Where's the data? @missing-data-blog - Tumblr Blog

Like a bridge over troubled data...

The NICAR Database Library, part of Investigative Reporters & Editors, will release the National Inventory of Dams database. This is great for reporters. The database “has been inaccessible for over a decade” as the U.S. Army Corps of Engineers has refused to disclose the full database since 2002. The Corps did post part of the data online in a publicly available searchable (but not downloadable) table. Of course, the data release isn’t without complications.

Although this newly released data is very valuable, the USACE is still withholding some crucial information, such as the city nearest to the dam and the hazard ranking of the dam. The hazard ranking indicates how bad it would be if a dam fails; a dam is ranked "high hazard" if people will likely die in that event. While the USACE will not release that information on an individual basis, they do provide the total number of high hazard dams by state.

NICAR will release data on U.S. dams after a decade-long hiatus IRE.org (Sept. 23, 2015)

#infrastructure #ddj #data journalism #environmental reporting #federal data

When a soldier makes it home...

Information veterans who commit suicide after they’ve left combat is woefully incomplete, as The New York times reports.

No one knows whether the battalion’s suicide rate is abnormally high or a common trait of fighting units hit hard by combat, because no one monitors troops over time. In an era of Big Data, when algorithms can predict human patterns in startling detail, suicide data for veterans is incomplete and years old by the time it is available. The most recent data is from 2011.

In Unit Stalked by Suicide, Veterans Try to Save One Another The New York Times (Sept. 19, 2015)

#ddj #data journalism #veterans #military data

An unchecked bully

Iowa passed an anti-bullying law in 2007 that required districts to keep track of bullying incidents and any disciplinary action that occurred as a result. Nearly 8 years later, the Associated Press finds that information is incomplete. The state agency blames the lack of data on “limited staffing and funding.”

The Iowa Department of Education acknowledges that data available on school bullying is inconsistent, arguing that for years districts have had different interpretations of what they should collect. The agency also said a new data collection system aimed at improving reporting from local districts resulted in missing data.

Documents show Iowa has limited data on school bullying Associated Press (Sept. 14, 2015)

#ddj #education data #data journalism

An incomplete census

Designed to assess poverty in India to better allocate government programs, the 2011 Socio Economic and Caste Census released may be too incomplete to be of value, one newspaper reports.

“The final data is still missing for over half the districts, and the draft list has gaps and inaccuracies,” says economist Jean Dreze. Of 640 districts covered, data for 628 is in a draft list, and only 277 in the final list. Of the 35 states and Union territories, 21 are yet to publish their final list.

Socio Economic and Caste Census data unreliable or incomplete: Experts The Economic Times (July 20, 2015)

#data journalism #india #census #poverty

I want a new drug (adverse event reporting system)

The Adverse Event Reporting System used by the U.S. Food and Drug Administration is known to be incomplete. Still, it’s often the best source of information to assess a drug’s safety it is on the market. So, how incomplete is the data?

The last time the GAO did a comprehensive review of the data, in 2000, it estimated fewer than 10% of all incidents are reported to the FDA.

The Slippery Slope: Adverse Events and Runaway Diabetes Train MedPage Today (Dec. 22 2014)

#health data #health reporting #data journalism #fda #drugs data #medical reporting

Can’t bill, but still getting paid

More than one in five doctors who were banned from billing Medicare are still able to bill state Medicaid programs -- costing taxpayers millions. Despite the extensive data work, the number is likely low, as Reuters explains in this detailed investigation.

In all, Reuters found 1,800 banned providers that were still able to bill elsewhere on a given date in 2014. The figures almost certainly underestimate the phenomenon by thousands of providers because of inadequate state and federal data.

Banned from Medicare, still billing Medicaid Reuters (April 29, 2015)

#data journalism #health data #medicare #medical reporting #health reporting #medicaid

Clean up in aisle three

Americans spent more money eating out than buying groceries for the first time ever, according to retail sales data from the U.S. Commerce Department. There is something missing in the information reported last month, however, as Jo Craven McGinty reports in her column, The Numbers.

But as it turns out, reports on the decline of home cooking were half baked. They demonstrate, once again, that it is important to understand how the government compiles statistics to avoid jumping to conclusions the figures don't support.

So what was wrong with using the Commerce figures to project a major dining trend? Lots of things.

Perhaps the biggest is that the figures primarily represent sales at supermarkets and don’t include big-box stores such as Wal-Mart, the largest grocery retailer in the nation, where many Americans stock up on provisions for their cupboards and refrigerators.

Don’t Believe the Hype: Eating In Still Tops Eating Out The Wall Street Journal (May 15, 2015)

#federal data #data journalism #business data

Jukebox hero, he'll come alive tonight

A fun one for this Saturday. Not government data, nor exactly hard news, but The Wall Street Journal breaks down the most popular jukebox songs in New York City using data from TouchTunes. The e-jukebox company has “more than 500 publicly-accessible city venues,” which the Journal used to categorize trends by zip code.

Comparisons aren’t exact because some zip codes have fewer or even only one venue offering the high-tech jukeboxes. Venues can also limit music selections by genre and some locations attract more customers – including those from farther away – than others.

Still, the results offer yet another glimpse into the eclectic tastes that are part of the city’s fabric.

NYC Jukebox Heroes: Musical Map The Wall Street Journal (May 14, 2015)

#data journalism #missing data #entertainment reporting #just for fun

Dr...Who?

When the Centers for Medicare and Medicaid Services released the first batch of payment information about doctors who received payments from drug companies the data was woefully incomplete in actually identifying those physicians.

Forty percent of payments (more than 1.7 million records) did not include the names of the doctors or hospitals that received the payments; the rest did.

Calculated by the value of the payments, the redactions were even more extensive. About 64 percent of the total spending by companies wasn't attributable to a particular doctor or hospital (the names, addresses and other identifying information were removed). This was particularly true for research payments—90 percent of the value of those payments went to unnamed doctors or hospitals.

Our First Dive Into the New Open Payments System ProPublica (Sept. 30, 2014)

#health data #health reporting #data journalism #missing data #medicare #medical reporting

Can you find me now?

When people call 911 in an emergency, responders often cannot locate the caller’s location and in some states it is getting worse.

Your chance of 911 getting a quick fix on location ranges from as low as 10% to as high as 95%, according to hundreds of pages of local, state and federal documents obtained and reviewed by USA TODAY and more than 40 Gannett newspapers and television stations across the country.

911's deadly flaw: Lack of location data USA Today (Feb. 21, 2015)

#missing data #criminal justice #data journalism #emergency responders #geolocation

A database of reported rape cases is basically empty

Despite the FBI creating a database to track detailed case descriptions in rape cases, it is virtually unused by law enforcement officers.

The FBI also created a database to contain detailed case descriptions to help police capture serial rapists who operate across state lines. But it is seldom used. Of 79,770 rapes reported to police in 2013, only 240 cases were entered into the database — 0.3 percent.

Upon Further Review: Inside the Police Failure to Stop Darren Sharper’s Rape Spree ProPublica & The New Orleans Advocate (April 8, 2015)

#criminal justice #data journalism #missing data #federal data

Dead and not counted

Data on officer-involved shootings isn’t thoroughly tracked at a national level by the three federal agencies who keep that information.

A Wall Street Journal analysis of the latest data from 105 of the country’s largest police agencies found more than 550 police killings during those years were missing from the national tally or, in a few dozen cases, not attributed to the agency involved. The result: It is nearly impossible to determine how many people are killed by the police each year.

...

Three sources of information about deaths caused by police—the FBI numbers, figures from the Centers for Disease Control and data at the Bureau of Justice Statistics—differ from one another widely in any given year or state, according to a 2012 report by David Klinger, a criminologist with the University of Missouri-St. Louis and a onetime police officer.

Hundreds of Police Killings Are Uncounted in Federal Stats The Wall Street Journal (Dec. 3, 2014)

#criminal justice #data journalism #federal data #missing data

Wildcatting in Western PA

Natural gas wells throughout Pennsylvania in the Marcellus Shale region far outnumber the total number of permits the state’s Department of Environmental Protection had on record.

When PG Web content producer Laura Schneiderman downloaded DEP's production data, she discovered it says there are 495 more wells producing gas, or ready to produce gas, than DEP has recorded as ever being drilled, and 182 of those wells don't even show up on the state's Marcellus Shale permit list.

DEP's Marcellus Shale drilling numbers do not add up Pittsburgh Post-Gazette (Jan. 8, 2012)

#missing data #environment #state data #data journalism #environmental reporting

There are going to be some agencies that won't want to fill your requests. Just hound them for it. Make sure you call them out in the end.

Matthew Doig, Sarasota Herald-Tribune investigations editor, in Behind the Story: Tracking problem police officers in Florida on IRE's On The Road blog. Doig took time last month to talk with me about the paper's series looking at Florida cops' complaint records.

Trending Blogs

Recently Viewed Blogs

Where's the data?