The U.S. Cannot Keep Track of Police Shootings

A year ago, in a bureaucratic shift that went unremarked in the somnolent days before Michael Brown was shot dead in Ferguson, Missouri, the US government admitted a disturbing failure. The top crime-data experts in Washington had determined that they could not properly count how many Americans die each year at the hands of police. So they stopped.

The move did not make headlines. Before Brown was killed, a major government effort to count people killed by police could be mothballed without anybody noticing. The program was never fully funded, and no one involved was accustomed to their technical daily work drawing a spotlight.

But it had been a major effort. For the better part of a decade, a specialized team of statisticians within the US Bureau of Justice Statistics (BJS) – number-crunchers working several nesting dolls deep inside the Justice Department – had been collecting data on what they called arrest-related deaths. The ARD tally was more than a count of killings by police. It was meant to be the elusive key to a problem that seemed easy to understand but difficult to define. The program set out to track any death, of anyone, that happened in the presence of a local or state law enforcement officer.

A victim like Michael Brown, shot dead in the process of arrest, would make the count. A victim like Akai Gurley, shot dead in Brooklyn not in the process of arrest, would make the count. A victim like Eric Garner, choked and squeezed to death on Staten Island in the process of arrest, would make the count. A victim like Tamir Rice, shot dead in Cleveland at 12 years old with no arrest attempt made at all, would make the count, along with many other victims.

These people would make the US government’s authoritative count of people killed by police. If the count still existed. Which it does not.

With some states never participating, and major police departments such as the NYPD failing to report for some years, the Bureau of Justice (BJS) statisticians were never satisfied with their data pool. In March of last year, the bureau pulled the plug on the project, leaving the truth about the most high-profile year for police killings in US history – the truth about fatal police violence – to discarded spreadsheets, bad numbers and acronymed taskforces with little to show.

As revelations about patterns of abuse in Ferguson and beyond rattle the US criminal justice system from bottom to top, calls for a national police-killings database have once again gained urgency. But an awareness of what has been tried – and failed – remains elusive. And while even Barack Obama, after meeting with his post-Ferguson taskforce on policing, spoke of the “need to collect more data”, relatively little public discussion has centered on how such a count – a real count – would actually work.

A detailed look at what went wrong with the arrest-related deaths count reveals challenges that run deeper than the unwillingness of local police departments to file a report when they kill someone, although that is part of the problem. The structural and technical challenges to compiling uniform data from the 18,000-plus local law enforcement agencies in the US far exceeds the reporting problem, in some cases.

However, interviews across the lower rungs of government – with dozens of statisticians, police officers and politicians over the course of the seven months since Brown’s death – reveals a concentrated, multi-layered effort across multiple bureaucracies to arrive at a meaningful national number for arrest-related deaths that might be the start of a true national accounting.

Michael Planty, a senior official at BJS, said statisticians in the Justice Department would rather put out no number at all than put out a misleading number. “We’re concerned about quality and putting out a number,” said Planty. “We don’t want to put out just a number.”

The White House, however, does want them to put out a number. Last December, Obama signed a renewal of the Deaths in Custody Reporting Act (Dicra), which was created to monitor deaths in prisons and jails but which mandates that the Justice Department count arrest-related deaths. The new law might resurrect the BJS counting program in enlarged form, with new funding and new support.

Representative Bobby Scott, a Democrat from Virginia, the author of the bill in the House, spoke to the urgency of keeping up the program. Simply collecting data on fatal encounters with law enforcement could promote restraint on the part of police, he said: “The fact that the criminal justice system recognizes that we take deaths seriously may encourage people to be careful.”

The federal government counts many things very well. It counts the number of children who die each week from flu (11 during the week ending 17 January). It counts the average number of hours American men spend weekly on lawn care (almost two). It counts the monthly production of hens’ eggs (8.31bn in November). It counts nut consumption by non-Hispanic white men over the age of 20 (42.4% enjoyed nuts on any given day in 2009-2010). It counts how many women aged 15-44 use contraception (60.9 million, or 61.7%).

The US government is a virtuoso counter. So why can’t it count people killed by police?

Some people think Washington already does keep count. The government publishes two statistics every year that look an awful lot like, and are regularly mistaken for, comprehensive counts of deaths from interactions with police. The first is the FBI’s count of “justifiable homicides by law enforcement”, based on police reports submitted by state and local agencies.

The second is the Centers for Disease Control’s count of “deaths by legal intervention”, based on medical records and death certificates. Considerable scholarly exertion has gone into describing the flaws in each count. Not all police agencies participate in the FBI’s count, which is limited to deaths involving a weapon, for example, while the CDC misses cases, among others, in which medical examiners certify a homicide but fail to note police involvement.

Title 28 of the US Code, however, requires the FBI to publish the sum of whatever incomplete reports it receives. New objections to the bureau’s bad numbers were recently voiced by the FBI director himself, James Comey, in a speech last month at Georgetown University. Comey called crime statistics unreliable and said: “It’s ridiculous that I can’t tell you how many people are shot by police in this country right now.”

After Comey’s speech, Stephen L Morris, the FBI assistant director in charge of the uniform crime reporting program, said the bureau has been frustrated for decades by flaws in its national crime data collection system and a competing legal requirement to publish “the numbers provided to us”.

“We were ecstatic to have a director that has taken this issue on personally, because he was echoing, for the first time publicly, our frustration with this data,” Morris told the Guardian, speaking about “overall crime statistics” as well as “law-enforcement involved shootings”.

The FBI, which is a law enforcement and not a statistics agency, is working with a reporting program that is almost a century old and widely acknowledged to be inadequate. The program undertaken early in the George W Bush presidency by the BJS, in contrast, was designed to take advantage of current technology and techniques.

If it were possible to count people killed by police, it seemed, the BJS could do it. More than any other agency, the bureau is responsible for documenting how crime and punishment works in the US. Its best-known report, the annual National Crime Victimization Survey, is the country’s primary source of information on everything from rape and sexual assault among college-age women to firearm violence to crimes against the elderly.

But the arrest-related deaths program presented special problems. Just as there is no law compelling local law enforcement agencies to participate in the FBI’s uniform crime reporting, the BJS could not force participation in the count of arrest-related deaths.

Three states – Georgia, Montana and Maryland – declined all participation, which from the state’s perspective could mean extra work and compliance headaches. Other states submitted numbers for some years but not others. Washington DC dropped out of the program as resources and willpower dwindled. In other cases local numbers were reported to state agencies but then not up the next rung, to the federal government.

Erica L Smith, chief of the bureau’s law enforcement statistics unit, recalled the decision to end the count. “The data that we were collecting didn’t meet our data-quality standards, because we had little way of verifying that we were capturing all of the arrest-related deaths with that methodology,” she said.

The program had an unexpected benefit, however. It allowed the statisticians to estimate just how bad the FBI’s numbers were. The statistics bureau used a comparison between the FBI count of justifiable homicides by law enforcement and its own count, the arrest-related deaths count, to estimate how far off each count was. They published the results of their study early this month.

The results were stunning. The FBI was counting fewer than half of homicides by police officers, BJS discovered. From 2003 to 2009, plus 2011, the FBI counted an average of 383 “justifiable homicides by law enforcement” each year. The actual number, as estimated by the BJS study, was closer to 928.

Asked whether the FBI had ever considered suspending publication of justifiable homicide figures, given that those figures capture fewer than half of the deaths they purport to, the FBI declined to add to Morris’s earlier comments about frustrations at the “holes and gaps” in the data.

University of Missouri criminologist David Klinger, who has studied the BJS, FBI and CDC counts over time, said efforts to analyze their respective flaws were not likely to be repaid with insight. “Pardon my language, but if there’s a steaming pile of feces over in the corner, I don’t want to dig through it to find out whether it’s carrots or peas,” Klinger said. “I just know it’s steaming.”

‘It’s our responsibility’: searching for an almost perfect number

Hilary O Shelton, director of the Washington bureau of the NAACP, sees the country as having arrived at a crossroads comparable to what followed the beating of Rodney King by Los Angeles police in 1991. After the brutal assault, which was taped and broadcast on national news and showed King on the ground as multiple officers beat him with batons and kicked him, the NAACP conducted a series of hearings across the country on community-police relations.

“People perceived that African Americans were treated differently, more gruffly, more disrespectfully,” Shelton said. “That they felt that they were often targeted simply because they were African American. That was the perception.”

The perception, however, was not grounded in hard numbers. “That’s when we realized that we needed data,” Shelton said. The NAACP supported legislation, the End Racial Profiling Act, that would have awarded grants for data collection on racial profiling, but the bill was never signed into law.

Almost 25 years after Rodney King, there is a new call for data – and many suggested models for what might work. There may be a way to use federal money as an incentive for local law enforcement agencies to improve reporting. There may be a way to conduct a new arrest-related deaths count with blind spots eliminated. Technology may also provide an answer, through better video surveillance of police activities, or through internet searches of media reports that could document the overall problem.

“The absence of that database right now is a really tough thing, for everybody,” said John Firman, research director of the International Association of Chiefs of Police, the 120-year old police membership organization. “Because you’d love to be able to say, let’s look at trends, let’s look at incidents, let’s look at officer-involved shootings.”

Would the ideal count be top-down, with the federal government improving surveys of law enforcement agencies and other local data sources? Or would it be bottom-up, with law enforcement agencies improving their reporting? Would media reports or crowd sourcing play a role?

One flaw of the bottom-up counts is that they are based on a single source of information: the FBI uses data from local law enforcement, while the CDC relies on medical examiners and coroners. The statisticians running the top-down counts were encouraged to tap multiple information sources. That meant police and medical officials, but also media reports and close-at-hand agencies likely to be in the know, such as the offices of local and state prosecutors.

“That’s one of the challenges we had with the current [arrest-related deaths] program, was that states did different things to identify and collect this information,” said Planty, the senior BJS official. “And we’re trying to understand, did it really make a difference when we used, say, one source versus four or five sources? And I think it’s not a stretch to come to the conclusion that when you start relying on multiple sources of information, you’re going to get more and better information.”

Klinger, the University of Missouri criminologist, is working with the Los Angeles police department to develop a pilot program that would amount to a more muscular bottom-up approach.

“It doesn’t require, tomorrow, an act of Congress,” said Klinger. “Because what’s going to happen then is, it becomes a big political football, it becomes now, is it gonna be FBI, BJS, CDC – they’re going to be fighting over who gets the money. We don’t want any of that. What we want is, we’re going to have a program that is in place that has worked out as many of the bugs as we can think of. The federal government would merely have to pass legislation to fund a continuing effort.”

A key recommendation of the president’s post-Ferguson policing taskforce was that local law enforcement agencies be required to report to the federal government when their officers kill someone.

“There’s no reason for us not to have this data available,” the Philadelphia police commissioner, Charles H Ramsey, a taskforce co-chair, told the Guardian. “I know when this issue first came up, I was a bit surprised that we didn’t have a federal database that actually had all this data available. Now that we know that that does not exist, now it’s our responsibility to do everything we possibly can to develop that information and make it available to the public. And that’s what we’re in the process of working on now.”

For some people, the government’s failure to track officer-involved homicides is especially painful because it seems part of the institutional racism visited on African Americans by the US criminal justice system.

Of the many examples of racial disparity in criminal justice that BJS tracks – black drivers pulled over, black men imprisoned, black people sentenced on drug charges, black people murdered, black people sentenced to death – the arrest-related deaths data points to extra risk for African Americans. Black people die in disproportionate numbers at the hands of police, they are more prone to “accidents” around police, and their deaths are more likely to manifest as holes in police records.

“There’s a big problem here,” said Shelton, of the NAACP. “We saw what happened to Eric Garner. We saw what happened to Michael Brown. We saw what happened to Tamir Rice, and so many others who have died while in the custody of law enforcement. We have a problem.

“But in order to solve the problem, you have to have good data.”

Before long, the question of how many people are killed by police leads to the door of the only people who really know: the police.