Homicides by Police: Comparing Counts From the National Violent Death Reporting System, Vital Statistics, and Supplementary Homicide Reports

Abstract

Objective. To evaluate the National Violent Death Reporting System (NVDRS) as a surveillance system for homicides by law enforcement officers.

Methods. We assessed sensitivity and positive predictive value of the NVDRS “type of death” variable against our study count of homicides by police, which we derived from NVDRS coded and narrative data for states participating in NVDRS 2005 to 2012. We compared state counts of police homicides from NVDRS, Vital Statistics, and Federal Bureau of Investigation Supplementary Homicide Reports.

Results. We identified 1552 police homicides in the 16 states. Positive predictive value and sensitivity of the NVDRS “type of death” variable for police homicides were high (98% and 90%, respectively). Counts from Vital Statistics and Supplementary Homicide Reports were 58% and 48%, respectively, of our study total; gaps varied widely by state. The annual rate of police homicide (0.24/100 000) varied 5-fold by state and 8-fold by race/ethnicity.

Conclusions. NVDRS provides more complete data on police homicides than do existing systems.

Policy Implications. Expanding NVDRS to all 50 states and making 2 improvements we identify will be an efficient way to provide the nation with more accurate, detailed data on homicides by law enforcement.

Recent media attention to the killings of civilians by police^1,2 has given rise to a national conversation on the use of lethal force by law enforcement: when it is justified and how best to reduce its use while ensuring the greatest possible safety to police. In remarks delivered at the Justice Department in January 2015, then–attorney general Eric Holder called for a national data system on police homicide, stating, “The first step . . . is to obtain better, more accurate data on the scope of the challenges we face.”³

Every year in the United States, approximately 100 000 people are treated in hospital emergency departments for nonfatal injuries inflicted by law enforcement officers.⁴ Two longstanding national data systems capture the fatal injuries—Supplementary Homicide Reports (SHRs) from the Federal Bureau of Investigation (FBI) Uniform Crime Reports system and mortality data from the Centers for Disease Control and Prevention (CDC) National Vital Statistics System⁵—but an evaluation found that both seriously underreport these deaths,⁶ as does a newer effort by the Bureau of Justice Statistics.⁷ The CDC’s National Violent Death Reporting System (NVDRS) offers promise in delivering a more accurate count, both because it draws from multiple data sources rather than a single source⁸ and because it reports larger numbers of police homicides than does Vital Statistics according to the CDC’s data query system that provides access to both.⁹ The NVDRS captures detailed coded data and rich narratives that describe the precipitating circumstances and incident dynamics for all suicides and homicides occurring in participating states. Unlike Vital Statistics and SHRs, the NVDRS is not yet implemented in all 50 states and has not been widely used by reporters or policymakers.

We evaluated the accuracy and completeness of the NVDRS in identifying homicides by law enforcement. We used the coded and narrative data to identify cases. We compared our count with that derived from the NVDRS “type of death” variable and—aggregated at the state level—with counts derived from other official sources. We also reported the population-based incidence rate of police homicide in NVDRS states and estimated the total number of homicides by police annually in the United States.

METHODS

How each of the data sources captured and defined law enforcement officer homicides is shown in Table A (available as a supplement to the online version of this article at http://www.ajph.org).

Data Sources

National Violent Death Reporting System.

The CDC established the NVDRS in 2003; it captures all deaths in participating states that result from homicide, suicide, legal intervention (excluding legal execution), unintentional shooting, and injuries of unknown intent.¹⁰ In 2003, 6 states participated; as of 2015, 32 participate. NVDRS has been fully described elsewhere.¹¹ Briefly, abstractors at the state level code detailed information at a minimum from the death certificate, coroner or medical examiner’s report, and police report to summarize a violent death incident’s victims, weapons, suspects, victim–suspect relationships, location, and precipitating circumstances. The abstractor assigns a “type of death” code to the case; an option is “legal intervention,” which the NVDRS coding manual specifies abstractors should use when the “decedent was killed by a police officer or other peace officer (persons with specified legal authority to use deadly force), including military police, acting in the line of duty.”¹² The abstractor also writes 2 brief narratives on each incident to summarize the coroner or medical examiner report and the police report.

Vital Statistics.

The CDC’s National Center for Health Statistics assembles Vital Statistics mortality data, which are derived from the death certificate. The coroner or medical examiner in the jurisdiction in which the incident occurred investigates homicides and other suspicious deaths at the local level. The coroner or medical examiner fills out the portion of the death certificate that captures the manner and causes of death.¹³ The funeral director fills out the demographic portion. The death certificate is filed with local authorities and the state vital statistics registry.

State registry officials forward the data to the National Center for Health Statistics, where software programs code the multiple causes, and underlying cause, of death using the World Health Organization’s International Classification of Disease (ICD-10).¹⁴ The ICD-10 defines a category for “legal intervention” deaths (“injuries inflicted by the police or other law-enforcing agents, including military on duty, in the course of arresting or attempting to arrest lawbreakers, suppressing disturbances, maintaining order, and other legal action”¹⁵); however, it can only be assigned if the police involvement is explicitly mentioned on the death certificate. Although the report that the coroner or medical examiner writes will typically explicitly mention if a decedent died at the hands of the police, this information is not always mentioned on the death certificate itself, in which case the death is coded as a homicide for underlying cause.¹⁶ State counts are derived from the victim’s state of residence, not the state in which the incident occurred.

Supplementary Homicide Reports.

SHRs are supplemental reports that local police and sheriffs’ departments provide to the FBI as part of the voluntary Uniform Crime Reports system.⁵ Police homicides are identified with the circumstance code, “felon killed by police” and also receive a subcircumstance code that describes why the killing occurred (e.g., “felon attacked civilian”).¹⁷ Homicides occurring on federal property (e.g., federal prisons, tribal lands, and military bases) are not included in the system.

State counts are derived from the state in which the incident occurred, regardless of the victim’s state of residence. Reporting is voluntary; although the vast majority of departments participate in the Uniform Crime Reports, and participation is tracked by the FBI, many do not file supplemental reports such as the SHR.⁶

Sample

Data acquisition.

We obtained Vital Statistics counts for legal intervention deaths (excluding executions) by state for the years 2005 to 2012 from the CDC’s Wide-ranging Online Data for Epidemiologic Research Web site using ICD-10 codes Y35.0–Y35.4, Y35.6, Y35.7, and Y89.0. We downloaded SHR data from the Inter-University Consortium for Political and Social Research Web site for the same period and aggregated, by state, cases for which the precipitating circumstance was 81 (felon killed by police).

We received individual-level NVDRS data from the CDC using their Researcher Access Data request process. To report counts at the state level according to the NVDRS (as opposed to our count derived from expanded case finding), we selected NVDRS cases that were coded by the abstractor as “legal intervention” for type of death and excluded cases in which the incident occurred outside the reporting state’s borders.

Case definition.

We defined homicides by law enforcement as deaths that meet each of the following criteria:

1. The manner of death (on case review) was homicide, not suicide, accident, or natural. Accidents, such as a police officer unintentionally shooting another while cleaning his gun, did not qualify. Deaths that were unintentional but resulted from the intentional use of violent force against a person (e.g., an officer shot at a criminal but hit a bystander) qualified. Deaths resulting from police chase were a gray area. We applied the definition the Department of Justice used.⁷ That is, deaths resulting from chases qualified only if police action had a direct role in the death (e.g., police maneuver forced subject’s car off the road). “Suicide by cop” (so called when a suicidal person provokes an officer to kill him or her) met the homicide criterion. Natural deaths that occurred as the result of use of force (e.g., heart attack during attempted arrest) qualified only if the coroner or medical examiner determined the manner of death to be homicide (not natural death). We excluded legal executions. The term “homicide” is not the same as “murder” and does not of itself imply wrongdoing.

2. The suspect was a law enforcement officer. This included police officers, sheriffs, deputies, federal agents (e.g., FBI, Drug Enforcement Agency), and military police. We included corrections officers, although some states do not define them as sworn officers. (Our final data set included 4 legal intervention homicides involving corrections officers.) We excluded private security guards, as they were not sworn officers employed by a public agency with statutory authority to use lethal force.

3. The incident occurred in the line of duty. An off-duty officer who witnesses a crime and intercedes as an officer is acting in the line of duty. At the other end of the spectrum, an officer who, for example, kills his or her spouse during an argument is not acting in the line of duty regardless of whether the officer is on duty or off duty at the time. A handful of cases fell into a gray area in which an off-duty officer was the victim of a crime and responded by killing the alleged offender (e.g., an officer shot someone who broke into his house). If the off-duty officer was the victim of the crime, the incident did not meet our case definition.

Case finding.

We flagged potential police homicides on the basis of 5 NVDRS variables that all states consistently captured:

1. Type of death, an abstractor-coded variable, was “legal intervention.”

2. Relationship of victim to suspect was 50, “victim injured by law enforcement officer.”

3. Underlying cause of death, a variable provided by the state vital statistics registry, was in the legal intervention range (Y35.0–Y35.4, Y35.6, Y35.7, and Y89.0, excluding legal executions).

4. Victim in custody was coded as 1 “in jail or prison,” 2 “under arrest but not in jail,” 6 “injured before arrest,” or 8 “other” and manner of death (from the death certificate) was homicide or legal intervention. (The standard US death certificate form does not include “legal intervention” as an option for manner of death. Eight of the NVDRS states had at least 1 death coded to “legal intervention” in the death certificate manner of death field.)

5. Homicide circumstance was “justifiable homicide” and relationship was missing. If relationship was 50, it was captured in point 1. We reviewed a random 20% sample (n = 102) of cases coded as “justifiable homicide” for which the relationship was nonmissing and found only 1 legal intervention case. We therefore focused our search on justifiable homicides for which relationship was missing, nearly half of which were cases.

Four authors (C. B., D. A., D. T., and D. E. W.) read the incident narratives for potential legal intervention deaths in 2012 and identified case definition issues for the group to resolve. Two authors then coded 2006 and 2009 cases and identified those meeting our case definition. There was agreement across coders on case status for 98.3% of cases. We came to consensus on the remainder and refined the case definition. One of us (C. B.) then read the incident narratives for all cases 2005 to 2012 that carried any of the 5 markers for law enforcement homicides and identified those meeting our case definition. Gray area cases were brought to the group for vote.

We excluded cases that occurred outside an NVDRS state’s borders.

Statistical Analysis

To assess positive predictive value (PPV),¹⁸ we calculated the proportion of cases coded by the NVDRS abstractor as “legal intervention” that met our case definition as law enforcement homicides. When calculating PPV, we set aside 18 cases for which not enough information was provided in the incident narratives to assess case status. To assess sensitivity, we calculated the proportion of deaths that we identified as law enforcement homicides that the NVDRS abstractor coded as “legal intervention” for type of death.

Using the study-identified cases, we calculated the average annual rates of legal intervention homicides for each state, for Hispanics, and for non-Hispanics by race using US Census–derived annual population estimates supplied on the CDC Wide-ranging Online Data for Epidemiologic Research Web site. Hispanic ethnicity was unknown for 2% of victims. Among non-Hispanic victims, race was unknown for 1% and more than 1 race for 3%. We set these cases aside when calculating incidence rates by race/ethnicity.

We estimated the average annual number of police homicides in the United States by applying the combined rate observed in the NVDRS states to the US population. We included the 18 cases that were abstractor coded as legal interventions but had insufficient information to enable us to confirm case status; we included these because PPV was so high (98%) for abstractor-coded legal intervention deaths.

RESULTS

For the period 2005 to 2012, we identified 1552 law enforcement homicides that occurred in the 16 NVDRS states. Table 1 summarizes the counts by state according to each reporting source. The NVDRS abstractors identified a total of 1421 cases (92% of the study total), Vital Statistics reported 906 (58%), and SHRs reported 742 (48%). The ratio of cases Vital Statistics reported in each state to that we identified ranged from a low of 0.2 to 1.0 in North Carolina to a high of more than 1.0 to 1.0 in Oregon (Table 1).

TABLE 1—

Law Enforcement Homicides by Reporting Source and State: United States, 2005–2012

State	NVDRS Study,a No. (Rate)b	NVDRS Abstractor, No.c (Ratio)d	National Vital Statistics System, No. (Ratio)d	Supplementary Homicide Reports, No. (Ratio)d
Alaska	24 (0.43)	26 (1.08)	10 (0.42)	0 (0.00)
Colorado	119 (0.30)	100 (0.84)	77 (0.65)	60 (0.50)
Georgia	228 (0.30)	184 (0.81)	108 (0.47)	92 (0.40)
Kentucky	66 (0.19)	45 (0.68)	37 (0.56)	19 (0.29)
Maryland	177 (0.39)	178 (1.01)	128 (0.72)	164 (0.93)
Massachusetts	46 (0.09)	46 (1.00)	36 (0.78)	6 (0.13)
New Jersey	95 (0.14)	95 (1.00)	31 (0.33)	82 (0.86)
New Mexico	82 (0.51)	78 (0.95)	66 (0.80)	41 (0.50)
North Carolina	166 (0.22)	161 (0.97)	33 (0.20)	43 (0.26)
Oklahoma	103 (0.35)	103 (1.00)	37 (0.36)	78 (0.76)
Oregon	91 (0.30)	95 (1.04)	94 (1.03)	29 (0.32)
Rhode Island	13 (0.15)	13 (1.00)	e	2 (0.15)
South Carolina	77 (0.21)	35 (0.45)	45 (0.58)	20 (0.26)
Utah	67 (0.31)	66 (0.99)	50 (0.75)	21 (0.31)
Virginia	119 (0.19)	118 (0.99)	104 (0.87)	39 (0.33)
Wisconsin	79 (0.17)	78 (0.99)	50 (0.63)	46 (0.58)
Total	1552 (0.24)	1421 (0.92)	906 (0.58)	742 (0.48)

Note. NVDRS = National Violent Death Reporting System.

^aIncludes cases with any marker for legal intervention that, on review of the incident narratives, we confirmed as a case.

^bAverage annual rate per 100 000.

^cIncludes cases the NVDRS abstractor coded as “legal intervention” for type of death.

^dThe ratio of cases the data source reported to cases we identified.

^eCenters for Disease Control and Prevention policy requires that cell sizes of 1–4 not be reported.

The ratio of cases reported by SHRs to the study total also ranged widely by state, from lows of 0.00 in Alaska and 0.13 in Massachusetts to a high of 0.93 in Maryland. The ratio of NVDRS abstractor-reported cases to study cases was generally very high across the states but notably low in South Carolina (0.45).

Nearly all the cases reported by the NVDRS abstractors met our case definition. PPV was 98% (Figure 1). The 27 cases that did not meet our study definition failed largely because of definitional issues, not coding errors. These included 8 security guards, 6 off-duty law enforcement officers who were responding to a crime against themselves, and 9 people who died during police pursuit but not as the result of direct contact with police (e.g., struck by a vehicle while fleeing from police). Only 4 were coding errors.

FIGURE 1—

Sensitivity and Positive Predictive Value (PPV) of National Violent Death Reporting System (NVDRS) Abstractor Coding of Law Enforcement Homicide, 16 States (Using the NVDRS Study as the Gold Standard): United States, 2005–2012

^aThe figure excludes 18 decedents coded by the NVDRS abstractor as legal intervention deaths but for which there was insufficient information in the incident narrative for the authors to ascertain case status. These 18 are included in the other tables and figures because PPV for NVDRS abstractor was so high.

^bThe universe on which Sensitivity is calculated is NVDRS decedents with markers suggestive of legal intervention homicide, not on an objective gold standard. Because the universe is a subset of all NVDRS cases, Specificity and Negative Predictive Value are not calculated.

Sensitivity of the reporting system was also relatively high (90%), although NVDRS abstractors missed at least 158 cases. These were largely coded as homicides and were concentrated in South Carolina, Kentucky, Georgia, and Colorado. Among the missed cases, 78% were picked up by the justifiable homicide circumstance code, 63% by the relationship code, 39% by the in custody variable, and 13% by the ICD-10 code; 1 was the second victim in a 2-victim legal intervention incident in which the abstractor coded only 1 victim as such. A third of the missed cases were picked up by 1 of the alternative markers only, 44% by 2, 21% by 3, and only 3% by all 4.

The average annual rate of legal intervention homicides in the 16 NVDRS states over the study period was 0.24 per 100 000 population on the basis of the study count. Rates varied by race/ethnicity. Among Hispanics, the rate was 0.25/100 000. Among non-Hispanics, rates were 0.48 for Blacks, 0.25 for American Indians, 0.17 for Whites, and 0.06 for Asians. Among all groups combined, rates were stable across the 8-year period (Figure 2). State-specific rates, summarized in Table 1, varied more than 5-fold, with a low of 0.09/100 000 in Massachusetts to highs of 0.43 in Alaska and 0.51 in New Mexico. If the rate of legal intervention deaths for the nation is similar to that of the NVDRS states overall, an estimated 731 people were killed by law enforcement officials each year in the United States during the study period.

FIGURE 2—

Rate of Law Enforcement Homicides per 100 000 Population by Race/Ethnicity and Year, 16 National Violent Death Reporting System (NVDRS) States (on the Basis of NVDRS Study Count): United States, 2005–2012

DISCUSSION

The NVDRS is a more complete reporting system for identifying legal intervention homicides than is the Vital Statistics system or the FBI’s SHR. The NVDRS captured more than twice the legal intervention deaths reported by SHRs and 71% more than reported by Vital Statistics. False positives were rare in the NVDRS, and most were not errors per se but resulted from our applying a more restrictive case definition. The NVDRS abstractors did, however, miss at least 10% of the legal intervention deaths that we identified, and these were largely errors.

A technical solution could solve much of this underreporting. When another variable suggests a potential police homicide and the abstractor has not endorsed legal intervention, a pop-up reminder could ask if legal intervention is the appropriate code for abstractor-assigned type of death.

Researchers should be cautioned not to compare legal intervention homicide rates across states using Vital Statistics or SHR data because differences will be driven more by artifacts of reporting than by true differences in incidence rates. This is far less a problem in the NVDRS, with the exception of serious underreporting in South Carolina and, to a lesser extent, Kentucky.

We have not presented state-level counts obtained from the Bureau of Justice Statistics’ arrest-related deaths reporting system. Data were not available past 2009, and 5 of the 16 study states did not participate in that system for all or part of 2005 to 2009. Bureau of Justice Statistics personnel supplied us with state–year counts for law enforcement homicides. The Bureau of Justice Statistics’ count in NVDRS states from 2005 to 2009 (n = 418) was 57% of that identified in the NVDRS study data for the same state–years (n = 738). The Bureau of Justice Statistics suspended reporting in 2014 and is currently redesigning the system.¹⁹

Limitations

Our study was subject to limitations. We did not have a true gold standard against which to measure sensitivity and PPV of the NVDRS but instead identified cases within the NVDRS. Estimates of the national incidence of police homicides by the Washington Post²⁰ and the Web site Fatal Encounters²¹ suggest that even with our expanded case finding, the NVDRS is missing cases.

We reviewed a random 2% sample of NVDRS homicides that carried no markers suggesting police homicide and identified only 1 potential, but not definitive, case. This does not signal a major underreporting problem; however, there were many homicides for which the incident narratives were blank or uninformative. Some of these were probably unsolved homicides that will never have detailed information (e.g., a body is found and no information is available on the perpetrator or circumstances). But some were cases under active investigation, and so the abstractors typically would not have access to the police report until the case was closed. It is possible that among these some police homicides are lost.

Capture of these cases (if indeed they exist) may be improved if NVDRS sites use a flag to note cases that were not yet available for abstraction and consistently follow back until they are available. Future research could establish whether an aggressive follow back policy results in improved identification of police homicides and other homicide types.

The NVDRS is already funded in 32 states, and its feasibility has been well demonstrated.²² At an estimated cost of $25 million annually for a 50-state system, the NVDRS is an efficient mechanism to integrate data from coroners or medical examiners, police, and death certificates into a useable database to better track, understand, and prevent all violent deaths, including law enforcement homicides.

Conclusions

The NVDRS provides a more comprehensive count of law enforcement homicides than is provided by the National Vital Statistics System and the FBI’s SHR. Although there is some underreporting in the NVDRS, it appears likely that this could be remedied with technological fixes. Expanding the system from its current (as of 2016) 32 states to 50 and the District of Columbia will be an efficient way to supply the nation with both an accurate count of police homicides and detailed information from the coded and narrative data on the people, weapons, and circumstances involved.