Reply to Knox and Mummolo and Schimmack and Carlsson: Controlling for crime and population rates

Recently, we (1) published a report showing that, among civilians fatally shot, officer race does not predict civilian race and there is no evidence of anti-Black or anti-Hispanic disparities. As Knox and Mummolo (2) correctly state, this is different from the likelihood of being shot given a person’s race, Pr(shot$\mid$race). Although we are clear about the quantity we estimated and provide justification for calculating Pr(race$\mid$shot) in our introduction (see ref. 3), Knox and Mummolo (2) are right that our language in the significance statement of our report should be more careful.

However, Knox and Mummolo (2) are incorrect in their criticism that we did not “consider how many minority or White civilians are encountered [by police]” or that “The original paper infers no evidence of anti-Black or anti-Hispanic disparity simply because more fatally shot civilians are White.” Similarly, Schimmack and Carlsson (4) are incorrect in their criticism that our results “do not control for the fact that we would expect a higher number of White victims simply because the majority of US citizens are White.”

We agree that the estimate of Pr(race$\mid$shot) will be uninformative if it does not account for racial differences in police exposure. We discuss this in our introduction when we note that racial disparities in the likelihood of being fatally shot—Pr(shot$\mid$race)—depend the proxy for exposure used. Schimmack and Carlsson (4) make the strong claim that population proportions are the correct proxy for exposure: that all civilians are equally likely to encounter police in situations where fatal force is used.

However, researchers have repeatedly discussed the problems of using population proportions as a proxy for police exposure since the 1970s (5–9). As argued by Goff et al. (ref. 8, pp. 16–17) in the following:

Population benchmarks provide only a crude method for estimating disproportionality. They allow for an inference that force is being used in a manner that is disproportionate to presence in the general population, but do not allow for a clear inference as to whether the force is disproportionate to presence in any particular area or to legitimately provocative behavior. A more direct, albeit still limited, proxy for level of provocative behavior would be actual offending.

For this reason the analyses in our report account for racial differences in exposure by controlling for crime rates, a proxy for offending. However, as there is debate over which proxy is most appropriate, we also conduct analyses (in ref. 1, SI Appendix) where we account for racial differences in exposure by controlling for crime rates, population rates, or both simultaneously. That is, we test whether a person fatally shot is more likely to be Black or Hispanic than White if crime rates in a county were equal, if population rates in a county were equal, or if both were equal. Thus the claim that we somehow failed to account for encounters or population differences is unequivocally incorrect.

When using weighted-effect coding as recommended by Schimmack (10), Table 1 shows anti-White disparity when controlling for crime rates, anti-Black disparity when controlling for population rates, and no disparity when controlling for both crime and population rates. No analysis shows anti-Hispanic disparity.

Table 1.

Racial disparity in civilian race

	Black		Hispanic
Model controls for racial differences in	OR	95% CI	OR	95% CI
Crime	0.31	0.23, 0.42	0.36	0.28, 0.47
Population	2.03	1.21 3.41	0.30	0.14, 0.64
Crime and population	0.89	0.44, 1.80	0.32	0.12, 0.81

These intercept values test for racial disparity in a county where crime rates, population rates, or both do not vary by race. All other variables are centered or weighted-effect coded. OR, odds ratio; CI, confidence interval; N = 917.

Schimmack and Carlsson (4) control for population proportions and show anti-Black disparity in fatal shootings for their subgroup analyses. Besides relying on the questionable assumption that controlling for population proportions is an appropriate proxy for police exposure, such an analysis adds little to our understanding of fatal shootings because it is already widely known that Black citizens are shot more than White citizens on a per capita basis (see also ref. 11). Schimmack and Carlsson’s (4) analysis also ignores a major point of our report (the importance of crime rates) and that our supplemental analyses control for both population and crime rates.

Indeed, when we run the same subgroup analysis as Schimmack and Carlsson (4) but instead control for crime rates, we find that a person fatally shot is 1.21 [0.57, 2.58] times more likely to be Black than White and that a person fatally shot is 1.42 times (OR = 0.71 [0.57, 2.58]) times less likely to be Hispanic than White. Neither of these subgroup analyses shows significant anti-Black or anti-Hispanic disparity.

In addition, we investigated Pr(shot$\mid$race) in prior work and found anti-Black disparity when using population rates but no disparity or anti-White disparity when using crime rates as a proxy (11), the exact pattern obtained by others such as Goff et al.’s (8) Center for Policing Equity. We also showed in a prior response (3) that translating Pr(race$\mid$shot) to Pr(shot$\mid$race) via Bayes’ theorem when using reasonable estimates of police exposure again reveals no anti-Black or anti-Hispanic disparity.

One final point of clarification is an incorrect description of our work by Schimmack and Carlsson (4), who claim that “the intercept of [Johnson et al.’s] model corresponds to a country where White homicides rates equal 1) Black homicide rates and 2) Hispanic homicide rates and where victims are 3) average age (36.71 y) and White and Black victims are equally likely to 4) have mental health problems, 5) be suicidal, 6) be armed, and 7) attack an officer.”

Although 1) through 3) are accurate, 4) through 7) are wrong. Our model does not assume a county where White and Black civilians are equally likely to have mental health problems, be suicidal, be armed, or attacking. Because these categorical variables are weighted-effect coded, our model tests refer to disparity in the average shooting (i.e., the mean of these variables is zero).

In sum, we did control for racial differences in police encounters. Our results are consistent with past research estimating Pr(shot$\mid$race) when using population or crime rates, with the added benefit of being able to control for both simultaneously. This is a valuable contribution given debate over what proxy should be used to measure police exposure (11, 12). Rather than choose one, our approach allows researchers to control for multiple relevant proxies for police exposure.