High-quality Research on the Impact of Deaths Due to Firearms Is Critical for Fairly Evaluating Public Policy

I grew up in rural Ohio, where many people owned firearms with few legal restrictions. During my childhood, my father owned guns, and he personally ranked states by how restrictive their guns laws were. I have also lived in other states with very restrictive access to firearms. Given these experiences, I have often wondered: do more restrictive firearm laws make any impact, positively or negatively, on public health? During my career as a clinician, researcher and bioethicist, I have wanted to see more evidence on how policy impacts people, particularly with regard to firearm laws. Having more information would help policy makers and healthcare professionals alike have more meaningful discussions on the impact of different state policies with stakeholders involved in this controversial issue in the USA.

In order to have more and valid information, we need high-quality research. However, conducting research on the associations and impacts of state laws, on any topic, is challenging. Laws are almost never implemented in a prospective randomized control trial, allowing for careful evaluation of cause and effect. Implementation is usually based upon politics and funding, and other practical issues. Additionally, formal evaluation is rarely planned. Furthermore, there are so many confounding variables to consider, and data is often not readily available to accurately operationalize the key variables involved. Heated political controversy further increases the challenges related to conducting unbiased, high-quality research; this is particularly true when discussing firearm laws in the USA.

Laws are often central to public health and healthcare, and their impacts can be significant on people’s lives. We need more high-quality research that is ready to take-on controversial topics that impact people’s health. Siegel and colleagues accepted this challenge by conducting a national study evaluating the health impact of multiple state firearm laws on homicide and suicide.¹ Using data on state firearm laws in all 50 states from 1991 to 2016, they conducted an analysis using difference-in-differences; fixed effects and regression modeling to assess the impact of these laws on age-adjusted total homicide and suicide rates for each state.

Importantly, they were able to do this research using several key innovations. First, they used a unique database that was developed by combining data from the Thomson Reuters Westlaw database and the Everytown for Gun Safety database, along with secondary sources from the Bureau of Alcohol, Tobacco, Firearms and Explosives’ State Laws and Published Ordinances and the Law Center to Prevent Gun Violence.² This database includes 133 different provisions, covering 14 aspects of state policies, over a 26-year period. Importantly, these provisions are coded as to allow for variation and nuance in categorizing the types of provisions—this detailed coding had not be done before the creation of this database.

Second, the authors also carefully selected the laws they studied in terms of being currently considered by state legislators or previously studied before, and restricted the analysis to laws that were adopted by two or more states in the time-frame of the study. This thoughtful selection is key to conducting relevant research that influences policy, by ensuring that the laws studied are relevant to both the previous research and to policy development.

Third, the authors used a new proxy variable for rates of firearm ownership: they incorporated the ratio of states’ hunting license rates; the previously established proxy uses the ratio of suicides due to firearms to all suicides (FS/S). This new proxy results in a more accurate determination of firearm ownership, which has obvious implications for research in this area. Fourth, the authors used a difference-in-differences approach, across all 50 states, which provides a more robust analysis compared to previous investigations; allowing for the investigators to control for state effects that might impact the outcomes. For those readers who are not familiar with a difference-in-differences analysis, this approach attempts to approximate a randomized, experimental design while using observational data, with the hope of coming closer to determining causation. Researchers do this analysis by comparing the average change (difference) over time in the dependent variables within the control group and within the intervention group. Then, one determines if there is a meaningful difference between the differences in the control and intervention groups. For more information, please see Bret Zeldow and Laura Hatfield’s website for detailed descriptions and graphics.³

Interestingly, Siegel and colleagues found that some, but not all, laws were statically significantly associated with age-adjusted homicide and suicide rates. Laws requiring universal background checks prior to firearm purchase and laws restricting firearm ownership among people with violent misdemeanor convictions were associated with lower homicide rates. The relationship was in the opposite direction for shall-issue laws, which allow for concealed carry permits; in all cases where individuals meet specific pre-established criteria, which removes any discretion from the law enforcement authorities. Importantly, household firearm ownership was not associated with homicide rates.

One of the most interesting findings is that firearm ownership was not associated with overall homicide or suicide rates when adjusting for key variables. Although the authors are likely correct that the small changes in firearm ownership, and measurement error, make such analyses difficult. To me, this finding warrants further investigation, for many of the reasons I have stated above. If it is accurate that the number of people who own firearms is not associated with homicide and suicide rates, we may want to focus instead on the policies that do seem to have a significant impact by controlling who, not necessarily how many, people own firearms. Importantly, future investigations, as the authors note, should focus not only on the existence of certain laws but also on the methods of enforcement of those laws, given that enforcement can significantly impact the effectiveness of the laws to carry out their intended purposes.

In closing, I commend these authors for exploring this important public health issue, and for considering all types of laws, regardless of their place on the political spectrum. Such research not only provides information to help inform policy development, but it also allows stakeholders to come together in a more meaningful way to discuss ways to reduce unnecessary deaths. Research such as this study is vital to public health.