Simulating the Suicide Prevention Effects of Firearms Restrictions Based on Psychiatric Hospitalization and Treatment Records: Social Benefits and Unintended Adverse Consequences

Abstract

Objectives. To estimate the number of lives saved from firearms suicide with expansions of gun restrictions based on mental health compared with the number who would be unnecessarily restricted.

Methods. Agent-based models simulated effects on suicide mortality resulting from 5-year ownership disqualifications in New York City for individuals with any psychiatric hospitalization and, more broadly, anyone receiving psychiatric treatment.

Results. Restrictions based on New York State Office of Mental Health–identified psychiatric hospitalizations reduced suicide among those hospitalized by 85.1% (95% credible interval = 36.5%, 100.0%). Disqualifications for anyone receiving psychiatric treatment reduced firearm suicide rates among those affected and in the population; however, 244 820 people were prohibited from firearm ownership who would not have died from firearm suicide even without the policy.

Conclusions. In this simulation, denying firearm access to individuals in psychiatric treatment reduces firearm suicide among those groups but largely will not affect population rates. Broad and unfeasible disqualification criteria would needlessly restrict millions at low risk, with potential consequences for civil rights, increased stigma, and discouraged help seeking.

In 2016, 44 965 individuals died by suicide in the United States.¹ As many other leading causes of mortality have decreased in recent decades, the suicide rate has increased by more than 30% since 1999.² Firearm-related suicide, in particular, along with the greater burden of firearm violence,³ remains a leading cause of death among those aged 10 to 54 years.⁴ The population-wide increase in firearm suicide deaths in the United States warrants a public health response. However, empirical data to evaluate different approaches to suicide prevention in the population are limited.

Mental illness is a key reason that many people attempt suicide,^5,6 and access to a firearm is often the reason they die; guns are a highly lethal method of intentional self-injury.^7,8 Thus, restricting access to firearms for people at risk for suicide, including many with serious mental illnesses, can save many lives.^9–12 Gun prohibition linked specifically to involuntary psychiatric hospitalization is a longstanding federal policy. However, the fact that most gun suicide decedents have never been involuntarily hospitalized limits the effectiveness of point-of-sale prohibitions that are predicated on civil commitment alone. Gun restrictions that have been proposed or enacted have included people with records of any (voluntary or involuntary) psychiatric hospitalization,^13–15 and, indeed, a few states have broader criteria, such as having a record of any psychiatric hospitalization (Connecticut) or even a record of any mental health treatment (Hawaii).

Enforcement of gun prohibitions based on mental health treatment history alone, without the legal safeguards of an adjudicatory process, is problematic from both a practical and civil rights standpoint. An estimated 44.7 million US adults have a diagnosable mental illness, and about 43% of these individuals are receiving treatment.¹⁶ The large majority of people treated for a mental illness in the community will neither die of suicide nor perpetrate violent acts. Moreover, restrictions can have unintended consequences; mental illness remains a highly stigmatized condition.¹⁷ Restrictions on firearm access linked to mental illness services utilization per se could discourage help seeking, especially among gun owners who believe that receiving mental health treatment will result in loss of their gun rights. Furthermore, associations between gun violence and mental illness more generally are complex and imbued with a problematic social and political history linked with race/ethnicity and class disparities.¹⁸ Guiding policy on firearm ownership to reduce suicide must balance efficacy with fairness, the protection of civil rights, and the goal of reducing stigma attached to mental illness.

In this study, we simulated randomized trials to estimate the hypothetical population-level effects of firearm ownership disqualification policies based on mental health records. We began with a prohibition predicated on any psychiatric hospitalization—a criterion that is broader than the current federal restriction linked to involuntary commitment but that still affects only a small proportion of the population. We then broadened the range of individuals who would be disqualified and examined the effect of gun prohibition based on any mental health treatment. Our model thus allowed us to explore both the benefits and social costs of such a broad restriction—to estimate its effectiveness, but also the extent to which it would preemptively restrict the rights of millions of law-abiding, nondangerous people. We examined the potential effect of firearm ownership disqualifications on overall population rates of firearm-related suicide, and also the effect on the targeted population of restricted individuals considered as a discrete group.

METHODS

We developed an agent-based model simulating the dynamic processes contributing to firearm suicide among adults in New York City. Figure A (available as a supplement to the online version of this article at http://www.ajph.org) illustrates the relations included in the model.^19–22 We used data from New York City sources to calibrate the model when possible; we also used national or other community-based data (see data sources in Table A, available as a supplement to the online version of this article at http://www.ajph.org). Additional details about model parameters and processes include a description of the model following the Overview, Design Concepts, Details protocol (Appendix A),^23,24 initialization parameters and default values (Table B), and flow charts illustrating steps in the model (Figures B and C, available as supplements to the online version of this article at http://www.ajph.org), as well as final calibration formulae for key model parameters.

Agent Population and Neighborhoods

We initialized the population of 260 000 agents to approximate a 5% sample of the New York City adult population aged 18 to 64 years in the year 2000.^19,20 Agent attributes included age, gender, race/ethnicity, income, and education, as well as agent behaviors including substance use, a range of psychiatric disorders, gun carrying and ownership, and suicidal behavior. We assigned agents to neighborhoods, proportionate to size; distributions of age, gender, race/ethnicity, and household income matched Census data for each of the 59 New York City community districts in 2000.²⁵ We chose the year 2000 because most data used to parameterize agent behaviors were collected in the mid-2000s. Individual behaviors were influenced by neighborhood characteristics and vice versa. Also, the study period predates 2009, the year that New York State began reporting large numbers of gun-disqualifying mental health records to the Federal Bureau of Investigation’s National Instant Criminal Background Check System. As of 2017, the number of “adjudicated mental health” records from New York State that are active in the National Instant Criminal Background Check System Index exceeds 540 000.²⁶ The impact that the presence of such records in the background check system may have had is unknown, but it would not have affected the data inputs to our models, in any case.

Aging, Mortality, Movement

Each model step represented 1 year in time. At each time step, agents aged by 1 year, a proportion of agents moved to a new neighborhood, and agents died consistent with 2000 New York City adult all-cause mortality rates.²⁷ Agents’ probabilities of moving were based on income, current neighborhood duration of residence, and violent victimization at the last time step, calibrated with data from longitudinal studies in urban areas²⁸ and the Panel Study of Income Dynamics.²⁹

Gun Carrying and Ownership

Agents were calibrated to have firearms in their household and, separately, to carry firearms at each time step. We separated household guns and gun carrying to allow agents to access guns without legal purchase (e.g., in a nonlicensed or illegal market, or exchanged within their social network—for example, a firearm from family or a friend). We calibrated probabilities of household firearms and, separately, carrying from the National Comorbidity Survey Replication,³⁰ which included separate questions on household firearms and firearm carrying within the past 30 days. Probabilities were based on demographics, past household firearms and carrying at previous time steps, history of victimization and perpetration, and substance use. Furthermore, we specified that agents who had social network ties to agents who had household firearms or carried them were able to use those guns. Thus, even when an agent was disqualified from gun purchasing, the agent was still able to access a firearm. Furthermore, we varied the prevalence of gun access through sensitivity analyses, especially as New York City is estimated to have lower gun ownership rates compared with the United States.

Psychiatric Disorders and Treatment

Agents also had the possibility of having 1 or more psychiatric disorders at each time step. We calibrated probabilities of having major depressive disorder, generalized anxiety disorder, antisocial behavior, intermittent explosive disorder, posttraumatic stress disorder, mania, and psychosis from the National Comorbidity Survey Replication.³⁰ Probabilities at each time step were based on predictive equations as well as history of disorders at previous time steps.

At each time step, agents could receive mental health treatments, including overnight hospitalization, medication, and other treatments based on National Comorbidity Survey Replication calibration.

To measure the occurrence of psychiatric hospitalization, we used data from the Patient Characteristics Survey, a questionnaire that collects information on all individuals in the public mental health system in New York State, including those inpatients of state and locally operated psychiatric facilities.^31,32 These data capture those inpatient stays that are reported to the New York State Office of Mental Health (OMH). We calibrated the number of OMH-identified psychiatric hospitalizations to match the total number of New York City adults who were hospitalized for psychiatric reasons (as identified by OMH), based on race/ethnicity, age, and borough.

Suicide and Other Health Outcomes Related to Firearms

At each time step, agents died by suicide with or without a firearm according to New York City Office of the Chief Medical Examiner data.³³ We calculated probabilities based on race/ethnicity, gender, age, mental health and suicidal behavior, and drug and alcohol use.^34,35 Suicide was influenced by history of suicide ideation and suicide attempt; agents could also die by suicide without previous ideation or attempt according to previous literature about impulsivity of suicide.^10,36 Presence of a firearm increased the probability of a fatal suicide attempt.³⁷ We also calculated probabilities of suicide ideation and attempt from the National Comorbidity Survey Replication.³⁰ We based probabilities on sociodemographic characteristics, household firearm and carrying statuses, alcohol and drug use,³⁸ history of violent victimization and perpetration,³⁹ alcohol and drug abuse,³⁸ each of the 7 mental health disorders listed previously, and mental health treatment.^38,40

Social Network and Neighborhood Influences

We assigned each agent a target number of close social ties, with an average of 4 ties per agent.⁴¹ We matched agents on the basis of age, gender, race/ethnicity, education, firearm status (gun carrying and ownership), drinking status, and spatial proximity, such that agents who were more similar and geographically closer to each other were more likely to become social ties.^41,42 Social network members matched to a particular agent at baseline remained part of that agent’s social network for the duration of the model run. On the basis of empirical social network literature,⁴³ we modeled ties to other agents who were involved in gun violence, which informed suicide risk probabilities. The strength of the social network influence on model parameters was varied in sensitivity analyses (Appendix B, Table C, available as a supplement to the online version of this article at http://www.ajph.org).

Agents were also embedded in neighborhoods, and those neighborhoods had their own characteristics (e.g., demographics, average mental health and suicide, violence, and substance use). We used predictive equations to assess the strength of the relationship between neighborhood characteristics and suicide; the strength of the neighborhood relationships determined 10% of suicide risk; the remaining 90% was determined by individual-level risk factors. We then varied this distribution in sensitivity analyses (Appendix B).

Model Calibration and Intervention Scenarios

During model calibration, we compared agent-based model estimates with empirical data. We used an iterative process⁴⁴ to adjust predictive equations and initial conditions in the model (Table 1).

TABLE 1—

Annualized Estimates of Suicide, Mental Health Treatment, and Firearm Carrying and Ownership From the Agent-Based Model and Empirical Data Sources in New York City When Available and the National Comorbidity Study Replication

	ABM Estimate (95% CI)a	Empirical Estimates
Firearm-related suicide, rate/100[thin space]000	2.1 (1.9, 2.3)	2.0b
Any psychiatric hospitalization, rate/100 000	108.3 (105.2, 110.9)	108.0b
Any mental health treatment in past y, %	13.9 (13.9, 14.0)	15.8c
Firearm ownership, %	24.3 (24.1, 24.4)	24.0c
Firearm carrying status, %	4.0 (3.9, 4.0)	3.9c

Note. ABM = agent-based model; CI = credible interval.

^aMedian and 95% CI from 50 runs of ABM.

^bNew York City Office of the Chief Medical Examiner (2000–2006). Treatment from NY State Office of Mental Health.

^cNational Comorbidity Study Replication, 2001–2003.

After a burn-in period to stabilize estimates, we ran each model scenario 50 times for 30 years, and we report the median across runs; credible intervals (CIs) reflect variation across the 50 runs, which we varied in sensitivity analyses. To estimate the CI, we rank ordered the 50 estimates for each model run, and report the 2.5th and 97.5th percentile. We developed the model by using Recursive Porous Agent Simulation Toolkit for Java (RepastJ, version 3.0, University of Chicago, Chicago, IL), and implemented in Eclipse (version 4.2, Eclipse Foundation, Ottawa, Ontario, Canada). We implemented 2 ownership prohibitions; each year, an agent could meet disqualification criteria. Agents meeting criteria for each prohibition were restricted from gun ownership and purchasing for 5 years (based on the recommendation of the Consortium for Risk-Based Firearm Policy⁴⁵); gun carrying remained possible for prohibited agents and was calibrated in the model as described previously. The first scenario was to disqualify those with OMH-identified inpatient hospitalization, as defined previously, and constituting 0.11% of the simulated population each year. The second scenario was to disqualify those with past-year mental health treatment, as defined previously, and constituting 13.9% of the simulated sample. We also estimated the number of individuals needed to “treat”: that is, the number that would be exposed to the ownership restriction for each suicide to be prevented, as 1 divided by the suicide rate after implementation of the ownership disqualification minus the rate prior. We multiplied those estimates by the prevalence of the high-risk groups to estimate the number of disqualified individuals needed to prevent each suicide. We considered high-risk groups to be (1) OMH-identified psychiatric hospitalizations and (2) those with any mental health treatment.

RESULTS

Table 1 describes the distribution of demographics and key parameters of interest at baseline and compares these distributions to empirical data sources. Table 2 provides an overall summary of model scenarios and estimated intervention effects, which are graphically depicted in Figures 1 and 2.

TABLE 2—

Simulation of Gun-Related Suicide in New York City After Implementation of Mental Illness Related Ownership Disqualifications Among the Total Agent Population and High-Risk Groups

	Rate per 100 000 (95% CI)	Percentage Decrease (95% CI)
Among the total population
Baseline	2.1 (1.9, 2.3)
Ownership disqualification (5-y duration): OMH-identified psychiatric hospitalizations	2.0 (1.8, 2.2)	2.8 (–7.1, 14.5)
Any mental health treatment in past year	0.8 (0.7, 1.0)	59.8* (50.5, 64.8)
Among high-risk groups
OMH-identified psychiatric hospitalizations
Baseline	15.8 (7.9, 25.9)
5-y restricted ownership	2.4 (0.0, 10.0)	85.1* (36.5, 100.0)
Any mental health treatment in past year
Baseline	3.5 (3.0, 4.1)
5-y restricted ownership	0.6 (0.5, 0.9)	82.9* (76.1, 87.3)

Note. CI = credible interval; OMH = Office of Mental Health.

^*P < .05 significant decrease.

FIGURE 1—

Simulation of Gun-Related Suicide in New York City After Implementation of Mental Illness–Related Ownership Disqualifications

Note. OMH = Office of Mental Health. Based on a simulated sample of 260 000 residents of New York State.

FIGURE 2—

Simulation of Gun-Related Suicide in New York City After Implementation of Mental Illness–Related Ownership Disqualifications, by Targeted High-Risk Subpopulation

Note. OMH = Office of Mental Health.

Effects of Firearm Disqualification

Population rates of firearm suicide.

The average annual baseline rate of firearm-related suicide was 2.1 per 100 000 (95% CI = 1.9, 2.3; lower in New York City than the national rate; Figure 1). Restricting firearm ownership based on OMH-identified psychiatric hospitalizations had no statistically significant influence on the rate of firearm-related suicide (2.0/100 000; 95% CI = 1.8, 2.2). Number-needed-to-treat analysis indicated that approximately 144 000 individuals need to be disqualified to prevent 1 suicide. Disqualifying anyone with mental health treatment significantly decreased the population firearm suicide rate in New York City to 0.8 per 100 000 (95% CI = 0.7, 1.0; a 59.8% [95% CI 50.5%, 64.8%] decrease). Number-needed-to-treat analysis indicated that approximately 724 000 individuals need to be disqualified to prevent 1 suicide.

Rates of firearm suicide among high-risk groups.

We next tested whether restrictions affected suicide rates among very-high-risk groups considered separately (Figure 2): (1) OMH-identified psychiatric hospitalizations and (2) those with any mental health treatment. Restrictions significantly influenced firearm-related suicide among both of the high-risk groups. Restrictions based on OMH-identified psychiatric hospitalizations reduced suicide in this population from 15.8 per 100 000 (95% CI = 7.9, 25.9) to 2.4 per 100 000 (95% CI = 0.0, 10.0; an 85.1% [95% CI = 36.5%, 100.0%] decrease). Restrictions based on any mental health treatment reduced suicide among those with any mental health treatment from 3.5 per 100 000 (95% CI = 3.0, 4.1) to 0.6 per 100 000 (95% CI = 0.5, 0.9; an 82.9% [95% CI = 76.1%, 87.3%] decrease).

Sensitivity Analysis

Our 10 sensitivity analyses are consistent with our main findings, suggesting that model findings for firearm-related suicide did not depend on the number of model runs, the size of the contribution of neighborhood and network characteristics, the prevalence of firearms in the home, or the prevalence of gun carrying. Please see Appendix B for specific results.

DISCUSSION

Firearm disqualification based on OMH-identified inpatient hospitalization did not significantly influence population rates of suicide, given the low prevalence of inpatient psychiatric treatment in the population. However, among those who were hospitalized, suicide rates decreased by 85%. Our agent-based model allowed us to estimate the potential efficacy of 2 different disqualification criteria and to demonstrate the minimum and maximum bounds of impact that disqualification criteria could have on population rates of suicide by firearms. Indeed, we estimated based on the simulation that to have a population impact on suicide rates, disqualifying individuals based on broad, largely unenforceable and unfeasible criteria such as presence in any psychiatric treatment would theoretically reduce firearm suicide rates by more than half, but at large societal costs. For every life saved, approximately 724 000 individuals would be identified and restricted needlessly; they would not have died from suicide even without the restriction. While policymakers have proposed such broad criteria, and some states have enacted such criteria, though enforcement remains equivocal, concerns about stigma and civil rights remain paramount.

Restricting firearm access to those with relatively rare treatment regimens such as OMH-identified inpatient hospitalization is a high-risk approach to public health,⁴⁶ as control is concentrated among a small group of individuals among whom the burden of firearm injury is concentrated. While high-risk approaches are often efficacious in reducing burden among the affected groups,^47,48 these approaches are limited in population-level impact by the small size of the high-risk group relative to the large population that is unexposed to the policy, including many at risk for other reasons. In the case of firearm suicide, criteria to disqualify individuals from gun ownership based on psychiatric hospitalization alone affect an exceedingly small group of individuals. Broader disqualification criteria would theoretically have a more significant effect in reducing suicide at the population level, but identification and enforcement are impractical.

Furthermore, such approaches would hardly be equitable. Embedded inequities—longstanding disparities in access to treatment—would be replicated in gun-disqualifying records. For example, while racial/ethnic minorities have less access to mental health care and receive lower-quality care,^49,50 rates of involuntary psychiatric hospitalization and civil commitment are higher among racial minorities, including in New York State⁵¹; to the extent that such criteria are used for gun disqualification, those disparities will be reproduced in disqualification. Indeed, commonly held assumptions about mental illness and violence are historically embedded in race/ethnicity and class conflict, often based on untenable evidence,¹⁸ suggesting that expansions of gun disqualification on the basis of mental illness may exacerbate stigma linked to multiple marginalized social statuses. Further simulation modeling that assesses the potential unintended consequences on the emergence and exacerbation of disparities is an important gap in the literature for which agent-based modeling is well suited in future investigations.

Public health investments that have focused on population-based approaches often achieve larger reductions in health outcomes (e.g., seat-belt laws,^52,53 alcohol and cigarette taxes,^54–56 vaccine mandates) but often do so at a cost, in that not everyone will benefit from population approaches, and some may even be harmed. Population-based approaches to firearms control include potential policies such as ammunition taxes, gun-free zones, and product-safety engineering, and pursuit of population-based approaches are important adjuncts to reducing firearm violence.

We observed substantial decreases in rates of firearm-related suicide among those who were disqualified from gun ownership under all of our simulated scenarios. However, our agent-based model is calibrated on the basis of available data, which does not reflect all of those who are disqualified from gun ownership. Indeed, OMH data capture only a portion of those who are hospitalized because of psychiatric symptoms, and, furthermore, our model does not capture all those who are disqualified from gun ownership based on mental illness as historical commitment records may still prevent gun purchases during a background check.

New York City is also a geographic area with particularly low suicide rates—only about half the rate for the United States overall in 2002—and with a much smaller proportion of suicides caused by firearms (31% vs 54%).¹ Furthermore, legislative efforts such as the Secure Ammunition and Firearms Enforcement Act in New York that expand the ability of the state to restrict firearm access to those with psychiatric disorders could influence firearm ownership and subsequently rates of firearm suicides in New York City in ways that would not be generalizable to other areas.

Despite these limitations, 2 results from our model do have broad generalizability. First, our results demonstrate that among the small group of individuals that we could calibrate to be disqualified from gun purchase, the reduction in firearm-related suicides associated with firearm denial is very high. To the extent that cities have higher rates of gun ownership and carrying, we would expect greater numbers of deaths to be prevented among all groups with greater restrictions on gun ownership.

Second, the group of individuals at high risk for psychiatric hospitalization is likely to be small in any jurisdiction, indicating that there will be small effects of disqualification on population rates of suicide regardless of prevalence of gun ownership, carrying, or suicide. Across all jurisdictions, our results indicate that disqualifications would need to be broadly defined to have a population impact. Overall, however, were we to apply these results to other calibrations of high-risk individuals in other states, the effects on population rates could be greater as gun access and ownership increase.⁵⁷

Ownership disqualifications based on other risk factors beyond mental health should also be considered. Gun violence restraining orders, also known as extreme risk protection orders, are a promising approach to firearm violence prevention when individuals pose an imminent risk of harm to themselves or others—whether or not mental illness is involved.^58–60 In general, they allow courts to mandate temporary recovery of firearms under such circumstances, following specified procedures and rules of evidence. Formal evaluations⁶⁰ and anecdotal reports suggest they may be effective across a wide range of circumstances.^61,62 Gun ownership restrictions based on timely behavioral indicators of risk, rather than records of psychiatric treatment, are likely to be both more effective and fair, although unintended consequences and equity should be considered for any ownership disqualification.

Several other considerations in the modeling framework affect interpretation of these results. As in all simulations, our results are dependent on a series of modeling assumptions and on the quality of the parameters that we used from existing data. Data on gun ownership and carrying are not routinely collected, and we relied on select sources of survey data to calibrate these aspects of our model. Firearm purchases through illicit marketplaces are common. We calibrated the illicit firearm marketplace in our model by allowing agents to carry firearms without purchase; as data become more available on the dynamics of firearm marketplaces, the development and calibration of agent-based and other simulation models to understand firearm injury dynamics will improve. Furthermore, we focused on specific model dynamics around social networks that facilitate firearm use but did not focus on social network factors that may be protective, such as social support; further extensions of simulation work that include a specific focus on the role of social networks are warranted.

The limitations of gun ownership and carrying data are balanced by our rigorous and robust data on prediction equations for mental illness, treatment, and outcome estimates of suicide. The quality of our data for these parameters underscores the utility of routinely collected high-quality surveillance and survey data that can be used for multiple public health goals, including simulation of anticipated policy and intervention benefits. We drew data from a variety of sources—based in New York City to the extent possible—but to the extent that parameters were not available, we relied on national sources and assumed that the strength of associations would be similar in New York City. However, we conducted extensive calibration and sensitivity analyses of key parameters and assumptions, mitigating concerns about the dependence of the model results on assumptions.

The study suggests that while high-risk approaches such as firearms denial criteria based on OMH-identified psychiatric hospitalization are effective for preventing suicide for those at high risk, broader approaches may be necessary to reduce firearm deaths at the population level, carefully calibrated to mitigate unintended consequences for stigma and help seeking. Our models estimated that denying firearm rights of the much larger population with any mental health treatment would significantly reduce the gun suicide rate at the population level, but at the cost of stigmatizing and needlessly restricting the rights of millions of people at low risk. Designing laws and policies to restrict firearms from those who pose a substantial risk, without abridging the rights of too many people who do not, remains a difficult challenge for suicide prevention.