Comparison of National and Local Approaches to Detecting Suicides in Healthcare Settings

Abstract

Introduction:

In order to address the problem of suicide, healthcare providers and researchers must be able to accurately identify suicide deaths. Common approaches to detecting suicide in the healthcare setting include the National Death Index (NDI) and Root-Cause Analysis (RCA) methodology. No study has directly compared these two methods.

Materials and Methods:

Suicide reporting was evaluated within the Veterans Affairs (VA) healthcare system. All suicides were included that occurred within 7 days of discharge from an inpatient mental health unit and were reported to the VA through the NDI record linkage and/or RCA database between 2002 and 2014. The proportion of suicide deaths that were identified by NDI and found in the RCA database were calculated. Potential misclassification by the NDI was evaluated, whereby the RCA database identified a suicide case, but the NDI classified the death as a non-suicide.

Results:

In the study period, the NDI identified 222 patients who died by suicide within 7 days of discharge, while the RCA database only detected 95 reports of suicide. A comparison of cases across the two methods indicated that the RCA database identified only 35% (N = 78) of NDI detected suicides (N = 222). Conversely, the RCA database detected 13 suicide cases that the NDI had coded as deaths due to accidental poisoning or other causes. Importantly, RCA accounted for 13% (N = 7) of overdose suicides identified in all databases (N = 52).

Conclusions:

Combining national and local approaches to detect suicide may help to improve the classification of suicide deaths in the healthcare setting.

INTRODUCTION

Accurate classification of suicide deaths is critical both to understand the burden of suicide and to develop and evaluate effective strategies to prevent suicide. For example, researchers and policymakers must be confident that the troubling 30% increase in suicide in the USA from 2000 to 2016 truly reflects an increase in suicide mortality requiring allocation of limited resources.¹ Innovative suicide prevention strategies, such as identifying and targeting interventions to those at risk using predictive modeling, also rely heavily on accurate classification of suicide deaths,^2,3 and are of reduced value when there are misclassifications in the outcome.³ Finally, the recent rise in opioid overdose deaths has heightened the need for accurate estimation of suicide as studies have raised concerns that opioid-related suicides are often misclassified as undetermined or accidental deaths.⁴ Together these factors suggest that the validity of surveillance and prevention efforts is dependent upon accurate suicide classification.

Several methods exist to detect death by suicide. At the national level, medical examiners and coroners conduct death investigations and report these findings to state health departments, who in turn share their findings with the Center for Disease Control and Prevention (CDC). The CDC aggregates these data into the National Death Index (NDI), which allows researchers and policymakers to gain important insights into the national burden of suicide and identify emerging trends. There are, however, potential limitations to this method. First, there is a considerable lag time in NDI reporting, which precludes use of the NDI for real-time analysis and clinical improvement activities. For example, the CDC did not release the 2016 US suicide data until June of 2018 preventing us from learning from these events in realtime. Second, suicide death investigation is not standardized across the USA introducing variability into the accuracy of classification.⁵

Another source of information on death by suicide is adverse event reporting within health care systems. The Joint Commission requires that clinical teams perform root cause analysis (RCA) on any inpatient suicide and any outpatient suicide that occurs within 72 hours of discharge.⁶ RCAs were instituted to identify and mitigate systematic vulnerabilities that are associated with rare adverse (sentinel) events.⁷ As part of RCA, a multidisciplinary team uses a standardized approach to evaluate a suspected suicide death. This includes reviewing medical records, conducting site investigations (if applicable) and gathering data from key informants (e.g., providers, family, medical examiners, coroners, and law enforcement). Because a RCA is typically initiated soon after the suicide and must be completed within 45 days, the method allows for real-time analysis and clinical improvement. Yet, RCAs also have their limitations. As a source of epidemiologic data, RCA reports are at risk for measurement bias, as healthcare teams may not be aware of some postdischarge suicides. They also require significant resources and healthcare systems may not have the resources to conduct RCAs on all identified events.

Although a prior study has evaluated methods to detect suicide attempts,⁸ no study has combined NDI and RCA databases to compare their ability to detect suicide in the period following hospital discharge, when these two sources potentially overlap. The Department of Veterans Affairs (VA) provides a unique opportunity conduct these analyses as it maintains all RCAs in a centralized database and has developed the Suicide Data Report (SDR), a centralized suicide database that includes all NDI suicides and can be linked to VA administrative healthcare databases. We performed a study comparing contemporaneous cases of suicide mortality in the VA RCA database and the NDI. Our primary aim was to determine the proportion of post-discharge deaths by suicide found in the NDI that were captured in the VA RCA database. Our secondary aim was to determine whether the NDI misses any suicides detected by the VA RCA database. Finally, we explored potential reasons for discrepancies in reporting between these two methods. An improved understanding of the comparative strengths and weaknesses of these two methods of suicide detection may help in developing more robust strategies to assess and prevent suicide that balance timeliness with comprehensiveness.

METHODS

We conducted a comparison study evaluating suicide reporting within the VA. We included deaths by suicide that were reported to the VA through the RCA process and through the NDI record linkage between 2002 and 2014. While the Joint Commission requires RCAs to be completed for any suicide within 72 hours of discharge from an inpatient mental health unit, we extended our period of observation to 7 days post hospital discharge.⁶ This is because the VA instituted an additional policy in 2010 requiring that facilities complete a RCA for any known suicide within 7 days of inpatient mental health discharge.⁹ The project was approved by the Institutional Review Board at the Institutional Review Board at the Syracuse VA Medical Center.

Databases

The VA integrates multiple sources of data to attain the aim of accurately monitoring veteran suicide and support suicide prevention efforts. The VA collects NDI reported deaths (and causes of death) for all VA decedents via the VA/ Department of Defense (DoD) Suicide Data Repository (SDR). The SDR was created in collaboration with the NDI and can be linked directly to medical records aggregated in the VA corporate data warehouse (CDW) to assist in suicide surveillance and research efforts.¹⁰ In addition, the VA National Center for Patient Safety (NCPS) maintains a centralized RCA database that includes more than 17,000 RCA reports of adverse events occurring across VA medical centers. The RCA database contains free-text fields which include detailed information on the adverse event, the timeline of events leading to the event, the final understanding of the event, root causes, and contributing factors.⁷

Identification of Cohort

First, we searched the SDR for all suicides that occurred within 7 days of discharge from any VA inpatient mental health unit. These deaths were linked with medical information that was available in the CDW. We considered the individual to have died by suicide if any of the following ICD-10-CM codes were assigned: X60-X84 and Y87.0. Second, we searched the RCA database for RCA reports that mentioned suicide as the event type in the incident field. We supplemented information in the RCA event type field by using PolyAnalyst text mining software to review the entire text of each RCA report to search for reference to the term suicide. After locating RCA reports of suicide, we searched the free text fields to identify those reports that referred to a suicide that occurred within 7 days after discharge from a VA inpatient mental health unit. If the days between suicide and discharge were not described in the narrative report, we manually calculated the difference based on the dates provided. We matched cases across the SDR and RCA databases by date of death, facility, and cause of death as RCAs do not include identifying information. After matching was complete, we collected additional variables including ICD-10 codes for mental health diagnoses, and suicidal ideation and attempts from medical records in the CDW.

Analysis

For our primary analysis, we calculated the proportion of suicide deaths in the SDR that were found in the RCA database. We evaluated year-to-year changes in agreement using a line graph, and calculated confidence intervals using Wilson’s method, the preferred method for small samples.^11–13 We then compared patient diagnoses, prior suicidal behavior, and method of suicide for SDR suicides that were (and were not present) in the RCA database. For our secondary analysis, we looked for potential misclassification of suicide by the NDI. Specifically, we identified cases that were called suicide in the RCA database, but were classified as other causes in the SDR. In these cases, we abstracted the alternative cause of death (ICD-10) assigned by the NDI. For comparisons, we used Chi-squared test for dichotomous measures and Kruskal–Wallis test for continuous measures. Because the frequency of certain factors was very low (e.g., Schizophrenia), we used Fisher’s Exact test instead of chi-square. We considered a p < 0.05 to be statistically significant. Although we considered adjusting for multiple statistical tests, we decided against it due to the preliminary nature of the analyses and the small number of suicides available for analysis.

To complete our secondary analysis, we performed a qualitative review of RCA cases that were potentially misclassified by the NDI. Two independent reviewers (PB, NR) abstracted information that was used by the RCA team to determine whether the injury was self-inflicted and the individual had intent. We used this information to determine whether there was sufficient data in the RCA report to conclude that the death may have been misclassified as non-suicide by the NDI.

RESULTS

Between 2002 and 2014, the NDI detected a total of 222 patients who died by suicide within 7 days of discharge from a VA inpatient mental health unit. During this same period, the RCA database contained a total of 95 reports of death by suicide within 7 days of VA psychiatric discharge that could be linked to deaths identified in the linked CDW/ SDR database.

Percent of 7 Days NDI Suicides Receiving an RCA

Figure 1 describes the percentage agreement in suicide reporting between the NDI and the RCA database over the 12 -year study period. Overall, the RCA database detected only 35% (N = 78) of the 222 suicide cases identified through the NDI. While the percentage agreement between the two systems was generally low, there was a notable improvement in agreement between 2010 and 2012 (60% and 59%, respectively). This improvement overlaps with the VA instituting an additional policy requiring that facilities conduct RCA on any suicide occurring within 7 days of inpatient mental health discharge.⁹ The improvement, however, appears to be short lived and percentage agreement dropped to its lowest point (18%) in 2014.

FIGURE 1.

Percent National Death Index suicides within three and seven days that were also reported in Root Cause Analysis reports*. NDI = National Death Index; RCA = Root Cause Analysis. *Includes suicides reported within three days and seven days of discharge from any Veterans Affairs inpatient mental health unit.

As shown in Table I, there were no significant differences in mental health disorders between suicides present and missing in the RCA database. However, the suicide method did show differential effects on matching with suicides by firearm more likely to result in an RCA.

TABLE I.

Factors Associated with RCA Misses for NDI Suicide Cases (N = 222), 2002–2014

Diagnosis	RCA Match for NDI Suicide (N = 78)		RCA Misses for NDI Suicide (N = 144)
	%	n	%	n
Any depression	44.8	35	55.8	80
Major depression	35.9	28	36.8	53
Bipolar disorder	15.4	12	20.8	30
Schizophrenia	6.4	5	11.1	16
Other psychotic disorder	3.8	3	2.8	4
Posttraumatic stress disorder	21.8	17	23.6	34
Other anxiety disorder	14.1	11	12.5	18
Alcohol use disorder	37.2	29	44.4	64
Substance use disorder	30.8	24	30.0	43
Suicidal ideation	33.3	26	22.2	32
Suicide attempt	2.6	2	3.5	5
Comorbidity, mean (SD)	−0.12	0.8	−0.05	0.8
Method*
Firearm**	51.3	40	31.9	46
Overdose	14.1	11	23.6	34
Hanging	21.8	17	22.2	32
Other	12.8	10	22.2	32

n = number; N = total number; NDI = National Death Index; % = percent;

^*p < 0.05;

^**p < 0.01.

Potential Misclassification of Suicide by the NDI

There were 13 cases of suicide death that were detected by RCA, but coded as death by another cause in NDI. All discrepant misclassifications were categorized as accidents or assaults by the NDI. Specifically, the NDI coded 54% (N = 7) of these cases as deaths due to accidental poisoning and 46% (N = 6) as deaths due to other causes (e.g., pedestrian injured in collision, firearm assault). While there was a trend that the NDI more frequently misclassified deaths in patients with PTSD and any psychotic disorder, the results were not significant (p = 0.07). Overall, it was evident in reviewing the reports that the RCA team likely uncovered information in the course of their investigation that was not available to the medical examiner or coroner at the time of their assessment. These data led the RCA team to conclude that the death was a suicide.

In addition to the aforementioned cases, there were four cases that received a RCA because of suspected death by suicide, but were coded as death due to other causes by the NDI. In reviewing these RCA reports, we determined that the RCA team had, in fact, concluded during the course of their own investigation that the death was not a suicide. These findings were consistent with those reported by the NDI.

DISCUSSION

The purpose of this study was to compare and contrast two available approaches to detecting suicide in the healthcare setting. Despite existing policies that require RCA to be conducted on all post-discharge suicides, most suicides that occur within 7 days after discharge from an inpatient mental health unit are not captured by RCA, which raises questions regarding the representativeness of RCA for suicides that occur within a week of hospital discharge. Conversely, NDI may misclassify some RCA-identified suicides as non-suicides. Unlike medical examiners and coroners, RCA teams have direct knowledge about the patient as they have broad access to medical records, and communicate with a variety of collateral sources when completing the RCA process. As a result, they may obtain critical information regarding suicidal thoughts and behaviors prior to the event. This information may be especially helpful for determining intent of non-firearm deaths, particularly accidents or assaults. For example, medical examiners or coroners may be unaware that an individual shared thoughts about jumping to their death during their inpatient stay, which would help them classify a fall resulting in death within a week discharge as a suicide. Our findings argue that a multi-pronged approach to suicide detection may yield more precise insights into suicide burden than can be obtained from relying on a single assessment method.

The finding that RCA missed many suicides is not surprising. Although VA policy requires that RCA be conducted on all suicides occurring within 7 days of discharge from an inpatient mental health unit, it is difficult for VA medical centers to accomplish this for multiple reasons. First, formal policy requiring RCA on suicides within 7 days of psychiatric discharge was not implemented by the VA until 2010,⁹ although the gain from such policy appears to be short lived. Second, while we observed brief gains in 3-day reporting in 2003, 2006 and 2009, we are unable to draw any strong conclusions about the reasons for these findings. As our graph suggests, RCA reporting of 3-day postdischarge suicides has never been a stable process within the VA. We can only speculate that the decision to centralize oversight of hospital-related suicide deaths within NCPS in 2009 may explain the brief uptick in RCA reports that year. Third, healthcare teams may not be aware of all suicides because they are not informed about the event. This is especially true if staff at the facility were not informed of the death. Third, as opposed to the medical examiner or coroner, healthcare teams may have differing knowledge about the events that led up to the death. As a result, healthcare teams may determine that a post-discharge death was not due to suicide, which would not result in an RCA. Unfortunately, we do not have reliable data on whether VA facilities were aware of the post-discharge deaths described in the NDI or how they decided whether or not to perform an RCA. Lastly, hospitals may be aware of suicide deaths and simply elect not to do an RCA, likely due to preferential allocation of resources to problems that address patient problems as they occur rather than after they occur. Further research aimed at understanding these situations is clearly warranted.

While RCA missed many cases of suicides, it is notable that 6% (13/235) of the linked suicide records in the week following discharge from a psychiatric hospitalization were suicide detected by RCA but misclassified in the NDI as death due to accidents or other causes. This raises concern that small year-to-year changes in the suicide rate may be subject to misclassification bias and corroborates growing concerns in the scientific community that national surveillance systems may underestimate suicide burden.^14,15 It appears that certain methods of suicide may be more subject to misclassification, and also that misclassification is potentially more commonly associated with several specific psychiatric diagnoses, such as posttraumatic stress disorder and psychotic disorders. Although the overall percentage of suicides detected by RCA, but misclassified by NDI was small, it raises questions about the potential use of other healthcare and societal data sources to further improve classification efforts. Further research is therefore needed to better understand the potential of standardizing utilization of healthcare databases to improve cause of death classification.

The determination of suicide is particularly problematic in the case of a non-firearm suicide. Medical examiners and coroners determine whether an event such as an opioid overdose death was “intentional” or “unintentional.” When intent is not definitive, the default is to label an event an “accident” or “undetermined”, rather than suicide.¹⁴ In examinations of overdose deaths, it has been estimated that there may be a drastic misclassification of overdose deaths as accidental rather than due to suicide, leading to an under-identification of opioid-related suicides.^14,16 Such misclassifications are believed to largely be a result of underfunded emergency healthcare and forensic investigation systems.¹⁴ In addition, medical examiners and coroners (who are often elected officials) may face political and social pressures to classify a death as non-suicide. For example, survivors may be otherwise ineligible to receive insurance benefits.^14,16 Our finding that over 50% of misclassifications were overdose deaths suggest that local data including RCAs may be useful in reducing the under-identification of opioid-related suicides. Of the 52 overdose deaths due to suicide identified in all databases, RCA accounted for 13% (N = 7) which could make a small, but important improvement in reducing the under-identification of opioid-related suicides.

Our study has several strengths. We conducted our analysis using two well-established methods for suicide detection. Our study reports on suicide deaths that have occurred within the largest healthcare system in the U.S. and the study period spans 12 years. To the best of our knowledge, our study is the first to compare suicide reporting across several different methods for suicide classification in the postdischarge setting. Therefore, our study represents an important first step to utilizing our current knowledge to begin to address ongoing national concerns regarding suicide misclassification.¹⁵ Despite these strengths, there were some limitations. RCAs were only conducted on a small subset of suicides in the week after discharge, they were not chosen randomly, and there is no indication that they are representative of all VA suicides in the week after discharge. This suggests that there could have been some selection bias. It is also possible that despite the rigor of our methods to identify RCA of post-discharge suicides, we may have overlooked some RCA reports that met our inclusion criteria.

CONCLUSIONS

We believe that data from medical records and healthcare practitioners can potentially improve the classification of suicide in the post-discharge setting. While national surveillance systems are certainly a more comprehensive measure of suicide deaths in the USA, these systems may be at risk for misclassification in certain cases. This may lead to artificially depressed suicide rates and undermine the validity of surveillance and prevention efforts.^14,15 Conversely, given that medical examiners and coroners encounter important challenges in conducting medicolegal death investigations,¹⁴ it is also possible that some of these biases could contribute to an overestimation of actual suicide rates by national surveillance systems. Thus, we advise caution in considering any source of information on death by suicide to be a “gold standard.” Instead, it may be important to combine findings from these national surveillance systems with local information collected by healthcare systems, including through RCA methodology. Healthcare teams offer unique insight into suicide deaths because they more readily have access to the individuals thoughts and behaviors immediately preceding the suspected suicide, especially in the case of a post-discharge suicide. Thus, linking national and local suicide surveillance systems may improve our overall understanding of suicide burden, and permit more real time assessment and improvement to prevent future suicide, especially if system factors contributed to the suicide.