Improving Suicide Risk Screening and Detection in the Emergency Department

Abstract

Introduction

The Emergency Department Safety Assessment and Follow-up Evaluation Screening Outcome Evaluation examined whether universal suicide risk screening is feasible and effective at improving suicide risk detection in the emergency department (ED).

Methods

A three-phase interrupted time series design was used: Treatment as Usual (Phase 1), Universal Screening (Phase 2), and Universal Screening + Intervention (Phase 3). Eight EDs from seven states participated from 2009 through 2014. Data collection spanned peak hours and 7 days of the week. Chart reviews established if screening for intentional self-harm ideation/behavior (screening) was documented in the medical record and whether the individual endorsed intentional self-harm ideation/behavior (detection). Patient interviews determined if the documented intentional self-harm was suicidal. In Phase 2, universal suicide risk screening was implemented during routine care. In Phase 3, improvements were made to increase screening rates and fidelity. Chi-square tests and generalized estimating equations were calculated. Data were analyzed in 2014.

Results

Across the three phases (N=236,791 ED visit records), documented screenings rose from 26% (Phase 1) to 84% (Phase 3) (χ² [2, n=236,789]=71,000, p<0.001). Detection rose from 2.9% to 5.7% (χ² [2, n=236,789]=902, p<0.001). The majority of detected intentional self-harm was confirmed as recent suicidal ideation or behavior by patient interview.

Conclusions

Universal suicide risk screening in the ED was feasible and led to a nearly twofold increase in risk detection. If these findings remain true when scaled, the public health impact could be tremendous, because identification of risk is the first and necessary step for preventing suicide.

Introduction

The suicide rate in the U.S. has risen almost 30% over the past 20 years.¹ As discussed in a special issue of the American Journal of Preventive Medicine,² healthcare settings play a crucial role in identifying individuals who are at risk for suicide. Emergency departments (EDs) may be particularly important for such efforts.^3–9 Many ED patients have unrecognized risk incidental to their chief complaint.^10–12 In the two largest studies to date, point prevalence of active ideation among ED patients presenting with non-psychiatric complaints was 8%,^13,14 which went undetected by treating ED clinicians. This far exceeds community estimates of around 3% over a period of an entire year.¹⁵ Many individuals who are treated in an ED and then die by suicide do not present for a psychiatric problem and their risk is not detected during the visit.^16,17

The ED Safety Assessment and Follow-up Evaluation (ED-SAFE^18,19) is a multicenter collaborative study with two overarching objectives:

1. to develop and test a standardized approach to universal suicide risk screening within the general medical ED (the Screening Outcome Evaluation); and

2. to test an ED-initiated intervention with follow-up telephone contact to reduce suicidal behavior among people who screen positive for suicide risk (the Intervention Evaluation).

The current paper reports key findings from the Screening Outcome Evaluation.

Methods

A detailed review of the study’s design and methods has been published.¹⁸ ED-SAFE used an interrupted time series design with three phases:

1. Treatment as Usual;

2. Universal Screening; and

3. Universal Screening + Intervention.

Eight participating sites were randomly assigned to one of four cohorts, and each cohort was randomly assigned to a start date and progressed through the three phases sequentially (random stepped wedge design).²⁰ The Screening Outcome Evaluation used data from all three phases. The study was approved by the IRBs of all participating institutions. Because primary data were collected by chart review, the requirement for informed consent was waived. Patients who were approached for additional questioning provided verbal informed consent to be interviewed. All adult ED patients who were treated in one of the eight participating sites during any of the three study phases were eligible for chart review.

Setting

Eight hospitals located in seven states participated. Annual ED census ranged from 31,000 to 54,000. Like most U.S. EDs, participating sites did not have dedicated psychiatric EDs,²¹ to increase the representativeness of the findings.

At baseline, sites generally screened for suicide risk only among those who presented with frank psychiatric symptoms. One site had already implemented universal screening during Phase 1 using a single item screener created by the site. This implementation occurred after the site had been selected for participation (at the time of selection, it did not have universal screening) but before Phase 1 had started and was attributed to the hospital’s risk management concerns about complying with Joint Commission requirements.

Implementing Universal Screening

The investigative team and representatives from all eight sites used the best evidence available to create a screener feasible to implement in emergency settings. The Patient Safety Screener-3 (PSS-3²²) assesses depressed mood, active suicidal ideation in the past 2 weeks, and lifetime suicide attempt. If lifetime attempt is endorsed, the timing of the most recent attempt is identified. A positive screen was defined as active suicidal ideation in the past 2 weeks or a suicide attempt within the past 6 months. The depression item is not factored into screening results; it was included as segue to help “ease into” the suicide items, which can be jarring when asked without context. A positive screen on the PSS-3 has strong agreement (κ=0.95) with the Beck Scale for Suicide Ideation,²³ a well-established suicide risk instrument.²²

Prior to Phase 2, each site convened a performance improvement team to develop strategies the site could use to improve implementation of universal screening. The team composition and performance improvement methods followed those commonly used by the Institute for Healthcare Improvement,²⁴ an industry leader. The team was multidisciplinary, including practicing clinicians to administrators, and focused on using continuous quality improvement cycles. The Plan–Do–Check–Act (PDCA) strategy was used, which included planning the desired changes (Plan), implementing them (Do), measuring performance (Check), and making adjustments (Act). The PDCA process was repeated as often as needed. Each site’s team received centralized training on these methods and their relationship to suicide risk screening to assure a common knowledge base. Although the precise clinical protocols were not dictated, there were two requirements:

1. The site needed to use the PSS-3.

2. The PSS-3 was to be administered universally for all patients, with notation if the individual was inappropriate to screen because of his or her medical condition (e.g., unconscious, cognitively disabled).

Screening during triage for patients who presented with a psychiatric complaint was maintained. However, screening during triage for individuals with non-psychiatric complaints was strongly discouraged because of concerns about slowing ED flow. For most sites, the PSS-3 was administered during the primary nursing assessment once the patient was placed in a treatment area. Some sites chose to use the PSS-3 during triage for psychiatric patients as well. Each site’s performance improvement team created written clinical protocols, which stipulated how positive screens would be managed.

Nurses at each site were either trained by trainers recruited from the site’s clinical staff or through a webinar provided by the ED-SAFE trainer. Each site’s performance improvement team was responsible for collecting its own data on performance independent of the research data (e.g., screening rates), making adjustments to protocols as needed, and providing feedback, additional training, and other small incentives (e.g., $5 gift cards and lunches) to promote performance. An a priori “Excellence” screening rate of 75% was set by consensus.

During Phase 3, sites continued to improve screening using PDCA cycles. See Methods paper^{(pp 15–16)} for additional information on the additional interventions introduced during Phase 3.

Data Collection

Across all phases, sites staffed the ED with research assistants (RAs) at least 40 hours/week during peak volume hours (12:00pm to 10:00pm), with at least 1 weekend day/month. As both volume and enrollment rates decline after 10:00PM, this efficiently maximized the representativeness of the sample. All adult patients who entered the ED during data collection shifts were documented on a Screening Log, and the RA reviewed their medical charts in real time throughout the visit until the patient was discharged. This was done both to obtain screening data (goal of the present study) and to identify subjects for the longitudinal clinical trial. Although the RAs were not blinded to phase, additional data sources were used to expand and validate outcomes abstracted by chart review, including patient interview and random chart review. Information on the fidelity interviews used to validate medical record documentation is reported in Appendix A. Data were collected from 2009 through 2014 and analyzed in 2014.

Outcome Definitions

The study was directed at improving suicide risk screening and detection. There were two primary outcomes. The first outcome, documentation of intentional self-harm ideation or behavior screening, was defined broadly as any documentation of past or current intentional selfharm ideation or behavior appearing in the record as either present or absent. The second outcome, intentional self-harm risk detection, was defined as past or current intentional self-harm ideation or behavior documented as positive in the ED medical record. A broad initial threshold comprising any intentional self-harm ideation or behavior was used because ED documentation may be vague or incomplete as to whether intentional self-harm is suicidal or not, particularly during treatment as usual, and could introduce error and unreliability across raters (Limitations section). Consequently, the protocol tracked documentation of any intentional self-harm ideation or behavior, not just suicidal self-harm ideation or behavior, and clarified the nature of the selfharm using a subsequent direct research interview with the patient. Passive ideation only, such as a desire to go to sleep and not wake up, was not considered a positive screen.

Measures

The Screening Log extracted data from the medical record. The RA recorded demographic information, whether any screening for intentional self-harm ideation or behavior was documented anywhere on the patient’s ED medical record, and whether self-harm ideation or behavior was documented as present (positive screen) or absent (negative screen). A positive screen was further classified by the RA using chart documentation only as current (i.e., self-harm ideation or behavior occurring during or immediately preceding the current ED visit), past (i.e., noted in the past but not currently), or unknown time (i.e., documented without a clear time reference).

In addition, through all three phases, an RA approached all individuals with documentation reflecting intentional self-harm ideation or behavior and directly interviewed the patient. Patients who were incarcerated or too medically, cognitively, or emotionally ill to be interviewed were excluded. Intentional self-harm ideation or behavior was further clarified as suicidal ideation or behavior within the past week, including the day of the visit (yes/no).

Because RAs did not staff the ED 24 hours/day, 7 days/week, a separate sample of 2,400 randomly selected charts was collected (100/site × 8 sites × 3 phases) as a validation sample. The RA recorded whether there was documentation in the chart of:

1. any intentional self-harm ideation or behavior screening;

2. any intentional self-harm ideation or behavior detection;

3. suicidal ideation (past or current); and

4. suicide attempts (past or current).

More information on these procedures is published.²⁵

Statistical Analysis

Analyses were performed using Stata, version 13.1. Data are presented as proportions with 95% CIs and medians with interquartile ranges. Changes in documentation of screening and detection were evaluated by analyzing data from the Screening Log using chi-square tests. Analyses were repeated using random chart review data as a confirmation of trends. To examine trends across the three study phases and control for potential clustering by site, generalized estimating equations with a logit link function were used. All p-values are two-tailed, with p<0.05 considered statistically significant. Time series figures displaying screening and detection rates were generated from Screening Log data.

Results

Across the three phases, RAs reviewed 236,791 ED visit records (Figure 1, Table 1), identified 10,625 patients with any intentional self-harm ideation or behavior, and completed 3,101 patient interviews. The demographics were compared to the National Hospital Ambulatory Medical Care Survey,²⁶ which samples ED visits nationally. The ED-SAFE sample was very similar to the national sample.

Figure 1. CONSORT flow diagram.

Note: This was an interrupted time series trial that used data collected during an index emergency department visit; therefore, there was no follow-up or attrition.

Table 1.

Characteristics of Emergency Department Patients, Overall and by Study Phase

	Overall (n=236,791)		Treatment as usual (Phase 1) (n=94,257)		Universal screening (Phase 2) (n=57,215)		Universal screening + intervention (Phase 3) (n=85,319)		p- value
Characteristics	n	%	n	%	n	%	n	%
Age, in years
Age, mean		45		45		46		46
(SD)	236,744	(19)	94,246	(19)	57,194	(19)	85,304	(19)	<0.001
18–24	32,017	13.5	13,192	14.0	7,614	13.3	11,211	13.1	<0.001
25–34	46,728	19.7	18,861	20.0	11,310	19.8	16,557	19.4
35–44	40,934	17.3	16,667	17.7	9,859	17.2	14,408	16.9
45–54	45,118	19.1	18,124	19.2	10,737	18.8	16,257	19.1
55–64	32,133	13.6	12,047	12.8	7,813	13.7	12,273	14.4
65–74	17,930	7.6	6,859	7.3	4,466	7.8	6,605	7.7
>75	21,884	9.2	8,496	9.0	5,395	9.4	7,993	9.4
ND	47	0.02	11	0.01	21	0.04	15	0.02
Sex									<0.001
Female	129,078	54.5	51,435	54.6	31,583	55.2	46,060	54.0
ND	59	0.02	22	0.02	19	0.03	18	0.02
Race									<0.001
White	138,098	58.3	54,305	57.6	33,672	58.9	50,121	58.8
Black	51,657	21.8	21,708	23.0	12,844	22.5	17,105	20.1
Other	7,577	3.2	2,804	3.0	2,126	3.7	2,647	3.1
ND	39,459	16.7	15,440	16.4	8,573	15.0	15,446	18.1
Ethnicity									<0.001
Hispanic	36,264	15.3	14,535	15.4	8,727	15.3	13,002	15.2
Non-Hispanic	91,387	38.6	29,697	31.5	21,166	37.0	40,524	47.5
ND	109,140	46.1	50,025	53.1	27,322	47.8	31,793	37.3
Day of week									<0.001
Sunday	7,934	3.4	2,842	3.0	1,871	3.3	3,221	3.8
Monday	46,093	19.5	16,935	18.0	11,093	19.4	18,065	21.2
Wednesday	46,910	19.8	19,019	20.2	11,218	19.6	16,673	19.5
Thursday	44,327	18.7	17,916	19.0	11,674	20.4	14,737	17.3
Friday	30,502	12.9	13,494	14.3	6,950	12.2	10,058	11.8
Saturday	11,737	5.0	4,545	4.8	2,255	3.9	4,937	5.8
Site									<0.001
1	42,027	17.8	15,950	16.9	8,079	14.1	17,998	21.1
2	19,140	8.1	6,798	7.2	4,801	8.4	7,541	8.8
3	31,378	13.3	9,934	10.5	8,873	15.5	12,571	14.7
4	22,306	9.4	14,870	15.8	2,867	5.0	4,569	5.4
5	39,872	16.8	15,174	16.1	11,065	19.3	13,633	16.0
6	28,675	12.1	10,786	11.4	7,868	13.8	10,021	11.8
7	28,120	11.9	12,679	13.5	8,507	14.9	6,934	8.1
8	25,273	10.7	8,066	8.6	5,155	9.0	12,052	14.1

ND, not documented

Note: Boldface indicates statistical significance (p<0.05).

Documentation of screening for any intentional self-harm ideation or behavior improved from 26% in Phase 1 to 73% and 84% in Phases 2 and 3, respectively, χ² [2, n=236,789]=71,000, p<0.001, p for trend adjusted for site clustering <0.001). Similar increases were noted using random chart review data (24%, 72%, and 84% in Phases 1, 2, and 3, respectively [χ² (2, n=2,398)=679, p<0.001, p for trend adjusted for site clustering <0.001]). The rate of screening across phases for each site can be seen in Figure 2. The Phase 3 screening rates by site ranged from 63% to 98%, with seven sites reaching >75%.

Figure 2. Time series plot of screening rates by site across the three phases.

Lines represent the percentage of patients that were screened for intentional self-harm. One site (dark blue) achieved 95% screening in Treatment as Usual, because hospital administration interpreted the Joint Commission Patient Safety Goal 15 as requiring universal screening. This site implemented the screening after site selection had been completed for the study but before Treatment as Usual began. In Phase 2, this site switched over to the PSS-3 and continued universal screening.

Detection of any intentional self-harm ideation or behavior rose from 2.9% in Phase 1 to 5.2% and 5.7% in Phases 2 and 3, respectively (χ² [2, n=236,789]=902, p<0.001), with the largest increase occurring between Phases 1 and 2 (p with adjustment for site clustering=0.008) (Table 2). Documentation of intentional self-harm ideation or behavior in the week prior to the visit rose from 2.7% to 3.9% (χ² [2, n=236,789]=205, p<0.001), while past intentional self-harm ideation or behavior at any time other than the past week rose from 0.3% to 1.9% (χ² [2, n=236,789]=1,200, p<0.001). Analyses with random chart review data revealed similar increases (2.8%, 5.8%, and 7.3% in Phases 1, 2, and 3, respectively, χ² [2, n=2,398]=17, p<0.001). Appendix Figure B1 depicts the risk detection rates across all three phases over time. Multivariable logistic regression models adjusting for patient clustering by site and demographics did not materially affect the results (Appendix Table B1).

Table 2.

Screening Log and Random Chart Review Risk Detection Rates by Phase

	Treatment as Usual (Phase 1)				Universal Screening (Phase 2)				Universal Screening + Intervention (Phase 3)
	n	%	95% CI		n	%	95% CI		n	%	95% CI		p for trend adjusted for clustering by site
Screening Log (n=236,791) Documentation of…
Any self-harma (positive)	2,771	2.9	2.8	3.0	2,953	5.2	5.0	5.3	4,901	5.7	5.6	5.9	<0.001
Current self-harm b (positive)	2,516	2.7	2.6	2.8	1,863	3.3	3.1	3.4	3,299	3.9	3.7	4.0	<0.001
Past self-harmc (positive)	255	0.3	0.2	0.3	1,090	1.9	1.8	2.0	1,602	1.9	1.8	2.0	<0.001
Random Chart Review (n=2,400) Documentation of…
Any self-harma (positive)	22	2.8	1.8	4.1	46	5.8	4.3	7.6	58	7.3	5.6	9.3	0.009
Any suicidal ideation (positive)	18	2.3	1.4	3.5	27	3.4	2.3	4.9	22	2.8	1.8	4.1	0.62
Current suicidal ideationd (positive)	8	1.0	0.5	2.0	15	1.9	1.1	3.1	16	2.0	1.2	3.2	0.28
Past suicidal ideatione (positive)	10	1.3	0.7	2.3	12	1.5	0.9	2.6	6	0.8	0.3	1.7	0.17
Any suicide attempt (positive)	14	1.8	1.0	2.9	32	4.0	2.8	5.6	46	5.8	4.3	7.6	0.02
Current suicide attemptf (positive)	2	0.3	0.1	1.0	6	0.8	0.3	1.7	3	0.4	0.1	1.2	0.72
Past suicide attemptg (positive)	12	1.5	0.9	2.6	26	3.3	2.2	4.7	43	5.4	4.0	7.2	0.02

^aAny self-harm = Any intentional self-harm ideation or behavior noted on ED medical record

^bCurrent self-harm = any intentional self-harm ideation or behavior within the past 2 weeks, including day of visit

^cPast self-harm = any intentional self-harm ideation or behavior older than the past 2 weeks, or unknown/unspecified time

^dCurrent suicidal ideation = any thoughts about killing oneself within the past 2 weeks, including day of visit

^ePast suicidal ideation = any thoughts about killing oneself older than the past 2 weeks, or unknown/unspecified time

^fCurrent suicide attempt = any attempt to end one’s life within the past 2 weeks, including day of visit

^gPast suicide attempt = any attempt to end one’s life older than the past 2 weeks, or unknown/unspecified time

Note: Boldface indicates statistical significance (p<0.05).

Direct patient interview data (n=3,101) found that, across all three phases, approximately 74% (95% CI=73%, 76%) of patients with any intentional self-harm reported suicidal ideation or an attempt within the past week, supporting that the majority of the self-harm detection was relatively recent suicidal ideation or behavior.

Discussion

Although research has repeatedly suggested that ED patients have significant undetected suicide risk, this is the first study to address the key question of whether detection feasibly can be increased by implementing universal suicide risk screening protocols in the ED. Two main conclusions can be reached. First, increasing documented suicide risk screening rates in the general adult ED population during routine care was achieved using a simple screener and Institute for Healthcare Improvement performance improvement protocols. Second, increased screening led to nearly twice as many patients being identified as having suicide risk, either by virtue of endorsing active suicidal ideation or a past suicide attempt. If applied to the approximately 350,000 ED visits that occur annually at the eight participating EDs alone, nearly 10,000 additional patients with previously undetected suicide risk would be identified every year.

Documented screenings rose from an average of 26% in Phase 1 to an average of 84% in Phase 3, representing an increase of more than 300% (Figure 1). Moreover, this increase in screening led to a nearly twofold increase in identification of patients positive for suicide risk. Documentation of any past or current intentional self-harm ideation or behavior nearly doubled over the course of the study from 2.9% to 5.7%. Patient interviews confirmed that the majority of this intentional self-harm was suicidal ideation or behavior that occurred in the weeks before the ED visit. Findings using data obtained during active research shifts, which disproportionately captured hours from 9am to 10pm, were reinforced by findings from random chart reviews. Furthermore, the positive trends in screening and detection persisted even after adjusting for factors such as site, demographics, and fidelity (Appendix A).

Importantly, steps were taken to foster successful screening adoption of the screening. A simple screening instrument and clinical protocols designed to be easily integrated into the ED routine were developed. Training was brief and available through multiple channels, including in-person by a site trainer or online. Sites used widely available performance improvement methodologies, including PDCA cycles, for integrating the protocols into routine care and monitoring performance.

Limitations

There are several potential limitations with this study. The participating EDs, though quite diverse, may not represent the nation’s EDs. Further study will be needed to examine whether these protocols, and their subsequent increase in risk detection, can be successfully translated to other EDs.

The RAs were not blinded to study phase. The most important bias this could have caused was to artificially inflate the improvements in suicide risk screening and detection. However, the authors reduced the potential for this bias in several ways. First, RAs were trained on research protocols, which were highly structured, designed to minimize the role of RA interpretations, and closely monitored using both site-specific and centralized quality assurance procedures. Second, RAs were not involved in any of the intervention training or activities. Finally, RAs interviewed all patients who screened positive on the chart to validate the documentation with the patient. If the RAs had been influenced by their knowledge of phase and artificially inflated the documentation of positive screenings, there would have been more patients identified as a positive screen on the chart but who denied any intentional self-harm. This rarely happened (less than 1% of the time across the study).

The study did not randomize individuals or sites due to study design, ethical, and legal considerations (described in Methods paper¹⁸). The quasi-experimental design could have introduced a change in clinician behavior simply as a result of being observed (Hawthorne Effect). There was no way to eliminate or measure this, but research protocols were designed to be unobtrusive. The random chart reviews captured data from all days and times to provide an indicator of performance less likely to be influenced by RA presence. Finally, the time series plots show a trend of performance improving over time, which is the opposite of tapering trends usually observed with a Hawthorne Effect as clinicians acclimate to being observed.

The RAs were trained to first identify any documentation of intentional self-harm, whether it was specifically noted as suicidal or not, and to interview these patients to ascertain whether the self-harm was suicidal. This was done because clinical documentation may be inadequate to determine intent. Relying on RAs to ascertain if the self-harm noted was suicidal based solely on documentation might have excluded valid cases. This would have been especially true during treatment as usual prior to the systematic changes in protocols that trained clinicians to document suicidal ideation and behavior more precisely using the PSS-3. The most likely impact of including only charts with clear documentation of suicidal self-harm would be to artificially inflate the screening rates for Phases 2 and 3 relative to Phase 1. Clinicians simply could have become more specific in their self-harm documentation as a result of training and improved templates, rather than increasing screening. Consequently, the inclusion criterion was broad, followed by direct interview to ascertain whether the intentional self-harm documented was indeed suicidal in nature. Patient interviews and careful examination of PSS-3 response patterns in Phase 3 confirmed that the vast majority of intentional self-harm was composed of recent suicidal ideation or behavior.

Conclusions

Such a remarkable and robust increase in screening by clinicians during routine care in busy EDs, and the subsequent increase in risk detection that would not have been identified if universal screening had not been implemented, has never before been published. This landmark finding, if scaled beyond the ED-SAFE sites, has the potential to dramatically improve identification of nascent suicide risk in a highly vulnerable group, thereby paving the way for applying interventions to reduce subsequent suicidal behavior.