Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Dec 1.
Published in final edited form as: Suicide Life Threat Behav. 2020 Jul 24;50(6):1097–1104. doi: 10.1111/sltb.12657

C-SSRS Performance in Emergency Department Patients at High Risk for Suicide

Lily A Brown 1, Edwin D Boudreaux 2, Sarah A Arias 1, Ivan W Miller 1, Alexis M May 1, Carlos A Camargo Jr 3, Craig J Bryan 4, Michael F Armey 1
PMCID: PMC7746629  NIHMSID: NIHMS1639469  PMID: 32706437

Abstract

Objective:

To evaluate the psychometric and predictive performance of the Columbia-Suicide Severity Rating Scale (C-SSRS) in emergency department (ED) patients with suicidal ideation or attempts (SI/SA).

Methods:

Participants (n = 1,376, mean age 36.8, 55% female, 76.8% white) completed the CSSRS during the ED visit and were followed for one year. Reliability analyses, exploratory structural equation modeling, and prediction of future SA were explored.

Results:

Reliability of the Suicidal Ideation subscale was adequate, but was poor for the Intensity of Ideation and Suicidal Behavior subscales. Three empirically derived factors characterized the C-SSRS. Only Factor 1 (Suicidal Ideation and Attempts) was a reliable predictor of subsequent SA, though odds ratios were small (ORs: 1.09 – 1.10). The original CSSRS Suicidal Ideation and Suicidal Behavior subscales and the C-SSRS ED screen predicted subsequent SA, again with small odds ratios (ORs: 1.07 – 1.19, CI95%: 1.01, 1.29). In participants without a SA history, no C-SSRS subscale predicted subsequent SA. History of any SA (OR: 1.98, CI95%: 1.43, 2.75) was the strongest predictor of subsequent SA.

Conclusions:

The psychometric evidence for the C-SSRS was mixed. History of a prior SA, as measured by the C-SSRS, provided the most parsimonious and powerful assessment for predicting future SA.

Keywords: Columbia-Suicide Severity Rating Scale, Emergency Department, Suicide Attempts

C-SSRS Performance in Emergency Department Patients at High Risk for Suicide

Suicide risk is a common chief complaint in emergency departments (EDs; Ting, Sullivan, Boudreaux, Miller, & Camargo, 2012). The ED offers a critical opportunity for implementing brief, effective suicide risk assessments and interventions. However, the average ED is overcrowded, resulting in pressure for the most parsimonious evaluations possible (Boyle, Beniuk, Higginson, & Atkinson, 2012). Thus, ED physicians are reliant on suicide screening tools that balance efficiency and accuracy in selecting their clinical intervention.

A few prior studies have explored the performance of the Columbia-Suicide Severity Rating Scale (C-SSRS; Posner et al., 2011) in ED samples. The C-SSRS includes three subscales: Suicidal Ideation, Intensity of Ideation, and Suicidal Behavior. In adults presenting to a psychiatric ED, the C-SSRS correctly identified suicide attempts (SA) and non-suicidal self-injury (NSSI) (Brown, Currier, Jager-Hyman, & Stanley, 2015). However, this study was not designed to predict SA based on C-SSRS scores. In adolescent psychiatric ED patients, the Suicidal Ideation subscale, Intensity of Ideation subscale, lifetime NSSI, and history of SA were predictive of future SA over an 18-month follow-up period (Horwitz, Czyz, & King, 2015). The predictive power of the Intensity of Ideation subscale was driven by suicidal ideation (SI) frequency and duration (Horwitz, Czyz, & King, 2015). However, follow-up data was only gathered through chart review and was not supplemented with a clinical assessment; therefore, important classifying information may have been missed. In a pediatric ED, history of NSSI and the Intensity of Ideation subscale predicted future SA whereas history of SA and the Suicidal Ideation subscale did not (Gipson, Agarwala, Opperman, Horwitz, & King, 2015). In the ED Safety Assessment and Follow-up Evaluation (ED-SAFE) study (Boudreaux et al., 2013; Miller et al., 2017), the Suicidal Ideation subscale, history of NSSI, history of interrupted attempts, and history of actual SA had the most evidence for prediction of SA when examined independently (Arias et al., 2015). However, this study did not examine the psychometric performance of the CSSRS subscales or empirically derived subscales of the measure.

The psychometric performance and predictive power of C-SSRS subscales has also been explored in psychiatric inpatients, another group at higher risk for suicide (Madan et al., 2016). The Suicidal Ideation and Suicidal Behavior subscales had excellent internal consistency, but the Intensity of Ideation subscale had poor reliability (Madan et al., 2016). Two factors best characterized the data, separating the Intensity of Ideation subscale from other variables. The Suicidal Ideation and Suicidal Behavior factor had significant but small correlations with SA at 6-month follow-up. However, this study did not explicitly recruit patients with suicide risk. No study to our knowledge has examined the psychometric performance and predictive power of the C-SSRS subscales in patients presenting to the general ED with SI or SA.

The purpose of the current study was to build on prior findings (Madan et al., 2016) in a sample of ED patients who screened positive for SI or SA and were followed for one year through the ED-SAFE study (Boudreaux et al., 2013; Miller et al., 2017). This study extends a prior report (Arias et al., 2015) by analyzing the psychometric properties of the C-SSRS and by examining empirically derived components of the measure. We used confirmatory factor analysis (CFA) to test the two-factor structure derived from prior research (Madan et al., 2016). Additionally, we attempted to predict SA up to one year later using the following: 1) empirically derived components of the C-SSRS; 2) original C-SSRS subscale; 3) a single C-SSRS item reflecting history of SA; and 4) a restricted set of items reflected in the C-SSRS for ED Screen. We hypothesized that, consistent with prior research (Madan et al., 2016), a two-component structure would best fit the data and that each of the components and original subscales would be associated with SA up to one year after baseline. Consistent with a robust literature, (Klonsky, May, & Saffer, 2016; Ribeiro et al., 2016) we hypothesized that the single C-SSRS item reflecting history of prior actual SA would have the strongest predictive power.

Methods

Participants

Participants (n = 1,376) were mostly middle-aged (mean = 36.8, SD = 13.1), female (55%), white (76.8%), well-educated (79% with at least a high school diploma), and single (50.9% never married). They were recruited from eight hospital EDs following a chart review documenting a self-harm, SA or SI. Inclusion criteria were: at least 18 years of age; past-week thoughts of killing oneself; past-week actual, interrupted, or aborted suicide attempt (including at the time of the emergency department encounter); ability to consent; and willingness to complete telephone assessments at 6, 12, 24, 36, and 52 weeks later. Exclusion criteria were: inability to participate due to cognitive or medical reasons; dwelling in a non-community setting; in state custody; pending legal action; lack of permanent residence; lack of reliable phone service; or insurmountable language barrier. Following medical stabilization and approval from the attending physician, patients with any level of self-harm ideation were approached to participate in the study and assessed for eligibility. In the week prior to hospitalization, 460 (33.43%) participants reported an actual suicide attempt, 394 (28.63%) reported an interrupted attempt, 427 (31.03%) reported an aborted suicide attempt, and 358 (26.02%) reported preparatory behavior. After the ED evaluation, the disposition for participants was as follows: returned home (n = 345, 25.07%), admitted to medical ward for observation (n = 90, 6.54%), admitted to intensive care unit (n = 12, 0.87%), admitted to psychiatric ward/facility (n = 726, 52.76%), admitted to substance abuse treatment (n = 13, 9.45%), transferred to other mental health evaluation (n = 175, 12.72%), left against medical advice (n = 1, 0.07%), other (n = 10, 0.73%), and not documented (n = 4, 0.22%).

Measures

Columbia Suicide Severity Rating Scale (C-SSRS; Posner et al., 2011). The C-SSRS is a 20-item measure of suicidal thoughts and behaviors across three subscales which were rated for the past week and lifetime at baseline. The five Suicidal Ideation subscale items are binary. The 5 Intensity of Ideation subscale items are rated on either a 1–5 or a 0–5 point Likert scale. The 9 Suicidal Behavior subscale items include 5 binary items, 3 count variables, and 1 continuous lethality of actual SA variable (which was coded as 0 for individuals who denied any SA). A few extreme outliers were reported in frequency of actual (0 – 120), interrupted (0 – 3,000), and aborted SA (0 – 7,500). Counts of behavior ≤ 50 accurately reflected at least 98% of participants; therefore counts > 50 were Winsorized to 50. For follow-up assessments, any actual SA since the last assessment (coded either through the C-SSRS or chart review) was included.

Procedure

All procedures were approved by institutional review boards at each site. The ED-SAFE study was conducted across three-phases, discussed elsewhere (Boudreaux et al., 2016; Miller et al., 2017), and participants across all study phases completed the C-SSRS at parallel time-points (NCT01150994). After explanation of study procedures and completion of informed consent, highly trained and supervised research assistants administered the C-SSRS during the index ED visit. Trained and blinded technicians completed the C-SSRS over the phone at 6, 12, 24, 36, and 52 weeks after baseline. A chart review was conducted by trained medical record abstractors at 6- and 12-months after study enrollment to detect subsequent suicide attempts using a standardized form. Of the enrolled participants, 79.1% completed at least 1 interview over the phone during the follow-up year, and chart-reviews were conducted for all participants.

Data analytic plan

Analyses focused on past-week ratings for all C-SSRS items except lethality which referenced the most serious actual SA. Reliability was calculated using the tetrachoric and polychoric correlation matrices for the Suicidal Ideation and Intensity of Ideation subscales, and heterogeneous correlation matrix for Suicidal Behavior subscale and for all items.

Confirmatory Factor Analysis (CFA).

A CFA was conducted in MPlus to evaluate the 2-factor structure of the C-SSRS proposed by Madan and colleagues, wherein items from the past week Suicidal Ideation subscale and Suicidal Behavior subscale were entered on Factor 1, and Intensity of Suicidal Ideation items were entered on Factor 2 using weighted least squares estimator (WLSMV; Nussbeck, Eid, & Lischetzke, 2010). Model fit for the CFA was determined using the χ2 test, the Comparative Fit and Tucker-Lewis indices (CFI and TLI; recommended cutoff > .95), root-mean squared error of approximation (RMSEA; recommended value < .06), standardized root mean square residual (SRMR; recommended value < .08), and Akaike’s Information Criterion (AIC; Hu, & Bentler, 1999). Then, a CFA was conducted using only the theoretically-derived original C-SSRS subscales (the 5 binary Suicidal Ideation items, the 5 categorical Intensity of Ideation items, the 5 binary Suicidal Behavior items plus the categorical lethality of attempts items).

Exploratory Structural Equation Modeling (ESEM).

As the CFA did not result in ideal model fit, an ESEM model was run to determine empirically derived C-SSRS factors. The advantage of ESEM is that restrictions about factor loadings are reduced compared to CFA and exploratory factor analysis (Asparouhov & Muthén, 2009). This model was run using the past week Suicidal Ideation Items, Intensity of Ideation Items, and binary Suicidal Behavior Items (five items, including NSSI and preparatory behavior). Next, an ESEM was conducted in MPlus with the WLSMV estimator. Factors were described according to item loadings; a cutoff ≥ .40 was selected for inclusion on a component (Stevens, 1992).

Then, we calculated receiver operator characteristic curve (ROC) for the sensitivity and specificity of the original subscales (Suicidal Ideation, Intensity of Ideation and Suicidal Behavior) as well as the empirically derived subscales from the ESEM against the binary indicator of any future suicidal behavior using the sum of raw scores for each item. In this analysis, an Area Under the Curve (AUC) > .5 indicates an uninformative classifier, whereas 1 indicates perfect categorization of participants into groups based on either: 1) no suicide attempt in the follow-up period; or 2) suicide attempt in the follow-up period.

Due to the mixed nature of variable types, for prediction analyses, empirically derived and original subscale scores were generated by summing standardized scores. Empirical and original subscales were entered into logistic regressions to predict actual SA at all observations in independent models (e.g., the influence of Factor 1 on SA) and combined models (e.g., the influence of Factor 1 on SA above and beyond the influence of Factors 2–3), both of which were reported. At 6, 12, 24, 36 and 52 week follow-ups the frequency of cumulative actual SAs was 108 (7.84%), 157 (11.41%), 210 (15.26%), 244 (17.33%), and 283 (20.57%), respectively. To determine whether empirical factors and original subscales were predictive of participants who transitioned from no history of SA to making an actual SA, these analyses were repeated in restricted groups of participants, based on: 1) denying a history of an actual SA at the index ED visit; and 2) reporting a history of an actual SA at the index ED visit.

Then, history of any actual SA was entered as independent predictor of subsequent SA. Finally, we constructed a score reflecting the C-SSRS Emergency Department Screen Version with Triage Points (The Columbia Lighthouse Project, 2017). The C-SSRS for ED screen includes all Suicidal Ideation subscale items as well as any history of suicidal behavior (i.e., “Have you ever done anything, started to do anything, or prepared to do anything to end your life?”). The latter item was calculated as a total score from the lifetime suicidal behavior items.

Results

Reliability.

The sample was recruited on the basis of suicide risk at the index ED visit. Therefore, the eligibility items (Wish to be dead & Non-specific active suicidal thoughts) had restricted variability and prevented estimation of a full correlation matrix. These items were deleted in fitting the best model.

Reliability was acceptable for the Suicidal Ideation subscale (α = .83) and for all items (α = .76), but poor for the Intensity of Ideation (α = .42) and the Suicidal Behavior subscales (α = .42). When the NSSI item was removed from the Suicidal Behavior subscale, reliability remained poor (α = .43).

Confirmatory Factor Analysis (CFA)

Fit values for the CFA were below standard recommendations for the 2-factor model proposed by Madan and colleagues (2016; CFI: 0.733, TLI: 0.689, RMSEA: 0.085, CI90%: 0.080, 0.089, χ2: 1116.993, df = 103, WRMR = 2.717). Therefore, the two-factor solution was rejected. Similarly, fit values for the CFA were below standard recommendations for the 3-factor model based on the original C-SSRS subscales (CFI: 0.912, TLI: 0.892, and χ2: 304.160, df = 74, WRMR: 1.517), except for the RMSEA (.048, CI90%: 0.042, 0.053). Therefore, the factor structure of the original C-SSRS subscales was also rejected.

Exploratory Structural Equation Model (ESEM)

Factor Structure.

The three-factor structure had strong model fit and preserved parsimony (see Table 1). Model fit was not meaningfully improved in the four-factor model, which added unnecessary complexity and resulted in all factors with only two items per factor. Furthermore, the item loadings for the three-factor model were conceptually logical. Therefore, the three-factor model was retained.

Table 1.

Model Fit from Exploratory Structural Equation Model

1 Factor 2 Factor 3 Factor 4 Factor
χ2 (df) 377.117 (77) 208.055 (64) 119.859 (52) 58.304 (41)
Δ χ2 -- 142.954 76.412 56.97
Δ df -- 13 12 11
p -- <.001 <.001 <.001
CFI 0.885 0.945 0.974 0.993
Δ CFI −0.060 −0.029 −0.019
TLI 0.864 0.922 0.955 0.985
Δ TLI −0.058 -0.033 −0.030
RMSEA 0.053 (90% CI: .048 – .059) 0.040 (90% CI: .034 – .047) 0.031 (90% CI: .024 – .038) 0.018 (90% CI: .004 – .027)
Δ RMSEA 0.013 0.009 0.013
WRMR 1.694 1.197 0.864 0.586
Δ WRMR 0.497 0.333 0.278

Note. χ2 difference tests were conducted using the DIFF TEST option in MPlus. Model fit for the 3-factor model was superior to the 2-factor model, and the 3-factor model was more parsimonious than the 4-factor model. Therefore, the 3-factor model was retained

Factor 1, the Suicidal Ideation and Attempt Factor, included all 3 Suicidal Ideation items, actual suicide attempt, and preparatory behavior (see Table 2). The deterrents to suicidal ideation and lethality of suicide attempt also loaded most highly on this factor, but the loadings (.398 and .227) were less than the recommended cutoff and they were therefore not included. Factor 2, or the Intensity of Suicidal Ideation Component, included frequency and control of suicidal ideation. The duration of suicidal ideation and reasons for suicidal ideation also loaded most highly on this factor, but the loadings (.324 and .311) were less than the recommended cutoff and they were therefore not included. Factor 3, or the Stopped Suicide Attempts Component, included interrupted and aborted attempts. NSSI also loaded the highest on this factor (.212) but was below the recommended cutoff was therefore not included.

Table 2.

Factor Loadings for C-SSRS items

F1: Suicidal Ideation and Attempts F2: Intensity of Ideation F3: Stopped Suicide Attempts
Active Suicidal Ideation with Any Methods (Not Plan) without Intent to Act .650 .213 .022
Active Suicidal Ideation with Some Intent to Act, without Specific Plan .811 −.240 .138
Active Suicidal Ideation with Specific Plan and Intent .876 .135 −.104
Frequency .210 .427 −.003
Duration .311 .324 −.004
Controllability .002 .642 .181
Deterrents .398 .002 −.303
Reasons for Ideation .103 .311 −.101
Suicide Attempts .795 −.512 −.002
Interrupted .176 .002 .672
Aborted Attempts .000 .164 .429
Preparatory Behavior .535 −.002 .204
Attempt Lethality .227 .095 −.108
NSSI .086 .093 .212

Note: Bolded values indicate that a variable loaded on a given factor. Variables in italics indicate that they did not load on any factor.

Receiver Operating Characteristics (ROC)

All Area Under the Curve values failed to discriminate based on SA in the follow-up period (ROC area range = .528 – .589, see Figure 1).

Figure 1.

Figure 1.

Discrimination of the C-SSRS Factor- and Theoretically-Derived Scales against Suicide Attempts in the Follow-Up Period

Prediction of subsequent SA from ESEM Factors.

Across all participants, Factors 1 and 3 individually predicted subsequent SA (see Table 3). When all factors were included in a combined model, only Factor 1 remained significant.

Table 3.

Prediction of Subsequent Suicide Attempts based on History of Suicide Attempts and in the Total Sample

No History of Actual SA History of Actual SA Total Sample
Model with separate original C-SSRS subscales
Suicidal Ideation OR=1.04, SE: .07, z=.65, p=.516, CI95% .92, 1.19 OR=1.19, SE: .05, z=4.25, p<.001, CI95% 1.10, 1.29 OR=1.18, SE: .04, z=4.94, p<.001, CI95% 1.10, 1.26
Intensity of Ideation OR=1.11, SE: .07, z=1.67, p=.094, CI95% .98, 1.25 OR=1.01, SE: .03, z=.36, p=.719, CI95% .96, 1.07 OR=1.04, SE: .03, z=1.57, p=.117, CI95% .99, 1.09
Suicidal Behavior OR=1.01, SE: .07, z=.19, p=.846, CI95% .89, 1.15 OR=1.10, SE: .03, z=3.83, p<.001, CI95% 1.05, 1.16 OR=1.12, SE: .02, z=4.95, p<.001, CI95% 1.07, 1.17
Model with combined C-SSRS subscales
Suicidal Ideation OR=1.01, SE: .07,
z=.14, p=.887, CI95% .88, 1.16
OR=1.16, SE: .05, z=3.31, p=.001, CI95% 1.06, 1.27 OR=1.13, SE: .04, z=3.25, p=.001, CI95% 1.05, 1.22
Intensity of Ideation OR=1.11, SE: .07, z=1.63, p=.104 CI95% .98, 1.26 OR=.97, SE: .03, z= −1.07, p=.285, CI95% .91, 1.03 OR=.99, SE: .03, z=−.29, p=.772, CI95% .94, 1.05
Suicidal Behavior OR=.98, SE: .07, z=−.27, p=.790, CI95% .86, 1.13 OR=1.07, SE: .03, z=2.27, p=.023, CI95% 1.01, 1.13 OR=1.08, SE: .03, z=2.98, p=.003, CI95% 1.03, 1.13
Model with Separate Factors
Factor 1: Suicidal Ideation and Attempt OR=1.00, SE: 0.06, z=0.18, p=.860, CI95% 0.91, 1.13 OR=1.09, SE: 0.03, z=3.47, p<.01, CI95% 1.04, 1.15 OR=1.10, SE: 0.02, z=4.51, p<.001, CI95% 1.06, 1.15
Factor 2:
Intensity of Ideation
OR=1.10, SE: 0.12, z=0.91, p=.365, CI95% 0.89, 1.35 OR=1.05, SE: 0.05, z=1.09, p=.277, CI95% 0.96, 1.16 OR=1.08, SE: 0.05, z=1.76, p=.078, CI95% 0.99, 1.18
Factor 3:
Stopped Suicide Attempts
OR=1.08, SE: 0.10, z=0.78, p=.434, CI95% .90, 1.29 OR=1.08, SE: 0.05, z=1.61, p=.108, CI95% 0.98, 1.19 OR=1.09, SE: 0.05, z=2.11, p<.05, CI95% 1.01, 1.19
Model with Combined Factors
Factor 1: Suicidal Ideation and Attempt OR=0.98, SE: 0.06, z= −0.31, p=.754, CI95% 0.87, 1.10 OR=1.10, SE: 0.03, z=3.27, p<.01, CI95% 1.03, 1.15 OR=1.10, SE: 0.02, z= 4.07, p<.01, CI95% 1.05, 1.15
Factor 2:
Intensity of Ideation
OR=1.09, SE: 0.12, z=0.82, p=.415, CI95% 0.88, 1.36 OR=1.01, SE: 0.05, z=0.10, p=.916, CI95% 0.91, 1.11 OR=1.02, SE: 0.05, z=0.44, p=.660, CI95% 0.93, 1.11
Factor 3:
Stopped Suicide Attempts
OR=1.08, SE: 0.11, z=0.78, p=.438, CI95% 0.89, 1.32 OR=1.03, SE: 0.05, z=0.60, p=.546, CI95% 0.93, 1.14 OR=1.03, SE: 0.05, z=0.17, p=.480, CI95% .94, 1.13
Lifetime History of Suicide Attempts -- -- OR=1.98, SE: .33, z=4.09, p<.001, CI95% 1.43, 2.75
Baseline Past Week Suicide Attempt -- -- OR=1.55, SE: .21, z=3.19, p=.001, CI95% 1.18, 2.03
Brief C-SSRS (Ideation items 3–5 + history of suicide attempts) OR=1.06, SE: .06, z=.89, p=.372, CI95% .94, 1.19 OR=1.19, SE: .05, z=4.25, p<.001, CI95% 1.10, 1.29 OR=1.18, SE: .04, z=5.10, p<.001, CI95% 1.11, 1.26

Note: The ideation subscale does not include the original first 2 ideation items because of low variance in these items due to the study inclusion criteria. Bolded cells reflect statistical significance

In participants who denied a history of actual SA at baseline (n = 389), 52 (13%) made an actual SA over the follow-up period; no factor was independently associated with subsequent SA in this group in either separate or combined models.

In participants who reported a history of SA at baseline (n = 987), 231 (23%) made an actual SA over the follow-up period. Only Factor 1 was associated with SA in independent and combined models.

Prediction of subsequent SA from original CSSRS scales.

Across all participants, the Suicidal Ideation and Suicidal Behavior subscales were significant predictors of SA, whereas the Intensity of Ideation subscale was not.

In participants who denied a history of SA at baseline, none of the original C-SSRS subscales were significant predictors of SA.

In participants who reported a history of SA, the Suicidal Ideation and Suicidal Behavior subscales were significant predictors of SA, whereas the Intensity of Ideation subscale was not.

Prediction of subsequent SA from Actual SA

Across all participants, history of lifetime SA and past week SA were both predictive of subsequent SA.

Prediction of subsequent SA from the Brief C-SSRS Emergency Department Screen

Across the total sample, the C-SSRS for ED screen was predictive of subsequent SA. These results held in those with a history of SA, but not in those without a history of SA.

Discussion

History of a prior SA, as measured by the C-SSRS, provided the most parsimonious and powerful assessment for predicting future SA. This single item outperformed the standard and empirically derived C-SSRS subscales. Whereas some empirically derived and original C-SSRS subscales predicted future SAs in the total sample and in those with a history of SAs, none of the scales were predictive in patients without a history of SAs. The C-SSRS ED Screen also predicted subsequent SA, but not in those without a history of SA. Odds ratios were highest for the single history of SA item, consistent with a robust literature base (Klonsky, May, & Saffer, 2016; Ribeiro et al., 2016).

The psychometric performance of the C-SSRS in EDs was somewhat mixed and largely unsatisfactory. Reliability ranged from acceptable (Suicide Ideation subscale and for all items together) to poor (Intensity of Ideation and Suicidal Behavior subscales). These findings were partially consistent with prior research (Madan et al., 2016; Gutierrez et al., 2019) wherein questionable or unacceptable reliability for the Intensity of Ideation subscale, though strong reliability for both the Suicidal Ideation and Suicidal Behavior subscale, was reported (Madan et al., 2016). Attempts to replicate the component structure from prior research (Madan et al., 2016) and from the theoretically-derived scales were unsuccessful, perhaps due to sample differences. Whereas Madan and colleagues (2016) studied a diverse sample of psychiatric inpatients with varying degrees of suicide risk, the current study focused on ED patients who screened positive for SI or SA.

Three factors best characterized the data, though these factors diverged from the original C-SSRS subscales. Factor 1 included all of the valid Suicidal Ideation items, actual SA, and preparatory behavior. Factor 2 included frequency and controllability of SI. Thus, in prior research (Madan et al., 2016) and the current study, the Intensity of Ideation items loaded separately from other items. Also consistent with prior research in which the factor structure on the Intensity of Ideation subscale was poor (Gutierrez et al., 2019), three of the Intensity of Ideation items did not load on any factor. Factor 3 included interrupted and aborted SA. Thus, aborted attempts and interrupted attempts performed differently than actual attempts. This differs somewhat from a prior study in which few meaningful differences were reported in individuals on the basis of whether they had aborted, interrupted, or actual SAs (Burke, Hamilton, Ammerman, Stange, & Alloy, 2016). However, the prior study was cross-sectional and in undergraduates, perhaps accounting for the different findings. Finally, NSSI, duration of SI, reasons for SI and lethality did not load on any of component.

Receiver operating characteristics (ROC) were universally poor across the theoretically and empirically derived subscales. Whereas a score of 1 on this metric would indicate optimal ability to discriminate between individuals who made a suicide attempt versus those who did not in the follow-up period, all values were between .53 and .59. These ROC analyses indicated that the original and empirically-derived subscales were uninformative classifiers for future suicide attempts. This is further evidence of the poor performance of the subscales.

When the sample was reduced to only include those with a history of actual SA, the findings paralleled the full sample with only one difference. Specifically, in the analyses of separate components, only Factor 1 significantly predicted subsequent SA. Of note, Factor 1 included the actual suicide attempt item for the past week. This finding highlights another difference between aborted/interrupted attempts and actual attempts in predicting future attempts. As with the total sample, the C-SSRS for ED screen was predictive of subsequent SA in this restricted sample of patients with a history of SA.

A different pattern emerged in the sample that denied a history of actual SA at baseline. No empirically derived factor or original C-SSRS subscale was significantly associated with subsequent SA in combined or separate models. Similarly, the C-SSRS for ED screen was not predictive of subsequent SA in this restricted sample. This was concerning, as a goal of suicide prevention work is to interrupt the transition from thinking about suicide to engaging in suicidal behavior; in the current study, none of the C-SSRS variables were capable of predicting who would make that transition for the first time. Given that this group denied history of SA, the variable with the strongest predictive power for suicidal behavior, perhaps it is not surprising that none of the empirically derived or original C-SSRS subscales predicted SA in this group.

The limited utility of the Intensity of Ideation items is consistent with prior research (Madan et al., 2016). While these Intensity of Ideation items are intuitively meaningful and useful for research and characterization purposes, they performed poorly across most domains. Clinically, it may be more advantageous to spend valuable ED time on developing a safety plan (Chesin et al., 2017) or providing means safety counselling (Jin, Khazem, & Anestis, 2016) than in assessing Intensity of Ideation.

Several limitations of the current study require consideration. Participants were recruited on the basis of being at high-risk for suicide, restricting the floor of SI severity. Consequently, two of the C-SSRS items were removed from subsequent analyses due to this floor effect. While this can be perceived as a limitation, the focus of this research is on understanding the psychometrics of the C-SSRS in a high-risk sample. The sample in this study is consistent with prior studies, in which 20–30% of individuals who have made a SA engage in a subsequent attempt in the following year (e.g., Vaiva et al., 2006). In light of the high severity of the sample and the many items that loaded on Factor 1, one possibility is that Factor 1 reflects recent crises that may precipitate suicide risk. Future research should explore correlations of this factor with other indicators, such as recent relationship conflict. In addition, this sample was mostly white and well-educated. Therefore, it is unclear if the results will generalize to more diverse samples. Further, it is likely that some less serious SA may have been missed despite attempts at interviews and comprehensive medical chart reviews. Additionally, suicidal behaviors are likely influenced by a variety of interpersonal, intrapersonal, and contextual factors, though a recent meta-analysis revealed the difficulty in predicting suicidal ideation and attempts in general (Franklin et al., 2017). Comprehensive assessments of other risk factors beyond SI and SA may improve predictive power (Arias et al., 2016). Furthermore, the C-SSRS was originally developed as a tool for classifying suicidal thoughts and behaviors, which is a distinct goal from predicting future suicidal behavior. This may account for the poor predictive performance in the current study and in some prior studies. Finally, all participants received some level of intervention, ranging from discharge resources to extensive inpatient hospitalizations through either usual care or study-specific resources. The resources were intended to mitigate suicide risk, introducing an inevitable confound between the baseline assessment and follow-up assessment. Nevertheless, baseline scores remained significant predictors of SA.

In summary, the psychometric and predictive performance of the C-SSRS was limited in this sample of ED patients who screened positive for either SI or SA. On the one hand, reliability was acceptable for the Suicidal Ideation subscale and for all items together, and two empirically derived factors of the C-SSRS were meaningful predictors of subsequent SA in the total sample and in patients with a history of SA. The C-SSRS ED screen and the original Suicidal Ideation and Suicidal Behavior scales were also predictive of subsequent SA in the total sample and in those who reported a history of SA. On the other hand, this study revealed several critical weaknesses of the C-SSRS for this sample. Some subscales had poor reliability, and the Intensity of Ideation subscale was a poor prognostic indicator of subsequent SA. All of the subscales failed in receiver operator characteristics for discriminating those who had a suicide attempt in the follow-up period versus those who did not. Finally, this study provides further evidence that the most powerful predictor of SA was having a history of prior SA. Therefore, the utility of the C-SSRS in an ED setting is inadequate in terms of predictive power, though may be useful for classification of suicidal thoughts and behavior.

Disclosures and acknowledgements.

The authors have no financial conflicts to disclose. We would like to acknowledge the following contributors to the research: ED-SAFE Investigators: Michael H. Allen, M.D. (University of Colorado School of Medicine); Edward Boyer, M.D., Ph.D. (University of Massachusetts); Jeffrey Caterino, M.D., M.P.H. (Ohio State University Medical Center); Robin Clark, Ph.D. (University of Massachusetts); Mardia Coleman (University of Massachusetts Medical School); Barry Feldman, Ph.D. (University of Massachusetts Medical School); Amy Goldstein, Ph.D. (National Institute of Mental Health); Talmage Holmes, Ph.D. (University of Arkansas for Medical Sciences Medical Center); Maura Kennedy, M.D. (Beth Israel Deaconess Medical Center); Frank LoVecchio, D.O. (Maricopa Medical Center); Sarah Arias, Ph.D. (Massachusetts General Hospital); Lisa Uebelacker, Ph.D. (Memorial Hospital of Rhode Island); Wesley Zeger, D.O. (University of Nebraska Medical Center)

Contract grant sponsor: National Institute of Mental Health; Contract grant number: U01MH088278, R01 MH095786 & R01 MH 099741

Role of the sponsors: The supporters had no role in the design, analysis, interpretation, or publication of this study.

Footnotes

Clinical Trials Identifier: NCT01150994

References

  1. Arias SA, Miller I, Camargo CA Jr, Sullivan AF, Goldstein AB, Allen MH, … & Boudreaux ED (2015). Factors associated with suicide outcomes 12 months after screening positive for suicide risk in the emergency department. Psychiatric Services, 67, 206–213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Asparouhov T, & Muthén B (2009). Exploratory structural equation modeling. Structural equation modeling: a multidisciplinary journal, 16, 397–438. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Boudreaux ED, Millver I, Goldstein AB, Sullivan AF, Allen MH, Manton AP, … & Camargo CA Jr (2013). The emergency department safety assessment and follow-up evaluation (ED-SAFE): method and design considerations. Contemporary clinical trials, 36, 14–24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Boudreaux ED, Camargo CA Jr., Arias SA, Sullivan AF, Allen MH, Goldstein AB, … & Miller IW (2016). Improving suicide risk screening and detection in the emergency department. American journal of preventive medicine, 50, 445–453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Boyle A, Beniuk K, Higginson I, & Atkinson P (2012). Emergency department crowding: time for interventions and policy evaluations. Emergency medicine international, 2012, 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brown GK, Currier GW, Jager-Hyman S, & Stanley B (2015). Detection and classification of suicidal behavior and nonsuicidal self-injury behavior in emergency departments. The Journal of clinical psychiatry, 76¸ 1397–1403. [DOI] [PubMed] [Google Scholar]
  7. Burke TA, Hamilton JL, Ammerman BA, Stange JP, & Alloy LB (2016). Suicide risk characteristics among aborted, interrupted, and actual suicide attempters. Psychiatry Research, 242, 357–364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chesin MS, Stanley B, Haigh EA, Chaudhury SR, Pontoski K, Knox KL, & Brown GK (2017). Staff views of an emergency department intervention using safety planning and structured follow-up with suicidal veterans. Archives of Suicide Research, 21, 127–137. [DOI] [PubMed] [Google Scholar]
  9. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, … Nock MK (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2), 187–232. 10.1037/bul0000084 [DOI] [PubMed] [Google Scholar]
  10. Gipson PY, Agarwala P, Opperman KJ, Horwitz A, & King CA (2015). Columbia-suicide severity rating scale: predictive validity with adolescent psychiatric emergency patients. Pediatric Emergency Care, 31, 88. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Gutierrez PM, Joiner T, Hanson J, Stanley IH, Silva C, & Rogers ML (2019). Psychometric properties of four commonly used suicide risk assessment measures: applicability to military treatment settings. Military Behavioral Health, 1–8. [Google Scholar]
  12. Horwitz AG, Czyz EK, & King CA (2015). Predicting future suicide attempts among adolescent and emerging adult psychiatric emergency patients. Journal of Clinical Child & Adolescent Psychology, 44, 751–761. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Hu LT, & Bentler PM (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: A multidisciplinary journal, 6, 1–55. [Google Scholar]
  14. Jin HM, Khazem LR, & Anestis MD (2016). Recent advances in means safety as a suicide prevention strategy. Current psychiatry reports, 18, 96. [DOI] [PubMed] [Google Scholar]
  15. Klonsky ED, May AM, & Saffer BY (2016). Suicide, suicide attempts, and suicidal ideation. Annual review of clinical psychology, 12, 307–330. [DOI] [PubMed] [Google Scholar]
  16. Madan A, Freuh BC, Allen JG, Ellis TE, Rufino KA, Oldham JM, & Fowler JC (2016). Psychometric reevaluation of the Columbia-Suicide Severity Rating Scale: findings from a prospective, inpatient cohort of severely mentally ill adults. The Journal of Clinical Psychiatry, 77, 867–873. [DOI] [PubMed] [Google Scholar]
  17. Miller IW, Camargo CA, Arias SA, Sullivan AF, Allen MH, Goldstein AB, … & Boudreaux ED (2017). Suicide prevention in an emergency department population: the ED-SAFE study. JAMA psychiatry, 74, 563–570. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Nussbeck FW, Eid M & Lischetzke T (2010). Analysing multitrait–multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: What sample size is needed for valid results? British Journal of Mathematical and Statistical Psychology, 59, 195–213. [DOI] [PubMed] [Google Scholar]
  19. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, … & Mann JJ (2011). The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. American Journal of Psychiatry, 168, 1266–1277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Ribeiro JD, Franklin JC, Fox KR, Bentley KH, Kleiman EM, Chang BP, & Nock MK (2016). Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: a meta-analysis of longitudinal studies. Psychological Medicine, 46, 225–236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Stevens J (1992). Applied multivariate statistics for the social sciences. Lawrence Erlbaum Associates, Inc. [Google Scholar]
  22. The Columbia Lighthouse Project (2017). Screener with triage for emergency departments, Retrieved from http://cssrs.columbia.edu/the-columbia-scale-c-ssrs/cssrs-for-communities-and-healthcare/#filter=.general-use.english.
  23. Ting SA, Sullivan AF, Boudreaux ED, Miller I, & Camargo CA Jr. (2012). Trends in US emergency department visits for attempted suicide and self-inflicted injury, 1993. 2008. General hospital psychiatry, 34, 557–565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Vaiva G Ducrocq F, Meyer P Mathieu D, Philippe A, Libersa C, Goudemand M (2006). Effect of telephone contact on further suicide attempts in patients discharged from an emergency department: randomised controlled study. British Medical Journal, 332(7552), 1241–1245. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES