Initial Validation of Brief Measures of Suicide Risk Factors: Common Data Elements Used by the Military Suicide Research Consortium

Abstract

The Military Suicide Research Consortium (MSRC) developed a 57-item questionnaire assessing suicide risk factors, referred to as the Common Data Elements (CDEs), in order to facilitate data sharing and improve collaboration across independent studies. All studies funded by MSRC are required to include the CDEs in their assessment protocol. The CDEs include shortened measures of the following: current and past suicide risk, lethality and intent of past suicide attempts, hopelessness, thwarted belongingness, anxiety sensitivity, post-traumatic stress disorder symptoms, traumatic brain injury, insomnia, and alcohol abuse. This study aimed to evaluate the psychometric properties of the CDE items drawn from empirically validated measures. Exploratory factor analysis (EFA) was used to examine the overall structure of the CDE items, and confirmatory factor analyses (CFAs) were used to evaluate the distinct properties of each scale. Internal consistencies of the CDE scales and correlations with full measures were also examined. Merged data from 3,140 participants (81.0% military service members, 75.6% male) across 19 MSRC-funded studies were used in analyses. Results indicated that all measures exhibited adequate internal consistency, and all CDE shortened measures were significantly correlated with the corresponding full measures with moderate to strong effect sizes. Factor analyses indicated that the shortened CDE measures performed well in comparison with the full measures. Overall, our findings suggest that the CDEs are not only brief, but also provide psychometrically valid scores when assessing suicide risk and related factors that may be used in future research.

Suicide prevention is a worldwide public health priority that warrants considerable research and intervention (World Health Organization, 2015). According to the Centers for Disease Control and Prevention (CDC, 2013), suicide was ranked as the tenth leading cause of death across all age groups in the United States. Historically, the suicide rate among active duty military service members has been lower than that among civilians matched demographically for age and sex (Nock et al., 2013). However, the current national rate for suicide was 12.6 per 100,000 in 2013 (CDC, 2013). In comparison, the suicide rate for all active components of the military was 18.7 per 100,000 service members in 2013 and 26.2 per 100,000 for Reserve and National Guard members in 2013 (Smolenski, 2015). As the suicide rates among military populations have risen recently and appear to be higher than civilian rates, it is imperative to evaluate factors that may confer increased risk among this population.

Previous research has shown that there are unique stressors for active duty military service members, Reserve and National Guard members, and veterans as compared to civilian populations. Specifically, military culture includes distinct values, traditions, hierarchy, codes of conduct, and concerns about confidentiality that differ from civilian culture (Anestis & Green, 2015; Coll, Wiess, & Yarvis, 2011; Denneson et al. 2015; Pease, Billera, & Gerard, 2015). As such, it is vital to accurately assess not only suicidal ideation in this population but also suicidal behavior, defined as engagement in a behavior motivated by at least some intent to die. Importantly, assessing suicidality, or an individual’s risk for suicide which encompasses suicidal ideation, suicidal behavior, and other relevant risk factors, requires specific knowledge of how various risk factors may operate to increase this risk.

From a theoretical perspective, many risk factors may be contextualized within Joiner’s (2005) interpersonal theory of suicide (see also Van Orden et al., 2010; Joiner et al., 2016). This theory states that suicidal desire comprises both thwarted belongingness and perceived burdensomeness (i.e., a feeling of being a liability to others). Similarly, a third component known as capability for suicide refers to an individual’s ability to engage in lethal or near-lethal suicidal behavior and includes the individual’s familiarity/comfort with means as well as the ability to withstand the physiological and psychological discomfort associated with a suicide attempt. It is posited that an individual is at risk for a lethal or near-lethal suicide attempt if the individual has both high levels of suicidal desire and acquired capability (Joiner, 2005). Some of the above risk factors have established associations with aspects of the interpersonal theory of suicide, in which the risk factor has been shown to be associated with either thwarted belongingness, perceived burdensomeness, or capability. One example of this association is the relationship between insomnia, thwarted belongingness, and suicidal ideation. It has been demonstrated that thwarted belongingness accounts for the relationship between severity of insomnia symptoms and increased suicidal ideation (Chu et al., 2016).

A broad range of risk factors may be associated with significantly higher suicide risk. Studies have indicated that one of the most salient predictors for death by suicide across demographics was prior history of suicidal behavior (Cavanagh et al., 2003; Joiner et al., 2005; Ribeiro et al., 2016; Villatte et al., 2015). Other risk factors include symptoms of insomnia, which are associated with suicidal ideation and attempts (Pigeon, Pinquart, & Conner, 2012; Ribeiro et al., 2012) and higher levels of hopelessness (Holma et al., 2010; Beck, Steer, Kovacs, & Garrison, 1985). In addition to factors that confer increased risk for suicide in the general population, military populations may also exhibit unique vulnerabilities, which heighten suicide risk. Psychiatric diagnoses such as depression, anxiety, and substance use are significantly associated with higher risk of suicide among service members (LeardMann et al., 2013). Similarly, Posttraumatic Stress Disorder (PTSD) has been associated with death by suicide and with higher levels of suicidal thoughts and behaviors among veterans (Ramchand, Rudavsky, Grant, Tanielian, & Jaycox, 2015). Thwarted belongingness, a component of suicidal desire according to the interpersonal theory of suicide, involves a lack of interpersonal connections with others. This construct has been identified as a significant risk factor (Joiner, 2005; Van Orden et al., 2010) and found to predict suicidal thoughts and behaviors among service members (Anestis, Khazem, Mohn, & Green, 2015). Moreover, physical conditions, such as traumatic brain injury (TBI), are related to increased risk of death by suicide among veterans (Brenner, Ignacio, & Blow, 2011). These psychological and physical conditions are important suicide-related risk factors, as they may aid in better understanding an individual’s risk for suicide, and they may each be targeted for intervention.

One of the biggest challenges to understanding the complexities of suicide is data quality (WHO, 2015), as variation in data collection across research efforts, particularly the use of related but distinct measures, results in various obstacles. According to Villante and colleagues (2015), thousands of studies have attempted to characterize specific suicide risk and protective factors in an effort to improve prevention and intervention efforts. However, research supports the importance of implementing the same measures across studies to allow for collaborative comparisons, secondary data analyses, and meta-analyses (Maas et al., 2011; Maas et al., 2010). In response to these obstacles, the Military Suicide Research Consortium (MSRC) was established to enhance the military’s ability to quickly identify and to provide effective interventions to service members in need (MSRC, 2016). The MSRC’s goals include creating standards across studies; improving data quality; reducing the burden on investigators and participants through the use of empirically validated, brief measures; and ultimately, accelerating dissemination and implementation of research findings to impact change. The MSRC developed a standard set of questions to be used in all funded studies that assess a broad range of relevant factors, referred to as the Common Data Elements (CDEs). The CDEs are composed of shortened versions of validated measures assessing current and past suicidal ideation and planning, lethality and intent of past suicide attempts, hopelessness, thwarted belongingness, aspects of anxiety sensitivity, symptoms of post-traumatic stress disorder related to military experiences, traumatic brain injury, insomnia, and alcohol abuse.

As previously stated, the goals of the MSRC include improving data quality, creating standards across funded studies, and improving access to empirically validated and brief measures in hopes of facilitating the dissemination and implementation of findings. The objective of the current study was to empirically evaluate the psychometric properties and validity of the CDE items drawn from existing measures, in order to aid in the refinement of these shortened CDE scales and understanding of the performance of these scales in research and clinical settings. The items included in the CDEs target suicide risk factors; therefore, the current study sought to understand the interrelations between these risk factors and a range of suicide-related behaviors, by examining the utility of the CDEs drawn from existing measures in assessing these constructs. We also sought to compare and contrast the performance of the shortened CDE scales with the full measures in the context of closely related constructs.

Specifically, we conducted an exploratory factor analysis (EFA) to examine the structure of the items making up the CDEs drawn from existing scales and to evaluate how items from different scales assessing similar constructs are associated. Additionally, this EFA was conducted to establish divergent validity among constructs closely associated with suicide risk. Next, we conducted confirmatory factor analyses (CFAs) in which a specific shortened CDE measure of interest was referenced to two other constructs, namely suicidality and traumatic brain injury. These models using a particular abbreviated CDE measure of interest were compared to models including the full measure version when data were available for this comparison. This analytic strategy allowed us to compare how the abbreviated measures performed as compared to full measures within the same factorial space. Finally, we examined the psychometric properties of each shortened measure of the CDE and the magnitude of the correlation between each shortened measure and its corresponding full measure. Specifically, we predicted that the shortened CDE measures would be strongly correlated with the corresponding full measure versions and would exhibit acceptable reliability. We also predicted that the shortened CDE measures would exhibit comparable model fit similar to the full measure versions. In brief, the primary objective of the current study was to scrutinize the psychometrics and validity of the MSRC CDEs.

Method

Procedure and Participants

The MSRC CDEs were designed with the goal of broadly assessing suicide-related behavior and established suicide risk factors – constructs that have been consistently and empirically linked to suicide-related behaviors. The items making up the MSRC CDEs were selected and developed by the directors of the MSRC in consultation with MSRC’s senior advisors and experts from the field of suicide research. The CDEs are composed of 57 items assessing key suicide-relevant constructs and include items that were created specifically for the MSRC CDE (10 items) and those drawn from existing, validated measures (47 items). The current study examined the 47 items drawn from previously validated measures. The ten items excluded were not drawn from any existing scales, and many (e.g., a qualitative description of a previous suicide attempt) were not amenable to the current data-analytic approach. Permission to select individual items from existing measures was obtained from the copyright holders of all full measures, and permission to use existing aggregated data was obtained from contributing study PIs.

The MSRC CDE measure was included in all 26 studies funded by the MSRC to date; however, at the time of this paper’s writing, data were only available for 19 studies (see Table 1 in the supplemental materials for a summary of studies used in this manuscript). Administration of the CDE items varied by study site and included administration of the items over the phone, in-person, or online. Likewise, sample characteristics varied by site, and studies targeted the following populations for recruitment including veterans and activity duty service members, suicide survivors, veteran inpatients, and civilians. For studies with longitudinal designs, baseline CDE data were used. Abbreviated forms of full measures were developed by the MSRC directors, using two main heuristics – namely, factor loading in past research and content coverage. Items were selected that, in past research, displayed high factor loadings on the overall construct in question, with consideration to covering the full content of the overall construct. Some individual studies also included the full versions of the abbreviated CDE scales, which allowed for comparisons between both versions. Approval was acquired for all study protocols by each site’s institutional review board (IRB) and United States Army Medical Research and Materiel Command Human Research Protection Office. Protocols varied across research studies, but administration time for the CDE was approximately 20 minutes.

As seen in Table 1, this sample was primarily male, non-Hispanic, and White/Caucasian. It should be noted that the majority of participants were currently or had previously served in the military; however, some participants had no history of military service. In this sample, 53.9% (n = 1,698) reported currently serving in the military, 34.4% (n = 1084) reported previously serving in the military, and 11.7% (n = 368) reported never serving in the military. In this sample, 53.9% (n = 1,698) reported currently serving in the military, 34.4% (n = 1084) reported previously serving in the military, and 11.7% (n = 368) reported never serving in the military. Based on the data available regarding military branch of individuals currently serving in the military (n = 1696), the most commonly reported was the Army National Guard (n = 792, 43.9%). Based on available data among veterans (n =906), the most common reported was Active Duty Army (n = 411, 37.5%). Based on the data available regarding military branch of individuals currently serving in the military (n = 1129), the most commonly reported was the Army Reserves (n = 777, 91.7%). Based on available data among veterans (n =48), the most common reported was Active Duty Army (n = 21, 43.8%). Because there was no common demographics form across studies, the demographic data were aggregated, and new variables were created in order to accurately present demographic and military history variables. As a result, some demographic and military history variables have a large amount of missing data, but information on these variables is reported when known. Also of note, several studies included in this manuscript collected data from civilian participants who were recruited for various reasons depending on the objective of the individual study (e.g., family members of veterans were recruited for a primary study examining bereavement).”

Table 1.

Demographic Information

	N (%)
Sex (n = 3086)
Male	2330 (75.5%)
Female	753 (24.4%)
Transgender	3 (.1%)
Undeclared	30
Race/Ethnicity (n = 3035)
White/Caucasian	2009 (66.2%)
Black/African American	598 (19.7%)
Native American/Native Alaskan	30 (1.0%)
Asian	67 (2.2%)
Pacific Islander	7 (0.2%)
Multiracial	43 (1.4%)
Other	281 (9.3%)
Missing	81
Income (n = 1160)
$0 – $10,000	145 (12.5%)
$10,001 – $25,000	248 (21.4%)
$25,001 – $50,000	412 (35.5%)
$50,001 – $75,000	195 (16.8%)
$75,001 – $100,000	96 (8.3%)
Greater than $100,000	64 (5.5%)
Undeclared	1956
Relationship Status (n = 2871)
Married	882 (30.7%)
Single	1311 (45.7%)
Cohabitating	16 (0.6%)
Widowed	38 (1.3%)
Divorced/Separated	617 (21.5%)
Other	7 (0.2%)
Missing	245
Military Service (n = 3060)
Currently Serving	169833 (53.94%)
Reported history of deployment (n = 351)	160 (45.6%)
Veteran	108459 (34.46%)
Reported history of deployment (n = 137)	56 (40.9%)
No Military Service	368 (112.70%)
Missing	56

Note: Percentages were calculated based on data available.

Measures

Alcohol Use Disorders Identification Test (AUDIT; Bush, Kivlahan, McDonell, Fihn, & Bradley, 1998)

The AUDIT is a brief, 10-item screening measure for risky drinking and alcohol abuse and dependence (alcohol misuse). This measure was initially described as a screen for risky drinking or alcohol use disorders in male Veterans Affairs (VA) patients; however, it has since been validated in non-VA primary care settings and United States general population samples. Higher scores on the AUDIT reflect less safe drinking habits. Alcohol use disorders have been associated with increased risk for death by suicide across several studies (Schneider, 2009). The AUDIT has previously demonstrated overall good reliability and appears to be a valid measure of alcohol use problems (Meneses-Gaya, Zuardi, Loureiro, & Crippa, 2009). Three items measuring alcohol consumption were included in the CDE and exhibited good internal consistency in our sample (Cronbach’s α = 0.86).

Anxiety Sensitivity Index-3 (ASI-3; Taylor et al., 2007)

The ASI-3 is an 18-item measure of concerns regarding symptoms of anxiety (i.e., anxiety about having anxiety). Three domains of concerns are assessed, including physical (6 items), cognitive (6 items), and social concerns (6 items). Items are rated on a 5-point Likert scale ranging from 0 (very little) to 4 (very much), with subscale scores ranging from 0 to 24 and total scores ranging from 0 to 72. The ASI-3 has been shown to be adequately reliable across the three subscales, and has demonstrated adequate convergent, divergent, and criterion validity (Taylor et al., 2007). Five items from the cognitive concerns subscale were included in the CDEs, as previous research has indicated that this subscale is significantly associated with elevated suicide risk (Oglesby, Capron, Raines, & Schmidt, 2015). Internal consistency in our sample was excellent for this subscale (Cronbach’s α = 0.90).

Beck Scale for Suicidal Ideation (BSS; Beck & Steer, 1991)

The BSS is a 21-item scale that assesses severity of suicidal ideation within the previous week. Total scores, which comprise the first 19 items (scored 0 to 2), range from 0 (i.e., no suicidal ideation) to 38. The last two items refer to past attempts and do not contribute to the overall score. High levels of internal consistency (Cronbach’s α = .93) and concurrent validity have been reported with the Beck Depression Inventory (Beck, Steer, & Ranieri, 1993). Two items, assessing wish to live and wish to die, were included in the CDEs; internal consistency was good in our sample (Cronbach’s α = 0.80).

Beck Hopelessness Scale (BHS; Beck and Steer, 1988)

The BHS is a 20-item, true-false, self-report scale that measures the level of negative expectations about the future held by respondents over the previous week. Scores range from 0 to 20 representing nil (0–3), mild (4–8), moderate (9–14), and severe (>14) levels of hopelessness. The scale has strong concurrent validity with clinical ratings of hopelessness (r = 0.74) and other measures of hopelessness (r = 0.60, Beck & Steer, 1988). Hopelessness confers increased risk for death by suicide (Beck, Brown, Berchick, Stewart, & Steer, 1990). The internal consistency for the three hopelessness items included in the CDEs in this sample fell into a range which is generally considered workable but non-optimal (Cronbach’s α = 0.69).

Depressive Symptom Index – Suicidality Subscale (DSI-SS; Metalsky & Joiner, 1997)

The DSI-SS is a four-item self-report measure designed to assess the frequency and intensity of suicidal ideation and impulses in the past two weeks. Items are rated on a 4-point Likert scale ranging from 0 to 3 with total scores ranging from 0 to 12, and higher scores reflecting greater severity of suicidal ideation. The DSI-SS has demonstrated strong psychometric properties (Joiner, Pfaff, & Acres, 2002; Ribeiro, Braithwaite, Pfaff, & Joiner, 2012). All items were included in the CDEs, and internal consistency was excellent in this sample (Cronbach’s α = 0.91). As the entire full measure was included in the CDE, this scale will be referred to as the DSI-SS throughout this manuscript.

Insomnia Severity Index (ISI; Morin, 1993)

The ISI is a 7-item measure that examines an individual’s sleep difficulties and its effects on functioning. Symptoms of insomnia assessed include difficulty falling and staying asleep as well as waking up too early. In addition to assessing symptoms of insomnia, the scale also assesses satisfaction with sleep quality, distress over symptoms, and interference with daily functioning. Previous studies have indicated that the ISI exhibits good psychometric properties, including adequate internal consistency and concurrent validity. Furthermore, it appears to be sensitive to change over time and correlates with measures of insomnia (Bastíen, Vallieres, & Morin, 2001). Five items from the original scale were utilized in the CDE, and the internal consistency for the CDE version utilized in this sample was good (Cronbach’s α = 0.87).

Suicidal Behaviors Questionnaire – Revised (SBQ-R; Osman, Bagge, Gutierrez, Konick, Kopper, & Barrios, 2001)

The SBQ-R is a self-report measure of four constructs of suicidal behavior – lifetime ideation and attempt, recent frequency of ideation, suicide threats, and self-assessed likelihood of future suicidal behavior. The four items are rated on Likert scales of varying lengths, resulting in total scores between 3 and 18. It has been found to be valid and reliable across a range of clinical and non-clinical adolescent and adult samples. A cut-off score of 8 accurately discriminates between adult psychiatric inpatients with recent ideation or attempts and psychiatric controls (Osman et al., 2001). All four items were included in the CDEs; internal consistency was good in this sample (Cronbach’s α = 0.84). As the full scale was included in the CDE, this scale will be referred to as the SBQ-R throughout this manuscript.

Suicide Intent Scale (SIS; Beck, Schuyler, & Herman, 1974)

The SIS is a 15-item interviewer-administered instrument that assesses behavior before and during the most recent suicide attempt. Items 1–8 cover objective circumstances surrounding the attempt (e.g., preparation and execution of the attempt). Items 9–15 capture perceptions of potential lethality, expectations of rescue, purpose of the attempt, impulsivity, and reaction to the attempt. Each item is rated on a scale from 0 to 2 with the total score ranging from 0 to 30. A self-report version of this interview has been created and appears to be highly correlated with the original interview (Strosahl, Chiles, & Linehan, 1992). The SIS has high internal reliability (Cronbach’s α = .95; Beck, Schuyler & Herman, 1974) and high inter-rater reliability, ranging from .81 (Mieczowski et al., 1993) to .95 (Beck, Schuyler & Herman, 1974). Support for the concurrent validity of this measure has also been established (Beck, Morris, & Beck, 1974). Four items were included in the CDEs and were administered in self-report format. The items included in the CDE were drawn from the subjective domains of suicidal intent and consisted of items assessing lethality of intent. Internal consistency was good in this sample (Cronbach’s α = 0.89).

Interpersonal Needs Questionnaire-Thwarted Belongingness Subscale (INQ-TB; Van Orden, Cukrowicz, Witte, & Joiner, 2012)

The INQ is a 15-item measure of two components of Joiner’s (2005) interpersonal theory of suicide: perceived burdensomeness and thwarted belongingness. The thwarted belongingness subscale assesses the individual’s perceived feelings of alienation from others. All statements are rated on a 7-point Likert scale from 1 (not at all true for me) to 7 (very true for me). Higher scores reflect a higher degree of thwarted belongingness (Van Orden, Cukrowicz, Witte, & Joiner, 2012). Van Orden et al. (2008) reported good estimates of internal consistency for the thwarted belongingness subscale (Cronbach’s α = .85). In addition, the INQ exhibits good concurrent, divergent, and criterion validity. Furthermore, thwarted belongingness is associated with increased odds of reporting current suicidal ideation and suicidal ideation at a one-month follow-up (Van Orden, Cukrowicz, Witte, & Joiner, 2012). Five items assessing thwarted belongingness were included in the CDEs¹; internal consistency of the thwarted belongingness items was excellent in this sample (Cronbach’s α = 0.90).

Post-Traumatic Stress Disorder Checklist – Military Version (PCL-M; Weathers, Litz, Herman, Huska & Keane, 1993)

The PCL-M is a 17-item self-report measure of PTSD symptoms. Each of the 17 items corresponds to one of the 17 DSM-IV PTSD symptom criteria. Respondents rate the degree to which they were “bothered” by the problem or symptom in the past month on a 5-point Likert scale ranging from 1 (not at all) to 5 (extremely). A total symptom severity score can be obtained by summing the scores from each of the 17 items and ranges from 17 to 85. Subscale scores representing each of the three PTSD symptom clusters (i.e., re-experiencing, avoidance, and hyperarousal) can also be derived. Synthesis of the psychometric properties of the different versions of the PCL across a number of studies shows adequate total score internal consistency (Cronbach’s α > 0.75; Wilkins, Lang, & Norman, 2011). With respect to validity, the civilian version of the PCL demonstrated good convergence validity with the Clinician Administered PTSD Scale (CAPS), the gold standard in PTSD assessment (r = 0.79), and the Mississippi PTSD Scale (r = 0.90) in a sample of male Veterans with PTSD (Keen et al., 2008). PTSD is significantly associated with an elevated rate of death by suicide (Gradus et al., 2010). Eight questions were included in the CDEs on PTSD: 4 items on intrusions, 2 items on avoidance, and 2 items on hyperarousal. In this sample, the PCL-M was utilized, which focuses on symptoms associated with military experiences. The internal consistency of the eight item PCL-M was excellent (Cronbach’s α = 0.94).

The Traumatic Brain Injury-4 (TBI-4; Brenner et al., 2013)

The TBI-4 is a four-item screening tool that is included in the Eastern Colorado Health Care System (ECHCS) Mental Health Clinic (MHC) intake form. The four questions assess possible occurrences that may have led to traumatic brain injury. Ongoing research is being conducted by the Rocky Mountain Mental Illness Research, Education, and Clinical Center (previously known as VISN 19 MIRECC) to establish the concurrent validity of the TBI-4. Research examining veterans has shown that individuals who have experienced traumatic brain injury are significantly more likely to die by suicide (Brenner, Ignacio, & Blow, 2011). All four items were included in the CDEs, and internal consistency in this sample was good (Cronbach’s α = 0.77).

Data Analytic Strategy

Due to missing data on all CDE items, 122 cases were excluded. Before analyses were conducted, all shortened scales were examined for skewness and kurtosis, but neither were detected². Pearson’s r and Cronbach’s α were computed for all shortened and full measures to examine convergent/discriminant validity and internal consistency, respectively. Correlations could not be examined between some full scale measures as the data were merged from multiple sites, and not all sites administered all the full versions of the measures used in the study.

An EFA with a geomin rotation was conducted to examine the factor structure of all the items comprising the CDEs. Items that were developed specifically for the CDEs and were not drawn from a previously validated measure were not included in the factor analysis. The number of factors to retain were determined by examining model fit statistics including the Chi-square test (χ²), Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), the Root Mean Square Error of Approximation (RMSEA), residual variances, and eigenvalues. Previous recommendations for χ² have indicated that if a model exhibits good fit, this test should exhibit a low value and should not be significant; however, this test of model fit has been shown to have limitations when used in large samples (Hooper, Coughlan, & Mullen, 2008). As this study includes large samples sizes, we examined other fit indices in order to evaluate model fit, and followed the recommendations for cut-offs of greater than or equal to .95 for the CFI, of .95 for TLI, and equal to or lower than .06 for the RMSEA (Hooper, Coughlan, & Mullen, 2008).

Finally, models were developed in which a specific shortened CDE measure of interest was referenced to two other constructs, namely suicidality as assessed by the DSI-SS and traumatic brain injury as assessed by the TBI-4. These two scales were selected as referents because the full measure of each was included in the CDE, because they are well characterized constructs, and because they represented related, but distinct constructs from one another and from those included in the CDEs. These models including DSI-SS, TBI-4, and a particular abbreviated CDE measure of interest were compared to models including the DSI-SS, TBI-4, and the full version of the CDE measure of interest. This analytic strategy allowed us to compare how abbreviated measures performed, as compared to full measures, within the same factorial space. CDE subscales were compared using confirmatory factor analysis to the full scales if the full scales had enough completed data points (i.e., 100) to be included in a CFA. To the extent that the subscales acted as distinct constructs, they were expected to form distinct factors from the DSI-SS and the TBI-4 to a similar degree as their full-scale versions. After excluding the measures without enough data on the full scales, the scales selected for inclusion in these models were the AUDIT, BHS, BSS, PCL-M, and INQ-TB. Analyses were conducted in MPlus 6.11 (Muthen & Muthen, 2010). In addition to the fit statistics listed above, in the CFAs, we examined the Weighted Root Mean Square Residual (WRMR) and followed the recommendations of Yu (2002) for cut-offs of WRMR greater than or equal to 1.

Results

Correlations and Internal Consistency

Descriptive statistics, Cronbach’s α, and correlations of CDE measures and full measures are shown in Table 2. Pairwise deletion was utilized for these analyses. Examination of Pearson’s r correlations indicated that all shortened measures were significantly associated with the full measures.

Table 2.

Intercorrelations and Descriptive Statistics of All Shortened and Full Scales

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17
1. DSI - SS	-
2. SBQ – R	.72**	-
3. SIS	.47 **	.66 **	-
4. BSS	.67 **	.62 **	.46 **	-
5. BHS	.18 **	.14 **	.07*	.15**	-
6. INQ-TB	.50 **	.51 **	.41**	.48**	.11 **	-
7. ASI - 3	.35 **	.39 **	.14**	.31**	.23 **	.29**	-
8. PCL-M	.38 **	.41 **	.34**	.29**	.03	.43 **	.44**	-
9. TBI-4	.20 **	.26**	.30**	.16**	−.05**	.22 **	.13 **	.30**	-
10. AUDIT	.13**	.11**	.08**	.09**	.01	.11**	.06**	.10**	.10**	-
11. ISI	.42**	.46**	.42**	.40**	.02	.49**	.42**	.56**	.28**	.17**	_
12. Full BSS	.53 **	.42 **	.27**	.40**	.23 **	.20**	.17**	.18**	.11*	.06	.15**	-
13. Full BHS	.12	.22**	.08	.15*	.29**	.28**	.22*	.27**	.04	.20**	.31**	C	-
14. Full INQ-TB	.42**	.51**	.33**	.55**	−.17**	.64**	.43**	.34**	.12*	.10**	.46**	−.01	C	-
15. Full ASI-3	.29*	.39**	−.30	.32**	.37**	.28*	.99**	.46**	.09	.05	.30**	C	C	C	-
16. Full PCL-M	.68**	.75**	.61**	.76**	0.37**	.72**	.58**	.83**	.30**	.15**	.74**	.15*	C	.71**	C	-
17. Full AUDIT	.22**	.25**	.17**	.19**	−.01	.20**	.11**	.22**	.21**	.79**	.24**	−.03	C	.22**	.08	.25**	-
N	3032	2878	1601	2857	2856	3093	2169	2877	2869	2868	2871	374	267	1485	100	1049	1674
Mean	2.20	7.99	3.95	.89	1.90	16.13	12.62	17.74	1.40	3.00	9.24	19.47	7.46	22.49	10.77	21.58	5.69
SD	2.95	5.32	2.88	1.24	1.11	9.21	6.11	9.62	1.46	3.19	5.63	7.68	6.21	11.24	6.95	18.84	6.85
Minimum	0	0	0	0	0	2	1	1	0	0	0	0	0	3	0	1	0
Maximum	12	21	8	4	3	35	25	40	4	12	30	36	20	63	24	85	38
Skewness	1.12	.53	−.10	1.12	−.48	.30	.35	.70	.52	1.18	−.13	−.38	.54	1.03	.13	1.59	.189
Kurtosis	.15	−.88	−1.41	−.04	−1.19	−1.12	−1.02	−.74	−1.20	.66	−1.03	−.47	−1.03	1.03	−1.01	1.40	3.56

Note:

^*= p<.05

^**= p < .001

Bolded correlations indicate correlations between the parent and CDE measures; C = No subsample completed both measures; DSI-SS = Depressive Symptom Inventory – Suicidality Subscale; SBQ-R = Suicidal Behaviors Questionnaire – Revised; SIS = Suicide Intent Scale (CDE); BSS = Beck Scale for Suicide Ideation (CDE); BHS = Beck Hopelessness Scale (CDE); INQ-TB = Interpersonal Needs Questionnaire – Thwarted Belongingness (CDE); ASI-3 = Anxiety Sensitivity Index – Cognitive Concerns Subscale (CDE); PCL-M = PTSD Checklist-Military Version (CDE); TBI-4 = Traumatic Brain Injury – 4; AUDIT = Alcohol Use Disorder Identification Test (CDE); ISI = Insomnia Severity Index; Full BSS = Full Beck Scale for Suicide Ideation; Full BHS = Full Beck Hopelessness Scale; Full INQ-TB = Full Interpersonal Needs Questionnaire - Thwarted Belongingness; Full ASI-3 = Full Anxiety Sensitivity Index – Cognitive Concerns Subscale; Full PCL-M = Full PTSD Checklist-Military Version; Full AUDIT = Full Alcohol Use Disorder Identification Test; N = Number of Participants; SD = Standard Deviation

Correlations between the measures illustrated convergent validity across several measures. CDE measures directly assessing suicide-related variables, including the DSI-SS, the SBQ-R, the SIS, and the BSS, were moderately to highly correlated with each other. CDE measures assessing risk factors associated with suicide, such as anxiety sensitivity or hopelessness, were moderately correlated with measures directly assessing suicide risk but less so when compared to the magnitude of correlations between other direct measures of suicide risk as expected. The magnitude of correlations between direct measures of suicide risk were larger than the magnitude of correlations between risk factors associated with suicide and direct measures of suicide risk, with the exception of thwarted belongingness which exhibited slightly stronger correlations to the DSI-SS (r = .50) than the BSS (r = .47). Furthermore, all of the measures were significantly correlated with the parent measure, including the CDE versions of the BSS, PCL-M, AUDIT, ASI-3, BHS, and INQ-TB. These findings illustrate that the shortened measures are assessing similar constructs. Although the correlations provided largely positive results illustrating good convergent and discriminant validity across scales, there was one pattern that should be noted. The CDE BSS was more strongly correlated with the DSI-SS and the SBQ-R than was its full version measure. This likely reflects that the BSS items included in the CDE are assessing current wish to live and wish to die, similar in content to the items on the DSI-SS and the SBQ-R.

Exploratory Factor Analysis

An EFA was conducted to examine the factor structure of the 47 CDE items drawn from existing measures. For this model, we utilized a Weighted Least Squares Means and Variance Adjusted (WLSMV) estimator. The CDE BHS and TBI-4 scales were modeled as categorical due to the rating scales of these items. Examination of eigenvalues and the scree plot indicated that after nine factors eigenvalues dropped below 1.00. Model fit statistics for all models with eigenvalues greater than 1.00 are shown in Table 2 in the supplemental materials. Models with eight or fewer factors were shown to exhibit less than optimal fit (χ² ≥ 4204.969, df ≥ 733, p < .001, CFI ≤ .86, TLI ≤ .80, RMSEA ≥ .04). Examination of model fit statistics indicated that a nine-factor solution provided the best model fit (χ² = 2,936.78, df = 694, p < .001, CFI = .91, TLI = .86, RMSEA = .03). Factor loadings for the nine-factor solution are shown in Table 3.

Table 3.

Factor Loadings for Exploratory Factor Analysis of CDE measures

CDE Item	Scale	1	2	3	4	5	6	7	8	9
1	DSI – SS	.792	−.018	.035	−.002	.042	.020	−.014	.021	.002
2	DSI – SS	.910	−.001	−.025	−.051	.017	−.057	.010	.012	−.028
3	DSI – SS	.891	−.028	.016	−.014	.009	−.017	.012	.013	−.009
4	DSI – SS	.821	−.015	.047	−.008	.001	−.014	−.033	.009	.038
5	SBQ – R	.354	.436	.039	.093	.036	.034	.023	.006	−.103
6	SBQ – R	.535	.319	.020	.091	−.011	.036	.065	.000	−.065
7	SBQ – R	.427	.152	.056	.061	.055	−.010	−.032	−.044	−.019
8	SBQ – R	.569	.147	.017	.100	.045	.041	−.094	.014	.112
9	SIS	.003	.768	.034	−.012	.039	−.048	−.017	.016	−.047
10	SIS	−.014	.745	.014	−.042	.045	−.035	.012	.012	−.008
11	SIS	.030	.763	.020	−.008	.058	−.013	−.040	.008	.035
12	SIS	−.025	.585	.035	−.015	.060	.002	−.043	−.004	.004
19	BSS	.398	.520	−.053	.010	−.155	.078	.079	−.029	.089
20	BSS	.468	.541	−.082	.000	−.112	.048	.106	−.003	.068
21	BHS	.057	−.008	.024	−.042	.008	.032	−.009	−.034	.644
22	BHS	−.001	.064	.024	.004	.017	−.020	.000	.012	.854
23	BHS	.006	−.008	−.011	.079	−.042	.005	.031	.019	.845
24	INQ – TB	.008	.045	.021	−.045	−.021	.775	.007	−.022	−.001
25	INQ – TB	.132	.130	.050	.034	−.049	.638	.065	.008	.015
26	INQ – TB	.033	−.025	.013	−.010	.018	.845	.003	−.002	−.026
27	INQ – TB	.034	.037	.019	−.003	.003	.813	.057	.024	−.039
28	INQ – TB	−.095	−.059	−.032	.024	.054	.847	−.088	.000	.201
29	ASI – 3	.01	.034	.044	.696	−.064	−.011	.076	−.004	.015
30	ASI – 3	.006	−.031	.006	.731	.053	−.002	.026	−.027	.006
31	ASI – 3	.025	.013	.003	.826	−.003	−.007	−.038	.002	−.016
32	ASI – 3	.014	−.006	−.021	.851	.026	.003	.001	.014	.037
33	ASI – 3	−.018	−.008	.012	.829	−.011	.033	.000	.007	.013
34	PCL-M	.008	.025	.900	−.043	−.004	−.005	.000	−.012	.013
35	PCL-M	.002	−.032	.860	−.052	.029	−.007	.025	−.010	.051
36	PCL-M	.022	−.024	.835	.029	−.012	−.009	−.008	−.047	.007
37	PCL-M	.003	.015	.858	.055	−.021	−.001	.013	−.006	−.004
38	PCL-M	.005	.002	.814	−.027	−.008	.033	.001	.020	−.011
39	PCL-M	.017	.008	.846	.011	−.026	.032	−.018	−.005	−.022
40	PCL-M	−.018	−.002	.556	.071	.095	.028	.122	.043	−.014
41	PCL-M	.004	.051	.554	.125	.048	.015	.128	.047	−.002
42	TBI-4	.060	−.048	−.021	.001	.884	−.026	.011	−.042	.038
43	TBI-4	−.010	.088	.050	.002	.823	.003	−.009	.029	−.001
44	TBI-4	.041	−.007	−.037	−.002	.767	.051	.033	−.037	−.028
45	TBI-4	−.043	.101	.018	.012	.717	.029	.053	.054	−.007
46	AUDIT	.006	−.046	.005	−.024	.002	.006	−.007	.757	.015
47	AUDIT	.041	.034	−.005	−.001	−.007	.012	.033	.723	−.023
48	AUDIT	−.016	.003	−.005	.016	.001	−.009	−.013	.988	.011
52	ISI	.016	−.034	.036	.006	.010	−.004	.796	.016	.003
53	ISI	.017	.013	.081	.013	.028	−.005	.761	.003	.007
54	ISI	.036	.002	.090	.058	−.001	.001	.369	.049	−.013
55	ISI	−.017	.020	−.081	−.020	.043	.041	.871	.002	.011
56	ISI	.019	.041	.055	.090	−.010	.002	738	−.026	−.008

Note: Factor Loadings over .30 appear in bold

DSI-SS = Depressive Symptom Inventory - Suicidality Subscale; SBQ-R = Suicidal Behaviors Questionnaire – Revised; SIS = Suicide Intent Scale (CDE); BSS = Beck Scale for Suicide Ideation (CDE); BHS = Beck Hopelessness Scale (CDE); INQ-TB = Interpersonal Needs Questionnaire – Thwarted Belongingness (CDE); ASI-3 = Anxiety Sensitivity Index – Cognitive Concerns Subscale (CDE); PCL-M = PTSD Checklist-Military Version (CDE); TBI-4 = Traumatic Brain Injury – 4; AUDIT = Alcohol Use Disorder Identification Test; ISI = Insomnia Severity Index

The nine-factor solution was also interpretable. Items were considered to load onto a specific factor if the factor loading was greater than 0.3. The first factor was the only factor comprising multiple scales and included the DSI-SS and the last three items of the SBQ-R. Because these items appeared to encompass both characteristics of current suicidal ideation and behavior, this factor may be characterized as current suicidality. The second factor comprised items of the SIS, the BSS, and the first item of the SBQ-R, which assessed lifetime attempts and ideation. This factor was, therefore, characterized as suicide attempt intent and lethality and may represent more longitudinal risk. Factors three through nine were comprised of the following distinct subscales, which are presented in order: PCL-M items, ASI-3 items, TBI-4 items, INQ-TB items, ISI items, AUDIT items, and BHS items.

Regarding cross-loadings, the items that cross-loaded were limited to the first (current suicidality) and second factor (suicide attempt intent and lethality). The first item of the SBQ-R, which assessed lifetime suicidal ideation and attempts, loaded marginally stronger on the second factor, characterized as suicide attempt intent and lethality, than on the first (current suicidality). The second item of the SBQ-R, which assessed suicidal ideation within the past year, loaded more strongly on the first factor, current suicidality. The BSS items, which assessed wish to live and wish to die, loaded much stronger onto the second factor than the first; therefore, these items were considered as part of the factor representing suicide attempt intent and lethality.

Confirmatory Factor Analyses

CFAs were conducted to compare the performance between the five full-scale measures and their respective shortened CDE measures, specifically examining the constructs of alcohol use, hopelessness, wish to live/die (BSS), PTSD symptoms, and thwarted belongingness. In each of the CFAs, a three-factor model was specified: (1) either the selected CDE or full-length measure (e.g., CDE AUDIT vs. full length AUDIT), (2) current suicidality as assessed by the DSI-SS, and (3) traumatic brain injury positive screen as assessed by the TBI-4. In all, ten CFAs were conducted across the five measures and their CDE counterparts. Fit indices for these 10 CFAs can be found in Table 4, and an example path diagram may also be found in Figure 1.

Table 4.

Comparison of Fit Statistics Between CDE and Full Measures When Included in Confirmatory Factor Analysis Model with DSI-SS and TBI

Scale	N	χ2	df	p	CFI	TLI	RMSEA[90% CI]	WRMR
AUDIT	2909	161.668	41	<.001	.982	.976	.032[.027–.037]	1.184
Full AUDIT	2909	642.940	132	<.001	.938	.928	.036[.034–.039]	1.579
BHS	2909	95.716	41	<.001	.993	.991	.021[.016–.027]	.962
Full BHS	2909	746.292	347	<.001	.964	.961	.020[.018–.022]	1.359
BSS	3115	573.782	31	<.001	.906	.867	.074[.069–.079]	2.611
Full BSS	2909	730.921	321	<.001	.953	.948	.021[.019–.023]	1.325
PCL-M	2954	439.521	101	<.001	.954	.945	.034[.030–.037]	1.206
Full PCL-M	2909	923.783	272	<.001	.885	.873	.029[.027–.031]	1.979
INQ-TB	2954	304.417	62	<.001	.968	.959	.036[.032–.041]	1.333
Full INQ-TB	2953	915.424	116	<.001	.882	.861	.048[.045–.051]	1.915

AUDIT = Alcohol Use Disorder Identification Test (CDE); Full AUDIT = Full Alcohol Use Disorder Identification Test; BHS = Beck Hopelessness Scale (CDE); Full BHS = Full Beck Hopelessness Scale; BSS = Beck Scale for Suicide Ideation (CDE); Full BSS = Full Beck Scale for Suicide Ideation; PCL-M = PTSD Checklist-Military Version (CDE); Full PCL-M = Full PTSD Checklist-Military Version; INQ-TB = Interpersonal Needs Questionnaire – Thwarted Belongingness (CDE); Full INQ-TB = Full Interpersonal Needs Questionnaire – Thwarted Belongingness; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMSEA = Root Meat Square Error of Approximation; WRMR = Weighted Root Mean Square Residual

Figure 1.

Path diagram of confirmatory factor analysis of CDE AUDIT, DSI-SS, and TBI

Note: Unstandardized coefficients were reported; DSI-SS = Depressive Symptom Inventory-Suicidality Subscale; AUDIT = Alcohol Use Disorder Identification Test (CDE); TBI = Traumatic Brain Injury-4

Overall, the shortened CDE measures exhibited good model fit; regarding four of the five variables, fit was at least as good for the shortened measure as for the full measure model. Regarding the one exception, the CFA model examining the full measure of the BSS exhibited good model fit; however, the CFA model including the shortened CDE version of the BSS, which assessed wish to live/die, did not exhibit comparable fit to this model. Alternatively, two CDE scales exhibited better model fit than the full versions. The CFA model examining the shortened CDE version, which assessed PTSD symptoms associated with military experiences, exhibited adequate model fit; however, the CFA model examining the full measure of the PCL-M did not exhibit comparable fit. Similarly, the model assessing the shortened CDE version of thwarted belongingness exhibited good model fit, but the model assessing the full measure version of thwarted belongingness did not exhibit comparable fit. Conversely, the models examining alcohol misuse and hopelessness exhibited comparable fit across both models including the shortened CDE measure version and the full measure version.

Discussion

The purpose of the present study was to examine the psychometric properties and validity of the shortened CDE measures in order to determine the utility of the CDEs in assessing various factors associated with suicidality. Results indicated that the shortened CDE measures exhibited adequate-to-good internal consistency, and correlations between the shortened CDE and full measure versions were strong, which demonstrated that the shortened CDE measures are reliable and valid. Additionally, results of an EFA also showed that the shortened CDE measures comprised nine factors which were consistent with distinct constructs associated with suicide risk: current suicidality, suicide attempt lethality and intent, PTSD symptoms, anxiety sensitivity, hopelessness, traumatic brain injury, insomnia, thwarted belongingness, and alcohol misuse. Follow-up CFAs revealed that each of the shortened CDE scales demonstrated good fit in a three three-factor model in which the shortened measure was referenced to two other constructs. In four of five cases, the fit for the shortened version models was at least as good as that for the full measure models. The exception involved the BSS; even here, however, fit for the abbreviated CDE model was adequate.

Our findings supported the validity and reliability of all shortened CDE measures for use in future research. In comparison to the full measures, the majority of the shortened CDE scales performed either comparably or better than the full measure version. Despite these positive results, however, further research is needed to investigate two trends. Foremost, it should be noted that the CDE PCL-M does not assess numbing symptoms, which may limit its utility when assessing symptoms of PTSD. This shortened measure performed well in the current sample, but future research is needed in order to further clarify its sensitivity in detecting PTSD symptomatology in other clinical samples. Also of note, although the internal consistency of the shortened CDE version of the BHS was only adequate, analyses of model fit indicated that it performed in a manner comparable to the full measure version. Likewise, in the EFA, the CDE BHS loaded separately onto its own factor. Therefore, results suggested that this subscale is an accurate measure of hopelessness in this sample. Future research to evaluate both the CDE BHS and CDE PCL-M would be informative; however, the current study supported the validity and utility of all of the CDE shortened measures for use in research. A considerable strength demonstrated by the CDEs in this study included the brief, reliable assessment of a comprehensive range of suicidality and related risk factors. The administration of all items included in the CDE would provide one basis for a thorough understanding of suicide risk and related psychopathology for research. Similarly, in the context of examining individual risk factors, administering specific shortened measures may provide a reliable and valid assessment without increasing burden on participants and researchers.

As noted previously, there are numerous advantages to using the shortened CDE scales, particularly through the implementation of the shortened CDE scales in research settings. Findings suggest that the CDEs are psychometrically valid, brief measures that may be used to accurately assess suicide risk and related risk factors (e.g., hopelessness, thwarted belongingness). The brevity of the CDEs and its coverage of a broad range of constructs allow it to be easily incorporated into existing protocols without increasing the burden on researchers and participants. The CDE measures facilitate combination of data across individual studies, thereby providing opportunities for collaboration between researchers and decreasing the demands of data management. Additionally, the shortened CDE measures may facilitate comparisons of findings across individual studies, and may allow researchers to examine distinct populations with identical measures. The resulting improvement of data quality and facilitation of comparison across studies may accelerate the process of disseminating findings.

Furthermore, given the brevity and the strong psychometric properties of the CDE measures, the CDE measures may be useful for screening for suicide risk in clinical settings, particularly those serving military service members. The use of the shortened CDE scales can help to reduce the burden on the patients, and increase clinicians’ ability to detect and manage suicide risk. In particular, the implementation of the CDEs in clinical settings not only allows the clinician to assess immediate suicide risk, but also provides important information regarding risk factors and characteristics of suicidal thoughts and behaviors. Future research evaluating the use of the CDEs in clinical settings would inform the use of these measures outside of research. Importantly, emphasis on establishing clinical cut-off scores for the shortened CDE measures would be beneficial. Future studies may also seek to apply item response theory models, including Rasch analysis, in order to further refine suicidality measures to reduce redundancy as well as examine possible item bias among varying populations.

This study has many strengths, including the use of a large sample comprised of civilians, veterans, and active members of multiple military branches located in diverse regions of the U.S. Additionally, full versions of many of the shortened CDE measures were available to test the validity of scores acquired from these abbreviated scales. However, this study is not without limitations, including the inconsistent gathering of demographic data, the cross-sectional design, and the self-report nature of all measures. Demographics from each study were aggregated, and many variables, such as those examining a history of combat experience, had missing data. Future studies should incorporate identical demographic forms in order to facilitate accurate reporting of demographic variables across individual studies.

Additionally, it should also be noted that the Pearson’s r correlations between full scale measures and shortened CDE scales were not corrected for correlated errors. Thus, it is possible that these correlations may be spuriously high due to correlated errors, a problem which has been noted when evaluating coefficients of short form tests (Levy, 1967). For example, based on calculations examining possible inflations, the correlation between the full PCL-M and the CDE PCL-M may be inflated by .07. The adjusted Pearson’s r, based on these calculations, would be approximately .76. However, the CDE items were administered separately from the full scale measures, and the correlations of the full scales and shortened CDE scales included subsamples who completed the full measure and subsamples who did not. Smith, McCarthy, and Anderson (2000) indicated that administering both forms to the same participants may reduce this problem, and thus, the current study’s method of administration may have decreased the effects of correlated errors. Nevertheless, correlated errors should be considered when interpreting the correlations between the full scale measures and shortened CDE scales. Although the cross-sectional, self-report design allowed us to test our specific hypotheses and to compare the shortened CDE measures to the full versions, we are limited in our ability to determine the abbreviated measures’ ability to predict future behavior, our eventual goal. Implementing a longitudinal design will also allow future studies to examine the test-retest reliability of the CDEs.

The results of this study represent an important first step in providing empirical support for the MSRC’s shortened CDE scales as psychometrically valid measures. The incorporation of these abbreviated measures has the potential to greatly improve researchers’ ability to analyze and collect data from multiple studies, which increases the ease, quality, and speed of data aggregation. Additionally, the brevity of the shortened CDE measures ensures that these measures can be incorporated into studies without significantly increasing the burden on participants. The comprehensive areas of suicide risk factors and related psychopathology assessed by the CDE measures further support the utility of these measures in research and suggests utility for screening purposes in clinical settings.

What is the public significance of this article?

The current study supports the use of CDEs as brief measures of suicide risk and a comprehensive range of risk factors of suicide in research settings among military populations. The CDEs facilitate collaboration among researchers and dissemination of research findings, aiding suicide prevention efforts.