Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Feb 1.
Published in final edited form as: J Abnorm Psychol. 2018 Feb;127(2):139–149. doi: 10.1037/abn0000317

Risk Factors for the Transition from Suicide Ideation to Suicide Attempt: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS)

Matthew K Nock 1, Alexander J Millner 1, Thomas E Joiner 1, Peter M Gutierrez 1, Georges Han 1, Irving Hwang 1, Andrew King 1, James A Naifeh 1, Nancy A Sampson 1, Alan M Zaslavsky 1, Murray B Stein 1, Robert J Ursano 1, Ronald C Kessler 1, On behalf of the STARRS-LS collaborators
PMCID: PMC5851467  NIHMSID: NIHMS908948  PMID: 29528668

Abstract

Prior research has shown that most known risk factors for suicide attempts in the general population actually predict suicide ideation rather than attempts among ideators. Yet clinical interest in predicting suicide attempts often involves the evaluation of risk among patients with ideation. We examined a number of characteristics of suicidal thoughts hypothesized to predict incident attempts in a retrospective analysis of lifetime ideators (n=3,916) drawn from a large (n=29,982), representative sample of U.S. Army soldiers. The most powerful predictors of first nonfatal lifetime suicide attempt in a multivariate model controlling for previously known predictors (e.g., demographics, mental disorders) were: recent onset of ideation, presence and recent onset of a suicide plan, low controllability of suicidal thoughts, extreme risk-taking or “tempting fate,” and failure to answer questions about the characteristics of one’s suicidal thoughts. A predictive model using these risk factors had strong accuracy (AUC=.93), with 66.2% of all incident suicide attempts occurring among the 5% of soldiers with highest composite predicted risk. This high concentration of risk in this retrospective study suggests that a useful clinical decision support model could be constructed from prospective data to identify those with highest risk of subsequent suicide attempt.

Keywords: suicide, suicidal, attempt, Army, military, prediction

Suicide is among the leading causes of death worldwide (World Health Organization, 2017). In the U.S., the suicide rate is especially high among military service members and veterans (Bossarte, Claassen, & Knox, 2010; Nock et al., 2013). Although the number of studies aimed at improving the understanding and prediction of suicidal behavior has increased dramatically over the past several decades (Franklin et al., 2017), the majority of studies seeking to predict suicidal behaviors have repeatedly examined the same, relatively small, set of risk factors (e.g., socio-demographics, mental disorders) in samples of the general population (Franklin et al., 2017). Recent research has shown that the majority of these risk factors are actually risk factors for suicide ideation in the total population rather than for suicide attempts among ideators (Nock, Borges, & Ono, 2012; Nock, Kessler, & Franklin, 2016). This is an important limitation given that most suicide prevention efforts focus on the prevention of attempts among ideators.

The small amount of research that has been carried out on the prediction of suicide attempts among ideators suggests that a number of characteristics of suicidal thoughts are important (Ribeiro et al., 2016). Most notably, recent onset of suicidal thoughts, duration and uncontrollability of these thoughts, severity of ideation at its worst point, and presence of a suicide plan have all been found to predict increased risk of subsequent suicide attempts (Beck, Brown, Steer, Dahlagaard, & Grisham, 1999; Joiner et al., 2003; Nock, Borges, Bromet, Alonso, et al., 2008). However, these findings have been mixed, perhaps due to the small and selected nature of the samples used in those studies (e.g., pediatric emergency patients, bipolar patients) (Horwitz, Czyz, & King, 2015; Jiménez et al., 2016). It also has been proposed that engaging in dangerous and intentionally self-injurious behaviors is associated with increased risk of suicidal behaviors by removing the natural fear and aversion toward self-harm (Joiner, 2005). However, there is also a view that intentional non-suicidal self-injury (NSSI) might be protective in that it can serve as an emotion regulation strategy that decreases a person’s risk for suicide (Suyemoto, 1998). Consistent with this view, a recent study of temporal sequences and trajectories of suicide related thoughts and behaviors in a sample of military personnel and veterans enrolled in college courses found that the transition time from suicide ideation to suicide attempt was significantly longer for those who engaged in NSSI compared to ideators without NSSI (Bryan, Bryan, May, & Klonsky, 2015).

The primary purpose of the current study was to examine the associations of these proposed predictors of first lifetime attempts among ideators in a much larger sample than in previous studies. Notably, most prior studies in this area have examined predictors of lifetime suicide attempt among clinical samples, rather than onset of a first suicide attempt in some general population, such as a large representative sample of Army soldiers. Our focus on the latter was intentional and designed to provide new information that could inform suicide prediction and prevention efforts.

The analysis was carried out in a representative sample of U.S. Army soldiers participating in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS; http://starrs-ls.org). In the interest of focusing on associations of self-injurious thoughts and behaviors (SITB) with subsequent incident nonfatal suicide attempts beyond known risk factors, we controlled for socio-demographics, mental disorders, and other factors shown in a prior analysis of this sample to predict this outcome (Millner et al., in press, 2017). This allowed us to rule out the possibility that any observed association between SITBs and subsequent suicide attempts were due to these other predictors, such as the possibility that SITBs predict suicide attempt because both are caused by mental disorders. Given that different types of SITB may be related, we examined associations of each SITB considered in the analysis both in isolation and in a multivariate model to determine unique associations involving of each SITB type. Importantly, the data analyzed here were retrospective, which means that results need to be considered provisional due to the possibility of recall bias. However, the sample is the baseline for an in-progress follow-up survey that will make it possible subsequently to evaluate the replicability of the results found here in a prospective analysis.

METHOD

Sample

Data are from the Army STARRS Consolidated All-Army Survey (AAS), a study that combines three large survey samples, each administered to a large and representative sample of active duty soldiers. The first of these three surveys was the All-Army Survey (AAS), in which a sample of active duty soldiers (excluding those in Basic Combat Training or deployed to a combat theatre) completed a self-administered questionnaire (SAQ). The AAS sample was drawn in quarterly replicates during the years 2011–2012, with the addition of activated National Guard and Army Reserve (G/R) units in 2013. The AAS sample was stratified by Army Command-location. The survey was administered to all soldiers in units or sub-units selected with probabilities proportional to authorized unit strength (excluding units with <30, which was the case for <2% of Army personnel). All soldiers in selected units were asked to attend an informed consent session (20.2% could not attend because of conflicting duty assignments) covering the study purpose, confidentiality, and voluntary participation before requesting: informed consent, linkage with administrative records (not used in the current study/report), and participation in future/longitudinal data collection. Most attendees at the consent sessions (95.0%) agreed to participate, 97.3% of consenters completed the survey, and 63.1% of completers consented to record linkage. In total, 17,462 AAS soldiers completed the survey and consented to administrative data linkage.

As noted in the previous paragraph, the main AAS excluded soldiers deployed to a combat theatre. The second survey included in the Consolidated AAS was designed to fill that gap by administering the AAS questionnaire to Soldiers in Kuwait waiting to be processed for transit to and from their mid-deployment leave to Afghanistan. In addition to the difference in location of the survey, the sampling differed in that the sample selected individual soldiers rather than units. Each day during the field period, a sample of soldiers either transitioning out of or back into Afghanistan was selected at the airport and recruited in a group session to participate in the survey using the same procedures as in the main AAS. As in the main AAS, most soldiers (80.9%) provided informed consent, 86.5% of consenters completed the survey, and 55.6% of completers provided record linkage. In total, 3,987 soldiers completed the survey and provided consent for administrative data linkage.

The third survey included in the Consolidated AAS was the baseline wave of the longitudinal STARRS Pre-Post Deployment Survey (PPDS) of soldiers in three Brigade Combat Teams who were assessed before deploying to Afghanistan and then again three times after returning from deployment. Recruitment, consent, and data collection procedures were the same as in the other two surveys. Most soldiers invited to participate in the baseline PPDS attended the consent session (96.7%), 98.7% of those attending provided informed consent, 99.2% of consenters completed the survey, and 90.9% of completers providing record linkage. In total, 8,558 soldiers completed the baseline PPDS survey and providing consent for administrative data linkage.

Study procedures were approved by all collaborating organizations involved in this research. Survey responses of soldiers agreeing to administrative data linkage were doubly-weighted before combining to adjust for discrepancies between the sample and population. The first weight (W1) adjusted for differences in survey responses between the respondents who agreed to record linkage and those who did not. The second weight (W2) adjusted for differences in administrative record profiles of weighted (W1) survey completers with record linkage and the target population. The latter weight adjusted the sample to be representative of all active duty soldiers during the years 2011–2012 on the cross-classification of socio-demographics (age, sex, race-ethnicity, education, marital status), command (e.g., Forces Command, Training and Doctrine Command, Reserve Command, Component Commands), occupation (Combat Arms, Combat Support, Combat Service Support), rank (E1–E4, E5–E9, W1–W4, O1–O10), and deployment status-history (never-deployed, currently-deployed, previously deployed). The doubly-weighted (W1×W2) data were combined to create the Consolidated AAS. A more detailed description of AAS weighting is presented elsewhere (Kessler, Heeringa, et al., 2013). Finally, participants (n=25) with unknown survey dates were omitted from the final analytic sample.

Measures

Self-injurious thoughts and behaviors (SITB)

SITBs were measured using a modified version of the Columbia Suicidal Severity Rating Scale (C-SSRS; Posner et al., 2011). The assessment began by asking about lifetime occurrence and age-of-onset (AOO) of suicide ideation (Did you ever in your life have thoughts of killing yourself? Did you ever wish you were dead of would go to sleep and never wake up?), suicide plans (Did you ever think of how you might kill yourself [e.g., taking pill, shooting yourself] or work out a plan of how to kill yourself?), and suicide attempts (Did you ever make a suicide attempt; that is, purposefully hurt yourself with at least some intention to die?). Suicide plan and attempt were only assessed among those who reported lifetime suicide ideation. We also assessed several characteristics of suicide ideation hypothesized to predict subsequent suicide attempt: worst week duration of suicide ideation (i.e., “Think of the one week in your life when you thought most about killing yourself or wanting to be dead. How many days during that worst week did you have those thoughts?” and “How long during that worst week did those thoughts usually last on the days you had them?,” which were combined to produce one duration variable) and worst week controllability of suicide ideation (i.e., “During that worst week, how easy was it for you to control those thoughts or push them out of your mind when you wanted to?”). We also assessed two forms of self-harm behavior that we hypothesized would predict suicide attempts: extreme risk-taking or “tempting fate” (i.e., “People who wish they were dead or think about killing themselves sometimes do dangerous things as a way to tempt fate (e.g., take a lot of drugs, drive too fast, volunteer for dangerous missions, or act recklessly). How often in your life did you ever do dangerous things related to wishing you were dead?”) and non-suicidal self-injury (NSSI; i.e., “Did you ever do something to hurt yourself on purpose, but without wanting to die [e.g., cutting yourself, hitting yourself, or burning yourself]?”).

Mental disorders

We assessed lifetime prevalence and age of onset of five internalizing disorders (major depressive episode [MDE], bipolar disorder [BPD], panic disorder [PD], generalized anxiety disorder [GAD], post-traumatic stress disorder [PTSD]) and three externalizing disorders (attention-deficit/hyperactivity disorder [ADHD], intermittent explosive disorder [IED], and substance use disorders [SUD]). The Composite International Diagnostic Interview (CIDI)(Kessler & Üstün, 2004) screening scales were used to assess BPD, PD and IED, along with ADHD in the 6 months before SAQ completion. All other disorders were assessed with a modified self-report version of the Family History Screen (FHS)(Weissman et al., 2000). The CIDI screening scales have good concordance with independent clinical diagnoses (AUC = .69–.79 across diagnoses)(Kessler, Santiago, et al., 2013). The FHS also has been shown to have good concordance with clinical diagnoses (Weissman et al., 2000); however, the modified version used here yielded implausibly high prevalence estimates and so the diagnoses based on the FHS should be considered combinations of threshold and sub-threshold disorders.

Army career variables

We examined three Army career variables: component (Regular Army versus Reserve Component [i.e., activated G/R]), Military Occupational Specialty (MOS) collapsed into the broad categories of combat arms, combat support, and combat service support, rank (junior enlisted E1–E4, senior enlisted E5–E9, and officers [combining Warrant officers and Commissioned officers]), and deployment status-history (never deployed to a combat theatre, currently deployed, previously deployed). Consistent with prior research on differences in health across different occupations (Gubata, Piccirillo, Packnett, & Cowan, 2013; Lindstrom et al., 2006; Niebuhr et al., 2011), we grouped soldiers into three types of occupations: combat arms (involved directly in ground combat), combat support (provide operational assistance to combat arms), and combat service support (all other occupations) (Kirin & Winkler, 1992; Layne, Naftel, Thie, & Kawata, 2001).

Analysis Methods

We first examined the probability and speed of transitioning from suicide ideation to subsequent suicide attempt among all soldiers. We then used discrete-time survival analysis with person-year the unit of analysis, time-varying predictors, and a logistic link function (Efron, 1988) to examine associations of predictors with first lifetime onset of suicide attempt (n=803) among those with suicide ideation (n=3,916). Notably, we assessed AOO for all predictor variables examined and they were considered to have been present only if their AOO occurred before the outcome being predicted in each model, with the exception of ideation duration, controllability, and tempting fate. Therefore, these last three variables can only be considered correlates of (not risk factors for) the outcomes of interest. We estimated a series of multivariate models that included the hypothesized risk factors (i.e., mental disorders, characteristics of suicide ideation, and self-harm behaviors) after statistically controlling for a number of known risk factors including: time since onset of ideation, presence of a suicide plan, time since onset of the plan, and a range of socio-demographic (i.e., sex, race, marital status) and Army history (i.e., timing of enlistment, years of service, Army component, rank, MOS, and deployment history) factors documented to be significant predictors of suicidal behavior in our prior work (Millner et al., in press, 2017). We examined the association between each hypothesized risk factor and the presence of suicide attempts: (a) after controlling for these previously studied factors (Gross Models), and (b) all together in a final multivariate model including all putative risk factors (Net Model) in order to examine the unique association between each factor and the occurrence of suicide attempts.

Survival coefficients in all models were exponentiated to create odds-ratios (ORs) with 95% confidence intervals. As the survey data are both clustered and weighted, the design-based Taylor series linearization method was used to produce standard errors (Wolter, 1985). Design-based F tests were used to evaluate the significance of predictor sets, with numerator degrees of freedom equal to the number of variables in the set and denominator degrees of freedom equal to the number of sampling error calculation units (SECUs; n=395) with observed attempted suicides minus the sum of primary sample units from which these SECUs were selected (n=208) and one less than the number of variables in the predictor set (Reed, 2007), resulting in 188 denominator degrees of freedom in evaluating univariate associations and fewer in evaluating multivariate associations.

The odds-ratios from the multivariate models described above provide information about the magnitude of the individual-level associations between risk factors and suicide attempts, but do not take into account the prevalence of each predictor or overlap among these predictors. We did this by calculating population attributable risk proportions (PARPs), which represent the proportions of suicide attempts that would be prevented if each set of risk factors were prevented assuming a causal association between risk factor and outcome.

Finally a predicted probability of making a nonfatal attempt among ideators was generated for each respondent from Net model coefficients. A receiver-operating characteristic (ROC) curve was estimated from these predicted probabilities and an area under the ROC curve (AUC) was calculated to quantify the accuracy of the model predictions. We evaluated the sensitivity and positive predictive values at the 1%, 5%, and 10% threshold of respondents with highest predicted probabilities to determine how well the model performs in identifying cases of suicide attempt among ideators. Sensitivity was the proportion of observed attempts found among the 1%, 5%, and 10% of respondents with highest predicted probabilities. Positive predictive value was the prevalence of suicide attempt among these respondents. We used replicated 10-fold cross-validation with 20 replicates (i.e. 200 separate estimates of model coefficients) to correct for the over-estimation of prediction accuracy when both estimating and evaluating model fit (Smith, Seaman, Wood, Royston, & White, 2014).

RESULTS

Descriptive Statistics

As we noted in a previous report, a weighted 13.7% (n = 3,916) of consolidated AAS respondents had a lifetime history of suicide ideation and 20.5% (n = 803) of lifetime ideators made one or more lifetime suicide attempts subsequent to the onset of ideation (Millner et al., 2017). Median (inter-quartile range) age-of-onset (AOO) was 17 (15–23) in the total sample (n = 3,916), 17 (14–22) among ideators who later made a suicide attempt (n = 803), and 17 (15–23) among ideators who did not. Median duration (in years) between onset of ideation and first attempt was 0 (0–2) years. Median duration between onset of a plan and first attempt was 0 (0-0) years. Median AOO of suicide attempt was 18 (15–23). The distribution of demographic and Army career variables at the AOO of all suicide ideators is presented in Appendix Table 1.

Appendix Table 1.

Distribution of demographic and army career characteristics at the age of onset, among ideators in the full AAS consolidated sample, weighted analysis

Total Sample No Attempt Attempt
(n=3,916) (n=3,113) (n=803)
% SE % SE % SE
Demographics
 Age
  ≤14 22.8 0.9 21.7 1.2 27.1 2.7
  15–17 29.9 1.4 29.4 1.3 31.9 3.0
  ≥18 47.3 1.2 48.9 1.4 41.0 2.6
 Sex: female 19.7 1.1 18.5 1.0 24.2 2.4
 Race
  White 66.8 1.3 67.5 1.6 64.4 2.3
  Black 15.2 0.7 15.1 0.9 15.8 2.0
  Hispanic 10.9 1.1 10.9 1.4 10.8 1.6
  Other 7.0 0.6 6.5 0.6 9.1 1.5
Army characteristics
 Pre-enlistment 63.8 1.1 63.3 1.4 66.0 2.3
 Post-enlistment: before 2000 2.9 0.4 3.3 0.5 1.6 0.6
 Post-enlistment: after 2000 33.2 1.0 33.5 1.2 32.4 2.2
 Component: RA vs. G-R
 MOSa
  Combat arms 33.2 2.2 33.2 2.1 33.2 6.1
  Combat support 23.8 1.9 23.6 1.9 24.9 4.5
  Combat service support 42.9 2.2 43.2 2.1 41.9 4.8
 Ranka
  Junior 54.3 1.8 52.2 2.3 63.1 5.5
  Senior 31.9 1.6 33.0 2.1 27.4 5.5
  Officer 13.8 1.5 14.8 1.6 9.5 2.4
 Deploymenta
  Currently 8.0 1.1 8.9 1.2 4.8 1.8
  Previously 36.9 2.3 38.3 2.7 31.3 2.7
  Never 21.4 1.8 20.3 1.9 26.1 4.7
  Unknown (before 2000) 33.5 2.4 32.5 2.4 37.7 5.4
Open in a new tab
a

There were n=1,332 Respondents who had enlisted in the Army by the age of their ideation onset. Of enlisted ideators, there were n=1,101 who did not make a subsequent suicide attempt and n=231 who made a suicide attempt.

Probability and speed of transition from ideation to first lifetime attempt

More than half (68.2%) of the observed transitions from suicide ideation to first suicide attempt occur in the year of onset of ideation and more than three quarters (91.1%) occur in the first five years after onset of ideation. Notably, these trends differ depending on whether or not the ideation is accompanied by a plan and whether the plan began in the same year as the ideation or only after the ideation (Figure 1). Among ideators who develop a plan in the same year as the onset of their ideation, 28.4% make an attempt at the AOO of the plan and 35.3% within five years of onset of the plan. Among ideators who develop a plan in a year after the AOO of their ideation, 19.9% make an attempt at the AOO of the plan and 25.2 within five years of onset of the plan. Among ideators without a plan, 6.9% make an attempt at the AOO of ideation and 9.3% do so within five years of onset of ideation.

Figure 1.

Figure 1

Open in a new tab

Probability and timing of transition to first suicide attempt among lifetime ideators (n=3,916).

Note. Respondents of the third curve curve (in green) were followed from onset of plan until onset of attempt or censoring, whereas respondents of the first two curves (in blue and green) were followed from onset of ideation until onset of attempt or censoring.

Evaluation of the joint predictive effects of these factors along with AOO of ideation shows that AOO of ideation is not a significant predictor of a subsequent attempt (F2,186 = 1.0, p=.36) and that information about the presence and timing of AOO of a plan relative to AOO of ideation is significant (F5,183 = 138.3, p<.001). The year of highest odds of onset of a first attempt is the year of onset of a plan among soldiers whose plan began at the AOO of ideation (OR = 5.6) followed by the year of onset of a plan among soldiers whose plan began at a later year than their AOO of ideation (OR = 3.6), followed by the year of onset of ideation among soldiers without a plan (OR = 1.0). Conditional odds of making a first attempt in later years are dramatically lower both among soldiers with (OR = 0.6-0.6) and without (OR = 0.1) a plan. In addition, there is a meaningfully negative time trend in odds of incident attempts in the years after onset of ideation and plans (F3,185 = 12.7, p <.001).

Predictors of the transition from ideation to attempt

The associations between each set of the putative risk factors and the transition from suicide ideation to first suicide attempt are presented in Table 1. In the Gross Models, we examined the association between each set of risk factors shown (entered one set at a time) after adjusting for socio-demographic and Army history factors documented in our prior work (Millner et al., in press, 2017) as well as for time-since-onset of ideation and plan. In the Net Model we entered all of these variables simultaneously in one multivariate model to examine the unique association between each predictor and the transition from ideation to attempt.

Table 1.

Predictors of Suicide Attempt among Ideators (n=3,916)

Gross Models Net Model
OR (95% CI) OR (95% CI)
Mental disorders
 Major depressive episode (SC) 1.5* (1.2–1.9) 1.4* (1.1–1.8)
 Generalized anxiety disorder (SC) 1.0 (0.8–1.3) 1.0 (0.8–1.2)
 Post-traumatic stress disorder (SC) 1.3* (1.1–1.7) 1.3* (1.0–1.7)
 Bipolar disorder 1.2 (0.9–1.8) 1.1 (0.7–1.6)
 Panic disorder 1.7* (1.2–2.5) 1.4 (1.0–2.0)
 Intermittent explosive disorder 1.4* (1.1–1.8) 1.4* (1.1–1.8)
 Attention deficit hyperactivity disorder 0.8 (0.6–1.1) 0.6* (0.5–0.9)
 Substance use disorder (SC) 2.1* (1.5–2.9) 1.7 (1.2–2.3)
 F8,180 11.3* <0.001 6.3* <0.001
Worst week ideation duration (reference: just a few minutes, during 1 day)
 Just a few minutes, during 2–3 days 1.3 (0.4–3.6) 1.0 (0.4–2.5)
 Just a few minutes, during 4+ days 0.9 (0.3–2.6) 0.6 (0.2–1.6)
 Less than 1 to 4 hours, during 1 day 1.7 (0.7–4.1) 1.5 (0.7–3.0)
 Less than 1 to 4 hours, during 2–3 days 1.3 (0.4–3.9) 0.9 (0.3–2.5)
 Less than 1 to 4 hours, during 4–6 days 1.7 (0.7–3.8) 1.0 (0.5–2.2)
 Less than 1 to 4 hours, during 7 days 1.8 (0.6–5.3) 0.8 (0.3–2.3)
 More than 5 hours, during 1–3 days 2.6* (1.1–6.3) 1.4 (0.6–3.1)
 More than 5 hours, during 4–6 days 2.9 (1.0–8.6) 1.1 (0.4–3.3)
 More than 5 hours, during 7 days 3.6* (1.3–10.4) 1.1 (0.4–2.9)
 Missing 5.9* (1.9–18.6) 6.7* (2.5–18.3)
 F10,178 12.0* <0.001 8.3* <0.001
Controllability of thoughts during worst week (reference: easy)
 A little difficult 0.9 (0.6–1.3) 1.1 (0.7–1.6)
 Somewhat difficult 1.3 (0.9–1.9) 1.4 (0.9–2.2)
 Very difficult 2.4* (1.6–3.7) 2.3* (1.4–3.5)
 Impossible 3.4* (2.0–5.8) 3.0* (1.9–4.9)
 F4,184 15.4* < 0.001 7.1* < 0.001
Frequency of tempting fate (reference: never)
 Rarely 1.5* (1.2–2.0) 1.6* (1.2–2.1)
 Sometimes 2.1* (1.6–2.8) 1.7* (1.2–2.5)
 Often 3.3* (2.3–4.9) 2.7* (1.7–4.1)
 F3,185 6.0* < 0.001 6.4* < 0.001
Nonsuicidal self-injury 1.5* (1.2–1.9) 1.0 (0.8–1.3)
Open in a new tab

Note: There are 395 sampling error calculation units and 208 primary sampling units (i.e. a net difference of 187).

Disorders assessed with screening questions are designated by “(SC).”

Several important predictive associations are observed in the Gross models. Five of the eight mental disorders examined are significantly associated with the onset of a suicide attempt, with effect sizes ranging from OR=1.3 for PTSD to OR=2.1 for substance use disorder. In the case of worst week suicide ideation, associations are small and insignificant for shorter durations of ideation, but become larger and statistically significant if lasting more than five hours, with ORs increasing monotonically as the number of days with ideation increase from 1–3 days/week (OR=2.6), to 4–6 days/week (OR=2.9), to 7 days/week (OR=3.6). There also is a significant association between not answering this survey item and odds of suicide attempt (OR=5.9). A similar pattern is observed in the case of worst week controllability of suicidal thoughts, such that there are small and non-significant ORs among those who do not find it difficult to control their suicidal thoughts, but larger, significant, and increasing ORs for those who report it is “very difficult” (OR=2.4) or “impossible” (OR=3.4) to control their thoughts of suicide. Engaging in behavior that tempted fate (ORs=1.5–3.3) and engaging in NSSI (OR=1.5) also are associated with significantly increased odds of transitioning from suicide ideation to attempt.

In the Net Model that includes all risk factors simultaneously, most ORs are slightly attenuated compared to in the Gross Models, whereas others decrease substantially and are no longer significantly predictive of suicide attempts. ORs are largely unchanged for mental disorders, with the three of the initial five disorders being significantly associated with suicide attempt, and now the (negative) association between ADHD and suicide attempt is significant (OR=0.6). The effects of worst week ideation duration are largely gone (with the only remaining significant effect for having missing data on this item), as are the association between NSSI and suicide attempt. In contrast, the ORs are only slightly attenuated for controllability of ideation and tempting fate. This pattern of findings suggests that mental disorders, controllability of suicidal thoughts, and engaging in behavior that tempts fate all contribute uniquely (relative to the other variables in the model) to the transition from suicide ideation to suicide attempt. Duration of suicide ideation (which is strongly correlated with controllability; r=.48) and NSSI (which is significantly associated with tempting fate; r=.17) are not predictive when these other variables are taken into account.

Population attributable risk proportions (PARPS)

As noted above in the section on analysis methods, PARPs represent the proportions of observed cases of suicide attempt among ideators that would be prevented if specific predictor variables were prevented based on the assumption that the odds-ratios for these predictors represent causal effects. Focusing first on the PARPs for the Gross Models, we estimate that approximately 54% of suicide attempts among ideators are associated with having a suicide plan, 43% are associated with prior mental disorders, 61% with ideation duration, 40% with ideation controllability, 42% with tempting fate, and 11% with NSSI (Table 2). As in the predictive models described above, these effects are attenuated in the Net Model, with the strongest effects observed for suicide plan (47%), ideation controllability (39%), and tempting fate (38%). Assuming that these are causal associations, preventing these predictors from occurring or blocking their effects would prevent 86% of suicide attempts among ideators.

Table 2.

Population Attributable Risk Proportion (PARP) for Suicide Attempt among Ideators

Gross Models Net Model
PARP PARP
Suicide Plan 53.5% 47.2%
Mental disorders 42.8% 27.5%
Ideation duration 60.5% 8.8%
Controllability 39.6% 38.6%
Tempting fate 42.0% 37.8%
Nonsuicidal self-injury 10.6% 1.0%
All of the above 86.2%
Open in a new tab

Note: n=803 observed suicide attempts in this sample of n=3,916 ideators.

Prediction accuracy in the net model

We evaluated the predictive accuracy of our final Net Model by calculating individual-level predicted probabilities with 20 replicates of 10-fold cross-validation and then estimating ROC curves and AUC based on these predictive probabilities. The overall estimated AUC was .93, meaning that if we used the results of this model to classify two randomly selected respondents, one ideator who made an attempt and one who did not, we would classify them correctly 93% of the time (Figure 2). The 1% of respondents with the highest predicted probability of suicide attempt (according to our model) include 30.0% of all attempters (sensitivity)(Table 3). The 5% of respondents with the highest probability include 66.2% of all attempters, and the top 10% include 80.2% of all attempters. Positive predictive value (the proportion of predicted positives who made an attempt) for the top 1%, 5%, and 10% are 39.5%, 22.4%, and 14.9%, respectively.

Figure 2.

Figure 2

Open in a new tab

ROC curve for final model predicting suicide attempts among ideators, simulated sample of full person-year file, weighted analysis (n=690,500).

Table 3.

Concentration of Risk of Suicide Attempt among Ideators, Weighted Analysis

Simulated sample (n=690,500)
Sensitivity Positive predictive value
% of suicide attempts (SE) % with suicide attempt (SE)
Top 99th percentile 30.0 (2.9) 39.5 (3.1)
Top 98th percentile 44.2 (2.9) 32.5 (2.2)
Top 97th percentile 55.8 (3.1) 28.8 (1.7)
Top 96th percentile 61.4 (3.0) 25.1 (1.4)
Top 95th percentile 66.2 (3.1) 22.4 (1.3)
Top 90th percentile 80.2 (2.2) 14.9 (0.8)
Open in a new tab

Note: Estimates calculated from 20 replicates of 10-fold cross-validation of the final model.

DISCUSSION

There are three key findings from this study. First, we identified several different characteristics of suicidal thinking that are associated with increased risk of suicide ideators making incident suicide attempts: recency of ideation onset, presence of a suicide plan, and worst week duration and controllability of suicidal thoughts. Second, we found that two self-harm related behaviors, tempting fate and NSSI, were associated with increased risk of incident suicide attempts among ideators. Third, we found that, as a group, all the risk factors studied here predicted which people with suicide ideation went on to make a suicide attempt with a fairly high degree of accuracy (AUC=.93). Each of these findings warrants additional comment.

For decades, theories of suicidal behavior have proposed that the risk factors for suicide ideation differ from those for suicide attempt (Joiner, 2005; Mann, Waternaux, Haas, & Malone, 1999; Shaffer & Pfeffer, 2001). Prior research has supported this idea by showing that most risk factors for suicide attempt and death actually predict suicide ideation, but not the transition from suicidal thought and behavior (Nock, Borges, Bromet, Cha, et al., 2008; Nock et al., 2016). Factors predicting the latter transition have been much more difficult to identify. As such, the identification in this study of several characteristics of suicidal thinking that predict this transition is an important finding. The findings that recent onset of suicide ideation and the presence of a suicide plan predict suicide attempts among ideators is consistent with results from prior studies (Nock, Borges, Bromet, Alonso, et al., 2008; Nock et al., 2012). New findings in this study are that worst point duration and controllability of suicide ideation statistically predict the transition to suicide attempt. Specifically, relatively brief periods of suicidal thinking lasting <5 hours are not associated with an increased risk of going on to make a suicide attempt, whereas longer periods of time are. If replicated in a prospective study, this finding would have important clinical implications regarding the prediction of suicide attempts. It is perhaps not surprising that greater duration of suicide ideation also is strongly associated with greater difficulty controlling suicidal thoughts, but less intuitive is the finding that when examined together, it is controllability that is the significant predictor of suicide attempts – even after controlling for mental disorders – suggesting that controllability is an especially important clinical predictor of suicidal behavior.

It also is noteworthy that a refusal to answer the question about duration of suicidal thinking had an especially strong association with subsequent risk of suicidal behavior. This finding is in line with those from a recent study showing that those who choose not to answer questions about suicide show elevated scores on other measures of suicide risk (Podlogar et al., 2016). These findings suggest that non-response to questions about suicidal thinking may provide useful information in the prediction of suicidal behavior. This important finding should be followed up in future studies and, if replicated, should factor into clinical risk assessments.

Researchers and clinicians have long sought to understand what psychological or behavioral factors might increase the risk of acting on one’s suicidal thoughts. It has been suggested that the tendency or willingness to intentionally hurt oneself (in the form of NSSI) or engage in potentially harmful behaviors (in the form of risk-taking) may increase the risk of suicide attempt (Ammerman, Steinberg, & McCloskey, 2016; Ribeiro et al., 2016). The current study suggests that extreme risk-taking, or “tempting fate,” and NSSI are significant predictors of the transition from suicidal thought to suicidal action. Importantly, tempting fate was an especially strong predictor of this transition and was one of the few significant predictors in the final multivariate model, whereas NSSI was not. Indeed, the single-item assessment of tempting fate used in this study was more strongly associated with suicide attempt among ideators than each of the mental disorders examined in this study and had a larger association with suicide attempt in the PARP calculations than did all mental disorders combined. To our knowledge, this is the first study to examine the role of tempting fate in predicting suicidal behavior. As such, additional studies are needed to test the reliability and generality of this association, and if supported, to better understand how and why tempting fate is associated with this risk.

The ultimate goal of this line of work is to improve the prediction and prevention of suicidal behavior. Toward this end, the finding in this study that nearly nine out of 10 (86%) of suicide attempts among ideators are attributable to this relatively small handful of risk factors suggests that they may be useful in improving the prediction of suicide attempts. Moreover, the finding that an algorithm using these factors identified a small proportion of soldiers with suicide ideation who account for the majority of subsequent suicide attempts suggests that these risk factors can help with clinical prediction of such behavior. On balance, however, only 39.5% of the soldiers in the highest-risk group identified by the model (top 1% of risk scores) went on to make a suicide attempt, a false positive rate that presents a significant challenge to the current feasibility of using such an algorithm in actual clinical settings (Nielssen, Wallace, & Large, 2017; Quinlivan et al., 2017). Although the results of the current study represent a significant advance in understanding and prediction of suicidal behavior, we need much greater understanding and accuracy to aid prediction and prevention efforts.

The results of this study should be interpreted in the context of six important limitations. The most obvious of these is that the analysis was based on retrospective data. Recall error and differential survivorship could have biased results. Moreover, several key predictors (ideation duration, controllability, and tempting fate) were not time stamped and could have occurred after respondents’ first lifetime suicide attempt – in which case they would represent correlates of making a suicide attempt, but not necessarily predictors of that outcome. As a result, it will be important to evaluate the consistency of these results in future prospective studies that carefully document the time of onset of each predictor and outcome. Second, the response and linkage rates were relatively low, limiting inferences that can be made about the external validity of these findings to other Army soldiers. Post-stratification was used to adjust for differences between the sample and the population; however, this limitation remains a concern. Third, it is likely that respondents under-reported suicidal thoughts and behaviors due to concerns about confidentiality or stigma (Zinzow et al., 2013) and/or displayed errors or bias in their recollections of the timing of the AOO of suicidal thoughts and behaviors. Both concerns would introduce imprecision into our predictive models. Fourth, although we included a large number of covariates in our models, we included a relatively small set of types of SITBs that might predict the transition from suicide ideation to suicide attempt. Therefore, it will be important to test the associations between other SITBs (e.g., preparatory behavior, interrupted/aborted suicide attempt) and suicide attempts in future studies. Fifth, we used screeners rather than full diagnostic interviews to assess mental disorders. Although several of these screeners showed acceptable-to-good concordance with full diagnostic interviews, some had lower concordance rates (with implausibly high prevalence estimates) and so should be considered combinations of threshold and subthreshold disorders. Sixth, because the focus of this study was on Army soldiers, the generality of these findings to other populations also is not known. Prior studies have revealed that there is much overlap between the risk factors for suicidal behavior between service members and civilians (Nock et al., 2013); nevertheless, the replication of these results in other populations represents an important goal for future research, as some differences in risk factors between service members and civilians have been observed (Nock et al., 2014; Ursano et al., 2016).

General Scientific Summary.

Suicide is a leading cause of death worldwide. One of the greatest challenges in this area is predicting which people who think about suicide will act on those thoughts and make a suicide attempt. In this study we documented several characteristics of suicide-related thoughts that predict incident nonfatal suicide attempts with good accuracy.

Acknowledgments

The Army STARRS Team consists of Co-Principal Investigators: Robert J. Ursano, MD (Uniformed Services University of the Health Sciences) and Murray B. Stein, MD, MPH (University of California San Diego and VA San Diego Healthcare System)

Site Principal Investigators: Steven Heeringa, PhD (University of Michigan), James Wagner, PhD (University of Michigan), and Ronald C. Kessler, PhD (Harvard Medical School)

Army liaison/consultant: Kenneth Cox, MD, MPH (USAPHC (Provisional))

Other team members: Pablo A. Aliaga, MA (Uniformed Services University of the Health Sciences); COL David M. Benedek, MD (Uniformed Services University of the Health Sciences); Laura Campbell-Sills, PhD (University of California San Diego); Chia-Yen Chen DSc (Harvard Medical School); Carol S. Fullerton, PhD (Uniformed Services University of the Health Sciences); Nancy Gebler, MA (University of Michigan); Joel Gelernter (Yale University); Robert K. Gifford, PhD (Uniformed Services University of the Health Sciences); Paul E. Hurwitz, MPH (Uniformed Services University of the Health Sciences); Sonia Jain, PhD (University of California San Diego); Tzu-Cheg Kao, PhD (Uniformed Services University of the Health Sciences); Lisa Lewandowski-Romps, PhD (University of Michigan); Holly Herberman Mash, PhD (Uniformed Services University of the Health Sciences); James E. McCarroll, PhD, MPH (Uniformed Services University of the Health Sciences); James A. Naifeh, PhD (Uniformed Services University of the Health Sciences); Tsz Hin Hinz Ng, MPH (Uniformed Services University of the Health Sciences); Matthew K. Nock, PhD (Harvard University); Nancy A. Sampson, BA (Harvard Medical School); CDR Patcho Santiago, MD, MPH (Uniformed Services University of the Health Sciences); Jordan W. Smoller, MD, ScD (Harvard Medical School); and Alan M. Zaslavsky, PhD (Harvard Medical School).

Financial Support: Army STARRS was sponsored by the Department of the Army and funded under cooperative agreement number U01MH087981 (2009–2015) with the U.S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (NIH/NIMH). Subsequently, STARRS-LS was sponsored and funded by the Department of Defense (USUHS grant number HU0001-15-2-0004). The contents are solely the responsibility of the authors and do not necessarily represent the views of the Department of Health and Human Services, NIMH, the Department of the Army, the Department of Veterans Affairs, or the Department of Defense. Additional Information: A complete list of Army STARRS publications can be found at http://www.STARRS-LS.org. This work also was supported in part by funding from the Military Suicide Research Consortium (MSRC), an effort supported by the Office of the Assistant Secretary of Defense for Health Affairs (Award No. W81XWH-10-2-0181). Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the MSRC, the Department of Veterans Affairs, or the Department of Defense.

Conflict of Interest: Dr. Stein has in the past three years been a consultant for Actelion, Dart Neuroscience, Healthcare Management Technologies, Janssen, Oxeia Biopharmaceuticals, Pfizer, Resilience Therapeutics, and Tonix Pharmaceuticals. In the past 3 years, Dr. Kessler received support for his epidemiological studies from Sanofi Aventis; was a consultant for Johnson & Johnson Wellness and Prevention, Shire, Takeda; and served on an advisory board for the Johnson & Johnson Services Inc. Lake Nona Life Project. Kessler is a co-owner of DataStat, Inc., a market research firm that carries out healthcare research. The remaining authors have no financial disclosures.

References

  1. Ammerman BA, Steinberg L, McCloskey MS. Risk-taking behavior and suicidality: The unique role of adolescent drug use. Journal of Clinical Child & Adolescent Psychology. 2016:1–11. doi: 10.1080/15374416.2016.1220313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Beck AT, Brown GK, Steer RA, Dahlagaard KK, Grisham JR. Suicide ideation at its worst point: A predictor of eventual suicide in psychiatric outpatients. Suicide and Life-Threatening Behavior. 1999;29:1–9. [PubMed] [Google Scholar]
  3. Bossarte RM, Claassen CA, Knox KL. Evaluating evidence of risk for suicide among veterans. Military Medicine. 2010;175:703–704. doi: 10.7205/milmed-d-10-00247. [DOI] [PubMed] [Google Scholar]
  4. Bryan CJ, Bryan AO, May AM, Klonsky ED. Trajectories of suicide ideation, nonsuicidal self-injury, and suicide attempts in a nonclinical sample of military personnel and veterans. Suicide and Life-Threatening Behavior. 2015;45:315–325. doi: 10.1111/sltb.12127. [DOI] [PubMed] [Google Scholar]
  5. Efron B. Logistic regression, survival analysis, and the Kaplan-Meier curve. Journal of the American Statistical Association. 1988;83:414–425. [Google Scholar]
  6. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, … Nock MK. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin. 2017;143:187–232. doi: 10.1037/bul0000084. [DOI] [PubMed] [Google Scholar]
  7. Gubata ME, Piccirillo AL, Packnett ER, Cowan DN. Military occupation and deployment: Descriptive epidemiology of active duty U.S. Army men evaluated for a disability discharge. Military Medicine. 2013;178:708–714. doi: 10.7205/MILMED-D-12-00542. [DOI] [PubMed] [Google Scholar]
  8. Horwitz AG, Czyz EK, King CA. Predicting future suicide attempts among adolescent and emerging adult psychiatric emergency patients. Journal of Clinical Child & Adolescent Psychology. 2015;44:751–761. doi: 10.1080/15374416.2014.910789. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Jiménez E, Arias B, Mitjans M, Goikolea JM, Ruíz V, Brat M, … Benabarre A. Clinical features, impulsivity, temperament and functioning and their role in suicidality in patients with bipolar disorder. Acta Psychiatrica Scandinavica. 2016;133:266–276. doi: 10.1111/acps.12548. [DOI] [PubMed] [Google Scholar]
  10. Joiner TE. Why people die by suicide. Cambridge: Harvard University Press; 2005. [Google Scholar]
  11. Joiner TE, Steer RA, Brown G, Beck AT, Pettit JW, Rudd MD. Worst-point suicidal plans: A dimension of suicidality predictive of past suicide attempts and eventual death by suicide. Behaviour Research and Therapy. 2003;41:1469–1480. doi: 10.1016/s0005-7967(03)00070-6. [DOI] [PubMed] [Google Scholar]
  12. Kessler RC, Heeringa SG, Colpe LJ, Fullerton CS, Gebler N, Hwang I, … Ursano RJ. Response bias, weighting adjustments, and design effects in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) International Journal of Methods in Psychiatric Research. 2013;22:288–302. doi: 10.1002/mpr.1399. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Kessler RC, Santiago PN, Colpe LJ, Dempsey CL, First MB, Heeringa SG, … Ursano RJ. Clinical reappraisal of the Composite International Diagnostic Interview Screening Scales (CIDI-SC) in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS): Clinical reappraisal of the CIDI-SC in Army STARRS. International Journal of Methods in Psychiatric Research. 2013;22:303–321. doi: 10.1002/mpr.1398. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kessler RC, Üstün TB. The World Mental Health (WMH) Survey Initiative version of the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI) International Journal of Methods in Psychiatric Research. 2004;13:93–121. doi: 10.1002/mpr.168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kirin SJ, Winkler JD. The Army military occupational specialty database. DTIC Document; 1992. Retrieved from http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA428376. [Google Scholar]
  16. Layne M, Naftel S, Thie HJ, Kawata JH. Military occupational specialties. RAND Corporation; 2001. Retrieved from http://www.rand.org/content/dam/rand/pubs/monograph_reports/2009/MR977.pdf. [Google Scholar]
  17. Lindstrom KE, Smith TC, Wells TS, Wang LZ, Smith B, Reed RJ, … Ryan MAK. The mental health of U.S. military women in combat support occupations. Journal of Women’s Health. 2006;15:162–172. doi: 10.1089/jwh.2006.15.162. [DOI] [PubMed] [Google Scholar]
  18. Mann JJ, Waternaux C, Haas GL, Malone KM. Toward a clinical model of suicidal behavior in psychiatric patients. American Journal of Psychiatry. 1999;156:181–189. doi: 10.1176/ajp.156.2.181. [DOI] [PubMed] [Google Scholar]
  19. Millner AJ, Ursano RJ, Hwang I, King AJ, Naifeh JA, Sampson NA … the STARRS-LS Collaborators. Lifetime suicidal behaviors and career characteristics among U.S. Army soldiers: results from the army study to assess risk and resilience in servicemembers (Army STARRS) Suicide and Life-Threatening Behavior. 2017 doi: 10.1111/sltb.12363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Millner AJ, Ursano RJ, Hwang I, King A, Naifeh JA, Sampson NA, … Nock MK. Prior mental disorders and lifetime suicidal behaviors among U.S. Army soldiers in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) doi: 10.1111/sltb.12394. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Niebuhr DW, Krampf RL, Mayo JA, Blandford CD, Levin LI, Cowan DN. Risk factors for disability retirement among healthy adults joining the US Army. Military Medicine. 2011;176:170–175. doi: 10.7205/milmed-d-10-00114. [DOI] [PubMed] [Google Scholar]
  22. Nielssen O, Wallace D, Large M. Pokorny’s complaint: The insoluble problem of the overwhelming number of false positives generated by suicide risk assessment. BJPsych Bull. 2017;41:18–20. doi: 10.1192/pb.bp.115.053017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Nock MK, Borges G, Bromet EJ, Alonso J, Angermeyer M, Beautrais A, … Williams D. Cross-national prevalence and risk factors for suicidal ideation, plans and attempts. The British Journal of Psychiatry. 2008;192:98–105. doi: 10.1192/bjp.bp.107.040113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Nock MK, Borges G, Bromet EJ, Cha CB, Kessler RC, Lee S. Suicide and suicidal behavior. Epidemiologic Reviews. 2008;30:133–154. doi: 10.1093/epirev/mxn002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Nock MK, Borges G, Ono Y. Suicide: Global perspectives from the WHO World Mental Health Surveys. Cambridge University Press; 2012. [Google Scholar]
  26. Nock MK, Deming CA, Fullerton CS, Gilman SE, Goldenberg M, Kessler RC, … Ursano RJ. Suicide among soldiers: A review of psychosocial risk and protective factors. Psychiatry: Interpersonal and Biological Processes. 2013;76:97–125. doi: 10.1521/psyc.2013.76.2.97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Nock MK, Kessler RC, Franklin JC. Risk factors for suicide ideation differ from those for the transition to suicide attempt: The importance of creativity, rigor, and urgency in suicide research. Clinical Psychology: Science and Practice. 2016;23:31–34. [Google Scholar]
  28. Nock MK, Stein MB, Heeringa SG, Ursano RJ, Colpe LJ, Fullerton CS, … Kessler RC. Prevalence and correlates of suicidal behavior among soldiers: Results from the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) JAMA Psychiatry. 2014;71:514. doi: 10.1001/jamapsychiatry.2014.30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Podlogar MC, Rogers ML, Chiurliza B, Hom MA, Tzoneva M, Joiner T. Who are we missing? Nondisclosure in online suicide risk screening questionnaires. Psychological Assessment. 2016;28:963–974. doi: 10.1037/pas0000242. [DOI] [PubMed] [Google Scholar]
  30. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, … Shen S. The Columbia–Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. American Journal of Psychiatry. 2011;168:1266–1277. doi: 10.1176/appi.ajp.2011.10111704. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Quinlivan L, Cooper J, Meehan D, Longson D, Potokar J, Hulme T, … Kapur N. Predictive accuracy of risk scales following self-harm: Multicentre, prospective cohort study. The British Journal of Psychiatry. 2017 doi: 10.1192/bjp.bp.116.189993. bjp.bp.116.189993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Reed JF. Better binomial confidence intervals. Journal of Modern Applied Statistical Methods. 2007;6:153–161. [Google Scholar]
  33. Ribeiro JD, Franklin JC, Fox KR, Bentley KH, Kleiman EM, Chang BP, Nock MK. Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: A meta-analysis of longitudinal studies. Psychological Medicine. 2016;46:225–236. doi: 10.1017/S0033291715001804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Shaffer D, Pfeffer CR. Practice parameter for the assessment and treatment of children and adolescents with suicidal behavior. Journal of the American Academy of Child & Adolescent Psychiatry. 2001;40:24S–51S. doi: 10.1097/00004583-200107001-00003. [DOI] [PubMed] [Google Scholar]
  35. Smith GCS, Seaman SR, Wood AM, Royston P, White IR. Correcting for optimistic prediction in small data sets. American Journal of Epidemiology. 2014;180:318–324. doi: 10.1093/aje/kwu140. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Suyemoto KL. The functions of self-mutilation. Clinical Psychology Review. 1998;18:531–554. doi: 10.1016/s0272-7358(97)00105-0. [DOI] [PubMed] [Google Scholar]
  37. Ursano RJ, Kessler RC, Stein MB, Naifeh JA, Aliaga PA, Fullerton CS, … Heeringa SG. Risk factors, methods, and timing of suicide attempts among US Army soldiers. JAMA Psychiatry. 2016;73:741–749. doi: 10.1001/jamapsychiatry.2016.0600. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Weissman MM, Wickramaratne P, Adams P, Wolk S, Verdeli H, Olfson M. Brief screening for family psychiatric history: The Family History Screen. Archives of General Psychiatry. 2000;57:675–682. doi: 10.1001/archpsyc.57.7.675. [DOI] [PubMed] [Google Scholar]
  39. Wolter KM. Introduction to variance estimation. New York, NY: Springer-Verlag; 1985. [Google Scholar]
  40. World Health Organization. Suicide. WHO; 2017. Retrieved from http://www.who.int/mediacentre/factsheets/fs398/en/ [Google Scholar]
  41. Zinzow HM, Britt TW, Pury CLS, Raymond MA, McFadden AC, Burnette CM. Barriers and facilitators of mental health treatment seeking among active-duty Army personnel. Military Psychology. 2013;25:514–535. [Google Scholar]

RESOURCES