Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Aug 6.
Published in final edited form as: Psychiatry Res. 2018 Jan 31;262:175–183. doi: 10.1016/j.psychres.2018.01.045

Identifying the Relative Importance of Non-Suicidal Self-Injury Features in Classifying Suicidal Ideation, Plans, and Behavior Using Exploratory Data Mining

Taylor A Burke a, Ross Jacobucci b, Brooke A Ammerman a, Marilyn Piccirillo c, Michael S McCloskey a, Richard G Heimberg a, Lauren B Alloy a
PMCID: PMC6684203  NIHMSID: NIHMS1043244  PMID: 29453036

Abstract

Individuals with a history of non-suicidal self-injury (NSSI) are at alarmingly high risk for suicidal ideation (SI), planning (SP), and attempts (SA). Given these findings, research has begun to evaluate the features of this multi-faceted behavior that may be most important to assess when quantifying risk for SI, SP, and SA. However, no studies have examined the wide range of NSSI characteristics simultaneously when determining which NSSI features are most salient to suicide risk. The current study utilized three exploratory data mining techniques (elastic net regression, decision trees, random forests) to address these gaps in the literature. Undergraduates with a history of NSSI (N = 359) were administered measures assessing demographic variables, depression, and 58 NSSI characteristics (e.g., methods, frequency, functions, locations, scarring) as well as current SI, current SP, and SA history. Results suggested that depressive symptoms and the anti-suicide function of NSSI were the most important features of NSSI for predicting SI and SP. The most important features in predicting SA were the anti-suicide function of NSSI, NSSI-related medical treatment, and scarring. Overall, results suggest that NSSI functions, scarring, and medical lethality may be more important to assess than commonly regarded NSSI severity indices when ascertaining suicide risk.

Keywords: non-suicidal self-injury, suicidal ideation, suicide plan, suicide attempt, exploratory data mining, elastic net regression, decision trees

1. Introduction

Suicide is a major public health problem, ranking as the second leading cause of death among adolescents and adults ages 15–24 (Centers for Disease Control, 2014). Although a large body of research has identified numerous risk factors for suicidal behavior, a recent meta-analysis suggests that these risk factors are weak predictors of suicidal thoughts and behaviors, resulting in a relatively poor ability to predict its occurrence (Franklin et al., 2016). In turn, rates of suicide have continued to increase (Curtin et al., 2016). However, exploratory data mining (EDM) techniques recently have allowed researchers to examine risk factors simultaneously in multivariate predictive models that can meaningfully augment the prediction of suicidal behavior (e.g., Kessler et al., 2017; Kessler et al., 2015; Walsh et al., 2017). Indeed, such procedures have recently significantly improved accuracy (e.g., AUC = 0.84; Walsh et al., 2017) in predicting suicidal behavior and have resulted in substantially larger effect sizes than studies of singular risk factors (Franklin et al., 2016).

One population exhibiting elevated suicide risk that may particularly benefit from the application of these novel techniques comprises individuals who engage in non-suicidal self-injury (NSSI). NSSI is the direct, deliberate destruction of one’s body tissue performed without suicidal intent (Nock et al., 2006). The behavior is often carried out based on its interpersonal (e.g., interpersonal influence) and/or intrapersonal (e.g., affect regulation) functions (Nock et al., 2006). Although without suicidal intent, engagement in NSSI significantly increases one’s risk for subsequent engagement in suicidal behaviors (Hamza et al., 2012; Klonsky et al., 2013). Indeed, among individuals with a history of NSSI, a staggering percentage also has engaged in suicidal behavior (up to 70%; Brausch and Gutierrez, 2010; Cheung et al., 2013; Cloutier et al., 2012; Nixon, and Muehlenkamp, 2010; Nock et al., 2006; Paul et al., 2015). Despite the clear high-risk nature of this population, EDM techniques have yet to be employed to improve suicide risk prediction among individuals with a history of NSSI.

A recent meta-analysis of risk factors for suicide attempts (SA) among those with a history of NSSI found that NSSI features (i.e., frequency of engagement, number of methods employed) were the most potent predictors of SAs, second only to suicidal ideation (SI; Victor and Klonsky, 2014). This provides strong evidence that the characteristics of NSSI itself may be more important in quantifying suicide risk among self-injurers, compared to other clinical indices (e.g., depression, impulsivity, borderline personality disorder) considered important risk factors for suicidal behaviors in the general population (Victor and Klonsky, 2014). Overall, this research underscores the importance of confirming the relationships between NSSI characteristics previously demonstrated to be associated with SI and SA. Furthermore, it sheds light on the importance of exploring the potential risk associated with more varied and understudied facets of NSSI, as they may further augment our ability to predict SI and SA in this population.

Given the consistent findings of a risk relationship between NSSI and suicide-related outcomes, a small, but growing, body of research has begun to evaluate what features of this multi-faceted behavior may be most important to assess when quantifying risk for SI and SA. This research has identified the following NSSI features as potentially important: frequency (Paul et al., 2015), method of cutting (Victor and Klonsky, 2014), the experience of pain (Ammerman et al., 2016; Nock et al., 2006), medical severity (Burke et al., 2015), scarring (Burke et al., 2015), and engaging in NSSI alone as opposed to with others (Glenn and Klonsky, 2009). Research also has suggested that motivations (e.g., functions) for NSSI may be important predictors of SI and SA (Nock and Prinstein, 2005; Paul et al., 2015); however, this research has been somewhat mixed, likely due to variability in operationalizing NSSI functions. For example, one study examining the relationships between 17 specific NSSI functions and SI/SA found that most of the functions were predictive of SA, whereas only the interpersonal communication and anti-dissociation (i.e., feeling generation) functions were predictive of SI (Paul et al., 2015). In another study examining four classes of functions, as opposed to individual functions, only intrapersonal negative reinforcement (i.e., reducing negative affect) was associated with recent SA (Nock and Prinstein, 2005). These findings were further supported through research utilizing latent class analysis, which identified a subgroup of self-injurers characterized by high levels of intrapersonal functions as well as SI and SA (Klonsky and Olino, 2008). Generally, the small body of research examining NSSI functions and SI/SA suggests the importance of intrapersonal functions, yet, it remains unclear which intrapersonal functions are most important and the extent to which interpersonal functions also play a role. Thus, research is needed to clarify which functions may be most associated with SI and SA.

Despite the growing body of research examining the association between NSSI features and SI/SA, several NSSI features remain understudied (Victor and Klonsky, 2014). For example, no studies, to our knowledge, have directly examined whether NSSI location is associated with SI or SA. However, a recent study found that individuals with borderline personality disorder (BPD), a population at increased risk for SI/SA (Oldham, 2006), engaged in self-injury in locations that are more visible/exposed than self-injurers without BPD (Stroehmer et al., 2015), offering indirect support for the idea that location may be related to SI/SA. Additional features deserving further attention with respect to prediction of suicide-related outcomes are NSSI medical severity and scarring from NSSI (Burke et al., 2015), as well as time from urge to action, desire to cease NSSI, likelihood of engaging in future NSSI, and age of NSSI onset. Given the limited research examining these characteristics coupled with their promise in improving SI/SA predictive models, the current study aimed to investigate the importance of such NSSI features in the occurrence of suicide-related outcomes.

1.1. The Current Study

Although research has begun to utilize EDM techniques to improve the ability to predict suicidal behavior, no studies, to our knowledge, have applied these techniques to improve suicide risk prediction among individuals with a history of NSSI. Moreover, no studies have examined the previously identified NSSI characteristics simultaneously within a predictive model to determine which features are most salient to consider when performing suicide risk assessment. Such information would be useful to researchers and providers who currently have minimal guidance on which NSSI features among many may denote high suicide risk classification. Investigating these NSSI characteristics simultaneously is important given their significant shared variance. The current study aimed to identify which NSSI features are most important in predicting SI, suicide planning (SP), and SA, considering a total of 58 NSSI features. In order to be able to comment on the relative importance of NSSI features as compared to well-established predictors of SI, SP, and SA, we also included depressive symptoms and demographic variables in all predictive models, in addition to including SI and SP as indicators when predicting SA. Based on prior literature (Victor and Klonsky, 2014), we hypothesized that the NSSI characteristics of frequency and number of methods would emerge as important predictors of SI, SP, and SA. Additionally, we hypothesized that the intrapersonal functions of anti-suicide and anti-dissociation would also emerge as important predictors of SI, SP, and SA (Paul et al., 2016). We predicted that these NSSI features would emerge as important, even after considering depressive symptom severity in all models and SI and SP in the SA model.

2. Method

2.1. Participants and Procedures

Participants (N = 1082) were undergraduate students enrolled in a large, northeastern university recruited from psychology classes in exchange for class credit; participants completed all study procedures online. All procedures were reviewed and approved by the university Institutional Review Board. Participants were eligible for the study if they were able to complete study measures (i.e., read and speak proficiently in English, maintain normal or corrected vision) and if they were age 18 or older. Validity items were included in the online study to ensure data integrity. Participants who failed greater than 50% of these data integrity items were excluded from the analyses.

The final sample consisted of 359 (33.2%) participants who completed the online screening survey and reported a history of NSSI. There were no significant demographic differences between individuals who reported a history of NSSI and individuals who did not. Participants who reported a history of NSSI also reported elevated suicidal ideation, 90 (25.1%), as measured by the Beck Scale for Suicidal Ideation (BSS; Beck and Steer, 1991) compared to those who did not report a history of NSSI, 53 (7.4%), and this difference was statistically significant, X2(1) = 65.34, p < .001. Additionally, participants who reported a history of NSSI, 33 (9.2%), were significantly more likely than those without a history of NSSI, 7 (1.0%), to report having a suicide plan, X2(1) = 45.34, p < .001. Likewise, participants with a history of NSSI, 51 (14.2%), were significantly more likely to report a past suicide attempt compared to 15 (2.17%) participants without a history of NSSI, X2(1) = 61.31, p < .001. Demographic characteristics (age, gender, race) for the sample are reported in Table 1. Of the 359 participants who endorsed a history of NSSI, 14 participants evidenced a small portion of missing data (a total of 2% missingness among the 14 participants). Because the rate of missingness was low, we opted for single imputation using default methods from the mice package in R (Buuren and Groothuis-Oudshoorn, 2011). The NSSI characteristics of the final sample (n = 359) have been summarized in Supplementary Table 1. Among the final sample, 25.1% reported past week SI, 9.2% reported past week SP, and 14.2% reported a lifetime history of SA. Descriptive statistics were run to determine whether the demographic characteristics were significantly correlated with the study outcomes of SI, SP, and SA. Race was significantly associated with SI (X2(4, N = 359) = 17.33, p = .004)1, but was not associated with SP or SA.

Table 1.

Demographic and clinical characteristics of the NSSI sample (N = 359).

No SA History (n = 308)* SA History (n =51)*
Age (Mean, range) 20.40 (2.70) 20.38 (5.25)
Female gender (N, %) 229 (74.40) 40 (78.40)
Race/Ethnicity (N, %)
White 193 (62.70) 35 (68.60)
Black/African-American 43 (14.00) 4 (7.80)
Asian 39 (12.70) 4 (7.80)
Biracial 19 (6.20) 6 (11.80)
Other 14 (4.50) 2 (3.90)
Open in a new tab
*

Age statistics were calculated excluding participants who reported inconsistent values (i.e., values less than 0).

2.2. Measures

2.2.1. Predictor Variables

2.2.1.1. Demographics.

Age in years, gender, and race were included as indicators in all models.

2.2.1.2. Depression.

The Beck Depression Inventory-II (BDI-II; Beck et al., 1996) is a self-report questionnaire that measures depressive symptom severity. Participants rate each of the 21-items on a 4-point Likert Scale and higher scores signify greater depressive symptom severity. The BDI-II has evidenced strong test-retest reliability and convergent validity (Beck et al., 1996; Storch et al., 2004). The internal consistency in the current sample was excellent (α = 0.93).

2.2.1.3. NSSI forms and characteristics.

A modified version of the Deliberate Self-Harm Inventory (DSHI; Gratz, 2001) was used to assess 17 different forms of NSSI behavior. The DSHI was amended to ensure that participants did not report on any behaviors enacted with suicidal intent. If a participant endorsed a NSSI behavior, seven follow-up questions assessed the age at onset, frequency (lifetime and past year) and recency of behavior, years of engagement (not used in the current study), and whether the behavior has ever resulted in hospitalization or medical treatment. Additionally, participants were queried on 18 potential locations of their NSSI, answered with a yes/no response, a section that was added to the original DSHI. Example locations included arm/wrist, neck/throat, thigh, and lower leg/ankles. Previous studies have supported the psychometric characteristics of the DSHI in a university-student sample (Fliege et al., 2006; Gratz, 2001). Finally, the individuals’ behavioral forecast of future engagement in any form of NSSI was assessed through an additional item added to the DSHI (e.g., “If you were feeling bad in the future, how likely would you hurt yourself intentionally (without intending to kill yourself) as a way to cope?”), measured on a five-point Likert-type scale (“Not at all likely” to “Very likely”).

2.2.1.4. NSSI scarring.

Participants were asked to report on whether NSSI behaviors resulted in a visible mark or scar via an additional question added to the DSHI. If participants endorsed having a visible mark or scar from NSSI, participants were asked to report number of current scars.

2.2.1.5. NSSI subjective experiences.

The second section of the Inventory of Statements about Self-Injury (ISAS; Klonsky and Glenn, 2009) was used to assess additional characteristics associated with an individual’s subjective experience of their NSSI, including experience of pain, presence of others during NSSI, time elapsed from urge to action, and desire to stop self-injurious behavior. We adapted these questions to assess an individual’s overall subjective experience of NSSI (as opposed to their experience only in relation to their main form of NSSI as in the original ISAS). Questions regarding subjective pain and presence of others during NSSI had a multiple choice format (e.g., Yes, Sometimes, and No). Desire to stop self-injurious behavior was assessed with a yes/no response.

2.2.1.6. NSSI functions.

The third section of the ISAS (Klonsky and Glenn, 2009; Klonsky and Olino, 2008) was used to assess functions of NSSI behavior. This scale includes 39 self-report items that assess 13 functions associated with NSSI. Each function loads onto either an intrapersonal (e.g., affect regulation, self-punishment, anti-dissociation, distress, and anti-suicide) or an interpersonal factor (e.g., interpersonal boundaries, autonomy, peer bonding, interpersonal influence, sensation-seeking, strength, revenge, and self-care; Klonsky and Glenn, 2009). The intrapersonal factor (α = 0.85) and the interpersonal factor (α = 0.74) both demonstrated good internal consistency in this sample.

2.2.2. Outcome Variables

2.2.2.1. Suicidal ideation and suicide planning.

The Beck Scale for Suicidal Ideation (BSS; Beck and Steer, 1991), a 19-item self-report scale, was used to assess SI and SP within the past week. In the current study, we created a composite score of current (prior one-week) SI (14 items) and SP (4 items) based on corresponding items in the BSS. We excluded one item reflecting access to means, as this does not indicate SI or SP. We then categorized participants as having SI if they had a non-zero score on the SI items and categorized participants as having SP if they had a non-zero score on the SP items. Previous studies have determined that the BSS has good psychometric properties (Chioqueta and Stiles, 2006). The full BSS demonstrated good internal consistency in this sample (α = 0.79). After modification, the BSS-SI items and the BSS-SP items also each demonstrated adequate to good internal consistency (SI = 0.73; SP = 0.62).

2.2.2.2. Suicide attempts.

Suicide attempts were assessed using the following yes/no question, “Have you ever attempted to kill yourself?”

2.3. Data Analysis

In the current analyses, we employed three EDM (McArdle and Ritschard, 2013) methods: elastic net regression (Zou and Hastie, 2005), decision trees (DT; Therneau et al., 2017), and random forests (Breiman, 2001). To prevent selecting a model too complex for our data (overfitting), we used repeated 10-fold cross-validation to select a final model. Given the nonlinear (DT, random forests) and adaptive (elastic net) nature of the methods, as well as the large ratio of predictors (62 for SI and SP models; 64 for SA model) to sample size (359), variants of DTs, random forests, and elastic net were chosen that explicitly incorporate forms of cross-validation in order to control for the propensity to overfit and to perform variable selection as a way to reduce the predictor to sample size ratio.2

2.3.1. Elastic Net Regression.

In order to perform subset selection, we searched for methods beyond either backward or forward stepwise selection. The shortcomings of stepwise procedures have been sufficiently documented (e.g., Harrel, 2001), so no further detail is provided. Instead, detail regarding an alternative, more recently developed procedure is given. This procedure, elastic net regression (Zou and Hastie, 2005), is a form of regularized regression that shrinks coefficients to zero while also overcoming problems when predictors are highly correlated. Elastic net regression can be seen as an extension of linear regression, but with a penalty added to induce sparsity, or more beta coefficients set to zero than would be the case with ordinary least squares (OLS) regression.

Elastic net regression adds penalties to each of the regression coefficients in a linear regression model. By increasing the penalty, each coefficient is driven towards zero. Through the use of cross-validation, running the model on a subset of the sample, then testing this model on a holdout sample, a final model (with a single penalty value) is chosen that is thought to demonstrate the best generalizability. In many cases, this final model has regression coefficients estimated as zero, thus performing subset selection. Given the propensity for this procedure to result in estimates biased towards zero (Hastie et al., 2009), we first identified the nonzero paths, and then ran an unconstrained model (linear regression with no penalties) using only these variables in the model. This is an extension of the relaxed lasso (Meinshausen, 2007). Although the elastic net does not estimate p-values to test the significance of each predictor, using cross-validation to find regression coefficients with non-zero parameter estimates can be thought of as a form of identifying which predictors are “important” (e.g., Laurin et al., 2016). Using repeated cross-validation to choose a final model attempts to derive results with an eye on generalizing to alternate samples. We used the glmnet package (Friedman et al., 2010), interfaced through the caret package (Kuhn, 2008), to run the elastic net regression models, with tuning parameters of both the penalty and alpha (mixing between ridge and lasso penalties) in the R statistical environment (R Core Team, 2016). In all elastic net regression models, we selected the simplest model within 1 standard error of the minimum fit.

2.3.2. Decision Trees and Random Forests.

So as not to limit our hypothesized relationship between predictors and outcomes to linear models, we also used a form of DTs. DTs are one of the most popular methods that fall under the umbrella of EDM, and they form the basis for a number of more flexible and advanced methods. DTs can be thought of as simple nonparametric regression models for use with both continuous and categorical outcomes. DTs select a subset of the predictors to split on, creating binary splits at cutoff values of the selected covariates. This allows for easily interpretable tree structures that allow for the inclusion of nonlinear relationships and interactions between predictors.

To create tree models, we used the rpart package (Therneau et al., 2017) in the R statistical environment (R Core Team, 2017); the algorithm builds binary trees for both classification and regression models. One of the most popular DT algorithms, it is based on the commercial software implementation of Classification and Regression Trees (CART; Breiman et al., 1984). We interfaced rpart through the caret package, using repeated cross validation to select a final model among 50 complexity parameters. Similar to the elastic net regression models, we selected the simplest model within 1 standard error of the minimum fit.

Finally, given the propensity for parameter instability in DTs (e.g., Berk, 2008; Hastie et al., 2009), we included random forests (Breiman, 2001) as a way to derive a more stable estimate of variable importance using tree models. Random forests can be thought of as creating a large number of individual decision trees (e.g., 100), with a bootstrap (or subset) of the sample and a subset of the predictors used to create each tree. After all of the trees are created, predictions are made by aggregating predictions across the individual trees. Given the small sample size, we do not assess the performance of random forests; instead, we only include the variable importance, which quantifies the reduction in prediction accuracy that results from removing each predictor from the set of trees, scaled relative to the most important predictor (given a value of 100). Within the R statistical environment, the caret package was used as a wrapper around the randomForest (Liaw and Wiener, 2002) package in order to facilitate testing using repeated cross-validation with 3,000 trees and three values of the number of random predictors selected.

Analysis scripts to run both elastic net regression and DTs are available at the second author’s website.3

3. Results

3.1. Suicidal Ideation

3.1.1. Elastic Net Regression.

Across the 100 values of penalty tested, we chose the model with the largest penalty within one standard error of the lowest mean squared error (MSE). This model (alpha=0.25, penalty=0.35) had two non-zero regression coefficients: the anti-suicide function of NSSI and depression. In re-running the model using OLS, the standardized coefficients were as follows: 0.72 for the anti-suicide function and 0.88 for depression. Due to high correlations among predictors (rs up to .54), we did not run a linear regression model for comparison. We used the area under the receiver operating characteristic curve (AUC) to calculate model performance. This metric balances both sensitivity and specificity, which results in a better representation of performance in comparison to Accuracy when there is class imbalance. This model had an AUC of 0.85.

3.1.2. Decision Trees.

The resultant tree is displayed in Figure 1. This tree selected some of the same variables as the elastic net analyses: depression, anti-suicide function, affective regulation function, and NSSI social context (e.g., engaging alone or with others). The five subgroups of participants are as follows: (1) those who reported a depression score of 20 or more and the anti-suicide function values of 0.5 or more had a high (86%) probability of SI, (2) those who reported a depression score of 20 or more, anti-suicide function values of less than 0.5, affect regulation function values of less than 3.5, and reported engaging in NSSI alone sometimes or all the time had high (82%) probability of SI, (3) those who reported a depression score of 20 or more, anti-suicide function values of less than 0.5, affect regulation function values of less than 3.5, and did not report engaging in NSSI alone had a low (22%) probability of SI, (4) those who reported a depression score of 20 or more, anti-suicide function values of less than 0.5, and affect regulation function values of 3.5 or more had low (21%) probability of SI, and (5) those who reported depression scores less than 20 had a low (12%) probability of SI. This model had an AUC of 0.77.

Figure 1.

Figure 1

Open in a new tab

Decision Tree for Suicidal Ideation

Note. Each node shows the predicted class (“yes” or “no” SI), the predicted probability of belonging to the “yes” class, and the percentage of observations in each node.

3.1.3. Random Forests.

The relative variable importance value from the Random Forest analyses are depicted in Table 2. Depression demonstrated greatest influence on SI compared to all other variables, followed by the anti-suicide function of NSSI and the anti-dissociation function of NSSI.

Table 2.

Random Forest top ten most important variables for suicidal ideation, suicide plans, and suicide attempt history

Suicidal Ideation Suicide Plans Suicide Attempt History
Variable Importance Value Variable Importance Value Variable Importance Value
Depression 100.00 Depression 100.00 NSSI medical treatment 100.00
Anti-suicide function 39.60 Anti-suicide function 47.45 Anti-suicide function 97.50
Anti-dissociation function 18.10 Age 40.37 Number NSSI scars 96.85
Future NSSI likelihood 16.72 Self-punishment function 37.78 Anti-dissociation function 76.65
Gender 16.13 Future NSSI likelihood 36.87 Suicide plan 70.41
Revenge function 13.50 Desire to cease NSSI 36.07 Interpersonal influence function 70.00
Autonomy function 12.23 Toughness function 34.62 Marking distress function 58.22
Self-carve words method 11.15 Gender 33.65 Time between urge and NSSI 57.20
Self-care function 9.28 Number NSSI scars 31.13 Self-cutting method 56.27
Self-punishment function 8.94 Interpersonal influence function 29.30 NSSI on hands/knuckles/fingers 55.32
Open in a new tab

3.2. Suicide Plan

3.2.1. Elastic Net Regression.

Across the 100 values of penalty tested, we chose the model with the largest penalty within one standard error of the lowest mean squared error (MSE). This model (alpha=0.75, penalty=0.03) had six non-zero regression coefficients: number of NSSI scars, anti-suicide function of NSSI, revenge function of NSSI, desire to cease NSSI, NSSI likelihood, and depression. In re-running the model using OLS, the standardized coefficients were as follows: 0.78 for number of NSSI scars, 0.72 for anti-suicide function, 3.05 for revenge function, 12.15 for desire to cease NSSI, 0.52 for NSSI likelihood, and 0.89 for depression. Due to high correlations among predictors (rs up to .54), we did not run a linear regression model for comparison. This model had an AUC of 0.89.

3.2.2. Decision Trees.

The resultant tree is displayed in Figure 2. This tree selected some of the same variables as the elastic net analyses: depression, anti-suicide function, and toughness function. The four subgroups of participants are as follows: (1) those who reported a depression score of 26 or more, anti-suicide function values of 0.5 or more, and a toughness function value of less than 1.5 had a high (76%) probability of SP, (2) those who reported a depression score of 26 or more, anti-suicide function value of 0.5 or more, and toughness function value of 1.5 or more had a low (23%) probability of SP, (3) those who reported a depression score of 26 or more and anti-suicide function values of less than 0.5 had a low (11%) probability of SP, and (5) those who reported depression scores less than 26 had a very low (4%) probability of SP. This model had an AUC of 0.77.

Figure 2.

Figure 2

Open in a new tab

Decision Tree for Suicide Plan

Note. Each node shows the predicted class (“yes” or “no” SP), the predicted probability of belonging to the “yes” class, and the percentage of observations in each node.

3.2.3. Random Forests.

The relative variable importance value from the Random Forest analyses are depicted in Table 2. Depression demonstrated greatest influence on SP compared to all other variables, followed by the anti-suicide function of NSSI and participant age.

3.3. Suicide Attempts

3.3.1. Elastic Net Regression.

For elastic net regression, we chose the model with the largest penalty within one standard error of the lowest binomial deviance. This model (alpha=0.25, penalty=0.32) had five non-zero regression coefficients: number of NSSI scars, history of medical treatment due to NSSI, anti-suicide function of NSSI, anti-dissociation function of NSSI, and history of suicide plan. In re-running the model using logistic regression, the odds ratios were as follows: 1.18 for number of NSSI scars, 4.20 for history of medical treatment due to NSSI, 1.14 for anti-suicide function of NSSI, 1.26 for anti-dissociation function of NSSI, and 3.35 for history of suicide plan. This model had an AUC value of 0.75.

3.3.2. Decision Trees.

For the decision tree model, the resultant tree is displayed in Figure 3. The final model selected five variables, three of which also were presented in the elastic net results: the anti-suicide function of NSSI, number of NSSI scars, history of medical treatment due to NSSI, the toughness function of NSSI, and depressive symptoms. The seven resultant subgroups and their probability of SA are as follows: (1) those with values less than 1.5 on the anti-suicide function of NSSI exhibited a very low (6%) probability of SA, (2) those with values of 1.5 or more on the anti-suicide function of NSSI, reported less than 20 NSSI scars, no history of medical treatment due to NSSI, and values of greater than or equal to 1.5 on the toughness function exhibited a low (9%) probability of SA, (3) those with values of 1.5 or more on the anti-suicide function of NSSI, reported less than 20 NSSI scars, no history of medical treatment due to NSSI, values of less than or equal to 1.5 on the toughness function, and values of greater than 30 on depression exhibited a very low (2%) probability of SA, (4) those with values of 1.5 or more on the anti-suicide function of NSSI, reported less than 20 NSSI scars, no history of medical treatment due to NSSI, values of less than or equal to 1.5 on the toughness function, and values of less than 22 on depression exhibited a low (26%) probability of SA, (5) those with values of 1.5 or more on the anti-suicide function of NSSI, reported less than 20 NSSI scars, no history of medical treatment due to NSSI, values of less than or equal to 1.5 on the toughness function, and values of less than 30 on depression but greater than 22 exhibited a high (75%) probability of SA, (6) those with values of 1.5 or more on the anti-suicide function of NSSI, reported less than 20 NSSI scars, and a history of medical treatment due to NSSI exhibited a high (78%) probability of SA, and finally, (7) those with values of 1.5 or more on the anti-suicide function of NSSI and reported 20 or more NSSI scars exhibited a high (72%) probability of SA. This model had an AUC value of 0.74.

Figure 3.

Figure 3

Open in a new tab

Decision Tree for Suicide Attempts

Note. Each node shows the predicted class (“yes” or “no” SA), the predicted probability of belonging to the “yes” class, and the percentage of observations in each node.

3.3.3. Random Forests.

The relative variable importance value from the Random Forest analyses are depicted in Table 2. Medical treatment for NSSI behavior demonstrated greatest influence on SA compared to all other variables, followed by the anti-suicide function of NSSI and the number of scars due to NSSI.

4. Discussion

The current study aimed to utilize EDM techniques to inform suicide risk prediction among individuals with a history of NSSI by simultaneously examining numerous NSSI characteristics as risk factors for SI, SP, and SA. Although research has demonstrated NSSI as a high-risk behavior for the occurrence of suicidal ideation and behavior (e.g., Hamza et al., 2012), little research has attempted to determine which NSSI features may be most important to assess when conducting suicide risk assessments in this population. Furthermore, such research has relied heavily on designs that preclude the simultaneous examination of a large number of factors; the current study addressed this gap through the implementation of EDM statistical analysis techniques. We hypothesized that NSSI frequency and number of methods would emerge as important predictors of SI, SP and SA, and, further, that the intrapersonal functions of anti-suicide and anti-dissociation would also emerge as important predictors (Paul et al., 2016), even after considering depressive symptom severity for all models and SI and SP in the SA model. These hypotheses were partially supported.

In predicting the occurrence of SI and SP, decision trees, elastic net, and random forests results identified participants’ depressive symptoms and the endorsement of the anti-suicide function of NSSI as the two most important predictors. In the DT analysis, the first split occurred between those endorsing a score of less than 20 and a score of 20 or more on depressive symptoms, with the latter group at increased risk for reporting current SI and SP. Further, among those indicating high depressive symptom severity, individuals endorsing a particularly high identification with the NSSI anti-suicide function experienced particularly elevated probability for current SI and SP, whereas those endorsing lower identification with the NSSI anti-suicide function were less likely to report experiencing SI and SP. Given past research demonstrating the strong relationship between depressive symptoms and SI (e.g., Kandel et al., 1991), it is not surprising that individuals who report severe depressive symptoms would be most likely to experience current SI and SP. Relatedly, findings with regards to the anti-suicide function are consistent with past research demonstrating that the anti-suicide function of NSSI has one of the strongest relationships with SI as compared to other functions of the behavior (Victor et al., 2015).

Findings from the current study suggest that individuals self-reporting that they are experiencing severe depressive symptoms and have a history of engaging in NSSI as a way to resist suicide are at the highest risk of experiencing SI and engaging in SP. These results extend previous research by demonstrating that there is an additive effect of the anti-suicide function among those experiencing depression. It is possible that these individuals may be using NSSI to express suicidal thoughts and plans without risking death (Klonsky, 2007) and that they believe it is effectively serving this function, thus increasing the likelihood of continuing the behavior. Self-injurers who experience SI and engage in SP may engage in NSSI to help to reduce their experience of SI in the short-term, similar to the manner in which self-injurers report that engagement in NSSI temporarily reduces negative affect (Klonsky, 2007). Given this, it is then possible that the negative reinforcement involved in ameliorating one’s experience of SI and SP may, in turn, lead to greater NSSI engagement. Future research should explore this association further.

Whereas depression was an important indicator of SI and SP, it was identified as important in predicting SA by only one of the three EDM methods employed in this study. Rather, NSSI features were selected by all three methods to best predict SA. Even with the inclusion of SI and SP in the models predicting SA, the variables that were identified as important across all three EDM methods were three NSSI features: anti-suicide function of NSSI, number of NSSI scars, and history of medical treatment due to NSSI. The DT analysis suggests that the anti-suicide function of NSSI was the most important predictor of history of SA, demonstrating a first split between those endorsing low identification with the anti-suicide function and those endorsing high identification with the function. Results suggest that participants endorsing high identification with the anti-suicide function were over five times more likely to report a SA than those in the low endorsement group. Given that the anti-suicide function of NSSI was identified as an important predictor of SAs in all three analysis methods, NSSI may present as a vehicle for individuals to express their suicidal thoughts while resisting the urge to actually act on them (Klonsky, 2007). However, it may also be a way for those individuals to practice engaging in intentional self-injury, which may, in turn, make it more likely that they will carry out suicidal behavior through habituation to the fear and pain involved in carrying out self-directed lethal acts (Joiner, 2005). Notably, given that the current study is cross-sectional, it is important to consider that the direction of this relationship may be otherwise. Indeed, it is possible that after attempting suicide, individuals aiming to avoid engaging in a repeat attempt may begin to engage in NSSI for anti-suicide motivations.

Findings further show that among individuals reporting high identification with the anti-suicide function, reporting 20 or more scars from the behavior increased probability of reporting a SA history by almost threefold as compared reporting 20 or fewer scars, highlighting the role of NSSI scarring as a significant correlate of SAs. Indeed, the elastic net, DT, and random forests results support the number of NSSI scars as an important indicator of SA probability. To date, there has been limited research on NSSI scarring in the empirical literature. However, one empirical study did link NSSI scarring with SI and SA (Burke et al., 2016). Finally, results suggested that among individuals with high anti-suicide function endorsement who have less than 20 NSSI scars, but have obtained medical treatment for NSSI in the past also have increased SA probability (78%). The importance of medical treatment for NSSI in the occurrence of SA, which was demonstrated across all statistical methods, is in line with past research. These findings demonstrating the importance of NSSI scarring and medical severity are congruent with the prevailing theory (Joiner, 2005) that the experience of painful and provocative events (e.g., engagement in severe NSSI) may lead to an increase in one’s acquired capability to carry out suicidal behavior (Joiner, 2005). Past research has supported this theory, finding that more severe NSSI behavior, as defined by greater NSSI frequency (Paul et al., 2015) and more NSSI methods (Victor and Klonsky, 2014), is related to SA; the current findings add to this body of research suggesting that having extensive scarring from NSSI and receiving medical attention for NSSI may be other important indicators of NSSI severity (Burke et al., 2015). Indeed, it is plausible that those who engage in NSSI to the point of wounds producing scars or requiring medical intervention may have habituated to the pain involved in enacting lethal self-injury more than those engaging in frequent NSSI but resulting in fewer (or no) scars and requiring no medical treatment, thus, increasing their likelihood of SA.

Across all three SA models, which included SI and SP as indicators, SI was not identified as an important predictor. SP was identified as important in one model predicting SA, but less important relative to the aforementioned NSSI features. Our results provide further support for Joiner’s theory that holds acquired capability as the mechanism that meaningfully differentiates between those experiencing SI or engaging in SP and those who actually act on those thoughts/plans. The current study’s findings strongly suggest the importance of collecting detailed information about NSSI history when ascertaining suicide risk, as it may provide important insight into suicidal behavior beyond SI and SP.

It is important to consider that the three methods of EDM employed in the current study, elastic net, DT, and random forests, resulted in slightly varying findings. These results are likely divergent due to the fact that whereas elastic net will only find additive and linear effects, DT and random forests can also capture nonlinear effects and interactions (Hastie et al., 2009). Indeed, whereas in predicting SA, the anti-suicide function evidenced the lowest odds-ratio of important predictors in a logistic regression (OR = 1.03), it emerged as the top one or two most important features in the DT and random forests models. This outcome is likely due to this feature’s nonlinear relationship with SA, as well as its interaction with, or additive effects on other variables such as scarring secondary to NSSI and necessitating medical treatment for NSSI (as opposed to its independent relationship with SA).

4.1. Limitations and Future Directions

Limitations of the current study should be noted. First, the study is cross-sectional in design, limiting our ability to draw conclusions about the temporal direction of our findings. There have been foundational studies demonstrating that NSSI prospectively predicts attempted suicide more strongly than other suicide risk factors (Asarnow et al., 2011; Wilkinson et al., 2011). Future prospective studies should employ EDM techniques to determine whether the important NSSI characteristics identified in this study remain important when considering other suicide risk factors outside those included in the current study (e.g., hopelessness). Second, our outcomes of interest were measured across divergent timeframes: SI and SP were measured over the prior one week, whereas SA and NSSI were assessed across the lifetime. Third, the current study employed a one-item measure of lifetime suicide attempts. Given that this item did not specify intent or perceived consequences of the behavior, it is possible that we are overestimating suicidal behavior in our sample (Millner et al., 2015). Fourth, we did not examine single versus multiple attempter status in the current study. Given research suggesting that risk factors for single and multiple suicide attempts may be distinct (e.g., Esposito et al., 2003), future research should examine whether the predictive models in the current study apply to both groups of attempters. Fifth, the current study employed amended versions of the DSHI, ISAS, and BSS, which require further psychometric examination. Sixth, the sample employed was composed of a large proportion of females (75%). Given differences in NSSI characteristics and suicidal behavior rates between genders (Andover et al., 2010; O’Connor and Nock, 2014), it will be important for future research to examine whether the results can be replicated in an evenly-proportioned sample of males and females. Finally, given that the undergraduate sample employed in the current study may not be representative of all individuals who engage in NSSI, future research should replicate these findings in community and clinical samples.

Despite the significant limitations outlined, our findings reveal that NSSI functions, scarring, and medical lethality may be more important to assess than commonly regarded NSSI severity indices (e.g., number of methods, frequency) when ascertaining risk for SI, SP, and SA. Our findings also have important implications for the development of suicide risk algorithms. Given our analysis primarily included NSSI characteristics as indicators (in addition to demographics and depressive symptoms), without accounting for any other clinical or diagnostic indicators, our findings suggest that fairly accurate decisions about suicide risk can be made in a self-injuring sample with primarily NSSI feature information alone. Thus, the current findings highlight the potential clinical importance of collecting nuanced information about self-injurers’ experience of their NSSI. Perhaps more importantly, the current findings suggest that including specific information about NSSI history may augment the predictive accuracy of extant suicide risk algorithms (e.g., Barak-Corren et al., 2016).

Supplementary Material

1

Acknowledgements:

Taylor A. Burke was supported by a National Science Foundation Graduate Research Fellowship. Ross Jacobucci was supported by the National Institute of Aging T32AG0037. Brooke A. Ammerman was supported by National Research Service Award F31MH107156 from the National Institute of Mental Health. Lauren B. Alloy was supported by National Institute of Mental Health grant MH101168.

Footnotes

1

Post-hoc follow-up analyses were conducted to determine the effect of ethnicity on SI using Fisher’s exact chi-square tests. White, South Asian, Biracial, and participants who identified as “other race” endorsed SI more frequently than Black participants. Additionally, South Asian, Biracial, and participants who identified as “other race” endorsed SI more frequently than East Asian participants. Results from these post-hoc analyses are available from the corresponding author. As cell counts were low, caution is advised when interpreting these results

2

It is important to distinguish the proposed EDM analyses from Latent Class Analysis (LCA; McCutcheon, 1987). Although some studies have employed LCA to identify combinations of NSSI risk factors that may best differentiate high and low suicide risk groups (e.g., Klonsky and Olino, 2008), LCA is unable to offer information on the relative importance of specific factors in suicide-related outcomes, whether there are non-linear relationships between factors and outcomes, or whether there are particularly important interactions between factors in predicting an outcome.

Referencess

  1. Ammerman BA, Burke TA, Alloy LB, McCloskey MS, 2016. Subjective pain during NSSI as an active agent in suicide risk. Psychiatry Res. 236, 80–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ammerman BA, Jacobucci R, Kleiman EM, Uyeji LL, McCloskey MS, In Press. The relationship between NSSI age of onset and severity of self-harm. Suicide Life-Threat. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Andover MS, Primack JM, Gibb BE, Pepper CM, 2010. An examination of non-suicidal self-injury in men: do men differ from women in basic NSSI characteristics?. Arch. Suicide Res 14, 79–88. [DOI] [PubMed] [Google Scholar]
  4. Anestis MD, Khazem LR, and Law KC, 2015. How many times and how many ways: The impact of number of nonsuicidal self-injury methods on the relationship between nonsuicidal self-injury frequency and suicidal behavior. Suicide Life Threat. Behav 45, 164–177. [DOI] [PubMed] [Google Scholar]
  5. Asarnow JR, Porta G, Spirito A, Emslie G, Clarke G, Wagner KD, Vitiello B, Keller M, Birmaher B, McCracken J, Mayes T, 2001. Suicide attempts and nonsuicidal self-injury in the treatment of resistant depression in adolescents: Findings from the TORDIA study. J. Am. Acad. Child Adolesc. Psychiatry 50, 772–781. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Barak-Corren Y, Castro VM, Javitt S, Hoffnagle AG, Dai Y, Perlis RH, Nock MK, Smoller JW, Reis BY, 2017. Predicting suicidal behavior from longitudinal electronic health records. Am. J. Psychiatry 174, 154–162. [DOI] [PubMed] [Google Scholar]
  7. Beck AT, Steer RA, Brown GK, 1996. Manual for the Beck Depression Inventory, Second Edition (BDI-II). San Antonio, TX: Psychological Association. [Google Scholar]
  8. Berk RA, 2008. Statistical learning from a regression perspective. Springer Science and Business Media; New York. [Google Scholar]
  9. Breiman L, Friedman JH, Olshen RA, Stone CJ, 1984. Classification and regression trees. New York, Chapman and Hall. [Google Scholar]
  10. Burke TA, Stange JP, Hamilton JL, Cohen JN, O’Garro-Moore J, Daryanani I, Abramson LY, Alloy LB, 2015. Cognitive and emotion-regulatory mediators of the relationship between behavioral approach system sensitivity and nonsuicidal self-injury frequency. Suicide Life Threat. Behav 45, 495–504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Buuren S, & Groothuis-Oudshoorn K, 2011. mice: Multivariate imputation by chained equations in R. J. Stat. Softw 45, 1–67. [Google Scholar]
  12. Curtin SC, Warner M, and Hedegaard H, 2016. Increase in suicide in the United States, 1999–2014. NCHS data brief, no 241. Hyattsville, MD: National Center for Health Statistics. [PubMed] [Google Scholar]
  13. Esposito C, Spirito A, Boergers J, Donaldson D, 2003. Affective, behavioral, and cognitive functioning in adolescents with multiple suicide attempts. Suicide Life Threat. Behav 33, 389–399. [DOI] [PubMed] [Google Scholar]
  14. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, Musacchio KM, Jaroszewski AC, Chang BP, Nock MK, 2017. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychol. Bull 143, 187–232. [DOI] [PubMed] [Google Scholar]
  15. Friedman JH, Hastie T, Tibshirani R, 2010. glmnet: lasso and elastic-net regularized generalized linear models. URL http://CRAN.R-project.corg/package=glmnet.Rpackageversion,1-1.
  16. Glenn CR, Klonsky ED, 2011. Prospective prediction of nonsuicidal self-injury: A 1-year longitudinal study in young adults. Behav. Ther 42, 751–762. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Himber J, 1994. Blood rituals: Self-cutting in female psychiatric inpatients. Psychotherapy. 31, 620–631. [Google Scholar]
  18. Kandel DB, Raveis VH, Davies M, 1991. Suicidal ideation in adolescence: Depression, substance use, and other risk factors. J. Youth Adolesc 20, 289–309. [DOI] [PubMed] [Google Scholar]
  19. Kessler RC, Stein MB, Petukhova MV, Bliese P, Bossarte RM, Bromet EJ, Fullerton CS, Gilman SE, Ivany C, Lewandowski-Romps L, Bell AM, 2017. Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Service members (Army STARRS). Mol. Psychiatry 22, 544–551. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kessler RC, Warner CH, Ivany C, Petukhova MV, Rose S, Bromet EJ, Brown M, Cai T, Colpe LJ, Cox KL and Fullerton CS, Fullerton CS, 2015. Predicting suicides after psychiatric hospitalization in US Army soldiers: The Army Study to Assess Risk and Resilience in Service members (Army STARRS). JAMA Psychiatry. 72, 49–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Klonsky DE The function of deliberate self-injury: A review of the evidence. Clinical Psychol. Rev 2007; 27: 226–239. [DOI] [PubMed] [Google Scholar]
  22. Klonsky ED, May AM, Glenn CR, 2013. The relationship between nonsuicidal self-injury and attempted suicide: Converging evidence from four samples. J. Abnorm. Psychol 122, 231–237. [DOI] [PubMed] [Google Scholar]
  23. Kuhn M, 2008. Caret package. J Stat. Softw 28, 1–26.27774042 [Google Scholar]
  24. Laurin C, Boomsma D, Lubke G, 2016. The use of vector bootstrapping to improve variable selection precision in Lasso models. Stat. Appl. Genet. Mol. Biol 15, 305–320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Liaw A, Wiener M, 2002. Classification and regression by randomForest. R news, 2, 18–22. [Google Scholar]
  26. Lundh L, Karim J, Quilisch E, 2007. Deliberate self-harm in 15-year-old adolescents: A pilot study with a modified version of the Deliberate Self-Harm Inventory. Scand. J. Psychol 48, 33–41. [DOI] [PubMed] [Google Scholar]
  27. McCutcheon AL (Ed.)., 1987. Latent class analysis (Vol. 64). Thousand Oaks, CA: Sage. [Google Scholar]
  28. Meinshausen N, 2007. Relaxed lasso. Comput. Stat. Data Anal 52, 374–393. [Google Scholar]
  29. Nock MK, 2009. Why do people hurt themselves? New insights into the nature and functions of self-injury. Curr. Dir. Psychol. Sci 18, 78–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Millner AJ, Lee MD, Nock MK, 2015. Single-item measurement of suicidal behaviors: validity and consequences of misclassification. PloS one, 10, e0141606. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Nock MK, Joiner TE Jr., Gordon KH, Lloyd-Richardson E, Prinstein MJ, 2006. Non-suicidal self-injury among adolescents: Diagnostic correlates and relation to suicide attempts. Psychiatry Res. 144, 65–72. [DOI] [PubMed] [Google Scholar]
  32. Nock MK, Prinstein MJ, 2005. Contextual features and behavioral functions of self-mutilation among adolescents. J. Abnorm. Psychol 114, 140–146. [DOI] [PubMed] [Google Scholar]
  33. O’Connor RC, Nock MK, 2014. The psychology of suicidal behaviour. Lancet Psychiatry. 1, 73–85. [DOI] [PubMed] [Google Scholar]
  34. Oldham JM, 2006. Borderline personality disorder and suicidality. Am. J. Psychiatry 163, 20–26. [DOI] [PubMed] [Google Scholar]
  35. Paul E, Tsypes A, Eidlitz L, Ernhout C, Whitlock J, 2015. Frequency and functions of non-suicidal self-injury: Associations with suicidal thoughts and behaviors. Psychiatry Res. 225, 276–282. [DOI] [PubMed] [Google Scholar]
  36. Quinlan JR, 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo. [Google Scholar]
  37. R Core Team. R: A language and environment for statistical computing [Computer software manual]. 2016. Vienna, Austria: Retrieved from http://www.R-project.org/ [Google Scholar]
  38. Rosen PM, Heard KV, 1995. A method for reporting self-harm according to level of injury and location on the body. Suicide Life Threat. Behav 25, 381–385. [PubMed] [Google Scholar]
  39. Sornberger MJ, Heath NL, Toste JR, McLouth R, 2012. Nonsuicidal self-injury and gender: Patterns of prevalence, methods, and locations among adolescents. Suicide Life Threat. Behav 42, 266–278. [DOI] [PubMed] [Google Scholar]
  40. Stroehmer R, Edel MA, Pott S, Juckel G, Haussleiter IS, 2015. Digital comparison of healthy young adults and borderline patients engaged in non-suicidal self-injury. Ann. Gen. Psychiatry 14, 47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Storch EA, Roberti JW, Roth DA, 2004. Factor structure, concurrent validity, and internal consistency of the Beck Depression Inventory-Second Edition in a sample of college students. Depress. Anxiety 19, 187–189. 10.1002/da.20002 [DOI] [PubMed] [Google Scholar]
  42. Therneau T, Atkinson B, Ripley B, 2017. rpart: Recursive partitioning and regression rrees. R package version 4.1–11. [Google Scholar]
  43. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE Jr., 2010. The interpersonal theory of suicide. Psychol. Rev 117, 575–600. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Victor SE, Styer D, Washburn JJ, 2015. Characteristics of nonsuicidal self-injury associated with suicidal ideation: Evidence from a clinical sample of youth. Child Adolesc. Psychiatry Ment. Health 9, 20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Victor SE, Klonsky ED, 2014. Correlates of suicide attempts among self-injurers: A meta-analysis. Clin. Psychol. Rev 34, 282–297. [DOI] [PubMed] [Google Scholar]
  46. Wilkinson P, Kelvin R, Roberts C, Dubicka B, Goodyer I, 2011. Clinical and psychosocial predictors of suicide attempts and nonsuicidal self-injury in the Adolescent Depression Antidepressants and Psychotherapy Trial (ADAPT). Am. J. Psychiatry 168, 495–501. [DOI] [PubMed] [Google Scholar]
  47. Whitlock J, Muehlenkamp J, Purington A, Eckenrode J, Barreira P, Baral Abrams G, Marchell T, Kress V, Girard K, Chin C Knox K, 2011. Nonsuicidal self-injury in a college population: General trends and sex differences. J. Am. Coll. Health 59, 691–698. [DOI] [PubMed] [Google Scholar]
  48. Whitlock J, Muehlenkamp J, Eckenrode J, Purington A, Abrams GB, Barreira P, Kress V, 2013. Nonsuicidal self-injury as a gateway to suicide in young adults. J. Adolesc. Health 52, 486–492. [DOI] [PubMed] [Google Scholar]
  49. Whitlock J, Eckenrode J, Silverman D, 2006. Self-injurious behaviors in a college population. Pediatrics. 117, 1939–1948. [DOI] [PubMed] [Google Scholar]
  50. Zou H, Hastie T, 2005. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B Stat. Method 67, 301–320. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1043244-supplement-1.pdf (190.3KB, pdf)

RESOURCES