Development and Validation of Empirically Derived Frequency Criteria for NSSI Disorder Using Exploratory Data Mining

Abstract

Research suggesting non-suicidal self-injury (NSSI) may belong in a distinct diagnostic category has led to the inclusion of NSSI Disorder in the DSM-5 section for future study. There has been limited research, however, examining the validity of criterion A (the frequency criterion). The current study aimed to examine the validity of the frequency criterion of NSSI Disorder through the use of an Exploratory Data Mining method, Structural Equation Modeling Trees, as a way to determine a NSSI frequency that optimally discriminates pathological NSSI from normative behavior among undergraduate students (n = 3,559), 428 who engaged in NSSI in the previous year. The model included psychopathology symptomology found to be comorbid with NSSI and cognitive-affective deficits commonly associated with NSSI. Results demonstrated a first split between individuals with 0 and 1 act of NSSI in the past year, as was expected. Among individuals with 1 or more previous acts, the optimal split was between those with 5 and 6 NSSI acts in the past year. Results from the current study suggest that individuals with 6 acts of NSSI in past year, compared to those with 5 acts or less, may represent a more severe group of self-injurers. These individuals reported higher levels of related psychopathology symptomology and cognitive-affective deficits, in addition to decreased quality of life. Findings have potential implications for the proposed frequency criteria of NSSI Disorder and how pathological NSSI is characterized.

Non-suicidal self-injury (NSSI) is the direct, intentional destruction of one’s own body tissue without the intent to die (Klonsky & Muehlenkmap, 2007; Nock, 2009) and is a major public health concern. The behavior occurs in up to 4–6% of adults (Klonsky, 2011; Swannell, Martin, Page, Hasking, & St. John, 2014) and 12–18% of adolescents and young adults (Claes, Luyckx, & Bijttebier, 2014; Kuentzel et al., 2012; Muehlenkamp, Claes, Havertape, & Plener, 2012; Taliaferro & Muehlenkamp, 2015; Whitlock, Eckenrode, & Silverman, 2006) in community samples. Previous research on NSSI has demonstrated its associations with several adverse effects, including interpersonal difficulties (Adrian, Zeman, Erdley, Lisa & Sim, 2011; Tatnell, Kelada, Hasking, & Martin, 2014), cognitive-affective deficits (Andover & Morris, 2014; Bresin, 2014; Davis et al., 2014; Plener, Schumacher, Munz & Groschwitz, 2015), comorbid psychopathology (Braga & Goncalves, 2014; Bentley, Cassiello-Robbins, Vittorio, Saur-Zavala, & Barlow, 2015; Kerr & Muehlenkamp, 2010), and increased risk for suicide (Andover & Gibb, 2010; Klonsky, May, & Glenn, 2013; Whitlock et al., 2013).

Although mentioned as a symptom of Borderline Personality Disorder (BPD) in previous versions of the DSM (e.g., DSM-IV-TR; American Psychiatric Association [APA], 2000), recent research has begun to shed light on the nature of NSSI outside of the context of BPD. NSSI has been found to occur in individuals without BPD (Andover et al., 2005; Briere & Gil, 1998; Muehlenkamp et al., 2011; Selby et al., 2011) and across of range of psychological disorders (Glenn & Klonsky, 2011; Kerr & Muehlenkamp, 2010; Klonsky, 2011). Furthermore, research has suggested that NSSI may belong in a distinct diagnostic category. Selby and colleagues (2011) found that individuals who engaged in NSSI, but did not meet criteria for BPD, had similar levels of functional impairment in comparison to those with BPD. Those with a history of NSSI also reported higher levels of depressive and anxiety symptoms, greater suicidality, and lower functioning relative to other diagnoses (Selby, Bender, Gordon, Nock, & Joiner, 2011).

In response to this research, DSM-5 has included NSSI disorder as a disorder for future study (APA, 2013). According to the diagnostic criteria, an individual must engage in acts of NSSI on five or more separate days in the past year in order to meet for criterion A. Further, the behavior must be performed for at least one of the following reasons (criterion B): 1) relieve negative thoughts or feeling, 2) resolve an interpersonal problem, or 3) cause a positive feeling or emotion. It must also be associated with negative thoughts or feelings and/or interpersonal problems immediately prior to engaging in the behavior, preoccupation with the behavior that is difficult to resist, or the frequent urge to engage in the behavior (criterion C). Finally, the behavior cannot be socially sanctioned or occur exclusively within the context of another mental or medical disorder, and must cause clinically significant distress or impairment (APA, 2013).

The development of diagnostic criteria specific for NSSI disorder represents a significant advancement in the study of NSSI as it provides a clear set of guidelines by which researchers and clinicians can characterize pathological NSSI. Although these criteria have only recently been proposed, previous research has widely supported both criterions B and C. For example, NSSI has been identified to serve specific the functions of negative affect relief, interpersonal communication of distress, and providing a sense of reality or feeling (Gratz, 2003; Klonsky 2007; Muehlenkamp, Brausch, Quigley, & Whitlock, 2013; Nock & Prinstein, 2005). The behavior has also been found to occur in the context of negative mood (e.g., worry, pressure) and interpersonal conflict (Nock, Prinstein & Sterba, 2009).

There is limited research, however, examining the utility and discriminant validity of the behavior frequency (criterion A) necessary for NSSI disorder. No research has specifically examined the presence of a categorical demarcation based on the number of days NSSI has occurred; though, a strong literature maintains a distinction between single and repeated NSSI acts (e.g., Evans, Platts, & Liebenau, 1996; Hawton, Kingsbury, Steinhardt, James & Fagg, 1999), supported by the implication that NSSI must be repeated several times to receive the desired effect (Shaffer & Jacobson, 2009). More explicitly, previous researchers have proposed that for NSSI to be considered repetitive, and potentially indicative of its own disorder, an individual must have engaged in five or more acts (Favazza & Rosenthal, 1990; Graff & Mallin, 1967). Despite this, there is a dearth of empirical evidence surrounding this specific cutoff, which may potentially be considered as arbitrary (Shaffer & Johnson, 2009). Studies examining NSSI disorder have provided valuable descriptive information about those who meet the diagnostic criteria (e.g., feel more pain during act of NSSI; In-Albon, Ruf, & Schmid, 2013; Zetterquist, Lulndh, Dahlstrom, & Svedin, 2013), but have done little to shed light on the importance of past year behavior frequency for diagnostic status. For example, 8.7% of individuals in a community sample reported five or more acts of NSSI in the past year, but did not fulfill NSSI disorder criterion B or C, and, of those meeting criteria for NSSI disorder, approximately 74% performed 11 or more acts in the past year (Zetterquist et al., 2013). Similarly, in another study, approximately 4.6% of participants reported five or more acts of NSSI in the previous year with only 2.6% meeting criteria for NSSI disorder (Andover, 2014). Within a clinical sample seeking treatment for NSSI, over 80% of participants self-injured on five or more days in the previous year, with 71.8% of the overall sample meeting criteria for NSSI Disorder (Washburn, Potthoff, Juzwin, & Styer, 2015). As such, there is a need to further explore the validity of criterion A for NSSI disorder, in addition to how this cutoff may relate to individual severity or impairment.

One way to explore the validity of the frequency criterion is through the use of Exploratory Data Mining (EDM; McArdle & Ritschard, 2013). The past 10 years have seen an emergence of EDM applications in the social and behavioral science (for an overview, see McArdle & Ritschard, 2013). The main distinction between EDM and more traditional methods, such as linear regression, is allowing data a greater role in modeling the relationships, with less of an emphasis on prescribing the functional form a priori. Recently, reviews of different EDM models have been tailored for psychological researchers (King & Resick, 2014; McArdle, 2012; Strobl et al., 2009), along with a proliferation of EDM implementation, such as the use of Classification and Regression Trees (CART; Breiman, Friedman, Olshen, & Stone, 1984) to predict suicidal ideation in sexually abused female adolescents (Brabant, Hébert, & Chagnon, 2013) and treatment outcomes for eating disorders (Hannover, Richard, Hansen, Martinovich, & Kordy, 2002), among many others.

The current study used one specific EDM method, Structural Equation Modeling Trees (SEM Trees; Brandmaier, von Oertzen, McArdle, & Lindenberger, 2013) as a way to determine a NSSI frequency that optimally discriminates pathological NSSI from typical behavior within an empirically informed model. The model included psychopathology symptomology found to be comorbid with NSSI, including depression, suicidal thoughts and behaviors, anxiety, borderline personality symptomology, and disordered eating (Bentley, Cassiello-Robbins, Vittorio, Saur-Zavala, & Barlow, 2015; Glenn & Klonsky, 2011; Kerr & Muehlenkamp, 2010; Serras et al., 2010), in addition to cognitive-affective deficits commonly associated with NSSI, including emotion dysregulation and emotion reactivity (Andover & Morris, 2014; Gratz & Roemer, 2004; Franklin, Aaron, Arthur, Shorkey, & Prinstein, 2012; Kerr & Muehlenkamp, 2010; Nock, Wedig, Holmberg & Hooley, 2008; Selby et al., 2011). We expected that individuals with a history of NSSI would report higher levels of the overall factor comprised of psychopathology symptoms and cognitive-affective deficits. Further, we thought that the model would discriminate between individuals with and without a history of NSSI in the past year (zero acts versus one act); although no specific hypotheses were made beyond this initial classification, it was expected the model would also discriminate between those with lower NSSI frequency and higher NSSI frequency.

Method

Participants and Procedures

Participants were 3,559 undergraduate students (63% female, M age = 20.78, SD = 3.18, range = 18–57), ranging from freshman to seniors, from a large urban university. They completed a series of self-report measures as part of a larger, IRB-approved study on a secure website. Participants received course credit for their participation. Approximately 61% of the sample identified as Caucasian, 13% Asian, 13% African American, and 4% mixed race. Approximately 2% of the sample indicated they preferred not to answer, and the remaining 7% self-identified as “other.”

Measures

Forms and Functions of Self-Injury (FAFSI; Jenkins, Connor & Alloy, 2011)

The FAFSI is a self-report measure that inquires about different forms and associated functions of NSSI. Participants were asked, “Have you ever, intentionally or on purpose, hurt yourself in the following ways, without the intention of killing yours?” The FAFSI was used to assess number of NSSI acts in the past year (“How many times, in the last year, have you done this?).

Participants were asked to write in the number of times they engaged in NSSI in the past year. The internal consistency of the measure has been found to be strong (Jenkins et al., 2011), which was replicated in the current study (KR-20 = .81). Number of NSSI acts in the past year ranged from 0 – 10,000.

Quick Inventory of Depressive Symptomology (QIDS; Rush et al., 2003)

The QIDS is a 16-item self-report measure used to assess depressive symptomology based on the DSM-IV-TR major depressive episode criteria. The internal consistency and construct validity of this measure have been supported (Rush et al., 2003). In the current study, the scale had acceptable reliability, α = .78, and responses ranged from 0 – 24.

Suicidal Behavior Questionnaire – Revised (SBQ-R; Osman et al., 2001)

The SBQ-R was used to assess suicidality. It is a self-report that consists of 4 items, each assessing a dimension of suicidality: lifetime ideation/attempt; frequency of ideation; threat of suicidal behavior; and likelihood of suicidal behavior. The total score was used in the current study. Internal consistency of the full scale has been found to be strong in previous studies (Osman et al., 2001). In the current study, the SBQ-R had acceptable reliability, α = .79, and responses ranged from 0 – 18.

Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004)

The DERS is a 36-item self-report measure, which assessed six areas of emotion dysregulation: nonacceptance of negative emotions, inability to engage in goal-directed behaviors, difficulties controlling impulsive behaviors, limited access to emotion regulation strategies, lack of emotional awareness, and lack of emotional clarity. The DERS internal consistency, re-test reliability, and construct and predictive validity have been found to be strong (Gratz & Roemer, 2004). In the current study the DERS total score was used; the measure demonstrated good reliability, α = .92, and responses ranged from 23–162.

Emotion Reactivity Scale (ERS; Nock et al., 2008)

The ERS is a 21-item self-report measure that assessed emotion reactivity, including sensitivity, intensity, and duration of emotions (e.g. “When I experience emotions, I feel them very strongly/intensely.”). All items were summed to a single scale where higher scores equal higher levels of emotion reactivity. The ERS has shown strong internal consistency and convergent validity (Nock et al., 2008). In the current study the ERS demonstrated excellent reliability, α = .96, and responses ranged from 0 – 84.

McLean Screening Instrument for Borderline Personality Disorder (MSI-BPD; Zanarini et al., 2003)

The MSI-BPD is a 10-item self-report measure of BPD features and was used to assess BPD symptoms. Higher scores equal greater severity of symptoms. Previous studies found high convergent and concurrent validity of the measure (Gardner & Qualter, 2009). Further, when compared with a validated structured interview of BPD diagnosis, both sensitivity and specificity of the MSI-BPD were above .90 (Sanislow, Grilo, & McGlashan, 2000). In the current study, the MSI-BPD demonstrated good reliability, α = .83, and responses ranged from 0–10.

Eating Disorder Examination-Quetionnaire (EDE-Q; Fairburn & Beglin, 1994; Fairburn & Cooper, 1993)

The EDE-Q is a 36-item self-report questionnaire that focused on eating behaviors, including dietary restraint, eating concerns, concerns about weight, and concerns about shape in the past 28 days. The total score for the EDE-Q was used in the current study. The psychometric properties of this measure have been supported (Luce & Crowther, 1999; Peterson et al., 2007). In the current study the EDE-Q demonstrated excellent reliability, α = .96, and responses ranges from 0 – 6.

Penn State Worry Questionnaire (PSWQ; Meyer, Miller, Metzger, & Borkovec, 1990)

The PSWQ is a 16-item self-report measure that assessed the tendency to engage in excessive, uncontrollable, and generalized worry. The questionnaire has demonstrated excellent internal consistency, test-retest reliability, and validity (Brown, Antony, & Barlow, 1992; Meyer et al., 1990; Molina & Borkovec, 1994). In the current study, the scale demonstrated excellent reliability, α = .92, and responses ranged from 15 – 80.

Quality of Life Enjoyment and Satisfaction Questionnaire (Q-LES-Q; Endicott, Nee, Harrison, & Blumenthal, 1993)

The Q-LES-Q is a 16-item self-report measure that assessed enjoyment and satisfaction experienced in daily functioning. Items evaluated satisfaction with areas including physical health, relationships, leisure activities, etc. The questionnaire has demonstrated good reliability (Endicott et al., 1993) and predicted psychiatric outcomes above depression and anxiety (Hope, Page, & Hooke, 2009). The Q-LES-Q has also been associated with emotional distress, self-efficacy, and social support (Ritsner, 2007). In the current study the measure demonstrated excellent reliability, α = .95, and responses ranged from 0 – 64.

Exploratory Data Mining Methods

Decision Trees

Decision trees (DT) are one of the most popular methods that fall under the umbrella of EDM, and they form the basis for a number of more flexible and advanced methods. DT’s can be thought of as simple nonparametric regression models for use with both continuous and categorical outcomes. In this, the predictor variables are recursively partitioned to create groupings of individuals with similar response values. This results in inequalities on the individual dimensions of the covariate space, which can then be read like a rule set. An example of this is using height and weight as predictors of gender: “if height > 68 inches and weight > 150 pounds then gender = male, otherwise gender = female.” In this example, the covariate space is exhaustively searched, testing splits between each successive value of height and weight, to find the split that maximizes the chosen cost function.

The two most popular DT algorithms were independently introduced in the 1980’s, Classification and Regression Trees (CART; Breiman, Friedman, Olshen, & Stone, 1984) and ID3 (predecessor of C4.5; Quinlan, 1986, 1993). Terminology for different tree-based methods varies widely across published research; however, for the purposes of this paper, we will refer to the general EDM method as DT, and save the use of the term CART for the specific algorithm. In addition to providing better prediction accuracy than traditional linear regression in many cases, the popularity of DT can be attributed to the graphical nature and ease of interpretability. There are a number of DT packages available in the R (R Core Team, 2014) statistical environment. The specific R package we used that implements the CART algorithm was the tree package (Ripley, 2014).

Structural Equation Model Trees

Structural Equation Model Trees (SEM Trees; Brandmaier, et al., 2013) are a generalization of DT that build tree structures which separate a dataset recursively into subsets that are maximally different with respect to the fit of the SEM. Instead of fitting a single SEM model to the observed dataset, the dataset is split into subsets (of participants or cases) based on the splitting on covariates. This is analogous to the fitting of a multi-group model in SEM, except that the groups are not set a priori with SEM Trees. The SEM Tree algorithm proceeds in 3 steps:

1. Fit a parametric template model to the current data set via a chosen optimization procedure;

2. for each covariate, split the data set with respect to the covariate and compare the fit of the compound model of all submodels against the fit of the template model; and

3. choose among the compound models the one that is the best description of the data set according to the chosen criterion. If this model fits significantly better than the template model, repeat procedure with Step 1 for all submodels, otherwise terminate. (Brandmaier et al., 2013)

SEM Trees combines the exploratory benefits of DT with the confirmatory aspects of SEM. By including SEMs, it allows researchers to specify a confirmatory model, based on prior hypotheses. Through this marrying of exploratory and confirmatory, it adds a level of flexibility not inherent in most methods. It allows researchers the ability to constrain the search to results based on prior knowledge, while still giving the freedom to search for relationships not previously thought of and generate new ideas. Additionally, there is no need to compute factor scores, which can be difficult to estimate (i.e., factor score indeterminacy) and hard to interpret with multiple factor models.

It is worth noting the distinction between the procedure we used and others, such as Latent Class Analysis (LCA; McCutcheon, 1987). Our analysis created hierarchical subgroups based on the observed covariate number of NSSI acts, whereas if this variable would have been used as a covariate in LCA it would only partially influence the creation of subgroups. In SEM Trees, the subgroups are split solely based on values of the covariate. As creating subgroups based on number of NSSI acts was the main intention of this study, we view using the number of NSSI acts as the sole variable to derive hierarchical subgroups as optimal as compared to the use of LCA.

Methods Plan

Prior to running analyses, all data were checked for validity. As all procedures were completed online, those participants who completed the questionnaires quicker than two standard deviations below the mean response time were not included in the analyses. Further, validity items (e.g., “Select ‘1’ on this item.”) were included throughout the questionnaires, and only those participants who responded appropriately to 70% of the validity items were included in the analyses. As this study is exploratory, we tried three different statistical methods using the model aimed to indicate pathological NSSI. As there is no one best theoretical model of what would be considered pathological NSSI, our model was based on previous literature of NSSI risk factors. The model consisted of seven variables and assessed psychopathology symptomology and cognitive-affective deficits related to NSSI: depressive symptomology, suicidality, borderline personality disorder symptoms, disorder eating, anxiety symptomology, emotion dysregulation, and emotion reactivity. Higher mean scores are more indicative of pathology. The three methods used were SEM Trees, DT with explicitly estimating factor scores, and DT using summed scores. Although the use of SEM Trees theoretically provides the best form of estimation (the explicit estimation factor scores can be problematic; see Grice, 2001) for the question that we are asking, we wanted to determine whether similar results were derived across procedures (i.e., if the differences were to due differences in model fit across groups or differences in factor means across groups). The use of DT with factor scores and summed scores represent a form of validation, in an attempt to make sure the derived cutoffs were not solely due to the methodology used, especially given the relative novelty of SEM Trees. All analyses were conducted in R: CFA analyses utilized full information maximum likelihood with the lavaan package (Rosseel, 2012); DT analyses utilized the tree package (Ripley, 2014); SEM Tree analyses utilized the semtree package (Brandmaier et al., 2013). The only default that was changed in the semtree package was the use of the “fair” splitting criterion as opposed to “naive” in order to control for the large number of response options in the covariate and prevent an overly complex model. Of note, the variable assessing number of NSSI acts in the past year was significantly skewed (range 0–10,000); however, this does not impact the SEM Trees analysis as the method is sensitive to outliers and monotonic transformations (e.g., distributions that would be skewed/have outliers) in predictor variables (Hastie, Tibshirani, & Friedman, 2009). After determining what cutoff may best indicate pathological NSSI, group comparisons were examined based on both the proposed NSSI disorder criteria and the current model cutoffs using ANOVAs. Due to the multiple bivariate post-hoc comparisons p <.01 was utilized for interpretation. Finally, given the importance NSSI disorder criterion E (i.e., presence distress and impairment), a local regression curve, by means of a local scatter plot smoother (loess) based on a least-squares fit, with a tension of 66%, was calculated for the relationship between NSSI frequency and quality of life to assess for possible thresholds. This analysis included only those with a history of NSSI in the past year. NSSI acts in the past year was significantly skewed; as such the variable was windsordized to 2 standard deviations above the mean (correction applied to four values), which resulted in a normal distribution.

Results

Preliminary Results

Intercorrelations for study variables are displayed in Table 1. All study variables demonstrated small to moderate positive correlations (r’s .11 to .61). In line with the proposed NSSI disorder criteria, participants were categorized as having a history of NSSI if they reported engaging in the behavior within the past 12 months. Of the overall sample, 428 participants (11.72%) reported engaging in at least one act of NSSI in the past year. Among participants with a history of NSSI, the range of acts in the past year was 1–10,000 (M = 43.57, SD = 490.22, Median = 3). Those with and without a history of NSSI in the past year were compared on demographic variables. There was not a significant difference on gender between groups, χ2 (2) = .42, p = .81. However, there were significant differences on race between groups, χ2 (4) = 12.96, p = .01; those with a history of NSSI in the past year were more likely to be Caucasian, and those without a history of NSSI in the past year were more likely to be African American. Participants without a history of NSSI in the past year were also more likely to be older (M = 20.85, SD = 3.27) than those who engaged in NSSI in the past year (M = 20.26, SD = 2.32), t (3559) = 3.61, p <.001.

Table 1.

Intercorrelations for study variables

	1	2	3	4	5	6	7	8
1. NSSI History	--
2. Suicidal Thoughts/Behavior (SBQ-R)	.30**	--
3. Depression Symptomology (QIDS)	.24**	.37**	--
4. Emotion Dysregulation (DERS)	.21**	.27**	.53**	--
5. Emotion Reactivity (ERS)	.17**	.32**	.46**	.61**	--
6. Borderline Personality Symptomology (MSI-BPD)	.28**	.49**	.55**	.48**	.50**	--
7. Eating Disorder Symptomology (EDE-Q)	.16**	.22**	.35**	.29**	.30**	.33**	--
8. Anxiety Symptomology (PSWQ)	.14**	.25**	.43**	.43**	.55**	.38**	.31**	--

Note: NSSI History = Engaged in non-suicidal self-injury in past year; SBQ-R = Suicidal Behavior Questionnaire – Revised; QIDS = Quick Inventory of Depressive Symptomology; DERS = Difficulties in Emotion Regulation Scale; ERS = Emotion Reactivity Scale; MSI-BPD = McLean Screening Instrument for Borderline Personality Disorder; EDE-Q = Eating Disorder Examination – Questionnaire; PSWQ = Penn State Worry Questionnaire; AUDIT = Alcohol Use Disorders Identification Test;

^**p < .01

Confirmatory Factor Analysis

Before using SEM Trees to determine the optimal split on the NSSI frequency, we examined the fit of the confirmatory factor analysis model using the seven outcome variables. See Table 2 for factor loadings. The model achieved adequate fit, (χ2 (13) = 512, p < 0.001) with CFI = 0.925, RMSEA = 0.104 (95% CI: 0.095, 0.113), and SRMR = 0.042. Although the RMSEA is above recommendations for an adequate fit (< 0.1; Browne & Cudeck, 1993), we determined that the fit was good enough to proceed as it has been previously noted that the RMSEA has the potential to be positively biased when the number of indicators is small (as in the current model) (MacCallum, Browne, & Sugawara, 1996). It should be noted that some level of misfit must initially exist to use the SEM Trees algorithm. If none existed, there would be no possible covariate split that could increase the fit of the model, suggesting there is one homogenous group of participants. However, we were attempting to improve the fit of the model based on the assumption there is heterogeneity among participants.

Table 2.

Factor loadings of the confirmatory factor analyses

	Unstandardized	Standardized	Standard Error	p-value
Depression	1	0.72	0.02	<.001
Suicidality	0.79	0.57	0.02	<.001
Emotion Dysregulation	1.00	0.72	0.02	<.001
Emotion Reactivity	1.02	0.74	0.02	<.001
Borderline Personality	0.98	0.71	0.02	<.001
Disordered Eating	0.61	0.44	0.02	<.001
Anxiety	0.85	0.62	0.02	<.001

Note: Standard errors and p-values are based on the standardized factor loadings

SEM Tree

See Figure 1 for the tree diagram. The first split was between individuals with 0 and 1 act of NSSI in the past year, creating a large subgroup of individuals with no history of NSSI (n = 3,118). Among individuals with 1 or more previous acts, the optimal split took place between those with 5 and 6 NSSI acts. This resulted in a more symmetric distribution of individuals, with 289 people having 1 to 5 acts, and 137 with 6 or more. Across the three resulting subgroups, it is interesting to note the differences in general increases in the variable level means in the SEM model. It is of note that we allowed the factor means to vary across groups; therefore, less emphasis should be placed on comparing variable level means as discrepancies between groups manifest in the factor means. As represented in Figure 1, the factor means also increase across the three groups.

Figure 1.

SEM Tree

Note: LR = Likelihood Ratio; N = subgroup size; ddf = difference in degrees of freedom; =~ =factor loadings; ~~ =residuals; ~1 =means; Dep = depression; Suic = suicide; EmoDys = emotion dysregulation; EmoReac = emotion reactivity; BPD = borderline personality symptomology; DisEat = disordered eating; Anx = anxiety

DT with Factor and Summed Score

The results of the DT models demonstrated splits that correspond with the splits at the first two levels in Figure 1 from the SEM Trees model. The mean factor scores for those without a history of NSSI was 0.12, those with 1–5 acts of NSSI in the past year had a mean factor score of 0.57, and finally individuals with 6 or more acts of NSSI in the past had a mean of 1.2. Similar to the SEM Tree model results, as the NSSI cutoff criteria increased so did the factor scores on the severity variables. The use of summed scores, instead of factor scores, resulted in almost identical results; the cutoffs were identical, with only small differences in the means of subgroups.

Group Comparisons

Across the three methods, we found consistent results for two splits: the first between individuals with and without any NSSI in the past year, and among those with 1 or more NSSI acts in the past year, a second optimal split between those with 5 and 6 NSSI acts in the past year. Given this, potential differences in groups based on both the proposed NSSI disorder criteria (those with 1–4 acts in the past year versus 5 or more acts) and the cutoffs determined in the model (those with 1–5 acts in the past year versus 6 or more acts) were examined with regard to study variables and an additional measure assessing quality of life. First, when examining groups based on the current NSSI disorder criteria, those without a history of NSSI reported significantly lower levels of all study variables compared to those with a history of NSSI. Among those with a history of NSSI, participants having engaged in 5 or more acts of NSSI in the past year, compared to those with 1–4 NSSI acts, reported significantly greater suicidality, depression symptomology, emotion dysregulation, borderline personality symptoms, and eating disorder symptoms. No significant differences emerged for emotion reactivity, anxiety symptoms, or quality of life between these groups. See Table 3. In examining groups based on the model cutoffs, the same pattern of results with regard to those with and without a history of NSSI were found. Among those with a history of NSSI, individuals having engaged in 6 or more acts of NSSI in the past year, compared to those with 1–5 NSSI acts, reported significantly greater suicidality, depression symptomology, emotion dysregulation, borderline personality symptoms, and eating disorder symptoms. They also reported significantly lower quality of life. No significant differences emerged for emotion reactivity or anxiety symptoms between these groups. See Table 4. The loess regression plot showed a negative association between NSSI acts in the past year and quality of life. The loess regression curve illustrates a decline in quality of life with increased NSSI acts. See Figure 2. Potential thresholds emerge after 6 acts of NSSI in past year and near 25 acts of NSSI in the past year.⁺

Table 3.

Means and standard deviations of study variables by NSSI status based on NSSI disorder criteria

	No NSSI (n = 3137) M (SD)	1–4 NSSI acts (n = 267) M (SD)	5+ NSSI acts (n = 153) M (SD)	F - value	η2	d
Suicidality	4.36 (2.13) a	6.28 (3.06) b	8.24 (3.69) c	269.74***	.14	0.58
Depression	5.07 (3.78) a	7.15 (4.34) b	9.65 (5.18) c	124.25***	.07	0.52
Emotion Dysregulation	82.10 (.21.66) a	92.53 (20.86) b	103.55 (23.69) c	96.71***	.05	0.49
Emotion Reactivity	27.42 (18.07) a	36.04 (18.17) b	39.98 (18.56) b	44.88***	.03	0.21
Borderline Personality	2.95 (2.86) a	5.03 (2.96) b	6.50 (2.88) c	123.65***	.08	0.50
Disorder Eating	1.25 (1.30) a	1.70 (1.44) b	2.31 (1.58) c	43.48***	.03	0.40
Anxiety	48.85 (13.37) a	53.98 (13.41) b	55.45 (14.62) b	29.70***	.02	0.10
Quality of Life	44.28 (11.08) a	40.85 (10.65) b	37.22 (10.25) b	29.40***	.02	0.33

Note: NSSI = non-suicidal self-injury in past year; d = Cohen’s d based on 1–4 NSSI acts versus 5+ NSSI acts comparison;

^***p < .001; means with different superscripts within rows are significantly different at the p <.01 based on Tukey’s post hoc paired comparisons

Table 4.

Means and standard deviations of study variables by NSSI status based on model cutoffs

	No NSSI (n = 3137) M (SD)	1–5 NSSI acts (n = 286) M (SD)	6+ NSSI acts (n = 136) M (SD)	F - value	η2	d
Suicidality	4.36 (2.13) a	6.32 (3.09) b	8.41 (3.69) c	272.97***	.15	0.61
Depression	5.07 (3.79) a	7.16 (4.33) b	9.92 (5.23) c	127.41***	.07	0.57
Emotion Dysregulation	82.10 (21.66) a	92.80 (20.71) b	104.44 (24.22) c	97.44***	.05	0.52
Emotion Reactivity	27.42 (18.07) a	35.88 (18.24) b	40.90 (18.31) b	45.82***	.03	0.27
Borderline Personality	2.95 (2.86) a	5.01 (2.98) b	6.76 (2.73) c	126.91***	.08	0.61
Disorder Eating	1.25 (1.30) a	1.68 (1.44) b	2.43 (1.57) c	46.69***	.03	0.50
Anxiety	48.85 (13.37) a	53.76 (13.49) b	57.30 (14.51) b	29.83***	.02	0.25
Quality of Life	44.28 (11.08) a	40.85 (10.20) b	36.66 (10.61) c	30.37***	.02	0.40

Note: NSSI = non-suicidal self-injury in past year; d = Cohen’s d based on 1–5 NSSI acts versus 6+ NSSI acts comparison;

^***p < .001; means with different superscripts within rows are significantly different at the p <.01 based on Tukey’s post hoc paired comparisons

Figure 2.

Locally weighted regression smoothing curve

Note: NSSI = non-suicidal self-injury; the vertical reference line represents 6 acts of NSSI in the past year

Discussion

There has been a recent increase in the exploration of NSSI Disorder since its addition as a disorder for further study in the latest edition of the DSM (APA, 2013). The diagnostic criteria for NSSI Disorder specify the presence of five or more days with instances of NSSI behavior over a one-year period. However, there has been no study to validate if this cutoff actually distinguishes pathological NSSI from typical behavior. We addressed this question in the present study utilizing Exploratory Data Mining (McArdle & Ritschard, 2013), a set of statistical techniques that allows the grouping of participants in a dataset based on their scores of relevant variables. We utilized two different statistical techniques to determine optimal NSSI frequency cutoffs: structural equation modeling trees (SEM Trees) and decision trees (DT). Both methods agreed on optimal cutoffs of one and six acts. Thus, indicating that there are three distinct groups of NSSI individuals: 1) those who do not engage in NSSI, 2) those who seldom engage in NSSI (1–5 times per year), and 3) those who frequently engage in NSSI (6 or more times per year).

Our findings have two main implications for the diagnosis of NSSI Disorder when examined in an undergraduate sample. First, the optimal cutoff derived from this sample is six, not five acts of NSSI per year. Although this finding is generally supportive of the DSM criteria when considering the possible range from 1 – 365 (i.e., the number of days in a year), it suggests that under the current criteria, individuals who engaged in one NSSI act on exactly five days in the past year may be incorrectly categorized as having NSSI Disorder. Second, because the one to five acts group differed from both those who had not engaged in NSSI and those who had engaged in six or more acts, there may be a subclinical form of NSSI Disorder, which is consistent with a dimensional approach to considering psychopathology. These individuals are of particular interest for future study because they may represent those who are at high-risk for development of NSSI Disorder. Alternatively, this group may also represent individuals who are in remission from NSSI Disorder. Future research is needed on this subclinical group.

The distinctiveness of the three NSSI groups was further supported by differences on a variety of relevant psychopathology and cognitive-affective variables. Compared to those who engaged in NSSI, individuals who did not engage in NSSI had lower levels of suicidality; depressive, anxiety, borderline personality disorder, and eating disorder symptoms; and emotion dysregulation and reactivity. This is in line with a large literature suggesting differences in cognitive-affective and psychopathology variables between those who do and do not engage in NSSI (e.g., Gratz & Roemer, 2004; Kerr & Muehlenkamp, 2010). With the exception of a few variables (i.e., anxiety symptoms, emotion reactivity), individuals who seldom engaged in NSSI had significantly less severe cognitive-affective difficulties and psychopathology symptoms. Further, when comparing group differences between those with one to four versus five past year NSSI acts with differences between those with one to five versus six past year NSSI acts, the effect sizes for latter group comparisons (based on the model cutoffs) are larger for all significant group differences. When specifically looking at reported quality of life it was found that group differences existed when examining groups based on the model cutoff, but not as proposed by the NSSI disorder criteria. This suggests that using a cutoff of six, instead of five, acts of NSSI to determine clinical severity may produce better distinction in severity between those who frequently engage in NSSI from those who seldom engage in NSSI. Although this provides an initial examination of the validation of the cutoff of 6 acts, future research is warranted to further support this cutoff, particularly among more diverse samples.

It is not necessarily problematic that those who frequently engaged in NSSI and those who seldom engaged in NSSI did not differ on all psychopathology and cognitive-affective variables because we would not expect any clinical cutoff to distinguish groups on all potential variables. Our findings may suggest that variables such as anxiety symptoms may be useful to distinguish those who engage in NSSI from those who do not, but are not useful in determining NSSI severity. This is in line with research showing greater anxiety symptoms distinguish those who have engaged in NSSI from those who have not, but does not predict severity of NSSI beyond that point (Selby et al., 2012).

Taken together, these findings are highly relevant to the general literature on NSSI disorder. The cutoff of five or more days with NSSI in a 12-month period is largely arbitrary (Shaffer & Jacobson, 2009) and not all studies find differences in impairment using a cutoff of five acts of NSSI (e.g., Andover 2014). Because of these issues, Selby, Kranzler, Fehling, and Panza’s (2015) recent review of the literature on NSSI disorder calls for empirically derived cutoffs that might better characterize impairment in NSSI disorder. Our study serves to answer that call. It is important to note, however, that the current study examined number of NSSI events in the past year as opposed to number of days in the past year, as stated in the diagnostic criteria. Given this, it is possible that some individuals in the current sample engaged in NSSI multiple times within a few days, not fulfilling criterion A. Future research will be needed to validate the current findings using an assessment of days versus events in the past year, and, moreover, to determine if number of days in the past year is the best diagnostic metric by comparing those with multiple acts on four or less days to those with singular acts on five or more days.

Our findings have several other limitations that should be considered. The tree structures cannot necessarily be seen as optimal, as the split at the second level is dependent on the first split. Said in another way, the second level split between 5 and 6 is optimal given that the first split was between 0 and 1. If the first split was at a different value, then the second level split could be completely different. However, a substantial body of literature comparing those who have never engaged in NSSI to those who have at least once finds a pattern of lesser psychopathology and emotion reactivity among those who have never engaged in NSSI (Nock et al., 2008; Selby et al., 2011). These findings can be taken as support for the empirically derived split between zero and one acts in the current paper. Relatedly, the cutoffs established are for a single (albeit large) undergraduate sample responding to all questionnaires online. It is possible that the non-clinical nature of our sample, and restricted age range, may have influenced the current findings. Although the rate of NSSI within the current sample is comparable to that found in an outpatient treatment-seeking sample (e.g., Selby et al., 2011), significant NSSI history is often found at higher rates among inpatient clinical samples (Ferrara, Terrinoni, & Williams, 2012; Muehlenkamp & Brausch, 2012). This is important to consider in the context of the NSSI disorder diagnosis and highlights the need of replication of the current findings. Further, although we do not believe our online data collection method negatively impacted the validity of the data as validity checks were included, verification of these cutoffs across additional samples in multiple settings, including within treatment-seeking or inpatient clinical populations, will be needed to confirm the current findings and increase confidence in the validity of the frequency cutoff point for diagnostic decisions.

There are several strengths to the present study. First, we used a relatively large sample of participants. We also used a novel, but well-established, data analysis method to derive empirically informed cutoffs for NSSI disorder in a way that would not be possible with other statistical methods. Outside of the specific findings, our study provided an application of a novel statistical technique that can be used to answer other questions in clinical psychology. Finally, the current data provide one of the first empirical analyses to validate the frequency cutoff, criterion A, for NSSI disorder. From these analyses, it appears the suggested cutoff noted in DSM-5 has relevance, but increasing the cutoff to six days in which an act of NSSI occurs may strengthen the validity of this criterion. Overall, the current study offers data that advances the field of NSSI research and calls for continued, critical analysis of the proposed DSM-5 NSSI disorder criteria.