Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jun 1.
Published in final edited form as: J Psychopathol Behav Assess. 2015 Sep 14;38(2):284–296. doi: 10.1007/s10862-015-9514-x

Development and Validation of a Brief Version of the Difficulties in Emotion Regulation Scale: The DERS-16

Johan Bjureberg 1,*, Brjánn Ljótsson 1, Matthew T Tull 2, Erik Hedman 1, Hanna Sahlin 1, Lars-Gunnar Lundh 3, Jonas Bjärehed 3, David DiLillo 4, Terri Messman-Moore 5, Clara Hellner Gumpert 1, Kim L Gratz 2
PMCID: PMC4882111  NIHMSID: NIHMS723083  PMID: 27239096

Abstract

The Difficulties in Emotion Regulation Scale (DERS) is a widely-used, theoretically-driven, and psychometrically-sound self-report measure of emotion regulation difficulties. However, at 36-items, the DERS may be challenging to administer in some situations or settings (e.g., in the course of patient care or large-scale epidemiological studies). Consequently, there is a need a briefer version of the DERS. The goal of the present studies was to develop and evaluate a 16-item version of the DERS – the DERS-16. The reliability and validity of the DERS-16 were examined in a clinical sample (N = 96) and two large community samples (Ns = 102 and 482). The validity of the DERS-16 was evaluated comparing the relative strength of the association of the two versions of the DERS with measures of emotion regulation and related constructs, psychopathology, and clinically-relevant behaviors theorized to stem from emotion regulation deficits. Results demonstrate that the DERS-16 has retained excellent internal consistency, good test-retest reliability, and good convergent and discriminant validity. Further, the DERS-16 showed minimal differences in its convergent and discriminant validity with relevant measures when compared to the original DERS. In conclusion, the DERS-16 offers a valid and brief method for the assessment of overall emotion regulation difficulties.

Keywords: emotion regulation, short version, self-report measure, assessment, experiential avoidance

Emotion regulation broadly refers to the intrinsic and extrinsic processes involved in monitoring, evaluating, and modulating emotional reactions in order to accomplish one's goals (Thompson, 1994). Inherent within this definition of emotion regulation is the idea that emotions are functional, providing information about our environment and motivating behaviors that may facilitate adaptation to situational demands (Izard & Ackerman, 2000). Conversely, difficulties in the awareness, understanding, or modulation of emotion may interfere with adaptation and contribute to a wide range of negative outcomes. Indeed, a rapidly growing body of research offers support for the role of emotion regulation difficulties in multiple forms of psychopathology and maladaptive behaviors (for reviews, see Cichetti, Ackerman, & Izard, 1995; Gratz & Tull, 2010; Gross & Jazaieri, 2014; Kring & Werner, 2004; Sheppes, Suri, & Gross, 2015). Given the obvious clinical relevance of emotion regulation, the past decade has also seen the development and validation of numerous self-report measures designed to assess various aspects of the emotion regulation construct.

Although there are several empirically-supported measures of various dimensions of emotion regulation difficulties (e.g., emotional nonacceptance, lack of emotional awareness and clarity; Bagby, Taylor, & Parker, 1994; Hofmann & Kashdan, 2010; Simons & Gaher, 2005) and putatively adaptive and maladaptive emotion regulation strategies (e.g., emotional avoidance, expressive suppression; Gross & John, 2003; Taylor, Laposa, & Alden, 2004), one comprehensive measure of emotion regulation difficulties that has received extensive attention within the literature is the Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004). The DERS is a 36-item self-report measure that assesses individuals' typical levels of difficulties in emotion regulation. The DERS is based on a clinically-useful conceptualization of emotion regulation that was developed to be applicable to a wide variety of psychological difficulties and relevant to clinical applications and treatment development (Gratz, 2007; Gratz & Tull, 2010). Specifically, the conceptual definition of emotion regulation on which the DERS is based emphasizes the functionality of emotions and focuses on adaptive ways of responding to emotional distress, including the: (a) awareness, understanding, and acceptance of emotions; (b) ability to control behaviors when experiencing negative emotions; (c) flexible use of situationally-appropriate strategies to modulate the intensity and/or duration of emotional responses, rather than to eliminate emotions entirely; and (d) willingness to experience negative emotions as part of pursuing meaningful activities in life (see Gratz & Roemer, 2004; Gratz & Tull, 2010).

There is extensive support for the construct validity of the DERS. Scores on the DERS have been found to be associated with multiple forms of psychopathology, including posttraumatic stress disorder (Tull, Barrett, McMillan, & Roemer, 2007), borderline personality disorder (Gratz, Rosenthal, Tull, Lejuez, & Gunderson, 2006), major depression (Ehring, Tuschen-Coffier, Schnüller, Fischer, & Gross, 2010), eating disorders (Whiteside et al., 2007), and generalized anxiety disorder (Salters-Pedneault, Roemer, Tull, Rucker, & Mennin, 2006). In addition, emotion regulation difficulties as assessed by the DERS are related to a number of maladaptive behaviors thought to serve an emotion regulating function, such as deliberate self-harm (Gratz & Roemer, 2008), substance use (Tull, Bardeen, DiLillo, Messman-Moore, & Gratz, 2015), risky sexual behavior (Tull, Weiss, Adams, & Gratz, 2012), and purging behavior (Lavender et al., 2014). The DERS has also demonstrated associations with biological (e.g., magnitude of rostral anterior cingulate cortex activation; Li et al., 2008), psychophysiological (e.g., heart rate variability; Berna, Ott, & Nandrino, 2014), neurological (e.g., amygdala habituation; Goodman et al., 2014), and behavioral (e.g., persistence on laboratory-based distress-inducing tasks; Gratz et al., 2006; Tull, Gratz, Latzman, Kimbrel, & Lejuez, 2010) indices of emotion regulation. Finally, scores on the DERS have been found to change in response to treatments that target emotion regulation, including an emotion regulation group therapy (Gratz, Bardeen, Levy, Dixon-Gordon, & Tull, 2015; Gratz, Tull, & Levy, 2014) and dialectical behavior therapy (Ben-Porath, Federici, Wisniewski, & Warren, 2014), suggesting that the measure is sensitive to change over time and has utility in treatment outcome research.

In order for a measure to have broad clinical and research utility, however, it must be brief. Indeed, given the limited time and resources of community clinicians, only measures that are brief and can be easily incorporated into standard assessments or administered during ongoing sessions are likely to be utilized. Moreover, shorter measures are often preferable in certain research contexts, such as clinical trials that require repeated assessments (e.g., assessment of change in symptoms on a weekly basis) or the measurement of change across multiple indices and epidemiological studies and other large-scale research endeavors. Thus, although the DERS has strong empirical support, its clinical and research utility would likely be further improved by shortening the measure. To this end, we sought to develop a short form of the DERS that would adequately assess the overarching construct of emotion regulation difficulties in as brief a way as possible, thereby broadening its applicability across a wider range of clinical and research settings.

Primary Aims of this Research

The primary aims of the present research were to develop a brief version of the DERS (the DERS-16) and evaluate its psychometric properties. Consistent with recommendations for scale development in general and the development of short forms of scales in particular (Hinkin, 1998; Smith, McCarthy, & Anderson, 2000), we used independent samples for scale development and evaluation, as well as independent administrations of the short version and original version of the DERS. We also examined the convergent and discriminant validity of the DERS-16 item in two additional samples, in order to provide further information on the construct validity of this measure in large community samples.

In Study 1, both the DERS-16 and the original DERS were administered and their associations with measures of relevant constructs (including psychiatric symptoms and clinically-relevant behaviors thought to stem from emotion dysregulation) were examined and compared. In Study 2, the relative strength of the associations of the 36-item and 16-item versions of the DERS with measures of (a) emotion regulation and related constructs (including emotional functioning, experiential avoidance, and mindfulness), (b) psychopathology (including depression, anxiety, and borderline personality disorder symptoms), and (c) clinically-relevant behaviors theorized to stem from emotion regulation deficits (including deliberate self-harm and substance abuse) were examined in two large community samples. We hypothesized that the strength of these associations would be comparable for the 36-item and 16-item versions of the DERS.

Development of the DERS-16

Items for the DERS-16 were selected from the 36 items in the original DERS on the basis of both item-total correlations and considerations regarding content validity. This procedure improves the chance of preserving high internal consistency and excluding items with the most error variance while simultaneously maintaining the breadth of the construct being measured (Smith et al., 2000). Items that were very similar in wording or that correlated highly (r > .90) with another item were also excluded to reduce scale length and redundancy. The scale reduction procedure was led by the principal investigator of the present study and developer of the full-length DERS (KLG) to assure content preservation.

Item-total correlations and inter-item correlations were obtained from the original study of the 36-item DERS (Gratz & Roemer, 2004). Item reduction based on item-total correlations less than r = .50 resulted in the elimination of 9 items. The remaining 27 items represented five of the six original subscales (i.e., all except the lack of emotional awareness subscale). Findings that none of the items from the lack of emotional awareness subscale was retained on the basis of the item-total correlations are consistent with past research indicating that this subscale is less strongly associated with the overall DERS score than the other subscales (Neumann, van Lier, Gratz, & Koot, 2010; Tull et al., 2007; Tull et al., 2010). Given evidence that this subscale (as operationalized in the original measure and distinct from emotional clarity) may be less relevant to the overarching construct of emotion dysregulation as assessed with the DERS (see Bardeen, Fergus, & Orcutt, 2012), no additional items from this subscale were added to the short version.

Of the remaining 27 items, no items were excluded due to a high correlation (r > .90) with another item. To balance content and brevity considerations, the 15 items with the highest item-total correlation were initially retained in order to maintain high internal consistency. Of these 15 items, one item was excluded due to overlapping content and similarity with the other retained items, and another was excluded because its overarching subscale was already sufficiently represented by other items. Three additional items from the original DERS were then included in order to retain at least two items from each subscale (except for the lack of emotional awareness subscale) and cover the broad scope of emotion regulation difficulties. These additional items were chosen based on the following criteria: (a) a high item-total correlation (at least r > .50), and (b) complete agreement by study authors that the item best represented or improved the construct validity of the subscale in question. This procedure resulted in a 16-item solution that was considered optimal for a short version of the DERS. The items included in the DERS-16 are presented in the Appendix. This 16-item version of the DERS demonstrated good internal consistency (α = .92) and was strongly associated with the original 36-item version (r = .93) in the original validation sample (Gratz & Roemer, 2004).

Study 1

In order to provide evidence for the utility and validity of the DERS-16, both the original 36-item DERS and the DERS-16 were administrated in a clinical sample and their respective correlations with relevant constructs compared.

Participants and Procedure

Study 1 included 96 women enrolled in an acceptance-based emotion regulation group therapy for deliberate self-harm at 14 different psychiatric outpatient clinics throughout Sweden. Inclusion criteria included: (a) being a woman over 18 years of age; (b) meeting at least three diagnostic criteria for borderline personality disorder (BPD); and (c) having a history of deliberate self-harm with at least three episodes in the past six months. Exclusion criteria consisted of: (d) primary diagnoses of a psychotic disorder, bipolar I disorder, current (past month) substance dependence, or another acute primary diagnosis (e.g. anorexia nervosa) that required immediate treatment; (e) current life circumstances that would interfere with treatment (e.g. ongoing domestic abuse); and/or (f) insufficient Swedish language skills.

The study was approved by the Regional Ethics Review Board in Stockholm, Sweden. All participants enrolled in the treatment study completed the Mini-International Neuropsychiatric Interview Version 6 (Sheehan et al., 2010) in order to assess the presence of mood and anxiety disorders and the Structured Clinical Interview for DSM-IV Axis II Personality Disorders – Borderline Personality Disorder Module (First, Gibbon, Spitzer, Williams, & Benjamin, 1997) to assess symptoms of BPD. Participant characteristics are presented in Table 1. Participants also completed a series of internet-administered self-report measures prior to the initiation of treatment. The original 36-item DERS and the DERS-16 were administrated separately on the same day or up to six days apart from each other; the median and mean number of days between assessments was 0 (IQR = 0-1.5) and 0.84 (SD = 1.39), respectively. The DERS-16 was presented before the original DERS for 34 participants and in the reversed order for the remaining 64 participants. The DERS-16 was also administered twice for 31 participants to assess its test-retest reliability, with the second administration occurring between 7 to 14 days (Mean = 8.32; SD = 1.58) after the first administration.

Table 1.

Participant demographic and clinical characteristics.

Study 1 Study 2
N = 96 Sample 1 (N = 102) Sample 2 (N = 482)
Age
 Mean (SD) 25.37 (6.63) 24.68 (10.27) 21.75 (2.23)
 Range 18-49 18-60 18-25
Sex
 Female 100 % 63.64 % 100 %
Ethnicity
 White - 52.04 % 55.6 %
 African Am - 24.49 % 32.2 %
 Asian - 9.18 % 2.7 %
 Latina - 4.08 % 5.8 %
Relationship status
 Single 28.12 % 92.86 % 83.33 %
Educational level
 High school graduate 49.47 % 6.12 % 16.63 %
 Some higher education 10.53 % 69.39 % 51.14 %
Occupation
 Full-time student 32.29 % 65.98 % 51.98 %
Diagnoses
 BPD 68.75 % - -
 Any anxiety disorder 81.25 % - -
 Mood disorder 54.17 % - -
Open in a new tab

Note. BPD = Borderline personality disorder.

Measures

Emotion regulation

The original Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) is a 36-item self-report measure that assesses individuals' typical levels of emotion dysregulation across six domains: nonacceptance of negative emotions, inability to engage in goal-directed behaviors when distressed, difficulties controlling impulsive behaviors when distressed, limited access to emotion regulation strategies perceived as effective, lack of emotional awareness, and lack of emotional clarity. Respondents rate the extent to which each item applies to them on a 5-point Likert-type scale from 1 (almost never) to 5 (almost always). As such, total scores on the DERS can range from 36 to 180. The DERS has been found to demonstrate good test-retest reliability (ρI = .88, p < .01) and adequate construct and predictive validity (Gratz & Roemer, 2004; Gratz & Tull, 2010). Items were recoded so that higher scores indicate greater emotion dysregulation, and a sum was calculated. The Swedish version of the DERS was back-translated into English and scrutinized by KLG to ensure no discrepancies between the English and Swedish versions.

As described previously, the shortened version of the Difficulties in Emotion Regulation Scale (DERS-16) consists of 16 items that assess the following dimensions of emotion regulation difficulties: nonacceptance of negative emotions (three items), inability to engage in goal-directed behaviors when distressed (three items), difficulties controlling impulsive behaviors when distressed (three items), limited access to emotion regulation strategies perceived as effective (five items), and lack of emotional clarity (two items). As with the original DERS, respondents rate the extent to which each item applies to them on a 5-point Likert-type scale from 1 (almost never) to 5 (almost always). Total scores on the DERS-16 can range from 16 to 80, with higher scores reflecting greater levels of emotion dysregulation.

Psychiatric symptoms

The 21-item version of the Depression Anxiety Stress Scales (DASS-21; Henry & Crawford, 2005) was used to assess the severity of depression, anxiety, and stress symptoms in the past week. Items are rated on a 4-point Likert-type scale ranging from 0 (Did not apply to me at all) to 3 (Applied to me very much, or most of the time). Scores are summed separately for each subscale, and multiplied by two. The DASS-21 demonstrates adequate reliability and construct and discriminant validity (Henry & Crawford, 2005).

Clinically-relevant behaviors

The Deliberate Self Harm Inventory (DSHI; Gratz, 2001) is a 17-item self-report measure that assesses different aspects of deliberate self-harm (including frequency and type of self-harm behavior) over specified time periods (e.g., lifetime or past four months [used in the present study]). The DSHI has demonstrated good internal consistency, adequate test–retest reliability, and adequate construct, discriminant, and convergent validity among undergraduate student, community adult, and patient samples (Fliege et al., 2006; Gratz, 2001; Gratz et al., 2011). Consistent with past research (Dixon-Gordon, Tull, & Gratz, 2014; Gratz & Tull, 2012; Turner, Layden, Butler, & Chapman, 2013), a deliberate self-harm frequency variable was computed by summing the total number of deliberate self-harm episodes reported in the past four months and a deliberate self-harm versatility index was computed by summing the number of different types of deliberate self-harm behaviors in the past four months (Turner et al., 2013).

The 11-item behavior supplement to the Borderline Symptom List (BSL; Bohus et al., 2001) assesses past-week engagement in a range of impulsive, self-destructive behaviors (e.g., risky sexual behavior, binge eating, purging, and substance abuse). Items are rated on a 5-point scale ranging from 0 (Not at all) to 4 (Daily or more often) and summed to obtain a total score (Gratz & Tull, 2011).

Observed internal consistency coefficients (Cronbach's α) for all relevant measures are presented in Table 2.

Table 2.

Cronbach's alphas, intercorrelations, differences between correlation coefficients (r-diff), and significance tests of the difference between correlation coefficients (p) in Study 1.

Study 1 (N=96)
Self-report measure α DERS-16 DERS r-diff (95% C.I.) p
Emotion regulation
 DERS-16 .92 .80***
 DERS .90
Related constructs
  DASS anxiety .78 .43*** .40*** .03 [-.09, 14] .67
  DASS depression .87 .55*** .54*** .01 [-.09, .12] .79
  DASS stress .76 .56*** .53*** .03 [-.08, .14] .60
DSHI frequencya,b - .22* .20* .02 [-.11, .14] .80
DSHI versatilitya,b - .30** .32** -.02 [-.15, .10] .69
BSL-supplementb - .36*** .40*** -.04 [-.16, .08] .50
Open in a new tab

Note. DERS = Difficulties in Emotion Regulation Scale; DASS = Depression Anxiety Stress Scales; DSHI = Deliberate Self Harm Inventory; BSL = Borderline Symptom List.

a

Time period < 4 months.

b

Given the checklist nature of the DSHI and the BSL, it was not considered appropriate to calculate Cronbach's alpha for these measures.

***

p < .001

**

p < .01

*

p < .05

Analysis Plan

Statistical analyses were conducted using STATA version 13.1 (StataCorp, College Station, Texas, USA) and R version 3.1.1 (R Development Core Team, R Foundation for Statistical Computing, Vienna, Austria).

Internal consistency of the DERS and DERS-16 was examined by calculating Cronbach's α. Intraclass correlation coefficients were computed to determine the test-retest reliability of the DERS-16. To examine the feasibility of the DERS-16 and its relative brevity compared to the original DERS, the mean and median administration times for both DERS versions in a subsample of participants for which those data were available (n = 94 for the DERS-16; n = 71 for the DERS; n = 70 for both) were calculated and the differences in the mean administration time of both measures was examined using a paired sample t-test. Pearson's product-moment correlations were conducted to evaluate associations between the DERS variables and other relevant self-report measures. Confidence intervals for differences between correlations of the original 36-item DERS and DERS-16 with the relevant self-report measures were calculated using a method for comparing overlapping correlations for dependent groups (Zou, 2007). We also tested whether the correlations differed significantly using a method for comparing dependent (single sample) correlation differences (Steiger, 1980).

Results

Preliminary Analyses

All continuous variables fell within the acceptable range of normality according to Tabachnick and Fidell (2001), with the exception of deliberate self-harm frequency (skewness = 5.03). Following logarithm transformation, this variable approximated a normal distribution (skewness = -0.05). Means and standard deviations of the DERS and DERS-16 in this sample are presented in Table 3.

Table 3. Means (M) and standard deviations (SD) of the DERS-16 and original DERS scores in study samples.

Study 1 Study 2
N = 96 Sample 1 (N = 102) Sample 2 (N = 482)
M SD M SD M SD
DERS-16 57 13.05 42.90 12.83 33.57 13.14
Original DERS 126.03 19.28 93.87 22.42 78.03 23.95
Open in a new tab

Note. DERS = Difficulties in Emotion Regulation Scale

Reliability of the DERS-16

The internal consistency of the DERS-16 was excellent (α= .92). Likewise, test-retest reliability of the DERS-16 was good (ρI = .85; p < .001).

Validity

Table 2 presents the correlation between the original 36-item DERS and the DERS-16, as well as their respective correlations with the other measures, the differences between these correlations, and significance tests of these differences. As shown in Table 2, the two versions of the DERS were significantly correlated with one another and with all other measures. Furthermore, the correlations of the DERS-16 and original 36-item DERS with the other measures were very similar (with only minor differences ranging from r = .00 to r = .04) and did not differ significantly from one another (see Table 2). These findings provide evidence for construct validity equivalence across the DERS-16 and original DERS.

Feasibility of the DERS-16

Providing support for the feasibility of the DERS-16 and its relative brevity compared to the original 36-item DERS, the average administration time for the DERS-16 was 138.99 seconds (SD = 133.89) with a median administration time of 100 seconds. In contrast, the average administration time of the DERS was 328.51 seconds (SD = 219.29), with a median time of 269 seconds. Results of the t-test revealed a significant difference in the mean administration time of the 16-item and 36-item versions, t (69) = 6.51, p < .001.

Study 2

To provide further support for the construct validity of the 16-item version of the DERS, scores on this shortened version of the measure were extracted from the original DERS administered to two additional community samples within the United States, and the relations of both the 36-item and 16-item versions of the DERS with additional measures of emotion regulation and related constructs were examined and compared.

Participants and Procedures

All methods received prior approval by the Institutional Review Boards of all participating institutions and all participants provided written consent. Participant characteristics for both samples are presented in Table 1. Participants in the first sample were 102 adults from the greater Washington D.C. area recruited using advertisements posted throughout the community and on-line. The original purpose of this study was to investigate emotional and cognitive functioning in psychopathology; consequently, advertisements targeted individuals with mood or behavioral dysregulation.

Participants in the second sample included a community sample of 482 young adult women drawn from a large, multi-site, prospective study of emotion dysregulation and sexual revictimization. Participants were recruited from four sites in the Southern and Midwestern United States using random sampling from the community (based on residential mailing addresses) as well as community advertisements. All questionnaires were completed online in the laboratory of one of the study sites.

Measures

In addition to the measures administered in Study 1 (i.e., the DERS, DASS-21, DSHI [assessing lifetime prevalence of deliberate self-harm], and BSL Supplement), the following measures were also administered to one or both of the samples in Study 2 (see Table 4).

Table 4.

Cronbach's alphas, intercorrelations, and differences between correlation coefficients (r-diff) in Study 2.

Sample 1 (N = 99-102) Sample 2 (N = 465-482)
Self-report measure α DERS-16 DERS r-diff α DERS-16 DERS r-diff
Emotion regulation
 DERS-16 .92 .95*** .94 .96***
 DERS .93 .95
Emotion-related constructs
 AAQ-9 .61 .70*** .76*** -.06 .67 .68*** .72*** -.04
 AIM total .93 .43*** .37*** .06 .90 .32*** .26*** .06
  AIM positive .93 .19 .14 .05 .94 .03 -.05 .08
  AIM negative .85 .64*** .59*** .05 .80 .58*** .56*** .02
 FFMQ Total .88 -.63*** -.70*** -.07 -- -- -- --
  FFMQ Nonreactivity .74 -.54*** -.55*** -.01 -- -- -- --
  FFMQ Nonjudgement .91 -.54*** -.55*** -.01 -- -- -- --
  FFMQ Acting .86 -.58*** -.56*** .02 .87 -.57*** -.58*** -.01
  FFMQ Describing .91 -.32*** -.45*** -.13 .91 -.36*** -.48*** -.12
  FFMQ Observing .77 .07 .02 .05 -- -- -- --
 WBSI .87 .54*** .53*** .01 -- -- -- --
 ALS-SF .90 .58*** .60*** -.02 -- -- -- --
 ASI-3 .90 .52*** .53*** -.01 .97 .57*** .57*** .00
 TEARS reduction -- -- -- -- .90 -.50*** -.54*** .04
 TEARS amplification -- -- -- -- .90 .04 -.05 .10
 BEQ negative expressivity -- -- -- -- .67 .13** .10* .03
 BEQ impulse strength -- -- -- -- .73 .40*** .34*** .06
 Psychopathology
  BEST .83 .61*** .63*** -.02 .85 .46*** .48*** -.02
  DASS anxiety .73 .44*** .48*** -.04 .91 .27*** .28*** -.01
  DASS depression .86 .47*** .45*** .02 .90 .45*** .45*** .00
  DASS stress .78 .56*** .56*** .00 .89 .56*** .55*** .01
 Clinically relevant behaviors
  DSHI frequencya,b -- .11 .10 .01 -- .33*** .33*** .00
  DSHI versatilitya,b -- .17 .15 .02 -- .30*** .29*** .01
  BSL-supplementb -- .38*** .39*** -.01 -- -- -- --
  AUDIT -- -- -- -- .83 .31*** .32*** -.01
Open in a new tab

Note. DERS = Difficulties in Emotion Regulation Scale; AAQ = Acceptance and Action Questionnaire; AIM = Affect Intensity Measure; TEARS = The Emotion Amplification and Reduction Scales; FFMQ = Five Facet Mindfulness Questionnaire; WBSI = White Bear Suppression Inventory; ALS = Affective Lability Scale; ASI-3 = Anxiety Sensitivity Index 3; BEQ = Berkeley Expressivity Questionnaire; BEST = Borderline Evaluation of Severity over Time; DASS = Depression Anxiety Stress Scales; DSHI = Deliberate Self Harm Inventory; BSL = Borderline Symptom List; AUDIT = Alcohol Use Disorders Identification Test.

-- Self-report measure not collected in this sample.

a

Different time periods were specified in the samples. Sample 1: Past 4 months; Sample 2: lifetime.

b

Given the checklist nature of the DSHI and BSL-supplement, it was not considered appropriate to calculate Cronbach's alphas for these measure.

***

p < .001

**

p < .01

*

p < .05.

Emotion regulation and related constructs

The Acceptance and Action Questionnaire (AAQ-9; Hayes et al., 2004) is a nine-item self-report measure of experiential avoidance, or the tendency to avoid unwanted internal experiences (particularly emotions). Items are rated on a 7-point Likert-type scale ranging from 1 (“never true”) to 7 (“always true”), and higher scores indicate greater levels of experiential avoidance. The AAQ-9 has been found to have adequate convergent, discriminant, and concurrent validity (Hayes et al., 2004). This measure was administered to both samples.

The Affect Intensity Measure (AIM; Larsen & Diener, 1987) is a 40-item self-report measure that assesses trait affect intensity and reactivity. Participants rate items on a 6-point Likert-type scale, ranging from 1 (”never”) to 6 (”always”), with higher scores indicating greater affect intensity and reactivity. Although originally developed as a unidimensional measure of affect intensity, research indicates that the AIM is multidimensional, assessing both positive and negative affect intensity and reactivity. This measure was administered to both samples.

The Five Factor Mindfulness Questionnaire (FFMQ; Baer, Smith, Hopkins, Krietemeyer, & Toney, 2006) is a 39-item measure that assesses five distinct facets of mindfulness, including nonreactivity to inner experience, nonjudgment of inner experience, acting with awareness, describing, and observing. Items are rated on a 5-point Likert-type scale ranging from 1 (never or very rarely true) to 5 (very often or always true). Scores on the FFMQ have been found to have good psychometric properties within samples of undergraduates, adults from the community, and experienced meditators (Baer et al., 2006; 2008). The full FFMQ was administered to sample 1 and the acting with awareness and observing subscales were administered to sample 2.

The White Bear Suppression Inventory (WBSI; Wegner & Zanakos, 1994) is a 15-item, self-report measure designed to assess the extent to which individuals suppress, and experience the intrusion of, thoughts. Items are rated on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Evidence has been provided for the test-retest reliability (r = .80; Muris, Merckelbach, & Horselenberg, 1996) and construct validity (Muris et al., 1996; Schmidt et al., 2009; Wegner & Zanakos, 1994) of this measure. This measure was administered to sample 1 only.

The Affective Lability Scale short form (ALS-SF; Oliver & Simmons, 2004) is an 18-item self-report measure of affective instability and rapid shifts in emotional experience. Items are rated on a 4-point Likert-type scale ranging from 0 (very uncharacteristic of me) to 3 (very characteristic of me). The ALS-SF total score demonstrates good construct, convergent, and discriminant validity (Look, Flory, Harvey, & Siever, 2010). This measure was administered to sample 1 only.

The Anxiety Sensitivity Index–3 (ASI-3; Taylor et al., 2007) is an 18-item self-report measure that assesses the fear of anxiety-related sensations due to physical, cognitive, and social concerns. Respondents are asked to rate each item on a 5-point scale from 0 (very little) to 4 (very much) based on their level of agreement. Higher scores are indicative of higher levels of anxiety sensitivity. The ASI-3 has been found to demonstrate adequate reliability and discriminant, convergent, and criterion-related validity (Taylor et al., 2007). This measure was administered in both samples.

The Emotion Amplification and Reduction Scales (TEARS; Hamilton et al., 2007) is an 18-item self-report measure that assesses an individual's perceived ability to modulate the course and intensity of an emotional response. Items are rated on a 4-point Likert-type scale from 1 (not at all true for me) to 4 (very true for me). The measure contains two subscales: emotion reduction (i.e., the perceived ability to reduce the intensity or duration of an emotional response) and emotion amplification (i.e., the perceived ability to prolong or intensify an emotional response). The TEARS demonstrates good internal consistency and construct validity (Hamilton et al., 2007), and was administered to sample 2 only.

The Berkeley Expressivity Questionnaire (BEQ; Gross & John, 1995) is a 16-item self-report measure that assesses three facets of trait emotional expressivity: positive expressivity, negative expressivity, and impulse strength. Participants rate items on a 7-point Likert-type scale ranging from 1 (strongly disagree) to 7 (strongly agree). The BEQ demonstrates adequate convergent and discriminative validity (Gross & John, 1997). The negative expressivity and impulse strength subscales were administered to sample 2 only.

Psychiatric symptoms

The Borderline Evaluation of Severity over Time (BEST; Pfohl et al. 2009) is a 15-item self-report measure of BPD symptom severity, or the degree of impairment from each of the 9 BPD criteria during the past month. Higher scores indicate greater BPD pathology. The BEST has demonstrated good test-retest reliability (r = .62) and convergent and discriminant validity (Pfohl et al. 2009). This measure was administered to both samples.

Clinically-relevant behaviors

The Alcohol Use Disorders Identification Test (AUDIT; Saunders, Aasland, Babor, De La Fuente, & Grant, 1993) is a 10-item self-report measure that assesses hazardous alcohol use and alcohol-related problems over the past year. Items are summed to provide an overall score of alcohol problem severity. The AUDIT has demonstrated good internal consistency and criterion-based validity across a range of clinical and community settings (Reinert & Allen, 2007). This measure was administered to sample 2 only.

Internal consistency values (Cronbach's α) for all relevant measures are presented in Table 4.

Analysis Plan

As in Study 1, Cronbach's α was calculated to evaluate the internal consistency of the 16-item and original 36-item versions of the DERS, and Pearson's product-moment correlations were conducted to evaluate the relative strength of the associations of the DERS versions with the other self-report measures.

In contrast to Study 1, however, we did not conduct significance tests of differences between correlations of the 36-item and 16-item versions of the DERS with the other self-report measures in this study. Specifically, because the 16-item version of the DERS used in these samples was extracted from the original DERS (vs. being administrated separately), all responses to the items in the short form are counted twice in these samples; thus, any random variance or systematic effects that may have occurred in the long form are reproduced in the short form (see Smith, et al., 2000).

Results

Preliminary Analyses

All continuous variables fell within the acceptable range of normality (Tabachnick & Fidell, 2001), with the exception of the deliberate self-harm frequency in both samples 1 and 2 (skewness ≥ 4.26 ; 17.35). Following logarithm transformation, both variables approximated a normal distribution (skewness ≤ 1.18; 1.70). Means and standard deviations of the DERS-16 and the original 36-item DERS in these samples are presented in Table 3.

Reliability

The internal consistency of the 36-item and 16-item versions of the DERS was comparable and excellent in both samples, with a Cronbach's α ranging from .92 to .95.

Validity

The correlations between the 36-item and 16-item versions of the DERS, as well as their respective correlations with the other measures and the differences between these correlations are presented in Table 4. As shown in Table 4, the two versions of the DERS were highly correlated in both samples and evidenced significant correlations of comparable magnitude with most of the other measures of emotion regulation and related constructs (including experiential avoidance, mindfulness, and negative emotionality), psychiatric symptoms, and most clinically-relevant behaviors. Providing some evidence for the discriminant validity of both DERS versions, neither was significantly correlated with positive affect intensity/reactivity (as assessed by the AIM), emotion amplification (as assessed by the TEARS) or the tendency to notice or attend to internal or external experiences (as assessed by the FFMQ Observe subscale). Furthermore, although deliberate self-harm frequency and versatility were not significantly associated with the DERS in sample 1, the strength of the associations between these variables and the 36-item and 16-item versions of the DERS were comparable. Furthermore, as shown in Table 4, the magnitude of the correlations of the 36-item and 16-item versions of the DERS with all included measures were comparable, with the differences between these correlations ranging from r = .00 to r = .13 suggesting minimal differences in the construct validity of both versions.

Discussion

The overarching goal of the present studies was to develop and validate a brief version of the DERS – the DERS-16. Overall, the DERS-16 was shown to have satisfactory psychometric properties, similar to those of the original DERS. In addition, evidence was obtained for the construct validity and reliability of the DERS-16 within three independent and diverse samples. Specifically, the DERS-16 consistently demonstrated high magnitude correlations with the original 36-item version of the DERS, excellent internal consistency, good test-retest reliability, and convergent and discriminant validity. Results suggest that the DERS-16 is a valid and reliable brief self-report measure of overall emotion regulation difficulties.

In terms of convergent validity, the two versions of the DERS demonstrated significant correlations of comparable strength with the majority of the other measures of emotion regulation and related constructs, as well as clinically-relevant behaviors. The only unexpected non-significant association was the lack of a significant association between deliberate self-harm frequency and versatility and both versions of the DERS in sample 1 of Study 2. However, the strength of the associations between these variables and both versions of the DERS were comparable, suggesting that the 16-item version was not less strongly related to this behavior than the original 36-item DERS.

With regard to discriminant validity, neither version of the DERS was significantly correlated with measures of positive emotionality, emotion amplification, or mindful observing. The absence of significant relations between both DERS versions and the measure of positive emotionality is consistent with the emphasis on negative emotions within the conceptualization of emotion regulation difficulties upon which the DERS is based. Likewise, although efforts to control or change emotional responses are inconsistent with the acceptance-based conceptualization of emotion regulation on which the DERS is based, the focus on controlling behaviors in the context of emotional distress and accessing effective strategies for modulating the intensity and/or duration of emotional responses is more relevant to the down-regulation of emotions (as assessed with the TEARS emotion reduction subscale) than the up-regulation of emotions (as assessed with the TEARS emotion amplification subscale). Finally, the absence of significant correlations between both DERS versions and the FFMQ observe subscale is in-line with previous research (Baer et al., 2006), and suggests that the omission of items from the lack of emotional awareness subscale from the DERS-16 did not negatively influence the strength of its relation with measures of awareness-related constructs (relative to the original 36-item DERS).

The DERS-16 was developed and validated in a manner consistent with recommendations for scale development in general (Hinkin, 1998) and addresses several specific methodological issues that are common in short-form development in particular (Smith, et al., 2000). First, when items for the DERS-16 were selected, we combined our review of item-total correlations with a thorough content analysis with several judges to ensure content preservation, removing weak items while simultaneously retaining maximal content coverage. Second, we used independent samples for scale development and evaluation, and Study 1 used independent administrations of the 16-item and original 36-item versions of the DERS. Third, it is often assumed that a high correlation between the short and long form automatically implies evidence of validity for the short form (Smith, et al., 2000). This was not assumed in the present studies. On the contrary, the validity of the two versions of the DERS was assessed in multiple samples (clinical and nonclinical) using a range of measures assessing emotion regulation and related constructs. Finally, we presented data demonstrating that the short form provides meaningful time savings compared to the original DERS without losing reliability or validity.

Several limitations warrant discussion. First, to maximize the strength of the association of the DERS-16 with the original 36-item DERS, no items from the original lack of emotional awareness were included in the DERS-16 (due to lower item-total correlations). Although this decision resulted in a measure in which this theoretically-relevant dimension of emotion regulation is not represented, past research suggests that the exclusion of this dimension from a measure of overall emotion regulation difficulties would not negatively affect its construct validity. For example, Fowler and colleagues (2014) conducted a confirmatory factor analysis of the DERS and found the 6-factor DERS model to be equivalent to a 5-factor DERS model from which the lack of emotional awareness subscale was removed. This finding is consistent with evidence that the lack of emotional awareness subscale may be less relevant to the overarching construct of emotion regulation difficulties as assessed by the DERS (Bardeen et al., 2012). Moreover, there is evidence to suggest that lack of emotional clarity (a dimension of emotion regulation that is represented in the DERS- 16) is more relevant to both psychopathology and the overarching emotion regulation construct than lack of emotional awareness when emotional awareness and clarity are separated as in the DERS (e.g., Ehring, Fischer, Schnülle, Bösterling, & Tuschen-Coffier, 2008; Fox, Axelrod, Paliwal, Sleeper, & Sinha, 2007; Salters-Pedneault, et al., 2006; Tull et al., 2007; Tull, et al., 2010; Tull & Roemer, 2007; Whiteside et al., 2007). Finally, our results demonstrate that, even in the absence of items assessing lack of emotional awareness, the DERS-16 is comparable to the original 36-item DERS in its associations with other related constructs, including a measure of mindful attending. Nonetheless, it is important to note that there are some psychiatric disorders (e.g., autism spectrum disorders) for which lack of emotional awareness may be a particularly relevant dimension of emotion regulation difficulties. Thus, future research is needed to examine the relevance of a lack of emotional awareness to a broad range of psychiatric disorders, as well as the extent to which the DERS-16 adequately captures emotion regulation difficulties across diverse forms of psychopathology.

Second, and in contrast to Study 1, the two versions of the DERS were not administrated separately in Study 2. For this reason, any random variance or systematic effects that may have occurred in the long form were reproduced in the short form. Despite this limitation, Smith et al. (2000) point out that correlating short and long forms of a measure from a single administration is a meaningful step in the pilot phase of research collecting preliminary validity evidence for a new measure. Third, the DERS-16 only provides an overall score of emotion regulation difficulties, whereas the original DERS provides a total score and six subscale scores. Although this may limit the amount of information one can obtain from the DERS-16, there is a substantial amount of research that supports the utility of using a global score to evaluate emotion regulation difficulties within various forms of psychopathology (e.g., Kuo, Khoury, Metcalfe, Fitzpatrick, & Goodwill, 2015; Stevens et al., 2013; Tull, 2006; Tull, Stipelman, Salters-Pedneault, & Gratz, 2009) and in predicting maladaptive behaviors (Gratz & Tull, 2010; Messman-Moore, Walsh, & DiLillo, 2010). That said, future studies with larger samples than used here may benefit from analyzing the factor structure of the DERS-16 to determine if meaningful lower-order factors can be identified.

Finally, two of the three samples in the present studies consisted only of women; thus, the extent to which the results of these studies generalize to men is unclear. However, it is important to note that although previous studies of the original DERS revealed gender differences in some of the DERS subscales (e.g., Gratz & Roemer, 2004; Neumann et al., 2010; Weinberg & Klonsky, 2009), these differences have not generally been found for the total score on this measure (Gratz & Roemer, 2004; Tull et al., 2007; Weinberg & Klonsky, 2009). Moreover, Neumann et al. (2010) found preliminary evidence of gender invariance in the measurement of emotion regulation difficulties on this measure. Nonetheless, future research examining the validity and reliability of the DERS-16 in larger male or mixed-gender samples is needed.

In conclusion, given that the two versions of the DERS show similar psychometric properties, the DERS-16 is a viable replacement for the full-length DERS when a brief assessment of overall emotion regulation difficulties is required. In addition, within certain contexts, the DERS-16 has several advantages compared to the original DERS. For example, clinicians and researchers may find the DERS-16 easier to administer during ongoing sessions or repeatedly when monitoring treatment progress. Patients would also likely find the DERS-16 to be less of a burden than the original 36-item version. Finally, the DERS-16 may be more feasible for large scale studies where the space available for measures is limited (e.g., epidemiological studies).

Acknowledgments

This research was supported by National Institute of Child Health and Human Development Grant R01 HD062226, awarded to the eighth author (DD). Additional funding was provided by the Stockholm County Council (ALF project; grant number: 20140428) and the Fredrik och Ingrid Thurings stiftelse.

Appendix

Difficulties in Emotion Regulation Scale – 16 item version (DERS-16)

graphic file with name nihms723083u1.jpg
Please indicate how often the following statements apply to you by writing the appropriate number from the scale above (1 – 5) in the box alongside each item.

  1. I have difficulty making sense out of my feelings. [CLARITY*]

  2. I am confused about how I feel. [CLARITY]

  3. When I'm upset, I have difficulty getting work done. [GOALS]

  4. When I'm upset, I become out of control. [IMPULSE]

  5. When I'm upset, I believe that I will remain that way for a long time. [STRATEGIES]

  6. When I'm upset, I believe that I'll end up feeling very depressed. [STRATEGIES]

  7. When I'm upset, I have difficulty focusing on other things. [GOALS]

  8. When I'm upset, I feel out of control. [IMPULSE]

  9. When I'm upset, I feel ashamed with myself for feeling that way. [NONACCEPTANCE]

  10. When I'm upset, I feel like I am weak. [NONACCEPTANCE]

  11. When I'm upset, I have difficulty controlling my behaviors. [IMPULSE]

  12. When I'm upset, I believe that there is nothing I can do to make myself feel better. [STRATEGIES]

  13. When I'm upset, I become irritated with myself for feeling that way. [NONACCEPTANCE]

  14. When I'm upset, I start to feel very bad about myself. [STRATEGIES]

  15. When I'm upset, I have difficulty thinking about anything else. [GOALS]

  16. When I'm upset, my emotions feel overwhelming. [STRATEGIES]
Open in a new tab

CLARITY = Lack of Emotional Clarity; GOALS = Difficulties Engaging in Goal-Directed Behavior; IMPULSE = Impulse Control Difficulties; STRATEGIES = Limited Access to Effective Emotion Regulation Strategies; NONACCEPTANCE = Nonacceptance of Emotional Responses.

*

Factor as described in the original study of the 36-item DERS (Gratz & Roemer, 2004). This information was not presented to participants.

References

  1. Baer RA, Smith GT, Hopkins J, Krietemeyer J, Toney L. Using self-report assessment methods to explore facets of mindfulness. Assessment. 2006;13:27–45. doi: 10.1177/1073191105283504. [DOI] [PubMed] [Google Scholar]
  2. Baer RA, Smith GT, Lykins E, Button D, Krietemeyer J, Sauer S, Walsh E, Duggan D, Williams JMG. Construct validity of the five facet mindfulness questionnaire in meditating and nonmeditating samples. Assessment. 2008;15:329–342. doi: 10.1177/1073191107313003. [DOI] [PubMed] [Google Scholar]
  3. Bagby RM, Taylor GJ, Parker JA. The twenty-item Toronto Alexithymia Scale: II. Convergent, discriminant, and concurrent validity. Journal of Psychosomatic Research. 1994;38:33–40. doi: 10.1016/0022-3999(94)90006-x. [DOI] [PubMed] [Google Scholar]
  4. Bardeen JR, Fergus TA, Orcutt HK. An examination of the latent structure of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment. 2012;34:382–392. [Google Scholar]
  5. Ben-Porath DD, Federici A, Wisniewski L, Warren M. Dialectical Behavior Therapy: Does it bring about improvements in affect regulation in individuals with eating disorders? Journal of Contemporary Psychotherapy. 2014;44:245–251. [Google Scholar]
  6. Berna G, Ott L, Nandrino JL. Effects of Emotion Regulation Difficulties on the Tonic and Phasic Cardiac Autonomic Response. PloS one. 2014;9:e102971. doi: 10.1371/journal.pone.0102971. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bohus M, Limberger MF, Frank U, Sender I, Gratwohl T, Stieglitz R. Development of the Borderline Symptom List (BSL) Psychotherapie Psychosomatik Medizinische Psychologie. 2001;51:201–221. doi: 10.1055/s-2001-13281. [DOI] [PubMed] [Google Scholar]
  8. Cicchetti D, Ackerman BP, Izard CE. Emotions and emotion regulation in developmental psychopathology. Development and Psychopathology. 1995;7:1–10. [Google Scholar]
  9. Dixon-Gordon KL, Tull MT, Gratz KL. Self-injurious behaviors in posttraumatic stress disorder: An examination of potential moderators. Journal of Affective Disorders. 2014;166:359–367. doi: 10.1016/j.jad.2014.05.033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Ehring T, Fischer S, Schnülle J, Bösterling A, Tuschen-Caffier B. Characteristics of emotion regulation in recovered depressed versus never depressed individuals. Personality and Individual Differences. 2008;44:1574–1584. [Google Scholar]
  11. Ehring T, Tuschen-Caffier B, Schnülle J, Fischer S, Gross JJ. Emotion regulation and vulnerability to depression: Spontaneous versus instructed use of emotion suppression and reappraisal. Emotion. 2010;10:563–572. doi: 10.1037/a0019010. [DOI] [PubMed] [Google Scholar]
  12. First MB, Gibbon M, Spitzer RL, Williams JBW, Benjamin LS. Structured clinical interview for DSM-IV. Washington, DC: American Psychiatric Press; 1997. [Google Scholar]
  13. Fliege H, Kocalevent RD, Walter OB, Beck S, Gratz KL, Gutierrez PM, Klapp BF. Three assessment tools for deliberate self-harm and suicide behavior: Evaluation and psychopathological correlates. Journal of Psychosomatic Research. 2006;61:113–121. doi: 10.1016/j.jpsychores.2005.10.006. [DOI] [PubMed] [Google Scholar]
  14. Fowler JC, Charak R, Elhai JD, Allen JG, Frueh BC, Oldham JM. Construct validity and factor structure of the difficulties in emotion regulation scale among adults with severe mental illness. Journal of Psychiatric Research. 2014;58:175–180. doi: 10.1016/j.jpsychires.2014.07.029. [DOI] [PubMed] [Google Scholar]
  15. Fox HC, Axelrod SR, Paliwal P, Sleeper J, Sinha R. Difficulties in emotion regulation and impulse control during cocaine abstinence. Drug and Alcohol Dependence. 2007;89:298–301. doi: 10.1016/j.drugalcdep.2006.12.026. [DOI] [PubMed] [Google Scholar]
  16. Goodman M, Carpenter D, Tang CY, Goldstein KE, Avedon J, Fernandez N, et al. Hazlett EA. Dialectical behavior therapy alters emotion regulation and amygdala activity in patients with borderline personality disorder. Journal of Psychiatric Research. 2014;57:108–116. doi: 10.1016/j.jpsychires.2014.06.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gratz KL. Measurement of deliberate self-harm: Preliminary data on the deliberate self-harm inventory. Journal of Psychopathology and Behavioral Assessment. 2001;23:253–263. [Google Scholar]
  18. Gratz KL. Targeting emotion dysregulation in the treatment of self-injury. Journal of Clinical Psychology. 2007;63:1091–1103. doi: 10.1002/jclp.20417. [DOI] [PubMed] [Google Scholar]
  19. Gratz KL, Bardeen JR, Levy R, Dixon-Gordon KL, Tull MT. Mechanisms of change in an emotion regulation group therapy for deliberate self-harm among women with borderline personality disorder. Behaviour Research and Therapy. 2015;65:29–35. doi: 10.1016/j.brat.2014.12.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gratz KL, Hepworth C, Tull MT, Paulson A, Clarke S, Remington B, Lejuez CW. An experimental investigation of emotional willingness and physical pain tolerance in deliberate self-harm: The moderating role of interpersonal distress. Comprehensive Psychiatry. 2011;52:63–74. doi: 10.1016/j.comppsych.2010.04.009. [DOI] [PubMed] [Google Scholar]
  21. Gratz KL, Roemer L. Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment. 2004;26:41–54. [Google Scholar]
  22. Gratz KL, Roemer L. The relationship between emotion dysregulation and deliberate self-harm among female undergraduate students at an urban commuter university. Cognitive Behaviour Therapy. 2008;37:14–25. doi: 10.1080/16506070701819524. [DOI] [PubMed] [Google Scholar]
  23. Gratz KL, Rosenthal MZ, Tull MT, Lejuez CW, Gunderson JG. An experimental investigation of emotion dysregulation in borderline personality disorder. Journal of Abnormal Psychology. 2006;115:850–855. doi: 10.1037/0021-843X.115.4.850. [DOI] [PubMed] [Google Scholar]
  24. Gratz KL, Tull MT. Emotion regulation as a mechanism of change in acceptance- and mindfulness-based treatments. In: Baer RA, editor. Assessing mindfulness and acceptance: Illuminating the processes of change. Oakland, CA: New Harbinger Publications; 2010. pp. 107–134. [Google Scholar]
  25. Gratz KL, Tull MT. Extending research on the utility of an adjunctive emotion regulation group therapy for deliberate self-harm among women with borderline personality pathology. Personality Disorders: Theory, Research, and Treatment. 2011;2:316–326. doi: 10.1037/a0022144. [DOI] [PubMed] [Google Scholar]
  26. Gratz KL, Tull MT. Exploring the relationship between posttraumatic stress disorder and deliberate self-harm: The moderating roles of borderline and avoidant personality disorders. Psychiatry Research. 2012;199:19–23. doi: 10.1016/j.psychres.2012.03.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Gratz KL, Tull MT, Levy R. Randomized controlled trial and uncontrolled 9-month follow-up of an adjunctive emotion regulation group therapy for deliberate self-harm among women with borderline personality disorder. Psychological Medicine. 2014;44:2099–2112. doi: 10.1017/S0033291713002134. [DOI] [PubMed] [Google Scholar]
  28. Gross JJ, Jazaieri H. Emotion, emotion regulation, and psychopathology: An affective science perspective. Clinical Psychological Science. 2014;2:387–401. [Google Scholar]
  29. Gross JJ, John OP. Facets of emotional expressivity: Three self-report factors and their correlates. Personality and Individual Differences. 1995;19:555–568. [Google Scholar]
  30. Gross JJ, John OP. Revealing feelings: Facets of emotional expressivity in self-reports, peer ratings, and behavior. Journal of Personality and Social Psychology. 1997;72:435–448. doi: 10.1037//0022-3514.72.2.435. [DOI] [PubMed] [Google Scholar]
  31. Gross JJ, John OP. Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology. 2003;85:348–362. doi: 10.1037/0022-3514.85.2.348. [DOI] [PubMed] [Google Scholar]
  32. Hamilton NA, Karoly P, Gallagher M, Stevens N, Karlson C, McCurdy D. The assessment of emotion regulation in cognitive context: The emotion amplification and reduction scales. Cognitive Therapy and Research. 2007;33:255–263. [Google Scholar]
  33. Hayes SC, Strosahl K, Wilson KG, Bissett RT, Pistorello J, Toarmino D, et al. McCurry SM. Measuring experiential avoidance: A preliminary test of a working model. The Psychological Record. 2004;54:553–578. [Google Scholar]
  34. Henry JD, Crawford JR. The short-form version of the depression anxiety stress scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British Journal of Clinical Psychology. 2005;44:227–239. doi: 10.1348/014466505X29657. [DOI] [PubMed] [Google Scholar]
  35. Hinkin TR. A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods. 1998;1:104–121. [Google Scholar]
  36. Hofmann SG, Kashdan TB. The affective style questionnaire: Development and psychometric properties. Journal of Psychopathology and Behavioral Assessment. 2010;32:255–263. doi: 10.1007/s10862-009-9142-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Izard CE, Ackerman BP. Motivational, organizational, and regulatory functions of discrete emotions. In: Lewis M, Haviland-Jones JM, editors. Handbook of emotions. 2nd. New York: Guilford Press; 2000. pp. 253–264. [Google Scholar]
  38. Kuo JR, Khoury JE, Metcalfe R, Fitzpatrick S, Goodwill A. An examination of the relationship between childhood emotional abuse and borderline personality disorder features: the role of difficulties with emotion regulation. Child Abuse and Neglect. 2015;39:147–155. doi: 10.1016/j.chiabu.2014.08.008. [DOI] [PubMed] [Google Scholar]
  39. Kring AM, Werner KH. Emotion regulation in psychopathology. In: Philippot P, Feldman RS, editors. The regulation of emotion. Mahwah, NJ: Lawrence Erlbaum Associates; 2004. pp. 359–385. [Google Scholar]
  40. Larsen RJ, Diener E. Affect intensity as an individual difference characteristic: A review. Journal of Research in Personality. 1987;21:1–39. [Google Scholar]
  41. Lavender JM, Wonderlich SA, Peterson CB, Crosby RD, Engel SG, Mitchell JE, et al. Berg KC. Dimensions of emotion dysregulation in bulimia nervosa. European Eating Disorders Review. 2014;22:212–216. doi: 10.1002/erv.2288. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Li CR, Huang C, Yan P, Bhagwagar Z, Milivojevic V, Sinha R. Neural correlates of impulse control during stop signal inhibition in cocaine-dependent men. Neuropsychopharmacology. 2008;33:1798–1806. doi: 10.1038/sj.npp.1301568. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Look AE, Flory JD, Harvey PD, Siever LJ. Psychometric properties of a short form of the affective lability scale (ALS-18) Personality and Individual Differences. 2010;49:187–191. doi: 10.1016/j.paid.2010.03.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Messman-Moore TL, Walsh KL, DiLillo D. Emotion dysregulation and risky sexual behavior in revictimization. Child Abuse and Neglect. 2010;34:967–976. doi: 10.1016/j.chiabu.2010.06.004. [DOI] [PubMed] [Google Scholar]
  45. Muris P, Merckelbach H, Horselenberg R. Individual differences in thought suppression. The white bear suppression inventory: Factor structure, reliability, validity, and correlates. Behaviour Research and Therapy. 1996;34:501–513. doi: 10.1016/0005-7967(96)00005-8. [DOI] [PubMed] [Google Scholar]
  46. Neumann A, van Lier PAC, Gratz KL, Koot HM. Multidimensional assessment of emotion regulation difficulties in adolescents using the difficulties in emotion regulation scale. Assessment. 2010;17:138–149. doi: 10.1177/1073191109349579. [DOI] [PubMed] [Google Scholar]
  47. Oliver MN, Simmons JS. The affective lability scales: Development of a short-form measure. Personality and Individual Differences. 2004;37:1279–1288. [Google Scholar]
  48. Pfohl B, Blum N, St John D, McCormick B, Allen J, Black DW. Reliability and validity of the borderline evaluation of severity over time (BEST): A self-rated scale to measure severity and change in persons with borderline personality disorder. Journal of Personality Disorders. 2009;23:281–293. doi: 10.1521/pedi.2009.23.3.281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Reinert DF, Allen JP. The alcohol use disorders identification test: An update of research findings. Alcoholism: Clinical and Experimental Research. 2007;31:185–199. doi: 10.1111/j.1530-0277.2006.00295.x. [DOI] [PubMed] [Google Scholar]
  50. Salters-Pedneault K, Roemer L, Tull MT, Rucker L, Mennin DS. Evidence of broad deficits in emotion regulation associated with chronic worry and generalized anxiety disorder. Cognitive Therapy and Research. 2006;30:469–480. [Google Scholar]
  51. Saunders JB, Aasland OG, Babor TF, De La Fuente JR, Grant M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption-II. Addiction. 1993;88:791–804. doi: 10.1111/j.1360-0443.1993.tb02093.x. [DOI] [PubMed] [Google Scholar]
  52. Schmidt RE, Gay P, Courvoisier D, Jermann F, Ceschi G, David M, Brinkmann K, Van der Linden M. Anatomy of the white bear suppression inventory (WBSI): A review of previous findings and a new approach. Journal of Personality Assessment. 2009;91:323–330. doi: 10.1080/00223890902935738. [DOI] [PubMed] [Google Scholar]
  53. Sheehan D, Janavs J, Baker R, Lecrubier Y, Hergueta T, Weiller E. MINI international neuropsychiatric interview, Swedish version 5.0 2010 [Google Scholar]
  54. Sheppes G, Suri G, Gross JJ. Emotion Regulation and Psychopathology. Annual Review of Clinical Psychology. 2015;11:379–405. doi: 10.1146/annurev-clinpsy-032814-112739. [DOI] [PubMed] [Google Scholar]
  55. Simons JS, Gaher RM. The distress tolerance scale: Development and validation of a self-report measure. Motivation and Emotion. 2005;29:83–102. [Google Scholar]
  56. Smith GT, McCarthy DM, Anderson KG. On the sins of short-form development. Psychological Assessment. 2000;1:102–111. doi: 10.1037//1040-3590.12.1.102. [DOI] [PubMed] [Google Scholar]
  57. Steiger JH. Tests for comparing elements of a correlation matrix. Psychological Bulletin. 1980;87:245–251. [Google Scholar]
  58. Stevens NR, Gerhart J, Goldsmith RE, Heath NM, Chesney SA, Hobfoll SE. Emotion regulation difficulties, low social support, and interpersonal violence mediate the link between childhood abuse and posttraumatic stress symptoms. Behavior Therapy. 2013;44:152–161. doi: 10.1016/j.beth.2012.09.003. [DOI] [PubMed] [Google Scholar]
  59. Tabachnick BG, Fidell LS. Using multivariate statistics. California State University Northridge: HarperCollins College Publishers; 2001. [Google Scholar]
  60. Taylor CT, Laposa JM, Alden LE. Is avoidant personality disorder more than just social avoidance? Journal of Personality Disorders. 2004;18:571–594. doi: 10.1521/pedi.18.6.571.54792. [DOI] [PubMed] [Google Scholar]
  61. Taylor S, Zvolensky MJ, Cox BJ, Deacon B, Heimberg RG, Ledley DR, et al. Cardenas SJ. Robust dimensions of anxiety sensitivity: Development and initial validation of the anxiety sensitivity index-3. Psychological Assessment. 2007;19:176–188. doi: 10.1037/1040-3590.19.2.176. [DOI] [PubMed] [Google Scholar]
  62. Thompson RA. Emotion regulation: A theme in search of definition. In: Fox NA, editor. The development of emotion regulation: Biological and behavioral considerations. Chicago: University of Chicago Press; 1994. pp. 25–52. [PubMed] [Google Scholar]
  63. Tull MT. Extending an anxiety sensitivity model of uncued panic attack frequency and symptom severity: The role of emotion dysregulation. Cognitive Therapy and Research. 2006;30:177–184. [Google Scholar]
  64. Tull MT, Bardeen JR, DiLillo D, Messman-Moore T, Gratz KL. A prospective investigation of emotion dysregulation as a moderator of the relation between posttraumatic stress symptoms and substance use severity. Journal of Anxiety Disorders. 2015;29:52–60. doi: 10.1016/j.janxdis.2014.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Tull MT, Barrett HM, McMillan ES, Roemer L. A preliminary investigation of the relationship between emotion regulation difficulties and posttraumatic stress symptoms. Behavior Therapy. 2007;38:303–313. doi: 10.1016/j.beth.2006.10.001. [DOI] [PubMed] [Google Scholar]
  66. Tull MT, Gratz KL, Latzman RD, Kimbrel NA, Lejuez CW. Reinforcement sensitivity theory and emotion regulation difficulties: A multimodal investigation. Personality and Individual Differences. 2010;49:989–994. [Google Scholar]
  67. Tull MT, Roemer L. Emotion regulation difficulties associated with the experience of uncued panic attacks: Evidence of experiential avoidance, emotional nonacceptance, and decreased emotional clarity. Behavior Therapy. 2007;38:378–391. doi: 10.1016/j.beth.2006.10.006. [DOI] [PubMed] [Google Scholar]
  68. Tull MT, Stipelman BA, Salters-Pedneault K, Gratz KL. An examination of recent non-clinical panic attacks, panic disorder, anxiety sensitivity, and emotion regulation difficulties in the prediction of generalized anxiety disorder in an analogue sample. Journal of Anxiety Disorders. 2009;23:275–282. doi: 10.1016/j.janxdis.2008.08.002. [DOI] [PubMed] [Google Scholar]
  69. Tull MT, Weiss NH, Adams CE, Gratz KL. The contribution of emotion regulation difficulties to risky sexual behavior within a sample of patients in residential substance abuse treatment. Addictive Behaviors. 2012;37:1084–1092. doi: 10.1016/j.addbeh.2012.05.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Turner BJ, Layden BK, Butler SM, Chapman AL. How often, or how many ways: Clarifying the relationship between non-suicidal self-injury and suicidality. Archives of Suicide Research. 2013;17:397–415. doi: 10.1080/13811118.2013.802660. [DOI] [PubMed] [Google Scholar]
  71. Wegner DM, Zanakos S. Chronic thought suppression. Journal of Personality. 1994;62:615–640. doi: 10.1111/j.1467-6494.1994.tb00311.x. [DOI] [PubMed] [Google Scholar]
  72. Weinberg A, Klonsky ED. Measurement of emotion dysregulation in adolescents. Psychological Assessment. 2009;21:616–621. doi: 10.1037/a0016669. [DOI] [PubMed] [Google Scholar]
  73. Whiteside U, Chen E, Neighbors C, Hunter D, Lo T, Larimer M. Difficulties regulating emotions: Do binge eaters have fewer strategies to modulate and tolerate negative affect? Eating Behaviors. 2007;8:162–169. doi: 10.1016/j.eatbeh.2006.04.001. [DOI] [PubMed] [Google Scholar]
  74. Zou GY. Toward using confidence intervals to compare correlations. Psychological Methods. 2007;12:399–413. doi: 10.1037/1082-989X.12.4.399. [DOI] [PubMed] [Google Scholar]

RESOURCES