Confirmatory Validation of the Factor Structure and Psychometric Properties of the Difficulties in Emotion Regulation Scale – Positive

Abstract

Objectives:

Emotion dysregulation is a transdiagnostic factor central to the etiology and treatment of various clinical difficulties. Yet, research in this area has focused almost exclusively on emotion dysregulation stemming from negative emotions. The current study confirmed the factor structure of the Difficulties in Emotion Regulation Scale – Positive (DERS-P) and further examined its reliability and validity.

Method:

Participants in Study 1 were 229 college students (M age = 19.37 years; 66.8% female; 67.2% White). Participants in Study 2 were 353 trauma-exposed community individuals (M age = 35.77 years; 57.8% female; 71.2% White).

Results:

Findings supported the three-factor structure of the DERS-P. Mean levels of the DERS-P scales demonstrated convergent and discriminant validity and differentiated individuals with (vs. without) probable PTSD, depression, alcohol use, and drug use disorders.

Conclusions:

Findings provide additional support for the factor structure, reliability, and validity of the DERS-P, thereby adding to its clinical utility.

Since the early 1990s, there has been exponential growth in research on emotion dysregulation (Tull & Aldao, 2015). The findings of these studies provide support for a robust association between emotion dysregulation and a wide range of psychological difficulties (e.g., posttraumatic stress disorder [PTSD], depression; Mennin, Heimberg, Turk, & Fresco, 2002; Weiss, Tull, Anestis, & Gratz, 2013) and risky behaviors (e.g., substance use; Weiss, Tull, Viana, Anestis, & Gratz, 2012). The relevance of emotion dysregulation to these outcomes has been documented among diverse populations, such as those that vary in age (Orgeta, 2009), gender (Turk, Heimberg, Luterek, Mennin, & Fresco, 2005), and race (Weiss, Tull, Davis, et al., 2012). Further, research indicates that emotion dysregulation is an important mechanism in the treatment of various psychological difficulties and risky behaviors (Gratz, Weiss, & Tull, 2015). The above literature highlights emotion dysregulation as a transdiagnostic vulnerability factor underlying psychological difficulties and risky behaviors across an array of populations.

As defined here, emotion dysregulation is a multi-faceted construct involving maladaptive ways of responding to emotions, including (a) lack of emotional awareness, clarity and acceptance; (b) behavioral dyscontrol in the context of intense emotions; (c) unwillingness to pursue meaningful activities in the context of emotional distress; and (d) inflexible use of adaptive strategies to modulate (vs. eliminate) the intensity and/or duration of emotional experiences (Gratz & Roemer, 2004). Although consistent with well-established conceptualizations that define emotion dysregulation as maladaptive responses to emotions (e.g., Cole, Michel, & Teti, 1994; Thompson & Calkins, 1996), this definition differs from other extant frameworks of emotion dysregulation. For instance, emotion regulation is not defined here as the control of emotions and reduction of emotional arousal (e.g., Zeman & Garber, 1996). Such a framework implies that intense emotions are in some way problematic and must be controlled, a solution that actually runs counter to effective emotion regulation as defined here. Indeed, empirical evidence suggests that efforts to control emotions have paradoxical effects, increasing their frequency, intensity, and duration (Hayes, Luoma, Bond, Masuda, & Lillis, 2006). Thus, the conceptualization proposed here defines emotion regulation as control of behavior in the context of emotions, rather than efforts to control emotions themselves.

Additionally, the definition of emotion regulation described here does not equate emotion dysregulation with the temperamental characteristic of emotional intensity/reactivity (e.g., Livesley, Jang, & Vernon, 1998). Such a framework implies that intense, reactive, or long-lasting emotions are fundamentally dysregulated. While there is some evidence that individuals who have these emotional experiences are more likely to exhibit emotion dysregulation (e.g., Salsman & Linehan, 2012; Weiss, Williams, & Connolly, 2015), other studies suggest that this relation is not direct, with intense, reactive, or last-lasting emotions being related to emotion dysregulation through other key mechanisms (e.g., Gratz & Tull, 2010). Indeed, as defined here, nature/quality of emotions is separate from responses to emotions.

Notably, research to date has focused almost exclusively on emotion dysregulation stemming from negative emotions. Positive emotion disturbance has been linked to a wide array of clinically relevant outcomes (e.g., Brown and Barlow, 2009; Gruber, 2011). As examples, individuals with depressive disorders (e.g., Brown, 2007), PTSD (e.g., Miller, 2003), and substance use disorders (e.g., Swendsen, Conway, Rounsaville, & Merikangas, 2002) have been shown to exhibit lower levels of positive affectivity, whereas elevated levels of positive affectivity have been found among those with bipolar disorder (e.g., Gruber et al., 2008). Thus, although we generally expect people to desire and enjoy positive emotions, some individuals may struggle with the experience and regulation of such emotions. Taken together, findings provide support for the clinical utility of examining other aspects of positive emotion disturbance, such as difficulties regulating positive emotions.

As has been true for other emotion regulation constructs (e.g., Cyders et al., 2007 [UPPS-P Positive Urgency]; Gross & John, 2003 [Emotion Regulation Questionnaire]; Weiss, Tull, Dixon-Gordon, & Gratz, 2018 [Risky Behavior Questionnaire Negative and Positive Scales]), the conceptualization and measurement of emotion regulation applies to both negative and positive emotional experiences. Thus, to explore individual’s responses to positive emotions, Weiss, Gratz, and Lavender (2015) developed the Difficulties in Emotion Regulation Scale – Positive (DERS-P), a comprehensive measure that assesses abilities underlying the effective regulation of positive emotions that aligns with the Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004). Exploratory factor analysis (EFA) of the DERS-P provided support for a three-factor structure: nonacceptance of positive emotions (Accept), difficulties controlling impulsive behaviors when experiencing positive emotions (Impulse), and difficulties engaging in goal-directed behaviors in the context of positive emotions (Goals). In an initial psychometric study (Weiss, Gratz, et al., 2015), which used the derivation sample of college students, the DERS-P facets demonstrated high internal consistency (αs = .87, .83, and .86 for Accept, Goals, and Impulse, respectively) and construct validity. For instance, each of the DERS-P scales were significantly related to difficulties regulating negative emotions (rs ranging from .28 to .34) and expectancies for negative mood regulation (rs ranging from −.19 to −.25). Further, none of the DERS-P scales were significantly associated with negative emotional intensity and reactivity (rs ranging from −.03 to .09) or positive self-expressiveness in the family context (rs ranging from −.10 to .07). However, only some of the DERS-P scales were significantly related to other clinically relevant constructs. As an example, Accept (r = −.15) and Goals (r = −.10), but not Impulse (r = .04), were significantly related to less positive emotional expressivity, albeit each of these relations were small in magnitude. These findings provide support for the convergent and discriminant validity of the DERS-P. Of note, while Impulse demonstrates conceptual overlap with specific subscales of other measures (i.e., positive urgency [Cyders et al., 2007]; the positive emotion-dependent nature of risky behaviors [Weiss, Tull, Dixon-Gordon, & Gratz, 2018]), we are not aware of any other measure examining Accept or Goals, or any comprehensive assessment of difficulties regulating positive emotions.

Regarding the specific subscales, Accept refers to the tendency to take a judgmental or evaluative stance toward positive emotions (e.g., “When I’m happy, I become scared and fearful of those feelings”). Individuals who report elevated levels of Accept may judge positive emotion states to be undesirable, unpredictable, and/or frightening. Although somewhat counterintuitive – in that one might expect individuals to be more accepting of positive emotional experiences – evidence for this construct is provided in the literature. For instance, studies have found that individuals may experience negative emotion states in response to positive stimuli (Frewen, Dean, et al., 2012; Frewen, Dozois, et al., 2012). Other research suggests that individuals may be nonaccepting of the arousal associated with positive emotions in particular. For instance, Roemer et al. (2001) hypothesized that, through stimulus generalization, fear of physiological arousal originally associated with negative emotional experiences may expand to positive emotions.

Goals refers to the ability to continue present moment activities in the context of positive emotions (e.g., “When I’m happy, I have difficulty focusing on other things”). Support for this domain of difficulties regulating positive emotions comes from research on the cognitive and attentional consequences of positive emotions. For instance, individuals have been shown to exhibit increased distractibility (Dreisbach & Goschke, 2004) and less discriminative use of information (Forgas, 1992) in the context of positive emotions. These changes following positive emotions may interfere with the ability to attend to goal-directed behavior.

Lastly, Impulse refers to the tendency to engage in rash action in the context of positive emotional experiences (e.g., “When I’m happy, I have difficulty controlling my behaviors”). Existing literature suggests that elevated levels of Impulse are related to a failure to take into account future consequences and a poor capacity for prepotent response inhibition in the context of positive emotions. Regarding the former, following the experience of positive emotions, individuals tend to have a heightened focus on immediate needs, without consideration of long-term consequences (Cyders & Coskunpinar, 2012). As a result, individuals may be more likely to engage in behaviors that address their short- (but not long-) term needs in the context of positive emotions. Regarding the latter, research has found that it is more difficult to deliberately control or suppress an automatic response in the context of positive emotions (Billieux et al., 2010). This suggests that individuals may be less likely to inhibit rash action in positive emotional contexts.

Preliminary evidence suggests that difficulties regulating positive emotions may be particularly relevant to individuals who exhibit psychological difficulties and risky behaviors. For instance, PTSD has been linked to heightened physiological responding to positive emotional stimuli (Litz, Orsillo, Kaloupek, & Weathers, 2000), which may be experienced as aversive because of its association with trauma-related symptoms (Taylor, Koch, & McNally, 1992). Positive emotions may also impair behavioral control in PTSD, including the capacity to inhibit impulsive or reward-seeking behaviors, perhaps due to impaired regulatory control capacity (Frewen & Lanius, 2006) or disadvantageous decision-making (Goschke, 2014). Symptoms of depression have been found to be related to secondary negative affective emotional states in response to stimuli that are typically positive (DePierro et al., 2017). Further, Beblo et al. (2012) found that individuals with (vs. without) depression were more likely to engage in efforts to suppress positive emotions, and negative evaluations of positive emotions (i.e., judging them as frightening) was related to greater suppression of positive emotions. Finally, a tendency toward delay discounting and prepotent response inhibition in the context of positive emotions may increase risk for alcohol and drug misuse, which demonstrate associations with these mechanisms (Fillmore & Rush, 2003; Kollins, 2003).

In summary, emerging theoretical and empirical literature supports the clinical utility of examining difficulties regulating positive emotions. In an initial examination, the DERS-P was found to have good reliability and validity (Weiss, Gratz, et al., 2015). As an essential next step, the current study sought to confirm the factor structure of the DERS-P beyond the initial validation study, as well as further test the reliability and validity of the DERS-P across two samples, namely both community and undergraduate samples. In Study 1, we examine the factor structure, reliability, and convergent and discriminant validity of the DERS-P. In Study 2, we examine the DERS-P factor structure, reliability, and its utility in differentiating samples based on level of probable clinical disturbance. Consistent with findings from the initial validation study (Weiss, Gratz, et al., 2015), we expected to find factorial support for three facets of positive emotion dysregulation (i.e., Accept, Impulse, and Goals). Further, we hypothesized that these three facets would have high internal consistency. Finally, regarding validity, we expected that the DERS-P facets would demonstrate good convergent and discriminant validity as well as differentiate between individuals with (vs. without) clinically significant levels of psychological difficulties (i.e., PTSD and depression) and risky behaviors (i.e., alcohol and drug use disorders). We expected the DERS-P to be significantly, moderately associated with other measures of positive emotion dysregulation, particularly those domains that exhibit greater conceptual overlap: Impulse with positive urgency and Accept with positive emotional avoidance. Further, as has been found to be true of measures of emotion dysregulation (Gratz & Roemer, 2004), we expected the DERS-P to be statistically related to measures of negative emotion dysregulation, albeit weakly. Finally, consistent with research on negative emotion dysregulation, we expected higher levels of DERS-P among individuals with vs. without probable PTSD (Weiss et al., 2013), depression (Dixon-Gordon et al., 2015), alcohol use disorders (Fox et al., 2008), and drug use disorders (Fox, Axelrod, Paliwal, Sleeper, & Sinha, 2007).

Method

Participants and Procedures

Study 1

All procedures were reviewed and approved by the University of New Haven Institutional Review Board. Participants were recruited using a psychology subject pool and completed measures through an online survey. As compensation, participants received extra credit. Participants were 311 young adults enrolled in a large private university located in the northeast United States. Of these, 58 participants were excluded for completing the questionnaire in under 30 minutes (as a validity index to ensure attentive responding; remaining n = 253); an additional 24 participants were excluded for missing more than 30% item-level data on any primary variable of interest (see Measures). The final college student sample included 229 respondents. See Table 1 for demographic data.

Table 1.

Sample Characteristics

	Study 1		Study 2
	M (SD)	n (%)	M (SD)	n (%)
Age	19.37 (4.12)		35.76 (11.12)
Sex
Male		76 (33.2%)		132 (40.8%)
Female		153 (66.8%)		187 (57.8%)
Other		0 (0.0%)		4 (1.4%)
Ethnicity
Hispanic/Latino/a		42 (13.0%)		34 (14.8%)
Non-Hispanic/Latino/a		276 (85.3%)		187 (81.7%)
Racial/Ethnic Background
Caucasian/White		154 (67.2%)		230 (71.2%)
African American/Black		27 (11.8%)		34 (10.5%)
Asian		9 (3.9%)		38 (11.9%)
American Indian/Alaskan Native		1 (0.4%)		16 (5.1%)
Native Hawaiian/other Pacific Islander		0 (0.0%)		2 (0.6%)
Hispanic		21 (9.2%)		42 (13.0%)
Years of School Completed	12.45 (0.77)		15.31 (2.48)
Current Employment Status
Full-time		7 (3.1%)		232 (71.8%)
Part-time		78 (34.1%)		50 (15.5%)
Retired		0 (0.0%)		11 (3.4%)
Unemployed		140 (61.1%)		30 (9.3%)
Income – Study 2
Less than $15,000				26 (8.2%)
$15,000 to $24,999				46 (14.2%)
$25,000 to $34,999				49 (15.3%)
$35,000 to $49,999				43 (13.3%)
$50,000 to $64,999				62 (19.3%)
$65,000 to $79,999				28 (8.8%)
$80,000 or higher				68 (21.0%)
Income – Study 1
Less than $9,999		6 (3.6%)
$10,000 to $19,999		4 (2.4%)
$20,000 to $29,999		14 (8.4%)
$30,000 to $39,999		9 (5.4%)
$40,000 to $49,999		11 (6.6%)
$50,000 to $59,999		15 (9.0%)
$60,000 to $69,999		15 (9.0%)
$70,000 to $79,999		11 (6.6%)
$80,000 to $89,999		7 (4.2%)
$90,000 to $99,999		8 (4.8%)
$100,000 or higher		66 (39.8%)

Study 2

Participants were recruited from Amazon’s Mechanical Turk (MTurk) platform. Beyond generating reliable data (Buhrmester, Kwang, & Gosling, 2011; Shapiro, Chandler, & Mueller, 2013), MTurk’s subject pool is generally more demographically representative of the general population than typical convenience samples of undergraduate students (Paolacci & Chandler, 2014). Further, MTurk’s subject pool represents the prevalence of some mental health problems such as trauma exposure rates, clincial depression, and substance use in the general population (Shapiro et al., 2013). Findings of Mishra and Carleton (2017) indicate that MTurk’s subject pool is more representative than most convience samples used in behavioral research.

The Institutional Review Board of University of North Texas approved this study. It was described as a 45- to 60-minute self-report survey to develop a novel measure assessing risky behaviors among individuals with stressful life experiences. Participants 18 years and older were screened for three inclusionary criteria: (1) living in North America; (2) working knowledge of the English language; and (3) experience of a traumatic event screened with the Criterion A question of the Primary Care PTSD Screen (Prins et al., 2015). Participants who met eligibility criteria provided informed consent and completed the survey on Qualtrics. Participants were compensated $1.25 USD for study participation in accordance with recommended compensation parameters (Barger, Benrend, Sharek, & Sinar, 2011; Schmidt, 2015).

Of the obtained 891 responses, duplicate responses from 18 participants who attempted to answer the questionnaire multiple times were excluded (47 responses; remainder n = 844). We then excluded 150 participants not meeting one or more inclusionary criteria (remainder n = 694), 122 participants who failed any of four validity checks interspersed in the study to ensure attentive responding and comprehension (remainder n = 572; Meade & Craig, 2012; Oppenheimer, Meyvis, & Davidenko, 2009; Thomas & Clifford, 2017), and 97 participants for missing data on all measures (remainder n = 475). Using data obtained from the Life Event Checklist for DSM-5 (LEC-5; Weathers, Blake, et al., 2013), we excluded 11 participants who either did not endorse a traumatic event, or did not identify their most distressing traumatic event (remainder n = 464). Finally, we excluded 111 participants missing more than 30% item-level data on any primary variable of interest (see Measures). The final MTurk sample included 353 trauma-exposed participants. See Table 1 for additional demographic data.

Measures

The DERS-P (Weiss, Gratz, et al., 2015) is a 13-item self-report measure that assesses individuals’ typical levels of positive emotion dysregulation across three domains: DERS-P Accept, DERS-P Goals, and DERS-P Impulse. Participants rate each item using a 5-point Likert-type scale (1 = almost never, 5 = almost always). Higher scores indicate greater difficulties regulating positive emotions (see Introduction for reliability and validity data from the measure development study). The DERS-P was administered in Study 1 and Study 2. Cronbach’s α was 0.89 in Study 1 and 0.95 in Study 2.

The Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004) is a 36-item self-report measure that assesses individuals’ typical levels of negative emotion dysregulation including nonacceptance of negative emotions (DERS Accept), difficulties engaging in goal-directed behaviors when distressed (DERS Goals), and difficulties controlling impulsive behaviors when distressed (DERS Impulse). Participants rate the extent to which each item applies to them on a 5-point Likert-type scale (1 = almost never, 5 = almost always). Of note, the DERS has six domains; two domains (emotional clarity and awareness) do not reference emotion valence; items assessing a third domain (limited access to effective emotion regulation strategies) were not included in the DERS-P. Higher scores indicate greater difficulties regulating negative emotions. The DERS was administered in Study 1 to test the validity of the DERS-P. Cronbach’s α was 0.87.

The Emotional Avoidance Questionnaire (EAQ; Taylor, Laposa, & Alden, 2004) is a 10-item self-report measure that assesses avoidance of positive (EAQ-Positive; e.g., “If I start feeling strong positive emotions, I prefer to leave the situation”) and negative (EAQ-Negative; e.g., “When I feel anxious or worried about something, I try to ignore it as much as I can”) emotions. Each subscale is composed of 5 items. Participants rate items on a 5-point scale (1 = not true of me, 5 = very true of me). The EAQ was administered in Study 1 to test the validity of the DERS-P. Cronbach’s α was 0.87 and 0.79 for the positive and negative scales, respectively.

The Urgency, Premeditation, Perseverance, Sensation Seeking, and Positive Urgency Impulsive Behavior Scale (UPPS-P; Cyders et al., 2007; Whiteside & Lynam, 2001) is a 59-item self-report questionnaire that assesses multiple dimensions of impulsivity, including negative urgency, positive urgency, sensation seeking, lack of premeditation, and lack of perseverance. Participants rate each item on a 4-point Likert-type scale (1 = rarely/never true, 4 = almost always/always true). Given their overlap with DERS and DERS-P Impulse, we used the negative and positive urgency scales, which measure the tendency to engage in risky behaviors when experiencing intense negative and positive emotions, respectively (Cyders et al., 2007; Weiss et al., 2015). The UPPS-P was administered in Study 1 to test the validity of the DERS-P. Cronbach’s α was 0.94 and 0.87 for the positive and negative urgency scales, respectively.

The LEC-5 (Weathers, Blake, et al., 2013) is a 17-item self-report measure designed to screen for traumatic events in a respondent’s lifetime. It assesses exposure to 16 traumatic events and the 17^th item assesses any other stressful event not captured in the first 16 items. For each event, the respondent is asked to indicate if: (a) it happened to them, (b) they witnessed it, (c) they learned about it, (d) they experienced it as part of their job, (e) they aren’t sure if they experienced it, or (f) they didn’t experience it. Either of the first four response options indicated a positive Criterion A traumatic event endorsement (American Psychiatric Association, 2013). The LEC-5 was utilized in Study 1 and Study 2 to test the validity of the DERS-P.

The PTSD Checklist for DSM-5 (PCL-5; Weathers, Litz, et al., 2013) is a 20-item self-report measure that assesses past-month PTSD symptoms consistent with the DSM-5 (American Psychiatric Association, 2013). Participants completed the PCL-5 in response to the most distressing traumatic event endorsed on the LEC-5. Each item was rated using a 5-point Likert-type scale (0 = not at all, 4 = extremely). The PCL-5 has a recommended cut-off score of 31 or higher to identify probable PTSD diagnosis (Blevins, Weathers, Davis, Witte, & Domino, 2015; Bovin et al., 2016). The PCL-5 was utilized in Study 1 and Study 2 to test the validity of the DERS-P. Cronbach’s α was .94 in Study 1 and .96 in Study 2.

The Depression Anxiety Stress Scales (DASS-21; Lovibond & Lovibond, 1995) depression scale is a 6-item self-report measure assessing depression symptoms. The four response options range from 0 (did not apply to me at all) to 3 (applied to me very much or most of the time). Scores of 10 or higher indicate probable depression. The DASS-21 was utilized in Study 1 to test the validity of the DERS-P. Cronbach’s α was .91 in Study 1.

The Patient Health Questionnaire-9 (PHQ-9; Kroenke & Spitzer, 2002) is a 9-item self-report measure assessing depression symptoms. The four response options range from 0 (not at all) to 3 (nearly every day) (Kroenke, Spitzer, & Williams, 2001). The PHQ-9 has a cut-off score of 10 or higher to identify probable depression (Fine et al., 2013). The PHQ-9 was utilized in Study 2 to test the validity of the DERS-P. Cronbach’s α was .91 in Study 2.

The Alcohol Use and Disorders Identification Test (AUDIT; Saunders, Aasland, Barbor, De la Fuente, & Grant, 1993) is a 10-item self-report measure assessing heavy drinking and active alcohol use disorder with a 5-point Likert scale, with higher scores indicating greater alcohol misuse. In Study 1, all AUDIT items were administered. In Study 2, only the Consumption items were administered (consistent with recommendations by Bush, Kivlahan, McDonell, Fihn, & Bradley, 1998). A score of 8 or higher in Study 1 was associated with probable alcohol use disorder (Babor, Higgins-Biddle, Saunders, & Monteiro, 2001). A score of 5 or higher in Study 2 was associated with probable alcohol use disorder (Dawson, Grant, Stinson, & Zhou, 2005). The AUDIT was utilized in Study 1 and Study 2 to test the validity of the DERS-P. Cronbach’s α was .79 in Study 1 and .92 in Study 2.

The Drug Abuse Screening Test (DAST; Skinner, 1982) is a 10-item self-report measure used to assess drug misuse, including occupational or relational problems, illegal activities, or regret in the past 30 days. Responses to each item have 1 (yes) and 0 (no) options. A score of 3 or higher indicates probable drug use disorder. The DAST was utilized in Study 1 and Study 2 to test the validity of the DERS-P. Cronbach’s α was .59 in Study 1 and .84 in Study 2.

Demographic information.

Information regarding age, gender, ethnicity, race, income, educational level, and employment status was obtained for Study 1 and Study 2.

Data Analysis

Preliminary analyses.

Preliminary analyses were conducted using SPSS 24 to examine assumptions of a general linear model (i.e., normality, linearity, homoscedasticity, and multicollinearity) and missing data patterns, as outlined by Tabachnick and Fidell (2007). Little’s MCAR test indicated that the data in Study 2 were not missing at random, so multiple imputation was used to complete the dataset (13.2% missing data imputed).

Confirmatory factor analysis.

The factor structure of the DERS-P was tested in R 3.5.1 using confirmatory factor analysis (CFA) with the weighted least squares estimation method (WLSMV). WLSMV was utilized due to the ordinal nature of item responses (i.e., Likert scale); WLSMV has been shown to be less biased and more accurate than robust maximum likelihood in estimating factor loadings for ordinal data (Li, 2016). All other analyses were conducted in SPSS 24. In both studies, three CFA models were examined: (1) correlated model with three factors, (2) unifactorial model with inter-correlations between the three factors fixed at 1.0, and (3) orthogonal factor model with inter-correlations between the three factors fixed at zero. Because the hypothesized, three-factor model was nested within the alternative models (i.e., unifactorial and orthogonal), it allowed for the direct comparison of model fit using chi-square difference tests (Kline, 2011). However, chi-square difference tests are only meaningful when comparing nested models (Schermelleh-Engel, Moosbrugger, & Müller, 2003). Therefore, we first compared non-nested models (i.e., unifactorial and orthogonal models) with global fit indices; next the better fitting non-nested model (i.e., unifactorial model) was directly compared to the hypothesized, three-factor model using a chi-square difference test.

Across models, each factor had a loading fixed at 1.0 to help identify the factor with a scale/metric. Model fit was assessed by considering parameter estimates (factor loadings and covariances) as well as global fit indices, including the χ² test, comparative fit index (CFI), standardized root mean-square residual (SRMR), and root square error of approximation (RMSEA) with corresponding 90% confidence intervals (Kline, 2011). A well-fitting model had a CFI value ≥ 0.95, a SRMR value of ≤ .05, and a RMSEA value ≤ 0.06 (Hu & Bentler, 1999).

Demographic comparisons.

Following this, for both studies, possible differences in demographic patterns (age, education, gender [male vs. female], ethnicity [Hispanic/Latino/a vs. non-Hispanic/Latino/a], race [White vs. non-White], employment [employed full-time vs. not employed full-time], income [≥ $50,000 vs. < $50,000]) related to DERS-P variables were examined using correlations. Due to unbalanced group sizes, categorical demographic variables were dichotomized so that statistically meaningful comparisons could be made across groups. Then, internal consistency for the DERS-P was calculated for both studies.

Convergent and discriminant validity.

Next, the convergent and discriminant validity of the DERS-P was evaluated by examining its relations with clinically relevant variables in Study 1 (i.e., DERS, EAQ, UPPS-P, PCL-5, DASS-21, AUDIT, DAST) and Study 2 (i.e., PCL-5, PHQ-9, AUDIT, DAST). Lee and Preacher’s (2013) online calculator was used to test the difference between two dependent correlations with one variable in common, which provides a z-score and p-value to statistically compare correlations.

Finally, in Study 2, we examined whether the DERS-P differentiated subsamples varying in symptom severity. Specifically, ANOVAs were conducted to examine whether levels of DERS-P varied among individuals characterized by probable PTSD versus no PTSD, probable depression versus no depression, probable alcohol use disorder versus no alcohol use disorder, and probable drug use disorder versus no drug use disorder. Given the number of analyses (N = 16), a Benjamini-Hochberg adjustment was utilized to minimize both Type I and Type II errors (Benjamini & Hochberg, 1995). This method preserves an overall Type I error without increasing the risk for Type II error and unnecessarily reducing statistical power.

Results

Preliminary Analyses

The data were normally distributed, as indicated by skewness and kurtosis values and visual inspection of data plots. Correlations among variables ranged from .13 to .91 in Study 1, and from .12 to .96 in Study 2 (see Table 2).

Table 2.

Intercorrelations among the Primary Study Variables for Study 1 (Upper Matrix) and Study 2 (Lower Matrix)

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
1. DERS-P Total	--	--	--	--	.27***	.32***	.12	.27***	.32***	.25***	.27***	.39***	.38***	.14*	.23***	.56***	.35***	.33***	.07
2. DERS-P Accept	--	--	.45***	.64***	.38***	.41***	−.01	.19**	.36***	.34***	.25***	.43***	.38***	.14*	.23***	.35***	.31***	.46***	−.01
3. DERS-P Goals	--	.75***	--	.66***	.18***	.22***	.20**	.25***	.25***	.19**	.25***	.26***	.22***	.06	.16*	.50***	.31***	.20**	.12
4. DERS-P Impulse	--	.90***	.80***	--	.19***	.24***	.09	.23***	.25***	.17*	.19**	.35***	.21***	.05	.15*	.52***	.29***	.26***	.05
5. PTSD+	.46***	.50***	.38***	.46***	--	.73***	.13*	.29***	.47***	.41***	.42***	.53***	.47***	.21***	.45***	.24***	.48***	.21***	.14*
6. Depression	.48***	.52***	.42***	.46***	.73***	--	.07	.18**	.51***	.46***	.51***	.51***	.56***	.21**	.21**	.23***	.50***	.26***	.15*
7. Alcohol Misuse	.18***	.21***	.12*	.21***	.16**	.16**	--	.46***	.01	.03	.08	.07	−.01	.08	.08	.19**	.13	−.06	.04
8. Drug Misuse	.32***	.33***	.24***	.33***	.28***	.35***	.28***	--	.18**	.12	.19**	.26***	.17**	.17**	.11	.35***	.32***	.04	.08
9. DERS Total	.43***	.44***	.40***	.40***	.56***	.74***	.15**	.34***	--	--	--	--	--	--	--	.28***	.54***	.24***	.12
10. DERS Accept										--	.56***	.54***	.67***	.67***	.61***	.22***	.45***	.31***	.14*
11. DERS Goals											--	.57***	.71***	.71***	.46***	.28***	.53***	.23***	.23***
12. DERS Impulse												--	.73***	.73***	.55***	.33***	.53***	.29***	−.01
13. DERS Strategy													--	.26***	.64***	.26***	.56***	.34***	.12
14. DERS Awareness														--	.52***	.06	−.13*	.25***	.15*
15. DERS Clarity															--	.20**	−.45***	.20**	.13*
16. Positive Urgency																--	.52***	.23***	.03
17. Negative Urgency																	--	−.18**	−.06
18. EAQ Positive																		--	.04
19. EAQ Negative																			--

Note

^*p < .05.

^**p <.01.

^***p < .001.

DERS-P= Difficulties in Emotion Regulation Scale – Positive. PTSD = posttraumatic stress disorder. DERS = Difficulties in Emotion Regulation Scale. EAQ= Emotional Avoidance Questionnaire. ⁺PTSD symptoms were only assessed among participants who endorsed a traumatic event.

Confirmatory Factor Analysis

The proposed three-factor structure of the DERS-P (i.e., Accept, Goals, Impulse) was assessed using CFA. Fit indices for each examined model are presented in Table 3. Results indicated that the correlated factor model provided the best fit to the data, ꭓ2(62) = 61.84, p = 0.48; CFI > 0.99; SRMR = 0.05; RMSEA= 0.00, 90% CI [0.00, 0.04]. The nonsignificant χ2 as well as obtained values for CFI, SRMR, and RMSEA suggest good fit. In addition, the nested nature of the hypothesized, correlated factor model allowed for model comparison by calculating χ ² difference tests. Significant differences in χ² values revealed that the more complex, correlated factor model best explained the data. Factor loadings for the correlated factor model in Study 1 are presented in Table 4. A similar pattern emerged in Study 2, with results indicating that the correlated factor model provided good fit to the data, ꭓ²(62) = 81.66, p = 0.05; CFI = 0.99; SRMR = 0.02; RMSEA = 0.03, 90% CI [0.00, 0.05]. Although the correlated model in Study 2 had a significant χ2, this pattern is common among moderate to large samples (Harlow, 2014). Factor loadings for the correlated factor model in Study 2 are presented in Table 5. Taken together, the findings from Study 1 and Study 2 provide support for a three-factor structure of the DERS-P.

Table 3.

Goodness-of-fit Indicators of Models

Model	χ2	df	CFI	SRMR	RMSEA (90% CI)	χ2diff
Study 1
1. Orthogonal model	1064.29*	65	0.75	0.51	0.26 (0.25, 0.27)
2. Unifactorial model	190.29*	65	0.97	0.16	0.09 (0.08, 0.11)
3. Correlated factor model	61.84	62	0.99	0.05	0.00 (0.00, 0.04)	128.45*
Study 2
1. Orthogonal model	16,082.88*	65	0.34	0.62	0.84 (0.83, 0.85)
2. Unifactorial model	504.03*	65	0.98	0.09	0.14 (0.13, 0.15)
3. Correlated factor model	81.66*	62	>0.99	0.02	0.03 (0.00, 0.05)	422.37*

Note: CFI = comparative fit index. SRMR = standardized root mean square residual. RMSEA = root mean square error of approximation.

^*p < .001.

χ²_diff compared the unifactorial and correlated factor models.

Table 4.

Factor Loadings for the Correlated Factor Model in Study 1 (N= 229)

Item	Accept	Goals	Impulse
2. When I’m happy, I become angry with myself for feeling that way.	0.89
4. When I’m happy, I feel ashamed with myself for feeling that.	0.77
6. When I’m happy, I become scared and fearful of those feelings.	0.79
12. When I’m happy, I feel guilty for feeling that way.	0.87
1. When I’m happy, I have difficulty focusing on other things.		0.63
7. When I’m happy, I have difficulty concentrating.		0.84
9. When I’m happy, I have difficulty thinking about anything else.		0.73
11. When I’m happy, I have difficulty getting work done.		0.80
3. When I’m happy, I worry that I will lose control.			0.81
5. When I’m happy, I become out of control.			0.91
8. When I’m happy, I have difficulty controlling my behaviors.			0.89
10. When I’m happy, I feel out of control.			0.86
13. When I’m happy, I lose control over my behaviors.			0.81

Table 5.

Factor Loadings for the Correlated Factor Model in Study 2 (N= 353)

Item	Accept	Goals	Impulse
2. When I’m happy, I become angry with myself for feeling that way.	0.95
4. When I’m happy, I feel ashamed with myself for feeling that.	0.95
6. When I’m happy, I become scared and fearful of those feelings.	0.92
12. When I’m happy, I feel guilty for feeling that way.	0.92
1. When I’m happy, I have difficulty focusing on other things.		0.81
7. When I’m happy, I have difficulty concentrating.		0.93
9. When I’m happy, I have difficulty thinking about anything else.		0.83
11. When I’m happy, I have difficulty getting work done.		0.93
3. When I’m happy, I worry that I will lose control.			0.91
5. When I’m happy, I become out of control.			0.95
8. When I’m happy, I have difficulty controlling my behaviors.			0.92
10. When I’m happy, I feel out of control.			0.94
13. When I’m happy, I lose control over my behaviors.			0.97

Demographic Comparisons for DERS-P Total Scale and Subscales

Table 6 provides results for the associations between demographic factors and the DERS-P scores. In Study 1, employment was significantly associated with DERS-P Impulse, such that individuals who were employed (vs. unemployed) reported higher levels of this DERS-P scale. In Study 2, DERS-P scores differed across all demographic factors except education (for all scales) and employment (for Accept). Specifically, greater difficulties regulating positive emotions were found among individuals who were younger, male, Hispanic, non-White, employed full-time, and making < $50,000 a year.

Table 6.

Demographic Comparison Table

	DERS-P Total		DERS-P Accept		DERS-P Goals		DERS-P Impulse
	Study 1	Study 2	Study 1	Study 2	Study 1	Study 2	Study 1	Study 2
Age	.06	−.30***	.11	−.29***	−.01	−.26***	.07	−.30***
Education	.01	.05	.03	.07	−.02	.06	.01	.02
Gender1	.03	.19***	−.04	.17**	−.01	.17**	.08	.18**
Ethnicity2	−.05	.15**	−.09	.17**	−.03	.11*	−.03	.18**
Race3	−.01	.20***	.08	.30***	−.08	.12*	.02	.24***
Employment4	.11	.11*	.10	.10	.04	.14*	.15*	.13*
Income5	.09	−.13*	.02	−.16**	.07	−.14**	.12	−.15**

Note.

^*p < .05.

^**p <.01.

^***p < .001.

DERS-P = Difficulties in Emotion Regulation Scale – Positive. ¹0 = female, 1 = male. ²0 = non-Hispanic/Latina/o, 1 = Hispanic/Latina/o. ³0 = White, 1 = non-White. ⁴0 = not employed full time, 1 = employed full time. ⁵0 = < $50,000, 1 = ≥ $50,000.

Internal Consistency

In Study 1, results revealed high internal consistency for the total scale (α = .92) and good to high internal consistency for subscales, with Cronbach’s alphas of .89 for Accept, .82 for Goals, and .93 for Impulse. Similarly, results of Study 2 indicated excellent internal consistency for the total scale (α = .96). Subscales also demonstrated good to high internal consistency, with coefficient alphas of .92 for Accept, .87 for Goals, and .94 for Impulse.

Convergent and Discriminant Validity

Emotion constructs.

The DERS-P scales were significantly positively associated with the DERS scales, negative and positive urgency, and positive (but not negative) emotional avoidance in Study 1 (see Table 2). However, the strength of these relations varied considerably, providing support for both convergent and discriminant validity. For instance, while the DERS-P scales were significantly positively associated with the DERS scales, the strength of these associations were small to medium (r = 0.17 to r = 0.43), suggesting that, while related, these constructs are distinct. Examination of the specific DERS-P and DERS scales showed that the association between those with greater conceptual overlap (i.e., DERS-P Impulse and DERS Impulse) was stronger compared to the association of those with less conceptual overlap (i.e., DERS-P Impulse to DERS Accept), z = 2.97, p = .003. The strength of the relations between the DERS-P scales and positive urgency were generally large, whereas the strength of the relations among the DERS-P scales and negative urgency were generally medium, and the association between those with greater conceptual overlap (i.e., DERS-P Impulse and positive urgency) was stronger compared to the association of those with less conceptual overlap (i.e., DERS-P Accept and positive urgency), z = −3.45, p = .001. Lastly, the DERS-P scales were significantly and positively associated with positive emotional avoidance, but not negative emotional avoidance, and the association between those with greater conceptual overlap (i.e., DERS-P Accept and positive emotional avoidance) was stronger than the association of those with less conceptual overlap (i.e., DERS-P Goals and positive emotional avoidance), z = 4.08, p = 0.001.

Mental and behavioral health outcomes.

The DERS-P scales were significantly positively associated with severity of PTSD symptoms, depression, alcohol misuse, and drug misuse in Studies 1 and 2 with a few exceptions: the DERS-P total and DERS-P Accept and Impulse subscales were not significantly related to alcohol misuse in Sample 1 (see Table 2). The strength of the associations between the DERS-P scales and mental/behavioral health outcomes were small to medium in Study 1 (rs = −.01 to.41), and medium in Study 2 (rs = .12 to .52).

Differentiation of Samples Based on Level of Clinical Disturbance

In Study 2, ANOVA results indicated that the DERS-P scores were significantly higher among individuals with (vs. without) probable PTSD, probable depression, probable alcohol use disorder, and probable drug use disorder (see Table 7). For PTSD, the total scale and Impulse subscale were strongly associated, the Goals subscale was moderately associated, and the Accept subscale was weakly associated. For depression, the Accept subscale was strongly associated, whereas the total scale and the Impulse and Goals subscales were moderately associated. For alcohol and drug misuse, the total scale and subscales were all weakly associated.¹

Table 7.

Differentiation of Samples Based on Clinical Disturbance by DER-P Total and Subscale Scores

	Scale	Groups	n	M	SD	F	Eta Squared (η2)
Posttraumatic Stress Disorder
	DERS-P Total	PCL-5 score ≥ 31	120	25.25	12.63	68.57***	0.24
		PCL-5 score < 31	224	15.33	4.91
		Total	334	18.79	9.67
	DERS-P Accept	PCL-5 score ≥ 31	127	8.32	4.61	7.74**	0.03
		PCL-5 score < 31	224	4.43	1.53
		Total	351	5.84	3.56
	DERS-P Goals	PCL-5 score ≥ 31	123	8.04	4.10	49.58***	0.17
		PCL-5 score < 31	224	5.25	2.18
		Total	347	6.24	3.28
	DERS-P Impulse	PCL-5 score ≥ 31	124	9.81	5.47	67.54***	0.24
		PCL-5 score < 31	224	5.65	1.86
		Total	348	7.13	4.10
Depression
	DERS-P Total	PHQ-9 score ≥ 10	104	25.95	12.81	61.41***	.24
		PHQ-9 score < 10	240	15.68	5.56
		Total	344	18.79	9.67
	DERS-P Accept	PHQ-9 score ≥ 10	110	8.50	4.76	68.04***	.26
		PHQ-9 score < 10	241	4.63	1.85
		Total	351	5.84	3.56
	DERS-P Goals	PHQ-9 score ≥ 10	107	8.36	4.14	52.24***	.19
		PHQ-9 score < 10	240	5.29	2.25
		Total	347	6.24	3.28
	DERS-P Impulse	PHQ-9 score ≥ 10	107	9.96	5.56	53.77***	.21
		PHQ-9 score < 10	241	5.88	2.37
		Total	348	7.13	4.10
Alcohol Misuse
	DERS-P Total	AUDID score ≥ 5	82	21.78	11.83	7.74**	.03
		AUDIT score < 5	262	17.85	8.70
		Total	344	18.79	9.67
	DERS-P Accept	AUDIT score ≥ 5	85	7.14	4.57	10.49**	.04
		AUDIT score < 5	266	5.42	3.06
		Total	351	5.84	3.56
	DERS-P Goals	AUDIT score ≥ 5	84	6.94	3.81	4.13*	.01
		AUDIT score < 5	263	6.01	3.07
		Total	347	6.24	3.28
	DERS-P Impulse	AUDIT score ≥ 5	124	9.81	5.47	9.88**	.04
		AUDIT score < 5	224	5.65	1.86
		Total	348	7.13	4.10
Drug Misuse
	DERS-P Total	DAST score ≥ 3	52	23.71	12.32	14.13***	.08
		DAST score < 3	262	17.05	7.53
		Total	314	18.16	8.84
	DERS-P Accept	DAST score ≥ 3	54	7.76	4.56	16.41***	.09
		DAST score < 3	265	5.15	2.71
		Total	319	6.00	2.25
	DERS-P Goals	DAST score ≥ 3	53	7.55	3.89	10.59**	.05
		DAST score < 3	263	5.72	2.85
		Total	316	6.03	3.12
	DERS-P Impulse	DAST score ≥ 3	53	9.30	5.33	15.09***	.09
		DAST score < 3	263	6.36	3.12
		Total	316	6.85	3.74

Note:

^*p<.05.

^**p<.01.

^***p<.001.

Results are reported as significance in accordance with the B-H correction.

Discussion

Emotion dysregulation has been identified as a transdiagnostic factor central to the etiology and treatment of diverse psychological difficulties and risky behaviors. Yet, research in this area has focused exclusively on emotion dysregulation stemming from negative emotions. There is growing evidence that individuals may also experience emotion dysregulation stemming from positive emotions (Cyders et al., 2007; Gruber & Moskowitz, 2014). Advancing research in this area, Weiss, Gratz, et al. (2015) developed a comprehensive measure of positive emotion dysregulation: the DERS-P. The goal of the current study was to confirm the factor structure of the DERS-P and further examine its reliability and validity in college and community samples.

Results confirmed the three-factor structure of the DERS-P, which included nonacceptance of positive emotions (Factor 1; Accept), difficulties engaging in goal-directed behaviors when experiencing positive emotions (Factor 2; Goals), and difficulties controlling impulsive behaviors when experiencing positive emotions (Factor 3; Impulse). These findings provide further support for the multi- (vs. uni-) dimensional nature of difficulties regulating positive emotions. Additionally, consistent with results of Weiss, Gratz, et al. (2015), the DERS-P scale scores demonstrated high internal consistency, providing support for its reliability.

Supporting the convergent and discriminant validity of the DERS-P, scale scores were found to demonstrate differential relations with emotion constructs. Generally, the relations among the DERS-P total and scales were generally medium, whereas scales that are more conceptually similar were generally more strongly associated, and those with less conceptual similarity were less strongly associated. Further, the relations between the DERS-P and positive urgency were generally large, while the relations between the DERS-P and negative urgency were moderately strong, and relations were generally stronger among scales that are more conceptually similar. In addition, the strength of the relations of the DERS-P to positive emotional avoidance were moderate, but small and non-significant for negative emotional avoidance. Finally, providing further support for convergent validity, the DERS-P scales were significantly and positively associated with mental and behavioral health outcomes, although the strength of these relations varied across the two samples (i.e., medium in the community sample; low in the college sample, with only one of the DERS-P subscales being significantly related to alcohol misuse). These latter findings suggest that the DERS-P may be more strongly associated with health outcomes in a community versus college sample. The non-significant association with alcohol is inconsistent with a prior study of college students (Weiss, Forkus, Contractor, & Schick, 2018), although relations in this other study were weak in magnitude. Non-significant or weak associations in college samples may reflect different patterns and correlates of alcohol use. Indeed, college students report more problematic alcohol use (e.g., binge drinking) compared to same-aged young adults not enrolled in full-time college (Substance Abuse and Mental Health Services Administration, 2014), and findings of Read et al. (2003) suggest that motives for alcohol use may diverge among college students compared to other populations (e.g., drinking among college students is often motivated by enhancement, and less often coping).

In addition, the DERS-P was found to differentiate community individuals with (vs. without) clinically significant levels of psychological difficulties (i.e., PTSD and depression) and risky behaviors (i.e., alcohol and drug misuse). Examination of effect sizes suggests that difficulties regulating positive emotions may be more useful in identifying individuals with probable PTSD and depression (generally medium to large effects) than those with probable alcohol or drug misuse (small effects). Further, large effect sizes highlight the contributions of DERS-P Impulse and Accept in particular to probable PTSD and depression, respectively. Finally, our findings for probable alcohol and drug misuse, while statistically significant, suggest a weak association with the DERS-P. It may be that negative emotions drive disordered levels of alcohol and drug use among community individuals, consistent with negative reinforcement (Baker, Piper, McCarthy, Majeskie, & Fiore, 2004). Research examining the unique roles of difficulties regulating negative and positive emotions to alcohol and drug misuse is warranted.

DERS-P scores were also found to vary as a function of demographic variables, specifically age, gender, ethnicity, race, employment status, and income. There is some support for these differences. Regarding age, consistent with our findings, younger individuals have been found to report lower acceptance of emotions (Nolen-Hoeksema & Aldao, 2011) and greater behavioral dyscontrol (Cyders et al., 2010) than older individuals. Further, the initial validation study (Weiss, Gratz, et al., 2015) examined gender differences in the DERS-P; as was also found here, men reported greater positive emotion dysregulation than women. Greater fear of positive emotions has been found to be related to more masculine gender role stress (Jakupcak, Salters, Gratz, & Roemer, 2003). Thus, men, who typically hold more masculine gender roles, may be socialized to control their positive emotions, which may lead them to be non-accepting of these emotional experiences. Men also display greater behavioral dyscontrol than women (e.g., impulsivity; Cross, Copping, & Campbell, 2011). Racial and ethnic differences also were found, such that individuals who identified as non-White (vs. White) and/or Hispanic/Latino/a (vs. non-Hispanic/Latino/a) reported greater positive emotion dysregulation. There is some evidence that emotion dysregulation differs across racial/ethnic groups (Butler, Lee, & Gross, 2007; Gross & John, 2003). For instance, Hispanic/Latino/a individuals report more attempts to hide, inhibit, or reduce emotions than Whites (Gross & John, 2003). Differences for employment status and income are less clear; future research is needed to explain these differences.

Results should be considered in the context of study limitations. First, the current study examined only specific aspects of reliability and validity. Future research on the DERS-P is needed to examine other indices of reliability and validity, such as test-retest reliability and predictive validity. Relatedly, the DERS-P subscales were strongly intercorrelated in Study 2, perhaps due to the fact that this sample was composed of trauma-exposed individuals. Future research is needed to examine the discriminant validity of the DERS-P scales across diverse (including non-trauma exposed) samples. Further, the DERS-P does not measure lack of emotional awareness, lack of emotional clarity, and limited access to effective strategies for regulating positive emotions, domains assessed by the DERS (Gratz & Roemer, 2004). Research would benefit from developing scales to assess these potential facets of positive emotion dysregulation and exploring their relation to mental and behavioral health outcomes.

Second, this study relied exclusively on self-report measures, which may be hampered by biased or inaccurate reporting. Future investigations should include behavioral and physiological measures of emotion regulation difficulties (Gratz et al., 2006; Vasilev, Crowell, Beauchaine, Mead, & Gatzke‐Kopp, 2009), as well as experimental paradigms to assess the influence of positive emotions on emotion regulation and its consequences. Third, we used self-report measures of PTSD, depression, and alcohol and drug misuse to assess symptom severity and probable diagnosis. Future studies would benefit from the use of standardized diagnostic interviews. Fourth, the DERS-P scales are conceptually similar and use very similar wording and phrases. Thus, stronger relations among conceptually similar scales may be due to shared method variance. Fifth, a plethora of work has linked personality factors (e.g., neuroticism) to emotion constructs. Additional work is needed to understand the relation of personality factors to the DERS-P. Sixth, further research is needed to speak to the robustness and reproducibility of our findings in larger, more diverse samples, including samples of verified clinical populations, for whom emotion dysregulation stemming from positive emotions may be most relevant.

Seventh, there may be a concern regarding data quality and motivation for study participation given the low compensation rate ($1.25 for an hour of work) and high income levels of the sample (72% of the sample reported being employed full-time and approximately 50% reported an income beyond $50,000). However, participants all consented to the present study after learning about compensation, and research indicates that compensation rates do not negatively influence data quality (Buhrmester et al., 2011) and that MTurk workers may not perceive MTurk as their primary source of income (Mason & Sun, 2012). Future research can explore the use of recommended performance-based pay rather than time-based pay in relation to data quality (Brawley & Pury, 2016). Relatedly, although limiting MTurk samples based on attention checks and degree of missing data improves data quality (Aust et al., 2013; Buhrmester et al., 2011; Oppenheimer et al., 2009), it may create a selection bias and limit generalizability. To counter such selection bias while maintaining data quality, restricting participation to MTurk workers with high reputations (>95% approval ratings) is preferred to validity checks (Peer, Vosgerau, & Acquisti, 2014); such a technique could be used in future research.

Finally, the DERS-P does not assess many other facets of positive emotional dysfunction. For instance, the DERS-P does not assess the frequency or intensity of positive emotions, which are linked to forms of psychopathology (American Psychiatric Association, 2013). Neither does the DERS-P assess the up-regulation or the down-regulation of positive emotions, which are distinct processes (Kim & Hamann, 2007). For instance, to maintain or up-regulate positive emotions, individuals may engage in savoring (i.e., reflecting on the positive aspects of positive experiences; Bryant, 2003) and capitalizing (i.e., positive event sharing or celebrating; Langston, 1994). Alternatively, to down-regulate positive emotions, they may use dampening (i.e., focusing on negative thoughts, such as having the thought that you do not deserve to feel good; Feldman, Joorman, & Johnson, 2008), and suppression, or attempts to reduce positive emotional reactions (Roemer, Litz, Orsillo, & Wagner, 2001). Although context plays a key role in determining emotion regulation effectiveness (Aldao, 2013), in general, research has found more adaptive outcomes among individuals who report greater maintenance/up-regulation and less down-regulation of positive emotions (Tugade & Fredrickson, 2007). Future studies that distinguish between these two processes are warranted given evidence that clinical disorders may be differentially characterized by down- (e.g., bipolar disorder [Johnson, McKenzie, & McMurrich, 2008]) versus up- (e.g., depression [Heller et al., 2009]) regulation of positive emotions.

Despite study limitations, the results of the current study significantly add to the literature on emotion dysregulation, confirming the factor structure, reliability, and validity of the DERS-P. The DERS-P has important research and clinical utility. The DERS-P may be used to identify (a) processes underlying the etiology and maintenance of emotion dysregulation stemming from positive emotions and (b) individuals who exhibit (or are at risk for exhibiting) psychological difficulties and risky behaviors. Future research would benefit from examining the contribution of the DERS-P to other disorders characterized by positive emotional disturbance, such as mood (e.g., bipolar), anxiety (e.g., social anxiety), psychotic (e.g., schizophrenia), and personality (e.g., schizotypal) disorders. Of note, while emotion dysregulation encompasses difficulties regulating both negative and positive emotions, the DERS-P was developed as a stand-alone measure, with the idea that researchers and clinicians could add the DERS-P as they saw fit, whether that be alongside the DERS, alone (perhaps because positive emotional experiences are more relevant to the specific question/problem), or not at all (perhaps because negative emotional experiences are more relevant to the specific question/problem). We believe that this flexibility enhances the feasibility and utility of the DERS-P. With regard to clinical implications, emotion dysregulation stemming from positive emotions is often overlooked in clinical settings. If replicated, our findings may suggest the utility of considering emotion dysregulation stemming from positive emotions in the assessment and treatment of psychological difficulties and risky behaviors.