Abstract
Emotion dysregulation, the pervasive difficulty managing negative emotions, is a core problem across mood and anxiety disorders. Anger, shame, and disgust are particularly problematic emotions, impacting both disorder severity and treatment outcome. We previously found that a 16-week dialectical behavior therapy skills training group (DBT-ST) was superior to an activities-based support group (ASG) in decreasing emotion dysregulation in 44 adults with high emotion dysregulation who met diagnostic criteria for an anxiety or depressive disorder. We presently examine these participants’ changes in anger, shame, disgust, and distress using self-reports collected over 6 months during and after treatment. Hierarchical linear modeling analyses show that DBT-ST was superior to ASG in decreasing anger suppression (d = 0.93) and distress (d = 1.04). Both conditions significantly reduced shame, disgust propensity, and disgust sensitivity, but neither was superior for these outcomes. The treatments did not significantly reduce anger expression. Mediation analyses suggest that condition indirectly influenced 4-month anger suppression, shame, and distress through its effect on 2-month emotion dysregulation. These findings suggest that DBT-ST is efficacious for certain problematic emotions and distress in depressed and anxious adults and that common factors may account for some, but not all, of its benefits.
Keywords: dialectical behavior therapy, anger, shame, disgust, anxiety, depression
Mood and anxiety disorders are widely prevalent and debilitating problems that affect a significant number of adults in the United States each year. Anxiety disorders are most common, affecting 18.1% of adults in the general population, followed by mood disorders, which affect 9.5% of adults in the United States (Kessler, Chiu, Demler, & Walters, 2005). Across disorders, 22–45% of adults report severe impairments, indicating that in the previous 12 months, their mental health problems prevented them from engaging in normal daily activities on average 88 days out of the year (Kessler, Chiu, Demler, & Walters, 2005). There is high comorbidity between mood and anxiety disorders (Kaufman & Charney, 2000; Kessler et al., 2005), and recent evidence suggests similar underlying problems are likely to contribute to the development and severity of these disorders. One such core problem, which has been identified as a precursor and maintenance factor for mood and anxiety disorders, is emotion dysregulation (Hawkins & Cougle, 2011; Hoffmann, Sawyer, Fang, & Asnaani, 2012). Examining emotion dysregulation across disorders can provide important insight into the treatment of anxiety and mood disorders.
Emotional Dysregulation in Mood and Anxiety Disorders
Emotion dysregulation refers to the predisposition to use maladaptive strategies to moderate unwanted emotional intensity (Gratz & Roemer, 2004; Kuo & Linehan, 2009; Linehan, 1993). According to Linehan (1993) emotional dysregulation is more likely to occur for those who experience emotions more easily, with greater intensity, and for a longer duration. These aspects of emotional vulnerability and dysregulation for both positive and negative emotions are relevant to both the course and treatment outcomes of anxiety and depression. Anxiety and depression are highly correlated with maladaptive emotion regulation strategies such as avoidance, rumination, and suppression (Aldao, Nolen-Hoeksema, & Schweizer, 2010). In addition, dysregulation of specific emotions are strongly implicated in the development and maintenance of both mood and anxiety disorders (Kring & Bachorowski, 1999; Olatunji, Cisler, McKay, Phillips, 2010; Pasquini, Picardi, Biondi, Gaetano, & Morosini, 2004).
Experimental findings support the theoretical connection between dysregulated anger, shame, guilt and depressive and anxiety disorders. A recent meta-analysis has found moderate correlations between depression severity and different dimensions of shame and low correlations between depression and guilt (Kim, Thibodeau, & Jorgensen, 2011). Group comparisons also highlight that adults who meet criteria for depressive or anxiety disorders report higher anger dysregulation when compared to healthy controls (Koh, Kim, & Park, 2002). These studies fail to account for the fact that emotions such as shame, guilt, or irritability are already part of the pathology of anxiety and depressive disorders. Nevertheless, studies where anxiety and depression symptomatology is controlled for, still find clinically significant emotional dysregulation that warrants to be directly addressed in treatment. For example, Pasquini and colleagues (2004) completed a factor analysis for problems reported by a depressed group and found a stand-alone factor for anger/irritability that was independent of depression and that was clinically relevant to 23% of the patients examined. Moscovitch, McCabe, Anthony, Rocca, and Swinson, (2008) identified significant differences in anger expression/suppression in patients who met criteria for panic disorder and social anxiety disorder (but not other types of anxiety disorder) and healthy controls, even when controlling for depression severity. An investigation of anxiety disorders highlighted a relationship and relevance to treatment for shame-proneness in generalized anxiety disorder and social anxiety disorder when controlling for anxiety and depression symptomatology (Fergus, Valentiner, McGrath, & Jencius, 2010). Taken together these findings suggest that some but not all people who meet criteria for an anxiety or depressive disorder report anger, shame, and to a lower extent, clinically significant guilt dysregulation that is independent of their reported disorder.
In addition to specific negative emotional dysregulation, general psychopathology-induced distress and disability is an important clinical concern for those with mood and anxiety disorders due to its association with treatment retention and outcomes (Hannan et al., 2005). Depression and anxiety are often associated with a variety of impairments (e.g., problems with work, school, or relationships) that contribute to an individual’s perceived level of psychopathology-related disability (Ansell, Sanislow, McGlashan, & Grilo, 2007; Kaufman & Charney, 2000; Kessler et al., 2005). Those who report greater degrees of distress and disability are also more likely to discontinue treatment (Hannan et al., 2005). Given the evidence suggesting that negative emotions and distress influence the outcomes of mood and anxiety disorder treatments, further research on whether targeting emotion dysregulation improves specific emotions and distress may yield insights for achieving better treatment outcomes.
Anger Dysregulation in Depression and Anxiety
Anger is a frequent clinical problem associated with increased depression (Koh et al., 2002), anxiety (Hawkins & Cougle, 2011), and interpersonal problems (DiGiuseppe, Tafrate, & Eckhardt, 1994). Problematic anger leads to treatment dropout, therapeutic ruptures, and additional psychosocial stressors (DiGiuseppe et al., 1994; Newman, 2011). According to Spielberger (1998), anger expression refers to overt behaviors associated with anger (i.e., verbal communication, physical aggression), whereas anger suppression refers the inhibition of the expression of this emotion. Anger expression is associated with depressive disorders (Koh et al., 2002), and anger suppression is associated with higher rates of psychosocial impairment and treatment noncompliance (Erwin, Heimberg, Schneier, & Liebowitz, 2003). Additionally, dysregulated anger is correlated with posttraumatic stress (PTSD), social anxiety (SAD) and panic (PD) disorders severity (Baker, Holloway, Thomas, Thomas, & Owens, 2004; Erwin, Heimberg, Schneier, & Liebowits, 2003; Kulkarni, Porter, & Rauch, 2012). For example, when compared to non-anxious controls, SAD participants report more anger suppression and react with more anger to provocation (Erwin et al., 2003). Adults with PTSD who report more problems with anger prior to treatment benefit less from exposure treatment than clients who are less angry (Foa, Riggs, Massie, & Yarczower, 1995). Difficulties with suppressing a wide variety of emotions is independent from depression and anxiety severity, suggesting that adults who meet criteria for affective disorders have an independent difficulty with emotional regulation (Baker et al., 2004). Furthermore, increased anger experience leads to higher risk to drop out of CBT or to have poor outcomes for SAD participants (Erwin et al., 2003). Conversely, people who drop out of cognitive behavior therapy (CBT) are more likely to have greater difficulty with anger problems than those who complete treatment (Erwin et al., 2003).
The connection between anger and anxiety disorders is further supported by the similarity in biological pathways that these emotions evoke. Kunimatsu and Marsee (2012) propose a model based on a thorough literature review that describes pathological anxiety and aggression as two behavioral manifestations of a dysfunctional fight/flight system. In particular, difficulty controlling attention in an emotional situation and heightened emotional reactivity are some examples of dimensions of their model that underline both anxiety and anger dysregulation. Moons, Eisenberger, & Taylor (2010) show that both anger and anxiety affect similar biological targets (proinflammatory cytokinesis and cortisol) although in different ways (i.e., fear leads to increase in proinflammatory cytokinesis and decrease in cortisol, while anger leads to increase in cortisol). Therefore, thinking about anxiety and anger as behavioral manifestations of dysfunction of the fight/flight system can benefit treatment, since similar treatment mechanisms may address the common dimensions and biological pathways that underline these emotions when they become problematic (Kunimatsu et al., 2012).
Shame Dysregulation in Depression and Anxiety
High levels of shame are also associated with depression (Orth, Berking, & Burkhardt, 2006), anxiety (Blum, 2008), and difficulties seeking help (Gilbert & Procter, 2006). Shame is a problematic emotion for the treatment of mood and anxiety disorders because the nature of this emotion may lead to self-devaluation (Kim et al., 2011) which in turn may impede treatment (Schomerus, Matschinger, & Angermeyer, 2009). Several research studies link the experience of shame with depression (Kim et al., 2011; Orth et al., 2006), posttraumatic stress, social anxiety, and self-harm (Blum, 2008; Gilbert, 2000). The effect of shame on anxiety and depression severity is significantly mediated by rumination (Nolen-Hoeksema, 2000; Orth et al., 2006), and low mood has been found to trigger self-devaluative thoughts in previously depressed patients (Teasdale & Cox, 2001). These findings suggest that shame is associated with anxiety and depression, may impede treatment, and may lead to relapse for those who recover from depression.
Disgust Dysregulation in Depression and Anxiety
Disgust is also associated with the development and maintenance of anxiety and depression. Adults with phobias and obsessive-compulsive disorder experience greater levels of disgust as well as an increased tendency towards avoidance, which impacts the severity of mood and anxiety difficulties (Olatunji, Cisler, Deacon, Connolly, & Lohr, 2007; Olatunji, Lohr, Sawchuk, & Tolin, 2007). Cavanagh and Davey (2000) propose a promising model of disgust with two dimensions that are relevant to psychopathology: disgust propensity (the frequency of experiencing disgust) and disgust sensitivity (the emotional impact of experiencing disgust; as cited in Van Overveld, de Jong, Peters, Cavanagh, & Davey, 2006). In clinical samples, self-reported disgust sensitivity and disgust propensity are each associated with anxiety severity, above and beyond the presence of overall negative affect (Olatunji, Cisler, et al., 2007).
Specific Emotions and Psychotherapy for Anxiety and Depression
These findings highlight the importance of determining whether targeting emotion dysregulation among anxious and depressed adults reduces difficulties managing specific emotions such as anger, shame, or disgust. Most psychotherapies utilize cognitive-behavioral techniques to target dysfunctional cognitions and rumination (Hallion & Ruscio, 2011), enhance mindfulness and self-awareness (Grossman, Niemann, Schmidt, & Walach, 2004; Hofmann, Sawyer, Witt, & Oh, 2010), or increase behavioral activation (Cuijpers, Van Straten, & Warmerdam, 2007). Despite apparent efficacy of CBT-based interventions on depression and anxiety (Butler, Chapman, Forman, & Beck, 2006), many participants continue to have problems with specific emotions after receiving CBT. For example, adolescents who completed a 16-week course of CBT reported improvements in anxiety and overall emotion regulation at posttreatment, but did not report significant changes in feelings of sadness or anger (Suveg, Sood, Comer, & Kendall, 2009). Additionally, many participants with dysregulated anger, shame, or disgust do not complete treatment (Erwin et al., 2003; Hedman, Ström, Stünkel, & Mörtberg, 2013). Thus, it is important to evaluate whether targeting emotion dysregulation directly improves difficulties with anger, shame, and disgust among adults with mood and anxiety problems.
Current Study
In a previous study we showed that emotion dysregulation is indeed a key transdiagnostic problem for adults with depression and anxiety and that emotion dysregulation can be improved through a skills-based, transdiagnostic intervention (Neacsiu, Eberle, Kramer, Wiesmann, & Linehan, 2014). Non-borderline personality disorder (BPD) participants who met criteria for a depressive or anxiety disorder and who reported high emotion dysregulation were randomized into a dialectical behavior therapy skills training group (DBT-ST) or an activities-based support group (ASG). DBT-ST is a shortened version of standard DBT (Linehan, 1993; Linehan, 2014`), an evidence-based treatment that broadly aims to improve emotion dysregulation by teaching four modules of skills: mindfulness, emotion regulation, interpersonal effectiveness, and distress tolerance. Minimal research has evaluated the impact of DBT skills on specific dysfunctional emotions, yet the skills target a wide variety of emotions, including anger, shame, and disgust. Participants in both conditions reported significant reductions in emotion dysregulation, anxiety, and depression severity throughout the course of treatment, but those in the DBT-ST group reported significantly greater improvements earlier in treatment than those in the control group on emotion dysregulation and anxiety severity.
The current study examines changes in specific negative emotions (anger, shame, and disgust) and psychopathology-induced distress in this sample of anxious and depressed adults randomized into DBT-ST or ASG. The primary hypothesis was that DBT-ST would lead to superior improvements in negative emotions and distress in comparison to ASG. Finally, we hypothesized that condition would indirectly influence each of these outcomes at 4 months through its effect on emotion dysregulation at 2 months.
Method
Participants, procedures, and treatment conditions are presented here in brief; detailed demographics and additional information regarding participant recruitment are described elsewhere (Neacsiu et al., 2014).
Participants
Intent-to-treat (ITT) participants were 44 men and women between the ages of 19 and 70 (M = 35.55; SD = 12.43) who were selected for meeting criteria for high emotion dysregulation a primary diagnosis of mood or anxiety disorder. Participants were recruited from the community with fliers, brochures, and online ads that advertised a treatment study for those who have difficulties managing their emotions. Providers in the area were contacted to advertise the study as a possible low cost treatment option for people who were referred out of their clinic. In order to enter the study, participants could not be in any other type of psychotherapy, had to be interested in receiving group treatment for their psychiatric disorders, and could continue on their psychiatric medication provided that they did not change the dosage throughout the study.
Participants were primarily single (72%), heterosexual (68%), Caucasian (93%), and female (66%) and met criteria for multiple DSM-IV Axis I disorders. Participants met criteria for major depression (n = 22), dysthymic disorder (n = 11), depressive disorder NOS (n = 1), generalized anxiety disorder (n = 29), social phobia (n = 16), specific phobia (n = 8), panic disorder (n = 6), obsessive-compulsive disorder (n = 5), posttraumatic stress disorder (n = 4), anxiety disorder NOS (n = 4), and agoraphobia (n = 3) according to the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I; First, Spitzer, Gibbon, & Williams, 1995). Primary diagnoses were major depression (n = 14), dysthymic disorder (n = 5), generalized anxiety disorder (n = 17), social phobia (n = 6), obsessive-compulsive disorder (n = 1), and posttraumatic stress disorder (n = 1). On average, participants met criteria for 2.68 diagnoses in DBT-ST (SD = 1.21) and 2.59 diagnoses in ASG (SD = 1.44). High emotion dysregulation was operationally defined as a score of 97 or higher on the Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004); only participants who scored over 96 on the DERS at the phone screen were included in the study (see Neacsiu et al., 2014, pp. 42–43, for the cutoff calculation and rationale).
Participants were excluded at phone or in-person screening if they (a) were actively suicidal (n = 26); (b) were mandated to psychological treatment (n = 3); (c) had scheduling issues (n = 11), were unable to come to group regularly (n = 1), (d) had received more than five sessions of outpatient DBT (n = 15); (e) met criteria for bipolar disorder (n = 4), psychotic or other life-threatening disorders (n = 0); (f) obtained a verbal IQ of less than 70 on the Peabody Picture Vocabulary Test-Revised (PPVT-R; Dunn, 1981; n = 0). Adults who scored above a preset cutoff on the Borderline Symptom List-23 (BSL-23; Bohus et al., 2009; n = 77) or who met criteria for BPD on the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II; First, Gibbon, Spitzer, Williams, & Benjamin, 1997; n = 28) were also excluded in order to test this intervention on those who were unlikely to meet criteria for BPD, given that the effect of DBT for BPD is already known. Participants who scored below the DERS cutoff (n = 74), were under age 18 (n = 1), did not have a primary current mood/anxiety disorder (n = 1), and did not consent (n = 2) were excluded, as were those who did not agree to remain on the same dosage of psychotropic medication (n = 0) and to refrain from ancillary psychotherapy (n = 16). Finally, participants were excluded if they declined the phone screening (n = 58), in-person screening (n = 32), or study (n = 9). All participants provided informed consent using protocols approved by the University of Washington Human Subjects Division.
Procedures
Participants were randomly assigned to DBT-ST (n = 24) or ASG (n = 24). A total of 44 ITT participants were included in the study after 4 people (2 in each condition) withdrew before the first treatment session. Participants were matched according to: (a) gender, (b) primary diagnosis (mood or anxiety disorder), and (c) use of psychotropic medication (yes or no).
Participants were asked to complete assessments before treatment started (pretreatment), halfway through treatment (2-month), at the end of treatment (4-month), and 2 months post treatment (6-month). Assessments were scheduled independent of therapy sessions and included measurement of emotion dysregulation, skills use, general and emotional distress, and psychopathology. Assessors were blind to randomization and had been trained to reliability (for more details on reliability see Neacsiu et al., 2014). Participants could drop out of therapy and continue completing the study assessments. Participants were compensated for each assessment they completed. Treatment occurred weekly, in a group setting, and the group sessions were 2 hours long. All 22 participants enrolled in the DBT-ST condition completed all the study assessments. Three of the 22 ASG participants had incomplete assessment data.
Treatment Conditions
DBT-ST
DBT-ST retained the essence of DBT (Linehan, 1993) and remained didactically focused. It emphasized modeling and utilized instructions, structured exercises, feedback, and homework assignments to practice new skills. DBT-ST included all four modules of standard DBT (Linehan, 2014) with two modifications: (1) skills from the interpersonal effectiveness module were removed to include only three weeks’ worth of material, and (2) the duration of treatment was shortened from 24 to 16 weeks. Each session consisted of (a) mindfulness practice, (b) homework review, (c) skills instruction, and (d) a new homework assignment. Two groups (one on Tuesday and one on Wednesday) were continuously run throughout the duration of the study, and participants could join either group whenever the group was learning about mindfulness (see curriculum in Neacsiu et al., 2014). Participants paid on a sliding scale for treatment.
ASG
ASG was developed after DBT, excluding active ingredients such as homework review, skills teaching and skills coaching, but preserving principles of client-centered supportive therapy and CBT, such as offering unconditional positive regard and empathy, providing psychoeducation, encouraging awareness and problem solving, and facilitating support-building in a structured environment. Common factors such as structure, empathy, and therapeutic alliance are therapeutic in treating depression (Illardi & Craigheard, 1994), and the activities integrated into ASG contained a behavioral activation component known to improve anxiety and depression (Hopko, Lejuez, Ruggiero, & Eifert, 2003; Hopko, Robertson, & Lejuez, 2006). The ASG group met for 16 weeks, and each session consisted of (a) a 30-minute check-in, (b) a support-building activity chosen from the treatment manual, (c) discussion about a topic chosen by participants the week prior (e.g., anxiety, anger, goals, support), and (d) homework that was assigned but not discussed during following sessions. There were two groups run continuously (one on Thursday and one on Friday), and participants could join either at the beginning or at the middle of each pass through the curriculum. Participants paid on a sliding scale for treatment.
Measures
Difficulties in Emotion Regulation Scale (DERS; Gratz & Roemer, 2004)
The DERS is a 36-item self-report measure of individuals’ typical levels of emotion dysregulation across six domains. Participants respond on a Likert scale ranging from 1 (almost never) to 5 (almost always). A psychometric study of the DERS found high internal consistency (Cronbach’s α = .93), good test-retest reliability (r = .88, p < .01), and adequate construct and predictive validity (Gratz & Roemer, 2004). The total score and subscale scores are sums of relevant items. In the present study, Cronbach’s alpha for the total score at phone screen was .82.
State-Trait Anger Expression Inventory (STAXI; Spielberger, 1988)
The STAXI is a 44-item self-report measure that provides scores on four subscales related to anger: Trait Anger (frequency of experiencing anger), Anger-In (frequency of suppressing anger), Anger-Out (frequency of expressing anger), and Anger Control (attempts to control anger). Items are rated on a Likert scale ranging from 1 (never) to 4 (almost always). The STAXI demonstrates high internal consistency (Cronbach’s α between subscales ranges from .73 to .93) and construct validity (Spielberger, 1988). Only the Anger-In and Anger-Out subscales were included in our analyses, as these subscales directly measure anger suppression and expression. At pretreatment, Cronbach’s alphas for these subscales were .68 and .85, respectively. We did not analyze changes in Trait Anger because it is considered stable. Anger Control was also excluded because it more closely measures emotion regulation than emotional distress and we intended to measure emotional distress as outcome mediated by changes in emotional dysregulation.
Experience of Shame Scale (ESS; Andrews, Qian, & Valentine, 2002)
The ESS is a 25-item self-report measure that assesses several aspects of shame, including nonphysical characterological shame (feeling ashamed of one’s habits, manner with others, character, or ability), behavioral shame (feeling shame about doing something wrong, saying something stupid, or experiencing failure in competitive situations), and bodily shame. Items are rated on a Likert scale ranging from 1 (not at all) to 4 (very much), and total scores range from 25 to 100. The ESS demonstrates high internal consistency (Cronbach’s α = .92) and good test-retest reliability (ρI = .83; Andrews, Qian, & Valentine, 2002). Total scores on the ESS are moderately correlated with self-reported depression (Andrews, Qian, & Valentine, 2002). At pretreatment for the current study, Cronbach’s alpha for the total score was .94.
Disgust Propensity and Sensitivity Scale—Revised (DPSS-R; van Overveld et al., 2006)
The DPSS-R is a 16-item self-report measure that assesses two factors related to disgust: Disgust Propensity (the frequency of feeling disgust) and Disgust Sensitivity (the emotional impact of disgust). Items are rated on a Likert scale ranging from 1 (never) to 5 (always). The DPSS-R demonstrates excellent internal consistency (Cronbach’s α = .90) and adequate test-retest reliability for both subscales (ρI = .69–.77) and demonstrates convergent validity with self-reported anxiety (Olatunji, Cisler, et al., 2007). At pretreatment, Cronbach’s alphas for these subscales were .86 and .76, respectively. While disgust propensity and sensitivity have not yet been examined as treatment outcome measures, how likely and how intensely one is to experience disgust should also be changeable with the use of emotion regulation skills. Therefore, we opted to include both subscales into the study.
Outcome Questionnaire-45 (OQ-45; Lambert, Huefner, & Reisinger, 1996)
The OQ-45 is a 45-item self-report measure used to track severity of psychopathology throughout treatment. It consists of subscales that identify three types of problems that lead to general stress: psychological symptoms, interpersonal conflicts, and problems with social roles (Wells, Burlingame, Lambert, Hoag, & Hope, 1996). Items are rated on a Likert scale ranging from 0 (never) to 4 (almost always). The OQ-45 demonstrates adequate test-retest reliability over a 3-week interval (ρI = .84) and excellent internal consistency (Cronbach’s α = .93; Wells, Burlingame, Lambert, Hoag, & Hope, 1996). At pretreatment, Cronbach’s alpha for the total score was .90.
Statistical Analysis
Power
We used Optimal Design (Raudenbush et al., 2011) to conduct a priori power analyses aimed at detecting longitudinal emotion dysregulation differences between conditions. Data from two randomized controlled trials suggested an expected effect size from 0.89 to 1.53 (Gratz & Gunderson, 2006; unpublished data from Linehan, 2015). We found that, to reach 80% power with an effect size of 0.89, 42 participants would need to complete the DERS at least once. We therefore recruited 48 participants, allowing for a 10% loss of data (from participants who completed no assessments). (For more information see p. 43 of Neacsiu et al., 2014).
Longitudinal outcomes
We used hierarchical linear modeling (HLM; Bryk & Raudenbush, 1992) to assess differences between conditions over time. Appropriate covariance structures were analytically determined (Verbeke, 1997), and analyses included a restricted estimated maximum likelihood model to account for missing data (Schafer & Graham, 2002; i.e., cases with missing data were not discarded, but slopes for each participant were computed with the data available). Assuming that the main effects were different during the active phase of treatment versus the follow-up phase of the study, we separated the time variable into two legs. The first leg (time1) was coded as continuous throughout treatment and constant at follow-up, whereas the second leg (time2) was coded as constant throughout treatment and continuous at follow-up. Time1 and time2 were both used as fixed effects and together yielded two time-by-condition interactions (one for treatment and one for follow-up). Each analysis included four a priori contrasts—two for each study phase—to assess the significance of each condition slope. Effect sizes were computed using Feingold (2009)’s formula and interpreted using Cohen (1988)’s specifications. Because each treatment condition included two days when groups occurred, we examined whether there were any effects of day within each condition. Depending on these analyses, we chose an HLM model with two nested levels rather than three.
Mediation
We used ordinary least squares path analysis to estimate simple mediation models based on Model 4 of Hayes’s (2013) PROCESS macro (SPSS v. 2.16). Each analysis estimated the indirect effect of condition assigned at the beginning of treatment (independent variable) on outcome measured at the end of treatment (dependent variable) through its effect on emotion dysregulation measured after 2 months of treatment (mediator) controlling for baseline emotion dysregulation and outcome (covariates). Therefore, the mediation model included variables that were temporally sequenced. Furthermore, the mediator was assessed during the course of the treatment. Assessments collected at other time points were not included in this analysis. Baseline scores were covaried in the analyses. The indirect effect (αβ) is the product of the effect of condition on emotion dysregulation, controlling for the covariates, (α) and the effect of emotion dysregulation on outcome, controlling for condition in addition to the covariates, (β). Each analysis also estimated the direct effect of condition on outcome measured at 4 months, controlling for emotion dysregulation measured at 2 months and the covariates measured at pretreatment (c'). The sum of the direct and indirect effects is the total effect of condition on outcome measured at 4 months (c = c' + αβ). Although we report unstandardized metrics for all of these effects (Table 3), the indirect effect is deemed significant only if its bias-corrected 95% bootstrap confidence interval based on 10,000 bootstrap samples excludes zero. Given that an independent variable need not be associated with a dependent variable to indirectly affect the dependent variable through its effect on a mediator (e.g., the sum of a small direct effect and potentially multiple indirect effects, some positive and others negative, may be close to or equal to zero; Hayes, 2013, pp. 87–88, 169), we estimated mediation models for all outcomes without requiring a total effect of condition on outcome as a prerequisite. Each model excluded participants who missed data for any variable in the model.
Table 3.
Outcome | n | c (SE), p | α (SE), p | β (SE), p | c' (SE), p | αβ (SE), 95% CI |
---|---|---|---|---|---|---|
STAXI Anger-In | 36 | −2.55 (1.31), .06 | −15.88 (5.59), < .01 | 0.14 (0.03), < .001 | −0.40 (1.22), .74 | −2.15 (0.88), [−4.66, −0.86] |
STAXI Anger-Out | 36 | 0.70 (0.90), .44 | −11.86 (5.94), .05 | 0.04 (0.03), .12 | 1.19 (0.93), .21 | −0.49 (0.41), [−1.62, 0.06] |
ESS | 36 | −4.76 (3.95), .24 | −13.02 (6.06), .04 | 0.30 (0.10), < .01 | −0.87 (3.82), .82 | −3.89 (2.23), [−9.22, −0.41] |
DPSS-R Propensity | 36 | 2.86 (1.32), .04 | −9.49 (5.94), .12 | 0.07 (0.04), .06 | 3.55 (1.31), .01 | −0.69 (0.56), [−2.40, 0.03] |
DPSS-R Sensitivitya | 36 | 0.26 (0.15), .09 | −11.12 (6.17), .08 | 0.01 (0.00), .06 | 0.35 (0.15), .03 | −0.09 (0.07), [−0.31, 0.00] |
OQ-45 | 37 | −17.61 (6.25), < .01 | −13.01 (5.50), .02 | 0.37 (0.19), .06 | −12.84 (6.50), .06 | −4.77 (3.42), [−13.99, −0.21] |
Note. All results are based on a simple mediation model estimated by ordinary least squares path analysis using Model 4 of Hayes’s (2013) PROCESS macro. Condition is the independent variable, emotion dysregulation (Difficulties in Emotion Regulation Scale) at 2 months is the mediator, outcome at 4 months is the dependent variable, and both emotion dysregulation and outcome at pretreatment are covariates. Each analysis excluded participants missing data for any variable in the model. Reported effects are unstandardized. c = coefficient estimate for the total effect of condition on outcome, controlling for the covariates; α = coefficient estimate for the effect of condition on emotion dysregulation, controlling for the covariates; β = coefficient estimate for the effect of emotion dysregulation on outcome, controlling for condition in addition to the covariates; c' = coefficient estimate for the direct effect of condition on outcome, controlling for emotion dysregulation and the covariates; αβ = estimate for the indirect effect of condition on outcome through emotion dysregulation, controlling for the covariates and tested with a bias-corrected 95% bootstrap confidence interval (CI) based on 10,000 bootstrap samples; STAXI = State-Trait Anger Expression Inventory; ESS = Experience of Shame Scale; DPSS-R = Disgust Propensity and Sensitivity Scale—Revised; OQ-45 = Outcome Questionnaire-45.
Transformed ( ).
Results
Preliminary Analyses
Randomization successfully matched participants on the chosen variables. No significant demographic differences emerged. DBT-ST and ASG sessions were rated as adherent to their respective models. Dropout rates were 32% in DBT-ST and 59% in ASG, and compliance rates were 38% in DBT-ST and 46% in ASG (for more information, see Neacsiu et al., 2014). There were no significant time by group interaction (ps >.05, analyses available upon request) within each condition for any outcome, indicating that participating in either the Tuesday or Wednesday DBT group and participating in either the Thursday or Friday ASG group led to similar outcomes. Therefore, we examined longitudinal results with a two-level HLM model combining all participants within each condition.
Table 1 shows raw outcome score means for the ITT sample. Given that the DPSS-R Disgust Sensitivity subscale was not normally distributed at the 4-month time point (Shapiro-Wilk W = .87), we transformed the variable using and used the transformed variable, which was normal at all time points, in all subsequent analyses. All other variables were normally distributed at all time points.
Table 1.
DBT-ST | ASG | ||||||
---|---|---|---|---|---|---|---|
Outcome | Period | n | M | SD | n | M | SD |
STAXI Anger-In | Pretreatment | 22 | 21.82 | 4.17 | 22 | 19.77 | 3.89 |
2 months | 22 | 19.05 | 4.88 | 19 | 20.53 | 2.65 | |
4 months | 19 | 18.05 | 5.53 | 17 | 19.53 | 3.86 | |
Follow-up | 21 | 18.62 | 4.98 | 17 | 18.88 | 3.79 | |
STAXI Anger-Out | Pretreatment | 22 | 15.91 | 4.53 | 22 | 14.59 | 5.37 |
2 months | 22 | 15.23 | 4.16 | 19 | 14.58 | 5.18 | |
4 months | 19 | 15.21 | 3.88 | 17 | 13.47 | 3.92 | |
Follow-up | 21 | 15.62 | 4.17 | 17 | 14.24 | 4.10 | |
ESS | Pretreatment | 22 | 68.64 | 17.12 | 22 | 69.77 | 15.84 |
2 months | 22 | 61.95 | 14.96 | 19 | 69.42 | 14.50 | |
4 months | 19 | 58.68 | 17.01 | 17 | 64.12 | 13.52 | |
Follow-up | 21 | 58.52 | 16.23 | 17 | 57.59 | 13.72 | |
DPSS-R Propensity | Pretreatment | 22 | 20.59 | 5.97 | 22 | 16.82 | 4.00 |
2 months | 22 | 19.82 | 4.81 | 19 | 17.37 | 2.79 | |
4 months | 19 | 18.37 | 4.32 | 17 | 14.24 | 4.29 | |
Follow-up | 21 | 18.33 | 4.99 | 17 | 14.88 | 3.84 | |
DPSS-R Sensitivity | Pretreatment | 22 | 16.73 | 5.61 | 22 | 13.68 | 3.83 |
2 months | 22 | 15.23 | 5.50 | 19 | 14.58 | 3.29 | |
4 months | 19 | 14.63 | 5.39 | 17 | 10.88 | 2.91 | |
Follow-up | 21 | 14.38 | 5.12 | 17 | 12.41 | 4.78 | |
OQ-45 | Pretreatment | 22 | 87.14 | 17.77 | 22 | 84.86 | 20.51 |
2 months | 22 | 62.55 | 21.46 | 19 | 74.42 | 18.70 | |
4 months | 19 | 61.16 | 20.55 | 18 | 77.72 | 20.73 | |
Follow-up | 21 | 64.90 | 20.67 | 17 | 72.12 | 26.63 |
Note. DBT-ST = dialectical behavior therapy skills training; ASG = activities-based support group; STAXI = State-Trait Anger Expression Inventory; ESS = Experience of Shame Scale; DPSS-R = Disgust Propensity and Sensitivity Scale—Revised; OQ-45 = Outcome Questionnaire-45.
At pretreatment, emotion dysregulation significantly correlated with anger suppression (r = .58, p < .001), shame (r = .48, p = .001), and distress (r = .38, p = .01), each of which also significantly correlated with one another: anger suppression with shame (r = .49, p < .001) and distress (r = .57, p < .001), and shame with distress (r = .56, p < .001). Anger expression marginally correlated with shame (r = −.28, p = .06) and disgust propensity (r = .27, p = .07) and significantly correlated with disgust sensitivity (r = .30, p < .05). Disgust propensity significantly correlated with disgust sensitivity (r = .74, p < .001).
Changes in Emotional Distress as a Function of Emotion Dysregulation
Anger
Longitudinal ITT analyses of the STAXI revealed a main effect of time during treatment such that all participants reported significantly less anger suppression over time (Table 2). Furthermore, a significant time-by-condition interaction indicated that participants in DBT-ST improved more and faster (i.e., declined more steeply in anger suppression over time) than those in ASG, d = 0.93. In fact, when considered separately, only DBT-ST participants significantly improved, as shown by a significant slope for DBT-ST but not for ASG: estimate = −1.97, SE = 0.58, t(119.53) = −3.40, p < .001; estimate = −0.07, SE = 0.62, t(112.56) = −0.11, p = .92; respectively. A simple mediation analysis revealed that condition indirectly influenced anger suppression through emotion dysregulation, controlling for pretreatment levels of both variables: DBT-ST participants reported less emotion dysregulation at 2 months than ASG participants (α = -15.88), and participants who reported less emotion dysregulation at 2 months reported less anger suppression at 4 months (β = 0.14, Table 3). The confidence interval for the indirect effect (αβ = −2.15) did not include zero (−4.66 to −0.86). No evidence suggested condition influenced anger suppression independent of its effect on emotion dysregulation (c' = −0.40, p = .74). During follow-up, participants did not report significant changes in anger suppression, suggesting they maintained their gains; neither the main effect of time nor the time-by-condition interaction was significant (Table 2). With regard to anger expression, participants did not report any significant changes; neither the main effect of time nor the time-by-condition interaction was significant during treatment or follow-up (Table 2). A simple mediation analysis revealed no indirect or direct effect of condition on anger expression at 4 months (Table 3).
Table 2.
Treatment | Follow-Up | ||||||
---|---|---|---|---|---|---|---|
|
|||||||
Outcome | Effect | df | F | p | df | F | p |
STAXI Anger-In | Time | 1, 115.83 | 5.79 | .02 | 1, 135.99 | 0.00 | .95 |
Interaction | 1, 115.83 | 5.06 | .03 | 1, 135.99 | 1.48 | .23 | |
STAXI Anger-Out | Time | 1, 97.31 | 1.55 | .22 | 1, 132.11 | 0.60 | .44 |
Interaction | 1, 97.31 | 0.00 | .99 | 1, 132.11 | 0.00 | .97 | |
ESS | Time | 1, 95.13 | 7.55 | < .01 | 1, 138.68 | 1.80 | .18 |
Interaction | 1, 95.13 | 0.66 | .42 | 1, 138.68 | 3.52 | .06 | |
DPSS-R Propensity | Time | 1, 129.29 | 7.82 | < .01 | 1, 120.77 | 0.00 | 1.00 |
Interaction | 1, 129.29 | 0.01 | .93 | 1, 120.77 | 0.00 | .96 | |
DPSS-R Sensitivitya | Time | 1, 121.96 | 9.26 | < .01 | 1, 133.49 | 0.42 | .52 |
Interaction | 1, 121.96 | 0.04 | .84 | 1, 133.49 | 0.03 | .85 | |
OQ-45 | Time | 1, 127.99 | 18.71 | < .001 | 1, 130.60 | 0.41 | .52 |
Interaction | 1, 127.99 | 5.64 | .02 | 1, 130.60 | 2.95 | .09 |
Note. HLM = hierarchical linear modeling; STAXI = State-Trait Anger Expression Inventory; ESS = Experience of Shame Scale; DPSS-R = Disgust Propensity and Sensitivity Scale—Revised; OQ-45 = Outcome Questionnaire-45.
Transformed ( ).
Shame
ITT analyses of the ESS revealed a main effect of time during treatment such that, when considered together, all participants reported significantly less shame over time (Table 2). A nonsignificant time-by-condition interaction indicated that the difference between conditions over time was not significant. A simple mediation analysis revealed that condition indirectly influenced shame through emotion dysregulation, controlling for pretreatment levels of both variables: DBT-ST participants reported less emotion dysregulation at 2 months when compared with ASG participants (α = −13.02, Table 3), and participants who reported less emotion dysregulation at 2 months reported less shame at 4 months (β = 0.30). The confidence interval for the indirect effect (αβ = −3.89) did not include zero (−9.22 to −0.41). No evidence suggested that condition influenced shame independent of its effect on emotion dysregulation (c' = −0.87, p = .82). During follow-up, the main effect of time was nonsignificant, suggesting participants maintained their gains (Table 2).
Disgust
ITT analyses of the DPSS-R revealed main effects of time during treatment such that participants in both conditions reported significantly less disgust propensity and disgust sensitivity over time (Table 2). Nonsignificant time-by-condition interactions indicated that there were no significant differences in either outcome between conditions over time. Simple mediation analyses revealed that condition did not indirectly influence either outcome at 4 months through its effect on emotion dysregulation at 2 months (Table 3). However, condition influenced both outcomes independent of its effect on emotion dysregulation, controlling for pretreatment levels of each variable: DBT-ST participants reported greater disgust propensity and disgust sensitivity than ASG participants at 4 months when covarying emotion dysregulation at 2 months (c' = 3.55, p = .01; c' = 0.35, p = .03; respectively). Still, when considered together, all participants reported significantly less disgust propensity and disgust sensitivity over time in treatment, and at follow-up, participants did not report significant changes in either outcome, suggesting they maintained their gains (Table 2).
Changes in Distress as a Function of Emotion Dysregulation
ITT analyses of the OQ-45 revealed a main effect of time during treatment such that, when considered together, all participants reported significantly less distress over time (Table 2). Moreover, a significant time-by-condition interaction indicated that participants in DBT-ST improved significantly more and faster (i.e., declined more steeply in distress over time), d = 1.04. In fact, when considered separately, only DBT-ST participants significantly improved, as shown by a significant slope for DBT-ST but not for ASG: estimate = −13.96, SE = 2.89, t(133.01) = −4.84, p < .001; estimate = −4.06, SE = 3.00, t(123.18) = −1.35, p = .18; respectively. A simple mediation analysis showed that condition indirectly influenced distress through emotion dysregulation, controlling for pretreatment levels of both variables: Participants in DBT-ST reported less emotion dysregulation at 2 months than those in ASG (α = −13.01), and participants who reported less emotion dysregulation at 2 months reported less distress at 4 months (β = 0.37, Table 3). The confidence interval for the indirect effect (αβ = −4.77) did not include zero (−13.99 to −0.21). No evidence suggested that condition influenced distress independent of its effect on emotion dysregulation (c' = −12.84, p = .06). During follow-up, a nonsignificant main effect of time indicated that, when considered all together, participants did not report significant changes in distress (Table 2).
Discussion
This study evaluated the efficacy of DBT skills training (DBT-ST) as a treatment for emotion dysregulation with a transdiagnostic sample of anxious and depressed adults. In the original study, we demonstrated that DBT-ST was more effective in reducing emotion dysregulation compared to an activities-based support group (ASG; Neacsiu et al., 2014). Building on this previous finding, in this study we examined whether targeting emotion dysregulation also improves dysregulation of distress and of specific emotions such as anger, shame, and disgust. As we hypothesized, DBT-ST led to greater and faster reductions in participants’ self-reported anger suppression and distress compared to ASG, and emotion dysregulation explained these differences. All participants, regardless of condition, improved in shame, disgust propensity, and disgust sensitivity. Contrary to our hypothesis, there were no significant changes in anger expression in either condition. Taken together, these findings suggest that DBT-ST is an effective treatment for dysfunctional emotions that are commonly associated with anxiety and depression, although its benefits may be less pervasive than hypothesized when compared to an active supportive therapy condition.
The finding that DBT-ST (but not ASG) reduces distress significantly over time (medium effect size) extends the current body of literature. The average OQ-45 score in the sample at pretreatment was comparable to the typical distress reported by community mental health clients (e.g., Lunnen, Ogles, & Pappas, 2008). Although individual distress was not directly targeted, as part of their homework clients were encouraged to use skills on current distressing problems. Given that the OQ-45 questionnaire assesses problems with psychological distress, interpersonal conflicts, and social roles, improvements in this outcome measure demonstrate better functioning in a variety of life domains. This finding may therefore support the importance of learning and using new skills as a means of coping effectively with social and psychological stressors.
Only participants in the DBT-ST condition significantly decreased anger suppression, a maladaptive regulation strategy for anger. This suggests that the skills taught in this treatment may have helped participants regulate their anger more effectively, possibly by engaging in more goal-directed behaviors instead of using suppression as a coping strategy. Previous research has demonstrated that adults with anxiety disorders have a tendency to suppress their angry feelings and engage in avoidance behaviors in response to negative affect (Erwin et al., 2003). Furthermore, suppressing anger may lead to increased sympathetic activation of the cardiovascular system, exacerbating the physiological responses associated with anger (Gross, 1998; Gross & Levenson, 1997). Therefore, teaching emotion regulation skills to manage anger more effectively than suppression may play a key role in treating emotional dysfunction within anxiety and depression. In addition, we found that there were no significant changes in the participants’ anger suppression at follow-up, suggesting that they may have maintained their skills even after the treatment phase.
We also found that participants did not change significantly in their self-reported anger expression. This finding contradicts existing data that indicated superiority of DBT over control conditions in reducing anger expression in incarcerated or BPD patients (Evershed et al., 2003; Koons et al., 2001). In our study, participants were primarily anxious and depressed adults who overall reported higher levels of anger suppression (M = 20.80, SD = 4.11) than anger expression (M = 15.25, SD = 4.96) at pretreatment. This suggests that our sample experienced greater problems with suppressing their angry feelings than expressing them towards others, which is consistent with research demonstrating the association between mood disorders and the tendency to suppress anger (Erwin et al., 2003). Our results also suggest that emotion dysregulation was associated with anger suppression, but not anger expression, in our sample. As a result, participants in both conditions may not have specifically addressed aggression or outward expressions of anger as a target of treatment. Therefore, these findings may suggest that teaching skills may improve problems with anger by offering effective strategies to replace maladaptive anger regulation, regardless of whether the maladaptive regulation is suppression or problematic expression. This might account for the efficacy of DBT-ST with different facets of anger dysregulation in different samples. Future research should also investigate whether DBT-ST has an effect on sensitivity and reactivity to anger cues, not only anger suppression or expression.
These results also have clinical implications for the treatment of anger with DBT-ST. The skills taught within this treatment may help individuals more effectively identify and manage anger using emotion regulation and problem-solving strategies. This may improve treatment compliance, since anxious and depressed patients often react to anger with avoidance behaviors that prevent them from fully engaging in therapy (Erwin, Heimberg, Schneier, & Liebowitz, 2003; Newman, 2011). This hypothesis is supported by the lower dropout rate and the higher compliance with research protocols in the DBT-ST than in the ASG in this sample. Furthermore, therapists can use clinical skills drawn from DBT to effectively address patients’ anger as it manifests in therapy-interfering behaviors (Chapman & Rosenthal, 2016).
There was no significant difference between ASG and DBT-ST in changing dysregulated shame, and participants in both conditions improved in their experience of shame. This surprising finding suggests that the factors common to both therapies may have led to this improvement, which is consistent with previous literature demonstrating the powerful effects of common factors of therapy on treatment outcomes (Hubble, Duncan, & Miller, 1999; Luborsky et al., 2002). For example, participants in both conditions learned to discuss and accept difficult feelings, a process of change called emotional experiencing that leads to improvement within mood disorders (Klein, Matthieu-Coughlan, & Kiesler, 1986; Castonguay, Goldfried, Wiser, Rau, & Hayes, 1996). Addressing problems within the supportive environments of ASG and DBT-ST may have led to the reductions in shame. Therefore, the similar outcomes from both conditions suggest that changes in shame may be explained by common factors of therapy and do not reflect unique effects of DBT-ST. This explanation is only a hypothesis as we did not specifically assess and test for the effects of common factors (e.g., therapeutic alliance). In support of this hypothesis comes the finding that emotion dysregulation did mediate changes in shame. Alternatively, the study may be insufficiently powered to detect a difference between conditions. Of note, the average ESS score at pretreatment in this sample (M = 69.2, SD = 16.3) is lower than in other clinical trials where DBT has been shown to effectively reduce shame (e.g., M = 85.8, SD = 10.2, in Harned, Korslund, Foa, & Linehan, 2012). Therefore, future research should examine this hypothesis in larger samples and should investigate if DBT-ST has particularly beneficial impacts on levels of shame dysfunction higher than in this current study.
There was no significant difference between conditions in changing participants’ frequency of feeling disgust (disgust propensity) and their emotional impact of disgust (disgust sensitivity). As with shame, the therapeutic factors common to both ASG and DBT-ST may account for improvements across all participants. For example, in both types of therapy, exposure to negative thoughts and feelings may have been particularly helpful for reducing negative interpretations and aversion responses associated with disgust (Smits, 2002).. Disgust measured by the DPSS-R is more often studied within anxiety disorders that involve an aversion to a specific stimulus, such as phobias (Thorpe & Salkovskis, 1998; Smits, Telch, & Randall, 2002; Olatunji, 2006; Olatunji, Sawchuk, de Jong, & Lohr, 2006), or contamination-based obsessive-compulsive disorder (Berle & Phillips, 2006; Olatunji, Williams, Lohr, & Sawchuk, 2005). Furthermore, psychometric data with the DPSS-R indicate that this scale might not be a useful measure of disgust or longitudinal changes in disgust within populations that do not suffer from these particular disorders (Olatunji et al., 2007). Therefore, our results may reflect a failure to accurately capture potential changes in disgust within our transdiagnostic sample where a specific aversion to a stimulus was not frequently reported. The final composition of our transdiagnostic sample may also explain the lack of finding between conditions. Future studies may consider different measures for disgust or narrowing the transdiagnostic sample to include people who report specific aversions to better understand the effects of DBT skills on disgust. Alternatively, future studies may consider measuring affect sensitivity and difficulty with tolerating affect (e.g., Bernstein, Zvolensky, Vujanovic, & Moos, 2009) more broadly and examine the effects of DBT skills on such measures.
Taken together, our results suggest that the treatment of emotion dysregulation may be promising for reducing distress and some problematic emotions associated with anxiety and depression. In particular, DBT-ST is superior to supportive therapy in improving participants’ self-reported problematic emotions that were most relevant to the included sample, while both therapies are effective in improving shame and disgust, emotions that affected participants in the current sample less than others in other DBT studies. Additionally, changes in emotion dysregulation significantly mediated the relationship between condition and improvements in distress and anger suppression. These results suggest that emotion dysregulation may be a mechanism of change for these outcomes.
Furthermore, these findings have important implications for using a transdiagnostic approach to treat emotion dysregulation. According to a dimensional model of emotional disorders, negative emotions are regulated ineffectively by strategies (e.g., suppression or avoidance) that actually increase distress and lead to emotion dysregulation (Barlow, Allen, & Choate, 2004). Thus, problems with specific emotions (e.g., anger suppression) are different manifestations of this emotion dysregulation that is common across different anxiety and depressive disorders. The fundamental components of treating this emotion dysregulation consist of changing cognitive appraisals, preventing emotional avoidance, and facilitating behaviors that are not associated with the problematic emotion (Barlow et al., 2004). Therefore, the success of DBT-ST in treating these specific problematic emotions may be attributed to this intervention’s incorporating these fundamental components of transdiagnostic therapy. Additionally, the discussion of homework assignments (e.g., daily use of skills) in the DBT-ST group could have also contributed to its positive effects. As participants in DBT-ST learned to regulate their emotions with skills more generally, this improvement may also have affected the specific negative emotions that caused them distress. These changes in problems with specific emotions may in turn have led to decreases in anxiety, as the results from the previous study demonstrated (Neacsiu et al., 2014).
This study also leads to interesting hypotheses about how the common and specific factors of therapy are related to treatment outcomes. While DBT-ST was effective in treating a range of negative emotions, supportive therapy was equally effective in treating shame and disgust. This suggests that some negative emotions may be treated by the supportive elements common across different types of therapy. Future research should replicate these findings and examine finer grained mechanisms of change for problematic emotions.
This study has several limitations that should be taken into consideration. First, therapist characteristics may have had an impact on the treatment outcomes since different therapists delivered the interventions (Wampold & Serlin, 2000). Second, we had a small sample size (although, the sample size was comparable to other studies evaluating the efficacy of DBT skills interventions, e.g., Soler et al., 2009). Third, the dropout rate in ASG was high (59%). Fourth, although the conditions did not differ significantly in the number of participants with primary depressive disorders, ASG had significantly more dysthymic participants than DBT-ST. Fifth, anger suppression had low internal consistency. Other studies have also found this subscale of the STAXI to have lower internal consistency (e.g., see Wongtongkam, Ward, Day, & Winefield, 2013 and research cited in their discussion). Possible explanations are heterogeneity of the sample (Streiner, 2003) or more sensitivity to cultural factors (Wongtongkam et al., 2013). Implications are that this particular subscale may not be the best fit to measure anger suppression or that an analysis that accounts for internal consistency (such as clinically reliable change) should be proposed and utilized if this scale is included in treatment studies in the future. Sixth, we focused only on negative emotions although positive emotions and their regulation/dysregulation may be as important in the treatment of anxiety and depression (e.g., Carl, Fairholme, Gallagher, Thompson-Hollands, & Barlow, 2014).
In summary, treatment for emotion dysregulation improves problems with negative emotions within anxiety and depression. Our findings suggest that specific factors of DBT-ST have a superior effect on psychopathology induced distress and anger suppression, while factors common to both ASG and DBT-ST may improve shame and disgust. These findings may fit within a more comprehensive model of psychotherapy effects on psychopathology that needs further testing. Specifically, DBT skills training reduces emotion dysregulation, which in turn improves problems with specific emotions and general distress. Furthermore, these improvements with the management of specific emotions may impact presence of disorders and function of those who meet criteria for these disorders, which, in essence, is the ultimate goal of mental health treatment. This second part of the model could not be tested in the current study. Nevertheless, future research containing all aspects of such a model could provide clearer answers about the mechanisms of change that are essential in the treatment of anxiety and depression.
Highlights.
We examine a group treatment for transdiagnostic emotion dysregulation.
We conducted a randomized controlled trial using an activities group as control.
DBT skills training was superior in reducing anger suppression and general distress.
Shame, disgust propensity, and disgust sensitivity changed similarly over time.
Emotion dysregulation explained differences between conditions.
Acknowledgments
This research and the completion of the manuscript was supported by an American Psychological Association Dissertation Research Award and by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number 5KL2TR001115 granted to the first author. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The authors would like to thank the therapists and clients who took part in this study and to acknowledge Megan Smith, Laura Murphy, Sara Miller, Annika Benedetto, William Kuo, Dan Finnegan, Kevin King, Heidi Heard, and Nancy Whitney for their contributions.
Footnotes
Data on the present secondary outcomes were also presented as part of the first author’s doctoral dissertation and in conference talks. The main outcomes were reported in Neacsiu, Eberle, Kramer, Wiesmann, & Linehan (2014). Andrada D. Neacsiu receives fees for DBT trainings.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Andrada D. Neacsiu, Duke University Medical Center and University of Washington.
Joscelyn Rompogren, Alliant International University, San Diego, and University of Washington.
Jeremy W. Eberle, University of Washington
Kibby McMahon, Duke University.
References
- Aldao A, Nolen-Hoeksema S, Schweizer S. Emotion-regulation strategies across psychopathology: A meta-analytic review. Clinical Psychology Review. 2010;30(2):217–237. doi: 10.1016/j.cpr.2009.11.004. [DOI] [PubMed] [Google Scholar]
- Andrews B, Qian M, Valentine JD. Predicting depressive symptoms with a new measure of shame: The Experience of Shame Scale. The British Journal of Clinical Psychology. 2002;41(1):29–42. doi: 10.1348/014466502163778. [DOI] [PubMed] [Google Scholar]
- Ansell EB, Sanislow CA, McGlashan TH, Grilo CM. Psychosocial impairment and treatment utilization by patients with borderline personality disorder, other personality disorders, mood and anxiety disorders, and a healthy comparison group. Comprehensive Psychiatry. 2007;48(4):329–336. doi: 10.1016/j.comppsych.2007.02.001. [DOI] [PubMed] [Google Scholar]
- Baker R, Holloway J, Thomas PW, Thomas S, Owens M. Emotional processing and panic. Behavior Research and Therapy. 2004;42:1271–1287. doi: 10.1016/j.brat.2003.09.002. [DOI] [PubMed] [Google Scholar]
- Barlow DH, Allen LB, Choate ML. Toward a unified treatment for emotional disorders. Behavior Therapy. 2004;35(2):205–230. doi: 10.1016/S0005-7894(04)80036-4. [DOI] [PubMed] [Google Scholar]
- Blum A. Shame and guilt, misconceptions and controversies: A critical review of the literature. Traumatology. 2008;14(3):91–102. doi: 10.1177/1534765608321070. [DOI] [Google Scholar]
- Berle D, Phillips ES. Disgust and obsessive–compulsive disorder: An update. Psychiatry: Interpersonal and Biological Processes. 2006;69(3):228–238. doi: 10.1521/psyc.2006.69.3.228. [DOI] [PubMed] [Google Scholar]
- Bernstein A, Zvolensky MJ, Vujanovic AA, Moos R. Integrating anxiety sensitivity, distress tolerance, and discomfort intolerance: A hierarchical model of affect sensitivity and tolerance. Behavior Therapy. 2009;40(3):291–301. doi: 10.1016/j.beth.2008.08.001. [DOI] [PubMed] [Google Scholar]
- Bryk AS, Raudenbush SW. Hierarchical linear models: applications and data analysis methods. Thousand Oaks, California: Sage Publications, Inc; 1992. [Google Scholar]
- Bohus M, Kleindienst N, Limberger MF, Stieglitz RD, Domsalla M, Chapman AL, … Wolf M. The short version of the Borderline Symptom List (BSL-23): Development and initial data on psychometric properties. Psychopathology. 2009;42(1):32–39. doi: 10.1159/000173701). [DOI] [PubMed] [Google Scholar]
- Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical Psychology Review. 2006;26(1):17–31. doi: 10.1016/j.cpr.2005.07.003. [DOI] [PubMed] [Google Scholar]
- Carl JR, Fairholme CP, Gallagher MW, Thompson-Hollands J, Barlow DH. The effects of anxiety and depressive symptoms on daily positive emotion regulation. Journal of Psychopathology and Behavioral Assessment. 2014;36(2):224–236. [Google Scholar]
- Castonguay LG, Goldfried MR, Wiser S, Raue PJ, Hayes AM. Predicting the effect of cognitive therapy for depression: A study of unique and common factors. Journal of Consulting and Clinical Psychology. 1996;64(3):497–504. [PubMed] [Google Scholar]
- Cavanagh K, Davey GCL. The development of a measure of individual differences in disgust. Paper presented to the British Psychological Society.2000. [Google Scholar]
- Chapman AL, Rosenthal MZ. Managing therapy-interfering behavior: Strategies from dialectical behavior therapy. Washington, DC: American Psychological Association; 2016. [Google Scholar]
- Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988. [Google Scholar]
- Cuijpers P, Van Straten A, Warmerdam L. Behavioral activation treatments of depression: A meta-analysis. Clinical Psychology Review. 2007;27(3):318–326. doi: 10.1016/j.cpr.2006.11.001. [DOI] [PubMed] [Google Scholar]
- DiGiuseppe R, Tafrate R, Eckhardt C. Critical issues in the treatment of anger. Cognitive and Behavioral Practice. 1994;1(1):111–132. doi: 10.1016/S1077-7229(05)80089-6. [DOI] [Google Scholar]
- Dunn LM, Dunn LM. Manual for the Peabody Picture Vocabulary Test-Revised. Circle Pines, MN: American Guidance Service; 1981. [Google Scholar]
- Erwin BA, Heimberg RG, Schneier FR, Liebowitz MR. Anger experience and expression in social anxiety disorder: Pretreatment profile and predictors of attrition and response to cognitive-behavioral treatment. Behavior Therapy. 2003;34(3):331–350. doi: 10.1016/S0005-7894(03)80004-7. [DOI] [Google Scholar]
- Evershed S, Tennant A, Boomer D, Rees A, Barkham M, Watson A. Practice-based outcomes of dialectical behaviour therapy (DBT) targeting anger and violence, with male forensic patients: A pragmatic and non-contemporaneous comparison. Criminal Behaviour and Mental Health. 2003;13(3):198–213. doi: 10.1002/cbm.542. [DOI] [PubMed] [Google Scholar]
- Feingold A. Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychological Methods. 2009;14(1):43–53. doi: 10.1037/a0014699. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fergus TA, Valentiner DP, McGrath PB, Jencius S. Shame-and guilt-proneness: Relationships with anxiety disorder symptoms in a clinical sample. Journal of anxiety disorders. 2010;24(8):811–815. doi: 10.1016/j.janxdis.2010.06.002. [DOI] [PubMed] [Google Scholar]
- First MB, Spitzer RL, Gibbon M, Williams JB. The structured clinical interview for DSM-III-R personality disorders (SCID-II). Part I: Description. Journal of Personality Disorders. 1995;9(2):83–91. doi: 10.1521/pedi.1995.9.2.83. [DOI] [Google Scholar]
- Foa EB, Riggs DS, Massie ED, Yarczower M. The impact of fear activation and anger on the efficacy of exposure treatment for posttraumatic stress disorder. Behavior Therapy. 1995;26(3):487–499. doi: 10.1016/S0005-7894(05)80096-6. [DOI] [Google Scholar]
- Gilbert P. The relationship of shame, social anxiety and depression: The role of the evaluation of social rank. Clinical Psychology & Psychotherapy. 2000;7(3):174–189. [Google Scholar]
- Gilbert P, Procter S. Compassionate mind training for people with high shame and self-criticism: Overview and pilot study of a group therapy approach. Clinical Psychology & Psychotherapy. 2006;13(6):353–379. doi: 10.1002/cpp.507. [DOI] [Google Scholar]
- Gratz KL, Gunderson JG. Preliminary data on an acceptance-based emotion regulation group intervention for deliberate self-harm among women with borderline personality disorder. Behavior Therapy. 2006;37:25–35. doi: 10.1016/j.beth.2005.03.002. http://dx.doi.org/10.1016/j.beth.2005.03.002. [DOI] [PubMed] [Google Scholar]
- Gratz KL, Roemer L. Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the Difficulties in Emotion Regulation Scale. Journal of Psychopathology and Behavioral Assessment. 2004;26(1):41–54. doi: 10.1023/B:JOBA.0000007455.08539.94. [DOI] [Google Scholar]
- Gross JJ. Antecedent-and response-focused emotion regulation: Divergent consequences for experience, expression, and physiology. Journal of Personality and Social Psychology. 1998;74(1):224–237. doi: 10.1037//0022-3514.74.1.224. [DOI] [PubMed] [Google Scholar]
- Gross JJ, Levenson RW. Hiding feelings: The acute effects of inhibiting negative and positive emotion. Journal of Abnormal Psychology. 1997;106(1):95–103. doi: 10.1037//0021-843x.106.1.95. [DOI] [PubMed] [Google Scholar]
- Grossman P, Niemann L, Schmidt S, Walach H. Mindfulness-based stress reduction and health benefits: A meta-analysis. Journal of Psychosomatic Research. 2004;57(1):35–43. doi: 10.1016/S0022-3999(03)00573-7. [DOI] [PubMed] [Google Scholar]
- Hallion LS, Ruscio AM. A meta-analysis of the effect of cognitive bias modification on anxiety and depression. Psychological Bulletin. 2011;137(6):940–958. doi: 10.1037/a0024355. [DOI] [PubMed] [Google Scholar]
- Hannan C, Lambert MJ, Harmon C, Nielsen SL, Smart DW, Shimokawa K, Sutton SW. A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology. 2005;61(2):155–163. doi: 10.1002/jclp.20108. [DOI] [PubMed] [Google Scholar]
- Harned MS, Korslund KE, Foa EB, Linehan MM. Treating PTSD in suicidal and self-injuring women with borderline personality disorder: Development and preliminary evaluation of a dialectical behavior therapy prolonged exposure protocol. Behaviour Research and Therapy. 2012;50(6):381–386. doi: 10.1016/j.brat.2012.02.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hawkins KA, Cougle JR. Anger problems across the anxiety disorders: Findings from a population-based study. Depression and Anxiety. 2011;28(2):145–152. doi: 10.1002/da.20764. [DOI] [PubMed] [Google Scholar]
- Hayes AF. Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York: Guilford; 2013. [Google Scholar]
- Hedman E, Ström P, Stünkel A, Mörtberg E. Shame and guilt in social anxiety disorder: Effects of cognitive behavior therapy and association with social anxiety and depressive symptoms. PloS one. 2013;8(4):162–188. doi: 10.1371/journal.pone.0061713. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hofmann SG, Sawyer AT, Witt AA, Oh D. The effect of mindfulness-based therapy on anxiety and depression: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2010;78(2):169–183. doi: 10.1037/a0018555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hofmann SG, Sawyer AT, Fang A, Asnaani A. Emotion dysregulation model of mood and anxiety disorders. Depression and Anxiety. 2012;29(5):409–416. doi: 10.1002/da.21888. [DOI] [PubMed] [Google Scholar]
- Hopko DR, Lejuez CW, Ruggiero KJ, Eifert GH. Contemporary behavioral activation treatments for depression: Procedures, principles, and progress. Clinical Psychology Review. 2003;23(5):699–717. doi: 10.1016/S0272-7358(03)00070-9. [DOI] [PubMed] [Google Scholar]
- Hopko DR, Robertson S, Lejuez CW. Behavioral activation for anxiety disorders. The Behavior Analyst Today. 2006;7(2):212–232. Retrieved from http://dx.doi.org/10.1037/h0100084. [Google Scholar]
- Hubble MA, Duncan BL, Miller SD. The heart and soul of change: What works in therapy. Washington, DC: American Psychological Association; 1999. [Google Scholar]
- Ilardi SS, Craighead WE. The role of nonspecific factors in cognitive-behavior therapy for depression. Clinical Psychology: Science and Practice. 1994;1(2):138–155. doi: 10.1111/j.1468-2850.1994.tb00016.x. [DOI] [Google Scholar]
- Kunimatsu MM, Marsee MA. Child & Youth Care Forum. 3. Vol. 41. Springer; US: 2012. Jun, Examining the presence of anxiety in aggressive individuals: The illuminating role of fight-or-flight mechanisms; pp. 247–258. [Google Scholar]
- Kuo JR, Linehan MM. Disentangling emotion processes in borderline personality disorder: Physiological and self-reported assessment of biological vulnerability, baseline intensity, and reactivity to emotionally evocative stimuli. Journal of Abnormal Psychology. 2009;118(3):531. doi: 10.1037/a0016392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaufman J, Charney D. Comorbidity of mood and anxiety disorders. Depression and Anxiety. 2000;12(S1):69–76. doi: 10.1002/1520-6394(2000)12:1+<69::AID-DA9>3.0.CO;2-K. [DOI] [PubMed] [Google Scholar]
- Kessler RC, Chiu WT, Demler O, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62(6):617–627. doi: 10.1001/archpsyc.62.6.617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klein MH, Mathieu-Coughlan PL, Kiesler DJ. The Experiencing Scale. In: Greenberg LS, Pinsof WM, editors. The psychotherapy process: A research handbook. New York: Guilford Press; 1986. pp. 21–71. [Google Scholar]
- Kim S, Thibodeau R, Jorgensen RS. Shame, guilt, and depressive symptoms: A meta-analytic review. Psychological Bulletin. 2011;137(1):68–96. doi: 10.1037/a0021466. [DOI] [PubMed] [Google Scholar]
- Koh KB, Kim CH, Park JK. Predominance of anger in depressive disorders compared with anxiety disorders and somatoform disorders. Journal of Clinical Psychiatry. 2002;63(6):486–492. doi: 10.4088/jcp.v63n0604. [DOI] [PubMed] [Google Scholar]
- Koons CR, Robins CJ, Tweed JL, Lynch TR, Gonzalez AM, Morse JQ, … Bastian LA. Efficacy of dialectical behavior therapy in women veterans with borderline personality disorder. Behavior Therapy. 2001;32(2):371–390. doi: 10.1016/S0005-7894(01)80009-5. [DOI] [Google Scholar]
- Kring AM, Bachorowski JA. Emotions and psychopathology. Cognition and Emotion. 1999;13(5):575–599. doi: 10.1080/026999399379195. [DOI] [Google Scholar]
- Kulkarni M, Porter KE, Rauch SA. Anger, dissociation, and PTSD among male veterans entering into PTSD treatment. Journal of Anxiety Disorders. 2012;26(2):271–278. doi: 10.1016/j.janxdis.2011.12.005. [DOI] [PubMed] [Google Scholar]
- Lambert MJ, Huefner JC, Reisinger CW. Quality improvement: Current research in outcome management. In: Stricker G, Troy WG, Shueman SA, Stricker G, Troy WG, Shueman SA, editors. Handbook of quality management in behavioral health. Dordrecht, Netherlands: Kluwer Academic Publishers; 1996. pp. 95–110. [DOI] [Google Scholar]
- Linehan M. Cognitive–behavioral treatment of borderline personality disorder. New York: Guilford Press; 1993. [Google Scholar]
- Linehan M. DBT skills training manual. Guilford Publications; New York. N.Y: 2014. [Google Scholar]
- Linehan MM, Korslund KE, Harned MS, Gallop RJ, Lungu A, Neacsiu AD, … Murray-Gregory AM. Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: a randomized clinical trial and component analysis. JAMA psychiatry. 2015;72(5):475–482. doi: 10.1001/jamapsychiatry.2014.3039. [DOI] [PubMed] [Google Scholar]
- Luborsky L, Rosenthal R, Diguer L, Andrusyna TP, Berman JS, Levitt JT, … Krause ED. The dodo bird verdict is alive and well—mostly. Clinical Psychology: Science and Practice. 2002;9(1):2–12. doi: 10.1093/clipsy.9.1.2. [DOI] [Google Scholar]
- Moons WG, Eisenberger NI, Taylor SE. Anger and fear responses to stress have different biological profiles. Brain, behavior, and immunity. 2010;24(2):215–219. doi: 10.1016/j.bbi.2009.08.009. [DOI] [PubMed] [Google Scholar]
- Moscovitch DA, McCabe RE, Antony MM, Rocca L, Swinson RP. Anger experience and expression across the anxiety disorders. Depression and anxiety. 2008;25(2):107–113. doi: 10.1002/da.20280. [DOI] [PubMed] [Google Scholar]
- Muris P. The pathogenesis of childhood anxiety disorders: Considerations from a developmental psychopathology perspective. International Journal of Behavioral Development. 2006;30(1):5–11. doi: 10.1177/0165025406059967. [DOI] [Google Scholar]
- Neacsiu AD, Eberle JW, Kramer R, Wiesmann T, Linehan MM. Dialectical behavior therapy skills for transdiagnostic emotion dysregulation: A pilot randomized controlled trial. Behaviour research and therapy. 2014;59:40–51. doi: 10.1016/j.brat.2014.05.005. [DOI] [PubMed] [Google Scholar]
- Newman CF. When clients' morbid avoidance and chronic anger impede their response to cognitive-behavioral therapy for depression. Cognitive and Behavioral Practice. 2011;18(3):350–361. doi: 10.1016/j.cbpra.2010.07.004. [DOI] [Google Scholar]
- Nolen-Hoeksema S. The role of rumination in depressive disorders and mixed anxiety/depressive symptoms. Journal of Abnormal Psychology. 2000;109(3):504–511. doi:10I037//0021-843X.1093.504. [PubMed] [Google Scholar]
- Orth U, Berking M, Burkhardt S. Self-conscious emotions and depression: Rumination explains why shame but not guilt is maladaptive. Personality and Social Psychology Bulletin. 2006;32(12):1608–1619. doi: 10.1177/0146167206292958. [DOI] [PubMed] [Google Scholar]
- Olatunji BO. Evaluative learning and emotional responding to fearful and disgusting stimuli in spider phobia. Journal of Anxiety Disorders. 2006;20:858–876. doi: 10.1016/j.janxdis.2006.01.005. [DOI] [PubMed] [Google Scholar]
- Olatunji BO, Cisler JM, Deacon BJ, Connolly K, Lohr JM. The Disgust Propensity and Sensitivity Scale-Revised: Psychometric properties and specificity in relation to anxiety disorder symptoms. Journal of Anxiety Disorders. 2007;21(7):918–930. doi: 10.1016/j.janxdis.2006.12.005. [DOI] [PubMed] [Google Scholar]
- Olatunji BO, Cisler J, McKay D, Phillips ML. Is disgust associated with psychopathology? Emerging research in the anxiety disorders. Psychiatry Research. 2010;175(1):1–10. doi: 10.1016/j.psychres.2009.04.007. [DOI] [PubMed] [Google Scholar]
- Olatunji BO, Lohr JM, Sawchuk CN, Tolin DF. Multimodal assessment of disgust in contamination-related obsessive-compulsive disorder. Behaviour Research and Therapy. 2007;45(2):263–276. doi: 10.1016/j.brat.2006.03.004. [DOI] [PubMed] [Google Scholar]
- Olatunji BO, Sawchuk CN, de Jong PJ, Lohr JM. The structural relation between disgust sensitivity and blood–injection–injury fears: A cross-cultural comparison of US and Dutch data. Journal of Behavior Therapy and Experimental Psychiatry. 2006;37(1):16–29. doi: 10.1016/j.jbtep.2005.09.002. [DOI] [PubMed] [Google Scholar]
- Olatunji BO, Williams NL, Lohr JM, Sawchuk CN. The structure of disgust: Domain specificity in relation to contamination ideation and excessive washing. Behaviour Research and Therapy. 2005;43(8):1069–1086. doi: 10.1016/j.brat.2004.08.002. [DOI] [PubMed] [Google Scholar]
- Pasquini M, Picardi A, Biondi M, Gaetano P, Morosini P. Relevance of anger and irritability in outpatients with major depressive disorder. Psychopathology. 2004;37(4):155–160. doi: 10.1159/000079418. [DOI] [PubMed] [Google Scholar]
- Preacher KJ, Kelley K. Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological Methods. 2011;16(2):93–115. doi: 10.1037/a0022658. [DOI] [PubMed] [Google Scholar]
- Raudenbush SW, Spybrook J, Congdon R, Liu X, Martinez A, Bloom H, et al. Optimal design software for multi-level and longitudinal research (Version 3.01) [Software] 2011 Retrieved from http://www.wtgrantfoundation.org/
- Rizvi SL, Linehan MM. The treatment of maladaptive shame in borderline personality disorder: A pilot study of “opposite action”. Cognitive and Behavioral Practice. 2005;12(4):437–447. doi: 10.1016/S1077-7229(05)80071-9. [DOI] [Google Scholar]
- Royston P. Estimating departure from normality. Statistics in Medicine. 1991;10(8):1283–1293. doi: 10.1002/sim.4780100811. [DOI] [PubMed] [Google Scholar]
- Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods. 2002;7(2):147–177. doi: 10.1037//1082-989X.7.2.147. [DOI] [PubMed] [Google Scholar]
- Schomerus G, Matschinger H, Angermeyer MC. The stigma of psychiatric treatment and help-seeking intentions for depression. European archives of psychiatry and clinical neuroscience. 2009;259(5):298–306. doi: 10.1007/s00406-009-0870-y. [DOI] [PubMed] [Google Scholar]
- Shapiro SS, Wilk MB. An analysis of variance test for normality (complete samples) Biometrika. 1965;52:591–611. doi: 10.2307/2333709. [DOI] [Google Scholar]
- Smits JAJ, Telch MJ, Randall PK. An examination of the decline in fear and disgust during exposure-based treatment. Behaviour Research and Therapy. 2002;40(11):1243–1253. doi: 10.1016/S0005-7967(01)00094-8. [DOI] [PubMed] [Google Scholar]
- Soler J, Pascual JC, Tiana T, Cebrià A, Barrachina J, Campins MJ, … Pérez V. Dialectical behaviour therapy skills training compared to standard group therapy in borderline personality disorder: A 3-month randomised controlled clinical trial. Behaviour Research and Therapy. 2009;47(5):353–358. doi: 10.1016/j.brat.2009.01.013. [DOI] [PubMed] [Google Scholar]
- Spielberger CD. State-Trait Anger Expression Inventory (STAXI). Research edition. Odessa, FL: Psychological Assessment Resources; 1988. [Google Scholar]
- Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. Journal of personality assessment. 2003;80(1):99–103. doi: 10.1207/S15327752JPA8001_18. [DOI] [PubMed] [Google Scholar]
- Suveg C, Sood E, Comer JS, Kendall PC. Changes in emotion regulation following cognitive-behavioral therapy for anxious youth. Journal of Clinical Child & Adolescent Psychology. 2009;38(3):390–401. doi: 10.1080/15374410902851721. [DOI] [PubMed] [Google Scholar]
- Teasdale JD, Cox SG. Dysphoria: Self-devaluative and affective components in recovered depressed patients and never depressed controls. Psychological Medicine. 2001;31(07):1311–1316. doi: 10.1017/s003329170100424x. [DOI] [PubMed] [Google Scholar]
- Thorpe SJ, Salkovskis PM. Studies on the role of disgust in the acquisition and maintenance of specific phobias. Behaviour Research and Therapy. 1998;36(9):877–893. doi: 10.1016/s0005-7967(98)00066-7. [DOI] [PubMed] [Google Scholar]
- Van Overveld WJM, de Jong PD, Peters ML, Cavanagh K, Davey GCL. Disgust propensity and disgust sensitivity: Separate constructs that are differentially related to specific fears. Personality and Individual Differences. 2006;41(7):1241–1252. [Google Scholar]
- Verbeke G. Linear mixed models for longitudinal data. In: Verbeke G, Molenberghs G, editors. Linear mixed models in practice. New York, NY: Springer; 1997. pp. 108–114. [Google Scholar]
- Wampold B, Serlin RC. The consequences of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods. 2000;5(4):425–433. doi: 10.1037/1082-989x.5.4.425. [DOI] [PubMed] [Google Scholar]
- Wells MG, Burlingame GM, Lambert MJ, Hoag MJ, Hope CA. Conceptualization and measurement of patient change during psychotherapy: Development of the Outcome Questionnaire and Youth Outcome Questionnaire. Psychotherapy: Theory, Research, Practice, Training. 1996;33(2):275–283. [Google Scholar]
- Wongtongkam N, Ward PR, Day A, Winefield H. Reliability and vailidity of self-reported questionnaires related to adolescent violence and consequences, Thailand. International journal of social science studies. 2013;1(2):82–92. [Google Scholar]