Quality or quantity? A multistudy analysis of emotion regulation skills deficits associated with Borderline Personality Disorder

Abstract

Linehan’s (1993) biosocial theory posits that people with Borderline Personality Disorder (BPD) have emotion regulation skills deficits characterized by 1) less frequent use of adaptive emotion regulation strategies, 2) more frequent use of maladaptive strategies, or 3) poorer quality strategy implementation (i.e., strategies implemented less skillfully). We tested these possibilities among participants with BPD, Major Depressive Disorder (MDD), or no disorder (controls). Study 1 participants (N = 272) were recruited online; Study 2 participants (N = 90) completed in-person diagnostic assessments. The BPD groups reported greater use of maladaptive strategies than the MDD (d = .35) and control (d = 1.54) groups and lower quality implementation than the MDD (d = −.33) and control groups (d = −.97). BPD participants reported similar use of adaptive strategies as the MDD group (d = −.09) but less use than controls (d = −.47). BPD may be uniquely characterized by overuse of maladaptive strategies and poorer quality emotion regulation implementation.

Borderline Personality Disorder (BPD) is a severe psychiatric disorder characterized, in part, by chronic suicidality, affective lability, interpersonal dysfunction, and impulsivity (American Psychiatric Association [APA], 2013). It is estimated to affect 1.6% of the general population (Torgersen, 2009) but up to 20% of psychiatric inpatients (Gunderson & Links, 2008). While there may be no gender differences in the rates of BPD in the general population (Grant et al., 2008), 75% of those presenting with BPD for clinical treatment are female (Widiger & Trull, 1993). In Linehan’s biosocial theory (Crowell, Beauchaine, & Linehan, 2009; Linehan, 1993), BPD is posited to involve dysregulated emotion regulation functioning (Linehan, 1993, p. 43). For example, people with BPD consistently self-report greater difficulties in emotion regulation than healthy controls (HCs) and people with anxiety and mood disorders (Fletcher et al., 2014; Kuo & Linehan, 2009; Neacsiu et al., 2015), Obsessive-Compulsive Personality Disorder (Steenkamp et al., 2015), and people who engage in non-suicidal self-injury (NSSI) behaviors (Turner et al., 2015). However, emotion dysregulation has been shown to play a role in many disorders (Aldao, Nolen-Hoeksema, & Schweizer, 2010), emphasizing the need to characterize its specific presentations transdiagnostically, as some researchers have begun to do.

Emotion Regulation Skills Deficits

Frequency of skill use.

Emotion regulation strategies are used to influence the onset, duration, and/or type of emotion experienced (Gross & Thompson, 2007). Emotion regulation strategies are often classified as generally adaptive (e.g., acceptance, cognitive reappraisal, problem-solving) or generally maladaptive (e.g., avoidance, rumination, self-harm; Aldao & Nolen-Hoeksema, 2012). In one study of participants recruited online (Neacsiu & Tkachuck, 2016), those who likely met criteria for BPD (n = 29) reported using adaptive emotion regulation strategies less frequently than those who likely met criteria for another cluster B personality disorder (PD; n = 22; i.e., Antisocial PD, Histrionic PD, or Narcissistic PD) and those who likely met criteria for no PD (n = 77). Conversely, those in the likely BPD group and those in the other PD group reported using maladaptive strategies more frequently than those in the no PD group.

Daros, Guevara, Uliaszek, McMain and Ruocco (2018) compared participants diagnosed with BPD (n = 30), a DSM-IV-TR (APA, 2000) mood or anxiety disorder (n = 30), and no psychiatric disorder (HCs; n = 32). In this study, participants with BPD reported using maladaptive strategies more frequently than participants in either of the other groups, while those with BPD reported a similar frequency of adaptive strategy use as those with a mood or anxiety disorder and a lower frequency of adaptive strategy use than those in the HC group.

In a separate study of participants with BPD (n = 24), MDD (n = 19), or neither disorder/HCs (n = 32), participants in the BPD group and the MDD group reported using adaptive emotion regulation strategies less frequently and maladaptive responses more frequently than those in the HC group (Sauer et al., 2016). There were no significant differences between participants in the BPD and MDD groups on these measures. Further, when presented with negatively-valenced images in a laboratory task, there were no differences among the three groups in the frequency of use of cognitive reappraisal or distraction. The findings from these three studies suggest those with BPD use adaptive strategies less frequently and maladaptive strategies more frequently than HCs. People with BPD may use adaptive strategies as frequently as those with mood or anxiety disorders but less frequently than those with another Cluster B PD. People with BPD may use maladaptive strategies either more frequently or with similar frequency as those with a mood or anxiety disorder or a Cluster B PD.

Quality of skill use.

While it is important to measure the frequency of emotion regulation strategy use, frequency is at best “a proxy for skillful behavior” (Neacsiu, Rizvi, & Linehan, 2010). One direct method of measuring skillful behavior is for independent coders to rate the quality of participants’ emotion regulation skill implementation in response to standardized hypothetical scenarios without regard to the outcome of the skill use. This approach has several advantages. First, the use of independent coders reduces the likelihood that differences in the implementation of skills could be due to participants in different diagnostic groups systematically under- or over-estimating the quality of their skill use (Morey, 2014; Winter et al., 2015). Second, standardized hypothetical scenarios enhance internal validity by presenting the same stimulus to all participants, while potentially enhancing external validity compared to standard laboratory tasks (e.g., image or video presentation) by describing real-life situations. Finally, rating the quality of skills without regard to the outcome distinguishes skill quality from skill effectiveness. This is an important distinction because, while skill effectiveness describes the outcome of emotion regulation skill use (e.g., reductions in negative affect), skill quality describes the mechanism or process by which that skill is used. Understanding the process by which participants use skills allows researchers and clinicians to identify if a skill is being used as intended and, if not, at what point in the process the breakdown occurs.

One validated measure of emotion regulation skill quality is the Ways of Responding scale (WOR; Barber & DeRubeis, 1992). This measure presents participants with six hypothetical stressful scenarios. Examples include being turned down for a job for the third time in a week, learning that one’s family is disappointed by one’s decision to postpone a visit, and coming home to a note from a romantic partner saying that they have left the relationship. Participants are provided with initial negative thoughts about each scenario (e.g., “Relationships never turn out well for me. I’m never going to be happy.”) and are asked to write down their further thoughts in response to the scenario as well as what they would do to handle the scenario. Independent coders then rate the overall quality of the responses to each scenario based on how likely the response would be to improve the average person’s mood or circumstances. The WOR is often used in studies of participants diagnosed with MDD. Across two studies (Adler, Strunk, & Fazio, 2015; Barber & DeRubeis, 1992), participants with MDD demonstrated medium-to-large sized differences in WOR scores (ds: −.63 - −1.82) compared to HCs. This preliminary evidence suggests that the quality of emotion regulation skill implementation, as measured by the WOR, may differ between healthy controls and those with mood disorders. However, there is no evidence, to our knowledge, of how the quality of emotion regulation skill implementation among those with BPD compares to either of these groups.

Current Studies

In the current studies, we aimed to compare both the frequency and quality of implementation of emotion regulation skills among three groups of participants: those with likely or diagnosed BPD, those with likely or diagnosed MDD, and healthy control participants (HCs). We recruited MDD groups to determine if any emotion regulation skills deficits were unique to BPD or were more indicative of general psychopathology. Based on prior theory and research, BPD is associated with prolonged experiences of a wider variety of negative emotions (e.g., anxiety, anger, shame) than MDD (e.g., primarily sadness) that may lead to more difficulties engaging in adaptive emotion regulation (Dixon-Gordon, Weiss, Tull, DiLillo, Messman-Moore, & Gratz, 2015). We hypothesized that participants in the BPD group would report using adaptive emotion regulation strategies less frequently and demonstrate poorer quality emotion regulation implementation than those in the HC or MDD groups. Similarly, we further hypothesized that participants in the BPD group would report using maladaptive emotion regulation strategies more frequently than those in the HC or MDD groups.

In line with calls for replication studies in clinical psychology (Tackett et al., 2017), we conducted two studies. In Study 1, we recruited participants online from Amazon’s Mechanical Turk (MTurk) website¹ to obtain a relatively large sample of mixed gender participants. We divided participants into three groups based on scores on validated measures of depressive symptoms and BPD features with established threshold scores. In Study 2, we recruited female participants to complete in-person structured diagnostic interviews to validate the results from Study 1 in a clinical sample. Participants in each study completed the same measures of emotion regulation frequency and quality. Finally, we conducted a mini meta-analysis from both studies to synthesize the results.

Materials and Methods – Study 1

Participants

An initial sample of 717 participants was recruited in three waves from MTurk between November and December 2016. Participants were 36.36 years old (SD = 10.94), on average. The majority of participants identified as female (60.4%) and Caucasian (75.8%). A plurality of participants reported they were either married (33.8%) or single (29.6%). Similar proportions of participants reported they had either attained a four-year college degree (34.4%) or attended some college (31.5%). The median estimated annual income in the current sample was $40,000 - $49,999. Participants were excluded if they were younger than 18 years old at the beginning of the study; if they could not read English; if they were registered with MTurk as living outside the United States; or if their MTurk approval rating was less than 95%.

Measures

Demographics.

Participants self-reported their age, gender, ethnicity, marital status, income, education, and history of psychotherapy and psychotropic medication utilization.

DBT Ways of Coping Checklist (DBT-WCCL; Neacsiu, Rizvi, Vitaliano, Lynch, & Linehan, 2010).

The DBT-WCCL is a 59-item self-report scale designed to assess participants’ use of DBT skills (e.g., acceptance, cognitive reappraisal, problem-solving) and maladaptive coping responses (e.g., avoidance, denial, self-blame). Participants rate how often they used each coping response in the previous month using a four-point scale, resulting in two subscales: a DBT Skills Subscale (DSS) and a Dysfunctional Coping Subscale (DCS). In the current sample, both DSS (α = .95) and DCS (α = .92) items exhibited excellent internal consistency.

Patient Health Questionnaire (PHQ-9; Kroenke, Spitzer, & Williams, 2001).

The PHQ-9 is a 9-item self-report scale designed to assess DSM-IV-TR (APA, 2000) MDD criteria. Participants use a 0 (not at all) to 3 (nearly every day) scale to rate how often they have experienced each symptom in the past two weeks, resulting in a total score. In the current sample, PHQ-9 items exhibited excellent internal consistency (α = .92).

Personality Assessment Inventory – Borderline Subscale (PAI-BOR; Morey, 1991).

The PAI-BOR is a 24-item self-report scale designed to assess features of Borderline Personality Disorder, including affective instability, identity problems, negative relationships, and NSSI. Participants use a four-point scale to rate how true each statement is of them, resulting in a total score. In the current sample, PAI-BOR items exhibited excellent internal consistency (α = .94).

Ways of Responding Scale (WOR; Barber & DeRubeis, 1992).

The WOR is a written measure of compensatory skills taught in cognitive therapy. Participants read six hypothetical stressful situations² in which initial negative thoughts regarding the event are provided. Participants then generate responses describing what they would think and do next. Two trained female undergraduate coders, blind to participant information, independently rated the overall quality of each response. Quality is defined in the WOR as how likely the response would be to improve the participant’s mood and circumstances. Higher quality responses typically include more adaptive strategies and more detail regarding how these strategies would be used. In the current sample, coders demonstrated excellent agreement regarding overall quality (intraclass correlation coefficient [ICC] = .93) when averaged across all six responses for each participant.

Study Flow and Procedures

All participants were users of Amazon’s MTurk website and provided informed consent before beginning the study. This study was approved by the local Institutional Review Board, and all participants were compensated $4.00 upon study completion.

Participants were recruited in three waves. In Wave 1, participants who did not meet the exclusion criteria were recruited. These participants completed all measures in a randomized order. In Wave 2, participants were recruited if they did not meet the exclusion criteria and scored above Morey’s (1991) threshold score on the PAI-BOR (≥ 38) suggesting the likely presence of BPD. Previously, we found that this PAI-BOR threshold score accurately categorizes those with and without BPD (Southward & Cheavens, 2018). These participants completed the PAI-BOR first and then completed all remaining measures in a randomized order. In Wave 3, participants were recruited if they did not meet the exclusion criteria, scored below Morey’s (1991) threshold score on the PAI-BOR (< 38), and met likely criteria for MDD based on responses to the PHQ-9³. These participants completed the PAI-BOR and the PHQ-9 in a randomized order before completing all other measures in a randomized order.

Although 769 unique responses were originally recorded, 52 of these were removed because they had a duplicate Internet Service Provider address and/or geographical location (n_remaining = 717). Participants were further deemed ineligible who scored < 38 on the PAI-BOR in Wave 2 (n = 43; n_remaining = 674), who did not meet likely MDD criteria on the PHQ-9 in Wave 3 (n = 165; n_remaining = 509), who met likely MDD criteria but scored ≥ 38 on the PAI-BOR in Wave 3 (n = 126; n_remaining = 383), and who provided no data after consenting to the study across waves (n = 27; n_remaining = 356).

Demographic Comparisons

Of the 356 participants deemed eligible for the study, 272 (76.4%) provided complete data. There were no differences in average age between completers (M = 36.40, SD = 10.61) and non-completers (M = 36.24, SD = 12.03), t(354) = 0.12, p = .90, 95% CI [–2.86, 2.53], Cohen’s d = .01; the proportion of women among completers (59.2%) and non-completers (64.3%), χ²(1) = 0.69, p = .40, 95% CI [–7.56, 16.95]; or the proportion of Caucasian participants among completers (75.7%) and non-completers (76.2%), χ²(1) = 0.01, p = .93, 95% CI [–11.20, 10.70].

We then examined the PAI-BOR and PHQ-9 scores from the completers⁴ in each wave to form three new groups: 1) Healthy Controls (HCs; n = 149), who scored < 38 on the PAI-BOR and did not meet likely MDD criteria on the PHQ-9; 2) MDD group (n = 54), who scored < 38 on the PAI-BOR and met likely MDD criteria on the PHQ-9; 3) BPD group (n = 69), who scored ≥ 38 on the PAI-BOR. Of note, 56.5% (n = 39) of the BPD group also met criteria for likely MDD. There were no significant differences in average age among the HC group (M = 35.83, SD = 9.85), the MDD group (M = 39.47, SD = 12.79), and the BPD group (M = 35.25, SD = 10.03), F(2, 269) = 2.92, p = .06. There was a smaller proportion of women in the HC group (49.0%) than in the MDD group (72.2%; χ²(1) = 8.58, p < .01, 95% CI [7.86, 36.07]) and the BPD group (73.6%; χ²(1) = 11.58, p < .01, 95% CI [10.67, 36.54]), χ²(2) = 15.96, p < .01, but there were no differences in the proportion of participants who identified as Caucasian in the HC group (74.5%), the MDD group (83.3%), or the BPD group (72.5%), χ²(2) = 2.22, p = .33.

While there were no significant differences in the proportion of the BPD (5.8%), MDD (9.3%), and HC (2.0%) groups who had ever participated in DBT, χ²(2) = 5.36, p = .07, there were significant between-group differences in the proportion of participants currently engaged in talk therapy, χ²(2) = 7.79, p = .02. A similar proportion of the BPD (20.3%) and MDD (18.5%) groups reported current talk therapy, χ²(1) = 0.60, p = .81, 95% CI [−12.81, 15.40], while a larger proportion of both the BPD, χ²(1) = 6.69, p = .01, 95% CI [2.69, 23.68], and MDD, χ²(1) = 4.47, p = .03, 95% CI [0.65, 23.24], groups reported current talk therapy than those in the HC group (8.1%). There was also a significant between-group difference in the proportion of participants utilizing psychotropic medications, χ²(2) = 30.49, p < .01. A similar proportion of the BPD (33.3%) and MDD (37.0%) groups reported current psychotropic medications, χ²(1) = 0.18, p = .67, 95% CI [−12.81, 20.36], while a larger proportion of both the BPD, χ²(1) = 22.26, p < .01, 95% CI [13.88, 37.50], and MDD, χ²(1) = 24.95, p < .01, 95% CI [16.14, 42.74], groups reported current psychotropic medications than those in the HC group (8.1%).

Analytic Method

We first examined differences in BPD features and depression symptoms among groups. We checked assumptions of ANOVA by examining the normality of residuals with the Shapiro-Wilk test; the homogeneity of variances with Levene’s test; and the correlations among the dependent variables. If these assumptions were satisfied, we conducted one-way ANOVAs^5,6; otherwise, we conducted one-way Kruskal-Wallis H tests using SPSS Version 25 (IBM Corp., 2017) to compare 1) adaptive emotion regulation strategy frequency, 2) maladaptive emotion regulation strategy frequency, and 3) quality of emotion regulation implementation among the HC, MDD, and BPD groups. We followed up these omnibus tests with post hoc t-tests using Fisher’s least significant differences method. Effect sizes were calculated using Cohen’s equations (Cohen, 1988) where .20 ≤ d ≤ .49 indicates a small effect size, .50 ≤ d ≤ .79 indicates a medium effect size, and d ≥ .80 indicates a large effect size. As an exploratory analysis, we re-ran ANOVAs with four groups: HC, MDD, BPD, and BPD+MDD.

Results – Study 1

First, we compared PAI-BOR and PHQ-9 total scores among the three groups (Table 1). There were significant between-group differences in PAI-BOR and PHQ-9 scores. Participants in the BPD group reported significantly higher PAI-BOR scores than those in the MDD group, M_diff = 16.91, SE_diff = 1.57, p < .01, 95% CI [13.81, 20.00], d = 2.56, and those in the HC group, M_diff = 29.57, SE_diff = 1.26, p < .01, 95% CI [27.08, 32.05], d = 3.27. Participants in the BPD group reported similar PHQ-9 scores as those in the MDD group, M_diff = −1.29, SE_diff = .74, p = .08, 95% CI [−2.74, .17], d = −.10, but higher PHQ-9 scores than those in the HC group, M_diff = 11.50, SE_diff = .59, p < .01, 95% CI [10.34, 12.67], d = 1.21. Participants in the MDD group reported significantly higher PAI-BOR scores, M_diff = 12.66, SE_diff = 1.38, p < .01, 95% CI [9.95, 15.37], d = 1.36, and PHQ-9 scores, M_diff = 12.79, SE_diff = .64, p < .01, 95% CI [11.52, 14.06], d = 3.71, than those in the HC group.

Table 1.

Means, standard deviations, and group comparison statistics of psychopathology, emotion regulation skill frequency, and emotion regulation quality measures.

Variable	HC M (SD)	MDD M (SD)	BPD M (SD)	F	df	p
Study 1	(n = 149)	(n = 54)	(n = 69)
PAI-BOR	16.84 (10.03)a	29.50 (6.89)b	46.41 (6.37)c	278.15	2, 269	< .01
PHQ-9	3.32 (3.60)a	16.11 (2.96)b	14.83 (5.49)b	299.60	2, 269	< .01
DSS	1.66 (0.58)a	1.55 (0.52)a,b	1.42 (0.58)b	4.40	2, 269	.01
DCS	1.07 (0.57)a	1.69 (0.40)b	1.89 (0.48)c	70.22	2, 269	< .01
WOR	3.72 (0.81)a	3.26 (0.67)b	2.97 (0.85)c	22.82	2, 269	< .01
Study 2	(n = 30)	(n = 30)	(n = 30)
PAI-BOR	14.10 (8.15)a	31.67 (7.94)b	45.33 (8.34)c	110.97	2, 87	< .01
PHQ-9	2.43 (3.68)a	15.87 (5.40)b	12.90 (4.78)c	68.36	2, 87	< .01
DSS	2.09 (0.43)a	1.67 (0.46)b	1.78 (0.52)b	6.19	2, 87	< .01
DCS	1.06 (0.55)a	1.84 (0.32)b	1.92 (0.51)b	30.79	2, 87	< .01
WOR	3.97 (0.82)a	3.16 (0.95)b	2.93 (0.98)b	10.55	2, 87	< .01

Note. HC = Healthy controls. MDD = Major Depressive Disorder. BPD = Borderline Personality Disorder. PAI-BOR = Personality Assessment Inventory – Borderline subscale. PHQ-9 = Patient Health Questionnaire – 9. DSS = DBT-WCCL DBT Skills Subscale. DCS = DBT-WCCL Dysfunctional Coping Subscale. WOR = Ways of Responding scale. Different subscripts (i.e., a, b, c) indicate statistically significant between-group differences in mean values of that measure, ps < .05.

The distribution of three sets of residuals were significantly non-normal: WOR Quality scores in the HC group, W = .97, df = 149, p < .01; DBT-WCCL-DSS scores in the HC group, W = .97, df = 149, p < .01; and DBT-WCCL-DCS scores in the BPD group, W = .95, df = 69, p = .01. All other residuals were normally distributed, Ws > .97, ps > .05. DBT-WCCL-DCS scores were also significantly heteroscedastic across groups, F(2, 269) = 4.62, p = .01. Because ANOVA is relatively robust to violations of assumptions (Schmider, Ziegler, Danay, Beyer, & Bühner, 2010), we only ran both a one-way ANOVA and a Kruskal-Wallis H test for DBT-WCCL-DCS scores. Because the correlations among DBT-WCCL subscale and WOR Quality scores were mostly nonsignificant and near zero (Table 2), we did not conduct a MANOVA.

Table 2.

Correlations among emotion regulation quality and frequency by group.

Variable Pair	HC r	MDD r	BPD r	Full sample r
Study 1	(n = 149)	(n = 54)	(n = 69)	(n = 272)
DSS-DCS	.32**	.16	.14	.10
DSS-WOR	.23**	.12	.06	.22**
DCS-WOR	−.04	−.01	−.45**	−.33**
Study 2	(n = 30)	(n = 30)	(n = 30)	(n = 90)
DSS-DCS	.20	−.28	.36	−.10
DSS-WOR	.31	.31	.42*	.44**
DCS-WOR	−.24	−.14	.08	−.34**

Note. HC = Healthy controls. MDD = Major Depressive Disorder. BPD = Borderline Personality Disorder. DSS = DBT-WCCL DBT Skills Subscale. DCS = DBT-WCCL Dysfunctional Coping Subscale. WOR = Ways of Responding scale.

^*p < .05

^**p < .01.

We then tested our main hypotheses (Table 1). There were significant between-group differences on all three measures of emotion regulation skills: DBT-WCCL-DSS, F(2, 269) = 4.40, p = .01; DBT-WCCL-DCS, F(2, 269) = 70.22, p < .01; and the WOR Quality scale, F(2, 269) = 22.82, p < .01^7,8. Of note, the Kruskal-Wallis H test for the DBT-WCCLS-DCS was also significant, H(2) = 99.30, p < .01. We probed each scale with post hoc tests in turn.

On the DBT-WCCL-DSS (Figure 1), the BPD group reported similar frequencies of adaptive strategy use as the MDD group, M_diff = −.13, SE_diff = .10, p = .20, 95% CI [−.34, .07], d = −.23, but significantly lower frequency of adaptive strategy use than the HC group, M_diff = −.24, SE_diff = .08, p < .01, 95% CI [−.41, −.08], d = −.41. On the DBT-WCCL-DCS, the BPD group reported a greater frequency of maladaptive strategy use than the MDD, M_diff = .19, SE_diff = .09, p = .04, 95% CI [.01, .38], d = .45, and HC groups, M_diff = .82, SE_diff = .08, p < .01, 95% CI [.67, .97], d = 1.51. Finally, on the WOR Quality scale, the BPD group provided responses that were rated as lower quality than the MDD, M_diff = −.29, SE_diff = .14, p = .04, 95% CI [−.58, −.01], d = −.37, and HC groups, M_diff = −.75, SE_diff = .12, p < .01, 95% CI [−.98, −.52], d = −.91.

Figure 1.

A) Study 1 group comparisons of frequency of adaptive strategy use. B) Study 1 group comparisons of frequency of maladaptive strategy use. C) Study 1 group comparisons of emotion regulation strategy quality. DBT-WCCL = Dialectical Behavior Therapy Ways of Coping Checklist. WOR = Ways of Responding scale. HC = Healthy Controls. MDD = Major Depressive Disorder. BPD = Borderline Personality Disorder. Lines represent group means and standard errors. d = Cohen’s d * p < .05 ** p < .01

Exploratory Analyses

When examining HC, MDD, BPD, and BPD+MDD groups, we again found significant between-group differences on all measures of emotion regulation skills (Table S1, Supplemental Materials). The BPD+MDD and BPD groups did not significantly differ on any measure of emotion regulation skill, ps > .05. Only the BPD+MDD group reported significantly greater frequency of maladaptive skill use than the MDD group, M_diff = .29, SE_diff = .11, p = .01, 95% CI [.08, .51], d = .74, and demonstrated lower quality emotion regulation skills than the MDD group, M_diff = −.39, SE_diff = .17, p = .02, 95% CI [−.72, −.06], d = −.52.

Discussion – Study 1

In an online sample of participants recruited into three groups (HC, MDD, and BPD), we found significant group differences in the frequency of adaptive and maladaptive emotion regulation strategy use and emotion regulation skill quality. Compared to those in the HC group, participants in the BPD group reported using adaptive strategies less frequently, maladaptive strategies more frequently, and demonstrated lower quality emotion regulation implementation. Compared to those in the MDD group, participants in the BPD group reported using adaptive strategies just as frequently, maladaptive strategies more frequently, and demonstrated lower quality emotion regulation implementation. These results suggest that the emotion regulation skills deficits that may uniquely characterize BPD, relative to MDD, are greater frequency of maladaptive strategy use and lower quality of emotion regulation implementation, while the frequency of adaptive strategy use may be more indicative of general psychopathology.

Although the threshold scores used in this study to determine group membership have been validated in previous clinical samples, these results may be limited in how well they generalize to a clinical population. To address this concern, we attempted to replicate these results in a second study in which we recruited participants from the community to complete a structured diagnostic interview to determine group membership.

Method & Materials – Study 2

Participants

An initial sample of 206 people completed a phone screening assessment to determine likely study eligibility between November 2016 and December 2017. Of these, 106 were deemed potentially eligible and completed an in-person structured diagnostic interview at a large Midwestern university. Of those 106 participants, 90 were determined to be eligible for study participation. These 90 participants were 32.14 (SD = 12.33) years old on average. All participants identified as female (100.0%) and the majority of participants identified as Caucasian (66.7%). A plurality of participants reported they had either some college experience (41.1%) or had completed a four-year college degree (31.1%), with a median household income of $40,000 - $49,999. A plurality of participants reported being single/never married (43.3%).

Measures

Demographics.

Participants self-reported their age, gender, ethnicity, marital status, income, education, and history of psychotherapy and psychotropic medication utilization.

DBT Ways of Coping Checklist (DBT-WCCL; Neacsiu et al., 2010).

Details regarding the DBT-WCCL are provided under Study 1. In Study 2, both DSS (α = .93) and DCS (α = .92) items exhibited excellent internal consistency.

Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001).

Details regarding the PHQ-9 are provided under Study 1. In Study 2, PHQ-9 items exhibited excellent internal consistency (α = .91).

Personality Assessment Inventory – Borderline Subscale (PAI-BOR; Morey, 1991).

Details regarding the PAI-BOR are provided under Study 1. In Study 2, PAI-BOR items exhibited excellent internal consistency (α = .93).

Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID-I; First, Spitzer, Miriam, & Williams, 2002).

This semi-structured diagnostic assessment is used to determine the presence or absence of DSM-IV-TR Axis I psychiatric disorders. Previous researchers have reported fair-to-excellent agreement (κs: .61 - .83) on diagnoses assigned using the SCID-I among a sample of participants in inpatient treatment, outpatient treatment, and healthy controls (Lobbestael, Leurgans, & Arntz, 2011). In this study, the first author, who received extensive training over four years by the second author, conducted all SCID-I assessments.

Structured Clinical Interview for DSM-IV-TR Axis II Disorders (SCID-II; First, Spitzer, Gibbon, Williams, & Benjamin, 1997).

This semi-structured diagnostic assessment is used to determine the presence or absence of DSM-IV-TR Axis II psychiatric disorders. Previous researchers have reported excellent agreement (κs: .77 - .94) on diagnoses assigned using the SCID-II among the same sample of participants in inpatient treatment, outpatient treatment, and healthy controls (Lobbestael, Leurgans, & Arntz, 2011). The first author also conducted all SCID-II assessments.

Ways of Responding Scale (WOR; Barber & DeRubeis, 1992).

Details regarding the WOR are provided under Study 1. Two trained female undergraduate coders, blind to participant information, independently rated the overall quality of each response. In Study 2, coders demonstrated excellent agreement regarding overall quality (ICC = .94) when averaged across all six responses for each participant⁹.

Study Flow and Procedures

Participants were recruited via referrals from the local university DBT training clinic, community flyers, and online advertisements. Potential participants completed a phone screening assessment to determine likely eligibility. Potential participants were excluded if they met full DSM-IV-TR (APA, 2000) diagnostic criteria for bipolar I disorder, any psychotic disorder, or a primary substance use disorder. Participants were recruited such that there were no statistically significant group differences in gender, age, or education level, as these demographics may be differentially related to patterns of emotion regulation (John & Gross, 2004; Nolen-Hoeksema & Aldao, 2011) or complexity of written responses. Participants who did not identify as female were excluded because the majority of people with BPD who present for clinical services are female (Widiger & Trull, 1993). Participants were also excluded if they were younger than 18 years old at the beginning of the study or if they could not read English.

Potential participants then attended an in-person assessment. Participants first provided informed consent and then completed the SCID-I and SCID-II. Group assignment was determined by diagnostic status. Participants who met no DSM-IV-TR criteria for any current Axis I or Axis II disorder were assigned to the HC group. Participants who met DSM-IV-TR criteria for current MDD and who met no more than three criteria for BPD were assigned to the MDD group. Participants who met DSM-IV-TR criteria for BPD were assigned to the BPD group. Participants were blind to group assignment. Following the diagnostic assessments, participants completed all remaining measures online and were thanked, debriefed, and compensated at a rate of $15 per hour.

Analytic Method

We first compared demographic and diagnostic information among groups. We then checked assumptions of ANOVA by examining the normality of residuals with the Shapiro-Wilk test; the homogeneity of variances with Levene’s test; and correlations among dependent variables. We conducted one-way ANOVAs (and one-way Kruskal-Wallis H tests if ANOVA assumptions were not met) using SPSS Version 25 (IBM Corp., 2017) to compare 1) adaptive emotion regulation strategy frequency, 2) maladaptive emotion regulation strategy frequency, and 3) emotion regulation quality among the HC, MDD, and BPD groups. We followed up significant omnibus tests with post hoc t-tests using Fisher’s least significant differences method. As an exploratory analysis, we re-ran ANOVAs with four groups: HC, MDD, BPD, and BPD+MDD.

Results – Study 2

We first compared demographic and diagnostic information among groups. Groups did not differ by average age, F(2, 87) = .01, p = .99 or proportion of Caucasian participants, χ²(2) = 1.20, p = .55. Groups did differ by the number of DSM-IV-TR diagnoses, F(2, 87) = 92.90, p < .01. The BPD group (M = 6.03, SD = 2.19) met criteria for significantly more diagnoses than the MDD group (M = 3.87, SD = 2.06), M_diff = 2.17, SE_diff = .45, p < .01, 95% CI [1.28, 3.06], d = 1.02, and HCs (M = 0.00, SD = 0.00), M_diff = 6.03, SE_diff = .45, p < .01, 95% CI [5.14, 6.92], d = 3.89 (Table 3). More participants in the BPD group had ever participated in DBT (46.7%) than the MDD group (20%), χ²(1) = 4.72, p = .03, 95% CI [2.80, 46.81], or the HC group (0%). Similar proportions of the BPD (66.7%) and MDD groups (50.0%) reported current psychotropic medication use, χ²(1) = 1.69, p = .19, 95% CI [−7.90, 38.64], and similar proportions of the BPD (33.3%) and MDD groups (43.3%) reported currently engaging in talk therapy, χ²(1) = 0.62, p = .43, 95% CI [−13.97, 32.45]. More participants in the BPD group (66.7%) reported current psychotropic medication use than HCs (16.7%), χ²(1) = 15.17, p < .01, 95% CI [25.39, 66.91], while a similar proportion of participants in the BPD group (33.3%) reported current psychotherapy utilization as HCs (13.3%), χ²(1) = 3.30, p = .07, 95% CI [−1.59, 39.60].

Table 3.

Study 2 sample diagnostics.

Diagnosis	HC (n = 30) % (n)	MDD (n = 30) % (n)	BPD (n = 30) % (n)	χ2	df	p
Bipolar I Disorder	0 (0)a	0 (0)a	0 (0)a	–	–	–
Bipolar II Disorder	0 (0)a	0 (0)a	0 (0)a	–	–	–
Major Depressive Disorder	0 (0)a	100 (30)b	40.0 (12)c	61.07	2	< .01
Dysthymia	0 (0)a	36.7 (11)b	53.3 (16)b	21.27	2	< .01
Psychotic Disorder	0 (0)a	0 (0)a	0 (0)a	–	–	–
Substance Use Disorder	0 (0)a	0 (0)a	6.7 (2)a	4.09	2	.13
Substance Dependence	0 (0)a	3.3 (1)a	13.3 (4)a	5.51	2	.06
Panic Disorder	0 (0)a	10.0 (3)b	36.7 (11)c	16.41	2	< .01
Agoraphobia	0 (0)a	16.7 (5)a,b	23.3 (7)b	7.50	2	.02
Social Anxiety Disorder	0 (0)a	26.7 (8)b	43.3 (13)b	16.03	2	< .01
Specific Phobia	0 (0)a	10.0 (3)a,b	26.7 (8)b	10.59	2	.01
Obsessive-Compulsive Disorder	0 (0)a	10.0 (3)a	13.3 (4)a	4.03	2	.13
Posttraumatic Stress Disorder	0 (0)a	23.3 (7)b	26.7 (8)b	9.12	2	.01
Generalized Anxiety Disorder	0 (0)a	66.7 (20)b	56.7 (17)b	32.04	2	< .01
Anorexia Nervosa	0 (0)a	0 (0)a	0 (0)a	–	–	–
Bulimia Nervosa	0 (0)a	0 (0)a	0 (0)a	–	–	–
Binge Eating Disorder	0 (0)a	10.0 (3)a	0 (0)a	6.21	2	.05
Avoidant PD	0 (0)a	26.7 (8)b	33.3 (10)b	11.67	2	< .01
Dependent PD	0 (0)a	0 (0)a	3.3 (1)a	2.02	2	.36
Obsessive-Compulsive PD	0 (0)a	20.0 (6)b	33.3 (10)b	11.55	2	< .01
Paranoid PD	0 (0)a	10.0 (3)a	46.7 (14)b	23.64	2	< .01
Schizotypal PD	0 (0)a	0 (0)a	10.0 (3)a	6.21	2	.05
Schizoid PD	0 (0)a	6.7 (2)a	0 (0)a	4.09	2	.13
Histrionic PD	0 (0)a	0 (0)a	13.3 (4)a	8.37	2	.02
Narcissistic PD	0 (0)a	0 (0)a	6.7 (2)a	4.09	2	.13
Antisocial PD	0 (0)a	6.7 (2)a	16.7 (5)a	5.89	2	.05
PD - Not Otherwise Specified	0 (0)a	0 (0)a	0 (0)a	–	–	–

Note. HC = Healthy Controls. MDD = Major Depressive Disorder. BPD = Borderline Personality Disorder. PD = Personality Disorder. Different subscripts indicate statistically significant group differences, ps < .05.

As in Study 1, there were significant between-group differences in both PAI-BOR and PHQ-9 scores (Table 1). The BPD group reported significantly higher PAI-BOR scores than the MDD group, M_diff = 13.67, SE_diff = 2.10, p < .01, 95% CI [9.49, 17.84], d = 1.68, and HCs, Mdiff = 31.23, SE_diff = 2.10, p < .01, 95% CI [27.06, 35.41], d = 3.79. The BPD group reported significantly lower PHQ-9 scores than the MDD group, M_diff = −2.97, SE_diff = 1.21, p = .02, 95% CI [−5.37, −0.57], d = −.58, but significantly higher PHQ-9 scores than HCs, M_diff = 10.47, SE_diff = 1.21, p < .01, 95% CI [8.07, 12.87], d = 2.45. The MDD group reported significantly higher PAI-BOR scores, M_diff = 17.57, SE_diff = 2.10, p < .01, 95% CI [13.39, 21.74], d = 2.18, and PHQ-9 scores, M_diff = 13.43, SE_diff = 1.21, p < .01, 95% CI [11.03, 15.83], d = 2.91, than HCs.

All sets of residuals were relatively normally distributed, Ws > .94, ps > .12. As in Study 1, DBT-WCCL-DCS scores demonstrated significant heteroscedasticity, F(2, 87) = 3.43, p = .02.

We then tested our main hypotheses. There were significant between-group differences on all three measures of emotion regulation skills: DBT-WCCL-DSS, F(2, 87) = 6.19, p < .01; DBT-WCCL-DCS, F(2, 87) = 30.79, p < .01; and the WOR Quality scale, F(2, 87) = 10.55, p < .01. The Kruskal-Wallis H test for the DBT-WCCLS-DCS was also significant, H(2) = 36.75, p < .01. We probed each scale with post hoc tests in turn.

On the DBT-WCCL-DSS (Figure 2), the BPD group reported a similar frequency of adaptive strategy use as the MDD group, M_diff = .11, SE_diff = .12, p = .37, 95% CI [−.13, .35], d = .22, but a significantly lower frequency of adaptive strategy use than HCs, M_diff = −.30, SE_diff = .12, p = .02, 95% CI [−.55, −.06], d = −.65. On the DBT-WCCL-DCS, the BPD group reported a similar frequency of maladaptive strategy use as the MDD group, M_diff = .08, SE_diff = .12, p = .53, 95% CI [−.17, .32], d = .19, but a significantly greater frequency than HCs, M_diff = .87, SE_diff = .12, p < .01, 95% CI [.62, 1.11], d = 1.62. Finally, on the WOR Quality scale, the BPD group described strategies similar in quality to the MDD group, M_diff = −.24, SE_diff = .24, p = .24, 95% CI [−.71, .24], d = −.24, but significantly lower in quality than HCs, M_diff = −1.04, SE_diff = .24, p < .01, 95% CI [−1.51, −.57], d = −1.15.

Figure 2.

A) Study 2 group comparisons of frequency of adaptive strategy use. B) Study 2 group comparisons of frequency of maladaptive strategy use. C) Study 2 group comparisons of emotion regulation strategy quality. DBT-WCCL = Dialectical Behavior Therapy Ways of Coping Checklist. WOR = Ways of Responding scale. HC = Healthy Controls. MDD = Major Depressive Disorder. BPD = Borderline Personality Disorder. Lines represent group means and standard errors. d = Cohen’s d * p < .05 ** p < .01

Exploratory Analyses

When examining HC, MDD, BPD, and BPD+MDD groups, we again found significant between-group differences on all measures of emotion regulation skills (Table S1, Supplemental Materials). The BPD+MDD and BPD groups did not significantly differ from each other or from the MDD group on any measure of emotion regulation skill, ps > .15.

Discussion – Study 2

In a sample of participants assigned to three groups (HC, MDD, and BPD) based on diagnostic status, we found significant group differences in the frequency of adaptive and maladaptive emotion regulation strategy use and emotion regulation skill quality. Compared to those in the HC group, participants in the BPD group reported using adaptive strategies less frequently, maladaptive strategies more frequently, and demonstrated lower quality emotion regulation implementation. Compared to those in the MDD group, however, participants in the BPD group reported using both adaptive and maladaptive strategies just as frequently and described emotion regulation implementation of similar quality. Unlike the findings in Study 1, these results suggest that the emotion regulation skills deficits associated with BPD may be more indicative of general psychopathology than BPD in particular.

A notable strength of Study 2 is the use of validated in-person diagnostic assessments. However, relying on in-person participation limits the pool of available participants and the study sample size. Studies with smaller sample sizes typically have less statistical power to detect smaller effects, which might be expected when comparing the two clinical groups. To address the limitations and capitalize on the strengths of both Study 1 and Study 2, we conducted a mini meta-analysis of our results. Finally, to better specify the emotion regulation differences among groups, we identified the most and least frequently used adaptive and maladaptive strategies by each group using the full sample.

Methods & Materials – Mini Meta-Analysis

Analytic Method

We first conducted independent samples t-tests to compare PAI-BOR and PHQ-9 scores between studies. We then entered all information into Goh, Hall, and Rosenthal’s (2016) mini meta-analysis spreadsheet (version 2). We entered group means and standard deviations for each study from the DBT-WCCL-DSS, DBT-WCCL-DCS, and the WOR. This allowed us to calculate Cohen’s d for each comparison involving a BPD group. We then entered these ds and sample size information from each study to calculate meta-analytic ds, weighted by study sample size, as well as standard errors and 95% confidence intervals.

Results – Mini Meta-Analysis

There were no significant between-study differences on PAI-BOR or PHQ-9 scores among the BPD groups, ps > .09, the MDD groups, ps > .19, or the HC groups, ps > .11. On the DBT-WCCL-DSS, there were no significant differences between the BPD groups and the MDD groups, $\overline{d}$ = −.09, SE = .15, p = .56, 95% CI [−.38, .21], although the BPD groups reported using adaptive strategies significantly less frequently than the HC groups, $\overline{d}$ = −.47, SE = .13, p < .01, 95% CI [−.72, −.22]. On the DBT-WCCL-DCS, the BPD groups reported using maladaptive strategies significantly more frequently than the MDD, $\overline{d}$ = .35, SE = .15, p = .02, 95% CI [.05, .64], and HC groups, $\overline{d}$ = 1.54, SE = .14, p < .01, 95% CI [1.26, 1.82]. Finally, on the WOR, the BPD groups described strategies lower in quality than both the MDD, $\overline{d}$ = −.33, SE = .15, p = .03, 95% CI [−.63, −.04], and HC groups, $\overline{d}$ = −.97, SE = .13, p < .01, 95% CI [−1.23, −.71].

Specific DBT-WCCL Descriptives

The most frequently used adaptive strategy in both the BPD and MDD groups was occupying one’s mind with something else (M_BPD = 1.94, SD_BPD = .91; M_MDD = 2.26, SD_MDD = .79); in the HC group, it was focusing on the good things in life (M = 2.29, SD = .84). The least frequently used adaptive strategy in all groups was surrounding oneself with a nice fragrance to soothe oneself (M_BPD = 1.02, SD_BPD = 1.01; M_MDD = .70, SD_MDD = .89; M_HC = .93, SD_HC = 1.01).

The most frequently used maladaptive response in the BPD group was self-blame (M = 2.34, SD = .80); in the MDD and HC groups, it was keeping feelings to oneself (M_MDD = 2.44, SD_MDD = .70; M_HC = 1.67, SD_HC = .92). The least frequently used maladaptive response in the BPD and HC groups was refusing to believe something had happened (M_BPD = 1.08, SD_BPD = 1.00; M_HC = .52, SD_HC = .71); in the MDD group, it was finding out who was responsible (M = .69, SD = .73).

Exploratory Analyses

When examining HC, MDD, BPD, and BPD+MDD groups, we found significant differences in the expected direction between each clinical group and HCs (|d|s > .35, ps < .01; Table S2, Supplemental Materials). Among clinical groups, the only significant differences were between MDD and BPD+MDD groups, with those in the BPD+MDD group reporting more frequent use of maladaptive strategies, d = .64, SE = .18, p < .01, 95% CI [.28, 1.00] and lower quality strategies, d = –.38, SE = .18, p = .03, 95% CI [–.74, –.03] than those in the MDD group.

Discussion

Across two studies, we examined whether BPD was characterized by unique deficits in emotion regulation strategy use. We found meta-analytic evidence that BPD may be uniquely characterized by the overuse of maladaptive emotion regulation strategies and lower quality implementation of emotion regulation strategies, relative to those classified as MDD or HCs. The frequency with which those in the BPD group used adaptive emotion regulation strategies was not significantly different from those in the MDD group but was lower than those in the HC group, indicating a deficit that may be characteristic of psychopathology more generally.

These results provide further empirical support for and help specify Linehan’s (1993) biosocial theory of BPD. “The thesis [of the biosocial theory] is that borderline individuals are… deficient in emotion modulation skills. Failures to inhibit maladaptive, mood-dependent actions are by definition part of the borderline syndrome” (Linehan, 1993, p. 43). These results provide support for the claim that those with BPD engage more frequently in maladaptive strategy use than people with mood disorders or no disorder. These results also specify that the deficiency in emotion modulation skills in BPD may be more accurately characterized as a unique deficiency in the quality of implementation of skills (i.e., how well one uses skills) rather than the frequency with which those skills are used. People with BPD may report a deficit in adaptive emotion regulation skill frequency relative to HC participants but report a similar frequency of use relative to those with MDD.

The greater statistical power of these studies also clarifies previous empirical results. In line with Sauer et al. (2014) and Daros et al. (2018), our results suggest that those with BPD use adaptive emotion regulation strategies as frequently as those with mood or anxiety disorders. It is possible that people with BPD only exhibit a unique deficit in the frequency of adaptive strategy use relative to those with other PDs (Neacsiu & Tkachuck, 2016). Given the relative dearth of research on personality disorders other than BPD, future researchers are encouraged to recruit other PD samples to enhance our understanding of baseline emotion regulation deficits among people with other PDs. Our results also support those of Daros et al. (2018) regarding the unique deficit characterized by an overuse of maladaptive emotion regulation strategies by those with BPD relative to those with mood or anxiety disorders. Given the small-to-medium effect size found in the current studies, it is unsurprising that this difference may not be consistent across smaller samples (cf. Sauer et al., 2014). Finally, our results extend and complement previous studies by providing evidence that people with BPD exhibit unique deficits in the quality of their emotion regulation implementation.

Taken together, these results suggest that people with BPD may be using adaptive strategies as frequently as those with MDD but doing so less skillfully. In line with developmental aspects of the biosocial theory, chronic invalidation of those with greater emotional sensitivity (i.e., those most likely to develop BPD) may impair learning about how to effectively regulate one’s emotions (Fruzzetti, Shenk, & Hoffman, 2005; Mazursky & Schul, 2000). The differences in emotion regulation quality and use between those with BPD and MDD further suggests that those with BPD may be experiencing a wider variety of longer-lasting negative emotions than people with MDD that may make it more difficult to adaptively tolerate the accompanying distress. Such difficulties tolerating distress may lead those with BPD to rely more habitually on strategies that reduce immediate distress, even at the cost of longer-term goals. In contrast, people with MDD may use fewer maladaptive and higher quality strategies because they tend to experience a smaller variety of negative emotions (i.e., predominantly sadness). Future researchers are encouraged to test these hypotheses.

The results of the current studies should be considered in light of their limitations. Although previously validated in several clinical samples, our measure of adaptive and maladaptive emotion regulation frequency relied on self-reported responses regarding the previous month, which may be subject to recall biases (cf. Stone et al., 1998; Todd, Tennen, Carney, Armeli, & Affleck, 2004). Second, our measure of emotion regulation quality relied on written responses to hypothetical stressful scenarios. While this method allows for direct between-subject comparisons, we cannot say how representative these scenarios are of situations in which participants would regulate their emotions or the degree to which participants’ written responses reflects their actual emotion regulation behaviors. Thus, it is unclear whether the skills deficits described here reflect participants’ behaviors or their appraisals of their behaviors. Further, because coders rated the overall quality of participants’ responses, it is unclear what specific aspects, if any, of participants’ emotion regulation strategies were of higher or lower quality. Future researchers may conduct standardized behavioral assessments of participant skill use to compare both observer and participant ratings of the quality of participants’ emotion regulation behaviors. Third, because participants in Study 1 completed all study procedures online, we cannot verify the veracity of their responses. We attempted to mitigate these concerns by a) excluding potential participants who completed pre-screening questionnaires multiple times with different answers and b) including messages emphasizing the need for high quality data (Zhou & Fishbach, 2016). Fourth, the first author completed all screenings and diagnostic assessments in Study 2, which may have influenced the reliability and validity of the diagnostic assessments. Finally, by recruiting participants with MDD as a clinical control group, our findings do not necessarily generalize beyond mood disorders. In Study 2, participants in the MDD group met criteria for nearly three comorbid diagnoses on average, suggesting reasonable variability in diagnostic presentation. Future researchers may recruit participants with disorders other than MDD or use transdiagnostic criteria to recruit a more varied comparison group.

Despite these limitations, this set of studies embodies principles of reproducible clinical psychological science advocated for by Tackett et al. (2017). It capitalizes on the strengths of a larger sample size assessed on dimensional measures (Study 1) along with a verified diagnostic sample (Study 2). Further, these studies extend previous theoretical work on the biosocial theory of BPD and empirical work on skills deficits by comparing not only the frequency of adaptive and maladaptive strategy use but also the quality with which these strategies are used. Our results provide initial evidence that the quality of emotion regulation implementation among people with BPD may be as much of a unique deficit as the frequency of maladaptive emotion regulation use. By specifying these skill deficits, our results offer clear targets for developmental researchers to understand how such skill deficits arise. These results also offer clinical assessors more detailed and normative baseline information to consider when evaluating skills deficits among patients with BPD and MDD as well as validated measures for such assessments.