Differential Role of CBT Skills, DBT Skills and Psychological Flexibility in Predicting Depressive versus Anxiety Symptom Improvement

Abstract

Objective

Studies have reported associations between cognitive behavioral therapy (CBT) skill use and symptom improvement in depressed outpatient samples. However, little is known regarding the temporal relationship between different subsets of therapeutic skills and symptom change among relatively severely depressed patients receiving treatment in psychiatric hospital settings.

Method

Adult patients with major depression (N=173) receiving combined psychotherapeutic and pharmacological treatment at a psychiatric hospital completed repeated assessments of traditional CBT skills, DBT skills and psychological flexibility, as well as depressive and anxiety symptoms.

Results

Results indicated that only use of behavioral activation (BA) strategies significantly predicted depressive symptom improvement in this sample; whereas DBT skills and psychological flexibility predicted anxiety symptom change. In addition, a baseline symptom severity X BA strategies interaction emerged indicating that those patients with higher pretreatment depression severity exhibited the strongest association between use of BA strategies and depressive symptom improvement.

Conclusions

Findings suggest the importance of emphasizing the acquisition and regular use of BA strategies with severely depressed patients in short-term psychiatric settings. In contrast, an emphasis on the development of DBT skills and the cultivation of psychological flexibility may prove beneficial for the amelioration of anxiety symptoms.

The efficacy of cognitive behavioral therapy (CBT) for the treatment of depression has been evaluated in numerous clinical trials (DeRubeis, Webb, Tang, & Beck, 2010; Epp & Dobson, 2010). Despite the large body of evidence supporting the overall efficacy of CBT in alleviating depressive symptoms (Cuijpers et al., 2013a), the mechanisms that account for symptom improvement remain poorly understood. One hypothesis is that the acquisition and regular use of core cognitive and behavioral skills represents a central mechanism through which patients improve.

Several relatively brief patient-report measures of CBT skills have recently been developed, including the Skills of Cognitive Therapy scale (SoCT; Jarrett et al., 2011), the Cognitive Behavioral Therapy Skills Questionnaire (CBTSQ; Jacob et al., 2011) and the Competencies of Cognitive Therapy Scale (CCTS; Strunk et al., 2014). Studies using these measures have reported that greater depressive symptom improvement is associated with greater acquisition and use of CBT skills, as assessed by the SoCT (Jarrett et al., 2011; 2013), CBTSQ (Jacob et al., 2011; Webb et al., 2013) and CCTS (Strunk et al., 2014). However, causal inferences regarding the role of CBT skills in contributing to depressive symptom improvement are limited given that most of the associations reported within these studies are based on one or two concurrent assessments (e.g., only pre- and post-treatment) of CBT skills and depressive symptoms. Given the cross-sectional designs used within most studies to date, a significant association between CBT skills and symptom improvement could be due to skill use causing symptom change or vice-versa (or be the result of an unmeasured third variable confound).

In addition to the above common temporal confounds in the CBT skills literature, the bulk of studies investigating the association between skills and symptom change are conducted within the context of outpatient settings or in carefully controlled clinical trials (for an exception, see Jacob et al., 2011). We know surprisingly little about the extent to which different subsets of CBT skills predict symptom improvement among more severely depressed patients receiving treatment in psychiatric hospital settings. Data from these treatment contexts are critical to informing our understanding of the mechanisms that account for symptom improvement in naturalistic settings and to complement clinical trial data. In terms of clinical implications, findings regarding which CBT skills predict symptom improvement may ultimately inform which treatment elements and interventions are emphasized by CBT therapists in different settings. Null findings may also be theoretically and clinically informative. Namely, if patient acquisition and use of core cognitive or behavioral skills fails to predict depressive symptom improvement, then it may suggest that some of these skills are either therapeutically inert, or are moderated by important patient characteristics (e.g., pretreatment depression severity [Webb et al., 2012], comorbid personality disorders [Keefe et al., 2016]) or treatment setting (e.g., short-term, intensive inpatient or partial hospital treatment vs. longer-term outpatient treatment involving weekly therapy sessions).

The goal of the present study was to expand on prior research and examine the role of cognitive and behavioral skills in predicting depressive symptom improvement within the context of a naturalistic psychiatric setting treating severely depressed patients. As noted above, previous studies commonly rely on one or two (e.g., pre- and post-treatment) assessment timepoints to represent the state of patient CBT skills throughout treatment. To achieve a more fine-grained and statistically powerful test of skill-outcome associations, we included repeated assessments of CBT skills and depressive symptoms over the course of treatment. To our knowledge, the present study is the first to test whether patient-reported CBT skill use, assessed repeatedly throughout the course of therapy, predicts subsequent depressive symptom improvement. It should be noted that Jarrett et al. (2011; 2013) reported that their SoCT measure, assessed at one mid-treatment timepoint, prospectively predicted post-treatment depression response (controlling for pre-treatment depression scores). However, the authors did not control for (1) concurrent symptoms (i.e., assessed at the time at which the SoCT was measured) or (2) prior symptom change (i.e., symptom improvement prior to the SoCT assessment), both of which represent plausible confounds. Indeed, others have noted how measures of self-reported CBT skills may inadvertently measure and be confounded with concurrent symptoms (Strunk et al., 2014). Similarly, the abovementioned studies examining the association between self-reported CBT skills and symptom change did not control for prior depressive symptom improvement. Prior symptom change may represent a third variable confound in so far as it predicts both subsequent symptom change and CBT skill use. For example, a patient in CBT treatment who has experienced significant depressive symptom improvement may be more likely to endorse using cognitive and/or behavioral skills (whether or not they in fact acquired and are using these skills). These two plausible symptom confounds are included as covariates in the analyses reported below.

Assessing both Traditional and “Newer Generation” Cognitive Behavioral Skills

Recent developments in transdiagnostic treatment have supported the integration of newer generation cognitive behavioral strategies for managing common depressive and anxiety symptoms. For example, there is evidence that acceptance and commitment therapy (ACT) is efficacious for both depressive and anxiety symptoms (Ost, 2014; Swain, Hancock, Hainsworth, & Bowman, 2013). Further, there is growing evidence of the therapeutic benefit of dialectical behavior therapy (DBT) skills in alleviating depression and anxiety in mixed samples of depressed and anxious adults (Neacsiu, Eberle, Kramer, Weismann, & Linhean, 2014; Ritschel, Cheavens, & Nelson, 2012). As such, modern cognitive behavioral treatment packages for affective disorders have moved to incorporate both traditional CBT interventions (i.e., cognitive restructuring [CR] and behavioral activation [BA] techniques) as well as those from ACT and DBT that emphasize mindfulness- and acceptance-based strategies and decreasing experiential avoidance (e.g., Unified Protocol for Emotional Disorders; Barlow et al., 2010). The expansion of traditional CBT protocols and increased utilization of a variety of interventions in the treatment of depression in real-world psychiatric settings raises the question: What elements of treatment are most effective for different subsets of symptoms most commonly experienced by depressed patients (in particular, depressive versus anxiety symptoms)? Such knowledge may ultimately help inform treatment planning and assist providers in selecting the most effective and efficient interventions for different symptom domains.

The current study examined associations between different subsets of cognitive behavioral skills (both traditional and newer generation) emphasized within the context of a multi-faceted intervention package, and depression and anxiety outcomes. Specifically, in addition to assessments of CBT (BA and CR) skills, the study included repeated assessments of DBT skills and ACT-based psychological flexibility in order to investigate differential associations with symptom change in a sample of depressed patients receiving treatment in a naturalistic psychiatric setting incorporating CBT-, DBT- and ACT-based group and individual therapy (see Participants and Treatment Setting below).

Assessing both Depressive and Anxiety Symptom Improvement as Outcomes

Given the exceptionally high rates of co-occurrence between depression and anxiety (Kessler et al., 2003), we were interested in examining the specificity of CBT skills, DBT skills, and psychological flexibility in predicting depressive versus anxiety symptom improvement. Of particular interest is the relative contributions of CBT skills compared to the development of psychological flexibility, given growing evidence to support the role of experiential avoidance in anxiety and anxiety disorders. For example, a recent meta-analysis across 63 studies found a significant association between experiential avoidance and anxiety (Bluett et al., 2014). Several reviews and meta-analyses have concluded that ACT has comparable outcomes to CBT for anxiety (A-Tjak, Davis, Morina, Powers, Smits & Emmelkamp, 2015; Ruiz, 2012; Swain, Hancock, Hainsworth, & Bowman, 2013). Further, there is evidence that ACT is associated with greater decreases in experiential avoidance relative to cognitive therapy (Lappalainen, Lehtonen, Skarp, Taubert, Ojanen, & Hayes, 2007) and such decreases mediate anxiety and depression outcomes (Forman et al., 2007). Researchers have speculated that the development of greater psychological flexibility may account for the therapeutic benefits of ACT on anxiety symptoms (For a review, see Bluett et al., 2014; Forman et al., 2007). As previously noted, there is also emerging evidence that DBT skills training is effective for decreasing anxiety in adults with affective disorders (Neacsiu, Eberle, Kramer, Weismann, & Linhean, 2014; Ritschel, Cheavens, & Nelson, 2012). Thus, an additional exploratory aim of the study is to examine differential effects of CBT (BA and CR) skills, DBT skills and increased psychological flexibility on anxiety symptom improvement in our depressed sample.

Hypotheses

The theoretical literature on CBT (Beck et al., 1979; Coffman et al., 2007) and empirical findings (Dimidjian et al., 2006) highlight the benefit of BA strategies for more severely depressed patients. Moreover, BA may be particularly beneficial in briefer treatment contexts, in comparison to relatively more complex CR skills, which may require more time to acquire and learn to competently utilize. Thus, given the depression severity of the present hospitalized sample (see Participants and Treatment Setting below), coupled with the short-term nature of the treatment setting (average length of treatment = 11.7 days), we hypothesized that only BA strategies would predict subsequent depressive symptom improvement (Hypothesis 1). We further hypothesized that a pretreatment depression severity by BA - but not CR - strategies interaction would emerge, such that those patients with higher intake depression severity would exhibit the strongest associations between use of BA strategies and subsequent depressive symptom improvement (Hypothesis 2).

Exploratory Hypothesis

Finally, given the high comorbidity between depression and anxiety and to test the specificity of our findings to depression, we also examined whether BA, CR, DBT skills, or the development of psychological flexibility predicted subsequent anxiety symptom change. Given that depression was the primary treatment target, analyses regarding anxiety symptoms were exploratory. In light of growing data on the efficacy of ACT for anxiety symptoms (Öst, 2014; Swain et al., 2013), in particular the therapeutic benefits of the development of psychological flexibility for anxiety (Bluett et al., 2014), we expected that increased psychological flexibility may be associated with improvement in anxiety symptoms.

Method

Participants and Treatment Setting

Participants were patients receiving treatment within the Behavioral Health Partial Hospital Program (BHP) at McLean Hospital (Belmont, MA), a Harvard Medical School teaching hospital. To be included in the present study, patients had to be admitted to the BHP, complete the assessment battery described below, and meet criteria for a current diagnosis of Major Depressive Disorder (MDD). Patients with Bipolar Disorder (i.e., current or past Manic/Hypomanic episode), or a current or past Psychotic Disorder were excluded. A total of 173 patients (ages 18–72 years, M = 35.47, SD = 13.61; 56% females) met these criteria during the study period (July, 2013 to July, 2014) and provided written informed consent for their clinical data to be used for research studies. Participants classified themselves as 88% White, 5% African-American, 5% Asian, 3% Latino, 3% Other (total percentage exceeds 100% as some participants identified with multiple racial/ethnic categories). With regards to current marital status, 30% were married, 57% were never married, 11% were separated or divorced, and 2% were living with a partner. With regards to employment, 29% of the sample reported being currently unemployed (not due to being a current student).

Previous episodes of depression were very common in our sample, with a mean of 5.90 (SD = 8.52) reported previous episodes. Mean age of first major depressive episode was 18.1 years old (median = 16 years; range 5–50 years). The pretreatment mean CES-D-10 depression score for the sample (M = 20.30) was double the suggested clinical cutoff of 10 (Andresen et al., 1994). Moreover, diagnostic comorbidity was also common, in particular for anxiety disorders (61.3% of the sample met criteria for a current anxiety disorder). Current diagnoses at the time of intake were as follows: 27.2% (n=47) of our sample met criteria for concurrent Generalized Anxiety Disorder, 28.3% (n=49) for Social Anxiety Disorder, 12.1% (n=21) for Post-Traumatic Stress Disorder, 22.5% (n=39) for Panic Disorder, and 11.6% (n=20) for Obsessive Compulsive Disorder. In addition, 13.3% (n=23) of the sample met criteria for Alcohol Abuse or Dependence. Twenty-nine percent (n=51) of the sample scored above the cut-off (total score ≥7) suggesting a BPD diagnosis on the McLean Screening Instrument for Borderline Personality Disorder (MSI-BPD; Zanarini et al. 2003). Nearly half (47%) of the sample had previously been hospitalized (inpatient) at least once for their psychiatric problems (30% received inpatient psychiatric treatment in the week prior to admission to the BHP). Twenty-nine percent (51/173) received prior psychiatric treatment within either a partial hospital program or intensive outpatient program. With regards to data on pharmacological treatment, we began tracking medications prescribed to patients after data collection for the current study began. However, we report descriptive information from our medication database to provide some detail about the typical pharmacological treatment received in the BHP. Upon admission, all MDD patients were prescribed at least one antidepressant (77% SSRI, 4% Tricyclic, 4% Tetracyclic, 15% other). MDD patients were also prescribed the following types of medications upon admission: 17% benzodiazepine, 3% non-benzo anti-anxiety medication, 8% atypical antipsychotic, 7% anti-epileptic mood stabilizer, 1% other mood stabilizer, 6% stimulant/ADHD medication.

The BHP delivers CBT and related behavioral approaches (both in group and individual formats) and pharmacological treatment to patients suffering from a wide range of psychiatric disorders (principally mood and anxiety disorders). Individual treatment plans were constructed for each patient by clinical team managers who conducted intake assessments, developed an initial conceptualization, and oversaw all aspects of treatment. Treatment consisted primarily of CBT-based groups provided by psychologists, social workers, occupational therapists, postdoctoral and graduate level psychology trainees, and mental health counselors. Patients attended up to five 50-minute groups each day, five days per week (Monday-Friday). Of these, one group per day focused on behavioral activation (BA) strategies, based on a protocol adapted from Martell, Dimidjian, and Herman-Dunn (2010). A second group focused on identifying and challenging negative automatic thoughts related to depression and was guided by a protocol adapted from Beck, Rush, Shaw, and Emery (1979). Although depression is the primary treatment target, given the elevated prevalence of co-occurring anxiety patients also attended a number of CBT anxiety-related groups. Specifically, treatment included groups focused on worry and stress management (Craske & Barlow, 2006). Patients also attended a number of groups focused on cultivating core DBT skills (i.e., distress tolerance, emotion regulation, and interpersonal effectiveness; Linehan, 1993). Finally, patients also attended groups focused on an ACT approach to emotional problems (Hayes & Smith, 2005). The ACT-based groups focused on the ineffectiveness of emotional avoidance and on cultivating psychological flexibility, or the ability to accept distressing internal experiences and act in accordance with one’s values. Of note, mindfulness groups were relevant to both DBT and ACT approaches. The remaining group content included psychoeducation, self-monitoring, and wellness. We began collecting data on group attendance after data collection for the current study began. We report descriptive information from this database to provide more detail about the typical treatment received at this partial hospital. For these descriptive data, and similar to the pharmacological treatment data reported above, we applied the same inclusion criteria as the current study (current MDD, consented to research study). For MDD patients, the average number of groups attended over the course of their stay at the BHP was 31 (SD = 10). Approximately 88% of MDD patients attended CBT, DBT, and ACT-based groups. On average, patients attended the following number of groups for the skills examined in the current paper: four behavioral activation, three cognitive restructuring, two distress tolerance, one emotion regulation, one interpersonal effectiveness, one mindfulness, and one acceptance and values-oriented.

Group leaders were rated for adherence to the group protocols twice per year by trained research assistants. Inter-rater reliability is excellent (r = .99), and group leaders, on average, addressed 83% of protocol components during each group (Garner et al., 2014). It is important to note that given the short-term nature of the BHP unit (i.e., average length of stay = 11.7 days in the present sample), the abovementioned groups are short-term skills-based groups, and do not represent the “full-package” or comprehensive treatment in each of the above psychotherapy modalities (e.g., DBT typically requires both a skills group and DBT-specific individual therapy). In addition to group therapy and medication consults with a psychiatrist, all patients also received two to three weekly individual therapy sessions from graduate-level psychologists to reinforce and tailor the skills and lessons learned in groups to the patient’s unique needs (for additional details on the BHP program, see Beard & Björgvinsson, 2013).

Measures

Mini International Neuropsychiatric Interview (MINI; Sheehan et al., 1998)

The MINI is a structured interview assessing for DSM-IV Axis I symptoms (e.g., mood, anxiety, substance abuse, psychosis). Each MINI diagnostic module consists of a series of screening items followed by questions about specific symptomatology. The MINI has strong reliability and validity in relation to the Structured Clinical Interview for DSM-IV (SCID-IV), with inter-rater reliabilities ranging from kappas of .89–1.0 (Sheehan et al., 1998). For the partial hospital patients, inter-rater reliability between the MINI and the program psychiatrists is .69 for MDD and .75 for Bipolar Disorder-Depressed (Kertz, Bigda-Peyton, Rosmarin, & Björgvinsson, 2012). The MINI was administered by doctoral practicum students and interns in clinical psychology who received weekly supervision from a postdoctoral psychology fellow. Training included reviewing administration manuals and completing mock interviews. All clinicians were required to pass a final training interview with their supervisor before administering MINIs for the program.

Center for the Epidemiological Studies of Depression-10 (CES-D-10; Andersen, Malmgren, Carter, & Patrick, 1994)

The CES-D-10 is a widely used, brief patient-report instrument for measuring depressive symptoms. Items assess for symptoms of depression (e.g., “I felt depressed”) and response anchors range temporally from 0 (rarely or none of the time) to 3 (most or all of the time). The CES-D-10 has been shown to have high internal consistency in similar psychiatric samples (α = .87; Webb et al., 2013) and in the current study (α = .85). Given that patients completed the CES-D-10 daily, we modified the instructions and asked patients to rate the frequency of symptoms over the past 24 hours (with the exception of the initial admission, where we used the original CES-D-10 temporal anchor, i.e., over the past week). We altered the 4-point Likert scale accordingly by removing the number of days from each anchor. Internal consistency for the measure was good (α = .88). The same modification to the instructions of a similar scale did not affect its psychometric properties (Beard & Björgvinsson, 2014).

Generalized Anxiety Disorder 7-item Scale (GAD-7; Spitzer et al., 2006)

The GAD-7 is a commonly used, brief self-report questionnaire designed to assess generalized anxiety symptoms. Respondents rate how often they were bothered by different symptoms. Upon initial admission, patients were assessed regarding the past two weeks, with responses anchors ranging from 0 (not at all) to 3 (nearly every day). During all other assessments, patients were assessed regarding the past 24 hours, with response anchors modified to range from 0 (not at all) to 3 (nearly all the time). Scores can range from 0–21, with higher scores indicating greater anxiety severity. The GAD-7 has shown good reliability and validity (Kroenke, Spitzer, Williams, Monohan, & Lowe, 2007) and as measure of general anxiety in our partial hospital population, including the 24-hour version (Beard & Björgvinsson, 2014). The internal consistency for the GAD-7 in our sample was good (α = .84).

Cognitive Behavior Therapy Skills Questionnaire (CBTSQ-16; Jacob, Christopher, & Neuhaus, 2011)

The CBTSQ is a 16-item measure designed to assess cognitive behavioral skills. The measure is divided into two subscales assessing cognitive restructuring skills (CR; “Catch myself when I jump to conclusions”) and behavioral activation strategies (BA; “Socialize even though I don’t feel like it”). Items are rated on a 1 (I don’t do this) to 5 (I always do this) Likert-type scale. Participants were asked how much they actually used said skills/strategies in the past month at admission and were asked about how much they actually used them in the past 24 hours at all other assessments. The CBTSQ-16 was validated among patients within the BHP and demonstrated high internal consistency (α = .84 for cognitive restructuring, α = .80 for behavioral activation; see Jacob et al., 2011). In our sample, the internal consistency of the overall measure was good (α = .89), with acceptable to good consistency of the subscales for behavioral activation (α = .76) and cognitive restructuring (α = .83).

Dialectical Behavior Therapy Ways of Coping Checklist (DBT-WCCL; Neacsiu, Rivzi, Vitaliano, Lynch, & Linehan, 2010)

The DBT Skills Subscale (DSS) of the DBT-WCCL is a 38-item subscale assessing frequency of DBT skills use over the past month. Responses are rated on a 4-point Likert-type scale, from 0 (never used) to 3 (regularly used). Examples of items include: “Stepped back and tried to see things as they really are”, “Talked to someone about how I’ve been feeling”, “Did something to feel a totally different emotion (like going to a funny movie)”, “Tried to distract myself by getting active.” For daily assessments and at discharge, the patients were asked to consider skill usage over the past 24 hours in the face of stressors. The measure has demonstrated adequate to excellent reliability and validity, including the ability to discriminate from treatment conditions providing DBT skills versus those that do not (Neacsiu, Rizvi, & Linehan, 2010; Neacsiu, Rizvi, Vitaliano, Lynch, & Linehan, 2010). The DSS has also demonstrated adequate psychometric properties in a transdiagnostic sample from the BHP (Stein, Hearon, Beard, Hsu, & Björgvinsson, 2016). The internal consistency of this subscale in our sample was excellent (α = .94).

Acceptance and Action Questionnaire—II (AAQ-II; Bond et al., 2011)

The AAQ-II is a 7-item scale designed to assess psychological flexibility/inflexibility. Individuals rate how true a list of statements are for them (e.g., “I’m afraid of my feelings”) on a scale of 1 (never true) to 7 (always true). For daily assessments and at discharge, patients were asked how true the statements were for them in the past 24 hours. Total scores range from 7–49, with higher scores reflecting greater levels of psychological inflexibility. The reliability and validity of the measure have been found to be satisfactory (Bond et al., 2011). Internal consistency in our sample was good (α = .84). The AAQ-II is the most commonly used measure to assess the extent to which patients have developed greater psychological flexibility in ACT (for reviews, see Bluett et al., 2014; Ciarrochi, Bilich, & Godsel, 2010; Hayes, Levin, Plumb-Vilardaga, Villatte, & Pistorello, 2013). It is important to note that the included CBT (CBTSQ) and DBT (DBT-WCCL) measures are designed to assess acquisition and use of cognitive and behavioral skills and DBT coping skills, respectively. However, the AAQ-II is designed to assess psychological flexibility, which is typically conceptualized as a psychological “ability” or a plausible treatment mechanism or “process” of change in ACT rather than a therapeutic “skill” (see Bluett et al., 2014; Hayes et al., 2006).

Procedure

Patients completed a program orientation on the first day of treatment. During the orientation, patients were informed that they would complete daily computerized questionnaires to assess their symptoms and functioning, and that this information would be used by the treatment team. They were also given the opportunity to consent for their clinical data to be used for research purposes. Only patients who provided informed written consent are included in the current report. The local Institutional Review Board at McLean Hospital approved all study procedures. Study data were collected and managed using the REDCap (Research Electronic Data Capture) application hosted at McLean Hospital. REDCap is a web-based application designed to facilitate and securely streamline data capture for research studies (Harris et al., 2009).

The average duration of treatment for these patients was 11.72 (SD = 3.31; Range 4–26) days. Patients completed the MINI on the second day of treatment. The CES-D-10 and GAD-7 were administered everyday of treatment, including admission and discharge. The CBTSQ, DBT-WCCL, and AAQ-II were administered at admission, discharge, and every third treatment day on a rotating basis. Given that the study was conducted in a naturalistic clinical (i.e., psychiatric hospital) rather than research setting, and to reduce assessment burden, measurements of the different skills were staggered such that patients completed only one skills measure each day. Specifically, participants were sequentially assigned to one of six measure rotation orders (e.g., Order 1 = CBTSQ on admission, day 2, day 5, and discharge; DBT-WCCL on admission, day 3, day 6 and discharge; AAQ-II on admission, day 4, day 7, and discharge). In addition, given the naturalistic clinical setting, some patients did not complete all of the self-report and diagnostic measures at one or more of the assessment points for a variety of reasons (e.g., clinical crises resulting in admission to inpatient unit, unexpected discharge). Given the abovementioned range in treatment stays, a number of patients did not have time to complete multiple midtreatment assessments (e.g., not completing day 7 assessment). Overall, and to allow us to test the association between skill acquisition/use and symptom change over the course of treatment, 75% (129/173) of our sample had at least 3 data points for CBT skills (Mean # of data points = 2.9; SD = .67). These values were 75% (130/173) for DBT skills (M = 2.9; SD = .61), 83% (143/173) for the AAQ-II (M = 3.0; SD = .61), and 100% for both the CES-D-10 (M = 6.9; SD = 1.26) and the GAD-7 (M = 6.8; 1.36). Neither the duration of treatment (i.e., number of days from partial hospital admission to discharge) nor the number of data points available moderated the associations tested below between skill use and symptom change (see Footnote 1 for details).¹

Analytic Strategy

To model the association between skill use (i.e., CBT BA vs. CR [CBTSQ], DBT coping skills [DBT-WCCL], and psychological flexibility [AAQ-II]) and symptom change over time, two sets of models were tested predicting subsequent depressive symptom change (CES-D-10) and separately, subsequent change in anxiety symptoms (GAD-7). Similar to Strunk et al. (2012) and Webb et al. (2014), a repeated measures regression was conducted using SAS (9.2) mixed procedure and maximum likelihood estimation. First, to test the association between skill use and subsequent depressive symptom change, a vector of lagged CES-D-10 scores for each patient served as the dependent variable (i.e., CES-D-10 at Time T+1), with CES-D-10 scores at the previous timepoint (Time T) entered as covariates. A vector of CBTSQ/DBT-WCCL/AAQ-II scores were entered as our predictor variables (i.e., Time T). That is, the latter model uses repeated assessments to statistically estimate the relation between the skill measures (Time T) and CES-D-10 scores the next treatment day (Time T+1), adjusting for CES-D-10 scores at the same timepoint as the skill measure assessment (Time T). To control for the influence of prior symptom change, a residualized prior change score was included as a covariate (i.e., CES-D-10 at Time T, adjusting for CES-D-10 at Time 1). In addition, the above models control for CBTSQ/DBT-WCCL/AAQ-II scores at baseline/intake. Models predicting anxiety symptom change were identical as above, but used a vector of lagged GAD-7 scores as the dependent variable. The CBTSQ (BA and CR subscales), DBT-WCCL and AAQ-II were entered into separate models given that, as described above, they were assessed at different timepoints (with the exception of the BA and CR subscales, as they were assessed using the same measure [CBTSQ] and thus at the same assessment timepoints). For the latter models, a negative t value indicates that higher scores on the skills measure predicted relatively lower subsequent symptoms.²

Studies including repeated assessments obtained from the same individuals over time (and consequently within-subject observations, and residuals, over time are likely to be correlated) commonly use covariance structures that include first-order autoregressive, heterogeneous autoregressive, banded Toeplitz, or compound symmetry. In order to select between these different covariance structure options, we fitted models using each structure and chose the best fit based on Akaike’s information criterion, Schwarz’s Bayesian criterion, and −2 log likelihood. For analyses predicting subsequent depressive symptom change, the best fit was banded Toeplitz; for analyses predicting subsequent change in anxiety symptoms, the best fit was first-order autoregressive.

Results

Means, standard deviations, and inter-correlations for skill and symptom variables at intake and discharge are listed in Table 1.

Table 1.

Means, Standard Deviations, and Correlations for all Variables at Admission (Pre) and Discharge (Post)

Variable	M	SD	2	3	4	5	6	7	8	9	10	11	12
1. CES-D-10_Pre	20.30	4.66	.28**	.50**	.29**	−.07	.04	−.28**	−.08	−.24**	−.13	.36**	.22**
2. CES-D-10_Post	12.58	5.80	-	.29**	.68**	.06	−.42**	−.16*	−.58**	−.17*	−.52**	.13	.58**
3. GAD-7_Pre	13.16	5.04		-	.53**	.03	.03	−.11	−.08	−.05	.05	.41**	.23**
4. GAD-7_Post	7.96	4.62			-	.08	−.19*	−.10	−.36**	−.07	−.30**	.28**	.55**
5. CBT-CR_Pre	24.32	6.81				-	.35**	.60**	.17*	.49**	.23**	−.04	−.07
6. CBT-CR_Post	30.44	7.21					-	.26**	.77**	.36**	.68**	−.06	−.40**
7. CBT-BA_Pre	17.66	4.95						-	.38**	.71**	.34**	−.24**	−.23**
8. CBT-BA_Post	22.90	5.57							-	.39**	.73**	−.15	−.53**
9. DBT Skills_Pre	1.39	.51								-	.54**	−.25**	−.22**
10. DBT Skills_Post	1.72	.57									-	−.18*	−.52**
11. ACT-AAQ_Pre	33.91	7.39										-	.40**
12. ACT-AAQ_Post	26.93	8.60											-

Note: CES-D-10 = Center for the Epidemiological Studies of Depression-10; GAD-7 = Generalized Anxiety Disorder 7-item scale; CBT-CR/BA = Cognitive Behavior Therapy Skills Questionnaire – Cognitive Restructuring and Behavioral Activation subscales; DBT Skills = Dialectical Behavior Therapy Ways of Coping Checklist - DBT Skills Subscale; ACT-AAQ = Acceptance and Action Questionnaire—II; Pre = Admission; Post = Discharge. Note that in contrast to the CBT and DBT skills measures for which higher scores reflect greater levels of skills, higher AAQ scores reflect lower levels of psychological flexibility.

^*p < .05.

^**p < .01.

Prediction of Subsequent Depressive Symptom Change

We conducted repeated measures regressions to test the association between skill use and subsequent depressive symptom change over time. As shown in Table 2 (top panel), greater self-reported use of BA strategies significantly predicted greater subsequent depressive symptom improvement. There were no significant associations between depressive symptom improvement and the other skill domain/predictor variables (CR, DBT, AAQ-II).

Table 2.

Repeated Measures Regression Analyses of the Relation between Skill Domain and Subsequent Symptom Change

Predictor	Dependent Variable	Parameter Estimate (b)	Standard Error (SE)	t-value	p-value
CBT-CR	Subsequent Depressive Symptom Improvement	.02	.05	0.44	.664
CBT-BA		−.16	.05	−2.82	.006
DBT Skills		−.08	.06	−1.37	.173
ACT-AAQ		.04	.06	0.71	.479
CBT-CR	Subsequent Anxiety Symptom Improvement	−.01	.05	−0.20	.841
CBT-BA		−.06	.05	−1.06	.290
DBT Skills		−.14	.05	−3.13	.002
ACT-AAQ		.15	.05	2.90	.004

Note: CBT-CR/BA = Cognitive Behavior Therapy Skills Questionnaire – Cognitive Restructuring and Behavioral Activations subscales; DBT Skills = Dialectical Behavior Therapy Ways of Coping Checklist - DBT Skills Subscale; ACT-AAQ = Acceptance and Action Questionnaire—II. All models control for symptoms at the time at which the predictor was assessed, prior symptom change and baseline (i.e., admission) predictor/skills values. Degrees of freedom (df) for CBT-Cognitive and Behavioral skills (df=123); DBT skills (df=116) and AAQ-II (df=128). In the above table, negative t values indicate that higher scores on the predictor variable are related to relatively larger improvements in symptom scores. (Note that in contrast to the CBT and DBT skills measures for which higher scores reflect greater levels of skills, higher AAQ scores reflect lower levels of psychological flexibility. Thus, a positive t-value in the bottom row indicates that lower AAQ scores [i.e., greater psychological flexibility] are associated with relatively larger improvements in anxiety symptoms).

Prediction of Subsequent Anxiety Symptom Change

As shown in Table 2 (bottom panel), both the AAQ-II and DBT skills significantly predicted subsequent anxiety symptom improvement. However, neither BA nor CR skills predicted anxiety symptom change.

Moderating Role of Pretreatment Depression Severity

To test whether pretreatment depression severity moderated the relation between use of BA strategies and depressive symptom improvement (Hypothesis 2), we added a baseline CES-D-10 X BA strategies interaction term to the model tested above and following the recommendation of Aiken and West (1991; i.e., prior to computing cross-products for inclusion in tests of interaction effects predicting subsequent CES-D-10 symptom change, relevant variables were mean-centered). The baseline CES-D-10 X BA strategies interaction term was significant (b = −0.12, standard error [SE] = .04, t = −3.01; p = .003). A median split of the baseline CES-D-10 (Median = 21), and when running a similar repeated measures regression as above, indicated that BA strategies were a significant predictor of subsequent depressive symptom improvement among those with relatively higher (b = −0.35, SE = .09, t = −4.00; p < .001), but not lower (b = −0.04, SE = .08, t = −0.53; p = .595), levels of pretreatment depressive symptoms. Highlighting the specificity of findings to BA strategies, there were no significant interactions between baseline depression severity and the CR subscale of the CBTSQ, nor between baseline depression and either AAQ-II or DBT-WCCL scores (all ps > .15).³

Discussion

The present study examined the association between patient CBT skills, DBT skills, psychological flexibility and symptom improvement while addressing several limitations of prior research. Specifically, in contrast to the bulk of previous studies, we (1) investigated differing subsets of plausibly therapeutic skills/abilities, (2) assessed both depressive and anxiety symptoms as outcome variables, and (3) assessed these variables at multiple timepoints throughout treatment, while (4) statistically controlling for temporal confounds. In addition, and complementing prior research testing skill-outcome associations within the context of clinical trials and outpatient settings, we examined the role of skills in predicting symptom change within a psychiatric hospital setting treating severely depressed patients.

Our hypotheses were partially supported. First, and as expected, results revealed that only BA strategies predicted subsequent depressive symptom improvement within our sample. Indeed, CBT theory emphasizes the benefits of BA for more severely depressed patients, in particular in the early phases of treatment to aid patients in gradually re-engaging with antidepressant sources of reward and positive reinforcement in their environment (Beck et al., 1979; Coffman et al., 2007). Perhaps not surprisingly given the psychiatric setting, the present hospitalized sample had high levels of depressive symptoms at intake (i.e., double the clinical cutoff on the CES-D-10). These findings indicate that BA strategies may be particularly beneficial for relatively severely depressed patients.

It is important to highlight that the psychiatric severity of the sample, coupled with the short-term treatment context (i.e., average length of stay = 11.7 days), may have made it more challenging for these patients to acquire and learn to competently implement cognitive restructuring (CR) skills, in comparison to relatively less complex BA strategies. Cognitive strategies are both more challenging for therapists to implement in a competent manner and, in particular, for patients to grasp and integrate into their day-to-day lives. This is particularly true for patients with severe levels of depressive symptoms who may lack the cognitive resources (e.g., due to impaired concentration, fatigue, anhedonia) and sufficient time (i.e., due to treatment within a short-term unit, as in the present study) to learn these relatively challenging cognitive skills. It should be noted, however, that our sample did report relatively large increases in their use of CR skills (Cohen’s d = .77) from intake to discharge (albeit the increase was numerically larger for BA strategies [d = .89]).

Second, and consistent with our hypothesis, a significant pretreatment depression severity X BA strategies interaction emerged in predicting depressive symptom improvement. The latter interaction was not significant for the other skill domains, including CR. More specifically, in addition to the overall association between use of BA strategies and subsequent depressive symptom change, the above interaction indicates that the relation between BA strategies and symptom improvement is stronger among those patients entering treatment with relatively higher levels of depressive symptoms. Indeed, a median split of pretreatment depression scores (median_CES-D-10 = 21) revealed that BA strategies significantly predicted depressive symptom improvement among patients with higher – but not lower – levels of depression at intake. These results are consistent with clinical trial findings highlighting the relative benefits of BA for higher - but not lower - levels of depression severity (Dimidjian et al., 2006). Taken together, and with regards to clinical implications, these findings suggest that relatively more depressed patients may benefit from treatments emphasizing BA strategies over cognitive restructuring. To date, there has been very little empirical work investigating the underlying mechanisms that may account for the therapeutic benefits of BA for depression. There is some evidence that BA may help normalize abnormalities in neural reward circuitry implicated in depression (e.g., striatal functioning; Dichter et al., 2009). Additional proposed mediators of BA include improvements in the cardinal depressive symptom of anhedonia (Treadway & Zald, 2011) and decreased rumination (Watkins & Nolen-Hoeksema, 2014).

In contrast, CBT-based CR skills, as well DBT skills and psychological flexibility, were not significantly associated with depressive symptom change in this sample. The development of psychological flexibility has been more frequently linked with anxiety symptom improvement (Bluett et al., 2014). However, there is a growing evidence of the efficacy of ACT for depressive symptoms (A-Tjak et al., 2015; Forman et al., 2007), which may in part be due to the core emphasis within ACT on the cultivation of psychological flexibility. Similarly, DBT skills use has previously been shown to mediate depressive symptom improvement (within a BPD sample; Neacsiu et al., 2010). Given these findings, it may seem somewhat surprising that neither DBT skills nor psychological flexibility predicted depressive symptom improvement. As noted above, the short-term nature of the BHP and the severity of depression may help account for why only BA strategies – which are relatively less complex to acquire and utilize – predicted depressive symptom improvement in this sample and setting. Moreover, and as described above, although the majority (approximately 88%) of patients with MDD within the BHP attend CBT-, DBT-, and ACT-based groups, a greater proportion of CBT-based groups (focusing on BA and CR) are provided.

Greater self-reported use of DBT skills and psychological flexibility did predict anxiety symptom improvement. The latter finding is consistent with the growing body of evidence supporting the efficacy of newer generation cognitive behavioral approaches, in particular ACT and DBT, for anxiety symptoms and disorders (Arch et al., 2012; Landy, Schneider & Arch, 2015; Roemer, Orsillo & Salters-Pedneault, 2008; Swain, Hancock, Hainsworth, & Bowman, 2013). Of note, the present depressed sample also presented with high levels of anxiety. Given the significant associations between experiential avoidance and anxiety (Bluett et al., 2014), newer generation mindfulness- and acceptance-based skills may have targeted such avoidance more directly and effectively than CR (or BA). These findings also suggest that interventions focused on cultivating acceptance, mindfulness, and psychological flexibility may be generally more effective than other cognitive interventions (e.g., cognitive restructuring) in decreasing anxiety when symptoms are acute and/or treatment is brief, as in the present study.

It is important to note that the present study focused on the prediction of depressive symptom improvement and our analyses exploring anxiety symptom change were exploratory. In addition, although commonly used, our measure of anxiety symptom change (GAD-7) is a brief 7-item measure. Thus, it is unclear to what extent these findings may generalize to other anxiety measures (e.g., clinician administered instruments). In addition, the items from the CBTSQ assess CR and BA skills more tailored and relevant to depression. Thus, it may be that a CBT skills measure more tailored to anxiety-specific skills (e.g., related to exposure, decreasing physiological reactivity) would have yielded significant associations with anxiety symptom improvement. Nevertheless, and overall, these findings suggest that the depressive and anxiety symptom improvement experienced by patients within our study may be due at least in part to the acquisition and use of the therapeutic skills emphasized within this behavioral health partial hospital program. Future research is needed to investigate - in a more fine-grained manner - which skills in particular are most strongly predictive of symptom improvement, including examining different subsets of symptoms as outcome variables (e.g., improvement in anhedonia vs. negative affect vs. cognitive symptoms).

Several unique characteristics and limitations of the present study should be highlighted to facilitate interpretation of our findings. First, the study was conducted within the context of a short-term naturalistic behavioral health partial hospital program in a sample characterized by high levels of depressive symptoms and comorbidities (in particular, anxiety disorders). The extent to which our findings generalize to other treatment settings is unclear. Although our results may not generalize to the typical outpatient setting, at the same time, a primary impetus for the current study was that the skills-outcome literature to date may not generalize to more acute settings (inpatient, residential, partial hospital units). Such settings represent important and highly utilized - yet understudied - levels of care that treat the most severe and chronic forms of psychiatric illness. Second, skill use was assessed via patient self-report. Thus, it is not clear to what extent our findings would generalize to other assessment methods (e.g., therapist report or observational coding of patient skill use from videotaped sessions). Third, patients were not assessed at follow-up timepoints following discharge. Thus it unclear whether the skill domains assessed may be differentially associated with acute symptom improvement versus longer-term outcomes (e.g., Do BA strategies also predict a reduced risk of depression relapse?). Fourth, we controlled for two plausible confounds: prior symptom change and concurrent symptoms (Strunk et al., 2014). However, given the observational, non-experimental nature of the present design there may have been other third variable confounds that accounted for some of the observed findings (e.g., unmeasured patient characteristics associated with both greater anxiety symptom change and self-reported DBT skills). Fifth, the study focused on the role of CBT/DBT skills and psychological flexibility in predicting symptoms improvement. There are undoubtedly other factors within this partial hospital treatment context that contributed to depressive and anxiety symptom improvement (e.g., increased daily structure and social contact, medication changes, reduced internalized stigma). Indeed, patients in our study did not only receive individual and group therapy, but were also prescribed psychiatric medications. In addition, although we administered the self-report MSI-BPD to assess BPD symptomatology, a structured diagnostic interview should be used in future research to increase the reliability of BPD diagnoses. Finally, as discussed in the Methods, within our partial hospital setting, patients were not assigned to separate therapy groups or tracks (e.g., patients assigned exclusively to CBT groups versus others assigned to ACT only) but rather the majority of patients (approximately 88%) attended CBT-based, DBT-based and ACT-based groups during their stay in the BHP. Thus, a mediation model could not be tested to examine whether any between-group differences in depressive and/or anxiety symptom improvement was mediated by DBT skills, CBT skills or psychological flexibility. Moreover, given the naturalistic setting, detailed information on group attendance was not collected for the present sample.

In conclusion, the current findings from an intensive CBT-based partial hospital treatment program are consistent with previous findings suggesting that BA strategies may be a more efficient and effective means of targeting depressive symptoms than cognitive interventions (Coffman et al., 2007; Dimidjian et al., 2006). In contrast, psychological flexibility and DBT skills predicted anxiety improvement in these depressed patients. These findings highlight the complementary utility of these different intervention strategies in treating depressive versus anxiety symptoms. Future studies should examine whether these relationships are moderated by patient characteristics other than depression severity and whether they replicate in other naturalistic treatment settings.

Research Highlights.

• Cognitive behavioral skills are understudied in naturalistic psychiatric settings.

• We tested whether skills predict depressive/anxiety symptom change in a hospital.

• Behavioral activation (BA) strategies predicted depressive symptom improvement.

• Patient severity moderated the relation between BA strategies and symptom change.

• Psychological flexibility and DBT skills predicted anxiety symptom change.