What Changes in Cognitive Therapy for Depression? An Examination of Cognitive Therapy Skills and Maladaptive Beliefs

Abstract

This study examined effortful cognitive skills and underlying maladaptive beliefs among patients treated with Cognitive Therapy for depression (CT). Depressed patients (n = 44) completed cognitive measures before and after 16 weeks of CT. Measures included: an assessment of CT skills (Ways of Responding Scale, WOR), an implicit test of maladaptive beliefs (Implicit Association Test, IAT), and a self-report questionnaire of maladaptive beliefs (Dysfunctional Attitude Scale, DAS). A matched sample of never-depressed participants (n = 44) also completed study measures. Prior to treatment, depressed patients endorsed significantly more undesirable cognitions on the WOR, IAT, and DAS compared to never-depressed participants. Patients displayed improvement on the WOR and DAS over the course of treatment, but showed no change on the IAT. Additionally, improvements on the WOR and DAS were each related to greater reductions in depressive symptoms. Results suggest that the degree of symptom reduction among patients participating in CT is related to changes in patients’ acquisition of coping skills requiring deliberate efforts and reflective thought, but not related to reduced endorsement of implicitly-assessed maladaptive beliefs.

Cognitive Therapy for depression (CT; Beck, Rush, Shaw, & Emery, 1979) is a well-studied treatment with substantial evidence for its efficacy (DeRubeis, Webb, Tang, & Beck, 2010). While cognitive change has long been suggested as important to the therapeutic benefits of CT, the nature of the cognitive changes produced by CT remains unclear. Clarifying the nature of these changes is important to advancing a basic understanding of depression as well as elucidating the process by which therapeutic gains are achieved in CT.

As Barber and DeRubeis (1989) suggested, different kinds of cognitive change might explain the therapeutic benefits of CT. First, CT could help patients develop skills to cope with negative thoughts when they occur. Such cognitive change would involve deliberate, ongoing efforts to employ cognitive and behavioral strategies. These efforts require patients to understand, practice, and generalize a variety of skills during appropriate situations (Jarrett, Vittengl, Clark, & Thase, 2011). Alternatively, CT could help patients change their underlying maladaptive beliefs (e.g., If others do not approve of me, I am worthless) through therapeutic procedures that allow them to see that these beliefs are not reasonable, accurate, or adaptive. If patients changed their beliefs, they would be unlikely to have future negative thoughts related to their self-worth and therefore have less need to engage in ongoing effortful cognitive strategies to cope with their negative thoughts. The purpose of this study was to evaluate cognitive change in CT by clarifying the role of effortful (i.e., skill acquisition) and underlying (i.e., maladaptive beliefs) cognitive processes.

Assessment of CT Skills

In CT, patients learn skills to apply when experiencing negative moods that allow them to handle these experiences in more adaptive ways. Much of the skills taught in CT are related to identifying negative automatic thoughts and evaluating the accuracy of these thoughts. However, CT also involves other skills, including behavioral strategies such as how to engage in certain kinds of activities to improve one's mood. Therapists providing CT continually help patients develop new skills and enhance their use of existing skills to cope with negative moods. While several efficient measures of CT skills have been developed recently (e.g., Skills of Cognitive Therapy in Jarrett et al., 2011; Cognitive-Behavioral Therapy Skills Questionnaire in Jacob, Christopher, & Neuhaus, 2011), these measures each rely on patients’ ability to report on their own mastery and use of CT skills (for a review, see Hundt, Mignona, Underhill, & Cully, 2013).

Barber and DeRubeis (1992) developed the Ways of Responding Scale (WOR) to assess patients’ mastery of the coping strategies taught in CT. The WOR presents respondents with six hypothetical scenarios and some initial negative thoughts one might have in each situation. Using an open-ended response format, the WOR asks respondents to describe what their further thoughts and actions might be in each scenario. Responses are parsed into thought units and coded for whether each thought unit reflects a positive or negative coping strategy. A total score is calculated as the difference between the number of positive and negative responses.

The WOR has been used to examine change in coping skills over the course of psychotherapy in several studies. Barber and DeRubeis (2001) found that patients who participated in 12 weeks of CT for depression showed significant pre- to post-treatment improvements in WOR total scores (d = .70). Furthermore, greater changes in WOR total scores were related to greater improvements in self-reported depressive symptoms (r = .54). Across both cognitive and dynamic psychotherapies, Connolly Gibbons et al. (2009) found that patients experienced positive changes in WOR total scores (d = .47). Consistent with Barber and DeRubeis's findings, change on the WOR was related to concurrent change in depressive symptoms (partial eta = -.23 for BDI and -.33 for HRSD). While these studies suggest that the acquisition of CT skills is related to depressive symptom reductions in CT, they do not address the possibility that changes in underlying maladaptive beliefs may also occur during CT. In fact, evidence of improvements in WOR scores could be due to changes in maladaptive beliefs that reduce patients’ tendency to report negative coping strategies because they are less likely to experience negative thoughts.

Assessment of Belief Change

Assessing changes in underlying maladaptive beliefs in a way that is not likely to be contaminated by individual differences in CT skills requires careful consideration of the available methods of measurement. Self-report questionnaires, which are often used to assess depression-related beliefs, can be susceptible to self-presentation and expectancy biases. For example, a patient who has completed a course of CT has likely learned what types of beliefs his or her therapist views as adaptive. We suspect some patients may be motivated to endorse these desirable responses even though they do not fully believe them. In this case, self-report measures would fail to capture important individual differences in underlying beliefs that are not contaminated by the desire to give adaptive responses. A number of researchers have developed methods of assessing implicit cognition that minimize or eliminate these potential self-presentational biases (see Petty, Fazio, & Brinol, 2008). One promising method of assessing implicit cognition is the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998). IATs are computer-based tasks that measure the ease with which individuals can associate two concepts. Participants’ rapid categorization judgments provide an indirect assessment of beliefs, even though the task does not involve a direct query regarding the belief of interest (Greenwald, Poehlman, Uhlmann, & Banaji, 2009). For example, an IAT assessing gender stereotypes might ask participants to categorize words into the categories of male versus female or science versus liberal arts – but do so using only one of two response keys. By altering the response mapping across blocks of the task, the gender-IAT examines whether participants are faster to respond when the keys are labeled “male / science” and “female / liberal arts” than when they are labeled “female / science” and “male / liberal arts.” Differential latencies across the response mappings provide an estimate of participants’ beliefs regarding the association of gender with science or the liberal arts – beliefs that some people may hold but be motivated to deny on self-report measures. Thus, an IAT offers a means of assessing changes in underlying maladaptive beliefs associated with depression that is not likely to be affected by individual differences in the use of effortful skills.

IATs have been utilized to capture beliefs related to depression in several studies. For example, De Raedt, Schacht, Franck, and De Houwer (2006) developed a self-worth IAT that assesses the ease of associating the “self” with being “valuable” versus “worthless”. The self-worth IAT was significantly related to depressive symptoms among a sample of college students selected for high and low depressive symptoms (r = -.46; De Raedt, Franck, Fannes, & Verstraeten, 2008). In another study, Franck, De Raedt, Dereu, and Van den Abbeele (2007) reported significantly lower self-worth on an IAT among currently-depressed patients compared with never-depressed controls.

While research has yet to examine changes in beliefs using IATs in the treatment of depression, we are aware of three such studies examining the treatment of anxiety disorders. Gamer and colleagues (Gamer, Schmukle, Luka-Krausgrill, & Egloff, 2008) reported significant reductions in self-anxious associations on an IAT following group cognitive-behavioral therapy (CBT) for social anxiety. Similarly, Teachman, Marker, and Smith-Janik (2008) reported significant reductions in panic associations on an IAT following group CBT for panic disorder. Change in panic associations on an IAT between assessments correlated with concurrent reductions in panic symptoms. Finally, in a study of participants with a public speaking phobia, Vasey, Harbaugh, Buffington, Jones, and Fazio (2012) found significant reductions in IAT-assessed negative attitudes towards public speaking following a one-session exposure treatment. Although these studies suggest that an IAT can capture change over the course of psychotherapy, these studies focused on attitudes central to anxiety disorders and may not generalize to depression. In addition, the first two of these studies assessed associations of the self with being anxious. A comparable depression-IAT would assess the association of the self with being depressed. However, more than just no longer seeing themselves as depressed, change in maladaptive beliefs central to the cognitive model of depression would involve depressed patients no longer holding maladaptive beliefs which constitute greater vulnerability to depression (such as a need for approval by others). Therefore, to test this model, an IAT assessing these types of maladaptive beliefs is needed. For example, an IAT that assesses the ease with which an individual associates the self with being valued by significant others would focus on a specific type of belief overlapping in content with the Dysfunctional Attitude Scale (discussed below). Although this is just one type of belief typical of depressed patients, it is often targeted in CT and would potentially change during a successful course of treatment.

Self-Reported Assessment of Maladaptive Beliefs

While we selected the WOR and IAT as our measures of CT skills and maladaptive beliefs, respectively, other measures of depressive cognition have often been examined in the context of CT. Of these other measures, the most commonly used is the Dysfunctional Attitude Scale (DAS; Weissman, 1979). The DAS is a self-report questionnaire that asks respondents to indicate the degree to which they endorse specific dysfunctional attitudes, which we also refer to as maladaptive beliefs. Based on factor analyses (see De Graaf, Roelofs, & Huibers, 2009), the DAS assesses two factors: Performance Evaluation and Need for Approval by Others. These factors include items assessing beliefs related to one's worth or happiness being influenced by either negative evaluations of one's performance or the approval of others, respectively. A number of studies have reported substantial change in DAS scores over the course of CT (Barber & DeRubeis, 2001: d = .46; Jarrett, Vittengl, Doyle, & Clark, 2007: d = 1.05). Moreover, changes on the DAS have been related to concurrent change in depressive symptoms (Barber & DeRubeis, 2001: r = .67; Jarrett et al., 2007: r = .60). While these findings appear to be consistent with the role of belief change in CT, it is possible that self-report measures like the DAS could be influenced by patients’ use of effortful processes (e.g., coping skills). Participants’ responses to self-report measures may reflect their reappraisal of cognitions rather than their primary evaluations (Barber & DeRubeis, 1989). However, given the widespread use of the DAS, we planned to examine the DAS along with our two primary cognitive measures, the WOR and IAT.

Purpose of this Study

The purpose of this study was three-fold: (1) to compare depressed and never-depressed participants across cognitive measures reflecting CT skills and maladaptive beliefs; (2) to assess the nature of cognitive changes (i.e., CT skill acquisition and belief change) occurring over a course of CT for depression; and (3) to examine the relationship of different kinds of cognitive change (i.e., CT skill acquisition and belief change) with symptom change over the course of CT. Compared to never-depressed participants, we hypothesized depressed patients would endorse poorer skills and more maladaptive beliefs. We expected pre-to-post treatment improvements on the WOR, IAT, and DAS. Finally, we expected that improvement on each cognitive measure would be related to reductions in depressive symptoms. While we have suggested that one might expect cognitive change to involve changes in CT skills or maladaptive beliefs, it need not be one or the other. Both kinds of cognitive change may occur and be related to patients’ symptom reduction. In fact, it may be that patients’ early use of CT skills facilitates changes in underlying beliefs or changes in underlying beliefs promote use of CT skills. While these complex relations are beyond the scope of our study, we endeavored to take a first step in characterizing the degree of change in CT skills and underlying beliefs. Insofar as both kinds of cognitive change are important, one would expect that the most successful treatments for depression would involve both an enhancement of coping skills and belief change.

Method

Participants

We recruited two samples of participants from the local community via flyers, newspaper, and online advertisements: a sample of depressed patients and a sample of never-depressed participants. The depressed sample consisted of patients with a primary diagnosis of current DSM-IV Major Depressive Disorder (MDD), no substance dependence in the past six months and no history of Bipolar I Disorder or psychosis. As the depressed sample flowchart in Figure 1 shows, 86 patients attended an initial evaluation and were assessed for study eligibility. Of the 67 patients who were enrolled in the study and began treatment, 23 patients (34%) dropped out of treatment prematurely. Dropout was defined as attending fewer than 10 sessions or failing to complete a post-treatment assessment. The remaining 44 patients (66%) completed 16 weeks of treatment. In comparing patients who completed treatment and those who discontinued prematurely, we found no significant differences in age, gender, income, marital status, or initial depressive symptom severity. However, compared with treatment completers, a greater proportion of patients who dropped out of treatment were not Caucasian (30% of dropouts vs. 9% of completers were non-Caucasian; χ² = 5.01, p = .03). Additionally, there was a non-significant trend for patients who dropped out of treatment to have less education compared to those who completed treatment (19% of dropouts vs. 5% of completers had a high school degree, 53% of dropouts vs. 43% of completers had some college/2-year degree, 28% of dropouts vs. 52% of completers had a 4-year degree/graduate school; χ² = 5.29, p = .07).

Figure 1.

Flow of Patients through Assessments and Treatment

The never-depressed sample consisted of 44 participants with no current or lifetime history of MDD or any anxiety disorder. These control participants were selected on the basis of an initial online screening indicating low symptoms of depression and anxiety [i.e., scoring less than 5 on both the Patient Health Questionnaire (PHQ; Kroenke & Spitzer, 2002) and Generalized Anxiety Disorder Questionnaire- IV (GAD-Q-IV; Newman et al., 2002)] and being a demographic match for one of the 44 depressed patients who completed treatment. Participants qualified as a demographic match by matching a depressed patient on age (within 3 years), gender, and education (i.e., high school degree, some college/2-year degree, or 4-year degree/higher). Following the online screening, 53 participants were assessed for lifetime history of MDD or any anxiety disorder. Of these, 9 were not utilized as control participants [8 due to DSM-IV diagnoses (5 past MDD; 2 social phobia; 1 both past MDD and social phobia) and 1 due to discontinuing prior to completion of the study assessment]. The resulting 44 matched control participants were included in the study. This study was approved by the university's Institutional Review Board.

Demographic and diagnostic data for the depressed and never-depressed samples are summarized in Table 1. As our primary analyses focus on treatment completers, we have limited our descriptive statistics for the depressed sample to depressed patients who completed treatment (n = 44).

Table 1.

Baseline Demographic Information for the Depressed (n = 44) and Never-Depressed (n = 44) Samples

Variable	Depressed	Never-depressed
Female, % (n)	57 (25)	57 (25)
Age in years, mean ± SD	37.8 ± 13.7	37 ± 13.6
Range	18 – 63	18 – 65
Ethnicity, % (n)
White	91 (40)	86 (38)
Minority	9 (4)	14 (6)
Married or cohabitating, % (n)	39 (17)	39 (17)
Income, mean ± SD (in thousands of US $)a,b	40.90 ± 30.84	60.95 ± 32.84
Education, % (n)
High-school degree	5 (2)	5 (2)
Some college/2-yr degree	43 (19)	43 (19)
4-yr degree/graduate school	52 (23)	52 (23)
Two or more past MDEs, % (n)	43 (19)	-
Current comorbid disorder, % (n)
Dysthymic Disorder	25 (11)	-
Any anxiety disorder	59 (26)	-
Any personality disorder	34 (15)

Note.

^amissing data for 5 depressed patients,

^bmissing data for 2 never-depressed participants,

MDE = Major Depressive Episode.

Measures

Diagnostic Evaluations

The Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I; First, Spitzer, Miriam, & Williams, 2002a, 2002b) and Structured Clinical Interview for DSM-IV Axis II Disorders (SCID-II; First, Gibbon, Spitzer, Williams, & Benjamin, 1997) were used to assess whether participants met formal criteria for current MDD and other Axis I and Axis II conditions. Reliability for a diagnosis of current MDD based on double-ratings for 12 randomly-selected cases was excellent (kappa = 1.00).

Depressive Symptom Severity

We used two measures of depressive symptom severity: the Hamilton Rating Scale for Depression (HRSD) and the Beck Depression Inventory - 2^nd Edition (BDI-II). The HRSD is an interviewer-administered measure. We used the 17-item HRSD modified to assess atypical symptoms (Hamilton, 1960; Williams, 1988). Reliability for total scores on the HRSD in the current sample based on double-ratings for 30 randomly-selected cases was excellent (ICC = .99). The BDI-II (Beck, Steer, & Brown, 1996) is a 21-item self-report measure. Respondents are asked to describe the way they have been feeling by rating each item on a scale from 0 to 3, with total scores ranging from 0 (minimal depression) to 63 (high depression). Although the BDI-II typically asks respondents about the past two weeks, to facilitate assessing the degree of change in depressive symptoms, we modified the instructions to inquire about the past week (see Dimidjian et al., 2006 for a similar modification). Internal consistency for the BDI-II was acceptable (depressed patients at intake: α = .84; never-depressed participants: α = .71).

Ways of Responding Scale (WOR)

The WOR (Barber & DeRubeis, 1992) measures coping skills cognitive therapists endeavor to develop in their patients. Participants are asked to respond to 6 hypothetical stressful situations (e.g., turned down for multiple jobs) in which their initial thoughts regarding the event are given (e.g., “There just doesn't seem to be any point in applying”). Participants are then asked to report any further thoughts they have regarding the situation and what they would do at that point. Responses were coded by three raters blind to patient identifiers or assessment point using procedures outlined by Barber and DeRubeis in the WOR Rater's Guide. Responses were first parsed into individual thought units which were then coded by two independent raters as being one of 25 specific positive (e.g., coming up with a specific plan or solution) or negative (e.g., avoiding the situation) categories. A third rater resolved any disagreements between the first two raters. These ratings were then used to calculate the total score, which is the number of positive responses minus the number of negative responses. Raters were all cognitive therapists with one or more years of experience providing CT. Training included approximately 30 hours of practice rating using non-study protocols. We used kappa to evaluate agreement on the specific categories assigned between the first and second raters and agreement on the use of positive versus negative categories. Both kappas indicated substantial agreement (.66 and .79, respectively; Landis & Koch, 1977).

Valued-Implicit Association Test (valued-IAT)

As described earlier, IATs are computer-based assessments in which participants are shown a list of words one at a time and asked to categorize each word as quickly as possible (Greenwald et al., 1998). The valued-IAT we used in this study was a personalized, single-category IAT for which stimuli were identified from a pre-test questionnaire (see Online Supplement). Participants were asked to provide information related to four different categories: “me,” “not me,” “valued by my associates,” and “not valued by my associates” (information related to the “not me” category was only used during the practice blocks to reduce the potential influence of this contrast category on the assessment of the association between “me” and “valued” vs. “not valued”; the single-category IAT has demonstrated at least comparable reliability and validity to traditional IATs; Karpinski & Steinman, 2006). The “valued” and “not valued” category labels were worded to reflect features of the DAS subscale assessing need for social approval. For the “valued” and “not valued” categories, participants were asked to consider what their associates (e.g., friends, family members, or other people whose opinion matters to them) find to have worth. Participants were given a list of 60 words (based in part on values research by Bardi and Schwartz, 2003) and asked to select 10 words that describe things that are “valued by my associates” and an additional 10 words that describe things that are “not valued by my associates.” Personalized stimuli were used for all categories to reduce the influence of extrapersonal associations (i.e., cultural values that are inconsistent with one's attitude but related to the stimuli), which have been shown to influence IAT scores (Olson & Fazio, 2004). Stimuli were presented in random order in the center of the computer screen with trial intervals set at 100 ms. On each trial, participants were presented with a word to categorize as belonging to one of three categories (using two response keys, with one key being used for either of two categories). Category labels were displayed at the bottom right and bottom left of the computer screen. Error feedback was provided such that when an incorrect categorization was made, participants saw a large red X on the screen and were required to press the correct key before the next stimulus was displayed.

The task involved two practice blocks of 20 words each followed by 8 test blocks of 30 words each. During the first practice block, participants identified stimuli as being “valued by my associates” or “not valued by my associates”. During the second practice block, participants identified stimuli as “me” or “not me”. During test blocks 3, 5, 8, and 10, participants were asked to use the right key to indicate when a stimuli belonged to either the “me” or “valued” category; the left key was to be used to identify stimuli that belonged to the “not valued” category. During test blocks 4, 6, 7, and 9, participants used the right key to categorize stimuli as “valued” and the left key to categorize stimuli as “me” or “not valued.” Thus, these critical test blocks varied whether “me” was mapped on to the same response key as “valued” or “not valued.” Scores were calculated by subtracting response times when “me” and “valued” were paired and when “me” and “not valued” were paired. These difference scores were taken to reflect the strength of participants’ belief that they are valued by others, with positive scores reflecting the belief that one is valued by others and negative scores reflecting the belief that one is not valued by others. IAT scores were calculated using a modified version of the improved scoring algorithm (Greenwald, Nosek, & Banaji, 2003) proposed by Fazio and colleagues (Han, Czellar, Olson, & Fazio, 2010; Olson & Fazio, 2004) that log-transformed reaction times and did not impose a 600 ms penalty for incorrect trials. The improved scoring algorithm with modifications included the following steps: 1) deleting trials with reaction times greater than 10,000 ms and eliminating participants for whom 10% or more of trials have reaction times faster than 300 ms, 2) deleting practice trials (blocks 1 and 2), 3) log-transforming reaction times, 4) computing the average log-transformed reaction time for correct trials separately for each test block, 5) computing pooled standard deviations for all trials in blocks 3 and 4, blocks 5 and 6, blocks 7 and 8, and blocks 9 and 10, 6) averaging the reaction time for each test block after replacement of error latencies, 7) computing four differences by subtracting the average latency for block 3 from 4, block 5 from 6, block 8 from 7, block 10 from 9, 8) dividing each block by its respective pooled standard deviation, and 9) calculating the final IAT score by averaging the differences computed in step 8. Reliability was evaluated by calculating Cronbach's alpha for the four difference scores computed from paired test blocks. For the depressed group, alphas at intake and post-treatment were .58 and .78, respectively. For the never-depressed group, alpha was .73.

Dysfunctional Attitude Scale (DAS)

The DAS (Weissman, 1979) is a 40-item self-report measure designed to assess stable and enduring maladaptive beliefs typical of depressed patients in accordance with cognitive theory. Respondents are asked to describe the way they think most of the time by rating each statement (e.g., “I cannot be happy unless most people I know admire me”) on a scale from 1 (disagree totally) to 7 (agree totally). Responses on the DAS are summarized with a total score, which is the sum of responses for the 40 items. Internal consistency for the DAS was excellent (depressed patients at intake: α = .93; never-depressed participants: α = .87).

Assessment and Treatment

Assessments were conducted by five advanced graduate students with one or more years of clinical experience. While several assessors also served as cognitive therapists, therapists did not conduct assessments for clients for whom they were providing CT. Interviewers completed a course of training in the use of the SCID-I, SCID-II, and HRSD. Assessment training involved approximately 50 hours of watching training videos, group consensus practice ratings, and weekly supervision provided by the second author.

Treatment was provided by four advanced graduate students with at least one year of clinical experience prior to treating patients in the study (range of one to two years). CT training involved approximately 100 hours of clinical training in CT (with a focus on experiential role playing). Over the course of the study, the second author provided weekly group and individual supervision. Treatment consisted of 16 weeks of Cognitive Therapy, which was delivered according to Beck et al. (1979). Patients with HRSD scores greater than or equal to 20 at intake were provided twice-weekly therapy sessions for the first four weeks, while those with HRSD scores less than 20 received once-weekly therapy sessions.

To evaluate therapist competence, we used the Cognitive Therapy Scale (CTS; Young & Beck, 1988). The CTS is an observer-rated scale containing 11 items rated on a 7-point Likert scale. Total scores range from 0-66, with experts suggesting scores of 40 or greater indicate adequate competence (Muse & McManus, 2013). To assess therapist competence in our study, for each study therapist we randomly chose one case from the therapist's caseload after two cases were removed from consideration (i.e., therapist-nominated cases judged likely to yield the lowest and the highest competence scores). Two evaluators (the study supervisor and a founding fellow of the Academy of Cognitive Therapy unaffiliated with the study) provided independent ratings of a randomly-selected session recording for each therapist using the CTS. The correlation between the four ratings made by both evaluators was large (r = .998). The mean CTS scores provided by each rater were 39.25 (SD = 2.8) and 42.0 (SD = 10.6), respectively. While two therapists scored above 40, two scored below 40.

It is important to note that competence is often evaluated based on therapist-nominated sessions (Academy of Cognitive Therapy, n.d.). CTS scores from randomly-selected sessions are likely to be lower than therapist-nominated sessions. To provide additional data on the two therapists whose sessions were rated below 40, a second randomly-selected session recording was evaluated by the study supervisor only. The second ratings each yielded a score of 40. The mean of the maximum CTS scores achieved by each therapist (as rated by the study supervisor) was 40.75 (with all ratings scoring 40 or higher).

Procedures

Depressed participants completed study measures at intake, week 4, and post-treatment assessments. Analyses of the HRSD and BDI-II draw on data from each of these three time-points. Cognitive measures (WOR, valued-IAT, and DAS) and diagnostic status (SCID-I) were assessed at intake and post-treatment only. Depressed patients were compensated at $10 per hour for the post-treatment assessment only. Participants in the never-depressed sample were compensated at $10 per hour for study assessments. All participants provided informed consent prior to completing study assessments.

Analytic Strategy

As noted above, our primary analyses examine cognitive changes from intake to post-treatment for depressed patients, which necessitates that data be available at both assessment points. Therefore, analyses were conducted based on treatment completers (n = 44). Due to computer error, data were missing for one depressed participant at intake on the WOR, DAS, and BDI-II. WOR data were also missing for two depressed participants who never completed this measure at intake. Thus, available data for the depressed sample at intake on each measure was as follows: n = 44 for the HRSD, n = 43 for the BDI-II, n = 41 for the WOR, n = 44 for the valued-IAT, and n = 43 for the DAS. Among treatment completers, all data was available at post-treatment.

Given that each set of analyses involved tests for three cognitive measures, we applied the Holm-Bonferroni method (Holm, 1979) to adjust for the three tests involving the WOR, valued-IAT, and DAS. Throughout the results section, we report p-values for each of these tests. With only one exception, significant effects remained significant after applying the Holm-Bonferroni method. We only comment on the Holm-Bonferroni correction for this one instance when an initially significant result was no longer significant after the correction.

Examination of the BDI-II data revealed evidence of a non-linear pattern of symptom change across time. We considered several transformations of time, including a log transformation and a square root transformation. The best fitting model was chosen by examining the Aikake Information Criteria (AIC) and Bayesian Information Criteria (BIC), two measures of model fit. The square root transformation provided the best fit (i.e., time values of 0, 2, and 4 were used for the intake, week 4, and post-treatment assessments, respectively). As change on the HRSD appeared linear and transformation did not yield better fitting models, we used an untransformed representation of time (values of 0, 4, and 16 to represent intake, week 4, and post-treatment, respectively). To examine concurrent change in cognitive measures from intake to post-treatment in relation to depressive symptom change, hierarchical linear modeling (HLM) was conducted using SAS Version 9.2 PROC MIXED. In separate models for the HRSD and BDI-II, we modeled subject specific intercepts and slopes of depressive symptom change. To generate scores reflecting change in cognitive measures between two time points, we used residual change scores (i.e., the residuals of a regression model that included the intake score as an independent variable and the post-treatment score as the dependent variable). In the HLM models, the main variable of interest was the interaction of time by cognitive residual change score. A significant, negative time by residual cognitive change interaction would indicate that change in the cognitive variable was related to a larger magnitude of change in depressive symptoms. For ease of interpretation, we have modified the signs for tests involving the DAS (where higher scores reflect more maladaptive responses). After this modification, a negative DAS residual change by time interaction would indicate improvements in DAS being associated with a larger degree of change in depressive symptoms. In addition to regression parameters, for HLM models we report r-type effect sizes that were estimated from the t-statistic and degrees of freedom from each test.

Results

Over the course of 16 weeks of treatment, depressed patients who completed treatment attended an average of 15.4 sessions (SD = 2.7, range = 11 – 22). Means and standard deviations for depressive symptoms and scores on the cognitive measures are presented in Table 2. As the table shows, there were significant, large decreases in depressive symptoms from intake to post-treatment as measured by the HRSD (d = -2.27, t(43) = 11.14, p < .0001) and BDI-II (d = -2.11, t(42) = 9.49, p < .0001). Of the 44 depressed patients, 29 (66%) responded to treatment (i.e., no longer met criteria for current MDD and obtained an HRSD < 12 at post-treatment). A total of 19 patients (43%) met remission criteria with a post-treatment HRSD less than or equal to 7. We also examined the relations among cognitive measures in the depressed sample. As shown in Table 3, scores on the WOR and DAS were significantly, negatively correlated at intake and post-treatment, while scores on the valued-IAT were not significantly related with scores on the other cognitive measures at either time point.

Table 2.

Means and Standard Deviations for Depressive Symptoms and Scores on Cognitive Measures for the Depressed Sample at Intake and Post-Treatment and the Never-Depressed Sample

	D-I	D-P	ND	D-I vs. D-P	D-I vs. ND	D-P vs. ND
Variable	M (SD)	M (SD)	M (SD)	d	d	d
HRSD	20.34 (4.58)	8.95 (5.43)	2.25 (2.08)	−2.27***+	5.09***+	1.63***+
BDI-II	26.95 (8.62)	9.73 (7.69)	1.30 (2.13)	−2.11***+	4.09***+	1.49***+
WOR	−5.12 (10.61)	12.84 (12.10)	15.02 (11.45)	1.53***+	−1.82***+	−.19
Valued-IAT	.31 (.20)	.33 (.22)	.42 (.25)	.10	−.49*+	−.38
DAS	140.98 (33.75)	123.32 (31.70)	94.20 (21.43)	−51**+	1.65***+	1.08***+

Note. Due to missing data at intake, sample sizes differ for the depressed sample at intake vs. post-treatment. All available data contributed to the means and standard deviations reported, but only paired data were used to calculate the effect-sizes for depressed at intake vs. post-treatment.

D-I = depressed at intake, D-P = depressed at post-treatment, ND = never-depressed, HRSD = Hamilton Rating Scale for Depression, BDI-II = Beck Depression Inventory – 2^nd Edition, WOR = Ways of Responding Scale, Valued-IAT = valued-Implicit Association Test, DAS = Dysfunctional Attitude Scale.

^*p < .05

^**p < .01

^***p < .0001

⁺significant after Holm-Bonferroni correction.

Table 3.

Correlations between Cognitive Measures at Intake and Post-treatment among the Depressed Sample

	1.	2.	3.	4.	5.	6.
1. WOR intake	41
2. WOR post-treatment	.02	44
3. Valued-IAT intake	−.13	.03	44
4. Valued-IAT post-treatment	.05	−.19	.62***	44
5. DAS intake	−.42**	−.22	−.01	.12	43
6. DAS post-treatment	.02	−.42**	−.01	.18	.65***	44

Note. Sample sizes for each measure at a particular time point are provided on the diagonal. WOR = Ways of Responding Scale, Valued-IAT = valued-Implicit Association Test, DAS = Dysfunctional Attitude Scale

^**p < .01

^***p < .0001.

Comparison of depressed patients and never-depressed participants on cognitive measures

We first examined differences on cognitive measures between depressed patients before treatment and never-depressed participants. As shown in Table 2, compared to never-depressed participants, depressed patients at intake reported significantly poorer coping skills on the WOR (d = -1.82, t(83) = 8.40, p < .0001), significantly less adaptive beliefs related to being valued by others on the valued-IAT (d = -.49, t(86) = 2.18, p = .03), and significantly more self-reported maladaptive beliefs on the DAS (d = 1.65, t(85) = -7.74, p < .0001). Thus, across all three cognitive measures, depressed patients exhibited more maladaptive scores than the never-depressed participants.

Cognitive changes over the course of treatment

Next, we examined whether depressed patients exhibited significant changes in cognition over the course of treatment. As shown in Figure 2 and Table 2, there was a significant, large improvement from intake to post-treatment on the WOR (d = 1.53, t(40) = -7.01, p < .0001). On the valued-IAT, we failed to find significant change from intake to post-treatment (d = .10, t(43) = -.48, p = .63). Finally, there was a significant, medium effect-sized decrease on the DAS from intake to post-treatment (d = -.51, t(42) = 3.98, p = .0003). Thus, over the course of treatment depressed patients showed substantial improvements in coping skills on the WOR and self-reported maladaptive beliefs on the DAS but no belief change on the valued-IAT.

Figure 2.

Changes on cognitive measures from intake to post-treatment reported as the absolute-value of Cohen's d effect sizes

We conducted analyses to examine the relations among change on the three cognitive measures using residual change scores. Change on the valued-IAT was not significantly correlated with change on the WOR (r = -.28, p = .08, n = 41). Similarly, change on the valued-IAT was not significantly related to change on the DAS (r = .24, p = .12, n = 43). Change on the DAS was correlated with change on the WOR (r = -.34, p = .03, n = 41), but this relation did not remain significant after applying the Holm-Bonferroni correction. In summary, no relation of changes in CT skills and maladaptive beliefs assessed by the three cognitive measures achieved the adjusted level of significance.

Although we lacked an a priori hypothesis, we compared the cognitive scores of depressed patients at post-treatment with those of never-depressed participants (see Table 2). There were no significant differences between groups on the WOR (d = -.19, t(86) = .87, p = .4) or the valued-IAT (d = -.38, t(86) = 1.83, p = .07). However, the depressed group continued to endorse significantly more self-reported maladaptive beliefs post-treatment compared to the never-depressed participants (d = 1.08, t(86) = -5.05, p < .0001). When we consider these analyses along with those comparing depressed patients at intake and never-depressed participants, differences in DAS scores were identified both at intake and post-treatment, whereas differences in WOR and valued-IAT scores were present at intake but no longer evident at post-treatment.

Cognitive change and change in depressive symptoms

Using separate HLM models for the HRSD and BDI-II, we examined the relationship between cognitive change from intake to post-treatment (measured as residual change scores) and concurrent change in depressive symptoms (measured at intake, week 4, and post-treatment). Results for the HRSD and BDI-II models are presented in Table 4. For analyses of the HRSD, change on the WOR and DAS were each significantly related to the degree of depressive symptom change (interaction of WOR change by Time: r = -.36, β = -.01, 95% CI [-.02, -.003], t(41) = -2.51, p = .02; interaction of DAS change by Time: r = -.39, β = -.007, 95% CI [-.002, -.01], t(43) = -2.78, p = .008), but change on the valued-IAT was not (interaction of valued-IAT change by Time: r = .23, β = .58, 95% CI [-.17, 1.33], t(44) = 1.55, p = .13). When depressive symptoms were measured using the BDI-II, all three cognitive measures were significantly related to the degree of depressive symptom change (interaction of WOR change by Time: r = -.30, β = -.07, 95% CI [-.15, -.001], t(41) = -2.05, p = .047; interaction of valued-IAT change by Time: r = .39, β = 6.84, 95% CI [1.94, 11.74], t(43) = 2.81, p = .007; interaction of DAS change by Time: r = -.56, β = -.07, 95% CI [-.04, -.10], t(43) = -4.45, p < .0001).

Table 4.

Parameter Estimates (and Standard Errors) for Mixed Models Examining Residual Cognitive Change Scores in Relation to Depressive Symptom Change

Dependent Variable	Independent Variable	Model 1 WOR as IV	Model 2 IAT as IV	Model 3 DAS as IV
HRSD	Intercept	19.70 (.73)***	20.18 (.74)***	19.99 (.73)***
	WOR	.08 (.06)
	IAT		−2.04 (4.39)
	DAS			.03 (.03)
	Time	−.68 (.06)***	−.70 (.06)***	−.69 (.06)***
	WOR × Time	−.01 (.01)*+
	IAT × Time		.58 (.37)
	DAS × Time			−.01 (.003)**+
BDI-II
	Intercept	27.05 (1.34)***	27.61 (1.33)***	27.23 (1.25)***
	WOR	.18 (.11)
	IAT		−12.54 (7.83)
	DAS			.12 (.05)*
	Time	−4.32 (.44)***	−4.37 (.41)***	−4.33 (.37)***
	WOR × Time	−.07 (.04)*+
	IAT × Time		6.84 (2.43)**+
	DAS × Time			−.07 (.02)***+

Note. HRSD = Hamilton Rating Scale for Depression, BDI-II = Beck Depression Inventory – 2^nd Edition, WOR = Ways of Responding Scale, IAT = valued-Implicit Association Test, DAS = Dysfunctional Attitude Scale, IV = independent variable.

^*p < .05

^**p < .01

^***p < .0001

⁺significant after Holm-Bonferroni correction.

It is worth noting that results were consistent when cognitive change was measured using difference scores (calculated by subtracting intake scores from post-treatment scores) rather than residual change scores as reported above. For analyses of the HRSD: interaction of WOR difference by Time: r = -.44, β = -.02, 95% CI [-.02, -.004], t(41) = -3.10, p = .004; interaction of IAT difference by Time: r = .20, β = .48, 95% CI [-.24, 1.19], t(44) = 1.35, p = .19; interaction of DAS difference by Time: r = -.44, β = -.007, 95% CI [-.01, -.003], t(43) = -3.25, p = .002. For analyses of the BDI-II: interaction of WOR difference by Time: r = -.49, β = -.09, 95% CI [-.14, -.04], t(41) = -3.63, p = .0008; interaction of IAT difference by Time: r = .40, β = 6.49, 95% CI [1.87, 11.12], t(43) = 2.83, p = .007; interaction of DAS difference by Time: r = -.67, β = -.07, 95% CI [-.10, -.05], t(43) = -5.98, p < .0001. Thus, cognitive improvement on the WOR and DAS were each related to a larger magnitude of depressive symptom change on both the HRSD and BDI-II. However, contrary to what was expected, the significant interaction of valued-IAT change by Time reflects that improvement on the valued-IAT was related to a smaller magnitude of depressive symptom change on the BDI-II.

Discussion

Both the acquisition of CT skills and the modification of underlying maladaptive beliefs have been proposed as potentially critical forms of cognitive change for patients who successfully respond to CT for depression. In this study, depressed patients showed substantial improvements in coping skills as measured by the WOR over the course of treatment, with patients’ post-treatment skill levels being comparable to those of never-depressed participants. Improvements in CT skills as assessed by the WOR were related to a larger degree of change in depressive symptoms. With regard to maladaptive belief change, while depressed patients showed significantly more negative associations on the valued-IAT than never depressed participants, we failed to find evidence of change on this measure over the course of treatment. Individual differences in the degree of improvement (or worsening) of maladaptive beliefs as assessed by the valued-IAT were not related to change in depressive symptoms as expected. Instead, in one of two tests, patients who exhibited more improvement in valued-IAT scores over time showed smaller degrees of symptom improvement (on the BDI-II only). Taken together, these results suggest the acquisition of CT skills is related to the course of symptom improvement among patients participating in CT. In contrast, change in implicitly-assessed maladaptive beliefs failed to exhibit the expected association with symptom improvement.

Although we were primarily focused on the WOR and valued-IAT as measures of CT skills and maladaptive beliefs, respectively, we also examined the DAS. We found that while depressed patients exhibited moderate change in DAS scores, at the end of treatment patients continued to endorse more self-reported maladaptive beliefs on the DAS compared to never-depressed participants. We also found that DAS improvements were related to a larger degree of depressive symptom change, consistent with the WOR but not the valued-IAT. However, it remains a possibility that DAS scores might reflect either underlying beliefs or CT skills or both.

We found evidence that depressed patients showed significantly less adaptive scores on all three cognitive measures as compared with never-depressed participants. Specifically, we found that prior to treatment, depressed patients endorsed poorer CT skills on the WOR, less adaptive beliefs related to being valued on the valued-IAT, and greater self-reported maladaptive beliefs on the DAS. Our findings for the WOR and DAS are consistent with previous research showing significant correlations between depressive symptoms and these measures (Barber & DeRubeis, 1992; Weissman, 1979, respectively). The significant difference between groups on the valued-IAT provides evidence for the validity of this IAT as a measure of depression-related beliefs. While we did not highlight this in the results, the average valued-IAT scores for both groups were positive, indicating a tendency to believe that one is valued by important others; however, depressed patients endorsed this belief significantly less than never-depressed participants.

As we noted earlier, the valued-IAT we used differed from previous IATs used to study CBT for anxiety disorders in a number of ways (i.e., Gamer et al., 2008; Teachman et al., 2008). The most notable difference was the nature of the beliefs assessed. We focused our efforts on assessing patients’ beliefs that their worth depends on social approval—a key factor of the DAS and a maladaptive belief likely to be addressed in CT. While we designed our IAT for this specific purpose, our labels (“valued by my associates” versus “not valued by my associates”) were similar to those used by De Raedt and colleagues for their self-worth IAT (i.e., “worth” vs. “worthless”; De Raedt, Schacht, Franck, & De Houwer, 2006). Thus, our IAT shared key features with IATs used in previous research on depression. In contrast, Gamer and Teachman used “anxious” or “panicked” versus “calm” as the attribute categories, thus making their IAT a measure of the degree to which patients viewed themselves as associated with anxiety. If one were to design an IAT to assess the content of panic disorder-relevant beliefs along the lines we did for depression, the IAT might assess the association between physical symptoms and threat. Our key point is that the type of maladaptive beliefs we endeavored to assess were those central to the maladaptive belief content associated with depression rather than beliefs about whether one currently has symptoms of a disorder. In comparison to beliefs about having symptoms, we suspect that beliefs about one's value involve longer-standing patterns of thinking that are likely to be more difficult to change.

When we examined cognitive changes over the course of CT, we found significant improvement on the WOR and DAS. In contrast, we failed to find evidence of improvements on the valued-IAT. Changes on the WOR and DAS over the course of CT have consistently been reported in the literature, and the effect sizes found in this study (WOR: d = 1.53; DAS: d = .51) are comparable to those reported by Barber and DeRubeis (2001) following 12 weeks of CT (d = .70, d = .46, respectively). Our results suggest that depression-relevant maladaptive beliefs assessed by the valued-IAT endured even as patients participated in (and benefited from) CT.

Our results highlight the potential importance of acquiring CT skills to achieve a good therapeutic outcome in CT for depression. Whereas improvement on the WOR predicted a larger degree of depressive symptom change, improvement on the valued-IAT predicted a smaller degree of depressive symptom change (as measured by the BDI-II specifically). Because this latter finding was unexpected and only evident when depressive symptoms were measured using the BDI-II but not the HRSD, we would encourage caution in interpreting this finding until it is replicated. Nonetheless, our results regarding CT skills were clear and consistent across both measures of depressive symptoms.

We note several limitations of this study. First, the WOR and valued-IAT each represent just one measure of CT skills and underlying beliefs. Other measures could yield different results. For the valued-IAT, it is important to note that there are multiple ways in which an implicit measure can be constructed and multiple domains of depressive beliefs one could assess (e.g., beliefs in the need for perfection). The internal consistency of our IAT at intake among depressed patients was lower than desirable. Beyond this, IATs are limited as assessments of beliefs insofar as they do not directly assess these beliefs but rather assess the strength of associations (Rothermund & Wentura, 2004). Second, the observed cognitive changes may have been due to the passage of time, the concurrent use of antidepressant medication, or common factors present in all psychotherapies. Only with a randomized trial including a psychotherapy control condition and excluding medication usage could we establish the extent to which the changes observed might be due to CT specifically. Third, the rate of dropout (34%) in the current study was somewhat higher than expected (Swift & Greenberg, 2012), and patients who completed treatment differed in ethnicity from non-completers. Our analyses of patients who completed treatment do not address the nature of cognitive changes among patients who discontinued treatment. Fourth, our sample size was small and powered to detect only moderate to large effects. Fifth, the length of treatment was limited to 16 weeks and treatment was provided by advanced graduate students. It is possible that greater evidence of belief change (and skill acquisition) would be obtained with a longer course of CT or more expert, highly competent therapists. While we are not aware of research examining the relation of therapist competence and cognitive change, therapist competence has been found to predict therapeutic outcomes (Strunk, Brotman, DeRubeis, & Hollon, 2010). Finally, this study did not examine risk of relapse or recurrence following treatment. Additional research is needed to examine the relation of CT skills and belief change with risk of relapse and recurrence.

Conclusion

Our results indicated that among patients participating in CT, large improvements in CT skills were significantly related to symptom improvement. When assessing maladaptive beliefs using the valued-IAT, we failed to find evidence of belief change or that individual differences in belief change were associated with positive therapeutic gains. Our results highlight the importance of CT skills as cognitive factors that change and are associated with positive therapeutic outcomes. Future research is needed to replicate these findings and extend them by examining how cognitive changes involving effortful skill use and underlying beliefs might exhibit more complex reciprocal relations over time.