Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Dec 1.
Published in final edited form as: J Consult Clin Psychol. 2012 Apr 2;80(3):373–381. doi: 10.1037/a0027663

Predictors of Patient Cognitive Therapy Skills and Symptom Change in Two Randomized Clinical Trials: The Role of Therapist Adherence and the Therapeutic Alliance

Christian A Webb 1, Robert J DeRubeis 2, Sona Dimidjian 3, Steven D Hollon 4, Jay D Amsterdam 5, Richard C Shelton 6
PMCID: PMC4250039  NIHMSID: NIHMS643997  PMID: 22468907

Abstract

Objective

Previous research has found that therapist adherence to concrete, problem-focused cognitive therapy (CT) techniques predicts depressive symptom change (e.g., Feeley, DeRubeis, & Gelfand, 1999). More recently, Strunk, DeRubeis, Chui, and Alvarez (2007) demonstrated that in-session evidence of patients’ use of CT skills was related to a lower rate of relapse in the year following CT for depression. The current investigation attempts to integrate and extend these findings within 2 separate samples of patients and therapists.

Method

Drawing from the CT samples (N = 105, mean age = 40 years, female = 62%, White = 82%) of 2 published randomized clinical trials of depression treatment, we conducted analyses to examine whether therapist adherence to concrete CT techniques (Collaborative Study Psychotherapy Rating Scale) and the quality of the therapeutic alliance (Working Alliance Inventory) predict patients’ use of CT skills (Performance of Cognitive Therapy Strategies) and subsequent Beck Depression Inventory symptom change.

Results

Results indicated a differential pattern of prediction in the 2 samples. In one, CT techniques exhibited a stronger association with patient CT skills and symptom change than did the alliance, whereas the reverse pattern emerged in the second sample. A baseline symptom severity × CT techniques interaction indicated that between-study differences in intake depression severity might in part explain the process– outcome differences.

Conclusions

The present findings suggest that the nature of the therapy sample examined may moderate process–outcome findings in psychotherapy research. The implications of these results and directions for future research are discussed.

Keywords: cognitive therapy, depression, adherence, patient skills

In the last three decades numerous studies have examined the efficacy of cognitive therapy (CT) in the treatment of clinical depression (see DeRubeis, Webb, Tang, & Beck, 2010, for a review). CT has been shown to be an effective treatment, with reductions in depressive symptoms of similar magnitude to those observed in pharmacotherapy for depression. Relatively less is known regarding the process through which CT exerts its beneficial effects. In two previous process– outcome studies, DeRubeis and colleagues (DeRubeis & Feeley, 1990; Feeley, DeRubeis, & Gelfand, 1999) have found that variability in therapist adherence early in treatment to concrete problem-focused CT techniques predicted subsequent symptom change. In contrast, a factor reflecting more abstract, less focused CT methods (e.g., engaging in a discussion about the relationship between thoughts and feelings) was not found to be significantly associated with symptom change in those investigations. A broad array of therapist procedures are prescribed in CT and assessed by measures of therapist adherence to CT. It is likely that these interventions differ in the extent to which they are therapeutically beneficial to patients with depression. The abstract procedures of CT may be very important, but the treatment places a premium on the concrete, active, and focused interventions of CT (e.g., helping patients to identify and challenge specific depressogenic cognitions). The latter set of procedures may be particularly beneficial, especially early in treatment. It may be that the common use of the overall mean of therapist adherence scales (assessing therapist adherence to the entire treatment package), rather than the examination of particular techniques or subsets of techniques, in part accounts for the inconsistent findings in previous psychotherapy process research examining adherence–outcome associations (see Webb, DeRubeis & Barber, 2010, for a review).

In contrast to the significant adherence–outcome findings reported by DeRubeis and his colleagues, very different findings were obtained by Castonguay, Goldfried, Wiser, Raue, and Hayes (1996) in examining the relation between several process variables and outcome in depressed patients treated with CT. They assessed one aspect of therapist adherence to CT, referred to as “intrapersonal consequences.” This measure is designed to assess the extent to which a cognitive therapist “highlights the cause and effect between two components of the client’s functioning, such as the impact of distorted cognitions on depressive affect …” (Castonguay et al., 1996, p. 499). Across most analyses, this measure was found to be unrelated to patient outcome. In the one analysis in which significant findings emerged, higher scores on this adherence measure were associated with higher post-treatment scores on the Hamilton Rating Scale for Depression (HRSD; Hamilton, 1960).

In addition, the therapeutic alliance has received a great deal of attention within the psychotherapy literature. The Working Alliance Inventory (WAI; Horvath & Greenberg, 1986, 1989) is perhaps the most frequently used alliance measure (Martin, Garske, & Davis, 2000). Meta-analytic reviews of alliance– outcome studies indicate that a stronger alliance is associated with better treatment outcomes. For example, in their recent meta-analysis, Horvath, Del Re, Flückiger, and Symonds (2011) reported a mean alliance–outcome correlation of .28. These findings suggest that the alliance may be one factor that contributes to symptom improvement in psychotherapy, including CT. However, it is important to note that in very few of these studies have the investigators controlled for temporal confounds. Among studies that have controlled for these confounds, alliance– outcome findings have been inconsistent (Barber, 2009). It should be noted that in a recent study of CT controlling for temporal confounds, Webb et al. (2011b) found that the alliance, assessed early in treatment, did significantly predict subsequent depressive symptom improvement. The latter study used a relatively large sample (n = 100), as it was based on the pooled data sets of the two clinical trials examined in the current investigation (DeRubeis et al., 2005; Dimidjian et al., 2006).

Although the association between the therapeutic alliance and symptom change has been examined in numerous studies, there have been relatively few tests of the clinical intuition that therapist techniques are more therapeutically beneficial in the context of a strong, relative to weak, therapeutic alliance (see Barber, 2009; Webb et al., 2010). When interactions between the alliance and techniques have been examined, the sample sizes employed have typically been too small to provide a powerful statistical test.

In contrast to research on therapist adherence to CT procedures and, in particular, the alliance, relatively little research has focused on patients’ acquisition and use of the central cognitive skills (e.g., the ability to identify and challenge depressogenic thoughts) and behavioral skills (e.g., the ability to schedule and structure activities) that are encouraged in CT. A core aim of CT is to promote the development of a particular set of skills and understandings that are hypothesized to alleviate distressing symptoms and prevent their recurrence following successful treatment (Barber & DeRubeis, 1989; Strunk, DeRubeis, Chui, & Alvarez, 2007). One of the first approaches to investigating patient competence in CT skills was initiated by Barber and DeRubeis (1992) through their development of the Ways of Responding (WOR) questionnaire. Designed to assess the extent to which patients have developed the compensatory or metacognitive skills taught in CT, the WOR presents patients with stressful scenarios, followed by initial negative thoughts to which they are asked to respond. Barber and DeRubeis (2001) have shown that WOR scores improve significantly over the course of CT and that these changes are associated with decreases in depressive symptoms. Although the WOR was designed to assess competence in the skills patients are meant to learn in CT, it does not address questions about the extent to which patients apply these skills in their daily lives. To fill this void, Strunk et al. (2007) developed the Performance of CT Strategies (PCTS), a measure applied by observers to recordings of therapy sessions, to assess the extent to which patients exhibit in session the cognitive and behavioral skills taught in CT or report having used these skills between sessions. Strunk et al. showed that among the CT patients who responded to treatment in the Cognitive Pharmachotherapy–II study (CPT-II; DeRubeis et al., 2005), higher scores on the PCTS near the end of treatment were significantly associated with a lower risk of relapse in the subsequent year. The Strunk et al. findings raise the following key questions: If patients in CT acquire and make use of skills that predict better outcomes, what processes contribute to the development of these skills? In particular, do specific therapist behaviors, such as therapist adherence to concrete CT techniques, predict the acquisition of such skills? In addition, given the substantial interest in the relation between the alliance and better treatment outcomes (Horvath et al., 2011), does the alliance predict patient use of CT skills?

There are several goals to the current study. First, we examine whether therapist adherence to concrete CT techniques (CT-Concrete factor) and the quality of the therapeutic alliance (WAI) predict patient use of CT skills (PCTS) in the CT condition of the CPT-II study. Second, to the extent that significant findings emerge, and in an effort to replicate these findings, we examine whether the CT-Concrete factor and the WAI predict the PCTS in a sample of patients from the CT condition of a trial recently conducted at the University of Washington (UW study; Dimidjian et al., 2006). Third, because the CT-Concrete factor and the alliance have both been examined in relation to symptom change in previous studies, these variables are examined in relation to depressive symptom improvement in both the CPT-II and UW samples. The majority of previous process– outcome studies have failed to control for temporal confounds (Webb et al., 2010). In the current study, a potential temporal confound is addressed by statistically controlling for symptom change that precedes the assessment of the predictor variables.

The UW study has generated discussion in the psychotherapy literature in regard to the relatively poor performance of the CT condition in comparison to the antidepressant medication and behavioral activation conditions (Coffman, Martell, Dimidjian, Gallop, & Hollon, 2007). In contrast to the CPT-II study, which, according to the classification delineated in the Handbook of Psychiatric Measures (American Psychiatric Association, 2000), included only patients with “severe” or “very severe” levels of depressive symptoms (HRSD ≥ 20), the UW study included patients with lower levels of depressive symptoms (HRSD ≥ 14; i.e., “moderate” or higher). Recent evidence indicates that, among patients entering treatment with lower depression severity, there are at most minor differences in symptom change between active treatments and placebo, both in pharmacotherapy (Fournier et al., 2010; Kirsch et al., 2008) and psychotherapy (Driessen, Cuijpers, Hollon, & Dekker, 2010) outcome studies. These findings suggest that, among patients with lower levels of depressive symptoms, the bulk of symptom change in pharmacotherapy and psychotherapy may be attributable to factors implicated in the placebo response (e.g., response expectancies, installation of hope, spontaneous remission), rather than to the putative active ingredients of anti-depressant medications or the theory-specified techniques of psychotherapy (in the case of CT, the core cognitive and behavioral methods emphasized in treatment; A. T. Beck, Rush, Shaw, & Emery, 1979). To the extent that this is the case, ratings of therapist use of cognitive and behavioral techniques may yield relatively small correlations with depressive symptom change among patients beginning CT with lower depression severity. To our knowledge, no study has examined whether the severity of symptoms at the beginning of treatment moderates process–outcome findings. Consequently, the inclusion of both the CPT-II and UW samples allows for the examination of whether depressive symptom severity at intake influences process–outcome associations.

In summary, we hypothesized that, within the context of a design that controls for temporal confounds, therapist adherence to concrete CT techniques would be a stronger predictor of depressive symptom improvement and patient CT skills in the CPT-II than the UW sample. Second, given the relatively broad range of depression severity represented in the UW sample, we hypothesized that therapist adherence to concrete CT techniques would be more strongly associated with symptom change among the more severely depressed UW patients (i.e., those with depressive symptom severity levels as high as the patients included in the CPT-II trial).

Finally, the current study allows for tests of interactions between alliance and therapist technique in predicting both patient CT skills and symptom change. Combining the CT conditions of two large randomized clinical trials afforded a relatively powerful test of Alliance × Technique interactions, or the notion that therapist techniques are more therapeutically beneficial in the presence of a strong therapist–patient alliance. We hypothesized that a significant Alliance × Technique interaction would emerge in the combined (CPT-II and UW), but not the individual, samples.

Method

Participants

Patients

Patients from the CT conditions of the CPT-II (N = 60) and UW (N = 45) studies were included.1 Both studies targeted adults with major depression, and CT was provided for a total of 16 weeks. Although the CT samples and procedures were similar in most respects, there were a few noteworthy differences. In the CPT-II trial, prior to intake patients had to receive a score of 20 or higher on the modified 17-item HRSD (Hamilton, 1960) at both the screen and baseline assessments, whereas in the UW trial, patients had to receive 14 or higher on the 17-item HRSD (Hamilton, 1960) and 20 or higher on the Beck Depression Inventory–II (BDI-II; A. T. Beck, Steer, & Brown, 1996) at a single intake assessment. (For more detailed information on each of the studies, see DeRubeis et al., 2005, CPT-II; Dimidjian et al., 2006, UW.) Local institutional review board (IRB) approval was obtained for all sites, and all patients provided written informed consent. IRB approval for the Seattle study inadvertently lapsed for approximately six weeks at the time of Principal Investigator Neil S. Jacobson’s death; approval for use and publication of data collected during that time period was subsequently granted by the IRB.

Therapists

In the CPT-II study, four male and two female clinicians (three therapists at each site) served as cognitive therapists. Five of the therapists were licensed PhD psychologists, and one was a psychiatric nurse practitioner (MSN). Four of the therapists had extensive CT experience (7–21 years) prior to the initiation of the study. Two of the therapists started the study with 2 years of CT experience and received additional training from the Beck Institute for Cognitive Therapy during the trial.

In the UW study, three licensed psychologists, with an average of 14 years of experience in clinical practice, served as cognitive therapists. Two of them were highly experienced cognitive therapists, had served as cognitive therapists in previous trials, and had received training at the Beck Institute. The third therapist had received training in CT for anxiety disorders. Each of the three therapists had acquired certification by the Academy of Cognitive Therapy during the course of the study. All therapists in both studies followed the procedures outlined in two standard texts of CT for depression (i.e., A. T. Beck et al., 1979; J. S. Beck, 1995).

Measures

Collaborative Study Psychotherapy Rating Scale (CSPRS; Hollon et al., 1988)

The CSPRS was designed to differentiate CT, interpersonal psychotherapy (Klerman, Rounsaville, Chevron, Neu, & Weissman, 1979), and pharmacotherapy management (Fawcett & Epstein, 1980) based on therapist behaviors observed in taped recordings of sessions. It has also been used to rate the extent to which therapists within a given a treatment modality exhibit behaviors of interest (e.g., Feeley et al., 1999). The CSPRS is considered a measure of therapist adherence, in that it assesses the extent to which therapists are delivering the prescribed procedures of a given treatment modality. It is not intended to be a measure of therapist competence, the skill with which these procedures are implemented. In the current study, we utilized the CT-Concrete factor (DeRubeis & Feeley, 1990) from the CSPRS. Comprising 10 of the 28 CBT subscale items, it assesses the therapist’s use of concrete, problem-focused CT techniques. The 10 CT-Concrete factor items assess the following therapist behaviors: set and followed agenda (1), reviewed homework (2), asked for specific examples of beliefs (21), asked patient to report cognitions verbatim (57), labeled cognitive errors (59), examined evidence concerning beliefs (62), practiced rational responses with patient (69), assigned homework (72), assigned/reviewed self-monitoring (75), and asked patient to record thoughts (76). All items are rated on a 7-point Likert-type scale. Higher scores indicate that a therapist engaged in a behavior more extensively or more thoroughly in the rated session. Previous research indicates that the CT-Concrete subscale can be rated relatively reliably (DeRubeis & Feeley, 1990; Feeley et al., 1999), with intraclass correlation coefficients (ICCs) ranging from .63 to .75.

WAI–short observer-rated version (WAI-O-S; Tracey & Kokotovic, 1989; Tichenor & Hill, 1989)

The WAI-O-S is a 12-item observer-rated measure designed to assess the quality of the therapeutic alliance. The widely used WAI scales are based on Bordin’s (1979) conceptualization of the working alliance as consisting of three components: (a) the bond between the therapist and patient, (b) agreement about the goals of therapy, and (c) agreement about the tasks. Correspondingly, the WAI-O-S comprises three subscales (Bond, Goal, Task), each with four items. Items are rated on a 7-point scale (0 = never to 6 = always). Previous research suggests that the WAI-O-S, referred to henceforth as the WAI, can be rated reliably. For example, with regard to interrater reliability, Andrusyna, Tang, DeRubeis, and Luborsky (2001) reported a Pearson correlation coefficient of .67 for the WAI, and Strunk, Brotman, and DeRubeis (2010) reported an ICC of .79 for the latter variable. As described in Footnote 1, Strunk et al., using a different set of raters who were trained independently from the raters in the current study, obtained ratings of the WAI based on a subset of the patients included in our study (only the CPT-II patients). The correlation between WAI ratings for the subset of patients (CPT-II) investigated in Strunk et al. and the WAI ratings in the current study was high (r = .67).

PCTS (Strunk et al., 2007)

The PCTS is a 16-item observer-rated assessment, gleaned from recordings of CT sessions, of the extent to which patients demonstrate their understanding and use of CT skills, either during the session or by reporting that they had employed CT skills between sessions. The PCTS items are grouped into subscales corresponding to three CT components: behavioral activation (3 items), automatic thoughts (11 items), and schemas (2 items; for a list of the PCTS items, please see the Appendix of Strunk et al., 2007). Items are rated on a Likert-type scale ranging from 0 to 6, with scores of 0 reflecting a patient’s reluctance or resistance to a given domain. For example, Item 6 (Relating Thoughts and Feelings) reads “Did the patient understand the relationship between his or her thoughts and feelings?” During the course of the session, if a patient appears to disagree with the idea that thoughts and feelings are related, he or she would receive a score of 0 on this item. Scores from 1 through 6 indicate, depending on the item in question, greater understanding/ability and, at higher scores, increasingly independent application of the material learned in a given domain. Strunk et al. (2007) reported an ICC of .85 for the overall score that combines the three domains. Strunk et al. rated 49 patients from the CT condition of the CPT-II study using the PCTS. The correlation between Strunk et al.’s PCTS ratings and those in the current study was .45. It should be noted that unlike the PCTS ratings from the current study, which were based on the third from last CT session, the Strunk et al. ratings were based on raters observing three consecutive sessions late in treatment.

BDI-II (Beck et al., 1996)

The BDI-II, a 21-item self-report measure of depressive symptoms, was administered to patients in both the CPT-II and UW studies prior to each session. It has exhibited excellent psychometric properties in prior work (Beck et al., 1996).

Procedures

Therapy sessions in both studies were video- and audiotaped. The third session (“early session”) and the third from last session (“late session”) were rated for each therapist–patient dyad. The CSPRS’s CT-Concrete subscale was rated at the early session. Early assessments (e.g., Sessions 2, 3, or 4) of therapist adherence are common in studies investigating adherence– outcome relations (see Webb et al., 2010, for a review). Moreover, DeRubeis and Feeley (1990) and Feeley et al. (1999) found, in two separate studies of CT for depression, that therapists were delivering at least as much CT-Concrete technique early in treatment (Session 2) as in several randomly selected sessions later in treatment. This may be due in part to the fact that in randomized clinical trials involving CT, it is common for comprehensive clinical assessments to have been completed prior to the first therapy session, freeing up the cognitive therapists to begin the active and structured work of CT very early in treatment. Moreover, the assessment of therapy processes early in treatment, before much symptom change has occurred, will tend to maximize subsequent variability in symptom change. As was the case with therapist adherence to CT-Concrete techniques, the WAI was assessed at the early session.

The PCTS was rated at the late session to allow time for patients to have acquired CT skills. Raters took detailed notes while viewing a full session, after which they assigned ratings on the PCTS items. If the requisite videotape was damaged or unavailable, a videotape from an adjacent session was substituted. If these recordings were damaged or unavailable, audiotapes were used. Two patients were not included because they dropped out of treatment prior to Session 2. All other dropouts (9 CPT-II patients and 5 UW patients) left treatment between Session 2 and the third from last session; these patients were included in analyses of early, but not late, sessions. In addition, recordings of early sessions for three patients and late sessions for two patients were unavailable from the UW study. In total, 100 early sessions and 87 late sessions were rated.

Random numbers were assigned to the tapes, from which all other identifying information was removed. As in previous studies examining therapist adherence to CT-Concrete techniques (DeRubeis & Feeley, 1990; Feeley et al., 1999) and the one previous published study of the PCTS (Strunk et al., 2007), five undergraduate psychology majors from the University of Pennsylvania served as raters. Each tape was coded by two raters independently according to a balanced incomplete block design (Fleiss, 1981), such that each rater was paired with each of the other four raters an equal number of times. Raters were assigned no more than one session tape per therapist–patient dyad. Hollon et al. (1988) demonstrated that two raters per tape were sufficient to achieve adequate reliability on the CSPRS, as did Strunk et al. (2007) for the PCTS. Raters were blind to treatment outcome, site, and the study aims, as well as to subject and session numbers. Raters read J. S. Beck’s (1995) Cognitive Therapy: Basics and Beyond, reviewed all available manuals for the rating scales, and received approximately thirty hours of training prior to the initiation of the study. In addition, in order to reduce rater drift, raters met approximately once each week with the study supervisor (CAW) to rate a tape independently and to discuss any discrepancies in ratings.

Data Analytic Strategy

Scores on the measures of patient CT skills (PCTS) and depressive symptom improvement (BDI) were utilized as dependent variables. First, multiple regression analyses were performed within both the CPT-II and UW samples to examine the prediction of the PCTS assessed at the third from last session from the WAI and the CT-Concrete factor, both assessed at the third session. To be consistent with the analyses below (see Prediction of Depressive Symptom Change section), we used as a covariate the BDI score obtained at the start of the session in which the CT-Concrete factor and the WAI were assessed (i.e., Session 3) in all analyses.

Similarly, multiple regression analyses were performed within each of the two samples to examine the prediction of BDI symptom change from the CT-Concrete and the WAI. Consistent with Feeley et al. (1999), a subsequent change score (i.e., from Session 3 onward) was computed for each patient by calculating the difference between the Session 3 BDI score (i.e., the time at which the CT-Concrete and WAI variables were assessed) and the end of treatment (i.e., Week 16) BDI score, adjusting for Session 3 BDI. Following DeRubeis et al. (2005) and Dimidjian et al. (2006), we used the last observation carried forward for patients who dropped out or failed to complete the final assessment (see Procedures).

Following the recommendation of Aiken and West (1991), prior to computation of cross-products for inclusion in tests of interaction effects (predicting, depending on the analysis, either BDI subsequent symptom change or PCTS scores), relevant variables were mean-centered. In all cases, positive partial correlations (pr) indicate that higher ratings on the given process variable were associated with greater symptom reduction. In addition, unstandardized regression coefficients (B) and 95% confidence intervals (CIs) for coefficients are reported. One univariate outlier was detected for the WAI (Z = −3.92). This score was replaced with the next most extreme (non-outlier) value in the data set for this variable. We performed all analyses with SAS Version 9.2 PROC GLM and PROC REG.

Results

Interrater Reliability

ICCs were estimated for the PCTS, the WAI, and the CT-Concrete subscale of the CSPRS, using a random effects model, for the mean ratings of two raters (Shrout & Fleiss, 1979). The ICCs, .71 for the PCTS and .70 for CT-Concrete, were similar to those reported in previous studies (e.g., DeRubeis & Feeley, 1990; Feeley et al., 1999; Strunk et al., 2007). Similarly, the ICC for the WAI (.73) was in a range of interrater reliability values similar to those reported in previous studies using observer-rated alliance scales (e.g., Hanson, Curry, & Bandalos, 2002; Strunk et al., 2010). When examined separately by study, ICCs for the primary investigated variables in the CPT-II study were .80 for the WAI, .77 for CT-Concrete, and .70 for the PCTS; whereas in the UW study ICCs were .63 for the WAI, .51 for CT-Concrete, and .71 for the PCTS. Means, standard deviations, and correlations for the investigated variables are listed in Table 1.

Table 1.

Means, Standard Deviations, and Correlations for Process Variables for Combined Sample

Variable M SD 1 2 3
1. CT-Concrete 2.83 0.48 — .43** .23*
2. WAI 4.38 0.46 — .40**
3. PCTS 1.77 0.36 —
Open in a new tab

Note. N = 100 for CT-Concrete and WAI; N = 87 for PCTS; CT-Concrete ≤ Cognitive Therapy–Concrete subscale (assessed at the third CT session); WAI = Working Alliance Inventory (assessed at the third CT session); PCTS = Performance of Cognitive Therapy Strategies (assessed at the third from last CT session).

*

p < .05.

**

p < .01.

Prediction of Patient Use of CT Skills

Analyses for the CPT-II sample

A comprehensive model was examined in which the CT-Concrete factor and the WAI, as well as a term representing their interaction, were included as simultaneous predictors. The overall model was significant, F(4, 45) = 3.05, p = .026, with an adjusted R2 of .14. The CT-Concrete factor was a significant predictor of PCTS scores (B = 0.21, 95% confidence interval [CI] = 0.052, 0.362; p = .010; pr = .37). The association between the WAI and the PCTS was not significant (B = 0.05, 95% CI = −0.151, 0.246; p = .632; pr = .07). The CT-Concrete factor × WAI interaction term was not significant (B = 0.00, 95% CI = −0.267, 0.270; p = .991; pr = .00).

Replication with UW study sample

The comprehensive model for the UW sample, in which CT-Concrete and the WAI variables were predictors, as was a term representing their interaction, was significant, F(4, 31) = 3.23, p = .025, with an adjusted R2 of .20. In contrast to the CPT-II sample analyses, in the UW sample the WAI was a significant predictor of PCTS scores (B = 0.48, 95% CI = 0.147, 0.816; p = .006; pr = .47), whereas CT-Concrete was not (B = −0.05, 95% CI = −0.442, 0.348; p = .811; pr = −.04). The CT-Concrete factor × WAI interaction term was not significant (B = −0.40, 95% CI = −1.239, 0.445; p = .344; pr = −.17). When we combined the CPT-II and UW samples to allow a more powerful statistical test, the CT-Concrete factor × WAI interaction remained nonsignificant (B = −0.08, 95% CI = −0.384, 0.234; p = .629; pr = −.05).2

Prediction of Depressive Symptom Change

Analyses for CPT-II sample

In a model that included both the CT-Concrete factor and the WAI, as well as a term representing their interaction, the overall model was significant, F(4, 54) = 4.56, p = .003, with an adjusted R2 of .20. Neither the CT-Concrete factor (B = 3.96, 95% CI = −1.433, 9.355; p = .147; pr = .20) nor the WAI (B = 1.69, 95% CI = −5.402, 8.790; p = .634; pr = .06) was significantly associated with symptom change in this analysis. The CT-Concrete factor × WAI interaction term was not significant (B = −0.69, 95% CI = −9.236, 7.860; p = .872; pr = −.02).

Replication with UW study sample

In a comprehensive model that included the CT-Concrete factor and the WAI, as well as a term representing their interaction, the overall model was significant, F(4, 36) = 2.80, p = .040, with an adjusted R2 of .15. In contrast to the CPT-II sample analyses, in the UW sample the WAI was significantly associated with subsequent symptom change (B = 8.57, 95% CI = 1.124, 16.014; p = .025; pr = .36), whereas the CT-Concrete factor was not (B = −1.57, 95% CI = −10.721, 7.582; p = .730; pr = −.06). The CT-Concrete × WAI interaction term was not significant (B = −8.05, 95% CI = −28.039, 11.936; p = .419; pr = −.13). To provide a relatively powerful test of the CT-Concrete factor × WAI interaction in comparison to previous process research, we combined the CPT-II and UW samples. Again, the interaction was not significant (B = −0.84, 95% CI = −8.276, 6.592; p = .823; pr = −.02). A power analysis indicated that with our sample size (N = 100) and at a power of 80% and an alpha of .05, we would have been able to detect an interaction that accounted for 6% (R2 = .058) or more of the variance in symptom change (or patient CT skills; SAS Power and Sample Size 3.1).3

Addressing Differences in Initial Depressive Symptom Severity

Twenty of the 45 UW patients entered treatment with HRSD scores below 20, whereas patients with scores below 20 were not included in the CPT-II study. Dimidjian et al. (2006) conducted separate analyses for the “low severity” (HRSD 14–19) and “high severity” subgroups (HRSD ≥ 20). To examine whether the process– outcome findings described above may be attributable at least in part to the inclusion of patients with lower levels of severity in the UW study, we tested intake HRSD × process variable (CT-Concrete or WAI) interactions in predicting BDI subsequent symptom change and PCTS scores in the UW study. Despite the relatively small UW sample size, a nonsignificant trend emerged for the intake HRSD × CT-Concrete factor interaction in predicting depressive symptom change (B = 2.17, 95% CI = −0.113, 4.447; p = .062; pr = .31). Although the latter interaction was only a nonsignificant trend, exploratory analyses were conducted to examine the direction of the effect; the CT-Concrete factor tended to be more strongly associated with symptom improvement among patients entering treatment with higher HRSD scores. The intake HRSD × WAI interaction was not significant in predicting symptom change (B = 1.59, 95% CI = −0.407, 3.596; p = .115; pr = .26). However, paralleling the pattern of findings for CT-Concrete, the association between the WAI and symptom change was in the direction that would suggest a stronger WAI–symptom change association among patients with higher intake HRSD scores. This pattern of findings may be due at least in part to the greater variability in BDI symptom change and WAI ratings in the high severity sample. (See Table 2 for means and standard deviations for independent variables and dependent variables in the UW study by intake depression severity subgroup.) Tests of these interactions in the prediction of the PCTS were not significant (ps > .27). However, as true for the pattern observed in the prediction of symptom change, the association between the independent variables and the PCTS were in the direction that suggests a stronger relation among patients with higher HRSD scores at intake.

Table 2.

Means (SDs) for All Variables for Each Site (by Dimidjian et al. 2006 Severity Grouping for UW)

Variable UW HRSD 14–19 UW HRSD > 19 CPT-II
CT-Concrete 2.81 (0.35) 2.75 (0.38) 2.87 (0.55)
WAI 4.41 (0.38) 4.35 (0.53) 4.38 (0.45)
PCTS 1.87 (0.45) 1.82 (0.44) 1.71 (0.27)
BDI-Change 7.88 (8.57) 11.52 (12.61) 11.58 (10.52)
Open in a new tab

Note. UW = University of Washington study; HRSD = Hamilton Rating Scale for Depression; CPT-II = Cognitive Pharmacotherapy–II study; CT-Concrete = Cognitive Therapy–Concrete subscale (assessed at the third CT session); WAI = Working Alliance Inventory (assessed at the third CT session); PCTS = Performance of Cognitive Therapy Strategies (assessed at the third from last CT session); BDI-Change = change in Beck Depression Inventory–II from Session 3 until treatment termination (or dropout).

Discussion

In the present investigation, therapist adherence to concrete CT techniques (CT-Concrete factor) and the therapeutic alliance (WAI) were examined as predictors of patient understanding and use of CT skills (PCTS) and depressive symptom improvement (BDI) in two independent samples. Different patterns of association were observed across the two investigated samples, with the CT-Concrete factor demonstrating a stronger association with outcome relative to the alliance in one sample, and the reverse pattern (i.e., alliance the stronger predictor) emerging in the second sample.

Our hypotheses were partially supported. In line with our hypotheses, among the CT patients in the CPT-II trial, the CT-Concrete factor was a significant predictor of patient CT skills, even after controlling for the influence of the therapeutic alliance. This is the first study to find support for the notion that therapist adherence to theory-specified CT techniques may contribute to patient acquisition and use of CT skills.

Contrary to our hypotheses, the CT-Concrete factor, although associated with symptom improvement at a level that was numerically higher than what was observed with the alliance (partial correlations of .20 and .06, respectively), was not significantly associated with symptom change when controlling for the influence of the alliance. These findings do not replicate those reported by DeRubeis and Feeley (1990) and Feeley et al. (1999). One of the strengths of the current investigation was that a potential temporal confound was addressed by statistically controlling for symptom change that preceded the assessment of the predictor variable. Temporal confounds were also controlled for in the analyses predicting patient use of CT skills. The quality of the therapeutic alliance was not a significant predictor of either symptom change or patient CT skills in the CPT-II sample.

Interestingly, the reverse pattern of findings emerged within the UW sample replication. The therapeutic alliance was a significant predictor of both symptom change and patient understanding and use of CT skills, whereas the CT-Concrete factor was not significantly related to either of these variables. This finding may reflect the importance of the alliance in contributing to symptom improvement in CT. Indeed, meta-analyses of alliance–outcome associations indicate that, on average, stronger therapeutic alliances are associated with better patient outcomes, across a variety of treatment modalities and mental health problems (Horvath et al., 2011). It is important to note that there was less variability and, perhaps as a result of such restricted variance, lower interrater reliability in CT-Concrete ratings in the UW study than the CPT-II sample. This may in part account for the nonsignificant CT-Concrete findings.

In the UW study, 22% of the CT patients exhibited “extreme nonresponse” (ENR), defined by a posttreatment BDI score greater than 30 (i.e., in the severe range of depressive symptoms; Coffman et al., 2007). In contrast, only 7% of the CPT-II sample exhibited ENR. The higher rates of ENR within the UW, relative to the CPT-II, CT sample may account in part for the differential pattern of WAI findings across the two study samples. Namely, two separate studies have found that ENR patients in the UW study, even early in CT, had significantly weaker therapeutic alliances (rated using the WAI) than did their non-ENR counterparts (Coffman et al., 2007; Webb et al., 2011a). Thus, the ENR patients had both significantly lower WAI scores and particularly poor treatment outcomes, which may help account for the finding of a significant alliance– outcome association for the UW sample in the current study. Consistent with this hypothesis, when the 10 ENR patients were eliminated from the UW sample, the alliance–outcome association in the UW study was markedly reduced (from β = .35, p = .025 to β = .12, p = .293; Webb et al., 2011a).

One of the goals in the current study was to examine whether severity of depression moderated adherence– outcome associations. The fact that the symptom (HRSD) severity levels at intake for nearly half the patients in the UW study were lower than those for any of the CPT-II patients may also help account for the pattern of findings we obtained. Given the evidence indicating that, among less severely depressed patients, there are generally only small, if any, differences in outcome between active treatments and placebo conditions, in studies of psychotherapy (Driessen et al., 2010) and of pharmacotherapy (e.g., Fournier et al., 2010; Kirsch et al., 2008), a relatively large proportion of the symptom improvement experienced by the CT patients in the UW study may have been due to factors associated with placebo response. Indeed, among the low severity UW group (HRSD 14–19) there was no evidence that the active treatment conditions outperformed the placebo condition (Dimidjian et al., 2006). To the extent that this was the case, one would not expect therapist adherence to CT techniques to account for substantial variability in symptom improvement among the lower severity group. Consistent with this claim, and in partial support of our hypothesis, a baseline symptom severity × CT techniques interaction, although only at the level of a nonsignificant trend, suggested that adherence to CT-Concrete techniques tended to be more strongly associated with symptom improvement among those UW patients who entered treatment with higher levels of depressive symptoms than those who began treatment with milder symptoms.

Moreover, with respect to the higher severity UW patients (HRSD ≥ 20), and in contrast to the CPT-II study findings, the antidepressant medication and behavioral activation conditions both significantly outperformed the CT condition. Thus, it may be that the patients within the CT condition did not evidence significantly greater improvement than what may have been accounted for by placebo response. (There was no direct comparison of CT vs. placebo reported in Dimidjian et al., 2006.) To the extent that this was the case, one would not expect variation in CT techniques to account for substantial variability in symptom improvement.

These findings suggest that the nature of the sample may influence process–outcome findings. For example, response expectancies and spontaneous remission may tend to account for a larger proportion of symptom improvement in patients with lower levels of depressive symptoms than those with more severe clinical depression. Consequently, specific techniques, or the “active” ingredients of a treatment, may explain less variability in symptom change in populations of depressed patients with milder symptoms. Moreover, to the extent that there is less variability in symptom change at lower, relative to higher, intake levels of a depression severity, reduced variance may in part account for relatively weak process– outcome findings that emerge within low severity samples. Furthermore, although speculative, it may be that those disorders that are relatively less responsive to placebo (e.g., severe obsessive-compulsive disorder, psychotic disorders) tend to reveal larger associations between techniques and symptom change than do those that are more responsive to placebo (e.g., panic disorder, depression; Khan et al., 2005).

Of the numerous process–outcome studies in the literature, relatively few have tested and reported Alliance × Technique interactions in predicting symptom change (Barber, 2009; Webb et al., 2010). In addition, many of these studies employed relatively small samples and thus may have been underpowered to detect statistically significant interactions (e.g., Castonguay et al., 1996). The present study provided several opportunities to test Alliance × Technique interactions in predicting both symptom change and patients’ use of CT skills. However, none were significant, even when both the CPT-II and UW samples were combined, providing a relatively powerful test of Alliance × Technique interactions in comparison to previous process research. A power analysis indicated that with our sample size, it is likely that we would have detected an interaction that accounted for even a rather small portion of the variability in symptom change (or patient CT skills). In contrast to clinical intuition, it may be that patients are not more therapeutically responsive to therapist techniques within the context of a strong, relative to a weak, therapeutic alliance. It is important to note, however, that the nonsignificant interaction findings emerged within a specific context. We examined the interaction between concrete CT techniques (as assessed by the CT-Concrete factor of the CSPRS) and the alliance (as assessed by the WAI) at the third session of a specific form of psychotherapy (CT) in the treatment of one disorder (clinical depression). Alliance × Technique interactions may be more likely to emerge in other contexts (e.g., within other treatment modalities; for disorders other than depression). Moreover, given that the therapists in both studies were, by and large, highly experienced and that the delivery of therapy was closely monitored, there may have been a restriction in the range of adherence and alliance, possibly limiting our ability to detect interactions between these variables.

Several limitations of the present study should be noted. Although the “no omitted variables” assumption is fundamental to the concept of causal modeling, it is not possible to verify with observational data. Thus, in the current study, one or more unmeasured third variables could have exerted important influences, and we would not be able to detect these effects. For example, there may be particular patient characteristics that evoke CT-Concrete techniques from therapists and at the same time contribute to better treatment outcomes. In addition, the process variables examined in the current study were assessed only at Session 3 (CT-Concrete factor and WAI) and the third from last session (PCTS). It may be that a different pattern of results would have emerged if these variables were assessed at different time points. Moreover, we investigated a particular subset of CT interventions (i.e., concrete techniques) and may have excluded other interventions delivered by cognitive therapists that are particularly effective. Similarly, a different pattern of findings may have emerged if measures of the alliance other than the WAI had been included. The WAI, based on Bordin’s (1979) transtheoretical conceptualization of the alliance, necessarily assesses aspects of the therapist–patient alliance that are deemed applicable to a range of different forms of psychotherapy (e.g., an affective bond, agreement on treatment goals and tasks). It will be important for future research to examine the process– outcome associations tested in this study using other alliance measures, particularly those that take into account CT-specific elements of the therapist–patient relationship, such as the emphasis on “collaborative empiricism.” Finally, it is not known to what extent the dropouts from the two studies may have influenced the strength of the associations we reported.

Future Directions

The current study raises issues that should be addressed in future research. It will be important to investigate the relationships we examined in other samples of depressed patients treated with CT, as well as within other treatment modalities, and with other patient populations. In addition, a more comprehensive examination of process variables and depressive symptoms across additional time points would provide for a more complete and accurate picture of how these phenomena change and interact with symptom change over time. Moreover, future research would benefit from the inclusion of variables that were not assessed in the current study. For example, it may be important to measure the skill or appropriateness of the delivery of these techniques (i.e., competence) in addition to the extent to which therapists adhere to CT techniques (i.e., adherence). The results of such future investigations could help researchers identify variables that play particularly important roles in contributing to symptom improvement. Such findings could ultimately yield important clinical implications for the treatment of depression with CT.

Acknowledgments

This article is based in part on Christian A. Webb’s doctoral dissertation, which was supported by a Social Sciences and Humanities Research Council of Canada doctoral fellowship. The CPT-II trial (DeRubeis et al., 2005) was supported by National Institute of Mental Health (NIMH) Grants MH50129 (R10) (to Robert J. DeRubeis) and MH55875 (R10) and MH01697 (K02) (to Steven D. Hollon). The UW trial (Dimidjian et al., 2006) was supported by NIMH Grant MH55502 (R01) first to Neil S. Jacobson and, after his death, to David L. Dunner. GlaxoSmithKline provided medications and pill placebos for both trials. We thank Keith S. Dobson, David L. Dunner, Robert J. Kohlenberg, and Karen B. Schmaling for their leadership of the UW trial and for making data available for this project.

Footnotes

1

There is partial overlap between the current study and Strunk, Brotman, and DeRubeis (2010). Using repeated measures regression, Strunk et al. examined the relation between ratings of several process variables, including measures of adherence to CT (but not the CT-Concrete factor) and the WAI, at Sessions 1–4 and BDI ratings at session n + 1. The participants in the Strunk et al. study were drawn from the CT condition of Cognitive Pharmacotherapy–II (CPT-II) and thus overlap partially with the sample employed in the current analyses. However, there was no overlap of the ratings made or the raters between the two investigations.

2

PCTS scores were associated with depressive symptom improvement at the level of a nonsignificant trend in both the CPT-II study, β = 0.22, t(47) = 1.75, p = .086, and the UW study, β = 0.30, t(34) = 1.93, p = .062.

We also obtained the PCTS ratings from the one previous study that utilized this measure (Strunk et al., 2007). In order to examine whether the CT-Concrete factor and WAI ratings obtained in the current study predicted the Strunk et al. PCTS ratings, we conducted a multiple regression in which CT-Concrete techniques and the WAI, as well as a term representing their interaction, were included as predictors simultaneously, and the Strunk et al. PCTS variable was the criterion. The pattern of findings that emerged was the same as that reported in the Results section above (i.e., the CT-Concrete factor, but not the WAI, significantly predicted the PCTS).

3

This power analysis also assumes a reduced R2 (i.e., the reduced model excluding the interaction term) of .22. This value was chosen given that it is the value of the observed R2 in the analysis predicting subsequent BDI change from CT-Concrete and the WAI (excluding the term representing their interaction) in the combined (CPT-II and UW) sample.

In addition to conducting the analyses reported in the Results section, we tested curvilinear (i.e., quadratic) adherence– outcome effects, as well as the interaction between a quadratic adherence term and the alliance, in predicting both symptom change and the PCTS for both samples, as well as the overall sample. All tests were nonsignificant.

Contributor Information

Christian A. Webb, University of Pennsylvania

Robert J. DeRubeis, University of Pennsylvania

Sona Dimidjian, University of Colorado at Boulder.

Steven D. Hollon, Vanderbilt University

Jay D. Amsterdam, University of Pennsylvania

Richard C. Shelton, Vanderbilt University

References

  1. Aiken LS, West SG. Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage; 1991. [Google Scholar]
  2. American Psychiatric Association. Handbook of psychiatric measures. Washington, DC: American Psychiatric Publishing; 2000. [Google Scholar]
  3. Andrusyna TP, Tang TZ, DeRubeis RJ, Luborsky L. The factor structure of the Working Alliance Inventory in cognitive-behavioral therapy. Journal of Psychotherapy Practice and Research. 2001;10:173–178. [PMC free article] [PubMed] [Google Scholar]
  4. Arbuckle JL. Amos 16.0 user’s guide. Chicago, IL: SPSS; 2007. [Google Scholar]
  5. Barber JP, DeRubeis RJ. On second thought: Where the action is in cognitive therapy for depression. Cognitive Therapy and Research. 1989;13:441–457. doi: 10.1007/BF01173905. [DOI] [Google Scholar]
  6. Barber JP, DeRubeis RJ. The Ways of Responding: A scale to assess compensatory skills taught in cognitive therapy. Behavioral Assessment. 1992;14:93–115. [Google Scholar]
  7. Barber JP, DeRubeis RJ. Change in compensatory skills in cognitive therapy for depression. Journal of Psychotherapy Practice and Research. 2001;10:8–13. [PMC free article] [PubMed] [Google Scholar]
  8. Baron RM, Kenny DA. The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51:1173–1182. doi: 10.1037/0022-3514.51.6.1173. [DOI] [PubMed] [Google Scholar]
  9. Beck AT, Rush AJ, Shaw BF, Emery G. Cognitive therapy of depression. New York, NY: Guilford Press; 1979. [Google Scholar]
  10. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory–II. San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
  11. Beck JS. Cognitive therapy: Basics and beyond. New York, NY: Guilford Press; 1995. [Google Scholar]
  12. Castonguay LG, Goldfried MR, Wiser S, Raue PJ, Hayes AM. Predicting the effect of cognitive therapy for depression: A study of unique and common factors. Journal of Consulting and Clinical Psychology. 1996;64:497–504. doi: 10.1037/0022-006X.64.3.497. [DOI] [PubMed] [Google Scholar]
  13. Coffman SJ, Martell CR, Dimidjian S, Gallop B, Hollon SD. Extreme nonresponse in cognitive therapy: Can behavioral activation succeed where cognitive therapy fails? Journal of Consulting and Clinical Psychology. 2007;75:531–541. doi: 10.1037/0022-006X.75.4.531. [DOI] [PubMed] [Google Scholar]
  14. DeRubeis RJ, Feeley M. Determinants of change in cognitive therapy for depression. Cognitive Therapy and Research. 1990;14:469–482. doi: 10.1007/BF01172968. [DOI] [Google Scholar]
  15. DeRubeis RJ, Hollon SD, Amsterdam JD, Shelton RC, Young PR, Salomon RM, Gallop R. Cognitive therapy vs medications in the treatment of moderate to severe depression. Archives of General Psychiatry. 2005;62:409–416. doi: 10.1001/archpsyc.62.4.409. [DOI] [PubMed] [Google Scholar]
  16. DeRubeis RJ, Webb CA, Tang TZ, Beck AT. Cognitive therapy. In: Dobson KS, editor. Handbook of cognitive-behavioral therapies. 3. New York, NY: Guilford Press; 2010. pp. 277–316. [Google Scholar]
  17. Dimidjian S, Hollon SD, Dobson KS, Schmaling KB, Kohlenberg RJ, Addis ME, Jacobson NS. Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology. 2006;74:658–670. doi: 10.1037/0022-006X.74.4.658. [DOI] [PubMed] [Google Scholar]
  18. Driessen E, Cuijpers P, Hollon SD, Dekker JJM. Does pretreatment severity moderate the efficacy of psychological treatment of adult outpatient depression? A meta-analysis. Journal of Consulting and Clinical Psychology. 2010;78:668–680. doi: 10.1037/a0020570. [DOI] [PubMed] [Google Scholar]
  19. Fawcett J, Epstein P. Unpublished manuscript. Department of Psychiatry, Rush-Presbyterian-St. Luke’s Medical Center; Chicago: 1980. Clinical management—imipramine/placebo administration manual. [Google Scholar]
  20. Feeley M, DeRubeis RJ, Gelfand L. The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. Journal of Consulting and Clinical Psychology. 1999;67:578–582. doi: 10.1037/0022-006X.67.4.578. [DOI] [PubMed] [Google Scholar]
  21. First M, Spitzer R, Gibbon M, Williams J. Structured Clinical Interview for DSM–IV–TR Axis I Disorders, Research Version, Patient Edition with Psychotic Screen (SCID–I/P W/PSY SCREEN). New York, NY: Biometrics Research, New York State Psychiatric Institute; 2001. [Google Scholar]
  22. Fleiss JL. Balanced incomplete block designs for inter-rater reliability studies. Applied Psychological Measurement. 1981;5:105–112. doi: 10.1177/014662168100500115. [DOI] [Google Scholar]
  23. Fournier JC, DeRubeis RJ, Hollon SD, Dimidjian S, Amsterdam JD, Shelton RC, Fawcett J. Antidepressant drug effects and depression severity: A patient-level meta-analysis. JAMA. 2010;303:47–53. doi: 10.1001/jama.2009.1943. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gelfand LA, Mensinger JL, TenHave T. Mediation analysis: A retrospective snapshot of practice and more recent directions. Journal of General Psychology. 2009;136:153–176. doi: 10.3200/GENP.136.2.153-178. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hamilton M. A rating scale for depression. Journal of Neurology, Neurosurgery & Psychiatry. 1960;23:56–62. doi: 10.1136/jnnp.23.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Hollon SD, DeRubeis RJ, Evans MD, Wiemer M, Garvey MJ, Grove WM, Tuason VB. Cognitive therapy and pharmacotherapy for depression: Singly and in combination. Archives of General Psychiatry. 1992;49:774–781. doi: 10.1001/archpsyc.1992.01820100018004. [DOI] [PubMed] [Google Scholar]
  27. Hollon SD, Evans MD, Auerbach A, DeRubeis RJ, Elkin I, Lowery A, Piasecki JM. Unpublished manuscript. Vanderbilt University; 1988. Development of a system for rating therapies for depression: Differentiating cognitive therapy, interpersonal psychotherapy, and clinical management pharmacotherapy. [Google Scholar]
  28. Horvath AO, Del Re AC, Flückiger C, Symonds D. Alliance in individual psychotherapy. Psychotherapy. 2011;48:9–16. doi: 10.1037/a0022186. [DOI] [PubMed] [Google Scholar]
  29. Horvath AO, Greenberg LS. The development of the Working Alliance Inventory. In: Greenberg LS, Pinsoff WM, editors. The psychotherapeutic process: A research handbook. New York, NY: Guilford Press; 1986. pp. 529–556. [Google Scholar]
  30. Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology. 1989;36:223–233. doi: 10.1037/0022-0167.36.2.223. [DOI] [Google Scholar]
  31. Khan A, Kolts RL, Rapaport MH, Krishnan KRR, Brodhead AE, Brown WA. Magnitude of placebo response and drug–placebo differences across psychiatric disorders. Psychological Medicine. 2005;35:743–749. doi: 10.1017/S0033291704003873. [DOI] [PubMed] [Google Scholar]
  32. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: A meta-analysis of data submitted to the Food and Drug Administration. PloS Medicine. 2008;5(2):e45. doi: 10.1371/journal.pmed.0050045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Klerman GL, Rounsaville B, Chevron E, Neu G, Weissman MM. Unpublished manuscript. 4. Yale University; 1979. A manual for the short term interpersonal psychotherapy of depression. [Google Scholar]
  34. Kline RB. Principles and practice of structural equation modeling. 2. New York, NY: Gilford Press; 2005. [Google Scholar]
  35. MacKinnon DP, Lockwood CM, Hoffman JM, West SG, Sheets V. A comparison of methods to test the significance of mediation and other intervening variable effects. Psychological Methods. 2002;7:83–104. doi: 10.1037/1082-989X.7.1.83. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. MacKinnon DP, Lockwood CM, Williams J. Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research. 2004;39:99–128. doi: 10.1207/s15327906mbr3901_4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Shrout PE, Bolger N. Mediation in experimental and non-experimental studies: New procedures and recommendations. Psychological Methods. 2002;7:422–445. doi: 10.1037/1082-989X.7.4.422. [DOI] [PubMed] [Google Scholar]
  38. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin. 1979;86:420–428. doi: 10.1037/0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
  39. Strunk DR, Brotman MA, DeRubeis RJ. The process of change in cognitive therapy for depression: Predictors of early inter-session symptom gains. Behaviour Research and Therapy. 2010;48:599–606. doi: 10.1016/j.brat.2010.03.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Strunk DR, DeRubeis RJ, Chui A, Alvarez JA. Patients’ competence in and performance of cognitive therapy skills: Relation to the reduction of relapse risk following treatment for depression. Journal of Consulting and Clinical Psychology. 2007;75:523–530. doi: 10.1037/0022-006X.75.4.523. [DOI] [PubMed] [Google Scholar]
  41. Tang TZ, DeRubeis RJ. Reconsidering rapid early response in cognitive behavioral therapy for depression. Clinical Psychology: Science and Practice. 1999;6:283–288. doi: 10.1093/clipsy.6.3.283. [DOI] [PubMed] [Google Scholar]
  42. Webb CA, DeRubeis RJ, Amsterdam JD, Shelton RC, Hollon SD, Dimidjian S. Differences in the implementation of cognitive therapy: A comparison of two randomized clinical trials. 2011a Manuscript in preparation. [Google Scholar]
  43. Webb CA, DeRubeis RJ, Amsterdam JD, Shelton RC, Hollon SD, Dimidjian S. Two aspects of the therapeutic alliance: Differential relations with depressive symptom change. Journal of Consulting and Clinical Psychology. 2011b;79:279–283. doi: 10.1037/a0023252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Webb CA, DeRubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2010;78:200–211. doi: 10.1037/a0018912. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES