The Process of Change in Cognitive Therapy for Depression when Combined with Antidepressant Medication: Predictors of Early Inter-Session Symptom Gains

Abstract

Objective

Previous studies of cognitive therapy (CT) for depression have examined therapist adherence and the therapeutic alliance as predictors of subsequent symptom change. However, little is known about these CT process variables when CT is delivered in combination with antidepressant medication.

Method

In a sample of 176 depressed outpatients randomized to the combined condition of a clinical trial, we examined ratings of adherence and alliance as predictors of subsequent session-to-session symptom change across early CT sessions.

Results

Both adherence to Behavioral Methods / Homework and the therapeutic alliance significantly predicted session-to-session symptom change. In models in which patients’ medication regimen and prior symptom change were covaried, only Behavioral Methods / Homework remained a significant predictor of subsequent symptom change.

Conclusions

Our findings highlight the utility of Behavioral Methods / Homework in combined treatment for promoting early session-to-session symptom change.

In the treatment of depression, cognitive therapy (CT) has been recognized as being among the psychotherapies with the greatest empirical support (Chambless & Ollendick, 2001; DeRubeis & Crits-Christoph, 1998; Department of Health, 2008). Having established the efficacy of a treatment, it is important to understand the process by which that treatment achieves its effects; such an understanding is likely to be useful in informing attempts to both refine and effectively disseminate the treatment (Rosen & Davison, 2003). Process researchers have attempted to address how CT for depression achieves its effects by capitalizing on naturally-occurring variability in therapeutic process and outcome. To date, research on CT for depression has shown: (1) variation in therapist adherence to specific elements of CT serves as an important predictor of variability in subsequent outcome, whereas (2) variation in the therapeutic alliance appears to play less, if any, role in predicting subsequent outcome (DeRubeis & Feeley, 1990; Feeley, DeRubeis, & Gelfand, 1999; Strunk, Brotman, & DeRubeis, 2010). While this research has been primarily conducted in the context of CT provided alone, there is evidence to suggest that the combination of CT and antidepressant medication (ADM) produces better clinical outcomes than does either treatment alone (Friedman, Wright, Jarrett, & Thase, 2006; Otto, Smits, & Reese, 2005) and is quite common among patients in CT (Olfson et al., 2002). Although one might assume any mechanisms operative in CT alone would also be important in combined treatment, this may not be the case. As we discuss, components of CT may be more or less important in combined treatment as compared with CT alone.

In this paper, we examine the relation of psychotherapy process variables with subsequent symptom change in CT combined with ADM, thereby extending previous research which has been conducted primarily in CT alone. We focus on two process variables: therapist adherence and the therapeutic alliance. We use repeated measures regression to examine these process variables as predictors of session-to-session symptom change early in treatment. In addition to these models, we also consider models that control for prior symptom change and patients’ ADM regimens. Finally, given the limitations of previous efforts to identify the factor structure of adherence, we first conduct an exploratory factor analysis to better elucidate the factors underlying adherence in CT (details of which are provided in Supplement A).

Adherence and Alliance in Process-Outcome Relations

An important goal of psychotherapy research is to understand the relationship between therapeutic process variables and outcomes. Two important process variables are the therapeutic alliance– the collaborative and affective bond between therapist and patient – and therapist adherence– the extent to which therapists utilize specific procedures described in a given treatment manual. The alliance has been found to have a small, reliable association with outcome across a number of different psychotherapies (r = .22 reported by Martin, Garske, & Davis, 2000). Though less studied, therapist adherence was recently reported to be unrelated to outcome across studies of different psychotherapies (r = .02, reported by Webb, DeRubeis and Barber, 2010) – with the caveat that there was considerable heterogeneity in this estimate.

As our study focuses on the process of change in CT for depression, rather than reviewing all estimates contributing to these meta-analyses, we focus on studies satisfying two criteria: (1) examination of CT for depression (whether the depression was acute or chronic); and (2) examination of adherence or alliance as predictors of subsequent symptom change. We emphasize subsequent symptom change for methodological reasons (Judd & Kenny, 1981). Without attention to the temporal precedence of process measures relative to the outcome they may predict, any process-outcome association might be attributable, at least in part, to symptom change that had already occurred at the time of the assessment of the therapeutic processes. Such a possibility becomes implausible when process variables are assessed prior to the period of time over which the associated changes in outcome are observed (see Barber, 2009).

Process Measures Predicting Subsequent Change in CT for Depression

In studies of CT for depression, the importance of therapist adherence to manual-specified behaviors has been suggested by the relation of these behaviors to subsequent change in depressive symptoms. Prototypical examples of such behaviors in CT include: setting and following an agenda for a session, helping patients to use activities to promote a sense of pleasure or mastery, and using evidence to evaluate the accuracy of automatic thoughts. The model underlying CT for depression suggests that while attention to the alliance may be important, therapist adherence is hypothesized to be the primary determinant of symptom change (Beck, Rush, Shaw, & Emery, 1979). Consistent with these ideas, three studies have found that some elements of adherence (but not the alliance) predicted subsequent symptom change in CT.

In the first of these studies, DeRubeis and Feeley (1990) examined adherence and alliance as predictors of subsequent symptom change in a sample of 25 patients participating in open duration CT for depression. The authors conducted a factor analysis of the 28 items assessing adherence to CT drawn from the Collaborative Study Psychotherapy Rating Scale (Hollon et al., 1988) and two factors emerged: Concrete adherence, which includes in-session methods and homework assignments using both cognitive and behavioral strategies; and Abstract adherence, which includes items assessing broader discussions of therapy-relevant issues, such as efforts to further understand the meaning of patients’ beliefs. However, this analysis was limited by insufficient sample size and was not conducted in accord with more recent recommendations on methods for conducting an exploratory factor analysis (Fabrigar, Wegener, MacCallum, & Strahan, 1999). DeRubeis and Feeley found that Concrete but not Abstract adherence (each rated at session 2) predicted subsequent symptom change occurring between session 2 and the 12^th week of CT; the alliance failed to predict subsequent symptom change. Although the majority of the patients in this study were medication-free, five patients had been taking ADM prior to beginning CT and continued on ADM over the 12 week period.

In a replication of DeRubeis and Feeley (1990), Feeley and colleagues (1999) examined adherence and alliance as predictors of subsequent symptom change in a sample of 32 CT patients from a clinical trial comparing CT alone and combined with ADM. As in the prior study, Concrete adherence (but not Abstract adherence or the alliance) predicted symptom change from session 2 through the week 12 assessment. The small sample precluded a well-powered test of difference between CT-alone and concurrent ADM.

Most recently, Strunk and colleagues (2010a) examined adherence and alliance as predictors of subsequent symptom change in a sample of 60 moderately to severely depressed outpatients participating in the CT alone condition of a clinical trial (DeRubeis et al., 2005). To more precisely model the relatively rapid effects of process variables, Strunk and colleagues examined subsequent symptom change over a brief interval – the time between sessions. As noted in their paper, Tang and colleagues’ (Tang & DeRubeis, 1999; Tang, DeRubeis, Beberman & Pham, 2005) examination of the relatively rapid effects observed in sudden gains in CT suggest that causal relations may play out in single, between-session intervals. Repeated measures regression models were used to examine whether repeated observations of process measures were predictors of session-to-session symptom change across the first four sessions of CT. The authors used adherence subscales based upon a new factor structure derived from the data we report on in the current paper¹. Two of the subscales (called Cognitive Methods and Negotiating / Structuring) predicted greater symptom improvement in the session-to-session model. The other adherence subscale, Behavioral Methods / Homework, was not a significant predictor of session-to-session symptom change. The alliance also failed to predict symptom change.

Taken together, these three studies examining adherence each found that some elements of adherence predicted subsequent symptom change. Use of different adherence subscales and the prediction of symptom change across different time periods complicate the comparison of process-outcome relations across studies. In contrast, the therapeutic alliance failed to predict subsequent symptom change in any of the studies reported above. However, although the lack of prediction of symptom change from the alliance has been a consistent finding, thus far, in studies of CT, we describe below a study focusing on a cognitive-behavioral treatment other than CT that warrants careful consideration with regard to alliance-outcome relations in the treatment of depression.

Prior Change as a Third Variable

When a process-outcome relation is obtained, such a relation may be explained by the process variable causing outcome, outcome causing the process variable (i.e., reverse causality), or a third variable leading to a spurious relation (Judd & Kenny, 1981). As noted above, others have emphasized the importance of process measures predicting subsequent symptom change to rule out reverse causality. While a large number of third variables exist, prior symptom change has been highlighted as a particularly important variable to address (Barber et al., 2000). If the association between a process variable and outcome were merely reflective of the influence of early symptom change, such a spurious process-outcome relation would say little about the active ingredients of treatment. Drawing from a large trial of treatments for chronic depression (Keller et al., 2000), Klein et al. (2003) have considered the role of prior symptom change in their analysis of the Cognitive Behavioral Analysis System of Psychotherapy (CBASP).

Klein et al. (2003) investigated the alliance as a predictor of subsequent symptom change among 367 depressed outpatients assigned to CBASP alone or CBASP combined with Nefazodone. All patients in this trial met criteria for MDD with chronicity (either chronic depression, dysthymia or 2 years of MDD with incomplete remission). Patient-reported alliance at week 2 predicted subsequent slope of symptoms through the end of treatment. This relation was obtained even when controlling for prior symptom change. As the authors note, this relation could be attributable to the more important role of the therapeutic alliance in CBASP relative to CT; a competing explanation is that the effect size obtained (ranging from .10 - .15 across the models the authors present) may have represented a population value that would be unlikely to be detected in smaller sample sizes, such as those used in previous studies of CT. A well-powered study of CT would allow for testing these alternative accounts.

Examining the Process of Change in CT when Combined with Antidepressant Medication

Although two of the CT studies reviewed above included some representation of patients participating in combined treatment, those studies did not examine possible differences in process-outcome relations in CT alone compared to combined treatment. There are several reasons one might expect that the results from CT alone may not generalize to combined treatment; however, whether process-outcome relations would be stronger or attenuated remains unclear.

There are reasons to suspect that ADM might strengthen process-outcome relations in CT. For instance, by reducing affective discomfort, ADM may help patients to tolerate discussion of emotionally-charged topics and thereby facilitate applying cognitive strategies to these topics (Klerman et al., 1994). ADM also appears to have very rapid effects (i.e., evident three hours after administration) on information processing biases (Harmer et al., 2009); changes in these biases could facilitate patients’ deliberate efforts to re-evaluate their automatic thoughts early in CT. ADM may also enhance the effects of behavioral strategies commonly used in CT (Beck et al., 1979), by helping patients to engage in these behaviors or to experience more marked benefits from doing so.

However, one might just as readily posit mechanisms that would lead to attenuated process-outcome relations. Cognitive strategies may be less strongly related to symptom change in the presence of ADM if, for example, patients taking ADM experience such reduced negative affect as to make accessing their negative cognitions difficult. Similarly, while ADM-induced changes in information processing could facilitate the use of cognitive strategies in CT, another possibility is that these information processing effects might nullify the effects of CT-mediated cognitive change. Although these possibilities are speculative and the direction of any differences is unclear, there is evidence of differential process-outcome relations between ADM patients and those not on ADM. Specifically, these differences have been reported in studies comparing medication to placebo, suggesting that the active medication, presumably the only key difference across those treatment conditions, gave rise to that pattern of results (Carroll, Nich, & Rounsaville, 1997; Strunk et al., 2010b). Thus, there is a need to examine process-outcome relations when patients are on active medications.

Purpose of this Study

We sought to investigate two psychotherapy process variables (viz., adherence and alliance) as predictors of subsequent symptom change in combined treatment. Drawing on the methods utilized by Strunk and colleagues (2010a), we examine process measures as predictors of subsequent session-to-session symptom change across multiple sessions early in treatment. We have two reasons for adopting this approach. First, examining session-to-session change focuses our analyses on a relatively brief time period (i.e., the period between one session and the next). We expect the relatively immediate effects of psychotherapy sessions might best be observed over such a brief interval. Second, we focus on the first three sessions, as symptom change over these sessions is likely to be great (Ilardi & Craighead, 1994; Feeley et al., 1999), and patients exhibit considerable variability in the degree of symptom change during this period. This variability maximizes the potential for detecting any process-outcome relations. Whereas Strunk and colleagues used the same raters to code sequential sessions of CT and thereby risked ratings being influenced by knowledge of prior symptom change, we ensured that each patient's sessions were rated by independent raters to guard against any possible contamination of ratings through knowledge of prior sessions. Additionally, we include models that test for process-outcome relations while controlling prior symptom change and differences in patients’ ADM regimens. Nevertheless, because we have concerns that these models have the potential to be overly conservative², we also plan to examine models without prior change included as a covariate.

Method

Participants

Patients

Participants were 176 of 227 patients (78%) in the combined condition of a three-site, randomized trial of CT combined with ADM and ADM alone (S. D. Hollon and R. J. DeRubeis, personal communication, 2012). All patients were given a primary current diagnosis of Major Depressive Disorder (MDD) as measured by the Structured Clinical Interview for DSM-IV (First, Spitzer, Gibbon, & Williams, 2001). In addition, all patients either met criteria for chronic depression or had at least one prior episode of depression. Individuals with dysthymia or double depression were included as long as they met the other inclusion criteria. Excluded from the study were: those with psychotic features; a history of bipolar disorder; current substance abuse; another primary Axis I disorder requiring treatment other than what was provided in the study; borderline, antisocial, or schizotypal personality disorder; subnormal intellectual potential (IQ ≤ 80); or current suicide risk sufficient to preclude treatment on an outpatient basis. For this study, the sample was limited to those combined treatment participants with at least one video recording available from the first three CT sessions. While patients could be followed for up to 7.5 years depending on their treatment response, we focus only on the first three CT sessions.

Of the 176 patients in the sample, 58% were women. The average age was 43.8 (SD = 13.0; range of 18 to 80). With regard to ethnicity, 85% of patients were White, 9% were African American, 2% were Asian, 1% were American Indian or Native Alaskan, and 2% identified themselves as “other”. A total of 86% of patients reported a previous depressive episode (with 51% having three or more prior episodes), and 32% met criteria for chronic depression.

Therapists

Fourteen therapists (9 of whom were women) provided CT in the trial. As described more fully in Supplement B, therapists had substantial variability in experience with and competence in CT. On-site supervision of CT was provided by: Robert J. DeRubeis at the University of Pennsylvania, Steven D. Hollon at Vanderbilt University, and Paula R. Young at Rush-Presbyterian-St. Luke's Medical Center..

Measures

Depressive Symptoms

Session-to-session change in depressive symptom severity was assessed using the Beck Depression Inventory-II (BDI-II; Beck, Steer & Brown, 1996), a widely-used and well validated self-report measure. Cronbach's alpha for the BDI-II ranged from .91 to .92 across the first four sessions.

Therapist Adherence

The 28 items included in the Cognitive Behavioral subscale of the Collaborative Study Psychotherapy Rating Scale (Hollon et al., 1988) were used to rate therapist adherence to CT. Items are scored on a 7-point Likert scale, with higher scores reflecting greater frequency and extent of therapist adherence. Based on our own factor analysis (see Supplement A), we utilize three subscales: Cognitive Methods, Negotiating / Structuring, and Behavioral Methods / Homework. The Cognitive Methods scale is comprised of 10 items assessing therapist efforts to help patients identify and evaluate the accuracy of their automatic thoughts. The Negotiating / Structuring scale consists of seven items reflecting therapists’ management of session structure, pacing, and efforts to collaborate with patients. The Behavioral Methods / Homework scale is composed of five items to assess therapists’ use of behavioral strategies and therapists’ assigning and reviewing homework, including but not limited to homework that employs behavioral interventions.

Alliance

The therapeutic alliance was assessed using the 12-item observer version of the Working Alliance Inventory (WAI; Horvath & Greenberg, 1989; Tracey & Kokotovic, 1989). In addition to the total score, the WAI has three subscales: Goals, which reflects the agreement between therapist and patient on overall goals of treatment; Tasks, which reflects the agreement on the appropriate tasks on which to focus (in order to achieve goals); and Bond, the quality of the affective relationship between the therapist and the patient.

Alliance scores (total and subscales) were raised to the power of three to reduce negative skew evident in this sample. In all analyses, the magnitude of effects was similar using transformed and raw scores.

Antidepressant Treatment Regimen

Pharmacotherapy was delivered flexibly, in a manner typical of high quality clinical practice. In general, pharmacotherapists began with a selective norepinephrine reuptake inhibitor (venlafaxine or duloxetine), with the option to augment or switch if indicated (S. D. Hollon and R. J. DeRubeis, personal communication, 2012). Implementation of pharmacotherapy was conducted in accord with the clinical management manual developed for the Treatment of Depression Collaborative Research Program (Fawcett, Epstein, Fiester, Elkin, & Autry, 1987). Pharmacotherapy was overseen by Jan Fawcett and an on-site supervisor (Jay D. Amsterdam at the University of Pennsylvania, Richard C. Shelton at Vanderbilt University, and John Zajecka at Rush-Presbyterian-St. Luke's Medical Center) with monthly conference calls used to facilitate consistency across the sites.

For the purpose of this study, we constructed a variable to reflect each patient's ADM regimen between intake evaluation and the last rated CT session. The protocol allowed multiple classes of ADMs as well as the use of other drugs not used to treat depression per se (e.g., chloral hydrate for sleep difficulties). Two experts (Jay D. Amsterdam and Richard C. Shelton) reviewed all medications used in the trial and identified the subset of 25 drugs which they deemed appropriate as primary in the treatment of depression. In our sample, a total of 22 of these drugs were used prior to the last rated CT session. For each of the drugs, our experts provided the “minimum therapeutic dose” and the “maximum routine dose.” By minimum therapeutic dose, our experts were identifying a dose that would indicate that a failure to show a therapeutic response reflects a failed trial of a particular drug. These ratings were used as a basis for converting doses of different medications to a common 0 to 10 scale. Doses between zero and the minimum therapeutic dose were assigned values between 0 and 5 (interpolating as needed). Doses between the minimum and maximum were assigned values between 5 and 10 (again interpolating as needed). Values were averaged to create an “ADM regimen” variable to reflect the average ADM dosing value per day for the period between the first pharmacotherapy session and the last rated CT session. These scores ranged from 0 to 19.43 (SD = 3.17) with a median of 4.55, suggesting the median participant early in treatment was receiving near a minimum therapeutic dose. Values above 10 could be achieved only if the patient was on more than one ADM.

Procedure

Although we planned to rate the first three CT sessions for all patients in combined treatment in the trial, due to missing or damaged recordings, the number of patients with one or more sessions available was reduced to 176. Across these patients, recordings of a total of 467 sessions were available, with approximately equal representation of each session (for session 1, n = 160; for session 2, n = 157; for session 3, n = 150). Ratings were provided by 50 advanced undergraduate students. Prior to providing ratings for the study, raters completed approximately 50 hours of training over a 10 week period. All sessions were viewed by four independent raters who were blind to each others’ ratings and outcome. Thus, raters made a total of 1,868 ratings. Each rater coded an average of 37 tapes for the study. Raters provided ratings for no more than one session per patient.

Inter-Rater Reliability of Process Scores

To evaluate the inter-rater reliability of process measures, random effects intraclass correlation coefficients (ICCs) were calculated (McGraw & Wong, 1996). Estimates of these ICCs for ratings made at each session were corrected for the number of raters (in this case, four raters per session). ICCs for the alliance were: .74 for the overall WAI total score, .64 for the Goal subscale, .71 for the Task subscale, and .72 for the Bond subscale. ICCs for adherence were: .80 for Cognitive Methods, .50 for Negotiating / Structuring, and .79 for Behavioral Methods / Homework.

Analytic Strategy

A repeated measures regression analysis was conducted to examine predictors of session-to-session symptom change using SAS PROC Mixed (without specification of random effects). In these analyses, a vector of lagged BDI-II scores for each participant (i.e., BDI-II scores from sessions 2 through 4) served as the dependent variable, with BDI-II scores from the previous sessions entered as a covariate (i.e., BDI-II at session 1 serves as a covariate in predicting BDI-II at session 2, BDI-II at session 2 serves as a covariate in predicting BDI-II at session 3, etc.). For each process measure examined as a predictor, a vector of scores (for sessions 1, 2, and 3) was entered. Thus, these models use repeated observations to estimate the association between process measures (across sessions 1 through 3) and BDI-II scores in the next session while controlling for BDI-II scores in the current session. In these analyses, a negative t-score would indicate that higher scores on the process measure are associated with greater symptom change at the next session, controlling for BDI-II scores at the current session. Based on comparison with alternative covariance structures (viz., compound symmetry, toeplitz, and first order autoregressive), unstructured was identified as achieving the best fit for this model (on the basis of Akaike's Information Criterion, Schwarz's Bayesian Criterion, and -2 Res Log Likelihood).

Results

We first examined the concurrent association among process measures and depressive symptoms across sessions 1 through 3. As summarized in Table 1, we conducted correlation analyses separately at each session. Process measures were largely unrelated to concurrent symptom severity, with the sole exception being the alliance at one of three sessions. Among process measures, no large correlations were identified. The three largest average correlations obtained were those of Cognitive Methods and Negotiating / Structuring and the correlations of the alliance with each of these two scores. We used the same approach to examine concurrent associations among the alliance subscales. Each of these correlations was significant, with the average correlations across sessions ranging from .67 to .85.

Table 1.

Average Correlations among Process Measures and Concurrent BDI-II scores Across Sessions 1 through 3

	1	2	3	4	5
1. Cognitive Methods	--
2. Negotiating / Structuring	32(2/3)	--
3. Behavioral Methods / Homework	-.08 (0/3)	.08(0/3)	--
4. Alliance	.21 (3/3)	.24(3/3)	.15(0/3)	--
5. BDI-II	.06 (0/3)	-.09(0/3)	.01(0/3)	-.05(1/3)	--

Note. Correlations reported are the average of the correlations at each of the first three sessions.

To compute averages, correlations were converted to z-scores. The z-scores reflecting associations of the same two constructs were averaged across sessions, and these averages were then converted back to the r-type effect sizes shown above. Significance tests were performed at each session. The fractions in superscript indicate at how many of the first three sessions the relationship was significant (p < .05).

Session-to-Session Analyses: Predicting Early Symptom Improvement

Prior to examining process-outcome relations, we considered the role of therapist in our model. Modeling therapist as a random effect, we calculated ICCs for therapist at each of the first three sessions. ICCs ranged from .006 to .007. While cutoffs for substantial ICCs are not well established, we consider these to be small values. We also tested the fit of models in which therapist was or was not specified as a random effect, using procedures described by West, Welch & Galecki (2007). Modeling therapist as a random effect failed to improve model fit (difference in likelihood ratio statistic = 0.3, df = 1, p = .29), and also failed to change the statistical significance of any predictors in our models. Thus, in the models reported below, therapist was not specified as a random effect.

In a series of separate models, we examined each process measure as a predictor of subsequent session-to-session symptom improvement on the BDI-II. As the left portion of Table 2 shows (see Model 1), the largest significant effect obtained was for Behavioral Methods / Homework serving as a predictor of greater session-to-session symptom change.³ The only other significant effect observed was for alliance total scores, with alliance scores also predicting greater session-to-session symptom change. Neither Cognitive Methods nor Negotiating / Structuring emerged as significant predictors.

Table 2.

Therapist Adherence and Alliance as Predictors of Subsequent Session-to-Session Symptom Change

	Model 1		Model 2
	Basic Model (df= 154)		ADM Regimen and Prior Change Covaried (df =137)
Process Variables	r	t	r	t
Adherence
Cognitive Methods	.11	1.42	.14	1.60
Negotiating / Structuring	.07	.90	.13	1.57
Behavioral Methods / Homework	-.24	-3.01**	-.21	-2.52*
Alliance Total Score	-.16	-1.99*	-.14	-1.65

Note: Negative t-scores reflect a process measure being related to larger improvements in symptom scores. For both models, BDI-II scores at the next session are predicted by process scores (with current session BDI-II scores covaried). For Model 2, ADM regimen and symptom change (from intake to the current session) have been entered as additional covariates.

†p < .10.

^*p < .05.

^**p < .01.

Given the significant relation between alliance total scores and session-to-session symptom change, we also conducted follow-up analyses examining each of the alliance subscales as predictors in session-to-session models. Only the Task subscale was a significant predictor of session-to-session change (r = -.17, t(154) = -2.17, p = .03). Neither the Goal nor the Bond subscales were significant predictors (Goal: r = -.13, t(154) = -1.57, p = .12; Bond: (r = -.11, t(154) = -1.40, p = .16).

We then examined two models in which multiple process measures served as predictors: a model with all three adherence scores and a model with all process measures which had emerged as significant predictors in single predictor models described above. When all three adherence scores were entered into the same model, Behavioral Methods / Homework remained the sole significant predictor (r = -.23, t(154) = -2.99, p = .003). In the model with only those measures that emerged as significant predictors in the single predictor models, Behavioral Methods / Homework remained a significant predictor (r = -.22, t(154) = -2.78, p = .006) while the alliance did not (r = -.13, t(154) = -1.63, p = .105). Finally, we ran a model with Behavioral Methods / Homework and the Task subscale of the alliance as predictors; Behavioral Methods/Homework remained significant (r = -.22, t(154) = -2.74, p = .007), while the Task was reduced to a non-significant trend (r = -.14, t(154) = -1.79, p = .08). ⁴

Considering the Role of Prior Symptom Change and Concurrent Pharmacotherapy

To consider two potentially important third variables, we entered covariates reflecting prior symptom change and ADM regimen in our session-to-session model. In constructing the prior symptom change covariate, we outputted the residuals from a series of regression models in which BDI-II at each session (for sessions 1 through 3) served as the dependent variable with intake BDI-II scores serving as the sole predictor. These residuals thus reflect change in BDI-II from intake to each session. When entered as an additional covariate to the session-to-session model, these prior change values were not significant predictors (r = -.05, t (144) = -.63, p = .5). When we examined the ADM regimen covariate, this variable showed a non-significant trend towards prediction of subsequent change (r = .14, t(156) = 1.74, p = .08), with the direction of the effect indicating that higher doses were associated with less robust session-to-session symptom improvement. We suspect this association reflects the likelihood that the pharmacotherapists adjusted ADM regimens to be more aggressive when they detected poorer initial response.

As shown in Table 2 (Model 2), the previously significant effect for Behavioral Methods / Homework remained significant even after the two additional covariates were included in the model. However, the alliance no longer showed a significant relation to subsequent change (nor did any of the alliance subscales, with ps >.08). As in the model without the covariates, Behavioral Methods / Homework remained the sole significant predictor when we examined all three adherence scores in a model with ADM regimen and prior change covaried (r = -.22, t(139) = -2.63, p = .009). We also tested all two-way interactions among our four process measures. In both our initial model and the model with the additional covariates, none of these interactions was significant (all ps > .1).

Discussion

We found that Behavioral Methods / Homework and alliance predicted subsequent session-to-session symptom change early in the course of CT combined with ADM. However, the Behavioral Methods / Homework scale was the only significant predictor in the models in which we accounted for patients’ ADM regimens and prior symptom change. When these additional covariates were included, the relation of the alliance and outcome was no longer statistically significant, but the effect sizes were quite similar. Thus, we were unable to find any third variables that accounted for the relationship between Behavioral Methods / Homework and subsequent symptom change. We first consider the alliance finding in the context of previous research, and then discuss results involving adherence.

As noted previously, the alliance-outcome relation was estimated at an r of .22 in the Martin et al. meta-analysis (2000). However, insofar as the estimates contributing to this meta-analysis were not restricted to the prediction of subsequent change specifically, this may have led to an overestimate of the true effect of interest (Feeley et al, 1999; Barber, 2009). Estimates from both the large sample of Klein et al. (2003; ranging from rs of .10 to .15 across models) and the moderate-sized sample of Strunk et al. (2010; r = .15) are consistent with the effect sizes obtained in our models (rs of.14 and .16). Thus, consistent effect sizes have been obtained in CT or CBASP –whether provided alone or in combination with medication. In our view, the consistency of these effect size estimates is beginning to suggest that differences in statistical significance noted across studies might best be accounted for by differences in statistical power.

With respect to adherence, both the adherence scales we used and the analytic strategy we employed were most consistent with those of Strunk and colleagues’ study of CT alone (2010a). Our findings are in marked contrast to the results reported in that study. Strunk et al. found that Cognitive Methods was the most robust predictor of session-to-session symptom change, while Behavioral Methods / Homework was not predictive. Thus, our results highlight process-outcome relations in combined treatment, and these relations appear to differ in important ways from those obtained previously in CT alone.

Comparing Adherence-Outcome Relations in Combined Treatment with CT Alone

In Supplement B, we formally compared our process-outcome relations to those obtained by Strunk et al. (2010a) and noted significant differences across studies for both Cognitive Methods and Behavioral Methods / Homework. Cognitive Methods predicted more strongly in the CT alone sample and Behavioral Methods / Homework predicted more strongly in the sample of patients in combined treatment.

What might account for this pattern of results? While the inclusion of ADM is the most obvious difference between these studies, variability present in the delivery of both ADM and CT complicates our efforts to understand this difference. In our sample, we found patients receiving a more aggressive ADM regimen evidenced a non-significantly poorer initial response to treatment. Thus, pharmacotherapists may have increased ADM aggressiveness as a function of poor initial response. This could cause difficulty in the interpretation of our process-outcome relations. Specifically, any process-outcome relations observed might not be true instances of CT processes driving symptom change, but rather an association of patient characteristics with both ADM treatment regimen and the behaviors of cognitive therapists. For example, suppose patients with high levels of functional impairment elicit both a focus on behavioral techniques from their cognitive therapists and a more optimal medication strategy from their pharmacotherapists. Differences in medication strategy might cause greater symptom improvement and introduce a spurious correlation between Behavioral Methods / Homework and symptom change. However, our analyses revealed that no adherence-outcome relation was accounted for by ADM dosing; only the alliance-outcome relation was reduced to non-significance by the inclusion of our measure of ADM regimen (although the effect size was comparable). Thus, the adherence-outcome relations observed in combined treatment appear to be specific to CT when combined with ADM, and we failed to find any evidence these relations were an artifact related to differences in patients’ medication regimens.

It is important to note that our data and those of Strunk et al. (2010a) were drawn from independent clinical trials and that comparisons of group differences across trials are not equivalent to those made between groups within a trial. Although patients were randomized in their respective trials, they were not randomized to the conditions being compared. Therefore, rival explanations for the observed differences must be considered carefully. As discussed in Supplement B, two possible differences between these samples are: (1) differences in the inclusion criteria for the trials; and (2) differences in therapist competence across trials.

Briefly, differences in inclusion criteria across the two studies (with a primary difference being initial severity) did not explain the differences in process-outcome relations we obtained. In a restricted sample of patients eligible for both studies, a similar pattern of results was observed, such that Cognitive Methods was a more important predictor for patients treated with CT alone and Behavioral Methods / Homework was more important for those in combined treatment (albeit at the level of a non-significant trend). As detailed in Supplement B, our exploratory analyses of therapist competence (based on supervisor ratings) revealed a more complicated picture. While the primary measures reported in the body of this manuscript are based on therapist adherence, competence is a related but distinct construct of interest. Whereas adherence reflects the “degree of utilization of specialized procedures by the therapist,” competence has been defined as “the level of skill and judgment shown by the therapist in delivering treatment” (Perepletchikova & Kazdin, 2005). In the analyses reported in Supplement B, there was evidence of a competence by Cognitive Methods interaction, such that Cognitive Methods predicted more symptom change when delivered by more competent therapists. Nonetheless, this interaction did not fully explain the difference between CT and combined treatment with regard to process-outcome relations. There was no evidence of an interaction between competence and Behavioral Methods / Homework. Future research utilizing more fine-grained assessments of competence will be needed to shed further light on this issue.

Limitations

We note five limitations. First, as our independent variables were not experimentally manipulated, we must exercise caution in drawing causal inferences. We did ensure that our key process-outcome relations reflect the relation between process variables and subsequent change, thus ruling out reverse causality. In addition, we considered at least two potentially important third variables--differences in patients’ ADM regimens and prior symptom change--that might have spuriously given rise to the process-outcome relations observed in CT combined with ADM. Nonetheless, we are unable to definitively rule-out the possibility that an unknown third variable accounts for the effects observed.

Second, our analyses focus on early sessions of therapy. We think these sessions are of special interest insofar as symptom change tends to occur most rapidly over the course of these sessions. Nonetheless, the process-outcome relations we observed may not generalize to later sessions.

Third, although the ICCs we obtained for most of our process measures were adequate, the ICC for Negotiating / Structuring was .50. This lower level of reliability would likely have attenuated effects involving this scale. Thus, our failure to find a relation between Negotiating / Structuring and subsequent change should be interpreted with some caution.

Fourth, while we hope our results shed light on the relation between CT process variables and subsequent outcome in combined treatment, more complex questions remain about the mechanisms by which these associations may emerge. Earlier we suggested that a particularly strong relation of Behavioral Methods / Homework to subsequent symptom change in combined treatment (compared to CT alone) might be due to ADM facilitating patients’ engagement in and enjoyment of assigned behavioral activities. However, it is also possible this association could be due to Behavioral Methods / Homework having an unanticipated effect on patients’ attitudes towards ADM (e.g., enhancing patients’ willingness to tolerate side effects). We have little reason to suspect this to be the case, but our data do not address this issue. All psychotherapy process studies are open to such limitations insofar as any effects observed might be accounted for by other unmeasured aspects of treatment. Nonetheless, we highlight this issue because it may be especially worth noting when two treatments are being delivered concurrently.

Finally, although the adherence-outcome relation we obtained raises the possibility that therapist behaviors may influence subsequent symptom change, our findings do not address the client's role in this process (see Kazantzis, Deane, Ronan & L'Abate, 2005). In an effort to do so, we evaluated the role of client's expectations for therapeutic improvement prior to treatment. We found that covarying a measure of these early client expectations did not change the significance of our predictors in either our primary model or the model in which prior symptom change and ADM regimen were entered as covariates (details available upon request). Nonetheless, we acknowledge the need for future research on other client factors (e.g., motivation and engagement).

Implications and Conclusion

Our findings speak to the importance of Behavioral Methods / Homework early in the course of CT combined with ADM. As we have noted, this is in contrast to Strunk and colleagues’ (2010a) study of CT alone, where Cognitive Methods was the most robust predictor. The relation of Behavioral Methods / Homework and subsequent symptom change we observed was not accounted for by differences in patients’ ADM regimens or prior symptom change. The alliance-outcome relation, while initially significant, was reduced to a non-significant trend when these covariates were considered. If replicated, the implications of this study would be striking. Our results would suggest that Behavioral Methods / Homework is uniquely important in combined treatment, while the positive effects of Cognitive Methods are less robust, or negligible, in this context. A test of process-outcome relations in the context of a randomized comparison of CT alone versus combined treatment would be particularly compelling, and would have implications for therapists implementing CT with and without concurrent ADM.

Figure 1.

CONSORT Flow Diagram