Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Sep 1.
Published in final edited form as: Behav Res Ther. 2015 Jul 2;72:56–62. doi: 10.1016/j.brat.2015.06.011

A Session-to-Session Examination of Homework Engagement in Cognitive Therapy for Depression: Do patients experience immediate benefits?

Laren R Conklin a,*, Daniel R Strunk a
PMCID: PMC4529756  NIHMSID: NIHMS709195  PMID: 26183022

Abstract

Homework is a key component of Cognitive Therapy (CT) for depression. Although previous research has found evidence for a positive relationship between homework compliance and treatment outcome, the methods used in previous studies have often not been optimal. In this study, we examine the relation of specific aspects of homework engagement and symptom change over successive session-to-session intervals. In a sample of 53 depressed adults participating in CT, we examined the relation of observer-rated homework engagement and session-to-session symptom change across the first five sessions. Within patient (and not between patient) variability in homework engagement was significantly related to session-to-session symptom improvements. These findings were similar when homework engagement was assessed through a measure of general engagement with homework assignments and a measure assessing engagement in specific assignments often used in CT. Secondary analyses suggested that observer ratings of the effort patients made on homework and the completion of cognitive homework were the numerically strongest predictors of depressive symptom improvements. Patient engagement with homework assignments appears to be an important predictor of early session-to-session symptom improvements. Future research is needed to identify what therapist behaviors promote homework engagement.

Keywords: homework, cognitive therapy, depression

Cognitive Therapy (CT) has been established as an efficacious treatment for depression (DeRubeis, Webb, Tang, & Beck, 2010). The use of homework is an integral component of CT, with homework assignments serving as a critical way of encouraging patients to practice integrating the skills they learn in therapy into their everyday lives (Beck, Rush, Shaw, & Emery, 1979; Kazantzis & Lampropoulos, 2002). Common homework assignments in CT for depression include monitoring the relationship between activities and moods, engaging in activities to promote a sense of pleasure or accomplishment, recording and developing alternative responses to one’s automatic thoughts, and engaging in behavioral experiments designed to test one’s depression-related beliefs (Beck et al., 1979).

Across a variety of cognitive-behavioral treatments for diverse psychological problems, the available evidence indicates that homework is both correlated with and experimentally linked to therapeutic outcomes. As meta-analytic evidence has shown, individual differences in completing homework are related to more positive treatment outcomes (Kazantzis, Deane, & Ronan, 2000; Kazantzis, Whittington, & Dattilio, 2010). In addition, patients randomized to therapy including homework assignments have been found to experience superior outcomes relative to those randomized to therapy without such assignments (Kazantzis et al., 2010). For example, Neimeyer and Feixas (1990) examined homework in the context of a group version of CT for depression and found that groups assigned homework outperformed those not assigned homework. However, this study did not include an assessment of the degree to which patients engaged in homework assignments. The meta-analytic evidence of a homework-outcome relation was based on a combination of studies examining different patient populations and treatments under investigation, with these treatments differing markedly in the homework utilized. In our search for studies of homework in individual CT for depression, we identified a relatively small group of seven studies (Bryant, Simons, & Thase, 1999; Burns & Spangler, 2000; Coon & Thompson, 2003; Cowan et al., 2008; Detweiler-Bedell & Whisman, 2005; Persons et al., 1988; Startup & Edmonds, 1994). A review of these studies suggests that all but one (i.e., Detweiler-Bedell & Whisman, 2005) reported evidence of homework compliance in CT for depression being associated with positive outcomes at the end of treatment. However, it is important to consider the methodological and analytic features of these studies to understand what conclusions regarding the homework-outcome relation in CT for depression are currently warranted.

In our review of these seven studies, we identified five methodological or analytic features of the studies that we believe limit the conclusions one can draw regarding the nature of the relationship between homework engagement and therapeutic outcomes in CT for depression. First, earlier investigations of homework compliance utilized retrospective therapist ratings gathered at the end of treatment (Burns & Spangler, 2000; Persons et al., 1988). When therapists make a retrospective assessment of patients’ homework engagement, it is impossible to ensure that these ratings are not contaminated by knowledge of patients’ ultimate outcomes. One might expect that therapists’ estimates of homework engagement would be inflated as patients achieved more positive outcomes. Second, six of seven studies examined the relation of homework engagement with symptom change as assessed over relatively long periods of time. In these studies, symptom change was only evaluated from pre- to post-treatment. As the exception to this, Detweiler-Bedell and Whisman (2005) included analyses of symptom change over the first half of treatment; these analyses failed to identify a significant relationship between early homework compliance and improvement in depressive symptoms across the first half of treatment. The problem with relying on studies examining the relation of homework and pre- to post-treatment symptom change is that the relation identified in these studies could at least partly reflect the impact of early symptom change on homework engagement much later in treatment. On average, patients tend to experience greater symptom improvements early in treatment, with one estimate suggesting that that 32% of symptom change occurs in the first 2 weeks (and four sessions) of a 16-week course of CT of depression (Strunk, Brotman, & DeRubeis, 2010a). Insofar as homework might be expected to have relatively immediate effects, analyses of the relation of homework engagement and session-to-session symptom change would be most sensitive to detecting the effects of interest. However, we are not aware of any studies examining the relation of homework engagement and session-to-session symptom change—in CT for depression or otherwise.

Third, a number of researchers have studied homework by examining patients’ compliance with the assignments given. This approach compares the homework completed with the amount of homework assigned, either using the percentage of homework completed (e.g., Coon & Thompson, 2003) or a Likert-type item reflecting the amount of homework completed relative to that assigned (e.g. 0 “patient did not attempt the assigned homework” to 6 “the patient did more of assigned homework than was requested”; Primakoff, Epstein, & Covi, 1986). This means that a high score could reflect either a great deal of work on the patient’s part or very little work if the therapist had only assigned little homework. Similarly, a low score could reflect either little work on the patient’s part or a large amount of work, if the therapist had assigned an even larger amount. An alternative way of examining homework is to assess the extent to which patients engage in homework activities between sessions—without a comparison to what was assigned in the preceding session (e.g., Rees, McEvoy, & Nathan, 2005). As we noted, these approaches can lead to important differences in the assessment of homework. In our view, establishing a relation of “homework engagement,” reflecting the patient’s absolute efforts rather than their efforts relative to the amount of homework assigned, provides the clearest test of the hypothesis that the more patients engage with the homework activities, the more symptom improvement patients will experience.

Fourth, patients’ homework efforts are often assessed broadly using a single variable (Kazantzis, Deane, & Ronan, 2004). Most homework research has assessed the overall amount of homework completed. Single homework measures necessarily fail to reflect the degree to which patients engage in different types of homework assignments used in the treatment under investigation. Although Kazantzis et al. (2000) examined the role of homework type in their meta-analysis, the 11 studies included in that analysis were limited to those that exclusively used only one specific type of homework. Thus, homework type was confounded with other characteristics of the studies (e.g., the clinical problem being treated). The role of different kinds of homework within CT for depression is unknown.

To our knowledge, only two previous studies have examined patients’ homework engagement on multiple specific assignments within a given treatment (Cammin-Nowak et al., 2013; Rees et al., 2005). Rees and colleagues examined several different homework types (relaxation, reading, thought diaries, and behavioral tasks) as predictors of post-treatment symptoms in a mixed anxious and depressed sample participating in group cognitive-behavioral therapy. They found a greater number of behavioral tasks (pleasant activities and exposure exercises) was significantly associated with greater change in depressive symptoms whereas a greater number of thought diaries was significantly associated with greater change in symptoms of anxiety. Cammin-Nowak and colleagues also found situational exposure predicted outcomes in CBT for panic disorder more strongly than other homework types. While treatment developers presumably would predict that all homework assignments included in their approach would be therapeutic, one can only evaluate potential differences in the relation of different homework assignments and outcome by including measures that reflect the use of the different types of assignments given. The results of the two studies of multiple specific homework assignments attest to the importance of examining individual types of homework within the same treatment.

Finally, while some researchers have used repeated homework measures, as noted earlier, researchers have yet to use such repeated measures to examine the relation of homework engagement and session-to-session symptom change. Session-to-session analyses are appropriate for examining the relation of homework engagement between two sessions and the associated changes in depressive symptoms during that between-session interval. We think such an approach coincides with the time period over which we suspect the benefits of homework would be most likely to be observed. Whether a patient is asked to engage in a pleasant activity or reappraising his or her negative automatic thoughts, these activities are hypothesized to have immediate therapeutic impact (Beck et al., 1979). Previous research with other important therapeutic constructs supports the possibility that effects of psychological interventions can be detected over such short intervals. For example, both therapist competence and therapist adherence have been found to predict session-to-session symptom changes in CT (Strunk et al., 2010a; Strunk, Brotman, DeRubeis, & Hollon, 2010b; Strunk, Cooper, Ryan, DeRubeis, & Hollon, 2012).

In addition, using repeated measures also allows one to parse within- and between-patient variability to rule out the potential impact of any stable patient characteristics serving as confounding factors (Allison, 2005; Curran & Bauer, 2011). For example, certain types of patients (e.g., those who are motivated, agreeable, or relatively free of personality psychopathology) may be more likely to have higher homework engagement and respond positively to treatment. By focusing on within-patient variability in homework, any homework-outcome relation obtained could not be attributed to these (or any other) between-patient characteristics. By separating within- and between-patient variability in homework engagement, one also opens up the possibility of finding differing relations of within-patient and between-patient homework engagement and depressive symptom change (a point to which we will return later in this paper).

This Study

In consideration of the methodological decisions highlighted above, we explore session-to-session relationships between homework engagement and concurrent depressive symptom change. We utilize observer ratings, made by raters who were blind to subsequent outcomes. We focused our investigation on the early sessions of CT for two key reasons: (1) our assessments of homework engagement were labor intensive and therefore required a focused approach; and (2) symptom change occurs disproportionately early in treatment (Sasso, Strunk, Braun, DeRubeis, & Brotman, 2014). We examine assessments of homework that include both patients’ engagement with homework generally (e.g., the time and effort patients put into homework) and patients’ engagement with specific CT assignments. By using repeated measures and parsing within- and between-patient variability, we are also able to rule-out the potentially confounding influence of any stable patient characteristics. In using this approach, our primary hypothesis is that within-patient homework engagement will be related to session-to-session symptom change. We expected this to be the case across both general and CT-specific measures of homework engagement, but examined these measures to test these ideas empirically.

Methods

Participants

Drawing from a sample of 67 patients who participated a naturalistic study of CT (see Adler, Strunk, & Fazio, 2014), data for 53 patients were sufficient for inclusion in our primary analyses. Inclusion criteria into the original study were: (1) a primary diagnosis of current Major Depressive Disorder as assessed by the Structured Clinical Interview for the DSM-IV (SCID; First, Spitzer, Gibbon & Williams, 1994); (2) 18 years of age or older; and (3) willing and able to provide informed consent. Individuals were excluded from the study if any of the following were present: (1) a history of Bipolar I disorder or a psychotic disorder; (2) a primary diagnosis of a current Axis-I disorder other than Major Depressive Disorder only if it was judged to necessitate treatment other than that being offered; (3) meeting criteria for Substance Dependence in the 6 months prior to intake; (4) a below-normal intelligence (IQ < 80); (5) clear indications of secondary gain; or (6) an acute risk of suicide sufficient to preclude outpatient treatment. In addition, among patients on antidepressant medications, only those who agreed to stay on a stable dosage throughout the course of treatment were eligible. This study was approved by the Institutional Review Board at The Ohio State University.

As we describe more fully below (see Analytic Strategy), patients must have had session recordings available from at least four of their first five CT sessions given the requirements of our primary analyses. Fourteen patients failed to meet this criterion, thirteen of whom dropped out of the study prior to session five. One patient failed to meet the criterion due to missing session recordings. Of the 53 patients who had a sufficient number of recordings, 28 (53%) were female, with a mean age of 37.5 (SD = 14.0, range: 18–69). The sample was 88% Caucasian (n = 47), 8% African American (n= 4), 2% Asian American (n = 1), and 2% were Hispanic/Latino (n = 1). Two-thirds of the sample (68%, n = 36) was diagnosed as having at least one depressive episode prior to the current one, and 58% (n= 31) of the sample were diagnosed with one or more co-morbid anxiety disorders. Patients who had insufficient data for inclusion did not differ from those included in the analyses on age, sex, or intake BDI-II scores (all ps > .1). However, the representation of Caucasian vs. non-Caucasian patients differed significantly between those included and those not included in our analyses (two-tailed Fisher’s exact test, p = .04). Specifically, among patients not included, 9 of 14 (65%) were Caucasian; whereas 47 of 53 (88%) of those included in the analyses were Caucasian.

Therapists

Four advanced graduate students provided CT and were supervised by the second author (DRS). At the beginning of the study, therapists had one to two years of experience providing CT. As reported previously (Adler et al., 2014), therapists were rated on general competence using the Cognitive Therapy Scale (Young & Beck, 1988), and the mean scores obtained were comparable to those obtained by Strunk and colleagues (2010b) when rating more experienced CT therapists in a clinical trial. For more information, see Adler et al.

Measures

Homework Engagement Scale—General (HES-General)

The HES-General is rated using a three-item observer-rated measure that includes items assessing the following forms of homework engagement over the past week: (1) the estimated amount of time the patients spent on homework (Time; possible scores from 0 “no time” to 4 “more than 3 hours”); (2) the frequency with which the patients reported using therapy skills when they were sad or upset (Frequency; possible scores from 0 “not at all” to 4 “every time”); and (3) the estimated effort the patients put into homework assignments (Effort; possible scores from 0 “no effort” to 4 “great effort”). Each of these variables has the benefit of not being tied to CT-specific homework assignments (thus can be used more broadly) and assesses components theoretically related to higher engagement in general. Beyond merely completing a homework assignment, it may be important for patients to spend the time, frequency, and effort that therapists would recommend in completing assignments (as opposed to, for example, quickly completing homework in the waiting area just prior to the session). Sum scores for the HES-General could range from 0 to 12, with higher scores indicating greater time, greater effort, and more frequent engagement in homework activities between sessions. The internal consistency of HES-General at each session was excellent, with Cronbach’s alphas ranging from .82 to .86 across sessions.

Homework Engagement Scale—CT-Specific (HES-CT)

The HES-CT is a three-item observer-rated scale that assesses the degree to which patients engage in several specific types of homework assigned in CT. The homework assessed includes: cognitive homework (e.g., use of thought records), self-monitoring homework (i.e., tracking activities and corresponding moods), and behavioral homework other than self-monitoring (i.e., activities intended to produce a sense of pleasure or mastery; practicing assertiveness). While the HES-CT items were rated on a zero (no engagement) to six (extensive engagement) scale during the study, screening of the items suggested non-normality due to a high percentage of zero values ranging from 39% (self-monitoring homework engagement) to 68% (behavioral homework engagement). This was the result of therapists often assigning one or two of the three homework types. To reduce the non-normality of the data, the non-zero ratings were recoded. Scores from one to three were coded as a one and scores from four to six were coded a two, thus resulting in a scale with values ranging from zero to two. Total HES-CT scores were calculated as the sum of these three recoded items and therefore could range from zero to six. Copies of the HES-General and HES-CT with the rating manual are available upon request.

Beck Depression Inventory - 2nd Edition (BDI-II)

The BDI-II (Beck, Steer, & Brown, 1996) is the current version of the Beck Depression Inventory. The BDI-II is a reliable and well-validated 21-item self-report measure that assesses the severity of depressive symptoms according to the Diagnostic and Statistical Manual of Mental Disorders - Fourth Edition (DSM-IV; American Psychiatric Association, 1994). Total scores can range from 0 (minimal depression) to 63 (high depression). To capture rapid changes in depressive symptoms, the BDI-II was modified to assess symptoms in the past week. Patients were asked to complete a BDI-II prior to each CT session.

Procedures
CT Session Ratings

We utilized all existing recordings of the first five sessions for each of the 53 patients who were included in our analyses. Raters largely used video recordings, substituting audio recordings in cases when a video recording was missing or of poor quality. Among the 53 patients examined, a total of 263 sessions occurred between session one and five, and recordings of 260 sessions (99%) were available for coding.

Ratings of CT sessions were provided by 19 advanced undergraduate students trained to use the HES-General and HES-CT. Raters were trained for a total of 40 hours over a 10-week period and then began making study ratings (with three additional meetings occurring to minimize rater drift). Three raters were randomly assigned to code each of the sessions with the constraint that each rater coded no more than one session per patient and that each rater was paired with all other raters an approximately equal number of times. To reduce the likelihood that non-homework activities would be erroneously counted as homework engagement, raters identified the upcoming homework assignments and type of homework assignments for each session and entered a brief description of the assignments in a homework assignment log. Raters who were assigned to rate the following session consulted the homework assignment log to know which assignments had been given the previous week. They then rated homework engagement using the HES-General and HES-CT, with the instruction to use the previous homework assignments to determine which activities reviewed during the session had been part of a previous homework assignment. To allow for the fact that patients may continue with a prior homework assignment for more than one between-session interval, raters were instructed to count any tasks completed that clearly were tied to therapy-related homework (e.g., thought records or daily activity logs), even if not assigned explicitly in the previous session. The average of raters’ judgments for each session was used in the primary analyses. To assess inter-rater reliability, we utilized a reliability estimator, G(q,k), that yields more precise estimations than intraclass correlation coefficients when raters and those being rated are neither fully crossed or nested (Putka, Le, McCloy, & Diaz., 2008). The inter-rater reliability for the HES-General was excellent, G(.28, 3) = .91, as was the inter-rater reliability for the HES-CT, G(.28, 3) = .94.

Analytic Strategy

To examine the relation of homework engagement between successive sessions and corresponding session-to-session changes in depressive symptoms, we used repeated measures regression analyses implemented in SAS Proc Mixed (without specification of random effects). The dependent variable in these models consisted of current BDI scores for each participant, with BDI scores from the previous session entered as a covariate. We disaggregated within-patient and between-patient variability in each homework engagement variable into two separate variables and examined these within-patient and between-patient variables as separate predictors. In each model where we examined a given homework variable (i.e., HES-General or HES-CT) as a predictor, we included two variables: the within-patient and the between-patient scores for that homework variable. Following the approach described by Curran and Bauer (2011), we used procedures inspired by those described in detail by Sasso et al. (2014). See Sasso et al. for additional discussion of theses models. Specifically, for each patient, we utilized a separate regression model in which session number (centered) was entered as a predictor of each homework engagement variable (i.e., HES-General or HES-CT). This step required ratings of the first session where homework was initially assigned plus a minimum of three homework ratings per patient (for a total of four sessions). To obtain between-patient scores for a given homework engagement measure, we used the intercepts of the patient-specific models. To obtain within-patient scores, we used the residuals of the patient-specific models. By definition, within-person values varied across sessions for each patient, while each patient had only one between-patient score for each homework engagement variable. We used an unstructured covariance structure because it was the best-fitting structure of the four we examined (unstructured, compound symmetry, autoregressive, and Toepliz), decided by Akaike’s Information Criterion (AIC).

Results

Descriptive Statistics

Across sessions one through five, BDI-II scores improved an average of 7.5 points (d = .82, t(49) = 6.11, p < .0001). The average session-to-session change (i.e., difference score) for the four intervals we studied ranged from 1.6 to 2.8, with the SDs ranging from 6.4 to 8.1. Our primary analyses involve identifying predictors of this session-to-session variability in BDI symptom improvement across these early sessions.

Homework assignments were rated in sessions one through four. Across these sessions, homework was assigned in 207 of 208 sessions (nearly 100%). The average number of homework assignments was 2.1 (SD = 1.1), with a range of zero to seven assignments. There was variability in the percentage of clients who received a particular homework assignment type at each session, with no homework type having fewer than 30% of patients assigned that type at any of the rated sessions. The most commonly assigned type of homework during these sessions was cognitive homework, which was assigned in 75% of the sessions. Self-monitoring was assigned in 59% of sessions, and behavioral homework was assigned in 47% of sessions. Combining the self-monitoring and behavioral homework categories, one of these forms of homework was assigned in 68% of the sessions rated.

Across sessions two through five, patients reported that they completed some homework in 89% of sessions. Using the item anchors to interpret the mean homework scores reported in Table 1, on average, patients completed the equivalent of a partially completed self-monitoring form, two partially completed thought records, and one behavioral task between each session. Additionally, as reflected in the item anchors, raters estimated that on average patients had spent “less than 30 minutes” on homework, “occasionally” used therapy skills when they felt sad or upset, and put “some effort” into completing homework assignments between sessions. The HES-General and HES-CT were significantly correlated with each other across sessions, with a mean correlation of .75.

Table 1.

Means and Standard Deviations for Homework Measures

Variable Mean SD
Primary Homework Measures
    HES-CT 2.0 0.73
    HES-General 4.2 1.30
Items from the HES-CT
    Cognitive 0.64 0.42
    Behavioral 0.34 0.30
    Self-Monitoring 1.0 0.58
Items from the HES-General
    Time 1.3 0.52
    Frequency 1.6 0.59
    Effort 1.3 0.42
Open in a new tab

Note. The means reported are the means of the patient average scores. HES-CT = Homework Engagement Scale—CT-Specific; HES-General = Homework Engagement Scale—General.

As discussed in the Analytic Strategy section, we disaggregated the within-patient and between-patient effects for each homework variable of interest. Each patient had one between-patient score and four within-patient scores (one for each session) for each homework variable. Using intraclass correlations coefficients of patient’s scores on each homework measure, we examined the percentage of the overall variability in homework engagement scores reflected between-patient variability (with the remaining portion being within-patient variability). Overall, only a relatively small portion of variability in homework engagement scores was due to between-patient variability (17% for both HES-General and HES-CT), with the remaining portion reflecting within-patient variability (83%).

Homework Engagement Variables and Session-to-Session Depressive Symptom Change

As noted above and shown in Table 2, within-patient and between-patient variables were included in each model as predictors of depressive symptom change. For HES-General, within-patient variability was significantly related to greater session-to-session depression symptom improvement, whereas between-patient variability did not significantly predict session-to-session depressive symptom changes. For the HES-CT, within-patient variability was significantly related to greater session-to-session depression symptom improvement. Surprisingly, there was a non-significant trend for between-patient variability in HES-CT scores to predict less marked session-to-session symptom change.

Table 2.

Within-Patient and Between-Patient Homework Engagement as Predictors of Session-to-Session Symptom Change

Within-Patient Between-Patient
Est(SE) β t Est(SE) β t
Models using Primary Homework Measures
    HES-General −.91(.38) −.1.18 −.2.40* −.07(.24) −.09 −.28
    HES-CT −.1.39(.68) −.98 −.2.03* .78(.40) .56 1.96†
Models using Items from Homework Measures
Est(SE) β t Est(SE) β t
HES-General
    Time −.1.59(1.06) −.74 −.1.50 .43(.58) .23 .74
    Frequency −.1.45(.87) −.82 −.1.68† −.58(.52) −.34 −.1.13
    Effort −.3.17(1.03) −.1.50 −.2.40** −.23(.73) −.10 −.31
    HES-CT
    Cognitive −.2.47(1.23) −.98 −.2.01* −.78(.68) −.32 −.1.15
    Behavioral −.10(1.55) −.03 −.06 1.45(.99) .42 1.47
    Self-monitoring −.1.37(.99) −.66 −.1.38 1.19(.46) .69 2.61*
Open in a new tab

Note. Each line shows results from a separate model, in which within-patient and between-patient scores on the line were entered as predictors into the same model. As described in the Analytic Strategy section, current session BDI served as the dependent variable with previous session BDI included as a covariate. Est is the estimate (i.e., regression coefficient) for each within-patient or between patient predictor. β is the estimate obtained in the same model when predictors were standardized to a mean of 0 and a SD of 1. These standardized estimates show the change in BDI points associated with a one SD increase in the predictor (at each session). HES-CT = Homework Engagement Scale—CT-Specific; HES-General = Homework Engagement Scale—General. For each model, df = 51.

†

p < .1

*

p < .05

**

p < .01.

To better understand the relationships we observed, we conducted a set of secondary analyses to examine within-patient and between-patient variation in individual homework items as predictors of concurrent depressive symptom change (see Table 2). Within-patient variability in effort and cognitive homework engagement were the only significant predictors of symptom improvement. There was a nonsignificant trend for the relation of within-patient variability in frequency of using skills and depressive symptom improvement. With regard to between-patient variation in our predictors, only self-monitoring exhibited a relation. Self-monitoring was related to less marked symptom improvement across sessions. Taken together, these analyses suggested that the significant relation of within-patient variability in HES-CT and greater depressive symptom improvement was driven by cognitive homework engagement. The significant relation of within-patient variability in HES-General and depressive symptom improvement was driven by effort that patients put into homework (and perhaps the frequency of using therapy skills). Similarly, the trend level relation of between-patient HES-CT and less marked symptom improvements was driven primarily by self-monitoring homework engagement.1

Discussion

Consistent with our expectations, across early sessions of CT, between-session intervals in which patients showed greater homework engagement were marked by greater session-to-session symptom improvement. These findings were similar when homework engagement was assessed generally and when it was assessed by examining CT-specific homework. The similarity of findings is understandable given the relatively high correlation between these two homework scores. Our evidence of a relation of within-patient variability in homework engagement and symptom improvement is consistent with previous research documenting a relationship between homework and treatment outcome across mood and anxiety disorders (Kazantzis et al., 2000; 2010). Due to the methodological approach used in this study, we can rule out the possibility that the relationship between homework engagement and depressive symptoms were accounted for by stable patient characteristics. In addition, whereas previous studies have often measured patients’ homework activities in relation to the assignments given (e.g., percentage completed), we examined homework variables that characterize the absolute amount of homework completed. Consequently, our results provide a direct test of the relation of patients’ use of homework and symptom improvement.

Interestingly, secondary analyses of the within-patient variability in the individual types of CT-specific homework engagement indicated patients’ engagement in more cognitive homework and putting more effort into homework were each associated with greater depressive symptom improvements. While these analyses should be interpreted with some caution given the naturalistic variability in therapists’ use of homework assignments, these findings suggest that patients’ engagement in cognitive homework may be important to patient improvement in early CT sessions. This finding is consistent with the idea that patients’ homework efforts pay off in the short term (i.e., in a single between-session interval).

While we were primarily interested in the relation of within-patient homework scores and symptom change, it is worth considering how to interpret the results for the between-patient scores. There was a nonsignificant trend for patients who tended to engage in more CT-specific homework to exhibit less marked improvement in depressive symptoms. A closer examination of these results indicated that these findings were particularly driven by self-monitoring homework. Given that within-patient variability in CT-specific homework was significantly associated with greater improvements in depressive symptoms, we consider two possible explanations. First, a subset of patients may have been simply more likely to complete higher levels of CT-specific homework, especially self-monitoring, and were also less likely to improve as quickly as other patients. Alternatively, a second possibility is that therapists’ assignment of self-monitoring homework may have been due to the extent to which patients had trouble completing earlier assignments and were not improving as rapidly as hoped. To be clear, the relationship of between-patient variability in homework engagement and depressive symptom does not fundamentally alter the interpretation of our findings involving within-patient scores. Within-patient variability in homework engagement and concurrent session-to-session improvements in depressive symptoms were significantly related. This pattern does remind us that had we not parsed within- and between-patient variability, we would have failed to distinguish patient effects (which could be explained by stable patient characteristics) from the effects due to variability in homework engagement.

While our study focused on the role of homework engagement, it is important to recognize that a number of other potentially important psychotherapy processes may be important to consider. These include variables characterizing therapist behaviors such as therapist competence and therapist adherence. Some aspects of both variables are likely relevant to promoting homework engagement. Therapist competence has been found to predict subsequent symptom improvement, with at least one study showing that a single item assessing therapist behaviors related to homework also predicted symptom improvements (Strunk et al., 2010b). Future research is needed to provide more fine-grained characterizations of the specific therapist behaviors that might be useful in promoting patients’ homework engagement.

Much as competence can be distinguished from adherence by its assessment of quality as compared to quantity, one could similarly assess not only the quantity of homework engagement, but also its quality. Looking beyond studies of treatments for depression specifically, some research has addressed this issue. While Rees and colleagues (2005) failed to find that quality of thought record completion predicted outcomes for depression or anxiety symptoms above and beyond homework quantity, Schmidt and Woolaway-Bickel (2000) found quality of homework was a stronger predictor of outcomes than percentage of homework completed in their study of CBT for panic disorder. Additional research is needed to clarify whether it is the quality or quantity of homework that better predicts therapeutic outcomes.

There are several limitations of this study that warrant discussion. First, similar to previous studies, examination of concurrent changes in homework engagement and depressive symptom change do not rule out the possibility of a reverse causal relation; thus, it is possible that a relation between homework engagement and depressive symptom change could reflect improvements in depressive symptoms helping patients engage in more homework. However, to rule out this possibility of reverse causality in an observational study of CT, one would need repeated assessments of homework engagement and depressive symptoms within each between-session interval. Future studies using ecological momentary assessments could be used to achieve this goal. Second, because we examined only early sessions in CT, it is unclear whether the findings would generalize to later in treatment. While Startup and Edmonds (1994) used a different definition of early treatment (only the first two sessions instead of the first five), they did find early CT homework compliance was related to treatment outcome whereas homework compliance in the remainder of treatment was not related to outcome. Third, our analyses of specific homework types should be interpreted with caution. If homework assignments were made systematically across therapist-patient dyads as a function of third variables, the relation of CT specific homework and symptom change could partly reflect the impact of therapists’ selection of homework assignments. For our analysis of between-patient variables, any stable patient characteristic that informed therapists’ use of homework assignments could have contributed to the relations identified (as we discussed for self-monitoring above). For our analysis of within-patient variables, only time-varying covariates could bias the relations of interest. While we cannot rule out such a possibility, we have no reason to suspect such time-varying effects would have been present. Finally, because we utilized homework information obtained from session recordings, it is possible that patients may not have always discussed a completed homework assignment or therapists did not always conduct a thorough review of completed homework. While we lack data to address this possibility empirically, it is our impression that therapists were fairly comprehensive and little homework was likely to have been missed.

In closing, this study examined the relationship of within and between-patient variability in homework engagement and depressive symptom change over session-to-session intervals – a plausible interval to explore immediate effects of homework engagement. Our findings that within-patient variability in general and CT-specific homework engagement were related to concurrent depressive symptom improvements, particularly greater cognitive homework engagement and greater homework-related effort, are consistent with what one would expect if homework engagement contributed to the early symptom improvements patients experience in CT.

While the relations of general engagement and CT-specific homework with symptom change were quite similar in our study, the examination of individual items from these scales provided distinct information about the aspects of homework engagement that were most strongly related to depressive symptom improvement. Clinically, our results are consistent with the recommendation that patients should be encouraged to engage fully with homework assignments (Beck et al., 1979), perhaps particularly cognitive homework. Patients who manage to put considerable effort into their homework assignments are among those who benefit most from their early CT sessions. We encourage future investigations to examine specific therapist strategies for increasing the likelihood that clients maximally engage in completing homework assignments. Research in this area has been very limited to date (Bryant, Simons, & Thase, 1999). With a number of studies showing the benefits of homework engagement, this is a vital next step.

Highlights.

  • Examines the relationship of homework and session-to-session symptom improvements

  • CT-specific homework was related to improvement in depression symptoms

  • General homework (time, frequency, effort) also related to depression improvements

Acknowledgements

This project was supported by Award Number TL1RR025753 from the National Center for Research Resources. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Research Resources or the National Institutes of Health.

We thank our colleagues for making this research possible. Abby D. Adler, Andrew A. Cooper, Lizabeth A. Goldstein, and Elizabeth T. Ryan served with Laren R. Conklin as cognitive therapists and clinical interviewers. Abby D. Adler served as study coordinator. Daniel R. Strunk provided training in and supervision of clinical assessments and cognitive therapy.

Special thanks to those who served as raters of homework engagement; without their hard work and dedication, this research would not have been possible.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1

Our analytic strategy necessitated the removal of participants who dropped out of treatment early in the study. As dropout is also an outcome of interest, we examined whether early homework engagement variables predicted dropout using average scores on homework variables within the full sample of 67 participants. None of the homework variables significantly predicted dropout, though lower engagement in cognitive homework, Wald’s χ2 = 3.8, p = .05, b = −1.48, SE (.76), CI (.05; 1.01), and lower frequency of using skills to cope, Wald’s χ2 = 3.4, p = .07, b = −.89, SE (.49), CI (.16; 1.06), were related to a higher change of dropout at the level of nonsignificant trends.

References

  1. Adler AD, Strunk DR, Fazio RH. What changes in cognitive therapy for depression? An examination of cognitive therapy skills and maladaptive beliefs. Behavior Therapy. 2014 doi: 10.1016/j.beth.2014.09.001. Advance online copy. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Allison PD. Fixed effects regression methods for longitudinal data using SAS. Cary, NC: SAS Publishing; 2005. [Google Scholar]
  3. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington, DC: American Psychiatric Association; 1994. [Google Scholar]
  4. Beck AT, Rush AJ, Shaw BF, Emery G. Cognitive therapy of depression. New York: Guilford; 1979. [Google Scholar]
  5. Beck AT, Steer RA, Brown BK. Beck depression inventory manual. 2th edition. San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
  6. Bryant MJ, Simons AD, Thase ME. Therapist skill and patient variables in homework compliance: Controlling an uncontrolled variable in cognitive therapy outcome research. Cognitive Therapy and Research. 1999;23:381–399. [Google Scholar]
  7. Burns DD, Spangler DL. Does psychotherapy homework lead to improvements in depression in cognitive-behavioral therapy or does improvement lead to increased homework compliance? Journal of Consulting and Clinical Psychology. 2000;68:46–56. doi: 10.1037//0022-006x.68.1.46. [DOI] [PubMed] [Google Scholar]
  8. Cammin-Nowak S, Helbig-Lang S, Lang T, Gloster AT, Fehm L, Gerlach AL, Wittchen H. Specificity of homework compliance effects on treatment outcome in CBT: Evidence from a controlled trial on panic disorder and agoraphobia. Journal of Clinical Psychology. 2013;69:616–629. doi: 10.1002/jclp.21975. [DOI] [PubMed] [Google Scholar]
  9. Coon DW, Thompson LW. The relationship between homework compliance and treatment outcomes among older adult outpatients with mild-to moderate depression. American Journal of Geriatric Psychiatry. 2003;11:53–61. [PubMed] [Google Scholar]
  10. Cowan MJ, Freedland KE, Burg MM, Saab PG, Youngblood ME, Cornell CE, Czajkowski SM. Predictors of treatment response for depression and inadequate social support -- The ENRICHD randomized clinical trial. Psychotherapy And Psychosomatics. 2007;77:27–37. doi: 10.1159/000110057. [DOI] [PubMed] [Google Scholar]
  11. Curran PJ, Bauer DJ. The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology. 2011;62:583–619. doi: 10.1146/annurev.psych.093008.100356. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. DeRubeis RJ, Webb CA, Tang TZ, Beck AT. Cognitive therapy. In: Dobson KS, editor. Handbook of cognitive-behavioral therapies. 3rd ed. New York, NY: Guilford Press; 2010. pp. 277–316. [Google Scholar]
  13. Detweiler-Bedell JB, Whisman MA. A lesson in assigning homework: Therapist, client, and task characteristics in cognitive therapy for depression. Professional Psychology: Research and Practice. 2005;36:219–223. [Google Scholar]
  14. First MB, Spitzer RL, Gibbon M, Williams JBW. Structured clinical interview for DSM-IV-TR axis I disorders, research version, patient edition. (SCID- I/P) New York: Biometrics Research, New York State Psychiatric Institute; 2002. [Google Scholar]
  15. Kazantzis N, Deane FP, Ronan KR. Homework assignments in cognitive and behavioral therapy: A meta-analysis. Clinical Psychology: Science and Practice. 2000;7:189–202. [Google Scholar]
  16. Kazantzis N, Deane FP, Ronan KR. Assessing compliance with homework assignments: Review and recommendations for clinical practice. Journal of Clinical Psychology. 2004;60:627–641. doi: 10.1002/jclp.10239. [DOI] [PubMed] [Google Scholar]
  17. Kazantzis N, Lampropoulos GK. The use of homework in psychotherapy: An introduction. Journal of Clinical Psychology. 2002;58:487–488. doi: 10.1002/jclp.10026. [DOI] [PubMed] [Google Scholar]
  18. Kazantzis N, Whittington C, Dattilio F. Meta-analysis of homework effects in cognitive and behavioral therapy: A replication and extension. Clinical Psychology: Science and Practice. 2010;17:144–156. [Google Scholar]
  19. Neimeyer RA, Feixas G. The role of homework and skill acquisition in the outcome of group cognitive therapy for depression. Behavior Therapy. 1990;21:281–292. doi: 10.1016/j.beth.2016.08.013. [DOI] [PubMed] [Google Scholar]
  20. Persons JB, Burns DD, Perloff JM. Predictors of dropout and outcome in cognitive therapy for depression in a private practice setting. Cognitive Therapy and Research. 1988;12:557–575. [Google Scholar]
  21. Primakoff L, Epstein N, Covi L. Homework compliance: An uncontrolled variable in cognitive therapy outcome research. Behavior Therapy. 1986;17:433–446. [Google Scholar]
  22. Putka DJ, Le H, McCloy RA, Diaz T. Ill-structured measurement designs in organizational research: Implications for estimating interrater reliability. Journal of Applied Psychology. 2008;93:959–981. doi: 10.1037/0021-9010.93.5.959. [DOI] [PubMed] [Google Scholar]
  23. Rees CS, McEvoy P, Nathan PR. Relationship between homework completion and outcome in cognitive behaviour therapy. Cognitive Behaviour Therapy. 2005;34:242–247. doi: 10.1080/16506070510011548. [DOI] [PubMed] [Google Scholar]
  24. Sasso KE, Strunk DR, Braun JD, DeRubeis RJ, Brotman MA. A re-examination of process-outcome relations in cognitive therapy for depression: Disaggregating within-patient and between-patient effects. Psychotherapy Research. 2015 doi: 10.1080/10503307.2015.1026423. Advance online publication available. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Schmidt NB, Woolaway-Bickel K. The effects of treatment compliance on outcome in cognitive-behavioral therapy for panic disorder: Quality versus quantity. Journal of Consulting and Clinical Psychology. 2000;68:13–18. doi: 10.1037//0022-006x.68.1.13. [DOI] [PubMed] [Google Scholar]
  26. Startup M, Edmonds J. Compliance with homework assignments in cognitive-behavioral psychotherapy for depression: Relation to outcome and methods of enhancement. Cognitive Therapy and Research. 1994;18:567–579. [Google Scholar]
  27. Strunk DR, Brotman MA, DeRubeis RJ. The process of change in cognitive therapy for depression: Predictors of early inter-session symptom gains. Behaviour Research and Therapy. 2010a;48:599–606. doi: 10.1016/j.brat.2010.03.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Strunk DR, Brotman MA, DeRubeis RJ, Hollon SD. Therapist competence in cognitive therapy for depression: Predicting subsequent symptom change. Journal of Consulting and Clinical Psychology. 2010b;78:429–437. doi: 10.1037/a0019631. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Strunk DR, Cooper AA, Ryan ET, DeRubeis RJ, Hollon SD. The process of Change in cognitive therapy for depression when combined with antidepressant medication: Predictors of early intersession symptom gains. Journal of Consulting and Clinical Psychology. 2012;80:730–738. doi: 10.1037/a0029281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Young JE, Beck AT. Revision of the Cognitive Therapy Scale. Philadelphia, PA: Unpublished manuscript. University of Pennsylvania; 1988. [Google Scholar]

RESOURCES