Therapist Behaviors as Predictors of Immediate Homework Engagement in Cognitive Therapy for Depression

Abstract

Homework assignments are an integral part of cognitive therapy (CT) for depression, though facilitating homework engagement in patients with depression can be a challenge. We sought to examine three classes of therapist behaviors as predictors of homework engagement in early sessions of CT: therapist behaviors related to the review of homework, the assignment of homework, and efforts to help patients overcome obstacles to completing homework. In a sample of 66 depressed outpatients participating in CT, therapist behaviors involved in assigning homework predicted both CT-specific homework engagement and more general homework engagement. Therapist behaviors involved in homework review were not predictive of homework engagement. Our findings are consistent with the possibility that therapists’ emphasis of key elements of the homework assignment process enhances patients’ engagement in homework in early sessions of CT.

Homework assignments are an essential part of cognitive therapy (CT) for depression (Beck, Rush, Shaw, & Emery, 1979). In CT, therapists utilize homework to help patients practice using and integrating CT skills into their lives (Kazantzis & Lampropoulos, 2002). Common homework assignments in CT depression include self-monitoring activities to understand the connection between daily activities and mood; recording one’s automatic thoughts and developing alternative responses; and scheduling and engaging in activities to promote a sense of accomplishment or pleasure (Beck, 1995; Beck et al., 1979). Patients are encouraged to regularly practice skills learned in therapy through assigned homework; however, the extent to which they engage in homework varies considerably, often falling short of desirable levels (Kazantzis & Shinkfield, 2007).

Previous research has found that patients’ level of engagement with homework during CT is positively related to improvements in depressive symptoms. This relationship has been observed when symptom improvements were assessed over the duration of treatment (Kazantzis, Deane, & Ronan, 2000; Kazantzis et al., 2016; Mausbach, Moore, Roesch, Cardenas, & Patterson, 2010) and more immediately (i.e., from one session to the next; Conklin & Strunk, 2015). However, there is little empirical evidence regarding which therapist behaviors predict homework engagement, in CT or in other treatments. In this study, we examine therapist behaviors that might serve to predict homework engagement. We assess both general homework engagement and engagement with specific types of homework. We use the term homework engagement to refer to the frequency and extent of any homework activities (see Conklin & Strunk, 2015, for discussion of this conceptualization). It differs conceptually from compliance, which refers to the proportion of homework completed relative to what was assigned (see Primakoff, Epstein, & Covi, 1986). In considering candidate therapist behaviors that could predict homework engagement, we drew from both from the empirical evidence available and expert clinical opinion on the subject.

Empirical Evidence

To our knowledge, two previous studies of cognitive behavioral therapies for depression in adults have examined the relation of therapist behaviors and patients’ homework activities (Bryant, Simons, & Thase, 1999; Startup & Edmonds, 1994). In first of these studies, Startup and Edmonds (1994) examined the relation of patient ratings of therapist behaviors involved in promoting homework (i.e., providing a rationale, clear description, anticipating problems, and involving the patient in the process) and weekly therapist-rated homework compliance. In their sample of 25 patients in multimodal cognitive behavioral therapy, they failed to find evidence that any therapist behavior was associated with homework compliance. They suggested that ceiling effects in patients’ ratings of therapists may have limited their ability to detect the effects of interest.

In the second study, Bryant and colleagues (1999) examined observer ratings of therapist homework-promoting behaviors at three recorded sessions widely spaced over early, mid, and late-treatment periods in a sample of 26 patients participating in CT. Observers rated the quality of four therapist behaviors: (1) the review of previous homework, (2) the explanation of a rationale for assignments, (3) the clarity and tailoring of assignments to the patient’s reported problems, and (4) how well the therapist sought reactions to the assignment and helped the patient trouble-shoot possible problems that might affect compliance. They found a non-significant trend suggesting a possible relation of their overall measure of homework-related therapist behaviors and homework compliance (r = .24). An examination of individual items suggested that therapist’s review of the previous session’s homework was significantly positively associated with compliance (r = .39). After being dichotomized (due to low frequency), an item assessing therapists’ eliciting of reactions to the assignment and trouble-shooting obstacles also showed an association with compliance (r = .26).

While the focus of the current study is homework in CT for depression in adults, it is worth considering some important findings from the broader homework literature. One important experimental demonstration among behavioral medicine patients found that providing written instructions of the assigned homework led to higher compliance than verbal instructions alone (Cox, Tisdelle, & Culpert, 1988). In a study of depressed adolescents participating in CT, providing a strong rationale and allocating more time to assigning homework predicted subsequent compliance, particularly for those who were initially resistant. Following the first session, therapists’ rationale and eliciting reactions / troubleshooting obstacles predicted compliance among those who had been low in compliance initially (Jungbluth & Shirk, 2013).

Expert Clinical Opinion

While the empirical evidence reviewed above is limited, expert clinicians have offered numerous recommendations and guidelines for improving homework engagement (Addis & Jacobson, 2000; Beck et al., 1979; Detweiler & Whisman, 1999; Garland & Scott, 2002; Kazantzis et al., 2000; Kazantzis, Pachana, & Secker, 2003; Shelton & Levy, 1981; Tompkins, 2002). Among the variety of techniques suggested, we discerned three broad categories: (1) therapist behaviors associated with the review of homework; (2) therapist behaviors associated with the assignment of homework; and (3) therapist behaviors related to overcoming obstacles to completing homework assignments.

Although multiple strategies for using homework successfully have been suggested, experts have varied in the extent to which they have emphasized different strategies (Scheel, Hanson, & Razzhavaikina, 2004). When homework has been previously assigned, therapists are encouraged to review those homework assignments at the next session (Beck et al., 1979; Garland & Scott, 2002; Kazantzis et al., 2003), reinforce the patients’ efforts with praise, and assess the patients’ perceptions of the helpfulness of homework (Kazantzis et al., 2000; Kazantzis et al., 2003; Tompkins, 2002). Expert clinicians have also emphasized the value of clearly describing and assigning homework. Specific recommendations include: providing a rationale for the homework, devoting time in session to describing how to complete assignments, delineating clear and specific assignments, using in-session practice as needed, and ensuring that patients have a written reminder of their assignments (Beck et al., 1979; Detweiler and Whisman, 1999; Garland & Scott, 2002; Kazantzis et al., 2000; Kazantzis et al., 2003; Shelton & Levy, 1981; Tompkins, 2002). At the beginning of treatment when homework assignments are new to patients, these recommendations may be particularly important. Finally, some experts have recommended working with patients to identify and overcome potential obstacles that might impede homework engagement (Kazantzis et al., 2000; Tompkins, 2002). Therapist behaviors related to these three domains (assignment, review, addressing obstacles) were also identified as important to therapists’ competent use of homework by a panel of six experts in CT (see Schmidt, Strunk, DeRubeis, Conklin, & Braun, 2017).

This Study

In this study, we sought to identify therapist behaviors that may predict homework engagement in early sessions of CT for depression (specifically, in the first five sessions). We focused on early sessions of CT for three reasons: (1) previous research has suggested that much of the therapeutic progress made occurs early in the course of treatment (Strunk, Brotman, & DeRubeis, 2010); (2) we were interested in initial engagement, as we suspected that it is key to establishing a pattern of high levels of engagement throughout treatment; and (3) our assessments of therapist behaviors and homework engagement were labor intensive and therefore required a focused approach. We also planned to build on previous studies in which the authors assessed the relationship between in-session therapist behaviors and patient homework compliance that occurred prior to the next session (Bryant et al., 1999; Startup & Edmonds, 1994) by focusing on immediate consequents of therapist behaviors while utilizing a relatively large sample and leveraging repeated assessments to increase power. Based on our review of the existing literature, we evaluated three classes of therapist behaviors described above (viz., reviewing homework, assigning homework, and overcoming homework obstacles). We examine both general homework engagement and engagement with specific types of CT homework. We hypothesized that therapist behaviors in each of these domains would predict homework engagement as observed in patients’ discussions with therapists at the following session.

Methods

Participants

CT sessions utilized in this study were drawn from a naturalistic study of 67 adults with major depressive disorder (MDD; see Adler, Strunk, & Fazio, 2015; Conklin & Strunk, 2015). Because one patient discontinued participation prior to attending a session, the sample was necessarily reduced to 66. CT was provided by four doctoral students who were supervised by the second author (for more information about therapist training and the competence with which CT was provided, see Adler, Strunk, & Fazio [2015]). Study inclusion criteria were as follows: a diagnosis of current MDD as assessed by the Structured Clinical Interview for the DSM-IV (SCID; First, Spitzer, Gibbon & Williams, 1994) and agreeing to refrain from adding or modifying the dose of any psychiatric medications currently being used over the course of treatment. Individuals were excluded from the study if any of the following were present: Bipolar I or any psychotic disorder; substance dependence in the six months prior to intake; or a primary diagnosis other than MDD if it was judged to require treatment other than CT.

Of the 66 participants, 37 (56%) were female. The mean age was 36.3 (SD = 13.4, range: 18–69). The racial/ethnic make-up of the sample was 83% Caucasian (n = 55), 11% African American (n= 7), 5% Asian American (n = 3), and 1% were Hispanic/Latino (n = 1). A majority of the sample (91%) had at least some college education, and 45% had graduated from a four-year college. Two-thirds of the sample (67%, n = 44) were diagnosed as having at least one prior depressive episode, and 61% (n= 40) of the sample met criteria for one or more co-morbid anxiety disorders, with generalized anxiety disorder being the most common (33%, n = 22). The majority (79%, n = 52) attended the first five sessions, with the remainder of the patients discontinuing treatment prior to session five. All available data from participants were used in analyses.

Measures

Homework-Specific Therapist Behavior Scales (HSTBS)

Based on our review of the literature summarized in the introduction, we developed 16 observer-rated HSTBS items to assess three classes of therapist behaviors that may promote homework engagement: Review, Assignment, and Overcoming Obstacles. The first and second authors reviewed this literature and identified key classes of therapist behaviors thought to promote homework engagement. The first author drafted specific items and the second author, who has extensive experience in research utilizing observer ratings of psychotherapy, provided feedback regarding the degree to which the items covered the relevant therapist behaviors as well as suggesting any revisions to facilitate reliable coding of the behaviors by observers. The items were then revised to address these concerns. On the HSTBS, each scale item is rated from zero (not at all) to four (considerable), and then all items on each scale are averaged to create the scale score. In this study, three raters (drawn from a team of 17 raters; see the “ratings of homework-related characteristics” section below for details) used the HSTBS to evaluate each study session; thus, inter-rater reliability for each scale is reported below.

The Assignment scale consists of seven items assessing therapist behaviors thought to be important when assigning homework. The items assess the extent to which: (1) the therapist discussed the rationale for upcoming homework, (2) described how to do the tasks assigned for homework, (3) practiced the skills needed to do the homework, (4) ensured the patient understood how to perform the homework, (5) assigned specific homework, (6) clarified which activities or recommendations were homework assignments, and (7) ensured the patient had a written record of the homework. The Assignment scale had a median Cronbach’s alpha of .67 across sessions. The intraclass correlation coefficient (ICC; adjusted for three raters; see McGraw & Wong, 1996) for the Assignment scale was .75.

The Review scale consists of four items assessing therapist behaviors thought to be important in reviewing homework. These items assessed the extent to which therapist: (1) reviewed the homework, (2) involved the patient in reviewing homework, (3) discussed the helpfulness of previous homework, and (4) praised the patient for working on homework. The Review scale had a median Cronbach’s alpha of .76 across sessions. The ICC (adjusted for three raters) for the Review scale was .86.

Finally, the Overcoming Obstacles scale consists of five items assessing therapist behaviors important in ensuring patients would know how to work around possible obstacles to completing homework assignments. Overcoming Obstacles items assessed the extent to which therapists discussed the following with the patient: (1) obstacles that impeded previous homework progress, (2) the ways identified obstacles could be minimized in future homework, (3) the ways possible obstacles to upcoming homework could be worked around, (4) the ways of helping the patient remember to make time to do the homework, and (5) the ways of challenging thoughts that might interfere with the upcoming homework. The Overcoming Obstacles scale had a median Cronbach’s alpha of .75 across sessions. However, the ICC (adjusted for 3 raters) for the Overcoming Obstacles scale was .43. This low ICC may have been due to restriction of range with many zero scores, which was not evident in the other two scales. Because of the low ICC, we limit our presentation of analyses of this scale to basic descriptive statistics and do not present other analyses involving this scale score.

Homework Engagement Scale—General (HES-General; Conklin & Strunk, 2015)

The HES-General is a three-item observer-rated measure of patients’ between-session homework engagement. The initial evaluation of the HES-General and the HES-CT (described below) were drawn from the same parent dataset (Conklin & Strunk, 2015) as the current study. Three raters (drawn from a team of 19 raters) used the HES-General and HES-CT to rate each session, and inter-rater reliability is reported for each scale below.

The HES-General scale includes the following items: (1) the estimated amount of time the patients spent on homework (time); (2) the frequency with which the patients reported using CT skills when they were sad or upset (frequency); and (3) the estimated effort the patients put into homework assignments (effort). Items were summed so that the range possible on this measure was zero to twelve, with higher scores indicating greater time, greater effort, and more frequent engagement in homework activities between sessions. The HES-General had a median Cronbach’s alpha of .83 across sessions. The ICC for HES-General was .63.

Homework Engagement Scale—CT-Specific (HES-CT; Conklin & Strunk, 2015)

The HES-CT is a three-item observer-rated scale that assesses the degree to which patients engage in commonly assigned types of CT homework. The homework assessed includes: cognitive homework (primarily thought records), self-monitoring homework (primarily use of a daily activity log), and behavioral homework other than self-monitoring (e.g., practicing assertiveness, planning activities designed to produce a sense of pleasure or mastery). While the HES-CT items were rated on a zero (no engagement) to six (extensive engagement) scale, screening of the items suggested non-normality due to a high percentage of zero values. This is consistent with conventional CT implementation, in which therapists do not typically assign all three types of homework in every session. To reduce the non-normality of the data, the non-zero ratings were recoded. Scores from one to three were coded as a one and scores from four to six were coded a two, thus resulting in scale items with values ranging from zero to two and HES-CT total scores ranging from zero to six. The ICC for HES-CT was .64.

Procedures

Ratings of CT

Video (or audio) recordings from the first five therapy sessions were coded for this study. Audio recordings were used on occasions where the video quality was too poor for adequate rating or when a video recording was not available. All available recordings were utilized. A total of 288 sessions occurred between session one and five, and recordings of 282 sessions (99%) were available for coding. No recordings were available for sessions taking place via phone (n = 4) or when a recording device failed (n = 2).

Ratings of homework-related characteristics

A total of 36 undergraduates were on one of two CT rating teams, and both teams were equivalent with respect to the duration and extent of training. Raters were trained over 10 weeks prior to beginning the rating of study sessions. Training consisted of weekly meetings with the first author to review rating materials and practice making ratings using recordings of CT sessions drawn from an independent study of CT. After the training, raters made study ratings over a subsequent 10 week period. During this period, raters met periodically to discuss any difficulties that had arisen in making study ratings. They also rated clips of CT sessions not from this study to practice making ratings as a group in an effort to reduce any rater drift.

Each study session was rated by three members of each team, with each member rating no more than one session per patient. The averages of raters’ scores were used in analyses. One team (n = 17) rated therapist behaviors in sessions one through four using the HSTBS. The second team of raters (n = 19) rated engagement in sessions two through five using the HES-General and HES-CT. To complete their ratings, both teams of raters needed to know what homework had been assigned in the previous session. To provide this information, the second team of raters created a log of homework assignments. Logs for the prior session were provided to aid raters in making their judgments for the current session.

Overall Analytic Strategy

To model the dependence among repeated measures, we conducted primary analyses using the SAS Proc Mixed without specification of random effects. We evaluated four covariance structures (viz., unstructured, compound symmetry, autoregressive, and Toeplitz) and selected the best-fitting covariance structure for each dependent variable examined. The best fitting covariance structure, decided by Akaike’s Information Criterion (AIC), was autoregressive for both the HES-General and HES-CT.

We used repeated measures regression analyses to examine therapist behaviors at each session as predictors of homework engagement at the next session while controlling for homework engagement at the current session. For each of the analyses, lagged engagement scores (viz., engagement scores from sessions 2–5) served as the dependent variable with engagement scores from the previous session entered as a covariate. Thus, engagement at session two served as a covariate in predicting engagement at session three, engagement at session three served as a covariate in predicting engagement at session four, etc. HSTBS scores at sessions one through four were entered simultaneously as predictors in separate models for each homework engagement variable. To aid interpretation, both predictors and dependent variables were standardized to a mean of zero and a standard deviation of one prior to these analyses.

Results

Descriptive Statistics for Therapist Behaviors and Homework

Therapist behaviors involved in homework review would not have occurred in the first session, as homework was not assigned prior to that session. Across the remaining sessions (2 through 5), the Review scale had an average score of 1.90 (SD = .82). Across sessions 1 through 5, the Assignment scale had an average score of 2.49 (SD = .58) and the Overcoming Obstacles scale had the numerically lowest average score of .45 (SD = .54). Review and Assignment, the two scales included in the primary analyses, were moderately positively correlated on average (r = .26: range: .12 to .47), but this relationship was only significant at one session.¹

Similar to the description provided by Conklin and Strunk (2015) for a subset of this sample, homework was assigned in 99% of the 230 sessions that were rated between sessions one and four. Among the three types of homework we examined, cognitive homework, consisting predominantly of thought records, was most commonly assigned (73% of the sessions). Self-monitoring, involving use of daily activity logs, was the second most frequently assigned homework type (60%). Finally, behavioral homework, which focused on trying new between session activities, was a common, but less frequently assigned type (45%).

Across sessions two through five, patients reported that they completed homework in 88% of sessions. The average amount of total CT-specific homework completed by patients between each session was 1.93 (SD = .82) out of a possible 6. Breaking these ratings down by the type of homework completed, the average scores indicated that patients completed “some” of each type of homework, which according to the anchors the raters used was equivalent to a partially completed activity log (M = 1.03, SD = .65), two thought records (M = .57, SD = .44), and two behavioral tasks (M = .33, SD = .31).

When examining ratings of general homework engagement (HES-General), patients received an average score of 4.06 (SD = 1.56) out of 12 points. Broken down by item and interpreting amount based on the scale anchors, average ratings indicate that patients spent approximately 30 minutes on homework (M = 1.28, SD = .60), “occasionally” used therapy skills when they felt sad or upset (M = 1.56 SD = .64), and put “some” effort into completing homework assignments (M = 1.22, SD = .49).

Homework-Related Therapist Behaviors as Predictors of Session-to-Session Homework Engagement

Using a separate model for each dependent variable, we examined both Review and Assignment as predictors of HES-CT and HES-General. For the model of HES-CT scores, the HSTBS Assignment scale significantly predicted homework engagement, t(93) = 3.41, β = .36, p = .001, but the HSTBS Review scale did not, t(93) = .78, β = .10, p = .43. Similarly, for the model of HES-General, the HSTBS Assignment scale emerged as a significant predictor, t(93) = 3.83, β = .41, p = .0002, whereas HSTBS Review scale did not predict HES-General scores, t(93) = −.28, β = −.04, p = .78.

Discussion

In this study, we examined specific types of therapist behaviors as potential predictors of patients’ session-to-session homework engagement across early sessions of CT for depression. The consistent finding was that the scale aggregating therapist efforts to assign homework was the numerically strongest predictor of homework engagement. Assigning homework predicted both measures of general and CT specific homework engagement. These findings highlight the importance of therapist efforts to encourage patients to practice new skills through their assignment of homework: by providing a compelling rationale, a thorough description of any homework assigned, and ensuring a clear understanding on the part of the patient.

Contrary to expectation, homework review did not predict subsequent homework engagement, neither in the model assessing predictors of general homework engagement nor the model assessing predictors of CT-specific homework engagement. This is in contrast to two previous studies that found some evidence that the competence with which a therapist reviewed homework predicted next session homework compliance (Bryant et al., 1999; Weck et al., 2013). The measures and methods used varied considerably from those of the current study, which we suspect played a role in the differing findings. We consider three key differences between previous studies and the current one. First, there was a difference across studies in which sessions were rated. We rated early sessions, whereas Bryant et al. (1999) and Weck et al. (2013) examined sessions spread out across the course of treatment. It is possible that therapists establish a pattern of reviewing homework more frequently for patients who engage in homework regularly. If so, such patterns in later sessions may have contributed to the stronger evidence of a relation of homework review and compliance in studies that included later sessions than our own study. Second, both Bryant et al. and Weck et al. relied on assessments of the competence of homework review. It is unclear how strongly assessments of competence would correlate with measures of the extent to which a therapist engaged in specific behaviors involved in homework review. Finally, rather than homework engagement, authors of both of these previous studies assessed homework compliance. Unlike our assessments of homework engagement, assessments of homework compliance involve measuring the patients’ homework relative to what was assigned. As we have argued previously, this can sometimes lead to high compliance scores when small amounts of homework were completed and low compliance scores when moderate or even relatively high amounts of homework were completed (see Conklin & Strunk, 2015).

Limitations

We wish to acknowledge some limitations of this study. First, the Overcoming Obstacles scale exhibited poor inter-rater reliability and was therefore not used in subsequent analyses. As we mentioned, this may have been due to a restriction of range (i.e., a floor effect) that made it difficult for raters to reliably observe the infrequent occasions when therapists exhibited these behaviors. Informed by this evidence that therapist behaviors related to overcoming obstacles may be relatively less frequent or more difficult to identify suggests the need to refine our measure or bolster rater training in future research. Second, our ability to detect the relations of interest depends on both adequate representation of variability in the constructs of interest in the sample as well as the ability of our measures to capture the range of these behaviors. Insofar as some of our measures were created for this study, it is difficult to assess the extent to which this was present. Insofar as larger number of items may enhance a scale’s reliability and validity, one might expect that the Assignment scale would have an advantage over the Review Scale. Nonetheless, it is worth noting the two prior studies that found relationships between homework review and compliance (Bryant et al.,1999; Weck et al., 2013) utilized one-item assessments of homework review. Third, therapists involved in the project were therapists in training. The relations we studied may vary as a function of therapist expertise in providing CT. However, evaluations of therapist competence in this study suggested that competence appeared roughly on par with that observed among clinicians providing CT in clinical trials (Adler, Strunk, & Fazio, 2015). Fourth, our decision to focus on early sessions means we do not know whether our findings would generalize across the course of CT. Finally, and most importantly, it is important to note that this study was not experimental. Therefore, we cannot rule out the possibility that our findings might be accounted for by an unmeasured third variable.

Conclusion

Although the use of between-session homework activities has been a key component of CT for depression since its inception (Beck, Rush, Shaw, & Emery, 1979), clinicians have thus far had to largely rely on expert advice and clinical intuition to determine what elements of the homework process are important to emphasize to increase the likelihood that patients complete homework. Should one spend the majority of effort in session reviewing patients’ previous homework and helping them learn from past experience with the homework? Should one instead emphasize the practicing of new skills and ensure that the homework related to those skills is specific and understandable? How one answers these questions plays a key role in determining session structure and flow. We hope that the findings of this study help to facilitate making such decisions on the basis of empirical evidence.

To provide such evidence, we examined the relation of homework assignment and review-related therapist behaviors to general homework engagement and engagement with specific CT homework assignments. Therapist actions involved in assigning homework emerged as the more robust and numerically stronger predictor of both forms of homework engagement. Therefore, helping patients understand the nature of any homework assignments given (by describing the homework clearly, practicing it, ensuring patients have a written record of the assignments, etc.) appears to be a plausible determinant of session-to-session homework engagement. Our findings suggest that homework engagement could be enhanced when therapists emphasize key elements of the homework assignment process in early sessions of CT for depression.