What is the effect of homework engagement in group cognitive behavioral therapy for anxiety disorders and depression?

Abstract

Background

Homework is integral to cognitive behavioral therapy (CBT) treatment programs. Previous research has reported mostly positive findings regarding the effect of homework adherence on CBT outcomes, but only limited research has evaluated the effect in transdiagnostic CBT (tCBT).

Methods

This secondary study used data from 164 patients with major depressive disorder, social phobia, agoraphobia, or panic disorder, randomized to 14 sessions of diagnosis-specific CBT (dCBT) in groups or group tCBT (Unified Protocol). The number of times patients engaged in homework assignments was measured with a single-item homework engagement assessment. We evaluated the effects of total mean homework engagement on symptom outcomes using the Hopkins Symptom Checklist at the end of therapy, and the effect of intervention arm and diagnosis on homework engagement across three periods with repeated measures analysis of variance. Finally, we used cross-lagged panel modeling (CLPM), with the inclusion of baseline covariates and interaction variables, to investigate whether homework engagement predicted next-session symptom severity as measured with the Overall Depression Severity and Impairment Scale and the Overall Anxiety Severity and Impairment Scale.

Results

Patients’ general homework engagement significantly affected their symptoms at end of treatment (F_1,162: 3.944, p = 0.05), and had a significant cross-lagged effect on next-week symptoms (coefficient = − 0.23, se = 0.11, z = -2.16, p = 0.03, 95% CI [-0.44, − 0.02]). Initial analysis indicated that homework engagement was higher in dCBT than in the UP, and patients with depression in dCBT conditions were more engaged in homework assignments in the first period of therapy. However, these differences did not affect the overall treatment outcome in repeated measures or CLPM analyses.

Discussion

This is the first study to examine the effects of homework engagement on symptoms in group UP and also adds to the limited literature using such methods to isolate the unique effects of treatment engagement on symptom outcomes. Future studies should confirm these results and investigate other relevant aspects of homework engagement, such as the quality of said engagement and process variables such as therapist factors, group cohesion, and negative effects of homework assignments.

Background

Cognitive behavioral therapy (CBT) is recommended as a first-line treatment for anxiety and depression due to its extensive evidence base [1]. CBT protocols emphasize the use of between-session homework assignments for practicing and solidifying skills acquired in therapy. Furthermore, practicing in ‘real-life settings’ provides an opportunity to apply the new skills and ideally experience sustained symptom relief [2]. Homework in this context has been defined as ‘specific, structured, therapeutic activities that are routinely discussed in session, to be completed between sessions’ [3]. CBT therapists can assign many types of homework to patients. Broadly, these can be categorized as: 1) Psychoeducation, which often includes reading and educating oneself on the nature of one’s disorder; 2) self-assessment, e.g., mood monitoring, recognizing patterns in emotions, thoughts, and behaviors and their interconnectivity; and 3) modality-specific homework that pertains to the particular protocol and patient group, e.g., exposure-related homework for anxiety disorders [4]. It is hypothesized that engaging in homework assignments would lead to immediate symptom improvements in therapy [3, 5]. Homework engagement has been studied using quantity and quality as its indices [6, 7]. In this context, quantity refers to the amount of homework completed or partially completed, the amount of time involved, or the number of practice sessions undertaken to meet the homework assignment, whereas quality refers to the depth of engagement or acquired learning from homework assignments. Homework engagement can also be studied in terms of the patients’ engagement with homework in general or patients’ engagement with specific CBT assignments [6].

Previous research has identified engagement in homework as an important factor in treatment outcomes and lack of engagement as an important predictor of CBT non-response [8]. Studies comparing CBT protocols that include homework to those that do not have found larger effect sizes for the former [9, 10]. In addition, early homework engagement is associated with overall therapy engagement [11]. Meta-analytic studies in samples of patients with mood disorders [7, 10, 12] have consistently found both engagement in and the quality of homework to significantly predict patients’ symptoms at the end of the intervention. The most recent meta-analysis by Kazantzis et al. (2016) [8] found that both the quality and quantity of homework had a significant effect at the end of the treatment period (quality Hedges’ g = 0.78, 95% confidence interval [CI] = 0.03 to 1.53; quantity g = 0.79, 95% CI = 0.57 to 1.02) and at follow-up (quality g = 1.07, 95% CI = 0.06 to 2.08; quantity g = 0.51, 95% CI = 0.28 to 0.74) [8] across 17 studies on CBT (N = 2,312 clients). In line with these findings, a systematic review on the impact of homework engagement in CBT for OCD [13] found a “consistent medium to large significant association between CBT task adherence and post-treatment OCD symptom reduction”.

However, previous research on the importance of homework engagement has mainly relied on therapist-rated measures of overall engagement in homework post-treatment [14]. One may question the reliability of therapists retrospectively accounting for homework engagement, i.e., therapists’ estimates of it might become inflated as patients achieve more positive outcomes. Therefore, session-to-session ratings of engagement and symptom change may be a better way to understand the impact of homework on treatment outcomes [15]. Several studies of this type exist. Yovel and Safren (2007) [16] found no significant relationship between session-to-session homework engagement and symptom change among patients (N = 16) in individual CBT for attention deficit hyperactivity disorder (ADHD). Olatunji et al. (2015) [17] found that homework engagement in individual CBT predicted symptom change in the following session for young adults (N = 27) with obsessive-compulsive disorder (OCD), while Strunk et al. (2010) [18] reported that adherence to behavioral methods and homework in individual CBT predicted session-to-session symptom change among depression patients (N = 60). Schmidt and Woolaway-Bickel (2000) [19] found that homework engagement in individual CBT was positively associated with a change in panic disorder symptoms (N = 48). Conklin and Strunk (2015) [6] showed that observer-rated homework engagement predicted session-to-session change in symptoms among patients (N = 53) with depression receiving individual CBT. Finally, Yee et al. (2021) [20] found no effect of homework engagement on the next session’s depression scores among patients with depression (N = 50) in individual CBT.

Furthermore, the effect of homework engagement on treatment outcomes has primarily been discussed for diagnosis-specific CBT (dCBT) protocols [7]. In the past decade, transdiagnostic CBT (tCBT) protocols have gained momentum due to their potential benefits for the simultaneous treatment of comorbid disorders [21, 22]. tCBT interventions target underlying shared psychopathologies across related disorders and apply the same treatment principles without tailoring the interventions for specific symptoms [23]. The Unified Protocol for Transdiagnostic Treatment of Emotional Disorders (UP) developed by David Barlow and colleagues [24] is one of the best empirically supported tCBT protocols. The UP may even have advantages compared to dCBT protocols, since therapists can be trained to treat the full range of emotional disorders using a single treatment manual, and therapy groups can be formed in order of referral to the clinic instead of waiting for enough patients with the same diagnosis to start a group [25]. A single study by Conklin et al. (2021) has investigated the effect of homework engagement on treatment outcomes across the UP and dCBT programs [26]. They found that homework engagement and quality significantly predicted both self-reported and clinician-rated positive outcomes, but the effect on symptoms was not affected by treatment modality when comparing individual UP and individual dCBT for anxiety disorders and OCD [26].

The present study aimed to examine the impact of homework engagement on treatment outcomes in the UP and in direct comparison with dCBT protocols. Furthermore, it aimed to evaluate the the potential differences in the mediating/moderating effects of the homework across treatment type. In addition, while multiple researchers have examined the effect of CBT homework engagement in group therapy [27], this study is the first to explore the impact of homework engagement on outcome in the context of UP group therapy. Lastly, we conducted a mediation analysis to explore whether treatment protocol affects symptom improvement and whether this association is mediated by homework engagement.

We examined (a) patients’ engagement in homework in dCBT and UP group treatments; (b) the impact of diagnosis and intervention on the quantity of homework engagement; and (c) the association between patients’ engagement in homework assignments and symptom outcomes at end of therapy. We then isolated the unique effects of homework engagement on treatment outcomes using cross-lagged panel modeling (CLPM) to separate (d) between-persons and within-person effects and (e) effects from session to session. The within-person level allows for exploring the temporal changes in a potential effective characteristic of therapy relative to changes in treatment outcomes and is considered the appropriate level of analysis for understanding how psychological treatments assert their effects [28, 29]. Based on Kazantzis et al. (2016) [7] and Conklin & Strunk’s (2015) [6] definitions, we use the term ‘homework engagement’ to describe the quantity of patients’ general engagement in homework during treatment.

Considering previous research, we hypothesized that (1) the homework engagement between sessions would predict more symptomatic improvement at the end of treatment in both dCBT and UP conditions. Further, we hypothesized that (2) patients with higher scores on the homework engagement measure would have lower anxiety and depression ratings at the following session.

Methods

Context

The current research is a secondary analysis of a randomized controlled trial that included 291 participants with major depressive disorder, panic disorder, agoraphobia, or social phobia. The trial tested the relative effects of the transdiagnostic UP [22, 30] in a group format and dCBT in a group format, evaluated by participants’ subjective well-being, symptoms, and psychosocial functioning at the end of treatment. The results indicated non-inferiority between the interventions on all outcomes.

The main trial was registered on the ClinicalTrials.gov website (ID NCT02954731). See [22, 30] for details of the main study.

Participants

This study included the 164 treatment completers (i.e., patients who attended seven or more of the 14 group therapy sessions) with a primary diagnosis of major depressive disorder, panic disorder/agoraphobia, or social phobia. The Danish Mental Health Services both operate inpatient and outpatient facilities. Participants were treated in outpatient clinics in the free public Danish Mental Health Services. To obtain a referral for these clinics, patients must generally have attempted treatment with medication and/or psychotherapy in the primary sector first or have moderate-to-severe symptoms and psychosocial dysfunction.

Procedure

Patients in the main study were randomly assigned to receive treatment in transdiagnostic groups, following a Danish adaptation of the UP for groups, or in dCBT groups for depression, social anxiety, or panic disorder/agoraphobia. The dCBT was administered following three evidence-based treatment protocols for the specific disorders [31, 32, 33]. See Table 1 for the content of the individual sessions and see the protocol and main paper of the main trial for further details [22, 30].

Table 1.

Overview of homework assignments in the applied CBT protocols

Period/session	stdCBT group for DEP	stdCBT group for SAD	stdCBT group for PA/Ago	UP group
Period 1
Session 1*	Problem & goal registration	Problem & goal registration	Problem & goal registration Reading psychoeducation material	Reading psychoeducation material Symptom monitoring
Session 2	Activity registration Evening therapy	Reading psychoeducation material. Response prevention Critical appraisal	Registration NAT/ catastrophizing Reading psychoeducation material	Goal setting Motivation enhancement exercise Symptom monitoring
Session 3	Activity registration Mood diary	Registration NAT Situation analyses	Situation analysis. Alternative thoughts	Registering emotions in their context Symptom monitoring
Session 4	Activity planning Distraction techniques	Registration self-focus Situation analyses	Daily interoceptive exposure Situation analysis	Daily mindful emotion awareness exercise 3– point check Symptom monitoring
Period 2
Session 5	3-column situation analysis	Registration avoidance & security Behavior Small exposure tasks	Daily exposure Cognitive re-structuring and response prevention	Registration of NAT and Cognitive Flexibility Symptom monitoring
Session 6	Situation analysis/NAT Quantification of belief	Daily exposure & report	Diverting attention from body in exposure Graduated exposure	Practice countering emotional behavior Symptom monitoring avoidance
Session 7	3-column analysis repetition	Behavior experiments 5-column analysis	Behavior experiments Daily exposure & report	Interoceptive exposure exercises Symptom monitoring
Session 8	4-column analysis and alternative thoughts	Daily exposure & report	Behavior experiments Daily exposure & report	Emotion exposure exercises Symptom monitoring
Session 9	4-column analysis and alternative thoughts	Daily exposure & report	Behavior experiments Daily exposure & report	Emotion exposure exercises Symptom monitoring
Period 3
Session 10	Behavior experiments Negative appraisal	Daily smalltalk exposure	Behavior experiments Daily exposure & report	Emotion exposure exercises Symptom monitoring
Session 11	Rating of strategies and 4-column NAT analysis	Strategies identification	Behavior experiments Daily exposure & report	Emotion exposure exercises Symptom monitoring
Session 12	Behavior experiments elliciting strategies	Old-new strategies pro & con	Strategies identification	Emotion exposure exercises Symptom monitoring
Session 13	Modification of strategies	Behavior change Daily exposure	Old-new strategies pro & con	Emotion exposure exercises Symptom monitoring
Session 14	Relapse prevention	Relapse prevention	Relapse prevention	Relapse prevention New goal setting

DEP: depression [32] SAD: social anxiety disorder [33]. Pa/Ago: panic disorder/Agoraphobia [31, 33]. UN: unified protocol for emotional disorders [65]. NAT: negative automatic thoughts. *) homework at the first session was planned at the individual group Preparation session two weeks prior to group onset

Patients in both conditions received 14 sessions (two hrs. each) in groups of eight participants, led by two therapists. The therapists were licensed psychologists (N = 38) and other professionals trained in psychotherapy (N = 19) [22]. All sessions in the UP and dCBT groups included homework assignments. A brief outline of homework content described in each session for the four protocols is given in Table 1.

Homework engagement was evaluated with a single-item measure at the beginning of each group session, examining the extent to which the patients had engaged with the homework assignments between sessions. Baseline data encompassed participants’ sex, age, civil status, education, and current employment status.

Outcome assessments

Hopkins symptom checklist (HSCL-25)

The HSCL-25 is a widely used questionnaire of self-reported psychological distress, or more specifically, symptoms of anxiety and depression [34, 35]. It is a brief version of the original SCL-90 scale and includes an anxiety subscale (10 items), a depression subscale (13 items), and a somatization subscale (2 items). The items concern the amount of distress experienced in the last week, scored on a 5-point Likert scale from 0 = Not at all to 4 = Extremely bothered. The total score is calculated as the mean value of all 25 items, and a cut-off score of > 1.75 has been recommended as a valid predictor of a psychiatric illness. The HSCL-25 version has demonstrated good psychometric properties and has been validated in several Nordic countries, including Denmark [35, 36, 37, 38, 39].

Overall depression severity and impairment scale (ODSIS) and overall anxiety severity and impairment scale (OASIS)

The Overall Depression Severity and Impairment Scale (ODSIS) and the Overall Anxiety Severity and Impairment Scale (OASIS) are short, self-administered instruments that are designed to assess the severity of depression and anxiety symptoms, respectively, and their interference with psychosocial functioning [40, 41]. There are five items per questionnaire, each of which is rated based on the past week and scored on a 5-point Likert scale ranging from 0 to 4. A high score indicates a higher level of depressive or anxious symptoms and interference with daily functioning. The ODSIS has been validated in Danish, where it was found to have robust reliability and validity, accurately measuring both depression and functional impairment [42]. The OASIS is not yet validated in Danish, but one psychometric study has been performed for the English edition, which found robust correlations with global and disorder-specific measures of anxiety [40]. We averaged ODSIS and OASIS scores for each session into a single transdiagnostic combined OASIS/ODSIS score. As a collapsed instrument, the ODSIS/OASIS score is calculated as the mean of all 10 items.

Homework engagement measure

The homework engagement outcome measure was purpose-made for the current study [30]. This single-item, pen-and-paper, self-reported measure assesses patients’ homework engagement during the preceding week by asking, “To what extent have you worked on the homework from the last session?” This item is assessed on a 5-point Likert scale: 1 = I have not thought about the homework and did not work on it/I did not want to work on it; 2 = I have tried to do my homework, but I was unable to do so; 3 = I have worked on the assignments less than half of the days; 4 = I have worked on the assignments more than half of the days; 5 = I have worked on the homework every day. For statistical analysis, a ‘No fill-in’ on the measure was set as missing data. An average homework engagement score was calculated for the first, middle, and last periods of therapy: Period 1 (sessions 1–4), Period 2 (sessions 5–9), and Period 3 (sessions 10–14).

Data analysis

Statistical analyses were conducted using IBM SPSS Statistics version 27 and Mplus version 8, with significance level set to p < 0.05.

We calculated patients’ mean scores on the HSCL-25 at beginning and end of treatment and tested for a significant difference between groups using one-way ANOVA for main effects of treatment arm, and repeated measures analysis of variance (rmANOVA) for co-variate and exploratory co-factor analyses. The general effect of homework (HW) engagement on outcome was tested by adding the mean HW engagement across the full intervention period as a co-variate in the rmANOVA of HSCL-25 (baseline; end of treatment). We explored the effect of intervention arm and diagnostic category (depression or anxiety disorder) on mean homework engagement within three pre-defined periods of the psychotherapy course by including diagnostic category and intervention arm as co-factors in an rmANOVA of mean homework engagement score within the three periods (mean of beginning, middle and end period). Post-hoc, due to a significant triple interaction effect of Period×Diagnostic category×Intervention arm, we split the sample by diagnostic category to illustrate the development in HW engagement across the three periods and in each intervention arm individually.

Finally, we used cross-lagged panel modeling to test whether homework engagement changes in one session predicted symptom change, assessed with the composite ODSIS/OASIS score, in the following week. In the first step, we compared the fit of five different models to the data: the dynamic panel model (DPM) [43], the random intercept cross-lagged panel model (RI-CLPM) [44], the autoregressive latent trajectory model (ALT) [45], the latent curve model with structured residuals (LCM-SR) [46], and the general cross-lagged model (GCLM) [47]. These were estimated using maximum likelihood with robust standard errors. Fit was compared using the Akaike information criterion (AIC). The best-fitting model was further refined by testing whether loosening various constraints improved its fit. We started by allowing variation over time in (1) autoregressions, (2) cross-lagged regressions, and (3) slope loadings (i.e., allowing for non-linear trajectories). Residual variances were allowed to vary over time by default. We also tested whether adding second order auto- and cross-lagged regressions improved the model’s fit.

To evaluate differences between groups regarding the effects of homework engagement, we tested both a mediation and a moderation model [48]. The meditation model postulated that homework engagement is better in one treatment than the other, and this difference affects symptom outcomes via cross-lagged paths. In contrast, the moderation model postulated that the effect of homework engagement on next-session symptom severity is stronger in one treatment compared to the other. Stated differently, in the moderation model, the cross-lagged effect itself is assumed to differ between the included treatments. As recommended in the literature, we estimated the indirect effect using bootstrapping to account for the non-normality of the product of coefficients [49].

Results

A total of 164 patients were included in the study. Of those, 84 patients received UP and 80 received dCBT. The sample was predominantly young females (62.8%, N = 103), and 88 (53.7%) had a bachelor’s degree or higher. Most (49.4%, N = 81) had a primary diagnosis of unipolar depression, while the remaining 47 patients (28.7%) had social anxiety disorder and panic disorder or agoraphobia (22.0%, N = 36). Moreover, 45.1% (N = 74) of the sample were on a long-term sick leave. We found no statistically significant differences in demographic data between patients randomized to dCBT or UP. See Table 2 for further details. Patients had equal mean scores of symptoms in the two treatment arms. No significant difference was observed between groups at baseline or at the end of treatment. See Table 3.

Table 2.

Sample characteristics

Sample characteristics		tCBT	dCBT	Total
		N	N	N
Total		84	80	164
Sex
Male		28	33	61
Female		56	47	103
Age
Less than 35 years		57	47	104
35 years or more		27	33	60
Completed education
More than high school		40	48	88
High school or less		44	32	76
On long-term sick leave (3 months or more)
Yes		39	35	74
No		45	45	90
Previous hospitalization
Yes		11	10	21
No		73	70	143
Previous psychotherapy (min. five sessions)
Yes		73	72	145
No		11	8	19
Primary diagnosis
Depression		39	42	81
Social Anxiety Disorder		28	19	47
Panic Disorder / Agoraphobia		17	19	36

N: number. UP: unified protocol for transdiagnostic treatment of emotional disorders. dCBT: Diagnosis-specific cognitive behavior therapy

Table 3.

Symptomatology and homework adherence across UP and dCBT groups

	Baseline				End of treatment				Intervention effect
	Mean	STD	95% CI		Mean	STD	95% CI		F	df	p
Symptom level - HSCL-25
UP	1.97	0.67	1.82	2.11	1.58	0.80	1.41	1.76	26.206	1;83	2.0e-06
dCBT	1.98*	0.69	1.82	2.13	1.47**	0.78	1.29	1.64	43.664	1;79	4.2e-09
Homework and Attendance	UP				dCBT				Intervention effect
	Mean	STD	95% CI		Mean	STD	95% CI		F	df	p
HW frequency - total	2.87	0.61	2.74	3.00	3.15	0.77	2.98	3.32	6.467	1;163	0.01
HW frequency period 1	2.74	0.87	2.55	2.93	3.39	0.86	3.19	3.58	22.722	1;163	4.1e -06
HW frequency period 2	2.93	0.76	2.76	3.09	3.13	0.95	2.92	3.35	2.369	1;163	0.1
HW frequency period 3	2.94	0.90	2.75	3.14	2.92	0.98	2.70	3.14	0.027	1;163	0.9
Sessions attended	11.89	1.98	11.46	12.32	11.96	1.67	11.59	12.33	0.590	1;163	0.8
Missing HW data	1.26	1.36	0.97	1.56	1.36	1.44	1.04	1.68	0.211	1;163	0.7

HSCL-25: Hopkins Symptom Check List, 25 items. SD: standard deviation. CI: confidence interval. UP: Unified Protocol for Transdiagnostic Treatment of Emotional Disorders. dCBT: Diagnosis-specific Cognitive Behavior Therapy. *) Difference between intervention arms is significant F1,163: 6.467, p = 0.01.**) Difference between intervention arms is significant F1,163: 22.722, p = 4.1E-06

Homework engagement’s effects on symptom outcomes at the end of treatment

The mean number of attended sessions was 11.9 (SD = 1.8; 95% CI [11.6–12.2]) of a possible 14 sessions. Patients had a mean homework engagement of 3.0 (SD = 0.7; CI = 2.9–3.1]), corresponding to 50% of the maximum possible.

In UP, the patients’ mean HW engagement was 2.9 (SD = 0.6; CI = 2.7-3.0]). In dCBT, the mean HW engagement was 3.2 (SD = 0.8; 95% CI [3.0-3.3]). The difference between intervention arms was statistically significant (F_(1,163): 6.467, p = 0.01). See Table 3. In Period 1, the difference in HW engagement between the two intervention arms was also significant (F_1,163: 22.911, p < 0.001). Additionally, mean HW engagement across the course of psychotherapy interacted significantly with the effect of time on symptom outcome (rmANOVA HSCL-25: F_1,162: 3.944, p = 0.05) but adding co-factors intervention arm and diagnostic category yielded the entire analysis insignificant.

When exploring the development of HW engagement throughout the treatment course, no main within-subject effect of time was observed (F_2,160: 1.552, p = 0.2). However, both the interactions between time and intervention arm (F:_2,160: 9.861, p = 7.0e-05), between time and diagnostic category (F_2,162: 6.902, p = 0.001), and the triple interaction term between the factors (F_2,162: 4.886, p = 0.008), were significant. After splitting the sample by diagnostic category (see Fig. 1), no significant effects of intervention (F_1,81: 1.733, p = 0.2), nor time (F:_1,81: 2.094, p = 0.1) nor time*intervention arm interaction (F_2,79: 0.725, p = 0.5) were observed in the sub-sample with anxiety disorders (lower panel), while intervention (F:_1,79: 6.211, p = 0.02), time (F:_1,79: 6.338, p = 0.002) and time*intervention arm interaction (F_1,79: 13.954, p = 2.6e^-06) were significant in the sub-sample with depression (upper panel).

Fig. 1.

Homework Engagement by Intervention Arm in Participants with Depression or Anxiety Disorders. Upper panel: For patients with depression the significant interaction term Intervention Arm*Period, see text, is observed as a lower level of homework engagement in UP in the first period (F_1,79: 31.420, P = 2.9e^-07) that is not seen in the other periods. N = 81. Lower panel: For patients with anxiety disorders level of homework engagement did not differ depending on intervention arm. N = 83. UP: Unified Protocol for Transdiagnostic Treatment of Emotional Disorders. dCBT: Diagnosis-specific Cognitive Behavior Therapy

The effects of homework engagement on symptom outcomes from session to session

Initial model comparisons using the AIC indicated that the best-fitting structure was the ALT model (see Table 4). Model fit was improved by allowing loadings for the slope factor for combined ODSIS/OASIS to be estimated freely (the first loading was constrained to zero and the last to 12, to identify the model, see [50]). Adding lag-2 autoregressions for both ODSIS/OASIS and homework engagement also improved fit. None of the other modifications did so. The final model (see Fig. 2) showed good fit to the data according to fit indices, although the chi-square test was statistically significant (c²(364) = 483.99, p < 0.001; RMSEA = 0.04, 90% CI [0.03, 0.04]; CFI = 0.96, SRMR = 0.07). In Fig. 2, a cross-lagged panel model is estimated among the observed variables, while simultaneously accounting for between-patient differences in level (random intercept terms I_Oi and I_Hwi) and slope over time (random slope terms S_Oi and S_Hwi). Curved bidirectional arrows represent covariances, which are there to ensure the model is realistic and fits the data. Directed arrows represent either factor loadings (when going from random intercepts to observed variables); these are fixed to 1 for intercepts and to linear change (0, 1, 2, etc.) for slopes. Directed arrows between observed variables represent presumed causal effects on a within-patient level. The horizontal ones linking the same variable over time are called autoregression effects, while the diagonal ones between variables are called cross-lagged effects. The latter are the ones that are of most interest, as they indicate potential mechanisms of change.

Table 4.

Model fit comparisons

Model	AIC	BIC	Chi2(df), value, p	CFI	RMSEA[90% CI]	SRMR
DPM	20145.83	20384.65	618.82(367), < 0.001	0.92	0.05 [0.04, 0.06]	0.08
RI-CLPM	20157.79	20382.36	604.64(371), < 0.001	0.92	0.05 [0.04, 0.06]	0.08
ALT	20056.22	20255.84	522.91(378), < 0.001	0.95	0.04 [0.03, 0.05]	0.07
LCM-SR	20095.35	20259.32	578.18(388), < 0.001	0.93	0.04 [0.04, 0.05]	0.07
GCLM	20157.69	20474.94	586.68(345), < 0.001	0.92	0.05 [0.04, 0.06]	0.06

Note. DPM = Dynamic Panel Model, RI-CLPM = Random Intercept Cross-Lagged Panel Model, ALT = Autoregressive Latent Trajectory Model, LCM-SR = Latent Curve Model with Structured Residuals, GCLM = General Cross-Lagged Model. AIC = Akaike Information Criterion, CFI = Comparative Fit Index, RMSEA = Root Mean Square Error of Approximation, SRMR = Standardized Root Mean Square Residual

Fig. 2.

Final Autoregressive Latent Trajectory Model. Yit = Outcome variable (OASIS/ODSIS) for individual i at time t, Mit = Mediator variable(Homework) for individual i at time t, U0i, Y = random intercept for outcome variable for individual i, U0i, M = random intercept for mediator variable for individual i, U1i, Y = random slope for outcome variable for individual i, U1i, M = random slope for mediator variable for individual i, eYt = error variable for outcome variable for individual i at time t, eMt = error variable for mediator variable for individual i at time t, Treatmenti = dummy coded treatment variable for individual i

Parameter estimates showed a significant, but small cross-lagged effect of homework engagement on the combined next-week ODSIS/OASIS ratings (cross-lagged effect = − 0.23, se = 0.11, z = -2.16, p = 0.03, 95% CI [-0.44, − 0.02]). This effect corresponded to standardized coefficients between − 0.03 and − 0.04, which are considered in the small range [51]. The long-run effect of a stable one-point increase on the homework engagement rating would be expected to lead to 0.32 points of decrease in anxiety and depression symptoms, as assessed by the combined ODSIS/OASIS (se = 0.15, z = -2.17, p = 0.03, 95% CI [-0.61, − 0.03]). Using the pre-treatment standard deviation to calculate the Cohen’s d effect size for the long-run effect showed that this corresponds to a small effect (d = 0.05).

Since our previous analyses showed a significant difference in homework engagement between the two treatments, we next examined whether that difference would predict better outcomes for dCBT compared to UP. We first tested a mediation model [48], in which the treatment group was added as a predictor of the random intercepts and slopes for homework engagement and combined ODSIS/OASIS, and observed variables at Time 1. The mediation model is visualized in Fig. 3. The direct effects of treatment group were statistically significant for the random intercepts and slopes of homework engagement (intercept coefficient = 0.57, se = 0.11, z = 5.07, p < 0.001, slope coefficient = − 0.07, se = 0.02, z = -4.48, p < 0.001), although not for the observed homework variable at session one (coefficient = 0.25, se = 0.17, z = 1.43, p = 0.15). Mimicking the findings from the summative analysis of homework engagement on treatment groups, this means that dCBT had significantly higher initial levels of homework engagement (at least from session two), but that the groups became more similar in terms of engagement over time (the negative effect of treatment group on slope coefficient). Since none of the direct effects of treatment type on combined ODSIS/OASIS were significant, we constrained these to zero and re-estimated the model. In this new model, we calculated the total indirect effect of treatment type on end-of-therapy outcome (combined ODSIS/OASIS, at session 14); however, it was likewise not statistically significant (indirect effect = 0.05, se = 0.05, z = 1.04, p = 0.30). Thus, the difference in homework engagement between treatments did not appear to have significantly influenced the overall outcome of the study.

Fig. 3.

Autoregressive Latent Trajectory Mediation Model. Y_it = Outcome variable (OASIS/ODSIS) for individual i at time t, M_it = Mediator variable (Homework) for individual i at time t, U_{0i, Y} = random intercept for outcome variable for individual i, U_{0i, M} = random intercept for mediator variable for individual i, U_{1i, Y} = random slope for outcome variable for individual i, U_{1i, M} = random slope for mediator variable for individual i, e_Yt = error variable for outcome variable for individual i at time t, e_Mt = error variable for mediator variable for individual i at time t, Treatment_i = dummy coded treatment variable for individual i

Second, we tested whether treatment moderated the effect of homework engagement on next-session ODSIS/OASIS scores using a two-group model. The results of this model indicated almost identical effects in the two groups (-0.21 vs. -0.23, difference = -0.03, se = 0.21, z = -0.13, p = 0.90). Thus, the test provided no support for differential effects of homework engagement between the two groups.

Discussion

This study analyzed data on the patients’ engagement in treatment homework and its impact on overall and session-to-session symptom outcomes in group CBT. We found that patients randomized to the UP had lower homework engagement in the first treatment period than those randomized to dCBT groups. This was especially true for those with a primary diagnosis of depression in the UP groups. The different content of the UP versus dCBT protocols might explain this finding. UP focuses on symptom monitoring, psychoeducation, clarity of emotion exercises, and mindfulness skills acquisition from session one through the rest of Period 1, while the dCBT protocol for depression [32] instead prescribes problem and goal registration in session one and continues with registration of activities in the following sessions of this period (see Table 1). It might be easier for participants with depression to engage in the activity-focused homework used in dCBT than in the clarity of emotion strategies and mindfulness exercises used in UP. Furthermore, the dCBT protocol for depression [32] was developed for patients who received CBT in a day hospital—i.e., patients with a lower level of functioning compared to this study’s participants. It is possible that it was designed to include more accessible homework compared to the UP protocol, which was designed for outpatient administration. However, the result might also be explained by the therapists not sufficiently tailoring the treatment rationale and homework to participants with depression in the UP groups. Tailoring the assigned homework to the specific participant is also considered an important aspect of fostering homework compliance [52]. This speculation is supported by qualitative data showing that therapists found the UP group protocol easier to apply for participants with anxiety disorders than participants with depression [53]. A possible clinical implication of the current finding is that UP therapists should pay extra attention to homework completion for patients with depression and tailor the homework to these participants.

Although we found homework engagement to be lower in UP, this did not affect the outcome of treatment when examined with linear regression or cross-lagged panel modeling, nor across outcomes. This also reflects the findings of the main trial, which showed non-inferiority between UP and dCBT regarding symptom outcomes, functioning, and well-being [22].

For this paper, we utilized the HSCL-25 as an outcome for the first analyses, since it included items relating to both anxiety and depression and was therefore useful for capturing therapy outcome in a transdiagnostic sample. The main study employed the six-item Hamilton Anxiety and Depression Rating scales for diagnostic-specific outcomes, but we did not use these outcomes, as we did not have sufficient statistical power to analyze the level of diagnosis or for specific dCBT protocols.

For the second analysis (cross-lagged panel modeling), we used a combined ODSIS/OASIS score, since these short symptom outcome measures were administered at each session as an integrated part of the UP protocol, and in the main study, they were collected in both intervention arms. The broad transdiagnostic outcome measures applied in both types of analyses may have diluted the effects of homework engagement. Yet, we did find an overall effect of mean homework engagement on the symptom outcomes at the end of therapy, as well as a significant, but small, effect of homework engagement on the next-session combined OASIS/ODSIS measure scores in the cross-lagged panel modeling analysis. This corresponds to previous meta-analytic findings in other samples of patients with mood disorders [7, 10, 12], where homework engagement significantly predicted the patients’ symptoms at the end of the intervention.

Limitations

The homework engagement measure applied herein was purpose-made for the present study by the authors and has, therefore, not been tested in other studies or undergone formal validation. Relying on a single item, it captures the quantity aspect of general homework engagement measured as time spent on homework assignments. However, it does not capture other aspects of homework engagement quantity, including the number of homework tasks worked with or the amount of homework partially or fully completed [7]. In addition, the first response option on the homework engagement measure was phrased as, “I have not thought about the homework and did not work on it/I did not want to work on it.” This might combine two rather distinct aspects: intention or ability to perform the homework assignments versus motivation or engagement in the homework and/or the group therapy.

Furthermore, we did not assess the quality of patients’ homework engagement, but only to what degree it had been attempted or done. It is plausible that the quality of the completed homework, rather than simply the attempt at or engagement in performing it, is the true predictor of the outcome. In Kazantzis et al.’s (2016) [7] meta-analysis, they found that both the quality and quantity of homework engagement predicted treatment outcomes in CBT programs. However, quality might be more relevant than quantity, since it is more closely related to the treatment outcomes [54]. Previous research suggests that the quality of homework in CBT is a possible mediator of the effect on symptoms [7]. Moreover, we did not assess how patients engaged in different types of homework assignments specific to the treatment. A previous study [6] found that both general homework engagement and engagement in specific CBT homework assignments affected the treatment outcomes.

In addition to this, homework engagement was self-reported, which can be regarded as a limitation of the study. Although self-reporting is a widely used source of engagement data, it may overestimate the extent of homework completed relative to objective measures [7].

Further, Kazantzis et al. (2016) [7] found that the assessment of homework is highly heterogeneous across individual studies. The 17 studies included in their meta-analysis all tracked homework engagement. Ten studies tracked homework engagement and quality with a single-item Likert-type instrument, while the remaining seven trials tracked whether homework was completed. In most studies, engagement was self-reported, while in the remaining studies it was assessed by the therapist or research staff. Only three studies utilized established instruments—the Homework Compliance Scale (HCS) [55] and Patient EX/RP Adherence Scale (PEAS) [56]—to track homework engagement, while the remainder of the instruments used were purpose-made for their trials. The PEAS was the only multi-item instrument, having three items.

We also analyzed the ODSIS and OASIS scores as one combined score rather than conducting separate analyses of the two instruments. We did so inspired by previous research [57, 58] where other brief anxiety and depression scales have been combined yielding valid and sensitive composite measures of anxiety and depression. We furthermore wished to preserve the primary trial’s transdiagnostic perspective, albeit we realize that the composite ODSIS/OASIS measure has not been validated. This method is in line with a transdiagnostic perspective where anxiety and depression symptoms should be considered manifestations of a shared underlying neuroticism/negative affect construct [59]. However, it is possible that analyzing the two instruments separately would have yielded different results, and that our methods obscure possible symptom-specific effects of HW.

Finally, while the present study focused on the role of homework engagement in the outcome of group CBT interventions, it should be noted that other psychotherapy process variables such as client factors, group cohesion, working alliance, and patients’ beliefs about homework can also impact outcome and therapy and explain lack of significance, as found by McEvoy et al. (2024a) and McEvoy et al. (2024b) [11, 60]. Furthermore, therapist protocol adherence and therapist competence may indirectly promote engagement to homework, but may also influence outcomes via other routes [61]. It is, therefore, possible that our findings reflect the effects of HW being “drowned out” by other important factors known to influence group therapy (e.g., group coherence) [11, 60, 62].

To the best of our knowledge, this is the first study on homework’s effects on treatment outcomes in UP, and the first ever in group UP. Since this study used robust CLPM to account for the effect of homework engagement on symptom outcomes, our findings may reflect the isolated effects of homework on treatment outcomes and overcome methodological shortcomings in the area. Thus, they may reflect the actual effects of homework engagement rather than being explained by the previously proposed methodological issues. Future research should use an established and validated measure [63] such as the Homework Compliance Scale (HCS) [55] to track homework engagement. Future research could also incorporate general and specific factors known to affect the outcome of group therapy.

Lastly, it has been speculated that homework assignment could worsen symptoms for some patients. Barnes et al.’s (2013) qualitative research on homework in CBT for depression found that, for a substantial number of patients, homework may become an insurmountable burden that leads to feelings of failure and, hence, may intensify depressive experiences [64]. Therefore, patients’ feelings about homework must be monitored, and obstacles to engagement must be discussed continuously in therapy. This might be more difficult in group therapy, which allows less focus on individual challenges and where the therapist possibly has less insight into each patient. Nevertheless, future research should investigate the possible negative effects of homework in CBT.

Conclusion

This study examined the relationships between engagement in homework, symptoms, and different types of group CBT. We found lower homework engagement among patients with depression randomized to UP, but this had no impact on the patients’ overall symptom outcomes at the end of therapy. However, across CBT protocols, homework engagement had a small effect on end-of-treatment outcomes as well as on the next week’s ratings of anxiety and depression. This is the first study to examine homework engagement’s effects on symptoms in group UP and adds to the limited literature using robust methods to isolate treatment engagement’s unique influences on symptom outcomes. Future studies should confirm the results and include other relevant aspects of homework, such as homework quality, and other process variables, such as therapist factors, group cohesion, and negative effects of homework assignment.

Author contributions

SA, MA, MH, SP, BB, and NR conceived the project. ABC, AE, and NR collected data. SA and FF carried out statistical calculations. ORH was responsible for writing the first draft of this manuscript. All authors have discussed, reviewed, and approved the manuscript.

Funding

Open access funding provided by Copenhagen University. The original trial was funded by grant ID 114241 from TrygFonden; grants 5577 and 6215 from Jascha Fonden; grants RSSF2017-000667, RSSF2016-000342, and RSSF2015-000342 from Region Zealand Research Foundation; PhD. scholarship (Bryde) from Region Zealand Mental Health Services; and PhD. scholarship (Reinholt) from Mental Health Services Capital Region of Denmark.

Data availability

Data can be obtained upon reasonable request from the last author.

Declarations

Human ethics and consent to participate

This study was conducted from December 2016 to September 2019 under the approval of the Ethics Committee Region Zealand (reg. no. 3084871-SJ-582) and the Danish Data Protection Agency Region Zealand (reg. no. REG-104-2016). Informed written consent to participate was obtained from all participants after the objectives of the study were explained and the confidentiality of information was ensured. All methods were carried out in accordance with relevant guidelines and regulations, including the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.