Who Gets the Most Out of Cognitive-Behavioral Therapy for Anxiety Disorders?

Abstract

Objective

The present study explored treatment dose and patient engagement as predictors of treatment outcome in cognitive behavioral therapy (CBT) for anxiety disorders.

Method

Measures of high versus low treatment dose, and high versus low patient engagement in CBT were compared as predictors of 12 and 18 month outcomes for patients being treated for anxiety disorders with CBT (with or without concurrent pharmacotherapy) in primary care settings as part of a randomized controlled effectiveness trial of the Coordinated Anxiety Learning and Management (CALM) intervention. Measures of dose (attendance, exposure completion) and engagement in CBT (homework adherence, commitment) were collected throughout treatment, and blinded follow-up phone assessments of outcome measures (12-item Brief Symptom Inventory, Patient Health Questionnaire 8, Sheehan Disability Scale) were completed at 12 and 18 months. Propensity score weighting controlled for baseline differences in demographics and symptom severity between patients with high and low dose and engagement. These analyses included the 439 patients that selected CBT as treatment modality.

Results

Completing exposures, high attendance, and being more homework adherent predicted better outcomes across all measures at 12 and 18 months, and high CBT commitment predicted better outcomes on all measures at 18 months.

Conclusions

This study found that higher treatment dose and patient engagement in CBT for anxiety disorders were stable and robust predictors of greater reductions in anxiety symptoms, depression symptoms, and functional disability.

Cognitive behavioral therapy (CBT) is well established as an effective treatment for anxiety disorders (e.g., Hofmann & Smits, 2008). An important direction for future research is to establish predictors of response. The amount of treatment dose patients receive and the extent to which they engage in CBT are likely predictors of response. More specifically, one central tenet of CBT for anxiety disorders is that the acquisition of cognitive or somatic coping skills reduces anxiety and avoidance behavior; another is that repeated exposure to anxiety inducing cues leads to corrective learning and eventual anxiety reduction (Craske, 2009). Both central tenets rest upon a learning process. Storage of long-term learning is strengthened by repeated study and practice (i.e., Bjork and Bjork, 1992), so it is conceptualized that more time spent practicing CBT skills (i.e., high treatment dose: session attendance and exposure completion) and greater effort devoted to practice (i.e., high patient engagement: homework adherence and commitment to treatment) lead to more robust long-term storage of learning, which results in greater symptom reduction and functional improvement. Conversely, low treatment dose and lack of patient engagement are viewed as reasons for treatment nonresponse to CBT. Extant evidence regarding the roles of dose and engagement in predicting CBT outcomes is mixed, partly due to methodological limitations. The goal of the current study is to overcome those limitations by using sufficiently powered analyses and by accounting for baseline characteristics that influence dose and engagement, in a sample of participants who were afforded a wide range of dose and engagement. The findings may guide the development of strategies for those at greatest risk for low dose and lack of engagement.

Meta-analyses clearly indicate that CBT is effective for anxiety disorders. A review of meta-analyses demonstrates strong efficacy of CBT for anxiety disorders, with large effect sizes for the treatment of obsessive compulsive disorder, and at least medium effect sizes for social anxiety disorder, panic disorder, and post-traumatic stress disorder (Hofmann, Asnaani, Imke, Vonk, Sawyer, & Fang, 2012). CBT has also been found to be effective in comparison to no-treatment or “expectancy control” (pill placebo, attentional placebo, nonspecific therapy) for anxiety (Norton & Price, 2007) and in comparison to psychodynamic therapy for anxiety and depression (Tolin, 2010). Nonetheless, a substantial number of participants drop out of CBT or remain symptomatic at post-treatment (Arch & Craske, 2009).

The majority of these studies were efficacy studies conducted in research settings. In our recent effectiveness study conducted in primary care settings (Craske et al., 2011; Roy-Byrne et al., 2010), 1004 participants were randomized to an active treatment intervention (comprised of CBT, expert recommendations for psychotropic medications, or both) or to usual care. The majority (91%) of participants in the active intervention received CBT which was delivered by novice clinicians (mainly nurses and social workers) who were guided by a computerized program. Participants attended 7 CBT sessions on average (SD = 4.1). Outcomes from the intervention were superior to usual care across a range of symptom and functional measures at 6, 12 and 18 months after study entry (Craske et al., 2011; Roy-Byrne et al., 2010). Individualized treatment response was operationalized as 50% or more reduction on disorder-specific questionnaires. Even though the treatment was highly effective, and most participants (87%) in the intervention received CBT (with or without concurrent pharmacotherapy), a number of participants (46.7%. 45.3%, and 45.1%) who received CBT were classified as non-responders at 6, 12, and 18 months. Thus, findings from both efficacy studies described above and our recent primary care trial indicate substantial nonresponse rates.

Failure to respond to CBT may be due to a multitude of factors, including treatment-related (e.g., poorly conducted CBT, low dose of treatment received) and patient-related (e.g., comorbidity, low motivation and low engagement). In this report, we focus on the treatment-related factor of dose and the patient-related factor of engagement in CBT. We conceptualized treatment dose as the total amount of treatment received, and patient engagement as the extent to which patients actively participated in treatment.

One index of treatment dose is session attendance. Most available studies have found a positive relationship between number of sessions attended and outcomes for a range of disorders (e.g., Bowen, South, Fischer, & Looman, 1994; Taft, Murphy, Elliott, & Morrel, 2001). Between 13 to 18 sessions have been shown to be necessary for clinically significant change in 50% of patients (e.g., Hansen & Lambert, 2003). However, these studies failed to fully adjust for pre-existing influences (e.g., baseline demographics and symptom severity) upon treatment attendance or outcome. Two studies that fully controlled for baseline variables established a positive relationship between sessions attended and outcomes in the treatment of anxiety disorders. However, one study was limited by a relatively small sample size (n = 93) (Craske et al., 2006) and the other was limited by only assessing symptoms at post-treatment but not at follow-up time points (Buckner et al., 2009).

Another measure of treatment dose in CBT for anxiety disorders is the amount of exposure completed. Studies have demonstrated that exposure alone is as effective as exposure combined with other coping skills, such as cognitive restructuring (e.g., Longmore & Worrell, 2007; Norton & Price, 2007), but only one study to date has evaluated the degree to which amount of involvement in exposure therapy per se predicts outcome relative to other CBT strategies (Craske et al., 2006). In the current study, exposure completion was conceptualized as a related, but distinct, construct to session attendance, each providing a unique measure of treatment dose. Exposure completion was conceptualized as a specific measure of the active treatment ingredient dose (akin to whether the prescribed medication is taken), and session attendance as a broad measure of treatment dose (akin to how many scheduled medical appointments are attended).

One index of engagement is patient involvement in treatment, which has been operationalized in several different ways. Some studies have defined patient involvement as a shared decision-making process leading to mutual agreement by patient and clinician regarding the treatment approach (Elwyn et al., 2001). Findings have been inconclusive regarding the effectiveness of interventions aimed at increasing shared decision making in mental health treatment (Duncan, Best, & Hagen, 2010), but one study found that patient involvement in treatment decision-making contributed to better treatment outcomes for depression (Loh, Leonhart, Wills, Simon, & Härter, 2007). Better treatment outcomes (post-treatment diagnosis and impairment) are predicted by high treatment involvement by anxious child patients (e.g., Chu & Kendall, 2004; Chu & Kendall, 2009). Another approach has been to examine patient verbal commitment or reported desire to change and language indicating a commitment to change (coded by independent raters) (e.g., Aharonovich, Amrhein, Bisaga, Nunes, & Hasin, 2008; Amrhein, Miller, Yahne, Palmer, & Fulcher, 2003). In these studies, patient commitment predicted superior outcomes for drug abuse and eating disorders. However, the potential influences of baseline patient demographics and symptom severity on commitment were not adequately controlled in these studies.

Another index of engagement in CBT is adherence to homework assignments. Homework may be one of the most distinctive features of CBT compared to other forms of psychotherapy, and it is presumed to be critical to therapeutic change (for review see Kazantzis, Deane, & Ronan, 2004). The relationship between homework adherence and treatment outcome has generally yielded modest effect sizes; recent studies show a positive correlation between homework adherence (behavioral and cognitive skills practice) and treatment outcome in CBT for anxiety disorders (e.g., Anand, Sudhir, Math, Thennarasu, & Reddy, 2011; Westra, Arkowitz, & Dozois, 2009), depression (e.g., Cowan et al., 2008), substance dependence (e.g., Carroll et al., 2008) and hoarding (e.g., Ayers, Wetherell, Golshan, & Saxena, 2011). Also, homework compliance significantly mediated outcomes (Burns & Spangler, 2000) and was related to continued improvement after CBT was completed (e.g., Edelman & Chambless, 1995). Two recent meta-analyses by Kazantzis, Whittington, and Dattilio (2010; 46 studies reviewed, n = 1,072) and Mausbach, Moore, Roesch, Cardenas, and Patterson (2010; 23 studies reviewed; n = 2183) found robust effect sizes of homework compliance on treatment outcome across a variety of target symptoms. However, neither meta-analysis accounted for baseline demographics or symptom severity, as few of the individual studies reviewed examined these factors, which may have biased the findings. There have been mixed findings as to whether quality or quantity of homework compliance in CBT more strongly predicts outcome (Rees, McEvoy, & Nathan, 2005; Schmidt & Woolaway-Bickell, 2000).

As reviewed, studies to date were often limited by failure to adjust for baseline characteristics that influence dose and engagement, which is necessary to establish the independent effect of dose and engagement upon outcomes. Pretreatment severity of anxious and depressive symptoms has been found to predict poorer response to psychotherapy (e.g., Rosenkranz & Muller, 2011). Findings regarding the effect of demographics on treatment outcome have been inconsistent (Santana & Fontenelle, 2011), but in some cases response to psychotherapy has been influenced by pretreatment demographics including gender (e.g., Davis, Barlow, & Smith, 2010), marital status (e.g., Thase et al., 1992), employment status and age (e.g., Fournier et al., 2009). Ethnic differences have been found in number of therapy sessions attended and response to treatment (e.g., Sue, Fujino, Hu, Takeuchi, & Zane, 1991). Patients are not randomized to high or low dose and engagement, so without accounting for baseline difference in severity and demographics, which may correlate with or contribute to dose, engagement, and outcome, it is impossible to identify the unique effects of dose and engagement in CBT upon outcome. Other weaknesses of studies to date were limited sample sizes and range of dose afforded to patients by limiting the number of sessions provided. In this report, we aimed to overcome these limitations by using data from the CALM study (Roy-Byrne et al., 2010; Craske et al., 2011). In this large sample, patients were encouraged to complete CBT within 8 to 10 sessions, but the large range in number of CBT sessions actually received (1 to 21 sessions) provided an excellent opportunity to evaluate indices of treatment dose and engagement. Flexibility was given to therapists and patients in terms of how much of each CBT component was completed, which resulted in variance in the amount of exposure therapy conducted, providing a measure of treatment dose. Also, rather than relying on patient report of commitment and homework completion, therapists rated patient commitment and patient adherence to homework assignments on a session-by-session basis, providing measures of patient engagement in treatment. This is the first study to examine these two measures of dose (attendance, exposure completion) and two measures of engagement (homework adherence, commitment to CBT) within the same treatment sample, allowing for direct comparison of which aspects of dose and engagement best predict outcome.

Furthermore, propensity score analyses were used to obtain a more robust estimate of the association between measures of dose and engagement and outcomes. Propensity score weighting (Rosenbaum & Rubin, 1983) is an effective way of eliminating differences in observed characteristics (such as age, gender, severity at baseline, presence of chronic disorders and so on) between patients in low dose versus high dose groups, and low engagement versus high engagement groups. Propensity score weighting is preferred over regression models which rely too heavily on linear assumptions and are highly sensitive to model specification, such as the inclusion of important interaction terms. Here, we consider a large range of baseline demographic and severity variables in order to control for potential confounders of the effect of dose and engagement on outcomes. We hypothesized that attendance, exposure completion, homework adherence, and commitment to CBT would predict lower symptom and better functioning outcomes after controlling for the influence of baseline demographics and symptom severity.

Methods

Design

Our randomized controlled effectiveness trial compared the CALM intervention to usual care in 17 primary care clinics in 4 US cities. Blind assessments occurred at 6, 12, and 18 months after baseline. The analyses included here examined the 12 and 18 month assessment data but omitted the 6 month assessment data, as a number of patients were still receiving study treatment at 6 months.

Participants

Between June 2006 and April 2008, 1620 primary care patients consented to complete a study eligibility interview, and 1004 patients with panic disorder (with or without agoraphobia), generalized anxiety disorder, social anxiety disorder, or posttraumatic stress disorder were enrolled. Participating research institutions were: University of Washington (Seattle), University of California-Los Angeles, University of California-San Diego, University of Arkansas for Medical Sciences, and the RAND Corporation (an assessment site only) (for more details, see Sullivan et al., 2007).

Recruitment

Primary care providers (PCPs) and clinic nursing staff directly referred potential participants. At some sites, a five-question anxiety screener, the Overall Anxiety Severity and Impairment Scale (OASIS) (Campbell-Sills et al., 2009) was used to identify potential participants. A trained study clinician functioned as the main care manager, as well as the diagnostician who met with referred patients to determine eligibility. All participants gave informed, written consent to participate in this study, which was approved by each institution’s Institutional Review Board.

Inclusion Criteria

An eligible participant had to be a patient at a participating clinic, 18-75 years old, met DSM-IV criteria for one or more of panic disorder, generalized anxiety disorder, social anxiety disorder, or posttraumatic stress disorder based on the Mini International Neuropsychiatric Interview (Sheehan et al., 1998) administered by the clinician (after formal training and diagnostic reliability testing, and in consultation with study psychiatrists and psychology supervisors), and score at least 8 (moderate and clinically significant anxiety symptoms on a scale ranging from 0-20) on the OASIS.

Exclusion Criteria

Patients with unstable or life-threatening medical conditions, marked cognitive impairment, active suicidal intent or plan, psychosis, or bipolar I disorder were excluded. Alcohol or marijuana abuse (but not dependence) was permitted, but other drug abuse or dependence was exclusionary. Patients already receiving ongoing CBT (n = 7) were excluded, as were patients who could not speak English or Spanish (n = 2).

Randomization

After baseline assessment, participants were randomized, using stratified (by clinic and presence of co-morbid major depression) permuted block randomization, to intervention or usual care by an automated program at RAND. Block size was masked to all clinical site study members. Of the 1004 patients enrolled in the study, 503 (50%) were randomized into the intervention. Of the 1004 patients enrolled in the study 65 patients (7%) were 66 to 75 years old. For further details and consort chart of patient flow from referral through eligibility screening, consent and randomization in the CALM study see Roy-Byrne et al. (2010).

Intervention

Intervention participants received a treatment involving pharmacotherapy, computer-assisted CBT, or both, depending on their preference. Of the 503 patients randomized to the intervention, a total of 439 patients (87%) chose to receive CBT: 166 chose CBT alone, 273 chose to receive both CBT and pharmacotherapy concurrently, 43 patients chose pharmacotherapy alone, and 21 patients randomized to the intervention did not have any further study contact. Of the 439 patients who chose to receive CBT, 30 patients (7%) were 66 to 75 years old. Intervention participants who either chose pharmacotherapy alone or did not have any study treatment contacts were excluded from these analyses. The CBT program (called CALM Tools for Living, English and Spanish versions) contained eight modules. Participants selected their most distressing and disabling of the four anxiety disorders as the primary target for CBT. The cognitive restructuring and two exposure modules were tailored to each of the four anxiety disorders through branching mechanisms, whereas the remaining modules (i.e., self monitoring, psychoeducation, fear hierarchy, breathing retraining, relapse prevention) were mostly generic. The clinician sat side by side with the patient as they both viewed the program on screen (for more program details see Craske et al., 2009).

Occasionally, study clinicians used additional strategies, such as behavioral activation and cognitive restructuring for depressed mood. Motivational enhancement techniques were utilized by clinicians during the initial patient contact, and if a patient’s depressive symptoms became more severe than their anxious symptoms or began to significantly impair their ability to complete CBT for anxiety, or if the patient became suicidal. Such techniques were also implemented in response to irregular session attendance, low homework compliance, and low weekly patient ratings of outcome expectancy (“How likely is it that your anxiety can be successfully treated?”) and self-efficacy (“How likely is it that you can do what is necessary to make anxiety treatment work?”). After the first 10 to 12 weeks, symptomatic participants could receive more of the same modality (CBT or medication) or the alternative modality, for up to 3 more steps (i.e., another 10 to12 weeks) of treatment. For further details about other aspects of the CALM intervention, including medication, “continued care,” collaborative care, and web-based tracking, see Roy-Byrne et al. (2010) and Craske et al. (2011).

Study Clinician

Study clinicians (n = 14; 6 social workers, 5 registered nurses, 2 masters level psychologists, 1 doctoral level psychologist) had some patient care experience (although only 8 had prior mental health care experience) and exposure to primary care settings, but had no expertise in anxiety management or CBT. For further details about clinician training in the CALM intervention, see Rose et al. (2011). All clinicians provided CALM treatment within the participating primary care clinics.

Measures

The assessment battery was administered at baseline, 6, 12, and 18 months via a centralized phone survey by the RAND survey research group, blinded to group assignment and timing of assessment.

Dose and Engagement Variables

The following dose variables were derived from session-by-session data the clinicians entered into a web-based tracking system: (1) number of CBT sessions attended, and (2) number of CBT modules in which exposures were conducted to interoceptive cues, trauma memories, catastrophic images, or feared or avoided situations, either in-session or assigned as between-session practice. The following engagement variables were derived from session-by-session data the clinicians entered into a web-based tracking system: (1) clinician rating of homework adherence, rated after every CBT session on a 4 point scale (1 = Missed Most, 2 = Missed Half, 3 = Missed Few, 4 = Missed None) and (2) clinician rating of patient “overall commitment to CBT this session,” rated after every CBT session on a 0 to 10 point scale (0 = None, 10 = Complete). Mean ratings were calculated for both homework adherence and commitment to CBT across all completed CBT sessions. It should be noted that the two engagement variables were derived from clinician ratings (homework adherence, commitment to CBT) and were not blind to ongoing symptom status.

The frequency distribution was plotted for each raw dose and engagement variable, and based on measures of central tendency (mean, median) as well as skewness and kurtosis, cutoff points were selected that most cleanly split the variable into high versus low dose or engagement. Attendance was dichotomized as attending at least six CBT sessions or fewer than six sessions. Exposure was dichotomized as completing at least one exposure module or none. Homework Adherence was dichotomized as averaging 3.41 or higher or less than 3.41. Commitment was dichotomized as averaging 8 or higher or less than 8.

Outcome Variables

A battery of measures was administered at each assessment point. For these analyses, we included measures relevant to all anxiety disorders, rather than disorder-specific measures. The primary outcome measure was a generic measure of two key components of all anxiety disorders, psychic and somatic anxiety, as measured by the 12-item Brief Symptom Inventory (BSI-12; Derogatis, 2001). The BSI-12 is a subset of the BSI-18, including the 6-item anxiety scale and 6-item somatization scale from the BSI-18, but excluding the 6-item depression scale. The BSI-18 has been validated and found to be internally reliable in a community sample (Derogatis, 2001).

Secondary outcome measures included depression, as measured by the Patient Health Questionnaire 8 (PHQ-8; Kroenke, Spitzer & Williams, 2001) (the item assessing suicidal ideation and intent from the PHQ-9 was dropped at follow-up assessments), and functional status, as measured by the Sheehan Disability Scale (SDS; Sheehan, Harnett-Sheehan, & Raj, 1996). The PHQ-9 is a nine item self-report measure of depressive symptoms which has high internal reliability (Cronbach’s α of 0.86 to 0.89), high test-retest reliability (0.84), high construct, external and criterion validity, as well as 88% sensitivity and 88% specificity for major depression (Kroenke et al., 2001). The SDS is a self-report measure assessing impairment from symptoms in three areas (work/school, social life, home life/family). The SDS has been found to have adequate internal reliability, construct validity, criterion validity, and discriminant validity, as well as 83% sensitivity and 69% specificity in patients with mood, anxiety, and substance use disorders (e.g., Rush, Pincus, & First, 2000). Higher scores on all three outcome variables indicate worse outcomes; greater symptom severity on the BSI-12 and PHQ-9, and greater functional impairment on the SDS.

Potential Baseline Confounders

The following baseline measures were included to model the probability of patients demonstrating low treatment dose or low engagement in CBT (versus patients with high dose or high engagement in CBT): insurance status (uninsured or not), marital status (yes/no), employed full time (yes/no), years of education (less than12/12/more than 12), gender (female/male), minority status (yes/no), age, number of chronic medical diseases, psychotropic medication use during the 6 months prior to treatment, psychiatric diagnoses at baseline (generalized anxiety disorder, panic disorder, social anxiety disorder, posttraumatic stress disorder, major depressive disorder, dysthymia), and baseline scores for BSI-12, PHQ-8, and SDS.

Indices of Learning

Though not the primary focus of these analyses, several measures of patient learning were collected during each CBT module, including patient ratings of understanding of the CBT material, ratings of ability to apply the material, and performance on quizzes assessing knowledge of the material.

Statistical Approach

As stated above, the primary aim of this study was to assess the effect of CBT dose and engagement on outcomes among those patients that selected CBT as treatment modality. These analyses use the data from the 439 patients who completed at least one CBT session. Because this is an observational study, patients are not randomized to the low or high CBT dose or engagement groups, but rather self-selected into each dose and engagement group. Differences in patient characteristics might thus in part explain the observed differences between the low and high dose and engagement groups. Propensity score weighting is an effective way of eliminating differences in baseline characteristics between patients in the low and high dose and engagement groups.

In this study, the propensity score is defined as the probability that a patient selects into the low CBT dose or engagement group conditionally to a set of patient’s characteristics X. This probability is used to build weights (Hirano, Imbens, & Ridder, 2003; McCaffrey, Ridgeway, & Morral, 2004) for patients in the high dose or engagement group. Patients in the high dose or engagement group who have similar characteristics to patients in the low dose or engagement group have a large propensity score and are therefore “up-weighted” when computing the effect of dose or engagement. Patients with high dose or engagement with characteristics dissimilar to the low dose or engagement group are “down-weighted” when computing the effect of dose or engagement. We fitted the propensity score weights using the TWANG R package (Ridgeway, McCaffrey, & Morral, 2006) which uses a non-parametric regression technique instead of a logistic regression. The obtained propensity score weights eliminated differences in baseline characteristics between the low and high CBT dose and engagement groups. We then ran propensity score weighted linear regressions to assess whether CBT dose and engagement had effects on symptom outcomes. In order to control the possible inflation of Type I error due to multiple comparisons, p-values were adjusted using the False Discovery Rate method.

To account for possible differences between patients receiving CBT along with concurrent pharmacotherapy versus without concurrent pharmacotherapy, the propensity score weighted linear regressions were repeated for the concurrent pharmacotherapy subsample (n = 273), and no concurrent pharmacotherapy subsample (n = 166). The primary propensity analyses with the entire sample (n = 439) is reported in the Results section, and secondary analyses with the concurrent pharmacotherapy and no concurrent pharmacotherapy subsamples are reported only when significant findings and notable trends differed from those found in the entire sample. Tables reporting the secondary analyses are included as supplementary online materials.

Missing Data

There were two sources of missing data in this study: the CBT engagement variables and outcomes measures (there were no missing data for CBT dose variables). Because of the different nature of these two sets of variables, the missing data issue was addressed in a separate way for each one. For CBT engagement indicators, there was missing data for both homework adherence (51 missing out of 439 patients) and commitment (13 missing out of 439 patients). Close examination of the missing data revealed that in most cases engagement indicators were missing either because the patient attended only one CBT session or because the clinician forgot to record the information in the WBS. Therefore, it was possible to logically impute nearly all of the missing cases. For example, patients who attended only one CBT session were scored as not having adhered to homework, or as having no commitment, as they never had the opportunity to complete any homework assignments or to show commitment to treatment. Other cases were logically imputed through additional data about completion of specific homework assignments entered into the CALM Tools for Living computer program; patients with no completed assignments were scored as not having adhered to homework. The few cases that could not be imputed logically through the above described methods were imputed using information from the other engagement indicators.

Non-response weights were used to account for missing outcome measures. Non-response weights are an effective way to address missing data when it is due to unit non-response (Brick & Kalton, 1996), as was the case for the missing outcome measures. For example, missing 12-month outcomes were due to the fact that a patient failed to respond to the entire 12-month follow-up assessment, rather than a patient refusing to respond to specific questions within the 12-month assessment. Because some of the patients who did not respond at month 12 did respond at month 18, we computed two separate sets of non-response weights, one for each follow-up. The non-response weights were estimated in the same way as the propensity score weights. The aim of non-response weights is to weigh those patients with outcomes at month 12, or month 18, to represent the entire sample of patients that selected CBT (n = 439).

Results

Sample Characteristics

Twenty nine percent of the 439 patients who selected CBT were non-Caucasian and 19% were ethnically Latino, with a wide range of ages (M = 43.4, SD = 13.4). Eighty one percent had completed more than 12 years of education, 71% were female, and 88% had at least one comorbid medical condition at baseline. Among the 439 patients, the primary anxiety disorders treated were: generalized anxiety disorder = 234 patients (53%), panic disorder = 125 patients (29%), social anxiety disorder = 60 patients (14%), and posttraumatic stress disorder = 20 patients (5%). Eighty three percent of the sample had at least one comorbid mood or anxiety disorder: generalized anxiety disorder = 337 patients (77%), panic disorder = 203 patients (46%), social anxiety disorder = 185 patients (42%), posttraumatic stress disorder = 78 patients (18%), major depressive disorder = 278 patients (63%), and dysthymia = 17 patients (4%).

Inter-item Reliability of Commitment to CBT and Homework Adherence Rating Scales

For the first 10 session-by-session Commitment to CBT ratings, Cronbach’s α was 0.92, indicating excellent internal consistency. For the first 10 session-by-session Homework Adherence ratings, Cronbach’s α was 0.78, indicating acceptable internal consistency.

Correlations between Measures of Dose and Engagement

Commitment to CBT was significantly positively correlated with Homework Adherence (r = 0.38, p < 0.001) and with Exposure Completion (r = 0.13, p = 0.01), and Exposure Completion was significantly positively correlated with Session Attendance (r = 0.81, p < 0.001). All other phi correlations between measures of dose and engagement were non-significant.

Correlations between Indices of Learning and Measures of Dose and Engagement

Ratings of CBT understanding were positively correlated with all measures of dose and engagement (rs from 0.12 to 0.20; ps < 0.02), ratings of CBT ability were positively correlated with Homework Adherence (r = 0.21, p < 0.001) and Commitment to CBT (r = 0.29, p < 0.001), and quiz performance was positively correlated with Commitment to CBT (r = 0.16, p = 0.002).

Main Analyses

For all the main analyses conducted and reported below, the propensity score weights effectively eliminated imbalances between groups in the observed baseline characteristics

Total CBT Sessions

Patients with High Attendance (total CBT sessions ≥ 6; n = 328) were compared to those with Low Attendance (total CBT sessions < 6; n = 111). Table 1 shows the 12-month and 18-month outcomes (i.e., the difference in mean outcomes for Low Attendance versus High Attendance) after the propensity score adjustment in the entire sample. After the propensity adjustment, patients with Low Attendance scored significantly higher (ps < 0.029) on all outcome measures than patients with High Attendance at both 12 and 18 months. Medium to large effect sizes were observed for attendance at 12 months, with Cohen’s ds ranging from 0.49 to 0.77, and medium effect sizes were observed for attendance at 18 months, with Cohen’s ds ranging from 0.48 to 0.50. In the concurrent pharmacotherapy subsample, patients with Low Attendance scored significantly higher (ps < 0.011) across all measures at 12 months, but not at 18 months (all subsample outcome analyses are reported in Supplementary Tables 1 and 2). In the no concurrent pharmacotherapy subsample, patients with Low Attendance scored significantly higher on the PHQ-8 at 12 and 18 months, and on the SDS at 18 months.

Table 1.

Outcomes for high and low dose and engagement groups after propensity weighting in entire CBT sample

12 Month Outcomes
Outcome Variable	Low Attendance– High Attendance	No Exposure– Completed Exposure	Less Homework Adherent– More Homework Adherent	Low Commitment– High Commitment
BSI-12	11.30–6.60 = 4.7	12.70–6.68 = 6.02	8.65–6.47 = 2.18	9.03–8.02= 1.01
(0-48)	(p < 0.001)	(p < 0.001)	(p = 0.012)	(p = 0.321)
	(d = 0.77)	(d = 1.00)	(d = 0.35)	(d = 0.16)
PHQ-8	8.91–5.57 = 3.34	9.77–5.21 = 4.56	7.17–5.13 = 2.03	7.59–6.32= 1.27
(0-24)	(p < 0.001)	(p < 0.001)	(p = 0.002)	(p = 0.117)
	(d = 0.73)	(d = 1.01)	(d = 0.44)	(d = 0.27)
SDS	10.93–7.50 = 3.43	12.19–6.57 = 5.62	8.83–6.27 = 2.56	9.81–7.77 = 2.04
(0-30)	(p = 0.008)	(p < 0.001)	(p = 0.002)	(p = 0.080)
	(d = 0.49)	(d = 1.02)	(d = 0.45)	(d = 0.36)
18 Month Outcomes
Outcome Variable	Low Attendance– High Attendance	No Exposure– Completed Exposure	Less Homework Adherent– More Homework Adherent	Low Commitment– High Commitment
BSI-12	10.57–7.40 = 3.17	11.57–7.22 = 4.35	9.03–6.00 = 3.03	9.95–6.81 = 3.13
(0-48)	(p = 0.029)	(p = 0.004)	(p < 0.001)	(p = 0.006)
	(d = 0.48)	(d = 0.66)	(d = 0.47)	(d = 0.47)
PHQ-8	8.04–5.76 = 2.27	8.79–5.67 = 3.12	7.24–4.79 = 2.44	8.11–5.36 = 2.75
(0-24)	(p = 0.018)	(p = 0.003)	(p < 0.001)	(p = 0.001)
	(d = 0.48)	(d = 0.67)	(d = 0.52)	(d = 0.59)
SDS	10.49–7.43 = 3.06	11.35–7.46 = 3.89	9.37–5.82 = 3.55	10.50–7.36 = 3.14
(0-30)	(p = 0.018)	(p = 0.003)	(p < 0.001)	(p = 0.006)
	(d = 0.50)	(d = 0.65)	(d = 0.59)	(d = 0.53)

Note. p-values adjusted using False Discovery Rate method

Completed Exposure

Patients who Completed Exposure (total exposure modules ≥ 1; n = 339) were compared to those who completed No Exposure (total exposure modules = 0; n = 100). Table 1 shows the 12-month and 18-month outcomes after the propensity score adjustment for the entire sample. After the propensity adjustment, patients with No Exposures were significantly more symptomatic (ps < 0.004) across all measures at 12 and 18 months than patients with Completed Exposure. Large effect sizes were observed for exposure completion on all outcome measures at 12 months, with Cohen’s ds ranging from 1.00 to 1.02, and medium to large effect sizes were observed on all outcome measures at 18 months, with Cohen’s ds ranging from 0.65 to 0.67. Similar trends were obtained in the two subsamples.

Homework Adherence

Patients who were More Homework Adherent (mean homework adherence rating ≥ 3.41; n = 218) were compared to those who were Less Homework Adherent (mean homework adherence rating < 3.41; n = 221). Table 1 shows the 12-month and 18-month outcomes after the propensity score adjustment in the entire sample. After the propensity adjustment, Less Homework Adherent patients scored significantly higher (ps < 0.012) across all measures at 12 and 18 months than More Homework Adherent patients. Small to medium effect sizes were observed for homework adherence at 12 and 18 months, with Cohen’s ds ranging from 0.35 to 0.59. In the concurrent pharmacotherapy subsample, Less Homework Adherent patients scored significantly higher (ps < 0.022) across all measures at 18 months, but not at 12 months. In the no concurrent pharmacotherapy subsample, Less Homework Adherent patients scored significantly higher (ps < 0.01) across all measures at 12 months, but not at 18 months.

Commitment to CBT

Patients with High Commitment to CBT (mean CBT commitment rating ≥ 8; n = 298) were compared to those with Low Commitment to CBT (mean CBT commitment rating < 8; n = 141). Table 1 shows the 12-month and 18-month outcomes after the propensity score adjustment in the entire sample. After the propensity adjustment, Low Commitment patients scored significantly higher across all measures at 18 months (ps < 0.006), but not at 12 months. Medium effect sizes were observed for commitment across all measures at 18 months, with Cohen’s ds ranging from 0.47 to 0.59. Similar trends were obtained in both subsamples.

Discussion

The goal of this study was to investigate the role of treatment dose and patient engagement in CBT for anxiety disorders as predictors of long-term treatment outcomes. A number of previous studies have examined dose and engagement in CBT, but most did not adequately control for the influence of baseline demographics and baseline symptom severity and therefore were unable to partial out the effects unique to dose and engagement. This study overcame those methodological limitations by using propensity score analyses, which adjust for group differences in baseline characteristics, with a large sample of participants afforded a wide range of dose and engagement. As hypothesized, higher treatment dose (attending more sessions, completing exposures) and patient engagement (being more adherent to assigned homework, demonstrating commitment to CBT) predicted significantly better symptom and functioning outcomes. The predictive effect of treatment dose was particularly robust; after controlling for the influence of baseline demographics and baseline symptom severity, completing exposures and higher attendance predicted better outcomes across multiple measures (BSI-12, PHQ-8, SDS) and multiple time points (12 and 18 months). There was also a strong predictive effect of patient engagement: after controlling for the influence of baseline demographics and baseline symptom severity, high attendance predicted better outcomes across all measures and time points, while higher commitment to CBT predicted better outcomes across all measures at a single time point (18 months). These findings are consistent with the conceptualization that treatment dose and engagement contribute to greater learning during CBT, and particularly to greater storage strength of learning (i.e., Bjork and Bjork, 1992). Having completed at least one exposure, relative to not having completed any exposures, was the strongest predictor of sustained long-term symptom reduction. Large effect sizes were observed for exposure completion at 12 months and medium to large effect sizes were observed at 18 months. This finding is consistent with the widely accepted finding that exposure is one of the most important treatment components in CBT for anxiety disorders (Longmore & Worrell, 2007). Overall, the results emphasize the value of dose and engagement in CBT as predictors of outcomes above and beyond baseline variables that may influence degree of dose, engagement, and outcome.

The subsamples of patients with versus without concurrent pharmacotherapy generally showed similar trends as each other, though the results in these smaller samples were less frequently statistically significant than in the entire sample due to diminished statistical power. In the subsample receiving CBT without concurrent pharmacotherapy, homework adherence predicted better outcomes at 12 months but was of limited significance for the 18-month outcomes. This contrasted with the subsample with concurrent pharmacotherapy, in which homework adherence predicted better outcomes at 18 months but was of limited significance for the 12-month outcomes. It is difficult to explain why the presence or absence of psychotropic medication would alter the benefits of homework adherence in CBT at 12 versus 18 months, but this trend suggests that future research should examine a potential interaction between the benefits of homework in CBT and concurrent medication status.

As outlined in the introduction, measures of dose and engagement are proxies for the time and effort devoted to practicing CBT skills, which underlay the learning processes believed to be central to outcomes in CBT. Most measures of patient learning were positively correlated with measures of dose and engagement. This finding supports the notion that dose and engagement are related to learning during CBT. However, the correlation coefficients were relatively small, and it is unclear whether that represents limitations in the measures or discordance between constructs.

The findings from this study have several important implications for improving clinical outcomes in CBT. First, given that high treatment dose and engagement predicted robust long-term symptom reduction, it may be beneficial to add a brief preliminary phase to CBT to augment dose and engagement. One possibility is motivational interviewing (Miller & Rollnick, 2002), forms of which were conducted in the CALM study. The addition of motivational interviewing to CBT for generalized anxiety disorder was shown to significantly improve outcome, mediated by increased homework compliance (Westra, Arkowitz, & Dozois, 2009). Another possibility is the pretreatment stage used in Dialectical Behavior Therapy (DBT; Linehan, 1993) in which several sessions preceding the beginning of formal therapy are dedicated to addressing issues of treatment dose and engagement including attendance, homework adherence, and commitment to treatment.

Second, dedicating clinician, institutional, and technological resources to addressing issues of dose and engagement may be an effective means of improving treatment outcomes. CBT for anxiety disorders may benefit from increased therapist attention to issues of dose and engagement, such as problem solving around barriers to attendance and homework adherence, and frequent elicitation of patient feedback about the therapeutic process. Risk for low CBT dose and engagement may be addressed in primary care settings through quality improvement interventions (i.e., training nurses and medical staff to educate patients about the importance of does and engagement in CBT), as quality improvement has been successfully used to improve treatment of depression for underserved ethnic minorities in primary care settings (Miranda et al., 2003). Novel means for making therapeutic materials more interesting, understandable, and useable may also be explored, such as presenting treatment materials through multi-media computer programs (as was done in the CALM study; Roy-Byrne et al., 2010) or through mobile phone applications (Bang, Timpka, Eriksson, Einar, & Nordin, 2010; Morris et al., 2010).

Several potential limitations of this study should be noted. First, these analyses examined exposure completion as a dichotomous variable, comparing the effect of completing at least one exposure with completing no exposures, and thus do not address whether there is an added benefit of completing multiple exposures over a single exposure. It is compelling that such robust effects were found for completing exposures with such a loose criterion for group membership in the Completed Exposure group. Given the consistency of findings about the importance of completing exposures in the treatment of anxiety disorders (Longmore & Worrell, 2007) it seems likely that completing multiple exposures would be more beneficial than completing just one. Second, the propensity score analyses only corrected for observed baseline confounders. Though the analyses accounted for a wide range of potential differences in demographics and symptom severity, unobserved or unaccounted variables may have explained differences between patients high and low in dose and engagement, such as personality traits, or attitudes towards psychotherapy. Third, the dose variables included in this study (exposure completion and session attendance) had some overlap with one another. This overlap was predictable given that exposures were conducted in the latter part of treatment, meaning the likelihood of completing an exposure increased as patients attended more CBT sessions. Nevertheless, given that completing an exposure predicted large effect sizes at 12 months and medium to large effect sizes at 18 months, while high attendance predicted medium to large effect sizes at 12 months and medium effect sizes at 18 months, it seems that exposure completion and session attendance are at least partially distinct constructs. A fourth limitation of this study is that homework adherence may have differed from other dose and engagement measures due to variance in the methods through which the measures were collected; homework adherence was a clinician rating of patient behavior outside of the therapy setting, while commitment, attendance and exposure completion were measures of in-session patient behavior. Homework adherence ratings may have been more biased by patient self-report than the other measures. Also, there may be a fundamental distinction between in-session engagement and dose (i.e. commitment, attendance, exposure completion) and between-session engagement (homework adherence). Also, this study examined quantity, rather than quality, of homework completion. Quality and quantity of homework compliance appear to be highly correlated aspects of patient engagement in CBT, though there have been mixed findings as to which better predicts outcome (Rees et al., 2005; Schmidt & Woolaway-Bickell, 2000). Future research should further examine the extent to which quality versus quantity of homework compliance better predict outcome. A fifth limitation of this study is the failure to measure the impact of therapeutic alliance (e.g. mutual sense of trust, joint agreement of treatment goals and treatment approach) on dose, engagement, and outcome. Dearing, Barrick, Dermen, and Walitzer (2005) found that in addition to the effects of session attendance and positive expectations about therapy, greater therapeutic alliance contributed to better treatment outcome. The role of the therapeutic alliance in CBT has not yet been thoroughly tested, and future research should examine whether alliance is best understood as a non-specific factor that contributes directly to treatment outcome, or as a factor that interacts with dose or engagement to influence outcome. A sixth potential limitation is that the clinicians occasionally used strategies such as behavioral activation and cognitive restructuring to treat symptoms of depression, and motivational enhancement strategies to maintain patient engagement, which may have altered the relationship between aspects of treatment dose, patient engagement, and outcome. While there is a potential confound of specific interventions utilized by clinicians having influenced the dose and engagement variables, the primary target of our analyses was with treatment-related aspects of dose (i.e. how much treatment was received) and patient-related aspects of engagement (i.e. how actively involved were patients in treatment) rather than clinician-related aspects of dose and engagement (i.e. how did clinicians themselves influence outcomes). A seventh limitation is that no formal reliability ratings were collected for data entered by clinicians into the web-based tracking system, although study supervisors regularly reviewed data entry. An eighth limitation is the measures of engagement (homework adherence and commitment to CBT) were unvalidated. There is a lack of well validated measures of homework adherence: the Homework Rating Scale (HRS; Kazantzis et al., 2004) has been proposed as a measure of homework compliance but has yet to be psychometrically validated, and the Observer-Based Behavioral Activation Homework Completion Measure (Busch, Uebelacker, Kalibatseva, & Miller, 2010) has only been psychometrically tested in a small sample (24 therapy sessions drawn from 12 patients). Several validated measures are available to assess patient involvement/commitment to therapy, such as the “observing patient involvement in decision making” scale (OPTION scale; Elwyn et al, 2003) and the Working Alliance Inventory (Fenton, Cecero, Nich, Frankforter, & Carroll, 2001), but these were not ideal matches for the current study as they are measures of shared patient and clinician decision making while the goal of this study was to examine the patient-related factors of CBT engagement. A final limitation is that delivery of CBT was moderately flexible, with clinicians able to guide treatment within the constraints of the computer-guided CALM Tools for Living program and weekly supervision. It is possible that different results might have been found had CBT delivery been completely standardized across all patients.

Overall, these current findings highlight the sizeable and lasting effects of treatment dose and patient engagement on outcomes in CBT. After controlling for baseline differences in demographics and symptom severity, greater symptom reductions and functioning improvements were predicted by higher dose and engagement, and particularly by completing at least one exposure component, having high session attendance, and adhering to assigned homework. Future research should target identifying and clarifying which measures of treatment dose and patient engagement best predict long-term outcome, and which individual characteristics best predict dose and engagement. As the contribution of dose and engagement to treatment outcomes is more fully understood, the development of interventions which augment dose and engagement may be an especially fruitful means of optimizing long-term CBT outcomes.