Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Mar 1.
Published in final edited form as: Behav Ther. 2017 May 1;49(2):295–307. doi: 10.1016/j.beth.2017.04.004

Efficacy of Guided iCBT for Depression and Mediation of Change by Cognitive Skill Acquisition

Nicholas R Forand 1, Jeffrey G Barnett 2, Daniel R Strunk 3, Mohammed U Hindiyeh 2, Jason E Feinberg 4, John R Keefe 5
PMCID: PMC5853137  NIHMSID: NIHMS919812  PMID: 29530267

Abstract

Guided internet CBT (iCBT) is a promising treatment for depression; however, it is less well known through what mechanisms iCBT works. Two possible mediators of change are the acquisition of cognitive skills and increases in behavioral activation. We report results of an 8-week waitlist controlled trial of guided iCBT, and test whether early change in cognitive skills or behavioral activation mediated subsequent change in depression. The sample was 89 individuals randomized to guided iCBT (n = 59) or waitlist (n = 30). Participants were 75% female, 72% Caucasian, and 33 years old on average. The PHQ9 was the primary outcome measure. Mediators were the Competencies of Cognitive Therapy Scale-Self Report and the Behavioral Activation Scale for Depression-Short Form. Treatment was Beating the Blues plus manualized coaching. Outcomes were analyzed using linear mixed models, and mediation with a bootstrap resampling approach. The iCBT group was superior to waitlist, with large effect sizes at post-treatment (Hedge’s g = 1.45). Dropout of iCBT was 29% versus 10% for waitlist. In the mediation analyses, the acquisition of cognitive skills mediated subsequent depression change (indirect effect = −.61, 95% bootstrapped biased corrected CI: −1.47, −0.09), but increases in behavioral activation did not. iCBT is an effective treatment for depression, but dropout rates remain high. Change in iCBT appears to be mediated by improvements in the use of cognitive skills, such as critically evaluating and restructuring negative thoughts.

Keywords: depression, guided iCBT, mediation, cognitive skills, behavioral activation

Guided, computerized cognitive behavioral therapy (CBT) delivered over the internet (iCBT) is an effective treatment for depression that might improve access to empirically supported treatments (Andersson & Cuijpers, 2009; Andrews, Cuijpers, Craske, McEvoy, & Titov, 2010). When iCBT is provided with therapist support (hereafter referred to simply as guided iCBT), the treatment is cost-effective and acceptable to users (Andersson & Cuijpers, 2009; Andrews et al., 2010), and might have similar effects to group or standard CBT (Andersson et al., 2013; Spek et al., 2007; Wagner, Horn, & Maercker, 2014).

Despite these positive outcomes, few of the commercially available iCBT programs in the United States have been adequately tested. One available program is Beating the Blues (BtB), which was originally developed and tested in the United Kingdom. Early studies in primary and secondary care settings showed promising outcomes (Cavanagh et al., 2006; Proudfoot et al., 2004), however, a recent large comparative effectiveness study found BtB was no better than primary care alone (Gilbody et al., 2015). Further, the way in which BtB was delivered in early studies, in primary care offices on computers without coaching, differs markedly from current practices in mobile health, where interventions are typically delivered at home and can include manualized coaching. The United States version of BtB has been “Americanized,” and had its content, interface and platform modernized to bring the program in line with current trends in mobile health (see Figure 1). In contrast to the well-studied United Kingdom version of BtB, the efficacy of the updated American version has not been tested. The validation of BtB and other commercially available programs is needed to determine their effectiveness for the United States population.

Figure 1. Differences between Beating the Blues Studies and Versions.

Figure 1

Open in a new tab

Mediation of Change in iCBT

There are also questions about the mechanisms of change in guided iCBT. Mechanisms of change in psychological treatments are best studied through a test of mediation (Baron & Kenny, 1986; Lemmens, Müller, Arntz, & Huibers, 2016). Mediation analyses test whether the pattern of relations observed is consistent with a potential mediator serving as a causal link between an independent and a dependent variable, such as would be expected if the mediator were a mechanism through which the independent variable impacted the dependent variable.

One potential mechanism of change in iCBT is the acquisition of cognitive skills. The process of teaching cognitive therapy skills is intended to provide patients with a novel set of cognitive responses that minimize distress, reduce maladaptive emotional and behavioral responses, and help prevent relapse (Hundt, Mignogna, Underhill, & Cully, 2013). In standard CBT for depression, patient performance of cognitive skills has been related to reductions in depressive symptoms and lower risk of relapse (Jarrett, Vittengl, Clark, & Thase, 2011; Strunk, DeRubeis, Chiu, & Alvarez, 2007). Skills change has also been found to mediate change in depression relative to a control condition in a CBT group therapy (Gallagher-Thompson, Gray, Dupart, Jimenez, & Thompson, 2008).

There is debate about whether changes in CBT are achieved through individuals applying cognitive skills in response to stressors (a compensatory skills model) or through changes in the content of dysfunctional cognition (see Adler, Strunk, & Fazio, 2015; Barber & DeRubeis, 2001; Barber & DeRubeis, 1989). In contrast to measures of skill use, measures of dysfunctional cognition assess the content and severity of maladaptive automatic thoughts and underlying beliefs (Beck, Rush, Shaw, & Emery, 1979). The cognitive model hypothesized that change in cognition was key to the therapeutic benefits of CBT. However, some authors have argued that consideration of effortful use of skills may be needed to better understand how change occurs in CBT, and by extension, iCBT (Hundt et al., 2013). Unlike measures of dysfunctional cognition, which often change equally irrespective of treatment modality (Lorenzo-Luaces, German, & DeRubeis, 2015), there is some evidence that change in skills may be specific to CBT (Hunter et al., 2002; Kuyken et al., 2010; Powers, Thompson, & Gallagher-Thompson, 2008). Further, CBT skill use might inform two potential mechanisms of change in depression, the first being the use of skills to reduce distressing emotions and maladaptive behaviors that can result from stressors, and the second the use of skills to directly change underlying beliefs (for a discussion, see Barber & DeRubeis, 1989; Hundt et al., 2013 and Jacobson et al., 1996). The acquisition of skills might be even more important as a potential change mechanism in iCBT, where there is little opportunity for other change mechanisms such as the therapeutic alliance to exert influence. Thus far, no studies have measured the acquisition of cognitive skills as a mediator of change in iCBT, and methodological problems have often undermined efforts to study cognitive skills as a mediator in the CBT more generally (Hundt et al., 2013).

Another possible mediator of change in iCBT is behavioral activation (BA). BA includes strategies to increase the frequency of reinforcing activities and reduce behavioral avoidance (Addis, Jacobson & Martell, 2001). Critics of cognitive therapy have noted that much change in CBT occurs early, before many cognitive interventions have been offered and during a period often prescribed for behavioral strategies (Ilardi & Craighead, 1994; Jacobson et al., 1996). Jacobson et al. (1996) found that BA was no different from a full CBT package and that they lead to similar amount of change in dysfunctional thinking, suggesting that the BA component of CBT was sufficient to produce change. Although there are several studies of internet-based BA treatment packages and many iCBT packages include BA strategies, none have yet tested whether BA mediates outcome in iCBT for depression.

Studies that have assessed mediation of change in iCBT for depression have focused on changes in cognitive content or repetitive negative thinking. Warmerdam, van Straten, Jongsma, Twisk & Cuipers (2010) found that dysfunctional attitudes, worry, negative problem orientation, and perceived control mediated change in both iCBT and an online problem solving therapy versus waitlist. In a study of transdiagnostic iCBT for depression and anxiety, Newby, Williams, & Andrews (2014) found that reductions in positive beliefs about repetitive negative thinking (rumination and worry) were associated with reductions in negative thinking, which mediated change in depression in comparison to a waitlist control. These studies are notable because they have a number of the recommended design characteristics for a test of mediation (Kazdin, 2007; Lemmens et al., 2016), including: 1) randomized trial, 2) control group (waitlist), 3) N ≥ 40, 4) testing multiple mediators, and 5) fulfilling the temporal criterion (change in the mediator precedes change in the dependent variable). Thus far, no study of iCBT has met these criteria when measuring the acquisition of CBT skills or BA.

The Current Study

In the current study, we present results from a randomized trial of guided iCBT versus wait-list control. Our objectives are to: (1) test the efficacy of iCBT supported using TeleCoach versus waitlist, and (2) determine whether change in iCBT is mediated through the acquisition of cognitive skills or through increases in BA. Beating the Blues includes both cognitive and behavioral strategies early in treatment, but tends to emphasize cognitive strategies (see Figure 2). Therefore, we hypothesize that iCBT will be (1) superior to waitlist and (2) will be mediated through cognitive skill acquisition.

Figure 2. Beating the Blues v2.5 Session Summaries and Project.

Figure 2

Open in a new tab

Note. Topics listed include only those introduced in that session. Some topics are reviewed or added to in subsequent sessions, including the ABC model (sessions 2 and 3), pleasurable activities (session 3), and problem solving (sessions 4 and 5).

Methods

Design

Participants were randomly assigned to eight weeks of guided iCBT or waitlist. Waitlist participants who continued to meet the initial inclusion after 8 weeks were offered the iCBT treatment (delayed iCBT). For information on outcomes for the delayed treatment, please see the supplemental materials. Because an aim of the study was to identify predictors of response, 2/3rds of participants were randomized to immediate treatment and 1/3 were randomized to waitlist. The trial was conducted between April 2014 and September 2015 at The Ohio State University Wexner Medical Center in Columbus, OH. Randomization was implemented through data capture software (REDCap). The trial was approved by the Ohio State University Institutional Review Board and registered on ClinicalTrials.gov with identifier NCT01990053.

Sample

Treatment-seeking adults were recruited by: (a) online advertisements and notices (33 of 89, 37%), (b) flyers placed in the community (31 of 89, 35%), (c) ads in newspapers (1 of 89, 1%), and (d) ResearchMatch.org (9 of 89, 10%).1 To be eligible for a full assessment, potential participants need to report the following during an initial phone screen: (a) a score of ≥ 2 on the first 2 questions of the PHQ9 (Kroenke, Spitzer, & Williams, 2001) and ≥8 on the rest of the measure (Davidson et al., 2006). Such participants were reporting the “cardinal” depression symptoms of low mood and anhedonia and meeting an overall severity cutoff of ≥10 (see Kroenke, Spitzer, & Williams, 2001). Additional inclusion criteria were: (b) 18 years of age or older; (c) able and willing to give informed consent; and (d) having access to a computer with an internet connection. Exclusion criteria were: (a) history of bipolar disorder or psychosis; (b) current primary Axis I disorder other than major depressive disorder; (c) substance dependence within the past six months; (d) subnormal intellectual potential (IQ below 80) as measured by the Shipley Scale for Living (Shipley, 1940); (e) clear indication of secondary gain such as court-ordered therapy; (f) current imminent suicide risk or significant intentional self-harm in the last six months; (g) discharge within 6 months from a higher level of care; (h) currently undergoing outpatient psychotherapy; (i) a change in antidepressant medication over the past month or a planned change over the duration of the study, and (j) inability to read and write English. Diagnostic exclusions were determined by the MINI 6.0.0 diagnostic interview. Following the baseline in-person evaluation, all subsequent contacts were via phone or email.

Treatment

The iCBT program used in this study was the Americanized version of Beating the Blues United States v2.5 (BtB) licensed by U2 Interactive, LLC. BtB is an 8-session computerized, internet-delivered, cognitive behavioral treatment for depression. The changes made to the program for the United States population are detailed in Figure 1. Figure 2 lists the topics covered in each session and the between-session projects. The program has interventions directed primarily towards addressing mood symptoms, but also includes some interventions for anxiety (e.g., diaphragmatic breathing and optional graded exposure). Sessions take approximately 45 minutes to an hour to complete, and participants were encouraged to complete one session per week. The program is not tailored; although participants are given the option of choosing one of four different techniques in session 4 (see Figure 2).

A coach (NRF or DRS) was assigned to each participant to provide support and guidance using the TeleCoach protocol adapted for BtB by author NRF (Duffecy, Kinsinger, Ludman, & Mohr, 2011). The protocol is based on the supportive accountability model (Mohr, Cuijpers, & Lehman, 2011), which focuses on building reciprocity in the coaching relationship, informing the user about expected behaviors, setting expectations for progress, building expectations for accountability, and rewarding adherence. The TeleCoach protocol (Duffecy et al., 2011) was found to improve adherence (but not outcomes) in a randomized trial of iCBT for depression (Mohr et al., 2013). The coaching schedule included 3 pre-arranged calls and 1 pre-arranged email before week 4, additional contact as needed after week 4, and on-call support. Coaches spent about 1 to 2 hours in total working with each participant.

Measures

Mini-International Neuropsychiatric Diagnostic Interview 6.0.0 (MINI600)

The MINI (Sheehan et al., 1998) is a brief structured diagnostic interview covering 19 disorders commonly seen in psychiatric practice. Diagnoses correspond to Diagnostic and Statistical Manual Fourth Edition (DSM-IV) and International Statistical Classification of Diseases and Related Health Problems (ICD-10) psychiatric disorders. The MINI has shown good concurrent validity with respect to these instruments (Sheehan et al., 1998).

Patient Health Questionnaire-9 (PHQ9)

The PHQ9 (Kroenke & Spitzer, 2002) is a 9-item self-report measure based on the diagnostic criteria for major depression from the Diagnostic and Statistical Manual Fourth Edition (DSM-IV). It was the primary self-report measure of depression symptoms.

Hamilton Rating Scale for Depression (HRSD)

The 17-item HRSD is an interview measure of depressive symptom severity (Hamilton, 1960; Williams, 1988). The current study used the version modified to include reverse vegetative symptoms (Reimherr et al., 1998), which has been used in previous trials of depression (DeRubeis et al., 2005; Dimidjian et al., 2006). HRSD raters (authors JGB and JEF) were trained by authors NRF and DRS. Based on a sample of 20 double-rated HRSDs, the ICC for the HRSD scores was .98.

Competencies of Cognitive Therapy Scale – Self Report (CCTS-SR)

The CCTS-SR (Strunk, Hollars, Adler, Goldstein, & Braun, 2014) is a 29-item measure of skills taught in cognitive therapy for depression. Items, rated on a 7-point scale, measure three areas in which patients are expected to acquire competencies from participating in CBT: coping with automatic thoughts, knowledge of schemas or core beliefs, and BA. The measure includes two items related to BA, but overall is heavily oriented towards assessing cognitive skills. The original validation study indicated a one factor solution (Strunk et al., 2014). An example item is: “I examined evidence from my past or present to more carefully consider whether my negative thoughts are accurate or not.” The CCTS-SR has good internal reliability and predicts and correlates with change in cognitive therapy for depression (Strunk et al., 2014).

Behavioral Activation for Depression Scale– Short Form (BADs-SF)

The BADS-SF. The BADS-SF (Manos, Kanter, & Luo, 2011) was derived from the original Behavioral Activation for Depression Scale (Kanter, Mulick, Busch, Berlin, & Martell, 2007), which was developed to measure skills from the BA theory of depression. Some subscales showed considerable overlap with depressive symptoms, leading Manos et al. (2011) to develop short form including items less highly correlated with symptoms. The BADS-SF includes Activation and Avoidance/Rumination subscales, and has good internal reliability, correlates with behavioral changes over treatment, and predicts changes in depression during treatment (Manos et al., 2011).

Assessment Schedule

The HRSD was collected at intake and post-treatment. The intake assessment occurred prior to randomization; however, the HRSD raters were not blind to treatment condition at the post-treatment assessment. The PHQ9 was completed at intake and weeks 3, 5, and 8. CCTS-SR, and BADS-SF were completed at intake, week 3 and week 8. All self-report measures were administered via computer or mobile device using the REDCap survey tool. The Waitlist group had the same assessment schedule as the iCBT group. Week 3 was chosen to measure potential mediator variables in order to capture early change processes (cf. Newby et al., 2014). These week 3 assessments could occur either prior to or after session 3. Most change in depression takes place early in treatment, and early change predicts both shorter and longer-term outcomes (Tadić et al., 2010). In addition, key process measures have been found to predict symptomatic changes very early in treatment (e.g., Strunk, Brotman, DeRubeis, & Hollon, 2010). BtB has both cognitive and behavioral components over its first three sessions, including psychoeducation on influence of thoughts and behaviors on mood, thought recording and cognitive distortions, and the importance of pleasurable activities.

Data Analysis

iCBT vs Waitlist comparison

The iCBT and waitlist samples were compared using linear mixed models, adjusting for depression scores and the use of antidepressant medication at intake in the model (Raudenbush & Bryk, 2002). These models were estimated in SAS PROC MIXED using full maximum likelihood and an unstructured covariance structure. Rates of dropout, reliable and clinically significant change, and remission were compared using Fisher’s exact tests. In iCBT, individuals may determine how much treatment they need, discontinuing sessions when they decide to do so. In light of this, participants were classified as having dropped out only if the study team lost contact with them during the treatment period or if they did not complete a week 8 phone interview. To be consistent with previous investigations using the PHQ9, reliable change was set at ≥5 points, and clinically significant improvement required a score of <10 (McMillan, Gilbody, & Richards, 2010). Primary remission was defined as a final PHQ9 score of <5, as recommended by Kroenke et al., (2001). A secondary measure of remission was defined as a week 8 HRSD score of <8 as recommended by Frank et al. (1991).

Mediation Analyses

Mediation analyses were conducted using PROCESS (Hayes, 2013). Hayes (2009) recommended the use of bootstrapped confidence interval to detect indirect (mediation) effects over these methods, especially for smaller samples. Bootstrapping is a nonparametric resampling approach used in PROCESS to generate an estimate of the indirect effect (the ‘ab’ term). In multiple bootstrap samples sampled with replacement from the original sample, PROCESS calculates the ‘ab’ term and then creates 95% bias corrected confidence intervals from this distribution of terms. If zero is not within the 95% confidence interval, the indirect effect is significant, meaning that that effect of treatment on outcome is at least partially contingent on change in the mediator (Preacher & Hayes, 2008). We used the PROCESS macro for SAS and constructed 10,000 bootstrapped resamples, as recommended by Hayes (2013). Mediators were tested simultaneously in a parallel or competing mediator model.

Several authors have argued that change in a mediator must precede change in an outcome variable to provide a powerful test of whether the potential causal effect of treatment on outcome may have been achieved through the mediator (Hollon & DeRubeis, 2009; Kazdin, 2007; Lemmens et al., 2016). In the current analysis, the mediator is change in skills and change in BA from intake to week 3, and the outcome variable is residualized change in depression symptoms on the PHQ9 from weeks 3 to 8, controlling for residualized change in depression from intake to week 3. Thus, this study fulfills the temporal criterion. Further, it meets 4 of the remaining 5 quality metrics for mediation research as outlined by Lemmens et al. (2016): 1) the use of an randomized controlled trial design, 2) the use of a control group, 3) sample size > 40, and 4) examination of multiple potential mediators. The unmet metric is direct experimental manipulation of the mediator.

Missing Data

PROCESS has no provision for handling missing data, so missing data were imputed for the mediation analyses. Random forest imputation (Stekhoven & Bühlmann, 2012) was used under the assumption that data were missing at random (Rubin & Little, 2002). Random forest imputation is a single-dataset imputation method that can produce nominally equivalent or more accurate imputations compared to standard multiple imputation by chained equations (Liao et al., 2014; Stekhoven & Bühlmann, 2012; Waljee et al., 2013). Random forest algorithms produce nested prediction or decision trees based on splits in continuous or categorical predictor variables (King & Resick, 2014) Each individual tree selects a bootstrapped subset of the main data set, and at each potential split point allows a random subset of possible predictors to be used, until there are no more possible splits to be made. This process is repeated across many different forests of trees, wherein predictions for the missing variables are iteratively updated and improved with successive models, until a stopping criterion is reached. A given imputed data value is based on the average prediction made across several decision trees. For this data set, a normalized root mean square error of prediction was estimated at 0.27, indicating that the imputation was adequately accurate (Stekhoven & Bühlmann, 2012).

Results

Participants flow through the trial is reported in the CONSORT diagram (Figure 3). The randomized iCBT sample (n = 89) was 75% women (67 of 89) and 72% White (64 of 89), with 16% identifying as Black (16 of 89). A total of 29% were married or cohabiting (26 of 89), 89% employed (79 of 89) and 46% (41 of 89) had a degree from a 4-year college or higher. Based on assessment with the MINI, 87 (of 89) participants (98%) were diagnosed with major depression and 52% (46 of 89) were diagnosed with another Axis I disorder. A total of 38% (34 of 89) reported currently taking antidepressant medications. The average intake PHQ9 score was 17.07 (SD = 4.34) and the average intake HRSD was 21.12 (SD = 5.19). Full descriptive statistics are provided in the online supplement. There were no significant differences between the groups on any variables (all ps > .10). Individuals initially randomized to iCBT completed an average of 6.22 sessions (SD = 2.32). The rate of participants completing all 8 sessions of the program was 56% (33 of 59). The rate of participants dropping out of iCBT was greater than that observed in the waitlist condition, but only at the level of a nonsignificant trend (17 of 59, 29% vs. 3 of 30, 10%; Fishers exact test p = .06. One individual experienced worsening depression symptoms and was rescued from the waitlist and started on iCBT, at which point the data from this individual are censored from the analyses.

Figure 3. CONSORT Diagram.

Figure 3

Open in a new tab

Correlations

The PHQ9 and HRSD were correlated at r = .46 (p < .001). The CCTS-SR was uncorrelated with either the PHQ9 (r = −.01, p = .91) or the HRSD (r = .17, p = .11), indicating little overlap between cognitive skills and depression severity. However, the BADS-SF was moderately correlated with the PHQ9 (r = −.50, p < .001) and the HRSD (r = −.44, p < .001), suggesting that a portion of variance in the BADS-SF may have been attributable to the severity of depression symptoms. Of note, the BAD-SF and CCTS-SR were uncorrelated at intake (r = .07, p = .54), suggesting that they measure different sets of responses.

Randomized Comparison of iCBT vs. Waitlist

Table 1 shows the means and standard deviations of depression scores, the CCTS-SR, and the BADS-SF. A visual inspection of depression scores over time revealed a linear pattern of change. An unconditional model showed that the linear time model fit better than a model using log time so a linear model was selected. The intercept of this model was set at week 8 to enable a test of treatment on post-treatment depression. Intake PHQ9 scores and whether the patient was taking antidepressant medication were included as covariates in the model. The model showed a significant effect of time (b = −.70, t = −2.27(74), p < .05), as well as a time x treatment interaction (b = 1.05, t = 7.49(75.4), p < .001), indicating that iCBT was associated with greater depression change over 8 weeks compared to waitlist (Hedge’s g = 1.67). The iCBT group was also associated with significantly lower PHQ9 scores at week 8 (b = −7.70, t = −6.47(78.8), p < .001, Hedge’s g = 1.45). A graph of PHQ9 scores is provided in the online supplement. This figure also displays PHQ9 scores for the Waitlist participants who entered into delayed iCBT after week 8. There was no effect of current depression medication on the slope or the intercept.

Table 1.

Pre and Post-Treatment Means for iCBT vs. Waitlist

iCBT
M (SD)		 Waitlist
M (SD)		 t-Tests
PHQ9 intake 16.55 (3.99) 18.22 (4.84) t = −1.73, p = .09, g = −0.31
PHQ9 week 3 11.77 (4.60) 14.28 (5.35) t= −2.31, p<.01, g = −0.47
PHQ9 post 6.45 (4.20) 14.75 (6.68) t = −7.16, p < .001, g = −1.59
HRSD intakea 20.90 (5.20) 21.57 (5.23) t = −.57, p = .57, g = −0.13
HRSD post 9.20 (5.28) 18.56 (6.96) t = −6.45, p < .001, g = −1.55
CCTS-SR intake 90.57 (30.62) 81.03 (18.17) t = 1.57, p = .12, g = 0.35
CCTS-SR week 3 109.40 (28.34) 81.67 (24.48) t = 4.57, p < .001, g = 1.03
CCTS-SR post 139.11 (27.71) 82.18 (33.78) t = 8.50, p<.001, g = 1.89
BADS-SF intake 20.86 (7.13) 21.00 (7.92) t = −.08, p = .93, g = −0.02
BADS-SF week 3 28.49 (7.09) 21.41 (10.47) t = 3.77, p < .001, g = 0.84
BADS-SF post 33.08 (7.63) 23.74 (9.65) t = 4.98, p < .001, g = 1.11
Open in a new tab

Note. Post-treatment scores use imputed data unless otherwise noted. PHQ9, Patient Health Questionnaire – 9; HRSD, Hamilton Rating Scale for Depression; CCTS-SR, Competencies of Cognitive Therapy Scale – Self Report; BADS-SF, Behavioral Activation for Depression Scale – Short Form; g = Hedges’s g effect size

a

HRSD scores were not imputed. Post-treatment scores represent completers only, iCBT n=45, Waitlist n = 27.

Secondary Outcomes

In a linear mixed model using the HRSD, there was again a significant effect of time (b = −3.65, t = −3.29(74.4), p < .01), and time x treatment, with iCBT being superior to waitlist (b = −9.40(75.2), t = −6.96, p < .001, Hedge’s g = 1.56). The iCBT treatment also had lower depression scores at post treatment (b = −9.84 (77), t = −6.96, p<.001, Hedge’s g = 1.56). Current antidepressant medication had no effect on the slope or intercept.

Improvement and Remission

In the iCBT sample, 83% of participants (49 of 59) experienced reliable change in symptoms, compared to 30% (9 of 30) in the waitlist condition (Fishers exact test p <.001). In the iCBT sample, 73% (43 of 59) experienced both reliable and clinically significant change, compared to 23% (7 of 30) of the waitlist condition (Fishers exact test p <.001). Using PHQ9<5, the remission rate of iCBT was 34% (20 of 59) versus 7% (2 of 28) for waitlist (Fishers exact test p < .01). Using the HRSD<8, the remission rate of iCBT was 31% (18 of 59) versus 0% (0 of 28) for waitlist (Fishers exact test p < .001).

Mediation of Change

For illustration, we present the results of the mediation model in Figure 4. As seen in the figure, treatment had a significant effect on cognitive skills change and increase in BA, with iCBT resulting in more change than waitlist. Further, skills change predicted greater subsequent depression change, but BA did not. Skills change was also estimated to mediate change in depression. The indirect effect of treatment on outcome via skills change was estimated to be −.61 (SE = .34), and the 95% bootstrapped bias corrected confidence interval was −1.47, −0.09. This indicates that the change in depression in iCBT was partially contingent on change in cognitive skills. The corresponding and non-significant indirect effect for BA was −.16 (SE=.46) and the 95% bootstrapped bias corrected confidence interval was −1.04, 0.79.2

Figure 4. Mediation of Change in Depression by Change in CBT Skills and Behavioral Activation.

Figure 4

Open in a new tab

Note. Betas are unstandardized, and the parentheses contain 95% bootstrapped biased corrected confidence intervals. All models controlled for change in PHQ9 from weeks 0–3. PHQ9, Patient Health Questionnaire – 9; CCTS-SR, Competencies of Cognitive Therapy Scale – Self Report; BADS-SF, Behavioral Activation for Depression Scale – Short Form.

Discussion

In this study, we presented the results of a randomized trial of guided iCBT versus waitlist. Guided iCBT outperformed the waitlist condition on both primary and secondary measures of depression, with large effect sizes associated with both change in depressive symptoms and post-treatment levels of depressive symptoms. The observed remission rate of 34% is slightly lower but comparable to the estimated remission rate of 43% in face to face therapy in depression calculated by Cuijpers et al. (2014). These results add to the growing literature that guided iCBT is an effective treatment for depression.

Also consistent with the literature, iCBT had a high rate of dropout (29%). This rate was similar to the 31% weighted average rate for iCBT interventions, despite our use of a coaching protocol specifically designed to promote adherence to the intervention (Mohr et al., 2013, 2011). In previous studies, dropout rates for Beating the Blues were 27.4 % when delivered in primary care offices (Proudfoot et al., 2004), 38.4% when delivered in secondary care offices (Cavanagh et al., 2006) and 21.4% when delivered over the internet in conjunction with coaching and standard care (Gilbody et al., 2015). Each of these are higher than the estimated 17.5% dropout rate in face to face psychotherapy for depression (Cooper & Conklin, 2015). The rate of completing all 8 iCBT sessions in the current study (56%) is also within the range of estimates reported by Christensen, Griffiths, and Farrer (2009) for iCBT for depression (50–70%). We did not include a no-coaching condition, so a comparison to unguided iCBT was not possible; however, it is notable that the coaching protocol did not appear to lower rates of dropout substantially below that observed in trials of this program without coaching. It is possible that we have reached the upper limit of adherence to iCBT programs that can be achieved with manualized coaching.

Programs like BtB are designed to mimic face to face psychotherapy; for example, they recommend one session per week and deliver their content in a specific sequence. However, some have argued that such an approach is at odds with how people prefer to use mobile technology (Lattie et al., 2016). A recent study showed that a suite of simple “single use” smartphone apps, including one for BA and one for cognitive restructuring, accessible whenever the user needed them and delivered with coaching, had a dropout rate of less than 10% over 8 weeks (Mohr et al., 2017). Successful implementation of digital behavioral health might therefore require the development of new models of delivering care, rather than a simple digitizing of traditional models (Schueller, Muñoz, & Mohr, 2013).

For sites using more convention iCBT interventions, the effectiveness of these treatments must be weighed against the risk of individuals dropping out. It is likely that these treatments significantly increase the reach of evidence-based therapies, including reaching those who would not seek out traditional face-to-face therapy and those who would find such treatment overly burdensome. A possible direction for addressing the dropout problem is to develop methods of identifying individuals who are at risk for poor outcomes before treatment initiation or early in its course, so that they might be directed to alternatives. Such methods might include identifying risk factors, or methods that can identify deviation from acceptable symptom change trajectories (De Jong et al., 2014; Delgadillo, Moreea, & Lutz, 2016).

In the meditation analyses, we found that early improvement in cognitive skills mediated change in subsequent depression scores in guided iCBT versus waitlist. These cognitive skills are specifically related to identifying negative thoughts and their relationship with negative emotions, rationally evaluating these thoughts, and engaging in active coping efforts in response to problems or negative mood. Patients who reported a greater change in the use of these skills from intake to week 3 improved more on subsequent depression scores, suggesting that those individuals who learned and applied the cognitive skills taught in Beating the Blues tended to benefit more from the program.

In contrast, although iCBT was related to greater improvements in BA than waitlist, we failed to find that change in BA mediated subsequent change in depression symptoms. Given the program’s relative emphasis on cognitive strategies early in treatment, it is possible that there is not enough BA content to promote meaningful change in these behaviors. The change that did occur in BA might also have been a consequence of improving depression symptoms. Indeed, the BA scale was highly correlated with depression at intake, a problem that was also observed with original version of the BADS (Manos et al., 2011). Given the high level of covariation, disentangling whether the effects of BA are a cause or consequence of (or the same as) improving depression symptoms will require additional research.

Cognitive skills, in contrast, were unrelated to depression at intake. Our findings suggest that skills measures might be good candidates for investigating mechanisms of change. In addition to being unrelated to depression in this sample, cognitive skills have also been shown to have specificity to CBT treatments in comparison to other treatments (Hunter et al., 2002; Kuyken et al., 2010; Powers et al., 2008). Identifying which skills are most closely related to improvements in depression might provide a means of increasing the effectiveness of iCBT. By tracking improvements in skills, it might be possible to identify individuals who are at risk for poor response early in treatment, and provide them with additional help.

Limitations and Future Directions

The current manuscript has several strengths. It tested an updated version of a well-studied iCBT intervention, it used a validated coaching protocol designed to improve adherence to the programs (TeleCoach), it included both self-report and interviewer-rated measures of depression severity, and it met five of the six criteria described by Lemmens et al. (2016) as markers of quality for research on mediators. These included testing multiple mediators and meeting the temporal criterion.

Nevertheless, there were also several limitations. The primary limitation is the use of a waitlist instead of an active treatment comparator, which might have led to an overestimate of the treatment effect. Our study also relied primarily on self-report instruments, and is therefore susceptible to the problems common in these measures. Further, the HRSD raters were not blinded to treatment condition, introducing the possibility of bias. Our data structure also did not support a more fine-grained investigation of hypotheses related to change, e.g., whether session-to-session change in mediators predicted improvement in depression (e.g., Strunk, Brotman, DeRubeis, & Hollon, 2010). Also, because the iCBT treatment is non-tailored (i.e., everyone receives the same treatment), we are unable to determine what specific components of the treatment resulted in larger improvements in cognitive skills.

The recruited sample was also relatively young, highly educated, female, and employed. Although they tended to have depression symptoms in the moderate to severe range, our results might not generalize outside of similar groups. A further problem with generalizability is the limited availability of coaching services offered for publically available iCBT programs. Access to iCBT for depression in the United States is largely limited to free services or programs purchased by insurers, employee assistance programs, or health systems. Within these contexts, the availability of coaching is highly variable and the likelihood of receiving manualized coaching is likely low. Efforts to disseminate programs and coaching into practice are needed. Finally, we did not report on longer-term outcomes. Longer term outcomes for this trial will be the subject of a subsequent report.

Future directions include refining efforts to predict or prevent dropout in iCBT, as well as efforts to incorporate measures of skills into iCBT programs themselves as potential markers of improvement. Future studies might also seek to randomize therapy components or conduct dismantling studies to determine which specific modules or strategies within iCBT programs are the most effective at increasing cognitive skills.

Supplementary Material

supplement

Highlights.

  • Guided iCBT for depression was superior to waitlist, with large effect sizes

  • Dropout in iCBT was 29% despite a coaching protocol designed to improve adherence

  • Depression change in iCBT was mediated by improvement in cognitive skills

  • We failed to find evidence that depression change in iCBT was mediated by improvement in behavioral activation

Acknowledgments

We would like to thank the following individuals for their help in conducting the iCBT trial: Lola Gollapudy, Emily Graydon, Kara Neff, Claudia Villena, Olivia Fitzpatrick, and Iony Schmidt. We would also like to thank U2 Interactive, LLC for support and training in implementing Beating the Blues. Finally, we would like to thank Dr. Stephen Schueller and Dr. David Mohr for providing the TeleCoach iCBT coaching protocol and instructing the study team in its use.

Financial support

This work was supported by the National Center for Advancing Translational Sciences Award Number Grant KL2TR001068, awarded to NRF. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclosure Statement

The authors declare that there are no conflicts of interest.

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

1

9 (10%) were recruited from other sources, e.g., through friends or referred from other research study coordinators, and another 5 (6%) could not recall where they heard about the study.

2

At the suggestion of a reviewer, we also conducted this analysis removing two items on the CCTS-SR related to behavioral activation: “When I was upset, I made an effort to engage in enjoyable activities that would be likely to improve my mood,” and “When I have felt down, I engaged in activities that were enjoyable or gave me a sense of accomplishment (to try to help my mood).” Removing these items to eliminate any conceptual overlap between the proposed mediators, the indirect effect for the cognitive skills remained significant and the estimated 95% CI was similar (−1.50, −0.12). BA remained nonsignificant in this analysis.

References

  1. Adler AD, Strunk DR, Fazio RH. What Changes in Cognitive Therapy for Depression? An Examination of Cognitive Therapy Skills and Maladaptive Beliefs. Behavior Therapy. 2015;46(1):96–109. doi: 10.1016/j.beth.2014.09.001. https://doi.org/10.1016/j.beth.2014.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cognitive Behaviour Therapy. 2009;38(4):196–205. doi: 10.1080/16506070903318960. https://doi.org/10.1080/16506070903318960. [DOI] [PubMed] [Google Scholar]
  3. Andersson G, Hesser H, Veilord A, Svedling L, Andersson F, Sleman O, Carlbring P. Randomised controlled non-inferiority trial with 3-year follow-up of internet-delivered versus face-to-face group cognitive behavioural therapy for depression. Journal of Affective Disorders. 2013;151(3):986–994. doi: 10.1016/j.jad.2013.08.022. https://doi.org/10.1016/j.jad.2013.08.022. [DOI] [PubMed] [Google Scholar]
  4. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PloS One. 2010;5(10):e13196. doi: 10.1371/journal.pone.0013196. https://doi.org/10.1371/journal.pone.0013196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Barber JP, DeRubeis RJ. Change in compensatory skills in cognitive therapy for depression. The Journal of Psychotherapy Practice and Research. 2001;10(1):8–13. [PMC free article] [PubMed] [Google Scholar]
  6. Barber Jacques P, DeRubeis RJ. On second thought: Where the action is in cognitive therapy for depression. Cognitive Therapy and Research. 1989;13(5):441–457. https://doi.org/10.1007/BF01173905. [Google Scholar]
  7. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51(6):1173–1182. doi: 10.1037//0022-3514.51.6.1173. https://doi.org/10.1037/0022-3514.51.6.1173. [DOI] [PubMed] [Google Scholar]
  8. Beck AT, Rush AJ, Shaw BT, Emery G. Cognitive Therapy of Depression. Guilford Press; 1979. [Google Scholar]
  9. Cavanagh K, Shapiro DA, Van Den Berg S, Swain S, Barkham M, Proudfoot J. The effectiveness of computerized cognitive behavioural therapy in routine care. The British Journal of Clinical Psychology / the British Psychological Society. 2006;45(Pt 4):499–514. doi: 10.1348/014466505X84782. https://doi.org/10.1348/014466505X84782. [DOI] [PubMed] [Google Scholar]
  10. Christensen H, Griffiths KM, Farrer L. Adherence in Internet Interventions for Anxiety and Depression: Systematic Review. Journal of Medical Internet Research. 2009;11(2):e13. doi: 10.2196/jmir.1194. https://doi.org/10.2196/jmir.1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cooper AA, Conklin LR. Dropout from individual psychotherapy for major depression: A meta-analysis of randomized clinical trials. Clinical Psychology Review. 2015;40:57–65. doi: 10.1016/j.cpr.2015.05.001. https://doi.org/10.1016/j.cpr.2015.05.001. [DOI] [PubMed] [Google Scholar]
  12. Cuijpers P, Karyotaki E, Weitz E, Andersson G, Hollon SD, van Straten A. The effects of psychotherapies for major depression in adults on remission, recovery and improvement: A meta-analysis. Journal of Affective Disorders. 2014;159:118–126. doi: 10.1016/j.jad.2014.02.026. https://doi.org/10.1016/j.jad.2014.02.026. [DOI] [PubMed] [Google Scholar]
  13. Davidson KW, Kupfer DJ, Bigger JT, Califf RM, Carney RM, Coyne JC National Heart, Lung, and Blood Institute Working Group. Assessment and treatment of depression in patients with cardiovascular disease: National Heart, Lung, and Blood Institute Working Group Report. Psychosomatic Medicine. 2006;68(5):645–650. doi: 10.1097/01.psy.0000233233.48738.22. https://doi.org/10.1097/01.psy.0000233233.48738.22. [DOI] [PubMed] [Google Scholar]
  14. De Jong K, Timman R, Hakkaart-Van Roijen L, Vermeulen P, Kooiman K, Passchier J, Busschbach JV. The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research: Journal of the Society for Psychotherapy Research. 2014 doi: 10.1080/10503307.2013.871079. https://doi.org/10.1080/10503307.2013.871079. [DOI] [PubMed]
  15. Delgadillo J, Moreea O, Lutz W. Different people respond differently to therapy: A demonstration using patient profiling and risk stratification. Behaviour Research and Therapy. 2016;79:15–22. doi: 10.1016/j.brat.2016.02.003. https://doi.org/10.1016/j.brat.2016.02.003. [DOI] [PubMed] [Google Scholar]
  16. DeRubeis RJ, Hollon SD, Amsterdam JD, Shelton RC, Young PR, Salomon RM, Gallop R. Cognitive therapy vs medications in the treatment of moderate to severe depression. Archives of General Psychiatry. 2005;62(4):409–416. doi: 10.1001/archpsyc.62.4.409. https://doi.org/10.1001/archpsyc.62.4.409. [DOI] [PubMed] [Google Scholar]
  17. Dimidjian S, Hollon SD, Dobson KS, Schmaling KB, Kohlenberg RJ, Addis ME, Jacobson NS. Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology. 2006;74(4):658–670. doi: 10.1037/0022-006X.74.4.658. https://doi.org/10.1037/0022-006X.74.4.658. [DOI] [PubMed] [Google Scholar]
  18. Duffecy J, Kinsinger S, Ludman E, Mohr DC. Brief telephone support program to enhance patient adherence to Technology Assisted Behavioral Interventions (TABIs): Therapist manual. Unpublished Manuscript 2011 [Google Scholar]
  19. Frank E, Prien RF, Jarrett RB, Keller MB, Kupfer DJ, Lavori PW, Weissman MM. Conceptualization and rationale for consensus definitions of terms in major depressive disorder. Remission, recovery, relapse, and recurrence. Archives of General Psychiatry. 1991;48(9):851–855. doi: 10.1001/archpsyc.1991.01810330075011. [DOI] [PubMed] [Google Scholar]
  20. Gallagher-Thompson D, Gray HL, Dupart T, Jimenez D, Thompson LW. Effectiveness of Cognitive/Behavioral Small Group Intervention for Reduction of Depression and Stress in Non-Hispanic White and Hispanic/Latino Women Dementia Family Caregivers: Outcomes and Mediators of Change. Journal of Rational-Emotive and Cognitive-Behavior Therapy: RET. 2008;26(4):286–303. doi: 10.1007/s10942-008-0087-4. https://doi.org/10.1007/s10942-008-0087-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, White D. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. The BMJ. 2015:351. doi: 10.1136/bmj.h5627. https://doi.org/10.1136/bmj.h5627. [DOI] [PMC free article] [PubMed]
  22. Hamilton M. A rating scale for depression. Journal of Neurology. 1960;23:56–61. doi: 10.1136/jnnp.23.1.56. https://doi.org/10.1136/jnnp.23.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hayes AF. Beyond Baron and Kenny: Statistical Mediation Analysis in the New Millennium. Communication Monographs. 2009;76(4):408–420. https://doi.org/10.1080/03637750903310360. [Google Scholar]
  24. Hayes AF. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. New York, NY: Guilford Press; 2013. Retrieved from http://www.guilford.com/books/Introduction-to-Mediation-Moderation-and-Conditional-Process-Analysis/Andrew-Hayes/9781609182304. [Google Scholar]
  25. Hollon SD, DeRubeis RJ. Mediating the Effects of Cognitive Therapy for Depression. Cognitive Behaviour Therapy. 2009;38(sup1):43–47. doi: 10.1080/16506070902915667. https://doi.org/10.1080/16506070902915667. [DOI] [PubMed] [Google Scholar]
  26. Hundt NE, Mignogna J, Underhill C, Cully JA. The Relationship Between Use of CBT Skills and Depression Treatment Outcome: A Theoretical and Methodological Review of the Literature. Behavior Therapy. 2013;44(1):12–26. doi: 10.1016/j.beth.2012.10.001. https://doi.org/10.1016/j.beth.2012.10.001. [DOI] [PubMed] [Google Scholar]
  27. Hunter MS, Ussher JM, Cariss M, Browne S, Jelley R, Katz M. Medical (fluoxetine) and psychological (cognitive-behavioural therapy) treatment for premenstrual dysphoric disorder: a study of treatment processes. Journal of Psychosomatic Research. 2002;53(3):811–817. doi: 10.1016/s0022-3999(02)00338-0. [DOI] [PubMed] [Google Scholar]
  28. Ilardi SS, Craighead WE. The role of nonspecific factors in cognitive-behavior therapy for depression. Clinical Psychology: Science and Practice. Vol 1(2) 1994;1:138–156. [Google Scholar]
  29. Jacobson NS, Dobson KS, Truax PA, Addis ME, Koerner K, Gollan JK, Prince SE. A component analysis of cognitive-behavioral treatment for depression. Journal of Consulting and Clinical Psychology. 1996;64(2):295–304. doi: 10.1037//0022-006x.64.2.295. [DOI] [PubMed] [Google Scholar]
  30. Jarrett RB, Vittengl JR, Clark LA, Thase ME. Skills of Cognitive Therapy (SoCT): a new measure of patients’ comprehension and use. Psychological Assessment. 2011;23(3):578–586. doi: 10.1037/a0022485. https://doi.org/10.1037/a0022485. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kanter JW, Mulick PS, Busch AM, Berlin KS, Martell CR. The Behavioral Activation for Depression Scale (BADS): Psychometric Properties and Factor Structure. Journal of Psychopathology and Behavioral Assessment. 2007;29(3):191. https://doi.org/10.1007/s10862-006-9038-5. [Google Scholar]
  32. Kazdin AE. Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology. 2007;3:1–27. doi: 10.1146/annurev.clinpsy.3.022806.091432. https://doi.org/10.1146/annurev.clinpsy.3.022806.091432. [DOI] [PubMed] [Google Scholar]
  33. King MW, Resick PA. Data mining in psychological treatment research: A primer on classification and regression trees. Journal of Consulting and Clinical Psychology. 2014;82(5):895–905. doi: 10.1037/a0035886. https://doi.org/10.1037/a0035886. [DOI] [PubMed] [Google Scholar]
  34. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kuyken W, Watkins E, Holden E, White K, Taylor RS, Byford S, Dalgleish T. How does mindfulness-based cognitive therapy work? Behaviour Research and Therapy. 2010;48(11):1105–1112. doi: 10.1016/j.brat.2010.08.003. https://doi.org/10.1016/j.brat.2010.08.003. [DOI] [PubMed] [Google Scholar]
  36. Lattie EG, Schueller SM, Sargent E, Stiles-Shields C, Tomasino KN, Corden ME, Mohr DC. Uptake and Usage of IntelliCare: A Publicly Available Suite of Mental Health and Well-Being Apps. Internet Interventions. 2016;4(2):152–158. doi: 10.1016/j.invent.2016.06.003. https://doi.org/10.1016/j.invent.2016.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lemmens LHJM, Müller VNLS, Arntz A, Huibers MJH. Mechanisms of change in psychotherapy for depression: An empirical update and evaluation of research aimed at identifying psychological mediators. Clinical Psychology Review. 2016;50:95–107. doi: 10.1016/j.cpr.2016.09.004. https://doi.org/10.1016/j.cpr.2016.09.004. [DOI] [PubMed] [Google Scholar]
  38. Liao SG, Lin Y, Kang DD, Chandra D, Bon J, Kaminski N, Tseng GC. Missing value imputation in high-dimensional phenomic data: imputable or not, and how? BMC Bioinformatics. 2014;15:346. doi: 10.1186/s12859-014-0346-6. https://doi.org/10.1186/s12859-014-0346-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Lorenzo-Luaces L, German RE, DeRubeis RJ. It’s complicated: The relation between cognitive change procedures, cognitive change, and symptom change in cognitive therapy for depression. Clinical Psychology Review. 2015;41:3–15. doi: 10.1016/j.cpr.2014.12.003. https://doi.org/10.1016/j.cpr.2014.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Manos RC, Kanter JW, Luo W. The behavioral activation for depression scale-short form: development and validation. Behavior Therapy. 2011;42(4):726–739. doi: 10.1016/j.beth.2011.04.004. https://doi.org/10.1016/j.beth.2011.04.004. [DOI] [PubMed] [Google Scholar]
  41. McMillan D, Gilbody S, Richards D. Defining successful treatment outcome in depression using the PHQ-9: A comparison of methods. Journal of Affective Disorders. 2010;127(1-3):122–129. doi: 10.1016/j.jad.2010.04.030. https://doi.org/10.1016/j.jad.2010.04.030. [DOI] [PubMed] [Google Scholar]
  42. Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. Journal of Medical Internet Research. 2011;13(1):e30. doi: 10.2196/jmir.1602. https://doi.org/10.2196/jmir.1602. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, Begale M. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PloS One. 2013;8(8):e70086. doi: 10.1371/journal.pone.0070086. https://doi.org/10.1371/journal.pone.0070086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K, Schueller SM. IntelliCare: An Eclectic, Skills-Based App Suite for the Treatment of Depression and Anxiety. Journal of Medical Internet Research. 2017;19(1):e10. doi: 10.2196/jmir.6645. https://doi.org/10.2196/jmir.6645. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Newby JM, Williams AD, Andrews G. Reductions in negative repetitive thinking and metacognitive beliefs during transdiagnostic internet cognitive behavioural therapy (iCBT) for mixed anxiety and depression. Behaviour Research and Therapy. 2014;59:52–60. doi: 10.1016/j.brat.2014.05.009. https://doi.org/10.1016/j.brat.2014.05.009. [DOI] [PubMed] [Google Scholar]
  46. Powers DV, Thompson LW, Gallagher-Thompson D. The Benefits of Using Psychotherapy Skills Following Treatment for Depression: An Examination of “Afterwork” and a Test of the Skills Hypothesis in Older Adults. Cognitive and Behavioral Practice. 2008;15(2):194–202. https://doi.org/10.1016/j.cbpra.2007.01.002. [Google Scholar]
  47. Preacher KJ, Hayes AF. Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods. 2008;40(3):879–891. doi: 10.3758/brm.40.3.879. https://doi.org/10.3758/BRM.40.3.879. [DOI] [PubMed] [Google Scholar]
  48. Proudfoot J, Ryden C, Everitt B, Shapiro DA, Goldberg D, Mann A, Gray JA. Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. The British Journal of Psychiatry. 2004;185(1):46–54. doi: 10.1192/bjp.185.1.46. https://doi.org/10.1192/bjp.185.1.46. [DOI] [PubMed] [Google Scholar]
  49. Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. 2. Newbury Park, CA: Sage; 2002. [Google Scholar]
  50. Reimherr FW, Amsterdam JD, Quitkin FM, Rosenbaum JF, Fava M, Zajecka J, Sundell K. Optimal length of continuation therapy in depression: a prospective assessment during long-term fluoxetine treatment. The American Journal of Psychiatry. 1998;155(9):1247–1253. doi: 10.1176/ajp.155.9.1247. [DOI] [PubMed] [Google Scholar]
  51. Rubin DB, Little RJA. Statistical analysis with missing data. 2. New York, NY: Wiley; 2002. [Google Scholar]
  52. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Dunbar GC. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry. 1998;59(Suppl 20):22–33. quiz 34-57. [PubMed] [Google Scholar]
  53. Shipley WC. A self-administering scale for measuring intellectual impairment and deterioration. Journal of Psychology: Interdisciplinary and Applied. 1940;9:371–377. https://doi.org/10.1080/00223980.1940.9917704. [Google Scholar]
  54. Spek V, Nyklícek I, Smits N, Cuijpers P, Riper H, Keyzer J, Pop V. Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychological Medicine. 2007;37(12):1797–1806. doi: 10.1017/S0033291707000542. https://doi.org/10.1017/S0033291707000542. [DOI] [PubMed] [Google Scholar]
  55. Stekhoven DJ, Bühlmann P. MissForest—non-parametric missing value imputation for mixed-type data. Bioinformatics. 2012;28(1):112–118. doi: 10.1093/bioinformatics/btr597. https://doi.org/10.1093/bioinformatics/btr597. [DOI] [PubMed] [Google Scholar]
  56. Strunk DR, Brotman MA, DeRubeis RJ, Hollon SD. Therapist competence in cognitive therapy for depression: predicting subsequent symptom change. Journal of Consulting and Clinical Psychology. 2010;78(3):429–437. doi: 10.1037/a0019631. https://doi.org/10.1037/a0019631. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Strunk DR, DeRubeis RJ, Chiu AW, Alvarez J. Patients’ competence in and performance of cognitive therapy skills: Relation to the reduction of relapse risk following treatment for depression. Journal of Consulting and Clinical Psychology. Vol 75(4) 2007;75:523–530. doi: 10.1037/0022-006X.75.4.523. https://doi.org/10.1037/0022-006X.75.4.523. [DOI] [PubMed] [Google Scholar]
  58. Strunk DR, Hollars SN, Adler AD, Goldstein LA, Braun JD. Assessing Patients’ Cognitive Therapy Skills: Initial Evaluation of the Competencies of Cognitive Therapy Scale. Cognitive Therapy and Research. 2014;38(5):559–569. doi: 10.1007/s10608-014-9617-9. https://doi.org/10.1007/s10608-014-9617-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Tadić A, Helmreich I, Mergl R, Hautzinger M, Kohnen R, Henkel V, Hegerl U. Early improvement is a predictor of treatment outcome in patients with mild major, minor or subsyndromal depression. Journal of Affective Disorders. Vol 120(1–3) 2010;120(2010):86–93. doi: 10.1016/j.jad.2009.04.014. https://doi.org/10.1016/j.jad.2009.04.014. [DOI] [PubMed] [Google Scholar]
  60. Wagner B, Horn AB, Maercker A. Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. Journal of Affective Disorders. 2014;152–154:113–121. doi: 10.1016/j.jad.2013.06.032. https://doi.org/10.1016/j.jad.2013.06.032. [DOI] [PubMed] [Google Scholar]
  61. Waljee AK, Mukherjee A, Singal AG, Zhang Y, Warren J, Balis U, Higgins PD. Comparison of imputation methods for missing laboratory data in medicine. BMJ Open. 2013;3(8) doi: 10.1136/bmjopen-2013-002847. https://doi.org/10.1136/bmjopen-2013-002847. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Williams JB. A structured interview guide for the Hamilton Depression Rating Scale. Archives of General Psychiatry. 1988;45(8):742–747. doi: 10.1001/archpsyc.1988.01800320058007. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
NIHMS919812-supplement.docx (22.1KB, docx)

RESOURCES