Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Sep 1.
Published in final edited form as: Cognit Ther Res. 2018 Nov 16;43(3):620–630. doi: 10.1007/s10608-018-9979-5

Predictors of Dropout in Internet-Based Cognitive Behavioral Therapy for Depression

Iony D Schmidt 1, Nicholas R Forand 2, Daniel R Strunk 1
PMCID: PMC7462416  NIHMSID: NIHMS1512910  PMID: 32879540

Abstract

Internet-based cognitive behavioral therapy (iCBT), provided with guidance, has been shown to outperform wait-list control conditions and appears to perform on par with face-to-face psychotherapy. However, dropout remains an important problem. Dropout rates for iCBT programs for depression have ranged from 0% to 75%, with a mean of 32%. Drawing from a recent study in which 117 people participated in iCBT with support, we examined participant characteristics, participants’ use of iCBT skills, and their experience of technical difficulties with iCBT as predictors of dropout risk. Educational level, extraversion, and participant skill use predicted lower risk of dropout; technical difficulties and openness predicted higher dropout risk. We encourage future research on predictors of dropout in the hope that greater understanding of dropout risk will inform efforts to promote program engagement and retention.

Keywords: internet-based cognitive behavioral therapy, depression, dropout

Predictors of Dropout in Computerized Cognitive Behavioral Therapy for Depression

Internet-based cognitive behavioral treatments (iCBT) are efficacious treatments for depression and anxiety, with a number of studies demonstrating these treatments outperform wait-list control conditions and achieve effects similar to face-to-face CBT (Andersson et al., 2013; Andrews, Cuijpers, Craske, McEnvoy, & Titov, 2010; Cuijpers, Donker, Van Straten, Li, & Andersson, 2010; Spek et al., 2007; Wagner, Horn, & Maercker, 2014; Wright et al., 2005). Researchers studying iCBT have also shown that guided iCBT programs (i.e., those delivered with the support of a coach or helper) are more effective than self-guided programs (Andersson & Cuijpers, 2009; Cuijpers et al., 2010; Gellatly et al., 2007; Richards and Richardson, 2012; Spek et al., 2007). Thus, guided iCBT is a promising treatment approach.

Beating the Blues

Beating the Blues (BtB) is a program consisting of eight modules that involve a series of CBT lessons (Forand et al., 2017; Proudfoot et al., 2004). Topics covered include cognitive restructuring, behavioral activation, problem solving, and sleep hygiene. Though the program is directed primarily towards addressing depressive symptoms, it also includes interventions aimed at anxiety symptoms, such as relaxation training and exposure exercises. Homework is assigned between modules that is similar to assignments used in traditional, face-to-face CBT.

Given the evidence that providing guidance or support increases the therapeutic benefits of iCBT (Andersson & Cuijpers, 2009; Johansson and Andersson, 2012), BtB participants are often assigned coaches to aid them throughout the program. Support can be text-based or occur via phone contacts, with the coach offering reminders, encouragement to complete modules and assignments, and troubleshooting any other problems that might develop as one completes the program. Although there has been variability in the support provided across iCBT studies, protocols have now been developed to detail the procedures for providing support. Based on the supportive accountability model (Mohr, Cuijpers, & Lehman, 2011), the TeleCoach protocol calls for coaches to build reciprocity with the participant, set expectations for participant behaviors and progress, develop expectations for participant accountability, and reward participant adherence (Duffecy, Kinsinger, Ludman, & Mohr, 2011). Findings from a recent clinical trial showed that use of this protocol with iCBT was found to improve adherence (though not outcomes) when compared to iCBT alone (Mohr et al., 2013).

Therapeutic Outcomes with BtB

In a randomized trial of 274 participants with anxiety or depression in the United Kingdom, BtB was found to be superior to treatment as usual in reducing symptoms of depression (Proudfoot et al., 2003; Proudfoot et al., 2004). In the more recent randomized trial from which data for this project were drawn, we compared iCBT and wait-list (Forand et al., 2017). Among those randomized to BtB or wait-list, the BtB condition outperformed the wait-list condition in terms of change in depressive symptoms (Hedge’s g = 1.67) and post-treatment depressive symptoms (Hedge’s g = 1.45; Forand et al.). Additional analyses showed that the acquisition of cognitive skills mediated the advantage of BtB over wait-list on change in improvements in depressive symptoms.

Dropout in BtB.

One of the greatest concerns regarding iCBT is the relatively high dropout rate associated with these programs. The literature on iCBT for depression has shown the dropout rates are considerable, with a review of the available literature that aggregated 16 studies reporting a mean dropout rate of 32% (Kaltenthaler et al., 2008). Using a more liberal definition of dropout, Richards and Richardson (2012) conducted a meta-analysis of 7,313 participants across 40 studies and found an overall dropout rate of 57%. When examining the dropout rates by the different levels of support, the dropout rate was 74% for those provided treatment without support, 38% for those provided only administrative support, and 28% for those provided therapeutic support.

These dropout rates draw attention to the need for understanding why such a substantial minority of participants do not complete these programs. One approach to understanding dropout is to simply ask participants who dropout to state their reasons for doing so (Richards & Richardson, 2012). Proudfoot and colleagues (2003) and Waller and Gilbody (2009) reported that participants who dropped out cited unhappiness with treatment allocation, illness, lack of time, work commitments, family problems, moving, and treatment progress as reasons for their dropout. Warmerdam, van Straten, Twisk, Riper, and Cuijpers (2008) also noted that patients had reported other reasons, including participants seeking other treatments, feeling better before the end of treatment, scheduling difficulties, and having problems with understanding the program. However, participants who drop out may not always be available or willing to report their reason for dropping out. In addition, it is unclear whether participants can correctly identify the factors that cause them to drop out.

Predictors of dropout in BtB.

While a number of studies have examined the rate of dropout in iCBT, less research has examined potential predictors of dropout. In an effort to summarize the work that has been conducted on predictors of dropout, Karyotaki and colleagues’ (2015) conducted a meta-analysis of predictors across 10 trials involving 2,705 participants in self-guided iCBT. They found that male gender (relative risk [RR] 1.08), lower educational level (RR 1.26), age (RR 0.94; indicating high risk for younger participants), and comorbid anxiety symptoms (based on elevated anxiety symptom ratings at baseline; RR 1.18) significantly predicted the risk of dropout before adequate dose (i.e., completing 75% of treatment modules). However, these findings were obtained from studies of self-guided treatments. Predictors of dropout in samples participating in guided iCBT may differ. In this study, we examine predictors of dropout and extend the variables studied to include personality characteristics as well as process variables such as the amount of support provided and the use of skills during program participation.

Personality characteristics.

There has been interest in better understanding how normal personality factors might impact the course of treatment. There is some evidence that higher levels of self-reported neuroticism predict less symptom improvement in combined medication and psychotherapy (Klein, Kotov, & Bufferd, 2011; Quilty et al., 2008). Using thin-slice observer ratings of personality characteristics, Sasso and Strunk (2013) failed to find that neuroticism predicted symptom change among patients in CBT for depression but found that lower levels of neuroticism were related to dropping out of treatment. Perhaps clients higher in neuroticism are more strongly motivated to complete the full length of treatment because of their negative affect, even if they may benefit less from treatment. Despite the importance of identifying predictors of dropout, the Sasso and Strunk study is one of very few that have examined personality traits as potential predictors of dropout.

Process variables.

Another way that participants might determine whether to continue with the program would be to assess whether they are utilizing the information and skills they have acquired through the program in their lives. Acquisition of the skills taught in CBT has been found to be related to overall symptom improvement as well as risk of relapse following face-to-face CBT (Jarrett, Vittengl, Clark, & Thase, 2011; Strunk, DeRubeis, Chui, & Alvarez, 2007). Insofar as a participant is not learning the skills of the treatment, one might expect that they may find the program less useful and be at increased risk of dropout. In our recent trial, we found that acquisition of cognitive skills mediated the superiority of iCBT over the wait-list condition (Forand et al., 2017). We suspect that patients might use skill acquisition as a way to gauge the potential benefits of treatment; if they have not learned new skills, they may be more pessimistic about experiencing benefits from treatment if they continue. To our knowledge, no studies have yet examined the relationship between acquisition of CBT skills and treatment dropout.

With its computerized delivery, clients participating in iCBT may experience technical problems, ranging from a link in a module not functioning properly to a glitch in the program that denies access to a module or resets one’s progress. Clearly, technical issues can have an effect on a participant’s access to treatment and may also influence a participant’s risk of terminating treatment early. Perhaps because it helps participants to overcome such obstacles, the provision of support increases the therapeutic benefits of iCBT (Andersson & Cuijpers, 2009; Johansson and Andersson, 2012).

Purpose

In an effort to understand risk for dropout in iCBT, we planned to examine several potential predictors of dropout drawing data from the project described by Forand et al. (2017). First, we expected to replicate the findings of Karyotaki and colleagues’ (2015) meta-analysis, that male gender, younger age, lower educational level, and comorbid anxiety symptoms would increase likelihood of dropping out. Second, we planned to examine how personality characteristics might predict likelihood of dropout; we specifically expected lower levels of neuroticism to predict higher risk of dropout. Third, we expected to find that evidence of greater participant skill use and fewer encountered technical difficulties would predict lower risk of dropout.

Methods

Participants

We drew data from both the immediate and delayed treatment samples reported by Forand and colleagues’ (2017) and pooled those data with those from a group of participants who participated in iCBT in a subsequent non-randomized phase of the study. Thus, the current sample is composed of three study segments: (1) participants randomized to BtB in phase 1 of the study (n = 59); (2) participants who completed BtB after being randomized to wait-list in phase 1 (n = 20); and (3) participants offered BtB as part of an open trial constituting phase 2 (n = 38). A total of 117 people participated in guided BtB in this study. Participants were on average 33 years old (SD = 12 range: 18 – 67), women (74%; n = 87), and employed (81%; n = 95). The sample was 72% Caucasian (n = 84), with the next largest group being African-American at 17% (n = 20). Seven participants were Asian (6%), three were American Indian or Native Alaskan (3%), one was Native Hawaiian or other Pacific Islander (1%), one indicated “other” (Egyptian; 1%), and one participant preferred not to answer (1%). At intake, 109 (93.2%) participants met criteria for current major depressive disorder (MDD). The average Patient Health Questionnaire Depression Module (PHQ-9) score at intake was 16.8 (SD = 4.1).

Inclusion criteria for the trial were: (1) the presence of significant mood or anhedonic symptoms of depression, as indicated by a score of greater than or equal to two on the first two questions of the PHQ-9 (Kroenke, Spitzer, & Williams, 2001) and a score of eight or greater on the rest of the PHQ-9 (Davidson et al., 2006; Thombs & Ziegelstein, 2014), (2) 18 years of age or older; (3) able and willing to provide informed consent; (4) access to a computer with an internet connection. Exclusion criteria were: (1) history of bipolar disorder or psychosis; (2) current primary Diagnostic and Statistical Manual Fourth Edition (DSM-IV) Axis I disorder other than MDD; (3) substance dependence within the past six months; (4) subnormal intellectual potential (Intelligence Quotient below 80) as measured by the Shipley Scale for Living (Shipley, 1940); (5) clear indication of secondary gain; (6) current imminent suicide risk or significant intentional self-harm in the last six months; (7) discharge within six months from a higher level of care; (8) current outpatient psychotherapy; (9) a change in antidepressant medication over the past month or a planned change over the duration of the study; (10) inability to read and write English (for additional details, see Forand et al., 2017).

iCBT

iCBT was provided using an Americanized version of BtB (Beating the Blues US v2.5) licensed by U2 Interactive, LLC. Participants were encouraged to complete one module per week for eight continuous weeks. Each module included homework assignments to encourage practicing CBT skills in daily life.

Each participant was also assigned an iCBT coach. In segments 1 and 2, two coaches with Ph.D.s provided support; in segment 3, supervised undergraduate-level coaches provided support. Coaching was provided in line with the TeleCoach protocol (Duffecy et al., 2011) adapted for BtB. Over the first four weeks, the protocol called for coaches to make three phone calls and initiate one email exchange. Plans for additional email or phone contact for the remainder of the program was decided collaboratively by the coach and participant. All interactions with coaches and study personnel were audio recorded and electronically logged. The electronic record included notes regarding clients’ progress through the program, any obstacles to progressing through the program including any technical difficulties with the program, the clients’ engagement with homework assignments, and the clients’ use of CBT skills in their daily lives.

Measures

Diagnoses.

Diagnoses were obtained using the Mini-International Neuropsychiatric Diagnostic Interview 6.0.0 (MINI). The MINI (Sheehan et al., 1998) is a brief structured diagnostic interview covering 19 disorders that are commonly seen in psychiatric practice. Administration of this measure consists of a series of “yes or no” questions and takes approximately 15 minutes to complete. Diagnoses correspond to psychiatric disorders in the DSM-IV and International Statistical Classification of Diseases and Related Health Problems. The MINI has demonstrated good concurrent validity (Sheehan et al., 1998).

Depressive symptoms.

Depressive symptoms were measured using the PHQ-9 (Kroenke, Spitzer, & Williams, 2001). The PHQ-9 is a nine item self-report measure that assesses depressive symptoms considered in making a DSM-IV diagnosis of MDD. The measure assesses symptom severity over the past two weeks (e.g., “little interest or pleasure in doing things, “feeling down, depressed or hopeless”). Items are measured on a zero- to three-point scale, with higher scores reflecting being bothered by specific symptoms more frequently. PHQ-9 scores 10 and above suggest probable MDD diagnosis. The PHQ-9 has demonstrated good concurrent validity (Kroenke, Spitzer, & Williams, 2001).

Personality characteristics.

Personality characteristics were measured using the Big Five Inventory (BFI; John, Donahue, & Kentle, 1991), a 44-item inventory that assesses five dimensions of personality (i.e., agreeableness, conscientiousness, extraversion, neuroticism, and openness; Goldberg, 1993). Participants are asked to indicate how much each characteristic (e.g., “is talkative”) is self-descriptive using a five-point scale ranging from “disagree strongly” to “agree strongly.” These scales have shown satisfactory reliability and validity (Benet-Martinez & John, 1998; John & Srivastava, 1999).

CBT skills use and technical difficulties.

Informed by a review of each clients’ program records (i.e., notes from coaches or other study personnel), we obtained ratings of both clients’ use of CBT skills and clients’ experience of technical difficulties encountered with the program. The review of these records was completed by two independent raters, who had no contact with participants over the course of the study and were blind to clinical outcomes. Each variable was rated on a zero- to six-point Likert scale, with higher scores reflecting encountering more substantial technical difficulties or greater use of CBT skills between iCBT modules. Raters reviewed each participants full study record and rated the items. The CBT skill use item asked the rater to “rate the extent to which the participant showed evidence of CBT skill use.” The technical difficulties item asked the rater to “rate the extent to which the records showed evidence that the participant experienced technical difficulties.” Additional item information including detailed item anchors are available from the corresponding author upon request. Random effects intraclass correlation coefficients (ICCs; McGraw & Wong, 1996) for these ratings (corrected for two raters) were:.84 for technical difficulties and.72 for participant CBT skill use.

Dropout

For the primary analyses, we defined dropout as completion of less than 75% of the treatment modules (i.e., completing fewer than six modules). Overall, 53 participants (45.3%) dropped out of treatment.1

In secondary analyses, we analyzed two additional definitions of dropout: (1) those who did not complete more than half of the eight treatment modules (dropout n = 38) and (2) those who both did not experience treatment response (PHQ-9 score < 10) and completed fewer than 75% of the modules (dropout n = 19).

Analytic Strategy

For our primary analyses, we used procedures similar to those described by Karyotaki and colleagues (2015). We first conducted a series of bivariate analyses to assess the relative risks (RR) of each of the 11 predictors. Then, we ran a Poisson model of the complete list of 11 predictor variables of interest: age, sex, level of education (based on the levels described in Table 1), presence of comorbid anxiety disorder(s), personality traits, the rating of CBT skills use, and the rating of technical difficulties encountered. Finally, we examined a parsimonious model in which we retained the predictors that were significant in the complete model. We also examined the role of three covariates in the complete model: study segment (i.e., phase 1 immediate, phase 1 delayed, and phase 2 immediate), PHQ-9 score at baseline of the active treatment, and number of contacts between the participant and coach (phone call or email)2. Models were implemented in PROC GENMOD in SAS with the REPEATED statement (to obtain a standard error for each RR). The resulting confidence intervals are somewhat more conservative than those that would be obtained surrounding odds ratios obtained via logistic regression. This method has been supported as efficient in synthesizing and estimating the effect of included predictors (Zou, 2004).For ease of interpretation, all predictors were standardized (M = 0, SD = 1).

Table 1.

Demographic and clinical characteristics of sample

Characteristics
Age, M (SD) 33.47 (12.2)
Gender, females, n (%) 87 (74.4)
Education, n (%)
    Less than or = high school diploma 13 (11.1)
    Some college 36 (30.8)
    Graduated two-year college 12 (10.3)
    Graduated four-year college 37 (31.6)
    More than four-year college 19 (16.2)
Employed, n (%) 95 (81.2)
PHQ-9 at intake, M (SD) 16.76 (4.1)
Co-morbid anxiety, n (%) 30 (25.9)
Dropout, n (%) 53 (45.3)
Open in a new tab

Note. PHQ-9, Patient Health Questionnaire; M, mean; SD, standard deviation; n, number of participants.

We acknowledge there is not widespread agreement on a specific definition of dropout. With this in mind, we conducted a secondary set of analyses in which we considered alternative definitions of dropout. Those alternatives were: (1) those who did not complete more than half of the eight treatment modules and (2) those who completed fewer than 75% of the modules and did not experience treatment response (PHQ-9 score < 10). Following the same analytic strategy to examine a complete model in our primary analyses, we ran a Poisson model containing all 11 potential predictor variables for each of these alternative dropout definitions.

Results

Descriptive Statistics

We report Ms and SDs of the predictor variables in Table 1. Regarding personality characteristics, mean BFI scores were 32.6 (SD = 6.1) for agreeableness, 29.4 (SD = 6.1) for conscientiousness, 20.9 (SD = 6.7) for extraversion, 30.5 (SD = 5.9) for neuroticism, and 36.6 (SD = 6.9) for openness. We then converted the sample means to percentage of maximum possible (POMP) scores, a linear transformation of a raw metric into a 0 to 100 scale, where 0 represents the minimum possible score and 100 represents the maximum possible score (Cohen, Cohen, Aiken, & West, 1999). Comparing our POMP values to the scores obtained by Srivastava, John, Gosling, and Potter’s (2003) in a large, normative sample (N = 132,515) of North American adults, our sample scored significantly higher across all personality characteristics (z range: 2.33 – 10.20; all ps <.01), except for openness, which did not differ from that reported by Srivastava and colleagues (z = 1.02, p =.31).

Regarding the reviewed process variables, about half of the sample (49%; n = 57) did not report any technical difficulties, with 29 participants (25%) experiencing minor technical difficulties (e.g., personal computer issues, not remembering their password), 21 participants (18%) experiencing moderate technical difficulties (e.g., BtB program glitches), and seven participants (6%) experiencing severe technical difficulties (e.g., BtB program becoming inaccessible). Regarding participant skill use, 10 participants (9%) were rated as showing no evidence of skill usage outside of the BtB program, 81 (69%) showed evidence of minimal to moderate progress at skill usage outside of the BtB program, and 26 participants (22%) showed evidence of mastery of skill use outside of the BtB program.

Correlations among Variables

Prior to conducting our main analyses, we also examined correlations among the potential predictors of dropout. As reported in Table 2, age was significantly correlated with education, agreeableness, and conscientiousness. Comorbid anxiety was related to being female. Neuroticism was significantly correlated with agreeableness, conscientiousness, and PHQ-9 at intake. Openness was significantly related to agreeableness. Extraversion was related to neuroticism. PHQ-9 at intake was also correlated with conscientiousness and technical difficulties encountered through treatment.

Table 2.

Correlation coefficients among predictor and baseline participant variables

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.
1. Age --
2. Sex −.01 --
3. Education .23* .05 --
4. Comorbid anxiety −.03 .21* .11 --
5. Agreeableness .21* −.03 .11 −.20 --
6. Conscientiousness .21* .01 .18 −.16 .09 --
7. Extraversion .05 .08 −.11 −.02 .03 .27* --
8. Neuroticism −.07 .35 −.05 .24 −.23* −.24* −.07 --
9. Openness .07 −.16 .15 −.11 .11 .10 .22* −.09 --
10. Skill use −.18 .14 −.07 .05 −.04 −.01 −.04 .17 −.11 --
11. Technical difficulties .04 −.04 .06 −.13 .03 −.09 −.06 .05 .06 .08 --
12. PHQ-9 at intake −.05 −.07 −.09 .20 −.07 −.30* −.12 .20* −.07 −.07 .19*
Open in a new tab

Note. Spearman correlations were calculated between continuous variables, point biserial correlations were calculated between binary and continuous variables, and phi coefficients were calculated between binary variables. Items 5 – 9 were measured with the Big Five Inventory. PHQ-9: Patient Health Questionnaire Depression Module.

*

p <.05.

Primary Analyses

We examined the 11 variables of interest as predictors of dropout using modified Poisson regression to calculate RRs for each variable. For every one standard deviation increase in a predictor variable, the RR describes the change in the probability of risk for dropout. As shown in Table 3, results of the bivariate analyses indicated that male gender (RR 1.62, 95% CI 1.11 – 2.37) predicted higher risk of dropout whereas higher levels of education (RR 0.81, 95% CI 0.66 – 0.99) and participant skill use (RR 0.52, 95% CI 0.44 – 0.62) predicted lower risk of dropout.

Table 3.

Model of dropout risk

Bivariate Model
 Complete Model
 Parsimonious Model
Predictors RR 95% CI p RR 95% CI p RR 95% CI P
Age 1.00 0.98–1.02 .87 1.00 0.98–1.01 .69 -- -- --
Gender (male) 1.62 1.11–2.37 .01 0.99 0.64–1.52 .95 -- -- --
Education 0.81 0.66–0.99 .04 0.75 0.62–0.91 <.01 0.77 0.65–0.91 <.01
Comorbid anxiety 1.05 0.65–1.68 .85 0.91 0.61–1.36 .64 -- -- --
Agreeableness 1.10 0.89–1.35 .38 1.15 0.96–1.37 .13 -- -- --
Conscientiousness 0.89 0.73–1.08 .23 1.01 0.84–1.20 .95 -- -- --
Extraversion 0.81 0.66–1.01 .06 0.70 0.58–0.85 <.001 0.71 0.59–0.86 <.001
Neuroticism 0.88 0.73–1.05 .16 0.96 0.82–1.12 .63 -- -- --
Openness 1.19 0.97–1.46 .09 1.27 1.08–1.48 <.01 1.29 1.11–1.50 <.001
Technical difficulties 1.04 0.86–1.26 0.69 1.33 1.13–1.58 <.001 1.31 1.11–1.55 <.01
Participant skill use 0.52 0.44–0.62 <.001 0.46 0.38–0.56 <.001 0.48 0.39–0.58 <.001
Open in a new tab

In the complete model, five variables emerged as significant predictors of dropout risk. Higher levels of education (RR 0.75, 95% CI 0.62 – 0.91), extraversion (RR 0.70, 95% CI 0.58 – 0.85), and participant skill use (RR 0.46, 95% CI 0.38 – 0.56) predicted lower risk of dropout. Higher levels of openness (RR 1.27, 95% CI 1.08 – 1.48) and technical difficulties (RR 1.33, 95% CI 1.13 – 1.58) predicted greater risk of dropout. As an example of how to interpret the RR in this context, a one standard deviation increase in extraversion was associated with 0.70 times the risk of dropping out. Each of the other six variables (i.e., sex, age, co-morbid anxiety symptoms, agreeableness, conscientiousness, and neuroticism) failed to predict likelihood of dropout. As shown in Table 4, results of a complete model that also included three covariates did not differ in direction or statistical significance of any of the 11 predictors of interest. For simplicity, we did not include these covariates in the parsimonious model.

Table 4.

Model of dropout risk with covariates

Predictors RR 95% CI p
Age 1.00 0.98–1.01 .55
Gender (male) 0.99 0.62–1.58 .97
Education 0.76 0.62–0.93 <.01
Comorbid anxiety 0.89 0.58–1.37 .60
Agreeableness 1.13 0.93–1.37 .23
Conscientiousness 1.03 0.85–1.25 .75
Extraversion 0.70 0.55–0.88 <.01
Neuroticism 0.95 0.81–1.12 .54
Openness 1.30 1.10–1.54 <.01
Technical difficulties 1.35 1.09–1.68 <.01
Participant skill use 0.45 0.35–0.59 <.001
Study segment 1.05 0.59–1.89 .86
Total contacts 1.01 0.95–1.09 .70
PHQ-9 at intake 1.01 0.96–1.05 .81
Open in a new tab

Note. RR = risk ratio; CI = confidence interval.

In the parsimonious model, education (RR 0.77, 95% CI 0.65 –0.91), extraversion (RR 0.71, 95% CI 0.59 –0.86), participant skill use (RR 0.48, 95% CI 0.39 –0.58), openness (RR 1.29, 95% CI 1.11 –1.50), and technical difficulties (RR 1.31, 95% CI 1.11 –1.55) remained statistically significant predictors of dropout from treatment. Given their primacy in our analytic strategy, we focus primarily on findings from the complete and parsimonious models.

Secondary Analyses

We then examined complete models including the same 11 variables of interest as predictors of two different definitions of dropout: (1) those who did not complete more than half of the eight treatment modules and (2) those who both did not experience treatment response (PHQ-9 score < 10) and completed fewer than 75% of the modules. As reported in Table 5: Model 1, higher levels of education (RR 0.77, 95% CI 0.59 – 0.99), extraversion (RR 0.74, 95% CI 0.59 – 0.94), and participant skill use (RR 0.41, 95% CI 0.31 – 0.55) significantly predicted lower risk of dropout using the first new dropout definition. As shown in Table 5: Model 2, when we used the second dropout definition, only higher levels of extraversion (RR 0.50, 95% CI 0.29 – 0.86) and participant skill use (RR 0.60, 95% CI 0.43 – 0.83) significantly predicted lower risk of dropout.

Table 5.

Predictors of dropout using two alternative definitions of dropout.

Model 1 Model 2
Predictors RR 95% CI p RR 95% CI p
Age 1.01 0.99–1.03 .52 1.00 0.97–1.04 .85
Gender (male) 1.41 0.84–2.36 .20 1.39 0.58–3.34 .46
Education 0.77 0.59–0.99 .045 0.96 0.64–1.42 .83
Comorbid anxiety 0.80 0.48–1.34 .40 1.75 0.53–5.81 .36
Agreeableness 0.95 0.73–1.23 .69 1.08 0.67–1.73 .76
Conscientiousness 0.93 0.71–1.21 .59 1.04 0.70–1.55 .85
Extraversion 0.74 0.59–0.94 .01 0.50 0.29–0.86 .01
Neuroticism 1.07 0.85–1.34 .57 1.20 0.81–1.78 .36
Openness 1.07 0.95–1.48 .13 1.04 0.71–1.52 .84
Technical difficulties 1.23 0.957–1.58 .09 1.29 0.90–1.85 .17
Participant skill use 0.41 0.31–0.55 <.001 0.60 0.43–0.83 <.01
Open in a new tab

Note. In model 1, dropout is defined as those who did not complete more than half of the treatment modules (dropout n = 38). In model 2, dropout is defined as those who both did not experience treatment response and completed fewer than 75% of the modules (dropout n = 19). N = 117.

Discussion

To our knowledge, this study is the first to examine predictors of dropout in guided iCBT. Consistent with expectations, lower educational level predicted greater risk of dropout. This is consistent with Karyotaki and colleagues’ (2015) meta-analytic finding that lower levels of education predict dropout in unguided CBT, as well as Jarrett and colleagues’ (2013) finding that lower levels of education predict dropout in face-to-face CBT. This might be understood in terms of low educational status potentially identifying those with greater difficulties in understanding the program material and iCBT technology (Waller & Gilbody, 2009).

Although we did not have specific hypotheses about openness and extraversion, we found that lower levels of extraversion and higher levels of openness predicted increased risk of dropout. It is important to note that dropout may not always reflect a poor symptom outcome. However, due to the limited amount of research on dropout, we look to the broader outcome research to help put our findings in context. For example, our finding that low extraversion predicts increased risk of dropout is broadly consistent with findings that higher levels of extraversion are associated with more positive treatment outcomes, such as greater treatment response (Quilty et al., 2008). Although we found higher openness predicted greater dropout risk and Bagby and colleagues (2008) found openness to experience to be associated with greater symptom change over the course of combined mediation and face-to-face psychotherapy, perhaps more open participants in our study were more willing to try treatment, even though they were less committed to seeing the program through than those lower in openness. Additionally, and contrary to expectation, neuroticism did not predict risk of dropout. Our expectation that neuroticism would predict dropout was based largely on a previous study in which Sasso and Strunk (2013) found that observer ratings of neuroticism in short video segments from initial evaluations predicted dropout. It is unclear to what extent such ratings correspond to self-reported neuroticism.

Although we examined a variety of predictors, predictors that are modifiable such as skill use and technical difficulties encountered are of particular interest. These two factors were especially robust predictors of dropout. Research has shown that acquisition of therapy skills is important in face-to-face CBT (Hundt, Mignogna, Underhill, & Cully, 2013), though acquisition of these skills has not been tied to dropout specifically. Whereas therapists have tremendous latitude in how to deliver face-to-face CBT, iCBT is more standardized, with coaches having only brief contacts. Our results are consistent with the idea that developing and practicing CBT skills is beneficial in retaining or engaging people in treatment. Our finding that greater technical difficulties predicted increased risk of dropout is also consistent with Waller and Gilbody’s (2009) suggestions that poor provision of information technology can be a barrier to iCBT. Technical difficulties are a preventable problem that may lead to frustration and ultimately disengagement from the program. Minimizing such difficulties is an obvious, if not always easy way to facilitate a positive user experience. In their review of eight computerized programs for depression and anxiety, Knowles and colleagues (2014) concluded that user experience needs to be a priority for iCBT programs; this includes a focus on personalization, as well as the participant’s sense of connection and collaboration in the program. These aspects of user experience may be strengthened by both coaching and technological improvements.

It is also important to consider the degree to which our findings may be impacted by our definition of dropout and the type of iCBT program studied. As we describe in the secondary set of analyses, our findings are similar, though not identical when we used two more restrictive definitions of dropout. Across all complete models, extraversion and participant skill use emerged as predictors. As the definition of dropout is more restrictive, fewer participants are considered dropouts and the power to detect effects is reduced. Future research systematically examining alternative definitions of dropout in larger datasets is needed. Future studies may also strive to use ecological momentary assessment in addition to more powerful data analysis techniques to better understand dropout in iCBT programs, such as complex network analysis (see Lutz et al., 2018). Moreover, there have been recent moves to change the format of iCBT programs. For example, Mohr et al. (2017) introduced and examined a mobile app suite program named IntelliCare that emphasizes the use of learning individual skills that could be completed in a few minutes at a time, as compared to programs such as BtB, which suggest participants complete the program in larger time blocks. In future research, such program differences will need to be considered in conceptualizing and operationalizing dropout.

Implications

In face-to-face CBT, a number of demographic, personality, and clinical variables have been found to predict dropout (Sasso & Strunk, 2013; Swift & Greenberg, 2012). Consequently, some research efforts have begun to focus on using this information in personalizing or selecting alternative treatments (e.g., DeRubeis et al., 2014; Zilcha-Mano et al., 2016) that may be best for individuals exhibiting particular characteristics that place them at high risk of dropout. Zilcha-Mano and colleagues (2016) developed a multivariable model for predicting dropout using a number of pre-treatment characteristics across supportive-expressive therapy, antidepressant medication, and placebo. Research might investigate the value of providing coaches with additional tools to help them monitor factors such as technical difficulties and CBT skills use, which portend greater risk of dropout as treatment progresses. Technical problems predicted dropout even in the context of coaches being there to help; we suspect the relation of technical problems and dropout would be even stronger in unguided iCBT programs. Developers of both unguided and guided iCBT programs have special reason to ascertain that all technical issues of a program are resolved to the extent possible before providing it to participants. In addition, study coaches and personnel might be provided specific guidance on troubleshooting skills prior to beginning the programs so they can provide ongoing support and help ensure a smooth user experience.

Limitations

We acknowledge a few key limitations of this study. First, the sample over-represented highly educated and employed women. Whether the predictors we identified will generalize to other populations needs to be addressed. On a similar note, those who participated in BtB following a wait-list were drawn from a larger group, raising the possibility that those who participated may not have been representative of the full group. Second, our analyses may have been underpowered. It has been suggested that a minimum of 10 events (or 10 dropouts in this case) per parameter should be present in models to ensure variance around parameter estimates are not too large and to avoid bias in confidence intervals related to the number of events being too low. However, other researchers have suggested that this rule of 10 may be too conservative and using five to nine events per parameter is still acceptable (see Hosmer, Lemeshow, & Sturdivant, 2013). In our sample, the number of dropouts was 53 with nearly five events (4.8) for the each of the 11 predictors we examined. Regardless, this issue may be addressed by using a smaller number of predictors or a larger sample size. Third, study eligibility criteria precluded us from testing characteristics only present among those excluded from the study as predictors of dropout (e.g., being recently discharged from a higher level of care). Fourth, results from our secondary analyses should be considered with caution. As we noted earlier, analyses in which the definition dropout is more restrictive result in fewer dropouts being identified and thereby lower our power to detect effects. Finally, the measures of technical difficulties and skill use were based on ratings following chart-review, an approach developed for this study. Future studies should look to further examine the psychometric properties of such ratings.

Conclusions

Our findings support the notion that certain participant characteristics and experiences (i.e., extraversion, openness, participant skill acquisition, technical difficulties, and education level) are predictive of dropout in guided iCBT. Again, this was the first attempt we know of that has examined predictors of dropout in a guided BtB program. If the results are replicated in additional guided iCBT samples, the findings might be used to inform efforts to improve guided iCBT program retention. Taken together, this approach to assessing predictors of dropout may help those providing iCBT to better identify participants that are more likely to dropout and might ultimately inform an approach of providing such participants with additional resources or alternative treatments.

Acknowledgments

Funding: This study was funded in part by the National Center for Advancing Translational Sciences Award Number Grant KL2TR001068, awarded to NRF. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.

Footnotes

Conflict of Interest: All authors declare that they have no conflict of interest.

Compliance with Ethical Standards

Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent: Informed consent was obtained from all individual participants included in the study.

Data availability: Pending IRB approval for sharing, the dataset described in this paper is available from the corresponding author on reasonable request.

1 Dropout rates across study segments were: 39% (23 of 59) among those randomized to Btb, 65% (13 of 20) among those randomized to wait-list and subsequently offered Btb, and 45% (17 of 38) among non-randomized participants offered Btb. The 20 participants who had initially been assigned to wait-list were a subset a larger group of 30, as 10 opted to not continue participation. An initial test suggested that study segment was associated with differential dropout risk (RR = 1.67; CI: 1.06–2.62, p =.03). However, study segment failed to predict dropout in our multivariate model (see Table 4).

2 We included the number of contacts as a covariate out of concern for the possibility that our measure of CBT skills could have been confounded with clients who had more contact with their coach and thus more opportunities to convey their use of CBT skills. There was an average of 2.6 (SD = 1.54; range = 0 –9) calls completed between participants and their coaches and an average of 5.05 (SD = 2.72; range = 0 –12) emails exchanged between participants and their coaches.

References

  1. Andersson G, & Cuijpers P (2009). Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cognitive Behaviour Therapy, 38, 196–205. doi: 10.1080/16506070903318960 [DOI] [PubMed] [Google Scholar]
  2. Andersson G, Hesser H, Veilord A, Svedling L, Andersson F, Sleman O, … Carlbring P (2013). Randomised controlled non-inferiority trial with 3-year follow-up of internet-delivered versus face-to-face group cognitive behavioural therapy for depression. Journal of Affective Disorders, 151, 986–994. doi: 10.1016/j.jad.2013.08.022 [DOI] [PubMed] [Google Scholar]
  3. Andrews G, Cuijpers P, Craske MG, McEvoy P, & Titov N (2010). Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PloS One, 5, e13196. doi: 10.1371/journal.pone.0013196 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bagby RM, Quilty LC, Segal ZV, McBride CC, Kennedy SH, & Costa PT (2008). Personality and Differential Treatment Response in Major Depression: A Randomized Controlled Trial Comparing Cognitive-Behavioural Therapy and Pharmacotherapy. Canadian Journal of Psychiatry, 53, 361–370. doi: 10.1177/070674370805300605 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Benet-Martinez V, & John OP (1998). Los Cinco Grandes across cultures and ethnic groups: Multitrait-multimethod analyses of the Big Five in Spanish and English. Journal of Personality and Social Psychology, 75, 729–750. doi: 10.1037/0022-3514.75.3.729 [DOI] [PubMed] [Google Scholar]
  6. Cohen P, Cohen J, Aiken LS, & West SG (1999). The problem of units and the circumstance for POMP. Multivariate Behavioral Research, 34, 315–346. doi: 10.1207/S15327906MBR3403_2 [DOI] [Google Scholar]
  7. Cuijpers P, Donker T, van Straten A, Li J, & Andersson G (2010). Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychological Medicine, 40, 1943–1957. doi: 10.1017/S0033291710000772 [DOI] [PubMed] [Google Scholar]
  8. Davidson KW, Kupfer DJ, Bigger JT, Califf RM, Carney RM, Coyne JC, … National Heart, Lung, and Blood Institute Working Group. (2006). Assessment and treatment of depression in patients with cardiovascular disease: National Heart, Lung, and Blood Institute Working Group Report. Psychosomatic Medicine, 68, 645–650. doi: 10.1097/01.psy.0000233233.48738.22 [DOI] [PubMed] [Google Scholar]
  9. DeRubeis RJ, Cohen ZD, Forand NR, Fournier JC, Gelfand LA, & Lorenzo-Luaces L (2014). The Personalized Advantage Index: translating research on prediction into individualized treatment recommendations. A demonstration. PloS one, 9, e83875. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Duffecy J, Kinsinger S, Ludman E, & Mohr DC (2011). Brief telephone support program to enhance patient adherence to Technology Assisted Behavioral Interventions (TABIs): Therapist manual. Unpublished Manuscript. [Google Scholar]
  11. Forand NR, Barnett JG, Strunk DR, Hindiyeh MU, Feinberg JE, & Keefe JR (2017). Efficacy of Guided iCBT for Depression and Mediation of Change by CBT Skill Acquisition. Behavior Therapy, 42, 612–623. doi: 10.1016/j.beth.2017.04.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gellatly J, Bower P, Hennessy S, Richards D, Gilbody S, & Lovell K (2007). What Makes Self-Help Interventions Effective in the Management of Depressive Symptoms? Meta-analysis and meta-regression. Psychological Medicine. 37, 1217–1228. doi: 10.1017/S0033291707000062 [DOI] [PubMed] [Google Scholar]
  13. Hosmer DW, Lemeshow S, & Sturdivant RX (2013). Sample Size Issues when Fitting Logistic Regression Models In Hosmer DW (Ed.), Applied Logistic Regression (3rd ed). (pp.399–347). New York, NY: Wiley-International Publication. [Google Scholar]
  14. Hundt NE, Mignogna J, Underhill C, & Cully JA (2013). The Relationship Between Use of CBT Skills and Depression Treatment Outcome: A Theoretical and Methodological Review of the Literature. Behavior Therapy, 44, 12–26. doi: 10.1016/j.beth.2012.10.001 [DOI] [PubMed] [Google Scholar]
  15. Jarrett RB, Minhajuddin A, Kangas JL, Friedman ES, Callan JA, & Thase ME (2013). Acute phase cognitive therapy for recurrent major depressive disorder: who drops out and how much do patient skills influence response? Behaviour Research and Therapy, 51, 221–230. doi: 10.1016/j.brat.2013.01.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Jarrett RB, Vittengl JR, Clark LA, & Thase ME (2011). Skills of Cognitive Therapy (SoCT): a new measure of patients’ comprehension and use. Psychological Assessment, 23, 578–586. doi: 10.1037/a0022485 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Johansson R, & Andersson G (2012). Internet-based psychological treatments for depression. Expert Review of Neurotherapeutics, 12, 861–869. doi: 10.7717/peerj.178 [DOI] [PubMed] [Google Scholar]
  18. John OP, Donahue EM, & Kentle RL (1991). The Big Five Inventory—Versions 4a and 54. Berkeley: University of California, Berkeley, Institute of Personality and Social Research. [Google Scholar]
  19. John OP, & Srivastava S (1999). The Big-Five trait taxonomy: History, measurement, and theoretical perspectives In Pervin LA & John OP (Eds.), Handbook of personality: Theory and research (pp.78–80). New York, NY: Guilford Press. [Google Scholar]
  20. Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, & Ferriter M (2008). The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychological Medicine, 38, 1521–1530. doi: 10.1017/S0033291707002607 [DOI] [PubMed] [Google Scholar]
  21. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G,… Christensen H (2015). Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data meta-analysis. Psychological Medicine, 45, 2717–2726. doi: 10.1017/S0033291715000665 [DOI] [PubMed] [Google Scholar]
  22. Klein DN, Kotov R, & Bufferd SJ (2011). Personality and depression: explanatory models and review of the evidence. Annual Review of Clinical Psychology, 7, 269–295. doi: 10.1146/annurev-clinpsy-032210-104540 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kroenke K, Spitzer RL, & Williams JB (2001). The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine, 16, 606–613. doi: 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Knowles SE, Toms G, Sanders C, Bee P, Lovell K, Rennick-Egglestone S,… & Gilbody S (2014). Qualitative meta-synthesis of user experience of computerised therapy for depression and anxiety. PLoS One, 9, e84323. doi: 10.1371/journal.pone.0084323 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lutz W, Schwartz B, Hofmann SG, Fisher AJ, Husen K, & Rubel JA (2018). Using network analysis for the prediction of treatment dropout in patients with mood and anxiety disorders: A methodological proof-of-concept study. Scientific Reports, 8, 7819. doi: 10.1038/s41598-018-25953-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. McGraw KO, & Wong SP (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. doi: 10.1037/1082-989X.1.1.30 [DOI] [Google Scholar]
  27. Mohr DC, Cuijpers P, & Lehman K (2011). Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. Journal of Medical Internet Research, 13, e30. doi: 10.2196/jmir.1602 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, & Begale M (2013). A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PloS One, 8, e70086. doi: 10.1371/journal.pone.0070086 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mohr DC, Tomasino KN, Lattie EG, Palac HL, Kwasny MJ, Weingardt K,… & Caccamo L (2017). IntelliCare: an eclectic, skills-based app suite for the treatment of depression and anxiety. Journal of Medical Internet Research, 19, e10. doi: 10.2196/jmir.6645 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Proudfoot J, Goldberg D, Mann A, Everitt B, Marks I, & Gray JA (2003). Computerized, interactive, multimedia cognitive-behavioural program for anxiety and depression in general practice. Psychological Medicine, 33, 217–227. doi: 10.1192/bjp.185.1.46 [DOI] [PubMed] [Google Scholar]
  31. Proudfoot J, Ryden C, Everitt B, Shapiro DA, Goldberg D, Mann A, … Gray JA (2004). Clinical efficacy of computerised cognitive–behavioural therapy for anxiety and depression in primary care: randomised controlled trial. The British Journal of Psychiatry, 185, 46–54. doi: 10.1192/bjp.185.1.46 [DOI] [PubMed] [Google Scholar]
  32. Quilty LC, De Fruyt F, Rolland J-P, Kennedy SH, Rouillon PF, & Bagby RM (2008). Dimensional personality traits and treatment outcome in patients with major depressive disorder. Journal of Affective Disorders, 108, 241–250. doi: 10.1016/j.jad.2007.10.022 [DOI] [PubMed] [Google Scholar]
  33. Richards D, & Richardson T (2012). Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clinical Psychology Review, 32, 329–342. doi: 10.1016/j.cpr.2012.02.004 [DOI] [PubMed] [Google Scholar]
  34. Sasso KE, & Strunk DR (2013). Thin slice ratings of client characteristics in intake assessments: predicting symptom change and dropout in cognitive therapy for depression. Behaviour Research and Therapy, 51, 443–450. doi: 10.1016/j.brat.2013.04.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, … Dunbar GC (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of Clinical Psychiatry, 59 Suppl 20, 22–33-57. [PubMed] [Google Scholar]
  36. Shipley WC (1940). A self-administering scale for measuring intellectual impairment and deterioration. Journal of Psychology: Interdisciplinary and Applied, 9, 371–377. doi: 10.1080/00223980.1940.9917704 [DOI] [Google Scholar]
  37. Strunk DR, DeRubeis RJ, Chui A, & Alvarez JA (2007). Patients’ competence in and performance of cognitive therapy skills: Relation to the reduction of relapse risk following treatment for depression. Journal of Consulting and Clinical Psychology, 75, 523–530. doi: 10.1037/0022-006X.75.4.523 [DOI] [PubMed] [Google Scholar]
  38. Spek V, Nyklícek I, Smits N, Cuijpers P, Riper H, Keyzer J, & Pop V (2007). Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychological Medicine, 37, 1797–1806. doi: 10.1017/S0033291707000542 [DOI] [PubMed] [Google Scholar]
  39. Srivastava S, John OP, Gosling SD, & Potter J (2003). Development of personality in early and middle adulthood: Set like plaster or persistent change? Journal of Personality and Social Psychology, 84, 1041–1053. doi: 10.1037/0022-3514.84.5.1041 [DOI] [PubMed] [Google Scholar]
  40. Swift JK, & Greenberg RP (2012). Premature discontinuation in adult psychotherapy: a meta-analysis. Journal of Consulting and Clinical Psychology, 80, 547–559. doi: 10.1037/a0028226 [DOI] [PubMed] [Google Scholar]
  41. Thombs BD, & Ziegelstein RC (2014). Does depression screening improve depression outcomes in primary care? BMJ (Clinical Research Ed.), 348, g1253. [DOI] [PubMed] [Google Scholar]
  42. Wagner B, Horn AB, & Maercker A (2014). Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. Journal of Affective Disorders, 152, 113–121. doi: 10.1016/j.jad.2013.06.032 [DOI] [PubMed] [Google Scholar]
  43. Waller R, & Gilbody S (2009). Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychological Medicine, 39, 705–712. doi: 10.1017/S0033291708004224 [DOI] [PubMed] [Google Scholar]
  44. Warmerdam L, van Straten A, Twisk J, Riper H, & Cuijpers P (2008). Internet-based treatment for adults with depressive symptoms: randomized controlled trial. Journal of Medical Internet Research, 10, e44. doi: 10.2196/jmir.1094 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, & Otto MW (2005). Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. The American Journal of Psychiatry, 162, 1158–1164. doi: 10.1176/appi.ajp.162.6.1158 [DOI] [PubMed] [Google Scholar]
  46. Zilcha-Mano S, Keefe JR, Chui H, Rubin A, Barrett MS, & Barber JP (2016). Reducing dropout in treatment for depression: Translating dropout predictors into individualized treatment recommendations. The Journal of Clinical Psychiatry, 77, e1584–e1590. doi: 10.4088/JCP.15m10081 [DOI] [PubMed] [Google Scholar]
  47. Zou G (2004). A modified Poisson regression approach to prospective studies with binary data. American Journal of Epidemiology 159, 702–706. doi: 10.1093/aje/kwh090 [DOI] [PubMed] [Google Scholar]

RESOURCES