The Effectiveness of Internet-Based Cognitive Behavioral Therapy for the Treatment of Depression in a Large Real-World Primary Care Practice: A Randomized Trial

Abstract

Purpose

The purpose of this study was to compare the effectiveness of internet-based cognitive behavioral therapy (iCBT) for depression, to that of treatment as usual (TAU) alone.

Methods

In this study, 302 depressed adult Family Medicine outpatients were randomized to receive either TAU or additional access to Thrive, a fully automated iCBT program with three video-based modules, each containing 10 lessons using behavioral activation, cognitive restructuring, and social skills training. The primary outcome was the change in the score on an online patient health questionnaire (PHQ-9), measured at 0, 8, and 24 weeks.

Results

The intervention group saw a relative improvement of 2.5 points in PHQ-9 scores at 8 weeks (p = 0.002, d = −0.48), was 6.0 times (p < 0.001) more likely to respond (defined as a ≥ 50% reduction in PHQ-9 score), and was 5.2 times (p = 0.04) more likely to have achieved remission (defined as a PHQ-9 score of < 5) at 8 weeks, but by 24 weeks, the control group had improved to a similar extent as the intervention group (d = −0.14). The intervention group improved in productivity at 8 weeks (p = 0.03), but by 24 weeks, the TAU group had also improved to a similar extent. No significant differences in anxiety, quality of life, or suicidal ideation were found. Patients reported high satisfaction with this iCBT tool, including ease of use, tailoring, and perceived helpfulness. However, only 43% of the intervention group and 58% of the TAU group had outcome measures at every time point.

Conclusions

iCBT was associated with greater depression response and remission at 8 weeks, compared with the control group. Depression scores in the intervention group remained similar at 24 weeks, at which time the control group also showed similar rate of response and remission.

Introduction

Depression is the most common mental disorder in the US, with a lifetime prevalence rate of 20.6%, ¹ and is the leading cause of disability globally. ^2,3 Higher mortality rates, ⁴ greater functional impairment, ⁵ substantial work/life productivity costs, ⁶ and major risk for suicide are all associated with depression. ² Depression substantially burdens the US health care system, where people with depression present for treatment most frequently in primary care settings ⁷ and total depression-related health care costs reach an estimated $71 billion annually. ⁸

Despite the existence of effective treatments for depression, 34% of individuals with depression receive no treatment. ⁹ There are numerous barriers to care including availability of practitioners, convenience, affordability, acceptability of treatments, stigma, and practitioner–patient relationships. ^10,11 Addressing barriers to psychotherapy treatment may be especially effective, as patients with depression prefer psychotherapy over medication by 3:1. ¹² Psychotherapy such as cognitive behavioral therapy (CBT) has similar efficacy to antidepressant medications. ¹³ Improving access to and use of psychotherapy, especially in primary care settings, could have a substantial impact on depression-related health care costs.

Internet-based CBT (iCBT) with some clinical provider contact shows promise for the delivery of evidence-based depression treatment in primary care settings, with significant improvements in depression symptoms compared to controls ^14–16 and with efficacy similar to that of face-to-face CBT. ^17–19 In addition, unguided iCBT (with no support related to the therapeutic content) may be more cost-effective and feasible for wide-spread delivery in large primary care settings.

In this randomized efficacy trial, the authors examined the effects of Thrive, an unguided iCBT program, owned by Pear Therapeutics (not associated with Kaiser Permanente). The authors compared depression scores between those who participated in iCBT plus treatment as usual (TAU) and those who participated in TAU alone for 302 adults with clinical depression, 48% of whom were of a racial/ethnic minority, in a large integrated primary care setting serving 4.6 million members. The study was designed to mimic the conditions under which iCBT would be implemented as much as possible so that, if efficacious, health care systems could easily adopt and implement it. The authors hypothesized that, compared to adult patients assigned to TAU alone, the intervention group would have improved depression symptoms during the 24-week study period. Symptoms were reassessed at 8 weeks (when the intervention ended) and at 24 weeks.

Methods

Participants

All recruitment was from the Southern California Kaiser Permanente, San Bernardino County Area, Department of Family Medicine. Queries from the electronic medical record used the following inclusion criteria: ≥ 18 years old, English-speaking, having an in-person visit with their primary care provider within the past 2 weeks, and current depression or related diagnosis associated with that visit. Patients with a current diagnosis of schizophrenia (or related disorder), delirium, bipolar disorder, any type of neurocognitive disorder, intellectual disability, personality disorder, or active substance use disorder were excluded. From this list, 5265 potential patients were initially contacted for the study. Of these, 469 accessed the study website and completed the online patient health questionnaire (PHQ-9) screening (8.9% response rate; see Figure 1). They were then contacted by email (or letter if email was unavailable), and 31 were further excluded because their PHQ-9 ²⁰ responses resulted in a total score of < 5 or because the patient stated that they were suicidal and could not keep themselves safe. An additional 136 individuals then refused to participate, leaving 302 patients with a PHQ-9 score of 6–27 (no upper limit) who were randomized to receive TAU alone (n = 154) or Thrive iCBT plus TAU (n = 148).

Figure 1:

Consort diagram. PHQ-9 = patient health questionnaire.

Procedures

Recruitment and enrollment

Between October 18, 2016, and July 18, 2017, potential participants logged on (using a unique code) to a study website designed to 1) inform them about the trial, 2) administer a PHQ-9 online survey to verify depression symptoms, 3) obtain electronic informed consent, 4) randomize patients to receive TAU alone or iCBT plus TAU, and 5) administer baseline assessments. Once patients had been randomized, those who were assigned to the intervention were given access immediately to Thrive iCBT by being redirected to a separate website.

Intervention

Thrive iCBT was designed to teach patients CBT techniques primarily through videos. Thrive includes an introductory lesson and then three modules, each containing 10 lessons (N = 31 lessons total), based on behavioral activation, cognitive restructuring, and social skills training. The intervention was designed for 8 weeks but can be completed in as little as 3 weeks and was available to patients for up to 6 months after enrollment. Depending on patient responses, Thrive iCBT software guides participants through the modules, progressing from “learn about the technique,” to “apply the technique online,” and finally to “apply the technique in real life.” The program personalizes the participant’s experience (eg, content, exercises, suggestions) based on usage patterns and answers to questions posed, and it provides regular feedback to patients on their progress.

In addition, automated “coaching” emails were sent to encourage participation in the intervention.

Usual care

Typical depression care in this study included medication management by a primary care physician, or treatment in a collaborative care program for depression embedded in an adult primary care department, or referral to psychiatry. Use of these treatments was monitored in the electronic medical record. All patients, regardless of randomization, had access to these services, and their primary care physicians were not alerted that the patients were participating in the study.

Measures

Data obtained from electronic medical records included age, gender, and ethnicity at the time of enrollment and use of antidepressants, psychotherapy, and/or collaborative care for depression during the study period. At 8 weeks and 24 weeks, all patients were reminded by email, and a $20 electronic gift card was sent as incentive to log on to the assessment website, which administered the PHQ-9, a generalized anxiety disorder screener (GAD-7), and measures of quality of life and productivity. The primary study outcome was change in PHQ-9 score, measured longitudinally from baseline to 24 weeks. Secondary outcomes included changes in anxiety, quality of life, and productivity.

Statistical analyses

Changes in PHQ-9 scores were analyzed using an intent-to-treat approach, with the linear model for correlated data. ²¹ Baseline, 8-week, and 24-week PHQ-9 measurements for all patients were included to model the change in PHQ-9 score from baseline to 24 weeks. This model contained fixed effects terms for treatment (iCBT plus TAU vs TAU alone), time (8 weeks and 24 weeks vs baseline), and the treatment × time interaction. This functional form for time was chosen so that trends over 6 months were not constrained to a specific parametric form.

The change in PHQ-9 scores over 6 months for each study group was compared by analyzing the test statistic, p value, and effect size of the treatment × time interaction. Because of the randomized, balanced study design and small number of time points, the correlation over time was modeled using an unstructured covariance. ²² Restricted maximum likelihood estimation and Type 3 tests of fixed effects were used. Secondary outcomes were analyzed in a similar manner, but only effect sizes are presented. Effect size estimates at each time point were calculated by dividing the time-specific coefficient from the treatment × time interaction in the linear model for correlated data by the baseline pooled standard deviation for each outcome. ²³

Results

Participants

Baseline characteristics were measured for all participants at the beginning of the study (Table 1). Patients were randomly assigned to receive Thrive iCBT plus TAU or TAU alone.

Table 1:

Baseline characteristics of the intention-to-treat population by study group

Characteristic	iCBT + TAU (n = 148)	TAU (n = 154)	All (N = 302)
Age (years)	47.5 ± 16.8	46.9 ± 15.5	47.2 ± 16.1
Female patients	83.1% (123)	80.5% (124)	81.1% (247)
Non-Hispanic White patients	52.7% (78)	50.6% (78)	51.7% (156)
Depression symptoms total score (range 5–27)	14.6 ± 5.1	14.9 ± 5.2	14.8 ± 5.1
Mild (total score 5–9)	19.6% (29)	16.2% (25)	17.9% (54)
Moderate (total score 10–14)	27.7% (41)	31.2% (48)	29.5% (89)
Moderate to severe (total score 15–19)	35.8% (53)	29.2% (45)	32.5% (98)
Severe (total score > 20)	16.9% (25)	23.4% (36)	20.2% (61)
Anxiety symptoms total score (range 0–21)	11.1 ± 4.7	11.6 ± 5.0	11.4 ± 4.8
Quality of life (range 0–10)	4.8 ± 1.9	4.8 ± 1.9	4.8 ± 1.9
Productivity (range 0–10)	5.1 ± 2.2	4.9 ± 2.2	5.0 ± 2.2
Taking antidepressants	71.6% (106)	76.0% (117)	73.8% (223)
Other depression-related treatment	50.0% (74)	55.8% (86)	53.0% (160)

iCBT, internet-based cognitive behavioral therapy; TAU, treatment as usual.

Study participants were 47.2 ± 16.1 years old and included 81.1% female patients and 51.7% non-Hispanic White patients. Most patients (regardless of study group) were taking antidepressants (73.8%) and/or were engaged in some other form of depression treatment (53.0%) while enrolled in the study.

At the 8-week assessment, 52% of iCBT plus TAU patients and 64% of TAU alone patients completed the primary and secondary outcome measures, with completion rates increasing at 24 weeks to 55% and 74%, respectively. A total of 63 (43%) iCBT plus TAU and 90 (58%) TAU alone patients had assessments at every time point. The average number of Thrive iCBT lessons completed was 6 ± 7 (range 1–30).

Primary outcome: Depression symptoms

Mean estimates from the linear models for correlated data are presented in Table 2 for depression symptoms over 6 months. The intervention had a statistically significant effect on PHQ-9 total scores (F = 6.28, p = 0.002), with patients in the iCBT plus TAU group experiencing a moderate mean reduction in PHQ-9 total score at 8 weeks (est = −2.54; 95% CI = −4.02, −1.07; d = −0.48) compared to patients in the TAU alone group (iCBT PHQ-9 total score of 14.6 at baseline decreased to 8.1 at 8 weeks vs TAU alone score of 14.9 at baseline decreased to 10.9 at 8 weeks). Although the between-group differences were not maintained at 24 weeks (est = −0.74; 95% CI = −2.31, 0.83; d = −0.14), this was because the intervention group maintained the 8-week symptom reduction while the TAU alone group exhibited similar improvements by 24 weeks (iCBT PHQ-9 total score of 8.9 vs TAU score of 9.9).

Table 2:

Estimates and 95% confidence intervals from linear models for correlated data for Thrive iCBT intervention plus TAU and TAU alone groups over 24 weeks for primary and secondary outcomes ^a

Outcome	Thrive iCBT + TAU						TAU alone
	n	Baseline	n	8 wks	n	24 wks	n	Baseline	n	8 wks	n	24 wks
Depression symptoms	148	14.6 (13.8, 15.5)	77	8.1 (7.0, 9.2)	82	8.9 (7.7, 10.1)	154	14.9 (14.1, 15.7)	99	10.9 (9.9, 11.9)	114	9.9 (8.9, 10.9)
Productivity	139	5.15 (4.8, 5.5)	77	6.5 (5.9, 6.9)	82	6.3 (5.8, 6.8)	150	4.9 (4.6, 5.3)	99	5.4 (4.9, 5.8)	114	6.0 (5.6, 6.4)
Anxiety symptoms	140	11.1 (10.3, 11.9)	77	7.2 (6.1, 8.2)	82	7.0 (6.0, 8.1)	151	11.7 (10.9, 12.4)	99	8.6 (7.6, 9.5)	114	7.99 (7.1, 8.9)
Quality of life	139	4.9 (4.5, 5.2)	77	6.3 (5.9, 6.7)	82	6.0 (5.6, 6.4)	150	4.8 (4.5, 5.1)	99	5.7 (5.3, 6.0)	114	5.9 (5.5, 6.2)

^aStatistical analyses were performed using SAS Enterprise Guide 7.1 (SAS Institute).

iCBT, internet-based cognitive behavioral therapy; TAU, treatment as usual.

Secondary outcomes: Productivity, anxiety, and quality of life

Mean estimates from the linear models for correlated data are presented in Table 2 for secondary outcomes over 6 months. The intervention group experienced a small to moderate increase in productivity at 8 weeks from 5.15 to 6.5 (est = 0.88; 95% CI = 0.14, 1.61; d = 0.39) compared to the control group (4.9 at baseline to 5.4 at 8 weeks). The intervention group remained as productive at 6 months; however, the control group made similar gains during the same time period (est = 0.11; 95% CI = −0.59, 0.82; d = 0.04). Effect sizes were clinically insignificant between groups in anxiety symptoms and quality of life over 6 months. Intervention patients reported high satisfaction with iCBT, reporting that it was easy to use (88%), tailored to their situation (80%), and helpful (80%).

Discussion

The intervention group experienced a moderate reduction in depression symptom severity as measured by the PHQ-9 total score at 8 weeks (d = −0.48) compared to patients receiving TAU alone (iCBT PHQ-9 total score of 14.6 at baseline and 8.1 at 8 weeks vs TAU alone total score of 14.9 at baseline and 10.9 at 8 weeks), but by 6 months, the two groups saw similar improvements from baseline. This faster response for those engaging in iCBT may have freed up needed resources, reducing the need for specialty referrals and lowering the risk of recurrence. ^24,25 Furthermore, patients that respond to traditional CBT in the acute phase of depression are less likely to relapse after withdrawal of treatment than those withdrawn from antidepressants, ¹³ so this intervention may have lowered the risk of recurrence, although this study was not designed to test this outcome.

The generalizability of these findings to an adult primary care population with depression may be limited because all patients contacted were English-speaking, of which only 8.9% responded, and 5.7% enrolled. Although our recruitment was better than those of other similar studies (1.8%–3.8% response and 1.8–2.1% enrollment), ^26–28 it was not as favorable as those found in studies in which referral was from a practitioner. ^29–31 Notably, our study recruitment included no phone calls or clinician involvement and was designed to place as little burden as possible on the primary care teams.

Another limitation was the high rates of missing outcome data, which were highest for patients in the intervention (only 52% and 55% had outcome data at 8 weeks and 24 weeks, respectively). This problem is not unique to the present study. Retention in comparable studies has varied from 53% to 90%. ^{26–28,30–36} It is difficult to conclude that the use of Thrive iCBT was as effective for those who did not complete study measures. It is possible that patients who completed assessments would do better during iCBT plus TAU because of greater motivation to adhere to the intervention. Higher attrition for intervention participants has also been reported for interventions for health behavior. ³⁷ Compared to the control group, our intervention group spent more time on study questionnaires, in addition to the time spent using Thrive. The greater amount of work required for participation in the intervention arm may explain some of the higher attrition in this group.

Compounding the missing rates for primary and secondary outcome measures, the average number of lessons completed was only 6/31 (19%). A recent meta-analysis of self-guided iCBT randomized controlled trials for depression reported that an average of 17% of participants completed all sessions. ³⁸ Five of the 13 published iCBT randomized controlled trials in primary care settings (both guided and unguided programs) reported similarly low intervention utilization, with the average number of sessions completed ranging from 2.6 to 8.5. ^{26–28,33,39}

Despite these limitations, this real-world iCBT intervention demonstrated the possibility of more rapid improvement in depression symptoms compared with TAU. Engagement in this pre-pandemic intervention was relatively low, but 2020 saw a sharp increase in demand for telehealth technologies, as patients have had to adapt to social distancing measures. Thus, repeating this trial in a post-pandemic environment may result in higher engagement and more relevant results.