A Randomized, Placebo-Controlled Trial of Online Cognitive Behavioral Therapy for Chronic Insomnia Disorder Delivered via an Automated Media-Rich Web Application

Abstract

Study Objectives:

The internet provides a pervasive milieu for healthcare delivery. The purpose of this study was to determine the effectiveness of a novel web-based cognitive behavioral therapy (CBT) course delivered by an automated virtual therapist, when compared with a credible placebo; an approach required because web products may be intrinsically engaging, and vulnerable to placebo response.

Design:

Randomized, placebo-controlled trial comprising 3 arms: CBT, imagery relief therapy (IRT: placebo), treatment as usual (TAU).

Setting:

Online community of participants in the UK.

Participants:

One hundred sixty-four adults (120 F: [mean age 49y (18-78y)] meeting proposed DSM-5 criteria for Insomnia Disorder, randomly assigned to CBT (n = 55; 40 F), IRT placebo (n = 55; 42 F) or TAU (n = 54; 38 F).

Interventions:

CBT and IRT each comprised 6 online sessions delivered by an animated personal therapist, with automated web and email support. Participants also had access to a video library/back catalogue of session content and Wikipedia style articles. Online CBT users had access to a moderated social network/community of users. TAU comprised no restrictions on usual care and access to an online sleep diary.

Measurements and Results:

Major assessments at baseline, post-treatment, and at follow-up 8-weeks post-treatment; outcomes appraised by online sleep diaries and clinical status. On the primary endpoint of sleep efficiency (SE; total time asleep expressed as a percentage of the total time spent in bed), online CBT was associated with sustained improvement at post-treatment (+20%) relative to both TAU (+6%; d = 0.95) and IRT (+6%: d = 1.06), and at 8 weeks (+20%) relative to IRT (+7%: d = 1.00) and TAU (+9%: d = 0.69) These findings were mirrored across a range of sleep diary measures. Clinical benefits of CBT were evidenced by modest superiority over placebo on daytime outcomes (d = 0.23-0.37) and by substantial improved sleep-wake functioning on the Sleep Condition Indicator (range of d = 0.77-1.20). Three-quarters of CBT participants (76% [CBT] vs. 29% [IRT] and 18% [TAU]) completed treatment with SE > 80%, more than half (55% [CBT] vs. 17% [IRT] and 8% [TAU]) with SE > 85%, and over one-third (38% [CBT] vs. 6% [IRT] and 0% [TAU]) with SE > 90%; these improvements were largely maintained during follow-up.

Conclusion:

CBT delivered using a media-rich web application with automated support and a community forum is effective in improving the sleep and associated daytime functioning of adults with insomnia disorder.

Clinical Trial Registration:

ISRCTN – 44615689.

Citation:

Espie CA; Kyle SD; Williams C; Ong JC; Douglas NJ; Hames P; Brown JSL. A randomized, placebo-controlled trial of online cognitive behavioral therapy for chronic insomnia disorder delivered via an automated media-rich web application. SLEEP 2012;35(6):769-781.

INTRODUCTION

Sleep disturbance is the most common symptom of mental illness in the UK.¹ Worldwide, epidemiologic studies report the prevalence of a clinical insomnia disorder at 10% to 12%,^2,3 and longitudinal investigation has shown that, once established, insomnia disorder tends to persist.⁴ Typically, insomnia is associated with increased fatigue, impaired work productivity, reduced quality of life and relationship satisfaction, as well as increased ill health.^5–9 Chronic insomnia may be a risk factor for the development of mental and physical health problems^10–13 and is possibly associated with all-cause mortality.^14–16 The importance of insomnia to public health is illustrated by national annual costs ($92 to $107 billion USD in USA),¹⁷ and its cost per untreated case ($5,000 CAD in Canada).¹⁸ Despite this, persistent insomnia often goes unrecognized, and care management is poorly developed.^19,20

Benzodiazepine hypnotics and sedative antidepressants are commonly prescribed, although long-term outcome data are relatively sparse.^21,22 Whereas the benzodiazepine receptor agonists confer some advantages, there is limited evidence that they are preferable for chronic insomnia.^19,23 On the other hand, there is compelling evidence that cognitive behavioral therapy (CBT) is a lastingly effective treatment,^19,24–27 a good conceptual fit for psychological factors that commonly underlie insomnia,^5,28 and an approach that many patients prefer over a pharmacological one.^29,30

There is support for CBT being made widely available^{19,24–27,31,32}; however, the outstanding challenge is its inherent lack of scalability to meet population need.³³ This is not untypical of the broader population interest in solutions that shifts the focus from the professional to the person and from the clinic to home implementation.³⁴ Traditionally, CBT is delivered face-to-face by a specialist psychological therapist, and so is dependent on a rare and expensive resource.^33,35 It seems inconceivable that any face-to-face therapy could replace, for example, the 12 million prescriptions for hypnotics that are written each year in England and Scotland for a combined population base of 47 million adults.^36,37 Moreover, “stepped care” models argue that expert professionals should consult on more complex cases rather than deal with routine referrals.^33,38 Although, there is evidence that nurses, trained to follow a CBT manual, can deliver effective treatment to small groups of patients^39–41 and that large community group interventions may also be feasible,⁴² this approach is unlikely to offer a realistic alternative to prescribing because it too relies on regular contact, and with professionals whose duties are already many and varied.³³

A review of 9 controlled studies, where written materials were distributed directly to people with insomnia, sometimes along with other media or telephone support, suggested some benefit, although effect sizes were generally small.⁴³ Six investigations of internet-based CBT offer encouraging results,^44–49 suggestive of the potential far-reaching benefits of this health technology. In perhaps the best-designed study, albeit on a small total sample (n = 45), a 16% improvement in sleep efficiency (SE: proportion of time in bed spent asleep) relative to baseline was observed following CBT (an absolute increase of 12% from pre- to post-treatment), compared with 3% (2% in absolute terms) in a wait-list group.⁴⁶ These data were mirrored by significant reductions in insomnia severity. Uncontrolled data also suggested gains were maintained. In the 2 largest studies,^47,49 significant effects of CBT over a wait-list condition were also encouraging, although limited to improvements in sleep quality and reductions in fatigue, rather than sleep parameters per se,⁴⁷ or showing small-to-moderate effects, with 49% dropout in the electronic CBT group.⁴⁹

The literature on internet CBT for insomnia remains small and lacks a definitive study. In particular, a placebo-controlled trial is required if we are to be sure that reported improvements are not simply the result of a novel mode of treatment, participant enthusiasm, expectations, or experimental demand characteristics.⁵⁰ More than this, however, the intervention platforms evaluated so far, may not fully reflect the levels of sophistication that might be expected by contemporary internet users, for example, full web and mobile interactivity and the use of social networking.⁵¹ Indeed, offering CBT within a supportive self-help environment may be crucial in helping people apply what they are learning.⁵²

The objectives of this study were to address these scientific and technical imperatives by addressing the following questions: Is CBT for chronic insomnia disorder—delivered via an automated, media-rich web application—superior to a credible placebo intervention, as well as to a treatment as usual condition, in improving nighttime sleep and associated daytime functioning? Are these effects durable and clinically important?

METHODS

Participants from the UK community (18+ years), who had completed the online Great British Sleep Survey (GBSS), and who met proposed DSM-5 criteria for persistent Insomnia Disorder were invited to take part.^53,54 The GBSS utilized pre-established algorithms to screen for: (a) a current complaint of poor sleep (difficulty initiating and/or maintaining sleep, early morning wakening, or non-restorative sleep); with (b) significant daytime effects in ≥ 1 of 6 domains (fatigue, daytime sleepiness, cognitive impairment [e.g., concentration problems], mood disturbance, impaired occupational or academic functioning [e.g., poor productivity], impaired interpersonal/social functioning); and (c) affecting them ≥ 3 nights per week for ≥ 3 months.⁵⁵ People who reported being in “poor” or “very poor” physical or mental health, or who exceeded a threshold for other sleep disorders on our screening algorithm (published in Wilson et al.²⁴) were excluded. Items from the AUDIT⁵⁶ and CAGE⁵⁷ were applied to identify heavy alcohol use, and cutoff scores on the Depression Anxiety Stress Scales provided supplementary data on mental health status.⁵⁸ The use of sleeping pills or other sleep aids was permitted. Usual care that participants had been receiving via their medical advisers, including medical prescriptions and any counselling or psychotherapy, continued in all arms. The website www.sleepio.com/research hosts illustrative material of the study evaluation and intervention procedures.

The GBSS was launched online in February 2010 by Sleepio Ltd (a company dedicated to helping people sleep better, through raising awareness, research, and the dissemination of behavioral treatment advice), in association with Boots UK (an international pharmacy-led health and beauty group) and the Mental Health Foundation (a leading UK mental health research, policy and service improvement charity). Growth was “organic in nature,” driven for example by links to Boots WebMD (www.webmd.boots.com), Mental Health Foundation campaigning, and newspaper media exposure.

A total of 10,071 adults completed the GBSS from April 2010 to 25 February 2011, of whom 6,609 provided email addresses. In total, 1,342 of this latter group (20.3%) met preliminary screening criteria and were invited by email to consider taking part; of these, 276 (20.1%) replied and consented to further screening. The majority (n = 228, 82.6%) then completed further standardized assessments. Finally, to confirm current eligibility, participants completed prospectively a 7-day online sleep diary, during which they had to have a mean (baseline) sleep efficiency (SE) ≤ 79% to reflect a sleep problem of clinical severity on our primary outcome variable. Sixty-four participants were excluded during these final stages, and the remaining sample of 164 eligible, consenting participants was randomized (see participant flowchart: Figure 1).

Figure 1.

Flow of Participants in the Trial.

In order to ensure real-world evaluation of the online intervention, participant enrolment was confined to email contact, and all eligibility and baseline data were automatically obtained without face-to-face verification. Ineligible participants were provided with a report comprising tailored sleep hygiene advice, and all participants were advised to contact their doctor if they had concerns about any aspect of their health. The website also hosted a list of telephone contact numbers for mental health helplines. Technical support to participants was provided by email contact or via the online community forum, where there was a dedicated discussion thread for identifying and resolving problems.

Study Design

The study was a pragmatic, parallel-group, randomized controlled trial comprising 3 treatment arms: (1) online CBT; (2) online imagery relief therapy (IRT: placebo); (3) treatment as usual (TAU), with blind assignment to group determined by a computer-generated random allocation schedule, operated by a remote independent technical operator. The trial followed CONSORT 2010 guidelines.⁵⁹ Consistent with the inclusion/exclusion criteria, the study design in effect was CBT+TAU v. IRT+TAU v. TAU alone. Major assessments were at baseline, post-treatment, and follow-up 8 weeks later. Participants randomized to the IRT placebo or TAU arm were offered the online CBT package upon completion of the study. All assessment, treatment, and data-gathering procedures were conducted online, and all queries/enquiries were managed electronically. These procedures ensured that the trial was genuinely an evaluation of a completely online CBT approach. The study protocol was approved by the University of Glasgow, Faculty of Medicine Research Ethics Committee and all participants provided informed consent online (see www.sleepio.com/research).

Assessment Measures

Participants accessed an online daily sleep diary throughout the study, to be completed each morning upon rising. They could set automated SMS (mobile text message) and/or email prompts as reminders. Such diaries are the staple tool of insomnia assessment.^60–62 Participants completed items by selecting from a drop-down menu of possible values. “How long did it take you to fall asleep last night” and “how long were you awake in total last night due to wakenings after you first fell asleep” (each variable offered as 0 min, 5 min, 10 min, 15 min, 30 min, and so on, thereafter in 15-min increments) assessed the central insomnia dimensions of difficulty initiating sleep (sleep onset latency [SOL], min) and difficulty maintaining sleep (wake time after sleep onset [WASO], min). The diary also enquired about bedtime and rising time, from which total time in bed (TIB), and thence sleep efficiency (SE, %) were calculated; [1 − (SOL + WASO / TIB)] × 100. Total Sleep Time (TST, h) was also estimated from diary data [TIB − (SOL + WASO)]. A sensitive rating of “overall sleep quality” was obtained by dragging a slider along a dimensional analogue scale (0–100) between the poles of “very unsatisfactory” and “very satisfactory.”

Diary data yielded the dependent variables of SOL, WASO, SE, TST, and sleep quality, averaged across 7 nights for each of the 3 major assessment points: baseline, post treatment, and follow-up (see www.sleepio.com/research). The primary study endpoint was change in SE from pre- to post-treatment, and from pre-treatment to follow-up for 2 reasons. First, sleep efficiency provides an overall index of insomnia by capturing both difficulties getting to sleep and staying asleep, and so was relevant for all participants of whatever insomnia subtype; second, endpoints relating to achievement of sleep efficiencies of 80%, 85%, and 90% reflect clinically important improvement and not merely statistical change.⁶³

Importantly, because Insomnia Disorder must incorporate defined consequences, 6 domains of daytime function recommended by DSM-5 (energy, relationships, mood, concentration, productivity, sleepiness) were rated on a 5-point scale at each assessment phase (0 = not at all affected through to 4 = very much affected). Principal components analysis of data from our UK sample (n = 11,129) suggests that these items load (r ≥ 0.64) on 2 factors: “daytime performance,” comprising concentration, productivity, and sleepiness ratings (64.9% of the variance); and “social functioning,” comprising ratings on mood, relationships, and energy (12.0% of the variance).⁵⁵ Accordingly, we used these 2 factor scores to evaluate daytime impact of treatment.

The Sleep Condition Indicator (SCI) is a new patient-reported outcome measure, specifically based upon DSM-5 Insomnia Disorder criteria.^53,54 It is brief (8-item) and has shown preliminary reliability in a field study of 11,129 participants (α = 0.894, range of α-if-items systematically deleted = 0.877–0.898).⁵⁵ The SCI generates scores in the range 0 to 10, with higher values reflecting a person's sleep being in “better condition.” SCI ≤ 5.9 identifies 95.4% of people with insomnia disorder, whereas a score ≥ 6.0 correctly identifies 76.8% of individuals without insomnia disorder.⁵⁵ The SCI also correlates with the Pittsburgh Sleep Quality Index⁶⁴ (r = −0.78, n = 256) and the Insomnia Severity Index⁶⁵ (r = −0.79, n = 256).⁶⁶ The SCI provided a secondary, clinically focused outcome measure for the study.

Finally, we selected the Depression Anxiety Stress Scale (DASS: 21 items)⁵⁸ because it takes a dimensional view of symptoms, including stress, which often co-present with insomnia. Internal consistency for the DASS is satisfactory (α = 0.82-0.93). Items from the Sleep Disturbance Questionnaire (SDQ)^67,68 and the Glasgow Content of Thoughts Inventory (GCTI)⁶⁹ were also completed to inform CBT treatment algorithms.

The integrity of all data was assured by the online acquisition system and supporting software application. Clear guidance was provided including pop-up “tool tip” explanations for many items to ensure that they were correctly understood. All data entries were time stamped for all participants for the duration of the online course.

Treatment Groups

CBT

Participants received 6 weekly sessions delivered by an animated “virtual therapist” (The Prof). The program comprised a fully automated media-rich web application, driven dynamically by baseline, adherence, performance, and progress data. At the start of each session, The Prof conducted a progress review with the participant, explored the diary data submitted during the week, the participant's current sleep status and pattern, and progress achieved against goals previously set. Underlying algorithms fed the delivery of information, support, and advice in a personally tailored manner. CBT content was consistent with the literature,⁶⁰ and covered behavioral (e.g., sleep restriction, stimulus control) and cognitive (e.g., putting the day to rest, thought re-structuring, imagery, articulatory suppression, paradoxical intention, mindfulness) strategies, as well as additional relaxation strategies (progressive muscle relaxation and autogenic training) and advice on lifestyle and bedroom factors (sleep hygiene). The intervention was based upon a previously validated manual.^39–41 The following illustrations may be helpful. In sleep restriction, The Prof proposes a new “window” for sleep, calculated from available sleep diary data, and engages with the participant to help them select the timing (onset/offset) of this window from a set of personalized options. An example of a cognitive technique, is where another animated character (with insomnia) presents to the Prof their concerns, dysfunctional beliefs, and associated emotions. The Prof then asks the participant to choose some solutions from a menu of options and delivers this as advice to the character, who is seen to revise his thinking. The Prof then reveals to the participant that the scenario was based upon his/her own sleep-related attributions and thoughts (from baseline SDQ and GCTI data). In this way the participant is helped to learn how to restructure dysfunctional thinking. Table 1 summarizes the content and features of the intervention, permitting comparison of CBT with IRT and TAU conditions (with further illustration available at http://www.sleepio.com/research).

Table 1.

Summary of treatment conditions

IRT

Imagery relief therapy was also delivered by The Prof, using the same application platform, and design and execution principles as for CBT, but with no known active therapeutic ingredient. IRT was based on a well-established and credible non-pharmacological placebo intervention⁵⁰ used in several clinical trials.^70–72 The term imagery relief therapy was selected to enhance credibility of an active and novel therapy. For example, if participants were to enter this as an internet search term, they would come upon material which would appear to be valid for a psychological problem such as sleep. However, IRT contained no active components of imagery training, or of systematic desensitization. Likewise, it did not include detailed relaxation instruction or behavioral advice about what to do during the sleep period. The participant was trained to visualize neutral objects (e.g., a key) or shapes (e.g., a yellow square) in conjunction with thinking about sequential aspects of their evening routine (e.g., setting the table for dinner), and was asked to practice these pairings for 20 min/day early in the evening. The rationale for this “quasi-desensitization” framework was that successful sleep was associated with good preparation, and that neutralizing unhelpful associations with evening routines would recondition them towards automatic sleep engagement and sleep maintenance. IRT participants also received e-mail reminders from The Prof and had access to Wikipedia-style articles on sleep, its functions, and its disorders.

Protocol standardization

The integrity and fidelity of treatment allocations was assured by the online procedures which delivered the interventions. In addition, in session 1 of both CBT and IRT conditions, participants were invited to commit (or not) to the course following an explanation of the therapeutic rationale (all did so). CBT and IRT participants did not have contact with each other, nor did they have access to alternative treatment materials. Both CBT and IRT were scripted and automated, so support and length of treatment was similar. All web-based interactions were electronically stored to provide time-stamped data on participant activity (e.g., diary entries, session activities, engagement with the community, adherence to tasks).

TAU alone

In real world practice, insomnia patients often have some concurrent physical and psychological symptoms, as well as concurrent treatments. Therefore, to reflect validity, and to permit greater generalizability of findings, the protocol explicitly permitted continuation of treatment as usual health care for all participants. Physicians were free to offer appointments, to prescribe, and to maintain/discontinue prescriptions. Aside from this, TAU alone participants comprised, effectively, a wait-list group who completed measures but received no additional help for their insomnia. The only contact received by the TAU arm was reminders to complete evaluations. After the trial was completed TAU and IRT participants were offered access to the CBT intervention.

Data Management and Analysis

The study was designed to have 80% power to detect a medium effect size (Cohen⁷³; consistent with published meta-analytic data²⁷), based upon a 3-group ANOVA model with fixed effects, main effects, and interactions, on our primary outcome measure of SE. These criteria implied recruiting a total sample of 159 participants. All comparisons were planned and tests were 2-sided, with P < 0.05 considered to indicate statistical significance. Where appropriate, to control for multiple comparisons, a per family error rate was adopted to maintain the nominal error rate (0.05/n of comparisons). Analyses were performed using PASW Statistics 18 (SPSS Inc., Chicago, IL).⁷⁴ Linear mixed effects models were used, to avoid imputation of missing data (estimated at 16.1% of those commencing the trial at post-treatment and 19.9% at follow-up), predicting mean values at each assessment point (baseline, post-treatment, 8-week follow-up) and to test our hypotheses with respect to between group differences. In each model, time and treatment group were included as fixed effects, with time and group × time interaction terms. For variables exhibiting between-group differences at baseline, the baseline value was entered as a covariate. Clinical response to treatment was evaluated on an intention to treat basis in relation to proportions of participants achieving the clinical endpoints of 80%, 85%, and 90% for SE at post-treatment and follow-up.

RESULTS

Participant Characteristics

Information on the allocated sample of 164 (120 F) adults (mean age 49 y [18–78y]) is provided in Table 2. Approximately two-thirds were employed either full-time or part-time. Post-code data provided a proxy for socioeconomic status by deriving an index of multiple deprivation (IMD). Mean IMD was 16.7 (SD 11.8); somewhat less deprived (by < 0.5 SD) than national norms (21.7 [SD 15.5]).⁷⁵ Participants were in at least average health (as per selection criteria), although around 30% and 10%, respectively, took medication for a physical or mental health problem. One in 5 participants sometimes used prescribed sleeping pills, and 40% made some use of over-the-counter (OTC) sleep aids. Twenty-nine people provided additional free text on the strategies they used to manage their insomnia. Much of this was amplification on their medication or OTCs; however, 9 described using relaxation, meditation, or yoga, and 4 used devices (e.g., ear plugs, a heat pad).

Table 2.

Demographic and clinical characteristics of participants (n = 164)

All participants had DSM-5 Insomnia Disorder, the great majority being of difficulty maintaining sleep or of mixed subtypes. Two-thirds had had insomnia > 6 years, and almost 50% for > 11 years. Participants randomized to CBT (n = 55), IRT (n = 55), and TAU (n = 54) were similar in all demographic and clinical respects. Eight of the 164 participants did not start their treatment, so the final sample receiving the allocated intervention was 156 (CBT = 53, IRT = 52, TAU = 51). There were no significant differences on any variable between this sample and the allocated sample of 164.

Treatment Attrition and Integrity

Of those receiving their allocated intervention, 43 CBT participants (82%) completed all their online therapy sessions, and 47 (88%) completed ≥ 4 sessions. This compared to 41 (79%) and 48 (92%), respectively, in the IRT group. Thus, there were similar modest levels of attrition during the treatment phase. Just under 80% (n = 125) completed follow-up assessment, comprising similar proportions of CBT and IRT, but a higher proportion of TAU (CBT [n = 40, 75%]; IRT [n = 38, 73%]; TAU [n = 47, 92%]). This was perhaps due to the latter group's anticipation of receiving active intervention immediately thereafter, as per provision of ethical approval. Reasons for withdrawal during treatment are summarized (where known) in Figure 1. There were no significant differences between treatment completers (defined as completing ≥ 4 sessions) and those who dropped out, on any variable. No harm-related or serious adverse events were reported.

Participants in the CBT group took an average of 50 days to complete the course compared with 48 days in IRT, with TAU participants typically taking one further week (58 days). Participants were generally adherent in completion of diaries. The study generated 11,278 daily diary records, of which only 239 (2.2%) had to be estimated by participants, upon prompting by The Prof, at the weekly progress review point.

Baseline Characteristics

Baseline data indicated current insomnia in the severe clinical range, with an average total wake time (SOL + WASO) of 136.5 min (SD 72.5) and mean self-reported SE of 61.3% (SD 16.2) for the sample as a whole (Table 3). Average estimated TST was 5.09 h (SD 1.47). On the SCI, the overall mean score of 2.98 (SD 1.04) was > 4 SD below the mean for good sleepers, based on our UK sample.⁵⁵ Consistent with diagnostic criteria, substantial impact was observed on daytime performance and social functioning. In descending rank order of mean (SD) impact at baseline, insomnia had a negative effect on energy (2.71 [0.80]), mood (2.50 [0.93]), concentration (2.40 [0.92]), productivity (2.12 [0.92]), relationships (1.72 [1.06]), and staying awake (1.28 [1.00]). There was modest symptomatology on the DASS, consistent with selection criteria, with stress scores significantly higher than depressive (7.80 [3.70] vs 5.05 [3.01], t₁₆₃ = 11.1, P < 0.001) or anxiety (2.70 [2.20], t₁₆₃ = 21.0, P < 0.001) scores, and depressive scores higher than anxiety scores (t₁₆₃ = 11.5, P < 0.001).

Table 3.

Treatment outcomes for sleep and daytime measures. Baseline, post-treatment, and follow-up data (actual mean [SE]) are presented for each group along with change scores (95% CI) and within group effect sizes (Cohen's d)

One-way ANOVA revealed differences in pre-treatment scores for SOL, SE, and TST (all P < 0.01), in each case accounted for by the TAU group having more symptomatic scores (see Table 3). Consequently, baseline values were introduced conservatively as covariates in subsequent hypothesis testing on these variables.

Impact of Treatment on Self-Reported Sleep

Summary sleep diary data comprising pre-treatment, post-treatment and follow-up actual mean (SE) values for each group are presented in Table 3. Change scores (with 95% CI) and within group effect sizes [ES: (M₁−M₂/δ_pooled)] are also provided. ES were regarded as large (d = 0.8), moderate (d = 0.5), or small (d = 0.3), consistent with recognized definitions.⁷³ In Table 4, relative ES, representing changes over baseline observed at post-treatment and follow-up, are provided for each comparison (CBT-TAU, IRT-TAU, CBT-IRT). For all variables, significant effects in favor of CBT were observed, and these remained significant when taking account of baseline values.

Table 4.

Relative effect sizes (Cohen's d) for each treatment group comparison (CBT-TAU, IRT-TAU, CBT-IRT) at post-treatment and follow-up for sleep and daytime variables

CBT was associated with an absolute post-therapy increase of 19.5% (95%CI, 15.3 to 23.7) in SE (a 30.8% increase over baseline), compared with a 5.7% (95%CI, 2.79 to 8.52) gain following IRT, and 6.4% (95%CI, 2.88 to 9.86) in TAU (Table 3). A near 20% level of improvement was sustained in the CBT group at follow-up (95%CI, 15.7 to 23.6), compared with 7% (95%CI, 4.53 to 10.1) and 9% (95%CI, 4.89 to 13.7) gains in IRT and TAU. The mixed effects model confirmed a main effect for time (F_2,151 = 92.54, P < 0.0001) and a significant treatment × time interaction (F_4,304 = 15.97, P < 0.0001), with between-group comparisons favoring CBT at post-treatment relative to both TAU (d = 0.95) and IRT (d = 1.06), each representing large ES (Table 4). At follow-up, CBT again yielded superior outcome relative to IRT (d = 1.00) and TAU (d = 0.69).

Substantial reductions in SOL and WASO were observed in the CBT group, of around 26 min and 48 min, respectively, at both post-treatment and follow-up (see Table 3 for detailed data). By comparison, a more modest (20 min) but sustained reduction in WASO (only) was observed following IRT. TAU participants reduced their SOL by around 10 min. Mixed model analysis supported the superiority of CBT relative to TAU and IRT across time points for both SOL (F_4,304 = 4.55, P < 0.001) and WASO (F_4,306 = 8.53, P < 0.0001). At both time points for WASO, CBT exhibited a large ES relative to TAU (d = −0.77), and a moderate to large ES relative to IRT (d = −0.41). For SOL, there was a large effect in favor of CBT relative to IRT (d = −0.86) and a modest ES relative to TAU (d = −0.45). At follow-up, IRT was associated with lower WASO than TAU (d = −0.41). To further quantify these medium term improvements in sleep continuity, TWT reduced by some 75 min (95%CI, −56.8 to −92.7 min) following CBT, exhibiting large ES compared with either IRT (d = −0.98: 24 min; 95%CI, −9.61 to −30.6) or TAU (d = −0.81: 9 min; 95%CI, −3.16 to −32.8) (F_4,306 = 9.56, P < 0.001).

Mixed models analysis also revealed significant interaction effects on TST (F_4,304 = 2.81, P = 0.026). At post-treatment, TST was increased by approximately 40 min in both CBT and TAU compared with 20 min in IRT. However, by follow-up, TST had increased by 70 min in the CBT group compared with 28 min (d = 0.73) and 47 min (d = 0.32) in IRT and TAU, respectively. Self-reported sleep quality also increased to a greater extent in CBT than in either IRT or TAU (F_4,306 = 4.06, P = 0.003), with the latter comparison representing a moderate-large effect both at post-treatment and follow-up (d = 0.70). Applying a Bonferroni correction to maintain the 0.05 error rate across all sleep diary variables (adjusted P < 0.01) would result in the TST main effects (only) failing to attain statistical significance. It should be noted that, as for the primary outcome of SE, time main effects were observed for SOL, WASO, TST, and sleep quality (all P < 0.001) in addition to the interaction terms reported above.

Impact of Treatment on Daytime Functioning

Comparative data on daytime outcomes are presented in Table 3; inspection of which indicate that there was a main effect of time for daytime performance (F_2,151 = 63.47, P < 0.0001) and for social functioning: (F_2,151 = 91.98, P < 0.0001). Visual impression suggests that both the CBT and the IRT groups improved relative to TAU, and this was confirmed by interaction effects for daytime performance (F_4,316 = 5.73, P < 0.001) and social functioning (F_4,316 = 3.78, P = 0.005). In relation to daytime performance a moderate-large effect in favor of CBT had developed by post-treatment for the CBT-TAU (d = −0.72) comparison (Table 4), and this was consolidated by follow-up (d = −0.85). However, a moderate-large effect was also evident for IRT relative to TAU by this point (d = −0.40 to −0.72). The ES for the CBT-IRT contrast, therefore, is important and reveals a small additional ES benefit favoring CBT (d = −0.23 to −0.32). A similar pattern of results was obtained with the social functioning data (Table 4). DASS total score data also suggest some generalized impact of CBT on participants' (mild) symptoms of psychopathology, with small effects for the CBT-IRT comparison observed at post-treatment (d = −0.33) and follow-up (d = −0.28).

Clinical Effects of Treatment

The SCI exhibited > 2-fold sustained improvement following CBT, represented by large CBT-TAU effects at post-treatment (d = 1.20) and follow-up (d = 1.11: Table 4). The CBT-IRT comparison also yielded large ES at both time points (d = 0.65 and d = 0.77), and placebo demonstrated a small effect (d = 0.34) relative to TAU (F_4,316 = 12.22, P < 0.0001). In terms of clinically relevant change associated with CBT, mean SCI score at post-treatment and follow-up was higher than our suggested threshold score of 6.0 used to identify normal sleepers on this measure.⁵⁵

To be eligible for the study, all participants had to have initial SE < 80%. Therefore, it was of interest to determine the proportion within each group who exceeded this value (SE ≥ 80%) following intervention. These data are presented in Figure 2, along with comparisons on the more stringent clinical cutoff scores of SE ≥ 85% and SE ≥ 90%. Three-quarters of participants allocated to CBT completed the course with SE ≥ 80%, compared with less than one-third of those in IRT, and one in 5 of the TAU group (χ²(2) = 33.0, P < 0.001). Likewise, 55% of CBT participants achieved a SE of 85% (χ²(2) = 23.8, P < 0.001), and approaching 40% achieved SE ≥ 90% (χ²(2) = 13.4, P = 0.001). These advantages of CBT over placebo and the passage of time alone, all represent large ES (w = 0.86, w = 0.72, and w = 0.51 for the 80%, 85%, and 90% criteria, respectively), where w is the square root of the standardized χ² statistic (ES conventions: w = 0.10 [small], 0.30 [medium], 0.50 [large]).⁷³ At follow-up, large effects were maintained for the 80% (w = 0.81) and 85% criteria (w = 0.62), with a medium effect observed for the 90% criterion (w = 0.38).

Figure 2.

Percentage of patients within each treatment arm achieving sleep efficiency (SE) clinical end-points.

Finally, we wish to report that 38 IRT participants (79% of IRT completers) and 39 TAU participants (83% of those who provided post-treatment data) created CBT user accounts subsequent to completion of the trial period.

DISCUSSION

We compared CBT for insomnia, delivered via an automated media-rich web application, with a similarly delivered, credible placebo (IRT), and a waitlist TAU control group.

On our primary endpoint of SE, large pre- to post-treatment effect sizes were observed for CBT relative to IRT and TAU. These effects remained robust at follow-up. SOL was reduced by 56% (compared to 5% and 17% in IRT and TAU), and WASO was reduced by 63% (compared to 29% in IRT and a 4% increase in TAU). Global sleep-wake function, assessed by the Sleep Condition Indicator, similarly favored CBT, and clinical significance of findings was confirmed by the proportion of patients achieving SE values > 85% (post-treatment: 55% [CBT] v. 17.3% [IRT] v. 7.8% [TAU]; follow-up: 42% [CBT] v. 15% [IRT] v. 3.9% [TAU]), as well as improvements in daytime functioning. Moreover, the mean sleep parameter scores for CBT (Table 3) were within normative values (i.e., SOL < 30 min, WASO < 30 min, TST > 6 h,⁷⁶ and SCI > 6)⁷⁶ at follow-up.

Such outcomes appear comparable in magnitude to therapist-delivered CBT²⁷ and greater than the majority of online CBT studies.^44,45,47,49 Our findings are most similar to those of Ritterband et al.,^46,48 suggesting that there may be benefits associated with the design and delivery of online CBT, such as engaging animations and graphics and reminder prompts. Importantly, we also had comparatively low attrition rates (12% to 20%), in contrast with some other studies, where dropout rates have been as high as 33% pre-to post treatment,⁴⁷ and up to 49% pre-treatment to follow-up.⁴⁹

Thus, CBT delivered using advanced web-based tools, and tested within a placebo-controlled design, had a positive and durable impact. It should be noted that IRT placebo relative to TAU, did also show some positive effects, particularly in reducing WASO, and in achieving SE endpoints for around 15% of participants. Such findings help to confirm that there was a placebo effect for at least a proportion of participants, on some outcomes. It should also be borne in mind that usual care continued in all groups, consistent with a real-world trial. Although we cannot be certain of the effects of such uncontrolled factors, it seems unlikely that usual care would systematically differ across our groups.

In contrast to substantial improvements in quantitative estimates of sleep, we observed more modest improvements on ratings of sleep quality. In the CBT group, mean scores increased by around 15 points (on a 100-point scale). Although statistically greater than IRT or TAU, the degree of absolute change and the final endpoint seem low. We cannot readily explain this, given (a) the global improvement observed on the SCI and the generalized benefits to daytime function observed with CBT, and (b) the literature that suggests that sleep quality can be more amenable than sleep pattern to improvement with CBT.^26,27 One possibility is that our dimensional, bipolar measure of sleep (“very unsatisfactory” to “very satisfactory”) was not sufficiently sensitive. People tended to use mainly the central area of the scale, and we did not provide definition of intermediate points. Also the sleep quality rating was the first item on our diary, when it is more usual to be near the end.⁷⁷

We did not investigate the association between treatment and use of medication. Indeed, our website specifically advised people not to adjust medication without consulting their doctor. Twenty percent reported taking hypnotics at baseline (Table 2) and slightly fewer (16%) reported using them at post-treatment, but this appeared unrelated to group allocation. Further research is required to consider how, if at all, online CBT may be used as an alternative to prescription medication.

We would suggest that our design was particularly rigorous, providing the first placebo-controlled evidence that online CBT for insomnia can be clinically effective. We have demonstrated that CBT effects are not merely associated with user engagement on an attractive programme, or with the demand characteristics and expectations of benefit associated with receiving treatment. IRT was considered a credible treatment by participants, reflected in low levels of attrition, faithful recording of sleep diary information, and good session completion rates. Furthermore, we included a comprehensive assessment of daytime outcomes, based on proposed revised DSM-5 criteria and research recommendations.⁷⁸ We would also suggest that there were methodological advantages associated with our technology. For example, treatment fidelity was likely enhanced because The Prof's interactions and recommendations, though highly tailored, were all pre-programmed. Indeed, standardization of protocols in online CBT may offer quality assurance that is superior to training therapists to consistently follow a manual. Technology offers greatly improved precision of measuring adherence (e.g., time stamps of page views, entries, and interactions).

In addition to demonstrating reliable interaction effects, our data also reveal main effects of time (independent of group allocation). In this respect it should be noted that all our participants started the trial in February 2011, completing in May/June 2011. One explanation of this finding may be an underlying seasonal improvement in sleep (or a reduced concern about insomnia) from late winter through to early summer. This requires further systematic study because, in most RCTs, participants are recruited sequentially, often over months or years. Thus any seasonal effect is likely to be randomly represented in the data. Of course, the time main effect may simply relate to spontaneous improvement over the study period. We cannot differentiate these possibilities at this stage. The feasibility of simultaneously commencing entire cohorts online affords potential advantages, and disadvantages. In relation to the former, online recruitment and in-parallel processing of many participants may permit highly efficient use of research resources. In terms of disadvantages, gathering a cohort for the purposes of research may not reflect the real world, where patients want to start their treatment whenever they feel ready to do so. A large scale, open trial of online CBT, with participants enrolling at the time of signing up to the site, would be welcome to address this point.

This study has a number of important limitations. Subjects were recruited by online survey and may represent a cohort unusually interested in addressing sleep problems. They certainly all had access to, and competencies in, using the internet, thus restricting the sample targeted from that of the wider population with insomnia. We also did not, for example, include polysomnography in our design, and therefore, we are unable to rule out occult sleep disorder pathology. Further work evaluating CBT with respect to objective sleep outcomes would be valuable. We also acknowledge that our selection of SE as the primary endpoint could have unduly favored CBT because the sleep restriction component of CBT can lead to improved SE, in the absence of other evidence. Whereas our outcome data across other sleep variables do corroborate significant sleep pattern gains, we would advocate using SE as part of a group of sleep endpoints in the future.

In setting the inclusion criterion of ≤ 79% for baseline SE on the sleep diary, we attempted to ensure that our participants had a current, prospectively monitored problem with sleep upon entry to the trial. In so doing, however, we recognize that we excluded those with better sleep efficiency, who may nonetheless have benefitted from CBT. In this regard, and also in our exclusion of those who reported being in either “poor” or “very poor” physical or mental health, our study departed from a real-world evaluation and limiting the generalizability of our findings. Future investigations with patients with active comorbidities, who represent the majority of patients in clinical practice, are required. Finally, we acknowledge that our follow-up period, though experimentally controlled, was relatively short. Most face-to-face CBT-I studies have demonstrated maintenance of gains between 6 and 12 months post-treatment, with some showing durability up to 2 years.²⁷ Future work should assess the stability of gains over longer periods.

In keeping with a real world framework for online CBT, we wanted to have minimal contact with participants. Communication, therefore, was by email and questionnaire completion, with no face-to-face contact during the trial. We feel that demonstrating robust effects in the absence of formal contact strengthens the ecological validity of the study as well as the applicability of the approach. Contacts between the intervention system and the participants (e.g., text reminders from The Prof) and among the participants (the social community) on the other hand were integral to the program. Of course, the community here was necessarily limited, to the 53 CBT trial participants. Nevertheless, 37 (70%) posted comments on the site, indicating that this element of the program was valued, and it should be borne in mind that social networks generally increase in perceived value as they expand in scale. We do, however, recognize that it will be important to assess the specific impact of personal tailoring and community support in further online intervention studies. Consistent with the stepped care approach,³³ such work should bear in mind that personal preference is likely to play a role in motivation and adherence, such that some people may prefer to have personal support (regardless of whether or not they actually would “need” it). Indeed, there is recent evidence that brief behavioral intervention, involving only two in-person contacts, can be very effective.⁷⁹ There would be value in comparing efficacy, preference, and satisfaction between such minimal contact models and online CBT.

In conclusion, CBT delivered using an online media-rich web application with automated support and a community forum appears effective in improving the sleep and associated daytime functioning of adults with insomnia disorder. Further work is required to evaluate the objective changes associated with treatment delivered in this way. Treatment trials of insomnia associated with complex clinical presentations and associated with physical and/or mental health problems are also needed to establish any necessary pre-screening requirements for access to online, compared with, in-person CBT.

DISCLOSURE STATEMENT

This was not an industry supported study. The software and web development for this study was supported by Sleepio Limited. Dr. Espie is Clinical and Scientific Director of Sleepio Limited and a shareholder, but has not received any income from the company. He has also participated in speaking engagements and has served as a consultant for Boots Pharmaceuticals. He has also received free use of actigraphs by Philips Respironics. Dr. Hames is Managing Director and CEO of Sleepio Limited and has received a salary from the company. Dr. Williams has written workbook and online modules addressing sleep problems and is the Director of Five Areas Limited a company that provides access and training to CBT resources. The other authors have indicated no financial conflicts of interest.

SUPPLEMENTAL MATERIAL

CONSORT 2010 checklist of information to include when reporting a randomized trial*