Feasibility of a UK community-based, eTherapy mental health service in Greater Manchester: repeated-measures and between-groups study of ‘Living Life to the Full Interactive’, ‘Sleepio’ and ‘Breaking Free Online’ at ‘Self Help Services’

Abstract

Objectives

There is increasing evidence to support the effectiveness of eTherapies for mental health, although limited data have been reported from community-based services. Therefore, this service evaluation reports on feasibility and outcomes from an eTherapy mental health service.

Setting

‘Self Help Services’, an Increasing Access to Psychological Therapies (IAPT) eTherapy service in Greater Manchester.

Participants

1068 service users referred to the service for secondary care for their mental health difficulties.

Interventions

Participants were triaged into one of three eTherapy programmes: ‘Living Life to the Full Interactive’ for low mood, stress and anxiety; ‘Sleepio’ for insomnia; and ‘Breaking Free Online’ for substance misuse, depending on clinical need.

Primary outcomes measures

Standardised psychometric assessments of depression, anxiety and social functioning, collected as part of the IAPT Minimum Data Set, were conducted at baseline and post-treatment.

Results

Data indicated baseline differences, with the Breaking Free Online group having higher scores for depression and anxiety than the Living Life to the Full Interactive (depression CI 1.27 to 3.21, p<0.0001; anxiety CI 077 to 1.72, p<0.0001) and Sleepio (depression CI 1.19 to 4.52, p<0.0001; anxiety CI 2.16 to 5.23, p<0.0001) groups. Promising improvements in mental health scores were found within all three groups (all p<0.0001), as were significant reductions in numbers of service users reaching clinical threshold scores for mental health difficulties (p<0.0001). Number of days of engagement was not related to change from baseline for the Living Life to the Full or Sleepio programmes but was associated with degree of change for Breaking Free Online.

Conclusion

Data presented provide evidence for feasibility of this eTherapy delivery model in supporting service users with a range of mental health difficulties and suggest that eTherapies may be a useful addition to treatment offering in community-based services.

Strengths and limitations of this study.

• Large sample size

• Data from community-based mental health service and service users

• High ecological validity

• Examines feasibility of a novel treatment modality within a community-based mental health service

• Outcomes have implications for reducing waiting list times for mental health services

• Unequal sample sizes across three eTherapy groups

• Lack of control/comparison group

• Lack of randomisation to treatment arms

• Lack of longer-term follow-up data

Introduction

For individuals in the UK with mental health difficulties, waiting list times remain a significant barrier to accessing psychosocial support.¹ Data indicate that since the 2008 recession, funding for mental health provision in some regions of the UK has been cut by as much as 32%,² a trend that has occurred alongside increasing prevalence of common mental health difficulties such as anxiety and depression³ and increased demand for mental health service.⁴ Despite the introduction of the Increasing Access to Psychological Therapies (IAPT) programme in England, which was intended to alleviate pressures on primary mental health services,⁵ as many as 1 in 10 patients may still be waiting for over a year for face-to-face psychosocial treatment.⁶ The IAPT approach is designed to widen access to lower-intensity interventions for mild to moderate depression and anxiety, which are delivered by specially trained Psychological Wellbeing Practitioners (PWPs). These PWPs work alongside higher-intensity mental health professionals such as high-intensity therapists and Clinical Psychologists, who provide support to individuals with more complex needs.⁷ Although the IAPT approach has reduced waiting list times within primary mental health services, IAPT services themselves are now facing significant oversubscription.⁶

One possible solution to the growing issue of waiting lists in mental health services are ‘computer-assisted therapies’ (CATs) or ‘eTherapies.’ These therapies deliver evidence-based psychosocial interventions and behavioural change techniques through digital technologies such as web and mobile applications. Such interventions have the potential to deliver highly individualised treatment, by tailoring intervention content to the specific needs of the individual, and there is now a growing literature demonstrating the clinical and cost-effectiveness of tailorable eTherapy programmes for the treatment of a wide range of health difficulties.^{8 9}

Three examples of such eTherapy programmes are ‘Living Life to the Full Interactive’ (www.llttf.com/),^{10 11} a programme for low mood, stress and anxiety, and depression associated with physical health problems; ‘Sleepio’ (www.sleepio.com),^12–15 a sleep improvement programme; and ‘Breaking Free Online’ (www.breakingfreeonline.com),^16–20 a programme that helps people overcome substance misuse difficulties. All three of these programmes have been delivered to service users via an innovative eTherapy service in Greater Manchester, ‘Self Help Services’, a service-user-led mental health charity that provides primary care mental health services across the north of England. A number of National Health Service (NHS) Trusts have commissioned services provided by the charity as part of the IAPT initiative, and therefore, this study reports on the feasibility of the delivery of these three eTherapies by Self Help Services in community-based mental health treatment settings, via the use of clinical outcomes data collected for service evaluation purposes in the IAPT Minimum Data Set.

Advantages and disadvantages of delivering eTherapies for mental health

There are a number of advantages to providing eTherapies, such as significantly reducing waiting times within healthcare services and being more cost-effective than one-to-one therapy, given that multiple users can access an eTherapy simultaneously.^21–23 Providing interventions as eTherapy also ensures optimal treatment fidelity as therapeutic techniques are delivered using a computer in a highly standardised manner,^{21 24} without the human-related variance in delivery often seen in traditional human-facilitated interventions.^{25 26} However, despite these advantages of delivering eTherapies, there are still some limitations with the approach. Some studies have demonstrated adherence to be low, with numbers of service users dropping out of treatment being high.^27–29 Additionally, when such interventions are provided with minimal or no practitioner support, there is little opportunity for a positive therapeutic alliance to be built between the service user and and a practitioner, which may reduce effectiveness.^{30 31}

The introduction of digital health interventions such as eTherapies into existing healthcare systems is also often perceived as ‘disruptive’, meaning it can take considerable time for such innovations to be implemented and incorporated into standard practice.^32–34 Additionally, research has demonstrated that information technology infrastructure within NHS mental health services may be inadequate for the effective delivery of eTherapies.^{35 36} Nevertheless, a recent report by the King’s Fund highlighted the potential of eTherapies as part of effective practice,³⁷ with some eTherapy programmes having been demonstrated to be tailorable to the need of the individual and clinically effective, with the National Institute for Health and Care Excellence now recommending such approaches for anxiety and depression.³⁸ This is due to the fact that despite difficulties with implementation, some eTherapies have crossed the divide between research-based innovation and implementation in clinical settings, including those eTherapies delivered at Self Help Services, namely Living Life to the Full Interactive, Sleepio and Breaking Free Online.

Mental health eTherapy provision at Self Help Services

Living Life to the Full Interactive^{10 11} is a licensed eTherapy programme demonstrated as being effective in helping people cope with low mood, stress and anxiety via the inclusion of techniques informed by cognitive behaviour therapy (CBT) principles.^{39 40} The programme is an online interactive self-help skills package, comprising a number of modules covering areas of life and well-being commonly affected by low mood and stress. It uses both interactive text and video formats to accommodate different learning styles and provides psychoeducation alongside more practical CBT techniques such as relaxation and guidance on how to make life changes.

Sleepio is an online sleep improvement programme demonstrated as being effective in helping people with insomnia,^12–15 which can be used as a self-help programme. It comprises intervention techniques informed by CBT principles and provides users with 6 weeks of access to tailored clinical content, and 12 weeks of support from an online community. The programme includes 10 online tools and a ‘library’ of articles written by sleep experts, in addition to a personal 12-week sleep diary. Users are encouraged to log on once a week for a personalised 20 min session with an avatar that guides them through a personalised programme.

Breaking Free Online is an online treatment programme that has a growing evidence base to support its effectiveness in helping people overcome difficulties with alcohol and drugs,^16–20 which can be delivered as CAT or self-help, and targets 39 different substances, including alcohol. The programme provides multiple interactive psychosocial interventions, drawing on CBT, mindfulness and relapse prevention techniques, via a six-domain biopsychosocial model, the Lifestyle Balance Model,⁴¹ which conceptualises various aspects of functioning associated with substance misuse and comorbid mental health difficulties.

Although the three eTherapy programmes provided by Self Help Services each have different primary clinical targets, they all contain cognitive-behavioural interventions that are likely to be generally helpful to individuals in addressing underlying anxiogenic or depressogenic thinking and unhelpful behaviour patterns. For example, Living Life to the Full Interactive was developed specifically for individuals experiencing low mood, stress and anxiety but contains clinical techniques that could also be effective for addressing associated issues such as poor sleep. Additionally, previous research conducted at Self Help Services with individuals using Breaking Free Online for their substance misuse demonstrated significant improvements in depression (r=0.59), anxiety (r=0.63) and general social functioning (r=0.68).¹⁶ Additionally, outcomes studies of Sleepio indicate that as well as resulting in improved sleep for users, the programme also facilitates general improvements in mental health (d=−0.33)⁴² and workplace functioning (d=0.77).¹²

Aims of the study

Over the past 4 years, Living Life to the Full Interactive, Sleepio and Breaking Free Online have been delivered to service users via the novel eTherapy delivery model developed by Self Help Services, which has provided an additional, digital treatment modality within the Greater Manchester IAPT service provision. Therefore, this service evaluation sought to explore feasibility and outcomes of the Self Help Services eTherapy delivery model and its potential to provide a useful addition to traditional IAPT treatment offerings. This is done via examination of psychosocial outcomes for service users engaging with each of the three eTherapy programmes using the IAPT Minimum Data Set, which is intended to facilitate service evaluation and development, and measures depression, anxiety and social functioning.

Guidance from the Medical Research Council (MRC) recommends that alongside examining clinical effectiveness of complex interventions via randomised controlled trials (RCTs), feasibility studies allow examination of acceptability of a new intervention, service user compliance and different delivery approaches. These are all important considerations that can impact on recruitment and retention of service users and, ultimately, clinical outcomes.^{43 44} Additionally, the MRC framework recommends that feasibility and piloting work be conducted, both within research and community-based treatment delivery settings, in order to contribute to further development of clinical content of such interventions, and development of appropriate delivery models within the healthcare system. In this way, this service evaluation takes a pragmatic approach by examining feasibility of delivering eTherapies in a community mental health service, using clinical outcomes data from service users, as opposed to data collected within the highly controlled context of a research study such as a clinical trial, where ecological validity may be low.^{45 46}

Method

Design

This study had a pre-test post-test design, using standardised psychometric assessments from the IAPT Minimum Data Set, to examine mental health and social functioning outcomes, in three separate groups of service users accessing different eTherapy treatment programmes in a community-based mental health service.

Participants

Participants were 1068 service users receiving support for a range of mental-health-related issues from Self Help Services, an eTherapy service, between 2011 and 2015. Inclusion criteria included any service user over the age of 18 years accessing one of the three eTherapy programmes provided by Self Help Services (Living Life to the Full Interactive, Sleepio or Breaking Free Online), who had completed their eTherapy treatment period, provided post-treatment assessment data and consented for their anonymised data to be used for service evaluation purposes at the start of their treatment. Self Help Services provides services across Greater Manchester and some areas of Liverpool, including self-help and peer support groups, face-to-face counselling and a mental health crisis centre, alongside access to a eTherapies.

Services users had either been referred to Self Help Services by a healthcare practitioner or self-referred. On entering the service, they completed an initial assessment comprising a battery of standardised psychometric assessments, which forms the IAPT Minimum Data Set, and a consultation with a Self Help Services practitioner to establish their principal area of difficulty: depression, anxiety and/or stress, sleep disruption or problems with alcohol or drugs. These initial assessments were conducted either face to face or via telephone depending on the service users’ preference. Practitioners were all trained to provide guidance and support to service users using the eTherapy programmes, with some practitioners also having lived experience of mental health difficulties. Following a collaborative discussion between the service user and practitioner, service users were triaged into the most appropriate of the three eTherapies and supported in setting up an account on the relevant programme.

Procedure

Service users entered the service and were evaluated as above during a 4-year period between 2011 and 2015. Primary outcome measures came from the Minimum Data Set of standardised psychometric assessments completed in NHS IAPT services throughout England to facilitate service evaluation, and included:

1. The Patient Health Questionnaire (PHQ-9)⁴⁷: This nine-item scale measures levels of depression and contains validated clinical norms, with a possible score range of 0–27. Internal reliability of the PHQ-9 has been found to be excellent (α=0.89), with test–retest reliability also being excellent (r=0.84). Score ranges for severity of depression are: 0–4, ‘minimal’; 5–9, ‘mild’; 10–14, ‘moderate’; 15–19, ‘moderately severe’; 20–27 ‘severe.’

2. The General Anxiety Disorder Scale (GAD-7)⁴⁸: This seven-item scale measures levels of anxiety and also contains validated clinical norms and has a possible score range of 0–21. Factor analyses revealed the GAD-7 to have a one-dimensional factor structure with item factor loadings ranging between 0.69 and 0.81, with internal consistency being excellent (α=0.92). Score ranges for severity of anxiety are: 0–4, ‘minimal’; 5–9, ‘mild’; 10–14, ‘moderate’; 15–21, ‘severe.’

3. The Work and Social Adjustment Scale (WASA)⁴⁹: This five-item scale measures levels of social impairment and has a possible score range of 0–40. Cronbach's α measure of internal scale consistency ranged from 0.70 to 0.94, with test–retest correlation being 0.73.

This baseline assessment was completed with a total of 1787 service users. Once the baseline assessment had been completed, service users were triaged into the most appropriate eTherapy programme and provided with full access. They were then followed up with a telephone call once a week from the service coordinator and were offered the opportunity to come into the service for face-to-face support if required. Then, following a period of engagement with the eTherapy programme each service user was triaged into, each service user was contacted to arrange a time to complete the last treatment session, during which the same measures were completed a second time as part of a post-treatment assessment. Of the 1786 service users completing the baseline assessment, a total of 1068 (59.8%) started treatment and provided post-treatment assessment data, with 719 (40.2%) completing the baseline assessment but not completing treatment and providing post-treatment assessment. Reasons for service users not completing treatment and providing post-treatment data were: 216 (38.5%) disengaged/dropped out from the service following baseline assessment, 251 (34.9%) were discharged from the service following baseline assessment because treatment was not required; and 252 (35%) were referred to another service, for example, higher-intensity IAPT or non-IAPT services. Comparisons between those service users providing post-treatment assessment data and those who did not, indicated no significant baseline differences between the groups in terms of psychometric assessment scores (PHQ-9, p=0.234; GAD-7, p=0.061; WASA, p=0.54).

Data analyses

Shapiro-Wilk tests revealed data from the main outcomes measures (PHQ-9, GAD-7, WASA), both at baseline and post-treatment, to be non-normally distributed (all p<0.05); therefore non-parametric tests were run to analyse data. Two main sets of analyses were conducted. Kruskal-Wallis analyses of variance (ANOVA) were conducted to examine baseline differences between the three eTherapy groups on the mental health and social functioning assessment contained within the IAPT Minimum Data Set. Analyses of covariance (ANCOVA) were also conducted to examine whether eTherapy group assignment was predictive of the extent to which scores for post-treatment mental health and social functioning differed from baseline scores, that is, the degree of change in functioning, whilst controlling for the participant characteristics of age and gender. Additionally, separate within-group, repeated-measures Wilcoxon signed-rank ANOVA tests were conducted to examine changes in the same psychometric outcomes from baseline to post-treatment assessment within each of the three eTherapy groups. Pearson’s effect sizes (r) were calculated using test statistics from ANOVA and ANCOVA tests run in SPSS (Z) and sample size (n) using the following formula⁵⁰:

$${\displaystyle r=\frac{Z}{\sqrt{n}}}$$

Results

Clinical outcomes data from a total of 1068 service users from the Self Help Services eTherapy service were included in the analyses, with 866 (81%) having accessed Living Life to the Full Interactive, 85 (8%) having accessed Sleepio and 117 (11%) having accessed Breaking Free Online. Across the entire sample, 679 (63.6%) were female; by group, 572 (66.1%) of those allocated to Living Life to the Full Interactive were female, compared with 60 (70.6%) to Sleepio and 47 (40.2%) of Breaking Free Online users. Across the whole sample, mean age was 37.38 years (range 16–79 years; SD, 11.98), with a mean age of 36.11 years (range 16–73 years; SD, 11.51) for Living Life to the Full Interactive users, 45.21 years (range 20–79 years; SD, 15.04) for Sleepio users, and 41.21 years (range 19–60 years; SD, 9.80) for Breaking Free Online users.

Time periods of engagement with the service varied, with some service users engaging in treatment for longer periods than others, depending on their need. Engagement periods for the whole sample ranged from 4 days to 288 days (0.64–41.14 weeks), with a median engagement treatment period of 62.39 days (IQR=40.18). For each individual eTherapy group, engagement periods were as follows: for Living Life to the Full Interactive users, 4–288 days (0.64–41.14 weeks) with a median of 66.29 days (IQR=43.06); for Sleepio users, 29–148 days (4.19–21.08 weeks) with a median of 66.35 days (IQR=39.06); and for Breaking Free Online users, 6–205 days (0.92–29.35 weeks) with a median of 58.29 days (IQR=48.64). At the end of each service user’s period of engagement with the service, the same battery of assessment measures was completed at point of discharge, which was between 4 days and 288 days following the baseline assessment.

When the three eTherapy groups were compared using Kruskal-Wallis ANOVAs on their psychometric assessment scores for mental health and social functioning at baseline, some significant differences between the groups were found (table 1). The Breaking Free Online group was found to have significantly higher scores on the PHQ-9 assessment for baseline depression (Breaking Free Online mean=14.20) than the other two eTherapy groups (Living Life to the Full Interactive mean=11.89, CI 1.27 to 3.21, p<0.0001; Sleepio mean=11.34; CI 1.19 to 4.52, p<0.0001). The Breaking Free Online group was also found to have significantly higher scores at baseline on the GAD-7 assessment for anxiety than the other two eTherapy groups (Breaking Free Online mean=12.18; Living Life to the Full Interactive mean=11.32, CI. 077 to 1.72, p<0.0001; Sleepio mean=8.49, CI 2.16 to 5.23, p<0.0001).

Table 1.

Comparison of baseline psychometric outcomes for the three eTherapy programmes

	Breaking Free Online mean (SD)	Living Life to the Full Interactive mean (SD)	Sleepio mean (SD)	Z	p Value	r
PHQ-9 baseline	14.20 (6.43)	11.89 (4.77)	11.34 (5.12)	12.00	<0.0001	0.36
GAD-7 baseline	12.18 (5.75)	11.32 (3.98)	8.49 (5.00)	9.80	<0.0001	0.31
WASAS baseline	16.84 (9.74)	16.14 (7.91)	15.65 (9.12)	0.56	0.569	0.00

In addition to comparing the three eTherapy groups at baseline, degrees of psychometric score change from baseline to post-treatment were also compared across the three eTherapy groups, with baseline scores on each of the psychometric assessments being regressed against post-treatment scores, while controlling for age and gender. ANCOVAs revealed no significant differences between the groups in terms of degree of change in any of the baseline psychometric assessments including the following: PHQ-9 F=2.373, p=0.094; GAD-7 F=3.239, p=0.052; WASAS F=0.164, p=0.848. Additionally, when outcomes from baseline to post-treatment were compared within each of the three eTherapy groups, using Wilcoxon signed-rank tests, significant reductions in scores on PHQ-9, GAD-7 and WASAS were found within all three groups, with p values across all three eTherapy groups being <0.0001. In addition to conducting these Wilcoxon analyses, effect sizes were calculated. In all three eTherapy groups and for each of the psychometric assessments, effect sizes were in the moderate to large range. See table 2 for full details of these within-group analyses.

Table 2.

Within-group comparison for each of the three eTherapy groups on baseline and post-treatment psychometrics assessment scores

		Z	p Value	r
PHQ-9	Breaking Free Online	−6.771	<0.0001	0.63
	Living Life to the Full Interactive	−21.450	<0.0001	0.73
	Sleepio	−7.226	<0.0001	0.78
GAD-7	Breaking Free Online	−6.449	<0.0001	0.60
	Living Life to the Full Interactive	−21.463	<0.0001	0.73
	Sleepio	−6.365	<0.0001	0.69
WASAS	Breaking Free Online	−5.558	<0.0001	0.51
	Living Life to the Full Interactive	−13.729	<0.0001	0.47
	Sleepio	−4.967	<0.0001	0.54

Given that service users accessed each eTherapy programme for varying lengths of time (4–288 days), regression analyses were conducted to examine whether number of days engaging with each programme was associated with degree of change in depression, anxiety and social functioning from baseline to post-treatment assessment. These regression analyses revealed that length of period of engagement (in days) was not related to degree of change from baseline for service users accessing either the Living Life to the Full Interactive or Sleepio groups. However, number of days of engagement was associated with degree of change in scores for depression, anxiety and social functioning from baseline to post-treatment assessment for those service users accessing Breaking Free Online. For Breaking Free Online users, the greater the number of days of engagement with the programme, the greater the reduction in scores for depression,anxiety and social impairment (table 3).

Table 3.

Regression analyses demonstrating associations between number of days of engagement with each eTherapy programme and degree of change in psychometric scores

		F	β	p	95% CI
Breaking Free Online	PHQ-9 GAD-7 WASAS	34.387 28.396 26.648	−0.341 −0.330 −0.190	<0.0001 <0.0001 0.020	−0.214 to 0.039 −0.193 to 0.027 −0.170 to 0.176
Living Life to the Full Interactive	PHQ-9 GAD-7 WASAS	168.814 95.392 127.071	−0.038 −0.041 −0.021	0.185 0.178 0.475	−0.039 to 0.013 −0.036 to 0.016 −0.037 to 0.058
Sleepio	PHQ-9 GAD-7 WASAS	31.996 40.010 30.856	−0.130 −0.138 −0.050	0.129 0.083 0.556	−0.065 to 0.056 −0.060 to 0.047 −0.147 to 0.097

Alongside statistically significant within-group changes in scores on PHQ-9, GAD-7 and WASAS (table 2), the percentages of service users reaching clinical thresholds scores for mild, moderate and severe depression and anxiety at baseline were compared with the percentages reaching each threshold at post-treatment (see tables 4 and 5). Chi-square analyses demonstrated that within each of the three eTherapy groups, the percentages of service users reaching threshold scores for clinically relevant depression and anxiety (a score of 5≤) after treatment reduced significantly from baseline: Living Life to the Full Interactive (PHQ-9 χ²=260.30, p<0.0001; GAD-7 χ²=105.44, p<0.0001), Breaking Free Online (PHQ-9 χ²=68.77, p<0.0001; GAD-7 χ²=45.88, p<0.0001), Sleepio (PHQ-9 χ²=57.24, p<0.0001; GAD-7 χ²=44.23, p<0.0001). Specifically, the percentages of service users in the minimal and mild categories of symptom severity increased after treatment, while the percentages of service users in the categories between moderate and severe symptom severity decreased (see tables 4 and 5 for score ranges for each severity category).

Table 4.

Changes in percentages of service users reaching clinical threshold scores for depression by eTherapy group

	PHQ-9 baseline threshold (%)			PHQ-9 post-treatment threshold (%)
PHQ-9 score thresholds	Breaking Free Online	Living Life to the Full Interactive	Sleepio	Breaking Free Online	Living Life to the Full Interactive	Sleepio
Minimal (range 0–4)	7.7	6.9	10.6	37.6	40.5	42.4
Mild (range 5–9)	21.4	23.1	20	15.4	33.5	38.7
Moderate (range 10–14)	20.5	40.3	44.7	22.2	15.5	11.8
Moderately severe (range 15–19)	26.5	24.2	18.8	11.1	7.8	2.4
Severe (range 20–27)	23.9	5.5	5.9	13.7	2.7	4.7

Table 5.

Changes in percentages of service users reaching clinical threshold scores for anxiety by eTherapy group

	GAD-7 baseline threshold (%)			GAD-7 post-treatment threshold (%)
GAD-7 thresholds	Breaking Free Online	Living Life to the Full Interactive	Sleepio	Breaking Free Online	Living Life to the Full Interactive	Sleepio
Minimal (range 0–4)	11.1	3.6	27.1	38.5	43.3	58.9
Mild (range 5–9)	26.5	41.1	43.4	28.2	37.9	23.5
Moderate (range 10–14)	28.2	40.2	20	15.4	11.3	14.1
Severe (range 15–21)	34.2	15.1	10.6	17.9	7.5	3.5

Discussion

Given that eTherapy is still a relatively novel treatment modality within the UK mental health sector, this service evaluation explored the feasibility of a novel treatment delivery model developed by one of the UK’s only eTherapy mental health services: Self Help Services. This service evaluation used outcomes from community-based service users receiving treatment via three eTherapy programmes provided by Self Help Services: Living Life to the Full Interactive for low mood, stress and anxiety; Sleepio for insomnia; and Breaking Free Online for substance misuse. Statistically significant within-group reductions in scores for anxiety, depression and social impairment were demonstrated for each of the three eTherapy programmes. Additionally, regardless of eTherapy group allocation, degrees of reduction in mental health scores were comparable across each of the three eTherapy programmes, (table 1) with eTherapy group assignment not being predictive of degree of change in depression, anxiety or social functioning scores between baseline and post-treatment assessment.

Despite the equivalent outcomes across the three eTherapy groups, there were some significant differences between the groups in terms of scores on the assessment measures and hence the severity of their mental health difficulties. The Breaking Free Online group was found to have significantly higher baseline scores for depression and anxiety than the Living Life to the Full Interactive and Sleepio groups, and significantly higher scores for anxiety at post-treatment assessment. These findings may make sense, as the Breaking Free Online group may have presented with more complex difficulties than the Living Life to the Full Interactive and Sleepio groups, given the extent to which substance use can impair mental health^{51 52} and the often chaotic lifestyle that is common for individuals with drug and alcohol difficulties.^{41 53} The significantly higher post-treatment anxiety for the Breaking Free Online group may also be explained by the fact that, in the early stages of substance misuse recovery, many individuals may experience a temporary worsening of mental health symptoms^{54 55} when the anti-depressant effects of previously consumed drugs are removed, particularly in the case of opiates.⁵⁶

When clinical threshold scores for depression and anxiety were examined, there were reductions across all three groups in percentages of service users reaching threshold scores for clinically significant anxiety and depression at post-treatment assessment. These findings reinforce the statistically significant reductions in scores for mental health found across the three eTherapy groups. Moreover, it is important to note that anxiety and depression scores reduced in association with all programmes, despite these symptoms not being the principal clinical targets of the Breaking Free Online and Sleepio programmes.

The findings from this service evaluation would appear to support the feasibility and effectiveness of eTherapy programme delivery in mental health services, and would appear to support findings from previous research, including outcomes studies demonstrating effectiveness of each of the three eTherapy programmes,10 11 13 14 16 17 and findings related to delivery of eTherapies more generally. For example, eTherapies have been demonstrated as being useful additions to adult mental health services,³⁸ services for children and young people^{57 58} and also treatment provision delivered by social care workers.⁵⁹ However, some authors have advised caution around the potential of eTherapies, as many may not be grounded in psychological theory and may lack a solid evidence base.⁶⁰ Additionally, in some cases, during the development process, the challenges of implementation and uptake may not have been adequately considered, meaning that some eTherapies may not fulfil their promise of widening access to treatment.⁶¹ However, despite this caution, a recent survey of NHS mental health services showed that the provision of eTherapies is an emerging and growing trend, meaning that it is becoming increasingly important to put structures in place to ensure that only evidence-based eTherapies are commissioned and delivered.⁶²

Limitations to the study

Although the findings from this study are promising, there are a number of limitations that merit discussion. Firstly, the sample sizes across the three programmes varied, with the Living Life to the Full Interactive group (n=866) being considerably larger than the Sleepio and Breaking Free Online groups (n=85 and n=117, respectively). However, given that Self Help Services is an eTherapy service for individuals with mental health problems that are common among the general population, it is unsurprising that Living Life to the Full Interactive is accessed by more service users, as it is the only eTherapy programme of the three included in this service evaluation that is designed specifically for addressing mental health difficulties, such as low mood, stress and anxiety. Additionally, Living Life to the Full Interactive is a more established programme and has been provided in mental health services for a number of years, in contrast to Sleepio and Breaking Free Online, which are still relatively new eTherapy programmes.

Further limitations are the lack of follow-up data from participants and the fact that the study did not include randomisation and control groups. However, although methodologies such as RCTs are an important part of the development and evaluation process for complex interventions,^{14 63 64} there is now a growing literature to suggest that additional methodologies, employed alongside RCTs, may be required for evaluating complex, multi-component interventions such as eTherapies for mental-health-related conditions.^{44 65–68} When eTherapies are evaluated via RCT designs, there may be some methodological limitations due to the tailorability of these programmes, which though may enhance clinical effectiveness,^69–71 may result in within-group variation in terms of the personalised sets of intervention strategies that each user may complete.

This study was also restricted to analyses of data routinely collected for service evaluation purposes at IAPT services via the Minimum Data Set, meaning that the authors were not able to make decisions around which psychometric measures should be used. It may be that there are limitations with the psychometric properties of the measures that have been selected for inclusion in the Minimum Data Set, and therefore, other measures may be more appropriate. Additionally, some data that would have been informative were not available in the Minimum Data Set, such as around severity of insomnia for the Sleepio group and severity of substance dependence and substance consumption for the Breaking Free Online group. The Minimum Data Set also does not record whether service users have received face-to-face or telephone support during each contact with Self Help Services, and some information was not available around reasons for service users disengaging with the service or, reasons why service users engaged with each of the programme for the time periods they did. For example, it would have been informative to understand service user satisfaction with each programme, or why they stopped using each one.

An additional factor that could be viewed as a limitation is that service users engaged with the eTherapy programmes for varying lengths of time, between 4 days and 288 days. However, eTherapy programmes are designed to offer such patient-centred flexibility, with this being associated with their potential to be clinically effective.²¹ Additionally, regression analyses revealed that number of days of engagement did not appear to be associated with degree of change in scores for depression, anxiety and social impairment, from baseline to treatment assessment, for the Living Life to the Full Interactive and Sleepio users. However, there did appear to be a significant association between number of days of engagement with Breaking Free Online and degree of improvement in depression, anxiety and social impairment from baseline to post-treatment assessment. This may be explained by the fact that the Breaking Free Online group had more severe mental health problems at baseline and were being treated for a particularly challenging mental-health-related condition, that being substance misuse and comorbid mental health difficulties.

Finally, just over 40% of service users who initially engaged with the service did not provide post-treatment data, although for the majority of these service users, they were either discharged due to the service not being required or were referred to more appropriate services. Attrition is common in eTherapies^{29 72 73} with this now a focus for research in the digital health sector more generally. The problem of attrition may also be an issue in mental health interventions,^{74 75} with drop-out from psychological therapies being associated with poorer outcome for service users.⁷⁶

Implications of the findings

The data presented here demonstrate that the Self Help Services eTherapy model may have the potential to inform future mental health service delivery, given the encouraging clinical outcomes reported and the potential cost-effectiveness of such an approach. In addition, as there are now significant and lengthening waiting lists—even for IAPT services that were originally intended to reduce waiting times for mental health services^6—widening service provision to incorporate eTherapies may increase access to evidence-based psychosocial treatment for large numbers of people who could benefit from it.

PWPs, who are trained specifically to deliver lower-intensity interventions within IAPT services, may be ideally placed to incorporate the delivery of eTherapies into their current roles. This is because they are trained to conduct assessments, build a therapeutic alliance with service users, work collaboratively with them to identify areas in which they wish to see change, deliver assisted self-help and provide information about other services that may be beneficial to each individual service user’s recovery.⁷⁷ By expanding their therapeutic repertoire to include the provision of eTherapies, PWPs would be able to deliver comprehensive, evidence-based programmes that are highly standardised and not subject to the variation in fidelity of delivery that is common to more traditional psychosocial interventions,²⁵ given that all clinical content is delivered via the computer.⁷⁸

Clearly, it is important to provide access to psychosocial interventions to address the increasingly pressing issue of waiting times for mental health services and to ensure these interventions are effective and evidence based. This service evaluation has demonstrated that evidence-based eTherapy programmes can be effective, using clinical outcomes data from service users in a community-based mental health service, as opposed to data from a highly controlled study, enhancing ecological validity and generalisability of findings. Future research is planned to explore the longer-term clinical outcomes of providing eTherapies as part of an IAPT stepped-care model and potential waiting list time and cost implications of such a service.