Abstract
Background
The Wellbeing Plus Course is an internet-delivered psychological intervention for older adults with anxiety or depression.
Aims
To compare the effectiveness of the Wellbeing Plus Course in a public health setting (clinic group) with its efficacy in a randomised controlled trial (research group).
Method
Participants (n=949) were Australian adults aged 60 and above. Primary outcome measures were the Patient Health Questionnaire (PHQ-9) and Generalized Anxiety Disorder scale (GAD-7).
Results
Initial symptom severity was higher in the clinic group and course completion was lower. Both groups showed significant symptom reductions at post-treatment and were satisfied with the treatment. Results were maintained at 3-month follow-up. Within-group symptom changes were comparable between settings; there were no between-group differences on primary outcomes or satisfaction.
Conclusions
The Wellbeing Plus Course is as effective and acceptable in routine clinical care, as it is in controlled research trials.
Declaration of interest
N.T. and B.F.D developed the Wellbeing Plus Course but derived no financial benefit from it.
Copyright and usage
© The Royal College of Psychiatrists 2016. This is an open access article distributed under the terms of the Creative Commons Non-Commercial, No Derivatives (CC BY-NC-ND) license.
Symptoms of depression and anxiety are common in older adults and are associated with increased disability and decreased quality of life1,2; however, the proportion of this population accessing treatment is low, and therefore represents a significant public health problem.3,4 One way to increase treatment uptake is to provide evidence-based cognitive behavioural therapy via the internet (iCBT).5,6 Research trials conducted by the eCentreClinic, an online research unit at Macquarie University Australia, have developed and evaluated iCBT for older adults. These trials have demonstrated that the Wellbeing Plus Course, a transdiagnostic intervention designed for adults aged 60 years and above with symptoms of anxiety or low mood, is both efficacious and acceptable.7–12 Although the research outcomes are encouraging, and some studies have demonstrated successful translation of online treatment for adults in general,13–16 other studies have reported less successful implementation,17–19 and there is little published evidence of the real-world effectiveness of internet-delivered therapy specifically for adults aged 60 years and above. Implementation data are critical when translating treatments from highly controlled research settings to routine clinical settings, as lower levels of compliance and more complicated or severe presentations may be expected.17,20
As part of the Australian Government’s e-mental health strategy,21 the MindSpot Clinic was funded to provide online treatment to Australian adults with anxiety and depression, including iCBT specifically designed for adults aged 60 and above.22 Between January 2013 and June 2015, the Wellbeing Plus Course was provided to over 500 people. This paper compares outcomes from the MindSpot Clinic (‘clinic group’) with a group consisting of participants from the most recent randomised controlled trial of the course conducted by the eCentreClinic11 (‘research group’). Based on previous published research,11,22 it was expected that initial symptom severity would be higher in the clinic group, compared with the research group where more stringent inclusion criteria were used. It was hypothesised that the clinic group would show significant improvements in symptoms comparable to the results obtained in the research trial.
Method
Participants
Participants in the clinic group were consecutive patients starting the Wellbeing Plus Course at MindSpot from 1 January 2013 to 30 June 2015. A total of 516 participants were eligible for analysis. Patients were eligible if they met the following criteria: (1) identified as an Australian resident eligible for publicly funded health services, (2) were aged 60 years and above, (3) reported a principal complaint of anxiety or depression and (4) agreed for their de-identified data to be reported. Individuals who were acutely suicidal, preferred face-to-face services, were currently participating in CBT or had clinical presentations (e.g. untreated and disabling psychotic symptoms, or severe symptoms across four or more types of anxiety or depressive disorders) deemed to require face-to-face assessment were referred to suitable alternative services. Participants self-referred through the MindSpot Clinic website (www.mindspot.org.au) and completed an online assessment, providing demographic details and completing standardised self-report symptom questionnaires. These questionnaires were used in conjunction with assessment by the clinicians to determine the likely benefits of treatment for individual applicants.
Participants forming the research group (n=433) were enrolled in a randomised controlled trial conducted by the eCentreClinic to test the effect of providing varying levels of clinician contact prior to and during the Wellbeing Plus Course.11 Participants were recruited online via the eCentreClinic website (www.ecentreclinic.org). Consistent with the clinic group, participants in the research group were Australian residents aged 60 years and above who reported a principal complaint of anxiety or depression. In contrast to the clinic group, patients were excluded from the research trial if they had a total score ≥24 on the 9-item Patient Health Questionnaire (PHQ-9) or a score of 3 on the question regarding suicidal ideation. Participant flow is summarised in Fig. 1. Approval to conduct the study was provided by the Human Research Ethics Committee at Macquarie University.
Fig. 1. Participant flow.
Procedure
The Wellbeing Plus Course is a transdiagnostic iCBT intervention designed to treat core symptoms of anxiety and depression in adults aged 60 years and above. The treatment consists of five lessons and homework assignments, made available over an 8-week period. Each lesson is presented as a series of slides that include didactic text and case-enhanced learning examples, photos and images, which illustrate the principles of CBT, together with supplementary material on related topics. The case-enhanced stories detail the experiences of older adults recovering from symptoms of anxiety and depression. The content of the course lessons and related procedures have been described previously.11
Clinicians in the research unit were registered clinical psychologists with doctoral-level qualifications and several years of experience in internet-delivered treatments. Clinicians at the MindSpot Clinic were mainly nationally registered psychologists (n=15; 75%), three were provisional psychologists in training, one was an indigenous mental health worker and one was a counsellor. All had previous clinical experience of 1–15 years. Clinicians in both the research unit and the MindSpot Clinic received specific training in the principles of online psychological assessment and treatment, as well as detailed course instructions. Supervision and ongoing training was provided in both settings. For the clinic group, clinicians attempted to contact all the participants each week via telephone or email to provide guidance in completing the course, although the level of contact varied widely.22 For the research group, participants were originally allocated to one of three levels of clinician contact. As the published results from this trial showed that the level of clinician contact did not affect outcomes,11 the participants were analysed as one group for the purposes of this study.
Outcome measures
Two primary measures were used to assess the presence and severity of symptoms: the first, the Patient Health Questionnaire 9-item scale (PHQ-9), was used to measure symptoms of depression.23 Scores on the PHQ-9 range from 0 to 27, with higher scores indicating more severe symptoms. A score ≥10 on the PHQ-9 has been identified as the threshold for identifying clinically significant depression, and a score ≥20 indicates symptoms of severe depression.23 A score ≥24 on the PHQ-9 can be used to define very severe symptoms of depression,11 and patients scoring ≥24 were excluded from the research group (but not the clinic group). The PHQ-9 criterion originally applied to the research group has good internal consistency and is sensitive to change.24
The second, the Generalized Anxiety Disorder 7-item scale (GAD-7), was used to measure symptoms of anxiety.25 Scores on the GAD-7 range from 0 to 21, with higher scores indicating more severe symptoms. A score ≥8 has been identified as the threshold for identifying clinically significant anxiety, and a score ≥15 indicates severe symptoms.26 The GAD-7 also has good internal consistency and good convergent and divergent validity with other anxiety and disability scales.24,27 The PHQ-9 and GAD-7 were administered at assessment, weekly throughout the course, post-treatment and at 3-month follow-up.
The Kessler 10-item scale (K-10) was used as a secondary measure to assess general psychological distress.28 Scores range from 10 to 50, with higher scores indicating more severe distress. The K-10 was administered at assessment, mid-treatment, post-treatment and follow-up. Treatment satisfaction and acceptability were assessed at post-treatment by two questions requiring a ‘yes’ or ‘no’ response: (1) ‘Would you feel confident in recommending this course?’ and (2) ‘Was it worth your time doing the course?’
Statistical analyses
Group differences in demographics, clinician time and satisfaction were assessed by general linear models. Treatment outcomes over time on the PHQ-9, GAD-7 and K-10 were analysed with generalised estimating equation (GEE) models. Consistent with the principles of intention-to-treat analyses, separate GEE models utilising random intercepts were employed to impute missing data. An unstructured working correlation matrix and maximum likelihood estimation were used, and a gamma distribution with a log link response scale was specified to address positive skewness in the dependent variable distributions. Outcome measures were analysed at assessment, post-treatment and at 3-month follow-up, with assessment scores included as a covariate.
To assess clinical significance, Cohen’s d effect sizes and 95% confidence intervals were calculated for within-group and between-groups effects, based on the estimated marginal means derived from the GEE models. Average percentage improvements across time were also calculated. To provide data about negative outcomes,29 the proportion of participants reporting deteriorations ≥30% on the primary measures as well as symptoms in the clinical ranges at post-treatment (i.e. ≥10 for the PHQ-9 and ≥8 for the GAD-7) were calculated.
Results
Demographic characteristics, course engagement and clinician contact time
Demographic characteristics are included in Table 1. The mean age of both groups was 65.7 years (clinic group: s.d. 5.1; range 60–88; research group: s.d. 4.8; range 60–93). There were no significant differences between the clinic and research groups in age, gender, marital status, education or vocational status (P’s>0.05). There was a significant difference between groups in psychotropic medication use and initial symptom severity (Table 1). At assessment, 42% of the clinic group reported taking medication for anxiety or depression, compared with 33% of the research group. In addition, 14% and 25% of the clinic group reported severe symptoms of depression or anxiety, respectively (≥20 on the PHQ-9 or ≥15 on the GAD-7), compared with 4% and 19% of the research group (P’s>0.05). All other participants scored in the mild range or above on either the PHQ-9 or GAD-7. Distributions of PHQ-9 and GAD-7 scores according to the established severity categories for each measure are shown in Fig. 2.
Table 1. Demographic characteristics and symptom and treatment histories.
| Clinic group (n=516) | Research group (n=433) | Statistical comparison between groups | |
|---|---|---|---|
| Age | |||
| Mean (s.d.) | 65.7 (5.1) | 65.7 (4.8) | F1,947=0.0, P=0.993 |
| Range | 60–88 | 60–93 | |
| Gender | |||
| Female | 59% (302/516) | 64% (278/433) | Wald’s χ² =3.2, P=0.074 |
| Marital status | |||
| Single/never married | 12% (62/516) | 4% (19/433) | Wald’s χ² =0.1, P=0.820 |
| Married/de facto | 56% (290/516) | 68% (293/433) | |
| Separated/divorced/widowed/other | 26% (133/516) | 28% (121/433) | |
| No answer | 6% (31/516) | 0% (0/433) | |
| Education | |||
| High school or less | 21% (110/516) | 20% (85/433) | Wald’s χ² =0.0, P=0.970 |
| Certificate/diploma/university | 75% (389/516) | 80% (348/433) | |
| No answer | 3% (17/516) | 0% (0/433) | |
| Vocational statusa | |||
| Retired | 50% (257/516) | 53% (229/433) | Wald’s χ² =1.0, P=0.318 |
| Employed full-time or part-time | 31% (159/516) | 33% (143/433) | |
| Other/no answer | 19% (100/516) | 14% (61/433) | |
| Treatment history | |||
| Seen mental health professional in previous 12 months | 34% (176/516) | 34% (149/433) | Wald’s χ² =0.0, P=0.922 |
| Taking psychotropic medication | 42% (216/503) | 33% (145/433) | Wald’s χ² =7.0, P<0.01 |
| Symptom severity at assessmentb | |||
| Severe symptoms of depression (PHQ-9 ≥20) | 14% (70/516) | 4% (19/433) | Wald’s χ² =21.1, P<0.001 |
| Severe symptoms of anxiety (GAD-7 ≥15) | 25% (127/516) | 19% (81/433) | Wald’s χ² =4.8, P<0.05 |
Fig. 2. Distributions of PHQ-9 and GAD-7 scores by category.
There were statistically significant between-group differences in adherence and attrition (Fig. 1). The clinic group completed a mean of 4.4 lessons (s.d. 1.2) compared with a mean of 4.7 lessons (s.d. 1.1) in the research group (P<0.001). Post-treatment and follow-up questionnaire completion rates were also lower in the clinic group (81% and 50%, respectively) compared with the research group (97% and 96%; P’s<0.001), and course withdrawal rates were higher (13% compared with <1%; P<0.001).
Clinician contact comprised making and receiving phone calls, and reading and responding to secure emails. There was a significant difference between groups in clinician contact time (F=520.6, P<0.001). The clinic group had more contact with a clinician throughout the course, with a mean total clinician time per participant of 129 min (s.d. 76.3), compared with 36 min (s.d. 40.0) for the research group.
Treatment outcomes
PHQ-9 scores at assessment were higher for the clinic group compared with the research group (Wald’s χ²=1066.4, P<0.001). There was a significant main effect over time on the PHQ-9 (Wald’s χ²=2117.1, P<0.001), with both groups showing a significant decrease in scores from assessment to post-treatment and from assessment to 3-month follow-up (P’s<0.001). There were no group (Wald’s χ²=0.4, P=0.549) or interaction effects (Wald’s χ²=2.4, P=0.304).
Initial scores on the GAD-7 were higher for the clinic group compared with the research group (Wald’s χ²=1045.1, P<0.001). There was a significant main effect over time (Wald’s χ²=1963.7, P<0.001), with both groups showing a significant decrease in GAD-7 scores from assessment to post-treatment, from assessment to 3-month follow-up and from post-treatment to follow-up (P’s<0.001). There were no group (Wald’s χ²=0.1, P=0.737) or interaction effects (Wald’s χ²=3.5, P=0.179).
Initial scores on the K-10 were higher for the clinic group compared with the research group (Wald’s χ²=3015.2, P<0.001). There was a significant main effect of time (Wald’s χ²=12566.0, P<0.001), with both groups showing a significant decrease in symptoms from assessment to post-treatment, assessment to 3-month follow-up and from post-treatment to follow-up (P’s<0.001). There was no main group effect (Wald’s χ²=1.9, P=0.164), but there was an interaction effect (Wald’s χ²=6.6, P=0.037). Pairwise comparisons indicated that the improvement in K-10 scores at 3-month follow-up was greater for the clinic group than the research group (P=0.020).
Clinical significance
Effect sizes based on estimated marginal means are given in Table 2. Within-group effect sizes ranged from 1.4 to 1.7 for the clinic group and 1.2 to 1.6 for the research group. Between-group effect sizes were ≤0.3. Percentage changes are also shown in Table 2. Percentage improvements (i.e. reduction in symptoms) for the clinic group ranged from 57% to 62%. For the research group, percentage improvements ranged from 55% to 60%.
Table 2. Means and effect sizes (Cohen’s d) for the primary and secondary outcome measures.
| n | Estimated means | Percentage change from assessment | Within-group effect sizes from assessment | Between-group effect sizes | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Assessment | Post-treatment | 3-month follow-up | To post-treatment | To 3-month follow-up | To post-treatment | To 3-month follow-up | Post-treatment | 3-month follow-up | ||
| PHQ-9 | ||||||||||
| Clinic group | 516 | 12.7 (6.0) | 5.3 (4.1) | 5.3 (3.8) | 58% [55–61] | 58% [56–61] | 1.4 [1.3–1.6] | 1.5 [1.3–1.6] | 0.2 [0.1–0.3] | 0.3 [0.1–0.4] |
| Research group | 433 | 10.5 (4.8) | 4.5 (3.7) | 4.4 (3.3) | 57% [54–61] | 58% [55–61] | 1.4 [1.3–1.6] | 1.5 [1.3–1.6] | ||
| GAD-7 | ||||||||||
| Clinic group | 516 | 10.5 (5.0) | 4.5 (3.5) | 4.0 (2.8) | 57% [54–60] | 62% [59–64] | 1.4 [1.3–1.5] | 1.6 [1.5–1.7] | 0.0 [−0.2–0.1] | 0.0 [−0.2–0.1] |
| Research group | 433 | 10.2 (4.4) | 4.6 (3.9) | 4.1 (3.3) | 55% [51–58] | 60% [57–63] | 1.4 [1.2–1.5] | 1.6 [1.4–1.7] | ||
| K-10 | ||||||||||
| Clinic group | 516 | 28.1 (6.9) | 19.1 (6.2) | 17.8 (5.0) | 1.4 [1.2–1.5] | 1.7 [1.6–1.9] | 0.2 [0.1–0.4] | 0.3 [0.1–0.4] | ||
| Research group | 433 | 25.0 (6.5) | 17.6 (6.0) | 16.5 (5.5) | 1.2 [1.0–1.3] | 1.4 [1.3–1.6] | ||||
Note: Standard deviations are shown in round parentheses for the means and 95% confidence intervals are shown in square parentheses for percentage changes and effect sizes.
PHQ-9, Patient Health Questionnaire 9-item scale; GAD-7, Generalized Anxiety Disorder 7-item scale; K-10, Kessler 10-item scale.
On both the PHQ-9 and GAD-7, 1.2% (6/516) of participants in the clinic group showed a deterioration in scores of ≥30% and also scored above the clinical cut-off at post-treatment. For the research group, the deterioration rate was 1.4% (6/427) for both the PHQ-9 and GAD-7.
Treatment satisfaction
Of the participants who completed the evaluation questions, 97.3% (389/400) of the clinic group and 93.5% (361/386) of the research group reported that they would recommend the course to a friend. Similarly, 98.7% (390/395) of the clinic group and 94.3% (363/385) of the research group reported that the course was worth their time. There were no differences between groups in treatment satisfaction (P’s>0.05).
Discussion
This study evaluated the effectiveness of the Wellbeing Plus Course, an iCBT intervention designed for older adults with symptoms of anxiety and depression, following its implementation at a national online mental health service. The clinic group was compared with a research group, comprising participants from a randomised controlled trial. At assessment, psychotropic medication use was higher for the clinic group, as were symptoms of depression and anxiety, though for both groups, mean scores on the PHQ-9 and GAD-7 were in the moderate range.
Course completion was lower and clinician contact times were higher for the clinic group; however, both groups showed significant reductions in symptoms of anxiety and depression at post-treatment and 3-month follow-up and reported high levels of satisfaction with the intervention. There were no differences between the groups on the primary outcomes or in terms of satisfaction.
The results of this study indicate that iCBT for older adults with anxiety and depression can be successfully translated from the research environment into a public health setting, highlighting the considerable potential of iCBT to improve treatment access for older adults. Importantly, as well as being effective and acceptable, deterioration rates were low (<1.5%). The results are consistent with several previous studies showing the effectiveness of iCBT in routine clinical care for adults in general14,16 and older adults in particular.15 The results are also consistent with the significant symptom improvements observed in older adults completing earlier versions of the course.7,5,10,12,30 Furthermore, the differences in clinician contact times observed in this and previous studies8,11 suggest that there is scope for improving the efficiency of routine clinical care.
There are some limitations to this study. First, it is based on self-reported data from treatment-seeking individuals, and the absence of a formal diagnosis means that some care is needed in generalising the results. Second, in both the clinic group and the research group, the participants were drawn from the ‘young elderly’, with a mean age of around 66 years. It would be of interest to replicate the results in an older group, although previous research in this population suggests that age cohorts (60–64 years, 65–74 years and 75 years and above) respond similarly to the intervention.11 However, age-associated neurological disorders could be expected to affect symptom presentation in a more elderly sample, and the potential impact of age-related cognitive impairment in an elderly sample would be an important consideration in future studies. It is also important to note that participants with very severe symptoms of depression and thoughts of suicide or self-harm were referred to other services. Therefore, we cannot extend the current results to this group. Another limitation is the relatively short 3-month follow-up period, although similar studies have found that the gains made following iCBT are still evident 24 months after treatment.31,32 The short follow-up period also limits the ability to determine the cost-effectiveness of the intervention, although again the economic analyses after longer-term follow-up in similar studies have been favourable.8
Although the study’s design can partially inform about the effectiveness of interventions in real-world conditions, there are limitations associated with the lack of randomisation of participants and the lack of formal matching of participants between conditions. These limitations are partly mediated by sample size; however, the use of alternative designs (e.g. pragmatic randomised controlled trial) and sampling methods (e.g. the use of stepwise-matching of participants) could address these issues in future trials.
It is also important to recognise the specialist nature of the MindSpot Clinic itself, which includes psychologists, psychiatrists, nurses and indigenous health workers trained in iCBT, a system of careful supervision of clinicians and a unique and well-supported software platform capable of delivering online assessment and treatment to more than 15 000 people nationally per year.22 This may limit the generalisability of the results to service providers that provide comparable levels of infrastructure, specialisation or support.17–19 Notwithstanding this, a version of the service model developed at the MindSpot Clinic has been successfully implemented in Saskatchewan, Canada,13 highlighting the real-world potential of iCBT for people from a range of age groups and localities.
The growing availability of online mental health services, combined with the increasing use of the internet by older adults and an ageing population, highlights the need for real-world data from this age group. Overall, this study confirms that the effectiveness and acceptability of iCBT for older adults can be maintained when delivered as part of routine clinical care. Such studies play an essential role in informing researchers, health service providers, policy makers and consumers.
References
- 1.Goncalves DC, Pachana NA, Byrne GJ. Prevalence and correlates of generalized anxiety disorder among older adults in the Australian National Survey of Mental Health and Well-Being. J Affect Disord 2011; 132: 223–30. [DOI] [PubMed] [Google Scholar]
- 2.Pirkis J, Pfaff J, Williamson M, Tyson O, Stocks N, Goldney R, et al. The community prevalence of depression in older Australians. J Affect Disord 2009; 115: 54–61. [DOI] [PubMed] [Google Scholar]
- 3.Mackenzie CS, Reynolds K, Cairney J, Streiner DL, Sareen J. Disorder-specific mental health service use for mood and anxiety disorders: associations with age, sex, and psychiatric comorbidity. Depress Anxiety 2012; 29: 234–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Trollor JN, Anderson TM, Sachdev PS, Brodaty H, Andrews G. Prevalence of mental disorders in the elderly: the Australian National Mental Health and Well-Being Survey. Am J Geriatr Psychiatry 2007; 15: 455–66. [DOI] [PubMed] [Google Scholar]
- 5.Jones SL, Hadjistavropoulos HD, Soucy JN. A randomized controlled trial of guided internet-delivered cognitive behaviour therapy for older adults with generalized anxiety. J Anxiety Disord 2016; 37: 1–9. [DOI] [PubMed] [Google Scholar]
- 6.Spek V, Cuijpers P, Nyklíček I, Smits N, Riper H, Keyzer J, et al. One-year follow-up results of a randomized controlled clinical trial on internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years. Psychol Med 2008; 38: 635–9. [DOI] [PubMed] [Google Scholar]
- 7.Dear BF, Zou J, Titov N, Lorian C, Johnston L, Spence J, et al. Internet-delivered cognitive behavioural therapy for depression: a feasibility open trial for older adults. Aust N Z J Psychiatry 2013; 47: 169–76. [DOI] [PubMed] [Google Scholar]
- 8.Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Sheehan J, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety: a randomized controlled trial. Behav Ther 2015; 46: 206–17. [DOI] [PubMed] [Google Scholar]
- 9.Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Terides MD, et al. Examining self-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety and depression: two feasibility open trials. Internet Interv 2015; 2: 17–23. [Google Scholar]
- 10.Titov N, Dear BF, Ali S, Zou JB, Lorian CN, Johnston L, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial. Behav Ther 2015; 46: 193–205. [DOI] [PubMed] [Google Scholar]
- 11.Titov N, Fogliati VJ, Staples LG, Gandy M, Johnston L, Wootton B, et al. Treating anxiety and depression in older adults: randomised controlled trial comparing guided v. self-guided internet-delivered cognitive-behavioural therapy. BJPsych Open 2016; 2: 50–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Zou JB, Dear BF, Titov N, Lorian CN, Johnston L, Spence J, et al. Brief internet-delivered cognitive behavioral therapy for anxiety in older adults: a feasibility trial. J Anxiety Disord 2012; 26: 650–5. [DOI] [PubMed] [Google Scholar]
- 13.Hadjistavropoulos HD, Nugent MN, Alberts NM, Staples LG, Dear BF, Titov N. Transdiagnostic Internet-delivered cognitive behaviour therapy in Canada: an open trial comparing results of a specialized online clinic and nonspecialized community clinics. J Anxiety Disord 2016; 42: 19–28. [DOI] [PubMed] [Google Scholar]
- 14.Hedman E, Ljótsson B, Kaldo V, Hesser H, Alaoui S, Kraepelien M, et al. Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J Affect Disord 2013; 155: 49–58. [DOI] [PubMed] [Google Scholar]
- 15.Mewton L, Sachdev P, Andrews G. A naturalistic study of the acceptability and effectiveness of internet-delivered cognitive behavioural therapy for psychiatric disorders in older Australians. PLoS One 2013; 8: e71825. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ruwaard J, Lange A, Schrieken B, Dolan CV, Emmelkamp P. The effectiveness of online cognitive behavioral treatment in routine clinical practice. PLoS One 2012; 7: e40089. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. BMJ 2015; 351: h5627. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kenter RMF, van de Ven PM, Cuijpers P, Koole G, Niamat S, Gerrits RS, et al. Costs and effects of Internet cognitive behavioral treatment blended with face-to-face treatment: results from a naturalistic study. Internet Interv 2015; 2: 77–83. [Google Scholar]
- 19.Twomey C, O’Reilly G, Byrne M, Bury M, White A, Kissane S, et al. A randomized controlled trial of the computerized CBT programme, MoodGYM, for public mental health service users waiting for interventions. Br J Clin Psychol 2014; 53: 433–50. [DOI] [PubMed] [Google Scholar]
- 20.Mewton L, Wong N, Andrews G. The effectiveness of internet cognitive behavioural therapy for generalized anxiety disorder in clinical practice. Depress Anxiety 2012; 29: 843–9. [DOI] [PubMed] [Google Scholar]
- 21.Australian Government. E-Mental Health Strategy for Australia. Department of Health and Ageing, 2012. [Google Scholar]
- 22.Titov N, Dear B, Staples LG, Bennett-Levy J, Klein B, Rapee R, et al. MindSpot Clinic: an accessible, efficient and effective online treatment service for anxiety and depression. Psychiatr Serv 2015; 66: 1043–50. [DOI] [PubMed] [Google Scholar]
- 23.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001; 16: 606–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Kroenke K, Spitzer RL, Williams JB, Lowe B. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. Gen Hosp Psychiatry 2010; 32: 345–59. [DOI] [PubMed] [Google Scholar]
- 25.Spitzer R, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006; 166: 1092–7. [DOI] [PubMed] [Google Scholar]
- 26.Kroenke K, Spitzer RL, Williams JB, Monahan PO, Lowe B. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Ann Intern Med 2007; 146: 317–25. [DOI] [PubMed] [Google Scholar]
- 27.Dear BF, Titov N, Sunderland M, McMillan D, Anderson T, Lorian C, et al. Psychometric comparison of the Generalized Anxiety Disorder Scale-7 and the Penn State Worry Questionnaire for Measuring Response during Treatment of Generalised Anxiety Disorder. Cogn Behav Ther 2011; 40: 216–27. [DOI] [PubMed] [Google Scholar]
- 28.Kessler R, Andrews G, Colpe L, Hiripi E, Mroczek D, Normand S, et al. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32: 959–76. [DOI] [PubMed] [Google Scholar]
- 29.Rozental A, Andersson G, Boettcher J, Ebert DD, Cuijpers P, Knaevelsrud C, et al. Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interv 2014; 1: 12–9. [Google Scholar]
- 30.Dear B, Zou J, Ali S, Lorian C, Johnston L, Terides MD, et al. Examining self-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety and depression: two feasibility open trials. Internet Interv 2015; 2: 17–23. [Google Scholar]
- 31.Dear BF, Staples LG, Terides MD, Karin E, Zou J, Johnston L, et al. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: a randomized controlled trial. J Anxiety Disord 2015; 36: 63–77. [DOI] [PubMed] [Google Scholar]
- 32.Titov N, Dear B, Staples LG, Terides MD, Karin E, Sheehan J, et al. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: a randomized controlled trial. J Anxiety Disord 2015; 35: 88–102. [DOI] [PubMed] [Google Scholar]