Abstract
Background
Major depression is defined as a period of depression lasting at least 2 weeks characterized by depressed mood, most of the day, nearly every day, and/or markedly diminished interest or pleasure in all, or almost all, activities. Anxiety disorders encompass a broad range of disorders in which people experience feelings of fear and excessive worry that interfere with normal day-to-day functioning.
Cognitive behavioural therapy (CBT) is a form of evidence-based psychotherapy used to treat major depression and anxiety disorders. Internet-delivered CBT (iCBT) is structured, goal-oriented CBT delivered via the internet. It may be guided, in which the patient communicates with a regulated health care professional, or unguided, in which the patient is not supported by a regulated health care professional.
Methods
We conducted a health technology assessment, which included an evaluation of clinical benefit, value for money, and patient preferences and values related to the use of iCBT for the treatment of mild to moderate major depression or anxiety disorders. We performed a systematic review of the clinical and economic literature and conducted a grey literature search. We reported Grading of Recommendations Assessment, Development, and Evaluation (GRADE) ratings if sufficient information was provided. When other quality assessment tools were used by the systematic review authors in the included studies, these were reported. We assessed the risk of bias within the included reviews. We also developed decision-analytic models to compare the costs and benefits of unguided iCBT, guided iCBT, face-to-face CBT, and usual care over 1 year using a sequential approach. We further explored the lifetime and short-term cost-effectiveness of stepped-care models, including iCBT, compared with usual care. We calculated incremental cost-effectiveness ratios (ICERs) from the perspective of the Ontario Ministry of Health and Long-Term Care and estimated the 5-year budget impact of publicly funding iCBT for mild to moderate major depression or anxiety disorders in Ontario. To contextualize the potential value of iCBT as a treatment option for major depression or anxiety disorders, we spoke with people with these conditions.
Results
People who had undergone guided iCBT for mild to moderate major depression (standardized mean difference [SMD] = 0.83, 95% CI 0.59–1.07, GRADE moderate), generalized anxiety disorder (SMD = 0.84, 95% CI 0.45–1.23, GRADE low), panic disorder (small to very large effects, GRADE low), and social phobia (SMD = 0.85, 95% CI 0.66–1.05, GRADE moderate) showed a statistically significant improvement in symptoms compared with people on a waiting list. People who had undergone iCBT for panic disorder (SMD= 1.15, 95% CI: 0.94 to 1.37) and iCBT for social anxiety disorder (SMD=0.91, 95% CI: 0.74–1.07) showed a statistically significant improvement in symptoms compared with people on a waiting list. There was a statistically significant improvement in quality of life for people with generalized anxiety disorder who had undergone iCBT (SMD = 0.38, 95% CI 0.08–0.67) compared with people on a waiting list. The mean differences between people who had undergone iCBT compared with usual care at 3, 5, and 8 months were −4.3, −3.9, and −5.9, respectively. The negative mean difference at each follow-up showed an improvement in symptoms of depression for participants randomized to the iCBT group compared with usual care. People who had undergone guided iCBT showed no statistically significant improvement in symptoms of panic disorder compared with individual or group face-to-face CBT (d = 0.00, 95% CI −0.41 to 0.41, GRADE very low). Similarly, there was no statistically significant difference in symptoms of specific phobia in people who had undergone guided iCBT compared with brief therapist-led exposure (GRADE very low). There was a small statistically significant improvement in symptoms in favour of guided iCBT compared with group face-to-face CBT (d= 0.41, 95% CI 0.03–0.78, GRADE low) for social phobia. There was no statistically significant improvement in quality of life reported for people with panic disorder who had undergone iCBT compared with face-to-face CBT (SMD = −0.07, 95% CI −0.34 to 0.21).
Guided iCBT was the optimal strategy in the reference case cost–utility analyses. For adults with mild to moderate major depression, guided iCBT was associated with increases in both quality-adjusted survival (0.04 quality-adjusted life-years [QALYs]) and cost ($1,257), yielding an ICER of $31,575 per QALY gained when compared with usual care. In adults with anxiety disorders, guided iCBT was also associated with increases in both quality-adjusted survival (0.03 QALYs) and cost ($1,395), yielding an ICER of $43,214 per QALY gained when compared with unguided iCBT. In this population, guided iCBT was associated with an ICER of $26,719 per QALY gained when compared with usual care. The probability of cost-effectiveness of guided iCBT for major depression and anxiety disorders, respectively, was 67% and 70% at willingness-to-pay of $100,000 per QALY gained. Guided iCBT delivered within stepped-care models appears to represent good value for money for the treatment of mild to moderate major depression and anxiety disorders.
Assuming a 3% increase in access per year (from about 8,000 people in year 1 to about 32,000 people in year 5), the net budget impact of publicly funding guided iCBT for the treatment of mild to moderate major depression would range from about $10 million in year 1 to about $40 million in year 5. The corresponding net budget impact for the treatment of anxiety disorders would range from about $16 million in year 1 (about 13,000 people) to about $65 million in year 5 (about 52,000 people).
People with depression or an anxiety disorder with whom we spoke reported that iCBT improves access for those who face challenges with face-to-face therapy because of costs, time, or the severity of their condition. They reported that iCBT provides better control over the pace, time, and location of therapy, as well as greater access to educational material. Some reported barriers to iCBT include the cost of therapy; the need for a computer and internet access, computer literacy, and the ability to understand complex written information. Language and disability barriers also exist. Reported limitations to iCBT include the ridigity of the program, the lack of face-to-face interactions with a therapist, technological difficulties, and the inability of an internet protocol to treat severe depression and some types of anxiety disorder.
Conclusions
Compared with waiting list, guided iCBT is effective and likely results in symptom improvement in mild to moderate major depression and social phobia. Guided iCBT may improve the symptoms of generalized anxiety disorder and panic disorder compared with waiting list. However, we are uncertain about the effectiveness of iCBT compared with individual or group face-to-face CBT. Guided iCBT represents good value for money and could be offered for the short-term treatment of adults with mild to moderate major depression or anxiety disorders. Most people with mild to moderate depression or anxiety disorders with whom we spoke felt that, despite some perceived limitations, iCBT provides greater control over the time, pace, and location of therapy. It also improves access for people who could not otherwise access therapy because of cost, time, or the nature of their health condition.
OBJECTIVE
This health technology assessment looked at the effectiveness, safety, cost-effectiveness, budget impact of publicly funding, and patient preferences and values associated with internet-delivered cognitive behavioural therapy (iCBT) for the treatment of mild to moderate major depression and anxiety disorders.
This health technology assessment has been registered on PROSPERO, the international prospective register of systematic reviews (CRD42018096042), available at https://www.crd.york.ac.uk/PROSPERO.
BACKGROUND
Health Condition
Major depression is one of the most common mental illnesses, imposing a huge human and economic burden on people and society. Each year, about 7% of people in Canada meet the diagnostic criteria for major depression, and about 13% to 15% of these people will experience major depression for the rest of their lives.1 The essential feature of major depression is the occurrence of one or more major depressive episodes, defined as periods lasting at least 2 weeks characterized by depressed mood most of the day, nearly every day, and/or a markedly diminished interest or pleasure in all, or almost all, activities.2 To receive a diagnosis of major depression, within the same 2-week period a person must experience five or more symptoms from the criteria for a major depressive episode as described in the current version of the Diagnostic and Statistical Manual of Mental Disorders (DSM).3
Major depression is both chronic (lasting 3 months or more) and episodic (consisting of separate episodes) in nature. It consists of initial phases (i.e., the acute and continuation phases, each lasting approximately 3 months) and a maintenance phase (lasting approximately 6 to 24 months, with an average 9 to 12 months).2,4–6 The category of anxiety disorders includes a broad range of disorders, including generalized anxiety disorder (GAD), panic disorder (with and without agoraphobia), social anxiety disorder, and specific phobias (common phobias include fear of animals, insects, germs, heights, thunder, driving, public transportation, flying, dental or medical procedures, and elevators). People with anxiety disorders experience feelings of fear and excessive worry that impacts their overall well-being and functioning. The DSM classifies post-traumatic stress disorder (PTSD) and obsessive-compulsive disorder (OCD) outside of the category of anxiety disorders.7,8 Anxiety disorders can exist in isolation or coexist with other anxiety and depressive disorders.9,10
Clinical Need and Target Population
Approximately 11.3% of the Canadian population have been classified as meeting criteria for major depression at some point in their life.2,11 Major depression affects not only individuals and families but also occupational functioning, through absenteeism and presenteeism (loss of productivity from attending work while unwell).2 Major depression also negatively affects people's ability to perform personal activities such as parenting and housekeeping.
As of 2006, the lifetime prevalence rates of panic disorder, agoraphobia, and social phobia in Canada were 3.7%, 1.5%, and 8.1%, respectively.12 One Ontario study estimated that 12% of adults between the ages of 15 and 64 years—9% of men and 16% of women—experience an anxiety disorder during any 12-month period.12
Current Treatment Options
Treatment for acute major depression consists of pharmacological and psychological interventions. The use of antidepressant medications has increased over the last 20 years, mainly due to the advent of selective serotonin reuptake inhibitors, as well as newer agents.13 While antidepressants continue to be a mainstay in the treatment of major depression, adherence rates are low, in part because of patients' concerns about side effects and possible dependency. Surveys have demonstrated patients' preference for psychological therapies over antidepressants.13
Psychotherapy is the treatment of mental or emotional illness through psychological methods rather than through drugs. There are many types of psychotherapy. Cognitive behavioural therapy (CBT) is an evidence-based, structured, intensive, time-limited, symptom-focused form of psychotherapy recommended for the treatment of major depression and anxiety disorders.14
Cognitive behavioural therapy helps people become aware of how certain negative automatic thoughts, attitudes, expectations, and beliefs contribute to feelings of sadness and anxiety. People undergoing CBT learn how their thinking patterns, which may have developed in the past to deal with difficult or painful experiences, can be identified and changed to reduce unhappiness.14
Barriers to face-to-face CBT include stigmas around people seeking help in person, geography (distance from health care professional), time, and cost. Increasingly, there is a desire to pursue internet delivery as an option to increase access to treatment.15
The treatment of major depression can be divided into acute and maintenance phases.2,4–6 The aim of treatment in the acute and continuation phases is the reduction or elimination (remission) of symptoms and a return to the level of psychological and social functioning (psychosocial functioning) experienced before the onset of major depression.2 The aim of treatment in the maintenance phase is to prevent symptoms from recurring.2
A Health Quality Ontario quality standard on major depression1 recommends that people with major depression have timely access to either antidepressant medication or evidence-based psychotherapy, based on their preference. Clinical guidelines suggest that CBT may be offered as an initial treatment for anxiety disorders. Pharmacological treatment may be considered if the person has a poor response to CBT treatment.16
The National Institute for Health and Care Excellence (NICE) guidelines recommend a stepped-care approach that starts with low-intensity treatments such as guided iCBT for people with mild to moderate major depression or anxiety disorders.17 They recommend higher-intensity interventions (face-to-face psychotherapies such as CBT or interpersonal therapy) alone or in combination with medications for people who do not respond to treatment or who progress to more severe depression or anxiety.17
Health Service Under Review
Internet-delivered CBT is based on the principles of CBT, consists of structured modules with clearly defined goals, and is delivered via the internet.14 Although there are many types of iCBT programs, each are goal oriented sessions that typically consist of 8 to 12 modules and can be guided or unguided.14 Internet-delivered CBT programs are made available by computer, smartphone, or tablet, for a fee.14 With unguided iCBT, patients are informed of a website through which they can participate in an online self-directed program. Guided iCBT involves support from a regulated health professional (e.g., social worker, psychologist, psychotherapist, occupational therapist, nurse, or physician). In guided iCBT, people complete modules and communicate (via email, text messages, or telephone calls) their progress to a regulated health professional.14
Current recommendations indicate that iCBT is not appropriate for severely ill people.14
Regulatory Information
Internet-delivered CBT does not require regulatory approval from Health Canada.
Ontario Context
Internet-delivered CBT is not currently publicly funded in a systematic manner in Ontario. Guided iCBT is currently provided by some hospitals and in the private sector. There are several pilot programs underway or recently completed in Canada, funded through public and private sources.
CLINICAL EVIDENCE
Research Question
What are the effectiveness and safety of iCBT for improving outcomes for adults with mild to moderate major depression or anxiety disorders?
Methods
We developed the research questions in consultation with health care providers and clinical experts.
Clinical Literature Search
We performed a literature search on February 15, 2018, to retrieve studies published from January 1, 2000 to the search date. We used the Ovid interface to search the following databases: MEDLINE, Embase, Cochrane Database of Systematic Reviews, Health Technology Assessment, National Health Service Economic Evaluation Database (NHS EED), and PsycINFO. We used the EBSCOhost interface to search the Cumulative Index to Nursing & Allied Health Literature (CINAHL).
Medical librarians developed the search strategies using controlled vocabulary (e.g., Medical Subject Headings) and relevant keywords. We applied a search filter to limit results to systematic reviews, meta-analyses, and health technology assessments. The final search strategy was peer reviewed using the PRESS Checklist.18 We created database auto-alerts in MEDLINE, Embase, PsycINFO, and CINAHL and monitored them for the duration of the assessment period.
We performed targeted grey literature searching of health technology assessment agency websites and PROSPERO. See Appendix 1 for the literature search strategies, including all search terms.
Literature Screening
Two reviewers conducted an initial screening of titles and abstracts using Covidence management software and obtained full-text articles that appeared eligible according to the inclusion criteria. The reviewers then examined the full texts of articles that appeared eligible to identify studies eligible for inclusion.
Inclusion and Exclusion Criteria
Studies
We conducted an overview of systematic reviews of randomized controlled trials. We considered publications to be systematic reviews if they met all the following criteria:
Clearly described inclusion and exclusion criteria
Undertook a reproducible search of two or more electronic literature databases
Assessed and documented the quality of the included randomized controlled trials
We included English-language full-text systematic reviews of randomized controlled trials published between January 1, 2000, and February 15, 2018.
Participants
We included studies of outpatient adults aged 16 years and older with a primary diagnosis of mild to moderate major depression or anxiety disorder according to validated diagnostic instruments such as the DSM, the International Statistical Classification of Diseases and Related Health Problems (ICD), the Centre for Epidemiological Scale for Depression, the Beck Depression Inventory, or the Patient Health Questionnaire, Structured Diagnostic Interview Schedule.1 We included studies of people with a primary diagnosis of anxiety disorder or of mild to moderate major depression coexisting with other mental health conditions (excluding OCD and PTSD).
We excluded studies of people less than 16 years old or had participants with postpartum depression, bipolar disorder, dysthymia, seasonal affective disorder, a psychotic disorder, drug or alcohol dependence–related depression or anxiety, obsessive compulsive disorder, posttraumatic stress disorder, major depression or anxiety comorbid with physical disorders (e.g., cancer, stroke, or acute coronary syndrome).
Intervention
We included reviews that assessed iCBT. We excluded non-traditional CBT (e.g., mindfulness CBT), transdiagnostic interventions, CBT delivered via bibliotherapy, and CBT described as computerized but for which there was no analysis specifically for iCBT.
Comparators
We included the following comparators:
Face-to-face CBT, defined as individual or group face-to-face CBT
Usual care, defined as any treatment prescribed by a general practitioner
Waiting list, defined as participants receiving iCBT at a later date
Combination of usual care, waiting list, and/or information control
Outcomes of Interest
We included the following outcomes of interest:
Remission of depression or anxiety symptoms (acute phase)
Prevention of relapse following a successful acute treatment (maintenance phase)
Response to therapy (50% reduction in symptoms from baseline)
Safety
Quality of life
Satisfaction with care
Patient adherence
Data Extraction
Two reviewers extracted relevant data using a data extraction form that included the following study characteristics: study population, inclusion and exclusion criteria, description of the interventions, types of comparators, outcomes, results, and quality assessment as conducted by authors of the systematic reviews. For reviews where a portion of the participants or the intervention did not match the population or intervention of interest, we extracted the results specific to our population or intervention of interest
We contacted authors of the systematic reviews to provide clarification as needed.
Evidence Synthesis
We undertook a narrative summary of the results reported in the included systematic reviews. We did not perform an analysis of primary studies. Results for guided and unguided iCBT were reported separately where available. Unless specified, “iCBT” refers to both guided and unguided iCBT.
Critical Appraisal of Evidence
A single reviewer assessed risk of bias using the Risk of Bias in Systematic Reviews (ROBIS) tool.19 See Appendix 2 for details of the ROBIS assessment.
We assessed the quality of the evidence within the included reviews by extracting the review authors' Grading of Recommendations Assessment, Development, and Evaluation (GRADE) ratings if sufficient information was provided. If other quality assessment tools were used by the systematic review authors in the included studies, these were reported.
Expert Consultation
Consulted experts included physicians in the specialty areas of psychiatry and psychology and regulated mental health professionals. Their role was to review the clinical review plan, contextualize the evidence, and provide feedback on the appropriate use of iCBT.
Results
Literature Search
The literature search yielded 393 citations published between January 1, 2000, and February 15, 2018, after removing duplicates. We obtained the full text of 143 articles for further assessment. Seven systematic reviews met the inclusion criteria.7,20–25 The primary reasons for exclusions are provided below. See Appendix 3 for a selected list of studies excluded after full-text review that includes the primary reason for exclusion.
Figure 1 presents the flow diagram for the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA).
Figure 1: PRISMA Flow Diagram—Clinical Search Strategy.
Source: Adapted from Moher et al.26
Characteristics of Included Systematic Reviews
We identified one systematic review22 that evaluated iCBT for mild to moderate major depression and five systematic reviews7,21,23–25 that evaluated iCBT for anxiety disorders. One systematic review reported data on both mild to moderate major depression and anxiety disorders.20 There was inconsistency across the systematic reviews in reporting and analyzing the level of support associated with iCBT. One systematic review reported the degree of support as therapist-guided (i.e., clinical support).20 The type of therapist support included email correspondence and weekly phone conversations of 10 to 20 minutes for each participant.
The instruments used to diagnose mild to moderate major depression and anxiety disorders in the inclusion criteria of the included systematic reviews varied. There was additional variation across the scales used to measure the change in symptoms from pre- to post-treatment.
Characteristics of the included systematic reviews are summarized in Table 1.
Table 1:
Characteristics of Included Systematic Reviews
| Author, Year | Objective | Study Design and Methods | Comparators | Outcomes of interest |
|---|---|---|---|---|
| Andrews et al, 201821 | Replication and extension of 2010 meta-analysis to examine whether computerised therapy for the anxiety and depressive disorders is effective, acceptable, and practical health care. |
Inclusion criteria Adutls ≥18 yrs with a primary diagnosis of either major depression, GAD, PD with or without agoraphobia, or SAD Diagnosis could be determined by a clinician, through telephone interview or by meeting a recognised cut-off on a validated self-report questionnaire. Exclusion criteria Studies of treatments aimed at a range of diagnoses (transdiagnostic studies), and studies of depressive or anxiety symptoms in which no data on the probability of satisfying diagnostic criteria were supplied. |
Waitlist control, information control, care as usual, or placebo. | Subgroup analyses examining the effects of iCBT on change in symptom severity for depression and anxiety disorders. |
| Arnberg et al, 201420 | (1) Is internet-delivered psychological treatment efficacious, safe, and cost-effective for mood and anxiety disorders in children, adolescents and adults? (2) Is internet-delivered treatment noninferior to established psychological treatments? |
Inclusion Criteria Participants: children, adolescents, and adults with anxiety or mood disorders, including major depression, dysthymia, bipolar disorder, social phobia, PD, GAD, PTSD, OCD, specific phobia, and separation anxiety (in children and adolescents). Intervention: internet-delivered psychological treatments that are based on an explicit psychological theory and not conducted at a clinic. Any support had to be remotely delivered (e.g. email-like messages or telephone). The degree of support was categorized into pure self-help (no support), technician-assisted (e.g., nonclinical), or therapist-guided (i.e., clinical support). Study design: for short-term effects and risk of adverse events, only RCTs were included. For long-term follow-up assessments (i.e., ≥6 mo post-assessment), RCTs and observational studies were included because of the ethical and practical dilemmas of conducting long-term RCTs. For cost-effectiveness data, economic evaluations based on individual-level data and decision models were used. Exclusion Criteria Studies where the participants were selected primarily because of a specific physical illness. |
Any established psychological treatments, waiting list, usual care, or attention control. | Change in symptoms of the primary disorder, adverse events, and cost per effect and per quality-adjusted life year. |
| Kampmann et al, 201625 | Evaluate the efficacy of technology-assisted interventions for individuals with a diagnosis of SAD. |
Inclusion Criteria Participants: lts ≥18 yrs who meet the criteria for a diagnosis of SAD and who had their SAD symptoms assessed during or after the initial assessment. Interventions: treatments targeting SAD symptoms. Study design: RCTs with at least 10 participants per treatment condition, no language restrictions. Exclusion Criteria Dissertation abstracts, reviews, and study protocols. |
Passive control, active control. | Symptoms of depression at post-assessment, efficacy and changes in quality of life. |
| Adelman et al, 20147 | Examine cCBT efficacy for non-PTSD, non-OCD anxiety disorders along multiple dimensions, including treatment efficacy by comparison condition, diagnostic target, level of therapist involvement, study quality, and participant age group. |
Inclusion Criteria RCTs assessing efficacy of cCBT for anxiety disorders, subjects who meet criteria for GAD, PD, SAD, or a specific phobia based on DSM-IV criteria, trials recorded to compare cCBT to waiting list or in-person CBT control condition. Exclusion Criteria OCD and PTSD trials, due to the underlying neuropathology of these conditions, the CBT techniques used to treat these conditions were considered to be sufficiently different from the other anxiety disorders, OCD, and PTSD within diagnostic categories that are distinct from the anxiety disorders. Trials with <10 participants. |
Waiting list or in-person CBT. | Endpoint score on a rating scale used to measure anxiety. Results were stratified by comparator. |
| Kaltenthaler et al, 200822 | Systematically review RCTs of computerized CBT (cCBT) software packages for the treatment of mild to moderate depression. |
Inclusion Criteria Adults with mild to moderate depression, with or without anxiety, as defined by individual studies. Exclusion Criteria Studies on postnatal depression, bipolar disorder, depression with psychotic symptoms or current major depression, or serious suicidal thoughts. |
Current standard treatments including therapist-led CBT, non-directive counselling, primary care counselling, routine management (including drug treatment), and alternative methods of CBT delivery such as bibliotherapy and group CBT. | Improvement in psychological symptoms, quality of life, patient satisfaction. |
| Dedert et al, 201323 | (1) For adults with depressive disorder, PTSD, PD, or GAD, what are the effects of cCBT interventions compared with inactive controls? (2) For cCBT interventions, what level, type, and modality of user support is provided (e.g., daily telephone calls, weekly email correspondence), who provides this support (e.g., therapist, graduate student, peer), what is the clinical context (primary intervention, adjunct), and how is this support related to patient outcomes? Examine the influence of support-related factors on treatment outcomes, including satisfaction, response, and completion. (3) For adults with major depression, PTSD, PD, or GAD, what are the effects of cCBT interventions compared with face-toface therapy? Compare the effectiveness of cCBT with face-to-face CBT. |
Inclusion Criteria Participants: Adults ≥18 yrs with one or more of the following conditions:
Interventions: CBT delivered primarily by a computerized (i.e., electronic) mechanism. Interventions may be self-guided or with clinician support, but the computerized mechanism must be the key intervention that differs from the control group. Study design: RCTs with N > 20. Exclusion Criteria Participants: people with test anxiety, phobias, or SAD. Interventions: interpersonal therapy designed to prevent the onset or relapse of mental illness; interventions that are primarily telemedicine-based (e.g., therapy via video chat or phone interactions, including those by interactive voice response); interventions that use virtual reality as the primary therapeutic mode, do not use the key components of CBT, disease management interventions where CBT is only one component of a more comprehensive intervention, are delivered primarily in face-to-face encounters but supplemented by text messages, or use online materials that do not meet the definition of CBT or CBT-related intervention. |
Usual care not involving psychotherapy; waitlist control; attention/information control, cCBT with a different level of therapist support, face-to-face CBT. | Patient satisfaction, safety, symptom measure, health related quality of life. |
| Richards et al, 201524 | Systematically review and conduct a meta-analysis of internet-delivered psychological therapy for GAD compared to waiting list control groups. |
Inclusion Criteria Adults ≥18yr who have a clinical diagnosis of GAD and may have comorbidity with depression and/or impairment in functioning. Study design: RCTs. |
Waiting list | Clinical efficacy |
Abbreviations: cCBT, computerized CBT; GAD, generalized anxiety disorder; iCBT, internet-delivered cognitive behavioural therapy; OCD, obsessive compulsive disorder; PD, panic disorder; PTSD, posttraumatic stress disorder; RCT, randomized controlled trial; SAD, social anxiety disorder; SD, standard deviation.
Mild to Moderate Major Depression
Guided Internet-Delivered CBT Compared to Waiting List
Symptoms and Response to Treatment
Arnberg et al20 identified five randomized control trials comparing guided iCBT to waiting list. People on the waiting list received iCBT after participants randomized to receive iCBT completed the study. Guided iCBT was delivered in the form of email correspondence or telephone conversations lasting about 10 to 20 minutes for each participant on a weekly basis. Participants who received guided iCBT experienced a reduction in symptoms of depression compared with waiting list (standardized mean difference [SMD] = 0.83, 95% confidence interval [CI] 0.59–1.07). This reduction was statistically significant.
Internet-Delivered CBT Compared to Weekly Telephone Calls
Symptoms and Response to Treatment
Kaltenthaler et al22 reported data from one randomized controlled trial that assessed changes in psychological outcomes before and 6 weeks after treatment for participants with mild to moderate major depression randomized to an iCBT program or control group. The study did not specify whether participants received guided or unguided iCBT. The control group received phone calls from interviewers once a week to discuss lifestyle and environmental factors that may have had an influence on depression. The authors found that participants who received iCBT showed a significant reduction in symptoms of depression compared to the control group (mean difference 3.2, 95% CI 0.9–5.4). This randomized controlled trial used the Centre for Epidemiologic Studies Depression Scale as a primary outcome measure of symptoms of depression.
Patient Satisfaction
Kaltenthaler et al22 reported that participants overall accepted the iCBT program.
Patient Dropout and Treatment Adherence
Kaltenthaler et al22 analyzed one randomized controlled trial that reported on patient dropout when iCBT was compared with participants who received weekly telephone calls from interviewers about environmental and lifestyle factors that may have had an influence on depression. Of 525 participants randomized, 25.3% dropped out of the iCBT program, and 10% of participants who received weekly telephone calls from interviewers about environmental and lifestyle factors were loss to follow-up.
Internet-Delivered CBT Compared to Usual Care
Symptoms and Response to Treatment
Kaltenthaler et al22 reported data from a randomized controlled trial that assessed change in psychological outcomes before and after treatment for participants with mild to moderate major depression randomized to an iCBT program or usual care. Usual care was defined as whatever treatment the general practitioner prescribed, such as medications, or referral to a counsellor or a health professional. The authors found that the mean score using the Beck Depression Inventory at 3, 5, and 8 months after treatment demonstrated an improvement in symptoms for participants receiving iCBT compared to usual care. The mean difference between iCBT and usual care was −4.3, −3.9, and −5.9 at 3, 5, and 8 months, respectively.22 The negative mean difference at each follow-up signifies improvement in symptoms of depression for participants randomized to the iCBT group compared to usual care.
Patient Satisfaction
Kaltenthaler et al22 reported that participants receiving iCBT were significantly more satisfied with treatment compared to individuals randomized to usual care.22
Patient Dropout and Treatment Adherence
Kaltenthaler et al22 reported data from a randomized controlled trial that assessed patient drop out for participants with mild to moderate major depression randomized to an iCBT program or usual care. Of the 274 participants, 35% dropped out during the study.22 The proportion of participants who were lost to follow-up in the iCBT program versus the usual care program is unclear.
See Table 2 for summary of results for major depression.
Table 2:
Summary of Results of Included Systematic Reviews for Mild to Moderate Major Depression
| Author, Year | No. of Studies/No. of Participants | Results | Quality Assessment |
|---|---|---|---|
| Guided internet-deliverd CBT compared with waiting list | |||
| Symptoms and Response to Treatment | |||
| Arnberg et al, 201420 | 5 RCTs/159 | SMD = 0.83 (95% CI 0.59–1.07) | GRADE: ⊕⊕⊕ moderate |
| Internet-delivered CBT compared with weekly telephone calls | |||
| Symptoms and Response to Treatment | |||
| Kaltenthaler et al, 200822 | 1 RCT/525 | SMD = 3.2 (95% CI 0.9–5.4) | CASP Tool Randomization method: statistical software program used Masking: no maskedAssessment Power calculation: yes Loss to follow-up loss: number and some reasonsreported |
| Patient Dropout and Treatment Adherence | |||
| Kaltenthaler et al, 200822 | 1 RCT/525 | 25.3% of participants dropped out of the iCBT program;10% of participants dropped out from the control group, who received weekly telephone calls | CASP Tool Randomization method: statistical software program used Masking: no masked assessment Power calculation: yes Loss to follow-up loss: number and some reasons reported |
| Internet-delivered CBT compared with usual care | |||
| Symptoms and Response to Treatment | |||
| Kaltenthaler et al, 200822 | 1 RCT/274 | iCBT vs. usual care 3 mos: −4.3 (95% CI not reported)a 5 mos: −3.9 (95% CI not reported)a 8 mos: −5.9 (95% CI not reported)a |
CASP Tool Randomization: sealed envelopes, stratified for medication and duration of current episode Masking: no masked assessment Power calculation: yes Loss to follow-up: number and some reasons reported |
| Patient Dropout and Treatment Adherence | |||
| Kaltenthaler et al, 200822 | 1 RCT/274 | 35% dropped out during the study | CASP Tool Randomization: SPSS function Masking: no masked assessment Power calculation: yes Loss to follow-up: loss and reasons reported |
Abbreviations: CI, confidence interval; iCBT, internet-delivered cognitive behavioural therapy; SMD, standardized mean difference.
A positive improvement in symptoms in favour of iCBT. The mean differences were calculated by the authors of the health technology assessment.
Anxiety Disorders
Internet-Delivered CBT Compared to Waiting List
Symptoms and Response to Treatment
Arnberg et al20 identified four randomized controlled trials that reported on changes in symptoms for participants with generalized anxiety disorder. A large statistically significant pooled effect was found for guided iCBT that involved therapist (i.e., clinical) support compared to waiting list (SMD = 0.84, 95% CI 0.45–1.23).
Similarly, Adelman et al7 identified four randomized controlled trials that reported on anxiety symptoms for generalized anxiety disorder. There was a large statistically significant standardized mean difference that showed an improvement in anxiety symptoms for participants who received iCBT compared to waiting list (SMD = 1.06, 95% CI 0.82–1.30).
Arnberg et al20 found small to very large effects in favour of guided iCBT, which involved therapist (i.e., clinical) support, compared to waiting list for individuals with panic disorder.32
Adelman et al7 identified eight randomized controlled trials that reported on panic disorder. The authors found a large statistically significant standardized mean difference showing an improvement in anxiety symptoms for participants who received iCBT compared to waiting list (SMD = 1.15, 95% CI 0.94— 1.37).
Arnberg et al20 reported on changes in symptoms for eight randomized controlled trials on social phobia that compared guided iCBT, which involved therapist (i.e., clinical) support, to waiting list. A large statistically significant pooled effect was found in favour of guided iCBT compared to waiting list (SMD = 0.85, 95% CI 0.66–1.05).
Adelman et al7 identified nine randomized controlled trials that reported on anxiety symptoms for social anxiety disorder. The authors found a large statistically significant standardized mean difference that showed improvement in anxiety symptoms for participants who received iCBT compared to waiting list (SMD = 0.91, 95% CI 0.74–1.07).
Dedert et al23 identified four trials (five comparisons) in people with generalized anxiety disorder and found that those who had undergone iCBT experienced a large statistically significant difference in symptoms of anxiety compared with people on a waiting list (SMD −0.94, 95% CI −1.34 to −0.54).
Quality of Life
Richards et al24 identified two studies on generalized anxiety disorder that reported on the difference in quality of life using the Quality-of-Life Inventory pre- and post-assessment between people receiving iCBT compared to waiting list. Particiapnts that received iCBT had a statistically significant improvement in quality of life at post assessment compared to waiting list (SMD = 0.38, 95% CI 0.08–0.67).
Patient Dropout and Treatment Adherence
Dedert et al23 reported treatment adherence as the percentage of patients completing all planned sessions stratified by the level of support. The authors identified two randomized controlled trials that reported on treatment adherence for generalized anxiety disorder, and there was variation based on the level of support provided alongside iCBT. Support provided alongside iCBT included feedback by a technician (nonlicensed staff) or clinician (licensed professional), based on the participant's previous interactions with the program, and psychoeducation. This type of support was delayed (not live). Live support comprised phone sessions, a scheduled chat on internet forums, or instant messaging with either technicians or clinicians. Seventy-five percent of participants completed all sessions when there was live support, compared to only 11% when support was provided alongside iCBT.23 These results suggest that participants who received instant communication with either technicians or clinicians completed more sessions than those who received delayed communication.
Internet-Delivered CBT Compared to a Combination of Usual Care, Waiting List, and/or Information Control
Symptoms and Response to Treatment
Andrews et al21 identified 12 randomized controlled trials that reported on changes in symptom severity for panic disorder. The control group was comprised of both usual care and waiting list participants. There was a statistically significant improvement in symptoms of panic disorder among participants who received iCBT compared to control groups (Hedges' g = 1.31; 95% CI 0.85–1.76]).
Dedert et al23 identified seven randomized controlled trials that reported on panic disorder. The control group was defined as waiting list, usual care, or attention/information control. Attention/information controls received support or psychoeducation on the symptoms or disorder being targeted. A large SMD, indicative of a significant reduction in symptoms, was found in favour of participants who received iCBT compared to waiting list, usual care, or attention control (SMD = −1.08, 95% CI −1.45 to −0.72).23
Andrews et al21 identified nine randomized controlled trials that reported on improvement in symptoms for generalized anxiety disorder. Internet-delivered CBT demonstrated a statistically significant improvement in symptoms compared to usual care and waiting list (Hedge's g = 0.70, 95% CI 0.39–1.01).
Andrews et al21 also identified 11 randomized controlled trials that reported on changes in symptom severity for social anxiety disorder. Internet-delivered CBT demonstrated a statistically significant improvement in symptoms compared to usual care and waiting list (Hedge's g = 0.92, 95% CI 0.76–1.08).
Quality of Life
Dedert et al23 identified three randomized controlled trials that reported on health-related quality of life for generalized anxiety disorder. A statistically significant standardized mean difference was found in favour of iCBT compared with control (SMD = 0.57, 95% CI 0.27–0.87).23
Dedert et al23 identified six randomized controlled trials that reported on health-related quality of life for panic disorder. A statistically significant standardized mean difference was found in favour of iCBT compared to control (SMD = 0.49; 95% CI 0.23–0.75).
Patient Dropout and Treatment Adherence
Dedert et al23 reported treatment adherence as the percentage of patients completing all planned sessions and as the mean number of sessions completed stratified by the level of support. Dedert et al23 also identified four randomized controlled trials that reported on treatment adherence for panic disorder, and there was variation based on the level of support. Support provided alongside iCBT included feedback provided by a technician (nonlicensed staff) or clinician (licensed professional) based on the participant's previous interactions with the program, and psychoeducation. Live support comprised phone sessions, a scheduled chat on internet forums, or instant messaging with either technicians or clinicians. Eighty percent of participants completed sessions when there was live support, and 24% to 95% of participants completed sessions when there was delayed support.
Internet-Delivered CBT Compared to Face-to-Face Cognitive Behavioural Therapy
Symptoms and Response to Treatment
Dedert et al reported23 data on four randomized controlled trials on panic disorder that assessed changes in symptoms. There was variation in the face-to-face CBT ranging from ten 2-hour group sessions in one study and 6 to 12 individual sessions in three other studies. Three randomized controlled trials provided support (e.g., feedback on the participant's previous interactions with iCBT and/or psychoeducation), and one trial used a mobile palmtop as an adjunct to face-to-face CBT. The authors found no statistically significant difference in symptoms of panic disorder for participants who had undergone iCBT compared with those who had undergone face-to-face CBT for three randomized controlled trials (SMD = 0.06, 95% CI −0.19 to 0.31).23 Similarly, one randomized controlled trial that provided iCBT as an adjunct to face-to-face CBT found no difference in symptoms of panic disorder compared with face-to-face CBT (SMD = −0.42, 95% CI −0.87 to 0.02).
Arnberg et al20 found no difference in symptoms for panic disorder between individuals randomized to guided iCBT that included therapist (i.e., clinical) support when compared to group face-to-face CBT.20 Noninferiority was not established as the confidence interval included the predefined noninferiority margin of d = −0.20. Similarly, no difference was found in symptoms of panic disorder between individuals randomized to guided iCBT that included therapist (i.e., clinical) support compared with individual face-to-face CBT. This study was not designed to assess noninferiority.
Arnberg et al20 identified one randomized controlled trial that reported on individuals with social phobia. There was a statistically significant difference in symptoms of social phobia for participants who received guided iCBT that included therapist (i.e., clinical) support when compared to group face-to-face CBT (d = 0.41, 95% CI 0.03 to 0.78).
Quality of Life
Dedert et al23 identified three randomized controlled trials that reported on health-related quality of life for panic disorder, comparing iCBT to face-to-face CBT. There was no statistically significant difference in quality of life for participants who received iCBT compared to face-to-face CBT (SMD = −0.07, 95% CI −0.34 to 0.21).
Internet-Delivered CBT Compared to Active Control
Symptoms and Response to Treatment
Kampmann et al25 reported on symptoms of social anxiety disorder, comparing iCBT with active control. The authors did not define the active control condition.
Internet-delivered CBT demonstrated a small to medium statistically significant improvement in symptoms of social anxiety disorder compared to active control conditions post-assessment (Hedges's g = 0.38, 95% CI 0.13–0.62). Similarly, a small to medium statistically significant improvement in symptoms of social anxiety disorder occurred compared to active control conditions at ≥6 months follow-up (Hedges's g = 0.23, 95% CI 0.04–0.43).25
Quality of Life
Kampmann et al25 found a small statistically significant improvement in quality of life in favour of iCBT compared to active control conditions post-assessment (Hedges'g = 0.30, 95% CI 0.10–0.50).
Internet-Delivered CBT Compared to Passive Control
Symptoms and Response to Treatment
Kampmann et al25 reported on symptoms of social anxiety disorder, comparing iCBT with passive control. However, the authors did not define the passive control condition. The authors of the systematic review reported that, compared with passive control conditions, iCBT demonstrated a large statistically significant improvement in symptoms of social anxiety disorder at post-assessment (Hedges's g = 0.84; 95% CI 0.72–0.97). An exploratory analysis of two studies did not show a statistically significant improvement in symptoms of social anxiety disorder for iCBT relative to passive control conditions until 5 months (Hedges's g = 0.12; 95% CI −0.17 to 0.42).25
Quality of Life
Kampmann et al25 found a moderate statistically significant improvement in quality of life in favour of iCBT compared to passive control (Hedges's g = 0.57; 95% CI 0.2–0.93).
See Table 3 for a summary of results for anxiety disorders.
Table 3:
Summary of Results of Included Systematic Reviews for Anxiety Disorders
| Author, Year | No., Type of Studies/No. of Participants | Results | Quality Assessment |
|---|---|---|---|
| Internet-delivered CBT compared with waiting list | |||
| Symptoms and Response to Treatment | |||
| Arnberg et al, 201420 |
Generalized anxiety disorder 4 RCTs/132 |
Guided iCBT vs. waiting list SMD = 0.84, 95% CI 0.45–1.23 |
GRADE: ⊕⊕ low |
| Adelman et al, 20147 |
Generalized anxiety disorder 4 RCTs/317 |
iCBT vs. waiting list SMD = 1.06, 95% CI 0.82–1.30 |
Jadad score unclear for the subgroup of studiesa |
| Dedert et al, 201323 |
Generalized anxiety disorder 4 RCTs/321 |
iCBT vs. waiting list SMD = −0.94, 95% CI −1.34 to −0.54 |
Stregnth of evidence: moderate |
| Arnberg et al, 201420 |
Panic disorder 4 RCTs/132 |
Guided iCBT vs. waiting list Small to very large effects |
GRADE: ⊕⊕ low |
| Adelman et al, 20147 |
Panic disorder 8 RCTs/406 |
iCBT vs. waiting list SMD = 1.15, 95% CI 0.94–1.37 |
Jadad score unclear for the subgroup of studiesa |
| Arnberg et al, 201420 |
Social phobia 8 RCTs/356 |
Guided iCBT vs. waiting list SMD = 0.85, 95% CI 0.66–1.05 |
GRADE: ⊕⊕⊕ moderate |
| Adelman et al, 20147 |
Social anxiety disorder 9 RCTs/not specified |
iCBT vs. waiting list SMD = 0.91, 95% CI 0.74–1.07 |
Jadad score unclear for the subgroup of studiesa |
| Quality of Life | |||
| Richards et al, 201524 |
Generalized anxiety disorder 2 RCTs/157 |
SMD = 0.38, 95% CI 0.08–0.67 | Risk of bias using the Cochrane Collaboration criteria was not reported for the subgroup of studies |
| Patient Dropout/Adherence | |||
| Dedert et al, 201323 |
Generalized anxiety disorder 2 RCTs/137 |
Guided iCBTb: 75% completion Guided iCBTc: 11% completion |
Strength of evidence not reported |
| Internet-delivered CBT compared with a combination of usual care, waiting list, and/or information control | |||
| Symptoms and Response to Treatment | |||
| Andrews et al, 201821 |
Panic disorder 12 RCTs/584 |
Hedges's g = 1.31, 95% CI 0.85–1.8 |
Low/unclear |
|
Social anxiety disorder 11 RCTs/950 |
Hedges's g = 0.92, 95% CI 0.76–1.08 |
Low/unclear | |
|
Generalized anxiety disorder 9 RCTs/1,103 |
Hedges's g = 0.70, 95% CI 0.39–1.0 |
Low | |
|
Panic disorder 7 RCTs/333 |
SMD = −1.08, 95% CI −1.45 to −0.72 |
Strength of evidence: moderate | |
| Quality of Life | |||
| Dedert et al, 201323 |
Generalized anxiety disorder 3 RCTs/176 |
SMD = 0.57, 95% CI 0.27–0.87 | Strength of evidence: low |
|
Panic disorder 6 RCTs/250 |
SMD = 0.49, 95% CI 0.23–0.75 | Strength of evidence: moderate | |
| Patient Dropout and Treatment Adherence | |||
| Dedert et al, 201323 |
Panic disorder 4 RCTs/313 |
80% of participants completed sessions when there was live support; 24–90% completed sessions when there was delayed support | Strength of evidence: not reported |
| Internet-delivered CBT compared with face-to-face CBT | |||
| Symptoms and Response to Treatment | |||
| Arnberg et al, 201420 |
Specific phobia 1 RCT/30 |
Guided iCBT vs. brief therapist-led exposure No change (no effect size reported) |
GRADE ⊕ very low |
|
Social phobia 1 RCT/126 |
Guided iCBT vs. group face-to-face CBT d = 0.41, 95% CI 0.03–0.78 |
GRADE: ⊕⊕ low | |
| Dedert et al, 201323 |
Panic disorder 1 RCT/49 |
Guided iCBT vs. individual face-to-face CBT Not statistically significant |
GRADE: ⊕ very low |
|
Panic disorder 1 RCT/113 |
Guided iCBT vs. group face-to-face CBT d = 0.00 (95% CI −0.41 to 0.41) |
GRADE: ⊕ Very low | |
|
Panic disorder 3 RCTs/248 |
iCBT vs. face-to-face CBT SMD = 0.06, 95% CI −0.19 to 0.31 |
Strength of evidence: moderate | |
| 1 RCT/121 | SMD = −0.42, 95% CI −0.87 to 0.02d | ||
| Quality of Life | |||
| Dedert et al, 201323 |
Panic disorder 3 RCTs/239 |
SMD = −0.07, 95% CI −0.34 to 0.21 | Strength of evidence: moderate |
| Internet-delivered CBT compared with active control | |||
| Symptoms and Response to Treatment | |||
| Kampmann et al, 201625 |
Social anxiety disorder Number of studies not specified/number of participants not specified |
A small to medium effect was found when iCBT was compared with active control conditions at post-assessment (Hedges's g = 0.38, 95% CI 0.13–0.62, SE = 0.13, P < 0.01, k = 8) and at follow-up 2e (Hedges's g = 0.23, 95% CI 0.04–0.43, SE = 0.10, P = 0.02; k = 5) | Risk of bias using the Cochrane Collaboration criteria was not reported for the subgroup of studies |
| Quality of Life | |||
| Kampmann et al, 201625 |
Social anxiety disorder Number of studies not specified/number of participants not specified |
Hedges's g = 0.30, 95% CI 0.10–0.50, SE = 0.10, P < 0.01, k = 3 | Risk of bias using the Cochrane Collaboration criteria was not reported for the subgroup of studies |
| Internet-delivered CBT compared to passive control | |||
| Symptoms and Response to Treatment | |||
| Kampmann et al, 201625 |
Social anxiety disorder Number of studies not specified/number of participants not specified |
Hedges's g = 0.84, 95% CI 0.72–0.97, SE = 0.07, P < 0.001, k = 16 Exploratory analysis including only two studies of iCBT relative to passive control conditions at follow-up 1f (Hedges's g = 0.12, 95% CI −0.17 to 0.42, SE = 0.15, P = 0.412, k = 2) |
Risk of bias using the Cochrane Collaboration criteria was not reported for the subgroup of studies |
| Quality of Life | |||
| Kampmann et al, 201625 |
Social anxiety disorder Number of studies not specified/number of participants not specified |
A medium effect for iCBT compared to passive control (Hedges's g = 0.57, 95% CI 0.21–0.93, SE = 0.31, P < 0.01, k = 2) | Risk of bias using the Cochrane Collaboration criteria was not reported for the subgroup of studies |
Abbreviations: CI, confidence interval; iCBT, internet-delivered cognitive behavioural therapy; SE, standard error; SMD, standardized mean difference.
The authors of the health technology assessment were unable to extract the Jadad score for the subgroup of studies.
Real-time interactions with study technicians (nonlicensed staff) or clinicians (licensed professionals), including phone sessions, a scheduled chat on internet forums, or instant messaging.
Real-time communication with technician (nonlicensed staff) or clinician (licensed professionals) was delayed.
Internet-delivered CBT as an adjunct vs. face-to-face CBT.
Follow-up period was defined as 6 months and greater.
Follow-up period was defined as less than 5 months.
Adverse Events
Dedert et al23 planned to assess outcomes of safety of iCBT across disorders (e.g., emergency department visits, hospital admissions related to the disorder being treated, and self-harm behaviors); however, the data were unavailable from the included studies and were not reported by the systematic review authors. No other systematic reviews included in this assessment reported any data on safety.
Discussion
Internet-delivered CBT is one type of psychotherapy for the treatment of mild to moderate major depression and anxiety disorders. Of the seven systematic reviews included in this clinical evidence review, we found little evidence on guided iCBT. None of the systematic reviews presented data on unguided iCBT. One systematic review evaluated the noninferiority of iCBT compared to group face-to-face CBT and individual face-to-face CBT for anxiety disorders.20 Overall, the level of support (guided or unguided) associated with iCBT in the included systematic reviews was limited.
There is moderate-quality evidence suggesting that, compared with waiting list, guided iCBT is effective and likely results in symptom improvement for mild to moderate major depression and social phobia. Compared with waiting list, guided iCBT may also improve the symptoms of generalized anxiety disorder and panic disorder. However, there is uncertainty about the effectiveness of iCBT compared with individual or group face-to-face CBT.
Although there was a statistically significant improvement in quality of life for individuals with generalized anxiety disorder who received iCBT compared to waiting list, these results should be interpreted with caution as the lower limit of the 95% confidence interval (i.e., 0.08) is very close to the zero cut-off. While iCBT demonstrated an improvement in quality of life for people with generalized anxiety disorder or panic disorder compared to a combination of usual care, waiting list, and information control, the strength of evidence is low to moderate.
Guided iCBT may provide a promising option for the treatment of mild to moderate major depression and select anxiety disorders. Moreover, iCBT may expand treatment for individuls unable or unwilling to access face-to-face CBT.
Limitations
The characteristics of the study populations and the number of randomized controlled trials for each subgroup analysis varied in the included systematic reviews. Characteristics of the study participants included a high level of educational attainment and employment, and recruitment was via the internet or advertisements.20 The potential selection bias identified limits the generalizability of these findings. Furthermore, results from single-trial efficacy studies conducted on guided iCBT compared to group face-to-face CBT and individual face-to-face CBT for panic disorder, and guided iCBT compared to brief therapist-led exposure for specific phobias have limited external validity due to the small sample sizes.20
We undertook an overview of systematic reviews. There may be randomized controlled trials published since then that we have not captured. A rapid review by CADTH provides an assessment of the randomized controlled trials published since the systematic reviews assessed in this report.27
Conclusions
Compared with usual care, iCBT:
Significantly improves symptoms for mild to moderate major depression (GRADE not conducted)
Compared with waiting list, guided iCBT:
Significantly improves symptoms of mild to moderate major depression (GRADE moderate)
Significantly improves symptoms of generalized anxiety disorder (GRADE low)
Significantly improves symptoms of panic disorder (GRADE low)
Significantly improves symptoms of social phobia (GRADE moderate)
Compared with face-to-face CBT, guided iCBT:
Did not significantly improve symptoms of panic disorder (GRADE very low)
Did not significantly improve symptoms of social phobia (GRADE low)
ECONOMIC EVIDENCE
Research Questions
What is the cost-effectiveness of unguided or guided iCBT compared with face-to-face CBT or usual care in the management of adults with mild to moderate major depression?
What is the cost-effectiveness of unguided or guided iCBT compared with face-to-face CBT or usual care in the management of adults with anxiety disorders?
Methods
Economic Literature Search
We performed an economic literature search on February 21, 2018, for studies published from January 1, 2000, to the search date. To retrieve relevant studies, we developed a search using the clinical search strategy with an economic filter applied.
We created database auto-alerts in MEDLINE, Embase, PsycINFO, and CINAHL and monitored them for the duration of the health technology assessment. We performed a targeted grey literature search of health technology assessment agency websites, PROSPERO, and Tufts Cost-Effectiveness Analysis Registry. See the Clinical Evidence literature search, above, for further details on methods used. See Appendix 1 for literature search strategies, including all search terms.
Literature Screening
A single reviewer screened titles and abstracts, and, for those studies likely to meet the eligibility inclusion criteria, we obtained full-text articles and performed further assessment for eligibility.
Inclusion Criteria
English-language, full text articles published between January 1, 2000, and the search date
Individual-level economic evaluations conducted alongside randomized controlled trials (i.e., trial-based) or economic analyses based on decision analytic models (i.e., model-based)
Studies in adults with major depression or an anxiety disorder that falls within the DSM-5 criteria
Studies comparing unguided or guided iCBT (by a therapist or a coach) with face-to-face CBT, pharmacologic therapies, treatment as usual, or no treatment (e.g., waiting list or placebo)
Exclusion Criteria
Narrative reviews of the literature, study protocols, guidelines, conference abstracts, commentaries, letters, and editorials
Economic evaluations examining iCBT used solely for the treatment of postnatal depression or depression or anxiety co-occurring with chronic conditions (e.g., depression coexisting with chronic diseases such as cardiovascular disease, dementia, diabetes mellitus, or inflammatory bowel disease)
Economic evaluations examining the value of non-traditional CBT (e.g., mindfulness CBT), CBT delivered via bibliotherapy, or CBT described as computerized CBT for which no further analysis was done specifically for iCBT
Feasibility studies exploring different models of care for the treatment of major depression or anxiety disorders that do not report economic outcomes
Noncomparative studies reporting the costs of iCBT or cost-of-illness studies
Outcomes of Interest
Incremental costs
Incremental effectiveness outcomes (e.g., quality-adjusted life-years [QALYs], disability-adjusted life-years [DALYs])
Incremental cost-effectiveness ratio (ICER)
Incremental net benefit (INB)
Data Extraction
We extracted relevant data on the following:
Publication source (i.e., author's name, location, publication year)
Study design
Study population
Interventions and comparators
Outcomes (e.g., health outcomes, costs, and incremental cost-effectiveness ratio)
Study Applicability and Limitations
We determined the usefulness of each identified study for decision-making by applying a modified quality appraisal checklist for economic evaluations that was originally developed by the National Institute for Health and Care Excellence (NICE) in the United Kingdom to inform development of NICE's clinical guidelines. We modified the wording of the questions to remove references to guidelines and to make it Ontario-specific. Next, we separated the checklist into two sections. In the first section, we assessed the applicability of each study to the research question (directly, partially, or not applicable). A summary is presented in Appendix 4. In the second section, we assessed the limitations (minor, potentially serious, or very serious) of the studies that we found to be directly or partially applicable.
Results
Literature Search
The literature search yielded 504 citations published from January 1, 2000, to February 21, 2018, after removing duplicates. We excluded a total of 456 articles based on information in the title and abstract and obtained 48 potentially relevant articles for further assessment. Forty studies met the inclusion criteria and were assessed to establish the applicability of their findings to the Ontario context. Figure 2 presents the PRISMA diagram for the economic evidence review.
Figure 2: PRISMA Flow Diagram—Economic Search Strategy.
Source: Adapted from Moher et al.26
Review of Included Economic Studies
Of the 40 eligible studies, 9 were systematic literature reviews20,28–35 that examined the cost-effectiveness of iCBT among other psychological therapies for either major depression or anxiety disorders, and 31 were original research studies focusing on the cost-effectiveness of iCBT in adults with major depression (n = 15)36–50 or in adults with an anxiety disorder (n = 16).9,50–64
Systematic Reviews: The Cost-Effectiveness of iCBT for the Management of Major Depression or Anxiety Disorders
We identified nine relevant systematic reviews including data on the potential cost-effectiveness of iCBT published between 2012 and 2018.20,28–35 All reviews examined other psychological treatments in addition to iCBT: four reviews were focused on major depression,28–31 two examined anxiety disorders,32,33 and three reviews examined internet interventions for mental health conditions in general or for mood and anxiety disorders altogether.20,34,35 The reviews examining depression or anxiety disorders included mixed study populations and types of psychological treatments in addition to guided and unguided iCBT, which were examined in a wide range of included studies. In general, results were mixed. Most reviews suggested that iCBT could represent an economically viable treatment alternative over control.30 The one exception, Kolovos et al,30 included individual-level participant data and conducted a meta-analysis and concluded that guided iCBT was not a cost-effective option compared with control. None of the studies described in detail the differences between unguided and guided iCBT, and there was a great variability of cost-effectiveness estimates. We performed our own systematic review of the literature to inform our primary economic evaluation and to gain a better understanding of relevant values for our model input parameters, such as changes in health-related quality of life weights associated with progression of depression and anxiety and with iCBT treatment.
We reviewed all original studies relevant to our research questions. We then compared their study designs, including perspective, the time horizon of analysis, study populations, and comparative strategies. Lastly, we summarized the cost-effectiveness findings. Tables 4 and 5 summarize the characteristics and results of the included original studies.
Table 4:
Results of Economic Literature Review—Cost-Effectiveness of iCBT for the Treatment of Major Depression
| Name, Year, Location | Methods | Results | ||||
|---|---|---|---|---|---|---|
| Study Design and Perspective | Population | Interventions/Comparators | Health Outcomes | Costs | Cost-Effectiveness | |
| Duarte et al, 201737; Littlewood et al, 2015,36 United Kingdom |
|
|
|
|
|
|
| Romero-Sanchiz et al, 2017,43 Spain |
|
|
|
|
|
|
| Lee et al, 2017,49 Australia |
|
|
|
|
|
|
| Brabyn et al, 2016,38 United Kingdom |
|
|
|
|
|
|
| Dixon et al, 2016,39 United Kingdom |
|
|
|
|
|
|
| Titov et al, 2015,44 Australia |
|
|
|
|
|
|
| Geraedts et al, 2015,45 Netherlands |
|
|
|
|
|
|
| Solomon et al, 2015,48 Australia |
|
|
|
|
|
|
| Phillips et al, 2014,41 United Kingdom |
|
|
|
|
|
|
| Gerhards et al, 2010,46 Netherlands |
|
|
|
|
|
|
| Warmerdam et al, 2010,47 Netherlands |
|
|
|
|
|
|
| Hollinghurst et al, 2010,40 United Kingdom |
|
|
|
|
|
|
| Kaltenthaler et al, 2006,50 United Kingdom |
|
|
|
|
|
|
| McCrone et al, 2004,42 United Kingdom |
|
|
|
|
|
|
Abbreviations: CCBT, computerized CBT; EVPI, expected value of perfect information; EVPPI, expected value of partial perfect information; GP, general practitioner; iCBT, internet-delivered cognitive behavioural therapy; ICER, incremental cost-effectiveness ratio; INB, incremental net benefit; NHS, National Health Service; NICE, National Institute of Health and Care Excellence; NR, not reported; PSS, personal social services; PST, problem-solving therapy; QALY, quality-adjusted life-year; RCT, randomized controlled trial; REEACT, Randomised Evaluation of the Effectiveness, cost-effectiveness and Acceptability of Computerised Therapy; ROI, return on investment; SD, standard deviation; SD, standard deviation; TAU, treatment as usual; wtp, willingness to pay.
Table 5:
Results of Economic Literature Review—Cost-Effectiveness of iCBT in the Treatment of Anxiety Disorders
| Name, Year, Location | Methods | Results | ||||
|---|---|---|---|---|---|---|
| Study Design and Perspective | Population | Interventions/Comparators | Health Outcomes | Costs | Cost-Effectiveness | |
| Kumar et al, 2018,63 United States |
|
|
|
|
|
|
| El Alaoui et al, 2017,61 Sweden |
|
|
|
|
|
|
| Hedman et al, 2016,60 Sweden |
|
|
|
|
|
|
| Dear et al, 2015,62 Australia |
|
|
|
|
|
|
| Nordgren et al, 2014,56 Sweden |
|
|
|
|
|
|
| Hedman et al, 2014,59 Sweden |
|
|
|
|
|
|
| Hedman et al, 2013,57 Sweden |
|
|
|
|
|
|
| National Collaborating Centre for Mental Health and NICE, 2013,64 United Kingdom |
|
|
|
|
|
|
| Joesch et al, 2012,55 United States |
|
|
|
|
|
|
| Hedman et al, 2011,58 Sweden |
|
|
|
|
|
|
| National Collaborating Centre for Mental Health and NICE, 2011,9 United Kingdom |
|
|
|
|
|
|
| Bergstrom et al, 2010,54 Sweden |
|
|
|
|
|
|
| Titov et al, 2009,53 Australia |
|
|
|
|
|
|
| McCrone et al, 2009,52 United Kingdom |
|
|
|
|
|
|
| Mihalopoulos et al, 2005,51 Australia |
|
|
|
|
|
|
| Kaltenthaler, 2006,50 United Kingdom |
|
|
|
|
|
|
Abbreviations: CALM, Coordinated Anxiety Learning and Management; CBT, cognitive behavioural therapy; CCBT, computerized CBT; DALY, disability-adjusted life-year; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, 4th edition; EQ-5D, European Quality of Life questionnaire in five dimensions; ESEMeD, European Study of the Epidemiology of Mental Disorders; GAD, generalized anxiety disorder; GP, general practitioner; iCBT, internet-delivered cognitive behavioural therapy; ICER, incremental cost-effectiveness ratio; INB, incremental net benefit; NA, not applicable; NHS, National Health Service; NICE, National Institute of Health and Care Excellence; NR, not reported; PSS, personal social services PST, problem-solving therapy; QALY, quality-adjusted life-year; RCT, randomized controlled trial; ROI, return on investment; SAD, social anxiety disorder; SD, standard deviation; SF-6D, Short-Form Health Survey in 6 Dimensions; TAU, treatment as usual; wtp, willingness to pay; YLD, years lived with disability.
Original Studies: The Cost-Effectiveness of iCBT for the Management of Mild to Moderate Major Depression
Study Design
Of the 15 included studies (see Table 4), 12 were individual-level cost-effectiveness analyses conducted alongside clinical trials. Six were from the United Kingdom36–42 (the report by Littlewood et al36 was later published by Duarte et al37; it was counted as the same publication in Table 4), one study was from Spain,43 one from Australia,44 and three were from the Netherlands.45–47 Three studies were model-based cost-effectiveness analyses (two from Australia48,49 and one from the United Kingdom50).
Study Population
All economic evaluations were done with adult participants who had major depression, and the majority included mild to moderate disease severity in the inclusion criteria. Prior episodes of major depression and use of medications at baseline was reported by participants in four studies.36–40 One study was specifically conducted in older adults aged 60+ years,44 and another study45 targeted employed individuals (mean age: 43 years).45 The remaining studies recruited participants from hospitals or the general population. All but one study in older Australian adults44 included relatively large samples of about 300 to 700 participants, with the mean age ranging from 34 to 50 years, and with females composing between 50% and 80% of the study populations.
Analysis Perspective
The analysis perspective varied between the studies and may have been affected by the features of each country's health care system. Most studies conducted in the United Kingdom were done from a health sector perspective. Most of the studies done in the Netherlands, Australia, and Spain were done from a societal perspective.
Time Horizon
The duration of follow-up in trial-based cost-effectiveness analyses or of the time horizon in model-based cost-effectiveness analyses was short, ranging from 8 weeks to 12 months in most studies. This indicates a short-term, non-repetitive use of iCBT. Only two cost–utility analyses (one trial-based,36,37 and another model-based50) examined the benefits and costs of 6- to 8-session courses of unguided or coach-guided iCBT over a period of 18 months.
Interventions and Comparators
Unguided iCBT. Six studies used unguided iCBT as an intervention strategy. The unguided iCBT consists of 4 to 8 sessions, along with homework assignments. In most studies, the sessions were designed to be completed weekly. In three individual-level cost-effectiveness studies, usual care was compared with unguided or minimally guided iCBT (where minimal guidance was defined as technical guidance on use of the software).36,37,41,43,46 One study by Gerhards et al46 compared iCBT alone to usual care alone and to iCBT combined with usual care. Another study by McCrone et al42 compared iCBT combined with usual care to usual care alone. Only one model-based study by Solomon et al48 examined the benefits and costs of unguided iCBT to usual care and to face-to-face CBT.48
Guided iCBT. In total, nine studies examined therapist-guided iCBT.38–40,43–45,47,49,50 Durations varied from 6 to 16 sessions completed on a weekly basis, along with homework assignments. In four studies,38,39,47,50 including three from the United Kingdom, iCBT was guided by a coach, a non-regulated mental health worker trained to provide low-intensity CBT. Highly trained regulated therapists (e.g., clinicians or psychologists) were employed in five studies done in Spain,43 the Netherlands,45 Australia44,49 and the UK40. In seven studies, guided iCBT was compared with usual care only. One study47 compared guided iCBT with problem-solving therapy;47 another38 compared guided iCBT supported by a coach (i.e., a non-regulated mental health worker) with unguided iCBT.38
Control Treatment. In general, usual care consisted of treatment provided by a general practitioner and included active surveillance, medications or counselling, and psychoeducation according to country-specific clinical guidelines. Another frequently used control option was waiting list. One study by Phillips et al41 that examined unguided iCBT used a self-help mental health website as a control comparator.41
Cost-Effectiveness Results
In 11 of the 14 included studies, ICER estimates associated with the incremental cost-effectiveness of unguided or guided iCBT were below country-specific willingness-to-pay thresholds, indicating that these types of CBT could be economically attractive options in the management of mild to moderate depression. However, results should be interpreted with caution as there were some uncertainties around the ICER in the probabilistic sensitivity analyses. Depending on the duration and type of cost–utility analysis, the probability of unguided or guided iCBT being cost-effective ranged from 52% to more than 95% (at country-specific willingness to pay thresholds). Moreover, the probability of cost-effectiveness of guided iCBT versus unguided iCBT was 55% at a willingness to pay of £30,000/QALY.38
In most studies, the costs associated with guided iCBT were greater than the cost of usual care; however, the incremental benefits (expressed in QALYs) remained uncertain. For instance, in several studies, unguided or guided iCBT alone was associated with increments of 0.01 to 0.02 QALYs. Comparatively, slightly larger increases of 0.03 to 0.04 QALYs were shown in two studies that examined the cost-effectiveness of guided or unguided iCBT when combined with usual care.40,42 Exceptionally high increases in QALYs were also determined in a modeling study by Kaltenthaler et al50 and an individual-level cost–utility analysis by Romero-Sanchiz.43 In these studies, increments in QALYs were shown to be about 0.08 for guided or unguided iCBT compared with usual care. Lastly, a study by Brabyn et al38 that compared guided to unguided iCBT found a small gain in QALYs of 0.003 with guided iCBT.
Three cost–utility analyses suggested that unguided or guided iCBT is probably economically unattractive compared with usual care for the management of mild to moderate depression36,37,39,45 (see Table 5). A cost–utility analysis alongside a large randomized controlled trial of 691 participants with prevalent or new major depression36,37 found that two commercially available unguided iCBT software programs (“MoodGYM” and “Beating the Blues”), which included 6 to 8 online sessions with homework assignments, were no better than usual care after 18 months of follow-up.36,37 Another cost–utility analysis alongside a large randomized controlled trial of 609 participants with mild to moderate major depression39 found that coach-guided iCBT software (“Living Life to the Full”) was not cost-effective compared with usual care after 12 months of follow-up.39 After accounting for a large amount of missing data, the reported ICER was £132,630/QALY, and the probability of cost-effectiveness of iCBT versus usual care at a willingness-to-pay threshold of £20,000/QALY was 30%.39 In contrast, the results were economically attractive in the complete case analysis (ICER: £3,850/QALY), but the authors indicated that the results of this complete-case analysis should be considered biased.
Gerhards et al45 examined the cost-effectiveness of guided iCBT (software: “Happy@Work”) in a convenience sample of 231 employed adults with mild to moderate depression and found no incremental benefit of guided iCBT versus usual care. With the mean difference in QALYs estimated at −0.001 (95% CI −0.04 to 0.04), the authors determined that iCBT did not seem to be economically attractive despite some cost savings. Overall, the study estimated an ICER of €533,000/QALY lost—a savings of over €530,000 per QALY lost (Table 4). Finally, in a cost–utility modeling study by Solomon et al48, unguided iCBT (software: “My Compass”) was cost-saving compared with usual care; however, despite decreases in the mean costs of about $2,000, due to a decrement in the mean QALYs of 0.03, unguided iCBT was found to be less cost-effective than face-to-face CBT at the commonly used willingness-to-pay threshold of $50,000 AUD/QALY in Australia. An ICER of an estimated $66,000 AUD/QALY indicated a savings of $66,000 AUD per QALY lost.
Original Studies: The Cost-Effectiveness of iCBT for the Management of Anxiety Disorders
Study Design
Of the 16 included studies (see Table 5), 11 were individual-level cost-effectiveness analyses conducted alongside randomized controlled trials (seven from Sweden,54,56–61 two from Australia,53,62 one from the United States,55 and one from the United Kingdom52), and five were model-based cost-effectiveness analyses (one from the United States,63 three from the United Kingdom,9,50,64 and one from Australia51). Some studies were performed by the same research groups, presenting the short-term and long-term results in separate publications. For example, Hedman at al (2011),58 Hedman at al (2014),58,59 and El Alaoui et al (2017)61 reported 6-month to 4-year follow-up results for social anxiety disorder. Hedman et al (2013)57 and Hedman et al (2016)57,60 reported 12-week and 1-year follow-up results for health anxiety disorder.
Study Populations
A majority of the economic evaluations we found focused on adults with panic disorder (n = 5)9,50–52,54 and social anxiety disorder (n = 5),53,57,58,61,64 followed by generalized anxiety disorder (n = 2),62,63 severe health anxiety (n = 2),57,60 and other anxiety (n = 2).55,56 All economic evaluations alongside randomized controlled trials that examined iCBT as a single intervention had small to moderately large samples of 72 and 193 participants. The exception was a study by Joesch et al55 examining the cost-effectiveness of therapist-guided iCBT for any anxiety disorder within a collaborative care program, which included a sample of 690 participants. In many of the studies, participants were between 34 and 49 years of age and were predominantly female (> 70%). Almost all study participants had a long-standing anxiety disorder, used medications, and/or had already been treated with a psychological therapy.
Analysis Perspective
The analytic perspective varied between studies. Most studies conducted in Sweden adopted a societal perspective, while studies conducted in the United Kingdom, Australia, and the United States adopted a health sector perspective.
Time Horizon
The duration of follow-up depended on the type of anxiety disorders even though the course of an iCBT program was short and occurred only one time. Studies of adults with panic disorder lasted from 6 weeks to 12 months. Studies of adults with social phobia lasted for 6 months to 5 years. Studies of middle-aged adults with severe health anxiety, any anxiety, and of older adults with GAD lasted for 8 weeks to 18 months. Only one modeling study63 examined the lifetime cost-effectiveness of iCBT in adults with GAD. Kumar et al63 conducted a Markov cohort decision analytic study assuming continuous benefits of iCBT over 5 years based on the estimates from a 3-month pilot coach-based iCBT program (software: “Lantern”).
Interventions and Comparators
Unguided iCBT. Only three studies, two model-based50,64 and one trial-based,52 examined the cost-effectiveness of unguided iCBT versus control in adults with social anxiety or panic disorders. The duration of iCBT course was between 6 and 12 weeks.
Guided iCBT. The remaining 13 studies examined the cost-effectiveness of guided iCBT over the control. One of these studies was embedded within a collaborative care program that also included case management and coordination of psychiatric care.55 Overall, the iCBT programs consisted of 5 to 15 modules with homework assignments that were taken over a period of 8 to 15 weeks.
Control. Benefits and costs of iCBT programs were compared with treatment as usual, including medications, waiting list control, online relaxation/stress management programs or discussion forums, and face-to-face CBT (group or individual format).
Cost-Effectiveness Results
Most studies showed that guided or unguided iCBT for the treatment of anxiety disorders represented good value for money (i.e., cost-saving or cost-effective at a country-specific willingness-to-pay threshold) when compared with control. When compared with usual care, guided iCBT was associated with QALY increments from 0.01 to 0.05. Guided iCBT for social phobia had equal benefits to group CBT in noninferiority trials by Hedman et al.57,58 A 12-month modeling study by Kaltenthaler et al50 indicated a QALY increment of 0.06 for unguided iCBT (software: “FearFighter”) versus a computerized relaxation program, and a decrease in QALY of 0.01 for unguided iCBT versus individual face-to-face CBT. In many of the studies, the probability of cost-effectiveness of iCBT versus control was high and ranged from 70% to over 95%.
Applicability and Limitations of the Included Studies
Appendix 4 presents the results of the methodology checklist for economic evaluations applied to the included articles. (Tables A2 and A3 present data for the 14 evaluated cost-effectiveness studies for major depression, and Tables A4 and A5 present data for the 16 evaluated cost-effectiveness studies for anxiety disorders.) All studies were deemed partially applicable to our two research questions. We considered none of the included studies relevant or directly applicable for the Ontario or Canadian setting.
We also assessed the methodological quality of these studies. Regarding the 14 cost-effectiveness analyses of iCBT for major depression, six studies had minor limitations and the rest had potentially serious limitations (Appendix 4, Table A3). With respect to the 16 cost-effectiveness analyses of iCBT for anxiety disorders, one study had minor limitations, 12 studies had potentially serious limitations, and three had very serious limitations (Appendix 4, Table A5). Eligible psychological interventions (e.g., individual or group face-to-face CBT) were not compared against each other in most studies, which led to potential overestimation of the QALY associated with guided CBT, particularly in patients with depression. Also, the additional value of iCBT over usual care was difficult to estimate given that: 1) study populations were exposed to prior treatments (medications or psychotherapies), and 2) sample sizes were relatively small and selected.
In all studies, iCBT was provided over a short time frame (usually 3 months or less); as such, long-term extrapolations of its effect contain substantial uncertainty around the estimate of the ICER. Internet-delivered CBT had a larger QALY benefit when it was combined with usual care or provided within a collaborative care program. This creates the possibility of contamination of iCBT treatment effects when there is a cointervention with other case management approaches.
A few large studies had a substantial amount of missing data that biased the estimation of the ICER. Also, direct medical costs associated with iCBT and management of depression or anxiety were not estimated in some studies. Studies that conducted probabilistic sensitivity analyses did not explore in detail changes in some important input parameters (e.g., changes in utility values or effectiveness of the iCBT intervetions) that could affect the robustness of cost-effectiveness results. Modeling studies that examined the cost-effectiveness of guided iCBT over 5 years, or over the participants' lifetimes, made relatively bold assumptions regarding the constancy and equality of the treatment effect over time between face-to-face CBT and iCBT. These assumptions were without strong support in the evidence.
Discussion
Our review of the economic evidence suggests that guided iCBT probably represents good value for money for the management of adults with mild to moderate major depression or anxiety in the short-term (< 3 months). There is larger uncertainty around the economic estimates for people with mild to moderate major depression compared with anxiety disorders. Limited evidence from studies in patients with major depression suggests that unguided iCBT is as attractive as usual care and that guided iCBT provides a slight increase in QALYs when compared with unguided iCBT.
No study examined a repetitive use of iCBT for the treatment of recurrent episodes of major depression and anxiety despite the fact that these diseases are episodic and recurrent. This observation is in line with the current recommendations that are positioning iCBT as a first-step in a stepped-care treatment model.17
Some patients may prefer iCBT over face-to face CBT because it can be more compatible with their busy schedules or because they have a social phobia or other anxiety disorder. The greater privacy of iCBT may also reduce concerns around the stigma associated with mental illness.65
Given the similar course of both major depression and anxiety disorders, our economic evaluation aimed to determine whether guided or unguided iCBT represents good value for money over a short-term horizon and over a lifetime. More specifically, we examined the cost-effectiveness of guided versus unguided iCBT against all available treatment alternatives that are used in Ontario for the initial management of mild to moderate major depression or anxiety disorders. Finally, we explored the lifetime cost-effectiveness of a stepped-care approach over usual care, starting with guided iCBT in adults with mild to moderate major depression and anxiety disorders.
Conclusions
Our evidence synthesis of the economic studies revealed that when compared with usual care, guided iCBT probably represents good value for money for the short-term management of mild to moderate major depression or anxiety disorders (e.g., panic disorder, social phobia, generalized anxiety disorder, and health anxiety disorder). It is unclear whether unguided iCBT is economically attractive in the treatment of adults with mild to moderate major depression and anxiety disorders. Given limitations of the model-based studies with respect to the extrapolation of the treatment effect of iCBT long term, the lifetime cost-effectiveness of guided iCBT remains unclear and warrants further investigation in the Ontario context.
PRIMARY ECONOMIC EVALUATION
Our literature search identified 40 potentially relevant studies but none directly applicable to our research questions. Therefore, we conducted a primary economic evaluation.
Research Questions
Primary Research Questions
What is the cost-effectiveness of unguided or guided iCBT compared with face-to-face CBT or usual care in the management of adults with mild to moderate major depression?
What is the cost-effectiveness of unguided or guided iCBT compared with face-to-face CBT or usual care in the management of adults with anxiety disorders?
Secondary Research Questions
What is the cost-effectiveness of guided iCBT within a stepped-care model, including face-to-face CBT, compared with usual care in the management of adults with mild to moderate major depression or anxiety disorders?
What is the cost-effectiveness of unguided iCBT followed by guided iCBT compared with usual care in the management of adults with mild to moderate major depression or anxiety disorders?
Methods
The information presented in this report follows the reporting standards set out by the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement.66,67 The methodological approaches follow the recent recommendations set out by the Canadian Agency for Drugs and Technologies in Health (CADTH) Guidelines for the Economic Evaluation of Health Technologies, fourth edition,68 and align with Health Quality Ontario's Health Technology Assessments Methods and Process Guide.
Type of Analysis
For each research question, we conducted cost–utility analyses to estimate ICERs for the interventions versus comparator strategies. In these analyses, the ICER estimates are expressed as the cost per QALY gained.
Target Population
Our two study populations consist of individuals 18 years or older with a primary diagnosis of mild to moderate major depression or with an anxiety disorder as defined by the DSM-5 criteria.69 In addition, the baseline severity of the disease is determined by the score on the nine-item Patient Health Questionnaire (PHQ-9),70 which also corresponds to the diagnostic criteria of the DSM-5.71–74 Individuals with mild and moderate major depression account for approximately 80% of all individuals with major depression.75,76 The DSM-569 includes adults with the following anxiety disorders: specific phobias, social anxiety disorder (social phobia), panic disorder, agoraphobia, and generalized anxiety disorder. Anxiety disorders can exist in isolation, but they coexist with another type of anxiety disorder in more than 70% of patients, including about 50% who also have major depression. Since the treatment for these conditions is similar,77 our reference case analyses considered individuals with either primary diagnosis of mild to moderate major depression or one primary anxiety disorder.
Our cohort of individuals, ready and able to engage in iCBT, is treated with iCBT one time only during an active episode of the disease. Ability to engage in iCBT is defined as the ability to:
Speak and understand at least one language in which the program is available (most applications are in English and require understanding of the English language at a grade 10 level)
Work with computers and use the internet
Internet-delivered CBT is not suitable for people with severe learning disabilities, people with high risk of suicide, or people primarily diagnosed with a substance-related or addictive disorder.14 A generic program of iCBT tailored for mild to moderate major depression or anxiety disorders is unsuitable for people who are actively psychotic or manic; this program would need to be modified for people whose principal diagnosis is OCD, PTSD, an eating disorder, substance use, pain, or a somatoform disorder and would require a rigorous initial assessment (P. Farvolden, PhD, email communication, November 9, 2018). Our study also did not consider people with chronic conditions (i.e., cardiovascular disease, chronic pain, or diabetes) that caused or contributed to their depression or anxiety. We also did not consider populations with dementia or other cognitive impairment. Due to legal issues related to consent and confidentiality, there might be challenges in offering guided iCBT to individuals younger than 18 (P. Farvolden, PhD, personal communication, January 19, 2018).
Perspective
We conducted the reference case analysis from the perspective of the Ontario Ministry of Health and Long-Term Care. We considered a societal perspective in a scenario analysis.
Interventions and Comparators
In the reference case analysis, we compared unguided and guided iCBT interventions with face-to-face CBT or usual care. Based on expert consultation and published literature,78–80 most participants self-refer (i.e., refer themselves) to treatment. Self-referral was the main method of accessing iCBT in the reference case analysis.
Treatment of depression and anxiety disorders is provided in a stepped-care approach, depending on the disease severity, using low-intensity treatments such as unguided (standalone) or guided iCBT as first treatment options for people with mild to moderate depression or anxiety.17 Our secondary research questions included an examination of the stepped-care approach. The rationale for comparing face-to-face CBT to iCBT options in the reference case analysis was based on the NICE recommendations, patient preferences, and possible equivalency between various CBT treatments as summarized here.
NICE recommends using either iCBT (an individual-based treatment) or group face-to-face CBT as equally valid options for step 2 of the stepped-care approach in patients who present with mild to moderate depression or anxiety.17 Of note, step 1 of the NICE model does not involve any psychological therapy, but active surveillance and education only.
Some research suggests a possible equivalency in the effectiveness of in-person group CBT and guided iCBT for milder forms of depression.28,81,82
In the following sections, we provide examples of iCBT programs and more detailed descriptions of each strategy.
Internet-Delivered CBT
Internet-delivered CBT is CBT delivered via the internet by an individual or program remote from the client. Table 6 summarizes various iCBT applications in Ontario and across the world.
Table 6:
Internet-Delivered CBT Models Used in Clinical Practice
| Type of iCBT | Description |
|---|---|
| Unguided iCBT | Patients are provided information about website and participate in an online self-help program. |
| Guided iCBT | Patients undergo self-directed exercises with minimal/some guidance or coaching on how to use the website services. Generally, guides are peers and are not regulated health professionals (coaches) |
| Patients complete iCBT modules and regularly discuss their progress with a regulated health professional therapist (e.g., who has a master's degree in social work or psychology). |
Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.
Unguided iCBT
Unguided iCBT is operationalized without any therapist support. Patients are told about the website and they independently guide themselves through an online CBT program. Some commercially available unguided programs offer minimal, technical support via help-line.
Guided iCBT
A guided iCBT intervention is operationalized as a structured psychological treatment delivered to a patient through the internet. The patient works through a course more or less independently, but with some kind of support from a therapist or trained coach.14,83–85 Based on the level of assistance and involvement provided to the patient, guided iCBT is divided into two categories, lower intensity coaching, and higher intensity therapist guidance.
Coach-Guided iCBT
Coaches are unregulated mental health workers trained in CBT. Their involvement in supporting and teaching iCBT is minimal. For example, a coach-guided iCBT program, Improving Access for Psychological Therapies (IAPT), is funded by National Health Service (UK).78,80,86,87 It involves a five-step approach to psychological care for people with depression and anxiety. First, mental health services match treatment intensity to patient needs.78,80,86,87 Step 2 allows access to low-intensity psychological support to patients presenting with mild to moderate symptoms of depression and anxiety.78,80,86,87 These interventions are often delivered by psychological well-being practitioners, who are predominantly psychologists with further training in delivering low-intensity CBT-based interventions.78,80,86,87 Coaches in the IAPT program have a university degree and are rigorously trained in CBT (S. Rao, MD, written and oral communications, January 25 2018). They are unregulated (nonregistered) clinicians or unregulated mental health workers whose main role is providing support for iCBT (with low contact with patients). They monitor and coach patients through the iCBT with the goal of providing structured CBT via internet modules so that the therapy is not influenced during inperson interactions. Coaches do not provide feedback on homework and do not text patients during the therapy. Their involvement is measured weekly, about 10 to 15 minutes per session. They may also perform many other administrative tasks during a regular work day. Training of coaches varies from 1 to 2 days to 1 year, but they work under the supervision of licensed health care professionals (registered experienced psychotherapists) and are supervised on a weekly basis.
Therapist-Guided iCBT
Therapist-guided iCBT is provided by regulated health care nonphysicians who are well-trained and certified in delivery of CBT. They spend more time with patients as compared to coaches. They monitor patient outcomes over time and communicate with patients, including regular check-ins and feedback on homework.
Face-to-Face CBT
Face-to-face CBT for major depression and anxiety is a structured, intensive, time-limited, symptom focused psychotherapy built on the premise that distorted beliefs about the self, the world, and the future maintain depressive affect. Once patients learn to recognize these automatic thoughts, they are taught more adaptive ways of responding.71,73
We previously conducted a cost-effectiveness analysis focused on the structured face-to-face approach to CBT, delivered through either individual or group therapy by a publicly funded, regulated (registered and professionally trained) health care professional.77 Face-to-face CBT was provided in 14 weekly sessions lasting an hour if provided by an individual therapist, or 2 hours if provided as group therapy.77 The training of nonphysician therapists is regulated by professional organizations in Ontario.
Usual Care
Treatments for major depression and anxiety disorders are similar.77 Usual care depends on the severity of disease. Active surveillance and education are recommended for mild major depression (no medication); pharmacotherapy may be recommended for moderate major depression and anxiety.71–74
Summary of Strategies Considered in Our Economic Analysis
Table 7 summarizes all strategies evaluated in the economic model. The intervention and usual-care strategies did not depend on the severity of the disease, given the mix of disease severity in populations in whom iCBT is provided. Pharmacotherapy was initiated with the lowest daily dose of medication and in accordance with the 2016 Canadian Network for Mood and Anxiety Treatments (CANMAT) guidelines71–74 and clinical practice (as established through expert consultation, and analysis77). We describe the variability of medication pathways examined in the reference case and scenario analyses and the costs associated with additional care in Tables 11 and 12 and in Costs of Usual Care and Follow-Up, below.
Table 7:
Interventions and Comparators Evaluated in the Primary Economic Model in Individuals With Mild to Moderate Major Depression or Anxiety Disorders
| Interventions/Comparators |
|---|
1. Unguided individual iCBT
|
2. Guided individual iCBT (with therapist)
|
| 3. Face-to-face individual CBT by nonphysician |
| 4. Face-to-face group CBT by nonphysician |
5. Usual care
|
Abbreviation: CBT, cognitive behavioural therapy.
Example of medication used in our analysis is sertraline starting at an initial dose of 50 mg/d.
14 in-person sessions of CBT for major depression and 10 in-person sessions of CBT for an anxiety disorder, with 2 additional sessions, one for the assessment and the last for planning and conclusion/relapse prevention.77
Table 11:
Health State Utilities Considered for the Economic Model
| Model Parameter: Utilities | Mean (SD/SE/95% CI)a | Source |
|---|---|---|
| Treated major depression | Schaffer et al, 2002102 | |
| Mild MDE | 0.79 (SD: 0.28) | |
| Moderate MDE | 0.67 (SD: 0.36) | |
| Unguided iCBT, major depression/anxiety | ||
| Unguided iCBT vs. usual care | –0.01 (–0.06 to 0.09) | Duarte, 2017,37 Littlewood, 201536 |
| Guided iCBT, major depression, at 12 mo | 0.84 (0.81 to 0.86) | Duarte, 2017,37 Littlewood, 201536 |
| Guided iCBT, anxiety | Hedman et al, 201158,104 | |
| Baseline | 0.77 (SD: 0.18) | |
| 4 mo | 0.82 (SD: 0.14) | |
| 6–12 mo | 0.85 (SD: 0.14) | |
| Utility at 12 mo | 0.83 (0.81 to 0.84) | |
| Individual face-to-face CBT, major depression/anxiety | King et al, 2000103 | |
| Baseline | 0.73 | |
| 4 mo | 0.85 (NR) | |
| 12 mo | 0.85 (NR) | |
| Utility at 12 mo | 0.83b | |
| Group CBT, major depression/anxietyc | Hedman et al, 201158,104 | |
| Baseline, usual care | 0.74 (SD: 0.19) | |
| 4 mo | 0.80 (SD: 0.17) | |
| 6–12 mo | 0.81 (SD: 0.17) | |
| Utility at 12 mo | 0.80 (0.77–0.82) | |
| Normal health, well health state | 0.94 (SE: 0.03) | Lenert et al (2000)105 |
Abbreviations: CBT, cognitive behavioural therapy; CI, confidence interval; iCBT, internet-delivered CBT; MDE, major depressive episode; NR, not reported; SD, standard deviation; SE, standard error.
Beta distributions assigned in probabilistic sensitivity analysis (please see above formulas for calculations of the beta distribution's parameters: α and β.
Standard error assumed to be the same as the one estimated for the group CBT.
Assumed to be the same for anxiety disorders and major depression, based on results of a study in patients with social anxiety disorder.
Table 12:
Estimates of Web Portal Maintenance Costs for iCBT
| Maintenance Costs ($) | Mean, $a | Source | |
|---|---|---|---|
| Low End | High End | ||
| Annual maintenance grant | 50,000 | 100,000 | Consultationb |
| Total cost per week (52 wk) | 961 | 1,923 | NA |
| Cost per week per patient accounting for iCBTcapacity,100 patientsc | 9.61 | 19.23 | NA |
| Total: 8-wk guided iCBT course | 76.92 | 153.85 | NA |
Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.
All costs were estimated in 2018 Canadian dollars.
Communication with experts, names and affiliations withheld to protect proprietary information.
Assumed capacity is the midpoint between 90 to 120 patients weekly (Applicant, email communication, March 24, 2018).
As previously mentioned, unguided iCBT is delivered via the internet and does not involve any therapist support. Guided iCBT could be provided either by therapists or coaches. Our reference case analysis assumed that guided iCBT was provided by therapists (i.e., nonphysicians, regulated health care professionals) from Ontario, trained and certified to deliver CBT for depression and anxiety. In sensitivity analysis, we assumed that guided iCBT was provided by coaches, non-regulated mental health care workers, trained for delivery of CBT (i.e., the provision of CBT similarly to the UK IAPT program78,80,86,87). Further explanations related to differences between regulated registered therapists and non-regulated mental health workers are described in the sections related to modeling main assumptions and cost input parameters.
Discounting and Time Horizon
Internet-delivered CBT is a first step in the management of depression and anxiety and is not repeated in individuals who subsequently sustain a relapse or recurrence. Therefore, in the reference case analysis, we used a short time horizon of 1 year for the duration of iCBT, or CBT and follow-up. No discount rate is applied to accumulated costs and QALYs. This follow-up time is based on the data provided by most studies examining the effectiveness of iCBT. It is further confirmed through our expert consultations.
We explored the cost-effectiveness of guided iCBT for two different stepped-care models with different time horizons:
In the first analysis, we examined the cost-effectiveness of guided iCBT with a therapist followed by face-to-face CBT and compared it with usual care within a long-term stepped-care model. The time horizon for this analysis was lifetime; consequently, we applied a discounting rate of 1.5% for both costs and QALYs, as suggested by the CADTH Guidelines for the Economic Evaluation of Health Technologies.68
In the second analysis, we examined the cost-effectiveness of unguided iCBT followed by guided iCBT with a therapist solely in individuals who are likely to drop out. This model had a short 1-year time horizon with no discounting applied to the cost-effectiveness outcomes.
Model Structure: Reference Case Analysis
Reference Case Analysis: Primary Research Questions
We developed a decision-tree model to determine the incremental cost per QALY gained of the iCBT strategies versus usual care or face-to-face CBT for one episode of mild to moderate major depression or an anxiety disorder. Due to limited evidence with respect to iCBT, this short-term model does not account for repetitive use of iCBT in a subsequent recurrence of the disease,2,5,6 or the risk of hospitalizations. This decision was supported by our economic evidence review, in which most studies provided a single course of iCBT (6–10 weeks on average). The disease progression and repetitive use of CBT should be addressed within the stepped-care treatment approach with the use of face-to-face CBT or other types of psychotherapies.2,5,6 In our prior analysis of structured psychotherapies for major depression and generalized anxiety disorder in response to which the Ontario Health Technology Advisory Committee made a positive funding recommendation with respect to structured psychotherapies, we examined the long-term economic impact of repetitive use of group and individual face-to-face CBT for populations with major depression and anxiety.77
Since iCBT and face-to-face CBT interventions are provided using similar methodology across major depression and anxiety disorders, and their effectiveness is measured using the same outcomes, we used the same model structure to address our primary research questions. It is worth noting that although anxiety disorders can exist in isolation, they coexist with major depression in at least half of the cases.17,88,89 Therefore, the definition of target population was based on the primary diagnosis (e.g., major depression). Also, a subgroup analysis differentiating health-related utility weights between mild and moderate stages of the disease may not be plausible, as our economic evidence review found that unguided and guided iCBT were provided in the mixed sets of populations in terms of baseline disease severity.
Figure 3 represents an example of the model schematic used to evaluate the cost-effectiveness of iCBT over control treatments in adults with mild to moderate major depression. As mentioned above, the model structure is similar for the two primary research questions. The differences are the smaller number of face-to-face sessions used for the treatment of anxiety disorders and values of some input parameters. At the beginning of the simulation, adults start with mild to moderate major depression, with a possibility of medication for those with a more severe disease (i.e., about 50% of patients with moderate major depression, accounted only through modeling of the treatment costs). The model accounts for the probability of participant drop out after each intervention and assumes that 25% to 30% of patients recover spontaneously after discontinuation of the therapy.77 In the next step, if patients do not respond to CBT (or iCBT) and progress with the disease, a medication is added to the clinical pathway (our model assumes about 50% of participants receive medication; costs of medications are included in our model. More details are provided in later sections related to the costing of usual care. The outcomes associated with the effectiveness of the interventions within this short time horizon are response (i.e., 50% reduction in depression scales during the first weeks of therapy initiation) and remission/recovery (i.e., no new episode of depression or no relapse within the 3-month continuation phase of treatment).2,5,6
Figure 3: Simplified Model Schematic: Reference Case Analysis.
Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.
Note: Figure 3 describes a decision-tree model where the square denotes a decision node, the circle denotes a chance node and the triangle denotes a terminal node (a health state that accumulates utilities and costs associated with a specific pathway and strategy, assuming a 12-month time horizon). The same model structure was applied to the reference case analysis in adults with an anxiety disorder. A person with mild to moderate major depression who enters this model chooses between five different strategies: unguided iCBT, guided iCBT, individual face-to-face CBT, group face-to-face CBT (these four tracks compose the upper branch), and usual care. Within each strategy, pathways account for differences in drop out, response and recovery rates, utilities, and costs. People who respond and recover enter the well health state. People who do not respond are offered medication. Those who recover progress to the better health state associated with increments in health-state utility, and those who do not recover do not accumulate this utility increment (i.e., return to the baseline health state).
Main Assumptions: Reference Case Analysis
The major assumptions for this model are as follows:
Each individual in the iCBT intervention received only one course of treatment, regardless of disease progression (one-time non-repetitive treatment)
No stepping-up the care—those who begin with medication remain on their medication throughout the course and those receiving iCBT do not switch to an in-person psychological therapy
In adults who are taking medication, an increment in the costs was accounted for, but not an increment in utility. The effect of medication on quality of life or clinical outcomes was not separated from the effects of other treatments (see our economic evidence review)
In adults started with CBT or iCBT only, medication was added after no response to CBT treatment. In a scenario analysis, we explored the addition of medication from the beginning of the iCBT/CBT intervention and the change in medication pathway after no response to CBT/iCBT treatment (more details provided in Table 12 and Costs of Usual Care and Follow-Up, below)
- Booster iCBT sessions (i.e., an additional course of treatment) were not accounted for the following reasons:
- ○ They require the full fee for iCBT
- ○ They suggest deterioration or reappearance of symptoms (recurrence of the disease)
- ○ They require modeling over a longer time horizon
- ○ There is limited evidence of the efficacy of repeated iCBT on the risk of relapse or recurrence
- ○ There is limited evidence of the long-term effects of iCBT on the risk of recurrence of major depression and anxiety or other health outcomes such as rate of suicide or hospitalization
Probability of suicide and excess mortality were assumed to be small in patients with mild forms of disease and was not accounted in the short-term model. Psychotherapies were not suitable for people with severe major depression or at risk of suicide77
- Assumptions related to program delivery and costs of treatment:
- ○ The number of sessions and costs of providing iCBT for anxiety disorders or major depression were assumed to be similar (a total of eight therapist-guided sessions over 8 weeks)
- ○ The cost of iCBT was incurred with the program initiation; there was no reduction of cost from people who dropout
- ○ Training costs for iCBT:
- ■ High-intensity guided iCBT included the training costs in therapist salaries (expert consultation);
- ■ If iCBT was guided by coaches, then the training cost would not be accounted in their salaries, but would be based on findings from the literature data90 because training costs are treated as proprietary information (P. Farvolden, PhD, email communication, March 2018)
- ○ Based on our prior HTA of structured psychotherapies for major depression and generalized anxiety disorder (GAD)77, the face-to-face CBT strategy incurred costs for psychological assessment with referral
- ○ Self-referral was assumed for all types of iCBT interventions, except for the coach-based program (described in Total Costs: Unguided or Guided iCBT, below)
- ○ We assumed that only patients who had been prescribed medications have regular follow-up visits with their physicians77
- ○ Internet-delivered CBT was assumed to be provided via a central portal (e.g., Ontario Telemedicine Network [OTN]) to account for the maintenance costs
Secondary Exploratory Analyses: Model Structures
Long-Term Cost-Effectiveness of Guided iCBT Provided Within a Stepped-Care Model
We combined our short-term reference case iCBT model with a previously developed probabilistic Markov microsimulation model77 to examine the lifetime cost-effectiveness of guided iCBT with a therapist and face-to-face CBT versus usual care in people with new and recurrent major depression and anxiety. We used a stepped-care approach starting with therapist-guided iCBT followed by with group or individual face-to-face CBT if the disease recurs or progresses. In this analysis, we compared three strategies:
Guided iCBT followed by individual CBT provided by a regulated health care professional (nonphysician)
Guided iCBT followed by group CBT provided by a regulated health care professional (nonphysician)
Usual care (no CBT)
The methods including model input parameters are described in detail in Appendix 5.
Cost-Effectiveness of Unguided iCBT Followed by Guided iCBT
Based on the literature15,91 and expert consultation (J. Hensel, MD, oral communication, February 7, 2018), unguided iCBT could be used as a first step in stepped care. It could be used to select and help those with minor issues and identify others who need more help with CBT assignments, are not motivated enough to self-manage their CBT, or are at risk of early dropout. Alternatively, as highlighted with the UK Improving Access for Psychological Therapies program (IAPT) and the literature, using a stepped-care approach with counselling/education as step 1 and iCBT as step 2 may increase access to mental health services and treatment.17,78–80
We developed a stepped-care decision-tree model comparing the costs and effectiveness of unguided and guided iCBT modeled as a stepped-care strategy versus treatment as usual over 1 year. This scenario did not include face-to-face CBT because this type of CBT is part of stepping-up the care from iCBT to high-intensity structured in-person psychotherapy for those who do not respond to prior treatments or experience a relapse or recurrence (step 3 under NICE guidelines17).
Clinical Outcomes and Utility Parameters: Reference Case Analysis
We used several different input parameters to populate the model. These parameters describe the course of one major depression or anxiety episode, including changes in the severity of initial disease with the treatment (e.g., from mild to moderate major depression). To estimate the cost-effectiveness of iCBT versus usual care or face-to-face CBT, we populated the model with parameters related to the effectiveness of interventions (e.g., probabilities of response and recovery), health state utilities, and direct medical costs.
Natural History
To model the natural history of one episode of major depression, we used literature sources and data from current clinical practice provided through expert consultation (Table 8). Additional input parameters for the life-time cost-effectiveness analysis (secondary research questions) are described in detail in our prior analysis of structured psychotherapies for major depression and GAD and Appendix 6.77
Table 8:
Natural History Inputs, Reference Case Analysis: Major Depression and Anxiety Disorders
| Model Parameter | Mean | Source |
|---|---|---|
| Probability of dropout, major depression/anxiety | van Ballegooijen et al, 201492 | |
| Face-to-face CBT | 0.16 | |
| Unguided iCBT | 0.32/0.75a | |
| Guided iCBT | 0.19 | |
| Probability of dropout, major depression/anxiety | Machado et al, 200694 | |
| Sertraline | 0.10 | |
| All SSRIs | 0.28 (0.19–0.38)a | |
| Probability of self-recovery after dropout (no treatment) | 0.25 | Guidelines, expert opinion77 |
| Probability of adding medication | Koeser et al, 201593 | |
| iCBT/CBT | 0.23 | |
| Usual care | 0.70 |
Abbreviations: CBT, cognitive behavioural disorder; iCBT, internet-delivered CBT; CI, confidence interval; SSRI, selective serotonin reuptake inhibitor.
Values used in sensitivity analysis.
The probabilities of dropout from face-to-face and iCBT are based on a meta-analysis conducted by van Ballegooijen et al,92 which included 24 studies with 981 participants allocated to face-to-face or guided iCBT. This study was not included in our clinical review but is described here to address potentially high drop out in unguided iCBT compared with the rates of drop out from guided iCBT and face-to-face CBT. The probability of adding a medication after failing to respond to CBT is based on meta-analyses conducted within the most recent UK modelling analysis in people with moderate major depression.93 After dropout, major depression could resolve on its own or become more severe. We assumed that a person has a 25% chance of recovery without any treatment after a dropout.77
Intervention Effects
The clinical evidence review of this report provides data related to the effectiveness of guided iCBT as compared to usual care.7,20,21,23,50 Data related to the effectiveness of unguided iCBT was obtained from a systematic review by Karyotaki et al91 (this study that was excluded from the clinical review because of the lack of a precise definition of the target population with major depression). We assumed that the recovery rate for guided iCBT is the same for these two disorders, and as reported in a systematic review by Andrews et al.21 In their review, the authors reported the effectiveness estimate for the follow-up of 9–18 months for people with major depression and anxiety disorders. The effectiveness of face-to-face CBT versus usual care was based on data from our prior report.77 The effectiveness of these therapies was measured using a Hedge's g statistic or standardized mean difference (see column 3 in Tables 9 and 10). Below, we describe the probabilities of response and recovery calculated from the identified effect sizes using the recommended effect-size conversion method.95 Based on data from three systematic reviews,7,21,23 the probability of response to guided iCBT was assumed to be equal to face-to-face CBT. This assumption was based on data from Dedert et al,23 Andrews et al,21 and Adelman et al,7 that reported no statistically significant difference in the response between guided iCBT and face-to-face CBT. Of note, Dedert et al warned that despite no difference between two types of treatments, the evidence on equality of face-to-face CBT and guided iCBT is very limited, particularly for adults with major depression. Although we made our assumption of no difference in responses to guided iCBT and face-to-face CBT in our target populations, we distinguished the effects of guided iCBT and face-to-face CBT on recovery. Therefore, we remained in line with the overall conclusions of our clinical review.
Table 9:
Effectiveness of Interventions, Reference Case Analysis: Mild to Moderate Major Depression
| Model Parameter | Mean (95%CI/SE)a | SMD (95%CI)* | Source |
|---|---|---|---|
| Probability of response (improvement) | |||
| Unguided iCBT | 0.50 (0.28–0.62) | 0.27 (0.17–0.37) | Karyotaki, 201791 |
| Guided iCBT | 0.73 (0.66–0.77) | 0.83 (0.59–1.07) | Arnberg et al, 201420 |
| Individual face-to-face CBT | 0.73 (0.66–0.77) | NA | Dedert et al, 201723; Andrews et al, 201821; Adelman et al, 20147 |
| Group face-to-face CBT | 0.73 (0.66–0.77) | NA | Dedert et al, 201723; Andrews et al, 201821; Adelman et al, 20147 |
| Medication | 0.70 | Koeser et al, 201593 | |
| Probability of recovery | |||
| Unguided iCBT | 0.38 (0.09) | NA | Kaltenthaler et al, 200650 |
| Guided iCBT | 0.48 (0.23–0.60) | 0.22 (0.59–1.07) | Andrews et al, 201821 |
| Individual face-to-face CBT | 0.66 (0.64–0.71) | NAc | Biesheuvel-Leliefeld, 201696 |
| Group face-to-face CBT | 0.63 (0.59–0.66) | NAc | Churchill, 200197 |
| Medication | 0.62 (0.58–0.66) | NAc | Cipriani, 201498 |
Abbreviations: CBT, cognitive behavioural therapy, CI, confidence interval; iCBT, internet-delivered CBT; NA, not applicable; SE, standard error; SMD, standardized mean difference.
Original value, reported in the article and clinical review: probability of response/recovery was calculated by transforming SMD to log odds ratio using the formula: d × π/sqrt(3),95 and further calculated the probability of event based on the data of baseline probability of response.50
Beta distributions assigned in probabilistic sensitivity analysis to all parameters for which 95% CI or SE were specified in Table 9a. Two parameters of the beta distribution (α, β) are derived from the mean and SE (stated for each model parameter). Formulas for these calculations, derived from the mean and SE (provided in the Tree Age software) are α = ([Mean2] × [1 – Mean])/([SE2] – Mean); β = ([{1 – Mean} × {1 – Mean}] × Mean)/([SE2] – 1).
Reported risk of recurrence, in-person CBT vs. usual care in Biesheuvel-Leliefeld, 2016 (relative risk: 0.68, 95% CI 0.53–0.76) and group CBT, in Churchill, 2011 (0.78, 95% CI 0.69–0.93), and sertraline in Cipriani, 2014 (relative risk: 0.80, 95% CI 0.69–0.93) details provided in our prior report.77
Table 10:
Effectiveness of Cognitive Behavioural Therapy Interventions, Reference Case Analysis: Anxiety Disorders
| Model Parameter | Mean (95%CI/SE)a | SMD (95%CI)b | Source |
|---|---|---|---|
| Probability of response (improvement) | |||
| Unguided iCBT | 0.67 (0.58–0.73) | 0.66 (0.32–1.00) | Adelman, 20147 |
| Guided iCBT | 0.74 (0.69–0.78) | 0.84 (0.45–1.23) | Arnberg et al, 201420 |
| Individual face-to-face CBT | 0.73 (0.66–0.77) | NA | Dedert et al, 201723; Andrews et al, 201821; Adelman et al, 20147 |
| Group face-to-face CBT | 0.73 (0.66–0.77) | NA | Dedert et al, 201723; Andrews et al, 201821; Adelman et al, 20147 |
| Medication | 0.70 | Koeser et al, 201593 | |
| Probability of recovery | |||
| Unguided iCBT | 0.38 (0.09) | NA | Kaltenthaler et al, 200650 |
| Guided iCBT | 0.48 (0.23–0.60) | 0.22 (0.59–1.07) | Andrews et al, 201821 |
| Individual face-to-face CBT | 0.66 (0.62–0.68) | NAc | Biesheuvel-Leliefeld, 201696 |
| Group face-to-face CBT | 0.63 (0.59–0.66) | NAc | Churchill et al, 200197 |
| Medication | 0.62 (0.58–0.66) | NAc | Cipriani et al, 201498 |
Abbreviations: CBT, cognitive behavioural disorder; CI, confidence interval; iCBT, internet-delivered CBT; NA, not applicable; SMD, standardized mean difference.
Beta distributions assigned in probabilistic sensitivity analysis to all parameters for which 95% CI or SE were specified in Table 9b. Two parameters of the beta distribution (α, β) are derived from the mean and SE (stated for each model parameter). Formulas for these calculations, derived from the mean and SE (provided in the Tree Age software) are α = ([Mean2] × [1 – Mean])/([SE2] – Mean); β = ([{1 – Mean} × {1 – Mean}] × Mean)/([SE2] – 1).
Original value, reported in the article and clinical review.
Reported risk of recurrence, in-person CBT vs. usual care in Biesheuvel-Leliefeld, 2016 and group CBT, in Churchill, 2011, and sertraline in Cipriani, 2014, details provided in our prior report.77
Health State Utilities
A health state utility is a measure of health-related quality of life and reflects the strength of preference for specified health states. By convention, health state utilities are anchored on death and best possible health (death is assigned a utility weight of 0, and perfect health is assigned a utility weight of 1).99 The value of a QALY for a certain health state is calculated by multiplying time spent in that health state with the utility assigned to that health state (e.g., 1 year of untreated schizophrenia with a utility weight of 0.5 equals a QALY of 0.5). We performed a targeted literature search for utility values on February 28, 2018, for studies published from inception to the search date in MEDLINE. The search was based on the intervention of the clinical search strategy with a methodological filter applied to limit retrieval to health state utility values.100 See Appendix 1 for literature search strategies, including all search terms. We also used the utility weights specific to disease states that were reported in economic evaluations examined in our economic review of the literature.
The utility weights related to health states of depression and anxiety disorders considered for our analysis are presented in Table 11. We present the original values found in the literature and the model input parameter values estimated using the area-under-the-curve method to account for changes in the utility estimate over time.101 For example, in Table 11, the utility value for guided iCBT for anxiety disorders was estimated for the follow-up period of 12 months using the following formula: ([{0.77 + 0.82}/2] · [1/3]) + ([{0.82 + 0.85}/2] · [1/6]) + ([{0.85+0.85}/2] · [1/2]) = 0.265 + 0.139 + 0.465 = 0.83.
Utility weights associated with medication therapy are obtained from a study including Canadian patients with past mild or moderate major depressive episodes.102 The face-to-face CBT-related utility value is based on a UK study of 457 adults with major depression and generalized anxiety disorder who received psychological therapy.103 Based on studies found in our economic literature review and targeted search, compared with treatment as usual, the increments in utilities are nonsignificant with unguided iCBT compared with usual care,36 ranging from 0.01 to 0.05 with guided iCBT.36,58,104 In studies that examined iCBT, utility weights were elicited using the EQ-5D instrument.
People who are “well” are assumed to be stable and not receiving treatment. Lenert et al105 conducted a mapping study using a sample of 295 people with major depression to provide the utility weights for those who are “near normal health.”
Cost Parameters: Reference Case Analysis
The reference case analysis included additional costs relevant to CBT treatment. Scenario analyses considered the costs of follow-up and costs borne by patients.
Overall Components of Cognitive Behavioural Therapy Treatment: Reference Case Analysis
Depending on the type of CBT, the costs of treatment potentially include the following components:
- Initial assessment, primarily applicable to face-to-face CBT
- ○ The visit to a general practitioner or nurse practitioner to obtain a referral; this is applicable to face-to-face CBT and one type of guided iCBT (i.e., coach-guided iCBT)
- ○ The psychiatric assessment by a psychiatrist or psychologist
- Labour
- ○ Face-to-face CBT by regulated nonphysician therapists
- ○ Costs of supervision (1 hour weekly)
- Internet-delivered CBT
- ○ Non-regulated mental health care workers for coach-guided (low intensity-guided) iCBT
- ○ Regulated nonphysician therapists (e.g., those with a Master's of Social Work degree) for therapist-guided (high-intensity guided) iCBT
Physician follow-up, applicable to people taking medications (a description of specific medication pathways is presented in Table 16)
Maintenance and licencing specific to the delivery of iCBT
Table 16:
Estimates of Annual Per-Person Costs of Additional Care Including Medicationsa
| Additional Care in UC/CBT | Per-Person Cost, $a | |||
|---|---|---|---|---|
| Physicianb | Drug 1b | Drug 2b | Totalc | |
| Not using drugs | 125.5 | NA | NA | 125.5 |
| Same medication throughout year | 564.75 | 356.2 | NA | 666.3 |
| Starting with one medication and adding another during follow-up | 878.5 | 178.1 | 193.44 | 1,039.5 |
| Adding medication during follow-up | 564.75 | 178.1 | NA | 615.5 |
Abbreviations: CBT, cognitive behavioural therapy; NA, not available; UC, usual care.
All costs were estimated in 2018 Canadian dollars.
All costs related to physician fees and medication are considered fixed in the analysis, meaning that no distributions were assigned.
Total costs are not a simple sum of the values across the rows; they are smaller because they are adjusted to account for public funding of drugs in 57% of the population, for compliance, and the increasing number of follow-up physician visits after changes in medication dose during the last 6 months of follow-up. Another medication dose was considered during follow-up for people who did not benefit from treatment.
Labour Costs
We used a previously described method to estimate the labour costs associated with clinical activities. We also applied an hourly salary where appropriate.77 Applied cost recognizes that salaried clinicians spend time on nonpatient activities, so less than 100% of a clinician's time accounts for clinical work.106 It is recognized that the applied rate (i.e., the ratio of applied time to total time) is 85% for medical staff. Using the applied rate, we calculated hourly costs for publicly funded regulated therapists providing face-to-face CBT.77 The following is an example of these calculations that are further expanded to estimate the hourly rate for therapists who deliver guided iCBT (regulated professionals and coaches).
Assumptions:
A full-time equivalent (FTE) works 1,950 hours per year (7.5 hours per day, 5 days per week, 52 weeks per year)
Using the applied rate of 85%, the applied time is thus 1,950 × 0.85 = 1,658 hours annually
Costs:
Individual face-to-face CBT by regulated nonphysician therapist at an annual salary of $110,000 with 30% benefits ($143,000 per year total) yields an hourly cost of $86.25 (14 weekly 1-hour sessions for major depression and 10 weekly sessions for anxiety disorders)
Group face-to-face CBT including 10 participants by one therapist yields an hourly cost of $17.25 per participant (14 weekly 2-hour sessions for major depression; 10 weekly 2-hour sessions for anxiety disorders)
Coach-based iCBT therapist at an annual salary of $35,000 with 30% benefits ($45,500 per year total) yields an hourly cost of $27.44
Therapist-based iCBT by regulated MSW therapist at an annual salary of $54,455 with 30% benefits ($70,791 per year total) yields an hourly cost of $42.70 (P. Farvolden, PhD, email communication, March 24, 2018)
Supervision for face-to-face CBT was $86.25 for each 1-hour weekly session by an experienced clinical psychologist (10 weeks for major depression and 8 weeks for anxiety disorders). Supervision costs do not include the first and last week of CBT, which were primarily related to treatment planning
Overall costs of supervision for iCBT (not per patient) consisted of 1-hour weekly supervision by a clinical psychologist ($86.25 per hour cost) over 8 weeks of iCBT for all but the coach-based program, which is provided for a maximum of six sessions
Maintenance Costs: Internet-Delivered CBT
The maintenance and IT support required for iCBT to be provided through a central portal may be supported by an annual government grant ranging from $50,000 to $100,000. It was assumed that people willing to engage in iCBT would have internet access. The capacity for guided iCBT may depend on the number of users and may range between 90 and 120 users per week (P. Farvolden, PhD, written communication, March to May 2018). Although the capacity for unguided iCBT over the online or research portal should be effectively unlimited, we assumed the same estimated cost of maintenance as for the guided iCBT. Table 12 presents the per-person estimates of maintenance costs for an iCBT course. The lower-range estimated cost of $77 was used for the reference case.
Total Costs: Unguided or Guided iCBT
Table 13 describes the estimates for per-person costs for different types of iCBT. Based on expert consultation and literature findings,107 the amount of time a therapist spends providing support for iCBT to a client is at least three times shorter than the time spent for face-to-face CBT. As previously mentioned, guided iCBT can involve a therapist (i.e., high intensity of communication between a client and the therapist) or a coach (i.e., low intensity of communication). A therapist (an example of high-intensity guidance) spends about 20 minutes on each client per guided iCBT session. Based on the UK IAPT data,78,80,86,87 a coach spends half as much time as a highly qualified therapist (i.e., 10 minutes per client per session, written and oral communication, S Rao, MD, January 25, 2018, see description in intervention section). Applying previous calculations of their hourly cost, we estimate the labour costs associated with one 8-session coach-guided or therapist-guided iCBT course is $37 and $114, respectively (Table 13). The cost of supervision is about $690 per course (assuming 8 weeks of sessions).
Table 13:
Cost Estimates for Unguided and Guided iCBT Courses
| Type of iCBT Course | Mean Costs, $a | Total Cost of iCBT Course, $a,b (Low–High)b | |||||
|---|---|---|---|---|---|---|---|
| Licence (Low–High) | Referral | Therapist Training | Therapist Salary | Supervision | Maintenance | ||
| Reference case analysis | |||||||
| Unguided iCBT course | 60–90 | 0c | NA | NA | NA | 77 | 137–167 |
| Therapist-guided iCBT (high intensity) course via portal | 3–5d | 0c | 579 | 114 | 690 | 77 | 1,466–1,470 |
| Therapist-guided iCBT (high intensity) employer-led therapist course via portal | 3–5d | 0c | NA | 114 | 690 | 77 | 887–891 |
| Coach-guided iCBT (low intensity) course | 3–5d | 0c | 579 | 37 | 690 | 77 | 1,389–1,393 |
| Scenario analyses | |||||||
| A. Coach (low intensity) guided iCBT course with referrale | 60–90 | 62.75 | NA | 27 | 517 | 77 | 744–774 |
| B. GP- and coach-guided iCBTf | 3–5d | 62.75 | 579e | 167f | 690 | 77 | 1,582–1,587 |
| C. Employer-led, publicly provided iCBT course | 500–800 | 0c | NA | NA | NA | 77 | 577–877 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy, all assumed to have 8 sessions except for the Bounce Back program (6 sessions); NA, not applicable.
All costs were estimated in 2018 Canadian dollars.
Gamma distributions assigned in probabilistic sensitivity analysis, assuming standard error to the mean cost of 25%. Two parameters of the gamma distribution (α, λ) are derived from the mean and standard error. Formulas for these calculations are: α = (Mean‸2)/(SE‸2); λ= Mean/([Mean × SE]‸2).
We assumed patient referred self and costs of assessment were included in price of the program (applicant, in-person communication January 19, 2018).
Costs provided by the applicant (email communication March 24, 2018, and updated in May 2018): these are the monthly costs ($3-$5 per person), the total therapist-guided iCBT assumed 8 weekly sessions of therapy and were accounted in the estimates of the total per-person costs.
Costs estimated for the one example coach-based program in Ontario.
Costs estimated for the primary care iCBT model, enabling a circle of care and assuming that the coach is a family physician (one session/one visit of $62.75).
We also accounted for the cost of training of coaches based on data from an Australian research study by Crome et al.90 In this study, a 2-day highly intensive training workshop was about $579 CAD ($640 2014 AUD converted to 2018 CAD using purchasing power parity and consumer price index conversion factors.
Scenario analyses include other models of coach- or therapist-based iCBT:
Scenario A considers coach-based iCBT as provided by one 6-session program in Ontario conducted in adults with mood disorders. This program is not an exemplar of structured coach-guided iCBT; therefore, it is explored in a scenario analysis for adults with major depression only. The probability of response is about 40% at the end of the program and the recovery rate is unknown. Consequently, we assumed that this program will have the same probability of recovery as that calculated for therapist-guided iCBT (mean 0.48, 95% CI 0.23–0.60, see Tables 9 and 10).
Scenario B represents an example of a possible primary care iCBT model whereby a family physician is trained to be the coach for iCBT and to overview the whole patient care. This iCBT model is important to consider as patient care in many rural and Northern Ontario locations are dependent on primary care physicians.77 This scenario could be the easiest and most reliable way for implementation in some regions of Ontario. An example of this type of model is rolling out in Newfoundland (in-person/e mail communication, Dr. P. Cornish, 01 Feb, 2018).108
Scenario C describes one type of privately (employer) funded therapist-guided iCBT currently existing in Ontario. The employer provides a therapist-guided iCBT program through their own portal (not publicly funded) (P. Farvolden, PhD, multiple communications, January to March 2018). Depending on the employer's capacity to provide care for people with major depression and anxiety, this could be also an implementation option to consider for Ontario.
The reference case analysis used the mean costs estimated from the low range values (e.g., $137 unguided iCBT). The upper range value was used in sensitivity analyses to examine the robustness of cost effectiveness results to changes in the costs of iCBT. All scenario analyses were done from a conservative perspective, using the upper-level mean cost estimates (i.e., the mean cost for the scenario A: $774). All cost estimates were considered probabilistically in our analyses using the gamma distribution to account for uncertainty around the mean estimate.
Total Costs: Face-to-Face Cognitive Behavioural Therapy
Table 14 describes the components, calculations, and estimates of total per-person costs for individual and group CBT in people with major depression or an anxiety disorder. The calculation methods for therapist cost (i.e., hourly cost per CBT session) and costs of supervision are described above. The costs of in-person CBT represent the conservative estimate. Actual costs may be lower. For instance, it is possible that supervision costs will not be needed for experienced-regulated CBT therapists. In sensitivity analyses, we used a lower range for costs associated with individual and group CBT (hourly cost estimates for individual face-to-face CBT of $77.60 and for group CBT of $15.50), a total cost amounting to $2,242 and $1,372, for major depression and $1,219 and $598, for anxiety disorders, respectively, for individual and group CBT.
Table 14:
Estimated Per-Person Costs of Face-to-Face Cognitive Behavioural Therapy Course by Regulated Nonphysiciana
| CBT Format | No. of Sessions, Course/Supervision | Per-Person Cost, $a | |||
|---|---|---|---|---|---|
| Initial Assessment | Therapist Salary | Weekly Supervision | Totalb | ||
| Individual CBT, major depression | 14/12 | 223.35 | 1,207.5 | 1,034.9 | 2,466 |
| Group CBT, major depression | 14/12 | 223.35 | 241.5 | 1,034.9 | 1,500 |
| Individual CBT, anxiety | 10/8 | 223.35 | 862.5 | 690 | 1,776 |
| Group CBT, anxiety | 10/8 | 223.35 | 172.5 | 690 | 1,086 |
Abbreviations: CBT, cognitive behavioural therapy.
All costs were estimated in 2018 Canadian dollars.
Gamma distributions assigned in probabilistic sensitivity analysis, assuming SE to the mean cost of 25%. Two parameters of the gamma distribution (α, λ) are derived from the mean and standard error. Formulas for these calculations are: α = (Mean2)/(SE2); λ= Mean/([Mean × SE]2).
Source: Health Quality Ontario Health Technology Assessment.77
Costs of Usual Care and Follow-Up
As in our prior analysis,77 we accounted for the cost of medication and follow-up used in usual care and in certain patients (e.g., 48–50% of people with moderately severe disease take medications). We based the costs of follow-up by physicians on the fee-for-service schedule and number of visits. As per the 2016 CANMAT guidelines72 and expert consultation,77 the number of follow-up visits depended on the clinical pathway:
People on medication were assumed to have biweekly follow-up visits with a general practitioner in the first month and then monthly visits until the end of the time horizon. We assumed that the consultation with a psychiatrist to update the initial medication plan occurred once, at the completion of face-to-face CBT (at 4 months)
People who received no medication were assumed to have follow-up visits with a general practitioner every 6 months
We based drug costs on the Ontario Drug Benefit program after accounting for dispensing fees and markup costs, as suggested in the literature.77
Tables 15 and 16 describe input values for cost parameters used in calculations of total yearly costs per person, related to medication use and follow-up by a GP in the usual care or other intervention strategies in the reference case analysis. It was assumed that drug coverage is publicly funded in 57% of cases based on the 2017 Canadian report by Sutherland and Thy.109 The costing of medications was also adjusted for patient adherence. It was assumed that 50% of people with mild to moderate major depression or anxiety take medications at baseline20 or when prescribed for the first time during the follow-up. We assumed that the compliance is 100% in people who were on medication at baseline and had to switch to a higher dose in the second half of the year. This usual care medication pathway was accounted for in individuals on CBT treatment in a scenario analysis. In the reference case analysis, we assumed that people would start with medication after no response to iCBT or CBT treatment (all pathways shown in Table 16).
Table 15:
Costs Used in the Economic Model: Usual Care and Follow-Up
| Model Parameter | Mean, $a | Distribution | Source |
|---|---|---|---|
| One-time costs related to CBT | |||
| Initial assessment, total | 223.35 | Fixed | HQO HTA77 |
| GP referral | 62.75 | — | OHIP code K005, 30 min110 |
| Assessment/consultation by psychiatrist or psychologist | 80.30 | — | OHIP code K197, 30 min110 |
| Weekly costs of pharmacotherapyb | |||
| Sertraline, 50 mg/d | 6.85 | Fixed | ODB111; Kolber (2014);112 HQO |
| Sertraline, 100 mg/d | 7.44 | — | HTA77 |
Abbreviations: CBT, cognitive behavioural therapy; GP, general practitioner; HQO, Health Quality Ontario; HTA, health technology assessment; ODB, Ontario Drug Benefit Program; OHIP, Ontario Health Insurance Plan Schedule of Benefits and Fees.
All costs were estimated in 2018 Canadian dollars.
Costs of drugs include a dispensing fee of $10.22 and a mark-up of $1.71 for 30-day use.112
Table 17:
Per-Person Societal Costs Associated With Receiving Face-to-Face Cognitive Behavioural Therapy
| Model Parameter | Meana | Distributionb | Source |
|---|---|---|---|
| Productivity loss, per session | |||
| Hourly rate based on median income in Ontario of $34,000/yr with 30% benefits ($1,950 hr/yr) | $22.67 | NA | Statistics Canada, 2016113 |
| Costs associated with productivity loss | $1,275 | — | — |
| Daily travel and wage costs | |||
| Parking | $26 | NA | |
| Public transport | $5 | NA | |
| Traveling and treatment time | 4 hr | NA | Assumption |
| Minimum wage in Ontario | $15/hr | NA | Ministry of Labor, 2017114 |
| Total travel and wage costs (15 days) | |||
| Parking costs (conservative estimate)b | $390 | NA | NA |
| Public transport | $75 | NA | |
| Wage costs | $904 | NA | |
| Total societal costs (CBT including travel costs, conservative estimate)b | $3,454 | Gammac | NA |
| Total societal costs (CBT including public transportation travel costs) | $2,254 | Gammac | NA |
| Total societal costs (CBT only) | $2,179 | Gammac | NA |
Abbreviations: CBT, cognitive behavioural therapy; NA not applicable.
Original nondiscounted costs in 2018 Canadian dollars.
Used in probabilistic sensitivity scenario analyses; standard error (SE) based on an assumption that mean costs vary by ±25%.
Table 16 describes the annual costs of additional care, including medications, for the four different pathways considered in the model. The reference case analysis considered costs related to two clinical pathways for additional or usual care: costs associated with people who were not on medication and costs of adding medication during follow-up. A separate model and scenario considered the costs of additional care, including all four possible clinical pathways.
Societal Perspective
In a scenario analysis, we used a societal perspective and included the following costs:
Out-of-pocket costs
Cost of people's time spent travelling and receiving CBT
Cost of lost productivity
For out-of-pocket costs, we considered the cost of parking at the hospital. We estimated the number of days patients receive CBT in person and applied a maximum daily parking rate of $26 ($390 for 15 days of therapy). The public transport costs were assumed to range from $4 and $6 (median $5 used in the estimation of the travel costs: $75).
We assumed that people spend about 2 hours traveling to and from the hospital and an additional 2 hours to receive CBT treatment. To value the time that people spend in face-to-face CBT, we used the Ontario minimum wage of $11.60 per hour plus a 30% employee benefit (about $15.10 per hour). We assumed that people would be able to work part time while receiving face-to-face CBT; thus, the value of time spent on face-to-face CBT totaled about $60 per day of treatment ($904 for 15 days of therapy).
We assumed that people would not be able to work full time while receiving face-to-face CBT and used the median income in Ontario to estimate the cost of lost productivity. Assuming an income of $33,840 (plus 30% benefits), we calculated a $22.67 hourly rate. The cost of productivity loss associated during CBT treatment was about $1,275 (assuming 15 days of therapy and part time work including 56.25 hours spent away from work: 56.25 × $22.67 = $1,275).
Analysis
The primary outcome of the economic evaluation is the ICER, reported as the incremental cost per QALY gained. Following CADTH guidelines,68 we used probabilistic sensitivity analysis to address both primary and secondary economic research questions. The reference case analysis simulated 100,000 trials. Uncertainty in the model estimates was also evaluated using a series of probabilistically computed scenario analyses for some important parameters (e.g., utilities associated with internet and face-to-face CBT, efficacy of iCBT, dropout rates, iCBT costs). The analyses are conducted using TreeAge Pro 2018.R1 (TreeAge Software, Williamstown, MA).
Generalizability
The findings of this economic analysis may not be generalizable to all patients with major depression or anxiety disorder. Internet-delivered CBT may not be suitable for people who are suffering from severe mood or anxiety disorders that are unlikely to respond to outpatient treatment. These may require specialized multidisciplinary intensive outpatient or inpatient care.
Expert Consultation
Expert consultations were conducted throughout the review process to feed into the development and refinement of the research questions, review methods, and review results, as well as to contextualize the evidence on iCBT to Ontario. Expertise was sought from clinical and methodological experts within the CBT field.
Results
Below we present the results of our primary economic evaluation: the base case and sensitivity analyses.
Reference Case Analysis
We present results of two cost–utility analyses, for adults with mild to moderate major depression (Table 18), and adults with anxiety (Table 19).
Table 18:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and Usual Care for Major Depression
| Strategy | Mean Costs, $a | Mean QALYs | Incremental Costs, $a,b | Incremental QALYsc | ICER, $/QALY Gaineda |
|---|---|---|---|---|---|
| Unguided iCBT | 274.65 | 0.778 | — | — | — |
| Usual care | 409.40 | 0.787 | 134.75 | 0.009 | 14,972 |
| Guided iCBT (therapist) | 1,666.26 | 0.826 | 1,256.86 | 0.039 | 31,575 |
| Group CBT | 1,691.23 | 0.817 | 24.97 | –0.009 | Dominatedd |
| Individual CBT | 2,654.00 | 0.826 | 987.79 | 0.00001 | 59,377,114e |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done given the 12-month time horizon.
Incremental cost = average cost (strategy of guided CBT) – average cost (strategy of usual care).
Incremental effect = average effect (strategy of guided CBT) – average effect (strategy of usual care).
Negative ICER means this strategy was considered dominated and was excluded from the analysis.
This estimate of the high ICER implies that individual CBT was just slightly better when it comes to QALY increments and is essentially dominated by guided iCBT because it provides benefit almost equal to that of guided iCBT, but it is more costly.
Note: Results may appear incorrect because of rounding.
Table 19:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and Usual Care for Anxiety Disorders
| Strategy | Mean Costs, $a | Mean QALYs | Incremental Costs, $a,b | Incremental QALYsc | ICER, $/QALY Gaineda |
|---|---|---|---|---|---|
| Unguided iCBT | 269.70 | 0.802 | — | — | — |
| Usual care | 409.40 | 0.787 | 139.71 | –0.015 | Dominatedd |
| Group CBT | 1,275.37 | 0.817 | — | — | Extended dominancee |
| Guided iCBT | 1,665.17 | 0.834 | 1,395.47 | 0.032 | 43,214 |
| Individual CBT | 1,952.78 | 0.8267 | 287.61 | –0.007 | Dominatedd |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (strategy of guided CBT) – average cost (strategy of unguided iCBT).
Incremental effect = average effect (strategy of guided CBT) – average effect (strategy of unguided iCBT).
The negative ICER, this and usual care strategies were considered dominated.
ICER of group CBT vs. usual care: $65,070/QALY.
Note: Results may appear incorrect because of rounding.
Cost–Utility Analysis for Mild to Moderate Major Depression
In adults with mild to moderate major depression, in a sequential cost–utility analysis ranking five strategies by increasing costs, unguided iCBT and group CBT were dominated and excluded from final calculations. Of note, QALYs associated with all guided iCBT and in-person CBT were similar. Compared with usual care, guided iCBT (i.e., guided via a web portal with a therapist who is a non-physician, regulated health care professional) was associated with an increased quality-adjusted survival of 0.04 QALYs (95% credible interval [Crl] 0.01–0.07) and increased mean costs of $1,257 (95% Crl $630–$2,061), yielding an ICER of $31,575 per QALY gained.
In the reference case analysis including adults with major depression, the probability of cost-effectiveness of guided iCBT was 63% at a willingness-to-pay value of $50,000/QALY and increased to 67% at a willingness-to-pay value of $100,000/QALY (Figure 4). The probability of cost-effectiveness of guided iCBT was the highest (87% at willingness-to-pay of $50,000/QALY) when we assumed that guided iCBT was provided solely by the employer.
Figure 4: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT, In-Person CBT, and Usual Care for Mild to Moderate Major Depression.
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. Note: The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care across various willingness-to-pay values. The x-axis shows the probability of cost-effectiveness (range: 0 to 1) and the y-axis represents willingness-to-pay values (range: $0 to $100,000 per QALY gained).
All other analyses that considered guided iCBT by a different type of therapist (i.e., coach-guided) or a different type of provider (i.e., employer-led) identified guided iCBT as the best option of all strategies with the following ICERs (compared with usual care):
$17,092 per QALY if guided iCBT was provided solely by the employer
$29,651 per QALY if guided iCBT was provided by a coach-therapist
$31,691 per QALY if a higher cost range was used for all iCBT strategies (cost estimates presented in Table 13)
In addition, the results remained robust in the sensitivity analysis considering the lower costs of delivery of individual or group CBT, showing guided iCBT as the optimal strategy, with an ICER for guided versus unguided iCBT of $39,670 per QALY (Appendix 7, Table A11).
Finally, the results remained robust in the sensitivity analysis considering that a portion of individuals take medications from baseline (Table 19). Guided iCBT was the optimal strategy, with an ICER for guided iCBT versus usual care of $33,397 per QALY (Appendix 8, Table A13).
Cost–Utility Analysis for Anxiety Disorders
In the second population of adults with an anxiety disorder, a sequential cost–utility analysis that ranked five strategies by increasing costs, usual care was dominated by unguided iCBT and group CBT was excluded by extended dominance from the final calculations (ICER of group CBT vs unguided iCBT: $65,000 per QALY). If compared with usual care (i.e., the only publicly funded option), guided iCBT was associated with an ICER of $26,719/QALY gained. Compared with unguided iCBT, guided iCBT (i.e., therapist-guided via web portal) was associated with an increased quality-adjusted survival of 0.03 QALYs (95% Crl 0.01–0.06) and increased mean costs of $1,395 (95% Crl $767–$2,192), yielding an ICER of $43,214 per QALY gained (Table 19).
In the reference case analysis including adults with anxiety disorders, the probability of cost-effectiveness of guided iCBT was 49% at willingness-to-pay of $50,000 per QALY gained and increased to 70% at willingness-to-pay of $100,000 per QALY gained (Figure 5). The probability of cost-effectiveness of guided iCBT was the highest (80% at a willingness-to-pay value of $50,000/QALY) when we assumed that guided iCBT was solely provided by the employer.
Figure 5: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT, In-Person CBT, and Usual Care for Anxiety Disorders.
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-guided CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. Note: The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care across various willingness-to-pay values on the x–y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0–1) and the y-axis represents willingness-to-pay values (range: $0–$100,000 per QALY gained).
All other analyses that considered guided iCBT by a different type of therapist (i.e., coach-guided) or a different type of provider (i.e., employer-led) identified guided iCBT as the best option of all strategies with an ICER (comparing it with unguided iCBT) of:
$25,349 per QALY if guided iCBT was provided solely by the employer
$40,960 per QALY if guided iCBT was provided by a coach-therapist
$40,069 per QALY if a higher cost range was used for all iCBT strategies (Table 13)
In addition, in the sensitivity analysis considering lower costs of delivery of individual or group CBT, group CBT and guided iCBT were economically attractive with the ICERs below $50,000 per QALY. The ICER of group CBT versus usual care was $38,515 per QALY (incremental mean costs of $597 and incremental mean QALYs of 0.015) and the ICER for guided iCBT versus group CBT was $47,753 (incremental mean costs of $800 and incremental mean QALYs of 0.017) (Appendix 7, Table A12).
The results remained robust in the sensitivity analysis considering that a portion of individuals take medications from baseline (see Table 16). Guided iCBT was the optimal strategy, with the ICER of guided versus unguided iCBT of $43,250 per QALY (Appendix 8, Table A14).
Sensitivity Analysis: Reference Case Analysis
Coach-Based Ontario Program (Scenario A)
The coach-based program used as an example for this scenario assumed a probability of response to iCBT at 40%. We found that all guided CBT programs (therapist or face to face) were associated with increments in QALYs and costs as compared to each other (in a ranking order). However, the estimated ICERs were around or below a willingness-to-pay value of $100,000 per QALY gained (Table 20). The probability of this program being cost-effective compared to other options was 48% at a willingness-to-pay value of $50,000 per QALY gained and 28% at a willingness-to-pay value of $100,000 per QALY gained.
Table 20:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and Usual Care for Major Depression
| Strategy | Mean Costs, $a | Mean QALYs | Incremental Costs, $a,b | Incremental QALYsc | ICER, $/QALY Gaineda |
|---|---|---|---|---|---|
| Unguided iCBT | 381.62 | 0.777 | – | – | – |
| Usual care | 409.40 | 0.787 | 27.78 | 0.0092 | 3,004 |
| Ontario coach-based iCBT | 989.321 | 0.806 | 579.92 | 0.0197 | 29,387 |
| Group CBT | 1,690.15 | 0.817 | 700.83 | 0.0105 | 66,836 |
| Individual CBT | 2,654.08 | 0.827 | 963.93 | 0.0096 | 100,312 |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (strategy of individual CBT) – average cost (strategy of group CBT).
Incremental effect = average effect (strategy of individual CBT) – average effect (strategy of group CBT).
Note: Results may appear incorrect because of rounding.
Physician-Guided iCBT (Scenario B) and Company-Led Program (Scenario C)
Both scenarios found that guided iCBT was the optimal strategy. In scenario B, the ICER was $34,625 per QALY gained (guided iCBT versus usual care) in adults with major depression and $47,045 per QALY gained (guided vs unguided iCBT). In scenario C, the ICER was $16,790 per QALY (guided iCBT vs usual care) in adults with major depression, and $33,435 per QALY gained in adults with anxiety disorders (guided vs unguided iCBT).
Additional Scenario: Societal Perspective
Guided iCBT was the optimal strategy for adults with major depression, associated with an ICER of $31,660 per QALY gained (Table 21). The probability of guided iCBT being cost-effective as compared to other strategies was 78% and 92% at willingness-to-pay values of $50,000 and $100,000 per QALY gained. Similar findings were found in a second analysis that included adults with anxiety disorders.
Table 21:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and Usual Care for Major Depression, From the Societal Perspective
| Strategy | Mean Costs, $a,b | Mean QALYs | Incremental Costs, $b,c | Incremental QALYsd | ICER, $/QALY Gaineda |
|---|---|---|---|---|---|
| Unguided iCBT | 274.53 | 0.777 | — | — | — |
| Usual care | 409.40 | 0.787 | 134.79 | 0.009 | 14,551 |
| Guided iCBT | 1,667.41 | 0.826 | 1,258.80 | 0.039 | 31,660 |
| Group CBT | 3,574.92 | 0.817 | — | — | Dominated |
| Individual CBT | 4,179.31 | 0.826 | 2,511.80 | 0.000 | Dominatede |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
Costs estimated in Table 16, additional societal costs associated with in-person CBT of $2,179.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (strategy of guided CBT) – average cost (strategy of usual care).
Incremental effect = average effect (strategy of guided CBT) – average effect (strategy of usual care).
This is essentially dominated by guided iCBT because, after rounding, it provides a benefit equal to that of guided iCBT but is more costly; the estimated ICER for individual CBT vs. guided iCBT would be approximately $65 million if we used an actual increment in QALYs of 0.0004.
Note: Results may appear incorrect because of rounding.
Additional Sensitivity Analyses
We performed additional sensitivity analyses for each parameter and found that, while the cost-effectiveness results remained robust to changes in dropout rates from iCBT or CBT strategies, they were sensitive to changes in the probability of recovery and changes in the utility weight associated with guided iCBT.
Influential Parameters
Recent findings of the UK IAPT program79,86,115 suggested that the recovery rate of guided iCBT could be much higher than the rate found in the published evidence. Thus, if the recovery rate with guided iCBT increased to 60% from the 48% found in the reference case for major depression, the ICER would substantially decrease, from $31,575 to $25,100 per QALY gained (Table 22). In addition, the probability of cost-effectiveness of guided iCBT over other therapies would increase to over 90% at a willingness-to-pay value of $50,000 per QALY gained (from 63% in the reference case analysis).
Table 22:
Sensitivity Analysis, Sequential Approach: Increased Recovery Rate With Guided iCBT
| Strategy | Mean Costs, $a,b | Mean QALYs | Incremental Costs, $b,c | Incremental QALYsd | ICER, $/QALY Gainedb |
|---|---|---|---|---|---|
| Unguided iCBT | 274.62 | 0.777 | — | — | — |
| Usual care | 409.40 | 0.787 | 134.78 | 0.009 | 14,644 |
| Guided iCBT | 1,660.09 | 0.838 | 1,278.64 | 0.051 | 25,110 |
| Group CBT | 1,688.05 | 0.817 | — | — | Dominated |
| Individual CBT | 2,657.27 | 0.827 | — | — | Dominated |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
Costs estimated in Table 16, additional societal costs associated with in-person CBT of $1,244.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (strategy of guided CBT) – average cost (strategy of usual care).
Incremental effect = average effect (strategy of guided CBT) – average effect (strategy of usual care).
Note: Results may appear incorrect because of rounding.
Publication bias has been seen in some studies.77 If the utility weight of guided iCBT therapy decreased by 5% (from 0.83 in the reference case to 0.79), the ICER of guided iCBT versus usual care would increase to about $39,200 per QALY gained (compared to $31,575 in the reference case). The ICER of group versus guided CBT would be smaller, resulting in exclusion of guided iCBT from the calculations by extended dominance. Group CBT versus usual care is associated with an ICER of about $43,000 per QALY gained and becomes less favorable as compared to individual CBT (with the ICER of about $38,700 per QALY gained). Individual CBT versus usual care becomes optimal associated with an ICER of about $56,000 per QALY gained when compared with usual care (Table 23).
Table 23:
Sensitivity Analysis, Sequential Approach: Change in Utility Weight Associated With Guided iCBT
| Strategy | Mean Costs, $a | Mean QALYs | Incremental Costs, $a,b | Incremental QALYsc | ICER, $/QALY Gaineda | ICER, cont. $/QALY Gaineda,d |
|---|---|---|---|---|---|---|
| Unguided iCBT | 274.62 | 0.777 | — | — | — | — |
| Usual care | 409.40 | 0.787 | 134.79 | 0.009 | 38,931 | |
| Guided iCBT | 1,666.86 | 0.81701 | 1,257.46 | 0.030 | 39,184 | Extended dominance |
| Group CBT | 1,690.15 | 0.81704 | 23.29 | 0.00003 | 39,162d | 42,700 |
| Individual CBT | 2,654.08 | 0.8266 | 963.93 | 0.0096 | 38,678 | 56,115 |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (strategy of guided CBT) – average cost (strategy of usual care).
Incremental effect = average effect (strategy of guided CBT) – average effect (strategy of usual care).
The ICER of group CBT vs. usual care changed to $42,700/QALY after exclusion of guided iCBT by extended dominance.
Note: Results may appear incorrect because of rounding.
Additionally, the probability of cost-effectiveness of guided iCBT over other therapies would decrease to 38% at a willingness-to-pay value of $50,000 per QALY gained (from 63% in the reference case; Figure 6). In this analysis, the probability of cost-effectiveness of all CBT therapies, when compared to each other, remains quite low (up to 40%) over a wide range of willingness-to-pay values.
Figure 6: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT, In-Person CBT, and Usual Care for Major Depression, a Sensitivity Analysis Assuming a 5% Decrease in the Utility Weight Associated With Guided iCBT.
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care across various willingness-to-pay values on the x–y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0–1) and the y-axis represents willingness-to-pay values (range: $0 to $100,000 per QALY gained).
Secondary Research Questions – Exploratory Scenario Analyses
Lifetime Cost-Effectiveness of Guided iCBT Within a Stepped-Care Model for Adults With Mild to Moderate Major Depression as Primary Diagnosis
Both strategies that included guided iCBT were cost saving and associated with increments in QALYs when compared with usual care. Over a lifetime time horizon using a stepped-care model, guided iCBT followed by individual face-to-face CBT represented an economically attractive option when compared with guided iCBT followed by group CBT (ICER of $1,100/QALY gained). The results remained robust in scenario analyses addressing parameter uncertainty (e.g., changes in the efficacy of iCBT and CBT, dropout rates, utilities, and medication costs) and duration of time horizon. We present detailed results of this analysis in Appendix 9.
Unguided Followed by Guided iCBT
Adults With Mild to Moderate Major Depression
In adults with mild to moderate major depression, a sequential cost–utility analysis ranking four strategies by increasing costs found that all CBT strategies were associated with increments in cost and benefits as compared with usual care (Table 24). Compared with usual care, unguided iCBT followed by guided CBT was associated with an increased quality-adjusted survival of 0.02 QALYs and increased mean costs of $306, yielding the most favourable ICER, $19,454 per QALY gained. Further, the ICER for group CBT versus iCBT was about $67,000 per QALY gained, and the ICER for individual versus group CBT was about $100,000 per QALY gained.
Table 24:
Cost–Utility Analysis: Stepped-Care iCBT, In-Person CBT, and Usual Care for Major Depression
| Strategy | Mean Costs, $a | Mean QALYs | Incremental Costs, $a,b | Incremental QALYsc | ICER: Strategy vs. Usual Care, $/QALY Gained |
|---|---|---|---|---|---|
| Usual care | 409.40 | 0.7869 | – | – | – |
| Unguided followed by guided iCBT | 715.41 | 0.8026 | 306.00 | 0.0157 | 19,454 |
| Group CBT | 1,690.18 | 0.8171 | 974.77 | 0.0145 | 67,161 |
| Individual CBT | 2,654.39 | 0.8267 | 964.22 | 0.0096 | 100,316 |
Abbreviations: CBT, cognitive behavioural therapy; CrI, credible interval; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (strategy unguided CBT) – average cost (strategy usual care).
Incremental effect = average effect (strategy guided CBT) – average effect (strategy usual care).
Note: Results may appear incorrect because of rounding.
In analyses that included adults with major depression, the probability of cost-effectiveness of unguided followed by guided iCBT in those who tend to dropout of unguided iCBT was 48% at a willingness-to-pay value of $50,000 per QALY gained. This decreased to 27% at a willingness-to-pay value of $100,000 per QALY gained (Figure 7). At these two willingness-to-pay values, the probability of cost-effectiveness of group CBT was 28% and 33%, and individual CBT was 10% and 39%. If the dropout rate from unguided iCBT raised from 32% (in the base case) to 75%, the probability of cost-effectiveness of unguided followed by guided iCBT was 94% at a willingness-to-pay of $50,000/QALY and of $100,000/QALY.
Figure 7: Cost-Effectiveness Acceptability Curve: Internet-Delivered CBT (Unguided Followed by Guided), In-Person CBT, and Usual Care for Mild to Moderate Major Depression.
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care across various willingness to pay values on the x–y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0–1) and the y-axis represents willingness to pay values (range: $0 to $100,000 per QALY gained).
Adults With Anxiety Disorders
In adults with an anxiety disorder, a sequential cost–utility analysis ranking four strategies by increasing costs found that group CBT was dominated by stepped-care iCBT (Table 25). Compared with usual care, unguided iCBT followed by guided CBT was associated with an increased quality-adjusted survival of 0.04 QALYs and increased mean costs of $301, yielding the most favorable ICER of $7,670 per QALY gained. Further, the ICER of individual CBT versus stepped-care iCBT was over $2 million per QALY gained due to a small increment in QALYs with individual CBT. Stepped-care iCBT, therefore, seems to be the optimal economic option for the initial treatment of adults with anxiety.
Table 25:
Cost–Utility Analysis: Stepped-Care iCBT, In-Person CBT, and Usual Care for Anxiety
| Strategy | Mean Costs, $a | Mean QALYs | Incremental Costs, $a,b | Incremental QALYsc | ICER $/QALY Gained |
|---|---|---|---|---|---|
| Usual care | 409.40 | 0.787 | — | — | — |
| Unguided followed by guided iCBT | 710.62 | 0.826 | 301.21 | 0.039 | 7,667 |
| Group CBT | 1,276.23 | 0.817 | 565.62 | –0.009 | Dominated |
| Individual CBT | 1,954.99 | 0.827 | 1,244.37 | 0.001 | 2,223,063 |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (strategy of guided CBT) – average cost (strategy of usual care).
Incremental effect = average effect (strategy of guided CBT) – average effect (strategy of usual care). The exact incremental QALYs were about 0.00056, which we rounded to 0.001.
Note: Results may appear incorrect because of rounding.
In an analysis including adults with anxiety disorders, the probability of cost-effectiveness of unguided followed by guided iCBT in those who tend to dropout from unguided iCBT was 86% at a willingness-to-pay value of $50,000 per QALY gained. This decreased to 75% at a willingness-to-pay value of $100,000 per QALY gained (Figure 8). At these two willingness-to-pay values, the probability of cost-effectiveness of group CBT was 8% and 7% and individual CBT was 5% and 18%, respectively. If the dropout from unguided iCBT raised from 32% (in the base case) to 75%, the probability of cost-effectiveness of unguided followed by guided iCBT was 70% and 72% at willingness-to-pay values of $50,000 and $100,000 per QALY gained, respectively.
Figure 8: Cost-Effectiveness–Acceptability Curve: Internet-Delivered CBT, In-Person CBT, and Usual Care for Anxiety Disorders.
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay. The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care across various willingness-to-pay values on the x–y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0–1) and the y-axis represents willingness-to-pay values (range: $0 to $100,000 per QALY gained).
Discussion
Our analysis employed several decision-modeling approaches to examine the cost-effectiveness of iCBT for the treatment of mild to moderate major depression or anxiety disorders. We showed that guided iCBT may represent an economically attractive strategy option when used as a one-time, non-repetitive short-term treatment or when combined with inperson CBT over a long-term time horizon. In all analyses, guided iCBT was associated with ICERs of less than $50,000 per QALY gained.
When all CBT strategies and usual care were compared using a sequential approach in the reference case cost–utility analyses, guided iCBT was selected as the best ranking option, while its clinical benefits (QALYs) and those associated with in-person CBT strategies were similar. For instance, over a 1-year time horizon in adults with mild to moderate major depression, guided iCBT was associated with increases in mean QALY of 0.04 (95% credible interval [Crl] 0.01–0.07) and mean costs of $1,257 (95% Crl $630 to $2,061) compared with usual care (the ICER: about 31,000 per QALY gained). Similar results were shown in the cost–utility analysis in adults with anxiety disorders (the ICER for guided vs unguided iCBT was about $43,000 per QALY gained). Over a short-term time horizon, uncertainty around the cost-effectiveness estimates was moderately high, depending on the maximum amount a decision-maker would be willing to pay for this treatment (e.g., about 60% for major depression and about 50% for anxiety at $50,000 per QALY gained).
We conducted several sensitivity analyses to explore the robustness of the reference case results. In most analyses, the ICER for guided iCBT remained robust and at similar probabilities of cost-effectiveness below the commonly used willingness to pay value of $50,000 per QALY gained. However, the probability of cost-effectiveness substantially increased to over 80% at a willingness to pay value of $50,000 per QALY gained if the cost of guided iCBT decreased by about half (a scenario analysis with employer-led guided iCBT). In addition to the cost of delivery, a decrease in the probability of response to guided iCBT versus usual care from around 70% to 40% as reported in scenario A, would lead to a very high uncertainty around the estimate of ICER (i.e., 40% chance of guided iCBT to be cost-effective at willingness to pay of $50,000/QALY and 28% of guided iCBT to be cost-effective at willingness to pay of $100,000/QALY). For the reference case analysis, the recovery rate after guided iCBT was assumed to be around 48% based on estimates reported in a study21 included in our review.21
Recovery rate is an important outcome and real-word data from the UK Improving Access for Psychological Therapies (IAPT) program79,86,115 suggest that guided iCBT could be associated with much higher recovery rates. Our sensitivity analysis addressing this change, assuming an increase in the recovery rate of 60% (as shown in the UK IAPT program79,86,115), suggests a substantially large increment in certainty around the ICER estimate of approximately 90% at a willingness to pay of $50,000/QALY. If the improvement in health state utility after the treatment with iCBT was smaller by 5%, then guided iCBT would not represent the best ranking option and would be dominated by group CBT (see Table 23). This finding indicates the importance of determining a true change in the utility weight for health states when guided iCBT is used instead of in-person CBT, as the evidence is very limited, and the studies included in our review suggested very small differences in patients' quality of life between iCBT and in-person CBT options. Further studies need to examine the advantages of iCBT with respect to overcoming barriers to access and stigma, as well as its convenience of use and the correlation of these specific iCBT features with changes in quality of life for different subgroups of individuals suffering from major depression and anxiety disorders.
Although our secondary analyses were more complex and exploratory in nature, their findings are in line with the results of the reference case analysis. They show that guided iCBT is optimal and economically attractive over a patient's lifetime when delivered as the first-step of a stepped-care model and is followed by in-person CBT for people who have recurrent episodes of anxiety or major depression (the ICER, depending on the format of in-person CBT and the comparator, was either cost saving or $1,100 per QALY gained). Another secondary analysis exploring a subsequent delivery of guided iCBT in those who tend to drop out from unguided iCBT also showed that a stepped-care iCBT approach represents good value for money versus usual care (ICER: $19,500 per QALY gained). This analysis partially addressed the issue related to high attrition rates in people on unguided iCBT, including a lack of understanding of patient readiness for iCBT.
Equity
Barriers to in-person CBT include stigma, geography, time, cost, and lack of treatment fidelity. Access to specialized providers varies depending upon where someone lives within Ontario. Internet-delivered CBT represents an additional option that may help overcome these barriers and potentially increase access to treatment.116
Although iCBT reduces barriers to access to mental health services, it is important to acknowledge that people who are not comfortable with technology may not be ideal candidates for iCBT. However, recent patient-preference surveys65 suggest that iCBT guided by therapists could become a preferred option over in-person CBT or medication. Current therapist-guided iCBT is developed in English language and is available in French as of July 3, 2018 (P. Farvolden, PhD, written and oral communications, January to July 2018); however, some inequities may remain for recent immigrants and non-native English or French speakers.117,118 Tailoring iCBT programs for different populations, including Aboriginal and First Nation populations, will be required to enable equal and effective access to all eligible people with depression and anxiety.
Limitations
Although we used a comprehensive analytic approach to this economic analysis, we recognize several limitations:
Our analysis did not examine the effectiveness and cost-effectiveness of any Ontario-specific iCBT program; our input parameters related to the effectiveness of iCBT were based on the literature. The clinical utility of guided iCBT should be carefully examined during the procurement and implementation process (with weekly monitoring of clinical outcomes as implemented in the IAPT program), given the wide variety of commercial and non-commercial iCBT packages available
In the reference case, the effectiveness of iCBT, based on the published evidence, was lower than estimates recognized in real-word effectiveness programs such as the UK Improving Access for Psychological Therapies program (IAPT).79,86,115 However, the cost-effectiveness of guided iCBT based on finding of the UK IAPT program (considered in a scenario analysis) was in line with the results of reference case analysis
The evidence is limited for the relative effectiveness of guided iCBT versus in-person CBT because there were no large noninferiority studies published in the literature. We made potentially strong assumptions related to the probability of response after guided iCBT, assuming no difference between guided iCBT and face-to-face CBT. However, our estimates related to the probability of recovery reflected differences in the effects between these two CBT treatments. To account for the limitations of the evidence, we defined the use of iCBT in our modeling studies as one-time, non-repetitive therapy provided within a short time frame. Future studies need to confirm the equivalence of face-to-face CBT and guided iCBT for the range of target populations that we examined in our analyses
The costs associated with delivery of iCBT are based on plausible but hypothetical scenarios potentially overestimating the costs of training and supervision for the reference case analysis considering therapist-guided iCBT. Nevertheless, we made assumptions related to the cost of delivery of iCBT in various scenarios, and guided iCBT remained the optimal strategy in all scenarios
The lifetime cost-effectiveness of guided iCBT with a therapist as a single therapy for the treatment of reccurent major depression and anxiety remains unclear. We examined the long-term cost-effectiveness of guided iCBT over usual care in a stepped-care modelling approach (secondary research question) because this approach was most clinically plausible, was evidence-based, and was supported by expert consultations and current policy development in Ontario
In the long-term lifetime model, we did not fully address the complexity of secondary psychiatric care, nor did we model the use of CBT in complex depression or the effectiveness of various additional types of treatment for treatment-resistant depression. Due to limitations in the existing literature, we also assumed that the efficacy of CBT is the same for every new episode and that there is no difference in the effectiveness of CBT delivery among the various types of regulated providers of psychotherapy services
Waiting time costs associated with in-person CBT were not included in our analyses
A scenario analysis addressing a societal perspective did not include time loss and costs incurred by caregivers
Conclusions
Compared with other available options, guided iCBT may represent an optimal and economically attractive alternative and could be offered as an initial step for the short-term treatment of eligible adults with mild to moderate major depression and/or anxiety disorders.
BUDGET IMPACT ANALYSIS
We conducted a budget impact analysis to estimate the cost burden of publicly funding iCBT for individuals with mild to moderate major depression or anxiety disorders in Ontario over the next 5 years. The analysis was done from the perspective of the Ontario Ministry of Health and Long-Term Care. All costs were reported in 2018 Canadian dollars. Reporting and analysis were done in accordance with the 2012 International Society for Pharmacoeconomics and Outcomes Research good-practice guidelines for budget impact analysis.119
Research Questions
What is the net budget impact of publicly funding unguided or guided iCBT for adults with mild to moderate major depression in Ontario over the next 5 years?
What is the net budget impact of publicly funding unguided or guided iCBT for adults with anxiety disorders in Ontario over the next 5 years?
Methods
Analytic Framework
We estimated the budget impact of unguided and guided iCBT interventions using the cost difference between two scenarios: current clinical practice without publicly funded iCBT (the current scenario), and the anticipated clinical practice with the publicly funded iCBT for individuals with mild to moderate major depression or anxiety disorders (the new scenario). All strategies related to current and new scenarios were described in our primary economic evaluation (see Table 7), where they are referred to as comparator (usual care) and intervention (iCBT). The model schematic is shown in Figure 9.
Figure 9: Budget Impact Model Schematic.
Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.
We conducted a reference case analysis and several sensitivity analyses. For all analyses, we used outputs from our cost-effectiveness model to estimate budget impact. By doing so, we ensured that our budget impact analysis accounted for heterogeneity in the patient populations with respect to disease severity, and potential differences in response to treatments, dropout rates, and subsequent differences in resource use and costs.
Key Assumptions
The assumptions in this analysis are described in the primary economic evaluation. In addition, we considered the following:
Our target population was based on published prevalence data; the prevalence of anxiety disorders in the reference case may be underestimated
A certain percentage of people with substance-abuse disorders are considered eligible for iCBT (S. Rao, MD, and J. Hensel, MD, written communications, May 2018)
Eligibility of the target population was not restricted by patient readiness for iCBT; we assumed that people self-referred and were therefore ready and willing to try this type of CBT
Because the course of iCBT was paid upon initial sign up, public funding would still cover the program costs of people who did not complete the full course
Target Population
The target population is adults diagnosed with mild to moderate major depression or any anxiety disorder (as defined by DSM-5 criteria) who are eligible to use iCBT in Ontario. The lifetime prevalence of major depression in Canada is 10.8%; annual and 1-month prevalence estimates are about 4.0% and 1.3%, respectively.120–123 Mild to moderate major depression occurs in approximately 80% of patients with prevalent major depression.75,76 Anxiety disorders are among the most common mental disorders, with lifetime prevalence rates ranging from 10.4% to 28.8%, and 12-month prevalence rates of about 18%.88 Other Canadian data, accounting for differences in diagnosis of anxiety between ICD-10 and DSM-IV suggest a lower 12-month prevalence of any anxiety disorder of 10.2%.124 Generalized anxiety disorder is the most common form, with a 12-month prevalence rate of 2.5%.10
Approximately 50% of people with major depression also suffer from an anxiety disorder.10 The percentage of coexisting anxiety disorders is even higher, with 59% to 79% of people with anxiety disorder having more than one. In our estimation of the target populations, we will focus on the primary diagnosis of either major depression (with or without a coexisting anxiety disorder) or anxiety alone.
Based on population-based data, the incidence of intellectual disability co-occurring in patients with a mental illness that cannot engage in CBT ranges between 3.7% and 5.2%.125 Eligibility for iCBT is dependent on access to a computer. About 89% of people in Canada have access to computers and routinely access the internet.126 We adjusted the population for ineligibility to iCBT due to learning disabilities (5%)125 and due to a substance use disorder (10%).
Tables 26 and 27 present the prevalence of major depression and anxiety disorders, adjusted for the population growth and the corresponding number of patients eligible for iCBT with major depression and/or an anxiety disorder. The annual number of eligible individuals with mild to moderate major depression ranges from about 266,400 to 277,200.
Table 26:
Number of People With Mild to Moderate Major Depression (Combined With Anxiety) Eligible for iCBT in Ontario, 2018 to 2022
| Year | Estimated No. of Adults in Ontarioa | No. of People With Major Depressionb | No. of People With Major Depression Eligible for CBTc | No of People With Mild to Moderate Major Depression Eligible for iCBTd |
|---|---|---|---|---|
| 2018 | 11,287,810 | 440,225 | 333,030 | 266,424 |
| 2019 | 11,400,688 | 444,627 | 336,360 | 269,088 |
| 2020 | 11,514,695 | 449,073 | 339,724 | 271,779 |
| 2020 | 11,629,842 | 453,564 | 343,121 | 274,497 |
| 2021 | 11,746,140 | 458,099 | 346,552 | 277,242 |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT.
Assuming 1% growth in population per year (Statistics Canada).
Assuming major depression prevalence of 3.9%.
After excluding 11% with no access to the internet and 15% unable to engage in CBT because of disability or substance abuse.
Assuming the proportion of mild to moderate major depression is 80% of eligible cases.
Table 27:
Number of People With Anxiety Disorder Eligible for iCBT in Ontario, 2018 to 2022
| Year | Estimated No. of Adults in Ontarioa | No. of People With Anxiety Disordersb | Combined major depression and Anxiety, % | No. of People With Anxiety Disorders Only | No. of People With Anxiety Disorder Alone Eligible for iCBTc |
|---|---|---|---|---|---|
| 2018 | 11,287,810 | 1,151,357 | 50 | 575,678 | 435,500 |
| 2019 | 11,400,688 | 1,162,870 | 50 | 581,435 | 439,856 |
| 2020 | 11,514,695 | 1,174,499 | 50 | 587,249 | 444,254 |
| 2020 | 11,629,842 | 1,186,244 | 50 | 593,122 | 448,697 |
| 2021 | 11,746,140 | 1,198,106 | 50 | 599,053 | 453,186 |
Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.
Assuming 1% growth in population per year (Statistics Canada).
Assuming prevalence of 10.2%.
After excluding 11% with no access to the internet and 15% unable to engage in CBT because of disability or substance abuse.
Of note, there is no restriction in eligibility to iCBT in people with anxiety disorders with respect to the severity of the disease. As a result, the number of potentially eligible people with anxiety is much greater when compared with those with major depression. The annual number of potentially eligible individuals with anxiety disorders ranges from about 435,500 to 453,200.
Current Intervention Mix
We are not aware of any intervention mix of publicly funded iCBT and in-person CBT for people with mild to moderate major depression or anxiety disorders. We used the change in uptake of iCBT over time and calculated the budget impact separately for guided and unguided iCBT strategies. We assumed no changes in the costs of usual care with the introduction of iCBT.
Uptake of the New Intervention and Market Effects
In the reference case, we assumed that access to iCBT would increase by 3% each year (from 0% to 15% over 5 years). This indicates very limited access to publicly funded iCBT at baseline.116 The degree of the influence of an increasing access rate over time on the net budget impact will be tested in our sensitivity analysis. Based on expert consultation and experience in other Canadian provinces, a growth of up to 25% could be expected with proper government funded (P. Farvolden, MD, written communication, May 2018).
Assuming 3% access to iCBT, we estimated the number of people with major depression or an anxiety disorder in year 1 to be about 7,990 and 13,060, respectively (Tables 28 and 29). The estimate of all people at risk over a 5-year period took into account the uptake rate, prevalence of mild to moderate major depression/anxiety disorder in the first year, and subsequent changes in the distribution of eligible individuals (assuming a 15% recurrence each year127 accounting for people who become ineligible for iCBT due to disease progression).
Table 28:
Estimate of Eligible Cohort of Patients With Mild to Moderate Major Depression in Ontario, Years 1–5
| Year | No. of People With Mild to Moderate Major Depression Eligible for iCBT | Cohort Estimate, Accounting for Disease Progression and People who Had iCBT in Prior Year, Na | Yearly Uptake of iCBT | Target Population Eligible for iCBT, N | Target Population Continuing with Usual Care, N |
|---|---|---|---|---|---|
| Year 1 | 266,424 | 266,424 | 0.03 | 7,993 | 258,431 |
| Year 2 | 269,088 | 220,732 | 0.06 | 13,244 | 255,844 |
| Year 3 | 271,779 | 217,768 | 0.09 | 19,599 | 252,180 |
| Year 4 | 274,497 | 213,723 | 0.12 | 25,647 | 248,850 |
| Year 5 | 277,242 | 210,009 | 0.15 | 31,501 | 245,740 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy.
Assuming 15% had recurrence and progressed to more severe major depression and became ineligible for iCBT course.
Table 29:
Estimate of Eligible Cohort of Patients With Anxiety Disorders in Ontario, Years 1–5
| Year | No. of People With Mild to Moderate Major Depression Eligible for iCBT | Cohort Estimate, Accounting for Disease Progression and People who Had iCBT in Prior Year, Na | Yearly Uptake of iCBT | Target Population Eligible for iCBT, N | Target Population Continuing with Usual Care, N |
|---|---|---|---|---|---|
| Year 1 | 435,500 | 435,500 | 0.03 | 13,065 | 422,435 |
| Year 2 | 439,856 | 360,812 | 0.06 | 21,649 | 418,207 |
| Year 3 | 444,254 | 355,967 | 0.09 | 32,037 | 412,217 |
| Year 4 | 448,697 | 349,355 | 0.12 | 41,923 | 406,774 |
| Year 5 | 453,184 | 343,283 | 0.15 | 51,493 | 401,691 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy.
Assuming 15% had recurrence and progressed to more severe major depression and became ineligible for iCBT course.
Future Intervention Mix
No mix of psychological interventions is expected for the future scenario in the management of mild to moderate major depression or anxiety disorder. Without knowing the exact changes in treatment mix over time, we assumed that iCBT would not substitute for any type of in-person CBT in Ontario over the next 5 years and would be only an addition to usual care.
Resource and Costs
The costs were derived from our deterministic cost–utility analysis. As shown in Figure 9, for the new scenario, we assumed that the costs of iCBT varied by the type of delivery (guided vs unguided). The total cost estimates were based on the model outputs of undiscounted direct medical costs. In adults with major depression, guided (therapist) and unguided iCBT were associated with a mean cost of $1,666 and $275, respectively. Usual care was associated with a mean cost of $409. In adults with anxiety disorders, guided (therapist) and unguided iCBT were associated with a mean cost of $1,665 and $270, respectively. Usual care was associated with a mean cost of $409. Table 30 presents the total costs associated with the future and current scenarios over the next 5 years for adults with mild to moderate major depression. Table 31 presents the corresponding costs for adults with anxiety disorders.
Table 30:
Future and Current Scenarios, Cost Estimates Over 5 Years—Reference Case Analysis: Patients With Mild to Moderate Major Depression
| Year | Future Scenario | Current Scenario | ||||
|---|---|---|---|---|---|---|
| Guided iCBT | Unguided iCBT | Usual Care | Guided iCBTa | Unguided iCBTa | Usual Care | |
| Year 1 | $13,317,921 | $2,195,237 | $105,802,833 | $119,120,754 | $107,998,070 | $109,075,086 |
| Year 2 | $22,067,795 | $3,637,507 | $104,743,714 | $126,811,509 | $108,381,221 | $110,165,837 |
| Year 3 | $32,657,207 | $5,382,995 | $103,243,523 | $135,900,730 | $108,626,517 | $111,267,495 |
| Year 4 | $42,734,124 | $7,044,006 | $101,880,269 | $144,614,394 | $108,924,275 | $112,380,170 |
| Year 5 | $52,489,280 | $8,651,980 | $100,607,200 | $153,096,481 | $109,259,181 | $113,503,972 |
Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.
Totals for future scenarios with guided or unguided iCBT. They represent sums of data presented for guided or unguided iCBT and for usual care.
Table 31:
Future and Current Scenarios, Cost Estimates Over 5 Years—Reference Case Analysis: Patients With Anxiety Disorders
| Year | Future Scenario | Current Scenario | ||||
|---|---|---|---|---|---|---|
| Guided iCBT | Unguided iCBT | Usual Care | Guided iCBTa | Unguided iCBTa | Usual Care | |
| Year 1 | $21,755,489 | $3,523,618 | $172,946,846 | $194,702,335 | $176,470,464 | $178,295,717 |
| Year 2 | $36,048,860 | $5,838,637 | $171,215,659 | $207,264,519 | $177,054,296 | $180,078,743 |
| Year 3 | $53,347,166 | $8,640,349 | $168,763,323 | $222,110,490 | $177,403,673 | $181,879,422 |
| Year 4 | $69,808,378 | $11,306,482 | $166,535,072 | $236,343,449 | $177,841,554 | $183,698,374 |
| Year 5 | $85,743,881 | $13,887,469 | $164,454,035 | $250,197,916 | $178,341,504 | $185,535,290 |
Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.
Totals for future scenarios with guided or unguided iCBT. They represent sums of data presented for guided or unguided iCBT and for usual care.
Analysis
To address the Ontario context for the different scenarios, we conducted the following budget impact analyses:
Reference case analysis—to estimate the net budget impact of therapist-guided iCBT provided by regulated health care professionals (nonphysicians) compared with usual care for adults with mild to moderate depression and adults with anxiety disorders
Reference case analysis—to estimate the net budget impact of unguided iCBT compared with usual care for adults with mild to moderate depression and adults with anxiety disorders
Sensitivity analysis, scenario 1—to estimate how increases in access to iCBT over time (i.e., increases in the uptake rate) affect the net budget impact of therapist-guided iCBT. The uptake rate used in this scenario was 5% in year 1, increasing 5% per year to 25% in Year 5 (the reference case analysis assumed 3% in year 1, increasing 3% per year, reaching 15% in year 5)
Sensitivity analysis, scenario 2—to estimate the net budget impact of coach-guided iCBT provided by nonregulated mental health workers. The costs associated with coach-guided iCBT were estimated at about $1,590 per person per year (the corresponding cost for therapist-based iCBT used in the reference case analysis was about $1,670 per person per year). The costs of usual care remained the same as in the reference case analysis ($409 per person per year)
Sensitivity analysis, scenario 3—to estimate the net budget impact of guided iCBT provided by regulated health care professionals, assuming no training costs. The costs associated with guided iCBT were estimated at about $1,087 per person per year (the corresponding cost for therapist-based iCBT used in the reference case analysis was about $1,670 per person per year). The costs of usual care remained the same as in the reference case analysis ($409 per person per year)
Sensitivity analysis, scenario 4—to estimate the net budget impact of stepped-care iCBT versus usual care. In this scenario, unguided was followed by therapist-guided iCBT for those individuals who tended to drop out from the unguided iCBT. The estimated annual cost of step-level iCBT was about $715 per person with mild to moderate major depression and about $711 per person with an anxiety disorder
Sensitivity analysis, scenario 5—to estimate how an increase in the prevalence of anxiety disorders may change the net budget impact of therapist-guided iCBT provided by regulated health care professionals. The prevalence of anxiety disorders used in this scenario was 18% and 25%
The last four scenario analyses assume the same rate of uptake for iCBT (3% per year) as in the reference case analyses. Scenarios 2–4 assume the same prevalence estimates for major depression and anxiety disorders as in the reference case.
All analyses were conducted using Excel 2013 (Microsoft, Redmond, WA).
Results
Reference Case: Budget Impact for Mild to Moderate Major Depression
Table 32 presents calculations of the net budget impact in detail, using the data presented in Table 30 for the scenario comparing therapist-guided iCBT with usual care. This strategy was associated with the lowest ICER in our cost–utility analysis for people with mild to moderate major depression. Adopting therapist-guided iCBT at a 3% uptake rate in year 1, increasing to 15% in year 5 would lead to an increase in costs of about $10 million in year 1 to about $40 million in year 5.
Table 32:
Net Budget Impact of Adopting Therapist-Guided iCBT in Ontario for Adults With Mild to Moderate Major Depression: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $109,075,086 | $110,165,837 | $111,267,495 | $112,380,170 | $113,503,972 | $556,392,560 |
| Guided iCBT | $119,120,754 | $126,811,510 | $135,900,730 | $144,614,394 | $153,096,481 | $679,543,869 |
| NBI | $10,045,669 | $16,645,673 | $24,633,235 | $32,234,224 | $39,592,509 | $123,151,309 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Table 33 presents calculations of the net budget impact for the scenario comparing unguided iCBT with usual care in adults with mild to moderate major depression. In our cost–utility analysis, this strategy was associated with a smaller mean cost as compared with usual care but also a slight decrement in QALYs. As a result, adopting an unguided iCBT strategy at a 3% uptake rate in year 1, increasing to 15% in year 5 would lead to cost savings of about $1 million in year 1 to about $4 million in year 5.
Table 33:
Net Budget Impact of Adopting Unguided iCBT in Ontario for Adults With Mild to Moderate Major Depression: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $109,075,086 | $110,165,837 | $111,267,495 | $112,380,170 | $113,503,972 | $556,392,560 |
| Unguided iCBT | $107,998,070 | $108,381,221 | $108,626,517 | $108,924,275 | $109,259,181 | $543,189,265 |
| NBI | −$1,077,016 | −$1,784,615 | −$2,640,978 | −$3,455,895 | −$4,244,791 | −$13,203,295 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Reference Case: Budget Impact for Anxiety Disorders
Table 34 presents calculations on the net budget impact in detail, using data presented in Table 31 for the scenario comparing guided iCBT with usual care in adults with a primary diagnosis of anxiety disorder. In our cost–utility analysis, therapist-guided iCBT was the best strategy. Adopting therapist-guided iCBT at a 3% uptake rate in year 1, increasing to 15% in year 5 would lead to an increase in costs of about $16 million in year 1 to about $65 million in year 5.
Table 34:
Net Budget Impact of Adopting Therapist-Guided iCBT in Ontario for Adults With Anxiety Disorders: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $178,295,718 | $180,078,743 | $181,879,422 | $183,698,374 | $185,535,290 | $909,487,546 |
| Guided iCBT | $194,702,335 | $207,264,519 | $222,110,490 | $236,343,449 | $250,197,916 | $1,110,618,710 |
| NBI | $16,406,618 | $27,185,777 | $40,231,068 | $52,645,075 | $64,662,627 | $201,131,165 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Table 35 presents calculations of the net budget impact for the scenario comparing unguided iCBT with usual care in adults with anxiety disorders. In our cost–utility analysis, this strategy was associated with a smaller mean cost as compared with usual care, and a slight increment in QALYs. As a result, adopting unguided iCBT strategy at a 3% uptake rate in year 1, increasing to 15% in year 5 would lead to cost savings of about $2 million in year 1 to about $7 million in year 5.
Table 35:
Net Budget Impact of Adopting Unguided iCBT in Ontario for Adults With Anxiety Disorders: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $178,295,718 | $180,078,743 | $181,879,422 | $183,698,374 | $185,535,290 | $909,487,546 |
| Unguided iCBT | $176,470,464 | $177,054,297 | $177,403,673 | $177,841,554 | $178,341,504 | $887,111,492 |
| NBI | −$1,825,254 | −$3,024,446 | −$4,475,749 | −$5,856,820 | −$7,193,785 | −$22,376,054 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Sensitivity Analysis
Scenario 1: Budget Impact for Major Depression and Anxiety Disorders Assuming Larger Access to Therapist-Guided Internt CBT Over Time
In adults with mild to moderate major depression, therapist-guided iCBT at a 5% uptake rate in year 1, increasing to 25% in year 5, would lead to costs of about $17 million in year 1, increasing to about $61 million in year 5 (Table 36).
Table 36:
Net Budget Impact of Adopting Therapist-Guided iCBT in Ontario for Adults With Mild to Moderate Major Depression, Increase in Access by 5% Per Year: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $109,075,086 | $110,165,837 | $111,267,495 | $112,380,170 | $113,503,972 | $556,392,560 |
| Guided iCBT | $125,817,867 | $137,238,914 | $150,758,776 | $163,132,267 | $174,862,019 | $751,809,843 |
| NBI | $16,742,781 | $27,073,077 | $39,491,281 | $50,752,097 | $61,358,047 | $195,417,283 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
In with anxiety disorders, therapist-guided iCBT at a 5% uptake rate in year 1, increasing to 25% in year 5, would lead to costs of about $27 million in year 1, increasing to about $100 million in year 5 (Table 37).
Table 37:
Net Budget Impact of Adopting Therapist-guided iCBT in Ontario for Adults With Anxiety Disorders, Increase in Access by 5% per year: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $178,295,718 | $180,078,743 | $181,879,422 | $183,698,374 | $185,535,290 | $909,487,546 |
| Guided iCBT | $205,640,081 | $224,294,596 | $246,376,690 | $266,586,926 | $285,745,465 | $1,228,643,758 |
| NBI | $27,344,363 | $44,215,853 | $64,497,268 | $82,888,552 | $100,210,175 | 319,156,212 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Scenario 2: Budget Impact for Major Depression and Anxiety Disorders Assuming Coach-Delivered Guided iCBT
In adults with mild to moderate major depression, adopting coach-guided iCBT at a 3% uptake rate per year would lead to costs of about $9 million in year 1, increasing to about $37 million in year 5 (Table 38).
Table 38:
Net Budget Impact of Adopting Coach-Guided iCBT in Ontario for Adults With Mild to Moderate Major Depression: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $109,075,086 | $110,165,837 | $111,267,495 | $112,380,170 | $113,503,972 | $556,392,560 |
| Guided iCBT | $118,509,898 | $125,799,321 | $134,402,835 | $142,654,298 | $150,688,943 | $672,055,296 |
| NBI | $9,434,812 | $15,633,484 | $23,135,340 | $30,274,128 | $37,184,972 | $115,662,736 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
In adults with anxiety disorders, adopting coach-guided iCBT at a 3% uptake rate per year would lead to costs of about $15 million in year 1, increasing to about $61 million in year 5 (Table 39).
Table 39:
Net Budget Impact of Adopting Coach-Guided iCBT in Ontario for Adults With Anxiety Disorders: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $178,295,718 | $180,078,743 | $181,879,422 | $183,698,374 | $185,535,290 | $909,487,546 |
| Guided iCBT | $193,710,097 | $205,620,379 | $219,677,397 | $233,159,583 | $246,287,253 | $1,098,454,710 |
| NBI | $15,414,379 | $25,541,637 | $37,797,975 | $49,461,209 | $60,751,963 | $188,967,164 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Scenario 3: Budget Impact for Major Depression and Anxiety Disorders for Guided iCBT Assuming No Additional Spending on Therapist Training
In adults with mild to moderate major depression, adopting therapist-guided iCBT at a 3% uptake rate per year (whereby the course of iCBT excluded the training costs), would lead to costs of about $5 million in year 1, increasing to about $21 million in year 5 (Table 40).
Table 40:
Net Budget Impact of Adopting Guided iCBT (Assuming No Additional Costs on Therapist Training) in Ontario for Adults With Mild to Moderate Major Depression: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $109,075,086 | $110,165,837 | $111,267,495 | $112,380,170 | $113,503,972 | $556,392,560 |
| Guided iCBT | $114,486,833 | $119,133,102 | $124,537,776 | $129,745,214 | $134,833,031 | $622,735,958 |
| NBI | $5,411,747 | $8,967,266 | $13,270,281 | $17,365,044 | $21,329,059 | $66,343,398 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
In adults with anxiety disorders, adopting therapist-guided iCBT at a 3% uptake rate per year (the course of inernet CBT excludes training costs), would lead to costs of about $9 million in year 1, increasing to about $35 million in year 5 (Table 41).
Table 41:
Net Budget Impact of Adopting Guided iCBT (Assuming No Additional Costs on Therapist Training) in Ontario for Adults With Anxiety Disorders: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $178,295,718 | $180,078,743 | $181,879,422 | $183,698,374 | $185,535,290 | $909,487,546 |
| Guided iCBT | $187,145,381 | $194,742,641 | $203,579,898 | $212,094,915 | $220,414,050 | $1,017,976,885 |
| NBI | $8,849,664 | $14,663,898 | $21,700,476 | $28,396,541 | $34,878,760 | $108,489,339 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Scenario 4: Budget Impact for Stepped-Care Delivery of Unguided and Therapist-Guided iCBT for Major Depression and Anxiety Disorders
In adults with mild to moderate major depression, adopting unguided followed by therapist-guided iCBT via a stepped-care approach (at a 3% uptake rate per year) would lead to costs of about $2.5 million in year 1, increasing to about $10 million in year 5 (Table 42).
Table 42:
Net Budget Impact of Adopting Guided iCBT via a Stepped-Care Delivery in Ontario for Adults With Mild to Moderate Major Depression: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $109,075,086 | $110,165,837 | $111,267,495 | $112,380,170 | $113,503,972 | $556,392,560 |
| Unguided followed by guided iCBT | $111,520,903 | $114,218,556 | $117,264,946 | $120,228,232 | $123,143,555 | $586,376,192 |
| NBI | $2,445,818 | $4,052,720 | $5,997,450 | $7,848,062 | $9,639,583 | $29,983,633 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
In adults with anxiety disorders, adopting unguided followed by therapist-guided iCBT via a stepped-care approach would lead to costs of about $4 million in year 1, increasing to about $16 million in year 5 (Table 43).
Table 43:
Net Budget Impact of Adopting Guided iCBT via a Stepped-Care Delivery in Ontario for Adults With Anxiety Disorder: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $178,295,718 | $180,078,743 | $181,879,422 | $183,698,374 | $185,535,290 | $909,487,546 |
| Unguided followed by guided iCBT | $182,231,105 | $186,599,682 | $191,529,481 | $196,326,130 | $201,045,646 | $957,732,045 |
| NBI | $3,935,387 | $6,520,940 | $9,650,059 | $12,627,756 | $15,510,356 | $48,244,499 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Scenario 5: Budget Impact Associated With Higher Estimates of Prevalence of Anxiety Disorders
Assuming a prevalence of 18% for anxiety disorders, adopting therapist-guided iCBT at a 3% uptake rate per year would lead to costs of about $28 million in year 1, increasing to about $114 million in year 5 (Table 44). Assuming a prevalence of 25% for anxiety disorders, adopting therapist-guided iCBT would lead to an increase in costs of about $40 million in year 1 to $158 million in year 5 (a total 5-year net budget impact of $493 million, data not shown).
Table 44:
Net Budget Impact of Adopting Guided iCBT Assuming Higher Prevalence of Anxiety Disorders: Years 1–5
| Scenario | Year 1 | Year 2 | Year 3 | Year 4 | Year 5 | Total |
|---|---|---|---|---|---|---|
| Usual care | $314,639,702 | $317,786,090 | $320,964,068 | $324,173,638 | $327,415,417 | $1,604,978,915 |
| Guided iCBT | $343,592,576 | $365,761,000 | $391,960,159 | $417,076,713 | $441,526,013 | $1,959,916,460 |
| NBI | $28,952,874 | $47,974,910 | $70,996,091 | $92,903,075 | $114,110,596 | $354,937,546 |
Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; NBI, net budget impact.
Discussion
We conducted a model-based budget impact analysis to examine the range of investment needed to enable access to iCBT for adults with a primary diagnosis of mild to moderate major depression or anxiety disorders, as defined by the DSM-5 criteria.
Assuming a 3% increase in access per year (from 0% at baseline to 15% in year 5) and a target population increase from about 8,000 in year 1 to about 32,000 in year 5, we estimated the net budget impact of publicly funding guided iCBT with a therapist for the treatment of mild to moderate major depression would be about $10 million in year 1, increasing to about $40 million in year 5. The latter estimate of budget impact corresponds to treating about 12% of people with mild to moderate major depression in 2021 (year 5). The net budget impact would be about $16 million in year 1, increasing to about $65 million in year 5 for the treatment of anxiety disorders (the target population of 13,000 in year 1 would increase to 52,000 in year 5, corresponding to about 12% of people with anxiety disorders in 2021). As expected, increases in the access over time or prevalence of the disease were associated with greater annual budget impact estimates.
Public funding for unguided iCBT would be associated with cost savings in both populations. However, the budget impact analysis was model-based. It adopted the same parameter assumptions as were used for our cost-effectiveness analysis. Thus, unguided iCBT was associated with smaller total costs than usual care because of the smaller probability of medication use in the CBT strategies. Nevertheless, unguided iCBT is not a preferred option because it has relatively high dropout rates and there is a lack of evidence with respect to incremental clinical benefits compared with usual care.
Interestingly, using a stepped-care model for iCBT (i.e., guided iCBT only in individuals who are considered at risk of dropping out from the unguided program) represented the most affordable option for Ontario. Over the next 5 years, the annual net budget impact ranged from about $2 million in year 1 to about $10 million in year 5 in adults with mild to moderate depression, and from about $4 million to about $16 million over the same time period in adults with anxiety disorders. Although this modeling approach may seem to be the most affordable option for Ontario, we found limited evidence of effectiveness or cost-effectiveness with unguided iCBT. Consequently, this analysis indicates the importance of individuals' readiness for iCBT, which should be carefully assessed and considered in the implementation program.
It is important to recognize that guided iCBT was assumed to be delivered by any regulated registered health care professional providing CBT for depression or anxiety and that the projected costs of guided therapy do not include and costs for training. However, the cost of iCBT includes potential maintenance costs of the portal through which e-telehealth therapy could be delivered. Types of professionals providing guided iCBT widely range across the countries. Some countries, such as the United Kingdom, have organized mental health programs (i.e., the UK IAPT79,86,115) in which internet and in-person CBT are provided by highly trained non-regulated mental health workers called coaches, who are supervised by regulated health care professionals.
When we explored this scenario for Ontario, we found relatively small decreases in the annual net budget impact. If coaches deliver guided iCBT (instead of regulated registered psychotherapists), the savings will range between $1 million and $3 million per year (e.g., $9 million vs $10 million in year 1, increasing to $37 million vs $40 million in year 5 for major depression). We also find that guided iCBT would be more affordable if no additional costs are spent on therapist training (e.g., $5 million in year 1, increasing to $21 million in year 5 for major depression).
Our budget impact analysis considered that, regardless of prior disease history, anyone diagnosed with mild to moderate major depression or anxiety disorder at the time of entrance to the publicly available mental health program would have only one full course of iCBT. After that, depending on the disease progression, individuals could continue with other forms of treatment.
Conclusions
Over the next 5 years, the annual net budget impact of publicly funding guided iCBT in Ontario would be between $10 million and $40 million for the treatment of mild to moderate major depression, and between $16 million and $65 million for the treatment of anxiety disorders. A relatively small cost savings may be achieved if guided iCBT is provided by a coach (i.e., non-regulated mental health worker), while moderately high cost savings may be achieved if no additional spending is required for therapist training or if the delivery of guided iCBT is organized through a stepped-care iCBT model or is provided to a selected population who is ready to comply with therapy.
PATIENT PREFERENCES AND VALUES
Objective
The objective of this analysis was to explore the underlying values, needs, impacts, preferences, and perceptions of iCBT among people with lived experience of depression or an anxiety disorder.
Background
Patient, caregiver, and public engagement provides a unique source of information about people's experiences of a health condition and the health technologies or interventions used to manage or treat that health condition. This information includes the impact of the condition and its treatment on the patient, the patient's family and other caregivers, and the patient's personal environment. Engagement also provides insights into how a health condition is managed by the province's health system.
Information shared by people with lived experience can also identify gaps or limitations in published research (e.g., sometimes typical outcome measures do not reflect what is important to those with lived experience).128–133 Additionally, lived experience can provide information and perspectives on the ethical and social values implications of health technologies or interventions.
Because the needs, priorities, preferences, and values of those with lived experience in Ontario are often not adequately explored in published literature, we contact and speak directly with people who live with a given health condition, including those who may have experience with the intervention we are exploring.
Methods
Engagement Plan
The engagement plan for this health technology assessment focused on examining the experiences of people with depression or anxiety and those of their family, including their understanding and perceptions of iCBT. We engaged people via face-to-face and telephone interviews.
We used a qualitative interview, as this method of engagement allows us to explore the meaning of central themes in the experiences of people with anxiety or depression, as well as those of their family. Our main task in interviewing is to understand what people tell us and to gain an understanding of the meaning of their experiences.134 The sensitive nature of exploring people's experiences of a health condition and their quality of life are other factors that supported our choice of an interview methodology.
Participant Outreach
We used an approach called purposive sampling,135 which involves actively reaching out to patients, families, and caregivers with direct experience of the health condition and health technology or intervention being reviewed. We approached more than 30 organizations and groups involved in providing iCBT or providing care for anxiety or depression using other treatments to spread the word about this engagement opportunity.
Inclusion Criteria
We sought to speak with people who had been actively managing depression or anxiety with iCBT or another type of treatment.
Exclusion Criteria
We did not set specific exclusion criteria.
Participants
For this project, we spoke with 17 people with depression or anxiety living across Ontario, as well as 1 family member of people with depression or anxiety. Participants varied in terms of socioeconomic background, gender, and language preference.
We spoke with people who had experience with iCBT, as well as with other treatments such as medication, in-person CBT, and self-help. The people with anxiety or depression with whom we spoke fell into two groups: one who had undergone treatment with iCBT and another who were interested in trying iCBT but had not yet done so.
Fourteen participants had direct experience with iCBT, and all participants had experience with medication. All participants had lived experience of depression or anxiety and with treatment for their condition. Participants shared their experiences and perceptions either in person or over the telephone.
Gaining an understanding of the day-to-day functioning of people with depression or anxiety and their experiences with available treatments, including iCBT, helped us assess the potential value of iCBT from the perspective of people living with depression or anxiety and their family members.
Approach
At the beginning of the interview, we explained the role of Health Quality Ontario, the purpose of this health technology assessment, the risks of participation, and how participants' personal health information would be protected. We gave this information to participants both verbally and in a letter of information (Appendix 10). We obtained participants' verbal consent before starting the interview. With participants' consent, we audio-recorded and then transcribed the interviews.
Interviews lasted about 20 to 40 minutes. Interviews were semistructured and consisted of a series of open-ended questions. Questions were based on a list developed by the Health Technology Assessment International Interest Group on Patient and Citizen Involvement in Health Technology Assessment.136 Questions focused on the impact of depression or anxiety on participants' quality of life and their perceptions of the benefits and limitations of various treatment options for managing their condition. See Appendix 11 for our interview guide.
Data Extraction and Analysis
We used a modified version of a grounded-theory methodology to analyze interview transcripts.137,138 The grounded-theory approach allows us to organize and compare information across participants. This method consists of a repetitive process of obtaining, documenting, and analyzing responses while simultaneously collecting, analyzing, and comparing information. We used the qualitative data analysis software program NVivo (QSR International, Doncaster, Victoria, Australia) to identify and interpret patterns in interview data. The patterns we identified then allowed us to highlight the impact of depression or anxiety on the people we interviewed.
Results
The people with depression or anxiety and the family members with whom we spoke emphasized the constant struggle of managing the condition. They reported medications were a widely available treatment option, but although medications helped manage symptoms, they also had side effects that could be severe at times. In addition, all interviewees expressed that medications do not address the underlying cause of a person's depression or anxiety. People with depression or anxiety reported being interested in exploring treatment options in addition to or as an alternative to medication to manage their condition in the long term. Some participants reported experience with in-person CBT or self-help methods in addition to taking medication. Some participants reported one or many socioeconomic barriers to accessing treatment.
People with experience of iCBT were able comment on the similarities and differences of this treatment compared with in-person CBT. They commented that iCBT was appealing as it provided control over the location, time, and pace of therapy. Some people stated that the lack of face-to-face interaction with one's iCBT therapist was difficult. Some also noted that despite learning techniques to manage their condition, they sometimes fell back into their old patterns of thinking and reported needing peer support and follow-up after completing therapy.
Day-to-Day Impact of Depression or Anxiety
Participants noted that their depression or anxiety had an immense impact on their day-to-day life. Most interviewees discussed struggles in the daily management of their condition, as well as challenges associated with low self-esteem, maintaining personal relationships, and financial difficulties.
Struggles in the Day-to-Day Management of Depression or Anxiety
People with depression or anxiety expressed difficulty understanding their condition and with seeking help. They mentioned that, following diagnosis, they experienced a consistent struggle to cope with their condition. They often felt lethargic from the effort it took to manage their emotions on a daily basis:
The first challenge that I faced in terms of living with anxiety was, sort of like, understanding it myself.
I would get home from work, when I was still working, be exhausted, have to lie down, couldn't start to prepare meals until I'd had a rest.
I would need to sedate myself to be able to open the door and get outside, and I would have to make numerous attempts to just try to open the door. So that one thing could take me hours to be able to get to the mailbox and back, and it would take everything out of me that I had.
Participants often attributed other health problems, such as high blood pressure, blackouts, panic attacks, insomnia, and lethargy, to unmanaged depression or anxiety:
I also lost my driver's licence because…of the blackouts. I think I was falling asleep, personally, just exhaustion from trying to cope with this depression.
I believe the blood pressure was the result of the anxiety.
[I] went to the hospital, the emergency department, many, many times with chest pain, so then we got into exploring did I have a heart problem. So, one thing led to another. As I look back, as I've improved, all of those health problems, I think, were a problem of…the anxiety and the depression.
I have no deep sleep. I've had sleep studies done; I have no deep sleep; I'm waking every 2 to 3 minutes because of the hypervigilance. So, I have no sleep.
It [depression] also affects our sleep. We don't sleep, and not having enough sleep also…just makes everything worse.
Participants noted that depression or anxiety caused them to lose interest in other aspects of life such as meal preparation, managing finances, taking part in recreational activities, and travelling. They noted that things that had previously interested them were no longer appealing, with one person referring to this feeling as “drifting”:
It has kind of affected my desire to travel and enjoy some of the things that I have enjoyed in the past.
Mentally it affected my memory. I could not sit down and play the piano; I couldn't organize my finances; I couldn't balance my bank accounts or my credit card statements or whatever. At home, I would drift from room to room, incapable of actually performing a task. But then, I just…I'm generally a very good housekeeper, a good cook as well, [and] I like to bake. So, none of those hobbies interested me, and, of course, I just didn't seem to have the energy to do any of those things.
I would buy a ticket for a concert and one of my friends would come to pick me up, and I couldn't go; just, I just couldn't go.
Low Self-Esteem
Participants reported feeling isolated, stigmatized, and disconnected from other people, especially those who did not understand their condition. One person commented that their condition was a “huge hit” to their self-esteem and sense of self-worth. Participants commented that low self-esteem and a feeling of isolation also made it difficult to seek help for their condition:
And there are people in my life who don't understand mental illness, or they're dismissive of it. And that, of course, impacts my relationship with them, because this has become such a huge part of my day-to-day life, living and trying to cope with this condition.
At my lowest point, I definitely felt utterly alone. I felt like I was the only person in the world going through this, and I couldn't access all the resources I felt that I needed.
You know, I have moments where I definitely still doubt myself, and even my closest relationships with friends and family.
You feel very isolated; you feel like a total “nut bar.” So, when you meet people, doctors and nurses and other kinds of professionals, [and] there is no one [who] understands your suffering…words just don't give it justice.
My relationship with myself is impacted…because I know I could be doing so much better and living a more wholesome life and doing [many] more things. So, it affects you 360 degrees all around your life.
Managing Personal Relationships
People with depression or anxiety mentioned that their condition also impacted their personal relationships. They noted that they often found it challenging to communicate with people about their condition:
One of the bigger challenges now is…communicating that with other people. Because it is something that people are going to experience tangentially through you, you know, like it's never going to be a thing that people don't notice, or people don't see, and it's going to always be something that (they) feel.
Some interviewees noted that their condition caused them to become increasingly dependent on family and friends. They noted that this caused stress for their families, and that their condition limited their ability to take part in daily activities and travel:
Without the friends, I possibly might have committed suicide when my husband wasn't home; it had gotten quiet, quite frightening. So, it got to the point where my husband, of course, was very fearful and concerned; he became overprotective, constantly asking me how I was. [He] didn't like me to go anywhere by myself because he was afraid I might have one of these episodes…[I was] pretty much withdrawn from social activities, especially if it was about me going anywhere, and basically I rely on my husband, heavily, when he's home…to do everyday activities together such as groceries or any kind of thing, and [we] pretty much [stay] at home, for the most part. Not really going anywhere.
Financial Impact
Some participants whose depression or anxiety had begun early in their lives mentioned that their condition had impacted their ability to complete their education and find a job:
I felt like my life had been derailed because of my anxiety disorder. It took me an extra year to complete my undergraduate degree, which obviously came with added financial stress.
Because my agoraphobia started when I was 12, I never finished high school… because I couldn't leave the house. So, you can imagine a young person [who] has not finished school because they can't leave the house, what kind of an income that they would be making; it would be virtually nothing…It affected everything.
Participants whose depression or anxiety began later in life reported that the condition impacted their ability to be productive at work and stay employed. All participants noted some impact on their finances owing to their condition:
Well, we're [people with depression or anxiety] often really poor. We can't get a job…For a brief period of time, I was on disability myself, so I know what that's like, and it feels hopeless. You don't have any means or ability to earn an income; you barely make it.
My work was severely affected; I was exhausted all the time. Normally I'm a very organized, prepared person, [and] my work is up to date. But as I became more and more depressed, I became disorganized, I lost important papers, and I couldn't get my work done on time.
Experience with Medication and In-Person Cognitive Behavioural Therapy
Participants reported having tried various treatment options, including medication and in-person CBT. People reported that medication was the most accessible treatment option.
Medication
Participants mentioned that medications were helpful in managing the severity of their condition and stabilizing their mood:
Medication helps, and it reduces the impact when you are suffering from anxiety.
One participant used the metaphor of a “fire in my house” to explain the severity of her condition. She stated that she needed to “put out the fire in my house before I can stand back and analyze why I might have had a fire in my house,” referring to the importance of using medication to regulate her mood before pursuing other types of treatment.
However, some participants mentioned that medications sometimes only partially helped alleviate symptoms:
I felt like they [medications] met my needs somewhat, in terms of helping with the mood and anxiety. You know, obviously, I still wasn't at top mental and emotional shape, but I was definitely able to function better in day-to-day life in terms of attending classes and things like that.
Many participants described medications as “hit or miss” in terms of their ability to improve symptoms. They mentioned having to experiment with a variety of medications before finding one or more that they perceived as having a positive impact on their mood. Some participants felt that medication alone, without also pursuing another type of treatment, did not help:
I've tried. In the last 10 or 15 years, I've tried all kinds of different antidepressants, but nothing helps.
Medication Side Effects
Participants noted a range of medication side effects, including weight gain and an urge to cut oneself. They also noted having to take additional medications to manage their side effects:
I tried many, many drugs, some with some really bad side effects…and that's why I had to change so often.
In terms of side effects…it took a long time to find [the] right meds. I went through a whole lot. Some of these meds cause some really severe side effects [including] nausea [and] constipation, and I would have to, in turn, go and get over-the-counter medication to help with those side effects, if you know what I mean. So, it's just medication piling up, piling up on each other. So that wasn't good.
It got so bad once that I went to our hospital's emergency mental health crisis centre, and at that time I was told that I was [taking] a drug that was making things worse, I think, than better.
One family member also reported on the negative impact medication can have on a person:
I was strongly against it [medication]…She insisted on taking it. She tried it and started cutting herself in 3 weeks. I saw the marks and inquired. She told me, and I asked her to stop the meds.
Barriers to Accessing Medication
Participants reported experiencing barriers to therapy with medication related to cost and access.
The ongoing cost of medications was reported as a barrier for people without drug coverage or living on a fixed income. Most interviewees with depression or anxiety reported that the cost of medication was a substantial barrier:
I felt like, for me, again, cost was another factor in terms of pharmacotherapy, the medication. Again, I was not covered on an insurance plan and had to rely on the Trillium public drug plan, which…in my case had pretty high deductibles.
I'm living in poverty, and…my husband has mental health issues as well, so that again impacts me because of his issues. So basically, we just talk because I have a lot of stresses in my life that I can't do anything about. I can't do anything about the poverty.
Some participants, especially those from Northern Ontario, noted that it was difficult for them to receive a diagnosis and to access mental health care:
Well, it was very, very difficult here [in Northern Ontario]…to get a psychiatrist, but my doctor recognized that; he…attempted to find the right mix [of medication and/or treatment] that would work for me, but we're very short of psychiatrists.
Because neither family doctors nor [we], the general public, are aware of places where you can go for help or get treatment and how you get involved in it, you know. So, there is a disconnect somewhere between the services that are available and the public and even doctors working within the health care system; they don't know about everything that's there.
Alternatives to Medication
Many participants reported seeking out treatments as an alternative or in addition to medication. Some mentioned that medication helped alleviate symptoms during an episode of anxiety or depression; however, they felt medication was not a long-term solution for their condition. Some also noted feeling “trapped” by their medications. They expressed interest in developing ways to manage their condition without having to rely solely on their medications. Several interviewees also reported trying self-help methods such as relaxation therapy, meditation, and self-help books to supplement their medication or while they were waiting to receive therapy such as CBT:
I do have friends who have tried…meds, and I feel like it doesn't necessarily…address the root of the problem.
When you have really bad anxiety like I do, you don't want to be under the control of drugs either, so pharmaceuticals were really not much of an option for me, because I felt equally as trapped taking drugs as I did with the anxiety itself. So, I tried all kinds of things, but I didn't stick with very many things for that reason.
In-Person Cognitive Behavioural Therapy
For the purposes of this project, we were interested in comparing participants' experiences with in-person CBT versus iCBT. Participants reported finding in-person CBT helpful in improving their mood and anxiety but that it was a challenging treatment to undergo:
I have found cognitive behavioural therapy to be…challenging, because it requires a lot of…interpersonal reflection and…thinking, and it can be really…emotionally exhausting.
There's a great deal of work that has to go into helping yourself get better; nobody can do it for you. You've got to put a lot of…work in it, and [you've] got to be really determined to improve, that you don't want your life to be like this anymore.
One-on-One Versus Group Sessions
Participants who had experience with both one-on-one and group therapy sessions reported that each has its own benefits and challenges. They noted that one-on-one sessions allow for a more personalized approach to therapy, whereas group sessions provide peer support and a chance to relate with others with similar issues. However, group therapy does not provide the privacy some felt is necessary for sharing personal information. Some reported that this lack of privacy limited their ability to work on certain issues:
Because it is a group session, you are not given as much personal care. Time is allocated between people. Even though after weeks you are more at ease…you are still not as transparent [as] if you are in [a] private session.
Barriers to Accessing In-Person Cognitive Behavioural Therapy
Despite participants' interest in receiving in-person CBT, many faced barriers of cost and access to this type of therapy. Most participants found they were often unable to afford the cost of in-person CBT:
What I really wanted was somebody to…help me in a nonpharmaceutical way, but…I didn't have the financial means to be able to hire a psychologist or anybody [who] would be able to provide me with any help
I wasn't covered by insurance, and the psychologist [who] I would reach out to or [had] researched, their average starting rate was something like $200 an hour, which obviously, for a student, you can't really afford.
Some participants also mentioned the costs associated with travel to and from CBT sessions:
I definitely had to pay for parking and sometimes get a ride, ask a relative to give me a lift.
Some of those unable to access in-person CBT were advised by their physicians to visit the emergency department to receive care in addition to medication:
And the person that she [physician] spoke with…said, “You know, there are no community psychologists covered by OHIP…You'd be better off sending your patient to the emergency, saying that they are a suicide risk.” And that is the way that you will get services.
Limitations of In-Person Cognitive Behavioural Therapy
Participants reported several limitations to in-person CBT, including the time commitment required, the number of sessions included in a course of CBT, and the lack of follow-up support. They also reported that the severity of one's symptoms could make it difficult to get to therapy sessions.
Time Commitment
Participants with busy schedules found it challenging to accommodate therapy appointments and travel:
He [psychiatrist], in fact, was the one who suggested that I take a CBT course for this [anxiety], but since I work full-time, I was not able to choose any timing during my work hours, obviously.
Number of Sessions in a Course of In-Person CBT
Some participants mentioned that a single 8- to 12-week course of in-person CBT was sometimes insufficient. They reported sometimes struggling to incorporate the skills they had learned into their daily lives and that refresher sessions were needed to maintain their skills. One patient reported the perception that, in a 10-week course, a person usually gets less than 5 hours of actual therapy:
I guess another sort of drawback would be, like, it was an eight-week-long session, and I felt that it's really towards the end that you sort of start experiencing the benefits of those sessions, and then it sort of ended.
She [therapist] gets 10 sessions with one [client], so the first four or five [sessions] she can do planning work, and then she gets four or five sessions to do CBT…You may get
40 minutes with them, and to do four or five sessions of 40 minutes of CBT, that's just useless…Less than five hours of CBT with a social worker: they're [clients] not going to learn anything.
Follow-Up Support
Participants mentioned that even though they had learned coping skills to manage their symptoms through CBT, a high-stress environment or change of routine after completing therapy could cause them to revert to their previous habits of thinking. Participants felt that following a routine and practising self-care helped them manage their condition. A few people mentioned peer support following therapy might help them stay accountable and reinforce the skills learned in CBT. It was mentioned that acting as a peer support might also assist in a person's recovery through a change in mindset from “victim” to “helping others”:
Because the way you deal with anxiety and stuff [depression], those are all skills that you have to learn, and you have to practise just like any other skill. And if you get really busy, just like any other skill, it's something that kind of takes the back burner a little bit. And it takes a little bit of time to [try] to get back on the horse and like be, like, “Okay, I have to remember that I have to be doing this and managing my thinking and managing…those thoughts.”
I have a certain routine that I stick to that I know…lends to me being more calm and confident and capable. So, if I have to make, for example, medical appointments or things I have to get to, either before or after work, I find that really challenging or frustrating. Or, it definitely throws me off and makes me feel more anxious and frustrated, and that becomes difficult.
They [therapists] give tools and exercises and avenues to explore. After 8 weeks [of therapy], you feel that you start back in the loop of [a] depressive phase. You are not using the exercises in the more scheduled manner anymore.
Nature of Condition
Some interviewees reported finding it difficult to attend in-person therapy, owing to the nature of the symptoms of depression and anxiety. This theme was especially common for people with anxiety and those who were homebound owing to the severity of their condition:
…a lot of people with anxiety or agoraphobia…can't get treatment…can't even get there, we can't even leave our houses, so even if there were centres…[or] communities that offered different programs, I couldn't get there to be able to get any benefit from them.
[An organization] had a therapy group and I went to that; it was like an eight-week course…so I would sign up for things like that, but it's very hard when you have that type of anxiety to even leave the house. So…it didn't mean [I would] be able to take advantage of all eight weeks, because I may only be able to brave it two or three of those eight…times…So I missed a lot of the sessions.
Internet-Delivered CBT
Participants who had undergone iCBT described their experiences with the treatment process and what they felt were its benefits and limitations. During the treatment period, most had access to their therapist via email and text messaging. Some participants also had in-person check-ins with their therapist.
Treatment Process
Most participants who had experienced iCBT reported that it improved the management of their mood and anxiety:
It helped me a lot. It helped me to overcome a few of the things from my condition.
Internet-based cognitive behavioural therapy is…the majority of the work is being done by the individual and managing anxiety and managing anxious thoughts really helps with a whole bunch of other mental health issues, like depression…I would say…after learning how to manage my anxiety, I became…significantly less depressed because I wasn't spiralling down into these…negative-thinking loops.
Interviewees mentioned that the earlier sessions in the course were simple, having been designed to help them better understand their condition, whereas the latter sessions were more complex and required intense reflection and thinking:
So, it has information about the condition that is not just downloaded in a dump, but it's bit by bit…depending on where you are in the treatment…certain things you need to know about that. And then a space on understanding that your thoughts…affect your behaviours. And then you go to a very concrete place where you start analyzing your thoughts with regard to the different situations that you're afraid [of].
Participants noted that each learning module was built on the previous one. For each module, they were required to complete homework assignments such as thought records during the week and send them to their therapist.
Participants reported that the design, layout, and structure of the iCBT program were essential to keeping them engaged throughout the course of therapy. The usability and interactivity of the online platform were also important:
I didn't find that one [an iCBT program] particularly helpful…it was just not very…compelling or interactive; I didn't really feel like there was a lot of structure to it. It felt really…overly self-motivated…you really had to…force yourself to do it, and I found that [made it] really challenging to stay motivated. The other one [another iCBT program] basically…has a much better…design and layout. They have…steps so you can see the entire program laid out before you, so you know what you're getting into. You know how much there is to do…it's all planned out for you.
Accountability and Support
Many participants felt that motivation was key to completing an iCBT course. As many people with depression struggle with motivation, this is an important factor to consider for both practitioners and people considering pursuing iCBT. Interviewees who had experience with iCBT appreciated that the program provided access to a therapist who kept them accountable for their work at regular check-ins and provided support when needed:
The other thing that I found super helpful…[was] having…some sort of community support as well…however often you needed it…You are also able to text them [therapist/coach] whenever you have…an anxious moment. So, I found that…really helpful, because that helped me stay motivated, because I knew how much there was to do.
Having weekly check-ins with a coach was also super helpful, because that helped me…to have somebody holding me accountable for what I was doing…It was just motivating to have somebody else out there who knew what you were going through and could help you through it and was…holding you accountable for it.
Condition Severity
Several participants with anxiety noted that the ability to do iCBT from home allowed them to access treatment that would otherwise have been inaccessible owing to the severity of their condition, which in some cases resulted in their being homebound. Some participants expressed that iCBT enabled them to manage their thoughts and readied them seek further treatment, if needed:
It's kind of like you have two broken legs…you can't get to the doctor to go get casts on your legs; you need some help to be able to even get to the doctor's office. It's the same thing…a program inside your home that would help you develop some crutches that [would allow you] to at least get out the door…to get additional help that's needed…At least…develop skills to…leave and participate in something bigger.
It [iCBT]…prompted me to…explore deeper and try to…figure out where the anxiety was coming from, like a root cause of it. And for that, I decided to go back to my therapist and…explore those reasons more.
Control Over the Time and Location of Therapy
Most participants who had experienced iCBT were satisfied with the amount of control over the time and location that iCBT offered. People who were in school or working reported that not needing to travel to appointments saved them time, making it easier for them to fit therapy into their busy schedules:
The biggest plus point was the convenience and the flexibility of the timing.
The convenience of deciding when to put it in the schedule…it just so nice to have the control.
The first [benefit] is, when I was doing online therapy, what I really loved about that was the flexibility of it and the fact that I could do it from anywhere. So, I actually started it when I was…backpacking in Australia…and then I was able to continue when I got back to Canada, so the flexibility of that was great.
I was not keen on taking leave from work and going and doing this [CBT] course because that would mean that I [would have to] take that leave from work, and I just wanted to do it online…at my own pace and convenience.
I wanted to not miss work. I wanted to do it online at my own pace and convenience. There is no excuse…you can do appointments at any time, even at midnight.
Participants also found that the online iCBT platform reduced anxiety associated with scheduling appointments and travel time:
Being at home and in a place where you're comfortable [makes it] much easier to function, but as soon as you go out [in] public and you have to deal with things that you're not used to or things that change…noise levels or people or crowds, that is extremely difficult to deal with.
For me…having to go to preplanned appointments…what if something [is] wrong on that day, [and] you're not in the mood to do it, then it can bring anxiety on.
Control Over Pace and Length of Sessions
With online access to iCBT educational materials, those pursuing a course of iCBT have access to their learning materials at all times. Participants reported that this encouraged them to regularly practise their skills and thus consolidate their learning and gain confidence in their new skills:
As we learn more, as we practise, then we gain some confidence, we gain some confidence in that our symptoms become very predictable…You know, the internet is kind of nice because you can walk yourself 10 times [through] the same issue, and you can document that each time, or 10 times [through] the same crazy thinking…and it's all documented…Then you can go back over that history, and you can start building the little muscles like going to the gym…a little bit at a time, build on your own confidence by practising.
Participants reported that communicating with their therapist via email was helpful, in that it allowed them to plan what they wanted to say in advance. They also found the online interface useful, as it allowed them to save useful links, educational materials, and personal reflections on their computers for future reference:
I get very flustered when I speak…so it was good just to…be able to…communicate in a way that I was able to get everything out, get the questions out that I needed. I like that she [therapist] was able to send links…that I could save onto my computer and I could just have…if I was having a bad day and I needed the techniques [to] kind of help to keep the anxiety down.
There is a lot of thinking and analysis [in the iCBT] process, [one has to mentally] prepareto go through experiments. Therapists gives you feedback on [the] plan. There is [an] open line for questions. It is online through the system. Everything you write is kept there.
Ease of Access
Many noted that having online access to therapy was beneficial, given that when people are experiencing symptoms of depression or anxiety, they often want to be alone and also often spent time online. One participant noted that when he was experiencing symptoms of anxiety, he often browsed the internet. Then, when he had access to online therapy, he would choose to do therapy instead of randomly browse websites. Participants reported appreciating the ability to receive therapy and track their progress in the comfort and privacy of their homes with their personal devices:
The big benefit that I found was how accessible it was, so again because you can access it through your laptop, tablet, or phone. That was a big thing for me…Especially… the younger generation I feel would really benefit from that because, I mean, for me, especially when you're having anxiety or something, you tend to go and distract yourself by browsing different sites or going on YouTube. So rather than doing that, I would often use the app and just look over the material there to sort of cope with the anxiety or depression.
Even though you didn't meet with your therapist face to face, it was really easy to communicate and…get…pretty detailed feedback, and [the process was] just pretty timely and responsive. And again, you sort of had a history of what you both talked about and the different exercises and recall for yourself. Since it was an app, you could actually see a graph and see a trend of how you were doing week by week. So, I found that was pretty handy, to kind of see how you'd improved in certain areas or also how certain different exercises helped improve your mood.
I think check-in calls were…exactly what I needed…After a certain point, when I was certainly done with the [iCBT] program…I had these skills to…deal with my anxiety and my anxious thoughts, and…a way to…manage my thinking.
Barriers to iCBT
Although participants noted many benefits to iCBT, some barriers were also reported.
Cost
Although iCBT was less expensive than face-to-face CBT, the cost was considered a burden and potentially a barrier for people on limited income, students, or those who did not have insurance.
There were some things that I could not work on. I had lost my job, and I had to drop it [therapy].
It was really pricy. I [student] believe I paid…around $250 for [each session]…basically, it still is a 12-week session, so I felt that was pretty expensive, especially if you consider the younger crowd who might be at school or college or…might not have that income to afford that app.
Another important cost associated with iCBT is the cost of a computer, which some living on a low income reported being unable to afford:
I could hardly afford a flip phone, let alone a computer.
Though we think the whole world is wired, it's not yet, and not everybody has a computer.
There may be technology challenges because…all I have is a little phone to use the internet with, and my phone is so basic…[that] I can't even load any kind of an app or anything on it because there's no room. The operating system takes up the whole phone, so it makes it really difficult for me to try and do things online.
They [people] may not be able to afford one [a computer]. So, access to a computer I think might be a challenge, it might not be, but it could be, and even if they could borrow a laptop or something from someone, I think understanding how to use it or something like that would be important.
Computer and Internet Access
Although free access to internet-accessible computers is available in public spaces such as libraries, some participants expressed concerns of privacy and confidentiality with regard to pursuing therapy in a public space on a shared computer:
This isn't something I'd want to do over a public-access computer…Every time I talk to anybody about the lack of connectivity for those of us in the rural and remote areas, you know, the answer is always, “Well, there are public-access computers.” Well, I'm sorry, I'm not going to do psychotherapy over a public-access computer with people looking over my shoulder and people reading my stuff after I leave; that's not a suitable option.
Participants in Northern Ontario also commented on internet speed as a barrier to pursuing iCBT:
That's the biggest challenge with anything over the ‘net, because it's fine in the cities, but when you get up here, up north, if you do have the means to have a computer and highspeed internet, the internet isn't always available. For a lot of people, though, it's just simply not accessible because of where they live.
Computer Literacy
Another important barrier to accessing iCBT is computer literacy. For those without the skills necessary to operate a computer or navigate the internet, iCBT is not an option. Participants noted that basic computer skills training would be needed for such individuals:
Many have had a different past than I have. If they started very young before this whole internet age, they may not have the skill or ability to navigate a computer; they may not even own a computer.
[There should be] a training manual of sorts…but sort of dumb it down as to the very basics of being able to use a tool, because some people may never have even had a computer.
I know there [are] a lot of folks out there [who] don't have the computer knowledge to do any online courses. I think that would overwhelm my partner because he's not very computer savvy.
Disabilities Affecting One's Ability to Use a Computer
Disability was also mentioned as a barrier to accessing iCBT:
[For people] with disabilities, if you can't speak, you can't type…[or] you can't see, those are other challenges that come through with anything where it's online or you have to read the contents [of educational materials].
Language
Participants who did not speak English fluently commented that language could be a barrier to receiving treatment. Some predominantly French-speaking participants indicated that they would not have been able to complete their course of iCBT, offered only in English, without the additional in-person assistance offered by their therapist.
Limitations of iCBT
Lack of Face-to-Face Interaction
Some participants indicated that, based on their personality, they would have preferred face-to-face interaction with their therapist. Some mentioned that when they had been feeling particularly low, they wished they could have seen their therapist in person. One person mentioned that an in-person session would have allowed for an “emotional release.” Another expressed interest in using video-call applications such as Skype or FaceTime to interact with their therapist:
I definitely felt like there could have been more person-to-person interaction, if not one-on-one, [then] in a group therapy setting.
At first when I got the response back from my social worker, [it] made me…realize, “Oh, I kind of understand the way of my thinking, how it's affecting me psychologically.” But then after a while [of] just receiving emails and never really even [hearing] the voice of the person [therapist], it just didn't really have anything…It didn't really help.
Definitely the online…discussion forum helped, but…especially [at] my lower point…when I felt really isolated and alone, I think…being able to…maybe even…Skyping…the person running the course or something like that…I think would have been beneficial for me during those low points, or FaceTime or whatever software you want to use.
Having that human connection [would be helpful], just being able to talk face to face with a professional, ask them questions, open [up] to them about what I'm experiencing. So, for me, I'm a very social person. I like to talk to people. I like being able to connect on that level. So just being able to talk to them, being able to open up, to cry if need be, just [to] have that emotional release [would be helpful].
Given the format of iCBT, in which clients and therapists interact via email, participants sometimes had to wait for responses to questions about learning modules. Some noted that this wait could cause anxiety. As each iCBT learning module builds on the previous one, not getting clarification in an appropriate amount of time can impede a person's progress:
Emailing and waiting to receive a response to a question would decrease effectiveness of the therapy. If you had misunderstood and perceived something incorrectly [during a module] and moved on [to the next module] as if you had perceived something incorrectly…it would decrease effectiveness of your learning.
Every week she [therapist] would take a look at the homework and offer some suggestions. If I got stuck, usually it was the interpretation of something, [and] she would help me with that…People who are in trouble cannot always work [their] way
through [the content]…sometimes simply because we do not perceive the things the same way.
Program Rigidity
Some participants reported that the iCBT program appears to be prescriptive and rigid. Unlike in-person CBT, which allows clients the opportunity to explore issues as they surface, iCBT follows a more rigid course that does not allow for issues to be explored as they arise. One person reported feeling they had to “stick to the regimen of the computer program.”
A few interviewees compared iCBT with a self-help book, in that neither allows for flexibility in terms of therapeutic content (what topics are covered) or personal preference (e.g., for face-to-face interaction with a therapist). One person mentioned that waiting for a text or email response to a question did not meet their expectation of receiving immediate responses:
Doing it online would be just the same as picking up the book and reading it. And, you know, I guess that's my thing with it…I think the contact part is important for somebody like me. Maybe not everybody, but [for] somebody like me, it's important.
The whole program made me feel like I could just buy a self-help book and just do it that way. So, for me, being totally internet based and never being in contact [with the therapist], you know, face to face or anything like that, I think that was the main reason it did not help.
I stopped around halfway [through the program]…because…the person [therapist] is actually really not helping me. So that's why I'm actually doing it [therapy] in person at the moment.
Condition Severity
Most participants indicated that because undergoing CBT is an emotionally challenging experience, internet-based therapy may not be the best option for people with severe depression. Participants mentioned that when they are extremely depressed, they often lack the motivation to undergo iCBT, as this therapy requires more motivation than others:
And I feel like, definitely, my case was a bit more…I guess a bit of a complex case…So, I definitely feel…I guess [if] the province were to move forward with an OHIP-covered CBT program, there should be an [option for] individual or group therapy…[and a] face-to-face component for more complex cases. It shouldn't be a one-size-fits-all solution.
Participants also felt that people with certain anxieties related to privacy implications from the government or internet communication may not be able to benefit from iCBT:
Given the online platform and people recording their thoughts online, this can cause anxiety for people who are not comfortable sharing their inner thinking over the internet.
Some people are not confident with the government. Folks, too, might not be from Canada [and may] feel like they can't trust the government to open up and be honest about things that they don't want to be discriminated against. That could be a barrier for some people, too.
As with in-person CBT, interviewees noted that the length of a course of iCBT treatment is important, stating that those whose condition is more severe may benefit from a longer course of treatment:
Other people get [understand] it the first time, but it may be their first episode of anxiety or depression for…but for somebody who has a serious illness, whether it's bipolar or a serious anxiety from a phobia or…PTSD [post-traumatic stress disorder]…Like in the early ‘90s and through the ‘90s…if somebody needed longer-term care, they got that, and they weren't just kicked out after 10 or 12 sessions.
Comprehension of Information
A few participants noted frustration at their inability to comprehend long emails and sessions as therapy became more complex. They explained that their condition impacted their cognitive functioning and that in-person sessions would have allowed them to interrupt their therapist and ask questions when they needed clarification:
Just getting…emails explaining the way of thinking, it didn't really…do anything… compared to…[a] personal one [in-person therapy]. It [in-person therapy] helps a lot more…because…I'm there. I can even ask if I don't understand something, what she's explaining; then I can…interrupt and ask what that really means, because having this condition, my comprehension is really, I would say, degraded, and I think having human contact actually helps me more to really understand the concept of the whole therapy.
It was user friendly, but the amount of information was getting more and more in-depth, which [it] is supposed to be, I assume, and again it's based on how much I can comprehend…It was just too much as well for me. It was…a lot more complex and halfway done already, and I just felt lost, basically, in all that information.
Follow-Up Support
As mentioned, some participants reported that, following therapy, increased stress and changes in routine could have a negative impact on their thought processes that could result in succumbing to their old ways of thinking. As with in-person CBT, participants mentioned that people could likely benefit from some type of follow-up, coaching, or peer support group following the completion of iCBT. Further, they felt that people could benefit from this process as both the person receiving the follow-up support and as a person providing such support:
For me personally, I found that I didn't have a whole lot of…support outside. And…I guess since we're talking about…online cognitive therapy, I think that stuff is really good…I think a key to it is follow-up. It's almost like you learn these things, and then you need…some coaching or mentoring on how to put the theory into action in some cases.
I also think that there is a need for…having…a physical presence and physical engagement, some sort of…supportive community around it as well.
You know the best way to keep it going is to teach it to others. So, if you have an online [therapy program] that you're creating, I would create some kind of community [to go along with it]. People like this have had nobody, and if they start learning to have confidence, they're going to learn from each other, and…in teaching other people, they reinforce their own learning, and they no longer become the victim; they become, you know, like their life has some meaning.
Technical Difficulties
Participants noted that their experience of therapy could be disrupted owing to technical difficulties with the online iCBT program:
There [were] a few weeks when there was some bug or something where I had problems accessing the app, so even though I was sort of able to go and log in through their site and was able to do the weekly stuff, there [were] some technical issues [that] were annoying.
Discussion
People with depression or anxiety shared their experiences of the burden their condition has on their daily life, self-esteem, relationships, work, and finances, as well as their experiences with a number of treatment options.
Interviewees indicated that medication and in-person therapy are currently available treatment options, with most reporting being able to access medication. However, many found that medications often had undesirable side effects, and many felt that medication was not a long-term solution for their condition. People with experience of in-person CBT indicated that it had therapeutic benefits, but that barriers such as cost, and access exist. Several limitations to inperson CBT were reported, including the time for therapy and the time for travel to and from appointments, the number of sessions in a course of treatment, symptom severity sometimes making it difficult or impossible to attend therapy, and the lack of follow-up support.
Most interviewees with experience of iCBT reported that it had a positive impact on their condition. Many reported perceiving that their iCBT therapist had provided them with an appropriate degree of accountability and support. Reported benefits of iCBT versus in-person CBT include increased flexibility in and control over the time, pace, and location of therapy. People who were homebound owing to the severity of their condition reported that iCBT provided them an option for therapy that otherwise would not be available to them, and that it helped them develop the skills needed to leave home for further treatment if needed. The online platform of iCBT allowed easier access and reference to learning materials on personal devices. Reported barriers to accessing iCBT included the cost of therapy and the need for a computer, internet access, computer literacy, and the ability to comprehend complex written information. Language was also reported as a barrier for the people we interviewed as they reported that iCBT was offered only in English. Reported limitations of iCBT included the lack of in-person interaction, the rigidity of the program, symptom severity affecting people's motivation to pursue therapy, the number of sessions in a course of treatment, the lack of follow-up support, and the potential for technical difficulties with the online platform.
Conclusions
People with depression or anxiety with whom we spoke viewed both in-person CBT and iCBT as effective treatment options for adults with depression or anxiety. Internet-delivered CBT may be especially beneficial for those with whose symptoms may prevent them from leaving home. Internet-delivered CBT may also be beneficial for those with busy schedules who would find it difficult to arrange travel time to and from appointments, and for those for whom the cost of travel to and from appointments would be a barrier. However, iCBT is associated with important barriers and limitations, including the need for a computer, internet access, computer literacy, and the ability to comprehend complex written information.
For both in-person CBT and iCBT, the cost of treatment, the number of sessions in a course of treatment, and the lack of follow-up support were reported as substantial drawbacks.
In-person CBT may be preferred by people who feel they would benefit from in-person interaction and the opportunity to deal with issues as they arise, whereas iCBT may be preferred by those wanting to pursue therapy from the comfort of home and who are comfortable adhering to a more structured course of therapy.
For people with mild to moderate depression or anxiety, iCBT provided control over the time, pace, and location of therapy. Despite some perceived limitations, iCBT was felt to improve access for those people who could not otherwise access treatment due to costs, time, or nature of their health condition.
CONCLUSIONS OF THE HEALTH TECHNOLOGY ASSESSMENT
Compared to waiting list, guided iCBT significantly improves symptoms of mild to moderate major depression (GRADE moderate), generalized anxiety disorder (GRADE low), panic disorder (GRADE low), and soical phobia (GRADE low). Compared to group or individual face-to-face CBT, guided iCBT did not significantly improve symptoms of panic disorder (GRADE very low).
In various cost–utility analyses, we found that over a short-term time horizon guided iCBT represents good value for money compared with other available options such as unguided iCBT, face-to-face CBT, and usual care. Guided iCBT could be offered as an initial step for the short-term treatment of eligible adults with mild to moderate major depression or anxiety disorders. Assuming a 3% increase in access per year over the next 5 years, the annual net budget impact of publicly funding guided iCBT in Ontario would be between $10 million and $40 million for the treatment of mild to moderate major depression, and between $16 million and $65 million for the treatment of anxiety disorders.
People with depression or anxiety with whom we spoke viewed iCBT as an effective treatment option. Internet-delivered CBT may be preferred over in-person CBT by those wanting to pursue therapy from home, those with busy schedules, those who are comfortable adhering to a more structured course of therapy, and those with severe anxiety preventing them from leaving home. Despite some perceived limitations, iCBT was felt to improve access to treatment for people who may not otherwise be able to access treatment owing to cost, time, or the severity of their health condition.
Acknowledgments
Bernice Tsoi and Cody Black, health economists at the Canadian Agency for Drugs and Technologies in Health (CADTH), reviewed and provided input on the economic analysis. David Kaunelis, methods specialist, and Kaitryn Campbell, research information specialist, provided peer review for the literature search strategy.
The medical editors were Tim Maguire, Kara Cowan, and Elizabeth Jean Betsch. Others involved in the development and production of this report were Merissa Mohamed, Kellee Kaulback, Saleemeh Abdolzahraei, Claude Soulodre, Andrée Mitchell, Sarah McDowell, Vivian Ng, Nancy Sikich, and Irfan Dhalla.
We are grateful to the following experts for sharing their expertise as we prepared this health technology assessment: Peter Cornish (Memorial University), Z. J. Daskalakis (Centre for Addiction and Mental Health), Peter Farvolden (CBT Associates), David Gratzer (Centre for Addiction and Mental Health), Jennifer Hensel (University of Manitoba), Muhammad Mamdani (St. Michael's Hospital), Farooq Naeem (Centre for Addiction and Mental Health), Sanjay Rao (Royal Ottawa Mental Health Centre), and David Tannenbaum (Sinai Health System).
ABBREVIATIONS
- CADTH
Canadian Agency for Drugs and Technologies in Health
- CI
confidence interval
- DSM
Diagnostic and Statistical Manual of Mental Disorders
- GAD
generalized anxiety disorder
- GP
general practitioner
- GRADE
Grading of Recommendations Assessment, Development, and Evaluation
- IAPT
Improving Access for Psychological Therapies
- iCBT
internet-delivered cognitive behavioural therapy
- ICER
incremental cost-effectiveness ratio
- MDE
major depressive episode
- MSW
Master's of Social Work
- OCD
obsessive-compulsive disorder
- OR
odds ratio
- PTSD
post-traumatic stress disorder
- QALY
quality-adjusted life-year
- RR
relative risk
- SD
standard deviation
- SMD
standardized mean difference
GLOSSARY
- Budget impact analysis
A technique to estimate the financial impact of a planned action over a specified time period by calculating the costs and savings of different options.
- Cost-effective
Good value for money. The overall benefit of the technique or intervention justifies the cost.
- Cost–utility analysis
A type of analysis that estimates the value for money of an examined intervention versus control treatment (e.g., usual care). The result is expressed as a dollar amount per “quality-adjusted life-year” or QALY.
- Discounting
A method that considers that costs and health benefits are worth more today than in the future.
- Dominant
An intervention is dominant when it is both more effective and less costly than its alternative comparator(s).
- Economically attractive
See Cost-effective.
- Incremental cost-effectiveness ratio (ICER)
Determines “a unit of benefit” for an intervention by dividing the incremental cost by the incremental effectiveness. The incremental cost is the difference between the mean costs of the treatment under study and an alternative treatment. The incremental effectiveness is the additional unit of health effect gained (e.g., life-years or QALYs).
- Markov model
A type of modelling that measures the health state of a patient over the course of treatment. A patient may stay in one health state or move from one health state to another, depending on the effect of the treatment and the progression of the disease.
- Ministry of Health and Long-Term Care perspective
The Ministry of Health and Long-Term care perspective includes treatment costs (e.g., drug, administration, monitoring, and health service resource use, such as hospital stays and general practitioner visits) and costs associated with managing adverse events caused by treatment. This perspective does not include patients' out-of-pocket costs relating to obtaining care (e.g., transportation) or loss of productivity costs (e.g., absenteeism, presenteeism)
- Net budget impact
In a budget impact analysis, the net budget impact is the difference in cost between a future scenario (new treatment) as compared with the current scenario (usual care).
- Quality-adjusted life-year (QALY)
A measurement that considers both the number of years gained by a patient from a procedure and the quality of those extra years (considering such factors as ability to function and freedom from pain). The QALY is commonly used as an outcome measure in cost–utility analyses.
- Reference case
A set of recommended methods used for all evaluations that promote uniformity and transparency and that enable the comparison of results for different technologies and different decisions.
- Societal perspective
The societal perspective considers the full effect on society of a condition, including all costs, regardless of who pays, and all benefits, regardless of who receives the benefits.
- Time horizon
Costs and outcomes are examined within a chosen time frame. In an economic evaluation, this time frame is referred to as the time horizon.
APPENDICES
Appendix 1: Literature Search Strategies
Clinical Evidence Search
Search date: February 15, 2018
Databases searched: All Ovid MEDLINE, Embase, Cochrane Database of Systematic Reviews, CRD Health Technology Assessment Database, NHS Economic Evaluation Database, PsycINFO, CINAHL
Ovid Database: EBM Reviews – Cochrane Database of Systematic Reviews <2005 to February 14, 2018>, EBM Reviews – Health Technology Assessment <4th Quarter 2016>, EBM Reviews – NHS Economic Evaluation Database <1st Quarter 2016>, Embase <1980 to 2018 Week 07>, All Ovid MEDLINE(R) <1946 to Present>, PsycINFO <1806 to February Week 1 2018>
Search Strategy
-
1
Depression/(431591)
-
2
Depressive Disorder/(128784)
-
3
Depressive Disorder, Major/(30842)
-
4
Depressive Disorder, Treatment-Resistant/(1071)
-
5
(depress* or MDD).ti,ab,kf. (1169437)
-
6
Anxiety/(288598)
-
7
Anxiety Disorders/(55241)
-
8
Agoraphobia/(10906)
-
9
Anxiety, Separation/(4124)
-
10
Panic Disorder/(26098)
-
11
exp Phobic Disorders/(37106)
-
12
Mutism/(3684)
-
13
(anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,kf. (636431)
-
14
or/1–13 (1733300)
-
15
Cognitive Therapy/(76265)
-
16
(((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition therap* or CBT*).ti,ab,kf. (142208)
-
17
15 or 16 (171612)
-
18
Internet/(185887)
-
19
Therapy, Computer-Assisted/(10344)
-
20
Computer-Assisted Instruction/(90169)
-
21
Mobile Applications/(7010)
-
22
Telemedicine/(38968)
-
23
Remote Consultation/(11963)
-
24
(internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart phone* or mobile based or e mail* or email* or electronic mail* or “Information and communication technology” or “Information and communication technologies” or emedicine or e medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele psychol*).ti,ab,kf. (279138)
-
25
((technolog* or computer* or digital* or webbased or web based or webdeliver* or web deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or deliver* or technique* or training)).ti,ab,kf. (162924)
-
26
or/18–25 (635529)
-
27
14 and 17 and 26 (5032)
-
28
(eCBT* or cCBT* or iCBT*).ti,ab,kf. (1908)
-
29
14 and 28 (1194)
-
30
(MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,kf. (314)
-
31
27 or 29 or 30 (5318)
-
32
Meta Analysis.pt. (84720)
-
33
Meta-Analysis/or Meta-Analysis as Topic/or exp Technology Assessment, Biomedical/(286547)
-
34
(((systematic* or methodologic*) adj3 (review* or overview*)) or pooled analysis or published studies or published literature or hand search* or handsearch* or medline or pubmed or embase or cochrane or cinahl or data synthes* or data extraction* or HTA or HTAs or (technolog* adj (assessment* or overview* or appraisal*))).ti,ab. (645525)
-
35
(meta analy* or metaanaly* or health technolog* assess*).mp. (437423)
-
36
or/32–35 (881233)
-
37
31 and 36 (680)
-
38
Case Reports/or Congresses.pt. (1930375)
-
39
37 not 38 (680)
-
40
limit 39 to english language [Limit not valid in CDSR; records were retained] (655)
-
41
limit 40 to yr=“2000 -Current” (647)
-
42
41 use ppez,cleed (204)
-
43
limit 31 to english language [Limit not valid in CDSR; records were retained] (5120)
-
44
limit 43 to yr=“2000 -Current” (5011)
-
45
44 use coch,clhta (15)
-
46
depression/(431591)
-
47
major depression/(162267)
-
48
(depress* or MDD).tw,kw. (1186504)
-
49
anxiety/(288598)
-
50
anxiety disorder/(104805)
-
51
generalized anxiety disorder/(11358)
-
52
panic/(25444)
-
53
phobia/(26970)
-
54
agoraphobia/(10906)
-
55
claustrophobia/(11329)
-
56
social phobia/(14558)
-
57
separation anxiety/(7268)
-
58
anxiety neurosis/(25355)
-
59
“mixed anxiety and depression”/(538)
-
60
selective mutism/(1622)
-
61
(anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).tw,kw. (650368)
-
62
or/46–61 (1750576)
-
63
cognitive therapy/(76265)
-
64
cognitive behavioral therapy/(25567)
-
65
(((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition therap* or CBT*).tw,kw,dv. (150690)
-
66
or/63–65 (180094)
-
67
Internet/(185887)
-
68
computer assisted therapy/(11486)
-
69
computer program/(279888)
-
70
e-mail/(18195)
-
71
mobile application/(7881)
-
72
computer/(121808)
-
73
personal computer/(15611)
-
74
software/(120953)
-
75
telecommunication/(28846)
-
76
teleconference/(5799)
-
77
teleconsultation/(12436)
-
78
telemedicine/(38968)
-
79
telepsychiatry/(453)
-
80
teletherapy/(7848)
-
81
(internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart phone* or mobile based or e mail* or email* or electronic mail* or “Information and communication technology“ or ”Information and communication technologies“ or emedicine or e medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele psychol*).tw,kw,dv. (289515)
-
82
((technolog* or computer* or digital* or webbased or web based or webdeliver* or web deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or deliver* or technique* or training)).tw,kw,dv. (167557)
-
83
or/67–82 (977112)
-
84
62 and 66 and 83 (5851)
-
85
(eCBT* or cCBT* or iCBT*).tw,kw,dv. (1960)
-
86
62 and 85 (1239)
-
87
(MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or Ecompass or Deprexis or Moodkit or Living Life to the Full).tw,kw. (326)
-
88
84 or 86 or 87 (6143)
-
89
Meta Analysis/or “Meta Analysis (Topic)”/or Biomedical Technology Assessment/(280947)
-
90
(((systematic* or methodologic*) adj3 (review* or overview*)) or pooled analysis or published studies or published literature or hand search* or handsearch* or medline or pubmed or embase or cochrane or cinahl or data synthes* or data extraction* or HTA or HTAs or (technolog* adj (assessment* or overview* or appraisal*))).ti,ab. (645525)
-
91
(meta analy* or metaanaly* or health technolog* assess*).mp. (437423)
-
92
or/89–91 (880146)
-
93
88 and 92 (1113)
-
94
Case Report/or conference abstract.pt. (6872119)
-
95
93 not 94 (1079)
-
96
limit 95 to english language [Limit not valid in CDSR; records were retained] (1051)
-
97
limit 96 to yr=“2000 -Current” (1011)
-
98
97 use emez (302)
-
99
“depression (emotion)”/(123407)
-
100
major depression/(162267)
-
101
recurrent depression/(749)
-
102
treatment resistant depression/(4640)
-
103
(depress* or MDD).ti,ab,id. (1170012)
-
104
anxiety/(288598)
-
105
anxiety disorders/(55241)
-
106
generalized anxiety disorder/(11358)
-
107
panic disorder/(26098)
-
108
panic attack/(28367)
-
109
social anxiety/(75650)
-
110
exp phobias/(22840)
-
111
separation anxiety disorder/(5612)
-
112
Elective Mutism/(1485)
-
113
(anxiet* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,id. (617046)
-
114
or/99–113 (1636314)
-
115
cognitive behavior therapy/(42521)
-
116
cognitive therapy/(76265)
-
117
(((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition therap* or CBT*).ti,ab,id. (145381)
-
118
or/115–117 (176194)
-
119
Internet/(185887)
-
120
Computer Assisted Therapy/(11486)
-
121
Computer Mediated Communication/(5218)
-
122
computer assisted instruction/(90169)
-
123
computer software/(105562)
-
124
computer applications/(11089)
-
125
mobile devices/(1843)
-
126
electronic communication/(3031)
-
127
human computer interaction/(15088)
-
128
Information Technology/(16892)
-
129
electronic learning/(660)
-
130
online therapy/(2362)
-
131
Telemedicine/(38968)
-
132
(internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart phone* or mobile based or e mail* or email* or electronic mail* or “Information and communication technology” or “Information and communication technologies” or emedicine or e medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele psychol*).ti,ab,id. (278495)
-
133
((technolog* or computer* or digital* or webbased or web based or webdeliver* or web deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or deliver* or technique* or training)).ti,ab,id. (163192)
-
134
or/119–133 (747179)
-
135
114 and 118 and 134 (4972)
-
136
(eCBT* or cCBT* or iCBT*).ti,ab,id. (1899)
-
137
114 and 136 (1176)
-
138
(MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,id. (315)
-
139
135 or 137 or 138 (5256)
-
140
(((systematic* or methodologic*) adj3 (review* or overview*)) or pooled analysis or published studies or published literature or hand search* or handsearch* or medline or pubmed or embase or cochrane or cinahl or data synthes* or data extraction* or HTA or HTAs or (technolog* adj (assessment* or overview* or appraisal*))).ti,ab. (645525)
-
141
(meta analy* or metaanaly* or health technolog* assess*).mp. (437423)
-
142
(systematic review or meta analysis).md. (33994)
-
143
or/140–142 (866376)
-
144
139 and 143 (673)
-
145
case report/(4089061)
-
146
144 not 145 (671)
-
147
limit 146 to english language [Limit not valid in CDSR; records were retained] (644)
-
148
limit 147 to yr=“2000 -Current” (636)
-
149
148 use psyh (126)
-
150
42 or 45 or 98 or 149 (647)
-
151
150 use ppez (204)
-
152
150 use coch (7)
-
153
150 use clhta (8)
-
154
150 use cleed (0)
-
155
150 use emez (302)
-
156
150 use psyh (126)
CINAHL
| # | Query | Results |
|---|---|---|
| S1 | (MH “Depression”) | 81,242 |
| S2 | depress* OR MDD | 135,715 |
| S3 | (MH “Anxiety”) | 30,406 |
| S4 | (MH “Anxiety Disorders”) | 8,352 |
| S5 | (MH “Generalized Anxiety Disorder”) | 289 |
| S6 | (MH “Panic Disorder”) | 1,841 |
| S7 | (MH “Phobic Disorders+”) | 4,368 |
| S8 | (MH “Separation Anxiety”) | 546 |
| S9 | (MH “Mutism”) | 223 |
| S10 | anxiet* OR anxious* OR panic* OR phobi* OR agoraphobi* OR GAD OR mute OR mutism | 75,834 |
| S11 | S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 | 178,201 |
| S12 | (MH “Cognitive Therapy”) | 15,055 |
| S13 | ((cognitive OR behavio*) N2 (therap* OR psychotherap*)) OR cognitive behavio* OR cognition therap* OR CBT* | 39,189 |
| S14 | S12 OR S13 | 39,189 |
| S15 | (MH “Internet”) | 38,850 |
| S16 | (MH “World Wide Web”) | 63,831 |
| S17 | (MH “World Wide Web Applications”) | 4,682 |
| S18 | (MH “Therapy, Computer Assisted”) | 4,731 |
| S19 | (MH “Mobile Applications”) | 2,897 |
| S20 | (MH “Telehealth”) | 5,018 |
| S21 | (MH “Telemedicine”) | 7,577 |
| S22 | (MH “Telepsychiatry”) | 231 |
| S23 | (MH “Remote Consultation”) | 1,481 |
| S24 | internet* OR Beacon OR app OR apps OR (mobile N2 application*) OR smartphone* OR smart phone* OR mobile based OR e mail* OR email* OR electronic mail* OR “Information and communication technology” OR “Information and communication technologies” OR emedicine OR e medicine OR ehealth* OR e health* OR emental health* OR e mental health* OR etherap* OR e therap* OR epsychiatr* OR e psychiatr* OR epsychol* OR e psychol* | 72,532 |
| S25 | telemedicine OR tele medicine OR telehealth* OR tele health* OR telemental health* OR tele mental health* OR telecare OR tele care OR teletherap* OR tele therap* OR telepsychiatr* OR tele psychiatr* OR telepsychol* OR tele psychol* | 14,496 |
| S26 | ((technolog* OR computer* OR digital* OR webbased OR web based OR webdeliver* OR web deliver* OR online) N6 (therap* OR psychotherap* OR CBT OR intervention* OR treatment* OR deliver* OR technique* OR training)) | 24,487 |
| S27 | S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR S24 OR S25 OR S26 | 165,602 |
| S28 | S11 AND S14 AND S27 | 997 |
| S29 | eCBT* or iCBT* or cCBT* | 205 |
| S30 | S11 AND S29 | 148 |
| S31 | MoodGym OR Big White Wall OR Beating the Blues OR Fear Fighter OR E compass OR Ecompass OR Deprexis OR Moodkit OR Living Life to the Full | 144 |
| S32 | S28 OR S30 OR S31 | 1,123 |
| S33 | (MH “Meta Analysis”) | 31,096 |
| S34 | (PT “Meta Analysis”) or (PT “Systematic Review”) | 74,036 |
| S35 | ((systematic* or methodologic*) N3 (review* or overview*)) or pooled analysis or published studies or published literature or hand search* or handsearch* or medline or pubmed or embase or cochrane or cinahl or data synthes* or data extraction* or HTA or HTAs or (technolog* N1 (assessment* or overview* or appraisal*)) | 148,301 |
| S36 | S33 OR S34 OR S35 | 159,578 |
| S37 | S32 AND S36 | 124 |
| S38 | (MH “Case Studies”) OR (MH “Congresses and Conferences”) | 89,938 |
| S39 | S37 NOT S38 | 124 |
| S40 | S37 NOT S38 Limiters – Published Date: 20000101-; English Language | 123 |
Economic Evidence Search
Search date: February 21, 2018
Databases searched: All Ovid MEDLINE, Embase, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, CRD Health Technology Assessment Database, NHS Economic Evaluation Database, PsycINFO, CINAHL
Ovid Database: EBM Reviews – Cochrane Central Register of Controlled Trials <January 2018>, EBM Reviews – Cochrane Database of Systematic Reviews <2005 to February 14, 2018>, EBM Reviews – Health Technology Assessment <4th Quarter 2016>, EBM Reviews – NHS Economic Evaluation Database <1st Quarter 2016>, Embase <1980 to 2018 Week 08>, All Ovid MEDLINE(R) <1946 to Present>, PsycINFO <1806 to February Week 2 2018>
Search Strategy
-
1
Depression/(439538)
-
2
Depressive Disorder/(133466)
-
3
Depressive Disorder, Major/(33905)
-
4
Depressive Disorder, Treatment-Resistant/(1225)
-
5
(depress* or MDD).ti,ab,kf. (1218104)
-
6
Anxiety/(295392)
-
7
Anxiety Disorders/(57570)
-
8
Agoraphobia/(11298)
-
9
Anxiety, Separation/(4190)
-
10
Panic Disorder/(26885)
-
11
exp Phobic Disorders/(38122)
-
12
Mutism/(3694)
-
13
(anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,kf. (664501)
-
14
or/1–13 (1800358)
-
15
Cognitive Therapy/(82916)
-
16
(((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition therap* or CBT*).ti,ab,kf. (155741)
-
17
15 or 16 (186922)
-
18
Internet/(189187)
-
19
Therapy, Computer-Assisted/(11416)
-
20
Computer-Assisted Instruction/(91414)
-
21
Mobile Applications/(7298)
-
22
Telemedicine/(40471)
-
23
Remote Consultation/(12309)
-
24
(internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart phone* or mobile based or e mail* or email* or electronic mail* or “Information and communication technology“ or ”Information and communication technologies“ or emedicine or e medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele psychol*).ti,ab,kf. (288011)
-
25
((technolog* or computer* or digital* or webbased or web based or webdeliver* or web deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or deliver* or technique* or training)).ti,ab,kf. (172863)
-
26
or/18–25 (655134)
-
27
14 and 17 and 26 (6066)
-
28
(eCBT* or iCBT* or cCBT*).ti,ab,kf. (2272)
-
29
14 and 28 (1459)
-
30
(MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,kf. (393)
-
31
27 or 29 or 30 (6424)
-
32
economics/(276061)
-
33
economics, medical/or economics, pharmaceutical/or exp economics, hospital/or economics, nursing/or economics, dental/(795242)
-
34
economics.fs. (400763)
-
35
(econom* or price or prices or pricing or priced or discount* or expenditure* or budget* or pharmacoeconomic* or pharmaco-economic*).ti,ab,kf. (933453)
-
36
exp “costs and cost analysis”/(572001)
-
37
(cost or costs or costing or costly).ti. (252770)
-
38
cost effective*.ti,ab,kf. (294433)
-
39
(cost* adj2 (util* or efficacy* or benefit* or minimi* or analy* or saving* or estimate* or allocation or control or sharing or instrument* or technolog*)).ab,kf. (199167)
-
40
models, economic/(11111)
-
41
markov chains/or monte carlo method/(72649)
-
42
(decision adj1 (tree* or analy* or model*)).ti,ab,kf. (39787)
-
43
(markov or markow or monte carlo).ti,ab,kf. (120475)
-
44
quality-adjusted life years/(34469)
-
45
(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).ti,ab,kf. (64871)
-
46
((adjusted adj2 (quality or life)) or (willing* adj2 pay) or sensitivity analys*s).ti,ab,kf. (104322)
-
47
or/32–46 (2532592)
-
48
31 and 47 (1027)
-
49
Case Reports/or Comment.pt. or Editorial.pt. or Letter.pt. or Congresses.pt. (4894150)
-
50
48 not 49 (1016)
-
51
limit 50 to english language [Limit not valid in CDSR; records were retained] (993)
-
52
limit 51 to yr=“2000 -Current” (977)
-
53
52 use ppez,coch,cctr,clhta (416)
-
54
limit 31 to english language [Limit not valid in CDSR; records were retained] (6153)
-
55
limit 54 to yr=“2000 -Current” (6037)
-
56
55 use cleed (17)
-
57
53 or 56 (433)
-
58
depression/(439538)
-
59
major depression/(162568)
-
60
(depress* or MDD).tw,kw. (1236728)
-
61
anxiety/(295392)
-
62
anxiety disorder/(107266)
-
63
generalized anxiety disorder/(11394)
-
64
panic/(25750)
-
65
phobia/(27948)
-
66
agoraphobia/(11298)
-
67
claustrophobia/(12286)
-
68
social phobia/(15545)
-
69
separation anxiety/(7339)
-
70
anxiety neurosis/(25407)
-
71
“mixed anxiety and depression”/(541)
-
72
selective mutism/(1629)
-
73
(anxiet* or anxious* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).tw,kw. (679930)
-
74
or/58–73 (1819572)
-
75
cognitive therapy/(82916)
-
76
cognitive behavioral therapy/(25701)
-
77
(((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition therap* or CBT*).tw,kw,dv. (165843)
-
78
or/75–77 (196948)
-
79
Internet/(189187)
-
80
computer assisted therapy/(12560)
-
81
computer program/(280859)
-
82
e-mail/(18512)
-
83
mobile application/(7961)
-
84
computer/(121857)
-
85
personal computer/(15849)
-
86
software/(122285)
-
87
telecommunication/(28938)
-
88
teleconference/(5880)
-
89
teleconsultation/(12782)
-
90
telemedicine/(40471)
-
91
telepsychiatry/(456)
-
92
teletherapy/(7959)
-
93
(internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart phone* or mobile based or e mail* or email* or electronic mail* or “Information and communication technology” or “Information and communication technologies” or emedicine or e medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele psychol*).tw,kw,dv. (298897)
-
94
((technolog* or computer* or digital* or webbased or web based or webdeliver* or web deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or deliver* or technique* or training)).tw,kw,dv. (178265)
-
95
or/79–94 (998502)
-
96
74 and 78 and 95 (6982)
-
97
(eCBT* or iCBT* or cCBT*).tw,kw,dv. (2326)
-
98
74 and 97 (1512)
-
99
(MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or Ecompass or Deprexis or Moodkit or Living Life to the Full).tw,kw,dv. (409)
-
100
96 or 98 or 99 (7346)
-
101
Economics/(276061)
-
102
Health Economics/or Pharmacoeconomics/or Drug Cost/or Drug Formulary/(130901)
-
103
Economic Aspect/or exp Economic Evaluation/(424121)
-
104
(econom* or price or prices or pricing or priced or discount* or expenditure* or budget* or pharmacoeconomic* or pharmaco-economic*).tw,kw. (962778)
-
105
exp “Cost”/(548008)
-
106
(cost or costs or costing or costly).ti. (252770)
-
107
cost effective*.tw,kw. (305828)
-
108
(cost* adj2 (util* or efficac* or benefit* or minimi* or analy* or saving* or estimate* or allocation or control or sharing or instrument* or technolog*)).ab,kw. (206718)
-
109
Monte Carlo Method/(57252)
-
110
(decision adj1 (tree* or analy* or model*)).tw,kw. (43842)
-
111
(markov or markow or monte carlo).tw,kw. (125756)
-
112
Quality-Adjusted Life Years/(34469)
-
113
(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).tw,kw. (68696)
-
114
((adjusted adj2 (quality or life)) or (willing* adj2 pay) or sensitivity analys*s).tw,kw. (123838)
-
115
or/101–114 (2176900)
-
116
100 and 115 (1497)
-
117
Case Report/or Comment/or Editorial/or Letter/or conference abstract.pt. (9640928)
-
118
116 not 117 (1417)
-
119
limit 118 to english language [Limit not valid in CDSR; records were retained] (1393)
-
120
limit 119 to yr=“2000 -Current” (1344)
-
121
120 use emez (352)
-
122
“depression (emotion)”/(130566)
-
123
major depression/(162568)
-
124
recurrent depression/(750)
-
125
treatment resistant depression/(4805)
-
126
(depress* or MDD).ti,ab,id. (1218676)
-
127
anxiety/(295392)
-
128
anxiety disorders/(57570)
-
129
generalized anxiety disorder/(11394)
-
130
panic disorder/(26885)
-
131
panic attack/(29198)
-
132
social anxiety/(75770)
-
133
exp phobias/(23810)
-
134
separation anxiety disorder/(5684)
-
135
Elective Mutism/(1492)
-
136
(anxiet* or panic* or phobi* or agoraphobi* or GAD or mute or mutism).ti,ab,id. (644312)
-
137
or/122–136 (1702449)
-
138
cognitive behavior therapy/(42695)
-
139
cognitive therapy/(82916)
-
140
(((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition therap* or CBT*).ti,ab,id. (158916)
-
141
or/138–140 (191538)
-
142
Internet/(189187)
-
143
Computer Assisted Therapy/(12560)
-
144
Computer Mediated Communication/(5262)
-
145
computer assisted instruction/(91414)
-
146
computer software/(106540)
-
147
computer applications/(11097)
-
148
mobile devices/(1868)
-
149
electronic communication/(3045)
-
150
human computer interaction/(15139)
-
151
Information Technology/(16974)
-
152
electronic learning/(662)
-
153
online therapy/(2372)
-
154
Telemedicine/(40471)
-
155
(internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart phone* or mobile based or e mail* or email* or electronic mail* or “Information and communication technology” or “Information and communication technologies” or emedicine or e medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele psychol*).ti,ab,id. (287368)
-
156
((technolog* or computer* or digital* or webbased or web based or webdeliver* or web deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or deliver* or technique* or training)).ti,ab,id. (173133)
-
157
or/142–156 (767401)
-
158
137 and 141 and 157 (5986)
-
159
(eCBT* or iCBT* or cCBT*).ti,ab,id. (2264)
-
160
137 and 159 (1441)
-
161
(MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,id. (394)
-
162
158 or 160 or 161 (6342)
-
163
economics/or economy/(373151)
-
164
pharmacoeconomics/or health care economics/(176143)
-
165
(econom* or price or prices or pricing or priced or discount* or expenditure* or budget* or pharmacoeconomic* or pharmaco-economic*).tw. (938408)
-
166
exp “costs and cost analysis”/(572001)
-
167
cost*.ti. (273267)
-
168
cost effective*.tw. (301920)
-
169
(cost* adj2 (util* or efficacy* or benefit* or minimi* or analy* or saving* or estimate* or allocation or control or sharing or instrument* or technolog*)).ab,id. (198711)
-
170
markov chains/(18227)
-
171
(decision adj1 (tree* or analy* or model*)).tw. (42927)
-
172
(markov or markow or monte carlo).tw. (123251)
-
173
(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).tw. (68076)
-
174
((adjusted adj2 (quality or life)) or (willing* adj2 pay) or sensitivity analys*s).tw. (122344)
-
175
or/163–174 (2091190)
-
176
162 and 175 (955)
-
177
(editorial or comment reply or letter).dt. (182549)
-
178
176 not 177 (950)
-
179
limit 178 to english language [Limit not valid in CDSR; records were retained] (927)
-
180
limit 179 to yr=“2000 -Current” (912)
-
181
180 use psyh (154)
-
182
57 or 121 or 181 (939)
-
183
182 use ppez (241)
-
184
182 use coch (0)
-
185
182 use cctr (173)
-
186
182 use clhta (2)
-
187
182 use cleed (17)
-
188
182 use emez (352)
-
189
182 use psyh (154)
-
190
remove duplicates from 182 (525)
CINAHL
| # | Query | Results |
|---|---|---|
| S1 | (MH “Depression”) | 81,441 |
| S2 | depress* OR MDD | 135,892 |
| S3 | (MH “Anxiety”) | 30,504 |
| S4 | (MH “Anxiety Disorders”) | 8,372 |
| S5 | (MH “Generalized Anxiety Disorder”) | 290 |
| S6 | (MH “Panic Disorder”) | 1,848 |
| S7 | (MH “Phobic Disorders+”) | 4,374 |
| S8 | (MH “Separation Anxiety”) | 547 |
| S9 | (MH “Mutism”) | 223 |
| S10 | anxiet* OR anxious* OR panic* OR phobi* OR agoraphobi* OR GAD OR mute OR mutism | 75,954 |
| S11 | S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 | 178,448 |
| S12 | (MH “Cognitive Therapy”) | 15,083 |
| S13 | ((cognitive OR behavio*) N2 (therap* OR psychotherap*)) OR cognitive behavio* OR cognition therap* OR CBT* | 39,254 |
| S14 | S12 OR S13 | 39,254 |
| S15 | (MH “Internet”) | 38,889 |
| S16 | (MH “World Wide Web”) | 63,859 |
| S17 | (MH “World Wide Web Applications”) | 4,685 |
| S18 | (MH “Therapy, Computer Assisted”) | 4,739 |
| S19 | (MH “Mobile Applications”) | 2,936 |
| S20 | (MH “Telehealth”) | 5,036 |
| S21 | (MH “Telemedicine”) | 7,602 |
| S22 | (MH “Telepsychiatry”) | 232 |
| S23 | (MH “Remote Consultation”) | 1,485 |
| S24 | internet* OR Beacon OR app OR apps OR (mobile N2 application*) OR smartphone* OR smart phone* OR mobile based OR e mail* OR email* OR electronic mail* OR “Information and communication technology” OR “Information and communication technologies” OR emedicine OR e medicine OR ehealth* OR e health* OR emental health* OR e mental health* OR etherap* OR e therap* OR epsychiatr* OR e psychiatr* OR epsychol* OR e psychol* | 72,643 |
| S25 | telemedicine OR tele medicine OR telehealth* OR tele health* OR telemental health* OR tele mental health* OR telecare OR tele care OR teletherap* OR tele therap* OR telepsychiatr* OR tele psychiatr* OR telepsychol* OR tele psychol* | 14,535 |
| S26 | ((technolog* OR computer* OR digital* OR webbased OR web based OR webdeliver* OR web deliver* OR online) N6 (therap* OR psychotherap* OR CBT OR intervention* OR treatment* OR deliver* OR technique* OR training)) | 24,533 |
| S27 | S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR S24 OR S25 OR S26 | 165,807 |
| S28 | S11 AND S14 AND S27 | 997 |
| S29 | eCBT* or iCBT* or cCBT* | 205 |
| S30 | S11 AND S29 | 148 |
| S31 | MoodGym OR Big White Wall OR Beating the Blues OR Fear Fighter OR E compass OR Ecompass OR Deprexis OR Moodkit OR Living Life to the Full | 144 |
| S32 | S28 OR S30 OR S31 | 1,123 |
| S33 | (MH “Economics”) | 11,576 |
| S34 | (MH “Economic Aspects of Illness”) | 7,284 |
| S35 | (MH “Economic Value of Life”) | 533 |
| S36 | MH “Economics, Dental” | 110 |
| S37 | MH “Economics, Pharmaceutical” | 1,833 |
| S38 | MW “ec” | 148,396 |
| S39 | (econom* or price or prices or pricing or priced or discount* or expenditure* or budget* or pharmacoeconomic* or pharmaco-economic*) | 230,982 |
| S40 | (MH “Costs and Cost Analysis+”) | 90,943 |
| S41 | TI cost* | 42,599 |
| S42 | (cost effective*) | 32,110 |
| S43 | AB (cost* N2 (util* or efficacy* or benefit* or minimi* or analy* or saving* or estimate* or allocation or control or sharing or instrument* or technolog*)) | 22,836 |
| S44 | (decision N1 (tree* or analy* or model*)) | 5,873 |
| S45 | (markov or markow or monte carlo) | 4,053 |
| S46 | (MH “Quality-Adjusted Life Years”) | 3,210 |
| S47 | (QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs) | 7,837 |
| S48 | ((adjusted N1 (quality or life)) or (willing* N2 pay) or sensitivity analys?s) | 14,000 |
| S49 | S33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR S42 OR S43 OR S44 OR S45 OR S46 OR S47 OR S48 | 310,969 |
| S50 | S32 AND S49 | 144 |
| S51 | PT Case Study or Commentary or Editorial or Letter or Proceedings | 441,336 |
| S52 | S50 NOT S51 | 141 |
| S53 | S50 NOT S51 Limiters – Published Date: 20000101-; English Language | 139 |
Search for Intervention-Related Health State Utilities
Search date: February 28, 2018
Database: All Ovid MEDLINE(R) <1946 to Present>
Search Strategy
-
1
Cognitive Therapy/(21328)
-
2
(((cognitive or behavio*) adj2 (therap* or psychotherap*)) or cognitive behavio* or cognition therap* or CBT*).ti,ab,kf. (36053)
-
3
1 or 2 (43955)
-
4
Internet/(63558)
-
5
Therapy, Computer-Assisted/(6202)
-
6
Computer-Assisted Instruction/(11053)
-
7
Mobile Applications/(2672)
-
8
Telemedicine/(16845)
-
9
Remote Consultation/(4334)
-
10
(internet* or Beacon or app or apps or (mobile adj2 application*) or smartphone* or smart phone* or mobile based or e mail* or email* or electronic mail* or “Information and communication technology” or “Information and communication technologies” or emedicine or e medicine or ehealth* or e health* or emental health* or e mental health* or etherap* or e therap* or epsychiatr* or e psychiatr* or epsychol* or e psychol* or telemedicine or tele medicine or telehealth* or tele health* or telemental health* or tele mental health* or telecare or tele care or teletherap* or tele therap* or telepsychiatr* or tele psychiatr* or telepsychol* or tele psychol*).ti,ab,kf. (98140)
-
11
((technolog* or computer* or digital* or webbased or web based or webdeliver* or web deliver* or online) adj6 (therap* or psychotherap* or CBT or intervention* or treatment* or deliver* or technique* or training)).ti,ab,kf. (62759)
-
12
or/4–11 (215025)
-
13
3 and 12 (3242)
-
14
(eCBT* or cCBT* or iCBT*).ti,ab,kf. (634)
-
15
(MoodGym or Big White Wall or Beating the Blues or Fear Fighter or E compass or Ecompass or Deprexis or Moodkit or Living Life to the Full).ti,ab,kf. (111)
-
16
13 or 14 or 15 (3521)
-
17
Quality-Adjusted Life Years/(9861)
-
18
(quality adjusted or adjusted life year*).tw. (12827)
-
19
(qaly* or qald* or qale* or qtime*).tw. (8263)
-
20
(illness state$1 or health state$1).tw. (5340)
-
21
(hui or hui1 or hui2 or hui3).tw. (1231)
-
22
(multiattribute* or multi attribute*).tw. (729)
-
23
(utility adj3 (score$1 or valu* or health* or cost* or measure* or disease* or mean or gain or gains or index*)).tw. (11528)
-
24
utilities.tw. (5792)
-
25
(eq-5d or eq5d or eq-5 or eq5 or euro qual or euroqual or euro qual5d or euroqual5d or euro qol or euroqol or euro qol5d or euroqol5d or euro quol or euroquol or euro quol5d or euroquol5d or eur qol or eurqol or eur qol5d or eurqol5d or euro?qul or eur?qul5d or euro* quality of life or European qol).tw. (8210)
-
26
(euro* adj3 (5 d or 5d or 5 dimension* or 5dimension* or 5 domain* or 5domain*)).tw. (2817)
-
27
(sf36* or sf 36* or sf thirtysix or sf thirty six).tw. (18845)
-
28
(time trade off$1 or time tradeoff$1 or tto or timetradeoff$1).tw. (1610)
-
29
((qol or hrqol or quality of life).ti. or *quality of life/) and ((qol or hrqol* or quality of life) adj2 (increas* or decreas* or improve* or declin* or reduc* or high* or low* or effect or effects of worse or score or scores or change$1 or impact$1 or impacted or deteriorate$)).ab. (25226)
-
30
Cost-Benefit Analysis/and (cost effectiveness ratio* and (perspective* or life expectanc*)).tw. (2682)
-
31
*quality of life/and (quality of life or qol).ti. (45114)
-
32
quality of life/and ((quality of life or qol) adj3 (improve* or chang*)).tw. (19685)
-
33
quality of life/and ((quality of life or qol) adj (score$1 or measure$1)).tw. (9722)
-
34
quality of life/and health-related quality of life.tw. (24914)
-
35
quality of life/and ec.fs. (8752)
-
36
quality of life/and (health adj3 status).tw. (7471)
-
37
(quality of life or qol).tw. and cost-benefit analysis/(10102)
-
38
models, economic/(8557)
-
39
or/17–38 (130798)
-
40
16 and 39 (158)
-
41
limit 40 to english language (157)
Grey Literature Search
Search dates: February 9–14, 2018
Websites searched: HTA Database Canadian Repository, Alberta Health Technologies Decision Process reviews, Canadian Agency for Drugs and Technologies in Health (CADTH), Institut national d'excellence en santé et en services sociaux (INESSS), Institute of Health Economics (IHE), McGill University Health Centre Health Technology Assessment Unit, National Institute for Health and Care Excellence (NICE), Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Centers, Australian Government Medical Services Advisory Committee, Centers for Medicare & Medicaid Services Technology Assessments, Institute for Clinical and Economic Review, Ireland Health Information and Quality Authority Health Technology Assessments, Washington State Health Care Authority Health Technology Reviews, PROSPERO International prospective register of systematic reviews, Tufts Cost-Effectiveness Analysis Registry
Keywords used: Internet, computer*, online, digital*, webbased, web based, app, apps, smartphone*, smart phone*, CBT, behavioral therapy, behavioural therapy, behavior therapy, behaviour therapy, cognitive therapy, psychotherapy, psychotherapies
Results: 13 (17 PROSPERO systematic review protocols not counted in PRISMA)
Appendix 2: Critical Appraisal of Clinical Evidence
Table A1:
Risk of Biasa Among Systematic Reviews (ROBIS Tool)
| Author, Year | Phase 2 | Phase 3 | |||
|---|---|---|---|---|---|
| Study Eligibility Criteria | Identification and Selection of Studies | Data Collection and Study Appraisal | Synthesis and Findings | Risk of Bias in the Review | |
| Arnberg et al, 201420 | Highb | Low | Low | Low | Low |
| Adelman et al, 20147 | Low | Low | Low | Low | Low |
| Andrews et al, 201821 | Low | Low | Low | Low | Low |
| Richards et al, 201524 | Low | Low | Low | Low | Low |
| Kampmann et al, 201625 | Low | Low | Highc | Low | High |
| Dedert et al, 201323 | Higha | Low | Low | Low | Low |
| Kaltenthaler et al, 200822 | Low | Highd | Highe | Low | Low |
Abbreviation: ROBIS, Risk of Bias in Systematic Reviews.
Possible risk of bias levels: low, high, unclear.
Potential bias due to language restrictions.
Potential bias as additional methods to database searching were not used. Unclear how passive and active control is defined.
Potential bias due to single reviewer for data extraction.
Unlcear if double reviewer for data extraction.
Appendix 3: Selected Excluded Studies
| Citation | Primary Reason for Exclusion |
|---|---|
| Andersson et al, 201482 | No subgroup analysis of clinical diagnosis studies |
| Carlbring et al, 2018139 | No subgroup analysis of clinical diagnosis studies |
| Olthuis et al, 2016140 | No details on mild to moderate major depression |
| Baumeister et al, 2014141 | Unclear if iCBT programs were included in subgroup analysis |
| Karyotaki et al, 201791 | No details on mild to moderate major depression |
Appendix 4: Results of Applicability and Limitation Checklists for Studies Included in Economic Literature Review
Table A2:
Assessment of the Applicability of Studies Assessing the Cost-Effectiveness of iCBT for Major Depression
| Author, Year, Country | Is the study population similar to the question? | Are the interventions similar to the question? | Is the health care system in which the study was conducted sufficiently similar to the Ontario context? | Were the perspectives clearly stated? What were they? | Are estimates of relative treatment effect from the best available source? |
|---|---|---|---|---|---|
| Duarte et al, 2017,37 Littlewood, et al, 2015,36 United Kingdom | Yes | Yes, iCBT plus usual care (drug allowed) vs usual care, low-guided | No | Yes, NHS | Unclear |
| Romero-Sanchiz et al, 2017,43 Spain | Yes | Yes, unguided/guided vs TAU | No | Yes, Spain, societal | Unclear |
| Lee et al, 2017,49 Australia | Yes | Partially | No | Yes, health care payer | Unclear |
| Brabyn et al, 2016,38 United Kingdom | Yes | Partially | No | Yes, NHS and PPS | Unclear |
| Dixon et al, 2016,39 United Kingdom | Yes | Partially | No | Yes, NHS and PPS | Unclear |
| Solomon et al, 2015,48 Australia | Yes | Partially | No | No, health care payer | Unclear |
| Titov et al, 2015,44 Australia | Yes | Partially | No | No, health care payer | Unclear |
| Gerhards et al, 2015,45 Netherlands | Yes | Partially | No | Yes, employer and societal | Unclear |
| Phillips et al, 2014,41 United Kingdom | Partially | Partially | No | Yes, societal | Unclear |
| Gerhards et al, 2010,46 Netherlands | Yes | Partially | No | Yes, societal | Unclear |
| Warmerdam et al, 2010,47 Netherlands | Yes | Partially | No | Yes, societal | Unclear |
| Hollinghurst et al, 2010,40 United Kingdom | Yes | Partially | No | Yes, societal | Unclear |
| Kaltenthaler et al, 2006,50 United Kingdom | Yes | Partially | No | Yes, NHS | Unclear |
| McCrone et al, 2004,42 United Kingdom | Partially | Partially | No | Yes, NHS | Unclear |
| Author, Year, Country | Are all future costs and outcomes discounted? (If yes, at what rate?) | Is the value of health effects expressed in terms of quality-adjusted life-years? | Are costs and outcomes from other sectors fully and appropriately measured and valued? | Overall judgement (directly applicable/partially applicable/not applicable) |
|---|---|---|---|---|
| Duarte et al, 2017,37 Littlewood, et al, 2015,36 United Kingdom | Yes (3.5%) | Yes | Yes, microcosting approach accounting for cost of CBT, medications, health and social care services | Partially applicable |
| Romero-Sanchiz et al, 2017,43 Spain | No, 12 mo, NA | Yes | Yes, microcosting approach accounting for both direct and indirect costs | Partially applicable |
| Lee et al, 2017,49 Australia | No, 12 mo, NA | Yes | Yes | Partially applicable |
| Brabyn et al, 2016,38 United Kingdom | No, 12 mo, NA | Yes | Yes | Partially applicable |
| Dixon et al, 2016,39 United Kingdom | No, 12 mo, NA | Yes | Yes | Partially applicable |
| Solomon et al, 2015,48 Australia | No, 6 mo, NA | Yes | Partially | Partially applicable |
| Titov et al, 2015,44 Australia | No, 8 wk, NA | Yes | Partially | Partially applicable |
| Gerhards et al, 2015,45 Netherlands | No, 12 mo, NA | Yes | Partially | Partially applicable |
| Phillips et al, 2014,41 United Kingdom | No, 12 mo, NA | Yes | Yes | Partially applicable |
| Gerhards et al, 2010,46 Netherlands | No, 12 mo, NA | Yes | Yes | Partially applicable |
| Warmerdam et al, 2010,47 Netherlands | No, 12 wk, NA | Yes | Unclear | Partially applicable |
| Hollinghurst et al, 2010,40 United Kingdom | No, 8 mo, NA | Yes | Unclear | Partially applicable |
| Kaltenthaler et al, 2006,50 United Kingdom | No, 18 mo, NA | Yes | Yes | Partially applicable |
| McCrone et al, 2004,42 United Kingdom | No, 8 mo, NA | No | Yes | Partially applicable |
Abbreviations: CBT, cognitive-behavioural therapy; iCBT, internet-delivered CBT; NA, not applicable; NHS, National Health Service; PPS, personal social services; TAU, treatment as usual.
Table A3:
Assessment of the Limitations of Studies Assessing the Cost-Effectiveness of iCBT for Major Depression
| Author, Year, Country | Does the model structure adequately reflect the nature of the health condition under evaluation? | Is the time horizon sufficiently long to reflect all important differences in costs and outcomes? | Are all important and relevant health outcomes included? | Are the estimates of relative treatment effects obtained from best available sources? | Do the estimates of relative treatment effect match the estimates contained in the clinical report? | Are all important and relevant (direct) costs included in the analysis? | Are the estimates of resource use obtained from best available sources? |
|---|---|---|---|---|---|---|---|
| Duarte et al, 2017,37 Littlewood, et al, 2015,36 United Kingdom | NA | Partially | Yes | Unclear | NA | Yes | Unclear |
| Romero-Sanchiz et al, 2017,43 Spain | NA | No | Yes | Unclear | NA | Yes | Unclear |
| Lee et al, 2017,49 Australia | No | No | Partly | Unclear | No | Yes | Unclear |
| Brabyn et al, 2016,38 United Kingdom | NA | No | Partly | Unclear | No | Yes | Unclear |
| Dixon et al, 2016,39 United Kingdom | NA | No | Partly | Unclear | No | Yes | Unclear |
| Solomon et al, 2015,48 Australia | Partially | No | No | Unclear | NA | Yes | Unclear |
| Titov et al, 2015,44 Australia | NA | No | No | Unclear | NA | Yes | Unclear |
| Gerhards et al, 2015,45 Netherlands | NA | No | Yes | Unclear | Yes | Unclear | Unclear |
| Phillips et al, 2014,41 United Kingdom | NA | No | No | Unclear | NA | Yes | Unclear |
| Gerhards et al, 2010,46 Netherlands | NA | No | No | Unclear | NA | Yes | Unclear |
| Warmerdam et al, 2010,47 Netherlands | NA | No | No | Unclear | NA | Yes | Unclear |
| Hollinghurst et al, 2010,40 United Kingdom | NA | No | No | Unclear | NA | Yes | Unclear |
| Kaltenthaler et al, 2006,50 United Kingdom | Partially | No | No | Unclear | NA | Yes | Unclear |
| McCrone et al, 2004,42 United Kingdom | NA | No | No | Unclear | No | Yes | Unclear |
| Author, Year, Country | Are the unit costs of resources obtained from best available sources? | Is an appropriate incremental analysis presented, or can it be calculated from the reported data? | Are all important and uncertain parameters subjected to appropriate sensitivity analysis? | Is there a potential conflict of interest? | Overall assessment including applicability to the project (minor limitations/potentially serious limitations/very serious limitations) |
|---|---|---|---|---|---|
| Duarte et al, 2017,37 Littlewood, et al, 2015,36 United Kingdom | Yes | Yes, multiple imputation and adjustment from baseline | Yes | Unclear | Minor limitations, interventions may be contaminated, decreasing the effect of cCBT, pragmatic trial |
| Romero-Sanchiz et al, 2017,43 Spain | Yes | Yes, ITT | Yes | Unclear | Minor limitations, pragmatic trial |
| Lee et al, 2017,49 Australia | Unclear | No | Unclear | Unclear | Potentially serious limitations |
| Brabyn et al, 2016,38 United Kingdom | Yes | Yes, multiple imputation | Yes | Unclear | Minor limitations, RCT, not long-term |
| Dixon et al, 2016,39 United Kingdom | Yes | Yes | Yes | Unclear | Potentially serious limitations due to large amounts of missing data, analyses are comprehensive and accounting for it, but results are uncertain |
| Solomon et al, 2015,48 Australia | Yes | Yes | Yes | Unclear | Potentially serious limitations, no clear model structure, paper not well written, short time horizon, number of limitations |
| Titov et al, 2015,44 Australia | Yes | No | Yes | Unclear | Potentially serious limitations, small study |
| Gerhards et al, 2015,45 Netherlands | Yes | No | Unclear | Unclear | Minor limitation to HE analysis, but selected population |
| Phillips et al, 2014,41 United Kingdom | Yes | No | Yes | Unclear | Major limitations |
| Gerhards et al, 2010,46 Netherlands | Unclear | Partially | Yes | Unclear | Potentially serious limitations, small trial, not sure how they allocated, short-term, no INB |
| Warmerdam et al, 2010,47 Netherlands | Yes | No | Yes | Unclear | Potentially serious limitations |
| Hollinghurst et al, 2010,40 United Kingdom | Yes | No | Yes | Unclear | Minor limitations, RCT, not long-term |
| Kaltenthaler et al, 2006,50 United Kingdom | Yes | Partially | Yes | Unclear | Potentially serious limitations |
| McCrone et al, 2004,42 United Kingdom | Yes | No | Yes | Unclear | Potentially serious limitations |
Abbreviations: CBT, cognitive-behavioural therapy; iCBT, internet-delivered CBT; HE, health economics; INB, incremental net benefit; ITT, intention-to-treat analysis; NA, not applicable; RCT, randomized controlled trial.
Table A4:
Assessment of the Applicability of Studies Assessing the Cost-Effectiveness of iCBT for Anxiety Disorders
| Author, Year, Country | Is the study population similar to the question? | Are the interventions similar to the question? | Is the health care system in which the study was conducted sufficiently similar to the current Ontario context? | Were the perspectives clearly stated? What were they? | Are estimates of relative treatment effect from the best available source? |
|---|---|---|---|---|---|
| Kumar et al, 2018,63 United States | Yes, GAD | Yes | No | Yes, health care payer and societal | Partially |
| El Alaoui et al, 2017,61 Sweden | Yes, SAD | Yes | No | Yes, health care payer | NA |
| Hedman et al, 2016,60 Sweden | Yes, SHA | Yes | No | Yes, societal | Partially |
| Dear et al, 2015,62 Australia | Partially, older age anxiety | Yes | No | Yes, health sector | Unclear |
| Nordgren et al, 2014,56 Sweden | Partially, anxiety including comorbid | Yes | No | Yes, societal | Unclear |
| Hedman et al, 2014,59 Sweden | Yes, SAD | Yes | No | Yes, societal | Unclear |
| Hedman et al, 2013,57 Sweden | Yes, SHA | Yes | No | Yes, societal | Unclear |
| National Collaborating Centre for Mental Health and NICE, 2013,64 United Kingdom | Yes, SAD | Yes | No | Yes, NHS and PPS | Partially |
| Joesch et al, 2012,55 United States | Partially, collaborative care, sick at baseline | Partially, iCBT embedded in collaborative care program | No | Yes, health care payer | Unclear |
| Hedman et al, 2011,58 Sweden | Yes, SAD | Yes | No | Yes, societal | Unclear |
| National Collaborating Centre for Mental Health and NICE, 2011,9 United Kingdom | Yes, panic | Yes | No | Yes, NHS & PPS | Unclear |
| Bergstrom et al, 2010,54 Sweden | Yes, panic | Yes | No | Yes, societal | Unclear |
| Titov et al, 2009,53 Australia | Yes, SAD | Partially | No | Yes, health sector | No |
| McCrone et al, 2009,52 Sweden | Yes, panic | Partially | No | Unclear | No |
| Mihalopoulos et al, 2005,51 Australia | Yes, panic | Yes | No | Yes, health sector | Unclear |
| Kaltenthaler et al, 2006,50 United Kingdom | Yes, panic | Yes | No | Yes, NHS | Unclear |
| Author, Year, Country | Are all future costs and outcomes discounted? (If yes, at what rate?) | Is the value of health effects expressed in terms of quality-adjusted life-years? | Are costs and outcomes from other sectors fully and appropriately measured and valued? | Overall judgement (directly applicable/partially applicable/not applicable) |
|---|---|---|---|---|
| Kumar et al, 2018,63 United States | Yes, 3% | Yes | Yes | Partially applicable |
| El Alaoui et al, 2017,61 Sweden | Yes, 0% (3% to 5% in sensitivity analysis) | NA | Yes | Partially applicable |
| Hedman et al, 2016,60 Sweden | No, 12 wks | Yes (data not presented) | Yes | Partially applicable |
| Dear et al, 2015,62 Australia | No, 2 mo | Yes | Yes | Partially applicable |
| Nordgren et al, 2014,56 Sweden | No, 10 wks | Yes | Yes | Partially applicable |
| Hedman et al, 2014,59 Sweden | Unclear | Yes | Yes | Partially applicable |
| Hedman et al, 2013,57 Sweden | No | Yes | Yes | Partially applicable |
| National Collaborating Centre for Mental Health and NICE, 2013,64 United Kingdom | Unclear | Yes | Yes | Partially applicable |
| Joesch et al, 2012,55 United States | No | Yes | Yes | Partially applicable |
| Hedman et al, 2011,58 Sweden | Unclear | Yes | Yes | Partially applicable |
| National Collaborating Centre for Mental Health and NICE, 2011,9 United Kingdom | No, 1 yr | No | Yes | Partially applicable |
| Bergstrom et al, 2010,54 Sweden | No | No | Yes | Partially applicable |
| Titov et al, 2009,53 Australia | No | No | Partially | Partially applicable |
| McCrone et al, 2009,52 Sweden | No | No | Unclear | Partially applicable |
| Mihalopoulos et al, 2005,51 Australia | No | No | Unclear | Partially applicable |
| Kaltenthaler et al, 2006,50 United Kingdom | No, 1 yr | Yes | Unclear | Partially applicable |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; GAD, generalized anxiety disorder; NA, not applicable; NHS, National Health Service; NICE, National Institute for Health and Care Excellence; PPS, personal social services; SAD, social anxiety disorder; SHA, social health anxiety.
Table A5:
Assessment of the Limitations of Studies Assessing the Cost-Effectiveness of iCBT for Anxiety Disorders
| Author, Year, Country | Does the model structure adequately reflect the nature of the health condition under evaluation? | Is the time horizon sufficiently long to reflect all important differences in costs and outcomes? | Are all important and relevant health outcomes included? | Are the estimates of relative treatment effects obtained from best available sources? | Do the estimates of relative treatment effect match the estimates contained in the clinical report? | Are all important and relevant (direct) costs included in the analysis? | Are the estimates of resource use obtained from best available sources? |
|---|---|---|---|---|---|---|---|
| Kumar et al, 2018,63 United States | Partially | Yes, lifetime (assumption made on 5-yr efficacy of iCBT) | Partially, no recurrence | Partially | No | Yes | Unclear |
| El Alaoui et al, 2017,61 Sweden | NA | No | NA | NA | No | Yes | Unclear |
| Hedman et al, 2016,60 Sweden | NA | No | Unclear | Partially | No | Yes | Unclear |
| Dear et al, 2015,62 Australia | NA | No | Partially, no relapse | Unclear | No | Yes | Yes |
| Nordgren et al, 2014,56 Sweden | NA | No | No | Unclear | No | Yes | Unclear |
| Hedman et al, 2014,59 Sweden | NA | Partially | Yes | Unclear | No | Yes | Unclear |
| Hedman et al, 2013,57 Sweden | NA | No | No | Unclear | No | Unclear | Unclear |
| National Collaborating Centre for Mental Health and NICE, 2013,64 United Kingdom | Partially | Yes (assumption made on long-term efficacy of iCBT) | Yes | Partially | Unclear | Yes | Yes |
| Joesch et al, 2012,55 United States | NA | No | Yes | Unclear | No | Yes | Unclear |
| Hedman et al, 2011,58 Sweden | NA | Partially | Yes | Unclear | No | Yes | Unclear |
| National Collaborating Centre for Mental Health and NICE, 2011,9 United Kingdom | Partially | No | No | Unclear | No | Yes | Partially |
| Bergstrom et al, 2010,54 Sweden | NA | No | No | Unclear | No | Partially | Unclear |
| Titov et al, 2009,53 Australia | NA | No | No | No | No | No | No |
| McCrone et al, 2009,52 Sweden | NA | No | No | No | No | No | No |
| Mihalopoulos et al, 2005,51 Australia | NA | No | No | Unclear | No | Unclear | Unclear |
| Kaltenthaler et al, 2006,50 United Kingdom | Partially | No | No | Unclear | No | Partially | Unclear |
| Author, Year | Are the unit costs of resources obtained from best available sources? | Is an appropriate incremental analysis presented or can it be calculated from the reported data? | Are all important and uncertain parameters subjected to appropriate sensitivity analysis? | Is there a potential conflict of interest? | Overall assessment including applicability to the project (Minor limitations/potentially serious limitations/very serious limitations) |
|---|---|---|---|---|---|
| Kumar et al, 2018,63 United States | Unclear | No | No | Unclear | Potentially serious limitations |
| El Alaoui et al, 2017,61 Sweden | Unclear | Yes | Unclear | Unclear | Potentially serious limitations due to CMA design and duplication of data analysis |
| Hedman et al, 2016,60 Sweden | Unclear (self-reported questionnaire) | Yes | No | Unclear | Potentially serious limitations, selected sample, self-reported costs, no sensitivity analysis, no CEAC, no all comparators, no clear influence of medications |
| Dear et al, 2015,62 Australia | Yes | Yes | No | Unclear | Potentially serious limitations |
| Nordgren et al, 2014,56 Sweden | Unclear | Yes | Unclear | Unclear | Potentially serious limitations, selected population, only TAU, Unclear estimation of QALYs, very large increment as compared to other studies |
| Hedman et al, 2014,59 Sweden | Unclear | Yes | Unclear | Unclear | Potentially serious limitations, selected population, not clear cost calculations, only group CBT, drug comparator probably not used for this condition |
| Hedman et al, 2013,57 Sweden | Yes | Unclear | Partially | Unclear | Potentially serious limitations, short duration, estimates not presented well |
| National Collaborating Centre for Mental Health and NICE, 2013,64 United Kingdom | Partially | Partially | Partially | Unclear | Minor limitations, large modeling assumptions, but all strategies are modeled, results for year 1 are not presented, but are discussed as cost-effective, uncertainty analysis not presented |
| Joesch et al, 2012,55 United States | Unclear | Yes | Unclear | Unclear | Potentially serious limitations, study population quite sick, other comparators not well defined, iCBT mixed with other interventions |
| Hedman et al, 2011,58 Sweden | Unclear | Yes | Unclear | Unclear | Potentially serious limitations, selected population, no clear cost calculations, only group CBT, drug comparator probably not used for this condition, not clear quality of this RCT |
| National Collaborating Centre for Mental Health and NICE, 2011,9 United Kingdom | Partly | Yes | Unclear | Unclear | Potentially serious limitations, simple decision tree analysis, short modeling time, costs well estimated but may not be applicable to Ontario, utilities—EQ-5D but change based on assumption and linear extrapolation due to lack of data |
| Bergstrom et al, 2010,54 Sweden | Unclear | No | No | Unclear | Potentially serious limitations, study population selected, no QALY, costs not well estimated, analysis not well done |
| Titov et al, 2009,53 Australia | No | No | No | Unclear | Very serious limitations, selected population, cost estimates not well done |
| McCrone et al, 2009,52 Sweden | No | No | Unclear | Unclear | Very serious limitations, costs not measured, QALYs not estimated, selected population |
| Mihalopoulos et al, 2005,51 Australia | Unclear | No | No | Unclear | Very serious limitations, unclear how costs or outcomes were measured, QALYs not estimated, pilot project |
| Kaltenthaler et al, 2006,50 United Kingdom | Unclear | Partially | No | Unclear | Potentially serious limitations, simple model, 1-yr time horizon, utilities—EQ-5D but changed based on assumption and linear extrapolation due to lack of data from a study in individuals with prior panic disorder |
Abbreviations: CBT, cognitive-behavioural therapy; CEAC, cost-effectiveness acceptability curve; iCBT, internet-delivered CBT; NA, not applicable; NICE, National Institute for Health and Care Excellence; QALY, quality-adjusted life year; RCT, randomized controlled trial; TAU, treatment as usual.
Appendix 5: Long-Term Cost-Effectiveness of Guided iCBT Provided Within a Stepped-Care Model
Model Structure
We combined our short-term reference case iCBT model with a previously developed probabilistic Markov microsimulation model77 to examine the lifetime cost-effectiveness of guided iCBT by a therapist and face-to-face CBT versus usual care in people with new and recurrent major depression and anxiety. Our model simulates a hypothetical cohort of women and men aged 18 years and older with a primary diagnosis of mild to moderate major depression who are eligible for a course of iCBT. We use a stepped-care approach starting with therapist-guided iCBT followed by with group or individual face-to-face CBT if the disease recurs or progresses. In this analysis, we compared three strategies: (1) guided iCBT followed by individual CBT provided by a regulated health professional (nonphysician); (2) guided iCBT followed by group CBT provided by a regulated health professional (nonphysician); and (3) usual care that does not include CBT. We accounted for the possibility of repeating CBT up to three times in people with recurrent disease, as suggested by the guidelines.72,73 The model used a short weekly cycle to monitor changes in the progression of disease and adequately reflect what is being done in current clinical practice; we applied a half-cycle correction to balance the distribution of people who transition between health states at the beginning or end of each cycle.
The model schematic is provided in Figure A1. The current model was adapted from our published model77 and it incorporated a delivery of guided iCBT in the initial phase of a major depressive episode (MDE), and the benefits and costs of iCBT during all phases of the MDE. Prior models77 included in-person CBT only, starting from the first MDE. In the current model, we also allow for a possibility to drop out from the iCBT or CBT course each time the courses were provided (i.e., initial or recurrent episodes). The input parameters for face-to-face CBT were derived from our prior HTA.77 The input parameters relevant to guided iCBT were based on the current review. All parameter values used in the current model are described in Appendix 6 (Tables A6 to A9).
Figure A1: Simplified Markov Model Schematic: Guided iCBT with a Stepped-Care Approach.
Abbreviation: MDE, major depressive episode.
Note: Figure A1 depicts an individual-level (microsimulation) model that includes 11 Markov health states, each represented by an oval, reflecting the course of major depression. The simulation starts with a hypothetical person aged between 18 and 75 years with a mild or moderate major depressive episode. Each week cycle, the person has a chance to move among health states. Death is an absorbing Markov health state. In the acute initial phase lasting 12 weeks, a course of iCBT is applied. In two recurrent states, in-person CBT is the therapy of choice. People are allowed to drop out. The model accounts for the age at onset of major depression, counts the number of recurrent events, and modifies the risk of future major depressive episodes and changes in treatment accordingly. The model also takes into account the probability of death as a result of suicide according to the severity of disease and the probability of hospitalization in the complex depression health state. Transfer to the complex depression health state depends on the number of prior major depressive episodes-.
Our Markov model was developed through expert consultation and in accordance with the current clinical guidelines.2,5,6 It captures the episodic nature of major depression and anxiety disorders142,143 and distinguishes relapse from recurrence, which are components of the natural course of major depression. It follows the course of major depression, which consists of the following:
Two initial phases (acute and continuation), each lasting approximately 3 months, during which response and relapse are monitored with the goal of achieving remission
The maintenance phase, lasting approximately 6–24 months, during which full recovery can be established or recurrence of a full MDE can occur.2,5,6
Recurrence is defined as a full episode of depression after full recovery. Recurrence can happen in the maintenance phase 6 months after the initial diagnosis or later after a person has achieved a period of remission including at least 2 months with no significant symptoms.2,5,6 This model accounts for factors affecting the risk of recurrence (e.g., age at disease onset and number of prior episodes), changes in the severity of initial disease (mild to severe major depression), and consequent changes in treatment options after a transition from mild to moderate or severe disease. It also allows for the addition of pharmacotherapy for people who progress to more severe health states, for medication dose increase, or medication switch for people whose symptoms are not improving.77 The frequency of disease progression monitoring (by a general practitioner or psychiatrist) depends on the initial disease severity. In general, monitoring occurs every 2 weeks in the first 4 months and monthly thereafter for the remainder of the first year after a diagnosis of major depression. If a person achieves full remission and remains stable during the maintenance phase, they enter the well health state, in which they are considered to remain stable (without depression and receiving no treatment twice yearly follow-ups with a general practitioner). A patient has a chance of experiencing a recurrent episode in the maintenance phase, from the well health state, or while in one of the two recurrent health states (mild or moderate to severe, Figure 4). Our model accounted for age-dependent background mortality in Ontario and for differences in suicide rate based on the severity of major depression.144,145
Appendix 6: Model Inputs Used for the Life-Time Cost-Effectiveness Analysis of Guided iCBT Followed by Face-to-Face CBT Versus Usual Care
Table A6:
Input Parameters Associated With the Natural History of Major Depression: Probabilities and Risks
| Model Parameter Probabilities/Rates | Mean | Distribution | Source |
|---|---|---|---|
| Probability of major depression, by disease severity, no prior episode | NA | Ferrari et al, 2013146 | |
| Mild | 0.68 | ||
| Moderate to severe | 0.32 | ||
| Probability of major depression, by disease severity, with prior episode | NA | Hardeveld et al, 2013147 | |
| Mild | 0.24 | ||
| Moderate to severe | 0.76 | ||
| Probability of dropout: | NA | van Ballegooijen et al, 201492; Koeser et al, 201593 | |
| Guided iCBT | 0.19 | ||
| Face-to-face CBT | 0.16 | ||
| CBT plus pharmacotherapy | 0.23 | ||
| Pharmacotherapy only | 0.30 | ||
| No treatment | 0.29 | ||
| Probability of being not well after dropout | 0.33 | Expert consultation | |
| Probability of no response, acute phase | NA | Koeser et al, 201593 | |
| CBT only | 0.23 | ||
| CBT plus pharmacotherapy | 0.18 | ||
| Pharmacotherapy only | 0.70 | ||
| Probability of response (no relapse), continuation phase | NA | Koeser et al, 201593 | |
| CBT only | 0.69 | ||
| CBT plus pharmacotherapy | 0.75 | ||
| Pharmacotherapy only | 0.70 | ||
| No treatment | 0.43 | ||
| Annual rate of recurrence, year 1, maintenance phase | 0.15 | NA | Eaton et al, 2008127 |
| Annual rate or recurrence, long-term | NA | Eaton et al, 2008127 | |
| Year 2 | 0.20 | ||
| Year 4 | 0.75 | ||
| Year 5 | 0.70 | ||
| Year 10 | 0.50 | ||
| Annual rate or recurrence, long term, after 6 months of remissiona | NA | Hardeveld et al, 2013147 | |
| Year 1 | 0.025 | ||
| Year 2 | 0.045 | ||
| Year 5 | 0.13 | ||
| Year 10 | 0.23 | ||
| Year 15 | 0.23 | ||
| Year 20 | 0.42 | ||
| Rate of hospitalization | 0.10 | NA | Health Quality Ontario, 20161 |
| Background rate of death | Ontario life tables | NA | Statistics Canada, 2011148 |
| Rate of death by suicide | NA | Khan et al, 2003145; Khan and Schwartz, 2007144 | |
| General population | 0.000166 | ||
| Patients with depression | 0.001 | ||
| Patients with chronic/complex depression | 0.0024 | ||
| Risks | Risk Ratio (95% CI) | Distribution | Source |
| Relative risk of recurrent major depression, by age of disease onset, for each additional year | 0.96 (0.93–0.97) | Normalb (log-odds ratio) | Eaton et al, 2008127 |
| Relative risk of recurrent MDE, for each additional episode | 1.18 (1.06–1.31) | Normalb (log-odds ratio) | Mueller et al, 1999149 |
Abbreviations: CBT, cognitive behavioural disorder; CI, confidence interval; iCBT, internet-delivered CBT; MDE, major depressive episode; NA, not applicable.
Used in sensitivity scenario analysis only.
Distributions assigned in probabilistic sensitivity analysis.
Table A7:
Probability of Response, Initial Phase, Mild to Moderate Major Depression: Internet-Delivered CBT, CBT, and Medication
| Model Parameter | Mean (95% CI/SE)a | SMD (95%CI)b | Source |
|---|---|---|---|
| Probability of response (improvement) | |||
| Guided iCBT | 0.73 (0.66–0.77) | 0.83 (0.59–1.07) | Arnberg et al, 201420 |
| Individual face-to-face CBT | 0.73 (0.66–0.77) | NAc | Dedert et al, 2013,23 Andrews et al, 2018,21 Adelman et al, 20147 |
| Group face-to-face CBT | 0.73 (0.66–0.77) | NAc | Dedert et al, 2013,23 Andrews et al, 2018,21 Adelman et al, 20147 |
| Medication | 0.70 | Koeser et al, 201593 | |
Abbreviations: CBT, cognitive behavioural disorder; iCBT, internet-delivered CBT; CI, confidence interval; SMD, standardized mean difference; NA, not applicable; SE, standard error; SMD, standardized mean difference.
Beta distributions assigned in probabilistic sensitivity analysis to all parameters for which 95% CI or SE were specified in Table A6. Two parameters of the beta distribution (α,β) are derived from the mean and SE (stated for each model parameter). Formulas for these calculations, derived from the mean and SE (provided in the Tree Age software) are: α = ([Mean‸2] × [1 – Mean])/([SE‸2] – Mean); β = (1 – Mean) × ([(1 – Mean) × Mean]/([SE‸2] – 1).
Original value, reported in the article and clinical review.
Reported risk of recurrence, in-person CBT vs usual care in Biesheuvel-Leliefeld et al77,96 (relative risk [RR] 0.68, 95% CI 0.53–0.76), group CBT in Churchill et al97 (0.78, 95% CI 0.69–0.93), and sertraline in Cipriani & Geddes98 (RR 0.80, 95% CI 0.69–0.93). Details provided in our prior report.77
Table A8:
Efficacy Estimates Used in the Economic Model: Risk of Recurrence
| Model Parameter | Relative Risk (95% CI) | Distribution | Source |
|---|---|---|---|
| Risk of recurrence, major depression: CBT only vs no treatment | 0.68 (0.53–0.76) | Normal (log-odds ratio) | Biesheuvel-Leliefeld et al, 201596 |
| Risk of recurrence, major depression: CBT plus pharmacotherapy vs pharmacotherapy only | 0.94a (0.72–1.21) | Normal (log-odds ratio) | Amick et al, 2015150 |
| Risk of recurrence, major depression: Individual CBT vs group CBT | 0.78b (0.69–0.93) | Normal (log-odds ratio) | Churchill et al, 200197 |
| Risk of recurrence, major depression: Guided iCBT vs usual care | 0.67 (0.54–0.83) | Normal (log-odds ratio) | Andrews et al, 201821 |
| Risk of recurrence, second-generation antidepressant, sertraline | 0.80 (0.69–0.93) | Normal (log-odds ratio) | Cipriani et al, 2009151 |
Abbreviations: CBT, cognitive behavioural therapy, relates to face-to-face CBT; CI, confidence interval; iCBT, internet-delivered CBT.
Recalculated from the reported relative risk of 1.06 (95% CI 0.82–1.38).
Relative risk estimated from the odds ratio after accounting for a mean baseline probability of major depressive episode in the placebo arm of 0.57.152
Table A9:
Health State Utilities Used in the Economic Model
| Model Parameter | Mean (SD/SE, 95% CI)a | Distributionb | Source |
|---|---|---|---|
| Utilities | |||
| Acute phase, untreated new major depression | Beta | Mohiuddin and Payne, 2014153 | |
| Mild MDE | 0.69 (SD 0.14) | ||
| Moderate to severe MDE | 0.52 (SD 0.28) | ||
| Past major depression | Beta | Schaffer et al, 2002102 | |
| Mild MDE | 0.79 (SD 0.28) | ||
| Moderate to severe MDE | 0.67 (SD 0.36) | ||
| Guided iCBT | Beta | Hedman et al, 201158,104 | |
| 4 months | 0.82 (SD 0.18) | ||
| 12 months | 0.85 (SD 0.14) | ||
| Face-to-face CBT | Fixed | King et al, 2000103 | |
| 4 months | 0.85 (NR) | ||
| 12 months | 0.85 (NR) | ||
| Pharmacotherapy only–treated major depression | 0.63 (SD 0.19) | Beta | Revicki and Wood, 1998154 |
| Normal health, well health state | 0.94 (SE 0.03) | Beta | Lenert et al, 2000105 |
| Complex depression state, with treatment, nonhospitalized | 0.52 (SD 0.29) | Beta | Mann et al, 2009,155 Vallejo-Torres et al, 2015156 |
| Complex depression state, severe phase, hospitalized | 0.34 (SE 0.02) | Beta | Mann et al, 2009,155 Vallejo-Torres et al, 2015156 |
Abbreviations: CBT, cognitive behavioural therapy; CI, confidence interval; MDE, major depressive episode; NR, not reported; SD, standard deviation, SE, standard error.
Assumed to be the same for anxiety disorders and major depression, based on results of a study in patients with social anxiety disorder.
Beta distributions assigned in probabilistic sensitivity analysis (see above formulas for the calculations of the beta distribution's parameters: α and β.
Table A10:
Costs Used in the Economic Modela: Interventions, Usual Care, Follow-Ups, Complex Depression, and Hospitalization
| Model Parameter | Mean, $ (SE) | Distribution | Source |
|---|---|---|---|
| Costs of providing iCBT and CBT | |||
| Initial assessment, one-time costs, total: | 223.35 (55.84) | Gammab | |
| GP referral | 62.75 | OHIP code K005110 | |
| Assessment by psychiatrist or psychologist | 80.30 | OHIP code K197110 | |
| CBT provided by nonphysician, weeklyc | Gamma | Expert consultation, literature157–160: annual salary of $130,000 plus 30% benefits160,c | |
| Individual session (1 hr/wk) | 101.93 (25.48) | ||
| Group session (2 hr/wk) | 20.93 (5.10) | ||
| Therapist-guided iCBT, one-time cost | 1,466 | Gamma | Table 13 |
| Coach-guided iCBT, one-time costd | 1,389 | ||
| Employer-led iCBT, one-time costd | 887 | ||
| Costs of pharmacotherapy,e per week | |||
| Sertraline, 50 mg/d | 6.85 | Fixed | ODB111; Kolber et al, 2014112 |
| Sertraline, 100 mg/d | 7.44 | Fixed | ODB111; Kolber et al, 2014112 |
| Venlafaxine, 75 mg/d | 5.93 | Fixed | ODB111; Kolber et al, 2014112 |
| Costs of follow-up, per week | |||
| CBT only (mild major depression) | 20.85 (5.21) | Gamma | Calculated based on OHIP codes K005 and K198110; details provided in text |
| CBT plus pharmacotherapy (moderate to severe major depression) | 33.20 (8.30) | Gamma | Calculated based on OHIP codes K005 and K198110; details provided in text |
| Pharmacotherapy, usual care (moderate to severe major depression) | 33.20 (8.30) | Gamma | Calculated based on OHIP codes K005 and K198110; details provided in text |
| No treatment, usual care (mild major depression) | 7.24 (1.71) | Gamma | Calculated based on OHIP codes K005 and K198110; details provided in text |
| Costs: Complex depression | |||
| Responders, weekly | 163.78 (40.92) | Gamma | Health Quality Ontario, 2016161 |
| Nonresponders, weeklyb | 222.47 (55.61) | Gamma | Health Quality Ontario, 2016161 |
| One-time costs of hospitalization (per 10 days) | 5,422.88 (1,355.72) | Gamma | Bereza et al, 2012162 |
| Costs: Well health state, per week | |||
| Follow-up by GP (2 visits per year) | 2.41 (0.60) | Gamma | Calculated based on OHIP code K005110 |
Abbreviations: CBT, cognitive behavioural therapy, relates to face-to-face CBT; iCBT, internet-delivered CBT; GP, general practitioner; ODB, Ontario Drug Benefit program; OHIP, Ontario Health Insurance Plan Schedule of Benefits and Fees; SE, standard error.
Original nondiscounted costs in 2017 Canadian dollars, estimated per weekly cycle; standard error based on an assumption that the mean costs vary by ±25%. Two parameters of the gamma distribution (α,Λ) are derived from the mean and SE. Formulas for these calculations are: α = (Mean2)/(SE2); Λ= Mean/([MeanxSE]2).
Used in probabilistic sensitivity scenario analyses.
Labour costs after applied salary adjustment for a full-time professional working 1,658 hours per year.
Used in scenario analyses
Costs of drugs include a dispensing fee of $10.22 and a mark-up of $1.71 for 30-day use.112
Appendix 7: Scenario Analysis: Lower Costs of Face-to-Face Delivery Compared With iCBT
Table A11:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and Usual Care for Major Depression, Lower Delivery Costs of In-Person CBT
| Strategy | Mean Costsa ($) | Mean QALYs | Incremental Costsb($) | Incremental QALYsc | ICER $/QALY gained |
|---|---|---|---|---|---|
| Unguided iCBT | 274.62 | 0.778 | – | – | |
| Usual care | 409.40 | 0.789 | – | – | – |
| Group CBT | 1,582.54 | 0.817 | Dominated | ||
| Guided iCBT | 1,668.91 | 0.827 | 1,259 | 0.04 | 39,661 |
| Individual CBT | 2,467.08 | 0.827 | – | – | Dominated |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (guided CBT) – average cost (usual care).
Incremental effect = average effect (guided CBT) – average effect (usual care).
Note: Results may appear incorrect due to rounding.
Table A12:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, and Usual Care for Anxiety Disorders, Lower Delivery Costs of In-Person CBT
| Strategy | Mean Costsa ($) | Mean QALYs | Incremental Costsb ($) | Incremental QALYsc | ICER $/QALY gained |
|---|---|---|---|---|---|
| Unguided iCBT | 269.69 | 0.801 | – | – | |
| Usual care | 409.40 | 0.787 | – | – | Dominated |
| Group CBT | 866.52 | 0.817 | 596.8 | 0.015 | 38,515 |
| Individual CBT | 1495.53 | 0.827 | – | – | Extended Dominance |
| Guided CBT | 1666.59 | 0.833 | 800.1 | 0.017 | 47,753 |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (guided CBT) – average cost (unguided iCBT).
Incremental effect = average effect (guided CBT) – average effect (unguided iCBT).
Note: Results may appear incorrect due to rounding.
Appendix 8: Scenario Analysis: Medication Starting From Baseline
Table A13:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, Usual Care for Major Depression, and Medication From Beginning of Intervention
| Strategy | Mean Costsa ($) | Mean QALYs | Incremental Costsb ($) | Incremental QALYsc | ICER, $/QALY gained |
|---|---|---|---|---|---|
| Unguided iCBT | 794.22 | 0.778 | – | – | |
| Usual care | 862.23 | 0.789 | – | – | |
| Group CBT | 2,132.44 | 0.817 | – | – | Extended Dominance |
| Guided iCBT | 2,188.82 | 0.827 | 1,328 | 0.04 | 33,397 |
| Individual CBT | 3,096.02 | 0.827 | 907 | 0.0001 | 9,216,211 |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (guided CBT) – average cost (usual care).
Incremental effect = average effect (guided CBT) – average effect (usual care).
Note: Results may appear incorrect due to rounding.
Table A14:
Cost–Utility Analysis, Sequential Approach: Internet-Delivered CBT, In-Person CBT, Usual Care for Anxiety Disorders, and Medication From Beginning of Intervention
| Strategy | Mean Costsa ($) | Mean QALYs | Incremental Costsb ($) | Incremental QALYsc | ICER $/QALY gained |
|---|---|---|---|---|---|
| Unguided iCBT | 790.43 | 0.801 | – | – | |
| Usual care | 862.26 | 0.788 | – | – | Dominated |
| Group CBT | 1,719.15 | 0.817 | – | – | Extended Dominance |
| Guided iCBT | 2,188.54 | 0.833 | 1,398 | 0.03 | 43,250 |
| Individual CBT | 2,398.23 | 0.827 | – | – | Dominated |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. No discounting was done because of the 12-month time horizon.
Incremental cost = average cost (guided CBT) – average cost (unguided iCBT).
Incremental effect = average effect (guided CBT) – average effect (unguided iCBT).
Note: Results may appear incorrect due to rounding.
Appendix 9: Results: Life-Time Cost-Effectiveness Analysis of Guided iCBT Followed by In-Person CBT Versus Usual Care
In a sequential cost–utility analysis ranking three strategies by increasing costs, usual care was dominated (the highest costs and lowest benefits) and excluded from the final calculations (Table A15). Therefore, as compared to usual care, both types of guided CBT—internet and inperson—strategies were cost saving and associated with increments in QALYs. Over a lifetime time horizon using a stepped-care model, guided iCBT by a therapist followed by individual face-to-face CBT represented an economically attractive option when compared with guided iCBT followed by group CBT. It was associated with increases in both the mean discounted quality-adjusted survival of 0.036 QALYs (95% Crl −0.11 to 0.20) and the mean discounted costs of $39.82 (95% Crl −$2,886 to $2,351), yielding an ICER of $1,098 per QALY gained.
Table A15:
Lifetime Cost-Effectiveness of Guided iCBT Followed by Face-To-Face CBT, Cost–Utility Analysis With Sequential Approach: Stepped-Care Model
| Strategy | Mean Costs, $a | Mean QALYs | Incremental Costs, $a,b | Incremental QALYsc | ICER, $/QALY Gaineda |
|---|---|---|---|---|---|
| Guided iCBT followed by group CBT | 280,498.29 | 18.300 | – | – | – |
| Guided iCBT followed by individual CBT | 280,538.11 | 18.336 | 39.82 | 0.036 | 1,098 |
| Usual care | 283,651.49 | 18.090 | 3,113.37 | −0.246 | Dominated |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year.
All costs in 2018 Canadian dollars. Costs and QALYs discounted at 1.5% rate.
Incremental cost = average cost (strategy of guided CBT followed by individual CBT) – average cost (strategy of guided CBT followed by group CBT).
Incremental effect = average effect (strategy of guided CBT followed by individual CBT) – average effect (strategy of guided CBT followed by group CBT).
Note: Results may appear incorrect because of rounding.
In this cost–utility analysis considering a stepped-care modeling of two different strategies of internet-delivered and in-person CBT over a lifetime time horizon, the probability of cost-effectiveness of guided iCBT followed by individual face-to-face CBT was 60% at a willingness-to-pay value of $50,000 per QALY gained, increasing to 63% at $100,000 per QALY gained (Figure 8).
Figure A2: Cost-Effectiveness Acceptability Curve: Guided iCBT in a Stepped-Care Model Including In-Person CBT (Individual or Group).
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered CBT; QALY, quality-adjusted life-year; WTP, willingness to pay.
Note: The cost-effectiveness acceptability curve graphically presents the probability of cost-effectiveness of the examined CBT strategies and usual care across various willingness-to-pay values on the x–y coordinate system. The x-axis shows the probability of cost-effectiveness (range: 0–1) and the y-axis represents willingness-to-pay values (range: $0 to $100,000 per QALY gained).
Sensitivity Scenario Analyses: Lifetime Cost-Effectiveness of Guided iCBT Within a Stepped-Care Model
We examined parameter and structural model uncertainty in scenario analyses (Table A16). The results remained robust in all scenarios with guided iCBT followed by individual CBT as the most economically attractive strategy. The probability of cost-effectiveness of this strategy at a willingness-to-pay value of $100,000 per QALY gained ranged from 63% to 65% in scenarios addressing parameter uncertainty (e.g., changes in the efficacy of iCBT and CBT, dropout rates, utilities, and medication costs). The probability of cost-effectiveness of guided iCBT followed by individual CBT increased from 60% (reference case) to 99% (5-yr time horizon).
Table A16:
Sensitivity Scenario Analysis Results: iCBT Followed by In-Person CBT in a Stepped-Care Modela
| Parameter/Assumption: | Guided iCBT With Individual CBT vs. Guided iCBT With Group CBT, ICER ($/QALY) | Guided iCBT With Individual CBT vs. Usual Care, ICER ($/QALY) | Guided iCBT With Group CBT vs. Usual Care, ICER ($/QALY) |
|---|---|---|---|
| 1. Efficacy of CBT: decreased by 25% | |||
| a. Base case | 1,098 | Cost saving | Cost saving |
| b. Scenario: 0.25 × RR_iCBT & RR_CBT | Dominant | Cost saving | Cost saving |
| 2. Dropout: increased 2 times in CBT | |||
| a. Base case: 19% (iCBT) and 16% (CBT) | 1,098 | Cost saving | Cost saving |
| b. Scenarios: 2 × base case probability | Dominant | 846 | Dominated |
| 3. Utility of health states in which iCBT/CBT was provided: 10% lower | |||
| a. Base case: 0.85 at 12 months | 1,098 | Cost saving | Cost saving |
| b. Scenarios: 10% lower | 560 | Cost saving | Cost saving |
| 4. Medication costs | |||
| a. Base case | 1,098 | Cost saving | Cost saving |
| b. Scenario: Increased 2x (all strategies) | 1,097 | Cost saving | Cost saving |
| 5. Time horizon | |||
| a. Base case: lifetime | 1,098 | Cost saving | Cost saving |
| b. Scenario: 5 years | 6,088 | Cost saving | Cost saving |
Abbreviations: CBT, cognitive behavioural therapy; iCBT, internet-delivered cognitive behavioural therapy; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year; RR, relative risk.
All costs in 2018 Canadian dollars, discounted at 1.5%.
Appendix 10: Letter of Information
Appendix 11: Interview Guide
Background
Provide information on Health Quality Ontario's mandate.
Explain the health technology assessment program and part of Patient, Caregiver and Public Engagement. Explain the purpose of the interview.
Confirm consent for audio-recording
Restate options of withdrawal, freedom of sharing and not-sharing of information
Lived Experience
What are the biggest challenges of living/caring for someone with anxiety or depression?
How does it impact your day to day routine? How would you describe your quality of life?
Therapies
What are the current therapies/treatments that you aware of?
What therapies/treatments are accessible to you? Did you face any barriers?
Which therapies/treatments have you explored? And why did you explore these?
How did the therapies/treatments meet your needs?
How did the therapies impact your quality of life?
What were the side-effects and benefits?
Were there any equity issues related to cost, access, knowledge of health care system?
Internet-Delivered Cognitive Behavioural Therapy
Please explain the process of therapy
How did this therapy meet/not meet your needs? How was it adequate/inadequate? Quality of life? Empowerment? Ownership? Adherence? Lifestyle?
What were the side-effects and benefits? Anxiety, painful, intrusiveness?
Were there equity issues related to cost, access, knowledge of health care system, etc.? Travel, repeat visits?
What challenges did this procedure address?
Author contributions
This report was developed by a multidisciplinary team from Health Quality Ontario in collaboration with the Canadian Agency for Drugs and Technologies in Health (CADTH). The clinical epidemiologists were Amanda Manoharan and Kristen McMartin. Calvin Young, clinical research officer at CADTH, performed a second review of titles, abstracts, and full-text articles, as well as a second review of the data extracted. The health economist was Olga Gajic-Veljanoski, the patient and public partnering analyst was Arshia Ali, and the medical librarian was Melissa Walter.
KEY MESSAGES
What Is This Health Technology Assessment About?
Major depression is characterized by depressed mood and diminished interest or pleasure in all, or almost all, activities. Anxiety disorders are characterized by feelings of distress and fear that interfere with day-to-day functioning. Cognitive behavioural therapy (CBT) is a form of evidence-based psychotherapy used to treat major depression and anxiety disorders. Internet-delivered CBT (iCBT) is structured CBT delivered via the internet. Guided iCBT involves communication with a regulated health care professional, whereas unguided iCBT is provided without the support of a regulated health care professional.
This health technology assessment looked at how safe, effective, and cost-effective guided and unguided iCBT are for the treatment of adults with mild to moderate major depression or anxiety disorders. We also looked at the budget impact of publicly funding iCBT and the preferences, values, and experiences of people with mild to moderate major depression or anxiety disorders.
What Did This Health Technology Assessment Find?
Compared with waiting list, guided iCBT improves symptoms of mild to moderate major depression and select anxiety disorders.
Guided iCBT represents the most economical option for the short-term treatment of adults with mild to moderate major depression or anxiety disorders. Over the next 5 years, we estimate that publicly funding guided iCBT will result in additional annual costs of between $10 million and $40 million for mild to moderate major depression and between $16 million and $65 million for anxiety disorders.
People with anxiety or depression with whom we spoke viewed iCBT as an effective treatment option. Internet-delivered CBT may be especially beneficial for those whose symptoms may prevent them from leaving home. However, participants reported important barriers and limitations to using iCBT, including the need for a computer, internet access, and computer literacy, as well as the ability to understand complex written information. Participants found that the cost of treatment, the number of sessions in a course of treatment, and the lack of follow-up support were also substantial drawbacks for iCBT.
Contributor Information
Health Quality Ontario:
Amanda Manoharan, Kristen McMartin, Calvin Young, Olga Gajic-Veljanoski, Arshia Ali, and Melissa Walter
About Health Quality Ontario
Health Quality Ontario is the provincial lead on the quality of health care. We help nurses, doctors and other health care professionals working hard on the frontlines be more effective in what they do – by providing objective advice and data, and by supporting them and government in improving health care for the people of Ontario.
We focus on making health care more effective, efficient and affordable through a legislative mandate of:
Reporting to the public, organizations, government and health care providers on how the health system is performing,
Finding the best evidence of what works, and
Translating this evidence into clinical standards; recommendations to health care professionals and funders; and tools that health care providers can easily put into practice to make improvements.
Health Quality Ontario is governed by a 12-member Board of Directors with a broad range of expertise – doctors, nurses, patients and from other segments of health care – and appointed by the Minister of Health and Long-Term Care.
In everything it does, Health Quality Ontario brings together those with first-hand experience to hear their experiences and views of how to make them better. We partner with patients, residents, families and caregivers to be full participants in designing our programs and services, to ensure they are aligned to their needs and priorities. We work collaboratively with organizations across the province to encourage the spread of innovative and proven programs to support high quality care, while also saving money and eliminating redundancy. And, we work with clinicians on the frontlines to use their collective wisdom and experience to bring about positive change in areas important to Ontario – such as addressing the challenges of hallway health care and mental health.
For example, 29 Ontario hospitals participated in a pilot program last year that reduced infections due to surgery by 18% – which in turn reduces the number of patients returning to hospital after surgery and alleviating some of the challenges faced in hallway health care. This program enabled surgeons to see their surgical data and how they perform in relation to each other and to 700 other hospitals worldwide. We then helped them identify and action improvements to care. Forty-six hospitals across Ontario are now part of this program, covering 80% of hospital surgeries.
Health Quality Ontario also develops quality standards for health conditions that demonstrate unnecessary gaps and variations in care across the province, such as in major depression or schizophrenia. Quality standards are based on the best evidence and provide recommendations to government, organizations and clinicians. They also include a guide for patients to help them ask informed questions about their care.
In addition, Health Quality Ontario's health technology assessments use evidence to assess the effectiveness and value for money of new technologies and procedures, and incorporate the views and preferences of patients, to make recommendations to government on whether they should be funded.
Each year, we also help hospitals, long-term care homes, home care and primary care organizations across the system create and report on the progress of their annual Quality Improvement Plans, which is their public commitment on their priorities to improve health care quality.
Health Quality Ontario is committed to supporting the development of a quality health care system based on six fundamental dimensions: efficient, timely, safe, effective, patient-centred and equitable.
Our goal is to challenge the status quo and to focus on long-lasting pragmatic solutions that improve the health of Ontarians, enhance their experience of care, reduce health care costs, and support the well-being of health care providers. A quality health system results in Ontarians leading healthier and more productive lives, and a vibrant society in which everyone benefits.
REFERENCES
- (1).Health Quality Ontario. Major depression: care for adults and adolescents [Internet]. Toronto: Queen's Printer for Ontario; 2016. [cited 2018 Aug 18]. Available from: http://www.hqontario.ca/portals/0/documents/evidence/quality-standards/qs-depression-clinical-guide-1609-en.pdf [Google Scholar]
- (2).Patten SB, Kennedy SH, Lam RW, O'Donovan C, Filteau MJ, Parikh SV, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for the management of major depressive disorder in adults. I. Classification, burden and principles of management. J Affect Disord. 2009;117 Suppl 1:S5–S14. [DOI] [PubMed] [Google Scholar]
- (3).American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Washington (DC): The Association; 2013. [Google Scholar]
- (4).Kennedy SH, Lam RW, Parikh SV, Patten SB, Ravindran AV. Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for the management of major depressive disorder in adults. Introduction. J Affect Disord. 2009;117 Suppl 1:S1–2. [DOI] [PubMed] [Google Scholar]
- (5).Lam RW, Kennedy SH, Grigoriadis S, McIntyre RS, Milev R, Ramasubbu R, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for the management of major depressive disorder in adults. III. Pharmacotherapy. J Affect Disord. 2009;117 Suppl 1:S26–43. [DOI] [PubMed] [Google Scholar]
- (6).Parikh SV, Segal ZV, Grigoriadis S, Ravindran AV, Kennedy SH, Lam RW, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) clinical guidelines for the management of major depressive disorder in adults. II. Psychotherapy alone or in combination with antidepressant medication. J Affect Disord. 2009;117 Suppl 1:S15–25. [DOI] [PubMed] [Google Scholar]
- (7).Adelman CB, Panza KE, Bartley CA, Bontempo A, Bloch MH. A meta-analysis of computerized cognitive-behavioral therapy for the treatment of DSM-5 anxiety disorders. J Clin Psychiatry. 2014;75(7):e695–704. [DOI] [PubMed] [Google Scholar]
- (8).Kupfer DJ. Anxiety and DSM-5. Dialogues Clin Neurosci. 2015;17(3):245–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (9).National Collaborating Centre for Mental Health. Generalised anxiety disorder in adults: management in primary, secondary and community care. NICE clinical guidelines, No. 113 [Internet]. Leicester (UK): British Psychological Society; 2011. [cited 2016 Jun 5]. Available from: https://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0041616 [Google Scholar]
- (10).Pelletier L, O'Donnell S, McRae L, Grenier J. The burden of generalized anxiety disorder in Canada. Health Promot Chronic Dis Prev Can. 2017;37(2):54–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (11).Statistics Canada. Table 13-10-0465-01: Mental health indicators [Internet]. Ottawa (ON): Statistics Canada; 2018. [cited 2018 Oct 5]. Available from: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310046501 [Google Scholar]
- (12).Mood Disorders Society of Canada. Chapter 5 Anxiety Disorders. Belleville, Ontario: N.D. [Google Scholar]
- (13).Churchill R, Moore TH, Furukawa TA, Caldwell DM, Davies P, Jones H, et al. ‘Third wave’ cognitive and behavioural therapies versus treatment as usual for depression. Cochrane Database Syst Rev. 2013;10:CD008705. [DOI] [PubMed] [Google Scholar]
- (14).Gratzer D, Khalid-Khan F. Internet-delivered cognitive behavioural therapy in the treatment of psychiatric illness. CMAJ. 2016;188(4):263–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (15).Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev. 2015;3:CD011565. [DOI] [PubMed] [Google Scholar]
- (16).Katzman MA, Bleau P, Blier P, Chokka P, Kjernisted K, Van Ameringen M, et al. Canadian clinical practice guidelines for the management of anxiety, posttraumatic stress and obsessive-compulsive disorders. BMC Psychiatry. 2014;14(1):S1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (17).National Institute for Health and Care Excellence. Depression in adults: recognition and management [Internet]. London: The Institute; 2009. [cited 2018 Apr 6]. Available from: https://www.nice.org.uk/guidance/cg90 [PubMed] [Google Scholar]
- (18).McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6. [DOI] [PubMed] [Google Scholar]
- (19).Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (20).Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PLoS One. 2014;9(5):e98118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (21).Andrews G, Basu A, Cuijpers P, Craske MG, McEvoy P, English CL, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord. 2018;55:70–8. [DOI] [PubMed] [Google Scholar]
- (22).Kaltenthaler E, Parry G, Beverley C, Ferriter M. Computerised cognitive-behavioural therapy for depression: systematic review. Br J Psychiatry. 2008;193(3):181–4. [DOI] [PubMed] [Google Scholar]
- (23).Dedert E, McDuffie JR, Swinkels C, Shaw R, Fulton J, Allen KD, et al. Computerized cognitive behavioral therapy for adults with depressive or anxiety disorders [Internet]. Washington (DC): U.S. Department of Veterans Affairs; 2013. [cited 2018 Oct 11]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK269001/ [PubMed] [Google Scholar]
- (24).Richards D, Richardson T, Timulak L, McElvaney J. The efficacy of internet-delivered treatment for generalized anxiety disorder: a systematic review and meta-analysis. Internet Interv. 2015;2(3):272–82. [Google Scholar]
- (25).Kampmann IL, Emmelkamp PM, Morina N. Meta-analysis of technology-assisted interventions for social anxiety disorder. J Anxiety Disord. 2016;42:71–84. [DOI] [PubMed] [Google Scholar]
- (26).Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(6):e1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (27).Young C, Helis E, Williams D. Internet-delivered cognitive behavioural therapy for major depressive disorder and anxiety disorders: an environmental scan (environmental scan no. 78) [Internet]. Ottawa (ON) 2019.
- (28).Ahern E, Kinsella S, Semkovska M. Clinical efficacy and economic evaluation of online cognitive behavioral therapy for major depressive disorder: a systematic review and meta-analysis. Expert Rev Pharmacoecon Outcomes Res. 2018;18(1):25–41. [DOI] [PubMed] [Google Scholar]
- (29).Paganini S, Teigelkotter W, Buntrock C, Baumeister H. Economic evaluations of internet- and mobile-based interventions for the treatment and prevention of depression: a systematic review. J Affect Disord. 2018;225:733–55. [DOI] [PubMed] [Google Scholar]
- (30).Kolovos S, van Dongen JM, Riper H, Buntrock C, Cuijpers P, Ebert DD, et al. Cost effectiveness of guided internet-based interventions for depression in comparison with control conditions: an individual-participant data meta-analysis. Depress Anxiety. 2018;35(3):209–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (31).Brettschneider C, Djadran H, Harter M, Lowe B, Riedel-Heller S, Konig HH. Cost-utility analyses of cognitive-behavioural therapy of depression: a systematic review. Psychother Psychosom. 2015;84(1):6–21. [DOI] [PubMed] [Google Scholar]
- (32).Ophuis RH, Lokkerbol J, Heemskerk SC, van Balkom AJ, Hiligsmann M, Evers SM. Cost-effectiveness of interventions for treating anxiety disorders: a systematic review. J Affect Disord. 2017;210:1–13. [DOI] [PubMed] [Google Scholar]
- (33).Lewis C, Pearce J, Bisson JI. Efficacy, cost-effectiveness and acceptability of self-help interventions for anxiety disorders: systematic review. Br J Psychiatry. 2002;200(1):15–21. [DOI] [PubMed] [Google Scholar]
- (34).Donker T, Blankers M, Hedman E, Ljotsson B, Petrie K, Christensen H. Economic evaluations of internet interventions for mental health: a systematic review. Psychol Med. 2015;45(16):3357–76. [DOI] [PubMed] [Google Scholar]
- (35).Hedman E, Ljotsson B, Lindefors N. Cognitive behavior therapy via the internet: a systematic review of applications, clinical efficacy and cost-effectiveness. Expert Rev Pharmacoecon Outcomes Res. 2012;12(6):745–64. [DOI] [PubMed] [Google Scholar]
- (36).Littlewood E, Duarte A, Hewitt C, Knowles S, Palmer S, Walker S, et al. A randomised controlled trial of computerised cognitive behaviour therapy for the treatment of depression in primary care: the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Health Technol Assess. 2015;19(101):viii, xxi-171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (37).Duarte A, Walker S, Littlewood E, Brabyn S, Hewitt C, Gilbody S, et al. Cost-effectiveness of computerized cognitive-behavioural therapy for the treatment of depression in primary care: findings from the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Psychol Med. 2017;47(10):1825–35. [DOI] [PubMed] [Google Scholar]
- (38).Brabyn S, Araya R, Barkham M, Bower P, Cooper C, Duarte A, et al. The second Randomised Evaluation of the Effectiveness, cost-effectiveness and Acceptability of Computerised Therapy (REEACT-2) trial: does the provision of telephone support enhance the effectiveness of computer-delivered cognitive behaviour therapy? A randomised controlled trial. Health Technol Assess. 2016;20(89):1–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (39).Dixon P, Hollinghurst S, Edwards L, Thomas C, Foster A, Davies B, et al. Cost-effectiveness of telehealth for patients with depression: evidence from the healthlines randomised controlled trial. BJPsych Open. 2016;2(4):262–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (40).Hollinghurst S, Peters TJ, Kaur S, Wiles N, Lewis G, Kessler D. Cost-effectiveness of therapist-delivered online cognitive-behavioural therapy for depression: randomised controlled trial. Br J Psychiatry. 2010;197(4):297–304. [DOI] [PubMed] [Google Scholar]
- (41).Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, et al. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychol Med. 2014;44(4):741–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (42).McCrone P, Knapp M, Proudfoot J, Ryden C, Cavanagh K, Shapiro DA, et al. Cost-effectiveness of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry. 2004;185:55–62. [DOI] [PubMed] [Google Scholar]
- (43).Romero-Sanchiz P, Nogueira-Arjona R, Garcia-Ruiz A, Luciano JV, Garcia Campayo J, Gili M, et al. Economic evaluation of a guided and unguided internet-based CBT intervention for major depression: results from a multi-center, three-armed randomized controlled trial conducted in primary care. PLoS One. 2017;12(2):e0172741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (44).Titov N, Dear BF, Ali S, Zou JB, Lorian CN, Johnston L, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial. Behav Ther. 2015;46(2):193–205. [DOI] [PubMed] [Google Scholar]
- (45).Geraedts AS, van Dongen JM, Kleiboer AM, Wiezer NM, van Mechelen W, Cuijpers P, et al. Economic evaluation of a web-based guided self-help intervention for employees with depressive symptoms: results of a randomized controlled trial. J Occup Environ Med. 2015;57(6):666–75. [DOI] [PubMed] [Google Scholar]
- (46).Gerhards SA, de Graaf LE, Jacobs LE, Severens JL, Huibers MJ, Arntz A, et al. Economic evaluation of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry. 2010;196(4):310–8. [DOI] [PubMed] [Google Scholar]
- (47).Warmerdam L, van Straten A, Jongsma J, Twisk J, Cuijpers P. Online cognitive behavioral therapy and problem-solving therapy for depressive symptoms: exploring mechanisms of change. J Behav Ther Exp Psychiatry. 2010;41(1):64–70. [DOI] [PubMed] [Google Scholar]
- (48).Solomon D, Proudfoot J, Clarke J, Christensen H. e-CBT (myCompass), antidepressant medication, and face-to-face psychological treatment for depression in Australia: a cost-effectiveness comparison. J Med Internet Res. 2015;17(11):e255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (49).Lee YC, Gao L, Dear BF, Titov N, Mihalopoulos C. The cost-effectiveness of the online MindSpot Clinic for the treatment of depression and anxiety in Australia. J Ment Health Policy Econ. 2017;20(4):155–66. [PubMed] [Google Scholar]
- (50).Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, et al. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technol Assess. 2006;10(33):iii, xi-xiv, 1–168. [DOI] [PubMed] [Google Scholar]
- (51).Mihalopoulos C, Kiropoulos L, Shih ST, Gunn J, Blashki G, Meadows G. Exploratory economic analyses of two primary care mental health projects: implications for sustainability. Med J Aust. 2005;183(10 Suppl):S73–S6. [DOI] [PubMed] [Google Scholar]
- (52).McCrone P, Marks IM, Mataix-Cols D, Kenwright M, McDonough M. Computer-aided self-exposure therapy for phobia/panic disorder: a pilot economic evaluation. Cogn Behav Ther. 2009;38(2):91–9. [DOI] [PubMed] [Google Scholar]
- (53).Titov N, Andrews G, Johnston L, Schwencke G, Choi I. Shyness programme: longer term benefits, cost-effectiveness, and acceptability. Aust N Z J Psychiatry. 2009;43(1):36–44. [DOI] [PubMed] [Google Scholar]
- (54).Bergstrom J, Andersson G, Ljotsson B, Ruck C, Andreewitch S, Karlsson A, et al. Internet-versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: a randomised trial. BMC Psychiatry. 2010;10:54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (55).Joesch JM, Sherbourne CD, Sullivan G, Stein MB, Craske MG, Roy-Byrne P. Incremental benefits and cost of coordinated anxiety learning and management for anxiety treatment in primary care. Psychol Med. 2012;42(9):1937–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (56).Nordgren LB, Hedman E, Etienne J, Bodin J, Kadowaki A, Eriksson S, et al. Effectiveness and cost-effectiveness of individually tailored internet-delivered cognitive behavior therapy for anxiety disorders in a primary care population: a randomized controlled trial. Behav Res Ther. 2014;59:1–11. [DOI] [PubMed] [Google Scholar]
- (57).Hedman E, Andersson E, Lindefors N, Andersson G, Ruck C, Ljotsson B. Cost-effectiveness and long-term effectiveness of internet-based cognitive behaviour therapy for severe health anxiety. Psychol Med. 2013;43(2):363–74. [DOI] [PubMed] [Google Scholar]
- (58).Hedman E, Andersson E, Ljotsson B, Andersson G, Ruck C, Lindefors N. Cost-effectiveness of internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: results from a randomized controlled trial. Behav Res Ther. 2011;49(11):729–36. [DOI] [PubMed] [Google Scholar]
- (59).Hedman E, Axelsson E, Gorling A, Ritzman C, Ronnheden M, El Alaoui S, et al. Internet-delivered exposure-based cognitive-behavioural therapy and behavioural stress management for severe health anxiety: randomised controlled trial. Br J Psychiatry. 2014;205(4):307–14. [DOI] [PubMed] [Google Scholar]
- (60).Hedman E, Andersson E, Ljotsson B, Axelsson E, Lekander M. Cost effectiveness of internet-based cognitive behaviour therapy and behavioural stress management for severe health anxiety. BMJ Open. 2016;6(4):e009327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (61).El Alaoui S, Hedman-Lagerlof E, Ljotsson B, Lindefors N. Does internet-based cognitive behaviour therapy reduce healthcare costs and resource use in treatment of social anxiety disorder? A cost-minimisation analysis conducted alongside a randomised controlled trial. BMJ Open. 2017;7(9):e017053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (62).Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Sheehan J, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety: a randomized controlled trial. Behav Ther. 2015;46(2):206–17. [DOI] [PubMed] [Google Scholar]
- (63).Kumar S, Jones Bell M, Juusola JL. Mobile and traditional cognitive behavioral therapy programs for generalized anxiety disorder: a cost-effectiveness analysis. PLoS One. 2018;13(1):e0190554. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (64).National Collaborating Centre for Mental Health. Social anxiety disorder recognition, assessment and treatment. NICE clinical guidelines, No. 159 [Internet]. Leicester (UK): British Psychological Society; 2013. [cited 2016 Jun 05]. Available from: https://www.ncbi.nlm.nih.gov/pubmedhealth/PMH0071627 [PubMed] [Google Scholar]
- (65).Apolinario-Hagen J, Harrer M, Kahlke F, Fritsche L, Salewski C, Ebert DD. Public attitudes toward guided internet-based therapies: web-based survey study. JMIR Ment Health. 2018;5(2):e10735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (66).Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—explanation and elaboration: a report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health. 2013;16(2):231–50. [DOI] [PubMed] [Google Scholar]
- (67).Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)–explanation and elaboration: a report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health. 2013;16(2):231–50. [DOI] [PubMed] [Google Scholar]
- (68).Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada, 4th edition [Internet]. Ottawa (ON): The Agency; 2017. [cited 2017 Apr 04]. Available from: https://www.cadth.ca/guidelines-economic-evaluation-health-technologies-canada-4th-edition [Google Scholar]
- (69).Katz C, Stein MB, Sareen J. Anxiety disorders in the DSM-5: new rules on diagnosis and treatment [Internet]. Toronto (ON): Canadian Network for Mood and Anxiety Treatments; 2013. [cited 2018 Apr 13]. Available from: http://www.moodandanxietyrounds.ca/crus/144-010%20English.pdf [Google Scholar]
- (70).Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient health questionnaire. JAMA. 1999;282(18):1737–44. [DOI] [PubMed] [Google Scholar]
- (71).Lam RW, Kennedy SH, Parikh SV, MacQueen GM, Milev RV, Ravindran AV, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016. clinical guidelines for the management of adults with major depressive disorder: introduction and methods. Can J Psychiatry. 2016;61(9):506–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (72).Kennedy SH, Lam RW, McIntyre RS, Tourjman SV, Bhat V, Blier P, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016. clinical guidelines for the management of adults with major depressive disorder: section 3. Pharmacological treatments. Can J Psychiatry. 2016;61(9):540–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (73).Parikh SV, Quilty LC, Ravitz P, Rosenbluth M, Pavlova B, Grigoriadis S, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) 2016. clinical guidelines for the management of adults with major depressive disorder: section 2. Psychological treatments. Can J Psychiatry. 2016;61(9):524–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (74).Patten SB. Updated CANMAT guidelines for treatment of major depressive disorder. Can J Psychiatry. 2016;61(9):504–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (75).Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure J Gen Intern Med. 2001;16:606–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (76).Shim RS, Baltrus P, Ye J, Rust G. Prevalence, treatment, and control of depressive symptoms in the United States: results from the National Health and Nutrition Examination Survey (NHANES), 2005–2008. J Am Board Fam Med. 2011;24(1):33–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (77).Health Quality Ontario. Psychotherapy for major depressive disorder and generalized anxiety disorder: a health technology assessment. Ont Health Technol Assess Ser. 2017;17(15):1–167. [PMC free article] [PubMed] [Google Scholar]
- (78).Chan SW, Adams M. Service use, drop-out rate and clinical outcomes: a comparison between high and low intensity treatments in an IAPT service. Behav Cogn Psychother. 2014;42(6):747–59. [DOI] [PubMed] [Google Scholar]
- (79).Clark DM, Canvin L, Green J, Layard R, Pilling S, Janecka M. Transparency about the outcomes of mental health services (IAPT approach): an analysis of public data. Lancet. 2018;391(10121):679–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (80).Clark DM. Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: the IAPT experience. Int Rev Psychiatry. 2011;23(4):318–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (81).Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010;40(12):1943–57. [DOI] [PubMed] [Google Scholar]
- (82).Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014;13(3):288–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (83).Cuijpers P, van Straten A, Andersson G. Internet-administered cognitive behavior therapy for health problems: a systematic review. J Behav Med. 2008;31(2):169–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (84).Cuijpers P, van Straten A, Andersson G, van Oppen P. Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol. 2008;76(6):909–22. [DOI] [PubMed] [Google Scholar]
- (85).Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Rev Neurother. 2012;12(7):861–70. [DOI] [PubMed] [Google Scholar]
- (86).Clark DM. Overview of the UK Improving Access to Psychological Therapy (IAPT) program. Centre for Addiction and Mental Health; 2018. [Google Scholar]
- (87).Richards D, Duffy D, Blackburn B, Earley C, Enrique A, Palacios J, et al. Digital IAPT: the effectiveness & cost-effectiveness of internet-delivered interventions for depression and anxiety disorders in the Improving Access to Psychological Therapies programme: study protocol for a randomised control trial. BMC Psychiatry. 2018;18(1):59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (88).Canadian Psychiatric Association. Clinical practice guidelines. Management of anxiety disorders. Can J Psychiatry. 2006;51(8 Suppl):9S–91S. [PubMed] [Google Scholar]
- (89).Excellence NIfHaC. Generalised anxiety disorder and panic disorder in adults: management [Internet]. London (UK): National Institute for Health and Care Excellence; 2011. [cited 2018 Apr 07]. Available from: https://www.nice.org.uk/guidance/cg113/resources/generalised-anxiety-disorder-and-panic-disorder-in-adults-management-pdf-35109387756997 [PubMed] [Google Scholar]
- (90).Crome E, Shaw J, Baillie A. Costs and returns on training investment for empirically supported psychological interventions. Aust Health Rev. 2017;41(1):82–8. [DOI] [PubMed] [Google Scholar]
- (91).Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry. 2017;74(4):351–9. [DOI] [PubMed] [Google Scholar]
- (92).van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, et al. Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One. 2014;9(7):e100674. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (93).Koeser L, Donisi V, Goldberg DP, McCrone P. Modelling the cost-effectiveness of pharmacotherapy compared with cognitive-behavioural therapy and combination therapy for the treatment of moderate to severe depression in the UK. Psychol Med. 2015;45(14):3019–31. [DOI] [PubMed] [Google Scholar]
- (94).Machado M, Iskedjian M, Ruiz I, Einarson TR. Remission, dropouts, and adverse drug reaction rates in major depressive disorder: a meta-analysis of head-to-head trials. Curr Med Res Opin. 2006;22(9):1825–37. [DOI] [PubMed] [Google Scholar]
- (95).Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. 1st ed. Chichester (UK): John Wiley & Sons Ltd.; 2009. Apr 27. [Google Scholar]
- (96).Biesheuvel-Leliefeld KE, Kok GD, Bockting CL, Cuijpers P, Hollon SD, van Marwijk HW, et al. Effectiveness of psychological interventions in preventing recurrence of depressive disorder: meta-analysis and meta-regression. J Affect Disord. 2015;174:400–10. [DOI] [PubMed] [Google Scholar]
- (97).Churchill R, Hunot V, Corney R, Knapp M, McGuire H, Tylee A, et al. A systematic review of controlled trials of the effectiveness and cost-effectiveness of brief psychological treatments for depression. Health Technol Assess. 2001;5(35):1–173. [DOI] [PubMed] [Google Scholar]
- (98).Cipriani A, Geddes JR. Placebo for depression: we need to improve the quality of scientific information but also reject too simplistic approaches or ideological nihilism. BMC Med. 2014;12:105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (99).Drummond MF, Sculpher MJT, Sculpher G.W., O'Brien BJ, Stoddart GL. Methods for the economic evaluation of health care programmes. 3rd ed. New York: Oxford University Press; 2005. [Google Scholar]
- (100).Glanville J, Arber M, Veale T, Garcia S. Sensitivity of a search filter designed to identify studies reporting health state utility values. 116th Annual Meeting, Medical Library Association, Inc; 2016. May 15–20; Toronto, ON: Journal of the Medical Library Association. [Google Scholar]
- (101).Glick HA, Doschi JA, Sonnad SS. Economic evaluations in clinical trials 2nd ed. Oxford (UK): Oxford University Press; 2007. [Google Scholar]
- (102).Schaffer A, Levitt AJ, Hershkop SK, Oh P, MacDonald C, Lanctot K. Utility scores of symptom profiles in major depression. Psychiatry Res. 2002;110(2):189–97. [DOI] [PubMed] [Google Scholar]
- (103).King M, Sibbald B, Ward E, Bower P, Lloyd M, Gabbay M, et al. Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care. Health Technol Assess. 2000;4(19):1–83. [PubMed] [Google Scholar]
- (104).Hedman E, Andersson G, Andersson E, Ljotsson B, Ruck C, Asmundson GJ, et al. Internet-based cognitive-behavioural therapy for severe health anxiety: randomised controlled trial. Br J Psychiatry. 2011;198(3):230–6. [DOI] [PubMed] [Google Scholar]
- (105).Lenert LA, Sherbourne CD, Sugar C, Wells KB. Estimation of utilities for the effects of depression from the SF-12. Med Care. 2000;38(7):763–70. [DOI] [PubMed] [Google Scholar]
- (106).Gidwani RA. A budget impact analysis of chronic disease screening: the business care of rapid HIV testing in VA emergency departments. Ann Arbor (MI): UMI Dissertation Publishing; 2011. 160 p. [Google Scholar]
- (107).Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, Raffield T, et al. Computerassisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry. 2005;162(6):1158–64. [DOI] [PubMed] [Google Scholar]
- (108).Cornish P. Innovating online with stepped care: a research-based stepped care mental health community of practice proposal [Internet]. Palo Alto (CA): Issuu Inc.; 2014. [cited 2018 Apr 6]. Available from: https://issuu.com/cacuss/docs/communique_fall_2014_-_web/9 [Google Scholar]
- (109).Sutherland G, Dinh T. Understanding the gap: a pan-Canadian analysis of prescription drug insurance coverage [Internet]. Ottawa (ON): Conference Board of Canada; 2017. [cited 2018 Apr 6]. Available from: http://innovativemedicines.ca/wp-content/uploads/2017/12/20170712-understanding-the-gap.pdf [Google Scholar]
- (110).Ministry of Health and Long-Term Care. Schedule of benefits: physician services under the Health Insurance Act [Internet]. Toronto (ON): The Ministry; 2015. [cited 2017 Oct 11]. Available from: http://www.health.gov.on.ca/en/pro/programs/ohip/sob/physserv/sob_master20160401.pdf [Google Scholar]
- (111).Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One. 2010;5(6):e10939. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (112).Kolber MC, Lee J, Nickonchuk T. Price comparison of commonly prescribed pharmaceuticals in Alberta 2014. [Internet]. Edmonton (AB): Alberta College of Family Physicians; 2014. [cited 2017 Apr 4]. Available from: https://www.acfp.ca/wp-content/uploads/2014/06/ACFPPricingDoc2014.pdf [Google Scholar]
- (113).Statistics Canada. Table 14-10-0320-02: Average usual hours and wages by selected characteristics, monthly, unadjusted for seasonality (x 1,000) [Internet]. Ottawa (ON): Statistics Canada; 2018. [cited 2018 Oct 2]. Available from: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1410032002 [Google Scholar]
- (114).Ministry of Labour. Minimum wage [Internet]. Toronto (ON): Queen's Printer for Ontario; 2017. [cited 2018 Oct 2]. Available from: https://www.ontario.ca/document/your-guide-employment-standards-act-0/minimum-wage#section-0 [Google Scholar]
- (115).Clark DM. Realizing the mass public benefit of evidence-based psychological therapies: the IAPT program. Annu Rev Clin Psychol. 2018;14:159–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (116).Serhal E, Crawford A, Cheng J, Kurdyak P. Implementation and utilisation of telepsychiatry in Ontario: a population-based study. Can J Psychiatry. 2017;62(10):716–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (117).Ahmad F, Shakya Y, Ginsburg L, Lou W, Ng PT, Rashid M, et al. Burden of common mental disorders in a community health centre sample. Can Fam Physician. 2016;62(12):e758–e66. [PMC free article] [PubMed] [Google Scholar]
- (118).Alavi N, Hirji A, Sutton C, Naeem F. Online CBT is effective in overcoming cultural and language barriers in patients with depression. J Psychiatr Pract. 2016;22(1):2–8. [DOI] [PubMed] [Google Scholar]
- (119).Sullivan SD, Mauskopf JA, Augustovski F, Jaime Caro J, Lee KM, Minchin M, et al. Budget impact analysis-principles of good practice: report of the ISPOR 2012. Budget Impact Analysis Good Practice II Task Force. Value Health. 2014;17(1):5–14. [DOI] [PubMed] [Google Scholar]
- (120).Patten SB, Lee RC. Describing the longitudinal course of major depression using markov models: data integration across three national surveys. Popul Health Metr. 2005;3:11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (121).Patten SB, Williams JV, Lavorato DH, Fiest KM, Bulloch AG, Wang J. The prevalence of major depression is not changing. Can J Psychiatry. 2015;60(1):31–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (122).Patten SB, Williams JV, Lavorato DH, Wang JL, McDonald K, Bulloch AG. Descriptive epidemiology of major depressive disorder in Canada in 2012. Can J Psychiatry. 2015;60(1):23–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (123).Patten SB, Williams JV, Lavorato DH, Wang JL, McDonald K, Bulloch AG. Major depression in Canada: what has changed over the past 10 years? Can J Psychiatry. 2016;61(2):80–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (124).Tempier R, Meadows GN, Vasiliadis HM, Mosier KE, Lesage A, Stiller A, et al. Mental disorders and mental health care in Canada and Australia: comparative epidemiological findings. Soc Psychiatry Psychiatr Epidemiol. 2009;44(1):63–72. [DOI] [PubMed] [Google Scholar]
- (125).Morgan VA, Leonard H, Bourke J, Jablensky A. Intellectual disability co-occurring with schizophrenia and other psychiatric illness: population-based study. Br J Psychiatry. 2008;193(5):364–72. [DOI] [PubMed] [Google Scholar]
- (126).Wozney L, McGrath P, Newton A, Hartling L, Curran J, Huguet A, et al. RE-AIMing emental health: a rapid review of current research [Internet]. Ottawa (ON): Mental Health Commision of Canada; 2017. [cited 2018 Apr 4]. Available from: https://www.mentalhealthcommission.ca/sites/default/files/2017-08/eMH%20Literature%20Review_FINAL%20EN.pdf [Google Scholar]
- (127).Eaton WW, Shao H, Nestadt G, Lee HB, Bienvenu OJ, Zandi P. Population-based study of first onset and chronicity in major depressive disorder. Arch Gen Psychiatry. 2008;65(5):513–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (128).Casey L, Wright M-A, Clough BA. Comparison of perceived barriers and treatment preferences associated with internet-based and face-to-face psychological treatment of depression. Int J Cyber Behav Psychol Learning. 2014;4(4):16–22. [Google Scholar]
- (129).Khatri N, Marziali E, Tchernikov I, Shepherd N. Comparing telehealth-based and clinic-based group cognitive behavioral therapy for adults with depression and anxiety: a pilot study. Clin Interv Aging. 2014;9:765–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (130).Williams C, McClay CA, Martinez R, Morrison J, Haig C, Jones R, et al. Online CBT life skills programme for low mood and anxiety: study protocol for a pilot randomized controlled trial. Trials. 2016;17(1):220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (131).Barham L. Public and patient involvement at the UK National Institute for Health and Clinical Excellence. Patient. 2011;4(1):1–10. [DOI] [PubMed] [Google Scholar]
- (132).Messina J, Grainger DL. A pilot study to identify areas for further improvements in patient and public involvement in health technology assessments for medicines. Patient. 2012;5(3):199–211. [DOI] [PubMed] [Google Scholar]
- (133).Ontario Health Technology Advisory Committee Public Engagement Subcommittee. Public engagement for health technology assessment at Health Quality Ontario—final report from the Ontario Health Technology Advisory Committee Public Engagement Subcommittee [Internet]. Toronto (ON): Queen's Printer for Ontario; 2015 Apr. [cited 2018 Apr 30]. Available from: http://www.hqontario.ca/Portals/0/documents/evidence/special-reports/report-subcommittee-20150407-en.pdf [Google Scholar]
- (134).Kvale S. Interviews: an introduction to qualitative research interviewing. Thousand Oaks (CA): Sage; 1996. [Google Scholar]
- (135).Kuzel AJ. Sampling in qualitative inquiry. In: Miller WL, Crabtree BF, editors. Doing qualitative research. Thousand Oaks (CA): Sage; 1999. p. 33–45. [Google Scholar]
- (136).Health Technology Assessment International Interest Group on Patient and Citizen Involvement in HTA. Introduction to health technology assessment [Internet]. Edmonton (AB): Health Technology Assessment International; 2015. [cited 2016 Jan]. Available from: http://www.htai.org/fileadmin/HTAi_Files/ISG/PatientInvolvement/v2_files/Resource/PCISG-Resource-Intro_to_HTA__KFacey__Jun13.pdf [Google Scholar]
- (137).Strauss AL, Corbin JM. Grounded theory research: procedures, canons, and evaluative criteria. Qual Sociol. 1990;13(1):3–21. [Google Scholar]
- (138).Strauss AL, Corbin JM. Grounded theory methodology: an overview. In: Denzin NK, Lincoln YS, editors. Handbook of qualitative research. Thousand Oaks (CA): Sage; 1994. p. 273–85. [Google Scholar]
- (139).Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018;47(1):1–18. [DOI] [PubMed] [Google Scholar]
- (140).Olthuis JV, Watt MC, Bailey K, Hayden JA, Stewart SH. Therapist-supported internet cognitive behavioural therapy for anxiety disorders in adults. Cochrane Database Syst Rev. 2016(3):CD011565. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (141).Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on internet-based mental health interventions — a systematic review. Internet Interventions. 2014;1(4):205–15. [Google Scholar]
- (142).Patten SC. The duration of major depressive episodes in the Canadian general population. Chronic Dis Can. 2001;22(1):6–11. [PubMed] [Google Scholar]
- (143).Patten SB. Recurrence risk in major depression. Depress Anxiety. 2013;30(1):1–4. [DOI] [PubMed] [Google Scholar]
- (144).Khan A, Schwartz K. Suicide risk and symptom reduction in patients assigned to placebo in duloxetine and escitalopram clinical trials: analysis of the FDA summary basis of approval reports. Ann Clin Psychiatry. 2007;19(1):31–6. [DOI] [PubMed] [Google Scholar]
- (145).Khan A, Khan S, Kolts R, Brown WA. Suicide rates in clinical trials of SSRIs, other antidepressants, and placebo: analysis of FDA reports. Am J Psychiatry. 2003;160(4):790–2. [DOI] [PubMed] [Google Scholar]
- (146).Ferrari AJ, Charlson FJ, Norman RE, Flaxman AD, Patten SB, Vos T, et al. The epidemiological modelling of major depressive disorder: application for the Global Burden of Disease study 2010. PLoS One. 2013;8(7):e69637. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (147).Hardeveld F, Spijker J, De Graaf R, Nolen WA, Beekman AT. Recurrence of major depressive disorder and its predictors in the general population: results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Psychol Med. 2013;43(1):39–48. [DOI] [PubMed] [Google Scholar]
- (148).Statistics Canada. Life tables, Canada, provinces and territories (2009 to 2011, no. 5) [Internet]. Ottawa (ON): Statistics Canada; 2011. Available from: http://www.statcan.gc.ca/access_acces/alternative_alternatif.action?loc=http://www.statcan.gc.ca/pub/84-537-x/2013005/tbl_2009_2011-eng.xlsx [Google Scholar]
- (149).Mueller TI, Leon AC, Keller MB, Solomon DA, Endicott J, Coryell W, et al. Recurrence after recovery from major depressive disorder during 15 years of observational follow-up. Am J Psychiatry. 1999;156(7):1000–6. [DOI] [PubMed] [Google Scholar]
- (150).Amick HR, Gartlehner G, Gaynes BN, Forneris C, Asher GN, Morgan LC, et al. Comparative benefits and harms of second generation antidepressants and cognitive behavioral therapies in initial treatment of major depressive disorder: systematic review and meta-analysis. BMJ. 2015;351:h6019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (151).Cipriani A, Furukawa TA, Salanti G, Geddes JR, Higgins JP, Churchill R, et al. Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet. 2009;373(9665):746–58. [DOI] [PubMed] [Google Scholar]
- (152).Williams N, Simpson AN, Simpson K, Nahas Z. Relapse rates with long-term antidepressant drug therapy: a meta-analysis. Hum Psychopharmacol. 2009;24(5):401–8. [DOI] [PubMed] [Google Scholar]
- (153).Mohiuddin S, Payne K. Utility values for adults with unipolar depression: systematic review and meta-analysis. Med Decis Making. 2014;34(5):666–85. [DOI] [PubMed] [Google Scholar]
- (154).Revicki DA, Wood M. Patient-assigned health state utilities for depression-related outcomes: differences by depression severity and antidepressant medications. J Affect Disord. 1998;48(1):25–36. [DOI] [PubMed] [Google Scholar]
- (155).Mann R, Gilbody S, Richards D. Putting the ‘Q’ in depression QALYs: a comparison of utility measurement using EQ-5D and SF-6D health related quality of life measures. Soc Psychiatry Psychiatr Epidemiol. 2009;44(7):569–78. [DOI] [PubMed] [Google Scholar]
- (156).Vallejo-Torres L, Castilla I, Gonzalez N, Hunter R, Serrano-Perez P, Perestelo-Perez L. Cost-effectiveness of electroconvulsive therapy compared to repetitive transcranial magnetic stimulation for treatment-resistant severe depression: a decision model. Psychol Med. 2015;45(7):1459–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (157).Kates N, McPherson-Doe C, George L. Integrating mental health services within primary care settings: the Hamilton Family Health Team. J Ambul Care Manage. 2011;34(2):174–82. [DOI] [PubMed] [Google Scholar]
- (158).Rosser W, Godwin M, Seguin R. Family medicine research capacity building: five-weekend programs in Ontario. Can Fam Physician. 2010;56(3):e94–e100. [PMC free article] [PubMed] [Google Scholar]
- (159).Rosser WW, Colwill JM, Kasperski J, Wilson L. Progress of Ontario's Family Health Team model: a patient-centered medical home. Ann Fam Med. 2011;9(2):165–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- (160).Association of Family Health Teams of Ontario, Association of Ontario Health Centres, Nurse Practitioners’ Association of Ontario. Toward a primary care recruitment and retention strategy for Ontario: compensation structure for Ontario's interprofessional primary care organizations2014 [cited 2018. Apr 04]:[12 p.].
- (161).Health Quality Ontario. Repetitive transcranial magnetic stimulation for treatment-resistant depression: a systematic review and meta-analysis of randomized controlled trials. Ont Health Technol Assess Ser. 2016;16(5):1–66. [PMC free article] [PubMed] [Google Scholar]
- (162).Bereza BG, Machado M, Papadimitropoulos M, Sproule B, Ravindran AV, Einarson TR. A markov model approach assessing the cost of illness of generalized anxiety disorder in Canada. Neurol Ther. 2012;1(1):1. [DOI] [PMC free article] [PubMed] [Google Scholar]