Abstract
Background:
Internet-based interventions show clinical effectiveness for treating anxiety disorders and depression and could make mental healthcare more affordable.
Methods:
We searched databases including PubMed; EMBASE; Cochrane Central; PsychINFO; CINAHL; EconLit; and Web of Science from January 1, 2000 to August 21, 2020. Inclusion criteria were: 1) pertained to the treatment or prevention of anxiety disorders or depression; 2) evaluated the use of an internet-delivered psychological intervention; 3) recruited participants; and 4) reported costs or cost-effectiveness.
Results:
Of the 6,069 articles identified, 33 targeted anxiety (N=13) and depression (n=20) and met final inclusion criteria. All studies were from high-income countries. The control conditions and cost components included were heterogeneous. Only eight studies reported costs of developing the intervention. Of 27 studies that made a conclusion about cost-effectiveness, 81% of interventions were cost-effective. The quality of studies included was high based on a quality assessment checklist of economic evaluations, although many studies did not include definitions of cost components or differentiate between patient-side and system-level costs.
Limitations:
Studies varied in methodology, making conclusions about cost-effectiveness difficult. The generalizability of these results is unclear as studies were clustered in a small number of high-income countries and costs vary over time and between regions.
Conclusions:
Internet-delivered interventions appeared to be cost-effective although control conditions and cost component reporting were variable. We propose a checklist of cost components for future cost analyses to better compare intervention costs. More research is needed to describe development costs, cost-effectiveness in low-resource settings, and cost-effectiveness of newer technologies.
Introduction
Globally, mental disorders are a leading cause of disability (Vigo et al., 2016), with depression and anxiety disorders accounting for the greatest burden of disease within mental disorders (Whiteford et al., 2013). Common mental disorders also cost the world approximately US$1.15 trillion per year (Chisholm et al., 2016), and healthcare spending around the world continues to increase (Xu et al., 2018). Healthcare systems in most countries are unable to deliver adequate mental health services due to workforce shortages, few available mental health professionals, high costs of existing evidence-based treatments, and barriers to accessing care for patients including long wait-times and stigma (Kazdin, 2017; Patel et al., 2018). In recent years, digital technologies have emerged as a highly promising collection of approaches for supporting the delivery of mental health care across a wide range of settings, and there is growing evidence supporting the use of these technologies for delivering treatments for depression and anxiety disorders (Arnberg et al., 2014).
Among different types of digital interventions, internet interventions are increasingly popular given growing internet access and use around the world. Multiple reviews and meta-analyses have shown that internet interventions are effective in reducing symptoms of anxiety and depression (Karyotaki et al., 2017; Newby et al., 2015; Păsărelu et al., 2017; Richards and Richardson, 2012).
Despite mounting evidence supporting the clinical effectiveness of internet interventions for anxiety and depression, less is known about related costs or their cost-effectiveness. Knowledge about economic evaluation of internet interventions is crucial for informing provider organizations, health systems, and policy makers on adopting effective internet interventions for treating anxiety and depression. Insufficient cost data has been cited as a major barrier to the successful uptake of digital mental health interventions (Gehring et al., 2017; Lennon et al., 2017), and perceived lower costs are noted as an incentive for stakeholders to implement digital mental health programs (Topooco et al., 2017). Some literature has suggested that internet interventions for mental health conditions are cost-effective (Baumann et al., 2020; Donker et al., 2015; Ophuis et al., 2017; Paganini et al., 2018), but prior reviews have been limited by few available studies with rigorous economic evaluations (Ahern et al., 2018; Arnberg et al., 2014; Hedman et al., 2012; Kolovos et al., 2018). Additionally, these reviews have been largely narrative, with little reporting of average costs of interventions or incremental cost-effectiveness ratios (ICERs). There has also been less attention given to the cost components included in cost analyses and differentiating between system-level and patient-level costs. Given the differences in costs of resources between settings, more detailed knowledge of the cost components required for intervention implementation may be useful for anyone interested in replicating or adapting an intervention in a new setting. Lastly, discussion has been limited surrounding the costs of developing the interventions, which is important when dealing with new and highly adaptable interventions such as internet interventions.
The purpose of this systematic review was to examine the literature on economic evaluations of internet interventions for anxiety disorders and depression, with a focus on presenting costs of developing and delivering the interventions and their components, and the cost-effectiveness of the interventions. Anxiety and depression were the focus of this review as they are prevalent, significant contributors to the global burden of disease, and well-studied.
Methods
Search Strategy
In this systematic review, we followed the PRISMA reporting guidelines (see Checklist in the Appendix) and registered our protocol on the PROSPERO register of systematic reviews (CRD42015027179). We searched the following databases from January 1, 2000 to March 16, 2018: PubMed (MEDLINE); EMBASE; Cochrane Central; PsychINFO (EBSCOhost); CINAHL; EconLit; Web of Science; Health Economic Evaluations Database (HEED); National Health Service Economic Evaluation Database (NHSEED); Cost-Effectiveness Analysis Registry; Research Papers in Economics (RcPEc); European Network of Health Economic Evaluation Database (EURONHEED). We then updated our search from March 16, 2018 to August 21, 2020. The start date of January 1, 2000 was selected because based on several systematic reviews of the literature, no published studies using digital technology for mental health, including anxiety, depression, and other mental disorders were identified before 2001 (Ahern et al., 2018; Firth et al., 2017a; Firth et al., 2017b; Hedman et al., 2012; Naslund et al., 2017; Naslund et al., 2015).
We combined search terms for “mental disorders” and “technology” and “costs, cost- effectiveness, or economic analysis/evaluation”. Each term was entered as a key word and corresponding medical subject heading (MeSH) term. MeSH terms refer to the United States Library of Medicine’s controlled vocabulary thesaurus and are primarily used for indexing articles in Medline. Combining MeSH terms with general free text search terms is important in order to identify as many relevant records as possible (Higgins et al., 2019). No language limits were applied. The complete search strategy used in Medline is shown in Figure 1, and the search terms in Table 1. We also searched reference lists of included studies, prior systematic reviews, and Google Scholar to identify additional relevant studies.
Figure 1.
Flow diagram of studies included in the systematic review
Table 1.
Key terms used in Medline search
| Search | Search Terms |
|---|---|
| #1 (Mental disorders) | “serious mental illness” OR “serious and persistent mental illness” OR “severe mental illness” OR “mental illness” OR “mental health” OR “mental disorder” OR “schizophrenia” OR “bipolar disorder” OR “schizoaffective disorder” OR “major depressive disorder” OR “depression” OR “anxiety” OR “affective disorder” OR “psychotic disorders” OR “psychosis” OR “post-traumatic stress disorder” OR “PTSD” OR “stress disorder” OR “antipsychotics” OR “Dementia” OR “Epilepsy” OR “mental disorders”[Mesh Terms] OR “Anxiety Disorders”[Mesh] OR “Stress Disorders, Traumatic”[Mesh] OR “Mood Disorders”[Mesh] OR “Depressive Disorder”[Mesh] OR “Schizophrenia and Disorders with Psychotic Features”[Mesh] OR “Psychotic Disorders”[Mesh] OR “Dementia”[Mesh] OR “Epilepsy”[Mesh] |
| #2 (Digital technology) | “Telemetry” OR “Telemedicine” OR “Telepsychiatry” OR “Telehealth” OR “Telecare” OR “Tele mental health” OR “Connected Health” OR “Internet” OR “Internet health” OR “Web Browser” OR “Website” OR “Web-based” OR “Social Media” OR “Facebook” OR “Mobile health” OR “Mobile technology” OR “Mobile phone” OR “Cellular Phone” OR “Cellphone” OR “Smartphone” OR “text message” OR “Text Messaging” OR “Wireless Technology” OR “Remote Sensing Technology” OR “Remote consultation” OR “big data” OR “Telemetry”[Mesh] OR “Telemedicine”[Mesh] OR “Internet”[Mesh] OR “Web Browser”[Mesh] OR “Social Media”[Mesh] OR “Cellular Phone”[Mesh] OR “Text Messaging”[Mesh] OR “Wireless Technology”[Mesh] OR “Remote Sensing Technology”[Mesh] |
| #3 (Costs, cost-effectiveness, and economic evaluation) | “cost” OR “cost-effectiveness” OR “cost-benefit” OR “cost-minimization” OR “cost-utility” OR “economic” OR “finance” OR “pricing” OR “spending” OR “expenditure” OR “Costs and Cost Analysis”[Mesh] OR “Cost-Benefit Analysis”[Mesh] OR “Economics”[Mesh] OR “Fees and Charges”[Mesh] |
| #4 (Final Search) | #1 AND #2 AND #3 |
Mesh indicates Medical Subject Heading
Study Selection Criteria
Our original search criteria included all studies that pertained to mental disorders and involved any form of digital technology. Because of the large number of studies that met these criteria, we decided to focus on only studies evaluating internet interventions for anxiety disorders or depression. We thus only included studies that met the following eligibility criteria: 1) pertained to interventions for anxiety disorders or depression, including anxiety or depressive symptoms; 2) evaluated the use of an internet-delivered intervention; 3) recruited participants; and 4) assessed the costs of an internet intervention. We defined internet interventions as those using any internet or web-enabled platform for diagnosis, screening, treatment, prevention, training, education, or facilitating self-management of mental disorders. All types of studies were included, so long as the study included primary data collected from participants. Thus, modeling studies using secondary data were excluded. Participants consisted of individuals living with mental disorders. Costs could be related to intervention development, delivery, evaluation, or maintenance, as well as out-of-pocket expenses or indirect costs incurred by patients related to seeking mental health care. Commentaries, opinion pieces, editorials, or perspective articles were not included. Reference lists of relevant meta-analyses and review articles were searched for articles that met the inclusion criteria.
Data Extraction and Analysis
After the databases were searched, all entries were imported into reference management software. Duplicate entries were removed. Two members of the review team independently screened titles and abstracts of remaining entries to identify studies that met inclusion criteria. All entries deemed as not meeting inclusion criteria were reviewed by another member of the review team to confirm exclusion. An additional member of our team reviewed the final list of full text articles that met the inclusion criteria. The final list of included studies was then circulated to other authors not directly involved in these preliminary steps. We then retrieved full-text articles for all studies in this final list. Full text articles were also retrieved for any citations for which eligibility could not be determined. Any abstracts for which a full text article was not available were excluded.
Two researchers then extracted the following data from the included studies: publication year; country where the study was conducted; year the study was conducted; study design and duration; mental illness studied; participant characteristics (e.g., number, age, gender); description of the internet intervention and control condition; cost components; cost related outcomes (development costs, patient-level and system-level costs); and cost-effectiveness outcomes (incremental cost-effectiveness ratios, willingness-to-pay thresholds). These details were summarized into a table, and the full text articles and summary tables were sent to additional members of the review team to confirm accuracy in data extraction and that the details of included studies were objectively reported. In many cases, results from a single study are published as multiple manuscripts, such as reporting of secondary outcomes. We did not count these as unique studies but rather used the secondary analyses to supplement data extraction from the primary study.
All numerical costs extracted from the studies were converted to 2015 United States dollars (USD). We first converted any foreign currency into USD based on the World Bank exchange rate for the given year (The World Bank, 2020b). Then, we converted those USD to 2015 USD with the World Bank GDP deflator (The World Bank, 2020a). If a currency year was not reported, we used the intervention year (or median intervention year) as a proxy for currency year; if intervention year was also not reported, we used the year of publication.
We did not perform a meta-analysis of quantitative findings and economic outcomes of those studies due to their heterogeneity, as summarized in the results section, in control conditions (usual care, another internet intervention, in-person treatment), cost components included, and timeframe of cost collection. Additionally, the Cochrane Collaborative does not recommend meta-analysis because there lacks consensus on best approaches for pooling combined estimates of cost data across economic evaluations with varied study designs and reported outcomes (Shemilt et al., 2008). We present as quantitative a summary as possible given these limitations.
Quality Assessment
We assessed quality of cost evaluation studies included in this review using a 35-item checklist for quality assessment of economic evaluations (Drummons and Jefferson, 1996). We selected this checklist because it has been widely used in prior systematic reviews of economic evaluations (Ahern et al., 2018; Donker et al., 2015) and is recommended by the Cochrane Collaboration (Ahern et al., 2018). An important advantage with this checklist is that it captures study characteristics related to quality, design, and risk of bias. The checklist is divided into 9 domains: study design (3 items); selection of alternatives (2 items); economic evaluation (2 items); effectiveness data (8 items); costing data (4 items); modeling (2 items); adjustment for timing of costs/benefits (4 items); uncertainty estimates (5 items); and presentation of results (5 items). Each item on the checklist has the following possible responses: “yes”, “no”, “not clear”, or “not applicable”. To calculate a quality score for each study based on the checklist, “yes” responses were scored as 1, “no” as 0, “not clear” as 0.5, and “not applicable” responses were excluded from the calculation (Gonzalez-Perez, 2002). The average of these responses resulted in a composite score between 0 and 1 for each study. Three authors reviewed the quality ratings.
Results
Our initial database search yielded 6,069 articles after removal of duplicates, of which 318 were relevant and required full-text review (Figure 1). We then updated this initial search by repeating our database search on August 21, 2020, which yielded an additional 13 relevant studies requiring full-text review. In total, 73 studies involved use of digital technologies for mental disorders, of which 35 were internet-delivered interventions targeting anxiety disorders or depression and met our final inclusion criteria. Three of these articles were reports of secondary outcomes and were included to supplement data extraction but not counted as unique studies. One study reported results from two separate trials (one involving patients with anxiety, and one involving patients with depression) and these two trials are reported as two separate studies.
Of the 33 unique studies, most (n=28; 85%) were from Europe, including Sweden (n=11; 33%), Netherlands (n=9; 27%), United Kingdom (n=5; 15%), Germany (n=2; 6%), and Spain (n=1; 3%), followed by Australia (n=4; 12%) and one study from the United States (n=1; 3%). All studies were published in English. The sample sizes ranged from 20 to 4448 participants with an average of 392 and median of 158. Thirteen studies (39%) focused on anxiety disorders, including health anxiety, obsessive-compulsive disorder (OCD), panic disorder, and social anxiety, and 18 studies (55%) focused on depression. One study (3%) enrolled patients with either anxiety or depression. Thirty studies (91%) were randomized controlled trials, with two pilot/feasibility studies (6%) and one observational study (3%).
Internet-delivered Psychological Interventions for Anxiety Disorders (N=13)
Intervention Characteristics and Effectiveness
Thirteen studies assessed costs of internet-delivered interventions for anxiety disorders (see Table 2). Twelve studies were RCTs, with one feasibility trial (Dear et al., 2015b). All thirteen interventions were internet-delivered Cognitive Behavioral Therapy (CBT) self-help courses. Two interventions were completely self-guided (Dear et al., 2015b; Powell et al., 2020), and one study evaluated an unguided internet intervention, a therapist-guided internet intervention, and a paper-based intervention (Axelsson et al., 2018). The remaining ten studies involved some form of regular remote contact with a therapist or coach. Comparison conditions included usual care/waitlist (n=5; 38%) (Andersson et al., 2015a; Axelsson et al., 2018; Dear et al., 2015a; Lenhard et al., 2017; Powell et al., 2020), in-person CBT (n=2; 15%) (Bergström et al., 2010; Hedman et al., 2011), an alternative internet-based intervention (n=1; 8%) (Hedman et al., 2016), and a less intensive internet-based supportive therapy (n=4; 31%) (Andersson et al., 2015b; Hedman et al., 2013; Jolstedt et al., 2018; Nordgren et al., 2014). All the interventions with usual care/waitlist, an alternative internet intervention, or a less-intensive internet-based therapy control condition were found to be clinically effective compared to control (n=10; 77%) (Andersson et al., 2015a; Andersson et al., 2015b; Axelsson et al., 2018; Dear et al., 2015a; Hedman et al., 2013; Hedman et al., 2016; Jolstedt et al., 2018; Lenhard et al., 2017; Nordgren et al., 2014; Powell et al., 2020). For the two interventions that were compared to in-person CBT, there was no significant difference in effectiveness between intervention and control (Bergström et al., 2010; Hedman et al., 2011).
Table 2.
Cost evaluations of internet-delivered psychological interventions for anxiety disorders*
| Study | Intervention country, year, and duration | Study design and sample | Internet intervention description | Control condition | Costs of developing intervention and components | Costs of delivering intervention and components | Total costs of intervention and components | Total costs of control and components | Measure of effectiveness | ICER | WTP Threshold |
|---|---|---|---|---|---|---|---|---|---|---|---|
| USUAL CARE/WAITLIST CONTROL CONDITION | |||||||||||
| (Andersson et al., 2015b) | Sweden; year not reported; 3 weeks | RCT; 93 patients with OCD | Online booster program added to Internet CBT 6 months after completion. Booster was same format as ICBT (with written self-help material, homework, and therapist guidance via email) with goal of preventing relapse of OCD symptoms | Usual care | Not reported | $249 per patient; Components: therapist time | $1089 per patient at 12-month follow up; Components: delivery costs, direct medical, healthcare visits, medications, direct non-medical, indirect non-medical, unemployment, sick leave, work loss, domestic | $786 per patient at 12-month follow up; Components: direct medical, healthcare visits, medications, direct non-medical, indirect non-medical, unemployment, sick leave, work loss, domestic | Number of avoided relapses | $1533 per avoided relapse | At WTP of $1030, 90% probability of cost-effectiveness |
| (Axelsson et al., 2018) | Sweden; 2013 - 2016; 12 weeks | RCT; 132 patients with severe health anxiety | Three self-help interventions: 1) G-ICBT (therapist-guided internet CBT), 2) U-ICBT (unguided internet CBT), 3) BIB-BCT (CBT in paper book form, no therapist guidance) | Waitlist (WL) | Not reported | G-ICBT: $1001.. U-ICBT: $364. BIB-CBT: $579 per patient; Components: therapist time, web platform or self-help book, patient time | G-ICBT: $602.. U-ICBT: $940. BIB-CBT: $846 per patient over 12-week intervention period; Components: healthcare visits, medication, direct non-medical costs, indirect costs (unemployment, sick leave, work cutback, domestic) | $1091 per patient over 12-week intervention period; Components: healthcare visits, medication, direct non-medical costs, indirect costs (unemployment, sick leave, work cutback, domestic) | (1) Cases of remission, (2) QALYs | G-ICBT vs WL: $692 per case of remission. U-ICBT v WL: −$223 per case of remission (lower costs, higher efficacy).. G-ICBT vs BIB-CBT: $902,164 per case of remission. U-ICBT vs BIB-CBT: −$22716 per case of remission (higher cost, lower efficacy) | At WTP of $6652 per remission, >95% probability of cost-effectiveness of both G-ICBT and U-ICBT compared to WL |
| (Dear et al., 2015a) | Australia; year not reported; 8 weeks | RCT; 72 adults aged 60 and over with anxiety | Managing Stress and Anxiety Course, an internet CBT course with case-enhanced learning, email contact with therapists, and goal to learn and adopt psychological skills for managing anxiety. | Waitlist | Not reported | Not reported separately; Components: therapist time, supervisor time, internet access, computer and phone use | $228 per participant over 8-week treatment period; Components: delivery costs, healthcare consultations and admissions, antidepressant and anxiolytic medications | $137 per participant over 8-week treatment period; Components: healthcare consultations and admissions, antidepressant and anxiolytic medications | QALYs | $8755 per QALY gained | At WTP of $49,709 per QALY, >95% probability of cost-effectiveness |
| (Lenhard et al., 2017) | Sweden; 2014;12 weeks | RCT; 67 patients with pediatric OCD | Parent-assisted internet CBT course. Contains text, videos, exercises, and therapist provides written feedback via secure internet portal and occasionally via phone. | Waitlist control | Not reported | $195 per patient; Components: clinician time and intervention platform maintenance costs | $970 per patient over 12-week intervention period; Components: delivery costs, healthcare use, medications, supportive resources, prescription-free drugs and supplements, school absence, academic production loss | $888 per patient over 12-week intervention period; Components: healthcare use, medications, supportive resources, prescription-free drugs and supplements, school absence, academic production loss | Number of treatment responders | $77 per responder from healthcare perspective (including costs of clinician time, maintenance costs, healthcare use, medications) | At WTP of $3957 per responder, 100% probability of cost-effectiveness |
| (Powell et al., 2020) | UK; 2016–2018; 6 weeks | RCT; 2166 patients with social anxiety symptoms | E-couch is a fully self-guided internet CBT program. Can be used as a mobile app or on computer browser. Received automated email reminders to complete the program | Waitlist | $65 per patient (including delivery costs); Components: developing, modifying, delivering, and maintaining the intervention | Not reported separately; Components: delivering and maintaining the intervention | $353 per patient over 6 weeks; Components: healthcare visits, work days lost to sick leave | $412 per patient over 6 weeks; Components: healthcare visits, work days lost to sick leave | QALYs | Intervention dominated the control. Not including development costs | Not reported |
| OTHER CONTROL CONDITIONS | |||||||||||
| (Andersson et al., 2015a) | Sweden; year not reported; 10 weeks | RCT; 101 patients with OCD | Internet CBT with written self-help material, homework, and therapist guidance via email. Includes self-help modules, psychoeducation, cognitive restructuring, exposure with response prevention, and relapse prevention. | Online non-directive supportive therapy with email communication with a therapist. | Not reported | $436 per patient; Components: therapist time | $1642 per patient over 10-week intervention period; Components: delivery costs, direct medical, healthcare visits, medications, direct non-medical, indirect non-medical, unemployment, sick leave, work loss, domestic | $1125 per patient over 10-week intervention period; Components: therapist time, direct medical, healthcare visits, medications, direct non-medical, indirect non-medical, unemployment, sick leave, work loss, domestic | (1) Number of remissions, (2) QALYs | (1) $959 per additional remission, (2) $7400 per QALY gained | At WTP of $2678 per remission and $26,775 per QALY, 90% probability of cost-effectiveness |
| (Bergström et al., 2010) | Sweden; year not reported; 10 weeks | RCT; 113 patients with panic disorders | Internet CBT course with homework, based on psychoeducation, cognitive restructuring, interceptive exposure, exposure in-vivo, and relapse prevention. Included email messaging with therapists and online discussion forum with other patients in treatment | Group CBT led by two clinical psychologists | Not reported | $124 per patient; Components: therapist time | $124 per patient over 10-week intervention period; Components: delivery costs | $470 per patient over 10-week intervention period; Components: therapist time | Number of treatment responders | Not calculated | N/A |
| (Dear et al., 2015b) | Australia; year not reported; 8 weeks | Feasibility trial; 27 patients with anxiety | Managing Stress and Anxiety Course, an internet CBT courses with lessons, case-studies, and automated emails to help participants learn and practice core CBT skills | None | Not reported | Not reported separately; Components: resource use associated with the ICBT treatment | Difference in total cost from baseline to post-treatment (8 weeks): $69 more per patient; Components: delivery costs, healthcare consultations and admissions, antidepressant and anxiolytic medications | N/A | (1) Cases of clinical improvement, (2) QALYs | Not calculated | N/A |
| (Hedman et al., 2013) | Sweden; year not reported; 12 weeks | RCT; 81 patients with severe health anxiety | Internet CBT with self-help content delivered in 12 modules and access to an online therapist | 12-week online discussion forum where participants can discuss health anxiety | Estimated to be $657 per participant; Components: writing the treatment program, computer programming, and establishing the treatment unit | $354 per patient; Components: therapist time, participant time spent on intervention | $14,020 per patient over 12-week intervention period; Components: delivery costs, direct medical (healthcare visits and medication), direct non-medical, indirect (unemployment, sick leave, work cutback, domestic) | $13,380 per patient over 12-week intervention period; Components: direct medical (healthcare visits and medication), direct non-medical, indirect (unemployment, sick leave, work cutback, domestic) | (1) Number of remissions (2) QALYs | (1) -$2096 for each additional case of remission, (2) -$11,006 per QALY gained (not including development costs). Negative values indicated lower costs, higher efficacy | At WTP of $8423 per one case of remission, 96% probability of cost-effectiveness. At WTP of $8423 per QALY, 77% probability of cost-effectiveness |
| (Hedman et al., 2016) | Sweden; year not reported; 12 weeks | RCT; 158 patients with severe health anxiety | Internet exposure-based CBT course using mindfulness, self-help materials and an online therapist who provided email feedback and guidance. | Internet behavioral stress management course using relaxation and stress management strategies | Not reported | $588 per patient; Components: therapist time | $1655 per patient over 12-week intervention period; Components: delivery costs, direct medical (healthcare visits and medication), direct non-medical, indirect (unemployment, sick leave, work cutback, domestic) | $1380 per patient over 12-week intervention period; Components: therapist time, direct medical (healthcare visits and medication), direct non-medical, indirect (unemployment, sick leave, work cutback, domestic) | (1) Cases of clinically significant improvement, (2) QALYs | (1) $2280 per case of clinically significant improvement, (2) $10,298 per QALY gained | At WTP of $10,298 per case of clinically significant improvement, 91% probability of cost-effectiveness. No probability reported for QALYs |
| (Hedman et al., 2011) [Secondary studies: (El Alaoui et al., 2017; Hedman et al., 2014)] |
Sweden; year not reported; 15 weeks | RCT; 126 patients with social anxiety disorder | Internet CBT with 15 self-help modules and therapist guidance via email. Program emphasizes role of safety behaviors, cognitions, internal focus of attention | In-person cognitive behavioral group therapy (CBGT) with one individual session and 14 group sessions led by therapists | Estimated to be $662 per participant; Components: writing the treatment program, computer programming, and establishing the treatment unit | $512 per patient; Components: therapist time, participant time spent on intervention | $7278 per patient over 15-week intervention period; Components: delivery costs, direct medical (healthcare visits and medication), direct non-medical, and indirect (unemployment, sick leave, work cutback, domestic) | $9539 per patient over 15-week intervention period; Components: therapist time, participant time, direct medical (healthcare visits and medication), direct non-medical, and indirect (unemployment, sick leave, work cutback, domestic) | (1) Cases of clinical improvement, (2) QALYs | (1) -$7772 per incremental clinical improvement, (2) -$19,659 per QALY gained. Not including development costs. Negative values indicated lower costs, higher efficacy | At WTP of $3309 per case of improvement, 89% probability of cost-effectiveness. At WTP of $0 per QALY, 81% probability of cost-effectiveness |
| (Jolstedt et al., 2018) | Sweden; 2015–2016; 12 weeks | RCT; 131 children with pediatric anxiety disorders | Internet CBT (therapist-supported and parent-assisted). Therapist support consisted of messages, feedback, and encouragement. Parents also access to separate parent-directed modules learning about parental behavior | Internet delivered, therapist-guided child-directed play education for parents | Not reported | $174 per patient; Components: therapist time | $4591 per patient over 12-week intervention period; Components: delivery costs, healthcare visits, supportive resources, medication, dietary supplements, school absence, parental work absence, productivity losses from school and work | $5129 per patient over 12-week intervention period; Components: therapist time, healthcare visits, supportive resources, medication, dietary supplements, school absence, parental work absence, productivity losses from school and work | (1) Cases of remissions, (2) QALYs | (1) -$1573 per case of remission, (2) Not calculated. Negative values indicate lower costs, higher efficacy | At WTP of $5475 per remission, 100% probability of cost-effectiveness |
| (Nordgren et al., 2014) | Sweden; recruitment from 2010–2011; 10 weeks | RCT; 100 patients with anxiety disorder | Internet CBT with 7–10 weekly individually assigned self-help modules guided by online therapists. | Received weekly email questions about their wellbeing from their therapist | Estimated to be $641 per participant; Components: developing ICBT and establishing the treatment in a healthcare context | $541 per patient; Components: therapist time | $2418 per patient over 10-week intervention period; Components: delivery costs, direct medical (healthcare visits and medication), direct non-medical, indirect (unemployment, sick leave, work cutback, domestic) | $2292 per patient over 10-week intervention period; Components: therapist time, direct medical (healthcare visits and medication), direct non-medical, indirect (unemployment, sick leave, work cutback, domestic) | (1) Number of treatment responders, (2) QALYs | (1) -$1948 per treatment responder, (2) -$8034 per QALY. Not including development costs. Negative values indicated lower costs, higher efficacy | At WTP of $0 for one treatment responder and at WTP of$3204 per QALY, 95% probability of cost-effectiveness |
Anxiety disorders includes: OCD, panic disorders, health anxiety, and social phobia or social anxiety.
All costs displayed in 2015 USD
CBT: Cognitive Behavioral Therapy
ICER: Incremental Cost Effectiveness Ratio
OCD: Obsessive Compulsive Disorder
QALY: Quality adjusted life year
RCT: Randomized control trial WTP: Willingness-to-pay
Intervention costs
Ten studies, all from Sweden, reported a value for delivery costs separately from total costs, which ranged from $124 to $1001 per participant (mean $413; SD=$250) as seen in Figure 2 (Andersson et al., 2015a; Andersson et al., 2015b; Axelsson et al., 2018; Bergström et al., 2010; Hedman et al., 2013; Hedman et al., 2011; Hedman et al., 2016; Jolstedt et al., 2018; Lenhard et al., 2017; Nordgren et al., 2014). Many studies did not specify whether costs were borne by patients or by the system. Eleven studies reported total costs of the intervention during the intervention period (Andersson et al., 2015a; Axelsson et al., 2018; Bergström et al., 2010; Dear et al., 2015a; Hedman et al., 2013; Hedman et al., 2011; Hedman et al., 2016; Jolstedt et al., 2018; Lenhard et al., 2017; Nordgren et al., 2014; Powell et al., 2020). These total costs ranged from $124 to $14,020 per participant, with mean $2902 (SD=$4081). Control condition costs ranged from $137 to $13,380, with mean $3258 (SD=$4363). These eight intervention periods lasted between 6–15 weeks. One study only reported total costs at 12-month follow-up, for a total of $1089 per participant in the intervention and $786 in control over 12 months (Andersson et al., 2015b). Only one completely self-directed study reported delivery costs, which was $364 per participant (Axelsson et al., 2018).
Figure 2. Delivery costs.
Per participant costs. All dollar amounts in 2015 USD. [] indicates publication year if data collection year was not reported. Some studies reported two interventions; these are shown on separate lines.
Cost components
All thirteen studies reported components of the delivery costs. Of the delivery costs, all 11 interventions with therapist or coach support reported personnel time, five studies reported website-related costs (maintenance of the site, computer use, and/or internet access costs) (Axelsson et al., 2018; Dear et al., 2015a; Dear et al., 2015b; Lenhard et al., 2017; Powell et al., 2020), and two reported cost of participant time spent on the intervention (Axelsson et al., 2018; Hedman et al., 2011). For total costs, 12 studies included direct medical costs (including healthcare visits and medications), ten included indirect non-medical costs (Andersson et al., 2015a; Andersson et al., 2015b; Axelsson et al., 2018; Hedman et al., 2013; Hedman et al., 2011; Hedman et al., 2016; Jolstedt et al., 2018; Lenhard et al., 2017; Nordgren et al., 2014; Powell et al., 2020), and eight also included direct non-medical costs (Andersson et al., 2015a; Andersson et al., 2015b; Axelsson et al., 2018; Hedman et al., 2013; Hedman et al., 2011; Hedman et al., 2016; Lenhard et al., 2017; Nordgren et al., 2014). Direct non-medical costs were inconsistently defined across studies but generally included costs such as parking and transportation. Indirect non-medical costs included costs such as unemployment, sick leave, work cutback, and domestic work loss.
Cost-effectiveness
Among these 13 studies, 11 performed a cost-effectiveness analysis and calculated incremental cost-effectiveness ratios (ICER) (Andersson et al., 2015a; Andersson et al., 2015b; Axelsson et al., 2018; Dear et al., 2015a; Hedman et al., 2013; Hedman et al., 2011; Hedman et al., 2016; Jolstedt et al., 2018; Lenhard et al., 2017; Nordgren et al., 2014; Powell et al., 2020). No studies included development costs in their cost-effectiveness analyses. Of seven studies which used Quality Adjusted Life-Years (QALYs) as the measure of effectiveness, one ICER was dominated in favor of the intervention (Powell et al., 2020), and other ICERs ranged from −$19,659 per QALY to $10,298 per QALY (mean −$2041; SD=$12,527) as seen in Figure 3 (Andersson et al., 2015a; Dear et al., 2015a; Hedman et al., 2013; Hedman et al., 2011; Hedman et al., 2016; Nordgren et al., 2014). All negative ICERs were cases of lower costs and higher QALYs in the intervention group. We were unable to draw a conclusion about ICERs of studies with usual care/waitlist versus other control conditions as the studies with usual care/waitlist all reported ICERs with different units. Studies reported a variety of Willingness-to-Pay (WTP) thresholds (all are presented here in 2015 USD). Three studies had greater than 90% probability of cost-effectiveness at WTP thresholds of $3204 (Nordgren et al., 2014), $26,775 (Andersson et al., 2015a), and $49,709 per QALY (Dear et al., 2015a). One intervention had a 77% probability of cost-effectiveness at WTP threshold of $8423 per QALY (Hedman et al., 2013) and another had 81% probability at a WTP threshold of $0 per QALY (Hedman et al., 2011). Four studies reported ICERs with clinical outcomes, and ICERs were $1533 per avoided relapse (Andersson et al., 2015b), $77 per treatment responder (Lenhard et al., 2017), −$1573 per case of remission (Jolstedt et al., 2018). Axelsson et al compared two internet interventions (one guided, one unguided) with a waitlist control condition and with a paper booklet intervention. ICERs were $692 per case of remission (guided intervention vs control) (Axelsson et al., 2018), −$233 per case of remission (unguided intervention vs control; negative value indicates lower costs and higher efficacy) (Axelsson et al., 2018), $902,164 (guided vs paper booklet intervention), and −$22,716 (unguided vs paper booklet intervention; negative value indicates higher costs and lower efficacy). All 11 studies that performed a cost-effectiveness analysis concluded that the intervention was cost-effective compared to control.
Figure 3. Intervention ICERs.
Per QALY gained. All dollar amounts in 2015 USD. All negative ICERs indicate lower costs and greater QALYs in the intervention. [] indicates publication year if data collection year was not reported. *Indicated development costs included in ICER. Some studies reported two interventions; these are shown on separate lines.
Internet-delivered Psychological Interventions for Depression (N=20)
Intervention Characteristics and Effectiveness
Twenty studies, summarized in Table 3, evaluated costs of internet-delivered interventions for depression, including one study that enrolled subjects with either depression or anxiety (Kenter et al., 2015). Five studies enrolled subjects with depressive symptoms or subjects at risk of developing depression (Buntrock et al., 2017; Nobis et al., 2018; Phillips et al., 2014; Ruby et al., 2013; Wijnen et al., 2018). Eighteen studies were RCTs, while one study was a feasibility trial (Dear et al., 2015b) and another an observational study (Kenter et al., 2015). Seventeen interventions were internet-delivered self-help courses, two were blended interventions with both internet self-help and in-person sessions (Kenter et al., 2015; Kooistra et al., 2019), and one was therapist-delivered sessions via internet messaging (Hollinghurst et al., 2010). The majority of interventions were based on CBT (n=19; 95%) (Bolier et al., 2014; Brabyn et al., 2016; Buntrock et al., 2017; Dear et al., 2015b; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Kenter et al., 2015; Klein et al., 2018; Kooistra et al., 2019; Littlewood et al., 2015; Nobis et al., 2018; Phillips et al., 2014; Romero-Sanchiz et al., 2017; Ruby et al., 2013; Titov et al., 2015; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018) and problem-solving therapy (n=5; 25%) (Bolier et al., 2014; Buntrock et al., 2017; Geraedts et al., 2015; Kolovos et al., 2016; Warmerdam et al., 2010). Sixteen studies (80%) evaluated interventions that employed therapist or coach support, while three were completely self-guided (Bolier et al., 2014; Dear et al., 2015b; Wijnen et al., 2018), and one study evaluated both a guided and unguided intervention (Romero-Sanchiz et al., 2017).
Table 3.
Cost evaluations of internet-delivered psychological interventions for depression*
| Study | Intervention country, year, and duration | Study design and sample | Internet intervention description | Control condition | Costs of developing intervention and components | Costs of delivering intervention and components | Total costs of intervention and components | Total costs of control and components | Measure of effectivene ss | ICER | WTP Threshold |
|---|---|---|---|---|---|---|---|---|---|---|---|
| USUAL CARE/WAITLIST CONTROL CONDITION | |||||||||||
| (Bolier et al., 2014) | Netherlands; year not reported; 2 months | RCT; 284 patients with depression | “Psyfit” is an internet self-help intervention with no therapist support to reduce depressive symptoms. Uses principles derived from positive psychology, mindfulness, CBT, and problem-solving therapy. | Usual care waitlist control | Not reported | $85 per patient; Components: website maintenance, recruitment costs, participant time spent on intervention | $10,597 per patient at 6-month follow-up; Components: delivery costs, direct medical costs (healthcare services and medication), direct non-medical costs (traveling and parking expenses, patient time), absenteeism, presenteeism | $8334 per patient at 6-month follow-up; Components: direct medical costs (healthcare services and medication), direct non-medical costs (traveling and parking expenses, patient time), absenteeism, presenteeism | Number of treatment responders | $32,798 per treatment responder | At WTP of $46,153 per treatment responder, 61% probability of cost-effectiveness |
| (Geraedts et al., 2015) | Netherlands; 2011–2013; up to 7 weeks | RCT; 231 patients with depression | “Happy@Work” is an internet self-help intervention, including problem solving treatment, cognitive therapy, and guidelines to help prevent work-related stress. Participants received feedback on assignments from coaches. | Care as usual | Not reported | $318 per patient; Components: website hosting and maintenance and coaches’ time | $30,160 per patient over 12 months; Components: delivery costs, healthcare visits, hospital admissions, medications, domestic tasks, absenteeism, and presenteeism | $31,120 per patient over 12 months; Components: healthcare visits, hospital admissions, medications, domestic tasks, absenteeism, and presenteeism | (1) Decrease in depressive symptoms, (2) QALYs | (1) $423 per point decrease in depressive symptoms, (2) $717,530 per QALY | At WTP of $2693 per point decrease in depressive symptoms, 95% probability of cost-effectiveness. At any WTP per QALY, maximum probability of cost effectiveness is 62% |
| (Hollinghurst et al., 2010) | UK; recruitment 2005–2008; up to 4 months | RCT; patients with depression | Therapist delivered (via internet messaging), individualized internet CBT with 10 sessions | Usual care waitlist control | Not reported | $1118 per patient; Components: therapist and supervisor time | $1732 per patient over 8 months; Components: delivery costs, primary healthcare visits, secondary healthcare mental health visits, medications, social services use, private sector healthcare, over the counter medications, social and domestic help, travel costs, loss of earnings, lost productivity | $669 per patient over 8 months; Components: primary healthcare visits, secondary healthcare mental health visits, medications, social services use, private sector healthcare, over the counter medications, social and domestic help, travel costs, loss of earnings, lost productivity | (1) Cases of recovery, (2) QALYs | (1) $7998 per extra patient recovering, (2) $38,933 per QALY gained | At WTP of $68,013 per QALY, 71% probability of cost-effectiveness |
| (Holst et al., 2018) | Sweden; 2010 – 2014; 12 weeks | RCT; 90 patients with depression | Internet CBT with therapist support via email or telephone. Support concentrated on validation, encouragement, and reinforcing progress | Treatment as usual | Not reported | $1368 per patient; Components: intervention software cost, technical support, therapist time | $6876 per patient over 12 months; Components: delivery costs, general practitioner visits, therapist visits, phone counseling, medications, productivity loss, participant time, transportation expenses | $7260 per patient over 12 months; Components: general practitioner visits, therapist visits, phone counseling, medications, productivity loss, participant time, transportation expenses | (1) Change in BDI-II score, (2) QALYs | (1) $562 per point reduction on BDI-II scale, (2) $7368 per QALY | Not reported |
| (Klein et al., 2018) | Netherlands; 2010–2013; duration not specified | RCT; 264 patients with recurrent depression | Internet CBT with minimal therapist support over the phone and mood monitoring via text messaging | Treatment as usual | $98 per participant (including delivery costs); Components: therapist time and training, developing intervention, periodic software upgrades, server costs | Not reported separately; Components: therapist time and training, periodic software updates, server costs | $11,142 per participant over 24 months; Components: development costs, delivery costs, mental healthcare services, other healthcare visits, medications, informal care, home care, travel expenses, unpaid work, absenteeism, presenteeism | $9796 per participant over 24 months; Components: mental healthcare services, other healthcare visits, medications, informal care, home care, travel expenses, unpaid work, absenteeism, presenteeism | (1) Depression-free days, (2) QALYs | (1) $240 per depression-free day, (2) $309,918 per QALY. Includes development costs | At a WTP of $6714 per depression-free day, 65% probability of cost-effectiveness. At a WTP of $67,135 per QALY, 35% probability of cost-effectiveness |
| (Kolovos et al., 2016) | Netherlan ds; year not reported; 5 weeks | RCT; 269 patients with depression | “Taking Control” is an internet program based on problem solving therapy and self-examination therapy to help participants learn skills to regain control over their problems. Consists of 5 weekly sessions and email feedback from a coach. | Enhanced usual care | Not reported | $345 per patient; Components: intervention maintenance and hosting, coaches’ time | $22,279 per patient over 12 months; Components: delivery costs, healthcare visits to primary care, secondary care, and mental healthcare, informal care, household help, presenteeism, absenteeism | $20,119 per patient over 12 months; Components: healthcare visits to primary care, secondary care, and mental healthcare, informal care, household help, presenteeism, absenteeism | (1) Improvement on the CES-D scale, (2) QALYs | (1) $4407 per 1-point improvement on the CES-D scale, (2) $215,962 per QALY | At WTP $20,516 per point of improvement in CES-D, 57% probability of cost-effectiveness. At WTP $41,031 per QALY, 30% probability of cost-effectiveness |
| (van Luenen et al., 2019) | Netherlands; recruitment in 2015; 10 weeks | RCT; 188 patients living with HIV and depression | Internet CBT with psychoeducation based on activation, relaxation, changing negative cognitions, and goal attainment. Also received telephone support and motivation from coach | Usual care with check-in phone calls from a coach | $14,368 total; Components not reported | $34 per patient over 6 months; Components: coaches’ time and training, and supervision; website updates, domain, and security | $2285 per patient over 6 months; Components: delivery costs, healthcare visits, mental healthcare visits, medication, domestic help, absenteeism, presenteeism, unpaid work | $3087 per patient over 6 months; Components: coach time, training, and supervision, healthcare visits, mental healthcare visits, medication, domestic help, absenteeism, presenteeism, unpaid work | QALYs | ICER not reported | At WTP $5483, >95% probability of cost-effectiveness |
| (Littlewood et al., 2015) [Secondary studies: (Duarte et al., 2017)] |
UK; 2009–2013; duration not specified | RCT; 691 patients with depression | Two internet interventions: 1) Beating the Blues interactive CBT with 8 sessions and homework assignments; and 2) MoodGYM interactive CBT with 6 modules with exercises and assignments. Both interventions included weekly phone calls or emails to provide technical support and encouragement | Usual care | Not reported | Beating the Blues: $83. MoodGYM: $1 per patient; Components: intervention licensing fee, personnel time for telephone calls | Beating the Blues: $1192. MoodGYM: $1151 per patient over 12 months; Components: delivery costs, GP visits, nurse visits, out-of-hours GP services, hospital inpatient stays, outpatient visits, mental health services, other community services, depression-related medication | $973 per patient over 12 months; Components: GP visits, nurse visits, out-of-hours GP services, hospital inpatient stays, outpatient visits, mental health services, other community services, depression-related medication | QALYs | For MoodGYM, $11,476 per QALY vs. usual care. Beating the Blues was dominated by usual care | At a WTP of $33,106 per QALY, usual care had highest probability of being cost-effective (55%) followed by MoodGYM (42%) and Beating the Blues (4%) |
| (Romero-Sanchiz et al., 2017) | Spain; 2013–2015; 3 months | RCT; 296 patients with depression | “Smiling is fun” internet self-help program consisting of 10 CBT modules covering different psychological techniques to cope with depression. Both a self-directed version without guidance (TSG) and a version with low-intensity therapist guidance via email (LITG) were compared | Improved usual care | Not reported | TSG: $0. LITG: $0 per patient | TSG: $1755. LITG: $2359 per patient over 12-month period with ITT analysis; Components: medication consumption, medical tests, and use of health-related services (outpatient and inpatient), lost productivity | $2304 per patient over 12-month period with ITT analysis; Components: medication consumption, medical tests, and use of health-related services (outpatient and inpatient), lost productivity | (1) Improvement on BDI-II scale, (2) QALYs | (1) -$132 (TSG) and $13 (LITG) vs control per point improvement on BDI-II, (2) -$6929 (TSG) and $667 (LITG) vs control per QALY. Negative values indicated lower costs, higher efficacy | Authors state that costs are far below the threshold proposed by the National Institute for Clinical Excellence (WTP of $33,261) |
| (Titov et al., 2015) | Australia; year not reported; 8 weeks | RCT; 54 adults aged 60 and over with depression | Managing Your Mood Course: internet CBT (iCBT) including didactic lessons and case-enhanced learning, and email messaging with a therapist. | Waitlist control | Not reported | Not reported separately; Components: therapist time, supervisor time, internet access, computer and telephone use | $ 197 per patient over the 8-week intervention period; Components: delivery costs, primary and secondary healthcare consultations and admissions, use of antidepressants and anxiolytic medications | $ 146 per patient over the 8-week intervention period; Components: primary and secondary healthcare consultations and admissions, use of antidepressants and anxiolytic medications | QALYs | $4366 per QALY gained | At WTP of $49,709 per QALY, >95% probability of cost-effectiveness |
| (Warmerdam et al., 2010) | Netherlands; year not reported; 8 and 5 weeks | 3-arm RCT; 263 patients with depression | Two internet interventions: 1) 8-week CBT course consisting of psychoeducation and focused on skills such as relaxation, cognitive restructuring, social skills training, and behavioral activation; and 2) 5-week problem solving therapy (PST) course focused on accomplishing goals. Both interventions supported by a life coach via email for feedback and questions | Waitlist control | Not reported | CBT: $778. PST: $525 per patient; Components: website maintenance, personnel support, participant time spent on intervention. | CBT: $4370. PST: $4200 per patient over 12-week period; Components: delivery costs, healthcare service utilization, medication, direct non-medical (travel and parking), indirect non-medical (work loss, work cutback, domestic loss) | $3972 per patient over 12-week period; Components: healthcare service utilization, medication, direct non-medical (travel and parking), indirect non-medical (work loss, work cutback, domestic loss) | (1) Number of reliably improved patients, (2) QALYs | (1) $2821 (CBT) and $1938 (PST) vs control per reliably improved participant. (2) $35,107 (CBT) and $17,893 (PST) vs control per QALY gained | At WTP of $15,528 per improved participant, 91% (CBT) and 89% (PST) probability of cost-effectiveness. At WTP of $46,584 per QALY, 52% (CBT) and 61% (PST) probability of cost-effectiveness |
| (Wijnen et al., 2018) | Netherlands; recruitment from 2014 to 2015; 4 weeks | RCT; 329 patients with depressive symptoms | Three internet self-help complaint-directed mini interventions (CDMIs) with no therapist support. Participants could choose a CDMI based on their symptoms (sleep, stress, or worry). Largely based on CBT | Waitlist treatment as usual | Not reported | $4 per patient; Components: costs of hosting and updating website, helpdesk cost | $2293 per patient over 3-month period; Components: delivery costs, healthcare visits, mental healthcare visits, medication, home care, informal care at home, absenteeism, presenteeism | $2442 per patient over 3-month period; Components: Healthcare visits, mental healthcare visits, medication, home care, informal care at home, absenteeism, presenteeism | (1) Number of treatment responders, (2) QALYs | (1) Dominated in favor of intervention, (2) Dominated in favor of intervention | Not reported |
| OTHER CONTROL CONDITIONS | |||||||||||
| (Brabyn et al., 2016) | UK; 2013; up to 8 weeks | RCT; 369 patients with depression | MoodGYM internet interactive CBT with the goal to overcome patterns of unhelpful thinking, combined with 8 telephone support calls by trained workers to offer guidance and encouragement. | MoodGYM with minimal support (helpline for technical problems, no phone calls) | Not reported | $66 per patient; Components: personnel time for telephone calls | $2838 per patient over 12 months; Components: delivery costs, mental health hospital costs, non-mental health hospital costs, GP costs, other primary care, medications | $1887 per patient over 12 months; Components: mental health hospital costs, non-mental health hospital costs, GP costs, other primary care, medications | QALYs | Dominated in favor of intervention (incremental mean costs and QALYs for CEA analysis were taken from a regression that excluded outliers) | At WTP of $48,298 per QALY, 55% probability of cost-effectiveness |
| (Buntrock et al., 2017) | Germany; recruitment 2013–2014; up to 6 weeks | RCT; 406 patients with subthreshold depression | GET.ON internet self-help intervention with CBT and problem-solving therapy supported by text messages and an online trainer. | Web-based psychoeducation | Not reported | $409 per patient; Components: coach time, market price of intervention | $6367 per patient. Mean annual costs. Components: delivery costs, healthcare visits, hospital admissions, medications, patient travel, opportunity cost, informal care by family members, productivity loss (absenteeism and presenteeism) | $6172 per patient. Mean annual costs. Components: control web-based psychoeducation cost, healthcare visits, hospital admissions, medications, patient travel, opportunity cost, informal care by family members, productivity loss (absenteeism and presenteeism) | (1) Number of depression free years, (2) QALYs | (1) $1528 per depression free year, (2) $18,327 per QALY | At WTP of $27,354 per depression-free-year, 99% probability of cost-effectiveness. At WTP of $27,354 per QALY, 60% probability of cost-effectiveness |
| (Dear et al., 2015b) | Australia; year not reported; 8 weeks | Feasibility trial; 20 patients with depression | Managing Your Mood Course, an 8-week internet CBT courses with lessons, case-studies, and automated emails, to help participants learn and practice core CBT skills. | None | Not reported | Not reported separately; Components: resource use associated with the ICBT treatment | Difference in total cost from baseline to post-treatment (8 weeks): $55 more per patient; Components: delivery costs, healthcare consultations and admissions, antidepressant and anxiolytic medications | N/A | (1) Cases of clinical improvement, (2) QALYs | Not calculated | N/A |
| (Kenter et al., 2015) | Netherlands; 2009–2012; duration not specified | Descriptive naturalistic study; 4448 patients with depression or anxiety | Blended CBT with Internet- delivered treatment sessions and face-to-face treatment. Therapist support via internet platform. During face-to-face sessions, CBT was offered, and online activities of the patient were discussed. | Standard face-to-face CBT | Not reported | Depression: $2842. Anxiety: $2619 per patient; Components: therapist time | Depression: $2842 per patient. Anxiety:_$2619. Over duration of treatment (unspecified). Components: delivery costs | Depression: $1973 per patient. Anxiety: $1714. Over duration of treatment (unspecified). Components: therapist time | Improvement in GAF scores | Not calculated | N/A |
| (Kooistra et al., 2019) | Netherlands; 2014–2016; 30 weeks | RCT; 103 patients with depression | Blended CBT with 10 face to face and 9 internet based sessions focusing on psychoeducation, behavioral activation, cognitive restructuring, and relapse prevention. Therapist provided online feedback regarding content and progress. | In-person CBT, on average 18 sessions | Not reported | Not reported separately; Components: therapist time | $16,651 per patient over 30 weeks; Components: delivery costs, healthcare visits, medication, help from friends and family, presenteeism and absenteeism for paid and unpaid work | $11,404 per patient over 30 weeks; Components: healthcare visits, medication, help from friends and family, presenteeism and absenteeism for paid and unpaid work | (1) Number of treatment responses, (2) QALYs | (1) $52,947 per treatment response, (2) $249,582 per QALY | At WTP $33,568 per treatment response, 37% probability of cost-effectiveness. At WTP $33, 568 per QALY, 2% probability of cost-effectiveness |
| (Nobis et al., 2018) | Germany; year not reported; 6 weeks | RCT; 260 patients with type I or II diabetes and depressive symptoms | GET.ON.M.E.D. is 6 CBT-based internet sessions with support from a coach. The coach was supervised by an experienced clinical psychologist. | Usual care plus access to a self-help, internet psychoe ducation session acout depression | $205 per patient (including delivery cost of hosting website). Components: development and hosting of website | Not reported separately; Components: hosting website, coaches’ time, participant time | $7105 per patient over 6 months; Components: development costs, delivery costs, healthcare visits, hospitalizations, rehabilitation centers, additional services, medication, transportation, opportunity costs, absenteeism, presenteeism, domestic help | $6973 per patient over 6 months; Components: development costs, patient time, healthcare visits, hospitalizations, rehabilitation centers, additional services, medication, transportation, opportunity costs, absenteeism, presenteeism, domestic help | (1) Number of treatment responders, (2) QALYs | (1) $319 per treatment responder, (2) $14,645 per QALY. Includes development costs | At a WTP of $6839 per responder, 97% probability of cost-effectiveness. At a WTP of $19,148 per QALY, 51% probability of cost-effectiveness |
| (Phillips et al., 2014) | UK; recruitment 2009–2011; 6 weeks | RCT; 637 patients with depressive symptoms | MoodGYM internet interactive CBT with 5 weekly modules. Received weekly phone calls to maintain engagement and screen for risk | Were sent websites with information about mental health. Received weekly phone calls to maintain engagement and screen for risk | Not reported | Not reported; no components reported | $211 per patient over 6-week intervention period; Components: inpatient and outpatient hospital services, community healthcare visits, medication, lost work | $251 per patient over 6-week intervention period; Components: inpatient and outpatient hospital services, community healthcare visits, medication, lost work | (1) WSAS (work performance) scores, (2) QALYs | Not calculated | N/A |
| (Ruby et al., 2013) | USA; 2004–2008; duration not specified | RCT; 83 adolescents at risk of developing depression | CATCH-IT internet depression prevention intervention for at risk adolescents with 14 modules using behavioral activation, CBT, interpersonal training and family intervention. Two forms: CATCH-IT with motivational interviewing (MI) and CATCH-IT with brief advice (BA). MI group was contacted 3 times with encouragement. Both groups received safety calls | Costs compared to standard group CBT costs | Total: $157,131; Components: salaries of researchers and web designers and a data analyst, presentation services, database construction and supplies, domestic travel for feedback, consulting and promotion for feedback, publication costs, refreshments, other miscellaneous supplies, indirect costs | Not reported separately; Components: salaries of clinicians, administrators and data analyst, website maintenance, two physician visits, cost of safety training and safety calls | $677 per patient over 12-month intervention period; Components: delivery costs, indirect costs | The cost of a 15-session group CBT intervention was $1849 per patient (this figure taken from a different study) | Not reported | Not calculated | N/A |
Studies targeting both depression and anxiety were included in this table.
All costs displayed in 2015 USD
CBT: Cognitive Behavioral Therapy
ICER: Incremental Cost Effectiveness Ratio
QALY: Quality adjusted life year
RCT: Randomized control trial
WTP: Willingness-to-pay
Control conditions were heterogeneous but mostly involved usual care/waitlist (n=11; 57%) (Bolier et al., 2014; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Klein et al., 2018; Kolovos et al., 2016; Littlewood et al., 2015; Romero-Sanchiz et al., 2017; Titov et al., 2015; Warmerdam et al., 2010; Wijnen et al., 2018). Four studies compared an intervention to a less-intensive web-based or telephone intervention (Buntrock et al., 2017; Nobis et al., 2018; Phillips et al., 2014; van Luenen et al., 2019), one study compared an intervention with telephone support to the intervention without support (Brabyn et al., 2016), and two studies compared an intervention to face-to-face CBT (Kenter et al., 2015; Kooistra et al., 2019). One study did not have a control group but compared costs to standard in-person group CBT costs (Ruby et al., 2013).
Ten interventions (50%) were clinically effective compared to control (Bolier et al., 2014; Brabyn et al., 2016; Buntrock et al., 2017; Geraedts et al., 2015; Hollinghurst et al., 2010; Nobis et al., 2018; Romero-Sanchiz et al., 2017; Titov et al., 2015; Warmerdam et al., 2010; Wijnen et al., 2018). Eight studies (40%) found no significant difference in effectiveness between intervention and control, which included face-to-face CBT (Kenter et al., 2015; Kooistra et al., 2019), usual care (Holst et al., 2018; Klein et al., 2018; Kolovos et al., 2016; Littlewood et al., 2015), or a less-intensive web-based or telephone intervention (Phillips et al., 2014; van Luenen et al., 2019).
Intervention costs
Thirteen studies reported delivery costs separately from total costs (Bolier et al., 2014; Brabyn et al., 2016; Buntrock et al., 2017; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Kenter et al., 2015; Kolovos et al., 2016; Littlewood et al., 2015; Romero-Sanchiz et al., 2017; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018), and these ranged from $0 to $2842 per participant (mean $623; SD=$892), as seen in Figure 2. The study reporting $0 as a delivery cost noted that the interventions were developed by the government and thus would be free for use in a government-run health service (Romero-Sanchiz et al., 2017). Studies that were completely self-guided reported delivery costs of $85 per patient (Bolier et al., 2014), $0 per patient (Romero-Sanchiz et al., 2017), and $4 per patient (Wijnen et al., 2018). Nineteen studies reported total costs of the intervention, but over different time periods, which make comparison difficult. Eight studies reported total costs over a twelve-month period (Brabyn et al., 2016; Buntrock et al., 2017; Geraedts et al., 2015; Holst et al., 2018; Kolovos et al., 2016; Littlewood et al., 2015; Romero-Sanchiz et al., 2017; Ruby et al., 2013), and the mean cost of the intervention group was $7565 per participant (SD=$10,227) with a range of $677 to $30,160. The mean control group cost was $8961 per participant (SD=$10,921) with a range from $973 to $31,120. Four studies reported total costs over the intervention time period (range 6–8 weeks), and on average the intervention group incurred costs of $2407 per participant (SD=$1844), range $197 to $4370 (Kenter et al., 2015; Phillips et al., 2014; Titov et al., 2015; Warmerdam et al., 2010). Control conditions incurred costs of $1611 per participant (SD=$1558), range $146 to $3972. The studies that evaluated an intervention for depressive symptoms/prevention of depression had total costs of $6367 per patient over 12 months (Buntrock et al., 2017), $7105 per patient over 6 months (Nobis et al., 2018), $211 per patient over 6-weeks (Phillips et al., 2014), $677 per patient over 12 months (Ruby et al., 2013), and $2442 per patient over 3 months (Wijnen et al., 2018).
Cost components
Eighteen studies reported components of delivery costs. These delivery costs comprised personnel time (n=15) (Brabyn et al., 2016; Buntrock et al., 2017; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Kenter et al., 2015; Klein et al., 2018; Kolovos et al., 2016; Kooistra et al., 2019; Littlewood et al., 2015; Nobis et al., 2018; Ruby et al., 2013; Titov et al., 2015; van Luenen et al., 2019; Warmerdam et al., 2010), website hosting/maintenance and internet access/computer use (n=11) (Bolier et al., 2014; Dear et al., 2015b; Geraedts et al., 2015; Klein et al., 2018; Kolovos et al., 2016; Nobis et al., 2018; Ruby et al., 2013; Titov et al., 2015; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018), purchase or licensing fee of the intervention (n=3) (Buntrock et al., 2017; Holst et al., 2018; Littlewood et al., 2015), participant time spent on the intervention (n=3) (Bolier et al., 2014; Nobis et al., 2018; Warmerdam et al., 2010), and recruitment costs (n=1) (Bolier et al., 2014). For total costs, 18 studies included direct medical costs such as healthcare visits, hospitalizations, and medication use (Bolier et al., 2014; Brabyn et al., 2016; Buntrock et al., 2017; Dear et al., 2015b; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Klein et al., 2018; Kolovos et al., 2016; Kooistra et al., 2019; Littlewood et al., 2015; Nobis et al., 2018; Phillips et al., 2014; Romero-Sanchiz et al., 2017; Titov et al., 2015; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018). Seven reported direct non-medical costs (e.g., patient travel and parking) (Bolier et al., 2014; Buntrock et al., 2017; Hollinghurst et al., 2010; Holst et al., 2018; Klein et al., 2018; Nobis et al., 2018; Warmerdam et al., 2010), and 15 included indirect non-medical costs (e.g., lost productivity, absenteeism, domestic work loss) (Bolier et al., 2014; Buntrock et al., 2017; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Klein et al., 2018; Kolovos et al., 2016; Kooistra et al., 2019; Nobis et al., 2018; Phillips et al., 2014; Romero-Sanchiz et al., 2017; Ruby et al., 2013; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018).
Cost-effectiveness
Sixteen out of the 20 studies performed a cost-effectiveness analysis and calculated an ICER (Bolier et al., 2014; Brabyn et al., 2016; Buntrock et al., 2017; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Klein et al., 2018; Kolovos et al., 2016; Kooistra et al., 2019; Littlewood et al., 2015; Nobis et al., 2018; Romero-Sanchiz et al., 2017; Titov et al., 2015; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018). Two studies included development costs in their cost-effectiveness analyses (Klein et al., 2018; Nobis et al., 2018). Fifteen studies used QALYs as the measure of effectiveness (Brabyn et al., 2016; Buntrock et al., 2017; Geraedts et al., 2015; Hollinghurst et al., 2010; Holst et al., 2018; Klein et al., 2018; Kolovos et al., 2016; Kooistra et al., 2019; Littlewood et al., 2015; Nobis et al., 2018; Romero-Sanchiz et al., 2017; Titov et al., 2015; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018). Two studies reported that the ICER was dominated in favor of the intervention (Brabyn et al., 2016; Wijnen et al., 2018), one that the ICER was dominated in favor of the control (Littlewood et al., 2015), and the ICER range of the remaining studies was −$6929 to $717,530 per QALY (mean $116,775; SD=$202,445). ICERs can be seen in Figure 3. Negative ICERs represented lower costs and higher QALYs gained in the intervention group. Studies with usual care/waitlist control condition had an average ICER of $122,936 per QALY (SD=$222,400) and one study where the intervention dominated the control and another where the control dominated the intervention. Three of the studies which evaluated interventions for depressive symptoms/depression prevention calculated ICERS: they were $14,645 per QALY (Nobis et al., 2018), $18,327 per QALY (Buntrock et al., 2017), and ICER dominated in favor of intervention (Wijnen et al., 2018). Studies reported a variety of WTP thresholds (all are presented here in 2015 USD). Two interventions had a probability of cost-effectiveness that was greater than 95% at the study’s reported WTP threshold, which were $49,709 per QALY (Titov et al., 2015) and $5483 per QALY (van Luenen et al., 2019). All other interventions had a probability of cost-effectiveness ranging from 2% to 71% at reported WTP thresholds of $27,354 to $68,013 per QALY. One ICER was calculated with a clinical outcome and was $32,798 per treatment responder (Bolier et al., 2014). Eleven studies concluded that the interventions were cost-effective compared to control (Bolier et al., 2014; Brabyn et al., 2016; Buntrock et al., 2017; Geraedts et al., 2015; Hollinghurst et al., 2010; Nobis et al., 2018; Romero-Sanchiz et al., 2017; Titov et al., 2015; van Luenen et al., 2019; Warmerdam et al., 2010; Wijnen et al., 2018), and five concluded that the interventions were not cost-effective compared to control (Holst et al., 2018; Klein et al., 2018; Kolovos et al., 2016; Kooistra et al., 2019; Littlewood et al., 2015).
Intervention Development Costs
Eight studies reported the costs of developing the internet interventions within a research context. Two studies reported development costs as a total of $157,131 (Ruby et al., 2013) and $14,368 (van Luenen et al., 2019). Three studies presented development costs combined with intervention delivery costs: $65 per participant (Powell et al., 2020), $98 per participant (Klein et al., 2018), and $205 per participant (Nobis et al., 2018). Three studies reported estimated development costs, which were $662 per participant (Hedman et al., 2011), $657 per participant (Hedman et al., 2013), and $641 per participant (Nordgren et al., 2014). One study provided a breakdown of the unit costs for each cost component, such as personnel salaries, travel, consulting, and indirect costs (Ruby et al., 2013). Two studies included development costs in cost-effectiveness analyses (Klein et al., 2018; Nobis et al., 2018), and three studies included development costs only in sensitivity analyses (Hedman et al., 2013; Hedman et al., 2011; Nordgren et al., 2014).
Quality of studies
We calculated quality scores for all 33 studies, which are summarized in Table 4. The scores ranged from 0.57 to 0.98 with 94% (n=31) greater than or equal to 0.6, a suggested threshold for inclusion in an economic evaluation database (Gonzalez-Perez, 2002). The mean score was 0.85 (SD=0.13). Study design was strong, as most studies used an RCT to evaluate intervention effectiveness and 82% performed a cost-effectiveness analysis. However, there were some weaknesses in the reporting of data: over half of the studies did not report the year that the study was conducted, three studies did not report the year of the currency used, and many studies did not specify the meaning of the cost component terms they reported or what was included in each component. Some studies reported $0 for pertinent cost components or excluded pertinent cost components. Although most studies reported a societal perspective, there was little distinction of patient-level versus system-level costs in the studies.
Table 4.
Quality assessment of cost evaluation studies of internet-delivered psychological interventions for anxiety disorders and depression
| Part A. | |||||||||||||||||
| Quality Assessment Items | Andersson 2015a | Andersson 2015b | Axlesson 2018 | Bergstrom 2010 | Bolier 2014 | Brabyn 2016 | Buntrock 2017 | Dear 2015a | Dear 2015b - Anxiety | Dear 2015b-Depression | Geraedts 2015 | Hedman 2013 | Hedman 2016 | Hedman 2011 | Hollinghurst 2010 | Holst 2018 | |
| Study Design | |||||||||||||||||
| 1. Research question stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| 2. Economic importance of research question is stated | Y | Y | Y | Y | Y | Y | Y | N | NC | NC | Y | Y | Y | Y | Y | Y | |
| 3. Viewpoints clearly stated and justified | Y | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| Selection of alternatives | |||||||||||||||||
| 4. Rationale for choosing interventions compared are clearly described | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| 5. Alternatives compared are clearly described | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | NC | Y | |
| Economic evaluation | |||||||||||||||||
| 6. The form of economic evaluation used is stated | Y | Y | Y | Y | Y | Y | Y | Y | NC | NC | Y | Y | Y | Y | Y | Y | |
| 7. The choice of economic evaluation is justified | Y | Y | Y | N | Y | N | Y | N | N | N | Y | Y | Y | Y | Y | Y | |
| Effectiveness data | |||||||||||||||||
| 8. Sources of effectiveness estimates are stated | Y | Y | Y | Y | Y | Y | Y | Y | N | N | Y | Y | Y | Y | Y | Y | |
| 9. Details of design and results of effectiveness are given | Y | Y | Y | Y | Y | Y | Y | Y | N | N | Y | Y | Y | Y | Y | Y | |
| 10. Details of methods of synthesis/meta-analysis stated | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 11. Primary outcomes for economic evaluation stated | Y | Y | Y | Y | Y | Y | Y | Y | NC | NC | Y | Y | Y | Y | NC | Y | |
| 12. Methods to value benefits stated | N | Y | Y | N | Y | Y | Y | Y | NC | NC | Y | Y | Y | Y | Y | Y | |
| 13. Details of subjects | N | Y | Y | N | Y | Y | Y | Y | NC | NC | Y | Y | NC | Y | Y | Y | |
| 14. Productivity changes (if included) reported separately | Y | Y | Y | N/A | Y | N/A | Y | N/A | N/A | N/A | Y | Y | Y | Y | Y | Y | |
| 15. Relevance of productivity changes to the research question is discussed | N | Y | Y | N/A | N | N/A | N | N/A | N/A | N/A | N | N | NC | Y | Y | N | |
| Costing data | |||||||||||||||||
| 16. Quantities of resource use are reported separately to unit costs | N | Y | Y | N | Y | Y | Y | N | Y | Y | N | Y | Y | N | Y | N | |
| 17. Methods for estimating quantities of unit costs detailed | Y | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| 18. Currency and price data recorded | Y | Y | Y | Y | Y | Y | Y | Y | N | N | Y | Y | Y | Y | Y | Y | |
| 19. Details of inflation and/or currency conversion given | Y | Y | Y | N | N | N | Y | N | N | N | N | Y | Y | N | NC | Y | |
| Modelling | |||||||||||||||||
| 20. Details of any model used given | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 21. The choice of model used and the key parameters on which it is based are justified | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| Adjustments for timing of costs/benefits | |||||||||||||||||
| 22. Horizon of costs/benefits stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| 23. Discout rates stated | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 24. Choice of discount rates justified | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | |
| 25. Explanation given if no discounts applied | N | N | N | N | N | N | Y | N | N | N | Y | N | N | N | Y | Y | |
| Uncertainty estimates | |||||||||||||||||
| 26. Details of statistical tests and confidence intervals | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| 27. Approach to sensitivity analysis given | Y | Y | Y | N | Y | Y | Y | N | NC | NC | Y | Y | NC | Y | Y | Y | |
| 28. Choice of variables for sensitivity analysis justified | Y | Y | Y | N/A | Y | Y | Y | N/A | NC | NC | Y | Y | NC | Y | Y | N | |
| 29. Range over which variables are varied justified | Y | Y | Y | N/A | Y | Y | Y | N/A | NC | NC | Y | Y | NC | Y | Y | Y | |
| 30. Relevant alternatives compared | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | |
| Presentation of results | |||||||||||||||||
| 31. Incremental analysis reported | Y | Y | Y | Y | Y | Y | Y | Y | N | N | Y | Y | Y | Y | Y | Y | |
| 32. Major outcomes are presented in a disaggregated as well as aggregated form | Y | Y | Y | N | Y | Y | Y | N | N | N | Y | Y | Y | Y | Y | Y | |
| 33. Answer to research question given | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| 34. Conclusions follow from the data reported | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| 35. Conclusions are accompanied by caveats | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | |
| Total Quality Score | 0.83 | 0.97 | 0.97 | 0.62 | 0.9 | 0.89 | 0.97 | 0.73 | 0.57 | 0.57 | 0.9 | 0.93 | 0.88 | 0.9 | 0.95 | 0.87 | |
| Part B. | |||||||||||||||||
| Quality Assessment Items | Jolstedt 2018 | Kenter 2015 | Klein 2018 | Kolovos 2016 | Kooistra 2019 | Lenhard 2017 | Littlewood 2015 | van Luenen 2019 | Nobis 2018 | Nordgren 2014 | Phillips 2014 | Powell 2020 | Romero-Sanchiz 2017 | Ruby 2013 | Titov 2015 | Warmerdam 2010 | Wijnen 2018 |
| Study Design | |||||||||||||||||
| 1. Research question stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 2. Economic importance of research question is stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | NC | Y | N | Y | Y | Y | Y | Y |
| 3. Viewpoints clearly stated and justified | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y | Y | Y | Y |
| Selection of alternatives | |||||||||||||||||
| 4. Rationale for choosing interventions compared are clearly described | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 5. Alternatives compared are clearly described | Y | NC | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Economic evaluation | |||||||||||||||||
| 6. The form of economic evaluation used is stated | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | NC | Y | Y | Y |
| 7. The choice of economic evaluation is justified | Y | N | Y | Y | Y | Y | Y | Y | Y | N | N | Y | N | Y | Y | Y | Y |
| Effectiveness data | |||||||||||||||||
| 8. Sources of effectiveness estimates are stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 9. Details of design and results of effectiveness are given | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | NC | Y |
| 10. Details of methods of synthesis/meta-analysis stated | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 11. Primary outcomes for economic evaluation stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | NC | Y | Y | Y |
| 12. Methods to value benefits stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y |
| 13. Details of subjects | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y |
| 14. Productivity changes (if included) reported separately | Y | N/A | Y | Y | Y | Y | N/A | Y | Y | Y | Y | Y | Y | N/A | N/A | Y | Y |
| 15. Relevance of productivity changes to the research question is discussed | N | N/A | Y | NC | N | N | N/A | Y | N | N | Y | N | N | N/A | N/A | N | Y |
| Costing data | |||||||||||||||||
| 16. Quantities of resource use are reported separately to unit costs | Y | N | N | Y | Y | Y | Y | N | N | Y | N | N | Y | Y | N | Y | N |
| 17. Methods for estimating quantities of unit costs detailed | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y | NC | Y | Y |
| 18. Currency and price data recorded | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | NC | Y | Y |
| 19. Details of inflation and/or currency conversion given | Y | N | N | Y | Y | Y | N | Y | N | N | N | N | N | N | N | N | N |
| Modelling | |||||||||||||||||
| 20. Details of any model used given | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 21. The choice of model used and the key parameters on which it is based are justified | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| Adjustments for timing of costs/benefits | |||||||||||||||||
| 22. Horizon of costs/benefits stated | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 23. Discout rates stated | N/A | N/A | Y | N/A | N/A | N/A | Y | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 24. Choice of discount rates justified | N/A | N/A | Y | N/A | N/A | N/A | Y | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
| 25. Explanation given if no discounts applied | Y | N | N/A | Y | N | N | N/A | N | N | N | N | Y | Y | N | N | N | Y |
| Uncertainty estimates | |||||||||||||||||
| 26. Details of statistical tests and confidence intervals | Y | NC | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | Y |
| 27. Approach to sensitivity analysis given | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y | N | N | Y | Y |
| 28. Choice of variables for sensitivity analysis justified | Y | N/A | Y | Y | Y | Y | Y | Y | Y | Y | N/A | Y | Y | N/A | N/A | NC | Y |
| 29. Range over which variables are varied justified | Y | N/A | Y | Y | Y | Y | Y | Y | Y | NC | N/A | Y | Y | N/A | N/A | NC | Y |
| 30. Relevant alternatives compared | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | N | NC | Y |
| Presentation of results | |||||||||||||||||
| 31. Incremental analysis reported | Y | N | Y | Y | Y | Y | Y | N | Y | Y | N | Y | Y | N | Y | Y | Y |
| 32. Major outcomes are presented in a disaggregated as well as aggregated form | Y | NC | Y | Y | N | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | Y | Y |
| 33. Answer to research question given | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 34. Conclusions follow from the data reported | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| 35. Conclusions are accompanied by caveats | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y | Y |
| Total Quality Score | 0.97 | 0.6 | 0.94 | 0.98 | 0.9 | 0.93 | 0.97 | 0.9 | 0.87 | 0.83 | 0.68 | 0.87 | 0.9 | 0.65 | 0.73 | 0.83 | 0.93 |
NC: Not Clear
N/A: Not Applicable
Discussion
This systematic review summarized costs and cost-effectiveness of internet-delivered interventions for depression and anxiety disorders reported by 33 studies that met inclusion criteria. Delivery costs ranged from $124 to $1001 per participant for the anxiety disorder interventions and $0 to $2842 per participant for depression interventions. The ICERs for anxiety interventions were between −$19,659 and $10,298 per QALY, and for depression interventions were between −$6929 and $717,530 per QALY. Of 27 studies that reported on cost-effectiveness, 22 (81%) concluded that the intervention was cost-effective compared to the control condition. A detailed description of the cost components included and costs of developing the interventions were absent from many studies.
The wide ranges of delivery costs are likely due to several factors. First, studies included different cost components. Personnel and therapist costs were a commonly included component, but some studies also included costs such as website maintenance, intervention licensing fees, and participant time. Second, the interventions were different in terms of length and depth. The number of sessions and treatment length varied, and interventions also had differing levels of therapist involvement: some were completely participant run with no therapist support, some had limited therapist guidance, and some included therapist-delivered sessions.
We also observed a wide range of ICERs and were not able to compare the ICERs across the interventions due to large variation in reported cost components and heterogeneous control conditions (e.g., usual care/waitlist, in-person interventions, and other internet interventions). This limited our ability to assess which online intervention was most cost-effective. Additionally, five studies performed a cost-effectiveness analysis for a completely self-directed intervention (Axelsson et al., 2018; Bolier et al., 2014; Powell et al., 2020; Romero-Sanchiz et al., 2017; Wijnen et al., 2018), but these interventions varied in which disorder they targeted and what units were used for reporting ICERs. Therefore, we were unable to assess the cost-effectiveness of therapist-supported interventions compared to interventions without therapist support. Among included studies, the cost-effectiveness data appeared more favorable for anxiety interventions compared to depression interventions (100% of anxiety interventions were found to be cost-effective, versus 69% of depression interventions). This could partly be because there was less heterogeneity among the studies of internet interventions for anxiety, with the majority of these studies coming from Sweden and consistently showing clinical effectiveness of the interventions compared to the control conditions. By contrast, the studies of internet interventions for depression were from a wider range of countries and settings, and only about half showed clinical effectiveness compared to the control conditions. It is also possible that anxiety disorders may be more responsive to internet interventions, though further research would be needed to determine if the promising cost-effectiveness findings can generalize to treatment of anxiety disorders in other settings.
Most studies reported a societal perspective but did not differentiate between patient-side and system-level costs, which is consistent with the findings of prior reviews as well (Ahern et al., 2018; Donker et al., 2015; Ophuis et al., 2017). This makes it difficult to provide necessary information to policy makers who are interested in those interventions and would like to know the costs of implementing interventions. Relatively few studies reported development costs, which highlights the gap in knowledge about the cost of developing digital mental health interventions. Development costs are sometimes thought of as sunk costs and their inclusion in cost-effectiveness analyses is a matter of debate (Garrison Jr et al., 2010; McNamee et al., 2016; Ronckers et al., 2005). However, it is important to report these costs at least as cost estimates. If these interventions become commercially produced, having data about the development costs can help ensure pricing is fair and transparent. For example, the lack of transparency surrounding research and development costs of drugs creates difficulties in assessing fair drug pricing (Chit et al., 2014; Light and Warburton, 2011). Also, given the need to tailor psychotherapy interventions to different settings (such as changing language or cultural context), there will likely be adaptations required to disseminate these internet interventions to a global audience. Furthermore, there are constant advances in technology and constant efforts to improve and update available treatments for medical conditions (Bhugra et al., 2017; McNamee et al., 2016), suggesting that development of new internet interventions could be an ongoing rather than a sunk cost.
This review also shows that there is a lack of evidence surrounding the effectiveness and cost-effectiveness of internet interventions in low-resource settings, as all studies included were conducted in high-income countries. Internet interventions are potentially highly scalable, making them especially well-suited for low resource settings where there are few trained mental health providers (Carter et al., 2021; Fairburn and Patel, 2017; Naslund et al., 2019). Scalability allows for greater ease of access and implementation of internet interventions compared to in-person interventions in both high- and low-resource settings. This is an important benefit that may not be fully captured in cost-effectiveness analyses.
Only one of the interventions described in this review was formulated specifically as a smartphone app (Powell et al., 2020). Over 10,000 mental health apps exist (Bhugra et al., 2017), and the number of smartphone users as well as the number of individuals who rely solely on their smartphone for internet access continues to grow (Smith, 2015). Despite emerging evidence that smartphone apps can offer clinical benefit for treating anxiety disorders and depression (Linardon et al., 2019), there has been little study of the costs or cost-effectiveness of these smartphone apps for addressing mental disorders. Additionally, all studies evaluated a self-help or therapist-delivered internet course. There is a need for cost-effectiveness studies of other types of digital interventions for mental health, such as training or support interventions for mental health providers. Finally, there were also only five studies evaluating the costs of treating depressive symptoms and none for treating anxiety symptoms. Early interventions to prevent the development of depression and anxiety have the potential to be highly cost-effective but this area needs further research.
Our findings support prior literature, which suggest that internet interventions for mental health are cost-effective compared to usual care (Ahern et al., 2018; Donker et al., 2015; Hedman et al., 2012). Prior reviews have also noted that there is great heterogeneity among studies in how they conduct and report their cost-effectiveness analyses (Donker et al., 2015) and have called for standardization of how costs are collected, analyzed, and reported (Arnberg et al., 2014; Donker et al., 2015; Paganini et al., 2018). We agree that standardization of cost-effectiveness methodology, including which cost components are reported, would allow for comparison and analysis of intervention costs and cost-effectiveness. Based on the WHO’s Six Building Blocks of a Health System (World Health Organization, 2010) and the cost components included in the studies in this review, we propose a checklist of cost components to be included in future cost analyses of internet-based interventions for anxiety and depression (Table 5). We would also recommend that future studies report development costs and a breakdown of cost components with clear definitions, including a differentiation between patient-side costs and system-level costs.
Table 5.
Checklist of Cost Components
| Costs at the health system level | |
| Components | Examples |
| Human Resources (costs of those involved in developing or delivering online interventions, including volunteers or free experts) | • Costs of training personnel to deliver online interventions • Costs of medical personnel’s time in delivering online interventions • Costs of technician’s time for establishing and maintaining online interventions • Costs of supervisor’s time for managing the delivery |
| Health Information (technology equipment and software needed for intervention delivery, including free goods) | • Costs of using new or used technology equipment (e.g. computers, phones) or sofware • Costs of internet access • Market price or licensing fee of intervention • Costs of communication fees (e.g. internet and phone use) |
| Other related costs in delivering interventions (including free goods) | • Costs of office supplies • Costs of printing, copying, and scanning |
| Costs at the patient level | |
| Components | Examples |
| Costs of participating in intervention | • Time spent on interventions • Other costs (e.g. cell phone fees) |
| Out-of-pocket payments for received medical care | • Outpatient care • Inpatient care • Medications • Lab tests |
| Non-medical costs related to received medical care | • Traveling costs or parking expenses • Travel time • Food costs • Waiting time |
| Opportunity costs of illness | • Unemployment • Sick leave • Domestic work loss • Informal care by family members |
Despite the lack of standardization across studies, the conclusion that interventions are generally cost-effective suggests that governments and insurers should consider including internet interventions in their covered services. With an overwhelming number of unstudied digital mental health interventions available to consumers (Bhugra et al., 2017; Mathews et al., 2019), this review highlights which internet interventions have rigorous cost data to support their use. However, uptake of internet interventions remains low among service users and clinicians (Batterham et al., 2019; Mathews et al., 2019). Many apps that have been evaluated and described in academic journals are not publicly available (Lattie et al., 2016), and many clinicians may be unaware of how to incorporate internet health interventions into treatment plans for their patients (Bhugra et al., 2017). In high-income settings with more readily accessible mental health clinicians, incorporating internet interventions as an adjunct to in-person care in routine clinical settings may be an appropriate first step to expanding access to evidence-based services (Etzelmueller et al., 2020). In some cases, this approach is referred to as blended interventions that include both internet and in-person services. Initial reports and literature reviews summarizing evidence on these types of blended services show feasibility and promising clinical outcomes, though it is not clear how to achieve optimal balance of internet and face-to-face services to ensure treatment success (Erbe et al., 2017). More research is needed on how best to integrate internet interventions into existing mental health infrastructure in both high-income and low-income settings, as well as how internet interventions can reach individuals who are not already connected to formal care. This research should take care to also explore the ethical implications of recommending internet-based mental health interventions, such as data privacy in commercially available products, the validity of interventions, the training level of personnel involved in digital products, and the issue of asking patients with mental illness to self-monitor and self-treat (Bauer et al., 2017). Efforts to disseminate internet-based interventions should draw on research detailing the acceptability and user experience of digital interventions to help mitigate ethical concerns (Patel et al., 2020).
There are limitations of this review to consider. The variability in methodology among studies makes comparison and conclusions about cost-effectiveness difficult. Additionally, although quality of studies was largely high, study quality could impact cost-effectiveness estimates. Of concern would be the comprehensiveness of cost components included, accuracy of cost estimation methods, clarity of the description of currency and price data, adjustment for inflation and discounting, and quality of the effectiveness data. Although we recognize that there might be a relationship between quality of included studies and cost-effectiveness estimates, this review is limited in that we were unable to determine this relationship. Future studies are needed to explore this further.
Studies were concentrated in only a small number of high-income countries, most with universal health coverage, so the generalizability of their data to other settings is unclear. Also, costs are subject to variation according to how technological components of a program are priced and how equipment and connectivity costs vary over time and between regions, making it difficult to apply these findings to inform current health system decision-making.
Conclusion
This review represents an important step towards understanding the current literature on economic evaluations of internet interventions for anxiety disorders and depression and can inform future efforts to implement effective programs in real-world clinical settings. Implementing these programs has the potential to improve clinical outcomes for persons living with anxiety disorders and depression. Although the internet interventions reviewed were generally cost-effective, cost components and control conditions were not standardized across studies and cost components were not well-defined. Use of this review’s cost component checklist in future cost analyses can allow for improved comparison and synthesis of intervention costs. More research is needed to understand the costs of developing these interventions, their acceptability and cost-effectiveness in low-resource settings, and cost-effectiveness of newer technologies like mobile phone apps or wearable devices.
Supplementary Material
Highlights.
All studies from high-income countries; eight reported intervention development costs
Studies varied in control conditions and cost components
81% of studies concluded that interventions were cost-effective
Policy makers should consider including internet interventions in health systems
We propose a checklist of cost components for standardization of future cost analyses
Acknowledgements:
There are no other acknowledgements.
Funding: Funding for this study was received from the National Institute of Mental Health (Grant number: 5U19MH113211). The funder played no role in the study design; collection, analysis, or interpretation of data; writing of the manuscript; or decision to submit the manuscript for publication.
Submission Declaration: We confirm that this work has not been published previously, is not under consideration for publication elsewhere, and has been approved by all authors. If accepted to the Journal of Affective Disorders, this work will not be published elsewhere in the same form, in English or any other language, including electronically without the written consent of the copyright-holder.
Declarations of Interest: Funding for this study was received from the National Institute of Mental Health (Grant number: 5U19MH113211). The funder played no role in the study design; collection, analysis, or interpretation of data; writing of the manuscript; or decision to submit the manuscript for publication.
Funding for this study was received from the National Institute of Mental Health (Grant numbers: 5U19MH113211). The funder played no role in the study design; collection, analysis, or interpretation of data; writing of the manuscript; or decision to submit the manuscript for publication. There are no other financial or personal conflicts of interest.
Footnotes
Conflicts of Interest
Limitations: There are several limitations of this review. The variability in methodology among studies makes comparison and conclusions about cost-effectiveness difficult. The generalizability of these results is also unclear, as costs vary over time and between regions, and studies were concentrated in only a small number of high-income countries, most with universal health coverage.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Ahern E, Kinsella S, Semkovska M, 2018. Clinical efficacy and economic evaluation of online cognitive behavioral therapy for major depressive disorder: a systematic review and meta-analysis. Expert review of pharmacoeconomics & outcomes research 18(1), 25–41. [DOI] [PubMed] [Google Scholar]
- Andersson E, Hedman E, Ljótsson B, Wikström M, Elveling E, Lindefors N, Andersson G, Kaldo V, Rück C, 2015a. Cost-effectiveness of Internet-based cognitive behavior therapy for obsessive-compulsive disorder: results from a randomized controlled trial. Journal of Obsessive-Compulsive and Related Disorders 4, 47–53. [Google Scholar]
- Andersson E, Ljótsson B, Hedman E, Mattson S, Enander J, Andersson G, Kaldo V, Lindefors N, Rück C, 2015b. Cost-effectiveness of an internet-based booster program for patients with obsessive–compulsive disorder: Results from a randomized controlled trial. Journal of Obsessive-Compulsive and Related Disorders 4, 14–19. [Google Scholar]
- Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U, 2014. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PloS one 9(5), e98118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Axelsson E, Andersson E, Ljótsson B, Hedman-Lagerlöf E, 2018. Cost-effectiveness and long-term follow-up of three forms of minimal-contact cognitive behaviour therapy for severe health anxiety: Results from a randomised controlled trial. Behaviour research and therapy 107, 95–105. [DOI] [PubMed] [Google Scholar]
- Batterham PJ, Calear AL, O’Dea B, Larsen ME, J Kavanagh D, Titov N, March S, Hickie I, Teesson M, Dear BF, 2019. Stakeholder perspectives on evidence for digital mental health interventions: Implications for accreditation systems. Digital Health 5, 2055207619878069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bauer M, Glenn T, Monteith S, Bauer R, Whybrow PC, Geddes J, 2017. Ethical perspectives on recommending digital technology for patients with mental illness. International journal of bipolar disorders 5(1), 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baumann M, Stargardt T, Frey S, 2020. Cost–Utility of Internet-Based Cognitive Behavioral Therapy in Unipolar Depression: A Markov Model Simulation. Applied health economics and health policy, 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bergström J, Andersson G, Ljótsson B, Rück C, Andréewitch S, Karlsson A, Carlbring P, Andersson E, Lindefors N, 2010. Internet-versus group-administered cognitive behaviour therapy for panic disorder in a psychiatric setting: a randomised trial. BMC psychiatry 10(1), 54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bhugra D, Tasman A, Pathare S, Priebe S, Smith S, Torous J, Arbuckle MR, Langford A, Alarcón RD, Chiu HFK, 2017. The WPA-lancet psychiatry commission on the future of psychiatry. The Lancet Psychiatry 4(10), 775–818. [DOI] [PubMed] [Google Scholar]
- Bolier L, Majo C, Smit F, Westerhof GJ, Haverman M, Walburg JA, Riper H, Bohlmeijer E, 2014. Cost-effectiveness of online positive psychology: Randomized controlled trial. The Journal of Positive Psychology 9(5), 460–471. [Google Scholar]
- Brabyn S, Araya R, Barkham M, Bower P, Cooper C, Duarte A, Kessler D, Knowles S, Lovell K, Littlewood E, 2016. The second Randomised Evaluation of the Effectiveness, cost-effectiveness and Acceptability of Computerised Therapy (REEACT-2) trial: does the provision of telephone support enhance the effectiveness of computer-delivered cognitive behaviour therapy? A randomised controlled trial. Health technology assessment, 1–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buntrock C, Berking M, Smit F, Lehr D, Nobis S, Riper H, Cuijpers P, Ebert D, 2017. Preventing depression in adults with subthreshold depression: health-economic evaluation alongside a pragmatic randomized controlled trial of a web-based intervention. Journal of medical Internet research 19(1). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carter H, Araya R, Anjur K, Deng D, Naslund JA, 2021. The emergence of digital mental health in low-income and middle-income countries: A review of recent advances and implications for the treatment and prevention of mental disorders. Journal of Psychiatric Research 133, 233–246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, Saxena S, 2016. Scaling-up treatment of depression and anxiety: a global return on investment analysis. The Lancet Psychiatry 3(5), 415–424. [DOI] [PubMed] [Google Scholar]
- Chit A, Parker J, Halperin SA, Papadimitropoulos M, Krahn M, Grootendorst P, 2014. Toward more specific and transparent research and development costs: The case of seasonal influenza vaccines. Vaccine 32(26), 3336–3340. [DOI] [PubMed] [Google Scholar]
- Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Sheehan J, Staples LG, Gandy M, Fogliati VJ, Klein B, 2015a. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety: a randomized controlled trial. Behavior Therapy 46(2), 206–217. [DOI] [PubMed] [Google Scholar]
- Dear BF, Zou JB, Ali S, Lorian CN, Johnston L, Terides MD, Staples LG, Gandy M, Fogliati VJ, Klein B, 2015b. Examining self-guided internet-delivered cognitive behavior therapy for older adults with symptoms of anxiety and depression: Two feasibility open trials. Internet Interventions 2(1), 17–23. [Google Scholar]
- Donker T, Blankers M, Hedman E, Ljotsson B, Petrie K, Christensen H, 2015. Economic evaluations of Internet interventions for mental health: a systematic review. Psychological medicine 45(16), 3357–3376. [DOI] [PubMed] [Google Scholar]
- Drummons M, Jefferson T, 1996. Guidelines for authors and peer reviewers of economic submissions to the BMJ. British Medical Journal 313(7052), 275–284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Duarte A, Walker S, Littlewood E, Brabyn S, Hewitt C, Gilbody S, Palmer S, 2017. Cost-effectiveness of computerized cognitive–behavioural therapy for the treatment of depression in primary care: findings from the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Psychological medicine 47(10), 1825–1835. [DOI] [PubMed] [Google Scholar]
- El Alaoui S, Hedman-Lagerlöf E, Ljótsson B, Lindefors N, 2017. Does internet-based cognitive behaviour therapy reduce healthcare costs and resource use in treatment of social anxiety disorder? A cost-minimisation analysis conducted alongside a randomised controlled trial. BMJ open 7(9), e017053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Erbe D, Eichert H-C, Riper H, Ebert DD, 2017. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: systematic review. Journal of medical Internet research 19(9), e306. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Etzelmueller A, Vis C, Karyotaki E, Baumeister H, Titov N, Berking M, Cuijpers P, Riper H, Ebert DD, 2020. Effects of internet-based cognitive behavioral therapy in routine care for adults in treatment for depression and anxiety: Systematic review and meta-analysis. Journal of medical Internet research 22(8), e18100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fairburn CG, Patel V, 2017. The impact of digital technology on psychological treatments and their dissemination. Behaviour research and therapy 88, 19–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Firth J, Torous J, Nicholas J, Carney R, Pratap A, Rosenbaum S, Sarris J, 2017a. The efficacy of smartphone‐based mental health interventions for depressive symptoms: a meta - analysis of randomized controlled trials. World Psychiatry 16(3), 287–298. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Firth J, Torous J, Nicholas J, Carney R, Rosenbaum S, Sarris J, 2017b. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. Journal of affective disorders 218, 15–22. [DOI] [PubMed] [Google Scholar]
- Garrison LP Jr, Mansley EC, Abbott III TA, Bresnahan BW, Hay JW, Smeeding J, 2010. Good Research Practices for Measuring Drug Costs in Cost‐Effectiveness Analyses: A Societal Perspective: The ISPOR Drug Cost Task Force Report—Part II. Value in Health 13(1), 8–13. [DOI] [PubMed] [Google Scholar]
- Gehring ND, McGrath P, Wozney L, Soleimani A, Bennett K, Hartling L, Huguet A, Dyson MP, Newton AS, 2017. Pediatric eMental healthcare technologies: a systematic review of implementation foci in research studies, and government and organizational documents. Implementation Science 12(1), 76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Geraedts AS, van Dongen JM, Kleiboer AM, Wiezer NM, van Mechelen W, Cuijpers P, Bosmans JE, 2015. Economic evaluation of a Web-based guided self-help intervention for employees with depressive symptoms: results of a randomized controlled trial. Journal of occupational and environmental medicine 57(6), 666–675. [DOI] [PubMed] [Google Scholar]
- Gonzalez-Perez J, 2002. Developing a scoring system to quality assess economic evaluations. The European Journal of Health Economics 3(2), 131–136. [DOI] [PubMed] [Google Scholar]
- Hedman E, Andersson E, Lindefors N, Andersson G, Rück C, Ljótsson B, 2013. Cost-effectiveness and long-term effectiveness of internet-based cognitive behaviour therapy for severe health anxiety. Psychological medicine 43(2), 363–374. [DOI] [PubMed] [Google Scholar]
- Hedman E, Andersson E, Ljótsson B, Andersson G, Rück C, Lindefors N, 2011. Cost-effectiveness of Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: results from a randomized controlled trial. Behaviour research and therapy 49(11), 729–736. [DOI] [PubMed] [Google Scholar]
- Hedman E, Andersson E, Ljótsson B, Axelsson E, Lekander M, 2016. Cost effectiveness of internet-based cognitive behaviour therapy and behavioural stress management for severe health anxiety. BMJ open 6(4), e009327. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hedman E, El Alaoui S, Lindefors N, Andersson E, Rück C, Ghaderi A, Kaldo V, Lekander M, Andersson G, Ljótsson B, 2014. Clinical effectiveness and cost-effectiveness of Internet-vs. group-based cognitive behavior therapy for social anxiety disorder: 4-year follow-up of a randomized trial. Behaviour research and therapy 59, 20–29. [DOI] [PubMed] [Google Scholar]
- Hedman E, Ljótsson B, Lindefors N, 2012. Cognitive behavior therapy via the Internet: a systematic review of applications, clinical efficacy and cost–effectiveness. Expert review of pharmacoeconomics & outcomes research 12(6), 745–764. [DOI] [PubMed] [Google Scholar]
- Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, 2019. Cochrane handbook for systematic reviews of interventions. John Wiley & Sons. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hollinghurst S, Peters TJ, Kaur S, Wiles N, Lewisand G, Kessler D, 2010. Cost-effectiveness of therapist-delivered online cognitive–behavioural therapy for depression: randomised controlled trial. The British Journal of Psychiatry 197(4), 297–304. [DOI] [PubMed] [Google Scholar]
- Holst A, Björkelund C, Metsini A, Madsen J-H, Hange D, Petersson E-LL, Eriksson MC, Kivi M, Andersson P-ÅÅ, Svensson M, 2018. Cost-effectiveness analysis of internet-mediated cognitive behavioural therapy for depression in the primary care setting: results based on a controlled trial. BMJ open 8(6). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jolstedt M, Wahlund T, Lenhard F, Ljótsson B, Mataix-Cols D, Nord M, Öst L-G, Högström J, Serlachius E, Vigerland S, 2018. Efficacy and cost-effectiveness of therapist-guided internet cognitive behavioural therapy for paediatric anxiety disorders: a single-centre, single-blind, randomised controlled trial. The Lancet Child & Adolescent Health 2(11), 792–801. [DOI] [PubMed] [Google Scholar]
- Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, Mackinnon A, Meyer B, Botella C, Littlewood E, 2017. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA psychiatry 74(4), 351–359. [DOI] [PubMed] [Google Scholar]
- Kazdin AE, 2017. Addressing the treatment gap: A key challenge for extending evidence-based psychosocial interventions. Behaviour research and therapy 88, 7–18. [DOI] [PubMed] [Google Scholar]
- Kenter RM, van de Ven PM, Cuijpers P, Koole G, Niamat S, Gerrits RS, Willems M, van Straten A, 2015. Costs and effects of Internet cognitive behavioral treatment blended with face-to-face treatment: results from a naturalistic study. Internet Interventions 2(1), 77–83. [Google Scholar]
- Klein NS, Bockting CL, Wijnen B, Kok GD, van Valen E, Riper H, Cuijpers P, Dekker J, van der Heiden C, Burger H, 2018. Economic evaluation of an internet-based preventive cognitive therapy with minimal therapist support for recurrent depression: randomized controlled trial. Journal of medical Internet research 20(11), e10437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kolovos S, Kenter RM, Bosmans JE, Beekman AT, Cuijpers P, Kok RN, van Straten A, 2016. Economic evaluation of Internet-based problem-solving guided self-help treatment in comparison with enhanced usual care for depressed outpatients waiting for face-to-face treatment: A randomized controlled trial. Journal of affective disorders 200, 284–292. [DOI] [PubMed] [Google Scholar]
- Kolovos S, van Dongen JM, Riper H, Buntrock C, Cuijpers P, Ebert DD, Geraedts AS, Kenter RM, Nobis S, Smith A, 2018. Cost effectiveness of guided Internet‐based interventions for depression in comparison with control conditions: An individual–participant data meta‐analysis. Depression and anxiety 35(3), 209–219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kooistra LC, Wiersma JE, Ruwaard J, Neijenhuijs K, Lokkerbol J, van Oppen P, Smit F, Riper H, 2019. Cost and effectiveness of blended versus standard cognitive behavioral therapy for outpatients with depression in routine specialized mental health care: Pilot randomized controlled trial. Journal of medical Internet research 21(10), e14261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lattie EG, Schueller SM, Sargent E, Stiles-Shields C, Tomasino KN, Corden ME, Begale M, Karr CJ, Mohr DC, 2016. Uptake and usage of IntelliCare: a publicly available suite of mental health and well-being apps. Internet interventions 4, 152–158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lenhard F, Ssegonja R, Andersson E, Feldman I, Rück C, Mataix-Cols D, Serlachius E, 2017. Cost-effectiveness of therapist-guided internet-delivered cognitive behaviour therapy for paediatric obsessive–compulsive disorder: results from a randomised controlled trial. BMJ open 7(5), e015246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lennon MR, Bouamrane M-M, Devlin AM, O’Connor S, O’Donnell C, Chetty U, Agbakoba R, Bikker A, Grieve E, Finch T, 2017. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. Journal of medical Internet research 19(2), e42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Light DW, Warburton R, 2011. Demythologizing the high costs of pharmaceutical research. BioSocieties 6(1), 34–50. [Google Scholar]
- Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz M, 2019. The efficacy of app‐supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 18(3), 325–336. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Littlewood E, Duarte A, Hewitt C, Knowles S, Palmer S, Walker S, Andersen P, Araya R, Barkham M, Bower P, 2015. A randomised controlled trial of computerised cognitive behaviour therapy for the treatment of depression in primary care: the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Health Technology Assessment (Winchester, England) 19(101), 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB, 2019. Digital health: a path to validation. NPJ digital medicine 2(1), 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McNamee P, Murray E, Kelly MP, Bojke L, Chilcott J, Fischer A, West R, Yardley L, 2016. Designing and undertaking a health economics study of digital health interventions. American journal of preventive medicine 51(5), 852–860. [DOI] [PubMed] [Google Scholar]
- Naslund JA, Aschbrenner KA, Araya R, Marsch LA, Unützer J, Patel V, Bartels SJ, 2017. Digital technology for treating and preventing mental disorders in low-income and middle-income countries: a narrative review of the literature. The Lancet Psychiatry 4(6), 486–500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Naslund JA, Gonsalves PP, Gruebner O, Pendse SR, Smith SL, Sharma A, Raviola G, 2019. Digital innovations for global mental health: opportunities for data science, task sharing, and early intervention. Current Treatment Options in Psychiatry, 1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Naslund JA, Marsch LA, McHugo GJ, Bartels SJ, 2015. Emerging mHealth and eHealth interventions for serious mental illness: a review of the literature. Journal of mental health 24(5), 321–332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Newby JM, McKinnon A, Kuyken W, Gilbody S, Dalgleish T, 2015. Systematic review and meta-analysis of transdiagnostic psychological treatments for anxiety and depressive disorders in adulthood. Clinical psychology review 40, 91–110. [DOI] [PubMed] [Google Scholar]
- Nobis S, Ebert DD, Lehr D, Smit F, Buntrock C, Berking M, Baumeister H, Snoek F, Funk B, Riper H, 2018. Web-based intervention for depressive symptoms in adults with types 1 and 2 diabetes mellitus: a health economic evaluation. The British Journal of Psychiatry 212(4), 199–206. [DOI] [PubMed] [Google Scholar]
- Nordgren LB, Hedman E, Etienne J, Bodin J, Kadowaki Å, Eriksson S, Lindkvist E, Andersson G, Carlbring P, 2014. Effectiveness and cost-effectiveness of individually tailored Internet-delivered cognitive behavior therapy for anxiety disorders in a primary care population: a randomized controlled trial. Behaviour research and therapy 59, 1–11. [DOI] [PubMed] [Google Scholar]
- Ophuis RH, Lokkerbol J, Heemskerk SC, van Balkom AJ, Hiligsmann M, Evers SM, 2017. Cost-effectiveness of interventions for treating anxiety disorders: A systematic review. Journal of affective disorders 210, 1–13. [DOI] [PubMed] [Google Scholar]
- Paganini S, Teigelkoetter W, Buntrock C, Baumeister H, 2018. Economic evaluations of internet-and mobile-based interventions for the treatment and prevention of depression: a systematic review. Journal of affective disorders 225, 733–755. [DOI] [PubMed] [Google Scholar]
- Păsărelu CR, Andersson G, Bergman Nordgren L, Dobrean A, 2017. Internet-delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and depression: a systematic review and meta-analysis of randomized controlled trials. Cognitive Behaviour Therapy 46(1), 1–28. [DOI] [PubMed] [Google Scholar]
- Patel S, Akhtar A, Malins S, Wright N, Rowley E, Young E, Sampson S, Morriss R, 2020. The acceptability and usability of digital health interventions for adults with depression, anxiety, and somatoform disorders: Qualitative systematic review and meta-synthesis. Journal of medical Internet research 22(7), e16228. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Patel V, Saxena S, Lund C, Thornicroft G, Baingana F, Bolton P, Chisholm D, Collins PY, Cooper JL, Eaton J, Herrman H, Herzallah MM, Huang Y, Jordans MJD, Kleinman A, Medina-Mora ME, Morgan E, Niaz U, Omigbodun O, Prince M, Rahman A, Saraceno B, Sarkar BK, De Silva M, Singh I, Stein DJ, Sunkel C, UnÜtzer J, 2018. The Lancet Commission on global mental health and sustainable development. The Lancet 392(10157), 1553–1598. [DOI] [PubMed] [Google Scholar]
- Phillips R, Schneider J, Molosankwe I, Leese M, Foroushani PS, Grime P, McCrone P, Morriss R, Thornicroft G, 2014. Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention. Psychological medicine 44(4), 741–752. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Powell J, Williams V, Atherton H, Bennett K, Yang Y, Davoudianfar M, Hellsing A, Martin A, Mollison J, Shanyinde M, 2020. Effectiveness and Cost-Effectiveness of a Self-Guided Internet Intervention for Social Anxiety Symptoms in a General Population Sample: Randomized Controlled Trial. Journal of medical Internet research 22(1), e16804. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Richards D, Richardson T, 2012. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clinical psychology review 32(4), 329–342. [DOI] [PubMed] [Google Scholar]
- Romero-Sanchiz P, Nogueira-Arjona R, García-Ruiz A, Luciano JV, Campayo JG, Gili M, Botella C, Baños R, Castro A, López-Del-Hoyo Y, 2017. Economic evaluation of a guided and unguided internet-based CBT intervention for major depression: Results from a multi-center, three-armed randomized controlled trial conducted in primary care. PloS one 12(2), e0172741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ronckers E, Groot W, Ament A, 2005. Systematic review of economic evaluations of smoking cessation: standardizing the cost-effectiveness. Medical Decision Making 25(4), 437–448. [DOI] [PubMed] [Google Scholar]
- Ruby A, Marko-Holguin M, Fogel J, Van Voorhees BW, 2013. Economic analysis of an internet-based depression prevention intervention. The journal of mental health policy and economics 16(3), 121–130. [PMC free article] [PubMed] [Google Scholar]
- Shemilt I, Mugford M, Byford S, Drummond M, Eisenstein E, Knapp M, Mallender J, McDaid D, Vale L, Walker D, 2008. 15 Incorporating economics evidence. Cochrane handbook for systematic reviews of interventions, 449. [Google Scholar]
- Smith A, 2015. US Smartphone Use in 2015. Pew Research Center: Washington, DC, USA. [Google Scholar]
- The World Bank, 2020a. GDP deflator (base year varies by country). The World Bank Group, Washington, DC. [Google Scholar]
- The World Bank, 2020b. Official exchange rate. World Bank Group, Washington, DC. [Google Scholar]
- Titov N, Dear BF, Ali S, Zou JB, Lorian CN, Johnston L, Terides MD, Kayrouz R, Klein B, Gandy M, 2015. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial. Behavior therapy 46(2), 193–205. [DOI] [PubMed] [Google Scholar]
- Topooco N, Riper H, Araya R, Berking M, Brunn M, Chevreul K, Cieslak R, Ebert DD, Etchmendy E, Herrero R, 2017. Attitudes towards digital treatment for depression: a European stakeholder survey. Internet interventions 8, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- van Luenen S, Kraaij V, Garnefski N, Spinhoven P, van den Akker–van ME, 2019. Cost-utility of a guided Internet-based intervention in comparison with attention only for people with HIV and depressive symptoms: A randomized controlled trial. Journal of psychosomatic research 118, 34–40. [DOI] [PubMed] [Google Scholar]
- Vigo D, Thornicroft G, Atun R, 2016. Estimating the true global burden of mental illness. The Lancet Psychiatry 3(2), 171–178. [DOI] [PubMed] [Google Scholar]
- Warmerdam L, Smit F, van Straten A, Riper H, Cuijpers P, 2010. Cost-utility and cost-effectiveness of internet-based treatment for adults with depressive symptoms: randomized trial. Journal of medical Internet research 12(5). [DOI] [PMC free article] [PubMed] [Google Scholar]
- Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, Charlson FJ, Norman RE, Flaxman AD, Johns N, 2013. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. The Lancet 382(9904), 1575–1586. [DOI] [PubMed] [Google Scholar]
- Wijnen BF, Lokman S, Leone S, Evers SM, Smit F, 2018. Complaint-Directed Mini-Interventions for depressive symptoms: a health economic evaluation of unguided web-based self-help interventions based on a randomized controlled trial. Journal of medical Internet research 20(10), e10455. [DOI] [PMC free article] [PubMed] [Google Scholar]
- World Health Organization, 2010. Monitoring the building blocks of health systems: A handbook of indicators and their measurement strategies, Geneva, Switzerland. [Google Scholar]
- Xu K, Soucat A, Kutzin J, Brindley C, Vande Maele N, Touré H, Aranguren Garcia M, Li D, Barroy H, Flores G, Roubal T, Indikadahena C, Cherilova V, Siroka A, 2018. Public spending on health: a closer look at global trends. World Health Organization, Geneva, Switzerland. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.