Internet-Delivered Cognitive Behavioural Therapy for Post-traumatic Stress Disorder or Acute Stress Disorder: A Health Technology Assessment

Abstract

Background

Post-traumatic stress disorder (PTSD) and acute stress disorder (ASD) are mental health conditions that may emerge following a frightening or traumatic event in a person's life. We conducted a health technology assessment of internet-delivered cognitive behavioural therapy (iCBT) for adults with PTSD or ASD, which included an evaluation of effectiveness, safety, cost-effectiveness, the budget impact of publicly funding iCBT for PTSD or ADS, and patient preferences and values.

Methods

We performed a systematic literature search of the clinical evidence. We assessed the risk of bias of systematic reviews using ROBIS and of randomized controlled trials (RCTs) using the Cochrane Risk of Bias Tool, and the quality of the body of evidence according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group criteria.

We performed a systematic economic literature search to summarize the economic evidence on the cost-effectiveness of iCBT for adults with PTSD or ASD. We did not conduct a primary economic evaluation on iCBT for adults with PTSD, as an existing cost–utility analysis is directly applicable to this research question. We did not conduct a primary economic evaluation on iCBT for adults with ASD, as there is limited clinical evidence on this topic and because evidence on iCBT for PTSD may be generalizable to iCBT for ASD at risk of progressing to PTSD. We analyzed the budget impact of publicly funding iCBT for adults with PTSD or ASD in Ontario over the next 5 years.

To contextualize the potential value of iCBT for PTSD, we reviewed relevant literature on patients’ preferences and values and spoke with people who have lived experience with PTSD to explore their values, needs, and priorities.

Results

We identified no studies on the use of iCBT for prevention of PTSD or studies on the use of iCBT to treat ASD, nor studies that directly compared iCBT with face-to-face CBT for the treatment of PTSD. We included one systematic review of the use of iCBT to treat PTSD (10 RCTs, N = 720). Overall, iCBT is more effective than wait-list (waiting for iCBT) or usual care alone for reducing the severity of PTSD symptoms (standardized mean difference [SMD] = −0.60 [95% CI −0.97 to −0.24]; N = 560, 8 RCTs) (GRADE: Very low). Internet-delivered CBT is not more effective than non–CBT internet-delivered interventions for reducing the severity of PTSD symptoms (SMD = −0.08 [−0.52 to 0.35]; N = 82, 2 RCTs) (GRADE: Very low).

We identified one economic evaluation on the cost-effectiveness of iCBT for adults with PTSD. For adults with PTSD, iCBT was found to be dominant (i.e., less costly and more effective) compared with usual care. The model used a Canadian public health care payer perspective, and there were no major limitations to the model structure, time horizon, or source of model inputs. The annual budget impact of publicly funding iCBT in Ontario over the next 5 years ranges from an additional $2.43 million in year 1 to $2.37 million in year 5, for a total additional cost of $16.53 million over the next 5 years. If treatment costs alone are considered, the annual budget impact ranges from an additional $3.37 million in year 1 to $17.84 million in year 5, for a total additional cost of $52.61 million over the next 5 years.

Our review of the quantitative literature on patient preferences found that adults with PTSD may experience iCBT as a generally acceptable form of treatment, but there is uncertainty in the evidence due to incomplete follow-up in studies and variability in the nature and extent of the therapist–patient relationship. The 10 people we spoke with had all been diagnosed with PTSD. They reported on its negative impact on their quality of life, including difficulty in managing everyday activities, relationships, and employment. Participants viewed iCBT as beneficial to managing their PTSD symptoms but stressed the importance of combining it with face-to-face CBT. However, wait times for PTSD services are long, and out-of-pocket expenses could be a barrier for people without private insurance.

Conclusions

Internet-delivered CBT may reduce the severity of PTSD symptoms compared with wait-list or usual care, but the evidence is very uncertain, and iCBT may have little to no effect on improving PTSD symptoms compared with non–CBT interventions delivered online, but here as well the evidence is very uncertain.

For adults with PTSD, iCBT may be cost-effective compared with usual care. We estimate that publicly funding iCBT in Ontario would result in additional costs of between $2.37 million and $2.43 million per year over the next 5 years.

People with PTSD seem to generally find iCBT as an acceptable treatment option. People with PTSD with whom we spoke viewed iCBT to be effective and recommended it be combined with in-person psychotherapy.

Objective

This health technology assessment evaluates the effectiveness, safety, and cost-effectiveness of internet-delivered cognitive behavioural therapy (iCBT) for adults with post-traumatic stress disorder (PTSD) or acute stress disorder (ASD). It also evaluates the budget impact of publicly funding iCBT and the experiences, preferences, and values of people with PTSD or ASD.

Background

Health Condition

Post-traumatic stress disorder (PTSD) is a mental health condition that may emerge after a frightening or traumatic event such as physical or sexual assault, combat or warfare, motor vehicle collisions, or other threats to a person's life.¹ Symptoms, which can vary over time and from person to person, include re-experiencing the event through intrusive thoughts, memories, images, nightmares, and flashbacks; avoiding situations that are reminders of the event; negative changes in thinking and mood such as hyperarousal and hypervigilance, anger, and irritability; and other changes in physical and emotional reactions.² Most people who experience a traumatic event will experience some symptoms of post-traumatic stress, but most will recover over time without intervention. People are said to develop PTSD if they continue to experience symptoms that cause significant distress and significantly interfere with functioning a month or more after the event. Symptoms of PTSD usually appear within 3 months of the event but may not appear for years.¹ People can recover from PTSD. Some recover in 6 months, while others take much longer.¹

Acute stress disorder (ASD) is an intense, unpleasant reaction to an overwhelming traumatic event, beginning shortly afterwards and lasting less than a month.³ Symptoms of ASD are similar to PTSD and may include feelings of detachment from reality (derealization), and/or feelings of detachment from oneself and one's experiences (depersonalization).² Derealization and depersonalization can also occur in PTSD, but they are often more prominent in ASD. If these symptoms persist longer than a month, people are diagnosed as having PTSD.³ Most people recover from ASD once they are removed from the traumatic situation and given appropriate support.

Clinical Need and Target Population

People with PTSD may have considerable distress and problems with social, educational, and occupational functioning.⁴ PTSD may lead to somatization (physical symptoms), chronic pain, and poor health.⁴ People with PTSD are at greater risk of a variety of medical problems, including circulatory and musculoskeletal disorders, and have more medical conditions than people without PTSD.⁴ It is estimated that 9.2% of people in Canada will have PTSD at some point in their adult lifetime.⁵

The point prevalence of ASD (the proportion of people with the condition at a certain point in time) has been estimated to range from 5% to 20%, depending on the nature and severity of trauma and the instrument used to identify the disorder.⁶

Current Treatment Options

Psychotherapy is the treatment of mental or emotional illness by using psychological methods (i.e., talking about problems) rather than medication. There are several types of psychotherapy for PTSD including cognitive behavioural therapy (CBT), cognitive processing therapy (CPT), and eye movement desensitization and reprocessing (EMDR). Cognitive processing therapy is a variation of CBT, and research on EMDR suggests some components of EMDR may be similar to those of CBT.⁷

Trauma-focused CBT includes therapies aimed at helping a person challenge their thoughts, beliefs, and/or behaviour so they can function well despite the trauma. These therapies typically include psychoeducation (to help people understand and cope with their problems), cognitive and exposure work (to break patterns of fear and avoidance), stress/relaxation management, and homework. Trauma-focused CBT is widely considered an effective treatment for PTSD⁸ and recommended for ASD or PTSD in international and Canadian clinical guidelines.^9–11 Trained providers deliver trauma-focused CBT over 8 to 12 sessions based on a validated manual, with additional booster sessions after the first course of treatment if needed, particularly in relation to significant dates such as an anniversary of the trauma.¹¹

Trauma-focused CBT interventions delivered as individual (as opposed to group) therapy are effective for improving PTSD symptoms in adults who experienced a traumatic event within the previous month.¹¹ Trauma-focused CBT may also reduce the number of adults who meet the criteria to be diagnosed with PTSD after 1 month. In 2018, the UK National Institute for Health and Care Excellence (NICE) recommended offering individual, trauma-focused CBT interventions to adults who have ASD or clinically important symptoms of PTSD and have been exposed to one or more traumatic events within a month of an initial traumatic event.¹¹ People with clinically important symptoms of PTSD refers to people assessed as having PTSD on a validated scale, as indicated by baseline scores above the clinical threshold, but who do not necessarily have a diagnosis of PTSD. They are typically referred to in studies that have not used a clinical interview to arrive at a formal diagnosis of PTSD and instead have used only self-report measures of PTSD symptoms.

Health Technology Under Review

Barriers to face-to-face CBT include access (e.g., long wait list for publicly funded CBT), cost (e.g., CBT provided by nonphysician registered professionals in private practice), stigma, and geography (e.g., remote location). Another option is computerized CBT, which includes both offline formats (e.g., CD ROMs) and internet-delivered CBT (iCBT). Increasingly, there is a desire to pursue internet delivery as an option to increase access to treatment.¹²

Internet-delivered CBT is delivered online, remotely from the people using the service, and can either be guided by a therapist or designed for people to complete on their own.¹³ Since CBT uses self-contained modules and clearly defined goals, it is uniquely suited for implementation online.¹³ Various types of iCBT programs have been developed, but they share many features and consist of short-term, skills-based, goal-oriented sessions, typically delivered as 8 to 12 modules.¹³ The modules can be made available online or via a smartphone or tablet application, free or for a fee, and, as mentioned, with or without guided support.¹³

To treat people with PTSD, internet-delivered programs use both CBT and trauma-focused CBT (including CPT) approaches. Internet-delivered CBT interventions for PTSD generally offer less therapist contact than conventional (face-to-face) trauma-focused CBT, and the extent of guidance varies across iCBT interventions.¹⁴

Internet-delivered CBT may not be suitable for people who are suffering from severe symptoms of PTSD or ASD and who require specialized, intensive, multidisciplinary outpatient or inpatient care.

Additionally, iCBT may not be suitable for people with suicidal ideation, a history of self-harm (moderate to severe), or a comorbidity such as personality disorder, bipolar disorder, psychopathology, or psychosis (Anna Baranowsky, PhD, email communication, March 20, 2020).

Although iCBT reduces some barriers to access to mental health services, including stigma, geography, time, and cost, it is important to acknowledge that, because this is a technology-based treatment, people who are not comfortable with technology or who have limited access to a current device or internet services may not be ideal candidates for iCBT.

Regulatory Information

Delivery of iCBT to people with PTSD or ASD does not require regulatory approval from Health Canada.

Ontario, Canadian, and International Context

In Ontario, the delivery of face-to-face CBT for PTSD or ASD from a psychiatrist or other physician trained in psychotherapy is publicly funded.¹⁵ Face-to-face CBT provided by other trained, registered practitioners in regulated professions (e.g., nurses, occupational therapists, psychologists, psychotherapists, social workers) may be free to patients if the services are offered in government-funded hospitals, clinics, or agencies. However, many publicly funded services have long wait lists. Face-to-face CBT provided by registered, regulated nonphysicians in private practice is not publicly funded. Private or workplace insurance may cover the fees, although these plans may not cover the full amount or may provide coverage for only certain types of therapists.¹⁵ Prior to May 5, 2020, iCBT for PTSD or ASD was not publicly funded in Ontario. On that date, iCBT for PTSD or ASD became publicly funded when delivered by certain service providers during the COVID-19 pandemic.^16,17 The Ministry of Health is currently planning how best to deliver iCBT for PTSD and other mental health conditions as part of a structured provincial program.

Based on the 2019 optimal use report by the Canadian Agency for Drugs and Technology in Health (CADTH),¹⁸ the Health Technology Expert Review Panel (HTERP), an advisory body to CADTH, made the following recommendations on the use of iCBT for the treatment of PTSD¹⁹:

• HTERP suggests that there is a potential role for iCBT in the treatment of adults with PTSD; however, HTERP considers that, at present, the relevant evidence is insufficient and of low quality. Better-quality evidence is required to inform future implementation and policy decisions for the use of iCBT in the treatment of adults with PTSD. Future studies should report:

  • Standardized outcomes stratified by patient characteristics

  • Short- and long-term evaluation of clinical effectiveness, safety, and cost-effectiveness

• Regarding the possible implementation of iCBT for the treatment of PTSD, HTERP recommends:

  • Initial diagnostic assessment and referral to establish patient suitability with regard to the appropriateness of the iCBT intervention, including safety and access considerations

  • That the iCBT be therapist guided

  • The use of iCBT as one component of a stepped-care model or in conjunction with other therapies as appropriate

  • Ensuring the appropriateness of programs based on symptom severity, culture, context, and the type of trauma

  • Ensuring that personal health information is appropriately safeguarded and securely managed in accordance with the privacy regulations in the jurisdiction where the care is being provided

In 2019, as part of the Government of Canada's Action Plan on Post-traumatic Stress Injuries, a pilot project was initiated to provide access to iCBT to public safety personnel, such as firefighters, police, and paramedics. It is unclear if this program will be extended to people other than public safety personnel.^20,21

Internationally, several bodies have recommended that iCBT for treatment of PTSD be therapist guided. Among them is the International Society for Traumatic Stress Studies, which recommends therapist-guided iCBT with a trauma focus in the 2019 treatment guidelines for PTSD.²²

Regarding the use of iCBT for adults with a diagnosis of PTSD or clinically important symptoms of PTSD, an expert committee of NICE issued the following as part of its guidance in 2018 (based on members’ clinical experience and the systematic review by NICE)¹¹:

• Consider supported trauma-focused computerized CBT for adults with a diagnosis of PTSD or clinically important symptoms of PTSD who have presented more than 3 months after a traumatic event if they prefer it to face-to-face trauma-focused CBT or EMDR as long as:

  • They do not have severe PTSD symptoms, in particular dissociative symptoms, and

  • They are not at risk of harm to themselves or others

• Supported trauma-focused computerized CBT interventions for adults should:

  • Be based on a validated program

  • Typically be provided over 8 to 10 sessions

  • Involve elaboration and processing of the trauma memories; processing trauma-related emotions; restructuring trauma-related meanings for the individual; helping to overcome avoidance; and re-establishing adaptive functioning (for example, work and social relationships)

  • Include guidance and support from a trained practitioner to encourage people to complete the intervention, give feedback on homework assignments, and review progress and outcomes

The United States Veterans Administration/Department of Defense recommends iCBT for the treatment of PTSD with feedback provided by a qualified facilitator, as an alternative to no treatment.²³

Expert Consultation

We engaged with experts in the specialty areas of psychiatry and psychology to help inform our understanding of aspects of the health technology and our methodologies and to contextualize the evidence.

PROSPERO Registration

This health technology assessment has been registered in PROSPERO, the international prospective register of systematic reviews (CRD 42020193117), available at https://www.crd.york.ac.uk/PROSPERO.

Clinical Evidence

Research Question

What are the effectiveness and safety of internet-delivered cognitive behavioural therapy (iCBT) compared with alternative treatments for adults with post-traumatic stress disorder (PTSD) or acute stress disorder (ASD)?

Methods

Clinical Literature Search

During initial scoping for this report, we identified two recent, relevant, and high-quality systematic reviews of randomized controlled trials (RCTs). One, by the UK National Institute for Health and Care Excellence (NICE) in 2018,²⁴ was a broad review focused on psychological, psychosocial, and other nonpharmacological therapies for the prevention of PTSD. The other, by the Canadian Agency for Drugs and Technologies in Health (CADTH) in 2019,¹⁸ focused on iCBT for the treatment of PTSD and was an update to a 2018 Cochrane systematic review by Lewis et al.²⁵ Our aim was to leverage these previously published systematic reviews with an updated literature search. In addition to updating the CADTH systematic review on iCBT for the treatment of PTSD, we also updated the NICE systematic review specifically focusing on any studies that addressed the use of iCBT to treat ASD or prevent PTSD in people diagnosed with ASD.

We performed a clinical literature search on June 1, 2020, to retrieve studies published from January 1, 2018, until the search date. We used the Ovid interface in the following databases: MEDLINE, Embase, the Cochrane Central Register of Controlled Trials, the Cochrane Database of Systematic Reviews, the Health Technology Assessment Database, the National Health Service Economic Evaluation Database (NHS EED), and APA PsycInfo.

A medical librarian reproduced the search strategy derived from the CADTH report,¹⁸ which used controlled vocabulary (e.g., Medical Subject Headings) and relevant keywords. We used methodological filters to limit retrieval to systematic reviews, meta-analyses, health technology assessments, and randomized controlled trials. The final search strategy was peer-reviewed using the PRESS Checklist.²⁶

We created database auto-alerts in MEDLINE, Embase, and APA PsycInfo and monitored them for the duration of the assessment period. We also performed a targeted grey literature search of health technology assessment agency websites as well as clinical trial and systematic review registries. See Appendix 1 for our literature search strategies, including all search terms.

Eligibility Criteria

STUDIES

Inclusion Criteria

• English-language full-text publications

• Studies published from January 1, 2018, to present. This date was chosen because the date of the last search for the systematic review by NICE (prevention²⁴) was January 29, 2018, while the last search date for the systematic review by CADTH (treatment only¹⁸) was June 21, 2019.

• Randomized controlled trials, randomized crossover trials, systematic reviews, health technology assessments

Exclusion Criteria

• Animal and in vitro studies

• Nonsystematic reviews, narrative reviews, abstracts, editorials, letters, case reports, and commentaries

PARTICIPANTS

Inclusion Criteria

• Adults aged 16 years or older diagnosed with PTSD or ASD

• At least 70% of participants in any given study met diagnostic criteria for either PTSD or ASD according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-III, DSM-III-R, DSM-IV, or DSM-V) or the International Classification of Diseases (ICD-9 or ICD-10), as assessed by the study's clinical interview or a validated questionnaire

Exclusion Criteria

• No restrictions placed on sex or gender, ethnicity, comorbidities, setting, type of traumatic event, severity of symptoms, or time since trauma

INTERVENTIONS

Inclusion Criteria

• Guided or unguided iCBT (trauma-focused or non–trauma-focused) delivered via a computer or mobile device

Exclusion Criteria

• Interventions based on eye movement desensitization and reprocessing (EMDR) alone or online psychoeducation alone

• Interventions using mindfulness-based approaches, apart from mindfulness-based iCBT

COMPARATOR

Inclusion Criteria

• Face-to-face psychological therapy (CBT based, including cognitive processing therapy [CPT]); face-to-face psychological therapy (non–CBT based, e.g., EMDR, supportive therapy, nondirective counselling, psychodynamic therapy, and present-centred therapy); wait-list (i.e., waiting for iCBT); repeated assessment; usual care (as defined by included studies); internet-delivered psychoeducation; internet-delivered psychological therapy (non–CBT)

Exclusion Criteria

• None

OUTCOME MEASURES

• Severity of ASD or PTSD symptoms (as measured by standardized scales)

• Prevention of PTSD after diagnosis with ASD (i.e., proportion of participants with ASD who do not progress to PTSD)

• Diagnosis of PTSD after treatment (i.e., proportion of participants who continued to meet the diagnostic criteria for PTSD following treatment)

• Severity of depression symptoms in people diagnosed with PTSD (as measured by standardized scales)

• Severity of anxiety symptoms in people diagnosed with PTSD (as measured by standardized scales)

• Dropout rates (i.e., proportion of study participants who completed treatment and all post-treatment assessments)

• Quality of life

• Adverse events (e.g., worsening symptoms, relapses to substance use, hospitalizations, suicide attempts, work absenteeism)

TIMING

• Short-term (≤ 3 months) and long-term (> 3 months) outcomes after treatment

SETTING

• No restriction

Literature Screening

A single reviewer conducted an initial screening of titles and abstracts using Covidence²⁷ and then obtained the full texts of studies that appeared eligible for review according to the inclusion criteria. A single reviewer then examined the full-text articles and selected studies eligible for inclusion.

Data Extraction

A single reviewer extracted relevant data on study characteristics and risk-of-bias items using a data form to collect information on the following:

• Source (e.g., citation information, study type)

• Methods (e.g., study design, study duration and years, participant allocation, allocation sequence concealment, blinding, reporting of missing data, reporting of outcomes, whether the study compared two or more groups)

• Outcomes (e.g., outcomes measured, number of participants for each outcome, number of participants missing for each outcome, outcome definition and source of information, unit of measurement, upper and lower limits [for scales], time points at which the outcomes were assessed)

Statistical Analysis

For the outcomes listed above, we reported summary statistics as stated in the included systematic reviews.

We reanalyzed CADTH's subanalyses for one outcome: severity of PTSD symptoms in the iCBT versus wait-list/usual care comparison. These subanalyses looked at the post-treatment effect of trauma-focused or non-trauma-focused iCBT compared with usual care or wait-list and guided or unguided iCBT compared with usual care or wait-list. In our reanalysis, we used the same methods as Lewis et al²⁵ in their Cochrane systematic review, applying a random-effects meta-analysis of continuous data analyzed using standardized mean differences (SMD). The reanalysis of the subgroups was carried out using Cochrane Review Manager software (version 5.3).²⁸ No further reanalyses were conducted.

Critical Appraisal of Evidence

We assessed risk of bias in each systematic review using the Risk of Bias in Systematic Reviews (ROBIS) tool²⁹ (Appendix 2).

We reported the quality of the body of evidence and risk-of-bias assessments for each outcome as reported by the authors of the included systematic reviews (Appendix 2). Both the NICE²⁴ and CADTH¹⁸ systematic reviews assessed risk of bias using the Cochrane Risk of Bias tool³⁰ and the quality of the body of evidence using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Handbook.³¹ The GRADE rating is based on the following considerations: risk of bias, inconsistency, indirectness, imprecision, and publication bias. The overall rating reflects our certainty in the evidence.

Results

Clinical Literature Search

The database search of the clinical literature yielded 1,036 citations published from January 1, 2018, until June 1, 2020. We identified four additional studies from other sources, for a total of 681 after removing duplicates.

We identified one systematic review of RCTs by CADTH,¹⁸ evaluating iCBT for the treatment of PTSD, that met our inclusion criteria. No further studies on the use of iCBT to treat PTSD that met our inclusion criteria were identified in the updated literature search. We identified no studies, either in the broad prevention-focused NICE systematic review²⁴ or in our literature search update, that met our inclusion criteria regarding the use of iCBT to treat ASD or prevent PTSD in individuals diagnosed with ASD. See Appendix 3 for a list of selected studies excluded after full-text review.

Figure 1 presents the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram for the clinical literature search.

Figure 1: PRISMA Flow Diagram—Clinical Search Strategy.

Source: Adapted from Moher et al, 2009.³²

Abbreviation: CADTH, Canadian Agency for Drugs and Technologies in Health; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses; PTSD, post-traumatic stress disorder.

^aResults from the systematic review by CADTH (treatment of PTSD).

Characteristics of Included Studies

CADTH¹⁸ updated a Cochrane systematic review and meta-analysis by Lewis et al²⁵ (published in December 2018) of the effectiveness iCBT for the treatment of PTSD. Since no further studies were identified that met the inclusion criteria in their update, CADTH summarized the Cochrane systematic review and meta-analysis.

Table 1 shows the characteristics of the 10 RCTs^33–42 included in the CADTH¹⁸ review. Eight studies^{33–38,41,42} compared iCBT with a wait-list control group (study participants who receive the same treatment as the experimental group but at a later time). Two studies^39,40 compared iCBT with a non–CBT internet-delivered intervention (e.g., internet-delivered supportive therapy) for the treatment of PTSD.

Table 1:

Characteristics of Included Studies in the Systematic Review by CADTH

Author, Year, Country	Study Design, Objective	Participants, Type of Trauma	Intervention	Comparator	Outcomes
iCBT vs. Wait-List or Usual Care
Krupnick et al, 201736 United States	RCT, open label Follow-up at 12 wk and 24 wk post-treatment To determine feasibility, acceptability, safety, and preliminary effectiveness of an online writing intervention based on principles of CBT, compared with usual treatment	N = 34 (iCBT n = 18; usual care n = 16) Veterans with military-related PTSD Mean time since trauma: NR	Therapist-guided iCBT plus treatment as usual (details not provided) Number of session/modules: 10 Treatment duration: NR Therapist guidance: Support provided by psychologist. Short response and instructions sent by therapist after each writing session Guidance provided online	Treatment as usual with no restrictions Chart review at end of study showed comparison patients received cognitive processing therapy (n = 4), antidepressant drugs (n = 8), or acupuncture (n = 1)	PCL-M PHQ-9 AUDIT
Kuhn et al, 201737 United States	RCT, open label Follow-up at 3 mo and 6 mo post-treatment To evaluate efficacy of a freely available smartphone app (PTSD Coach) that includes CBT-based tools for the treatment of PTSD	N = 120 (iCBT n = 62; wait-list n = 58) Adults who experienced physical assault (n = 56), sexual assault (n = 17), serious accident (n = 25), life-threatening illness or injury (n = 7), disaster exposure (n = 3), combat exposure (n = 4), other events (n = 8) Mean time since trauma: iCBT 9.88 y (SD = 11.59); wait-list 9.77 y (SD = 10.22)	Unguided iCBT. While PTSD Coach included sections with CBT-based tools, program appears to be much less structured than other iCBT software Number of sessions/modules: not divided into sessions Treatment duration: 12 wk	Wait-list control received no intervention during treatment period	Primary outcome: PCL Secondary outcomes: PTSD symptom coping self-efficacy PHQ-8 B-IPF
Lewis et al, 201738 United Kingdom	RCT, single blind Follow-up at 10, 14, 22 wk post-treatment To evaluate a novel trauma-focused, internet-based, guided self-help program for PTSD	N = 42 (iCBT n = 21; wait-list n = 21) Adults who experienced transportation accidents (n = 9), witnessing a sudden, violent, or accidental death (n = 9), traumatic childbirth or stillbirth (n = 8), sexual assault or rape (n = 5), physical attack	Therapist-guided, trauma-focused iCBT. Modules included psychoeducational materials, grounding techniques, relaxation exercises, imaginal exposure, cognitive techniques to address negative thoughts, and graded in vivo exposure work Number of sessions/modules: 8	Wait-list control received no intervention during treatment period	Primary outcome: CAPS-5 Secondary outcomes: PTSD symptoms (PCL-5) Depression symptoms (BDI)
		(n = 4), life threatening illness or injury (n = 3), serious accident (n = 1), learning of the violent death of a loved one (n = 1), seeing a mutilated body (n = 1), being held hostage or detained (n = 1) Mean time since trauma: 37.33 mo (SD = 46.95; range = 3–228)	Treatment duration: 8 wk Therapist guidance: up to 3 hr of therapist assistance (support, monitoring motivation, and problem-solving) provided by a psychiatrist, clinical psychologist, and 3 cognitive behavioural therapists experienced in trauma-focused CBT. Guidance provided in person, by telephone, or by email		Anxiety symptoms (BAI) Signs of harmful drinking or dependence (AUDIT) Perceived social support (SSQ) Functional impairment (SDS)
Miner et al, 201641 United States	RCT, open label Follow-up at 1 mo post-treatment To assess feasibility, acceptability, and preliminary efficacy of iCBT-based app (PTSD Coach) to inform a larger trial	N = 49 (iCBT n = 25; wait-list n = 24) Type of trauma: NR Mean time since trauma: NR	Unguided iCBT (PTSD Coach). The program appears to be much less structured than other iCBT software Number of sessions/modules: not divided into sessions Treatment duration: 4 wk	Wait-list control group received no intervention during treatment period	Primary outcome: PCL-C Secondary outcomes: Acceptability Feasibility
Engel et al, 201533 United States	RCT, single blind Follow-up at 6, 12, and 18 wk post-treatment To examine effectiveness of a nurse-assisted iCBT intervention for war-related PTSD compared with optimized usual care PTSD treatment	N = 80 (iCBT n = 43; usual care n = 37) Type of trauma: war-related trauma (including military sexual trauma) Mean time since trauma: NR	Nurse-guided iCBT (DESTRESS-PC) plus optimized usual primary care PTSD treatment. The non–trauma-focused program included educational information about PTSD, stress, trauma, depression, survivors’ guilt, strategies to manage anger and promote better sleep hygiene, and cognitive reframing techniques Number of sessions/modules: 18 (3/wk for 6 wk) Therapist guidance: participants encouraged to contact study nurses for assistance if needed. Study nurses had access to a private portion of the website where they could monitor compliance and symptom severity	Optimized usual PTSD care consisted of usual primary care PTSD treatment augmented with low-intensity management, feedback to the primary care provider, and training of the clinic providers in management of PTSD Treatment designed to approximate level of PTSD care normally available in primary care while incorporating nonspecific treatment elements of the DESTRESS intervention (e.g., participants received 3 × 15 min phone calls from a DESTRESS nurse)	Primary outcome: PCL-C Secondary outcomes: PHQ-8 PHQ-15 SF-36
Knaevelsrud et al, 201535 Iraq	RCT, open label Follow-up: 5 wk and 3 mo post-treatment To evaluate effectiveness of iCBT for treatment of PTSD in a highly unstable setting (Iraq)	N = 159 (iCBT n = 79; wait-list n = 80) Type of trauma: war-related; specifically killing of a family member (n = 24), sexual violence related to war or sexual abuse (n = 63), violence of war or torture (n = 30), other (e.g., kidnapping, witnessing bomb attacks) n = 42 Mean time since trauma: NR as mean, but reported as % of participants by time since trauma iCBT group < 6 mo: 13%; 6 mo to 3 y: 22%; > 3 y: 65% Wait-list group < 6 mo: 10%; 6 mo to 3 y: 18%; > 3 y: 70%	Therapist-guided, trauma-focused iCBT (translated into Arabic and culturally adapted). Treatment involved structured writing activities over 3 phases: (1) self confrontation with the traumatic event, (2) cognitive restructuring, (3) social sharing Number of sessions/modules: 10 writing assignments Treatment duration: 5 wk Therapist guidance: support provided weekly either in person or via Skype; assignment reminders provided by email and telephone	Wait-list control group received no intervention during treatment period	Primary outcome: PDS Secondary outcomes: HSCL-25 SCL EUROHIS-QOL
Ivarsson et al, 201434 Sweden	RCT, single blind Follow-up: 8 wk and 1 y post-treatment To investigate effectiveness of guided iCBT for treatment of PTSD	N = 62 (iCBT n = 31; wait-list n = 31) Type of trauma: sexual, physical, and/or psychological abuse by partner (n = 14), life-threatening disease (n = 8), severe offense by significant other (perceived as threatening to integrity) n = 6, life-threatening accident (n = 5), nonsexual assault by a stranger (n = 5), murder of close relative (n = 4), nonsexual assault by a family member (n = 3), death of a close relative (n = 3), severe maltreatment in health care (n = 3), life-threatening disease of close relative (n = 2), military combat (n = 2), torture (n = 1), rape by stranger (n = 1), rape by family member (n = 1), tsunami disaster (n = 1) Mean time since trauma: NR	Therapist-guided, trauma-focused iCBT, included psychoeducation, anxiety coping skill training, imaginal exposure, cognitive restructuring Number of sessions/modules: 8, text-based Treatment duration: 8 wk Therapist guidance: support provided by therapist students in their last semester of a 5-y clinical psychology program who had received clinical supervision in CBT. Support consisted of guidance, encouragement, and individual feedback on completed assignments. Therapist feedback was provided once a week via an encrypted web service (through emails). Average time spent with participants: 28 min/wk	Minimal support via internet for the control group. Participants presented weekly with general questions on well-being, stress, and sleep Purpose of this contact was to stay in touch and provide support during the waiting period	Primary outcomes: IES-R PDS Secondary outcomes: BDI BAI QOLI CGI-I
Spence et al, 201142 Australia	RCT, open label Follow-up: 8 wk and 3 mo post-treatment To explore efficacy of iCBT for treatment of PTSD	N = 42 (iCBT n = 23; wait-list n = 19) Type of trauma: various; most participants experienced multiple types. Most common were physical assault (74%), unwanted sexual experience (70%), sexual assault (57%), transportation accidents (52%), other stressful experiences (52%) Mean time since trauma: NR	Therapist-guided, trauma-focused iCBT. Program included psychoeducational materials, strategies for monitoring and challenging thoughts, education and guidelines about practicing exposure and challenging dysfunctional beliefs, and information about relapse prevention Number of sessions/modules: 7 Treatment duration: 8 wk Therapist guidance: support provided by clinical psychologist via telephone, email, and forum posts. Purpose of guidance was to monitor mood and provide support and encouragement. Mean therapist time per participant: 103.91 min (SD = 96.53) over course of program	Wait-list control group received no intervention during treatment period	Primary outcome: PCL-C Secondary outcomes: PHQ-9 GAD-7 SDS
iCBT vs. Non–CBT Internet-Based Interventions
Littleton et al, 201639 United States	RCT, open label Follow-up: 14 and 24 wk post-treatment To determine effectiveness of therapist-facilitated, online CBT tailored to meet needs of rape victims with PTSD	N = 87 (iCBT n = 46; comparison group n = 41) Type of trauma: a completed rape since the age of 14 y Mean time since trauma: NR	Therapist-guided iCBT (From Survivor to Thriver program). Program had 3 phases: (1) psychoeducation relating to PTSD, (2) introduction to the cognitive model and how to identify and respond to distorted or unhelpful automatic thoughts, (3) cognitive behavioural techniques to address specific concerns common among women following sexual assault (e.g., difficulties with trust, self-blame for the assault) Number of sessions/modules: 9 Treatment duration: 14 wk Therapist guidance: provided by doctoral students in psychology via scheduled check-in phone calls about once every 2 wk. Aim of calls to assess participants’ mood, substance use, suicidal or self-harming thoughts, frequency of logging into the program, time spent in enjoyable activities, and to discuss technical problems or distress related to the program	Access to psychoeducational website that contained information from the first 3 treatment modules (relaxation, grounding, coping strategies). Website did not contain multimedia content or interactive exercises from iCBT program Also received scheduled check-in phone calls from doctoral students in psychology about once every 2 wk	Primary outcome: PSS-I Secondary outcomes: Interference (at school, work, relationships, and overall; scored between 0 and 3) CES-D FDAS Therapist competence Therapist and treatment satisfaction (STTS-R) Working alliance (WAI-S)
Litz et al, 200740 United States	RCT, single blind Follow-up: 8 wk, 3 mo, 6 mo post-treatment To evaluate effectiveness of therapist-assisted iCBT vs. internet-based supportive counselling for the treatment of PTSD	N = 45 (iCBT n = 24; comparison group n = 21) Type of trauma: combat exposure (9/11 attack on Pentagon or combat in Iraq or Afghanistan) Mean time since trauma: NR	Therapist-guided, trauma-focused iCBT (DESTRESS). Program included stress management strategies and graduated, self-guided in vivo exposure Number of sessions/modules: 7 trauma writing sessions Treatment duration: 8 wk Therapist guidance: support provided by therapist using initial face-to-face contact, telephone, and email (both scheduled and when requested by participant)	Internet-delivered supportive counselling. This group received monitoring of non– trauma-related concerns and online writing about these experiences. Psychoeducational materials were available. Participants asked to visit website daily to log their symptoms, read about stress and stress management, and write about current concerns. Support provided at participants’ request through initial in-person contact, telephone, and email. Therapists instructed to be empathetic and validating, nondirective and supportive and to focus on non–trauma-related present day concerns	Primary outcome: PSS-I Secondary outcomes: BDI BAI

Abbreviations: AUDIT, Alcohol Use Disorders Identification Test; BAI, Beck Anxiety Inventory; BDI, Beck Depression Inventory; B-IPF, Brief Inventory of Psychosocial Functioning; CADTH, Canadian Agency for Drugs and Technologies in Health; CAPS-5, Clinician-Administered PTSD Scale – 5 Items; CBT, cognitive behavioural therapy; CES-D, Center for Epidemiological Studies – Depression Scale; CGI-I, Clinical Global Impression – Improvement; EUROHIS-QOL, European Health Interview Survey – Quality of Life; FDAS, Four Dimensional Anxiety Scale; GAD-7, Generalized Anxiety Disorder – 7 Items; hr, hour(s); HSCL-25, Hopkins Symptom Checklist – 25 Items; iCBT, internet-delivered cognitive behavioural therapy; IES-R, Impact of Event Scale – Revised; min, minute(s); mo, month(s); N, number of people in study; n, number of people in group; NR, not reported; PCL-5, Post-Traumatic Stress Disorder Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition; PCL-C, PTSD Checklist – Civilian; PCL-M, PTSD Checklist-Military; PDS, Post-traumatic Diagnostic Scale; PHQ-8, Patient Health Questionnaire – 8 Items; PHQ-9, Patient Health Questionnaire – 9 Items; PHQ-15, Patient Health Questionnaire – 15 Items; PSS-I, PTSD Symptom Scale – Interview; QOLI, Quality of Life Inventory; PTSD, post-traumatic stress disorder; RCT, randomized controlled trial; SCL, Symptom Checklist; SD, standard deviation; SDS, Sheehan Disability Scale; SF-36, Medical Outcomes Study Short Form-36; SSQ, Social Support Questionnaire; STTS-R, Satisfaction with Therapy and Therapist Scale – Revised; vs., versus; WAI-S, working Alliance Inventory – Short Form; wk, week(s); y, year(s).

Source: Adapted from CADTH, 2019.¹⁸

We identified no studies that compared iCBT with face-to-face CBT for the treatment of PTSD.

Risk of Bias in the Included Studies

We rated the risk of bias for the CADTH systematic review¹⁸ as low, using ROBIS (Appendix 2,Table A2).

The authors of the Cochrane systematic review²⁵ assessed risk of bias in the individual studies using the Cochrane Risk of Bias tool and certainty of the evidence using GRADE. These, in turn, CADTH reported in their own systematic review and we also report them in our results below.

Overall, there was a high risk of bias in the primary studies, which may have overestimated the treatment effects. Among other risk-of-bias concerns, the studies were open-label trials (unblinded to both researchers and participants), and several had incomplete outcome data due to high rates of dropout without adequate explanation or appropriate handling in the statistical analysis. The risk-of-bias assessment also found imprecision (due to small samples sizes) and inconsistency in effect estimates between the studies.^18,25

Full details of the risk of bias for each study and the GRADE ratings for each outcome are in Appendix 2 (Tables A1, A3, and A4).

Severity of ASD Symptoms

None of the included studies reported this outcome.

Prevention of PTSD After Diagnosis With ASD

None of the included studies reported this outcome.

Severity of PTSD Symptoms

INTERNET-DELIVERED CBT VERSUS WAIT-LIST OR USUAL CARE

From a meta-analysis of eight studies^{33–38,41,42} (N = 560), iCBT was more effective than wait-list or usual care alone in alleviating the severity of PTSD symptoms post-treatment (standardized mean difference [SMD] −0.60; 95% confidence interval [CI] −0.97 to −0.24).¹⁸ CADTH examined the clinical significance of this finding from the Cochrane systematic review.²⁵ Based on a supplemental literature search, CADTH found that there is no widely accepted threshold for defining clinically significant change on the CAPS (Clinician-Administered PTSD Scale) or PCL (Post-Traumatic Stress Disorder Checklist) scales. However, based on proposed thresholds proposed in several publications,^43–46 CADTH suggested that the estimated SMD, despite being statistically significant, does not indicate a clinically significant change in the severity of PTSD symptoms after treatment with iCBT versus wait-list or usual care alone.¹⁸

There was no statistically significant difference in severity of PTSD symptoms at a second follow-up at less than 6 months between groups receiving iCBT versus wait-list or usual care alone (SMD [95% CI] = −0.84 [−2.15 to 0.47]; N = 95, 2 studies^33,38).¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and inconsistency (Appendix 2, Table A3).

As reported by CADTH,¹⁸ the authors of the Cochrane systematic review²⁵ published two subanalyses of their original meta-analysis in a subsequent publication.⁴⁷ These subanalyses assessed the post-treatment effects of trauma-focused iCBT or non–trauma-focused iCBT and therapist-guided iCBT or unguided iCBT. We reanalyzed the subanalyses from the publication by Lewis et al⁴⁷ since the summary effect size for the non–trauma-focused iCBT subgroup was not reported and it was unclear which studies were included in each subgroup. We found the post-treatment effect size was greater for studies that used trauma-focused iCBT compared with wait-list/usual care (SMD [95% CI] = −0.94 [−1.38 to −0.49]; N = 305, 4 studies^34,35,38,42) than non–trauma-focused iCBT compared with wait-list/usual care (SMD [95% CI] = −0.22 [−0.54 to 0.09]; N = 255, 4 studies^33,36,37,41) (Table 2). Guided iCBT had greater effect post-treatment compared with wait-list/usual care (SMD [95% CI] = −0.80 [−1.18 to −0.42]; N = 391, 6 studies^{33–36,38,42}) than unguided iCBT compared with wait-list/usual care (SMD [95% CI] = −0.09 [−0.39 to 0.22]; N = 169, 2 studies^37,41). These results were similar to those reported by Lewis et al.⁴⁷

Table 2:

Severity of PTSD Symptoms From Subanalyses Comparing Trauma-Focused and Non–Trauma-Focused iCBT and Therapist-Guided and Unguided iCBT

Comparison	No. of Participants, Studies	Post-treatment Effect: Summary Estimate, SMD (95% CI)
Trauma-Focused or Non–Trauma-Focused iCBT
Trauma-focused iCBT vs. wait-list or usual care	305, 4 RCTs	-0.94 (-1.38 to −0.49)
Non–trauma-focused iCBT vs. wait-list or usual care	255, 4 RCTs	-0.22 (-0.54 to 0.09)
Guided or Unguided iCBT
Guided iCBT vs. wait-list or usual care	391, 6 RCTs	-0.80 (-1.18 to −0.42)
Unguided iCBT vs. wait-list or usual care	169, 2 RCTs	-0.09 (-0.39 to 0.22)

Abbreviation: CI, confidence interval; iCBT, internet-delivered cognitive behavioural therapy; PTSD, post-traumatic stress disorder; RCT, randomized controlled trial; SMD, standardized mean difference; vs., versus.

Data source: Reanalysis of data reported by primary studies included in Lewis et al, 2019.⁴⁷

INTERNET-DELIVERED CBT VERSUS NON–CBT INTERNET-DELIVERED INTERVENTIONS

There was no significant difference in the severity of PTSD symptoms post-treatment between participants in the iCBT group compared with the group receiving non–CBT treatment delivered online (SMD [95% CI] = −0.08 [−0.52 to 0.35]; N = 82, 2 studies^39,40) or at a second follow-up of less than 6 months (SMD [95%CI] = 0.08 [−0.41 to 0.57]; N = 65, 2 studies^39,40).¹⁸

There was a significant difference in favour of iCBT at the follow-up between 6 and 12 months, as measured with the PTSD Symptom Scale – Interview (PSS-I) (mean difference or MD [95% CI] = −8.83 [−17.32 to −0.34]; N = 18, 1 study⁴⁰). However, the number of participants lost to follow-up was more than 50%.¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and imprecision (Appendix 2, Table A4).

Diagnosis of PTSD After Treatment

INTERNET-DELIVERED CBT VERSUS WAIT-LIST OR USUAL CARE

There was no statistically significant difference for iCBT compared with wait-list for the risk of continued PTSD diagnosis post-treatment (i.e., PTSD diagnosis remained following treatment) (relative risk [RR] [95% CI] = 0.53 [0.28 to 1.00]; N = 62, 1 study³⁴).¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and imprecision (Appendix 2, Table A3).

INTERNET-DELIVERED CBT VERSUS NON–CBT INTERNET-DELIVERED INTERVENTIONS

None of the included studies reported this outcome.

Severity of Depression Symptoms in People Diagnosed With PTSD

INTERNET-DELIVERED CBT VERSUS WAIT-LIST OR USUAL CARE

iCBT was more effective than wait-list or usual care for the reduction of depressive symptoms post-treatment (SMD [95% CI] = −0.61 [−1.17 to −0.05]; N = 425, 5 studies^{34,35,37,38,42}) and at a second follow-up at less than 6 months (MD [95% CI] = −8.95 [−15.57 to −2.33]; N = 42, 1 study³⁸).¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and inconsistency (Appendix 2, Table A3).

INTERNET-DELIVERED CBT VERSUS NON–CBT INTERNET-DELIVERED INTERVENTIONS

There was no statistically significant difference in the severity of depressive symptoms in people treated with iCBT compared with non–CBT internet-delivered interventions either post-treatment (SMD [95% CI] = −0.12 [−0.78 to 0.54]; N = 84, 2 studies^39,40) or at a second follow-up of less than 6 months (SMD [95% CI] = 0.20 [−0.31 to 0.71]; N = 61, 2 studies^39,40).¹⁸

There was a statistically significant difference favouring iCBT for severity of depressive symptoms compared with non–CBT interventions online, when the follow-up was between 6 and 12 months (MD [95% CI] = −8.34 [−15.83 to −0.85]; N = 18, 1 study⁴⁰). However, more than 50% of participants dropped out of the study before follow-up of 6 to 12 months.¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and imprecision (Appendix 2, Table A4).

Severity of Anxiety Symptoms in People Diagnosed With PTSD

INTERNET-DELIVERED CBT VERSUS WAIT-LIST OR USUAL CARE

iCBT was more effective than wait-list or usual care at reducing symptoms of anxiety post-treatment (SMD [95% CI] = −0.67 [−0.98 to −0.36]; N = 305, 4 studies^34,35,38,42) and at a second follow-up at less than 6 months (MD [95% CI] = −12.59 [−20.74 to −4.44]; N = 42, 1 study³⁸).¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and imprecision (Appendix 2, Table A3).

INTERNET-DELIVERED CBT VERSUS NON–CBT INTERNET-DELIVERED INTERVENTIONS

There was no statistically significant difference in severity of anxiety symptoms in people who received iCBT compared with non–CBT internet-delivered interventions, either post-treatment (SMD [95% CI] = 0.08 [−0.78 to 0.95]; N = 74, 2 studies^39,40) or at a second follow-up of less than 6 months (SMD [95% CI] = −0.16 [−0.67 to 0.35]; N = 60, 2 studies^39,40).¹⁸

There was a statistically significant difference in the severity of anxiety symptoms, favouring iCBT compared with non–CBT internet-delivered treatment, when the follow-up was between 6 and 12 months (MD [95% CI] = −8.05 [−15.20 to −0.90]; N = 18, 1 study⁴⁰). However, the number of dropouts lost prior to follow-up between 6 and 12 months was greater than 50%.¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and imprecision (Appendix 2, Table A4).

Dropout Rates

INTERNET-DELIVERED CBT VERSUS WAIT-LIST OR USUAL CARE

There was a statistically significant increase in the proportion of participants who dropped out from iCBT compared with those receiving wait-list or usual care (RR [95% CI] = 1.39 [1.03 to 1.88]; N = 585, 8 studies^{33–38,41,42}).¹⁸ Participants who did not complete post-treatment assessments for any reason (e.g., discontinued, withdrew, lost to follow-up) were considered to have dropped out.

The certainty of the evidence was rated as low, downgraded due to risk of bias (Appendix 2, Table A3).

INTERNET-DELIVERED CBT VERSUS NON–CBT INTERNET-DELIVERED INTERVENTIONS

There was no statistically significant difference in dropout rates between people receiving iCBT compared with non–CBT online interventions (RR [95% CI] = 2.14 [0.97 to 4.73]; N = 1,325, 2 studies^39,40).¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and imprecision (Appendix 2, Table A4).

Quality of Life

INTERNET-DELIVERED CBT VERSUS WAIT-LIST OR USUAL CARE

Quality of life was measured in two studies using the Quality of Life Inventory and the EUROHIS-QUOL scales. iCBT was more effective than wait-list or control for improving quality of life post-treatment (SMD [95% CI] = 0.60 [0.08 to 1.12]; N = 221, 2 studies^34,35).¹⁸

The certainty of the evidence was rated as very low, downgraded due to risk of bias and inconsistency (Appendix 2, Table A3).

INTERNET-DELIVERED CBT VERSUS NON–CBT INTERNET-DELIVERED INTERVENTIONS

None of the included studies reported this outcome.

Adverse Events

INTERNET-DELIVERED CBT VERSUS WAIT-LIST OR USUAL CARE

The RCTs by Lewis et al³⁸ and Krupnick et al³⁶ reported that no adverse events were documented in either the iCBT or wait-list groups.¹⁸

None of the other six included studies reported adverse event data.

The certainty of the evidence was not rated due to lack of data.

INTERNET-DELIVERED CBT VERSUS NON–CBT INTERNET-DELIVERED INTERVENTIONS

None of the included studies reported this outcome.

According to CADTH,¹⁸ one RCT reported that 4.3% (2 of 46 participants) treated with iCBT reported clinically significant increases in depression at post-treatment. However, the authors (Littleton et al³⁹) stated these two participants experienced the death of an immediate family member while completing iCBT and this may have contributed to the feelings of depression.

The certainty of the evidence was not rated due to lack of data.

Summary of Evidence

Table 3 summarizes the CADTH¹⁸ results.

Table 3:

Summary of Results for iCBT Compared With Wait-List/Usual Care or Non–CBT Internet-Delivered Interventions for Adults With PTSD or ASD

Outcome	No. of Participants, Studies	Summary Estimate (95% CI)	GRADE
iCBT vs. Wait-List or Usual Care
Severity of ASD symptoms	No studies reported this outcome
Prevention of PTSD (in individuals diagnosed with ASD)	No studies reported this outcome
Severity of PTSD symptoms, post-treatment			Very low
At first follow-up	560, 8 RCTs	SMD = −0.60 (-0.97 to −0.24). CADTH determined that, while this result favours iCBT and is statistically significant, it is not considered clinically significant
At second follow-up of < 6 mo	95, 2 RCTs	SMD = −0.84 (-2.15 to 0.47). No difference
Diagnosis of PTSD after treatment	62, 1 RCT	RR = 0.53 (0.28 to 1.00). No difference	Very low
Severity of depressive symptoms, post-treatment			Very low
At first follow-up	425, 5 RCTs	SMD = −0.61 (-1.17 to −0.05). Favours iCBT
At second follow-up of < 6 mo	42, 1 RCT	MD = −8.95 (-15.57 to −2.33). Favours iCBT
Severity of anxiety symptoms, post-treatment			Very low
At first follow-up	305, 4 RCTs	SMD = −0.67 (-0.98 to −0.36). Favours iCBT
At second follow-up of < 6 mo	42, 1 RCT	MD = −12.59 (-20.74 to −4.44). Favours iCBT
Dropout rates	585, 8 RCTs	RR = 1.39 (1.03 to 1.88). Favours wait-list or usual care	Low
Quality of life (post-treatment)	221, 2 RCTs	SMD = 0.60 (0.08 to 1.12). Favours iCBTa	Very low
Adverse events	No studies reported this outcome
iCBT vs. Non–CBT Online Interventions
Severity of ASD symptoms	No studies reported this outcome
Prevention of PTSD (in people diagnosed with ASD)	No studies reported this outcome
Severity of PTSD symptoms, post-treatment			Very low
At first follow-up	82, 2 RCTs	SMD = −0.08 (-0.52 to 0.35). No difference
At second follow-up of < 6 mo	65, 2 RCTs	SMD = 0.08 (-0.41 to 0.57). No difference
At follow-up of 6–12 mo	18, 1 RCT	MD = −8.83 (-17.32 to −0.34). Favours iCBT
Diagnosis of PTSD after treatment	No studies reported this outcome
Severity of depressive symptoms, post-treatment			Very low
At first follow-up	84, 2 RCTs	SMD = −0.12 (-0.78 to 0.54). No difference
At second follow-up of < 6 mo	61, 2 RCTs	SMD = 0.20 (-0.31 to 0.71). No difference
At follow-up of 6–12 mo	18, 1 RCT	MD = −8.34 (-15.83 to −0.85). Favours iCBT
Severity of anxiety symptoms, post-treatment			Very low
At first follow-up	74, 2 RCTs	SMD = 0.08 (-0.78 to 0.95). No difference
At second follow-up of < 6 mo	60, 2 RCTs	SMD = −0.16 (-0.67 to 0.35). No difference
At follow-up of 6–12 mo	18, 1 RCT	MD = −8.05 (-15.20 to −0.90). Favours iCBT
Dropout rates	132, 2 RCTs	RR = 2.14 (0.97 to 4.73). No difference	Very low
Quality of life	No studies reported this outcome
Adverse events	No studies reported this outcome

Abbreviations: ASD, acute stress disorder; CADTH, Canadian Agency for Drugs and Technologies in Health; CBT, cognitive behavioural therapy; CI, confidence interval; GRADE, Grading of Recommendations Assessment, Development, and Evaluation; iCBT, internet-delivered cognitive behavioural therapy; MD, mean difference; mo, month(s); PTSD, post-traumatic stress disorder; RCT, randomized controlled trial; RR, relative risk; SMD, standardized mean difference; vs., versus.

^aQuality of life was measured using the Quality of Life Inventory and the EUROHIS-QUOL scales.

Source: Adapted from CADTH, 2019.¹⁸

Ongoing Studies

We are aware of the following ongoing studies that may affect this review:

• Online therapy for post-traumatic stress symptoms in WTC [World Trade Center] responders and survivors. ClinicalTrials.gov identifier: NCT03154151. Available from: https://clinicaltrials.gov/ct2/show/NCT03154151

• An internet-delivered cognitive-behavioral intervention provided soon after trauma: an RCT. ClinicalTrials.gov identifier: NCT04101942. Available from: https://clinicaltrials.gov/ct2/show/NCT04101942

Discussion

Our updated review found some beneficial effects of iCBT compared with wait-list or usual care for the treatment of PTSD (e.g., improvement in quality of life or symptoms of PTSD, depression, or anxiety), although the evidence was very uncertain. Internet-delivered CBT compared with non–CBT (e.g., supportive therapy) interventions delivered online showed little to no effect on PTSD symptoms, depression/anxiety symptoms, or study dropout rates, but here as well the evidence was very uncertain. The GRADE ratings for all the outcomes ranged from low to very low. As noted by both the Cochrane²⁵ and CADTH¹⁸ systematic reviews, the included primary studies had a high risk of bias (e.g., they were open-label trials or had incomplete outcomes data due to high rates of dropout without adequate explanation), potentially overestimating their treatment effects, and they also had imprecision (due to small samples sizes) and inconsistency in effect estimates between the studies. It should be noted that it is impossible to blind study participants and therapists in psychological treatment trials, and therefore all the studies in the review were at high risk of performance bias.²⁵ As is common in studies of psychological therapies, concurrent pharmacotherapy was used by participants in the RCTs and all the studies stated that dosage had been constant for a stipulated duration.²⁵ The time from treatment initiation to final follow-up ranged from 1 month to 1 year, but most of the studies did not measure outcomes beyond 3 months post-treatment.²⁵

Clinical variability among the iCBT programs was high in terms of program content, number of modules, duration, type of support (e.g., telephone, email, face-to-face conversation, or a combination thereof), and frequency of support.¹⁸ In addition, many of the included studies demonstrated a lack of independent evaluation as all but one of the programs were evaluated by the program developers themselves.⁴⁷

No studies were identified that assessed iCBT for the treatment of ASD or the prevention of PTSD in people diagnosed with ASD. Additionally, no studies were identified that compared iCBT with face-to-face CBT to treat PTSD or ASD.

Recruitment bias should also be considered in interpreting these findings. Most studies recruited participants through advertisements rather than via clinical services. People who volunteer for a trial may engage more with iCBT than the broader population of people with PTSD.⁴⁷ They may also have less severe or complex symptoms than people presenting to clinical services.⁴⁷

Interestingly, a subanalysis indicated evidence of greater treatment effect for improvement of PTSD symptoms from the use of trauma-focused iCBT than iCBT without a trauma focus. Lewis et al⁴⁷ suggested this supports the view that iCBT interventions for PTSD benefit from the addition of exposure work (a component of trauma-focused iCBT). Another subanalysis suggested therapist-guided iCBT also increased the treatment effect for improving PTSD symptoms, compared with unguided iCBT.

Strengths and Limitations

As noted by Lewis et al,⁴⁷ the systematic reviews focused on studies where the majority of participants met diagnostic criteria for PTSD either through a clinical interview or self-reported answers on a validated questionnaire. This resulted in the exclusion of studies of traumatized people with subthreshold PTSD symptoms. It may be argued that this may limit the generalizability of the findings; however, Lewis et al⁴⁷ suggested that interventions effective for people meeting the criteria for a diagnosis will also be effective in reducing traumatic stress symptoms among people with a subthreshold condition.

Conclusions

• We found no studies that assessed iCBT for the treatment of ASD or the prevention of PTSD in individuals diagnosed with ASD

• We identified no studies that compared iCBT with face-to-face CBT for the treatment of PTSD

• For iCBT compared with wait-list or usual care:

  • iCBT may improve PTSD symptoms, but the evidence is very uncertain (GRADE: Very low)

  • The evidence suggests iCBT results in a slight increase in dropout rates (GRADE: Low)

  • iCBT may have little to no effect on post-treatment diagnosis of PTSD (that is, the diagnosis persists after treatment), but the evidence is very uncertain (GRADE: Very low)

  • iCBT may improve depression and anxiety symptoms or quality of life, but the evidence is very uncertain (GRADE: Very low)

• For iCBT compared with non–CBT internet-delivered interventions:

  • iCBT may have little to no effect on PTSD symptoms, dropout rates, or depression and anxiety symptoms, but the evidence is very uncertain (GRADE: Very low)

Economic Evidence

Research Question

What is the cost-effectiveness of internet-delivered cognitive behavioural therapy (iCBT) compared with alternative treatments for adults with post-traumatic stress disorder (PTSD) or acute stress disorder (ASD)?

Methods

Economic Literature Search

We performed an economic literature search on June 2, 2020, to retrieve studies published from January 1, 2018, until the search date. To retrieve relevant studies, we developed a search using the clinical search strategy with an economic and costing filter applied.

We created database auto-alerts in MEDLINE, Embase, and APA PsycInfo and monitored them for the duration of the assessment period. We also performed a targeted grey literature search of health technology assessment agency websites, systematic review registries, and the Tufts Cost-Effectiveness Analysis Registry. See Clinical Literature Search, above, for further details on methods used. See Appendix 1 for our literature search strategies, including all search terms.

Eligibility Criteria

STUDIES

Inclusion Criteria

• English-language full-text publications

• Studies published between January 1, 2018, and June 2, 2020

• This date limit was chosen because the date of the last search for the systematic review by the National Institute for Health and Care Excellence (NICE) discussed in the Clinical Evidence section of this report was January 29, 2018,²⁴ and because we identified a systematic review of economic evaluations by the Canadian Agency for Drugs and Technologies in Health (CADTH) with a search date of May 23, 2019¹⁸

• Cost–benefit analyses, cost-effectiveness analyses, cost-minimization analyses, or cost–utility analyses

Exclusion Criteria

• Narrative reviews, letters, editorials, case reports, commentaries, abstracts, posters, or unpublished studies

POPULATION

• Adults aged 16 years or older diagnosed with PTSD or ASD

• At least 70% of participants in any given study met diagnostic criteria for either PTSD or ASD according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-III, DSM-III-R, DSM-IV, or DSM-V) or the International Classification of Diseases (ICD-9 or ICD-10), as assessed by the study's clinical interview or a validated questionnaire

INTERVENTIONS

Inclusion Criteria

• Guided or unguided iCBT (trauma-focused or non–trauma-focused) delivered via a computer or mobile device

Exclusion Criteria

• Interventions based on eye movement desensitization and reprocessing (EMDR) alone or online psychoeducation alone

• Interventions using mindfulness-based approaches, apart from mindfulness-based iCBT

COMPARATOR

Inclusion Criteria

• Face-to-face psychological therapy (CBT based, including cognitive processing therapy [CPT]); face-to-face psychological therapy (non–CBT based, e.g., EMDR, supportive therapy, nondirective counselling, psychodynamic therapy, and present-centred therapy); wait-list (people waiting for CBT); repeated assessment; usual care (people without active iCBT); internet-delivered psychoeducation; internet-delivered psychological therapy (non–CBT)

Exclusion Criteria

• None

OUTCOME MEASURES

• Costs

• Health outcomes (e.g., quality-adjusted life-years)

• Incremental costs

• Incremental effectiveness

• incremental cost-effectiveness ratio

Literature Screening

A single reviewer conducted an initial screening of titles and abstracts using Covidence²⁷ and then obtained the full texts of studies that appeared eligible for review according to the inclusion criteria. A single reviewer then examined the full-text articles and selected studies eligible for inclusion. The reviewer also examined reference lists and consulted content experts for any additional relevant studies not identified through the search.

Data Extraction

We extracted relevant data on study characteristics and outcomes to collect information about the following:

• Source (e.g., citation information, study type)

• Methods (e.g., study design, analytic technique, perspective, time horizon, population, intervention[s], comparator[s])

• Outcomes (e.g., health outcomes, costs, incremental cost-effectiveness ratios)

Study Applicability and Limitations

We determined the usefulness of each identified study for decision-making by applying a modified quality appraisal checklist for economic evaluations originally developed by the NICE in the United Kingdom to inform the development of their clinical guidelines.⁴⁸ We modified the wording of the questions to remove references to guidelines and to make it specific to Ontario. Next, we separated the checklist into two sections. In the first section, we assessed the applicability of each study to the research question (directly, partially, or not applicable). In the second section, we assessed the limitations (minor, potentially serious, or very serious) of the studies that we found to be directly applicable.

Results

Economic Literature Search

The database search of the economic literature yielded 94 citations published between January 1, 2018, and June 2, 2020. We identified five additional studies from other sources. In total, we identified one study (a cost–utility analysis) that met our inclusion criteria.¹⁸ Figure 2 presents the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram for the economic literature search.

Figure 2: PRISMA Flow Diagram—Economic Search Strategy.

Source: Adapted from Moher et al, 2009.³²

Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses.

Overview of the Included Study

The target population of the identified economic evaluation was adults aged 16 years or older diagnosed with PTSD and treated in the community or an outpatient setting.¹⁸ This analysis considered substance abuse and depression as major comorbidities of PTSD.

In the study,¹⁸ the treatments compared (iCBT and control) were based on findings from an accompanying systematic review.²⁵ The study reported on both guided iCBT and unguided iCBT.¹⁸ The analysis assumed that iCBT was provided to adults with PTSD at only one point. The comparator to iCBT was usual care without active iCBT treatment.

Figure 3 shows the structure of CADTH's cohort-level state-transition model.¹⁸ The model simulated the transition of adults with PTSD between health states. The model used a lifetime horizon and a cycle length of 6 months. The following health states were used: PTSD in remission, active PTSD, active PTSD with substance abuse, active PTSD with depression, and death. All people entered the model with active PTSD, with or without comorbidities. Depending on treatment effect, people with PTSD and related comorbidities may improve (“PTSD in remission”) or remain active (“active PTSD”). At any point in the model, people have a probability of death due to age-specific background mortality or psychological distress.

Figure 3: Structure of CADTH's Model.

Source: Adapted from CADTH, 2019.¹⁸

Abbreviations: CADTH, Canadian Agency for Drugs and Technologies in Health; PTSD, post-traumatic stress disorder.

The model assumed that the treatment effect lasted 1 year and was independent of the type of trauma and the number of traumatic exposures experienced.¹⁸ Further, it was assumed that people with comorbidities would maintain the same comorbidities until their PTSD was in remission and that in the remission state, people would not experience new comorbidities. The probability of PTSD recurrence was assumed to be the same for people with and without comorbidities. Another major assumption was that the treatment effect estimates used in the model already accounted for the impact of dropout during treatment.

The treatment effect of iCBT was based on a systematic review²⁵ reported in a separate section of the CADTH report.¹⁸ The researchers converted treatment effect, represented as a standard mean difference (SMD) of symptom score change, into an odds ratio (OR) to estimate the possibility of remission from PTSD.²⁵ The mortality risks and natural history of PTSD, with and without comorbidities, were retrieved from Statistics Canada Life Tables and the literature.^18,49–52

Utility values were derived from the literature.^18,53,54 For the health states of active PTSD, PTSD with depression, and PTSD in remission, utility values were derived from an Australian study by Gospodarevskaya.⁵³ This study (a cost–utility analysis) included 993 patients and assessed health state values with the Assessment of Quality of Life instruments. The CADTH report identified another study reporting utility value for substance abuse, which was elicited using the EuroQol Five Dimensions (EQ-5D) tool from patients diagnosed with alcohol dependence.⁵⁴ This utility value was combined with the health state utility for active PTSD to determine the utility of the joint health state of PTSD with substance abuse.

All costs were reported in 2019 Canadian dollars. The analysis considered cost items including treatment costs and health state costs.¹⁸ Treatment costs were estimated to be $250.25, based on the licence fees for online iCBT modules ($5 per person), the referral cost ($40.15), therapist salaries ($42.70 per hour, with an average of 3 hours assumed), and program maintenance costs ($77 per person). The cost estimates were based on a previous health technology assessment on iCBT for major depression and anxiety disorders conducted by Health Quality Ontario.⁵⁵ For those receiving usual care, no intervention costs were incurred. Treatment costs were assumed to be the same for those with and without comorbidities. Health state costs were estimated for active PTSD ($838/year in 2016 Canadian dollars) based on a study of 1,456 adults aged 65 years or older recruited in a primary care setting in Quebec.⁵⁶ Health state costs were estimated based on inpatient and outpatient medical costs and medication costs retrieved from health administrative databases in Quebec. The CADTH cost–utility analysis also considered societal cost in a scenario analysis, capturing costs related to loss of productivity.¹⁸

The CADTH model estimated lifetime costs and quality-adjusted life-years (QALYs) for adults receiving iCBT and usual care.¹⁸ The analysis ran 5,000 Monte Carlo simulations using a probabilistic sensitivity approach to capture the impact of parameter uncertainty. We obtained the mean values of the outputs through simulations and use these values in the reference case.

Cost-Effectiveness Findings

Table 4 summarizes the main results of this analysis.¹⁸ The results suggested that, from a Canadian health care payer perspective, iCBT was likely to be dominant (i.e., less costly and more effective) compared with usual care over a person's lifetime.

Table 4:

Results of Economic Literature Review—Summary

Author, Year, Country	Analytic Technique, Study Design, Perspective, Time Horizon	Population	Intervention and Comparator	Results
				Health Outcomes	Costsa	Cost-Effectiveness
CADTH, 2019, Canada18	Cost-utility analysis Cohort-level statetransition (Markov) model Publicly funded health care system (Canada) Lifetime horizon with a cycle length of 6 mo	Adults (≥ 16 y) with a primary diagnosis of PTSD treated in the community or an outpatient setting	iCBT versus usual care	Total QALYs (mean per person)  • iCBT: 23.12  • Usual care: 22.81  • Incremental QALY: 0.31 Discount rate: 1.5%	iCBT: $15,998 Usual care: $16,501 Incremental cost: −$504 Discount rate: 1.5%	iCBT dominant (probability of dominance > 95%)b

Abbreviations: CADTH, Canadian Agency for Drugs and Technologies in Health; iCBT, internet-delivered cognitive behavioural therapy; ICER, incremental cost-effectiveness ratio; mo, month(s); PTSD, post-traumatic stress disorder; QALY, quality-adjusted life-year; y, year(s).

^aCosts are in 2019 CAD.

^bDominant = less costly and more effective.

In addition to the reference case, the authors conducted many scenario analyses, including analyses on guided iCBT and unguided iCBT, and the use of a 1-year time horizon (Table 5). These scenario analyses suggested that iCBT remained cost-effective compared with usual care. The scenarios examining guided and unguided iCBT considered different treatment costs and treatment effects. For guided iCBT, the results indicated that this treatment was dominant compared with usual care. For unguided iCBT, the incremental cost-effectiveness ratio (ICER) was $6,042 per QALY gained compared with usual care. For the scenario with a 1-year time horizon, the estimated ICER was $17,435 per QALY gained. A scenario analysis comparing iCBT with internet-delivered non–CBT-based psychological therapy estimated an ICER of $8,624 per QALY gained for iCBT. Overall, compared with usual care, iCBT was found to be dominant or had an ICER below a willingness-to-pay value of $50,000 per QALY gained.

Table 5:

Major Scenario Analyses of the CADTH Model

	Results
Scenario	Health Outcomes	Costsa	Cost-Effectiveness
Scenario 1: Guided iCBT vs. usual care	Total QALYs (mean per person):  • Guided iCBT: 23.30  • Usual care: 22.79 Incremental QALY: 0.51	Guided iCBT: $15,549 Usual care: $16,201 Incremental cost: −$652	iCBT dominantb Probability of iCBT being cost-effective at a WTP of $50,000/QALY: 100%
Scenario 2: Unguided iCBT vs. usual care	Total QALYs (mean per person):  • Unguided iCBT: 22.85  • Usual care: 22.79 Incremental QALY: 0.05	Unguided iCBT: $16,524 Usual care: $16,199 Incremental cost: $325	ICER: $6,042/QALY Probability of iCBT being cost-effective at a WTP of $50,000/QALY: 72%
Scenario 3: 1-year time horizon	Total QALYs (mean per person):  • Guided iCBT: 0.62  • Usual care: 0.60 Incremental QALY: 0.028	Guided iCBT: $999 Usual care: $511 Incremental cost: $487	ICER: $17,435/QALY Probability of iCBT being cost-effective at a WTP of $50,000/QALY: 91%
Scenario 4: iCBT vs. internet-delivered non–CBT-based psychological therapy	Total QALYs (mean per person)  • iCBT: 22.88  • Internet-delivered non–CBT-based psychological therapy: 22.84 Incremental QALY: 0.04	iCBT: $16,634 Internet-delivered non–CBT-based psychological therapy: $16,292 Incremental cost: $342	ICER: $8,624/QALY Probability of iCBT being cost-effective at a WTP of $50,000/QALY: 58%

Abbreviations: CADTH, Canadian Agency for Drugs and Technologies in Health; iCBT, internet-delivered cognitive behavioural therapy; ICER, incremental cost-effectiveness ratio; QALY, quality-adjusted life-year; WTP, willingness-to-pay value.

^aCosts are presented in 2019 CAD.

^bDominant = less costly and more effective.

Applicability and Limitations of the Included Study

The included study¹⁸ was directly applicable to adults with PTSD in the Ontario setting (Appendix 4, Table A5). The study population, intervention, alternative treatment options, and health care system used in the analysis were all similar to our target setting. Thus, the clinical pathway modelled in this analysis is representative of Ontario practice. Second, the cost parameters were appropriately valued from a Canadian health care payer perspective, including parameters based on the Ontario setting (e.g., physician fee, therapist fee, program maintenance costs). Further, the health effects were expressed in QALYs, and both costs and outcomes were discounted at a rate of 1.5% per year.

The model used in the study was assessed to have no apparent methodological limitations (Appendix 4, Table A6).¹⁸ It simulated transitions between health states for adults with PTSD, with and without comorbidities, through a lifetime horizon, which was sufficiently long to reflect all important differences in costs and outcomes. The model considered PTSD remission and death, with clinical inputs based on an accompanying high-quality systematic review and meta-analysis of randomized clinical trials.⁴⁷ It considered important cost components, including therapist guidance costs, licence fees, maintenance costs, and costs related to health states.¹⁸ A variety of sensitivity and scenario analyses were conducted to address important assumptions and uncertain parameters. The results were robust and suggested iCBT is likely to be cost-effective. However, the model was limited by the very low quality of clinical evidence on the effectiveness of iCBT and by limited evidence on comorbidities (i.e., substance abuse and depression) and the natural history of PTSD. Overall, the model was designed and executed with rigor, adhering to the Guidelines for the Economic Evaluation of Health Technologies.⁵⁷

Discussion

We conducted a systematic review to evaluate the economic evidence on the use of iCBT for adults with PTSD or ASD. Our review identified one cost–utility analysis on the use of iCBT for adults with PTSD.¹⁸ The analysis concluded that, compared with usual care, iCBT was cost-effective.

However, the model used in this study¹⁸ was limited by its clinical parameters, as the quality of the clinical evidence was rated as very low due to a very serious risk of bias and serious risk of inconsistency according to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) criteria.³¹ Further, there was limited evidence on comorbidities (i.e., substance abuse and depression) and the natural history of PTSD (i.e., long-term outcomes for PTSD).¹⁸ The included study compared iCBT with control strategies based on findings from an accompanying systematic review⁴⁷ because there was no clinical evidence specific to iCBT programs in Ontario or Canada.¹⁸

Our systematic review found no economic evaluations on the cost-effectiveness of iCBT for ASD, and there is limited clinical evidence on the use of iCBT to treat adults with ASD. This lack of evidence undermines our confidence in developing a de novo model for the ASD setting. This lack of evidence may exist because CBT (including both face-to-face and internet-delivered CBT) is not commonly used to treat ASD, and when it is used, it is used primarily to prevent the onset of PTSD. Thus, the target populations for analyses of iCBT for PTSD and for ASD are similar. Further, the important outcomes in a model for adults treated with iCBT for ASD would be active PTSD, no PTSD, and PTSD in remission. This implies that the clinical evidence on adults with PTSD, and the model structure for adults with PTSD, are likely to be generalizable to adults with ASD at risk of developing PTSD. Therefore, although the CADTH analysis did not directly address the cost-effectiveness of iCBT for adults with ASD, the cost-effectiveness results found for the use of iCBT for adults with PTSD are likely generalizable to adults with ASD at risk for developing PTSD.

Conclusions

In summary, we identified one study that was deemed directly applicable to our research question and had no major methodological limitations. This analysis suggested that iCBT was cost-effective compared with usual care for adults with PTSD in the Canadian setting. However, this analysis was limited by the very low quality of its clinical parameters. We found no direct evidence addressing the cost-effectiveness of iCBT for adults with ASD. The analysis on PTSD may be generalizable to adults with ASD at risk for developing PTSD.

Primary Economic Evaluation

In our economic evidence review, we identified one economic evaluation on the cost-effectiveness of internet-delivered cognitive behavioural therapy (iCBT) for adults with post-traumatic stress disorder (PTSD).¹⁸ The model used a Canadian public health care payer perspective, and we judged it to have no major limitations. Our confidence in the model was further strengthened by the fact that it was based on a systematic review²⁵ that comprehensively summarized the available clinical evidence on the use of iCBT for adults with PTSD. Given this study¹⁸ and its model are directly applicable to our research question, and that the methodology was judged to be rigorous, there was no need to conduct a de novo economic evaluation. Although the analysis was limited by the very low quality of clinical evidence used to populate the model and a lack of evidence on the natural history of PTSD and its comorbidities (i.e., substance abuse and depression), our clinical evidence review did not identify any additional clinical evidence. Thus, a de novo economic evaluation would suffer from the same issues and would likely generate the same or very similar results. We therefore decided not to conduct a primary economic evaluation of iCBT for PTSD.

We did not identify any evidence evaluating the cost-effectiveness of iCBT for adults with acute stress disorder (ASD). Although CBT or iCBT may be used to treat adults with ASD, currently this treatment is used primarily for those with persistent symptoms who are likely to be subsequently diagnosed with PTSD. This implies that the clinical evidence on adults with PTSD may be generalizable to adults with ASD at risk for developing PTSD. Thus, although the CADTH analysis¹⁸ we reviewed did not directly address the cost-effectiveness of iCBT for ASD, its results may be generalizable to this population. However, given the lack of direct clinical evidence on the effects and safety of iCBT for ASD, any generalization will have a degree of uncertainty. We anticipated that a primary economic evaluation of iCBT for adults with ASD would not add to what is already known. Therefore, we did not complete a primary economic evaluation for the cost-effectiveness of iCBT for ASD.

Budget Impact Analysis

Research Question

What is the potential 5-year budget impact for the Ontario Ministry of Health of publicly funding internet-delivered cognitive behavioural therapy (iCBT) for adults with post-traumatic stress disorder (PTSD) or acute stress disorder (ASD)?

Methods

Analytic Framework

We estimated the budget impact of publicly funding iCBT for both PTSD and ASD using the cost difference between two scenarios: (1) current clinical practice with no public funding for iCBT for adults with PTSD or ASD (the current scenario) and (2) anticipated clinical practice with public funding for iCBT for adults with PTSD or ASD (the new scenario). Figure 4 presents the budget impact model schematic.

Figure 4: Schematic Model of Budget Impact.

Abbreviations: ASD, acute stress disorder; iCBT, internet-delivered cognitive behavioral therapy; PTSD, post-traumatic stress disorder.

We conducted a reference case analysis and sensitivity analyses. Our reference case analysis represents the analysis with the most likely set of input parameters and model assumptions. Our sensitivity analyses explored how the results are affected by varying input parameters and model assumptions.

Key Assumptions in Reference Case Analysis

• Estimated volumes of adults eligible for iCBT for PTSD or ASD were based on data reported for PTSD due to a lack of prevalence data for ASD

• A regulated health care professional (a nonphysician therapist) provides iCBT

• The standard of care is usual care without active iCBT treatment

• In the current scenario, due to an existing lack of programmatic funding, no one with PTSD or ASD receives iCBT

• In the new scenario, iCBT is provided one time only within the 5-year time frame to adults with new or existing PTSD or ASD

• Adults who have access to and are currently receiving face-to-face CBT would adhere to this strategy (i.e., in the new scenario, there is no switching from face-to-face CBT to iCBT, and there is no change in the uptake of face-to-face CBT)

• Other than iCBT, people would receive the same care in the current and new scenarios; therefore, no active iCBT treatment costs would be incurred

• For adults responding to iCBT, there is no cost related to maintenance therapy. For adults not responding to iCBT, there is no stepped care

• Adults not responding to iCBT have higher health state costs due to their active PTSD condition

Target Population

Our target population is adults aged 15 years or older diagnosed with PTSD or ASD. Table 6 summarizes the data used to estimate the target population. The age of 15 years was chosen here because population projections obtained from the Ontario Ministry of Finance categorized adults as people aged 15 years or older.⁵⁸

Table 6:

Target Population and Volume of Intervention

		2021	2022	2023	2024	2025
Total population, na		12,746,315	12,952,196	13,143,292	13,318,835	13,479,594
With PTSD diagnosis, nb		599,077	608,753	617,735	625,985	633,541
With access to iCBT and able to engage, nc		443,317	450,477	457,124	463,229	468,820
Current scenario	Uptake of iCBT, %	0	0	0	0	0
	iCBT, n	0	0	0	0	0
	No iCBT, n	443,317	450,477	457,124	463,229	468,820
New scenario	Uptake of iCBT, %	3	6	9	12	15
	iCBT, n	13,300	27,029	41,141	55,587	70,323
	No iCBT, n	430,017	423,449	415,983	407,642	398,497

Abbreviations: iCBT, internet-delivered cognitive behavioral therapy; n, number; PTSD, post-traumatic stress disorder.

^aPopulation projection of adults aged 15 years or older, based on data from the Ontario Ministry of Finance.⁵⁸

^bAnnual prevalence of 4.7%, assuming adults with ASD who need treatment have been included in the number of people diagnosed with PTSD.

^cAdjusted for 11% without access to the internet and 15% unable to engage in iCBT due to a learning disability or substance abuse.

We estimated the annual number of adults with PTSD who would need treatment based on Ontario population projections for 2021 to 2015 and prevalence data from a US study. We first obtained the Ontario population projection of the total number of Ontarians 15 years of age or older from the Ontario Ministry of Finance.⁵⁸ The 12-month prevalence of PTSD has been estimated to be 4.7% in the United States.⁵⁹ Since iCBT is used in the ASD population primarily for those with persistent symptoms, we assumed that adults with ASD being treated with iCBT would have symptoms lasting longer than 1 month and would subsequently be diagnosed with PTSD.

We also accounted for people without access to iCBT and those without the cognitive capacity to engage with an iCBT program. Approximately 89% of Canadians have access to a computer and routine access to the internet.⁶⁰ Therefore, we assumed 11% of Ontario adults with PTSD or ASD would not have access to an iCBT program. Further, we assumed that 5% of adults with PTSD or ASD would be unable to engage in an iCBT program due to a learning disability and that 10% would be unable to engage owing to substance abuse. For these individuals, iCBT may not be an appropriate treatment option.^55,61 For people eligible for and having access to an iCBT program, we assumed that the uptake of iCBT would increase from 3% to 15% over 5 years.⁵⁵ This assumption was based on small capacity and low current uptake of CBT (including both face-to-face CBT and iCBT) for adults with PTSD in Ontario.^62,63

Current Intervention Mix

There is currently no programmatic public funding in Ontario for iCBT for adults with PTSD or ASD. Therefore, we assumed that adults with PTSD or ASD do not receive iCBT in the current scenario. There is no intervention mix of iCBT and face-to-face CBT in the current scenario, and there are no changes in the costs of usual care with the introduction of iCBT.

Uptake of the New Intervention and New Intervention Mix

There is no empirical evidence on the potential uptake rate of iCBT programs for PTSD or ASD. However, iCBT is not intended to displace face-to-face CBT (see the Clinical Evidence section). Rather, it is meant to be provided as a supplementary treatment modality for those without access to face-to-face CBT. Thus, we assumed there will be no change to the uptake of face-to-face CBT, and the costs related to face-to-face CBT will be the same in both the current and new scenarios. We assumed the introduction of iCBT would expand service capacity, providing mental health care to those who would otherwise have no access. This means the number of adults using iCBT would be the main driving factor for the cost difference between the current and new scenarios. In the reference case, assuming a 3% uptake rate for iCBT, we estimated that the number of adults with PTSD in year 1 would be approximately 13,300. The estimated volume of adults with PTSD or ASD using iCBT over a 5-year period accounted for the uptake rate increase in subsequent years, while the prevalence of PTSD or ASD, access to iCBT, and eligibility for iCBT programs remained constant.

Resources and Costs

We based our budget impact analysis on the CADTH model-based cost–utility analysis, which was summarized in the Economic Evidence section.¹⁸ We derived cost parameters (related to treatment costs and health state costs) from the CADTH analysis. We used CADTH's model to account for (1) the difference in health state costs due to iCBT treatment effect (the iCBT treatment effect was based on the same body of evidence in both the CADTH analysis and our clinical evidence review); and (2) the natural history of PSTD and its comorbidities. We undertook the budget impact analysis differently from the CADTH cost–utility analysis in two respects: (1) We used undiscounted costs; and (2) where applicable, we inflated cost parameters to 2020 Canadian dollars.

Treatment costs were estimated based on the patient referral cost ($43.43), licence fees for online iCBT modules ($5), therapist salaries ($42.70), and maintenance costs ($77.00), for a total of $253.53 (Table 7). Notably, the reference case assumed an average of 3 hours of electronically provided therapist support; thus, the reference case represented a mixture of unguided and guided iCBT. We also explored the budget impact of unguided and guided iCBT separately, and we examined the budget impact of adding more therapist support hours for guided iCBT.

Table 7:

Costs Parameters for Budget Impact Analysis

Variable	Unit Cost, $a	Duration or Quantity, n	Total Cost, $	Reference(s)
Physician and Therapist Costs
Primary care physician	86.85	0.5b	43.43	Ontario Schedule of Benefits (Code: K197)64
e-Therapist hourly rate	42.70	Mean (SD): 3 (3)c	128.10	CADTH, 201918; HQO, 201955
Psychotherapy Program
Licence cost, per patient	5.00	1	5.00	CADTH, 201918; HQO, 201955
iCBT program maintenance cost, per patient	77.00	1	77.00
Total cost of iCBT (reference case)	-	-	253.53
Annual Health State Costs
Active PTSD	879.83 (SD: 175.97)d,e	-	879.83	CADTH, 201918; Lamoureux-Lamarche 201656
Active PTSD with comorbidities	No additional cost
PTSD in remission (1 follow-up visit)	86.85	1	86.85	Ontario Schedule of Benefits (Code: K197)64

Abbreviations: CADTH, Canadian Agency for Drugs and Technologies in Health; HQO, Health Quality Ontario (now Ontario Health); iCBT, internet-delivered cognitive behavioural therapy; n, number; PTSD, post-traumatic stress disorder; SD, standard deviation.

^aAll costs are presented in 2020 Canadian dollars.

^bIt was assumed that half of those receiving iCBT were referred by a primary care physician. A beta distribution was applied for the probability of being referred to iCBT by a primary care physician in the probabilistic sensitivity analysis, with α = 50 and β = 50.

^cA gamma distribution was used in the probability sensitivity analysis, with a mean of 3 and a standard deviation of 0.3.

^dA gamma distribution was used in the probability sensitivity analysis, assuming the standard deviation equals 20% of the mean value.

^eThe cost was estimated to be $838 in 2016 CAD; $1 in 2016 CAD = $1.0499 in 2020 CAD.⁶⁷

Health state costs were estimated for active PTSD and PTSD in remission. The cost for active PTSD was estimated to be $879.83.⁵⁶ This estimate was based on adults aged 65 years or older recruited in primary medical clinics in the province of Quebec.^18,56 We assumed this figure was applicable to adults experiencing active PTSD in the Ontario setting. Assuming one physician visit per year for those with PTSD in remission, the annual physician visit cost was estimated to be $86.85.⁶⁴

We assumed that the new scenario of publicly funded iCBT would expand capacity and access to mental health care but would not replace existing face-to-face CBT treatment. Therefore, the budget impact of publicly funding iCBT represents the additional cost of funding iCBT but does not impact existing funding for face-to-face CBT. The probabilities of the health states of active PTSD and PTSD in remission were based on the transition between health states in the first 5 years of funding. Compared with usual care without active iCBT treatment, the odds ratio (OR) of PTSD remission in the first year for adults receiving iCBT was 2.97, with a 95% confidence interval (CI) of 1.55 to 5.81.¹⁸

We estimated the cost of publicly funding iCBT over the next 5 years (2021–2025). Table A7 (Appendix 5) summarizes the annual cost per person for those receiving iCBT and those receiving usual care. We determined our estimates using undiscounted direct medical costs. We estimated unit costs from year 1 to year 5 for adults receiving usual care and for those receiving iCBT. Specifically, the treatment cost of $253.53 was applied only to those receiving iCBT in the first year of iCBT treatment. The overall health state cost component combined health state–related costs for those with active PTSD and those with PTSD in remission. The CADTH cost–utility analysis reported the mean values of the probabilistic sensitivity analysis based on 5,000 Monte Carlo simulations as the reference case.¹⁸ To be consistent with costs estimated within this approach, our reference case also reflected the probabilistic results. And, as suggested in the current guidelines,^65,66 we also reported the deterministic results in a scenario analysis.

Analysis

Our reference case analysis represented the analysis with the most likely set of input parameters and model assumptions. It estimated the budget impact of the most likely iCBT strategy, including both unguided and guided iCBT, compared with usual care. All costs were reported in 2020 Canadian dollars; where necessary, costs were inflated to 2020 Canadian dollars using the Consumer Price Index for Health and Personal Care in Canada.⁶⁷ To explore how the results are affected by varying input parameters and model assumptions, we conducted the following scenario analyses (Appendix 5, Table A8):

• Scenario 1: Results based on deterministic analysis

  • Our reference case was conducted using the probabilistic approach; scenario 1 estimates the budget impact using a deterministic analysis approach

• Scenario 2: Unguided iCBT

  • The iCBT program in the reference case includes 3 hours of therapist time provided electronically (“e-therapist time”) to account for a mix of guided and unguided iCBT programs. We evaluated the budget impact for a scenario in which all iCBT programs would be unguided, which would cost less than the reference case (i.e., 0.5 e-therapist hours vs. 3 hours in the reference case). Unguided iCBT led to a smaller clinical benefit compared with the reference case (OR of PTSD remission for unguided iCBT compared with no treatment, 1.28 [95% CI, 0.67 to 2.48] vs. the reference case: OR, 2.97 [95% CI, 1.55 to 5.81])¹⁸

• Scenario 3: Guided iCBT

  • We evaluated the budget impact for a scenario in which all iCBT programs would be guided by a therapist, which would cost more than the reference case (i.e., 3.5 e-therapist hours vs. 3 hours in the reference case). Guided iCBT led to a larger clinical benefit compared with the reference case (OR of PTSD remission for guided iCBT compared with no treatment, 4.27 [95% CI, 2.14 to 8.50] vs. the reference case: OR, 2.97 [95% CI, 1.55 to 5.81])¹⁸

• Scenario 4: Moderate uptake rates

  • We assumed moderate uptake rates in this scenario, increasing from 10% to 30% over the next 5 years, with a 5% annual increase

• Scenario 5: High uptake rates

  • We assumed high uptake rates in this scenario, increasing from 30% to 50% over the next 5 years, with a 5% annual increase

• Scenario 6: Varying number of e-therapist hours

  • In the reference case, we applied 3 hours of e-therapist support. In scenario 6, we explored the robustness of the budget impact by varying the number of e-therapist hours from 4 to 10. With more e-therapist hours invested, we assumed the treatment effect would be the same as for guided iCBT (scenario 3)

• Scenario 7: Mixed program of face-to-face CBT and iCBT

  • In scenario 7, we explored the impact of introducing a mixed program, including both face-to-face CBT and iCBT, for all adults receiving iCBT. For the cost of this program, we assumed 6 therapist hours and 5 supervision hours for the face-to-face CBT component and 0.5 e-therapist hours for the iCBT component. The hourly rate for face-to-face nonphysician therapists was estimated to be $86.25.^18,55,64 We assumed the same uptake rate as for the reference case. The treatment effect was assumed to be the same as for the guided iCBT program due to the therapist support received in face-to-face CBT

Internal Validation

A secondary health economist conducted formal internal validation. This process included checking for errors and ensuring the accuracy of parameter inputs and equations in the budget impact analysis.

Results

Reference Case

Table 8 summarizes the results of our reference case budget impact analysis. In the new scenario, adopting iCBT (including both guided and unguided programs) at an uptake rate of 3% to 15% over 5 years would lead to additional costs of $2.43 million in year 1 to $2.37 million in year 5, for a total additional cost of $16.53 million. The total budget impact was driven by the increased treatment costs associated with implementing iCBT. Given that these costs could be partially offset by savings in health state costs in later years, the annual budget impact decreased in years 4 and 5, although treatment costs increased. If iCBT treatment-related costs alone are considered (i.e., excluding the health state–related cost offset), the budget impact would be an additional cost of $3.37 million in year 1 to $17.84 million in year 5, for a total additional cost of $52.61 million over 5 years. The budget impact results considering iCBT treatment-related costs alone did not account for cost savings due to improved health outcomes, which would spread out across the health system (e.g., fewer family physician visits).

Table 8:

Budget Impact Analysis Results

	Budget Impact, $ Milliona,b
Scenario	Year 1	Year 2	Year 3	Year 4	Year 5
Current scenario: treatment costsc	0	0	0	0	0
Current scenario: health state costsd	374.17	735.06	1,083.90	1,421.72	1,749.45
Current scenario: total costse	374.17	735.06	1,083.90	1,421.72	1,749.45
New scenario: treatment costsc	3.37	6.86	10.44	14.10	17.84
New scenario: health state costsd	373.23	732.01	1,077.67	1,411.33	1,733.99
New scenario: total costse	376.60	738.87	1,088.11	1,425.43	1,751.83
Budget impact	2.43	3.81	4.21	3.71	2.37

^aAll costs are presented in 2020 Canadian dollars.

^bResults may appear inexact due to rounding.

^cTreatment costs included iCBT costs. For the current scenario, treatment costs were estimated to be $0; for the new scenario, treatment costs represented the cost for those receiving iCBT each year.

^dHealth state costs included costs due to PTSD recurrence and for follow-up visits for PTSD in remission (one visit per person), for both those who received iCBT and those who did not. For each year, the total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered the analysis in previous years. For example, the calculations for year 2 are as follows. Current scenario: ($844.03 × 450,477) + ($772.78 × 0) + ($800.43 × 443,317) + ($715.82 × 0) = $735.06 million; new scenario: ($844.03 × 423,449) + ($772.78 × 27,029) + ($800.43 × 430,017) + ($715.82 × 13,300) = $732.01 million (according to the annual health state costs per person provided in Appendix 5, Table A7, and volume of uptake provided in Table 6).

^eFor each year, total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered the analysis in previous years. For example, in year 3, the total costs represent the total costs of three cohorts: health state costs for adults who entered the analysis in years 1 and 2, and both treatment costs and health state costs for those who entered the analysis in year 3.

Scenario Analysis 1: Deterministic Results

Table 9 summarizes the deterministic results of the budget impact analysis. Publicly funding iCBT for adults with PTSD or ASD (including both unguided and guided programs) over the next 5 years would require an additional $2.31 million in year 1 and $1.40 million in year 5, for a total cost increase of $13.77 million over 5 years. When iCBT treatment-related costs alone were considered, compared with the reference case, the budget impact did not change. This suggests that the driving factor for the discrepancy between the probabilistic and deterministic approaches was changes in health state costs.

Table 9:

Budget Impact Analysis Deterministic Results

	Budget Impact, $ Milliona,b
Scenario	Year 1	Year 2	Year 3	Year 4	Year 5
Current scenario: treatment costsc	0	0	0	0	0
Current scenario: health state costsd	370.93	725.44	1,065.96	1,394.49	1,712.73
Current scenario: total costse	370.93	725.44	1,065.96	1,394.49	1,712.73
New scenario: treatment costsc	3.37	6.85	10.43	14.09	17.83
New scenario: health state costsd	369.86	722.07	1,059.18	1,383.32	1,696.31
New scenario: total costse	373.24	728.92	1,069.61	1,397.41	1,714.13
Budget impact	2.31	3.48	3.65	2.92	1.40

^aAll costs are presented in 2020 Canadian dollars.

^bResults may appear inexact due to rounding.

^cTreatment costs included iCBT costs. For the current scenario, treatment costs were estimated to be $0; for the new scenario, treatment costs represented the cost for those receiving iCBT each year.

^dHealth state costs included costs due to PTSD recurrence and for follow-up visits for PTSD in remission (one visit per person).

^eFor each year, total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered the analysis in previous years. For example, in year 3, the total costs represent the total costs of three cohorts: health state costs for adults who entered the analysis in years 1 and 2, and both the treatment costs health state costs for adults who entered the analysis in year 3.

Scenarios 2 and 3: Unguided and Guided iCBT

Table 10 summarizes the scenario results of introducing unguided and guided iCBT. We assumed that the unguided iCBT scenario included 0.5 hours of e-therapist time and that the guided iCBT scenario included 3.5 hours, compared with 3 hours for the reference case. With the uptake of iCBT increasing from 3% in year 1 to 15% in year 5, publicly funding unguided iCBT led to a cost increase of $1.79 million in year 1 to $7.65 million in year 5, for a total cost increase of $24.22 million over 5 years. Implementing guided iCBT led to a cost increase of $2.12 million in year 1 and a cost savings of $5.61 million in year 5, for a total cost savings of $1.22 million over 5 years. If iCBT treatment-related costs alone were considered, the budget impact would be an additional cost of $1.95 million in year 1 to $10.32 million in year 5 for unguided iCBT, and $3.66 million in year 1 to $19.33 million in year 5 for guided iCBT. The driving factor for cost savings for the guided iCBT scenario was lower health state costs due to higher probabilities of people experiencing remission from PTSD.

Table 10:

Budget Impact Analysis—Unguided and Guided iCBT

	Budget Impact, $ Milliona,b,c
Scenario	Year 1	Year 2	Year 3	Year 4	Year 5
Unguided iCBT
Current scenario: treatment costsc	0	0	0	0	0
Current scenario: health state costsd	374.20	735.16	1,084.10	1,422.08	1,750.02
Current scenario: total costse	374.20	735.16	1,084.10	1,422.08	1,750.02
New scenario: treatment costsd	1.95	3.97	6.04	8.16	10.32
New scenario: health state costse	374.04	734.63	1,083.03	1,420.29	1,747.35
New scenario: total costsf	375.99	738.60	1,089.07	1,428.45	1,757.67
Budget impact	1.79	3.44	4.97	6.37	7.65
Guided iCBT
Current scenario: treatment costsd	0	0	0	0	0
Current scenario: health state costse	374.21	735.16	1,084.09	1,422.03	1,749.91
Current scenario: total costsf	374.21	735.16	1,084.09	1,422.03	1,749.91
New scenario: treatment costsd	3.66	7.43	11.31	15.28	19.33
New scenario: health state costse	372.67	730.23	1,074.04	1,405.27	1,724.98
New scenario: total costsf	376.32	737.66	1,085.35	1,420.55	1,744.31
Budget impact	2.12	2.49	1.25	-1.48	-5.61

^aAll costs are presented in 2020 Canadian dollars.

^bNegative costs indicate savings.

^cResults may appear inexact due to rounding.

^dTreatment costs included iCBT costs. For the current scenario, treatment costs were estimated to be $0; for the new scenario, treatment costs represented the cost for those receiving iCBT each year.

^eHealth state costs included costs due to PTSD recurrence and for follow-up visits for PTSD in remission (one visit per person).

^fFor each year, total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered the analysis in previous years. For example, in year 3, the total costs represent the total costs of three cohorts: health state costs for those who entered the analysis in years 1 and 2, and both treatment costs and health state costs for those who entered the analysis in year 3.

Scenarios 4 and 5: Moderate and High Uptake

Table 11 presents the budget impact analyses for the scenarios of moderate and high uptake. We assumed that in the moderate uptake scenario, the uptake of iCBT increased from 10% to 30% over 5 years. For this scenario, the budget impact of introducing iCBT would be a cost increase of $8.09 million in year 1 to $1.67 million in year 5, for a total cost increase of $32.64 million over 5 years.

Table 11:

Budget Impact Analysis—Moderate and High Uptake

	Budget Impact, $ Milliona,b,c
Scenario	Year 1	Year 2	Year 3	Year 4	Year 5
Moderate Uptake
Current scenario: number of adults receiving iCBT	0	0	0	0	0
Current scenario: treatment costsc	0	0	0	0	0
Current scenario: health state costsd	374.17	735.06	1,083.90	1,421.72	1,749.45
Current scenario: total costse	374.17	735.06	1,083.90	1,421.72	1,749.45
New scenario: treatment costsd	11.25	17.14	23.19	29.38	35.68
New scenario: health state costse	371.01	726.50	1,068.31	1,397.61	1,715.44
New scenario: total costsf	382.26	743.64	1,091.50	1,426.99	1,751.12
Budget impact	8.09	8.58	7.60	5.27	1.67
High Uptake
Current scenario: number of adults receiving iCBT	0	0	0	0	0
Current scenario: treatment costsd	0	0	0	0	0
Current scenario: health state costse	374.17	735.06	1,083.90	1,421.72	1,749.45
Current scenario: total costsf	374.17	735.06	1,083.90	1,421.72	1,749.45
New scenario: treatment costsd	33.74	40.00	46.39	52.88	59.47
New scenario: health state costse	364.70	712.57	1,047.46	1,370.44	1,682.51
New scenario: total costsf	398.44	752.57	1,093.84	1,423.32	1,741.98
Budget impact	24.26	17.51	9.95	1.60	-7.47

^aAll costs are presented in 2020 Canadian dollars.

^bNegative costs indicate savings.

^cResults may appear inexact due to rounding.

^dTreatment costs included iCBT costs. For the current scenario, treatment costs were estimated to be $0; for the new scenario, treatment costs represented the cost for those receiving iCBT each year.

^eHealth state costs included costs due to PTSD recurrence and for follow-up visits for PTSD in remission (one visit per person).

^fFor each year, total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered the analysis in previous years. For example, in year 3, the total costs represent the total costs of three cohorts: health state costs for those who entered the analysis in years 1 and 2, and both the treatment costs and health state costs for those who entered the analysis in year 3.

If iCBT treatment-related costs alone were considered, the budget impact would be an additional $11.25 million in year 1 to $35.68 million in year 5, for a total additional cost of $116.64 million over 5 years.

In the high uptake scenario, with uptake increasing from 30% to 50%, introducing iCBT led to a cost increase of $24.26 million in year 1 and a cost savings of $7.47 million in year 5, for a total cost increase of $45.85 million over 5 years. If iCBT treatment-related costs alone were considered, the budget impact would be an additional cost of $33.74 million in year 1 to $59.47 million in year 5, for a total additional cost of $232.48 million over 5 years.

Scenario 6: Varying Number of e-Therapist Hours

In scenario 6, we explored the influence of increased e-therapist hours on the budget. In this scenario analysis, we assumed the treatment effect of increasing the number of e-therapist hours for guided iCBT would be the same as seen in scenario 3 (i.e., guided iCBT), which accounted for 3.5 hours of e-therapist support. Table 12 summarizes the analysis results for scenario 6. Providing 4 hours of e-therapist support led to a cost increase of $2.39 million in year 1 and a cost savings of $4.31 million in year 5, for a total additional cost of $2.73 million over 5 years. The cost savings observed in years 4 and 5 resulted from a cost offset due to an increase in PTSD remission. As the number of e-therapist hours increased, the treatment costs and budget impact also increased. When the number of e-therapist hours increased from 5 to 10, the budget impact of guided iCBT over 5 years increased from an additional $11.58 million to $55.86 million.

Table 12:

Budget Impact Analysis—Varying Number of e-Therapist Hours

	Budget Impact, $ Milliona,b,c
Number of e-Therapist Hours Provided	Year 1	Year 2	Year 3	Year 4	Year 5
4	2.39	3.03	2.05	-0.43	-4.31
5	2.96	4.18	3.81	1.94	-1.31
6	3.52	5.34	5.56	4.32	1.70
7	4.09	6.49	7.32	6.69	4.70
8	4.66	7.65	9.08	9.06	7.70
9	5.23	8.80	10.83	11.44	10.71
10	5.80	9.95	12.59	13.81	13.71

^aAll costs are presented in 2020 Canadian dollars.

^bNegative costs indicate savings.

^cFor each year, total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered the analysis in previous years. For example, in year 3, the total costs represent the total costs of three cohorts: health state costs for those who entered the analysis in years 1 and 2, and both the treatment costs and health state costs for those who entered the analysis in year 3.

Table A9 (Appendix 5) summarizes the breakdown of costs included in the budget impact calculations. If guided iCBT treatment-related costs alone were considered, the budget impact over 5 years would range from an additional $61.44 million (4 hours of e-therapist support) to $114.57 million (10 hours of e-therapist support).

Scenario 7: Mixed Program of Face-to-Face CBT and Unguided iCBT

Table 13 summarizes the budget impact results for a scenario in which a mixed program combining face-to-face CBT and unguided iCBT is introduced. Implementing this program led to an additional cost of $16.19 million in year 1 to $68.86 million in year 5, for a total additional cost of $218.34 million over 5 years. Our analysis suggested that the budget impact was driven by the cost of the face-to-face component.

Table 13:

Budget Impact Analysis—Mixed Program of Face-to-Face CBT and Unguided iCBT

	Budget Impact, $ Milliona,b
Scenario	Year 1	Year 2	Year 3	Year 4	Year 5
Current scenario: treatment costsc	0	0	0	0	0
Current scenario: health state costsd	374.15	735.02	1,083.84	1,421.64	1,749.37
Current scenario: total costse	374.15	735.02	1,083.84	1,421.64	1,749.37
New scenario: face-to-face component costsc	15.78	32.06	48.80	65.94	83.42
New scenario: iCBT component costsc	1.95	3.97	6.04	8.16	10.32
New scenario: health state costsd	372.62	730.10	1,073.81	1,404.92	1,724.48
New scenario: total costse	390.35	766.13	1,128.65	1,479.02	1,818.23
Budget impact	16.19	31.10	44.81	57.37	68.86

^aAll costs are presented in 2020 Canadian dollars.

^bResults may appear inexact due to rounding.

^cTreatment costs included face-to-face CBT costs and unguided iCBT costs. For the current scenario, treatment costs were estimated to be $0; for the new scenario, we have presented the breakdown of treatment costs.

^dHealth state costs included costs due to PTSD recurrence and for follow-up visits for PTSD in remission (one visit per person).

^eFor each year, total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered analysis in previous years. For example, in year 3, the total costs represent the total costs of three cohorts: health state costs for those who entered the analysis in years 1 and 2, and both the treatment costs and health state costs for those who entered the analysis in year 3.

Discussion

We conducted a model-based budget impact analysis to examine the range of costs related to publicly funding iCBT as a treatment option for adults with PTSD or ASD. Assuming an annual prevalence of 4.7%, and the uptake rate increasing from 3% to 15% over 5 years, publicly funding iCBT in Ontario would lead to additional costs of $2.43 million in year 1 to $2.37 million in year 5, for a total additional cost of $16.53 million over 5 years.

Currently, there is limited access to CBT for adults with PTSD or ASD in Ontario. However, the Ontario government has recently launched the MINDABILITY program,⁶⁸ which is designed to provide access to CBT through internet-based modules, personal workbooks, telephone coaching, and online clinical counselling. Our budget impact analysis may help estimate the resources needed to deliver these online resources for adults with PTSD or ASD.

For our analyses, we adapted a previously published CADTH model.¹⁸ While the CADTH cost–utility analysis suggested that iCBT (guided or unguided) was cost saving compared with usual care over a person's lifetime, in the short term, it would lead to a cost increase. This finding underscores the importance of accounting for the treatment costs associated with iCBT (either guided or unguided) in assessing the budget impact. When the total budget impact of iCBT treatment-related costs alone are considered, the total budget impact over the next 5 years would be an additional $52.61 million.

We conducted scenario analyses to examine differences between providing guided and unguided iCBT. Our budget impact analysis found that publicly funding guided iCBT would be cost saving over the next 5 years. This is because we assumed that guided iCBT would be more effective than unguided iCBT; accordingly, cost savings would result from an increased clinical benefit that would lead to lower health state–related costs. However, the evidence on the treatment effect of iCBT for PTSD is of very low quality (see the Clinical Evidence section); moreover, there is currently no long-term evidence on the treatment effect of iCBT. As such, the results from our budget impact analysis for guided iCBT should be interpreted with caution. A further reason to interpret our results with caution is that our scenario analysis for guided iCBT assumed 3.5 hours of e-therapist support, while more e-therapist hours may be needed for a trauma-focused iCBT program. However, we are unsure if an added treatment effect would be associated with an increase in e-therapist time for guided iCBT.

In further scenario analyses, we examined potential differences in budget impact associated with varying the number of e-therapist hours provided in a guided iCBT program (i.e., 4–10 hours vs. 3 hours in the reference case). If 10 hours of e-therapist support are provided, the 5-year budget impact would be an additional $55.86 million, even after assuming a larger treatment effect than found with unguided iCBT. However, given that the attrition rate for any psychotherapeutic intervention is typically higher in the real world than it is in randomized controlled trials and that people with PTSD may experience clinical benefit before completing a full course of iCBT (Peter Farvolden, PhD, email communication, September 5, 2020), 3 to 5 hours of e-therapist support would probably represent real-world clinical practice.

If a mixed program including face-to-face CBT and iCBT were introduced and publicly funded, the 5-year budget impact would be an additional cost of more than $200 million because more therapist support and supervision hours would be necessary.

We also assessed potential differences in budget impact associated with different rates of program uptake. As expected, the budget impact would increase as the rate of uptake increases. In our high uptake scenario, we assumed that uptake increased from 30% in year 1 to 50% in year 5. We considered this a reasonably high rate of uptake, given that we estimated the volume of eligible adults with PTSD based on annual prevalence (i.e., the proportion of adults with PTSD in a given year, rather than just those who are newly diagnosed).

Although publicly funding iCBT may reduce barriers to accessing mental health services, we acknowledge that there are people for whom iCBT may not be a suitable treatment option. This particularly includes those who have already tried CBT (whether face-to-face or internet-delivered) and did not respond to treatment, as well as people without internet access, among other reasons. Our assumption on the rate of uptake of iCBT over the next 5 years did not consider the specific needs of these populations. Tailored iCBT programs may be needed for different populations to facilitate equitable access to treatment and to ensure treatment is as effective as possible, which may result in a larger budget impact.

Additionally, we assumed that the standard of care was usual care without active iCBT treatment and that adults receiving face-to-face CBT would not switch to iCBT. This strategy overlooked the cost offset within the health system and thus overestimated the iCBT treatment-related budget impact, as switching from publicly funded face-to-face CBT to iCBT could result in cost savings. However, we have limited knowledge on the uptake of publicly funded face-to-face CBT and were therefore unable to account for switching from face-to-face CBT to iCBT. Further, it is unclear whether the treatment effect of iCBT differs from that of face-to-face CBT, and this limited our ability to examine the budget impact of iCBT programs potentially replacing some face-to-face CBT programs.

Strengths and Limitations

There were several strengths in our budget impact analysis. First, this analysis was based on a modelbased budget impact analysis, which considered both treatment costs and health state costs. Second, we conducted scenario analyses to examine the budget impact of different potential iCBT programs, and we examined the impact of varying the number of e-therapist support hours. Additionally, our cost parameters were derived from Ontario or Canada settings.

Our budget impact analysis was limited by some uncertainties. First, there is uncertainty related to clinical and cost parameters, resulting primarily from the very low quality of evidence on treatment effect. Further, the long-term treatment effects of iCBT are unclear. To overcome this limitation, we analyzed the budget impact if only treatment costs were considered. Second, there is very limited evidence regarding the use of iCBT for the treatment of adults with ASD and to estimate the proportion of adults who would benefit from CBT or iCBT. Third, we have limited knowledge on the capacity of an iCBT program to treat eligible adults with PTSD or ASD. Theoretically, unguided iCBT has an unlimited capacity, but some people may need trauma-focused treatment, which would require greater therapist guidance and thus could not be expanded without limitation. An assessment of capacity should also consider the needs of adults with other mental health disorders, such as depression or anxiety disorders, who may also benefit from iCBT.

The design of a publicly funded iCBT program is beyond the scope of this analysis; additional understanding is required on what types of iCBT program could be used and how many hours of e-therapist support these programs would require. Additionally, we have limited knowledge on the likely uptake of a publicly funded iCBT program. For example, adults receiving face-to-face CBT, paid for only in part by private insurance or paid for out of pocket, may choose to switch to iCBT if it is publicly funded, which may further influence uptake and downstream costs. That said, the uptake rate for face-to-face CBT for adults with PTSD in Ontario is low, and we therefore expect that this rate would have only a minimal impact on our analysis.

Conclusions

Our budget impact analysis suggests that publicly funding iCBT (guided or unguided) for adults with PTSD or ASD in Ontario would lead to an additional cost of $2.43 million in year 1 to $2.37 million in year 5, for a total additional cost of $16.53 million over the next 5 years. If treatment-related costs alone are considered, we estimate a budget impact of an additional $3.37 million in year 1 to $17.84 million in year 5, for a total additional cost of $52.61 million over the next 5 years. Increasing the number of e-therapist support hours and/or the rate of uptake may further increase the budget impact. The results of this analysis should be interpreted with caution, as there is uncertainty with regard to the treatment effect of iCBT for PTSD and limited clinical evidence on the use of iCBT to treat ASD.

Preferences and Values Evidence

Objective

The objective of this analysis was to explore the underlying values, needs, and priorities of adults with lived experience of post-traumatic stress disorder (PTSD) or acute stress disorder (ASD), as well as the preferences and perceptions of people who have received internet-delivered cognitive behavioural therapy (iCBT) for either of these mental health conditions.

Background

Exploring patient preferences and values provides a unique source of information about people's experiences of a health condition and the health technologies or interventions used to manage or treat that health condition. It includes the impact of the condition and its treatment on the person with the health condition, their family and other caregivers, and the person's personal environment. Engagement also provides insights into how a health condition is managed by the province's health system.

Information shared from lived experience can also identify gaps or limitations in published research (e.g., outcomes important to those with lived experience that are not reflected in the literature).^69–71 Additionally, lived experience can provide information and perspectives on the ethical and social values implications of health technologies or interventions.

Because the needs, preferences, priorities, and values of those with lived experience in Ontario are often inadequately explored in the published literature, we may speak directly with people who live with a given health condition, including those with experience of the technology or intervention we are exploring.

For this analysis, we examined in two ways the preferences and values of people with PTSD or ASD who sought iCBT:

• A review by Ontario Health of the quantitative evidence on patient preferences and values

• Direct engagement by Ontario Health through interviews with adults with PTSD or ASD

Quantitative Evidence

Research Question

What is the relative preference of adults with PTSD or ASD for treatment with iCBT compared with usual care?

Methods

LITERATURE SEARCH

We performed a literature search for quantitative evidence on preferences on June 5, 2020, for studies published from inception to the search date. We used the Ovid interface of MEDLINE and the EBSCO interface in the Cumulative Index to Nursing & Allied Health Literature (CINAHL). The search was based on the population and intervention of the clinical search strategy with a methodological filter applied to limit retrieval to quantitative evidence of preference and values (modified from Selva et al⁷²). See Appendix 1 for literature search strategies, including all search terms.

ELIGIBILITY CRITERIA

Studies

Inclusion Criteria

• English-language full-text publications

• Studies published between January 2010 and June 5, 2020. This date was chosen since iCBT is a relatively new technology and studies published prior to 2010 may not be generalizable to current iCBT practice

• Randomized controlled trials, observational studies, systematic reviews, health technology assessments that examined:

  • Patients’ preferences for iCBT to treat PTSD or ASD

  • Utility measures: direct techniques (standard gamble, time trade-off, rating scales) or conjoint analysis (discrete choice experiment, contingent valuation and willingness-to-pay, probability trade-off), or

  • Non-utility quantitative measures: direct-choice techniques, decision aids, surveys, questionnaires

Exclusion Criteria

• Animal and in vitro studies

• Editorials, commentaries, letters, case reports, conference abstracts

Participants

Inclusion Criteria

• Adults aged 16 years or older diagnosed with PTSD or ASD

• At least 70% of participants in any given study met diagnostic criteria for either PTSD or ASD according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-III, DSM-III-R, DSM-IV, or DSM-V) or the International Classification of Diseases (ICD-9 or ICD-10), as assessed by the study's clinical interview or a validated questionnaire

Exclusion Criteria

• No restrictions placed on sex or gender, ethnicity, comorbidities, setting, type of traumatic event, severity of symptoms, or length of time since trauma

Interventions

Inclusion Criteria

• Guided or unguided iCBT (trauma-focused or non–trauma-focused) delivered via a computer or mobile device

Exclusion Criteria

• Interventions based on eye movement desensitization and reprocessing (EMDR) alone or online psychoeducation alone

• Interventions using mindfulness-based approaches, apart from mindfulness-based iCBT

Comparator

• No specific comparator was required

Outcome Measures

• Patient preference

• Acceptability of intervention

• Satisfaction with treatment

Timing

• Short-term (≤ 3 months) and long-term (> 3 months) outcomes after treatment

Setting

• No restriction

LITERATURE SCREENING

A single reviewer conducted an initial screening of titles and abstracts using Covidence²⁷ and then obtained the full text of studies that appeared eligible for review according to the inclusion criteria. A single reviewer then examined the full-text articles and selected studies eligible for inclusion.

DATA EXTRACTION

A single reviewer extracted relevant data on study characteristics using a data form to collect information about the following:

• Source (e.g., citation information, contact details, study type)

• Methods (e.g., study design, study duration, participant recruitment)

• Outcomes (e.g., outcomes measured, outcome definition and source of information, unit of measurement, upper and lower limits [for scales], time points at which the outcomes were assessed)

STATISTICAL ANALYSIS

Results are summarized narratively. No additional statistical analyses were conducted beyond those reported in the primary studies.

CRITICAL APPRAISAL OF EVIDENCE

We did not undertake a formal critical appraisal of the included studies.

Results

LITERATURE SEARCH

The literature search of the quantitative evidence of preferences and values yielded 973 citations published from inception until June 5, 2020. We did not identify any additional studies from other sources. We identified one study¹⁴ (a systematic review) that met our inclusion criteria. Figure 5 presents the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram for the literature search for quantitative evidence of preferences and values.

Figure 5: PRISMA Flow Diagram—Quantitative Evidence of Preferences and Values Search Strategy.

Source: Adapted from Moher et al, 2009.³²

Abbreviation: PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses.

^aSystematic review by Simon et al, 2019.¹⁴

CHARACTERISTICS OF INCLUDED STUDIES

We identified one systematic review (10 RCTs, N = 720), which included the same 10 studies contained in the systematic reviews by CADTH¹⁸ and Cochrane.²⁵ Table 1 in the Clinical Evidence section of this report presents characteristics of the studies included in the systematic review by Simon et al.¹⁴

All 10 studies reported the outcomes related to patient preferences as secondary outcomes.

No studies were identified that examined iCBT for the treatment of ASD. We identified no studies that compared iCBT with face-to-face CBT for the treatment of PTSD.

PATIENT PREFERENCE

No studies reported on the outcome of patient preference.

ACCEPTABLITY OF INTERVENTION

Four studies reported this outcome.^35,36,41,42 None of them used a standardized/validated acceptability scale; however, three studies used measures developed specifically for their studies (Krupnick et al,³⁶ Miner et al,⁴¹ Spence et al⁴²).

Krupnick et al³⁶ asked participants who received iCBT four open-ended questions about their experience with the iCBT program, including how they felt about the intervention, its perceived strengths and weaknesses, and whether there was anything about the approach they would recommend changing. Five of 16 people (31%) who completed all 10 writing sessions in the iCBT group answered that they were “extremely enthusiastic” about the iCBT program.³⁶ One person who dropped out of the iCBT group but wrote a qualitative response indicated that it was “difficult to find time to write.”³⁶ He felt that doing it online was “difficult” and felt “kind of impersonal.”³⁶

Miner et al⁴¹ stated that nearly 83% (19/23) of participants in the iCBT arm reported they had learned new tools to cope with their symptoms. Responses to an open-ended question about which features of the iCBT app they found most useful were reported in the following categories (responses could be in more than one category): symptom self-management, 47.8% (11/23); accessibility, 13.0% (3/23); education, 8.7% (2/23); and other, 39.1% (9/23).⁴¹ One participant (4.3%) replied that the app was not useful to them, and four (17.4%) did not respond to the open-ended question.⁴¹

Spence et al⁴² reported that 95.2% (20/21) of the iCBT group participants who completed a post-treatment questionnaire stated doing the program was worth their time, and 95% (20/21) reported they would recommend the iCBT program to a friend with PTSD.

Knaevelsrud et al³⁵ used the Distress/Endorsement Validation Scale⁷³ to ask participants about their experience of the treatment. Most participants (78%, 37/47) considered the duration to be sufficient, and 74% (35/47) spent up to 2 hours a week on the writing therapy.³⁵ Furthermore, 87% (41/47) of participants regarded the therapy as clearly understandable and effective for reducing tension and exhaustion, 74% (35/47) experienced a marked decrease in their symptoms, and 76% (36/47) would recommend the treatment to others.³⁵ No other data were reported by the authors.

SATISFACTION WITH TREATMENT

Two studies reported this outcome using different methods to measure satisfaction.^39,42

Littleton et al³⁹ assessed participant satisfaction at post-treatment using the Satisfaction With Therapy and Therapist Scale–Revised (STTS-R).⁷⁴ The STTS-R is a 12-item measure that provides scores on two scales: satisfaction with the therapist and satisfaction with the treatment received. Participants rate their agreement with each item on a Likert scale from 1 (strongly disagree) to 5 (strongly agree). Participants reported high levels of satisfaction with the online program (mean = 4.33, standard deviation [SD] = 0.64) and their online therapist (mean = 4.43, SD = 0.62).³⁹ Seven of 21 (33%) participants gave the online program the maximum possible satisfaction rating of 5, and 9 of 21 (42.9%) reported a top rating of 5 for their satisfaction with their online therapist.³⁹ No participants reported a satisfaction rating below the midpoint, and 1 of 21 (4.8%) reported satisfaction ratings at the scale midpoint.³⁹

Spence et al⁴² assessed participants’ treatment satisfaction using a questionnaire based on the Credibility/Expectancy Questionnaire.⁷⁵ Overall, iCBT participants who completed the post-treatment satisfaction questionnaire reported a high level of satisfaction with the overall program, with 17 of 21 (81%) reporting they were either “very satisfied” or “mostly satisfied” and 4 of 21 (19%) were “neutral” or “somewhat satisfied,” with no participants rating the program as “unsatisfactory.”⁴² Nineteen of 21 (90%) participants rated the quality of the treatment modules as “excellent” or “good” and 2 of 21 (10%) rated them as “satisfactory.”⁴² No participants reported the modules to be “unsatisfactory.” Twenty of 21 (95%) rated the quality of internet correspondence with the therapist as “excellent” or “good,” 1 of 21 (5%) rated it as “satisfactory,” and no participants rated the contact as “unsatisfactory.”⁴²

Discussion

Overall, in their review of participants’ experiences with iCBT for PTSD, Simon et al¹⁴ stated that the available studies found high levels of general acceptability with the treatment, according to measures of acceptability and satisfaction.

There are several limitations to the systematic review by Simon et al.¹⁴ All the outcomes related to the preferences and perceptions of people receiving treatment for PTSD were secondary outcomes assessed in the treatment arms of the included RCTs. That is, these outcomes were planned for in the overall study design, but the studies were statistically designed to evaluate the primary outcomes. Although the primary studies used various scales to assess the acceptability of iCBT by people receiving it for PTSD, Simon et al¹⁴ cautioned that there is a general lack of standardized scales to measure acceptability in health care interventions.⁷⁶ Additionally, not all study participants completed post-treatment questionnaires on treatment acceptability. For example, Littleton et al³⁹ reported high levels of satisfaction but did not obtain this information from participants who did not complete the post-treatment assessment. Finally, among the primary studies, the nature and extent of the guided iCBT programs varied widely, as did the training of guiding clinicians.¹⁴ The type of relationship between the therapist providing guidance and the patient may impact a person's sense of how acceptable the treatment was and how satisfied they were with it.

Conclusions

Internet-delivered CBT may be a generally acceptable treatment for adults with PTSD based on measures of acceptability and satisfaction, but there is uncertainty in the evidence based on incomplete follow-up of study participants and variability in the nature and extent of the relationships between the guiding therapists and the patients.

Direct Patient Engagement

Methods

PARTNERSHIP PLAN

The partnership plan for this health technology assessment focused on consultation to examine the experiences of people with PTSD and those of their families and other caregivers. We engaged people via phone interviews.

We used a qualitative interview, as this method of engagement allowed us to explore the meaning of central themes in the experiences of people with PTSD, as well as those of their families and caregivers.⁷⁷ The sensitive nature of exploring people's experiences of a health condition and their quality of life are other factors that support our choice of an interview methodology.

PARTICIPANT OUTREACH

We used an approach called purposive sampling,^78–81 which involves actively reaching out to people with direct experience of the health condition and health technology or intervention being reviewed. We approached a variety of PTSD support groups, the Traumatology Institute, and patient networks to spread the word about this engagement activity and to contact people with PTSD, family members, and caregivers, including those with experience of iCBT.

Inclusion Criteria

We sought to speak with people and their caregivers who have been actively managing PTSD or ASD. People were not required to have had direct experience with iCBT to participate.

We sought broad geographic, cultural, and socioeconomic representation to elicit possible equity issues in access to treatment for PTSD or ASD.

Exclusion Criteria

We did not set specific exclusion criteria.

Participants

For this project, we spoke by phone with 10 people diagnosed with PTSD living in Ontario. We spoke with people who had experience with various treatment options for PTSD including iCBT. Four of the 10 participants had experience using therapist-guided iCBT.

APPROACH

At the beginning of the interview, we explained the role of Ontario Health, the purpose of this health technology assessment, the risks of participation, and how participants’ personal health information would be protected. We gave this information to participants both verbally and in a letter of information (Appendix 6). We then obtained participants’ verbal consent before starting the interview. With participants’ consent, we audio-recorded and then transcribed the interviews. Interviews lasted approximately 45 to 60 minutes. The interview was loosely structured and consisted of a series of open-ended questions. Questions were based on a list developed by the Health Technology Assessment International Interest Group on Patient and Citizen Involvement in Health Technology Assessment.⁸² Questions focused on the impact of PTSD on the quality of life of people with this condition, their experiences with treatments to manage or treat PTSD, their experiences with the iCBT, and their perceptions of the benefits or limitations of iCBT. See Appendix 7 for our interview guide.

DATA EXTRACTION AND ANALYSIS

We used a modified version of a grounded-theory methodology to analyze interview transcripts. The grounded-theory approach allowed us to organize and compare information on experiences across participants. This method consists of a repetitive process of obtaining, documenting, and analyzing responses while simultaneously collecting, analyzing, and comparing information.^83,84 We used the qualitative data analysis software program NVivo⁸⁵ to identify and interpret patterns in the data. The patterns we identified allowed us to highlight the impact of PTSD and treatments on the people with PTSD we interviewed.

Results

On the guidance of a clinical psychologist, no questions specifically asked participants about their history with PTSD or its social, emotional, or day-to-day impact, to prevent the possibility of reigniting their past trauma. Our questions focused on the treatment journey and the experiences with different treatment options. Results presented in the “Impact of PTSD” section below and summarized here were derived from comments that organically came up in conversation regarding the treatment journey.

Participants reported that PTSD had a significant impact on their quality of life. Those who disclosed the reason for their PTSD indicated it stemmed from workplace disturbances, several traumatic incidents, or physical or sexual assault that had occurred either in their childhood or adult lives. The trauma they experienced had a profound impact on their mental health.

The people we interviewed reported on the unique and complex nature of PTSD. Participants were in various stages of recovery, ranging from those needing a significant amount of support to those able to manage with minimal assistance. All participants stated that managing PTSD is an ongoing struggle. Those who said their PTSD is now manageable still receive ongoing therapy, especially when they experience triggers in their day-to-day lives.

DIAGNOSIS

Participants reported being frustrated with how challenging it had been to navigate the mental health space. Several people stated their primary care provider wasn't informed about the mental health services available in the community and, in one instance, resisted referring them to a psychiatrist. Patients had to find services on their own or self-advocate to get the support they needed:

None of the help I found was referred to me by any funded or Ministry of Health body. I had to do all the legwork and then go to them [primary care physician] for the referrals.

The [family doctor] had absolutely no training or understanding whatsoever of mental health issues. I went out on my own and got my diagnosis.

I had a lot of difficulty with my family doctor, getting them to set up a consultation with my psychiatrist.

Several participants described the difficulty in getting an accurate diagnosis, a process that often took years. The complicated nature of trauma and the complexity of the symptoms made PTSD difficult to diagnose. A few patients were misdiagnosed with other mental health conditions, which made it challenging to manage their symptoms and get the support they needed. The misdiagnosis also had an impact on the treatment options that were available to them:

So my first diagnosis, I presented with a dissociative state and I was diagnosed with a manic episode and treated as bipolar.

I was seeing somebody before I was diagnosed [with PTSD] and it was purely cognitive psychotherapy and it was not directed at trauma. It just didn't help me. But that's nobody's fault. I just hadn't been diagnosed yet.

The psychiatrist said to me, you have chronic PTSD. I've been trying to get somebody to acknowledge that I have PTSD for 50 years.

In one case, a participant noted the re-traumatization that they experienced by having to retell their story over and over again in the process of seeking care:

I would have to tell my trauma to some random person who picked up the phone and was doing an assessment on me … I had to relive it for my doctor … and then I had to relive it for the paralegal … I was just constantly having to tell multiple stories.

IMPACT OF PTSD

Participants reported experiencing various symptoms due to PTSD, including memory loss, depression, pain, anxiety, suicidal thoughts, and limited control over their emotions. These symptoms made it difficult to manage day-to-day activities, relationships, and employment:

I was in bad shape. I was already deep, and I had severe anxiety, and depression was kicking in.

Suicidal ideation thoughts come frequently when you're down. They easily overwhelm you.

You know your emotional control is broken and you get really angry at certain things.

Changes in their emotional states also led participants to self-destructive behaviour, feelings of helplessness, and avoidance of leaving their homes:

I didn't know what I wanted. I was messy, I didn't wash, I was rude … I didn't want to die, I didn't want to quit drinking, I didn't want to get therapy … I wasn't able to stay in any particular room for more than 20 minutes.

I still get stuck in a rut of staying at home and not going out. Because it's a safety thing. And you get into a pattern of staying home for safety.

Managing Relationships

Those who spoke about the impact of PTSD on their relationships indicated that they emotionally detached themselves from others, which deprived them of a personal support system. They also mentioned the negative impact of their PTSD on their families:

I have a daughter. She was injured as well because of my injuries. I'm a single mother, she was so scared and she's the only child.

I never had a support system before. I wouldn't allow it.

Managing Employment

Employment was difficult to manage for those with complications related to their PTSD symptoms. If they continued to work while experiencing symptoms, this could affect their on-the-job performance. For participants whose PTSD was related to workplace trauma, going back to the work environment added another dimension of complexity to their condition:

I was stable and productive and working full time, but I was having a lot of problems because some of the triggers and the things that were in my life were still present in my life.

I was struggling with having to go back to a work situation where I was attacked. So it was quite a difficult time for me to get over that and constantly being forced to come back when I couldn't.

One of the things with having PTSD is your inability to retain information or to really listen because it's like a lot of noise going on.

TREATMENTS OPTIONS

Once they had received an accurate diagnosis of PTSD, the people we spoke with reported difficultly in getting the support they needed. Participants tried various treatment options that included CBT, iCBT, group therapy, yoga, meditation, and medication, as well as alternative forms of treatment such as reiki. They described several barriers that limited the availability of these treatment options.

Medication

Personal experiences with medication varied. Participants reported taking antidepressants, mood stabilizers, sleeping pills, and cannabis to manage their PTSD. A few reported trying medications that were unsuccessful due to the side effects, while others had positive experiences. Others said they needed medication early in their treatment journey but were able to taper off their use of the drugs as they developed other coping mechanisms and strategies through psychotherapy:

We've tried all kinds of medications with no success or limited success. Either they don't work, or they worked briefly, or the side effects become overwhelming, or they stop working.

When it first happened, I couldn't get out of the bed. So I did need to have some meds at the beginning.

In some cases, participants preferred psychotherapy over medication or would use medication on an as-needed basis due to concerns over addiction:

Psychiatrists wanted me on benzodiazepines … I was concerned about addictions, but I don't sleep. So basically what I'll do is, I'll use them if I have a really bad day, or there's some sort of crisis in the household and I'm just really struggling.

I was hospitalized several times, I was on medication in the hospital. [The hospital] did offer us different options of the different treatment[s], but I still found CBT was most effective and so I stick with it.

Group Therapy

Group psychotherapy was another treatment that interviewees reported to be beneficial. Those who had used group therapy stated that it complemented their individual therapy and had additional benefits:

I never even knew you could have group treatment for PTSD … People just share what it's like living with it. Like the memory loss or whatever funny stories that come with PTSD, but not the trauma itself. It was very, very beneficial because you feel like you're not isolated.

My one-on-one therapy wouldn't be half as productive if I didn't have the support group … I used to look at it that my therapist was the architect and manager, and the support group was a construction crew.

Face-to-Face Cognitive Behavioural Therapy

All participants had experience with in-person CBT and reported it to be a beneficial and effective form of treatment for their PTSD. This therapy led to significant positive changes in their quality of life. In some cases, it gave them the hope that they will eventually get past their trauma even if they weren't at that point yet:

I can't make light of what changed. What changed was everything. I was able to be a mother, I was able to leave a toxic environment job, I was able to leave a toxic relationship, I was able to learn coping skills.

For the first time I had hope. I actually believed that this might work. And having spent decades believing it won't.

Participants said CBT gave them the coping mechanisms, skills, and strategies to manage their PTSD symptoms and, in some cases, they preferred it over medication:

Going through the behavioural therapy and going through these mental exercises and things, it's really helped me live with day-to-day problems, far better than [I] ever was without that kind of training and without that kind of ability to use CBT.

It [CBT] was also bringing me … into the situation [where] I visually could get myself to bring out those emotions. On top of that, [CBT helped me] work through those emotions so that I could eventually, hopefully, get past it to a point, but I'll never forget [the trauma].

INTERNET-DELIVERED CBT

Participants who had used iCBT got access to it without cost as long as they were accessing psychotherapy services from the participating clinic. All four participants who had experience with iCBT had used it along with face-to-face CBT. They found the online experience to be beneficial but emphasized that it needs to be a complement to face-to-face CBT, to fully reap the benefits of this therapy. One person suggested that iCBT by itself could be used to support people who were further along in their treatment and no longer needed frequent face-to-face psychotherapy:

You can't do one without the other. Not for this type of condition, in my opinion … But I think for PTSD, the online [together] with the face-to-face is very important.

I don't think the online was enough. And I wouldn't have known how to use it because I just never dealt with this before. Now that I've dealt with it, yeah, I love the program.

As long as the online program is linked to a therapist and they are following all the same philosophy and stuff like that, it's really good.

All four people described a positive experience with iCBT. The interface was easy to use and allowed them to get to the module they needed. They also mentioned developing various coping strategies and skills by going through the different modules and felt these tools were sustainable for the long term:

If you have a setback, you go back to it and you learn it faster and you can get yourself out of it quicker. And then when it happens a third time, it's even quicker and so you may not have to go back to it. You might start to remember the activity.

The tools on the online program, some of those exercises, worksheets were excellent, because they reinforce what you learned within the therapy session. One hundred percent they're helpful.

Some participants used iCBT as a “filler” between appointments or while they were on a wait-list to access a therapist. Some also said the internet-based CBT program was a good place to find reliable information on PTSD:

It was a bridge for the waiting period. That was awesome.

It was a filler for me to help me through the times when I couldn't see her. Because I just can't afford to see her every day.

For participants who had not used iCBT, the interviewer described what iCBT entails in both guided and unguided options. These participants reported a preference for therapist-guided iCBT:

In the earlier days, without being guided through it, it would not have been helpful at all.

Non-users of iCBT also said they would welcome any sort of mental health support that is publicly funded. Some mentioned that an online CBT program would be a trusted place to find reputable information and resources:

I think for someone to be able to get any support, a bit of a diagnosis or some kind of help over the internet, I think that would work for many people.

I can't judge what I see on the internet—if it's good information or not so good information—but if this is approved and researched, that would be good.

BARRIERS TO ACCESSING TREATMENT

As noted above, one of the biggest barriers to accessing care was navigating the mental health system. Participants identified several key issues, including limited access to information on where they can get support, long wait times, lack of mental health training for family physicians, finding a therapist who specializes in PTSD, and finding a therapist who fits their needs:

You cannot functionally or compassionately help somebody with long-term, complex PTSD with strategies that are meant for simple trauma.

I didn't know that it was so important to get connected with the right therapist, and I think I wasted a lot of time just going around to trying to find someone.

These issues were exacerbated for participants living in a rural or remote community. They found it difficult to get the support they needed within their area:

[The therapist] comes up once a month. Part of my problem is, with PTSD, it's episodic. I get to see her once every two months and, if I'm having a really good day, what's the point. But in that two-month period, I can crash … and I don't have access to her then.

You're dependent on waiting months for an appointment and [the therapist] has to drive from Toronto, so if there's a blizzard and the highway's closed, too bad. See you in two months.

Cost was another important barrier reported by participants. Some had private insurance that offset the cost of treatment, but that insurance typically comes with annual limits, so participants said they often paid out-of-pocket for ongoing support after maxing out their benefits. Those without private insurance mentioned the difficulty in obtaining publicly funded services. They often had to do their own research to find these programs and then discovered that they faced a long wait for publicly funded services:

If I hadn't been lucky enough to go to [a health centre] and take both the core program and trauma program while I still had the insurance, I would have lost that opportunity. Because getting an OHIP bed [at that health centre], it is almost impossible. But it takes years and years on the waiting list.

I've been in charge of my own self-care … simply because of the lack of services available to me. it's partly because of poverty and partly because they just don't exist.

My biggest concerns around mental health lie with the fact of, if you can't get a therapist, number one, that's a problem. Number two, you cannot pay for the therapist. That's the problem.

Discussion

Outreach for this health technology assessment yielded engagement with 10 patients diagnosed with post-traumatic stress disorder. Patients discussed the impact of their condition and treatment journey. Those who voluntarily chose to disclose the impact of PTSD on their quality of life mentioned the struggle it created for their relationships, employment, and day-to-day activities. The people we spoke with had experienced various treatment options for PTSD including medication, cognitive behavioural therapy, and internet-delivered CBT.

Participants who had used iCBT discussed the benefits of the program; it provided tools, strategies and coping mechanisms to help them manage their PTSD symptoms. However, patients stated that the full benefits can only be achieved if iCBT is used alongside face-to-face CBT. They emphasized the need for face-to-face CBT for people experiencing PTSD symptoms, especially in the beginning of the treatment journey.

A majority of participants reported cost, wait times, a shortage of trained providers, and the challenge of navigating the mental health system as barriers to accessing treatment for PTSD, including any form of CBT. Getting an accurate diagnosis took years for some people, and some felt unsupported by their primary care provider as they sought help for their condition. Those without private insurance found publicly funded treatment options but were put on long wait-lists. Even those with private insurance expressed frustration with long waits to access CBT. Patients had to self-advocate and conduct their own research to find the support they needed, particularly in remote and rural areas where services were limited.

Bias in the recruitment of participants who had used iCBT is one of several limitations in our direct patient engagement for this health technology assessment. These people were referred to us through their iCBT provider, and this may have biased our results towards more positive experiences with this form of treatment. In addition, recruitment was restricted due to the ongoing COVID-19 pandemic in the province, which may have limited the range of experiences captured. Finally, all participants who had used iCBT had also used face-to-face CBT. Therefore, the preferences and values for iCBT captured in this engagement exercise reflect the experiences of people receiving multiple forms of cognitive behavioural therapy.

Conclusions

Patients with PTSD felt that internet-based CBT was a beneficial form of treatment if used in conjunction with face-to-face CBT. Several of those interviewed spoke about the positive impact of iCBT in managing their PTSD. They emphasized the use of iCBT as a “filler” between in-person sessions or while they were on a wait-list.

A majority reported cost as a barrier when accessing treatment for PTSD. Many also spoke about other access barriers that included lengthy wait times, misdiagnosis, geography, and a lack of information and support from their primary health care provider.

Preferences and Values Evidence Discussion

Overall, both the quantitative published evidence and the direct patient engagement indicated that patients with PTSD have high levels of general acceptability with iCBT according to measures of acceptability and satisfaction.¹⁴ Interviewees indicated a preference for guided iCBT over unguided iCBT.

Similar to the results of our direct patient engagement, patients in published studies reported that iCBT provided tools, strategies, and coping mechanisms to help them manage their PTSD symptoms.⁴¹ Additionally, ease of access to treatment was a feature of iCBT that individuals with PTSD found useful,⁴¹ and they stressed the importance of finding a therapist who fits a patient's needs.⁴² Those interviewed reported an improvement in their quality of life when they had access to the additional support from iCBT and emphasised the need for face-to-face psychotherapy in conjunction with iCBT.

While there is a lack of follow-up studies in the quantitative literature, when we asked interviewees about the long-term impact of iCBT, participants reported that it added sustainability to their ability to manage their PTSD symptoms but also that they continue to need follow-up support from time to time.

Preferences and Values Evidence Conclusions

Internet-delivered CBT may be a generally acceptable treatment for adults with PTSD based on measures of acceptability and satisfaction, but there is uncertainty in the evidence due to incomplete follow-up of study participants and variability in the nature and extent of the relationships between the guiding therapists and the patients. In our direct engagement, patients reported that iCBT supported the sustainability of their learnings in managing their PTSD symptoms.

Conclusions of the Health Technology Assessment

In our clinical evidence review:

• We found no studies that assessed internet-delivered cognitive behavioural therapy (iCBT) for the treatment of acute stress disorder (ASD) or the prevention of post-traumatic stress disorder (PTSD) in individuals diagnosed with ASD

• We identified no studies that compared iCBT with face-to-face CBT for the treatment of PTSD

• For iCBT compared with wait-list or usual care:

  • iCBT may improve PTSD symptoms, but the evidence is very uncertain (GRADE: Very low)

  • The evidence suggests iCBT results in a slight increase in dropout rates (GRADE: Low)

  • iCBT may have little to no effect on post-treatment diagnosis of PTSD (that is, the diagnosis persists after treatment), but the evidence is very uncertain (GRADE: Very low)

  • iCBT may improve depression and anxiety symptoms or quality of life, but the evidence is very uncertain (GRADE: Very low)

• For iCBT compared with non–CBT internet-delivered interventions:

  • iCBT may have little to no effect on PTSD symptoms, dropout rates, or depression and anxiety symptoms, but the evidence is very uncertain (GRADE: Very low)

In our economic assessments:

• We identified one previously published economic evaluation on the cost-effectiveness of iCBT for the treatment of adults with PTSD that was applicable to the Ontario context

• We found no studies evaluating the cost-effectiveness of iCBT for the treatment of adults with ASD

• Compared with usual care, iCBT may be cost-effective for the treatment of adults with PTSD

  • However, given the very low quality of clinical evidence used to inform the model in this study, and given the limited evidence on comorbidities and the natural history of PTSD, this finding should be interpreted with caution

• iCBT may be cost-effective for the treatment of adults with ASD at risk for developing PTSD, given that results for PTSD may be generalizable to the context of ASD, but this is very uncertain

• Publicly funding iCBT for adults with PTSD or ASD would lead to additional costs of $2.43 million in year 1 to $2.37 million in year 5, for a total additional cost of $16.53 million over the next 5 years

  • If treatment-related costs alone are considered, the total budget impact over the next 5 years would be an additional $52.61 million

  • The budget impact analysis results should be interpreted with caution, given the very low quality of evidence on the treatment effect of iCBT for PTSD, and because there is currently no long-term evidence on the treatment effect of iCBT

In our review of preferences and values evidence:

• Internet-delivered CBT may be a generally acceptable treatment for adults with PTSD based on measures of acceptability and satisfaction

  • However, there is uncertainty in the evidence based on incomplete follow-up of study participants and variability in the nature and extent of the relationships between the guiding therapists and the patients

  • These findings are based on a systematic review of 10 studies, all of which reported patient satisfaction and acceptability as secondary outcomes

• In interviews for this report, people with PTSD felt that internet-based CBT was a beneficial form of treatment if used in conjunction with face-to-face CBT

• Most people we spoke with reported cost as a barrier when accessing treatment for PTSD. Many also spoke about other access barriers, including lengthy wait times, misdiagnosis, geography, and a lack of information and support from their primary health care provider

ABBREVIATIONS

ASD

Acute stress disorder

CADTH

Canadian Agency for Drugs and Technologies in Health

CBT

Cognitive behavioural therapy

CI

Confidence interval

CPT

Cognitive processing therapy

EMDR

Eye movement desensitization and reprocessing

GRADE

Grading of Recommendations Assessment, Development, and Evaluation

iCBT

Internet-delivered cognitive behavioural therapy

ICER

Incremental cost-effectiveness ratio

MD

Mean difference

NICE

National Institute for Health and Care Excellence

OR

Odds ratio

PTSD

Post-traumatic stress disorder

QALY

Quality-adjusted life-year

RCT

Randomized controlled trial

ROBIS

Risk of Bias in Systematic Reviews tool

RR

Relative risk

SD

Standard deviation

SMD

Standardized mean difference

GLOSSARY

Adverse event

An adverse event is an unexpected medical problem that happens during treatment for a health condition. Adverse events may be caused by something other than the treatment.

Budget impact analysis

A budget impact analysis estimates the financial impact of adopting a new health care intervention on the current budget (i.e., the affordability of the new intervention). It is based on predictions of how changes in the intervention mix will impact the level of health care spending for a specific population. Budget impact analyses are typically conducted for a short-term period (e.g., 5 years). The budget impact, sometimes referred to as the net budget impact, is the estimated cost difference between the current scenario (i.e., the anticipated amount of spending for a specific population without using the new intervention) and the new scenario (i.e., the anticipated amount of spending for a specific population following the introduction of the new intervention).

Cost-effective

A health care intervention is considered cost-effective when it provides additional benefits, compared with relevant alternatives, at an additional cost that is acceptable to a decision-maker based on the maximum willingness-to-pay value.

Cost-effectiveness analysis

Used broadly, “cost-effectiveness analysis” may refer to an economic evaluation used to compare the benefits of two or more health care interventions with their costs. It may encompass several types of analysis (e.g., cost-effectiveness analysis, cost–utility analysis). Used more specifically, “cost-effectiveness analysis” may refer to a type of economic evaluation in which the main outcome measure is the incremental cost per natural unit of health (e.g., life-year, symptom-free day) gained.

Cost–utility analysis

A cost–utility analysis is a type of economic evaluation used to compare the benefits of two or more health care interventions with their costs. The benefits are measured using quality-adjusted life-years, which capture both the quality and quantity of life. In a cost–utility analysis, the main outcome measure is the incremental cost per quality-adjusted life-year gained.

Deterministic sensitivity analysis

Deterministic sensitivity analysis is an approach used to explore uncertainty in the results of an economic evaluation by varying parameter values to observe the potential impact on the cost-effectiveness of the health care intervention of interest. One-way sensitivity analysis accounts for uncertainty in parameter values one at a time, whereas multiway sensitivity analysis accounts for uncertainty in a combination of parameter values simultaneously.

Discounting

Discounting is a method used in economic evaluations to adjust for the differential timing of the costs incurred and the benefits generated by a health care intervention over time. Discounting reflects the concept of positive time preference, whereby future costs and benefits are reduced to reflect their present value. The health technology assessments conducted by Ontario Health use an annual discount rate of 1.5% for both future costs and future benefits.

Dominant

A health care intervention is considered dominant when it is more effective and less costly than its comparator(s).

EuroQol–Five Dimensions (EQ-5D)

The EQ-5D is a generic health-related quality-of-life classification system widely used in clinical studies. In economic evaluations, it is used as an indirect method of obtaining health state preferences (i.e., utility values). The EQ-5D questionnaire consists of five questions relating to different domains of quality of life: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. For each domain, there are three response options: no problems, some problems, or severe problems. A newer instrument, the EQ-5D-5L, includes five response options for each domain. A scoring table is used to convert EQ-5D scores to utility values.

Health state

A health state is a particular status of health (e.g., sick, well, dead). A health state is associated with some amount of benefit and may be associated with specific costs. Benefit is captured through individual or societal preferences for the time spent in each health state and is expressed in quality-adjusted weights called utility values. In a Markov model, a finite number of mutually exclusive health states are used to represent discrete states of health.

Incremental cost

The incremental cost is the additional cost, typically per person, of a health care intervention versus a comparator.

Incremental cost-effectiveness ratio (ICER)

The incremental cost-effectiveness ratio (ICER) is a summary measure that indicates, for a given health care intervention, how much more a health care consumer must pay to get an additional unit of benefit relative to an alternative intervention. It is obtained by dividing the incremental cost by the incremental effectiveness. Incremental cost-effectiveness ratios are typically presented as the cost per life-year gained or the cost per quality-adjusted life-year gained.

Markov model

A Markov model is a type of decision-analytic model used in economic evaluations to estimate the costs and health outcomes (e.g., quality-adjusted life-years gained) associated with using a particular health care intervention. Markov models are useful for clinical problems that involve events of interest that may recur over time (e.g., stroke). A Markov model consists of mutually exclusive, exhaustive health states. Patients remain in a given health state for a certain period of time before moving to another health state based on transition probabilities. The health states and events modelled may be associated with specific costs and health outcomes.

Ministry of Health perspective

The perspective adopted in economic evaluations determines the types of costs and health benefits to include. Ontario Health develops health technology assessment reports from the perspective of the Ontario Ministry of Health. This perspective includes all costs and health benefits attributable to the Ministry of Health, such as treatment costs (e.g., drugs, administration, monitoring, hospital stays) and costs associated with managing adverse events caused by treatments. This perspective does not include out-of-pocket costs incurred by patients related to obtaining care (e.g., transportation) or loss of productivity (e.g., absenteeism).

Monte Carlo simulation

Monte Carlo simulation is an economic modelling method that derives parameter values from distributions rather than fixed values. The model is run several times, and in each iteration, parameter values are drawn from specified distributions. This method is used in microsimulation models and probabilistic sensitivity analysis.

Probabilistic sensitivity analysis (PSA)

A probabilistic sensitivity analysis (PSA) is used in economic models to explore uncertainty in several parameters simultaneously and is done using Monte Carlo simulation. Model inputs are defined as a distribution of possible values. In each iteration, model inputs are obtained by randomly sampling from each distribution, and a single estimate of cost and effectiveness is generated. This process is repeated many times (e.g., 10,000 times) to estimate the number of times (i.e., the probability) that the health care intervention of interest is cost-effective.

Quality-adjusted life-year (QALY)

The quality-adjusted life-year (QALY) is a generic health outcome measure commonly used in cost–utility analyses to reflect the quantity and quality of life-years lived. The life-years lived are adjusted for quality of life using individual or societal preferences (i.e., utility values) for being in a particular health state. One year of perfect health is represented by one quality-adjusted life-year.

Reference case

The reference case is a preferred set of methods and principles that provide the guidelines for economic evaluations. Its purpose is to standardize the approach of conducting and reporting economic evaluations, so that results can be compared across studies.

Scenario analysis

A scenario analysis is used to explore uncertainty in the results of an economic evaluation. It is done by observing the potential impact of different scenarios on the cost-effectiveness of a health care intervention. Scenario analyses include varying structural assumptions from the reference case.

Sensitivity analysis

Every economic evaluation contains some degree of uncertainty, and results can vary depending on the values taken by key parameters and the assumptions made. Sensitivity analysis allows these factors to be varied and shows the impact of these variations on the results of the evaluation. There are various types of sensitivity analysis, including deterministic, probabilistic, and scenario.

Time horizon

In economic evaluations, the time horizon is the time frame over which costs and benefits are examined and calculated. The relevant time horizon is chosen based on the nature of the disease and health care intervention being assessed, as well as the purpose of the analysis. For instance, a lifetime horizon would be chosen to capture the long-term health and cost consequences over a patient's lifetime.

Utility

A utility is a value that represents a person's preference for various health states. Typically, utility values are anchored at 0 (death) and 1 (perfect health). In some scoring systems, a negative utility value indicates a state of health valued as being worse than death. Utility values can be aggregated over time to derive quality-adjusted life-years, a common outcome measure in economic evaluations.

Wait-list control

A group of study participants who will receive the same treatment as people in the active intervention or experimental group, but at a later time. This method is often used to create a comparison group in mental health studies, where it would be unethical to deny all participants access to potentially effective treatment.

Willingness-to-pay value

A willingness-to-pay value is the monetary value a health care consumer is willing to pay for added health benefits. When conducting a cost–utility analysis, the willingness-to-pay value represents the cost a consumer is willing to pay for an additional quality-adjusted life-year. If the incremental cost-effectiveness ratio is less than the willingness-to-pay value, the health care intervention of interest is considered cost-effective. If the incremental cost-effectiveness ratio is more than the willingness-to-pay value, the intervention is considered not to be cost-effective.

Appendices

Appendix 1: Literature Search Strategies

Clinical Evidence Search

Search date: June 1, 2020

Databases searched: Ovid MEDLINE, Embase, APA PsycInfo, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, CRD Health Technology Assessment Database, and NHS Economic Evaluation Database

Database: EBM Reviews – Cochrane Central Register of Controlled Trials <April 2020>, EBM Reviews −Cochrane Database of Systematic Reviews <2005 to May 29, 2020>, EBM Reviews – Health Technology Assessment <4th Quarter 2016>, EBM Reviews – NHS Economic Evaluation Database <1st Quarter 2016>, Embase <1980 to 2020 Week 22>, Ovid MEDLINE(R) ALL <1946 to May 29, 2020>, APA PsycInfo <1967 to May Week 4 2020>

Search strategy:

• 1Stress Disorders, Post-Traumatic/ (53897)
• 2Stress Disorders, Traumatic/ (37442)
• 3Combat Disorders/ (38273)
• 4Stress Disorders, Traumatic, Acute/ (1503)
• 5((posttrauma∗ or post-trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom∗)).ti,ab,kw. (122825)
• 6PTSD.ti,ab,kw. (93238)
• 7(acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,kw. (2749)
• 8or/1-7 (162438)
• 9(android or app or apps or audio∗ or blog or iCBT or cCBT or i-CBT or c-CBT or CD-ROM or cell phone∗ or cellphone or chat or computer∗ or cyber∗ or distance∗ or DVD or eHealth or e-health or electronic health∗ or e-Portal or ePortal or etherap∗ or etherap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or internet∗ or interapy or ipad or i-pad or iphone or i-phone or ipod or i-pod or web∗ or WWW or smart phone or smartphone or mobile phone∗ or e-mail∗ or email∗ or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant∗ or PDA or SMS or social medi∗ or Facebook or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗or teletherap∗ or text messag∗ or texting or tape or taped or video∗ or YouTube or podcast or virtual∗ or remote).ti,ab,kw. (4082091)
• 10(self adj3 (care or change or guide∗ or help or intervention or manag∗ or support∗ or train∗)).ti,ab,kw. (225962)
• 119 or 10 (4260976)
• 128 and 11 (13601)
• 13limit 12 to yr="2018 −Current” (3873)
• 1413 use cctr,coch,clhta,cleed (366)
• 15posttraumatic stress disorder/ (124131)
• 16(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,kw. (138494)
• 17(((acute or traumatic) adj stress∗) and (expos∗ or psyc∗)).ti,ab,kw. (53059)
• 18(traumati#ed adj (victim? or survivor?)).ti,ab,kw. (170)
• 19(trauma∗ adj2 (event? or memor∗ or flashback∗ or nightmare?)).ti,ab,kw. (37125)
• 20((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure adj3 (therap∗ or psychotherap∗ or training or counsel∗))).ti,ab,kw. (4321)
• 21or/15-20 (198981)
• 22(((internet or web or online) adj3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT).ti,ab,kw. (16259)
• 23(android or app or apps or blog∗ or CD-ROM or cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or e-therap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or i-phone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW).ti,ab,kw,hw. (4807237)
• 24internet/ (213019)
• 25blogging/ or e-mail/ or social media/ or text messaging/ or videoconferencing/ or webcast/ or wireless communication/ (79657)
• 26telecommunication/ or teleconference/ (31903)
• 27telemedicine/ or telehealth/ or telepsychiatry/ or teletherapy/ (67669)
• 28mobile phone/ or smartphone/ (45498)
• 29mobile application.hw. (11508)
• 30∗technology/ (51291)
• 31computer program/ or digital computer/ or personal computer/ or computer assisted therapy/ (377398)
• 32∗computer/ (42767)
• 33(telecomm∗ or tele-comm∗).ti,ab,kw. (11478)
• 34(eLearning or blended learning).ti,ab,kw. (3974)
• 35(videoconferenc∗ or video conferenc∗).ti,ab,kw. (9654)
• 36(synchronous or asynchronous or (electronic adj2 deliver∗)).ti,ab,kw. (105137)
• 37or/23-36 (4932938)
• 38(behavio∗ or cognitive).ti. or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or ((cognitive or behavio∗) adj2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) adj3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ adj3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety adj3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ adj3 suppress∗) or rumination).mp. (4714730)
• 3937 and 38 (381789)
• 40Systematic review/ or “systematic review (topic)"/ or exp Meta Analysis/ or “Meta Analysis (Topic)"/ or Biomedical Technology Assessment/ (606997)
• 41(meta analy∗ or metaanaly∗ or health technolog∗ assess∗ or systematic review∗).hw. (623000)
• 42((systematic∗ or methodologic∗) adj3 (review∗ or overview∗)).tw,kw. (488507)
• 43(meta analy∗ or metaanaly∗ or met analy∗ or metanaly∗ or meta review∗ or metareview∗ or health technolog∗ assess∗ or HTA or HTAs or (technolog∗ adj (assessment∗ or overview∗ or appraisal∗))).tw,kw. (513201)
• 44(evidence adj (review∗ or overview∗ or synthes#s)).tw,kw. (17991)
• 45(review of reviews or overview of reviews).tw,kw. (1975)
• 46umbrella review∗.tw,kw. (967)
• 47((pool∗ adj3 analy∗) or published studies or published literature or hand search∗ or handsearch∗ or manual search∗ or ((database∗ or systematic∗) adj2 search∗) or reference list∗ or bibliograph∗ or relevant journals or data synthes∗ or data extraction∗ or data abstraction∗).tw,kw. (529987)
• 48(medline or pubmed or medlars or embase or cinahl or web of science or ovid or ebsco∗ or scopus).ab. (509740)
• 49cochrane.tw,kw. (221690)
• 50(meta regress∗ or metaregress∗).tw,kw. (23010)
• 51(((integrative or collaborative or quantitative) adj3 (review∗ or overview∗ or synthes∗)) or (research adj3 overview∗)).tw,kw. (36309)
• 52(cochrane or (health adj2 technology assessment) or evidence report or systematic review∗).jw. (66248)
• 53((comparative adj3 (efficacy or effectiveness)) or relative effectiveness or ((indirect or indirect treatment or mixed-treatment) adj comparison∗)).tw,kw. (79367)
• 54or/40-53 (1435882)
• 55randomized controlled trial/ (1108607)
• 56randomization.de. (86733)
• 57controlled clinical trial/ and (Disease Management or Drug Therapy or Prevention or Rehabilitation or Therapy).fs. (296353)
• 58∗clinical trial/ (24332)
• 59placebo.de. (342619)
• 60placebo.ti,ab. (851935)
• 61trial.ti. (855068)
• 62(randomi#ed or randomi#ation or randomi#ing).ti,ab,kw. (2517583)
• 63(RCT or “at random” or (random∗ adj3 (administ∗ or allocat∗ or assign∗ or class∗ or control∗ or determine∗ or divide∗ or division or distribut∗ or expose∗ or fashion or number∗ or place∗ or recruit∗ or subsitut∗ or treat∗))).ti,ab,kw. (2110812)
• 64((singl$ or doubl$ or trebl$ or tripl$) adj3 (blind$ or mask$ or dummy)).mp. (943372)
• 65(control∗ and (trial or study or group) and (placebo or waitlist∗ or wait∗ list∗ or ((treatment or care) adj2 usual))).ti,ab,kw,hw. (824010)
• 66or/55-65 (4158637)
• 67((animal or nonhuman) not (human and (animal or nonhuman))).de. (5575656)
• 6866 not 67 (4002395)
• 6954 or 68 (5050408)
• 7021 and (22 or 39) and 69 (2432)
• 71(2018∗ or 2019∗ or 2020∗).yr,dp,dt,ed,ep. (8574921)
• 7270 and 71 (826)
• 7372 use emez (224)
• 74“Trauma and Stressor Related Disorders"/ or stress disorders, traumatic/ or combat disorders/ or psychological trauma/ or stress disorders, post-traumatic/ or stress disorders, traumatic, acute/ (93249)
• 75(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,kf. (136669)
• 76(((acute or traumatic) adj stress∗) and (expos∗ or psyc∗)).ti,ab,kf. (52311)
• 77(traumati#ed adj (victim? or survivor?)).ti,ab,kf. (170)
• 78(trauma∗ adj2 (event? or memor∗ or flashback∗ or nightmare?)).ti,ab,kf. (36943)
• 79((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure adj3 (therap∗ or psychotherap∗ or training or counsel∗))).ti,ab,kf. (4228)
• 80or/74-79 (200236)
• 81(((internet or web or online) adj3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT).ti,ab,kf. (16069)
• 82(android or app or apps or blog∗ or CD-ROM or cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or e-therap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or i-phone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW).ti,ab,kf,hw. (4797547)
• 83computer communication networks/ or internet/ or blogging/ or social media/ (268023)
• 84cell phones/ or smartphone/ or text messaging/ or videoconferencing/ or webcasts as topic/ or wireless technology/ (64406)
• 85(telecomm∗ or tele-comm∗).ti,ab,kf. (11082)
• 86Telemedicine/ (53099)
• 87(eLearning or blended learning).ti,kf. (1780)
• 88(videoconferenc∗ or video conferenc∗).ti,kf. (2345)
• 89(synchronous or asynchronous or (electronic adj2 deliver∗)).ti,kf. (26026)
• 90or/82-89 (4823648)
• 91(behavio∗ or cognitive).ti. or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or cognitive processing or ((cognitive or behavio∗) adj2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) adj3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ adj3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety adj3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ adj3 suppress∗) or rumination).mp. (4728054)
• 9290 and 91 (376233)
• 93(Systematic Reviews or Meta Analysis).pt. (115838)
• 94Systematic Review/ or Systematic Reviews as Topic/ or Meta-Analysis/ or exp Meta-Analysis as Topic/ or exp Technology Assessment, Biomedical/ (626476)
• 95((systematic∗ or methodologic∗) adj3 (review∗ or overview∗)).ti,ab,kf. (472313)
• 96(meta analy∗ or metaanaly∗ or met analy∗ or metanaly∗ or meta review∗ or metareview∗ or health technolog∗ assess∗ or HTA or HTAs or (technolog∗ adj (assessment∗ or overview∗ or appraisal∗))).ti,ab,kf. (479341)
• 97(evidence adj (review∗ or overview∗ or synthes#s)).ti,ab,kf. (17490)
• 98(review of reviews or overview of reviews).ti,ab,kf. (1755)
• 99umbrella review∗.ti,ab,kf. (919)
• 100GRADE Approach/ (475)
• 101((pool∗ adj3 analy∗) or published studies or published literature or hand search∗ or handsearch∗ or manual search∗ or ((database∗ or systematic∗) adj2 search∗) or reference list∗ or bibliograph∗ or relevant journals or data synthes∗ or data extraction∗ or data abstraction∗).ti,ab,kf. (497627)
• 102(medline or pubmed or medlars or embase or cinahl or web of science or ovid or ebsco∗ or scopus).ab. (509740)
• 103cochrane.ti,ab,kf. (217904)
• 104(meta regress∗ or metaregress∗).ti,ab,kf. (22046)
• 105(((integrative or collaborative or quantitative) adj3 (review∗ or overview∗ or synthes∗)) or (research adj3 overview∗)).ti,ab,kf. (34974)
• 106(cochrane or (health adj2 technology assessment) or evidence report or systematic review∗).jw. (66248)
• 107((comparative adj3 (efficacy or effectiveness)) or relative effectiveness or ((indirect or indirect treatment or mixed-treatment) adj comparison∗)).ti,ab,kf. (57574)
• 108or/93-107 (1384224)
• 109controlled clinical trial.pt. (185179)
• 110randomized controlled trial.pt. (1000853)
• 111(randomi#ed or randomi#ation or randomi#ing).ti,ab,kf. (2419263)
• 112(RCT or at random or (random∗ adj3 (assign∗ or allocat∗ or control∗ or crossover or cross-over or design∗ or divide∗ or division or number))).ti,ab,kf. (1852590)
• 113placebo∗.ab,ti,kf. (857078)
• 114trial.ab,ti,kf. (2124866)
• 115groups.ab. (5774112)
• 116(control∗ and (trial or study or group∗) and (placebo or waitlist∗ or wait∗ list∗ or ((treatment or care) adj2 usual))).ti,ab,kf,hw. (827675)
• 117double-blind method/ or random allocation/ or single-blind method/ (705493)
• 118((single or double or triple or treble) adj2 (blind∗ or mask∗ or dummy)).ti,ab,kf. (702845)
• 119or/109-118 (8907966)
• 120exp animals/ not humans.sh. (29735588)
• 121119 not 120 (4874257)
• 122108 or 121 (6021598)
• 12380 and (81 or 92) and 122 (2261)
• 124(2018∗ or 2019∗ or 2020∗).yr,dp,dt,ed,ep. (8574921)
• 125123 and 124 (783)
• 126125 use medall (296)
• 127posttraumatic stress disorder/ or complex ptsd/ or acute stress disorder/ or combat experience/ or “debriefing (psychological)"/ or emotional trauma/ or post-traumatic stress/ or exp stress reactions/ or traumatic neurosis/ (373474)
• 128exp DISASTERS/ (124279)
• 129(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,id. (137253)
• 130(((acute or traumatic) adj stress∗) and (expos∗ or psyc∗)).ti,ab,id. (52661)
• 131(trauma∗ adj2 (event? or memor∗ or flashback∗ or nightmare?)).ti,ab,id. (37179)
• 132((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure adj3 (therap∗ or psychotherap∗ or training or counsel∗))).ti,ab,id,hw. (5067)
• 133(traumati#ed adj (victim? or survivor?)).ti,ab,id. (174)
• 134or/127-133 (551565)
• 135(((internet or web or online) adj3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT).ti,ab,kf. (16069)
• 136(android or app or apps or blog∗ or CD-ROM or cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or etherap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or iphone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW).ti,ab,id,hw. (4790539)
• 137(telecomm∗ or tele-comm∗).ti,ab,id. (11054)
• 138(eLearning or blended learning).ti,ab,id. (3837)
• 139(videoconferenc∗ or video conferenc∗).ti,ab,id. (9505)
• 140(synchronous or asynchronous or (electronic adj2 deliver∗)).ti,ab,id. (104489)
• 141internet/ or websites/ (216468)
• 142mobile devices/ or cellular phones/ (15202)
• 143social media/ or online social networks/ or blog/ or online community/ or text messaging/ (51508)
• 144electronic communication/ or exp computer mediated communication/ or electronic learning/ (18558)
• 145online therapy/ or telemedicine/ (55635)
• 146telecommunications media/ (1417)
• 147teleconferencing/ (2430)
• 148technology/ or information technology/ or exp computer applications/ or computer software/ (339157)
• 149computers/ or computer games/ or digital computers/ or microcomputers/ (161493)
• 150or/136-149 (4997351)
• 151(behavio∗ or cognitive).ti. or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or cognitive processing or ((cognitive or behavio∗) adj2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) adj3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ adj3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety adj3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ adj3 suppress∗) or rumination).ti,ab,id,hw. (4679439)
• 152(self adj (care or change or guide∗ or help or intervention or manag∗ or support∗ or train∗)).ti,id. (50992)
• 153150 and (151 or 152) (388593)
• 154(Systematic Review or Meta Analysis).md. (43487)
• 155meta analysis/ (307551)
• 156((systematic∗ or methodologic∗) adj3 (review∗ or overview∗)).ti,ab,id. (471330)
• 157(meta analy∗ or metaanaly∗ or met analy∗ or metanaly∗ or meta review∗ or metareview∗ or health technolog∗ assess∗ or HTA or HTAs or (technolog∗ adj (assessment∗ or overview∗ or appraisal∗))).ti,ab,id. (478062)
• 158(evidence adj (review∗ or overview∗ or synthes#s)).ti,ab,id. (17341)
• 159(review of reviews or overview of reviews).ti,ab,id. (1732)
• 160umbrella review∗.ti,ab,id. (904)
• 161((pool∗ adj3 analy∗) or published studies or published literature or hand search∗ or handsearch∗ or manual search∗ or ((database∗ or systematic∗) adj2 search∗) or reference list∗ or bibliograph∗ or relevant journals or data synthes∗ or data extraction∗ or data abstraction∗).ti,ab. (496529)
• 162(medline or pubmed or medlars or embase or cinahl or web of science or ovid or ebsco∗ or scopus).ab. (509740)
• 163cochrane.ti,ab. (217869)
• 164(meta regress∗ or metaregress∗).ti,ab. (21984)
• 165(((integrative or collaborative or quantitative) adj3 (review∗ or overview∗ or synthes∗)) or (research adj3 overview∗)).ti,ab. (34920)
• 166((comparative adj3 (efficacy or effectiveness)) or relative effectiveness or ((indirect or indirect treatment or mixed-treatment) adj comparison∗)).ti,ab. (56974)
• 167or/154-166 (1290258)
• 168clinical trials.sh. (11674)
• 169(randomi#ed or randomi#ation or randomi#ing).ti,ab,id. (2417986)
• 170(RCT or at random or (random∗ adj3 (assign∗ or allocat∗ or control∗ or crossover or cross-over or design∗ or divide∗ or division or number))).ti,ab,id. (1850783)
• 171(control∗ and (trial or study or group) and (placebo or waitlist∗ or wait∗ list∗ or ((treatment or care) adj2 usual))).ti,ab,id,hw. (823304)
• 172((single or double or triple or treble) adj2 (blind∗ or mask∗ or dummy)).ti,ab,id. (702771)
• 173trial.ti. (855068)
• 174placebo.ti,ab,id,hw. (993841)
• 175treatment outcome.md. (20440)
• 176treatment effectiveness evaluation.sh. (24182)
• 177mental health program evaluation.sh. (2121)
• 178or/168-177 (3687086)
• 179167 or 178 (4633927)
• 180134 and (135 or 153) and 179 (2626)
• 181(2018∗ or 2019∗ or 2020∗).yr,an. (7724266)
• 182180 and 181 (846)
• 183182 use psyb (150)
• 18414 or 73 or 126 or 183 (1036)
• 185184 use cctr (362)
• 186184 use coch (4)
• 187184 use clhta (0)
• 188184 use cleed (0)
• 189184 use emez (224)
• 190184 use medall (296)
• 191184 use psyb (150)
• 192remove duplicates from 184 (725)

Economic Evidence Search

Search date: June 2, 2020

Databases searched: Ovid MEDLINE, Embase, APA PsycInfo, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Centre for Reviews and Dissemination (CRD) Health Technology Assessment Database, and National Health Service (NHS) Economic Evaluation Database

Database: EBM Reviews – Cochrane Central Register of Controlled Trials <April 2020>, EBM Reviews −Cochrane Database of Systematic Reviews <2005 to May 29, 2020>, EBM Reviews – Health Technology Assessment <4th Quarter 2016>, EBM Reviews – NHS Economic Evaluation Database <1st Quarter 2016>, Embase <1980 to 2020 Week 22>, Ovid MEDLINE(R) ALL <1946 to June 01, 2020>, APA PsycInfo <1967 to May Week 4 2020>

Search strategy:

• 1“Trauma and Stressor Related Disorders"/ or stress disorders, traumatic/ or combat disorders/ or psychological trauma/ or stress disorders, post-traumatic/ or stress disorders, traumatic, acute/ (93253)
• 2(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,kf. (136702)
• 3(((acute or traumatic) adj stress∗) and (expos∗ or psyc∗)).ti,ab,kf. (52327)
• 4(traumati#ed adj (victim? or survivor?)).ti,ab,kf. (170)
• 5(trauma∗ adj2 (event? or memor∗ or flashback∗ or nightmare?)).ti,ab,kf. (36952)
• 6((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure adj3 (therap∗ or psychotherap∗ or training or counsel∗))).ti,ab,kf. (4230)
• 7or/1-6 (200283)
• 8(((internet or web or online) adj3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT).ti,ab,kf. (16075)
• 9(android or app or apps or blog∗ or CD-ROM or cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or e-therap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or i-phone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW).ti,ab,kf,hw. (4798652)
• 10computer communication networks/ or internet/ or blogging/ or social media/ (268043)
• 11cell phones/ or smartphone/ or text messaging/ or videoconferencing/ or webcasts as topic/ or wireless technology/ (64416)
• 12(telecomm∗ or tele-comm∗).ti,ab,kf. (11085)
• 13Telemedicine/ (53112)
• 14(eLearning or blended learning).ti,kf. (1780)
• 15(videoconferenc∗ or video conferenc∗).ti,kf. (2347)
• 16(synchronous or asynchronous or (electronic adj2 deliver∗)).ti,kf. (26041)
• 17or/9-16 (4824768)
• 18(behavio∗ or cognitive).ti. or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or cognitive processing or ((cognitive or behavio∗) adj2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) adj3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ adj3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety adj3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ adj3 suppress∗) or rumination).mp. (4729100)
• 1917 and 18 (376343)
• 207 and (8 or 19) (6346)
• 21economics/ (278899)
• 22economics, medical/ or economics, pharmaceutical/ or exp economics, hospital/ or economics, nursing/ or economics, dental/ (865363)
• 23economics.fs. (434246)
• 24(econom∗ or price or prices or pricing or priced or discount∗ or expenditure∗ or budget∗ or pharmacoeconomic∗ or pharmaco-economic∗).ti,ab,kf. (1112092)
• 25exp “costs and cost analysis"/ (642088)
• 26(cost or costs or costing or costly).ti. (292424)
• 27cost effective∗.ti,ab,kf. (366312)
• 28(cost∗ adj2 (util∗ or efficacy∗ or benefit∗ or minimi∗ or analy∗ or saving∗ or estimate∗ or allocation or control or sharing or instrument∗ or technolog∗)).ab,kf. (246466)
• 29models, economic/ (13605)
• 30markov chains/ or monte carlo method/ (88195)
• 31(decision adj1 (tree∗ or analy∗ or model∗)).ti,ab,kf. (50127)
• 32(markov or markow or monte carlo).ti,ab,kf. (146288)
• 33quality-adjusted life years/ (42978)
• 34(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).ti,ab,kf. (87019)
• 35((adjusted adj1 (quality or life)) or (willing∗ adj2 pay) or sensitivity analys∗s).ti,ab,kf. (139817)
• 36or/21-35 (2918673)
• 3720 and 36 (371)
• 38animals/ not humans/ (5512869)
• 3937 not 38 (369)
• 4039 use medall,coch,cctr (169)
• 4120 use cleed,clhta (2)
• 4240 or 41 (171)
• 43limit 42 to english language [Limit not valid in CDSR; records were retained] (124)
• 44limit 43 to yr="2018 −Current” (36)
• 45posttraumatic stress disorder/ (124134)
• 46(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,kw. (138527)
• 47(((acute or traumatic) adj stress∗) and (expos∗ or psyc∗)).ti,ab,kw. (53075)
• 48(traumati#ed adj (victim? or survivor?)).ti,ab,kw. (170)
• 49(trauma∗ adj2 (event? or memor∗ or flashback∗ or nightmare?)).ti,ab,kw. (37134)
• 50((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure adj3 (therap∗ or psychotherap∗ or training or counsel∗))).ti,ab,kw. (4323)
• 51or/45-50 (199028)
• 52(((internet or web or online) adj3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT).ti,ab,kw. (16265)
• 53(android or app or apps or blog∗ or CD-ROM or cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or e-therap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or i-phone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW).ti,ab,kw,hw. (4808331)
• 54internet/ (213035)
• 55blogging/ or e-mail/ or social media/ or text messaging/ or videoconferencing/ or webcast/ or wireless communication/ (79662)
• 56telecommunication/ or teleconference/ (31903)
• 57telemedicine/ or telehealth/ or telepsychiatry/ or teletherapy/ (67682)
• 58mobile phone/ or smartphone/ (45505)
• 59mobile application.hw. (11508)
• 60∗technology/ (51291)
• 61computer program/ or digital computer/ or personal computer/ or computer assisted therapy/ (377426)
• 62∗computer/ (42767)
• 63(telecomm∗ or tele-comm∗).ti,ab,kw. (11481)
• 64(eLearning or blended learning).ti,ab,kw. (3976)
• 65(videoconferenc∗ or video conferenc∗).ti,ab,kw. (9662)
• 66(synchronous or asynchronous or (electronic adj2 deliver∗)).ti,ab,kw. (105167)
• 67or/53-66 (4934058)
• 68(behavio∗ or cognitive).ti. or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or cognitive processing or ((cognitive or behavio∗) adj2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) adj3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ adj3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety adj3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ adj3 suppress∗) or rumination).mp. (4729100)
• 6967 and 68 (383550)
• 7051 and (52 or 69) (6438)
• 71Economics/ (278899)
• 72Health Economics/ or Pharmacoeconomics/ or Drug Cost/ or Drug Formulary/ (133481)
• 73Economic Aspect/ or exp Economic Evaluation/ (471261)
• 74(econom∗ or price or prices or pricing or priced or discount∗ or expenditure∗ or budget∗ or pharmacoeconomic∗ or pharmaco-economic∗).tw,kw. (1143582)
• 75exp “Cost"/ (600441)
• 76(cost or costs or costing or costly).ti. (292424)
• 77cost effective∗.tw,kw. (379416)
• 78(cost∗ adj2 (util∗ or efficac∗ or benefit∗ or minimi∗ or analy∗ or saving∗ or estimate∗ or allocation or control or sharing or instrument∗ or technolog∗)).ab,kw. (258212)
• 79Monte Carlo Method/ (68692)
• 80(decision adj1 (tree∗ or analy∗ or model∗)).tw,kw. (54337)
• 81(markov or markow or monte carlo).tw,kw. (151757)
• 82Quality-Adjusted Life Years/ (42978)
• 83(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).tw,kw. (91002)
• 84((adjusted adj1 (quality or life)) or (willing∗ adj2 pay) or sensitivity analys∗s).tw,kw. (161017)
• 85or/71-84 (2526715)
• 8670 and 85 (379)
• 87(exp animal/ or nonhuman/) not exp human/ (10683800)
• 8886 not 87 (377)
• 8988 use emez (150)
• 90limit 89 to english language [Limit not valid in CDSR; records were retained] (148)
• 91limit 90 to yr="2018 −Current” (42)
• 92posttraumatic stress disorder/ or complex ptsd/ or acute stress disorder/ or combat experience/ or “debriefing (psychological)"/ or emotional trauma/ or post-traumatic stress/ or exp stress reactions/ or traumatic neurosis/ (373516)
• 93exp DISASTERS/ (124293)
• 94(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,id. (137285)
• 95(((acute or traumatic) adj stress∗) and (expos∗ or psyc∗)).ti,ab,id. (52677)
• 96(trauma∗ adj2 (event? or memor∗ or flashback∗ or nightmare?)).ti,ab,id. (37188)
• 97((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure adj3 (therap∗ or psychotherap∗ or training or counsel∗))).ti,ab,id,hw. (5069)
• 98(traumati#ed adj (victim? or survivor?)).ti,ab,id. (174)
• 99or/92-98 (551664)
• 100(((internet or web or online) adj3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT).ti,ab,kf. (16075)
• 101(android or app or apps or blog∗ or CD-ROM or cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or etherap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or iphone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW).ti,ab,id,hw. (4791619)
• 102(telecomm∗ or tele-comm∗).ti,ab,id. (11057)
• 103(eLearning or blended learning).ti,ab,id. (3839)
• 104(videoconferenc∗ or video conferenc∗).ti,ab,id. (9512)
• 105(synchronous or asynchronous or (electronic adj2 deliver∗)).ti,ab,id. (104519)
• 106internet/ or websites/ (216484)
• 107mobile devices/ or cellular phones/ (15203)
• 108social media/ or online social networks/ or blog/ or online community/ or text messaging/ (51512)
• 109electronic communication/ or exp computer mediated communication/ or electronic learning/ (18558)
• 110online therapy/ or telemedicine/ (55648)
• 111telecommunications media/ (1417)
• 112teleconferencing/ (2430)
• 113technology/ or information technology/ or exp computer applications/ or computer software/ (339186)
• 114computers/ or computer games/ or digital computers/ or microcomputers/ (161495)
• 115or/101-114 (4998459)
• 116(behavio∗ or cognitive).ti. or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or cognitive processing or ((cognitive or behavio∗) adj2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) adj3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ adj3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety adj3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ adj3 suppress∗) or rumination).ti,ab,id,hw. (4680473)
• 117(self adj (care or change or guide∗ or help or intervention or manag∗ or support∗ or train∗)).ti,id. (51008)
• 118115 and (116 or 117) (388711)
• 11999 and (100 or 118) (8106)
• 120economics/ or economy/ (378443)
• 121pharmacoeconomics/ or health care economics/ (197522)
• 122(econom∗ or price or prices or pricing or priced or discount∗ or expenditure∗ or budget∗ or pharmacoeconomic∗ or pharmaco-economic∗).tw. (1115507)
• 123exp “costs and cost analysis"/ (642088)
• 124cost∗.ti. (314325)
• 125cost effective∗.tw. (373521)
• 126(cost∗ adj2 (util∗ or efficacy∗ or benefit∗ or minimi∗ or analy∗ or saving∗ or estimate∗ or allocation or control or sharing or instrument∗ or technolog∗)).ab. (243949)
• 127markov chains/ (23177)
• 128(decision adj1 (tree∗ or analy∗ or model∗)).tw. (53083)
• 129(markov or markow or monte carlo).tw. (148689)
• 130(QOLY or QOLYs or HRQOL or HRQOLs or QALY or QALYs or QALE or QALEs).tw. (90139)
• 131((adjusted adj1 (quality or life)) or (willing∗ adj2 pay) or sensitivity analys∗s).tw. (158047)
• 132or/120-131 (2445457)
• 133119 and 132 (462)
• 134(animal not human).po. (350251)
• 135133 not 134 (461)
• 136135 use psyb (65)
• 137limit 136 to english language [Limit not valid in CDSR; records were retained] (62)
• 138limit 137 to yr="2018 −Current” (16)
• 13944 or 91 or 138 (94)
• 140139 use medall (28)
• 141139 use emez (42)
• 142139 use psyb (16)
• 143139 use cctr (6)
• 144139 use coch (2)
• 145139 use cleed (0)
• 146139 use clhta (0)
• 147remove duplicates from 139 (72)

Quantitative Evidence of Preferences and Values Search

Search date: June 5, 2020

Databases searched: Ovid MEDLINE, Cumulative Index to Nursing & Allied Health Literature (CINAHL)

Search filter used: Quantitative preference evidence filter, modified from Selva et al⁷²

Database: Ovid MEDLINE(R) ALL <1946 to June 04, 2020>

Search strategy:

• 1“Trauma and Stressor Related Disorders"/ or stress disorders, traumatic/ or combat disorders/ or psychological trauma/ or stress disorders, post-traumatic/ or stress disorders, traumatic, acute/ (35363)
• 2(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) adj3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗).ti,ab,kf. (38568)
• 3(((acute or traumatic) adj stress∗) and (expos∗ or psyc∗)).ti,ab,kf. (13445)
• 4(traumati#ed adj (victim? or survivor?)).ti,ab,kf. (37)
• 5(trauma∗ adj2 (event? or memor∗ or flashback∗ or nightmare?)).ti,ab,kf. (9818)
• 6((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure adj3 (therap∗ or psychotherap∗ or training or counsel∗))).ti,ab,kf. (1054)
• 7or/1-6 (57327)
• 8(((internet or web or online) adj3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT).ti,ab,kf. (3859)
• 9(android or app or apps or blog∗ or CD-ROMor cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or e-therap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or i-phone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW).ti,ab,kf,hw. (1640369)
• 10computer communication networks/ or internet/ or blogging/ or social media/ (91715)
• 11cell phones/ or smartphone/ or text messaging/ or videoconferencing/ or webcasts as topic/ or wireless technology/ (19644)
• 12(telecomm∗ or tele-comm∗).ti,ab,kf. (4427)
• 13Telemedicine/ (22267)
• 14(eLearning or blended learning).ti,kf. (575)
• 15(videoconferenc∗ or video conferenc∗).ti,kf. (858)
• 16(synchronous or asynchronous or (electronic adj2 deliver∗)).ti,kf. (11997)
• 17or/9-16 (1654050)
• 18(behavio∗ or cognitive).ti. or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or cognitive processing or ((cognitive or behavio∗) adj2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) adj3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ adj3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety adj3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ adj3 suppress∗) or rumination).mp. (1643862)
• 1917 and 18 (113780)
• 207 and (8 or 19) (1606)
• 21Attitude to Health/ (83231)
• 22Health Knowledge, Attitudes, Practice/ (110442)
• 23Patient Participation/ (25619)
• 24Patient Preference/ (8374)
• 25Attitude of Health Personnel/ (121160)
• 26∗Professional-Patient Relations/ (11609)
• 27∗Physician-Patient Relations/ (35424)
• 28Choice Behavior/ (32236)
• 29(choice or choices or value∗ or valuation∗ or knowledg∗).ti. (262010)
• 30(preference∗ or expectation∗ or attitude∗ or acceptab∗ or point of view).ti,ab,kf. (584809)
• 31((patient∗1 or user∗1 or men or women or personal or provider∗ or practitioner∗ or professional∗1 or (health∗ adj2 worker∗) or clinician∗ or physician∗ or doctor∗ or nurse∗ or practitioner∗ or counselor∗ or counsellor∗ or therapist∗) adj2 (participation or perspective∗ or perception∗ or misperception∗ or perceiv∗ or view∗ or understand∗ or misunderstand∗ or value∗1 or knowledg∗)).ti,ab,kf. (154814)
• 32health perception∗.ti,ab,kf. (2746)
• 33∗Decision Making/ (41956)
• 34(patient∗1 or user∗1 or men or women or personal or provider∗ or practitioner∗ or professional∗1 or (health∗ adj2 worker∗) or clinician∗ or physician∗ or doctor∗ or nurse∗ or practitioner∗ or counselor∗ or counsellor∗ or therapist∗).ti. (2541961)
• 3533 and 34 (8373)
• 36(decision∗ and mak∗).ti. (28906)
• 37(decision mak∗ or decisions mak∗).ti,ab,kf. (147338)
• 3836 or 37 (148825)
• 39(patient∗1 or user∗1 or men or women or personal or provider∗ or practitioner∗ or professional∗1 or (health∗ adj2 worker∗) or clinician∗ or physician∗ or doctor∗ or nurse∗ or practitioner∗ or counselor∗ or counsellor∗ or therapist∗).ti. (2541961)
• 4038 and 39 (30304)
• 41(discrete choice∗ or decision board∗ or decision analy∗ or decision-support or decision tool∗ or decision aid∗ or latent class∗ or decision∗ conflict∗ or decision∗ regret∗).ti,ab,kf. (35945)
• 42Decision Support Techniques/ (20173)
• 43(health and utilit∗).ti. (1471)
• 44(gamble∗ or prospect theory or health utilit∗ or utility value∗ or utility score∗ or utility estimate∗ or health state or feeling thermometer∗ or best-worst scaling or time trade-off or TTO or probability trade-off).ti,ab,kf. (13264)
• 45(preference based or preference score∗ or preference elicitation or multiattribute or multi attribute).ti,ab,kf. (2830)
• 46or/21-32,35,40-45 (1247073)
• 4720 and 46 (228)
• 48limit 47 to english language (221)

CINAHL

#	Query	Results
S1	(MH “Stress Disorders, Post-Traumatic+”)	24,538
S2	(MH “Psychological Trauma”)	1,286
S3	(PTSD or ((posttrauma∗ or post-trauma∗ or post trauma∗) N3 (stress∗ or disorder∗ or psych∗ or symptom?)) or acute stress disorder∗ or combat disorder∗ or war neuros∗)	29,894
S4	(((acute or traumatic) N1 stress∗) and (expos∗ or psyc∗))	7,993
S5	(traumati#ed N1 (victim? or survivor?))	27
S6	(trauma∗ N2 (event? or memor∗ or flashback∗ or nightmare?))	4,916
S7	((trauma∗ or posttrauma∗ or post-trauma∗ or victim∗ or survivor?) and (exposure N3 (therap∗ or psychotherap∗ or training or counsel∗)))	649
S8	S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7	34,877
S9	(((internet or web or online) N3 (cognitive or behavio∗)) or iCBT or i-CBT or ePsych∗ or e-Psych∗ or cCBT or c-CBT)	29,595
S10	(android or app or apps or blog∗ or CD-ROM or cell phone or cellphone or chat room or computer∗ or cyber∗ or digital or technology based or DVD or eHealth or e-health or electronic health or e-mail∗ or email∗ or e-Portal or ePortal or eTherap∗ or e-therap∗ or forum∗ or gaming or information technolog∗ or instant messag∗ or messaging or internet∗ or ipad or i-pad or iphone or i-phone or ipod or i-pod or podcast or smart phone or smartphone or social network∗ site∗ or social networking or mHealth or m-health or mobile or multi-media or multimedia or online∗ or on-line or personal digital assistant or PDA or SMS or social medi∗ or software or telecomm∗ or telehealth∗ or telemed∗ or telemonitor∗ or telepsych∗ or teletherap∗ or tele-health∗ or tele-med∗ or tele-monitor∗ or tele-psych∗ or tele-therap∗ or text messag∗ or texting or virtual∗ or web∗ or WWW)	1,112,376
S11	(MH “Computer Communication Networks”)	3,015
S12	(MH “Internet”)	52,212
S13	(MH “Blogs”)	3,752
S14	(MH “Social Media”)	15,453
S15	(MH “Smartphone”)	2,720
S16	(MH “Text Messaging”)	3,028
S17	(MH “Videoconferencing”)	1,777
S18	(telecomm∗ or tele-comm∗)	3,616
S19	(MH “Remote Consultation”) OR (MH “Telemedicine”) OR (MH “Telenursing”) OR (MH “Telepsychiatry”)	15,309
S20	TI(eLearning or blended learning)	485
S21	TI(videoconferenc∗ or video conferenc∗)	542
S22	TI(synchronous or asynchronous or (electronic N2 deliver∗))	1,878
S23	S10 OR S11 OR S12 OR S13 OR S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22	1,116,095
S24	TI(behavio∗ or cognitive) or (psychotherap∗ or psychological therap∗ or cognitive behavio∗ or cognitive processing or ((cognitive or behavio∗) N2 (activat∗ or component? or defusion or modif∗ or restructur∗ or technique∗ or intervention or treatment∗ or therap∗ or train∗)) or ((acceptance∗ or commitment∗) N3 therap∗) or rational emotive or RET or problem sol∗ or PST or problem focus∗ or solution focus∗ or trauma focus∗ or psychoeducat∗ or psycho-educat∗ or psychodrama or psycho-drama∗ or mindfulness∗ or third wave or self-control or (self∗ N3 (control or efficacy)) or stress manage∗ or exposure or reality therap∗ or (anxiety N3 (management or therap∗ or train∗)) or relaxation or guided imagery or present cent∗ or person cent∗ or person∗ construct∗ or therapeutic process∗ or schema? or schemata or (thought∗ N3 suppress∗) or rumination)	486,812
S25	S23 AND S24	104,161
S26	S8 AND (S9 or S25)	2,386
S27	(MH “Attitude to Health”)	44,265
S28	(MH “Health Knowledge”)	33,931
S29	(MH “Consumer Participation”)	20,881
S30	(MH “Patient Preference”)	709
S31	(MH “Attitude of Health Personnel”)	45,976
S32	(MM “Professional-Patient Relations”)	14,189
S33	(MM “Physician-Patient Relations”)	16,828
S34	(MM “Nurse-Patient Relations”)	14,926
S35	TI (choice or choices or value∗ or valuation∗ or knowledg∗)	101,147
S36	(preference∗ or expectation∗ or attitude∗ or acceptab∗ or point of view)	481,212
S37	((patient or patients or user or users or men or women or personal or provider∗ or practitioner∗ or professional or professionals or (health∗ N2 worker∗) or clinician∗ or physician∗ or doctor∗ or nurse∗ or practitioner∗ or counselor∗ or counsellor∗ or therapist∗) N2 (participation or perspective∗ or perception∗ or misperception∗ or perceiv∗ or view∗ or understand∗ or misunderstand∗ or value or values or knowledg∗))	866,825
S38	health perception∗	4,716
S39	(MH “Decision Making, Shared”)	1,522
S40	(MH “Decision Making, Patient”)	16,656
S41	(MH “Decision Making, Family”)	4,417
S42	(MM “Decision Making”)	24,127
S43	TI (patient or patients or user or users or men or women or personal or provider∗ or practitioner∗ or professional or professionals or (health∗ N2 worker∗) or clinician∗ or physician∗ or doctor∗ or nurse∗ or practitioner∗ or counselor∗ or counsellor∗ or therapist∗)	1,183,240
S44	S42 AND S43	4,671
S45	TI (decision∗ and mak∗)	18,815
S46	(decision mak∗ or decisions mak∗)	162,308
S47	S45 OR S46	162,531
S48	(patient or patients or user or users or men or women or personal or provider∗ or practitioner∗ or professional or professionals or (health∗ N2 worker∗) or clinician∗ or physician∗ or doctor∗ or nurse∗ or practitioner∗ or counselor∗ or counsellor∗ or therapist∗)	3,443,344
S49	S47 AND S48	115,277
S50	(discrete choice∗ or decision board∗ or decision analy∗ or decision support or decision tool∗ or decision aid∗ or latent class∗ or decision∗ conflict∗ or decision∗ regret∗)	29,174
S51	(MH “Decision Support Techniques”)	6,912
S52	TI (health and utilit∗)	943
S53	(gamble∗ or prospect theory or health utilit∗ or utility value∗ or utility score∗ or utility estimate∗ or health state or feeling thermometer∗ or best worst scaling or time trade off or TTO or probability trade off)	17,308
S54	(preference based or preference score∗ or preference elicitation or multiattribute or multi attribute)	1,554
S55	S27 OR S28 OR S29 OR S30 OR S31 OR S32 OR S33 OR S34 OR S35 OR S36 OR S37 OR S38 OR S39 OR S40 OR S41 OR S44 OR S49 OR S50 OR S51 OR S52 OR S53 OR S54	1,318,538
S56	S26 AND S55	758
	S26 AND S55
557	Limiters – English Language	752

Grey Literature Search

Search dates: May 29–June 2, 2020

Websites searched: Alberta Health Evidence Reviews, Alberta Health Services, BC Health Technology Assessments, Canadian Agency for Drugs and Technologies in Health (CADTH), Institut national d'excellence en santé et en services sociaux (INESSS), Institute of Health Economics (IHE), McGill University Health Centre Health Technology Assessment Unit, Centre Hospitalier de l'Universite de Quebec-Universite Laval, Health Technology Assessment Database, National Institute for Health and Care Excellence (NICE), Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Centers, Australian Government Medical Services Advisory Committee, Council of Australian Governments Health Technologies, Centers for Medicare & Medicaid Services Technology Assessments, Institute for Clinical and Economic Review, Ireland Health Information and Quality Authority Health Technology Assessments, Washington State Health Care Authority Health Technology Reviews, Health Technology Wales, Oregon Health Authority Health Evidence Review Commission, Veterans Affairs Health Services Research and Development, Italian National Agency for Regional Health Services (AGENAS), Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIPS), Belgian Health Care Knowledge Centre, Ludwig Boltzmann Institute for Health Technology Assessment, Ministry of Health Malaysia Health Technology Assessment Section, Swedish Agency for Health Technology Assessment and Assessment of Social Services, PROSPERO, EUnetHTA, Tufts Cost-Effectiveness Analysis Registry, SickKids Paediatric Economic Database Evaluation (PEDE) database

Keywords used: cognitive behavioural therapy, CBT, post-traumatic stress disorder, PTSD, stress, psychology

Clinical results (included in PRISMA): 4

Economic results (included in PRISMA): 5

Ongoing health technology assessments (PROSPERO/EUnetHTA): 1

Ongoing randomized controlled trials (clinicaltrials.gov): 8

Appendix 2: Critical Appraisal of Clinical Evidence

Table A1:

Risk of Bias^a Among Randomized Controlled Trials (Cochrane Risk-of-Bias Tool)

Author, Year	Random Sequence Generation	Allocation Concealment	Blinding of Participants and Personnel	Incomplete Outcome Data	Selective Reporting	Other Bias
Engel et al, 201533	Low	Unclear	High	Low	Low	Low
Ivarsson et al, 201434	Low	Unclear	High	Low	Low	High
Knaevelsrud et al, 201535	Low	Unclear	High	High	Low	Low
Krupnick et al, 201736	Unclear	Unclear	High	High	Low	High
Kuhn et al, 201737	Low	Unclear	High	Low	Low	Low
Lewis et al, 201738	Low	Low	High	Low	Low	High
Littleton et al, 201639	Low	Unclear	High	Low	Low	Low
Litz et al, 200740	Unclear	Unclear	High	High	Low	High
Miner et al, 201641	Unclear	Unclear	High	Low	Low	Low
Spence et al, 201142	Low	Unclear	High	High	Low	Unclear

^aPossible risk-of-bias levels: low, high, and unclear.

Source: As reported in the systematic review by CADTH, 2019.¹⁸

Table A2:

Risk of Bias^a Among Systematic Reviews (ROBIS Tool)

	Phase 2				Phase 3
Author, Year	Study Eligibility Criteria	Identification and Selection of Studies	Data Collection and Study Appraisal	Synthesis and Findings	Risk of Bias in the Review
CADTH, 201918	Low	Low	Low	Low	Low
NICE, 201824	Low	Low	Low	Low	Low

Abbreviation: ROBIS, Risk of Bias in Systematic Reviews.

^aPossible risk-of-bias levels: low, high, unclear.

Table A3:

GRADE Evidence Profile for the Comparison of iCBT Versus Wait-List or Usual Care for PTSD or ASD

Number of Studies (Design)	Risk of Bias	Inconsistency	Indirectness	Imprecision	Publication Bias	Upgrade Considerations	Quality
Severity of ASD Symptoms
No studies	-	-	-	-	-	-	-
Prevention of PTSD After Diagnosis With ASD
No studies	-	-	-	-	-	-	-
Severity of PTSD Symptoms
8 (RCTs)	Very serious limitations (-2)a	Serious limitations (-1)b	No serious limitations	No serious limitations	Undetected	-	Very low
Diagnosis of PTSD After Treatment
1 (RCT)	Very serious limitations (-2)c	No serious limitations	No serious limitations	Serious limitations (-1)d	Undetected	-	Very low
Severity of Depression Symptoms in People Diagnosed With PTSD
5 (RCTs)	Very serious limitations (-2)e	Serious limitations (-1)b	No serious limitations	No serious limitations	Undetected	-	Very low
Severity of Anxiety Symptoms in People Diagnosed With PTSD
4 (RCTs)	Very serious limitations (-2)f	No serious limitations	No serious limitations	Serious limitations (-1)g	Undetected	-	Very low
Dropout Rates
8 (RCTs)	Very serious limitations (-2)a	No serious limitations	No serious limitations	No serious limitations	Undetected	-	Low
Quality of Life
2 (RCTs)	Very serious limitations (-2)h	Serious limitations (-1)b	No serious limitations	No serious limitations	Undetected	-	Very low
Adverse Events
No studies	-	-	-	-	-	-	—

Abbreviations: ASD, acute stress disorder; CADTH, Canadian Agency for Drugs and Technologies in Health; GRADE, Grading of Recommendations Assessment, Development, and Evaluation; iCBT, internet-delivered cognitive behavioural therapy; PTSD, post-traumatic stress disorder; RCT, randomized controlled trial.

^aDowngraded 2 levels due to high risk of performance bias in all 8 studies, high risk of attrition bias in 2 studies (Knaevelsrud et al,³⁵ Krupnick et al³⁶), and high risk of other bias in 3 studies (Ivarsson et al,³⁴ Krupnick et al,³⁶ Lewis et al³⁸).

^bDowngraded 1 level for inconsistency; high levels of heterogeneity.

^cDowngraded 2 levels due high risk of performance bias and other bias (Ivarsson et al³⁴).

^dDowngraded 1 level for imprecision due to small sample size; in addition, the confidence interval around the effect estimate includes both little or no effect.

^eDowngraded 2 levels due to high risk of performance bias in all 5 studies, high risk of attrition bias in 1 study (Krupnick et al³⁶), and high risk of other bias in 2 studies (Krupnick et al,³⁶ Lewis et al³⁸).

^fDowngraded 2 levels due to high risk of performance bias in all 4 studies, high risk of attrition bias in 1 study (Knaevelsrud et al³⁵), and high risk of other bias in 2 studies (Lewis et al,³⁸ Spence et al⁴²).

^gDowngraded 1 level for imprecision due to small sample size.

^hDowngraded 2 levels due to high risk or performance bias in both studies, high risk of attrition bias (Knaevelsrud et al³⁵), and high risk of other bias (Ivarsson et al³⁴).

Source: As reported in the systematic reviews by CADTH, 2019¹⁸ and Lewis et al, 2018.²⁵

Table A4:

GRADE Evidence Profile for the Comparison of iCBT Versus Non–CBT Internet-Delivered Interventions for PTSD or ASD

Number of Studies (Design)	Risk of Bias	Inconsistency	Indirectness	Imprecision	Publication Bias	Upgrade Considerations	Quality
Severity of ASD Symptoms
No studies	-	-	-	-	-	-	-
Prevention of PTSD After Diagnosis With ASD
No studies	-	-	-	-	-	-	-
Severity of PTSD Symptoms
2 RCTs	Very serious limitations (-2)a	No serious limitations	No serious limitations	Very serious limitations (-2)b	Undetected	-	Very low
Diagnosis of PTSD After Treatment
No studies	-	-	-	-	-	-	—
Severity of Depression Symptoms in People Diagnosed With PTSD
2 RCTs	Very serious limitations (-2)a	No serious limitations	No serious limitations	Very serious limitations (-2)b	Undetected	-	Very low
Severity of Anxiety Symptoms in People Diagnosed With PTSD
2 RCTs	Very serious limitations (-2)a	Serious limitations (-1)c	No serious limitations	Very serious limitations (-2)b	Undetected	-	Very low
Dropout Rates
2 RCTs	Very serious limitations (-2)a	No serious limitations	No serious limitations	Very serious limitations (-2)b	Undetected	-	Very low
Quality of Life
Not studies	-	-	-	-	-	-	-
Adverse Events
No studies	-	-	-	-	-	-	-

Abbreviations: ASD, acute stress disorder; CBT, cognitive behavioural therapy; GRADE, Grading of Recommendations Assessment, Development, and Evaluation; iCBT, internet-delivered cognitive behavioural therapy; PTSD, post-traumatic stress disorder; RCT, randomized controlled trial.

^aDowngraded 2 levels for high risk of performance bias due to lack of blinding participants and personnel in both studies (Littleton et al³⁹; Litz et al⁴⁰), high risk of detection bias due to lack of blinding outcome assessors in 1 study (Littleton et al³⁹), and high risk of attrition bias and other bias in 1 study (Litz et al⁴⁰).

^bDowngraded 2 levels for imprecision due to small sample size; in addition, the confidence intervals of the effect estimate includes both little or no effect.

^cDowngraded 1 level for inconsistency due to high levels of heterogeneity.

Source: As reported in the systematic reviews by CADTH, 2019¹⁸ and Lewis et al, 2018.²⁵

Appendix 3: Selected Excluded Studies—Clinical Evidence

For transparency, we provide a list of studies that readers might have expected to see but that did not meet the inclusion criteria, along with the primary reason for exclusion.

Citation	Primary Reason for Exclusion
Ehlers A, Wild J, Warnock-Parkes E, Grey N, Murray H, Kerr A et al. A randomised controlled trial of therapist assisted online psychological therapies for posttraumatic stress disorder (STOP-PTSD): trial protocol. Trials. 2020;21(1):355.	Protocol only
Bangpan M, Felix L, Dickson K. Mental health and psychosocial support programmes for adults in humanitarian emergencies: a systematic review and meta-analysis in low and middle income countries. BMJ Global Health. 2019;4:e001484.	Included nonrandomized studies; PTSD not primary question
Andersson G, Carlbring P, Titov N, Lindefors N. Internet interventions for adults with anxiety and mood disorders: a narrative umbrella review of recent meta-analyses. Can J Psychiatry. 2019 Jul;64(7):465-470.	Included one systematic review of iCBT for PTSD from 2016
Lewis C, Roberts NP, Andrew M, Starling E, Bisson JI. Psychological therapies for post-traumatic stress disorder in adults: systematic review and meta-analysis. Eur J Psychotraumatol. 2020;11(1):1729633.	Multiple interventions assessed; focus not on iCBT
Stefanopoulou E, Lewis D, Mughal A, Larkin J. Digital interventions for PTSD symptoms in the general population: a review. Psychiatr Q. 2020 May 14.	No formal diagnosis of PTSD required in included studies
Sijbrandij M, Kunovski I, Cuijpers P. Effectiveness of internet-delivered cognitive behavioral therapy for posttraumatic stress disorder: a systematic review and meta-analysis. Depress Anxiety. 2016;33(9):783-91.	No formal diagnosis of PTSD required in included studies
Niemeyer H, Knaevelsrud C, Schumacher S, Engel S, Kuester A, Burchert S et al. Evaluation of an internet-based intervention for service members of the German armed forces with deployment related posttraumatic stress symptoms. BMC Psychiatry. 2020 May 6;20(1):205.	Combined intervention and control groups for statistical analysis

Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; PTSD, post-traumatic stress disorder.

Appendix 4: Results of Applicability and Limitation Checklists for Studies Included in the Economic Literature Review

Table A5:

Assessment of the Applicability of Studies Evaluating the Cost-Effectiveness of iCBT for PTSD in the Canadian Setting

Author, Year	Is the study population similar to the question?	Are the interventions similar to the question?	Is the health care system studied sufficiently similar to Ontario?	Were the perspectives clearly stated? If yes, what were they?	Are all direct effects included? Are all other effects included where they are material?	Are all future costs and outcomes discounted? If yes, at what rate?	Is the value of health effects expressed in terms of quality-adjusted life-years?	Are costs and outcomes from other sectors fully and appropriately measured and valued?	Overall Judgmenta
CADTH, 201918	Yes	Yes	Yes	Yes, Canadian health care payer	Yes	Yes, 1.5%	Yes	Yes Directly applicable

Abbreviations: CADTH, Canadian Agency for Drugs and Technologies in Health; iCBT, internet-delivered cognitive behavioral therapy; PTSD, post-traumatic stress disorder.

Note: Response options for all items were “yes,” “partially,” “no,” “unclear,” and “NA” (not applicable).

^aOverall judgment may be “directly applicable,” “partially applicable,” or “not applicable.”

Table A6:

Assessment of the Limitations of Studies Evaluating the Cost-Effectiveness of iCBT for PTSD in the Canadian Setting

Author, Year	Does the model structure adequately reflect the nature of the health condition under evaluation?	Is the time horizon sufficiently long to reflect all important differences in costs and outcomes?	Are all important and relevant health outcomes included?	Are the clinical inputsa obtained from the best available sources?	Do the estimates of relative treatment effect match the estimates contained in the clinical sources?	Are all important and relevant (direct) costs included in the analysis?	Are the estimates of resource use obtained from best available sources?	Are the unit costs of resources obtained from best available sources?	Is an appropriate incremental analysis presented or can it be calculated from the reported data?	Are all important and uncertain parameters subjected to appropriate sensitivity analysis?	Is there a potential conflict of interest?	Overall Judgmentb
CADTH, 201918	Yes	Yes, lifetime horizon	Yes	Yes, systematic review	Yes	Yes	Yes (Canada and Ontario)	Yes	Yes	Yes	No	Minor limitations

Abbreviations: CADTH, Canadian Agency for Drugs and Technologies in Health; iCBT, internet-delivered cognitive behavioral therapy; PTSD, post-traumatic stress disorder.

Note: Response options for all items were “yes,” “partially,” “no,” “unclear,” and “NA” (not applicable).

^aClinical inputs include relative treatment effects, natural history, and utilities.

^bOverall judgment may be “minor limitations,” “potentially serious limitations,” or “very serious limitations.”

Appendix 5: Budget Impact Analysis Calculations

Table A7:

Annual Unit Costs for Adults Receiving and Not Receiving iCBT

		Unit Cost ($)a
Cost Item	Type of Analysis	Year 1	Year 2	Year 3	Year 4	Year 5
iCBT
Treatment costs	Probabilisticb	253.69	0	0	0	0
	Deterministicc	253.53	0	0	0	0
Health state costsd	Probabilisticb	772.78	715.82	685.56	658.34	633.82
	Deterministicc	756.89	695.00	665.11	639.45	617.38
Total	Probabilisticb	1026.47	715.82	685.56	658.34	633.82
	Deterministicc	1,010.42	695.00	665.11	639.45	617.38
Usual care
Treatment costs	Probabilisticb	0	0	0	0	0
	Deterministicc	0	0	0	0	0
Health state costsd	Probabilisticb	844.03	800.43	761.29	726.11	694.47
	Deterministicc	836.70	786.16	742.88	705.78	673.94
Total	Probabilisticb	844.03	800.43	761.29	726.11	694.47
	Deterministicc	836.70	786.16	742.88	705.78	673.94

Abbreviation: iCBT, internet-delivered cognitive behavioural therapy.

^aAll costs are presented in 2020 Canadian dollars.

^bThe probabilistic approach represented the average value of 5,000 Monte Carlo simulations following the probabilistic sensitivity analysis approach. These values were used in the reference case.

^cThe deterministic approach represented the value used in the scenario analysis.

^dThe health state costs combined costs for both people with active PTSD and those in remission.

Table A8:

Scenario Analysis Parameters

Scenario	Cost	Treatment Effect	Uptake Rate
Reference case	$253. 53 (3 h of therapist support; Gamma distribution, mean = 3, SD = 3)	OR = 2.97 (1.55 to 5.81)	Year 1: 3% Year 2: 6% Year 3: 9% Year 4: 12% Year 5: 15%
Unguided iCBT	$146.78 (0.5 h of therapist support; beta distribution, α = 50, β = 50)	OR = 1.28 (0.67 to 2.48)	-a
Guided iCBT	$274.88 (3.5 h of therapist support; gamma distribution, mean = 3.5, SD = 3)	OR = 4.27 (2.14 to 8.50)	-a
Moderate uptake	-a	-a	Year 1: 10% (44,332) Year 2: 15% (67,572) Year 3: 20% (91,425) Year 4: 25% (115,807) Year 5: 30% (140,646)
High uptake	-a	-a	Year 1: 30% (265,990) Year 2: 35% (315,334) Year 3: 40% (365,699) Year 4: 45% (416,906) Year 5: 50% (468,820)
Varying number of e-therapist hours	4 h: $296.23 5 h: $338.93 6 h: $381.63 7 h: $424.33 8 h: $467.03 9 h: $509.73 10 h: $552.43	OR = 4.27 (2.14 to 8.50)	-a
Mixed program (face-to-face CBT and iCBT)	$1,386.59 (6 face-to-face sessions with nonphysician therapist, 5 h supervision [half of a conventional 12-session face-to-face CBT program]55 plus cost of unguided iCBT program)	OR = 4.27 (2.14 to 8.50)	Year 1: 3% Year 2: 6% Year 3: 9% Year 4: 12% Year 5: 15%

Abbreviations: iCBT, internet-delivered cognitive behavioural therapy; CBT, cognitive behavioural therapy; h, hour(s); OR, odds ratio; PTSD, post-traumatic stress disorder; SD, standard deviation.

^aAssumed to be the same as reference case.

Table A9:

Budget Impact—Varying Number of e-Therapist Hours

	Budget Impact, $ Milliona,b,c,d
Scenario	Year 1	Year 2	Year 3	Year 4	Year 5
Current scenario: treatment costse	0	0	0	0	0
Current scenario: health state costse	374.25	735.27	1,084.30	1,422.36	1,750.38
Current scenario: total costse	374.25	735.27	1,084.30	1,422.36	1,750.38
4 hours
New scenario: treatment costs	3.94	8.01	12.19	16.47	20.83
New scenario: health state costs	372.69	730.29	1,074.16	1,405.46	1,725.24
New scenario: total costs	376.64	738.30	1,086.35	1,421.93	1,746.08
Budget impact	2.39	3.03	2.05	-0.43	-4.31
5 hours
New scenario: treatment costs	4.51	9.16	13.95	18.84	23.84
New scenario: health state costs	372.69	730.29	1,074.16	1,405.46	1,725.24
New scenario: total costs	377.20	739.46	1,088.10	1,424.30	1,749.08
Budget impact	2.96	4.18	3.81	1.94	-1.31
6 hours
New scenario: treatment costs	5.08	10.32	15.70	21.22	26.84
New scenario: health state costs	372.69	730.29	1,074.16	1,405.46	1,725.24
New scenario: total costs	377.77	740.61	1,089.86	1,426.67	1,752.08
Budget impact	3.52	5.34	5.56	4.32	1.70
7 hours
New scenario: treatment costs	5.64	11.47	17.46	23.59	29.84
New scenario: health state costs	372.69	730.29	1,074.16	1,405.46	1,725.24
New scenario: total costs	378.34	741.76	1,091.62	1,429.05	1,755.08
Budget impact	4.09	6.49	7.32	6.69	4.70
8 hours
New scenario: treatment costs	6.21	12.62	19.22	25.96	32.85
New scenario: health state costs	372.69	730.29	1,074.16	1,405.46	1,725.24
New scenario: total costs	378.91	742.92	1,093.37	1,431.42	1,758.09
Budget impact	4.66	7.65	9.08	9.06	7.70
9 hours
New scenario: treatment costs	6.78	13.78	20.97	28.34	35.85
New scenario: health state costs	372.69	730.29	1,074.16	1,405.46	1,725.24
New scenario: total costs	379.47	744.07	1,095.13	1,433.80	1,761.09
Budget impact	5.23	8.80	10.83	11.44	10.71
New scenario: treatment costs	7.35	14.93	22.73	30.71	38.85
New scenario: health state costs	372.69	730.29	1,074.16	1,405.46	1,725.24
New scenario: total costs	380.04	745.22	1,096.89	1,436.17	1,764.09
Budget impact	5.80	9.95	12.59	13.81	13.71

^aAll costs are presented in 2020 Canadian dollars.

^bNegative costs indicate savings.

^cResults may appear inexact due to rounding.

^dFor each year, total costs were estimated as the cumulative costs for those who newly entered the model-based budget impact analysis and those who had entered the analysis in previous years. For example, in year 3, the total costs represent the total costs of three cohorts: health state costs for those who entered the analysis in years 1 and 2, and both the treatment costs and health state costs for those who entered the analysis in year 3.

^eBecause this scenario (considering varying numbers of e-therapist hours) had no impact on the current scenario (i.e., usual care), the current scenario would remain constant while the number of e-therapist hours changed.

Appendix 6: Letter of Information

Appendix 7: Interview Guide

Author contributions

This report was developed by a multidisciplinary team from Ontario Health. The clinical epidemiologist was Kristen McMartin, the primary health economist was Yuan Zhang, the secondary health economist was Olga Gajic-Veljanoski, the patient and public partnership analyst was Jigna Mistry, and the medical librarians were Corinne Holubowich and Caroline Higgins.

About Us

Ontario Health is an agency of the Government of Ontario. Our mandate is to connect and coordinate our province's health care system in ways that have not been done before to help ensure that Ontarians receive the best possible care. We work to support better health outcomes, patient experiences, provider experiences and value for money spent.

For more information about Ontario Health, visit ontariohealth.ca