Disorder-specific internet-based cognitive-behavioral therapy in treating panic disorder, comorbid symptoms and improving quality of life: A meta-analytic evaluation of randomized controlled trials

Abstract

Compared to conventional face-to-face psychological treatments, internet-based cognitive-behavioral therapy (iCBT) presents an innovative alternative that has been found to be effective in the treatment of anxiety disorders. The current study provides a meta-analysis investigating the efficacy of disorder-specific guided self-help (Gsh) iCBT compared to various active and inactive control conditions, with focus on adult panic disorder sufferers with or without agoraphobia (PD/A). Systematic literature search yielded 13 randomized controlled trials (RCTs) (N = 1214) that met the eligibility criteria for this study. We found no statistically significant differences between Gsh iCBT and various active CBT interventions in reducing PD/A symptoms at both post-test (g = 0.015, k = 10) and follow-up (g = 0.113, k = 6) levels. Also, comorbid anxiety and depression were reduced equivalently at post-test (g = 0.004, k = 6) and follow-up (g = 0.004, k = 6). Quality of life was equally improved at post-test (g = −0.100, k = 5) and follow-up (g = 0.074, k = 2). When compared to inactive controls, we found large effect sizes in PD/A (g = −0.892, k = 9) and comorbid anxiety and depression (g = −0.723, k = 9) symptoms, and moderate change in quality of life (g = −0.484, k = 3) at post-test. There was no difference between Guided self-help iCBT and Self-help iCBT in PD/A (g = −0.025, k = 3) and comorbid anxiety and depression (g = −0.025, k = 3) at post-test. Baseline severity, country of original research and adherence to the treatment in form of initial uptake were identified as statistically significant moderators of the iCBT treatment.

Highlights

• •Disorder-specific Gsh iCBT is equally effective in treating panic disorder with or without agoraphobia (PD/A), comorbid anxiety and depression symptoms, and improving quality of life, when compared to various active CBT interventions at post-test and follow-up
• •Gsh iCBT is highly effective in treating PD/A and comorbid anxiety and depression, and moderately effective in improving quality of life, when compared to inactive controls
• •Gsh iCBT is equally effective when compared to Sh iCBT in PD/A and comorbid anxiety and depression at post-test
• •Clinical significance, initial login into the treatment, adherence and attrition is equal in Gsh iCBT and psychological treatments with CBT elements
• •Baseline severity, country of original research and initial uptake to the treatment are moderators of the iCBT treatment effect

1. Introduction

1.1. Panic disorder

With a lifetime prevalence of 2–5% (Bienvenu, 2006; Kessler et al., 2006; McEvoy et al., 2011; Slade et al., 2009; Taylor, 2000), panic disorder (PD) is one of the most prevalent anxiety disorders worldwide. It is characterized by recurrent and unexpected panic attacks, resistance to spontaneous remission and high comorbidity with other mental health disorders (APA, 2013, Baillie and Rapee, 2005, Goodwin and Gotlib, 2004, Goodwin and Hamilton, 2001, Goodwin et al., 2004, Mattis and Ollendick, 2001). PD is associated with great discomfort in professional and social life (Mitte, 2005; Tsao et al., 2005), which leads, as a result, to significant deterioration in general quality of life (Rangé et al., 2011).

Kessler et al. (2006) report 45.0% comorbidity with other anxiety disorders, with highest comorbidity rates for specific phobia (21.0%) and social phobia (18.8%). Between 35 and 65% of individuals with panic disorder also meet criteria for agoraphobia, which is defined by intense fear and avoidance of situations where escaping or getting help may be difficult (APA, 2013; Wittchen et al., 2010).

1.2. Cognitive-behavioral therapy for panic disorder

Cognitive-behavioral therapy (CBT) is considered as “gold standard” among psychological treatments for anxiety disorders, particularly for its comprehensive scientific evidence base and superiority over alternative therapies such as psychodynamic therapy (Tolin, 2010). In treating panic disorder, traditional delivery of CBT in a face-to-face setting proves CBT to be effective not only in reducing panic symptoms, but also in co-occurring conditions such as residual anxiety or depression and improvements in quality of life (Sánchez-Meca et al., 2010; Soares et al., 2013). In addition, various randomized controlled trials (RCTs) and meta-analyses have documented its positive long-term effects (DiMauro et al., 2013; Durham et al., 2005; Soares et al., 2013). Although there are various CBT treatment programs for PD, the common denominator is identifying and changing maladaptive beliefs about physical symptoms and their consequences resulting from dysfunctional conditioning processes. Furthermore, conceptualizing avoidance behavior as the maintaining factor of PD is what all formats of CBT for PD have in common (Clark and Beck, 2010; Stein et al., 2010). Therefore, confronting feared stimuli is a fundamental part of CBT for PD (Bouton et al., 2001).

According to Kessler and Greenberg (2002), approximately two-thirds of individuals affected by PD remain untreated, reporting high treatment costs as one of the most common reasons for not entering psychological treatment. Also, long travel distances for people living in rural areas make the probability of entering a traditional face-to-face psychotherapy lower (Shapiro et al., 2003). As only about one third of individuals suffering from anxiety disorders receive treatment (Roberge et al., 2011), there is a great demand for an innovation in the treatment of this mental health disorder (Kazdin, 2015).

1.3. Internet-based CBT (iCBT) for panic disorder

The delivery of CBT interventions via the internet presents both an innovational and advantageous approach for treatment of various mental health disorders (Andersson and Titov, 2014). Benefits of iCBT include general availability, accessibility and flexibility in self-pacing, anonymity and appeal to individuals who prefer not to connect with the therapists, reduced travel time and costs for therapists and clients and reducing waiting times (Andersson and Titov, 2014; Cuijpers et al., 2008; Hedman et al., 2012).

As iCBT interventions utilize the same theoretical and practical principles as traditional face-to-face interventions, they provide the same therapeutic information and skills. Corresponding to the number of fCBT sessions, iCBT interventions programs generally incorporate 5–16 text modules. Therapist guidance has a mainly practical and supportive role, rather than explicitly therapeutic in orientation, depending on the degree of structure in the model of internet intervention adopted (Andersson and Titov, 2014).

1.3.1. Self-help iCBT

A typical online self-help program for panic disorder includes modules designed for psychoeducation, cognitive restructuring, various behavioral experiments and exposure, problem-solving techniques and lifestyle modification, motivational enhancement, repetition and relapse prevention (Barlow and Craske, 2000; Becker and Margraf, 2002; ⁎Carlbring et al., 2001; Stangier et al., 2003). In a self-help program, various additional features such as video clips, audio files or other interactive elements can be involved. Furthermore, participants can oftentimes engage in an online discussion forum concerning current mental health condition (e.g. ⁎Ciuca et al., 2018; ⁎Fogliati et al., 2016; ⁎Klein et al., 2006).

The new generation iCBT programs work well as self-guided format, since they are grounded on well-established protocols, evaluated over various clinical trials (⁎Fogliati et al., 2016; ⁎Ciuca et al., 2018). They typically involve measures involving screening assessment, monitoring, and engaging patients throughout treatment via automated messages. Considering this, acceptable and effective self-guided iCBT interventions have a great potential in treating various mental health conditions.

1.3.2. Guided self-help iCBT

When providing internet-based interventions, the key role of an online therapist is to offer guidance and feedback on homework assignments and queries to the clients who signed up for the treatment (Andersson et al., 2014). The amount of therapist support in existing internet-based programs varies widely from no support, through small amounts of contact via telephone and/or e-mail (e.g. ⁎Allen et al., 2016; Carlbring et al., 2006; ⁎Fogliati et al., 2016), to intensive involvement with levels similar to those seen in the face-to-face modalities (⁎Ciuca et al., 2018; ⁎Klein et al., 2009).

Several meta-analyses found that iCBT with incorporated therapist guidance generated larger effect sizes and higher completion rates than unguided programs in treating anxiety disorders (Andersson and Cuijpers, 2009; Cuijpers et al., 2009; Palmqvist et al., 2007; Spek et al., 2007). However, more recent research suggested that the impact of therapist guidance may not be as great as previously thought, especially for newer generation iCBT programs. There are numerous high-quality RCTs, which found Gsh and Sh interventions resulting in similar clinical outcomes for principal anxiety disorder (Baumeister et al., 2014; Berger et al., 2011a, Berger et al., 2011b; Dear et al., 2015; ⁎Fogliati et al., 2016; Titov et al., 2014; Titov et al., 2015). However, rather than disorder-specific approaches, these programs use a transdiagnostic approach, based on the premise that commonalities across disorders outweigh the differences (McEvoy et al., 2009). From this perspective, the two treatment aproaches differ in their designs and target groups.

Previous research confirms that iCBT for PD/A is overall an effective treatment, when compared to inactive waitlists (WL) and/or information control (IC) (Andrews et al., 2018; Hedman et al., 2012; Olthuis et al., 2016). Others found similar efficacy of Gsh iCBT compared to fCBT in reducing PD/A symptoms at post-test levels (Andersson et al., 2014; Carlbring et al., 2018; Hedman et al., 2012). Likewise, a meta-analysis from O'Kearney et al. (2019) confirmed non-inferiority (Δ = 0.26) of iCBT compared to fCBT in four RCTs, at post-test. Yet, one of the four studies (Haug et al., 2015) used a stepped-care approach, where iCBT was blended with fCBT in treating PD and social anxiety disorder (SAD). Effect size in this study was in favor of fCBT (g = 0.3). Finally, a recent narrative review from Apolinário-Hagen (2019) commented on the efficacy of guided, unguided, transdiagnostic and disorder-specific internet interventions (k = 8, n = 1013) for PD/A. The treatment groups included seven iCBT-based and one acceptance-based intervention, however, no meta-analytic estimates were made.

A recent meta-analysis from Stech et al. (2019) investigated the efficacy of Gsh iCBT in a large sample (k = 27, N = 2590) of efficacy and effectiveness trials, compared to active and inactive controls. When compared to active controls, results from three RCTs suggested similar outcomes in Gsh iCBT compared to fCBT in reducing panic symptoms (g = 0.14), but not agoraphobia symptoms (k = 2, g = 0.38). Compared to inactive controls, the authors found a large change in panic (g = 1.22) and agoraphobia (g = 0.91) symptoms in a data set of nine RCTs. When computing the within-group efficacy in a data set consisting of open, non-randomized and randomized trials (k = 14), the change in panic symptoms remained large (g = 0.98), however, with a high heterogeneity among effect sizes. The authors investigated the change in panic and agoraphobia symptoms in a mixed sample of clinical and subclinical panic disorder and/or agoraphobia sufferers, which could have contributed to the high heterogeneity. Moreover, changes in comorbid anxiety and depression symptoms and improvements in quality of life, at post-test and follow-up levels were not investigated.

1.4. Objectives

Our own meta-analysis will investigate the change in panic and agoraphobia symptoms in a sample of clinical panic disorder with or without agoraphobia sufferers. We will focus on the efficacy of disorder-specific Gsh iCBT for PD/A compared to both active and inactive controls, at post-test and follow-up levels. Moreover, this study will assess changes in secondary and tertiary symptoms, namely, comorbid anxiety and depression symptoms and improvements in quality of life, at post-test and follow-up levels, using between group comparisons in carefully selected 13 RCTs. We will investigate Gsh iCBT compared to various active CBT interventions in a sufficiently large data set (k ≥ 6), which previous meta-analyses omitted. Furthermore, clinical significance and treatment uptake in both treatment and control groups will be explored. Publication bias and moderator analysis will be investigated. Regarding moderator analysis, we will perform subgroup analyses of within-group pre- to post-treatment effect in the iCBT group focusing on assessment scale (PDSS or pooled effect sizes from all scales targeting PD/A symptoms); adherence to the treatment defined as initial uptake (low vs. high percentage of login to Gsh iCBT treatment programs); study context (country of original research), size of included samples (samples larger or smaller as N = 60) and baseline severity (higher or lower pre-test symptom severity on PDSS scale).

2. Methods

2.1. Identification of studies

In order to find eligible individual studies for our meta-analytical calculations, we adopted the recommendations from Lipsey and Wilson (2001) and proceeded as follows. First, we conducted a computer search through databases Medline, PsycINFO, PSYNDEX, PsycARTICLES, ProQuest, PubMed, Scopus, and Web of Science for articles published on our topic of interest. The search included the terms “internet-based cognitive-behavioral therapy” OR “web-based cognitive-behavioral therapy” OR “online cognitive-behavioral therapy” OR “guided self-help cognitive-behavioral therapy” OR “self-help cognitive-behavioral therapy” AND “cognitive-behavioral treatment” OR “face-to-face cognitive-behavioral therapy” AND “panic disorder” OR “panic disorder with or without agoraphobia” OR “agoraphobia”. Second, we searched through so-called “grey literature” and examined abstracts of conference contributions and posters, and screened reference lists of the found literature (snowball search) to further identify potentially relevant studies. Third, we engaged in personal and e-mail communication with experts in the field, to get access to their articles regarding this topic. The searches were repeated several times, up to March 2020.

2.2. Eligibility criteria

In the meta-analytical calculations, the included studies: a) were RCTs; b) aimed at patients who met the DSM-IV or DSM-5 criteria for PD/A; c) used clinician-administered screening interviews; d) had to include at least one disorder-specific iCBT (computer- or mobile-based) intervention designed for treating PD/A; e) investigated changes in symptoms in an adult population; and f) used valid and reliable measures for assessing levels of PD/A, comorbid anxiety and depression, and improvements in quality of life.

We also formulated additional exclusion criteria: a) non-randomized controlled trials, pilot studies, open trials and feasibility studies; b) trials using virtual reality interventions; c) transdiagnostic iCBT and “third-wave iCBT” (e.g., internet-delivered mindfulness or acceptance and commitment therapy) programs for PD/A, due to a different treatment approach used in these treatment programs; d) RCTs written in a language other than English; and e) unpublished studies or studies published in other than peer-reviewed journals.

2.3. Study selection

The first author (MP) initially screened all titles and abstracts of the studies to determine their relevance to this paper. Studies that could be immediately excluded based on the title and abstract were discarded. Both MP and NKT independently reviewed remaining studies (k = 41) for inclusion eligibility. Finally, 13 studies were included.

2.4. Quality assessment

Both the first and the second author independently assessed risk of bias in the included studies (k = 13). In agreement with the Cochrane tool for assessing risk of bias (Higgins et al., 2011), five default areas and one additional area (other risk of bias) of potential risk of bias were investigated. However, as guided internet-based interventions cannot be blinded from the clinicians' point of view, we did not include blinding of participants and personnel. Hence, the included studies were assessed for: a) random sequence generation, b) concealment of allocation; c) blinding of outcome assessors; d) incomplete outcome data, e) selective outcome reporting, and f) inclusion of a comparator. In the last category, we explored whether the selected RCTs included inactive comparators (WL, IC, care-as-usual, placebo), which is a common limitation across studies comparing iCBT to other active treatments. We interpreted all areas in terms of low, high, or unclear risk of bias (see Table 2). If the risk of bias was rated as high or unclear in more than three domains, a study was rated with an overall high risk. All differences were discussed and reconciled.

Table 2.

Risk of bias across the included studies.

Study	Selection bias Random sequence generation	Selection bias Allocation concealment	Reporting bias Selective reporting	Detection bias Blinding (outcome assessment)	Attrition bias Incomplete outcome data	Other bias Inclusion of a comparator
1. ⁎Allen et al. (2016)	Low	Low	Low	High	Low	Low
2. ⁎Bergström et al. (2010)	Low	Low	Unclear	Low	Low	High
3. ⁎Carlbring et al. (2001)	Unclear	Low	Unclear	Low	Low	Low
4. ⁎Carlbring et al. (2005)	Low	Unclear	Unclear	Low	Low	High
5. ⁎Carlbring et al. (2006)	Low	Unclear	Unclear	Low	Low	Low
6. ⁎Carlbring et al. (2003)	Low	Unclear	Low	High	Low	High
7. ⁎Ciuca et al. (2018)	Low	Low	Low	Low	Low	Low
8. ⁎Fogliati et al. (2016)	Low	Low	Low	High	Low	Low
9. ⁎Kiropoulos et al. (2008)	Low	Unclear	Unclear	Unclear	Low	Low
10. ⁎Klein et al. (2009)	Low	Low	Unclear	Low	Low	High
11. ⁎Klein et al. (2006)	Unclear	Low	Unclear	Unclear	Low	Low
12. ⁎Richards et al. (2006)	Unclear	Unclear	Unclear	Low	Low	Low
13. ⁎Wims et al. (2010)	Low	Low	High	High	Low	Low

Note: low = low risk of bias; high = high risk of bias; unclear = unclear risk of bias; N = no; Y = yes.

2.5. Data extraction

The following data was extracted from each study: authors and year of publication; primary diagnosis and percentage of agoraphobia symptoms; number of patients prior to randomization; treatment type and setting; type of patient contact; information on therapist-time per patient; number of modules and length of treatment; type of control condition; study design and statistical analysis; type of clinician screening; outcome scales assessing PD/A, comorbid anxiety and depression and quality of life; data regarding post-test and follow-up measures; initial treatment uptake and adherence; attrition; therapist experience and country (see Table 1). Information on sample sizes, means, standard deviations and standardized mean differences were transferred to Microsoft Excel spreadsheet and then to the Comprehensive Meta-Analysis software (Version 3.0; Biostat Inc.). (See Table 3, Table 4.)

Table 1.

Characteristics of the RCTs included in the meta-analytic calculations.

Study	Diagnosis	N	Treatment type	Treatment setting	Patient contact	Therapist time per patient	Modules (weeks)	Control condition	Analysis	Clinician screening	Scales	Outcome measure	Experience	Country
1. ⁎Allen et al. (2016)	PD	63	Guided self-help	Internet (Online program: Panic Program); Virtual clinic	E-mail + telephone	M = 6.04 min (SD = 10.66) vs. -	5 (8)	Inactive (WL)	ITT	MINI	1. PD/A: PDSS 2. A&D: K-10, PHQ-9	Pre, post, 3-month follow-up	Clinician	Australia
2. ⁎Bergström et al. (2010)	PD/A (A: 83%)	113	Guided self-help	Internet (Online program)	E-mail	M = 35.4 min (SD = 19) vs. M = 360 min (SD = -)	10 (10) vs. 10 (10)	Active (Face-to-face group CBT in psychiatric clinic)	ITT	PDSS	1. PD/A: PDSS 2. A&D: MADRS, ASI, SDS	Pre, post, 6-month follow-up	Psychologist	Sweden
3. ⁎Carlbring et al. (2001)	PD/A (A: not stated)	41	Guided self-help	Internet (Self-help guide)	E-mail	M = 90 min (SD = -) vs. -	6 (7–12)	Inactive (WL)	ITT	SCID	1. PD/A: ACQ, BSQ, MI 2. A&D: BAI, BDI, 3. QoL: QOLI	Pre, post	Therapists	Sweden
4. ⁎Carlbring et al. (2005)	PD/A (A: 51%)	49	Guided self-help	Internet (Online program)	E-mail	M = 150 min (SD = -) vs. M = 528 min (SD = -)	10 (10) vs. 10 (10)	Active (Face-to-face individual CBT in university setting)	ITT	SCID	1. PD/A: ACQ, BSQ, MI 2.A&D: BAI, BDI 3. QoL: QOLI	Pre, post, 12-month follow-up	Graduate psychology students, clinical psychologists	Sweden
5. ⁎Carlbring et al. (2006)	PD/A (A: not stated)	60	Guided self-help	Internet (Online program)	E-Mail + telephone	M = 234 min (SD = -) vs. -	10 (10)	Inactive (WL)	ITT	CIDI	1. PD/A: ACQ, BSQ, MI 2. A&D: BAI, BDI, MADRS 3. QoL: QOLI	Pre, post, 9-month follow-up	Psychologist, graduate psychology students	Sweden
6. ⁎Carlbring et al. (2003)	PD/A (A: 91%)	22	Guided self-help	Internet (Online program)	E-mail	M = 30 min (SD = -) vs. -	6 (−) vs. 9 (−)	Active (Applied relaxation)	ITT	SCID	1. PD/A: ACQ, BSQ, MI 2. A&D: BAI, BDI 3. QoL: QOLI	Pre, post	Therapists	Sweden
7. ⁎Ciuca et al. (2018)	PD/A (A: 52%)	73	Guided self-help	Internet (Online program: PAXPD); academic setting	Video call sessions	M = 247.2 min (SD = 129.6) vs. -	16 (12) vs. 16 (12)	Active (Self-help online program: PAXPD) orInactive (WL)	ITT	PDSQ	1. PD/A: ACQ, BSQ, BVS, PACQ, PDSS 2. A&D: PHQ-9	Pre, post, 6-month follow-up	Licensed psychotherapists	Romania
8. ⁎Fogliati et al. (2016)	PD	145	Guided self-help	Internet (Online program: Panic Course); eCentreClinic	E-mail + telephone	M = 36.79 min (SD = 21.35) vs. M = 0.55 min (SD = 1.88)	5 (8) vs. 5 (8)	Active (Self-help transdiagnostic online program: Wellbeing Course) orActive (Self-help online program: Panic Course)	ITT	MINI	1. PD/A: PDSS 2. A&D: PHQ-9, K-10, SDS	Pre, post, 24-month follow-up	Clinical psychologists, CBT-therapist	Australia
9. ⁎Kiropoulos et al. (2008)	PD/A (A: 59%)	86	Guided self-help	Internet (Online program: Panic Online); academic setting	E-mail	M = 352 min (SD = 240) vs. M = 568 min (SD = 255.12)	6 (12) vs. 12 (12)	Active (Face-to-face CBT in university setting)	ITT	ADIS	1. PD/A: ACQ, BVS, PDSS 2. A&D: ASP, DASS 3. QoL: WHO-QOLI	Pre, post	Psychologist, therapists	Australia
10. ⁎Klein et al. (2009)	PD/A (A: 74%)	57	Guided self-help	Internet (Online program: Panic Online)	E-mail	M = 205.28 min (SD = 120.01) vs. M = 308.3 min (SD = 222.67)	6 (8) vs. 6 (8)	Active (Guided self-help online program: Panic Online + frequent contact with therapist)	ITT	ADIS	1. PD/A: ACQ, BVS, PDSS 2. A&D: ASP, DASS 3. QoL: WHO-QOL-BREF	Pre, post	Therapists	Australia
11. ⁎Klein et al. (2006)	PD/A (A: 82%)	37	Guided self-help	Internet (Online program: Panic Online)	E-mail	M = 332.5 min (SD = 131.8) vs. M = 245.27 min (SD = 192.2)	6 (6) vs. -	Active (Guided self-help manualized CBT workbook) orInactive (IC)	ITT	PDSS	1. PD/A: ACQ, BVS, PDSS 2. A&D: ASP, DASS	Pre, post, 3-month follow-up	Clinical psychology graduate students, clinical psychologist	Australia
12. ⁎Richards et al. (2006)	PD/A (A: 78%)	32	Guided self-help	Internet (Online Program: Panic Online)	E-mail	M = 376.3 min (SD = 156.8) vs. M = 309.3 min (SD = 111.3)	6 (8) vs. 12 (8)	Active (Guided self-help + stress management Panic and Stress Online) orInactive (IC)	ITT	ADIS	1. PD/A: ACQ, BVS, PDSS 2. A&D: ASP, DASS 3. QoL: QOLI	Pre, post, 3-month follow-up	Clinical psychology doctoral students, clinical psychologist	Australia
13. ⁎Wims et al. (2010)	PD/A (A: not stated)	59	Guided self-help	Internet (Online program: Panic Online)	E-mail	M = 75 min (SD = -) vs. -	6 (8)	Inactive (WL)	ITT	MINI	1. PD/A: ACQ, BSQ, MI, PDSS 2. A&D: PHQ-9, SDS	Pre, post, 1-month follow-up	Clinician	Australia

Note: A = Agoraphobia; ACQ = The Agoraphobic Cognitions Questionnaire; A&D = Anxiety and Depression (comorbid); ADIS = The Anxiety Disorder Interview Schedule; ASI = The Anxiety Sensitivity Index; ASP = The Anxiety Sensitivity Profile; BAI = The Beck Anxiety Inventory; BDI = The Beck Depression Inventory; BSQ = The Body Sensations Questionnaire; BVS = The Body Vigilance Scale; CIDI = The Composite International Diagnostic Interview; DASS = The Depression, Anxiety, Stress Scale; IC = Information control; ITT = Intention-to-treat; K-10 = The Kessler 10-Item Psychological Distress Scale; MADRS = The Montgomery–Åsberg Depression Rating Scale; MI = The Mobility Inventory for agoraphobia; MINI = The Mini-International Psychiatric Interview; PD = Panic disorder; PD/A = Panic disorder with agoraphobia; PDSS = The Panic Disorder Severity Scale; PHQ-9 = The Patient Health Questionnaire-9 Item; PSWQ = The Penn State Worry Questionnaire; QoL = Quality of life; QOLI = The Quality of Life Inventory; SCID = The Structured Clinical Interview for the DSM; SDS = The Sheehan Disability Scale; WHO-QOL = The World Health Organization's Quality of Life; WHO-QOL-BREF = The World Health Organization Quality of Life Questionnaire-BREF, WL = waitlist control.

Table 3.

Clinical significance at post-test and follow-up in both treatment and control groups.

Study	Classification of responders	iCBT treated (n)	iCBT treated positive (post-test)	iCBT treated positive (follow-up)	Controls treated (n)	Controls treated positive (post-test)	Controls treated positive (follow-up)
1. ⁎Allen et al. (2016)	CC, RCI	27	75%	82%	37	29%	–
2. ⁎Bergström et al. (2010)	CSR, RFB	50	57%	70%	54	62%	62%
3. ⁎Carlbring et al. (2001)	RCI	21	81%	–	20	33%	–
4. ⁎Carlbring et al. (2005)	CSR, RCI	24	62%	92%	25	69%	88%
5. ⁎Carlbring et al. (2006)	CSR, RCI	30	71%	64%	30	4%	–
6. ⁎Carlbring et al. (2003)	CC, CSR	11	55%	–	11	37%	–
7. ⁎Ciuca et al. (2018)	CC, CSR	36	63%	–	37	31%	–
8. ⁎Fogliati et al. (2016)	RFB	68	37%	54%	64	44%	53%
9. ⁎Kiropoulos et al. (2008)	CC, CSR	46	33%	–	40	33%	–
10. ⁎Klein et al. (2009)	CC, CSR	28	34%	–	29	30%	–
11. ⁎Klein et al. (2006)	CC, CSR	19	58%	84%	18	44%	73%
12. ⁎Richards et al. (2006)	CC, CSR	12	47%	66%	11	64%	51%
13. ⁎Wims et al. (2010)	CC	29	70%	–	25	–	–

Note: CC = clinical cutoff; CSR = clinician severity rating; RCI = Reliable Change Index; RFB = reduction from baseline.

Table 4.

Uptake from treatment in iCBT and control group.

Study	Allocated iCBT (n)	Log-in iCBT (n, %)	Adherence iCBT (M, SD, %)	Attrition iCBT post (%)	Attrition iCBT follow-up (%)	Allocated controls (n)	Log-in controls (n, %)	Adherence controls (M, SD, %)	Attrition controls post (%)	Attrition controls follow-up (%)
1. ⁎Allen et al. (2016)	30	27 (90%)	–	43%	61%	37	36 (97%)	–	16%	–
2. ⁎Bergström et al. (2010)	53	50 (94%)	6.7 (2.5) of 10 (67%)	12%	14%	60	54 (90%)	8.1 (2.1) of 10 (81%)	9%	19%
3. ⁎Carlbring et al. (2001)	21	17 (81%)	–	19%	–	20	19 (95%)	–	5%	–
4. ⁎Carlbring et al. (2005)	24	21 (88%)	7.4 (2.2) of 10 (74%)	–	–	25	24 (96%)	9.0 (2.7) of 10 (90%)	–	–
5. ⁎Carlbring et al. (2006)	30	29 (97%)	8.9 (2.6) of 10 (89%)	7%	13%	30	30 (100%)	–	3%	–
6. ⁎Carlbring et al. (2003)	11	8 (72%)	- of - (56%)	27%	–	11	9 (82%)	- of - (57%)	18%	–
7. ⁎Ciuca et al. (2018)	36	33 (92%)	10.89 (5.08) of 16 (68%)	7%	36%	37	33 (89%)	6.89 (6.33) of 16 (43%)	32%	32%
8. ⁎Fogliati et al. (2016)	–	73 (−)	4.30 (1.16) of 5 (86%)	20%	23%	–	72 (−)	4.32 (1.24) of 5 (86%)	11%	24%
9. ⁎Kiropoulos et al. (2008)	46	41 (89%)	–	11%	–	40	38 (95%)	10.96 (−) of 12 (92%)	5%	–
10. ⁎Klein et al. (2009)	28	–	–	28%	–	29	–	–	21%	–
11. ⁎Klein et al. (2006)	19	18 (95%)	–	5%	–	18	15 (83%)	–	28%	–
12. ⁎Richards et al. (2006)	12	–	–	17%	–	11	–	–	9%	–
13. ⁎Wims et al. (2010)	32	29 (91%)	–	24%	28%	25	–	–	12%	–

2.6. Meta-analytical procedure

For the metric outcome measures, we calculated as effect size Hedges' g, which is a bias-adjusted estimate of the standardized mean difference particularly eligible for trials with small samples. Hedges' g represents the difference between means of a treatment intervention and comparison condition, divided by the pooled standard deviation (Hedges, 1981). Positive values of g (with the 95% confidence interval) indicate superiority of treatment condition over control condition. Effect sizes of 0.2, 0.5, and 0.8 were considered small, moderate, and large respectively (Cohen, 1988). For clinical significance, we used the risk difference as effect size (Borenstein et al., 2009).

To compute an effect size across studies, we used random effects model (REM) as it assumes that a treatment effect in each study is randomly selected from a normal distribution and that it varies from study to study (Borenstein et al., 2009). Each statistical analysis included a mean effect size with 95% confidence interval and a heterogeneity analysis, which assessed the degree of dispersion of the effect sizes around the mean effect (Hedges and Olkin, 1985; Rothstein et al. (2005). We examined the heterogeneity using the Q-statistic. Here, we considered the proposition from Borenstein et al., 2009 and set the level of significance to p < .05, indicating presence of heterogeneity. Furthermore, the I²-index was used in estimating of the observed variance proportion that reflects true differences in effect sizes between the studies. We interpreted the heterogeneity values of 25%, 50%, and 75% as low, moderate, and high, respectively (Crombie and Davies, 2009). In case a moderate to high heterogeneity complicated interpretation of mean effect sizes, a moderator analysis was performed (Borenstein et al., 2009; Crombie and Davies, 2009).

2.7. Outcome measures

We computed effect sizes for different symptoms both at post-test and follow-up: a) panic with or without agoraphobia; b) comorbid anxiety and depression; c) improvements in quality of life, and d) clinical significance. If more outcome scales were used within a symptom level, effect sizes were pooled.

2.7.1. Primary outcome assessment

In order to assess the panic symptoms and their severity, various well-validated screening interviews were used. The interviews were based upon DSM criteria, ranging from DSM-IV to DSM-5 and administered by clinicians, mostly via telephone. These were Structured Clinical Interview (SCID; First et al., 1995), the Anxiety Disorder Interview Schedule (ADIS; Brown et al., 1994), the Composite International Diagnostic Interview (CIDI; World Health Organization, 1997a) and the Mini International Neuropsychiatric Interview (MINI; Sheehan et al., 1998). Several disorder-specific scales were used. The Panic Disorder Severity Scale (PDSS; Shear et al., 2001) is a well-validated scale used for assessing panic disorder symptoms. The scale is known for its good internal consistency and test–retest reliability. Also, it is sensitive to change with treatment. The PDSS-SR was adapted to be used in a patient self-report format. The Body Sensations Questionnaire (BSQ; Chambles et al., 1984), the Panic Attack Cognition Questionnaire (PACQ; Clum et al., 1990) and the Agoraphobic Cognitions Questionnaire (ACQ; Chambles et al., 1984) were used to measure fear of bodily sensations and maladaptive cognitions associated with panic and agoraphobia. When examining agoraphobic avoidance, the Mobility Inventory (MI; Chambless et al., 1985) was administered. The interview consists of two subscales: MIA (avoidance alone) and MIB (avoidance accompanied). The Body Vigilance Scale (BVS; Schmidt et al., 1997) is a self-report scale designed to investigate attentional focus to bodily sensations.

2.7.2. Secondary outcome assessment

To measure depressive symptoms and general anxiety, the Beck Depression Inventory (BDI; Beck et al., 1961), the Beck Anxiety Inventory (BAI; Beck et al., 1988), the Montgomery-Åsberg Depression Rating Scale-Self-Rated (MADRS-S; Svanborg & Åsberg, 1994), the Anxiety sensitivity profile (ASP; Taylor and Cox, 1998), the Anxiety Sensitivity Index (ASI; Reiss et al., 1986), the Depression, Anxiety, Stress Scales (DASS; Lovibond & Lovibond, 1995), the Kessler 10-Item Psychological Distress Scale (K-10; Kessler et al., 2003), the Patient Health Questionnaire-9 Item (PHQ-9; Kroenke et al., 2001), the Penn State Worry Questionnaire (PSWQ; Meyer et al., 1990) and the Sheehan Disability Scale (SDS; Sheehan, 2000) were used.

2.7.3. Tertiary outcome assessment

To investigate quality of life of affected individuals, the Quality of Life Inventory (QOLI; Frisch et al., 1992), the World Health Organization Quality of Life scale (WHOQOL; World Health Organization, 1997b), the World Health Organization Quality of Life Questionnaire-BREF (WHOQOL-BREF; WHOQOL Group, 1998) were used.

2.7.4. Clinical significance assessment

We investigated the clinical significance of both treatment and control groups, in data set where iCBT was compared to active controls. In the included trials, there were various measures employed in defining diagnostic status at post-test and follow-up. According to Furukawa et al. (2009) and Shear et al. (2001), clinical levels of PD were defined by cut-off scores ≥8 in the Panic Disorder Severity Scale (PDSS, Shear et al., 1997). Hence, scores below 8 were considered not clinical. Also, patients were regarded as responders when a 40% reduction from baseline to post-test in the PDSS was observed (Barlow et al., 2000; Milrod et al., 2007). In addition, PD sufferers were defined as panic free by a PD clinician severity rating of ≤2 (Craske et al., 1991). Also, the Reliable Change Index (Jacobson and Truax, 1991) was used across studies. In cases where different methods were used when classifying patients as responders, we extracted data from all available measures on panic-free status and pooled them, as suggested in the literature (Brown and Barlow, 1995; ⁎Klein et al., 2006; ⁎Klein et al., 2009).

2.8. Additional analyses

2.8.1. Treatment uptake

We analyzed treatment uptake on various levels.

2.8.1.1. Initial phase

Defined as the number or percentage of patients who did not log into the intervention program after allocation in a condition.

2.8.1.2. Adherence

There are differences across studies in definition of adherence. Focus is either on completed lessons, or on number of individuals, who completed the treatment. These are two different things. Particularly the definition of adherence as number of completed modules can lead to heterogeneity, as there are iCBT programs with different number of treatment modules (e.g. Stech et al., 2019). However, when defined as percentage of completed modules per treatment intervention, a comparison across treatments is possible.

2.8.1.3. Attrition

Defined as percentage of patients who did not fill-in questionnaires at post-test and/or follow-up levels.

2.8.2. Therapist-time

We extracted data on averaged therapist-time per patient in both treatment and control group in the data sets where iCBT was compared to active controls. We searched for mean (with SD) therapist-time per patient. More information regarding therapist-time can be found in Table 1.

2.8.3. Publication bias

As detection of publication bias is based on a homogeneity assumption and is unreliable if less than six studies are analyzed, a publication bias analysis was carried out only in data sets fulfilling the requirements (Sterne et al., 2005). When addressing publication bias, we used Orwin's fail-safe N (Orwin, 1983) analysis.

2.8.4. Moderator analysis

To explore possible moderators of treatment effect in the iCBT group, subgroup analyses were used in categories such as a) type of scale (PDSS or pooled effect sizes from scales targeting PD/A symptoms); b) sample size (N > or < 60); c) country of original research (either Australia, or Sweden or rest of the world), d) adherence to the treatment defined as initial uptake (low or high percentage of login to the treatment program), and e) baseline severity (score higher or lower as 15 points on PDSS scale).

3. Results

3.1. Literature search and study selection

The literature search yielded 838 records. Of these, 336 records were considered not relevant and were therefore excluded. The first author reviewed titles and abstracts of the remaining 502 records. Of these, 461 records were excluded, mostly because of a not suitable design, such as PD/A being a secondary diagnosis, adolescent samples, transdiagnostic iCBT interventions or mixed interventions, data replication, open and/or non-randomized trials. This led to 41 studies that were reviewed in detail by both the first and second author. Subsequently, 28 studies were excluded for following reasons: 11 trials included no control group; five were not randomized; in two studies the sample was mixed of adults and adolescents; two used not eligible intervention such as iCBT mixed with face-to-face contact; seven listed PD/A as not primary diagnosis and one was a replication study using no original data. Finally, 13 RCTs that met our eligibility criteria were included in the meta-analytical calculations. These 13 RCTs included 19 comparisons. All RCTs are described in Table 1 and marked with an asterisk in the reference list. See Fig. 1 for a visual overview of the selection process according to “Preferred Reporting Items for Systematic Reviews and Meta-Analyses” (PRISMA) checklist, which is a consensus statement on meta-analytic reporting (Moher et al., 2009).

Fig. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Diagram of selected studies.

3.2. Study characteristics

In the 13 RCTs (N = 1214), we investigated a total of 19 comparisons that were eligible for the final analysis. We included comparisons where Gsh iCBT was compared to a) fCBT (⁎Bergström et al., 2010; ⁎Carlbring et al., 2005; ⁎Kiropoulos et al., 2008); b) applied relaxation (⁎Carlbring et al., 2003); c) Gsh iCBT with frequent contact to an online therapist (⁎Klein et al., 2009), Gsh as a manualized workbook (⁎Klein et al., 2006) and Gsh + stress management course (⁎Richards et al., 2006); d) self-help iCBT (⁎Ciuca et al., 2018; ⁎Fogliati et al., 2016; Andersson et al., 2009) and transdiagnostic self-help iCBT (⁎Fogliati et al., 2016), e) WL (⁎Allen et al., 2016; ⁎Carlbring et al., 2001; ⁎Carlbring et al., 2006; ⁎Ciuca et al., 2018; Andersson et al., 2009; ⁎Wims et al., 2010); f) IC (⁎Klein et al., 2006; Andersson et al., 2009; ⁎Klein et al., 2006; Andersson et al., 2014; ⁎Richards et al., 2006; Andersson et al., 2009); and g) self-help iCBT compared to WL (⁎Ciuca et al., 2018; Andersson et al., 2014). Regarding the study from ⁎Allen et al. (2016), we used only the first from two studies, as the second study was an open trial. In the trial from ⁎Fogliati et al. (2016), we used two comparisons, Gsh disorder-specific vs. Sh transdiagnostic and disorder-specific Gsh vs. Sh.

In the iCBT treatment group, 18 comparisons included disorder-specific Gsh iCBT programs (Panic Program, Panic Online, Panic Course, PAXPD) and one comparison included self-help disorder-specific iCBT program (PAXPD unguided). All iCBT programs included modules on psychoeducation and cognitive restructuring, controlled breathing or progressive muscle relaxation, gradual exposure, and relapse prevention. Therapist guidance varied from e-mail and/or telephone contact (scheduled or frequent) to video contact. One Gsh iCBT program included additional stress management modules. Therapists experience varied from psychology master students to licensed psychotherapists or clinical psychologists. More information regarding trials characteristics are provided in Table 1.

3.3. Risk of bias

From the 13 included RCTs, 10 trials reported adequate generation of random sequence; 8 RCTs reported low risk at concealment of allocation; 4 out of 13 trials were rated with low risk of selective reporting of results; 7 studies were rated as low risk for blinding of an outcome assessment; 13 RCTs were coded as low risk for handling of incomplete data; 9 RCTs included an inactive comparator, which was investigated among other risk for bias. From a global perspective (unclear or high risk of bias in more than three domains), no study was rated with a high overall risk of bias.

3.4. Gsh iCBT vs. active controls

3.4.1. Panic and agoraphobia symptoms at post-test and follow-up levels

3.4.1.1. Post-test

In nine RCTs (10 comparisons, N = 744), Gsh iCBT was compared to active controls at post-test [studies 2, 4, 6–12]. No statistically significant difference between groups (g = 0.015, SE = 0.073, 95% CI: −0.128, 0.157, p = .838) and no significant heterogeneity between effect sizes (Q = 7.968, I² = 0.00%, p = .537) were found. The meta-analysis can be seen in Fig. 2.

Fig. 2.

Meta-analysis of Gsh iCBT compared to active controls in panic and/or agoraphobia symptoms at post-test.

In terms of clinical significance, nine RCTs providing data on end-state functioning were analyzed [studies 2, 4, 6–12]. On average 49.56% (SD = 11.92) of affected individuals in the Gsh iCBT group and 46% (SD = 16.23%) in active treatments responded positively to the treatment and showed no clinically significant PD/A symptoms at post-test levels. The risk difference (0.030) between both groups was not statistically significant (p = .543).

3.4.1.2. Follow-up

The sample consisted of five RCTs (six comparisons) and included 512 participants. No statistically significant difference between Gsh iCBT and other active controls was found (g = −0.113, SE = 0.088, 95% CI: −0.285, 0.059, p = .200). The heterogeneity among effect sizes was statistically not significant (Q = 2.024, I² = 0.00%, p = .846).

In terms of clinical significance, five RCTs were analyzed [studies 2, 4, 8, 11, 12]. Here, 73.20% (SD = 15.01) in the Gsh iCBT group and 65.4% (SD = 15.34) in the control group were panic free at follow-up. The risk difference (0.056) between both groups was not statistically significant (p = .231).

3.4.2. Comorbid anxiety and depression at post-test and follow-up levels

3.4.2.1. Post-test

When compared exclusively to active controls in nine RCTs (10 comparisons, N = 744), Gsh iCBT was equally effective at post-test in reducing comorbid anxiety and depression (g = −0.026, SE = 0.073, 95% CI: −0.168, 0.117, p = .726). Heterogeneity was statistically not significant (Q = 8.319, I² = 0.00%, p = .502). The meta-analysis can be seen in Fig. 3.

Fig. 3.

Meta-analysis of Gsh iCBT compared to active controls in comorbid anxiety & depression symptoms at post-test.

3.4.2.2. Follow-up

In five RCTs (six comparisons) (N = 512), no statistically significant difference between Gsh iCBT and active controls was found (g = 0.004, SE = 0.156, 95% CI: −0.303, 0.310, p = .981). A statistically significant heterogeneity between effect sizes was found (Q = 13.661, I² = 63.40%, p = .018), but was no longer significant (Q = 1.268, I² = 0.00%, p = .867) after removing one comparison arm. Publication bias was investigated in Fig. 6.

Fig. 6.

Funnel plot of Gsh iCBT vs. active controls in comorbid anxiety and depression symptoms at follow-up.

3.4.3. Improvements in quality of life at post-test and follow-up levels

3.4.3.1. Post-test

This analysis included five RCTs (N = 231) with Gsh iCBT compared to active controls. Here, no significant difference between treatments (g = −0.100, SE = 0.130, 95% CI: −0.355, 0.154, p = .439) and no significant heterogeneity between effect sizes (Q = 3.457, I² = 0.00%, p = .484) were found. The meta-analysis can be seen in Fig. 4.

Fig. 4.

Meta-analysis of Gsh iCBT compared to active controls in improvements of quality of life at post-test.

3.4.3.2. Follow-up

The sample consisted of two RCTs comparing Gsh iCBT with active treatments and included 72 participants. No statistically significant difference was found between the effect sizes (g = 0.074, SE = 0.231, 95% CI: −0.378, 0.526, p = .749) and heterogeneity (Q = 0.209, I² = 0.00%, p = .647).

3.5. Gsh iCBT vs. inactive controls

3.5.1. Panic and agoraphobia symptoms at post-test levels

Gsh iCBT and Sh iCBT was compared to inactive controls, in seven RCTs (9 comparisons, N = 470). A statistically significant difference between iCBT and inactive controls was found (g = −0.892, SE = 0.165, 95% CI: −1.215, −0.568, p = .000), in favor of iCBT. The heterogeneity among effect sizes was statistically significant (Q = 22.445, I² = 64.35%, p = .004) but was no longer significant after removing one study with small effect size in favor for iCBT (Q = 11.288, I² = 37.98%, p = .127).

3.5.2. Comorbid anxiety and depression at post-test levels

Compared to inactive controls in seven RCTs (9 comparisons, N = 470), Gsh iCBT showed significantly larger effect sizes (g = −0.723, SE = 0.112, 95% CI: −0.943, 0.503, p = .000) with no significant heterogeneity (Q = 10.977, I² = 27.12%, p = .203).

3.5.3. Improvements in quality of life at post-test levels

When compared to inactive controls in three RCTs (N = 122), Gsh iCBT was significantly better in improving quality of life (g = −0.484, SE = 0.181, 95% CI: −0.838, −0.129, p = .008), with no heterogeneity among effect sizes (Q = 1.156, I² = 0.00%, p = .561).

3.6. Gsh iCBT vs. Sh iCBT

3.6.1. Panic and agoraphobia symptoms at post-test levels

In the analysis of two RCTs with three comparisons (N = 363), no significant difference between both treatment formats was found at post-test levels in reducing PD/A symptoms (g = −0.025, SE = 0.168, 95% CI: −0.353, 0.304, p = .883). No significant heterogeneity between effect sizes was found (Q = 4.894, I² = 59.13%, p = .087). The meta-analysis can be seen in Fig. 5.

Fig. 5.

Meta-analysis of Gsh iCBT compared to Sh iCBT in panic and/or agophobia symptoms at post-test.

In terms of clinical significance, two RCTs were analyzed [studies 7, 8]. Here, 63% in the Gsh iCBT group and 37% in the Sh iCBT were panic free at post-test. The risk difference (0.075) between both groups was not statistically significant (p = .265).

3.6.2. Comorbid anxiety and depression at post-test levels

When compared to Sh iCBT, Gsh iCBT showed no significantly different effect size in changes of comorbid anxiety and depression symptoms (g = −0.025, SE = 0.104, 95% CI: −0.227, 0.182, p = .832). No significant heterogeneity between effect sizes was found (Q = 0.802, I² = 0.00%, p = .669).

3.7. Additional analyses

3.7.1. Uptake from treatment

3.7.1.1. Login

In nine RCTs [studies 1–7, 9, 11], on average 88.9% (SD = 7.02) of the participants treated by the Gsh iCBT logged into the treatment program. Similarly, 93% (SD = 4.81) of the participants allocated to active control group began the treatment. The risk difference (−0.024) between both groups was not statistically significant (p = .276).

3.7.1.2. Adherence

When investigating a dataset of five RCTs [studies 2, 4, 6–8], in the Gsh iCBT group 70.2% (SD = 10.96) of the treatment modules were completed. In the active control group, 71.4% (SD = 20.40) of treatment modules were completed. The risk difference (−0.019) between both groups was not statistically significant (p = .728).

3.7.1.3. Adherence in Gsh iCBT vs. Sh iCBT

Gsh iCBT was compared to Sh iCBT in two trials [studies 7–8], with no statistically significant risk difference (0.107, p = .388) between both treatment approaches.

3.7.1.4. Attrition

Comparing Gsh iCBT to active and inactive controls in a mixed sample of 13 RCTs [studies 1–3, 5–13] the mean attrition rate in the iCBT group at post-test was 18.31% (SD = 10.58). In the control group, attrition at post-test was 14.08% (SD = 8.83). The risk difference (0.040) between both groups was not statistically significant (p = .270). When comparing Gsh iCBT to active controls in eight RCTs [studies 2, 6–12], the risk difference was not significant (−0.005, p = .909).At follow-up, we compared iCBT to active controls only and found that 24.33% (SD = 11.06) in the Gsh iCBT and 25% (SD = 6.56) in the control group did not fill in the questionnaires, in three RCTs [studies 2, 7–8]. The risk difference (−0.032) between both groups was not statistically significant (p = .720).

3.7.1.5. Attrition in Gsh iCBT vs. Sh iCBT

When Gsh iCBT was compared to Sh iCBT in two trials [studies 7, 8], no statistically significant risk difference was found between the two formats at post-test (−0.074, p = .664).

3.7.2. Therapist-time

In the data set [studies 2, 4, 8–12] consisting of Gsh iCBT compared to active controls, the Gsh iCBT included 212.57 min (SD = 145.43) averaged therapist-time per patient. The control conditions consisting of various active psychological treatments with CBT elements accumulated 331.35 min (SD = 188.57) therapist-time. Using a paired t-test, the difference in therapist-time between the two treatment formats was statistically not significant (t = 1.32, p = .23).

3.7.3. Publication bias

As there was a significant heterogeneity in the data set of Gsh iCBT compared to active controls when investigating changes in the comorbid anxiety and depression at follow-up levels, we investigated the data set for a publication bias here. We set the criterion for a trivial Hedges's g to −0.4 and found that 28 missing studies with a mean Hedges's g of −0.5 would have been needed to bring our g over −0.4. Fig. 6 presents a funnel plot relating effect sizes of the comorbid anxiety and depression outcomes to the standard errors of the estimates, at follow-up levels. Furthermore, publication bias was investigated in the sample of Gsh iCBT and Sh iCBT compared to inactive controls in seven RCTs with nine comparisons (N = 470). Using Orwin's fail-safe N, we set the criterion for a trivial Hedges's g to −0.4 and found that 40 missing studies with a mean Hedges's g of −0.3 would have been needed to bring our g over −0.4.

3.8. Moderator analysis

In order to explore possible moderators of treatment effect in the iCBT group, subgroup analyses were used in categories such as a) type of scale (PDSS or pooled effect sizes from scales targeting PD/A symptoms); b) sample size (N > or < 60); c) country of original research (either Australia, or Sweden or rest of the world), d) adherence to the treatment defined as initial uptake (low or high percentage of login to the treatment program), and e) baseline symptom severity (higher or lower pre-test symptom severity on PDSS scale). Three variables were found significant. Regarding country of the original research, research conducted in Australia (k = 8) showed an overall large effect size of g = 0.998. However, effect sizes from trials conducted in Sweden (k = 5) were significantly larger (g = 1.382). The largest effect within-group effect size was found in the Gsh iCBT group in a study from Romania (⁎Ciuca et al., 2018). Here, Hedges's g was 1.909. Regarding initial or login to treatment programs, there was a statistically significant difference between within-group effect sizes in the iCBT group. In the trials, where fewer than 80% of allocated individuals logged into the treatment program, the effect size was g = 0.877. In case more than 80% logged into the Gsh iCBT treatment programs, the within-group effect size was significantly larger (g = 1.494). Additionally, we found significantly larger within-group effect sizes in iCBT trials with higher baseline symptom severity (k = 7, g = 1.578), as it was in those trials with lower baseline severity (k = 6, g = 1.022) on PDSS scale. The results are shown in Table 5.

Table 5.

Categorical moderator analysis.

Panic and agoraphobia
Subgroup analysis		k	g	95% CI	p
Scale	PDSS	5	−0.129	−0.442, 0.185	0.23
	Pooled	5	−0.080	−0.393, 0.233
Sample	N < 60	7	1.137	0.758, 1.515	0.56
	N > 60	7	1.288	0.948, 1.629
Country	Australia	8	0.998	0.757, 1.240	0.01
	Sweden	5	1.382	0.882, 1.881
	Rest	1	1.909	1.178, 2.640
Initial uptake	Low login (<80%)	6	0.877	0.633, 1.121	0.00
	High login (>80%)	6	1.494	1.196, 1.792
Baseline	Lower severity	6	1.022	0.703, 1.341	0.00
	Higher severity	7	1.578	1.231, 1.926

4. Discussion

4.1. Key findings

As summarized in Table 6, the current meta-analysis investigated disorder-specific iCBT for PD/A compared to various active control condition comprised of some form of CBT, using between-group comparisons in carefully selected 13 RCTs (19 comparisons). In all comparisons, Gsh iCBT served as primary treatment interventions. All active controls were comprised of some form of CBT and included significantly more therapist-time as the Gsh iCBT group (166.96 vs. 331.35 min). Gsh iCBT was found equally effective in reducing clinical levels of PD/A, when compared to active controls (post-test: k = 10, g = 0.015; follow-up: k = 6, g = −0.113) and highly effective, when compared to inactive controls (post-test: k = 9, g = −0.892). Furthermore, this study found disorder-specific Gsh iCBT to be equally effective as other active CBT treatments reducing comorbid anxiety and depression (post-test: k = 10, g = −0.026; follow-up: k = 6, g = 0.004) and moderately to highly effective when compared to inactive controls (post-test: k = 9, g = −0.723). In improving quality of life of affected individuals, iCBT was found equally effective as active controls (post-test: k = 5, g = −0.100; follow-up: k = 2, g = 0.074) and moderately effective when compared to inactive controls (post-test: k = 3, g = −0.484). Additional to PD/A symptoms, this study establishes Gsh iCBT as an overall effective treatment for comorbid anxiety and depression symptoms when compared to inactive controls, and equally effective when compared to active controls with CBT elements.

Table 6.

Summary of study findings.

		Panic disorder/ agoraphobia	Anxiety & depression	Quality of life
1. Gsh iCBT vs. active controls	Post-test	g = 0.015, k = 10, N = 744	g = 0.004, k = 6, N = 512	g = −0.100, k = 5, N = 231
	Follow-up	g = 0.113, k = 6, N = 512	g = 0.004, k = 6, N = 512	g = 0.074, k = 2, N = 72
2. Gsh iCBT vs. inactive controls	Post-test	g = −0.892, k = 9, N = 470	g = −0.723, k = 9, N = 470	g = −0.484, k = 3, N = 122
	Follow-up	–	–	–
3. Gsh iCBT vs. Sh iCBT	Post-test	g = −0.025, k = 3, N = 363	g = −0.025, k = 3, N = 363	–
	Follow-up	–	–	–

The results from our study appear in contrast to the previous meta-analyses from Stech et al. (2019), Andrews et al. (2018), Hedman et al. (2012), and Olthuis et al. (2016), when iCBT was compared to inactive controls. In all previous meta-analyses, the mean between-group differences for PD/A severity were larger. Whereas Stech et al. (2019) found large effect sizes for PD/A symptoms (g = 1.22; g = 0.91), the other meta-analyses found even larger effects. For instance, Hedges's g in Andrews et al. (2018) was 1.31, in Hedman et al. (2012) it was 1.42 and the study from Olthuis et al. (2016) found the effect size to be 1.52. We have two explanation for this. First, all previous meta-analyses computed effect sizes for PD using one scale (primary PDSS, but also BSQ), whereas in our meta-analysis, we pooled effect sizes from all scales assessing primary PD/A symptoms (PDSS, BSQ, PACQ, ACQ, MI, BVS). For instance, the effect sizes in changes in PD and agoraphobia symptoms in Stech et al. (2019) were computed separately, showing changes in panic disorders symptoms to be larger than in agoraphobia. Second, the previous meta-analyses investigated changes in symptoms in mixed datasets consisting of open trials, non-randomized and randomized trials. In our meta-analysis, we only included RCTs with patients with clinical levels of PD/A. As the study from Stech et al. (2019) showed, after excluding participants with subsyndromal PD/A at baseline, the effect size was reduced. With our meta-analysis, this was the case as well.

In contrast to the previous meta-analyses on this topic, one of the major differences of this study was the investigation of Gsh iCBT compared to various active interventions with CBT elements, in a larger data set of included trials. In our meta-analysis, we compared Gsh iCBT to active CBT interventions (fCBT, applied relaxation, Gsh iCBT + stress management, Gsh iCBT with frequent contact to therapist, disorder-specific and transdiagnostic self-help) in nine RCTs (10 comparisons, N = 744). We found Gsh iCBT to be equally effective in reducing clinical levels of PD/A, with no significant heterogeneity between effect sizes at post-test (k = 10, g = 0.015) and follow-up levels (k = 6, g = −0.113). These findings confirm the findings from previous meta-analyses on Gsh iCBT compared to another active CBT interventions, although to our knowledge, there are only four meta-analytic comparisons of Gsh iCBT and fCBT (Carlbring et al., 2018; Hedman et al., 2012; O'Kearney et al., 2019; Stech et al., 2019) published, using repeatedly the same three RCTs (with the exception of the RCT from Haug et al., 2015). Our meta-analysis suggests equal efficacy of Gsh iCBT compared to active psychological CBT treatments, in a larger data set of RCTs (k ≥ 6).

Another difference to previous studies covering this topic is the focus on comorbidity and improving of quality of life in panic individuals. When compared exclusively to active controls in nine RCTs with 10 comparisons (N = 744), Gsh iCBT was equally effective in reducing comorbid anxiety and depression (g = −0.026) at post-test, with no heterogeneity. At follow-up levels, (k = 6, N = 512), no statistically significant difference between Gsh iCBT and active controls was found (g = 0.004). Compared to inactive controls in seven RCTs (nine comparisons) (N = 470), Gsh iCBT showed significantly larger effect sizes (g = −0.723), with no significant heterogeneity. Therefore, we conclude that Gsh iCBT is an overall effective treatment in reducing comorbid anxiety and depression in panic individuals, at both post-test and follow-up levels. This confirms the observation that disorder-specific treatments also have substantial transdiagnostic treatment effects (⁎Fogliati et al., 2016).

Moreover, we investigated improvements in quality of life of individuals with clinical levels of PD/A. As such, there are no other meta-analyses covering this issue. When compared to active controls only in five RCTs (N = 231), Gsh iCBT was equally effective (g = −0.100), with no heterogeneity between effect sizes. At follow-up, only two RCTs (N = 72) comparing Gsh iCBT against fCBT and Gsh iCBT with stress management modules were included. No statistically significant difference in effect sizes (g = 0.074) and no heterogeneity was found. When compared to inactive controls in three RCTs (n = 122), Gsh iCBT was significantly larger in improving quality of life (g = −0.484) with no heterogeneity among effect sizes. As this data set was not large enough (k ≥ 6) for reliable and valid generalization, the findings must be interpreted with caution.

Another contribution of this study is that the direct comparison of Gsh iCBT with Sh iCBT. Although this was possible only in two RCTs (⁎Ciuca et al., 2018; ⁎Fogliati et al., 2016; Andersson et al., 2009) within three comparisons (N = 363), we found no significant difference between both interventions in reducing PD/A symptoms (g = −0.025) and comorbid anxiety and depression symptoms (g = −0.025) at post-test levels. Follow-up efficacy was not investigated due to insufficient comparisons (k ≤ 2). Heterogeneity between effect sizes was not significant. Although newer trials have shown good outcomes can be obtained with very little clinician contact (⁎Fogliati et al., 2016; ⁎Klein et al., 2009), there are still very few studies that directly compare disorder-specific self-help and Gsh iCBT in treating PD and comorbid symptoms. Therefore, these findings must be interpreted with caution.

Alongside with determining of clinical significance of an intervention, deterioration or adverse effects of an intervention can be investigated (Rozental et al., 2014). From this perspective, deterioration of an intervention can be understood as a logical counterpart for responders. Evidence from face-to-face treatments suggests that negative effects occur in 5–10% of all patients undergoing treatment (Rozental et al., 2014). When investigating deterioration of iCBT for anxiety disorders, depression, erectile dysfunction, relationship problems, and gambling disorder, in a large meta-analysis (k = 29), Rozental et al. (2019) found that 26.8% of treated individuals were non-responders. The authors of the study found that higher symptom severity on the primary outcome measure at baseline, anxiety disorder as primary disorder and a male gender were predictors for not responding to treatment. In our study, it was not possible to calculate the deterioration rates, due to missing individual patient data on adverse effects. However, we investigated clinical significance or responder rates and our results are comparable with the findings from a large systematic review of CBT for anxiety disorders from Loerinc et al. (2015). The study found that the mean response rate to CBT treatment was 49.5%. In our study, approximately 50% of the patients were regarded as responders at post-test, and 73% at follow-up, considering the iCBT group. Also, we found no significant difference between iCBT and control treatments with CBT elements.

In our study, we investigated the uptake from treatment on various levels. More specifically, we investigated the initial phase between allocation to a treatment group and login into the treatment program. Then we investigated the adherence to the treatment program, defined as percentage of completed modules, in both treatment and control group. Lastly, we examined the attrition rates among participants in treatment and control group. In terms of treatment uptake, we focused on the phase between allocation to a group (randomization) and login into the treatment program. The reason for that was that frequently, the adherence after commencing of treatment is well documented, however, the timeframe before login is not. In nine RCTs, on average 88.9% (SD = 7.02) of the participants allocated to iCBT logged into the treatment program. Similarly, 93% (SD = 4.81) of the participants allocated to active control group began the treatment. There was no statistically significant risk difference found between the two treatment formats. There are various possible explanations for dropouts before login, with the most frequent explanation being lack of time and start of medication (k = 6). Other reasons were busy with other commitments, lack of motivation, computer problems, lost contact to therapists, reported spontaneous remission and/or starting another treatment. In the literature, information regarding treatment uptake of internet interventions are still scarce. A recent study from Lin et al. (2018) investigated acceptance, uptake, and adherence of internet- and mobile-based interventions for individuals with chronic pain. This study found lower uptake rates, with 67% (38/57) participants from the intervention group and 62% (36/58) from the control group, who had logged into the intervention program.

We investigated adherence to the treatment programs, which we defined as percentage of completed modules of intervention programs. Several trials (k = 6) reported the mean or percentage of completed treatment modules. As the number of modules across intervention programs varied between 5 and 16 (iCBT: M = 10.2, SD = 3.9; Active controls: M = 10.6, SD = 4), we computed the overall rates of completed modules in each treatment and compared both treatment groups using a risk difference analysis. No significant differences were found in both iCBT and active controls, as well as between Gsh iCBT and Sh iCBT. Here, a possible explanation is that automated emails in self-help iCBT programs not only improve treatment outcomes, but also increase rates of course completion (Titov et al., 2013).

Attrition was defined as percentage of patients who did not fill-in questionnaires at post-test and/or follow-up. We calculated the attrition rates in RCTs (k = 13) displaying the attrition data for both iCBT and active control conditions. On average 18.31% (SD = 10.58) of the participants treated by the iCBT did not fill in the questionnaires at post-test. Similarly, 14.08% (SD = 8.83) of the participants allocated to other active control conditions. We found no statistically significant difference between Gsh iCBt and active controls with CBT elements. As relatively high attrition rates are commonly reported in iCBT trials for anxiety disorders (Peñate and Fumero, 2016), our findings are in line with previous meta-analyses on PD/A symptoms (Apolinário-Hagen, 2019). Here, attrition rates between 9.8 and 42.1% were found. When investigating attrition between Gsh iCBT and Sh iCBT, we found no statistically significant difference.

In meta-analyses investigating moderators of iCBT, oftentimes the focus lies on amount of therapeutic contact (Păsărelu et al., 2017; Probst et al., 2019; Stech et al., 2019), average age of participants (Păsărelu et al., 2017; Probst et al., 2019), treatment duration (Păsărelu et al., 2017; Probst et al., 2019; Stech et al., 2019), adherence to treatment (Păsărelu et al., 2017; Probst et al., 2019), severity of symptoms (O'Kearney et al., 2019; Păsărelu et al., 2017) or study quality (Carlbring et al., 2018; Păsărelu et al., 2017). In this study, we wanted to extend the research scope by exploring other variables such as type of scale (PDSS or pooled effect sizes from scales targeting PD/A symptoms), sample size (N > or < 60), country of original research (either Australia, or Sweden or rest of the world), adherence to the treatment in form of an initial uptake (low or high percentage of login to the treatment program) and baseline severity for potential moderators. We found country of original research and initial uptake to be significant moderators of iCBT treatment. Among research trials conducted in Australia, Sweden or Romania, there were significant differences in within-group effect sizes, with the largest within-group effect size found in the Romanian trial from ⁎Ciuca et al. (2018) and the smallest in the Australian trials (g = 1.909 vs. g = 1.382 vs. g = 0.998). Our results contrast with the findings from Păsărelu et al. (2017). Whereas in our study, we investigated studies conducted in three countries, Păsărelu et al. (2017) included five countries of original research and found that researchers' country had no effect on anxiety, depression, or quality of life in anxious and/or depressed individuals. Regarding initial or login to treatment programs, there was a statistically significant difference between within-group effect sizes in the iCBT group. In the trials where fewer than 80% of allocated individuals logged into the treatment program, the effect size was g = 0.877. In case more than 80% logged into the Gsh iCBT treatment programs, the within-group effect size was significantly larger (g = 1.494). To our knowledge, beside the study from Lin et al. (2018) which found lower rates of login into treatment (and control) intervention, there are no studies investigating initial uptake in mobile- or internet-based interventions. Interestingly, the findings from our study suggest a relationship between higher login rates and larger treatment effects. We recommend exploring the role of uptake facilitating interventions during mobile- and internet-based treatment programs, as it seems that there might be a connection between initial login into an intervention program, treatment adherence and range of a treatment effect. Additionally, our findings confirm the assumption about higher treatment efficacy for patients with higher baseline symptom severity. In trials with PDSS score above 15 points, there were significantly larger within-group effect sizes found (g = 1.578) as opposed to those trials with lower baseline symptom severity (g = 1.022). Across the literature on iCBT, there have been similar conclusions in case of depression (Attridge, 2020; Button and Wiles, 2011; Spek et al., 2007; Warmerdam et al., 2013), as well as other conditions. For instance, the study from Attridge (2020) identified higher baseline severity to be a predictor for higher treatment efficacy in social anxiety, insomnia and general anxiety. Here, participants with clinical scores on SPIN, MOS and GAD-7 performed better as those with subclinical scores. With regards to panic severity at baseline, a study from El Alaoui et al. (2013) found that predictors of positive treatment response for both iCBT and group fCBT were having low levels of symptom severity, which is in contrast with our findings. Nonetheless, our study is the first meta-analysis investigating the moderators of iCBT treatment effect in panic disorder with or without agoraphobia.

4.2. Limitations

One of the limitations concerns the number of eligible RCTs and the number of participants included in our analyses. For instance, when comparing Gsh iCBT to active controls, we were able to investigate changes in quality of life in a data set not larger than k = 5 at post-test and k = 2 at follow-up. When compared to inactive controls, the data set was small as well (k = 3). When comparing Gsh iCBT to Sh iCBT in PD/A, comorbid anxiety and depression and improvements in quality of life, we were able to perform analyses including fewer than six RCTs. Generally, more high-quality RCTs investigating Gsh and/or Sh iCBT compared to other active and inactive controls are needed. Focus of these RCTs should be on post-test and long-term efficacy. Another problem are trials with small sample sizes, as larger samples have more power and therefore contribute more to a pooled outcome, which in turn leads to biased results (Cuijpers et al., 2019a). Hence, future RCTs with larger sample sizes (n > 30; Chambless and Hollon, 1998) are needed to establish a general efficacy of iCBT in panic disorder sufferers.

Another possible limitation is the number of Sh iCBT interventions included. Although unguided iCBT programs are a re-emerging topic, we were able to include only two RCTs which investigated the efficacy of Sh iCBT (⁎Ciuca et al., 2018; ⁎Fogliati et al., 2016). In these trials, the initial uptake was reported only in one study. Here, 89% of participants logged into the treatment program after randomization. In both studies, mean 65% of treatment modules were completed. Attrition rates were high at post-test (22%) and follow-up (28%). The mean percentage of panic-free individuals at post-test was rather low (38%). In conclusion, more RCTs investigating Sh iCBT for PD/A are needed, in order to establish an overall efficacy in primary symptoms, secondary comorbid symptoms and improvements in quality of life, as well as to establish the overall clinical significance, initial uptake, adherence and attrition rates in samples treated with this intervention. Therapeutic alliance is oftentimes regarded as a key predictor for change in face-to-face therapy (Cuijpers et al., 2019b; Horvath et al., 2011; Martin et al., 2000; Wampold et al., 2016; Wampold et al., 2017). As such, it is particularly self-help iCBT that needs to be a topic of future research exploration, as it directly upends the conventional wisdom about the role of therapeutic relationship in the symptom change of individuals suffering with a mental health disorder.

4.3. Implications for future research

The current study focused on disorder-specific Gsh iCBT. Transdiagnostic iCBT programs for PD/A were not investigated, although this is a promising topic. A meta-analysis from Păsărelu et al. (2017) investigated changes in anxiety and depression symptoms and found no differences in anxiety and quality of life outcomes between disorder-specific and transdiagnostic iCBT. Regarding changes in depression outcomes, transdiagnostic iCBT was found more effective. Moreover, evidence from few RCTs suggests that unguided transdiagnostic iCBT programs appear to be an efficient option for treating primary, as well as comorbid symptoms in panic disorder sufferers. Currently, there is no meta-analysis with the focus solely on PD/A sufferers. There is a meta-analysis in progress from the authors of the current study, covering this issue.

Mobile-based interventions present another emerging topic in treating PD/A with comorbid symptoms. Currently, there are only few existing studies (Christoforou et al., 2017; Ebenfeld et al., 2020; Ivanova et al., 2016) with focus on this issue. The RCT from Christoforou et al. (2017) reports that individuals with self-identified agoraphobia benefited equally from a disorder-specific and a transdiagnostic mobile-based intervention. Although the first of its kind, this study used no screening interview and the only inclusion criterion was self-identification as PD sufferer. Ivanova et al. (2016) investigated Acceptance and Commitment Therapy provided via the Internet and smartphone app (“ACT-smart”), in an unguided and clinician-guided setting. No differences between both formats in treating PD/A and social anxiety disorder was found. Adherence to the treatment was low (30 vs. 7.8%). Furthermore, a new large multi-methods feasibility study from (Ebenfeld et al., 2020) introduced a newly developed mobile app addressing symptoms of PD (“GET.ON Panic”). Here, conclusions about the clinical effectiveness were not possible due to a missing control group and insufficient power of the study. Overall, usability, user satisfaction, motivational value and technology acceptance of the app were perceived as high. We recommend that future studies focus on further investigating efficacy of mobile-based interventions, using RCTs with large sample sizes, screening interviews and reliable measurement scales. Particularly in the case of PD/A the use of mobile interventions is promising, as this treatment approach intervenes where panic attacks take place, namely in the everyday lives of affected individuals.

Declaration of competing interest

All authors declare no conflict of interest.

Appendix A. Appendices

Table A.1.

Benchmark effect size estimates of available treatments for panic disorder with or without agoraphobia.

Meta-analysis	Conditions	Trials	Panic with or without agoraphobia
			Between-group effect size	Within-group effect size
Andrews et al. (2018)	iCBT vs. CAU or inactive controls	Efficacy	g = 1.31	–
Caldirola et al. (2017)	Pharmacotherapy vs. inactive controls	Efficacy	g = 1.09	–
Hedman et al. (2012)	iCBT vs. inactive controls	Efficacy	–	d = 1.42
Hofmann and Smits (2008)	CBT vs. inactive controls	Efficacy	g = 0.35	–
Mitte (2005)	Pharmacotherapy vs. inactive controls	Efficacy	g = 0.51	–
Norton and Price (2007)	CBT vs. inactive controls	Efficacy	–	d = 1.56
Olthuis et al. (2016)	Gsh iCBT vs. inactive	Efficacy	d = 1.5	–
Stech et al. (2019)	Gsh iCBT vs. inactive controls	Efficacy	g = 1.22	–
Stech et al. (2019) (2)	Gsh iCBT	Effectiveness	–	g = 0.98
Stewart and Chambless (2009)	CBT	Effectiveness	–	g = 1.02

Table A.2.

Baseline and post-test estimates from PDSS scale.

Study	iCBT (PDSS)			Controls (PDSS)
	n	Baseline M (SD)	Post-test M (SD)	n	Baseline M (SD)	Post-test M (SD)
1. ⁎Allen et al. (2016)	27	13 (5.73)	6.36 (5.2)	36	12.14 (5.74)	11.75 (5.57)
2. ⁎Bergström et al. (2010)		14.1 (4.3)	6.3 (4.7)		14.2 (4.0)	6.3 (5.6)
3. ⁎Carlbring et al. (2001)	–	–	–	–	–	–
4. ⁎Carlbring et al. (2005)	–	–	–	–	–	–
5. ⁎Carlbring et al. (2006)	–	–	–	–	–	–
6. ⁎Carlbring et al. (2003)	–	–	–	–	–	–
7. ⁎Ciuca et al. (2018)	36	16.42 (4.76)	4.93 (3.64)	38	15.74 (4.91)	11.26 (6.38)
7. ⁎Ciuca et al. (2018) (2)	37	15.54 (4.51)	7.36 (5.28)	38	15.74 (4.91)	11.26 (6.38)
8. ⁎Fogliati et al. (2016)	68	11.77 (5.05)	7.36 (5.94)	64	12.95 (5.96)	7.23 (5.79)
8. ⁎Fogliati et al. (2016) (2)	65	12.32 (5.80)	7.94 (6.45)	67	12.36 (5.24)	6.68 (5.16)
9. ⁎Kiropoulos et al. (2008)	43	14.85 (4.40)	9.92 (5.88)	38	14.80 (5.04)	9.24 (5.65)
10. ⁎Klein et al. (2009)	23	14.96 (4.8)	11.13 (6.21)	28	14.14 (5.19)	10.60 (5.39)
11. ⁎Klein et al. (2006)	55	21.11 (3.7)	6.18 (6.2)	55	21.70 (4.50)	9.47 (7.90)
11. ⁎Klein et al. (2006) (2)	55	21.70 (4.50)	9.47 (7.90)	55	19.14 (4.50)	19.78 (4.40)
12. ⁎Richards et al. (2006)	32	16.54 (4.2)	9.54 (5.7)	32	17.00 (5.3)	16.78 (4.8)
12. ⁎Richards et al. (2006) (2)	32	19.00 (4.0)	6.77 (4.0)	32	17.00 (5.3)	16.78 (4.8)
13. ⁎Wims et al. (2010)	29	17.10 (4.84)	12.14 (5.74)	25	16.44 (4.63)	15.56 (5.77)

Table A.3.

Pooled between-group and within-group effect sizes at post-test and follow-up levels.

Study	Conditions	Between-group Hedges' g post-test	Between-group Hedges' follow-up	Within-group Hedges' post-test
Panic disorder with or without agoraphobia
1. ⁎Allen et al. (2016)	Gsh iCBT vs. WL	−0.1	–	−1.24
2. ⁎Bergström et al. (2010)	Gsh iCBT vs. fCBT	0.0	−0.19	−1.73
3. ⁎Carlbring et al. (2001)	Gsh iCBT vs. WL	−0.98	–	−1.78
4. ⁎Carlbring et al. (2005)	Gsh iCBT vs. fCBT	−0.11	−0.15	−0.98
5. ⁎Carlbring et al. (2006)	Gsh iCBT vs. WL	−1.18	–	−1.92
6. ⁎Carlbring et al. (2003)	Gsh iCBT vs. Applied relaxation	0.19	–	−0.67
7. ⁎Ciuca et al. (2018)	Gsh iCBT vs. WL	−1.29	–	−1.91
7. ⁎Ciuca et al. (2018) (2)	Sh iCBT vs. WL	−0.84	–	−1.07
7. ⁎Ciuca et al. (2018) (3)	Gsh iCBT vs. Sh iCBT	−0.42	–	–
8. ⁎Fogliati et al. (2016)	Gsh iCBT vs. Sh iCBT	0.22	0.01	−1.08
8. ⁎Fogliati et al. (2016) (2)	Gsh iCBT vs. Sh iCBT transdiagnostic	0.02	−0.11	−0.79
9. ⁎Kiropoulos et al. (2008)	Gsh iCBT vs. fCBT	0.19	–	−0.65
10. ⁎Klein et al. (2009)	Gsh iCBT vs. Gsh iCBT frequent	−0.03	–	−0.68
11. ⁎Klein et al. (2006)	Gsh iCBT vs. Gsh workbook	−0.37	−0.45	−1.58
11. ⁎Klein et al. (2006) (2)	Gsh iCBT vs. IC	−1.53		−1.58
12. ⁎Richards et al. (2006)	Gsh iCBT vs. IC	−0.85	–	−1.38
12. ⁎Richards et al. (2006) (2)	Gsh iCBT vs. Gsh iCBT + stress management	0.34	0.08	−1.38
13. ⁎Wims et al. (2010)	Gsh iCBT vs. WL	−0.33	–	−0.92
Anxiety and depression symptoms
1. ⁎Allen et al. (2016)	–	–	–	–
2. ⁎Bergström et al. (2010)	Gsh iCBT vs. fCBT	−0.26	−0.14	–
3. ⁎Carlbring et al. (2001)	Gsh iCBT vs. WL	−1.18	–	–
4. ⁎Carlbring et al. (2005)	Gsh iCBT vs. fCBT	−0.35	−0.32	–
5. ⁎Carlbring et al. (2006)	Gsh iCBT vs. WL	−0.94	–	–
6. ⁎Carlbring et al. (2003)	Gsh iCBT vs. Applied relaxation	0.18	–	–
7. ⁎Ciuca et al. (2018)	Gsh iCBT vs. WL	−1.04	–	–
7. ⁎Ciuca et al. (2018) (2)	Sh iCBT vs. WL	−0.8	–	–
7. ⁎Ciuca et al. (2018) (3)	Gsh iCBT vs. Sh iCBT	−0.21	–	–
8. ⁎Fogliati et al. (2016)	Gsh iCBT vs. Sh iCBT	0.03	−0.09	–
8. ⁎Fogliati et al. (2016) (2)	Gsh iCBT vs. Sh iCBT transdiagnostic	0.02	0.56	–
9. ⁎Kiropoulos et al. (2008)	Gsh iCBT vs. fCBT	0.2	–	–
10. ⁎Klein et al. (2009)	Gsh iCBT vs. Gsh iCBT frequent	0.16
11. ⁎Klein et al. (2006)	Gsh iCBT vs. Gsh workbook	−0.23	−0.29	–
11. Klein, Richards, & Austin (2006) (2)	Gsh iCBT vs. IC	−0.63	–	–
12. ⁎Richards et al. (2006)	Gsh iCBT vs. IC	−0.06	–	–
12. ⁎Richards et al. (2006) (2)	Gsh iCBT vs. Gsh iCBT + stress management	0.63	0.16	–
13. ⁎Wims et al. (2010)	Gsh iCBT vs. WL	−0.28	–	–
Quality of life
1. ⁎Allen et al. (2016)	–	–	–	–
2. ⁎Bergström et al. (2010)	–	–	–	–
3. ⁎Carlbring et al. (2001)	Gsh iCBT vs. WL	−0.74	–	–
4. ⁎Carlbring et al. (2005)	Gsh iCBT vs. fCBT	−0.21	−0.15	–
5. ⁎Carlbring et al. (2006)	Gsh iCBT vs. WL	−0.42	–	–
6. ⁎Carlbring et al. (2003)	Gsh iCBT vs. Applied relaxation	−0.3	–	–
7. ⁎Ciuca et al. (2018)	–	–	–	–
8. ⁎Fogliati et al. (2016)	–	–	–	–
8. ⁎Fogliati et al. (2016) (2)	–	–	–	–
9. ⁎Kiropoulos et al. (2008)	Gsh iCBT vs. fCBT	0.03	–	–
10. ⁎Klein et al. (2009)	Gsh iCBT vs. Gsh iCBT frequent	−0.29	–	–
11. ⁎Klein et al. (2006)	–	–	–	–
12. ⁎Richards et al. (2006)	Gsh iCBT vs. IC	−0.19	–	–
12. ⁎Richards et al. (2006) (2)	Gsh iCBT vs. Gsh iCBT + stress management	−0.55	−0.08	–
13. ⁎Wims et al. (2010)	–	–	–	–