App-Based Psychotherapy of Panic Disorder With Self-Guided Exposure in Virtual Reality

Abstract

Background

Patients with panic disorder often suffer from temporary unavailability of care. The smartphone app Invirto (IVT) provides digital treatment for panic disorder comprising self-guided exposure in virtual reality. The aim of this trial was to assess the efficacy of Invirto.

Methods

In a randomized, controlled, non-blinded trial, IVT was compared with care as usual (CAU) in patients with panic disorder (preregistration: DRKS00027585). The endpoints were assessed online before treatment (t0) and at 3 months (t1). The primary endpoint was the change in symptoms of anxiety, as measured with the Beck Anxiety Inventory (BAI), between the groups. The secondary endpoints were the patients’ scores on the following assessment instruments, all in their German versions: the Panic and Agoraphobia Scale (PAS), the Beck Depression Inventory (BDI-II), a questionnaire on patient satisfaction (Client Satisfaction Questionnaire, CSQ-8), the Acceptance and Action Questionnaire (AAQ-II), and quality of life as a global item in the World Health Organization Quality of Life Questionnaire (WHOQOL-BREF).

Results

One hundred twenty-four participants were included. The intention-to-treat analysis revealed greater improvement with IVT than with CAU with respect to both the primary endpoint (BAI, d = –0.46; 95% confidence interval [–0.87; –0.04]) and the secondary endpoints (PAS, d = –0.63 [–1.05; –0.22]; BDI-II, d = –0.44 [–0.86; –0.02]; AAQ-II, d = –0.42 [–0.84; –0.01]), except for WHOQOL-BREF (p = 0.216).

Conclusion

A digital treatment with virtual exposure can lessen anxiety, panic, and depressive symptoms and improve mental flexibility. In further studies, IVT should be compared with an active control group.

Panic disorder is characterized by recurrent and unpredictable panic attacks and constant worrying about further attacks. The 12-month prevalence in the general population is 1.7–2.4% (1, 2). Panic attacks are frequently accompanied by physical symptoms such as feelings of suffocation, tachycardia, or chest pain. Persons with panic disorder often adopt dysfunctional forms of behavior, e.g., avoidance behavior, to prevent future attacks. This may lead to substantial restrictions of their daily activities and reduced quality of life (3, 4).

The treatment guidelines recommend cognitive behavioral therapy (CBT) with exposure therapy (5). With or without therapist guidance, patients confront themselves with anxiety-inducing situations, thus gaining insight into their physical symptoms, safety strategies, and avoidance behavior (6). However, roughly one-third of individuals with panic disorder do not receive psychological treatment (7). The barriers to treatment include a shortage of trained therapists, high treatment costs, and shame and stigmatization around seeking psychological help (8).

Internet-based cognitive behavioral therapy (iCBT) is a promising option for the treatment of persons with panic disorder. iCBT comprises structured web- or app-based programs and specifically addresses the barriers to obtaining a place for face-to-face psychotherapy (9). iCBT programs have been shown to be effective in reducing anxiety, with moderate to large effect sizes when compared with inactive control conditions and small effect sizes when compared with active control groups with conventional CBT (10). Carlbring et al. (11) reported comparable efficacy in the treatment of panic symptoms (g = 0.05), with iCBT proving less costly (12).

The exposure for persons with panic disorder focuses mainly on interoceptive exposure, although external cues can also trigger panic attacks. The resulting intensification of agoraphobic behavior, often unrecognized and untreated, may be associated with panic disorder (13–15). Exposure therapy is considered one of the most effective elements of classical CBT (16, 17) and has been replicated for the iCBT setting. Virtual reality (VR) can be integrated into exposure therapy to simulate real scenarios. This permits better control over the stimuli presented and thus enables greater precision during the exercise.

For anxiety-related disorders, exposure therapy in VR (VRET) shows an overall large effect size compared with wait-list conditions (g = 0.90) and a medium to large effect size compared with active control conditions (g = 0.78) (18). While VRET might be useful in enhancing the efficacy of iCBT for persons with panic disorder, to our knowledge no study has yet investigated the efficacy of self-guided VRET. German health insurance funds currently cover the costs of only one app-based program that integrates self-guided VRET for panic disorder: Invirto. Initial data on the smartphone app Invirto (IVT) have been published on the website of the Federal Institute for Drugs and Medical Devices (available in German from https://diga.bfarm.de/de/verzeichnis/00300). However, patients with panic disorder were slightly underrepresented, so more data are needed.

Objective

The aim of the study was to investigate the efficacy of IVT in comparison with a control group with access to care as usual (CAU). Based on meta-analytic data (19), the following assumptions were made:

• Greater improvement in both the primary endpoint and the secondary endpoints in the IVT group than in the CAU group over the 3-month intervention period (t0–t1)

• After delayed receipt of the intervention in the CAU group, a decrease in both the primary and secondary endpoints between t1 and t2

• High satisfaction with IVT (as measured using the Client Satisfaction Questionnaire, CSQ-8).

The primary endpoint was the difference between the baseline and postassessment scores on the Beck Anxiety Inventory (BAI).

The secondary endpoints were changes in the following:

• Panic-specific symptoms (measured on the Panic and Agoraphobia Scale, PAS)

• Psychological flexibility (Acceptance and Action Questionnaire, AAQ-II)

• Symptoms of depression (Beck Depression Inventory, BDI-II)

• Quality of life (global item of the World Health Organization Quality of Life Questionnaire, WHOQOL-BREF)

Methods

Study design

The study was a two-armed randomized controlled trial (RCT) comparing IVT with CAU. The IVT group received access to the intervention immediately after the beginning of the study (t0) and its members were instructed to follow the IVT treatment pathway (Table 1). The CAU group received no instructions and were informed that they could make use of their CAU as needed. After 3 months (t1), cross-over of the treatments was performed and the CAU group also received access to IVT. A pre–post comparison was carried out to analyze within-group effects (the results of the follow-up survey and the satisfaction rates can be found in the eResults, eFigure 1, eTable 1, and eTable 2). The study was approved by the local ethics committee (LPEK-0415) and preregistered with the German clinical trials registry (#DRKS00027585). Further details can be found in the eMethods and in the published study protocol (20).

Table 1. Modules and Content of IT.

Module no.	Content and goals of the individual modules
1	• Overview of treatment, explanation of psychotherapy • Guidance on use of the app • Development of a digital therapeutic relationship • Psychoeducation about panic disorder, reduction of stigma • Managing expectations
2	• Visualization of anxiety-related limitations in everyday life (to create cognitive dissonance about the status quo) • Problem definition, summarizing symptoms and fears • Validation of the difficulty of change • Costs and benefits of changing behavior • Definition of goals for the treatment
3	• Psychoeducation: healthy and pathological anxiety • Self-reinforcing feedback loops • Conditions that develop and maintain anxiety • Self-observation of anxiety situations and occurrence of anxiety
4	• Problems due to avoidance/safety behavior • Breathing technique for panic attacks, relaxation techniques • Psychoeducation: autonomic nervous system • Psychoeducation: sympathetic and parasympathetic ner•vous systems
5	• Typical thoughts in anxiety states • Effects of anxious thoughts • Exploration of one’s own anxious thoughts
6	• Rationale for exposure • Interoceptive exposure (e.g., hyperventilation) and exposure in virtual reality (e.g., checking fears and practicing strategies against anxiety in scenarios such as a train ride or the supermarket)
7	• Function of emotions • Relationship between emotion avoidance and fear • Diversity of emotions • Emotions as indicators of needs • Acceptance of emotions
8	• Review of the treatment • How to deal with difficult situations • Preparation of an emergency kit • Tips for further practice • Planning for the next phase of exercises

eResults.

Results supplement

Demographic characteristics

There was no difference between the IVT group and the CAU group in terms of demographic characteristics (Table 2), except for the BAI baseline score, which was higher in the CAU than in the IVT group. Analysis of regression to the mean revealed a significant negative correlation between the BAI baseline scores (r = -0.459, p < 0.001) and the difference scores from t0 to t1. This indicates that higher baseline scores are associated with greater reduction of symptoms from t0 to t1.

Altogether, 26.61% of the participants (n = 33) reported having comorbid depression (IVT group 34.92%, n = 22; CAU group 18.03%, n = 11), and 29.03% (n = 36) also reported other comorbid disorders (four persons with attention deficit/hyperactivity syndrome [ADHS], seven with personality disorder, six with eating disorder, six with obsessive–compulsive disorder, and 13 with post-traumatic stress disorder [PTSD]). A high proportion (43.55%, n = 54) of participants had experience with face-to-face psychotherapy, including some who were currently undergoing psychotherapy (8.1%, n = 10), and about a third of the participants reported taking antidepressants (31.4%, n = 39).

ITT analyses: within-group analysis after cross-over

After treatment with IVT at t1, the CAU group showed a decrease in anxiety symptoms (BAI) from t1 (M = 26.5, SD = 11.6) to t2 (M = 20.5, SD = 9.6), with a moderate effect size (t = –3.25, p = -0.001, d = -0.53, n= 61). The results on secondary outcomes within the CAU group from t1 to t2 are presented in eTable 1.

Complete-case analyses

The CC analyses (eTable 2) differ in approach from the ITT analyses (Table 4). While ITT analysis includes all participants, regardless of their intensity of participation, CC analysis includes only those who completed the study or adhered to the treatment protocol. ITT analysis thus tends to show conservative effects, whereas CC analysis tends to provide more optimistic results because it takes only active participants into consideration.

Treatment satisfaction with the content of Invirto (CSQ-8)

The agreement rates for each item of the CSQ-8 (defined as the percentage of participants selecting positive anchor points as agreement) are presented in eFigure 1. CC analysis of satisfaction ratings within the intervention group (t1, n = 47) showed an average score of 21.69 points out of a possible 32. Most of the participants were satisfied with the quality of IVT support (85%) and the help they received (87%).

Within-group effect (CAU group)

Within the CAU group there was a decrease in anxiety symptoms from t0 (M = 31.8, SD = 12.9) to t1 (M = 26.4, SD = 11.6) (BAI; t = -3.37, p < 0.001, d = –0.39, [-0.62; -0.17], n = 61).

Negative effects

The following obstacles were reported by patients who stated that they used Invirto less than once a week:

• Two persons were less motivated and were reluctant to use the app several times a week because they found the depiction of anxiety as an “anxiety monster” ridiculous and unpleasant. They did not feel that their anxiety was taken seriously or did not find the strategies helpful.

• Three persons stated that they were fine at the moment and therefore did not need the app or had not used it.

• Five persons could not concentrate or motivate themselves enough to use the app.

• Eight persons were detrimentally affected by external obstacles such as life events (e.g., separation from partner).

• One person needed more time to process the content of Invirto.

• One person had no time and did not want to deal with their own feelings.

Discussion

The patients’ overall high satisfaction with the content may be advantageous with regard to treatment adherence given that iCBT adherence rates tend to be low (e11, e12) and higher adherence correlates with better treatment results (e13, e14). It should be taken into account, however, that this finding is potentially biased, as dissatisfied patients may not have provided feedback; moreover, 25% did not participate in the t1 assessment. Future studies could evaluate whether gamification (e.g., adding a visual reward system) (e15) would encourage participants to use IVT more frequently.

eFigure 1.

Client Satisfaction Questionnaire (CSQ)

eTable 1. Anxiety, panic, depression, psychological flexibility, quality of life at t2, after treatment cross-over^*1.

	IVT (n = 63)		CAU (n = 61)
	M (SD)	ITT*2 t1 to t2	M (SD)	ITT*2 t1 to t2
BAI	15.6 (8.4)	t = 2.76, p =0.006, d = −0.439 5% CI [−0.74; −0.13]	20.5 (9.6)	t = −3.25, p = 0.001, d = −0.53 95% CI [−0.84; −0.21]
PAS	11.4 (7.6)	t = −1.62, p = 0.11, d = −0.33 95% CI [−0.73; −0.07]	15.1 (8.7)	t = −3.31, p = 0.001, d = −0.70 95% CI [−1.12; −0.29]
BDI-II	12.9 (10.1)	t = −3.44, p = 0.001, d = −0.71 95% CI [−1.11; −0.30]	15.4 (9.6)	t = −5.17, p = 0.000, d = −1.06 95% CI [−1.47; −0.66]
WHOQOL-BREF	3.4 (0.9)	t = 1.27, p = 0.20, d = 0.27 95% CI [−0.15; 0.69]	3.1 (0.9)	t = 1.14, p = 0.26, d = 0.24 95% CI [−0.18; 0.66]
AAQ-II	19.5 (8.6)	t = −3.01, p = 0.003, d = −0.63 95% CI [−1.04; −0.22]	23.4 (8.7)	t = −4.37, p = 0.000, d = −0.99 95% CI [−1.43; −0.55]

^*1 Mean (standard deviation) at t2

^*2 The value d is a measure of the effect size for standardization of the mean difference between two samples in an at least interval-scaled variable. Example: The within-group difference of the IVT group for BAI scores revealed a reduction from t1 to t2 (t = -2.76, p = 0.006), with an effect size of d = -0.43. The 95% confidence interval (95% CI) for the effect size ranged from -0.74 to -0.13, indicating an improvement in anxiety levels within the group.

AAQ-II, Acceptance and Action Questionnaire-II; BAI, Beck Anxiety Inventory; BDI-II, Beck Depression Inventory-II; CAU, care as usual; ITT, intention to treat; IVT, Invirto therapy; PAS, Panic and Agoraphobia Scale; WHOQOL-BREF, global item of the World Health Organization Quality of Life Questionnaire

eTable 2. Results of analyses of variance (ANCOVA) for the completed cases (CC) across time for primary and secondary endpoints (means and standard deviations refer to CC analyses).

	IVT (n = 47)		CAU (n = 43)		CC t0 to t1
	t0	t1	t0	t1
BAI	26.91 (13.37)	18.91 (11.84)	32.23 (12.29)	27.81 (13.43)	F(1, 87) = 7.01, p < 0.001. η²p = 0.075
PAS	19.96 (10.32)	13.02 (8.41)	23.09 (11.29)	21.28 (10.40)	F(1, 87) = 17.81, p < 0.001. η²p = 0.170
BDI-II	23.32 (10.50)	18.39 (11.27)*	27.12 (10.36)	25.81 (11.17)	F(1, 86) = 6.41, p = 0.010. η²p = 0.075
WHOQOL-BREF	2.97 (0.90)	3.24 (0.79)*	2.85 (0.87)	2.88 (0.98)	F(2, 86) = 4.32, p =0.041. η²p = 0.048
AAQ-II	27.18 (8.89)	23.70 (9.57)*	30.79 (8.60)	30.91 (8.62)	F(1, 86) = 9.89, p = 0.002. η²p = 0.103

Example (BAI): The ANCOVA shows a difference between the groups.

The partial eta squared (e.g., η²_p = 0.075) suggests a moderate effect.

* n = 46

AAQ-II, Acceptance and Action Questionnaire-II; BAI, Beck Anxiety Inventory; BDI-II, Beck Depression Inventory-II; CAU, control group (care as usual); IVT, intervention group (Invirto therapy); PAS, Panic and Agoraphobia Scale; WHOQOL-BREF, World Health Organization Quality of Life Questionnaire—global item

eMethods.

Methods supplement

No artificial intelligence (AI) was used while the study was ongoing, but AI was employed for some aspects of manuscript preparation (e.g., data compilation).

Study procedure

Recruitment was conducted via flyers and social media posts. Persons interested in study participation were able to contact the provider, Sympatient, via the study website, by telephone, or by sending an e-mail. The inclusion and exclusion criteria were then reviewed by Sympatient, and participants were referred to psychologists employed by Sympatient for the diagnostic interview. All participants provided electronic informed consent at t0. Data collection took place between 17 February 2022 and 30 December 2023. As shown in the Figure, 143 persons took part in the online survey. Of these, 19 were excluded (two did not provide informed consent, eight terminated the assessment early, six reported psychotic symptoms, and three reported suicidal tendencies). Eventually 124 persons met all the criteria and were randomized to either the IVT group (n = 63) or the CAU group (n = 61). The response rate was 72.58% (90/124) at t1 and 55.65% (69/124) at t2. Thirty-four participants (27.42%) did not take part in the follow-up assessment because they could not be reached (n = 31) or did not want to continue with the study (n = 3). Fifteen persons in the CAU group stated at t1 that they had taken advantage of psychological care services outside of the study. These included day clinic or inpatient treatment, individual and/or group therapy, exercise therapy, and relaxation methods. Twenty-one persons (16.93%) did not participate at t2 without giving a reason.

As soon as participants had completed all three assessments, they were compensated with a voucher for € 60. Later in the study, that amount was split between t1 (€ 30) and t2 (€ 30) in an effort to reduce drop-outs.

A total of 143 persons took part in the online survey (Figure). It proved challenging to reach the original recruitment target of 128 participants. Reasons included the fact that some of the potential participants were not willing to be assigned to the control group and therefore preferred to use IVT outside of the study. To avoid extending the data collection period, we decided to end recruitment earlier than planned, blind to the data.

Sample size calculation

In a meta-analysis, Haug, Nordgreen, Öst, and Havik (30) showed an average effect size of Hedges’ g = 0.55 (≈ f = 0.28) for the effectiveness of iCBT interventions for treatment of anxiety states compared with (wait-list) control groups. A power calculation using G*Power software (e2) indicated that a sample size of N = 103 would be necessary to achieve a moderate effect size (f = 0.28) with an alpha level of 0.05 and power of 0.80 in an ANCOVA. Anticipating a drop-out rate of 20% on the basis of meta-analytic findings from similar studies (30), we set out to recruit N = 128 participants.

Randomization

To ensure balance between the groups (IVT or CAU), randomization was performed using the automated survey program, so that allocation remained concealed. Neither the participants nor the researchers could influence or predict group allocation. The study participants were allocated to the two groups with automatic ratio adjustment via Qualtrics according to block randomization with a predetermined block size of 5:5. After the t0 assessment, participants were automatically randomized and notified about their group allocation. After randomization, participants remained in their allocated group until the end of the study. Participants were allowed to leave the study at any time they wanted. Since the endpoints in this study were captured by self-report questionnaires, no blinded assessors were needed. The data analysts were not blinded to randomization.

Psychopathology

Primary endpoint

Beck Anxiety Inventory (BAI)

To measure the change in anxiety symptoms, the German version of the Beck Anxiety Inventory (BAI; [25]) was used. The BAI assesses anxiety-associated sensations and cognitions. It measures the intensity of these sensations and thoughts on a four-point Likert scale. The self-report questionnaire was administered at all three time points (t0, t1, t2). BAI scores can be categorized based on symptom severity: minimal anxiety (0–7), mild anxiety (8–15), moderate anxiety (16–25), and severe anxiety (30–63). This assessment, with a reliability of α = 0.90, was shown to be valid and change sensitive (e3).

Secondary endpoints

Panic and Agoraphobia Scale (PAS)

The Panic and Agoraphobia Scale (PAS; [e4]) assessed the intensity of panic symptoms (with or without agoraphobia) over the previous week. Five domains that impact the quality of life of persons with panic disorder are assessed using 13 items on a five-point Likert scale: panic attacks, agoraphobic avoidance, anticipatory anxiety, restrictions, and health fears. The possible scores range from 0 to 52, with higher scores corresponding to heightened levels of panic (0–8 = no panic, 9–18 = minimal panic, 19–28 = moderate panic, 29–39 = severe panic, > 40 = very severe panic). The PAS has been proved to be valid and has reliability of α = 0.86 (e4). The self-report questionnaire was administered at all three time points (t0, t1, t2).

Beck Depression Inventory-II (BDI-II)

The Beck Depression Inventory-II (BDI-II; [e5]) was used to evaluate depressive symptoms experienced within the previous 2 weeks on the basis of 21 items. The BDI scale spans from 0 to 63, with higher scores indicating increased levels of depression. Symptom severity is categorized as follows: 0–8 indicates an absence of depression, 9–13 shows minimal depression, 14–19 reflects mild depression, 20–28 suggests moderate depression, and 29–63 signifies severe depression. The internal reliability is strong, with a Cronbach’s α of 0.89 (e6). The self-report questionnaire was administered at all three time points (t0, t1, t2).

Acceptance and Action Questionnaire-II (AAQ-II)

The Acceptance and Action Questionnaire-II (AAQ-II; [37]) was used to measure experiential avoidance and psychological flexibility. It encompasses negative assessments of emotions (e.g., “Anxiety is bad”) and the avoidance of thoughts and feelings (e.g., “I try to suppress thoughts and feelings that I don’t like by just not thinking about them”). The questionnaire comprises seven items and utilizes a seven-point Likert scale ranging from “(1) never true” to “(7) always true.” In the German version, internal consistency reveals a Cronbach’s α of 0.84 for persons with social phobia and 0.97 for students (38). The self-report questionnaire was administered at all three time points (t0, t1, t2).

Quality of Life–global item (WHOQOL-BREF)

Quality of life was measured with the Quality of Life–global item (WHOQOL-BREF) developed by the WHOQOL Group of the World Health Organization (e7). It is a self-report item with answers ranging from “(1) very poor” to “(5) very good.” With a Cronbach’s α of 0.7, internal consistency can be seen as acceptable (e8). The item was administered at all three time points (t0, t1, t2).

Appraisal of the intervention

Client Satisfaction Questionnaire-8 (CSQ-8)

To evaluate users’ subjective appraisal of the intervention, the Client Satisfaction Questionnaire–8 (CSQ-8¸ [e9]) was used. It comprises eight questions, each presenting four response options (from “(1) most unfavorable” to “(4) most positive”) without a neutral stance; negative and positive responses are each dichotomized into “agreement” or “disagreement.” Summation yields a total score ranging from 8 to 32. Various studies and samples have demonstrated the reliability and validity of the questionnaire, with Cronbach’s α values ranging between 0.87 and 0.92 (e9). The self-report questionnaire was administered at the time points t1 and t2.

Data analysis

SPSS 25.0 (IBM Corp., Armonk, NY) software was used for all analyses. The secondary analyses were complete-case (CC) analyses. CC analyses drew on data from participants who completed the assessments at all three time points. The effect sizes are reported using partial eta squared, referring to η²_p ≈ 0.01 as small, η²_p ≈ 0.06 as moderate, and η²_p ≈ 0.14 as large effect sizes (e10).

Participants and procedure

Potential study participants underwent a clinical diagnostic interview for DSM-5 (SCID-5) (DSM, Diagnostic and Statistical Manual of Mental Disorders) (21) with a certified psychologist to confirm the diagnosis of panic disorder. After clarification of inclusion and exclusion criteria (Box), a survey via the online platform Qualtrics were carried out at baseline (t0) to document self-reported sociodemographic and psychopathological data (Tables 2 and 3). The health-related primary and secondary endpoints were assessed again 3 months after baseline (t1) and in the follow-up survey 6 months after baseline (t2) (Figure).

Box.

• Inclusion criteria

  • Diagnosis of a panic disorder, verified by SCID-5

  • Panic symptoms for at least 12 months

  • Age between 18 and 80 years

  • Electronic informed consent given

  • Possession of a smartphone with internet access

• Exclusion criteria

  • Acute suicidality, assessed by SCID-5

  • Diagnosis of schizophrenia or a bipolar disorder, verified by SCID-5

Table 2. Descriptive characteristics and group comparisons of the participants at baseline (t0)^*1.

	IVT (n = 63)	CAU (n = 61)
Gender  (female/male)*2	39 (62%)/ 24 (38%)	47 (77%)/ 14 (23%)
Age in years	36.3 (10.6)	37.6 (10.3)
Education
University degree	13 (20.6%)	12 (19.7%)
High school diploma	12 (19%)	16 (26.2%)
Other secondary school	37 (58.7%)	31 (50.9%)
No school qualification	1 (1.6%)	2 (3.3%)
Employment status
Working full-time	23 (36.5%)	26 (42.6%)
Working part-time	11 (17.5%)	10 (16.4%)
Other*3	21 (33.3%)	11 (18%)
Unemployed	8 (12.7%)	14 (23%)
Relationship status
No relationship	12 (19%)	10 (16.4%)
In a relationship	51 (81%)	51 (83.6%)
General internet usage  (more/less than 5 hours daily)	44 (69.8%)/ 19 (30.2%)	41 (67.3%)/ 20 (32.8%)

^*1 Expressed as frequency (percentage) or mean (standard deviation)

^*2 Participants were able to choose from “male,” “female,” and “gender-diverse”; since none of them chose “gender-diverse,” it was excluded from the analysis

^*3 “Other” includes care work, retired, in vocational training, unemployed CAU, Control group (care as usual); IVT, intervention group (Invirto therapy)

Table 3. Participants’ clinical characteristics at baseline (t0)*.

	IVT (n = 63)	CAU (n = 61)
Medication
Antidepressants	23 (36.5%)	16 (26.2%)
None	40 (63.5%)	45 (73.8%)
Psychotherapy (other than Invirto)
Currently	5 (7.9%)	5 (8.2%)
In the past	22 (34.9%)	22 (36.1%)
No psychotherapy	36 (51.1%)	34 (55.7%)
Duration of panic disorder (in years, since diagnosis)	4.21 (7.87)	5.36 (7.75)
Psychopathology
BAI	26.3 (13.0)	31.8 (12.9)
PAS	20.6 (10.7)	23.7 (11.1)
BDI-II	23.7 (10.2)	26.5 (10.2)

* Expressed as frequency (percentage) or mean (standard deviation)

BAI, Beck Anxiety Inventory; BDI-II, Beck Depression Inventory-II; CAU, control group (care as usual); PAS, Panic and Agoraphobia Scale; IVT, intervention group (Invirto therapy)

Figure.

Flow chart: overview of the study

CAU,Care as usual; CC, complete cases; ITT, intention to treat; IVT, Smartphone app Invirto

Intervention

IVT is an app-based iCBT program with self-guided VRET for the treatment of panic disorder (eight modules, approx. 15 hours; Table 1), to be completed over a period of 12 weeks. The contents of IVT are in line with the national guidelines for the treatment of anxiety disorders (5) and are based on established German treatment manuals (22–24). IVT comprises audio and video lessons recorded with experienced psychotherapists together with practical exercises in virtual and interoceptive exposure as well as cognitive restructuring. The app can be implemented as often as the user wishes. For use of the app (e.g., at home), each patient receives VR glasses and headphones. To support their self-guided VRET, patients have the option of taking part in two videoconference sessions (each lasting 50 minutes) with a clinical psychologist from the provider, Sympatient.

Measures

All instruments are described in full in the eMethods. The methodology and the endpoints were selected for comparability with the prior study on IVT. The primary endpoint was the difference between the two groups in changes in anxiety symptoms from t0 to t1, as measured by the BAI (25, 26).

The secondary endpoints were changes in the following:

• Anxiety symptoms (PAS)

• Depression (BDI-II)

• Quality of life (WHOQOL-BREF)

• Psychological flexibility (AAQ-II)

• Satisfaction with the program (CSQ-8)

Negative effects were assessed with the aid of the Reliable Change Index (RCI), which determines clinically significant changes (deterioration) in a person’s scores, accounting for test reliability and measurement error (27). In this study, the RCI was used to measure significant symptom deterioration in BAI scores between t0 and t1. On the assumption that negative experiences with IVT would lead to lower adherence, patients who reported using IVT less than once a week were asked about barriers to its use (eMethods).

Data analysis

Intention-to-treat (ITT) analyses with multiple imputation were conducted as primary analyses. Group allocation, sociodemographic variables (Table 2), and psychopathological variables at baseline (Table 3) were entered into the model as predictors for imputation; 100 imputations were run. To test the hypotheses, analyses of covariance (ANCOVA) were calculated using the respective baseline scores as covariate. The dependent variable was the change in BAI values from t0 to t1. Combined p-values and effect sizes are reported using Cohen’s d; values from 0.2 to < 0.5 indicate a small, 0.5 to < 0.8 a moderate, and > 0.8 a large effect size (28).

Results

Demographic characteristics

The demographic data are displayed in Table 2 and described in more detail in the eResults.

Efficacy

ITT analyses

For the primary endpoint anxiety (BAI), analyses showed a greater improvement in the IVT group than in the CAU group, with a small effect (p < 0.029; d = –0.46; 95% confidence interval [-0.87; -0.04]; Table 4, eFigure 2). Improvement of secondary endpoints was also greater in the IVT group, with a moderate effect for panic symptoms (PAS; p < 0.003; d = –0.63, [-1.05; -0.22]) and small effects for depressive symptoms (BDI-II; p = 0.040; d = –0.44, [-0.86; -0.02]) and for experiential avoidance and psychological flexibility (AAQ-II; p = 0.045; d = –0.42, [-0.84; -0.01]). Change in quality of life did not differ between the groups (WHOQOL-BREF; p = 0.216; d = 0.24, [-0.15; 0.63]).

Table 4. Primary endpoint anxiety and secondary endpoints panic, depression, psychological flexibility, quality of life—results of covariance analysis (ANCOVA) for the ITT analysis at t0 and t1.

	IVT (n = 63)		CAU (n = 61)		ITT*1*2 t0 to t1
	t0	t1	t0	t1
BAI	26.3 (13.0)	19.9 (10.5)	31.8 (12.9)	26.5 (11.6)	F(1,1022.533) = 4.789; p = 0.029, d = −0.46 95% CI [−0.87, −0.04]
PAS	20.6 (10.7)	13.8 (8.7)	23.7 (11.7)	20.6 (10.2)	F(1,656.521) = 8.725; p = 0.003, d = −0.63 95% CI [−1.05, −0.22]
BDI-II	23.7 (10.2)	19.3 (11.2)	26.5 (10.2)	25.1 (10.9)	F(1,1086.427) = 4.217; p =0.040, d = −0.44 95% CI [−0.86, −0.02]
WHOQOL-BREF	2.97 (0.90)	3.2 (0.8)	2.85 (0.87)	2.9 (0.9)	F(2,1984.933) = 1.533; p =0.216, d = 0.24 95% CI [−0.15, 0.63]
FAH-II	27.18 (8.89)	24.4 (9.6)	30.8 (8.6)	30.5 (8.8)	F(1,1234.716) = 4.032; p =0.045, d = −0.42 95% CI [−0.84, −0.01]

^*1 F indicates that the test procedure uses an F statistic based on a theoretical F distribution to calculate the p-value. The F-distribution has two parameters, in parentheses, that influence its appearance and thus also the limit of significance, e.g., F(2,131)

^*2 The value d is a measure of the effect size and standardizes the mean difference between two samples in an at least interval-scaled variable. Example: The analysis of the between-group results revealed a significant difference (F(1,1022.533) = 4.789; p < 0.029) with a small effect size (d = -0.46). The 95% confidence interval (95% CI) for the effect size ranged from -0.87 to -0.04, suggesting a meaningful difference between the groups.

AAQ-II, Acceptance and Action Questionnaire-II;

BAI, Beck Anxiety Inventory;

BDI-II, Beck Depression Inventory-II;

CAU, care as usual;

ITT, intention to treat;

IVT, Invirto therapy;

PAS, Panic and Agoraphobia Scale;

WHOQOL-BREF, global item of the World Health Organization Quality of Life Questionnaire

eFigure 2.

Changes in the primary endpoint, anxiety (BAI), in the two groups

The analyses showed a greater improvement in the primary endpoint, symptoms of anxiety (BAI), in the IVT group than in the CAU group, with a small effect size (p < 0.029; d = -0,46; 95% confidence interval [-0.87; -0.04])

Negative effects: symptom deterioration (RCI)

The ITT analyses showed that one person in the IVT group and two persons in the CAU group experienced clinically meaningful symptom deterioration from t0 to t1, as measured by the RCI.

Discussion

App-based iCBT represents a promising tool for addressing panic symptoms and broadening the application of evidence-based treatment, by virtue of its low-threshold accessibility, flexibility, and independence from time and place. Patients might find smartphone apps such as IVT easier to use, as other digital interventions are available only in web-based form. The study reported here revealed greater improvement for IVT than for CAU across all primary (BAI) and secondary (PAS, BDI-II, AAQ-II) endpoints, except for quality of life (WHOQOL-BREF), over a 3-month period.

For the primary endpoint, the findings show that IVT was superior to CAU in reducing anxiety symptoms (BAI), although the effect size was small (d = -0.46) and thus lower than the moderate to large effect sizes reported in previous studies (29, 30). This could be due to various factors:

• First, unexpected external factors, e.g., the pandemic occurring during the study period, may have amplified patients’ anxiety.

• Second, although IVT is based on an empirically validated treatment, some participants experienced technical problems that may have have reduced its efficacy.

• Third, methodological factors may have played a role as panic symptoms (measured using the PAS) in particular showed improvement with a moderate effect size (d = -0.63), which is in line with the results of previous research (10) and may better reflect the objectives of IVT, as this scale is explicitly aimed at panic symptoms.

Although the BDI-II t0 scores may seem high in the present sample, the data are in line with other research showing a high comorbidity rate of 38.5% between panic disorder and major depressive disorder (31). Depressive symptoms at t0 may have been more severe than usual as the survey took place during the COVID-19 pandemic (32, 33). In line with research showing that patients who receive panic-focused psychotherapy often experience clear remission of their diagnosed depression (34), the use of IVT resulted in a decrease in depressive symptoms with a small effect size (d = -0.44). However, lower adherence was associated with more pronounced depressive symptoms, which could have also influenced drop-out rates in the present sample. Additional optimization may be needed to keep patients with comorbid depression in treatment (34).

Panic disorders can impact quality of life (3, 4), particularly when left untreated (35, 36). In the present study, no group differences in quality of life were found (p = 0.2). Even within the groups, there was only a slight difference between t0 and t1. However, there was a slightly greater absolute difference in the IVT group from t0 to t1, which could be increased through further improvements such as additional modules geared to the needs of individual patients.

The level of experiential avoidance (AAQ-II) showed improvement with a small effect size (d = -0.42). In the context of exposure, this might be critical in regard to willingness to continue with aversive experiences and to take action instead of engaging in avoidance (37, 38).

Limitations

Besides the present study’s strengths (e.g., compliance with preregistration), some general limitations must be acknowledged.

Although study inclusion was based on diagnosis verification via structured clinical interviews by trained clinicians, all results are based on self-reports. The data may thus differ from clinical assessment. Not requiring face-to-face interviews can avoid alienating those who feel intense shame or fear of stigma; such persons tend to benefit greatly from internet-based treatments. Expert assessments based on telephone interviews, allowing for anonymity, should therefore be considered for future research.

In addition, we did not use face-to-face therapy as an active control condition, which should be considered in a follow-up study. Although all participants had the option of taking part in conventional treatment, only 8.1% of them chose to do so.

The present study did not acquire objective user data (e.g., number of conducted exposures in VR). This would therefore be an important consideration for follow-up studies (e.g., number of modules completed or order of completion), as usage outside the study setting might differ from engagement in research (39, 40).

Despite the two groups’ baseline differences in the primary endpoint (BAI score), randomization was considered successful as it was automatically carried out via Qualtrics with a ratio adjustment. We therefore consider that the psychopathological differences in BAI baseline score were coincidental.

Furthermore, as in many online studies and in line with epidemiological data (e1), women were over-represented in the present study, limiting generalizability.

Conclusion

To the best of our knowledge, this is the first study to investigate the efficacy of an unguided app-based iCBT program for panic disorder with self-guided exposure therapy. IVT was found superior to CAU in reducing anxiety, comorbid depressive symptoms, and experiential avoidance, but not in improving quality of life. This study offers additional evidence for the efficacy of app-based iCBT and its incorporation into treatment guidelines for panic disorder as an evidence-based intervention.