This randomized clinical trial examines the effectiveness of a self-guided app-based virtual reality cognitive behavior therapy using low-cost goggles.
Key Points
Question
Is fully self-guided app-based virtual reality cognitive behavior therapy using low-cost (cardboard) virtual reality goggles user friendly and effective in reducing acrophobia symptoms compared with a wait-list control group?
Findings
In a single-blind randomized clinical trial that included 193 participants with acrophobia symptoms, app-based therapy demonstrated a large and significant reduction in acrophobia symptoms compared with wait-list controls and was rated as user friendly.
Meaning
Acrophobia cognitive behavioral therapy can be effectively delivered without therapist intervention through standard smartphones and low-cost virtual reality goggles at a fraction of the cost of current face-to-face treatment or high-end virtual reality exposure therapy.
Abstract
Importance
Globally, access to evidence-based psychological treatment is limited. Innovative self-help methods using smartphone applications and low-cost virtual reality have the potential to significantly improve the accessibility and scalability of psychological treatments.
Objective
To examine the effectiveness of ZeroPhobia, a fully self-guided app-based virtual reality cognitive behavior therapy (VR CBT) using low-cost (cardboard) virtual reality goggles compared with a wait-list control group and to determine its user friendliness.
Design, Setting, and Participants
In a single-blind randomized clinical trial, participants were enrolled between March 24 and September 28, 2017, and randomly assigned (1:1) by an independent researcher to either VR CBT app or a wait-list control group. A total of 193 individuals aged 18 to 65 years from the Dutch general population with acrophobia symptoms and access to an Android smartphone participated. The 6 animated modules of the VR-CBT app and gamified virtual reality environments were delivered over a 3-week period in participants’ natural environment. Assessments were completed at baseline, immediately after treatment, and at 3-month follow-up. Analysis began April 6, 2018, and was intention to treat.
Intervention
Self-guided app-based VR CBT.
Main Outcomes and Measures
The primary outcome measure was the Acrophobia Questionnaire. The hypothesis was formulated prior to data collection.
Results
In total, 193 participants (129 women [66.84%]; mean [SD] age, 41.33 [13.64] years) were randomly assigned to intervention (n = 96) or a wait-list control group (n = 97). An intent-to-treat analysis showed a significant reduction of acrophobia symptoms at posttest at 3 months for the VR-CBT app compared with the controls (b = –26.73 [95% CI, −32.12 to −21.34]; P < .001; d = 1.14 [95% CI, 0.84 to 1.44]). The number needed to treat was 1.7. Sensitivity and robustness analysis confirmed these findings. Pretreatment attrition was 22 of 96 (23%) because of smartphone incompatibility. Of the 74 participants who started using the VR-CBT app, 57 (77%) completed the intervention fully.
Conclusions and Relevance
A low-cost fully self-guided app-based virtual reality cognitive behavioral therapy with rudimentary virtual reality goggles can produce large acrophobia symptom reductions. To our knowledge, this study is the first to show that virtual reality acrophobia treatment can be done at home without the intervention of a therapist.
Trial Registration
Trialregister.nl identifier: NTR6442
Introduction
Access to evidence-based psychological treatment for mental health disorders is a global challenge because of high-treatment costs1 and the limited availability of mental health professionals.2 Treatment coverage is below 50% and usually substantially lower.2 Novel technologies may contribute to accessible and affordable treatment options in important ways.
Specific phobias rank among the most prevalent mental health disorders,3,4 of which acrophobia is the most prevalent of all subtypes.5,6 Worryingly, specific phobias may increase the risk of developing other anxiety disorders7 and major depression.8
Several evidence-based therapies exist, most of which use cognitive behavior therapy (CBT), involving exposing individuals to their feared object or situation.9 An emergent form of exposure is virtual reality exposure therapy (VRET), in which immersive virtual environments replace real-life exposure settings.10 Meta-analyses indicate VRET is as effective as conventional exposure therapy11,12,13 with large effect sizes11,12,13,14,15 and similar dropout and deterioration rates.11,12,16,17 However, VRET has thus far required therapist involvement and uses high-end virtual reality (VR) equipment.18
Recently, efforts toward reducing therapist involvement using VRET have been undertaken. A randomized clinical trial19 targeting acrophobia demonstrated a large effect size (d = 2.0) but still required expensive technology, and treatment was delivered in a clinic under the supervision of a therapist. Hence, the relative effect of the intervention itself remains unknown and costs are not reduced. A literature search (eAppendix 1 in Supplement 2) yielded no results on previous studies using a mobile app for acrophobia treatment (except for the present study’s protocol20).
The present study tested the effectiveness and user friendliness of ZeroPhobia, a fully self-guided VR CBT for acrophobia symptoms that is delivered through a smartphone. To ensure scalability, the VR-CBT app relies on participants’ own smartphone and basic ($10) cardboard VR goggles, while the program can be followed at home. We hypothesized that the app would be associated with greater overall response at posttest compared with a wait-list control group and that the treatment gains would be maintained at 3-month follow-up. For robustness, we included a second questionnaire assessing acrophobia symptoms. Depressive symptoms were examined to assess whether the VR-CBT app could also influence depression levels. We also tested whether variation in perceived user friendliness, general anxiety, and cyber sickness when using the app affected acrophobia symptoms at posttest.
Methods
Study Design and Procedure
In this single-blind randomized clinical trial, participants were recruited from the Dutch general population through websites, magazines, and local media. Ethical approval was received from the Medical Ethics Committee of the Vrije Universiteit Medical Center (the trial protocol has been previously published20 and is available in Supplement 1). Participants provided written informed consent by email or mail.
Participants
To be considered for inclusion, individuals had to score at least 45.45 on the Acrophobia Questionnaire (AQ)–Anxiety (1 SD below the mean of a previous acrophobic sample),21,22 have access to an Android smartphone (Android version 5.1 Lollipop or higher, 4.7- to 5.5-inch screen, and gyroscope), be aged 18 to 65 years, and have provided informed consent. Exclusion criteria were insufficient Dutch language skills, current phobia treatment or receiving psychotropic medication, or having severe depression (total score >19 on the Patient Health Questionnaire–9 item23) or suicidality (score ≥3 on the Web Screening Questionnaire24). Enrollment commenced March 24 and ceased September 28, 2017. Analysis began in April 2018.
Randomization and Masking
A randomization list was created with Random Allocation Software25 using block randomization of 6, 8, 10, and 12 blocks at an allocation ratio of 1:1. Participants were randomized into 2 groups: intervention or wait-list. The randomization list was kept by an independent researcher who revealed the next randomization outcome after every inclusion, thus ensuring that the research team was blind for treatment allocation. All materials were completed online without researcher intervention.
Intervention: VR-CBT App
Participants received 6 animated CBT-based modules using 2-dimensional animations and a voice-over provided by a virtual therapist. The modules took between 5 and 40 minutes to complete. Participants were asked to complete the entire intervention within 3 weeks. Aside from the psychoeducation and CBT techniques, the VR-CBT app included a gamified immersive VR environment and four 360° videos covering the entire exposure spectrum. The participants started using VR and 360° videos from module 3 onwards and navigated through the virtual environment using gaze control. Because previous research found better effects when the treatment was delivered twice a week compared with once a week,26 the 6 app modules were offered over a period of 3 weeks. However, participants could practice with the VR-CBT app as often as they wanted. During the 3-week period, participants received weekly standardized motivational emails with reminders to start or continue with the app (Donker et al20 includes intervention details; eFigure in Supplement 2 includes VR screen shots). The VR environment was created with the Unity game engine (version 2017.3.0f3; Unity Technologies).
Outcomes
All measures at baseline, posttest, and 3-month follow-up were completed online. The participants were asked for demographic information and at each assessment point, whether they received any other professional acrophobia treatment.
The primary outcome was the AQ.21 In a previous study, the mean (SD) AQ-Anxiety scores in a Dutch population were 59.06 (17.12).27 Secondary outcomes included the Attitudes Toward Heights Questionnaire,28 the Beck Anxiety Inventory,29 the System Usability Scale30 to assess user friendliness, the Igroup Presence Questionnaire31 to assess presence in VR, Mastery32 to assess subjective feelings of control, and the Patient Health Questionnaire–9 item23 to assess depressive symptoms. The use of professional treatment during the VR-CBT app, the 3-item Simulator Sickness Questionnaire (SSQ),33,34 and vision impairment (eg, wearing glasses) were included as control variables. The SSQ is commonly used to assess simulator sickness but not in the context of anxiety treatment; the items completely overlap with anxiety symptoms (eg, nausea, dizziness).19 We used a simple raw score of whether the symptom was present or not. All assessments were programmed with Survalyzer software.35
Study Power
The primary outcome measure, the AQ, was used for the power calculations. In previous meta-analyses in which VRET vs a wait-list control group was compared using the AQ, effect sizes ranged from 0.93 to 1.23.12,14,15 However, the unguided nature of the VR-CBT app, the use of low-end equipment, and the lack of comparable studies led us to opt for a conservative estimate of Cohen d = 0.50. With 80% power at a 5% significance level (2-sided), 64 participants in each condition were required (128 participants in total). Because of the high dropout rates often encountered in self-guided interventions,36 we estimated a 40% dropout rate, meaning 180 participants were required.20
Statistical Analyses
Demographic and clinical characteristics were compared between the intervention and the wait-list control groups, with χ2 or variance analysis as appropriate. To assess whether pretreatment attrition (participants who failed to commence treatment after randomization37) and dropout (participants who discontinued treatment after commencing treatment37) were nonrandom, a balancing table was constructed comparing background characteristics, prescores, and other covariates between participants with and without missing outcome observations. Missing outcome observations were accounted for by using a single best regression-imputed value. Missing outcome values for the pretreatment attrition sample were imputed using the wait-list control sample, whereas for the dropout sample initial treatment assignment was used. The empirical evaluation was performed on an intention-to-treat (ITT) basis, using ordinary least squares regression models with prescores and background characteristics included. Standardized effect sizes (Cohen d) and confidence intervals were calculated for each end point.
Two additional robustness analyses were performed. First, because nonrandom sample attrition may bias the estimated treatment effects, nonparametric treatment effect bounds were estimated.38 Second, multiple imputation using an iterative Markov chain Monte Carlo method based on initial treatment assignment was performed to infer the statistical significance of the reported differences between the intervention and control groups. Furthermore, potential heterogeneity and mechanisms of effective treatment were addressed by estimating how treatment effects vary with prescores, general anxiety, and experiences when using the app. We also analyzed the predicted reductions in the posttest for 10-point bins of prescores using regression analyses to examine whether severity at baseline AQ predicts outcome. Sensitivity analyses using participants who completed posttest or follow-up after the intervention examined whether there was a difference in the results compared with the ITT analysis.39 We exploratively assessed clinically meaningful change on the AQ using the clinically significant change formula (method C40) with an AQ score less than 31.67.41 The reliable change criterion was assessed based on the AQ pretest SD scores with a Cronbach α of .91. This yielded an SE of change of 7.98, with a corresponding reliable change criterion on the AQ score of 15.65 (7.98 × 1.96).42,43 We also assessed the number needed to treat using the formula by Furukawa et al44 based on the effect size of the AQ. Two-sided P value less than .05 indicated statistical significance. Stata version 14.2 (StataCorp) was used for the analyses except for number needed to treat, which was conducted in R version 3.4.3 (R Project for Statistical Computing). A data monitoring committee was not required by the ethics committee because of the expected low safety risk of the participants.
Deviations From the Original Protocol
We deviated on some issues from the ethics protocol (Supplement 1). For details, see eAppendix 2 in Supplement 2.
Results
Sample Characteristics
Of 663 individuals who signed up for participation, 291 were ineligible and excluded from participation. In total, 193 participants filled in the baseline assessment and were randomly assigned to the VR-CBT app (n = 96) or wait-list control (n = 97). The pretreatment attrition rate was 23% in the app condition because of illness (1 [1%]) or an incompatible smartphone (21 [22%]). Several participants with smartphones lacking a gyroscope (required for experiencing VR) were erroneously included. To ensure that this did not jeopardize randomization, these participants were included in the ITT analyses, and Lee bounds38 were estimated to address potential bias. Figure 1 shows the participant flowchart.
Figure 1. Flow of Participants.
AQ indicates Acrophobia Questionnaire; PHQ, Patient Health Questionnaire.
Baseline demographics are provided in Table 1. Data were balanced between the groups. None of the participants reported receiving professional acrophobia treatment at baseline, posttest, or follow-up.
Table 1. Demographic and Baseline Characteristics.
| Variable | VR CBT App (n = 96) | Wait-list Control Group (n = 97) |
|---|---|---|
| Age, mean (SD), y | 41.53 (13.73) | 41.12 (13.62) |
| Female, No. (%) | 66 (68.75) | 63 (64.95) |
| Education, No. (%) | ||
| None or primary | 2 (2.08) | 0 (0) |
| Secondary | 10 (10.42) | 8 (8.25) |
| Postsecondary | 84 (87.50) | 89 (91.75) |
| Psychotropic medication,a No. (%) | 1 (1.04) | 4 (4.12) |
| Primary outcome, mean (SD) | Baseline | Baseline |
| AQ-Total | 85.16 (18.42) | 84.18 (18.25) |
| AQ-Anxiety | 68.62 (14.53) | 67.91 (14.44) |
| AQ-Avoidance | 16.54 (5.19) | 16.27 (4.83) |
| Secondary outcomes, mean (SD) | ||
| ATHQ | 44.89 (8.72) | 44.47 (9.64) |
| BAI | 32.00 (18.69) | 33.74 (18.00) |
| Masteryb | 25.56 (7.59) | 26.66 (5.81) |
| PHQ | 2.29 (3.31) | 2.23 (3.01) |
Abbreviations: AQ, Acrophobia Questionnaire; ATHQ, Attention to Height Questionnaire; BAI, Beck Anxiety Inventory; PHQ, Patient Health Questionnaire; VR CBT, virtual reality cognitive behavioral therapy.
Missing 1 observation.
A higher score means a higher sense of mastery.
Treatment Adherence and Attrition
There were no significant differences in the demographics between participants (n = 193) and those who were ineligible or chose not to participate (n = 346) (data not shown). In the intervention group, 59% (57 of 96) of the participants completed the posttest and 49% (47 of 96) completed follow-up, compared with 91% (87 of 97) posttest responses for the wait-list group. Attrition (22 of 96 [23%]) and dropout (17 of 96 [18%]) were not related to background characteristics, prescores, and other covariates (eTable 1 in Supplement 2), as indicated by the likelihood ratio test for predicting missing outcomes (χ2 (9) = 3.47; P = .94), supporting the assumption that outcomes are missing completely at random.45
Primary Outcome
The intervention condition showed a significant reduction in acrophobia symptoms compared with the control on the AQ in the ITT analysis at posttest (b = −26.73 [95% CI, −32.12 to −21.34]; t191 = −9.79; P < .001; adjusted R2 = 0.52) with an effect size of d = 1.14 (95% CI, 0.84-1.44). The number needed to treat was 1.7 (Table 2 and Table 3).
Table 2. Inferential Statistics of Treatment Outcome Measures, Intention to Treat (n = 193).
| Outcomes | Posttest | |
|---|---|---|
| VR CBT App (n = 96) | Wait-list Control Group (n = 97) | |
| Primary, mean (SD) | ||
| AQ-Total | 48.46 (24.34) | 74.69 (21.55) |
| AQ-Anxiety | 39.55 (19.12) | 60.41 (17.02) |
| AQ-Avoidance | 8.62 (5.73) | 14.19 (5.13) |
| Secondary, mean (SD) | ||
| ATHQ | 32.83 (12.71) | 45.16 (9.73) |
| BAI | 33.56 (8.98) | 37.93 (13.95) |
| Masterya | 28.02 (4.10) | 27.54 (4.86) |
| PHQ | 2.46 (2.34) | 2.87 (3.48) |
Abbreviations: AQ, Acrophobia Questionnaire; ATHQ, Attention to Height Questionnaire; BAI, Beck Anxiety Inventory; PHQ, Patient Health Questionnaire; VR CBT, virtual realtiy cognitive behavioral therapy.
A higher score means a higher sense of mastery.
Table 3. Virtual Reality Cognitive Behavioral Therapy App Treatment Effect Estimates.
| Measure | Posttesta | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Model 1 (n = 193) | Model 2 (n = 134-142) | Model 3 (n = 193) | |||||||
| b ITT (SE) | Adjusted R | P Valuea | Effect Size (95% CI) | b Lower Bound (SE) | b Upper Bound (SE) | b MI (SE) | P Valueb | Effect Size (95% CI) | |
| Primary outcome | |||||||||
| AQ total | –26.7 (2.73) | 0.52 | <.001 | 1.14 (0.84 to 1.44) | –47.50 (4.80) | –22.34 (4.55) | –36.54 (3.29) | <.001 | 1.53 (1.15 to 1.91) |
| Secondary outcomes | |||||||||
| ATHQ | –12.59 (1.41) | 0.46 | <.001 | 1.09 (0.79 to 1.39) | –22.43 (2.05) | –12.06 (2.46) | –16.07 (2.57) | <.001 | 1.50 (1.12 to 1.88) |
| BAI | –3.87 (1.48) | 0.32 | .001 | 0.37 (0.09 to 0.66) | –15.07 (3.10) | 3.03 (2.18) | –6.45 (2.06) | .002 | 0.51 (0.15 to 0.86) |
| Mastery | 1.01 (0.50) | 0.44 | .047 | –0.11 (–0.39 to 0.8) | –2.21 (0.92) | 3.87 (1.05) | 0.77 (0.86) | .37 | –0.20 (–0.54 to 0.15) |
| PHQ | –0.49 (0.31) | 0.51 | .12 | 0.14 (–0.14 to 0.42) | –2.46 (0.77) | 1.15 (0.46) | –0.58 (0.61) | .35 | 0.27 (–0.07 to 0.62) |
Abbreviations: AQ, Acrophobia Questionnaire; ATHQ, Attention to Height Questionnaire; BAI, Beck Anxiety Inventory; MI, Multiple Imputation; PHQ, Patient Health Questionnaire.
Because of ethical reasons, we do not have data on the follow-up for the wait-list control group participants because they were given access to the app after the posttest. Therefore, we could not analyze the follow-up results for the intent-to-treat sample.
Two-sided. Treatment effects reported in the preferred conservative model 1 are intention-to-treat effects, using dual-strategy regression imputation based on the source of missing data. Treatment effects reported in model 2 are Lee bounds estimates. Treatment effects in model 3 are derived through multiple (20) imputations, assuming missing at random, applying multivariate normal regression, imputation by experimental group, and accommodating arbitrary missing outcome value patterns using iterative Markov chain Monte Carlo. Prescore and background controls are included for improved precision of the regression point estimates reported in models 1 and 3. Background control variables are sex, age, education level, and a dummy for whether a participant had missing values on 1 or more background characteristics. Effect sizes reported in model 1 and 3 are all Cohen d for unadjusted group means based on the premeasure-postmeasure difference.
Secondary Outcomes
The results of the ITT analyses (Table 3) demonstrated a significant intervention effect compared with the controls on acrophobia symptoms, Attention to Height Questionnaire (b = −12.59; t182 = −8.92; P < .001; d = 1.091; 95% CI, 0.787-1.393), general anxiety symptoms, Beck Anxiety Inventory (b = −3.87; t182 = −2.62; P < .001; d = 0.37; 95% CI, 0.087-656), and a greater sense of mastery (b = 1.01; t182 = −2.00; P = .047; d = −0.11; 95% CI, −0.389 to 0.176). Depressive symptoms did not change significantly between both groups, Patient Health Questionnaire–9 item (b = −0.49; t182 = −1.58; P ≥ .99; d = 0.14; 95% CI, −0.143 to 0.422).
Robustness and Sensitivity Analyses
When estimating the Lee bounds,38 the resulting upper and lower bounds (model 2) indicated that the treatment effect on the primary outcome measure (AQ) remained statistically significant. The ITT estimate in model 1 (b = −26.73, SE = 2.73) was close to the conservative upper Lee bound (b = −22.34, SE = 4.55) after imputing outcomes for the attrition sample using wait-list control participants. Applying multiple imputation techniques for the attrition and dropout sample, based on initial treatment assignment yielded somewhat bigger SEs (model 3), but statistical significance of the ITT effect on AQ was maintained (d = 1.53; 95% CI, 1.15-1.91; Table 3).
Heterogeneous Treatment Effects and Mechanisms
Adding an interaction term to the ITT treatment estimation model of Table 3 revealed that treatment effectiveness related positively to baseline acrophobia symptoms (ie, as a modifier) (b = –0.306; SE = 0.15; P = .04), indicating higher potential benefits for participants with more severe acrophobia symptoms (eTable 3 in Supplement 2).
The experience-related results indicate that a reduction in acrophobia symptoms tends to be larger when the usability of the app was higher (System Usability Scale), when the feeling of being present in the virtual environment was higher (Igroup Presence Questionnaire), and when participants experienced more symptoms of cyber sickness (SSQ) (Figure 2). The robustness analysis further confirmed that the VR-CBT app had an impact on the anxiety for heights and that the general anxiety effect did not drive the results (eTable 4 in Supplement 2). eTable 5 in Supplement 2 provides regression-predicted reductions in posttest for 10-point bins of prescores suggesting that moderation is quantitative. Those with relatively low levels of baseline acrophobia did benefit significantly from the treatment.
Figure 2. Heterogeneous Treatment Effects (n = 193) With Respect to Initial Acrophobia Score and Self-reported Experience Mechanisms (n = 52-55).
Complete Cases
For individuals who returned the posttest, the between-group AQ effect size was d = 1.53 (95% CI, 1.15-1.91; number needed to treat: 1.4), and for the app individuals, the within-group effect size was d = 2.68 (95% CI, 2.09-3.22) between baseline and follow-up. Effect sizes on secondary outcomes for app individuals were also large (eTable 2 in Supplement 2). The VR-CBT app participants’ total mean (SD) Igroup Presence Questionnaire score was 42.69 (10.40) (range, 14-70).
Exploratory Analysis
For individuals who returned the AQ posttest (n = 56), all participants showed reliable change. Furthermore, 44 of 56 (79%) experienced clinically significant change, a change of 57.97 or more points on the AQ.
User Friendliness and Adverse Effects
The VR-CBT app was rated as user friendly (mean [SD], 75.35 [14.74], n = 55) and can be interpreted as a good and usable system.22 No deterioration or negative effects as defined by Rozental et al46 were identified, except for 24 participants who reported 1 or more symptoms of transient cyber sickness.
Discussion
To our knowledge, this randomized clinical trial tests the first fully self-guided treatment for acrophobia. We showed that an app-based VR-CBT program using low-cost VR goggles is both effective and user friendly. Large reductions in acrophobia symptoms on both the self-reported primary outcome measure AQ and secondary outcome measure Attention to Height Questionnaire were obtained. At 3-month follow-up, the results were maintained for the intervention group as demonstrated by the large within-group effect size. Importantly, these findings are in line with previous meta-analyses of guided VR acrophobia studies using high-end VR equipment12,14,15 and similar or better compared with therapist-guided exposure in vivo.47,48 In the current study, dropout rates were similar compared with dropout rates in previous VR studies.16 The app participants also experienced a greater reduction in general anxiety and an increased sense of mastery compared with the controls. As expected, because the baseline level of depression was low, the symptoms did not decrease; hence, there were no between-group differences. App participants perceived the VR environment as realistic although these results are slightly lower compared with previous research.49 This might be because of the quality of the cardboard VR goggles. However, the VR-CBT app was rated as user friendly and a good and usable system. Interestingly, the participants who experienced more cyber sickness also experienced a larger reduction in acrophobia symptoms. This might be an artifact of the SSQ because it overlaps with symptoms of anxiety. For exposure to be beneficial, experiencing anxiety is required. Therefore, the SSQ might not be a valid measure of cyber sickness for treating anxiety.
Strengths and Limitations
One of the key strengths of the present study was its ecological validity because the intervention was conducted in the participants’ natural environments instead of a research laboratory. Furthermore, participants received no guidance when using the VR-CBT app or filling in assessments, thereby ruling out any influence of human contact. Another strength was that the missing outcome observations for participants assigned to treatment to the VR-CBT app but unable to use the app because of incompatible smartphones were based on wait-list control group characteristics (ie, no treatment effect was imputed), yielding conservative overall treatment effect estimates. Furthermore, the ITT design has strong credibility in terms of aggregate effects when the intervention is scaled and adopted in real life.
The current study had several limitations. First, attrition was relatively high in the intervention condition because of incompatible smartphones. However, attrition was statistically unrelated to all observable characteristics at baseline. To retain a balanced experimental sample, the results were imputed for missing participants using regression-based imputation. Robustness and sensitivity analyses confirmed that potential bias concerns and precision concerns did not compromise the statistical significance of the results. Second, the data rely exclusively on self-reported measurements. However, the behavioral measurement effect sizes were similar to those calculated from other self-reported measures.13 Because the AQ has no validated cutoff scores, we were unable to provide results on the clinical threshold for acrophobia. We did not include a diagnostic interview because we wanted to isolate the true effect of VR; even in the assessment phase, human involvement may already have influenced the results.50,51 Third, because our follow-up period was only 3 months and because of dropout, the results are exploratory, and longer-term effects remain unknown. Fourth, the generalizability of the results is limited to Dutch Android smartphone users. However, most of the Dutch population (57%) uses Android.52
Future research is needed to replicate the VR-CBT app to draw firm conclusions about its effectiveness, compare the app directly with high-end VRET and exposure in vivo treatment, examine its long-term effects, determine cost effectiveness compared with treatment as usual, and examine whether the app can also be effective in the treatment of other phobias and mental health disorders.
Conclusions
In sum, our findings support the hypothesis that a fully self-guided app-based VR-CBT, which can be done at home at a fraction of the cost of existing evidence-based treatment options, strongly reduces acrophobia symptoms. The current study adds to the development of innovative and scalable delivery methods of evidence-based treatments and underlines that new technologies have the potential to transform mental health care worldwide.
Trial protocol.
eAppendix 1. Literature search for app-based acrophobia treatment
eAppendix 2. Deviations from the original trial protocol submitted to the ethic committee and the research protocol
eTable 1. Comparison of missing outcome participants to non-missing outcome participants after randomisation
eTable 2. Inferential statistics of treatment outcome measures, complete cases (n=134-144)
eTable 3. Heterogeneity of treatment effects of ZeroPhobia with respect to baseline acrophobia symptoms (n=193)
eTable 4. Estimation results when covarying for potential general anxiety symptoms (n=193)
eTable 5. The regression-predicted reductions in post-test for 10-point bins of pre-scores
eFigure. Virtual reality screen shots
Data sharing statement
References
- 1.Kessler RC, Berglund PA, Bruce ML, et al. The prevalence and correlates of untreated serious mental illness. Health Serv Res. 2001;36(6 pt 1):987-1007. [PMC free article] [PubMed] [Google Scholar]
- 2.Fairburn CG, Patel V. The impact of digital technology on psychological treatments and their dissemination. Behav Res Ther. 2017;88:19-25. doi: 10.1016/j.brat.2016.08.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.de Graaf R, ten Have M, van Gool C, van Dorsselaer S. Prevalence of mental disorders and trends from 1996 to 2009: results from the Netherlands Mental Health Survey and Incidence Study-2. Soc Psychiatry Psychiatr Epidemiol. 2012;47(2):203-213. doi: 10.1007/s00127-010-0334-8 [DOI] [PubMed] [Google Scholar]
- 4.Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, Wittchen HU. Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. Int J Methods Psychiatr Res. 2012;21(3):169-184. doi: 10.1002/mpr.1359 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Depla MF, ten Have ML, van Balkom AJ, de Graaf R. Specific fears and phobias in the general population: results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Soc Psychiatry Psychiatr Epidemiol. 2008;43(3):200-208. doi: 10.1007/s00127-007-0291-z [DOI] [PubMed] [Google Scholar]
- 6.Stinson FS, Dawson DA, Patricia Chou S, et al. The epidemiology of DSM-IV specific phobia in the USA: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Psychol Med. 2007;37(7):1047-1059. doi: 10.1017/S0033291707000086 [DOI] [PubMed] [Google Scholar]
- 7.Trumpf J, Margraf J, Vriends N, Meyer AH, Becker ES. Specific phobia predicts psychopathology in young women. Soc Psychiatry Psychiatr Epidemiol. 2010;45(12):1161-1166. doi: 10.1007/s00127-009-0159-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Choy Y, Fyer AJ, Goodwin RD. Specific phobia and comorbid depression: a closer look at the National Comorbidity Survey data. Compr Psychiatry. 2007;48(2):132-136. doi: 10.1016/j.comppsych.2006.10.010 [DOI] [PubMed] [Google Scholar]
- 9.Abramowitz JS. The practice of exposure therapy: relevance of cognitive-behavioral theory and extinction theory. Behav Ther. 2013;44(4):548-558. doi: 10.1016/j.beth.2013.03.003 [DOI] [PubMed] [Google Scholar]
- 10.Slater M, Sanchez-Vives MV. Enhancing our lives with immersive virtual reality. Frontiers Robotics AI. 2016;3:76. doi: 10.3389/frobt.2016.00074 [DOI] [Google Scholar]
- 11.Fodor LA, Coteț CD, Cuijpers P, Szamoskozi Ș, David D, Cristea IA. The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: a meta-analysis. Sci Rep. 2018;8(1):10323. doi: 10.1038/s41598-018-28113-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Opriş D, Pintea S, García-Palacios A, Botella C, Szamosközi Ş, David D. Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis. Depress Anxiety. 2012;29(2):85-93. doi: 10.1002/da.20910 [DOI] [PubMed] [Google Scholar]
- 13.Morina N, Ijntema H, Meyerbröker K, Emmelkamp PM. Can virtual reality exposure therapy gains be generalized to real-life? a meta-analysis of studies applying behavioral assessments. Behav Res Ther. 2015;74:18-24. doi: 10.1016/j.brat.2015.08.010 [DOI] [PubMed] [Google Scholar]
- 14.Carl E, Stein AT, Levihn-Coon A, et al. Virtual reality exposure therapy for anxiety and related disorders: a meta-analysis of randomized controlled trials. J Anxiety Disord. 2019;61:27-36. doi: 10.1016/j.janxdis.2018.08.003 [DOI] [PubMed] [Google Scholar]
- 15.Parsons TD, Rizzo AA. Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. J Behav Ther Exp Psychiatry. 2008;39(3):250-261. doi: 10.1016/j.jbtep.2007.07.007 [DOI] [PubMed] [Google Scholar]
- 16.Benbow AA, Anderson PL. A meta-analytic examination of attrition in virtual reality exposure therapy for anxiety disorders. J Anxiety Disord. 2019;61:18-26. doi: 10.1016/j.janxdis.2018.06.006 [DOI] [PubMed] [Google Scholar]
- 17.Fernández-Álvarez J, Rozental A, Carlbring P, et al. Deterioration rates in virtual reality therapy: an individual patient data level meta-analysis. J Anxiety Disord. 2019;61:3-17. doi: 10.1016/j.janxdis.2018.06.005 [DOI] [PubMed] [Google Scholar]
- 18.Lindner P, Miloff A, Hamilton W, et al. Creating state of the art, next-generation virtual reality exposure therapies for anxiety disorders using consumer hardware platforms: design considerations and future directions. Cogn Behav Ther. 2017;46(5):404-420. doi: 10.1080/16506073.2017.1280843 [DOI] [PubMed] [Google Scholar]
- 19.Freeman D, Haselton P, Freeman J, et al. Automated psychological therapy using immersive virtual reality for treatment of fear of heights: a single-blind, parallel-group, randomised controlled trial. Lancet Psychiatry. 2018;5(8):625-632. doi: 10.1016/S2215-0366(18)30226-8-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Donker T, Van Esveld S, Fischer N, Van Straten A. 0Phobia: towards a virtual cure for acrophobia: study protocol for a randomized controlled trial. Trials. 2018;19(1):433. doi: 10.1186/s13063-018-2704-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Cohen DC. Comparison of self-report and overt-behavioral procedures for assessing acrophobia. Behav Ther. 1977;8(1):17-23. doi: 10.1016/S0005-7894(77)80116-0 [DOI] [Google Scholar]
- 22.Steinman SA, Teachman BA. Cognitive processing and acrophobia: validating the Heights Interpretation Questionnaire. J Anxiety Disord. 2011;25(7):896-902 .doi: 10.1016/j.janxdis.2011.05.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606-613. doi: 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Donker T, van Straten A, Marks I, Cuijpers P. A brief Web-based screening questionnaire for common mental disorders: development and validation. J Med Internet Res. 2009;11(3):e19. doi: 10.2196/jmir.1134 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Saghaei M. Random allocation software for parallel group randomized trials. BMC Med Res Methodol. 2004;4:26. doi: 10.1186/1471-2288-4-26 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Cuijpers P, Huibers M, Ebert DD, Koole SL, Andersson G. How much psychotherapy is needed to treat depression? a metaregression analysis. J Affect Disord. 2013;149(1-3):1-13. doi: 10.1016/j.jad.2013.02.030 [DOI] [PubMed] [Google Scholar]
- 27.Emmelkamp PM, Krijn M, Hulsbosch AM, de Vries S, Schuemie MJ, van der Mast CA. Virtual reality treatment versus exposure in vivo: a comparative evaluation in acrophobia. Behav Res Ther. 2002;40(5):509-516. doi: 10.1016/S0005-7967(01)00023-7 [DOI] [PubMed] [Google Scholar]
- 28.Abelson JL, Curtis GC. Cardiac and neuroendocrine responses to exposure therapy in height phobics: desynchrony within the ‘physiological response system’. Behav Res Ther. 1989;27(5):561-567. doi: 10.1016/0005-7967(89)90091-0 [DOI] [PubMed] [Google Scholar]
- 29.Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56(6):893-897. doi: 10.1037/0022-006X.56.6.893 [DOI] [PubMed] [Google Scholar]
- 30.Bangor A. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24(6):574-594. doi: 10.1080/10447310802205776 [DOI] [Google Scholar]
- 31.Schubert T, Friedmann F, Regenbrecht H. The experience of presence: factor analytic insights. Presence Teleop Virt. 2001;10(3):266-281. doi: 10.1162/105474601300343603 [DOI] [Google Scholar]
- 32.Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav. 1978;19(1):2-21. doi: 10.2307/2136319 [DOI] [PubMed] [Google Scholar]
- 33.LaViola JJ., Jr A discussion of cybersickness in virtual environments. ACM SIGCHI Bullet. 2000;32(1):47-56. doi: 10.1145/333329.333344 [DOI] [Google Scholar]
- 34.Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG. Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol. 1993;3(3):203-220. doi: 10.1207/s15327108ijap0303_3 [DOI] [Google Scholar]
- 35.Survalyzer Nederland BV. Utrecht, the Netherlands. http://www.survalyzer.com. Accessed Augst 27, 2018.
- 36.Donker T, Bennett K, Bennett A, et al. Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: randomized controlled noninferiority trial. J Med Internet Res. 2013;15(5):e82. doi: 10.2196/jmir.2307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Issakidis C, Andrews G. Pretreatment attrition and dropout in an outpatient clinic for anxiety disorders. Acta Psychiatr Scand. 2004;109(6):426-433. doi: 10.1111/j.1600-0047.2004.00264.x [DOI] [PubMed] [Google Scholar]
- 38.Lee DS. Training, wages, and sample selection: estimating sharp bounds on treatment effects. Rev Econ Stud. 2009;76(3):1071-1102. doi: 10.1111/j.1467-937X.2009.00536.x [DOI] [Google Scholar]
- 39.Karyotaki E, Riper H, Twisk J, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA Psychiatry. 2017;74(4):351-359. doi: 10.1001/jamapsychiatry.2017.0044 [DOI] [PubMed] [Google Scholar]
- 40.Reliable and clinically significant change. Mounted by Chris Evans, 1998. http://www.psyctc.org/stats/rcsc.htm. Accessed August 18, 2018.
- 41.Cohen DC. Personality predictors and the outcome of varieties of desensitization. Unpublished doctoral dissertation, Harvard University, Cambridge, MA; 1972. [Google Scholar]
- 42.Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991;59(1):12-19. doi: 10.1037/0022-006X.59.1.12 [DOI] [PubMed] [Google Scholar]
- 43.Christensen L, Mendoza JL. A method of assessing change in a single subject: an alteration of the RC index. Behav Ther. 1986;17(3):305-308. doi: 10.1016/S0005-7894(86)80060-0 [DOI] [Google Scholar]
- 44.Furukawa TA, Leucht S. How to obtain NNT from Cohen’s d: comparison of two methods. PLoS One. 2011;6(4):e19070. doi: 10.1371/journal.pone.0019070 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Pedersen AB, Mikkelsen EM, Cronin-Fenton D, et al. Missing data and multiple imputation in clinical epidemiological research. Clin Epidemiol. 2017;9:157-166. doi: 10.2147/CLEP.S129785 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Rozental A, Castonguay L, Dimidjian S, et al. Negative effects in psychotherapy: commentary and recommendations for future research and clinical practice. BJPsych Open. 2018;4(4):307-312. doi: 10.1192/bjo.2018.42 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Arroll B, Wallace HB, Mount V, Humm SP, Kingsford DW. A systematic review and meta-analysis of treatments for acrophobia. Med J Aust. 2017;206(6):263-267. doi: 10.5694/mja16.00540 [DOI] [PubMed] [Google Scholar]
- 48.Wolitzky-Taylor KB, Horowitz JD, Powers MB, Telch MJ. Psychological approaches in the treatment of specific phobias: a meta-analysis. Clin Psychol Rev. 2008;28(6):1021-1037. doi: 10.1016/j.cpr.2008.02.007 [DOI] [PubMed] [Google Scholar]
- 49.Krijn M, Emmelkamp PMG, Biemond R, de Wilde de Ligny C, Schuemie MJ, van der Mast CA. Treatment of acrophobia in virtual reality: the role of immersion and presence. Behav Res Ther. 2004;42(2):229-239. doi: 10.1016/S0005-7967(03)00139-6 [DOI] [PubMed] [Google Scholar]
- 50.Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry. 2014;13(1):4-11. doi: 10.1002/wps.20083 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Buntrock C, Ebert DD, Lehr D, et al. Effect of a Web-based guided self-help intervention for prevention of major depression in adults with subthreshold depression: a randomized clinical trial. JAMA. 2016;315(17):1854-1863. doi: 10.1001/jama.2016.4326 [DOI] [PubMed] [Google Scholar]
- 52.Statcounter GlobalStats Mobile operating system market share Netherlands. Oct 2017- Oct 2018. http://gs.statcounter.com/os-market-share/mobile/netherlands. Accessed November 29, 2018.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Trial protocol.
eAppendix 1. Literature search for app-based acrophobia treatment
eAppendix 2. Deviations from the original trial protocol submitted to the ethic committee and the research protocol
eTable 1. Comparison of missing outcome participants to non-missing outcome participants after randomisation
eTable 2. Inferential statistics of treatment outcome measures, complete cases (n=134-144)
eTable 3. Heterogeneity of treatment effects of ZeroPhobia with respect to baseline acrophobia symptoms (n=193)
eTable 4. Estimation results when covarying for potential general anxiety symptoms (n=193)
eTable 5. The regression-predicted reductions in post-test for 10-point bins of pre-scores
eFigure. Virtual reality screen shots
Data sharing statement