Abstract
Introduction:
Traditional physical therapy (PT) requires patients to attend weekly in-office supervised physical therapy appointments. However, between 50% and 70% of patients who would benefit do not receive prescribed PT due to barriers to access. Virtual Reality (VR) provides a platform for remote delivery of PT to address these access barriers.
Methods:
We developed a VR-PT program consisting of training, games, and a progress dashboard for 3 common lower extremity physical therapy exercises. We enrolled orthopaedic trauma patients with lower extremity injuries. Patients completed a VR-PT session, consisting of training and one of the exercise-based games. Pre- and post-VR-PT questionnaires were completed.
Results:
We enrolled 15 patients with an average age of 51 years. Fourteen patients said they would enroll in a randomized trial in which they had a 50% chance of receiving VR-PT vs receiving standard of care. When asked to rate their experience using the VR-PT module on a scale from 0-10—with 0 being anchored as “I hated it” and 10 being anchored as “I loved it”—the average rating was 7.5. Patients rated the acceptability of VR-PT as a 3.9 out of 5, the feasibility as a 4.0 out of 5, and the usability as a 67.5 out of 100.
Conclusion:
The response to VR-PT in this pilot study was positive overall. A VR-based PT program may add value for both patients and clinicians in terms of objective data collection (to aid in compliance monitoring, progression toward goals and exercise safety), increased engagement and increased access.
Keywords: acceptability, feasibility, lower extremity injuries, physical therapy, rehabilitation, virtual reality
Introduction
Extremity injuries account for the majority of injuries sustained both in combat and in civilian trauma with approximately 15 million fractures per year in the United States.[1–5] Long-term impairments are often substantial with research demonstrating moderate to severe levels of disability, low rates of return to employment, and chronic pain up to 7 years after the injury.[6–10]
Physical therapy positively affects functional recovery after traumatic injury.[1,11–13] Patients unable to participate in prescribed physical therapy experience significantly less functional improvement compared with those who complete physical therapy.[13] Furthermore, in patients with traumatic amputations, a focused rehabilitation protocol results in substantial functional benefit and improved rate of return to work or duty.[4,11,12,14] Physical therapy also has a favorable impact on pain in patients with painful musculoskeletal conditions.[15–18]
Despite the established positive effect of physical therapy (PT) on functional recovery after traumatic injury, between 50% and 70% of patients who would benefit do not receive therapy due to access barriers.[13,19] During the current COVID-19 pandemic, this number is likely substantially higher due to social distancing-associated restrictions that have limited in-person physical therapy. The most frequently utilized means of administering physical therapy is with supervised clinic-based therapy. However, this is extremely resource intensive and is associated with many barriers both in military and civilian populations including high cost (for patients with and without health insurance), travel, time away from work or caregiver duties, and limited access in rural areas and internationally.[13,19,20] These barriers appear to result in substantial under-utilization of physical therapy among lower and upper extremity trauma patients.[13,19] These financial and access barriers are particularly salient for individuals with low socioeconomic status or lack of health insurance who are overrepresented among patients with unmet physical therapy need and who are at highest risk for poor functional and pain-related outcomes.[19,21–29]
As digital and communication technologies have improved, several iterations of telemedicine have been implemented to address access issues in several fields of medicine and rehabilitation.[30–39] Immersive virtual reality (VR) is an emerging technology that has potential to address many of the barriers associated with supervised clinic-based physical therapy. Specifically, the goal of VR is to give patients the illusion that they are inside a 3D computer-generated world. Motion and pressure sensors can be used to monitor movement and weight-bearing as well as provide real-time feedback on patients’ accuracy at performing tasks and success in achieving goals. VR may provide an ideal platform for remote delivery of physical therapy with opportunities for both in-program instruction and remote consultation with a physical therapist. The VR platform can take advantage of opportunities to improve engagement through use of goal-oriented “games” designed around physical therapy protocols. Given the high cost associated with supervised in-office physical therapy, particularly at the intervals at which it is administered (often 2–3 times per week), transition of patients to in-home VR-based physical therapy presents an enormous opportunity for cost savings (both direct cost of therapy as well as indirect costs associated with travel time and lost work for patients and their caretakers). Moreover, VR-based physical therapy has the additional benefits of: improved access through a home-based approach, ease of use, improved engagement and patient adherence through “gamification,” increased uniformity in interventions administered, the ability to remotely monitor compliance and progress with therapy, and identify potential problems.
The purpose of the present pilot study is to establish feasibility and patient acceptability. Our long-term goal is to establish equivalent functional and pain-related outcomes as well as improved cost-effectiveness and patient compliance with an interactive in-home VR-based physical therapy platform as opposed to supervised physical therapy in musculoskeletal trauma patients.
Methods
We obtained approval from our local institutional review board (IRB) prior to conducting this research.
Virtual reality (VR)-based physical therapy platform
We contracted KairosXR[40] to collaborate on the development of a virtual reality-based physical therapy (VR-PT) program consisting of training, games, and a progress dashboard for three common lower extremity physical therapy exercises: squat, heel slide, and leg raise (Figs. 1 and 2). The program was designed with input from Orthopaedic Surgeons, Physical Therapists, and VR developers around 4 critical components: data collection; patient instruction; safety; entertainment. The VR-based physical therapy application was designed to orient patients with no prior experience using VR technology and teach proper technique for performing exercises using a narrator and on-screen indications. Motion tracking “pucks” were used to collect data on relative success at exercise and to ensure correct and safe execution of exercises. Patients were alerted when a motion exceeded predetermined range of acceptability. For each exercise there was a training module with detailed instruction and feedback regarding accuracy of performing the exercise as well as a game module with difficulty settings and points to be earned (Fig. 1). A summary screen with achievement data was reported at the end of each activity (Fig. 2).
Patient population
Inclusion criteria: adult patients with operative fractures of the femur or tibia (to include plateau, pilon, and ankle injuries) presenting to the orthopaedic surgeon for either acute care or for follow-up of surgical fixation performed elsewhere within 14 days of the injury who would be referred to physical therapy as standard of care.
Exclusion criteria: patients with ISS > 18, bilateral lower extremity injuries that preclude crutch ambulation, associated spine fractures, GCS < 15 at the time of discharge, major peripheral nerve injury or planned admission to a skilled nursing facility or inpatient rehabilitation facility.
Study procedures
Eligible patients were approached for enrollment during their postoperative visits and informed consent was obtained. Patients completed an entire VR-PT session in a single clinic visit. Sessions began with a presurvey about previous experiences with both physical therapy and virtual reality, as well as demographic information, training, and participating in one of the exercise-based games. Participants then began the VR-PT program, consisting of an orientation to the user interface, a training session to teach proper exercise form, and a virtual game using the same exercise to complete a physical therapy objective. Patients were monitored throughout this session by a research coordinator. After completing the VR-PT program, patients completed a survey about their experience with VR-PT. The postsurvey contained 3 validated implementation outcome measures—the Acceptability of Intervention Measure (AIM),[41] the Feasibility of Intervention Measure (FIM),[41] and the System Usability Scale (SUS).[42]
Results
Fifteen patients agreed to participate in this pilot study (Table 1) and 2 patients declined participation. The 2 patients who declined participation reported time constraints. All 15 patients had prior experience with physical therapy; 14 had experience with physical therapy to help them recover from their current injury and had completed 12.5 weeks of physical therapy on average. When asked to rate their experience with traditional physical therapy on a scale of 0 to 10, with 0 anchored as “Physical therapy did not help at all” and 10 anchored as “Physical therapy was extraordinarily helpful,” the average patient score was 8.2 with a standard deviation of 2.0. Only 4 patients (26.7%) indicated they had prior experience with virtual reality.
Table 1.
Characteristic | |
---|---|
Age: mean (SD) | 51.1 (15.6) |
Sex: n (%) | |
Female | 6 (40.0%) |
Male | 9 (60.0%) |
Race: n | |
Black | 1 (6.7%) |
White | 14 (93.3%) |
Education: n | |
9th to 12th grade, no diploma | 1 (6.7%) |
High school or GED | 2 (13.3%) |
Some college | 5 (33.3%) |
Bachelors/college degree | 2 (13.3%) |
Graduate degree | 5 (33.3%) |
Work status just prior to injury | |
Working | 11 (73.3%) |
Taking care of house | 2 (13.3%) |
Something else | 2 (13.3%) |
Smoking status | |
Yes | 11 (73.3%) |
No | 4 (26.7%) |
Adequate social and emotional support | |
Always | 2 (13.3%) |
Usually | 4 (26.7%) |
Sometimes | 8 (53.3%) |
Rarely | 1 (6.7%) |
Marital status | |
Living with spouse/partner | 8 (53.3%) |
Separated or divorced | 2 (13.3%) |
Never married | 4 (26.7%) |
Other | 0 |
Injury Severity Score: mean (SD) | 5.1 (3.3) |
After completing the VR-PT session, patients were asked to rate their experience from 0 to 10, with 0 anchored as “I hated it” and 10 anchored as “I loved it.” The average patient rating was 7.5 with a standard deviation of 1.6 (95% CI: 6.69–8.31). Fourteen patients (93.3%) indicated that they would enroll in a randomized trial in which they had a 50/50 chance of receiving a virtual reality-based physical therapy or standard of care. All 15 patients completed the AIM and FIM measures. However, only 12 patients completed the SUS measure. Overall, patients rated the VR-PT program highly (Table 2). After having completed the VR-based PT program, the majority of patients reported that their preferred method of PT would be VR-based in-home PT with supplemental office-based PT (87%), compared with either VR-based or office-based PT alone. On the validated implementation measures, patients rated the acceptability of VR-PT at a mean 3.9 (SD 0.7, 95% CI: 3.55–4.25) out of 5 (item-by-item responses outlined in Fig. 3); patients rated the feasibility of VR-PT at a mean 4.0 (SD 0.4; 95% CI: 3.8–4.2) out of 5 (item-by-item responses outlined in Fig. 4); patients rated the usability of VR-PT at a mean 67.5 (SD 13.6; 95% CI: 59.8–75.2) out of 100 (item-by-item responses outlined in Fig. 5), which is right at the threshold for acceptable usability.[43]
Table 2.
Outcome measure | |
---|---|
Enjoyment rating∗ | 7.53 (1.55) |
Preferred PT method | |
VR alone | 1 (6.7%) |
VR plus office-based | 13 (86.7%) |
Office-based alone | 1 (6.7%) |
Acceptability of intervention measure, mean (SD)† | 3.9 (0.7) |
Feasibility of intervention measure, mean (SD)† | 4.0 (0.4) |
System usability scale, mean (SD)‡ | 67.5 (13.6) |
Scale 0 to 10, 0 anchored at “I hated it” and 10 anchored at “I loved it.”
Maximum value of 5.
Maximum value of 100.
Discussion
The results of this pilot study suggest that a VR-PT model is acceptable and practical to patients and that a randomized control trial using VR-PT is feasible. The majority of patients had a positive experience during their VR-PT session and indicated they would like to incorporate VR-PT into their rehabilitation and would also be willing to enroll in a clinical trial where they had a chance to receive VR-PT instead of a traditional PT model. Patients scored the system as acceptable on the usability scale.
There are several benefits associated with VR-PT that could be increasingly realized in future iterations. The VR system has the capability to collect quantitative data on system utilization/compliance with prescribed PT protocol, accuracy of performing exercises, and sequential data monitoring of progression toward achieving rehabilitation goals using motion and pressure sensors. Data can be transmitted cumulatively and in real time to both patients and care providers. To date, objective reporting regarding compliance with prescribed PT and objective measures of progress in functional improvement have been extremely limited or nonexistent. VR-PT can be used to facilitate remote supervision regarding functional improvement and can provide patients with objective measures of improvement to boost engagement. This creates opportunities for improving our understanding of the relative benefits of various physical therapy protocols and to improve resource utilization and value (e.g., matching individual patient therapy needs with available resources). The objective data collected can help clinicians and therapists to evaluate which exercises are most effective in achieving specific goals and define the potential gains achieved by highly compliant and motivated patients versus less motivated or compliant patients. Based off this remote data collection there is also the potential for patients to be identified as “succeeding” or “failing” at the remote physical therapy intervention, prompting referral for in-person PT if indicated.
There are substantial benefits to a VR-based intervention with regards to engagement. Reporting sequential improvement in physical therapy goals is likely to be motivating and engagement for many patients. Furthermore, VR platforms can improve engagement through use of goal-oriented “games” designed around physical therapy protocols.[44–47]
VR platforms are highly scalable and can be administered remotely. This is particularly important with the increasing emphasis on telemedicine or remotely administered interventions in the context of the COVID-19 pandemic as well as access issues in many rural (and urban) settings as well as internationally. Exercise instruction and safety monitoring can be executed independent of a therapist, providing the opportunity to distribute the same intervention to hundreds (or thousands) of patients. An intervention such as this may serve to provide physical therapists with more time and resources to focus on patients with more intensive physical therapy needs.
Given the high cost of supervised in-office physical therapy, particularly when administered 1 to 3 times per week, transition of patients to in-home VR-based PT presents an enormous opportunity for cost savings (both direct cost of therapy and indirect costs associated with travel time and lost work for patients and caregivers).[13,19] Castillo et al demonstrated that patients with an unmet physical therapy need, as assessed by a physical therapist, were statistically significantly less likely to achieve functional improvement[13] and “too expensive” was the most commonly cited reason for not receiving PT among patients who felt they would have benefited from PT.[19]
During this pilot study we identified several opportunities for improvement in future research, particularly as the technology is improving at a rapid rate. To track the knee and ankle position and rotation in this application, sensors using infrared outside-in tracking were used on the foot, lower leg, and thigh. This required a VR headset that had infrared lighthouse sensors and a computers graphics processing unit to render the application's graphics and thus required a more complex setup with a laptop computer and cables. During this pilot study the cables were identified as potentially problematic both for setup and safety. In conjunction with advancement of VR technology, we have developed strategies to address this issue and eliminate the cables by using new standalone VR headsets with inside-out tracking capabilities with sensors that use Bluetooth connectivity. This will minimize concerns and potential issues with setup and is likely to improve the usability score associated with this application. This study also revealed potential safety issues around the completely immersive environment in that patients wearing the VR headset cannot see their real environment such as furniture. Because of this we are planning for future iterations to be translated into Augmented Reality, which is an interactive experience of the real world where digital or computer-generated elements are added to the real world. This platform would also benefit from multiuser capabilities in which patients can compete with each other in the goal-oriented games and in which therapists or physicians can join the patient to assist in assessment or treatment.
There are several limitations associated with this pilot study. The VR hardware and software used in this study are not labeled by the FDA for telemedicine. This is an observational study assessing the feasibility and acceptability of this intervention and there is, therefore, no control group and small subject size by design. This study was designed to lay the groundwork for future implementation-type studies which will include intervention and comparative groups with adequate power to detect differences. There also is a possibility of selection bias and patients who choose to enroll in our study may be meaningfully different than those who declined participation. However, only 2 patients declined participation and they reported time constraints as their reason rather than any substantial bias against the intervention. Finally, this initial study was completed in a clinic setting and, although 87% of participants reported that they believe they could independently set up the VR system in their home without difficulty, we do not yet fully understand the potential barriers that may exist when attempting to implement this system in patient homes. Future home-based implementation studies are needed to better assess this.
Conclusions
The response to VR-PT in this pilot study was positive and may be a practical supplement or alternative to traditional physical therapy. A VR-based PT program may add value for both patients and clinicians in terms of objective data collection (to aid in compliance monitoring, progression toward goals and exercise safety), increased engagement, and increased access. A future implementation study will be required to demonstrate that a VR-PT model can be applied practically and effectively.
References
- 1.Owens BD, Kragh JF, Wenke JC, et al. Combat wounds in operation iraqi freedom and operation enduring freedom. J Trauma. 2008;64:295–299. [DOI] [PubMed] [Google Scholar]
- 2.Rice DP, MacKenzie EJ, Jones AS, et al. Cost of injury in the United States: a report to congress. 1989. [Google Scholar]
- 3.Dillingham TR, Pezzin LE, MacKenzie EJ. Incidence, acute care length of stay, and discharge to rehabilitation of traumatic amputee patients: an epidemiologic study. Arch Phys Med Rehabil. 1998;79:279–287. [DOI] [PubMed] [Google Scholar]
- 4.Pezzin LE, Dillingham TR, MacKenzie EJ. Rehabilitation and the long-term outcomes of persons with trauma-related amputations. Arch Phys Med Rehabil. 2000;81:292–300. [DOI] [PubMed] [Google Scholar]
- 5.Watkins-Castillo S, Andersson G. United States Bone and Joint Initiative: The Burden of Musculoskeletal Diseases in the United States (BMUS). 2014. [Google Scholar]
- 6.Cross JD, Ficke JR, Hsu JR, et al. Battlefield orthopaedic injuries cause the majority of long-term disabilities. J Am Acad Orthop Surg. 2011;19 (suppl 1):S1–S7. [DOI] [PubMed] [Google Scholar]
- 7.Masini BD, Waterman SM, Wenke JC, et al. Resource utilization and disability outcome assessment of combat casualties from Operation Iraqi Freedom and Operation Enduring Freedom. J Orthop Trauma. 2009;23:261–266. [DOI] [PubMed] [Google Scholar]
- 8.MacKenzie EJ, Bosse MJ, Pollak AN, et al. Long-term persistence of disability following severe lower-limb trauma: results of a seven-year follow-up. J Bone Jt Surg A. 2005;87:1801–1809. [DOI] [PubMed] [Google Scholar]
- 9.Bosse MJ, MacKenzie EJ, Kellam JF, et al. An analysis of outcomes of reconstruction or amputation of leg-threatening injuries. N Engl J Med. 2002;347:1924–1931. [DOI] [PubMed] [Google Scholar]
- 10.Castillo RC, MacKenzie EJ, Wegener ST, et al. Prevalence of chronic pain seven years following limb threatening lower extremity trauma. Pain. 2006;124:321–329. [DOI] [PubMed] [Google Scholar]
- 11.Bedigrew KM, Patzkowski JC, Wilken JM, et al. Can an integrated orthotic and rehabilitation program decrease pain and improve function after lower extremity trauma? Clin Orthop Relat Res. 2014;472:3017–3025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Blair JA, Patzkowski JC, Blanck RV, et al. Return to duty after integrated orthotic and rehabilitation initiative. J Orthop Trauma. 2014;28:e70–e74. [DOI] [PubMed] [Google Scholar]
- 13.Castillo RC, MacKenzie EJ, Archer KR, et al. Evidence of beneficial effect of physical therapy after lower-extremity trauma. Arch Phys Med Rehabil. 2008;89:1873–1879. [DOI] [PubMed] [Google Scholar]
- 14.Patzkowski JC, Owens JG, Blanck RV, et al. Deployment after limb salvage for high-energy lower-extremity trauma. J Trauma Acute Care Surg. 2012;73:S112–S115. [DOI] [PubMed] [Google Scholar]
- 15.Hurwitz EL, Aker PD, Adams AH, et al. Manipulation and mobilization of the cervical spine: a systematic review of the literature. Spine (Phila Pa 1976). 1996;21:1746–1760. [DOI] [PubMed] [Google Scholar]
- 16.Bergman GJD, Winters JC, Groenier KH, et al. Manipulative therapy in addition to usual medical care for patients with shoulder dysfunction and pain: A randomized, controlled trial. Ann Intern Med. 2004;141:432–439. [DOI] [PubMed] [Google Scholar]
- 17.Hayden JA, Van Tulder MW, Tomlinson G. Systematic review: strategies for using exercise therapy to improve outcomes in chronic low back pain. Ann Intern Med. 2005;142:776–785. [DOI] [PubMed] [Google Scholar]
- 18.Minkoff J, Harris GR, Susman JL. Managing musculoskeletal complaints with rehabilitation therapy: summary of the Philadelphia Panel evidence-based clinical practice guidelines on musculoskeletal rehabilitation interventions. J Fam Pract. 2002;51:1042–1046. [PubMed] [Google Scholar]
- 19.Castillo RC, MacKenzie EJ, Webb LX, et al. Use and perceived need of physical therapy following severe lower-extremity trauma. Arch Phys Med Rehabil. 2005;86:1722–1728. [DOI] [PubMed] [Google Scholar]
- 20.McCarthy ML, Ewashko T, MacKenzie EJ. Determinants of use of outpatient rehabilitation services following upper extremity injury. J Hand Ther. 1998;11:32–38. [DOI] [PubMed] [Google Scholar]
- 21.Horn W, Yoels W, Bartolucci A. Factors associated with patients’ participation in rehabilitation services: a comparative injury analysis 12 months post-discharge. Disabil Rehabil. 2000;22:358–362. [DOI] [PubMed] [Google Scholar]
- 22.Wrigley JM, Yoels WC, Webb CR, et al. Social and physical factors in the referral of people with traumatic brain injuries to rehabilitation. Arch Phys Med Rehabil. 1994;75:149–155. [PubMed] [Google Scholar]
- 23.Harada ND, Chun A, Chiu V, et al. Patterns of rehabilitation utilization after hip fracture in acute hospitals and skilled nursing facilities. Med Care. 2000;38:1119–1130. [DOI] [PubMed] [Google Scholar]
- 24.Hoenig H, Rubenstein L, Kahn K. Rehabilitation after hip fracture–equal opportunity for all? Arch Phys Med Rehabil. 1996;77:58–63. [DOI] [PubMed] [Google Scholar]
- 25.Crichlow RJ, Andres PL, Morrison SM, et al. Depression in Orthopaedic Trauma Patients<sbt aid="1118572">Prevalence and Severity</sbt>. J Bone Jt Surg. 2006;88:1927. [DOI] [PubMed] [Google Scholar]
- 26.Archer KR, Abraham CM, Obremskey WT. Psychosocial factors predict pain and physical health after lower extremity trauma. Clin Orthop Relat Res. 2015;473:3519–3526. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Clay FJ, Watson WL, Newstead SV, et al. A systematic review of early prognostic factors for persistent pain following acute orthopedic trauma. Pain Res Manag. 2012;17:35–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Wegener ST, Castillo RC, Haythornthwaite J, et al. Psychological distress mediates the effect of pain on function. Pain. 2011;152:1349–1357. [DOI] [PubMed] [Google Scholar]
- 29.Vranceanu A-M, Hageman M, Strooker J, et al. A preliminary RCT of a mind body skills based intervention addressing mood and coping strategies in patients with acute orthopaedic trauma. Injury. 2015;46:552–557. [DOI] [PubMed] [Google Scholar]
- 30.Ponce BA, Brabston EW, Shin Z, et al. Telemedicine with mobile devices and augmented reality for early postoperative care. Annu Int Conf IEEE Eng Med Biol Soc. 2016;2016:4411–4414. [DOI] [PubMed] [Google Scholar]
- 31.Ponce BA, Jennings JK, Clay TB, et al. Telementoring: use of augmented reality in orthopaedic education: AAOS exhibit selection. J Bone Joint Surg Am. 2014;96:e84. [DOI] [PubMed] [Google Scholar]
- 32.Tchero H, Teguo MT, Lannuzel A, et al. Telerehabilitation for stroke survivors: systematic review and meta-analysis. J Med Internet Res. 2018;20:e10867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Chen J, Jin W, Dong WS, et al. Effects of home-based telesupervising rehabilitation on physical function for stroke survivors with hemiplegia: a randomized controlled trial. Am J Phys Med Rehabil. 2017;96:152–160. [DOI] [PubMed] [Google Scholar]
- 34.Redzuan NS, Engkasan JP, Mazlan M, et al. Effectiveness of a video-based therapy program at home after acute stroke: a randomized controlled trial. Arch Phys Med Rehabil. 2012;93:2177–2183. [DOI] [PubMed] [Google Scholar]
- 35.Van Den Berg M, Crotty M, Liu E, et al. Early supported discharge by caregiver-mediated exercises and e-health support after stroke: a proof-of-concept trial. Stroke. 2016;47:1885–1892. [DOI] [PubMed] [Google Scholar]
- 36.Piron L, Turolla A, Agostini M, et al. Exercises for paretic upper limb after stroke: a combined virtual-reality and telemedicine approach. J Rehabil Med. 2009;41:1016–1102. [DOI] [PubMed] [Google Scholar]
- 37.Huijgen BCH, Vollenbroek-Hutten MMR, Zampolini M, et al. Feasibility of a home-based telerehabilitation system compared to usual care: arm/hand function in patients with stroke, traumatic brain injury and multiple sclerosis. J Telemed Telecare. 2008;14:249–256. [DOI] [PubMed] [Google Scholar]
- 38.Powers MB, Emmelkamp PMG. Virtual reality exposure therapy for anxiety disorders: a meta-analysis. J Anxiety Disord. 2008;22:561–569. [DOI] [PubMed] [Google Scholar]
- 39.Gonçalves R, Pedrozo AL, Coutinho ESF, et al. Efficacy of virtual reality exposure therapy in the treatment of PTSD: a systematic review. PLoS One. 2012;7:e48469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.KairosXR: Custom VR/AR applications for employee training. Available at: https://www.kairosxr.com/. Accessed April 17, 2020. [Google Scholar]
- 41.IAM I. Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), & Feasibility of Intervention Measure. ictp.fpg.unc.edu. Available at: https://ictp.fpg.unc.edu/sites/ictp.fpg.unc.edu/files/resources/Triple P System Implementation Outcomes_Acceptability%2C Appropriateness%2C Feasibility.pdf. Accessed April 17, 2020. [Google Scholar]
- 42.Brooke J. Sus: a “quick and dirty" usability scale. Usability Evaluation in Industry. 1996;189. [Google Scholar]
- 43.Sauro J, Lewis JR. Quantifying user research. In: Quantifying the User Experience: Practical Statistics for User Research. 2016;Morgan Kaufmann, 198. [Google Scholar]
- 44.Hoffman HG, Chambers GT, Meyer WJ, et al. Virtual reality as an adjunctive non-pharmacologic analgesic for acute burn pain during medical procedures. Ann Behav Med. 2011;41:183–191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Jeffs D, Dorman D, Brown S, et al. Effect of virtual reality on adolescent pain during burn wound care. J Burn Care Res. 2014;35:395–408. [DOI] [PubMed] [Google Scholar]
- 46.Cuthbert JP, Staniszewski K, Hays K, et al. Virtual reality-based therapy for the treatment of balance deficits in patients receiving inpatient rehabilitation for traumatic brain injury. Brain Inj. 2014;28:181–188. [DOI] [PubMed] [Google Scholar]
- 47.Zimmerli L, Jacky M, Lünenburger L, et al. Increasing patient engagement during virtual reality-based motor rehabilitation. Arch Phys Med Rehabil. 2013;94:1737–1746. [DOI] [PubMed] [Google Scholar]