Extended Reality in Medical Education: Driving Adoption through Provider-Centered Design

Abstract

Simulation is a widely used technique for medical education. Due to decreased training opportunities with real patients, and increased emphasis on both patient outcomes and remote access, demand has increased for more advanced, realistic simulation methods. Here, we discuss the increasing need for, and benefits of, extended (virtual, augmented, or mixed) reality throughout the continuum of medical education, from anatomy for medical students to procedures for residents. We discuss how to drive the adoption of mixed reality tools into medical school's anatomy, and procedural, curricula.

Simulation emerged as a best practice for training in the early 1900s with the development of flight simulators [1]. Over the past 3 decades, simulated training environments have become highly sophisticated, incorporating visual, audio, tactile, and motion components into a “virtual” reality [2]. The term virtual reality (VR) refers to “…the interactions between an individual and computer-generated environment stimulating multiple sensory modalities, including visual auditory, or haptic experiences” [3]. Recently, training environments have been built to incorporate both real and virtual objects. The latter simulation environment is called augmented reality, or “AR,” where “the virtual world is superimposed on the real one so both are experienced simultaneously” [4].

Here, we will use the term extended reality (XR) to include VR, AR, or any computer-generated reality [5]. Demand for XR in high-risk vocations, such as healthcare, has grown exponentially due to the need for skilled operators, and diminishing opportunities, for real-world training experiences [6]. XR provides educational solutions in a number of industries, but this review will focus on the use of XR across the continuum of medical education.

XR solutions have tremendous potential to transform the education of medical students and trainees in human anatomy and invasive procedures. For example, using XR, students can look inside the human body to study physiological processes. XR simulation can also integrate other digital biomedical data, such as patient-specific CT scans and MRIs. XR can be particularly helpful for learning anatomy and procedures because it allows trainees to prepare, practice, and review their performance repeatedly in an environment with no real-world consequences. While the use of XR is not yet routine in medical education, there have been numerous pilots and experiments over the past 3 decades centered around anatomy [3]. Examples of several of these pilots are outlined in Table 1.

Table 1.

Examples of XR anatomy innovations at medical schools

Description	School
Stanford Neurosurgical Simulation and Virtual Reality Center[22] XR type: VR Level of learners: medical students, residents, surgeons Focus: neuroanatomy, neurosurgical procedure training	Stanford
HoloAnatomy with Microsoft Hololens[18] XR type: MR Level of learners: medical students Focus: general anatomy	Case Western/Cleveland Clinic
Immersive Education at CHLA with Oculus Go[23] XR type: VR Level of learners: all incoming residents, optional for medical students Focus: pediatric trauma procedures, pediatric resuscitation training	Children's Hospital of Los Angeles
Virtual Reality Anatomy at USCF with HTC Vive[14] XR type: VR Level of learners: first-year medical students Focus: general anatomy	University of California, San Francisco
Enduvo VR Teaching and Learning Platform using HTC Vive[24] XR: VR Level of learners: medical students, surgeons, faculty Focus: general anatomy	University of Illinois College of Medicine Peoria

Medical schools recognize that new technologies, such as XR, can augment anatomical education, but their use remains low. The answer to why they have not yet been universally adopted is complex. Some schools are resistant to change because of the cost, time, and inherent risks associated with implementing new technologies. Others see no reason to stop using traditional educational approaches that have continually demonstrated efficacy [7]. For example, a meta-analysis that compared different methodologies of anatomical education showed that dissection versus prosection versus the use of digital media did not impact medical students' academic performance [7].

While implementation costs, faculty acceptance, and student adoption are important considerations, educators must demonstrate that XR provides value and an educational advantage to drive adoption [8]. One potential framework to address the barriers to adoption is a “go-to-market strategy,” used in the business and startup world. It is a plan of action for delivering a tool to a target population that solves a problem, and provides value to that particular user group [9]. We break down this strategy into three sequential steps: faculty-driven product design, student-centered iteration, and validation of multi-stakeholder value.

Faculty-Driven Design

Innovation in medical education is difficult for several reasons. One of the biggest challenges is faculty adoption of technology. Risk-averse and conservative by training, the majority of faculty and physicians are resistant to changing their approach to teaching medicine. The first step to driving adoption of a novel educational tool, such as XR, is faculty recruitment and collaboration.

Most of the existing medical training programs in XR were designed by faculty using VR in their own clinical practice. “[XR] applications have been designed for the surgical specialties due to the heavy involvement and support of physicians” [4]. We propose that strong faculty engagement, early on, would motivate educators to stay involved and advocate for the integration of XR into medical education [10]. Faculty need to assume the role of “champion” for XR in undergraduate medical education. A faculty champion would “[interact] with developers, the intended users and other stakeholders and [function] as the main negotiator between these parties. In this context, it is crucial that a ‘champion clinician’ participates in each phase of the development process, from identifying training objectives, through system design and implementation, to validation and integration of the training and assessment system into formal training” [11]. This faculty champion would not only guide the development process, ensuring that educational objectives were met at every step, but also use his or her position within the medical system to advocate for the development, iteration, and use of the XR simulation tool. For example, the champion clinician would work with the medical school administration, dean, and educators to ensure increased value to both the students and school.

While crucial to sponsorship and overall vision for XR in medical education, the faculty champion cannot drive the product's development and implementation alone. Introducing a new mode of learning into a traditional curriculum requires a multidisciplinary approach. Clinicians bring medical expertise and administrative clout, but need engineers' technical skills, educators' knowledge of training principles and curricula development, and students' willingness to test the new technology to create a viable pilot. It is critical to involve the end-users of an XR tool, including clinical educators, physicians, and students, early on, giving them both a say and stake in the final product. Lettl and Gemünden [12] argue that “the users' ability and willingness to participate is the highest when users are inventors and play an entrepreneurial role in realizing the initial concepts of the technology into a product.”

The need for multidisciplinary collaboration extends beyond the walls of the medical school. XR is rapidly evolving, with new startups, software, and hardware constantly emerging. For example, headsets that are popular right now, such as Oculus Rift for VR, and Microsoft Hololens, or Magic Leap, for AR, may not exist, or may be drastically different, in the next 2 years. We have outlined several technologies in Table 2 to highlight this issue. This poses a challenge for medical schools, who are skeptical of investing money and time in technologies that are not only disruptive to implement, but may become quickly obsolete. Medical institutions should not avoid piloting XR technologies because of this risk, but rather, should work with industry to try to predict the growth and staying power of the XR tools that they choose to test. As mentioned previously, we suggest that the XR physician faculty champions identify internal allies who specialize in validating, commercializing, and adopting new technologies to aid them in this process. These leaders' departmental affiliations differ from institution to institution. For example, some medical centers' IT departments are heavily involved in working with industry to implement outside technologies, whereas other medical centers rely on different resources, such as “innovation centers” [13]. The XR educators and these internal allies, with deep expertise in bringing healthcare technology products from idea to implementation, need to work together with external, or internal, companies to identify, test, demonstrate evidence of efficacy, and eventually co-build these novel education programs.

Table 2.

Selected XR technology

Technologyname	Estimated cost, USD	Description
Microsoft Hololens	3,000	Head-mounted wireless computer system including AR display
GoogleCardboard	5–10	Compact inexpensive cardboard adapter to use smartphones as VR glasses
Oculus Rift	350	High-fidelity VR headset display; requires a powerful connected computer
Oculus Go	200	Standalone VR headset; works wirelessly without a computer
HTC Vive	500	High-fidelity VR headset display; requires a powerful connected computer
Magicleap	2,295	Standalone AR headset; works wirelessly without a computer
Google Glass	1,500	Standalone AR headset; works wirelessly without a computer
zSpace	4,000	AR desktop display monitor; can be viewed by multiple users at once; requires a powerful connected computer

Student-Centered Iteration

The ultimate end-user for XR technology in medical education is the learner – the medical student, or trainee. XR may address many issues in anatomical and procedural education, such as ubiquitous access and real-time feedback, but it will not be integrated into medical curricula unless it demonstrates educational value and student adoption. The creation of an engaging, effective medical educational tool, like XR, warrants continuous medical student feedback throughout the product's development.

We propose that just as there must be faculty champions, there must also be medical student champions directing the XR tool development process from the beginning. Medical students should not just help initially to define educational goals, but should also provide feedback at each stage of development using “iterative prototype design.” Described in the literature as the optimal process to pilot VR training and assessment systems, “iterative design helps provide formative evaluations… of a system, generate feedback back to the process and develop new, improved versions for further testing” [11].

When considering medical student involvement, schools must consider barriers to participation. Medical students' overwhelming workload significantly impacts their ability to participate in additional educational activities, beyond what is required. Given that each medical school has its own distinct student population, academic culture, and educational goals, each will likely choose a different approach to how it pilots a product. For example, while UCSF made the XR component of their anatomy course optional [14], other universities may choose to require it, or vary the number of participants. Regardless of how schools tailor their product testing approach, we suggest that they identify students willing to dedicate a substantial amount of additional time to engaging with the XR tool. The school should create additional opportunities for these student champions to stay involved in the development, and eventual rollout, of the XR product throughout their education. For example, in exchange for their continued participation, the school could choose to grant elective credit or research opportunities, which have been shown to be associated with high match rates in a student's specialty of choice [15]. We believe that medical students are more likely to engage with the XR tool, and use it throughout the continuum of their medical education, if they believe it is an investment in their future as a physician.

Validation of Multi-Stakeholder Value: Administration, Student, and Educator

While student and faculty champions are integral to introducing XR into medical education, demonstrating improved learning and return-on-investment furthers administrative acceptance, and subsequent implementation, into medical school curricula. Ideally, the tool should show reduced long-term costs, improved access, more effective assessment, and an overall higher-quality learning experience.

Cost Reduction: Value for Administration

Most undergraduate medical institutions still rely heavily on cadavers as a primary instructional modality for anatomy and procedural training [16]. While cadavers allow students to interact with a real human body, which significantly contributes to students' understanding of anatomy and humility when dealing with mortality, cadavers are extremely costly to acquire and maintain [17]. By introducing XR into anatomy and procedural training, medical schools could reduce resources spent on the procurement of, and infrastructure to, maintain cadavers. While upfront costs may rise to support the implementation of an XR tool, in the long-run, the opportunities of decentralized or asynchronous instruction may allow schools to more flexibly integrate anatomy instruction with the broader curriculum and even integrate learning into point-of-care environments, such as the operating room.

Access and Educational Experience: Value for Students

Introducing XR into the anatomy curriculum could vastly improve student access to educational resources. Both time in the cadaver lab and opportunities to practice procedures are limited. XR may grant medical students the opportunity to see and interact with the human body, inside and outside of the classroom, or even in the hospital, while caring for patients. Currently, students who are studying anatomy can only access the cadaver lab during certain hours of the day. During this time, students are expected to learn all the organs, bones, muscles, vessels, and nerves for a particular part of the body. Students need access to a human body that they, and their peers, can collaboratively interact with outside of the lab. Unlike other educational tools, such as mannequins or online video tutorials, students using XR headsets can familiarize themselves with the spatial orientation of the 3D holographic human body. XR also allows learners to dynamically remove certain layers of tissue, or organ systems, to better isolate specific structures of interest. Another advantage that XR has over cadavers is that, like a living patient, the holographic body's systems can move. For example, using the Microsoft Hololens, medical students at Case Western can “see systems not only separately and together but also in motion, allowing users to see how the heart moves and observe the heart valves opening and closing while hearing the sounds of a heartbeat” [18]. Because it allows the user to integrate virtual and real objects into the learning environment, AR specifically allows students to engage with the dynamic, holographic human body at their own pace, easily delving into details that they might have missed in the lab. AR also allows trainees to collaborate on the same body with multiple peers. If access to headsets is limited, a tablet, such as the iPad, can be used to capture the real-world objects on camera while displaying a virtual body on the screen [19]. The controlled, risk-free environment that XR provides, tailored to an individual student's needs, is an ideal environment to learn a new skill, for both novices and experts. XR allows for more realistic, sensory feedback, real-time assessment of performance, and access to training outside of the anatomy lab, bedside, or operating room [2, 20].

Effective Evaluation: Value for Educators

XR is not just an educational tool for medical students. It can also be used for assessment. Medical students are constantly tested through institution-specific, and national, board exams. Standardized tests can demonstrate a student's mastery of clinical information with relative accuracy, but this mode of evaluation is limited. XR, on the other hand, can help both students and faculty to continuously moderate progress through “repeated testing, feedback and self-controlled practice” [21]. For example, if a student is assigned to learn a specific system of the human body and her professor receives feedback that she is struggling with a particular subject, the student can modify the way in which she interacts with the XR tool to improve her mastery. XR could also provide significant advancements to the evaluation of students' procedural skills. If educators used XR patient and surgical simulations to evaluate students, they could standardize these evaluations, making assessments of students both more objective, accurate, and equitable.

The Anatomy Course of the Future

Should schools effectively implement the steps above, they would have an opportunity to create a dynamic educational program for teaching anatomy and procedures. This course of the future would no longer be defined by the time and space of the anatomy lab, and could be woven into the context of other curricular areas such as radiology, procedures, and surgical skills. Faculty would have a multitude of educational choices from which they could deliver “blended learning,” which mixes traditional cadavers, 3D-printed models, XR, and live patients. This modularity would likely result in diverse courses across schools and programs.

Learning in this course would be both student and teacher controlled. Teachers would continue to provide the framework and guidance of the overall course, but XR would give students more control over their pace of learning and interaction with the material. Collaboration and teaching would happen in virtual and live spaces. Lastly, XR represents an opportunity for an additional set of faculty teachers, with novel skillsets, to participate in the core anatomy program. Faculty who use XR in clinical practice, such as radiologists, surgeons, pathologists, and interventionalists, would now all have roles as educators in this new future.

Conclusion

XR offers significant potential value to medical education, but use remains low. If the barriers to adoption can be overcome, XR could transform medical education by introducing on-demand learning, remote access to educational materials, and objective evaluation. We advise that medical schools who want to implement XR into their curricula dedicate the majority of their efforts toward identifying faculty and student champions, before attempting to pilot the technology. Lasting educational technology that leads to meaningful change should be valuable to its end-users not just at one specific point in time, but across the continuum of education. XR is useful not just to medical students, but residents, and even attendings, who not only want, but are mandated, to learn and practice new skills throughout their careers. XR promises a mode of education that is dynamic, accessible, and adaptable to student needs. It gives trainees repeatable access to life-like experiences without risk, and gives evaluators a tool to benchmark and customize the learner experience.

Disclosure Statement

The authors declare no conflicts of interest.