A New Virtual World? The Future of Immersive Environments in Anesthesiology

See Article, page 220

Immersive virtual environments (IVEs), including virtual reality (VR) and augmented reality (AR), have gained immense popularity in medical education health care.¹ Over the last 10 years, the expansion of gaming and development platforms, such as Unreal Engine and Unity, along with head-mounted displays (HMDs), has been fueled by major commercial interests, such as Meta, Microsoft, and Google, with huge investments in this immersive reality market. The immersive reality market is currently valued at 6.1 billion US dollars (USD) and is predicted to reach >20 billion USD by 2025.² This commercial expansion has led to the rapid advancement of affordable, accessible, high-powered VR/AR technology and development tools supporting IVEs in medical education including anesthesia.

IVEs date back to the early 19th century with the advent of the stereoscope. This ingenious device allowed 2 separate images, taken at slightly different angles, to be viewed as a single image with depth. The stereoscope led to the creation of a children’s toy: the View-Master (View-Master Ideal Group, Inc), a pioneer for modern-day VR. Subsequently, the invention of the microchip represented a significant leap, leading to a myriad of devices and technologies that make up IVEs today. Virtual environments have been described in 2 domains: immersive (HMD-based) and nonimmersive (desktop-based). Nonimmersive, desktop-based virtual environments do not use an HMD, but instead, the participant controls and manipulates the virtual environment on a screen with a traditional keyboard and mouse or a joystick.^3,4 Playing a video game on one’s television screen using popular hardware such as the Sony PlayStation or the Microsoft Xbox is a basic example of desktop-based VR. In medical education, the American Society of Anesthesiologists (ASA) recently partnered with CAE Healthcare to create desktop-based virtual environments for managing anesthesia emergencies,⁵ which, when completed, fulfill the American Board of Anesthesiology Maintenance of Certification in Anesthesiology (MOCA) 2.0 Part 2 and 4 requirements.⁶ Although nonimmersive desktop-based virtual environments have been popular for decades and are an important part of our current medical education system, the focus of this article is on HMD-based IVEs.

HMD-based IVEs have been described as “typically multi-modal in nature by providing a sense of immersion in the environment through 360° visuals by aid of an HMD, auditory stimulation through the use of earphones, and increasing the proprioception of limbs by way of controllers and tracking.”⁴ Using HMDs, IVEs “transport” users to alternate simulated environments while creating a sense of immersion. Imagine playing a video game, but instead of watching it on a 2-dimensional screen, actually being transported into the game itself, akin to the experience of the characters in the movie “Jumanji.” The HMDs used vary in complexity from simple cardboard-based stereoscopic lenses that pair with a mobile device to expensive helmets and goggles with advanced haptics.

A tenet of IVE is presence. Presence is the extent to which a participant feels that they are submerged within the simulated IVE. Presence is a key term that describes the difference between VR and AR (Table). VR is the process of being completely immersed in a “new” environment (filmed or computer-generated), while AR overlays digital elements on real-life surroundings. As an example, VR would completely alter the sensory input of users via an HMD and transport them into a virtual trauma room, while AR would project, again via an HMD, a virtual patient into an actual real-world operating room (OR).

Table.

Virtual Reality Versus Augmented Reality: Differences and Similarities

Virtual Reality	Augmented Reality
Creates entire virtual world	Creates a combination of virtual and real world
Immersive	Holographic
Computer generated or filmed using 360° camera elements	Computer-generated elements that are superimposed onto real life
Requires a headset	Requires multiple devices
Immersed completely into the projected environment using a head-mounted display that completely suppresses the real world (eg, Google Cardboard and Oculus Rift)	Immersed partially from screen-based devices with cameras (ie, smartphones, laptop, or tablet) using a head-mounted display with see-through lenses, allowing for visualization of the real world and virtual elements overlaid into it (eg, Microsoft Hololens)
Object manipulation	Object manipulation
Only digital objects in virtual world	Combines physical simulation with virtual objects
Pricing	Pricing
Cost for creation and equipment can vary from a few hundred to thousands of dollars, depending on the level of immersion and complexity.	Cost for creation and equipment can vary from a few hundred to thousands of dollars, depending on the level of immersion and complexity.

In the same manner that IVEs have evolved from the stereoscope, Anesthesia & Analgesia has evolved over the past 100 years, from being the world’s first anesthesia journal to one that is a leading source for innovation and education. Using lessons learned from other health care specialties and the non-health care industry, this article serves as a futuristic report that describes how IVEs, specifically VR and AR, can enhance medical training and subsequent delivery of patient care within the field of anesthesia. We suggest future directions and next steps in the evolution of this technology within the specialty of anesthesia. Technical aspects on “how-to” create and develop immersive environments are beyond the scope of this article; however, we hope to create excitement around the potential for the use of IVEs in anesthesia.

IVEs AND ANESTHESIA EDUCATION

Before the coronavirus disease 2019 (COVID-19) pandemic, the staple discourse in medical education included 3 prevalent themes. The first dealt with health care worker fatigue and working hour restrictions implemented to prevent human errors and to maintain learning.⁷ The second reflected the seismic shift in medical curriculums that have moved toward competency-based rather than time-based training.⁸ The third dominant theme revolved around the growing need for continuing education programs, especially considering recent retrospective literature, which has suggested correlations of increasing physician age and duration from graduation with patient complications.^9,10 The result of these significant issues has tasked educators with finding innovative and effective methods to enhance learning curriculums. Despite this need, health care delivery continues to fall behind other service industries with regard to its investment into, and adoption of, new disruptive technologies. Furthermore, the COVID-19 pandemic has only accelerated the need for innovative educational approaches and emphasized the need for technology, including Zoom and Teams. IVE is a technology that is likely to grow beyond the pandemic.^11,12 Discovering and using adjuncts to trainee curriculums and creating pathways to extend the reach of health care beyond the physical boundaries of the hospital are some of the new themes of medical education.^11–13

Technical Skills

Training for a career in anesthesiology requires expertise in both technical and nontechnical skills. The field of anesthesiology has led the way in creating and adapting mannequin-based simulation as a gold standard for training without risk to patients.¹⁴ However, to date, procedural dominant specialties such as surgery have pioneered the use of IVEs to teach and practice surgical skills and to instruct anatomy in an interactive manner.^15,16

IVE-based anesthesia training has been reported for procedural areas such as bronchoscopy, central line insertion, and regional anesthesia simulators.^17–20 Studies investigating such procedural training have shown that practicing technical skills in virtual environments can lead to effective knowledge acquisition and retention with subsequent successful clinical application.^15,17 Furthermore, IVEs can be easily manipulated, allowing iterative skill-building within evolving virtual scenarios. If a trainee typically takes 50 attempts at intubation to become proficient, that goal may be achieved more quickly using a VR simulator that evolves in difficulty and variance. The learning curve would be enhanced because the scenarios change (as they would in the real world). However, they do so repeatedly, over a shorter time period compared to a traditional training system that would have the trainee wait hours for each surgery to complete before having a chance to perform another intubation procedure.

Nontechnical Skills

IVEs may have significant potential in teaching and assessing nontechnical skills, such as empathy, communication, and decision-making. At a basic level, observational learning, in which a learner watches a teacher demonstrate or model “correct” behavior or steps, has been used as a teaching tool for decades.²¹ However, with busier clinical environments, reduced trainee working hours, and amid pandemic-based restrictions, direct observation opportunities are also becoming limited.^22,23 One solution to supplement observation learning is the use of VR-360 video (Figure 1). Asynchronously, we can use VR-360 cameras to film core concepts of a teaching curriculum for learners to view using HMDs, allowing them to feel as though they are physically present²⁴ in the learning environment, learning from the teacher, when not there.^25,26 Synchronously, such technology has been used to broadcast live surgeries to trainees when not present in the OR. VR-360 streams do not need advanced HMDs; instead, they can be viewed using simple HMDs such as the Google Cardboard combined with a user’s mobile device.

Figure 1.

VR-360 video of an operating room. This is a fisheye perspective, as it is not being viewed through an HMD (source: www.CHISIL.ca). HMD indicates head-mounted display.

Filming core concepts of a teaching curriculum for learners to watch is not new. VR-360 videos have been used to create IVEs to raise the participation factor and improve the learning experience.^25,27–30 Existing examples are already in place, such as the Oxford Medical Education lab (www.oxfordmedicaleducation.com) and the Collaborative Human ImmerSive Interaction Laboratory (CHISIL; www.chisil.ca) at the University of Toronto. These sites have released teaching programs in an effort to improve decision-making skills around topics such as advanced life support, trauma management, and cardiac arrest/code blue scenarios using VR-360 video simulations.

Fully immersive computer-generated video game software mimicking clinical environments can also be used to effectively teach decision-making around uncommon crises, such as OR fires and malignant hyperthermia, and more common scenarios, such as trauma management, anaphylaxis, and obstetrical emergencies.³¹ Unlike passive VR-360 video, these advanced immersive crisis management “video games” require advanced HMDs to allow users to interact with their environment and the virtual components to respond to their actions. In these virtual environments, scenarios can change quickly or adapt to suit learners’ needs or specific objectives. Virtual patients will improve or deteriorate in real time, depending on the actions taken by learners (Figure 2A). Like a video game, learners can “restart” and manage crises repeatedly with feedback processes built into the software. Furthermore, these scenarios can be recorded for future viewing or even observed live by a facilitator to provide coaching or assessment remotely (ie, from another location).

Figure 2.

IVEs for clinical education. A, An advanced IVE depicting a trauma crisis with interactive avatars and responsive patients (source: www.CHISIL.ca). B, Dr Julian Wiegelmann demonstrates the use of augmented reality by using an HMD, converting the operating room into a crisis scenario with a virtual patient and responsive hemodynamics broadcast live on anesthesia monitors (source: www.Holosim.ca). HMD indicates head-mounted display; IVE, immersive virtual environment.

Similarly, augmented virtual environments can make any physical space a virtual learning environment. For instance, at our local site, the onset of the pandemic required training and implementation of multidisciplinary COVID intubation teams within a short time frame. Using proprietary, locally developed AR software (www.HOLOSIM.ca), we converted our ORs and native monitoring equipment into virtual COVID care rooms with virtual patients requiring intubation and hemodynamic management (Figure 2B). Their simulated vitals were broadcast on our live OR monitors in response to the actions of the team.

VALIDATING IVEs

A significant challenge with IVEs is that their implementation may be outpacing the ability of the education community to validate them as effective teaching tools. Validation of IVEs for anesthesia by medical educators is vital to ensure they offer teaching interventions that align with curriculum goals using relevant educational frameworks. The current literature has shown that research in this field has been heterogeneous and lacks a basis in foundational learning theory.^32,33 Opportunity exists to test and validate the use of IVEs for anesthesia education, including improved patient outcomes. To help advance appropriate validation when implementing a VR teaching program, medical educators should consider sound learning theory and learning outcomes as the structural foundation of the project.

Learning theory, at its basic level, can be thought of as a foundation that helps achieve learning outcomes. Similar to biological research, it can be thought of as “biological plausibility.” Like all new technologies, the urge to just “build an app” without really considering the learning outcomes and how intelligent design can best facilitate achieving these outcomes is a common setup in which this “just do” mentality will fail. Learning theory is a complex topic, but for this article, we highlight that one must consider normative and cognitive learning theories when designing immersive virtual applications. Normative theories emphasize that the educator must consider the type of learner they are targeting, and cognitive theories “are concerned with how the brain actively processes information to produce effective learning.”³⁴ Common theories in technology are multimedia design learning and cognitive load, meaning one must consider how learners process the new 3-dimensional audio and visual stimuli in immersive environments and how easily the content can be filtered by a learner and hence, absorbed.³⁴

Our laboratory has developed a design framework called “LOOP” to be applied to every project. The designers are obliged to create based on sound learning theory, considering what learning outcomes the educator is aiming for, and to shape and guide the actual immersive software output (Figure 3).

Figure 3.

LOOP framework. Design framework developed to advance appropriate validation when implementing IVE teaching programs by considering learning theory, desired learning outcomes and the actual immersive software output. AR indicates augmented reality; IVE, immersive virtual environment; VR, virtual reality.

Future directions of anesthesia-based training with immersive environments and emerging educational areas for development include: (1) remote teaching/coaching, (2) teaching communication skills, and (3) equity, diversity, and inclusivity perspective learning. Using live broadcast features of VR-360 video, a teacher or expert may be capable of joining a health care team in a remote location by simply donning an HMD and linking digitally with their headset.³⁵ The teacher or expert can then directly assist, in a digital “first-person” capacity, with procedures or decision-making during certain clinical crises to address specific clinical needs or offer learning opportunities. More work is warranted to determine the optimal method of coaching using learning theory in the context of these potential developments. Lessons can be learned from previous telesimulation-based studies and can be applied to immersive coaching. For instance, Burckett-St-Laurent et al³⁶ (2016) used 2-dimensional screen-based telesimulation software to teach ultrasound-guided regional anesthesia skills. They incorporated the theories of observational learning and deliberate practice into their design to have a positive learning impact on their learners. This can be taken further and, as an example, developed into a virtual immersive teaching tool with which coaches and learners (in different geographical locations) can be immersed in the “same room” using VR. This way, the learner can simultaneously observe the expert demonstrating the skill and then practice under their direct supervision. This work is currently being done with ongoing investigations at our tertiary academic center of the University of Toronto.

Furthermore, teaching communication skills using immersive and interactive virtual environments can potentially be a valuable tool.³⁷ For instance, in one study, learners and practicing primary care providers interacted with avatars, and changes in simulated prescribing behaviors were seen in some of the primary care providers, despite participants knowing they were interacting with simulated patients.³⁸ Thus, this can be applied to anesthesia clinic-based teaching curriculums. Virtual patients can be voiced by live facilitators in remote locations or respond to live learners when practicing communication techniques such as difficult conversations or history taking. However, even beyond this, technology is evolving so that computer-controlled patients can detect and respond appropriately to a live learner’s communication technique with the appropriate sentiment.

An important and often overlooked issue in the future of using immersive environments for teaching is a lack of diversity in anesthesia-based simulation training and the lack of diversity training as a whole. The literature has revealed that simulation technology has been limited in presenting the full spectrum of diversity seen in the general patient population.^39,40 The use of VR, in which an avatar’s appearance and responses can be easily altered, may allow clinicians to understand diversity and the culture in which they practice. A relatively simple but profound example would be learning intravenous cannulation in a dark-skinned patient. The potential for immersive VR-360 videos or entirely digitized interactive environments is significant in diversity training. The presence and the sense of immersion that these technologies provide may allow for a more empathetic perspective for the learner who can more realistically “walk in the shoes” of an individual from a very different background. Outside of medicine, it has been seen that White participants exhibited less implicit and unconscious bias scores after taking part in immersive VR experiences during which they were asked to embody characters that were a different skin color than their own while completing routine tasks.^41–43 Furthermore, medical schools have used VR to allow their trainees to experience what their patients face when it comes to their illnesses and how they navigate the health system.⁴⁴ This is a form of empathy training to familiarize medical students with information resources related to the health of older adults. Overall, through 360° videos, creating digital avatars, donning an HMD, and taking on the role of marginalized individuals and patients, health care providers can begin to take steps toward “feeling” what these groups experience.

PATIENT EDUCATION

In addition to medical education for health care workers, IVEs are beginning to disrupt clinical anesthesia management and health care delivery vis-à-vis patient engagement and education. IVEs show promise as interventions to reduce pain and anxiety and provide stress relief to patients.

Studies have shown that IVEs viewed through HMDs can be an effective supplement to analgesic regimens of adult and pediatric patients undergoing wound dressing changes in burn management, in early labor pain, for dental procedures, intravenous insertion, procedures requiring sedation such as endoscopy, and for acute ischemic pain.^45–50 Using IVEs for pain management has been mainly based on the principle of distraction through the sense of presence in another nonpainful or nonstressful environment.⁵⁰ Functional magnetic resonance imaging studies have graphically revealed the additive effect on pain relief using distraction-based IVE.⁵¹ Currently, there are more questions than answers in this nascent field, and the issue of how to properly develop “pain curriculum”-based tools remains uncertain. To achieve complete “distraction,” the patient should feel “present” in the simulated IVE they are viewing with the HMD. Questions of how best to achieve this using, for instance, computer-generated environments versus real-world VR-360 camera film, remain to be investigated. In addition, does the “dose” of IVE correlate to the sense of presence and realism of the IVE? In other words, how capable is the IVE of fully engaging the subject’s mind, so it no longer relates to its actual location and becomes wholly engrossed within the new reality presented to it?

Importantly, as with any pharmacological intervention, the optimal prescribing protocols for pain relief have yet to be delineated, and how their impact will vary with different presenting conditions must also be ascertained. Furthermore, most of these studies investigated acute pain conditions, and further research is needed to assess the optimal approach for using IVEs and their effectiveness in the chronic pain population. Commercial IVE software tools have been developed to target chronic pain specifically, but there is a dearth of literature and research in this specific area that needs to be addressed.

Closely related to pain management are the conditions of anxiety and stress. Up to 90% of patients experience anxiety during the perioperative period, and this anxiety has been correlated with increased morbidity and, remarkably, mortality in the postoperative period.⁵² Research in psychiatry has already demonstrated that IVEs can be used to treat anxiety and posttraumatic stress disorder as well as to reduce stress.^53–55

IVEs have been applied to the perioperative experience as a patient education tool. The literature has shown them to be an effective method to relieve anxiety and reduce stress for patients undergoing surgery.⁵⁶ This evidence has provided clinicians and educators with a tempting target for nonpharmacological interventions using exposure therapy and distraction therapy. IVEs have been used as a form of exposure therapy by educating patients about their hospital experience with virtual tours of ORs or first-person point-of-view films of the procedure before they even set foot in a hospital.⁵⁷ Other studies have used IVEs as a form of distraction to reduce patient anxiety during hospital admission and while undergoing procedures.⁵⁸ Trying to demonstrate whether these reductions in subjective and objective anxiety measures may translate into improved clinical outcomes will be an enormous logistical undertaking, and so it remains an intriguing goal.

Using the principles of accessibility and virtual presence described beforehand, future uses of IVEs in clinical management can leverage the advantage of dismantling geographic boundaries and connecting expertise to remote areas. The ability to virtually connect a physician with a remote patient and the added immersive first-person experience that traditional 2-dimensional video cannot offer can potentially change the practice of medicine significantly. Expert anesthesia providers located in urban hospital settings can use HMDs to assess rural patients for surgery without requiring either party to travel very far. Improving patient education and accessibility to care is another area in which VR may be of benefit. Immersive high-definition virtual broadcasting capabilities can allow health care providers to take part in a whole new manner of remote consultation for patients or postoperative care with family members who are not easily accessible. A provider can be immersed in the patient’s 3D environment to truly appreciate the patient in their environment and overcome geographical barriers. Similarly, an anesthesiologist may not have to work independently all the time while in the OR, as a colleague, mentor, or teacher can be transported into their virtual setting to offer “just-in-time” coaching and teaching to help with a spectrum of issues, such as a specific clinical question, an unfamiliar or daunting clinical scenario, as well as an unanticipated clinical crisis. Overall, multidisciplinary teams with representation from both remote and larger tertiary centers as well as from the virtual software development and telecommunication industry are needed to collaborate on determining the gaps in remote education and care. They can then determine the optimal approach to create and scale these solutions for underserviced areas.

In addition, IVEs may prove useful in the subject of brain health in the perioperative period. IVEs have been used in screening, diagnosing, and rehabilitation of memory deficits, and the neurocognitive literature has investigated the impact of IVEs on improving sleep patterns and cognitive impairments linked with age or trauma.^59–63 IVEs have also been implemented as strategies to aid neurorehabilitation.^53,64 A recent systematic review concluded that VR can be a tool for cognitive rehabilitation after brain injury.⁶⁵

Extending these study designs to assess problems of brain health in the perioperative period, specifically, the profound malady of postoperative delirium, is another tantalizing possibility. Postoperative delirium is a widespread and highly debilitating condition that has a significant impact on mortality and results in a significant material burden to hospitals in terms of patient length of stay and labor requirements.⁶³ IVEs are now being investigated as part of a multimodal approach in screening for and preventing cognitive impairments as well as treating delirium in critical care settings.^65–68 Ideally, as these studies evolve, anesthesiologists will be able to use IVE software in their clinics to stratify delirium risk preoperatively, educate patients and their families about risks, and then implement a protocol (potentially with IVEs as a structural element) postoperatively to prevent or treat delirium. Imagine virtually transporting a delirious patient back into their home, where they can interact with their native environment and virtually present family?

Overall, IVEs have the potential to be a positively disruptive technology in the anesthesia education and health care delivery fields. Development is happening at a rapid pace, fueled by the commercial industry. To ensure the optimal development of IVEs in the context of education, medical educators must guide this process using as its foundation sound learning theory and allowing for appropriate guidance by our end users: the learners.

ACKNOWLEDGMENTS

We thank Drs Lilia Kaustov, Julian Wiegelmann, Bill Kapralos, Paul McHardy, Jordan Tarshis, Stephen Choi, and Shelly Au, as well as Monica Caldeira and Kyle Wilcocks, for their help with our research programs. We are grateful to Khatidja Moloo for advanced edits of the manuscript.

DISCLOSURES

Name: Fahad Alam, MD, MHSC, FRCPC.

Contribution: This author developed the initial idea and wrote the manuscript.

Name: Clyde Matava, MBChB, DA, MMed.

Contribution: This author completed advanced edits.