Innovations Through Virtual Reality Simulation

Abstract

The need to augment standardized learner outcomes related to performance and clinical competency led to creating curricular elements that would provide instruction and assessment from multiple perspectives. The COVID-19 pandemic brought about needs for re-imagination of standardized simulated clinical experiences given the need for increased distance-learning and asynchronous formats. Our goal was to identify activities that would engage pre-clinical simulation through asynchronous virtual reality (VR) case scenarios. The intent was to provide additional resources whereby competencies could be more defined through performance metrics and standardized assessments additive to our established simulation-based curriculum throughout all curricular phases. Student reflection and metacognition identified gaps to guide future performance improvement through the VR activities. Learner outcomes encompassing history-taking, physical assessment, evidence-based clinical reasoning, and medical decision-making guided the instructional objectives. The composite data showed progressive improvements over five scenarios delivered in our second-year clinical medicine curriculum.

Challenges

Take a moment to reflect on the following question: “How can we teach medical students to become physicians without them having the opportunity to touch a patient?” It is felt that in our collective experience this notion is unrealistic. Yet, given the COVID-19 pandemic, with its ongoing tumultuous associations, we have stepped into challenges surrounding such questions that were otherwise beyond our comprehension.

Many medical school curriculums across the globe were faced with similar challenges in looking at facilitating transitions to distance-education where curriculum delivery took on the form of remote, virtual, synchronous, and asynchronous deliveries. Innovations and re-imaginative processes were suddenly thrust into our planning and strategy. The rapidity with which these aspects needed to reach a form of implementation was something many have not experienced. The Kansas City University College of Osteopathic Medicine (COM) was faced with the task of realigning our curriculum to maintain the goal of optimal medical education delivery during the pandemic while addressing needs across all programmatic phases of our institution.

Beyond our concerns in augmenting our didactic activities to distance-learning formats, we realized the enormous need to align our simulation-based activities given the importance of “in-person” and “hands-on” modalities that defined our norms at the time. Faculty derived questions such as, “How do we move our present activities to provide meaningful remote activities that propagate learning within clinical skills development?” We likewise sought to construct appropriate assessments in determining clinical skills performance within remote environments. How would our assessments need to be developed as we transitioned to remote, or virtual, environments?

The purpose of this project was to consider ways the current simulation-based activities could be augmented through VR simulation. The intent was to consider developing VR with implementation not only in the pre-clinical years, but for those in clinical clerkship rotations as well. With the pandemic, we saw students pulled out of their clerkships putting them in a position whereby they were unable to continue their journey with real patient experiences. Our pre-clinical students were placed in a distance format where the usual clinical skills development involving direct contact with standardized patients became a Zoom room of activities promoting their development- without ever touching a patient.

Additional challenges surrounded our ability to establish standardized assessments which would appropriately identify performance outcomes and competencies throughout our entire program. These assessments are integral to the functionality of the virtual reality simulation as we worked to integrate all our simulation-based activities.

Approach

Our move to develop the VR project was based on a needs assessment where we partnered with Oxford Medical Simulation (OMS) in providing a virtual reality simulation platform. This commercial entity allowed for integration of VR scenarios with options allowing for VR headsets as a potential immersion experience. Once constructing a plan of deliverable elements, it was decided to pilot the OMS platform to objectively establish student responses as we felt integrating an activity that propagated student engagement would be paramount in continuing skills development.

The Scenario

It is useful to understand how these VR experiences are conducted as we compare its utility with real-world patient experiences. Remember, the students need to have a clinical experience that will simulate a live patient encounter since they are not able to be in front of an actual patient. Playing the role of a medical student takes us to our computer or laptop. We log in to the system and once loaded, we choose our assigned scenario. The screen changes to a clinical setting, commonly based in an Emergency Room bay. A nurse approaches you and explains they have triaged a patient who is febrile and does not look well. You move to the bedside and click on the patient’s head which opens a communication pick list giving options for questions that can be asked of the patient. You choose to ask them “How are you feeling?” The patient responds with some mumbled words where they basically ask where they are. You then turn to the nurse and click to bring up a menu for nursing tasks such as placing the patient on a monitor to obtain vital signs while choosing the task to start a peripheral IV and start a normal saline bolus at 500 ml/hr. You ask for an electrocardiogram and basic laboratory samples for complete blood count, comprehensive metabolic panel, lactate level, and a urinalysis. Additionally, you ask for a point-of-care glucose since you recognize the patient has altered mentation.

Having set up these tasks, you move to the computer in the room and click to pull up the patient’s electronic medical record where you find they have a diagnosis of testicular cancer and type I diabetes mellitus. The home medications are reviewed, and you find the patient has just finished a round of chemotherapy one week prior.

As you turn back to the monitor, you realize the patient has a temperature of 101.3 °F (38.5° C) and displays tachycardia and tachypnea. It is decided you want to perform an initial physical exam by clicking on the patient’s chest where a menu comes up allowing you to choose the system you wish to examine. The platform will then give you heart and lung sounds as you listen. As you palpate and inspect, the platform verbalizes those findings as they are examined.

By this time, you can go back to the computer and pull up the laboratory results ordered. The patient has leukopenia, acute kidney injury, and an elevated lactate level of 4.0 mEq/L. You are concerned for sepsis; thus, in going back to the computer you navigate to guidelines in treating sepsis and find a list of appropriate antibiotics. You choose the antibiotics you want given and the nurse responds, “Those medications have been given.”

At this point, available options are to order diagnostic imaging such as a chest x-ray while you can update the nurse on your plan as well as choosing to discuss things with the family. There is an option to call the attending and report on the patient’s presentation and receive further guidance as necessary.

As you come to a point where you exit the room, the platform takes you to a reflection phase where you, as the medical student, can evaluate the experience. Once that is complete, the scenario goes into a debriefing mode where you can see the objectives for the scenario giving the clinical summary involving neutropenic fever and sepsis. Figure 1 illustrates the important technical and non-technical tasks that you should have performed during the experience (Figure 1).

Figure 1.

Technical and Non-Technical Tasks

After reviewing the scenario, you move down the page to your performance scores. The platform gives you a green checkmark for those things you addressed appropriately while adding a red checkmark for the tasks you missed. There is an option to click on each performance task where you can review rationales and references as to why these items are important.

Phased Implementation

To provide a structured environment in which to implement this activity, we developed three phases to move toward effective curriculum integration with our simulation-based activities. Phase I consisted of introducing the scenarios to our second-year clinical medicine curriculum by choosing three scenarios per semester. The initial scenarios were implemented using student reflective responses while monitoring performance metrics. We experienced some degree of hesitation in that many of the scenarios were developed for residency-level training. With that in mind, we developed pre-session fact sheets for the students to review along with a platform orientation given its technical elements in navigating the scenario environment. The VR scenarios were made available through asynchronous learning via the web-based OMS platform.

Phase II added formal synchronous faculty-facilitated simulation debriefing sessions in small group remote environments to explore student recognition of learning gaps through metacognitive aspects while continuing individual reflective processes associated with each scenario experience.

Phase III will serve to define how the VR platform can be utilized outside of the second year, which is especially important to our third- and fourth-year programming with students in clinical rotations. Delivering clinical education in a distributive model, where our students are positioned all over the nation, serves as a driving force in providing those standardized processes in assessments, outcome designations, and performance levels in formulating objective evaluation of clinical skills throughout our curriculum.

Results

The focus of Phase I included student reflective responses about their experiences as they worked through the assigned scenarios. We were able to collate performance data from the OMS platform which served to identify overall student progress based on algorithmic results tied to various tasks required for completion within each scenario as described above. The thematic representations brought out student responses such as “stress,” “think,” “learn,” and “good.”

We stated previously that our intent was to create a simulation-based activity that would meet expectations of learners and establish appropriate learning environments with elements that would propagate student motivation. In comparing the overall student satisfaction, we realized a progressively positive approval rating between our first and last scenario experience within the first semester (Figure 2).

Figure 2.

A progressively positive approval rating is illustrated between our first and last scenario experience within the first semester.

Performance metrics were evaluated in stages of first and second attempts. It was felt there were issues with navigating the virtual platform whereby students had to learn where various resources were located within the environment. To determine issues with performance results, we elected to appraise the various attempts for purposes of evaluating the activity regarding further development, delivery, and assessment.

The means of the performance metrics were established through the OMS platform in our overall data retrieval. The scenario outcomes were graded through scenario specific items required in each case related to history-taking, treatment, and diagnosis. These tasks required transfer of biomedical knowledge, processes in critical-thinking and problem-solving, and medical decision-making for successful completion. Students could work through the scenario as often as they desired which we felt would take the form of deliberate practice and cognitive assimilation. The overall scores in the first, second, and third VR scenarios were 61%, 74%, and 77% respectively. In review of these scores, we were able to assess areas of performance within the second year with plans to extrapolate these outcomes for our third- and fourth-year experiences. Future development will likely involve constructing our own scenarios whereby appropriate competency-level learning can be highlighted.

Phase II of our project is not yet completed at the time of this paper; however, preliminary comparisons have been noted between the overall mean scores for those scenarios completed in the first and second semesters showing an average of successful performance of 66% in the first semester with 79% in the second semester.

In performing a two-tailed t-test of the means comparing our first semester performance scores with those from the second semester, there was significance with a t value equal to 1.96 (p = 0.004). While this shows positive progression, the significance related to improvement in competencies has yet to be determined. Our intent is to further define the data points related to individual tasks to make determinations of specific competencies and their performance levels.

Conclusions

Our goal in this project was to identify activities that would engage pre-clinical simulation through asynchronous virtual reality (VR) case scenarios. The intent was to provide additional resources whereby competencies could be more defined through performance metrics and standardized assessments additive to our established simulation-based curriculum throughout all curricular phases. We chose to integrate the virtual reality simulation through a pilot study into the curricular programming and obtain student responses. Given our student-centered approach, it was felt the VR activity would be a positive addition for student learning during transitioning to distance-learning formats due to the COVID-19 pandemic. To evaluate the VR simulation piece within our current simulation-based activities, we integrated the VR simulation platform into our existing second-year clinical medicine course to further assess and evaluate potential learning enhancements in a controlled environment. The impetus relates to how this activity could then be used in other phases of our curriculum, specifically as it relates to students rotating through clinical clerkships.

The pandemic experience has required creative thinking and innovations in how we deliver medical education. Faculty felt that more robust simulation activities using VR simulation would allow for greater opportunities in clinical skills development, and additional meaningful and effective assessment tools for outcome measures throughout the COM.

Our initial data shows preliminary favorable outcomes. Outcomes will need to be assessed through data mining related to individual competencies. Further development of the virtual reality simulation will require development of specific tools that allow for accurate assessment and evaluation of performance levels and their associated learning dynamics.