VR Simulation leads to enhanced procedural confidence for surgical trainees

Abstract

Objectives:

Active learning techniques result in greater knowledge acquisition compared to passive methods. For medical students with limited hands-on operative experiences, virtual reality (VR) platforms represent active learning and may enhance procedural training. We hypothesize that VR simulators like TIPS (Toolkit for Illustration of Procedures in Surgery) are a more effective modality in teaching laparoscopic surgical techniques to medical students when compared to passive learning tools like videos.

Design:

In this crossover study participants were randomly assigned to perform either a TIPS laparoscopic appendectomy followed by video of a laparoscopic cholecystectomy, or video of a laparoscopic appendectomy followed by TIPS laparoscopic cholecystectomy. A knowledge assessment followed each intervention. A post-survey was used to gather feedback and subjective impressions of the learning experience.

Setting:

University of Central Florida College of Medicine (UCF COM).

Participants:

Second, third, and fourth-year medical students (n=37).

Results:

Validation of the content assessments revealed strong internal consistency (Cronbach’s α = 0.73 ). A two-tailed Fisher’s exact test revealed that the video had greater ease of use ( p = 0.032 ), but TIPS had greater utility as a learning tool ( p < 0.001 ) and instilled greater confidence in the ability to reproduce procedural steps ( p < 0.001 ). A two-tailed T-test of the average content quiz scores revealed no significant difference in percentage correct between groups on the laparoscopic appendectomy quiz ( p = 0.772 ), but a difference favoring video learning on the laparoscopic cholecystectomy quiz ( p = 0.042 )

Conclusion:

Video and TIPS both enhanced different aspects of student learning; however, the active TIPS platform produced greater confidence in the ability to reproduce the steps of the procedure and had greater utility as a learning strategy. Videos are simple to use and can serve a complementary role in curriculum design.

Introduction

As minimally invasive surgical techniques become more popular across all surgical specialties, there arises a concordant need to appropriately train residents and medical students to master these skills. Studies of visuospatial and psychomotor learning curves reveal distinct levels of learner aptitude for these techniques¹. Trainees, regardless of aptitude, could benefit from deliberate practice models to help hone these skills². Simulation is appealing as it reduces reliance on patients, cadaveric, and animal models for skills practice and acquisition. Historically, these skills have been refined under the supervision of a surgical mentor³, but as mentorship opportunities have decreased, a larger emphasis has been placed on the use of alternative teaching modalities including textbooks, anatomical atlases and training videos⁴; while valuable, these resources are limited in their ability to demonstrate nuanced information such as the texture of different types of tissue and necessary force to be exerted during dissection⁵. Additionally, it has been repeatedly shown that active learning techniques result in greater knowledge acquisition and retention than passive methods^6,7. Confidence in ones’ abilities alone have proven to enhance the learning process^8,9, and can be improved by using simulations prior to the transfer of skills to actual patients.

Virtual reality (VR) training can be a valuable tool in the development of technical skills while minimizing risk to patients, particularly for rare conditions that the learner might not encounter during their training¹⁰. Given that laparoscopic surgery has a longer learning curve than open surgery, it would be beneficial to give trainees interested in procedural specialties the opportunity to develop their skills^4,11–19. As graduating residents consistently report not only decreased time in the operating room but also feeling under-prepared for independent practice, it is important to provide modes of supplemental practice ^16–20. We hypothesized that a VR technology-based procedural experience would complement commonly available educational videos leading to increased confidence and knowledge. Understanding the role of each of these technologies could inform curricular development efforts.

Materials and Methods

We aimed to test two different learning modalities – hands-on active learning vs. passive video learning. We wanted to reduce the chance that students struggled with the procedure due to inadequate knowledge of the anatomy, so participation was limited to only 2^nd through 4^th year medical students at the UCF College of Medicine who had successfully completed the 1^st year anatomy course. Power analysis was based on an assumption of tolerance for type I α error of 0.05 and a type II error, or β power, of 80. The effect size was determined after developing the assessment. The research team concluded that a 5% difference in the assessment would yield the desired minimum required effect given the anticipated size of study cohorts. These calculations necessitated a sample size of at least 36 subjects.

The Institutional Review Board for UCF gave permission (UCF IRB # SBE-18–14544). Enrollment was made by inviting all 2^nd–4^th year medical students to participate via email and posting to the class-specific Facebook page. Thirty-seven students consented to participate. Students were told the experiment would take no longer than one hour to complete and would be compensated for their time with a $20 giftcard.

This experiment employed the use of the TIPS (Toolkit for Illustration of Procedures in Surgery) VR software. TIPS has been developed as part of a longstanding collaboration between authors (JC and JB); it is an interactive surgical illustration platform which allows the user to engage with anatomic renderings using what appears to be laparoscopic instruments²¹. The user engages the environment using two three-dimensional (3D) haptic drawing pens, which act as facsimiles for laparoscopic instruments (see figure 1). Participants can choose from several common laparoscopic instrument tips (scissors, Maryland dissector, etc.) on either hand by toggling a single button. Haptic feedback is generated when the participant engages with an organ or tissue.

Figure 1.

A Student Participant Utilizing the TIPS Simulation

TIPS does not provide step-wise instructions for the user; directions for procedural completion were printed showing the surgical sequence and matching screen shots taken from the TIPS program. The other learning modality utilized in this experiment is the WISE-MD²² videos. These videos are used as online learning tools currently at the UCF College of Medicine, as well as other learning institutions, and display step-by-step directions on how surgical procedures are completed with real life video paired with auditory voice-over and some animations. The laparoscopic appendectomy WISE-MD video was 1 minutes 11 seconds in runtime, and the laparoscopic cholecystectomy video was 2 minutes and 30 seconds long. Students were blocked for hour-long sessions for the TIPS interaction; exact time-on-task was not measured though most sessions lasted about half hour.

In this experiment, students were assigned to a group in an alternating fashion based on their selection of a timeslot. Participants completed the first intervention, post-quiz, second intervention, post-quiz, and then a survey (see figure 2).

Figure 2.

Experiment Flow

The two post-quizzes were developed by the authors after review of the content covered in both the TIPS simulation and the WISE-MD videos to ensure equal opportunity for correct answers across the groups. As a form of psychometric assessment validation Cronbach’s alpha test can be viewed as the expected correlation of two tests that measure the same construct. It is a useful analysis of the internal reliability of an assessment; in this case the Cronbach’s alpha test demonstrated good internal reliability for the post-tests (α=0.73)²³.

In the survey portion of this experiment, students were asked for their feedback based on how much they liked using each learning modality, the ease of use of each, and other subjective questions based on a 5-point Likert scale. There was also a section where participants could provide any comments or suggestions.

Statistical analysis included descriptive analysis of demographics, as well as a two-tailed Fisher’s test for evaluation of 5-point Likert scale data²⁴. A two-tailed T-test was utilized to analyze the post-quiz data²⁵.

Results

For a total of 37 students, there were 19 in Group A and 18 in Group B. 67.6% of participants were male, and there were 15 2^nd year students (40.5%), eight 3^rd year students (21.6%), and 14 4^th year students (37.8%). Roughly a third of the students (13 students, 35.1%) reported they intend to pursue a surgical career. 62.1% of participants had already completed a surgical rotation during their 3^rd year and 67.6% had seen a laparoscopic appendectomy or laparoscopic cholecystectomy in person.

Participants performed overall well on the post-appendectomy quiz with no significant difference (81.6% for TIPS vs 83.3% for WISE-MD, p=0.772). However, the post-cholecystectomy quiz did demonstrate a difference in favor of the WISE-MD videos (84.2% vs 76.6%, p=0.042).

Overall, students favored the utility of TIPS as a learning tool compared to the videos (p=0.009) (Figure 3). This effect was particularly pronounced in the case of the laparoscopic cholecystectomy (p=0.001). However, participants had no significant preference between the two modalities for the laparoscopic appendectomy (p=0.536).

Figure 3.

Survey Questions

Students’ confidence in reproducing the procedure was assessed (Figure 3). Overall students reported higher confidence after completing the TIPS simulation (p=0.001). The overall effect was, again, largely driven by the large impact shown by TIPS in laparoscopic cholecystectomy training (p=0.002), with less effect noted in the laparoscopic appendectomy scenario (p=0.146).

As a summary question, participants were asked to report which modality they would prefer learning surgical techniques from and 83.8% (31 of 37) reported they would prefer using the TIPS simulation.

Assessment of ease of use of the technology revealed a preference towards WISE-MD (p=0.032) (Figure 3). Specifically, participants reported that the WISE-MD video was easier to use than the laparoscopic appendectomy TIPS simulation (p=0.004); there was no significant difference in the reported ease of use for the laparoscopic cholecystectomy TIPS simulation (p=0.507). It is worth noting that 43.2% of participants reported having technical difficulties while using the TIPS simulation. Research staff were available at all times during the investigation to intervene in typically minor technical issues.

Conclusion

In this study, we aimed to examine the TIPS simulation software and evaluate its potential place in the learning environment for surgical trainees. When asked whether participants thought the TIPS simulation had good utility as a learning tool and whether they could confidently verbally reproduce the procedural steps after using the simulation, collectively the response was in favor of the TIPS simulation. Not only did the TIPS simulation appear to be a useful adjunct to the learning environment, but it improved student confidence. Confidence, itself, is a vital component to the learning process^8,9. Educators commonly seek ways not only to improve testable knowledge, but improve learners’ confidence in their skills.

Participants felt more confident in their ability to verbally reproduce procedural steps after using the TIPS simulation, but this did not translate directly into improved post-quiz scores. In fact, there was no difference for the laparoscopic appendectomy post-quiz, while a meaningful difference was noted in the laparoscopic cholecystectomy post-quiz in favor of the WISEMD videos. The videos were roughly 1.5–2.5 minutes long and paired not only real-life visuals with animations, but had a matching voice-over, pairing auditory and visual learning. The TIPS simulation paired only visual and kinesthetic learning, where participants followed step-by-step written directions with screen shots from the simulation as guide posts. Interestingly, the shorter video (laparoscopic appendectomy) did not have significant differences in post-quiz scores, while the longer video (laparoscopic cholecystectomy) did, which could demonstrate that the longer the auditory and visual reinforcement, the more likely you were to perform better on knowledge-based questions. During medical school, students’ education is nearly entirely auditory and visual for the first two years of the curriculum, so students are far more used to learning from these methods when compared to the kinesthetic learning they begin experiencing as they progress through their third year. The TIPS simulation software could be improved by integrating auditory learning with the other two modalities. This could be accomplished by adding spoken instructions that triggered at the completion of the previous procedural step.

With nearly half of the participants experiencing some form of technical difficulty while using the TIPS simulation, it comes as no surprise that participants reported the WISEMD videos as easier to use. Again, though, when broken down by individual procedure, there was not a significant difference when comparing ease-of-use between WISEMD videos and the TIPS simulation for the laparoscopic cholecystectomy procedure. We experienced some difficulties with platform stability of the new software, as well as instrument calibration and orientation, which likely reflects these results. The laparoscopic appendectomy TIPS procedure, especially, proved more technically challenging to participants. There are improvements that need to be made to the TIPS simulation software to improve usability and decrease technical difficulties, despite these participants still preferred learning via the TIPS simulation.

Also collected was free text written feedback from participants. Common themes derived from these comments include technical difficulties, the learning curve associated with the TIPS simulation, enjoyment with using the TIPS simulation, and preference for combination of the WISEMD videos with the simulation. One participant reported that the video moved too quickly through the steps for adequate knowledge retention, so they relied on previous knowledge of the procedure to answer the post-quiz questions. This brings up an interesting potential confounding factor for the results of the post-quizzes but does not change that participants generally preferred utilizing the TIPS simulation as a learning tool. Many reported that the video was very concise, but they would have liked to have been able to repeat it for better knowledge solidification. Overall, students enjoyed using the TIPS simulation once they overcame the learning curve. A vital point echoed by many participants was the suggestion to integrate both modalities: “maybe getting to read through the steps, then watching a video, and then using the simulator would have been most effective.”

Through this experiment we evaluated a new learning tool for hands-on teaching of laparoscopic techniques when compared to traditional passive learning from a video. There are clearly merits to both methods of learning and the greatest improvement on learning, confidence, and eventually performance may be the integration of the two instead of either individually.

HIGHLIGHTS.

• Virtual reality is an emerging method to supplement surgical education

• TIPS is a simulation that allows practice of laparoscopic techniques on a computer

• When compared against videos, participants preferred learning using TIPS

• Participants also had greater confidence in their learning after using TIPS

• Virtual reality trainers like TIPS can complement existing educational platforms