A Pilot Study of Surgical Training Using a Virtual Robotic Surgery Simulator

Training on a virtual reality robotic simulator dry lab robotic surgery platform resulted in significant improvement in time to completion and economy of motion for novice robotic surgeons.

Abstract

Background and Objectives:

Our objectives were to compare the utility of learning a suturing task on the virtual reality da Vinci Skills Simulator versus the da Vinci Surgical System dry laboratory platform and to assess user satisfaction among novice robotic surgeons.

Methods:

Medical trainees were enrolled prospectively; one group trained on the virtual reality simulator, and the other group trained on the da Vinci dry laboratory platform. Trainees received pretesting and post-testing on the dry laboratory platform. Participants then completed an anonymous online user experience and satisfaction survey.

Results:

We enrolled 20 participants. Mean pretest completion times did not significantly differ between the 2 groups. Training with either platform was associated with a similar decrease in mean time to completion (simulator platform group, 64.9 seconds [P = .04]; dry laboratory platform group, 63.9 seconds [P < .01]). Most participants (58%) preferred the virtual reality platform. The majority found the training “definitely useful” in improving robotic surgical skills (mean, 4.6) and would attend future training sessions (mean, 4.5).

Conclusion:

Training on the virtual reality robotic simulator or the dry laboratory robotic surgery platform resulted in significant improvements in time to completion and economy of motion for novice robotic surgeons. Although there was a perception that both simulators improved performance, there was a preference for the virtual reality simulator. Benefits unique to the simulator platform include autonomy of use, computerized performance feedback, and ease of setup. These features may facilitate more efficient and sophisticated simulation training above that of the conventional dry laboratory platform, without loss of efficacy.

INTRODUCTION

Recent advancements in surgical technology, including the introduction of robotic-assisted laparoscopic surgery, afford gynecologists the ability to treat an increasing number of patients with minimally invasive surgical approaches. Robotic-assisted laparoscopic surgery has emerged as an essential minimally invasive technology with several purported advantages over conventional laparoscopic surgery.^1–4 Published reports suggest that robotic surgery may allow more surgeons to perform complex gynecologic procedures and may lead to improved patient outcomes compared with conventional laparoscopy or laparotomy.^5–8 Adoption of this surgical approach is rising rapidly, with almost 50% of US robotic procedures performed by gynecologists.^9,10 Given the relative merits of robotic surgery and its widespread adoption, it is paramount that gynecology residency and fellowship programs offer comprehensive training in this modality for their trainees.¹¹

In a recent survey of US obstetrics and gynecology residency program directors, most believed the use of robotic surgery would increase and play a more integral role in gynecologic surgery.¹² However, the question of how to best introduce new surgical technology in the operating room and safely instruct novice surgeons is challenging. Other issues, including the recent limitation of US resident work hours and decreased hysterectomy rates, compound this problem.^13,14 Surgical training involves the development of technical skills that are traditionally acquired through a Halstedian apprenticeship model in the operating room, as well as the development of cognitive and clinical skills.¹⁵ Fewer mentoring opportunities and decreased surgical volumes have led to the use of online tutorials, models, and surgical skills laboratories to replicate surgical situations. However, most of these methods are limited in their ability to simulate live surgery, particularly for robotic procedures. In the past decade, high-tech surgical simulators have been developed that have revolutionized the approach to surgical education and may help overcome many of these challenges. For instance, the da Vinci Surgical System (dVSS) (Intuitive Surgical, Sunnyvale, CA, USA) can be used in conjunction with training models to create a dry laboratory platform that allows trainees to practice various robotic skills before operating on patients.

Perhaps the most exciting development in simulation science is the advent of computer or virtual reality (VR) surgical simulators. The most sophisticated of the available robotic surgical simulators is the da Vinci Skills Simulator (Intuitive Surgical), which uses Mimic VR training software (MdVT) (Mimic Technologies, Seattle, WA, USA). Hung et al.¹⁶ recently demonstrated the face, content, and construct validity of this novel VR simulator. The da Vinci Skills Simulator contains >35 training exercises including modules that test basic skills such as camera and clutch control, as well as more advanced tasks such as energy control, needle driving, and suturing. The modules were designed to be applicable for surgeons of any subspecialty; however, the needle-driving and suturing exercises are particularly relevant to gynecologists, who perform these tasks routinely in gynecologic surgery. Therefore the study objectives were to compare the utility of learning a specific suturing task on the da Vinci Skills Simulator versus training on the dVSS dry laboratory platform among novice robotic surgeons and to determine the attitudes of gynecologic trainees regarding the different training platforms.

METHODS

Participants

This was a single-institution, institutional review board–approved prospective pilot study performed at Greater Baltimore Medical Center and Johns Hopkins Hospital (Baltimore, MD, USA). Medical students and gynecology trainees (postgraduate years 1–6) who had participated in <10 robotic-assisted surgeries were enrolled. Participants were assigned to train on either the da Vinci Skills Simulator with MdVT or the dVSS dry laboratory platform.

Training Platforms

The da Vinci Skills Simulator is a VR simulation system developed as a collaborative effort by Mimic Technologies and Intuitive Surgical. The da Vinci Skills Simulator uses MdVT and contains a variety of exercises designed for users to improve their proficiency with the da Vinci surgeon console controls. These exercises are organized by the following skill categories: EndoWrist manipulation, camera and clutching, fourth-arm integration, system settings, needle control and driving, and energy and dissection. The simulator, which is packed in a portable case, or “backpack,” measures 57.2 cm × 60.3 cm and attaches directly on the back of the dVSS Si or Si-e surgeon console (Intuitive Surgical) so that the console can be used for VR training without the need for the patient-side cart or instruments (Figure 1). No additional system components are required.

Figure 1.

da Vinci Skills Simulator shown attached to da Vinci Surgical Si console.¹⁷

The dVSS dry laboratory platform involves the use of all the components of the dVSS, which include the surgical console, the patient-side cart, EndoWrist instruments, and the vision cart. All of the components are set up as they would be for live surgery, with a training model in place of a live patient. Various training models are available, such as The Chamberlain Group's Robotic Oval Hex Pod model and Robotic Sea Spikes model (The Chamberlain Group, Great Barrington, MA, USA). A demonstration of the dVSS dry laboratory training platform setup is shown in Figure 2.

Figure 2.

dVSS dry laboratory training platform setup.

Pretest and Post-Test

Before and after completion of the training session, both groups completed the same pretests and post-tests, consisting of suturing an incision in a vaginal cuff dry laboratory model using a delayed absorbable, barbed suture on the dVSS dry laboratory platform (Figure 3). Three targets were placed along each side of the incision, each approximately 1 cm apart. The testing model was distinct from the training exercises for both groups, and no participants were given the opportunity to practice on the testing model. Suturing was selected as the testing task because it is a complex but integral skill required in many gynecologic procedures. Performance on the pretests and post-tests was monitored and recorded by 2 members of the research team watching the participants (A.I.T. and A.N.F.). Metrics measured included time to complete the task, number of instrument movements and instrument collisions, number of instances when instruments were out of view, and number of missed targets. Participants were timed from when they picked up the needle with the needle driver to when they passed through the last target and pulled the suture for a tight closure. A “missed target” was defined as passing the tip of the needle through an area that was not demarcated on the vaginal cuff model.

Figure 3.

Vaginal cuff model used for pretests and post-tests (inferior incision).

Training Sessions

The MdVT cohort trained on the VR simulator for 60 minutes using a needle-driving exercise known as “suture sponge level 1.” This is an exercise in which the participant sits at the robotic console and drives the needle through entrance and exit targets on a virtual sponge (Figure 4) using both hands. The exercise required the participant to drive the needle at many different angles and with the left and right hands. After completion of the exercise, the participant was given a detailed performance report, an example of which is shown in Figure 5. Measured metrics included time to complete exercise, economy of motion, instrument collisions, excessive instrument force, instruments out of view, master workspace range, drops, missed targets, critical errors, and overall score. These metrics were used to provide feedback to the participants.

Figure 4.

Screen shot of da Vinci Skills Simulator “suture sponge level 1” needle-driving exercise.

Figure 5.

Example of da Vinci Skills Simulator performance evaluation report.¹⁸

The training session for the dVSS group was also 60 minutes and consisted of suturing around the perimeter of geometric shapes on a dry laboratory model (Figure 6) distinct from the testing model. A proctor was present at all times and provided performance-improvement feedback to participants in both training cohorts.

Figure 6.

Robotic Oval and Hex Pod model (The Chamberlain Group) used for dVSS dry laboratory platform training session.¹⁹

The dVSS dry laboratory platform required the presence of a scrub technician, Intuitive Surgical representative, or an attending physician with knowledge of the robotic system to set up the robotic arms in the appropriate configuration, install the appropriate instruments, white-balance and calibrate the camera, oversee the training session, and undock the instruments and power down the system at the completion of the training. Conversely, the MdVT session required the surgeon console with the attached backpack and powering the console and simulator on and off.

Survey

Finally, the participants completed an online anonymous survey via the SurveyMonkey Web site. The survey assessed perceived clinical utility of the simulator platforms and queried participants regarding their overall experience with the training sessions and satisfaction with the training modalities. For the purposes of assessing satisfaction with the MdVT, the dVSS group was allowed to spend time using the VR simulator after completion of the post-test.

Statistics

Performance metrics were analyzed with the Student paired t test, and P < .05 was considered statistically significant. Statistical analysis was performed with Stata statistical software, version 11.1 (StataCorp, College Town, TX, USA).

RESULTS

Twenty trainees participated in the study: 10 medical students and 10 obstetrics and gynecology residents or fellows. Table 1 shows participant characteristics. Regarding previous surgical experience, 80% of the participants had no previous robotic surgery experience and 50% also had no experience with laparoscopy.

Table 1.

Participant Characteristics

Characteristics	Data
Mean age (SDa), yr	29 (4.8)
Gender
Female	70%
Male	30%
Level of training
Medical students	7
Junior residentsb	5
Senior residentsc	5
Fellowsd	3
Mean no. of cases (SD)	1.9 (3.6)

^aSD = standard deviation.

^bFirst- and second-year obstetrics and gynecology residents.

^cThird- and fourth-year obstetrics and gynecology residents.

^dFirst-year obstetrics and gynecology fellows.

The results of the pretests and post-tests for the 2 simulation groups are shown in Table 2. The mean pretest completion times were comparable between the MdVT and dVSS groups (189.8 seconds and 235.0 seconds, respectively; P = .390). Training on both the MdVT and the dVSS was associated with a significant and comparable decrease in time to completion of the tasks (64.9-second decrease in time to completion for MdVT [P = .044] and 63.9-second decrease for dVSS [P = .002]). An improvement in the number of movements used to complete the task was seen in both groups (mean of 12.1 fewer movements in the MdVT group [P = .022] and mean of 17.3 fewer movements in the dVSS group [P < .001]). However, training on the dVSS, but not the MdVT, was associated with a significant decrease in the number of missed targets (0.8 fewer missed targets for MdVT [P = .153] and 1.1 fewer missed targets for dVSS [P = .007]). Similarly, training on the dVSS, but not the MdVT, was associated with a significant decrease in instances of instruments out of view (0.4 fewer instances for MdVT [P = .443] and 0.7 fewer instances for dVSS [P = .045]). The mean number of instrument collisions for the pretest was less than 1 for both groups, and therefore no meaningful improvement occurred as a result of the training sessions. Subjectively, setup for the dVSS dry laboratory training sessions required substantially more time and effort and the presence of a proctor with a high working knowledge and familiarity with the robotic system than setup of the MdVT simulation session. As described earlier, there were many more metrics measured with the MdVT that could not be calculated or measured with the dVSS and therefore were not recorded for the purpose of comparison of performance between the 2 groups.

Table 2.

Pretest and Post-test Results for MdVT and dVSS Groups

Metric	MdVT Group				dVSS Group
	Pretest	Post-test	Difference	P Value	Pretest	Post-test	Difference	P Value
Time to completion, s	189.8	124.9	64.9	.044a	235.0	171	63.9	.002a
Instrument movements (no.)	43.7	31.6	12.1	.023a	54.0	36.7	17.3	<.001a
Missed targets (no.)	2.3	1.5	0.8	.153	2.5	1.4	1.1	.007a
Instruments out of view (no.)	1.1	0.7	0.4	.443	1.1	0.4	0.7	.045a
Instrument collisions (no.)	0	0.1	−0.1	.343	0.7	0.2	0.5	.052

^aStatistically significant.

Almost all of the participants (19 of 20) completed the 23-item survey. On a 5-point Likert scale, most participants found the training “definitely useful” in improving robotic surgical skills (mean, 4.67) and would “definitely” attend future training sessions (mean, 4.5). Respondents believed that the MdVT improved hand-eye coordination (95.8%), dexterity (100%), and instrument multitasking (87.5%) and that the VR software provided high-fidelity, realistic training scenarios (87.5%). Specifically, the respondents believed that the following aspects of the simulator were realistic: instrument movement (mean, 4.1), depth perception (mean, 3.8), and interaction of instruments with other objects (mean, 3.7). We asked the obstetrics and gynecology trainees only how important they consider training in robotic surgery; these respondents believe that training in robotic surgery is “extremely” important (mean, 4.6). Regarding user preference, most participants (58%) preferred the MdVT training platform, with 33% preferring the dVSS training platform and 8% having no preference. Interestingly, when queried regarding whether they had had any adverse effects from the 60-minute study training sessions, 42% of the participants reported mild hand and wrist soreness, 16% reported neck or back pain, and 11% reported a transient headache.

DISCUSSION

This is one of the first prospective studies in gynecology comparing the utility of a VR robotic simulator with that of a standard dVSS dry laboratory platform in the training of gynecologic residents and fellows. In this pilot study, training on either the VR da Vinci Surgical Simulator or the dVSS dry laboratory platform resulted in significant improvements in technical performance for novice robotic surgeons, including a decrease in time to completion and number of instrument movements. Training on the dVSS dry laboratory platform also resulted in a decrease in the number of missed targets and instances of instruments out of view. However, both groups had such low mean numbers of both metrics that this additional benefit seen in the dVSS group is difficult to interpret and may not be “clinically” significant. Our results show that training on both simulation platforms improved economy of motion, an important metric that may help surgeons operate more efficiently and potentially minimize tissue handling. However, most of the study participants preferred the MdVT training platform, although both platforms were found to be “definitely useful” for improving robotic surgical skills.

Currently, there are 4 VR robotic surgical simulators available in the United States: RoSS (Simulated Surgical Systems, Williamsville, NY, USA), SEP Robot (SimSurgery, Boston, MA, USA), da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA, USA), and Mimic dV-Trainer (Mimic Technologies, Seattle, WA, USA). VR robotic surgery simulators are an emerging technology, and the research on their use as an educational tool in robotic surgery is limited. Schreuder et al.²⁰ recently published a systematic review on training and learning strategies for robotic-assisted laparoscopic surgery. Most the available reported studies were small and comprised heterogeneous study populations. Of the 13 studies of training using VR robotic surgical simulators that were included in this review, 8 had fewer than 20 participants and 6 had groups that consisted of both surgical novices and experts. The results of our study are consistent with this review of the current literature, which shows that surgical skills training on VR simulators does have a significant learning effect.²⁰ Furthermore, our results are also consistent with a study by Lerner et al.,²¹ which shows that training on the VR Mimic dV-Trainer improved performance on the robot system equal to training with the robot itself with regard to improvements in completion times and number of instrument movements. A similar equivalent improvement in technical performance was shown in a systematic review of conventional laparoscopy surgical simulation training modalities.¹⁵ In this review, simulation training modalities generally showed results superior to no training at all but were not convincingly superior to standard training or watching videos when assessed by operative performance.

Although this study and the current data on VR robotic surgery simulators do not support the notion that this technology is better than existing training modalities, there are benefits unique to the VR simulators that may not be captured in quantitative studies of technical performance. Most of our study participants preferred the VR simulator over the dVSS. Expert surgeons rated the simulator highly for visual field, movement, and precision compared with live surgery.¹⁶ VR simulators also provide detailed, objective feedback that can be stored and used to evaluate surgeon performance over time. The variety of exercises available provides a wide range of training tasks and may prevent training from becoming dull and repetitive. Both of these features may improve surgeon engagement and motivation to use the trainer. Furthermore, independence from a moderator is key to facilitate use and may not be necessary to improve surgeon performance.²² The many unique benefits of a VR surgical simulator should be weighed against its potential drawbacks, perhaps the most significant of which is cost, which is approximately $100,000 according to the manufacturer of the da Vinci Skills Simulator. For some institutions, this may be cost-prohibitive. Furthermore, in their current state of development, the VR simulators are not sophisticated enough to perform specific procedural tasks, such as hysterectomy or prostatectomy or even knot-tying. If these issues can be mitigated, then the utility of the VR simulators would improve substantially.

Strengths of the study include a prospective study design and comparison groups that were well matched in terms of robotic console experience and baseline performance. Moreover, use of the same pretest and post-test between the groups allowed for a direct comparison of performance after the training sessions on the respective platforms. However, this study was limited by the small number of participants, as well as the short duration and frequency of training sessions. The optimal duration and frequency of simulation training sessions are unknown. On the basis of the musculoskeletal complaints and transient headaches reported by several of our participants, perhaps the duration of a training session should be less than 60 minutes to minimize strain caused by repetitive movements.

In the past decade, the rapid expansion and evolution of robotic surgery have posed a challenge for surgical educators: what is the optimal approach to teaching robotic surgery so that trainees achieve proficiency? Several other issues, such as mandated limited resident work hours and fewer procedures to be performed, contribute to this challenge. To address this, it is imperative that structured and standardized robotic surgery training curricula are developed and implemented in residency and fellowship programs.²⁰ Guzzo and Gonzalgo²³ described a 3-phase structured robotic training program: the preclinical phase, the beside-assistant phase, and the operative console phase. VR simulators would be useful in both the preclinical phase and the operative console phase. After the trainee has gained familiarity with the robotic system through didactics and videos, VR simulators can provide a stress-free environment in which they can gain familiarity and then proficiency with the console hand controls and foot pedals. During the operative console phase, the trainee may fine-tune and build on the technical skills that they have gained in the other phases of training and while performing live surgery.

CONCLUSION

In our study the VR simulator was shown to be a valuable learning tool that trainees preferred over the dry laboratory platform with similar efficacy. Additional benefits unique to the VR platform include autonomy of use, computerized performance feedback, and ease of setup. These features may facilitate more efficient and sophisticated simulation training above that of the conventional dry laboratory simulators, with no loss of efficacy. Prospective validation studies of surgical curricula that incorporate VR robotic surgical simulators are warranted.