A prospective study of the effect of video games on robotic surgery skills using the high-fidelity virtual reality RobotiX simulator

Abstract

Background

Robot-assisted surgery is a growing field. Prior video game experience might give advantage to novice robotic surgeons.

Aim

Assessing if prior video gaming experience gives advantage in performing high-fidelity virtual reality (VR)-simulated robotic surgery.

Methods

In this observational study, 30 medical students and 2 interns (17 females; 15 males) with median age 25 years (range, 24–26 years) were recruited and subsequently divided into groups according to prior gaming experience; gamers (≥6 video game hours/week) vs nongamers (<6 video game hours/week). Participants performed VR-simulated urethrovesical anastomosis on RobotiX Mentor, which measured performance parameters. Participants answered a questionnaire for demographics and gaming experience. Groups were compared using Mann–Whitney U and multiple regression.

Results

Gamers significantly outperformed nongamers in 3 of 24 performance metrics (p<0.05), and there was a trend toward better results for 7 of the 21 remaining metrics. Males outperformed females in 5 of 24 metrics (p<0.05) but were overrepresented among gamers.

Conclusion

Prior video game experience >6 hrs/week might give advantage in simulated robotic surgery. We recommend future studies testing this hypothesis to develop simulator programs for certification of robotic surgeons.

Introduction

Robot-assisted surgery has grown through the last decades and will most likely continue this growth in the future. The field involves a number of specialities including urology, cardiothoracic surgery, gynecology, general surgery and otorhinolaryngology and is still expanding.¹ Performing robotic surgery using the da Vinci^® Surgical System (Intuitive Surgical, Sunnydale, CA, USA) has proven to cause less complications (eg, reduced blood loss and shorter hospital stay) than traditional laparoscopic surgery.² However, the steep learning curve held together with the still rising awareness of patient safety, reduced working hours and financial constraints in health care organizations causes the need for training outside the operating theater.³

Previous studies suggest that video game players have better baseline laparoscopic skills and that video games could be a way to train laparoscopic skills in surgical residents.⁴ Both observational studies discussing the effect of prior gaming experience and controlled experiments using video games as an intervention have found an advantage of playing video games on laparoscopic skills. When testing on virtual reality (VR) and box-trainer, prior and recent video game experiences have both been associated with better baseline laparoscopic skills, but when testing on porcine models, there has not been identified any advantage with prior or recent video game experience.⁴

The latest robotic VR simulator, RobotiX Mentor (RM) (3D Systems, Simbionix Products, Cleveland, OH, USA), offers high-fidelity procedure-specific simulations for hysterectomy, lobectomy, vaginal cuff closure, inguinal hernia repair and prostatectomy using the da Vinci interface.⁵ These procedure-specific 3D simulations provide a very realistic platform to practice procedural tasks safely in a VR environment.⁷ Since robotic naive will never be allowed to perform real-time surgery, VR simulation might offer the most authentic scenario to test robotic skills besides the more expensive porcine models and human cadavers. VR simulators have the benefit of an inherent objective performance evaluation report, whereas when testing on physical models the presence of an observer to evaluate performance is needed.⁵

The development of video games is a steep and ever-changing process; lately, we have seen VR playing a greater role in the gaming world. VR resembles the intuitive interface seen in the da Vinci robot and so they might improve the psychometric skills necessary for robotic surgery. If so, video games might be incorporated into programs for training and certification of future robotic surgeons.

Other studies have assessed the effect of video game experience on simulated robotic surgery skills.^6–8 However, doing robotic surgery requires many other skills. Therefore, it would be of interest to investigate the effect of video game experience on more complex tasks in robotic surgery, eg, suturing an anastomosis on a high-fidelity simulator.

Methods

Study design and participants

This prospective, observational, comparative and multi-institutional study took place in Aalborg University Hospital and at Copenhagen Academy for Medical Education and Simulation located at Rigshospitalet in Copenhagen, Denmark.

We obtained written informed consent from robotic naive participants who were recruited locally in the period between January and April 2017. It was not necessary to get an approval from a review board or ethics committee. We used a sample size of convenience; 30 medical students and 2 interns were recruited and subsequently categorized into groups according to prior video game play in hours/week (eg, computer, PlayStation, Wii and X-box). Participants were divided into two groups according to prior video game experience and tested for a difference in the performance of a simulated urethrovesical anastomosis. The students or interns have never participated or assisted in laparoscopy or robotic surgery.

Exclusion criteria were experience with VR simulation in robotic surgery and prior performance of robotic surgery as these might affect the performance outcome; for the same reason, we did not include robotic surgeons. Demographics of the participants are shown in Table 1.

Table 1.

Demographics of participants

	Total NO	Gamers NO	Nongamers NO	p-value
Number (NO) of participants	32	12	20
Median age (IQR) years	25 (24–26)	24 (23.8–25)	25 (24–26)	0.111
Male/female	15/17	10/2	5/15
Handedness (left/right/ambidextrous)	4/26/2	2/8/2	2/18/0
Medical student/intern	30/2	11/1	19/1
Median year of study (IQR)	3.5 (3.5–4.5)	3.5 (3.5–3.8)	3.5 (3.5–4.5)	0.732
Experience with simulated surgery* (Y/N)	8/24	4/8	4/16
VR game experience (Y/N)	4/28	3/9	1/19
Prior video gameplay in hours/week (IQR)	3.9 (0.6–13)	17 (12.8–21.4)	1 (0–3)	0.004

Notes: *Porcine models, VR (not robotic surgery), box-trainer, physical models.

Abbreviations: Y, yes; N, no; IQR: interquartile range; VR, virtual reality.

Performance evaluation and outcome measures

The simulation was executed via the MentorLearn courseware which provides metrics and benchmark feedback for the simulation upon completion of a task. Outcome measures were based upon 25 provided and validated performance metrics including 11 general metrics such as task length and instrument collision as well as 14 task-specific metrics such as injury to the urethra and accuracy of the suturing technique (Table 2). The simulator we used did not have the possibility for participants to break the suture, and so this 25th metric is not included in the study.

Table 2.

Comparison of performance metrics between gamers and non-gamers

Performance metrics (unit)		Gamers		Nongamers		p-value
Median	IQR	Median	IQR
Time and economy
Time, s	3728.5		3055–4698.3	4229.5	3815–4869.5	0.432
Movements of left ins., NO	2122.5		2038.3–2443	2571.5	2288.8–3450.8	0.012
Movements of right ins., NO	2504		2165.8–3074.8	2996.5	2633.8–3720.3	0.103
Path length of left ins., mm	20,178.0		17,558.5–22,132.6	25,304	20,375.4–36,771.8	0.015
Path length of right ins., mm	21,920.9		18,891.1–25,208.5	28,473.2	22,724.5–34,211.2	0.063
Distance by camera, mm	3905.1		2598–5473	4628.8	1522.3–5829.5	0.527
Instrument collisions, NO	152.5		94.8–213	199	146.8–273.2	0.081
Path instruments are out of view, mm	4880.9		1727.5–7437.2	4266.7	2957.3–9213.7	0.161
Times instruments are out of view, NO	208		108–355.8	210	93–328.5	0.527
Time instruments are out of view, s	564.5		143.5–1234	463	279–1111.3	0.578
Clutch usage, NO	25		20.8–32.8	26.5	18–53.5	0.618
Safety and complications
Misapplied energy, NO	0		0	0	0	0.790
Injury to urethra NO	16		11.5–31.5	17.5	10.8–35.8	0.192
Injury to bladder neck, NO	3		2–4.8	4	2–8.5	0.211
Suspected injury to bladder, NO	0		0	0	0–1	0.104
Injury to the NVB, NO	0		0–3.8	3,5	1.8–8	0.071
Respect for tissue NO	13		7.3–27	14	8.5–35.3	0.046
Suspected injury to the UO, NO	1		0.8–2	1	1–2	0.599
Suspected injury to the EF/US, NO	10.5		4.5–16.3	13	6.8–18.5	0.442
Wound separation, mm	2.09		1.05–3.14	2.09	1.05–3.14	0.652
Improper suturing technique, NO	2		0–4	3	1–5.5	0.134
Needle handling
Percentage of stitches in optimal depth	56.6		52.6–75	62.3	55.9–69.8	0.388
Unnecessary needle piercing points, NO	134.5		80–194.3	182	101.8–252.8	0.234
Entrance and exit points for needle, NO	52		49.5–58	56	51.5–60.5	0.136

Note: P-values for metrics showing significant differences are marked with bold.

Abbreviations: ins., instrument(s); NVB, neurovascular bundle; UO, urethral orifice; EF, endopelvic fascia; US, urethral sphincter; NO, number; IQR: interquartile range.

Comparison of participants

We compared the performance of participants with low video game experience to performance of those with higher video game experience. Participants were divided into two groups: gamers with prior ≥6 hrs/week of video game play and nonamers with prior <6 hrs/week. This way of measuring prior video game play experience has been used by several previous studies.^4,9,10

Our definition of prior video game experience does not include the present amount of video game play but only the amount played somewhere in the past of the participants' lives. This time in life may vary between participants, as the questionnaire did not specify age. Moreover, performance between genders was compared.

Equipment

The RobotiX Mentor^TM (3D Systems; Simbionix Products, Cleveland, OH) system is a VR simulator using the da Vinci interface. The platform consists of an adjustable headset with integrated stereoscopic visors, nonfixed hand controls, and foot pedals all integrated into a single console. It is a novel and latest high-fidelity simulator for assessment and training of trainees in robotic surgical skills with validation for this use.^2,11,12

Process

We evaluated the influence of prior video game experience on VR simulation performance using the module “Guided urethrovesical anastomosis” (Figure 1) on the high-fidelity RM. Anastomosis is the last part of a robot-assisted radical prostatectomy, and the simulation displays the anatomy within pelvis after the prostate has been dissected and removed.

Figure 1.

Guided urethrovesical anastomosis on RobotiX Mentor. The initial target point for guided sewing is marked with yellow and with an arrow indicating the direction (A). A green target point designates accurate needle puncture (B), whereas a red target point designates inaccurate needle puncture (C). The guided task provides step-by-step instructions (D).

We performed all testing during daylight hours with slight background noise from personnel at both testing sites. Participants were guided orally for 5 mins about the hand controls, foot pedals and movement of the camera, according to a written standardized instruction. They were familiarized with the simulator using two modules for robotic basic skills. The first task “Camera 0” and the second task “Wristed manipulation level 2” were both tried once each, and participants were guided orally if in doubt. No data were collected during these modules. Participants then completed the module “Guided urethrovesical anastomosis” following a step-by-step instruction given by the simulator while it simultaneously collected data. On the screen, yellow target points would appear to show where the suture should be put and a yellow arrow to indicate the sewing direction (Figure 1). The task was complete when the participant had put 5 stitches on either side of the anastomosis (ie, left and right). This module alongside with the simulator is described more thoroughly in a previous study.¹³ If in doubt, participants were guided orally by an observer blinded to their amount of prior video game experience.

Upon completion of the module, participants fulfilled a questionnaire gathering demographics, prior experience with simulated surgery, handedness, experience with VR games, console games and PC-games (Table 1).

Statistical analysis

We stored all questionnaire data and data from MentorLearn courseware anonymously in a specially designed Excel 2011 spreadsheet (Microsoft Corp., Redmond, WA, USA).

Data were analyzed using SPSS for MAC, version 24 (IBM Corp., Armonk, NY, USA) and Microsoft Excel 2011 (Microsoft Corp.). We considered a p-value ≤0.05 significant.

Results

Participants

Thirty-three medical students and two intern volunteers (19 females; 16 males) participated in this study. One was excluded due to prior experience with VR robotic surgery and two were excluded due to technical difficulties during the simulation.

Finally, 30 medical students and two intern volunteers (17 females; 15 males) with a median age of 25 years (range, 24–26 years) participated in this study (Table 1). They were collected through convenience sampling in the period January to April 2017. All but one had passed their third year of medical school.

The group of nongamers (N=20) was almost twice as big as for gamers (N=12). Males were overrepresented among gamers and females were overrepresented among nongamers.

Performance outcome for participants

The gamers significantly outperformed the nongamers in 3 of 24 performance metrics (p<0.05), which included “Number of movements of left instrument”, “Path length of left instrument” and “Respect for tissue” (Table 2, Figures 2 and 3). For the remaining 21 metrics, there was no difference between the groups. It was not possible in the simulation to break the suture, and therefore both groups have a total of 0 occurrences. “Suture breakage” is for the same reason left out.

Figure 2.

Mean performance metrics for Time and economy. Time is measured in second; path length left/right ins. measured in 10² mm. Distance camera and path ins. out of view measured in mm. Clutch usage measured in numbers*10^-2. Metrics showing significant difference are marked with *.

Abbreviation: ins., instrument(s).

Figure 3.

Mean performance metrics for safety and complications. Metrics showing significant difference are marked with *.

Abbreviations: NVB, neurovascular bundle; UO, urethral orifice; EF, endopelvic fascia; US, urethral sphincter.

When comparing gender, males significantly outperformed females in 5 of 24 performance metrics (“Time”, “Number of movements of left instrument”, “Path length of left instrument”, “Injury to the bladder” and “Injury to the neurovascular bundle”), but not after adjustment for experience with video games.

When comparing males in the two groups, male gamers significantly outperformed the male nongamers in 1 of 24 performance metrics (“Injury to the neurovascular bundle”).

We could not compare females in the two groups since there are only 2 female gamers. The same accounts for handedness since there are only 4 left-handed and 2 ambidextrous.

Seven of the 24 performance metrics were normally distributed, and multiple regressions were conducted to compare the adjusted effect of being a gamer with being male. Males showed only significantly better performance in “Percentage of stitches within optimal depth” when adjusted for gaming experience.

Discussion

We demonstrated a difference in performance among medical students with ≥6 hrs/week of game play compared to medical students with <6 hrs/week of game play. Gamers performed significantly better in 3 of 24 metrics (“Number of movements of left instrument”, “Path length of left instrument” and “Respect for tissue”) and showed better results for the majority of the remaining metrics (Figures 2–3). At best, nongamers performed just as good as gamers in the “Safety and complication metrics” (Figure 3). This group of metrics especially requires motoric skills and control of the robotic arms as well as visuospatial skills. However, our study is not robust enough to conclude that this difference in single parameters has any relevance, but the role of video gaming is very relevant when making educational curriculum for future surgeons. A recent paper proposes a new standard in analyzing and reporting VR outcome data — the Heidelberg VR score.¹⁴ The scoring algorithm shows the results on the importance of different skill aspects in laparoscopic surgery are universially applicable and that it can be transferred to any simulator or task. By incorporating specific expert baseline data for the respective task, one will be able to compare between tasks, studies and simulators.

The present study performing a vesicourethral anastomosis (VUA) as in robotic prostatectomy is a challenging task for novices due to delicate tissue and difficult suturing angles. A recent study was testing 10 surgical residents without prior robotics training where 5 residents received structured VR training, 5 did not and 4 were faculty robotic surgeons “experts”. They found mean (range) completion percentage was 20% (10–30%), 54% (40–70%) and 96% (85–100%) by the untrained, trained and expert groups, respectively.¹⁵ Anastomosis integrity was rated as excellent (as opposed to moderate or poor) in 40%, 60% and 100% of untrained, trained and expert groups, respectively. It would have been interesting to know whether some of these had previous gamer experience which then could support our study. On-gamers sewed better in terms of “Percentages needle is in optimal depth”. However, given their higher number of “Unnecessary needle piercing points”, gamers might have been easier satisfied with the position of the needle even though the needle was not in optimal depth (red target point, Figure 1C), whereas nongamers have tried to perform the needle puncture as good as possible, piercing more times to get the needle in optimal depth (green target point, Figure 1B). Gamers had their instruments out of view for a longer time and a longer path, which is considered a lesser performance (Figure 2). But given the better performance in the “Safety and complication metrics”, they might have been more aware of the instrument position even though not having them in view (Figure 3).

Previous studies have a tendency of asking whether video game experience improves surgical skills even though participants are only tested on simulators which do not resemble real-time surgery and which are only validated for the purpose of training basic motoric skills. Considering the studies showing a correlation between video game play and simulator performance, this might mean one of two things. First, it has been shown that practicing on a VR simulator improves real-time theater skills and therefore video game playing might do as well. Second, VR simulators are similar to video games, using the same principles; therefore, video game players have an advantage due to familiarity with the VR concept.¹⁶

Testing laparoscopic skills on LapSim^® VR simulator, van Dongen et al found a significant difference in terms of the total score, efficiency and speed scores between interns with different levels of prior video game experience but not between school children with different video game experience.¹⁷ Neither the group of interns nor the school children differed in scores concerning precision. Unlike our findings, performance did not differ between genders.

Using VR laparoscopy on a MIST-VR, Grantcharov et al showed a correlation between surgeons' prior video game experience and less errors but not the time taken or numbers of unnecessary movements.¹⁸ They did not find any difference in performance between gender.

Willis et al tested medical students using VR and a physical model.¹⁹ They found a correlation between performance on Nintendo Wii and VR but not between Nintendo Wii and the physical model. Nor did the performance on VR correlate with performance on the physical model.

Earlier studies have tested the effect of video game experience on simulated robotic skills as the primary outcome, using the da Vinci robot.^6,20 They did not find prior video game experience to be of any aid in simulated robotic surgery, but experience with different sports and skills of playing musical instruments seemed to benefit the robotic naive. Because the da Vinci robot uses an intuitive interface, the authors suggested that activities developing manual dexterity and intuitive hand–eye coordination such as sports and music could facilitate a more rapid acquisition of the skills and coordination necessary for robotic surgery. The intuitive interface in robotic surgery is different from laparoscopic surgery. Counterintuitive 3D motions are transferred to a 2D video screen, requiring the surgeon to sometimes work against the camera and perform complex procedures with the instrument moving on the projected video image opposite to what may seem natural. This type of movement may mimic what takes place during certain video games and hence aiding the laparoscopic naive but not the robotic naive. Their outcome measures were based upon skills of knot tying, and as known, there are many other skills required for the performance of robotic surgery. A randomized study was performed in order to evaluate the benefits of a combined multimodality training program for surgical residents with laparoscopic cholecystectomy. The Global Operative Assessment of Laparoscopic Skills (GOALS) showed training had improved GOALS scores while control did not.⁷ Another study explored the potential advantages of sequential learning of psychomotor and visuospatial skills for laparoscopic suturing and knot tying compared to simultaneous learning.⁸ They found after the laparoscopy-naïve medical students were randomized into a sequential learning group (SEQ) or a simultaneous learning group (SIM) that SEQ was faster than SIM at the beginning of the learning curve, but SEQ did not reduce the total training time needed to reach a proficiency level. However, SEQ needed less training on the box-trainer.

There are 6 VR simulators available for robot-assisted surgery; the da Vinci skills simulator, the mimic dV trainer, the ProMIS simulator, the simsurgery educational platform simulator, the robotic surgical simulator and the RM. Their efficacy is limited by the lack of comparative studies, validation and high cost. There is a need for a standard in analyzing and reporting VR outcome data, eg, the Heidelberg score.¹⁴ These simulators have to be incorporated into the urology training curriculum globally to shorten the learning phase.^11,12

There are several limitations in our study. First, we had a study population, with a skewed gender distribution in our two groups, which made it difficult to assess whether the better performance was due to gender or due to prior video game experience. We used a new high-fidelity robotic simulator, the RM, and had no possibilities to use other simulators. Furthermore, participants varied in the type of games being played and the type of gaming unit(s) used.

Finally, a common denominator in the articles included from our search is the assessment of laparoscopic and robotic skills, which takes place on a simulator. Hence, even if a relationship between performance and video game playing is established, it will only refer to simulated performance. We suggest a future randomized controlled trial assessing the impact of video game playing on robotic skills in a theater.

Our study has several important implications and shows the need for unifying robotic surgery training curriculum that combines both basic skills development and procedure-specific training. The European Association of Urology and America Urological Association have identified and implemented a comprehensive training curricula with a 12-week training curriculum for robotic surgery, including VR, dry laboratory and wet laboratory simulations of all steps of the robotic prostatectomy.^21,22

We used a high-fidelity simulator which was not the case in most previous studies. This simulator has shown to be useful and might offer the most authentic scenario for a VUA in robotic naive doctors to be tested on except for the expensive and ethical difficult porcine models and human cadavers. Furthermore, the RobotiX measures a broad variety of parameters and have shown its usefulness in the recent study by Shee.¹⁵

To our knowledge, this study is the first to evaluate the effect of prior video game experience on complex tasks in high-fidelity VR simulated robotic surgery. Future studies should agree on a common definition of prior video game experience, and a common way to measure performance outcome.

Conclusion

We demonstrated that prior video game experience might give an advantage in performing high-fidelity VR-simulated robotic surgery. We recommend future randomized studies with larger populations and common validated methods in order to incorporate video games in simulator programs for training and certification of robotic surgeons.

Disclosure

The authors report no conflicts of interest in this work.