Abstract
Introduction
We aimed to assess the transferability of basic robotic skills from the simulator to the operating room (OR) while performing robotic-assisted radical prostatectomy (RARP).
Methods
Fourteen urology residents were randomized into two groups: group A was required to practice three sessions (nine tasks each) on the simulator, whereas group B was required to practice (same nine tasks) until they reached competency. Both groups were recorded while practicing on the da Vinci Surgical Skills Simulator. Both groups were then recorded while performing bladder mobilization during RARP. Senior residents from both groups were also recorded while performing urethro-vesical anastomosis during RARP. Recordings were assessed blindly using the validated Global Evaluative Assessment of Robotic Skills (GEARS) tool by C-SATS. Spearman’s correlation coefficient (rho) was used to assess correlation between GEARS scores from practice sessions on the da Vinci Simulator and the GEARS scores from bladder mobilization and urethro-vesical anastomosis during RARP.
Results
There was no difference in total GEARS scores between the two groups in the OR. Total GEARS scores for “ring and rail 2” and “suture sponge” tasks correlated with the total GEARS scores during urethro-vesical anastomosis (rho=0.86, p=0.007; rho=0.90, p=0.002, respectively). GEARS’ efficiency component during “energy and dissection” task on the da Vinci Simulator correlated with GEARS’ efficiency component during bladder mobilization (rho=0.62, p=0.03). GEARS’ force sensitivity component during “ring and rail 2” and “dots and needles” tasks on the da Vinci Simulator correlated with GEARS’ force sensitivity component during bladder mobilization (rho=0.58, p=0.047; rho =0.65, p=0.02, respectively).
Conclusions
Objective assessments of urology residents on the da Vinci Surgical Skills Simulator tasks ring and rail 2 and suture sponge correlated with their objective assessments of bladder mobilization and urethro-vesical anastomosis. Therefore, basic robotic skills could be transferred from the simulator to the OR.
Introduction
Prostate cancer is the most common malignancy in men and radical prostatectomy represents a standard treatment for clinically localized prostate cancer, with acceptable complication rates and functional outcomes.1,2 Since the introduction of robotic surgery in the early 2000s, it has been widely adopted in urology. Recently, robotic-assisted radical prostatectomy (RARP) has become a standard procedure for localized prostate cancer. However, similar to the open approach, it has a steep learning curve. Although there is no widely accepted definition of a learning curve for RARP, several authors have attempted to define it. A number of outcomes could be used to define the number of cases required to achieve the plateau on the learning curve: estimated blood loss was found to plateau in novice surgeons at 100–200 cases, operative time plateaued at 40–120 cases, length of stay plateaued at 200 cases, and positive surgical margins plateaued at 50–1600 cases.3–6 Despite these findings being established at an early stage of RARP, the fact remains that it is not an easy procedure to learn and poses a challenge to surgical trainees and trainers. This challenge is further compounded by the reduction in working hours and the recent introduction of competency-based learning in urology training programs. Therefore, competency threshold in basic robotic skills is needed to assess competency of urology residents in basic robotic skills. To date, there is no validated curriculum that has been shown to determine competency of urology residents.
Multiple studies attempted to define some elements of the challenge. First, virtual reality (VR) simulators were shown to correlate well with dry lab performances; they are on par as a training modality and are less costly.7,8 Second, simulators, including VR simulators, have been shown to improve technical skills with more training regardless of the curriculum implemented.9–12 Third, an objective form of assessment was established with the introduction and validation of the Global Evaluative Assessment of Robotic Skills (GEARS) scale and was shown to have good correlation with operative performance in surgeons; higher scores resulted in better surgical outcomes.13–15
The assessment of these surgical performances using objective tools is very time-consuming. In these studies, the volume of performances is sometimes too large to be analyzed in a timely manner by experts. Therefore, crowd-sourced scoring of these videos is being used and has been validated to correlate well with expert scoring in RARP.16–18 However, to date, there are no studies demonstrating that basic robotic skills could be transferred directly from the simulator to the operating room (OR).
The first objective of this study was to establish competency cutoffs on the da Vinci Surgical Skills Simulator for basic robotic skills. The second objective was to correlate global scores from the da Vinci Simulator for each task with the GEARS evaluations of the same tasks. The third objective was to assess the transferability of basic robotic skills from the da Vinci Simulator to the OR using the GEARS evaluations.
Methods
After obtaining institutional research ethics board approval, 14 urology residents and five attending urologists (experts) were enrolled in this longitudinal study that spanned from June 2015 to January 2019. The da Vinci Surgical Skills Simulator (Intuitive Surgical, Inc., Sunnyvale, CA, U.S.) was used to assess basic robotic surgical skills and to train robotic-naive urology residents. Most residents at the time of enrollment had participated in bedside assistance for robotic procedures; however, none of them had previous exposure to the robotic console prior to the study Based on previous construct validation studies,19 the following nine tasks were used: peg board level 2, match board level 2, needle targeting, ring and rail level 2, dots and needles level 1, suture sponge level 2, energy dissection level 1, ring walk level 3, and tubes. Two of the five experts had previously participated as experts in the previous validation study and had performed all nine tasks.19 At the start of present study, all five experts were recorded while performing the nine tasks. Competency cutoff thresholds were established for the global scores for each of the nine tasks on the da Vinci Simulator using the norm-referenced method (the mean of the expert group minus 1 standard deviation = competency cutoff).20
Participating residents were then randomized into two groups using https://www.randomizer.org/. Group A could perform all nine tasks three times on three separate days. Group B could perform all nine tasks until competency was reached for each of the nine tasks. Prior to starting their training on the da Vinci Simulator, all residents received the same 10-minute, hands-on introduction session to become familiar with the clutches and finger controls of the simulator. In addition, prior to starting each task, residents watched a short tutorial video provided by the simulator explaining the objectives and the difficulty of each task. For each task, the simulator generated a total score ranging from 0 (worst) to 100 (best). The score is based on a set of predetermined metrics, including the following parameters: time to complete task, economy of motion, instrument collision, excessive instrument force, master work space range, object drops, instruments out of view, missed targets, misplaced energy, broken vessels, and blood loss volume. In addition to the global scores, all of the tasks performed by both groups of residents and experts were recorded.
Once the simulator phase of the study was completed, robotic skills of the console-naive residents were assessed during RARP. For each resident, their first three bladder mobilizations during RARP were recorded. Once the residents became senior residents (postgraduate years 4 and 5), their first three urethro-vesical anastomoses were also recorded. When the OR phase concluded, all of the video recordings from the simulator and from the OR were blinded and sent for objective assessment by crowd sourced group (C-SATs) using the validated GEARS assessment tool with an overall score of 30. GEARS has six components — efficiency, bimanual dexterity, depth perception, force sensitivity, robotic control, and autonomy — each with a maximum score of 5. The autonomy component was not assessed in this study since blinded endoscopic videos were used. Since the autonomy component was not used, the maximum overall GEARS score was 25.
There were 50 C-SATs reviewers who blindly assessed and scored each video. For each resident, GEARS scores from the simulator tasks were compared with the GEARS scores in the OR to assess the transferability of basic robotic skills from the simulator to the OR.
Sample size calculation and statistical analysis
Sample size was calculated using the G* Power 3.1 for Mac, which was downloaded from http://www.gpower.hhu.de. A sample size of 14 residents was required to detect a correlation with an effect size of 0.7 between the GEARS scores of the simulator and the OR, with a two-tailed α error probability of 0.05, and power (1-β error probability) of 0.80.
Descriptive data are presented in terms of number, percentages, medians, and interquartile range (IQR). Spearman’s rank correlation was used to assess the association between da Vinci Simulator scores on the simulator tasks with the GEARS scores from the OR while performing bladder mobilization or urethro-vesical anastomosis. The Wilcoxon-Mann-Whitney test was performed to assess scoring differences between groups. The Wilcoxon signed-rank test was performed to assess scoring differences between the first and last attempt on the simulator tasks. Correlation between the simulator global scores and the GEARS scores on the videos of a given exercise was analyzed using the mixed linear model method to account for repeated measures.21 All tests were two-sided, with a significance threshold of 5%. Statistical analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC, U.S.).
Results
Five expert robotic surgeons were recruited to establish competency cutoffs for each of the nine tasks on the da Vinci Surgical Skills Simulator. Their mean global scores with standard deviations for the da Vinci Simulator tasks 1–9 were: 83.0±22.8; 73.2±19.4; 86.6±11.9; 57.8±28.1; 84.2±25.2; 73.6±18.4; 79.4±24.6; 55.6±24.3; 68.8±27.8, respectively. Therefore, the following competency cutoffs were established for the da Vinci Simulator global scores: 61, 54, 75, 29, 59, 55, 55, 31, and 41 for tasks 1–9, respectively. The study included 14 urology residents. Seven residents were randomized to group A and seven residents to group B. There were no significant differences between groups A and B in terms of baseline characteristics (Table 1). Furthermore, there was no significant differences between both groups in terms of the mean number of practice sessions needed to reach competency (2 vs. 1.8 practice sessions, for groups A and B, respectively, p=0.7) (Table 1). During this longitudinal study, eight residents became senior residents (postgraduate year 4–5) and performed the urethro-vesical anastomosis during RARP.
Table 1.
Resident characteristics
| Variables | Group A (n=7) | Group B (n=7) | p |
|---|---|---|---|
| Age (mean) | 28 | 28 | 0.9 |
| Gender | |||
| Male | 5 (71%) | 5 (71%) | 1.0 |
| Female | 2 (29%) | 2 (29%) | |
| Pre-study number of laparoscopic cases as first assistant (mean) | 16.7 | 13.9 | 0.8 |
| Practice sessions needed to reach competency in all 9 tasks (mean) | 2 | 1.86 | 0.7 |
Comparison of performances between both groups across all simulation tasks
For both groups of residents, total global scores obtained from the da Vinci Simulator correlated with the total GEARS scores for all nine tasks (p<0.01) (Table 2). However, there was no significant difference between groups A and B in their best global scores from the da Vinci Simulator across all nine tasks (p=0.05) (Table 3). Similarly, when the GEARS tool was used to assess blinded videos of residents practicing on the da Vinci Simulator, there was no significant difference between the two groups (p=0.05) (Table 3). When analyzing the global scores of all residents pooled together, there were significant improvements from the first to the last attempt in the following tasks: peg board level 2, match board level 2, needle targeting, ring and rail level 2, suture sponge level 2, energy dissection level 1, and ring walk level 3 (all p<0.05) (Table 4). However, these did not translate to significant improvements in the total GEARS scores on the simulator videos (all p>0.05) (Table 4).
Table 2.
Correlation between global scores and total GEARS scores for each simulator task
| Task | Task name | Correlation coefficient | p |
|---|---|---|---|
| 1 | Pegboard level 2 | 0.75 | <0.001 |
| 2 | Match board level 2 | 0.60 | <0.001 |
| 3 | Needle targeting | 0.64 | <0.001 |
| 4 | Ring & rail level 2 | 0.49 | 0.002 |
| 5 | Dots & needles level 1 | 0.50 | <0.001 |
| 6 | Suture sponge level 2 | 0.61 | <0.001 |
| 7 | Energy & dissection level 1 | 0.63 | <0.001 |
| 8 | Ring walk level 3 | 0.34 | 0.009 |
| 9 | Tubes | 0.58 | <0.001 |
Table 3.
Simulator global and GEARS total scores for each of the tasks for groups A and B
| Simulator global scores | |||
|---|---|---|---|
|
| |||
| Task # | Group A | Group B | p |
| 1 | 96 (89–98) | 96 (94–97) | 0.934 |
| 2 | 79 (67–83) | 82 (78–86) | 0.559 |
| 3 | 94 (85–100) | 97 (87–99) | 0.593 |
| 4 | 82 (74–93) | 74 (52–83) | 0.175 |
| 5 | 82 (63–87) | 85 (74–93) | 0.209 |
| 6 | 81 (64–86) | 74 (71–92) | 0.730 |
| 7 | 85 (85–91) | 95 (80–96) | 0.419 |
| 8 | 67 (47–81) | 73 (50–90) | 0.473 |
| 9 | 63 (56–71) | 74 (53–92) | 0.400 |
|
| |||
| Simulator GEARS total scores | |||
|
| |||
| Task # | Group A | Group B | p |
|
| |||
| 1 | 20.0 (19.7–21.6) | 21.5 (20.4–21.9) | 0.383 |
| 2 | 20.0 (19.0–20.3) | 20.0 (19.6–21.2) | 0.902 |
| 3 | 19.8 (18.3–20.9) | 19.9 (19.3–20.2) | 1.000 |
| 4 | 19.3 (19.0–19.7) | 19.2 (19.2–19.5) | 0.805 |
| 5 | 18.2 (16.9–20.0) | 18.7 (17.9–19.1) | 0.620 |
| 6 | 19.3 (18.7–20.0) | 19.2 (18.2–19.4) | 0.318 |
| 7 | 20.5 (19.7–20.7) | 19.6 (19.1–20.6) | 0.259 |
| 8 | 18.9 (18.4–19.5) | 19.5 (18.6–19.8) | 0.620 |
| 9 | 18.5 (18.2–19.6) | 18.9 (18.0–19.6) | 0.900 |
Data are presented as medians (interquartile range).
Table 4.
Differences in global scores and GEARS score from the first attempt to the last attempt for each simulator task for all residents in both goups
| Task | Global score median (IQR) | p | GEARS median (IQR) | p | |
|---|---|---|---|---|---|
| 1 | Pegboard level 2 | 4.5 (3, 11) | 0.001 | 0.95 (−0.09, 1.63) | 0.127 |
| 2 | Match board level 2 | 9.5 (1, 19) | 0.028 | 0.78 (−0.51, 2.65) | 0.147 |
| 3 | Needle targeting | 10 (1, 17) | 0.005 | 0.66 (−0.55, 1.76) | 0.217 |
| 4 | Ring & rail level 2 | 8 (−3, 25) | 0.049 | 0.32 (−0.35, 0.80) | 0.417 |
| 5 | Dots & needles level 1 | 3.5 (−4, 17) | 0.095 | 0.26 (−0.75, 1.95) | 0.326 |
| 6 | Suture sponge level 2 | 11 (−1, 22) | 0.047 | 0.40 (−0.44, 2.45) | 0.127 |
| 7 | Energy & dissection level 1 | 3 (−2, 9) | 0.042 | −0.18 (1.33, 0.82) | 1 |
| 8 | Ring walk level 3 | 15.5 (0, 27) | 0.023 | 0.41 (−0.95, 1.89) | 0.426 |
| 9 | Tubes | 6.5 (1, 12) | 0.235 | 0.31 (−0.36, 0.75) | 0.241 |
IQR: interquartile range.
Comparison of performance between the two groups in the OR
There were no significant differences between groups A and B in their total GEARS scores from their first three OR procedures. The median total GEARS scores for the bladder mobilization were 19.9 (IQR 19.7–20.8) and 20.0 (IQR 19.9–21.2) for groups A and B, respectively (p=0.431), and for the urethro-vesical anastomosis were 20.4 (IQR 18.6–21.1) and 20.0 (IQR 19.3–20.3) for Groups A and B, respectively (p=0.774) (Table 5). Similarly, there were no significant differences in the GEARS domains between the two groups (Table 5).
Table 5.
Comparison of GEARS scores for groups A and B for their first operating room cases
| Operation room procedure | GEARS domain | Group A | Group B | p |
|---|---|---|---|---|
| Bladder mobilization | Depth perception | 4.1 (4.0–4.2) | 4.0 (3.9–4.1) | 0.876 |
| Bimanual dexterity | 4.1 (3.9–4.2) | 4.2 (4.1–4.3) | 0.343 | |
| Efficiency | 3.7 (3.5–3.9) | 3.6 (3.3–3.8) | 0.639 | |
| Force sensitivity | 4.0 (3.9–4.1) | 4.2 (4.1–4.4) | 0.149 | |
| Robotic control | 4.3 (4.0–4.4) | 4.2 (4.2–4.6) | 0.432 | |
| Total | 19.9 (19.7–20.8) | 20.0 (19.9–21.2) | 0.431 | |
| Urethro-vesical anastomosis | Depth perception | 4.0 (3.8–4.2) | 4.1 (3.9–4.1) | 1.000 |
| Bimanual dexterity | 4.1 (3.7–4.3) | 4.0 (3.8–4.1) | 1.000 | |
| Efficiency | 3.9 (3.2–3.9) | 3.5 (3.4–3.6) | 0.571 | |
| Force sensitivity | 4.0 (3.9–4.2) | 4.1 (3.9–4.1) | 1.000 | |
| Robotic control | 4.4 (4.0–4.5) | 4.5 (4.1–4.5) | 0.786 | |
| Total | 20.4 (18.6–21.1) | 20.0 (19.3–20.3) | 0.774 |
Data are presented as medians (interquartile range).
Comparison of performances between residents and attending urologists (experts) using GEARS in the OR
GEARS was used to compare the performance of bladder mobilization and urethro-vesical anastomosis in the OR. There was no significant difference in the total GEARS scores between attending urologists and residents during bladder mobilization (21.3 vs. 19.9, p>0.05) and urethro-vesical anastomosis (20.8 vs. 20.4, p>0.05).
Correlation between residents’ simulator performance and performance in the OR
For each simulator task, residents’ global scores were compared with their total GEARS scores in the OR (Table 6). There was no significant correlation between the global scores across nine simulator tasks and total GEARS scores during bladder mobilization. However, a positive correlation was found between simulator task 4 (ring and rail 2) and urethro-vesical anastomosis (rho=0.74, p=0.037) (Table 6).
Table 6.
Correlation between simulator global scores and operating room total GEARS scores
| Task | Bladder mobilization | Urethro-vesical anastomosis | |||
|---|---|---|---|---|---|
|
| |||||
| rho | p | rho | p | ||
| 1 | Pegboard level 2 | −0.31 | 0.322 | 0.18 | 0.668 |
| 2 | Match board level 2 | −0.10 | 0.753 | 0.61 | 0.108 |
| 3 | Needle targeting | −0.47 | 0.122 | 0.18 | 0.668 |
| 4 | Ring & rail level 2 | 0.08 | 0.812 | 0.74 | 0.037 |
| 5 | Dots & needles level 1 | −0.07 | 0.829 | 0.24 | 0.570 |
| 6 | Suture sponge level 2 | −0.34 | 0.276 | 0.60 | 0.120 |
| 7 | Energy & dissection level 1 | −0.24 | 0.445 | 0.35 | 0.396 |
| 8 | Ring walk level 3 | −0.40 | 0.199 | 0.10 | 0.822 |
| 9 | Tubes | −0.14 | 0.672 | 0.32 | 0.435 |
For each simulator task, residents’ total GEARS scores were correlated with their total GEARS scores from the OR (Table 7). There was no significant correlation between the total GEARS scores on all nine tasks and total GEARS scores during bladder mobilization. However, significant positive correlations were noted between total GEARS scores during simulator task 4 (ring and rail 2; rho=0.86, p=0.007) and task 6 (suture sponge 2; rho=0.90, p=0.002) with total GEARS scores during urethro-vesical anastomosis (Table 7).
Table 7.
Correlation between simulator total GEARS scores and operating room total GEARS scores
| Task | Bladder mobilization | Urethro-vesical anastomosis | |||
|---|---|---|---|---|---|
|
| |||||
| rho | p | rho | p | ||
| 1 | Pegboard level 2 | −0.27 | 0.391 | 0.17 | 0.693 |
| 2 | Match board level 2 | −0.11 | 0.729 | 0.29 | 0.493 |
| 3 | Needle targeting | −0.41 | 0.183 | 0.07 | 0.867 |
| 4 | Ring & rail level 2 | 0.15 | 0.649 | 0.86 | 0.007 |
| 5 | Dots & needles level 1 | 0.25 | 0.430 | 0.50 | 0.207 |
| 6 | Suture sponge level 2 | −0.22 | 0.485 | 0.90 | 0.002 |
| 7 | Energy & dissection level 1 | −0.08 | 0.795 | 0.29 | 0.493 |
| 8 | Ring walk level 3 | −0.24 | 0.457 | 0.00 | 1.000 |
| 9 | Tubes | −0.37 | 0.236 | 0.31 | 0.456 |
Correlation between GEARS individual domains on the simulator and in the OR
GEARS individual domains were compared between the simulator and the OR and multiple positive correlations were noted (Fig. 1). Bimanual dexterity domain during the urethrovesical anastomosis correlated significantly with bimanual dexterity domain during tasks 5 (dots and needles 1) and 6 (suture sponge 2) (rho=0.74, p=0.037 and rho=0.76, p=0.028) (Fig. 1).
Fig. 1.
Spearman rho correlations for individual GEARS domains between the simulator and the operating room for each simulator task. BM: bladder mobilization; UVA: urethro-vesical anastomosis.
The efficiency domain during task 4 (ring and rail 2) correlated significantly with efficiency domain during urethrovesical anastomosis (rho=0.90, p<0.001) (Fig. 1). In addition, the efficiency domain during task 7 (energy dissection 1) correlated significantly with efficiency domain during bladder mobilization (rho=0.62, p=0.033) (Fig. 1). The force sensitivity domain during task 4 (ring and rail 2) and task 5 (dots and needles 1) correlated with force sensitivity domain during bladder mobilization (rho=0.58, p=0.047 and rho=0.65, p=0.022). Finally, the robotic control domain during task 6 (suture sponge 2) correlated with the same domain during urethro-vesical anastomosis (rho=0.81, p=0.015).
Discussion
The first objective of the study was to establish competency cutoffs. Using the norm-referenced method, the following competency cutoffs for da Vinci Simulator global scores were established: 61, 54, 75, 29, 59, 55, 55, 31, and 41 for tasks 1–9, respectively. These global scores are out of 100. The lowest competency cutoff score was 29 for task 4 (ring and rail 2), indicating that this was one of the most difficult tasks on the simulator. Interestingly, this was also the task for which global and GEARS scores correlated significantly with the GEARS scores during the urethro-vesical anastomosis. At this point, urology residents were randomized to either practice three sessions (group A) or practice until they reached competency (group B). Our hypothesis was that residents would need at least three sessions to achieve competency. Surprisingly, both groups achieved competence after two and 1.8 sessions, respectively. Therefore, it was not surprising that both groups had no significant differences in their total global scores from the da Vinci Simulator and total GEARS scores on the simulator and in the OR. Perhaps, if the study was designed with one arm as no practice or practice one session only, there would have been significant differences seen between the two groups. Other explanations for the lack of difference between the two groups could be that the competency cutoffs were set too low. When compared with a previous study including more robotic surgeons from different surgical specialties, the mean global scores for the expert cohort in the current study is similar to mean global scores from intermediates.19 This could be explained by the fact that at the time of the study, robotic surgery was still novel and, therefore, an expert performance at the time would equate to an intermediate performance in this era. It is possible that if higher competency cutoffs were chosen, there may have been a significant difference between the two groups. Furthermore, it is possible that the sensitivity of the assessment tools used failed to detect smaller differences that would translate into a significant difference. Nevertheless, practice did, in fact, significantly improve global scores for most tasks practiced.
The second objective of the study was to compare the global scores from the simulator to the blinded C-SATS evaluations of the simulator tasks using the GEARS tool. This was an important step in assessing transferability of competency in basic robotic skills from the simulator to the OR since there are no equivalency to the global scores in the OR. In fact, there was statistically significant correlation of the global scores from the da Vinci simulator to the GEARS scores for each of the nine tasks (p<0.01). Therefore, GEARS could be used to objectively assess performance on the da Vinci Simulator for the above nine tasks. The third objective of the study was to demonstrate transferability of basic robotic skills from the simulator to the OR. In the present study, da Vinci simulator global score and GEARS total scores from the ring and rail 2 task significantly correlated with residents’ GEARS total scores during urethro-vesical anastomosis. In addition, GEARS total scores from ring and rail 2 and suture sponge 2 tasks significantly correlated with residents’ GEARS total scores during urethro-vesical anastomosis, demonstrating the transfer of basic robotic skills from these two simulator tasks to the OR. When individual domains of the GEARS tool were analyzed, more positive correlations were found. Good bimanual dexterity on the simulator tasks dots and needle and suture sponge level 2 translated into good bimanual dexterity while performing the urethro-vesical anastomosis. Furthermore, the efficiency domain in energy dissection translated into good efficiency score when mobilizing the bladder, indicating that this task mimics several components of the bladder mobilization step in terms of use of energy and control of bleeding vessels using both robotic arms.
These findings have not been reported previously. In fact, the transferability of basic robotic skills was investigated previously with mostly negative findings. When 11 attending surgeons were assessed on the simulator and in the OR, there was no correlation found between the global scores of the simulator to the GEARS scores in the OR.22 However, in that study, there was no GEARS assessment of the simulator tasks, whereas in the present study, global scores from the simulator were first correlated to the GEARS scores on the simulator then the GEARS scores from the simulator were correlated to the GEARS in the OR. Another reason for the lack of correlation could have been that multiple robotic surgeries were compared (thoracic, gynecologic, and urologic), whereas in the present study, only RARP was used. In another study, when medical students were randomized to practice to reach competency vs. one practice session, there was no difference in the GEARS scores between both groups when they performed urethro-vesical anastomosis on a porcine model.23 This is similar to the present study, where there was no difference in the GEARS scores between the two groups of residents. Finally, one study demonstrated that a lengthy training protocol of training for one hour for 60 sessions on a simulator task (Tubes 3) resulted in faster performance of the urethro-vesical anastomosis.24 However, in that study, there was no other objective assessment of technical skills other than measuring time taken to perform the anastomosis.24
Limitations
This study is not without limitations. The first limitation is the lack of control group without practice on the da Vinci Simulator. However, it was not ethical to deny robotic-naive residents the opportunity to practice on the simulator prior to performing bladder mobilization during RARP. Therefore, it was decided to randomize residents into two groups, one with practice for three sessions and the second group with practice until competency. Furthermore, from a practical and patient safety standpoint, it would have been difficult to accommodate a console-naive resident in a time-restricted environment of the OR. The strengths of this study are the fact that residents were randomized into two groups of training, in which all residents were console-naive at the time of enrollment, thus capturing the very first attempts on the simulator, bladder mobilization, and urethra-vesical anastomosis.
The second limitation was the small sample size of residents and attending urologists despite recruiting all urology residents and all robotic surgeons in the urology training program.
The third limitation was that this study was conducted during the initial robotic experience of the urology department and the mean global scores of the experts in this cohort was similar to intermediates in previous publications. This could explain the lower competency cutoffs and the lack of statistically significant difference between the GEARS scores of the experts and the trainees in the OR. Another explanation for the lack of statistical difference is that the GEARS tool, as assessed by lay C-SATS, was not sensitive enough to detect small differences in performance in the OR.
The fourth limitation was that residents were not reassessed on the da Vinci simulator as they progressed in their residency training. Therefore, it is possible that senior residents’ robotic skills improved since their assessment on the simulator during the simulator phase of the study. This could explain the lack of correlation of total GEARS scores on tasks other than ring and rail 2 with total GEARS scores during urethro-vesical anastomosis. Nevertheless, this study correlated simulator global scores with total GEARS scores for all nine tasks. In addition, the da Vinci simulator global score and total GEARS scores from ring and rail 2 task significantly correlated with residents’ total GEARS scores during urethro-vesical anastomosis, demonstrating the transfer of basic robotic skills from the simulator to the OR.
Conclusions
Objective assessments of urology residents on the da Vinci Surgical Skills Simulator tasks ring and rail 2 and suture sponge correlated with their objective assessments of bladder mobilization and urethro-vesical anastomosis. Therefore, basic robotic skills could be transferred from the simulator to the OR.
Acknowledgements
Grant received from The Kuwaiti Foundation for the Advancement of Science (KFAS).
Footnotes
Competing interests: The authors report no competing personal or financial interests related to this work.
This paper has been peer-reviewed.
References
- 1.DeSantis CE, Lin CC, Mariotto AB, et al. Cancer treatment and survivorship statistics, 2014. CA Cancer J Clin. 2014;64:252–71. doi: 10.3322/caac.21235. [DOI] [PubMed] [Google Scholar]
- 2.Heidenreich A, Bastian PJ, Bellmunt J, et al. EAU guidelines on prostate cancer. Part 1: Screening diagnosis and treatment of clinically localized disease. Eur Urol. 2011;59:61–71. doi: 10.1016/j.eururo.2010.10.039. [DOI] [PubMed] [Google Scholar]
- 3.Abboudi H, Khan MS, Guru KA, et al. Learning curves for urological procedures: A systematic review. BJU Int. 2014;114:617–29. doi: 10.1111/bju.12315. [DOI] [PubMed] [Google Scholar]
- 4.Gumus E, Boylu U, Turan T, et al. The learning curve of robot assisted radical prostatectomy. J Endourol. 2011;25:1633–7. doi: 10.1089/end.2011.0071. [DOI] [PubMed] [Google Scholar]
- 5.O’Malley PJ, Van Appledorn S, Bouchier-Hayes DM, et al. Robotic radical prostatectomy in Australia: Initial experience. World J Urol. 2006;24:165–70. doi: 10.1007/s00345-006-0064-4. [DOI] [PubMed] [Google Scholar]
- 6.Sooriakumaran P, John M, Leung R, et al. A multi-institutional study of 3794 patients undergoing robotic-assisted radical prostatectomy shows the learning curve is not as short as previously thought. Eur Urol Suppl. 2011;10:1569–9056. doi: 10.1016/S1569-9056(11)60345-X. [DOI] [Google Scholar]
- 7.Newcomb L, Bradley M, Troung T, et al. Correlation of virtual reality simulation and dry lab robotic technical Sskills. J Min Invasive Gynecol. 2018;25:689–96. doi: 10.1016/j.jmig.2017.11.006. [DOI] [PubMed] [Google Scholar]
- 8.Whiteburst S, Lockrow E, Lendvay T, et al. Comparison of two simulation systems to support roboticassisted surgical training: A pilot study (Swine model) J Min Invasive Gynecol. 2015;23(5):483–8. doi: 10.1016/j.jmig.2014.12.160. [DOI] [PubMed] [Google Scholar]
- 9.Volpe A, Ahmed K, Dasgupta P, et al. Pilot validation study of the European Association of Urology robotic training curriculum. Eur Urol. 2015;68:292–9. doi: 10.1016/j.eururo.2014.10.025. [DOI] [PubMed] [Google Scholar]
- 10.Stegemann AP, Ahmed K, Syed JR, et al. Fundamental skills of robotic surgery: A multi-institutional randomized controlled trial for validation of a simulation-based curriculum. Urology. 2013;81:767–74. doi: 10.1016/j.urology.2012.12.033. [DOI] [PubMed] [Google Scholar]
- 11.Foell K, Finelli A, Yasufuku K, et al. Robotic surgery basic skills training: Evaluation of a pilot multidisciplinary simulation-based curriculum. Can Urol Assoc J. 2013;7:430–4. doi: 10.5489/cuaj.222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Dioun S, Fleming N, Munsell M, et al. Setting benchmarks for the new user: Training on the robotic simulator. JSLS. 2017;21(4):1–5. doi: 10.4293/JSLS.2017.00059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Goh A, Goldfarb D, Sander J, et al. Global evaluative assessment of robotic skills: Validation of a clinical assessment tool to measure robotic surgical skills. J Urol. 2012;187:247–52. doi: 10.1016/j.juro.2011.09.032. [DOI] [PubMed] [Google Scholar]
- 14.Ghani KR, Miller DC, Linsell S, et al. Measuring to improve: Peer and crowd-sourced assessments of technical skill with robot-assisted radical prostatectomy. Eur Urol. 2016;69:547–50. doi: 10.1016/j.eururo.2015.11.028. [DOI] [PubMed] [Google Scholar]
- 15.Goldenberg MG, Goldenberg L, Grantcharov TP. Surgeon performance predicts early continence after robot-assisted radical prostatectomy. J Endourol. 2017;31:858–63. doi: 10.1089/end.2017.0284. [DOI] [PubMed] [Google Scholar]
- 16.Powers MK, Boonjindasup A, Pinsky M, et al. Crowdsourcing assessment of surgeon dissection of renal artery and vein during robotic partial nephrectomy: A novel approach for quantitative assessment of surgical performance. J Endourol. 2016;30:447–52. doi: 10.1089/end.2015.0665. [DOI] [PubMed] [Google Scholar]
- 17.Ghani K, Miller D, Linsell S, et al. Measuring to improve: Peer and crowd-sourced assessments of technical skill with robot-assisted radical prostatectomy. Eur Urol. 2016;69:547–50. doi: 10.1016/j.eururo.2015.11.028. [DOI] [PubMed] [Google Scholar]
- 18.Holst D, Kowalewski TM, White LW, et al. Crowd-sourced assessment of technical skills: An adjunct to urology resident surgical simulation training. J Endourol. 2014;29:604–9. doi: 10.1089/end.2014.0616. [DOI] [PubMed] [Google Scholar]
- 19.Alzahrani T, Haddad R, Alkhayal A, et al. Validation of the da Vinci Surgical Skill Simulator across three surgical disciplines: A pilot study. Can Urol Assoc J. 2013;7:E520–9. doi: 10.5489/cuaj.419. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.George S, Haque MS, Oyebode F. Standard setting: Comparison of two methods. BMC Med Educ. 2006;6:46. doi: 10.1186/1472-6920-6-46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Hamlett A, Ryan L, Serrano-Trespalacios P, et al. Mixed models for assessing correlation in the presence of replication. J Air Waste Manag Assoc. 2003;53:442–50. doi: 10.1080/10473289.2003.10466174. [DOI] [PubMed] [Google Scholar]
- 22.Mills JT, Hougen HY, Bitner D, et al. Does robotic surgical simulator performance correlate with surgical skill? J Surg Educ. 2017;74:1052–6. doi: 10.1016/j.jsurg.2017.05.011. [DOI] [PubMed] [Google Scholar]
- 23.Vargas M, Moawad G, Denny K, et al. Transferability of virtual reality, simulation-based, robotic suturing skills to a live porcine model in novice surgeons: A single-blind, randomized controlled trial. J Min Invasive Gynecol. 2017;24:420–5. doi: 10.1016/j.jmig.2016.12.016. [DOI] [PubMed] [Google Scholar]
- 24.Shim JS, Noh TI, Kim JY, et al. Predictive validation of a robotic virtual reality simulator: The tube 3 module for practicing vesicourethral anastomosis in robot-assisted radical prostatectomy. Urology. 2018;122:32–6. doi: 10.1016/j.urology.2018.08.01. [DOI] [PubMed] [Google Scholar]