Abstract
Purpose:
Simulation-based training improves clinical skills, while minimizing the impact of the educational process on patient care. We present results of a pilot multidisciplinary, simulation-based robotic surgery basic skills training curriculum (BSTC) for robotic novices.
Methods:
A 4-week, simulation-based, robotic surgery BSTC was offered to the Departments of Surgery and Obstetrics & Gynecology (ObGyn) at the University of Toronto. The course consisted of various instructional strategies: didactic lecture, self-directed online-training modules, introductory hands-on training with the da Vinci robot (dVR) (Intuitive Surgical Inc., Sunnyvale, CA), and dedicated training on the da Vinci Skills Simulator (Intuitive Surgical Inc., Sunnyvale, CA) (dVSS). A third of trainees participated in competency-based dVSS training, all others engaged in traditional time-based training. Pre- and post-course skill testing was conducted on the dVR using 2 standardized skill tasks: ring transfer (RT) and needle passing (NP). Retention of skills was assessed at 5 months post-BSTC.
Results:
A total of 37 participants completed training. The mean task completion time and number of errors improved significantly post-course on both RT (180.6 vs. 107.4 sec, p < 0.01 and 3.5 vs. 1.3 sec, p < 0.01, respectively) and NP (197.1 vs. 154.1 sec, p < 0.01 and 4.5 vs. 1.8 sec, p = 0.04, respectively) tasks. No significant difference in performance was seen between specialties. Competency-based training was associated with significantly better post-course performance. The dVSS demonstrated excellent face validity.
Conclusions:
The implementation of a pilot multidisciplinary, simulation-based robotic surgery BSTC revealed significantly improved basic robotic skills among novice trainees, regardless of specialty or level of training. Competency-based training was associated with significantly better acquisition of basic robotic skills.
Introduction
Since the platform was first introduced in 1999,1,2 robotic surgery using the da Vinci Surgical System (Intuitive Surgical Inc., Sunnyvale, CA) has gained widespread adoption in surgical fields, such as urology and gynecology.2,3 Recently, we have also seen increasing utilization in other specialties, such as otolaryngology, general and cardiothoracic surgery.4 Improvements in surgical precision, dexterity, optics, as well as the ergonomic advantages of robotic surgery, have prompted surgeons to adopt this novel technology, more so than after the introduction of other surgical technologies (e.g., laparoscopy).5 Despite the “intuitive” nature of robotic surgery, the integration of this innovative technology into clinical practice still requires appropriate training and is associated with a real learning curve.6–9 Since robotic surgery, unlike traditional open or laparoscopic surgery, requires familiarity with a unique surgical interface, training is required not only for procedural, but robotic systems-based competencies as well.
Despite the need for comprehensive, structured training curricula, few validated robotic basic skills training curricula (BSTC) exist, particularly for specialties that have not fully embraced robotic surgery (e.g., cardiothoracic surgery) and within countries where the adoption of robotics is still in its relative infancy (e.g., Canada). To address this general gap, our group has begun to develop and implement a robotic surgery BSTC for robotic novices at our institution. Unlike in the United States, the adoption of robotic surgery in Canada has been much less widespread and its acceptance into clinical practice much less fervent. While robotic surgery is currently offered at a few of the hospitals affiliated with the University of Toronto, robotic surgery exposure has not reached the levels seen in most American training programs. We present a preliminary evaluation of our pilot multidisciplinary, simulation-based robotic surgery BSTC.
Methods
A 4-week robotic surgery BSTC was offered to residents, fellows, and staff surgeons at the University of Toronto, Departments of Surgery and Obstetrics and Gynecology (ObGyn). The curriculum consisted of various instructional strategies: didactic lecture, self-directed online-training modules, introductory hands-on training with the da Vinci robot (dVR), and dedicated, simulation-based training on the virtual reality da Vinci Surgical Simulator (dVSS).
All participants completed an initial survey detailing demographic and training-related information. The didactic and self-directed online training modules10 focused on the cognitive objectives of the BSTC: benefits and limitations of robotic technology, review of the various robotic systems and standard equipment available for use, introductions to the patient cart, surgeon’s console and vision cart, review of the principles of robot setup, trocar placement, docking, instrument exchange, clutching and troubleshooting of common technical problems.
Participants then engaged in several hands-on training sessions to address the skills objectives of the BSTC. Firstly, all participants were given a 2-hour standardized, hands-on introduction to the dVR, which included a review of its functionalities (including dedicated time to practice docking the dVR, camera setup and instrument exchange) and 30 minutes to practice basic robotic skills on inanimate models (endowrist and camera manipulation, instrument clutching, object manipulation, needle driving, suturing and knot tying). Three individual 1-hour sessions on the dVSS were then organized for each participant, at weekly intervals, during which each participant performed a standardized set of dVSS exercises addressing the following skills: camera navigation, instrument clutching, third arm functionality, endowrist manipulation of objects, needle handling and driving, cautery, and dissection.
Two-thirds of participants progressed through the dVSS simulated exercises in a traditional, time-based training model while one-third (urology group) were engaged in a competency-based dVSS curriculum. Using the built-in software scoring algorithm, participants achieved a 80% score before proceeding to the next exercise; moreover, immediate formative feedback from an expert robotic surgeon was provided after each exercise.
Pre- and post-course skills testing were conducted on the dVR using 2 standardized skill tasks with inanimate models: ring transfer (RT) and needle passing (NP). The RT task involved moving rings from a peg to 1 of 2 alternating pegs. The NP task involved driving a needle (RB-1) through a series of small rings, from one hand to the other. Performance assessment included time to completion and number of errors (i.e., dropped objects, collisions, excessive force, missed targets) for both tasks. A post-course survey was administered to all study participants.
The urology participants were reassessed 5 months after course completion, with repeat performance of the RT and NP tasks.
Statistical analysis was performed using Excel StatPlus (AnalystSoft Inc.). The pre- and post-course skills results were compared using a 2-tailed paired student’s t-test. For non-parametric variables, the Mann Whitney U test was used for independent samples, and the Wilcoxon matched pairs test for related samples. Results between specialties were compared using ANOVA. A p value <0.05 was considered significant for all tests.
Results
In total, 37 participants completed the robotic surgery BSTC: 13 urology, 12 ObGyn and 12 thoracic surgery (Table 1). Participants’ level of training ranged from junior resident to staff surgeon. Of the participants, 22 (59.5%) had no clinical robotic experience whatsoever, and 30 (81.1%) had no robotic console experience. Despite limited previous console experience, all participants self-identified as robotic novices.
Table 1.
Robotic surgery BSTC participant demographic data
| Survey question | Response | No. (%) |
|---|---|---|
| Gender | Male | 24 (64.9) |
| Female | 13 (35.1) | |
| Handedness | Right-hand dominant | 31 (83.8) |
| Left-hand dominant | 3 (8.1) | |
| Ambidextrous | 3 (8.1) | |
| Level of training | Junior resident (R1–R3) | 7 (18.9) |
| Senior resident (R4–R5) | 12 (32.4) | |
| Fellow | 15 (40.5) | |
| Staff surgeon | 3 (81) | |
| Specialty | Urology | 13 (35.1) |
| ObGyn | 12 (32.4) | |
| Thoracics | 12 (32.4) | |
| Previous MIS-laparoscopic/thoracoscopic | None/minimal | 8 (21.6) |
| Moderate | 11 (29.7) | |
| Significant | 15 (40.5) | |
| Previous robotic surgery experience | Fellowship-trained in MIS | 3 (8.1) |
| None | 22 (59.5) | |
| Yes | 15 (40.5) | |
| If yes, no. operative cases as surgical assistant | 0 cases | 0 (0) |
| <10 cases | 9 (60) | |
| 10–20 cases | 3 (20) | |
| >20 cases | 3 (20) | |
| If yes, no. operative cases at robotic console for at least 30 minutes | 0 cases | 8 (53.3) |
| <10 cases | 6 (40) | |
| 10–20 cases | 0 (0) | |
| >20 cases | 1 (6.7) |
MIS: minimally invasive surgery; BSTC: basic skills training curriculum.
The dVSS demonstrated excellent face validity, as all trainees felt the dVSS exercises looked realistic. Most participants (88%) felt that the dVSS was as effective as using the dVR with inanimate models for robotic surgery basic skills training.
While all 37 participants completed the BSTC, only 14 (37.8%) completed both pre- and post-course standardized skills tasks on the dVR; this was due to participant availability. The mean times and number of errors, both pre- and post-course, did not differ between the specialties. Overall, the participants demonstrated significantly improved mean times to completion and number of errors, post-course, for both tasks (p < 0.01) (Table 2). Previous robotic experience did not affect the results, as even the participants with some robotic experience demonstrated significant improvements. Neither level of surgical nor minimally-invasive surgery training significantly affected the results.
Table 2.
Performance parameters for participants completing both pre- and post-course standardized robotic tasks
| Task | Parameter | Pre-course | Post-course | p value |
|---|---|---|---|---|
| RT | Time (mean ± SD) | 180.6 ± 58.2 s | 107.4 ± 32.8 sec | <0.001 |
| Number of errors (mean ± SD) | 3.5 ± 2.4 | 1.3 ± 1.1 | 0.006 | |
| NP | Time (mean ± SD) | 197.1 ± 45.2 s | 153.9 ± 45.2 sec | 0.005 |
| Number of errors (mean ± SD) | 4.5 ± 3.5 | 1.8 ± 1.8 | 0.005 |
RT: ring transfer; NP: needle passing; SD: standard deviation.
While both the competency-based dVSS curriculum participants and traditional, time-based curriculum participants demonstrated significant improvements post-BSTC, the competency-based training group demonstrated better skill acquisition than the time-based training group (Table 3). At baseline, pre-BSTC testing, the proficiency-based group demonstrated a higher number of errors on both tasks, but similar times for task completion when compared to the time-based training group.
Table 3.
Post-BSTC testing results for competency- vs. time-based dVSS training
| Timing | Parameter | Competency-based dVSS training** | Time-based dVSS training | p value |
|---|---|---|---|---|
| Pre-BSTC | RT time (mean ± SD) | 157.5 ± 40.5 s | 198.0 ± 65.7 s | 0.181 |
| RT errors(mean) | 5.0 | 2.4 | 0.045 | |
| NP time (mean ± SD) | 188.7 ± 39.9 s | 203.4 ± 50.6 s | 0.554 | |
| NP errors (mean) | 6.7 | 2.9 | 0.039 | |
| Post-BSTC | RT time (mean ± SD) | 87.0 ± 13.3 s | 122.8 ± 35.3 s | 0.022 |
| RT errors (mean) | 1.0 | 1.5 | 0.651 | |
| NP time (mean ± SD) | 132.5 ± 22.3 s | 170.0 ± 35.9 s | 0.034 | |
| NP errors (mean) | 2.8 | 1.0 | 0.053 |
BSTC: basic skills training curriculum; RT: ring transfer; NP: needle passing; SD: standard deviation. dVSS: da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA).
Competency-based group was either equivalent or worse than Time-based group on all pre-BSTC parameters.
Among the participants that were assessed 5 months after the BSTC, the improvements were durable for RT and NP time. There was a significant increase in RT errors and a trend towards increased NP errors (Table 4).
Table 4.
Performance parameters for participants repeating the standardized robotic tasks five months after BSTC, who also completed pre- and post-curriculum (n=6)
| Metric | Immediately post-BSTC | 5 months post-BSTC | p value |
|---|---|---|---|
| RT time (mean ± SD) | 87.0 ± 13.3 s | 64.5 ± 16.0 s | 0.008 |
| RT errors (mean) | 1.0 | 3.0 | 0.043 |
| NP time (mean ± SD) | 132.5 ± 22.3 s | 136.2 ± 26.1 s | 0.566 |
| NP errors (mean) | 2.8 | 4.3 | 0.075 |
BSTC: basic skills training curriculum; RT: ring transfer; NP: needle passing; SD: standard deviation.
Discussion
Despite the rapid adoption of robotic surgery in clinical practice,4,11 comprehensive training for the waves of novice robotic surgeons emerging from surgical training programs has often been inadequate. Most programs lack a validated robotic surgery training curriculum, in large part due to a lack of widespread robotic expertise. As many faculty members are going through their own robotic surgery learning curves, the downstream effect results in limited exposure and a lack of formal, structured curricula for today’s trainees. This is particularly true in Canada where robotic surgery has not seen the rate of diffusion as in the United States.
For post-graduate surgeons looking to adopt robotic surgery, most available BSTC are industry-led rather than being designed by robotic surgery content experts, and usually involve a short 1-day training session that lacks any formal assessment of competency. The process of robotic surgery certification lacks consensus or consistency across jurisdictions4,12,13 and often lacks a true assessment of individual competency. In certain regions of the world, such as Canada, where the integration of robotic surgery into clinical practice is still in its early stages, the lack of formal robotic surgery training opportunities, for trainees and faculty, is even more profound.
Others have also noted this lack of structured training and have commented on the need to develop more comprehensive, validated training and credentialing programs.13 To address this need, several groups have begun the process of developing and validating robotic surgery training curricula. At the University of California Irvine, the short- and long-term benefits of procedure-specific 5-day robotic surgery training courses have been reported.14,15 More recently, educators at the University of Texas Southwestern have published feasibility, validity, and reliability data on a competency-based inanimate training program for robotic surgery, which also addresses the need for structured training programs.16
Surgical training literature has clearly demonstrated the superior educational outcomes associated with curricula that adhere to the principles of spaced-learning and competency-based training.17–20 The utilization of virtual reality simulation-based training strategies to minimize the footprint of surgical training and associated learning curves has also gained momentum and acceptance. By providing an opportunity for low-stakes, deliberate practice,21 simulation-based training allows surgeons to work through the early parts of their learning curve outside of the clinical care setting, thereby optimizing patient care.13,17 As such, simulation-based training is likely to have an increasing role in all future surgical education initiatives, regardless of discipline.
Preliminary evaluation of our pilot multidisciplinary, simulation-based robotic surgery BSTC demonstrates a clear educational benefit, regardless of surgical specialty, level of training, or previous robotic experience; overall post-course performance metrics revealed a significant improvement among our participants. These improvements were seen in the laboratory setting only and, as such, further research is required to examine the impact on performance in the clinical setting. This ability to demonstrate the clinical benefits of simulation-based training on patient care is the crux of the debate on the role of surgical simulation in today’s training paradigm. With increasing use of simulation-based training strategies, clear evidence that demonstrates its clinical value is beginning to emerge.18,19,22–24
In our preliminary evaluation, trainees completing either the competency-based or traditional time-based training curriculum demonstrated improvements post-BSTC. These preliminary results suggest greater improvements using competency-based training, particularly with time to task completion, and warrant further evaluation. While most training curricula lack this framework, in part due to personnel and financial resource limitations, the value of competency-based training has become a central tenet within the surgical education community.17,25
Our simulation-based training curriculum used the dVSS, a previously validated virtual reality surgical simulator.26–29 The dVSS had excellent face validity and was rated by 88% of participants as effective as the dVR itself for basic skills training. Unlike other robotic surgery simulators, the dVSS combines the actual dVR surgeon console with proprietary, validated training software (Mimic Technologies, Seattle, WA); this improved the fidelity of the simulation significantly and perhaps added to its educational benefit.
This study has several limitations. The results are from a pilot study, and require further validation in a larger cohort of participants. Though all participants completed the curriculum, the rate of participation in both pre- and post-BSTC standardized skills tasks was low (37.8%); this could have introduced selection bias if participants not benefitting from the BSTC were disinclined to undergo post-BSTC testing. Though performance improved on 2 standardized, robotic skills tasks, it has yet to be demonstrated that the BSTC contributes to improved clinical performance. While the improvement in time for task completion appears durable after 5 months, there was a decline in performance with respect to number of errors. However, this was in a limited number of subjects and warrants further evaluation. Also, our initial evaluation has demonstrated improvements in technical skill; however, acquisition and retention of cognitive learning objectives were not fully assessed. Finally, RT and NP performance skills relied on a single faculty rater, so further reliability evidence is required for this curriculum.
Conclusion
Preliminary evaluation of a 4-week, multidisciplinary, simulation-based robotic surgery BSTC demonstrates improved robotic surgical skills among robotic trainees, regardless of specialty, previous robotic experience, and level of training. Competency-based training was associated with better post-course performance compared to traditional time-based training, though there were improvements in performance with both types of training. Further validation studies are required and it is imperative that we ultimately determine the impact of such simulation-based training curricula on clinical performance.
Footnotes
This paper has been peer-reviewed.
Competing interests: Dr. Pace is a member of an Advisory Board for Janssen and Amgen. He has also received support for a fellowship from Cook Medical. Dr. Finelli is Advisory Board member for Amgen, Astellas and Janssen. He has also received honoraria from Amgen, Astellas, Janssen, Paladin and Astra Zeneca. Dr. Finelli has also participated in clinical trials in the past 2 years for Amgen, Astellas, Janssen and Ferring. Dr. Foell has recieved an honorarium from Actavis. Dr. Honey is currently involved in an outcome study in shock wave lithotripsy. Dr. Lee hs received honoraria from Takeda Inc. Dr. Yasufuku, Dr. Bernardini and Dr. Waddell declare no competing financial or personal interests.
References
- 1.Cichon R, Kappert U, Schneider J, et al. Robotically enhanced “Dresden technique” with bilateral internal mammary artery grafting. Thorac Cardiovasc Surg. 2000;48:189–92. doi: 10.1055/s-2000-6903. [DOI] [PubMed] [Google Scholar]
- 2.Yates DR, Vaessen C, Roupret M. From Leonardo to da Vinci: The history of robot-assisted surgery in urology. BJU Int. 2011;108:1708–13. doi: 10.1111/j.1464-410X.2011.10576.x. [DOI] [PubMed] [Google Scholar]
- 3.Weinberg L, Rao S, Escobar PF. Robotic surgery in gynecology: An updated systematic review. Obstet Gynecol Int. 2011;2011:852061. doi: 10.1155/2011/852061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Orvieto MA, Marchetti P, Castillo OA, et al. Robotic technologies in surgical oncology training and practice. Surg Oncol. 2011;20:203–9. doi: 10.1016/j.suronc.2010.08.005. [DOI] [PubMed] [Google Scholar]
- 5.Kho RM. Comparison of robotic-assisted laparoscopy versus conventional laparoscopy on skill acquisition and performance. Clin Obstet Gynecol. 2011;54:376–81. doi: 10.1097/GRF.0b013e31822b46f6. [DOI] [PubMed] [Google Scholar]
- 6.Lavery HJ, Small AC, Samadi DB, et al. Transition from laparoscopic to robotic partial nephrectomy: The learning curve for an experienced laparoscopic surgeon. JSLS. 2011;15:291–7. doi: 10.4293/108680811X13071180407357. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kang BH, Xuan Y, Hur H, et al. Comparison of surgical outcomes between robotic and laparoscopic gastrectomy for gastric cancer: The learning curve of robotic surgery. J Gastric Cancer. 2012;12:156–63. doi: 10.5230/jgc.2012.12.3.156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Meyer M, Gharagozloo F, Tempesta B, et al. The learning curve of robotic lobectomy. Int J Med Robot. 2012;8:448–52. doi: 10.1002/rcs.1455. [DOI] [PubMed] [Google Scholar]
- 9.Patel VR, Tully AS, Holmes R, et al. Robotic radical prostatectomy in the community setting--the learning curve and beyond: Initial 200 cases. J Urol. 2005;174:269–72. doi: 10.1097/01.ju.0000162082.12962.40. [DOI] [PubMed] [Google Scholar]
- 10. Fundamentals of robotic surgery: Module 1 [video]. www.intuitivesurgical.com/assets/training_materials/dVS/Fundamentals_Exam_S/Fundamentals_Exam_S.htm. Accessed November 12, 2013.
- 11.Wedmid A, Llukani E, Lee DI. Future perspectives in robotic surgery. BJU Int. 2011;108:1028–36. doi: 10.1111/j.1464-410X.2011.10458.x. [DOI] [PubMed] [Google Scholar]
- 12.Erickson BK, Gleason JL, Huh WK, et al. Survey of robotic surgery credentialing requirements for physicians completing OB/GYN residency. J Minim Invasive Gynecol. 2012;19:589–92. doi: 10.1016/j.jmig.2012.05.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lee JY, Mucksavage P, Sundaram CP, et al. Best practices for robotic surgery training and credentialing. J Urol. 2011;185:1191–7. doi: 10.1016/j.juro.2010.11.067. [DOI] [PubMed] [Google Scholar]
- 14.McDougall EM, Corica FA, Chou DS, et al. Short-term impact of a robot-assisted laparoscopic prostatectomy ‘mini-residency’ experience on postgraduate urologists’ practice patterns. Int J Med Robot. 2006;2:70–4. doi: 10.1002/rcs.71. [DOI] [PubMed] [Google Scholar]
- 15.Gamboa AJR, Santos RT, Sargent ER, et al. Long-term impact of a robot assisted laparoscopic prostatectomy mini fellowship training program on postgraduate urological practice patterns. J Urol. 2009;181:778–82. doi: 10.1016/j.juro.2008.10.018. [DOI] [PubMed] [Google Scholar]
- 16.Dulan G, Rege RV, Hogg DC, et al. Developing a comprehensive, proficiency-based training program for robotic surgery. Surgery. 2012;152:477–88. doi: 10.1016/j.surg.2012.07.028. [DOI] [PubMed] [Google Scholar]
- 17.Gallagher AG, Ritter EM, Champion H, et al. Virtual reality simulation for the operating room: Proficiency-based training as a paradigm shift in surgical skills training. Ann Surg. 2005;241:364–72. doi: 10.1097/01.sla.0000151982.85062.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: Results of a randomized, double-blinded study. Ann Surg. 2002;236:458–64. doi: 10.1097/00000658-200210000-00008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Zendejas B, Cook DA, Bingener J, et al. Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: A randomized controlled trial. Ann Surg. 2011;254:502–9. doi: 10.1097/SLA.0b013e31822c6994. discussion 509–11. [DOI] [PubMed] [Google Scholar]
- 20.Kerfoot BP, Baker HE, Koch MO, et al. Randomized, controlled trial of spaced education to urology residents in the united states and canada. J Urol. 2007;177:1481–7. doi: 10.1016/j.juro.2006.11.074. [DOI] [PubMed] [Google Scholar]
- 21.Ericsson KA. Deliberate practice and acquisition of expert performance: A general overview. Acad Emerg Med. 2008;15:988–94. doi: 10.1111/j.1553-2712.2008.00227.x. [DOI] [PubMed] [Google Scholar]
- 22.Ahlberg G, Enochsson L, Gallagher AG, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193:797–804. doi: 10.1016/j.amjsurg.2006.06.050. [DOI] [PubMed] [Google Scholar]
- 23.Grantcharov TP. Validation of a structured training and assessment curriculum for technical skill acquisition in minimally invasive surgery. Ann Surg. 2013;257:224–30. doi: 10.1097/SLA.0b013e31827051cd. [DOI] [PubMed] [Google Scholar]
- 24.Grantcharov TP, Kristiansen VB, Bendix J, et al. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2003;91:146–50. doi: 10.1002/bjs.4407. [DOI] [PubMed] [Google Scholar]
- 25.Palter VN, Graafland M, Schijven MP, et al. Designing a proficiency-based, content validated virtual reality curriculum for laparoscopic colorectal surgery: A Delphi approach. Surgery. 2012;151:391–7. doi: 10.1016/j.surg.2011.08.005. [DOI] [PubMed] [Google Scholar]
- 26.Liss MA, Abdelshehid C, Quach S, et al. Validation, correlation, and comparison of the da vinci trainer(™) and the da vinci surgical skills simulator(™) using the mimic(™) software for urologic robotic surgical education. J Endourol. 2012;26:1629–34. doi: 10.1089/end.2012.0328. [DOI] [PubMed] [Google Scholar]
- 27.Kelly DC, Margules AC, Kundavaram CR, et al. Face, content, and construct validation of the da vinci skills simulator. Urology. 2012;79:1068–72. doi: 10.1016/j.urology.2012.01.028. [DOI] [PubMed] [Google Scholar]
- 28.Finnegan KT, Meraney AM, Staff I, et al. Da vinci skills simulator construct validation study: Correlation of prior robotic experience with overall score and time score simulator performance. Urology. 2012;80:330–5. doi: 10.1016/j.urology.2012.02.059. [DOI] [PubMed] [Google Scholar]
- 29.Hung AJ, Zehnder P, Patil MB, et al. Face, content and construct validity of a novel robotic surgery simulator. J Urol. 2011;186:1019–25. doi: 10.1016/j.juro.2011.04.064. [DOI] [PubMed] [Google Scholar]