Abstract
Purpose
In this study, we investigate the effect of trainee involvement on surgical performance, as measured by automated performance metrics (APMs), and outcomes after robot-assisted radical prostatectomy (RARP).
Methods
We compared APMs (instrument tracking, EndoWrist® articulation, and system events data) and clinical outcomes for cases with varying resident involvement. Four of 12 standardized RARP steps were designated critical (“cardinal”) steps. Comparison 1: Cases where the attending surgeon performed all four cardinal steps (Group A) and cases where a trainee was involved in at least one cardinal step (Group B). Comparison 2, where Group A is split into Groups C and D: Cases where attending performs whole case (Group C) vs. cases where a trainee performed at least one non-cardinal step (Group D). Mann-Whitney U and chi-squared tests were used for comparisons.
Results
Comparison 1 showed significant differences in APM profiles including camera movement time, third instrument usage, dominant instrument moving time, velocity, articulation, as well as nondominant instrument moving time and articulation (all favoring Group A p <0.05). There was a significant difference in re-admission rates (10.9% in Group A vs 0% in Group B, p < 0.02), but not for post-operative outcomes. Comparison 2 demonstrated a significant difference in dominant instrument articulation (p <0.05) but not in post-operative outcomes.
Conclusions
Trainee involvement in RARP is safe. The degree of trainee involvement does not significantly affect major clinical outcomes. APM profiles are less efficient when trainees perform at least one cardinal step but not during non-cardinal steps.
Keywords: Automatic performance metrics, resident surgical training, surgical education, robotic surgical procedures, prostatectomy
Introduction
Clinical outcomes after surgery have been correlated to surgical technique [1]. Therefore, it is imperative for surgical trainees to develop technical skills in order to maximize patient outcomes and safety. The development of technical skills requires participation by trainees in the operating room and multiple studies in surgery have compared cases with and without resident involvement and found that resident involvement is safe [2–3]. Additional studies looking specifically at patient outcomes following urologic procedures have found increased operative times in cases involving a resident, but mixed results when examining differences in complication rates [4–7].
Previous studies assessing the impact of resident involvement in surgery have relied on databases such as the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) which provides only a binary variable to indicate the presence or absence of residents [2–4]. Robotic-assisted radical prostatectomy (RARP), is one of the more common robotic procedures performed in urology. This procedure can be broken down into 9 steps: dropping of the bladder, opening of endopelvic fascia, bladder neck dissection (BND), seminal vesicle dissection, posterior plane dissection, prostate pedicle dissection (PD), apical dissection (AD), vesicourethral anastomosis (VUA), and pelvic lymph node dissection. Of these, four steps that involve direct handling of the prostate and reconstruction have been identified and labeled as the “cardinal steps”. These include BND, PD, AD, and VUA. Superior performance during these specific steps by more experienced surgeons appear to lead to superior peri-operative outcomes [8].
The evaluation of surgical performance has classically relied upon subjective evaluation by peer surgeons, however the ability to record surgical video and systems data through the robotic platform has greatly enhanced our ability to evaluate surgical technique [9]. We build upon previous work of automated performance metrics, derived from instrument kinematics and systems event data, as an objective measurement of surgeon performance. APMs have been shown to differentiate between novice and expert surgeon as well as predict time to continence after RARP [10–11].
In this study, we present an evaluation of trainee impact on clinical outcomes and surgeon performance (as measured by APMs) while accounting for the specific degree of involvement (i.e., how many and which steps of the RARP were performed by a trainee). many and which steps of the RARP were performed by a trainee).
Methods
Study Design
Using an institutional review board approved protocol, synchronized surgical video and systems data for the whole RARP procedure were recorded using a data recorder (Intuitive Surgical) directly from the da Vinci Si and Xi systems consecutively, for RARP procedures from August 2016 to October 2017. RARP cases performed without the da Vinci system data recorder were excluded. Additionally, trainee involvement was identified for each step of the RARP. Outcomes were recorded prospectively.
Participants
Participants in the present study were attending faculty surgeons and trainees (minimally invasive surgery fellows, and urology residents). In “Comparison 1”, we compared cases where the attending surgeon performed all four cardinal steps (Group A) to cases where the trainee performed at least one cardinal step (Group B) (Fig. 1a). In a second comparison, “Comparison 2”, Group A was then further subdivided into 2 subgroups, where the attending surgeon either performed the whole case (Group C) or where the attending surgeon performed all cardinal steps with trainee involvement for non-cardinal steps (Group D). In “Comparison 2” we examined differences between Group C and Group D (Fig. 1b).
Fig. 1.
a Comparison 1 b Comparison 2: Group A is further split into Groups C and D
Data Collection
Previously developed and validated APMs were derived from the data recorder. These APMs included kinematic data (instrument travel time, path length, velocity, EndoWrist® movements) and system events data (camera movements, third arm usage). In this study, a total of nine previously validated APMs [11] were analyzed across the entire procedure.
Clinical data (patient characteristics, outcomes) for each case were prospectively obtained for all consented patients, with only those with a minimum 3-month follow-up being included for analysis. Information on functional outcomes was obtained via self-reported questionnaire at each postoperative clinic visit (3, 6, 12, 18, and 24 month visits). Continence was defined as requiring no or 1 safety pad [12]. A positive technical surgical margin was defined as a positive surgical margin in the absence of extracapsular extension or where the site of the positive margin did not coincide with the site of extracapsular extension.
Statistical Analysis
Statistical analysis was conducted using IBM SPSS v24, with P values < 0.05 taken to indicate statistical significance. Mann-Whitney U, chi-squared and log-rank tests were used to compare patient demographic and outcomes data.
Results
A total of 99 cases were included in analysis. For Comparison 1, there were 46 cases where the attending surgeon performed at least all cardinal steps (Group A) and 53 cases where a trainee performed at least one cardinal step (Group B). For Comparison 2, Group A was then further subdivided into 2 groups where the attending surgeon performed the entire case 18 times (Group C) and trainees were involved in non-cardinal steps in 28 cases (Group D) (Group A = Group C + Group D).
Patient characteristics and disease profiles were similar in Comparison 1 and Comparison 2 with the exception of BMI in Comparison 1 (Table 1).
Table 1.
Patient demographic and clinicopathologic data
| Comparison 1 | |||
|---|---|---|---|
| Group A | Group B | p value | |
| n=46 | n=53 | ||
| Age, median (IQR) | 66 (60–73) | 64 (60–68) | 0.183 |
| BMI, median (IQR) | 30 (28–37) | 29 (26–32) | 0.015* |
| PSA, median (IQR) | 7.9 (5.2–10.6) | 6.6 (5.4–8.7) | 0.189 |
| ASA, median (IQR) | 3 (2–3) | 3 (2–3) | 0.520 |
| Prostate Volume, median (IQR) | 46 (35–63) | 49 (38–63) | 0.567 |
| Final Gleason, n (%) | 0.426 | ||
| 6 | 1/46 (2.2%) | 4/52 (7.7%) | |
| 7 | 36/46 (78.3%) | 40/52 (76.9%) | |
| ≥8 | 9/46 (19.6%) | 8/52 (15.4%) | |
| Pathologic Stage, n (%) | 0.404 | ||
| pT2a | 2/46 (4.3%) | 1/53 (1.9%) | |
| pT2b | 1/46 (2.2%) | 0 | |
| pT2c | 18/46 (39.1%) | 19/53(36.5%) | |
| pT3a | 13/46 (28.3%) | 23/53 (44.2%) | |
| pT3b | 12/46 (26.1%) | 9/53 (17.3%) | |
| pT4 | 0 | 1/53 (1.9%) | |
| Extracapsular Extension | 25/46 (54.3%) | 33/53 (62.3%) | 0.425 |
| Comparison 2 | |||
|---|---|---|---|
| Group C | Group D | p value | |
| n=18 | n=28 | ||
| Age, median (IQR) | 68 (60–73) | 65 (60–72) | 0.457 |
| BMI, median (IQR) | 33 (27–37) | 29 (28–37) | 0.701 |
| PSA, median (IQR) | 8.2 (6.4–12.4) | 7.5 (4.3–10.9) | 0.246 |
| ASA, median (IQR) | 3 (2–3) | 3 (2–3) | 0.523 |
| Prostate Volume, median (IQR) | 54 (40–68) | 46 (31–60) | 0.105 |
| Final Gleason, n (%) | 0.648 | ||
| 6 | 0 | 1/28 (3.6%) | |
| 7 | 15/18 (83.3%) | 21/27 (75.0%) | |
| ≥8 | 3/18 (16.7%) | 6/27 (21.4%) | |
| Pathologic Stage, n (%) | 0.579 | ||
| pT2a | 1/18 (5.6%) | 1/28 (3.6%) | |
| pT2b | 1/18 (5.6%) | 0 | |
| pT2c | 8/18 (44.4%) | 10/28 (35.7%) | |
| pT3a | 5/18 (27.8%) | 8/28 (28.6%) | |
| pT3b | 3/18 (16.7%) | 9/28 (32.1%) | |
| pT4 | 0 | 0 | |
| Extracapsular Extension | 8/18 (44.4%) | 17/28 (60.7%) | 0.280 |
Group A: attending surgeon performed at least all cardinal steps of prostatectomy
Group B: trainee performed at least one cardinal step of prostatectomy
denotes p < 0.05
Group C: attending surgeon performed all steps of prostatectomy
Group D: attending surgeon performed all cardinal steps and trainee performed at least 1 non-cardinal step of prostatectomy
APM profile in Comparison 1
There were significant differences in APMs when comparing cases where an attending surgeon performed at least all cardinal steps (Group A) and those where a trainee performed at least cardinal step (Group B) (Table 2). Console time was shorter for Group A vs Group B (120 vs 137 min, p < 0.02). In the dominant instrument, moving time was less (100 vs 117 min, p < 0.01), velocity was higher (1.9 vs 1.7 cm/s, p < 0.01), and articulation was greater (53.2 vs 46.5 rad/min, p < 0.01) for Group A vs Group B. In the non-dominant instrument, moving time was less (102 vs 123, p < 0.01) and articulation was greater (29.7 vs 24.1 rad/min, p < 0.01) for Group A vs Group B. Both camera movement time and third instrument usage was lower (p < 0.02 and p < 0.03, respectively) for Group A vs Group B.
Table 2.
Comparison of Automatic Performance Metric (APM) Profile between Group A and B (Comparison 1)
| APMs | Group A | Group B | p value |
|---|---|---|---|
| n=46 | n=53 | ||
| Median (IQR) | Median (IQR) | ||
| Console time, min | 120 (101–153) | 137 (119–161) | 0.022* |
| Dominant instrument | |||
| Moving time, min | 100 (81–117) | 117 (103–136) | 0.003* |
| Velocity, cm/s | 1.9 (1.7–2.0) | 1.7 (1.6–1.8) | 0.003* |
| Articulation, rad/min | 53.2 (49.6–61.6) | 46.5 (41.3–51.4) | <0.001* |
| Non-dominant instrument | |||
| Moving time, min | 102 (81–119) | 123 (106–140) | <0.001* |
| Velocity, cm/s | 1.1 (1.0–1.2) | 1.0 (0.9–1.1) | 0.084 |
| Articulation, rad/min | 29.7 (26.0–40.5) | 24.1 (22.5–27.8) | <0.001* |
| Other | |||
| Camera movement time, min | 5.8 (4.7–8.3) | 7.1 (6.3–8.4) | 0.018* |
| Third instrument usage count | 319 (167–750) | 358 (306–438) | 0.029* |
Group A: attending surgeon performed at least all cardinal steps of prostatectomy
Group B: trainee performed at least one cardinal step of prostatectomy
denotes p < 0.05
Post-operative outcome in Comparison 1
There were no significant differences in estimated blood loss (EBL), length of hospital stay (LOS), positive surgical margin (PSM), days to continence, or complication rates amongst Groups A and B (Table 3). The only significant differences were captured in surgery time and readmission rates with longer time (226 vs 247 min, p = 0.016) in Group B., but greater readmission rate in Group A(10.9% vs 0%) (p = 0.014 for both). The reasons for re-admission were urinary tract infection, hemoptysis, lymphocele, dehydration, and incisional hernia.
Table 3.
Comparison of post-operative outcomes between Group A and B (Comparison 1)
| Post operative outcomes | Group A | Group B | p value |
|---|---|---|---|
| n=46 | n=53 | ||
| Median (IQR) | Median (IQR) | ||
| Surgery time, min | 226 (197–256) | 247 (22–277) | 0.016* |
| Estimated blood loss, ml | 100 (95–150) | 100 (100–150) | 0.361 |
| Length of hospital stay, days | 2 (1–2) | 1 (1–2) | 0.238 |
| Days to continence | 187 (82–304) | 171 (85–345) | 0.882 |
| Positive surgical margin, % | 28.3 | 20.8 | 0.385 |
| Complication, % | 10.8 | 3.8 | 0.170 |
| Readmission, % | 10.9 | 0 | 0.014* |
Group A: attending surgeon performed at least all cardinal steps of prostatectomy
Group B: trainee performed at least one cardinal step of prostatectomy
denotes p < 0.05
APM profile in Comparison 2
In cases comparing an attending surgeon who performed the whole case (Group C) and those with trainee involvement during a non-cardinal step (Group D), there were no significant differences in console time, camera movement time, third instrument usage count or dominant instrument moving time and velocity (Table 4). There were no significant differences in non-dominant instrument moving time, velocity, or articulation. There was a significant difference in dominant instrument articulation (56.3 vs 51.7 rad/min, p < 0.05), which was greater in Group C vs Group D.
Table 4.
Comparison of Automatic Performance Metric (APM) Profile between Group C and D (Comparison 2)
| APMs | Group C | Group D | p value |
|---|---|---|---|
| n=18 | n=28 | ||
| Median (IQR) | Median (IQR) | ||
| Console time, min | 105 (82–149) | 124 (108–153) | 0.084 |
| Dominant instrument | |||
| Moving time, min | 89 (66–121) | 103 (93–116) | 0.091 |
| Velocity, cm/s | 1.9 (1.8–2.1) | 1.8 (1.7–2.0) | 0.173 |
| Articulation, rad/min | 56.3 (50.5–66.2) | 51.7 (47.2–57.7) | 0.042* |
| Non-dominant instrument | |||
| Moving time, min | 92 (60–113) | 103 (86–124) | 0.141 |
| Velocity, cm/s | 1.2 (1.0–1.3) | 1.1 (0.9–1.2) | 0.363 |
| Articulation, rad/min | 31.1 (25.1–42.4) | 29.5 (26.2–38.1) | 0.928 |
| Other | |||
| Camera movement time, min | 5.2 (3.8–8.7) | 5.1 (6.7–8.4) | 0.091 |
| Third instrument usage count | 309 (139–424) | 330 (162–391) | 0.870 |
Group C: attending surgeon performed all steps of prostatectomy
Group D: attending surgeon performed all cardinal steps and trainee performed at least 1 non-cardinal step of prostatectomy
denotes p < 0.05
Post-operative outcome in Comparison 2
There were no significant differences in EBL, LOS, PSM, complication rates, or re-admission rates in Comparison 2 (Table 5). Surgery time (skin to skin) was shorter in cases where the attending surgeon performed the entire case (p < 0.05).
Table 5.
Comparison of post-operative outcomes between Group C and D (Comparison 2)
| Post operative outcomes | Group C | Group D | p value |
|---|---|---|---|
| n=18 | n=28 | ||
| Median (IQR) | Median (IQR) | ||
| Surgery time, min | 209 (172–234) | 228 (210–269) | 0.048* |
| Estimated blood loss, ml | 125 (100–200) | 100 (80–150) | 0.121 |
| Length of hospital stay, days | 2 (1–3) | 2 (1–2) | 0.199 |
| Days to continence | 178 (94–247) | 187 (82–359) | 0.417 |
| Positive surgical margin, % | 22.2 | 32.1 | 0.522 |
| Complication, % | 11.1 | 10.7 | 1.000 |
| Readmission, % | 10.7 | 10.9 | 1.000 |
Group C: attending surgeon performed all steps of prostatectomy
Group D: attending surgeon performed all cardinal steps and trainee performed at least 1 non-cardinal step of prostatectomy
denotes p < 0.05
Discussion
Our study shows that trainee involvement in RARP does not jeopardize patient safety or clinical outcomes. While there was a difference in re-admission rates between cases where the attending performed all cardinal steps and cases where a trainee performed at least one cardinal step, the reasons for re-admission were urinary tract infection, hemoptysis, lymphocele, dehydration, and incisional hernia -- none of which seem to be related to robotic performance. We also found that APM profiles differed significantly with trainee involvement in cardinal steps, but not with trainee involvement in non-cardinal steps. This suggests that these four steps can better distinguish surgeons of varying skill levels, consistent with our previous analysis [8].
To our knowledge, this is the first study with identification of trainees as operating surgeon during a specific step of urologic surgery. Previous studies have either used internal or ACS-NSQIP databases with binary markers of resident participation without the ability to account for degree of resident involvement [2–7]. With the variability in trainee autonomy between residency programs, it is difficult to determine the extent a resident is involved in a procedure even if the case is labeled as including a resident. Our findings confirm trainee involvement, which is also detectable via APM profile, with demonstration of no significant difference in EBL, LOS, or complication rate. More follow-up will be necessary to evaluate whether APMs may be tied to long-term data such as oncologic and functional outcomes. Our results support continued hands-on involvement of trainee surgeons, especially in robotic surgical training where a significant learning curve is known to exist [13]. Moreover, this involvement will lead to greater experience and improved future patient outcomes for trainees.
The present study has some limitations. We are limited by a small sample size at a single institution. Also, the skill of the trainee surgeons (i.e., junior resident versus chief resident) vary and this was not accounted for in this study. These limitations limit the generalizability of our study and external validation is needed. We are also limited by a short follow-up for oncologic and functional outcomes, but hope to address this in future studies. Additionally, the study did not draw any association or correlation between APMs and clinical outcomes. These were compared independently of each other.
Future directions of study include an evaluation of long term outcomes (i.e. functional and oncologic survival) as well as a more granular analysis of degree of trainee involvement and trainee level. The ultimate goal of this analysis would be to create a robotic surgical curriculum that could provide trainees with actionable feedback while maintaining a safe training environment for patients.
Conclusion
Trainee involvement in RARP is safe. APM profiles differ significantly when trainees perform at least 1 cardinal step but do not change as much when trainees are responsible for non-cardinal steps. Regardless of the degree of trainee involvement, there is no significant differences in major clinical outcomes.
Acknowledgements
Research reported in this publication was supported in part by the National Institute Of Biomedical Imaging And Bioengineering of the National Institutes of Health under Award Number K23EB026493 and an Intuitive Surgical Clinical Research Grant. Anthony Jarc and Liheng Guo (Intuitive Surgical, Inc.) assisted with automated performance metric processing.
Footnotes
Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.
Disclosure of potential conflicts of interest The study was supported in part by an Intuitive Surgical, Inc. clinical grant. Intuitive Surgical, Inc. provided the systems events data recorder.
Research involving Human Participants and/or Animals All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed Consent Informed consent was obtained from all individual participants included in the study.
References
- 1.Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ, Michigan Bariatric Surgery C (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369 (15):1434–1442. doi: 10.1056/NEJMsa1300625 [DOI] [PubMed] [Google Scholar]
- 2.Castleberry AW, Clary BM, Migaly J, Worni M, Ferranti JM, Pappas TN, Scarborough JE (2013) Resident education in the era of patient safety: a nationwide analysis of outcomes and complications in resident-assisted oncologic surgery. Ann Surg Oncol 20 (12):3715–3724. doi: 10.1245/s10434-013-3079-2 [DOI] [PubMed] [Google Scholar]
- 3.Raval MV, Wang X, Cohen ME, Ingraham AM, Bentrem DJ, Dimick JB, Flynn T, Hall BL, Ko CY (2011) The influence of resident involvement on surgical outcomes. J Am Coll Surg 212 (5):889–898. doi: 10.1016/j.jamcollsurg.2010.12.029 [DOI] [PubMed] [Google Scholar]
- 4.Matulewicz RS, Pilecki M, Rambachan A, Kim JY, Kundu SD (2014) Impact of resident involvement on urological surgery outcomes: an analysis of 40,000 patients from the ACS NSQIP database. J Urol 192 (3):885–890. doi: 10.1016/j.juro.2014.03.096 [DOI] [PubMed] [Google Scholar]
- 5.Ruhotina N, Dagenais J, Gandaglia G, Sood A, Abdollah F, Chang SL, Leow JJ, Olugbade K Jr., Rai A, Sammon JD, Schmid M, Varda B, Zorn KC, Menon M, Kibel AS, Trinh QD (2014) The impact of resident involvement in minimally-invasive urologic oncology procedures. Can Urol Assoc J 8 (9–10):334–340. doi: 10.5489/cuaj.2170 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Allard CB, Meyer CP, Gandaglia G, Chang SL, Chun FK, Gelpi-Hammerschmidt F, Hanske J, Kibel AS, Preston MA, Trinh QD (2015) The Effect of Resident Involvement on Perioperative Outcomes in Transurethral Urologic Surgeries. J Surg Educ 72 (5): 1018–1025. doi: 10.1016/j.jsurg.2015.04.012 [DOI] [PubMed] [Google Scholar]
- 7.Caveney M, Matthews C, Mirzazadeh M (2017) The Effect of Resident Involvement in Pelvic Prolapse Surgery: A Retrospective Study From a Nationwide Inpatient Sample. Female Pelvic Med Reconstr Surg 23 (6):387–391. doi: 10.1097/SPV.0000000000000436 [DOI] [PubMed] [Google Scholar]
- 8.Hung AJ, Oh PJ, Chen J, Ghodoussipour S, Lane C, Jarc A, Gill IS (2019) Experts vs superexperts: differences in automated performance metrics and clinical outcomes for robot-assisted radical prostatectomy. BJU Int 123 (5):861–868. doi: 10.1111/bju.14599 [DOI] [PubMed] [Google Scholar]
- 9.Chen J, Cheng N, Cacciamani G, Oh P, Lin-Brande M, Remulla D, Gill IS, Hung AJ (2019) Objective Assessment of Robotic Surgical Technical Skill: A Systematic Review. J Urol 201 (3):461–469. doi: 10.1016/j.juro.2018.06.078 [DOI] [PubMed] [Google Scholar]
- 10.Hung AJ, Chen J, Jarc A, Hatcher D, Djaladat H, Gill IS (2018) Development and Validation of Objective Performance Metrics for Robot-Assisted Radical Prostatectomy: A Pilot Study. J Urol 199 (1):296–304. doi: 10.1016/j.juro.2017.07.081 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Aj Hung, Chen J, Ghodoussipour S, Oh PJ, Liu Z, Nguyen J, Purushotham S, Gill IS, Liu Y (2019) A deep-learning model using automated performance metrics and clinical features to predict urinary continence recovery after robot-assisted radical prostatectomy. BJU Int. doi: 10.1111/bju.14735 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Patel VR, Sivaraman A, Coelho RF, Chauhan S, Palmer KJ, Orvieto MA, Camacho I, Coughlin G, Rocco B (2011) Pentafecta: a new concept for reporting outcomes of robot-assisted laparoscopic radical prostatectomy. Eur Urol 59 (5):702–707. doi: 10.1016/j.eururo.2011.01.032 [DOI] [PubMed] [Google Scholar]
- 13.Herrell SD, Smith JA Jr., (2005) Robotic-assisted laparoscopic prostatectomy: what is the learning curve? Urology 66 (5 Suppl):105–107. doi: 10.1016/j.urology.2005.06.084 [DOI] [PubMed] [Google Scholar]