Key Points
Question
How are mentorship and a formal robotics curriculum associated with the learning curves of the robotic pancreaticoduodenectomy and patient safety?
Findings
In this review, compared with no mentorship/curriculum, operating room time for the robotic whipple procedure was seen to decrease in surgeons with mentorship but no robotics curriculum. A further decrease was noted in surgeons with both mentorship and a formal curriculum.
Meaning
Mentorship and a formal skills curriculum may decrease the learning curves of newer surgeons without compromising patient safety.
Abstract
Importance
Learning curves are unavoidable for practicing surgeons when adopting new technologies. However, patient outcomes are worse in the early stages of a learning curve vs after mastery. Therefore, it is critical to find a way to decrease these learning curves without compromising patient safety.
Objective
To evaluate the association of mentorship and a formal proficiency-based skills curriculum with the learning curves of 3 generations of surgeons and to determine the association with increased patient safety.
Design, Setting, and Participants
All consecutive robotic pancreaticoduodenectomies (RPDs) performed at the University of Pittsburgh Medical Center between 2008 and 2017 were included in this study. Surgeons were split into generations based on their access to mentorship and a proficiency-based skills curriculum. The generations are (1) no mentorship or curriculum, (2) mentorship but no curriculum, and (3) mentorship and curriculum. Univariable and multivariable analyses were used to create risk-adjusted learning curves by surgical generation and to analyze factors associated with operating room time, complications, and fellows completing the full resection. The participants include surgical oncology attending surgeons and fellows who participated in an RPD at University of Pittsburgh Medical Center between 2008 and 2017.
Main Outcomes and Measures
The primary outcome was operating room time (ORT). Secondary outcomes were postoperative pancreatic fistula and Clavien-Dindo classification higher than grade 2.
Results
We identified 514 RPDs completed between 2008 and 2017, of which 258 (50.2%) were completed by first-generation surgeons, 151 (29.3%) were completed by the second generation, and 82 (15.9%) were completed by the third generation. There was no statistically significant difference between groups with respect to age (66.3-67.3 years; P = .52) or female sex (n = 34 [41.5%] vs n = 121 [46.9%]; P = .60). There was a significant decrease in ORT (P < .001), from 450.8 minutes for the first-generation surgeons to 348.6 minutes for the third generation. Additionally, across generations, Clavien-Dindo classification higher than grade 2 (n = 74 [28.7%] vs n = 30 [9.9%] vs n = 12 [14.6%]; P = .01), conversion rates (n = 18 [7.0%] vs n = 7 [4.6%] vs n = 0; P = .006), and estimated blood loss (426 mL vs 288.6 mL vs 254.7 mL; P < .001) decreased significantly with subsequent generations. There were no significant differences in postoperative pancreatic fistula.
Conclusions and Relevance
In this study, ORT, conversion rates, and estimated blood loss decreased across generations without a concomitant rise in adverse patient outcomes. These findings suggest that a proficiency-based curriculum coupled with mentorship allows for the safe introduction of less experienced surgeons to RPD without compromising patient safety.
This study evaluates the association of mentorship and a formal proficiency-based skills curriculum with the learning curves of 3 generations of surgeons and the association with increased patient safety.
Introduction
The introduction of the surgical robot in 2000 was a major advancement in minimally invasive surgery and has become a disruptive technology in medical practice.1,2,3 Patient outcomes are often worse in earlier phases of a learning curve compared with the later phases.4,5 In parallel with the rise of robotic surgery, there has been increased scrutiny of quality and outcomes. This highlights the need for new ways to teach new techniques without compromising patient outcomes.
The pancreaticoduodenectomy (PD) is one of the more complex intra-abdominal procedures performed by surgeons. The open PD (OPD) has a learning curve of 50 to 80 cases,6,7,8 with outcomes correlating directly with surgeon and center experience. The institutional learning curve for robotic PD (RPD) was similarly found to be 80 cases3 at our center, the University of Pittsburgh Medical Center. When done by experienced surgeons past the institutional learning curve, RPD has been shown to be noninferior to OPD,9,10,11 and in combination with enhanced recovery after surgery (ERAS) protocol, RPD significantly decreased length of stay and cost.12 Despite these benefits, the significant learning curve is prohibitive to many surgeons looking to adopt the procedure in their practice. It is thus critical to develop strategies that mitigate the learning curve for safe intuitional adoption of the technique.
Despite multiple studies analyzing the learning curve for open PD, little work has examined strategies to minimize its effect on patient care. With recognition of the importance of technical skills to patient outcomes,13,14 the surgical oncology group at the University of Pittsburgh Medical Center (UPMC) developed a 5-step proficiency-based robotic training curriculum.15 The curriculum consists of a simulation component,16 inanimate biotissue practice sessions,17,18 video library review of cases,19 intraoperative feedback, and ongoing quality assurance. Introduction of the formal training program has increased surgical oncology fellows’ involvement in cases15 and technical skills.16,17 However, to date, there has been no study on mentorship or the training curriculum’s association with learning curves or patient safety. The primary aim of this study is to evaluate the association of mentorship and a proficiency-based robotic curriculum with the learning curves of subsequent generations. A secondary aim is to determine whether integrating trainees was adversely associated with patient safety and outcomes. We hypothesize that a combination of formal mentorship and a proficiency-based robotic skills curriculum would decrease the learning curve for trainees without compromising patient safety.
Methods
Surgical Generations
Three generations of surgeons were defined for this study. The first generation consisted of 2 surgeons (H.J.Z. and A.J.M.) who were early adopters of the RPD but received no mentorship and did not undergo a formal curriculum. These 2 surgeons overcame the institutional learning curve.3 The second generation consisted of 2 surgeons (A.H.Z. and M.E.H.) who started after the institutional learning curve and benefited from mentorship of the first-generation surgeons. In 2013, the formal robotics curriculum was implemented at UPMC as previously described.15,16,17,18,19 The third generation benefitted from both mentorship and the formal robotics proficiency-based curriculum. The third generation includes all UPMC Complex General Surgical Oncology and hepatobiliary (HPB) fellows in graduating classes 2014 to 2017.
Database Creation
After obtaining approval from the UPMC institutional review board, a retrospective review of prospectively collected data on consecutive RPDs performed from October 2008 to December 2017 was conducted. Patient consent was not obtained because this was not an institutional requirement from deidentified retrospective databases. All patient information, outcomes, and intraoperative data were included in the initial database. Trainee console involvement was then added to the outcomes database. Trainees tracked their own case participation using case logs15 (eFigure in the Supplement). Logged cases were checked against fellow rotation schedules to ensure they were on service and further checked against the intraoperative report to ensure the fellows were recorded as being in the room during the case. Any step not logged by a fellow on the console was assumed to be done by the primary attending surgeon, defined as the surgeon who dictated the operative note.
Defining Third-Generation Fellow Cases
Fellows rotate on the pancreas service during their second year. At this point they have completed simulation, performed numerous biotissue drills, observed surgeries and videos, and rotated on other surgical oncology services where robotics surgeries are performed as part of the proficiency-based curriculum. The RPD is divided into 7 major steps: (1) mobilization, (2) portal dissection, (3) uncinate dissection, (4) cholecystectomy, (5) pancreaticojejunostomy, (6) hepaticojejunostomy, and (7) gastrojejunostomy. More broadly, the RPD can be subdivided into resection (steps 1-4) and reconstruction (steps 5-7). For this study, a case was defined as third generation or a fellow case if the fellow completed the entire resection (steps 1-4).
Outcomes and Data Collection
Operating room time (ORT) has been shown to be the most difficult outcome to improve3 and is also the most consistent variable reported in most learning curve articles3,20,21,22,23,24,25 and was therefore used as the primary outcome in assessment of the learning curve. Recorded ORT was from skin incision to closure. Secondary outcomes used to track patient safety are clinically relevant postoperative pancreatic fistula (CR-POPF) as defined by the International Study Group on Pancreatic Fistula Definition,26 major complications (Clavien-Dindo classification higher than grade 2),27 and fellows completing the entire resection. Portal vein (PV) clamp use was performed in patients with venous abutment and a PV resection: side bite, patch, or end-to-end anastomosis. Given the small numbers, these 3 types of PV resections were added as a composite variable of PV clamp, which usually consisted of clamping the portal vein, superior mesenteric vein, and splenic vein for vascular control.
Protocol Evolution
Patients in all 3 generations underwent the same perioperative protocols. There were notable changes in the study period. After 2009, neoadjuvant protocols were in place for high-risk patients with pancreatic cancer, and after 2013, neoadjuvant protocols were in place for resectable pancreatic cancer. In 2014, a “no intensive care unit” policy was initiated for whipple procedures, and in 2015, an “early recovery after surgery” protocol was instituted. Additionally, the International Study Group on Pancreatic Fistula Definition study group revised their nomenclature in 2016 to eliminate “grade A leaks” and the “Verona” protocol for drain management had updated from 3× serum amylase on postoperative day 3 to less than 5000 on postoperative day 1. The group universally adapted to all these modifications.
Statistical Analysis
Differences in continuous variables between generations were all analyzed using Kruskal-Wallis. For categorical variables, likelihood ratio χ2 tests were used. Univariable and multivariable logistic regression modeling was performed for CR-POPF and greater than 2 Clavien complications. Generalized linear modeling was used to model ORT. Multivariable analysis was performed in a backward stepwise analysis and presented as odds ratios (ORs) with 95% confidence intervals, estimated probability, coefficients, or difference from baseline. For multivariable analysis, all calculations were done using only the nonmissing values except for gland texture and duct size, which were replaced with mean and median imputations, respectively. Estimated blood loss (EBL) also had 1 missing value that was imputed at the median. The learning curves display the predicted ORT by case number after adjusting for all the other variables in the model. All statistical tests were 2-sided, and differences were considered significant when P was less than .05. Statistical analyses were performed using Stata/SE statistical software package, version 15.1 (StataCorp).
Results
Cohort Characteristic by Surgical Generation
A total of 514 consecutive RPD cases were performed. Two attending surgeons were in the first generation, 2 attending surgeons were in the second generation, and 24 surgical trainees in the third generation. Of the 514 cases, 258 (50.2%) were completed by the first generation, 151 (29.3%) were completed by the second generation, and 82 (15.9%) were completed by the third generation. Twenty-three cases were omitted from analysis, which were performed by 5 other surgeons learning from first- and second-generation attending surgeons without knowledge of who performed the resection phase.
Patient Demographics
Preoperative characteristics were similar except for higher American Society of Anesthesiologists scores greater than 3 (78.3% vs 88.1% vs 84.1% for first, second, and third generations, respectively; P = .04) and more neoadjuvant chemotherapy receipt (26.0% vs 37.1% vs 36.6%; P = .03) in subsequent generations (Table 1). Perioperative outcomes improved with subsequent generations: decrease in ORT (450.8 minutes vs 388.5 minutes vs 348.6 minutes; P < .001), conversion (7.0% vs 4.6% vs 0%; P = .006), EBL (426 mL vs 288.6 mL vs 254.7 mL; P < .001), and packed red blood cell units transfused (18.2% vs 11.3% vs 7.3%; P = .02) and increase in total lymph nodes harvested (27.4 lymph nodes vs 30.8 lymph nodes vs 34.0 lymph nodes; P < .001) (Table 1). There were no differences in pathology, concomitant procedure, tumor size, or positive margin status. Postoperatively, no significant differences were seen between generations for CR-POPF, readmission, wound infection, or mortality within 90 days. However, there was a statistically significant reduction in length of stay (11.0 days vs 9.7 days vs 8.1 days; P < .001) and more than 2 Clavien complications, from 28.7% in the first generation to 14.6% in the third generation (P = .01) (Table 1).
Table 1. Cohort Characteristics by Surgeon Generations.
| Variables | Generation, No. (%) | |||
|---|---|---|---|---|
| First (n = 258) | Second (n = 151) | Third (n = 82) | P valuea | |
| Preoperative characteristics | ||||
| Age, mean (SD), y | 67.3 (11.7) | 66.3 (11.1) | 66.9 (10.0) | .52 |
| Female | 121 (46.9) | 71 (47.0) | 34 (41.5) | .66 |
| Prior abdominal surgery | 143 (55.4) | 85 (56.3) | 37 (45.1) | .21 |
| ASA score (≥3) | 202 (78.3) | 133 (88.1) | 69 (84.1) | .04 |
| BMI, mean (SD) | 27.7 (5.4) | 28.1 (6.3) | 26.2 (5.0) | .05 |
| Neoadjuvant chemotherapy | 67 (26.0) | 56 (37.1) | 30 (36.6) | .03 |
| Perioperative outcomes | ||||
| ORT, mean (SD), min | 450.8 (121.0) | 388.5 (76.7) | 348.6 (52.6) | <.001 |
| Conversion | 18 (7.0) | 7 (4.6) | 0 | .006 |
| EBL, mean (SD), mL | 426.6 (525.8) | 288.6 (335.0) | 254.7 (274.1) | <.001 |
| pRBC units transfused | 47 (18.2) | 17 (11.3) | 6 (7.3) | .02 |
| Pathology | ||||
| PDA | 129 (50.0) | 80 (53.0) | 37 (45.1) | .46 |
| AMP | 41 (15.9) | 16 (10.6) | 14 (17.1) | |
| IPMN | 33 (12.8) | 13 (8.6) | 11 (13.4) | |
| NET | 17 (6.6) | 13 (8.6) | 8 (9.8) | |
| CCA | 16 (6.2) | 15 (9.9) | 3 (3.7) | |
| Other | 22 (8.5) | 14 (9.3) | 9 (11.0) | |
| PV isolation clamp | 41 (15.9) | 32 (21.2) | 8 (9.8) | .07 |
| Concomitant procedure | 4 (1.6) | 6 (4.0) | 5 (6.1) | .09 |
| Tumor size, mean (SD), cm | 2.8 (1.5) | 2.7 (1.4) | 2.8 (1.4) | .88 |
| Positive margin | 35 (13.6) | 18 (11.9) | 7 (8.5) | .45 |
| Total nodes, mean (SD) | 27.4 (13.4) | 30.8 (10.0) | 34.0 (13.7) | <.001 |
| Postoperative outcomes | ||||
| Wound infection | 31 (12.1) | 21 (13.9) | 11 (13.4) | .85 |
| CR-POPF | 21 (8.1) | 11 (7.3) | 5 (6.1) | .82 |
| Any Clavien-Dindo complication | 180 (69.8) | 103 (68.2) | 50 (61.0) | .34 |
| Clavien-Dindo >2 complications | 74 (28.7) | 30 (19.9) | 12 (14.6) | .01 |
| Length of stay, mean (SD), d | 11.0 (9.3) | 9.7 (7.0) | 8.1 (7.2) | <.001 |
| Readmission within 90 d | 94 (37.2) | 46 (31.7) | 30 (40.0) | .40 |
| Mortality within 90 d | 7 (2.8) | 5 (3.4) | 2 (2.7) | .93 |
Abbreviations: AMP, ampullary; ASA, American Society of Anesthesiologists; BMI, body mass index (calculated as weight in kilograms divided by height in meters squared); CCA, cholangiocarcinoma; CR-POPF, clinically relevant post-operative pancreatic fistula; EBL, estimated blood loss; IPMN, intrapapillary mucinous neoplasm; NET, pancreatic neuroendocrine tumor; ORT, operating room time; PDA, pancreatic ductal adenocarcinoma; pRBC, packed red blood cells; PV, portal vein.
Statistically significant values at P less than .05.
Operating Room Time Multivariable Analysis
Several patient factors contributed to increased ORT (Table 2) including decreased age (coefficient = 0.008; P < .001), increased EBL, decreased lymph nodes harvested (coefficient = −0.002; P = .003), and PV clamp use (coefficient = 0.078; P < .001). Pathology of ampullary cancers (coefficient = −0.051; P = .02) and intrapapillary mucinous neoplasm (coefficient = −0.072; P = .002) showed decrease in ORT compared with pancreas cancer and cholangiocarcinoma, where pancreatic neuroendocrine tumors had a trend toward decreased ORT (coefficient = −0.053; P = .09). Table 3 illustrates the magnitude of time change by variable. For example, 1000 mL of blood loss was associated with 93.8 more minutes of surgery, a concomitant procedure added 35.5 minutes to the procedure, and surgery for intrapapillary mucinous neoplasm was 29.4 minutes shorter.
Table 2. Multivariate Analysis of Factors Associated With Operating Room Time.
| Factors | Coefficienta | P valueb |
|---|---|---|
| ln(EBL)c | 0.076 | <.001 |
| Age, y | 0.008 | <.001 |
| Age, y2 | −0.0001 | <.001 |
| Lymph nodes harvested | −0.002 | .003 |
| Vascular clamp/resection | 0.078 | <.001 |
| Concomitant procedure | 0.082 | .07 |
| Pathology | ||
| PDA | 1 [Reference] | NA |
| AMP | −0.051 | .02 |
| IPMN | −0.072 | .002 |
| NET | −0.053 | .09 |
| CCA | −0.002 | .95 |
| Other | −0.024 | .37 |
| sqrt (case No.) | −0.012 | .08 |
| Resecting surgeon group | ||
| 1st vs 2nd generation | 0.510 | <.001 |
| 3rd vs 2nd generation | 0.138 | .02 |
| sqrt (case No.) by groupc | ||
| 1st vs 2nd generation | −0.038 | <.001 |
| 3rd vs 2nd generation | 0.002 | .92 |
| Interceptd | 5.521 | <.001 |
Abbreviations: AMP, ampullary; CCA, cholangiocarcinoma; EBL, estimated blood loss; IPMN, intrapapillary mucinous neoplasm; ln, natural logarithm transformation; NA, not applicable; NET, pancreatic neuroendocrine tumor; PDA, pancreatic ductal adenocarcinoma; sqrt, square root transformation.
A positive coefficient indicates a predicted increase in average operation time, while a negative coefficient indicates a predicted decrease in average operation time.
Statistically significant values at P less than .05.
One missing value for EBL was substituted with the median of the nonmissing EBL values.
The intercept applies to all surgeries.
Table 3. Factors Associated With Operating Room Time.
| Variable | Category/value | Mean predicted operative time, min | Difference from baseline category/value |
|---|---|---|---|
| Age, y | 40 | 426.3 | 0 |
| 50 | 429.1 | 2.8 | |
| 60 | 424.8 | −1.5 | |
| 70 | 413.7 | −12.6 | |
| 80 | 396.3 | −30.0 | |
| EBL, mL | 50 | 366.0 | 0 |
| 150 | 397.9 | 31.9 | |
| 250 | 413.7 | 47.7 | |
| 400 | 428.8 | 62.8 | |
| 1000 | 459.8 | 93.8 | |
| Lymph nodes, No. | 10 | 425.8 | 0 |
| 20 | 418.9 | −6.9 | |
| 30 | 412.2 | −13.6 | |
| 40 | 405.5 | −20.3 | |
| 50 | 399.0 | −26.8 | |
| Clamp | No | 407.9 | 0 |
| Yes | 441.2 | 33.3 | |
| Concomitant procedure | No | 412.2 | 0 |
| Yes | 447.4 | 35.5 | |
| Pathology | PDA | 422.5 | 0 |
| AMP | 401.5 | −21.0 | |
| IPMN | 393.1 | −29.4 | |
| NET | 400.6 | −21.9 | |
| CCA | 421.8 | −0.7 | |
| Other | 412.5 | −10.0 | |
| Case No. if generation 1 | 1 | 668.8 | 251.8 |
| 10 | 599.9 | 182.9 | |
| 50 | 492.8 | 75.8 | |
| 90 | 436.4 | 19.4 | |
| 150 | 379.8 | −37.2 | |
| 225 | 330.7 | −86.3 | |
| Case No. if generation 2 | 1 | 417.0 | 0 |
| 10 | 405.9 | −11.1 | |
| 50 | 386.6 | −30.4 | |
| 90 | 375.1 | −41.9 | |
| Case No. if generation 3 | 1 | 364.0 | −53.0 |
| 10 | 355.9 | −61.1 |
Abbreviations: AMP, ampullary; CCA, cholangiocarcinoma; EBL, estimated blood loss; IPMN, intrapapillary mucinous neoplasm; NET, pancreatic neuroendocrine tumor; PDA, pancreatic ductal adenocarcinoma.
Surgeon factors were independently associated with ORT as well. Later generation and later case number by surgeon were associated with lower ORT (Table 2). When setting the first case done by second-generation surgeons as the reference, the ORT was 251.8 minutes faster than first-generation surgeons and 53 minutes slower than third-generation surgeons. The starting point for the second generation was greater than 90 cases compared with first-generation surgeons. The starting point for third-generation surgeons was greater than 90 cases compared with second-generation surgeons and greater than 150 cases from first-generation surgeons (Table 3). Operating room time was risk adjusted for significant variables on multivariable analysis and graphed by generation (Figure). The first-generation learning curve is very steep compared with the second- and third-generation learning curves, which are more flat, but each starts at a much lower ORT than the previous generation.
Figure. Risk-Adjusted Learning Curves by Surgical Generation.
Major Postoperative Complications: Clavien and CR-POPF Multivariate Analysis
The most significant factor associated with increased Clavien-Dindo classifications higher than grade 2 was increased ORT (OR, 3.869; 95% CI, 1.673-8.949; P = .002). Other factors associated with increased Clavien-Dindo classifications higher than grade 2 were increased age, increased BMI, prior abdominal surgery, and PV clamping. Factors associated with decreased Clavien-Dindo classifications higher than grade 2 were lower ASA classification, female sex, hard glands, and neoadjuvant chemotherapy (Table 4).
Table 4. Factors Associated With Clavien-Dindo Classification Higher Than Grade 2 and Clinically Relevant Postoperative Pancreatic Fistula.
| Variablesa | Clavien-Dindo (n = 514) | CR-POPF (n = 514) | ||||
|---|---|---|---|---|---|---|
| Odds ratio (95% CI) | P value | Estimated probability of Clavien-Dindo >2, % | Odds ratio | P value | Estimated probability of CR-POPF, % | |
| Age, y | ||||||
| 50 | 1.028 (1.007-1.050) | .008 | 16.5 | NA | NA | NA |
| 60 | 20.5 | |||||
| 70 | 25.1 | |||||
| 80 | 30.3 | |||||
| BMIb | ||||||
| 20 | 2.637 (0.841-8.269) | .10 | 19.2 | NA | NA | NA |
| 25 | 22.5 | |||||
| 30 | 25.5 | |||||
| 35 | 28.2 | |||||
| ORT, minb | ||||||
| 300 | 3.869 (1.673-8.949) | .002 | 17.4 | 8.278 | .003 | 4.1 |
| 400 | 23.4 | 7.1 | ||||
| 500 | 28.9 | 10.4 | ||||
| 600 | 33.9 | 14.0 | ||||
| ASA classification | ||||||
| ≤2 | 0.614 (0.344-1.097) | .10 | 31.3 | NA | NA | NA |
| ≥3 | 22.5 | |||||
| Prior abdominal surgery | ||||||
| No | 1.491 (0.950-2.340) | .08 | 20.4 | NA | NA | NA |
| Yes | 27.1 | |||||
| Clamp | ||||||
| No | 1.987 (1.102-3.585) | .02 | 22.0 | NA | NA | NA |
| Yes | 34.7 | |||||
| Sex | ||||||
| Male | 0.652 (0.409-1.041) | .07 | 27.2 | NA | NA | NA |
| Female | 20.1 | |||||
| Gland texturec | ||||||
| Soft | 0.696 (0.427-1.136) | .15 | 26.7 | 0.471 | .12 | 9.1 |
| Hard | 20.6 | 4.8 | ||||
| Neoadjuvant | ||||||
| No | 0.607 (0.357-1.032) | .07 | 26.3 | 0.066 | .008 | 10.0 |
| Yes | 18.3 | 0.8 | ||||
| Duct size, mmd | ||||||
| 2 | NA | NA | NA | 0.432 | .05 | 12.3 |
| 4 | 7.7 | |||||
| 6 | 5.7 | |||||
| 8 | 4.6 | |||||
| CCI (age adjust)e | ||||||
| 2 | NA | NA | NA | 0.451 | .01 | 10.9 |
| 4 | 10.1 | |||||
| 6 | 7.0 | |||||
| 8 | 4.0 | |||||
| Model fit (McFaddens R2) | 0.067 | 0.177 | ||||
Abbreviations: ASA, American Society of Anesthesiologists; BMI, body mass index (calculated as weight in kilograms divided by height in meters squared); CCI, Charlson Comorbidity Index; CR-POPF, clinically relevant postoperative pancreatic fistula; NA, not applicable; ORT, operating room time.
Other variables considered in each model were estimated blood loss, lymph nodes, concomitant procedure, percentage of surgery completed by fellow, red blood cell transfusion, surgery conversion, pathology, cancer, other attending surgeon present, tumor size, case number, and surgeon. However, the inclusion of any of these variables did not result in a better model. Statistically significant values were P <.05. All factors listed improved model fit.
A natural log transformation was used on this variable in the models.
36 Missing values of gland texture were replaced with a mean imputation.
46 Missing values of duct size were replaced with a median imputation.
Both a nontransformed and a square root transformed version of CCI (age adjusted) were used in the model.
The most significant factor associated with increased CR-POPF was increased ORT (OR, 8.278; 95% CI, 2.098-32.675; P = .003). Factors associated with decreased CR-POPF were hard glands, neoadjuvant chemotherapy, larger pancreatic ducts, and higher age-adjusted Charlson comorbidity scores (Table 4).
Surgeon factors of generation and case numbers were not associated with increased Clavien-Dindo classifications higher than grade 2 or CR-POPF. In both models, EBL, lymph nodes, concomitant procedure, percentage of surgery completed by fellow, red blood cell transfusion, conversion, pathology, tumor size, case number, and surgeon were all included. However, their inclusion did not result in a better model.
Factors Associated With Fellow Resection
Fellow case logs were collected after 2013, and fellows logged 298 of 349 RPDs during this time. Fellow cases logged were analyzed for the 3 primary surgeons during this time, H.Z., A.Z., and M.H., of which the fellows performed on average 49%, 40%, and 54% of the whipple procedures, respectively. Multivariable analysis was performed to predict resections for third-generation surgeons. Fellows were more likely to complete the entire resection from the console if they had completed a previous resection (OR, 3.142; 95% CI, 2.185-4.517; P < .001). Fellows were less likely to complete the entire resection if more than 1 attending surgeon (indicative of attending surgeon’s learning curve or difficult resection, ie, PV clamp) was present (OR, 0.176; 95% CI, 0.092-0.334; P < .001) or if the patient had a high body mass index (OR, 0.141; 95% CI, 0.033-0.599; P = .008). Subgroup analysis was done on each individual surgeon. For the most junior surgeon (M.H.), attending case number was also associated with fellow completion of resection (OR, 3.348; 95% CI, 1.657-6.764; P < .001) but not for more senior attending surgeons beyond their learning curve in the time after 2013. For this attending surgeon, before case 15, fellows were likely to complete 5.7% of resections, whereas after 75 cases, fellows were likely to complete 58.1% of resections.
Discussion
This analysis demonstrates that implementation of formal mentorship and a robotic curriculum was associated with a decreased learning curve for subsequent generations. Several patient outcomes, including Clavien-Dindo classifications higher than grade 2, improved over generations despite surgeries being performed by less experienced surgeons. Operative time was not only a critical variable in determining the learning curve, but it was also an important factor associated with of 2 key outcomes after PD: major complications and CR-POPF. Outcomes of CR-POPF were all associated with patient-specific factors and not with surgeon generation or case experience. These results indicate that subsequent generations of attending surgeons and fellows can benefit from surgical mentorship as well as a formal curriculum.
To our knowledge, Boone et al3 published the first article on RPD learning curve for the institution and the largest in the literature. Four other studies have also been published, all of which use ORT as their primary end point and have less than 70 patients. Essentially, the smaller the study, the shorter the learning curve, which likely means as the numbers grow in future studies, so likely will the reported learning curve. Boone et al3 showed a learning curve of 80 for ORT, 40 for POPF and lymph node harvest, and 20 for EBL and conversion.3 It is essentially the first generation and institution’s learning curve. An institutional learning curve goes beyond the mere technical and procedural competence of the surgeon to encompass optimal port placement, optimal sequencing, training of operating room staff, having appropriate and efficient access to equipment, and postoperative management strategies and is reflective of a more pioneering discovery curve for the surgeons and institution. To our knowledge, this study is the first study to look at the learning curve of the next generation of adopters as well as the first to look at the learning curve of trainees who have undergone a formal curriculum. However, all generations are reflective of 1 institution. A cumulative sum control chart analysis was not performed as was done previously for comparison purposes, but to our knowledge, this is the first study to perform multivariable analysis and report risk-adjusted learning curves for ORT. Operative time as a variable can be controversial because it could reflect operative skill, as reported by the Birkmeyer et al study,13 looking at bariatric surgery complications and many other studies.13 It can also be a surrogate for more difficult cases.28,29,30 These data show that ORT is associated with Clavien-Dindo classifications higher than grade 2 and CR-POPF, likely reflecting more difficult cases, and these factors were accounted for in the generational analysis of ORT.
The steep 80-case learning curve for ORT was not repeated in the second generation and third generation (Figure). Additionally, each subsequent generation ORT starting point is beyond 90 cases from the previous generation and does not repeat the subsequent learning curves related to EBL, conversion, and lymph node harvest. In fact, the outcomes are better in subsequent generations for these variables. The learning curves for the second and third generation are more flat and perhaps plateau earlier (10-20 cases). The third generation likely does not have enough cases to truly identify a curve yet and are likely in part reflective of the first and second generations’ learning curves’ comfort to turn over the case. Third-generation cases are as much a reflection of the learning curve for generations 1 and 2 and are thus interesting to study for the reason of generation 1 and 2’s willingness to let a trainee perform. Each generation does have a later time, but all groups overlap at later periods and have standardized procedures and perioperative protocols, making comparisons meaningful (as mentioned in the Methods section). The key difference is the second generation with formal mentorship did not have to recreate port placement or procedure sequencing, and had coaching through their learning curve. The third generation had that mentorship but also underwent a rigorous 5-step proficiency-based skills curriculum for more than a year prior to being on the console for RPD. This allowed them more opportunity than previous fellows to operate as well as allowed them better technical skills, which we argue are responsible for improved outcomes and console time.
National database studies would not support that most centers performing minimally invasive PD (MIPD) are growing programs. Torphy et al31 used the National Cancer Database, which showed that, in 2010, 153 centers were performing MIPD with a median of 1 per hospital, and in 2015, 266 centers were performing MIPD with a median of 2 per year.31 In addition to the negligent growth in the 5-year period, 462 centers had performed MIPD during that time, showing that many centers were likely abandoning the technique. Adams et al32 used the National Inpatient Sample to show that 229 hospitals performed 865 cases and 20% only performed 1 per year,32 and an earlier National Cancer Database study33 showed 49% of hospitals performed 1 per year. Although impossible to know from these data sets, absence of formal training and subsequently poor outcomes were potentially large contributing factors to slow growth and/or abandonment of this technique.
Training has largely fallen to industry, and their standard is to (1) perform online modules, (2) complete an 8-hour pig laboratory, (3) do 1 case observation,34 and (4) post 3 cases to get proctored. This paradigm is inadequate for an RPD and other complex operations. That was the training paradigm used for the first-generation surgeons, and the proctor was not an HPB surgeon performing RPDs who could provide guidance on how to place ports, set-up, or sequence operation for an RPD. In 2019, this type of training regimen should no longer be acceptable. In 2008, RPD based on the Idea, Development, Exploration, Assessment, Long-term study (IDEAL) framework was in phase 1 as “proof of concept”; in 2019, it is in or nearing Phase 3, “assessment.”35 For low-volume, high-complexity procedures and for techniques and technologies acquired after training, procedure-targeted technical curriculums are ideal approaches.36 Hospital credentialing bodies need to be more rigorous before providing privileges for new technologies and should be procedure specific, as seen for open surgery. Granting robotic privileges after a surgeon performed 3 proctored cholecystectomies will have the transition to performing RPD be essentially invisible to the institution. Proficiency-based credentialing or mandates requiring completion of a procedure- or discipline-specific training program should be encouraged and supported to continue to advance technology without compromising safety.37,38
Limitations
This study had several limitations; an inherent limitation being that it is a retrospective review with selection bias and missing data. Additionally, it is a single-center study and may be difficult to replicate at another institution. Another limitation was that the program had a high volume and increased experience over time, so it is difficult to isolate the surgeon technical component from other factors. A secondary outcome was CR-POPF used to indicate patient safety. However, a 2015 study14 by the group showed a statistically significant correlation between technical skill during the pancreaticojejunostomy and development of a CR-POPF. Subsequently, all pancreaticojejunostomies were completed by the attending surgeon, regardless of fellow technical skill. Therefore, the CR-POPF outcome likely affected the second-generation surgeons, who did perform pancreaticojejunostomies, but not the third-generation surgeons. An additional limitation was that fellow cases were determined by a self-reported case log, which may be inaccurate (eg, wrong date). While efforts were made to ensure accuracy of case logs through review of operative records and surgeon operative notes, there may be minor inaccuracies. Despite these limitations, we describe an important step in safely integrating attending surgeons and trainees into complex operations while creating a learning curve possible to attain with fewer cases.
All 3 generations are a reflection of a single institution’s overall growth and learning spectrum. All 3 generational learning curves continue to decrease, showing this ongoing improvement on all levels within the program. There is no way to completely isolate each generation to include only mentorship and a curriculum because growth and time continue to occur in tandem for all 3 generations, which disproportionately affect later generations more owing to smaller cohorts and ongoing optimization. Including fellows as generation 3 has numerous implications. All fellows had a generation 1 or 2 attending surgeon mentoring them and assisting them from a console and an institutional infrastructure beyond the pioneering curve; therefore, this does not reflect their isolated performance. A true reflection of generation 3’s experience will be when this study is repeated for these surgeons in their practice as attending surgeons.
Conclusions
Mentorship and a formal proficiency-based robotics curriculum was associated with a decreased learning curve of subsequent generation of surgeons for the RPD without compromising patient safety. While the curriculum described here is designed for robotic pancreatic resections, its structure can likely be extrapolated to a broad variety of procedures. This makes it of interest to not just those in the fields of HPB and surgical oncology but also to a wide range of specialties and credentialing bodies. As is seen by the difference between the second and third generations, the mentorship and proficiency-based curriculum are both critically important to decreasing the learning curve, and we recommend these are implemented together. The “see one, do one, teach one” model of surgical training will slowly give way to more proficiency-based programs tailored for specific procedures and technologies. As training programs evolve to accommodate these changes; formal mentorship and proficiency-based curricula should become commonplace across specialties as a validated way to decrease learning curves while maintaining quality patient outcomes.
eFigure 1. Representative Fellow Robotic Whipple Case Log
References
- 1.Morris B. Robotic surgery: applications, limitations, and impact on surgical education. MedGenMed. 2005;7(3):72. [PMC free article] [PubMed] [Google Scholar]
- 2.Shah J, Vyas A, Vyas D. The history of robotics in surgical specialties. Am J Robot Surg. 2014;1(1):12-20. doi: 10.1166/ajrs.2014.1006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Boone BA, Zenati M, Hogg ME, et al. . Assessment of quality outcomes for robotic pancreaticoduodenectomy: identification of the learning curve. JAMA Surg. 2015;150(5):416-422. doi: 10.1001/jamasurg.2015.17 [DOI] [PubMed] [Google Scholar]
- 4.Gofton WT, Papp SR, Gofton T, Beaulé PE. Understanding and taking control of surgical learning curves. Instr Course Lect. 2016;65:623-631. [PubMed] [Google Scholar]
- 5.Hopper AN, Jamison MH, Lewis WG. Learning curves in surgical practice. Postgrad Med J. 2007;83(986):777-779. doi: 10.1136/pgmj.2007.057190 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Fisher WE, Hodges SE, Wu MF, Hilsenbeck SG, Brunicardi FC. Assessment of the learning curve for pancreaticoduodenectomy. Am J Surg. 2012;203(6):684-690. doi: 10.1016/j.amjsurg.2011.05.006 [DOI] [PubMed] [Google Scholar]
- 7.Schmidt CM, Turrini O, Parikh P, et al. . Effect of hospital volume, surgeon experience, and surgeon volume on patient outcomes after pancreaticoduodenectomy: a single-institution experience. Arch Surg. 2010;145(7):634-640. doi: 10.1001/archsurg.2010.118 [DOI] [PubMed] [Google Scholar]
- 8.Tseng JF, Pisters PW, Lee JE, et al. . The learning curve in pancreatic surgery. Surgery. 2007;141(5):694-701. doi: 10.1016/j.surg.2007.04.001 [DOI] [PubMed] [Google Scholar]
- 9.Zureikat AH, Postlewait LM, Liu Y, et al. . A multi-institutional comparison of perioperative outcomes of robotic and open pancreaticoduodenectomy. Ann Surg. 2016;264(4):640-649. doi: 10.1097/SLA.0000000000001869 [DOI] [PubMed] [Google Scholar]
- 10.McMillan MT, Zureikat AH, Hogg ME, et al. . A propensity score-matched analysis of robotic vs open pancreatoduodenectomy on incidence of pancreatic fistula. JAMA Surg. 2017;152(4):327-335. doi: 10.1001/jamasurg.2016.4755 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Magge D, Zenati M, Lutfi W, et al. . Robotic pancreatoduodenectomy at an experienced institution is not associated with an increased risk of post-pancreatic hemorrhage. HPB (Oxford). 2018;20(5):448-455. doi: 10.1016/j.hpb.2017.11.005 [DOI] [PubMed] [Google Scholar]
- 12.Kowalsky SJ, Zenati MS, Steve J, et al. . A Combination of robotic approach and ERAS pathway optimizes outcomes and cost for pancreatoduodenectomy. Ann Surg. 2018. [DOI] [PubMed] [Google Scholar]
- 13.Birkmeyer JD, Finks JF, O’Reilly A, et al. ; Michigan Bariatric Surgery Collaborative . Surgical skill and complication rates after bariatric surgery. N Engl J Med. 2013;369(15):1434-1442. doi: 10.1056/NEJMsa1300625 [DOI] [PubMed] [Google Scholar]
- 14.Hogg ME, Zenati M, Novak S, et al. . Grading of surgeon technical performance predicts postoperative pancreatic fistula for pancreaticoduodenectomy independent of patient-related variables. Ann Surg. 2016;264(3):482-491. doi: 10.1097/SLA.0000000000001862 [DOI] [PubMed] [Google Scholar]
- 15.Mark Knab L, Zenati MS, Khodakov A, et al. . Evolution of a novel robotic training curriculum in a complex general surgical oncology fellowship. Ann Surg Oncol. 2018;25(12):3445-3452. doi: 10.1245/s10434-018-6686-0 [DOI] [PubMed] [Google Scholar]
- 16.Hogg ME, Tam V, Zenati M, et al. . Mastery-based virtual reality robotic simulation curriculum: the first step toward operative robotic proficiency. J Surg Educ. 2017;74(3):477-485. doi: 10.1016/j.jsurg.2016.10.015 [DOI] [PubMed] [Google Scholar]
- 17.Tam V, Zenati M, Novak S, et al. . Robotic pancreatoduodenectomy biotissue curriculum has validity and improves technical performance for surgical oncology fellows. J Surg Educ. 2017;74(6):1057-1065. doi: 10.1016/j.jsurg.2017.05.016 [DOI] [PubMed] [Google Scholar]
- 18.Rice MK, Zenati MS, Novak SM, et al. . Crowdsourced assessment of inanimate biotissue drills: a valid and cost-effective way to evaluate surgical trainees. J Surg Educ. 2019;76(3):814-823. doi: 10.1016/j.jsurg.2018.10.007 [DOI] [PubMed] [Google Scholar]
- 19.Al Abbas AI, Jung JP, Rice MK, Zureikat AH, Zeh HJ III, Hogg ME. Methodology for developing an educational and research video library in minimally invasive surgery. J Surg Educ. 2019;76(3):745-755. doi: 10.1016/j.jsurg.2018.10.011 [DOI] [PubMed] [Google Scholar]
- 20.Chen S, Chen JZ, Zhan Q, et al. . Robot-assisted laparoscopic versus open pancreaticoduodenectomy: a prospective, matched, mid-term follow-up study. Surg Endosc. 2015;29(12):3698-3711. doi: 10.1007/s00464-015-4140-y [DOI] [PubMed] [Google Scholar]
- 21.Kim SC, Song KB, Jung YS, et al. . Short-term clinical outcomes for 100 consecutive cases of laparoscopic pylorus-preserving pancreatoduodenectomy: improvement with surgical experience. Surg Endosc. 2013;27(1):95-103. doi: 10.1007/s00464-012-2427-9 [DOI] [PubMed] [Google Scholar]
- 22.Speicher PJ, Nussbaum DP, White RR, et al. . Defining the learning curve for team-based laparoscopic pancreaticoduodenectomy. Ann Surg Oncol. 2014;21(12):4014-4019. doi: 10.1245/s10434-014-3839-7 [DOI] [PubMed] [Google Scholar]
- 23.Dokmak S, Ftériche FS, Aussilhou B, et al. . The Largest European single-center experience: 300 laparoscopic pancreatic resections. J Am Coll Surg. 2017;225(2):226-234.e2. [DOI] [PubMed] [Google Scholar]
- 24.Shyr BU, Chen SC, Shyr YM, Wang SE. Learning curves for robotic pancreatic surgery-from distal pancreatectomy to pancreaticoduodenectomy. Medicine (Baltimore). 2018;97(45):e13000. doi: 10.1097/MD.0000000000013000 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Takahashi C, Shridhar R, Huston J, Meredith K. Outcomes associated with robotic approach to pancreatic resections. J Gastrointest Oncol. 2018;9(5):936-941. doi: 10.21037/jgo.2018.08.04 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Bassi C, Dervenis C, Butturini G, et al. ; International Study Group on Pancreatic Fistula Definition . Postoperative pancreatic fistula: an international study group (ISGPF) definition. Surgery. 2005;138(1):8-13. doi: 10.1016/j.surg.2005.05.001 [DOI] [PubMed] [Google Scholar]
- 27.Clavien PA, Barkun J, de Oliveira ML, et al. . The Clavien-Dindo classification of surgical complications: five-year experience. Ann Surg. 2009;250(2):187-196. doi: 10.1097/SLA.0b013e3181b13ca2 [DOI] [PubMed] [Google Scholar]
- 28.Abu Hilal M, Jain G, Kasasbeh F, Zuccaro M, Elberm H. Laparoscopic distal pancreatectomy: critical analysis of preliminary experience from a tertiary referral centre. Surg Endosc. 2009;23(12):2743-2747. doi: 10.1007/s00464-009-0499-y [DOI] [PubMed] [Google Scholar]
- 29.de Rooij T, van Hilst J, Boerma D, et al. ; Dutch Pancreatic Cancer Group . Impact of a Nationwide Training Program in Minimally Invasive Distal Pancreatectomy (LAELAPS). Ann Surg. 2016;264(5):754-762. doi: 10.1097/SLA.0000000000001888 [DOI] [PubMed] [Google Scholar]
- 30.de Rooij T, van Hilst J, Topal B, et al. ; Dutch Pancreatic Cancer Group . Outcomes of a Multicenter Training Program in Laparoscopic Pancreatoduodenectomy (LAELAPS-2). Ann Surg. 2019;269(2):344-350. doi: 10.1097/SLA.0000000000002563 [DOI] [PubMed] [Google Scholar]
- 31.Torphy RJ, Friedman C, Halpern A, et al. . Comparing short-term and oncologic outcomes of minimally invasive versus open pancreaticoduodenectomy across low and high volume centers. Ann Surg. 2018. [DOI] [PubMed] [Google Scholar]
- 32.Adam MA, Thomas S, Youngwirth L, Pappas T, Roman SA, Sosa JA. Defining a hospital volume threshold for minimally invasive pancreaticoduodenectomy in the United States. JAMA Surg. 2017;152(4):336-342. doi: 10.1001/jamasurg.2016.4753 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Hogg ME, Besselink MG, Clavien PA, et al. ; Minimally Invasive Pancreatic Resection Organizing Committee . Training in minimally invasive pancreatic resections: a paradigm shift away from “see one, do one, teach one”. HPB (Oxford). 2017;19(3):234-245. doi: 10.1016/j.hpb.2017.01.016 [DOI] [PubMed] [Google Scholar]
- 34.Giulianotti PC, Mangano A, Bustos RE, et al. . Operative technique in robotic pancreaticoduodenectomy (RPD) at University of Illinois at Chicago (UIC): 17 steps standardized technique : lessons learned since the first worldwide RPD performed in the year 2001. Surg Endosc. 2018;32(10):4329-4336. doi: 10.1007/s00464-018-6228-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.McCulloch P, Altman DG, Campbell WB, et al. ; Balliol Collaboration . No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009;374(9695):1105-1112. doi: 10.1016/S0140-6736(09)61116-8 [DOI] [PubMed] [Google Scholar]
- 36.Kishiki T, Lapin B, Wang C, et al. . Teaching peroral endoscopic myotomy (POEM) to surgeons in practice: an “into the fire” pre/post-test curriculum. Surg Endosc. 2018;32(3):1414-1421. doi: 10.1007/s00464-017-5823-3 [DOI] [PubMed] [Google Scholar]
- 37.Tam V, Borrebach J, Dunn SA, Bellon J, Zeh HJ, Hogg ME. Proficiency-based training and credentialing can improve patient outcomes and decrease cost to a hospital system. Am J Surg. 2019;217(4):591-596. doi: 10.1016/j.amjsurg.2018.07.053 [DOI] [PubMed] [Google Scholar]
- 38.Tam V, Zeh HJ III, Hogg ME. incorporating metrics of surgical proficiency into credentialing and privileging pathways. JAMA Surg. 2017;152(5):494-495. doi: 10.1001/jamasurg.2017.0025 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eFigure 1. Representative Fellow Robotic Whipple Case Log