Skip to main content
PMC home page
Annals of Surgery Open logoLink to Annals of Surgery Open
. 2022 Jan 27;3(1):e111. doi: 10.1097/AS9.0000000000000111

Learning Curves in Open, Laparoscopic, and Robotic Pancreatic Surgery

A Systematic Review and Proposal of a Standardization

P C Müller *, C Kuemmerli †, A Cizmic ‡, S Sinz §, P Probst ‡, M de Santibanes ∥, S V Shrikhande , C Tschuor #, M Loos ‡, A Mehrabi ‡, K Z’graggen *, B P Müller-Stich ‡, T Hackert ‡, M W Büchler ‡, F Nickel ‡,✉
PMCID: PMC10431463  PMID: 37600094

Abstract

Objective:

To depict and analyze learning curves for open, laparoscopic, and robotic pancreatoduodenectomy (PD) and distal pancreatectomy (DP).

Background:

Formal training is recommended for safe introduction of pancreatic surgery but definitions of learning curves vary and have not been standardized.

Methods:

A systematic search on PubMed, Web of Science, and CENTRAL databases identified studies on learning curves in pancreatic surgery. Primary outcome was the number needed to reach the learning curve as defined by the included studies. Secondary outcomes included endpoints defining learning curves, methods of analysis (statistical/arbitrary), and classification of learning phases.

Results:

Out of 1115 articles, 66 studies with 14,206 patients were included. Thirty-five studies (53%) based the learning curve analysis on statistical calculations. Most often used parameters to define learning curves were operative time (n = 51), blood loss (n = 17), and complications (n = 10). The number of procedures to surpass a first phase of learning curve was 30 (20–50) for open PD, 39 (11–60) for laparoscopic PD, 25 (8–100) for robotic PD (P = 0.521), 16 (3–17) for laparoscopic DP, and 15 (5–37) for robotic DP (P = 0.914). In a three-phase model, intraoperative parameters improved earlier (first to second phase: operating time –15%, blood loss –29%) whereas postoperative parameters improved later (second to third phase: complications –46%, postoperative pancreatic fistula –48%). Studies with higher sample sizes showed higher numbers of procedures needed to overcome the learning curve (rho = 0.64, P < 0.001).

Conclusions:

This study summarizes learning curves for open-, laparoscopic-, and robotic pancreatic surgery with different definitions, analysis methods, and confounding factors. A standardized reporting of learning curves and definition of phases (competency, proficiency, mastery) is desirable and proposed.

Keywords: pancreatic surgery, learning curve, minimally invasive pancreatic surgery, robotic surgery, pancreaticoduodenectomy, pancreatoduodenectomy, distal pancreatectomy

graphic file with name as9-3-e111-g001.jpg

Open in a new tab

Mini Abstract: Learning curves in pancreatic surgery have not been systematically evaluated and standardized. This systematic review summarizes learning curves for open-, laparoscopic-, and robotic pancreatic surgery with different definitions, analysis methods, and confounding factors. A standardized reporting of learning curves and definition of phases (competency, proficiency, mastery) is proposed.

Supplemental Digital Content is available in the text.

INTRODUCTION

In pancreatic surgery, it has been shown that centralization with higher case volumes for both individual surgeons as well as centers leads to better patient outcomes.1–3 Formal training is recommended for the safe introduction of pancreatic surgery and particularly minimally invasive or robotic pancreatic surgery.4,5 However, definitions of learning curves vary considerably and have not been systematically evaluated and standardized with regard to evaluated outcomes, number of procedures necessary to reach proficiency, influence of training as well as surgeons previous experience, complexity of procedures, and number of phases of the learning curve. Depending on the definition, learning curves of seven to 250 cases for pancreatoduodenectomy (PD) and 10 to 40 cases for distal pancreatectomy (DP) have been described so far.5–12 Looking at the available surgical approaches (open, laparoscopic, and robotic), these numbers vary even further. The learning curve of a surgical procedure represents an important and underrepresented bias in clinical studies and is often neglected even in randomized controlled trials (RCT).13–15 There is no established definition of what should constitute a rigorous learning curve analysis in pancreatic surgery and how the learning curve can be standardized and defined to rule out learning curve associated bias in pancreatic surgery trials. Furthermore, a definition of various phases of the learning curve observed in pancreatic surgery is currently missing. The aim of this systemic review was to depict and analyze learning curves for PD and DP for the available surgical approaches (open, laparoscopic, robotic). Furthermore, we aimed to identify target parameters representing learning curves in pancreatic surgery and potentially define different phases of learning (competency, proficiency, mastery) according to operative complexity to standardize reporting of learning curves in pancreatic surgery.16

METHODS

Systematic Literature Search Methodology

This review complies with the recommendations of the Cochrane Handbook for Systematic Reviews and Interventions and specific recommendations for surgical systematic reviews17 and is reported in line with the PRISMA guidelines.18 A protocol was developed a priori and published on researchregistry.com on April 6, 2020 (Unique identifying number: reviewregistry865). The systematic literature search was performed using MEDLINE (via PubMed), Web of Science, and CENTRAL databases.19 The search terms were connected with Boolean operators and used in combination with medical subject headings. The systematic literature search included contributions listed in the above-mentioned databases until February 5, 2021. No language restrictions were applied. Cross-referencing and manual search of the bibliographies of eligible publications was actively performed until February 2021 to identify further relevant studies for the review.

The following search strategy for Medline (via PubMed) was used:

(pancreatectomy OR pancreatic surgery OR pancreatoduodenectomy OR pancreaticoduodenectomy OR pancreatic head resection OR distal pancreatectomy OR pancreatic left resection) AND (open OR laparoscopy OR minimally invasive OR robotic surgery OR robotic-assisted surgery OR da Vinci) AND (learning curve OR training OR Proficiency OR Mastery OR Competency OR Learning phase)

Study Selection and Data Extraction

Selection of relevant articles was performed in two stages. Two of the authors independently screened the title and abstracts of all retrieved references. Duplicates were deleted before further review. Studies considered irrelevant were discarded. Full text articles for each of the selected abstracts were analyzed. In cases where clarification was needed, a consensus was reached through either discussion or a third reviewer. For data extraction, a dedicated predefined spreadsheet was used. The selection process is illustrated in the PRISMA flow diagram (Figure 1).18

FIGURE 1.

FIGURE 1.

Open in a new tab

PRISMA flow diagram for the systematic review.

Inclusion and Exclusion

Eligible for inclusion were RCTs, clinical controlled trials (including more than 10 patients), case series (including more than 10 patients) with a specific number or range of procedures characterizing any learning curve on PD (open, laparoscopic, robotic) or DP (open, laparoscopic, robotic) performed for benign or malignant pancreatic pathologies. All parameters used and described in each study for learning curve determination were included. The definition of the learning curve was adopted according the individual study.

Exclusion criteria were (1) studies based on other pancreatic surgeries than PD or DP, (2) articles not providing a specific case number or range at which the learning curve was attained, (3) articles which compared preexisting data (systematic reviews and meta-analyses), (4) articles reporting emergency procedures. Abstracts and further material not associated with a full-text manuscript such as congress abstracts were only included in the systematic review when sufficient data concerning the characteristics of the learning curve were available but were used carefully in further discussion. Studies reporting on experimental or cadaveric models were excluded.

Outcome Parameters

Primary outcome was the number of procedures needed to surmount the “general learning curve” as the definition of the individual study was adopted for this analysis. Secondary outcomes were the endpoints that were used for the definition of the learning curve, the methods of learning curve analysis (statistical calculation/arbitrary), the classification of different learning phases and the selection of patients. Further outcomes were extracted and evaluated according to the parameters for the learning curve according to availability: Operative time, blood loss, conversion rate, overall complications, minor and major complications, 30- and 90-day mortality, postoperative pancreatic fistula (POPF) according to international study group of pancreatic surgery (ISGPS),20,21 postpancreatectomy hemorrhage (PPH) according to ISGPS,22 delayed gastric emptying, and chyle leak23 according to ISGPS,24 bile leakage according to international study group of liver surgery,25 reoperation rate, readmission rate, lymph node harvest, oncologic parameters, length of stay (LOS). If specified, surgeon specific parameters were also evaluated including surgical experience, yearly case volume as well as index procedures, specific fellowships, participation in general and specific mentoring activities and proctoring.

Quality Assessment

There were no RCT included in this analysis. Therefore, ROBINS-I tool was used for quality assessment of all nonrandomized studies in the systematic review.26 Seven domains—(1) reporting, (2) selection of participants, (3) statistical tests, (4) data dredging, (5) deviations of intended interventions, (6) measurement of outcomes, (7) selection bias—were rated with either low, moderate, or high risk of bias. The quality of evidence was used for grading the quality of evidence.27

Statistical Analysis

When studies reported more than one number of cases to overcome the learning curve, the lowest number mentioned was used. Due to the heterogeneity of endpoints used the assess the learning curve in the original studies, we assessed if learning curves were based on statistical calculation (eg, cumulative sum [CUSUM] and risk-adjusted CUSUM [RA-CUSUM]) or arbitrarily chosen as an attempt to make the groups more homogeneous. These two groups were then analyzed separately. The endpoints were furthermore grouped into the following domains: intraoperative outcomes including operative time, blood loss and conversion, and postoperative outcomes such as complications, oncological outcomes, and LOS. Descriptive statistics were used to depict the data. Correlation was assessed using the Spearman-rank coefficient. The association was considered weak (coefficient weak 0.1 to <0.3), moderate (0.3 to <0.5), or strong (≥0.5). The Chi-square and Mann-Whitney U tests were used for categorical and continuous data, respectively. R version 4.0.2, R Core Team, was used for all analyses.28

RESULTS

Study Selection

The search yielded 1115 studies. After removing 46 duplicates 1069 studies were screened and 66 studies included. The PRISMA diagram is presented as Figure 1.

Study Characteristics

The 66 studies included 14,206 patients in total. Forty-seven studies reported on PD, thereof 12 on open PD, 19 on laparoscopic PD, and 17 on robotic PD. Furthermore, 21 studies reported on DP, 15 on laparoscopic DP, and 6 on robotic DP, none on open DP. Of note, two studies presented data from both PD and DP,7,29 one study from both laparoscopic and open PD,30 and one study analyzed laparoscopic PD with and without vascular resection as two separate patient cohorts.31 There were no RCTs on the subject, all studies had an observational design. The number of participating surgeons was specified in 54 studies (82%), while participating surgeons ranged from 1 to 43. Studies reported on the learning curve from individual surgeons (n = 30; 45%) or institutional learning curves (n = 36; 55%). The baseline study and learning curve characteristics are shown in Tables 1 and 2.

TABLE 1.

Baseline Characteristics of the Included Study

Reference Year Country Patients(n) Surgeons(n) Analysis of Learning Curve Learning Curve Phases(n) Length of Learning Curve(n° Procedures) Factors
Open pancreatoduodenectomy
Cameron et al. 2006 USA 1000 1 Arbitrary 5 2 Operative time*
Blood loss
LOS*
Coe et al. 2015 USA 1210 Multiple Arbitrary 4 10 Mortality
Ecker et al. 2018 USA 303 1 Arbitrary 4 50 POPF*
Fisher et al. 2012 USA 162 1 Arbitrary 2 19 Operative time
Blood loss
LOS*
Complications*
Hardacre et al. 2010 USA 60 1 Arbitrary 2 30 Operative time*
LOS*
Adjuvant therapy*
Noda et al. 2012 Japan 100 1 Arbitrary 2 50 POPF
Park et al. 2020 Korea 300 2 Arbitrary 3 50 Operative time*
Blood loss
Relles et al. 2013 USA 686 47 Arbitrary 3 >16 Mortality
Roberts et al. 2020 UK 519 8 Statistical/CUSUM 2 50 POPF*
Schmidt et al. 2010 USA 1003 19 Statistical/other 2 20 Operative time*
Blood loss*
Complications*
Tsamalaidze et al. 2018 USA 93 1 Statistical/CUSUM 4 30 Operative time*
Tseng et al. 2007 USA 650 3 Arbitrary 2 60 Operative time *
Blood loss*
LOS*
Laparoscopic pancreatoduodenectomy
Choi et al. 2020 Korea 171 1 Statistical/CUSUM 3 40 Operative time*
Conversion
POPF
Mortality
Dokmak et al. Arbitrary 2 10 Operative time*
Huang et al. 2020 China 98 1 Statistical/CUSUM 3 34 Operative time*
LOS*
Ke et al. Arbitrary 4 19 Operative time*
DGE*
Kim et al. 2013 Korea 100 1 Arbitrary 3 33 Operative time*
Complications
Kim et al. 2020 Korea 119 1 Statistical/CUSUM 2 47 Operative time
Kuroki et al. 2014 Japan 30 1 Arbitrary 3 10 Operative time*
Blood loss*
Liao et al. 2017 Taiwan 12 Arbitrary 2 5 Operative time *
Blood loss
Lu et al. 2016 China 120 1 Arbitrary 4 30 Operative time
Blood loss*
Morato et al. 2020 Spain 50 1 Statistical/CUSUM 4 21 Operative time*
Conversion*
Complications
Nagakawa et al. 2018 Japan 150 3 Statistical/CUSUM 2 20 Blood loss*
Operative time*
Nieuwenhuijs et al. 2020 Netherlands 20 3 Arbitrary 2 10 Anastomotic complications*
Song et al. 2020 Korea 500 - Statistical/CUSUM 4 55 Operative time*
Speicher et al. 2014 USA 56 5 Arbitrary 6 10 Operative time*
Blood loss*
Tsamalaidze et al. 2018 USA 31 1 Statistical/CUSUM 4 20 Operative time*
Wang et al. 2019 China 1029 - Statistical/CUSUM 4 40 Operative time*
Wang et al. 2016 China 57 1 Statistical/CUSUM 3 11 Operative time*
Wang et al. 2020 China 550 Statistical/CUSUM 3 47 Operative time*
Zhang et al. 2018 China 20 - Arbitrary 2 10 Operative time
Blood loss
LOS
Robotic pancreatoduodenectomy
Beane et al. 2019 USA 380 3 Statistical/CUSUM 2 35 Operative time*
Boone et al. 2015 USA 120 - Statistical/CUSUM 5 20 Operative time*
Chen et al. 2015 China 60 2 Statistical 2 40 Operative time *
Blood loss*
Guerra et al. 2019 Italy 59 1 Arbitrary 2 20 Conversion
Kim et al. Statistical/CUSUM 2 29 Operative time
Marino et al. 2020 Spain 60 1 Statistical/CUSUM 2 25 Operative time*
Blood loss*
Napoli et al. 2016 Italy 70 1 Statistical/CUSUM 2 33 Operative time*
Rice et al. 2020 USA 514 28 Arbitrary 3 80 Operative time*
Complications*
Schmidt et al. 2020 USA 40 2 Statistical/other - 40 Operative time*
Shi et al. 2019 China 450 3 Statistical/CUSUM 3 100 Operative time*
Blood loss*
Shyr et al. 2018 Taiwan 61 2 Statistical/CUSUM 2 20 Operative time *
Takahashi et al. 2018 USA 65 1 Statistical 2 10 Operative time
Complications
Watkins et al. 2018 USA, Italy 92 - Statistical/CUSUM 2 20 Operative time
Zhang et al. 2018 China 20 - Arbitrary 2 10 Operative time*
Zhang et al. 2019 China 100 1 Statistical/CUSUM 2 40 Operative time*
Zhou et al. 2020 China 41 1 Statistical/CUSUM 2 8 Operative time
Zwart et al. 2021 Netherlands 275 15 Statistical/CUSUM 2 22 Operative time*
Open in a new tab

*Statistically significant.

CUSUM, cumulative sum; DGE, delayed gastric emptying; LOS, length of stay; POPF, postoperative pancreatic fistula.

TABLE 2.

Baseline Characteristics of the Included Study

Reference Year Country Patients(n) Surgeons(n) Analysis of Learning Curve Learning Curve Phases(n) Length of Learning Curve(n° Procedures) Factors
Laparoscopic distal pancreatectomy
Barga et al. 2012 Italy 30 – Arbitrary 3 10 Operative time*
Blood loss*
Barrie et al. 2015 UK 25 1 Statistical/CUSUM 2 3 Operative time*
Blood loss*
de Rooij et al. 2017 UK 111 1 Arbitrary 3 30 POPF*
Complications*
LOS*
de Rooij et al. 2016 Netherlands 201 32 Arbitrary 2 Before/after training Operative time
Blood loss
LOS
Dokmak et al. 2017 France 165 3 Arbitrary 2 40 Operative time
Hasselgren et al. 2016 Sweden 37 2 Arbitrary 2 18 Operative time*
Complications*
Kim et al. 2019 Korea 65 - Statistical/CUSUM 2 16 Complications
Kneuertz et al. 2012 USA 132 - Arbitrary 2 66 Operative time*
Liao et al. 2020 Taiwan 64 1 Statistical/CUSUM 2 16 Operative time*
Lof et al. 2019 UK 570 12 Arbitrary 4 15 Complications*
ICU admission*
LOS*
Malleo et al. 2014 Italy 100 - Arbitrary 3 33 Operative time*
Nachmany et al. 2016 Israel 39 5 Arbitrary 4 17 Operative time
Park et al. 2019 Korea 26 1 Statistical/other 2 12 Operative time*
Ricci et al. 2015 Italy 32 1 Statistical/other 2 17 Operative time*
Sahakyan et al. 2021 Norway 640 4 Arbitrary 5 80 Operative time*
Robotic distal pancreatectomy
Benizri et al. 2013 France 11 5 Statistical/CUSUM 2 7 Operative time*
Conversion
Complications
Reoperation
Klompmaker et al. 2019 USA 80 3 Statistical/CUSUM 2 31 Operative time
Napoli et al. 2015 Italy 55 1 Statistical/CUSUM 2 10 Operative time
Shakir et al. 2015 USA 100 3 2 20 Operative time
Shyr et al. 2018 Taiwan 70 2 Statistical/CUSUM 2 37 Operative time*
Takahashi et al. 2018 USA 43 1 Statistical/other 2 5 Operative time*
Open in a new tab

CUSUM: Cumulative sum; DGE: Delayed gastric emptying; LOS: Length of stay; POPF: Postoperative pancreatic fistula;

* = statistically significant.

Main Outcomes

Definition of the Learning Curve

Overall, the definition of the learning curves in pancreatic surgeries was heterogenous among the included studies. Only in 35 studies (53%) the learning curve analysis was based on a statistical calculation (eg, CUSUM), the rest of the studies used an arbitrary split-group approach. Looking at the different phases of the learning curves, the majority of studies identified two phases of a learning curve (n = 35; 53%), while three (n = 15; 23%), and up to six (n = 1; 2%) phases were also described. To define the learning curve, a single metric was used in 36 studies (55%), while two metrics were used in 12 studies (18%) and three or more in 18 studies (27%). The most often used parameters to define the learning curve were operative time (n = 51; 77%), blood loss (n = 17; 26%), complications (n = 10; 15%), and LOS (n = 9; 14%). Interestingly, only 9 of 42 studies (21%) reporting on oncologic outcomes mentioned differences in oncologic parameters such as lymph node harvest, R0-rate or rate of patients undergoing adjuvant therapies between different learning phases with respective patient groups.10,35–42 None of the included studies specifically evaluated the learning curve for oncologic parameters. Looking at potential biases that are important for the interpretation of the learning curve, previous surgical experience of the participating surgeons, either as performed pancreatic cases or previous training (years of experience or specific training), was mentioned in 30 (46%) studies. Proctoring and mentoring were reported in 4 (6%) and 8 (12%) studies, respectively. Furthermore, patient demographics (n = 62; 94%) as well as disease specific information (n = 58; 88%) defined as information on neoadjuvant therapy, vascular resections, pathological data, or tumor diameter, were reported in most studies. However, preoperative resectability status was reported in a minority of studies (n = 11; 17%).

Phases of the Learning Curve

The evolution of intra- and postoperative outcome parameters was evaluated over the different phases of the learning curve. When including all studies irrespective of their analysis method, a decrease of intraoperative parameters was mainly observed from phase 1 to phase 2, as demonstrated by a reduction of operative time (phase 1/2: –15% and phase 2/3: –6%) and blood loss (phase 1/2: –29% and phase 2/3: –13%). Postoperative parameters such as complications (phase 1/2: –20% and phase 2/3: –46%) and POPF (phase 1/2: –4% and phase 2/3: –48%) decreased at a later stage and more pronounced from phase 2 to phase 3 (Figure 2).

FIGURE 2.

FIGURE 2.

Open in a new tab

Evolution of the intra- and postoperative outcome parameters over the different phases of the learning curve. Operative time (A) and blood loss (B), decreased at an earlier stage while postoperative parameters such as complications (C) and POPF (D) decreased at a later stage. POPF, postoperative pancreatic fistula.

Arbitrary Split Group Approach Versus Statistical Calculation of the Learning Curve

The number of procedures to overcome the general learning curve was different according to the used analysis for both PD (statistical calculation 30 [IQR 20–40] vs arbitrary split group approach 19 [IQR 10–41.5]; P < 0.001) and DP (statistical calculation 16 (IQR 8.5–18.5) vs arbitrary split group approach 18 (IQR 16–35); P = 0.001). A statistical calculation was more frequently used in recent years and especially in studies assessing the learning curve for the robotic approach. In the last 5 years (2017–2021), learning curve analysis comprised 28 (67%) studies with a statistical calculation and 14 (33%) studies with an arbitrary split group approach. Before 2017, 10 (37%) studies used statistical calculation and 17 (63%) the arbitrary split group approach (P = 0.016). Looking at studies evaluating the operative time learning curve, there was a correlation between study sample size and number of procedures to overcome the learning curve for both DP (rho = 0.86, P < 0.001) and PD (rho = 0.44, P = 0.004) (Figure 3).

FIGURE 3.

FIGURE 3.

Open in a new tab

Relation of the study population size and the minimal number of procedures to overcome the learning curve of the most frequently used endpoint (operative time) for pancreatoduodenectomy and distal pancreatectomy.

Individual Surgeon Versus Institutional Analysis of the Learning Curve

The number of procedures to overcome the general learning curve for the individual surgeon analysis was 30 (2–50) for open PD, 26 (10–60) for laparoscopic PD, and 29 (10–40) for robotic PD as compared to 20 (10–60) for open PD, 20 (5–55) for laparoscopic PD, and 21 (8–100) for robotic PD when an institutional analysis was used. Looking at the general learning curve for the individual surgeon analysis for DP, it was 17 (3–30) for the laparoscopic approach and 8 (5–10) for the robotic approach as compared to 17 (10–80) for the laparoscopic approach and 26 (7–37) for the robotic approach with an institutional analysis, respectively.

Learning Curve for Pancreatoduodenectomy

Evaluating learning curves based on statistical calculation, the number of procedures to overcome the learning curve for PD varied from 30 in open PD (range 20–50), to 39 in laparoscopic PD (range 11–60), and 25 in robotic PD (range 8–100) (P = 0.521). Based on split group analyses, the number of procedures to overcome the learning curve for PD ranged from 30 in open PD (range 2–60), to 10 in laparoscopic PD (range 5–33), and 50 in robotic PD (range 20–80) (P = 0.058). The different number of procedures to overcome the respective learning curve defined by intraoperative (OR time, blood loss, and conversion rate) or postoperative parameters (LOS, POPF, complications) is shown in Figure 4A. The trend of operative time from start until the late phases/procedures is depicted in Figure 4B.

FIGURE 4.

FIGURE 4.

Open in a new tab

A: Learning curve of pancreatoduodenectomies per endpoint, either intraoperative (operative time, blood loss, and conversion) or postoperative (overall complications, length of stay, and POPF). The size of the bubble depicts the study sample size. B: Trend of operative time from start until the late phases/procedures. PD indicates pancreatoduodenectomy; POPF, postoperative pancreatic fistula.

Learning Curve for Distal Pancreatectomy

Looking only at learning curves based on statistical calculation, the number of procedures to overcome the learning curve for DP was 16 in laparoscopic DP (range 3–17) and 15 in robotic DP (range 5–37) (P = 0.914). Based on split group analyses, the number of procedures to overcome the learning curve for laparoscopic DP was 18 (10–80).

The different number of procedures to overcome the learning curve divided into intraoperative or postoperative parameters is shown in Figure 5A. The trend of operative time from start until the late phases/procedures is depicted in Figure 5B.

FIGURE 5.

FIGURE 5.

Open in a new tab

A: Learning curve of distal pancreatectomies per endpoint, either intraoperative (operative time, blood loss, and conversion) or postoperative (overall complications, length of stay, and POPF). The size of the bubble depicts the study sample size. B: Trend of operative time from start until the late phases/procedures. PD, pancreatoduodenectomy; POPF, postoperative pancreatic fistula.

Risk of Bias Within Studies

Of the included studies, 23 had a low, 36 had a moderate, and 7 a high overall risk of bias. Moderate or high overall risk of bias were mainly due to bias in the selection of participants (n = 16) and unclear reporting of items relevant for the assessment and analysis of the study (n = 19) (Supplemental Figure 1, see http://links.lww.com/AOSO/A96). Due to the retrospective study design, the moderate to high risk of bias in most studies and the inconsistency in learning curve outcomes the overall certainty of evidence was grading as low.

DISCUSSION

The present systematic review found great heterogeneity in definitions and assessment of learning curves for open, laparoscopic, and robotic pancreatic surgery. In the included studies that based their learning curve analysis on a statistical calculation, the number of procedures to surpass a first phase of the learning curve was 30 for open PD, 39 for laparoscopic PD, and 25 for robotic PD. For laparoscopic DP, the number of procedures to surpass the learning curve was 16 and for robotic DP it was 15, while there was no study assessing the learning curve of open DP. There was no clear difference between the open and minimally invasive surgical approaches for PD and DP, respectively. However, the studies did not consider the previously negotiated learning curves from other approaches. Furthermore, the definitions and methods to assess learning curves were heterogenous.

Of the included studies, 53% based their learning curve analysis on a statistical calculation while the rest chose arbitrary group sizes. Technical indicators such as operative time (77%) or blood loss (26%) were most widely used to analyze the learning curve and only a minority of studies analyzed postoperative outcome parameters such as complications (15%) or LOS (14%). While oncologic outcome parameters were not used to evaluate learning curves, they were assessed in 42 of the included studies and 9 studies (21%) found differences in oncologic outcome parameters between different learning phases.

Interestingly, a significant difference was found in number of procedures to surpass the learning curve between statistically calculated and arbitrary defined learning curves. While for PD the learning curve was longer when statistically analyzed, in DP the opposite was observed. Furthermore, a significant correlation was found between study sample size and cases needed to surpass the learning curve. Thus, the learning curve was reached after more procedures in larger studies, potentially presenting an important bias to consider in future studies that will require adequate sample sizes to provide a more realistic learning curve assessment. For a critical interpretation of learning curve analyses, it seems of paramount importance to take the study sample size into account. Larger sample sizes and consideration of case complexity will likely lead to a more differentiated analysis of the learning curve phases. We suggest a standardization of learning curve assessment for PD and DP that will allow for better comparability between centers and for consideration of learning curves as a source of bias in (randomized) trials.

To allow an unbiased comparison of learning curves, studies on this topic should ideally report the participating surgeons baseline characteristics including previous surgical and procedural experience,9,43 as well as simulation,44 procedure specific training,12,45 and clinical fellowships.46 In the present review, these important surgeon characteristics were reported in only 46% of included studies, representing an important confounding factor that limits generalization of the reported outcomes. Interestingly, in a systematic review on learning curves in robotic surgery in general, Kassite et al likewise found that previous surgical experience and training were underreported and only described in 20% of the studies.47 Beside surgeon related factors, the institutional learning curve should not be underestimated when looking at potential biases.48,49 The institutional competency depends on the frequency of a specific surgery, team familiarity,50 and training of surgical assistants, operating room nurses, and technical staff.51,52 The relationship between hospital and surgeon volume, team familiarity, and decreased operative times up to improved postoperative outcomes has been shown repeatedly.39,53 To minimize potential bias, we therefore suggest, that studies reporting on learning curves of pancreatic surgery should include information on institutional factors, with at least data on frequency of the procedure and team familiarity.

Patient related aspects are a further important confounding factor determining the procedural complexity.54,55 Body composition (eg, BMI56), comorbidities and disease characteristics57–59 all influence the type and difficulty of pancreatic surgery and therefore the learning curve. BMI and comorbidities were reported in most studies (88%), but the preoperative resectability status was mentioned only in a minority of studies (17%). In the initial learning period, surgeons usually select low-risk patients with favorable anatomical (low BMI) and disease specific features (cystic lesions, small tumors, no vessel involvement).47 After accumulating experience, technically more challenging and more complex cases such as advanced tumors with vascular involvement59–61 or initially irresectable tumors after neoadjuvant treatment32,62 are selected and affect the learning curve in a later stage.58,59 Miskovic et al suggested a case selection algorithm according to disease complexity in colorectal surgery, taking into account tumor size and inflammatory disease that increased the difficulty of the surgery.55 To report the complexity of the different pancreatic resections, a system stratifying pancreatic head resections by technical difficulty has been proposed by Mihaljevic et al.59 Similarly, a difficulty scoring system for laparoscopic DP was introduced by Ohtsuka et al taking into account type of operation, resection line, proximity of tumor to major vessels, tumor extension to peripancreatic tissue, and left-sided portal hypertension.34 Reporting PDs and DPs by these systems would enhance the comparability of learning curves between studies and is therefore recommended (Table 3). However, little attention has been paid on the influence of different comorbidities and disease specific factors on the learning curve in pancreatic surgery.

TABLE 3.

Minimal information suggested for a standardized reporting of learning curves in pancreatic surgery

Surgeon specific characteristics
 Previous surgical experience in the field of pancreatic surgery
 Number of pancreatic procedures (total and with the used access/technique)
 Years of surgical experience
 Specific HPB training and procedure specific training (dry-/wet lab).
 Proctoring/Mentoring
Institutional characteristics
 Annual pancreatic case volume
 Team familiarity
Patient characteristics
 Age
 BMI
 Previous extensive upper abdominal surgery
 ASA
Disease characteristics
 Resectability status
 Size of the lesion
 Neoadjuvant treatment
 Pathology
 Complicating factors such as acute pancreatitis or cavernous transformation
Complexity of pancreatoduodenectomy 32
 Type 1 (Standard PD)
 Type 2 (PD + PV)
 Type 3 (PD + additional organ)
 Type 4 (PD + artery)
Complexity of pancreatic anastomoses 33
 Type A (Non-soft texture and MPD > 3 mm)
 Type B (Non-soft texture and MPD < 3 mm)
 Type C (Soft texture and MPD > 3 mm)
 Type D (Soft texture and MPD < 3 mm)
Complexity of distal pancreatectomy 34
 Type of operation
 DP for benign disease
 Spleen preserving DP
 Radical antegrade modular pancreatosplenectomy
 Pancreatic resection line
 Portal vein
 Pancreatic tail
 Tumor close to major vessels
 Tumor extension to peripancreatic tissue
 Left-sided portal hypertension and splenomegaly
Open in a new tab

ASA, American society of anesthesiologists; BMI, body mass index; MPD, main pancreatic duct; PD, pancreatoduodenectomy; PV, portal vein.

In accordance with the general recommendations by Kassite et al,47 we propose that standardized learning curve reporting in pancreatic surgery should include the baseline information detailed in Table 3.

For a rigorous assessment, the analysis of the learning curve should then be based on a statistical calculation. CUSUM analysis is based on sequential monitoring of cumulative performance over a time period. It shows the deviation from their respective group mean, visualizing trends in a continuous process.63 CUSUM currently represents the gold standard, while a risk-adjusted (RA-)CUSUM analysis gives a more in depth look at learning curves, however this type of analysis was seldomly used for pancreatic procedures.64,65 RA-CUSUM was proposed by Steiner et al and uses a likelihood-based scoring model taking into account the individual preoperative risk, therefore accounting for the heterogeneity among patients.66 Roberts et al. preoperatively calculated the risk of POPF and adjusted the individual learning curve according to the calculated risk.64 When evaluating the learning curve of POPF, this RA-CUSUM analysis represents a more individual approach and can be based on the various fistula risk scores or on the latest ISGPS classification (Table 3).54,67–70

The existence of a learning curve can have a major impact in surgical RCTs, particularly when the trial is evaluating new techniques.13 Recently, the LEOPARD-2 trial compared laparoscopic to open PD and had to be prematurely terminated due to a mortality of 10% in the laparoscopic group versus 2% in the open group. The study included surgeons after having performed at least 20 laparoscopic PD while the institutional volume was set at a minimum of 20 PD per year. Remarkably, the authors evaluated videos from the laparoscopic group and found that 22% of the graded videos scored below the suggested minimum, meaning that the surgical proficiency could have been further optimized before starting the trial.13,14 While, predefined surgeon and center credentials are used in a minority of surgical RCTs, they mainly include a specific job level or prior number of cases that are often set low to allow feasibility of the trial.71

Interestingly, little is often known about outcomes from patients that were operated during the learning curve before the start of RCTs.72 The clinical outcome of these patients is however no less important than that of patients outside of the learning curve and measures should be taken to improve the outcome and safety of these patients, while additionally reporting this data to evaluate how new techniques can be safely introduced.

To minimize the risk of bias due to learning curve effects, quality control by video, study-specific training with rigorous proficiency criteria and formal statistical calculation of the learning curve should be considered in all surgical RCTs as is recently done more frequently.33

While a complete description of the learning curve is difficult to achieve, we propose that learning curve assessment should be based on several key factors analyzed, reflecting specific changes over the respective learning curve phase:

  • i. Intraoperative parameters showing competency in an early phase (eg, operative time/blood loss).

  • ii. For proficiency and mastery (see later), it takes further experience, which is reflected by improved postoperative outcomes (eg, perioperative complications) AND improved oncologic outcomes (eg, resection margin/lymph node harvest) at a later stage.

  • iii. The uptake of more complex cases happens at later stages during the learning curve and can represent an additional learning phase as was shown also for arterial replacement in pancreatic surgery.58,59 Reporting the learning curve assessment would allow an objective assessment and benchmarking, in addition specific training could be tailored to the phase of the surgical learning curve.

According to the findings of our study and the work on different learning phases in bariatric surgery by Wehrtmann et al, we propose a three phase model to report the learning curve in pancreatic surgery: competency, proficiency, and mastery.16 While intraoperative parameters such as operative time and blood loss decreased mainly from phase 1 to phase 2, postoperative parameters were found with a greater reduction from phase 2 to phase 3.

In a first phase, the surgeon learns to carry out a surgical procedure under supervision and with the help of an experienced surgeon. At the end of this phase, the surgeon reaches competency (end of first learning phase) and is able to perform a specific procedure with-out supervision. While operative times diminish during this phase, this does not always translate into better patient outcomes.73 In the second phase, the surgeon is able the solve more complex intraoperative problems in less time through accumulated experience. This phase of proficiency (end of second learning phase) can be defined by reaching patient centered and expert-derived benchmark- or textbook outcomes.16,61,74 Reaching mastery (end of third learning phase), the surgeon is able to operate on more complex non-benchmark cases, for example, in pancreatic surgery advanced tumors requiring vascular resections.58,59 Adaption to changing circumstances happens quickly and intuitively. Technical and oncological outcomes of benchmark-cases are above average. Reporting of postoperative outcomes as textbook- or benchmark outcomes facilitates international comparison.61,74,75

The present study has a few limitations: First of all, the heterogenous definitions of the learning curve make comparisons difficult. Therefore, this study is mainly a qualitative review presenting the available data in an attempt to standardize reporting of learning curves in pancreatic surgery. Furthermore, the included studies were mostly of retrospective design, making them prone to significant selection bias.

In conclusion, this systematic review presents a detailed summary of learning curves for open, laparoscopic, and robotic pancreatic surgery. The number of procedures to surpass a first phase of learning curve was 30 for open PD, 39 for laparoscopic PD, 25 for robotic PD, 16 for laparoscopic DP, and 15 for robotic DP, while no statistical difference could be found between the approaches. Furthermore, it gives an overview of the different definitions and analysis methods of the learning curve, while evaluating the most important confounding factors. Along the learning curve, we recommend a stepwise introduction of different pancreatic resections according to the procedural complexity and propose a standardized reporting of learning curves in pancreatic surgery within a three-phase model (competency, proficiency, and mastery). Furthermore, assessment of learning curves should be based on an adequate statistical calculation (eg, CUSUM analysis) and should consider both individual surgeon and institutional learning curves. Procedural learning curves should be addressed in comparative studies and RCTs in pancreatic surgery to reduce learning curve related bias.

Supplementary Material

as9-3-e111-s001.pdf (781.3KB, pdf)

Footnotes

Published online 27 January 2022

P.C.M. and C.K. have contributed equally to this work.

The authors declare that they have nothing to disclose.

P.C.M. did conception and design, acquisition of data, interpretation of data, drafting the article, gave final approval. C.K. did acquisition of data, statistical analysis, interpretation of data, drafting the article, gave final approval. A.C. and S.S. did conception and design, acquisition of data, critical revision of the article, gave final approval. P.P. did conception and design, interpretation of data, drafting the article, gave final approval. M.d.S., S.V.S., C.T., M.L., A.M., K.Z’g., B.P.M.-S., and T.H. did interpretation of data, critical revision of the article, gave final approval. M.W.B. did conception and design, interpretation of data, critical revision of the article, gave final approval. F.N. did conception and design, interpretation of data, drafting the article, and gave final approval.

The data that support the findings of this study are available on request from the corresponding author, F.N.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.annalsofsurgery.com).

REFERENCES

  • 1.Krautz C, Nimptsch U, Weber GF, et al. Effect of hospital volume on in-hospital morbidity and mortality following pancreatic surgery in Germany. Ann Surg. 2018;267:411–417. [DOI] [PubMed] [Google Scholar]
  • 2.van Heek NT, Kuhlmann KFD, Scholten RJ, et al. Hospital volume and mortality after pancreatic resection: a systematic review and an evaluation of intervention in the Netherlands. Ann Surg. 2005;242:781–788, discussion 788–790. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Casciani F, Trudeau MT, Asbun HJ, et al.; Pancreas Fistula Study Group. Surgeon experience contributes to improved outcomes in pancreatoduodenectomies at high risk for fistula development. Surgery. 2021;169:708–720. [DOI] [PubMed] [Google Scholar]
  • 4.Asbun HJ, Moekotte AL, Vissers FL, et al.; International Study Group on Minimally Invasive Pancreas Surgery (I-MIPS). The miami international evidence-based guidelines on minimally invasive pancreas resection. Ann Surg. 2020;271:1–14. [DOI] [PubMed] [Google Scholar]
  • 5.Moekotte AL, Rawashdeh A, Asbun HJ, et al. Safe implementation of minimally invasive pancreas resection: a systematic review. HPB. 2020;22:637–648. [DOI] [PubMed] [Google Scholar]
  • 6.de Rooij T, Cipriani F, Rawashdeh M, et al. Single-surgeon learning curve in 111 laparoscopic distal pancreatectomies: does operative time tell the whole story? J Am Coll Surg. 2017;224:826–832.e1. [DOI] [PubMed] [Google Scholar]
  • 7.Dokmak S, Ftériche FS, Aussilhou B, et al. The largest European single-center experience: 300 laparoscopic pancreatic resections. J Am Coll Surg. 2017;225:226–234.e2. [DOI] [PubMed] [Google Scholar]
  • 8.Wang M, Meng L, Cai Y, et al. Learning curve for laparoscopic pancreaticoduodenectomy: a CUSUM analysis. J Gastrointest Surg. 2016;20:924–935. [DOI] [PubMed] [Google Scholar]
  • 9.Nagakawa Y, Nakamura Y, Honda G, et al. Learning curve and surgical factors influencing the surgical outcomes during the initial experience with laparoscopic pancreaticoduodenectomy. J Hepatobiliary Pancreat Sci. 2018;25:498–507. [DOI] [PubMed] [Google Scholar]
  • 10.Boone BA, Zenati M, Hogg ME, et al. Assessment of quality outcomes for robotic pancreaticoduodenectomy: identification of the learning curve. JAMA Surg. 2015;150:416–422. [DOI] [PubMed] [Google Scholar]
  • 11.Zureikat AH, Beane JD, Zenati MS, et al. 500 Minimally invasive robotic pancreatoduodenectomies: one decade of optimizing performance. Ann Surg. 2021;273:966–972. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Haney C, Karadza E, Limen E, et al. Training and learning curves in minimally invasive pancreatic surgery: from simulation to mastery. J Pancreatol. 2020;3:101–110. [Google Scholar]
  • 13.van Hilst J, de Rooij T, Bosscha K, et al.; Dutch Pancreatic Cancer Group. Laparoscopic versus open pancreatoduodenectomy for pancreatic or periampullary tumours (LEOPARD-2): a multicentre, patient-blinded, randomised controlled phase 2/3 trial. Lancet Gastroenterol Hepatol. 2019;4:199–207. [DOI] [PubMed] [Google Scholar]
  • 14.Nickel F, Haney CM, Müller-Stich BP, et al. Not yet IDEAL?-evidence and learning curves of minimally invasive pancreaticoduodenectomy. Hepatobiliary Surg Nutr. 2020;9:812–814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Nickel F, Haney CM, Kowalewski KF, et al. Laparoscopic versus open pancreaticoduodenectomy: a systematic review and meta-analysis of Randomized Controlled Trials. Ann Surg. 2020;271:54–66. [DOI] [PubMed] [Google Scholar]
  • 16.Wehrtmann FS, de la Garza JR, Kowalewski KF, et al. Learning curves of laparoscopic Roux-en-Y gastric bypass and sleeve gastrectomy in bariatric surgery: a systematic review and introduction of a standardization. Obes Surg. 2020;30:640–656. [DOI] [PubMed] [Google Scholar]
  • 17.Kalkum E, Klotz R, Seide S, et al. Systematic reviews in surgery-recommendations from the Study Center of the German Society of Surgery. Langenbecks Arch Surg. 2021;406:1723–1731. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Shamseer L, Moher D, Clarke M, et al.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647. [DOI] [PubMed] [Google Scholar]
  • 19.Goossen K, Tenckhoff S, Probst P, et al. Optimal literature search for systematic reviews in surgery. Langenbecks Arch Surg. 2018;403:119–129. [DOI] [PubMed] [Google Scholar]
  • 20.Bassi C, Marchegiani G, Dervenis C, et al.; International Study Group on Pancreatic Surgery (ISGPS). The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 Years After. Surgery. 2017;161:584–591. [DOI] [PubMed] [Google Scholar]
  • 21.Bassi C, Dervenis C, Butturini G, et al.; International Study Group on Pancreatic Fistula Definition. Postoperative pancreatic fistula: an international study group (ISGPF) definition. Surgery. 2005;138:8–13. [DOI] [PubMed] [Google Scholar]
  • 22.Wente MN, Veit JA, Bassi C, et al. Postpancreatectomy hemorrhage (PPH): an International Study Group of Pancreatic Surgery (ISGPS) definition. Surgery. 2007;142:20–25. [DOI] [PubMed] [Google Scholar]
  • 23.Strobel O, Brangs S, Hinz U, et al. Incidence, risk factors and clinical implications of chyle leak after pancreatic surgery. Br J Surg. 2017;104:108–117. [DOI] [PubMed] [Google Scholar]
  • 24.Wente MN, Bassi C, Dervenis C, et al. Delayed gastric emptying (DGE) after pancreatic surgery: a suggested definition by the International Study Group of Pancreatic Surgery (ISGPS). Surgery. 2007;142:761–768. [DOI] [PubMed] [Google Scholar]
  • 25.Koch M, Garden OJ, Padbury R, et al. Bile leakage after hepatobiliary and pancreatic surgery: a definition and grading of severity by the International Study Group of Liver Surgery. Surgery. 2011;149:680–688. [DOI] [PubMed] [Google Scholar]
  • 26.Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Guyatt GH, Oxman AD, Kunz R, et al.; GRADE Working Group. GRADE guidelines: 7. Rating the quality of evidence–inconsistency. J Clin Epidemiol. 2011;64:1294–1302. [DOI] [PubMed] [Google Scholar]
  • 28.Anonymous. R: a language and environment for statistical computing. GBIF.ORG. 2015. Available at: http://www.gbif.org/resource/81287. Accessed February 12, 2015. [Google Scholar]
  • 29.Takahashi C, Shridhar R, Huston J, et al. Outcomes associated with robotic approach to pancreatic resections. J Gastrointest Oncol. 2018;9:936–941. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Tsamalaidze L, Stauffer JA. Pancreaticoduodenectomy: minimizing the learning curve. J Vis Surg. 2018;4:64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Wang X, Cai Y, Jiang J, et al. Laparoscopic pancreaticoduodenectomy: outcomes and experience of 550 patients in a single institution. Ann Surg Oncol. 2020;27:4562–4573. [DOI] [PubMed] [Google Scholar]
  • 32.Hackert T, Strobel O, Michalski CW, et al. The TRIANGLE operation - radical surgery after neoadjuvant treatment for advanced pancreatic cancer: a single arm observational study. HPB (Oxford). 2017;19:1001–1007. [DOI] [PubMed] [Google Scholar]
  • 33.van Workum F, Verstegen MHP, Klarenbeek BR, et al.; ICAN Collaborative Research Group. Intrathoracic vs cervical anastomosis after totally or hybrid minimally invasive esophagectomy for esophageal cancer: a Randomized Clinical Trial. JAMA Surg. 2021;156:601–610. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Ohtsuka T, Ban D, Nakamura Y, et al. Difficulty scoring system in laparoscopic distal pancreatectomy. J Hepatobiliary Pancreat Sci. 2018;25:489–497. [DOI] [PubMed] [Google Scholar]
  • 35.Fisher WE, Hodges SE, Wu MF, et al. Assessment of the learning curve for pancreaticoduodenectomy. Am J Surg. 2012;203:684–690. [DOI] [PubMed] [Google Scholar]
  • 36.Hardacre JM. Is there a learning curve for pancreaticoduodenectomy after fellowship training? HPB Surg. 2010;2010:230287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Park HM, Han SS, Park SJ, et al. Learning curve for pancreatoduodenectomy: can it be generalized? ANZ J Surg. 2020;90:1414–1421. [DOI] [PubMed] [Google Scholar]
  • 38.Tseng JF, Pisters PW, Lee JE, et al. The learning curve in pancreatic surgery. Surgery. 2007;141:694–701. [DOI] [PubMed] [Google Scholar]
  • 39.Wang M, Peng B, Liu J, et al. Practice patterns and perioperative outcomes of laparoscopic pancreaticoduodenectomy in China: a retrospective multicenter analysis of 1029 patients. Ann Surg. 2021;273:145–153. [DOI] [PubMed] [Google Scholar]
  • 40.Marino MV, Podda M, Pisanu A, et al. Robotic-assisted pancreaticoduodenectomy: technique description and performance evaluation after 60 cases. Surg Laparosc Endosc Percutan Tech. 2020;30:156–163. [DOI] [PubMed] [Google Scholar]
  • 41.Napoli N, Kauffmann EF, Perrone VG, et al. The learning curve in robotic distal pancreatectomy. Updates Surg. 2015;67:257–264. [DOI] [PubMed] [Google Scholar]
  • 42.Shi Y, Wang W, Qiu W, et al. Learning curve from 450 cases of robot-assisted pancreaticoduocectomy in a high-volume pancreatic center: optimization of operative procedure and a retrospective study. Ann Surg. 2021;274:e1277–e1283. [DOI] [PubMed] [Google Scholar]
  • 43.Pernar LIM, Robertson FC, Tavakkoli A, et al. An appraisal of the learning curve in robotic general surgery. Surg Endosc. 2017;31:4583–4596. [DOI] [PubMed] [Google Scholar]
  • 44.Mark Knab L, Zenati MS, Khodakov A, et al. Evolution of a novel robotic training curriculum in a complex general surgical oncology fellowship. Ann Surg Oncol. 2018;25:3445–3452. [DOI] [PubMed] [Google Scholar]
  • 45.Klompmaker S, van der Vliet WJ, Thoolen SJ, et al. Procedure-specific training for robot-assisted distal pancreatectomy. Ann Surg. 2021;274:e18–e27. [DOI] [PubMed] [Google Scholar]
  • 46.Kennedy GT, McMillan MT, Sprys MH, et al. The influence of fellowship training on the practice of pancreatoduodenectomy. HPB (Oxford). 2016;18:965–978. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Kassite I, Bejan-Angoulvant T, Lardy H, et al. A systematic review of the learning curve in robotic surgery: range and heterogeneity. Surg Endosc. 2019;33:353–365. [DOI] [PubMed] [Google Scholar]
  • 48.Speicher PJ, Nussbaum DP, White RR, et al. Defining the learning curve for team-based laparoscopic pancreaticoduodenectomy. Ann Surg Oncol. 2014;21:4014–4019. [DOI] [PubMed] [Google Scholar]
  • 49.Jones LR, Zwart MJW, Molenaar IQ, et al. Robotic pancreatoduodenectomy: patient selection, volume criteria, and training programs. Scand J Surg. 2020;109:29–33. [DOI] [PubMed] [Google Scholar]
  • 50.Parker SH, Lei X, Fitzgibbons S, et al. The impact of surgical team familiarity on length of procedure and length of stay: inconsistent relationships across procedures, team members, and sites. World J Surg. 2020;44:3658–3667. [DOI] [PubMed] [Google Scholar]
  • 51.Xiao Y, Jones A, Zhang BB, et al. Team consistency and occurrences of prolonged operative time, prolonged hospital stay, and hospital readmission: a retrospective analysis. World J Surg. 2015;39:890–896. [DOI] [PubMed] [Google Scholar]
  • 52.Xu R, Carty MJ, Orgill DP, et al. The teaming curve: a longitudinal study of the influence of surgical team familiarity on operative time. Ann Surg. 2013;258:953–957. [DOI] [PubMed] [Google Scholar]
  • 53.Elbardissi AW, Duclos A, Rawn JD, et al. Cumulative team experience matters more than individual surgeon experience in cardiac surgery. J Thorac Cardiovasc Surg. 2013;145:328–333. [DOI] [PubMed] [Google Scholar]
  • 54.Callery MP, Pratt WB, Kent TS, et al. A prospectively validated clinical risk score accurately predicts pancreatic fistula after pancreatoduodenectomy. J Am Coll Surg. 2013;216:1–14. [DOI] [PubMed] [Google Scholar]
  • 55.Miskovic D, Ni M, Wyles SM, et al. Learning curve and case selection in laparoscopic colorectal surgery: systematic review and international multicenter analysis of 4852 cases. Dis Colon Rectum. 2012;55:1300–1310. [DOI] [PubMed] [Google Scholar]
  • 56.Zou SY, Wang WS, Zhan Q, et al. Higher body mass index deteriorates postoperative outcomes of pancreaticoduodenectomy. Hepatobiliary Pancreat Dis Int. 2020;19:163–168. [DOI] [PubMed] [Google Scholar]
  • 57.Beane JD, Zenati M, Hamad A, et al. Robotic pancreatoduodenectomy with vascular resection: outcomes and learning curve. Surgery. 2019;166:8–14. [DOI] [PubMed] [Google Scholar]
  • 58.Loos M, Kester T, Klaiber U, et al. Arterial resection in pancreatic cancer surgery: effective after a learning curve [published online ahead of print June 12, 2020]. Ann Surg. doi: 10.1097/SLA.0000000000004054. [DOI] [PubMed] [Google Scholar]
  • 59.Mihaljevic AL, Hackert T, Loos M, et al. Not all Whipple procedures are equal: proposal for a classification of pancreatoduodenectomies. Surgery. 2021;169:1456–1462. [DOI] [PubMed] [Google Scholar]
  • 60.Bockhorn M, Uzunoglu FG, Adham M, et al.; International Study Group of Pancreatic Surgery. Borderline resectable pancreatic cancer: a consensus statement by the International Study Group of Pancreatic Surgery (ISGPS). Surgery. 2014;155:977–988. [DOI] [PubMed] [Google Scholar]
  • 61.Raptis DA, Sánchez-Velázquez P, Machairas N, et al. Defining benchmark outcomes for pancreatoduodenectomy with portomesenteric venous resection. Ann Surg. 2020;272:731–737. [DOI] [PubMed] [Google Scholar]
  • 62.Müller PC, Frey MC, Ruzza CM, et al. Neoadjuvant chemotherapy in pancreatic cancer: an appraisal of the current high-level evidence. Pharmacology. 2021;106:143–153. [DOI] [PubMed] [Google Scholar]
  • 63.Noyez L. Cumulative sum analysis: a simple and practical tool for monitoring and auditing clinical performance. Health Care Curr Rev. 2013;2:1–3. [Google Scholar]
  • 64.Roberts KJ, Boteon APCS, Marcon F, et al. Risk adjusted assessment of individual surgeon’s pancreatic fistula outcomes. HPB (Oxford). 2020;22:452–460. [DOI] [PubMed] [Google Scholar]
  • 65.McMillan MT, Soi S, Asbun HJ, et al. Risk-adjusted outcomes of clinically relevant pancreatic fistula following pancreatoduodenectomy: a model for performance evaluation. Ann Surg. 2016;264:344–352. [DOI] [PubMed] [Google Scholar]
  • 66.Steiner SH, Cook RJ, Farewell VT. Risk-adjusted monitoring of binary surgical outcomes. Med Decis Making. 2001;21:163–169. [DOI] [PubMed] [Google Scholar]
  • 67.Mungroop TH, van Rijssen LB, van Klaveren D, et al.; Dutch Pancreatic Cancer Group. Alternative Fistula Risk Score for pancreatoduodenectomy (a-FRS): design and international external validation. Ann Surg. 2019;269:937–943. [DOI] [PubMed] [Google Scholar]
  • 68.Shrikhande SV, Sivasanker M, Vollmer CM, et al.; International Study Group of Pancreatic Surgery (ISGPS). Pancreatic anastomosis after pancreatoduodenectomy: a position statement by the International Study Group of Pancreatic Surgery (ISGPS). Surgery. 2017;161:1221–1234. [DOI] [PubMed] [Google Scholar]
  • 69.Roberts KJ, Hodson J, Mehrzad H, et al. A preoperative predictive score of pancreatic fistula following pancreatoduodenectomy. HPB (Oxford). 2014;16:620–628. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Schuh F, Mihaljevic AL, Probst P, et al. A simple classification of pancreatic duct size and texture predicts postoperative pancreatic fistula: a classification of the International Study Group of Pancreatic Surgery (ISGPS) [published online ahead of print March 12, 2021]. Ann Surg. doi: 10.1097/SLA.0000000000004855. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Conroy EJ, Rosala-Hallas A, Blazeby JM, et al. Randomized trials involving surgery did not routinely report considerations of learning and clustering effects. J Clin Epidemiol. 2019;107:27–35. [DOI] [PubMed] [Google Scholar]
  • 72.Meagher AP, Yang S, Li S. Is it right to ignore learning-curve patients? Laparoscopic colorectal trials. ANZ J Surg. 2017;87:898–902. [DOI] [PubMed] [Google Scholar]
  • 73.Chen W, Sailhamer E, Berger DL, et al. Operative time is a poor surrogate for the learning curve in laparoscopic colorectal surgery. Surg Endosc. 2007;21:238–243. [DOI] [PubMed] [Google Scholar]
  • 74.Sánchez-Velázquez P, Muller X, Malleo G, et al. Benchmarks in pancreatic surgery: a novel tool for unbiased outcome comparisons. Ann Surg. 2019;270:211–218. [DOI] [PubMed] [Google Scholar]
  • 75.van Roessel S, Mackay TM, van Dieren S, et al.; Dutch Pancreatic Cancer Group. Textbook outcome: nationwide analysis of a novel quality measure in pancreatic surgery. Ann Surg. 2020;271:155–162. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

as9-3-e111-s001.pdf (781.3KB, pdf)

Articles from Annals of Surgery Open are provided here courtesy of Wolters Kluwer Health

RESOURCES