Abstract
Background
Although several prediction models for the occurrence of postoperative pancreatic fistula (POPF) after pancreatoduodenectomy (PD) exist, all were established using Western cohorts. Large-scale external validation studies in Eastern cohorts that consider demographic variables including lower body mass index (BMI) are scarce. The purpose of this study was to externally validate POPF prediction models using nationwide large-scale Korean cohorts.
Methods
Nine tertiary university hospitals in the Republic of Korea participated. Patients’ preoperative characteristics, intraoperative factors, and pathologic findings were evaluated. POPF grades were determined according to the 2016 International Study Group on Pancreatic Surgery definition. Three POPF risk models (Callery, Roberts, and Mungroop) were selected for external validation.
Results
A total of 1,898 PD patients were enrolled. A non-pancreatic disease diagnosis [hazard ratio (HR), 1.856; 95% confidence interval (CI), 1.223–2.817; P=0.004), higher preoperative BMI (HR, 1.069; 95% CI, 1.019–1.121; P=0.006), and soft pancreatic texture (HR, 1.859; 95% CI, 1.264–2.735; P=0.002) were independent risk factors for clinically relevant POPF (CR-POPF). The area under the receiver operating characteristic curve (AUC) values were 0.61, 0.64, and 0.63 on the Callery, Roberts, and Mungroop models, respectively; all were lower than those published in each external validation study.
Conclusions
Western POPF prediction models performed less well when applied to Korean cohorts. Thus, a large-scale Eastern-specific and externally validated POPF prediction model is needed.
Keywords: Pancreatic fistula, pancreatoduodenectomy (PD), predictive score
Introduction
Postoperative pancreatic fistula (POPF) remains a lethal complication, and is related with increased hospital stays, costs, and surgery-related mortality rates (1-4). The definition of POPF was unified and clarified by The International Study Group for Pancreatic Surgery (ISGPS) in 2005 (5), and revised in 2016 (6).
The availability of a prediction model is required for the development of individualized programs for the postoperative management of POPF. Such models can also decrease unnecessary postoperative interventions and hospital costs in low-risk patients, enable appropriate evaluations and treatments, and decrease life-threatening events and mortality in high-risk patients (7). Callery et al. reported that pancreatic texture, pathologic diagnosis, main pancreatic duct (MPD) diameter, and intraoperative estimated blood loss (EBL) were associated with the development of POPF after pancreatoduodenectomy (PD) and subsequently established the Callery score of POPF prediction based on these four factors (2). The Callery score has been widely used and demonstrated reliable validity in some studies (8-10). Simpler risk prediction models of POPF were recently developed and reported in the United Kingdom (UK) and The Netherlands (11,12). The UK group proposed body mass index (BMI) and MPD diameter as POPF risk factors, while The Netherlands group reported pancreatic texture, BMI and MPD diameter. These two models performed moderately well in external validation studies (11,13).
However, these POPF prediction models were established using Western cohorts. Baseline patient characteristics, surgical techniques, and postoperative managements differ different between Eastern and Western countries, and large-scale external validation studies of Eastern cohorts are scarce. This study aimed to externally validate several POPF prediction models using large-scale nationwide Korean cohorts.
Methods
Patients
This retrospective cohort study examined data from a prospectively collected medical database. Nine tertiary university hospitals in the Republic of Korea participated in this study: Seoul National University Hospital, Samsung Medical Center, Asan Medical Center, Catholic Medical Center, Gangnam Severance Hospital, National Cancer Center in Korea, Korea University Guro Hospital, Hallym University Sacred Heart Hospital, and Seoul Metropolitan Government Seoul National University Boramae Hospital. The surgeons in each of these nine hospitals performed a minimum of 20 cases of PD annually. Patients who underwent PD or pylorus-preserving pancreatoduodenectomy (PPPD) due to periampullary disease were enrolled. Patients who did not undergo pancreatico-enteric anastomosis, who underwent combined other organ resections, who had a previous history of abdominal surgery before PD, and for whom insufficient medical data were available to investigate POPF were excluded.
This study was approved by our hospital’s institutional review board (C-1806-129-954).
Data collection and definition of POPF
Preoperative patient characteristics were investigated, including age, sex, BMI, preoperative co-morbidities, preoperative lab data, and MPD diameter on a cross-sectional view of a preoperative computed tomography (CT) image. Intraoperative factors included operation type, combined vascular resection, pancreatico-enteric anastomosis site (pancreaticojejunostomy or pancreaticogastrostomy), pancreatico-enteric anastomosis type (invagination or duct-to-mucosa), operation time, intraoperative EBL, and pancreatic texture. Postoperative data included pathologic diagnosis, drain amylase concentration on postoperative day 3, and complications with a Clavian-Dindo classification > grade II. Pancreatic cancer and pancreatitis were categorized as pancreatic disease, and the other pancreatic disease such as benign pancreatic cystic neoplasm, or neuroendocrine tumors were categorized in other periampullary disease of non-pancreatic disease (2).
POPF was determined based on the 2016 ISGPS guideline, while clinically relevant POPF (CR-POPF) was defined as grade B or C (6).
Statistical analysis
Nominal data were compared using χ2 tests, while continuous variables were examined using Student’s t-test. Only variables statistically significant on univariate analysis were included in the multivariate analysis. To calculate the performance of each model, the area under the receiver operating characteristic curve (AUC) was calculated. All statistical analyses were performed using R version 3.3.0 (R Foundation, Vienna, Austria), and two-sided P values <0.05 were considered statistically significant.
Results
Patient demographics
Between 2007 and 2014, a total of 1,898 patients from the nine tertiary pancreaticobiliary centers in Korea were enrolled in this study (Table 1). The mean patient age was 62.6 years; 1,116 patients (58.8%) were male; and 669 (35.2%) were diagnosed with pancreatic ductal adenocarcinoma, 407 (21.4%) with extrahepatic common bile duct cancer, 365 (19.2%) with ampulla of Vater cancer, 39 (2.1%) with duodenal cancer, and 396 (20.9%) with other periampullary diseases (Table 1). The mean patient BMI was 23.0. Of the cohort, 1,767 patients (93.1%) underwent pancreaticojejunostomy anastomosis and 1,630 (85.9%) underwent duct-to-mucosa anastomosis. A soft pancreatic texture was seen in 57.5% of cases. Overall, POPF occurred in 752 patients (39.6%), while CR-POPF (grade B or C) occurred in 275 (14.5%).
Table 1. Demographic and pathologic findings of modeling cohorts in four studies.
| Variables | Callery et al. (2) | Roberts et al. (11) | Mungroop et al. (12) | Present study |
|---|---|---|---|---|
| Modeling cohorts (No.) | 233 | 217 | 1,924 | 1,898 |
| Age, year, median (IQR) | – | 67 [60–73] | 67 [60–74] | 64 [56–70] |
| Male sex (%) | – | 120 (55.8) | 778 (57.0) | 1,116 (58.8) |
| Diagnosis (%) | ||||
| Pancreatic ductal adenocarcinoma | 145 (62.2) | 88 (40.6) | 816 (43.0) | 669 (35.2) |
| Pancreatitis | – | 50 (3.0) | 22 (1.2) | |
| Common bile duct cancer | 88 (37.8) | 33 (15.2) | 265 (14.0) | 407 (21.4) |
| Ampulla of Vater cancer | 40 (18.4) | 265 (14.0) | 365 (19.2) | |
| Duodenal cancer | 11 (5.1) | 110 (6.0) | 39 (2.1) | |
| Others | 45 (20.7) | 453 (20.0) | 396 (20.9) | |
| Preoperative BMI, kg/m2, median (IQR) | – | 25.0 (22.5–27.8) | 25 [22–28] | 22.9 (20.8–24.9) |
| Preoperative diabetes mellitus (%) | – | 31 (14.3) | 403 (26.0) | 467 (24.6) |
| Preoperative MPD diameter, mm, median (IQR) | – | – | 4 [2–5] | 3 [2–5] |
| Anastomotic site (%) | ||||
| Pancreaticojejunostomy | – | 117 (53.9) | – | 1,767 (93.1) |
| Pancreaticogastrostomy | – | 100 (46.1) | – | 131 (6.9) |
| Estimated blood loss (≥1,000 mL) (%) | 13 (5.6) | – | – | 236 (15.9) |
| Soft pancreatic texture (%) | 110 (47.2) | – | 970 (54.0) | 1,091 (57.5) |
| Overall POPF (%) | 60 (25.8) | 48 (22.1) | – | 752 (39.6) |
| Clinically relevant POPF (%) | 31 (13.3) | 27 (12.4) | 232 (12.0) | 275 (14.5) |
BMI, body mass index; MPD, main pancreatic duct; POPF, postoperative pancreatic fistula.
Predictive factors for CR-POPF after PD
In the univariate analysis, male sex, non-pancreatic disease diagnosis, preoperative BMI, preoperative DM, MPD diameter on preoperative CT images, operation type, absence of vessel resection, duct-to-mucosa anastomosis, operation time, intraoperative blood loss, and soft pancreatic texture were associated with CR-POPF (Table 2). In the multivariate analysis, a non-pancreatic disease diagnosis [hazard ratio (HR), 1.856; 95% confidence interval (CI), 1.223–2.817; P=0.004), higher preoperative BMI (HR, 1.069; 95% CI, 1.019–1.121; P=0.006), and soft pancreatic texture (HR, 1.859; 95% CI, 1.264–2.735; P=0.002) were the independent risk factors for CR-POPF after PD.
Table 2. Predictive factors for postoperative pancreatic fistula in univariate and multivariate analysis.
| Predictive factors | Total (N=1,898) | Univariate analysis | Multivariate analysis | |||||
|---|---|---|---|---|---|---|---|---|
| CR-POPF (−) (N=1,623) | CR-POPF (+) (N=275) | P value | HR | 95% CI | P value | |||
| Age (year, mean ± SD) | 62.6±10.7 | 62.5±10.9 | 63.2±9.8 | 0.280 | – | – | – | |
| Gender (%) | ||||||||
| Male | 1,116 (58.8) | 934 (83.7) | 182 (16.3) | 0.008 | 1.334 | 0.970–1.835 | 0.077 | |
| Female | 782 (41.2) | 689 (88.1) | 93 (11.9) | |||||
| Diagnosis (%) | ||||||||
| Others | 1,198 (63.1) | 975 (81.4) | 223 (18.6) | <0.001 | 1.856 | 1.223–2.817 | 0.004 | |
| Pancreatic | 693 (36.5) | 641 (92.5) | 52 (7.5) | |||||
| Preoperative BMI (kg/m2, mean ± SD) | 23.0±3.2 | 22.9±3.3 | 23.7±2.9 | <0.001 | 1.069 | 1.019–1.121 | 0.006 | |
| Preoperative DM (%) | 467 (24.6) | 413 (25.4) | 54 (19.6) | 0.041 | 1.124 | 0.765–1.653 | 0.552 | |
| Preoperative MPD diameter (mm, mean ± SD) | 3.6±2.2 | 3.7±2.3 | 3.2±1.6 | <0.001 | 0.936 | 0.855–1.025 | 0.153 | |
| Operation type (%) | ||||||||
| PPPD | 1,558 (82.1) | 1,314 (84.3) | 244 (15.7) | 0.002 | 1.231 | 0.750–2.019 | 0.411 | |
| Whipple’s operation | 340 (17.9) | 309 (90.9) | 31 (9.1) | |||||
| Vessel resection (%) | ||||||||
| No | 1,754 (92.4) | 1,486 (84.7) | 268 (15.3) | 0.001 | 1.623 | 0.657–4.009 | 0.294 | |
| Yes | 144 (7.6) | 137 (95.1) | 7 (4.9) | |||||
| Anastomotic site (%) | ||||||||
| Pancreaticojejunostomy | 1,767 (93.1) | 1,503 (85.1) | 264 (14.9) | 0.052 | – | – | – | |
| Pancreaticogastrostomy | 131 (6.9) | 120 (91.6) | 11 (8.4) | |||||
| Anastomotic method (%) | ||||||||
| Duct-to-mucosa | 1,630 (85.9) | 1,372 (84.2) | 258 (15.8) | <0.001 | 1.293 | 0.428–3.904 | 0.648 | |
| Invagination | 145 (7.6) | 138 (95.2) | 7 (4.8) | 0.845 | 0.074–9.643 | 0.893 | ||
| Others | 123 (6.5) | 113 (91.9) | 10 (8.1) | |||||
| Operation time (minute, mean ± SD) | 369.3±116.7 | 373.1±117.9 | 347.1±106.9 | <0.001 | 1.000 | 0.998–1.002 | 0.996 | |
| Intraoperative blood loss (mL, mean ± SD) | 593.0±653.8 | 607.3±677.0 | 521.8±517.7 | 0.026 | 1.000 | 1.000–1.000 | 0.688 | |
| Pancreatic texture (%) | ||||||||
| Soft | 1,091 (57.5) | 891 (81.7) | 200 (18.3) | <0.001 | 1.859 | 1.264–2.735 | 0.002 | |
| Hard | 716 (37.7) | 654 (91.3) | 62 (8.7) | |||||
| Malignancy (%) | ||||||||
| Yes | 1,603 (84.5) | 1,362 (85.0) | 241 (15.0) | 0.126 | – | – | – | |
| No | 295 (15.5) | 261 (88.5) | 34 (11.5) | |||||
CR-POPF, clinically-relevant postoperative pancreatic fistula; BMI, body mass index; DM, diabetes mellitus; MPD, main pancreatic duct; PPPD, pylorus-preserving pancreatoduodenectomy; HR, hazard ratio; CI, confidence interval.
Comparisons of discrimination ability and relationship between score severity and actual occurrence rates of POPF
Of the published POPF prediction models, we selected those with proven validity in clinical circumstances in both internal and large-scale external validation studies. The models of Callery et al., Roberts et al., and Mungroop et al. were selected and investigated (2,11,12). Table 3 shows the summarization of these three-scoring system and the comparisons of the AUC values. The three western models showed moderate discrimination ability that the AUC values were more than 0.7. Figure 1 shows the ROCs of the Korean cohorts based on the three-scoring system (Figure 1A, Callery model; Figure 1B Roberts model; Figure 1C, Mungroop model). The AUC values were calculated based on these ROCs that these models performed less well when the Korean cohorts were applied that the AUC values were between 0.61 and 0.64.
Table 3. Comparisons of the scoring system of postoperative pancreatic fistula after pancreatoduodenectomy.
| POPF model | Modeling cohorts (n) | POPF (2005 grade) | Predictive factors | AUC (internal) | External validation studies | |
|---|---|---|---|---|---|---|
| Validation cohorts (n) | AUC | |||||
| Callery et al. (2) | 233 | B, C | Pancreatic texture; MPD diameter; diagnosis; EBL | 0.94 | 594 [9]; 444 [8]; 1,898† | 0.72; 0.72; 0.61 |
| Roberts et al. (11) | 217 | A, B, C | MPD diameter; BMI | 0.75 | 630 [13]; 1,898† | 0.77; 0.64 |
| Mungroop et al. (12) | 1,924 | B, C | Pancreatic texture; MPD diameter; BMI | 0.75 | 924 [12]; 1,898† | 0.78; 0.63 |
†, the present study. POPF, postoperative pancreatic fistula; AUC, area under the receiver operating curve; MPD, main pancreatic duct; EBL, estimated blood loss; BMI, body mass index; Internal, indicates internal validation study.
Figure 1.
The receiver operating curves of the Korean cohorts based on the three models. (A) Callery model (2); (B) Roberts model (11); (C) Mungroop model (12).
Figure 2 shows the relationship between score severity and actual POPF occurrence rates of Korean cohorts for each scoring system. The higher the score, the more frequent the occurrence of POPF in each model. However, the actual CR-POPF rate in the high-risk group was lower in this Korean cohort than in the reported validation studies. Regarding Callery score, the actual rate of CR-POPF in the high-risk group (Callery score 7–10) was 20.3% in the Korean cohort (Figure 2A) versus 28.6% in one external validation study (9) and 42.3% in another study (8). Regarding Mungroop score, the actual CR-POPF rate was 19.3% in the Korean cohort (Figure 2C) versus 31.0% in an external validation study (11).
Figure 2.
Relationship between score severity and actual POPF rate of Korean cohorts. (A) Callery model; (B) Roberts model; (C) Mungroop model. POPF, postoperative pancreatic fistula; CR-POPF, clinically-relevant POPF.
Discussion
Predicting POPF is important because it enables the development of individualized treatment plans for affected patients. Numerous risk factors for POPF have been identified, while several POPF prediction models have been subsequently developed (2,11,12,14-16). However, most have not been externally validated. Furthermore, there have been few large-scale external validation studies of POPF in Eastern cohorts. The present study identified three large-scale validated models and investigated their applicability to a large-scale Korean cohort.
AUC values, a discriminatory index for the prediction models, range from 0.5 (no discrimination ability) to 1.0 (perfect discrimination) (17,18). One study categorized the discriminatory ability of AUC values of 0.5–0.7 as poor, 0.7–0.9 as reasonable, and 0.9–1.0 as very good (19). Table 2 summarized the outcomes of internal and external validation studies for Callery, Roberts, and Mungroop scores. An internal validation revealed an AUC of the Callery score of 0.94 (2). However, the predictability of internal validation could be overestimated because establishing the prediction model was based on the multivariate logistic regression analysis of the study modeling cohorts. Miller et al. performed an external validation using the data of 594 PD patients who underwent pancreaticojejunostomy anastomosis and reported an AUC of 0.72 (9). Grendar et al. reported an AUC of 0.71 (8). Regarding Roberts and Mungroop scoring, the AUC in the external validation study was 0.77 and 0.78, respectively (Table 2) (11,13). These values indicated that these three models demonstrated reasonable performance for predicting POPF or CR-POPF when applied to Western cohorts.
Miller et al. insisted that the variations in predictability among the validation studies might have been caused by differences in patient characteristics, POPF rates in the high-risk groups, and perioperative managements (9). The present study performed external validation of the three methods using the data of 1,898 consecutive Korean PD patients from nine tertiary hospitals. The incidence of POPF in this Korean cohort increased as score severity increased (Figure 2). However, the AUC values were 0.61–0.64, meaning that these Western POPF scoring models had poor discriminatory ability when applied to our Korean cohort (Table 3). This might have been due to differences in demographics such as BMI and the proportion of patients with non-pancreatic diseases (Table 1). In addition, in our high-risk group of our cohorts, the actual CR-POPF rates were <30% for Callery scoring (Figure 2A) and 20% for Mungroop scoring (Figure 2C). Therefore, these Western models must be revised to increase their applicability to Korean cohorts.
Pancreatic texture and pathologic diagnosis were the risk factors proposed by Callery et al. (2). Preoperative BMI was recently included in two prediction models (11,12). EBL was also a risk factor proposed by Callery et al. proposed, but two recent external validation studies refuted this claim (8,10). Furthermore, due to the advent of minimally invasive surgery, reported intraoperative blood loss volumes became lower than those of studies of the period when Callery scoring was developed (20-22). In the present study, preoperative valuables including a non-pancreatic disease diagnosis (HR, 1.856; 95% CI, 1.223–2.817; P=0.004), higher preoperative BMI (HR, 1.069; 95% CI, 1.019–1.121; P=0.006), and soft pancreatic texture (HR, 1.859; 95% CI, 1.264–2.735; P=0.002) were the independent risk factors for CR-POPF in the multivariate analysis (Table 2), and one study suggested the exact same risk factors (23). Because patient characteristics, surgical skills, and perioperative management methods differ among countries, the statistically significant factors and hazard ratios could be different. Therefore, it is necessary to establish new models that reflect the factors and hazard ratios of each cohort. To consider such differences, pathologic diagnoses, preoperative BMI, and pancreatic texture should be included in the new model.
The previous three scoring system mentioned MPD size as one of the risk factors (2,11,12). However, it was not consistent with the present study that the MPD size was not statistically associated with CR-POPF (HR 0.936; 95% CI, 0.855–1.025, P=0.153, Table 2). This was probably because the anastomotic method (duct-to-mucosa or invagination), or pancreatic duct stenting method were not unified around nine institutions. In addition, different surgeons investigated MPD size in the preoperative CT image. In order to overcome this limitation, prospective study design and unified method to evaluate the MPD size would be needed.
The present study has some limitations. First, surgical techniques and perioperative management methods were not unified among the nine hospitals. Thus, surgeon preference was a potential confounding factor. Second, data missing from the surgical and pathologic reports could not be controlled for in this retrospective study. To ensure high-quality collaboration studies, centralization of the electric medical database and regular monitoring are needed. Despite these limitations, this was the largest external validation study of Western POPF prediction models in an Eastern cohort.
In conclusion, Western POPF prediction models performed less well when applied to Korean cohorts. Risk factors for POPF in Eastern model were a higher BMI, soft pancreatic texture, and non-pancreatic disease diagnosis. Thus, the development of a large-scale externally validated Eastern-specific POPF prediction model using Eastern cohorts is needed.
Acknowledgments
Funding: This study was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI16C2037), and by the Collaborative Genome Program for Fostering New Post-Genome Industry of the National Research Foundation funded by the Ministry of Science and ICT (NRF-2017M3C9A5031591).
Ethical Statement: This study was approved by the institutional review board of Seoul National University Hospital (C-1806-129-954).
Footnotes
Conflicts of Interest: The authors have no conflicts of interest to declare.
References
- 1.Ahmad SA, Edwards MJ, Sutton JM, et al. Factors influencing readmission after pancreaticoduodenectomy: a multi-institutional study of 1302 patients. Ann surg 2012;256:529-37. 10.1097/SLA.0b013e318265ef0b [DOI] [PubMed] [Google Scholar]
- 2.Callery MP, Pratt WB, Kent TS, et al. A prospectively validated clinical risk score accurately predicts pancreatic fistula after pancreatoduodenectomy. J Am Coll Surg 2013;216:1-14. 10.1016/j.jamcollsurg.2012.09.002 [DOI] [PubMed] [Google Scholar]
- 3.Frymerman AS, Schuld J, Ziehen P, et al. Impact of postoperative pancreatic fistula on surgical outcome—the need for a classification-driven risk management. J Gastrointest Surg 2010;14:711-8. 10.1007/s11605-009-1147-5 [DOI] [PubMed] [Google Scholar]
- 4.McMillan MT, Vollmer CM, Asbun HJ, et al. The characterization and prediction of ISGPF grade C fistulas following pancreatoduodenectomy. J Gastrointest Surg 2016;20:262-76. 10.1007/s11605-015-2884-2 [DOI] [PubMed] [Google Scholar]
- 5.Bassi C, Dervenis C, Butturini G, et al. Postoperative pancreatic fistula: an international study group (ISGPF) definition. Surgery 2005;138:8-13. 10.1016/j.surg.2005.05.001 [DOI] [PubMed] [Google Scholar]
- 6.Bassi C, Marchegiani G, Dervenis C, et al. The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 years after. Surgery 2017;161:584-91. 10.1016/j.surg.2016.11.014 [DOI] [PubMed] [Google Scholar]
- 7.Callery MP, Pratt WB, Vollmer CM. Prevention and management of pancreatic fistula. J Gastrointest Surg 2009;13:163-73. 10.1007/s11605-008-0534-7 [DOI] [PubMed] [Google Scholar]
- 8.Grendar J, Jutric Z, Leal JN, et al. Validation of Fistula Risk Score calculator in diverse North American HPB practices. HPB 2017;19:508-14. 10.1016/j.hpb.2017.01.021 [DOI] [PubMed] [Google Scholar]
- 9.Miller BC, Christein JD, Behrman SW, et al. A multi-institutional external validation of the fistula risk score for pancreatoduodenectomy. J Gastrointest Surg 2014;18:172-79; discussion 179-80. 10.1007/s11605-013-2337-8 [DOI] [PubMed] [Google Scholar]
- 10.Shubert CR, Wagie AE, Farnell MB, et al. Clinical risk score to predict pancreatic fistula after pancreatoduodenectomy: independent external validation for open and laparoscopic approaches. J Am Coll Surg 2015;221:689-98. 10.1016/j.jamcollsurg.2015.05.011 [DOI] [PubMed] [Google Scholar]
- 11.Roberts KJ, Hodson J, Mehrzad H, et al. A preoperative predictive score of pancreatic fistula following pancreatoduodenectomy. HPB 2014;16:620-8. 10.1111/hpb.12186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mungroop TH, van Rijssen LB, van Klaveren D, et al. Alternative Fistula Risk Score for Pancreatoduodenectomy (a-FRS): Design and International External Validation. Ann Surg 2019;269:937-43. 10.1097/SLA.0000000000002620 [DOI] [PubMed] [Google Scholar]
- 13.Roberts KJ, Sutcliffe RP, Marudanayagam R, et al. Scoring system to predict pancreatic fistula after pancreaticoduodenectomy: a UK multicenter study. Ann Surg 2015;261:1191-7. 10.1097/SLA.0000000000000997 [DOI] [PubMed] [Google Scholar]
- 14.Chen JY, Feng J, Wang XQ, et al. Risk scoring system and predictor for clinically relevant pancreatic fistula after pancreaticoduodenectomy. World J Gastroenterol 2015;21:5926-33. 10.3748/wjg.v21.i19.5926 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Kim JY, Park JS, Kim JK, et al. A model for predicting pancreatic leakage after pancreaticoduodenectomy based on the international study group of pancreatic surgery classification. Korean J Hepatobiliary Pancreat Surg 2013;17:166-70. 10.14701/kjhbps.2013.17.4.166 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Yamamoto Y, Sakamoto Y, Nara S, et al. A preoperative predictive scoring system for postoperative pancreatic fistula after pancreaticoduodenectomy. World J Surg 2011;35:2747-55. 10.1007/s00268-011-1253-x [DOI] [PubMed] [Google Scholar]
- 17.Kumar R, Indrayan A. Receiver operating characteristic (ROC) curve for medical researchers. Indian Pediatr 2011;48:277-87. 10.1007/s13312-011-0055-4 [DOI] [PubMed] [Google Scholar]
- 18.Swets JA. Measuring the accuracy of diagnostic systems. Science 1988;240:1285-93. 10.1126/science.3287615 [DOI] [PubMed] [Google Scholar]
- 19.Pearce J, Ferrier S. Evaluating the predictive performance of habitat models developed using logistic regression. Ecol Modell 2000;133:225-45. 10.1016/S0304-3800(00)00322-7 [DOI] [Google Scholar]
- 20.Palanivelu C, Senthilnathan P, Sabnis SC, et al. Randomized clinical trial of laparoscopic versus open pancreatoduodenectomy for periampullary tumours. Br J Surg 2017;104:1443-50. 10.1002/bjs.10662 [DOI] [PubMed] [Google Scholar]
- 21.Peng L, Lin S, Li Y, et al. Systematic review and meta-analysis of robotic versus open pancreaticoduodenectomy. Surg Endosc 2017;31:3085-97. 10.1007/s00464-016-5371-2 [DOI] [PubMed] [Google Scholar]
- 22.Stauffer JA, Coppola A, Villacreses D, et al. Laparoscopic versus open pancreaticoduodenectomy for pancreatic adenocarcinoma: long-term results at a single institution. Surg Endosc 2017;31:2233-41. 10.1007/s00464-016-5222-1 [DOI] [PubMed] [Google Scholar]
- 23.Jang JY, Chang YR, Kim SW, et al. Randomized multicentre trial comparing external and internal pancreatic stenting during pancreaticoduodenectomy. Br J Surg 2016;103:668-75. 10.1002/bjs.10160 [DOI] [PubMed] [Google Scholar]