Development and validation of a novel predictive model for postpancreatectomy hemorrhage using lasso-logistic regression: an international multicenter observational study of 9631 pancreatectomy patients

Abstract

Background:

Hemorrhage following pancreatectomy represents a grave complication, exerting a significant impact on patient prognosis. The formulation of a precise predictive model for postpancreatectomy hemorrhage risk holds substantial importance in enhancing surgical safety and improving patient outcomes.

Materials and methods:

This study utilized the patient cohort from the American College of Surgeons National Surgical Quality Improvement Program database, who underwent pancreatectomy between 2014 and 2017 (n=5779), as the training set to establish the Lasso-logistic model. For external validation, a patient cohort (n=3852) from the Chinese National Multicenter Database of Pancreatectomy Patients, who underwent the procedure between 2014 and 2020, was employed. A predictive nomogram for postpancreatectomy hemorrhage was developed, and polynomial equations were extracted. The performance of the predictive model was assessed through the receiver operating characteristic curve, calibration curve, and decision curve analysis.

Results:

In the training and validation cohorts, 9.0% (520/5779) and 8.5% (328/3852) of patients, respectively, experienced postpancreatectomy hemorrhage. Following selection via lasso and logistic regression, only nine predictive factors were identified as independent risk factors associated with postpancreatectomy hemorrhage. These included five preoperative indicators [BMI, American Society of Anesthesiologists (ASA) ≥3, preoperative obstructive jaundice, chemotherapy within 90 days before surgery, and radiotherapy within 90 days before surgery], two intraoperative indicators (total operation time, vascular resection), and two postoperative indicators (postoperative septic shock, pancreatic fistula). The new model demonstrated high predictive accuracy, with an area under the receiver operating characteristic curve of 0.87 in the external validation cohort. Its predictive performance significantly surpassed that of the previous five postpancreatectomy hemorrhage risk prediction models (P<0.001, likelihood ratio test).

Conclusion:

The Lasso-logistic predictive model we developed, constructed from nine rigorously selected variables, accurately predicts the risk of PPH. It has the potential to significantly enhance the safety of pancreatectomy surgeries and improve patient outcomes.

Introduction

Highlights

• This study successfully developed a Lasso-logistic regression predictive model based on an international multicenter cohort of 9631 pancreatectomy patients, specifically designed to accurately predict the risk of postpancreatectomy hemorrhage.

• Compared to several existing models, the Lasso-logistic regression predictive model developed in this study exhibits greater accuracy in predicting postpancreatectomy hemorrhage, and it provides capabilities for risk stratification.

• The model is particularly adept at handling large datasets with high-dimensional variables, effectively resolving collinearity issues among these variables.

• User-friendly in design, the model can be conveniently implemented in clinical settings, both pre-and post-surgery, to identify and manage patients at a high risk of hemorrhage following pancreatectomy.

Pancreatectomy, encompassing pancreaticoduodenectomy, central pancreatectomy, distal pancreatectomy, and total/subtotal pancreatectomy, is a critical, intricate, and demanding surgical intervention indicated for a spectrum of pancreatic pathologies ranging from chronic pancreatitis to pancreatic ductal adenocarcinoma¹. Despite a notable decrease in postoperative mortality in the past decade, the prevalence of postoperative complications in high-volume pancreatic centers continues to hover around 30–50%². Frequent complications associated with pancreatectomy include pancreatic fistula, anastomotic leak, hemorrhage, intra-abdominal abscess, and delayed gastric emptying. Of these, postpancreatectomy hemorrhage (PPH) emerges as the most fatal, exhibiting an incidence of 3–20% and an associated mortality rate reaching 20–50%. Anticipating the risk of PPH is pivotal for tailoring individualized treatment approaches, which may involve selecting the optimal surgical resection extent, enhanced transfusion readiness, and intensified postoperative surveillance. Consequently, the development of a clinically feasible PPH risk prediction model is imperative for dynamically and accurately forecasting PPH risk, implementing preventive strategies, and diminishing the incidence and mortality associated with PPH in patients.

Previous studies have attempted to identify risk factors for PPH, as exemplified by the work of researchers like Uggeri, Ricci, and Lu, who employed logistic regression methods to select relevant risk factors. However, these studies exhibit certain limitations in terms of predictive accuracy and risk stratification, attributable to the limited number of considered indicators, small patient sample sizes, and the absence of external dataset validation^2–4. Similarly, while nomogram models utilizing logistic regression for PPH risk scoring have been developed, most are based on small cohort studies, constraining their external validation feasibility and potentially diminishing predictive accuracy due to a limited number of included indicators⁵. Moreover, when dealing with large datasets containing high-dimensional variables, traditional logistic regression may encounter issues of collinearity among variables, further challenging the reliability of the models⁶.

In our study, we analyzed data from pancreatectomy patients between 2014 and 2017 from the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database, which represents a large, multicenter sample set. We developed a novel predictive model for PPH using the Lasso-logistic regression method, incorporating a multitude of perioperative indicators. To ascertain the accuracy and reliability of the model, we employed an external validation set comprising data from patients who underwent pancreatectomy across multiple centers in China from 2014 to 2020. The validation results demonstrate that our model provides dynamic and accurate risk predictions for PPH, enhancing the basis for clinical decision-making.

Materials and methods

Patients study cohort

The foundational data for this study were derived from the ACS-NSQIP database, encompassing cases of pancreatectomy conducted between 2014 and 2017⁷. This database incorporates perioperative data within 30 days post-surgery from multiple national medical centers, maintained and quality-controlled by professional clinical auditors to ensure data integrity⁷. The study cohort was identified using specific CPT codes (48150, 48152, 48153, and 48154), precisely pinpointing the relevant surgical cases⁷. For external validation of the model, data from the National Pancreatectomy Patient Database established by the National Cancer Center of China were used, including patient information on surgeries performed between 2014 and 2020. Cases that did not involve a pancreatectomy, had incomplete information, or lacked clinically significant indicators (e.g. nursing methods) were excluded from the study. The research process and data filtration are detailed in Figure 1. Throughout the design and validation of the clinical predictive model, the TRIPOD guidelines for multivariable predictive model development were rigorously followed⁸. As the study exclusively utilized non-identifiable pre-existing data, it was exempt from informed consent procedures in accordance with the Declaration of Helsinki and the Ethical Guidelines for Clinical Studies (No. P-2021-1230). This research received formal approval from the Ethics Committee of the ***, with the ethical approval number 17-168/1424. Additionally, this study was registered on ResearchRegistry.com with the unique identification code ***, and data reporting was conducted in accordance with the STROCSS 2021, Supplemental Digital Content 1, http://links.lww.com/JS9/D165 guidelines⁹.

Figure 1.

The research process and data filtration. ACS-NSQIP, American College of Surgeons National Surgical Quality Improvement Program; PPH, postpancreatectomy hemorrhage; ROC, receiver operating characteristic.

Definitions and indicators

In accordance with the 2007 ISGPS guidelines, the severity of PPH is categorized into Grades A, B, and C. Grade A PPH only results in transient and marginal changes in the standard postoperative course of pancreatectomy patients, without significant clinical impact. Grade B PPH necessitates further diagnostic and interventional measures, such as blood transfusion, admission to intermediate or intensive care units, and potential invasive interventions like re-laparotomy or embolization, leading to prolonged hospital stays. Grade C PPH causes severe harm to the patient and is considered potentially life-threatening¹⁰. To comprehensively summarize the risk factors for PPH and develop a predictive model, we analyzed patients across all three grades collectively. Additionally, pancreatic fistula is defined according to the 2016 ISGPS guidelines, also categorized into Grades A, B, and C. Grade A, also known as biochemical leak, is not considered clinically significant, whereas Grades B/C pancreatic fistulas are deemed clinically relevant¹¹. The definition of septic shock follows the latest international consensus, which describes it as life-threatening organ dysfunction caused by a dysregulated host response to infection¹². The study cohort included 60 perioperative indicators, comprising 36 preoperative, 5 intraoperative, and 19 postoperative indicators. Table 1 provides specific information about each indicator. Furthermore, predictive factors for the Uggeri model, Ricci model, Lu model, Georges model, and Yekebas model were identified based on relevant literature^2–4,13,14.

Table 1.

Baseline characteristics of study cohorts.

	Perioperative variables	Training cohort	Validation cohort
	Total number of patients	5779	3852
Preoperative parameters	Age (year), mean (SD)	64.5 (11.5)	64.3 (12.2)
	Sex, n (%)
	Male	3072 (53.2)	1998 (51.9)
	Female	2707 (46.8)	1854 (48.1)
	Weight (kg), mean (SD)	66.2 (4.1)	66.1 (4.1)
	Height (cm), mean (SD)	171.8 (43.1)	171.8 (42.0)
	BMI (kg/m²), mean (SD)	27.5 (6.0)	27.6 (5.9)
	Diabetes, n (%)
	No	4279 (74.0)	2868 (74.5)
	Yes	1500 (26.0)	984 (25.5)
	Smoker (within one year), n (%)
	No	4681 (81.0)	3125 (81.1)
	Yes	1098 (19.0)	727 (18.9)
	Dyspnea, n (%)
	No	5478 (94.8)	3648 (94.7)
	Moderate/rest	301 (5.2)	204 (5.3)
	Ventilator-dependent, n (%)
	No	5776 (99.9)	3817 (99.1)
	Yes	3 (0.1)	35 (0.9)
	History of severe COPD, n (%)
	No	5540 (95.9)	3693 (95.9)
	Yes	239 (4.1)	159 (4.1)
	CHF in 30 days before surgery, n (%)
	No	5756 (99.6)	3845 (99.8)
	Yes	23 (0.4)	7 (0.2)
	Ascites, n (%)
	No	5758 (99.6)	3843 (99.8)
	Yes	21 (0.4)	9 (0.2)
	Hypertension, n (%)
	No	2662 (46.0)	1816 (47.1)
	Yes	3117 (54.0)	2036 (52.9)
	Dialysis before surgery, n (%)
	No	5766 (99.8)	3845 (99.8)
	Yes	13 (0.2)	7 (0.2)
	Steroid use before surgery, n (%)
	No	5639 (97.6)	3755 (97.5)
	Yes	140 (2.4)	97 (2.5)
	Bleeding disorders, n (%)
	No	5637 (97.5)	3742 (97.1)
	Yes	142 (2.5)	110 (2.9)
	>10% weight loss (within 6 months), n (%)
	No	4884 (84.5)	3233 (83.9)
	Yes	895 (15.5)	619 (16.1)
	Transfusion before surgery (within 72 h), n (%)
	No	5732 (99.2)	3830 (99.4)
	Yes	47 (0.8)	22 (0.6)
	Preoperative sepsis, n (%)
	No	5721 (99.0)	3809 (98.9)
	Yes	58 (1.0)	43 (1.1)
	ASA classification≥3, n (%)
	Yes	1368 (23.7)	879 (22.8)
	No	4411 (76.3)	2973 (77.2)
	Quarter of admission, n (%)
	1	1384 (23.9)	962 (25.0)
	2	1538 (26.6)	1013 (26.3)
	3	1429 (24.8)	945 (24.5)
	4	1428 (24.7)	932 (24.2)
	Preoperative obstructive jaundice, n (%)
	No	3313 (57.3)	2280 (59.2)
	Yes	2466 (42.7)	1572 (40.8)
	Preoperative biliary stent, n (%)
	No	2952 (51.1)	1942 (50.4)
	Yes	2827 (48.9)	1910 (49.6)
	Chemotherapy within 90 days before surgery, n (%)
	No	4721 (81.7)	3106 (80.6)
	Yes	1058 (18.3)	746 (19.4)
	Radiotherapy within 90 days before surgery, n (%)
	No	5356 (92.7)	3551 (92.2)
	Yes	423 (7.3)	301 (7.8)
	Pancreatic duct size, n (%)
	<4 mm	1941 (33.6)	1304 (33.9)
	≥4 mm	3838 (66.4)	2548 (66.1)
	Days from admission to operation (day), mean (SD)	0.48 (5.2)	0.45 (2.0)
	Selective operation, n (%)
	No	482 (8.3)	333 (8.6)
	Yes	5297 (91.7)	3519 (91.4)
	Preoperative serum sodium (mg/dl), mean (SD)	138.9 (3.1)	139.0 (3.2)
	Preoperative serum creatinine (mg/dl), mean (SD)	0.9 (0.5)	0.9 (0.3)
	Preoperative serum albumin (g/dl), mean (SD)	3.8 (0.6)	3.8 (0.6)
	Preoperative total bilirubin (mg/dl), mean (SD)	1.5 (2.4)	1.5 (2.3)
	Preoperative alkaline phosphatase (U/l), mean (SD)	171.7 (162.0)	166.2 (150.8)
	Preoperative WBC (109/l), mean (SD)	7.3 (2.7)	7.3 (2.6)
	Preoperative hematocrit (vol%), mean (SD)	38.0 (5.3)	37.9 (5.0)
	Preoperative platelet count (109/l), mean (SD),	255.1 (92.3)	255.2 (93.1)
Intraoperative parameters	Total operation time (min), mean (SD)	352.4 (135.4)	351.9 (133.5)
	Operative approach, n (%)
	Laparoscopic	283 (4.9)	170 (4.4)
	Robotic	278 (4.8)	181 (4.7)
	Open	5202 (90.0)	3487 (90.5)
	Hybrid	16 (0.3)	14 (0.4)
	Pancreatic texture, n (%)
	Soft	2823 (48.8)	1823 (47.3)
	Intermediate/hard	2956 (51.2)	2029 (52.7)
	Pancreatic reconstruction, n (%)
	Pancreaticojejunal duct-to-mucosa	5069 (87.7)	3419 (88.8)
	Other	710 (12.3)	433 (11.2)
	Vascular resection, n (%)
	No	4990 (86.3)	3317 (86.1)
	Yes	789 (13.7)	535 (13.9)
Postoperative parameters	PPH, n (%)
	No	5259 (91.0)	3524 (91.5)
	Yes	520 (9.0)	328 (8.5)
	Superficial incisional SSI, n (%)
	No	5378 (93.1)	3555 (92.3)
	Yes	401 (6.9)	297 (7.7)
	Deep SSI (deep incisional SSI/organ or space SSI), n (%)
	No	4793 (82.9)	3288 (85.4)
	Yes	986 (17.1)	564 (14.6)
	Wound disruption, n (%)
	No	5715 (98.9)	3817 (99.1)
	Yes	64 (1.1)	35 (0.9)
	Postoperative pneumonia, n (%)
	No	5387 (93.2)	3646 (94.7)
	Yes	392 (6.8)	206 (5.3)
	Pulmonary embolism, n (%)
	No	5710 (98.8)	3815 (99.0)
	Yes	69 (1.2)	37 (1.0)
	Postoperative mechanical ventilation > 48 h, n (%)
	No	5560 (96.2)	3744 (97.2)
	Yes	219 (3.8)	108 (2.8)
	Renal insufficiency, n (%)
	No	5749 (99.5)	3824 (99.3)
	Yes	30 (0.5)	28 (0.7)
	Renal failure, n (%)
	No	5736 (99.3)	3814 (99.0)
	Yes	43 (0.7)	38 (1.0)
	Urinary tract infection, n (%)
	No	5609 (97.1)	3756 (97.5)
	Yes	170 (2.9)	96 (2.5)
	Stroke with neurological deficit, n (%)
	No	5768 (99.8)	3839 (99.7)
	Yes	11 (0.2)	13 (0.3)
	Myocardial infarction, n (%)
	No	5721 (99.0)	3821 (99.2)
	Yes	58 (1.0)	31 (0.8)
	Deep vein thrombosis, n (%)
	No	5620 (97.2)	3740 (97.1)
	Yes	159 (2.8)	112 (2.9)
	Sepsis, n (%)
	No	5278 (91.3)	3523 (91.5)
	Yes	501 (8.7)	329 (8.5)
	Septic shock, n (%)
	No	5567 (96.3)	3746 (97.2)
	Yes	212 (3.7)	106 (2.8)
	Wound closure, n (%)
	All	5762 (99.7)	3848 (99.9)
	Not all	17 (0.3)	4 (0.1)
	Pancreatic fistula, n (%)
	No/Grade A	4848 (83.9)	3220 (83.6)
	Grade B/C	931 (16.1)	632 (16.4)
	Delayed gastric emptying, n (%)
	No	4901 (84.8)	3256 (84.5)
	Yes	878 (15.2)	596 (15.5)
	Histology, n (%)
	Pancreatic adenocarcinoma	4498 (77.8)	3032 (78.7)
	Other	1281 (22.2)	820 (21.3)

ASA, American Society of Anesthesiologists; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; PPH, postpancreatectomy hemorrhage; SSI, surgical site infection; WBC, white blood cell.

Statistical analysis

In this study, all statistical analyses were conducted using the R programming software (version 4.2.3). Continuous variables were described using mean values and standard deviations (SD), while categorical variables were presented as percentages. In dealing with continuous variables, we adhered to the recommendations of the latest risk assessment tool—Prediction Model Risk of Bias Assessment Tool (PROBAST)—avoiding arbitrary dichotomization or categorization, as this could lead to the loss of key information in nonlinear relationships with outcome variables¹⁵. For similar reasons, univariate analysis was not conducted to prevent overlooking important variables or misidentifying associations due to the lack of consideration for interactions among multiple variables¹⁵. Lasso regression was utilized for variable selection, which, by penalizing the regression, effectively reduces unnecessary predictive variables and addresses issues of collinearity, particularly suitable for high-dimensional datasets^6,16. After identifying significant predictive variables, logistic regression was used for further model construction, and a nomogram based on the logistic model was created to predict the risk of PPH. This process is succinctly termed lasso-logistic regression¹⁷. During this process, the nomogramEx package in R was also employed to extract polynomial equations from the model. For risk stratification, the optimalCutoff function and the Youden index were applied¹⁸. Model similarity was evaluated using the likelihood ratio test, while the model fit was assessed through the Akaike Information Criterion (AIC) and Hosmer and Lemeshow tests¹⁷. The model’s performance was evaluated using the area under the receiver operating characteristic curve (AUC) and misclassification error¹⁷. Additionally, the model’s net benefit was assessed through decision curve analysis (DCA)¹⁷, and its predictive capability was evaluated using clinical impact curves. In this study, statistical significance was defined as P less than 0.05.

Results

Cohort characteristics

The training cohort consisted of 5779 patients enrolled in the ACS-NSQIP database from 2014 to 2017, including 3072 males and 2707 females, with an average age of 64.5 years and a mean BMI of 27.5 kg/m². The external validation cohort, comprising 3852 patients from the National Multicenter Database of Pancreatectomy Patients in China between 2014 and 2020, included 1998 males and 1854 females, with an average age of 64.3 years and a mean BMI of 27.6 kg/m². The incidence of PPH in the training cohort was 9.0% (520/5779), while in the validation cohort, it was 8.5% (328/3852). In the training cohort, 90% of the surgeries were open procedures, 4.9% were laparoscopic, and 4.8% were robotic. Corresponding proportions in the validation cohort were 90.5%, 4.4%, and 4.7%, respectively. The average duration of surgery for patients in both the training and validation cohorts was 352.4 and 351.9 min, respectively. The baseline characteristics of patients undergoing pancreatectomy in both cohorts are summarized in Table 1.

Development of a novel predictive model

Lasso regression for variable selection

To address issues of multicollinearity and prevent overfitting in the presence of high-dimensional variables, lasso regression was utilized for the initial selection of 60 variables in the training cohort. The number of model variables was reduced to 59 when log(λ) reached the minimum mean squared error, and further decreased to 9 at the point of the minimum standard error (1-fold SE) as determined by internal cross-validation using lasso (10 k-fold) (Fig. 2A, B). We favored the selection at the 1-fold SE of the minimum distance (best λ value=0.012) for its simplicity and interpretability. The lasso regression ultimately selected 9 optimal variables (Table 2), including total operation time, BMI, ASA greater than or equal to 3, preoperative obstructive jaundice, chemotherapy within 90 days before surgery, radiotherapy within 90 days before surgery, postoperative septic shock, vascular resection, and pancreatic fistula.

Figure 2.

Lasso regression curves. (A) The curve of the regression. coefficient versus log (λ); (B) the curve of MSE versus log (λ). The λ.min represents the cutoff point at which MSE takes the minimum value, while λ.1se represents the point where MSE takes 1 x standard error.

Table 2.

Variables identified by Lasso-based penalization in the training cohort.

Best λ value	Variables in lasso model	Coefficient	OR
	Total operation time	0.003	1.003
	BMI	0.058	1.060
	ASA≥3	1.049	2.854
	Preoperative obstructive jaundice	0.798	2.220
0.012	Chemotherapy within 90 days before surgery	1.631	5.107
	Radiotherapy within 90 days before surgery	0.449	1.566
	Postoperative septic shock	2.111	8.255
	Vascular resection	0.705	2.024
	Pancreatic fistula	2.113	8.273

ASA, American Society of Anesthesiologists; OR, odds ratio.

Development of a predictive nomogram based on logistic regression, extraction of scoring equation

Subsequently, the 9 variables selected by lasso regression were incorporated into logistic regression for further modeling. The results indicated that all 9 variables were independent risk factors for PPH (P<0.01) (Table 3), with no collinearity issues among them (VIF<2) (Table 3, Supplementary Fig. 1, Supplemental Digital Content 2, http://links.lww.com/JS9/D166), suggesting successful model development. A predictive nomogram for the occurrence of PPH based on the logistic regression model was then developed (Fig. 3A). This nomogram included 9 predictive indicators: 5 preoperative (BMI, ASA≥3, preoperative obstructive jaundice, chemotherapy within 90 days before surgery, radiotherapy within 90 days before surgery), 2 intraoperative (total operation time, vascular resection), and 2 postoperative (postoperative septic shock, pancreatic fistula). However, due to limitations of the nomogram model, specific risk scores for predictive indicators could not be obtained, which restricted its clinical applicability. To address this, we extracted a polynomial equation from the predictive model, allowing for the calculation of specific risk scores for each indicator and the total PPH risk score (Fig. 3B). The ∑ score represents the sum of risk scores across the 9 predictive indicators.

Table 3.

Independent predictors for the nomogram were determined through Lasso-logistic regression within the training cohort.

Nomogram model	VIF	OR	95% CI	P
Total operation time	1.08	1.003	1.002–1.004	<0.001
BMI	1.07	1.060	1.042–1.078	<0.001
ASA≥3	1.01	2.854	2.025–4.136	<0.001
Preoperative obstructive jaundice	1.07	2.220	1.779–2.778	<0.001
Chemotherapy within 90 days before surgery	1.52	5.107	3.899–6.685	<0.001
Radiotherapy within 90 days before surgery	1.35	1.566	1.122–2.180	<0.01
Postoperative septic shock	1.04	8.255	5.690–11.955	<0.001
Vascular resection	1.11	2.024	1.561–2.618	<0.001
Pancreatic fistula	1.07	8.273	6.593–10.399	<0.001

ASA, American Society of Anesthesiologists; OR, odds ratio; VIF, variance inflation factor.

Figure 3.

The nomogram for predicting PPH based on the Lasso-logistic. regression. (A) Nomogram representing the point assignment for each variable to calculate the predicted probability of PPH, the corresponding three risk stratifications are labeled at the bottom; (B) Risk calculation formula: The value of each variable corresponds to different points, and the risk value can be obtained by summing the points and inputting them into the risk formula. PPH, postpancreatectomy hemorrhage.

$$\mathrm{PPH}=0.12+8.18\mathrm{e}-05*\sum {\mathrm{Points}}^{\wedge }2-1.49\mathrm{e}-07*\sum {\mathrm{Points}}^{\wedge }3-0.01*\sum \mathrm{Points}$$

Risk stratification using the new model for PPH

To further stratify the risk of PPH, the optimalCutoff function and the Youden index were applied to determine binary risk stratification thresholds for the predictive model in the training cohort. We identified two distinct thresholds, 0.07 and 0.13, which were respectively named as the Optimal-cutoff value and Best-cutoff value (Fig. 4A). Subsequently, we utilized the intersection of these two values to create three risk strata: low, medium, and high risk. These corresponded to predicted probabilities of less than or equal to 0.07, greater than 0.07 and less than or equal to 0.13, and greater than 0.13, respectively. The discriminative ability of this risk stratification method was assessed by displaying the model’s receiver operating characteristic (ROC) curves within each risk group in both the training and validation cohorts (Fig. 4B). The results indicated that the model exhibited good discriminative ability in all three risk groups, particularly in the low and high-risk groups. Further analysis of the actual incidence of PPH in each risk group revealed that this three-tiered method demonstrated effective differentiation in both the training and validation cohorts (Fig. 4C, Table 4).

Figure 4.

Establishment and evaluation of the risk stratification. model. (A) Two thresholds for risk stratification; (B) Receiver operating characteristic curves of the models within each stratification group in the training and validation cohorts; (C) Actual incidence of PPH in each risk stratification group. AUC, area under the curve; PPH, postpancreatectomy hemorrhage.

Table 4.

Distribution characteristics of patients in different risk stratifications.

Training cohort (n=5779)						Validation cohort (n=3852)
Risk group	N	PPH (+)	PPH (−)	Positive rate (%)	Negative rate (%)	N	PPH (+)	PPH (−)	Positive rate (%)	Negative rate (%)
Low risk	4068	97	3971	2.4	97.6	2552	50	2502	2.0	98.0
Moderate risk	637	47	590	7.4	92.6	503	66	437	13.1	86.9
High risk	1074	376	698	35.0	65.0	797	230	567	28.9	71.1

PPH, postpancreatectomy hemorrhage.

Evaluation and external validation of the new model

Advantages of lasso regression in model variable selection

We introduced a novel modeling approach, termed the Lasso-logistic model, which involves initial variable selection through lasso penalized regression, followed by incorporating these variables into logistic regression for modeling. Traditional variable selection methods, such as univariate analysis and inclusion of all variables, are contrasted here, though the latest PROBAST guidelines advise against univariate analysis. To further demonstrate the superiority of the Lasso-logistic model, we compared its performance with a full-variable logistic model that included all variables. The results revealed a statistically significant difference between the two predictive models (P<0.001, likelihood ratio test) (Table 5). However, the Lasso-logistic model had fewer predictors (9 vs. 15), better model fit (AIC, 2384.5 vs. 2389.9), and higher model discrimination (AUC, 87.0% vs. 83.0%) in the validation cohort (Table 5, Fig. 5A and B). Notably, the misclassification error of the Lasso-logistic model was also lower than that of the Full-logistic model (8.1% vs. 9.8%) (Table 5). These findings suggest that lasso regression is a superior method to traditional variable selection approaches, and the performance of the Lasso-logistic model surpasses that of the Full-logistic model.

Table 5.

Evaluation and comparison of two variable screening methods in the validation cohort.

Models	Variable inclusion methods	Independent variables number (P<0.05)	AIC	Misclass error (%)	AUC (%)	Likelihood ratio test (P)
Lasso-logistic model	Lasso regression	9	2384.5	8.1	87.0	<0.001
Full-logistic model	Full variables	15	2389.9	9.8	83.0

AIC, Akaike Information Criterion; AUC, area under the curve.

Figure 5.

ROC curves of seven different models in the validation. cohort. (A) Lasso-logistic model; (B) Full-logistic model; (C) Uggeri model; (D) Ricci model; (E) Lu model; (F) Georges model; (G) Yekebas model. The figure marks the best-cutoff value, specificity, sensitivity, and AUC value for each model. AUC, area under the curve; ROC, Receiver operating characteristic.

Advantages of the Lasso-logistic model compared to traditional models

The five previously reported PPH risk scoring models—Uggeri model, Ricci model, Lu model, Georges model, and Yekebas model—were all created based on logistic regression. We evaluated and validated these five logistic models. As shown in Table 6, all five traditional models demonstrated statistical differences compared to the Lasso-logistic model (P<0.001, likelihood ratio test). Compared to the five traditional models, the Lasso-logistic model exhibited better fit (AIC, 2384.5 vs. 3036.3, 3106.6, 3081.6, 3090.8, and 3110.5) (Fig. 6A-F), higher model discrimination in the validation cohort (AUC, 87.0% vs. 62.3%, 60.6%, 60.0%, 59.3%, and 58.9%) (Fig. 5A, C-G), and the lowest misclassification error (8.1% vs. 14.5%, 17.7%, 13.8%, 18.3%, and 19.5%) (Table 6). The DCA curves (Fig. 6G) revealed that, within a threshold probability range of 0.03-0.95, the net benefit of the Lasso-logistic model was significantly higher than the other five traditional models. Figure 5H illustrates the relationship between the number of PPH cases predicted by the Lasso-logistic model and the actual number of true positive cases at different threshold probabilities (with a base of 1000 cases). Thus, the Lasso-logistic model surpasses the five conventional models, and its slightly increased number of variables does not significantly impact its usability or clinical applicability.

Table 6.

Evaluation and comparison of six different models in the validation cohort.

Models	Independent variables number (P<0.05)	AIC	Misclass error (%)	AUC (%)	Hosmer and Lemeshow test (P)	Likelihood ratio test (P )
Lasso-logistic model	9	2384.5	8.1	87.0	0.087
Uggeri model	4	3036.3	14.5	62.3	0.005	<0.001
Ricci model	2	3106.6	17.7	60.6	0.017	<0.001
Lu model	4	3081.6	13.8	60.0	0.001	<0.001
Georges model	2	3090.8	18.3	59.3	0.002	<0.001
Yekebas model	1	3110.5	19.5	58.9	0.020	<0.001

AIC, Akaike Information Criterion; AUC, area under the curve.

Figure 6.

Evaluation of different models in the validation cohort. (A–F) Calibration curves of six different models: (A) Lasso-logistic model; (B) Uggeri model; (C) Ricci model; (D) Lu model; (E) Georges model; (F) Yekebas model. The P value is obtained from the Hosmer and Lemeshow goodness of fit (GOF) test. A P value > 0.05 indicates a good model fit. (G) Decision curve analysis of six different models; (H) Clinical impact curve of the Lasso-logistic model. The red curve represents the number of predicted cases of postpancreatectomy hemorrhage (PPH) using the Lasso-logistic model under the threshold probability, while the blue curve represents the actual number of cases of PPH that occurred under the threshold probability.

Discussion

Despite significant advancements in surgical techniques and postoperative care over the past decade, the incidence and associated mortality of PPH remain high¹⁰. Prevention and timely management of postoperative bleeding are crucial for reducing mortality following pancreatectomy, and an accurate PPH predictive model can provide proactive strategies for preventive management and early intervention. In this study, we incorporated a total of 60 potential indicators that may affect PPH preoperatively, intraoperatively, and postoperatively, and utilized reliable analogous analytical methods for modeling. We identified nine independent risk factors impacting PPH—total operation time, BMI, ASA greater than or equal to 3, preoperative obstructive jaundice, chemotherapy within 90 days before surgery, radiotherapy within 90 days before surgery, postoperative septic shock, vascular resection, and pancreatic fistula. Subsequently, a nomogram for predicting PPH risk was established, with an extracted calculation formula and stratification of risk. This model is capable of dynamically and accurately predicting a patient’s risk of PPH, holding significant clinical applicability.

In 2007, Yekebas et al. ¹⁴ found that the prognosis of PPH primarily depends on the presence of pancreatic fistula, marking an early logistic regression model for PPH risk prediction. In 2012, Ricci et al. ⁴ enriched the risk prediction model for PPH, suggesting that pancreatic reconstruction might be a new potential risk factor for pancreatic fistula. In 2019, Lu and colleagues and Uggeri and colleagues’ teams each proposed new PPH risk prediction models, further enhancing the accuracy of PPH prediction. The Lu model incorporated four independent risk factors: pancreatic duct diameter less than 0.4 cm, pancreatic fistula, intra-abdominal abscess, and delayed gastric emptying², while the Uggeri model included pancreatic fistula, male gender, ASA greater than or equal to 3, and hypertension³. The most recent model, proposed by Georges and colleagues in 2021, included only pancreatic fistula and intra-abdominal abscess as predictive indicators¹³. In all these five models, pancreatic fistula is a consistent risk factor, underscoring its significant value in PPH prediction. Postpancreatectomy, patients with pancreatic cancer may produce more erosive pancreatic juice from the remnant pancreas, and insufficient drainage of pancreatic fluid can cause pseudoaneurysms, vascular erosion, and other vascular abnormalities, ultimately leading to catastrophic bleeding. A plausible pathophysiological explanation is that vessel stumps or walls can be eroded by proteases such as trypsin, elastase, and other pancreatic exocrine secretions¹⁴.

The American Society of Anesthesiologists (ASA) classification system, first introduced in 1941, serves as a subjective assessment of a patient’s overall health status and is currently divided into five categories (I–V). Category I represents a completely healthy, fit patient. Category II includes patients with mild systemic disease. Category III encompasses patients with severe systemic disease that is not incapacitating. Category IV is assigned to patients with incapacitating disease that constantly threatens life. Category V describes a moribund patient who is not expected to live 24 h with or without surgery¹⁹. Numerous studies have identified the ASA classification as an effective tool for stratifying the risk of postoperative complications and mortality, with higher ASA classifications correlating with increased probabilities of postoperative complications and mortality²⁰. Both the logistic regression model developed by Uggeri and our Lasso-logistic regression model identified ASA greater than or equal to 3 as an independent risk factor for PPH.

Compared to previous PPH predictive models, our model incorporates several new predictive indicators, significantly elevating the AUC value to 87.0% in the external validation cohort, surpassing the previous five models. One novel indicator is total operation time, a key factor affecting surgical outcomes. Prolonged surgery has been linked to increased risks of surgical site infections, anastomotic leaks, and transfusions. The potential mechanism involves extended surgical field exposure, leading to increased bleeding, stress responses, immune system impact, and systemic inflammatory responses, all of which are potential risk factors for postoperative bleeding^21,22. Patients with a higher BMI, typically indicating obesity, are associated with increased risks of metabolic syndrome and cardiovascular diseases, as well as being risk factors for complications in abdominal surgeries²³. Obesity can complicate surgical exposure due to increased visceral fat and omental weight, thereby increasing surgical difficulty and the risk of postoperative bleeding²⁴. Preoperative obstructive jaundice is another newly identified independent risk factor for PPH. Previous research has established a close association between post-pancreaticoduodenectomy hemorrhage and hyperbilirubinemia, especially severe cases. Obstructive jaundice is a major cause of hyperbilirubinemia, leading to immune response impairment, disturbances in fat and fat-soluble vitamin absorption, nutritional deficiencies, biliary infections, endotoxemia, liver and renal dysfunction, and coagulation disorders, all contributing to an increased risk of PPH²⁵. Neoadjuvant chemotherapy can cause peritoneal tissue exudation and edema, increasing tissue fragility and blurring anatomical planes, thus increasing surgical complexity, prolonging operation time, and augmenting intraoperative blood loss²⁶. Neoadjuvant radiotherapy likewise elevates risks of postoperative wound infection and intra-abdominal abscess due to localized tissue fibrosis, edema, and exacerbated inflammatory responses in irradiated areas, making surgical removal more challenging and a potential risk factor for PPH²⁷. Septic shock triggers severe inflammation, liver dysfunction, and disseminated intravascular coagulation (DIC) similar to macrophage activation syndrome²⁸. This coagulation abnormality, a common complication in patients with septic shock, can further lead to inadequate perfusion, major organ dysfunction, and death²⁹. In pancreatectomy patients, septic shock-induced DIC is a significant cause of PPH. In recent years, the number of vascular resections involving the superior mesenteric vein, portal vein, celiac axis, superior mesenteric artery, and hepatic artery has substantially increased, improving pancreatic cancer resectability³⁰. However, due to the complexity of these surgeries, some studies have found an increased postoperative mortality rate in patients undergoing vascular resection compared to standard pancreatectomy, with postoperative hemorrhage being one of the primary causes^31,32. In addition, the time of removal of the abdominal drainage tube is also an important variable. We considered including this variable in the data analysis, but unfortunately, due to the serious lack of data on the time of removal of postoperative drainage tube, we had to give up the analysis of this important variable. However, we still explored the relationship between early drainage tube removal and post-pancreatic resection complications such as PPH. The study by Chen et al. ³³ found that early removal of drainage tubes could lower the incidence of postoperative intra-abdominal infections, as well as reduce the risks of delayed gastric emptying and the need for reoperations, concurrently shortening the postoperative hospital stay. Moreover, research by Wu et al. ³⁴ suggests that early drain removal is significantly associated with a reduced incidence of clinically relevant postoperative pancreatic fistula, in addition to decreasing the occurrences of PPH, readmissions, and reoperations. These advantages may be related to early drain removal reducing traumatic stimulation, mechanical damage, the duration and intensity of inflammatory responses near the drainage tube, and facilitating a quicker return to the normal abdominal environment³³. We also found that bleeding disorders are not independent risk factors for PPH. We conducted a thorough analysis of the data and propose the following possible explanations: (1) The sample size of patients with bleeding disorders in our study cohort may be too small to detect a significant association between bleeding disorders and the risk of PPH. A smaller sample size could result in insufficient statistical power to reveal potential relationships. (2) Doctors may have implemented effective preventive and management measures for patients with bleeding disorders, thereby reducing their risk of PPH. These interventions might include the pre-arrangement of blood products and the use of hemostatic medications. (3) Although our study controlled for several known risk factors, there may still be unaccounted confounding factors. These confounders could obscure the true relationship between bleeding disorders and the risk of PPH.

In patient treatment, to avoid or minimize the occurrence of PPH, we can optimize management strategies and surgical planning based on the aforementioned risk indicators. To reduce total operation time, thorough preoperative planning and simulation should be conducted, along with enhancing the surgical team’s expertise, to swiftly address intraoperative complications and minimize unnecessary surgical duration²². For patients with high BMI, implementing weight reduction programs and providing specialized postoperative nutritional support and monitoring can promote wound healing²⁴. Patients with high ASA scores necessitate comprehensive preoperative assessment and risk management, with perioperative multidisciplinary team collaboration to manage their overall condition³⁵. Patients with severe obstructive jaundice should receive focused liver function and coagulation assessments, with preoperative biliary drainage if necessary, coupled with active liver protection and vitamin K supplementation to improve physical condition²⁵. For patients undergoing neoadjuvant chemotherapy and radiotherapy, comprehensive preoperative assessment is essential to ensure sufficient recovery time and to address potential tissue damage caused by these treatments^36,37. To prevent postoperative septic shock, enhanced postoperative monitoring, early identification and treatment of infections, and optimized antibiotic use are crucial²⁹. In pancreatectomy patients undergoing concomitant vascular resection, selecting experienced surgical teams for vascular resections and employing meticulous vascular anastomosis techniques during surgery are vital³². To avoid and reduce postoperative pancreatic fistula, preventive intraoperative measures such as appropriate anastomosis techniques and drainage strategies should be implemented, followed by close postoperative monitoring of pancreatic function and the nature of drainage fluid, ensuring timely and effective drainage³⁸.

This study has several limitations. Firstly, the new model has not been extensively validated in other studies, necessitating further clinical evidence to support the universality and reliability of these new indicators. Secondly, the retrospective nature of the study may be prone to recording bias, with the accuracy of data collection and recording directly impacting the model’s reliability. Additionally, our exclusion of missing data introduces the possibility of non-random distribution of missing information, precluding the elimination of residual confounding factors. Furthermore, the loss of some data related to variable time sequences may have impacted the model’s predictive capacity. Despite these limitations, the study’s uniqueness lies in the utilization of extensive data from the multicenter ACS-NSQIP database in the United States and the multicenter Pancreatectomy Patient Database in China for modeling and validation within an international cohort. In constructing the new predictive model, rigorous statistical measures were employed to screen high-dimensional variables. The new model focuses on simplicity and clinical practicability, comprising only nine predictive indicators, and precisely calculates patient PPH risk scores for risk stratification. Its ease of application in clinical settings enhances its potential for widespread use.

Conclusion

Our study, focusing on the risk of PPH, has developed a predictive model based on Lasso-logistic regression. This model is constructed from a rigorously selected set of variables that are not only statistically significant but also clinically relevant, enabling accurate prediction of PPH risk. The model is simple and practical, facilitating its use by clinicians pre- and post-surgery to identify high-risk patients and implement preventive measures to reduce the incidence of PPH. Moreover, the dynamic nature of the model allows for real-time risk assessment tailored to the changing conditions of patients. Overall, the development of this model represents a significant advancement in the management of PPH risk, with the potential to substantially enhance the safety of pancreatectomy procedures and improve patient outcomes.

Ethical approval

The Ethics Committee of the Cancer Hospital of the Chinese Academy of Medical Sciences (National Cancer Center) (17-168/1424).

Consent

As the study exclusively utilized non-identifiable pre-existing data, it was exempt from informed consent procedures in accordance with the Declaration of Helsinki and the Ethical Guidelines for Clinical Studies (No. P-2021-1230).

Source of funding

This research was funded by the CAMS Innovation Fund for Medical Sciences (CIFMS), 2022-I2M-1-010 and Beijing Physician Scientist Training Project（BJPSTP-2024-11）.

Author contribution

Y.D. conceived and designed the study. Y.D., Z.G., Y.M. and X.G. performed the analyses. Y.D. and Y.D. wrote the manuscript. C.W., J.Z. supervised the study. All authors have read and agreed to the published version of the manuscript.

Conflicts of interest disclosure

All other authors declare that they have no competing interests.

Research registration unique identifying number (UIN)

This study was registered on ResearchRegistry.com with the unique identification code researchregistry9650.

Guarantor

Chengfeng Wang and Yunjie Duan.

Data availability statement

The datasets generated during and/or analyzed during the current study are available in the National Surgical Quality Improvement Program (NSQIP) repository, https://www.facs.org/quality-programs/acs-nsqip/participant-use. Further inquiries can be directed to the corresponding author.

Provenance and peer review

Not commissioned, externally peer-reviewed.

Supplementary Material

js9-111-0791-s001.pdf ^{(796.9KB, pdf)}