Development and internal validation of a practical model to predict 30 days mortality of severe acute pancreatitis patients

Abstract

Background

Severe acute pancreatitis (SAP) is a common disease in the intensive care unit (ICU) accompanied by high mortality, the purpose of this study was to build a prediction model for the 30 days mortality of SAP.

Methods

We retrospectively reviewed 149 patients with SAP after admission in 48 h to the ICU of the First Affiliated Hospital of Nanjing Medical University between January 2015 and December 2019. Clinical variables including gender, age, blood routine, and biochemical indicators were collected. On the basis of these variables, stepwise regression analysis was carried out to establish the model. A bootstrapping technique was applied for internal validation.

Results

Age, aspartate aminotransferase (AST), alkaline phosphatase (ALP), triglycerides (TG), and creatinine (CREA) were differences between survivors and nonsurvivors groups (all p < 0.1). Multivariate analysis suggested that age, AST, ALP, TG, and CREA were independent variables. Then, a model was established. The area-under-the curve (AUC) of the model was 0.875 (95% confidence interval (CI): 0.811–0.924). After internal validation, the C-index was 0.859 (95% CI: 0.786–0.932).

Conclusion

Our study has built a refined model with easily acquired biochemical parameters to predict 30 days mortality of SAP admitted to ICU. This model will require external and prospective validation prior to translate into clinical management.

KEY MESSAGES

1. Severe acute pancreatitis is a common disease in the intensive care unit with high mortality.

2. A prediction model for the 30 days mortality of SAP was built with easily acquired parameters.

Background

Acute pancreatitis (AP) is a systemic inflammatory disease and is the most frequent gastrointestinal disease that contributes to hospitalization [1]. The annual incidence of global AP is considered to be 13–45 per 100,000 people, and the incidence rate increases annually [2]. The severity of AP ranges from self-limited mild illness to severe illness characterized by systemic complications and multiple organ failure. It is suggested that about 20%–30% of AP patients finally developed into severe AP (SAP) in clinical treatment, which experienced a more serious attack with progression to multiple organ dysfunction or local complications [3]. Without timely treatment, SAP patients are in an urgent and serious condition. Thus, early assessment and prediction of high-risk SAP combined with appropriate clinical intervention and timely treatment, will greatly improve the prognosis of SAP patients.

There are several scoring systems available for AP assessment, including Ranson, BISAP, the Acute Physiology and Chronic Health Evaluation (APACHE-II), and Sequential organ failure assessment (SOFA) scores [4]. Furthermore, many studies have demonstrated that laboratory indicators, such as the neutrophil-lymphocyte ratio (NLR) [5], C-reactive protein (CRP), serum calcium [6], UREA are also able to predict the prognosis of SAP. NLR can be evaluated as an early indicator of SAP. CRP has been associated with the severity of pancreatitis especially in interval change [5].

Nevertheless, the sensitivity and specificity of these prediction scoring systems are not high enough, and cumbersome items limit their clinical use. Some standards are not sensitive enough or high-cost, which would be unprocurable for most patients in an initial diagnosis. For example, the APACHE-II score has more than 10 variables and is complex in practical use. The Ranson criteria require 48 h hospitalization for observation, thereby resulting in a delay in triage and management [3]. Besides, pancreatic necrosis might not be discovered in early CT scans in 24 h [7]. In this case, the clinical evaluation of sensitive indicators for SAP is of great value. Up to now, no single biomarker can be competent to predict high-risk SAP in the early phase, thus complementary and combined markers are thus imperative.

The model is composed of various independent risk factors for disease progression, which can be used to calculate and predict the probability of event occurrence. A growing number of models are in widespread use to help identify patients with high risk in several diseases [8]. For all we know, there are few models suitable to predict the prognosis of SAP patients. Early detection of clinical deterioration is important to reduce the incidence rate and mortality of SAP. Thus, we aim to provide a basis for early clinical prevention and treatment of SAP by establishing a simple and useful prediction model.

Methods

Research design and patient selection

A retrospective database cohort study was performed on 149 adult patients (92 males and 57 females, aged 19–86-year-old, mean age 38.30 ± 12.63) with SAP after admission in 48 h to the intensive care unit (ICU) of the First Affiliated Hospital of Nanjing Medical University between January 2015 and December 2019. AP diagnosis was on the basis of the presence of two or more of the following criteria according to clinical symptoms, physical examination and laboratory data [1]: Upper abdominal pain during acute episodes [2]; Blood serum amylase and/or lipase activity at least threefold elevation the upper limit of normal [3]; characteristic findings of AP on abdominal imaging. Based on the 2012 revised Atlanta classification criteria, SAP was defined as AP with persistent single or multiple organ failure which lasted at least 48 h [9]. To minimize bias, exclusion criteria included [1]: Age <18 years [2]; Chronic pancreatitis or pancreas carcinoma [3]; Anemia patients [4]; Pregnancy [5]; Patients with malignant tumors [6]; Patients with chronic renal disease [7]; Pre-existing organ failure [8]; Chronic obstructive airway disease [9]; Immunosuppressive disease [10]; endoscopic retrograde cholangiopancreatography (ERCP) or trauma-induced pancreatitis. In addition, for patients with multiple ICU admission, only the first admission was included. Figure 1 presents a flow diagram of study participants. According to the 30-day prognosis, the patients were divided into the survival group and the non-survival group. The study protocol was authorized by the Ethics Committee of the First Affiliated Hospital of Nanjing Medical University (Nanjing, China) and in accordance with the Helsinki Declaration (2023-SR-043).

Figure 1.

Flow diagram of study participants.

Data collection

All documents ranging from clinical characteristics (age, gender, etiology of pancreatitis and comorbidities) and laboratory parameters (the first blood results at initial diagnosis and before the initiation of any treatment were collected, regardless of multiple same examination within 24 h after patients admission), including fibrinogen (FIB), D-Dimer (DD2), lymphocytes (LY), neutrophils (N), hemoglobin (HB), hematocrit (HCT); platelets (PLT), alanine aminotransferase (ALT), aspartate aminotransferase (AST), alkaline phosphatase (ALP), lactate dehydrogenase (LDH), creatine kinase (CK), total bilirubin (TB); direct bilirubin (DB); total cholesterol (TC), triglycerides (TG), high-density lipoprotein (HDL), low-density lipoprotein (LDL), albumin (ALB), urea nitrogen (UREA) and creatinine (CREA) of all patients were obtained and recorded. The physiological data (temperature, heart rate, blood pressure and respiratory rate) were made at the bedside from time-stamped, according to nurse-verified monitor records. We also collected the parameters needed to calculate the following scores: APACHE-II, the Modified Marshall Scoring System.

Patients were classified based on the revised Atlanta classification. SAP is classified based on the presence of persistent organ failure or local or systemic complications. The Modified Marshall Scoring System was used to assess organ failure admission.

Endpoint

The endpoint of the study was all-cause in-hospital mortality within 30 days.

Statistical analysis

Quantitative variables were presented as mean ± standard deviation and compared using independent samples t-test for normally distributed variables. For skewed distributions, the data are presented as the median (interquartile range) and compared using Mann-Whitney U nonparametric test. Categorical variables were described as counts and percentages and compared using the Chi-Square test as appropriate. The adjustment variables used in this model were those that achieved statistical significance at p < 0.1 univariate analysis or those considered to be clinically relevant. A bootstrapping technique was applied using 1000 random data sets (validation set) generated from the original data. The statistics were performed with IBM SPSS 21.0 statistical software (IBM SPSS Version 21.0. Armonk, NY). p value < 0.05 was regarded statistically significant. The sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) were calculated for best cutoff values.

Results

Clinical characteristics of the study subjects

A sum of 149 SAP patients were evaluated at enrollment who satisfied all inclusion criteria. Among them, 118 patients were stratified into survivor group and 31 patients died after 30-day follow-up which were nonsurvivor group. The demographic and clinical features of patients were illustrated in Table 1. The primary cohort was composed of 92 men (61.7%) and 57 women (38.3%) with SAP who were admitted to ICU. The total mortality rate was 20.8% (31/149). Compared with survivor group, nonsurvivor group were older (46 (19–86) vs.60 (24–86)) (p < 0.001). In comparison with survivor group, the levels of FIB in nonsurvivor group were apparently decreased (p < 0.001), while the level of ALT, AST, ALP, LDH, CK, DB, UREA, CREA, APACHE, Imire and Modified Marshall score in survivor group significantly increased (all p < 0.05). There was no difference in Gender, Systolic blood pressure, Diastolic blood pressure, Heart rate, Respiratory rate, Body temperature, Comorbidities, Etiology, DD2, LY, N, HB, HCT, PLT, TB, TC, TG, HDL, LDL, ALB, CRP and PCT between two groups (all p > 0.05) (Table 1).

Table 1.

Characteristics of patients.

Characteristics	Survivors (n = 118)	Nonsurvivors (n = 31)	p-value
Gender (male/female), n	74/44	18/13	0.638
Median age (range), y	46 (19–86)	60 (24–86)	0.001
Systolic blood pressure (mmHg)	129 ± 21	129 ± 30	0.875
Diastolic blood pressure (mmHg)	80 ± 15	82 ± 17	0.696
Heart rate	100 (56, 160)	88 (40, 162)	0.180
Respiratory rate (/min)	22 ± 7	22 ± 7	0.866
Body temperature (°C)	37.2 (36, 42)	37.1 (36, 39)	0.077
Comorbidities
Hypertension	39 (33.1%)	15 (48.4%)	0.115
Diabetes	21 (17.8%)	8 (25.8%)	0.319
Etiology of pancreatitis, n (%)			0.082
Alcoholic	11 (9.3%)	3 (9.7%)
Biliary	13 (11.0%)	6 (19.4%)
hypertriglyceridemia	24 (20.3%)	3 (9.7%)
Gallstones	32 (27.1%)	14 (45.2%)
Others	38 (32.2%)	5 (16.1%)
FIB (g/L)	5.12 ± 2.11	3.59 ± 1.96	<0.001
DD2 (mg/L)	6.28 ± 5.37	6.48 ± 4.98	0.855
LY (×109/L)	1.08 ± 0.64	1.23 ± 0.86	0.317
N (×109/L)	11.02 ± 5.97	11.37 ± 7.17	0.785
HB (g/L)	121.23 ± 29.78	124.38 ± 32.30	0.607
HCT	35.05 (21.0553.60)	37.70 (20.2, 56.3)	0.421
PLT (×109/L)	198.36 ± 90.85	176.65 ± 85.22	0.232
ALT (U/L)	28.5 (3.60–736.40)	55.9 (9.20–2516.30)	0.001
AST (U/L)	32.5 (8.30–2168.30)	262.8 (17.80–5907.00)	0.002
ALP (U/L)	125.24 ± 98.77	185.16 ± 165.88	0.011
LDH (U/L)	541.49 ± 334.91	723.32 ± 585.19	0.025
CK (U/L)	159 (4.00–14501.00)	278 (12.00–17159.00)	0.022
TB (μmol/L)	15.9 (4.8, 164.6)	21.8 (5.8, 396.5)	0.065
DB (μmol/L)	7.0 (0.7, 121.9)	11.7 (2.1, 269.0)	0.016
TC (mmol/L)	4.23 ± 3.03	3.44 ± 1.69	0.166
TG (mmol/L)	3.48 ± 3.59	2.29 ± 1.30	0.072
HDL (mmol/L)	0.72 ± 0.60	0.64 ± 0.36	0.454
LDL (mmol/L)	2.42 ± 1.22	2.37 ± 1.30	0.832
ALB (g/L)	32.77 ± 6.05	32.12 ± 7.48	0.614
UREA (mmol/L)	7.6 (0.4, 284.8)	13.3 (3.8, 30.1)	<0.001
CREA (μmol/L)	62.5 (23.1–745.4)	203.4 (48.60–954.20)	<0.001
CRP (mg/L)	90 (6.5, 90)	90 (3.8, 90)	0.491
PCT (ng/mL)	1.03 (0.06, 11.96)	5.69 (0.10, 75.54)	0.101
APACHE	15 (8, 26)	20 (13, 26)	<0.001
Imire	3 (1, 6)	4 (1, 7)	<0.001
Modified Marshall	2 (1, 9)	3 (1, 9)	<0.001

ALB: albumin; ALP: alkaline phosphatase; ALT: alanine aminotransferase; APACHE: acute Physiology and Chronic Health Evaluation; AST: aspartate aminotransferase; CK: creatine kinase; CREA: creatinine; CRP: c-reactive protein; DD2: D-Dimer; FIB: fibrinogen; HB: hemoglobin; HCT: hematocrit; HDL: high-density lipoprotein; LDH: lactate dehydrogenase; LDL: low-density lipoprotein; LY: lymphocytes; N: neutrophils; PCT: procalcitonin; PLT: platelets; TC: total cholesterol; TG: triglycerides; UREA: urea nitrogen.

Independent significant factors in the cohort

Variables in Table 1 with p < 0.1 were selected and transformed to categorical variable. Univariate analysis was first performed to selected variables with p < 0.1, the results presented that patients’ age, FIB, ALT, AST, ALP, LDH, CK, TB, DB, TG, UREA and CREA were associated with difference between the two groups (all p < 0.1, Table 2). Then, the above variables were included into multivariate logistic regression analysis, the results showed that the following five variables at admission were independently associated with increased in-hospital mortality: age: Hazard ratio (HR) = 3.225, 95% confidence interval (CI) 1.106 − 9.408, p = 0.032; AST: HR = 3.929, 95% CI 1.220–12.655, p = 0.022; ALP: HR = 5.317, 95% CI 1.506–18.772, p = 0.009; TG: HR =0.113, 95% CI 0.012–0.999; p = 0.049; CREA: HR = 8.694, 95% CI 2.665–28.361, p < 0.001 (Table 2).

Table 2.

Univariate and multivariate analyses in the study cohort.

Variables	Univariate analysis			Multivariate analysis
	HR	95% CI	p value	β	HR	95% CI	p value
Age (>53 y)	3.566	1.573–8.088	0.003	1.171	3.225	1.106–9.408	0.032
FIB (>4.3 g/L)	0.271	0.115–0.639	0.002
ALT (>92.4 U/L)	7.275	2.883–18.356	<0.001
AST (>86.5 U/L)	8.281	3.289–20.850	<0.001	1.368	3.929	1.220–12.655	0.022
ALP (>150.9 U/L)	3.872	1.674–8.959	0.002	1.671	5.317	1.506–18.772	0.009
LDH (>733 U/L)	2.687	1.161–6.217	0.035
CK (>42.7 U/L)	3.064	1.170–8.025	0.022
TB (>27.3 μmol/L	3.594	1.543–8.371	0.004
DB (>17.3 μmol/L)	4.959	2.025–12.146	0.001
TG (<4.69 mmol/L)	0.118	0.015–0.907	0.016	−2.179	0.113	0.012–0.999	0.049
UREA (>9.67 mmol/L)	5.568	2.335–13.276	<0.001
CREA (>79.4 μmol/L)	8.131	3.208–20.607	<0.001	2.163	8.694	2.665–28.361	<0.001
AUC				0.875

ALP: alkaline phosphatase; ALT: alanine aminotransferase; AST: aspartate aminotransferase; AUC: area under the cure; CI: confidence interval; CK: creatine kinase; CREA: creatinine; DB: direct bilirubin; FIB: fibrinogen; HR: hazard ratio; LDH: lactate dehydrogenase; TB: total bilirubin; TG: triglycerides; UREA: urea nitrogen; β: regression coefficient.

Establishment of clinical diagnosis regression model

After screening, the variables that entered the model included AST, ALP, CREA, age and TG. The prediction equation is built as follows:

$$\begin{array}{l}\text{Logit}\text{P}=1.171\times \text{Age}+1.368\times \text{AST}+1.671\times \text{ALP}+\\ 2.163\times \text{CREA-}2.179\times \text{TG}-7.887\end{array}$$

The cutoff point is 0.237.

Performance of the clinical model and independent markers

The ROC curve was usually applied to evaluate the predicted performance. An AUC > 0.7 was taken for useful, while an AUC between 0.8 and 0.9 suggested excellent predict accuracy.

Sensitivity and specificity were calculated to compare the prediction ability of the model. As shown in Figure 2, AUC of the model was 0.875 (95% CI 0.811–0.924). After internal validation, the C-index was 0.859 (95% CI: 0.786–0.932). Calibration curves for the clinical model was well. This model had a great differential predict efficiency.

Figure 2.

Performance of the clinical model for predicting 30 days mortality in SAP patients. (A) AUC for the clinical model; (B) AUC for the internal validation; (C) Calibration curves for the clinical mode.

In addition, the other statistics of the clinical model and independent markers were presented in Table 3.

Table 3.

Diagnostic statistics for independent markers.

Variable	AUC	Cutoff value	p value	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Age (year)	0.670 (0.588, 0.745)	53	0.006	58.06	72.03	35.3	86.7
AST (U/L)	0.729 (0.650, 0.798)	86.5	<0.001	48.39	89.83	55.6	86.9
ALP (U/L)	0.599 (0.516, 0.678)	196.3	0.112	45.16	79.66	36.8	84.7
TG (mmol/L)	0.541 (0.458, 0.623)	4.69	0.447	96.77	22.03	24.6	96.3
CREA (μmol/L)	0.779 (0.704, 0.843)	79.4	<0.001	77.42	70.34	40.7	92.2
CRP (mg/L)	0.511 (0.404, 0.618)	18.1	0.961	30	90	42.9	83.7
PCT (ng/mL)	0.680 (0.594, 0.758)	3.44	0.002	64.52	71.84	40.8	87.1
APACHE	0.747 (0.669, 0.814)	14	<0.001	93.55	44.07	30.5	96.3
Imire	0.727 (0.684, 0.796)	3	<0.001	74.19	64.41	35.4	90.5
Marshall	0.743 (0.665, 0.811)	1	<0.001	74.19	72.03	41.1	91.4
Clinical model	0.875 (0.811, 0.924)	0.237	<0.001	77.42	83.05	54.5	93.3

ALP: alkaline phosphatase; AST: aspartate aminotransferase; CREA: creatinine; CRP: C-reactive protein; TG: triglycerides; NPV: negative predictive value; PCT: Procalcitonin; PPV: positive predictive value; Youden’s Index: sensitivity + specificity-1.

Discussion

As the prevalence of AP increases, early, rapid, and effective screening of high-risk SAP patients poses a huge challenge to clinical treatment. Nearly 20%–30% AP patients will proceed into SAP, which is related to MODS, sepsis, and high mortality [10]. Therefore, to identify novel biomarkers to predict the severity and mortality with high accuracy of SAP is imperative. The aim of our model development is to transform patients with high mortality into low-risk patients through early identification and reasonable treatment.

Compared with imaging examination, biochemical parameters such as AST, ALP, TG and CREA can be faster and more efficiently detected from blood samples by the laboratory facility at low cost and are the routine inspection items in clinical practice. Our analysis of the collected SAP cases suggests that there are five indicators that are associated with the occurrence of SAP mortality. It is generally realized that age is a vital and useful indicator of adverse outcomes in a variety of entities. In addition, several universally acknowledged predictive models have incorporated age for predicting the severity or mortality of AP in the field of clinical medicine [11]. Our results as well demonstrate that age is an independent risk factor for in-hospital mortality in SAP patients.

AST is a nonspecific enzyme presented in many tissues such as mitochondria of human cardiomyocytes and hepatocytes, catalyzing the reversible reaction of amination reversible reaction of transamination [12]. The liver is a frequent major site of extra-pancreatic organ damaged during AP, and liver injury can deteriorate AP. A report has shown that liver injury occurs in about 80% of AP patients, and its severity is positively responsible for the severity of pancreatitis and prolongs pancreatitis [13]. In this study, serum AST in nonsurvivors was apparently higher than those in the survivors’ group, which is consisted with previous studies. In China, gallstones are universally been regarded as the commonest etiology of AP. ALP is an important indicator in biliary pancreatitis and can increase obstructive jaundice at the same time. The increase of ALP may be a signal of the obstruction in the ducts which causes biliary pancreatitis [14]. Most of it is produced by bone cells, and a small part comes from liver and is discharged into the intestine through bile. ALP is the most frequently used in biochemical factor linked with gallstone. Elevated serum levels of CREA are normal in the early period of AP [15]. Systemic inflammatory response and high catabolic rate lead to a significant increase in creatinine [16]. At the same time, systemic capillary leakage and reduction of effective circulating blood volume could activate the sympathetic nervous system, resulting in renal arterioconstriction, subsequently, the glomerular filtration rate is reduced and finally deterioration of renal function [17,18]. Jiang et al. indicated that CREA acts as an enduring index for persistent organ failure closest to 1-year mortality [6]. Yang et al. reported that CREA level worked as a more appropriate predictor for SAP [19]. Consistent with some previous studies, our study as well confirmed that the higher CREA is related to higher mortality in SAP. Elevated TG levels are a well-established risk factor for AP [20]. Isabel Pascual et al. [21] reported that TG levels are positively associated with any organ failure, persistent renal failure, persistent shock, persistent MOF, necrosis, development of acute collections, ICU admission and mortality. Nawaz et al. [22] suggested that elevated serum TG levels are independently associated with POF in acute pancreatitis. Our study also confirmed this.

In this study, we found that age, ALT, AST, ALP, LDH, CK and CREA in nonsurvivors were remarkably higher than that in survivors. Multivariate analysis shows that age, AST, ALP, TG and CREA were independent risk factors. Then a model was developed. Importantly, this diagnostic model may help simplify clinical management by avoiding over checking in patients which increases the financial burden of patients. All model data are readily obtained after admission. When the patients’ etiological analyses are not clear or clinical characteristics are not obvious, the model can help physicians make a prediction based on readily available clinical data.

However, there are also several inevitable limitations in our model. For example, there may be data bias since this is a single-center study with a limited sample size. Thus, this result also needs to be verified using larger patient queues and multicenter data. On the other hand, no analysis of proinflammation cytokines such as IL-6 had been performed because of a lack of data. In retrospective studies, experimental design and statistical analysis cannot always completely control for confounders. The observation of our study was set to the first 48 h of admission. Thus we could not infer a long-term association between the laboratory parameters above and AP severity and mortality. Future clinical work will further optimize and verify the model.

Conclusions

In summary, our study develops a useful and refined model with easily obtainable serological markers. The model highlights the need for more careful observation of SAP patients to ensure timely and sufficient medical intervention which can prevent disease progression.

Ethical statement

Ethical approval was obtained from the Ethical Committee of the First Affiliated Hospital of Nanjing Medical University. Due to the retrospective design of the study, a waiver of participant informed consent was granted by the Ethical Committee of the First Affiliated Hospital of Nanjing Medical University (2023-SR-043).

Funding Statement

This work was supported by grants from Jiangsu Provincial Research Hospital (YJXYY202201) and Jiangsu Provincial Medical Key Discipline (Laboratory) (ZDXK202239).

Author contributions

Conceptualization, Jun Zhou; Formal analysis, Qing Li, Zhenzhen Cai and Jun Zhou; Investigation, Ying Chen, Qing Li and Liang Ma; Methodology, Qing Li and Zhenzhen Cai; Resources, Ying Chen and Liang Ma; Writing—original draft, Ying Chen; Writing—review & editing, Qing Li, Liang Ma, Zhenzhen Cai and Jun Zhou. All the authors have read the manuscript and approved the final manuscript.

Disclosure statement

No potential conflict of interest was reported by the authors.

Data availability statement

The datasets are available from the corresponding author on reasonable request.