Development and validation of a nomogram for predicting survival in patients with acute pancreatitis

Abstract

BACKGROUND:

Acute pancreatitis (AP) is a complex and heterogeneous disease. We aimed to design and validate a prognostic nomogram for improving the prediction of short-term survival in patients with AP.

METHODS:

The clinical data of 632 patients with AP were obtained from the Medical Information Mart for Intensive Care (MIMIC)-IV database. The nomogram for the prediction of 30-day, 60-day and 90-day survival was developed by incorporating the risk factors identified by multivariate Cox analyses.

RESULTS:

Multivariate Cox proportional hazard model analysis showed that age (hazard ratio [HR]=1.06, 95% confidence interval [95% CI] 1.03–1.08, P<0.001), white blood cell count (HR=1.03, 95% CI 1.00–1.06, P=0.046), systolic blood pressure (HR=0.99, 95% CI 0.97–1.00, P=0.015), serum lactate level (HR=1.10, 95% CI 1.01–1.20, P=0.023), and Simplified Acute Physiology Score II (HR=1.04, 95% CI 1.02–1.06, P<0.001) were independent predictors of 90-day mortality in patients with AP. A prognostic nomogram model for 30-day, 60-day, and 90-day survival based on these variables was built. Receiver operating characteristic (ROC) curve analysis demonstrated that the nomogram had good accuracy for predicting 30-day, 60-day, and 90-day survival (area under the ROC curve: 0.796, 0.812, and 0.854, respectively; bootstrap-corrected C-index value: 0.782, 0.799, and 0.846, respectively).

CONCLUSION:

The nomogram-based prognostic model was able to accurately predict 30-day, 60-day, and 90-day survival outcomes and thus may be of value for risk stratification and clinical decision-making for critically ill patients with AP.

INTRODUCTION

Acute pancreatitis (AP) is a relatively prevalent disease that results in localized pancreatic injury, a systemic inflammatory response, and, in severe cases, organ failure.[1-3] Mortality rates for patients with AP vary with disease severity and can range from 2% to 20%.[4,5] Despite improvements in treatment options for AP in recent years, certain patients are at an elevated risk of poor outcomes.[6] Stratifying patients with AP based on their risk level is essential for clinical decision-making and treatments.[7,8] AP is a complex and heterogeneous disease, so it remains difficult to predict patient outcomes.

Several models have been developed for predicting outcomes in patients with AP, but these individual systems exhibit substantial drawbacks. Some systems employ digital interfaces that are user-friendly and accurate, enabling the calculation of nomogram-derived results that can guide treatments. Nomograms are graphical models that provide individualized risk estimation. These models are developed using regression analyses and are commonly used to evaluate prognostic outcomes associated with a range of illnesses, as they are simple, intuitive, and practical.[9-11] These tools can readily yield predictive outcomes without the need for complex calculation procedures.

The present study aimed to design and validate a nomogram capable of estimating the odds of survival in patients with AP.

METHODS

Database

The present study utilized the Medical Information Mart for Intensive Care (MIMIC)-IV Critical Care Database, which is a publicly available resource that compiles data pertaining to patients admitted to the Beth Israel Deaconess Medical Center intensive care unit (ICU) from 2001–2012.[12] The right of this database was approved by the Massachusetts Institute of Technology (Cambridge, MA) and Beth Israel Deaconess Medical Center (Boston, MA) and consent was obtained for the original data collection. Patients’ information in the MIMIC-IV database was anonymized; therefore, informed consent was not required. The present study was conducted as per the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis statement guidelines.[13]

Patient selection

Patients eligible for inclusion in this study were those diagnosed with AP as per the International Classification of Diseases, Ninth Revision code (ICD-9, code=5770) who were >18 years old and had been admitted to the ICU for >24 h. If multiple admissions were recorded for a single patient, only the data pertaining to the first ICU admission were analyzed. Patients were excluded if they suffered from chronic liver disease, malignant tumors, acquired immune deficiency syndrome (AIDS), hemolytic anemia, or end-stage kidney disease or if ≥20% of their data were missing from the database.

Data extraction

Structure query language (SQL) was used to extract raw data with DataGrip (v 2021.2.1) followed by further processing using R (v 4.1.1, R Foundation for Statistical Computing, Austria), which was also used to retrieve patient information from the database. All baseline data within 24 h after admission were collected. Data analyzed for this study included: (1) basic demographic characteristics, such as age, sex, ethnicity, and weight; (2) mean values for vital signs within 24 h following admission including temperature, heart rate, respiratory rate, systolic blood pressure (SBP), and diastolic blood pressure (DBP); (3) laboratory test results, including serum levels of creatinine, albumin, bilirubin, calcium, potassium, blood urea nitrogen (BUN), lactate, hemoglobin level, platelet count, and white blood cell (WBC) count; (4) Glasgow Coma Scale (GCS) scores; and (5) Simplified Acute Physiology Score II (SAPS-II), which was used as a measure of pancreatitis severity.

Study outcome

The primary outcome for this study was 90-day survival. The secondary outcomes were 30- and 60-day survival.

Management of missing data

Missing values are common in the MIMIC-IV database. When a given variable exhibited missing data for <20% of the included patients, multiple imputation was used to fill missing values with predictors to minimize bias.[14]

Statistical analysis

Continuous data are described as the mean±standard deviation (SD) or median (interquartile range [IQR]) as appropriate and were compared with Student’s t-test or rank-sum test. Categorical variables are presented as numbers (percentages) and were compared using the Chi-square test. The Shapiro-Wilk test was used to assess the normality of data distributions. Non-normally distributed data or data exhibiting heterogeneity of variance were compared via Kruskal-Wallis or Mann-Whitney U-tests. Cox regression models were used to identify independent predictors of 30-day, 60-day, and 90-day mortality, and hazard ratios (HRs) and 95% confidence intervals (95% CIs) were calculated. Receiver operating characteristic (ROC) curve analyses were used to evaluate the predictive utility of the developed nomogram. Bootstrap resampling-based internal validation test was utilized to evaluate the accuracy of the nomogram. A P-value <0.05 was the significance threshold, and all statistical analyses were performed using R (v 4.1.1).

RESULTS

Baseline characteristics of the study population

In total, 632 patient records that met our inclusion criteria were identified within the MIMIC-IV database and extracted for analysis. analysis. A total of 75 patients died within 90 days in the hospital. The following baseline characteristics differed significantly between survivors and non-survivors (all P<0.05): age, sex, ethnicity, temperature, WBC, serum levels of albumin, creatinine, BUN and lactate, SBP, DBP, and SAPS-II (supplementary Table 1).

Identification of prognostic factors

Univariate Cox proportional hazards regression model analysis showed that age, WBC, creatinine, BUN, SBP, DBP, lactate, and SAPS-II were potential predictors of 90-day mortality in patients with AP (all P<0.001). Multivariate Cox proportional hazard model analysis showed that age, WBC, SBP, serum lactate, and SAPS-II were independent predictors of 90-day mortality in patients with AP (Table 1).

Table 1.

Multivariate analysis of predictors of 90-day mortality in patients with acute pancreatitis

Development of the prognostic nomogram

The five prognostic variables identified above were next used to establish a prognostic nomogram for estimating the probability of 30-day, 60-day and 90-day survival in patients with AP. For this nomogram, a weighted score was assigned to each of these predictors, with the highest possible score being 240 points, and the survival probability scale ranged from 0.10 to 0.99. In this model, a higher total score derived by summing the five individual predictor scores was associated with lower odds of surviving for 30 days, 60 days or 90 days (Figure 1).

Figure 1.

A weighted score assigned to each of the predictors for the prognostic nomogram.

Evaluation of the performance of the prognostic nomogram

The predictive utility of the developed nomogram was assessed using ROC curve analysis. The area under the ROC curve (AUC) values for 30-day, 60-day, and 90-day survival were calculated to be 0.796 (95% CI 0.732–0.859), 0.812 (95% CI 0.755–0.869), and 0.854 (95% CI 0.805–0.903), respectively (supplementary Figures 1 A to C). When the nomogram was subjected to 1,000 bootstrap resampling-based internal validation, it was found to exhibit consistently good accuracy as a tool for predicting 30-day, 60-day, and 90-day survival odds in patients with AP, yielding bootstrap-corrected C-indexes of 0.782, 0.799, and 0.846, respectively (supplementary Figures 1 D to F). The calibration curves suggested that the model was acceptably calibrated, revealing good correlations between the odds of survival derived from this nomogram and those from estimates.

DISCUSSION

The present analysis leveraged clinical and survival-related data from 632 patients with AP included within the MIMIC-IV database to develop a model capable of predicting survival in patients with AP. Multivariate analysis identified five predictors of 90-day mortality among these patients with AP, namely, age, WBC, SBP, lactate, and SAPS-II. A predictive nomogram incorporating these five variables was then established and found to exhibit satisfactory performance in the prediction of 30-day, 60-day and 90-day survival according to AUC and calibration curve analyses. It is suggested that the nomogram might be useful in the clinical setting.

As a severe inflammatory condition with a rapid and heterogeneous clinical course, AP remains a major threat to the patients and imposes significant economic and health burdens on affected individuals.[1,2] A majority of patients with AP exhibit mild disease that resolves without any long-term complications, but an estimated 20% of patients will develop moderate or severe disease characterized by peripancreatic or pancreatic necrosis, organ failure, and mortality rates ranging from 20% to 40%.[3,4] These high-risk patients may be more likely to benefit from intensive monitoring for organ failure, aggressive fluid resuscitation efforts, appropriate antibiotic treatment, and other therapeutic interventions, including radiologic treatment or endoscopic sphincterotomy.[5] The ability to accurately evaluate AP severity during the early stage of the disease is thus essential as a means of facilitating timely intervention aimed at improving patient prognosis. Several scoring systems have been used to assess AP severity, including the Ranson score, Acute Physiology and Chronic Health Evaluation (APACHE)-score, Balthazar computed tomography severity index (CTSI), and Bedside Index for Severity in Acute Pancreatitis (BISAP) score.[15-17] However, these individual systems exhibit specific drawbacks and limitations, and no scoring system can currently achieve maximal sensitivity and specificity. The development of additional novel prognostic models is essential to further improve predictive accuracy for various clinical populations.

Nomograms can be readily employed to evaluate the odds of a given clinical outcome for an individual patient, leading to their increasingly frequent use as prognostic tools in oncological contexts.[9-11] In this study, we sought to develop a nomogram capable of quickly identifying high-risk patients with AP in an ICU setting. The developed nomogram consisted of five variables that can all be readily measured within 24 h following admission, and it was able to predict the odds of short-term survival for this patient population. This model exhibited moderate predictive utility, and the model-predicted odds of 30-day, 60-day and 90-day mortality were similar to the observed survival rates in this patient cohort.

Our final prognostic nomogram incorporated five predictors of AP patient survival. Age is well known as a primary predictor of risk among patients with AP.[3,18] An increase in WBCs may occur due to the inflammation associated with AP or due to pancreatic infection (a complication of AP). WBC count is significantly higher in patients with severe pancreatitis than in those with mild pancreatitis,[19] and WBC count is a predictor of mortality in patients with AP.[20] A low SBP is an indicator of organ failure secondary to systemic inflammation (i.e., more severe pancreatitis), and hypotension during the first week of AP is associated with an increased risk of infected pancreatic necrosis.[21] An elevated level of serum lactate is indicative of inadequate organ perfusion, and lactate level was reported to be a predictor of mortality in patients with AP,[22] including those with intraabdominal hypertension.[23] SAPS-II is an alternative version of the APACHE scale that is frequently used in the ICU setting.[24] Notably, SAPS-II is a predictor of mortality in patients with AP.[25] As such, all five of the predictors incorporated into our developed nomogram represent credible mortality-related risk factors worthy of consideration in clinical research.

There are several limitations to this study. First, several mortality risk factors that have been previously reported, including serum lipid level and serum calcium level, were not included as a means of reducing potential bias stemming from missing data. Second, we were unable to assess imaging findings, which potentially reduced the overall nomogram efficacy. Third, the nomogram model was established using a single dataset without a validation set, which will necessitate further large-scale studies aimed at validating its clinical utility. While internal bootstrap resampling validation was performed in this study, larger external cohorts will be critical to bolstering the overall efficacy of this predictive tool in the future. Fourth, given that these data came from a US population, whether these results can be generalized to non-US populations remains to be validated. In addition, other potentially relevant information, such as the ICU admission criteria, was not available from the database and hence could not be considered during the analysis.

CONCLUSIONS

We herein developed a bootstrap-resampling validated nomogram capable of predicting 30-day, 60-day and 90-day mortality rates in patients with AP who are critically ill and have been admitted to the ICU. This prognostic nomogram might be able to predict patient outcomes and thus may be used as a tool for risk stratification and clinical decision-making for patients with AP. However, external validation by an independent, prospective study with a large sample size will be vital to improve the reliability of this model.