A simple new scoring system for predicting the mortality of severe acute pancreatitis

Supplemental Digital Content is available in the text

Abstract

It is critical to accurately identify patients with severe acute pancreatitis (SAP) in a timely manner. This study aimed to develop a new simplified AP scoring system based on data from Chinese population.

We retrospectively analyzed a consecutive series of 585 patients diagnosed with SAP at the Changhai hospital between 2009 and 2017. The new Chinese simple scoring system (CSSS) was derived using logistic regression analysis and was validated in comparison to 4 existing systems using receiver operating characteristic curves.

Six variables were selected for incorporation into CSSS, including serum creatinine, blood glucose, lactate dehydrogenase, heart rate, C-reactive protein, and extent of pancreatic necrosis. The new CSSS yields a maximum total score of 9 points. The cut-offs for predicting mortality and severity (discriminating moderately SAP from SAP) were set as 6 points and 4 points respectively. Compared with 4 existing scoring systems, the area under the receiver operating characteristic of CSSS for prediction of mortality was 0.838, similar to acute physiology and chronic health evaluation II (0.844) and higher than Ranson's score (0.702, P < .001), bedside index of severity in acute pancreatitis (0.615), and modified computed tomography severity index (MCTSI) (0.736). For predicting SAP severity, CSSS was the most accurate (0.834), followed by acute physiology and chronic health evaluation II (0.800), Ranson's score (0.702), MCTSI (0.660), and bedside index of severity in acute pancreatitis (0.570). Further, the accuracy of predicting pancreatic infection with CSSS was the highest (0.634), similar to that of MCTSI (0.641).

A new prognostic scoring system for SAP was derived and validated in a Chinese sample. This scoring system is a simple and accurate method for prediction of mortality.

1. Introduction

Acute pancreatitis (AP) is a common inflammatory condition of the pancreas characterized by pancreatic enzyme activation.^[1] The global incidence of AP each year is 34 cases per 100,000 people,^[2] while the incidence in China was 39.2 cases per 100,000 in 2014.^[3] According to the Atlanta International Consensus in 2012, AP is clinically categorized into 3 types: mild acute pancreatitis, moderately severe (MSAP), and severe acute pancreatitis (SAP).^[4] Most mild acute pancreatitis patients are self-limited and have a good outcome. Those with SAP have pancreatic necrosis and/or distant organ failure; the mortality of SAP can reach up to 30%.^[5] According to Dellinger's classification system, the presence of both pancreatic necrosis and organ failure is associated with the highest mortality.^[6] It is widely accepted that early intervention and intensive care can decrease the mortality of SAP. Therefore, it is important to predict the severity and mortality of AP patients at an early stage. When SAP is considered, patients should be immediately transferred to an intensive care unit for early intervention and to ensure the maintenance of organ function.

Due to the variable and urgent presentation of SAP patients, it is difficult to make a correct clinical decision in a timely manner. Currently, there are four frequently used AP scoring systems for early identification of SAP, including the Acute Physiology and Chronic Health Evaluation (APACHE-II), Ranson's score, the Bedside Index of Severity in Acute Pancreatitis (BISAP), and the modified computed tomography (CT) severity index (MCTSI).^[7,8] The APACHE II score is initially applied in the dynamic evaluation of critical patients; it comprises 3 parts: acute physiology, age, and chronic health evaluation.^[9] APACHE II is widely used in the assessment of the severity of AP and has high predictive accuracy. However, the APACHE II measure is complicated and inconvenient to use, cannot distinguish infectious pancreatic necrosis from non-infectious lesions, and has poor predictive value within 24 hours of disease onset. Ranson's score was 1 of the earliest scoring systems to evaluate the severity of AP. It comprises 11 indicators that must be evaluated at admission and 48 hours after admission, respectively.^[10] However, some of these indicators are not collected routinely in the early stage of AP, making early prediction difficult. The BISAP score was proposed in 2008 and comprises 5 indicators to predict the mortality of AP within 24 hours of admission.^[11] It is easy to use but has low sensitivity for prediction of SAP (65%).^[12] In 2004, the MCTSI was proposed based on the CT severity index. It includes assessments of pancreatic inflammation and the area of pancreatic necrosis, as well as extra-pancreatic complications on the initial CT scans; these measures correlate closely with outcome measures of AP patients.^[13,14] However, peri-pancreatic necrosis and pseudocysts are not usually present in the early stage of AP. Thus, the MCTSI score must be evaluated 2 to 3 days after admission; this may delay the early diagnosis of SAP.^[15]

Although several scoring systems have been developed, each system has its specific applications and limitations. It is of clinical significance to develop a new and effective scoring system to predict the severity and mortality of SAP. This study aimed to construct a new AP scoring system based on the analysis of Chinese patients with SAP.

2. Methods

2.1. Patient enrollment

We conducted a retrospective cross-sectional study on a series of 585 patients diagnosed with MSAP or SAP who were admitted to the Pancreatic Intensive Care Unit at Changhai Hospital between July 2009 and August 2017. All patients were diagnosed with AP for the first time, and the possibility of other pancreatic diseases (recurrent AP, chronic pancreatitis, or pancreatic cancer) was excluded. After admission, all patients received routine management, including vital sign monitoring, fasting, antibiotics, acid suppression, inhibition of pancreas secretion, and enteral nutrition support. The following data were recorded: basic demographics, causes of the disease, vital signs (heart rate, blood pressure, respiration, blood oxygen saturation, and breathing), laboratory tests within 24 or 48 hours (blood count, blood gas analysis, blood biochemical examination, procalcitonin), and pancreatic examinations under CT or magnetic resonance imaging (MRI). The study was approved by the Ethics Committee of Shanghai Changhai Hospital. Written informed consents of all patients were obtained.

2.2. Diagnosis and existing scoring systems

The diagnostic and classification criteria for AP are based on the 2012 revision of the Atlanta classification.^[4] For a diagnosis of AP, 2 of the following must be met:

• (1)abdominal pain suggestive of acute pancreatitis;
• (2)serum amylase or lipase activity at least 3 times greater than the upper limit of normal; and
• (3)characteristic findings of AP on contrast-enhanced CT or MRI. MSAP and SAP patients should first meet the AP diagnostic criteria.

Then, MSAP can be diagnosed if the patient presents with transient organ failure resolving within 48 hours and/or local or systemic complications without persistent organ failure.^[4] Local complications include acute peripancreatic fluid collection, acute necrotic collections, pancreatic pseudocysts, and walled-off necrosis. Systemic complications are defined as exacerbations of pre-existing co-morbidities, such as chronic lung disease or coronary artery disease, which are caused by AP. SAP is characterized by persistent single or multiple organ failure for more than 48 hours.

In order to validate the new scoring system, 4 main scoring systems for AP were compared to the new scoring system. APACHE II and BISAP scores were determined through laboratory tests and radiological examinations within the first 24 hours of admission. Ranson score was calculated based on the results of laboratory tests within the first 48 hours of admission. The MCTSI score was calculated based on the results of CT scans performed within 72 hours of admission. The reporting of each CT scan was performed by the same set of radiologists in our hospital. For calculation of the new scoring system, laboratory and radiological test data were collected within the first 48 hours of admission. The extent of pancreatic necrosis was assessed by CT scans or MRI.

2.3. Model derivation and statistical methods

The statistical analyses were performed using SPSS (v19.0) software. Continuous variables are presented as means and standard deviations. Single variable 1-way logistic regression analyses were used to determine the associations between patient mortality and each variable. A variable was selected into the new scoring system if it was associated with patient mortality in the logistic regression analysis at a conservative threshold of 10%. To validate the efficiency and accuracy of the new scoring system, comparisons to existing scoring systems were performed using the area under the receiver operating characteristic (ROC) curve (AUC) in the same cohort. Youden's index was used to evaluate the performance of the new scoring system, which combines sensitivity and specificity into a single measure (Sensitivity + Specificity - 1). Further, sensitivity and specificity values with 95% confidence intervals (CIs) were calculated. A 2-sided P value less than .05 was taken to indicate a statistically significant difference.

3. Results

3.1. Basic characteristics of the patients

The demographic and clinical characteristics of the enrolled patients are summarized in Table 1. A total of 585 patients (496 cases with MSAP and 89 with SAP) were admitted to our hospital, including 372 males and 213 females. The average age of the patients was 53.05 ± 14.97 years. In terms of the etiology of SAP, cholelithiasis was the most common cause (274 cases; 46.8%), followed by alcohol in 134 cases (22.9%), and hyperlipemia in 93 cases (15.9%), while there were 70 (12.0%) endoscopic retrograde cholangiopancreatography-related cases, and 15 cases (2.6%) with other causes. Among the sample, 45 patients presented with infectious pancreatic necrosis (7.7%). A total of 31 (5.3%) patients died during hospitalization. The average length of hospitalization among all patients was 19.95 ± 20.43 days.

Table 1.

Demographic and clinical characteristics of patients (n = 585).

3.2. Derivation of the new scoring system

The single-variable logistic regression analyses of all clinical common data indicated that 6 variables were significantly associated with patient death during hospitalization; these variables included: serum creatinine (P < .001), blood glucose (P < .001), LDH (P = .004), C-reactive protein (P = .098), heart rate (P < .001), and the extent of pancreatic necrosis (P < .001). Aside from the extent of pancreatic necrosis, the thresholds for the remaining 5 variables were obtained from the maximum point of the Youden's index on the ROC curve. To facilitate the application of the new scoring system, we simply took the integer portion of the output for each variable (Table 2). Based on clinical experience and the MCTSI criterion, the extent of pancreatic necrosis was divided into 4 categories (<30%, 30%–50%, 50%–70%, and >70%) and corresponding scores (1 to 4 points) were assigned. Dichotomous scores (0 and 1 point) for the remaining 5 variables were assigned based on the optimal threshold (Table 2).

Table 2.

Variables included in the new scoring system and their assigned scores.

The new scoring system yields a total maximum score of 9 points, which is derived from the sum of the scores of each variable. Based on the calculated highest sensitivity and specificity values from the ROC curves, the determined cut-off values for predicting SAP in-hospital mortality and severity using the new scoring system were 6 points (the Youden's index is 0.56) and 4 points (the Youden's index is 0.53), respectively (Table 3).

Table 3.

The optimal threshold values for determining in-hospital mortality and severity of SAP according to the new scoring system.

3.3. Comparison between the new scoring system and existing systems

To evaluate the performance of the new scoring system, the new system was compared with 4 existing scoring systems for predicting disease mortality and severity (Table 4). The calculated AUC of the new scoring system for the prediction of disease mortality was 0.838 (95% CI 0.833 to 0.843) (Fig. 1), similar to APACHE II (0.844), and higher than Ranson's score (0.702, P < .01), BISAP (0.615, P < .01), and MCTSI (0.736, P < .01) (Fig. 2). The new scoring system was also the most accurate for predicting disease severity (AUC 0.834) (Fig. 1), followed by APACHE II (0.800, P < .01), Ranson's score (0.702, P < .01), MCTSI (0.660, P < .01), and BISAP (0.570, P < .01) (Fig. 3).

Table 4.

Comparison between the new scoring system and 4 existing scoring systems for accuracy of prediction of mortality and severity in SAP.

Figure 1.

ROC curves for in-hospital mortality and severity of disease (moderately SAP and SAP) using the new scoring system. ROC = receiver operating characteristic, SAP = severe acute pancreatitis.

Figure 2.

ROC curves for in-hospital mortality using existing scoring systems. A) Apache II; B) Ranson's score; C) BISAP; and D) MCTSI. BISAP = bedside index of severity in acute pancreatitis, MCTSI = modified computed tomography severity index, ROC = receiver operating characteristic.

Figure 3.

ROC curves for severity of disease (moderately SAP and SAP) using existing scoring systems. A) APACHE II; B) Ranson score; C) BISAP; and D) MCTSI. APACHE-II = acute physiology and chronic health evaluation, BISAP = bedside index of severity in acute pancreatitis, MCTSI = modified computed tomography severity index, ROC = receiver operating characteristic.

Finally, the accuracy of prediction of pancreatic infection was also evaluated (Supplementary Table 1, Available at). Based on the AUC, MCTSI was the most accurate scoring system (0.641), and yielded a similar AUC to the new scoring system (0.634), followed by Apache II (0.575, P < .01), Ranson's score (0.551, P < .01), and BISAP (0.551, P < .01).

4. Discussion

AP is a relatively common condition worldwide characterized by acute and severe upper abdominal pain. It is critical to accurately identify patients with SAP in a timely manner. Generally, it is difficult to clinically evaluate the severity and mortality of AP. Currently, there are 4 frequently used AP scoring systems to help clinicians identify SAP; these are APACHE-II, Ranson score, BISAP, and MCTSI. Each scoring system has specific applications and advantages, but each also has limitations.

Herein, we developed a new Chinese Simple Scoring System (CSSS) that comprises only 6 variables: serum creatinine, blood glucose, lactate dehydrogenase, C-reactive protein, heart rate, and extent of pancreatic necrosis; the data for these variables are collected within 48 hours of admission. The first 4 variables are informed by routine laboratory tests. Heart rate is a vital sign and is regularly measured during an admission. Finally, the extent of pancreatic necrosis is evaluated by CT scans or MRI. Using these variables, within 48 hours of admission we were able to stratify patients into risk groups for SAP presence, in-hospital mortality, and pancreatic infection.

The advantages of the CSSS include its simplicity and objectivity. The derived algorithm is very simple to use. The maximal total score is only 9 points. Scores for the first 5 variables are dichotomous (0 and 1 point) and can easily and rapidly be obtained over the course of hospitalization. Harnessing the advantages of the MCTSI, the CSSS contains an assessment of pancreatic infection under CT scans; the results are divided into 4 categories with corresponding scores (1 to 4 points). All variables are objective clinical measures. In contrast to APACHE II and BISAP, this system does not include assessment of mental status.

Compared to the 4 existing scoring systems, the performance of CSSS was good for the prediction of SAP severity and mortality. The calculated AUC of the CSSS for prediction of disease mortality was 0.838, similar to the AUC of APACHE II, and higher than that of Ranson score, BISAP, and MCTSI. The sensitivity of APACHEII for the prediction of mortality ranges between 63% and 82%.^[16] If the total score is less than 8 points, the mortality rate can be less than 4%, while if the total score is greater than 8 points, the mortality rate can range between 11% and 18%.^[16] After combining APACHEII with Ranson score, the sensitivity increases to 75% to 89%.^[17] The AUC for prediction of mortality with the BISAP score is reported to be 0.77 (95% CI: 0.73–0.80) according to a meta-analysis.^[12] AUC for BISAP in predicting SAP are 0.81 (95% CI: 0.74–0.87).^[18] In this study, the new scoring system was the most accurate in predicting disease severity according to the AUC, followed by APACHE II, Ranson score, MCTSI, and BISAP. The overall sensitivity and specificity for prediction of disease severity with Ranson score was 0.75 and 0.77, respectively, according to a meta-analysis.^[19] Our scoring system has similar sensitivity (0.669) and slightly higher specificity (0.851). However, Ranson score has twice the number of indicators compared to the CSSS, and requires 2 rounds of evaluations, making it difficult to use in the emergency department. For the prediction of both in-hospital mortality and disease severity, CSSS was found to perform better than most of the existing scoring systems and was comparable to the widely-used APACHE II. Further, for evaluating pancreatic infection, the accuracy of CSSS was similar to MCTSI, and higher than APACHE II, Ranson score, and BISAP. It is widely accepted that the MCTSI can effectively identify local lesions in AP and has a high predictive value for local complications such as pancreatic cysts and abscesses.^[20] Thus, CSSS harnesses the advantages of MCTSI, allowing for the accurate prediction of pancreatic infection.

There are several potential limitations of this study that should be noted. First, this was a single-center retrospective study, and although the data set was large, it did contain missing data. Second, we validate the performance of this new scoring system in ICU, rather than in general wards, which may lead overestimation of the effectiveness of this system. Third, we collected limited information regarding etiology and mental status, each of which may have prognostic value in SAP. Finally, this system includes CT or MRI evaluations that reveal pancreatic necrosis within 48 hours to 72 hours after onset of symptoms, which might limit its applications in clinical practice.

In summary, we have derived a new scoring system for predicting severity and mortality of SAP based on data from Chinese patients. All 6 variables included in this scoring system can be easily measured within 48 hours of admission. Compared to 4 existing scoring systems, the CSSS is accurate in predicting disease severity (moderately SAP and SAP), in-hospital mortality, and pancreatic infection.

Acknowledgments

This work was supported by the China-Europe International Business School (CEIBS) Healthcare Research Fund and the National Natural Science Foundation of China (Grant No. 81873588 [Z.S.L.]).

Author contributions

LW, YBZ and JYC LW, YBZ and JYC performed the data collection and statistical analysis, with substantial contributions from QL, RW, RZ, DZ, YHD and WBZ. LW and QX wrote the manuscript. YQD and ZSL revised the manuscript. ZSL and QX obtained the funding. All authors contributed to data interpretation and revision of the manuscript and approved the final manuscript. the data collection and statistical analysis, with substantial contributions from QL, RW, RZ, DZ, YHD and WBZ. LW and QX wrote the manuscript. YQD and ZSL revised the manuscript. ZSL and QX obtained the funding. All authors contributed to data interpretation and revision of the manuscript and approved the final manuscript.