A comparison of APACHE II, BISAP, Ranson’s score and modified CTSI in predicting the severity of acute pancreatitis based on the 2012 revised Atlanta Classification

Abstract

Objective

Our aim was to prospectively compare the Accuracy of Acute Physiology and Chronic Health Evaluation (APACHE) II, Bedside Index of Severity in Acute Pancreatitis (BISAP), Ranson’s score and modified Computed Tomography Severity Index (CTSI) in predicting the severity of acute pancreatitis based on Atlanta 2012 definitions in a tertiary care hospital in northern India.

Methods

Fifty patients with acute pancreatitis admitted to our hospital during the period of March 2015 to September 2016 were included in the study. APACHE II, BISAP and Ranson’s score were calculated for all the cases. Modified CTSI was also determined based on a pancreatic protocol contrast enhanced computerized tomography (CT). Optimal cut-offs for these scoring systems and the area under the curve (AUC) were evaluated based on the receiver operating characteristics (ROC) curve and these scoring systems were compared prospectively.

Results

Of the 50 cases, 14 were graded as severe acute pancreatitis. Pancreatic necrosis was present in 15 patients, while 14 developed persistent organ failure and 14 needed intensive care unit (ICU) admission. The AUC for modified CTSI was consistently the highest for predicting severe acute pancreatitis (0.919), pancreatic necrosis (0.993), organ failure (0.893) and ICU admission (0.993). APACHE II was the second most accurate in predicting severe acute pancreatitis (AUC 0.834) and organ failure (0.831). APACHE II had a high sensitivity for predicting pancreatic necrosis (93.33%), organ failure (92.86%) and ICU admission (92.31%), and also had a high negative predictive value for predicting pancreatic necrosis (96.15%), organ failure (96.15%) and ICU admission (95.83%).

Conclusion

APACHE II is a useful prognostic scoring system for predicting the severity of acute pancreatitis and can be a crucial aid in determining the group of patients that have a high chance of need for tertiary care during the course of their illness and therefore need early resuscitation and prompt referral, especially in resource-limited developing countries.

Introduction

Acute pancreatitis is a common and frequent inflammatory disorder of the pancreas with variable involvement of other regional tissues or remote organ systems [1]. The disease has a varying etiology with an overall mortality of 5–10%. Most cases (80–90%) are mild and self-limited with a good outcome. The remaining 10–20% of patients with severe disease characteristically have pancreatic necrosis or distant organ failure and can anticipate the need for intensive care and possible operative intervention with a mortality rate of up to 40% [2].

Early diagnosis and precise staging of disease severity are important goals in the initial evaluation and management of acute pancreatitis. While patients with mild acute pancreatitis can be managed with fluid resuscitation and supportive care, those with severe acute pancreatitis require maximal non-operative care and nutritional support in an intensive care unit (ICU). Due to the risk of rapid deterioration in severe acute pancreatitis, the assessment of severity becomes crucial to a clinician [3].

A clinically based classification system for acute pancreatitis was established in the International Symposium on Acute Pancreatitis in Atlanta, Georgia, in 1992. However, criticism of the Atlanta severity classification system was growing because it was retrospective, the duration of organ failure was unspecified and local complications did not seem to increase mortality. The Atlanta classification was revised via an international, web-based consensus in 2012 that provided clear definitions to classify acute pancreatitis using easily identifiable clinical and radiologic criteria. Greater emphasis was laid on organ failure and severity was graded as mild, moderately severe and severe acute pancreatitis [4].

Several multi-factorial scoring systems based on clinical and biochemical data have been used over the past few decades. These include Ranson’s score described in 1974, BISAP and APACHE II to name a few. Each of these scoring systems has its own limitations including the low sensitivity and specificity, complexity of the scoring system as well as inability to obtain a final score until 48 hours after admission [5].

With the advent of contrast enhanced scans, there has been major improvement in the grading system. Attenuation values of pancreatic parenchyma during an intra-venous bolus study can be used as an indicator of pancreatic necrosis and as a predictor of disease severity [6,7]. Contrast enhanced CT has shown an overall accuracy of 87% with a sensitivity of 100% for the detection of extended pancreatic necrosis. The sensitivity and specificity for diagnosing pancreatic necrosis increase with greater degrees of pancreatic non-enhancement, and complications have also been shown to correlate with the degree of non-enhancement [8]. However, early CT scans often fail to identify developing necrosis until such areas are better demarcated, which may become evident only 2–3 days after the initial clinical onset of symptoms. In 2004, modified CTSI was introduced to improve the staging of acute pancreatitis. A study of comparison between CTSI and modified CTSI and comparison of both with APACHE II concluded that modified CTSI was better than CTSI for assessing the severity of acute pancreatitis and the CTSI is better than APACHE II in assessing severe acute pancreatitis [9].

There have been few studies comparing these prognostic scoring systems based on the revised Atlanta classification. This study aimed to assess and compare the prediction of severity of acute pancreatitis based on multi-factorial scoring systems viz. Ranson, BISAP, APACHE II and modified CTSI in a tertiary care center.

Materials and methods

Data collection

Demographic, clinical, biochemical and radiographic data were prospectively collected from 50 patients admitted over the duration of March 2015 to September 2016 in the Department of General Surgery in Pt. B. D. Sharma PGIMS, Rohtak. The study was limited to 50 patients, since the it had to be completed during a fixed timeframe of 2 years and only patients admitted and treated under the direct supervision of the authors were considered.

The diagnoses of acute pancreatitis was based on the presence of two of the following three criteria: (i) abdominal pain characteristic of acute pancreatitis; (ii) serum amylase and/or lipase levels at least three times the upper limit of normal; and (iii) characteristic findings of acute pancreatitis on abdominal ultrasonography and/or computerized tomography (CT) scan. Patients who presented to the emergency department and were diagnosed as having acute pancreatitis based on the criteria mentioned above were informed about the study and written consent was taken. Patients who were diagnosed to have chronic pancreatitis based on their previous hospital records or found to have features of chronic pancreatitis upon radiological investigations during the course of their stay such as pancreatic calcifications, dilated pancreatic duct, areas of atrophy and pseudocysts were excluded from the study.

After detailed history and physical examination, laboratory investigations were sent at the time of admission—arterial blood gas analysis, hematocrit, kidney function test, liver function test, serum electrolytes, serum amylase, serum lipase and complete hemogram. All patients underwent abdominal ultrasonography at admission and contrast enhanced pancreatic protocol CT scan 72 hours after symptom onset.

Patients were subsequently examined daily and laboratory investigations relevant to APACHE II, Ranson’s criteria and BISAP score were sent. APACHE II score was evaluated for each patient within first 24, 48 and 72 hours of admission. BISAP was calculated within first 24 hours of admission. Ranson’s score was evaluated within first 48 hours of admission.

Definitions

At the time of discharge/death, patients were graded as having mild, moderately severe and severe acute pancreatitis based on the Atlanta 2012 classification. Patients with mild acute pancreatitis had neither local complications nor organ failure. Patients with moderately severe acute pancreatitis had transient organ failure or local complications or both, whereas patients with severe acute pancreatitis had persistent organ failure.

Organ failure was defined based on the Modified Marshall scoring system. A score of ≥2 for more than 48 hours was considered as persistent organ failure, whereas a score of ≥2 for less than 48 hours was considered as transient organ failure. Local complications included pancreatic necrosis, acute fluid collections, pseudocyst, acute necrotic collections and walled-off necrosis.

Management protocols

Patients presenting to the emergency department, suspected of having acute pancreatitis, were adequately resuscitated using crystalloids, primarily ringer’s lactate. Inotropes and colloids were added if the patients failed to respond to crystalloids.

All patients were catheterized to monitor the urine output and ascertain the adequacy of resuscitation. Central venous access was obtained for patients who failed to respond to initial resuscitation measures to monitor the central venous pressure and guide further fluid management. A nasogastric tube was placed for all patients. All patients were kept nil per oral for the first 24 hours. Subsequently, patients were examined daily and enteral feeding by means of a nasogastric tube or orally was initiated as soon as features of ileus resolved.

Patients with pancreatic necrosis who failed to improve were planned for necrosectomy and open drainage. A total of two patients underwent surgical intervention for pancreatic necrosis. Patients with cholelithiasis underwent pre-anesthetic checkup and pre-operative work-up prior to discharge and planned to undergo cholecystectomy after 6 weeks as per institutional protocol. Facilities for endoscopic retrograde cholangiopancreatography (ERCP) are not available at our institute.

Statistics

Severity of the disease was evaluated in terms of ICU admission, length of hospital stay, final grade as per Atlanta 2012 classification and presence of pancreatic necrosis. Data were collected prospectively in a Microsoft Excel Database. After completion of data collection, the database was imported into SPSS for Mac (v24.0, SPSS, Chicago, IL, USA). Continuous based line descriptive variables were expressed as mean with standard deviation and were compared using the Mann-Whitney Test and univariate ANNOVA test. Categorical variables were expressed as absolute numbers and proportions. Bivariate relationships for categorical variables were assessed using Fischer’s exact test and Pearson’s chi square test. Sensitivity, specificity, positive predictive value and negative predictive value were calculated for each scoring system. Receiver operating characteristics (ROC) curves for severe acute pancreatitis, ICU admission, pancreatic necrosis and organ failure were plotted for Ranson’s score, BISAP, APACHE II and modified CTSI, and predictive accuracy of each scoring system was measured by the area under ROC curve (AUC) with 95% confidence interval. AUC values were compared for statistical significance using De Long test. A p-value of <0.05 was considered statistically significant.

Results

Patient characteristics

The mean age of patients included in the study was 48.42 (19–80 years). Most of the patients were above the age of 50 years and females (66%). The most common etiology of acute pancreatitis was biliary (74%) followed by alcoholic (18%). Patients were classified as per Atlanta 2012 classification as mild acute pancreatitis (38%), moderately severe acute pancreatitis (34%) and severe acute pancreatitis (28%) (Table 1). Out of the 50 patients, 86% were discharged in satisfactory condition after recovery from acute phase. A mortality rate of 6% was recorded during the study. Four (8%) patients left against medical advice during the course of the study.

Table 1.

Patient characteristics

Characteristics	Category	No. of patients (%)
Sex	Male	17 (34%)
	Female	33 (66%)
Age group (years)	≥60	16 (32%)
	50–59	11 (22%)
	40–49	10 (20%)
	30–39	7 (14%)
	20–29	5 (10%)
	<20	1 (2%)
Etiology	Gall stone disease	37 (74%)
	Alcoholic	9 (18%)
	Traumatic	1 (2%)
	Idiopathic	3 (6%)
Presentation	Pain in abdomen	50 (100%)
	Radiating	32 (64%)
	Non-radiating	18 (36%)
	Peritonitis	44 (88%)
	Localized	26 (59%)
	Diffuse	18 (41%)
	Vomiting	39 (78%)
	Distension of abdomen	28 (56%)
	Non-passage of stool and flatus	22 (44%)
APACHE II (within first 72 hours)	≥8	30 (60%)
	<8	20 (40%)
BISAP score	≥3	23 (46%)
	<3	18 (36%)
Ranson’s score	≥3	22 (44%)
	<3	18 (36%)
Modified CTSI	0–2	14 (28%)
	4–6	22 (44%)
	8–10	14 (28%)
Atlanta 2012 grade	Mild	19 (38%)
	Moderately severe	17 (34%)
	Severe	14 (28%)
Outcome	Discharged	43 (86%)
	Death	3 (6%)
	Left against medical advice	4 (8%)
	ICU admission	14 (28%)
CT findings	Pancreatic necrosis	15 (30%)
	Pancreatic fluid collection	20 (40%)
	Pleural effusion	27 (54%)
	Ascites	24 (48%)

Based on contrast enhanced CT findings, pancreatic necrosis was noted in 30% of patients, pancreatic fluid collections were noted in 40%, pleural effusions were noted in 54% and ascites was noted in 48% (Table 1). ICU care was deemed necessary for 28% of patients.

The mean length of stay in the study was 6.98 days. The length of stay for those graded as having mild acute pancreatitis was 5.63 days, for moderately severe acute pancreatitis 6.58 days and for severe acute pancreatitis 9.28 days. This difference in length of stay was statistically significant (p < 0.05).

Comparison of scoring systems in predicting severe acute pancreatitis, organ failure, pancreatic necrosis and ICU admission

In predicting severe acute pancreatitis according to AUC, modified CTSI had the highest accuracy (0.919) followed by APACHE II (0.834), Ranson (0.754) and BISAP (0.684). In predicting pancreatic necrosis according to AUC, modified CTSI was the most accurate (0.993) followed by Ranson’s score (0.910), APACHE II (0.855) and BISAP (0.822). In predicting ICU admission according to AUC, modified CTSI was the most accurate (0.993) followed by Ranson’s score (0.910), APACHE II (0.885) and BISAP (0.877). In predicting organ failure according to AUC, modified CTSI was the most accurate (0.893) followed by APACHE II (0.831), BISAP (0.762) and Ranson’s score (0.762) (Table 2).

Table 2.

Area under the curve (with 95% confidence interval) of different scoring systems predicting severe acute pancreatitis, pancreatic necrosis, organ failure and ICU admission

Scoring system	Severe acute pancreatitisa (n = 31)	Pancreatic necrosis (n = 15)	Organ failureb (n = 27)	ICU admission (n = 14)
Ranson’s score	0.754 (0.606–0.901)	0.910 (0.767–1.000)	0.757 (0.602–0.912)	0.910 (0.767–1.000)
BISAP	0.684 (0.518–0.849)	0.822 (0.672–0.972)	0.762 (0.605–0.919)	0.877 (0.739–1.000)
APACHE II	0.834 (0.711–0.957)	0.855 (0.731–0.979)	0.831 (0.704–0.959)	0.885 (0.783–0.987)
Modified CTSI	0.919 (0.844–0.994)	0.993 (0.975–1.000)	0.893 (0.798–0.987)	0.993 (0.975–1.000)

^aIncluding patients with moderately severe (n = 17) and severe acute pancreatitis (n = 14). ^bIncluding transient (n = 13) and permanent organ failure (n = 14).

Based on the highest sensitivity and specificity values generated from the ROC curves, the following cut-offs were selected for further analysis: Ranson’s score (≥3), BISAP (≥3), APACHE II (≥8) and modified CTSI (>4), and the results are shown in Table 3.

Table 3.

Predictive value of different scoring systems for pancreatic necrosis, organ failure and ICU admission

	Sensitivity (%)	Specificity (%)	Positive predictive value (%)	Negative predictive value (%)
Pancreatic necrosis
Ranson’s score (≥3)	80.00 (44.39–97.48)	96.55 (82.24–99.91)	88.89 (53.21–98.25)	93.33 (80.18–97.98)
BISAP (≥3)	81.82 (48.22–97.72)	83.33 (65.28–94.36)	64.29 (43.55–80.77)	92.59 (77.94–97.79)
APACHE II (≥8)	93.33 (68.05–99.83)	71.43 (53.70–85.36)	58.33 (44.90–70.63)	96.15 (78.81–99.41)
Modified CTSI (>4)	93.33 (68.05–99.83)	77.14 (59.86–89.58)	63.64 (48.40–76.55)	96.43 (80.12–99.45)
Organ failure
Ranson’s score (≥3)	88.89 (51.75–99.72)	96.67 (82.78–99.92)	88.89 (53.46–98.24)	96.67 (82.03–99.46)
BISAP (≥3)	90.00 (55.50–99.75)	83.87 (66.27–94.55)	64.29 (44.00–80.48)	96.30 (80.10–99.41)
APACHE II (≥8)	92.86 (66.13–99.82)	69.44 (51.89–83.65)	54.17 (41.43–66.39)	96.15 (78.91–99.40)
Modified CTSI (>4)	92.86 (66.13–99.82)	75.00 (57.80–87.88)	59.09 (44.61–72.15)	96.43 (80.18–99.45)
ICU admission
Ranson’s score (≥3)	80.00 (44.39–97.48)	96.55 (82.24–99.91)	88.89 (53.21–98.25)	93.33 (80.18–97.98)
BISAP (≥3)	90.91 (58.72–99.77)	86.67 (69.28–96.24)	71.43 (49.63–86.38)	96.30 (79.96–99.41)
APACHE II (≥8)	92.31 (63.97–99.81)	65.71 (47.79–80.87)	50.00 (38.11–61.89)	95.83 (77.51–99.35)
Modified CTSI (>4)	92.86 (66.13–99.82)	75.00 (57.80–87.88)	59.09 (44.16–72.15)	96.43 (80.18–99.45)

The AUC derived were further compared using the De Long test. Accuracy of APACHE II was found to be significantly higher as compared to BISAP in terms of predicting the severity of acute pancreatitis (p = 0.02) and could be comparable to modified CTSI (p = 0.13). APACHE II was also comparable to BISAP and Ranson’s score in predicting pancreatic necrosis, organ failure and ICU admission (Table 4).

Table 4.

Pairwise comparison of AUC amongst APACHE II, BISAP and Ranson’s score using the De Long test

Comparison	Severe acute pancreatitis		Pancreatic necrosis		Organ failure		ICU admission
	Z statistic	P-value	Z statistic	P-value	Z statistic	P-value	Z statistic	P-value
APACHE II vs BISAP	2.321	0.02	0.198	0.84	0.890	0.33	0.017	0.98
APACHE II vs Ranson	0.607	0.54	0.152	0.87	0.835	0.40	0.365	0.71
Ranson vs BISAP	1.302	0.19	1.114	0.26	0.261	0.79	0.366	0.71
APACHE II vs modified CTSI	1.488	0.13	2.254	0.02	0.983	0.32	2.244	0.02
BISAP vs modified CTSI	3.039	0.002	2.298	0.02	1.54	0.12	1.618	0.10
Ranson vs modified CTSI	2.449	0.01	1.220	0.22	1.620	0.10	1.220	0.22

Discussion

Acute pancreatitis is a common ailment encountered by physicians in emergency departments all over the world. It is critical to identify patients with severe acute pancreatitis who will benefit from early intensive care therapy. In most cases, it is difficult to assess the severity clinically alone.

The mean age of the study population was 48.42 years and the male-to-female ratio was 0.51 (34% males). Gall stone disease (74%) followed by alcohol (18%) were the most common etiological factors in our study. The higher incidence of gall stone disease and female preponderance in our study as compared to similar studies in other parts of India could be attributed to the higher prevalence of gall stone disease in northern India, where our institute is located [10,11].

In this study, 17 (34%) patients are graded as moderately severe and 14 (28%) were graded as having severe acute pancreatitis. Pancreatic necrosis was present in 15 (30%) patients, while 14 (28%) developed persistent organ failure and 14 (28%) needed ICU admission. During the course of the study, mortality was recorded in three (6%) patients. All three patients were graded as having severe acute pancreatitis based on Atlanta 2012 criteria. The cause of death in all three patients was multiple organ failure. Similar mortality rates have been reported in large series by Carnovale et al. (4.8%) and Singh et al. (3.5%) [12,13].

Considering the poor availability of CT scanning and ICU facilities in our country, we aimed to compare various prognostic scoring systems, which may aid in decision making regarding which patients need to be referred to a tertiary care center at the earliest.

The AUC for modified CTSI was the highest for all the four parameters considered as markers for severity of acute pancreatitis, namely pancreatic necrosis (0.993), need for ICU admission (0.993), severe acute pancreatitis (0.919) and organ failure (0.893). Most other studies with similar study designs include CTSI rather than modified CTSI. It is important to note that CTSI has no weight for extra-pancreatic complications such as pleural effusions and vascular complications, while giving additional weight to pancreatic necrosis involving >50% of the pancreas. Banday et al. and Mortele et al. observed that modified CTSI is a simpler and more accurate scoring tool as compared to CTSI and has a stronger statistical correlation with length of stay, development of infection, organ failure and mortality [14,15]. In one of the few similar studies using modified CTSI, Yang et al. observed modified CTSI to have outstanding performance (AUC 0.791) in predicting local complications as compared to APACHE II and BISAP [16]. However, it performed poorly compared to these scoring systems in predicting severity and mortality, and contrast enhanced CT was performed within 3 days of onset, which may reduce its sensitivity. Moreover, the study population was exclusively limited to patients with hyperlipidemic acute pancreatitis.

In the present study, based on AUC comparisons, only APACHE II was found to be comparable to modified CTSI in terms of severity of acute pancreatitis (p = 0.13). On the other hand, the AUC of modified CTSI was significantly higher than Ranson’s score (p = 0.02) as well as BISAP (p = 0.002) in predicting the severity of acute pancreatitis. The AUC of APACHE II was also found to be significantly higher than BISAP score in predicting the severity of acute pancreatitis (p = 0.02). Even though the AUC of Ranson’s score was higher than APACHE II in predicting pancreatic necrosis and ICU admission, the difference was not significant (both p > 0.05). Mounzer et al., in a similar study, compared several prognostic scores and also found APACHE II to be more accurate as compared to Ranson’s and BISAP [17].

APACHE II was also found to have a high sensitivity and negative predictive value for predicting pancreatic necrosis (93.33% and 96.15%), organ failure (92.86% and 96.15%) and ICU admission (92.31% and 95.8%), which makes it an ideal scoring system for decision making regarding referral to higher centers.

The current study has a few limitations. The sample size is too small to make definitive comparisons amongst the scoring systems. The study population consists mostly of pancreatitis secondary to gall stone disease and therefore no meaningful comparisons can be made amongst the various scoring systems for different etiologies. Study differs from other similar studies in the use of modified CTSI instead of CTSI, which may make comparisons with other similar studies difficult.

In conclusion, although the study is limited by its small sample size, which makes it difficult to make any broad recommendations, it can be safely said that APACHE II can be a useful tool in predicting which patients are likely to develop severe disease early in the course of their illness and it may be somewhat better than Ranson’s score and BISAP in this regard.

Conflict of interest statement: none declared.