Comparison of the Performance of APACHE II, SOFA, and mNUTRIC Scoring Systems in Critically Ill Patients: A 2-year Cross-sectional Study

Abstract

Aims and objectives

Different severity scores are being used to assess outcomes in intensive care unit, but variable data had been reported so far per their performance. Main objective of this study is to compare performance of acute physiology and chronic health evaluation II (APACHE II), sequential organ failure assessment (SOFA), and modified nutrition risk in critically ill (mNUTRIC) scoring systems regarding the outcomes in the form of morbidity and mortality in medical intensive care unit (MICU) at rural tertiary-care health center.

Materials and methods

In this cross-sectional study, 1,990 patients older than 18 years admitted in the ICU were enrolled. Age, gender, diagnosis, intubation, comorbidities, APACHE II, SOFA scores, m NUTRIC score, MICU stays in days, and need of mechanical ventilation were noted.

Results

When we compared different score with mortality, APACHE-II was having sensitivity of 89.9% and specificity of 97.6%; SOFA had 90.1% sensitivity and 96.6% specificity; while mNUTRIC score had 97.2% sensitivity and 74.0% specificity. APACHE-II score had sensitivity of 93.4%, SOFA had 90.5%, and mNUTRIC score 92.3% with low specificity of 76.5% in predicting requirement of mechanical ventilation. mNUTRIC score and ICU length of stay showed moderate positive correlation (p value = <0.001).

Conclusion

All the three scores were comparable in sensitivity and specificity in predicting outcomes in the form of mortality, need of mechanical ventilation, and length of ICU stays. mNUTRIC score was more sensitive than others, and as it was based on nutritional status, hence more weightage should be given on this score.

How to cite this article

Kumar S, Gattani SC, Baheti AH, Dubey A. Comparison of the Performance of APACHE II, SOFA, and mNUTRIC Scoring Systems in Critically Ill Patients: A 2-year Cross-sectional Study. Indian J Crit Care Med 2020;24(11):1057–1061.

Introduction

For the assessment of patient outcome such as morbidity and mortality, as well as severity of illness in most of the medical intensive care units (MICU), various scoring systems had been studied.^1–6 To date, no consensus has been reached to point out the foremost tool for identifying the propensity of risk in critically ill patients.^3,4 More studies are required to evaluate various scoring tools, as their sensitivity and specificity are not defined to predict mortality in different patients across medical ICU setup. Nowadays sequential organ failure assessment (SOFA) and acute physiology and chronic health evaluation II (APACHE II) are being used for comparisons of different outcomes in different ICUs.^7–9 In 2011, Heyland et al. proposed, developed, and validated a widely accepted unique scoring system known as the NUTRIC or the Nutrition Risk in Critically ill score comprises of 6 variables, most of which are relatively obtained effortlessly in the ICUs.^10–12 Later on studies showed that eliminating interleukin 6 levels from the original NUTRIC Score did not have drastic effects on its performance, and a revised mNUTRIC (modified NUTRIC score) was formulated.^13–15 The patients who had a score ≥6 were considered to be at high risk. No studies had been reported so far while comparing APACHE II, SOFA, and mNUTRIC scoring systems as far as their performance is concerned. In this study, we had tried to evaluate performance APACHE II, SOFA, and m NUTRIC scoring systems regarding the outcomes in ICU, such as mortality, need of mechanical ventilations, and length of ICU stay.

Materials and Methods

This prospective cross-sectional study was conducted in the 42-bedded medical intensive care unit (MICU) under Department of Medicine at Jawaharlal Nehru Medical College, Wardha, in central India, from October 2017 to September 2019. Ethical clearance for this study was obtained from local institutional ethical committee, and written informed consent was obtained from the participants before commencement of the study.

Sample Size

A point biserial correlation for power analysis was conducted in G*Power to determine a sufficient sample size with an alpha of 0.05, a power of 0.80, a small effect size (p value = 0.1), and two tails.¹⁶ The optimal sample size was 779 based on the abovemnetioned assumptions. In this study, all the consecutive/sequential patients older than 18 years of age (n = 2100) admitted in the MICU were enrolled in this study. During this 2 year period, a total of 1,990 patients were analyzed in the present study. Patients who died within 24 hours were excluded. Critically ill surgical and patients of transplants were excluded because of admission at separate special ICU. Patients referred from other ICUs were also excluded. For patients with multiple admissions in MICU, first admission was included in data analysis. The study algorithm flow diagram is shown in Flowchart 1.

Flowchart 1.

Flow diagram of the study. APACHE II, acute physiology and chronic health evaluation II; ICU, intensive care unit; MICU, medical ICU; SOFA, sequential organ failure assessment

mNUTRIC SCORE was calculated in critically ill patients admitted to the MICU by taking various variables that are a part of the scoring system like age, thorough history and clinical examination, and preexisting comorbidities. SOFA score and APACHE II score were calculated using online software.

The cutoffs of mNUTRIC score may differ in different ICUs depending on the patient's characteristics, the disease dynamics, cultural, social, and environmental factors, and the type of treatment facilities available. Based on the data analyzed over 2 years, the results showed that a cutoff of mNUTRIC score >5.5 was associated with adverse outcome. The range of mNUTRIC score ranges from 0 to 9. Since the value 5.5 was not a whole number, we rounded of the value to the closest approximate higher value, i.e., 6. Hence, we divided our study population into two groups with patients having score ≥6 as high mNUTRIC score and those having score ≤5 as low mNUTRIC score. Accuracy of performance of APACHE II, SOFA, and mNUTRIC scoring system was measured by the area under the ROC curve for mortality and requirement of mechanical ventilation in MICU.

Statistical Analysis

Data were coded and recorded in MS Excel. Descriptive statistics were elaborated in the form of means ± standard deviation for continuous variables and frequencies/percentages for categorical variables. Group differences were compared using t test for continuously distributed data, and chi-square test for categorical data. IBM SPSS v23 was used for statistical analysis. Statistical significance was set at p value <0.05.

Result

In this study, out of 1,990 patients, 395 died while 1,595 survived and 799 had a high (≥6) mNUTRIC score, while 1,191 had a low (≤5) mNUTRIC score. Mean age of the patients were 50.48 ± 17.58 years, male being 63.2% and female being 36.8%. Other baseline characteristics are shown in Table 1.

Table 1.

Baseline characteristics

Parameters (n = 1990)	Nonsurvivors (n = 395)	Survivors (n = 1595)	p value
Age (years)	56.67 ± 16.92	48.95 ± 17.40	<0.0011
Gender			0.0232
Male	269 (68.1%)	988 (61.9%)
Female	126 (31.9%)	607 (38.1%)
APACHE-II score	40.00 ± 7.13	22.17 ± 4.61	<0.0011
SOFA score	15.86 ± 3.36	6.62 ± 1.93	<0.0011
ICU length of stay (days)	6.02 ± 6.04	6.76 ± 5.35	<0.0011
mNUTRIC score	7.15 ± 0.94	4.64 ± 1.50	<0.0011
mNUTRIC score category			<0.0012
High (≥6)	384 (97.2%)	415 (26.0%)
Low (≤5)	11 (2.8%)	1180 (74.0%)
Mechanical ventilation	345 (87.3%)	137 (8.6%)	<0.0012
Hypertension	282 (71.4%)	826 (51.8%)	<0.0012
Diabetes mellitus	45 (11.4%)	127 (8.0%)	0.0302
Renal diseases	267 (67.6%)	459 (28.8%)	<0.0012
Neurological diseases	168 (42.5%)	348 (21.8%)	<0.0012
Cardiovascular diseases	356 (90.1%)	1102 (69.1%)	<0.0012
Respiratory diseases	348 (88.1%)	248 (15.5%)	<0.0012
Sepsis	268 (67.8%)	603 (37.8%)	<0.0012
Other disorders	297 (75.2%)	893 (55.9%)	<0.0012

¹Kruskal-Wallis test;

²Chi-squared test. APACHE II, acute physiology and chronic health evaluation II; ICU, intensive care unit; mNUTRIC, modified nutrition risk in critically ill; SOFA, sequential organ failure assessment

When we compared different scores with mortality, APACHE-II was having sensitivity of 89.9% and specificity of 97.6% (AUC = 0.983), SOFA had 90.1% and 96.6% (AUC = 0.986), whereas mNUTRIC score had 97.2% and 74.0% (AUC = 0.938), respectively, shown in Figure 1. As far as the need for mechanical ventilation were considered, APACHE-II score had sensitivity of 93.4% and specificity of 89.7% (AUC = 0.966); SOFA had sensitivity of 90.5% and specificity of 95.8% (AUC = 0.976); and mNUTRIC Score had sensitivity of 92.3% and specificity of 76.5% (AUC = 0.901) as shown in Figure 2.

Fig. 1.

Comparison of the diagnostic performance of various scores and mortality. APACHE II, acute physiology and chronic health evaluation II; mNUTRIC, modified nutrition risk in critically ill; SOFA, sequential organ failure assessment

Fig. 2.

Comparison of the diagnostic performance of various scores and requirement of mechanical ventilation. APACHE II, acute physiology and chronic health evaluation II; mNUTRIC, modified nutrition risk in critically ill; SOFA, sequential organ failure assessment

Scatterplot shown in Figures 3 to 5 depicts the correlation between all three scoring systems and ICU length of stay (days). Individual points represented as individual cases. The blue trend line represented the general trend of correlation between the two variables. The shaded gray area represented the 95% confidence interval of this trend line. There was a weak positive correlation between APACHE-II score and SOFA score with ICU length of stay (days), and this correlation was statistically significant (ρ = 0.24, p value = <0.001), whereas between mNUTRIC Score and ICU length of stay (days), there was moderate positive correlation (ρ = 0.31, p value = <0.001). For every 1 unit increase in mNUTRIC score, the ICU length of stay in days increases by 0.98 units.

Fig. 3.

Correlation between acute physiology and chronic health evaluation II (APACHE II) score and intensive care unit (ICU) length of stay (days)

Fig. 5.

Correlation between modified nutrition risk in critically ill (mNUTRIC) score and intensive care unit (ICU) length of stay (days)

Discussion

Treating patient and predicting mortality in the ICUs is always a challenge as well as great concern for physicians. The effect of this prediction is on various aspects of patient care, such as medical treatment, triage, end-of-life ICU care, and many more. In recent years, there has been a growing interest in finding new prognostic markers like mNUTRIC score in critically ill ICU patients. Primary outcomes of this study were to compare mortality by different scoring system in critically ill patients. Secondary outcome was to analyze the need of mechanical ventilation and length of ICU stays which may have affected mortality. Most of the time because of variable institutional and population parameters, ICU mortality rates also varies which ranges from 6.4 to 46%.^17,18 In our study, mortality rate was 14.7% in less than 50 years of age, 20.7% among 50–75 years of age, and 42.0% in older than 75 years of age. Study by Leong and Tai and Ursavaş et al. had not shown any direct relation as far as increasing age and mortality rates are concerned. They further quoted that age was not a deciding factor in a ICU patient.^17–19

Estimated mortality by APACHE-II score was 46.47 ± 22.81%, 56.43 ± 20.21%, 64.04 ± 20.46% (p value < 0.0011) in the same groups, respectively. In this study, we had found trend of increase in APACHE II scores and mortality as patient's age increases.

Several studies have regarded mNUTRIC score to be one of the best scoring systems to adjudge nutritional adequacy in critically unwell patients and so the outcomes.^8,9,11,12 This score was valuable in categorizing illness according to its severity and estimating the approximate duration of ICU stay and need of mechanical ventilation.

On comparing different scoring system with mortality, APACHE-II and SOFA had better sensitivity and specificity than mNUTRIC Score, which had high sensitivity but low specificity. Same observation was in case of need for mechanical ventilation. In this study, there was a weak positive correlation between APACHE-II score and SOFA score with ICU length of stay but moderate positive correlation with mNUTRIC score.

Strength and Limitation

The present study had strength of single-center study with large number of patients having standard care. Main limitation of the study was not having predefined sample size, as we had compared the different scores regarding outcome in terms of mortality.

Fig. 4.

Correlation between sequential organ failure assessment (SOFA) score and intensive care unit (ICU) length of stay (days)

Conclusion

The discriminative performance of mNUTRIC score for assessing overall outcomes was found to be comparable in our study. Although having lower specificity, mNUTRIC score was more sensitive than others, and as it was based on nutritional status, more weightage should be given on this score. Further studies with multicenter trial are needed to develop national agreement and better clarification of the issues along with ICUs from different parts of the world taking more concern on nutritional status.

Ethics Statement

This study had been approved by Institutional Ethics Committee review; with approval number DMIMS (DU)/IEC/2018-19/6759.