Comparison of Various Severity Assessment Scoring Systems in Patients with Sepsis in a Tertiary Care Teaching Hospital

Abstract

Background:

Sepsis is a complex condition defined by the systemic response to infection. Severity assessment scoring systems are used to aid the physician in deciding whether aggressive treatment is needed or not. In this study, various severity assessment scoring systems, namely Acute Physiology and Chronic Health Evaluation II (APACHE II), Rapid Emergency Medicine Score (REMS), Sequential Organ Failure Assessment (SOFA), Multiple Organ Dysfunction Score (MODS), Predisposition, Infection, Response, and Organ Dysfunction (PIRO), and Mortality in Emergency Department Sepsis (MEDS), were compared to assess their sensitivity and specificity.

Materials and Methods:

A prospective cohort study was conducted over 6 months. The study was conducted in the intensive care unit (ICU) of a tertiary care teaching hospital. All patients above 18 years of age with confirmed sepsis diagnosis and a well-defined outcome were included in the study.

Results:

A total of 193 patients were included in the study. The mean age was 57.2 ± 15.3 (mean ± standard deviation) years. Majority of the patients were male, 125 (64.76%). Overall mortality was 108 (55.9%). The calculated area under the receiver operating characteristic curve was 0.86 (95% confidence interval [CI]: 0.80–0.90) for APACHE II, 0.81 (95% CI: 0.75–0.87) for REMS, 0.80 (95% CI: 0.74–0.86) for SOFA, 0.74 (95% CI: 0.67–0.80) for MODS, 0.78 (95% CI: 0.71–0.84) for PIRO, and 0.77 (95% CI: 0.71–0.83) for MEDS. Sensitivity and specificity for APACHE II were 81.5 and 75.3, respectively.

Conclusions:

In our study, APACHE II score was found to be the most sensitive and specific in predicting the severity of sepsis compared to other scores.

INTRODUCTION

Sepsis occurs when the body releases inflammatory mediators into the bloodstream to fight against an infection. Sepsis is a three-stage process beginning with sepsis, developing to severe sepsis and finally septic shock.

According to the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3), sepsis can be diagnosed when there is presence of increased heart rate, increased respiratory rate, and increased or decreased body temperature and white blood cell count.[1] The presence of all the symptoms of sepsis accompanied by multiple organ damage defines severe sepsis. Finally, septic shock arises when there is severe hypotension, unresponsive to simple fluid replacement, along with all the signs and symptoms of sepsis.

Globally, 31.5 million cases of sepsis are reported every year and the estimated mortality is 5.3 million.[2] Some studies have shown that 28.3% of patients present with severe sepsis or septic shock during hospital intensive care unit (ICU) stay in India and severe sepsis has a mortality rate of 59.26% in India.[3,4]

Severity assessment scoring systems are used to triage the patients presenting with sepsis so as to aid the physician in deciding whether aggressive treatment is needed. This can save time and cost for the patient and ensure that he receives adequate care. Different scoring systems have been introduced to determine the disease severity and prognosis of patients admitted in the ICU.

Scoring systems are used frequently in the hospital setting to predict the mortality. Each system uses a different combination of parameters to stratify the patient. Hence, it is essential to compare various scoring systems to determine which combination of parameters best predicts the status of the patient.

The objective of our study was to compare various severity assessment scoring systems, namely Acute Physiology and Chronic Health Evaluation II (APACHE II), Rapid Emergency Medicine Score (REMS), Sequential Organ Failure Assessment (SOFA), Multiple Organ Dysfunction Score (MODS), Predisposition, Infection, Response, and Organ Dysfunction (PIRO), and Mortality in Emergency Department Sepsis (MEDS).

In our study, APACHE II was found to be superior compared to all other scores with high sensitivity and specificity in predicting mortality among sepsis patients.

MATERIALS AND METHODS

Study design and setting

This was a prospective cohort study carried out in a South Indian tertiary care teaching hospital. Institutional Ethics Committee approval was obtained before the study. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Study population

Sepsis cases were identified from patients admitted in the ICU. Sepsis was diagnosed according to the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). All patients diagnosed with sepsis with a well-defined outcome, such as recovery or death, were included in the study. All patients below 18 years of age were excluded. A total of 193 patients were enrolled in the study.

Study protocol

The severity scores were calculated from the data recorded in the patient charts. The most abnormal value recorded in the ICU on the day of sepsis diagnosis was used in the score calculations. The data required for each scoring system were recorded on a separate document (case report form). The outcome was the severity score and predicted mortality.

Statistical analysis

Continuous data were expressed as mean ± standard deviation (SD) and median (interquartile range). Nominal data were expressed as frequency and percentage. Independent samples t-test was used to compare means between two groups, namely survivors and nonsurvivors. Receiver operating characteristic (ROC) curve analysis was used to analyze the discriminative power (ability to differentiate between survivors and nonsurvivors) of various severity scores. P < 0.05 was considered statistically significant. Statistical analysis was done using IBM SPSS Software version 20.0 (IBM Corp. IBM SPSS Statistics for Windows, Armonk, NY). MedCalc Statistical Software version 16.4 (MedCalc Software, Ostend, Belgium) was used for ROC curve analysis.

RESULTS

A total of 193 sepsis patients were included in the study during the study based on the inclusion and exclusion criteria. The demographic and clinical characteristics of the study population are shown in Table 1. The mean age of the population was found to be 57.2 ± 15.3 (mean ± SD) years. About 47.2% of the patients were elderly, i.e., over the age of 60. The mean length of hospital stay was found to be 9.3 ± 8.6 days. The mean length of ICU stay was 7.3 ± 7.1 (mean ± SD) days. The study population consisted of a higher number of males at 64.76%. About 31.1% of the patients developed multiple organ dysfunction and 72.5% developed septic shock.

Table 1.

Demograpic and clinical characteristics of the study population

Overall mortality was 108 (55.9%). Of 193 patients enrolled, only 30.1% had a positive blood culture. The most common organisms isolated from the blood were Escherichia coli 22.4%, followed by Klebsiella pneumoniae 17.2% and other unidentified Gram-negative bacilli 15.5%.

Comparison of the scores APACHE II, REMS, SOFA, MODS, PIRO, and MEDS between survivors and nonsurvivors is shown in Table 2. All the scores were found to be significantly higher in nonsurvivors.

Table 2.

Comparison of various scores between survivors and nonsurvivors

Area under the curve (AUC), sensitivity, and specificity of the various scoring systems are shown in Table 3. The calculated area under the ROC curve was 0.86 (95% confidence interval [CI]: 0.80–0.90) for APACHE II, 0.81 (95% CI: 0.75–0.87) for REMS, 0.80 (95% CI: 0.74–0.86) for SOFA, 0.74 (95% CI: 0.67–0.80) for MODS, 0.78 (95% CI: 0.71–0.84) for PIRO, and 0.77 (95% CI: 0.71–0.83) for MEDS. APACHE II and MODS showed highest and lowest discrimination power, respectively. APACHE II (81.48) had the highest sensitivity whereas MEDS (62.04) had the least sensitivity. ROC curves for all the models are depicted in Figure 1.

Table 3.

Area under the receiver operating characteristic curve, sensitivity, and specificity data of various scoring systems

Figure 1.

Comparison of ROC curves. ROC: Receiver operating characteristic; APACHE II: Acute Physiology and Chronic Health Evaluation II; REMS: Rapid Emergency Medicine Score; SOFA: Sequential Organ Failure Assessment; MODS: Multiple Organ Dysfunction Score; PIRO: Predisposition, Infection, Response, and Organ Dysfunction; MEDS: Mortality in Emergency Department Sepsis

DISCUSSION

Our study found mortality rate of sepsis to be 35.8%, septic shock to be 63.6%, and a total mortality rate of 56%. This is in accordance with the study conducted by Vincent and Moreno.[5]

Men have shown to be more likely to suffer from sepsis.[6] This is supported by our study as our study population consisted of 64.76% of males. A study by Martin et al. showed a higher incidence of sepsis in the elderly. This was observed in our study as 47.2% of the patients were in the elderly category.[7]

The scoring system APACHE II showed the best discriminative power and sensitivity. The AUC was found to be 0.856. Our study is in accordance with the results obtained by Ho et al. as APACHE II did better than the organ failure scores (SOFA and MODS) we used. Furthermore, the AUC obtained for SOFA is comparable with the results from the above-mentioned study. APACHE II showed higher sensitivity as it takes into account age, comorbidity, and surgical status of the patient. In addition, APACHE II performed better than organ failure scores as the latter is affected by therapeutic interventions.[8]

A study by Olsson et al. showed REMS to have an AUC of 0.852 ± 0.014, which is similar to our result of 0.814.[9] Ghanem-Zoubi et al. conducted a study comparing the modified early warning score, simple clinical score, MEDS, and REMS. The results showed REMS to have a similar discrimination power to that of our study. Moreover, the study revealed MEDS to have an AUC of 0.73–0.75, very close to our result of 0.774.[10]

The AUC obtained for the PIRO score was 0.779, which is lower than the results obtained by Macdonald et al. in their study. In addition, their study compared the models PIRO, MEDS, and SOFA and found PIRO to be the most superior. This is in contrast to our study where SOFA proved to be superior to PIRO and MEDS. This could be accounted to the difference in the study population, as our study consisted of ICU patients diagnosed with sepsis, while their study consisted of emergency department patients diagnosed predominantly with severe sepsis or septic shock, with only half of them being admitted to the ICU.[11]

A study conducted on surgical patients by Marshall et al. claimed MODS to have a very good discriminating power with AUC 0.93.[12] Our study reflected a lower AUC of 0.742. This small difference in AUC could be due to our study population consisting of surgical and medical patients. A study by Bota et al. showed APACHE II to have better discriminative power than MODS and SOFA; this is in accordance with our results.[13]

APACHE II showed the highest sensitivity, while MEDS had the lowest. This could be explained as APACHE II takes into account more patient parameters, such as temperature, blood pressure, serum creatinine, serum sodium, and serum potassium, as compared to MEDS. APACHE II assigns a score to each of these parameters depending on how far they vary from normal range, thus showing the severity of organ failure, whereas MEDS assigns a score based on the presence or absence of organ failure without taking into consideration the severity.

MEDS showed to be the most specific score and MODS the least. The MEDS scoring system uses presence of comorbid illness, age, and organ failure to stratify the patient, while MODS assigns a score based on the severity of organ failure only.

One of the primary limitations for our study is a limited sample size. A larger sample size could have better validated the various scoring systems. Our study was conducted in a single center. A multicentric study may be necessary. The data collected for all the scores were only done once on the day of sepsis diagnosis. Some scores such as SOFA and MODS require repeated data collection so as to calculate severity scores on different days. This was not carried out in our study. Hence, we may not have been able to identify trends in the results. Moreover, our study included adults of all ages. Thus, it is not possible to check the predictive power of the scoring systems in a particular age group, like the elderly. This study could have been affected by lead time bias. The study population included all surgical and medical patients.

CONCLUSION

In our study, the APACHE II score found to be more useful for stratifying sepsis patient as it considers laboratory parameters, chronic comorbidities, and surgical status of the patient. The illness severity and predicted mortality among sepsis patients by APACHE II score were very close to observed mortality. The finding of this study may have implications for identifying and managing high-risk sepsis patients.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.