Are scoring systems sufficient for predicting mortality due to sepsis in the emergency department?

Abstract

Objectives

Scoring systems have been used to risk stratify in intensive care units (ICU), but not routinely used in emergency departments. The aim of this study was to determine accuracy for predicting mortality in emergency medicine with Sequential Organ Failure Assessment (SOFA), Mortality in ED Sepsis (MEDS) score and Simplified Acute Physiology Score (SAPSII).

Methods

This is a prospective observational study. Patients presenting with evidence of sepsis were all included. SAPSII, MEDS, and SOFA scores were calculated. Analysis compared areas under the receiver operator characteristic (ROC) curves for 28-day mortality.

Results

Two hundred patients were included; consisting of 31 (14.3%) septic shock. 138 (69%) severe sepsis and 31 (15.5%) infection without organ dysfunction. 53 (26.5%) patients died within 28 days.

Area under the ROC curve for mortality was 0.76 for MEDS (0.69–0.82), 0.70 for SAPSII (0.62–0.78); and 1.68 for SOFA (0.60–0.76) scores. Pair wise comparison of AUC between MEDS, SAPSII, SOFA and Lactate were not significant.

Conclusion

According to our results; SOFA, SAPSII and MEDS were not sufficient to predict mortality. Also this result, MEDS was better than other scoring system.

1. Introduction

Sepsis is a common global health issue due to the increasing elderly population, which results in escalated healthcare costs and mortality.¹ According to the consensus guidelines, septic patients must be characterized by the severity of organ dysfunction and/or septic shock. Early Goal-Directed Therapy is required in the first 6 h; these golden hours often occur in the emergency department (ED).² Previous studies found that septic patients were managed in ED for 4.9–6 h, so it is important to understand, recognize, and manage sepsis in the ED.3, 4

Although septic patients are clinically established by the severity of organ dysfunction and/or septic shock, the admission criteria for inclusion in Intensive Care Unit (ICU) or non-critical hospital beds were not determined. Scoring systems have been used for critically ill patients in the ICU, but these are difficult to use in emergency medicine due to the abundance of data.

The Mortality in ED Sepsis (MEDS) score was developed for septic patients who were admitted to emergency medicine departments and predicts mortality within 28 days. MEDS was organized according to the recommended PIRO (Predisposition Infection Response Organ Dysfunction) classification approach by 9 characterized risks, according to Predisposition, Infection, Response, or Organ Dysfunction. Predisposition risk includes age over 65, nursing home residency, and rapidly terminal illness. Infection risk includes lower respiratory infection. Response risks are bands>5%, whereas Organ Dysfunction includes tachypnea, hypoxemia, septic shock, platelet counts<150, 000, and altered mental status. MEDS contains “Rapidly terminal illness,” which refers to a 50% mortality within 30 days; however, it is difficult to predict mortality with patient's illness because both sepsis and the other disease(s) can lead to mortality. Therefore, this variable was confusing and was not explained further.5, 6 The Sequential Organ Failure Assessment (SOFA) aims to define organ dysfunction in critically ill patients and describes scores with 6 organ functions; a normal score is 1 and scores can change 1–4.⁷ The SOFA accounts for organ dysfunction without patient's age, chronic disease status, and vital signs, so the SOFA may not have enough accuracy to predict mortality.⁸ Giannazzo et al. found that SOFA correlated with poor prognosis at 24 h after admission but not at 28 days.⁹

The Simplified Acute Physiology Score (SAPSII) is improved according to the SAPS model and includes 17 types of data about major organ systems and physiological data, age, admission type to ICU (scheduled/unscheduled surgery/medical), and chronic disease (metastatic/hematological malignancy). Each criterion is defined by a different point.¹⁰ Previous studies found that SAPSII is better than SAPS for predicting mortality but Acute Physiology and Chronic Health Evaluation (APACHE) scores are better than SAPSII scores. SAPSII is quite comprehensive and useful in the ICU but often requires information that is not readily available to an ED.¹¹

High levels of lactate are correlated with shock, low perfusion, and poor prognosis. Previous studies in emergency medicine declared that venous lactate level can predict mortality in 3 days; lactate levels greater than 4 mmol signified high-level critical illness. In an ED, lactate has an important value for the recognition and management of sepsis.¹²

In this study we aimed to compare scoring systems (SOFA, SAPSII, MEDS) for predicting mortality due to sepsis at emergency medicine department. MEDS is an emergency scoring system, so we group MEDS score and add lactate to form a new model for predict mortality.

2. Materials and methods

2.1. Study setting and population

This was a prospective, observational study at an urban, tertiary care, University Hospital and the study enrollment period was March 1st, 2014 through August 1st, 2014. The hospital's human research ethics committees approved the study.

2.2. Definitions4, 5, 18

Adult patients (aged 18 years or more) who had at least two systemic inflammatory response features, heart rate> 90 beats/min, respiratory rate > 20 breaths/min, temperature >38 °C or <36 °C, total white cell count >12 10⁹/<4 10⁹/L, and clinical infection were defined as suffering from sepsis. Infection source was categorized as respiratory tract, urinary tract and respiratory tract, urinary tract, intra-abdominal and soft tissue infections.

We further classified enrolled sepsis patients according to standard consensus definitions. Uncomplicated sepsis was defined as sepsis with no requirement for organ support. Severe sepsis was defined as associated organ dysfunction in the absence of shock. Septic shock was defined as associated hypotension (systolic blood pressure [sBP] < 90 mm Hg despite ≥ 1000 mL isotonic crystalloid bolus) and/or hypoperfusion (serum lactate ≥ 4 mmol/L).

MEDS score is developed for septic patients at emergency medicine and predicts mortality within 28 days, so we grouped MEDS as original described: group 1; low risk (0–7 point), group 2; moderate risk (8–12 point), group 3; high risk (13–15 point) and group 4; very high risk (>15 point).⁵

2.3. Inclusion into the study

Patients who had at least two systemic inflammatory response features and clinical infection were defined as suffering from sepsis and enrolled in the study by emergency physicians not only by researchers, in real time, prospectively, at the ED. According to sepsis' consensus definition, patients were defined as uncomplicated sepsis, severe sepsis, or septic shock. Patients' demographic information, including history, physical exam, vital sign information, and scoring system variables, was recorded by investigators using a structured data collection instrument in real time at the ED. Blood pressure, heart rate, respiratory rate, oxygen saturation was monitored and blood cell count, liver and renal function tests, electrolytes, coagulation (aPT, PTT), artery or venous blood gases, lactate was performed to all patients when initial diagnosis. First the patients were treated in emergency room and according to patient situation, admitted to ICU or non-critical hospital beds and the remain of the treatment was continued in there (see Fig. 1)

Fig. 1.

ROC curves for MEDS, SAPSII, SOFA and Lactate with 28 day mortality.

2.4. Exclusion of the study

Patients with a non-sepsis diagnosis (e.g., pulmonary embolism, trauma) and those who did not have at least two systemic inflammatory response features or were pregnant were excluded from the study.

2.5. Data analysis

The Chi-squared test was used for categorical data and each of the three scoring systems' descriptive values (mean, median). MEDS, SOFA, and SAPSII scores and lactate levels were compared by the method of area under the receiver operator characteristic (ROC) curves (AUC). Pairwise comparison of the groups and comparison of the AUCs were performed. MEDS is grouped as original to compare the groups' mortality ratio and the mortality distributions of the MEDS subgroups were performed using the Chi-squared test. MEDS components and lactate added to logistic regression analysis for compose a new model and compared with mortality. Statistical analyses were performed using Medcalc 13.

3. Results

The number of the patients who were admitted to emergency room with two criteria of SIRS were 485 and 182 of them did not want to include to the study. The number of the patients who were excluded from the study were 103. 200 patients were enrolled the study and 55% were male. The mean age was 77 (21–100).

3.1. Mortality

Two hundred patients were enrolled and 53 patients (26%) died. There was no significant difference in mean age between the survivors and non-survivors and the ages of both groups were older than 65 (Table 1).

Table 1.

Comparison of survivors and nonsurvivors group* P value < 0,05.

	Total (n = 200)	Nonsurvivors (n = 53)	Survivors (n = 147)	p-value
Age (yr) (mean ± SD)	74 ± 15	75 ± 14	73 ± 15	0.366
Gender
Female (%)	89 (%45)	19(%35)	70 (%48)	0.139
Male (%)	111 (%55)	34 (%65)	77 (%52)
Lactate (mmol)	2 ± 2.6 (0–1)	3.2 ± 2.2 (0–1)	2.3 ± 1.9 (0–1)	0.007*
Severe sepsis
Yes	137 (%68)	36(%68)	101(%69)	0,916
No	63 (%32)	17(%32)	46(%31)
Shock
Yes	32 (%16)	14 (%26)	18 (%12)	0.016*
No	168 (%84)	39 (%74)	129 (%88)
MEDS	9.3	12,11 ± 3.4	8,37 ± 3.9	<0.0001*
SAPSII	41.9	49,6 ± 14.3	39,2 ± 11.5	<0.0001*
SOFA	4	5,2 ± 2.5	3,7 ± 1.7	<0.0001*

3.2. Baseline health status

The chronic disease patterns of the patients were: 24% chronic pulmonary disease, 22% diabetes, 21% malignancy, 19% congestive heart failure, 19% cerebrovascular disease, 14%chronic renal disease, 5% chronic liver disease.

The most frequent infection source was the respiratory tract (47%), followed by the urinary tract (19%); however, the infection sources (respiratory, urinary, intra-abdominal and soft tissue infections) were not a significant factor for mortality (p > 0.05).

Pairwise comparison of the AUCs between the MEDS, SAPSII, and SOFA scores and lactate levels were not significant (p > 0.05, Table 2).

Table 2.

Groups pairwise comparison of ROC curves.

Variable	Difference between areas	95% confidence interval	Significance level
MEDS ∼ SAPS2	0,0512	−0,0312 to 0,134	P = 0,2236
MEDS ∼ SOFA	0,0751	−0,0211 to 0,171	P = 0,1259
MEDS ∼ lactate	0,104	−0,00629 to 0,215	P = 0,0645
SAPS2 ∼ SOFA	0,0532	−0,0498 to 0,156	P = 0,3116
SOFA ∼ lactate	0,0293	−0,0628 to 0,121	P = 0,5333

For mortality prediction, the chosen cutoff values for the SOFA, MEDS, and SAPSII scores and lactate levels were the values corresponding to the best respective Youden's index. The cutoff points were: SOFA 3, MEDS 11, SAPSII 43, and lactate 2.7 mmoL/L.

Scoring systems group comparison with mortality is in Table 3.

Table 3.

Scoring groups comparison with mortality.

Groups	Score	28 days mortalıty
		Survival	Nonsurvival	Mortality %
SOFA group	0–5	122	32	20%
	6–9	25	19	43%
	10–11	0	1	100%
	>11	0	1	100%
SAPS2 group	0–29	26	1	3%
	30–40	61	13	17.5%
	41–52	43	20	32%
	53–64	13	12	48%
	>64	4	7	63%
MEDS group	0–7	64	6	8.5%
	8–12	59	21	26%
	13–15	18	18	50%
	>15	6	8	57%

4. Discussion

International consensus guidelines define sepsis as a systemic inflammatory response in the context of suspected or proven infection, but the management of sepsis patients requires clinical judgement. Scoring systems have implications on the management of patients, admission decision, and admission type. In this study, we aimed to compare scoring systems and lactate levels to predict mortality in sepsis patients. The SOFA, SAPSII, and MEDS systems were not sufficient to predict mortality; MEDS worked better than the other scoring systems.

The MEDS score is aimed at risk stratification of sepsis patients in EDs. Its first derivation and validation were studied by Shapiro et al. The mortality rate was 5.4% and the AUC was 0.8–0.9.⁵Shapiro et al. studied the MEDS-predicted 1-year mortality rates. In that study, they did not measure the serum lactate levels, so they thought that lactate may be improved in the MEDS model.⁶ Sankoff et al. studied the validation of MEDS; the mortality rate was 9% and the AUC was 0.88. They found that the serum lactate concentration AUC was 0.78 (95% CI, 0.66–0.90), but they did not evaluate lactate concentrations in all of their patients. The MEDS score performed better at predicting mortality than serum lactate concentration alone, but the authors submitted that the MEDS score and serum lactate concentration may be combined to improve the ability to predict mortality in patients with SIRS or sepsis.¹³

The present study was a validation of MEDS in Istanbul/Turkey. We evaluated lactate values at the first admission to the ED. Statistically, lactate is significantly associated with mortality but not in pairwise comparisons with MEDS, SOFA, or SAPSII; therefore, lactate did not improve the MEDS model and is not related to mortality.

According to Shapiro et al., MEDS is useful in emergency medicine, but Jones et al. found that MEDS underestimated the risk of mortality and their mortality rates were higher than those of Shapiro et al. and the percentage of mortality groups in MEDS were higher than predicted; therefore, Jones et al. recommended that the MEDS efficiency be estimated in other populations and centers.¹⁴ MEDS validation in Turkey was similar to the literature in that MEDS was superior to the SAPSII and SOFA; in contrast, our MEDS group's predicted mortality percentages were higher than the predicted results.

Kofoed et al. explored predicting the values of the SOFA and SAPSII with systemic inflammatory response syndrome patients. They found that, when the cut-off was.

SAPS II > 22.5 (100% sensitive, 68% specific) and AUC was 0.89 (0.80–0.98) and SOFA >4.5 (44% sensitive, 95% specific) and AUC was 0.80 (0.65–0.94). This study did not find any significant differences with pairwise comparisons of the SOFA and SAPSII.¹¹ Viallon et al. observed that the SAPSII score was correlated with mortality. The mean SAPSII value was compared between the survivor and non-survivor groups; it was 22.17 ± 1.0 in the survivor group and 50.21 ± 3.6 in the non-survivor group.¹⁵

In the present study, the mean value of SAPSII was higher than Viallon et al.’s value in both the survivor and non-survivor groups. Our SAPSII and SOFA AUCs were lower than those of Kofoed et al. In the present study, the initial lactate level significantly predicted mortality. The non-survivor's lactate concentrations were lower than the predicted previous values. Marthy et al. found that mean blood lactate concentrations were lower in survivors than in non-survivors at admission (5 ± 3.1 mmoL/L vs. 6.9 ± 4.3 mmol; p = 0.049).¹⁶

In the present study, the lactate ROC curve was 0.654 and lactate cut-off point was 2.7, so the lactate threshold value was decreased in this study. Shapiro observed that a serum lactate level greater than or equal to 2.5 mmol/L was specific for death and the ROC AUC for lactate level was 0.67, similar to our study, but they also support Bakker in that blood lactate values, with a threshold of 4 mmo/L, were associated with in-hospital mortality.12, 17

Lactate evaluation may need widespread implementation to find the real cut-off value.

5. Limitation

The main limitation of this study relates to MEDS group selection. In the present study, the very low group (0–4 points) had too few patients for statistical analysis, so we placed them in the low group, changing the originally described MEDS groups. An additional limitation of the MEDS score is that a large weighting is given to a subjective assessment of short-term mortality by the treating clinician.¹⁸ The risk scores were not calculated in real time in the ED and were not used to guide clinical care.

Lactate values were obtained in this study, and we also planned to study the 2 h' lactate Clarence but patients' lactate values were not obtained after 2 h or performed several times. Therefore, we did not discuss consistent lactate clearance in all patients; we only obtained an initial lactate level.

Another limitation of the study was power analysis was not carried out.

6. Conclusion

The MEDS score predicts mortality better than SOFA and SAPSII scores. Adding lactate to the model did not improve the MEDS model. MEDS underestimated mortality in the moderate-high risk groups. MEDS needs to be validated for widespread use. These results suggest a need to develop validation and observational studies to correctly evaluate the usefulness of prognostic models in patients with suspected infection in EDs.