Does SOFA predict outcomes better than SIRS in Brazilian ICU patients with suspected infection? A retrospective cohort study

Abstract

We compared the discriminatory capacity of the sequential organ failure assessment (SOFA) versus the systemic inflammatory response syndrome (SIRS) score for predicting ICU mortality, need for and length of mechanical ventilation, ICU stay, and hospitalization in patients with suspected infection admitted to a mixed Brazilian ICU. We performed a retrospective analysis of a longitudinal ICU database from a tertiary hospital in Southern Brazil. Patients were categorized according to whether they met the criteria for sepsis according to SOFA (variation ≥2 points over the baseline clinical condition) and SIRS (SIRS score ≥2 points). From January 2008 to December 2014, 1487 patients were admitted to the ICU due to suspected infection. SOFA ≥2 identified more septic patients than SIRS ≥2 (79.0% [n = 1175] vs. 68.5% [n = 1020], p < 0.001). There was no difference between the two scores in predicting ICU mortality (area under the receiver operating characteristic curve (AUROC) = 0.64 vs. 0.64, p = 0.99). SOFA ≥2 was marginally better than SIRS ≥2 in predicting need for mechanical ventilation (AUROC = 0.64 vs. 0.62, p = 0.001), ICU stay > 7 days (AUROC = 0.65 vs. 0.63, p = 0.004), and length of hospitalization >10 days (AUROC = 0.61 vs. 0.59, p < 0.001). There was no difference between the two scores in predicting mechanical ventilation >7 days.

Sepsis results in significant mortality,1, 2, 3 morbidity, and resource utilization during and after critical illnesses.⁴

The previous consensus definitions of Sepsis (Sepsis-1 and Sepsis-2)5, 6 relied on the systemic inflammatory response syndrome (SIRS) to infection as a fundamental aspect of sepsis diagnosis. In 2016, the Sepsis-3 definition changed the focus from the presence of infection-related SIRS to the presence of infection-related organ dysfunction.⁷ The new consensus was mainly supported by the retrospective study by Seymor et al.,⁸ which showed that sequential organ failure assessment (SOFA) resulted in better predictive accuracy for mortality than SIRS among US and European ICU patients.

However, some health care societies⁹ and experts¹⁰ have expressed disappointment with the new recommendations, mainly due to the lack of studies on the performance of the organ dysfunction scores in developing countries. Thus, to safely adopt the new criteria proposed by Sepsis-3 in a broader context, it is necessary to validate it in scenarios different from those originally tested. In addition, other outcomes relevant to critical care patients besides death should be assessed. Therefore, we aimed to compare the discriminatory capacity of SOFA versus SIRS for predicting relevant outcomes among adult patients admitted to a mixed intensive care unit (ICU) of a tertiary hospital in Southern Brazil due to suspected infection.

We performed a retrospective analysis of a comprehensive prospectively collected longitudinal ICU database composed of 4221 patients admitted to the 31-bed mixed ICU of Hospital Moinhos de Vento, Porto Alegre, Brazil, over a 7-year period (January 2008 to December 2014). This study was approved by the Research Ethics Committee at Hospital Moinhos de Vento. The need for informed consent was waived. All patients with suspected infections at ICU admission were considered eligible for the study. Patients were identified using infection, sepsis, severe sepsis, septic shock, pneumonia, urinary tract infection, bloodstream infection, intra-abdominal infection, central nervous system infection, osteoarticular infection, and skin and soft tissue infection as search terms in the syndromic diagnosis field of the database.

The exposure variables were diagnosis of sepsis according to SOFA and SIRS definitions at ICU admission. Patients were classified as having sepsis according to SOFA if they had a score variation ≥2 points over the baseline clinical condition.⁷ The SOFA was assumed to be zero in patients not known to have preexisting organ dysfunction. In patients with chronic organ dysfunction, the baseline SOFA was assumed to be 4 in patients undergoing chronic renal replacement therapy, and 2, 3 or 4 in patients with cirrhosis, depending on baseline bilirubin levels. Patients were classified as having sepsis according to SIRS if they presented at least two of the following signs of systemic inflammation: temperature > 38 °C or <36 °C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per minute or PaCO₂ < 32 mmHg, abnormal white blood cell count (>12,000/μL or <4000/μL or >10% immature forms).⁵ In order to evaluate the prognosis of patients according to the presence or absence of organ dysfunction (SOFA variation ≥2 points over the baseline clinical condition) and SIRS (SIRS score ≥2 points) we created four groups: (1) patients with organ dysfunction and SIRS; (2) patients with SIRS but without organ dysfunction; (3) patients with organ dysfunction but without SIRS; and (4) patients with infection, but with neither SIRS nor organ dysfunction.

The outcomes evaluated were all-cause ICU mortality, need and length of for invasive mechanical ventilation (MV), length of ICU stay, and length of hospitalization. Prolonged MV, ICU stay, and hospitalization were defined as duration of these variables above the 75th percentile of the studied population. The accuracy of the different sepsis definitions for predicting these outcomes was evaluated through the area under the receiver operating characteristic (ROC) curve (AUROC). Univariate modeling of the two definitions was compared using the Chi-squared test. A comparison of outcomes among the study groups was made using the Kruskal–Wallis test for continuous variables and the Pearson's Chi-squared test for dichotomous variables. Among post hoc tests, Dunn's test was applied after the Kruskal–Wallis test, and Bonferroni correction was applied after the Pearson's Chi-squared test. A significance level of 0.05 was adopted for all comparisons. Stata v. 12 (StataCorp LP, College Station, TX, USA) was used for statistical analysis.

SOFA ≥2 identified a higher number of septic patients in comparison to the SIRS ≥2 (79.0% [n = 1175] vs. 68.5% [n = 1020], p < 0.001). Baseline characteristics of patients according to the different definitions of sepsis were comparable. There were no statistical differences in the proportion of males (SOFA ≥2, 55.6% vs. SIRS ≥2, 54.0%), median age (SOFA ≥2, 75.0 [interquartile range {IQR}, 63.0–83.0] vs. SIRS ≥2, 75.0 [IQR, 63.0–83.0]), median number of comorbidities (SOFA ≥2, 3.0 [IQR, 2.0–4.0] vs. SIRS ≥2, 3.0 [IQR, 2.0–4.0]), median APACHE-II score at ICU admission (SOFA ≥2, 21.0 [IQR, 16.0–25.0] vs. SIRS ≥2, 21.0 [IQR, 16.0–26.0]), and median SOFA score at ICU admission (SOFA ≥2, 5.0 [IQR, 3.0–8.0] vs. SIRS ≥2: 5.0 [IQR, 3.0–8.0]).

A comparison of the predictive accuracy for the study outcomes between SOFA ≥2 and SIRS ≥2 is shown in Fig. 1. SOFA ≥2 and SIRS ≥2 showed similar discriminatory capacity for ICU mortality (AUROC 0.64 [0.62–0.67] vs. 0.64 [0.62–0.67], p = 0.99). SOFA ≥2 had a marginally better discriminatory capacity than SIRS ≥2 for need for MV (AUROC 0.64 [95% CI, 0.62–0.65] vs. 0.62 [95% CI, 0.61–0.63], p = 0.001), length of ICU stay > 7 days (AUROC 0.65 [95% CI, 0.63–0.66] vs. 0.63 [95% CI, 0.62–0.64], p = 0.004), and length of hospitalization >10 days (AUROC 0.61 [95% CI, 0.60–0.63] vs. 0.59 [95% CI, 0.58–0.61], p < 0.001). There was no difference between the scores in terms of the predicting length of MV >7 days.

Fig. 1.

Comparison of predictive accuracy for outcomes among SOFA and SIRS criteria defining sepsis in 1487 critical care patients with infection at ICU admission. Abbreviations: AUROC, area under receiver operating characteristic curve; CI, confidence interval; ICU, intensive care unit; MV, mechanical ventilation.

Note: SOFA+, patients with SOFA variation ≥2 over their baseline clinical scores. SIRS+, patients with at least 2 of the following signs of SIRS: temperature > 38 °C or <36 °C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per minute or PaCO₂ < 32 mmHg, abnormal white blood cell count (>12,000/μL or <4000/μL or >10% immature forms).

ICU mortality: SOFA + AUROC 0.64 (95% CI, 0.62–0.67); SIRS + AUROC 0.64 (95% CI, 0.62–0.67). Need for MV: SOFA + AUROC 0.64 (95% CI, 0.62–0.65); SIRS + AUROC 0.62 (95% CI, 0.61–0.63). Length of MV >7 days: SOFA + AUROC 0.57 (95% CI, 0.55–0.60); SIRS + AUROC 0.58 (95% CI, 0.56–0.61). ICU stay >7 days: SOFA + AUROC 0.64 (95% CI, 0.62–0.65); SIRS + AUROC 0.62 [95% CI, 0.61–0.63). Hospital stay >10 days: SOFA + AUROC 0.61 (95% CI, 0.60–0.63); SIRS + AUROC 0.59 (95% CI, 0.58–0.61).

Table 1 shows a comparison of the outcomes according to the presence or absence of organ dysfunction and SIRS. Patients with both organ dysfunction and SIRS had worst outcomes compared to patients of other groups: higher ICU mortality, higher need of MV, and longer ICU stay. The comparison of outcomes among patients with SIRS but without organ dysfunction and patients with organ dysfunction but without SIRS showed no statistical difference. Patients with neither organ dysfunction nor SIRS had better outcomes: lower rates of mortality and MV and shorter length of ICU stay in comparison to patients from other groups.

Table 1.

Comparison of outcomes among SOFA and SIRS criteria defining sepsis in 1487 critical care patients with infection at ICU admission.

Outcomes	Group 1 SIRS+	Group 2 SIRS+	Group 3 SIRS−	Group 3 SIRS−	Kruskal–Wallis or Pearson Chi2	Comparison group	Post hoc p-value
	SOFA+	SOFA−	SOFA+	SOFA−
	(n = 867)	(n = 153)	(n = 308)	(n = 159)
ICU mortality, % (n)	26.8 (233)	16.3 (25)	14.2 (44)	5.6 (9)	Chi2 51.1	1 vs. 2	0.03
					p < 0.001	1 vs. 3	<0.001
						1 vs. 4	<0.001
						2 vs. 3	0.44
						2 vs. 4	0.01
						3 vs. 4	0.03
Need for MV, % (n)	68.5 (594)	45.7 (70)	54.2 (167)	20.1 (32)	Chi2 144.8	1 vs. 2	<0.001
					p < 0.001	1 vs. 3	<0.001
						1 vs. 4	<0.001
						2 vs. 3	0.18
						2 vs. 4	<0.001
						3 vs. 4	<0.001
Length of MV, days, median (IQR)	6 (3, 13)	6 (2, 11)	4 (1, 10)	2 (1, 7)	Chi2 19.3	1 vs. 2	0.90
					p < 0.001	1 vs. 3	0.004
						1 vs. 4	0.002
						2 vs. 3	0.22
						2 vs. 4	0.10
						3 vs. 4	0.32
Length of ICU stay, days, median (IQR)	10 (5, 19)	7 (4, 15)	7 (4, 15)	4 (2, 9)	Chi2 83.5	1 vs. 2	<0.001
					p < 0.001	1 vs. 3	<0.001
						1 vs. 4	<0.001
						2 vs. 3	0.99
						2 vs. 4	<0.001
						3 vs. 4	<0.001
Length of hospitalization, days, median (IQR)	18 (10, 34)	14 (8, 29)	16 (9, 32)	11 (7, 22)	Chi2 29.9	1 vs. 2	0.15
					p < 0.001	1 vs. 3	0.62
						1 vs. 4	<0.001
						2 vs. 3	0.68
						2 vs. 4	0.42
						3 vs. 4	<0.001

ICU, intensive care unit; IQR, interquartile range; MV, mechanical ventilation; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment.

Note: SOFA+, patients with SOFA variation ≥2 over their baseline clinical score. SIRS+, patients with at least 2 of the following signs of SIRS: temperature > 38 °C or <36° C, heart rate > 90 beats per minute, respiratory rate > 20 breaths per minute or PaCO₂ < 32 mmHg, abnormal white blood cell count (>12,000/μL or <4000/μL or >10% immature forms).

In this study constituted of Brazilian critical care patients with suspected infections who were admitted to a mixed medical-surgical ICU, the SOFA ≥2 criteria identified more patients with sepsis than the SIRS ≥2 criteria. There was no difference between the two scores in predicting ICU mortality; however, SOFA ≥2 showed a marginally better predictive accuracy for need for MV, length of ICU stay, and length of hospitalization in comparison to the SIRS ≥2 definition. The prognosis of patients classified as having sepsis by SOFA ≥2 but not by SIRS ≥2 (n = 308), was generally poorer than the prognosis of patients with infection but without SIRS or organ dysfunction. Conversely, some patients previously classified as having sepsis by SIRS ≥2 were not captured by the SOFA ≥2 criteria (n = 153), and these patients also had poorer prognosis in comparison to patients with infection but without SIRS or organ dysfunction.

Sepsis definitions based on SIRS criteria have been questioned due to low specificity.¹¹ Critical care patients may present signs of SIRS due to distinct causes of infection such as surgery, trauma, inflammatory conditions, blood transfusions, and medication use.¹² Conversely, immunosuppressed patients may not manifest SIRS even in more severe cases of infection due to an impaired host response to pathogens.¹³

In this sense, SOFA ≥2 criteria identified a higher number of septic patients in comparison to the SIRS ≥2 criteria. This finding may be related to the characteristics of our population, given that we evaluated mainly older patients with a relevant comorbidity burden. Particularly in this population, the signs of SIRS tend to be less prevalent due to weaker immune system activation.¹⁴ This hypothesis is supported in a study by Kaukonen et al., in which up to one in every eight patients with infection and organ dysfunction did not meet the SIRS criteria.¹⁵

Raith et al.¹⁶ showed that an increase in SOFA score of 2 or more points had a better prognostic accuracy for in-hospital mortality than SIRS criteria among adults with suspected infection admitted to Australian and New Zealand ICUs, suggesting that SIRS criteria may have limited utility for predicting mortality in the critical care setting. Similar results were found in a study by Cheng et al.,¹⁷ in which Sepsis-3 showed better predictive accuracy for in-hospital mortality than Sepsis-1 among ICU patients in China. The Brazilian study by Besen et al.¹⁸ showed that the new Sepsis-3 definitions were superior to the previous definitions in stratifying mortality as well. Our study failed to show a difference in the predictive accuracy for ICU mortality between SOFA ≥2 criteria and SIRS ≥2 criteria; however, the better performance of SOFA ≥2 criteria for predicting other important outcomes (i.e., need of MV, length of ICU, and hospital stay) indicates that ΔSOFA may be a useful index to assess the impact of therapies on clinically and economically relevant outcomes in ICUs. Interestingly, the use of SIRS ≥2 criteria identified a considerable number of patients with high mortality rates not classified as having sepsis by the SOFA ≥2 criteria. Also, the use of SIRS ≥2 in addition to the SOFA criteria allowed the identification of a subgroup of patients with both SIRS and organ dysfunction, which had the worst prognosis.

Some limitations must be acknowledged. First, a retrospective study design is susceptible to selection bias; notably the use of search terms in the ICU database as a strategy to identify patients admitted due to suspected infections may have contributed to selection bias given its low sensitivity. Second, this study was performed at a single center, which may limit its external validity. Third, we did not evaluate the impact of the different definitions of sepsis on long-term outcomes of ICU patients. The strengths of our study include the substantial number of patients evaluated and the assessment of relevant outcomes in addition to mortality.

In conclusion, in the ICU setting, the definition of sepsis based on the presence of organ dysfunction measured by SOFA identified more septic patients and was marginally superior to the definition based on the presence of SIRS criteria for the prediction of the need for MV, a prolonged length of stay in the ICU, and prolonged hospitalization time. These data externally validate SOFA as a diagnostic and prognostic tool in critically ill patients in a setting other than those originally published. However, the SIRS score is still important in stratifying the severity of patients with infection and should not be abandoned.

Conflicts of interest

The authors declare no conflicts of interest.