SOFA and qSOFA at admission to the emergency department: Diagnostic sensitivity and relation with prognosis in patients with suspected infection

Abstract

Objective

To evaluate the adequacy of SOFA and qSOFA for predicting unfavorable outcomes, and of qSOFA as a screening tool for sepsis in patients admitted to the emergency department (ED) of a Brazilian public hospital.

Methods

This was a single-center retrospective study conducted on a cohort of patients admitted to a Brazilian public hospital between August 2016 and November 2017 due to suspected infection. Exclusion criteria were: age <18 years, admission to the ED after 24 h of hospitalization, lack of information in the medical records, advanced comorbidities, or request of limited invasive care.

Results

A total of 184 patients were included; 84.24% had a SOFA score of 2 or higher. The relative risk of death, need for intensive care unit (ICU) and mechanical ventilation (MV) related to a positive SOFA on admission were: 5.17 (2.11–12.87), 1.45 (1.09–2.15) and 2.74 (1.63–5.16), respectively; sensitivity was 93.7% for death, 88.5% for ICU need and 93.6% for undergoing MV. The mean length of hospital stay was 38.83 days for patients with a positive SOFA score and 8.95 days for patients with a negative score (p = 0.02). The median SOFA value was higher for the patients who died; 41% of the patients had a positive qSOFA and its sensitivity for a positive SOFA was 46.4%. The relative risk of death, ICU and MV need related to qSOFA at admission were 1.83 (1.39–2.44), 0.98 (0.82–1.16) and 1.60 (1.23–1.97), respectively, and its sensitivity was 56.8% for death, 41.4% for ICU need and 53.6% for MV.

Conclusion

qSOFA did not perform well as a screening tool for sepsis and for predicting a poor prognosis in the ED. SOFA, on the other hand, showed reasonable sensitivity for predicting unfavorable outcomes and scores ≥2 were related to a poor prognosis.

1. Introduction

Sepsis is a highly prevalent public health care problem that affects millions of people each year. It accounts for 10% of Intensive Care Unit (ICU) admissions and is associated with 10%–20% in-hospital mortality.¹ Identifying this condition in the emergency room can be challenging because of its complexity and clinical heterogeneity^1,2. Current day protocols establish early detection and aggressive management as the accepted core to best practice for this disease.3, 4, 5 Despite the advances in early recognition and treatment, it still represents one of the major causes of morbidity and mortality worldwide, being the tenth cause of death in the general population.³

Initially, sepsis was defined as a Systemic Inflammatory Response Syndrome (SIRS) caused by infection.^6,7 This definition remained largely unchanged for more than two decades when, in 2016, the Third International Consensus Definitions for Sepsis and Septic Shock established the use of a Sequential (Sepsis-related) Organ Failure Assessment (SOFA) score to identify a life-threatening organ dysfunction caused by a dysregulated host response to infection. Indeed, sepsis diagnosis is based on an increase of at least 2 points in this score.⁸

SOFA is also used to assess the severity of the disease in critically ill patients and to predict their prognosis. Created in 1996, it works based on the sequential analysis of the respiratory, coagulation, hepatic, cardiovascular, renal and central nervous systems, evaluating the degree of organic dysfunction.^7,9,10

Despite its usefulness and the promising results shown by many ICU studies, the SOFA score has not been previously assessed for diagnostic purposes in developing countries and there are few data about its mortality prediction in non-ICU settings, such as Emergency Departments (ED), where most patients are admitted. Nonetheless, it should be mentioned that SOFA requires laboratory evaluation, which makes it more time consuming.^11,12 Delays in starting treatment worsen the prognosis and impair the Early Goal Directed Therapy (EGDT), particularly a prompt antibiotic therapy.^1,2

The latest Sepsis Consensus also introduced a variation of SOFA named quick SOFA (qSOFA) as a screening tool for patients likely to have sepsis.⁸ It consists of 3 clinical variables: systolic blood pressure of 100 mmHg or less, respiratory rate of 22 breaths/minute or higher, and altered mentation — originally described as Glasgow Coma Scale <15. The presence of 2 or more of these variables is considered to represent a positive test.²

Recently, studies that intended to evaluate the accuracy of qSOFA in predicting a poor prognosis of sepsis in prehospital and emergency settings have reached conflicting conclusions and many questions have been asked about the usefulness of qSOFA.^2,8,13

The use of SOFA and qSOFA for the diagnosis of sepsis requires the issue of quick results for prompt therapeutic decision making and a high sensitivity for a poor prognosis upon the first evaluation at the emergency department (ED), where most patients diagnosed with sepsis are admitted.³ Therefore, the objective of the present study was to evaluate the adequacy of SOFA and qSOFA in predicting unfavorable outcomes, and specifically the use of qSOFA as a screening tool for sepsis, in patients admitted to a tertiary emergency department of a Brazilian public hospital.

2. Methods

2.1. Study design and setting

This was a single-center retrospective cohort study of patients admitted to the emergency department of Instituto Hospital de Base between August 2016 and November 2017. This hospital is a 700 bed tertiary-care teaching facility considered to be a referral center in the capital of Brazil, Brasília. The city has a mainly urban and suburban population of 3,039,444 citizens.

The study was evaluated and approved by the Ethics Committee of Fundação de Ensino e Pesquisa em Ciências da Saúde do Distrito Federal (FEPECS). Informed consent was waived because the study involved a retrospective, observational and anonymous cohort.

2.2. Population and data sources

Data were collected from an electronic medical records network (Trakcare by InterSystems) used by the public health care system of Brasília. The charts were analyzed by four medical students and reviewed by an attending doctor.

Upon admission to the ED, the following data were obtained for each recruited patient, Glasgow Coma Scale (GCS), respiratory rate (RR), systolic blood pressure (SBP), arterial blood gas, platelet count, mean arterial pressure and use of vasopressors, bilirubin levels, urine output, and creatinine levels. The primary outcome was in-hospital mortality. Secondary end-points included length of hospitalization, admission to the ICU and need for mechanical ventilation support. Demographic data such as age and sex were also collected.

Patients primarily admitted to the ED due to suspected infection without an alternative diagnosis were included. The infection was confirmed with the final diagnosis recorded on the discharge form which was based on typical clinical characteristics, use of antibiotic therapy and the identification of the source of infection through laboratory, microbiological or imaging data.

Exclusion criteria were: age <18 years, admission to the ED after 24 h of hospitalization, refractory shock or cardiopulmonary arrest before admission, lack of information in the medical records, advanced comorbidities or request for limited invasive care. We considered as advanced comorbidities: metastatic cancer without treatment perspective, chronic liver disease (CHILD-PUGH C), previous stroke resulting in severe limitation of self-care, advanced dementia, chronic renal failure requiring dialysis, congestive cardiac failure with an ejection fraction < 30%, and chronic lung disease requiring continuous oxygen therapy.

The qSOFA score ranges from 0 to 3 with one point attributed to each of the following clinical signs: RR ≥ 22/min, SBP ≤100 mmHg and altered GCS from baseline. A score ≥2 was considered positive. The SOFA score ranges from 0 to 24, with a maximum of 4 points attributed to each of the 6 variables that ultimately represent the functionality of the cardiovascular, renal, respiratory, hepatic, coagulation and neurological systems. A score ≥2 was considered positive.

2.3. Statistical analysis

Nominal variables were summarized as frequency counts and percentages, ordinal and discrete variables as median and IQR (25-75%) and continuous variables as mean and standard deviation. Comparisons were made between two groups of patients categorized according to whether sepsis criteria were met or not in the ED (yes/no). Differences in normally distributed continuous variables were analyzed by the Student T Test while differences related to ordinal and discrete variables were determined using the non-parametric Mann-Whitney U test. The Chi-Squared or Fisher's Exact test was used to compare nominal variables between groups. The clinical performance of both scores was assessed using mainly sensitivity, specificity, relative risk and likelihood ratio values (with 95% confidence intervals, CIs).

All statistical tests were two-sided and conducted at the 0.05 level of significance. Data analysis were conducted using SAS Graphpad Prism 7 for Windows and online Medcalc calculator. The statistical power of the tests was calculated by G*Force software and was considered to be valid if greater than 80%.

3. Results

A total of 695 medical records were analyzed, 184 of which containing confirmed infections with the necessity of hospital admission were included (Fig. 1). Mean patient age was 58.8 years, with a predominance of male sex (58.15%). Overall sample mortality was 51.63%, and admission to the ICU and mechanical ventilation were needed in 76.09% and 67.93% of cases, respectively. The most prevalent sites of infection were respiratory (51.63%), urinary (24.46%) and abdominal (10.33%) (Table 1).

Fig. 1.

Patient flowchart.

Table 1.

Characteristics of the study population.

Demographic Data
Mean Age (SD)	58.78 (20.6)
Male sex (%)	58.15
Site of Infection
Pulmonary infection (%)	51.63
Urinary infection (%)	24.46
Intra-abdominal infection (%)	10.33
Soft tissue infection (%)	4.89
Bloodstream infection (%)	4.35
CNS infection (%)	3.8
Osteoarticular infection (%)	0.54
SOFA and qSOFA scores on admission
Median SOFA (IQR 25–75%)	6 (4-9)
Median qSOFA (IQR 25–75%)	1 (1-2)
Major Outcomes Evaluated
Need of MV at the ED (%)	67.93
Mean hospitalization time in days (SD)	27.00 (54.2)
Mean length of ICU stay in days (SD)	12.38 (193)
Mortality (%)	51.63

When the sample was divided into two groups of patients - survivors and non-survivors - significant differences were observed in some vital signs, laboratory tests and secondary outcomes, as shown in Table 2. Median SOFA (8.00 × 6.00 p = 0.0005) and median qSOFA (1 × 2 p < 0.01) scores were higher in patients who died.

Table 2.

Clinical and demographic characteristics of survivors and non-survivors.

	Survivors (n = 89)	SD or IQR25-75%	Non-survivors (n = 95)	SD or IQR25-75%	p value
Mean age (years)	55.54	21.1	61.81	19.6	0.02
Male sex (%)	55.05	-	61.05	-	0.42
Hemodynamic instabilitya (%)	27.3	-	42.1	-	0.02
Need of MVb (%)	42.7	-	91.6	-	< 0.01
Median RRc	20.00	18–23	23.00	20–24	< 0.01
Mean SBPd	130.00	39.9	124.2	37.9	0.16
Median platelets count (x 10³)	203.00	144–269	191.00	129–255	0.07
Median GCSe	14.00	9–15	10.00	3–14	< 0.01
Mean SpO2 (%)	94.7	6.8	94.1	8.8	0.28
Mean creatinine levels	1.84	2.1	2.10	2.8	0.16
Mean bilirubin levels	1.21	1.6	1.83	1.4	0.03
Mean hospitalization time (days)	28.82	43.58	25.29	62.78	0.33
Mean length of ICUf stay (days)	10.5	14.8	14.1	22.3	0.10
Median SOFA	5.0	2–7	8.0	5–9	< 0.01
Median qSOFA	1.0	1–2	2.0	1–2	< 0.01

^aMean arterial pressure < 70 or need for vasoactive drugs/

^bMechanical ventilation.

^cRespiratory rate/

^dSystolic blood pressure.

^eGlasgow Coma Scale/

^fIntensive care unit.

At admission, 155 (84.24%) patients had a SOFA score of 2 or higher. The relative risk of death, need for intensive care and mechanical ventilation related to the positive SOFA on admission were: 5.17 (95% CI: 2.11–12.87), 1.45 (95% CI: 1.09–2.15) and 2.74 (95% CI: 1.63–5.16), respectively; sensitivity was 93.7% for death, 88.5% for ICU need and 93.6% for undergoing mechanical ventilation. The mean length of hospital stay was 38.83 days for patients with a positive SOFA score and 8.95 for patients wuth a negative score (p = 0.02) (Table 3).

Table 3.

Performance of qSOFA and SOFA regarding screening for mortality.

	qSOFA		SOFA
	Value	95% CIa	Value	95% CIa
Sensitivity (%)	56.84	46.2–66.97	93.68	86.76–97.65
Specificity (%)	74.16	63.79–82.86	25.85	17.14–36.21
Positive predictive value (%)	70.13	61.31–77.67	57.42	54.13–60.64
Negative predictive value (%)	61.68	55.35–67.64	79.31	62.08–89.97
Positive Likelihood Ratio	2.20	1.48 to 3.26	1.26	1.11–1.44
Negative Likelihood Ratio	0.58	0.45 to 0.76	0.24	0.10–0.57

^aConfidence interval.

At admission, 77 patients (41%) had a positive qSOFA (≥2). The relative risk of death, need for intensive care and mechanical ventilation related to a positive qSOFA on admission were 1.83 (95% CI: 1.39–2.44), 0.98 (95% CI: 0.82–1.16) and 1.60 (95% CI: 1.23–1.97), respectively, with 56.8% sensitivity for death, 41.4% for ICU need and 53.6% for undergoing mechanical ventilation (Table 3).

The relative risk of a qSOFA greater than 2 being related to a positive SOFA was 1.21 (95% CI: 1.07–1.37), its sensitivity was 46.4% and its predictive value was 93.5% for a positive SOFA and 22.46% for a negative SOFA.

The statistical power related to total sample size was 88%.

4. Discussion

In a retrospective attempt to validate Sepsis-3, our results showed that a positive SOFA was related to a five-fold increase in the risk of death (5.17, 95% CI: 2.11–12.87) with 93.7% sensitivity for predicting mortality, data similar to those reported by Anami et al. and Rosa et al. respectively, in Brazilian intensive care units (ICU).^14,15 The literature has demonstrated a correlation between higher SOFA scores and increased in-hospital mortality rates, both in ICU settings^1,14 and in emergency departments.^3,9 Likewise, after dividing our patients into survivor and non-survivor groups, we observed higher SOFA scores among the latter (8.00 × 6.00 p = 0.0005).

Since clinical deterioration has been shown to be an independent predictor of mortality, another prognostic factor we analyzed was the need for mechanical ventilation (MV).¹⁶ Patients who require MV have higher vulnerability and may exhibit not only sepsis-associated hypoxemic respiratory failure, but also lower levels of consciousness, two important signs of organ dysfunction. In the present study a SOFA ≥2 on admission was related to a double relative risk of undergoing invasive ventilation (2.74, 95% CI: 1.63–5.16) with an expressive sensitivity (93.6%). Similar results were reported by Rosa et al. who demonstrated high rates of MV in the presence of a positive SOFA and an 88.2% sensitivity of the score for this outcome.¹⁵

Our data showed a high sensitivity of the score in predicting ICU admission (88.5%) and a relative risk of 1.45 (95% Cl: 1.09–2.15) for this outcome. Raith et al. detected a correlation between SOFA score, mortality and a prolonged stay in the ICU.¹⁷ Although our study did not detect this correlation, patients with a SOFA score of 2 or more had higher mortality rates compared to patients with lower scores, a fact that may have reduced their time in the ICU. It is also important to highlight that patients with a positive SOFA had a four-fold increase in total hospitalization time, which can indicate a higher rate of complications.

Despite the good prognostic value shown by SOFA in this cohort, logistical concerns should be raised since 4 of its 6 variables require laboratory measurements that can be often time-consuming and may not always be available depending on the features of different scenarios. The feasibility of such laboratory tests is a concern especially in low and middle income countries with resource-constrained settings.^16,18,19

An early identification of septic patients after arrival at the ED can be particularly challenging in an overcrowding situation. Delays in diagnosis can affect clinical management, resource allocation and prognosis.^3,11,20 Although the Sepsis 3 Task-Force has suggested the qSOFA as a bedside criterion that would prompt organ dysfunction investigation, it has also stated the need to validate the new proposed definitions and the utility of this score using non-US databases.^6,21

We found that a positive qSOFA at admission had a low sensitivity (56.8%) in predicting in-hospital mortality. Other studies that evaluated the sensitivity of the admission score for the same outcome reached similar results despite the time set for mortality analysis: Tusgul et al. found that qSOFA from admission had 68% sensitivity for mortality within 48 h, Rodriguez et al. showed a sensitivity of 64.4% within 72 h of hospitalization, and Hwang et al. a senstivity of 39% within 28 days, demonstrating the fragility and ineffectiveness of its use for an early recognition of critically ill patients (Table 4).^2,22, 23, 24

Table 4.

Reported qSOFA sensitivities.

	Tusgul et al.	Rodriguez et al.	Hwang et al.	Present study
Mortality	68% (48h)	64.4% (72h)	39% (28d)	56.8%c
ICU a	31.2%	54.4%	31%	41.4%
MV b	–	–	44.5%	53.6%

^aIntensive care admission.

^bNeed of mechanical ventilation.

^cIn-hospital mortality (despite the time of hospitalization).

Still considering the predictive capacity of qSOFA for the mortality outcome, a recent cohort study reported that when qSOFA was also obtained after 24 h of hospitalization its sensitivity increased significantly (91%).² However, considering that qSOFA was supposed to be used as a screening tool, the most important results are those obtained at admission.

In our sample, a positive qSOFA was not related to an increased risk of ICU admission (0.98, 95% CI: 0.82–1.16). qSOFA also performed poorly in identifying the need for intensive care, showing a low sensitivity of 41.4%. Other recent studies that analyzed the ability of this score to anticipate the need for intensive care reported similar results. Tusgul et al. detected a qSOFA sensitivity of 31.2% to predict ICU admission. Rodriguez et al. in a multicenter study, described the outcomes of infected patients and reported a sensitivity of 54.4% for ICU admission within 72 h of hospitalization. Hwang et al. assessed qSOFA repeatedly within the first 24 h after admission and found that its sensitivity at ED arrival, 31%, was the poorest one for ICU admission.^2,22,23

Our study also showed that a positive qSOFA is associated with a higher risk of needing mechanical ventilation (MV): 1.60 (95% CI: 1.23–1.97). However, the sensitivity for this outcome was only 53.6%. Likewise, Hwang et al. found that qSOFA at ED admission was not sensitive enough to screen for the need of MV (44.5%).²

qSOFA may vary temporally in the diagnosis of sepsis and concerns have been raised about the low sensitivity of this score to be employed as an early sepsis screening tool.²⁴ In our analysis, not only did the score prove to be ineffective in predicting bad outcomes, but it also performed poorly in predicting a positive SOFA, showing only 46.6% sensitivity.

4.1. Limitations

Some limitations of the present study should be acknowledged. First, since our study involved a single center retrospective cohort it was susceptible to selection bias. Second, it was conducted in a tertiary-care hospital, thus mostly including critically ill patients. Third, data were collected by analysis of the medical records, which may have been susceptible to variations depending on the evaluator's perception. The strengths of our study include the evaluation of SOFA and q-SOFA in an Emergency Department in a middle-income country and the assessment of other relevant outcomes besides mortality.

5. Conclusions

We concluded that the sensitivity of qSOFA was not appropriate enough for the score to be used as a screening tool for sepsis and for a poor prognosis in the emergency department. In contrast, SOFA showed a good performance for predicting mortality, need of ICU admission and need for mechanical ventilation. However, this score requires laboratory tests that may not be available in the ED, a fact that can delay the identification and treatment of sepsis. Therefore, our data suggest that more studies are necessary to solve the problem of a prompt diagnosis of sepsis in EDs of low income countries.

Funding

All funding was provided by Foundation of Education and Research in Health Sciences, Brasília - Brazil.

Author contribution statement

All authors verify and confirm that everyone who contributed to this manuscript is either listed as an author or acknowledged as a contributor in the acknowledgement section, and that the title page details any professional writing assistance or others paid to provide manuscript support.

All Authors have made important contributions to conceiving, designing and performing the analyses and interpretation of the data; providing materials, analysis tools or data and writing the paper.

Conflict of interest statement

The authors declare no conflict of interest.