Contrasting qSOFA and SIRS Criteria for Early Sepsis Identification in a Veteran Population

Abstract

The quick Sequential Organ Failure Assessment lacks sensitivity to be an effective replacement for the Systemic Inflammatory Response Syndrome criteria for sepsis screening.

Sepsis is a major public health concern: 10% of patients with sepsis die, and mortality quadruples with progression to septic shock.¹ Systemic inflammatory response syndrome (SIRS) criteria, originally published in 1992, are commonly used to detect sepsis, but as early as 2001, these criteria were recognized as lacking specificity.² Nonetheless, the use of SIRS criteria has persisted in practice. Sepsis was redefined in Sepsis-3 (2016) to guide earlier and more appropriate identification and treatment, which has been shown to greatly improve patient outcomes.^1,3 Key recommendations in Sepsis 3 included eliminating SIRS criteria, defining organ dysfunction by the Sequential Organ Failure Assessment (SOFA) score, and introducing the quick SOFA (qSOFA) score.¹

The qSOFA combines 3 clinical variables to provide a rapid, simple bedside score that measures the likelihood of poor outcomes, such as admission to an intensive care unit (ICU) or mortality in adults with suspected infection.^1,3 The qSOFA score is intended to aid health care professionals in more timely stratification of those patients who need escalated care to prevent deterioration.¹ The assessment also has been explored as a screening tool for sepsis in clinical practice; however, limited data exist concerning the comparative utility of qSOFA and SIRS in this capacity, and study results are inconsistent.^4–6

The most important attribute of a screening tool is high sensitivity, but high specificity also is desired. The qSOFA could supplant SIRS as a screening tool for sepsis if it maintained similarly high sensitivity but achieved superior specificity. Therefore, our primary objective for this study was to determine the effectiveness of qSOFA as a screening assessment for sepsis in the setting of a general inpatient medicine service by contrasting the sensitivity and specificity of qSOFA with SIRS in predicting sepsis, using a retrospective chart review design.

METHODS

Administrative data from the US Department of Veterans Affairs (VA) Corporate Data Warehouse were accessed via the VA Informatics and Computing Infrastructure (VINCI) and used to identify VA inpatient admissions and obtain the laboratory and vital sign data necessary to calculate SIRS, qSOFA, and SOFA scores. The data were supplemented by manual review of VA health records to obtain information that was not readily available in administrative records, including septic shock outcomes and laboratory and vital sign data obtained in the ICU. This study was approved by the institutional review board at the University of Iowa and the research and development committee at the Iowa City VA Medical Center (ICVAMC).

Patients

The study population included veterans admitted to the nonsurgical medicine unit at ICVAMC between August 1, 2014 and August 1, 2016, who were transferred to an ICU after admission; direct ICU admissions were not included as the qSOFA has been shown in studies to be more beneficial and offer better predictive validity outside the ICU. Excluding these direct admissions prevented any potential skewing of the data. To control for possible selection bias, veterans also were excluded if they transferred from another facility, were admitted under observation status, or if they had been admitted within the prior 30 days. These patients may have been more critically ill than those who presented directly to our facility and any prior treatment could affect the clinical status of the patient and assessment for sepsis at the time of presentation to the VA. Veterans were further required to have evidence of suspected infection based on manual review of the health record, which was determined by receipt of an antibiotic relevant to the empiric treatment of sepsis within 48 hours of admission.

Sepsis and Septic Shock Assessment Tools

As outlined in the Sepsis-3 guidelines, sepsis was defined as suspected or confirmed infection with an acute change in the SOFA score of ≥ 2 points, which is assumed to be 0 in those not known to have preexisting dysfunction.¹ The SOFA score includes variables from the respiratory, coagulation, hepatic, cardiovascular, renal, and central nervous systems.¹ Septic shock was defined as vasopressor administration and a serum lactic acid level > 2 mmol/L occurring up to 24 hours apart and within 3 days of the first antibiotic dose administered.

The SIRS assessment includes 4 clinical variables (temperature, heart rate, respiratory rate, and white blood cell count), while qSOFA is composed of 3 variables (respiratory rate, systolic blood pressure, and altered mental status).¹ With both assessments, a score ≥ 2 is considered positive, which indicates increased risk for sepsis in patients with suspected infection.¹ In keeping with existing studies, qSOFA and SIRS assessments were scored using maximum values found within 48 hours before and 24 hours after the first administered antibiotic dose.³

Outcomes

The primary outcome variable was the presence of sepsis in adults with evidence of infection within 48 hours of admission. Secondary outcome measures included 30-day mortality and septic shock.

Performance between the SIRS and qSOFA assessments was contrasted using sensitivity, specificity, and positive and negative predictive value measurements. Associations of qSOFA and SIRS with septic shock and 30-day mortality were evaluated using a 2-tailed Fisher exact test with a threshold of α = 0.05 to determine statistical significance.

RESULTS

The study sample of 481 veterans had a mean age of 67.4 years, 94% were male, and 91.1% were white (Table 1). When predicting risk for sepsis, the qSOFA demonstrated lower sensitivity than SIRS (44.7% vs 80.0%) but higher specificity (83.6% vs 25.7%) and higher positive predictive value (75.5% vs 54.8%) than did SIRS (Table 2). Specificity and positive predictive value results indicated a good probability that veterans with positive qSOFA assessments actually had sepsis.

TABLE 1.

Baseline Characteristics of Veterans Admitted to a Nonsurgical Inpatient Medicine Unit With Evidence of Suspected Infection (N = 481)

Characteristics	No. (%)
Sex
Male	452 (94.0)
Female	29 (6.0)
Race
White	438 (91.1)
Black	14 (2.9)
Other/unknown	29 (6.0)
Lactic acid
Not tested	431 (89.6)
> 2 mmol/L	21 (4.4)
≤ 2 mmol/L	29 (6.0)
Age, mean (SD), y	67.4 (11.6)
qSOFA score
0	145 (30.1)
1	185 (38.5)
2	122 (25.4)
3	29 (6.0)
SIRS score
0	28 (5.8)
1	81 (16.8)
2	118 (24.5)
3	152 (31.6)
4	102 (21.2)

Abbreviations: qSOFA, quick Sequential Organ Failure Assessment; SIRS, Systemic Inflammatory Response Syndrome.

TABLE 2.

Performance of Criteria for Predicting Sepsis^a

	Sepsis, No. (n = 255)	No Sepsis, No. (n = 226)	Measurement (%)
qSOFA +	114	37	Sensitivity 44.7 Specificity 83.6 PPV 75.5 NPV 57.3
qSOFA −	141	189
SIRS +	204	168	Sensitivity 80.0 Specificity 25.7 PPV 54.8 NPV 53.2
SIRS −	51	58

Abbreviations: NPV, negative predictive value; PPV, positive predictive value; qSOFA quick Sequential Organ Failure Assessment; SIRS, Systemic Inflammatory Response Syndrome.

^aSepsis = SOFA score ≥ 2 with suspected or confirmed infection;

“No Sepsis” = SOFA score < 2 with suspected or confirmed infection.

Scores for qSOFA but not SIRS were significantly associated with septic shock (Fisher exact test: qSOFA, P = .009; SIRS, P = .58) (Table 3). Both assessments were significantly associated with increased risk for 30-day mortality (Fisher exact test: qSOFA, P < .001; SIRS, P = .025). In an additional analysis, scores for SOFA were not significantly associated with septic shock (Fisher exact test, P = .13) but were significantly associated with an increased risk for 30-day mortality (Fisher exact test, P = .016) (Table 4).

TABLE 3.

Association of Assessment Scores With Septic Shock and 30-Day Mortality

	Septic Shock, No. (%)	Pa	30-Day Mortality, No. (%)	Pa
qSOFA + (n = 151)	4 (2.7)	.009	19 (12.6)	< .001
qSOFA − (n = 330)	0 (0)		12 (3.6)
SIRS + (n = 372)	4 (1.1)	.58	29 (7.8)	.025
SIRS − (n = 109)	0 (0)		2 (1.8)

Abbreviations: qSOFA, quick Sequential Organ Failure Assessment; SIRS, Systemic Inflammatory Response Syndrome

^aFisher’s exact test

TABLE 4.

Association of SOFA With Septic Shock and 30-Day Mortality

	Septic Shock, No. (%)	Pa	30-Day Mortality, No. (%)	Pa
SOFA ≥ 2 (Sepsis, n = 255)	4 (1.6)	.13	23 (9)	.016
SOFA < 2 (No sepsis, n = 226)	0 (0)		8 (3.5)

Abbreviation: SOFA, Sequential Organ Failure Assessment.

^aFisher exact test.

DISCUSSION

High sensitivity is critical for a sepsis screening tool. To be clinically useful, it has been suggested that biomarkers predicting poor outcomes for sepsis should have a sensitivity of > 80%.⁴ Although qSOFA demonstrated greater specificity than SIRS in our study (83.6% vs 25.7%), qSOFA showed lower sensitivity (44.7% vs 80.0%), which resulted in a greater potential for false negatives; 55.3% of those with sepsis would go undetected. Therefore, our study does not support qSOFA as a better screening assessment than SIRS for sepsis in the veteran population.

Most studies concur with our findings of low sensitivity and high specificity of qSOFA. In a systematic review and meta-analysis, Serafim and colleagues identified 10 studies published after Sepsis-3 that reported sensitivity or specificity of qSOFA and SIRS for sepsis diagnosis.⁵ Seven of the 10 studies reported sensitivities and favored SIRS in the diagnosis of sepsis (Relative risk: 1.32; 95% CI, 0.40–2.24; P < .0001; I² = 100%). The authors noted that substantial heterogeneity among studies, including differences in study design, sample size, and criteria for determination of infection, was an important limitation. In addition, most studies that contrast qSOFA and SIRS center on prognostic value in predicting mortality rather than as a screening test for a diagnosis of sepsis.

We concluded SIRS was more sensitive and thus superior to qSOFA when used as a screening tool for sepsis but conceded that more prospective and homogenous investigations were necessary. To our knowledge, only 1 published study has deviated from this conclusion and reported comparable sensitivity between SIRS (92%) and qSOFA (90%).⁶ Our study adds to existing literature as it is the first conducted in a veteran population. Additionally, we performed our investigation in a general medicine population with methods similar to existing literature, including the key study validating clinical criteria for sepsis by Seymour and colleagues.³

Limitations

This study is not without limitations, including potential misclassification of cases if essential data points were not available during data collection via health record review or the data points were not representative of a true change from baseline (eg, the Glasgow Coma Scale score for altered mental status in the qSOFA or the SOFA score for organ dysfunction). Generalizability of the results also may be limited due to our retrospective, single-center design and characteristics typical of a veteran population (eg, older, white males). Additionally, many veterans were excluded from the study if they transferred from another facility. These veterans may have been more critically ill than those who presented directly to our facility, which possibly introduced selection bias.

CONCLUSION

Our findings do not support use of the qSOFA as a suitable replacement for SIRS as a sepsis screening tool among patients with suspected infection in the general medicine inpatient setting. The clinical concern with SIRS is that unfavorable specificity leads to unnecessary antibiotic exposure among patients who are falsely positive. While qSOFA has demonstrated higher specificity, its use would cause many sepsis cases to go undetected due to the technique’s low sensitivity. Frequent false negative qSOFA results could thus serve to impede, rather than enhance, early recognition and intervention for sepsis.

The ideal sepsis screening tool is rapid and possesses high sensitivity and specificity to promptly identify and manage sepsis and avert unfavorable outcomes such as septic shock and death. While the SIRS criteria do not satisfy these ideal features, its measurement characteristics are more suitable for the application of sepsis screening than the qSOFA and should thus remain the standard tool in this setting. Future prospectively designed studies with more uniform methodologies are necessary to ascertain the most effective approach to identify sepsis for which novel screening approaches with more clinically suitable measurement properties are greatly needed.