qSOFA score: Predictive validity in Enterobacteriaceae bloodstream infections

Abstract

Purpose:

To determine whether the quick Sequential Organ Failure Assessment (qSOFA) retains predictive validity in patients with Enterobacteriaceae sepsis that all received appropriate initial antimicrobial therapy.

Materials and methods:

Retrospective cohort at Barnes-Jewish Hospital including individuals with Enterobacteriaceae sepsis receiving appropriate initial antimicrobial therapy between 6/2009–12/2013. Outcomes were com-pared according to qSOFA score and sepsis classification.

Results:

We identified 510 patients with Enterobacteriaceae sepsis; 67 (13.1%) died. Mortality was higher in patients with qSOFA scores of 2 or 3 than those with scores of 0 or 1 (13.3% and 42.4% versus 5.1% and 1.8%). In multivariate logistic regression analysis, altered mental status (AMS) alone or qSOFA score ≥ 2 were both predictors of mortality with odds ratios of 8.01 and 5.39, respectively. Regardless of sepsis severity, non-survivors were significantly more likely to have AMS than survivors. Sepsis severity, qSOFA, and AMS had comparable predictive validity for mortality.

Conclusions:

Our results support qSOFA score, AMS, and sepsis severity as acceptable bedside tools for prognostication during initial clinical assessment in patients with sepsis. qSOFA retained its predictive validity in this cohort, suggesting that appropriate initial antimicrobial therapy is not an effect modifier for mortality when using qSOFA for prognostication.

1. Introduction

In the most recent update to the sepsis guidelines, the quick Sequential Organ Failure Assessment (qSOFA) score was identified as a potentially easy to use bedside tool to determine which patients might be at risk for poor outcomes [1]. Since the release of the guidelines, there has been much debate about the merits of the qSOFA score [2–8]. The qSOFA score has been applied to various cohorts, with mixed results regarding its predictive validity [9–17]. One of the crucial factors that was not considered in derivation of the qSOFA score was the receipt of inappropriate initial antimicrobial therapy. Inappropriate initial antimicrobial therapy leads to higher mortality in patients with severe sepsis or septic shock [18–22]. Neglecting to account for inappropriate initial antimicrobial therapy during compilation of the qSOFA score could reduce its predictive validity. It was our goal to assess whether the qSOFA score would be a predictor of increased mortality in a cohort of patients that all received appropriate antimicrobial therapy. By studying only patients receiving appropriate antimicrobial therapy, inappropriate initial antimicrobial therapy will be removed as an effect modifier for qSOFA predictive validity. Our secondary aims were to assess the individual components of qSOFA in predicting mortality and to compare qSOFA to sepsis severity scoring.

2. Materials and methods

2.1. Study location and patient population

This study was conducted at Barnes-Jewish Hospital, a 1250 bed academic medical center located in St. Louis, MO. This was a secondary analysis of a cohort that we previously described [23]. The study period was June 1, 2009 through December 31, 2013, corresponding to the length of time for which an electronic medical record was available that could verify time of antibiotic administration. All consecutive hospitalized patients with sepsis, severe sepsis, or septic shock and a positive blood culture for an organism in the Enterobacteriaceae family during the study period were analyzed for eligibility. This study was approved by the Washington University School of Medicine Human Studies Committee.

2.2. Study design and data collection

The cohort and data collection has been previously described [23]. This was a retrospective cohort study of all patients age ≥ 18 with sepsis, severe sepsis, or septic shock (as defined by systemic inflammatory response syndrome (SIRS) criteria) and a positive blood culture for an organism in the Enterobacteriaceae family. Patients were included only if they had positive blood cultures with a single organism from the Enterobacteriaceae family. ICD-9 codes indicative of acute organ dysfunction or the need for vasopressors were used to classify patients as having severe sepsis or septic shock, respectively. The primary endpoint was all-cause 30-day mortality, calculated from the time that a positive blood culture was drawn. Only the first episode of sepsis, severe sepsis, or septic shock was evaluated. Baseline characteristics, including age, gender, race, place of origin, healthcare exposure, receipt of antibiotics within 30 days of positive culture, presence of immunosuppression, Acute Physiology and Chronic Health Evaluation (APACHE) II [24] scores (calculated based on clinical data present during the 24 h after positive blood cultures were drawn), Charlson Comorbidity Index, and medical comorbidities were obtained.

For qSOFA analysis, lowest systolic blood pressure and highest respiratory rate in the 24 h before or after the time at which a positive blood culture was drawn were collected. All charts were reviewed for documented episodes of altered mental status in the 24 h before or after positive blood culture.

2.3. Definitions

In order to meet criteria for altered mental status (AMS), a medical provider had to document a firsthand account of AMS in their note. Altered mental status as reported by witnesses or family members was not considered true AMS unless also documented by the medical provider. Patients with altered sensorium at baseline were considered to have AMS due to the inability to document change from baseline in this retrospective cohort.

All patients received appropriate initial antibiotic therapy, defined as antibiotics that had in vitro activity against the cultured organism (and were not single-agent aminoglycosides), that was administered within 12 h of when a positive blood culture was drawn and continued for at least 24 h. For extended-spectrum β-lactamase producing organisms, initial use of a carbapenem was required to be classified as appropriate treatment. Antimicrobial susceptibilities were determined using disc diffusion methodology. Appropriate antibiotics administered ≤12 h before positive blood cultures were drawn were considered to have a time of administration of 0 min.

Only the first episode of bacteremia during a hospitalization was considered. Patients who had an episode of bacteremia during their hospitalization prior to Enterobacteriaceae bacteremia were excluded (only two cases, one with Staphylococcus epidermidis, one with Enterococcus). The following organisms were considered contaminants if not recultured within 72 h: coagulase-negative Staphylococci, Corynebacterium, Propionibacterium acnes, or Viridans group Streptococcus. Patients were excluded if they were under 18 years of age or if they had a blood culture positive for more than one organism. All patients who did not receive antibiotics within 12 h of when positive blood cultures were drawn were excluded. Discharge on hospice was considered a mortality equivalent. All patients discharged on hospice were considered to expire at the time of hospital discharge.

2.4. Statistical analysis

Univariate analysis was performed by chi-square or Fisher’s exact test where appropriate for categorical values. Student’s t-test, Mann-Whitney U test, one-way ANOVA, or Kruskal-Wallis test was used where appropriate for continuous variables. Continuous variables were reported as means with standard deviations or median with interquartile range for non-normally distributed variables. Categorical data were expressed as frequencies. A P value of <0.05 was considered significant. Using our previous multivariate logistic regression model [23], we replaced sepsis severity with qSOFA score and each component of qSOFA in separate models to determine odds ratios (OR) for death. Area under the receiver operating characteristic (AUROC) curves for mortality were generated for each component of the qSOFA score, qSOFA scores ≥ 2 versus <2, and for sepsis severity scoring. AUROC curves were also generated for multivariate models that included qSOFA, AMS alone, or sepsis severity scores. Goodness-of-fit was assessed via the Hosmer-Lemeshow test. All tests were two-tailed. All analysis was done using SPSS v24 (IBM, Armonk, NY).

3. Results

Five-hundred ten patients with sepsis, severe sepsis, or septic shock (by SIRS criteria) due to Enterobacteriaceae met the inclusion criteria. Baseline characteristics of the patients stratified by qSOFA score are listed in Table 1. As qSOFA score increased from 0 to 3, patients were significantly more likely to require mechanical ventilation and have pneumonia as their source of infection, have CHF or COPD, have higher APACHE-II scores and lactate levels, were more likely to have severe sepsis or septic shock (by SIRS criteria), and less likely to have leukemia or sepsis (by SIRS criteria). Patients with a qSOFA score ≥ 2 had longer lengths of stay (LOS) and intensive care unit (ICU) LOS, and thirty day all-cause mortality than patients with a score < 2 (Table 1). Two hundred fifty-five (50%) patients had lactate results available. Mortality was significantly higher (Chi-square test P < 0.001) in the group of patients with lactate results available than those without (18.8% vs. 7.5%). APACHE II scores were significantly higher (Mann-Whitney test P b 0.001) in patients with lactate results available than those without (median 14.0 versus 12.0).

Table 1.

Characteristics by qSOFA score.

Characteristics	qSOFA = 0 (n = 79)	qSOFA = 1 (n = 166)	qSOFA = 2 (n = 180)	qSOFA = 3 (n = 85)	P value
Age, yrs	60.2 ± 16.3	58.7 ± 15.7	59.7 ± 15.5	63.1 ± 15.2	0.207
Male, % (#)	53.2 (42)	50 (83)	51.1 (92)	58.8 (50)	0.589
African-American, % (n)	30.4 (24)	28.9 (48)	30.6 (55)	38.8 (33)	0.432
Mechanical ventilation, % (n)	2.5 (2)	8.4 (14)	24.4 (44)	49.4 (42)	<0.001
Bone marrow transplant, % (n)	5.1 (4)	4.8 (8)	3.3 (6)	5.9 (5)	0.791
Solid organ transplant, % (n)	5.1 (4)	6.0 (10)	2.2 (4)	3.5 (3)	0.331
CHF, % (n)	8.9 (7)	12.0 (20)	16.1 (29)	23.5 (20)	0.036
COPD, % (n)	6.3 (5)	14.5 (24)	16.1 (29)	22.4 (19)	0.038
Diabetes mellitus, type 2, % (n)	31.6 (25)	27.7 (46)	30.6 (55)	25.9 (22)	0.798
CKD, % (n)	15.2 (12)	10.8 (18)	12.8 (23)	18.8 (16)	0.343
RRT, % (n)	0	3.0 (5)	2.8 (5)	7.1 (6)	0.074
Solid organ malignancy, % (n)	26.6 (21)	27.7 (46)	27.8 (50)	31.8 (27)	0.88
Leukemia, % (n)	27.8 (22)	23.5 (39)	13.9 (25)	10.6 (9)	0.004
Lymphoma, % (n)	3.8 (3)	6.6 (11)	6.7 (12)	4.7 (4)	0.753
Cirrhosis, % (n)	3.8 (3)	2.4 (4)	6.1 (11)	12.9 (11)	0.007
Antibiotics within 30 days, % (n)	41.8 (33)	39.2 (65)	37.2 (67)	35.3 (30)	0.833
Healthcare exposure, % (n)	63.3 (50)	71.1 (118)	67.2 (121)	76.5 (65)	0.260
MDR, % (n)	25.3 (20)	17.5 (29)	16.1 (29)	24.7 (21)	0.176
Time to appropriate antibiotics (hours)	2.4 [1.1–4.4]	2.7 [1.0–5.0]	2.3 [1.1–5.2]	2.6 [1.1–5.2]	0.872
Immunosuppressed, % (n)	46.8 (37)	41.0 (68)	30.6 (55)	31.8 (27)	0.094
Charlson Comorbidity Score	1.7 ± 1.3	1.5 ± 1.2	1.6 ± 1.3	1.9 ± 1.2	0.138
APACHE II score	10.0 ± 3.6	11.4 ± 4.1	14.7 ± 5.0	17.8 ± 5.9	<0.001
Patient origin, % (n)					<0.001
Nursing home, SNF, or LTACH	3.8 (3)	6.6 (11)	8.9 (16)	16.5 (14)
Community	59.5 (47)	62.7 (104)	51.1 (92)	41.2 (35)
OSH	7.6 (6)	7.2 (12)	13.9 (25)	10.6 (9)
In hospital	29.1 (23)	23.5 (39)	26.1 (47)	31.8 (27)
Infection source, % (n)
Central venous catheter	8.9 (7)	12.0 (20)	7.8 (14)	7.1 (6)	0.471
Genitourinary	39.2 (31)	37.3 (62)	48.3 (87)	41.2 (35)	0.195
Pulmonary	5.1 (4)	1.8 (3)	7.2 (13)	10.6 (9)	0.025
Gastrointestinal	12.7 (10)	18.1 (30)	16.1 (29)	16.5 (14)	0.763
Unknown	27.8 (22)	28.9 (48)	19.4 (35)	22.4 (19)	0.176
Othera	6.3 (5)	1.8 (3)	1.2 (2)	2.4 (2)	0.077
Sepsis	54.4 (43)	49.4 (82)	22.8 (41)	7.1 (6)	<0.001 for all categorizations by SIRS
Severe sepsis	45.6 (36)	43.4 (72)	35.0 (63)	23.5 (20)
Septic shock	0	7.2 (12)	42.2 (76)	69.4 (59)
SBP ≤100 mm Hg	0	60.8 (101)	86.7 (156)	100 (85)
RR ≥32 breaths per minute	0	33.7 (56)	90.6 (163)	100 (85)
AMS	0	5.4 (9)	22.8 (41)	100 (85)
Elevated lactate (>2.1)	23.8 (5/21 available)	43.4 (23/53 available)	56.4 (62/110 available)	73.2 (52/71 available)	<0.001
Lowest SBP	115 ± 10.8	99 ± 18.1	85.4 ± 17.6	74.6 ± 15.2	<0.001
Highest RR	19.4 ± 1.0	21.8 ± 5.4	29 ± 7.1	31.8 ± 8.8	<0.001
Highest lactate	1.2 [0.9–2.1]	1.7 [1.2–3.1]	2.5 [1.5–4.2]	3.2 [2.1–6.7]	<0.001
LOS	5.7 [3.8–19.8]	5.9 [4.4–17]	9.0 [5.3–20.5]	11.2 [5.9–24.5]	<0.001
ICU LOS	0	0 [0–1.6]	2.1 [0–6.3]	4.6 [2.1–11]	<0.001
Thirty-day mortality	5.1 (4)	1.8 (3)	13.3 (24)	42.4 (36)	<0.001

Age, Charlson – one way ANOVA. APACHE II, respirations, BP – violated Levene’s test of significance, so analyzed by Kruskal-Wallis. Time to appropriate antibiotics, lactate – Kruskal-Wallis.

^aTypes of infections in the other category include central nervous system, skin and soft tissue, vascular graft, muscle, joint, osteomyelitis, and gynecologic.

From our previous multivariate logistic regression analysis (MVLRA) [23], sepsis severity, African-American race, cirrhosis, solid organ malignancy, transfer from an OSH, and APACHE-II score were risk factors for mortality [23]. Replacing sepsis severity with a qSOFA score ≥ 2 in the MVLRA resulted in an OR of 5.39 for death. Each one point increase of the qSOFA score had an OR for death of 2.86. AMS alone had OR for death of 8.01. Additionally, across all categorizations of sepsis severity (SIRS criteria), non-survivors were significantly more likely to have AMS than survivors (Table 2). When replacing sepsis severity with AMS alone in MVLRA, the ORs for the other variables in the model were still statistically significant (Table 3).

Table 2.

Proportions of patients with altered mental status (AMS) by sepsis severity (as defined by the systemic inflammatory response syndrome criteria) according to survival status.

Sepsis classification	Survivors	Non-survivors	P value
Sepsis patients with AMS	7.8 (13)	66.7 (4)	<0.001
Severe sepsis patients with AMS	22.1 (38)	57.9 (11)	<0.001
Septic shock patients with AMS	35.2 (37)	76.2 (32)	<0.001
Any sepsis severity with AMS	19.9 (88)	70.1 (47)	<0.001

Table 3.

Odds ratios for death from the multivariate logistic regression model.

Factor	Odds ratio (95% confidence intervals)
AMS	8.01(4.13–15.55)
African American race	2.38 (1.23–4.62)
APACHE II (1-point increments)	1.14 (1.08–1.20)
Solid organ cancer	3.62 (1.88–7.00)
Cirrhosis	6.37 (2.31–17.59)
Patient origin from OSH	4.31 (1.79–10.39)

Hosmer-Lemeshow test = 0.696. When AMS was replaced by a qSOFA score ≥ 2 in the multivariate logistic regression analysis, the odds ratio for death was 5.39 (Hosmer-Lemeshow = 0.419). In our previous study, the odds ratio for each increase in class of sepsis severity was 2.07. Each one point increase in qSOFA score had an odds ratio for death of 2.86.

The AUROC curve for the various logistic regression analyses were 0.885, 95% CI (0.843–0.927) for the model incorporating AMS alone, 0.839, 95% CI (0.791–0.887) for the sepsis severity model, and 0.852, 95% CI (0.810–0.894) for the qSOFA model. There were no significant differences in the AUROC between these three multivariate models, but all had significantly higher AUROCs than AMS, qSOFA, or sepsis severity alone (p < 0.05). The AUROC for AMS alone was 0.751 and 0.716 for qSOFA ≥ 2, which was not statistically significant (Fig. 1). Sepsis severity alone as a predictor of mortality had an AUROC of 0.731, which was not statistically different from qSOFA or AMS. Non-survivors were significantly more likely to have AMS than survivors as assessed by Kaplan-Meier analysis in Fig. 2.

Fig. 1.

Receiver operating characteristic curves to predict mortality for altered mental status (AMS) alone and qSOFA score ≥ 2 (as univariate predictors). Area under the receiver operating characteristic (AUROC) for AMS: 0.751, 95% CI [0.684,0.819]. AUROC for qSOFA score ≥ 2: 0.716, 95% CI [0.660,0.773].

Fig. 2.

Kaplan-Meier curves for mortality according to the presence or absence of altered mental status.

4. Discussion

We found that qSOFA score ≥ 2 predicted mortality among patients with Enterobacteriaceae bloodstream infections receiving appropriate initial antimicrobial therapy. Additionally, AMS alone had OR for death of 8.01, comparable to the qSOFA score overall. The AUROC of 0.716 for the qSOFA score was similar to previously published results [1,9, 10,17]. In the context of the evaluation of the qSOFA score, comparable OR for mortality using AMS alone has not been previously reported. Septic encephalopathy, on the other hand, is a well-known cause of increased morbidity and mortality in sepsis patients, though morbidity and mortality attributable to septic encephalopathy in patients receiving appropriate initial antibiotics has not been studied [25–27]. Unsurprisingly, as qSOFA score increased, requirements for mechanical ventilation, number of comorbidities, APACHE-II scores, LOS, ICU LOS, lactate level, and thirty day mortality significantly increased.

We demonstrate that used in isolation, qSOFA, AMS, and sepsis severity have similar ability to discriminate between septic patients that will die and those that do not, as evidenced by similar AUROC curves. In addition, we show that adding these scoring systems to multivariable models improves the accuracy of mortality prediction, but that the AUROC for the individual multivariable models is similar regardless of the scoring system used. Based on this, we can conclude that these scoring systems are valuable as bedside tools during initial clinical assessments with the recognition that including other factors such as comorbidities, place of origin, and laboratory parameters (via APACHE-II score) provide more complete information regarding clinical outcome. As there were no differences in predictive validity between the scoring systems, it is reasonable for clinicians to select the scoring system of their choice without compromising prognostic accuracy.

In the sepsis-3 guidelines, the authors suggest that lactate could be used to risk stratify patients with a qSOFA score of one [1], an idea that was subsequently tested in a new cohort and found to be valid [15]. However, in our cohort of patients receiving appropriate therapy, lactate levels in patients whose qSOFA score would otherwise be one did not result in increased predictive validity for mortality. This could be explained by the fact that all patients received appropriate therapy within 12 h and did not progress to severe tissue level hypoxia that would result in the hyperlactatemia that was seen in other studies. As patients that had lactate results available had higher mortality and APACHE II scores, another possibility is that if the entire cohort had lactate checked, the predictive validity of qSOFA with lactate included might increase. Such a possibility needs to be explored in a cohort in which all sepsis patients, regardless of severity of illness, had lactate results available.

Our study is limited in several ways. The retrospective nature of the study makes it difficult to elucidate possible confounders that could have biased the outcome measures. This was a single-center study and results may not be generalizable to other centers. Altered mental status may have been documented in only the most severe cases, which would artificially inflate its association with mortality. We did not study out-comes in patients with Gram-positive infections or non-Enterobacteriaceae Gram-negative infections. It is possible that different components of the qSOFA score would have different predictive validity in different types of infections and this is an area ripe for future studies.

In conclusion, qSOFA score is an important predictor of mortality, even in patients that all receive appropriate initial antimicrobial therapy. AMS alone was strongly associated with mortality. Lactate levels for patients with a qSOFA score of one did not help predict mortality. Multivariate models for mortality prediction performed similarly, regardless of whether sepsis severity, qSOFA, or AMS alone were incorporated, suggesting that clinicians can use the bedside severity of illness scoring system of their choice without compromising prognostic accuracy. Our results will assist in prognostication for patients with Enterobacteriaceae bloodstream infections. Future studies can assess similar outcomes in patients with Gram-positive or Gram-negative non-Enterobacteraiaceae sepsis, and validate AMS as an important outcome predictor among patients receiving appropriate initial antimicrobial therapy.