Abstract
Background
We aimed to evaluate the accuracy of quick Sequential (sepsis-related) Organ Failure Assessment (qSOFA) for the diagnosis of sepsis-3, and to analyze the prognosis of infected patients in wards over-diagnosed with qSOFA but missed by sepsis-3, and those missed by qSOFA but in accordance with sepsis-3 criteria. We also intended to validate the performance of qSOFA as one predictor of outcome in patients with suspicion of infection.
Methods
We reviewed the medical records of 1,716 adult patients with infection who were hospitalized from July 1st, 2012 to June 30th, 2014 in the Yuetan subdistrict of Beijing, China. Based on the sepsis-3 criteria and qSOFA score proposed by the Third International Consensus Definitions for Sepsis and Septic Shock, these patients were categorized into four groups: qSOFA(−)sepsis(−), qSOFA(+)sepsis(−), qSOFA(−)sepsis(+), and qSOFA(+)sepsis(+). Multivariate logistic regression analysis was used to determine the independent risk factors for in-hospital mortality. The area under the receiver operating characteristic curves (AUROCs) of the qSOFA(+) group were compared with the sepsis(+) group for in-hospital mortality, ICU admission, and invasive ventilation.
Results
Among the 1,716 patients with infection, there were 935 patients (54.5%) with sepsis, and 640 patients (37.3%) with qSOFA ≥2. There were 610 patients in the qSOFA(−)sepsis(−) group, 171 in the qSOFA(+)sepsis(−) group, 466 in the qSOFA(−)sepsis(+) group, and 469 in the qSOFA(+)sepsis(+) group. In the logistic regression analysis, increasing age, bedridden status, and malignancy were all independent risk factors of hospital mortality. Sepsis and qSOFA ≥2 were also independent risk factors of hospital mortality, with an adjusted OR of 3.85 (95% CI: 2.70–5.50) and 13.92 (95% CI: 9.87–16.93) respectively. qSOFA had a sensitivity of 50.2% and a specificity of 78.1% for sepsis-3. The false-positive [qSOFA(+)sepsis(−)] group had 38 patients (22.2%) die during hospitalization, and an adjusted OR of 9.20 (95% CI: 4.86–17.38). In addition, the false-negative [qSOFA(−)sepsis(+)] group had a hospital mortality rate of 7.3% (34/466) and an adjusted OR of 2.59 (95% CI: 1.39–4.83). In comparison, patients meeting neither qSOFA nor sepsis criteria had the lowest hospital mortality [2.6% (16/610)], whereas patients with both qSOFA ≥2 and sepsis had the highest hospital mortality [56.5% (265/469)], with an adjusted OR of 42.02 (95% CI: 24.31–72.64). The discrimination of in-hospital mortality using qSOFA (AUROC, 0.846; 95% CI, 0.824–0.868) was greater compared with sepsis-3 criteria (AUROC, 0.834; 95% CI, 0.805–0.863; P<0.001).
Conclusions
In our analysis, the sensitivity(Se) of qSOFA for the diagnosis of sepsis was lower, and qSOFA score ≥2 might identify a group of patients at a higher risk of mortality, regardless of being septic or not.
Keywords: Sequential (sepsis-related) Organ Failure Assessment (SOFA), quick Sequential Organ Failure Assessment (qSOFA) score, sepsis-3, mortality, the area under the receiver operating characteristic curves (AUROCs)
Introduction
In 1991, sepsis was defined as systemic inflammatory response syndrome (SIRS) induced by infection (1). After more than two decades of widespread use in clinical practice and research, it is now well understood that both pro- and anti-inflammatory responses are involved in the pathogenesis of sepsis (2). Moreover, SIRS criteria are too sensitive and insufficiently specific to identify some severely infected patients (3,4). In 2001, the definitions of sepsis and septic shock were revised (5), incorporating the concept and diagnostic criteria of organ damage. However, owing to its complexity, the 2001 definition of sepsis has not been widely applied in clinical practice. In 2016, the Third International Consensus Definitions for Sepsis and Septic Shock Task Force (Sepsis-3) redefined sepsis as a “life-threatening organ dysfunction caused by a dysregulated host response to infection” (6). Organ dysfunction was defined as an acute increase in total Sequential (sepsis-related) Organ Failure Assessment (SOFA) score ≥2 points consequent to the infection. Because the components of SOFA were too complex and required multiple laboratory tests that might be impractical in daily clinical practice, the Task Force proposed a quick SOFA (qSOFA) score to facilitate easier identification of patients who were potentially at risk of dying from sepsis (6). The qSOFA score consists of only three criteria: Glasgow Coma Scale (GCS) <15, systolic blood pressure ≤ 100 mmHg, and respiratory rate ≥22/min. A qSOFA score of 2 or more points indicates organ dysfunction with predictive validity similar to that of the full SOFA score outside the intensive care unit (ICU) (7).
Since the definition of the new criteria was published, there have been controversies over the predictive values of qSOFA criteria in the diagnosis of sepsis-3 (8-13). Both retrospective studies and meta-analyses have demonstrated low sensitivity and high specificity of qSOFA score in the diagnosis of sepsis-3 (13-17). However, data concerning the prognosis of infected patients misdiagnosed by qSOFA score are still lacking (18). Based on a secondary analysis of a database of all hospitalized patients living in a subdistrict of Beijing, we performed the current study to investigate the predictive value of qSOFA score for sepsis-3, the clinical outcome of septic patients who are missed by the qSOFA score (false-negatives), and the clinical outcome of nonseptic patients who are misdiagnosed as sepsis-3 by the qSOFA score (false-positives).
Methods
The methodology of the current study has been previously reported in detail (19). In brief, this was a retrospective cohort study of all adult (≥18 years) residents of the Yuetan subdistrict of Beijing, China, who were hospitalized from July 1st, 2012, to June 30th, 2014. These patients were identified with the use of the hospital discharge database of the Beijing Public Health Information System. All available medical records of enrolled patients were manually reviewed independently by any two of three investigators, all who had more than 5 years of working experience in the ICU. Any disagreement was resolved by discussion among the three investigators, and then among the steering committee (XM, YA, and BD) if consensus could not be reached.
Retrieved data included demographic data, admission category (medical, elective surgery or emergency surgery), comorbidities (20), and in-hospital mortality. Derived from the above data, the severity of underlying illness was assessed by McCabe and Jackson classification (21), while chronic organ dysfunction or immunosuppression was defined based on the criteria from the Acute Physiology and Chronic Health Evaluation (APACHE) II score (22). In addition, body mass index (BMI) was calculated based on the height and weight on hospital admission.
For patients with infection, we collected data about the source of infection, relevant microbiological information, ICU admission, ICU length of stay, and all data necessary for the calculation of SOFA and qSOFA scores (6,23). For the purpose of this study, infection was diagnosed based on clinical manifestations, laboratory tests, and radiographic findings, including microbiologically documented (with definite positive results of microbial culture of body fluids or blood) and clinically documented (with no definite positive culture results but with imaging or pathological evidence of clinical infection) infections (24). Regardless of admission or discharge diagnoses, we identified cases with infection based on manual review of clinical manifestations, in addition to laboratory, imaging, and microbiologic parameters.
For patients with infection, we calculated maximum SOFA and qSOFA scores based on retrieved clinical data until 72 hours after hospital admission (for those who were admitted due to infection) or onset of infection (for those who developed infection during hospitalization). Sepsis was diagnosed as an acute change in total SOFA score ≥2 points consequent to the infection, according to the Third International Consensus Definitions for Sepsis and Septic Shock [6]. In addition, a patient meeting at least 2 of the 3 criteria of qSOFA score was deemed as qSOFA(+) [6].
Univariate and multivariate logistic regression analyses were used to identify independent risk factors associated with in-hospital mortality, such as age and chronic comorbidities. All potential risk factors were added into the model using stepwise conditional backward entry, if P<0.1 in univariate analysis. Age was categorized in three categories (18–64, 65–84, and ≥85 years) (25,26), because the assumption of linearity would be violated if age was included in the model as a continuous variable. The prognostic value of qSOFA and sepsis was also assessed by the area under the receiver operating characteristic curve (AUROC).
In order to delineate the performance of qSOFA for the diagnosis of sepsis, we calculated sensitivity and specificity. Moreover, clinical outcomes of those patients misdiagnosed by qSOFA score, including overdiagnosis (false-positives) and underdiagnosis (false-negatives), were also compared with adjusted odds ratio (OR).
Continuous variables were presented as median and interquartile range (IQR). Categorical variables were presented as a percentage of the group from which they were derived, and compared by the use of chi-square test or Fisher’s exact test. All comparisons were unpaired and all tests of significance were two-tailed. A P value <0.05 was considered as statistically significant.
Ethical approval
This study was approved by the ethics committee of Peking Union Medical College Hospital and informed consent was waived. This study was registered at ClinicalTrials.gov, with registration number NCT02285257.
Results
Patient enrollment
During the 2-year study period, a total of 22,552 adult residents in the Yuetan subdistrict were hospitalized, among whom 21,191 had their medical records manually reviewed. We were not able to review the medical records of the other 1,361 patients due to missing records (n=277), and refusal by the hospitals (n=1,084).
Out of the 21,191 adult patients, 1,716 infected patients with complete results of physical examination and laboratory examination required for the diagnosis of sepsis and calculation of qSOFA score were enrolled in this study.
Patient characteristics
Among 1,716 patients with infection, there were 935 patients (54.5%) with sepsis, 640 patients (37.3%) with qSOFA ≥2, and 610 patients (35.5%) meeting neither of the above criteria. Compared with patients without sepsis, patients with sepsis were older, more likely to be male, and had more comorbidities and chronic organ dysfunction. In addition, patients with sepsis were more likely to have lower respiratory tract infections, with more severe acute illness (as shown by more ICU admissions, more acute organ failures including septic shock and respiratory failure, higher mortality, and longer length of stay). Similar findings were also reported when patients with qSOFA ≥2 were compared with those with qSOFA <2 (Table 1).
Table 1. Characteristics of patients with infection, categorized by the presence or absence of qSOFA and sepsis criteria.
| Characteristics | qSOFA(−) (n=1,076) | qSOFA(+) (n=640) | Sepsis(−) (n=781) | Sepsis(+) (n=935) |
|---|---|---|---|---|
| Age, year, median [IQR] | 79 [61–85] | 82 [74–86]** | 78 [57–84] | 81 [74–86]‡ |
| Male sex, n (%) | 613 (57.0) | 375 (58.6) | 414 (53.0) | 574 (61.4)‡ |
| Body mass index (BMI), kg/m2, median [IQR] | 24 [21–26] | 22 [19–25]** | 24 [21–26] | 23 [20–26]† |
| Type of hospital admission, n (%) | ||||
| Medical | 970 (90.1) | 583 (91.1) | 691 (88.5) | 862 (92.2)‡ |
| Elective surgery | 93 (8.6) | 35 (5.5)* | 75 (9.6) | 53 (5.7)‡ |
| Emergency surgery | 13 (1.2) | 22 (3.4)** | 15 (1.9) | 20 (2.1)† |
| McCabe and Jackson classification, n (%) | ||||
| Nonfatal | 762 (70.8) | 448 (70.0) | 524 (67.1) | 686 (73.4)‡ |
| Ultimately fatal | 164 (15.2) | 125 (19.5)* | 112 (14.3) | 177 (18.9)† |
| Rapidly fatal | 17 (1.6) | 28 (4.4)** | 22 (2.8) | 23 (2.5) |
| Charlson comorbidity index, median [IQR] | 1 [1–3] | 2 [1–3]** | 1 [0–3] | 2 [1–3]‡ |
| Comorbidities, n (%) | ||||
| None | 176 (16.4) | 68 (10.6)* | 162 (20.7) | 82 (8.8)‡ |
| Hypertension | 579 (53.8) | 371 (58.0) | 395 (50.6) | 555 (59.4)‡ |
| Diabetes | 256 (23.8) | 198 (30.9)** | 206 (26.4) | 248 (26.5) |
| Malignancy | 188 (17.5) | 86 (13.4)* | 124 (15.9) | 150 (16.0) |
| Cerebrovascular disease | 394 (36.6) | 237 (37.0) | 273 (35.0) | 358 (38.3)‡ |
| Coronary heart disease | 315 (29.3) | 235 (36.7)** | 206 (26.4) | 344 (36.8)‡ |
| Chronic lung disease | 235 (21.8) | 150 (23.4) | 150 (19.2) | 235 (25.1)‡ |
| Peptic ulcer | 112 (10.4) | 36 (5.6)** | 87 (11.1) | 61 (6.5)‡ |
| Rheumatic disease | 43 (4.0) | 23 (3.6) | 19 (2.4) | 47 (5.0)‡ |
| Hematologic malignancy | 18 (1.7) | 14 (2.2) | 7 (0.9) | 25 (2.7)‡ |
| Dementia | 81 (7.5) | 42 (6.6) | 33 (4.2) | 90 (9.6)‡ |
| Chronic organ dysfunction, n (%) | ||||
| None | 889 (82.6) | 477 (74.5)** | 659 (84.4) | 707 (75.6)‡ |
| Cardiovascular | 13 (1.2) | 24 (3.8)** | 8 (1.0) | 29 (3.1)‡ |
| Liver | 23 (2.1) | 9 (1.4) | 13 (1.7) | 19 (2.0) |
| Respiratory | 42 (3.9) | 65 (10.2)** | 42 (5.4) | 65 (7.0) |
| Renal | 36 (3.3) | 19 (3.0) | 10 (1.3) | 45 (4.8)‡ |
| Immunosuppression | 86 (8.0) | 65 (10.2) | 62 (7.9) | 89 (9.5) |
| Site of infection, n (%) | ||||
| Lower respiratory tract infection | 724 (67.3) | 524 (81.9)** | 518 (66.3) | 730 (78.1)‡ |
| Urogenital tract infection | 104 (9.7) | 49 (7.7) | 102 (13.1) | 51 (5.5)‡ |
| Intra-abdominal infection | 83 (7.7) | 52 (8.1) | 55 (7.0) | 80 (8.6) |
| Upper respiratory infection | 51 (4.7) | 5 (0.8)** | 32 (4.1) | 24 (2.6) |
| Gastroenteritis | 50 (4.6) | 18 (2.8) | 38 (4.9) | 30 (3.2) |
| Skin and soft tissue infection | 29 (2.7) | 3 (0.5)** | 26 (3.3) | 6 (0.6)‡ |
| Bacteremia | 42 (3.9) | 42 (6.6)* | 36 (4.6) | 48 (5.1) |
| Other | 44 (4.1) | 20 (3.1) | 31 (4.0) | 33 (3.5) |
| Positive cultures, n (%) | 286 (26.6) | 343 (53.6)** | 250 (32.0) | 379 (40.5)‡ |
| Invasive ventilation, n (%) | 26 (2.4) | 203 (31.7)** | 17 (2.2) | 212 (22.7)‡ |
| ICU admission, n (%) | 44 (4.1) | 184 (28.8)** | 5 (0.6) | 223 (23.9)‡ |
| Septic shock, n (%) | 56 (5.2) | 261(40.8)** | 0 (0.0) | 317 (33.9)‡ |
| Hospital length of stay, median [IQR] | 16 [9–28] | 25 [12–43]** | 17 [9–29] | 20 [11–38] |
| In-hospital mortality, n (%) | 50 (4.6) | 303 (47.3)** | 54 (6.0) | 299 (32.0)‡ |
*P<0.05, **P<0.01, compared with qSOFA(−); †P<0.05, ‡P<0.01, compared with sepsis(−).
Performance of qSOFA for the diagnosis of sepsis
Among the 935 patients with sepsis, 469 (50.2%) met qSOFA criteria (qSOFA ≥2), whereas 171 (21.9%) out of 781 patients without sepsis also had qSOFA ≥2 (Table 2). As a result, for the diagnosis of sepsis, qSOFA criteria had a sensitivity of 50.2%, specificity of 78.1%, positive predictive value of 73.3%, and a negative predictive value of 56.7%.
Table 2. Characteristics of patients with infection categorized by qSOFA and sepsis.
| Characteristics | qSOFA(−)Sepsis(−) (n=610) | qSOFA(+)Sepsis(−) (n=171) | qSOFA(−)Sepsis(+) (n=466) | qSOFA(+)Sepsis(+) (n=469) |
|---|---|---|---|---|
| Age, year, median [IQR] | 76.5 [56–84] | 81 [68.5–85]**,‡ | 81 [73–85]**,† | 82 [75–87]** |
| Male sex, n (%) | 326 (53.4) | 88 (51.5)† | 287 (61.6)** | 287 (61.2)* |
| Body mass index (BMI), kg/m2, median [IQR] | 24 [21–27] | 21 [19–24]** | 23 [21–26] | 23 [19–25]** |
| Type of hospital admission, n (%) | ||||
| Medical | 533 (87.4) | 158 (92.4) | 437 (93.8)** | 425 (90.6) |
| Elective surgery | 68 (11.1) | 7 (4.1)** | 25 (5.4)** | 28 (6.0)** |
| Emergency surgery | 9 (1.5) | 6 (3.5) | 4 (0.9)‡ | 16 (3.4)* |
| McCabe and Jackson classification, n (%) | ||||
| Nonfatal | 405 (66.4) | 119 (69.6) | 357 (76.6)**,† | 329 (70.1) |
| Ultimately fatal | 91 (14.9) | 21 (12.3)‡ | 73 (15.7)† | 104 (22.2)** |
| Rapidly fatal | 10 (1.6) | 12 (7.0)**,† | 7 (1.5) | 16 (3.4) |
| Charlson comorbidity index, median [IQR] | 1 [0–3] | 2 [1–3]* | 2 [1–3]*,† | 2 [1–3]** |
| Comorbidities, n (%) | ||||
| None | 139 (22.8) | 23 (13.5)** | 37 (7.9)** | 45 (9.7)** |
| Hypertension | 305 (50.0) | 90 (52.6) | 274 (58.4)** | 281 (60.3)** |
| Diabetes | 159 (26.1) | 47 (27.5) | 97 (20.7)*,‡ | 151 (32.4)* |
| Malignancy | 93 (15.2) | 31 (18.1)† | 95 (20.3)* ,‡ | 55 (11.8) |
| Cerebrovascular disease | 196 (32.1) | 77 (45.0)**,† | 198 (42.2)**,‡ | 160 (34.3) |
| Coronary heart disease | 144 (23.6) | 62 (36.3)** | 171 (36.5)** | 173 (37.1)** |
| Chronic lung disease | 117 (19.2) | 33 (19.3) | 118 (25.2)* | 117 (25.1)* |
| Peptic ulcer | 71 (11.6) | 16 (9.4)† | 41 (8.7)‡ | 20 (4.3)** |
| Rheumatic disease | 17 (2.8) | 2 (1.2)† | 26 (5.5)* | 21 (4.5) |
| Hematologic malignancy | 7 (1.1) | 0 (0)† | 11 (2.3) | 14 (3.0)* |
| Dementia | 32 (5.2) | 1 (0.6)**,‡ | 49 (10.4)** | 41 (8.8)* |
| Chronic organ dysfunction, n (%) | ||||
| None | 525 (86.1) | 134 (78.4) | 364 (78.1) | 343 (73.1)** |
| Cardiovascular | 4 (0.7) | 4 (2.3) | 9 (1.9)† | 20 (4.3)** |
| Liver | 11 (1.8) | 2 (1.2) | 12 (2.6) | 7 (1.5) |
| Respiratory | 22 (3.6) | 20 (11.7)**,‡ | 20 (4.3)‡ | 45 (9.6)** |
| Renal | 10 (1.6) | 0 (0) | 26 (5.6)** | 19 (4.1)* |
| Immunosuppression | 45 (7.4) | 17 (10.0) | 41 (8.8) | 48 (10.2) |
| Site of infection, n (%) | ||||
| Lower respiratory tract infection | 382 (62.6) | 136 (79.5)** | 342 (73.4)**,‡ | 388 (82.7)** |
| Urogenital tract infection | 72 (11.8) | 30 (17.5)*,‡ | 32 (6.9)** | 19 (4.1)** |
| Intra-abdominal infection | 44 (7.2) | 11 (6.4) | 39 (8.4) | 41 (8.7) |
| Upper respiratory infection | 29 (4.8) | 3 (1.8) | 22 (4.7)‡ | 2 (0.4)** |
| Gastroenteritis | 36 (5.9) | 2 (1.2)* | 14 (3.0) | 16 (3.4) |
| Skin and soft tissue infection | 24 (3.9) | 2 (1.2) | 5 (1.1)** | 1 (0.2)** |
| Bacteremia | 30 (4.9) | 6 (3.5) | 12 (2.6)*,‡ | 36 (7.7) |
| Other | 25 (4.6) | 6 (3.5) | 19 (4.1) | 14 (3.0) |
| Positive cultures, n (%) | 168 (27.5) | 82 (48.0)** | 118 (25.3)‡ | 261 (55.7)** |
| Invasive ventilation, n (%) | 8 (1.3) | 9 (5.3)**,‡ | 18 (3.9)**,‡ | 194 (41.4)** |
| ICU admission, n (%) | 4 (0.7) | 1 (0.6)‡ | 40 (8.6)**,‡ | 183 (39.0)** |
| Septic shock, n (%) | 0 (0) | 0 (0) | 56 (12.0)‡ | 261 (55.7) |
| Hospital length of stay, median [IQR] | 16 [9–27] | 27 [15–40] | 17 [10–29] | 24 [12–45]* |
| In-hospital mortality, n (%) | 16 (2.6) | 38 (22.2)**,‡ | 34 (7.3)**,‡ | 265 (56.5)** |
*P<0.05, **P<0.01, compared with qSOFA(−)sepsis(−); †P<0.05, ‡P<0.01, compared with qSOFA(+)sepsis(+).
Clinical outcome of patients with infection stratified by sepsis and qSOFA criteria
A total of 353 patients (20.6%) died before hospital discharge. In logistic regression analysis, increasing age, bedridden status, and malignancy were all independent risk factors of hospital mortality. Including age as a continuous variable or intertertile range in the logistic regression model did not change the results (Tables S1,S2). Moreover, sepsis and qSOFA ≥2 were also independent risk factors of hospital mortality, with an adjusted OR of 3.85 (95% CI: 2.70–5.50) and 13.92 (95% CI: 9.87–19.63) respectively (Table 3). The qSOFA and sepsis criteria had similar prognostic value, with an AUROC of 0.846 (95% CI, 0.824–0.868) and 0.834 (95% CI, 0.805–0.863) respectively (Table 4).
Table S1. Risk factors of in-hospital mortality in patients with infection, by univariate and multivariate regression analysis.
| Variables | Control | Case | Model 1 | Model 2 | |||
|---|---|---|---|---|---|---|---|
| Crude OR (95% CI) | Adjusted ORa (95% CI) | Crude OR (95% CI) | Adjusted ORa (95% CI) | ||||
| Age groups (years), n | |||||||
| 18–74 | 518 | 59 | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | |
| 75–83 | 427 | 134 | 2.76 (1.98–3.84) | 2.21 (1.47–3.32) | 2.76 (1.98–3.84) | 2.22 (1.48–3.34) | |
| 84 | 418 | 160 | 3.36 (2.43–4.65) | 2.62 (1.73–3.97) | 3.36 (2.43–4.65) | 2.64 (1.74–4.00) | |
| BMI (kg/m2), n | |||||||
| 18.5–24.9 | 390 | 56 | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | |
| <18.5 | 90 | 29 | 2.24 (1.36–3.71) | 1.86 (0.99–3.49) | 2.24 (1.36–3.71) | 1.90 (1.01–3.57) | |
| >24.9 | 229 | 32 | 0.97 (0.61–1.55) | 1.41 (0.79–2.51) | 0.97 (0.61–1.55) | 1.39 (0.78–2.48) | |
| Bedriddenb | 654 | 236 | 2.51 (1.83–3.45) | 1.72 (1.15– 2.58) | 2.51 (1.83–3.45) | 1.75 (1.16–2.62) | |
| Comorbidities, n | |||||||
| Hypertension | 730 | 220 | 1.43 (1.13–1.82) | ||||
| Chronic heart disease | 409 | 141 | 1.44 (1.14–1.83) | 1.44 (1.14–1.83) | |||
| Chronic kidney disease | 53 | 25 | 1.88 (1.15–3.08) | 1.88 (1.15–3.08) | |||
| Malignancy | 175 | 99 | 2.65 (2.00–3.51) | 3.67 (2.51–5.36) | 2.65 (2.00–3.51) | 3.66 (2.50–5.34) | |
| Groups, n | |||||||
| Sepsis (vs. nonsepsis) | 636 | 299 | 6.33 (4.65–8.62) | 3.95 (2.76–5.63) | |||
| qSOFA ≥2 (vs. qSOFA <2) | 337 | 303 | 18.45 (13.35–25.50) | 14.34 (10.17–20.23) | |||
| qSOFA(−)Sepsis(−) | 594 | 16 | 1.00 (ref) | 1.00 (ref) | |||
| qSOFA(−)Sepsis(+) | 432 | 34 | 2.71 (1.48–4.97) | 2.60 (1.39–4.83) | |||
| qSOFA(+)Sepsis(−) | 133 | 38 | 10.61 (5.74–19.59) | 9.26 (4.90–17.50) | |||
| qSOFA(+)Sepsis(+) | 204 | 265 | 48.23 (28.41–81.85) | 43.72 (25.30–75.53) | |||
a, adjusted for gender, age in age categories (18–74, 75–83 and ≥84 years), comorbidities such as malignancy and the disease related groups; b, BMI could not be calculated because these patients were bedridden. CI, confidence interval; OR, odds ratio; SOFA, sequential organ failure assessment.
Table S2. Risk factors of in-hospital mortality in patients with infection, by univariate and multivariate regression analysis.
| Variables | Control | Case | Model 1 | Model 2 | |||
|---|---|---|---|---|---|---|---|
| Crude OR (95% CI) | Adjusted ORa (95% CI) | Crude OR (95% CI) | Adjusted ORa (95% CI) | ||||
| Age groups (years), n | 1,363 | 353 | 1.05 (1.03–1.06) | 1.04 (1.02–1.05) | 1.05 (1.03–1.06) | 1.04 (1.02–1.05) | |
| BMI (kg/m2), n | |||||||
| 18.5–24.9 | 390 | 56 | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | |
| <18.5 | 90 | 29 | 2.24 (1.36–3.71) | 1.80 (0.96–3.41) | 2.24 (1.36–3.71) | 1.84 (0.97–3.50) | |
| >24.9 | 229 | 32 | 0.97 (0.61–1.55) | 1.39 (0.78–2.48) | 0.97 (0.61–1.55) | 1.37 (0.77–2.46) | |
| Bedriddenb | 654 | 236 | 2.51 (1.83–3.45) | 1.62 (1.08–2.43) | 2.51 (1.83–3.45) | 1.64 (1.09–2.47) | |
| Comorbidities, n | |||||||
| Hypertension | 730 | 220 | 1.43 (1.13–1.82) | 1.43 (1.13–1.82) | |||
| Chronic heart disease | 409 | 141 | 1.44 (1.14–1.83) | 1.44 (1.14–1.83) | |||
| Chronic kidney disease | 53 | 25 | 1.88 (1.15–3.08) | 1.88 (1.15–3.08) | |||
| Malignancy | 175 | 99 | 2.65 (2.00–3.51) | 3.52 (2.42–5.13) | 2.65 (2.00–3.51) | 3.59 (2.46–5.23) | |
| Groups, n | |||||||
| Sepsis (vs. nonsepsis) | 636 | 299 | 6.33 (4.65–8.62) | 3.83 (2.68–5.47) | |||
| qSOFA ≥2 (vs. qSOFA <2) | 337 | 303 | 18.45 (13.35–25.50) | 14.06 (9.97–19.83) | |||
| qSOFA(−)Sepsis(−) | 594 | 16 | 1.00 (ref) | 1.00 (ref) | |||
| qSOFA(−)Sepsis(+) | 432 | 34 | 2.71 (1.48–4.97) | 2.55 (1.37–4.75) | |||
| qSOFA(+)Sepsis(−) | 133 | 38 | 10.61 (5.74–19.59) | 9.20 (4.86–17.40) | |||
| qSOFA(+)Sepsis(+) | 204 | 265 | 48.23 (28.41–81.85) | 41.89 (24.22–72.44) | |||
a, adjusted for gender, age in age categories (18–74, 75–83 and ≥84 years), comorbidities such as malignancy and the disease related groups; b, BMI could not be calculated because these patients were bedridden. CI, confidence interval; OR, odds ratio; SOFA, sequential organ failure assessment.
Table 3. Risk factors of in-hospital mortality in patients with infection, by univariate and multivariate regression analysis.
| Variables | Control | Case | Model 1 | Model 2 | |||
|---|---|---|---|---|---|---|---|
| Crude OR (95% CI) | Adjusted ORa (95% CI) | Crude OR (95% CI) | Adjusted ORa (95% CI) | ||||
| Age groups (years), n | |||||||
| 18–64 | 381 | 25 | 1.00 (ref) | 1.00 (ref) | 1.00 (Ref) | 1.00 (Ref) | |
| 65–84 | 633 | 189 | 4.55 (2.94–7.04) | 3.21 (1.91–5.40) | 4.55 (2.94–7.04) | 3.22 (1.91–5.43) | |
| ≥85 | 349 | 139 | 6.07 (3.87–9.52) | 3.86 (2.22–6.71) | 6.07 (3.87–9.52) | 3.86 (2.21–6.71) | |
| BMI (kg/m2), n | |||||||
| 18.5–24.9 | 390 | 56 | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | 1.00 (ref) | |
| <18.5 | 90 | 29 | 2.24 (1.36–3.71) | 1.72 (0.91–3.25) | 2.24 (1.36–3.71) | 1.76 (0.93–3.34) | |
| >24.9 | 229 | 32 | 0.97 (0.61–1.55) | 1.29 (0.72–2.30) | 0.97 (0.61–1.55) | 1.28 (0.71–2.28) | |
| Bedriddenb | 654 | 236 | 2.51 (1.83–3.45) | 1.63 (1.08–2.45) | 2.51 (1.83–3.45) | 1.65 (1.10–2.49) | |
| Comorbidities, n | |||||||
| Hypertension | 730 | 220 | 1.43 (1.13–1.82) | ||||
| Chronic heart disease | 409 | 141 | 1.44 (1.14–1.83) | 1.44 (1.14–1.83) | |||
| Chronic kidney disease | 53 | 25 | 1.88 (1.15–3.08) | 1.88 (1.15–3.08) | |||
| Malignancy | 175 | 99 | 2.65 (2.00–3.51) | 3.72 (2.55–5.45) | 2.65 (2.00–3.51) | 3.71 (2.53–5.42) | |
| Groups, n | |||||||
| Sepsis (vs. nonsepsis) | 636 | 299 | 6.33 (4.65–8.62) | 3.85 (2.70–5.50) | |||
| qSOFA (vs. qSOFA <2) | 337 | 303 | 18.45 (13.35–25.50) | 13.92 (9.87–19.63) | |||
| qSOFA(−)sepsis(−) | 594 | 16 | 1.00 (ref) | 1.00 (ref) | |||
| qSOFA(−)sepsis(+) | 432 | 34 | 2.71 (1.48–4.97) | 2.59 (1.39–4.83) | |||
| qSOFA(+)sepsis(−) | 133 | 38 | 10.61 (5.74–19.59) | 9.20 (4.86–17.38) | |||
| qSOFA(+)sepsis(+) | 204 | 265 | 48.23 (28.41–81.85) | 42.02 (24.31–72.64) | |||
a, adjusted for gender, age in age categories (15–64, 65–84, and ≥85 years), comorbidities such as malignancy, and the disease-related groups; b, BMI could not be calculated because these patients were bedridden. CI, confidence interval; OR, odds ratio; SOFA, sequential organ failure assessment.
Table 4. Comparison of the area under the receiver operating characteristic curves of qSOFA and Sepsis-3 criteria for in-hospital mortality, ICU admission, and invasive ventilation.
| Variable | qSOFA | Sepsis |
|---|---|---|
| In-hospital mortality | 0.846 (0.824–0.868) | 0.834 (0.805–0.863) |
| ICU admission | 0.805 (0.775–0.835) | 0.884 (0.863–0.906) |
| Invasive mechanical ventilation | 0.842 (0.816–0.869) | 0.875 (0.848–0.901) |
The false-positive group comprised 171 patients (SOFA ≥2) among whom 38 (22.2%) died during hospitalization, and had an adjusted OR of 9.20 (95% CI: 4.86–17.38). In addition, the false-negative group comprised 466 patients (qSOFA <2, met sepsis criteria), with a hospital mortality rate of 7.3% (34/466), and an adjusted OR of 2.59 (95% CI: 1.39–4.83). In comparison, patients meeting neither qSOFA nor sepsis criteria had the lowest hospital mortality [2.6% (16/610)], whereas patients with both qSOFA ≥2 and sepsis had the highest hospital mortality [56.5% (265/469)], with adjusted OR of 42.02 (95% CI: 24.31–72.64) (Table 3).
Discussion
When qSOFA score was used as a screening tool in the non-ICU setting, it had a sensitivity of 50.2%, and a specificity of 78.1% for the diagnosis of sepsis-3. In addition, both nonseptic patients with qSOFA ≥2 [false positives, or qSOFA(+)sepsis(−) group in our study] and septic patients with qSOFA <2 [false negatives, or qSOFA(−)sepsis(+) group in our study] had a significantly higher mortality than patients meeting neither criteria.
Previous studies reported an even lower sensitivity of qSOFA score to detect sepsis (13-15). For example, both Guirgis and Szakmany collected vital signs and laboratory data during the first 24 hours of admission to calculate SOFA and qSOFA scores, and reported a sensitivity of 16% and 18.4% respectively (13,15). Likewise, Williams and colleagues recorded the most abnormal values in the emergency department, and found that qSOFA score had poor sensitivity (29.7%) for organ dysfunction, i.e., sepsis (13). In comparison, we used the maximum qSOFA score within 72 hours after onset of infection, and also reported an unsatisfactory, albeit higher, sensitivity of 50.2%, which was consistent with those in the two recent meta-analyses (14,17). The low sensitivity of qSOFA score in the diagnosis of sepsis might raise serious concerns. The validity of qSOFA score as a screening tool for sepsis should be re-evaluated (16), because a high sensitivity should be a prerequisite for any screening tool, which would trigger extensive workup to search for evidence of infection-induced organ dysfunction, i.e., sepsis. In fact, such a low sensitivity of qSOFA suggests that it would miss about half of the patients with sepsis (false-negatives), precluding detection of these patients who are at higher risk of death (adjusted OR 2.59) until development of overt organ failure.
In the meanwhile, those patients fulfilling qSOFA but not sepsis criteria (false positives) also had a significantly higher mortality (adjusted OR 9.20). Previous studies reported the presence of shock and multiple organ failure (including altered mental status) as independent risk factors for mortality in cohorts of patients with sepsis (27,28). As a result, for any patients with infection and qSOFA score ≥2, regardless of sepsis diagnosis or not, clinicians should start supportive therapy as soon as possible, as well as further investigation of sepsis or other etiologies.
Our study had some strengths. This study was based on a secondary analysis of all hospitalized patients in a subdistrict of Beijing (19). Clinical and laboratory data from all enrolled patients in this study were collected through manual review of medical records by two independent investigators. Moreover, although many studies have been published to compare the prognostic performance of qSOFA with that of SOFA score in patients with infection, no studies had yet investigated the clinical outcomes of those septic patients missed by qSOFA score <2 (false-negatives) and those nonseptic patients with qSOFA score ≥2 (false-positives).
Our study was also subject to some limitations. First, this study was based on a secondary analysis of a database not originally designed for this purpose. Second, maximum qSOFA score was not necessarily obtained before onset of sepsis. However, this approach was still valid because the laboratory results required for SOFA score were not available every day in general wards. Therefore, qSOFA might still serve as a prompt of pending or unrecognized sepsis. Third, we only included patients with confirmed infection in our cohort. In clinical practice, patients with unconfirmed but suspected infection might also be screened for the presence of organ dysfunction, therefore compromising the specificity of qSOFA score for the diagnosis of sepsis, even if the sensitivity remained unchanged. Furthermore, by including patients without infection (regardless of meeting qSOFA criteria or not), the high mortality rate of septic patients not identified by qSOFA score [i.e., sepsis(+)qSOFA(−)] are unlikely to be affected, but whether the mortality rate of nonseptic patients meeting qSOFA criteria [i.e., sepsis(−)qSOFA(+)] will change remains uncertain. Last, the difference between crude and adjusted ORs was significant, indicating the possibility of including inappropriate covariates in the multivariate model.
In conclusion, the results of our study confirmed the low sensitivity of qSOFA score in the diagnosis of sepsis, therefore questioning its value as a screening tool. In addition, qSOFA score ≥2 might identify a group of patients at higher risk of mortality, regardless of being septic or not. Further prospective cohort studies are needed to confirm our findings and to evaluate the predictive value of qSOFA score in different settings.
Acknowledgments
Funding: The study was supported, in part, by the Capital Clinical Application Research Grant (Z131107002213112) from the Science and Technology Commission of Beijing, and the Chinese Academy of Medical Sciences (CAMS) Innovation Fund for Medical Sciences (CIFMS) 2016-I2M-1–014. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the article
Ethical Statement: The study was approved by the ethics committee of Peking Union Medical College Hospital and informed consent was waived.
Footnotes
Conflicts of Interest: The authors have no conflicts of interest to declare.
References
- 1.Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. Chest 1992;101:1644-55. 10.1378/chest.101.6.1644 [DOI] [PubMed] [Google Scholar]
- 2.Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol 2013;13:862-74. 10.1038/nri3552 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Churpek MM, Zadravecz FJ, Winslow C, et al. Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients. Am J Respir Crit Care Med 2015;192:958-64. 10.1164/rccm.201502-0275OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kaukonen KM, Bailey M, Pilcher D, et al. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med 2015;372:1629-38. 10.1056/NEJMoa1415236 [DOI] [PubMed] [Google Scholar]
- 5.Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Intensive Care Med 2003;29:530-8. [DOI] [PubMed] [Google Scholar]
- 6.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:801-10. 10.1001/jama.2016.0287 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Seymour CW, Liu V, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:762-74. 10.1001/jama.2016.0288 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Vincent JL, Martin GS, Levy MM. qSOFA does not replace SIRS in the definition of sepsis. Crit Care 2016;20:210. 10.1186/s13054-016-1389-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sartelli M, Kluger Y, Ansaloni L, et al. Raising concerns about the sepsis-3 definitions. World J Emerg Surg 2018;13:6. 10.1186/s13017-018-0165-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Freund Y, Lemachatti N, Krastinova E, et al. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department JAMA 2017;317:301-8. [DOI] [PubMed]
- 11.Churpek MM, Snyder A, Han X, et al. Quick Sepsis-related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients outside the Intensive Care Unit. Am J Respir Crit Care Med 2017;195:906-11. 10.1164/rccm.201604-0854OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Finkelsztein EJ, Jones DS, Ma KC, et al. Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit. Crit Care 2017;21:73. 10.1186/s13054-017-1658-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Williams JM, Greenslade JH, McKenzie JV, et al. SIRS, qSOFA and organ dysfunction: insights from a prospective database of emergency department patients with infection. Chest 2017;151:586-96. 10.1016/j.chest.2016.10.057 [DOI] [PubMed] [Google Scholar]
- 14.Song JU, Sin CK, Park HK, et al. Performance of the quick Sequential (sepsis-related) Organ Failure Assessment score as a prognostic tool in infected patients outside the intensive care unit: a systematic review and meta-analysis. Crit Care 2018;22:28. 10.1186/s13054-018-1952-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Guirgis FW, Puskarich MA, Smotherman C, et al. Development of a simple Sequential Organ Failure Assessment Score for risk assessment of emergency department patients with sepsis. J Intensive Care Med 2017. [Epub ahead of print]. 10.1177/0885066617741284 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Szakmany T, Pugh R, Kopczynska M, et al. Defining sepsis on the wards: results of a multi-centre point-prevalence study comparing two sepsis definitions. Anaesthesia 2018;73:195-204. 10.1111/anae.14062 [DOI] [PubMed] [Google Scholar]
- 17.Serafim R, Gomes JA, Salluh J, et al. A comparison of the quick-SOFA and systemic inflammatory response syndrome criteria for the diagnosis of sepsis and prediction of mortality: a systematic review and meta-analysis. Chest 2018;153:646-55. 10.1016/j.chest.2017.12.015 [DOI] [PubMed] [Google Scholar]
- 18.Du B, Weng L. Systemic inflammatory response syndrome, sequential organ failure assessment, and quick sequential organ failure assessment: more pieces needed in the sepsis puzzle. J Thorac Dis 2017;9:452-4. 10.21037/jtd.2017.02.99 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Zhou J, Tian H, Du X, et al. Population-based epidemiology of sepsis in a subdistrict of Beijing. Crit Care Med 2017;45:1168-76. 10.1097/CCM.0000000000002414 [DOI] [PubMed] [Google Scholar]
- 20.Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373-83. 10.1016/0021-9681(87)90171-8 [DOI] [PubMed] [Google Scholar]
- 21.McCabe WR, Jackson GG. Gram-negative bacteremia. I: etiology and ecology. Arch Intern Med 1962;110:847-55. 10.1001/archinte.1962.03620240029006 [DOI] [Google Scholar]
- 22.Knaus WA, Draper EA, Wagner DP, et al. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818-29. 10.1097/00003246-198510000-00009 [DOI] [PubMed] [Google Scholar]
- 23.Vincent JL, de Mendonca A, Cantraine F, et al. Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Crit Care Med 1998;26:1793-800. 10.1097/00003246-199811000-00016 [DOI] [PubMed] [Google Scholar]
- 24.Garner JS, Jarvis WR, Emori TG, et al. CDC definitions for nosocomial infections, 1988. Am J Infect Control 1988;16:128-40. 10.1016/0196-6553(88)90053-3 [DOI] [PubMed] [Google Scholar]
- 25.Orimo H, et al. Reviewing the definition of elderly. Nihon Ronen Igakkai Zasshi 2006;43:27-34. 10.1111/j.1447-0594.2006.00341.x [DOI] [PubMed] [Google Scholar]
- 26.Melzer D, Tavakoly B, Winder RE, et al. Much more medicine for the oldest old: trends in UK electronic clinical records. Age Ageing 2015;44:46-53. 10.1093/ageing/afu113 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Barriere SL, Lowry SF. An overview of mortality risk prediction in sepsis. Crit Care Med 1995;23:376-93. 10.1097/00003246-199502000-00026 [DOI] [PubMed] [Google Scholar]
- 28.Angus DC, Wax RS. Epidemiology of sepsis: an update. Crit Care Med 2001;29:S109-16. 10.1097/00003246-200107001-00035 [DOI] [PubMed] [Google Scholar]