Abstract
Background
We aim to compare the prognostic value of Quick Sepsis-Related Organ Failure Assessment (qSOFA) and the previous Systemic Inflammatory Response Syndrome (SIRS) criteria, the National Early Warning Score (NEWS) and along with their combinations in the emergency department (ED).
Methods
This single-centre prospective study recruited a convenience sample of unselected ED patients triaged as category 2 (Emergency) and 3 (Urgent). Receiver Operating Characteristic analyses were performed to determine the Area Under the Curve (AUC), along with sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios for the various scores.
Results
Of 1253 patients recruited, overall 30-day mortality was 5.7%. The prognostic value for prediction of 30-day mortality, with AUCs for qSOFA ≥2, SIRS ≥2, NEWS ≥5, qSIRS (qSOFA + SIRS) ≥2 and NSIRS (NEWS + SIRS) ≥5 of 0.56 (95%CI 0.53–0.58), 0.61 (95%CI 0.58–0.64), 0.61 (95%CI 0.58–0.64), 0.64 (95%CI 0.62–0.67) and 0.61 (95%CI 0.58–0.63), respectively. Using pairwise comparisons of ROC curves, NEWS ≥5 and qSIRS ≥2 were better than qSOFA ≥2 at predicting 30-day mortality.
Conclusions
Among unselected emergency and urgent ED patients, the prognostic value for NEWS and qSIRS were greater than qSOFA, Combinations of qSOFA and SIRS could improve the predictive value for 30-day mortality for ED patients.
Key messages
NEWS ≥5 and qSIRS ≥2 were better than qSOFA ≥2 at predicting 30-day mortality in ED patients.
Combinations of qSOFA and SIRS could improve the predictive value for 30-day mortality for ED patients.
Keywords: qSOFA, SIRS, NEWS, qSIRS, NSIRS, lactate, emergency department
Introduction
In 2016, the Third International Consensus meeting for Sepsis and Septic Shock (Sepsis-3) revised the definition of sepsis to “life-threatening organ dysfunction caused by a dysregulated host response to infection” [1]. Together with the updated definition of sepsis, new clinical criteria termed the ‘quick Sepsis-Related Organ Failure Assessment’ (qSOFA) were also introduced to identify high-risk patients outside the intensive care setting with suspected infection [1]. A positive qSOFA is defined as two or more of the following variables: (1) altered mentation (2) systolic blood pressure ≤100 mmHg, and (3) respiratory rate ≥22/min.
The previous criteria- Systemic Inflammatory Response Syndrome (SIRS) [2] - were removed from the current sepsis definition [1]. In parallel with SIRS, the Sepsis-3 task force has also removed the concept of “severe sepsis”, which was defined as ≥2 elements of SIRS and a blood lactate concentration of >2 mmol/l [2]. Lactate was not included for screening of sepsis in an infected patient on the basis that “lactate measurement offered no meaningful change in the predictive validity beyond two or more qSOFA criteria in the identification of patients likely to be septic” [1].
The original validation study of qSOFA was derived retrospectively from datasets in both the Intensive Care Unit (ICU) and non-ICU settings with suspected infection [3]. It was demonstrated that a score of qSOFA ≥ 2 may predict mortality in ICU and non-ICU setting with the area under the curve (AUC) of 0.66(0.64–0.68) and 0.81 (0.80–0.82), respectively. A recent prospective study shows qSOFA ≥ 2 predicts in-hospital mortality in emergency department (ED) patients with suspected infection with AUC of 0.80 (0.74–0.85), which is better than that of SIRS ≥ 2 (AUC: 0.65, 95%CI 0.59–0.70) [4].
In contrast, studies also demonstrated similar prognostic performances regardless of the choice of qSOFA or SIRS [5,6]. Recently a retrospective study demonstrated that combined qSOFA and SIRS are more accurate than qSOFA alone in predicting in-hospital mortality in patients with surgical sepsis [7]. In our systematic review of 45 studies with 431,634 patients, we showed that qSOFA for in-hospital mortality in all patients (with or without suspected infection) was poor [8].
Thus in the present study, we aimed to prospectively validate the qSOFA tool in urgent and emergency patients presenting to a university hospital ED in Hong Kong. We also compared the prognostic value of qSOFA and the previous criteria, SIRS, as well as a commonly used early warning score, the National Early Warning Score (NEWS) [9]. We also studied the prognostic performance of their combinations of qSIRS(qSOFA + SIRS) and NSIRS (NEWS + SIRS).
Finally, we investigated the potential incremental value of a single ED admission lactate level on the performance of the qSOFA, SIRS, NEWS, qSIRS, and NSIRS. We also investigate and report the prognostic value of a single lactate level for the study outcomes. Symptoms of infection are highly variable, detection of sepsis or infection may be clinically difficult, especially in the ED. Thus, in the present study, we recruited a convenience sample of unselected ED patients triaged as category 2 (Emergency) and 3 (Urgent), not just patients with suspected infection/sepsis. To our knowledge, this study is the first study to compare qSOFA, SIRS, NEWS, and their combinations (qSIRS and NSIRS) prospectively in a predominantly Chinese population in general emergency patients with or without infection.
Materials and methods
Study design
This was a prospective observational study in a university teaching hospital ED in Hong Kong. Ethics approval was obtained from the Joint Chinese University of Hong Kong – New Territories East Cluster Clinical Research Ethics Committee (CRE-2016.236). The ED receives over 144,000 new patients per annum and has an admission rate of 30%. Patients aged ≥18 years presenting to the ED between July 2016 to June 2017 triaged in the emergency (II) and urgent (III) categories were recruited. In Hong Kong, patients are triaged into five triage categories: (1) Critical, (2) Emergency, (3) Urgent, (4) Semi-urgent and (5) Non-urgent by trained ED nurses following local triage guidelines [10]. Patients aged <18 years, or pregnant, or patients presenting to the ED outside of office hours were excluded due to the limited availability of research staff.
Definition
A positive Quick Sepsis-Related Organ Failure Assessment (qSOFA) is defined as two or more of the following variables: (1) altered mental status, (2) systolic blood pressure ≤100 mmHg, and (3) respiratory rate ≥22/min [1].
The Systemic Inflammatory Response Syndrome (SIRS) is defined as the presence of two or more of the following variables: (1) temperature of more than 38 °C or less than 36 °C, (2) heart rate of more than 90 beats per minute, (3) respiratory rate of more than 20 breaths per minute, or a PaCO2 of less than 4.3 kPa, (4) abnormal white blood cell count (<4 × 109 cells/L or >12 × 109 cells/L), or more than 10% immature band forms [2].
The National Early Warning Score (NEWS) is a score made up of seven physiological parameters: (1) respiratory rate, (2) oxygen saturations, (3) systolic blood pressure, (4) pulse rate, (5) level of consciousness (AVPU score), (6) temperature, and (7) supplementary oxygen used [9].
Combination of scores
qSIRS scores were generated by combining the criteria of the SIRS and the qSOFA scores, while NSIRS scores were generated by combining the NEWS and SIRS criteria for each patient. For qSIRS, both qSOFA and SIRS include a respiratory rate, so only one respiratory rate was included in the final score. A cut-off of ≥20 breaths per min was used [7]. For NSIRS, if vital signs are duplicated in both scores, only one was included and the worst score was calculated. As NEWS includes all vital signs of qSOFA, the combination of the qSOFA and NEWS scores is the same as NEWS itself.
Outcome and other variables
The primary outcome was 30-day mortality. The secondary outcomes were 7-day mortality and ICU admission. The triage physiological measures were used to calculate qSOFA, SIRS, NEWS, qSIRS, and NSIRS. The 30-day mortality was retrieved from the centralized, computerized medical data system of the hospital, which captures and updates clinical information for all patients in the hospital in real time. The AVPU (Alert, responds to Voice, responds to Pain, Unresponsive) scale was used to identify and characterize altered mentation in the present study, with anything other than ‘Alert’ being taken to indicate altered mentation. Venous blood samples (∼1 ml) were collected by peripheral venepuncture. Lactate measurements were performed using an ED blood gas analyser [RAPIDPoint® 500 Blood Gas Systems (Siemens, US)].
Statistical analysis
Descriptive statistics of patient characteristics and clinical variables included frequencies, percentage, mean and standard deviations were calculated. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and positive and negative likelihood ratios were calculated. Receiver operating characteristic (ROC) analyses were performed to identify the area under the curve (AUC) to predict the probability of 30-day mortality, 7-day mortality, and ICU admission.
Multiple logistic regression analyses of qSOFA, SIRS, NEWS, qSIRS and NSIRS, and venous lactate levels using the primary outcome measure as the independent variable were performed. For all analyses, a two-tailed P < 0.05 was considered statistically significant. All data were analysed by using SPSS version 23.0 for Windows (IBM SPSS Statistics, Armonk, NY, USA) and MedCalc version 15 for Windows (MedCalc Software bvba).
Results
Basic characteristics
1253 patients were recruited during the study period. The median age was 72 years (IQR: 59–84), and 638 (50.9%) were male. Overall 30-day mortality was 5.7% (71/1253). The patient characteristics are shown in Table 1. Of 1253 patients, 18 (1.4%) patients had qSOFA scores ≥2, 276 (22.0%) patients had SIRS ≥2, and 175 (14.0%) patients had NEWS ≥ 5. In addition, 296 (25.5%) and 197 (16.9%) patients had qSIRS (qSOFA + SIRS) scores ≥2 and 197 (16.9%) patients had NSIRS (NEWS + SIRS) scores ≥5.
Table 1.
Comparison of factors for predicting 30-day mortality in 1253 patients presenting to the emergency department triaged in the urgent (III) and emergency (II) categories.
| All (n = 1253) | Survived (n = 1182) | 30-d mortality (n = 71) | p Value | |
|---|---|---|---|---|
| Male (No. %) | 638 (50.9) | 599 (50.7) | 39 (54.9) | 0.49 |
| Age | 72 [59–84] | 72 [58–83] | 79 [68–88] | 0.0002 |
| Triage vital sign | ||||
| Heart rate (bpm) | 85 [70–100] | 84 [70–100] | 96 [81–105] | <0.0001 |
| SBP (mmHg) | 141 [123–159] | 141 [124–160] | 133 [115–151] | 0.0199 |
| DBP (mmHg) | 77 [65–87] | 77 [65–87] | 73 [60–88] | 0.098 |
| Temperature (°C) | 63.8 [36.4–37.3] | 36.8 [36.4–37.3] | 37.0 [36.3–37.6] | 0.34 |
| Respiratory rate (bpm) | 16 [14–20] | 16 [14–19] | 20 [16–23] | <0.0001 |
| Score | ||||
| qSOFA (No. %) | ||||
| qSOFA ≥ 2 | 18 (1.4) | 10 (0.8) | 8 (11.2) | <0.0001 |
| qSOFA< 2 | 1193 (95.2) | 1134 (95.9) | 59 (83.1) | |
| Missing data | 42 (3.4) | 38 (3.2) | 4 (5.6) | |
| qSOFA ≥ 1 | 248 (19.8) | 221 (18.7) | 27 (38.0) | 0.0001 |
| qSOFA< 1 | 963 (76.9) | 923 (78.1) | 40 (56.3) | |
| Missing data | 42 (3.4) | 38 (3.2) | 4 (5.6) | |
| SIRS (No. %) | ||||
| SIRS ≥ 2 | 276 (22.0) | 248 (21.0) | 28 (39.4) | 0.0001 |
| SIRS < 2 | 904 (72.1) | 869 (73.5) | 35 (49.3) | |
| Missing data | 73 (5.8) | 65 (5.5) | 8 (11.3) | |
| NEWS (No. %) | ||||
| NEWS ≥ 5 | 175 (14.0) | 151 (12.8) | 24 (33.8) | <0.0001 |
| NEWS < 5 | 1034 (82.5) | 991 (83.8) | 43 (60.6) | |
| Missing data | 44 (3.5) | 40 (3.4) | 4 (5.6) | |
| NEWS ≥ 8 | 38 (3.0) | 29 (2.5) | 9 (12.7) | <0.0001 |
| NEWS < 8 | 1171 (93.5) | 1113 (94.2) | 58 (81.7) | |
| Missing data | 44 (3.5) | 40 (3.4) | 4 (5.6) | |
| qSIRS (No. %) | ||||
| qSIRS ≥ 2 | 296 (23.6) | 263 (22.2) | 33 (46.5) | <0.0001 |
| qSIRS< 2 | 882 (70.4) | 852 (72.1) | 30 (42.3) | |
| Missing data | 75 (6.0) | 67 (5.7) | 8 (11.3) | |
| NSIRS (No. %) | ||||
| NSIRS ≥ 5 | 197 (15.7) | 174 (14.7) | 23 (32.4) | <0.0001 |
| NSIRS < 5 | 981 (78.3) | 941 (79.6) | 40 (56.3) | |
| Missing data | 75 (6.0) | 67 (5.7) | 8 (11.3) | |
| Lactate level | ||||
| Venous blood lactate (VBG) | 1.7 [1.3 -2.3] | 1.7 [1.3–2.2] | 2.4 [1.8–3.9] | <0.0001 |
| VL-VBG ≥ 2 (No. %) | 436 (34.8) | 394 (33.3) | 42 (59.2) | <0.0001 |
| VL-VBG < 2 (No. %) | 812 (64.8) | 785 (66.4) | 27 (38.0) | |
| Missing data | 5 (0.4) | 3 (0.3) | 2 (2.8) | |
30-day mortality at various levels of qSOFA, SIRS, NEWS, qSIRS and NSIRS are shown in Figure 1 and Table 2. Median venous blood lactate level was 1.7 mmol/L [IQR: 1.7–2.3] and 34.8% (436/1253) of patients had lactate levels ≥2mmol/L. Among 18 patients with qSOFA ≥ 2, 44.4% (8/18) died within 30 days. Of 1193 patients with qSOFA < 2, 4.9% (59/1193) died within 30 days. The proportion of patients with missing data was small, with 42 (3.4%), 73 (5.8%) and 44 (3.5%) patients having missing parameters in qSOFA, SIRS and NEWS, respectively. Venous blood lactate was missing in 5 (0.4%) patients.
Figure 1.
Histogram of 30-day mortality at various level of qSOFA, SIRS, NEWS, qSIRS and NSIRS of 1253 patients.
Table 2.
30-day mortality at various level of qSOFA, SIRS, NEWS, qSIRS and NSIRS of 1253 patients.
| Survived (n = 1182) | 30-day mortality (n = 71) | |||
|---|---|---|---|---|
| qSOFA | 923 | (95.8%) | 40 | (4.2%) |
| 0 | 211 | (91.7%) | 19 | (8.3%) |
| 1 | 10 | (55.6%) | 8 | (44.4%) |
| 2 | 0 | (90.5%) | 4 | (9.5%) |
| Missing data | ||||
| SIRS | ||||
| 0 | 490 | (97.4%) | 13 | (2.6%) |
| 1 | 379 | (94.5%) | 22 | (5.5%) |
| 2 | 168 | (90.8%) | 17 | (9.2%) |
| 3 | 65 | (89.0%) | 8 | (11.0%) |
| 4 | 15 | (83.3%) | 3 | (16.7%) |
| Missing data | 65 | (89.0%) | 8 | (11.0%) |
| NEWS | ||||
| 0 | 357 | (97.5%) | 9 | (2.5%) |
| 1 | 251 | (96.9%) | 8 | (3.1%) |
| 2 | 157 | (94.6%) | 9 | (5.4%) |
| 3 | 132 | (95.0%) | 7 | (5.0%) |
| 4 | 94 | (90.4%) | 10 | (9.6%) |
| 5 | 58 | (93.5%) | 4 | (6.5%) |
| 6 | 41 | (83.7%) | 8 | (16.3%) |
| 7 | 23 | (88.5%) | 3 | (11.5%) |
| 8 | 14 | (82.4%) | 3 | (17.6%) |
| 9 | 9 | (75.0%) | 3 | (25.0%) |
| 10 | 3 | (60.0%) | 2 | (40.0%) |
| 11 | 2 | (100.0%) | 0 | (0.0%) |
| 12 | 1 | (50.0%) | 1 | (50.0%) |
| Missing data | 40 | (90.9%) | 4 | (9.1%) |
| qSIRS | ||||
| 0 | 471 | (97.7%) | 11 | (2.3%) |
| 1 | 381 | (95.2%) | 19 | (4.7%) |
| 2 | 169 | (89.4%) | 20 | (10.6%) |
| 3 | 73 | (89.0%) | 9 | (11.0%) |
| 4 | 21 | (87.5%) | 3 | (12.5%) |
| 5 | 0 | (0.0%) | 1 | (100.0%) |
| Missing data | 67 | (89.3%) | 8 | (10.7%) |
| NSIRS | ||||
| 0 | 314 | (98.7%) | 4 | (1.3%) |
| 1 | 240 | (95.6%) | 11 | (4.4%) |
| 2 | 161 | (95.8%) | 7 | (4.2%) |
| 3 | 137 | (96.5%) | 5 | (3.5%) |
| 4 | 89 | (87.3%) | 13 | (12.7%) |
| 5 | 69 | (97.2%) | 2 | (2.8%) |
| 6 | 46 | (82.1%) | 10 | (17.9%) |
| 7 | 20 | (90.9%) | 2 | (9.1%) |
| 8 | 21 | (84.0%) | 4 | (16.0%) |
| 9 | 9 | (90.0%) | 1 | (10.0%) |
| 10 | 5 | (71.4%) | 2 | (28.6%) |
| 11 | 2 | (66.7%) | 1 | (33.3%) |
| 12 | 1 | (100.0%) | 0 | (0.0%) |
| 13 | 1 | (50.0%) | 1 | (50.0%) |
| Missing data | 67 | (89.3%) | 8 | (10.7%) |
Prognostic value of qSOFA, SIRS and NEWS
30-Day mortality
Table 3 and Figure 2 show the prognostic value of qSOFA, SIRS, NEWS, qSIRS and NSIRS for prediction of 30-day mortality. The AUC of qSOFA ≥ 2, SIRS ≥ 2, NEWS ≥ 5, qSIRS ≥ 2 and NSIRS ≥ 5 to predict of 30-day mortality were 0.56 (95%CI 0.53–0.58), 0.61 (95%CI 0.58–0.64), 0.61 (95%CI 0.58–0.64), 0.64 (95%CI 0.62–0.67) and 0.61 (95%CI 0.58–0.63), respectively. Among patients with all scores available (n = 1177), pairwise comparison of ROC curves showed NEWS ≥ 5 (p = 0.036) and qSIRS (p = 0.004) were better at predicting 30-day mortality in ED patients than qSOFA ≥ 2, but no difference was found among other scores.
Table 3.
Prognostic value of qSOFA ≥ 2, SIRS ≥ 2, NEWS ≥ 5, qSIRS ≥ 2, NSIRS ≥ 5 and venous lactate ≥2 for predicting 30-day mortality, 7-day mortality and ICU admission in 1253 patients presenting to the emergency department triaged in the urgent (III) and emergency (II) categories.
| qSOFA ≥ 2 (n = 1211) | SIRS ≥ 2 (n = 1180) | NEWS ≥ 5 (n = 1209) | qSIRS ≥ 2 (n = 1178) | NSIRS ≥ 5 (n = 1178) | Venous lactate ≥ 2 (n = 1248) | |
|---|---|---|---|---|---|---|
| 30-day mortality | ||||||
| Sensitivity % (95%CI) | 11.94 (5.30 to 22.18) | 44.44 (31.92 to 57.51) | 35.82 (24.47 to 48.47) | 52.38 (39.41 to 65.12) | 36.51 (24.73 to 49.60) | 60.87 (48.37 to 72.40) |
| Specificity % (95%CI) | 99.13 (98.40 to 99.58) | 77.80 (75.24 to 90.20) | 86.78 (84.67 to 88.69) | 76.41 (73.81 to 78.88) | 84.40 (82.13 to 86.48) | 66.58 (63.81 to 69.27) |
| AUC (cutoff) (95%CI) | 0.56 (0.53 to 0.58) | 0.61 (0.58 to 0.64) | 0.61 (0.58 to 0.64) | 0.64 (0.62 to 0.67) | 0.61 (0.58 to 0.63) | 0.64 (0.61 to 0.66) |
| Positive likelihood ratio (95%CI) | 13.66 (5.57 to 33.48) | 2.00 (1.49 to 2.69) | 2.71 (1.90 to 3.86) | 2.22 (1.72 to 2.88) | 2.34 (1.64 to 3.33) | 1.82 (1.48 to 2.24) |
| Negative likelihood ratio (95%CI) | 0.89 (0.81 to 0.97) | 0.71 (0.57 to 0.89) | 0.74 (0.62 to 0.89) | 0.62 (0.48 to 0.81) | 0.75 (0.62 to 0.91) | 0.59 (0.44 to 0.79) |
| Positive predictive value % (95%CI) | 44.4 (21.53 to 69.24) | 10.14 (6.85 to 14.33) | 13.71 (8.99 to 19.72) | 11.15 (8.84 to 13.97) | 11.68 (8.50 to 15.84) | 9.63 (7.03 to 12.80) |
| Negative predictive value % (95%CI) | 95.05 (93.67 to 96.21) | 96.13 (94.66 to 97.29) | 95.84 (94.44 to 96.97) | 96.60 (95.63 to 97.36) | 95.92 (95.11 to 96.60) | 96.67 (95.20 to 97.80) |
| 7-day mortality | ||||||
| Sensitivity % (95%CI) | 15.63 (5.28 to 32.79) | 62.07 (42.26 to 79.31) | 50.00 (31.89 to 68.11) | 65.52 (45.67 to 82.06) | 48.28 (29.45 to 67.47) | 60.61 (42.14 to 77.09) |
| Specificity % (95%CI) | 98.90 (98.12 to 99.41) | 77.59 (75.06 to 79.96) | 86.49 (84.41 to 88.39) | 75.89 (73.31 to 78.34) | 84.07 (81.83 to 86.14) | 65.76 (63.02 to 68.43) |
| AUC (cutoff) (95%CI) | 0.57 (0.54 to 0.60) | 0.70 (0.67 to 0.72) | 0.68 (0.66 to 0.71) | 0.71 (0.68 to 0.73) | 0.66 (0.63 to 0.69) | 0.63 (0.61 to 0.66) |
| Positive likelihood ratio (95%CI) | 14.17 (5.37 to 37.38) | 2.45 (1.65 to 3.51) | 3.70 (2.54 to 5.39) | 2.72 (2.05 to 3.61) | 3.03 (2.03 to 4.52) | 1.77 (1.33 to 2.2.36) |
| Negative likelihood ratio (95%CI) | 0.85 (0.74 to 0.99) | 0.49 (0.31 to 0.78) | 0.58 (0.41 to 0.82) | 0.45 (0.28 to 0.75) | 0.62 (0.43 to 0.88) | 0.60 (0.39 to 0.92) |
| Positive predictive value % (95%CI) | 27.78 (12.73 to 50.36) | 6.52 (4.90 to 8.64) | 9.14 (6.47 to 12.77) | 6.42 (4.91 to 8.35) | 7.11 (4.88 to 10.24) | 4.59 (3.49 to 60.1) |
| Negative predictive value % (95%CI) | 97.74 (97.38 to 98.04) | 98.78 (98.07 to 99.23) | 98.45 (97.82 to 98.90) | 98.87 (98.14 to 99.31) | 98.47 (97.84 to 98.92) | 98.40 (97.57 to 98.84) |
| ICU admission | ||||||
| Sensitivity % (95%CI) | 0.00 (0.00 to 33.63) | 37.50 (8.52 to 75.51) | 44.44 (13.70 to 78.80) | 37.50 (8.52 to 75.51) | 50.00 (15.70 to 84.30) | 77.78 (39.99 to 97.19) |
| Specificity % (95%CI) | 98.48 (97.61 to 99.10) | 77.10 (74.56 to 79.49) | 85.96 (83.84 to 87.89) | 75.33 (72.73 to 77.79) | 83.72 (81.47 to 85.81) | 65.33 (62.58 to 68.00) |
| AUC (cutoff) (95%CI) | 0.49 (0.46 to 0.52) | 0.57 (0.54 to 0.60) | 0.65 (0.62 to 0.68) | 0.56 (0.54 to 0.59) | 0.67 (0.64 to 0.70) | 0.72 (0.69 to 0.74) |
| Positive likelihood ratio (95%CI) | 0.00 | 1.64 (0.67 to 4.03) | 3.17 (1.50 to 6.66) | 1.52 (0.62 to 3.74) | 3.07 (1.52 to 6.22) | 2.24 (1.57 to 3.21) |
| Negative likelihood ratio (95%CI) | 1.02 (1.01 to 1.02) | 0.81 (0.47 to 1.39) | 0.65 (0.36 to 1.16) | 0.83 (0.49 to 1.42) | 0.60 (0.30 to 1.20) | 0.34 (0.10 to 1.16) |
| Positive predictive value % (95%CI) | 0.00 | 1.12 (0.46 to 2.71) | 2.35 (1.13 to 4.83) | 1.04 (0.43 to 2.52) | 2.08 (1.04 to 4.13) | 1.63 (1.14 to 2.31) |
| Negative predictive value % (95%CI) | 99.23 (99.23 to 99.24) | 99.44 (99.05 to 99.67) | 99.51 (99.13 to 99.73) | 99.43 (99.03 to 99.67) | 99.59 (99.18 to 99.79) | 99.75 (99.16 to 99.93) |
Figure 2.
Receiver operating characteristic graphs for 30-day and 7-day mortality and ICU admission.
7-Day mortality
The prognostic values of qSOFA, SIRS, NEWS, qSIRS and NSIRS for prediction of 7-day mortality were also analysed and are shown in Table 3 and Figure 2. Overall 7-day mortality was 2.6%. The prognostic value for prediction of 7-day mortality, with AUCs for qSOFA ≥ 2, SIRS ≥ 2, NEWS ≥ 5, qSIRS ≥ 2 and NSIRS ≥ 5 were 0.57 (95%CI 0.54–0.60), 0.70 (95%CI 0.67–0.72), 0.68 (95%CI 0.66–0.71), 0.71 (95%CI 0.68to 0.73) and 0.66 (95%CI 0.63 to 0.69), respectively. In addition, when the ROC curves were compared, the AUC of NEWS ≥ 5 (p = 0.014) and SIRS ≥ 2 (p = 0.012), qSIRS ≥ 2 (p = 0.030) and NSIRS ≥ 5 (p = 0.003) were better than qSOFA ≥ 2 to predict 7-day mortality in ED patients.
ICU admission
Among 1253 patients, 9 (0.72%)were admitted to ICU from the ED, none of whom had a qSOFA score ≥2. AUC of qSOFA ≥ 2, SIRS ≥ 2, NEWS ≥ 5 qSIRS ≥ 2 and NSIRS ≥ 5 to predict ICU admission were 0.49 (95%CI: 0.46 to 0.52), 0.57 (95%CI: 0.54 to 0.60), 0.65 (95% CI: 0.62 to 0.68), 0.56 (95%CI 0.54 to 0.59) and 0.67 (95%CI 0.64 to 0.70), respectively (Table 3 and Figure 2).
Prognostic value of venous blood lactate and its incremental effect on qSOFA, SIRS, NEWS, qSIRS and NSIRS
The AUC of a lactate level ≥ 2mmol/l to predict 30-day mortality was 0.64 (95% CI 0.61–0.66). The prognostic value of the combination(both positive) of lactate levels and five scores to predict outcomes is shown in Table 4. The combination (both positive)of lactate level ≥ 2mmol/l with qSOFA ≥ 2, SIRS ≥ 2, NEWS ≥ 5, qSIRS ≥ 2 and NSIRS ≥ 5, AUC were 0.54 (95%CI 0.51–0.56), 0.61 (95%CI 0.58–0.63), 0.59 (95%CI 0.56–0.61), 0.59 (95%CI 0.56–0.62) and 0.59 (95%CI 0.56–0.62), respectively. To predict 7-day mortality, the AUC of lactate ≥2mmol/L was 0.63 (95% CI 0.61–0.66). The combination (both positive)of lactate level ≥2mmol/L with qSOFA ≥ 2, SIRS ≥ 2, NEWS ≥ 5, qSIRS ≥ 2 and NSIRS ≥ 5, AUC were 0.56 (95%CI 0.53–0.59), 0.65 (95%CI 0.62–0.67), 0.63 (95%CI 0.60–0.66), 0.63 (95%CI 0.60 to 0.65) and 0.62 (95%CI 0.60 to 0.65), respectively.
Table 4.
Prognostic value of combinations (both positive) of lactate level (VL) ≥2mmol/l with qSOFA ≥ 2, SIRS ≥ 2, NEWS ≥ 5, qSIRS ≥ 2 and NSIRS ≥ 5 for predicting 30-day mortality, 7-day mortality and ICU admission in 1253 patients presenting to the emergency department triaged in the urgent (III) and emergency (II) categories.
| qSOFA ≥ 2 + VL ≥ 2 (n = 1211) | SIRS ≥ 2+ VL ≥ 2 (n = 1175) | NEWS ≥ 5 + VL ≥ 2 (n = 1204) | qSIRS ≥ 2+ VL ≥ 2 (n = 1204) | NSIRS ≥ 5 + VL ≥ 2 (n = 1204) | |
|---|---|---|---|---|---|
| 30-d mortality | |||||
| Sensitivity % (95%CI) | 7.46 (2.47 to 16.56) | 31.15 (19.90 to 44.29) | 23.08 (13.53 to 35.19) | 24.59 (14.46 to 37.29) | 24.59 (14.46 to 37.29) |
| Specificity % (95%CI) | 99.56 (98.98 to 99.86) | 90.13 (88.22 to 91.81) | 94.03 (92.49 to 95.33) | 92.82 (91.14 to 94.27) | 93.53 (91.92 to 94.90) |
| AUC (cutoff) (95%CI) | 0.54 (0.51 to 0.56) | 0.61 (0.58 to 0.63) | 0.59 (0.56 to 0.61) | 0.59 (0.56 to 0.62) | 0.59 (0.56 to 0.62) |
| Positive likelihood ratio (95%CI) | 17.07 (5.07 to 57.54) | 3.15 (2.09 to 4.77) | 3.87 (2.34 to 6.37) | 3.80 (2.32 to 6.22) | 3.42 (2.10 to 5.58) |
| Negative likelihood ratio (95%CI) | 0.93 (0.87 to 0.99) | 0.76 (0.64 to 0.91) | 0.82 (0.72 to 0.94) | 0.81 (0.70 to 0.94) | 0.81 (0.70 to 0.93) |
| Positive predictive value % (95%CI) | 50.00 (18.71 to 81.29) | 14.73 (9.11 to 22.04) | 18.07 (10.48 to 28.05) | 15.79 (10.33 to 23.39) | 17.24 (11.29 to 25.43) |
| Negative predictive value % (95%CI) | 94.84 (93.43 to 96.02) | 95.98 (94.61 to 97.09) | 95.54 (94.16 to 96.67) | 95.74 (95.11 to 96.29) | 95.77 (95.14 to 96.32) |
| 7-d mortality | |||||
| Sensitivity % (95%CI) | 12.90 (3.63 to 29.83) | 39.29 (21.50 to 59.42) | 32.3 (16.68 to 51.37) | 32.14 (15.88 to 52.35) | 32.14 (15.88 to 52.35) |
| Specificity % (95%CI) | 99.49 (98.89 to 99.81) | 89.71 (87.81 to 91.41) | 93.78 (92.24 to 95.10) | 93.19 (91.58 to 94.58) | 92.50 (90.82 to 93.96) |
| AUC (95%CI) | 0.56 (0.53 to 0.59) | 0.65 (0.62 to 0.67) | 0.63 (0.60 to 0.66) | 0.63 (0.60 to 0.65) | 0.62 (0.60 to 0.65) |
| Positive likelihood ratio (95%CI) | 24.56 (7.29to 82.82) | 3.82 (2.34to 6.24) | 5.18 (2.97 to 9.04) | 4.72 (2.65 to 8.43) | 4.29 (2.41 to 7.62) |
| Negative likelihood ratio (95%CI) | 0.88 (0.77to 1.00) | 0.68 (0.50 to 0.91) | 0.72 (0.57 to 0.92) | 0.73 (0.56 to 0.94) | 0.73 (0.57 to 0.95) |
| Positive predictive value % (95%CI) | 40.00 (16.53 to 69.18) | 8.53 (5.40 to 13.22) | 12.05 (7.28 to 19.29) | 10.35 (6.07 to 17.08) | 9.47 (5.56 to 15.69) |
| Negative predictive value % (95%CI) | 97.74 (97.42 to 98.02) | 98.38 (97.82 to 98.79) | 98.13 (97.62 to 98.53) | 98.25 (97.76 to 98.64) | 98.24 (97.74 to 98.63) |
| ICU admission | |||||
| Sensitivity % (95%CI) | 0.00 (0.00 to 33.63) | 25.00 (3.19 to 65.09) | 22.22 (2.81 to 60.01) | 25.00 (3.19 to 65.09) | 25.00 (3.19 to 65.09) |
| Specificity % (95%CI) | 99.15 (98.45 to 99.59) | 89.41 (87.49 to 91.13) | 93.37 (91.80 to 94.73) | 92.88 (91.23 to 94.29) | 92.19 (90.48 to 93.67) |
| AUC (cutoff) (95%CI) | 0.50 (0.47 to 0.53) | 0.57 (0.54 to 0.60) | 0.58 (0.55 to 0.61) | 0.59 (0.56 to 0.62) | 0.59 (0.56 to 0.61) |
| Positive likelihood ratio (95%CI) | 0.00 | 2.36 (0.70 to 7.93) | 3.35 (0.70 to 11.60) | 3.51 (1.04 to 11.87) | 3.20 (0.95 to 10.80) |
| Negative likelihood ratio (95%CI) | 1.01 (1.00 to 1.01) | 0.84 (0.56 to 1.25) | 0.83 (0.59 to 1.18) | 0.81 (0.54 to 1.21) | 0.81 (0.55 to 1.21) |
| Positive predictive value % (95%CI) | 0.00 | 1.61 (0.49 to 5.22) | 2.50 (0.74 to 8.15) | 2.38 (0.72 to 7.62) | 2.17 (0.65 to 6.98) |
| Negative predictive value % (95%CI) | 99.24 (99.23 to 99.24) | 99.42 (99.14 to 99.61) | 99.37 (99.11 to 99.55) | 99.44 (99.17 to 99.63) | 99.44 (99.16 to 99.62) |
Discussion
In this prospective study of 1,253 unselected ED adult patients (with and without suspected infection), the prognostic values for NEWS and qSIRS (qSOFA + SIRS) were greater than qSOFA. In another retrospective study in 240,000 admitted patients with or without infection in the UK, NEWS performed significantly better at discriminating in–hospital death than qSOFA [11].
Studies of infected patients have also been in agreement with our findings. In a retrospective study of patients with surgical sepsis, the combination of qSOFA + SIRS (qSIRS) was more accurate than qSOFA alone in predicting mortality [7]. Other groups have demonstrated that NEWS and modified early warning score (MEWS) are more accurate than qSOFA for predicting poor outcomes for infected patients [12].
In the present study, we also found that NEWS, SIRS, qSIRS and NSIRS had a better performance than qSOFA for predicting 7-day mortality. However, in contrast, another study on patients with suspected infection who eventually required ICU admission showed that qSOFA had greater accuracy than SIRS [13]. In our study, the ICU admission rate was 0.7% while all patients were recruited from ICU in the study by Tusgul et al. [13]. These differences suggest that the performance of the scoring system could be affected by the availability of critical care resources and the admission policies for critical care units, which will vary enormously from country to country and from hospital to hospital.
In our study, the prognostic value of qSOFA for predicting 30-day mortality was poor. Our results show that the AUC of qSOFA ≥2 for 30-day mortality in unselected ED patients was 0.56 (0.53 to 0.58) suggesting that qSOFA was not good enough for practical clinical ED use.In line with 30-day-mortality, our study also reported poor AUCs for qSOFA ≥ 2 to predict 7-day mortality (0.57, 0.54 to 0.60) and ICU admission (0.49, 0.46 to 0.52). In our systematic review of more than 400000 ED patients, qSOFA was also found to be a poor predictor in ED patients with or without infection [8]. Other studies with infected patients also suggest that qSOFA is not a good predictor of poor outcomes such as 28/30-day mortality or ICU admission [14,15]. However, a retrospective study on ED patients with or without infection showed that qSOFA is good predictor of mortality and ICU admission [16]. In that study, the mean age of patients was 54 years; the mortality rate was 1.6% and the ICU admission rate was 7%. This is very different from our cohort, where the median age was 72 years, with 5.7% mortality rate at 30-days and a 0.7% ICU admission rate.
Among the three scores, qSOFA has the lowest sensitivity and highest specificity to discriminate mortality in ED patients. In a retrospective study of prehospital and ED triage of infected patients, qSOFA had a lower sensitivity and higher specificity than SIRS for 48 h mortality [13]. Similar to our findings, Goulden et al. [15] demonstrated that NEWS ≥ 5 had higher sensitivity but lower specificity than qSOFA ≥ 2 for the prediction of death in infected patients. In contrast, a qSOFA score ≥2 had a 90% sensitivity and 42% specificity for in-hospital mortality in another study on infected patients [17]. In a systematic review in patients with suspected infection, the pooled sensitivity and specificity for prediction of in-hospital mortality was 51.2% (95%CI: 43.6% to 58.7%) and 79.6% (95%CI: 73.3% to 84.7%), respectively, in non-ICU patients, while the pooled sensitivity and specificity was 87.2% (95%CI, 75.8% to 93.7%) and 33.3% (95%CI, 23.8% to 44.4%) [18]. The large difference in the prognostic performance of qSOFA varies between different settings (ICU vs no-ICU) and different patient cohorts.
It is important to note that the current population for our study differs from previous studies. qSOFA was introduced to identify high-risk patients with suspected infection. Most studies have only included patients with suspected infection. However, symptoms of infection are highly variable, detection of sepsis or infection may be clinically difficult, especially in the ED. Thus, in our present study, we recruited unselected urgent and emergency patients with both infection-related and non-infection related diseases. Patients in our cohort did not include the most severely ill patients (triage category 1, critical patients), which could explain the low percentage of qSOFA positive patients (1.4% patients met qSOFA ≥ 2 in our study) when compared to other ED-based studies (16.7 to 30.6%) [4,14,16].
Standalone lactate levels on ED admission are known to be associated with higher mortality. Our findings support the concept that lactate level can predict both 7- and 30-day mortality. However, the combinations(both positive) of lactate level with qSOFA, SIRS, NEWS, qSIRS and NSIRS did not improve the prediction of 7-day and 30-day mortality in ED patients. These findings are in keeping with the suggestion from the SEPSIS-3 group that there is no additional value of adding a lactate level to qSOFA to improve the prediction of mortality.
In contrast, the incremental value of lactate levels on qSOFA has been demonstrated on ED patients with suspected sepsis [19] and undifferentiated ICU patients [20]. In parallel with 7- and 30-day mortality, increased lactate levels were associated with ICU admission, but no additional value for qSOFA, SIRS and NEWS was found with respect to the prediction of ICU admission in our study. Like the present study, previous studies support the use of lactate levels to predict adverse outcome in ED patients regardless of the presence or absence of infection [21,22].
It should be noted that although qSIRS and venous lactate provide the superior results of AUC 0.64, both required a blood test and can be difficult to perform at triage or in busy departments. NEWS on the other hand can be calculated based on physiological parameters alone and might be more easily implemented in some EDs. However, due to the modest AUC of qSIRS and NEWS, it is not clear whether the routine use of these scores could improve clinical practice and patient outcomes.
Conclusions
In conclusion, among unselected emergency and urgent ED patients, the prognostic value for NEWS and qSIRS were greater than qSOFA, Combinations of qSOFA and SIRS could improve the predictive value for 30-day mortality for ED patients. High lactate levels were found to be associated with poor outcomes, but there is no incremental value to existing scores to improve outcome predictions. Our results suggest that qSOFA is not a good predictor of poor outcomes (7/30-day mortality and ICU admission) in unselected urgent and emergency ED patients in our setting. Our findings do not support the use of qSOFA as a screening tool in the ED.
Ethics approval and consent to participate: Ethics approval was obtained from the Joint Chinese University of Hong Kong – New Territories East Cluster Clinical Research Ethics Committee (CRE-2016.236). Written informed consent was obtained from all participants.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Author contributions
CG conceived the study and designed the trial. LL, RL, CY, SC conducted the study and performed data collection. CG, LL, RL, KH managed the data, including quality control. CG and LL provided statistical advice on study design and analysed the data. LL, RL drafted the manuscript, and all authors contributed substantially to its revision. KH and CG take responsibility for the integrity of the work as a whole, from inception to published article.
References
- 1.Singer M, Deutschman CS, Seymour CW, et al. . The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016; 315(8):801–810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Levy MM, Fink MP, Marshall JC, et al. ; International Sepsis Definitions Conference . International sepsis definitions conference. 2001 SCCM/ESICM/ACCP/ATS/SIS International sepsis definitions conference. Intensive Care Med. 2003;29(4):530–538. [DOI] [PubMed] [Google Scholar]
- 3.Seymour CW, Liu VX, Iwashyna TJ, et al. . Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):762–774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Freund Y, Lemachatti N, Krastinova E, et al. ; for the French Society of Emergency Medicine Collaborators Group . Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department. JAMA. 2017;317(3):301–308. [DOI] [PubMed] [Google Scholar]
- 5.Dorsett M, Kroll M, Smith CS, et al. . qSOFA Has Poor Sensitivity for Prehospital Identification of Severe Sepsis and Septic Shock. Prehosp Emerg Care. 2017;21(4):489–489. [DOI] [PubMed] [Google Scholar]
- 6.Williams JM, Greenslade JH, McKenzie JV, et al. Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection. Chest. 2017;151(3):586–596. [DOI] [PubMed] [Google Scholar]
- 7.Green SL, Smith MTD, Cairns C, et al. . The combined SIRS + qSOFA (qSIRS) score is more accurate than qSOFA alone in predicting mortality in patients with surgical sepsis in an LMIC emergency department. World J Surg. 2020; 44(1):21–29. [DOI] [PubMed] [Google Scholar]
- 8.Lo RSL, Leung LY, Brabrand M, et al. . qSOFA is a poor predictor of short-term mortality in all patients: a systematic review of 410,000 patients. JCM. 2019; 88(1):61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.National Early Warning Score Development and Implementation Group (NEWSDIG) 2010. National Early Warning Score (NEWS): standardising the assessment of acute-illness severity in the NHS. London: Royal College of Physicians. [Google Scholar]
- 10.Wai AKC, Chor CM, Lee ATC, et al. . Analysis of trends in emergency department attendances, hospital admissions and medical staffing in a Hong Kong university hospital: 5-year study. Int J Emerg Med. 2009;2(3):141–148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Redfern OC, Smith GB, Prytherch DR, et al. . A comparison of the quick Sequential (Sepsis-Related) organ failure assessment score and the national early warning score in non-ICU patients with/without infection. Crit Care Med. 2018;46(12):1923–1933. [DOI] [PubMed] [Google Scholar]
- 12.Churpek MM, Snyder A, Han X, et al. . Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. 2017;195(7):906–911. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Tusgul S, Carron PN, Yersin B, et al. . Low sensitivity of qSOFA, SIRS criteria and sepsis definition to identify infected patients at risk of complication in the prehospital setting and at the emergency department triage. Scand J Trauma Resusc Emerg Med. 2017; 25(1):108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Askim A, Moser F, Gustad LT, et al. . Poor performance of quick-SOFA (qSOFA) score in predicting severe sepsis and mortality: a prospective study of patients admitted with infection to the emergency department. Scand J Trauma Resusc Emerg Med. 2017; 25(1):56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Goulden R, Hoyle MC, Monis J, et al. . qSOFA, SIRS and NEWS for predicting inhospital mortality and ICU admission in emergency admissions treated as sepsis. Emerg Med J. 2018; 35(6):345–349. [DOI] [PubMed] [Google Scholar]
- 16.Singer AJ, Ng J, Thode HC, Jr, et al. . Quick SOFA scores predict mortality in adult emergency department patients with and without suspected infection. Ann Emerg Med. 2017;69(4):475–479. [DOI] [PubMed] [Google Scholar]
- 17.Finkelsztein EJ, Jones DS, Ma KC, et al. . Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit. Crit Care. 2017;21(1):73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Fernando SM, Tran A, Taljaard M, et al. . Prognostic accuracy of the quick sequential organ failure assessment for mortality in patients with suspected infection: a systematic review and meta-analysis. Ann Intern Med. 2018;168(4):266–275. [DOI] [PubMed] [Google Scholar]
- 19.Shetty A, MacDonald SP, Williams JM, et al. . Lactate ≥2 mmol/L plus qSOFA improves utility over qSOFA alone in emergency department patients presenting with suspected sepsis. Emerg Med Australas. 2017;29(6):626–634. [DOI] [PubMed] [Google Scholar]
- 20.Ho KM, Lan NS.. Combining quick sequential organ failure assessment with plasma lactate concentration is comparable to standard sequential organ failure assessment score in predicting mortality of patients with and without suspected infection. J Crit Care. 2017;38:1–5. [DOI] [PubMed] [Google Scholar]
- 21.Oedorf K, Day DE, Lior Y, et al. . Serum lactate predicts adverse outcomes in emergency department patients with and without infection. WestJEM. 2017;18(2):258–266.westjem. 2016.10.31397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Park YJ, Kim DH, Kim SC, et al. . Serum lactate upon emergency department arrival as a predictor of 30-day in-hospital mortality in an unselected population. PLoS One. 2018;13(1):e0190519. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.