Abstract
Objective
This study aimed to determine the prognostic accuracy of SOFA in comparison to quick-SOFA (qSOFA) and systemic inflammatory response syndrome (SIRS) in predicting 28-day mortality in the emergency department (ED) patients with infections.
Methods
A secondary analysis of data from a prospective study of adult patients with documented or suspected infections admitted to an ED in Denmark from Oct-2017 to Mar-2018. The SOFA scores were calculated after adjustment for chronic diseases. The prognostic accuracy was assessed by analysis of sensitivity, specificity, predictive values, likelihood ratios, and area under the receiver operating characteristic curve (AUROC) with 95% confidence intervals (CI).
Results
A total of 2045 patients with a median age of 73.2 (IQR: 60.9–82.1) years were included. The overall 28-day mortality was 7.7%. In patients meeting a SOFA score ≥2, qSOFA score ≥2, and SIRS criteria ≥2 the 28-day mortality was 13.6% (11.2–16.3), 17.8% (12.4–24.3) and 8.3% (6.7–10.2), respectively. SOFA ≥2 had a sensitivity of 61.4% (53.3–69.0) and specificity of 67.3% (65.1–69.4), qSOFA ≥2 had a sensitivity of 19.6% (13.7–26.7) and specificity of 92.4% (91.1–93.6), and SIRS ≥2 had a sensitivity of 52.5% (44.4–60.5) and specificity of 51.5% (49.2–53.7). The AUROC for SOFA compared to SIRS was: 0.68 vs 0.52; p<0.001 and compared to qSOFA: 0.68 vs 0.63; p=0.018.
Conclusion
A SOFA score of at least two had better prognostic accuracy for 28-day mortality than SIRS and qSOFA. However, the overall accuracy of SOFA was poor for the prediction of 28-day mortality.
Keywords: emergency department, infectious disease, SOFA, SIRS, qSOFA, sepsis, mortality, prognostic accuracy
Background
Sepsis is a serious clinical condition with high mortality.1,2 The incidence of sepsis has been reported to 83–731/100,000 person-years at risk depending on the definition and severity of sepsis.1,3,4
Since 1992, Systematic Inflammatory Response Syndrome (SIRS) (Appendix-1) criteria have been used to early identification of infected patients with sepsis. Due to the low predictive validity of SIRS5 and a better understanding of the pathobiological mechanism, sepsis has since 2016 (Sepsis-3) been defined as a life-threatening organ dysfunction caused by a dysregulated host response to infection.5 Organ dysfunction is defined as an increase of two points or more from the baseline of the Sequential Organ Failure Assessment (SOFA) score (Appendix 1).5 Since SOFA requires laboratory tests with the risk of delaying the identification of patients with sepsis, a simple bedside test, quick-SOFA (qSOFA)5 (Appendix 1) was proposed as a clinical tool for early identification of infected patients with increased risk of serious outcomes.
However, data from our institution6 and other studies7,8 have reported that qSOFA’s prognostic accuracy is not substantially better than SIRS, and qSOFA is nowadays considered a prognostic tool rather than a diagnostic tool.6,9,10 Thus, we still lack a method for early identification of infectious patients with a sepsis-like condition and an increased risk of a poor outcome.
The use of SOFA may be unfeasible. Clinicians often have to await laboratory results (appendix 1), and comorbidities (lung, kidney, liver diseases, and dementia) may influence the SOFA calculations, which may delay the identification of septic patients and treatment with antibiotics. With these considerations, SOFA seems, after all, to have superior prognostic accuracy.9,11–13 However, prospective evaluations of SOFA’s prognostic performance in patients with infections compared to other tools in the emergency department (ED) settings are scarce.9,11,14 Therefore, the present study aimed to compare the prognostic accuracy of SOFA compared to qSOFA and the SIRS criteria for predicting 28-day mortality in a study of patients with documented or suspected infection admitted to an ED. A secondary purpose was to estimate the incidence of sepsis based on the new Sepsis-3 criteria.
Method
The present study is a secondary analysis of data from a previously published prospective observational cohort study of the prognostic accuracy of qSOFA among adult (≥ 18 years) ED patients with an infection.6 All patients were admitted to the ED at Slagelse Hospital, Denmark, during the period October 1, 2017, to March 31, 2018.6 Slagelse Hospital is a tertiary care center that serves the entire population in the catchment area, and the ED has an uptake area of 198,000 adult inhabitants with 26,500 visits annually. The Danish healthcare system offers equal access to all residents. Patients who are admitted acutely are either referred by general practitioners or by ambulance to the hospital. Danish privately funded hospitals have no acute patient intake.15
Patients with Infections
During the study period, triage records and electronic patient records for all patients (N= 12,092) were every working-day reviewed by the authors (SMOBA, RHS) for inclusion. As previously described,6 all patients with either documented or suspected infection were diagnosed by an ED physician in charge. If delivered either intravenous or peroral antibiotics (AB) within 24 hours from arrival (N=3176), the patients were initially registered as patients with infectious diseases.
Patients who had a qSOFA score of at least two or fulfilled the SIRS criteria for sepsis, or were critically ill, were given priority for prompt medical examination by an ED physician. Septic patients were treated according to a protocol during the study period. If a physician suspected sepsis, treatment with oxygen, lactate measurement, blood tests, an electrocardiogram (ECG), treatment with intravenous fluids, and intravenous broad-spectrum AB after blood cultures were drawn was recommended initiated within one hour from recognition of sepsis irrespective of disease severity.6
Inclusion and Exclusion
Patients treated with antibiotics within 24 hours after ED admission and continued for at least 48 hours were considered to have an infection and were included in the present study. Among the 3176 patients initially registered as having an infection on admission patients with foreign nationality without a Danish Civil Registration number (n=6), if AB treatment was discontinued within 48 hours (n=174), prophylactic AB treatment with surgery (n=240), transfer to or from other hospitals (n=224), if previously included during the study period (n=354), missing data (n=123) and registration errors (n=10) were excluded.
Definitions
We used the Charlson Comorbidity Index (CCI) to classify the burden of chronic disorders.16
Patients who fulfilled a SOFA score of at least two on admission were considered to have sepsis.5 A baseline SOFA score of zero was registered for all patients without chronic disorders. An adjusted baseline score of SOFA was calculated for patients with known chronic diseases (kidney, liver, respiratory, or dementia). SOFA scoring was not a routine in our ED during the study period. The SOFA calculation method has been described in detail in Appendix 2. The definitions of qSOFA and SIRS followed the guidelines described in Appendix 1.
The altered mental state was defined as either a Glasgow Coma Scale (GCS) value less than 15 or AVPU (Alert, Verbal, Pain, Unresponsive) other than A on admission to the ED. Positive blood culture was defined as at least one positive blood culture during the stay in the ED (coagulase-negative staphylococci excluded).
Data Collection
All patients admitted to our ED were triaged immediately after the arrival. We obtained the following triage variable, laboratory test and medical history from the triage forms and the electronic medical record daily by the authors (SMOBA, RHS): Glasgow Coma Scale (GCS) or the Alert-Voice-Pain-Unresponsive (AVPU) scale, systolic blood pressure (SBP)(mm Hg), respiratory rate (RR)(breaths/min), heart rate (HR)(beats/min), body temperature (Tp)(°C), haemoglobin (mmol/L), C-reactive protein (CRP), leucocyte count (×109/L), paO2´(kPa), peripheral oxygen saturation (%), creatinine (μmol/L), bilirubin (μmol/L), platelets count (×109/L) and information regarding comorbidities (CCI).
Information on site of infection was based on a review of all electronic records at discharge with specific details on infectious sources diagnosed and documented in the records by the physicians during hospital stay. Foci of infection were specified by bacterial culturing of suspected infected tissues and body fluids. Other examinations (eg x-ray, ultrasound, computed tomography, gynecological examinations) were performed if relevant.
qSOFA and SIRS scores were calculated (appendix 1) based on the arrival variables. Survival status was obtained from the Danish Civil Registration System,17 a national registry with information on vital status among all Danish citizens. Each person has a unique personal registration number (CPR) in Denmark, making the linkage between health registries possible.17
Statistical Analysis
The primary outcome was 28-day mortality. In-hospital mortality and transfer to the ICU were secondary outcomes. Continuous data are reported as medians with interquartile ranges (IQR). Categorical variables are reported as counts and percentages with 95% confidence intervals (CI). Differences within medians with 95% CI and exact differences of proportions with 95% CI were used to compare the groups. Differences were assumed significant if the 95% CI for the median difference or the 95% CI for the difference of proportions did not include zero. We have calculated the incidence as (number of patients with sepsis defined by either SOFA, qSOFA, or SIRS/adult population in the area x 0.5) x 100.000 person-years at risk (95% CI). Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive- and negative likelihood ratio (LR+/LR-) with 95% CI and area under the receiver operating characteristic curve (AUROC) was used to assess the prognostic accuracy of the scores. The equality of the AUROC was reported by p-value. AUROC values at 0.5 suggest no ability to discriminate between patients with or without the outcome of interest, 0.6 to 0.7 was considered poor, adequate at 0.7 to 0.8, good at 0.8–0.9 and excellent at 0.9 or higher.12,18
Mortality rates among patients fulfilling the qSOFA score and SIRS criteria, and the prognostic accuracy of the scores, have previously been published6 but are included in the present analyses for comparison purposes.
All statistical analyses were performed using STATA 15.1.
Results
Study Population
A total of 2045 patients (48.6% male) with a median age of 73.2 years (IQR 60.9–82.1) were included (Figure 1).
Figure 1.
Study flow chart.
Baseline Characteristics
The baseline characteristics, according to the sepsis criteria (SOFA ≥2), are shown in Table 1. A total of 61 (3.0%) of the patients had systolic blood pressure ≤ 90 mmHg on admission, and 20 (1%) were treated with vasopressors (Table 1). The proportion of hypotensive (6.9% vs 0.9%) and vasopressor treated patients (2.2% vs 0.3%) was highest among sepsis patients (Table 1)
Table 1.
Baseline Characteristics According to Sepsis Classification
| All Patient N=2045 | Sepsis (SOFA ≥2) n=714 | No Sepsis (SOFA <2) n=1331 | Difference (95% CI) Between Sepsis and No Sepsis | |
|---|---|---|---|---|
| Male sex, n(%) | 993(48.6) | 417(58.4) | 576(43.3) | 15.1(10.6–19.6) |
| Age in years, median, (IQR) | 73.2(60.7–82.7) | 75.1(66.2–84.1) | 72.2(57.1–81.4) | 4.4(3.1–5.8) |
| Chronic disorders | ||||
| Charlson Comorbidity Index, n(%) | ||||
| 0 | 630(30.8) | 175(24.5) | 455(34.2) | 9.7(5.6–13.8) |
| 1–2 | 969(47.4) | 350(49.0) | 619(46.5) | 2.5(−2.0–7.0) |
| 3+ | 446(21.8) | 189(26.5) | 257(19.3) | 7.2(3.3–11.0) |
| Congestive heart failure | 222(10.9) | 110(15.4) | 112(8.4) | 7.0(4.0–10.0) |
| Ischemic heart diseasea | 225(11.0) | 95(13.3) | 130(9.8) | 3.5(0.5–6.5) |
| Cerebrovascular disease | 285(13.9) | 128(17.9) | 157(11.8) | 6.1(2.8–9.4) |
| Chronic pulmonary disease | 540(26.4) | 174(24.4) | 366(27.5) | 3.1(−0.9–7.1 |
| Diabetes mellitusb | 344(16.8) | 152(21.3) | 192(14.4) | 6.9(3.4–10.5) |
| Malignancyc | 275(13.5) | 100(14.0) | 175(13.2) | 0.8(−2.3–3.9) |
| Chronic kidney diseased | 116(5.7) | 61(8.5) | 55(4.1) | 4.4(2.1–6.7) |
| Chronic mild or severe liver disease | 36(1.8) | 14(2.0) | 22(1.7) | 0.3(−0.9–1.5) |
| Hypertension | 672(32.9) | 260(36.4) | 412(31.0) | 5.4(1.1–9.7) |
| Altered mental status, n(%) | 285(13.9) | 196(27.5) | 89(6.7) | 20.8(17.2–24.3) |
| qSOFA ≥2, n(%) | 174(8.5) | 140(19.6) | 34(2.6) | 17.0(14.0–20.0) |
| SIRS ≥2, n(%) | 999(48.9) | 425(59.5) | 574(43.1) | 16.4(11.9–20.9) |
| SBP ≤ 90 mmHg, (%) | 61(3.0) | 49(6.9%) | 12(0.9) | 6.0(4.1–7.9) |
| Treatment with vasopressor, n(%) | 20(1.0) | 16(2.2) | 4(0.3) | 1.9(0.8–3.0) |
| Laboratory results, median (IQR) | ||||
| Haemoglobin (mmol/L) | 8.1(7.2–8.8) | 8.0(7.0–8.9) | 8.1(7.3–8.8) | 0.1(0–0.2) |
| C-reactive protein (mg/L) | 65(20–137) | 78(23–148) | 61(18–130) | 9.0(3.6–14.5) |
| White blood cell count (x 109/L) | 11.1(8.5 −14.9) | 11.8 (8.6–16.4) | 10.8 (8.4–14.1) | 0.9(0.4–1.3) |
| Creatinine (µmol/L) | 82 (63 −113) | 114 (76–175) | 74 (60–93) | 36(31–42) |
| Platelet count (x 109/L) | 241 (187–315) | 213.5 (148–290.5) | 255 (206–323) | 46(37–55) |
| Bilirubin (µmol/L) | 9 (6–13) | 10 (7–17) | 8 (6–11) | 2(2–3) |
| Lactatee (mmol/L) | 1.2 (0.8–1.9) | 1.3 (0.9–2.1) | 1.0 (0.7–1.7) | 0.2(0.1–0.3) |
| Missing lactate values, n(%) | 1273(62.2%) | 301(42.2%) | 972(73.0%) | 30.8(26.5–35.1) |
| Length of stay, median (IQR) | 6.2(2.2–7.9) | 6.1(3.7–10.1) | 3.9(1.3–6.8) | 2.2(1.9–2.7) |
| Blood cultures, n(%) | ||||
| Number blood cultures obtained | 1101(53.8) | 438(61.3) | 663(49.8) | 11.5(7.0–16.0) |
| Positive blood cultures | 123(6.0) | 60(8.4) | 63(4.7) | 3.7(1.4–6.0) |
| Foci of infection, n(%)f | ||||
| Lungs | 1086(53.1) | 423(59.2) | 663(49.8) | 9.4(4.9–13.9) |
| Urinary | 525(25.7) | 157(22.0) | 368(27.7) | 5.7(1.8–9.6) |
| Abdominal | 217(10.6) | 71(9.9) | 146(11.0) | 1.1(−1.7–3.9) |
| Skin | 182(8.9) | 48(6.7) | 134(10.1) | 3.4(1.0–5.8) |
| Endocarditis | 9(0.4) | 3(0.4) | 6(0.5) | 0.1(−0.5–0.7) |
| Central nervous system | 9(0.4) | 5(0.7) | 4(0.3) | 0.4(−0.3–1.1) |
| Devices/implants | 2(0.1) | 1(0.1) | 1(0.08) | - |
| Facial/teeth | 11(0.5) | 2(0.3) | 9(0.7) | 0.4(−0.2–1.0) |
| Others | 9(0.4) | 3(0.4) | 6(0.5) | 0.1(−0.5–0.7) |
| Unknown | 139(6.8) | 60(8.4) | 79(5.9) | 2.5(0.1–4.9) |
| 2 infection foci | 135(6.6) | 54(7.6) | 81(6.1) | 1.5(−0.8–3.8) |
| 3+ infection foci | 5(0.2) | 3(0.4) | 2(0.2) | 0.2(−0.3–0.7) |
Notes: aA history with either myocardial infarction or other symptoms of ischemic heart disease. bDiabetes with or without organ damage. cMetastatic tumor or tumor without metastasis, leukemia and lymphoma. dSevere or mild chronic kidney disease. eA total of 772 (37.8%) patients had blood measurements of lactate on admission to the ED. fSome patients had more than one site of infection.
Abbreviations: CI, confidence interval; qSOFA, quick sequential organ failure assessment; SBP, systolic blood pressure; SOFA, sequential organ failure assessment; SIRS, systemic inflammatory response syndrome.
Sepsis patients were older, more often male, and had higher comorbidity scores (Table 1). The proportion of patients with congestive heart failure, ischemic heart disease, cerebrovascular disease, diabetes, chronic kidney disease, and hypertension was higher among patients with sepsis (Table 1). C-reactive protein, leucocyte count, creatinine, bilirubin, and lactate values were higher among septic patients while platelets were lower (Table 1). Blood cultures were more often obtained among septic patients, and positive blood cultures were more common in patients with sepsis (Table 1). Pulmonary origin of infection was more frequent in septic patients, while skin and urinary infections were more common in non-sepsis patients (Table 1). A total of 139 (6.8%) had unknown sites of infections (Table 1).
Incidence of Sepsis
A total of 714 (34.9%;32.8–37.0) patients had a SOFA score of ≥2 on admission, 174 (8.5%;7.3–9.8%) had a qSOFA score of ≥2, and 999 (48.9%; 46.6–51.0%) fulfilled at least two SIRS criteria (Table 2). The incidence of sepsis based on SOFA was 721 (663–768) per 100,000 person-years, based on qSOFA 176 (149–202) per 100,000 person-years, and based on SIRS criteria 1009 (938–1062) per 100,000 person-years (Table 2).
Table 2.
Incidence, Transfer to ICU and Mortality According to Different Sepsis Criteria
| n | Incidence/Person- Years at Risk (95% CI) | Transfer to ICU % (95% CI) | In-Hospital Mortality % (95% CI) | 28-Day Mortality % (95% CI) | |
|---|---|---|---|---|---|
| All patients | 2045 | 7.5 (6.4–8.7) | 3.8 (3.0–4.7) | 7.7 (6.6–9.0) | |
| SOFA ≥2 | 714 | 721/100,000 (663–768/100,000) | 13.5 (11.0–16.2) | 7.3 (5.5–9.4) | 13.6 (11.2–16.3) |
| qSOFA ≥2 | 174 | 176/100,000 (149–202/100,000) | 16.1 (11.0–22.4) | 9.2 (5.3–14.5) | 17.8 (12.4–24.3) |
| SIRS ≥ 2 | 999 | 1009/100,000 (938–1062/100,000) | 10.0 (8.2–12.0) | 4.2 (3.0–5.6) | 8.3 (6.7–10.2) |
Abbreviations: CI, confidence interval; ICU, Intensive care unit.
Mortality
A total of 158 (7.7%) patients died within 28 days, and the mortality in patients with a SOFA score ≥2, qSOFA ≥2, and SIRS ≥2 was 13.6% (11.2−16.3), 17.8% (12.4− 24.3), and 8.3% (6.7–10.2), respectively. The 28-day mortality increased with increasing SOFA scores (Table 3) and was significantly higher (13.6% vs 4.5%; 6.3–11.8%) in patients with sepsis (SOFA ≥ 2) compared to patients without sepsis (SOFA ≤ 2) (Table 2).
Table 3.
28-Day Mortality According to the SOFA Scores
| SOFA Score | N(%) | Mortality, n(%; 95% CI) |
|---|---|---|
| 0 | 834(40.8) | 33(4.0;2.7–5.5) |
| 1 | 497(24.3) | 28(5.6;3.7–8.0) |
| 2 | 337(16.5) | 30(8.9;6.1–12.5) |
| 3 | 183(9.0) | 22(12.0;7.7–17.6) |
| 4 | 100(4.9) | 13(13.0;7.1–21.2) |
| 5 | 55(2.7) | 20(36.4;23.8–50.4) |
| 6 | 21(1.0) | 4(19.0;5.4–41.9) |
| 7 | 7(0.3) | 3(42.9;9.9–81.6) |
| 8 | 6(0.3) | 3(50.0;11.8–88.2) |
| 9 | 4(0.2) | 2(50.0;11.8–88.2) |
| 10 | 1(0.05) | 0 |
Abbreviations: CI, confidence interval; SOFA, sequential organ failure assessment.
The 28-day mortality was highest among patients meeting a SOFA score ≥2 compared to SIRS ≥2 (13.6 vs 8.3%; 2.3−8.3%) on admission (Table 2). There was no difference in 28-day mortality between patients with SOFA ≥2 and qSOFA ≥ 2 (13.6 vs 17.8%; −2.0 − 10.4%) (Table 2).
The in-hospital mortality was also higher among patients with SOFA ≥2 than SIRS ≥2, (7.3% vs 4.2%; 0.8−5.4%) (Table 2). However, there was no difference in in-hospital mortality between patients with SOFA ≥2 and qSOFA ≥2 (7.3% vs 9.2%; −6.5 − 2.8) (Table 2).
Transfer to ICU
The proportion of patients transferred to the ICU was higher among patients with a SOFA score ≥2 than patients with SIRS ≥2 (13.5 vs 10.0%; 0.3−6.6%) (Table 2). However, there was no difference between patients with a SOFA score ≥2 and qSOFA ≥2 (13.5 vs 16.1%; -8.6−3.4%) (Table 2).
Prognostic Accuracy for 28-Day Mortality
Patients with SOFA ≥2 had a sensitivity of 61.4% (53.3−69.0) and a specificity of 67.3% (65.1−69.4) (Table 4). Patients meeting qSOFA ≥2 had a sensitivity of 19.6% (13.7−26.7) and a specificity of 92.4% (91.1−93.6) (Table 4). Patients with SIRS ≥2 had a sensitivity of 52.5% (44.4−60.5) and a specificity of 51.5% (49.2−53.7) (Table 4). Overall AUROC for SOFA was significantly higher compared to qSOFA (0.68 vs 0.63; P=0.018) and SIRS (0.68 vs 0.52; p<0.001) (Table 4). Likelihood ratios are close to 1.0 (for both LR+ and LR-) for SOFA and qSOFA (Table 4). PPV and NPV for the scores are shown in Table 4. The AUROC curves and corresponding AUROC values showed that SOFA and qSOFA had poor prognostic ability to predict 28-day mortality (Figure 2). SIRS could not predict mortality (Figure 2).
Table 4.
Prognostic Accuracy of SOFA, qSOFA, and SIRS for 28-Day Mortality
| SOFA ≥2 | qSOFA ≥2 | SIRS ≥2 | |
|---|---|---|---|
| Sensitivity (95% CI) | 61.4 (53.3–69.0) | 19.6 (13.7–26.7) | 52.5 (44.4–60.5) |
| Specificity (95% CI) | 67.3 (65.1–69.4) | 92.4 (91.1–93.6) | 51.5 (49.2–53.7) |
| PPV (95% CI) | 13.6 (11.2–16.3) | 17.8 (12.4–24.3) | 8.3 (6.7–10.2) |
| NPV (95% CI) | 95.4 (94.2–96.5) | 93.2 (92.0–94.3) | 92.8 (91.1–94.3) |
| Likelihood ratio (+) | 1.9 (1.6–2.2) | 2.6 (1.8–3.7) | 1.1 (0.93–1.3) |
| Likelihood ratio (-) | 0.6 (0.5–0.7) | 0.9 (0.8–0.9) | 0.9 (0.8–1.1) |
| SOFA | qSOFA | SIRS | |
| AUROC (95% CI) | 0.68 (0.64–0.73) | 0.63 (0.58–0.67) | 0.52 (0.47–0.56) |
Abbreviations: AUROC, area under the receiver operating characteristic curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.
Figure 2.
The area under the receiver operating characteristics (AUROC) curves.
Abbreviations: qSOFA, quick sequential organ failure assessment; ROC, receiver operating curve; SIRS, Systemic inflammatory response syndrome; SOFA, sequential organ failure assessment
Supplementary Analyses
We found no significant changes for AUROC values according to the three scores after excluding patients with systolic blood pressure ≤ 90 mmHg or need of vasopressor treatment (Supplement Table 1).
Prognostic Accuracy for in-Hospital Mortality and Transfer to ICU
The sensitivity for SOFA ≥2, qSOFA ≥2, SIRS ≥2 in predicting in-hospital mortality was 67.5% (55.9−77.8), 20.8% (12.4−31.5), 54.5% (42.8−65.9), respectively. The specificity was 67.5 (55.9−77.8), 92.0% (90.7–93.1), 51.4% (49.1−53.6), respectively. Analyses applying ICU-admission as endpoint did not show any significant differences across the scores (data not shown).
Patients without Chronic Disorders
In a sensitivity analysis excluding patients with chronic disorders and leaving patients with a baseline SOFA score of zero, we found a sensitivity of 64.5% (51.3−76.3), a specificity of 70.7% (67.9−73.4), and AUROC of 0.70 (0.63–0.77) for SOFA, which is not notable deviating values compared to the total group of patients (Table 4). The sensitivity analyses were repeated for qSOFA and SIRS without significant new findings (data not shown).
Discussion
In this study, we found that a SOFA score of at least two in patients with infectious diseases on admission to an ED had better prognostic accuracy than SIRS and qSOFA to predict 28-day mortality. However, the overall accuracy of SOFA was poor for the prediction of 28-day mortality. Our study also revealed that sepsis incidence based on the SOFA criteria was 721/100,000 person-years, which was considerably higher compared to estimates based on qSOFA, and lower compared to the SIRS criteria.
The Sepsis-3 guidelines were based on a retrospective study of both ICU and non-ICU patients.12 Since the introduction of SOFA in 2016, only one prospective study of SOFA’s prognostic accuracy in ED patients has been published.11 Freund et al11 found in a multicentre study of 879 ED patients with infectious diseases that SOFA outperformed SIRS; however, SOFA was not better than qSOFA. The sensitivity and specificity of SOFA for predicting in-hospital mortality were 73% and 70%, respectively.11 The AUROC values for SOFA, qSOFA, and SIRS for in-hospital mortality were 0.77, 0.80, and 0.65, respectively.11 The higher values of sensitivity, specificity, and AUROC compared to our study can be explained by differences in the SOFA measurement. We have used clinical and laboratory values on arrival to calculate the SOFA score. In contrast, Freund et al used the worst values during the ED stay, explaining the different performances of the scores. However, in our study, we have taken chronic disorders into account and examined the validity of the SOFA scores as a secondary data analysis of prospectively collected data. Freund et al did not describe the method behind the SOFA calculations.
A retrospective study by Boillat-Blanco et al of 519 infected patients admitted to an ED in Dar es Salaam concluded that SOFA had a prognostic accuracy for 28-day mortality similar to qSOFA but superior to SIRS.19 The study reported AUROC values for 28-day mortality for SOFA, qSOFA, and SIRS of 0.79, 0.80, and 0.61, respectively, which were also higher than our study. The higher discriminatory ability of SOFA and qSOFA was probably due to the high prevalence of various infectious diseases (Malaria, Dengue, HIV, etc.) among febrile adult patients.19
A screening tool for sepsis requires adequate sensitivity and specificity to identify sepsis patients. However, the study by Seymour et al12, which was the basis for the Sepsis-3 definition of sepsis, did not report sensitivity or specificity for SOFA. The sensitivity and specificity of SOFA score were higher in our study in comparison to SIRS. However, it was slightly lower than prior studies conducted among infected patients in the ED.11,19 Furthermore, we found that the sensitivity of the SOFA was substantially higher than qSOFA.
Few studies found that the prognostic accuracy (measured by the AUROC) of qSOFA was better than SOFA for non-ICU patients.11,19 However, recent prospective studies6–8 and a meta-analysis20 of qSOFA concluded that qSOFA cannot stand alone as a tool to identify infectious patients with a high risk of serious outcomes due to its low sensitivity.
AUROC for the SOFA was in our study lower than in prior studies (Table 4).9,11–14,19,21–25 The different findings can be explained by differences in methodology. Several studies9,11,12,22,23 have used the worst values of variables from the ED stay to calculate SOFA. Many studies were undertaken among patients with critical clinical conditions and admitted to the ICU21,22 or among mixed cohorts of ICU and non-ICU.13,14 Further on, the definition of infection varied across the studies.9,11–14,19,21–24 All these studies reported better prognostic accuracy of SOFA than SIRS, and seven out of ten studies reported slightly better prognostic performance of SOFA than qSOFA.
The apparent poor performance of the tools to identify patients with sepsis and the use of prognosticators rather than specific diagnostic tools calls for the development of other and better methods for early identification of sepsis patients. Two meta-analyses examining the role of more than 200 hundreds biomarkers have shown that the precise role of biomarkers in the management of sepsis patients has not been well-defined.26,27 The methodology used in the studies are not adequate, and most of the studies are based on old sepsis definition and are performed in different settings.26,27 The biomarkers with the highest AUROC’s were described in studies with limited sample sizes.27 Both meta-analyses call for new studies with a standardized methodology to evaluate the biomarker’s ability to identify septic patients.26,27 With rapid technological advancement within the point-of-care diagnostic tests of different biomarkers, it is desirable to look into their usefulness together with different scores of sepsis and, for example, combining biomarkers as Interleukin-6 (IL-6) and procalcitonin (PCT) with SOFA or qSOFA. IL-6 nor PCT cannot identify patients with sepsis alone; however, the combination of these with clinical criteria may improve their prognostic accuracy.26,28 There are also ongoing promising research initiatives in developing host-based molecular assays to detect sepsis at an early stage of the disease.29
The incidence of sepsis in our study was lower when the new sepsis-3 (SOFA) criteria were used compared to the previously used SIRS criteria, which is consistent with another large retrospective study conducted by Donnelly et al among non-ICU patients.23 Donnelly et al reported the sepsis incidence according to SOFA and SIRS criteria to 580- and 820 pr. 100.000 person-years at risk, respectively.23
Strength and Limitation
The main strength of this study was its prospective design of data collection. Although the SOFA calculations were performed as secondary analyses, the analyses were based on prospective data collection. The study was designed to identify all infected patients based on well-defined infection criteria and consecutively admitted to our ED. A complete follow-up was ensured by the CPR number of all the included patients, which reduced the risk of selection bias. Furthermore, we assume no referral bias as all acute patients from the uptake area were referred to our ED.
The study has potential limitations. First, the median age in our cohort seems higher compared to other studies (Table 5), which should be taken into account when comparing mortality rates.
Table 5.
Prognostic Performance of SOFA, qSOFA and SIRS Criteria
| Author | Setting Design | Sample Size | Age Years | Follow-Up Period for Mortality | Overall Mortality Rate (%) | Method of Sepsis Score Calculations | Definition of Infection | SOFA≥2 SE SP AUROC | qSOFA≥2 SE SP AUROC | SIRS≥2 SE SP AUROC |
|---|---|---|---|---|---|---|---|---|---|---|
| Abdullah Present study 2021 Denmark |
ED P |
2045 | 73a | 28-day | 8 | ED arrival. SOFA adjusted for chronic diseases |
Patients treated with antibiotics (AB) within 24 hours after ED admission and continued for at least 48 hours | 61 67 0.68 |
20 92 0.63 |
53 52 0.52 |
| Freund 201711 Europe |
ED P |
879 | 67a | IH | 8 | Worst values during ED stay |
Infection diagnosed by the on-duty physician |
73 70 0.77 |
70 79 0.80 |
93 27 0.65 |
| Seymour 201612 Australia |
Non-ICU R |
66,522 | 60b | IH | 3 | Worst values during ED stay |
Combination of AB treatment and cultures obtained | NR NR 0.79 |
NR NR 0.81 |
NR NR 0.76 |
| Raith 201721 New Zealand |
ICU R |
184,875 | 63b | IH | 19 | On ICU admission. SOFA calculation was adjusted for chronic diseases | Infection-related ICU-admission diagnosed according to Australian and New Zealand Intensive Care Society guidelines31 | NR NR 0.75 |
NR NR 0.61 |
NR NR 0.59 |
| April 201722 USA |
ICU R |
214 | 68b | IH | 18 | Worst values during ED stay | Administration of AB during ED stay and cultures obtained within 24h | NR NR 0.75 |
89.7 27.4 0.66 |
97.4 2.3 0.65 |
| Donnelly 201723 USA |
Non-ICU R |
2593 | 69c | IH | 6 | Worst values during the first 28 hours of admission | The criteria of Angus et al32 | NR NR 0.77 |
NR NR 0.76 |
NR NR 0.72 |
| Gaini 201824 Denmark |
HW R |
323 | 66a | IH | 6 | Not specified | Suspicion of infection by the on-duty physician | 100 44 0.83 |
38 89 0.67 |
81 28 0.61 |
| Blanco 201819 Tanzania |
ED R |
519 | 30a | 28-day | 6 | Not specified | Infections criteria derived from World Health Organization, Infectious Diseases Society of America guidelines and European Society of Clinical Microbiology and Infectious Diseases guidelines33 | 72 73 0.79 |
59 88 0.80 |
91 19 0.61 |
| Kovach 201913 USA |
ICU + Non-ICU R |
10,981 | 52a | IH | 7 | Not specified | Combination of intravenous AB treatment and cultures obtained. | NR NR 0.90 |
NR NR 0.84 |
NR NR 0.79 |
| Poutsiaka 201914 USA |
ICU + ED P |
176 | 61a | 30-day | 15 | Not specified | Two physicians from the department of infectious diseases determined infection | NR NR 0.84 |
NR NR 0.69 |
NR NR 0.54 |
| Machado 20209 Brazil |
Non-ICU P |
5460 | 64a | IH | 14 | Worst values from arrival until sepsis was suspected. For HW patients 24 hours prior to suspicion of sepsis. | Treatment with AB after blood cultures obtained and diagnosis determined by on-duty physician | 88 60 0.83 |
54 84 0.75 |
76 25 0.53 |
| Lopez 202025 Spain |
Selective cohort ED P |
870 | 70a | 2-day | 8 | Arrival variables or worst values during stay. | Not specified | 88 81 0.91 |
85 69 0.81 |
NR NR NR |
| Falsetti 202034 Italy |
ICU + Non-ICU R |
390 | 80b | IH | 37 | Not specified | Not specified | NR NR 0.69 |
NR NR 0.68 |
NR NR NR |
| Umemura 201735 Japan |
ICU R |
387 | 74a | IH | 28 | Not specified | Not specified | NR NR 0.70 |
91 19 0.62 |
NR NR 0.53 |
Notes: aMedian; bMean; cMean age for patients fulfilling SOFA ≥2.
Abbreviations: AB, antibiotic; AUROC, area under the receiver operating characteristic curve; CI, confidence interval; ED, emergency department; ICU, intensive care unit; IH, in-hospital; HW, hospital wards; NR, not reported; P, prospective; R, retrospective; SE, sensitive; SP, specificity; SIRS, systemic inflammatory response syndrome, SOFA, sequential organ failure assessment; qSOFA, quick sequential organ failure assessment.
Second, different definitions of infection should be considered when comparing prognostic performances of the scores across the studies. Other definitions used are suspicion of infection by the on-duty physician,11,14,24 infection criteria derived from national Infectious Diseases Societies19,21 and a combination of AB and culture obtained12,22,30 (Table 5). Third, the method used to adjust the SOFA score for chronic diseases with potential impact on the baseline SOFA value was not protocolized before the study start. The SOFA calculations were performed as secondary analyses, and the information on chronic diseases was based on diseases that were included in the CCI classification. We did not have precise information on the severity of chronic diseases before admission to ED, and our method used to adjust the SOFA values has not been validated. Therefore, we cannot exclude the risk of misclassification of septic patients based on the SOFA score. However, we have reanalyzed the prognostic accuracy of SOFA after excluding patients with chronic diseases with a potential impact on the baseline SOFA value and did not find any important differences. Furthermore, we only used admission variables to calculate the SOFA score. Serial SOFA measurements during the first hours of admission may have detected some patients with clinical deterioration and fulfilling the sepsis criteria during the initial ED stay. Fourth, sepsis treatment was primarily guided by a sepsis treatment protocol for patients fulfilling the qSOFA≥2 criteria, SIRS≥2 criteria, or if the patients were critically ill. According to the protocol, some of the septic patients identified by the SOFA criteria in our study did not fulfill the qSOFA and SIRS criteria, and sepsis treatment according to the protocol may therefore have been opted-out. The extent to which this may have affected the outcomes in patients only fulfilling the SOFA criteria is difficult to explore due to the observational study design and risk of bias. Fifth, the accuracy of our estimates could have been increased with larger sample size. Finally, the study was undertaken at a single Danish center and may not be generalizable to other centers in different countries.
Conclusion
A SOFA score of at least two among infectious patients admitted to an ED had better prognostic accuracy than SIRS and qSOFA in predicting 28-day mortality. However, the overall results of the scores indicate that SOFA is not an ideal sepsis screening tool and cannot stand alone in the identification of infected patients with the risk of serious outcomes. The incidence of sepsis depends on the definition used, and the findings from our and other studies indicate a need to rewrite parts of the epidemiology of sepsis.
Acknowledgments
Thanks to Dr. Lothar Wiese for his valuable and critical comments on the manuscript. Results from the study has been presented at the 13th European Emergency Medicine Conference 2019, Prague, Czech Republic.
Funding Statement
This work was supported by Region Zealand Health Research Foundation, Naestved, Slagelse and Ringsted Hospitals Research Fund, and the Department of Emergency Medicine, Copenhagen University Hospital, Bispebjerg and Frederiksberg, Denmark.
Abbreviations
AB, Antibiotic; AUROC, Area under the receiver operating characteristic curve; AVPU, Alert-Voice-Pain-Unresponsive; CCI, Charlson Comorbidity Index; CI, Confidence intervals; CPR, Personal registration number; CRP, C-reactive protein; ED, Emergency department; GCS, Glasgow Coma Scale; HR, Heart rate; IQR, Interquartile range; IL-6, Interleukin-6; LR-, Negative likelihood ratio; LR+, Positive likelihood ratio; NPV, Negative predictive value; PPV, Positive predictive value; PCT, Procalcitonin; qSOFA, Quick Sequential Organ Failure Assessment; RR, Respiratory rate; SBP, Systolic blood pressure; SIRS, Systemic Inflammatory Response Syndrome; SOFA, Sequential Organ Failure Assessment; Tp, Body temperature.
Ethics
This study was approved by the Danish Data Protection Agency (Project REG-105-2017). The study was the 16 May 2017 defined as a quality project by the Secretariat of The Committee on Health Research Ethics of Region Zealand, which acts as an Institutional Review Board. The project was therefore not covered by the Committee Act and without the obligation to report to the ethic committee system. Administrative permission to access the data was acquired 25 September 2018 from the head of Slagelse Hospital Administration.
Author Contributions
All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; agreed to submit to the current journal; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.
Disclosure
The authors do not report any conflicts of interest in relation to this study.
References
- 1.Stoller J, Halpin L, Weis M, et al. Epidemiology of severe sepsis: 2008–2012. J Crit Care. 2016;31(1):58–62. doi: 10.1016/j.jcrc.2015.09.034 [DOI] [PubMed] [Google Scholar]
- 2.Hermans MAW, Leffers P, Jansen LM, et al. The value of the Mortality in Emergency Department Sepsis (MEDS) score, C reactive protein and lactate in predicting 28-day mortality of sepsis in a Dutch emergency department. Emerg Med J. 2012;29(4):295–300. doi: 10.1136/emj.2010.109090 [DOI] [PubMed] [Google Scholar]
- 3.Martin GS, Mannino DM, Eaton S, et al. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348(16):1546–1554. doi: 10.1056/NEJMoa022139 [DOI] [PubMed] [Google Scholar]
- 4.Henriksen DP, Laursen CB, Jensen TG, et al. Incidence rate of community-acquired sepsis among hospitalized acute medical patients-a population-based survey. Crit Care Med. 2015;43(1):13–21. doi: 10.1097/CCM.0000000000000611 [DOI] [PubMed] [Google Scholar]
- 5.Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–810. doi: 10.1001/jama.2016.0287 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Abdullah SMOB, Sørensen RH, Dessau RBC, et al. Prognostic accuracy of qSOFA in predicting 28-day mortality among infected patients in an emergency department: a prospective validation study. Emerg Med J. 2019;36(12):722–728. doi: 10.1136/emermed-2019-208456 [DOI] [PubMed] [Google Scholar]
- 7.Askim Å, Moser F, Gustad LT, et al. Poor performance of quick-SOFA (qSOFA) score in predicting severe sepsis and mortality - a prospective study of patients admitted with infection to the emergency department. Scand J Trauma Resusc Emerg Med. 2017;25(1):56. doi: 10.1186/s13049-017-0399-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.González Del Castillo J, Julian-Jiménez A, González-Martínez F, et al. Prognostic accuracy of SIRS criteria, qSOFA score and GYM score for 30-day-mortality in older non-severely dependent infected patients attended in the emergency department. Eur J Clin Microbiol Infect Dis. 2017;36(12):2361–2369. doi: 10.1007/s10096-017-3068-7 [DOI] [PubMed] [Google Scholar]
- 9.Machado FR, Cavalcanti AB, Monteiro MB, et al. Predictive accuracy of the quick sepsis-related organ failure assessment score in Brazil: a Prospective Multicenter Study. Am J Respir Crit Care Med. 2020;201(7):789–798. doi: 10.1164/rccm.201905-0917OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Abdullah SMOB, Grand J, Sijapati A, et al. qSOFA is a useful prognostic factor for 30-day mortality in infected patients fulfilling the SIRS criteria for sepsis. Am J Emerg Med. 2019;38(3):512–516. doi: 10.1016/j.ajem.2019.05.037 [DOI] [PubMed] [Google Scholar]
- 11.Freund Y, Lemachatti N, Krastinova E, et al. Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department. JAMA. 2017;317(3):301–308. doi: 10.1001/jama.2016.20329 [DOI] [PubMed] [Google Scholar]
- 12.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):762–774. doi: 10.1001/jama.2016.0288 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kovach CP, Fletcher GS, Rudd KE, et al. Comparative prognostic accuracy of sepsis scores for hospital mortality in adults with suspected infection in non-ICU and ICU at an academic public hospital. PLoS One. 2019;14(9):e0222563. doi: 10.1371/journal.pone.0222563 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Poutsiaka DD, Porto MC, Perry WA, et al. Prospective observational study comparing sepsis-2 and sepsis-3 definitions in predicting mortality in critically ill patients. Open Forum Infect Dis. 2019;6(7):ofz271. doi: 10.1093/ofid/ofz271 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schmidt M, Schmidt SAJ, Adelborg K, et al. The Danish health care system and epidemiological research: from health care contacts to database records. Clin Epidemiol. 2019;11:563–591. doi: 10.2147/CLEP.S179083 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8 [DOI] [PubMed] [Google Scholar]
- 17.Schmidt M, Pedersen L, Sørensen HT. The danish civil registration system as a tool in epidemiology. Eur J Epidemiol. 2014;29(8):541–549. doi: 10.1007/s10654-014-9930-3 [DOI] [PubMed] [Google Scholar]
- 18.Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010;5(9):1315–1316. doi: 10.1097/JTO.0b013e3181ec173d [DOI] [PubMed] [Google Scholar]
- 19.Boillat-Blanco N, Mbarack Z, Samaka J, et al. Prognostic value of quickSOFA as a predictor of 28-day mortality among febrile adult patients presenting to emergency departments in Dar es Salaam, Tanzania. PLoS One. 2018;13(6):e0197982. doi: 10.1371/journal.pone.0197982 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lo RSL, Leung LY, Brabrand M, et al. qSOFA is a poor predictor of short-term mortality in all patients: a systematic review of 410,000 Patients. J Clin Med. 2019;8(1). doi: 10.3390/jcm8010061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Raith EP, Udy AA, Bailey M, et al. Prognostic Accuracy of the SOFA Score, SIRS Criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317(3):290–300. doi: 10.1001/jama.2016.20328 [DOI] [PubMed] [Google Scholar]
- 22.April MD, Aguirre J, Tannenbaum LI, et al. Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment. J Emerg Med. 2017;52(5):622–631. doi: 10.1016/j.jemermed.2016.10.012 [DOI] [PubMed] [Google Scholar]
- 23.Donnelly JP, Safford MM, Shapiro NI, et al. Application of the third international consensus definitions for sepsis (Sepsis-3) Classification: a retrospective population-based cohort study. Lancet Infect Dis. 2017;17(6):661–670. doi: 10.1016/S1473-3099(17)30117-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Gaini S, Relster MM, Pedersen C, et al. Prediction of 28-days mortality with sequential organ failure assessment (SOFA), quick SOFA (qSOFA) and systemic inflammatory response syndrome (SIRS) - A retrospective study of medical patients with acute infectious disease. Int J Infect Dis. 2019;78:1–7. doi: 10.1016/j.ijid.2018.09.020 [DOI] [PubMed] [Google Scholar]
- 25.López-Izquierdo R, Brio-Ibañez PD, Martín-Rodríguez F, et al. Role of qSOFA and SOFA scoring systems for predicting in-hospital risk of deterioration in the emergency department. Int J Environ Res Public Health. 2020;17:22. doi: 10.3390/ijerph17228367 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Pierrakos C, Velissaris D, Bisdorff M, et al. Biomarkers of sepsis: time for a reappraisal. Crit Care. 2020;24(1):287. doi: 10.1186/s13054-020-02993-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Liu Y, Hou J-H, Li Q, et al. Biomarkers for diagnosis of sepsis in patients with systemic inflammatory response syndrome: a systematic review and meta-analysis. Springerplus. 2016;5(1):2091. doi: 10.1186/s40064-016-3591-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Song J, Park DW, Moon S, et al. Diagnostic and prognostic value of interleukin-6, pentraxin 3, and procalcitonin levels among sepsis and septic shock patients: a prospective controlled study according to the Sepsis-3 definitions. BMC Infect Dis. 2019;19(1):968. doi: 10.1186/s12879-019-4618-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Ullrich E, Heidinger P, Soh J, et al. Evaluation of host-based molecular markers for the early detection of human sepsis. J Biotechnol. 2020;310:80–88. doi: 10.1016/j.jbiotec.2020.01.013 [DOI] [PubMed] [Google Scholar]
- 30.Churpek MM, Snyder A, Han X, et al. Quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med. 2017;195(7):906–911. doi: 10.1164/rccm.201604-0854OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):1619–1636. doi: 10.1378/chest.100.6.1619 [DOI] [PubMed] [Google Scholar]
- 32.Angus DC, Linde-Zwirble WT, Lidicker J, et al. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310. doi: 10.1097/00003246-200107000-00002 [DOI] [PubMed] [Google Scholar]
- 33.D’Acremont V, Kilowoko M, Kyungu E, et al. Beyond malaria--causes of fever in outpatient Tanzanian children. N Engl J Med. 2014;370(9):809–817. doi: 10.1056/NEJMoa1214482 [DOI] [PubMed] [Google Scholar]
- 34.Falsetti L, Martino M, Zaccone V, et al. SOFA and qSOFA usefulness for in-hospital death prediction of elderly patients admitted for suspected infection in internal medicine. Infection. 2020;48(6):879–887. doi: 10.1007/s15010-020-01494-5 [DOI] [PubMed] [Google Scholar]
- 35.Umemura Y, Ogura H, Gando S, et al. Assessment of mortality by qSOFA in patients with sepsis outside ICU: a post hoc subgroup analysis by the Japanese Association for Acute Medicine Sepsis Registry Study Group. J Infect Chemother. 2017;23(11):757–762. doi: 10.1016/j.jiac.2017.07.005 [DOI] [PubMed] [Google Scholar]