Abstract
Background
The diagnostic and prognostic utility of various sepsis scores varied among different cohorts and settings.
Methods
A prospective cohort study in adult patients with sepsis at Siriraj Hospital (Bangkok, Thailand) was conducted during January to July 2019. The performance of sepsis assessments, including systemic inflammatory response syndrome (SIRS) score, sequential organ failure assessment (SOFA) score, quick sepsis-related organ failure assessment (qSOFA) score, modified early warning score (MEWS), and national early warning score (NEWS), for sepsis detection and mortality prediction were compared with agreement between 2 infectious disease (ID) specialists to determine their sepsis and septic shock status as the reference standard.
Results
Among the 470 subjects included in this study, 206 patients (43.8%) were determined by 2 ID specialists to have sepsis. Systemic inflammatory response syndrome ≥2, qSOFA ≥2, and NEWS ≥5 yielded the highest sensitivity (93.2%), specificity (81.3%), and accuracy (72.6%), respectively, for detecting sepsis. The SIRS ≥2 had the highest sensitivity (97.8%), whereas qSOFA ≥2 had the highest specificity (61%) and accuracy (69.7%) for predicting mortality among sepsis patients. Receiver operating characteristic (ROC) curve showed MEWS to have the highest discriminatory power for sepsis detection (area under the ROC curve [AUROC], 0.79; 95% confidence interval [CI], 0.74–0.83), whereas SOFA had the highest discriminatory power for predicting hospital mortality (AUROC, 0.76; 95% CI, 0.69–0.79).
Conclusions
The NEWS ≥5 and qSOFA ≥2 were the most accurate scoring systems for sepsis detection and mortality prediction, respectively. Each scoring system is useful for different specific purposes relative to early detection and mortality prediction in sepsis patients.
Keywords: MEWS, NEWS, qSOFA, SIRS, SOFA
NEWS ≥5 and qSOFA ≥2 were the most accurate scoring systems for sepsis detection and mortality prediction, respectively. Each scoring system is useful for different specific purposes relative to early detection and mortality prediction in sepsis.
Sepsis is a life-threatening condition characterized by multiple organ dysfunction that is caused by an overwhelming host response to infection [1]. Sepsis remains a major public health problem worldwide with an estimated 11 million sepsis-related deaths in 2017 [2]. According to data from the Ministry of Public Health and National Health Security Office of Thailand, approximately 175 000 patients develop sepsis each year, and approximately 45 000 of those patients die. In Thailand, the mortality rate among patients with community-acquired sepsis was approximately 32% in 2017 [3]. There is substantial variation in sepsis incidence and mortality across regions, with the highest burden in low- to middle-income countries (LMICs) [2, 4]. Sepsis-specific epidemiological data in the Southeast Asian population are still limited [5]. Early recognition of sepsis can improve outcome via timely and appropriate interventions [6]. However, no gold standard criteria for detecting patients with sepsis have been established.
The systemic inflammatory response syndrome (SIRS) criteria have been used as a screening tool for the diagnosis of sepsis since 1991. However, due to the inadequate sensitivity and specificity of the SIRS criteria [7], the sequential organ failure assessment (SOFA) score and the quick sepsis-related organ failure assessment (qSOFA) score were proposed as new diagnostic criteria for sepsis by the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) in 2016. Performance of the qSOFA score for prediction of mortality in sepsis patients outside the intensive care unit (ICU) was reported from many studies in high-income countries [8–10]. An alternative scoring system for diagnosis of sepsis, the early warning score (EWS), was developed to meet the needs of different types of patients, such as the pediatric early warning score (PEWS), the national early warning score (NEWS), and the modified early warning score (MEWS). A comparative study reported that NEWS had greater prognostic accuracy than qSOFA or SIRS for hospital mortality, ICU transfer, and ICU length of stay in patients with sepsis [11]. However, there are limited data on the utility of the qSOFA score in LMICs [12].
The predictive validity of different sepsis scoring systems varies among patient cohorts and settings. Moreover, no studies have comprehensively evaluated and compared the different sepsis detection scoring systems in the Thai population. Accordingly, this study aimed to determine the diagnostic performance of SIRS score, qSOFA score, SOFA score, MEWS, and NEWS for sepsis detection and mortality prediction in adult patients suspected of having sepsis at Siriraj Hospital, Mahidol University, Bangkok, Thailand.
METHODS
Study Design and Population
This prospective observational study was conducted at Siriraj Hospital, which is the largest university-based national referral center in Thailand, during January 2019 to July 2019. Eligible patients were hospitalized adults aged 18 years or older who had blood cultures performed.
Patient Consent Statement
The protocol for this study was approved by the Institutional Review Boards of Siriraj Hospital (COA no. SI 597/2019), and written informed consent was waived because the research involves no more than minimal risk to the subjects.
Study Procedure
A list of hospitalized patients from all wards, including the ICU, who had blood culture performed was compiled, and patients were randomly selected using the random generator feature of R program software. A list of randomly selected patients was then provided to the research physician twice weekly. Approximately 100 to 120 subjects were recruited monthly. Clinical and laboratory data relating to SIRS score, qSOFA score, SOFA score, MEWS, and NEWS were collected at the time of blood draw for culture or within 6 hours before blood culture. The worst values for each item of the aforementioned scoring systems were used. Each patient was included only once on the date of suspected infection with bacteremia or sepsis or septic shock. The follow-up blood culture was not taken into account for determining diagnostic and prognostic utility compared among the different sepsis scoring systems. Patient clinical and laboratory data for each sepsis scoring system are shown in Supplementary Table S1. Presence of infection was determined by attending physicians at the time blood cultures were taken. The following data were also obtained from medical records: comorbidity, site and type of infection, causative pathogen, antimicrobial therapy, fluid and vasopressor therapy, and mortality.
Presence of sepsis and septic shock was determined by the opinion of 2 infectious disease (ID) specialists who reviewed the clinical, microbiological, and radiological data of each enrolled study subject, as well as the results of other diagnostic tests performed in the hospital until the patient died or was discharged from the hospital. Each of the 2 ID specialists independently reviewed the aforementioned data of each enrolled study subject to determine whether that patient had sepsis/septic shock. During that process, neither ID specialist knew the opinion of the other ID specialist. Any disagreement between the 2 ID specialists was decided by the opinion of a third ID specialist. In brief, the process involved first determining the presence of a particular kind of infection. The second criterion that had to be met to define sepsis was the presence of dysfunction in at least 1 organ/system, including cardiovascular, hematological, liver, neurological, respiratory, or renal involvement.
Definitions
Sepsis was defined as the presence of infection with organ dysfunction based on the clinical features, biochemical laboratory test results, microbiological findings, and the clinical course of the subject. Septic shock was defined as sepsis with persisting hypotension requiring vasopressor to maintain a mean arterial pressure of 65 mmHg or having a serum lactate level >2 mmol/L despite adequate volume resuscitation [8]. In our study, we classified infections into 3 groups according to the setting where the infection was acquired. Community-acquired infection was defined as an infection in a patient who was hospitalized ≤2 days, who had no healthcare-associated conditions, and who was not hospitalized in other hospitals longer than 2 days before transfer to this hospitalization. Hospital-acquired infection was defined as an infection in a patient who was hospitalized >2 days or hospitalized in other hospitals longer than 2 days before transfer to this hospitalization. Healthcare-associated infection was defined as an infection in a patient who was hospitalized ≤2 days that had a history of prior hospitalization or prior use of antibiotic within the preceding 90 days, who was a resident in a nursing home or extended care facility, or who had chronic dialysis within 30 days. Concordant empiric antibiotic therapy (ABT) was defined as the identified organism being susceptible to the given empiric ABT. Definite ABT was defined as the ABT given subsequently according to the result of susceptibility testing.
Outcomes
The primary outcome was the performance and accuracy of different sepsis scoring systems for detecting sepsis or septic shock compared with agreement in the opinions of 2 ID specialists as the gold standard. The secondary outcomes included (1) performance and accuracy of each scoring system for predicting hospital mortality in sepsis patients and (2) identification of factors independently associated with mortality in sepsis patients.
Sample Size Estimation
Based on the results of a previous study, we estimated the sensitivity of qSOFA for detecting sepsis to be 80% ± 10%. Using a type I error (2-sided) of 5%, the estimated number of sepsis patients was 62. That number was increased by 10% to compensate for missing data, which increased our minimum sample size to approximately 70 patients.
Information from the Global Antimicrobial Resistance Surveillance System (GLASS) at Siriraj Hospital in 2016 showed that the number of hospitalized patients who have blood cultures performed at Siriraj Hospital is approximately 5000 patients per year. The rate of sepsis in patients with positive blood culture specimens was approximately 15% [13]. Therefore, 467 adult subjects who had blood culture performed was estimated to yield at least 70 sepsis subjects and approximately 397 nonsepsis subjects.
Statistical Analysis
Characteristics of enrolled subjects were described using descriptive statistics. Continuous variables are reported as mean ± standard deviation for normally distributed variables and as median (25th percentile and 75th percentile) or range for nonnormally distributed variables. Student’s t test or Mann-Whiney U test was used for comparison of continuous variables, whereas χ 2 test or Fisher’s exact test was used for comparison of categorical variables. All statistical tests were 2 sided, with a P value less than .05 indicating statistical significance. Variables with a P < .05 were further analyzed for independent association with mortality using binary logistic regression analysis with a forward stepwise method. Those results are presented as odds ratio (OR) and 95% confidence interval (CI). Performance measures, including sensitivity, specificity, negative and positive predictive values, and accuracy, of various scoring systems for sepsis detection and mortality prediction were calculated as percentage and 95% CI. We evaluated the accuracy of various scoring systems for sepsis detection compared with agreement in the opinions of 2 ID specialists, which was considered the reference standard for sepsis and septic shock. Accuracy was calculated as the percentage of correctly classified instances (true positive + true negative)/(true positive + true negative + false positive + false negative). Diagnostic and prognostic accuracy of the various scoring systems were analyzed and compared by receiver operating characteristic (ROC) curve and area under the ROC curve (AUROC) values with 95% CI. All analyses were performed using PASW Statistics (SPSS) 18.0 (SPSS, Inc., Chicago, IL).
RESULTS
Among the 558 adult subjects with blood culture taken during the study period, 88 subjects were duplicate subjects who had already been previously recruited into the study. Among the 470 subjects included in this study, 409 were diagnosed with infection. Of those, 206 and 203 subjects were determined by 2 ID specialists to have and not have sepsis, respectively, for an incidence of sepsis of 43.8% among patients who had blood cultures performed. The concordance of interobserver agreement in the judgment of sepsis and septic shock was 78.6% (95% CI, 74.5%–82.2%) and 91.3% (95% CI, 88.3%–93.7%), respectively.
The baseline characteristics of subjects with and without sepsis are shown in Table 1. Two hundred thirty-seven (57.9%) patients had a positive culture result from any site, and 117 patients (28.6%) had a positive culture from blood, which was defined as true bacteremia. Thirty-one positive blood cultures were considered contaminated (20 in sepsis [9.7%], and 11 [5.4%] in nonsepsis). Sepsis subjects had significantly more culture-proven infection and bacteremia than those who did not have sepsis.
Table 1.
Demographic and Clinical Characteristics of 409 Patients Who Had Blood Cultures Performed Classified by Sepsis Category
| Total | Nonsepsis | Sepsis | ||
|---|---|---|---|---|
| Variables | (n = 409) | (n = 203) | (n = 206) | P Value |
| Mean (SD) age, years | 65.74 (17.84) | 63.81 (17.66) | 67.64 (17.85) | .03 |
| Male gender, n (%) | 210 (51.3) | 97 (47.8) | 113 (54.9) | .153 |
| Comorbidities, n (%) | 396 (96.8) | 195 (96.1) | 201 (97.6) | .383 |
| Hypertension | 230 (56.2) | 110 (54.2) | 120 (58.3) | .407 |
| Diabetes mellitus | 142 (34.7) | 74 (36.5) | 68 (33.0) | .465 |
| Chronic kidney disease | 99 (24.2) | 52 (25.6) | 47 (22.8) | .509 |
| Dyslipidemia | 82 (20.0) | 42 (20.7) | 40 (19.4) | .748 |
| Nonhematologic malignancy | 74 (18.1) | 40 (19.7) | 34 (16.5) | .401 |
| Received immunosuppressive agent | 67 (16.4) | 25 (12.3) | 42 (20.4) | .027 |
| Heart disease | 67 (16.4) | 33 (16.3) | 34 (16.5) | .946 |
| Cerebrovascular disease | 66 (16.1) | 36 (17.7) | 30 (14.6) | .383 |
| Hematologic malignancy | 49 (12.0) | 19 (9.4) | 30 (14.6) | .105 |
| Heart failure | 48 (11.7) | 25 (12.3) | 23 (11.2) | .718 |
| Chronic lung disease | 26 (6.4) | 10 (4.9) | 16 (7.8) | .239 |
| Chronic liver disease | 25 (6.1) | 13 (6.4) | 12 (5.8) | .807 |
| Autoimmune disease | 25 (6.1) | 13 (6.4) | 12 (5.8) | .807 |
| Degenerative brain diseases | 24 (5.9) | 12 (5.9) | 12 (5.8) | .970 |
| HIV infection | 9 (2.2) | 4 (2.0) | 5 (2.4) | .753 |
| Transplant | 8 (2.0) | 6 (3.0) | 2 (1.0) | .147 |
| Others | 128 (31.3) | 65 (32.0) | 63 (30.6) | .754 |
| Culture proven, n (%) | 237 (57.9) | 89 (43.8) | 148 (71.8) | <.001 |
| Bacteremia, n (%) | 117 (28.6) | 29 (14.3%) | 88 (42.7) | <.001 |
| Type of Infection, n (%) | ||||
| Hospital-acquired | 242 (59.2) | 116 (57.1) | 126 (61.2) | .408 |
| Community-acquired | 141 (34.5) | 73 (36.0) | 68 (33.0) | .530 |
| Healthcare-associated | 26 (6.4) | 14 (6.9) | 12 (5.8) | .657 |
| Site of Infection, n (%) | ||||
| Respiratory tract | 138 (33.7) | 65 (32.0) | 73 (35.4) | .465 |
| Urinary tract | 65 (15.9) | 35 (17.7) | 29 (14.1) | .312 |
| Gastrointestinal tract | 45 (11.0) | 31 (15.0) | 14 (6.9) | .008 |
| Primary bacteremia | 42 (10.3) | 33 (16.0) | 9 (4.4) | <.001 |
| Skin and soft tissue | 35 (8.6) | 17 (8.3) | 18 (8.9) | .824 |
| Catheter-related BSI | 12 (2.9) | 6 (2.9) | 6 (3.0) | .979 |
| Cardiovascular | 6 (1.5) | 2 (1.0) | 4 (2.0) | .401 |
| Central nervous system | 5 (1.2) | 1 (0.5) | 4 (2.0) | .172 |
| Others | 23 (5.6) | 3 (1.5) | 20 (9.9) | <.001 |
| Unknown | 60 (14.7) | 21 (10.2) | 39 (19.2) | .010 |
| Severity of Illness | ||||
| SIRS ≥2 | 323 (79.0) | 131 (64.5) | 192 (93.2) | <.001 |
| qSOFA ≥2 | 155 (37.9) | 38 (18.7) | 117 (56.8) | <.001 |
| SOFA ≥2 | 284 (69.4) | 113 (55.7) | 171 (83.0) | <.001 |
| MEWS ≥4 | 288 (70.4) | 105 (51.7) | 183 (88.8) | <.001 |
| NEWS ≥5 | 214 (52.3) | 60 (29.6) | 154 (74.8) | <.001 |
| Receiving empiric ABT | 386 (94.4) | 184 (89.3) | 202 (98) | .015 |
| Concordant empiric ABT | 146 (37.8) | 54 (29.3) | 92 (45.5) | .001 |
| Combination empiric ABT | 131 (33.9) | 45 (24.5) | 86 (42.5) | <.001 |
| Receiving definite ABT | 234 (57.2) | 97 (47.8) | 137 (66.5) | <.001 |
| Concordant definite ABT | 212 (91) | 91 (94.8) | 121 (88.3) | .183 |
| Combination definite ABT | 84 (35.9) | 24 (24.7) | 60 (43.8) | .003 |
| Duration of ABT (days), median (IQR) | 10 (6–15) | 10 (7–15) | 10 (6–15) | .434 |
| Discharge Status, n (%) | ||||
| Death | 117 (28.6) | 27 (13.3) | 90 (43.7) | <.001 |
Abbreviations: ABT, antibiotic therapy; BSI, blood stream infection; HIV, human immunodeficiency virus; IQR, interquartile range; MEWS, modified early warning score; NEWS, national early warning score; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SD, standard deviation; SOFA, sequential organ failure assessment.
Performance of Various Scoring Systems for Sepsis Detection and Mortality Prediction
The performance of various scoring systems for sepsis detection is shown in Table 2. The SIRS ≥2 had the highest sensitivity (93.2%) but the lowest specificity (35.5%) for sepsis detection. In contrast, qSOFA ≥2 had a lower sensitivity (56.8%), but it had the highest specificity (81.3%) compared with SIRS ≥2, SOFA ≥2, MEWS ≥4, and NEWS ≥5. The qSOFA had the lowest false positive (FP) rate for detection of sepsis (18.7%) compared with the other scores. The FP rate of NEWS, MEWS, SOFA, and SIRS was 26.6%, 51.7%, 55.7%, and 64.5%, respectively. All scoring systems with recommended cutoff scores, including SIRS≥2, qSOFA ≥2, SOFA ≥2, MEWS ≥4, and NEWS ≥5, had moderate accuracy for sepsis detection with scores that ranged from 63.8% by SIRS to 72.6% by NEWS. The discriminatory power of all evaluated scores for detecting sepsis was analyzed by area under the ROC (AUROC) curve. That analysis revealed the highest AUROC for MEWS (AUROC, 0.79; 95% CI, 0.74–0.83), followed by NEWS (AUROC, 0.78; 95% CI, 0.74–0.83), SIRS (AUROC, 0.75; 95% CI, 0.70–0.79), qSOFA (AUROC, 0.72; 95% CI, 0.67–0.77), and SOFA (AUROC, 0.71; 95% CI, 0.66–0.76).
Table 2.
Diagnostic Performance Compared Among Various Sepsis Detection Scoring Systems
| Sensitivity | Specificity | PPV | NPV | Accuracy | |
|---|---|---|---|---|---|
| Scoring System | % (95% CI) | % (95% CI) | % (95% CI) | % (95% CI) | % (95% CI) |
| SIRS ≥2 | 93.2 (88.9–96.2) | 35.5 (28.9–42.5) | 59.4 (53.9–64.8) | 83.7 (74.2–90.8) | 64.5 (59.7–69.2) |
| qSOFA ≥2 | 56.8 (49.7–63.7) | 81.3 (75.2–86.4) | 75.5 (67.9–82.0) | 65.0 (58.7–70.8) | 68.9 (64.2–73.4) |
| SOFA ≥2 | 83.0 (77.2–87.9) | 44.3 (37.4–51.5) | 60.2 (54.3–65.9) | 72.0 (63.3–79.7) | 63.8 (58.9–68.5) |
| MEWS ≥4 | 88.8 (83.7–92.8) | 48.3 (41.2–55.4) | 63.5 (57.7–69.1) | 81.0 (72.9–87.6) | 68.7 (64.0–73.2) |
| NEWS ≥5 | 74.8 (68.3–80.5) | 70.4 (63.7–76.6) | 72.0 (65.4–77.9) | 73.3 (66.5–79.4) | 72.6 (68.0–76.9) |
Abbreviations: CI, confidence interval; MEWS, modified early warning score; NEWS, national early warning score; NPV, negative predictive value; PPV, positive predictive value; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment.
The performance of the evaluated scoring systems for predicting mortality is shown in Table 3. All scoring systems had poor accuracy of lower than 60% for predicting mortality except for qSOFA score, which had an accuracy level of 70%. The discriminatory power of all evaluated scores for predicting hospital mortality was assessed by AUROC. The SOFA had the highest AUROC (AUROC, 0.76; 95% CI, 0.69–83), followed by qSOFA (AUROC, 0.74; 95% CI, 0.67–0.81), MEWS (AUROC, 0.72; 95% CI, 0.65–0.79), NEWS (AUROC, 0.70; 95% CI, 0.63–0.78), and SIRS (AUROC, 0.53; 95% CI, 0.45–0.61).
Table 3.
Performance Compared Among Various Scoring Systems for Predicting Mortality in Sepsis Patients
| Sensitivity | Specificity | PPV | NPV | Accuracy | |
|---|---|---|---|---|---|
| Scoring System | % (95% CI) | % (95% CI) | % (95% CI) | % (95% CI) | % (95% CI) |
| SIRS ≥2 | 97.8 (92.2–99.7) | 10.5 (5.3–18.0) | 48.4 (40.9–55.9) | 84.6 (54.6–98.1) | 50.8 (43.5–58.0) |
| qSOFA ≥2 | 80.0 (70.2–87.7) | 61.0 (50.9–70.3) | 63.7 (54.1–72.6) | 78.0 (67.5–86.4) | 69.7 (62.8–76.1) |
| SOFA ≥2 | 94.4 (87.5–98.2) | 25.7 (17.7–35.2) | 52.1 (44.2–60.0) | 84.4 (67.3–94.7) | 57.4 (50.2–64.4) |
| MEWS ≥4 | 95.6 (89.0–98.8) | 16.2 (9.7–24.7) | 49.4 (41.8–57.1) | 81.0 (58.1–94.6) | 52.8 (45.6–59.0) |
| NEWS ≥5 | 86.7 (77.9–92.9) | 33.3 (24.4–43.2) | 52.7 (44.3–61.0) | 74.5 (59.7–86.1) | 57.9 (50.7–65.0) |
Abbreviations: CI, confidence interval; MEWS, modified early warning score; NEWS, national early warning score; NPV, negative predictive value; PPV, positive predictive value; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment.
The cutoff values of ≥3 for SIRS, ≥4 for SOFA, and ≥6 for MEWS could increase the accuracy (sensitivity, specificity) of sepsis detection to 70.2% (68.5%, 71.9%), 66% (62.1%, 69.9%), and 71.9% (63.1%, 80.8%), respectively. The cutoff values of ≥5 for SOFA, ≥8 for MEWS, and ≥7 for NEWS could increase the accuracy (sensitivity, specificity) of mortality prediction to 71.3% (66.7%, 75.2%), 68.7% (51.1%, 83.8%), and 67.2% (70%, 64.8%), respectively.
Causative Pathogens in Sepsis Subjects
Bacteria were responsible for infection in 89.3% of all sepsis subjects, consisting of Gram-negative bacteria in 64.8% and Gram-positive bacteria in 24.5%. Escherichia coli was the most commonly isolated Gram-negative bacteria (16.8%), followed by Acinetobacter baumannii (13.0%) and Pseudomonas aeruginosa (11.5%). Staphylococcus aureus was the most commonly isolated Gram-positive bacteria (8.6%), followed by Enterococcus faecium (5.3%) and Enterococcus faecalis (2.4%). Fungus was identified as a causative pathogen in 8.7% of subjects. Candida spp (4.8%) was the most commonly identified fungi. Virus (1.5%) and mycobacteria (0.5%) were also identified in this study cohort. The causative pathogen isolated from patients classified by sepsis category was shown in Supplementary Table S2.
Treatment in Sepsis and Nonsepsis Subjects
The median time from blood culture collection to the first dose of antibiotics was 45 minutes (interquartile range, 0–3 hours), which was not significantly different between survivors and nonsurvivors. The information on concordance of ABT is shown in Table 1. Fluid resuscitation was significantly more often given in nonsurvivors (45.6% vs 14.3%, P < .001). Other treatments, such as vasoactive agent, corticosteroid, mechanical ventilator, and renal replacement therapy, were all significantly more often given to or used in nonsurvivors than in survivors (45.6% vs 13.3%, P < .001; 42.2% vs 15.2%, P < .001; 58.9% vs 22.9%, P < .001; and, 20% vs 8.6%, P < .001, respectively).
Comparison of Sepsis Subjects Between Those Who Died and Survived
Of the 206 sepsis patients, 11 with unknown outcome were excluded because they left the hospital against medical advice or were transferred to other hospitals, and 90 patients died in the hospital for an in-hospital case-fatality rate of 43.7%. Of those who died, 51 patients (56.7%) died during sepsis treatment, and 73 patients (81.1%) died within 28 days for a 28-day case-fatality rate of 35.4%. The outcomes of the remaining 195 patients who survived or died are shown in Table 4.
Table 4.
Comparison Between Sepsis Patients Who Survived and Dieda
| Variables | Total (n = 195) | Survivors (n = 105) | Died (n = 90) | P Value |
|---|---|---|---|---|
| Mean (SD) age, years | 67.64 (17.80) | 65.25 (18.83) | 70.43 (16.18) | .042 |
| Male gender, n (%) | 104 (53.3) | 52 (49.5) | 52 (57.8) | .249 |
| Comorbidities, n (%) | 190 (97.4) | 101 (96.2) | 89 (98.9) | .235 |
| Hypertension | 114 (58.5) | 65 (61.9) | 49 (54.4) | .292 |
| Diabetes mellitus | 66 (33.8) | 41 (39.0) | 25 (27.8) | .097 |
| Chronic kidney disease | 44 (22.6) | 25 (23.8) | 19 (21.1) | .653 |
| Received immunosuppressive agent | 41 (21.0) | 18 (17.1) | 23 (25.6) | .151 |
| Dyslipidemia | 38 (19.5) | 20 (19.0) | 18 (20.0) | .867 |
| Nonhematologic malignancy | 33 (16.9) | 16 (15.2) | 17 (18.9) | .498 |
| Heart disease | 31 (15.9) | 17 (16.2) | 14 (15.6) | .904 |
| Hematologic malignancy | 29 (14.9) | 14 (13.3) | 15 (16.7) | .514 |
| Cerebrovascular disease | 28 (14.4) | 16 (15.2) | 12 (13.3) | .705 |
| Heart failure | 21 (10.8) | 9 (8.6) | 12 (13.3) | .285 |
| Chronic lung disease | 16 (8.2) | 9 (8.6) | 7 (7.8) | .840 |
| Autoimmune disease | 12 (6.2) | 6 (5.7) | 6 (6.7) | .783 |
| Chronic liver disease | 12 (6.2) | 5 (4.8) | 7 (7.8) | .382 |
| Neurological diseases | 12 (6.2) | 8 (7.6) | 4 (4.4) | .358 |
| HIV infection | 4 (2.1) | 3 (2.9) | 1 (1.1) | .391 |
| Transplant | 2 (1.0) | 1 (1.0) | 1 (1.1) | .913 |
| Others | 59 (30.3) | 34 (32.4) | 25 (27.8) | .485 |
| Culture proven, n (%) | 140 (71.8) | 71 (67.6) | 69 (76.7) | .162 |
| Bacteremia, n (%) | 83 (42.6) | 43 (41.0) | 40 (44.4) | .755 |
| Type of Infection, n (%) | ||||
| Hospital-acquired | 121 (62.1) | 56 (53.3) | 65 (72.2) | .007 |
| Community-acquired | 62 (31.8) | 41 (39.0) | 21 (23.2) | .019 |
| Healthcare-associated | 12 (6.2) | 8 (7.6) | 4 (4.4) | .358 |
| Site of Infection, n (%) | ||||
| Respiratory tract | 70 (35.9) | 27 (25.7) | 43 (47.8) | .001 |
| Primary bacteremia | 33 (16.9) | 15 (14.3) | 18 (20.0) | .289 |
| Gastrointestinal tract | 30 (15.4) | 17 (16.2) | 13 (14.4) | .736 |
| Urinary tract | 27 (13.8) | 20 (19) | 7 (7.8) | .023 |
| Skin and soft tissue | 15 (7.7) | 10 (9.5) | 5 (5.6) | .300 |
| Catheter-related BSI | 4 (2.1) | 2 (1.9) | 2 (2.2) | .876 |
| Cardiovascular | 2 (1.0) | 1 (1.0) | 1 (1.1) | .913 |
| Central nervous system | 1 (0.5) | 1 (1.0) | 0 (0.0) | .353 |
| Others | 3 (1.5) | 2 (1.9) | 1 (1.1) | .654 |
| Unknown | 19 (9.7) | 13 (12.4) | 6 (6.7) | .180 |
| Number of Pathogens, n (%) | ||||
| Single pathogen | 98 (68.1) | 52 (72.2) | 46 (63.9) | .284 |
| Mixed pathogen | 46 (31.9) | 20 (27.8) | 26 (36.1) | .284 |
| Severity of Illness | ||||
| Septic shock, n (%) | 45 (23.1) | 10 (9.5) | 35 (38.9) | <.001 |
| SIRS ≥2 | 182 (93.3) | 94 (89.5) | 88 (97.8) | .021 |
| qSOFA ≥2 | 113 (57.9) | 41 (39.0) | 72 (80.0) | <.001 |
| SOFA ≥2 | 163 (83.6) | 78 (74.3) | 85 (94.4) | <.001 |
| MEWS ≥4 | 174 (89.2) | 88 (83.8) | 86 (95.6) | .008 |
| NEWS ≥5 | 148 (75.9) | 70 (66.7) | 78 (86.7) | .001 |
| Receiving empiric ABT | 191 (97.9) | 103 (98.1) | 90 (100) | .128 |
| Concordant empiric ABT | 89 (46.6) | 50 (48.5) | 39 (44.3) | .560 |
| Receiving definite ABT | 133 (68.2) | 74 (70.5) | 59 (65.6) | .606 |
| Concordant definite ABT | 118 (88.7) | 66 (89.2) | 52 (88.1) | .530 |
| Duration of ABT (days) | 10 (6–15) | 12 (7–16) | 8 (3–15) | <.001 |
| Median (IQR) | ||||
| Length of stay (days), median (IQR) | 20 (6.5–39) | 21 (8–42) | 17 (5–36) | .394 |
Abbreviations: ABT, antibiotic therapy; BSI, blood stream infection; HIV, human immunodeficiency virus; IQR, interquartile range; MEWS, modified early warning score; NEWS, national early warning score; qSOFA, quick sepsis-related organ failure assessment; SD, standard deviation; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment.
aExcluded 11 patients who left the hospital due to transfer or against medical advice.
The results of multivariate analysis for factors independently associated with death among sepsis patients are shown in Table 5. The independent factors associated with in-hospital mortality included presence of septic shock, qSOFA ≥2, and SOFA ≥2.
Table 5.
Multivariate Analysis for Factors Associated With Mortality in Sepsis Patients
| Factors | Crude OR (95% CI) | P Value | Adjusted ORa (95% CI) | P Value |
|---|---|---|---|---|
| Age | 1.02 (1.001–1.03) | .044 | - | - |
| Hospital-acquired infection | 2.28 (1.25–4.14) | .007 | - | - |
| Respiratory tract infection | 2.64 (1.45–4.83) | .002 | - | - |
| Urinary tract infection | 0.36 (0.14–0.89) | .028 | - | - |
| Septic shock | 6.45 (2.78–13.15) | <.001 | 4.72 (1.98–11.25) | <.001 |
| SIRS ≥2 | 5.15 (1.11–23.88) | .036 | 7.86 (1.33–46.59) | .023 |
| qSOFA ≥2 | 6.24 (3.27–11.94) | <.001 | 3.75 (1.84–7.65) | <.001 |
| SOFA ≥2 | 5.89 (2.16–16.04) | .001 | 3.68 (1.19–11.41) | .024 |
| MEWS ≥4 | 4.15 (1.34–12.85) | .013 | - | - |
| NEWS ≥5 | 3.25 (1.57–6.75) | .002 | - | - |
| Total duration of ABT | 0.95 (0.91–0.96) | .006 | - | - |
Abbreviations: ABT, antibiotic therapy; CI, confidence interval; OR, odds ratio; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment score.
aAdjusted for age, hospital-acquired infection, respiratory tract infection, urinary tract infection, septic shock, SIRS ≥2, qSOFA ≥2, SOFA ≥2, MEWS ≥, NEWS ≥5, and total duration of ABT.
DISCUSSION
To the best of our knowledge, this is the largest prospective cohort of sepsis patients in Thailand to be studied to determine the diagnostic and prognostic utility of various sepsis scoring systems. The early identification of sepsis is essential because earlier intervention improves patient outcomes. In the present study, we identified eligible subjects who might have sepsis from those who had blood culture performed because blood culture is recommended for all patients suspected of having sepsis and septic shock, bacteremia, or blood stream infection [14]. The presence of sepsis and septic shock was then determined by agreement between 2 ID specialists as the reference standard to evaluate the accuracy of the various aforementioned scoring systems. An approach to identifying sepsis in a patient who had blood culture performed is partly similar to that developed by the Centers for Disease Control and Prevention in the United States, called Adult Sepsis Event (ASE) [15]. The ASE is signified by obtaining blood culture and at least 4 consecutive days of antibiotics starting within 2 calendars days of when blood culture was obtained plus evidence of concurrent organ dysfunction.
In the present study, among the patients who had blood cultures performed, 43% of them were diagnosed as sepsis and one third had bacteremia. Approximately 36% and 44% of sepsis patients died within 28 days and in the hospital, respectively. Almost 60% of all death occurred during sepsis treatment. Regarding sepsis detection, SIRS ≥2 criteria had the highest sensitivity (93.2%) for sepsis detection; therefore, this might be a good screening tool for sepsis detection. Although qSOFA ≥2 had a lower sensitivity of 56.8%, it had the highest specificity (81.3%) compared with the other scoring systems. Therefore, qSOFA ≥2 was more accurate in sepsis detection than the other scores, with the exception of NEWS ≥5. When considering the AUROC, which calculated all possible cut-point values between sepsis and nonsepsis for each scoring system, all scores included in our study demonstrated good discrimination between sepsis and nonsepsis (AUC ≥0.7), especially MEWS ≥4 and NEWS ≥5. Although our results revealed that NEWS ≥5 exhibited the highest accuracy and almost the highest discriminatory ability for detecting sepsis, the relatively complicated calculation of this score could limit its use in daily clinical practice. Our study finding is consistent with that from a recent meta-analysis that found SIRS to be more sensitive but less specific than qSOFA for diagnosis of sepsis [16].
Concerning mortality prediction, SIRS ≥2 criteria had the highest sensitivity (97.8%) but unacceptably low specificity (10.5%). It is interesting to note that the other scores also had high sensitivity but lower specificity. Among those, qSOFA ≥2 had a high sensitivity (80%) with fair specificity (61%), which resulted in the highest accuracy for predicting mortality (69.7%). SOFA ≥2 was less accurate than qSOFA ≥2 for predicting mortality. However, an increase in the cut-point for SOFA from 2 or greater to 5 or greater could increase the accuracy to 71%. When considering the AUROC, which calculated all possible cut-point values between sepsis and nonsepsis for each scoring system, all assessments except for SIRS had good discriminatory power for predicting mortality (AUC ≥0.7). The SOFA and qSOFA outperformed the other scores.
Our results are similar to those from recent retrospective studies that included 2350 patients. Those studies found SOFA to be a better prognostic tool for predicting mortality and organ failure than qSOFA and SIRS among sepsis patients admitted to the ICU [17, 18]. However, a study in patients with suspected infection outside the ICU found qSOFA ≥2 to be a better predictor of mortality than SOFA ≥2 and SIRS ≥2 [8]. Other studies in other settings, including the emergency department [9], and a multicenter study from LMICs [19] also found qSOFA to be the best scoring system for predicting mortality. Similar findings were also observed in 2 recent systematic reviews and meta-analyses [16, 20].
Scoring systems each have their own strengths and limitations. For example, using SIRS, which had the highest sensitivity, might be most appropriate system for urgent screening of sepsis, especially in the emergency department. However, to fulfill the score, complete blood count is needed, which is time-consuming. In contrast, the poor sensitivity but high specificity of qSOFA for sepsis detection without any laboratory parameters may facilitate the delivery of early and appropriate interventions, especially among patients with a qSOFA ≥2. The MEWS and NEWS criteria have only clinical parameters, but they are more complex and less user-friendly scoring systems compared with qSOFA score. These obstacles limit their role in routine clinical practice, especially among physicians who work in urgent and high-throughput settings.
The strength of this study is that we evaluated adult sepsis patients from almost all units in the hospital; however, pediatric patients were excluded. Therefore, the results of this study should be considered generalizable to different adult patient populations in resource-limited settings. This study also has some mentionable limitations. First, because we used patients who had blood culture performed as a surrogate for suspected sepsis as our initial screening for the enrollment of subjects, our study could have in some way been influenced by selection bias. Some cases with sepsis or septic shock might have died early, or those who decided to receive palliative care without any treatment might not have had blood taken for culture. These could have led to an underestimation of hospitalized patients with sepsis. Second, we evaluated the score at the time of or within 6 hours before blood draw for culture, which might be later than the actual time of sepsis onset. Moreover, the sepsis score could have changed over time during the evolution of sepsis. Finally, there is currently no universally acceptable gold standard for sepsis diagnosis. Our study used the collective opinion of 2 ID specialists who reviewed all information of each subject from the time of presentation to the hospital until the patient died or was discharged from the hospital for sepsis diagnosis as the reference standard. This remains subjective and imperfect; however, there was substantial concordance between the opinions of the 2 ID specialists. In addition, some factors might have influenced the opinion of the ID specialists, such as overdiagnosis bias in cases with known blood culture results.
CONCLUSIONS
The SIRS ≥2 was the most sensitive, qSOFA ≥2 was the most specific, and NEWS ≥5 was the most accurate sepsis detection scoring system. The qSOFA ≥2 was the most accurate system for predicting hospital mortality in sepsis patients. The appropriate application of each scoring system is useful for different specific purposes relative to early detection and mortality prediction in sepsis.
Supplementary Data
Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Acknowledgments
We gratefully acknowledge the following individuals: Nerisa Thornsri and Orawan Supapeung of the Division of Clinical Epidemiology, Department of Research and Development, Faculty of Medicine Siriraj Hospital, Mahidol University for study assistance; and Khemajira Karaketklang of the Department of Medicine, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand for assistance with statistical analysis.
Authors contributions. P. P., L. T., N. A., W. W., R. S., and V. T. contributed to the study design, data collection, data analysis, and interpretation of the data. P. P. and N. A. wrote and revised the manuscript for important scientific content. All authors read and approved the final manuscript.
Disclaimer. The funding source had no role in the study design, data collection, analysis, or interpretation, conclusions drawn, or preparation of the manuscript.
Financial support. This work was funded by the Health Systems Research Institute, Nonthaburi, Thailand (Grant Number 60-073).
Potential conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest.
References
- 1. Rhodes A, Evans LE, Alhazzani W, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med 2017; 43:304–77. [DOI] [PubMed] [Google Scholar]
- 2. Rudd KE, Johnson SC, Agesa KM, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study. Lancet 2020; 395:200–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Ministry of Public Health of Thailand. Mortality of community-acquired sepsis Available at: http://data.ptho.moph.go.th/inspec/2561/inspec1/doc22dec/. Accessed 5 August 2019.
- 4. Fleischmann C, Scherag A, Adhikari NK, et al. ; International Forum of Acute Care Trialists Assessment of global incidence and mortality of hospital-treated sepsis. current estimates and limitations. Am J Respir Crit Care Med 2016; 193:259–72. [DOI] [PubMed] [Google Scholar]
- 5. Southeast Asia Infectious Disease Clinical Research Network. Causes and outcomes of sepsis in southeast Asia: a multinational multicentre cross-sectional study. Lancet Glob Health 2017; 5:e157–67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Bhattacharjee P, Edelson DP, Churpek MM. Identifying patients with sepsis on the hospital wards. Chest 2017; 151:898–907. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Kaukonen KM, Bailey M, Pilcher D, et al. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med 2015; 372:1629–38. [DOI] [PubMed] [Google Scholar]
- 8. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016; 315:762–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Freund Y, Lemachatti N, Krastinova E, et al. ; French Society of Emergency Medicine Collaborators Group Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department. JAMA 2017; 317:301–8. [DOI] [PubMed] [Google Scholar]
- 10. Raith EP, Udy AA, Bailey M, et al. ; Australian and New Zealand Intensive Care Society (ANZICS) Centre for Outcomes and Resource Evaluation (CORE) Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA 2017; 317:290–300. [DOI] [PubMed] [Google Scholar]
- 11. Kovach CP, Fletcher GS, Rudd KE, et al. Comparative prognostic accuracy of sepsis scores for hospital mortality in adults with suspected infection in non-ICU and ICU at an academic public hospital. PLoS One 2019; 14:e0222563. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Lamontagne F, Harrison DA, Rowan KM. qSOFA for identifying sepsis among patients with infection. JAMA 2017; 317:267–8. [DOI] [PubMed] [Google Scholar]
- 13. Sirijatuphat R, Sripanidkulchai K, Boonyasiri A, et al. Implementation of global antimicrobial resistance surveillance system (GLASS) in patients with bacteremia. PLoS One 2018; 13:e0190132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Lamy B, Dargère S, Arendrup MC, et al. How to optimize the use of blood cultures for the diagnosis of bloodstream infections? A state-of-the art. Front Microbiol 2016; 7:697. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Centers for Disease Control and Prevention. Hospital toolkit for adult sepsis surveillance. Available at: https://www.cdc.gov/sepsis/pdfs/Sepsis-Surveillance-Toolkit-Mar-2018_508.pdf. Accessed 15 January 2020. [Google Scholar]
- 16. Serafim R, Gomes JA, Salluh J, Póvoa P. A comparison of the quick-SOFA and systemic inflammatory response syndrome criteria for the diagnosis of sepsis and prediction of mortality: a systematic review and meta-analysis. Chest 2018; 153:646–55. [DOI] [PubMed] [Google Scholar]
- 17. Khwannimit B, Bhurayanontachai R, Vattanavanit V. Comparison of the performance of SOFA, qSOFA and SIRS for predicting mortality and organ failure among sepsis patients admitted to the intensive care unit in a middle-income country. J Crit Care 2018; 44:156–60. [DOI] [PubMed] [Google Scholar]
- 18. Gaini S, Relster MM, Pedersen C, Johansen IS. Prediction of 28-days mortality with sequential organ failure assessment (SOFA), quick SOFA (qSOFA) and systemic inflammatory response syndrome (SIRS) - a retrospective study of medical patients with acute infectious disease. Int J Infect Dis 2019; 78:1–7. [DOI] [PubMed] [Google Scholar]
- 19. Rudd KE, Seymour CW, Aluisio AR, et al. ; Sepsis Assessment and Identification in Low Resource Settings (SAILORS) Collaboration Association of the quick sequential (sepsis-related) organ failure assessment (qSOFA) score with excess hospital mortality in adults with suspected infection in low- and middle-income countries. JAMA 2018; 319:2202–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Song JU, Sin CK, Park HK, et al. Performance of the quick sequential (sepsis-related) organ failure assessment score as a prognostic tool in infected patients outside the intensive care unit: a systematic review and meta-analysis. Crit Care 2018; 22:28. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.