Abstract
Objective:
Early identification of sepsis is critical to improving patient outcomes. Impact of the new sepsis definition (Sepsis-3) on timing of recognition in the emergency department (ED) has not been evaluated. Our study objective was to compare time to meeting systemic inflammatory response syndrome (SIRS/Sepsis-2) criteria, Sepsis-Related Organ Failure Assessment (SOFA/Sepsis-3) criteria, and quick-SOFA (qSOFA) criteria using electronic health record (EHR) data.
Design:
Retrospective, observational study.
Setting:
The emergency department (ED) at the University of California, San Francisco.
Patients:
ED encounters between June 2012 and December 2016 for patients ≥18 years of age with blood cultures ordered, intravenous (IV) antibiotic receipt, and identification with sepsis via SIRS or SOFA within 72 hours of ED-presentation.
Interventions:
None.
Measurements and Main Results:
We analyzed timestamped EHR data from 16,612 encounters identified as sepsis by ≥2 SIRS criteria or a SOFA score ≥2. The primary outcome was time from ED-presentation to meeting ≥2 SIRS criteria, SOFA ≥2, and/or ≥2 qSOFA criteria. There were 9,087 patients (54.7%) that met SIRS-first a median of 26 minutes post-ED-presentation (IQR 0,109 minutes), with 83.1% meeting SOFA criteria a median of 118 minutes later (IQR 44,401 minutes). There were 7,037 patients (42.3%) that met SOFA-first a median of 113 minutes post-ED-presentation (IQR 60,251 minutes). qSOFA was met in 46.4% of patients a median of 351 minutes post-ED-presentation (IQR 67,1,165 minutes). Adjusted odds of in-hospital mortality were 39% greater in patients who met SIRS-first compared to those who met SOFA-first (OR 1.39, 95% CI 1.20 to 1.61).
Conclusions:
SIRS and SOFA initially identified distinct populations. Using SIRS resulted in earlier EHR sepsis identification in >50% of patients. Using SOFA alone may delay identification. Using SIRS alone may lead to missed sepsis presenting as acute organ dysfunction. Thus, a combination of inflammatory (SIRS) and organ dysfunction (SOFA) criteria may enhance timely EHR-based sepsis identification.
MeSH Keywords: Sepsis, Systemic Inflammatory Response Syndrome, Organ Dysfunction Scores, Electronic Health Records, Emergency Medicine, Critical Care
INTRODUCTION
Sepsis is a syndrome characterized by life-threatening organ dysfunction due to infection, with an estimated incidence of 6% in hospitalized patients and mortality of 15% in the United States (1). Early identification and treatment initiation for sepsis is critical to improving patient outcomes (2), as each hour delay in antibiotics is associated with an increase in absolute mortality of 0.3–1.8% (3). Recognizing sepsis in a timely manner is challenging because there is no gold standard for diagnosis, leading to reliance on consensus definitions to identify patients at highest risk.
Between 1992 and 2016, the consensus definitions of sepsis, Sepsis-1 (4) and Sepsis-2 (5), relied on systemic inflammatory response syndrome (SIRS) criteria to define sepsis. In 2016, to maximize the predictive validity of diagnostic criteria (6, 7), the new definition Sepsis-3 was proposed (8), which abandoned the emphasis on SIRS in favor of using the Sequential [Sepsis-Related] Organ Failure Assessment (SOFA) score to characterize organ dysfunction (9). Sepsis-3 also introduced the quick-SOFA (qSOFA) score to screen for non-intensive care unit (ICU) patients at greatest risk of poor sepsis outcomes (8, 10).
Electronic health record (EHR) data have been incorporated into automated sepsis alerts to identify patients and initiate swift treatment in those who are at risk for sepsis (11, 12). In addition, EHR data have been used to conduct epidemiologic studies of sepsis (1, 13, 14), including the analyses that informed the development of Sepsis-3 (8). Little work has been done in the emergency department (ED) setting to compare how the new definition performs against prior definitions at identifying sepsis in a timely manner.
Our primary objective was to compare the timing of sepsis identification via EHR data using SIRS, SOFA, qSOFA, and their components in patients presenting to an academic ED. In addition, we explored the association between first definition met and mortality.
MATERIALS AND METHODS
We conducted a retrospective cohort study of patients presenting to the University of California San Francisco (UCSF) Helen Diller Medical Center at Parnassus Heights ED with suspected sepsis between June 1, 2012 and December 31, 2016. Our institution is an 800 bed academic teaching hospital with approximately 30,000 ED encounters per year. The study was approved with a waiver of informed consent by the UCSF Human Research Protection Program.
Data Source:
Our institution uses an Epic-based EHR platform (Epic 2017, Epic Systems Corporation, Verona, Wisconsin) for clinical care that was implemented on June 1, 2012. All study data elements were obtained from Clarity, the relational database that stores Epic’s inpatient data.
Study Population:
The study included encounters for patients ≥18 years of age with blood cultures ordered, intravenous (IV) antibiotic receipt, and identification with sepsis via SIRS or SOFA within 72 hours of ED-presentation. We used orders for blood cultures and IV antibiotic receipt as a marker of suspicion of severe infection, enriching our patient sample for those likely presenting with sepsis, including those with possible delayed diagnosis. Repeat encounters were treated independently in our analysis.
Data Collection:
Sepsis identification was defined as meeting ≥ 2 SIRS criteria or having a SOFA score ≥ 2 based on vital signs, clinical assessments, and laboratory values collected during the index ED encounter along with timestamps from the EHR (Supplemental Digital Content- Table 1) within 72 hours of ED-presentation. All qSOFA component timestamps were collected during the first 72 hours following ED-presentation. For SIRS, SOFA and qSOFA, missing assessments were considered normal. Index blood culture results were collected and dates and timestamps of mechanical ventilation, fluid, vasopressor, and antibiotic administration were obtained for the duration of the encounter.
Demographic data were extracted including race, ethnicity, primary language, age, sex, insurance status, and ED/hospital discharge disposition. All International Classification of Diseases (ICD)-9/10 diagnosis codes were pulled from Clarity billing tables. We used the Elixhauser comorbidity groupings (15) of ICD-9/10 codes present on admission to identify pre-existing comorbidities and underlying organ dysfunction. To estimate burden of comorbid illnesses, we calculated the validated van Walraven comorbidity index (16) which provides an estimated risk of in-hospital death based on documented Elixhauser comorbidities. Admission level of care (acute, stepdown, or intensive care) was collected for inpatient admissions to assess initial illness severity (17). Discharge disposition and in-hospital mortality was also collected.
Statistical analysis:
For the primary analysis, we calculated the exact time from ED-presentation to meeting ≥ 2 SIRS criteria, to achieving a SOFA score of ≥ 2, and to meeting ≥ 2 qSOFA criteria. We compared the differences in time to meeting sepsis definitions using the Mann-Whitney rank-sum test. Chi-squared tests were used to evaluate associations between the first definition met, SIRS versus SOFA, and categorical variables. Mann-Whitney rank-sum tests were used to evaluate associations between the first definition met and continuous variables. Associations were considered statistically significant at the p<0.05 level. All analyses were conducted using standard programs in Stata 14.2 Special Edition (StataCorp, College Station, TX). For the secondary analysis, exact time from ED-presentation to meeting each SIRS, SOFA, and qSOFA component was calculated for all components available, regardless of the definition met. We summarized the time from ED-presentation to meeting each component stratified by the first definition met. Finally, in order to explore the association between first definition met and mortality, we conducted a multivariate logistic regression controlling for baseline patient demographic and clinical characteristics.
We performed four subgroup analyses, describing the timing of sepsis identification limited to patients with 1) a validated sepsis discharge diagnosis code (18), 2) a positive blood culture (excluding Staphylococcus epidermidis), 3) excluding patients who met SIRS criteria alone within 72 hours of ED-presentation, and 4) excluding SOFA points associated with bilirubin for those with chronic liver disease and excluding SOFA points associated with creatinine for those with chronic kidney disease based on Elixhauser comorbidity status present on admission.
RESULTS
We identified 16,612 patients who received IV antibiotics and were identified as possible sepsis via SIRS or SOFA criteria within 72 hours of ED-presentation (Figure 1). Frequency of missing components of SIRS, SOFA, and qSOFA can be found in Supplemental Digital Content-Table 2.
Figure 1. Identification of the study population.
All individuals ≥ 18 years of age who presented to the UCSF Emergency Department between June 1, 2012 and December 31, 2016 were included in the source population. We excluded those who did not receive IV antibiotics and who were not identified as sepsis by SIRS or SOFA.
Comparison of Timing to First Definition
Figure 2 provides a timeline for the median time to sepsis identification stratified by the first definition met. Time to first sepsis definition was significantly shorter for the 9,087 patients (54.7%) meeting SIRS-first compared to the 7,037 patients (42.3%) meeting SOFA-first (p<0.001). The median time from ED-presentation to sepsis identification was 26 minutes (IQR 0, 109) for patients meeting SIRS-first and 113 minutes (IQR 60, 251) for patients meeting SOFA-first. Among patients who met SIRS-first, 83.1% (n=7,550) subsequently met SOFA criteria at a median of 144 minutes post-ED-presentation. Among those who met SOFA-first, 67.7% (n=4,763) subsequently met SIRS at 347 minutes post-ED-presentation. Finally, those who met SIRS/SOFA simultaneously did so at a median of 106 minutes post-ED-presentation (IQR 51, 201). The 488 individuals that met SIRS/SOFA simultaneously were excluded from all subsequent comparisons presented.
Figure 2. Timeline for meeting SIRS and SOFA for the full study population (n=16,612).
Median time to identification of sepsis via SIRS or SOFA is displayed in this figure. The population is stratified by the first definition met, and when applicable, the median time to meeting the second definition is displayed.
The subgroup analyses limited to patients with a sepsis diagnostic code (n=5,004) and with a positive blood culture (n=873) exhibited similar patterns to the full cohort (Figure 3). When patients who met SIRS but never SOFA (n=1,537) were excluded, the timeline pattern was also unchanged (Supplemental Digital Content-Figure 1). Finally, when bilirubin and creatinine SOFA points were excluded for patients with chronic liver and/or renal disease the timeline pattern was again similar to the full cohort (Supplemental Digital Content-Figure 2).
Figure 3. Timeline for meeting SIRS and SOFA among patient subgroups.
In Panel A, the median time to identification of sepsis via SIRS or SOFA is displayed for the 5,004 patients (31.0%) that received a validated sepsis discharge diagnosis code. In Panel B, the median time to identification of sepsis via SIRS or SOFA is displayed for the 873 patients (5.4%) with positive blood cultures. The population is stratified by the first definition met, and when applicable, the median time to meeting the second definition is displayed.
Among the cohort of 16,124 meeting SIRS-first or SOFA-first, qSOFA was met first in only 274 encounters (1.7%). A total of 46.4% (n=7,487) met qSOFA criteria within the 72 hour follow-up period, a median of 351 minutes post-ED-presentation (IQR 67, 1165) which was significantly later than the cohort’s time to ever meeting SIRS (median 351 v. 83 minutes, p < 0.001) or time to ever meeting SOFA (median 351 v. 131 minutes, p < 0.001) (Supplemental Digital Content- Figure 3). qSOFA was met by 64.3% of the encounters that received a validated sepsis discharge code (n=3,217) and 61.2% of the encounters that included a positive blood culture (n=534). Supplemental Digital Content-Table 3 provides a summary of the time to first measurement and time to first qualifying measurement for qSOFA components.
Time from ED-presentation to meeting each component of SIRS, SOFA, and qSOFA for all components available are summarized in Figure 4 and stratified by first definition met. Heart rate was the first component met for both groups at a median of 0 minutes post-ED-presentation for the SIRS-first group and at 1 minute post-ED-presentation for the SOFA-first group. Both groups were found to have qualifying creatinine and bilirubin values at a median of approximately 2 hours post-ED-presentation. Qualifying vasopressor receipt occurred at a median of 482 minutes post-ED-presentation in those meeting SIRS-first and at a median of 367 minutes for those meeting SOFA-first.
Figure 4. Timeline for meeting each component of SIRS, SOFA, and qSOFA.
The median time to achieving each component of SIRS, SOFA, and qSOFA is displayed in this figure. The population is stratified by the first definition met.
Still excluding those who met SIRS/SOFA simultaneously, 13,850 patients met SIRS (85.9%), 14,587 patients met SOFA (90.5%), and 7,487 patients met qSOFA (46.4%) within 72 hours of ED-presentation. There were 1,537 patients who met SIRS but never SOFA (11.1%) and 2,274 patients who met SOFA but never SIRS (15.6%).
Comparison of Demographics, Clinical Characteristics, and Outcomes
Patients identified by SIRS-first were younger (59 years versus 65 years of age at admission, p<0.0001), more likely to be female (47.9% v. 46.3%, p = 0.042), less likely to be white (44.9% v. 47.0%, p=0.006), and less likely to have Medicare payer type (47.6% v. 60.7%, p<0.001) (Table 1). Those identified by SIRS-first were less likely to have chronic renal failure (16.6% v. 29.4%, p<0.001) or chronic liver disease (10.4% v. 18.2%, p<0.001), had a lower van Walraven comorbidity index (median 7 v. 10, p<0.001), and were more likely to be admitted to higher inpatient levels of care. Time from ED-presentation to IV antibiotic receipt was significantly shorter for those identified by SIRS-first (median 2.7 v. 4.1 hours, p<0.001).
Table 1.
Demographic Characteristics and Outcomes of 16,124 Subjects who met SIRS-first or SOFA-first
| Variable | SIRS-first n=9,087 | SOFA-first n=7,037 | p-value |
|---|---|---|---|
| Age at presentation (median, interquartile range) | 59 (44, 73) | 65 (54, 79) | <0.0001 |
| Female | 4,352 (47.9) | 3,257 (46.3) | 0.042 |
| Race | 0.006 | ||
| White | 4,077 (44.9) | 3,310 (47.0) | |
| Asian | 2,131 (23.5) | 1,507 (21.4) | |
| Black | 1,640 (18.1) | 1,296 (18.4) | |
| Other | 1,239 (13.6) | 924 (13.1) | |
| Payer type | <0.0001 | ||
| Medicare | 4,325 (47.6) | 4,274 (60.7) | |
| Medi-Cal | 2,324 (25.6) | 1,505 (21.4) | |
| Commercial | 2,232 (24.6) | 1,160 (16.5) | |
| Other/Self-Pay | 206 (2.3) | 98 (1.4) | |
| Chronic renal failurea | 1,512 (16.6) | 2,067 (29.4) | <0.0001 |
| Chronic liver diseasea | 942 (10.4) | 1,280 (18.2) | <0.0001 |
| Cancera | |||
| Lymphoma | 326 (3.6) | 210 (3.0) | 0.034 |
| Solid tumor | 963 (10.6) | 741 (10.5) | 0.890 |
| Metastatic cancer | 948 (10.4) | 617 (8.8) | <0.0001 |
| van Walraven Comorbidity Index value (median, interquartile range) | 7 (0, 15) | 10 (4, 17) | <0.0001 |
| Admission level of care | <0.0001 | ||
| Discharged from ED | 420 (4.6) | 189 (2.7) | |
| Admission to outside facility | 129 (1.4) | 72 (1.0) | |
| Acute care | 4,930 (54.3) | 4,386 (62.3) | |
| Stepdown care | 2,225 (24.5) | 1,426 (20.3) | |
| Intensive care | 1,383 (15.2) | 964 (13.7) | |
| Year of ED encounterb | 0.359 | ||
| 2012 | 964 (10.6) | 694 (9.9) | |
| 2013 | 1,936 (21.3) | 1,449 (20.6) | |
| 2014 | 1,952 (21.5) | 1,541 (21.9) | |
| 2015 | 2,164 (23.8) | 1,718 (24.4) | |
| 2016 | 2,071 (22.8) | 1,635 (23.2) | |
| Time to antibiotics in hours (median, interquartile range) | 2.7 (1.5, 4.9) | 4.1 (2.3, 7.9) | <0.0001 |
| Positive blood culture | 564 (6.2) | 309 (4.4) | <0.0001 |
| Receipt of validated sepsis discharge codec | 3,383 (37.2) | 1,621 (23.0) | <0.0001 |
| Site of infection coded on discharge | <0.0001 | ||
| None coded | 3,128 (34.4) | 2,660 (37.8) | |
| Skin and soft tissue infection | 592 (6.5) | 563 (8.0) | |
| Urinary tract infection | 523 (5.8) | 683 (9.7) | |
| Pneumonia | 1,237 (13.6) | 863 (12.3) | |
| Bacteremia | 837 (9.2) | 437 (6.2) | |
| Sepsis | 412 (4.5) | 208 (3.0) | |
| Otherd | 408 (4.5) | 426 (6.1) | |
| Multiple sites | 1,950 (21.5) | 1,197 (17.0) | |
| In-hospital mortality | 579 (6.4) | 479 (6.8) | 0.268 |
| ICU length of stay among survivors (median, interquartile range)e | 2.8 (1.7,5.1) | 3.0 (1.8,5.1) | 0.138 |
| Vasopressor receipt | 675 (7.4) | 560 (8.0) | 0.210 |
| Receipt of mechanical ventilation | 741 (8.2) | 606 (8.6) | 0.302 |
Defined using the Elixhauser comorbidity ICD-9/10 grouping method
Study period was June 1, 2012-December 31, 2016
Codes included ICD-10: A02.1, A22.7, A26.7, A32.7, A40.0, A40.1, A40.3, A40.8, A40.9, A41.01, A41.02, A41.1, A41.2, A41.3, A41.4, A41.50, A41.51, A41.52, A41.53, A41.59, A41.81, A41.89, A41.9, A42.7, A54.86, B37.7, R65.20, R65.2; ICD-9: 038.0, 038.1, 038.11, 038.12, 038.19, 038.2, 038.3, 038.4, 038.41, 038.42, 038.43, 038.44, 038.49, 038.8, 038.9, 995.91, 995.92.
Other sites included osteomyelitis, meningitis, peritonitis, Clostridium difficile infection, biliary infection, and device related infection
There were 1,636 SIRS-first and 1,209 SOFA-first patients who survived and were ever admitted to the ICU.
There was no statistically significant difference on bivariate analysis in in-hospital mortality comparing those who met SIRS-first to SOFA-first (6.4% v. 6.8%, p=0.268, Table 1). However, after adjustment for time to antibiotics, baseline SOFA score, age, race, ethnicity, and cancer, renal failure, and/or liver failure present on admission, patients who met SIRS-first had a 39% increased odds of in-hospital mortality compared to those who met SOFA-first (odds ratio 1.39, 95% confidence interval 1.20 to 1.61). The same relationship between SIRS-first and mortality was observed when time to antibiotics was excluded from the model (odds ratio 1.39, 95% confidence interval 1.20 to 1.60).
There was no statistically significant difference on bivariate analysis in the rate of vasopressor receipt (7.4% v. 8.0%, p=0.210), mechanical ventilation (8.2% v. 8.6%, p=0.302), or ICU length of stay among survivors (median 2.8 days v. 3.0 days, p=0.138) when comparing those who met SIRS-first to those who met SOFA-first (Table 1). In addition, there was no statistically significant difference in mortality (1.0% v. 1.25%, p=0.636), vasopressor receipt (0.7% v. 0.4%, p=0.373), or mechanical ventilation (0.8% v. 1.2%, p=0.636) on bivariate analysis when comparing the patients who met SIRS but never SOFA (n=1,537) to those who met SOFA but never SIRS (n=2,274).
Demographic and clinical characteristics for all combinations of first definition(s) met, including qSOFA, for all 16,612 patients identified can be found in Supplemental Digital Content- Table 4. Patients who met qSOFA-first (n=1,107), with or without SIRS or SOFA, were more likely to be transferred from the ED to the ICU (35.2% v. 13.2%, p<0.0001) and more likely to die in the hospital (16.4% v. 5.9%, p<0.0001) than those who met SIRS and/or SOFA-first. Patients who ever met qSOFA within 72 hours (n=7,487) were more likely to die (12.5% v. 1.4%, p<0.0001), more likely to be admitted directly to the ICU (29.8% v. 1.6%, p<0.0001), more likely to receive vasopressors (16.5% v. 1.0%, p<0.0001), and more likely to be mechanically ventilated (16.9% v. 1.0%, p<0.0001) when compared to those who never met qSOFA.
DISCUSSION
In this retrospective cohort study of 16,612 patients with suspected sepsis, SIRS identified patients with possible sepsis a median of 118 minutes before they met the definition of sepsis by SOFA. Using SIRS alone led to a 4-hour delay in identification among those who met SOFA-first. Finally, only 46.4% of the cohort met qSOFA, up to 5 hours after SIRS and SOFA were met.
Using SOFA alone in the ED may lead to a significant delay in the administration of antibiotics, the most important clinical intervention for reducing mortality from sepsis (3, 19). In this study, subjects who met SIRS-first met criteria more rapidly than those who met SOFA-first, by a median difference of 87 minutes. In addition, 83.1% of those who met SIRS-first did eventually meet SOFA criteria, but a median of 118 minutes later. Antibiotics were administered more rapidly to patients who met SIRS-first compared to those who met SOFA-first. A recent study by Liu et al. demonstrated that each hour delay in antibiotic receipt increased in-hospital mortality between 0.3% and 1.8% depending on sepsis severity (3). Applying these proportions to our study, if starting antibiotics is delayed until a patient meets the SOFA definition of sepsis, absolute mortality could increase by as little as 0.6% or as much as 7.2%. Our adjusted analysis which controlled for baseline patient level characteristics and comorbidities demonstrated that those who met SIRS-first experienced a 39% increased odds of mortality when compared to those who met SOFA-first. Given that the vast majority of patients who met SIRS-first went on to develop organ dysfunction and those who met SIRS-first experienced worse outcomes, SIRS still has significant clinical value when identifying the earliest opportunity to initiate treatment for sepsis.
Using SIRS alone may also lead to delays in sepsis identification and missed identification among patients with organ dysfunction. In our cohort, using SIRS would have delayed sepsis identification by 4 hours among those who met SOFA-first. In addition, 16% of patients who met SOFA never met SIRS within 72 hours of ED-presentation. Our results are consistent with a retrospective cohort study of adult ICU patients which demonstrated that requiring two or more SIRS criteria missed 1 in 8 patients with infection, organ failure, and substantial mortality (20). Given the potential for delays and missed identification, the use of SIRS alone would not be sufficient to screen for patients with sepsis.
All subgroup analyses revealed similar timeline patterns to those seen in the full cohort. The initial analysis used the definition of suspected sepsis; however, when we restricted the timeline analysis to those with a validated sepsis discharge code or those with positive blood cultures, there were no differences. Given that Sepsis-3 requires organ dysfunction for sepsis identification, we also excluded those who only met SIRS but this did not alter the timeline.
The population of patients identified with sepsis could be too limited if SOFA alone is utilized for early identification of sepsis. Given that Sepsis-3 did not incorporate baseline organ dysfunction but instead used the data gathered during the index encounter, patients who present with chronic organ dysfunction may be more likely to meet SOFA criteria even though they are not clinically septic (21). In our study, SOFA identified patients who were older and more likely to have underlying organ dysfunction present on admission.
While previous studies have reported how often SIRS or SOFA criteria were met within epochs of time since presentation and often within intensive care settings (3, 14, 22), data are lacking on the timing of sepsis identification comparing consensus definitions. One study conducted by Haydar et al. among 200 patients discharged with the Diagnosis Related Grouping for sepsis who were treated with antibiotics in the ED for suspected infection reported that the median time from ED-presentation to meeting SIRS was 12 minutes and qSOFA was 29 minutes (23). Our much larger study includes SOFA and replicates the finding that SIRS identifies sepsis patients quickly after ED-presentation. Together, these findings provide a compelling argument that changing screening criteria for suspected sepsis could delay EHR identification.
Data continue to emerge supporting the concept that sepsis is a heterogeneous syndrome with identifiable clinical phenotypes (24, 25). In addition, the relationship between mortality and organ dysfunction varies by organ system (26) and our work has demonstrated that the evolution of organ dysfunction over time varies by organ system as well. When evaluating time to meeting SIRS and SOFA components, as expected, vital signs-based SIRS components were met within minutes of ED-presentation while laboratory-based SOFA components were met later. Laboratory lag time explains some of the delay in EHR identification via SOFA, given that three of the SOFA criteria (bilirubin, creatinine, and platelet counts) are subject to reporting delay. However, our data also revealed that non-laboratory-based components of SOFA, such as hypotension and abnormal GCS, occurred hours post-ED-presentation, indicating that these markers of organ dysfunction developed over time. Abnormal temperature occurred last in the SOFA-first group, perhaps identifying patients unable to mount an immune response to infection due to underlying disease or age. Organ dysfunction manifests in many ways and timing Interestingly, elevated heart rate was the first qualifying component in both the SIRS-first and SOFA-first populations. Heart rate may be a component that should be conserved across definitions to identify the greatest number of patients eligible for intervention.
Only 46% of our study cohort ever met qSOFA within 72 hours of ED-presentation, which is similar to findings from other studies (23, 27, 28), but it was more commonly met in the populations that received a validated sepsis discharge code and that had a positive blood culture. In comparison, 86% met SIRS criteria and 91% met SOFA criteria within 72 hours of ED-presentation. qSOFA was also met significantly later than SIRS or SOFA, signifying that it may not be the most appropriate tool to use for triggering identification for timely treatment. It should be noted that GCS was missing for 25% of our study cohort, which may have affected the prevalence of patients meeting qSOFA at our institution. While not intended to be used as a screening tool for sepsis in the general population, the use of qSOFA was recommended by Sepsis-3 to identify patients with suspected infection who were at highest risk of poor outcomes from sepsis in the non-ICU population (8). Our study did demonstrate that patients who met qSOFA were more likely to experience poor outcomes, including in-hospital mortality, initial admission to the ICU, receipt of vasopressors, and receipt of mechanical ventilation. Taken together, qSOFA may not be the most appropriate tool to utilize for an EHR sepsis recognition alert in the ED but it is a useful indicator for identifying patients who would benefit most from intensive monitoring.
It is clinically relevant to determine the timing of sepsis identification based on EHR-derived implementations of consensus definitions in order to improve timeliness of treatment, halt progression of organ failure, and decrease the risk of mortality for patients with sepsis. Given the volume of patients seen in the ED, automating sepsis surveillance can lead to improved compliance with sepsis treatment guidelines and improved patient survival. At our institution, after implementation of an EHR SIRS-based severe sepsis alert in the ED, time to receipt of antibiotics decreased significantly from a mean of 61.5 minutes in the pre-implementation period to 29 minutes in the post-implementation period (p<0.001)(11). Appropriate treatment with the sepsis intervention bundle at our institution was also associated with a 33% decrease in risk of mortality (29). In this study, we found that those who met SIRS-first received antibiotics more quickly than those who met SOFA-first (median of 2.7 v. 4.1 hours, p<0.001), a difference that may be driven by the presence of our surveillance alert.
There are limitations to our study. It is a single-center retrospective analysis, factors which could reduce the generalizability. However, the study does include a large and diverse patient population and spans several years. Our analysis relies on the accuracy of EHR timestamped data. While the majority of SIRS and SOFA components are automatically entered, vital signs data that appear in the EHR are linked to automated alerts and clinical decision making. The validity of the timestamp associated with these data elements should be evaluated prospectively. Given our study’s focus on a population with suspected infection, we are unable to assess the specificity of SIRS or SOFA criteria in the general ED population. However, we have utilized a definition which is consistent with prior sepsis EHR research, including the Sepsis-3 derivation studies (10, 13). Despite these limitations, our study uses “real-world” EHR data that are directly relevant to the development of automated EHR-based screening approaches.
CONCLUSIONS
Among adults presenting to an ED with suspected sepsis, time to sepsis recognition differed depending on the consensus definition applied. SIRS provided the earliest indication of sepsis in 57.4% of patients presenting with suspected sepsis. However, the remainder of patients were identified with SOFA-first (42.8%) or SIRS/SOFA simultaneously (2.9%). qSOFA was only met by 46.4% of the cohort and it occurred after patients were already identified via SIRS or SOFA. Patients who met SIRS-first had a 39% increase in odds of mortality compared to those who met SOFA-first. Given the results of this study, additional research is needed to confirm whether meeting SIRS or SOFA-first is associated with differential treatment and outcomes and to determine whether a definition that includes elements of both SIRS and SOFA would lead to more timely and complete EHR identification of sepsis.
Supplementary Material
Supplemental Figure 1. Timeline for meeting SIRS and SOFA, excluding those who do not meet SOFA. The median time to identification of sepsis via SIRS or SOFA is displayed for the 14,587 patients that met SOFA. The population is stratified by the first definition met, and when applicable, the median time to meeting the second definition is displayed.
Supplemental Figure 2. Timeline for meeting SIRS and SOFA, adjusting SOFA points for those with chronic renal disease and chronic liver disease. The median time to identification of sepsis via SIRS or SOFA is displayed. SOFA points associated with bilirubin were excluded for those with chronic liver disease and SOFA points associated with creatinine were excluded for those with chronic kidney disease based on Elixhauser comorbidity status present on admission. The population is stratified by the first definition met, and when applicable, the median time to meeting the second definition is displayed.
Supplemental Figure 3. Timeline for meeting qSOFA. The median time to achieving qSOFA is depicted for the 7,487 patients that met qSOFA and stratified by the first definition met.
ACKNOWLEDGEMENTS
The authors would like to acknowledge Nader Najafi, MD, Zachary Jacobs, MD, and Nicole Van Groningen, MD, for their contributions to this work.
Financial Support: Dr. Prasad was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number #A127552. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. MC Fang was supported by grant K24HL141354 from NHLBI, CS Calfee was supported by HL140026 from NHLBI and KN Kangelaris was funded by 1K23HL116800 from NHLBI. Data acquisition for this publication was supported by UCSF Academic Research Systems, and by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number UL1 TR001872. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of UCSF or the NIH.
Copyright form disclosure: Dr. Prasad’s institution received funding from National Center for Advancing Translational Sciences, National Institutes of Health (NIH), through grant number UL1 TR001872, and she received funding from Epidemiologist for EpiExcellence, LLC (consultant). Drs. Prasad, Fang, Abe-Jones, Matthay, and Kangelaris received support for article research from the NIH. Dr. Fang’s institution received funding from the National Heart, Lung, and Blood Institute (NHLBI) K24HL141354 and Patient-Centered Outcomes Research Institute. Dr. Abe-Jones disclosed that data acquisition for this publication was supported by UCSF Academic Research Systems, and by the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number UL1 TR001872. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of UCSF or the NIH. Dr. Calfee’s institution received funding from the NHLBI HL140026, GlaxoSmithKline, and Bayer and she received funding from Bayer, CSL Behring, Prometic, Roche/Genentech, and Quark Pharmaceuticals. Dr. Matthay’s institution received funding from the NIH/NHLBI, GlaxoSmithKline, Bayer, and a Department of Defense grant. Dr. Kangelaris’s institution received funding from NHLBI 1K23HL116800
REFERENCES
- 1.Rhee C, Dantes R, Epstein L, et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009–2014. JAMA 2017;318(13):1241–1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Kumar A, Roberts D, Wood KE, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med 2006;34(6):1589–1596. [DOI] [PubMed] [Google Scholar]
- 3.Liu VX, Fielding-Singh V, Greene JD, et al. The Timing of Early Antibiotics and Hospital Mortality in Sepsis. Am J Respir Crit Care Med 2017;196(7):856–863. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992;101(6):1644–1655. [DOI] [PubMed] [Google Scholar]
- 5.Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med 2003;31(4):1250–1256. [DOI] [PubMed] [Google Scholar]
- 6.Raith EP, Udy AA, Bailey M, et al. Prognostic Accuracy of the SOFA Score, SIRS Criteria, and qSOFA Score for In-Hospital Mortality Among Adults With Suspected Infection Admitted to the Intensive Care Unit. JAMA 2017;317(3):290–300. [DOI] [PubMed] [Google Scholar]
- 7.Seymour CW, Angus DC. Clinical Criteria to Identify Patients With Sepsis--Reply. JAMA 2016;316(4):454. [DOI] [PubMed] [Google Scholar]
- 8.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315(8):801–810. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a New Definition and Assessing New Clinical Criteria for Septic Shock: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315(8):775–787. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315(8):762–774. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Narayanan N, Gross AK, Pintens M, et al. Effect of an electronic medical record alert for severe sepsis among ED patients. Am J Emerg Med 2016;34(2):185–188. [DOI] [PubMed] [Google Scholar]
- 12.Brandt BN, Gartner AB, Moncure M, et al. Identifying severe sepsis via electronic surveillance. Am J Med Qual 2015;30(6):559–565. [DOI] [PubMed] [Google Scholar]
- 13.Rhee C, Jentzsch MS, Kadri SS, et al. Variation in Identifying Sepsis and Organ Dysfunction Using Administrative Versus Electronic Clinical Data and Impact on Hospital Outcome Comparisons. Crit Care Med 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Johnson AEW, Aboab J, Raffa JD, et al. A Comparative Analysis of Sepsis Identification Methods in an Electronic Database. Crit Care Med 2018;46(4):494–499. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Elixhauser A, Steiner C, Harris DR, et al. Comorbidity measures for use with administrative data. Med Care 1998;36(1):8–27. [DOI] [PubMed] [Google Scholar]
- 16.van Walraven C, Austin PC, Jennings A, et al. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care 2009;47(6):626–633. [DOI] [PubMed] [Google Scholar]
- 17.Prin M, Wunsch H. The role of stepdown beds in hospital care. Am J Respir Crit Care Med 2014;190(11):1210–1216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Dunatchik A, Semler MW, Rice TW, et al. Accuracy of the Centers for Medicare and Medicaid Services ICD-10-CM Codes in Identifying Sepsis Among Critically Ill Adults. In: American Thoracic Society 2017. Annual Meeting. p. A5016–A5016. [Google Scholar]
- 19.Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med 2017;376(23):2235–2244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kaukonen KM, Bailey M, Pilcher D, et al. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med 2015;372(17):1629–1638. [DOI] [PubMed] [Google Scholar]
- 21.Donnelly JP, Safford MM, Shapiro NI, et al. Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study. Lancet Infect Dis 2017;17(6):661–670. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Scheer CS, Kuhn SO, Fuchs C, et al. Do Sepsis-3 criteria facilitate earlier recognition of sepsis and septic shock? A retrospective cohort study. Shock 2018. [DOI] [PubMed] [Google Scholar]
- 23.Haydar S, Spanier M, Weems P, et al. Comparison of QSOFA score and SIRS criteria as screening mechanisms for emergency department sepsis. Am J Emerg Med 2017;35(11):1730–1733. [DOI] [PubMed] [Google Scholar]
- 24.Seymour CW, Kennedy JN, Wang S, et al. Derivation, Validation, and Potential Treatment Implications of Novel Clinical Phenotypes for Sepsis. JAMA 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bhavani SV, Carey KA, Gilbert ER, et al. Identifying Novel Sepsis Subphenotypes Using Temperature Trajectories. Am J Respir Crit Care Med 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Gupta T, Puskarich MA, DeVos E, et al. Sequential Organ Failure Assessment Component Score Prediction of In-hospital Mortality From Sepsis. J Intensive Care Med 2018:885066618795400. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Usman OA, Usman AA, Ward MA. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the Emergency Department. Am J Emerg Med 2018. [DOI] [PubMed] [Google Scholar]
- 28.Askim Å, Moser F, Gustad LT, et al. Poor performance of quick-SOFA (qSOFA) score in predicting severe sepsis and mortality - a prospective study of patients admitted with infection to the emergency department. Scand J Trauma Resusc Emerg Med 2017;25(1):56. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Prasad PA, Shea ER, Shiboski S, et al. Relationship Between a Sepsis Intervention Bundle and In-Hospital Mortality Among Hospitalized Patients: A Retrospective Analysis of Real-World Data. Anesth Analg 2017;125(2):507–513. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Figure 1. Timeline for meeting SIRS and SOFA, excluding those who do not meet SOFA. The median time to identification of sepsis via SIRS or SOFA is displayed for the 14,587 patients that met SOFA. The population is stratified by the first definition met, and when applicable, the median time to meeting the second definition is displayed.
Supplemental Figure 2. Timeline for meeting SIRS and SOFA, adjusting SOFA points for those with chronic renal disease and chronic liver disease. The median time to identification of sepsis via SIRS or SOFA is displayed. SOFA points associated with bilirubin were excluded for those with chronic liver disease and SOFA points associated with creatinine were excluded for those with chronic kidney disease based on Elixhauser comorbidity status present on admission. The population is stratified by the first definition met, and when applicable, the median time to meeting the second definition is displayed.
Supplemental Figure 3. Timeline for meeting qSOFA. The median time to achieving qSOFA is depicted for the 7,487 patients that met qSOFA and stratified by the first definition met.