A sepsis early warning system is associated with improved patient outcomes

Abstract

In a real-world implementation of a machine-learning (ML)-based sepsis early warning system (EWS), Adams et al.1, 2 found that timely provider response to an alert was associated with improved mortality, highlighting the potential utility of these systems in patient care.

In a real-world implementation of a machine-learning (ML)-based sepsis early warning system (EWS), Adams et al.^1,2 found that timely provider response to an alert was associated with improved mortality, highlighting the potential utility of these systems in patient care.

Main text

Despite being a leading cause of hospitalization and death worldwide, there are few available interventions to improve patient outcomes in sepsis.² Multiple studies have shown that earlier sepsis recognition and antibiotic administration is associated with improved survival,^3,4 but development of a standard best practice to achieve this has been elusive. One small randomized trial found that use of a sepsis early warning system (EWS) was associated with reduced mortality and length of stay (LOS) among intensive care unit (ICU) patients.⁵ For a syndrome where more than 100 clinical trials have failed,² EWS may thus be one tool to improve outcomes.

EWSs are part of a broader family of clinical decision support tools, whose use for sepsis dates back to the 1970s.⁶ Underlying models are most commonly developed in retrospective or administrative data, can be used for risk modeling or detection of clinical deterioration, and can vary from regression-based approaches to complex neural networks.^7,8 EWSs vary in intended clinical environment, with systems designed for emergency departments (EDs), ICUs, and surgical units, among others. While many models have been retrospectively described, few have been evaluated prospectively in a clinical setting.

In a recent issue of Nature Medicine, Adams and colleagues report the results of a prospective, multi-center, two-arm cohort study evaluating the association between prompt provider confirmation of an EHR-embedded sepsis alert and patient outcomes.¹ The alert system, targeted real-time early warning system (TREWS), is an ML-based system deployed within two academic and three community hospitals in the US. TREWS continuously monitors patient vitals, laboratory data, medication orders, and clinical documentation to generate a real-time sepsis risk score.⁹ When an alert occurs, providers can either dismiss it or proceed to a dedicated page with further information. There, the provider enters an evaluation either confirming suspected infection or indicating they do not believe new or worsening infection is present.

From TREWS deployment in 2018 through September 2020, there were 590,736 adult inpatient encounters across the five hospitals. Of these, 6,877 met inclusion criteria: they triggered TREWS alert no more than 1 h prior to admission or ED triage, met retrospective sepsis criteria, received antibiotics not prior to but within 24 h after the alert, and were not admitted directly to the ICU. Among these 6,877 encounters, 4,220 (61%) had their TREWS alert evaluated and confirmed as suspected sepsis within 3 h (study arm) and 2,657 (39%) did not (comparison arm).

The primary outcome was adjusted all-cause hospital mortality. Secondary outcomes included change in SOFA score in the 72 h after alert and post-alert hospital LOS among survivors. These outcomes were adjusted for admitting hospital and patient demographics, comorbidities, and severity of illness. No adjustment was made for provider-level features. Alert confirmation within 3 h was associated with lower mortality, with 14.6% unadjusted mortality in the study arm and 19.2% in the comparison arm; adjusted mortality risk difference was −3.3% (CI −5.1, −1.7%, p < 0.001). There were significant differences between groups in both secondary outcomes: adjusted change in SOFA score in the 72 h after alert was −0.3 points (CI −.4, −0.1, p = 0.001) and adjusted median post-alert LOS among survivors was −11.6 h (CI, −18.1, −5.0, p = 0.001).

This is the first study of this scale to prospectively demonstrate the potential for ML-based alert systems to identify patients with sepsis early and meaningfully improve outcomes. Strengths of the study include its size, multi-center design, and low risk of surveillance bias through use of a natural contemporaneous control group. Additionally, use of “human-machine teaming” to integrate clinician impression made TREWS feel like more of a tool for care, rather than simply an alert.⁹

Important questions remain. While the EWS may indeed have led to earlier treatment, and thus to improved patient outcomes, it is possible that prompt response to the alert was simply indicative of a more attentive or better resourced care team in which we would already expect improved patient outcomes. While the authors appropriately adjusted for patient-level variability, this provider variability is more difficult to account for in observational data. A future study in which implementation is randomized by site or provider, such as a step-wedge design, may be helpful for reducing this potential confounding. In addition, the alert system was triggered 42,089 times, but sepsis criteria were met in only 13,680 encounters.¹ While companion papers sought to understand factors to increase alert usage, further work in both model refinement and implementation science is needed to ensure alert usage is maintained over time and not impacted by alert fatigue.^9,10

As use of EWS expands, there will be challenges in deployment into EHR systems that vary substantially between hospitals. The authors successfully implemented TREWS across 5 sites with a shared EHR, but there is not yet a straightforward way to scale implementation across dissimilar systems. Continued advancement of Health Level 7 (HL7) and Fast Healthcare Interoperability Resource (FHIR) standards offers exciting potential for the future development of Substitutable Medical Applications and Reusable Technologies (SMART).

In summary, real-time EWS such as the one created by Adams and colleagues have potential to improve patient outcomes by prompting earlier and more appropriate clinical care. Future work is needed to evaluate these systems in large-scale randomized trials, to better understand the human factors driving adoption and action at the provider level, and to develop technological frameworks to facilitate broader clinical implementation.