Machine learning and sepsis: on the road to revolution

With each passing day, the drumbeat of the advancing Big Data and machine learning revolution grows louder.(1) Already having conquered previously intractable tasks like interpreting complex images or voices, driving autonomous vehicles, and winning abstract strategy games, computationally-driven advances are producing disruptions across a wide swath of industries. These advances have arisen from a confluence of three major trends: rapid expansions in the availability of large-scale data; exponential increases in the accessibility of computational power; and critical progress in machine learning methods.

Naturally, many see healthcare as the next major industry ripe for innovation. And, as the National Academy of Medicine has highlighted in numerous landmark reports, advances in the use of data and information technology will be critically important for addressing systemic failures in US healthcare quality, safety, and value.(2) In particular, recent successes in clinical image recognition and ongoing advances in leveraging large-scale –omics data highlight the great promise of machine learning applications in the biomedical and healthcare enterprise.(3, 4)

In this issue of Critical Care Medicine, Churpek and colleagues further expand on the promise of machine learning approaches in sepsis: the single most common, costly, and deadly cause of US hospitalization.(5) Extending their prior work in developing and validating the electronic Cardiac Arrest Risk Triage (eCART) score among hospitalized ward patients – the score incorporates granular vital signs, laboratory values, and patient demographics extracted from electronic medical records (EMR) data(6) – they demonstrate that eCART scores uniformly provided the highest discrimination among inpatients with varying degrees of “suspected infection” for predicting a composite adverse outcome of intensive care unit (ICU) transfer or death within 48 hours. As is critical for innovation in this domain, their work capitalized on a large and highly granular dataset including >53,000 infected patients as well as access to the computational resources needed to develop and deploy a random forest-based machine learning approach to score development and determination.

Yet, even while the drumbeat of the machine learning revolution portends great promise, it is also accompanied by potential pitfalls.(7, 8) In this case, the discrimination of the eCART score was uniformly better than all other risk scores (median c-statistic, 0.73), however, it was only modestly better than the 7-element, vital sign-based NEWS score (National Early Warning Score; median c-statistic, 0.71). Thus, while the eCART was statistically superior to the NEWS scores, it remains unclear what the clinical impact of this advantage would be particularly since the eCART score also demonstrated modestly lower specificity in some scenarios. Additionally, the comparisons of the eCART versus other scores were repeated in a sample drawn from the same setting in which the score was derived. Whether eCART shows better performance in external validation cohorts remains unclear.

Beyond the statistical development of early warning scores, another major challenge remains identifying the appropriate interventions that should follow a high-risk alert. In a randomized trial deploying a real-time early warning score with excellent discrimination, Kollef and colleagues found that the transmitted alerts in the intervention arm neither decreased mortality nor increased the rate of ICU transfers. The alerts did result in a shorter overall hospital length of stay.(9) Given the impressive performance characteristics of the eCART score, we look forward to a comprehensive evaluation of its impact on clinical outcomes and, in particular, a detailed description of the efferent intervention arm. This will greatly improve our understanding of which elements of implementation are most likely to drive improved outcomes.

Finally, Churpek and colleagues found that eCART performance remained consistent even as the inclusion cohort criteria varied from minimal evidence of suspected infection (i.e., any culture ordered) to considerably stronger evidence (i.e., blood culture orders and intravenous antibiotics for at least four out of seven living days). Their findings confirm that the predictors conferring prognostic risk (i.e., the likelihood of experiencing an adverse event) in hospitalized patients are common across many conditions. At the same time, the determinants of predictive risk (i.e., the likelihood of responding to a specific therapy) are likely to vary across patient subgroups, particularly in heterogeneous syndromes like sepsis. Thus, future work incorporating biomolecular data for predictive enrichment will be critical to target therapies to specific sepsis endotypes who otherwise share the same general risk of adverse events.(10) These factors are certain to gain prominence in future consensus definitions of sepsis and will require carefully balancing the need for increasing predictive validity compared with preserving content validity, reliability, and low measurement burden.(11)

As Churpek and colleagues have shown, the machine learning revolution continues to draw closer, however, more work is needed to ensure that the future is one filled with promise and not pitfalls.