Risky Business: Predicting patient mortality at a glance

Abstract

Clinicians are constantly forecasting patient trajectories to make critical point of care decisions intended to influence clinical outcomes. Little is known, however, about how providers interpret mortality risk against validated scoring systems. This research aims to understand how providers forecast mortality specifically for that of patients with sepsis. Defined as life-threatening organ dysfunction caused by a dysregulated host response to infection, sepsis is commonly hard to diagnose, progresses rapidly, and lacks a “gold standard” test. Participants were nurses and doctors from the general medical and surgical floors of six different hospitals. Each was presented with ten different patient cases, categorized into low and high severity sepsis, and were asked about care decisions, along with estimations of mortality risk. The resulting data provides a unique look into the differences of risk forecasting between profession and patient severity.

INTRODUCTION

In many hospital environments providers and nurses rely on clinical gestalt and heuristics to predict a patient’s trajectory (Pommerening, et al., 2015). Understanding a patient’s risk of in-hospital mortality is critical to selecting proper therapies, monitoring resource utilization, and appropriately selecting when to discharge a patient (Ruger, Lewis, & Richter, 2007). Uncertainty permeates all facets of medicine but is especially present in understanding a patient’s risk of outcomes associated with sepsis, a disease that lacks universal agreement of its prognosis, best therapies, and even its very definition. Sepsis is defined as the body’s overwhelming and life-threatening response to infection, and if left untreated, it can lead to tissue damage, organ failure, and death (Dellinger, et al., 2013). It is commonly hard to diagnose, progresses rapidly, and lacks a “gold standard” test. An hour delay of antibiotics results in an 8% mortality risk (Kumar, et al., 2006) and is the most expensive condition associated with in-hospital stay (Pfuntner, Wier, & Steiner, 2006), making it critical to identify and treat sepsis as early into its progression as possible (Iwashyna, et al., 2014).

This relatively common, very expensive, and too-frequently life-threatening condition affects patients of all ages, comorbidities, infection types, and health statuses, contributing to a heterogeneity that makes accurate judgments of a patient’s risk of in-hospital mortality very difficult (Marshall, 2005; Iskander, et al., 2013). While many advances in scoring systems and predictive models have been made in recent years, providers often rely on their subjective impressions of a patient’s clinical appearance to make care decisions (Fine, et al., 1997). Accurate risk estimates of patient mortality are especially vital when it comes to sepsis, due to the dramatically steep progression of infection to septic shock, many care decisions need to be made expeditiously.

To make judicious and reliable point of care decisions, nurses and physicians must continuously forecast a patient’s clinical trajectory. Accurate and conscientious predictions of mortality risk have the potential to dramatically alter a patient’s clinical care and, ultimately, their outcome. The purpose of this study is to gain insight to nurse and physician in-hospital mortality prediction of patients presenting with sepsis on the general medical and surgical floors during a simulated use evaluation. Insights from findings have the potential to provide evidence to the cognitive processing of clinicians and subsequently influence cognitive support application development for health information technology.

METHODS

Participants

A cross-sectional usability study was conducted at six healthcare systems using a crossover experimental design. A total of 76 clinicians, physicians and nurses, participated in the study, recruited by an on-site coordinator from each site. Participants represent a cross-section of expert and novice clinical personnel from the general medical and surgical floor.

Patient Scenarios and Presentation

From two sepsis registries from two different Emergency Departments (ED), researchers collected ten clinical patient scenarios derived from real patient cases and outcomes. Because national data shows a higher mortality rate for patients that develop sepsis on the medical or surgical units rather than the ED, patient data was restricted to those conditions. Using the SOFA scoring system (Vincent, et al., 1996), patient cases were categorized into five low and five high severity sepsis scenarios. The presentation of the scenarios included variables from the first 12 hours of the patient’s stay, where “time 0” was considered the time of triage. Data elements presented to participants included patient demographics, operational details (e.g., arrival time and discharge disposition), medical history, assessments (e.g., neurological), vital signs, and outcome events (e.g., transfers in care, code blue, death), to preserve the integrity of the scenarios. All patient information was deidentified.

The patient scenarios were uploaded into a simulated electronic health record environment. Participants were given approximately three minutes to review the clinical scenarios before researchers asked questions about their clinical impressions including risk of mortality estimates (on a scale of 0–100%). All risk mortality estimates were then classified by participants as being a “low,” “moderate,” or “high” risk of death.

RESULTS

The collected data was comprised of two participant cohorts—nurses (n=46) and doctors (n=30)—low and high severity sepsis cases, numerical risk estimates (0–100) of the presented patient’s in-hospital mortality, and further categorization of that estimate into low (1), moderate (2), or high (3) risk.

Figure 1 shows the density function and boxplots of the risk projection (0–100) compared between nurses and doctors for each risk category (1, 2, or 3). Nurses consistently estimated higher levels of risk than doctors. Figure 2 displays risk estimation within each risk category, split by profession.

Figure 1.

Density function of estimated risk compared between nurses and doctors for each level of categorized severity (1, 2, or 3). Estimated risk shifts to the right as categorized risk gets more severe.

Figure 2.

Estimated risk distribution between risk categories split between profession (doctors vs. nurses)

All t-tests testing the different in the mean of the estimated risk between nurses and doctors show p<0.05. Differences between group estimated risk means are statistically significant at the level of alpha = 0.05.

DISCUSSION

Mortality rates, adjusted based on the predictions of mortality provided by prognostic scoring systems, are increasingly used to compare the quality of care provided by different intensive care units (ICU)s and hospitals. Little is known, however, about how providers interpret risk against validated scoring systems. This research aims to understand how providers forecast mortality, the correlation with sepsis scoring system mortality models, and the impact on clinical decision making. Accurately forecasting a patient’s risk of in-hospital mortality is crucial for clinicians to make point of care decisions intended to influence clinical outcomes.

For the present study, the research team presented clinical scenarios of real sepsis patient cases to both nurses and physicians and prompted participants to provide mortality estimates and categorize those estimates into “low,” “moderate,” and “high,” risks of in-hospital death. The findings suggest that the ability to estimate a patient’s mortality is highly associated with the profession of the estimator. Especially for high severity patients, nurses were more likely to predict higher risk of in hospital death. When predicting the trajectories for low severity patients, both nurses and physicians categorized similarly (between low, medium, and high), but when presented with the high severity cases, there is greater variability. This evidence suggests that care communication and coordination between medical personnel grow more critical as a patient’s condition worsens.

By evaluating comparisons of physicians versus nurses and levels of severity, we can better support clinicians in appropriately treating septic patients. Knowing which patients are at highest risk of mortality contributes to improved quality of care, appropriate resource allocation, and cost reduction. Especially in sepsis patients, who commonly decline rapidly if not given the proper treatment, it is vital that clinicians are able to understand a patient’s clinical trajectory in order to make appropriate treatment decisions.