Opportunities for Machine Learning to Improve Surgical Ward Safety

Abstract

Background:

Delayed recognition of decompensation and failure-to-rescue on surgical wards are major sources of preventable harm. This review assimilates and critically evaluates available evidence and identifies opportunities to improve surgical ward safety.

Data Sources:

Fifty-eight articles from Cochrane Library, EMBASE, and PubMed databases were included.

Conclusions:

Only 15–20% of patients suffering ward arrest survive. In most cases, subtle signs of instability often occur prior to critical illness and arrest, and underlying pathology is reversible. Coarse risk assessments lead to under-triage of high-risk patients to wards, where surveillance for complications depends on time-consuming manual review of health records, infrequent patient assessments, prediction models that lack accuracy and autonomy, and biased, error-prone decision-making. Streaming electronic heath record data, wearable continuous monitors, and recent advances in deep learning and reinforcement learning can promote efficient and accurate risk assessments, earlier recognition of instability, and better decisions regarding diagnosis and treatment of reversible underlying pathology.

Summary

Delayed recognition of decompensation and failure-to-rescue on surgical wards are major sources of preventable harm. Ward safety is compromised by coarse risk assessments leading to under-triage of high-risk postoperative patients to wards, where understaffed providers make infrequent patient assessments and use cognitive shortcuts, leading to bias and error-prone decision-making. Streaming electronic heath record data, wearable continuous monitors, and recent machine learning advances can promote efficient and accurate risk assessments, earlier recognition of instability, and better decisions regarding diagnosis and treatment of reversible underlying pathology.

Introduction

Following surgery, traumatic injury, or the onset of a surgical disease, patients who are not critically ill but are too ill for discharge home are admitted to surgical wards. Their recovery may be complicated by hemorrhage, respiratory failure, opiate toxicity, sepsis, and other life-threatening conditions. On surgical wards, infrequent patient assessments may lead to unrecognized decompensation and progression to cardiac arrest. Only 15–20% of patients suffering ward arrest survive and survivors often require prolonged hospitalization and rehabilitation.^{1, 2}

Some patients remain stable until suffering an acute event—like myocardial infarction and pulmonary embolism—where immediate recognition and rescue are the only paths to survival. In cases of gradual deterioration, subtle signs of physiologic instability often occur prior to organ failure and cardiac arrest, representing opportunities for prevention.^{3, 4} When risk for decompensation and arrest are underestimated, high-risk patients may be under-triaged to a ward rather than an intensive care unit (ICU).⁵ Traditional ward monitoring involves time-consuming, manual review of health records, infrequent patient assessments, high patient-to-provider ratios, outdated prediction models that lack accuracy and autonomy, and biased, error-prone decision-making.^6–10 Surgical ward patients continue to incur preventable harm from unrecognized decompensation, resulting in failure-to-rescue.^{11, 12} Machine learning systems and emerging technologies have the potential to address weaknesses inherent to traditional approaches and improve surgical ward safety, but have not yet gained widespread recognition or clinical adoption.

The purpose of this article is to promote deeper understanding of problems and potential solutions associated with surgical ward safety. The objectives of this article are to (1) summarize available evidence regarding the epidemiology of hospital ward decompensation and arrest, unique aspects of surgical ward patients, and weaknesses inherent to traditional ward monitoring, and (2) propose a framework in which machine learning and emerging technologies address these weaknesses, while emphasizing the importance of bedside assessment and human intuition in recognizing and managing a decompensating patient.

Methods

Cochrane Library, EMBASE, and PubMed databases were searched from their inception to December 2018, using keywords and terms defined in Supplementary Figure 1. Three authors with medical degrees screened articles by reviewing abstracts for the following criteria: (1) published in English and (2) primary literature or a review article, including the following study types: cross-sectional study, Delphi consensus, observational study, prospective observational study, prospective randomized trial, retrospective study, review article, and systematic review. Ninety-three articles were excluded by these criteria. Articles were selected for inclusion by manually reviewing abstracts and full texts for the following criteria: (1) topical relevance, (2) methodological strength, and (3) novel or meritorious contribution to existing literature describing the epidemiology of ward decompensation, traditional approaches to ward monitoring, new technologies that address the deficits of traditional approaches, and the importance of integrating bedside assessment and human intuition with prediction models. One hundred and thirty-one articles were excluded by these criteria. The screening and selection process was unblinded. Due to heterogeneity among study populations, methods, and results from published literature, the authors assessed topical relevance, methodological strength, and novelty by subjective means. Unpublished articles and abstracts without an accompanying full text were not eligible for inclusion. The authors also performed a manual review of articles cited by articles identified in the initial search using the same selection criteria. Eight articles were included by these criteria. Therefore, fifty-eight studies were included and manually synthesized into categories pertinent to study objectives, which were determined a priori by the authors. The design, population, sample size, major findings pertinent to this review, sources of funding, and conflicts of interest for all included studies are listed in Table 1.

Table 1:

Summary of included studies.

Primary Author	Study Design	Population	Sample Size	Major Findings Pertinent to this Scoping Review	Sources of Funding and Conflicts of Interest
Antink64	Observational	Arrhythmia alarms	n=1250	Machine learning techniques and multimodal rhythmicity estimation created a 95% TPR and 78% TNR for arrhythmia alarms	None reported
Arkin9	Retrospective	Aortic valve replacements	n=410,157	Higher nurse-to-patient ratios were protective against preventable complications (OR 0.94, 95% CI 0.90–0.99)	AHRQ
Bartkowiak47	Retrospective	Ward patients	n=32,537	Validation of the superiority of eCART vs. MEWS and NEWS; maximum respiratory rate was the most predictive vital sign for severe adverse events (AUC 0.67)	NHLBI, Philips Healthcare, Early Sense, Quant HC
Bechara71	Cross-Sectional	Patients with prefrontal cortex damage, healthy controls	n=16	In healthy individuals, activation of covert biases precedes overt reasoning, possibly reflecting unconscious access to previous experiences	None reported
Berlot4	Retrospective	In-hospital arrests	n=263	86% of in-hospital arrests had recognizable anticipating events, survival without neurologic sequelae was 5%	None reported
Brown38	Retrospective	Ward admissions	n=7,643	Continuous monitoring was associated with shorter hospital LOS, ICU LOS, and fewer respiratory events	EarlySense
Chen65	Retrospective	Hours of non-invasive vital sign monitoring	n=179,157	ML models can be taught to discriminate clinically relevant changes in vital signs from artifacts (AUC>0.87)	NINR, NHLBI, NSF
Churpek46	Retrospective	Ward admissions	n=269,999	The eCART early warning score was more accurate than the MEWS for predicting cardiac arrest, ICU transfer, and death	NHLBI, NIA, Philips Healthcare, AHA, Laerdal Medical, EarlySense
Churpek51	Retrospective	Ward patients	n=3,789	A >6 hour interval between onset of critical illness and ICU transfer was associated with increased mortality (33 vs. 25%)	NHLBI, Philips Healthcare, AHA, ownership interest in Quant HC
DeVita52	Retrospective	Rapid response team activations	n=3,269	Rapid response team adoption was associated with fewer ward arrests (5.4 vs. 6.5 per 1,000 admissions)	None reported
DeVita54	Consensus Conference	Experts	n=27	Vital sign aberrations can be used to predict risk of decompensation and improved monitoring may impact outcome, however the burden and characteristics of intensive monitoring system have yet to adequately studied	AHA, AHRQ, CHEST, AACN, Italian Resuscitation Council, Unilink, Delmarva Foundation, VA, The Learning Clinic, Patientrack, Philips Medical systems
Douw32	Systematic Review	Articles about clinical signs triggering nursing concern	n=18	Qualitative, intuitional assessment of a patient’s condition can trigger concern prior to changes in vital signs	None reported
Dybowski6	Retrospective	ICU patients	n=258	An artificial neural network outperformed a regression model in predicting in-hospital mortality (AUC 0.86 vs. 0.75)	Special Trustees for St. Thomas’ Hospital
Eerikainen63	Observational	Dysrhythmias	n=1,250	Simultaneous use of ECG, arterial BP, and PPG signals, along with Random Forest models, increases TPR and TNR for dysrhythmia alarms	None reported
Fernando56	Prospective	Ward patients	n=5,995	Older patients decompensating on wards were more likely to have delayed rapid response team activation and worse outcomes	None reported
Franklin3	Retrospective	Hospital admissions	n=21,505	66% of ward arrests were preceded by documented decompensation	None reported
Fry20	Retrospective	Patients undergoing major surgery	n=702,268	Decreased failure-to-rescue rates explained 64% of improvement in hospital mortality, a decrease in surgical complications accounted for 5%	NIH, AHRQ, ArborMetrix, PCORI
Ghaferi15	Retrospective	Major surgeries	n=107,899	Failure-to-rescue ranged from 6.8% for the top 20% of hospitals to 16.7% for the bottom 20% of hospitals	AHRQ, NCI
Ghaferi19	Retrospective	Pancreatectomies	n=16,900	Failure-to-rescue ranged from 6.4% for the top 20% of hospitals to 40.0% for the bottom 20% of hospitals	AHRQ, NCI, Robert Wood Johnson Clinical Scholars Program
Griffiths33	Retrospective	Ward admissions	n=138,133	The average patient-to-nurse ratio was 5.5, temporary staffing was associated with increased mortality	NIHR
Hassen34	Delphi Consensus	Experts	n=27	100% agreement that inadequate staffing negatively impacts surgical ward safety	NIHR
Heller53	Retrospective	Ward patients	n=3,827	Automated EWS paging system implementation was associated with fewer ward arrests (2.1 vs. 5.3 per 1,000 admissions)	Philips Healthcare
Helling12	Retrospective	Unexpected ICU transfers	n=111	Multiple physician notifications were required prior to 29% of all unexpected ICU transfers	None reported
Karpman55	Retrospective	Ward patients	n=20,745	Rapid response team implementation was associated with more ICU admissions and no change in outcomes of transferred patients	None reported
Kim7	Retrospective	ICU admissions	n=38,474	Machine learning algorithms were as good as APACHE III in predicting ICU mortality	NCRR
Komorowski60	Retrospective	Septic ICU patients	n=96,156	A reinforcement learning model recommending intravenous fluid and vasopressor strategies outperformed human clinicians	Orion Pharma, Amomed Pharma, Ferring Pharma, Tenax Therapeutics, Baxter Healthcare, Bristol-Myers Squibb, GSK, HCA International
Kumar31	Retrospective	Adult ICU patients with septic shock	n=2,731	Each hour of delay in antimicrobial administration decreased survival by 7.6%	Eli-Lilly, Pfizer, Merck, Astra-Zeneca, GSK, Arginox, DeepBreeze, Minimitter, Edwards, OrthoBioTech
Martin66	Review	Validation Studies for Wrist Actigraphy	n=15	Wrist actigraphy can be used to estimate sleep parameters and provide a useful adjunct to sleep diaries, polysomnography, and clinical interviews in sleep disorders	NIH, NIA, Cedars Siani Sleep Medicine Fellowship, VA GLAHS GRECC
McGaughey43	Systematic review	Trials about rapid response teams	n=2	EWS and rapid response team adoption was not associated with decreased mortality	Ireland Cochrane Fellowship
McGinley45	Review	-	-	Description of the National Early Warning Score and its application	None reported
McGloin11	Retrospective	Ward deaths and ICU transfers	n=415	65% of all unexpected ward deaths followed gradual decompensation	None reported
McQuillan57	Prospective	Unplanned ICU transfers	n=100	Pre-transfer care was suboptimal for 54/100 patients, 69% of this group had delayed transfer, and their mortality was 48%	NHS Trust, Lilly Industries
Merchant17	Cross-sectional	Adult IHCA with resuscitation response	n=209 million	Incidence study demonstrating ~200,000 cardiac arrests treated among US hospitalized patients annually	Philips Healthcare, Laerdal Medical, NIH, Cardiac Science, NHLBI, Medivance, Doris Duke Foundation, AHA, Lifebridge Medizintechnik, Gambro Renal
Mitchell50	Prospective	Ward admissions	n=2,142	Implementation of a ward monitoring system was associated with fewer unexpected ICU transfers and deaths	ACT Health
Mohammed Iddrisu23	Retrospective	Surgical ward patients	n=100	The most common reason for rapid response team activation was hypotension (26%)	None reported
Pearse5	Retrospective	High-risk surgical patients in the UK	n=513,924	High-risk patients accounted for 84% of deaths but for only 12.5% of procedures; highest mortality (39%) occurred in high-risk patients admitted to the ICU following initial admission to a standard ward	None reported
Peberdy14	Retrospective	In-hospital cardiac arrests	n=14,720	44% of adult in-hospital cardiac arrest victims had ROSC; 17% survived to hospital discharge; 86% of those with CPC-1 at time of admission had post arrest CPC-1 at time of discharge	AHA
Perman16	Retrospective	In-hospital cardiac arrests	n=85,201	41% of all in-hospital cardiac arrests occurred on wards with a rate of 0.1 events per 1,000 bed-days	NIH, Philips Healthcare, EarlySense, Quant HC, Physio-Control, Zoll Medical, Cardiac Science
Prgomet41	Observational	Clinical staff	n=84	Perceptions of hospital staff towards continuous monitoring is generally positive, but concerns remain for inappropriate escalations of care and patient discomfort	NHMRC
Rothman48	Retrospective	Hospitalized patients	n=148,985	Rothman Index predicted 24-hour mortality with AUC 0.93 or greater	Sarasota Memorial Healthcare Foundation, Greenfield Foundation
Rothman49	Retrospective	Hospitalized patients	n=42,302	12 types of nursing assessments were significant predictors of mortality	Rothman Healthcare Corporation, Sarasota Memorial Healthcare Foundation, Greenfield Foundation
Sandroni2	Review	Articles about in-hospital arrest	n=79	In most studies, survival following in-hospital cardiac arrest ranges from 15–20%	None reported
Sanfey69	Cross-Sectional	Players of a classic economic test in splitting sums of money	n=19	fMRI imaging shows simultaneous heightened activity in emotion and cognition centers when deciding between fair and unfair offers	None reported
Schein18	Prospective	Ward arrests	n=64	The most common physiologic sign of decompensation prior to arrest was respiratory (38%)	None reported
Shickel59	Retrospective	ICU admissions	n=85,164	A deep learning model predicted in-hospital mortality more accurately than traditional acuity score calculation	NIGMS, NSF, UF CTSI, NCATS, J. Crayton Pruitt Family Department of Biomedical Engineering, NVIDIA
Silber10	Retrospective	Hospital admissions	n=403,679	Higher bed-to-nurse ratios were associated with increased odds of failure-to-rescue	NHLBI, Veterans Affairs Health Services Research and Development
Silver61	Observational	Go boardgame	-	Deep reinforcement learning models provide high fidelity victories against previous Go algorithms and human experts	Google, Google DeepMind
Skogvoll1	Retrospective	In-hospital deaths	n=4,927	Survival after cardiopulmonary resuscitation outside of the ICU was 17%	None reported
Slight39	Retrospective	Modeled ward admissions	n=5,000	Costs and savings associated with a continuous ward monitoring system demonstrated favorable return on investment	EarlySense
Smith42	Systematic review	Early warning systems	n=33	AUC range for predicting adverse outcomes was 0.66–0.78	The Learning Clinic Ltd.
Subbe36	Review	Articles about ward decompensation	n=37	Among high-risk ward patients, continuous monitoring may facilitate early detection of decompensation	NHS Trust
Subbe44	Prospective	Medical emergency admissions	n=709	Higher EWS were associated with increased odds of death and ICU transfer	NHS Trust
Taenzer37	Prospective	Surgical ward admissions	n=13,398	Continuous pulse oximetry was associated with fewer rescue events and unplanned ICU transfers	The Hitchcock Foundation, Masimo Corporation
Van den Bruel74	Retrospective	Primary care patients	n=3,890	Clinician intuition identified patients with illness severity that was underrepresented by clinical parameters	Research Foundation Flanders, Eurogenerics, NIHR
Van den Bruel75	Systematic review	Articles about features of serious infections	n=30	Traditional clinical parameters associated with serious infection are often absent among patients with serious infections	Health Technology Assessment, NIHR
Watkinson40	Prospective randomized	Ward patients	n=402	Major adverse events were similar between continuous and standard monitoring groups	Oxford BioSignals, NHS Trust
Weenk62	Prospective	Ward patients	n=20	EWS calculated with data from wearable continuous monitors were similar to EWS calculated from vitals obtained by nurses	Radboud University Medical Center
Wesnes35	Prospective	Surgical house officers	n=10	House officers had impaired concentration and memory after a weekend on call	None reported

AAA = Abdominal Aortic Aneurysm; AHA: American Heart Association; AHRQ: Agency for Healthcare Research and Quality; APACHE: Acute Physiology and Chronic Health Evaluation; AUC: Area Under the Curve; CHEST: American College of Chest Physicians; AACN: American Association of Critical Care Nurses; CI: Confidence Interval; CPC: Cerebral Performance Category; eCART: electronic Cardiac Arrest Triage; EWS: Early Warning System; GSK: GlaxoSmithKline; HCA: Hospital Corporation of America; ICU: Intensive Care Unit; IHCA: In-hospital Cardiac Arrest; LOS: Length of Stay; MEWS: Modified Early Warning sSore; NCATS: National Center for Advancing Translational Science; NCI: National Cancer Institute; NCRR: National Center for Research Resources; NEWS: National Early Warning Score; NHLBI: National Heart, Lung, and Blood Institute; NHMRC: National Health and Medical Research Council; NHS: National Health Service; NIA: National Institute of Aging; NIGMS: National Institute for General Medical Sciences; NIH: National Institute of Health; NIHR: National Institute for Health Research; NINR: National Institute of Nursing Research; NSF: National Science Foundation; OR: Odds Ratio; PCORI: Patient-Centered Outcomes Research Institute; PPG: photoplethysmogram; TNR: True Negative Rate; TPR: True Positive Rate; UF CTSI: University of Florida Clinical and Translational Science Institute; VA GLAHS GRECC: Veteran’s Administration Greater Los Angeles Healthcare System Geriatric Research Education and Clinical Center

Results

Definitions and Epidemiology

Definitions of decompensation in published literature are heterogeneous and often subjective. An increase in the sequential organ failure assessment (SOFA) score by two or more can be used to identify the onset of organ dysfunction and critical illness, signifying decompensation.¹³ Cardiac arrest is defined as resuscitation requiring chest compressions and/or cardiac defibrillation.¹⁴ Failure-to-rescue is defined as the death of a patient following a complication.¹⁵

Approximately 41% of all in-hospital cardiac arrests occur on wards at a rate of 0.1 per 1,000 bed-days.¹⁶ On hospital wards, vital sign measurements and nursing assessments are often performed every four hours.^{1, 16, 17} This monitoring strategy may fail to identify early signs of physiologic deterioration, which often occur hours before arrest.³ In a review of 263 ward arrests, 86% were preceded by signs and symptoms of decompensation.⁴ The underlying disease process is often reversible.¹⁸

In an audit of unexpected ward deaths at a teaching hospital, almost two-thirds followed gradual decompensation.¹¹ In a review of surgical ward decompensation requiring unplanned ICU transfer, nearly one in three episodes were characterized by delayed recognition of critical illness.¹² When decompensation progresses to arrest, failure-to-rescue is the most likely outcome. In-hospital cardiac arrest occurring outside of the ICU is associated with 80–85% mortality.^{1, 2} Rates of failure-to-rescue for surgical patients following a complication is highly variable across US hospitals, suggesting that system-level factors contribute.^{15, 19} Although preventing complications is a laudable goal, decreasing failure-to-rescue may have a greater impact on mortality rates. In a recent analysis of Medicare beneficiary data, hospitals that performed well in reducing mortality after major surgery accomplished these reductions primarily by reducing failure-to-rescue rates rather than complications.²⁰

Unique Characteristics of Surgical Patients Contributing to Decompensation and Arrest

Surgical patients are uniquely vulnerable to postoperative hemorrhage, respiratory failure, opiate toxicity, and sepsis.^{12, 21, 22} These conditions impart a second physiologic insult following surgery, traumatic injury, or acute illness requiring hospitalization. Traumatic injury and major surgery increase risk for the interval development of brisk hemorrhage. Up to 20% of all circulating blood may be lost before compensatory tachycardia is evident and up to one-third of all circulating blood may be lost before hypotension develops. When vital signs are measured every four hours, a patient may progress to hemorrhagic shock and organ dysfunction between assessments.²³

Postoperative patients are susceptible to respiratory failure, especially following general anesthesia and painful thoracic and abdominal procedures. General anesthesia is associated with decreased functional residual capacity, alveolar macrophage dysregulation, increased alveolar capillary permeability, and increased tissue plasminogen activator (tPA) inhibitor levels. These factors predispose patients to atelectasis, pneumonia, deep vein thrombosis, and pulmonary embolism.^24–27 In addition, painful thoracic and abdominal procedures impair patients’ ability to take deep breaths, cough, clear secretions, and participate in respiratory therapy exercises. In a review of surgical patients requiring unplanned ICU transfer, respiratory events accounted for 69% of all alerts.¹² Respiratory depression, and eventual failure, among postoperative patients may be exacerbated by opiate consumption. Hospitalized surgical patients consume more opiates than hospitalized medical patients.^{21, 22} Patients with severe postoperative pain may receive high concentrations of opiates that saturate mu receptors and degradative enzymes, leading to dose-dependent respiratory depression.^{28, 29}

Surgical ward patients are also vulnerable to sepsis, which accounts for one in four deaths following elective surgery.³⁰ In a landmark 2006 study by Kumar et al.³¹ only half of all patients with septic shock received antibiotics within six hours of sepsis onset; each one-hour delay in antibiotic administration beyond six hours was associated with an 8% increase in mortality. Early diagnosis of postoperative sepsis may be confounded by difficulties in differentiating between sepsis and sterile inflammation. Vigilance and careful patient assessment are necessary to not only facilitate early recognition of postoperative sepsis but also avoid unnecessary intravenous fluid and antibiotic administration.

Factors contributing to Failure-to-Rescue

Limitations on Nursing and Physician Staffing

Low staffing levels on surgical wards are associated with failure-to-rescue. Nurses spend more time at the bedside than physicians and their intuitive sense of impending decompensation can precede objective signs of decompensation.³² This advantage may be abrogated by higher patient-to-nurse ratios, which averaged 5.5 in a review of over 138 thousand ward admissions.³³ Higher patient-to-nurse ratios are associated with significantly increased odds of preventable postoperative complications and failure-to-rescue.^{9, 10} In a Delphi Consensus, there was 100% agreement that both inadequate staffing levels negatively impact patient safety on surgical wards and access to doctors at night promotes patient safety.³⁴ However, the perceived benefits of access to doctors on call at night may be abrogated when low physician staffing levels and long hours on call negatively impact cognitive performance.³⁵ When staffing levels are low, early identification of physiologic instability often depends on vital sign monitoring.

Continuous Monitoring of Vital Signs

On hospital wards, vital signs are typically measured every four hours. Among high-risk ward patients, continuous monitoring may facilitate early detection of decompensation.³⁶ Continuous pulse oximetry has been associated with decreased frequency of rescue events and unplanned ICU transfers.³⁷ Continuous monitoring of heart and respiratory rate has been associated with shorter hospital length of stay, fewer ICU days, and fewer respiratory decompensation events.³⁸ Though continuous monitoring systems are expensive, the return on investment may be favorable.³⁹

However, level 1 evidence to support continuous monitoring is lacking.⁴⁰ Potential disadvantages of continuous monitoring include distracting false-positive alarms, patient discomfort, and impaired mobility.⁴¹ In addition, patient selection for continuous monitoring often depends on crude risk estimations using admission vital signs, time-consuming manual review of medical records, and initial impressions on clinical assessment. Current methods are inadequate for identifying high-risk, outlier patients that may benefit from close monitoring.³⁴ The frequency of false alarms may be partially attributable to data dimensionality. As the number of vital sign data sources increases linearly, the combinations of alarm parameters that may indicate instability increase exponentially. Finally, reduced patient contact may be an unintended consequence of continuous monitoring.⁴¹

Early Warning Scores and Clinical Decision-Support Systems

Early warning scores facilitate identification of decompensating patients by considering vital signs and patient assessments in aggregate. Many early warning scores exist, but few perform well on external validation.^{42, 43} Additive scores like the Modified Early Warning Score (MEWS)⁴⁴ and the National Early Warning Score (NEWS)⁴⁵ are easy to interpret and apply, but their accuracy is hindered by use of static variable thresholds for a small number of parameters. The electronic Cardiac Arrest Risk Triage (eCART) score was developed using regression coefficients to identify cut-off values for patient demographics, vital signs, and 18 laboratory values.⁴⁶ Notably, eCART outperformed MEWS and NEWS in a retrospective analysis of 32,537 postoperative surgical inpatients.⁴⁷

The Rothman Index uses vital signs, cardiac rhythms, laboratory values, and nursing assessments for subtle signs of illness—like loss of appetite and mild confusion—and accurately predicts death or discharge to hospice within 24 hours (AUC 0.93).^{48, 49} Blood oxygen saturation is included, but the influence of supplemental oxygen is not considered. Laboratory values are blended with a linear decay function for 48 hours and while this seems appropriate for daily laboratory values, it is less useful for serum albumin or liver enzymes, which are often measured infrequently. Parametric regression modeling assumes linear relationships among variables. However, relationships among variables are often complex and non-linear, limiting the accuracy of regression models.^{6, 7}

Decisions regarding ward patient management and ICU transfer may be affected by several forms of bias.⁸ The framing effect occurs when outcome predictions are affected by contextual information. A patient on a surgical ward who is mildly confused one day after an elective abdominal surgery, but has had stable vital signs since the operation, may be deemed appropriate for continued intermittent monitoring on a surgical ward. However, in the context of known liver dysfunction or history of crescendo transient ischemic attacks, this same patient may benefit from assessment of hepatic function, a focused neurologic exam, continuous monitoring, and consideration of ICU transfer. Overconfidence and optimistic bias occur when individuals perceive that things are better than they are and that adverse events disproportionately affect others. A surgeon may ignore signs that their patient is decompensating, and thus delay ICU transfer, due to overconfidence and unwarranted optimism in the likelihood of early recovery.

Anchoring bias, which occurs when outcome probability estimations gravitate toward an initial baseline value, may exacerbate the impact of overconfidence. A surgeon may observe that 95% of their patients recover uneventfully, leading them to believe that their next patient will also do well. However statistics mean nothing to an individual patient who is actively decompensating despite a 95% chance of uneventful recovery. Anchoring bias may be compounded when recent experiences have been positive, thus promoting recall bias in which outcome probability estimations are influenced by the ease with which prior experiences are recalled. Confirmation bias occurs when outcomes are predicted using personal beliefs rather than evidence. In the absence of high-quality evidence to guide surgical ward monitoring, personal beliefs may play a prominent role.

Rapid Response Teams

When ward patients develop acute organ dysfunction and decompensation, early ICU transfer is essential.⁵⁰ In a review of 3,789 medical and surgical ward patients at five different hospitals, the median interval between onset of critical illness and ICU transfer was 5.4 hours.⁵¹ Each additional 1-hour delay in transfer was associated with 3% increased mortality. Delay in transfer more than 6 hours was also associated longer hospital length of stay among survivors (13 vs. 11 days, p=0.01). Many hospitals employ rapid response teams that receive alerts, travel to the bedside, and facilitate early diagnosis and treatment, including ICU transfer if necessary. Use of rapid response teams has been associated with decreased incidence of cardiac arrest and unplanned ICU transfers.^{52, 53} However, rapid response teams are only as good as their activation criteria, clinical judgement, and the quality of transitions from hospital wards to the ICU.^54–56 In a review of medical and surgical patients with unplanned ICU admissions, pre-transfer care was suboptimal for 54%.⁵⁷ Patients receiving suboptimal pre-transfer care had 56% mortality, almost double that of patients receiving appropriate care.

Future Directions: Preventing the Need to Rescue

By current methods, high-risk patients who would benefit from close monitoring in an ICU may be under-triaged to a ward, where care is compromised by time-consuming manual review of health records, infrequent patient assessments, high patient-to-provider ratios, prediction models that lack accuracy and autonomy, and biased, error-prone decision-making (Figure 1).^{5, 9, 10} Integrating new technologies with human intuition and bedside assessment have the potential to transform surgical ward monitoring.

Figure 1:

Traditional approaches to surgical ward monitoring have several pitfalls and opportunities for improvement.

Use of Machine Learning for Phenotyping and Decision Support

Deep learning is an machine learning extension of regression-based techniques that is adept at learning and representing highly-dimensional data with non-linear functions.⁵⁸ The initial and final layers are inputs and outputs, respectively. The middle layers contain hidden nodes. A multilayer perceptron is a standard neural network with a single middle layer; deep models can have hundreds or thousands of middle layers. Each link connecting two nodes is assigned a weight that is influenced by previous layers and affects the output from that node. Weights are continually updated by an algorithm that optimizes the relationships between input and output layers, allowing accurate representation of complex, non-linear relationships among input variables. Deep models automatically learn optimal feature representations from raw data: a major advantage over models that require time-intensive, hand-crafted feature engineering. However, deep learning is difficult to interpret and is susceptible to overfitting and vanishing gradients unless corrective measures—like dropout and rectified linear units—are employed.

Deep models like the recurrent neural network—with its variants the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU)—capitalize on the availability of vast amounts of sequentially ordered, time-series vital sign data in electronic health records (EHRs). Recurrent neural networks update weights according to both the current inputs and previous states of each hidden unit.⁵⁸ GRU modeling has been used to predict in-hospital mortality for ICU patients using SOFA score variables from the electronic health record, and these models exhibit greater accuracy than the SOFA score itself.⁵⁹ A similar model could predict the onset of critical illness, unplanned ICU transfer, and death among surgical ward patients. Despite these capabilities, deep learning cannot suggest a course of action for a given clinical scenario, but reinforcement learning can.

In reinforcement learning, an algorithm learns that certain policies applied to given states lead to defined outcomes. This occurs in the context of a Markov decision process (MDP). Consisting of a finite set of states and actions, the MDP uses both the probability that a given action within a given state will lead to a new state along with the reward that results from the new state. Reinforcement learning derives a policy comparing the predicted utility of different actions under certain circumstances, identifying the choice or action with the highest probability of a defined outcome.

The Artificial Intelligence (AI) Clinician is a reinforcement learning model capable of recommending fluid resuscitation and vasopressor administration strategies among septic patients. This model was built on an MDP framework trained on Medical Information Mart for Intensive Care (MIMIC-III) data, and validated with Philips eICU data, including 96,156 ICU admissions.⁶⁰ Forty-eight variables—including vital signs, laboratory values, and comorbidities—were tracked by 4-hour increments over 72 hours and clustered into 750 distinct states. The model was then trained to predict the probability that, from any of the states, a given treatment would result in transition to another state. To establish the reward principle, positive scores were assigned to cases leading to survival. The model then experimented with fluid resuscitation and vasopressor administration strategies, learning which single strategy was associated with the highest probability of survival in each state. Mortality was lowest when actions taken by clinicians matched actions recommended by AI Clinician.

Combining deep and reinforcement learning offers unique advantages for complex scenarios, as demonstrated in the gaming industry.⁶¹ A similar approach could be used to augment surgical ward monitoring (Figure 2). Given the rarity of cardiac arrest on wards and the inherent difficulties associated with expeditious recognition and appropriate management of ward arrest, it seems prudent to seek prevention as a primary objective. Initial patient triage presents an important opportunity to achieve this objective. Rather than depending solely on the accuracy and reliability of triage decisions made by clinicians in the emergency department, operating room, or post-anesthesia care unit, deep models would identify high-risk patients who are likely to benefit from initial triage to a high level of care with continuous or frequent monitoring and patient assessments, and offer recommendations to clinicians. The same models could be used to avoid unnecessary over-triage of low-risk patients to an ICU. Continuous vital sign, actigraphy data from wearable devices, and live-streaming EHR data would fuel real-time, deep learning predictions of impending critical illness and in-hospital mortality with automated activation of rapid response teams. Reinforcement learning models would augment decisions regarding initial treatment of underlying pathology and ICU transfer.

Figure 2:

Wearable monitors, streaming electronic health record data, and machine learning can improve surgical ward monitoring.

Pervasive Sensing

The multi-billion dollar market for wearable fitness devices has fostered interest in medical applications, including continuous monitoring devices. Older wearable monitors used rigid, silicon-based hardware; newer devices use lightweight, flexible plastics and elastomers that interface well with human skin. Wearable devices have the potential to transform continuous monitoring of ward patients by streaming vital sign data to bedside and central monitoring systems while simultaneously minimizing patient discomfort and promoting mobility. Weenk et al.⁶² performed a pilot study in which 20 ward patients wore one of two devices. Vital sign measurements from the wearable devices were consistent with manual vital sign measurements by nurses. Data was missing or invalid 13–16% of the time, usually due to wireless connection failure and loss of skin contact. Data artifact can be mitigated by machine learning algorithms.^63–65

Modern physical activity and functional status assessments often rely on questionnaire-based evaluations along with daily interactions with nurses and physical and occupational therapists. Physical activity can also be monitored with non-invasive actigraphy devices that continuously collect activity data over long periods of time.⁶⁶ Beyond guiding mobilization interventions, these data could be used in concert with other vital sign and clinical assessments to identify patients at increased risk for decompensation and arrest.^{67, 68} In addition, incorporating data from operating room monitors may improve model accuracy by capturing important intraoperative events and physiologic changes.

Barriers to Implementation

Machine learning models are only as good as the quality and quantity of data used to train them.⁵⁸ Poorly trained models could provide a false sense of security, promote over-triage, or recommend ineffective or harmful treatments. Any of these scenarios could occur on a large scale. A clinician makes mistakes one patient at a time, but an errant model could harm hundreds or thousands of patients all over the world in a short period of time.

Traditional early warning scores, like the MEWS and NEWS, are easy to interpret by simple calculations using defined cut-off values.^44–46 Deep learning models provide limited insight regarding the relative importance of model inputs in determining model outputs. In discussing end-of-life care with patients and their caregivers, it would be inadequate to explain that the likelihood of functional recovery is low according to a computer, and that it is unclear why the computer reached this conclusion.

A deep reinforcement learning platform requires information technology infrastructure and expertise. These resources are not available in many clinical settings. Careful analysis of costs and savings would be necessary to determine whether this platform could be broadly applied.

Human Intuition

Bedside evaluation by an astute clinician is the most important element of patient assessment. Face-to-face interactions elicit signs and symptoms of worsening clinical status like diaphoresis, confusion, malaise, pain, and tenderness. Computer programs can also elicit some signs and symptoms of illness and their ability to do so will improve over time. However, human interactions will remain essential because they incite one of the greatest assets in surgical decision-making: human intuition, which appears to emerge from predictions made by flexible limbic system dopaminergic neurons that adjust their connections in response to positive and negative stimuli.^{69, 70}

Purely rational, evidence-based decision-making which uses a limitless supply of infallible data would be well suited to deep reinforcement learning. However, data and evidence are often lacking, particularly for surgical emergencies. In addition, optimal decision-making occurs under the influence of human intuition, as evident in decisions regarding card games, fight or flight survival responses, and naval warfare.^71–73 Intuition can also identify patients with severe, life-threatening conditions that would otherwise remain unrecognized by traditional clinical parameters.^{74, 75} Among ward patients, nursing intuition of imminent decompensation may precede objective signs of decompensation.³²

Conclusions

Early warning scores, continuous monitoring of high-risk patients, rapid response teams, and clinical decision-support systems promote early identification and appropriate management of decompensating surgical ward patients. Despite these advances, patients continue to incur preventable harm from delayed recognition of worsening clinical status, suboptimal decision-making, and failure-to-rescue. There is an urgent need for workflow-integrated decision-support tools to 1) ensure that high-risk patients are triaged to an ICU or continuously monitored on a ward, 2) predict decompensation prior to organ failure and arrest, and 3) recommend treatments for underlying pathology and ICU transfer for decompensating patients. Live-streaming EHR data, wearable continuous monitoring devices, and machine learning models have the potential to transform ward monitoring by providing accurate, autonomous, real-time patient assessments to augment these error-prone tasks and decision-making processes. The limitations of machine learning and the importance of human intuition should be emphasized throughout the clinical adoption process.

Highlights.

• Delayed recognition of decompensation on surgical wards is a major source of preventable harm

• Coarse risk assessments lead to under-triage of high-risk postoperative patients to wards

• Subtle signs of instability often precede ward arrest, underlying pathology is often reversible

• Understaffed providers use cognitive shortcuts, leading to bias and poor decisions

• Wearable monitors, streaming EHR data, and machine learning can improve ward safety