Coronavirus Disease 2019 Calls for Predictive Analytics Monitoring—A New Kind of Illness Scoring System

Abstract

Coronavirus disease 2019 can lead to sudden and severe respiratory failure that mandates endotracheal intubation, a procedure much more safely performed under elective rather than emergency conditions. Early warning of rising risk of this event could benefit both patients and healthcare providers by reducing the high risk of emergency intubation. Current illness severity scoring systems, which usually update only when clinicians measure vital signs or laboratory values, are poorly suited for early detection of this kind of rapid clinical deterioration. We propose that continuous predictive analytics monitoring, a new approach to bedside management, is more useful. The principles of this new practice anchor in analysis of continuous bedside monitoring data, training models on diagnosis-specific paths of deterioration using clinician-identified events, and continuous display of trends in risks rather than alerts when arbitrary thresholds are exceeded.

Wide-wasting Pest! that rages unconfin’d,

And crouds with Crimes the Record of Mankind

“The Vanity of Human Wishes”

—Dr. Samuel Johnson

The coronavirus disease 2019 (COVID-19) pandemic is a black swan event for the healthcare system. Overwhelmed hospitals may fail to meet community needs. Strained resources must be targeted to provide the sickest patients with the highest levels of care, while diverting others to outpatient protocols. Triage is imperative, and doctors face nightmarish decisions of allocating ventilators (1). Only on the battlefield is it so important to gauge the illness severity and trajectory of multiple patients simultaneously.

The first question we hear clinicians asking on arrival in a COVID-19 unit is, “Who is the sickest patient?” They ask because the illness can rapidly lead to lung failure, recognized in the COVID-specific Surviving Sepsis Guidelines that include the need for monitoring of incipient respiratory failure (2). Remarkably, there is but a single Best Practice Statement: “In adults with COVID-19 receiving non-invasive positive-pressure ventilation or high-flow nasal cannula, we recommend close monitoring for worsening of respiratory status, and early intubation in a controlled setting if worsening occurs.”

In the heart of this pandemic, what does “close monitoring for worsening of respiratory status” mean? Can we look to familiar illness severity scores for help?

In 1981, Knaus et al (3) introduced the Acute Physiology and Chronic Health Evaluation (APACHE) score and, with it, the durably appealing idea that a single number could inform on how sick an ICU patient was. The score grew with the times, evolving from pencil and paper, a tedious look at the first 24 hours, and weights decided upon by experts to a computerized, automated product founded on statistical analyses of many patients. Indeed, 1985’s APACHE-II (4) was more accurate than the Sequential Organ Failure Assessment score (5), Confusion, bUn, RR (respiratory rate), BP (blood pressure), age>65 score (6), and National Early Warning Score (7, 8) in assessing COVID-19 patients in the ICU of the Tongji Hospital in China (9, 10).

These scores, however, were not devised for illnesses like COVID-19 that can lead to rapidly accelerating lung failure. Most use only measurements made on the first day (11). Their dependence on values that are measured only when a clinician thinks to, like vital signs or laboratory tests, makes them sluggish with respect to the pace of the disease. They allow the illness a headstart that can be impossible to catch up to. Whatever advantage they offer in the calibrated synthesis of many kinds of information, they lose with their pace or lack of it.

There are other misalignments. As shown in Table 1, the targets that current scores are trained to detect are diffuse and include death (in the hospital [44] or for any cause up to a year later [29]), cardiac arrest (31), sepsis (34), septic shock in the ICU (24), hemorrhage (38, 45), and readmission (44). Their inputs are often intermittent, slowly moving, or static predictors. Their weighting of values and ranges is sometimes based on expert opinion from the pre-COVID-19 era. Their scoring ranges are often nonintuitive. Their impacts have often been untested even in non-COVID-19 settings. We note that trials that used triggered alerts rather than continuous displays have had, at best, mixed results (18, 19, 35, 46–48). These were APACHE-like tools and statistical models based on measured values taken when clinicians thought they needed them.

TABLE 1.

Illness Severity Scoring Systems

Name	Clinical Target	Sample Size	Years	Inputs	Strategy	Range	Impact	Original Citation(s)
Acute Physiology and Chronic Health Evaluation	Death in ICU	I: 805II: 5,815III: 17,440IV: 110,558	1981–2003	L, VS, D, C, and GCS	O, R	I: 0–130II: 0–71III: 0–299IV: 0–286	No trials	(3, 4, 12, 13)
Systemic inflammatory response syndrome	Death	0	1992	VS and L	O	0–4	RCT: negative (14, 15)	(16)
Sequential Organ Failure Assessment	Multiple organ failure	0	1996	VS, L, M, and GCS	O	0–24	No trials	(5)
Risk of Infection to Severe Sepsis and Shock Score	Infection to severe sepsis/shock	1,531	1997–1998	VS, L, C, and S	R	0–49	No trials	(17)
Oxford BioSignals/Visensia	None	150	2001–2003	VSa	+	0–10	RCT (18) and A/B (19) studies: no impact on mortality	(20)
Insight	Sepsis	1,394	2001–2007	VS and age	+	Not given	RCT (21) and A/B (22): mortality reduction and fewer readmissions	(23)
Targeted Real-time Early Warning Score	Septic shock in ICU	13,014	2001–2007	VSb, L, D, and C	+	0–1	No trials	(24)
Long short-term memory	Septic shock in ICU	50,373	2001–2012	VSb, L, D, and C	+	Not given	No trials	(25)
Heart rate characteristics index	Sepsis	316	2003	VSa and WF	R	0–6-foldX	RCT (26): mortality reduction	(33)
Etiometry	Inadequate oxygen delivery	0	2015	VSa	+	0–100	No trials	(28)
Rothman index	Death next 12 mo	22,265	2004	VS, L, and N	+	–91 to 100	No trials	(29)
Early warning score	ICU transfer	19,116	2007–2010	VSa, L, N, C, D, and M	R, +	0–1	RCT: (30) negative	(30)
(e)CART	Cardiac arrest in hospital	CART: 47,427eCART: 269,999	2008–2013	VS and L	R, +	0–1,000	No trials	(31, 32)
National Early Warning Score	Acute-illness severity	0	2012	VS, M, and alert, verbal, pain, unresponsive	O	0–20	RCT (33): negative	(7, 8)
Artificial intelligence sepsis expert	Sepsis	27,527	2013–2015	VSa, L, D, C, organ system scoring, and M	+	0–1	No trials	(34)
Continuous Monitoring of Event Trajectories	Sepsis, death, hemorrhage, intubation, and transfer to ICU	60,986	2013–2020	VSa, L, and WF	R	0–6-foldX	A/B: (37)Reduced septic shock	(37–42)
Ambient Clinical Aware	Severe sepsis	587	2015	VS, L, M	+	0–1	No trials	(43)
Google	Death in hospital, length of stay, and readmissions	126,000	2018	VS, L, D, C, N, and M	+		No trials	(44)

(e)CART = electronic Cardiac Arrest Risk Triage, + = other mathematical methods, A/B = before and after comparison, C = comorbidities, D = demographics, FoldX = fold-increase in risk compared with average, GCS = Glasgow Coma Scale, L = labs, M = medications, N = nursing notes, O = opinion, R = regression, RCT = randomized clinical trial, S = organ system scoring, VS = vital signs, WF = waveforms (continuous data inputs).

^aVS when recorded by nurses: q s.

^bVS when recorded by nurses: q 1 min.

We live, though, in the era of Artificial Intelligence and Big Data, and the promise of clinical decision support for bedside clinicians based on automated mathematical analysis of streaming data is known to us all. In addition, continuous cardiorespiratory monitoring is readily available in every ICU and many acute care ward settings. We have the appealing opportunity to analyze mathematically the voluminous continuous cardiorespiratory monitoring data to detect early signs of patient deterioration. The effort to collect, store, and analyze the 150 MB of data per patient per day seems worth the result—a real-time continuous assessment of patient status. In the rapidly moving world of COVID-19 patients, this makes more clinical sense than awaiting the results of nurse visits and blood draws.

In the exercise of predictive analytics monitoring, we seek pathophysiological signatures of illness (38, 49, 50). In the canonical example of respiratory sinus arrhythmia (51), the well-known coupling of the heart rate and the respiratory rate by the vagus nerve has long been taken as an index of health. In this context, the signature of health is slowing of the heart rate during expiration and speeding during inspiration, and the signature of illness is their absence. Godin et al found reduced heart rate variability in volunteers injected with endotoxin (52) and concluded that systemic inflammation uncoupled the heart and lungs, and presumably uncoupled others, leading to multiple organ dysfunction syndrome (53). A comprehensive modern view is that many organs are coupled in physiologic networks (54, 55) that can be modulated during sleep and illness.

Signatures differ from illness to illness, from hospital unit to hospital unit, and across the spectrum of age. In septic neonatal ICU (NICU) premature infants, for example, we identified the unique signature of abnormal heart rate characteristics (reduced variability and transient decelerations) hours prior to clinical presentation (56). A heart rate characteristic index (27) based on novel mathematical analytics (49, 57–59) led to a continuous display of the fold-increase in the risk of neonatal sepsis in the next 24 hours (26, 50, 60). In the largest randomized trial in neonatology, the display led to a more than 20% relative reduction in death in nine NICUs (26), a durable effect (61) mostly attributable to a reduction in deaths from sepsis (62).

Although this illness signature holds for several neonatal illnesses, the same is not true for adults (38). For example, the physiologic signature of acute respiratory acute failure differed from that of hemorrhage in adult ICUs. In addition, although these two illness signatures were similar in our medical and surgical ICUs, the signatures of sepsis in the two units differed—in the surgery ICU, sepsis presented more like respiratory failure, and in the medical ICU, more like circulatory shock. A display that we devised for other ICUs and wards—Continuous Monitoring of Event Trajectories—which reports two risks, an x,y plot of the 3-hour trajectory of the fold-increase in risk of a respiratory event as a function of the fold-increase in risk of a cardiovascular one, led to a 50% reduction of the rate of septic shock in a surgical and trauma ICU (37, 63).

On one of our hospital floors, the finding was the same—signatures of the most common reasons for patient deterioration leading to ICU transfer differed greatly from one another, and no single predictive model sufficed (64). For example, a model trained on all the ICU transfer events did not outperform the strategy of using multiple models, each of which was tuned to clinical deterioration scenarios specific to a hospital ward.

How should we monitor COVID-19 patients? Since the illness has physiologic features similar to other forms of viral sepsis (65) and acute respiratory distress syndrome (ARDS) (66), we might use predictive analytics monitoring models trained on patients who, on individual chart review, had sepsis using Surviving Sepsis Campaign criteria, or respiratory failure leading to emergent intubation as documented by procedure notes from attending anesthesiologists (38, 39). We note the recent finding that cytokine levels in patients with COVID-19 plus ARDS are lower than those in patients with sepsis plus ARDS (67), consistent with the clinical picture of primary respiratory deterioration. We propose that it may be better to follow lung function than to follow the markers of systemic inflammation in the blood.

Following lung function, like looking for signatures of illness, in our view requires continuous recording of organ function: the more highly resolved, the better. Pinsky et al recently demonstrated the additional information of noninvasive and invasive heart rate and waveform data in early detection of hemorrhage in pigs (68, 69), affirming clinical studies (45, 70). Heart rate analysis is directly applicable to clinical practice—each heartbeat sends an easily detected signal and allows for detailed analysis of long time-series of interbeat intervals using new and old mathematics (71, 72). A wealth of techniques have been applied in the time domain (73), frequency domain (74), and nonlinear dynamical domain (57, 58), and many machine learning tools from multivariable logistic regression (75) to artificial neural networks (76, 77) have long been used to combine the results.

Authoritative sources (78, 79) and clinical users (60, 63) have outlined what is required of clinical decision support in the era of artificial intelligence and of predictive analytics monitoring (Table 2). In the table, we propose how the new continuous predictive analytics monitoring systems can realize these requirements. Here, we add four principles that we believe to be of equally paramount importance to an effective monitoring system.

TABLE 2.

Requirements and Realizations of Clinical Decision Support Using Predictive Analytics Monitoring

Requirement	Realization
Authoritative sources
(78)
Black boxes are unacceptable	Guidelines for reporting studies (80–82)
Time is a scarce resource	No user keystrokes required
Complexity and lack of usability thwart use	Simple, intuitive displays
Relevance and insight are essential	Made by clinicians for clinicians
Delivery of knowledge and information must be respectful	Suggestions about patients that clinicians might wish to see next; no mandates for action
Scientific foundation must be strong	Models that are trained on events identified by clinicians
(79)
Provide measurable value in addressing a recognized problem area or area for improvement	Reduced mortality in premature infants (26) and reduced septic shock in adults (37)
Leverage multiple data types to bring the most current and relevant evidence to bear on clinical decisions	Use of all data inputs: labs, vital signs, and cardiorespiratory monitoring (40, 70)
Produce actionable insights from multiple data sources	Indications of respiratory vs cardiovascular vs other forms of instability
Deliver information to the user that allows the user to make final practice decisions	Indication of instability, not a diagnostic test
Demonstrate good usability, including clear displays	Simple, intuitive displays
Are testable in small settings with scalability	(26, 37)
Support quality and value improvement initiatives	(60, 63)
Clinical users
(63)
Understand the science	Publications on the algorithm development and validation (Table 1)
Trust the inputs	Data preprocessing to remove noise
Integrate into the EHR	Treat as a vital sign
Optimize clinical pathways	Change from reactive to proactive approaches
(60)
Reduce complexity	Provide guidelines for engaging with predictive analytics monitoring
Enhance compatibility	Align tasks with clinician experience
Foster trialability	Promote observation and association
Increase observability	Respected leaders serve as examples
Demonstrate relative advantage	Case examples

• 1) Predictive analytics monitoring for clinical decision support for rapidly moving illnesses should incorporate continuous cardiorespiratory monitoring in the ICU and on the floor when it is available, because it adds information to nurse-charted vital signs and laboratory tests (45, 68–70).

• 2) Predictive analytics monitoring models should be trained on specific targets, because there is no one-size-fits-all model (38, 64).

• 3) Clinical events that are used for training predictive analytics monitoring models should be identified by clinicians, because they are more accurate than computer searches of clinical databases (70, 83–87).

• 4) These new kinds of clinical information require new tools and methods for implementation and integration (60, 63, 88).

All of these elements are directly relevant to the problem of COVID-19 respiratory failure. First, patients presenting for acute flu-like illnesses have diagnoses ranging from common viral infection to potentially catastrophic COVID-19 respiratory failure. Just as high-risk scores might predict severe illness and lead to admission to a hospital floor or ICU (40), low-risk scores might predict benign courses and identify patients who can be treated at home. Second, COVID-19 patients admitted to wards can benefit from prediction of rapid, severe pulmonary failure occurring several days into the illness. Third, COVID-19 patients in ICUs treated noninvasively might benefit if predictive monitoring shows risk, allowing them to avoid intubation as they begin to improve on their own. In addition, novel therapies like the antiviral remdesivir and the interleukin-6 receptor antagonist tocilizumab are precious resources and should be reserved for the patients predicted to be at most need. Predictive analytics monitoring can help identify them before the illness is too far advanced. Finally, the illness is very fast-moving, and there is an urgent need to know if patients respond to a course of therapy so that a failing therapy can be quickly stopped and new ones substituted.

To conclude, COVID-19 infection—like other subacute potentially catastrophic illness—can cause rapid clinical deterioration for which early detection might improve outcomes. Volitional measurements of vital signs and labs can come too late. Predictive analytics monitoring that incorporates continuous cardiorespiratory monitoring data and uses targeted analytics that detect specific signatures of individual illnesses fit the clinical need better. Like all clinical decision support, effective predictive analytics monitoring requires intuitive and actionable displays of patient trajectories. It is time to advance these modern tools to the bedside.

ACKNOWLEDGMENTS

We thank Jessica Keim-Malpass, Rebecca Kitzmiller, and Liza Prudente Moorman for suggestions and insights about implementation of predictive analytics monitoring. We also thank our colleagues in the Center for Advanced Medical Analytics, in particular Prof. DE Lake, Sepsis Challenger, for many discussions and insights on predictive analytics monitoring.