Use of a Risk Analytic Algorithm to Inform Weaning From Vasoactive Medication in Patients Following Pediatric Cardiac Surgery

Supplemental Digital Content is available in the text.

OBJECTIVES:

Advanced clinical decision support tools, such as real-time risk analytic algorithms, show promise in assisting clinicians in making more efficient and precise decisions. These algorithms, which calculate the likelihood of a given underlying physiology or future event, have predominantly been used to identify the risk of impending clinical decompensation. There may be broader clinical applications of these models. Using the inadequate delivery of oxygen index, a U.S. Food and Drug Administration-approved risk analytic algorithm predicting the likelihood of low cardiac output state, the primary objective was to evaluate the association of inadequate delivery of oxygen index with success or failure of weaning vasoactive support in postoperative cardiac surgery patients.

DESIGN:

Multicenter retrospective cohort study.

SETTING:

Three pediatric cardiac ICUs at tertiary academic children’s hospitals.

PATIENTS:

Infants and children greater than 2 kg and less than 12 years following cardiac surgery, who required vasoactive infusions for greater than 6 hours in the postoperative period.

INTERVENTIONS:

None.

MEASUREMENTS AND MAIN RESULTS:

Postoperative patients were identified who successfully weaned off initial vasoactive infusions (n = 2,645) versus those who failed vasoactive wean (required reinitiation of vasoactive, required mechanical circulatory support, renal replacement therapy, suffered cardiac arrest, or died) (n = 516). Inadequate delivery of oxygen index for final 6 hours of vasoactive wean was captured. Inadequate delivery of oxygen index was significantly elevated in patients with failed versus successful weans (inadequate delivery of oxygen index 11.6 [sd 19.0] vs 6.4 [sd 12.6]; p < 0.001). Mean 6-hour inadequate delivery of oxygen index greater than 50 had strongest association with failed vasoactive wean (adjusted odds ratio, 4.0; 95% CI, 2.5–6.6). In patients who failed wean, reinitiation of vasoactive support was associated with concomitant fall in inadequate delivery of oxygen index (11.1 [sd 18] vs 8.9 [sd 16]; p = 0.007).

CONCLUSIONS:

During the de-escalation phase of postoperative cardiac ICU management, elevation of the real-time risk analytic model, inadequate delivery of oxygen index, was associated with failure to wean off vasoactive infusions. Future studies should prospectively evaluate utility of risk analytic models as clinical decision support tools in de-escalation practices in critically ill patients.

Critically ill pediatric patients with congenital and acquired cardiovascular disease have high rates of inhospital cardiac arrest and death (1). Clinical decision support tools aimed at identifying cardiac patients at high risk of clinical deterioration (2–6) have been described in the literature, including Cardiac Children’s Hospital Early Warning Score and Pediatric Index of Cardiac Surgical Intensive Care Mortality (7). Recent progress in acquisition, storage, and analysis of physiologic data streams have facilitated the development of newer high-frequency data monitoring and predictive analytic algorithms (8–11). Using this high-frequency physiologic data, several groups developed near real-time risk assessment algorithms for infants with single ventricle congenital heart disease (CHD), which have retrospectively identified patients at increased risk of cardiopulmonary decompensation (8, 9, 11). One of these risk models, the U.S. Food and Drug Administration (FDA) 510(k) cleared inadequate delivery of oxygen index (IDo₂) algorithm (Etiometry, Boston MA), uses available physiologic and laboratory variables to compute in near real-time the probability of the mixed venous oxygen saturation (Svo₂) lower than 40%. Elevated IDo₂ is associated with increased risk of cardiac arrest in neonates following surgical repair of CHD (9).

Predictive analytic algorithms such as IDo₂ may have utility beyond forecasting catastrophic events. By continuously analyzing real-time patient data to determine risk of underlying unstable physiology, these algorithms may conversely inform patient stability during de-escalation of care such as the weaning of vasoactive drugs and inotropes. By improving the efficiency of care de-escalation, patients may have shorter exposure to the ICU environment, shorter lengths of stay, and globally improved outcomes (12–14).

The purpose of this study is to explore the association of IDo₂ and vasoactive support weaning during de-escalation of care following pediatric cardiac surgery. The primary aim is to evaluate the association of IDo₂ in successful versus failed discontinuation of vasoactive support. The secondary aim of the study is to explore the impact of re-escalation of vasoactive support upon IDo₂ parameters.

MATERIALS AND METHODS

Design, Setting, and Patients

This is a multicenter retrospective cohort analysis of patients 0 days to 12 years and greater than 2 kilograms admitted to the cardiac ICU (CICU) following cardiac surgery at three tertiary care medical centers, Boston Children’s Hospital (Boston, MA), Children’s National Hospital (Washington, DC), and St. Louis Children’s Hospital (St. Louis, MO), between January 1, 2011, and December 31, 2019. Patients were included who received a postoperative vasoactive infusion (dopamine, epinephrine, norepinephrine, milrinone, or vasopressin) for at least 6 hours, at any point in their CICU admission, and that vasoactive infusion was discontinued prior to CICU transfer, discharge, or death. Patients were excluded who did not require postoperative vasoactive infusion, received a vasoactive infusion for fewer than 6 hours, or those who had a major complication (cardiac arrest, required extracorporeal membrane oxygenation [ECMO], renal replacement therapy [RRT]) prior to discontinuation of vasoactive infusions, or were transferred, discharged, or died prior to discontinuation of vasoactive infusions. Cases where IDo₂ data were available for less than half of the time interval were excluded from the analysis.

Physiologic data were captured and stored on secure servers using the Etiometry platform. Demographic, diagnostic, procedural, and CICU outcome data were compiled from local institutional databases. Surgical complexity was measured by the Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery (STAT) categorical score (15). This multisite study was verified as exempt (Secondary Research Uses of Data or Specimens) by the primary institutional review board (IRB) site at Boston Children’s Hospital (Boston Children’s Hospital IRB-P00028501).

Risk Analytic Algorithm

For this study, the risk analytic algorithm IDo₂ was used. IDo₂ is a naive Bayesian risk model calculating the likelihood of inadequate systemic oxygen delivery (defined as a Svo₂ < 40%) (10). It uses near-continuous physiologic data and intermittent laboratory data to calculate and graphically display IDo₂ index within the T3 platform. IDo₂ is recalculated and updated every 5 seconds. Compared with the gold standard of intermittently measured central venous oxygen saturation, IDo₂ area under the receiver operating characteristic curve (AUC) is 0.89 (0.88–0.90) (Appendix A, http://links.lww.com/CCX/A829). IDo₂ received FDA 510(k) clearance during the study period (March 2015 for infants up to 1 yr and June 2017 for patients 1–12 yr). After these dates, IDo₂ was available for display on the T3 platform. Prior to these dates, IDo₂ was not available in real-time and was retrospectively calculated using deidentified data. No institution had a standard method of IDo₂ display, IDo₂ utilization, nor standardized vasoactive weaning protocol over the study period. The Appendix (http://links.lww.com/CCX/A829) contains the data elements included in the IDo₂ calculation and further details about the collection and preprocessing of data including IDo₂ calculation, approach to artifacts, level of monitoring, and missing physiologic data.

Study Definitions

Episodes of vasoactive weaning and discontinuation were identified from the medical record. For this study, a “vasoactive infusion” is any continuous infusion of epinephrine, dopamine, dobutamine, milrinone, vasopressin, or norepinephrine started prior to admission (in the operating room) or during this admission and continuously maintained for more than 6 hours at any point during the postoperative course. “Vasoactive infusion discontinuation event” is the instance recorded in the medical record when the final remaining vasoactive infusion was discontinued. A “successful discontinuation event” is defined as an episode in which the final remaining vasoactive infusion was discontinued and not restarted at any subsequent point during the hospitalization. A “failed discontinuation event” is an episode where, after vasoactive infusion discontinuation but prior to CICU transfer/discharge, a vasoactive infusion was restarted or the patient experienced a major complication (cardiac arrest, ECMO, RRT, or death) (Fig. 1). Patients with failed discontinuation events were not reanalyzed for a subsequent successful event.

Figure 1.

Study design schematic of successful and failed vasoactive infusion wean “Patient A” successfully weaned off milrinone infusion, which was not restarted during same hospitalization. “Patient B” required reinitiation of dopamine infusion after initially weaning off vasoactive infusions. Inadequate delivery of oxygen index (IDo₂) is evaluated only in the final 6-hr time period of vasoactive infusion (blue brackets). CICU = cardiac ICU.

As IDo₂ represents a dynamic physiologic state, we prospectively calculated the total exposure or “dose” of IDo₂ as the area under the IDo₂ curve computed over a 6-hour window preceding the event (9).

Primary Analysis: Discontinuation of Vasoactive Support

For the primary analysis, we aimed to find an association between IDo₂ and failed discontinuation events. For each patient, the dose of IDo₂ was assessed for the 6-hour window preceding a vasoactive infusion discontinuation event. A 6-hour window was selected in study design, as it represented a reasonable time period to assess patient clinical stability during vasoactive infusion wean. To assess alternative characteristics of IDo₂ during the 6-hour window preceding the discontinuation that may be better associated with vasoactive wean failure, high total dose of IDo₂ (defined as a mean 6-hr IDo₂ > 50), excessive IDo₂ variability (defined as sd > 20), and prolonged exposure to moderately elevated IDo₂ (IDo₂ > 25 for > 50% of weaning time) were compared between the successful and failed weaning groups.

Secondary Analysis: Reinitiation of Vasoactive Support

The goal of the secondary analysis was to explore the association of IDo₂ and reinitiation of vasoactive support within the failed wean cohort. Patients who initially weaned off all vasoactive infusions and then subsequently had a vasoactive infusion restarted were analyzed. Specifically, patients were included who were on postoperative vasoactive infusions for at least 6 hours, then discontinued for a period of at least 6 hours, and then restarted for a period of at least 6 hours (Fig. 2). IDo₂ dose was calculated and compared between the 6-hour period off all vasoactive agents and then again 6 hours after the reinitiation of a vasoactive agent.

Figure 2.

Illustration of reinitiation of vasoactive support. Patient has weaned off initial postoperative vasoactive support but develops clinical indications to restart a vasoactive infusion. To compare how inadequate delivery of oxygen index (IDo₂) reflects physiologic change during these times, IDo₂ is captured in the final 6 hr off vasoactive infusions and compared with a 6-hr time period after restarting vasoactive infusions. We treat the 6 hr immediately following as a censor period to be ignored and then adopt the 6 hr following this censor as the “after” period.

Statistical Analysis

Normally distributed continuous data are reported as mean (sd). Non-normally distributed continuous data are reported as median (interquartile range [IQR]); categorical data as frequency (%). Univariate comparisons were performed using the Fisher exact test for categorical variables and the Wilcoxon rank-sum test and Moods median test for continuous variables. All analyses were performed using custom scripts written in the Python programming language. Packages used include pandas (a toolkit for managing and processing time series data), SciPy (a toolkit that provides statistical analysis methods), and scikit-learn (a toolkit that provides machine learning and additional statistical methods for AUC analysis) (16–18).

Total IDo₂ dose between successful and failed vasoactive discontinuation groups was compared by univariate and multivariate analyses, controlling for significant univariate associations. In the multivariable analysis, we employed subsample matching to create a roughly 3:1 match of successful and failed vasoactive wean, matched by STAT category. We further generated 100 bootstrappings of this subsample matching in order to more fully explore the successful vasoactive wean group. To compare the distributions of these IDo₂ metrics between successful and failed weans, odds ratio (OR) were calculated.

RESULTS

A total of 3,161 unique patient encounters were identified, with 2,645 successful and 516 failed vasoactive discontinuation events identified over the 9-year study period. Characteristics of the study population are reported in Table 1. The median age was 5.4 months (IQR, 1–28 mo) and median weight was 7.5 kg (IQR, 4.2–14 kg). The most frequently used vasoactive infusions were milrinone (n = 1,689, 53%), dopamine (n = 1,299, 41%), and epinephrine (n = 213, 5%). Most patients required one vasoactive infusion (n = 1,404, 44%), with fewer requiring two vasoactive (n = 1,139, 36%) or three or more vasoactive infusions (n = 618, 20%). Patients were on vasoactive infusions for a median of 48 hours (21–107 hr) prior to discontinuation. After subsample matching to balance the surgical complexity distribution between successful and failed weans, 1,732 patients remained in the successful wean group. Of the 516 patients who failed a vasoactive wean, n = 9 (1.7%) had a cardiac arrest, n = 9 (1.7%) required ECMO, n = 3 (0.6%) required RRT, and n = 38 (7.4%) died.

TABLE 1.

Characteristics of the Study Population

Variable	All Patients (n = 3,161)	Successful Wean (n = 2,645)	Failed Wean (n = 516)	p a
Maleb	1,509 (56%)	1,247 (56%)	262 (57%)	0.99
Age, mo	5.4 (1.1–28)	5.5 (1.1–28)	4.5 (0.7–25)	0.06
Weight, kg	4.5 (3.3–9.0)	4.6 (3.3–9.1)	4.2 (3.2–8.5)	0.04
Neonates (< 1 mo)	770	634	136	0.31
Infants (1–12 mo)	1,212	1,010	202	0.75
Children (1–12 yr)	1,179	1,001	178	0.25
Institution
Boston Children’s	2,660	2,221	439	0.80
Children’s National Medical Ctr	273	247	26	0.002
Washington University in St. Louis	228	177	51	0.01
STAT category
STAT 1	651 (21%)	584 (22%)	67 (13%)	< 0.001
STAT 2	810 (26%)	679 (26%)	131 (25%)	0.91
STAT 3	529 (17%)	448 (17%)	81 (16%)	0.53
STAT 4	926 (29%)	745 (28%)	181 (35%)	0.008
STAT 5	245 (8%)	189 (7%)	56 (11%)	0.006
Diagnosis
Single ventricle	435	360	75	0.62
Two ventricle	2,726	2,285	441	0.84
Vasoactive agentsc
Dopamine	1,299 (41%)	1,066 (40%)	233 (45%)	0.12
Milrinone	1,689 (53%)	1,472 (56%)	217 (42%)	< 0.001
Epinephrine	213 (7%)	121 (5%)	92 (18%)	< 0.001
Norepinephrine	51 (2%)	30 (1%)	21 (4%)	< 0.001
Vasopressin	11 (0%)	8 (0%)	3 (1%)	0.33
Dobutamine	0	0	0
Outcomes
Cardiac ICU length of stay	5.7 (2.9–11)	4.8 (2.5–9.0)	13 (6.7–26)	< 0.001
Hours of vasoactive support	48 (21–107)	47 (21–102)	56 (21–139)	0.07

STAT = Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery mortality category.

^aFor continuous variables, p values are derived from Mood’s median test that represents the probability that medians of the successful and failed wean groups are the same. For categorical variables, p values are derived from a χ² test.

^bEighty-five percent of patients had gender information available.

^cIncludes any episode of receiving a vasoactive infusion includes patients on multiple infusions. Total % is greater than 100%.

Values are n (%) and median (interquartile range).

Primary Analysis

The IDo₂ dose in the 6 hours preceding failed vasoactive infusion discontinuation was significantly higher when compared with the successful wean group (11.6 ± 19.0 vs 6.4 ± 12.6; p < 0.001). The distribution of IDo₂ dose for the successful and failed vasoactive wean groups is shown in Supplemental Figure 1 (http://links.lww.com/CCX/A826). Both groups are dominated by low IDo₂; Pearson’s skewness is 1.4 for failed weans and 1.3 for successful weans, demonstrating a significant skewed distribution of IDo₂ toward low values. The odds of failing a vasoactive wean versus increasing IDo₂ dose are plotted in Figure 3. There is a positive correlation between increasing IDo₂ dose and the odds of failed vasoactive wean.

Figure 3.

Plot of mean inadequate delivery of oxygen index (IDo₂) in 6 hr prior to vasoactive discontinuation and odds ratio for failed vasoactive infusion wean.

Alternative characteristics of IDo₂ were examined to determine the associations with vasoactive infusion discontinuation failure (Table 2). A 6-hour “high dose” of IDo₂ (defined as mean 6-hr IDo₂ > 50) had the highest association with weaning failure (adjusted OR, 4.0; 95% CI, 2.5–6.6). Additionally, increased IDo₂ variability was associated with failed vasoactive discontinuation (OR, 2.2; 95% CI, 1.4–3.6). Prolonged exposure to moderately elevated IDo₂ (IDo₂ > 25 for > 50% of weaning period) was associated with higher odds of failed discontinuation (OR, 2.0; 95% CI, 1.5–2.96).

TABLE 2.

Characteristics of Inadequate Delivery of Oxygen Index Preceding Vasoactive Discontinuation and Association With Weaning Failure

IDo2 Characteristic	Threshold	Adjusted OR (95% CI)a
High total dose IDo2	Mean 6-hr IDo2 > 50	4.0 (2.5–6.6)
IDo2 variability	sd > 20	2.2 (1.4–3.6)
Prolonged exposure to moderately elevated IDo2	IDo2 > 25 for > 50% of weaning time	2.0 (1.5–2.9)

IDo₂ = inadequate delivery of oxygen index, OR = odds ratio.

^aORs are adjusted for Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery mortality category.

Secondary Reinitiation of Vasoactive Support

From the original cohort, we identified 132 patients who initially weaned off all vasoactive support and subsequently were reinitiated on a vasoactive infusion. Characteristics of this population are described in Supplemental Table 1 (http://links.lww.com/CCX/A828). IDo₂ was significantly lower in patients 6 hours after restarting vasoactive infusions compared with the 6 hours prior to reinitiating vasoactive infusions (IDo₂ 8.9 [±16] vs 11.1 [±18]; p = 0.007) (Supplemental Fig. 2, http://links.lww.com/CCX/A827). Within this cohort, reinitiation of vasoactive infusions was associated with a fall in mean 6-hour IDo₂ in 75% of patients (p < 0.001).

DISCUSSION

In this multicenter retrospective cohort analysis of postoperative pediatric cardiac surgical patients, we found that an increased dose of IDo₂ in the 6 hours preceding discontinuation of vasoactive infusion was associated with discontinuation failure. In addition, high doses of IDo₂ in the weaning period, elevated IDo₂ variability, and prolonged exposure to moderately elevated IDo₂ were all associated with discontinuation failure. For those patients who failed vasoactive infusion wean, reinitiation of a vasoactive medication was correlated with a concomitant fall in IDo₂. This further suggests that IDo₂ may reflect underlying patient stability and the rescue of deteriorating physiology leads to a lower IDo₂.

Prior studies have demonstrated the utility of predictive analytics algorithms in the identification of patients at risk for instability or a catastrophic event (8, 9, 11, 19). To the best of our knowledge, this study is one of the first to suggest predictive analytics algorithm such as IDo₂ may be a valuable tool in the de-escalation of patient care. Current workflows require the clinician to monitor for improvements in vital signs, physical examination, and laboratory testing prior to and during de-escalating support. This process may be otherwise delayed as limited resources are allocated to the most acutely ill patients in a complex ICU environment. Weaning decisions may be delayed by variability in provider experience and opinions, fatigue, inattention, or information overload. Informed by predictive analytics algorithms, clinical decision support tools aimed at efficient de-escalation of care may result in shorter ICU length of stay, reduced rates of nosocomial infection, decreased sedative exposure, better neurodevelopmental outcome, and lower healthcare costs (12, 20–22).

Predictive analytic algorithms may be able to identify patients with earlier, subclinical signs of instability (8, 9, 11, 19). Our study suggests that predictive analytic algorithms such as IDo₂ may have utility in identifying patients at risk for deteriorating physiology during the de-escalation period of postoperative management (12, 20, 21). This clinical decision support tool has the potential to deliver relevant higher-order information on patient stability. When incorporated into a robust clinical decision support protocol, IDo₂ may be able to inform earlier weaning decisions and therefore improve on the accuracy and efficiency of care de-escalation (12, 13, 21, 23, 24). Future studies should aim to assess the clinical relevance of risk analytic algorithms during the de-escalation phases of care. We would suggest prospectively assessing the risk analytic algorithm as a part of a standardized de-escalation plan.

This study demonstrated the feasibility of using a common risk analytic tool across a heterogeneous cohort of postoperative cardiac surgical patients. Using the Etiometry platform, high-frequency physiologic data and risk analytic output could be analyzed among multiple institutions (25).

The results of this study are limited by its retrospective nature, which meant we could not track the real-time decision-making and human factors influencing patient management. Care de-escalation is not standardized at any of the three study centers, thus subject to multiple clinical and nonclinical variables (12). The initiation and use of respiratory support, sedation, and feeding can all alter the outcome of a vasoactive infusion wean, and we did not account for these variables. Variability in weaning practices between providers and among institutions may additionally confound results. IDo₂ received FDA clearance and became clinically available for infants and children in year 5 and year 7 of this retrospective 9-year study, respectively. As such, prior to these times, practitioners were blinded to IDo₂ in caring for patients. After, IDo₂ was available to clinicians; however, no institution regularly used IDo₂ in de-escalation of care protocols.

CONCLUSIONS

The real-time streaming risk analytic algorithm, IDo₂, successfully discriminated between successful and unsuccessful weans of vasoactive agents. To the best of our knowledge, this analysis is the first using a streaming analytic algorithm to find an association between success or failure of a de-escalation of patient care. The findings in this study are limited by retrospective design, lack of several important patient characteristic details, and did not capture practitioner decision making in stopping and starting inotropic agents. Future studies should prospectively evaluate streaming predictive algorithm platforms, like IDo₂, in informing clinical decisions on patient management.