A Near Real-Time Risk Analytics Algorithm Predicts Elevated Lactate Levels in Pediatric Cardiac Critical Care Patients

Abstract

BACKGROUND:

Postoperative pediatric congenital heart patients are predisposed to develop low-cardiac output syndrome. Serum lactate (lactic acid [LA]) is a well-defined marker of inadequate systemic oxygen delivery.

OBJECTIVES:

We hypothesized that a near real-time risk index calculated by a noninvasive predictive analytics algorithm predicts elevated LA in pediatric patients admitted to a cardiac ICU (CICU).

DERIVATION COHORT:

Ten tertiary CICUs in the United States and Pakistan.

VALIDATION COHORT:

Retrospective observational study performed to validate a hyperlactatemia (HLA) index using T3 platform data (Etiometry, Boston, MA) from pediatric patients less than or equal to 12 years of age admitted to CICU (n = 3,496) from January 1, 2018, to December 31, 2020. Patients lacking required data for module or LA measurements were excluded.

PREDICTION MODEL:

Physiologic algorithm used to calculate an HLA index that incorporates physiologic data from patients in a CICU. The algorithm uses Bayes’ theorem to interpret newly acquired data in a near real-time manner given its own previous assessment of the physiologic state of the patient.

RESULTS:

A total of 58,168 LA measurements were obtained from 3,496 patients included in a validation dataset. HLA was defined as LA level greater than 4 mmol/L. Using receiver operating characteristic analysis and a complete dataset, the HLA index predicted HLA with high sensitivity and specificity (area under the curve 0.95). As the index value increased, the likelihood of having higher LA increased (p < 0.01). In the validation dataset, the relative risk of having LA greater than 4 mmol/L when the HLA index is less than 1 is 0.07 (95% CI: 0.06-0.08), and the relative risk of having LA less than 4 mmol/L when the HLA index greater than 99 is 0.13 (95% CI, 0.12–0.14).

CONCLUSIONS:

These results validate the capacity of the HLA index. This novel index can provide a noninvasive prediction of elevated LA. The HLA index showed strong positive association with elevated LA levels, potentially providing bedside clinicians with an early, noninvasive warning of impaired cardiac output and oxygen delivery. Prospective studies are required to analyze the effect of this index on clinical decision-making and outcomes in pediatric population.

KEY POINTS

• Question: A Novel index to predict hyperlactatemia noninvasively elevated serum lactate.

• Findings: Multicenter retrospective study validating the hyperlactatemia (HLA) index. The HLA index predicts elevated lactate for pediatric patients with a high sensitivity and specificity.

• Meaning: A risk analytic algorithm predicts elevated serum lactate.

Low-cardiac output syndrome (LCOS) is frequently encountered in pediatric patients following cardiac surgery with a reported occurence of 25–60% (1). The hemodynamic instability associated with LCOS can be detrimental to a patient’s survival and clinical outcome (2, 3). Pediatric patients with congenital heart disease (CHD) are at increased risk for impaired cardiac output at variable stages of their disease. Several studies have explored the clinical predictors of LCOS for pediatric patients with CHD following surgery with cardiopulmonary bypass (CPB) (4, 5).

Serum lactic acid (LA) has been studied in patients with hemodynamic instability following cardiac surgery. Its prognostic role in identifying patients with LCOS and mortality risk has been validated (6–8). Although serum LA levels can offer insights into a patient’s physiologic condition, this requires frequent phlebotomy, which is associated with an increase in blood product transfusion (9, 10), especially in neonates with a small circulating blood volume.

This study uses a novel risk analytic algorithm to predict an elevated LA level based on near real-time patient data derived from bedside monitoring data, as well as other laboratory data. We evaluate the newly developed index and its association with elevated LA levels in 10 tertiary cardiac ICUs (CICUs) using data acquired by the T3 platform (Etiometry, Boston, MA).

MATERIALS AND METHODS

Algorithm

The hyperlactatemia (HLA) algorithm has been designed to incorporate data acquired in a near real-time manner by the T3 platform. As T3 continuously collects patient physiologic measures and laboratory test results, the algorithm uses Bayes’ theorem to interpret the newly acquired data given its own previous assessment of the physiologic state of the patient. It then uses the interpreted data to update its assessment of the patient state. The latter is represented by a set of patient physiologic variables, each of which is modeled with a probability distribution to account for patient-to-patient variation and measurement uncertainty. Physiologic variables representing the patient state are also related to one another using established relationships of human physiology. Finally, the physiologic variables are related to the physiologic measures and laboratory test results collected by T3 using models of the measurement sensors that account for potential errors in the data. These elements make up the software physiology model. Figure 1 shows the data flow of the risk analytics algorithm used to compute the HLA index. Additional details about the physiologic model and algorithm are provided in Supplemental tables, figures, and materials (http://links.lww.com/CCX/B277).

Figure 1.

Data flow is used by the risk analytics algorithm to compute the hyperlactatemia (HLA) index. BP = blood pressure, LA = lactic acid, SaO₂ = arterial oxygen saturation, ScvO_{2 =} central venous oxygen saturation.

Central venous oxygen saturation (ScvO₂), Paco₂, arterial pH (pHa), and whole blood concentration of lactate (LA) are four of the patient physiologic variables that the algorithm updates when given new data. Specifically, the algorithm maintains an estimate of the probability density of ScvO₂, Paco₂, pHa, and LA by computing the cumulative probability of LA being between 4 mmol/L and infinity.

Datasets

The HLA index is a product of a model-based approach to risk estimation, which means that the model is not fit to or learned from the data, as is the case with standard machine learning methodologies. Rather, the model is designed based on principles of physiology, and parameters are chosen to reflect those specified in medical literature. Therefore, the approach is not subject to the same risks of overfitting as machine learning techniques. However, to mitigate any risk of overfitting, the employed analysis uses development testing datasets and validation sets. Development testing sets are used to evaluate the impact of the changes during the innovation process. Validation sets are then used after all development is complete to validate that performance holds on an independent dataset. Note that because the model of the algorithms is designed, not learned, there is no need for a training dataset. The HLA was validated using a validation set that included data from 10 different clinical sites. A list of the participating centers and the dates for the validation dataset are provided in Supplemental table (http://links.lww.com/CCX/B277).

Validation Data Source and Acquisition

The data have been externally verified by the University of Alabama at Birmingham Institutional Review Board (IRB-300007877, approval date September 21, 2021, “Risk Analytics Algorithms to Predict Extubation Failure in the Cardiac Intensive Care Unit”) as being exempt at all included institutions, according to 45CFR46.101(b) (4): existing data and specimens—no identifiers, informed consent was waived. Data were acquired at each institution by the T3 Data Aggregation and Visualization software module and included monitoring data, laboratory data, and patient demographics. Data were deidentified by removing all protected health information and transferred to a central database managed by Etiometry. A query was generated to identify from the database all patients with the required data types within a particular encounter.

Analysis Preprocessing

The index was retrospectively computed in all deidentified patients. For each patient, venous or arterial serum lactate concentrations and their associated timestamps were identified. The average values of the HLA index were computed for a period of 30 minutes leading to but excluding the time stamp of LA laboratory test. The data from all patients were then collated in a table. Points and respective patients for which the HLA index was not computed due to the lack of required data were not included in the table. For the purposes of the subsequent analysis, the gold standard LA value was dichotomized into two categories: points positive for HLA corresponding to LA greater than 4 mmol/L and points negative for HLA corresponding to LA less than or equal to 4 mmol/L. Details about HLA index analysis approach, acceptance criteria, and the robustness of the HLA index are provided in Supplemental materials and tables (http://links.lww.com/CCX/B277).

RESULTS

Participating Centers and Cohort

All surgical and medical patients admitted to the CICUs in the participating centers between January 1, 2018, and December 31, 2020, were eligible for inclusion in the validation dataset (n = 4,562). Patients with no matching data were excluded (n = 1,062). Four patients were excluded due lack of serum lactate measurements. A total of 3,496 patients were included in the validation dataset with 58,168 serum lactate levels. Supplemental Figure 1 (http://links.lww.com/CCX/B277) summarizes the patient cohort used to validate the HLA index. The distribution of the points included in the study among the participating centers is summarized in Table 1.

TABLE 1.

Patients and Points Included in the Validation Analysis—Hyperlactatemia Index

Institution	Patients	Points	Points/Patient	Demographics (% Neonates % Infants % Children)
1	2	2	1.0	0 50 50
2	490	5671	11.6	27 31 42
3	569	7755	13.6	41 30 29
4	448	8,876	19.8	38 31 31
5	229	3,356	14.7	28 35 38
6	702	9,749	13.9	21 43 36
7	542	11,986	22.1	26 40 34
8	203	4,040	19.9	42 32 26
9	82	4,120	50.2	22 59 20
10	229	2,593	11.3	41 32 27
Total	3,496	58,168	16.6	31 36 33

General Performance

HLA Index as Predictor of Elevated LA

The ability of the HLA index to predict LA level greater than 4 mmol/L before a whole blood LA sample was evident in the receiver operating characteristic (ROC) curve. Figure 2 shows the ROC curve of predicting LA greater than 4 mmol/L by using the HLA index average 30 minutes before a whole blood lactate sample with high sensitivity and specificity. The area under the curve (AUC) for the full set of observations was 0.95. Supplemental Table 1 (http://links.lww.com/CCX/B277) depicts the resulting AUCs and 95% CIs. When we classified the HLA index in several categories based on its range of possible values, the probability of having LA level of more than 4 mmol/L was statistically increased with the increase of the HLA index value (p < 0.0001). The probability for LA greater than 4 mmol/L in different bins of the HLA index is summarized in Figure 3 and Supplemental Table 2 (http://links.lww.com/CCX/B277). The definition of the full set of observations is provided in Supplemental materials (http://links.lww.com/CCX/B277).

Figure 2.

Hyperlactatemia (HLA) receiver operating curves for full and minimum—validation sets. AUC = area under the curve.

Figure 3.

The risk of lactic acid (LA) is greater than 4 mmol/L for different bins of the hyperlactatemia (HLA) index.

The Relative Risk of LA in Relation to HLA Index Minimum and Maximum Values

Table 2 shows the minimum and maximum relative risk of LA greater than 4 mmol/L in relation to the HLA index value utilizing minimum and a full set of observations. In a full set of observations, the relative risk of having LA level greater than 4 mmol/L when the HLA index is less than 1 is 0.07 (95% CI, 0.06–0.08), and the relative risk of having LA level less than or equal to 4 mmol/L when the HLA index is greater than 99 is 0.13 (95% CI, 0.12–0.14). The definitions of the full set and minimum set of observations are provided in Supplemental materials (http://links.lww.com/CCX/B277).

TABLE 2.

Minimum and Maximum Relative Risk for Patients in the Validation Dataset for the Hyperlactatemia Index

Dataset	Minimum RR	LA Points Where HLA < 1	LA > 4 mmol/L Points Where HLA < 1	95% CI
Minimum	0.14	30111	410	0.13–0.15
Full	0.07	40,240	297	0.06–0.08
	Maximum RR	LA Points Where HLA > 99	LA ≤ 4 mmol/L Points Where HLA > 99
Minimum	0.15	991	132	0.13–0.17
Full	0.13	2,983	341	0.12–0.14

HLA = hyperlactatemia, LA = lactic acid, RR = relative risk.

Date presented as number.

Subpopulation Analysis

We divided patients into three groups: neonates (–28 d old), infants (from 28 d to 2 yr old), and children (from 2 yr to 12 yr old). Patients with a history of prematurity were included. The ROC AUC in both full and minimum sets of observations is shown in Table 3. The HLA index predicted a high LA with a high sensitivity and specificity. For a full set of observations; the ROC AUC was 0.9462 for neonates (95% CI, 0.9416–0.9506), 0.9533 for infants (95% CI, 0.944–0.9621), and 0.9525 for children (95% CI, 0.9495–0.9558).

TABLE 3.

Subpopulation Area Under The Curve Performance of Hyperlactatemia Index

Dataset	Population	Area Under the Curve	95% CI
Full	Neonates	0.9462	0.9416–0.9506
	Infants	0.9533	0.944–0.9621
	Children	0.9525	0.9495–0.9558
Minimum	Neonates	0.8853	0.879–0.8916
	Infant	0.8934	0.8802–0.9067
	Children	0.8884	0.8782–0.8994

DISCUSSION

Pediatric patients with congenital and acquired heart disease in the CICU often suffer hemodynamic instability and inadequate oxygen delivery in the postoperative period due to LCOS. Obtaining serial serum LA levels is key to evaluating changes in postoperative systemic oxygen delivery, and LA level greater than 4 mmol/L has been associated with higher morbidity demonstrated by longer ICU stays and mechanical ventilation duration (6). Our study validated the ability of a novel clinical index to predict lactate levels greater than 4 mmol/L in critically ill pediatric patients in the CICU with a high sensitivity and specificity.

Risk analytics algorithms and other machine learning methodologies have been explored as predictive and prognostic tools for poor outcomes or clinical worsening (11–14). For these risk analytics algorithms to be further explored and validated, making them publicly available is essential (14).

Recently, the inadequate delivery of oxygen (IDO₂) index has been described and used at many heart centers for pediatric patients following cardiac surgery (15). In a retrospective multicenter study including infants following CPB operations, Futterman et al (16) showed that a higher IDO₂ index dose over a monitoring period of 2 hours was associated with an increased risk of cardiac arrest even when censoring data 10–30 minutes before the event. Another retrospective multicenter study showed an association between higher IDO₂ index and failure to wean off vasoactive infusions in pediatric patients following cardiac surgery (17). The prospective phase of this study is underway.

Caring for pediatric patients with cardiac disease requires frequent laboratory blood draws, including arterial and venous blood gases and serum LA, to monitor their trajectory and ensure adequate oxygen delivery and recovery following heart surgery (18, 19). The initial LA level and the rate of change after surgery have been associated with outcomes (8,20,21). In our cohort, patients with LA levels greater than 4 mmol/L had higher HLA index preceding a serum LA level check. Using a large dataset across 10 pediatric CICUs, the HLA index was able to predict with high sensitivity and specificity a clinically significant elevation of LA level (> 4 mmol/L). Further analysis of our dataset shows high likelihood of having an elevated LA when the HLA index is high and low likelihood of having an elevated LA when the HLA index is low. The HLA index provides the ability to predict a patient’s LA level noninvasively. As a result, the HLA index has the potential to alert bedside clinicians to changes in clinical status, prompting escalation of care or further investigation to avoid further deterioration.

To the best of our knowledge, the HLA index is the first near real-time predictive algorithm able to predict lactate levels noninvasively in pediatric patients postcardiac surgery. The validation of this index in a more granular dataset is essential before deploying this broadly as a clinical tool.

There were some limitations to our study. These include the retrospective nature of the study, lack of granular clinical data such as vasoactive support and end-organ function, and a lack of homogeneity. For example, patients with different cardiac physiologies and pathologies (single ventricle vs. biventricle) were included to maximize the number of subjects. For the same reason, serum venous and arterial LA levels were compiled despite the discrepancy that they may have (22). Despite the low likelihood of having a false negative HLA value, the bedside clinician should not rely on the HLA index as the sole source of patient status information. The test only adds information in the context of the patient’s physiologic status in addition to other clinical and historical variables.

CONCLUSIONS

We showed the ability of the HLA index to predict a high LA. Specifically, a higher HLA index in pediatric patients admitted to the CICU was associated with higher serum LA levels. Although further study would be needed, by using readily available clinical data, the HLA index calculated by a risk analytics algorithm may reduce frequent laboratory blood draws typically used to follow serial LA levels. In addition, an elevated HLA index could potentially alert bedside staff to changes in clinical status, improving situational awareness. Changes in the continuous near real-time HLA index value may provide early warning and possibly prompt interventions to prevent further deterioration, such as cardiac arrest. Lastly, the physiology-based model that drives the index offers clinicians who are resistant to complex mathematical models driving clinical decision-making increased insight into the calculation beyond that offered by standard machine learning approaches. Prospective validation of the HLA index is necessary to test this new tool’s predictive ability and impact on patient management.

ACKNOWLEDGMENTS

We thank the Etiometry team for their essential role in sharing the algorithm used to develop this index and making it public.

Supplementary Material

cc9-5-e1013-s001.pdf ^{(366.8KB, pdf)}