Supervised Machine Learning Models Predicting Postoperative Low Cardiac Output Syndrome In Neonates

Abstract

OBJECTIVE:

To train and test supervised machine learning (ML) models to predict low cardiac output syndrome (LCOS) within the first 48 postoperative hours in neonates undergoing cardiothoracic surgery.

DESIGN:

Retrospective observational study. An efficient tree-based gradient-boosting algorithm (LightGBM) ML models were developed to predict LCOS occurrence at 2-, 4-, 6-, and 12-hour forecasting horizons, incorporating data from the prediction time and the two preceding hours. SHapley Additive exPlanations (SHAP) analyses were used for feature importance analyses.

SETTING:

Single center, January 2012 to April 2023.

PATIENTS:

Neonates 28 days old or younger who underwent cardiothoracic surgery.

INTERVENTIONS:

None.

MEASUREMENTS AND MAIN RESULTS:

A total of 181 neonates were included, with 14.9% experiencing LCOS. A multivariate time-series dataset was constructed using hourly clinical and laboratory variables recorded during the first 48 postoperative hours. The LightGBM ML models achieved area under the receiver operating characteristic curve values ranging from 0.91 to 0.98 and area under the precision-recall curve values ranging from 0.60 to 0.80 for LCOS prediction across 2-, 4-, 6-, and 12-hour forecasting horizons. SHAP analyses identified higher vasoactive inotrope score, lower urine output, and higher serum lactate as the most influential predictors.

CONCLUSIONS:

This study demonstrates that the supervised machine learning models can accurately predict LCOS in neonates, offering high interpretability. The findings support further validation in multicenter settings and integration into clinical workflows to enhance postoperative critical cardiac care neonates.

KEY POINTS

Question: Can supervised machine learning (ML) models accurately predict postoperative low cardiac output syndrome (LCOS) within the first 48 hours in neonates undergoing cardiothoracic surgery?

Findings: In this retrospective observational study of 181 neonates, an efficient tree-based gradient-boosting algorithm (LightGBM) ML models achieved strong predictive performance (area under the receiver operating characteristic curve 0.91–0.98, area under the precision-recall curve 0.60–0.80) for forecasting LCOS at 2-, 4-, 6-, and 12-hour horizons. Higher vasoactive inotrope scores, lower urine output, and elevated serum lactate significantly predicted LCOS.

Meaning: ML models reliably predict LCOS in neonates post-cardiothoracic surgery, supporting their potential integration into clinical workflows for enhanced neonatal critical cardiac care.

Low cardiac output syndrome (LCOS) occurs when there is mismatch between oxygen delivery and oxygen consumption in the post-cardiothoracic surgical patient. LCOS after cardiothoracic surgery for congenital heart disease (CHD) occurs in 24–71% of surgical cases, depending on the definition criteria used across studies (1–7). Half of the deaths after cardiothoracic surgery for CHD may be attributable to LCOS (8). Precisely defining LCOS remains a clinical challenge (9). The most used clinical variables to define LCOS include invasively measured cardiac index, elevated serum lactate levels, level of inotropic support or vasoactive inotropic score (VIS), and oliguria, all measured up to 72 postoperative hours (1, 2, 5, 7, 9). Rule-based indices such as the Inadequate Oxygen Delivery-2 score and other composite LCOS scales have been applied in pediatric cardiac patients, yet these tools depend on predefined physiologic thresholds and are not trained to forecast future LCOS events from retrospective time-series data (10, 11).

Although machine learning (ML) models to predict LCOS in adult cardiac surgical patients have been developed (12, 13), no such models exist for LCOS in neonates undergoing cardiothoracic surgery for CHD. The primary aim of this study was to develop ML models to predict LCOS in neonates undergoing cardiothoracic surgery for CHD. Accordingly, the work should be regarded as a technology readiness level 3–4 proof-of-concept that demonstrates algorithmic feasibility on retrospective data, rather than a deployment-ready solution (14).

METHODS

Study Design

This retrospective observational study received approval from the Cleveland Clinic Institutional Review Board (IRB Number 23-725; “Modeling of Post-Operative Low Cardiac Output Syndrome”; approved July 19, 2023). All procedures were conducted in accordance with the ethical standards of the responsible institutional and/or regional review committees and with the principles of the 1975 Declaration of Helsinki.

Patient Population

Neonates 28 days old or younger who underwent cardiothoracic surgery for CHD and were admitted to pediatric cardiac ICU (PCICU) at Cleveland Clinic Children’s were included between January 2012 and April 2023. Neonates admitted to the PCICU from the operating room on extracorporeal membrane oxygenation were excluded from the study.

Model Variables

The dataset was built by manual collection of all the variables from the electronic health records (EHRs). The initial input variables were categorized into demographic, preoperative, intraoperative, and postoperative domains, with measurements recorded hourly or at other predetermined intervals during the first 48 postoperative hours in the PCICU. Demographic variables included age in days and sex. Preoperative variables included weight and the presence of single ventricle physiology. Intraoperative variables included Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery (STAT) category (15) and cardiopulmonary bypass (CPB) duration in minutes. Postoperative data consisted of both physiologic and laboratory values collected over the first 48 postoperative hours. Postoperative physiologic variables included core and peripheral body temperature, heart rate, respiratory rate, peripheral arterial oxygen saturation by pulse oximetry (Spo₂), invasive arterial systolic, diastolic and mean blood pressures, central venous pressure (CVP), urine output (UOP) in mL/kg/hr, cerebral and renal near-infrared spectroscopy (NIRS), VIS and state behavioral scale (SBS) scores. These postoperative physiologic variables were collected hourly. Postoperative laboratory variables included serum lactate, bicarbonate, base excess, blood pH, Pao₂, and Paco₂ from the arterial blood gas, venous oxygen saturation (Svo₂) from the venous blood gas, hemoglobin, ionized calcium, and serum glucose from the arterial blood gas. Supplementary Material-Forecasting model’s input variables (https://links.lww.com/CCX/B557) listed those initial input variables.

Since there is no universally accepted definition of LCOS in children who have undergone cardiothoracic surgery for CHD, we defined LCOS based on the relevant literature (4, 5, 7, 9). Based on these data we defined LCOS after cardiothoracic surgery for CHD in the first 48 postoperative hours as follows: VIS greater than 15; UOP less than 0.5 mL/kg/hr, arterial blood serum lactate greater than or equal to 3 mmol/L. A patient is labeled as having LCOS if the patient meets all three LCOS criteria — VIS, UOP, and arterial blood serum lactate — at the same time at least once during the first 48 hours postoperatively.

Data Preprocessing

Missing data were handled using two main approaches to ensure the integrity and completeness of the dataset. The Last Observation Carried Forward (LOCF) method was applied for time-varying variables with occasional gaps, where missing values were filled with the most recent available measurement from a prior time point (16). For variables with no recorded values for a given patient Multiple Imputation by Chained Equations (MICE) was used (17). MICE method models each missing variable in relation to other observed variables, providing robust estimates when prior values are unavailable. These imputation methods ensured that both time-varying and static features were complete and ready for downstream analysis (Supplementary Material-Data Preprocessing and Missing Data Handling, https://links.lww.com/CCX/B557).

Feature Engineering

To enhance the model’s ability to leverage temporal dynamics, lagged features were incorporated into the dataset. For each time point, measurements from 1 and 2 hours prior were created as new variables, allowing the model to capture short-term physiologic trends that could predict LCOS (Supplementary Material—Feature Vector Construction https://links.lww.com/CCX/B557). This time-series approach, informed by strategies outlined by Qui et al (18), provided critical context for understanding how recent changes in variables such as lactate, urine output, and heart rate influence outcomes.

Following preprocessing, all features and their lagged versions were included in the initial dataset. An iterative feature selection process, guided by area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC) metrics, was then applied to refine the feature set. Variables with minimal contribution to predictive performance were removed, along with their associated lagged features.

Ultimately, 12 core features were identified as the strongest predictors of LCOS: age, weight, UOP, VIS, arterial blood lactate, heart rate, core temperature, invasive mean arterial pressure, respiratory rate, blood gas base excess, hemoglobin and SpO₂. When combined with their lagged versions, the final model included 32 features (Supplementary Material—Prediction model’s final input variables, https://links.lww.com/CCX/B557).

ML Model Development

LightGBM, an efficient tree-based gradient-boosting algorithm (19), was selected as the primary classifier after benchmarking against XGBoost, and CatBoost; it yielded the highest AUROC and AUPRC at every forecasting horizon (Supplementary Fig. 4, https://links.lww.com/CCX/B557). Separate LightGBM models were trained to predict LCOS 2, 4, 6, and 12 hours ahead, with each horizon beginning 1 hours after the prediction time. To avoid circular logic between the LCOS definition and the model inputs (lactate, VIS, UOP), we incorporated a 1-hour time-lagged prediction structure. Specifically, the models used lactate, VIS, and UOP values measured at 1 and 2 hours before the time that the prediction was made to forecast whether LCOS thresholds (i.e., lactate ≥ 3 mmol/L, VIS > 15, UOP < 0.5 mL/kg/hr) would be reached in the all forecasting horizons (Supplementary Material—Target Definition and Transformation, https://links.lww.com/CCX/B557). For instance, if the prediction is made at the fourth postoperative hour, the forecasting horizon begins at the fifth postoperative hour (Fig. 1). Consequently, the outcome values of lactate, VIS, and UOP that trigger an LCOS label occurred strictly after the time window used for the model’s input features. This ensured the prediction relied on earlier or current data rather than the same data that defines LCOS, thus preserving the integrity of the predictive modeling process.

Figure 1.

Example of the machine learning (ML) model’s prediction at 4 hours after arrival at the pediatric cardiac ICU (PCICU), illustrating how input variables are used and how the forecasting horizon (FH) extends to 2, 4, 6, and 12 hours.

Cross-validation and Hyperparameter Tuning

A nested cross-validation procedure was used to systematically evaluate model performance and optimize hyperparameters (20, 21). The dataset was split into ten outer folds, each serving as an independent test set. For each outer fold, the remaining training portion was further divided into Five inner folds for hyperparameter tuning. Hyperparameter search was conducted using Optuna with a Tree-structured Parzen Estimator (22). The best-performing hyperparameter configuration identified in the inner loop was then applied to retrain the model on the entire training set of the corresponding outer fold, and performance was subsequently assessed on the held-out outer test set (Supplementary Fig. 1 https://links.lww.com/CCX/B557). This approach reduced overfitting risks and provided reliable estimates of the model’s generalization capability.

ML Model’s Predictive Performance Evaluation and Feature Importance Analysis

Accuracy, AUROC, precision (positive predictive value), recall (sensitivity), AUPRC, F1 score, F2 score, and Brier loss score were reported with 95% CIs. To evaluate the relative impact of each input variable on model performance, a secondary analysis was performed using SHapley Additive exPlanations (SHAP) values (23).

Statistical Analysis

Continuous preoperative, intraoperative, and postoperative variables were summarized using medians and interquartile ranges, as appropriate, based on data distribution. Categorical variables were presented as frequencies and percentages. Appropriate statistical tests were applied to compare differences between LCOS and non-LCOS patients.

All statistical analyses were conducted using Python (Version 3.10.15), pandas (Version 2.0.3), NumPy (Version 1.22.1), scikit-learn (Version 1.4.2), LightGBM (Version 4.5.0), and Optuna (Version 4.0.0). This study adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis + Artificial Intelligence (TRIPOD+AI) guidelines (Supplementary Material—TRIPOD-AI, https://links.lww.com/CCX/B557), ensuring comprehensive and transparent reporting of the clinical prediction model development and evaluation processes (24).

RESULTS

Patient Characteristics

A total of 181 neonates were included in the dataset, of whom 27 (14.9%) met the definition of LCOS at least once within the first 48 postoperative hours (Supplementary Table 1, https://links.lww.com/CCX/B557). At 2- and 12-hour horizons, 142 and 275 LCOS episodes were identified, respectively. Within the 12-hour window, only 33% occurred in the first hour, with the remainder distributed over the next 11 hour (Supplementary Table 2, https://links.lww.com/CCX/B557). Statistically and clinically significant differences between LCOS and non-LCOS patients were observed: LCOS patients were significantly younger (median 4.0 vs. 7.0 d), weighed less (median 2.90 vs. 3.24 kg), were more likely to have single ventricle physiology (51.9% vs 22.7%), STAT 5 category surgery (48.1% vs 22.7%) and had longer CPB duration (median 154 vs 116 min). In addition, LCOS patients exhibited a significantly higher heart rate, reduced Spo₂ levels, lower UOP, increased VIS, lower SBS score, higher lactate levels, lower Pao₂ levels and lower Svo₂ (all p < 0.01). Although differences in core and peripheral temperatures, respiratory rate, diastolic blood pressure, NIRS values, arterial blood pH, bicarbonate, base excess, Paco₂, hemoglobin, serum ionized calcium, and blood glucose levels reached statistical significance (p < 0.05), these findings were not considered clinically significant. There were no statistically significant differences in sex, CVP, systolic, or mean arterial pressure between the LCOS and non-LCOS groups. LCOS patients also experienced higher occurance rate of cardiac arrest (33.3% vs 2.6%). These comparisons were listed in the Supplementary Table 1 (https://links.lww.com/CCX/B557).

The ML Model’s predictive performance

The LightGBM model demonstrated consistently strong performance across all forecasting horizons (Table 1). AUROC values ranged from 0.91 (95% CI, 0.85–0.96) to 0.98 (95% CI, 0.96–0.99), indicating excellent discriminative ability for predicting LCOS at all forecasting horizons. AUPRC values ranged from 0.80 (95% CI, 0.72–0.87) at 4-hour forecasting horizon to 0.60 (95% CI, 0.44–0.77) at 12-hour forecasting horizon, reflecting excellent performance in identifying true positives despite the imbalance in LCOS prevalence in the dataset. Sensitivity (recall) was poor and declined from 0.51 (95% CI, 0.39–0.62) at 4-hour forecasting horizon to 0.20 (95% CI, 0.03–0.38) at 12-hour forecasting horizon, whereas precision (positive predictive value) peaked at 0.75 (95% CI, 0.61–0.89) at 6-hour forecasting horizon and dropped to 0.29 (95% CI, 0.06–0.51) at 12-hour forecasting horizon. Brier scores were 0.01 (95% CI, 0.01–0.01) at the 2-, 4- and 6-hour horizons and 0.02 (0.01–0.02) at 12 hours. Other predictive metrics are shown in Table 1.

TABLE 1.

LightGBM Machine Learning Model’s Predictive Performance Results

LightGBM	2-hr FH, Mean (95% CI)	4-hr FH, Mean (95% CI)	6-hr FH, Mean (95% CI)	12-hr FH, Mean (95% CI)
Area under the receiver operating characteristic curve	0.98 (0.96–0.99)	0.97 (0.96–0.99)	0.93 (0.88–0.98)	0.91 (0.85–0.96)
Area under the precision-recall curve	0.72 (0.58–0.86)	0.80 (0.72–0.87)	0.72 (0.62–0.83)	0.60 (0.44–0.77)
Precision	0.54 (0.32–0.76)	0.74 (0.59–0.89)	0.75 (0.61–0.89)	0.29 (0.06–0.51)
Recall	0.39 (0.23–0.55)	0.51 (0.39–0.62)	0.46 (0.34–0.57)	0.20 (0.03–0.38)
Brier score	0.01 (0.01–0.01)	0.01 (0.01–0.01)	0.01 (0.01–0.01)	0.02 (0.01–0.02)
F1 score	0.42 (0.24–0.61)	0.58 (0.46–0.71)	0.55 (0.43–0.67)	0.22 (0.04–0.40)
F2 score	0.40 (0.23–0.57)	0.53 (0.41–0.65)	0.49 (0.37–0.60)	0.21 (0.04–0.38)
Accuracy	0.69 (0.61–0.78)	0.75 (0.69–0.81)	0.73 (0.67–0.79)	0.60 (0.51–0.69)

CI = confidence interval, FH = forecasting horizon.

LightGBM is an efficient tree-based gradient-boosting algorithm.

SHAP Analysis

The SHAP analysis identified higher blood gas lactate, higher VIS, and lower UOP at the time of the prediction as the most influential features in forecasting the development of LCOS at future time horizons (Fig. 2). In addition, lagged features including UOP, base excess (Supplementary Fig. 3, https://links.lww.com/CCX/B557). lactate, VIS, respiratory rate and heart rate recorded 1 and 2 hours before the time that the prediction was made also contributed to model predictions, emphasizing the importance of time-dependent physiologic trends in LCOS risk assessment (Supplementary Fig. 2, https://links.lww.com/CCX/B557).

Figure 2.

SHapley Additive exPlanations (SHAP) analysis of the input variables used by the machine learning model. The vertical axis lists feature names in descending order of importance from top to bottom, where “lag 1” represents the feature values from 1 hour prior and “lag 2” from 2 hours prior. On the horizontal axis, red points indicate higher feature values, with larger positive SHAP values signifying a stronger contribution toward predicting low cardiac output syndrome. MAP = invasive mean arterial blood pressure, Spo₂ = arterial oxygen saturation by pulse oximetry, UOP = urine output in mL/kg/hr, VIS = vasoactive inotrope score.

DISCUSSION

In this study, LightGBM-based supervised models were trained to forecast LCOS within the first 48 post-operative hours in neonates undergoing cardiothoracic surgery for CHD. Model performance remained high across 2–12 hours horizons (AUROC 0.91–0.98; AUPRC 0.60–0.80 despite class imbalance). To our knowledge, this represents the first application of ML specifically targeting LCOS prediction in neonates with CHD undergoing cardiothoracic surgery, addressing the gap in pediatric cardiac perioperative care by providing an early, reliable tool for risk stratification.

The ML model’s performance is consistent with previous work in adult populations, such as that by Hong et al (12), who reported an AUROC of 0.909 using a Random Forest model for LCOS prediction. Our model tackles the unique challenges of neonates with CHD, who have limited physiologic reserve and marked postoperative hemodynamic variability, by forecasting LCOS with time-lagged data. Unlike the clustering approach of Zhao et al (25), this temporal prediction yields more clinically actionable insights. Furthermore, our work complements recent advances in noninvasive cardiac output monitoring described by Tandon et al (26), offering a cohesive strategy for early risk stratification and intervention in high-risk neonates with CHD. Collectively, these features underscore the novelty and clinical relevance of our ML-based approach to LCOS prediction in this vulnerable pediatric patient group. Building on this point, the model draws only on data from the previous two hours and flags LCOS solely for events that occur after the prediction moment. This timing removes circularity by capturing sub-threshold trends in lactate, urine output, and VIS instead of their threshold-crossing values. The resulting one- to two-hour lead time could support closer monitoring, timely titration of vasoactive agents, and optimization of volume status; prospective studies are needed to confirm whether such early alerts improve outcomes.

The prevalence of LCOS in our cohort was 14.9%, creating a substantial class imbalance that warrants careful metric selection. Although AUROC is traditionally reported, it weights the majority and minority classes equally and can therefore overstate discrimination under skewed distributions. We consequently emphasized the area under the precision–recall curve (AUPRC), whose baseline is anchored to the positive‐class prevalence (14.9%), rendering it a more stringent and clinically informative indicator of rare‐event detection. Two adult LCOS investigations conducted by Hong et al (12, 13), reporting incidences of 15.8% and 18%, respectively, and they did not publish AUPRC, precluding a direct assessment of their capacity to identify true positives. By contrast, our models achieved AUPRC values of 0.80 (95% CI, 0.72–0.87) for the 4-hour horizon and 0.60 (95% CI, 0.44–0.77) for the 12-hour horizon, indicating favorable precision–recall trade-offs despite class imbalance and the increasing prognostic uncertainty inherent in longer horizons.

To explore whether recall could be further improved, we applied Synthetic Minority Over-sampling Technique –Tomek oversampling. Although this approach increased 12-hour recall by roughly 10%, it simultaneously decreased precision by a similar margin, expanded the training set by 40%, and introduced concerns about synthetic-data bias and physiologic plausibility. Net benefit, as assessed by F1 score and decision-curve analysis, did not improve; therefore, the final models were trained exclusively on observed data. This evidence underscores both the utility of AUPRC for imbalanced clinical datasets and the importance of balancing sensitivity gains against potential losses in predictive reliability.

Forecasting horizons of 2–12 hours afford clinicians critical lead time to intervene and mitigate LCOS progression in neonates, enabling proactive therapeutic adjustments that can improve outcomes. SHAP analysis improves interpretability by quantifying the influence of key predictors (lactate, VIS, and UOP), thereby satisfying current expectations for transparent machine learning (ML) and, as shown by Hong et al (13), reinforcing clinician trust. Consequently, the model combines high predictive performance with interpretability to support timely, evidence-based decisions in neonatal LCOS management.

Although a 12-hour forecasting horizon includes the 2-hour forecasting horizon, predicting events over a longer period introduces greater uncertainty due to evolving clinical conditions, changing interventions, and natural physiologic fluctuations. In a short horizon, immediate trends (e.g., rising lactate or decreased UOP) often provide clearer signals of imminent clinical deterioration, enabling the model to achieve higher accuracy. By contrast, as the forecast extends to 12 hours, acute trends may become “diluted” by additional variables leading to reduced predictive performance. Although the model sustained strong performance across all forecasting horizons, some of its discriminative power could plausibly stem from events that unfold shortly after the prediction time. Yet, our temporal-distribution analysis (Supplementary Table 2, https://links.lww.com/CCX/B557) showed that the majority of LCOS episodes in the 12-hour window occurred beyond the first hour, indicating that the model is capturing early pathophysiologic signals rather than merely sensing imminent deterioration. Consequently, while the 12-hour forecasting horizon can capture more future LCOS events, it also accumulates more noise, thus explaining why the 2-hour forecasting horizon often yields better discriminative power.

The study’s robustness rests on five pillars: rigorous preprocessing that preserves data integrity, inclusion of dynamic time-varying variables with lagged features to capture early physiologic shifts, deployment of the high-efficiency LightGBM algorithm for consistently strong discrimination across all forecasting horizons, a dedicated focus on neonates with CHD and the use of clinically actionable LCOS thresholds (lactate, VIS, and urine output) that facilitate real-world implementation. Several internal validation strategies were undertaken, and the highest performance was achieved with nested 10-fold cross-validation (Supplementary Table 3, https://links.lww.com/CCX/B557). In future work, greater generalizability may be achieved through external validation with multicenter datasets.

Despite its strengths, the study’s retrospective, single-center design constrains generalizability and may reflect institution-specific practices. The relatively small sample size increases uncertainty around effect estimates and raises the risk of overfitting despite internal validation. External validation in a multicenter cohort is necessary. Because study period spanned 2012–2023, clinical practices may have evolved during this period, introducing unmeasured temporal drift, these differences were not explicitly modeled and may affect model calibration. Although missingness was less than 3% and addressed using LOCF and MICE (Supplementary Table 4, https://links.lww.com/CCX/B557), residual confounding and latent temporal patterns may persist and could be better characterized with larger, longitudinal datasets. Prospective deployment within real-time EHR pipelines will be required to confirm clinical utility and enable timely intervention in high-risk neonates.

CONCLUSIONS

This study demonstrates that supervised ML models can accurately predict LCOS in neonates with CHD undergoing cardiothoracic surgery, delivering strong performance across multiple forecasting horizons. Furthermore, the study establishes a basis for future initiatives such as multicenter validation and real-time integration, which will broaden the application of predictive analytics in pediatric cardiac critical care.

Supplementary Material

cc9-7-e1327-s001.pdf ^{(1.3MB, pdf)}