Explainable machine learning for postoperative respiratory failure prediction in open-heart surgery patients — a study based on the MIMIC-IV database

Abstract

Background

Postoperative respiratory failure (PRF) is a severe complication after open-heart surgery, associated with increased mortality and prolonged ICU stays. While machine learning (ML) models have shown promise in predicting PRF, existing models often rely on fragmented data and lack interpretability. This study aimed to develop an interpretable ML model for early prediction of PRF using data from the first 24 h of ICU admission.

Methods

We analyzed data from the MIMIC-IV database, focusing on patients undergoing open-heart surgery with cardiopulmonary bypass (CPB). Patients with preoperative respiratory failure or significant missing data were excluded. Missing values (< 30%) were imputed using Predictive Mean Matching. Twelve features were selected through LASSO regression. We compared the performance of eight ML models using AUROC, AUPRC, and other metrics. The optimal model was further interpreted using Shapley Additive exPlanations (SHAP).

Results

Of the 4,488 patients, 339 (7.6%) developed PRF. The Gradient Boosting Machine (GBM) model demonstrated the best performance with an AUROC of 0.808, AUPRC of 0.369, and Youden’s index of 0.479, indicating balanced sensitivity (0.703) and specificity (0.776). SHAP analysis revealed that key predictors included minimum ionized calcium levels, vasopressor score, and central venous oxygen saturation (ScvO₂), with their impact varying across patient risk categories.

Conclusion

The GBM model, selected for its balanced performance across discrimination, calibration, and validation stability, provides a promising tool for early PRF risk stratification. The use of SHAP analysis enhances the interpretability of the model, highlighting the role of hemodynamic and metabolic markers in predicting PRF, thus improving clinical understanding and decision-making.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-026-03425-0.

Introduction

Postoperative respiratory failure (PRF) is a severe complication in patients after cardiac surgery [1]. It results in an increase in hospital mortality, ranging from 27% to 35%, and prolongs intensive care unit (ICU) stays to a median of 7 to 14 days. It can also increase the additional costs to $20,000 per case [2, 3]. Unfortunately, more than 1.5 million cardiac surgery patients are exposed to PRF annually [4, 5]. Hence, it’s essential to identify risk factors that predict PRF in patients after cardiac surgery.

The exact pathophysiological mechanisms underlying PRF remain poorly understood. However, these mechanisms relate to organ injury, involving inflammation, microcirculatory dysfunction, and metabolic reprogramming.

The pathogenic factors of respiratory failure are diverse and involve perioperative factors. Traditional models, such as logistic regression and APACHE-IV, can recognise perioperative risks, including age, overall health, smoking history, and lung function, which can influence the outcome after surgery [6, 7]. However, they have several limitations stemming from their inability to capture trends in vasopressor and lactate levels, which can subsequently affect the development of a predictive model [8, 9]. Although grade-based prediction models for postoperative morbidity were applicable, they failed to fully leverage time-dependent vasopressor doses and lactate levels [10]. It’s critical to improve postoperative risk-predicting models by incorporating the temporal changes of variables, such as inflammatory biomarkers, as noted by Agnes and colleagues [11].

In recent years, machine learning (ML) has become increasingly attention for predicting postoperative outcomes. Compared with traditional models, ML has shown significant improvements in AUC in patients undergoing continuous renal replacement therapy [12]. Furthermore, a high AUC was achieved in ML research that integrated perioperative laboratory results to predict postoperative mortality, as reported by Lim [13–15]. To address the limitations of traditional methods, Adhikari used time-series data to enhance predictive accuracy in dynamic preoperative risk assessments [16]. ML, particularly Gradient Boosting Machines (GBM), can extract key features from complex datasets, mainly features with temporal interactions and non-linear relations [17–19]. At the same time, the interpretability of ML models can deepen our understanding. SHAP (Shapley Additive exPlanations) provides clear insights into model predictions based on Shapley values, enhancing model transparency and fostering user trust [17, 18, 20]. Thus, combining ML with interpretability techniques, such as SHAP, holds significant potential to improve clinical decision-making [21].

Accordingly, the purpose of this study was to determine the best-performing ML models for predicting patients at high risk of PRF using data from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. The SHAP method highlights feature importance and interprets predictions, potentially enhancing its value for early clinical decisions. Finally, we planned to use interpretable ML via SHAP values to examine risk factors and quantitatively visualize their relationships with outcomes.

Methods and materials

Data source and study population

This study employed data from the publicly accessible MIMIC-IV database, developed by the MIT Laboratory for Computational Physiology [22]. The database contains deidentified medical records of critically ill patients from Beth Israel Deaconess Medical Center. Riliang Ma, one of the authors, has completed the necessary CITI training and is a credentialed user of the MIMIC-IV database (Record ID: 68629862). The author was granted access to the dataset and successfully extracted the relevant data. A retrospective analysis was conducted by 5,063 patients who underwent extracorporeal circulation as an auxiliary to open-heart surgery and were admitted to the ICU between 2008 and 2022, using ICD-9/10 codes. 19 cases of second admission to the ICU, 120 instances with preoperatively diagnosed respiratory failure, 18 cases of death within 72 h after surgery, and 418 instances with missing more than 50% of features were excluded, resulting in a final cohort of 4488 patients. The study protocol received approval from the MIT Institutional Review Board, with informed consent being waived — disease definitions adhered to established criteria [23] (refer to Supplementary Table 1).

Data preprocessing

Data extraction was performed using SQL queries executed in PostgreSQL (version 13.7.2) and Navicat Premium (version 16.1.12). Clinical variables obtained during the first 24 h of admission encompassed demographics, treatments, and laboratory measurements, which were used to develop the clinical prediction models. PRF was defined according to the recommended definition [23]. Features with more than 30% missing data were excluded to mitigate bias [24]. The potential features included in this research were categorized as follows: [1] Baseline demographic information: age, gender, BMI; [2] Comorbidities: COPD, renal insufficiency, diabetes, coronary artery disease, heart failure, liver disease, cerebrovascular disease; [3] Treatments: vasopressor score was calculated (vasopressor score = 1*dopamine_max_day + 1*dobutamine_max_day + 100*epinephrine_max_day + 100*norepinephrine_max_day + 10000*vasopressin_max_day + 10*milrinone_max_day); [4] Laboratory parameters: sodium, magnesium, ionized calcium, potassium, hemoglobin, platelets, lactate, blood creatinine, blood urea nitrogen (BUN), white blood cell count (WBC), lymphocytes, glucose coefficient variation, glucose, hyperglycemia percent, ScvO₂. Missing values were imputed using Predictive Mean Matching (PMM) on the full dataset prior to the train–test split. We acknowledge that this approach may introduce information leakage, as distributional characteristics of the test set could influence imputed values in the training data.

Model development and comparison

The dataset was divided into a training and a test set (70:30 ratio) using stratified sampling based on the binary outcome variable. Our analytical approach began with Least Absolute Shrinkage and Selection Operator (LASSO) regression for optimal feature selection, followed by the development of eight distinct machine learning models: Logistic Regression, Support Vector Machine (SVM), Gradient Boosting Machine (GBM), Neural Networks (NN), Extreme Gradient Boosting (XGBoost), Adaptive Boosting (AdaBoost), Light Gradient Boosting Machine (LightGBM), and Categorical Boosting (CatBoost). Hyperparameters for each model were optimized via a 10-fold cross-validation grid search on the training set, using the predefined search spaces detailed in Supplementary Table 1. Subsequently, the optimal hyperparameter was selected for each model (Table 3). Model performance was assessed using multiple metrics, including the Area Under the Receiver Operating Characteristic Curve (AUROC), the Area Under the Precision-Recall Curve (AUPRC), Brier score, accuracy, sensitivity, specificity, precision, and F1-score. The 95% confidence intervals for AUROC were estimated using DeLong’s method, and those for AUPRC were obtained via bootstrap resampling (1,000 iterations).

Table 3.

Optimal parameters of eight ML models in predicting PRF

Model	Optimal hyperparameters
Logistic	None (default)
SVM	sigma = 0.001, C = 0.09
GBM	n.trees = 100, interaction.depth = 3, shrinkage = 0.1, n.minobsinnode = 10
NN	size = 6, decay = 0.6
Xgboost	nrounds = 100, max_depth = 3, eta = 0.1, gamma = 0.5, colsample_bytree = 0.5, min_child_weight = 1, subsample = 0.6
Adaboost	mfinal = 10, maxdepth = 2, coeflearn = “Zhu”
LightGBM	objective = “binary”, metric = “auc”, num_leaves = 31, max_depth = -1, learning_rate = 0.1, min_data_in_leaf = 20, feature_fraction = 0.9, bagging_fraction = 0.8, lambda_l1 = 0, lambda_l2 = 0
CatBoost	iterations = 100, depth = 6, learning_rate = 0.03, l2_leaf_reg = 3, border_count = 32, random_strength = 1, use_best_model = TRUE, eval_metric = “AUC”, random_seed = 123

Model interpretation

The SHAP method was used to improve model interpretability by measuring feature contributions and ranking their importance, subsequently addressing the “black-box” nature of ML models. SHAP values were utilized to inform feature selection for each prediction task. SHAP provides both global and local interpretability. Global interpretability was demonstrated using summary plots, such as bar charts of mean absolute SHAP values, to highlight overall feature importance. For individual predictions, local interpretability was demonstrated with waterfall plots, enabling clinicians to understand how each variable influenced a patient’s risk score [25].

Statistical analysis

Data were analyzed using R (v4.4.3). Continuous numerical variables that conformed to a normal distribution were presented as the mean (standard deviation, SD). In contrast, nonnormally distributed data are reported as the median (interquartile range) [M (IQR)] and compared using the Mann–Whitney U test or Kruskal–Wallis test. Categorical variables were expressed as counts (n) and frequencies (%), and compared using Chi-square tests or Fisher’s exact test. Model performance was assessed with AUROC and AUPRC. The best cutoff point was identified by maximizing Youden’s index, which is calculated as (sensitivity + specificity − 1). A p-value<0.05 was deemed statistically significant.

Results

Baseline characteristics of the study population

As shown in the flowchart in Fig. 1, data from 5,063 patients were extracted from the MIMIC-IV database. Based on the established inclusion and exclusion criteria, 4,488 patients who underwent extracorporeal circulation during open-heart surgery were included in this study. These patients were divided into a training set (n = 3,143) and a test set (n = 1,345). The overall incidence of PRF was 7.6%, with a similar distribution of 7.6% in the training set and 7.5% in the test set after ICU admission. The baseline characteristics of the entire cohort, stratified by PRF occurrence and dataset, were summarized in Table 1. Statistically significant differences between the PRF and non-PRF groups were observed for several variables: higher prevalence of comorbidities (renal insufficiency, heart failure, liver disease, cerebrovascular disease), female gender, older age, and abnormalities in hemodynamic and metabolic stability (lower min_ ionized_calcium and ScvO₂, higher vasopressor score, BUN, and hyperglycemia percentage) (all p<0.05). The training and test sets were well-balanced for all baseline characteristics, including outcome distribution (PRF: 7.6% vs. 7.5%, p = 0.991). A detailed comparison was provided in Supplementary Table 3.

Fig. 1.

The screening flow of participants from the MIMIC-IV database. PRF: Postoperative respiratory failure, CPB: cardiopulmonary bypass

Table 1.

Comparison of demographic characteristics between PRF and Non-PRF

Characteristic	Overall	Non-PRF	PRF	p a
Result, n (%)	N = 4488	N = 4149(92.4)	N = 339(7.6)
Selected Comorbidities, n (%)
renal_insufficiency	645 (14.4)	553 (13.3)	92 (27.1)	< 0.0001
coronary_artery_disease	3261 (72.7)	3051 (73.5)	210 (61.9)	< 0.0001
heart_failure	1131 (25.2)	979 (23.6)	152 (44.8)	< 0.0001
liver_disease	111 (2.5)	77 (1.9)	34 (10.0)	< 0.0001
cerebrovascular_disease	471 (10.5)	404 (9.7)	67 (19.8)	< 0.0001
Demographics
Gender, Male	3126 (69.7)	2931 (70.6)	195 (57.5)	< 0.0001
age, years (M ± SD)	67.29 (12.05)	67.16 (11.96)	68.87 (12.90)	0.0121
Key Laboratory & Clinical Variables (Selected by LASSO)
min_ ionized_calcium, mmol/L (M ± SD)	1.07 (0.09)	1.08 (0.08)	1.00 (0.15)	< 0.0001
max_bun, mg/dL (Median [IQR])	17.00 [14.00, 22.00]	17.00 [14.00, 21.00]	21.00 [16.00, 28.00]	< 0.0001
hyperglycemia_percent, % (median [IQR])	0.00 [0.00, 14.29]	0.00 [0.00, 13.33]	14.29 [0.00, 33.33]	< 0.0001
ScvO2 (%, mean ± SD)	0.70 (0.08)	0.71 (0.08)	0.66 (0.09)	< 0.0001
vasopressor_score_day, (median [IQR])	0.00 [0.00, 2.00]	0.00 [0.00, 0.00]	4.00 [0.00, 16.01]	< 0.0001
Dataset Split Comparison		Testing N = 1345	Training N = 3143
Result: PRF, n (%)		101 (7.5)	238 (7.6)	0.991

IQR, Interquartile range; ScvO₂: central venous oxygen saturation; ^aStatistical tests: Continuous variables – Student’s t-test or Mann-Whitney U test; Categorical variables - Chi-square test or Fisher’s exact test. A complete detailed comparison of all baseline characteristics is provided in Supplementary Table 3

Features selected for model development

To identify key predictors of PRF after cardiac surgery, LASSO regression was used to automate feature selection in the training set. The coefficient profiles of the variable coefficients are shown in Fig. 2. Using tenfold cross-validation, 12 variables out of an initial 36 were selected at the optimal shrinkage parameter (lambda. 1se = 0.01198226) for inclusion in the subsequent machine learning models. The 12 features were renal insufficiency, coronary artery disease, liver disease, cerebrovascular disease, Gender, minimum ionized calcium, maximum haemoglobin, maximum BUN, minimum BUN, hyperglycemia percentage, ScvO₂, and vasopressor score.

Fig. 2.

The LASSO binary logistic regression model was utilized to select demographic and clinical features. A Graph depicting the paths of coefficients in LASSO regression, B Graph of cross-validation for LASSO regression

Model development and performance comparison

Eight ML models were developed and compared for their ability to predict PRF. A comprehensive visual comparison across both training (A, C, E, G) and test sets (B, D, F, H), including ROC curves, calibration plots, Precision–Recall curves, and decision curve analysis, was provided in Fig. 3. In the calibration plots (C & D), the x-axis shows the midpoint of binned predicted probabilities, and the y-axis shows the corresponding observed event percentage. This visual assessment was complemented by the quantitative Brier score, with all models’ scores reported in Table 2.

Fig. 3.

Performance of machine learning (ML) models predicting PRF in the training and test sets. ROC curve analysis (A, B), Calibration curve analysis (C, D), Precision-Recall Curve (E, F), and DCA curves (G, H) for each model in predicting PRF using eight ML algorithms in the training and test sets. Abbreviations: SVM: Support Vector Machine; GBM: Gradient Boosting Machine; NN: Neural Networks; XGBoost: Extreme gradient boosting; Adaboost: Adaptive boosting; LightGBM: Light gradient boosting machine; CatBoost: Categorical boosting. AUROC, area under the ROC curve; AUPRC, area under the precision-recall curve

Table 2.

Predictive performances of the eight ML models for predicting PRF on the Test set

Model	AUROC (95% CI)	AUPRC (95% CI)	Youden’s Index	Accuracy	Sensitivity	Specificity	Precision	F1 score	Brier Score
GBM	0.808 (0.764–0.852)	0.369 (0.265–0.467)	0.479	0.770	0.703	0.776	0.203	0.315	0.058
NN	0.814 (0.769–0.859)	0.359 (0.265–0.456)	0.497	0.804	0.683	0.814	0.230	0.344	0.058
CatBoost	0.810 (0.766–0.854)	0.349 (0.248–0.411)	0.491	0.781	0.703	0.788	0.212	0.326	0.264
Logistic	0.804 (0.759–0.849)	0.359 (0.268–0.465)	0.480	0.789	0.683	0.797	0.215	0.327	0.058
Xgboost	0.804 (0.759–0.849)	0.357 (0.263–0.453)	0.507	0.796	0.703	0.804	0.225	0.341	0.246
SVM	0.791 (0.745–0.837)	0.334 (0.246–0.439)	0.451	0.888	0.535	0.916	0.342	0.417	0.060
LightGBM	0.760 (0.711–0.809)	0.256 (0.185–0.336)	0.465	0.792	0.663	0.802	0.214	0.324	0.075
Adaboost	0.724 (0.673–0.775)	0.164 (0.123–0.213)	0.444	0.712	0.733	0.711	0.171	0.277	0.070

Optimal classification thresholds were determined by maximizing Youden’s index (Youden’s J = Sensitivity + Specificity − 1) to balance sensitivity and specificity for our imbalanced dataset. The full predictive performances of the eight ML models are provided in Supplementary Table 4

The performance metrics for all eight models on the test set are summarized in Table 2, and the corresponding optimal hyperparameters are detailed in Table 3. On the test set, all models showed reasonable discrimination for PRF prediction. The NN model achieved the highest AUROC of 0.814 (95% CI: 0.769–0.859), closely followed by CatBoost (0.810, 95% CI: 0.765–0.855) and GBM (0.808, 95% CI: 0.764–0.852). To select a model suitable for potential clinical interpretation, a multi-dimensional evaluation was performed, considering metrics relevant to imbalanced datasets and clinical application. The GBM model achieved the highest AUPRC (0.369, 95%CI: 0.265–0.467), which is a more informative metric for our imbalanced dataset (PRF prevalence: 7.6%). It also demonstrated satisfactory calibration, as evidenced by a low Brier score (0.058), a calibration curve closely aligned with the ideal diagonal in Fig. 3D, and a competitive Youden’s index (0.479; Table 2), indicating a reasonable balance between sensitivity and specificity. At the threshold maximizing Youden’s index, the GBM model achieved a sensitivity of 0.703 and a specificity of 0.776 on the test set.

An assessment of performance consistency between training and test sets (Supplementary Table 4) indicated that Although the GBM model showed a larger relative AUROC decrease (10.3%) than NN (5.7%) and CatBoost (6.4%), the absolute AUROC remained stable across datasets, and calibration and DCA analyses consistently supported its robustness. While this suggests a degree of overfitting, a visual inspection of Fig. 3 shows that the GBM model’s performance degradation across sets was minimal in practice. Furthermore, DCA (Fig. 3H) indicated that the GBM model provided competitive clinical utility across most threshold probabilities in the test set.

The test-set performance of the remaining models was as follows: Logistic Regression (AUROC = 0.804, Youden’s index = 0.480), which also demonstrated good calibration (Brier score = 0.058), XGBoost (AUROC = 0.804, Youden’s index = 0.507), SVM (AUROC = 0.791, Youden’s index = 0.451), LightGBM (AUROC = 0.760, Youden’s index = 0.465), and AdaBoost (AUROC = 0.724, Youden’s index = 0.444). Notably, the calibration curves for AdaBoost and CatBoost exhibited abrupt (Fig. 3D), and CatBoost had the highest Brier score (0.264) among all models (Table 2). The corresponding sensitivity, specificity, and Brier scores for all models are provided in Table 2.

Based on this comprehensive evaluation, which encompassed discrimination (AUROC, AUPRC), calibration (Brier score, calibration curves), clinical utility (Youden’s index, DCA), and generalization stability (training-test consistency), the GBM model was chosen for further interpretation in the context of this study.

Model interpretation

SHAP analysis was employed to interpret the GBM model’s predictions, ranking feature importance (Fig. 4), and illustrating individual feature contributions. Across the developed models, key factors consistently associated with PRF risk included ionized calcium levels, vasopressor score, ScvO₂, BUN, Gender, liver disease, and hyperglycemia percent, though their relative importance varied. Analysis of the test set (Fig. 5A) confirmed these patterns. Waterfall (Fig. 5B) and force plots (Fig. 5C) were generated to illustrate the predictions for individual cases, highlighting the directional impact of features such as low ionized calcium levels, low ScvO2, and high vasopressor scores.

Fig. 4.

Importance score ranking of characteristics in 8 models. A: GBM, B: NN, C: CatBoost, D: SVM, E: Logistic, F: Xgboost, G: LightGBM, H: Adaboost

Fig. 5.

SHAP interpretation of the GBM model. (A) SHAP summary dot plot. Every dot in a row represents a patient, and its color indicates the feature value—orange denotes a greater value, and dark red denotes a lower value. The more dispersed the points of the graph represent, the greater the impact of the variables on the model. (B) Personalized predictions for a patient. The protective and risk variables are defined by the orange and dark red bars, respectively. Higher functional significance is indicated by longer bars

To further substantiate the clinical plausibility and transparency of our model’s decision-making process, additional SHAP analysis were conducted (Figure. 6). The dependence plot (Figure. 6 A) suggested a non-linear relationship between minimum ionized calcium levels and the risk of PRF, with a sharp increase in risk observed as ionized calcium levels fell below approximately 1.0 mmol/L, especially in patients with high vasopressor scores. A risk-group comparison (Fig. 6B) showed that the same key features—minimum ionized calcium level and vasopressor score—were associated with PRF risk in both the high- and low-risk groups; however, their mean SHAP values were substantially greater in the high-risk group, indicating stronger contributions to predicted risk. The two most influential features were the minimum ionized calcium level (min_ ionized_calcium) and the high vasopressor score (vasopressor_score), which showed positive mean SHAP values (+ 0.083 and + 0.063, respectively) in the high-risk group, indicating that they increased the predicted risk of PRF. In the low-risk group, these features exhibited substantially smaller mean SHAP values, indicating weaker contributions to risk rather than opposite effects. The remaining features—maximum blood urea nitrogen (max_bun), liver disease, gender, and hyperglycemia percentage—also exhibited group-specific differences primarily in contribution magnitude, with varying magnitudes and directions of influence. Furthermore, the analysis of multi-sample waterfall plots for three representative cases (Fig. 6C) showed that the contributions of each feature lead to distinct risk predictions across the spectrum. In the high-risk case (predicted probability = 0.995), the prediction was primarily influenced by low ionized calcium (0.86 mmol/L), low ScvO₂ (0.65), and a high vasopressor score [26], which collectively pushed the prediction well above baseline. Conversely, the low-risk case (predicted probability = 0.011) was characterized by protective or neutral contributions from these features, including a higher ionized calcium level (1.21 mmol/L) and the absence of vasopressor use. The medium-risk case falls between these extremes.

Fig. 6.

Comprehensive analysis and validation of critical predictors. (A) Dependence plot illustrating the interaction between minimum ionized calcium levels and vasopressor scores, with color intensity indicating vasopressor score. (B) Comparison of mean SHAP values for the top 6 features between high-risk and low-risk groups. (C) SHAP force plot analysis of representative cases across high, medium, and low risk categories

In addition, Supplementary Fig. 1 provides an extensive visualization of the importance ranking of the top eight features. Meanwhile, Supplementary Fig. 2 elucidates the intricate interaction patterns among the top four predictors via a dependence plot matrix.

Discussion

In this study, we developed and validated ML models to predict PRF in patients undergoing open-heart surgery, using data from the MIMIC-IV database. Our analysis identified the GBM model as a competitive predictor, demonstrating an AUROC of 0.808 and an AUPRC of 0.369 on the test set, indicating reasonable discriminative ability. By employing SHAP, we sought to elucidate the model’s decision-making processes. Our analysis suggested that indicators of hemodynamic and metabolic instability—such as vasopressor requirements, low ionized calcium levels, and ScvO₂—were important contributors to model-predicted PRF risk.

Our findings align with the growing literature on ML in critical care prognostication. The performance of our GBM model (AUROC 0.808, AUPRC 0.369 on the test set) falls within the range reported in previous studies. For instance, Villar et al. reported AUROC values exceeding 0.85 for the prediction of acute respiratory failure [26]. Similarly, our results align with studies, such as that by Kong et al., indicating that ensemble methods like GBM can perform well in complex clinical prediction tasks [27]. This supports the notion that ML (like GBM) may be suitable for capturing non-linear patterns in clinical data. A distinctive aspect of our work is the focus on a very early prediction time window (within the first 24 h post-ICU admission), which may enable earlier risk stratification; however, further validation is required to confirm its clinical relevance.

The SHAP analysis identified hemodynamic and metabolic markers as key contributors to the model’s predictions, findings that are physiologically plausible and align with existing knowledge. The vasopressor score was a significant predictor, consistent with research linking vasopressor use to cardiovascular instability [28, 29]. As visualized in the SHAP summary plot (Fig. 4) and risk group comparison (Fig. 6B), high vasopressor requirements were a critical contributor to higher predicted risk in high-risk patients. Similarly, low ScvO₂ was also identified as a crucial feature, supporting prior associations between low perioperative ScvO₂ and postoperative complications [30–32]. Furthermore, the dependence plot (Fig. 6A) revealed a non-linear relationship between ionized calcium and PRF risk, with a sharp increase in predicted risk when ionized calcium levels fall below approximately 1.0 mmol/L. This threshold, coupled with the known physiological role of ionized calcium in cardiovascular function, positions hypocalcemia as a potentially modifiable marker associated with increased predicted risk, requireing further investigation and prospective validation. The consistency of these features across global and local explanation methods strengthens their internal validity within the context of our model.

The calibration curves (Figs. 3C and D) were used to assess the reliability of the predictions. Both the GBM and Logistic Regression models aligned closely with the ideal diagonal, indicating well-calibrated probabilities and yielding low Brier scores (0.058 for each; Table 2). In contrast, the NN and SVM models exhibited miscalibration, which may reflect less stable probability estimation and sensitivity to class imbalance without dedicated calibration procedures. AdaBoost and CatBoost showed step-like calibration curves, likely due to concentrated or discrete predicted probability distributions typical of boosting-based learners, leading to abrupt transitions between probability bins and higher Brier scores. These findings demonstrate that high discriminative performance does not guarantee well-calibrated probabilities, underscoring the importance of evaluating both discrimination and calibration metrics in assessing clinical prediction tools.

The potential long-term goal of a predictive model is to inform clinical practice. A possible clinical implication of our model is early identification of high-risk patients. While some predictors are non-modifiable (such as age, comorbidities), others may be actionable. Attention to vasopressor requirements and ScvO₂ may highlight the importance of vigilant hemodynamic monitoring. For high-risk patients, clinicians could consider hemodynamic optimization strategies, such as goal-directed fluid therapy and judicious use of vasopressors [33]. Additionally, the association between low ionized calcium levels and an increased risk suggests that protocolized monitoring and correction of hypocalcemia postoperatively may warrant evaluation in future studies [34]. However, it is crucial to emphasize that this is a preliminary study. Given the model’s current performance level (AUROC ~ 0.81) and the limitations inherent to its retrospective, single-center derivation, its findings should be interpreted with caution. The most appropriate role for such a model would be as an adjunct to clinical judgment for screening or decision support, rather than as an autonomous diagnostic tool. The clinical utility of the identified risk thresholds, such as for ionized calcium, requires prospective validation.

This study has several limitations. First, the retrospective design using the MIMIC-IV database may introduce selection bias and unmeasured confounding. Second, the lack of external validation limits the model’s generalizability beyond the MIMIC-IV population, raising concerns about overfitting. The performance reported here, particularly the modest AUPRC, reflects the challenge of imbalanced data and may not translate directly to other settings. Third, our analysis focused on the first 24 h of ICU data, which, although enabling early prediction, does not account for the evolving clinical trajectories. Fourth, despite using multiple imputation, variables with extensive missing data, such as preoperative lactate, were excluded, which might have omitted relevant predictors. Fifth, missing-value imputation was performed on the full dataset prior to the train–test split, which may introduce information leakage and slightly optimistic performance estimates. However, because the imputation model used only predictor distributions without outcome information, the impact on model discrimination and relative model ranking is likely limited. Sixth, the relatively low event rate (PRF incidence 7.6%), while handled with metrics like AUPRC, still limits the model’s positive predictive value in practice—a common challenge with imbalanced clinical outcomes. Finally, our feature selection was based on a single method (LASSO) regression; alternative approaches could provide a more robust feature set [35].

To address these limitations and advance toward potential clinical application, our future research should focus on: [1] externally validating the model using multi-center datasets like eICU-CRD to test generalizability; [2] developing a user-friendly clinical decision support tool, such as a web-based calculator or EHR dashboard, for practical risk assessment; and [3] establishing a model updating framework to allow periodic retraining with new data; and [4] prospectively evaluate the clinical impact of acting upon the model’s predictions, particularly the thresholds identified by SHAP analysis. The ultimate step would be a prospective interventional trial to determine whether this model can improve patient outcomes.

Conclusion

The GBM model, selected for its balanced performance in internal validation, demonstrated promising ability for early PRF risk stratification after cardiac surgery. Its interpretability, facilitated by SHAP analysis, provides clinically plausible insights focused on hemodynamic and metabolic stability. However, as an internally validated model derived from a single database, it remains at an early stage of development. Significant further work, including external validation and prospective evaluation, is essential to confirm its generalizability and clinical utility before any implementation can be considered.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

PRF

Postoperative respiratory failure

ML

Machine learning

ICU

Intensive Care Unit

CPB

Cardiopulmonary bypass

MIMIC-IV

Mart for Intensive Care IV

ScvO₂

Venous oxygen saturation

BUN

Blood urea nitrogen

AUC

Area under the curve

IQR

Interquartile range

SHAP

SHapley Additive exPlanations

ICD

International classification of diseases

ROC

Receiver operating characteristic

LASSO

Least absolute shrinkage and selection operator

GBM

Gradient Boosting Machine

SVM

Support Vector Machine

NN

Neural Networks

XGBoost

Extreme gradient boosting

Adaboost

Adaptive boosting

LightGBM

Light gradient boosting machine

CatBoost

Categorical boosting

Author contributions

YM designed the study. RM extracted clinical data from the MIMIC-IV database. GY, CL and NL performed data cleaning. YL, HF performed the statistical analysis. YH contributed to data curation, critical code modification, and analytical methodology. HW, HF edited and improved this original manuscript. YM, LM and YH examined and revised the paper. All authors reviewed and approved the manuscript.

Funding

This research was supported by Guangxi Science and Technology Department, Guangxi Natural Science Foundation Project (No. 2025GXNSFBA069340 and No. 2025GXNSFBA069401).

Data availability

Data is available in the manuscript or supplementary information files. For additional inquiries, please reach out to the corresponding author.

Declarations

Ethics approval and consent to participate

This study adhered to the guidelines set forth by the Declaration of Helsinki and received approval from the Institutional Review Boards of the Massachusetts Institute of Technology (MIT) and Beth Israel Deaconess Medical Center for the utilization of the MIMIC-IV database. Given that the data is publicly accessible, the requirement for ethics approval and informed consent was waived. Clinical trial number: not applicable.

Consent for publication

All authors agree to publish this work.

Competing interests

The authors declare no competing interests.