Abstract
Objectives
Postoperative major bleeding is a common problem in patients undergoing cardiac surgery and is associated with poor outcomes. We evaluated the performance of machine learning (ML) methods to predict postoperative major bleeding.
Methods
A total of 1,045 patients who underwent isolated coronary artery bypass graft surgery (CABG) were enrolled. Their datasets were assigned randomly to training (70%) or a testing set (30%). The primary outcome was major bleeding defined as the universal definition of perioperative bleeding (UDPB) classes 3–4. We constructed a reference logistic regression (LR) model using known predictors. We also developed several modern ML algorithms. In the test set, we compared the area under the receiver operating characteristic curves (AUCs) of these ML algorithms with the reference LR model results, and the TRUST and WILL-BLEED risk score. Calibration analysis was undertaken using the calibration belt method.
Results
The prevalence of postoperative major bleeding was 7.1% (74/1,045). For major bleeds, the conditional inference random forest (CIRF) model showed the highest AUC [0.831 (0.732–0.930)], and the stochastic gradient boosting (SGBT) and random forest models demonstrated the next best results [0.820 (0.742–0.899) and 0.810 (0.719–0.902)]. The AUCs of all ML models were higher than [0.629 (0.517–0.641) and 0.557 (0.449–0.665)], as achieved by TRUST and WILL-BLEED, respectively.
Conclusion
ML methods successfully predicted major bleeding after cardiac surgery, with greater performance compared with previous scoring models. Modern ML models may enhance the identification of high-risk major bleeding subpopulations.
Keywords: major bleeding, machine learning, prediction model, cardiac surgery, coronary artery bypass graft (CABG) surgery
Introduction
Postoperative bleeding is a significant cause of morbidity and mortality in patients undergoing cardiac surgery, particularly after coronary artery bypass grafting (CABG) because they often are treated with potent antithrombotic drugs (1). Excessive bleeding after cardiac surgery is related to significant blood product transfusion, infections, longer mechanical ventilation, prolonged intensive care unit (ICU) stay, emergency surgical re-exploration, acute kidney injury, sepsis, and thromboembolic events (2–4). In addition, several studies have reported that major bleeding after cardiac surgery was also associated with increased hospital costs (5, 6). Therefore, identifying patients at high risk of bleeding will allow clinicians to take additional precautionary action to mitigate risk.
Several risk stratification models have been developed to support clinical decision, making such as the TRUST and WILL-BLEED risk scores (7, 8). All the models in current use are based on logistic regression (LR), which relied on manually inputting variables. Missing the complex interactions among features may result in model misspecification (9). Artificial intelligence is a rapidly growing field in cardiovascular medicine (10). Machine learning (ML) is a sub-field of artificial intelligence that may produce better predictive ability than traditional statistical analysis to predict postoperative outcomes. These approaches better account for high-order, non-linear interactions between predictors and can gain more stable predictions (11). Despite the success of ML in other clinical applications, no study has yet examined the utility of modern ML models to predict postoperative major bleeding in patients with isolated CABG.
The goal of this study was to develop ML models to accurately predict major bleeding after isolated CABG. We also compared the performance of several ML approaches with the reference LR model, the and the TRUST and WILL-BLEED scores.
Materials and methods
Data source and study population
We analyzed electronic health record data from Fuwai Hospital, Beijing, China from 1,045 consecutive adult patients who underwent elective isolated CABG between September 2017 and May 2018. The subjects were assigned randomly to training and test sets at a ratio of 7:3. This retrospective cohort study was approved by the Institutional Review Board with waived written consent.
Study variables
We collected datasets that included demographics, surgery-related data, and clinical and laboratory parameters. The continuous anti-platelet therapy was provided only in patients with acute coronary syndrome presenting for CABG. All data were obtained from the electronic health records as in recent articles (12, 13). Our database initially comprised over 100 easily obtained variables. Features that were missing for more than 50% of cases were excluded. The final set of input features included 60 clinical variables. Missing values (Supplementary Table 2) were imputed using the mean imputation method, which replaces the missing values with the mean of the available cases.
Definition of major bleeding
Development of postoperative major bleeding was defined according to the universal definition of perioperative bleeding (UDPB) in classes 3 and 4 (14, 15). The UDPB defines 5 perioperative bleeding classes, which are intended to characterize the severity of bleeding, regardless of its source. The details of the UDPB are presented in Supplementary Table 1.
Statistical analysis
Data are presented as means with SD, medians and interquartile ranges (IQRs), or frequencies and proportions depending on variable type and distribution. Differences between groups were compared using the Student’s t-test for normally distributed continuous variables, the Mann-Whitney U-test for non-normally distributed continuous variables, and the Pearson’s chi-square test of independence for categorical variables.
To implement the ML algorithms, we randomly assigned patient data into a training set (70%) for model derivation and a testing set (30%) to be used for model validation. Assignment to the training/testing sets was stratified on primary outcome status.
To select features for LR models, we conducted a backward stepwise selection procedure based on the Akaike information criterion (AIC). Model development included several common ML methods, such as support vector machines (SVM), stochastic gradient boosting (SGBT), extreme gradient boosting (XGBoost), random forest (RF), conditional inference random forest (CIRF), boosted classification trees, Naïve Bayes (NB), and bagged classification and regression tree (CART). We report the variables included in each model in Supplementary Table 4. The details of the R package were presented in Supplementary Material. We conducted a fivefold cross-validation to tune the parameters of the ML models on the training set and determined the best hyperparameter. Discrimination was evaluated with the receiver-operating-characteristic curve (ROC) and the area under the ROC curve (AUROC) (16). To evaluate the performance of ML algorithms, we compared the ROC between the models using Delong’s test by the pROC package (17). Calibration, which is the agreement between predicted probabilities and observed frequencies of postoperative major bleeding, was assessed with calibration belts (18). The net reclassification improvement was used to quantify whether a new model provides clinically relevant improvements in prediction (19, 20). We graphically demonstrated the net benefit of each model through a range of threshold probabilities of the outcome as a decision curve. The variable importance from ensemble algorithms was also determined, with the top 20 important variables from each algorithm identified. Analysis code is available upon request. All statistical analyses were performed using R software (version 4.1.1; The Comprehensive R Archive Network).
Results
Baseline characteristics
We assigned 732 and 313 patients to the training and testing sets, respectively. The variables remained constant between the two sets (Supplementary Table 3). The postoperative major bleeding rate were similar between the training and test sets. Overall, the median age was 62 years (IQR 55–66 years) and 78.4% were men. At baseline, patients who developed a major bleeding event were older, had lower body surface area (BSA), and were more likely to have a history of anemia. Other demographics and perioperative variables of the training set are listed in Table 1.
TABLE 1.
Baseline characteristics of the training set.
| Variables | Total (n = 732) | No major bleeding (n = 680) | Major bleeding (n = 52) | P |
| Age (years) | 62 (55–67) | 61 (55–66) | 65 (59–69) | 0.017 |
| Male, n (%) | 559 (76.37) | 524 (77.06) | 35 (67.31) | 0.127 |
| Height (m) | 1.68 (1.62–1.72) | 1.69 (1.63–1.72) | 165 (1.60–1.70) | 0.007 |
| Weight (kg) | 72.00 (65.00–80.00) | 72.00 (65.00–80.00) | 67.25 (62.00–74.25) | 0.001 |
| BSA (m2) | 1.83 ± 0.17 | 1.84 ± 0.17 | 1.76 ± 0.2 | 0.002 |
| Smoking history, n (%) | 375 (51.23) | 349 (51.32) | 26 (50.00) | 0.886 |
| Angina, n (%) | 699 (95.49) | 647 (95.15) | 52 (100.00) | 0.160 |
| Myocardial infarction, n (%) | 62 (8.47) | 57 (8.38) | 5 (9.62) | 0.961 |
| Arrhythmia, n (%) | 22 (3.01) | 20 (2.94) | 2 (3.85) | 0.665 |
| Previous surgery, n (%) | 176 (24.04) | 164 (24.12) | 12 (23.08) | 0.866 |
| Diabetes, n (%) | 272 (37.16) | 260 (38.24) | 12 (23.08) | 0.036 |
| Hyperlipidemia, n (%) | 603 (82.38) | 559 (82.21) | 44 (84.62) | 0.850 |
| Hypertension, n (%) | 459 (62.70) | 434 (63.82) | 25 (48.08) | 0.026 |
| Kidney failure, n (%) | 21 (2.87) | 20 (2.94) | 1 (1.92) | > 0.99 |
| Dialysis, n (%) | 17 (2.32) | 17 (2.50) | 0 (0) | 0.625 |
| Chronic pulmonary disease, n (%) | 11 (1.50) | 11 (1.62) | 0 (0) | > 0.99 |
| Congestive heart failure, n (%) | 10 (1.37) | 8 (1.18) | 2 (3.85) | 0.155 |
| Anemia, n (%) | 183 (25.00) | 162 (23.82) | 21 (40.38) | 0.012 |
| Peripheral vascular disease, n (%) | 174 (23.77) | 161 (23.68) | 13 (25.00) | 0.866 |
| Venous disease, n (%) | 46 (6.28) | 43 (6.32) | 3 (5.77) | > 0.99 |
| Cerebrovascular disease, n (%) | 84 (11.48) | 77 (11.32) | 7 (13.46) | 0.651 |
| Previous PTCA, n (%) | 16 (2.19) | 14 (2.06) | 2 (3.85) | 0.316 |
| Previous thrombolysis, n (%) | 5 (0.68) | 4 (0.59) | 1 (1.92) | 0.309 |
| CHD family history, n (%) | 77 (10.52) | 69 (10.15) | 8 (15.38) | 0.240 |
| Preoperative statin use, n (%) | 307 (41.94) | 286 (42.06) | 21 (40.38) | 0.885 |
| Preoperative anticoagulant use, n (%) | 639 (87.30) | 596 (87.65) | 43 (82.69) | 0.284 |
| Antiplatelet drugs pause < 5 days, n (%) | 12 (1.64) | 10 (1.47) | 2 (3.85) | 0.207 |
| Left main coronary artery disease, n (%) | 134 (18.31) | 127 (18.68) | 7 (13.46) | 0.457 |
| RBC (×1012/L) | 4.44 ± 0.52 | 4.45 ± 0.51 | 4.32 ± 0.54 | 0.064 |
| WBC (×109/L) | 6.36 (5.31–7.41) | 6.36 (5.30–7.41) | 6.31 (5.35–7.41) | 0.934 |
| PLT (×109/L) | 208 (176–247) | 209 (177–249) | 199 (167–228) | 0.066 |
| Platelet distribution width (fL) | 12.30 (11.20–13.80) | 12.30 (11.20–13.80) | 12.35 (11.20–13.90) | 0.841 |
| Platelet volume (fL) | 10.50 (10.00–11.20) | 10.5 (10.00–11.20) | 10.45 (10.07–11.33) | 0.637 |
| Platelet-large cell ration (%) | 29.40 (24.70–34.90) | 29.40 (24.58–34.73) | 28.65 (26.15–35.42) | 0.653 |
| Thrombocytocrit (%) | 0.22 (0.19–0.26) | 0.22 (0.19–0.26) | 0.21 (0.19–0.23) | 0.096 |
| Hemoglobin (g/L) | 137 (126–146) | 137 (127–146) | 137 (120–143) | 0.045 |
| Total protein (g/L) | 66.05 (62.90–70.10) | 66.15 (63.00–70.30) | 65.20 (61.65–68.03) | 0.061 |
| Albumin (g/L) | 40.80 (38.70–43.42) | 40.85 (38.70–43.60) | 40.40 (38.48–42.20) | 0.097 |
| Potassium (mmol/L) | 4.03 (3.77–4.25) | 4.03 (3.77–4.26) | 4.02 (3.72–4.13) | 0.326 |
| Sodium (mmol/L) | 141.10 (139.12–143.01) | 141.17 (139.13–143.01) | 140.66 (138.74–142.74) | 0.630 |
| Calcium (mmol/L) | 2.26 (2.19–2.34) | 2.26 (2.19–2.34) | 2.27 (2.19–2.33) | 0.887 |
| Glucose (mmol/L) | 5.34 (4.69–6.54) | 5.36 (4.69–6.55) | 5.05 (4.64–6.18) | 0.167 |
| BUN (mmol/L) | 5.23 (4.19–6.44) | 5.29 (4.19–6.44) | 5.07 (4.22–6.30) | 0.431 |
| Creatine (μmol/L) | 82.00 (70.48–94.00) | 82.00 (70.27–94.00) | 84.29 (72.30–95.43) | 0.527 |
| GFR (ml/min/1.73 m2) | 84.29 (71.51–93.94) | 84.52 (72.14–94.24) | 78.37 (64.62–92.18) | 0.051 |
| HSCRP (mg/L) | 1.38 (0.66–3.10) | 1.42 (0.67–3.10) | 1.08 (0.50–2.99) | 0.293 |
| NT-proBNP (pg/ml) | 156.35 (62.98–369.52) | 154.05 (62.27–373.00) | 177.75 (86.90–315.28) | 0.532 |
| PT (s) | 13.10 (12.60–13.60) | 13.10 (12.60–13.53) | 13.30 (12.97–13.62) | 0.045 |
| INR (R) | 1.00 (0.96–1.04) | 1.00 (0.95–1.04) | 1.02 (0.99–1.06) | 0.047 |
| CPB or not, n (%) | 477 (65.16) | 444 (65.29) | 33 (63.46) | 0.765 |
| Operation time (h) | 3.80 (3.30–4.50) | 3.80 (3.30–4.40) | 3.90 (3.38–4.93) | 0.275 |
| Blood loss (ml) | 589.54 (75.17) | 587.76 (68.81) | 612.69 (131.82) | 0.021 |
| Intraoperative transfusion, n (%) | 14 (1.91) | 7 (1.03) | 7 (13.46) | < 0.001 |
| Intraoperative urine output (ml/kg/h) | 3.16 (2.02) | 3.13 (2.01) | 3.54 (2.11) | 0.159 |
| Hemoglobin decrease (g/L) | 25.00 (16.00–33.00) | 25.00 (16.00–33.00) | 27.00 (14.25–33.00) | 0.961 |
| Postoperative first Creatine (μmol/L) | 70.27 (60.88–81.90) | 70.31 (60.88–81.80) | 68.97 (60.85–84.61) | 0.707 |
| Postoperative first NT-proBNP (pg/ml) | 602.10 (350.15–1060.75) | 596.30 (342.08–1042.75) | 690.90 (409.08–1124.25) | 0.108 |
| Preoperative hospital LOS (d) | 6 (4–9) | 6 (4–9) | 6 (4–8) | 0.596 |
| TRUST score | 2 (1–3) | 2 (1–3) | 2 (1–3) | 0.002 |
| WILL-BLEED score | 1 (1–3) | 1 (1–3) | 3 (1–4) | 0.013 |
BSA, body surface area; PTCA, percutaneous transluminal coronary angioplasty; CHD, coronary heart disease; RBC, red blood cell; WBC, white blood cell; PLT, platelet; BUN, blood urea nitrogen; GFR, glomerular filtration rate; HSCRP, high sensitivity C reactive protein; NT-proBNP, N-terminal prohormone of brain natriuretic peptide; PT, prothrombin time; INR, international normalized ratio; LOS, length of stay.
Development of postoperative major bleeding prediction model
The overall major bleeding rate was 7.1% (74 of 1,045 patients) in this study population. The variables included in each model are presented in Supplementary Table 4. Multivariable LR analysis with backward stepwise variable selection based on AIC is shown in Table 2. ROC analysis showed an adequate discriminatory ability of the LR model [0.702 (0.577–0.827)]. The AUC of the CRIF and SVM models performed significantly better (p < 0.05) than the LR model in the test set. The discrimination ability of models selected in the testing set is presented in Table 3.
TABLE 2.
Final logistic regression model.
| Variables* | OR (95%CI) | P-value* |
| BSA | 0.018 (0.002–0.133) | <0.001 |
| Diabetes | 0.537 (0.256–1.055) | 0.083 |
| Hypertension | 0.426 (0.577–0.827) | 0.008 |
| Total protein | 0.931 (0.882–0.981) | 0.008 |
| Creatine | 1.023 (1.004–1.041) | 0.014 |
| NT-proBNP | 1.000 (0.999–1.000) | 0.025 |
| Blood loss | 1.004 (1.000–1.007) | 0.029 |
| Intraoperative transfusion | 21.968 (5.879–84.392) | <0.001 |
OR, odds ratio; NT-proBNP, N-terminal prohormone of brain natriuretic peptide.
*The backward stepwise selection procedure base on Akaike information criterion was performed to select variables.
TABLE 3.
Prediction models of postoperative major bleeding in the test set.
| Models | AUC (95%CI) | P-value* | P-value† | P-value‡ |
| TRUST | 0.629 (0.517–0.741) | |||
| WILL-BLEED | 0.557 (0.449–0.665) | |||
| Logistic regression | 0.702 (0.577–0.827) | 0.347 | 0.103 | |
| Support vector machine | 0.792 (0.678–0.907) | 0.016 | 0.002 | 0.031 |
| Xgboost | 0.802 (0.691–0.913) | 0.028 | <0.001 | 0.120 |
| Random forest | 0.810 (0.719–0.902) | 0.005 | <0.001 | 0.084 |
| Conditional inference random forest | 0.831 (0.732–0.930) | 0.002 | <0.001 | 0.027 |
| Stochastic gradient boosting | 0.811 (0.739–0.883) | <0.001 | <0.001 | 0.073 |
| Naïve Bayes | 0.687 (0.561–0.813) | 0.468 | 0.059 | 0.842 |
| Bagged CART | 0.791 (0.706–0.876) | 0.008 | <0.001 | 0.098 |
| Boosted classification trees | 0.794 (0.675–0.913) | 0.007 | 0.003 | 0.117 |
AUC, the area under the receiver operating characteristic curve; CART, classification and regression tree.
*Compared with TRUST.
†Compared with WILL-BLEED.
‡Compared with the logistic regression model.
We used the following ML methods in putting all the variables, including SVM, SGBT, XGBoost, RF, CIRF, boosted classification trees, NB, and bagged CART to predict major bleeding after isolated CABG. The ROC of the evaluated models are presented in Figure 1. The AUC of the CIRF model was 0.831 (0.732–0.930), which was the highest among the ML models. The SGBT and RF models achieved the next and the third-highest AUC value of 0.820 (0.742-0.899) and 0.810 (0.719–0.902), respectively. The AUC values of TRUST and WILL-BLEED for the prediction of major bleeding were 629 (0.517–0.741) and 0.557 (0.449–0.665), respectively. All ML models except the NB model performed remarkably better than the TRUST and WILL-BLEED risk scores (p < 0.05).
FIGURE 1.
Receiver-operating-characteristics (ROC) curves of the conventional model and machine learning models for postoperative major bleeding in the test set. The corresponding values of the area under the receiver-operating-characteristics curve (AUC) for each model are presented in Table 2.
The decision curve analysis (Figure 2) showed that the net benefit of ML surpassed that of the conventional scoring systems throughout the range of threshold probabilities, indicating that the ML models yielded a higher net benefit. The threshold probability is a level of certainty, above which the patient or physician would choose to intervene.
FIGURE 2.
Decision curve analysis of the conventional models and machine learning-based models. The X-axis indicates the threshold probability and Y-axis indicates the net benefit. The red solid line indicates the net benefit of all patients developing postoperative major bleeding events. Ochre’s solid line indicates the net benefit of no patients developing postoperative major bleeding events. LR, logistic regression; CIRF, conditional inference random forest; SGBT, stochastic gradient boosting.
The calibration belts of the ML models and the LR model for major bleeding are shown in Figure 3. The RF and SGBT models showed better calibration, while CIRF and LR models were not well-calibrated.
FIGURE 3.
Calibration belts of (A) logistic regression. (B) Random forest. (C) Conditional inference random forest. (D) Stochastic gradient boosting for postoperative major bleeding prediction in the test set.
Variable importance of machine learning models
Figure 4 shows the importance matrix plot of machine learning models. To gain insights into the relevance of each predictor, Figure 4A summarizes the 20 most important predictors used in the CIRF model. Of the top 20 most important features, 15 were preoperative variables and 5 were intraoperative variables. In the CIRF model, intraoperative transfusion was the most important variable in predicting postoperative major bleeding, followed by operation time and BSA. The determinative factors in the ML and logistic models were roughly the same.
FIGURE 4.
Variable importance of predictors in the ML models. The variable importance is a scaled measure to have a maximum value of 100. The predictors with variable importance of the top 20 are shown. (A) Conditional inference random forest. (B) Xgboost. (C) Random forest. (D) Stochastic gradient boosting.
Discussion
The clinical application of ML has emerged as a new tool to improve patient care. In this study, we compared the predictive accuracy of models for major bleeding after isolated CABG among ML techniques, a traditional statistical approach, and previous risk scoring models. The models developed using ML algorithms better predicted major bleeding events than conventional scoring systems, such as TRUST and WILL-BLEED.
To date, accurate prediction models that use statistical models for the risk of postoperative major bleeding have been lacking. The TRUST and WILL-BLEED scores using clinical and laboratory variables were developed using only preoperative factors (7, 8). These two prediction models were derived from multicenter databases and have demonstrated accurate prediction of bleeding or transfusion in cardiac surgery patients. However, these approaches showed poor performance in our dataset. Our study demonstrated the value of intraoperative and immediate postoperative data, which reflected the surgical-related factors to bleeding prediction. Integrating these features may elucidate the improved discrimination of our LR model compared with the TRUST and WILL-BLEED scores.
The LR model used in this study shares several risk factors for bleeding common to other prediction models. Interestingly, patients with hypertension could have a lower risk of postoperative major bleeding in this study. Patients with hypertension undergoing CABG had greater blood pressure variability, and lability as a protective marker demonstrating that autonomic nervous system integrity is associated with improved outcomes (21, 22). A retrospective analysis found that hypertension is a positive prognostic indicator in most patients undergoing anesthesia and surgery (23). A prior study has demonstrated that patients who developed major bleeding have lower systolic blood pressures than those who did not have major bleeding (24). These may be potential explanations for this contrary perception.
By capturing the non-linear association of predictors ML approaches may improve prediction in major bleeding (25). ML models in this study performed the best but at the cost of possibly increased complexity and lessened clinical significance. While overfitting in conventional models is often problematic, this study adopted a cross-validation method to mitigate the overfitting of ML models. A recent systematic review on the application of ML methods for predicting operative mortality following cardiac surgery demonstrated that ML models provide better discrimination ability (26). The XGBoost ML algorithm offered modest improvements across a variety of model performance measures compared with The Society of Thoracic Surgeons Predicted Risk of Mortality (STS PROM) (27). Zea-Vera and associates (28) recently developed an ML model to predict mortality, major morbidity, high total hospitalization cost, and 30-day re-admission. Nevertheless, Benedetto et al. (29) reported that the performance of ML techniques was not superior to an LR model in predicting operative mortality after cardiac surgery. Therefore, the additional value of ML in the development of prediction models for a variety of clinical conditions needs further investigation.
Decision curve analysis was developed as a method to determine whether the use of a prediction model in the clinic to inform decision-making would do more good than harm (30). This analysis indicated that the CIRF model improved the net benefit for predicting the postoperative major bleeding over the LR model and TRUST and WILL-BLEED scores. The threshold ranges displayed above the curves (in Figure 2) indicate how the ML models will apply to clinical practice. For example, if a patient’s threshold probability is 15%, according to the decision curve, the use of the CIRF model adds more benefit than either the treat-all-patients scheme or the treat-none scheme and even the other models.
Preventing postoperative bleeding is imperative to improve patient outcomes and decrease hospital costs. Prompt identification of patients at high-risk could help allow deployment of additional resources to achieve better outcomes, including precautions, such as preoperative correction of anemia, earlier discontinuation of antiaggregant drugs, detailed coagulation tests, better surgical homeostasis, and various procoagulant agent during the operation (31). Understanding which patients are at high risk for postoperative bleeding will also help guide future practice and mitigate worse outcomes associated with bleeding. Meticulous surgical technique aimed to reduce surgical site bleeding seems to be essential to prevent severe bleeding, exposure to blood products, and improve the prognosis after CABG.
Several potential limitations exist in our study. First, because it employed retrospective data from a single center at a tertiary hospital, the generalizability of these models will need further validation. Second, the study identified the most important variables concerning predicting postoperative major bleeding events, but we could not obtain certain degrees of risk, such as the relative risk, which is a common limitation of ML algorithms. In addition, we were only able to provide a comparison with the TRUST and WILL-BLEED scores and were unable to compute other risk scores because we lacked appropriate data. Finally, the most important variables reported in the ML model are not modifiable and accurate risk prediction may not be followed by improved patient outcomes. Thus, further prospective studies are required to evaluate the application of ML-based predictive models to clinical practice for the reduction of postoperative major bleeding risks.
Conclusion
Our study demonstrated that the ML technique of CIRF showed significantly better performance than the traditional LR analysis or previous scoring models in predicting postoperative major bleeding events for isolated CABG. By capturing the non-linear association of predictors, the ML approaches may facilitate optimal candidate selection and prognostication of patients undergoing isolated CABG. Future prospective studies are required to evaluate the application of predictive models based on ML to clinical practice and reduce bleeding risks.
Data availability statement
The dataset generated and analyzed during the current study is available from the corresponding author upon reasonable request.
Ethics statement
The studies involving human participants were reviewed and approved by the Ethics Committee of Fuwai Hospital. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.
Author contributions
YG and HA: designed the study, collected and analyzed the data, drafted the manuscript, and submitted the final version. XL and HA: writing–reviewing and editing. YG, XL, and LW: visualization, coordinated, and supervised data collection. SW, YY, YD, and JW: collected, checked, and validated the data. All authors read and approved the manuscript for submission.
Acknowledgments
We gratefully acknowledge Jon K. Moon (Ph.D., President, MEI Research, Ltd.) for his help with the language polishing.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2022.881881/full#supplementary-material
References
- 1.Knapik P, Knapik M, Zembala MO, Prybylowski P, Nadziakiewicz P, Hrapkowicz T, et al. In-hospital and mid-term outcomes in patients reoperated on due to bleeding following coronary artery surgery (from the KROK Registry). Interact Cardiovasc Thorac Surg. (2019) 29:237–43. 10.1093/icvts/ivz089 [DOI] [PubMed] [Google Scholar]
- 2.Billah B, Reid CM, Shardey GC, Smith JA. A preoperative risk prediction model for 30-day mortality following cardiac surgery in an Australian cohort. Eur J Cardiothorac Surg. (2010) 37:1086–92. 10.1016/j.ejcts.2009.11.021 [DOI] [PubMed] [Google Scholar]
- 3.Kinnunen EM, De Feo M, Reichart D, Tauriainen T, Gatti G, Onorati F, et al. Incidence and prognostic impact of bleeding and transfusion after coronary surgery in low-risk patients. Transfusion. (2017) 57:178–86. 10.1111/trf.13885 [DOI] [PubMed] [Google Scholar]
- 4.Stone GW, Clayton TC, Mehran R, Dangas G, Parise H, Fahy M, et al. Impact of major bleeding and blood transfusions after cardiac surgery: analysis from the acute catheterization and urgent intervention triage strategY (ACUITY) trial. Am Heart J. (2012) 163:522–9. 10.1016/j.ahj.2011.11.016 [DOI] [PubMed] [Google Scholar]
- 5.Newcomb AE, Dignan R, McElduff P, Pearse EJ, Bannon P. Bleeding after cardiac surgery is associated with an increase in the total cost of the hospital stay. Ann Thorac Surg. (2020) 109:1069–78. 10.1016/j.athoracsur.2019.11.019 [DOI] [PubMed] [Google Scholar]
- 6.Mehaffey JH, Hawkins RB, Byler M, Charles EJ, Fonner C, Kron I, et al. Cost of individual complications following coronary artery bypass grafting. J Thorac Cardiovasc Surg. (2018) 155:875.e–82.e. 10.1016/j.jtcvs.2017.08.144 [DOI] [PubMed] [Google Scholar]
- 7.Alghamdi AA, Davis A, Brister S, Corey P, Logan A. Development and validation of transfusion risk understanding scoring tool (TRUST) to stratify cardiac surgery patients according to their blood transfusion needs. Transfusion. (2006) 46:1120–9. 10.1111/j.1537-2995.2006.00860.x [DOI] [PubMed] [Google Scholar]
- 8.Biancari F, Brascia D, Onorati F, Reichart D, Perrotti A, Ruggieri VG, et al. Prediction of severe bleeding after coronary surgery: the WILL-BLEED Risk Score. Thromb Haemost. (2017) 117:445–56. 10.1160/TH16-09-0721 [DOI] [PubMed] [Google Scholar]
- 9.Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. (2019) 110:12–22. 10.1016/j.jclinepi.2019.02.004 [DOI] [PubMed] [Google Scholar]
- 10.Quer G, Arnaout R, Henne M, Arnaout R. Machine learning and the future of cardiovascular care: JACC state-of-the-art review. J Am Coll Cardiol. (2021) 77:300–13. 10.1016/j.jacc.2020.11.030 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med. (2019) 380:1347–58. 10.1056/NEJMra1814259 [DOI] [PubMed] [Google Scholar]
- 12.Qu J, Zhang D, Zhang H, Rao C, Chen S, Zhao Y, et al. Preoperative clopidogrel and outcomes in patients with acute coronary syndrome undergoing coronary artery bypass surgery. J Thorac Cardiovasc Surg. (2020) 163:1044.e–52.e. 10.1016/j.jtcvs.2020.03.118 [DOI] [PubMed] [Google Scholar]
- 13.Wang C, Gao Y, Tian Y, Wang Y, Zhao W, Sessler DI, et al. Prediction of acute kidney injury after cardiac surgery from preoperative N-terminal pro-B-type natriuretic peptide. Br J Anaesth. (2021) 127:862–70. 10.1016/j.bja.2021.08.015 [DOI] [PubMed] [Google Scholar]
- 14.Dyke C, Aronson S, Dietrich W, Hofmann A, Karkouti K, Levi M, et al. Universal definition of perioperative bleeding in adult cardiac surgery. J Thorac Cardiovasc Surg. (2014) 147:1458.e–63.e. 10.1016/j.jtcvs.2013.10.070 [DOI] [PubMed] [Google Scholar]
- 15.Bartoszko J, Wijeysundera DN, Karkouti K, Karkouti K, Callum J, Rao V, et al. Comparison of two major perioperative bleeding scores for cardiac surgery trials: universal definition of perioperative bleeding in cardiac surgery and European coronary artery bypass grafting bleeding severity grade. Anesthesiology. (2018) 129:1092–100. 10.1097/ALN.0000000000002179 [DOI] [PubMed] [Google Scholar]
- 16.Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. (2014) 35:1925–31. 10.1093/eurheartj/ehu207 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. (1988) 44:837–45. 10.2307/2531595 [DOI] [PubMed] [Google Scholar]
- 18.Nattino G, Finazzi S, Bertolini G. A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes. Stat Med. (2014) 33:2390–407. 10.1002/sim.6100 [DOI] [PubMed] [Google Scholar]
- 19.Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. (2015) 313:409–10. 10.1001/jama.2015.37 [DOI] [PubMed] [Google Scholar]
- 20.Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. 10.1177/0272989X06295361 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Krzych LJ, Krejca M, Bis J, Wasiak M, Vialek K, Szurlej D, et al. Perioperative blood pressure variability in patients undergoing coronary artery bypass grafting: its magnitude and determinants. Blood Press Monit. (2011) 16:129–33. 10.1097/MBP.0b013e328346d69f [DOI] [PubMed] [Google Scholar]
- 22.Filipovic M, Jeger R, Probst C, Girard T, Pfisterer M, Gürke L, et al. Heart rate variability and cardiac troponin I are incremental and independent predictors of one-year all-cause mortality after major noncardiac surgery in patients at risk of coronary artery disease. J Am Coll Cardiol. (2003) 42:1767–76. 10.1016/j.jacc.2003.05.008 [DOI] [PubMed] [Google Scholar]
- 23.Levin MA, Fischer GW, Lin HM, McCormick PJ, Krol M, Reich DL, et al. Intraoperative arterial blood pressure lability is associated with improved 30 day survival. Br J Anaesth. (2015) 115:716–26. 10.1093/bja/aev293 [DOI] [PubMed] [Google Scholar]
- 24.Mathews R, Peterson ED, Chen AY, Wang TY, Chin CT, Fonarow GC, et al. In-hospital major bleeding during ST-elevation and non-ST-elevation myocardial infarction care: derivation and validation of a model from the ACTION Registry®-GWTG™. Am J Cardiol. (2011) 107:1136–43. 10.1016/j.amjcard.2010.12.009 [DOI] [PubMed] [Google Scholar]
- 25.Kuhn M, Johnson K. Applied Predictive Modeling. New York, NY: Springer; (2013). [Google Scholar]
- 26.Benedetto U, Dimagli A, Sinha S, Cocomello L, Gibbison B, Caputo M, et al. Machine learning improves mortality risk prediction after cardiac surgery: systematic review and meta-analysis. J Thorac Cardiovasc Surg. (2020) 163:2075–2087e.9. 10.1016/j.jtcvs.2020.07.105 [DOI] [PubMed] [Google Scholar]
- 27.Kilic A, Goyal A, Miller JK, Gjekmarkaj E, Tam WL, Gleason TG, et al. predictive utility of a machine learning algorithm in estimating mortality risk in cardiac surgery. Ann Thorac Surg. (2020) 109:1811–9. 10.1016/j.athoracsur.2019.09.049 [DOI] [PubMed] [Google Scholar]
- 28.Zea-Vera R, Ryan CT, Havelka J, Corr SJ, Nguyen TC, Chatterjee S, et al. Machine learning to predict outcomes and cost by phase of care after coronary artery bypass grafting. Ann Thorac Surg. (2021) 25:S000304975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Benedetto U, Sinha S, Lyon M, Dimagli A, Gaunt TR, Angelini G, et al. Can machine learning improve mortality prediction following cardiac surgery? Eur J Cardiothorac Surg. (2020) 58:1130–6. 10.1093/ejcts/ezaa229 [DOI] [PubMed] [Google Scholar]
- 30.Vickers AJ, Holland F. Decision curve analysis to evaluate the clinical benefit of prediction models. Spine J. (2021) 21:1643–8. 10.1016/j.spinee.2021.02.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Pagano D, Milojevic M, Meesters MI, Venedetto U, Bolliger D, von Heymann C, et al. 2017 EACTS/EACTA guidelines on patient blood management for adult cardiac surgery. Eur J Cardiothorac Surg. (2018) 53:79–111. 10.1093/ejcts/ezx325 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The dataset generated and analyzed during the current study is available from the corresponding author upon reasonable request.