Abstract
Objectives:
To evaluate the association between preoperative heart failure and reoperative cardiac surgical outcomes in adult congenital heart disease and to develop a risk model for postoperative morbidity/mortality.
Methods:
Single-institution retrospective cohort study of adult congenital heart disease patients undergoing reoperative cardiac surgery 1/1/2010-3/30/2022. Heart failure defined clinically as preoperative diuretic use and either New York Heart Association Class II-IV or systemic ventricular ejection fraction <40%. Composite outcome included operative mortality, mechanical circulatory support, dialysis, unplanned noncardiac reoperation, persistent neurologic deficit, cardiac arrest. Multivariable logistic regression and machine learning analysis using gradient boosting technology were performed. Shapley statistics determined feature influence, or impact, on model output.
Results:
Preoperative heart failure was present in 376/1011 (37%); those patients had longer postoperative length of stay (6 [5,8] vs 5 [4,7], p<0.001), increased postoperative mechanical circulatory support (21/376 [6%] vs 16/635 [3%], p=0.015), and decreased long-term survival (84% [80%,89%] vs 90% [86%,93%] at 10 years, p=0.002). A 7-feature machine learning risk model for the composite outcome achieved higher AUC (0.76) than logistic regression, and ejection fraction was most influential (highest mean ∣Shapley value∣). Additional risk factors for the composite outcome included age, number of prior cardiopulmonary bypass operations, urgent/emergent procedure, and functionally univentricular physiology.
Conclusions:
Heart failure is common among adult congenital heart disease patients undergoing cardiac reoperation and associated with longer length of stay, increased postoperative mechanical circulatory support, and decreased long-term survival. Machine learning yields a novel 7-feature risk model for postoperative morbidity/mortality, in which ejection fraction was the most influential.
Keywords: Machine learning, survival, postoperative morbidity, mechanical circulatory support
Introduction
Adults with congenital heart disease (CHD) have been shown to have higher rates of cardiovascular risk factors and adverse cardiovascular events compared to the average population.1 These patients are also at high risk of developing heart failure due to long-standing effects of residual, recurrent, and palliated CHD lesions on ventricular function.2-5 Heart failure is now the most common late complication seen in adults with CHD, and is associated with increased rates of hospitalization and death.6-8
Therefore, it is important to evaluate the prevalence of heart failure among adult CHD patients undergoing reoperative cardiac surgery and its impact on postoperative outcomes. Heart failure is a known risk factor for postoperative morbidity and mortality following non-congenital adult cardiac surgery, and history of congestive heart failure and ejection fraction (EF) are included in the Society of Thoracic Surgeons (STS) adult cardiac surgery risk models.9,10
Also unclear is the impact of heart failure on reoperative cardiac surgical outcomes in adult CHD patients. Existing adult CHD surgical mortality risk models utilizing the STS Adult Cardiac Surgery Database (ACSD) focus solely on early postoperative mortality and have not been developed to assess the impact of heart failure on postoperative morbidity nor on late mortality.11-14 Risk prediction plays an important role in clinical patient assessment and preoperative counseling. Traditionally, most risk models have used regression models built upon a limited number of predictors which are assumed to operate in the same way on all patients and uniformly throughout their range. Machine learning (ML) overcomes these limitations by not assuming the uniform influence of predictors on outcomes and instead searching for the optimal fit, leading to more accurate and more clinically applicable risk prediction models.15 Another limitation of the existing STS adult CHD models is their complexity, which restricts their use to database-level research and case-mix adjustment for outcomes reporting.
Therefore, the objective of this study was to evaluate the association of preoperative heart failure with a composite morbidity/mortality outcome and late survival in adults with CHD undergoing cardiac reoperation. We sought to derive and validate a risk model that can be used in daily clinical practice to guide clinical decision making at the time of preoperative assessment in adults with CHD being considered for cardiac reoperation.
Methods
This retrospective cohort study was approved on 6/10/2022 by the Mayo Clinic Institutional Review Board with waiver of informed consent (IRB #22-005681). We identified 1,011 unique adults with CHD undergoing reoperative cardiac surgery at Mayo Clinic Rochester between 1/1/2010 – 3/30/2022. Isolated coronary artery bypass graft surgery, primary repair of CHD, and heart transplantation were excluded. Patients were categorized into the following four CHD groups: biventricular physiology and systemic left ventricle dysfunction, biventricular physiology and subpulmonary right ventricle dysfunction, biventricular physiology, and systemic right ventricle dysfunction, functionally univentricular physiology.
Heart failure was defined as preoperative diuretic use and either New York Heart Association (NYHA) Class II-IV symptoms or systemic ventricular EF <40%. The Bethesda classification for severe CHD was used.16,17 The updated 2020 STS-European Association for Cardio-Thoracic Surgery (STAT) mortality categories were included as covariates in our analysis.18 The chromosomal abnormality/syndrome risk groups defined by Jacobs et al for inclusion in the STS Congenital Heart Surgery Database (CHSD) Mortality Risk Model were assessed in our cohort.19 Patients were categorized as having pulmonary hypertension based on ICD-10 diagnosis codes. The study outcome was a composite of operative mortality, postoperative mechanical circulatory support, dialysis, unplanned noncardiac reoperation, neurologic deficit persisting at discharge, and cardiac arrest. The morbidity components for this composite outcome are a subset of the components included in the STS congenital heart surgery composite quality metric.20 Current vital status data was obtained using Accurint® (LexisNexis, Alpharetta, GA), therefore late survival outcomes reflect all-cause mortality.
Conventional statistical analysis was performed using RStudio Version 4.0.3 (RStudio, PBC; Boston, Massachusetts) with statistical significance defined as p<0.05. Categorical variables are reported as number (%) and continuous variables are reported as median (interquartile range). Kruskal-Wallis rank sum test and Fisher’s exact test were used for subgroup comparisons where appropriate and Kaplan-Meier survival analysis with log-rank test and Cox proportional hazards analysis were performed. Multivariable logistic regression using the leave-one-out cross-validation method was performed on the whole dataset to determine the predictive performance of the model. Predictors were defined a priori based on prior research, bivariate analysis of preoperative characteristics comparing groups with and without the composite outcome, and clinical expertise. Multivariable logistic regression using the same method was repeated with predictors identified from the ML model. EF, when considered separately from heart failure, was modeled as a continuous variable.
For the ML analysis, data were divided into training, testing, and validation cohorts comprised of 70%, 20%, and 10% of randomized total data with balanced proportions of true positive and true negative events. The training cohort was modeled using gradient boosting technology (XGBoost) and hyperparameter tuning was conducted by starting with XGBoost defaults and then iterating about eta, max_depth, subsample, and num_rounds to maximize area under the receiver operating characteristic curve (AUC) while minimizing variance.21 Bootstrapping was performed with 10,000 iterations to determine confidence intervals (CI) on both a full model with 18 predictors which were determined a priori based on conventional statistical analysis and a feature-reduced model of seven clinical variables selected based on feature contribution in the full model, estimated using Shapley statistics.22 Shapley statistics determined feature influence, or impact, on model output. In this manuscript the term “influence” indicates impact on model output. We selected a single model at the 90th percentile AUC from both the full and the reduced feature model distributions and assessed robustness using k-fold balanced cross validation with five folds.
Results
During the study period 1,011 adults with CHD underwent reoperative cardiac surgery at our institution. There were 482/1011 (47.7%) patients with biventricular physiology and systemic left ventricle dysfunction, 466/1011 (46.1%) with biventricular physiology and subpulmonary right ventricle dysfunction, 27/1011 (2.7%) with biventricular physiology and systemic right ventricle dysfunction, and 36/1011 (3.6%) with functionally univentricular physiology. EF <40% was present in only 33 patients: 16 patients with biventricular physiology and systemic left ventricle dysfunction, 10 patients with biventricular physiology and subpulmonary right ventricle dysfunction, and 7 patients with biventricular physiology and systemic right ventricle dysfunction. Data on preoperative NYHA Class symptoms were available in 711/1011 (70.3%) of the cohort: 116/711 (16.3%) had Class I, 258/711 (36.3%) had Class II, 294/711 (41.4%) had Class III, and 43/711 (6.0%) had Class IV symptoms. The number of patients with chromosomal abnormalities and/or syndromes in the risk groups included in the STS CHSD Mortality Risk Model were few: 41/1011 (4.1%) in risk group 1, 13/1011 (1.3%) in risk group 2, 20/1011 (2.0%) in risk group 3, and 7/1011 (0.7%) in risk group 4.
There were 376/1011 (37%) patients with preoperative heart failure. The heart failure group was older (41.5 [32.0, 57.0] years vs 37.0 [26.5, 49.0] years, p<0.001), less commonly male (179/376 [47.6%] vs 351/635 [55.3%], p=0.019), more likely to have diabetes (28/376 [7.4%] vs 23/635 [3.6%], p=0.011) and pulmonary hypertension (97/376 [25.8%] vs 121/635 [19.1%], p=0.014 ; Table 1). Preoperative testing revealed higher brain natriuretic peptide (BNP) levels and more frequent >moderate systemic atrioventricular valve regurgitation among patients with heart failure (Table 1). The status of the procedure, operative procedure type, and cardiopulmonary bypass (CPB) time did not differ between the groups, although patients with heart failure were more likely to undergo STAT Mortality Category 3 procedures (Table 1). The eight most common primary procedure codes, in decreasing order, were pulmonary valve replacement, tricuspid valve replacement, tricuspid valvuloplasty, mitral valve replacement, Ebstein’s repair, pulmonary artery branch reconstruction (central), aortic arch repair, and right ventricular outflow tract procedure.
Table 1:
Preoperative and Operative Details Stratified by Preoperative Heart Failure Diagnosis
| Preoperative Heart Failure | P-Value | ||
|---|---|---|---|
| No N=635 |
Yes N=376 |
||
| Age (years) | 37.0 (26.5, 49.0) | 41.5 (32.0, 57.0) | < 0.001 |
| Male | 351 (55.3%) | 179 (47.6%) | 0.019 |
| Severe Congenital Heart Disease | 280 (44.1%) | 167 (44.4%) | 0.948 |
| Preoperative Factors | |||
| # Prior CPB Operations | 1.0 (1.0, 2.0) | 1.0 (1.0, 2.0) | 0.431 |
| Mechanical Circulatory Support | 5 (0.8%) | 1 (0.3%) | 0.421 |
| Neurologic Deficit | 26 (4.1%) | 14 (3.7%) | 0.868 |
| Ventilator Support | 7 (1.1%) | 6 (1.6%) | 0.568 |
| Diabetes | 23 (3.6%) | 28 (7.4%) | 0.011 |
| Pulmonary Hypertension | 121 (19.1%) | 97 (25.8%) | 0.014 |
| ICD | 21 (3.3%) | 16 (4.3%) | 0.489 |
| PPM | 48 (7.6%) | 42 (11.2%) | 0.053 |
| Preoperative Testing | |||
| Creatinine | 0.9 (0.8, 1.1) | 0.9 (0.8, 1.1) | 0.608 |
| BNP (N=226) | 217 (137, 532) | 563 (229, 1344) | < 0.001 |
| Total Bilirubin (N=550) | 0.7 (0.5, 1.0) | 0.7 (0.5, 1.1) | 0.258 |
| >Moderate Systemic AV Valve Regurgitation (N=340) | 9 (5.0%) | 22 (13.7%) | 0.008 |
| Systemic Ventricle EF % | 60.0 (55.0, 64.0) | 59.0 (53.8, 63.0) | 0.003 |
| Predicted VO2 Max <60% (N=144) | 34 (47.9%) | 36 (49.3%) | 0.869 |
| VE/VCO2 >30 (N=143) | 19 (26.0%) | 21 (30.0%) | 0.710 |
| Operative Details | |||
| Urgent/Emergent Procedure | 36 (5.7%) | 19 (5.1%) | 0.898 |
| CPB Time (minutes) | 88 (60, 160) | 92 (61, 138) | 0.593 |
| Operative Procedures | 0.296 | ||
| Subaortic | 165 (26.0%) | 85 (22.6%) | |
| Subpulmonary | 346 (54.5%) | 207 (55.1%) | |
| Both | 124 (19.5%) | 83 (22.1%) | |
| Neither a | 0 (0.0%) | 1 (0.3%) | |
| STAT Mortality Category | 0.013 | ||
| 1 | 209 (32.9%) | 96 (25.5%) | |
| 2 | 212 (33.4%) | 118 (31.4%) | |
| 3 | 183 (28.8%) | 143 (38.0%) | |
| 4 | 31 (4.9%) | 19 (5.1%) | |
N provided for variables if not available for the whole cohort. (a) 1 pericardiectomy; AV=atrioventricular; BNP=brain natriuretic peptide; CPB=cardiopulmonary bypass; EF=ejection fraction; ICD=implantable cardioverter-defibrillator; PPM=permanent pacemaker; STAT=Society of Thoracic Surgery-European Association for Cardio-Thoracic Surgery; VO2=oxygen consumption; VE/VCO2=minute ventilation/carbon dioxide production
Statistically significant p values are italicized and in bold
The unadjusted operative mortality rate for the whole cohort was 18/1011 (1.8%) and this did not differ between patients with and without heart failure. Patients with heart failure had longer postoperative length of stay (6 [5, 8] days vs 5 [4, 7] days, p<0.001) and higher rates of postoperative mechanical circulatory support (21/376 [6%] vs 16/635 [3%], p=0.015; Table 2) despite having equivalent rates preoperatively (1/376 [0.3%] vs 5/635 [0.8%], p=0.421; Table 1). There were no statistically significant differences between the groups with and without heart failure for any of the four other major postoperative complications included in the composite morbidity/mortality outcome.
Table 2:
Early Outcomes Stratified by Preoperative Heart Failure Diagnosis
| Preoperative Heart Failure | P-Value | ||
|---|---|---|---|
| No N=635 |
Yes N=376 |
||
| Postoperative Length of Stay | 5.0 (4.0, 7.0) | 6.0 (5.0, 8.0) | < 0.001 |
| Operative Mortality | 9 (1.4%) | 9 (2.4%) | 0.325 |
| Postoperative Complications | |||
| Mechanical Circulatory Support | 16 (2.5%) | 21 (5.6%) | 0.015 |
| Cardiac Arrest | 8 (1.3%) | 11 (2.9%) | 0.090 |
| Dialysis | 13 (2.0%) | 14 (3.7%) | 0.156 |
| Neurologic Deficit at Discharge | 4 (0.6%) | 4 (1.1%) | 0.479 |
| Unplanned Noncardiac Reoperation | 28 (4.4%) | 16 (4.3%) | > 0.99 |
Statistically significant p values are italicized and in bold
Table 3 shows preoperative and operative details stratified by the composite morbidity/mortality outcome. Those with the composite outcome were older, had more prior CPB operations and were more likely to have preoperative heart failure, mechanical circulatory support, ventilator support, diabetes, pulmonary hypertension, implantable cardioverter-defibrillators (ICDs), and permanent pacemakers (PPMs; Table 3). These patients also had higher BNP levels preoperatively and were significantly more likely to be undergoing an urgent/emergent procedure, a combined subaortic and subpulmonary operation, and a higher STAT Mortality Category procedure (Table 3). Consistent with the differences in procedures, the CPB time was longer in the group with the composite outcome (Table 3).
Table 3:
Preoperative and Operative Details Stratified by Composite Outcome
| Composite Morbidity/Mortality Outcome |
P-Value | ||
|---|---|---|---|
| No N=922 |
Yes N=89 |
||
| Age (years) | 39.0 (28.0, 51.0) | 46.0 (31.0, 60.0) | 0.006 |
| Male | 488 (52.9%) | 42 (47.2%) | 0.319 |
| Severe Congenital Heart Disease | 407 (44.1%) | 40 (44.9%) | 0.911 |
| Preoperative Factors | |||
| Heart Failure | 334 (36.2%) | 42 (47.2%) | 0.050 |
| # Prior CPB Operations | 1.0 (1.0, 2.0) | 2.0 (1.0, 3.0) | < 0.001 |
| Mechanical Circulatory Support | 2 (0.2%) | 4 (4.5%) | < 0.001 |
| Neurologic Deficit | 33 (3.6%) | 7 (7.9%) | 0.078 |
| Ventilator Support | 9 (1.0%) | 4 (4.5%) | 0.022 |
| Diabetes | 41 (4.4%) | 10 (11.2%) | 0.010 |
| Pulmonary Hypertension | 187 (20.3%) | 31 (34.8%) | 0.003 |
| ICD | 29 (3.1%) | 8 (9.0%) | 0.012 |
| PPM | 72 (7.8%) | 18 (20.2%) | < 0.001 |
| Preoperative Testing | |||
| Creatinine | 0.9 (0.8, 1.1) | 0.9 (0.8, 1.2) | 0.095 |
| BNP (N=226) | 345 (153, 927) | 785 (335, 1648) | 0.033 |
| Total Bilirubin (N=550) | 0.7 (0.5, 1.0) | 0.8 (0.5, 1.1) | 0.055 |
| >Moderate Systemic AV Valve Regurgitation (N=340) | 31 (9.6%) | 0 (0.0%) | 0.383 |
| Systemic Ventricle EF % | 60.0 (55.0, 64.0) | 60.0 (50.5, 63.0) | 0.129 |
| Predicted VO2 Max <60% (N=144) | 66 (47.8%) | 4 (66.7%) | 0.432 |
| VE/VCO2 >30 (N=143) | 37 (27.0%) | 3 (50.0%) | 0.349 |
| Operative Details | |||
| Urgent/Emergent Procedure | 35 (4.0%) | 20 (22.0%) | < 0.001 |
| CPB Time (minutes) | 86 (59, 137) | 172 (100, 254) | < 0.001 |
| Operative Procedures | < 0.001 | ||
| Subaortic | 235 (25.5%) | 15 (16.9%) | |
| Subpulmonary | 517 (56.1%) | 36 (40.4%) | |
| Both | 169 (18.3%) | 38 (42.7%) | |
| Neithera | 1 (0.1%) | 0 (0.0%) | |
| STAT Mortality Category | < 0.001 | ||
| 1 | 292 (31.7%) | 13 (14.6%) | |
| 2 | 305 (33.1%) | 25 (28.1%) | |
| 3 | 284 (30.8%) | 42 (47.2%) | |
| 4 | 41 (4.4%) | 9 (10.1%) | |
N provided for variables if not available for the whole cohort. (a) 1 pericardiectomy; AV=atrioventricular; BNP=brain natriuretic peptide; CPB=cardiopulmonary bypass; EF=ejection fraction; ICD=implantable cardioverter-defibrillator; PPM=permanent pacemaker; STAT=Society of Thoracic Surgery-European Association for Cardio-Thoracic Surgery; VO2=oxygen consumption; VE/VCO2=minute ventilation/carbon dioxide production
Statistically significant p values are italicized and in bold
Multivariable analysis was used to assess the association between explanatory variables (age, heart failure diagnosis, number of prior CPB operations, diabetes, pulmonary hypertension, ICD/PPM, creatinine, urgent/emergent procedure, and STAT Mortality Category) and the composite outcome (Table 4, Logistic Regression Model 1). Significant associations included number of prior CPB operations (odds ratio [OR] 1.45 [1.17, 1.79], p<0.001), ICD/PPM (OR 1.98 [1.04, 3.61], p=0.030), urgent/emergent procedure (OR 5.25 [2.69, 10.01], p<0.001), and STAT Mortality Category (OR 1.70 [1.05, 2.77], p=0.031). This full model performed moderately well in predicting the composite outcome, AUC 0.708 (0.65, 0.77). The STAT Mortality Category alone performed poorly, AUC 0.371 (0.31, 0.43).
Table 4:
Multivariable Logistic Regression Models
| Odds Ratio (95% CI) |
P-Value | |
|---|---|---|
| Logistic Regression Model 1 (AUC = 0.708 [0.65-0.77]) | ||
| Age | 1.01 (1.00, 1.03) | 0.123 |
| Heart Failure Diagnosis | 1.22 (0.75, 1.98) | 0.413 |
| Number of Prior CPB Operations | 1.45 (1.17, 1.79) | < 0.001 |
| Diabetes | 2.22 (0.94, 4.83) | 0.055 |
| Pulmonary Hypertension | 1.23 (0.71, 2.07) | 0.455 |
| ICD/PPM | 1.98 (1.04, 3.61) | 0.030 |
| Creatinine | 1.08 (0.72, 1.47) | 0.667 |
| Urgent/Emergent Procedure | 5.25 (2.69, 10.01) | < 0.001 |
| STAT Mortality Category | 1.70 (1.05, 2.77) | 0.031 |
| Logistic Regression Model 2 (AUC = 0.727 [0.67, 0.79]) | ||
| Age | 1.02 (1.01, 1.04) | 0.006 |
| Systemic Ventricle Ejection Fraction | 0.97 (0.94, 0.99) | 0.007 |
| Number of Prior CPB Operations | 1.56 (1.26, 1.93) | < 0.001 |
| Creatinine | 1.23 (0.83, 1.80) | 0.264 |
| Urgent/Emergent Procedure | 5.36 (2.65, 10.61) | < 0.001 |
| Functionally Univentricular Physiology | 3.71 (1.34, 9.23) | 0.007 |
| Male Sex | 0.62 (0.38, 1.02) | 0.064 |
OR provided for 1 unit increase in continuous variables (age = 1 year, ejection fraction = 1%, prior CPB operations = 1 operation, creatinine = 1 mg/dL). AUC=area under the receiver operating characteristic curve; CI=confidence interval; CPB=cardiopulmonary bypass; ICD=implantable cardioverter-defibrillator; PPM=permanent pacemaker; STAT=Society of Thoracic Surgery-European Association for Cardio-Thoracic Surgery
Statistically significant p values are italicized and in bold
Unadjusted long-term survival was significantly lower in the heart failure group following cardiac reoperation when compared to the group without heart failure, p=0.002 (Figure 1). Survival among those with heart failure was 94% (92%, 97%) at 1 year, 88% (85%, 92%) at 5 years, and 84% (80%, 89%) at 10 years. In comparison, survival among those without heart failure was significantly higher; 97% (96%, 99%) at 1 year, 94% (93%, 97%) at 5 years, and 90% (86%, 93%) at 10 years. When adjusted for the covariates included in our seven-feature ML model, the increased mortality risk for patients with heart failure became a trend, HR 1.57 (0.98-2.50, p=0.059). Several of the covariates were also associated with statistically significantly increased risk for late mortality, including age, number of prior CPB operations, serum creatinine, urgent/emergent procedure, and functionally univentricular physiology (Supplementary Table).
Figure 1:
Kaplan-Meier Survival Analysis Stratified by Preoperative Heart Failure Diagnosis Adult CHD patients with heart failure undergoing reoperative cardiac surgery have worse late survival compared to those without heart failure. 95% confidence limits are shown with shading bars.
ML analysis was used to assess a more complete set of predictor variables. Gradient boosting modeling of the full feature complement (18 preoperative factors determined to be influential based on conventional statistics and clinical expertise) resulted in test and validate AUCs of 0.786 and 0.760 (95% CI 0.564 – 0.810) with a 5-fold cross validation range of 0.635 – 0.778. The features contributing most to the full model included systemic ventricle EF, age, number of prior CPB operations, serum creatinine, urgent/emergent procedure, sex, and functionally univentricular physiology (Figure 2). These factors were all included in the reduced seven-feature ML model. Total bilirubin appeared to be influential but was left out of the reduced model due to high rates of missingness (45.6%) and not being captured in the STS cardiac surgical databases. Heart failure diagnosis was left out of the reduced model because systemic ventricle EF was more influential and already included.
Figure 2:
Shapley Statistics and AUCs for Full Machine Learning Model
(A) Features (predictors) ranked in order of most to least influential in predicting the composite outcome according to each predictor’s mean ∣Shapley value∣. Location of red dots relative to x-axis shows impact of variable on model output, with positive SHAP values on x-axis indicating increased risk and negative values indicating decreased risk. Highly discriminative features will have more separation of red and blue, although some overlap is expected for continuous variables. (B) Receiver operating characteristic curve. (C) Histogram of AUCs after bootstrapping was performed with 10,000 iterations. AUC=area under the receiver operating characteristic curve; AV=atrioventricular; BiV=biventricular; CPB=cardiopulmonary bypass; EF=ejection fraction; ICD=implantable cardioverter-defibrillator; LV=left ventricle; NYHA=New York Heart Association; PPM=permanent pacemaker; RV=right ventricle; SHAP=Shapley
The reduced seven-feature ML model included systemic ventricle EF, serum creatinine, age in years, number of prior CPB operations, sex, urgent/emergent procedure, and functionally univentricular physiology. Similarly, the reduced seven feature ML model resulted in test and validate AUCs of 0.763 and 0.782 (95% CI 0.568 – 0.804) with a 5-fold cross validation range of 0.631 – 0.764 (Figure 3). Each of the seven features demonstrated acceptable discrimination in predicting the outcome. Systemic ventricle EF had the highest mean ∣Shapley value∣ in both the full (0.38) and reduced (0.41) models, signifying that it was the most influential feature in both models (Figure 4). Sex influence modeling was conducted to determine its Shapley contribution value which was −0.0823, indicating that on average being female influences the model away from the composite outcome (is protective). However, inter-feature dependence means that, in some situations, being female could actually influence the model toward the composite outcome, although this is less likely.
Figure 3:
Shapley Statistics and AUCs for Seven-Feature Machine Learning Model
(A) Features (predictors) ranked in order of most to least influential in predicting the composite outcome according to each predictor’s mean ∣Shapley value∣. Location of red dots relative to x-axis shows impact of variable on model output, with positive SHAP values on x-axis indicating increased risk and negative values indicating decreased risk. Highly discriminative features will have more separation of red and blue, although some overlap is expected for continuous variables. (B) Receiver operating characteristic curve. (C) Histogram of AUCs after bootstrapping was performed with 10,000 iterations. AUC=area under the receiver operating characteristic curve; CPB=cardiopulmonary bypass; EF=ejection fraction; SHAP=Shapley
Figure 4:
Graphical Abstract
Heart failure is common and associated with decreased long-term survival. Machine learning yields a novel 7-feature risk model for postoperative morbidity/mortality, in which ejection fraction is the most influential.
CI=confidence interval
Finally, we wanted to assess whether the ML model or logistic regression model with the same predictors would achieve a higher AUC in this dataset. We performed multivariable logistic regression using the same seven factors and the AUC was 0.727 (0.67, 0.79), which was lower than the AUC from the ML model, 0.763, although the 95% CI overlap (Table 4). The logistic regression model with these seven predictors did achieve a higher AUC than the first logistic regression model and five of the seven predictors were statistically significant (Table 4).
Discussion
Heart failure is common among adult CHD patients undergoing cardiac reoperation, observed in 37% of this cohort, and is associated with longer postoperative length of stay, higher rates of postoperative mechanical circulatory support, and worse late survival. ML yielded a novel seven-feature risk model for postoperative morbidity and mortality, in which systemic ventricular EF was the most influential factor in predicting the composite morbidity/mortality outcome. Additional independent risk factors in both multivariable logistic regression and ML analyses included age, number of prior CPB operations, urgent/emergent procedure, and functionally univentricular physiology. Notably, age, number of prior CPB operations, serum creatinine, urgent/emergent procedure, and functionally univentricular physiology were also independent risk factors for late mortality.
In the STS ACSD, approximately 45% of the adult CHD diagnoses are isolated bicuspid aortic valve and most of the remainder are left-sided lesions.11 In the STS CHSD, among adults with at least one prior CPB operation, the most common operations are right-sided procedures, such as pulmonary valve replacement or conduit placement/reoperation, and arrythmia or device procedures.23 While there are more adult CHD patients captured in the STS ACSD overall, many patients with complex CHD and functionally univentricular physiology are disproportionately captured in the STS CHSD.11 As the adult CHD population is spread across two databases, it is difficult to determine if the distribution of CHD diagnoses in this single-institution cohort is similar to the national cohort. Additionally, many patients with isolated bicuspid aortic valve are more similar to patients with acquired, senile aortic stenosis and may be reasonably grouped in the adult cardiac cohort, as we have chosen to do at our institution. Therefore, in our cohort, CHD physiology is relatively balanced between left- and right-sided lesions, with a minority of patients with systemic right ventricle and functionally univentricular physiology.
Prior studies have shown that NYHA class, BNP, and CHD lesion characteristics are important predictors of heart failure in adult CHD.24 In this surgical cohort, patients with heart failure were older, had higher BNP levels, were more often female, and were more likely to have diabetes, pulmonary hypertension, and >moderate systemic atrioventricular valve regurgitation. Systemic atrioventricular valve regurgitation has been previously associated with heart failure and this study supports that in a broad adult CHD surgical cohort.25-28
The unadjusted operative mortality rate for this cohort was 1.8%, which compares favorably to the unadjusted mortality rate of 2.6% seen in national adult CHD surgical cohorts.29,30 The complication rates observed in this cohort are lower than rates reported by Nasr et al for the adult CHD population in the Nationwide Inpatient Sample database.29 Importantly, while prior research has demonstrated worse late survival among adult CHD patients with heart failure, this is the first study to show that this association remains true in the subset of adult CHD patients considered candidates for corrective CHD reoperations, i.e., non-transplant.6-8 After adjusting for the covariates included in our 7-feature ML model for the composite morbidity/mortality outcome, this difference in survival between those with heart failure and those without became statistically lest robust (HR 1.57, p=0.059).
Although preoperative heart failure in this study was defined based on diuretic use at the time of admission and either systemic ventricular EF <40% or NYHA Class II-IV symptoms, ML analysis demonstrated that systemic ventricular EF was the most influential aspect in predicting the composite morbidity/mortality outcome. EF was also significant in the multivariable logistic regression, with increasing EF protective against the composite outcome. Diastolic dysfunction in the presence of preserved EF remains an important factor affecting symptoms and survival for adult CHD patients, but the lack of consistent and uniformly applied imaging techniques to quantify diastolic dysfunction in CHD make this difficult to capture in risk prediction models based on retrospective datasets.31,32 Given the prevalence of heart failure in this cohort, the high number of patients with NYHA Class III or IV symptoms, and the impact of heart failure on postoperative morbidity and mortality, guideline-directed medical therapy is an important aspect of patient management in the adult CHD population.
The seven-feature ML model performed moderately well in this population and was equivalent to the full 18-feature model. The seven-feature ML model is less burdensome and could potentially be applied in daily clinical practice once it has been externally validated and shown to be generalizable to a national adult CHD surgical population. It will be important to assess how this model performs in a large national dataset and relative to the STS adult CHD risk model published by Nelson et al.11 Interestingly, the XGBoost ML modeling technique AUC was higher, although with overlap of 95% CI, than that achieved with conventional multivariable logistic regression in discriminating risk for the composite outcome, highlighting an opportunity to continue to explore ML techniques as the field of congenital cardiac surgery continues to work towards better prediction of patient risk for operative mortality and postoperative morbidity. Importantly, the iterative tree-building process of gradient boosting allows it to capture intricate non-linear relationships that might influence patient outcomes. The built-in regularization mechanisms further enhance the model's robustness by minimizing overfitting. Our analysis, encompassing alternative ML models like decision tree and random forest, revealed that gradient boosting machine had the best prediction accuracy of the ML techniques and had favorable results relative to logistic regression. This underlines its capability to sequentially optimize for the outcome of interest, making it particularly suited for the prediction of postoperative morbidity/mortality in adult CHD patients.
It remains unclear the impact that sex has on patient risk for the composite morbidity/mortality outcome. In the multivariable logistic regression models female sex appears to potentially increase risk for the outcome although the odds ratio was nonsignificant, while in the ML models female sex appears to be modestly protective against the outcome. Inter-feature dependence in the ML model could potentially affect the way sex influences the outcome. External validation of these models with a larger, national cohort is needed, such as with the STS ACSD, and additional analysis of Shapley contribution values in those datasets will be necessary to further characterize the influence of sex on this composite morbidity/mortality outcome. Of note, in the unadjusted bivariate analysis, while sex did not differ between those with and without the composite outcome, female sex was more common in the group with heart failure than the group without heart failure.
Limitations
The single-institution retrospective cohort design leads to inherent limitations and reduces the generalizability of these findings. Our institution is a referral center for adult CHD, so this cohort may not be reflective of the adult CHD surgical population as a whole. Pulmonary hypertension as a risk factor may not be fully understood due to limitations with using administrative (ICD code) data. Due to the retrospective nature of the study and lack of consistently applied imaging techniques to quantify diastolic dysfunction in CHD, we are unable to capture its prevalence in this population. Diastolic dysfunction may be more common in the CHD population and could be significant confounder in our analysis. There is selection bias associated with surgical cohorts, potentially limiting applicability of these findings to the broader adult CHD population. However, when considering the differences between the adult CHD cohorts captured in the STS ACSD and CHSD, this cohort has the benefit of including all adult CHD reoperative cardiac surgery cases performed at our institution during the study period and long-term outcomes. The relatively few postoperative complications and operative mortalities limited our ability to include more predictors in our multivariable logistic regression models. Nonetheless, the large cohort size allowed us to incorporate ML techniques which permitted more comprehensive preoperative factor analysis. External validation of the seven-feature ML model with a larger national cohort will be necessary to further assess its clinical utility and generalizability.
Conclusions
Heart failure is common among adult CHD patients undergoing cardiac reoperation and is associated with longer postoperative length of stay, higher rates of postoperative mechanical circulatory support, and worse late survival. ML yielded a novel seven-feature risk model for the composite morbidity/mortality outcome, in which systemic ventricular EF was the most influential. Age, number of prior CPB operations, urgent/emergent procedure, and functionally univentricular physiology were also identified as significant independent predictors in multivariable logistic regression and ML analyses. Continued surveillance of adult CHD patients is warranted to plan reoperative cardiac surgery in an elective fashion and prior to the development of decreased ventricular function and heart failure.
Supplementary Material
Central Picture Legend.
7-feature risk model for morbidity/mortality after cardiac reoperation in adult CHD.
Central Message.
Machine learning yields a 7-feature risk model for morbidity/mortality after cardiac reoperation in adult CHD, in which ejection fraction is most influential. Heart failure shows worse late survival.
Perspective Statement.
The impact of heart failure on reoperative cardiac surgical outcomes in adult CHD has not been well-defined. This study used machine learning to develop a novel 7-feature risk model for postoperative morbidity/mortality, in which ejection fraction was the most influential. Heart failure patients had longer length of stay, increased postoperative mechanical circulatory support, and worse late survival.
Funding Statement:
This work was supported by the National Heart, Lung, and Blood Institute [R38HL150086].
Glossary of Abbreviations
- ACSD
Adult Cardiac Surgery Database
- AUC
area under the receiver operating characteristic curve
- BNP
brain natriuretic peptide
- CHD
congenital heart disease
- CHSD
Congenital Heart Surgery Database
- CI
confidence interval
- CPB
cardiopulmonary bypass
- EF
ejection fraction
- ICD
implantable cardioverter-defibrillator
- ML
machine learning
- NYHA
New York Heart Association
- OR
odds ratio
- PPM
permanent pacemaker
- STAT
Society of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery
- STS
Society of Thoracic Surgeons
- XGBoost
gradient boosting technology
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Disclosure Statement: The authors have no conflicts of interest to disclose.
IRB Approval: IRB #22-005681, Approved on 6/10/2022
Meeting Presentation: AATS 2023 Annual Meeting Poster Presentation
References
- 1.Saha P, Potiny P, Rigdon J, Morello M, Tcheandjieu C, Romfh A, et al. Substantial cardiovascular morbidity in adults with lower-complexity congenital heart disease. Circulation. 2019;139:1889–99. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Burchill LJ, Lee MGY, Nguyen VP, Stout KK. Heart failure in adult congenital heart disease. Cardiol Clin. 2020;38:457–69. [DOI] [PubMed] [Google Scholar]
- 3.Arnaert S, De Meester P, Troost E, Droogne W, Van Aelst L, Van Cleemput J, et al. Heart failure related to adult congenital heart disease: prevalence, outcome and risk factors. ESC Heart Failure. 2021;8:2940–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Menachem JN, Schlendorf KH, Mazurek JA, Bichell DP, Brinkley DM, Frischhertz BP, et al. Advanced heart failure in adults with congenital heart disease. J Am Coll Cardiol HF. 2020;8:87–99. [DOI] [PubMed] [Google Scholar]
- 5.Brida M, Lovric D, Griselli M, Gil FR, Gatzoulis MA. Heart failure in adults with congenital heart disease. Int J Cardiol. 2022;357:39–45. [DOI] [PubMed] [Google Scholar]
- 6.Burchill LJ, Gao L, Kovacs AH, Opotowsky AR, Maxwell BG, Minnier J, et al. Hospitalization trends and health resource use for adult congenital heart disease-related heart failure. J Am Heart Assoc. 2018;7:e008775. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Burstein DS, Rossano JW, Griffis H, Zhang X, Fowler R, Frischertz B, et al. Greater admissions, mortality and cost of heart failure in adults with congenital heart disease. Heart. 2021;107:807–13. [DOI] [PubMed] [Google Scholar]
- 8.Tsang W, Silversides CK, Rashid M, Roche SL, Alonso-Gonzalez R, Austin PC, et al. Outcomes and healthcare resource utilization in adult congenital heart disease patients with heart failure. ESC Heart Failure. 2021;8:4139–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Jacobs JP, Shahian DM, Badhwar V, Thibault DP, Thourani VH, Rankin JS, et al. The Society of Thoracic Surgeons 2021 adult cardiac surgery risk models for multiple valve operations. Ann Thorac Surg. 2022;113:511–18. [DOI] [PubMed] [Google Scholar]
- 10.Zhang Y, Liu X, Ding Q, Berguson M, Morris R, Liu H, et al. Perioperative renin-angiotensin system inhibitors improve major outcomes of heart failure patients undergoing cardiac surgery. Ann Surg. 2023;277:e948–54. [DOI] [PubMed] [Google Scholar]
- 11.Nelson JS, Thibault D, O’Brien SM, Feins EN, Jacobs JP, Mayer JE, et al. Development of a novel Society of Thoracic Surgeons adult congenital mortality risk model. Ann Thorac Surg. 2023;S0003-4975(23)00032–2. [DOI] [PubMed] [Google Scholar]
- 12.von Ohain JP, Sarris G, Tobota Z, Maruszewski B, Vida VL, Horer J. Risk evaluation in adult congenital heart surgery surgery: analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database risk models on data from the European Congenital Heart Surgeons Association Congenital Database. Eur J Cardiothorac Surg. 2021;60:1397–404. [DOI] [PubMed] [Google Scholar]
- 13.Fuller SM, He X, Jacobs JP, Pasquali SK, Gaynor JW, Mascio CE, et al. Estimating mortality risk for adult congenital heart surgery: an analysis of the Society of Thoracic Surgeons congenital heart surgery database. Ann Thorac Surg. 2015;100:1728–36. [DOI] [PubMed] [Google Scholar]
- 14.Jacobs JP, Nelson JS, Fuller S, Scholl FG, Kumar SR, Jacobs ML. Risk adjustment for cardiac surgery in adults with congenital heart disease: what do we know and what do we need to learn? Eur J Cardiothorac Surg. 2021;60:1405–7. [DOI] [PubMed] [Google Scholar]
- 15.Goldstein BA, Navar AM, Carter RE. Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges. Eur Heart J. 2017;38:1805–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Marelli AJ, Mackie AS, Ionescu-Ittu R, Rahme E, Pilote L. Congenital heart disease in the general population: changing prevalence and age distribution. Circulation. 2007;115:163–172. [DOI] [PubMed] [Google Scholar]
- 17.Warnes CA, Liberthson R, Danielson GK, Dore A, Harris L, Hoffman JI, et al. Task force 1: the changing profile of congenital heart disease in adult life. J Am Coll Cardiol. 2001;37:1170–1175. [DOI] [PubMed] [Google Scholar]
- 18.Jacobs ML, Jacobs JP, Thibault D, Hill KD, Anderson BR, Eghtesady P, et al. Updating an empirically based tool for analyzing congenital heart surgery mortality. World J Pediatr Congenit Heart Surg. 2021;12:246–81. [DOI] [PubMed] [Google Scholar]
- 19.Jacobs JP, O’Brien SM, Hill KD, Kumar SR, Austin EH, Gaynor JW, et al. Refining the Society of Thoracic Surgeons congenital heart surgery database mortality risk model with enhanced risk adjustment for chromosomal abnormalities, syndromes, and noncardiac congenital anatomic abnormalities. Ann Thorac Surg. 2019;108:558–66. [DOI] [PubMed] [Google Scholar]
- 20.Pasquali SK, Shahian DM, O’Brien SM, Jacobs ML, Gaynor JW, Romano JC, et al. Development of a congenital heart surgery composite quality metric: part 1-conceptual framework. Ann Thorac Surg. 2019;107:583–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Chen T, He T, Benesty M, Khotilovich V, Tang Y, Cho H, et al. Xgboost: extreme gradient boosting. R package version 0.4-2. OS Independent; 2015. [Google Scholar]
- 22.Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In: Proceedings of the advances in neural information processing systems. 2017;4765–4774. [Google Scholar]
- 23.Jacobs JP, Mavroudis C, Quintessenza JA, Chai PJ, Pasquali SK, Hill KD, et al. Reoperations for pediatric and congenital heart disease: an analysis of the Society of Thoracic Surgeons (STS) congenital heart surgery database. Semin Thorac Cardiovasc Surg Pediatr Card Surg Ann. 2014;17:2–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Wang F, Harel-Sterling L, Cohen S, Liu A, Brophy JM, Paradis G, et al. Heart failure risk predictions in adult patients with congenital heart disease: a systematic review. Heart. 2019;105:1661–69. [DOI] [PubMed] [Google Scholar]
- 25.Otto CM, Nishimura RA, Bonow RO, Carabello BA, Erwin JP 3rd, Gentile F, et al. 2020 ACC/AHA guideline for the management of patients with valvular heart disease: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. Circulation. 2021;143:e72–e227. [DOI] [PubMed] [Google Scholar]
- 26.Tseng SY, Siddiqui S, Di Maria MV, Hill GD, Lubert AM, Kutty S, et al. Atrioventricular valve regurgitation in single ventricle heart disease: a common problem associated with progressive deterioration and mortality. J Am Heart Assoc. 2020;9:e015737. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.King G, Ayer J, Celermajer D, Zentner D, Justo R, Disney P, et al. Atrioventricular valve failure in Fontan palliation. J Am Coll Cardiol. 2019;73:810–22. [DOI] [PubMed] [Google Scholar]
- 28.Fuller S. Comparing long-term sequelae of the systemic right ventricle: an overview of single versus biventricular arrangements. Semin Thorac Cardiovasc Surg Pediatr Card Surg Ann. 2022;25:2–10. [DOI] [PubMed] [Google Scholar]
- 29.Nasr VG, Faraoni D, Valente AM, DiNardo JA. Outcomes and costs of cardiac surgery in adults with congenital heart disease. Pediatr Cardiol. 2017;38:1359–64. [DOI] [PubMed] [Google Scholar]
- 30.Kim YY, He W, MacGillivray TE, Benavidez OJ. Readmissions after adult congenital heart surgery: frequency and risk factors. Congenit Heart Dis. 2017;12:159–65. [DOI] [PubMed] [Google Scholar]
- 31.Broberg CS, Burchill LJ. Myocardial factor revisited: the importance of myocardial fibrosis in adults with congenital heart disease. Int J Cardiol. 2015;289:204–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Burchill LJ, Mertens L, Broberg CS. Imaging for the assessment of heart failure in congenital heart disease: ventricular function and beyond. Heart Failure Clin. 2014;10:9–22. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.