Abstract
Objectives
To compare the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) risk score with the established Society of Thoracic Surgeons (STS) and EuroSCORE II risk prediction models regarding mortality discrimination after aortic and mitral valve surgery.
Design
Retrospective cohort study.
Setting
Single tertiary academic medical center.
Participants
A total of 259 patients who underwent open aortic valve replacement or open mitral valve repair/replacement from 2009–2014.
Interventions
Retrospective chart review.
Measurements and Main Results
MAGGIC, STS, and EuroSCORE II risk scores for each patient were studied using binary logistic regression and receiver operating characteristic analysis for the primary endpoint of one-year mortality and secondary endpoint of 30-day mortality. One-year mortality C-statistics were similar across risk scores (STS 0.709, 95% confidence interval [CI] 0.578–0.841; MAGGIC 0.673, 95% CI 0.547–0.799; EuroSCORE II 0.642, 95% CI 0.521–0.762; p = 0.56 between STS and MAGGIC; p = 0.20 between STS and EuroSCORE II; and p = 0.69 between MAGGIC and EuroSCORE II). Thirty-day mortality C-statistics also were similar between STS (0.797, 95% CI 0.655–0.939; p < 0.0001 v null hypothesis), MAGGIC (0.721, 95% CI 0.581–0.860; p = 0.33 v STS), and EuroSCORE II (0.688, 95% CI 0.557–0.818; p = 0.06 v STS; p = 0.68 v MAGGIC).
Conclusions
The MAGGIC risk score performs similarly to STS and EuroSCORE II risk models in mortality discrimination after aortic and mitral valve surgery, albeit in a small sample size. This finding has important implications in establishing MAGGIC as a viable prognostic model in this population subset, with fewer variables and ease of use representing key advantages over STS and EuroSCORE II.
Introduction:
Valvular heart disease exerts a significant burden on cardiovascular disease care, with an estimated prevalence of 2.5% in the United States alone, and a substantial predilection toward older patients1,2,3. Despite advances in minimally invasive percutaneous techniques, valvular heart surgery continues to play an integral role in the management of this heterogeneous disease process3,4. At present, the Society of Thoracic Surgeons (STS) risk score is the most widely used risk score in the United States to estimate mortality risk following cardiac surgery, including valvular heart surgery5,6. The European System for Cardiac Operative Risk Evaluation (with current iteration of EuroSCORE II) represents another well-known prognostic model of cardiac surgery risk stratification, and has likewise been widely adopted across Europe, Asia, and North America5,7.
In recent years, other risk prediction models have been developed for a variety of cardiovascular conditions, including the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) risk score8. The MAGGIC risk score is a well-validated prognostic model in patients with heart failure, and has even been shown to predict clinical outcomes following transcatheter valvular interventions8,9,10,11. However, the applicability of the MAGGIC risk score to valvular heart surgery has not been specifically studied to date, even though many such patients manifest with heart failure syndromes requiring surgical intervention. With only 13 independent predictor variables- each of which can be readily obtained from patient demographics, history, laboratory and echocardiographic data- the MAGGIC risk score could constitute a viable alternative risk prediction model for valvular heart surgery, with key advantages of simplicity and ease of use relative to STS and EuroSCORE II8,9.
Based on these observations, we hypothesized that the MAGGIC risk score was non-inferior to both STS and EuroSCORE II with respect to the discrimination of mortality following aortic and mitral valve surgery. We therefore sought to compare the MAGGIC risk score to these risk prediction models in this regard.
Methods:
Study design and population
A retrospective chart review was conducted regarding 413 patients who underwent open aortic valve replacement (AVR), or mitral valve repair or replacement (MVR), at a single tertiary academic medical center from 2009 to 2014, with follow-up through December 2015. Instead of a longer study period, the shorter 5-year study period was chosen to provide a more contemporary cohort of patients, from which study conclusions would be better applied to current practice. Further follow-up beyond this study period was not able to be obtained, due to electronic medical system changes in the reporting and capture of mortality data from state and national databases.
Of this initial cohort, 259 patients had complete data to calculate MAGGIC, STS and EuroSCORE II risk scores, and so were included in the final analysis. All patients included in the final analysis underwent operation on either the aortic valve, or the mitral valve, but not both aortic and mitral valves simultaneously. The expected 1-year MAGGIC mortality risk was chosen to maintain consistency with the outcomes of observed 30-day and 1-year mortality used in this analysis.
Study approval was obtained from the Institutional Review Board for Health Sciences Research at our academic medical center. The electronic medical record was then queried for pertinent clinical data, including patient baseline demographics, lab results, morbidity, and mortality.
Baseline characteristics
Continuous variables were described using median and interquartile range (IQR), and differences in continuous variables between subgroups were evaluated via the Kruskal-Wallis test. This non-parametric option was chosen to maintain consistency throughout the analysis, as some variables were not normally distributed. Categorical variables were described using frequency and percentage, and differences in categorical variables among subgroups were evaluated by either chi-square or Fisher’s exact tests. A P-value of less than 0.05 was chosen to define statistical significance in detecting differences between baseline characteristics.
Statistical analysis
The primary endpoint was the area under the receiver operating characteristic (ROC) curve (C-statistic) for 1-year mortality after aortic or mitral valve surgery. Secondary endpoints were C-statistics for each risk score with respect to 30-day mortality. Differences between C-statistics of each risk score were evaluated via a non-parametric approach, using the theory on generalized U-statistics to generate estimated covariance matrices12. Optimal cutoff points for each ROC curve were calculated using the Youden index (J), assuming equal weight to sensitivity and specificity (where J = sensitivity + specificity − 1)13,14. Post-hoc sample sizes were derived using a two-tailed Pearson’s chi-square test for two independent proportions, assuming a power of 0.80 and alpha of 0.05.
Kaplan-Meier survival curves were generated for the overall cohort, and the log-rank test was used to assess for potential differences in survival between subgroups. For C-statistic analysis, a P-value of less than 0.008 was chosen to define statistical significance, as a Bonferroni correction was employed to minimize Type I error by accounting for multiple testing procedures (P-value of 0.05 divided by 6 total hypotheses, with the hypotheses denoted as STS, MAGGIC and EuroSCORE II values for both 30-day mortality and 1-year mortality)15,16. Interaction terms via multivariable logistic regression were used to assess whether specific valvular surgery type (AVR or MVR) affected C-statistic interpretation. Statistical analysis was performed using both SAS 9.4 (SAS; Cary, NC, U.S.A.) and R Version 3.6.3 software (The R Foundation for Statistical Computing; Vienna, Austria).
Post-hoc sample size estimation
Assuming a two-tailed test with power of 0.80 and alpha of 0.05, the Pearson’s chi-square test of two independent proportions yielded a minimum estimated sample size of 254 patients, in order to detect a significant difference between the MAGGIC 1-year mortality C-statistic and the null hypothesis. Meanwhile, the minimum estimated sample size to detect a significant difference between the MAGGIC and STS 1-year mortality C-statistics was 5172 patients. For context, the actual sample size of this study was 259 patients.
Results:
Baseline characteristics
In all, 259 patients were included in the final analysis, of which 203 patients underwent AVR and 56 underwent MVR (Tables 1 and 2). The median age was 73.0 years (interquartile range or IQR 63.0–79.0 years), and 114 patients (44.0%) were female (Table 1). There was no significant difference in survival between AVR and MVR subgroups by log-rank test (P=0.45; Figure 1).
Table 1.
Baseline characteristics.
| Demographic Variable | Overall Cohort (N=259) |
AVR (n=203) |
MVR (n=56) |
P-value (AVR vs MVR) |
|---|---|---|---|---|
| Age (years) | 73.0 (63.0–79.0) | 75.0 (67.0–80.0) | 63.0 (54.5–71.5) | <0.0001 |
| Length of stay during index admission (days) | 8.0 (5.0–12.0) | 7.0 (5.0–12.0) | 9.0 (6.0–16.5) | 0.03 |
| Time to discharge (days) | 6.0 (5.0–9.0) | 6.0 (5.0–8.0) | 7.0 (5.5–10.5) | 0.05 |
| Gender | ||||
| • Male | 145 (56.0) | 122 (60.1) | 23 (41.1) | 0.01 |
| • Female | 114 (44.0) | 81 (39.9) | 33 (58.9) | 0.01 |
| Race/ethnicity | ||||
| • White | 236 (91.1) | 187 (92.1) | 49 (87.5) | 0.28 |
| • African-American | 19 (7.3) | 14 (6.9) | 5 (8.9) | 0.61 |
| • Hispanic | 1 (0.4) | 0 (0.0) | 1 (1.8) | 0.06 |
| • Other | 3 (1.2) | 2 (1.0) | 1 (1.8) | 0.62 |
| Coronary artery disease | 170 (65.6) | 144 (70.9) | 26 (46.4) | 0.0006 |
| Heart failure | 144 (55.6) | 105 (51.7) | 39 (69.6) | 0.02 |
| Atrial fibrillation | 84 (32.4) | 64 (31.5) | 20 (35.7) | 0.55 |
| Current cigarette smoking use | 29 (11.2) | 16 (7.9) | 13 (23.2) | 0.001 |
| Gastrointestinal bleeding history | 2 (0.8) | 1 (0.5) | 1 (1.8) | 0.33 |
| Beta-blocker use | 152 (58.7) | 117 (57.6) | 35 (62.5) | 0.52 |
| ACE inhibitor/ARB use | 119 (46.0) | 98 (48.3) | 21 (37.5) | 0.15 |
| Chronic kidney disease | ||||
| • CrCl >85 mL/min | 83 (32.1) | 67 (33.0) | 16 (28.6) | 0.53 |
| • CrCl 51–84 mL/min | 121 (46.7) | 89 (43.8) | 32 (57.1) | 0.08 |
| • CrCl <50 mL/min | 49 (18.9) | 43 (21.2) | 6 (10.7) | 0.08 |
| • Dialysis requirement | 6 (2.3) | 4 (2.0) | 2 (3.6) | 0.48 |
| Peripheral arterial disease | 54 (20.9) | 50 (24.6) | 4 (7.1) | 0.004 |
| Impaired mobility | 40 (15.4) | 32 (15.8) | 8 (14.3) | 0.79 |
| Lung disease (including COPD) | 91 (35.1) | 65 (32.0) | 26 (46.4) | 0.05 |
| Infective endocarditis | 8 (3.1) | 6 (3.0) | 2 (3.6) | 0.81 |
| Critical preoperative state | 14 (5.4) | 10 (4.9) | 4 (7.1) | 0.52 |
| Diabetes mellitus | 160 (61.8) | 132 (65.0) | 28 (50.0) | 0.04 |
| NYHA class | ||||
| • Class I | 17 (6.6) | 16 (7.9) | 1 (1.8) | 0.10 |
| • Class II | 122 (47.1) | 101 (49.8) | 21 (37.5) | 0.10 |
| • Class III | 98 (37.8) | 71 (35.0) | 27 (48.2) | 0.07 |
| • Class IV | 22 (8.5) | 15 (7.4) | 7 (12.5) | 0.22 |
| CCS Angina Grade IV | 11 (4.3) | 10 (4.9) | 1 (1.8) | 0.30 |
| Unstable angina | 10 (3.9) | 10 (4.9) | 0 (0.0) | 0.09 |
| Recent myocardial infarction (within 90 days of surgery) | 27 (10.4) | 23 (11.3) | 4 (7.1) | 0.36 |
| Pulmonary hypertension | ||||
| • <31 mm Hg | 40 (15.4) | 36 (17.7) | 4 (7.1) | 0.05 |
| • 31–55 mm Hg | 148 (57.1) | 122 (60.1) | 26 (46.4) | 0.07 |
| • >55 mm Hg | 71 (27.4) | 45 (22.2) | 26 (46.4) | 0.0003 |
| • >60 mm Hg | 54 (20.9) | 34 (16.8) | 20 (35.7) | 0.002 |
| BMI (kg/m2) | 29.9 (25.5–34.6) | 30.1 (25.5–35.2) | 28.9 (24.9–32.7) | 0.12 |
| SBP (mm Hg) | 129.0 (114.0–145.0) | 134.0 (117.0–148.0) | 113.5 (104.0–132.5) | <0.0001 |
| DBP (mm Hg) | 68.0 (60.0–78.0) | 69.0 (59.0–79.0) | 67.0 (60.0–74.5) | 0.60 |
| CrCl (mL/min) | 73.0 (53.0–93.0) | 72.0 (52.0–94.0) | 77.5 (59.0–88.5) | 0.44 |
| Creatinine (mg/dL) | 1.0 (0.8–1.3) | 1.0 (0.8–1.3) | 1.0 (0.8–1.2) | 0.91 |
| LVEF (%) | 55.0 (40.0–60.0) | 55.0 (40.0–60.0) | 55.0 (40.0–60.0) | 0.73 |
| PCWP (mm Hg) | 17.0 (12.0–24.0) | 15.0 (11.0–22.0) | 22.0 (16.0–26.0) | <0.0001 |
| Mean PAP (mm Hg) | 28.0 (21.0–37.0) | 26.0 (20.0–35.0) | 37.0 (30.0–43.0) | <0.0001 |
| PASP (mm Hg) | 43.0 (34.0–58.0) | 41.0 (33.0–54.0) | 53.5 (45.5–65.5) | <0.0001 |
| PADP (mm Hg) | 16.0 (10.0–22.0) | 15.0 (10.0–21.0) | 22.0 (17.0–28.0) | <0.0001 |
| RVSP (mm Hg) | 45.0 (36.0–58.0) | 43.0 (35.0–56.0) | 54.0 (40.0–67.0) | 0.0001 |
| RVDP (mm Hg) | 6.0 (2.0–11.0) | 5.0 (1.0–10.0) | 10.0 (5.0–14.0) | 0.0001 |
| CVP (mm Hg) | 7.0 (5.0–11.0) | 7.0 (5.0–10.0) | 8.0 (6.0–12.0) | 0.02 |
| MAGGIC integer score | 23.0 (19.0–27.0) | 24.0 (19.0–27.0) | 21.5 (17.0–25.0) | 0.002 |
| MAGGIC risk score decile | ||||
| • 1st to 2nd decile | 41 (15.8) | 28 (13.8) | 13 (23.2) | 0.09 |
| • 3rd to 4th decile | 47 (18.2) | 34 (16.8) | 13 (23.2) | 0.27 |
| • 5th to 6th decile | 59 (22.8) | 45 (22.2) | 14 (25.0) | 0.65 |
| • 7th to 8th decile | 65 (25.1) | 55 (27.1) | 10 (17.9) | 0.16 |
| • 9th decile | 29 (11.2) | 25 (12.3) | 4 (7.1) | 0.28 |
| • 10th decile | 18 (7.0) | 16 (7.9) | 2 (3.6) | 0.26 |
| Estimated % mortality | ||||
| • MAGGIC (1 year) | 13.4 (9.3–19.1) | 14.7 (9.3–19.1) | 11.7 (7.7–16.0) | 0.002 |
| • STS | 3.8 (2.2–6.1) | 3.9 (2.2–6.0) | 3.6 (2.0–7.7) | 0.78 |
| • EuroSCORE II | 4.5 (2.2–9.9) | 4.7 (2.2–10.2) | 3.6 (2.4–6.7) | 0.43 |
| 30-day mortality | 12 (4.6) | 8 (3.9) | 4 (7.1) | 0.31 |
| 1-year mortality | 20 (7.7) | 14 (6.9) | 6 (10.7) | 0.34 |
Categorical variables are displayed as number (percent), while continuous variables are displayed as median (interquartile range, or IQR). Abbreviations: ACE, angiotensin-converting enzyme; ARB, angiotensin receptor blocker; BMI, body mass index; CCS, Canadian Cardiovascular Society; COPD, chronic obstructive pulmonary disease; CrCl, creatinine clearance; CVP, central venous pressure; DBP, diastolic blood pressure; EuroSCORE II, European System for Cardiac Operative Risk Evaluation II; LVEF, left ventricular ejection fraction; MAGGIC, Meta-Analysis Global Group in Chronic Heart Failure; NYHA, New York Heart Association; PADP, pulmonary artery diastolic pressure; PAP, pulmonary artery pressure; PASP, pulmonary artery systolic pressure; PCWP, pulmonary capillary wedge pressure; RVDP, right ventricular diastolic pressure; RVSP, right ventricular systolic pressure; SBP, systolic blood pressure; STS, Society of Thoracic Surgeons.
Table 2.
Valvular heart surgery by type and surgical indication.
| N=259 | n | Percent (%) |
|---|---|---|
| Primary surgery type | ||
| • Aortic valve replacement | 203 | 78.3 |
| • Mitral valve replacement | 56 | 21.6 |
| • Concurrent CABG | 69 | 26.6 |
| - With aortic valve replacement | 63 | 24.3 |
| - With mitral valve replacement | 6 | 2.3 |
| Primary surgical indication | ||
| • Aortic regurgitation | 9 | 3.5 |
| • Aortic stenosis | 186 | 71.8 |
| • Aortic stenosis and aortic regurgitation | 3 | 1.2 |
| • Mitral regurgitation | 36 | 13.9 |
| • Mitral stenosis | 14 | 5.4 |
| • Mitral regurgitation and mitral stenosis | 6 | 2.3 |
| • Infective endocarditis | 5 | 1.9 |
Abbreviations: CABG, coronary artery bypass surgery.
Figure 1.
Kaplan-Meier survival analysis, for the overall cohort (A) and by aortic valve replacement (AVR) and mitral valve repair/replacement (MVR) subgroups (B).
When compared to the MVR subgroup, the AVR subgroup was older, had higher systolic blood pressure, and was more likely to have coronary artery disease, peripheral arterial disease and diabetes mellitus (Table 1). The MVR subgroup, meanwhile, had a higher percentage of female patients, was more likely to have underlying heart failure and endorse current cigarette smoking use, and experienced a longer length of stay during the index hospital admission. Of note, the MVR subgroup displayed a significant increase in filling pressures as defined by pre-operative right heart catheterization (Table 1).
In this study, 69 out of 259 patients underwent concurrent coronary artery bypass surgery (CABG) with their valve surgery, of which 63 were in the AVR subgroup, and 6 were in the MVR subgroup (Table 2). Patients with concurrent CABG in the overall cohort were older, had longer time to discharge, and were more likely to have pre-existing CAD, diabetes mellitus, myocardial infarction within 90 days of index surgery, higher STS risk score, and higher EuroSCORE II risk score, in addition to having a higher proportion of pulmonary artery systolic pressure in the 31–55 mm Hg range (Supplemental Table 1). Patients in the AVR subgroup with concurrent CABG had a longer length of stay and time to discharge, as well as a higher incidence of CAD, recent myocardial infarction, and higher STS and EuroSCORE II risk scores compared to those with AVR only (Supplemental Table 2).
The observed 30-day mortality was 4.6%, and the corresponding observed 1-year mortality was 7.7% (Table 1). There was no significant difference in observed mortality between AVR and MVR subgroups. By comparison, the expected 1-year mortality was 3.8% for STS, 4.5% for EuroSCORE II, and 13.4% for MAGGIC (Table 1). STS and EuroSCORE underestimated 1-year mortality by 51% and 41%, respectively, while MAGGIC overestimated 1-year-mortality by 74% (Table 1).
Discrimination of mortality by risk score
With respect to 30-day mortality in the overall cohort, C-statistics for STS, MAGGIC and EuroSCORE II were 0.797 (95% CI 0.655–0.939, P<0.0001 vs null hypothesis), 0.721 (95% CI 0.581–0.860, P=0.002) and 0.688 (95% CI 0.557–0.818, P=0.005), respectively (Table 3, Figure 2). The corresponding C-statistics for 1-year mortality were 0.709 for STS (95% CI 0.578–0.841, P=0.002 vs null hypothesis), 0.673 for MAGGIC (95% CI 0.547–0.799, P=0.007), and 0.642 for EuroSCORE II (95% CI 0.521–0.762, P=0.02; Table 3, Figure 2).
Table 3.
C-statistics.
| Risk Score | C-statistic | Standard Error | 95% CI | P-value (vs null hypothesis) | P-value (vs STS) | P-value (vs MAGGIC) | |
|---|---|---|---|---|---|---|---|
| 30-day mortality | |||||||
| Overall Cohort | STS | 0.797 | 0.073 | 0.655–0.939 | <0.0001* | - | 0.33 |
| MAGGIC | 0.721 | 0.071 | 0.581–0.860 | 0.002* | 0.33 | - | |
| EuroSCORE II | 0.688 | 0.066 | 0.557–0.818 | 0.005* | 0.06 | 0.68 | |
| 1-year mortality | |||||||
| Overall Cohort | STS | 0.709 | 0.067 | 0.578–0.841 | 0.002* | - | 0.56 |
| MAGGIC | 0.673 | 0.064 | 0.547–0.799 | 0.007* | 0.56 | - | |
| EuroSCORE II | 0.642 | 0.062 | 0.521–0.762 | 0.02 | 0.20 | 0.69 |
Abbreviations:
, denotes significant P-value of <0.008 by Bonferroni correction; 95% CI, 95% confidence interval; AVR, open aortic valve replacement; EuroSCORE II, European System for Cardiac Operative Risk Evaluation II; MAGGIC, Meta-Analysis Global Group in Chronic Heart Failure; MVR, open mitral valve repair/replacement; STS, Society of Thoracic Surgeons.
Figure 2.
Area under the Receiver Operating Characteristic (ROC) curves for 30-day and 1-year mortality. Abbreviations: EuroSCORE, European System for Cardiac Operative Risk Evaluation II; MAGGIC, Meta-Analysis Global Group in Chronic Heart Failure; STS, Society of Thoracic Surgeons.
Only STS and MAGGIC were significantly different from the null hypothesis for both 30-day and 1-year mortality in the overall cohort, when the Bonferroni correction P-value of 0.008 was applied; EuroSCORE II was only statistically different from the null hypothesis for 30-day mortality in this scenario (Table 3). There was no significant difference in either 30-day or 1-year mortality C-statistics between the risk prediction models (Table 3, Figure 2). Specific valve surgery type (AVR or MVR) did not significantly affect C-statistic models, when assessed by interaction terms from multivariable logistic regression analysis (Supplemental Table 3).
The optimal expected 30-day mortality cutoff points were 7.56% mortality for STS (sensitivity 66.7%, specificity 84.6%, J 0.512), 17.51% for MAGGIC (sensitivity 75.0%, specificity 65.2%, J 0.402) and 4.79% for EuroSCORE II (sensitivity 83.3%, specificity 53.4%, J 0.368; Table 4). The corresponding expected 1-year mortality cutoff points were 14.84% for STS (sensitivity 40.0%, specificity 96.2%, J 0.362), 17.50% for MAGGIC (sensitivity 70.0%, specificity 66.1%, J 0.361) and 4.29% for EuroSCORE II (sensitivity 75.0%, specificity 49.4%, J 0.244; Table 4).
Table 4.
Optimal cutoff points.
| Risk Score | Cutoff Point (% mortality) | Probability | Sensitivity (%) | Specificity (%) | Youden Index (J) | |
|---|---|---|---|---|---|---|
| 30-day mortality | ||||||
| Overall Cohort | STS | 7.56 | 0.0416 | 66.7 | 84.6 | 0.512 |
| MAGGIC | 17.51 | 0.0470 | 75.0 | 65.2 | 0.402 | |
| EuroSCORE II | 4.79 | 0.0405 | 83.3 | 53.4 | 0.368 | |
| 1-year mortality | ||||||
| Overall Cohort | STS | 14.84 | 0.1429 | 40.0 | 96.2 | 0.362 |
| MAGGIC | 17.50 | 0.0802 | 70.0 | 66.1 | 0.361 | |
| EuroSCORE II | 4.29 | 0.0672 | 75.0 | 49.4 | 0.244 |
Abbreviations: AVR, open aortic valve replacement; EuroSCORE II, European System for Cardiac Operative Risk Evaluation II; MAGGIC, Meta-Analysis Global Group in Chronic Heart Failure; MVR, open mitral valve repair/replacement; STS, Society of Thoracic Surgeons.
Discussion:
In this single-center retrospective review of aortic and mitral valve surgery patients, the MAGGIC risk score performed similarly to the established STS and EuroSCORE II risk prediction models in the discrimination of both 30-day and 1-year mortality, as assessed by C-statistic analysis. MAGGIC and STS were also predictive of both mortality metrics in the overall cohort. All three risk scores had similar issues with either overestimation or underestimation, as MAGGIC tended to overestimate both 30-day and 1-year mortality, while STS and EuroSCORE II underestimated 1-year mortality. Still, the observation that MAGGIC provides good relative estimation, but may need to be calibrated down for future clinical use, is nonetheless a novel and important finding of this study.
Currently, STS and EuroSCORE II represent the most widely used models for estimating peri-operative morbidity and mortality following cardiac surgery, including valvular heart surgery5,17,18. However, both incorporate variables which may not be readily available to clinicians, such as coronary artery anatomy for STS, and presence and specific degree of pulmonary hypertension for EuroSCORE II (Supplemental Tables 4 and 5)19,20,21. Accordingly, these missing variables may adversely affect the ability of STS and EuroSCORE II to estimate peri-operative risk5,18.
In this regard, MAGGIC may provide a viable alternative to these established risk prediction models in valvular heart surgery. With only 13 baseline demographic variables, MAGGIC provides a relatively straightforward and user-friendly tool for clinicians, qualities which can expand its utility beyond the original heart failure population from which it was derived (Supplemental Table 6)8,9,10,22,23. Our study contributes further to the existing literature by demonstrating the novel utility of MAGGIC in aortic and mitral valve surgery patients, many of whom manifest with heart failure as their presenting clinical syndrome.
There are a few study limitations worth mentioning. For one, selection bias was likely present at multiple levels, from the single-center retrospective lens, to the fact that several patients were excluded due to incomplete data to calculate each of the risk scores. Moreover, the distinctive demographics of the patient population at our academic medical center may not be as readily generalizable to other medical centers around the world. Finally, the small sample size of the overall cohort limited the power of this study to detect a significant difference between the STS and MAGGIC 1-year mortality C-statistics, though the absolute difference between these C-statistics was small, and the sample size here was nevertheless large enough to detect a difference between the 1-year MAGGIC C-statistic and the null hypothesis.
In conclusion, we found that the MAGGIC risk score performs similarly to the established STS and EuroSCORE II risk models in the discrimination of mortality following aortic and mitral valve surgery, albeit in a relatively small sample size. This finding has important implications in establishing the MAGGIC risk score as a viable prognostic model in this population subset, especially given the potential advantages of fewer variables over STS and EuroSCORE II.
Supplementary Material
References:
- 1.Iung B, Vahanian A. Epidemiology of valvular heart disease in the adult. Nat Rev Cardiol 2011;8:162–172. [DOI] [PubMed] [Google Scholar]
- 2.Nkomo VT, Gardin JM, Skelton TN, et al. Burden of valvular heart diseases: a population-based study. Lancet 2006;368:1005–1011. [DOI] [PubMed] [Google Scholar]
- 3.Kodali SK, Velagapudi P, Hahn RT, et al. Valvular Heart Disease in Patients ≥80 Years of Age. J Am Coll Cardiol 2018;71:2058–2072. [DOI] [PubMed] [Google Scholar]
- 4.Nishimura RA, Otto CM, Bonow RO, et al. 2017 AHA/ACC Focused Update of the 2014 AHA/ACC Guideline for the Management of Patients With Valvular Heart Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol 2017;70:252–289. [DOI] [PubMed] [Google Scholar]
- 5.Ad N, Holmes SD, Patel J, et al. Comparison of EuroSCORE II, Original EuroSCORE, and The Society of Thoracic Surgeons Risk Score in Cardiac Surgery Patients. Ann Thorac Surg 2016;102:573–9. [DOI] [PubMed] [Google Scholar]
- 6.Puskas JD, Kilgo PD, Thourani VH, et al. The Society of Thoracic Surgeons 30-Day Predicted Risk of Mortality Score Also Predicts Long-Term Survival. Ann Thorac Surg 2012;93:26–35. [DOI] [PubMed] [Google Scholar]
- 7.Duchnowski P, Hryniewiecki T, Kusmierczyk M, et al. Performance of the EuroSCORE II and the Society of Thoracic Surgeons score in patients undergoing aortic valve replacement for aortic stenosis. J Thorac Dis 2019;11:2076–2081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Pocock SJ, Ariti CA, McMurray JJ, et al. Predicting survival in heart failure: a risk score based on 39372 patients from 30 studies. Eur Heart J 2013;34:1404–13. [DOI] [PubMed] [Google Scholar]
- 9.Sartipy U, Dahlström U, Edner M, et al. Predicting survival in heart failure: validation of the MAGGIC heart failure risk score in 51,043 patients from the Swedish heart failure registry. Eur J Heart Fail 2014;16:173–9. [DOI] [PubMed] [Google Scholar]
- 10.Rich JD, Burns J, Freed BH, et al. Meta-Analysis Global Group in Chronic (MAGGIC) Heart Failure Risk Score: Validation of a Simple Tool for the Prediction of Morbidity and Mortality in Heart Failure With Preserved Ejection Fraction. J Am Heart Assoc 2018;7:e009594. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hioki H, Watanabe Y, Kozuma K, et al. The MAGGIC risk score predicts mortality in patients undergoing transcatheter aortic valve replacement: sub-analysis of the OCEAN-TAVI registry. Heart Vessels 2019;24:1976–1983. [DOI] [PubMed] [Google Scholar]
- 12.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–845. [PubMed] [Google Scholar]
- 13.Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32–35. [DOI] [PubMed] [Google Scholar]
- 14.Ruopp MD, Perkins NJ, Whitcomb BW, et al. Youden Index and Optimal Cut-Point Estimated from Observations Affected by a Lower Limit of Detection. Biom J 2008;50:419–430. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Dunn OJ. Multiple Comparisons among Means. J Am Stat Assoc 1961;56:52–64. [Google Scholar]
- 16.Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. BMJ 1995;310:170. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bouabdallaoui N, Stevens SR, Doenst T, et al. Society of Thoracic Surgeons Risk Score and EuroSCORE-2 Appropriately Assess 30-Day Postoperative Mortality in the STICH Trial and a Contemporary Cohort of Patients With Left Ventricular Dysfunction Undergoing Surgical Revascularization. Cir Heart Fail 2018;11:e005531. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Sullivan PG, Wallach JD, Ioannidis JP. Meta-Analysis Comparing Established Risk Prediction Models (EuroSCORE II, STS Score, and ACEF Score) for Perioperative Mortality During Cardiac Surgery. Am J Cardiol 2016;118:1574–1582. [DOI] [PubMed] [Google Scholar]
- 19.O’Brien SM, Shahian DM, Filardo G, et al. The Society of Thoracic Surgeons 2008 Cardiac Surgery Risk Models: Part 2-Isolated Valve Surgery. Ann Thorac Surg 2009;88:S23–S42. [DOI] [PubMed] [Google Scholar]
- 20.O’Brien SM, Feng L, He X, et al. The Society of Thoracic Surgeons 2018 Adult Cardiac Surgery Risk Models: Part 2-Statistical Methods and Results. Ann Thorac Surg 2018;105:1419–1428. [DOI] [PubMed] [Google Scholar]
- 21.Nashef SA, Roques F, Sharples LD, et al. EuroSCORE II. Eur J Cardiothorac Surg 2012;41:734–744. [DOI] [PubMed] [Google Scholar]
- 22.Khanam SS, Choi E, Son JW, et al. Validation of the MAGGIC (Meta-Analysis Global Group in Chronic Heart Failure) heart failure risk score and the effect of adding natriuretic peptide for predicting mortality after discharge in hospitalized patients with heart failure. PLoS One 2018;13:e0206380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Sawano M, Shiraishi Y, Kohsaka S, et al. Performance of the MAGGIC heart failure risk score and its modification with the addition of discharge natriuretic peptides. ESC Heart Fail 2018;5:610–619. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.