Abstract
Objective
To develop an inflammation-based risk stratification tool for operative mortality in patients with acute type A aortic dissection.
Methods
Between January 1, 2016 and December 31, 2021, 3124 patients from Beijing Anzhen Hospital were included for derivation, 571 patients from the same hospital were included for internal validation, and 1319 patients from other 12 hospitals were included for external validation. The primary outcome was operative mortality according to the Society of Thoracic Surgeons criteria. Least absolute shrinkage and selection operator regression were used to identify clinical risk factors. A model was developed using different machine learning algorithms. The performance of the model was determined using the area under the receiver operating characteristic curve (AUC) for discrimination, calibration curves, and Brier score for calibration. The final model (5A score) was tested with respect to the existing clinical scores.
Results
Extreme gradient boosting was selected for model training (5A score) using 12 variables for prediction—the ratio of platelet to leukocyte count, creatinine level, age, hemoglobin level, prior cardiac surgery, extent of dissection extension, cerebral perfusion, aortic regurgitation, sex, pericardial effusion, shock, and coronary perfusion—which yields the highest AUC (0.873 [95% confidence interval (CI) 0.845-0.901]). The AUC of 5A score was 0.875 (95% CI 0.814-0.936), 0.845 (95% CI 0.811-0.878), and 0.852 (95% CI 0.821-0.883) in the internal, external, and total cohort, respectively, which outperformed the best existing risk score (German Registry for Acute Type A Aortic Dissection score AUC 0.709 [95% CI 0.669-0.749]).
Conclusion
The 5A score is a novel, internally and externally validated inflammation-based tool for risk stratification of patients before surgical repair, potentially advancing individualized treatment.
Trial Registration
clinicaltrials.gov Identifier: NCT04918108
Abbreviations and Acronyms: 5A, Additive Anti-inflammatory Action for Aortopathy & Arteriopathy; ATAAD, acute type A aortic dissection; AUC, area under the receiver operating characteristics curve; AVR, aortic valve regurgitation; CT, computed tomography; GERAADA, German Registry for Acute Type A Aortic Dissection; ICU, intensive care unit; KNN, K-nearest neighbor; LASSO, least absolute shrinkage and selection operator; NB, naïve Bayes; RF, random forest; STI, systemic thrombo-inflammatory; SVM, support vector machine; WBC, white blood cell
Acute type A aortic dissection (ATAAD) is a major cardiovascular catastrophe; however, identification of patients with ATAAD at mortality risk remains a great challenge.1,2 Clinically used risk algorithms, such as additive and logistic EuroSCORE,3,4 Parsonnet score,5 Cleveland score,6 Ontario Province Risk score,7 SinoSCORE,8 and German Registry for Acute Type A Aortic Dissection (GERAADA) score,9 are based on traditional risk factors for mortality and predict future events with limited accuracy.
Despite adding individual laboratory biomarkers, such as brain natriuretic peptide, troponins, and C-reactive protein, to clinical risk scores, the overall improvement has been limited because of the lack of generalizability and impact analysis, omitting routinely assessed and powerful predictors.10,11 This may be explained by the fact that the vast majority of single markers are selected on the basis of specific pathophysiologic concepts, which do not reflect the true complexity of aortic dissections. In fact, mortality risk is the result of an interplay between organ malperfusion (coronary, renal, intestinal, and cerebral) and concomitant systemic responses propagated by a variety of pathophysiologic axes, comprising but not limited to coagulation, inflammation, and immune.12 The recognition of this pathophysiology has prompted efforts in the risk stratification of patients with ATAAD based on inflammatory assessments. Simultaneous additional assessment of inflammatory and thrombotic biomarkers may hold a promise to further refine risk assessment. We propose a novel systemic thrombo-inflammatory parameter, namely systemic thrombo-inflammatory (STI) index, calculated by the ratio of platelet count to leukocyte count, to predict the operative mortality, which might outperform the traditional hematologic signatures based on our clinical practice and current literature.13,14
The recent improvements in computation power and software technologies have led to the flourishing of machine learning, which seems to be a promising tool to meet this compelling demand.15 Machine learning refers to a collection of techniques that gives artificial intelligence the ability to learn complex rules and to identify patterns from multidimensional datasets without being explicitly programmed or applying any a priori assumptions. It has been effectively employed in many areas of cardiovascular conditions, such as precision phenotyping, diagnostics, and prognostication, including the prediction of hospital readmissions and mortality.16,17 Patients with ATAAD undergoing surgical repair represent another important target population for mortality prediction; however, there are only a few studies in which machine learning has been applied to tackle this issue.18
In the present study, we hypothesized that an additive inflammatory risk model can outperform prediction using traditional clinical risk factors only in the cardiovascular setting. The aim of this study was to develop a risk prediction model for the prediction of mortality in patients undergoing surgery for type A aortic dissection using advanced machine learning techniques to support clinical decision making. The findings were subsequently validated in the independent, external cohort.
Methods
We undertook a derivation and external validation study to develop an inflammation-based mortality prediction model for patients with ATAAD who underwent surgical repair. This study adhered to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis reporting guideline for diagnostic and prognostic studies.19
Study Population
The Additive Anti-inflammatory Action for Aortopathy & Arteriopathy (5A) study was a prospective, ongoing, multicenter, investigator-initiated registry in which eligible patients with aortic dissection were consecutively enrolled at 13 cardiovascular centers across the People’s Republic of China. We retrospectively included consecutive patients with ATAAD diagnosed by aortic computed tomography (CT) angiography if they were aged 18 years or older with complete blood count examined immediately at admission in the emergency department of participating hospitals from January 1, 2016 to December 31, 2021. Aortic dissection is classified according to the Stanford system: Type A involves the ascending aorta, regardless of the site of the primary intimal tear, and type B involves only the descending aorta. Acute aortic dissection is defined as less than or equal to 14 days from symptom onset to diagnosis. The exclusion criteria included type B aortic dissection, recurrent aortic dissection, and an onset time of more than 14 days (Figure 1). In addition, 357 patients who received anticoagulants or antiplatelet therapy in the most recent 3 months were excluded to help dodge their potential effects on the blood platelet count.
Figure 1.
Machine learning workflow of model construction and validation. AUC, the area under the receiver operating characteristic curve; Cr, creatinine; Hgb, hemoglobin; NLR, neutrophil-lymphocyte ratio; PLR, platelet-lymphocyte ratio; SII, systemic inflammatory-immune; STI, systemic thrombo-inflammatory; WBC, white blood cell.
∗The 12 Chinese university cardiovascular centers are listed in the Supplemental Materials.
The study protocol conformed to the ethical guidelines of the 1975 Declaration of Helsinki and was approved by the Institutional Review Board of Aortic Collaborative Institutions involved (2021-SR-381). This study was registered with clinicaltrials.gov (Identifier NCT04918108). Informed consent was waived for this retrospectively observational study.
End Point Definition
The primary outcome was operative mortality, defined as any death, regardless of cause, occurring within 30 days after surgery in or out of the hospital and after 30 days during the same hospitalization subsequent to the operation according to the Society of Thoracic Surgeons criteria.20 Secondary outcomes included the mechanical ventilation duration, intensive care unit (ICU) length of stay, and hospital length of stay.
Variable Selection
Baseline information of patients included age, sex, past medical history (hypertension, coronary heart disease, diabetes, stroke, and smoking and drinking history), physical examination, laboratory examination, imaging examination, and surgical procedures during the main period.
Briefly, aortic regurgitation was assessed using preoperative echocardiography and/or intraoperative inspection, as appropriate, and the most severe grade was determined as the final report in this study. Pericardial tamponade was mainly evaluated by clinical profiles, along with preoperative CT angiography, echocardiography, and/or intraoperative inspection, as appropriate, and the most severe grade was determined as the final report in this study. Pericardial effusion was determined using preoperative CT angiography, echocardiography, and/or intraoperative inspection, as appropriate, and the most severe grade was determined as the final report in this study. Shock was primarily determined on the basis of clinical vitals and symptoms (ie, a systolic blood pressure of <80 mmHg or needing large doses of vasopressors drugs to maintain circulatory stability). Malperfusion, including cerebral, coronary, and renal malperfusion, was defined as end-organ ischemia caused by branch vessel involvement and resulting in functional impairment,21 which ultimately was judged by a comprehensive assessment of clinical profiles; CT angiography; echocardiography; laboratory testing, including arterial blood gas analysis; and/or intraoperative inspection, as appropriate.
Conventional clinical variables to be considered for model development were chosen via the least absolute shrinkage and selection operator (LASSO) logistic regression from all the candidate variables a priori as those which appear in any of the existing clinical scores for mortality prediction, namely the additive and logistic EuroSCORE,3,4 Parsonnet score,5 Cleveland score,6 Ontario Province Risk score,7 SinoSCORE,8 and GERAADA score.9 The LASSO can minimize the potential collinearity of variables measured from the same patient and over-fitting of variables.22 To identify the optimal tuning parameter lambda in LASSO regression, we performed 5-fold cross-validation with 1 standard error rule of the minimum criteria. Using the suitable lambda value, variables with nonzero coefficients in the model were selected. Subsequently, we constructed the prediction scoring model by assigning each patient a risk score for mortality based on the product of the expression levels for the variables selected by the LASSO analysis and the respective regression coefficients weighted by logistic regression analysis in the derivation cohort. We also fitted the dose-response relationship between the risk score and mortality using generalized additive models.23
For selection of optimal inflammatory signature in predicting mortality, we compared the predictive performances of individual and collaborative hematologic signatures. Among laboratory inflammatory variables (the count of white blood cell [WBC], platelet, neutrophile, and lymphocyte; platelet-lymphocyte ratio; systemic inflammatory-immune index; neutrophil-lymphocyte ratio; and STI index calculated by the ratio of platelet count to WBC count), the best inflammatory classifier was identified as the candidate whose mortality probability estimates achieved the highest area under the receiver operating characteristic curve (AUC) analysis within the derivation cohort.
Algorithm Development
Based on the clinical variables selected by LASSO regression, 7 machine learning classifiers were used to develop the risk prediction model of mortality, consisting of extreme gradient boosting (XGBoost), adaptive boosting (AdaBoost), naïve Bayes (NB), linear regression, random forest (RF), K-nearest neighbor (KNN), and support vector machine (SVM).24 The best classifier was identified as the algorithm whose mortality probability estimates achieved the highest AUC on receiver operating characteristic curve analysis within the derivation cohort during cross-validation.
Two machine learning models were developed via the XGBoost technique: the first was based on clinical variables alone, and the second uses clinical variables plus STI. The STI index was included in the models as a continuous variable. To better understand how the gradient-boosted model worked, we also visualized feature importance in terms of the total decrease in node impurity due to branching over a given predictor, averaged over all trees and aggregated across all classifiers in the ensemble.
Model Evaluations
To assess algorithm discrimination, we calculated the AUC as primary performance metrics.25 We calculated 95% CI of the AUC and compared the AUCs of the models following the method of DeLong et al.26 For comparison of the 2 models, we also calculated the net reclassification improvement index and integrated the discrimination improvement index, which measures the improvement in predictive performance gained by adding the STI index to the base risk model.27 Algorithm calibration was assessed by plotting the predicted vs observed rate of operative mortality, using the goodness-of-fit test,28 and by calculating the scaled Brier score, which was defined as the mean squared difference between the observed and the predicted outcomes.29 The net benefit of the models was assessed using decision curve analysis, in which the existing clinical risk score models and the final inflammation-based model were converted to a logistic regression using probability theory.30,31 To assess other performance metrics of operational importance, we calculated the positive predictive value, the negative predictive value, sensitivity, and specificity, which were subjected by bootstrapping 100 times. We used the fourth quartile of the prediction score from the final inflammation-based model as the threshold to classify patients into high-risk and low-risk groups.
Subsequently, the predictive performance of the final risk model was compared with that of these existing clinical scores with respect to discrimination and calibration performances and decision curves.
Model Validations
The new mortality model was internally validated using data on 571 patients from the same hospitals as the derivation cohort (2016-2021). An independently and externally consolidated dataset of 1319 patients from 12 university hospitals was used to assess external generalizability (Figure 1). To examine the internal and external validity of model performances, we evaluated the discrimination and calibration performances and decision curves.
Statistical Analyses
Continuous data were presented as the mean (SD) or median (interquartile range) and compared using a t-test or Kruskal-Wallis test depending on the nature of variable, and categorical data were reported as percentages and compared using χ2 testing or Fisher’s exact testing. Because of covariates that were potentially missing, not completely at random, covariates were imputed for the multivariable analysis by means of a single imputation with 10 iterations with all the covariates using the “MICE” package for R. All statistical analyses were performed using R, version 3.6.1 (R Foundation for Statistical Computing). The R packages used were as follows: “glmnet” for LASSO logistic regression, “xgboost” for XGBoost, “adabag” for AdaBoost, “naivebayes” for NB, “mlr” for linear regression, “class” for KNN, “randomForest” for RF, and “e1071” for SVM.
Results
Patient Characteristics and Clinical Outcome Among Cohorts
A total of 5014 patients with ATAAD were finally included for model development and validation: 3124 patients for training, 571 patients for internal validation, and 1319 patients for external validation (Figure 1). The operative mortality was 7.0% for the total cohort, 5.8% for the derivation cohort, 6.5% for the internal cohort, and 10.1% for the external cohort. Baseline demographic, clinical, and procedural characteristics and clinical outcomes among cohorts are summarized in the Table. The secondary outcomes, including mechanical ventilation duration, ICU length of stay, and hospital length of stay, are summarized in the Table.
Table.
Baseline and Clinical Characteristics and Perioperative Outcomes of 3 Cohorts
| Variables | Derivation cohort (N=3124) | Internal cohort (N=571) | External cohort (N=1319) |
|---|---|---|---|
| Demographics | |||
| Age (y), median (IQR) | 50 (41-59) | 49 (39-57) | 49 (40-58) |
| Sex (male), n (%) | 2347 (75.1) | 423 (74.1) | 936 (71.0) |
| Body mass index (kg/m2), median (IQR) | 25.4 (23.0-27.8) | 25.0 (22.7-27.4) | 25.2 (22.5-27.8) |
| Medical history, n (%) | |||
| Hypertension | 2602 (80.9) | 458 (80.2) | 1054 (79.9) |
| Chronic lung diseases | 77 (2.5) | 12 (2.1) | 33 (2.5) |
| Diabetes mellitus | 165 (5.3) | 35 (6.1) | 86 (6.5) |
| Arrhythmia | 85 (2.7) | 8 (1.4) | 98 (7.4) |
| Stroke | 141 (4.5) | 28 (4.9) | 67 (5.1) |
| Coronary heart disease | 298 (9.6) | 29 (5.1) | 106 (8.0) |
| Previous cardiac surgery | 417 (13.4) | 87 (15.2) | 210 (15.9) |
| Specific conditions on admission | |||
| Extent of dissection extension, n (%) | |||
| Limited in the ascending aorta | 1054 (33.7) | 172 (30.1) | 423 (32.1) |
| Extended to the aortic arch | 406 (13.0) | 88 (15.4) | 180 (13.6) |
| Extended to the descending aorta | 1664 (53.3) | 311 (54.5) | 716 (54.3) |
| Aortic regurgitation, n (%) | |||
| Mild | 964 (32.8) | 172 (34.9) | 400 (33.6) |
| Moderate | 399 (13.6) | 82 (16.6) | 170 (14.3) |
| Severe | 614 (20.9) | 101 (20.5) | 227 (19.1) |
| Pericardial tamponade, n (%) | 255 (8.7) | 39 (7.8) | 95 (7.9) |
| Pericardial effusion, n (%) | |||
| Mild | 278 (9.0) | 75 (13.3) | 149 (11.4) |
| Moderate | 59 (1.9) | 9 (1.6) | 18 (1.4) |
| Severe | 30 (1.0) | 4 (0.7) | 15 (1.1) |
| Shock, n (%) | 26 (0.8) | 4 (0.7) | 13 (1.0) |
| Cerebral malperfusion, n (%) | 321 (10.3) | 58 (10.2) | 131 (9.9) |
| Coronary malperfusion, n (%) | 533 (17.1) | 158 (27.7) | 270 (20.5) |
| Renal malperfusion, n (%) | 168 (5.4) | 35 (6.1) | 66 (5.0) |
| Laboratory signatures, median (IQR)a | |||
| Hemoglobin level (g/L) | 138 (126-149) | 137 (125-148) | 128 (115-141) |
| Creatinine level (μmol/L) | 75 (63-90) | 75 (62-91) | 85 (62-110) |
| WBC count (109/L) | 11.4 (8.5-14.5) | 11.1 (7.9-15.3) | 11.4 (8.8-14.3) |
| Platelet count (109/L) | 161 (123-209) | 147 (91-216) | 166 (132-208) |
| Neutrophile count (109/L) | 9.6 (6.6-12.4) | 9.2 (6.1-12.9) | 9.7 (6.9-12.2) |
| Lymphocyte count (109/L) | 1.01 (0.66-1.47) | 1.00 (0.59-1.56) | 1.01 (0.70-1.42) |
| Platelet-lymphocyte ratio | 157 (108-2396) | 148 (94-233) | 163 (116-242) |
| Neutrophile-lymphocyte ratio | 10.1 (5.4-16.4) | 9.6 (5.2-18.0) | 10.2 (5.5-15.8) |
| SII indexb | 1483 (623-3449) | 1351 (519-3499) | 1596 (680-3413) |
| STI indexc | 23.3 (14.5-34.3) | 24.2 (14.5-35.5) | 24.1 (14.9-35.0) |
| Procedural variables | |||
| Root procedures, n (%) | |||
| AVR only | 107 (3.4) | 15 (2.6) | 72 (5.4) |
| Bentall | 1148 (36.7) | 222 (38.8) | 264 (20.1) |
| David | 47 (1.5) | 9 (1.6) | 10 (0.8) |
| Total arch replacement plus FET implantation, n (%) | 1506 (48.2) | 318 (55.7) | 1022 (77.5) |
| Hemi-arch replacement, n (%) | 371 (11.9) | 75 (13.1) | 106 (8.0) |
| Total arch replacement, n (%) | 1530 (49.0) | 326 (57.0) | 1015 (77.0) |
| Inclusion technique, n (%) | 2201 (70.5) | 461 (80.6) | 386 (29.3) |
| Concomitant CABG, n (%) | 225 (7.2) | 38 (6.6) | 75 (5.7) |
| Concomitant valve surgery, n (%) | 145 (4.6) | 25 (4.4) | 32 (2.4) |
| Cardiopulmonary bypass time (min), median (IQR) | 171 (137-206) | 177 (141-220) | 189 (138-236) |
| Aortic cross-clamp time (min), median (IQR) | 99 (77-123) | 100 (79-129) | 110 (82-140) |
| Circulatory arrest of the lower body, n (%) | 2029 (65.1) | 404 (71.0) | 1126 (85.4) |
| Circulatory arrest time (min), median (IQR) | 23(18-30) | 23 (18-30) | 28 (19-34) |
| Perioperative outcomes | |||
| Operative mortality, n (%) | 180 (5.8) | 37 (6.5) | 133 (10.1) |
| Mechanical ventilation time (h), median (IQR) | 18(14-38) | 20 (15-42) | 36 (17-92) |
| ICU stay (h), median (IQR) | 29 (19-64) | 36 (20-83) | 42 (26-95) |
| Hospital stay (d), median (IQR) | 16(11-22) | 15 (11-21) | 18 (13-26) |
AVR, aortic valve regurgitation; CABG, coronary artery bypass grafting; FET = frozen elephant trunk; ICU, intensive care unit; IQR, interquartile range; SII, systemic inflammatory-immune; STI, systemic thrombo-inflammatory; WBC, white blood cell.
Variables were collected during the first 24 hours after admission to test for risk factors associated with the end point of mortality.
SII index was calculated by platelet count multiplied by neutrophile count divided by lymphocyte count.
STI index was calculated by platelet count divided by WBC count.
Identifying the Conventional Clinical Risk Factors
On the basis of LASSO regression, we identified 11 conventional clinical covariates (creatinine level, age, hemoglobin level, prior cardiac surgery, extent of dissection extension, cerebral perfusion, aortic regurgitation, sex, pericardial effusion, shock, and coronary perfusion) associated with operative mortality in the derivation cohort, with the optimal k penalty (Figure 2A and B). A risk score was calculated for each patient: −3.345 + (0.021 × age [year]) + (−0.009 × male) + (0.463 × prior cardiac surgery) + (−0.493 × limited in ascending aorta + 0.399 × extended to descending aorta) + (−0.063 × no aortic valve regurgitation [AVR] + 0.071 × moderate AVR + 0.076 × severe AVR) + (1.289 × massive pericardial effusion) + (1.214 × shock) + (−0.004 × hemoglobin [g/L]) + (0.001 × serum creatinine [μmol/L]) + (−0.440 × no coronary malperfusion) + (0.586 × cerebral malperfusion). Dose dependency of mortality risk was identified for an increasing risk score (odds ratio, 3.325; 95% CI, 2.612-4.233; P<.0001) (Figure 2C). This LASSO-based risk score yielded the AUC (AUC, 0.715; 95% CI, 0.675-0.751) and a Brier score of 0.058 with good fit according to the Hosmer-Lemeshow test (P=.532).
Figure 2.
Characterization and performances of the LASSO model and the machine learning model. A, Coefficient profile plots of the LASSO model. B, Penalty plot for the LASSO model. C, Dose-response relationship between risk score and mortality. D, Relative importance of 12 variables predictive from machine learning (ML) inflammatory mode (5A risk score) in the derivation cohort. E, Prediction distributions of patients with acute type A aortic dissection according to the risk of mortality in ML inflammatory mode (5A risk score). F, The standard rate and odds ratio of operative mortality among the fourth quartile of ML inflammatory mode (5A risk score). AUC, the area under the receiver operating characteristic curve; LASSO, least absolute shrinkage and selection operator; OR, odds ratio; STI, systemic thrombo-inflammatory.
Selecting the Optimal Inflammatory Signature for Predicting Mortality
Of hematologic candidates, the AUC of STI index (0.664 [95% CI, 0.620-0.708]) was significantly greater than that of individual signatures (WBC count, 0.628 [0.584-0.673]; platelet count, 0.619 [0.575-0.662]; neutrophile count, 0.568 [0.521-0.614]; and lymphocyte count, 0.511 [0.468-0.554]; each vs STI index all P<.01) and collaborative signatures (platelet-lymphocyte ratio, 0.558 [0.512-0.604]; systemic inflammatory-immune index, 0.516 [0.472-0.560]; neutrophil-lymphocyte ratio, 0.503 [0.460-0.546]; each vs STI index all P<.01), respectively (Supplemental Figure 1A, available online at http://www.mayoclinicproceedings.org/). Considering the superior discrimination of STI index, we decided to select it as the final inflammatory variable to develop a novel risk model in combination with conventional clinical covariates.
Building the Inflammation-based Risk Model by Clinical and Inflammatory Candidates
On the basis of 11 identified clinical candidates, we develop an inflammation-based risk model using 7 machine learning classifiers. Among the evaluated machine learning classifiers based on 11 identified clinical candidates, the highest accuracy regarding the AUC analysis was achieved by the XGBoost classifier (AUC, 0.827; 95%CI, 0.795-0.859), followed by adaptive boosting, NB, logistic regression, RF, KNN, and SVM, all of which outperformed our LASSO-based risk model (for all, P<.05) (Supplemental Figure 1B and Supplemental Tables 1 and 2, available online at http://www.mayoclinicproceedings.org/). We, therefore, decided to use the XGBoost classifier as the final machine learning algorithm to create the risk model.
By adding STI index into this XGBoost-base model, we developed an additive inflammatory risk model, called 5A score (Figure 2D). In comparison, the 5A score model was superior to the base model with an incremental AUC of 0.046 from 0.827 to 0.873 and a decremental Brier score of 0.037 from 0.047 to 0.043 (Supplemental Table 3, available online at http://www.mayoclinicproceedings.org/), which was further confirmed by the net reclassification improvement index (0.072, [95% CI, 0.013-0.132], P=.018) and integrated discrimination improvement index (0.107, [95% CI, 0.045-0.168], P=.0007). The prediction distribution plot of 5A score with patient sorted in the order of risk showed positive clustering of patients who died (Figure 1E), asserting that the model accurately stratified patients at risk of mortality. With reference to the quartile 1 of 5A score, the quartiles 2, 3, and 4 conferred significantly higher risk of mortality in the derivation cohort (for trend, P<.001; Figure 2F). This 5A score yielded superior calibration performances with respect to good fit according to the Hosmer-Lemeshow test (P=.237) and calibration curves in comparison with LASSO-based model and base model (Figure 3). The decision curves also showed that the 5A score had better performance than these 2 models in clinical application (Figure 3). In terms of secondary outcomes, the discrimination performances of the 5A score were adequate for the prediction of a mechanical ventilation duration of greater than 48 hours (AUC, 0.837 [0.823-0.850]), an ICU stay duration of more than 3 days (AUC, 0.856 [0.843-0.868]), and hospital days of more than 30 days (AUC, 0.914 [0.903-0.923]).
Figure 3.
Comparison of the prediction performances of the LASSO-based model, ML-based clinical model, and ML-based inflammatory model in the derivation cohort. A, The AUC of the LASSO-based model. B, The AUC of the ML base model. C, The AUC of the ML inflammatory model. D, Calibration curves of the LASSO-based model. E, Calibration curves of the ML base model. F, Calibration curves of the ML inflammatory model. G, Decision curves of the LASSO-based model. H, Decision curves of the ML base model. I, Decision curves of the ML inflammatory model. AUC, area under the receiver operating characteristic curve; LASSO, least absolute shrinkage and selection operator; ML, machine learning.
Internal and External Validation of 5A Risk Model
The discriminatory ability of the 5A score from the derivation cohort was comparable with an AUC of 0.875 (0.814-0.936), 0.845 (0.811-0.878), and 0.852 (0.821-0.883) in the internal, external validation, and total cohort, respectively (Supplemental Table 4, available online at http://www.mayoclinicproceedings.org/). The calibration plots and Brier scores are showed in Figure 4. The Hosmer-Lemeshow test was not significant for the internal cohort (P=.544), the external cohort (P=.508), and the total cohort (P=.475), which indicated a good fit. The decision curves for mortality probability also showed relatively good performance in terms of clinical application (Figure 4).
Figure 4.
The prediction performances of the inflammatory model in the internal, external and total cohorts. A, The AUC of the inflammatory model in the internal cohort. B, The AUC of the inflammatory model in the external cohort. C, The AUC of the inflammatory model in the total cohort; D, Calibration curve of inflammatory model in the internal cohort; E, Calibration curve of inflammatory model in the external cohort; F, Calibration curve of inflammatory model in the total cohort; G, Decision curve of inflammatory model in the internal cohort; H, Decision curve of inflammatory model in the external cohort; I, Decision curve of inflammatory model in the total cohort. AUC = the area under the receiver operating characteristic curve.
Inflammatory Model Outperforms Existing Risk Models
This 5A score model’s predicted probabilities achieved higher accuracy for predicting mortality than that of currently available clinical scores in the total cohort: GERAADA score, Additive EuroSCORE, Logistic EuroSCORE, Parsonnet score, the Provincial Adult Cardiac Care Network of Ontario score, Cleveland score, and SinoSCORE (each vs inflammatory model, all P<0.001) (Supplemental Figure 2 and Supplemental Table 5, available online at http://www.mayoclinicproceedings.org/). The calibration performances and decision curves were assessed (Supplemental Figure 2, available online at http://www.mayoclinicproceedings.org/).
Discussion
In the present study, we developed and tested an inflammation-based risk stratification tool (5A score) to predict operative mortality of patients with ATAAD who underwent surgical repair by integrating the derived inflammatory hematologic variable (STI index) and 11 conventional clinical risk factors. Among the evaluated machine learning classifiers, XGBoost demonstrated the best performance; therefore, this algorithm was used to create the risk score. The 5A score denominated good discrimination performance for predicting operative mortality and mechanical ventilation duration, ICU stay, and hospital days. In the total cohort, the predictive value of the 5A score was confirmed and superior to that of other currently available clinical risk scores. Collectively, these data showed that a novel inflammatory risk score offers a significant improvement in mortality risk discrimination compared with a clinical risk model based on traditional risk factor.
Current Difficulties in Risk Stratification of Patients With ATAAD
Appropriate risk stratification is necessary for emergency critical patients’ providers to identify optimized and individualized treatment strategies for patients. The availability of scores in clinical-specific subpopulations is crucial for tailored risk stratification. Until now, risk stratification of patients with ATAAD has barely been reported and has not been comprehensively validated.9,18,19 The first attempt to derive a specific model for mortality prediction in patients with ATAAD was the Antonius Dissection Scoring System, which is a statistical model for mortality derived from a single-center database.32 To the best of our knowledge, this model has never been externally validated. An important prerequisite for the development of a generalizable risk stratification model is the availability of an external validation cohort. Although this validation cohort should not differ too much in terms of region and timing, it is highly advantageous to have a database that is created, filled in, and located independently from the derivation cohort, thus limiting investigator bias. Our study provided optimal conditions for the establishment of a risk stratification model because of its large size and the entirely independent data collection at the 13 participating sites.
The Strong Performance of the 5A Score in External Validation
Among all tested scoring systems, the novel 5A score provided the best risk stratification for operative mortality in patients with ATAAD. One possible explanation for the 5A score’s strong performance in comparison with other scores is that other scores, such as the logistic EuroSCORE or the SinoSCORE, were not specifically developed for patients with ATAAD. This suggests that patients with ATAAD exhibit additional risk factors beyond those that can be seen in patients undergoing general cardiac surgery, in whom these scores have been successfully applied.3, 4, 5, 6, 7, 8 The available scores specifically designed to risk-stratify patients with ATAAD, such as The International Registry of Acute Aortic Dissection score and GERAADA score,9,18,19,32, 33, 34, 35 were not able to reach the 5A score’s performance either, which may be caused by the fewer covariates used for the development of these scoring models. Thus, with the advantages of a larger population, the external validation and the specificity to patients with ATAAD, the 5A score may serve as a helpful risk stratification tool for daily clinical use. Precise preoperative risk stratification facilitates better decision making for surgery teams.
Particularly Predictive Features in the 5A Model
Machine learning facilitated hierarchical testing of inflammatory (STI index), metabolic (creatinine and hemoglobin), systemic (shock), cardiac (AVR, aortic regurgitation, and pericardial effusion), and specific (coronary perfusion and cerebral perfusion) parameters obtained during characterization of the dissection with respect to mortality, rather than classical cardiovascular risk factors, such as diabetes mellitus and hypertension. Thus, the 5A score model encompasses 12 parameters that can easily be evaluated in clinical routine and aid patient counseling. Some of the variables in our model resembled those used in previous cardiovascular scoring systems such as the EuroSCORE score (hemoglobin),3,4 GERAADA score (urea/blood urea nitrogen and blood pressure),9 and The International Registry of Acute Aortic Dissection score (prior cardiac surgery, shock, and myocardial ischemia).33,34 Specially, we introduced inflammatory signature (STI index) as a predictive factor and found that STI index was the most important predictor of mortality in the 5A score. Meanwhile, the STI index was identified as a decent predictor of mortality, whose addition into the base model significantly improved performance. This supports the suggestion that STI disorder, which is easily evaluated at the bedside, should be used clinically to predict operative mortality and should be collected in aortic dissection databases.36,37
The 5A risk model effectively identifies a gradient risk of operative mortality, thus supporting clinical decisions. A less aggressive arch repair or other alternative strategies, such as endovascular, hybrid, and staged operations, might be more appropriate for the high-risk patients predicted via 5A score.35,38 However, more aggressive strategy may be recommended for patients with the predicted low risk of operative mortality against the risk of future aortic events associated with less invasive aortic treatment.39
Study Limitations
Although this analysis provides major strengths, such as a large sample size and an external validation cohort, most limitations of this study are caused by the retrospective nature of the data source. The results of our study should not be generalized to patients with chronic stable type A aortic dissection or to patients who are selected to undergo surgery. Although the 5A score had a satisfying performance in the prediction of operative mortality, it was rather complicated, and the calculations were not conveniently available. Therefore, the application of 5A score may be restricted. Considering that the use of anticoagulants or antiplatelet therapy might have an impact on the platelet count, which might interfere with the STI index, we excluded those patients who received anticoagulants or antiplatelet therapy in the most recent 3 months, which might increase the potential bias of patient selection, potentially limiting the generalizability of these results to other institutions. Operative death was the only outcome that was assessed in this analysis, and, although important, it is not sufficient for full evaluation of patients with type A aortic dissection. Taken together, the application of the 5A score in additional complete, comprehensive, and prospective datasets will be of future interest.
Conclusion
In this study, we introduced the novel risk prediction system (5A score), which is an inflammation-based model to estimate operative mortality among patients with ATAAD undergoing open surgical repair. These results suggest that the 5A score can represent an important step toward individualized treatment in the field of aortic dissections.
Potential Competing Interests
The authors report no competing interests.
Acknowledgments
All authors fulfilled the International Committee of Medical Journal Editors Criteria for Authorship. Hong Liu contributed to the design of the study. All authors had access to the data presented in the manuscript. All authors verified the results of both datasets. Dr Liu drafted the manuscript, and all authors interpreted the data and results and intellectually and critically revised the manuscript and approved its final form. Drs Liu and Li were responsible for the decision to submit the manuscript for publication. The 5A Investigators are listed in the Supplemental Materials.
Footnotes
Drs Liu, Qian, Y. Zhang, Y. Wu, Hong, Yang, Zhong, Wang, D. Wu, Fan, Chen, S. Zhang, and Peng equally contributed to this work.
Grant Support: This work was supported by grants from the National Natural Science Foundation of China (82000305 and 82070483), Scientific Research Common Program of Beijing Municipal Commission of Education (KM202110025014), and Beijing Municipal Science and Technology Commission (Z211100002921010).
Data Availability Statement: The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available because of privacy or ethical restrictions.
Data Sharing: The datasets used to train and validate the 5A Score are legally restricted because of Chinese patient privacy and secrecy laws and are, therefore, not publicly available. The datasets used for training are available from the corresponding author on reasonable request.
Supplemental material can be found online at http://www.mcpiqojournal.org/. Supplemental material attached to journal articles has not been edited, and the authors take responsibility for the accuracy of all data.
Supplemental Online Material
References
- 1.Erbel R., Aboyans V., Boileau C., et al. 2014 ESC Guidelines on the diagnosis and treatment of aortic diseases: document covering acute and chronic aortic diseases of the thoracic and abdominal aorta of the adult. The Task Force for the Diagnosis and Treatment of Aortic Diseases of the European Society of Cardiology (ESC) Eur Heart J. 2014;35(41):2873–2926. doi: 10.1093/eurheartj/ehu281. [DOI] [PubMed] [Google Scholar]
- 2.Bossone E., Eagle K.A. Epidemiology and management of aortic disease: aortic aneurysms and acute aortic syndromes. Nat Rev Cardiol. 2021;18(5):331–348. doi: 10.1038/s41569-020-00472-6. [DOI] [PubMed] [Google Scholar]
- 3.Nashef S.A., Roques F., Michel P., Gauducheau E., Lemeshow S., Salamon R. European system for cardiac operative risk evaluation (EuroSCORE) Eur J Cardiothorac Surg. 1999;16(1):9–13. doi: 10.1016/s1010-7940(99)00134-7. [DOI] [PubMed] [Google Scholar]
- 4.Roques F., Michel P., Goldstone A.R., Nashef S.A. The logistic EuroSCORE. Eur Heart J. 2003;24(9):881–882. doi: 10.1016/s0195-668x(02)00799-6. [DOI] [PubMed] [Google Scholar]
- 5.Parsonnet V., Dean D., Bernstein A.D. A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease. Circulation. 1989;79(6 Pt 2):I3–12. [PubMed] [Google Scholar]
- 6.Higgins T.L., Estafanous F.G., Loop F.D., Beck G.J., Blum J.M., Paranandi L. Stratification of morbidity and mortality outcome by preoperative risk factors in coronary artery bypass patients. A clinical severity score. JAMA. 1992;267(17):2344–2348. [PubMed] [Google Scholar]
- 7.Tu J.V., Jaglal S.B., Naylor C.D. Multicenter validation of a risk index for mortality, intensive care unit stay, and overall hospital length of stay after cardiac surgery. Steering Committee of the Provincial Adult Cardiac Care Network of Ontario. Circulation. 1995;91(3):677–684. doi: 10.1161/01.cir.91.3.677. [DOI] [PubMed] [Google Scholar]
- 8.Zheng Z., Zhang L., Li X., Hu S., Chinese CABG Registry Study SinoSCORE: a logistically derived additive prediction model for post-coronary artery bypass grafting in-hospital mortality in a Chinese population. Front Med. 2013;7(4):477–485. doi: 10.1007/s11684-013-0284-0. [DOI] [PubMed] [Google Scholar]
- 9.Czerny M., Siepe M., Beyersdorf F., et al. Prediction of mortality rate in acute type A dissection: the German Registry for Acute Type A Aortic Dissection score. Eur J Cardiothorac Surg. 2020;58(4):700–706. doi: 10.1093/ejcts/ezaa156. [DOI] [PubMed] [Google Scholar]
- 10.Sodeck G., Domanovits H., Schillinger M., et al. Pre-operative N-terminal pro-brain natriuretic peptide predicts outcome in type A aortic dissection. J Am Coll Cardiol. 2008;51(11):1092–1097. doi: 10.1016/j.jacc.2007.12.015. [DOI] [PubMed] [Google Scholar]
- 11.Li G., Wu X.W., Lu W.H., et al. High-sensitivity cardiac troponin T: a biomarker for the early risk stratification of type-A acute aortic dissection? Arch Cardiovasc Dis. 2016;109(3):163–170. doi: 10.1016/j.acvd.2015.09.007. [DOI] [PubMed] [Google Scholar]
- 12.Zindovic I., Sjögren J., Bjursten H., et al. The coagulopathy of acute type A aortic Dissection: a Prospective, Observational Study. J Cardiothorac Vasc Anesth. 2019;33(10):2746–2754. doi: 10.1053/j.jvca.2019.02.013. [DOI] [PubMed] [Google Scholar]
- 13.Benedetto U., Dimagli A., Kaura A., et al. Determinants of outcomes following surgery for type A acute aortic dissection: the UK National Adult Cardiac Surgical Audit. Eur Heart J. 2021;43(1):44–52. doi: 10.1093/eurheartj/ehab586. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Allaire E., Schneider F., Saucy F., et al. New insight in aetiopathogenesis of aortic diseases. Eur J Vasc Endovasc Surg. 2009;37(5):531–537. doi: 10.1016/j.ejvs.2009.02.002. [DOI] [PubMed] [Google Scholar]
- 15.Al’Aref S.J., Anchouche K., Singh G., et al. Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging. Eur Heart J. 2019;40(24):1975–1986. doi: 10.1093/eurheartj/ehy404. [DOI] [PubMed] [Google Scholar]
- 16.Kilic A. Artificial intelligence and machine learning in cardiovascular health care. Ann Thorac Surg. 2020;109(5):1323–1329. doi: 10.1016/j.athoracsur.2019.09.042. [DOI] [PubMed] [Google Scholar]
- 17.Than M.P., Pickering J.W., Sandoval Y., et al. Machine learning to predict the likelihood of acute myocardial infarction. Circulation. 2019;140(11):899–909. doi: 10.1161/CIRCULATIONAHA.119.041980. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Guo T., Fang Z., Yang G., et al. Machine learning models for predicting in-hospital mortality in acute aortic dissection patients. Front Cardiovasc Med. 2021;8 doi: 10.3389/fcvm.2021.727773. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Collins G.S., Reitsma J.B., Altman D.G., Moons K.G.M. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BJS Open. 2015;102(3):148–158. doi: 10.1002/bjs.9736. [DOI] [PubMed] [Google Scholar]
- 20.Overman D.M., Jacobs J.P., Prager R.L., et al. Report from the Society of Thoracic Surgeons National Database Workforce: clarifying the definition of operative mortality. World J Pediatr Congenit Heart Surg. 2013;4(1):10–12. doi: 10.1177/2150135112461924. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Zindovic I., Gudbjartsson T., Ahlsson A., et al. Malperfusion in acute type A aortic dissection: an update from the Nordic Consortium for Acute Type A Aortic Dissection. J Thorac Cardiovasc Surg. 2019;157(4):1324–1333.e6. doi: 10.1016/j.jtcvs.2018.10.134. [DOI] [PubMed] [Google Scholar]
- 22.Tibshirani R.J. Regression shrinkage and selection via the Lasso. J R Stat Soc B. 1996;58(1):267–288. [Google Scholar]
- 23.Liu H., Zheng S.Q., Li X.Y., et al. Derivation and validation of a nomogram to predict in-hospital complications in children with tetralogy of Fallot repaired at an older age. J Am Heart Assoc. 2019;8(21) doi: 10.1161/JAHA.119.013388. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Churpek M.M., Yuen T.C., Winslow C., Meltzer D.O., Kattan M.W., Edelson D.P. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med. 2016;44(2):368–374. doi: 10.1097/CCM.0000000000001571. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Meurer W.J., Tolles J. Logistic regression diagnostics: understanding how well a model predicts outcomes. JAMA. 2017;317(10):1068–1069. doi: 10.1001/jama.2016.20441. [DOI] [PubMed] [Google Scholar]
- 26.DeLong E.R., DeLong D.M., Clarke-Pearson D.L. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–845. [PubMed] [Google Scholar]
- 27.Pencina M.J., D’Agostino R.B., Steyerberg E.W. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21. doi: 10.1002/sim.4085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Hosmer D.W., Hosmer T., Le Cessie S., Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16(9):965–980. doi: 10.1002/(sici)1097-0258(19970515)16:9<965::aid-sim509>3.0.co;2-o. [DOI] [PubMed] [Google Scholar]
- 29.Wu Y.C., Lee W.C. Alternative performance measures for prediction models. PLoS One. 2014;9(3) doi: 10.1371/journal.pone.0091249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Vickers A.J., Elkin E.B. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–574. doi: 10.1177/0272989X06295361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Alba A.C., Agoritsas T., Walsh M., et al. Discrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. 2017;318(14):1377–1384. doi: 10.1001/jama.2017.12126. [DOI] [PubMed] [Google Scholar]
- 32.Tan M.E., Kelder J.C., Morshuis W.J., Schepens M.A. Risk stratification in acute type A dissection: proposition for a new scoring system. Ann Thorac Surg. 2001;72(6):2065–2069. doi: 10.1016/s0003-4975(01)03214-3. [DOI] [PubMed] [Google Scholar]
- 33.Mehta R.H., Suzuki T., Hagan P.G., et al. Predicting death in patients with acute type a aortic dissection. Circulation. 2002;105(2):200–206. doi: 10.1161/hc0202.102246. [DOI] [PubMed] [Google Scholar]
- 34.Rampoldi V., Trimarchi S., Eagle K.A., et al. Simple risk models to predict surgical mortality in acute type A aortic dissection: the International Registry of Acute Aortic Dissection score. Ann Thorac Surg. 2007;83(1):55–61. doi: 10.1016/j.athoracsur.2006.08.007. [DOI] [PubMed] [Google Scholar]
- 35.Ghoreishi M., Wise E.S., Croal-Abrahams L., et al. A novel risk score predicts operative mortality after acute type A aortic dissection repair. Ann Thorac Surg. 2018;106(6):1759–1766. doi: 10.1016/j.athoracsur.2018.05.072. [DOI] [PubMed] [Google Scholar]
- 36.Kuang J., Yang J., Wang Q., Yu C., Li Y., Fan R. A preoperative mortality risk assessment model for Stanford type A acute aortic dissection. BMC Cardiovasc Disord. 2020;20(1):508. doi: 10.1186/s12872-020-01802-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Evangelista A., Isselbacher E.M., Bossone E., et al. Insights from the International Registry of Acute Aortic Dissection: a 20-year experience of collaborative clinical research. Circulation. 2018;137(17):1846–1860. doi: 10.1161/CIRCULATIONAHA.117.031264. [DOI] [PubMed] [Google Scholar]
- 38.Pollari F., Sirch J., Fischlein T. About usefulness of GERAADA score. Eur J Cardiothorac Surg. 2021;60(4):1005. doi: 10.1093/ejcts/ezab168. [DOI] [PubMed] [Google Scholar]
- 39.Heuts S., Adriaans B.P., Kawczynski M.J., et al. Editor’s Choice: extending aortic replacement beyond the proximal arch in acute type A aortic dissection: a meta-analysis of short term outcomes and long term actuarial survival. Eur J Vasc Endovasc Surg. 2022;63(5):674–687. doi: 10.1016/j.ejvs.2021.12.045. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.