Table 2. Performance Characteristics of Models for Predicting In-Hospital Mortality in Acute Myocardial Infarction.
| Characteristic | Logistic regression | LASSO | Neural network | XGBoost | Meta-classifier |
|---|---|---|---|---|---|
| Variables included in the model of McNamara et al21 | |||||
| Model performance metrics | |||||
| AUROC (95% CI) | 0.878 (0.875-0.881) | 0.874 (0.870-0.879) | 0.874 (0.870-0.878) | 0.886 (0.882-0.890) | 0.886 (0.882-0.890) |
| Precision-recall AUC | 0.372 | 0.367 | 0.371 | 0.395 | 0.398 |
| F score | 0.415 | 0.408 | 0.411 | 0.432 | 0.432 |
| Sensitivity | 0.42 (0.41-0.43) | 0.43 (0.42-0.45) | 0.41 (0.40-0.42) | 0.44 (0.43-0.45) | 0.43 (0.42-0.44) |
| Specificity | 0.97 (0.97-0.97) | 0.97 (0.97-0.97) | 0.97 (0.97-0.97) | 0.97 (0.97-0.97) | 0.98 (0.97-0.98) |
| PPV | 0.41 (0.40-0.42) | 0.38 (0.37-0.39) | 0.41 (0.40-0.42) | 0.42 (0.41-0.43) | 0.44 (0.43-0.45) |
| NPV | 0.97 (0.97-0.97) | 0.97 (0.97-0.98) | 0.97 (0.97-0.97) | 0.98 (0.97-0.98) | 0.97 (0.97-0.98) |
| Brier score | |||||
| Reliability, mean (SD), ×10−6 | 28.4 (9.2) | 96.3 (16.5) | 224.0 (26.1) | 9.5 (3.8) | 2.3 (2.1) |
| Resolution, mean (SD), ×10−3 | 5.6 (0.1) | 5.5 (0.1) | 5.4 (0.1) | 5.8 (0.1) | 5.9 (0.1) |
| Uncertainty | 0.04 | 0.04 | 0.04 | 0.04 | 0.04 |
| Overall, ×10−2 | 3.52 | 3.54 | 3.56 | 3.49 | 3.48 |
| Expanded variables included from the CP-MI Registry | |||||
| Model performance metrics | |||||
| AUROC (95% CI) | 0.888 (0.884-0.892) | 0.886 (0.882-0.890) | 0.885 (0.881-0.889) | 0.898 (0.894-0.902) | 0.899 (0.895-0.903) |
| Precision-recall AUC | 0.421 | 0.415 | 0.406 | 0.451 | 0.453 |
| F score | 0.436 | 0.436 | 0.428 | 0.458 | 0.459 |
| Sensitivity | 0.47 (0.45-0.48) | 0.42 (0.41-0.43) | 0.43 (0.42-0.44) | 0.45 (0.44-0.47) | 0.43 (0.42-0.44) |
| Specificity | 0.97 (0.97-0.97) | 0.98 (0.98-0.98) | 0.97 (0.97-0.98) | 0.98 (0.98-0.98) | 0.98 (0.98-0.98) |
| PPV | 0.41 (0.40-0.42) | 0.45 (0.44-0.46) | 0.43 (0.42-0.44) | 0.46 (0.45-0.47) | 0.49 (0.48-0.50) |
| NPV | 0.98 (0.98-0.98) | 0.97 (0.97-0.98) | 0.97 (0.97-0.98) | 0.98 (0.98-0.98) | 0.97 (0.97-0.98) |
| Brier score | |||||
| Reliability, mean (SD), ×10−6 | 229.4 (25.6) | 40.6 (10.3) | 55.7 (11.2) | 6.5 (3.5) | 4.3 (2.6) |
| Resolution, mean (SD), ×10−3 | 6.0 (0.1) | 5.9 (0.1) | 5.8 (0.1) | 6.4 (0.2) | 6.5 (0.2) |
| Uncertainty | 0.04 | 0.04 | 0.04 | 0.04 | 0.04 |
| Overall, ×10−2 | 3.50 | 3.49 | 3.50 | 3.43 | 3.42 |
Abbreviations: AUROC, area under the receiver operator characteristic curve; AUC, area under curve; CP-MI Registry, National Cardiovascular Data Registry’s Chest Pain–MI Registry; LASSO, least absolute shrinkage and selection operator; NPV, negative predictive value; PPV, positive predictive value.