Table 3. Additional performance metrics based on a 30% validation dataset for TIMI risk score and ML models with and without feature selection.
| PPV | NPV | Sensitivity | Specificity | Accuracy (Cl 95%) | Mcnemar’s test (p-value) | |
|---|---|---|---|---|---|---|
| In-hospital | ||||||
| Classifier | ||||||
| RF | 0.380 | 0.963 | 0.347 | 0.968 | 0.935 (0.923,0.946) | <0.0001 |
| RFvarImp-SBE-RF | 0.447 | 0.963 | 0.337 | 0.977 | 0.942 (0.931,0.952) | <0.0001 |
| RFE-RF | 0.350 | 0.963 | 0.347 | 0.964 | 0.931 (0.918,0.942) | <0.0001 |
| SVM | 0.242 | 0.976 | 0.614 | 0.892 | 0.877 (0.861, 0.891) | <0.0001 |
| SVMvarImp-SBE-SVM | 0.219 | 0.980 | 0.693 | 0.861 | 0.852 (0.835,0.868) | <0.0001 |
| RFE-SVM | 0.202 | 0.982 | 0.723 | 0.838 | 0.832 (0.815, 0.849) | <0.0001 |
| LR | 0.211 | 0.981 | 0.713 | 0.850 | 0.842 (0.825,0.858) | <0.0001 |
| LRstepwise—SBE-LR | 0.211 | 0.984 | 0.752 | 0.841 | 0.836 (0.814,0.852) | <0.0001 |
| RFE- LR | 0.185 | 0.978 | 0.663 | 0.834 | 0.825 (0.807,0.842) | <0.0001 |
| TIMI | 0.180 | 0.976 | 0.644 | 0.834 | 0.824 (0.806, 0.841) | <0.0001 |
| 30 days | ||||||
| Classifier | ||||||
| RF | 0.389 | 0.946 | 0.373 | 0.949 | 0.903 (0.882,0.921) | <0.0001 |
| RFvarImp-SBE-RF | 0.341 | 0.948 | 0.413 | 0.930 | 0.889 (0.867,0.909) | <0.0001 |
| RFE-RF | 0.414 | 0.947 | 0.387 | 0.952 | 0.907 (0.887,0.925) | <0.0001 |
| SVM | 0.258 | 0.972 | 0.733 | 0.817 | 0.810 (0.784,0.835) | <0.0001 |
| SVMvarImp-SBE-SVM | 0.258 | 0.983 | 0.840 | 0.790 | 0.794 (0.767,0.820) | <0.0001 |
| RFE-SVM | 0.261 | 0.980 | 0.813 | 0.800 | 0.801 (0.774,0.829) | <0.0001 |
| LR | 0.248 | 0.973 | 0.747 | 0.803 | 0.799 (0.771,0.824) | <0.0001 |
| LRstepwise—SBE-LR | 0.281 | 0.974 | 0.747 | 0.834 | 0.827 (0.802,0.851) | <0.0001 |
| RFE- LR | 0.248 | 0.971 | 0.720 | 0.810 | 0.803 (0.776,0.828) | <0.0001 |
| TIMI | 0.245 | 0.962 | 0.627 | 0.832 | 0.816 (0.789, 0.840) | <0.0001 |
| 1-year | ||||||
| Classifier | ||||||
| RF | 0.410 | 0.909 | 0.373 | 0.949 | 0.827 (0.801,0.852) | <0.0001 |
| RFvarImp-SBE-RF | 0.436 | 0.909 | 0.460 | 0.901 | 0.838 (0.811,0.861) | <0.0001 |
| RFE-RF | 0.425 | 0.913 | 0.492 | 0.889 | 0.8318 (0.805,0.856) | <0.0001 |
| SVM | 0.382 | 0.950 | 0.746 | 0.798 | 0.7909 (0.763,0.817) | <0.0001 |
| SVMvarImp-SBE-SVM | 0.357 | 0.950 | 0.754 | 0.773 | 0.771 (0.741,0.798) | <0.0001 |
| RFE-SVM | 0.387 | 0.953 | 0.762 | 0.798 | 0.793 (0.765,0.820) | <0.0001 |
| LR | 0.329 | 0.924 | 0.611 | 0.792 | 0.766 (0.737,0.794) | <0.0001 |
| LRstepwise—SBE-LR | 0.372 | 0.935 | 0.659 | 0.814 | 0.792 (0.764,0.818) | <0.0001 |
| RFE- LR | 0.344 | 0.926 | 0.619 | 0.802 | 0.776 (0.747,0.803) | <0.0001 |
| TIMI | 0.332 | 0.907 | 0.484 | 0.837 | 0.786 (0.758, 0.813) | <0.0001 |