Table 3.
Mean 10-fold cross-validation (CV) Training set performance and held-out test set performance of top models from raw and imputed datasets.
Training Performance | ||||||||||||
Raw Data | Mean | Median | KNN | Bagged Trees | ||||||||
Top Models | NB | XGBTree | SVM-RB | Log Reg | SVM-RB | Log Reg | DT | SVM-RB | Log Reg | NB | SVM-RB | Log Reg |
Brier Score | 0.156 | 0.157 | 0.117 | 0.172 | 0.115 | 0.251 | 0.175 | 0.115 | 0.228 | 0.176 | 0.119 | 0.190 |
Accuracy | 83.5% | 79.9% | 86.2% | 78.2% | 85.4% | 74.9% | 75.5% | 86.3% | 72.1% | 81.9% | 83.5% | 78.5% |
AUROC curve | 0.869 | 0.861 | 0.924 | 0.839 | 0.919 | 0.739 | 0.826 | 0.908 | 0.756 | 0.877 | 0.909 | 0.803 |
Test Performance | ||||||||||||
Raw Data | Mean | Median | KNN | Bagged Trees | ||||||||
Top Models | NB | XGBTree | SVM-RB | Log Reg | SVM-RB | Log Reg | DT | SVM-RB | Log Reg | NB | SVM-RB | Log Reg |
Brier Score | 0.117 | 0.166 | 0.069 | 0.069 | 0.066 | 0.066 | 0.183 | 0.077 | 0.077 | 0.105 | 0.068 | 0.068 |
Accuracy | 86.2% | 79.3% | 93.1% | 93.1% | 93.1% | 93.1% | 72.4% | 89.7% | 89.7% | 89.7% | 93.1% | 93.1% |
AUROC curve | 0.928 | 0.856 | 0.981 | 0.981 | 0.986 | 0.986 | 0.820 | 0.976 | 0.976 | 0.952 | 0.962 | 0.962 |