Table 3.
Model Assessment on Internal Validation Using 0.632 Bootstrapping With 1000 Resampled Datasets (n = 1451)
| Metric | Area under the curve |
Calibration slope | Calibration intercept | Brier Score | |
|---|---|---|---|---|---|
| Apparent | Internal Validation | ||||
| Elastic net | 0.687 (0.651-0.722) | 0.649 (0.647-0.651) | 0.967 (0.956-0.978) | 0.006 (0.004-0.008) | 0.14 (0.127-0.152) |
| Random forest | 0.966 (0.956- 0.97) | 0.710 (0.709-0.732) | 0.969 (0.964-0.975) | 0.006 (0.004-0.007) | 0.121 (0.11-0.133) |
| XGBoost | 0.995 (0.994-0.997) | 0.690 (0.687-0.699) | 0.969 (0.963-0.975) | 0.006 (0.004-0.007) | 0.126 (0.113-0.139) |
| SVM | 0.763 (0.761-0.764) | 0.633 (0.641-0.635) | 0.963 (0.951-0.974) | 0.007 (0.004-0.009) | 0.142 (0.129-0.155) |
| Neural Network | 0.692 (0.69-0.693) | 0.629 (0.627-0.631) | 0.987 (0.975-0.999) | 0.002 (0-0.005) | 0.142 (0.13-0.155) |
| Ensemble | 0.801 (0.8-0.802) | 0.722 (0.707-0.764) | 0.968 (0.965-0.971) | 0.006 (0.005-0.007) | 0.116 (0.104-0.128) |
GLM, generalized linear model; SVM, support vector machine; XGBoost, extreme gradient boosting.
Null model Brier score = 0.148