Table 2.
Internal and external performances of the 8 different trained models.
| Model | Internal validation (cross validation), mean (SD) | External temporal validation | |||||||
|
|
AUC-ROCa | Accuracy (%) | Sensitivity (%) | Specificity (%) | AUC-ROC | Accuracy (%) | Sensitivity (%) | Specificity (%) | |
| LRb | 0.670 (0.097) | 66.4 (7.1) | 76.3 (10.1) | 53.7 (9) | 0.85 | 69.2 | 76.9 | 61.5 | |
| BNCc | 0.624 (0.092) | 62.1 (7.5) | 66.3 (11.5) | 55.9 (11.3) | 0.83 | 69.2 | 69.2 | 69.2 | |
| RFd | 0.780 (0.084) | 74.7 (7.3) | 85.2 (7.1) | 61 (11.5) | 0.90 | 84.6 | 100.0 | 69.2 | |
| GBTe | 0.765 (0.092) | 73.5 (6.9) | 82.2 (7.8) | 62.4 (10.8) | 0.82 | 76.9 | 92.3 | 61.5 | |
| XGBf | 0.760 (0.087) | 72.2 (7.1) | 80.2 (7.8) | 62.1 (9.9) | 0.82 | 80.8 | 92.3 | 69.2 | |
| SVMg | 0.723 (0.094) | 67.6 (8.3) | 81.8 (9.8) | 49.1 (13.6) | 0.72 | 61.5 | 84.6 | 38.5 | |
| KNNh | 0.669 (0.089) | 63.8 (6.1) | 76.2 (9) | 48.3 (11.1) | 0.76 | 69.2 | 84.6 | 53.8 | |
| ANNi (64x32) | 0.690 (0.085) | 66.7 (7) | 77 (8.6) | 55.4 (10.9) | 0.65 | 65.0 | 54.0 | 77.0 | |
aAUC-ROC: area under the receiver operating characteristic curve.
bLR: logistic regression.
cBNC: Bayesian naive classification.
dRF: random forest.
eGBT: gradient-boosted tree.
fXGB: XGBoost.
gSVM: support vector machine.
hKNN: k-nearest neighbor.
iANN: artificial neural network.