Table 4.
General characteristics, methodological details, main results, and overall quality score for risk prediction studies.
| Reference | Country (population) | Task (prediction time) | Modality | Data amount (% cases) | Data amount external validation (% cases) | Inputs (no.) | Model | CV train / validation / test | K-fold cross validation | Evaluation metrics | Best result | Quality score (max 12) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Cary et al.72 | USA | HF Mortality (30 d, 1 yr) | Database | 17 140 (15%) | 15 | Logreg, MLP | 10-fold CV | 10-fold | Accuracy, AUC, precision, slope | AUC: 0.76 | 8 | |
| Chen et al.73 | China | VF (5 yr) | Database | 1603 (8%) | 147 | Logreg, SVM, DT, KNN, RF, ERT, GBDT, AdaBoost, CatBoost, XGB, MLP, hybrid | 80% train, 20% test | NR | Accuracy, precision, recall, F1-score, AUC | Accuracy: 0.90 | 9 | |
| Chen et al.74 | China | MOF (6 yr) | Database | 487 (NA) | 22 | RF, MLP, SVM, XGB, DT | 70% train, 30% test | 10-fold | AUC, DCA, CIC | AUC: 0.87 | 9 | |
| Cheng et al.75 | Taiwan | BMD Loss (6 yr) | Database | 23 497 (14%) | 17 | Logreg, XGB, RF, SVM | 80% train, 20% test | 10-fold | Sensitivity, specificity, AUC, accuracy, precision, f1-score | AUC: 0.75 | 7 | |
| Coco Martín et al.76 | Spain | MOF (> 1 yr) | Database | 993 (28%) | 25 | Logreg, MLP | 70% train, 30% test | Accuracy | Accuracy: 0.96 | 7 | ||
| De Vries et al.77 | Netherlands | MOF (3 yr, 5 yr) | Database | 7578 (11%) | 46 | Coxreg, RSF, MLP | 10-fold CV | 10-fold | C-index | C-Index: 0.70 | 11 | |
| DeBaun et al.78 | USA | HF Mortality (30 d) | Database | 19 835 (1051) | 43 | MLP, naive Bayes, Logreg | 80% train, 20% test | AUC | AUC: 0.92 | 4 | ||
| Du et al.79 | China | HF | Database | 120 (NA) | 13 | R2U-Net and SVM, RF, GBDT, AdaBoost, MLP, XGB | 80% train, 20% test | Accuracy, specificity, recall, precision | Accuracy: 0.96 | 5 | ||
| Forssten et al.80 | Sweden | HF Mortality (1 yr) | Database | 124 707 (17%) | 25 | Logref, SVM, RF, NB | 80%train, 20% test | 5-fold | Accuracy, sensitivity, specificity, AUC | AUC: 0.74 | 11 | |
| Galassi et al.81 | Spain | HF | Database | 137 (65%) | 38 | Logreg, SVM, DT, RF | 70% train, 30% test | Sensitivity, specificity, accuracy | Accuracy: 0.87 | 6 | ||
| Harris et al.82 | USA | HF Mortality (30 d) | Database | 82 168 (5%) | 46 | LASSO | 10-fold | 10-fold | Accuracy, C-Index | Accuracy: 0.76 | 8 | |
| Kitcharanant et al.83 | Thailand | HF Mortality (1 yr) | Database | 492 (13%) | 15 | GB, RF, MLP, Logreg, NB, SVM, KNN | 70% train, 30% test | Accuracy, sensitivity, specificity, AUC, PPV, NPV | AUC: 0.99 | 10 | ||
| Klemt et al.84 | USA | HF Revision Surgery (> 2 yr) | Database | 350 (5.2%) | NR | MLP, RF, KNN, PLR | 80% train, 20% test | 5-fold | AUC, intercept, calibration, Brier score | AUC: 0.81 | 6 | |
| Kong et al.85 | South Korea | VF | Database, X-ray | 1595 (7.5%) | IMG patches (+7) | HRNet + ResNet and DeepSurv, Coxreg | 89% train, 11% test | 5-fold | AUC, sensitivity, specificity, PPV, NPV, C-Index | C-Index: 0.61 | 11 | |
| Lei et al.86 | China | HF Mortality | Database | 391 (13.8%) | 165 (10.9%) | 27 | RF, GBM, DT, XGB | 67% train, 33% test | 10-fold | AUC, accuracy, sensitivity, specificity, Youden Index, intercept, calibration slope | AUC: 0.71 | 7 |
| Lu et al.87 | UK | MOF | Database | 345 (28%) | 359 | Logreg, RF | 80% train, 20% test | 5-fold | AUC, sensitivity, specificity | AUC: 0.90 | 7 | |
| Ma et al.88 | China | VF | Database | 529 (10.6%) | 27 | DT, RF, SVM, GBM, MLP, RDA, Logreg | 75% train, 25% test | 10-fold | AUC, Kappa, sensitivity, specificity | AUC: 0.94 | 9 | |
| Oosterhoff et al.89 | Netherlands | HF Mortality (90 d, 2 yr) | Database | 2478 (9.1% and 23.5%) | 14 | SGB, RF, SVM, MLP, PLR | 80% train, 20% test | 10-fold | AUC, intercept, calibration, Brier score | AUC: 0.74 (90 d), AUC: 0.70 (2 yr) | 9 | |
| Poullain et al.90 | France | VF | Database | 60 (50%) | 16 | RF, CART | k-fold | k-fold | Sensitivity, specificity, AUC | AUC: 0.92 | 5 | |
| Shimizu et al.91 | Japan | HF, FF (> 2 yr) | Database | 6590 (4.4%) | 10 | LightGBM, ANN | 75% train, 25% test | AUC | AUC: 0.75 | 4 | ||
| Shtar et al.92 | Israel | HF Rehabilitation (8 yr) | Database | 1896 (14%) | 18 | Linreg, Logreg, AdaBoost, CatBoost, ExtraTrees, KNN, RF, SVM, XGB, ensemble | NR | 10-fold | AUC, R2 | AUC: 0.86 | 10 | |
| Takahashi et al.93 | Japan | VF (Nonunion) | Database | 153 (17%) | 17 | Logreg, DT, XGB, RF | 70% train, 30% test | 5-fold | AUC, accuracy | AUC: 0.86 | 9 | |
| Ulivieri et al.94 | Italy | VF (9 yr) | Database | 174 (69) | 9 | MLP | 70% train, 30% test | Sensitivity, specificity, AUC, accuracy | AUC: 0.82 | 5 | ||
| Ulivieri et al.95 | Italy | VF (3 yr) | Database | 172 (54%) | 26 | MLP | NR | Sensitivity, specificity, accuracy, AUC | Accuracy: 0.79 | 6 |
Abbreviations: VF, vertebral fracture; HF, hip fracture; FF, forearm fracture; MOF, major osteoporotic fracture; BMD, bone mineral density; NA, non assessable; MLP, multilayer perceptron; Logreg, logistic regression; GB, gradient boosting; RF, random forests; NB, Naïve Bayes; SVM, support vector machines; KNN, k-nearest neighbors; SGB, stochastic gradient boosting; PLR, Elastic-Net Penalized Logistic Regression; DT, decision tree; Linreg, linear regression; XGB, extreme gradient boosting; Coxreg, Cox regression; RSF, random survival forests; ERT, extremely randomized trees; GBDT, gradient boosted decision trees; GBM, gradient boosting machines; RDA, regularized discriminant analysis; CART, classification and regression tree; LASSO, least absolute shrinkage and selection operator; AUC, area under the curve of the receiver operating characteristic; PPV, positive predictive value; NPV, negative predictive value; C-Index, concordance index; R2, coefficient of determination; DCA, decision curve analysis; CIC, clinical impact curve. If several models were evaluated for a given task, the best performing is highlighted in bold.