Table A3.
Metrics of the three models at the ideal and 0.20 thresholds for the in- and external validation. Metrics of the three models with 95% CI at the ideal (calculated with the Youden’s J statistic) and 0.20 threshold probability, for both the internal validation (5A–B) and external validation (5C–D). Random Forest (RF); Logistic Regression (LR); Naïve Bayes (NB); negative predictive value (NPV); Positive predictive value (PPV); Area under the receiver operating characteristics curve (AUROC).
| 5A | |||||||
|---|---|---|---|---|---|---|---|
| Model | AUROC | Sensitivity | Specificity | NPV | PPV | Misclassification | Ideal Threshold |
| RF model | 0.81 [0.75–0.91] | 0.83 [0.73–0.91] | 0.65 [0.49–0.80] | 0.90 [0.83–0.95] | 0.51 [0.40–0.65] | 0.30 [0.18–0.42] | 0.20 |
| LR model | 0.81 [0.76–0.91] | 0.85 [0.73–0.95] | 0.60 [0.51–0.71] | 0.91 [0.84–0.97] | 0.48 [0.42–0.55] | 0.32 [0.25–0.38] | 0.21 |
| NB model | 0.82 [0.74–0.91] | 0.85 [0.77–0.95] | 0.62 [0.43–0.75] | 0.90 [0.83–0.97] | 0.50 [0.39–0.61] | 0.31 [0.21–0.44] | 0.17 |
| 5B | |||||||
| Model | AUROC | Sensitivity | Specificity | NPV | PPV | Misclassification | Threshold |
| RF model | 0.81 [0.75–0.91] | 0.83 [0.73–0.91] | 0.65 [0.49–0.80] | 0.90 [0.83–0.95] | 0.51 [0.40–0.65] | 0.30 [0.18–0.42] | 0.20 |
| LR model | 0.81 [0.76–0.91] | 0.86 [0.73–0.95] | 0.59 [0.49–0.69] | 0.91 [0.84–0.97] | 0.48 [0.42–0.54] | 0.33 [0.26–0.40] | 0.20 |
| NB model | 0.82 [0.74–0.91] | 0.83 [0.73–0.91] | 0.67 [0.53–0.78] | 0.90 [0.84–0.95] | 0.53 [0.41–0.64] | 0.28 [0.18–0.40] | 0.20 |
| 5C | |||||||
| Model | AUROC | Sensitivity | Specificity | NPV | PPV | Misclassification | Threshold |
| RF model | 0.79 [0.72–0.86] | 0.90 [0.81–0.96] | 0.50 [0.38–0.62] | 0.85 [0.71–0.94] | 0.62 [0.52–0.72] | 0.31 [0.23–0.39] | 0.20 |
| LR model | 0.80 [0.73–0.87] | 0.90 [0.81–0.96] | 0.50 [0.38–0.62] | 0.85 [0.71–0.94] | 0.62 [0.52–0.72] | 0.31 [0.23–0.39] | 0.21 |
| NB model | 0.75 [0.67–0.82] | 0.89 [0.79–0.95] | 0.47 [0.36–0.59] | 0.82 [0.68–0.92] | 0.61 [0.51–0.70] | 0.33 [0.25–0.41] | 0.17 |
| 5D | |||||||
| Model | AUROC | Sensitivity | Specificity | NPV | PPV | Misclassification | Threshold |
| RF model | 0.79 [0.72–0.86] | 0.90 [0.81–0.96] | 0.50 [0.38–0.62] | 0.85 [0.71–0.94] | 0.62 [0.52–0.72] | 0.31 [0.23–0.39] | 0.20 |
| LR model | 0.80 [0.73–0.87] | 0.92 [0.83–0.97] | 0.49 [0.37–0.60] | 0.86 [0.73–0.95] | 0.62 [0.52–0.71] | 0.31 [0.23–0.39] | 0.20 |
| NB model | 0.75 [0.67–0.82] | 0.79 [0.68–0.88] | 0.56 [0.45–0.68] | 0.75 [0.62–0.85] | 0.63 [0.52–0.73] | 0.31 [0.25–0.41] | 0.20 |