Table 1.
Model 1, 2 and 3 performances. Classification performance in terms of AUC, accuracy, sensitivity, specificity, PPV, NPV, corresponding 95% confidence interval and statistical significance with respect to chance/random classification (p-value). Performances are reported for training, validation and internal testing.
| Model | Metric | Training | Validation | Internal Testing (Mean) |
Internal Testing (Majority Vote) |
|---|---|---|---|---|---|
| 1 | ROC-AUC (%) (95% CI) | 100 1 [99,100] | 83 2 [81–84] | 84 2 [80–87] | 85 |
| Accuracy (%) (95% CI) | 100 1 [99,100] | 79 2 [79,80] | 81 2 [77–85] | 83 | |
| Sensitivity (%) (95% CI) | 100 1 [99,100] | 67 2 [81–84] | 68 2 [64–71] | 70 | |
| Specificity (%) (95% CI) | 100 1 [99,100] | 84 2 [83–85] | 87 2 [82–91] | 88 | |
| PPV (%) (95% CI) | 100 1 [99,100] | 63 2 [60–66] | 66 2 [58–75] | 70 | |
| NPV (%) (95% CI) | 100 1 [99,100] | 87 2 [86–89] | 87 2 [86–89] | 88 | |
| 2 | ROC-AUC (%) (95% CI) | 84 2 [83–86] | 81 2 [79–83] | 79 2 [76–83] | 80 |
| Accuracy (%) (95% CI) | 77 2 [76–78] | 73 2 [71–76] | 75 2 [72–77] | 75 | |
| Sensitivity (%) (95% CI) | 78 2 [78,79] | 74 2 [72–77] | 76 2 [69–83] | 78 | |
| Specificity (%) (95% CI) | 76 2 [74–78] | 73 2 [70–76] | 74 2 [71–77] | 75 | |
| PPV (%) (95% CI) | 77 2 [76–78] | 53 2 [49–57] | 54 2 [51–56] | 54 | |
| NPV (%) (95% CI) | 77 2 [77,78] | 88 2 [87–89] | 89 2 [86–92] | 89 | |
| 3 | ROC-AUC (%) (95% CI) | 95 2 [94–96] | 62 2 [60–64] | 77 2 [73–81] | 79 |
| Accuracy (%) (95% CI) | 87 2 [85–88] | 58 2 [56–61] | 71 2 [67–74] | 75 | |
| Sensitivity (%) (95% CI) | 96 2 [95–98] | 57 2 [47–68] | 71 2 [60–82] | 73 | |
| Specificity (%) (95% CI) | 77 2 [74–79] | 59 2 [53–64] | 71 2 [66–76] | 75 | |
| PPV (%) (95% CI) | 81 2 [79–83] | 36 2 [35–37] | 49 2 [45–53] | 54 | |
| NPV (%) (95% CI) | 96 2 [94–97] | 79 2 [75–82] | 86 2 [82–90] | 88 |
1 p-value < 0.05/2 p-value < 0.005.