Table 2. Algorithm Performance Metrics.
Model | AUROC | Accuracy, % | F1 score, % | Sensitivity, % | Specificity, % | PPV, % | NPV, % |
---|---|---|---|---|---|---|---|
Teacher | |||||||
Messidor-2 internal validation 1 | 0.939 | 92.0 | 84.1 | 80.4 | 96.1 | 88.1 | 93.2 |
EyePACS internal validation 2 | 0.886 | 84.3 | 51.1 | 56.1 | 89.2 | 46.9 | 92.2 |
Egypt external validation | 0.964 | 93.3 | 96.4 | 94.4 | 75.0 | 98.4 | 45.0 |
Student | |||||||
Messidor-2 internal validation 1 | 0.951 | 93.7 | 87.6 | 84.8 | 96.9 | 90.7 | 94.7 |
EyePACS internal validation 2 | 0.916 | 74.4 | 52.0 | 95.1 | 70.8 | 35.8 | 98.8 |
Egypt external validation | 0.950 | 96.7 | 98.3 | 100 | 41.7 | 96.6 | 100 |
Bespoke | |||||||
Messidor-2 internal validation 1 | 0.985 | 96.5 | 94.8 | 96.5 | 96.9 | 89.1 | 96.1 |
EyePACS internal validation 2 | 0.853 | 74.4 | 83.2 | 74.2 | 75.6 | 94.7 | 33.3 |
Egypt external validation | 0.890 | 94.3 | 50.0 | 50.0 | 97.0 | 50.0 | 97.0 |
Abbreviations: AUROC, area under the receiver operating characteristic curve; NPV, negative predictive value; PPV, positive predictive value.