Table 2.
Retrospective validation studies of artificial intelligence in diabetic retinopathy
| References | Algorithm | Approach | Dataset | Images (n) | Detection | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | AUC (95% CI) |
|---|---|---|---|---|---|---|---|---|---|---|
| Larsen et al. [27] | Retinalyze | ML | WCDRS | 400 | DR/no DR | 96.7% | 71.4% | N/A | N/A | 0.903 |
| Larsen et al. [28] | Retinalyze | ML | Steno Diabetes Center, Denmark | 365 | DR/no DR | 93.1% | 71.6% | N/A | N/A | 0.936 |
| Oliveira et al. [23] | Retmarker | DL | ARS-Centro | 21,544 | DR/no DR | 96.1% (94.4–97.9) | 51.7% (50.3–53.1) | N/A | N/A | 0.849 |
| Abramoff et al. [29] | IDP | ML | Messidor-2 | 1748 | rDR | 96.8% (94.4–99.3) | 59.4% (55.7–63.0) | 39.8% (35.2–44.3) | 98.5% (97.4–99.7) | 0.937 (0.916–0.959) |
| Hansen et al. [30] | IDP | ML | Nakuru Eye Study | 6788 | DR/no DR | 91.0% (88.0–93.4) | 69.9% (68.3–71.6) | 32.1% (29.6–34.7) | 98.0% (97.4–98.6) | 0.878 (0.850–0.905) |
| Abramoff et al. [21] | IDx-DR X2.1 | DL | Messidor-2 | 1748 | rDR | 96.8% (93.3–98.8) | 87.0% (84.2–89.4) | 67.4% (61.5–72.9) | 99.0% (97.8–99.6) | 0.980 (0.968–0.992) |
| VTDR | 100% (96.1–100) | 90.8% (88.5–92.7) | 56.4% (48.4–64.1) | 100% (99.5–100) | 0.989 (0.984–0.994) | |||||
| Bhaskaranand et al. [33] | EyeArt v1.2 | ML | EyePACS | 40,542 | rDR | 90.0% (88.0–92.0) | 63.2% (61.7–64.6) | N/A | N/A | 0.879 (0.865–0.893) |
| Gulshan et al. [11] | ARDA | DL | Messidor-2 | 1748 | rDR | 87.0% (81.1–91.0) | 98.5% (97.7–99.1) | N/A | N/A | 0.990 (0.986–0.995) |
| EyePACS-1 | 9963 | rDR | 90.3% (87.5–92.7) | 98.1% (97.8–98.5) | N/A | N/A | 0.991 (0.988–0.993) | |||
| Ting et al. [32] | SELENA+ | DL | SIDRP | 71,896 | rDR | 90.5% (87.3–93.0) | 91.2% (88.0–93.6) | N/A | N/A | 0.936 (0.925–0.943) |
| VTDR | 100% (94.1–100) | 91.1% (90.7–91.4) | N/A | N/A | 0.958 (0.956–0.961) | |||||
| Li et al. [35] | DLA | DL | LabelMe, China | 35,201 | rDR | 92.5% | 98.5% | N/A | N/A | 0.955 |
| Bhaskaranand et al. [34] | EyeArt v2.0 | DL | EyePACS | 850,908 | rDR | 91.3% (90.9–91.7) | 91.1% (90.9–91.3) | 72.5% (71.9–73.0) | 97.6% (97.5–97.7) | 0.965 (0.963–0.966) |
| Ruamviboonsuk et al. [31] | ARDA | DL | Thailand’s national screening program | 25,326 | rDR | 96.8% (89.3–99.3) | 95.6% (98.3–98.7) | N/A | N/A | 0.987 (0.977–0.995) |
| Grzybowski and Brona [36] | Retinalyze strategy 1 | ML | Poznan, Poland | 680 | rDR | 89.7% (78.8–96.1) | 71.8% (62.4–80.0) | 62.7% | 92.9% | 0.807 |
| Retinalyze strategy 2 | ML | Poznan, Poland | 680 | rDR | 74.1% (61.0–84.7) | 93.6% (87.3–97.4) | 86.0% | 87.3% | 0.839 | |
| IDx-DR | DL | Poznan, Poland | 680 | rDR | 93.3% (83.8–98.2) | 95.5% (89.7–98.5) | 91.8% | 96.3% | 0.944 |
AI Artificial intelligence, AUC area under the curve, CI confidence interval, DL deep learning, DLA deep learning algorithm, IDP Iowa Detection Program, ML machine learning, NPV negative predictive value, PPV positive predictive value, rDR referable diabetic retinopathy, VTDR vision threatening diabetic retinopathy