Table 2.
Comparison of deep learning systems with cornea specialists in low-quality images
| One-versus-rest classification | PEDLS | NDLS | Cornea specialist A | Cornea specialist B | P1 | P2 | P3 | P4 | P5 |
|---|---|---|---|---|---|---|---|---|---|
| Keratitis versus others + normal | |||||||||
| Sensitivity (95% CI) | 70.5% (61.8–79.2) | 77.1% (69.1–85.2) | 72.4% (63.8–80.9) | 92.4% (87.3–97.5) | 0.815 | <0.001 | 0.383 | 0.001 | 0.016 |
| Specificity (95% CI) | 69.5% (63.1–76.0) | 90.9% (86.8–94.9) | 88.3% (83.8–92.8) | 94.4% (91.2–97.6) | <0.001 | <0.001 | 0.487 | 0.248 | <0.001 |
| Accuracy (95% CI) | 69.9% (64.7–75.0) | 86.1% (82.2–90.0) | 82.8% (78.5–87.0) | 93.7% (91.0–96.4) | <0.001 | <0.001 | 0.220 | 0.001 | <0.001 |
| Others versus keratitis + normal | |||||||||
| Sensitivity (95% CI) | 46.0% (35.5–56.4) | 80.5% (72.1–88.8) | 59.8% (49.5–70.1) | 90.8% (84.7–96.9) | 0.065 | <0.001 | 0.011 | 0.093 | <0.001 |
| Specificity (95% CI) | 66.5% (60.2–72.8) | 87.4% (83.0–91.9) | 74.0% (68.1–79.8) | 94.0% (90.8–97.1) | 0.073 | <0.001 | <0.001 | 0.034 | <0.001 |
| Accuracy (95% CI) | 60.6% (55.1–66.1) | 85.4% (81.5–89.4) | 69.9% (64.7–75.0) | 93.0% (90.2–95.9) | 0.009 | <0.001 | <0.001 | 0.004 | <0.001 |
| Normal versus keratitis + others | |||||||||
| Sensitivity (95% CI) | 35.5% (36.5–44.4) | 74.5% (66.4–82.7) | 68.2% (59.5–76.9) | 89.1% (83.3–94.9) | <0.001 | <0.001 | 0.360 | 0.009 | <0.001 |
| Specificity (95% CI) | 91.1% (87.1–95.2) | 87.5% (82.8–92.2) | 89.6% (85.3–93.9) | 97.9% (95.9–99.9) | 0.728 | 0.001 | 0.636 | <0.001 | 0.092 |
| Accuracy (95% CI) | 70.9% (65.7–76.0) | 82.8% (78.5–87.0) | 81.8% (77.4–86.1) | 94.7% (92.2–97.2) | 0.001 | <0.001 | 0.826 | <0.001 | <0.001 |
PEDLS, previously established deep learning system; NDLS, new deep learning system; CI, confidence interval. “Others” denotes other corneal abnormalities. “Normal” denotes normal cornea. P1 indicates the p value calculated between the PEDLS and cornea specialist A using the McNemar test. P2 indicates the p value calculated between the PEDLS and cornea specialist B using the McNemar test. P3 indicates the p value calculated between the NDLS and cornea specialist A using the McNemar test. P4 indicates the p value calculated between the NDLS and cornea specialist B using the McNemar test. P5 indicates the p value calculated between the PEDLS and NDLS using the McNemar test. Cornea specialist A has 3 years of clinical experience. Cornea specialist B has 7 years of clinical experience.