Table 6.
Diagnostic performance metrics of clinicians and DERM, using images from each camera, for all lesions in the Per Protocol population.
| Device | Lesions (N) | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | FNR (95% CI) | FPR (95% CI) | |
|---|---|---|---|---|---|---|---|---|
| Melanoma | Clinicians | 581 | 81.2% (53.7–95.0%) | 98.9% (97.6–99.6%) | 68.4% (43.5–86.4%) | 99.5% (98.3–99.9%) | 18.8% (5.0–46.3%) | 1.1% (0.4–2.4%) |
| iPhone 6S | 578 | 100% (74.7–100%) | 69.6% (65.6–73.4%) | 8.1% (4.7–13.2%) | 100% (98.8–100%) | 0% (0–25.3%) | 30.4% (26.6–34.4%) | |
| iPhone 11 | 571 | 93.3% (66.0–99.7%) | 73.6% (69.6–77.1%) | 8.7% (5.0–14.4%) | 99.8% (98.4–100%) | 6.7% (0.3–34.0%) | 26.4% (22.9–30.4%) | |
| Samsung | 578 | 100% (75.9–100%) | 65.5% (61.4–69.4%) | 7.6% (4.6–12.3%) | 100% (98.7–100%) | 0% (0–24.1%) | 34.5% (30.6–38.6%) | |
| SCC | Clinicians | 565 | 63.6% (47.7–77.2%) | 89.1% (86–91.5%) | 32.9% (23.4–44.1%) | 96.7% (94.5–98.0%) | 36.4% (22.8–52.3%) | 10.9% (8.5–14.0%) |
| iPhone 6S | 563 | 95.4% (83.3–99.2%) | 44.7% (40.4–49.1%) | 12.8% (9.5–17%) | 99.2% (96.6–99.9%) | 4.6% (0.8–16.7%) | 55.3% (50.9–59.6%) | |
| iPhone 11 | 556 | 93.2% (80.3–98.2%) | 45.7% (41.3–50.1%) | 12.8% (9.5–17.1%) | 98.7% (96–99.7%) | 6.8% (1.8–19.7%) | 54.3% (49.9–58.7%) | |
| Samsung | 562 | 90.9% (77.4–97%) | 50.6% (46.2–55%) | 13.5% (9.9–18.1%) | 98.5% (95.9–99.5%) | 9.1% (3–22.6%) | 49.4% (45–53.8%) | |
| BCC | Clinicians | 521 | 97.5% (93.9–99.1%) | 77.4% (72.4–81.8%) | 72.6% (66.7–77.7%) | 98% (95.2–99.3%) | 2.5% (0.9–6.1%) | 22.6% (18.2–27.6%) |
| iPhone 6S | 519 | 94.9% (90.6–97.4%) | 41.6% (36.2–47.2%) | 49.9% (44.7–55%) | 93.1% (87.3–96.4%) | 5.1% (2.6–9.4%) | 58.4% (52.8–63.8%) | |
| iPhone 11 | 512 | 95.8% (91.7–98%) | 45% (39.5–50.6%) | 51.1% (45.8–56.4%) | 94.7% (89.5–97.5%) | 4.2% (2–8.3%) | 55% (49.4–60.5%) | |
| Samsung | 518 | 94.4% (89.9–97%) | 54.5% (48.9–60%) | 55.6% (50.1–61%) | 94.1% (89.4–96.9%) | 5.6% (3.0–10.1%) | 45.5% (40–51.1%) | |
| Malignant | Clinicians | 583 | 93.8% (90–96.3%) | 77.4% (72.4–81.8%) | 77% (71.9–81.4%) | 94.3% (90.6–96.7%) | 5.8% (3.4–9.5%) | 22.6% (18.2–27.6%) |
| iPhone 6S | 580 | 95.7% (92.3–97.7%) | 41.6% (36.2–47.2%) | 56.8% (52–61.5%) | 92.4% (86.5–96%) | 4.3% (2.3–7.7%) | 58.4% (52.8–63.8%) | |
| iPhone 11 | 573 | 96.0% (92.6–98%) | 45% (39.5–50.6%) | 58% (53.1–62.7%) | 93.5% (88.1–96.7%) | 4% (2–7.4%) | 55% (49.4–60.5%) | |
| Samsung | 580 | 94.9% (91.3–97.2%) | 54.5% (48.9–60%) | 62.4% (57.4–67.2%) | 93.1% (88.3–96.1%) | 5.1% (2.8–8.7%) | 45.5% (40–51.1%) | |
| IEC | Clinicians | 323 | 90.9% (57.1–99.5%) | 78.8% (73.8–83.2%) | 13.2% (6.8–23.3%) | 99.6% (97.4–100%) | 9.1% (0.5–42.9%) | 21.1% (16.8–26.2%) |
| iPhone 6S | 322 | 100% (67.9–100%) | 43.1% (37.5–48.8%) | 5.9% (3.1–10.5%) | 100% (96.5–100%) | 0% (0–32.1%) |
56.9% (51.2–62.5%) | |
| iPhone 11 | 320 | 100% (67.9–100%) | 46.6% (41–52.3%) | 6.2% (3.3–11.2%) | 100% (96.8–100%) | 0% (0–32.1%) |
53.4% (47.7–59%) | |
| Samsung | 323 | 90.9% (57.1–99.5%) | 56.1% (50.4–61.6%) | 6.8% (3.5–12.5%) | 99.4% (96.4–100%) | 9.1% (0.5–42.9%) | 43.9% (38.4–49.6%) | |
| AK | Clinicians | 312 | 96.7% (87.6–99.4%) | 79.3% (73.6–84%) | 53.1% (43.5–62.6%) | 99% (96.1–99.8%) | 3.3% (0.6–12.4%) | 20.7% (16–26.4%) |
| iPhone 6S | 311 | 85.0% (72.9–92.5%) | 43.4% (37.2–49.8%) | 26.4% (20.5–33.3%) | 92.4% (85.6–96.2%) | 15% (7.5–27.1%) | 56.6% (50.2–62.8%) | |
| iPhone 11 | 309 | 84.8% (72.5–92.4%) | 47.2% (40.9–53.6%) | 27.5% (21.3–34.7%) | 92.9% (86.6–96.5%) | 15.2% (7.6–27.5%) | 52.8% (46.4–59.1%) | |
| Samsung | 312 | 83.6% (71.5–91.4%) | 51.4% (45–57.7%) | 29.5% (22.9–37%) | 92.8% (86.8–96.3%) | 16.4% (8.6–28.5%) | 48.6% (42.3–55%) | |
| Atypical | Clinicians | 251 | 76.2% (52.5–90.9%) | 73.9% (67.6–79.4%) | 21% (12.9–32.2%) | 97.1% (93.1–98.9%) | 23.8% (9.1–47.5%) | 26.1% (20.6–32.4%) |
| iPhone 6S | 251 | 86.4% (64.0–96.4%) | 39.3% (33.0–46.0%) | 12% (7.6–18.4%) | 96.8% (90.2–99.2%) | 13.6% (3.6–36.0%) | 60.7% (54.0–67.0%) | |
| iPhone 11 | 250 | 59.1% (36.7–78.5%) | 43.9% (37.4–50.6%) | 9.2% (5.2–15.6%) | 91.7% (84.5–95.9%) | 40.9% (21.5–63.3%) | 56.1% (49.4–62.6%) | |
| Samsung | 251 | 38.1% (19.0–61.3%) | 48.3% (41.7–54.9%) | 6.3% (3.0–12.4%) | 89.5% (82.4–94.1%) | 61.9% (38.7–81.0%) | 51.7% (45.1–58.3%) | |
| Premalignant | Clinicians | 323 | 91.4% (83.3–95.9%) | 73.9% (67.6–79.4%) | 58.6% (50.1–66.6%) | 95.5% (91.0–97.9%) | 8.6% (4.1–16.7%) | 26.1% (20.6–32.4%) |
| iPhone 6S | 322 | 87.1% (78.2–92.9%) | 39.3% (33.0–46.0%) |
36.8% (30.5–43.6%) | 88.2% (80.0–93.5%) | 12.9% (7.1–21.8%) | 60.7% (54.0–67.0%) | |
| iPhone 11 | 320 | 80.4% (70.6–87.7%) | 43.9% (37.4–50.6%) | 36.6% (30.1–43.7%) | 84.8% (76.7–90.5%) | 19.6% (12.3–29.4%) | 56.1% (49.4–62.6%) | |
| Samsung | 323 | 75.3% (65.0–83.4%) | 48.3% (41.7–54.9%) | 37% (30.2–44.4%) | 82.8% (75.1–88.6%) | 24.7% (16.6–35.0%) | 51.7% (45.1–58.3%) | |
| Benign | Clinicians | 581 | 73.9% (67.6–79.4%) | 93.7% (90.5–95.9%) | 88.5% (83.0–92.5%) | 84.6% (80.5–87.9%) | 26.1% (20.6–32.4%) | 6.3% (4.1–9.5%) |
| iPhone 6S | 578 | 39.3% (33.0–46.0%) | 94.3% (91.1–96.4%) | 81.8% (73.1–88.3%) | 70.3% (65.9–74.4%) | 60.7% (54.0–67.0%) | 5.7% (3.6–8.9%) | |
| iPhone 11 | 571 | 43.9% (37.4–50.6%) | 93.3% (90.0–95.6%) | 81.3% (73.1–87.5%) | 71.4% (67.0–75.5%) | 56.1% (49.4–62.6%) | 6.7% (4.4–10.0%) | |
| Samsung | 578 | 48.3% (41.7–54.9%) | 91.4% (87.8–94.0%) | 78.7% (70.9–85.0%) | 72.8% (68.3–76.8%) | 51.7% (45.1–58.3%) | 8.6% (6–12.2%) |
SCC, Squamous Cell Carcinoma; BCC, Basal Cell Carcinoma; IEC, Intraepidermal Carcinoma; AK, Actinic Keratosis; CI, Confidence Intervals Rate; PPV, Positive Predictive Value; NPV, Negative Predictive Value; FPR, False Positive Rate; FNR, False Negative.