Table 3.
Test characteristics of Annalise.ai interpretation compared to radiologist
Radiologist reporting of CXR | Annalise.ai | AUC (95% CI) | Radiologist & Annalise.ai combined | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
TP | FP | TN | FN | Sensitivity | Specificity | TP | FP | TN | FN | Sensitivity | Specificity | Sensitivity Specificity | |||
Pneumothorax | 57 | 8 | 1224 | 115 | 33.1% (26.2–40.7) |
99.4% (98.7–99.7) |
67 | 2 | 1227 | 104 | 39.2% (31.8–46.9) |
99.8% (99.4–100) |
0.926 (0.896–0.953) | 45.6% (38.0–53.4) |
99.2% (98.5–98.5-99.6) |
Pneumomediastinum | 3 | 1 | 1376 | 24 | 11.1% (2.4–29.2) |
99.9% (99.6–100) |
3 | 0 | 1373 | 24 | 11.1 (2.4–29.2) |
100% (99.7–100) |
0.872 (0.786–0.958) | 14.8% (4.2–33.7) |
99.9% (99.6–100) |
Rib fracture | 113 | 13 | 1042 | 236 | 32.4% (27.5–37.6) |
98.8% (97.9–99.3) |
143 | 75 | 977 | 205 | 41.1% (35.9–46.5) |
92.9% (91.2–94.4) |
0.749 (0.717–0.780) | 49.4% (44.1–54.8) |
91.9 (90.1–93.5) |
Clavicle fracture | 43 | 3 | 1322 | 36 | 54.4% (42.8–65.7) |
99.8% (99.3–100) |
44 | 37 | 1284 | 35 | 55.7% (44.1–66.9) |
97.2% (96.2–98.0) |
0.831 (0.775–0.887) | 69.6% (58.3–79.5) |
97.1 (96.0–97.9) |
Humerus fracture | 21 | 2 | 1371 | 10 | 67.7% (48.6–83.3) |
99.9% (99.5–100) |
10 | 8 | 1361 | 21 | 32.3% (16.7–51.4) |
99.4% (98.9–99.8) |
0.836 (0.743–0.929) | 74.2% (55.4–88.1) |
99.3% (98.8–99.7) |
Scapular fracture | 15 | 5 | 1344 | 40 | 27.3% (16.1–41.0) |
99.6% (99.1–99.9) |
19 | 64 | 1281 | 36 | 34.6% (22.2–48.9) |
95.2% (94.0–96.3) |
0.855 (0.790–0.920) | 45.5% (32.0–59.5) |
94.9 (93.6–96.1) |
Lobar/segmental collapse | 4 | 1 | 1366 | 33 | 10.8% (3.0–25.4) |
99.9% (99.6–100) |
13 | 21 | 1343 | 23 | 36.1% (20.8–53.8) |
98.5% (97.7–99.0) |
0.917 (0.856–0.979) | 36.1% (20.8–53.8) |
98.5% (97.7–99.0) |
FN, false negative; FP, false positive; TP, true positive; TP, true positive.