Table 2.
Sensitivity and specificity (a); accuracy and AUC (b); and time measurement results (c) for observer tests without and with AI-based devices as a concurrent reader.
| Author | Without AI-Based CAD | With AI-Based CAD | Change | Statistical Significance between Difference | ||
| Sensitivity (%) | Specificity (%) | Sensitivity (%) | Specificity (%) | |||
| a | ||||||
| Bai et al. [13] | 79 | 88 | 88 | 91 | ↑ | p < 0.001 |
| Beyer et al. [19] | 56.5 | - | 61.6 | - | ↑ | p < 0.001 |
| de Hoop et al. [20] | 56 * | - | 56 * | - | ↑ | - |
| Dorr et al. [14] | 47 | 79 | 61 | 75 | ↑ | p < 0.007 |
| Kim et al. [15] | 73.9 | 88.7 | 82.2 | 98.1 | ↑ | p < 0.014 |
| Koo et al. [21] | 92.4 | 93.1 | 95.1 | 97.2 | ↑ | - |
| Kozuka et al. [22] | 68 | 91.7 | 85.1 | 83.3 | ↑ | p < 0.01 ** |
| Lee et al. [23] | 84 | - | 88 | - | ↑ | - |
| Rajpurkar et al. [31] | 70 | 52 | 73 | 61 | ↑ | - |
| Singh et al. [28] | 68 * | 77.5 * | 73 * | 74 * | ↑ | - |
| Sung et al. [30] | 80.1 | 89.3 | 88.9 | 96.6 | ↑ | p < 0.01 |
| Yang et al. [17] | 89.5 | - | 94.2 | - | ↑ | p < 0.05 |
| Author | Without AI-Based CAD | With AI-Based CAD | Change | Statistical Significance between Difference | ||
| Accuracy (%) | AUC | Accuracy (%) | AUC | |||
| b | ||||||
| Bai et al. [13] | 85 | - | 90 | - | ↑ | p < 0.001 |
| Kim et al. [15] | - | 0.871 | - | 0.916 | ↑ | p = 0.002 |
| Koo et al. [21] | - | 0.93 | - | 0.96 | ↑ | p < 0.0001 |
| Li et al. [24] | - | 0.840 | - | 0.863 | ↑ | p = 0.01 |
| Li et al. [25] | - | 0.807 | - | 0.867 | ↑ | p < 0.001 |
| Liu et al. [26] | - | 0.66 * | - | 0.78 * | ↑ | - |
| Nam et al. [29] | 66.3 * | - | 82.4 * | - | ↑ | p < 0.05 |
| Rajpurkar et al. [31] | 60 | - | 65 | - | ↑ | p = 0.002 |
| Singh et al. [28] | - | 0.73 * | - | 0.74 * | ↑ | Not statistically significant |
| Sung et al. [30] | - | 0.93 | - | 0.98 | ↑ | p = 0.003 |
| Yang et al. [17] | 94.1 | - | 95.1 | - | ↑ | p = 0.01 |
| Author | Without AI-Based CAD | With AI-Based CAD | Change | Statistical Significance between Difference | ||
| Time | Time | |||||
| c | ||||||
| Beyer et al. [19] | 294 s (1) | 337 s (1) | ↓ | p = 0.04 | ||
| Kim et al. [15] | 165 min (2) | 101 min (2) | ↑ | - | ||
| Kozuka et al. [22] | 373 min(2) | 331 min (2) | ↑ | - | ||
| Liu et al. [16] | 100.5 min (3) | 34 min (3) | ↑ | p < 0.01 | ||
| Liu et al. [26] | 15 min (1) | 5–10 min (1) | ↑ | - | ||
| Martini et al. [27] | 194 s (1) | 154 s (1) | ↑ | p < 0.001 | ||
| Nam et al. [29] | 2771.2 s * (1) | 1916 s * (1) | ↑ | p < 0.002 | ||
| Sung et al. [30] | 24 s (1) | 12 s (1) | ↑ | p < 0.001 | ||
| Zhang et al. [18] | 3.623 min (2) | 0.744 min (2) | ↑ | - | ||
a: * our calculated average; ** for sensitivity only; - not applicable; ↑ positive change. b: * our calculated average; - not applicable; ↑ positive change. c: (1) per image/case reading time; (2) total reading time for multiple cases; (3) station survey time; * our calculated average; - not applicable; ↑ positive change; ↓ negative change.