Table 4.
Per-Lesion Performance Comparison with 95% Confidence Intervals for Readers, CAD Systems and in Combination.
| Study | Endpoint | Level | Zone | Readers(s) Alone | CAD System Alone | Combination | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Cut-Off | SN % | SP % | AUC | Chosen Threshold | SN % | SP % | AUC | Interaction | SN % | SP % | AUC | ||||
| ROI Classification (ROI-C) | |||||||||||||||
| Algohary [19] |
D’Amico ≥ Intermediate | Lesion | WP | PI-RADSv2, ≥3 | 71 (61–80) |
67 (52–80) |
NR | NR | 63 (52–72) |
91 (79–98) |
0.75 | NA | NA | NA | NA |
| Antonelli [20] |
GS 3 + 3 vs. 4 component | Index Lesion |
PZ | Suspected GS ≥ 3 + 4 |
72 | 40 | NR | Matched to reader SP in training set | 90 | 65 | 0.83 | NA | NA | NA | NA |
| TZ | 82 | 44 | NR | 92 | 56 | 0.75 | NA | NA | NA | NA | |||||
| Bonekamp [21] |
GS ≥ 3 + 4 | Lesion | WP | PI-RADSv2, ≥4 | 88 (77–95) |
50 (42–58) |
NR | Matched to reader SN in training set | 97 (88–100) | 58 (50–66) | 0.88 | NA | NA | NA | NA |
| Dinh [23] |
GS ≥ 3 + 4 | Lesion | WP | Likert (1–5), ≥3 | 100 (100–100) |
14 (8–19) |
0.84 (0.77–0.89) |
CAD SN of 95% in training set | 96 (91–100) | 44 (36–52) | 0.88 (0.82–0.93) | NA | NA | NA | NA |
| Dikaios [22] |
GS ≥ 3 + 4 or CCL ≥ 4 mm | Lesion | TZ | PI-RADSv1, ≥3 | 92 (74–99) |
37 (25–50) |
0.74 (0.63–0.86) |
Probability threshold > 0.5 | 60 | 73 | 0.67 (0.55–0.79) |
NA | NA | NA | NA |
| Hambrock [24] |
GS ≥ 3 + 3 and >0.5 cm3 | Lesion | WP | Likelihood scale (0–100), no cut-off | NR | NR | 0.88 (0.85–0.93) |
NA | NR | NR | 0.90 (0.83–0.96) |
CAD scores available to radiologist for interpretation |
NR | NR | 0.91 (0.86–0.97) |
| Iyama [25] |
GS ≥ 3 + 4 and >10 mm vs. BPH | Lesion | TZ | PI-RADSv2, no cut-off | NR | NR | 0.87 (0.81–0.93) |
NA | NR | NR | 0.97 (0.94–0.99) | NA | NA | NA | NA |
| Li [45] |
GS ≥ 3 + 3 | Index Lesion |
WP | PI-RADSv2.1, ≥4 | 91 | 68 | 0.85 | NR | 82 | 82 | 0.86 (0.75–0.94) |
LR model of PI-RADS, CAD score and PSA | 79 | 96 | 0.94 |
| Litjens [44] |
GS ≥ 3 + 4 | Lesion | WP | PI-RADSv1, ≥3 | 100 (98–100) |
9 (0–19) |
0.78 (0.70–0.85) |
NR | NA | NA | NA | LR model of PI-RADS and CAD score | 99 (98–100) |
26
(0–60) |
0.87
(0.81–0.93) |
| Niaf [26] |
GS ≥ 3 + 3 and >2 × 2 mm in-plane |
Lesion | PZ | Likelihood score (0–4), no cut-off |
NR | NR | 0.87 (0.81–0.92) |
NA | NR | NR | 0.82 (0.73–0.90) |
CAD scores available to radiologist for interpretation |
NR | NR | 0.89 (0.83–0.94) |
| Niu [27] |
GS ≥ 3 + 4 | Lesion | PZ | PI-RADSv2, ≥4 | 79 | 75 | 0.76 (0.74–0.83) |
NR | 87 | 89 |
0.89
(0.82–0.94) |
NA | NA | NA | NA |
| TZ | 73 | 77 | 0.73 (0.69–0.81) |
NR | 88 | 81 |
0.87
(0.81–0.92) |
NA | NA | NA | NA | ||||
| Transin [28] |
GS ≥ 3 + 4 | Lesion | PZ | PI-RADSv2, ≥3 | 97 (93–100) |
37 (22–52) |
0.74 (0.62–0.86) |
CAD SN of 95% in training set | 89 (82–97) | 42 (26–58) | 0.78 (0.69–0.87) | NA | NA | NA | NA |
| Wang [29] |
GS ≥ 3 + 3 and >0.5 cm3 | Index Lesion |
WP | PI-RADSv2, ≥3 | 76 (67–84) |
91 (87–94) | 0.86 (0.83–0.90) |
Youden statistics on test set |
90
(84–95) |
88 (85–93) |
0.95
(0.93–0.97) |
SVM model of PI-RADS and CAD score |
92
(87–96) |
95 (93–99) |
0.98
(0.95–0.99) |
| Winkel [30] |
GS ≥ 3 + 4 | Lesion | PZ | PI-RADSv2, ≥3 | 100 | 53 | 0.60 | NR | 100 | 58 | 0.90 | NA | NA | NA | NA |
| Woźnicki [31] |
GS ≥ 3 + 4 | Index Lesion |
WP | PI-RADSv2, ≥4 | NR | NR | 0.69 (0.43–0.89) |
NA | NA | NA | NA | Radiomics model ensembled with PI-RADS, PSAd and DRE models | NR | NR | 0.84 (0.60–1.00) |
| Zhong [32] |
GS ≥ 3 + 4 | Lesion | WP | PI-RADSv2, ≥4 | 86 | 48 | 0.71 (0.58–0.85) |
Point of best accuracy in test set | 64 | 80 | 0.73 (0.58–0.88) |
NA | NA | NA | NA |
| Lesion Localization and Classification (LL&C) | |||||||||||||||
| Cao [33] |
GS ≥ 3 + 4 | Lesion | WP | PI-RADSv2, ≥3 | 81 | NR | NR | FP per patient in test set matched to radiologist (0.62) | 79 | NR | 0.81 | NA | NA | NA | NA |
| Gaur [34] |
GS ≥ 3 + 3 | Index Lesion | WP | PI-RADSv2, ≥3 | 78 | NR | NR | NR | NA | NA | NA | CAD identified lesions reviewed by radiologist | 68 | NR | NR |
| Giannini [35] |
GS ≥ 3 + 4 | Lesion | WP | PI-RADSv2, ≥3 and max diameter ≥7 mm | 72 (61–81) |
NR | NR | Voxel likelihood of malignancy ≥60% and lesion candidate ≥ 100 voxels in size |
81 (61–93) |
NR | NR | CAD identified lesions reviewed by radiologist | 76 (65–85) |
NR | NR |
| Greer [36] |
GS ≥ 3 + 3 | Index Lesion | WP | PI-RADSv2, ≥3 | 78 (69–85) |
NR | NR | NR | NA | NA | NA | CAD identified lesions reviewed by radiologist | 78 (69–86) |
NR | NR |
| Mehralivand [38] |
GS ≥ 3 + 4 | Lesion | WP | PI-RADSv2, ≥3 | 51 (46–57) |
NR | 0.75 | NR | NA | NA | NA | CAD identified lesions reviewed by radiologist | 52 (45–61) |
NR | 0.78 |
| Schelb [39] |
GS ≥ 3 + 4 | Sextant | WP | Mix of PI-RADSv1/v2, ≥3 |
67 (55–78) |
68 (62–73) |
NR | Point that most closely matched PI-RADS ≥ 3 performance in training set | 59 (47–70) |
66 (61–72) |
NR | NA | NA | NA | NA |
| Schelb [40] |
GS ≥ 3 + 4 | Sextant | WP | PI-RADSv2, ≥3 | 71 (65–76) |
62 (60–65) |
NR | Iterative dynamic threshold that most closely matches PI-RADS ≥ 3 performance in most recent cases |
70 (64–75) |
66 (63–69) |
NR | NA | NA | NA | NA |
| Thon [41] |
GS ≥ 2 + 3 | Lesion | WP | PI-RADSv2, no cut-off | NR | NR | 0.68 (0.59–0.76) |
Youden statistics on test set | 47 | 75 | 0.64 (0.53–0.75) | NA | NA | NA | NA |
| Zhu [42] |
GS ≥ 3 + 4 | Lesion | WP | PI-RADSv2, ≥3 | 77 (68–84) |
NR | NR | NR | NA | NA | NA | Radiologist reported with but not limited by CAD probability map |
89
(82–94) |
NR | NR |
(AUC—Area under the receiver operating characteristic curve; BPH—benign prostatic hyperplasia; CAD—computer-aided diagnosis; CCL—cancer core length; FP—false positive; GS—Gleason score; LR—logistic regression; NR—not reported; PI-RADS—Prostate Imaging-Reporting and Data System; PZ—peripheral zone; ROI- region of interest; SN—sensitivity; SP—specificity; SVM—support vector machine; TZ—transition zone; WP—whole prostate). Bold results indicate statistically significant differences to that of reader(s) alone, p-value < 0.05.