Table 3.
Comparison of LNM predictions between radiologists with and without AI assistance in the validation cohort
| Radiologists | Accuracy | P value | Sensitivity | P value | Specificity | P value |
|---|---|---|---|---|---|---|
| Senior 1 | 0.03 | 0.86 | < 0.001 | |||
| Without AI |
60 (42/70) [48, 72] |
43(14/32) [27, 62] |
74 (28/38) [57, 86] |
|||
| With AI |
67 (47/70) [55, 78] |
50 (16/32) [32, 68] |
82 (31/38) [65, 92] |
|||
| Senior 2 | 0.09 | 0.04 | 0.09 | |||
| Without AI |
60(42/70) [48, 72] |
59 (19/32) [41, 76] |
61 (23/38) [43, 76] |
|||
| With AI |
61(43/70) [49, 73] |
53 (17/32) [35, 70] |
68 (26/38) [51, 82] |
|||
| Intermediate 1 | 0.82 | 0.31 | 0.78 | |||
| Without AI |
54 (38/70) [42, 66] |
47 (15/32) [30,65] |
61 (23/38) [43, 76] |
|||
| With AI |
50 (35/70) [38, 62] |
71 (23/32) [53, 86] |
32 (12/38) [18, 49] |
|||
| Intermediate 2 | 0.19 | 0.05 | > 0.99 | |||
| Without AI |
59 (41/70) [46, 70] |
69 (22/32) [50, 83] |
50 (19/38) [34, 66] |
|||
| With AI |
59 (41/70) [46, 70] |
69 (22/32) [50, 83] |
50 (19/38) [34, 66] |
|||
| Junior 1 | 0.10 | < 0.001 | 0.47 | |||
| Without AI |
56 (39/70) [43, 68] |
81 (26/32) [63, 92] |
34 (13/38) [20, 51] |
|||
| With AI |
64 (45/70) [52, 75] |
78 (25/32) [60, 90] |
53 (20/38) [36, 69] |
|||
| Junior 2 | 0.55 | 0.04 | 0.31 | |||
| Without AI |
53 (37/70) [41, 65] |
72 (23/32) [53, 86] |
37 (14/38) [22, 54] |
|||
| With AI |
56 (39/70) [43, 68] |
69 (22/32) [50, 83] |
45 (17/38) [30, 62] |
|||
| Average | < 0.001 | < 0.001 | 0.28 | |||
| Without AI |
57 (239/420) [52, 62] |
62 (119/192) [55, 69] |
53 (120/228) [46, 59] |
|||
| With AI |
60 (250/420) [55, 64] |
65 (125/192) [58, 72] |
55 (125/228) [48, 61] |
Unless otherwise specified, data are percentages with the number of patients in parentheses and 95% confidence intervals in brackets