Table 3.
Accuracy of automatically extracted imaging findings compared to manual gold standard review (validation sample)
| Imaging | Finding | Prevalence (%) | Precision (95 % CI) | Recall (95 % CI) | F-measure |
|---|---|---|---|---|---|
| Screening mammography n = 197 | Positive BI-RADSa | 16/197 | 1.0 | 1.0 (0.8, 1.0) | 1.0 |
| (8.1) | (0.8, 1.0) | ||||
| Calcification | 16/197 | 0.93 | 0.9 | 0.9 | |
| (8.1) | (0.7, 1.0) | (0.6, 1.0) | |||
| Mass | 9/197 | 1.0 | 0.7 | 0.8 | |
| (4.6) | (0.5, 1.0) | (0.3, 0.9) | |||
| Implants | 1/197 | 0.50 | 1.0 | 0.7 | |
| (0.5) | (0.0, 1.0) | (0.0, 1.0) | |||
| Architectural distortion | 0/197 | – | – | – | |
| (0.0) | |||||
| Asymmetry | 7/196 | 0.9 | 1.0 | 0.9 | |
| (3.6) | (0.5, 1.0) | (0.6, 1.0) | |||
| Diagnostic mammography n = 200 | Positive BI-RADSa | 75/198 | 1.0 | 1.0 | 1.0 |
| (37.9) | (0.9, 1.0) | (0.9, 1.0) | |||
| Calcification | 64/200 | 1.0 | 1.0 | 1.0 | |
| (32.0) | (0.9, 1.0) | (0.9, 1.0) | |||
| Mass | 49/200 | 0.9 | 0.9 | 0.9 | |
| (24.5) | (0.8, 1.0) | (0.8, 1.0) | |||
| Implants | 7/200 | 1.0 | 1.0 | 1.0 | |
| (3.5) | (0.6, 1.0) | (0.6, 1.0) | |||
| Architectural distortion | 18/200 | 0.9 | 0.9 | 0.9 | |
| (9.0) | (0.6, 1.0) | (0.6, 1.0) | |||
| Asymmetry | 25/200 | 0.6 | 0.8 | 0.7 | |
| (12.5) | (0.4, 0.8) | (0.6, 0.9) | |||
| Digital tomosynthesis n = 200 | Positive BI-RADSa | 24/200 | 1.0 | 1.0 | 1.0 |
| (12.0) | (0.9, 1.0) | (0.9, 1.0) | |||
| Calcification | 16/200 | 0.9 | 0.9 | 0.9 | |
| (8.0) | (0.6, 1.0) | (0.7, 1.0) | |||
| Mass | 30/200 | 1.0 | 1.0 | 1.0 | |
| (15.0) | (0.8, 1.0) | (0 .8, 1.0) | |||
| Implants | 3/200 | 1.0 | 1.0 | 1.0 | |
| (1.5) | (0.3, 1.0) | (0.3, 1.0) | |||
| Architectural distortion | 11/200 | 0.8 | 0.8 | 0.8 | |
| (5.5) | (0.5,1.0) | (0.5, 1.0) | |||
| Asymmetry | 10/200 | 0.6 | 1.0 | 0.8 | |
| (5.0) | (0.4, 0.8) | (0.7, 1.0) | |||
| Breast magnetic resonance imaging n = 145 | Positive BI-RADSa | 31/132 | 1.0 | 0.9 | 1.0 |
| (23.5) | (0.9, 1.0) | (0.8, 1.0) | |||
| Mass | 35/141 | 0.9 | 0.9 | 0.9 | |
| (24.8) | (0.7, 1.0) | (0.7, 1.0) | |||
| Cysts | 31/141 | 0.9 | 1.0 | 0.9 | |
| (22.0) | (0.7, 1.0) | (0.9, 1.0) | |||
| Implants | 12/141 | 0.9 | 1.0 | 1.0 | |
| (8.5) | (0.6, 1.0) | (0.7, 1.0) | |||
| NME | 18/141 | 0.8 | 1.0 | 0.9 | |
| (12.8) | (0.6, 0.9) | (0.8, 1.0) | |||
| Focus | 29/141 | 0.9 | 1.0 | 1.0 | |
| (20.6) | (0.8, 1.0) | (0.9, 1.0) | |||
| Breast ultrasound n = 199 | Positive BI-RADSa | 69/197 | 1.0 | 1.0 | 1.0 |
| (35.0) | (0.9, 1.0) | (0.9, 1.0) | |||
| Mass | 50/198 | 0.8 | 1.0 | 0.9 | |
| (25.3) | (0.6, 0.8) | (0.9, 1.0) | |||
| Cysts | 49/199 | 0.9 | 0.9 | 0.9 | |
| (24.6) | (0.8, 1.0) | (0.8, 1.0) | |||
| Architectural distortion | 0/199 | – | – | – | |
| (0.0) |
aA positive final assessment is defined as BI-RADS categories 0, 3, 4, or 5 on screening mammogram and BI-RADS categories 4 or 5 on diagnostic workup