Table 3.
Reader Groups | Area Under the Receiver Operating Characteristic Curve | Area Under the Alternative Free- response Receiver Operating Characteristic Curve | Sensitivity | Specificity | True Detection Rate |
---|---|---|---|---|---|
Session 1 (physician reading only) | |||||
Nonradiology physicians | 0.746 (0.552–0.940) | 0.664 (0.466–0.861) | 0.723 (0.677–0.765) | 0.670 (0.627–0.711) | 0.582 (0.543–0.620) |
P valuea | .0230 | .0088 | |||
Board-certified radiologists | 0.946 (0.911–0.982) | 0.900 (0.856–0.943) | 0.906 (0.874–0.932) | 0.948 (0.925–0.966) | 0.797 (0.764–0.827) |
P valuea | .0082 | .0003 | |||
Thoracic radiologists | 0.971 (0.948–0.993) | 0.925 (0.890–0.959) | 0.952 (0.927–0.970) | 0.930 (0.904–0.951) | 0.870 (0.842–0.894) |
P valuea | 0.0218 | 0.0001 | |||
Session 2 (physician reading with DLAD assistance) | |||||
Nonradiology physicians | 0.850 (0.694–1.005) | 0.781 (0.598–0.965) | 0.848 (0.810–0.881) | 0.800 (0.762–0.834) | 0.724 (0.688–0.758) |
P valueb | .0610 | .0236 | <.0001 | <.0001 | <.0001 |
Board-certified radiologists | 0.961 (0.933–0.988) | 0.924 (0.891–0.957) | 0.930 (0.901–0.953) | 0.954 (0.932–0.971) | 0.849 (0.819–0.875) |
P valueb | .0606 | .0353 | .0075 | .0833 | <.0001 |
Thoracic radiologists | 0.977 (0.957–0.997) | 0.942 (0.913–0.971) | 0.964 (0.941–0.980) | 0.936 (0.911–0.956) | 0.897 (0.871–0.919) |
P valueb | .1623 | .0036 | .0587 | .2568 | .0004 |
aComparison of performance with deep learning–based automatic detection (DLAD) algorithm.
bComparison of performance with session 1.