Table 2. Diagnostic performance for MDCS and radiologists with and without MDCS assistance in pooled external and prospective testing sets.
| Variables | Pooled external testing set | Fleiss’s Kappa |
Prospective testing set | Fleiss’s Kappa |
||||
| Sensitivity (95% CI) |
Specificity (95% CI) |
Accuracy (95% CI) |
Sensitivity (95% CI) |
Specificity (95% CI) |
Accuracy (95% CI) |
|||
| MDCS, multiprocess detection and classification system; R, radiologist; 95% CI, 95% confidence interval; *, indicates a statistically significant difference between MDCS and radiologists without MDCS assistance (*, P<0.05; **, P<0.001); #, indicates a statistically significant difference between radiologists without and with MDCS assistance (#, P<0.05; ##, P<0.001). | ||||||||
| MDCS | 0.954 (0.864−0.988) |
0.655 (0.457−0.841) |
0.852 (0.765−0.917) |
0.947 (0.845−0.986) |
0.714 (0.511−0.860) |
0.871 (0.780−0.934) |
||
| Radiologists without MDCS assistance | ||||||||
| Junior R1 | 0.908 (0.803−0.962)* |
0.400 (0.232−0.592)** |
0.747 (0.648−0.831) |
0.903 (0.804−0.957) |
0.310 (0.160−0.510)** |
0.733 (0.635−0.816)** |
||
| Junior R2 | 0.984 (0.906−1.000) |
0.333 (0.179−0.529)** |
0.779 (0.682−0.858) |
0.972 (0.894−0.995) |
0.276 (0.134−0.475)** |
0.772 (0.678−0.850)** |
||
| Junior R3 | 0.939 (0.844−0.980) |
0.552 (0.360−0.730) |
0.821 (0.729−0.892) |
0.931 (0.839−0.974) |
0.448 (0.270−0.640) |
0.792 (0.670−0.866) |
||
| Average junior | 0.943 (0.849−1.000) |
0.428 (0.150−0.707) |
0.782 (0.690−0.875) |
0.188 | 0.936 (0.849−1.000) |
0.345 (0.118−0.571) |
0.766 (0.691−0.840) |
0.123 |
| Senior R4 | 0.924 (0.825−0.972) |
0.621 (0.424−0.787) |
0.832 (0.741−0.901) |
0.889 (0.787−0.947) |
0.621 (0.424−0.787) |
0.812 (0.722−0.883) |
||
| Senior R5 | 0.924 (0.825−0.972) |
0.759 (0.561−0.890) |
0.874 (0.790−0.933) |
0.903 (0.804−0.957)* |
0.552 (0.360−0.730) |
0.802 (0.711−0.875) |
||
| Senior R6 | 0.939 (0.778−0.925) |
0.690 (0.490−0.840) |
0.863 (0.778−0.925) |
0.931 (0.839−0.974) |
0.414 (0.241−0.609)** |
0.782 (0.689−0.858) |
||
| Average senior | 0.929 (0.907−0.951) |
0.700 (0.519−0.861) |
0.856 (0.802−0.910) |
0.525 | 0.908 (0.855−0.961) |
0.529 (0.267−0.791) |
0.798 (0.761−0.837) |
0.328 |
| Radiologists with MDCS assistance | ||||||||
| Junior R1 | 0.939 (0.842−0.980) |
0.533 (0.346−0.712)# |
0.811 (0.717−0.884)## |
0.944 (0.857−0.982) |
0.379 (0.213−0.576) |
0.782 (0.689−0.858)## |
||
| Junior R2 | 0.984 (0.906−1.000) |
0.400 (0.232−0.592)# |
0.800 (0.705−0.875) |
0.958 (0.875−0.989) |
0.379 (0.213−0.576) |
0.792 (0.670−0.866) |
||
| Junior R3 | 0.955 (0.864−0.988) |
0.552 (0.360−0.730) |
0.832 (0.741−0.901) |
0.958 (0.875−0.989) |
0.483 (0.299−0.671) |
0.822 (0.733−0.891) |
||
| Average junior | 0.959 (0.903−1.000) |
0.495 (0.290−0.701) |
0.814 (0.774−0.855)# |
0.353 | 0.953 (0.933−0.973) |
0.414 (0.265−0.563) |
0.799 (0.747−0.850)# |
0.231 |
| Senior R4 | 0.955 (0.864−0.988) |
0.655 (0.457−0.841) |
0.863 (0.777−0.925) |
0.917 (0.821−0.966) |
0.655 (0.457−0.814) |
0.842 (0.756−0.907) |
||
| Senior R5 | 0.955 (0.864−0.988) |
0.724 (0.525−0.866) |
0.884 (0.802−0.941) |
0.931 (0.839−0.974) |
0.586 (0.391−0.759) |
0.832 (0.744−0.899) |
||
| Senior R6 | 0.970 (0.885−0.995) |
0.759 (0.561−0.890) |
0.905 (0.828−0.956) |
0.958 (0.875−0.989) |
0.448 (0.270−0.640) |
0.821 (0.722−0.883) |
||
| Average senior | 0.960 (0.938−0.982)## |
0.713 (0.581−0.844) |
0.884 (0.832−0.936)# |
0.659 | 0.935 (0.884−0.987) |
0.563 (0.301−0.825) |
0.832 (0.806−0.857)# |
0.438 |