Table 2.
Performance comparison of 36 doctors and the machine learning (ML) model in terms of sensitivity, specificity, and accuracy, with 95% CIs. The best performance of 1 clinician and our ML model in terms of accuracy is italicized.
| Doctor ID | Accuracy (95% CI) | Sensitivity (95% CI) | Specificity (95% CI) |
| 1 | 0.29 (0.12-0.46) | 0.42 (0.14-0.70) | 0.17 (0.00-0.40) |
| 2 | 0.54 (0.33-0.75) | 0.50 (0.21-0.80) | 0.58 (0.30-0.86) |
| 3 | 0.25 (0.08-0.42) | 0.33 (0.08-0.62) | 0.17 (0.00-0.40) |
| 4 | 0.33 (0.17-0.54) | 0.25 (0.00-0.50) | 0.42 (0.13-0.70) |
| 5 | 0.46 (0.29-0.67) | 0.50 (0.22-0.79) | 0.42 (0.14-0.73) |
| 6 | 0.42 (0.21-0.62) | 0.33 (0.08-0.64) | 0.50 (0.22-0.78) |
| 7 | 0.46 (0.29-0.67) | 0.50 (0.22-0.79) | 0.42 (0.14-0.73) |
| 8 | 0.62 (0.42-0.79) | 0.75 (0.50-1.0) | 0.50 (0.21-0.75) |
| 9 | 0.25 (0.08-0.42) | 0.33 (0.08-0.62) | 0.17 (0.00-0.40) |
| 10 | 0.62 (0.42-0.79) | 0.75 (0.50-1.0) | 0.50 (0.21-0.75) |
| 11 | 0.33 (0.17-0.54) | 0.25 (0.00-0.50) | 0.42 (0.13-0.70) |
| 12 | 0.42 (0.21-0.62) | 0.33 (0.08-0.64) | 0.50 (0.22-0.78) |
| 13 | 0.54 (0.33-0.75) | 0.50 (0.21-0.80) | 0.58 (0.30-0.86) |
| 14 | 0.42 (0.21-0.58) | 0.25 (0.00-0.50) | 0.58 (0.31-0.85) |
| 15 | 0.54 (0.33-0.75) | 0.83 (0.60-1.0) | 0.25 (0.00-0.50) |
| 16 | 0.50 (0.33-0.71) | 0.33 (0.08-0.62) | 0.67 (0.38-0.92) |
| 17 | 0.50 (0.33-0.71) | 0.33 (0.08-0.62) | 0.67 (0.38-0.92) |
| 18 | 0.62 (0.42-0.83) | 0.58 (0.30-0.88) | 0.67 (0.38-0.92) |
| 19 | 0.58 (0.38-0.79) | 0.50 (0.21-0.77) | 0.67 (0.40-0.92) |
| 20 | 0.62 (0.42-0.83) | 0.58 (0.30-0.88) | 0.67 (0.38-0.92) |
| 21 | 0.58 (0.38-0.79) | 0.50 (0.21-0.77) | 0.67 (0.40-0.92) |
| 22 | 0.50 (0.29-0.71) | 0.75 (0.50-1.0) | 0.25 (0.00-0.53) |
| 23 | 0.54 (0.33-0.75) | 0.42 (0.14-0.71) | 0.67 (0.38-0.91) |
| 24 | 0.50 (0.29-0.71) | 0.50 (0.18-0.78) | 0.50 (0.20-0.80) |
| 25 | 0.62 (0.42-0.83) | 0.50 (0.20-0.78) | 0.75 (0.50-1.0) |
| 26 | 0.54 (0.38-0.75) | 0.33 (0.08-0.62) | 0.75 (0.50-1.0) |
| 27 | 0.50 (0.29-0.71) | 0.50 (0.18-0.78) | 0.50 (0.20-0.80) |
| 28 | 0.46 (0.29-0.67) | 0.42 (0.12-0.70) | 0.50 (0.22-0.75) |
| 29 | 0.29 (0.12-0.46) | 0.42 (0.14-0.70) | 0.17 (0.00-0.40) |
| 30 | 0.62 (0.42-0.79) | 0.75 (0.50-1.0) | 0.50 (0.21-0.75) |
| 31 | 0.58 (0.38-0.79) | 0.42 (0.12-0.71) | 0.75 (0.50-1.0) |
| 32 | 0.58 (0.38-0.79) | 0.58 (0.29-0.86) | 0.58 (0.31-0.85) |
| 33 | 0.50 (0.29-0.71) | 0.33 (0.08-0.62) | 0.67 (0.38-0.92) |
| 34 | 0.50 (0.29-0.71) | 0.33 (0.08-0.62) | 0.67 (0.38-0.91) |
| 35 | 0.62 (0.42-0.83) | 0.58 (0.27-0.85) | 0.67 (0.42-0.92) |
| 36 | 0.71 (0.50-0.88) | 0.67 (0.36-0.92) | 0.75 (0.50-1.0) |
| ML model | 0.79 (0.62-0.92) | 0.75 (0.46-1.0) | 0.83 (0.58-1.0) |