Table 3.
Inter-rater agreement on correct answers, in total and by group of evaluators
| Group | SD | SD + DD |
|---|---|---|
| AC1 (95% CI) | AC1 (95% CI) | |
| Dermatologists | 0.51 (0.27, 0.75) | 0.66 (0.45, 0.87) |
| Residents | 0.46 (0.20, 0.72) | 0.57 (0.29, 0.84) |
| Board-certified dermatologists | 0.64 (0.39, 0.90) | 0.74 (0.53, 0.94) |
| Experts in nail diseases | 0.80 (0.62, 0.98) | 0.92 (0.83, 1.00) |
| Between groups | 0.54 (0.21, 0.87) | 0.69 (0.42, 0.95) |
| AI | 0.63 (0.34, 0.92) | 0.66 (0.39, 0.94) |
| Dermatologists versus AI | −0.08 (−0.58, 0.43) | −0.27 (−0.77, 0.22) |
AI artificial intelligence algorithms, DD differential diagnosis, SD suspected diagnosis.