Table 3. Index for patient-based analysis without AI and with AI of five nonspecialists.
| Without AI | With AI | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Rater 6 | Rater 7 | Rater 8 | Rater 9 | Rater 10 | Rater 6 | Rater 7 | Rater 8 | Rater 9 | Rater 10 | |
| TP | 130 | 130 | 132 | 131 | 131 | 130 | 132 | 133 | 132 | 132 |
| FP | 5 | 2 | 2 | 0 | 10 | 5 | 2 | 1 | 0 | 7 |
| FN | 5 | 5 | 3 | 4 | 4 | 5 | 3 | 2 | 3 | 3 |
| TN | 191 | 194 | 194 | 196 | 186 | 191 | 194 | 195 | 196 | 189 |
| Sensitivity* | 0.96 | 0.96 | 0.98 | 0.97 | 0.97 | 0.96 | 0.98 | 0.99 | 0.98 | 0.98 |
| Specificity | 0.97 | 0.99 | 0.99 | 1.00 | 0.95 | 0.97 | 0.99 | 0.99 | 1.00 | 0.96 |
| Accuracy | 0.97 | 0.98 | 0.98 | 0.99 | 0.96 | 0.97 | 0.98 | 0.99 | 0.99 | 0.97 |
| Precision | 0.96 | 0.98 | 0.99 | 1.00 | 0.93 | 0.96 | 0.99 | 0.99 | 1.00 | 0.95 |
| F** | 0.96 | 0.97 | 0.98 | 0.98 | 0.95 | 0.96 | 0.98 | 0.99 | 0.99 | 0.96 |
| MCC | 0.94 | 0.96 | 0.97 | 0.98 | 0.91 | 0.94 | 0.97 | 0.98 | 0.98 | 0.94 |
Raters 6 to 10 are nonspecialists.
AI: artificial intelligence, F: F-measure, FN: false negative, FP: false positive, MCC: Matthews correlation coefficient, TN: true negative, TP: true positive.
*p-value = 0.034, **p-value = 0.049.