Table 3.
Inter‐rater agreement and agreement between each rater and Ariol automated quantitative ER, PR scores for cores in the virtual TMA
| Marker | Comparison | N | % Pos. | Continuous automated score | Dichotomous automated score | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| AUC (95%CI) | Observed agreement | Kappa (95%CI) | Se (%) | Sp (%) | PPV (%) | NPV (%) | ||||
| ER | Rater 1 vs rater 2 | 615 | 76.3 | n/a | 96.7 | 0.91 (0.83, 0.99) | 98.3 | 91.8 | 97.5 | 94.4 |
| Ariol vs rater 1 | 587 | 75.0 | 0.97 (0.95, 0.98) | 90.1 | 0.76 (0.68, 0.84) | 89.5 | 91.8 | 97.0 | 74.6 | |
| Ariol vs rater 2 | 636 | 76.4 | 0.96 (0.95, 0.98) | 90.1 | 0.75 (0.67, 0.83) | 88.9 | 94.0 | 98.0 | 72.3 | |
| PR | Rater 1 vs rater 2 | 655 | 67.0 | n/a | 96.8 | 0.93 (0.85, 1.00) | 97.5 | 95.4 | 97.7 | 94.9 |
| Ariol vs rater 1 | 624 | 67.3 | 0.93 (0.91, 0.95) | 83.8 | 0.65 (0.57, 0.73) | 82.9 | 85.8 | 92.3 | 70.9 | |
| Ariol vs rater 2 | 634 | 66.6 | 0.93 (0.91, 0.95) | 84.4 | 0.66 (0.59, 0.74) | 83.6 | 85.8 | 92.2 | 72.5 | |
Raters scores are dichotomous (positive/negative), and Ariol automated scores are considered as continuous and dichotomous.
% Pos., % positive cores for reference rater; Se, sensitivity; Sp, Specificity; PPV, positive predictive value; NPV, negative predictive value.