Table 3.
Agreement (observed agreement, kappa statistic) and discriminatory accuracy (AUC) parameters for the automated and visual scores according to quality control status (satisfactory, N = 950 and suboptimal, N = 396) overall and among the 15 TMAs in the training set
| TMA Name | Satisfactory QC | Suboptimal QC | ||||||
|---|---|---|---|---|---|---|---|---|
| N | AUC (95% CI) | Observed agreement (95% CI) | Kappa (95% CI) | N | AUC (95% CI) | Observed agreement (95% CI) | Kappa (95% CI) | |
| TMA 1 | 65 | 82 (71, 92) | 78 (67, 88) | 0.31 (0.20, 0.43) | 37 | 79 (64, 94) | 84 (68, 94) | 0.42 (0.25, 0.58) |
| TMA 2 | 63 | 93 (85, 100) | 91 (82, 97) | 0.78 (0.66, 0.87) | 26 | 88 (74, 100) | 86 (65, 96) | 0.61 (0.41, 0.79) |
| TMA 3 | 73 | 92 (86, 98) | 87 (76, 93) | 0.61 (0.50, 0.73) | 47 | 82 (69, 95) | 79 (64, 89) | 0.28 (0.17, 0.44) |
| TMA 4 | 98 | 86 (79, 93) | 90 (83, 96) | 0.69 (0.59, 0.78) | 56 | 80 (67, 93) | 82 (70, 91) | 0.34 (0.25, 0.81) |
| TMA 5 | 76 | 91 (84, 97) | 90 (80, 95) | 0.70 (0.60, 0.81) | 13 | 69 (37, 100) | 89 (64, 100) | 0.51 (0.60, 0.81) |
| TMA 6 | 61 | 89 (77, 100) | 85 (74, 93) | 0.49 (0.36, 0.62) | 13 | 58 (14, 100) | 77 (46, 95) | 0.19 (0.10, 0.54) |
| TMA 7 | 84 | 88 (81, 95) | 91 (82, 96) | 0.69 (0.58, 0.79) | 17 | 79 (48, 100) | 88 (64, 99) | 0.57 (0.33, 0.81) |
| TMA 8 | 87 | 89 (81, 97) | 80 (71, 88) | 0.38 (0.28, 0.49) | 17 | 99 (95, 100) | 78 (50, 93) | 0.31 (0.10, 0.56) |
| TMA 9 | 44 | 100 (99, 100) | 95 (85, 99) | 0.85 (0.70, 0.93) | 26 | 96 (91, 100) | 95 (80, 100) | 0.79 (0.61, 0.93) |
| TMA 10 | 48 | 98 (95, 100) | 96 (86, 99) | 0.88 (0.75, 0.95) | 22 | 82 (63, 100) | 95 (77, 100) | 0.82 (0.60, 0.95) |
| TMA 11 | 48 | 92 (84, 99) | 93 (83, 99) | 0.81 (0.67, 0.91) | 21 | 91 (79, 100) | 85 (64, 97) | 0.54 (0.30, 0.74) |
| TMA 12 | 53 | 93 (86, 100) | 89 (77, 96) | 0.55 (0.40, 0.68) | 33 | 83 (65, 100) | 87 (72, 97) | 0.30 (0.16, 0.48) |
| TMA 13 | 45 | 86 (73, 99) | 89 (76, 96) | 0.68 (0.51, 0.80) | 27 | 97 (91, 100) | 96 (81, 100) | 0.85 (0.66, 0.95) |
| TMA 14 | 55 | 89 (78, 100) | 91 (80, 97) | 0.75 (0.61, 0.85) | 20 | 69 (44, 93) | 76 (51, 91) | 0.27 (0.11, 0.54) |
| TMA 15 | 50 | 91 (82, 100) | 90 (78, 97) | 0.71 (0.58, 0.84) | 21 | 49 (20, 78) | 78 (53, 92) | 0.03 (0.01, 0.23) |
| Overall | 950 | 86 (84, 89) | 89 (86, 91) | 0.68 (0.65, 0.71) | 396 | 82 (78, 86) | 85 (81, 88) | 0.51 (0.46, 0.56) |
Suboptimal QC were cores which did not meet the criteria to be considered satisfactory but which were sufficiently suitable for scoring, eg, cores with few tumour cells (50–500 cells), partially folded cores, staining artefact or suboptimal/poor fixation. N, Represents the number of cores on each TMA that have been classified as being either of satisfactory or suboptimal QC.