Table 4.
Frequency table of classifier performance
| Classifier predictions | Agreement | Disagreement | χ2(1) | P | P (permutation) |
|---|---|---|---|---|---|
| ACC & PCC | 30 (26) | 10 (14) | 1.76 | 0.185 | 0.155 |
| ACC & PMC | 30 (24) | 10 (16) | 3.75 | 0.053 | 0.039 |
| ACC & FG | 27 (23.5) | 13 (16.5) | 1.26 | 0.261 | 0.206 |
| PCC & PMC | 32 (24.8) | 8 (15.2) | 5.50 | 0.019 | 0.021 |
| PCC & FG | 29 (24.2) | 11 (15.8) | 2.41 | 0.121 | 0.139 |
| PMC & FG | 23 (22.8) | 17 (17.2) | 0.00 | 0.949 | 0.978 |
Note. Pair-wise observed frequencies of classifier agreement. Expected frequencies for independence are displayed in parentheses. Bold indicates statistically significant classifier concordance from permutation testing.