Table 7.
Interrater Agreement of the TrACE Analytic Annotation of the MEWS Corpus
| Interrater correlation Mean | Interrater correlation Median | Weighted Cohen’s Kappa Mean | Weighted Cohen’s Kappa Median | |
|---|---|---|---|---|
| Language quality | 0.75 | 0.77 | 0.72 | 0.74 |
| MEWS 1 (AD) | 0.73 | 0.73 | 0.71 | 0.71 |
| MEWS 2 (TE) | 0.75 | 0.76 | 0.72 | 0.73 |
| Organization | 0.72 | 0.73 | 0.72 | 0.73 |
| MEWS 1 (AD) | 0.68 | 0.70 | 0.68 | 0.70 |
| MEWS 2 (TE) | 0.76 | 0.76 | 0.77 | 0.76 |
| Content | 0.64 | 0.64 | 0.61 | 0.61 |
| MEWS 1 (AD) | 0.68 | 0.70 | 0.66 | 0.67 |
| MEWS 2 (TE) | 0.56 | 0.57 | 0.52 | 0.52 |