Table 3. Inter-rater reliability of 3CAT, WHOCAT and QUANT, measured with kappa, weighted kappa, adjusted weighted kappa, percentage agreement and/or the intraclass correlation coefficient mortality review of the contribution of healthcare-associated infections to death, 11 EU countries, April 2017–February 2018 (n = 291).
| 3CAT | n | Simple kappa (95% CI) | Weighted kappa (95% CI) | Weighted kappa, adjusted for hospitala (95% CI) | Agreement (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall | Definitely | Possibly | No | |||||||||
| Overall | 291 | 0.62 (0.54–0.70) | 0.68 (0.61–0.75) | 0.63 (0.55–0.71) | 76 | 84 | 73 | 65 | ||||
| BSI | 87 | 0.56 (0.41–0.72) | 0.60 (0.46–0.76) | 0.38 (0.20–0.56) | 76 | 81 | 73 | 44 | ||||
| Pneumonia | 113 | 0.66 (0.55–0.78) | 0.72 (0.62–0.82) | 0.82 (0.74–0.90) | 78 | 86 | 74 | 73 | ||||
| CDI | 71 | 0.49 (0.32–0.67) | 0.57 (0.41–0.73) | 0.55 (0.40–0.70) | 69 | 74 | 69 | 58 | ||||
| SSI | 20 | 0.87 (0.65–1.00) | 0.88 (0.70–1.00) | Not calculated | 95 | 100 | 89 | 0b | ||||
| WHOCAT | n | Simple kappa (95% CI) | Weighted kappa (95% CI) | Weighted kappa, adjusted for hospitala (95% CI) | Agreement (%) | |||||||
| Overall | Sole cause | Causal sequence | Contributory cause | No | ||||||||
| Overall | 288 | 0.58 (0.51–0.66) | 0.65 (0.58–0.73) | 0.75 (0.71–0.80) | 74 | 62 | 85 | 51 | 69 | |||
| BSI | 86 | 0.56 (0.41–0.71) | 0.60 (0.43–0.77) | 0.63 (0.55–0.71) | 75 | 70 | 86 | 53 | 46 | |||
| Pneumonia | 110 | 0.66 (0.54–0.78) | 0.72 (0.60–0.83) | 0.89 (0.83–0.96) | 80 | 0 a | 91 | 48 | 77 | |||
| CDI | 68 | 0.47 (0.30–0.64) | 0.52 (0.34–0.70) | 0.56 (0.46–0.66) | 65 | 36 | 76 | 53 | 62 | |||
| SSI | 17 | 0.56 (0.22–0.90) | 0.63 (0.29–0.97) | Not calculated | 72 | 83 | 80 | 0 a | 0b | |||
| QUANT | n | ICC (95% CI) for absolute agreement | ||||||||||
| Overall | 289 | 0.76 (0.71–0.81) | ||||||||||
| BSI | 87 | 0.75 (0.62–0.83) | ||||||||||
| Pneumonia | 111 | 0.85 (0.79–0.90) | ||||||||||
| CDI | 71 | 0.54 (0.35–0.69) | ||||||||||
| SSI | 20 | 0.71 (0.41–0.87) | ||||||||||
3CAT: three-category mortality review measure; CI: confidence interval; BSI: bloodstream infection; CDI: Clostridioides difficile infection; EU: European Union; ICC: intraclass correlation coefficient; QUANT: quantitative Likert scale from 0 to 10; SSI: surgical site infection; WHOCAT: World Health Organization death certification based measure.
a Excluding hospitals with fewer than six cases.
b Zero cases with agreement, one case in denominator; cases where one of the ratings was missing or ‘unknown’ were excluded.
Percentages are presented in italics.