Table 2.

Coverage Rates of 95% Confidence Intervals When Prevalence Rate Is 0.75, and the Number of Categories $q=3$.

		Sample size (n)
$\kappa$ a	Agreement coefficient	10	20	30	40	50	80	100
0.50	Cohen’s kappa	0.899	0.921	0.927	0.939	0.941	0.943	0.947
	Scott’s pi	0.907	0.925	0.930	0.940	0.943	0.943	0.947
	Gwet’s $\mathrm{AC}{}_1$	0.958	0.946	0.943	0.947	0.949	0.949	0.951
	Brennan–Prediger	0.849	0.935	0.925	0.944	0.951	0.945	0.950
	Krippendorff’s alpha	0.920	0.928	0.931	0.940	0.942	0.943	0.947
0.65	Cohen’s kappa	0.857	0.930	0.935	0.939	0.940	0.946	0.944
	Scott’s pi	0.863	0.935	0.938	0.942	0.941	0.947	0.945
	Gwet’s $\mathrm{AC}{}_1$	0.916	0.962	0.951	0.952	0.950	0.950	0.950
	Brennan–Prediger	0.775	0.933	0.936	0.950	0.954	0.948	0.946
	Krippendorff’s alpha	0.876	0.938	0.938	0.941	0.940	0.944	0.943
0.85	Cohen’s kappa	0.605	0.859	0.930	0.951	0.956	0.949	0.949
	Scott’s pi	0.607	0.861	0.933	0.952	0.957	0.950	0.950
	Gwet’s $\mathrm{AC}{}_1$	0.647	0.875	0.942	0.959	0.962	0.956	0.957
	Brennan–Prediger	0.500	0.744	0.856	0.915	0.936	0.951	0.955
	Krippendorff’s alpha	0.631	0.863	0.936	0.953	0.957	0.950	0.949

^a$\kappa$ = hypothesized agreement level, representing the nominal percent of subjects on which the raters agree for cause, as opposed to by chance.