Table 1.

Coverage Rates of 95% Confidence Intervals When Prevalence Rate is 0.75 and the Number of Categories $q=2$.

		Sample size (n)
$\kappa$ a	Agreement coefficient	10	20	30	40	50	80	100
0.50	Cohen’s kappa	0.864	0.925	0.934	0.943	0.937	0.945	0.943
	Scott’s pi	0.855	0.933	0.936	0.945	0.938	0.946	0.943
	Gwet’s $\mathrm{AC}{}_1$	0.926	0.946	0.945	0.950	0.948	0.951	0.949
	Brennan–Prediger	0.859	0.936	0.928	0.945	0.946	0.948	0.945
	Krippendorff’s alpha	0.872	0.932	0.935	0.944	0.938	0.946	0.942
0.65	Cohen’s kappa	0.796	0.924	0.934	0.941	0.942	0.949	0.942
	Scott’s pi	0.793	0.933	0.935	0.942	0.943	0.949	0.943
	Gwet’s $\mathrm{AC}{}_1$	0.852	0.947	0.948	0.950	0.951	0.954	0.949
	Brennan–Prediger	0.809	0.940	0.932	0.948	0.955	0.951	0.943
	Krippendorff’s alpha	0.808	0.933	0.934	0.942	0.942	0.949	0.942
0.85	Cohen’s kappa	0.521	0.772	0.886	0.924	0.937	0.945	0.946
	Scott’s pi	0.523	0.774	0.887	0.924	0.937	0.946	0.946
	Gwet’s $\mathrm{AC}{}_1$	0.555	0.781	0.895	0.932	0.946	0.952	0.952
	Brennan–Prediger	0.547	0.780	0.887	0.924	0.941	0.952	0.952
	Krippendorff’s alpha	0.542	0.775	0.886	0.924	0.937	0.946	0.947

^a$\kappa$ = hypothesized agreement level, representing the nominal percent of subjects on which the raters agree for cause, as opposed to by chance.