Table B3.

Computing Subject-Level Agreement Coefficients ${\kappa }_{G|i}^{☆}$ Between Raters 1 and 3 Based on the $\mathrm{AC}{}_1$ Coefficient, the Differences $d_i$ From Raters 1 and 2 Extent of Agreement.

		$(1-{\pi }_k)[{\epsilon }_{\mathrm{ik}}^{(1)}+{\epsilon }_{\mathrm{ik}}^{(3)}]/[2(q-1)]$
Subject $(i)$	$p_{a|i}$	$k=1$	$k=2$	$k=3$	$p_{e|i}$	${\kappa }_{G|i}^{(1,3)}$	${\kappa }_{G|i}^{☆(1,3)}$	$d_i$ a
1	0	0.0917	0.2	0	0.2917	–0.3616	–0.3809	–1.42158
2	1	0.1833	0	0	0.1833	1	1.0610	0.02032
3	1	0.1833	0	0	0.1833	1	1.0610	0.02032
4	1	0.1833	0	0	0.1833	1	1.0610	0.02032
5	1	0	0	0.4167	0.4167	1	0.8879	–0.04559
6	1	0.1833	0	0	0.1833	1	1.0610	0.02032
7	1	0.1833	0	0	0.1833	1	1.0610	0.02032
8	1	0.1833	0	0	0.1833	1	1.0610	0.02032
9	1	0.1833	0	0	0.1833	1	1.0610	0.02032
10	1	0	0.4	0	0.4	1	0.9003	–0.02499
11	1	0.1833	0	0	0.1833	1	1.0610	0.02032
12	0	0.0917	0.2	0	0.2917	–0.3616	–0.38092	0.05122
13	1	0	0.4	0	0.4	1	0.9003	–0.02499
14	1	0	0	0.4167	0.4167	1	0.8879	–0.04559
15	0	0.0917	0	0.2083	0.3	–0.3616	–0.3871	–0.01263
	$p_a$				$p_e$	${\hat{\kappa }}_G$	${\hat{\kappa }}_G$
Mean	0.8				0.266	0.728	0.728	$v(\overline{d})$
Variance of mean difference $\mathrm{var}(d_i)/n$								0.009090

^a$d_i={\kappa }_{G|i}^{(1,3)}-{\kappa }_{G|i}^{(1,2)}$, where ${\kappa }_{G|i}^{(1,2)}$ is obtained from Table B2.