Table 7.

Rater 2 intra-rater agreement: Cohen’s Kappa with 95% confidence intervals and overall criteria percentage agreement (n = 20 except *n = 19, **n = 18).

Test	Criteria	Left-hand side	Kappa (95% CI)	Right-hand side	Kappa (95% CI)	Criteria agreement (%)
1	A	.83	(.50-1.00)	1.00	(1.00-1.00)	97.5
	B	.89	(.69-1.00)	.76	(.45-1.00)	92.5
	C	.90	(.71-1.00)	.90	(.71-1.00)	95.0
	D	.79	(.52-1.00)	.89	(.69-1.00)	92.5
	E	.90	(.70-1.00)	.60	(.26-.94)	87.5
2	A	.86	(.59-1.00)	.83	(.50-1.00)	95.0
	B	.69	(.38-1.00)	.31	(-.08-.71)	75.0
	C	.70	(.39-1.00)	.68	(.35-1.00)	85.0
	D	N/A	N/A	N/A	N/A	100.0
	E	N/A	N/A	.00	N/A	97.5
3	A		1.00 (1.00-1.00)			100.0
	B		N/A N/A			100.0
	C	N/A	N/A	N/A	N/A	N/A
4	A		1.00 (1.00-1.00)			100.0
	B	N/A	N/A	N/A	N/A	N/A
	C	1.00	(1.00-1.00)	1.00	(1.00-1.00)	1.00
	D	.60	(.28-.92)	.60	(.28-.92)	.60
5	A	.64 *	(.00-1.00)	.77 *	(.35-1.00)	94.7
	B	.83 *	(.50-1.00)	.44 **	(-.21-1.00)	91.9
	C	N/A	N/A	N/A	N/A	100.0
	D	-.08 *	(-.23-.08)	.83*	(.50-1.00)	89.5
7	A	.69	(.29-1.00)	.53	(.14-0.92)	85.0
	B	.00	N/A	.00	N/A	90.0
	C	.79	(.53-1.00)	.67	(.34-1.00)	87.5
	D		0.77 (0.35-1.00)			95.0
	E	1.00	(1.00-1.00)	.69	(.31-1.00)	95.0
8	A		-0.053 (-0.18-0.07)			90.0
	B	1.00	(1.00-1.00)	1.00	(1.00-1.00)	1.00
	C	N/A	N/A	N/A	N/A	N/A
	D	.00	N/A	.00	N/A	.00
9	A	.89	(.67-1.00)	.88	(.64-1.00)	95.0
	B	.88	(.64-1.00)	.88	(.64-1.00)	95.0
	C	.77	(.35-1.00)	.77	(.35-1.00)	95.0
	D	.83	(.50-1.00)	.83	(.50-1.00)	95.0
	E	1.00	(1.00-1.00)	.64	(.01-1.00)	97.5
10	A	.89 *	(.69-1.00)	.90	(.71-1.00)	94.9
	B	.64	(.01-1.00)	N/A	N/A	97.5
	C	.64	(.01-1.00)	.64	(.01-1.00)	95.0
	D	.64	(.01-1.00)	.00	N/A	95.0
	E	1.00	(1.00-1.00)	N/A	N/A	100.0