Table 3:

Inter-rater reliability of PEMAT scores: item agreement

Item	Rater 1	Rater 2	n (%)	Perfect Agreement	Kappa (Standard Error)	95% Confidence Interval
1	0	0	9(17)	47(88.7%)	0.73(0.10)	(0.52,0.93)
	0	1	1(1.9)
	1	0	5(9.4)
	1	1	38(71.7)
3	0	0	43(81.1)	53(100.0%)	1.00(0.00)	(1.00,1.00)
	1	1	10(18.9)
4	0	0	24(45.3)	47(88.7%)	0.80(0.08)	(0.65,0.95)
	0	1	6(11.3)
	1	1	23(43.4)
5	0	0	53(100)	53(100.0%)	1.00(0.00)	(1.00,1.00)
8	0	0	30(56.6)	43(81.1%)	0.64(0.10)	(0.44,0.84)
	0	1	8(15.1)
	1	0	2(3.8)
	1	1	13(24.5)
9	0	0	23(43.4)	40(75.5%)	0.57(0.10)	(0.37,0.77)
	0	1	11(20.8)
	1	0	2(3.8)
	1	1	17(32.1)
10	0	0	6(11.3)	36(67.9%)	0.36(0.11)	(0.14,0.57)
	0	1	16(30.2)
	1	0	1(1.9)
	1	1	30(56.6)
11	0	0	41(77.4)	45(84.9%)	0.57(0.14)	(0.29,0.84)
	0	1	3(5.7)
	1	0	5(9.4)
	1	1	4(7.5)
12	NA	NA	53(100)	53(100.0%)	1.00(0.00)	(1.00,1.00)
13	0	0	1(1.9)	39(73.6%)	0.48(0.11)	(0.26,0.71)
	1	0	6(11.3)
	1	1	30(56.6)
	1	NA	2(3.8)
	NA	1	6(11.3)
	NA	NA	8(15.1)
14	0	0	4(7.5)	42(79.2%)	0.65(0.09)	(0.47,0.83)
	1	0	6(11.3)
	1	1	28(52.8)
	1	NA	3(5.7)
	NA	0	1(1.9)
	NA	1	1(1.9)
	NA	NA	10(18.9)
18	0	0	5(9.4)	44(83.0%)	0.75(0.07)	(0.60,0.89)
	0	1	4(7.5)
	1	0	3(5.7)
	1	1	17(32.1)
	NA	0	1(1.9)
	NA	1	1(1.9)
	NA	NA	22(41.5)
19	NA	1	1(1.9)	52(98.1%)	0.79(0.20)	(0.40,1.00)
	NA	NA	52(98.1)
20	0	0	23(43.4)	35(66.0%)	0.39(0.12)	(0.15,0.63)
	0	1	6(11.3)
	1	0	12(22.6)
	1	1	12(22.6)
21	0	0	15(28.3)	32(60.4%)	0.35(0.10)	(0.15,0.55)
	0	1	2(3.8)
	1	0	19(35.8)
	1	1	17(32.1)
22	0	0	27(50.9)	40(75.5%)	0.55(0.11)	(0.35,0.76)
	0	1	11(20.8)
	1	0	2(3.8)
	1	1	13(24.5)
25	NA	NA	53(100)	53(100.0%)	1.00(0.00)	(1.00,1.00)