. Author manuscript; available in PMC: 2018 Feb 1.

Published in final edited form as: Proc Annu Hawaii Int Conf Syst Sci. 2018 Jan 3;2018:1416–1422.

Table 3.

Inter-rater reliability

	AB	AC	AD	BC	BD	CD
N-comparisons	42	35	41	34	40	34
IRR	0.95	0.94	0.85	0.97	0.92	1.00
Same condition comparison average agreement			0.976
Different condition comparison average agreement			0.923
p-value for difference between mean % agreement			0.414