Table 3:

Interrater (between reviewers) κ statistic by top choice groups

Reviewer	1	2	3
Overall
1	1.0	0.35	0.40
2	0.35	1.0	0.16
3	0.40	0.16	1.0
Top choice
1	1.0	0.47	0.21
2	0.47	1.0	0.028
3	0.21	0.028	1.0
Top 2 choices
1	1.0	0.37	0.37
2	0.37	1.0	0.27
3	0.37	0.27	1.0
Top 3 choices
1	1.0	0.31	0.42
2	0.31	1.0	0.12
3	0.42	0.12	1.0