Table 12.

Inter-rater agreement

ROBIS question	No. of studies	Gwet’s AC1/Gwet’s AC2	95% CI	p-value
Domain 1: study eligibility criteria
1.1	16	0.57	0.17, 0.96	0.008
1.2	16	0.71	0.55, 0.87	< 0.001
1.3	16	0.76	0.61, 0.91	< 0.001
1.4	16	0.71	0.54, 0.87	< 0.001
1.5	16	0.49	0.20, 0.77	0.002
Domain 1Concerns regarding specification of study eligibility criteria	16	0.30	− 0.03, 0.63	0.072
Domain 2: identification and selection of studies
2.1	16	0.54	0.34, 0.73	< 0.001
2.2	16	0.64	0.37, 0.92	< 0.001
2.3	16	0.57	0.34, 0.81	< 0.001
2.4	16	0.50	0.27, 0.73	< 0.001
2.5	14	0.61	0.18, 1.00	< 0.001
Domain 2Concerns regarding methods used to identify and/or select studies	16	0.43	0.13, 0.73	0.008
Domain 3: data collection and study appraisal
3.1	14	0.82	0.51, 1.00	< 0.001
3.2	16	0.70	0.44, 0.96	< 0.001
3.3	16	0.72	0.52, 0.92	< 0.001
3.4	16	0.92	0.83, 1.00	< 0.001
3.5	15	0.66	0.21, 1.00	0.007
Domain 3Concerns regarding methods used to collect data and appraise studies	16	0.61	0.32, 0.89	< 0.001
Domain 4: synthesis and findings
4.1	16	0.65	0.45, 0.86	< 0.001
4.2	16	0.42	0.11, 0.73	0.011
4.3	16	0.73	0.58, 0.88	< 0.001
4.4	16	0.23	− 0.02, 0.48	0.072
4.5	16	0.40	0.22, 0.57	< 0.001
4.6	16	0.55	0.32, 0.77	< 0.001
Domain 4Concerns regarding the synthesis and findings	16	0.17	− 0.17, 0.50	0.305
Risk of bias in the review
A	16	0.47	0.28, 0.65	0.015
B	16	0.69	0.55, 0.82	< 0.001
C	16	0.54	0.37, 0.72	< 0.001
ROB	16	0.47	0.17, 0.77	0.004

Three ratings missing