Table 3.
Rater assessment of research reports based on the risk of bias guidelines.
| Risk of bias item | Risk of bias inter-rater agreement and reliability |
Risk of bias compliance |
|||||
|---|---|---|---|---|---|---|---|
| % Agreement (present or absent) | Kappa (95% CI) | Prevalence index | Bias index | PABAK (95% CI) | % Present rater 1 | % Present rater 2 | |
| Sequence generation | 86 | 0.68 (0.51, 0.86) | 0.36 | 0.08 | 0.72 (0.56, 0.88) | 72 | 64 |
| Allocation concealment | 69 | 0.44 (0.28, 0.60) | 0.03 | 0.31 | 0.39 (0.18, 0.60) | 67 | 36 |
| Blinding of participants, personnel and outcome assessors | 46 | 0.16 (0.06, 0.26) | 0.04 | 0.54 | −0.08 (−0.31, 0.15) | 75 | 21 |
| incomplete outcome data | 69 | 0.39 (0.18, 0.60) | 0.03 | 0.00 | 0.39 (0.18, 0.60) | 51 | 51 |
| Selective outcome reporting | 81 | 0.16 (−0.07, 0.40) | 0.75 | 0.17 | 0.61 (0.43, 0.79) | 21 | 4 |
| Other potential threats to validity | 86 | −0.03 (−0.07, 0.02) | 0.86 | 0.11 | 0.72 (0.56, 0.88) | 13 | 1 |