Table 4.
Intraclass correlations for each AGREE instrument domain as a function of the number of raters
Scope & Purpose | Stakeholder Involvement | Rigor of Development | Clarity & Presentation | Applicability | Editorial Independence | |||||||
Single Rater ICC | Average of Raters ICC | Single Rater ICC | Average of Raters ICC | Single Rater ICC | Average of Raters ICC | Single Rater ICC | Average of Raters ICC | Single Rater ICC | Average of Raters ICC | Single Rater ICC | Average of Raters ICC | |
2 Raters | 0.37 | 0.54 | 0.71 | 0.83 | 0.81 | 0.89 | 0.41 | 0.58 | 0.65 | 0.79 | 0.60 | 0.75 |
3 Raters | 0.38 | 0.64 | 0.71 | 0.88 | 0.82 | 0.93 | 0.54 | 0.78 | 0.66 | 0.85 | 0.38 | 0.65 |
4 Raters | 0.41 | 0.73 | 0.64 | 0.88 | 0.77 | 0.93 | 0.51 | 0.80 | 0.54 | 0.82 | 0.35 | 0.69 |
5 Raters | 0.42 | 0.78 | 0.69 | 0.92 | 0.79 | 0.95 | 0.51 | 0.84 | 0.57 | 0.87 | 0.38 | 0.76 |
6 Raters | 0.40 | 0.80 | 0.67 | 0.93 | 0.79 | 0.96 | 0.55 | 0.88 | 0.51 | 0.86 | 0.35 | 0.76 |
ICC = Intraclass correlation coefficient