Table 5.

Agreement on whether a CPG would be recommended or not

Pair of Raters	Kappa (unweighted)	Kappa (quadratic weights)
Generalists	0.20	0.34
Specialists	0.25	0.22
Researchers	0.39	0.47

A Kappa was calculated on the final overall rating question whether or not a CPG should be using with the data dichotomized as YES (strongly recommend or recommend with provisos) or NO (Would not recommend or unsure) or by using quadratic weighting to compare the strength of recommendation (Strongly, with provisos, would not, unsure).