Table 2.

Agreement between graders, using alternative standards of agreement. Κ values from 0.00 to 0.20 corresponded to slight agreement, 0.21 to 0.40 fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial, and 0.81 to 1.0 almost perfect agreement.⁷

Grader pair	Agreement	6-step Scale Κ Value (range)	9-step scale Κ Value (range)
1 (n=44)	Exact	0.39 (0.22–0.57)	0.42 (0.25–0.58)
	Modified-1-grade	0.79 (0.61–0.96)
	Within-1-grade	0.80 (0.53–1.000)	0.64 (0.45–0.82)
	Within-2-grade		0.92 (0.69–1.00)
2 (n=72)	Exact	0.81 (0.67–0.93)	0.63 (0.49–0.76)
	Modified-1-grade	0.85 (0.71–0.95)
	Within-1-grade	0.96 (0.83–1.00)	0.87 (0.74–0.97)
	Within-2-grade		0.86 (0.70–0.96)
3 (n=79)	Exact	0.40 (0.26–0.54)	0.38 (0.27–0.50)
	Modified-1-grade	0.49 (0.31–0.66)
	Within-1-grade	0.66 (0.46–0.83)	0.60 (0.45–0.75)
	Within-2-grade		0.81 (0.66–0.94)
4 (n=78)	Exact	0.28 (0.16–0.43)	0.42 (0.27–0.56)
	Modified-1-grade	0.46 (0.29–0.64)
	Within -1-grade	0.66 (0.46–0.82)	0.62 (0.45–0.76)
	Within-2-grade		0.86 (0.71–0.97)
5 (n=76)	Exact	0.41 (0.27–0.56)	0.15 (0.04–0.26)
	Modified-1-grade	0.52 (0.34–0.68)
	Within-1-grade	0.67 (0.48–0.81)	0.38 (0.22–0.54)
	Within-2-grade		0.79 (0.61–0.93)
Average (n=349)	Exact	0.46 (0.28–0.81)	0.40 (0.15–0.63)
	Within-1-grade	0.75 (0.66–0.96)	0.62 (0.38–0.87)
	Within-2-grade		0.85 (0.79–0.92)