Table 1.

Agreement between evaluators on the predicted disposition. The values in parentheses are the results obtained when the five categories are collapsed to three.

CBR DST predicted disposition: Do you agree with the suggested course of action?
Evaluator 5 point scale (3 point scale)	Observed Agreement	Agreement expected by chance alone	κ	95% C.I.	Interpretation
End users & Expert 1	93.5% (89.9%)	87.2% (79.6%)	0.49 (0.51)	0.25 – 0.69 (0.25 – 0.71)	Moderate (Moderate)
End users & Expert 2	93.6% (91.6%)	86.4% (79.9%)	0.53 (0.58)	0.33 – 0.68 (0.36 – 0.76)	Moderate (Moderate)
Expert 1 & Expert 2	94.5% (91.6%)	87.3% (80.9%)	0.56 (0.56)	0.38– 0.70 (0.33 – 0.74)	Moderate (Moderate)