Table 2.

Agreement between evaluators on the value of the explanatory case. The values in parentheses are the results obtained when the five categories are collapsed to three.

CBR DST explanatory example: Did you find the explanation case useful?
Evaluator5 point scale(3 point scale)	Observed Agreement	Agreement expected by chance alone	κ	95% C.I.	Interpretation
End users & Expert 1	89.2% (70.8%)	83.0% (58.4%)	0.36 (0.30)	0.19 – 0.53 (0.14 – 0.46)	Fair (Fair)
End users & Expert 2	83.96% (46.0%)	84.3% (43.2%)	−0.02 (0.05)	−0.10 – 0.04 (0.04 – 0.14)	None* (Poor)
Expert 1 & Expert 2	87.03% (59.2%)	87.3% (52.6%)	−0.01 (0.14)	−0.08 – 0.07 (0.03 – 0.26)	None* (Poor)