Table 3.

Interrater Reliability Estimates of UCLR Total and Subscores and OSATS Total Score

Rating Scale	ICC Consistency/95% CI	ICC Agreement/95% CI	Concordance Coefficient/95% CI	Weighted Kappa/95% CI
OSATS score	0.290/(–0.103 to 0.604)	0.217/(–0.0956 to 0.522)	0.210/(–0.0552 to 0.448)	0.233/(–0.012 to 0.478)
Total UCLR score	0.686/(0.413–0.846)	0.618/(0.237–0.820)	0.608/(0.342–0.784)	0.600/(0.3498–0.8507)
Marking subscore	0.728/(0.480–0.868)	0.726/(0.482–0.866)	0.194/(–0.009 to 0.382)	0.718/(0.541–0.895)
Performance subscore	0.682/(0.407–0.843)	0.584/(0.137–0.811)	0.575/(0.309–0.758)	0.548/(0.334–0.762)
Result subscore	0.507/(0.156–0.744)	0.420/(0.033–0.695)	0.410/(0.107–0.644)	0.410/(0.077–0.743)

Estimates in bold are at the moderate level of reliability. P-value < 0.05 was used as a threshold for significance.