Table 2.

Single rater agreement (ICC) between 11 raters, with 95% confidence intervals (upper and lower bounds) and average rater ICC for comparison.

VS N = 17	ICC	Control	Motivation	Distraction	Search pattern	Stamina	Indication	Confidence	Thoroughness	Independence	Speed	Detect & locate
Pre-benchmarks	Single value	0.930	0.744	0.837	0.755	0.690	0.900	0.722	0.719	0.561	0.662	0.597
	Lower bound	0.817	0.484	0.628	0.500	0.416	0.748	0.452	0.452	0.280	0.384	0.315
	Upper bound	0.991	0.961	0.977	0.963	0.950	0.987	0.957	0.956	0.918	0.944	0.928
	Average value	0.995	0.979	0.988	0.980	0.973	0.993	0.976	0.976	0.953	0.969	0.960
Post-benchmarks	Single value	0.877	0.892	0.633	0.782	0.764	0.854	0.850	0.554	0.826	0.780	0.772
	Lower bound	0.702	0.733	0.353	0.539	0.512	0.657	0.651	0.277	0.609	0.534	0.525
	Upper bound	0.984	0.986	0.937	0.968	0.965	0.980	0.980	0.915	0.976	0.968	0.966
	Average value	0.991	0.992	0.965	0.983	0.981	0.989	0.898	0.952	0.987	0.983	0.982
HAS N = 11	ICC	Control	Motivation	Distraction	Search pattern	Stamina	Indication	Confidence	Thoroughness	Independence	Speed	Detect & locate
Pre- benchmarks	Single value	0.843	0.706	0.874	0.912	0.770	0.797	0.826	0.922	0.385	0.687	0.857
	Lower bound	0.629	0.422	0.686	0.768	0.503	0.550	0.596	0.789	0.129	0.390	0.651
	Upper bound	0.979	0.954	0.983	0.989	0.966	0.971	0.976	0.990	0.620	0.950	0.985
	Average value	0.983	0.964	0.987	0.991	0.974	0.977	0.981	0.992	0.873	0.960	0.985
Post-benchmarks	Single value	0.823	0.912	0.823	0.786	0.779	0.852	0.682	0.911	0.629	0.837	0.884
	Lower bound	0.592	0.768	0.590	0.532	0.522	0.645	0.392	0.767	0.327	0.618	0.708
	Upper bound	0.975	0.989	0.975	0.969	0.968	0.980	0.949	0.989	0.937	0.978	0.985
	Average value	0.981	0.991	0.975	0.981	0.959	0.949	0.976	0.991	0.983	0.988	0.984

Highlighting indicates an increase (darker shading) or decrease (paler) in ICC of at least 0.1 in the post-benchmark condition. All ICCs were significant at P < 0.001.