. Author manuscript; available in PMC: 2015 Mar 1.

Published in final edited form as: Int J Psychophysiol. 2013 Oct 31;91(3):186–193. doi: 10.1016/j.ijpsycho.2013.10.015

Table 2.

Measures of predictive validity for all scoring methods.

	Rater Type	F	p	Estimate of Error Variance	Bayes factor (Manual)	Bayes factor (Autonomate)
Study 1	Manual	7.249	0.002	0.034	-	-
	Autonomate	7.129	0.002	0.035	1.006	-
	SCRalyze	14.207	0.000	0.174	1.426	1.417
	Ledalab	12.233	0.000	0.061	1.135	1.128
	AcqKnowledge	9.290	0.000	0.020	0.891	0.886

Study 2	Manual	0.292	0.831	0.014	-	-
	Autonomate	0.368	0.776	0.015	1.015	-
	SCRalyze	2.808	0.046	0.115	1.580	1.556
	Ledalab	0.438	0.726	0.074	1.436	1.414
	AcqKnowledge	0.713	0.548	0.017	1.043	1.028

Study 3	Manual	7.453	0.000	0.039	-	-
	Autonomate	7.376	0.000	0.045	1.032	-
	SCRalyze	6.945	0.000	0.109	1.250	1.212
	Ledalab	2.667	0.037	0.182	1.397	1.354
	AcqKnowledge	5.414	0.001	0.071	1.139	1.104

Note. Bayes factors were computed as $B F = e \frac{log σ_{e_{1}}^{2} - log σ_{e_{2}}^{2}}{2}$ using manual scoring or Autonomate as the reference (model 2). Bayes factors greater than one favor the reference (second) model.