Table 5.

Top 10 submissions in terms of Kendall’s tau, in order of increasing tau, for the two free energy sets in both stages of the challenge. Kendall’s tau error bars are illustrated in Fig. 6 and listed in Tables S13–S16. Methods include predictions from all scoring and free energy methods. Methods in bold are common methods submitted by the same participant across FE sets.

Stage 1
FE Set 1
Kendall’s Tau	Software Used	Submitter Name	Group/PI Name	Receipt ID
0.44	Smina	Bentley Wingert	Carlos Camacho	5nim5
0.39	Quasi exact method FE method	Bentley Wingert	Carlos Camacho	nrdge
0.39	Smina	Bentley Wingert	Carlos Camacho	f48cf
0.35	ligand-based 3D QSAR method	Flavio Ballante	Garland R. Marshall	kz0dz
0.35	Smina	Bentley Wingert	Carlos Camacho	ukdfw
0.31	Knowledge-based scoring method ITScore_v2_TF	Xiaoqin Zou	Xiaoqin Zou	r885q
0.29	MMPB/SA	Xiaoqin Zou	Xiaoqin Zou	axxmu
0.29	MMPB/SA	Xiaoqin Zou	Xiaoqin Zou	3qyiy
0.29	Knowledge-based scoring method ITScore_TF	Xiaoqin Zou	Xiaoqin Zou	c20xb
0.29	SeeSAR scoring function	Anonymous	Anonymous	h2w3q
FE Set 2
Kendall’s Tau	Software Used	Submitter Name	Group/PI Name	Receipt ID
0.52	Knowledge-based scoring method ITScore_v1	Xiaoqin Zou	Xiaoqin Zou	fvfe7
0.50	Smina	Matthew Baumgartner	David Evans	gzd7a
0.49	Knowledge-based scoring method ITScore_v2	Xiaoqin Zou	Xiaoqin Zou	33a8g
0.49	idock-RF-v3 scoring function with visual inspection	Ho Leung Ng	Ho Leung Ng	6pcik
0.49	Knowledge-based scoring method ITScore_v1_TF	Xiaoqin Zou	Xiaoqin Zou	q76s3
0.45	Knowledge-based scoring method ITScore_v2_TF	Xiaoqin Zou	Xiaoqin Zou	4ivv5
0.44	Ichem-GRIM score + HYDE score	Didier Rognan	Didier Rognan	4ynsp
0.44	Glide	Ashutosh Kumar	Kam Y.J. Zhang	ttgw7
0.41	RF-Score-VS machine learning score, Smina	Anonymous	Anonymous	f30wc
0.41	Knowledge-based scoring method ITScore_v2	Xiaoqin Zou	Xiaoqin Zou	mzwwt

Stage 2
FE Set 1
Kendall’s Tau	Software Used S	submitter Name	Group/PI Name	Receipt ID
0.41	Trained Random Forest Model, Rl-Score	Anonymous	Anonymous	4rbjk
0.35	Trained Random Forest Model, Rl-Score	Anonymous	Anonymous	bw4pj
0.33	Trained Random Forest Model, Rl-Score	Anonymous	Anonymous	jtsy2
0.33	Trained Random Forest Model, Rl-Score	Anonymous	Anonymous	n55eq
0.31	Trained Random Forest Model, Rl-Score	Anonymous	Anonymous	0aggj
0.31	Smina, CNN Model Scoring	David Koes	David Koes	0zno2
0.31	Knowledge-based scoring method ITScore_v2	Xiaoqin Zou	Xiaoqin Zou	pr2fp
0.31	Smina, CNN Model Scoring	David Koes	David Koes	tgmx1
0.31	QMMM energy, Schrodinger QSITE	Anonymous	Anonymous	eta0e
0.29	KRh-SCORPIO scoring function modeled using available affinity data	Jonathan Bohmann	Pharmaceuticals and Bioengineering Dept.	35yg0
FE Set 2
Kendall’s Tau	Software Used	Submitter Name	Group/PI Name	Receipt ID
0.62	Explicit solvent FE (Jarzynski pulling)	Oleksandr Yakovenko	Steve Jones	xk67c
0.55	QSAR method	Matthew Baumgartner	David Evans	hj31e
0.55	Schrodinger FEP	Anonymous	Anonymous	81n55
0.53	Glide-XP	Anonymous	Anonymous	ljdjm
0.53	Total Energy	Anonymous	Anonymous	67a3e
0.52	Xscore	Anonymous	Anonymous	qokw3
0.52	Glide	Ashutosh Kumar	Kam Y.J. Zhang	tbxzq
0.50	Schrodinger FEP	Christina Athanasiou	Zoe Cournia	x2j7p
0.50	QSAR method	Matthew Baumgartner	David Evans	naex2
0.50	Knowledge-based scoring method ITScore_v1	Xiaoqin Zou	Xiaoqin Zou	sb1dg