Table 6.

The extended LOO benchmark

			OLD				NEW				TEPITOPE
Allele	#	#bind	PCC	AUC	NN	dist	PCC	AUC	NN	dist	AUC
DRB1*0101	7685	4382	0.567	0.767	DRB1*0401	0.352	0.583	0.786	DRB1*1402	0.322	0.727
DRB1*0301	2505	649	0.433	0.727	DRB3*0101	0.277	0.499	0.765	DRB1*0302	0.156	0.718
DRB1*0401	3116	1039	0.563	0.787	DRB1*0405	0.066	0.594	0.804	DRB1*0405	0.066	0.762
DRB1*0404	577	336	0.592	0.806	DRB1*0401	0.091	0.595	0.804	DRB1*0401	0.091	0.747
DRB1*0405	1582	627	0.638	0.826	DRB1*0401	0.066	0.633	0.833	DRB1*0401	0.066
DRB1*0701	1745	849	0.659	0.831	DRB1*0901	0.504	0.648	0.826	DRB1*0901	0.504	0.780
DRB1*0802	1520	431	0.380	0.710	DRB1*1101	0.111	0.369	0.692	DRB1*0813	0.041	0.777
DRB1*0901	1520	622	0.539	0.757	DRB5*0101	0.431	0.517	0.762	DRB5*0101	0.431	0.645
DRB1*1101	1794	778	0.602	0.799	DRB1*1302	0.084	0.460	0.741	DRB1*1302	0.084
DRB1*1302	1580	493	0.338	0.691	DRB1*1101	0.084	0.323	0.671	DRB1*1101	0.084	0.793
DRB1*1501	1769	709	0.568	0.775	DRB1*0404	0.295	0.525	0.756	DRB1*0404	0.295	0.596
DRB3*0101	1501	281	0.339	0.672	DRB1*0301	0.277	0.374	0.702	DRB3*0301	0.223	0.731
DRB4*0101	1521	485	0.506	0.753	DRB1*0404	0.397	0.518	0.766	DRB1*0404	0.397
DRB5*0101	3106	1280	0.547	0.781	DRB1*1101	0.295	0.608	0.813	DRB1*1101	0.295
DRB1*0302	148	44	0.396	0.729	DRB1*0301	0.156	0.542	0.759	DRB1*1402	0.119	0.760
DRB1*0806	118	91	0.670	0.886	DRB1*0802	0.107	0.703	0.902	DRB1*0802	0.107
DRB1*0813	1370	455	0.505	0.735	DRB1*0802	0.041	0.340	0.666	DRB1*0802	0.041	0.884
DRB1*0819	116	54	0.567	0.789	DRB1*0802	0.107	0.566	0.813	DRB1*0813	0.083	0.750
DRB1*1201	117	81	0.626	0.786	DRB1*1101	0.445	0.609	0.798	DRB1*1202	0.045
DRB1*1202	117	79	0.623	0.814	DRB1*1101	0.399	0.713	0.879	DRB1*1201	0.045
DRB1*1402	118	78	0.570	0.793	DRB1*1101	0.148	0.659	0.846	DRB1*0302	0.119
DRB1*1404	30	16	0.393	0.594	DRB1*0404	0.311	0.646	0.679	DRB1*0806	0.240
DRB1*1412	116	63	0.640	0.845	DRB1*0802	0.180	0.738	0.897	DRB1*0813	0.139
DRB3*0301	160	70	0.395	0.738	DRB3*0101	0.223	0.545	0.765	DRB3*0101	0.223
Ave			0.527	0.766			0.554	0.780
Ave*			0.543	0.779			0.529	0.774			0.744
Ave**			0.539	0.771			0.606	0.800

The predictive performance of the pan-specific NN-align method when trained in a leave-one-out experiment and evaluated on the 24 alleles included in the new peptide binding data set.

# is the number of peptide binding data for each allele, #bind is the number of peptides with a binding affinity stronger than 500 nM. OLD is the method described here trained on the old peptide data set, NEW is the method described here trained on the new data set, and TEPITOPE is the method by Sturniolo et al. [1]. NN is the nearest neighbor as defined by the pseudo sequence distance, and dist is the nearest neighbor distance calculated as described in Materials and methods. Ave is the per allele average, Ave* is the per allele average of the 13 alleles characterized by the TEPITOPE method, and Ave** is the per-allele average performance of the 10 alleles included in the new peptide binding data set. In bold is highlighted the best performing method for each of the 24 alleles. AUC values were calculated using a binding threshold of 500 nM. Only AUC values are included for the TEPITOPE method since prediction values for this method are not linearly related to the binding affinity. The double line separates the 10 novel alleles from the original 14 alleles included in the development of the NetMHCIIpan-1.0 method.