Table 2:

Performance of GeneGPT compared to other LLMs on the GeneTuring benchmark.

GeneTuring task	GPT-2	BioGPT	BioMedLM	GPT-3	ChatGPT	New Bing	GeneGPT (ours)
							-full	-slim
Nomenclature
Gene alias	0.00	0.00	0.04	0.09	0.07	0.66	0.80 *	0.84 *
Gene name conversion	0.00	0.00	0.00	0.00	0.00	0.85	1.00	1.00
Average	0.00	0.00	0.02	0.05	0.04	0.76	0.90	0.92
Genomic location
Gene SNP association	0.00	0.00	0.00	0.00	0.00	0.00	1.00 *	1.00
Gene location	0.01	0.04	0.12	0.09	0.09	0.61	0.62	0.66
SNP location	0.03	0.05	0.01	0.02	0.05	0.01	1.00	0.98
Average	0.01	0.03	0.04	0.04	0.05	0.21	0.87	0.88
Functional analysis
Gene disease association	0.00	0.02	0.16	0.34	0.31	0.84	0.76 *	0.66
Protein-coding genes	0.00	0.18	0.37	0.70	0.54	0.97	0.76	1.00
Average	0.00	0.10	0.27	0.52	0.43	0.91	0.76	0.84
Sequence alignment
DNA to human genome	0.02	0.07	0.03	0.00	0.00	0.00	0.44 *	0.44 *
DNA to multiple species	0.02	0.00	0.00	0.20	0.00	0.00	0.86	0.88
Average	0.02	0.04	0.02	0.10	0.00	0.00	0.65	0.66
Overall average	0.00	0.04	0.08	0.16	0.12	0.44	0.80	0.83

^*One-shot learning for GeneGPT. Bolded and underlined numbers denote the highest and second-highest performance, respectively.