Table 6.

Test results in biomedical named entity recognition

				BERT	BioBERT v1.0			BioBERT v1.1
Type	Datasets	Metrics	SOTA	(Wiki + Books)	(+ PubMed)	(+ PMC)	(+ PubMed + PMC)	(+ PubMed)
Disease	NCBI disease	P	88.30	84.12	86.76	86.16	89.04	88.22
		R	89.00	87.19	88.02	89.48	89.69	91.25
		F	88.60	85.63	87.38	87.79	89.36	89.71
	2010 i2b2/VA	P	87.44	84.04	85.37	85.55	87.50	86.93
		R	86.25	84.08	85.64	85.72	85.44	86.53
		F	86.84	84.06	85.51	85.64	86.46	86.73
	BC5CDR	P	89.61	81.97	85.80	84.67	85.86	86.47
		R	83.09	82.48	86.60	85.87	87.27	87.84
		F	86.23	82.41	86.20	85.27	86.56	87.15
Drug/chem.	BC5CDR	P	94.26	90.94	92.52	92.46	93.27	93.68
		R	92.38	91.38	92.76	92.63	93.61	93.26
		F	93.31	91.16	92.64	92.54	93.44	93.47
	BC4CHEMD	P	92.29	91.19	91.77	91.65	92.23	92.80
		R	90.01	88.92	90.77	90.30	90.61	91.92
		F	91.14	90.04	91.26	90.97	91.41	92.36
Gene/protein	BC2GM	P	81.81	81.17	81.72	82.86	85.16	84.32
		R	81.57	82.42	83.38	84.21	83.65	85.12
		F	81.69	81.79	82.54	83.53	84.40	84.72
	JNLPBA	P	74.43	69.57	71.11	71.17	72.68	72.24
		R	83.22	81.20	83.11	82.76	83.21	83.56
		F	78.58	74.94	76.65	76.53	77.59	77.49
Species	LINNAEUS	P	92.80	91.17	91.83	91.62	93.84	90.77
		R	94.29	84.30	84.72	85.48	86.11	85.83
		F	93.54	87.60	88.13	88.45	89.81	88.24
	Species-800	P	74.34	69.35	70.60	71.54	72.84	72.80
		R	75.96	74.05	75.75	74.71	77.97	75.36
		F	74.98	71.63	73.08	73.09	75.31	74.06

Notes: Precision (P), Recall (R) and F1 (F) scores on each dataset are reported. The best scores are in bold, and the second best scores are underlined. We list the scores of the state-of-the-art (SOTA) models on different datasets as follows: scores of Xu et al. (2019) on NCBI Disease, scores of Sachan et al. (2018) on BC2GM, scores of Zhu et al. (2018) (single model) on 2010 i2b2/VA, scores of Lou et al. (2017) on BC5CDR-disease, scores of Luo et al. (2018) on BC4CHEMD, scores of Yoon et al. (2019) on BC5CDR-chemical and JNLPBA and scores of Giorgi and Bader (2018) on LINNAEUS and Species-800.