. Author manuscript; available in PMC: 2022 Nov 14.

Published in final edited form as: IEEE/ACM Trans Comput Biol Bioinform. 2022 Oct 10;19(5):2584–2595. doi: 10.1109/TCBB.2022.3173562

TABLE 6.

Performance Evaluation on an Independent Sample of 3000 Articles in litcovid

	Label-based measures				Instance-based measures		Time measures in the production environment (in sec)

	Macro-F1	Macro-AP	Micro-F1	Micro-AP	F1	Accuracy
Binary BERT	0.8116	0.7251	0.8570	0.7733	0.8719	0.6327	8912.22 (2.97 sec/doc)
Linear BERT	0.8299	0.7449	0.8539	0.7701	0.8759	0.6354	1536.24 (0.51 sec/doc)
LitMC-BERT (ours)	0.8413	0.7690	0.8628	0.7813	0.8862	0.6512	1673.98 (0.56 sec/doc)

Time was measured in the production environment where only cpus are available.