Table 2.

Accuracy evaluation of literature mined results in Cancer-Alterome.

Named entity recognition and normalization		Evaluation metric			Count of prediction
Tools	Entity type	Precision	Recall	F1 Score	Mention	Entity	Normalization
PubTator20	Gene	0.79	0.81	0.80	203,939	22,021	Entrez ID
PubTator20	Point mutations and SNPs	0.81	0.81	0.83	197,595	164,144	rsID
AGAC-NER24	General genetic alteration	0.74	0.57	0.64	3,002,082	549	Dictionary
AGAC-NER24	Trigger word	0.78	0.70	0.74	200,670	—	—
OGER++22	GO	0.72	0.17	0.27	52,140	22,874	GO ID
PhenoTagger23	HPO	0.79	0.70	0.74	24,261	8,345	HPO ID
PubTator20	MeSH	0.83	0.82	0.81	1,131,933	45,119	MeSH ID
Relation extraction		Evaluation metric			Count of prediction
Tools	Relation type	Precision	Recall	F1 Score	Relations
AGAC-RE24	Theme	0.87	0.84	0.91	37,411,752
AGAC-RE24	Cause	0.88	0.85	0.82	12,420,489
Regulatory events identification		Evaluation metric			Count of prediction
Method	Event type	Precision	Recall	F1 Score	Events
Template match	GARE	0.84	0.96	0.90	16,681,473

(a) Evaluation of NLP tools for named entity recognition and normalization. (b) Evaluation of NLP tools for relation extraction. (c) Evaluation of regulatory events (GAREs).