Table 2.
Bio-NER tools used in the study. MetaMap, MetaMap Lite and CLAMP provide configurable assertion detection (i.e., negation), hence the two performance values in the i2b2 2010 dataset.
| Bio-NER Tool | Description | Performance (F1 Score) | ||
|---|---|---|---|---|
| i2b2 2010 | SemEval 2014 | NCBI disease | ||
| MetaMap | An open-source software program developed by the NLM for finding UMLS concepts in biomedical text using dictionary lookup | 0.37, 0.38 (negation) | 0.469 | 0.641 |
| MetaMap Lite | A lightweight implementation of MetaMap, meant for applications that emphasize processing speed and ease of use | 0.38, 0.45 (negation) | 0.645 | 0.725 |
| CLAMP | A clinical NLP toolkit that provides state-of-the-art NLP components and a user-friendly graphic user interface to build customized NLP pipelines. CLAMP uses various technologies, including machine learning-based methods and rule-based methods | 0.857, 0.9398 (negation) | 0.632 | – |
| BERN (with Bio-BERT) | A neural biomedical named entity recognition and multi-type normalization tool. BERN uses the Bio-BERT NER models to tag genes/proteins, diseases, drugs/chemicals, and species | 0.865 | 0.779 | 0.8936 |