Table 5.
Extended measurements for concept recognition experiments on 39 Undiagnosed Diseases Program clinical notes. Neural Concept Recognizer models were trained on Human Phenotype Ontology. Largest values for each category are italicized.
| Method | Extended value (%) | Jaccard index (%) | ||
|
|
Precision | Recall | F1-score | |
| BioLarK | 58.9 | 42.6 | 49.5 | 29.5 |
| cTAKESa | 68.5 | 36.7 | 47.8 | 27.3 |
| OBOb | 59.2 | 46.4 | 52.0 | 31.3 |
| NCBOc | 69.8 | 37.2 | 48.5 | 27.2 |
| NCRd | 57.1 | 49.4 | 53.0 | 31.5 |
| NCR-He | 54.0 | 49.4 | 51.6 | 30.5 |
| NCR-Nf | 54.7 | 50.5 | 52.5 | 31.4 |
| NCR-HNg | 56.5 | 49.0 | 52.5 | 31.3 |
acTAKES: Clinical Text Analysis and Knowledge Extraction System.
bOBO: Open Biological and Biomedical Ontologies
cNCBO: National Center for Biomedical Ontology.
dNCR: Neural Concept Recognizer.
eNCR-H: variation of the NCR model that ignores taxonomic relations.
fNCR-N: variation of the NCR model that has not been trained on negative samples.
gNCR-HN: variation of the NCR model that ignores the taxonomy and has not been trained on negative examples.