Table 5.
Entity-level results of BiLSTM models.
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Random initialization | ||||
| DISEASE | 0.4387 | 0.2913 | 0.3502 | 454 |
| RAREDISEASE | 0.4592 | 0.4712 | 0.4651 | 1095 |
| SIGN | 0.3288 | 0.3224 | 0.3256 | 958 |
| SYMPTOM | 0.0000 | 0.0000 | 0.0000 | 54 |
| Micro-avg | 0.3668 | 0.3742 | 0.3705 | 2561 |
| Macro-avg | 0.2454 | 0.2170 | 0.2282 | 2561 |
| Macro-weighted | 0.3946 | 0.3742 | 0.3820 | 2561 |
| Google news | ||||
| DISEASE | 0.4432 | 0.3071 | 0.3628 | 454 |
| RAREDISEASE | 0.4796 | 0.4971 | 0.4882 | 1095 |
| SIGN | 0.3166 | 0.3419 | 0.3287 | 958 |
| SYMPTOM | 0.4571 | 0.3200 | 0.3765 | 54 |
| Micro-avg | 0.3724 | 0.4020 | 0.3866 | 2561 |
| Macro-avg | 0.3393 | 0.2932 | 0.3112 | 2561 |
| Macro-weighted | 0.4084 | 0.4020 | 0.4028 | 2561 |
| Glove | ||||
| DISEASE | 0.4246 | 0.3622 | 0.3909 | 454 |
| RAREDISEASE | 0.5194 | 0.5529 | 0.5356 | 1095 |
| SIGN | 0.3114 | 0.3971 | 0.3491 | 958 |
| SYMPTOM | 0.6154 | 0.4800 | 0.5393 | 54 |
| Micro-avg | 0.3850 | 0.4596 | 0.4190 | 2561 |
| Macro-avg | 0.3742 | 0.3584 | 0.3630 | 2561 |
| Macro-weighted | 0.4236 | 0.4596 | 0.4387 | 2561 |
| Wiki-pubmed-PMC | ||||
| DISEASE | 0.5794 | 0.4339 | 0.4962 | 454 |
| RAREDISEASE | 0.5378 | 0.5388 | 0.5383 | 1095 |
| SIGN | 0.3167 | 0.3570 | 0.3356 | 958 |
| SYMPTOM | 0.5946 | 0.4074 | 0.4835 | 54 |
| Micro-avg | 0.4170 | 0.4494 | 0.4326 | 2561 |
| Macro-avg | 0.4057 | 0.3474 | 0.3707 | 2561 |
| Macro-weighted | 0.4637 | 0.4494 | 0.4539 | 2561 |
Best micro and macro scores are in bold. Best scores for each entity type are also in bold