Table 7.
Entity-level results of BiLSTM-CRF models.
| Label | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Random initialization | ||||
| DISEASE | 0.5414 | 0.3780 | 0.4451 | 454 |
| RAREDISEASE | 0.6540 | 0.7144 | 0.6829 | 1095 |
| SIGN | 0.4892 | 0.4391 | 0.4628 | 958 |
| SYMPTOM | 0.8529 | 0.5800 | 0.6905 | 54 |
| Micro-avg | 0.5421 | 0.5494 | 0.5457 | 2561 |
| Macro-avg | 0.5075 | 0.4223 | 0.4563 | 2561 |
| Macro-weighted | 0.5748 | 0.5494 | 0.5582 | 2561 |
| Google news | ||||
| DISEASE | 0.5597 | 0.4304 | 0.4866 | 454 |
| RAREDISEASE | 0.6482 | 0.7548 | 0.6975 | 1095 |
| SIGN | 0.5327 | 0.4166 | 0.4675 | 958 |
| SYMPTOM | 0.6667 | 0.5600 | 0.6087 | 54 |
| Micro-avg | 0.5556 | 0.5654 | 0.5604 | 2561 |
| Macro-avg | 0.4815 | 0.4324 | 0.4521 | 2561 |
| Macro-weighted | 0.5887 | 0.5654 | 0.5711 | 2561 |
| Glove | ||||
| DISEASE | 0.4720 | 0.5092 | 0.4899 | 454 |
| RAREDISEASE | 0.7226 | 0.7240 | 0.7233 | 1095 |
| SIGN | 0.5068 | 0.4606 | 0.4826 | 958 |
| SYMPTOM | 0.5385 | 0.5600 | 0.5490 | 54 |
| micro-avg | 0.5489 | 0.5821 | 0.5650 | 2561 |
| Macro-avg | 0.4480 | 0.4508 | 0.4490 | 2561 |
| Macro-weighted | 0.5937 | 0.5821 | 0.5874 | 2561 |
| Wiki-pubmed-PMC | ||||
| DISEASE | 0.7208 | 0.4890 | 0.5827 | 454 |
| RAREDISEASE | 0.6339 | 0.7890 | 0.7030 | 1095 |
| SIGN | 0.4994 | 0.4562 | 0.4768 | 958 |
| SYMPTOM | 0.6739 | 0.5741 | 0.6200 | 54 |
| Micro-avg | 0.5564 | 0.6068 | 0.5805 | 2561 |
| Macro-avg | 0.5056 | 0.4617 | 0.4765 | 2561 |
| Macro-weighted | 0.5998 | 0.6068 | 0.5953 | 2561 |
Best micro and macro scores are in bold. Best scores for each entity type are also in bold