Table 5. IR results for fine-tuning BERT models on the full dataset.
Overlap Ratio char and Overlap Ratio word refer to the average overlap length between the predicted answers and the expected answers divided by the length of the expected answers, at character-level and word-level, respectively. Exact match rate refers to the proportion of perfect matches between the predicted answers and the expected answers.
| Model | Task | Overlap | Overlap |
|---|---|---|---|
| Ratio char | Ratio word | ||
| BERT | ABMR IR | 0.442 | 0.616 |
| BioBERT | ABMR IR | 0.519 | 0.667 |
| Clinical BERT | ABMR IR | 0.363 | 0.461 |
| Kidney BERT | ABMR IR | 0.363 | 0.461 |
| exKidneyBERT | ABMR IR | 0.604 | 0.833 |
| BERT | TCMR IR | 0.494 | 0.653 |
| BioBERT | TCMR IR | 0.494 | 0.653 |
| Clinical BERT | TCMR IR | 0.494 | 0.653 |
| Kidney BERT | TCMR IR | 0.494 | 0.653 |
| exKidneyBERT | TCMR IR | 0.664 | 0.792 |
| Model | Task | Exact match rate | |
| BERT | IFTA IR | 0.942 | |
| BioBERT | IFTA IR | 0.956 | |
| Clinical BERT | IFTA IR | 0.947 | |
| Kidney BERT | IFTA IR | 0.950 | |
| exKidneyBERT | IFTA IR | 0.958 |