Table 6. Comparison of F1-score of different BERT (Bidirectional Encoder Representations from Transformers) models.
| Model size and model name | All codes, 95% CI | Top 80% codes, 95% CI | ||
|---|---|---|---|---|
| Multilabel | Top-5 | Multilabel | Top-5 | |
| Base | ||||
| NorDeClin-BERT-base-NorICDa | 0.52 (0.51-0.53) | 0.79 (0.79-0.80) | 0.58 (0.57-0.59) | 0.88 (0.87-0.88) |
| SweDeClin-BERT-SweICD | 0.27 (0.26-0.27) | 0.55 (0.54-0.56) | 0.31 (0.30-0.32) | 0.63 (0.62-0.64) |
| SweClin-BERT-NorICD | 0.46 (0.45-0.47) | 0.76 (0.75-0.77) | 0.54 (0.53-0.55) | 0.86 (0.85-0.87) |
| ScandiBERT-NorICD | 0.45 (0.44-0.46) | 0.76 (0.75-0.77) | 0.54 (0.53-0.55) | 0.86 (0.85-0.87) |
| NorBERT3-base-NorICD | 0.50 (0.49-0.51) | 0.78 (0.78-0.79) | 0.57 (0.56-0.58) | 0.87 (0.86-0.88) |
| Large | ||||
| NorDeClin-BERT-large-NorICD | 0.54 (0.53-0.55)b | 0.81 (0.80-0.82)b | 0.60 (0.60-0.61)b | 0.89 (0.89-0.90)b |
| NorBERT3-large-NorICD | 0.50 (0.49-0.51) | 0.79 (0.78-0.80) | 0.58 (0.56-0.59) | 0.88 (0.87-0.89) |
ICD-10: International Statistical Classification of Diseases, Tenth Revision.
Highest score for each scenario.