Table 7.
Spatial role extraction results using gold spatial indicators: Average Precision (P%), Recall (R%), and F1 measures of 10-fold CV across 5 different fold variations. CI - 95% confidence intervals of the average F1 measures across 50 iterations. BLSTM-C - Bi-LSTM CRF, BERT-L - BERTlarge, BERT-LM - BERTlarge (MIMIC), XLNet-L - XLNetlarge.
| Models |
trajector |
landmark |
diagnosis |
hedge |
overall |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 (CI) | |
| BLSTM-C | 88.8 | 87.3 | 88.0 | 94.1 | 89.9 | 91.9 | 76.6 | 75.0 | 75.2 | 78.4 | 76.3 | 77.0 | 89.0 | 86.4 | 87.6 (±0.55) |
| BERT-L | 89.7 | 91.8 | 90.7 | 95.4 | 96.1 | 95.8 | 72.7 | 85.5 | 78.4 | 72.8 | 84.1 | 77.8 | 88.8 | 92.4 | 90.5 (±0.42) |
| BERT-LM | 91.2 | 93.1 | 92.1 | 95.6 | 96.6 | 96.1 | 72.3 | 83.9 | 77.4 | 75.0 | 86.1 | 80.1 | 89.5 | 93.3 | 91.4 (±0.54) |
| XLNet-L | 92.8 | 94.1 | 93.5 | 96.1 | 96.8 | 96.4 | 78.6 | 88.0 | 82.8 | 79.6 | 88.6 | 83.7 | 91.6 | 94.2 | 92.9 (±0.38) |