Table 8.
Spatial role extraction results using predicted spatial indicators: Average Precision (P%), Recall (R%), and F1 measures of 10-fold CV across 5 different fold variations. CI - 95% confidence intervals of the average F1 measures across 50 iterations. BLSTM-C - Bi-LSTM CRF, BERT-L - BERTlarge, BERT-LM - BERTlarge (MIMIC), XLNet-L - XLNetlarge.
| Models |
trajector |
landmark |
diagnosis |
hedge |
overall |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 (CI) | |
| BLSTM-C | 77.7 | 83.9 | 80.6 | 83.0 | 86.6 | 84.7 | 72.3 | 72.2 | 71.7 | 71.5 | 72.5 | 71.6 | 78.8 | 83.1 | 80.8 (±0.76) |
| BERT-L | 85.7 | 73.9 | 78.1 | 90.0 | 77.2 | 82.8 | 73.1 | 75.3 | 73.8 | 71.4 | 71.8 | 71.1 | 85.1 | 75.2 | 79.3 (±2.16) |
| BERT-LM | 85.8 | 85.9 | 85.7 | 89.6 | 89.2 | 89.3 | 72.5 | 83.0 | 77.3 | 72.0 | 81.6 | 76.3 | 84.8 | 86.6 | 85.6 (±0.65) |
| XLNet-L | 82.3 | 88.9 | 85.3 | 86.0 | 90.9 | 88.2 | 73.8 | 85.6 | 79.0 | 73.2 | 85.3 | 78.6 | 82.1 | 89.1 | 85.4 (±0.65) |