Table 5.
Comparison of different methods on development set of Subtask 1
| F1 (binary) (%) | AUC (binary) (%) | F1 (exact) (%) | F1 (relaxed) (%) | |
|---|---|---|---|---|
| NER, no classifier | - | - | 14.6 | 22.8 |
| SVM (baseline) | 38.4 | 62 | 14.9 | 23.4 |
| Logistic | 36 | 61 | 15.4 | 23.7 |
| SuRDE | 45.2 | 71 | 17.9 | 27.4 |
| SeRDE (200 refs, words) | 49.2 | 74.6 | 18.7 | 29.6 |
| SeRDE (200 refs, bigrams) | 48.8 | 74.2 | 18.5 | 29.7 |
| SeRDE (200 refs, words + bigrams) | 50.2 | 76.5 | 19.2 | 30.7 |
F1 (exact) and F1 (relaxed) are the official evaluation measures. The F1 (binary) and AUC (binary) are the performance on the binary sentence classification task defined in ‘Method’ and Table 1.