Table 7:
Ablation study: Results of the ablation study on the sentence classification level.
| Ignorance Category | Model | testing F1 score | testing support |
|---|---|---|---|
| ALL CATEGORIES BINARY | BERT | 0.33 | 2005 |
| answered question | BERT | 0.47 | 168 |
| explicit inquiry | BERT/BioBERT | 0.35 | 92 |
| unknown/novel | BERT/BioBERT | 0.25 | 63 |
| incompletely understood | BERT | 0.27 | 225 |
| indefinite relationship | BERT | 0.52 | 1072 |
| largely understood | BERT | 0.49 | 312 |
| anomalous/curious | BERT | 0.58 | 149 |
| alternative/controversy | BERT | 0.23 | 441 |
| difficult task | BERT | 0.35 | 93 |
| problem/complication | BioBERT | 0.43 | 202 |
| future work | BioBERT | 0.61 | 195 |
| future prediction | BERT | 0.22 | 55 |
| important consideration | BERT/BioBERT | 0.46 | 491 |