Table 5:
Sentence Classification: the best model for sentence classification for each approach: (1) ALL CATEGORIES BINARY: binary classification ignorance or not, (2) AN ENSEMBLE OF BINARY CLASSIFIERS: binary classification for each class (reported) combined to create the ensemble, and (3) ALL CATEGORIES COMBINED: one multi-classifier to all categories.
| Ignorance Category | Model | testing F1 score | testing support |
|---|---|---|---|
| ALL CATEGORIES BINARY | BioBERT | 0.95 | 2005 |
| answered question | BERT | 0.97 | 168 |
| explicit inquiry | BioBERT | 0.9 | 92 |
| unknown/novel | BioBERT | 0.88 | 63 |
| incompletely understood | ANN | 0.83 | 225 |
| indefinite relationship | BERT | 0.87 | 1072 |
| largely understood | BERT | 0.9 | 312 |
| anomalous/curious | BERT | 0.96 | 149 |
| alternative/controversy | BioBERT | 0.79 | 441 |
| difficult task | BERT | 0.95 | 93 |
| problem/complication | BioBERT | 0.9 | 202 |
| future work | BioBERT | 0.85 | 195 |
| future prediction | BERT | 0.88 | 55 |
| important consideration | BERT/BioBERT | >0.99 | 491 |
| ALL CATEGORIES COMBINED | BioBERT | 0.12 | 2005 |