Table 1.
Model | Full test set: 13 545 QAs |
Test HasAns: 12 376 QAs |
Test NoAns: 1169 QAs |
|||
---|---|---|---|---|---|---|
Exact | Partial | Exact | Partial | Exact | Partial | |
BERTbase + emrQAwhy | 0.633 | 0.688 | 0.599 | 0.659 | 0.995 | 0.995 |
BERTbase + SQuADwhy + emrQAwhy | 0.660 | 0.728 | 0.628 | 0.703 | 0.994 | 0.994 |
BERTbase + i2b2Notespre + emrQAwhy | 0.663 | 0.718 | 0.631 | 0.692 | 0.997 | 0.997 |
BERTbase + ClinBERTpre + emrQAwhy | 0.695 | 0.744 | 0.666 | 0.720 | 0.994 | 0.994 |
BERTbase + ClinBERTpre + SQuADwhy + emrQAwhy | 0.700 | 0.757 | 0.672 | 0.735 | 0.995 | 0.995 |
BERTbase + ClinBERTpre + SQuADwhy + emrQAwhy (10 epochs) | 0.707 | 0.760 | 0.679 | 0.737 | 0.999 | 0.999 |
Abbreviations: HasAns: answerable according to the gold; NoAns: unanswerable according to the gold.