Skip to main content
. 2020 Feb 4;3(1):16–20. doi: 10.1093/jamiaopen/ooz072

Table 1.

Accuracy of differently trained models on the test set

Model Full test set: 13 545 QAs
Test HasAns: 12 376 QAs
Test NoAns: 1169 QAs
Exact Partial Exact Partial Exact Partial
BERTbase + emrQAwhy 0.633 0.688 0.599 0.659 0.995 0.995
BERTbase + SQuADwhy + emrQAwhy 0.660 0.728 0.628 0.703 0.994 0.994
BERTbase + i2b2Notespre + emrQAwhy 0.663 0.718 0.631 0.692 0.997 0.997
BERTbase + ClinBERTpre + emrQAwhy 0.695 0.744 0.666 0.720 0.994 0.994
BERTbase + ClinBERTpre + SQuADwhy + emrQAwhy 0.700 0.757 0.672 0.735 0.995 0.995
BERTbase + ClinBERTpre + SQuADwhy + emrQAwhy (10 epochs) 0.707 0.760 0.679 0.737 0.999 0.999

Abbreviations: HasAns: answerable according to the gold; NoAns: unanswerable according to the gold.