Table 2.
Task and model type | Automatic transcripts AUROCa | Manually corrected transcripts AUROC | Change in AUROCb | |
Picture description | ||||
|
RFc | 0.617 | 0.687 | 0.070d |
|
GNBe | 0.662 | 0.725 | 0.063d |
|
LRf | 0.671 | 0.743 | 0.072d |
|
BERTg | 0.618 | 0.686 | 0.068d |
Experience description | ||||
|
RF | 0.503 | 0.636 | 0.133d |
|
GNB | 0.549 | 0.677 | 0.128d |
|
LR | 0.543 | 0.674 | 0.131d |
|
BERT | 0.630 | 0.650 | 0.020d |
aAUROC: area under the receiver operating characteristic curve.
bPositive change in AUROC indicates that the manually corrected transcript model outperformed the automatic transcript model.
cRF: random forest.
dIndicates P<.001.
eGNB: Gaussian naive Bayes.
fLR: logistic regression.
gBERT: Bidirectional Encoder Representations from Transformers.