TABLE 4. Additional Evaluation Measures of the Methods on the Litcovid and Hoc datsets.
Label-based measures | Instance-based measures | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Macro-Precision | Macro-Recall | Micro-Precision | Micro-Recall | Precision | Recall | |||||||
Mean | Max | Mean | Max | Mean | Max | Mean | Max | Mean | Max | Mean | Max | |
LitCovid BioCreative | ||||||||||||
ML-Net | 0.8364 | 0.8559 | 0.7309 | 0.7632 | 0.8756 | 0.8827 | 0.8142 | 0.8227 | 0.8849 | 0.8901 | 0.8514 | 0.8591 |
Binary BERT | 0.9103 | 0.9498 | 0.8350 | 0.8690 | 0.9304 | 0.9448 | 0.8969 | 0.9173 | 0.9349 | 0.9408 | 0.9210 | 0.9381 |
Linear BERT | 0.9071 | 0.9388 | 0.8354 | 0.8701 | 0.9276 | 0.9396 | 0.8870 | 0.9093 | 0.9368 | 0.9443 | 0.9143 | 0.9308 |
LitMC-BERT (ours) | 0.9131 | 0.9226 | 0.8574 | 0.8814 | 0.9313 | 0.9366 | 0.8952 | 0.9145 | 0.9418 | 0.9480 | 0.9212 | 0.9355 |
Hoc | ||||||||||||
ML-Net | 0.7949 | 0.8356 | 0.7389 | 0.7622 | 0.7667 | 0.8053 | 0.7253 | 0.7448 | 0.8045 | 0.8296 | 0.7826 | 0.8013 |
Binary BERT | 0.8471 | 0.8644 | 0.8661 | 0.8834 | 0.8363 | 0.8562 | 0.8548 | 0.8724 | 0.8565 | 0.8735 | 0.8909 | 0.9095 |
Linear BERT | 0.8772 | 0.8930 | 0.8475 | 0.8630 | 0.8614 | 0.8812 | 0.8496 | 0.8619 | 0.8929 | 0.9116 | 0.8955 | 0.9024 |
LitMC-BERT (ours) | 0.8868 | 0.8983 | 0.8641 | 0.8910 | 0.8718 | 0.8938 | 0.8582 | 0.8787 | 0.9035 | 0.9148 | 0.9038 | 0.9193 |