TABLE 6. Performance Evaluation on an Independent Sample of 3000 Articles in litcovid.
| Label-based measures | Instance-based measures | Time measures in the production environment (in sec) | |||||
|---|---|---|---|---|---|---|---|
| Macro-F1 | Macro-AP | Micro-F1 | Micro-AP | F1 | Accuracy | ||
| Binary BERT | 0.8116 | 0.7251 | 0.8570 | 0.7733 | 0.8719 | 0.6327 | 8912.22 (2.97 sec/doc) |
| Linear BERT | 0.8299 | 0.7449 | 0.8539 | 0.7701 | 0.8759 | 0.6354 |
1536.24 (0.51 sec/doc) |
| LitMC-BERT (ours) | 0.8413 | 0.7690 | 0.8628 | 0.7813 | 0.8862 | 0.6512 | 1673.98 (0.56 sec/doc) |
Time was measured in the production environment where only cpus are available.