Table 5.
No. | Criterion | Classifiers | Recall | Precision | F1 |
---|---|---|---|---|---|
1 | Costs | SVM | 79.26 | 83.69 | 72.43 |
CatBoost | 83.7 | 82.99 | 81.94 | ||
BERT | 74.07 | 69.89 | 70.99 | ||
2 | Benefits | SVM | 70.15 | 68.68 | 64.56 |
CatBoost | 70.15 | 67.98 | 66.8 | ||
BERT | 71.64 | 70.07 | 69.98 | ||
3 | Harms | SVM | 64.93 | 62.49 | 62.61 |
CatBoost | 69.4 | 68.36 | 65.29 | ||
BERT | 64.93 | 64.76 | 64.84 | ||
4 | Quality of the evidence | SVM | 67.41 | 66.67 | 65.39 |
CatBoost | 68.15 | 67.42 | 66.6 | ||
BERT | 64.44 | 63.51 | 63.61 | ||
5 | Disease-mongering | SVM | 82.96 | 68.83 | 75.24 |
CatBoost | 81.48 | 68.62 | 74.5 | ||
BERT | 82.96 | 68.83 | 75.24 | ||
6 | Identify conflicts of interest | SVM | 73.33 | 73.33 | 73.33 |
CatBoost | 76.3 | 76.27 | 76.27 | ||
BERT | 58.52 | 58.44 | 57.13 | ||
7 | Compare with existing alternative | SVM | 64.71 | 64.45 | 63.86 |
CatBoost | 58.09 | 57.16 | 55.17 | ||
BERT | 49.26 | 49.35 | 49.31 | ||
8 | Availability of the treatment | SVM | 68.42 | 66.48 | 64.25 |
CatBoost | 67.67 | 65.63 | 65.44 | ||
BERT | 66.17 | 62.78 | 59.4 | ||
9 | Novelty of the approach | SVM | 71.32 | 70.03 | 61.3 |
CatBoost | 73.53 | 71.98 | 68.09 | ||
BERT | 70.59 | 49.83 | 58.42 | ||
10 | Relying on a news release | SVM | 94.12 | 88.58 | 91.27 |
CatBoost | 95.59 | 95.79 | 94.32 | ||
BERT | 94.12 | 88.58 | 91.27 |