TABLE 2.
Performance of the classifiers on test set A (n = 9368), B (n = 7,85) and C (n = 5812)
| Classifier | Training criteria a | Set | RCTs | Non‐RCTs | Screening reduction % | ||
|---|---|---|---|---|---|---|---|
| Precision % | Recall % | Precision % | Recall % | ||||
| RCT classifier | Include RCT in any human health domain | A | 12.3 | 99.7 | 3.4 | 85.9 | 43.3 |
| B | 8.1 | 99.7 | 2.4 | 83.5 | 50.9 | ||
| C | 11.1 | 99.2 | 2.6 | 87.5 | 43.6 | ||
| Custom 1 | Include any studies that are in the health promotion domain (all studies without Ex1 code) | A | 11.7 | 99.1 | 3.8 | 99.5 | 40.9 |
| B | 7.8 | 100% | 2.8 | 100% | 49.3 | ||
| C | 13.6 | 99.5 | 3.5 | 96.9 | 54.2 | ||
| Custom 2 | Include any RCTs, non‐RCTs or reviews in the health promotion domain (all studies without Ex1, Ex2 or Ex3 codes) | A | 16.5 | 98.8 | 5.4 | 98.6 | 58.4 |
| B | 12.2 | 99.3 | 4.3 | 97.1 | 67.6 | ||
| C | 19.6 | 98.1 | 4.9 | 93.8 | 68.6 | ||
| Custom 3 | Include any studies that are RCTs or non‐RCTs in the health promotion domain (all studies without Ex1, Ex2, Ex3 or Ex4 codes) | A | 19.7 | 98.0 | 6.3 | 96.2 | 65.4 |
| B | 14.6 | 99.0 | 5.1 | 97.1 | 72.9 | ||
| C | 23.2 | 97.8 | 5.8 | 92.7 | 73.5 | ||
a
Ex1, Ex2, Ex3, Ex4 are described in Table 1.