Table 5:
Fine-grained NLP application accuracy for eligibility criteria information extraction
| Rule-based version | SVM-based version | |||
|---|---|---|---|---|
| Criterion | Recall (%) | Precision (%) | Recall (%) | Precision (%) |
| ECOG 0 | 96.9 | 96.9 | 100.0 | 100.0 |
| ECOG 1* | 85.7 | 85.7 | 85.7 | 100.0 |
| ER+ | 61.7 | 91.0 | 91.3 | 86.8 |
| ER− | 32.7 | 93.8 | 82.6 | 95.0 |
| PR+ | 80.0 | 96.8 | 96.0 | 86.6 |
| PR− | 57.5 | 90.2 | 92.1 | 86.8 |
| HER2+ | 24.0 | 100.0 | 85.0 | 89.5 |
| HER− | 31.0 | 98.0 | 90.5 | 74.5 |
| Postmenopausal | 72.1 | 88.0 | 86.89 | 84.1 |
| Not Postmenopausal** | - | - | 46.2 | 100.0 |
| T0 | 100.0 | 100.0 | 100.0 | 100.0 |
| T1 | 89.1 | 87.5 | 96.4 | 93.0 |
| T2 | 79.5 | 81.5 | 97.6 | 98.8 |
| T3* | 82.4 | 90.3 | 88.2 | 100.0 |
| N0 | 79.5 | 85.3 | 91.8 | 97.1 |
| N1 | 84.8 | 78.5 | 99.0 | 100.0 |
| N2 | 77.8 | 58.3 | 100.0 | 90.0 |
| N3 | 100.0 | 75.0 | 0.00 | 0.00 |
| M0 | 100.0 | 98.0 | 100.0 | 96.2 |
| M1 | 2.3 | 50.0 | 90.5 | 78.6 |
*
No positive instances occurred for ECOG 2–5 or T4
**
The rule-based system did not extract non-postmenopausal mentions.