Table 2.
Variable name | Spans | NLP model (Gate) | Fine-tuned NLP model (Prodigy) | ||||
Precision | Recall | F1 | Precision | Recall | F1 | ||
Dosage | 75 | 0.84 | 1.0 | 0.91 | 0.94 | 0.93 | 0.93 (0.02) |
Route | 6 | 0.79 | 1.0 | 0.88 | 0.40 | 0.33 | 0.36 (0.52) |
Medication | 152 | 0.69 | 0.92 | 0.79 | 0.90 | 0.91 | 0.90 (0.11) |
Frequency | 88 | 0.73 | 0.64 | 0.68 | 0.80 | 0.70 | 0.75 (0.07) |
Duration | 60 | 0.51 | 0.71 | 0.59 | 0.59 | 0.66 | 0.62 (0.03) |
History | 9 | 1.0 | 0.40 | 0.57 | 0.00 | 0.00 | 0.00 (0.57) |
Form | 24 | 0.60 | 0.38 | 0.46 | 0.84 | 0.91 | 0.86 (0.40) |
Diagnosis | 33 | 1.0 | 0.17 | 0.29 | 0.53 | 0.48 | 0.50 (0.21) |
Assessment scales | 6 | 1.0 | 0.13 | 0.22 | 0.66 | 0.33 | 0.44 (0.22) |
Symptoms | 272 | 0.93 | 0.08 | 0.14 | 0.65 | 0.71 | 0.68 (0.54) |
Time info | 173 | 1.0 | 0.03 | 0.06 | 0.73 | 0.78 | 0.75 (0.72) |
Adverse effects | 10 | 0.00 | 0.00 | 0.00 | 0.25 | 0.10 | 0.14 (0.14) |
Response | 30 | 0.00 | 0.00 | 0.00 | 0.47 | 0.30 | 0.36 (0.36) |
TOTAL | 0.73 | 0.43 | 0.54 | 0.74 | 0.73 | 0 .74 ( 0 .20) |
Total precision, recall and F1 are a micro-averaged measures
NLP, natural language processing.