Table 2. Performance of the drug NLP pipeline in manual validation.
| Drug | Accuracy | Precision | Recall | F1 | P | FN | FP | TN | TP |
|---|---|---|---|---|---|---|---|---|---|
| Warfarin | 0.94 | 0.87 | 0.97 | 0.92 | 69 | 2 | 10 | 121 | 67 |
| Aspirin | 0.96 | 0.90 | 0.98 | 0.94 | 62 | 1 | 7 | 131 | 61 |
| Rivaroxaban | 1.00 | 1.00 | 0.95 | 0.98 | 22 | 1 | 0 | 178 | 21 |
| Clopidogrel | 1.00 | 1.00 | 0.94 | 0.97 | 17 | 1 | 0 | 183 | 16 |
| Apixaban | 1.00 | 1.00 | 1.00 | 1.00 | 13 | 0 | 0 | 187 | 13 |
| Average | 0.98 | 0.95 | 0.97 | 0.96 |
Discharge summaries were selected at random (n = 200) and manually annotated for the prescription of the 10 drugs detected by the pipeline. Performance for the 5 drugs with > 10 positive examples in manual annotation is shown. P = total positive examples in manual annotation, FN = false negative, FP = false positive, TN = true negative, TP = true positive.