Table 1.
Accuracy of automated extraction of data elements
| Manual annotation | EHR structured data using traditional approaches | EHR unstructured data using advanced approaches | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Features | Encounter occurrence | Patient-level occurrence | Encounter occurrence | Recall | Precision | F1 score | Encounter occurrence | Recall | Precision | F1 score |
| Headache | 1,996 | 1,001 | 737 | 33.1% | 87.7% | 48.1% | 1,963 | 97.8% | 91.9% | 94.8% |
| Migraine | 1,314 | 809 | 95 | 58.5% | 82.6% | 68.5% | 1,233 | 93.5% | 95.1% | 94.3% |
| Headache severity* | ||||||||||
| Headache (mild) | 104 | 92 | 0 | 0.0% | NA | NA | 66 | 74.0% | 81.8% | 77.7% |
| Headache (severe) | 236 | 194 | 0 | 0.0% | NA | NA | 197 | 81.1% | 65.5% | 72.5% |
| Severe headache descriptors | ||||||||||
| Throbbing | 132 | 107 | 0 | 0.0% | NA | NA | 87 | 75.3% | 80.5% | 77.8% |
| Stabbing | 53 | 39 | 0 | 0.0% | NA | NA | 36 | 82.1% | 88.9% | 85.3% |
| Pulsating | 86 | 65 | 0 | 0.0% | NA | NA | 53 | 70.6% | 90.6% | 79.3% |
| Associated symptoms | ||||||||||
| Nausea | 1,195 | 757 | 39 | 2.5% | 75.0% | 4.9% | 1,144 | 95.1% | 92.1% | 93.6% |
| Vomiting | 886 | 607 | 12 | 1.0% | 56.3% | 2.0% | 864 | 97.2% | 89.2% | 93.1% |
| Photophobia | 457 | 335 | 0 | 0.0% | NA | NA | 429 | 86.6% | 87.7% | 87.1% |
| Phonophobia | 319 | 232 | 0 | 0.0% | NA | NA | 281 | 87.5% | 96.8% | 91.9% |
*Sample sizes for moderate headache, disabling headache, debilitating headache, and piercing headache were below requirements for inclusion (< 20 occurrences)
EHR Electronic health record, NA Not applicable