Table 5.
Target term (n-gram) |
Candidate terms (20 top-ranked n-grams) from CBOW neural embeddings for a target term | Candidate terms (20 top-ranked n-grams) from Skip-gram neural embeddings for a target term | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
MetaMap Experiment 1 | MetaMap Experiment 2 | MetaMap Experiment 1 | MetaMap Experiment 2 | |||||||||
P | R | F | P | R | F | P | R | F | P | R | F | |
anaemia | 84.21 | 94.12 | 88.89 | 95.00 | 100.00 | 97.44 | 94.74 | 94.74 | 94.74 | 95.00 | 100.00 | 97.44 |
arthritis | 93.33 | 73.68 | 82.35 | 100.00 | 75.00 | 85.71 | 94.44 | 89.47 | 91.89 | 100.00 | 90.00 | 94.74 |
asthma | 100.00 | 90.00 | 94.74 | 100.00 | 95.00 | 97.44 | 89.47 | 94.44 | 91.89 | 100.00 | 100.00 | 100.00 |
ckd | 68.75 | 73.33 | 70.97 | 100.00 | 95.00 | 97.44 | 62.50 | 71.43 | 66.67 | 100.00 | 100.00 | 100.00 |
diabetes | 76.47 | 81.25 | 78.79 | 100.00 | 90.00 | 94.74 | 88.89 | 88.89 | 88.89 | 94.74 | 94.74 | 94.74 |
epilepsy | 100.00 | 90.00 | 94.74 | 100.00 | 95.00 | 97.44 | 100.00 | 90.00 | 94.74 | 100.00 | 95.00 | 97.44 |
glaucoma | 87.50 | 77.78 | 82.35 | 94.74 | 94.74 | 94.74 | 93.33 | 73.68 | 82.35 | 94.74 | 94.74 | 94.74 |
heart_failure | 73.68 | 93.33 | 82.35 | 95.00 | 100.00 | 97.44 | 84.21 | 94.12 | 88.89 | 100.00 | 100.00 | 100.00 |
hypertension | 71.43 | 62.50 | 66.67 | 100.00 | 95.00 | 97.44 | 72.22 | 86.67 | 78.79 | 100.00 | 100.00 | 100.00 |
obesity | 75.00 | 100.00 | 85.71 | 85.00 | 100.00 | 91.89 | 84.21 | 94.12 | 88.89 | 89.47 | 94.44 | 91.89 |
osteoarthritis | 94.74 | 94.74 | 94.74 | 100.00 | 95.00 | 97.44 | 85.00 | 100.00 | 91.89 | 85.00 | 100.00 | 91.89 |
84.10 | 84.61 | 83.85 | 97.25 | 94.07 | 95.38 | 86.27 | 88.87 | 87.24 | 96.27 | 97.17 | 96.63 |
The table shows the performance of MetaMap in Experiment 1 (applying MetaMap to the candidate terms) and Experiment 2 (short form detection and expansion into long form before applying MetaMap to the candidate terms) for each target term (n-gram for a well-known medical condition). The candidate terms are a list of the 20 top-ranked terms (highest cosine value) obtained from the created neural embeddings with CBOW or Skip-gram taking the vector for a target term. The last row shows the average of each evaluation measure over all 11 medical conditions under study to get an overall measure of performance (a.k.a. macro-averaging). Abbreviations: P = precision; R = recall; and F = F measure