Table 6. 5-fold cross-validation NE results.
Exact Span Match | Relaxed Span Match | |||||||
---|---|---|---|---|---|---|---|---|
Category | BL-Ex | SM-Ex | FM-Ex | UL-Ex | BL-Rel | SM-Rel | FM-Rel | UL-Rel |
Environmental (P) | 0.66 | 0.63 | 0.64 | 0.65 | 0.79 | 0.76 | 0.77 | 0.79 |
Environmental (R) | 0.39 | 0.41 | 0.41 | 0.40 | 0.46 | 0.50 | 0.49 | 0.48 |
Environmental (F) | 0.49 | 0.50 | 0.50 | 0.49 | 0.58 | 0.60 | 0.60 | 0.60 |
Condition (P) | 0.78 | 0.80 | 0.81 | 0.81 | 0.88 | 0.89 | 0.90 | 0.89 |
Condition (R) | 0.68 | 0.75 | 0.74 | 0.74 | 0.76 | 0.83 | 0.82 | 0.82 |
Condition (F) | 0.73 | 0.78 | 0.78 | 0.77 | 0.82 | 0.86 | 0.85 | 0.86 |
Subject (P) | 0.70 | 0.79 | 0.80 | 0.80 | 0.87 | 0.87 | 0.87 | 0.87 |
Subject (R) | 0.70 | 0.70 | 0.71 | 0.70 | 0.76 | 0.76 | 0.77 | 0.76 |
Subject (F) | 0.74 | 0.74 | 0.75 | 0.75 | 0.81 | 0.81 | 0.82 | 0.81 |
Sign_or_Symptom (P) | 0.79 | 0.79 | 0.81 | 0.81 | 0.86 | 0.86 | 0.87 | 0.87 |
Sign_or_Symptom (R) | 0.56 | 0.62 | 0.61 | 0.60 | 0.61 | 0.66 | 0.65 | 0.65 |
Sign_or_Symptom (F) | 0.66 | 0.70 | 0.70 | 0.69 | 0.71 | 0.75 | 0.75 | 0.74 |
Anatomical (P) | 0.80 | 0.80 | 0.82 | 0.80 | 0.88 | 0.88 | 0.89 | 0.88 |
Anatomical (R) | 0.66 | 0.70 | 0.70 | 0.72 | 0.72 | 0.77 | 0.76 | 0.78 |
Anatomical (F) | 0.73 | 0.75 | 0.76 | 0.76 | 0.79 | 0.82 | 0.82 | 0.83 |
Therapeutic_or_Inv (P) | 0.71 | 0.73 | 0.72 | 0.74 | 0.79 | 0.79 | 0.80 | 0.81 |
Therapeutic_or_Inv (R) | 0.48 | 0.52 | 0.52 | 0.52 | 0.53 | 0.57 | 0.57 | 0.57 |
Therapeutic_or_Inv (F) | 0.57 | 0.61 | 0.60 | 0.61 | 0.64 | 0.66 | 0.67 | 0.67 |
Biological_Entity (P) | 0.91 | 0.91 | 0.91 | 0.91 | 0.93 | 0.94 | 0.93 | 0.93 |
Biological_Entity (R) | 0.57 | 0.57 | 0.61 | 0.62 | 0.58 | 0.66 | 0.62 | 0.63 |
Biological_Entity (F) | 0.70 | 0.70 | 0.73 | 0.74 | 0.72 | 0.78 | 0.74 | 0.75 |
TOTAL (P) | 0.77 | 0.77 | 0.79 | 0.78 | 0.86 | 0.85 | 0.86 | 0.86 |
TOTAL (R) | 0.58 | 0.63 | 0.63 | 0.63 | 0.65 | 0.70 | 0.69 | 0.70 |
TOTAL (F) | 0.67 | 0.70 | 0.70 | 0.70 | 0.74 | 0.77 | 0.77 | 0.77 |
BL = Baseline; SM = Selective MetaMap; FM = Full MetaMap; UL = UMLS Lookup; Ex = Exact span matching; Rel = Relaxed span matching; P = Precision; R = Recall; F = F-Score. The best Precision, Recall and F-Score results for each category are shown in bold type.