Table 1 –
Evaluation results of CLAMP Cancer Modules
| Type of information |
# of ent.* |
Entity only (Exact/Relaxed Matching) |
Entity and Relation | ||||
|---|---|---|---|---|---|---|---|
| Precision | Recall | F-measure | Precision | Recall | F-measure | ||
| Specimen | 310 | 0.99/0.99 | 0.99/0.99 | 0.99/0.99 | 0.97 | 0.98 | 0.98 |
| Primary-site | 351 | 0.98/0.99 | 0.98/0.99 | 0.98/0.99 | 0.98 | 0.98 | 0.98 |
| Sub-Site | 187 | 0.96/0.98 | 0.82/0.83 | 0.89/0.90 | 0.88 | 0.78 | 0.83 |
| Procedure | 339 | 0.98/0.99 | 0.98/0.99 | 0.98/0.99 | 0.97 | 0.97 | 0.97 |
| Histology | 553 | 0.91/1.00 | 0.85/0.93 | 0.88/0.97 | 0.90 | 0.85 | 0.86 |
| Tumor Grade | 92 | 0.96/1.00 | 0.88/0.91 | 0.92/0.96 | 0.91 | 0.83 | 0.86 |
| Tumor Size | 60 | 0.96/0.96 | 0.90/0.90 | 0.93/0.93 | 0.88 | 0.83 | 0.85 |
| Tumor Margin | 93 | 0.92/0.99 | 0.91/0.98 | 0.92/0.98 | 0.80 | 0.79 | 0.80 |
| Invasion | 71 | 0.92/1.00 | 0.83/0.90 | 0.87/0.95 | 0.86 | 0.78 | 0.82 |
| Biomarker | 107 | 0.95/0.99 | 0.90/0.94 | 0.92/0.96 | 0.88 | 0.84 | 0.86 |
|
(a) Performance of CLAMP-Cancer components on the VUMC test corpus * The number of each type of entities in the test corpus of 200 notes. | |||||||
| CLAMP-Cancer | MedKAT | ||||||
| Precision | Recall | F-1 | Precision | Recall | F-measure | ||
| Tumor size | 1.00 | 0.99 | 0.99 | 1.00 | 1.00 | 1.00 | |
| Dimension Extend | 1.00 | 0.99 | 0.99 | 0.99 | 1.00 | 0.99 | |
| Dimension Unit | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| Tumor Site | 0.94 | 0.89 | 0.92 | 0.96 | 0.95 | 0.96 | |
| Histology | 0.91 | 0.92 | 0.92 | 0.96 | 0.98 | 0.97 | |
| Grade | 1.00 | 0.88 | 0.94 | 0.93 | 0.97 | 0.99 | |
| Date | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | |
| (b) Performance comparison of CLAMP Cancer Modules and MedKAT on the same information extraction task from 302 pathology reports at Mayo Clinic | |||||||
*
The number of each type of entities in the test corpus of 200 notes.