Skip to main content
PMC home page
Proceedings of the AMIA Symposium logoLink to Proceedings of the AMIA Symposium
. 1999:864–867.

A large-scale evaluation of terminology integration characteristics.

F S McDonald 1, C G Chute 1, P V Ogren 1, D Wahner-Roedler 1, P L Elkin 1
PMCID: PMC2232475  PMID: 10566483

Abstract

OBJECTIVE: To describe terminology integration characteristics of local specialty specific and general vocabularies in order to facilitate the appropriate inclusion and mapping of these terms into a large-scale terminology. METHODS: We compared the sensitivity, specificity, positive predictive value, and positive likelihood ratios for Automated Term Composition to correctly map 9050 local specialty specific (dermatology) terms and 4994 local general terms to UMLS using Metaphrase. Results were systematically combined among exact matches, semantic type filtered matches, and non-filtered matches. For the general set, an analysis of semantic type filtering was performed. RESULTS: Dermatology exact matches defined a sensitivity of 51% (57% for general terms) and a specificity of 86% (92% general terms). Including semantic type filtered matches increased sensitivity (75% dermatology; 88% general); as did inclusion of non-filtered matches (98% and 99%). These inclusions correspondingly decreased specificity (filtered: 82% and 74%; non-filtered: 52% and 32%). Positive predictive values for exact matches (93.0% dermatology, 97.6% general) were improved by small but significant (p < 0.001) margins by including filtered matches (95.1% dermatology, 98.4% general) but decreased with non-filtered matches (89.2% dermatology, 87.8% general). Adding additional semantic types to the filtering algorithm failed to improve the positive predictive value or the positive likelihood ratio of term mapping, in spite of a 2.3% improvement in sensitivity. CONCLUSIONS: Automated methods for mapping local "colloquial" terminologies to large-scale controlled health vocabulary systems are practical (ppv 95% dermatology, 98% general). Semantic type filtering improves specificity without sacrificing sensitivity and yields high positive predictive values in every set analyzed.

Full text

PDF
864

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  1. Chute C. G., Cohn S. P., Campbell J. R. A framework for comprehensive health terminology systems in the United States: development guidelines, criteria for selection, and public policy implications. ANSI Healthcare Informatics Standards Board Vocabulary Working Group and the Computer-Based Patient Records Institute Working Group on Codes and Structures. J Am Med Inform Assoc. 1998 Nov-Dec;5(6):503–510. doi: 10.1136/jamia.1998.0050503. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Chute C. G., Elkin P. L. A clinically derived terminology: qualification to reduction. Proc AMIA Annu Fall Symp. 1997:570–574. [PMC free article] [PubMed] [Google Scholar]
  3. Cimino J. J. Desiderata for controlled medical vocabularies in the twenty-first century. Methods Inf Med. 1998 Nov;37(4-5):394–403. [PMC free article] [PubMed] [Google Scholar]
  4. Côté R. A., Robboy S. Progress in medical information management. Systematized nomenclature of medicine (SNOMED). JAMA. 1980 Feb 22;243(8):756–762. doi: 10.1001/jama.1980.03300340032015. [DOI] [PubMed] [Google Scholar]
  5. Elkin P. L., Bailey K. R., Chute C. G. A randomized controlled trial of automated term composition. Proc AMIA Symp. 1998:765–769. [PMC free article] [PubMed] [Google Scholar]
  6. Elkin P. L., Mohr D. N., Tuttle M. S., Cole W. G., Atkin G. E., Keck K., Fisk T. B., Kaihoi B. H., Lee K. E., Higgins M. C. Standardized problem list generation, utilizing the Mayo canonical vocabulary embedded within the Unified Medical Language System. Proc AMIA Annu Fall Symp. 1997:500–504. [PMC free article] [PubMed] [Google Scholar]
  7. Evans D. A., Rothwell D. J., Monarch I. A., Lefferts R. G., Cote R. A. Toward representations for medical concepts. Med Decis Making. 1991 Oct-Dec;11(4 Suppl):S102–S108. [PubMed] [Google Scholar]
  8. Tuttle M. S., Olson N. E., Keck K. D., Cole W. G., Erlbaum M. S., Sherertz D. D., Chute C. G., Elkin P. L., Atkin G. E., Kaihoi B. H. Metaphrase: an aid to the clinical conceptualization and formalization of patient problems in healthcare enterprises. Methods Inf Med. 1998 Nov;37(4-5):373–383. [PubMed] [Google Scholar]

Articles from Proceedings of the AMIA Symposium are provided here courtesy of American Medical Informatics Association

RESOURCES