Concept Coverage Analysis of Ophthalmic Infections and Trauma among the Standardized Medical Terminologies SNOMED-CT, ICD-10-CM, and ICD-11

Abstract

Purpose

Widespread electronic health record adoption has generated a large volume of data and emphasized the need for standardized terminology to describe clinical concepts. Here, we undertook a systematic concept coverage analysis to determine the representation of clinical concepts in ophthalmic infection and ophthalmic trauma among standardized medical terminologies, including the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT), the International Classification of Diseases (ICD) version 10 with clinical modifications (ICD-10-CM), and ICD version 11 (ICD-11).

Design

Extraction of concepts related to ophthalmic infection and ophthalmic trauma and structured search in terminology browsers.

Data Sources

The American Academy of Ophthalmology Basic and Clinical Science Course (BCSC), SNOMED-CT, and ICD-10-CM terminologies from the Observational Health Data Sciences and Informatics Athena browser, and the ICD-11 terminology browser.

Methods

Concepts pertaining to ophthalmic infection and ophthalmic trauma were extracted from the 2022 BCSC free text and index terms. We searched terminology browsers to identify corresponding codes and classified the extent of semantic alignment as equal, wide, narrow, or unmatched in each terminology. The overlap of equal concepts in each terminology was represented in a Venn diagram.

Main Outcome Measures

Proportions of clinical concepts with corresponding codes at various levels of semantic alignment.

Results

A total of 443 concepts were identified: 304 concepts related to ophthalmic infection and 139 concepts related to ophthalmic trauma. The SNOMED-CT had the highest proportion of equal coverage, with 82.0% (249 of 304) among concepts related to ophthalmic infection and 82.0% (115 of 139) among concepts related to ophthalmic trauma. Across all concepts, 28% (124 of 443) were classified as equal in ICD-10-CM and 52.8% (234 of 443) were classified as equal in ICD-11.

Conclusions

The SNOMED-CT had significantly better semantic alignment than ICD-10-CM and ICD-11 for ophthalmic infections and ophthalmic trauma. This demonstrates opportunity for continuing advancement of representation of ophthalmic concepts in standardized medical terminologies.

The widespread adoption of electronic health records (EHRs) and health information technology infrastructure has highlighted the need for standardized medical language and terminology to enable clinical interoperability across diverse information systems.^1,2 From a clinical standpoint, data standards are important for communicating information about patients across different systems and institutions. Furthermore, the use of standard terminologies also has important research applications, where standard concepts or codes can be used to define patient cohorts and uniformly evaluate outcomes across different data sources.^3,4 Other applications include building logic for clinical decision support interventions and natural language processing.^1,5,6

Standard terminologies are often employed in research databases that draw data from EHR systems and make them amenable for secondary data analysis.⁷ These enable important findings to be generated for public health reporting, epidemiological analyses, and health services research. Some prominent examples include the Medical Information Mart for Intensive Care III Database, a cohort of > 60 000 critical care patients, and the American College of Surgeons trauma registry.⁸ The use of these databases can help shed light on ophthalmic conditions encountered in emergent/inpatient settings, which are not as well represented in other ophthalmology “big data” sources, such as the IRIS® Registry (Intelligent Research in Sight), which primarily describes outpatient/ambulatory care. Ophthalmic infections and ophthalmic trauma are common reasons for admission to inpatient and critical care units and are associated with high morbidity and potential vision loss.⁵

Given the growing interest in advancing data standards in the ophthalmic community and the relative scarcity of prior studies examining standards representation in ophthalmic conditions encountered in inpatient settings,^9,10 we decided to pursue a systematic concept coverage analysis to determine the extent of representation of concepts related to ophthalmic infections and ophthalmic trauma among leading standardized medical terminologies in the United States.

Methods

This study entailed a review of clinically relevant concepts from medical texts and did not involve any human subjects. Therefore, it did not require institutional review board approval.

Identification of Relevant Concepts

We identified relevant concepts relating to ophthalmic infection and ophthalmic trauma by reviewing the 2022 Basic and Clinical Science Course (BCSC) textbooks from the American Academy of Ophthalmology.¹¹ This series of textbooks represents a core curriculum for ophthalmologists in the United States and forms the basis of annual Ophthalmic Knowledge Assessment Program in-service exams for ophthalmology residents across the country. In addition, the BCSC is generally considered testable content for written board examinations administered by the American Board of Ophthalmology. The BCSC covers a wide range of anatomic regions and types of pathology. Given its official nature as a publication of the national specialty society and its recognition as a core source of knowledge for ophthalmologists in training and practice, we designated this as our primary data source for the identification of relevant concepts. This approach of using medical texts to identify clinical concepts has been successfully employed by investigators in other fields as well.12, 13, 14, 15

To identify concepts, a primary physician grader (K.T.) systematically reviewed the free text narrative content of the BCSC textbooks and recorded any mention of concepts related to ophthalmic infection or ophthalmic trauma. We maintained a running list of all candidate concepts, which was then supplemented with an additional review of index terms of each textbook. The review of index terms (which by definition reflected specific annotation of importance by the textbook editors) ensured that no important terms were missed during the free text review. Two ophthalmologists (F.K. and S.L.B.) additionally reviewed the list of concepts to verify completeness and correct classification as ophthalmic infection or ophthalmic trauma.

Selection of Standardized Medical Terminologies

With input from a terminology expert (M.H.), we selected the following standardized medical terminologies to analyze: the Systemized Nomenclature of Medicine Clinical Terms (SNOMED-CT, version 20220901) and the International Classification of Diseases (ICD), including versions ICD-10-CM (version 10 with clinical modifications, 2023) and ICD-11 (version 11, 2022).

The SNOMED-CT was created in 1991 after fusing SNOMED and Clinical Terms version 3 and is considered one of the most comprehensive clinical terminologies in the world. It contains > 350 000 concepts as well as millions of relationship definitions between concepts.¹⁶ It is maintained and published by SNOMED-CT International, a nonprofit organization spanning 39 countries.¹⁷ The National Library of Medicine manages the United States version of SNOMED-CT, and it has been recommended as the terminology of choice for representing clinical concepts by the American Academy of Ophthalmology.¹⁸ We aimed to evaluate SNOMED-CT specifically, given its broad use in EHR systems and its recognized importance among the ophthalmology community.^19,20

The World Health Organization manages ICD, which for more than a century has been the coding system of choice for generating statistics regarding mortality and disease burden across the globe. It has been translated into 34 languages and is used in approximately 120 countries.^21,22 Its use has been critical for billing/payment and health services planning, operations, and research. The newest revision is version 11 (ICD-11), which was introduced by World Health Organization in May 2019 and came into effect in the United States on January 1, 2022. Because our analysis occurred in the spring/summer of 2022, we decided to analyze both ICD-10-CM, the version of ICD most commonly used in current practice in the United States, and additionally ICD-11,²³ which will likely be used for health statistics reporting in the coming years given its recent release at the time of analysis. Because ICD-11 is the newest version and has not been fully adopted in the United States, the “Clinical Modifications” for ICD-11 have not yet been released, unlike prior versions ICD-10-CM and ICD-9-CM.24, 25, 26

Coverage Analysis of Concepts across Terminologies

For each concept identified in the BCSC relating to ophthalmic infection or trauma, we searched for related concepts in each of the standardized medical terminologies listed above using the ICD-11 browser of the World Health Organization and ICD-10-CM and SNOMED-CT using the Athena browser,27, 28, 29 an online web application for browsing standardized vocabularies hosted by the Observational Health Data Sciences and Informatics organization.

We recorded the matching concept or code from each of the standard terminologies resulting from the search. If a concept was missing or not available from a given terminology, we recorded the concept as unmatched. If there was a matching code present, the extent of semantic alignment was determined as equal, wide, or narrow. Equal was defined in our study as a concept code mapping that was represented in equal fashion as the source clinical concept by the standardized terminology. Exact text matching was not required as long as meaning was equivalent. For instance, “adenoviral keratitis” and “keratitis due to adenovirus” were considered equal. The semantic alignment was considered wide if the related code in the terminology did not fulfill all the information and had a broader characterization in the terminology compared to the source clinical concept, reflecting some information loss. For example, for the source clinical concept “adenoviral keratitis,” the code for “infectious keratitis” in ICD-11 would be considered wide. We designated codes as narrow if they fulfilled all the information compared to the concept but with additional specification. However, this additional information may introduce potentially inaccurate representation, if that level of granularity was not indicated in the original source concept. Of note, we did not include laterality information (right eye, left eye, both eyes) in the semantic alignment analysis, as we were interested in alignment of clinical concepts other than laterality. In addition, to quantify the extent of wide matches that were potentially subjective, we calculated the proportion of concepts for which multiple concepts in the BCSC were mapped to a single standard concept in each terminology system.

Further, for ICD-11 we introduced the term subgroup as another way to characterize the mappings for some terms in this study. This designation arose because we found matching codes in the ICD-11 browser which were listed within a broader category (i.e., could be considered a “subgroup” of a wider code), but the subgroup with an equal meaning to the source concept had not been formally codified yet (e.g., no code number attached to the subgroup although it was specifically delineated in the browser). Because we wanted to reflect the fact that these concepts were specifically delineated in ICD-11, even if not formally codified yet, we labeled these as subgroup instead of wide.

The mappings and designations of semantic alignment were conducted in a tiered process, with primary mapping undertaken by 2 fellowship-trained graders (F.K., K.T.), secondary review by a faculty ophthalmologist (S.B.), and discrepancies reviewed by the entire group to establish consensus.

Statistical Analysis

We summed the total number of concepts identified in the BCSC about ophthalmic infection and ophthalmic trauma. To generate the proportions of coverage, we summed the number of concepts in each terminology classification and divided it by the total number of concepts in each group (ophthalmology infection or trauma). Correspondingly, this allowed us to derive the level of semantic alignment among each terminology. Then, we constructed a Venn diagram to illustrate the number of equal concepts overlapping between the terminologies. Statistical analyses and data visualizations were performed in R (version 2023.02.0 + 421).

Results

Identification of Relevant Concepts

In total, we identified 443 concepts, with 304 concepts related to ophthalmic infection and 139 concepts related to ophthalmic trauma based on a detailed review of free text and index terms from the BCSC. Specific concepts are listed in Tables S2 and S3.

Concept Coverage in Standardized Terminologies for Ophthalmic Infections and Trauma

Table 1 summarizes the coverage and semantic alignment among the selected terminologies. When comparing SNOMED-CT and ICD, SNOMED-CT had a better semantic alignment in both ophthalmic infections (82.0%) and trauma (82.0%).

Table 1.

Concept Coverage and Semantic Alignment for Ophthalmic Infection and Trauma across Standardized Medical Terminologies

	SNOMED-CT	ICD-10-CM	ICD-11
Infection (N = 304)
Equal	249 (82%)	78 (26%)	76 (25%) and additional 89 (29%) in subgroup
Narrow	13 (4%)	31 (10%)	13 (4%)
Wide	42 (14%)	195 (64%)	126 (41%)
Unmatched	0	0	0
Trauma (N = 139)
Equal	115 (82%)	46 (33%)	36 (26%) and additional 33 (24%) in subgroup
Narrow	6 (4%)	26 (19%)	5 (4%)
Wide	18 (13%)	67 (48%)	65 (46%)
Unmatched	1 (1%)	1 (1%)	1 (1%)

ICD-10-CM = International Classification of Diseases, version 10 Clinical Modification; ICD-11 = International Classification of Diseases, version 11; SNOMED-CT = Systemized Nomenclature of Medicine Clinical Terms.

In contrast, the proportions of equal matching concepts in ICD-10-CM and ICD-11 for ophthalmic infection and ophthalmic trauma were much lower (Table 1). Across all concepts for both ophthalmic infection and trauma, 28% (124 of 443) were classified as equal in ICD-10-CM, and 25.2% (112 of 443) were classified as equal in ICD-11. Interestingly, ICD-11 contained some terms that matched the source concepts equally but were not yet codified and were listed as subgroups under a broader code. If these codes were eventually codified individually, a total of 52.8% (234 of 443) concepts would be considered to have equal matches in ICD-11.

Regarding wide mappings, multiple concepts from the BCSC could be mapped to a single, less granular, standard concept. This applied to 61.3% (272 of 443) of the concepts when mapping to ICD-11, 49% (217 of 443) of concepts when mapping to ICD-10-CM, and 10.3% (46 of 443) of concepts when mapping to SNOMED.

Figure 1 depicts the level of overlap in equal concepts among the 3 terminology systems studied. There were 18% (22) mutually equal concepts in ophthalmic trauma, and 15% (40) mutually equal concepts in ophthalmic infection across all 3 terminologies.

Figure 1.

Venn diagrams of number of concepts in ophthalmic trauma (top) and infection (bottom) from the 2022 Basic and Clinical Science Course with equal/matching codes in standardized medical terminologies.

Discussion

In this study, we conducted a systematic identification of concepts related to ophthalmic infection and ophthalmic trauma based on a detailed review of free text and index terms in a set of medical textbooks broadly used by ophthalmologists and then analyzed the coverage of these concepts in standardized terminologies. Our key findings were, (1) there were still some gaps in coverage for ophthalmic infection and trauma among all of the standardized terminologies evaluated and (2) ICD-10-CM and ICD-11, despite being the newest terminologies examined, demonstrated the most gaps in semantic alignment.

First, all the terminologies we examined demonstrated some gaps in content coverage for ophthalmic infections and ophthalmic trauma. Although SNOMED-CT had approximately 82% of the concepts represented, this meant that there were still about 18% (19 [4%] narrow and 60 [14%] wide matches) of the concepts described in the BCSC without equal matching concepts in SNOMED-CT (Table 1). This represents a substantial gap in coverage, particularly given previous recommendations from the American Academy of Ophthalmology for SNOMED-CT to be the terminology of choice for representing clinical concepts in ophthalmology. This recommendation was based on a previous analysis of the coverage of ophthalmic clinical concepts in SNOMED-CT conducted by Chiang et al¹⁸ in 2005, which showed that SNOMED-CT had significantly better coverage of ophthalmic concepts than other terminologies studied, including ICD-9-CM, which was the most recent version of ICD at that time. Similarly, here we found that SNOMED-CT continues to demonstrate superior coverage of ophthalmic concepts compared to ICD-10-CM and ICD-11. The SNOMED-CT has also emerged as the preferred terminology in other clinical domains outside of ophthalmology, based on better content coverage, clinical orientation, flexible data entry, and retravel capabilities.^21,22 Nevertheless, the finding that 18% of clinical concepts in ophthalmic trauma and ophthalmic infection are still not coded equally in SNOMED-CT highlights opportunities for ongoing improvements in coverage.

The gaps seen in coverage for Unified Medical Language System (UMLS) and SNOMED-CT were significantly less than the levels of representation in ICD-10-CM and ICD-11, in which approximately 50% of the concepts identified in the BCSC related to ophthalmic infection and ophthalmic trauma did not have equal matches. Based on a qualitative review of the clinical concepts without equal matches, the lack of equal representation did not appear to correspond to specific disease etiologies or to vary substantially by anatomical regions. One main driver appeared to be mismatches in granularity. For example, the SNOMED included codes with specific infectious etiologies, such as “Acanthamoeba endophthalmitis,” whereas ICD included “endophthalmitis” and “parasitic endophthalmitis,” but not specifically “Acanthamoeba endophthalmitis.”

Another potential factor contributing to the reduced levels of equal coverage is varying approaches to precoordination versus postcoordination of codes. With a precoordinated approach, concepts are prespecified and can be highly specific and granular; multiple logical concepts are combined into a single code before storage or message transmission.³⁰ In contrast, postcoordination allows for making new concepts by adding modifiers to existing concepts, which means that concepts are kept as multiple attributes/codes and combined after storage or transmission of an instance.¹⁸ One straightforward example of postcoordination is laterality, where a single code or concept can be combined with modifiers of “right eye” or “left eye” instead of having 2 separate prespecified codes. This cuts down on the total number of codes that need to be maintained over time. Although laterality did not apply to our analysis since we did not incorporate laterality into any of the source concepts (see Tables S2 and S3), some concepts may not have had an equivalent match during our analysis if a terminology was designed with a postcoordination approach in mind. This is particularly salient for ICD-11, where reliance on postcoordination has been promoted as a strategy for reducing the need for revision.²⁴ In general, ICD-11 codes were less granular than ICD-10 and tended to describe classes/categories of conditions rather than specific conditions.

Most of the concepts in ICD terminologies were defined as wide codes. For example, there was no equal match for “retinitis,” and the closest match was “chorioretinal inflammation” in both ICD terminologies. Furthermore, in ICD-11, some concepts were identified as “subgroups,” as these concepts were not coded yet, but were available in the ICD-11 browser. For example, “syphilitic retinitis” was a source concept from the reference text and was listed as a subgroup of syphilitic disease in ICD-11 (listed but not codified individually). This has implications, given that ICD codes are the most common codes used for billing, and these codes are subsequently used for secondary EHR data analysis. Several prior studies have used ICD codes to define cohorts and outcomes in ophthalmic infections and trauma.31, 32, 33, 34, 35 Ophthalmic researchers should be aware of the implications of the lack of granularity in some ICD codes and consider that when determining study methodologies. Several EHR systems include mapping to SNOMED-CT,³⁶ which provides significantly higher proportions of equal representation of ophthalmic clinical concepts and may be more preferable when planning research studies. However, some well-known databases, such as the Medical Information Mart for Intensive Care (MIMIC) database and the American College of Surgeons Trauma Registry, use diagnosis codes based on ICD rather than SNOMED. Researchers should be aware of the limitations of ICD in terms of ophthalmic coverage, particularly those interested in studying ophthalmic infections or trauma. However, it should be noted that ICD-11 is new and with formal adoption in the United States in the future, we would anticipate “Clinical Modifications” to be developed in a manner similar to what has been done for prior versions of ICD. This would undoubtedly increase the granularity and representation of concepts for future use.

Limitations

Our study had some limitations. First, we used a single data source (BCSC textbooks) to generate the list of concepts. We may have been able to generate a broader array of concepts had we also searched through additional data sources such as peer-reviewed publications, other textbooks, or clinical notes. However, we decided to use the BCSC textbooks given their recognition as an official source of information for the specialty and their detailed vetting by a range of leading ophthalmologists. Further, using reference texts as a source of clinical concepts has been leveraged in other fields for similar analysis. Likewise, while some domains have a standardized nomenclature to guide formation of the list of source concepts (e.g., the Standardized Uveitis Nomenclature system for uveitis), unfortunately, to our knowledge, there is not standardized nomenclature in the domains of ophthalmic infection or trauma that are widely used for EHR data. We also acknowledge that the current analysis has a United States focus, both in terms of the data source used for concept identification and in some of the standardized terminologies analyzed. This may limit generalizability to other countries, which have specific and separate terminologies and coding systems that were not examined here. However, the terminologies that were included do have broad international use, and future investigations may extend upon this work to investigate ophthalmic concept representation in other terminologies. Furthermore, the hierarchical structure of the terminologies, where some concepts might be parent concepts of others, may cause percentages to be misleading, although our use of extent of semantic alignment (equal, wide, narrow, unmatched) may help depict some of those relationships better than a binary yes/no designation. Finally, there may have been gaps in coverage due to the search techniques themselves, as we used freely available online terminology browsers based on search engines where the underlying search algorithms were opaque.

Conclusions

In summary, here we analyzed the representation of clinical concepts related to ophthalmic infection and ophthalmic trauma in standardized terminologies and found better semantic alignment in SNOMED-CT than ICD-10-CM and ICD-11. This has important implications for future research in these patient populations and for the utility of databases in trauma and critical care that employ these terminologies. Awareness of these gaps will help inform future efforts in standards development and also potentially guide ophthalmic researchers in designing studies for these vision-threatening conditions.

Manuscript no. XOPS-D-22-00237.