A Survey of Direct Users and Uses of SNOMED CT: 2010 Status

Abstract

SNOMED CT is gaining momentum and endorsements as an international clinical terminology. However, many vendors await a clearer business case and clients’ demand. We conducted a survey of direct users of SNOMED CT to determine the current profile of users, modes of use, and attitudes towards different aspects of the terminology. A web-base survey, consisting of 43 questions was distributed in January 2010, and 215 responses were elicited. This paper summarizes findings regarding profiles of users and their SNOMED CT use. The results indicate significant use by non-researchers and by industry and government sectors. Many users are relative newcomers with less than 3 years experience with SNOMED CT, and production-related use was reported by 39% of respondents. Most users are satisfied with the level of content coverage. The results indicate that SNOMED CT has a solid footing in production systems, and that SCT is mostly used for concept searches and clinical coding.

Introduction

SNOMED CT¹ (Systematized Nomenclature of Medicine – Clinical Terms) is a large-scale comprehensive clinical healthcare terminology. It strives to offer a common terminological ground for the rapidly progressing applications in the fields of Electronic Health Records (EHR) and Clinical Information Systems (CIS). Coding systems such as the International Classification of Diseases (ICD) and other procedural terminologies have been playing a major role in the administrative and billing aspects of healthcare delivery in the United States and elsewhere in the world. However, the clinical use of standardized terminologies has been fragmented and non-comprehensive. SNOMED CT (SCT) represents an international effort to remedy these problems in the face of the inevitable, rapid, and about to be mandated progress related to clinical infrastructure aspects of healthcare delivery, both for inpatient and outpatient electronic systems.

Since 2007, the International Health Terminology Standards Development Organization (IHTSDO¹), has owned and is tasked with the continued development, promotion and implementation of SCT and its derivative products. IHTSDO has representation from member nations, vendors, standards organizations, researchers and other user groups.

While SNOMED in various incarnations has been around since 1965, a significant milestone for clinical use happened in 1999 when SNOMED RT (Reference Terminology) converged with the United Kingdom’s Clinical Terms Version 3 to create SNOMED CT. SCT contains more than 310,000 active concepts with an extensive network of attributes between them, and is implemented with Description Logic (DL). SCT has been endorsed by international and national standards organizations such as HL7, DICOM and ANSI, and by various organizations for clinical use in the context of patient medical information.

However, despite its prominent status and potential, the actual adoption of SCT in current clinical EHRs and CISs has been lagging. While a few vendors have implemented access to SCT (or subsets of SCT) in their products², actual use is rare and sporadic. Notable exceptions are Kaiser Permanente’s (KP) use of SCT as part of their Convergent Medical Terminology (CMT³), as well as the Veterans Health Administration’s Enterprise Reference Terminology (ERT⁴). However, while SCT is a core component of the CMT and ERT, actual system users do not interact with it directly.

The current profile of direct SCT users (i.e., not through systems that embed SCT) and their use of the terminology are not clear. We conducted a survey targeted at such users to provide new insight into the current, direct use of SCT and the profile of its users.

Methods

We designed a 43-question questionnaire consisting of five parts. The first part established basic demographics and employment experience information. The second part elicited responses regarding modes and purpose of use of SCT. The third part dealt with mean and frequency of access to SCT releases. A fourth part consisted of questions meant to investigate the respondent’s satisfaction with SCT content coverage and quality. The last part was designed to evaluate active users’ involvement and satisfaction with the change request mechanism of SCT. This paper covers the first three parts of the questionnaire (17 questions). To do justice to the wealth of knowledge obtained, the presentation of data from the last two parts is deferred to future publication.

The questionnaire contained 31 mandatory questions and 12 optional questions of which six were free-text questions. Ten questions contained an optional “Other” field; Eight “Other” fields were part of mandatory questions. Nine questions allowed for multiple responses. The survey was designed using freely available Google Docs tools and was presented as a Google Docs form on the web. Incomplete responses (i.e., to only some of the mandatory questions) were not accepted by the system. The questionnaire can be viewed online⁵.

The call to participate in the survey was sent by email to members of various user and discussion groups. Among the user groups were the IHTSDO user community, the UMLS user mailing list maintained by the NLM, ten AMIA discussion groups, as well as other ontology-related and nursing groups. Additionally, a mailing list was compiled based on a PubMed search of authors of SNOMED-related articles published since 2006. Each email message included a short introduction to our research center (SABOC⁶) and stressed the importance of participation. The respondents were encouraged to forward the questionnaire to their colleagues. All responses were strictly anonymous.

The request for participation was sent in three phases, each time to the same list of recipients. The initial call did not include a deadline. A second call 12 days later contained the same message, but included a deadline within seven days. The third call was a last-chance participation reminder within 24 hours of deadline expiration.

All responses were collected and summarized by Google Docs tools. Data were exported into an Excel spreadsheet for processing of information contained in “Other” fields and for further analysis. Sector analysis is displayed when appropriate.

Results

The survey was conducted between January 7^th and January 25^th, 2010. 215 complete responses (i.e., all mandatory questions answered) were elicited. The vast majority of respondents completed all non-mandatory questions as well. Figure 1 shows the number of daily responses. Of the four peaks in the graph, the first two represent the first phase. The trough between them corresponds to weekend days. The third and fourth peaks correspond to the second and third phases, respectively.

Figure 1:

Number of daily responses

Demographics, professions and employment

As can be seen in Figure 2, 65% of respondents were from North America, 28% were from Europe, 4% from Australia, and the rest from countries in Asia and South America. Researchers/educators comprised 48% of respondents, 38% were developers/engineers, 25% were physicians, 8% were nurses and 18% were administrators/managers (Fig. 3) (exceeds 100% due to multiple choices). Of all respondents, 36% were from academic or research institutions, 28% from the industry sector and 23% worked for governments. Of those working in the industry, 52% work for software vendors, and 51% work for medium-large healthcare providers and hospitals (exceeds 100% due to multiple choices).

Figure 2:

Respondents by country

Figure 3:

Distribution of professions

Experience with SNOMED CT

Of all respondents, 15% indicated that they are new users with less than one year of experience using SNOMED CT, 30% have been using it for two to three years, 14% for four to five years and 41% for more than five years (Fig. 4). Most users use SCT relatively frequently: 37% on a daily basis, 27% weekly (Fig. 5). Of 175 respondents who provided information, 149 also indicated that more than one person in their organization uses SCT. In 32% of sites more than 10 people use SCT, at 16% of sites 6 to 10 people use it, and at 37% of sites, two to five people actively use SCT.

Figure 4:

Years of experience (# of respondents, %)

Figure 5:

Frequency of use (# of respondents, %)

Modes, purpose of use and systems

Of the respondents, 65% indicated that SCT is used within their organization for research purposes, 50% of organizations use SCT for prototype/development purposes and 39% indicated current use in production systems (Fig. 6) (exceeds 100%). As expected, the academic sector dominated research use.

Figure 6:

Modes of organizational use

The most common uses of SNOMED CT were concept search (72%) and coding of clinical data (60%) (Fig. 7). Researchers and non-researcher users had a similar pattern of use.

Figure 7:

Common uses of SNOMED CT

Figure 8 shows the distribution of systems in which SCT was reported to be used. The two most dominant systems were CIS/EMR and Research Information Systems (RIS)/Data warehouses.

Figure 8:

Systems in which SNOMED CT is used.

Means of access

CliniClue browser was the most popular tool used to access SCT (54%, exceeds 100%), as well as the most preferred one (29%) (Fig. 9). Other common tools were non-commercial home-grown solutions (25%, 12%), direct table queries (32%, 12%), and the UMLSKS (31%, 8%). Other, non-listed tools were used less frequently. Tools such as Protégé, NCImetathesaurus, and SNOB, although somewhat popular where not among the preferred ones.

Figure 9:

Preferred methods to access SCT

Use of new releases

Interestingly enough, most respondents indicated that they do not rush to load new releases of SCT. Only 14% load new releases immediately, 13% within a week, while 49% indicated that they take three or more months before they load new releases (Fig 10). Simultaneous use of more than one release of SCT was indicated by 27% of users.

Figure 10:

Access to new SCT releases

User satisfaction

On a scale of 1 to 5 (Fig. 11) where 1 is “Not at all” and 5 is “very satisfied”, 58% of respondents indicated that they are at least satisfied (4+5) with coverage, and 10% were very satisfied. Figure 11 may suggest that physicians and developers/engineers are more extreme regarding their level of satisfaction with SCT’s coverage than researchers. Category 5 respondents for physicians and developers/engineers were almost double that of researchers (14.6%, 14.6%, 7.7%). Similar findings were observed for category 1.

Figure 11:

User perceptions regarding coverage per needs

Discussion

There is a paucity of surveys regarding the use of terminologies in general and SCT in particular. A PubMed search revealed only two previous surveys regarding the UMLS^7,8 and one related to SCT² which dealt with vendors’ SCT EMR/EHR implementation preparedness. The broader spectrum evaluations of Humphreys et al.⁹ and Chute et al.¹⁰ were completed more than a decade ago. Meanwhile SNOMED has made giant strides. No updated broad data exists as to SCT’s acceptance, uses, and coverage. After distribution, we received inquiries from organizations and individuals indicating interest in this study’s findings.

While Giannangelo and Fenton’s² more recent work showed that only 20% of vendors have functioning implementations of SCT within their EMR/EHR solutions, this number is expected to rise. The current survey was not designed to evaluate current dissemination of SCT in vendors’ systems. However, it seems to indicate, based on modes of use within organizations, that SCT plays a significant role in production systems. The observation that in many organizations multiple people interact directly with SCT, and that 45% of users have been using it for less than 3 years, attests to its rising importance.

Due to the nature of SCT’s integration in third-party systems, end users may not be aware that they interact with SCT (or an extract of it) just as users of KP’s clinical system do not know that they interact with SCT (through the CMT³). The current survey was not designed to reach such users; it was aimed at users of SCT who interact with it directly through its tables or via various GUI applications. As the practical and clinical use of SCT expands, it is expected that the profile of its users will change, and the portion of developers and administrator is likely to increase.

This survey provides insight at a single, recent point in time, and may suffer from many scientific deficiencies. Due to the method of distribution of our survey, we cannot accurately calculate the response rate and control for selection and response bias. However, we consider the overall number of respondents (215) very satisfactory, especially in light of prior surveys.^2,7

While the IHTSDO, the governing body of SNOMED CT, is an international organization with 15 member nations, the majority of respondents were from the US. This may represent a selection bias in the distribution of our survey. However, it is most probably an attribute of the size and structure of the healthcare industry in the US and the intense activity regarding healthcare infrastructure, and the drive to improve healthcare delivery via IT solutions in the US.

SCT’s structure with its DL underpinning and semantics lends itself to be used by sophisticated algorithms to drive applications and for reasoning and decision support. However, most respondents indicated much simpler uses. Concept search and clinical coding are the two most common uses. These types of uses are similar to the ones reported for the UMLS^7,8 but a direct comparison between the current study and the UMLS surveys is difficult. It is expected that as the use of SNOMED CT becomes a standard and potentially mandated, such as in Canada’s Infoway, more sophisticated applications will appear.

SCT must also keep up with the advances of modern medicine. IHTSDO is working to provide publicly open, easily accessible processes for content improvement and expansion. The subjective views of SCT users expressed in the current survey convey a perception of satisfaction with SCT’s coverage. However, direct comparison to past studies^9–12 is difficult since those were objective evaluations that were mostly conducted more than a decade ago. Moreover, most users indicated that they do not rush to load new releases of SCT, raising the possibility that content enhancements are not perceived as essential. As more users interact with SCT, areas of deficiencies may reveal themselves; this may reflect on coverage satisfaction and necessitate efficient and responsive feedback mechanisms.

This study, despite its limitations, provides a modest window into the many unknowns and assumptions regarding clinical and non-clinical uses of SCT. Such information is essential for informed decision making in light of current standardization initiatives. Our work also highlights the difficulties comparing results of different surveys. We propose that an agreed-upon survey template be developed, to be used in the future as part of periodic progress tracking of terminologies.

The survey contained significant number of additional questions which will be analyzed and published in the future.

Conclusions

Although SCT’s clinical use has not been fully realized yet, it has significant presence in production systems. However, most uses remain basic and do not capitalize on the rich semantics of this terminology. Many direct users are relative newcomers and most are satisfied with the scope of coverage.