Concept Dictionary Creation and Maintenance Under Resource Constraints: Lessons from the AMPATH Medical Record System

Abstract

The challenges of creating and maintaining concept dictionaries are compounded in resource-limited settings. Approaches to alleviate this burden need to be based on information derived in these settings. We created a concept dictionary and evaluated new concept proposals for an open source EMR in a resource-limited setting. Overall, 87% of the concepts in the initial dictionary were used. There were 5137 new concepts proposed, with 77% of these proposed only once. Further characterization of new concept proposals revealed that 41% were due to deficiency in the existing dictionary, and 19% were synonyms to existing concepts. 25% of the requests contained misspellings, 41% were complex terms, and 17% were ambiguous. Given the resource-intensive nature of dictionary creation and maintenance, there should be considerations for centralizing the concept dictionary service, using standards, prioritizing concept proposals, and redesigning the user-interface to reduce this burden in settings with limited resources.

Introduction

A central feature of many enterprise-quality electronic medical record (EMR) systems is a dictionary-driven database model. Creating and maintaining dictionaries with desirable properties¹ has been a long-standing challenge for both developers and implementers of Electronic Medical Record (EMR) systems. It is an undertaking which requires subject matter expertise, and a significant financial and human-resource commitment. For this reason, growing numbers of vendors and large organizations are developing their own enterprise-wide terminology centers.^2,3 However, these approaches are not feasible within resource-constrained environments. For such organizations, strategies and tools specifically aimed at alleviating the burden of creating well-formed dictionaries and ensuring their ‘graceful evolution’¹ are needed.

Three trends in medical informatics will increase the number of organizations with limited resources who have to deal with maintaining concept dictionaries. The first is the increasing adoption of EMRs in developing countries,⁴ and the second is the adoption in small practices in the developed world⁵ —because of limited human and financial resources, these places have a lot to gain from EMRs, but the same deficiencies make it difficult for them to manage a concept dictionary. The third trend is the growing role of open source software in medical informatics.⁶ When compared with commercial software, open source software is less likely to come with embedded vocabulary content, the support infrastructure, or dictionary maintenance tools. Open source software's low up-front costs make them attractive to institutions with limited-resources, further compounding the challenge of creating and maintaining the concept dictionary.

The medical informatics literature has not comprehensively dealt with approaches to help alleviate the burden of creating and maintaining concept dictionaries in resource-limited settings. Some of the questions that need to be addressed in this area include: (1) whether to use standard vocabularies, like SNOMED and ICD-9 in these settings — the answer to this is not very clear, as these standards perform variably in different settings,^7,8 do not contain local terms, and can be costly; (2) whether it is easier, cheaper, and more practical to create a central terminology service for various implementers as opposed to having vocabulary managed at each institution; (3) how best to prioritize suggested concept proposals to deal with the more important ones; and (4) how to optimally design a user-interface to accommodate differences in workflow and decrease the number of new concept proposals that result from misspellings, lexical variations, and synonyms to concepts already in the dictionary — this becomes particularly relevant in settings where providers are not interacting with the computer directly (e.g. settings where data-entry clerks, who usually have limited medical knowledge, input the data into the computer).

Approaches to these issues should be guided by lessons and demands on the ground. As a step in this direction, we describe the creation and evaluation of a concept dictionary and subsequent concept proposals for an open source EMR implemented in the resource-limited setting of Western Kenya. In this evaluation, we determine: (1) how well the initial concept dictionary met the needs in the local setting; (2) the characteristics of new concept proposal requests; and (3) the relationship between these characteristics and the number of times a new concept was requested.

Methods

Setting

We conducted this evaluation at the Academic Model for Prevention and Treatment of HIV-AIDS (AMPATH) program⁹ in Western Kenya. AMPATH is a collaborative initiative between Indiana University, U.S.A. and Moi University, Kenya which provides comprehensive care for patients infected with HIV. The program was initiated in 2001 and currently takes care of >40,000 HIV+ patients who have made >400,000 visits to 20 outpatient clinics.

AMPATH started using the AMPATH Medical Record System (AMRS) on Feb-15-2006 as its sole medical record. AMRS is an implementation of OpenMRS (http://openmrs.org), an open source EMR currently being widely deployed in Eastern and Southern Africa.¹⁰ All data in AMRS are stored as coded concepts to allow for easy retrieval and analysis. The concept dictionary is thus at the core of the EMR, and includes terms for diagnoses, tests, procedures, drugs, and other general terms for questions and potential answers.

We evaluated the initial concept dictionary terms which were diagnoses, findings and symptoms. We also analyzed new concept proposals for these categories during the first year of AMRS use, between Feb-15-2006 and Feb-14-2007. During this time, 237 providers recorded >9 million discrete clinical observations that were entered into AMRS by 28 data-entry clerks.

Creating the Initial Concept Dictionary

Two physicians, Drs. Paul G. Biondich (PGB) and Terry J. Hannan (TJH) created the concept dictionary for the initial installation using principles from Cimino’s desiderata¹, the ISO specifications,¹¹ and Chute’s framework¹². Both PGB and TJH have extensive experience in developing and implementing medical information systems. Working over a two year period, they compiled dictionary terms based on: (1) concepts used on AMPATH encounter forms, (2) the minimum dataset for HIV care in developing countries,¹³ (3) suggestions by AMPATH providers, and (4) the pre-existing concepts in the Mosoriot Medical Record system,¹⁴ an MS-Access®-based system previously used at one of the AMPATH clinics. This dictionary not only contained base concepts, but additionally described a rich synonym list, based upon reported local naming conventions.

Proposing New Concepts

When providers take care of a patient at an AMPATH clinic, they do not enter data into AMRS directly, but rather complete paper-encounter forms which contain questions and answers that have already been encoded into the AMRS concept dictionary. In some cases they are allowed to write down other answers as free text if these are not included as coded choices on the encounter form. Information on the encounter form is entered into AMRS by data-entry clerks who have little medical knowledge. When these clerks enter a term from the paper form, the system dynamically generates a list of all concepts that contain exact string matches to the letters already typed (Fig. 1).

Fig. 1.

Example for searching the concept dictionary terms in AMRS.

The only way a clerk can enter a datum for which there is no dictionary term is to select ‘Possible new concept?’ and type it in (Fig. 2). A concept is thus proposed as many times as it appears on the encounter forms. The proposals are stored in the encounter data as placeholders and copied into a queue. Once the proposals have been moderated, the placeholders are replaced with appropriate coded values.

Fig. 2.

Example for submitting a new concept proposal in AMRS.

Analyses of Initial Concept Dictionary

We determined the number of concepts (diagnoses, finding or symptoms) which were used at least once over the evaluation period. We also determined the percentage of all answers provided on encounter forms which used these concepts.

Analysis of New Concept Proposals

To allow for tabulation and analysis of new concept proposals, we chose not to add any of them into the initial concept dictionary during the 12-month period of this evaluation. This did not affect patient care as all encounter forms were stored in the patient’s paper chart and these were available to the providers.

The number of times each new concept was requested was determined. We then randomly sampled 20% of the new concept proposals, which were then characterized by one of the researchers, Martin C. Were, into categories modified from Wang et al.¹⁵ and Rob-inson et al.¹⁶ The categories included:

• Dictionary Deficiency: The new concept proposal did not exist in the concept dictionary.

• Synonym: The new concept proposal was either a true synonym of a term in the dictionary (e.g. ‘Boil’ as a synonym of ‘Furunculosis’), an impure synonym (e.g. ‘Enteritis’ classified as a synonym of ‘Gastroenteritis’), or a lexical variant (e.g. ‘asthmatic’ classified as synonym of ‘asthma’).

• Misspelling.

• Ambiguous term: The term entered did not make much sense (e.g. ‘Foot root’), was an uncommon term specifically used at that site (e.g. ‘FGC’), or had multiple possible meanings (e.g. ‘Adherence’).

• User Interface Failure: The new concept existed in the dictionary but the clerk still suggested it as a new concept proposal.

• Complex term: The new concept proposal included more than one potential concept (e.g. ‘Ascites/Bronchitis’), or had a modifier (e.g. ‘Severe Anemia’ and ‘Right sided lung collapse’).

A new concept could contain more than one of the characteristics above – for example, the concept proposal ‘HEMORHOIDS’ had a misspelling, and also represented a dictionary deficiency as the initial dictionary lacked the term ‘HEMORRHOIDS’.

We also looked for trends in the characteristics of new concept requests as a function of how many times they were proposed. This was done for all concepts proposed more than once.

Results

Analysis of Initial Concept Dictionary Use

The initial dictionary contained 424 concepts for clinical findings, symptoms, and diagnoses. Out of these, 380 (87%) were used at least once as answers on encounter forms. These concepts accounted for 79,336 of the 90,510 (88%) answers given on encounter forms over the study period. The other 11,174 answers were not in the concept dictionary, requiring the data-entry clerk to make a new concept request.

Analysis of New Concept Proposals

Over the study period, a total of 5,137 unique new concepts were requested. Out of these, 3,954 (77%) were requested one time, 484 (9%) twice, 209 (4%) three times, and the other 490 (10%) requested more than three times. 12 (0.2%) of the concepts were proposed over 100 times, with ‘ENTERITIS’ having the most requests at 461.

Table 1 shows the characteristics of a randomly selected sample of 1027 (20%) new concept proposals. Of the 199 terms judged as synonyms, 78 (39%) represented true synonyms, 89 (45%) impure synonyms, and 32 (16%) lexical variants. Out of the 420 complex terms, 93 (22%) were proposals which contained more than one concept, 149 (36%) had a location modifier (e.g. ‘Facial rash’ or ‘Right-sided pneumonia’), 84 (20%) had a quality modifier (e.g. ‘mild anemia’), and 94 had other types of modifiers (e.g. ‘r/o pneumonia’). Only 33 terms (18% of ambiguous terms and 3% of all terms) were classified as being site-specific. Fig. 3 shows the frequency of various characteristics as a function of how many times concepts were requested.

Table 1.

Analysis of New Concept Proposals by various characteristics^*

Characteristics	N	%
Dictionary Deficiency	418	41
Synonym	199	19
Misspelling	252	25
Complex Term	420	41
Ambiguous Term	179	17
User Interface Failure	10	1

^*A particular new concept proposal could have one or more of these characteristics.

Fig 3.

Relationship between the characteristics of concept proposals and the number of times a concept was requested.

Discussion

Creating and maintaining a well-structured concept dictionary is challenging. It is made even more difficult in settings of limited human and financial resources. In this paper, we report our experience and the lessons we have learned during the first year of using the AMRS system.

We show the benefit of investing time to create the initial dictionary. 380 concepts in the initial dictionary provided 88% of all the answers used over the first year. This was achieved after two years of hard work by experts in this area. Such a time-intensive approach might not be feasible in many settings and alternatives include: (1) implementers concentrating on concepts they think will be absolutely necessary for their practice (like those used on encounter forms), and adding any new concepts as the need arises; (2) Using standard vocabularies such as ICD-9 - the utility of these in resource-poor settings still needs formal evaluation; and (3) a consideration by implementers and developers who have already created dictionaries to share them with others (we provide a version of the OpenMRS dictionary at http://demo.openmrs.org/openmrs/dictionary/). Ideally, concept dictionaries put in the public domain should have links to standard vocabularies like ICD-9 and LOINC to make their adoption easier for those not familiar with a particular EMR implementation.

The AMRS concept dictionary was built on the premise that it was impossible to come up with all the concepts needed at the outset, but that the local community could help augment the dictionary. The 418 new proposals which revealed a dictionary deficiency support the feasibility of this approach. We also observed that the more frequently a concept was proposed, the more likely it was to represent a deficiency in the dictionary (Fig 3). This suggests that under resource constraints, time and resources could be better spent working on concepts that have been proposed more frequently. It seems to us that it is a good design principle to keep track of how many times a particular concept is requested as this can also help in deciding the level of granularity needed for a concept.

A significant number of the proposed concepts were complex terms. From these were learned that giving the providers some basic understanding of how data is encoded into the system could help reduce the numbers of complex terms suggested. Furthermore, our observation that there are a significant number of modifiers used point to the importance of addressing issues around pre- and post-coordination of concepts early during the development and implementation of a system. Advanced EMR developers might find it valuable to incorporate automated term dissection¹⁷ and composition¹⁸ algorithms into their software to help deal with complex terms.

The requested concepts also inform us on how user-interfaces should be designed when the person entering the data into the computer has little medical experience. Misspellings, which are usually off by one or two letters, and simple lexical variants can be matched correctly if approximate string comparator algorithms, like the Levenshtein Edit distance algorithm¹⁹, were incorporated into the system. Furthermore, it would be important to understand the social context within which the clerks operate, the role of memory, and the decision-making strategies they employ when proposing new concepts.²⁰

Finally, we observe that site-specific terms only made up a small percentage of all terms requested (3%). This finding is encouraging, since, in resource-limited settings, it might be preferable for several implementations to use a centralized vocabulary service. Such a service would: (1) facilitate data-sharing, which is critical for the success of such collaborative research projects like those being conducted by the International Epidemiological Databases to Evaluate AIDS (IeDEA),²¹ (2) reduce the number of terms that have slightly different meanings, thus making data easier to extract from the system, and (3) reduce the burden of creating the initial concept dictionary.

It is our hope that the lessons from our experience will be valuable for developers and implementers of EMRs in general, and provide specific benefit to those in resource-limited settings.