Evaluation of RxNorm for Representing Ambulatory Prescriptions

Abstract

Objective:

We evaluated the RxNorm standardized drug nomenclature for representing ambulatory e-prescriptions.

Methods:

Using a sample of 19743 primary care e-prescriptions, we estimated the coverage rate of RxNorm for representing clinical drugs, measured the 6-month replacement rate for RxNorm concepts, assessed the consistency of two independent concept mappings, and investigated inconsistent mappings.

Results:

RxNorm contained concepts for nearly all prescriptions in the sample (99.995%). Of 1419 concepts used, 8.1% were replaced between April and October 2009. Independent mappings produced different concepts for 676 e-prescriptions (3.4%), but most differences would have low clinical significance. Most mismatches were related to the use of extended-release form concepts with no duration specified, inhalers vs. their contents, and clinically inert salts.

Conclusions:

RxNorm provides concepts covering nearly all ambulatory e-prescriptions in this setting. Independent mappings were relatively consistent. Improvements could be made by enabling selection of the most-specific concepts when broader prescribable concepts exist.

Introduction

Ambulatory electronic prescribing (e-prescribing) is expected to deliver improvements in the quality and safety of drug prescribing,^1–4 but implementing robust e-prescribing systems capable of communicating unambiguous information requires the use of reliable standards for representing each prescribable drug.^5–7 Currently, the drugs in e-prescriptions are identified using the National Drug Code (NDC) Directory, which enumerates prescription drug products at the level of distinct packages. These granular distinctions have little clinical meaning, however, as clinicians rarely intend to prescribe at this level of detail. In addition, the NDC Directory is unreliable—one study found that 27% of the 123,856 codes in the Directory were erroneous, and that 14,337 additional prescription drug products were missing codes.⁸ The RxNorm standard, developed by the National Library of Medicine (NLM), is the first publicly available standardized nomenclature of prescribable clinical drugs.^9,10 Compared to existing commercial prescription drug nomenclatures, RxNorm is rigorously structured to represent prescribable concepts as unique {drug, dose, dose form} triples, independent of non-clinical elements such as inert ingredients or packaging (i.e. “cefdinir 300 MG Oral Capsule”). We sought to evaluate RxNorm’s completeness in representing a real-world sample of e-prescriptions and to highlight potential areas needing further refinement.

Methods

Overview

Using a sample of e-prescriptions, we assessed the degree to which those prescriptions could be accurately mapped to RxNorm generic concept unique identifiers (CUIs). As RxNorm concepts are being updated constantly, we assessed the rate of concept replacement over a six-month period. Finally, we assessed the consistency of two separate NDC-to-CUI mappings used for matching prescriptions to RxNorm concepts. E-prescriptions that match to different but synonymous CUIs suggest the existence of an error in RxNorm because a goal of RxNorm is to map all clinically-equivalent synonyms to a single CUI. E-prescriptions that match to more than one non-synonymous CUI indicate areas of ambiguity that need to be resolved.

Data Sources

We obtained samples of de-identified e-prescriptions that had been transmitted between March 6, 2007 and February 13, 2009 from ambulatory physician practices to retail pharmacies using either of 2 point-of-care software packages. A total of 82 providers, practicing at 48 sites in Kansas, Michigan, and Maryland contributed to the sample. Provider specialties included internal medicine, family practice, pediatrics, and general surgery. Data fields included NDC, drug name, strength, form, and patient instructions. Prescribers and patients were identified in the data only by code numbers; no patient clinical or demographic data, including diagnosis or age, was provided.

E-prescriptions either outside the scope of RxNorm or inadequately specified were excluded. E-prescriptions considered out of scope included non-prescriptions, non-drug items such as supplies and equipment, and multivitamins, which remain officially out of scope of RxNorm, but are partially represented. An inadequately specified e-prescription was one for which the specific drug being prescribed could not be determined, based either on the drug description or the representative NDC code. Examples include those missing the drug component (“Artificial tears. Gel”), the form (“Indocin 50MG”), or the strength (“K-Dur”), as well as those with errors in spelling, pack size, or units of measurement. Because it would not be possible to resolve these into single concepts, we excluded them as erroneous prescriptions.

Overall Coverage

We define “generic CUI” as an RxNorm Semantic Clinical Drug (SCD) or Generic Pack (GPCK) concept.¹⁰ We first attempted to automatically match e-prescriptions to generic CUIs by using an NDC-to-CUI mapping that we derived from the RxNorm distribution. A second round of automated matching was conducted using a proprietary NDC-to-CUI mapping obtained from a medical knowledge database vendor. The remaining unmatched prescriptions, particularly those with missing NDCs, were matched via manual searches of the RxNorm CUI database. In cases of uncertainty about the equivalence of drug concepts being prescribed vs. those represented in RxNorm, we sought specific product information about the drugs in question from a variety of sources, including the drug manufacturer, wholesaler and retailer websites, and drug reference compendia.

RxNorm Concept Replacement Rate

The RxNorm distribution is constantly being updated with new CUIs, some of which replace existing concepts. Our primary analyses and NDC-to-CUI mapping use the version of RxNorm from October 2009, which contained 18775 distinct generic CUIs. To measure RxNorm’s concept replacement rate, we obtained a distribution of RxNorm from April 2009, which contained 18398 generic CUIs. We developed an NDC-to-CUI mapping from this older distribution, and then used it to automatically match the same sample. We then took the CUIs that had been used in this mapping and checked whether they were present in the October 2009 CUI database. For CUIs that were present only in the older distribution, we determined whether each could be forward-mapped to a current CUI using RxNorm’s archival table for retired concepts (the RxNAtomArchive table) in the October 2009 RxNorm distribution.

Consistency of RxNorm and Vendor Mappings

We directly compared the RxNorm-derived NDC-to-CUI mapping and the vendor’s NDC-to-CUI mapping in order to assess their consistency and also to highlight possible cases of synonymy or ambiguity in RxNorm. Mismatches occurred when the same e-prescription was mapped to different CUIs by the two mapping sources; we classified these cases into distinct types, and we ordered the types according to the clinical significance of the dispensing error that would result if a prescription for one were filled with the other. We looked for both synonymy errors, where two distinct CUIs identify the same clinical drug, as well as ambiguous mappings, where two CUIs are clinically distinct, but for a given clinical drug, it is impossible to determine which of the two CUIs is the “correct” one.

Results

Data Disposition

The final analytic sample included 19743 e-prescriptions. 336 out-of-scope and 56 ambiguously specified e-prescriptions were excluded from the original sample of 20135.

Overall Coverage

The overall coverage rate of RxNorm is shown in Table 1. Only 1 e-prescription in this sample could not be linked to a generic CUI, resulting in an overall coverage rate of 99.995%. Overall, 98.8% of the sample matched using automatic mappings—94.4% with the RxNorm-derived mapping, and an additional 4.4% when the vendor’s mapping was applied to the remaining unmatched e-prescriptions. Of the remaining 1.2% that did not match automatically, we were able to manually find an appropriate RxNorm CUI for 236 of 237 e-prescriptions. A representative NDC was present in 98 of these, and missing in 139. A CUI match was found manually for 97 of the 98 with an NDC and for all 139 without an NDC.

Table 1.

Coverage of n=19743 e-prescriptions

Description	n	%
CUI found automatically	19506	98.8%
CUI found manually	236	1.2%
NDC missing	139	0.7%
NDC present	97	0.5%
CUI not found	1	0.0%*
Total e-prescriptions	19743	100.0%

^*exact value = 0.005%

The one e-prescription that could not be matched to any CUI was a combination pack of nicotine patches of three different strengths (NDC 00067503956). Each separate patch was represented by an existing RxNorm CUI, but this particular product—with all three combined in one box—did not have the expected pack CUI to represent it. We confirmed with the manufacturer that the missing combination package is still available on the market.

RxNorm Concept Replacement Rate

Using the April 2009 NDC-to-CUI mapping, we successfully matched e-prescriptions with non-missing NDCs to CUIs in 94.5% (18526/19604) of cases. Of the 1419 CUIs comprising those matches, 115 (8.1%)—representing 1065 (5.7%) e-prescriptions—had been replaced in RxNorm six months later. Forward-mapping to an updated CUI was achieved for all 115 (100%) of these CUIs.

Consistency of RxNorm and Vendor Mappings

Table 2 describes the two NDC-to-CUI mapping tables we used, in terms of the total number of NDC to CUI linkages available. The RxNorm-derived mapping tables utilized 65.7% of the 18775 distinct generic CUIs (SCDs or GPCKs) that existed in the October 2009 release; the vendor mapping table utilized 50.1% of these CUIs. This implies that about one-third of the theoretically-prescribable generic CUIs were not linked to any associated NDC in the mappings.

Table 2.

NDC-to-CUI Mappings

	RxNorm	Vendor
# NDC (n)	330,521	244,247
# SCD/GPCK (n)	12,329	9,411

Table 3 shows the cross-tabulated match count when these mapping tables were used to look up CUIs for our prescription sample. Of 19,604 prescriptions with an NDC code, 93.9% matched via both mappings, 1.2% matched only in the RxNorm-derived mapping, 4.4% matched only in the vendor mapping, and 0.5% did not match in either mapping.

Table 3.

RxNorm/Vendor Match Rates

		Vendor n (%)
		Match	No Match	Total
RxNorm n (%)	Match	18,408 (93.9)	234 (1.2)	18,642 (95.1)
	No Match	864 (4.4)	98 (0.5)	962 (4.9)
	Total	19,272 (98.3)	332 (1.7)	19,604 (100.0)

For 676 e-prescriptions (3.4%), the two mappings produced non-identical CUIs. The mismatches involved 106 distinct NDCs and represented 67 distinct clinical drugs. Because there should be only one unique CUI to represent each generic prescribable drug concept, each mismatch implies the existence of an error, either in one of the NDC-to-CUI mappings or in RxNorm itself, due to the existence of two concepts having the same meaning.

Table 4 categorizes the mismatches. Only one mismatch (topical vs. augmented topical betamethasone cream) was categorized as having more than minor clinical significance. Other mismatches included minor form differences, the use of general vs. time specific extended release forms, and differences in inhaler canister sizes and salts. Some mismatches appeared to be due to errors in RxNorm. For example, the sodium fluoride example in Table 4 shows two concepts that most likely represent the same solution, but one includes the weight of both elements, while the other only includes the weight of the fluoride active ingredient. RxNorm policy is to express drug strengths based on active ingredient(s) only, thus the concept with a strength of 1.1 MG/ML is likely an error in RxNorm.

Table 4.

Classification of Mismatches Between Two Mappings to Generic RxNorm Concepts

Mismatch Type	eRx n (%)	NDC n (%)	Unique Meds n (%)	RxNorm CUI Match (Example)	Vendor CUI Match (Example)
Form difference of moderate significance	3 (0.4)	2 (1.9)	1 (1.5)	Betamethasone 0.5 MG/ML Augmented Topical Cream	Betamethasone 0.5 MG/ML Topical Cream
Form difference of minor significance	9 (1.3)	6 (5.7)	5 (7.5)	chlorhexidine gluconate 40 MG/ML Medicated Liquid Soap	chlorhexidine gluconate 40 MG/ML Topical Solution
Use of potentially ambiguous general XR dose form	6 (0.9)	3 (2.8)	2 (3.0)	Verapamil 240 MG Extended Release Tablet	24 HR Verapamil 240 MG Extended Release Tablet
Use of non-ambiguous general XR dose form	113 (16.7)	17 (16.0)	12 (17.9)	24 HR Potassium Chloride 20 MEQ Extended Release Tablet	Potassium Chloride 20 MEQ Extended Release Tablet
Salt specified vs. unspecified	249 (36.8)	44 (41.5)	16 (23.9)	Metoprolol 100 MG Oral Tablet	Metoprolol Tartrate 100 MG Oral Tablet
MDI differing in canister size	54 (8.0)	5 (4.7)	4 (6.0)	60 ACTUAT Albuterol 0.09 MG/ACTUAT Metered Dose Inhaler	200 ACTUAT Albuterol 0.09 MG/ACTUAT Metered Dose Inhaler
MDI vs. its contents	213 (31.5)	22 (20.8)	21 (31.3)	120 ACTUAT fluticasone 0.22 MG/ACTUAT Metered Dose Inhaler	fluticasone 0.22 MG/ACTUAT Inhalant Solution
Prefilled syringe vs. its contents	7 (1.0)	2 (1.9)	1 (1.5)	0.3 ML Epinephrine 1 MG/ML Prefilled Syringe	Epinephrine 1 MG/ML Injectable Solution
Strength rounding difference	19 (2.8)	3 (2.8)	3 (4.5)	Azithromycin 16.7 MG/ML Oral Suspension	Azithromycin 20 MG/ML Oral Suspension†
Units difference	2 (0.3)	1 (0.9)	1 (1.5)	Nicotine 10 MG/ML Inhalant Solution	Nicotine 4 MG/ACTUAT Inhalant Solution†
Inactive ingredient strength issue	1 (0.1)	1 (0.9)	1 (1.5)	Sodium Fluoride 1.1 MG/ML Oral Solution†	Sodium Fluoride 0.5 MG/ML Oral Solution
Total	676	106	67

^†Probable erroneous RxNorm concept

Discussion

This evaluation of the RxNorm concept database using a real-world sample of e-prescriptions demonstrates that RxNorm could be useful for representing e-prescriptions, provided that caution is exercised around certain areas with incompletely resolved issues of ambiguity.

Completeness of RxNorm

Of 19,604 e-prescriptions transmitted with representative NDCs, 98 (0.5%) could not be automatically translated to a prescribable concept in RxNorm using NDC-to-CUI mappings. When using the vendor’s proprietary nomenclature instead of RxNorm, 107 (0.5%) failed to automatically translate from the NDC to the vendor’s proprietary, non-RxNorm prescribable concept. These “missing concepts” prevent pharmacies from relying on the representative NDC code to auto-populate their systems for dispensing, and creates extra work for the pharmacist. However, if RxNorm CUIs had been used as the primary identifiers for this prescription sample, then far fewer would fail to be auto-populated (on the order of 0.005%). A caveat is that frequent updating would be necessary. Although the completeness of RxNorm’s forward-mapping tables was perfect in this sample, 8.1% of CUIs used in April 2009 had been replaced with new CUIs six months later. Drug manufacturers could contribute to maintaining the completeness of RxNorm by ensuring that the FDA has complete and accurate information for each drug they make available on the market. This information could then feed directly into the RxNorm maintenance process. Manufacturers should also contribute to maintaining accurate NDC-to-RxNorm mappings.

Areas of Caution

Fixing persistent errors in RxNorm may require better mechanisms for investigating drug details. For example, rounding differences in concept mappings for the same product likely resulted in the azithromycin mismatch in Table 4 (20mg/ml vs. 16.7mg/ml). Eliminating nonspecific terms from RxNorm is another area for improvement. For example, the “Verapamil 240 MG Extended Release Tablet” concept could indicate either the 12-hour or 24-hour form of the medication, both of which are separately represented by other RxNorm concepts. One strategy to address this challenge could be to remove the ER forms having no specified duration of action from RxNorm. An alternative could be for e-prescribing systems to use the most specific concept for a given prescription, and to indicate to the prescriber when a more specific concept is available. To aid in this process, manufacturers should consistently specify the expected duration of action for ER forms in their product information.

Limitations

Several important limitations need to be considered in the interpretation of these results. First, our strategy for discovering ambiguities in RxNorm was based on only two NDC-to-CUI mappings; it is conceivable that additional mismatches and mismatch types would be discovered if additional vendor mappings were compared. There is no “gold standard” NDC-to-CUI mapping, so it was not possible to compute accuracy estimates or the sensitivity and specificity of each mapping. Second, the study sample consisted mainly of primary care physicians (PCPs). Although PCPs write a large majority of outpatient prescriptions in the U.S., it remains possible that the needs of some specialties may be less well-represented in RxNorm. Finally, this evaluation focused only on generic concepts. Cases in which physicians prescribe a branded drug with a “dispense as written” (DAW) order are relatively rare, so the generic concept is typically the central expression. However, further evaluation of brand concept accuracy in RxNorm may be warranted, using a larger sample that could be enriched for DAW events.

Conclusions

RxNorm was demonstrated to have a 99.995% coverage rate for 19743 e-prescribed clinical drugs in a real-world sample. This coverage rate indicates that the use of RxNorm as a drug identifier in prescriptions could yield gains in the ability of e-prescribing systems to accurately and unambiguously represent the intended clinical drugs, compared with the current use of “representative NDCs” for this purpose. Several challenges still need to be resolved in RxNorm, but most are of relatively minor clinical significance.