Comparison of three commercial knowledge bases for detection of drug-drug interactions in clinical decision support

Abstract

Objective: To compare 3 commercial knowledge bases (KBs) used for detection and avoidance of potential drug-drug interactions (DDIs) in clinical practice.

Methods: Drugs in the DDI tables from First DataBank (FDB), Micromedex, and Multum were mapped to RxNorm. The KBs were compared at the clinical drug, ingredient, and DDI rule levels. The KBs were evaluated against a reference list of highly significant DDIs from the Office of the National Coordinator for Health Information Technology (ONC). The KBs and the ONC list were applied to a prescription data set to simulate their use in clinical decision support.

Results: The KBs contained 1.6 million (FDB), 4.5 million (Micromedex), and 4.8 million (Multum) clinical drug pairs. Altogether, there were 8.6 million unique pairs, of which 79% were found only in 1 KB and 5% in all 3 KBs. However, there was generally more agreement than disagreement in the severity rankings, especially in the contraindicated category. The KBs covered 99.8–99.9% of the alerts of the ONC list and would have generated 25 (FDB), 145 (Micromedex), and 84 (Multum) alerts per 1000 prescriptions.

Conclusion: The commercial KBs differ considerably in size and quantity of alerts generated. There is less variability in severity ranking of DDIs than suggested by previous studies. All KBs provide very good coverage of the ONC list. More work is needed to standardize the editorial policies and evidence for inclusion of DDIs to reduce variation among knowledge sources and improve relevance. Some DDIs considered contraindicated in all 3 KBs might be possible candidates to add to the ONC list.

INTRODUCTION

Medications remain one of the most common modalities of treatment in modern health care, yet they are also an important iatrogenic cause of morbidity and even mortality. The incidence of adverse drug events has been estimated to be 6.5 per 100 admissions, 13% of which are fatal or life-threatening.¹ Inpatient adverse drug events alone have been estimated to cost a total of $2 billion per year.² Not all adverse events can be avoided, but drug-drug interactions (DDIs) may be among the most preventable and manageable because of their potential predictability. A clinically significant DDI is defined as an unintended modification in the effect of a drug when administered with another drug. It can be an increase or decrease in the action of either drug or an adverse effect not normally associated with the drugs,³ and is actionable (ie, some action should be taken or risk management plan considered). One study reported that DDIs accounted for 17% of adverse drug reactions leading to hospitalization.⁴ In another study, 4.4% of elderly patients received prescriptions with a risk of severe interactions.⁵

Aging populations, multiple comorbidities, polypharmacy, and frequent launching of new drugs are cited as factors that contribute to the frequency and seriousness of DDIs.^6–8 In the United States, 29% of adults are taking 5 drugs or more.⁹ Among those age 65 years or older, 17–19% take at least 10 drugs.⁹ A recent study confirmed that over 5 years (2006–2011), the risk of major DDIs nearly doubled in geriatric patients (8.4–15.1%).¹⁰ Polypharmacy directly affects the ability of health care professionals to recognize potential DDIs. As the numbers of coprescribed drugs increase, the potential pairwise interactions increase exponentially. Weideman et al. found that clinician DDI recognition rates decreased significantly as the number of drugs increased. Even among trained pharmacists, none detected all interactions when there were 8 or more drugs.¹¹ Furthermore, in a multiprovider situation, the prescribing clinician may not be familiar with, or even aware of, all of a patient’s medications. All these factors suggest that clinical information systems that can retrieve all medications a patient is using and can detect and remind providers about DDIs (by displaying warning messages or interruptive alerts) are becoming more necessary for patient safety. Indeed, computerized physician order entry with clinical decision support (CDS) capabilities has been mandated by national programs such as the Electronic Health Records (EHR) Meaningful Use incentive program, and strongly recommended by patient safety advocacy organizations such as the Leapfrog Group.^12,13

Background and significance

A comprehensive, accurate, and evidence-based knowledge base (KB) is a prerequisite for effective deployment of a CDS capable of preventing and managing DDIs. Since the resources and expertise needed to develop and implement a home-grown DDI KB are available to only a few large academic centers, most organizations choose to purchase KBs from commercial vendors.³ The editorial policies for DDI evidence inclusion and timeliness of KB updates, as well as unique EHR vendor implementation choices, directly affect a system’s alerting capabilities and CDS advice that is offered to clinicians. Previous studies have highlighted the variability between various sources of DDI knowledge, whether they are proprietary or in the public domain.^14–24 Vitry¹⁴ found that 14–44% of major DDIs listed in 1 compendium were not listed in other compendia. Hazlet et al.¹⁹ tested 9 DDI software programs and found that they failed to detect clinically relevant DDIs one-third of the time, with sensitivity and specificity ranging from 0.44–0.88 and 0.71–1.0, respectively. The variability was explained by both KB content and software implementation differences. In view of this variation, Smithburger et al.²⁰ suggested that more than 1 knowledge source should be used, and that information in proprietary KBs should be reviewed by clinical experts.²⁵ While these suggestions seem logical, financial constraints and availability of expertise could limit their feasibility. Currently, most health care organizations still rely on a single proprietary KB.²⁶ Inconsistent evaluation and classification of interactions have been cited as factors contributing to excessive DDI alerts.²⁷ As described in a recent publication, methodologies for unifying editorial policy decisions and criteria for evidence inclusion of DDI have also been pursued as a solution.²⁸ DDI alert customization capabilities are necessary because of the uniqueness of patient populations served by systems and local treatment guidelines, and are reported to help improve provider acceptance rates of interruptive alerts, but will also introduce more variability across institutions.²⁹

In this study, we performed a comprehensive comparison of 3 commercial DDI KBs widely used by hospitals, clinics, and pharmacies. First, we did a direct comparison of the KBs’ lists of interacting drug pairs and their severity rankings to assess overlap. Second, we used a reference list of highly significant DDIs to assess whether each KB alone would provide sufficient coverage of these high-priority cases.³⁰ Third, we applied the KBs to a prescription dataset to see whether the differences observed among them would translate into different rates and patterns of DDI alert generation. To our knowledge, such a comprehensive comparison of commercial DDI KBs has not been performed.

METHODS

Acquiring DDI information from vendors and mapping to RxNorm

We contacted 5 commercial drug KB vendors that provide prescription decision support to clinicians. First Databank (FDB), Micromedex, and Multum agreed to participate in our study, while MediSpan and Gold Standard declined. Our study considered only DDIs ranked as contraindicated, major/severe, and moderate by the KBs. Minor DDIs and interactions with herbal remedies were excluded, because they were less important and tended to be less consistently represented. We mapped the drugs in the KBs to RxNorm,³¹ the US interoperability standard drug terminology. We used 2 RxNorm clinical drug term types, semantic clinical drug (SCD, eg, azithromycin 500 mg oral tablet) and generic drug pack (GPCK, eg, 6-pack of azithromycin 500 mg oral tablet), which specified the ingredients, dose form, route, and strength. FDB and Multum provided their own RxNorm mapping tables. For Micromedex, we mapped first to RxNorm ingredients by lexical matching, supplemented by manual review, and then navigated to all corresponding SCDs and GPCKs, restricted to the dose form and route specified in Micromedex. We used the latest version of RxNorm when we acquired the KBs (May 2014).

Comparing interactions across KBs

A DDI could be represented in 3 ways: first, as a pair of clinical drugs, specifying the active ingredients, strength, and dose form (eg, trimipramine 100 mg capsule and albuterol 2 mg tablet); second, as a pair of ingredients (eg, trimipramine and albuterol); and third, as a pair of drug classes (eg, tricyclic antidepressants and sympathomimetics). In a KB, the DDIs were typically grouped into rules at the drug class level to facilitate content management and display of alert messages. Each DDI rule was associated with a textual description, known as a monograph (eg, “Concurrent use of tricyclic antidepressants and sympathomimetics may result in hypertension, cardiac arrhythmias, and tachycardia”). In this study, we analyzed DDIs at all 3 levels. Unless otherwise stated, drug pairs refer to drugs at the clinical drug level.

A master table was created for each KB, with all DDIs represented as pairs of RxNorm concept unique identifiers (RxCUIs) at the clinical drug level, together with their severity rankings. Each pair of drugs was listed only once, ie, (A, B) and (B, A) were considered equivalent. If a pair of drugs was assigned more than 1 severity ranking in a KB (eg, multi-ingredient formulation with several interactants), we kept only the highest-ranking entry. We assessed the overlap across KBs by matching the RxCUI pairs.

Clinical drugs were rolled up to their ingredients using the relationships in RxNorm. Multi-ingredient drugs were excluded in this analysis because it was not possible to pinpoint the interacting ingredient, and the ingredient-level interaction would be captured by the single ingredient formulations anyway. We rolled up clinical drug pairs to their corresponding rules (monographs in a KB), excluding those that were grouped under multiple rules (eg, multi-ingredient drugs). We considered DDI rules from 2 KBs to be overlapping if they shared at least 1 clinical drug pair.

Comparing interactions from KBs against a reference source

In order to address the challenges of alert burden and its impact on EHR adoption, the Office of the National Coordinator for Health Information Technology (ONC) commissioned a consensus-based effort to identify a subset of highly significant DDIs for which interruptive warnings should be generated in all EHRs.³⁰ The ONC high-priority list (ONC list) was developed from candidate interactions identified by Partners Healthcare that were then vetted by a stakeholder panel including medication KB vendors, EHR vendors, clinical experts, and representatives from federal and private agencies involved in the regulation of medication use. The panel attained consensus on 15 DDI rules at 3 levels of specification: drug class–drug class, ingredient–drug class, and ingredient-ingredient. The ONC list enumerates class members for all drug classes except QT-prolonging drugs and tricyclic antidepressants. We expanded all the ONC DDI rules to the ingredient level. For QT-prolonging drugs, we used the web resource CredibleMeds as recommended, limited to drugs associated with the highest risk of torsades de pointes (known as List 1).³² For tricyclic antidepressants, class members were determined by consulting pharmacology textbooks. To align with the KB master tables, we mapped the ONC list first to RxNorm ingredients, then propagated to SCD/GPCKs, restricting to systemic dose forms (eg, oral tablets, injections).

We analyzed the coverage of the ONC list by the KBs at the ingredient level. We considered that an ONC ingredient-level DDI was covered if at least 1 of the clinical drug pairs for that DDI was found in a KB. Since the ONC list was supposed to be highly significant and recommended to be used in all EHRs, any ONC DDI missing from a KB was reviewed by a KB expert to ascertain the reason for the absence. Conversely, we selectively reviewed DDIs that were ranked as contraindicated in all 3 KBs but not in the ONC list, as they could potentially reflect important DDIs missing from the ONC list.

Generating potential DDI alerts from actual prescription data

We used a dataset from Symphony Health Solutions with 1 year of prescription-filling data (from July 1, 2011, to June 30, 2012) for patients from Washington, DC, Maryland, Virginia, and West Virginia. The drugs in the dataset were mapped to RxNorm SCD and GPCK through the NDC codes included in the source data, supplemented by string matching and manual review. Drugs that were not administered systemically (eg, topical ointments) were excluded because they seldom caused significant interactions. The period that a patient was exposed to a drug was estimated based on the fill date and days of supply. Two drugs that might not be prescribed at the same time were considered to be co-administered if their period of exposure overlapped. The co-administered drug pairs were checked against the 3 KB and ONC tables to see if DDI alerts would have been generated. We reported the alert rates as a proportion of total prescriptions. We estimated the number of prescriptions by assuming that the drugs with the same fill date and physician ID belonged to the same prescription.

RESULTS

Comparing interactions across KBs

The number of unique clinical drugs (at the SCD/GPCK level) involved in any DDI ranged from 7427 to 13 133, among which 5754 drugs were common to all 3 KBs (Table 1). The size of the KBs varied considerably in terms of drug pairs. FDB had the least drug pairs (1.6 million), followed by Micromedex (4.5 million) and Multum (4.8 million). In all KBs, contraindicated was the smallest category and moderate the largest. Overall, the number of drug pairs that were commonly configured to generate interruptive alerts (contraindicated and major/severe categories together) was 490 260 (30.8%), 2 311 324 (51.9%), and 468 822 (9.8%) for FDB, Micromedex, and Multum, respectively. Altogether, the 3 KBs contained 8.6 million unique drug pairs, of which 6.8 million (79.4%) were unique to 1 KB, 1.3 million (15.5%) were found in 2 KBs, and 0.4 million (5%) were in all 3 KBs (Figure 1). The percentage of unique drug pairs (ie, not found in any other KB) was 35.6, 65, and 70.9% for FDB, Micromedex, and Multum, respectively.

Table 1.

Composition of the 3 knowledge bases

Component	FDB	Micromedex	Multum
Unique drugs	10 279	13 133	7427
Drug pairs × 1000 (% of total)
Contraindicated	101 (6.3)	192 (4.3)	100 (2.1)
Major/severe	390 (24.5)	2120 (47.6)	368 (7.7)
Moderate	1102 (69.2)	2139 (48.1)	4302 (90.2)
Total	1592 (100)	4450 (100)	4771 (100)

Figure 1.

Overlap of clinical drug pairs (numbers in 1000) between the knowledge bases.

In Table 2, the pairwise comparisons are shown as 6 grids of 3 × 4 boxes. The number in each box is the number of common drug pairs, and the percentage is based on the total in KB1. For example, comparing FDB to Micromedex (top middle grid), there are 48 673 shared contraindicated (sev1) DDI drug pairs, corresponding to 48% of all contraindicated DDIs (N = 100 697) in FDB. For better visualization, the boxes in which the severity rankings are the same for both KBs are shaded. The highest percentage of shared drug pairs within a severity category (ignoring those not found) is highlighted in bold type. So if the highlighted percentage falls within a shaded box, the severity rankings of the 2 KBs agree more often than disagree for that severity category. For example, between FDB and Micromedex (top middle and middle left grids), the rankings generally agree in most severity categories (5 out of 6 highlighted numbers are in shaded boxes), except that more major DDIs in Micromedex are classified in FDB as moderate (9%) than major (6%) or contraindicated (1%). Overall, 13 out of 18 highlighted percentages are in a shaded box, which is true for all contraindicated categories. Among the shared drug pairs between 2 KBs, 58% (FDB and Micromedex), 68% (FDB and Multum), and 57% (Micromedex and Multum) have the same severity rankings.

Table 2.

Pairwise comparison showing number of overlapping clinical drug pairs (numbers in thousands) between KBs

Each pairwise comparison is outlined by thick borders. The percentages are based on row totals (KB1). The highest percentage (excluding “not found”) in each severity category is highlighted in bold type. Shaded boxes are those in which the severity rankings in 2 KBs agree (sev1 = contraindicated, sev2 = major/severe, sev3 = moderate).

Detailed pairwise comparisons at the ingredient and DDI rule levels can be found in Appendix A (Online Supplementary Tables 1 and 2). Generally, the pattern of overlap at the ingredient level is similar to the clinical drug level. At the rule level, the degree of overlap and the agreement in severity ranking are considerably higher, with all but 1 of the 18 highlighted percentages in shaded boxes. Overall, 48.5% of DDI rules are shared by all 3 KBs, much higher than the ingredient (8.7%) and clinical drug (5%) levels (Supplementary Table 3).

Comparing interactions from KBs against a reference source

The 15 DDI rules in the ONC list expanded to 1027 pairs of ingredients. Overall, FDB, Micromedex, and Multum did not cover 10.3, 11.9, and 10.2% of the ONC ingredient pairs, respectively (Table 3). Versioning (ie, when the missing DDI was available in a newer version of the KB) accounted for many of the omissions. This shows that the KBs are updated quite frequently as new drug interaction information becomes available. Some drugs in the ONC list (eg, astemizole, terfenadine) were no longer available on the US market and were not in the KBs. The ONC list included every pair of QT-prolonging agents considered high risk for causing torsades de pointes. By editorial policy, some KBs only included QT-prolonger pairs corroborated by specific evidence (eg, specific warning on the drug label, clinical reports). Some KBs disagreed with the ONC list of CYP-450 inhibitors. For example, cimetidine and diltiazem were considered as strong CYP3A4 inhibitors in the ONC list, but only moderate inhibitors by 2 KBs. Some drugs were missed by RxNorm mapping. Some antiretroviral agents were given only as combinations (eg, tipranavir and ritonavir). For such cases, some KBs flagged DDI for only 1 component to avoid duplicate alerts. Some drug pairs were flagged as a different category of alert (eg, erythromycin and azithromycin were considered as duplicate therapy) and not as DDIs. Overall, if we adjust for the unintentional differences (versioning, not on market, mapping, combination therapy, and different category), the coverage of the ONC list becomes 98.7, 98.83, and 99.9% for FDB, Micromedex, and Multum, respectively.

Table 3.

Coverage of the ONC list ingredient pairs by KBs

Coverage	FDB (%)	Micromedex (%)	Multum (%)
Found in KB
Contraindicated	591 (57.5)	496 (48.3)	491 (47.8)
Major/severe	189 (18.4)	398 (38.8)	324 (31.5)
Moderate	141 (13.7)	11 (1.1)	107 (10.4)
Not found	106 (10.3)	122 (11.9)	105 (10.2)
Not found in KB because:
Versioning	38	92	20
Not on market	33	18	70
Editorial exclusion	13	12	1
Mapping problem	15	0	10
Combination therapy	5	0	4
Other	2	0	0
Total ONC ingredient pairs	1027 (100)	1027 (100)	1027 (100)

Generating potential DDI alerts from actual prescription data

Our prescription dataset covered 1.9 million patients with 14 million prescriptions and 19 million drug items. Considering all severity levels, the alerts generated by FDB, Micromedex, and Multum would be 163, 329, and 751 per 1000 prescriptions, respectively. Counting only contraindicated and major/severe DDIs (usually triggering interruptive alerts), 25, 145, and 84 alerts per 1000 prescriptions would be generated by FDB, Micromedex, and Multum, respectively (Supplementary Table 4).

Applying the ONC list to the prescription dataset would generate 43 047 alerts (3 alerts per 1000 prescriptions). The overwhelming majority (97.6%) were generated by 2 DDI rules: statins with protease inhibitors and 2 QT-prolonging drugs (Table 4). The remaining 13 ONC rules together accounted for only 2.4% of alerts. Overall, FDB, Micromedex, and Multum covered 97.9, 85.9, and 99.8% of the ONC list alerts, respectively. Adjusting for the unintentional differences (versioning, mapping issues, etc.), the overall coverage of ONC alerts became 99.8, 99.9, and 99.9% for FDB, Micromedex, and Multum, respectively. In FDB, most of the DDIs involving 2 QT prolongers were considered moderate, while they were generally ranked higher in the other 2 KBs.

Table 4.

DDI alerts generated by the ONC list and their coverage by KBs (sev1: contraindicated, sev2: major/severe)

DDI rule	ONC alerts (% of total alerts)	% of ONC alerts covered by KB
		FDB		Micromedex		Multum
		sev1 and sev2 (%)	All severities (%)	sev1 and sev2 (%)	All severities (%)	sev1 and sev2 (%)	All severities (%)
Lovastatin and simvastatin with CYP3A4 inhibitors	25 646 (59.6)	71.9a	100	100	100	100	100
QT-prolonging agents with QT-prolonging agents	16 362 (38)	11.5	95.1	63.4	63.5	93.5	100
All other ONC DDI rules	1039 (2.4)	90.8	90.8	89.4	90.8	88.1	92.9
Total	43 047 (100)	49.4	97.9	85.9	85.9	97.2	99.8

^aIn FDB, interactions between lovastatin/simvastatin and amiodarone, diltiazem, or verapamil are strength-specific; some strengths are contraindicated with these CYP3A4 inhibitors and some interact at a lower severity level.

There were 28 410 clinical drug pairs considered contraindicated in all 3 KBs, corresponding to 1213 ingredient pairs, of which 865 pairs (71%) were not in the ONC list. Among them, 160 pairs were actually coprescribed in our dataset (9 pairs coprescribed over 100 times). Had the 160 ingredient pairs been included in the ONC list, the total number of ONC alerts would have increased by 7,633 (17.7%). Some examples of these interacting ingredient pairs are as follows (frequency of coprescription in parenthesis):

• Gemfibrozil and simvastatin (2561)

• Duloxetine and rasagiline (223)

• Cyclosporine and simvastatin (163)

• Chlorpromazine and ziprasidone (110)

• Nitroglycerin and sildenafil (83)

• Azithromycin and dronedarone (64)

• Nitroglycerin and tadalafil (56)

• Alprazolam and ketoconazole (53)

• Bupropion and rasagiline (51)

• Bromocriptine and sumatriptan (50)

DISCUSSION

Differences among the KBs

The 3 commercial DDI KBs differ significantly in their numbers of clinical drug pairs and have limited overlap. About two-thirds of the clinical drug pairs in FDB can be found in the other 2 KBs. The converse is true for Micromedex and Multum, with two-thirds of the drug pairs being unique to the KB. Contrary to earlier studies, however, we find that there is generally more agreement than disagreement on severity ranking, especially for the most severe interactions. Not surprisingly, the degree of overlap and agreement in severity ranking are considerably higher at the DDI rule level.

Impact on clinical decision support

While it is interesting to compare the KB tables, it is more important to see what the differences mean when they are actually applied in a clinical context. After all, if the differences involve only rarely prescribed drugs, the impact would be small. We find that the number of drug pairs in a KB only has a weak correlation with the number of alerts generated. Multum and Micromedex are 3 times bigger than FDB, but Multum generates 5 times and Micromedex 2 times more alerts. Notably, the amount of alerts in the contraindicated category is disproportionately small for all KBs. Contraindicated DDIs constitute 6.3, 4.3, and 2.1% of drug pairs in FDB, Micromedex, and Multum, respectively, but account for only 3.2, 1, and 0.5% of the alerts. One possible explanation is that drugs with the most severe interactions are actively avoided by prescribers. It is also possible that prescribers are already using some DDI alerting software that avert contraindicated drug combinations.

All KBs cover over 99% of the alerts generated by the ONC list, which is supposed to be used in all EHRs. However, if the KBs are customized to alert only at the contraindicated and major/severe levels, 2.8–50.6% of the ONC alerts will not be fired. Users of KBs need to consider these cases carefully to see if suppression of these alerts is appropriate, otherwise some important DDIs could be missed. It is also worth noting that 2 statins (lovastatin and simvastatin) accounted for almost 60% of the ONC alerts. Replacing these statins with atorvastatin would reduce the number of DDI alerts considerably, as its magnitude of interaction is less than the other statins on the ONC list and it became available as a generic in 2011.

Discrepancies due to QT-prolonging drugs and CYP-450 metabolism

After adjusting for unintentional differences, there is very high coverage of the ONC list ingredient pairs by all 3 KBs (98–99.9%). The discrepancies that can be attributed to editorial policies are related to 2 classes of drugs: QT prolongers and CYP-450 inhibitors. The ONC list includes all combinations of a list of QT-prolonging drugs with high risk for torsades de pointes. This represents the single largest source of ONC ingredient pairs (61%) and 38% of alerts generated. It seems that this “broad-brush” approach of using CredibleMeds List 1 to determine DDI risk has not been substantiated with evidence. The CredibleMeds website also states, “Because these actions [QT prolongation and torsades de pointes] are highly dependent on the circumstances of each drug’s use AND each patient’s clinical characteristics, we do not attempt to rank-order the drugs within each category. Therefore, we do not recommend that these lists be used to rank-order the drugs for their relative toxicity.”³² The KB editors usually look for additional evidence before alerting against a specific combination of QT-prolonging drugs. Different QT prolonger combinations can be assigned different severity levels in a KB, depending on the supporting evidence. Among the 630 QT prolonger combinations in the ONC list, 33–44% are considered contraindicated, 25–50% major/severe, and 0.3–22% moderate by the KBs.

In the ONC list, CYP3A4 inhibitors are involved in 4 rules, 177 (17%) ingredient pairs, and 60% of alerts, while CYP1A2 inhibitors are involved in 2 rules, 11 (1%) ingredient pairs, and 1% of alerts. While the ONC list cites the Food and Drug Administration published list³³ and the Flockhart table from the University of Indiana³⁴ as authoritative sources for CYP-450 inhibitors, the enumerated lists of CYP3A4 and CYP1A2 strong inhibitors include agents not ranked as strong by these sources. Some KBs also use other reference sources³⁵ to determine the classification of CYP-450 inhibition, which can lead to different recommendations. Overall, CYP-450 inhibitor class membership causes 14 ingredient pairs to be excluded from at least 1 KB.

Refinement in identifying significant QT prolonger combinations and better agreement on the classification of CYP-450 inhibitors will improve the concordance between various DDI knowledge sources. The potential clinical impact of these discrepancies is big, as over 98% of alerts generated by the ONC list involve these 2 drug classes.

Completeness of the reference list and other limitations of our study

Since the ONC list was developed based on the knowledge source of a single health care institute and designed to be a minimum starter set of alerts, one would be justified in questioning its completeness as a reference list. We did find 865 ingredient pairs that were classified as contraindicated in all KBs but not in the ONC list; some of them were also in our prescription dataset. We reviewed a small sample and confirmed that some should be considered for addition to the ONC list. In addition, we recognize the following limitations. Our study is based on the 3 commercial DDI KBs that agreed to participate. The KBs were obtained at the beginning of the study and subsequent updates were not considered. Mapping to RxNorm may be incomplete. The extent of unintentional differences (eg, those caused by versioning and mapping) was assessed in the context of missing ONC DDIs from the KBs, but not for the entire KB. The prescription dataset we used was a regional dataset over a limited period, and the results may not be generalizable.

The way forward

In view of the variability among different sources of DDI knowledge, it has been suggested that an expert panel with a centralized organizer or convener should be established to develop and maintain a standard set of DDIs for CDS in the United States,³⁶ as has been done elsewhere.³⁷ The intensive logistics and trend toward DDI customization at individual institutions makes this effort difficult to implement. The Pharmacy Quality Alliance is convening stakeholder advisory panels for the purpose of creating and maintaining a consensus-based minimum DDI dataset.³⁸ The Pharmacy Quality Alliance develops medication-use measures in areas such as medication safety, medication adherence, and appropriateness. Future DDI KBs, though, will most benefit not from bigger or better consensus panels but from large-scale patient outcomes studies (eg, derived from EHRs) and population data. Improving the availability of DDI evidence in order to best capture high-priority drug pairs (or drug triplets), categorize by severity, or assign risk based on pharmacogenomics or phenotype context or other risk factors (eg, renal impairment) will be the future of not only DDI KB data curation, but other medication-related CDS as well.

CONCLUSION

The 3 commercial DDI KBs differ considerably in their gross size, and therefore have limited overlap. However, there was generally more agreement than disagreement in the severity rankings, especially in the contraindicated category. Coverage of the ONC high-priority list is very high for all 3 KBs, in both the number of interacting ingredient pairs and potential alerts generated. Disagreements involving QT-prolonging drugs and CYP-450 inhibitors account for most of the omissions of ONC DDIs from the KBs. There is evidence to suggest that the ONC list may not cover all highly clinically significant interactions.

Contributors

KWF and OB conceived and designed the study. JKU, JC, and SHB provided the source KB tables and documentation, helped with the mapping to RxNorm, and validated the results. KWF performed the data analysis. KWF drafted the manuscript, and all authors contributed substantially to its revision.

Funding

This research was supported in part by the Intramural Research Program of the National Institutes of Health National Library of Medicine.

Competing Interests

JK-U is an employee of First Databank, JC is an employee of Truven Health Analytics, and SH-B is an employee of Cerner Multum.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.