Comparison of Nine Tools for Screening Drug-Drug Interactions of Oral Oncolytics

Purpose:

Patients with cancer are an especially vulnerable population to potential drug-drug interactions (DDIs). This makes it important to adequately screen them for DDIs. The objective of this study was to compare the abilities of nine DDI screening tools to detect clinically relevant interactions with oral oncolytics.

Methods:

Subscription-based tools (ie, PEPID, Micromedex, Lexicomp, Facts & Comparisons) and free tools (ie, Epocrates Free, Medscape, Drugs.com, RxList, WebMD) were compared for their abilities to detect clinically relevant DDIs for 145 drug pairs including an oral oncology agent. Clinical relevance was determined by a pharmacist using Stockley’s Drug Interactions. Descriptive statistics were calculated for each tool, including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and then compared grouped by free or subscription-based tools for the secondary analysis and analyzed via generalized estimating equations.

Results:

For individual metrics, PPV had overall higher values (0.88 to 0.97) relative to the low values included for sensitivity (0.65 to 0.96), specificity (0.53 to 0.93) and NPV (0.38 to 0.83). The top-performing subscription and free tools, Lexicomp and Drugs.com, had no statistically significant differences in performance. Overall, subscription tools had a significantly higher sensitivity (0.85 ± 0.017 v 0.78 ± 0.017; P = .0082) and NPV (0.57 ± 0.039 v 0.48 ± 0.032; P = .031) than free tools. No differences were observed between the specificity and PPV.

Conclusion:

Due to the low performance of some tools for sensitivity, specificity, and NPV, individual performance should be examined and prioritized on the basis of the intended use when selecting a DDI tool. If a strong-performing subscription-based tool is unavailable, a strong-performing free option, like Drugs.com, is available.

INTRODUCTION

Drug-drug interactions (DDIs) are common in patients with cancer. Approximately 40% of patients with cancer have a DDI, and approximately 16% have a major DDI.¹ A variety of factors likely contribute to this number being so high. One such factor of increased DDI risk is polypharmacy,² which is common in oncology.^2,3 Another factor is the older age⁴ of patients with cancer,⁵ because age-related declines in renal⁶ and hepatic function⁷ increase risk of DDI. Finally, many anticancer agents have narrow therapeutic ranges and high toxicity.⁸ The combination of these factors makes patients with cancer an especially vulnerable population to potential DDIs that can cause additional toxicity or prevent treatment efficacy.

Despite their high risk of DDIs, patients with cancer are often not adequately screened for DDIs. Patients with cancer are typically screened at the time of order entry by the electronic medical record (EMR); however, warnings generated at the time of order entry by the EMR are often inappropriately bypassed in inpatient⁹ and outpatient settings.¹⁰ Missed clinically relevant DDIs can be resolved by a pharmacist or clinical pharmacologist using additional DDI screening tools,^11,12 helping mitigate DDI before they cause patient harm.

It is important for practitioners to have access to an accurate DDI screening tool, and a disease-specific approach to elucidate performance differences in DDI screening tools would provide valuable information to the oncology community. Only DDI-tool sensitivity has been compared in the oncology setting,¹³, to our knowledge. Numerous free and subscription-based tools are available, and the top-performing tools have yet to be determined for use in patients with cancer. This information is critical to better screen and consequently manage DDIs in patients with cancer. Our objective was to compare the abilities of nine DDI screening tools to detect clinically relevant interactions for a large sample of oral oncolytics.

METHODS

Drug Interaction Screening Tool Selection

The following subscription-based drug interaction screening tools (Table 1) were selected on the basis of availability from the University of Michigan, with the exception of PEPID, which was accessed through a 2-week free trial: Facts & Comparisons (Wolters Kluwer, Hudson, OH), Lexicomp (Wolters Kluwer), Micromedex (Truven Health Analytics, Greenwood Village, CO), and PEPID (Phoenix, AZ). The following free tools were included (Table 1): Drugs.com (Drugsite Trust, Auckland, New Zealand), Epocrates (Epocrates, San Francisco, CA), Medscape (WebMD, New York, NY), RxList (WebMD), and WebMD (WebMD).

Table 1.

Standardization of DDI Tool Severity Ratings

All free tools known to the authors were included for comprehensiveness. Each of the tools used a different scale to rate the severity of interactions. To standardize the levels of interactions, they were classified as either none, minor, moderate, and major/contraindicated on the basis of descriptions available in each respective tool (Table 1).

Drug-Pair Selection, Screening, and Clinical Relevance

Fifty-one oral oncolytics currently in use at the University of Michigan Rogel Cancer Center were paired with medications on the basis of known DDIs, anecdotal experience from clinicians, and common use as concomitant therapy in oncology. Medication pairs (n = 145) were screened by each of the nine drug-interaction screening tools (Data Supplement). Interaction levels were recorded. A few drug pairs were not present in the tools: n = 3 for Epocrates, n = 2 for Medscape, n = 15 for RxList, n = 2 for WebMD, and n = 1 for Facts & Comparisons. For drugs not present in a tool, it was considered that no interaction was detected.

Each drug pair was examined for clinical relevance by a pharmacist using the most recent version of Stockley’s Drug Interactions¹⁴ and PubMed. Stockley’s Drug Interactions¹⁴ discusses in depth the likelihood of a clinically relevant interaction occurring and this was supplemented by PubMed searches as necessary for clarity. Sixteen oral oncolytics were not included in Stockley’s Drug Interactions, likely because they were not approved at the time of publication in the United Kingdom, where the book is published.¹⁴

Drug-interaction pairs that were considered clinically relevant and detected by a DDI screening tool as a standardized moderate or major/contraindicated interaction were considered to be a true positive. Drug-interaction pairs not considered clinically relevant and either not detected or detected as a standardized minor interaction were considered to be a true negative.

Statistical Analysis

The sensitivities, specificities, positive predictive values (PPVs), and negative predictive values (NPVs) for each tool were calculated from true positives (TPs), true negatives (TNs), false positives (FPs), and false negatives (FNs), using the following formulas:

$$Sensitivity=TP/(TP+FN)$$

$$Specificity=TN/(FP+TN)$$

$$PPV=TP/(TP+FP)$$

$$NPV=TN/(TN+FN)$$

The tools were compared pairwise for the primary analysis. The McNemar test for equal sensitivity and specificity and the Wald test for equal PPV and NPV were used to determine p values to compare the free and subscription tools for the primary analysis and pairwise for the secondary analysis. A performance score, calculated by taking the average of each tool’s combined sensitivity, specificity, PPV, and NPV, was used to select the top-performing free and subscription tools. For the secondary statistical analysis, the screening tools were grouped by free or subscription tools. Generalized linear regression models were used to estimate the sensitivities, specificities, PPV, and NPV for the two groups. The number of patients needed to screen using a subscription tool compared with a free tool to prevent one missed clinically relevant interaction was calculated by using a number-needed-to-treat formula. This calculation was also performed to compare Lexicomp and Drugs.com, the top-performing free and subscription-based tools.

RESULTS

Performance of Individual Tools and Pairwise Comparisons

Estimates of sensitivity, specificity, NPV, and PPV for each of the nine tools are reported in Table 2. The tool with the highest sensitivity and NPV was Lexicomp, and for specificity and PPV was Facts & Comparisons. The tool with the lowest sensitivity and NPV was RxList, and for specificity and PPV was PEPID. The PPV had a high overall range (0.88 to 0.97) relative to the ranges encompassing low values of sensitivity (0.65 to 0.96), specificity (0.53 to 0.93), and NPV (0.38 to 0.83). Nominal differences detected in the performance metrics for each tool were statistically significant for pairwise comparisons of some tools (Table 3). The following pairs of tools did not have any statistically significant difference in any metric: Drugs.com and Lexicomp, Epocrates Free and Medscape, Epocrates Free and RxList, Epocrates Free and WebMD, Medscape and WebMD, Medscape and Facts & Comparisons, and WebMD and Micromedex.

Table 2.

Performance Metrics of Each Tool

Table 3.

P Value of Pairwise Comparison of Each Tool

Comparison of Top-Performing Free and Subscription Tools

The tools with the highest performance score from the subscription and free groups were Lexicomp (performance score, 0.89) and Drugs.com (performance score, 0.83), respectively (Table 2). When compared directly, there were no differences between the metrics for Lexicomp and Drugs.com (Table 3). On the basis of these metrics, it is estimated that 33.3 patients would need to be screened by Lexicomp instead of Drugs.com to detect one additional clinically relevant interaction.

Comparison of Grouped Free and Subscription Tools

The group estimates of sensitivity, specificity, NPV, and PPV for the free and subscription-based tools are reported in Table 2. Subscription tools had significantly higher sensitivity (0.85 ± 0.017 v 0.78±0.017; P = .0082) and NPV (0.57 ± 0.039 v 0.48±0.032; P = .031) than free tools (Table 2). No differences were observed between the two groups for specificity and PPV (both P > .05). On the basis of these findings, it is estimated that 14.3 patients would need to be screened by a subscription tool rather than a free tool to detect one additional clinically relevant interaction.

DISCUSSION

Missed DDIs pose a great risk to patients with cancer, because many are taking medications with a narrow therapeutic range and severe toxicity.⁸ It is important for health care providers to have an accurate DDI screening tool to detect potential interactions and ensure patient safety. This study highlights the individual performance of each tool and identifies the strongest performers, including one free tool, Drugs.com.

Lexicomp and Drugs.com were the top-performing subscription and free tools in our analysis. This is consistent with results from another study, which reported comparable sensitivity for Drugs.com and Lexicomp.¹³ On the basis of our results, one additional clinically relevant interaction would be detected for every 33.3 patients screened with Lexicomp instead of Drugs.com. This estimate is much higher than the comparable estimate for the combined free and subscription-based tools (one additional interaction detected per 14.3 patients screened), which indicates there is less difference between Lexicomp and Drugs.com than between the grouped free and subscription tools. These findings overall suggest that when a strong-performing subscription-based tool such as Lexicomp is unavailable, a high-performing free tool such as Drugs.com would be a suitable substitute.

It is critical that any DDI tool used in practice is appropriately calibrated for its intended use. Improper calibration can give rise to alert fatigue, in which clinicians ignore important DDI warnings^9,10 due to the deluge of irrelevant warnings, leading to inappropriate treatment combinations that can adversely affect patient outcomes.¹⁵ Overall, the tools had a high, narrow range of PPVs (0.88 to 0.97) but a relatively wide range of NPVs (0.38 to 0.83) with some low values. The differences in values included in the ranges indicate that the tools were better at ensuring interactions categorized as severe were actually clinically relevant (ie, PPV) than they were at ensuring that interactions not categorized as severe were truly not clinically relevant (ie, NPV). If an overall strong-performing tool is unavailable, each metric can be weighed carefully by the clinician for their practice setting to aid in tool selection.

A strong DDI screening tool could standardize screening for patients and decrease missed relevant drug interactions. Van Leeuwen et al¹¹ found that DDI screening by a clinical pharmacologist with a screening tool in patients newly starting oncolytics led to a clinical intervention by the oncologist in 13% of patients and an intervention suggested by the clinical pharmacologist in an additional 14% of patients. In another study, screening of patient medication lists by pharmacists using two different databases led to recommendations made for 35% of patients.¹² On the basis of our results, a drug-interaction expert, such as a pharmacist, with the freely available DDI tool Drugs.com may be able to greatly reduce clinically relevant interactions in patients with cancer.

One limitation of this analysis is the use of a single DDI resource, Stockley’s Drug Interactions, as the gold standard for classifying DDI clinical relevance. It is possible that using a different gold standard reference, or interpretation by a different pharmacist, could produce different results. In addition, our performance score calculation rated each metric equally, whereas in practice, some metrics might be of greater importance than others, as was discussed for PPV and NPV. Finally, automated EMR-based tools were not included, which clinicians might encounter frequently in their workflow. EMR-based tools can have institution-specific alerts and were considered to be outside the scope of this analysis.

In conclusion, DDI screening tools had highly variable performance, with the notable exception of PPV, when assessing DDI with oral oncolytics. The performance metrics of individual tools should be carefully considered when deciding which tool to use for DDI screening to ensure proper calibration for its intended use and prevention of alert fatigue. Overall, subscription-based tools had stronger performance than free tools. However, Drugs.com is a freely available tool with performance similar to the best-performing subscription tool that should be strongly considered for use in oncology practices that do not have access to subscription DDI tools.

Supplementary Material

jop-14-e368-s001.png ^{(94.7KB, png)}

AUTHOR CONTRIBUTIONS

Conception and design: Lauren A. Marcath, Shawna L. Kraft, Daniel L. Hertz

Collection and assembly of data: Lauren A. Marcath, Emily K. Hoylman, Shawna L. Kraft, Daniel L. Hertz

Data analysis and interpretation: Lauren A. Marcath, Jingue Xi, Kelley M. Kidwell, Daniel L. Hertz

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Comparison of Nine Tools for Screening Drug-Drug Interactions of Oral Oncolytics

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jop/site/ifc/journal-policies.html.

Lauren A. Marcath

Other Relationship: Working with PEPID to create a drug interaction screening tool for use during clinical trial enrollment.

Jingyue Xi

No relationship to disclose

Emily K. Hoylman

No relationship to disclose

Kelley M. Kidwell

No relationship to disclose

Shawna L. Kraft

No relationship to disclose

Daniel L. Hertz

Other Relationship: Working with PEPID to create a drug interaction screening tool for use during clinical trial enrollment.