Skip to main content
Journal of the American Medical Informatics Association : JAMIA logoLink to Journal of the American Medical Informatics Association : JAMIA
. 2019 Apr 26;26(10):934–942. doi: 10.1093/jamia/ocz033

Structured override reasons for drug-drug interaction alerts in electronic health records

Adam Wright 1,2,3,4,, Dustin S McEvoy 4, Skye Aaron 1, Allison B McCoy 5, Mary G Amato 1,6, Hyun Kim 7, Angela Ai 8, James J Cimino 9, Bimal R Desai 10, Robert El-Kareh 11, William Galanter 12, Christopher A Longhurst 11, Sameer Malhotra 13, Ryan P Radecki 14, Lipika Samal 1,2, Richard Schreiber 15, Eric Shelov 10, Anwar Mohammad Sirajuddin 16, Dean F Sittig 17
PMCID: PMC6748816  PMID: 31329891

Abstract

Objective

The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.

Materials and Methods

We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.

Results

Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: “will monitor or take precautions,” “not clinically significant,” and “benefit outweighs risk.”

Discussion

We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.

Conclusions

Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Keywords: drug-drug interactions, override reasons, clinical decision support, electronic health records, alerts

INTRODUCTION

Adverse drug events (ADEs) are a type of patient harm that commonly occur in both inpatient and outpatient settings.1–6 ADEs resulting from medication errors during prescribing, dispensing, or administration are considered preventable ADEs.4 Common types of medication errors include incorrect dosing or frequency, prescribing medications to which a patient is allergic, prescribing nephrotoxic drugs in patients with decreased renal function, and prescribing multiple drugs that have undesirable interactions with each other.

One strategy for preventing medication errors is clinical decision support (CDS) alerts, embedded within a computerized provider order entry (CPOE) system. Systematic reviews by Kaushal et al7 and Kuperman et al8 found that medication-related CDS alerts have potential to reduce the rates of ADEs. However, medication-related CDS alerts are frequently overridden,9,10 sometimes with good cause, as adverse events rarely occur when prescribers override even the highest-level alerts.11,12 Historically, the rationale for high override rates was alert fatigue and poor usability, but recent research suggests a more complex picture, including inaccurate warnings and incorrect judgments by the prescriber.12–17

One of the most common types of medication alerting is drug-drug interaction (DDI) checking, which the United States meaningful use incentive program required for adoption of electronic health records (EHRs).18,19 The evidence base for including DDIs as a required EHR functionality was not as strong as that for other types of CDS, such as renal dose adjustment.20–25 However, DDI checking is simple to implement, as a variety of medication knowledge bases are available with data on potential drug interactions, typically stratified by severity. In a prior study, we found that the clinical content of DDI alerts varies considerably across institutions and EHRs,26 even for a set of alerts identified as very high priority by a national consensus group.27

One controversy regarding CDS alerts is whether to employ hard stops—alerts a user cannot bypass.28 Hard stops are rare in practice, even for high-severity alerts.26 Instead, users can typically either accept an alert or override it.26 When users override an alert, there is often opportunity to provide a reason, typically drawn from a coded list. Users may also be able to supply a free-text reason instead of, or in addition to, a coded reason.

Literature on the rationale for collecting override reasons is scant; however, some common reasons include:

  • Documenting the ordering provider’s awareness of the potential for harm and rationale for overriding the alert; this contemporaneous documentation may be of use in the case that harm actually occurs or there is an allegation of malpractice

  • Communicating the provider’s awareness of the potential interaction downstream, such as to the pharmacy. Particularly in the inpatient setting, this may reduce the need for the pharmacist to call the ordering provider to verify the order. Override reasons rarely transmit to outpatient pharmacies, so this is less likely to be helpful in outpatient settings29

  • Trying to influence the behavior of the provider by forcing them to justify their decision to override an alert

  • Collecting data about what alerts prescribers commonly override, and why, to support improvements in knowledge bases and alerting systems30

In this study, we collected and analyzed data on DDI override reasons from organizations across the United States, with a goal of understanding available override reasons and how prescribers use them.

MATERIALS AND METHODS

Starting with the sample of organizations from our prior paper on DDIs,26 we identified sites across the country that used an EHR for the entirety of 2016, had active DDI alerts and allowed users to provide a coded reason for overriding alerts (whether optional or mandatory). We invited those organizations to participate in this new study by providing data on their DDI alert overrides.

Each participating site queried its data warehouse to find all DDI alerts shown during 2016 and determined alert acceptance and override rates. For overridden alerts, each site also determined whether an override reason was given and, if so, what it was. Sites aggregated the data to provide counts on the total number of DDI alerts shown to users, the number of alerts that were overridden, and the number of times prescribers chose each coded DDI override reason. Some sites also allowed free-text override reasons, but we did not collect these, as our prior work suggested that free-text entries frequently contained protected health information and sometimes even passwords. Further, several study sites were willing to participate only if the data sharing was limited to deidentified aggregate data.

Researchers at Brigham and Women’s Hospital collected, combined, and analyzed the data from each site. After collecting all of the override reasons from each site (shown in Supplementary Appendix) we conducted a multistage iterative card sort method to categorize the override reasons.31–33 We have previously used this method of empirical categorization in other studies of CDS.34–42 In some cases, reasons were not clear, and we sought clarification from the contributing site as needed. We also assessed whether each override reason was relevant to DDIs, as some sites offered override reasons that were clearly not relevant to DDIs, typically because they used a single list of override reasons for all alert types.

The Partners HealthCare Human Subjects Committee reviewed and approved this study.

RESULTS

Ten sites participated in the study. They are shown in Table 1.

Table 1.

Participating sites, including location and electronic health record used

Organization Location Electronic Health Record
Children's Hospital of Philadelphia Philadelphia, Pennsylvania Epic
Holy Spirit Hospital Camp Hill, Pennsylvania Allscripts (Sunrise)
Kaiser Permanente Northwest Portland, Oregin Epic
Memorial Hermann Houston, Texas Cerner
Partners HealthCare Boston, Massachusetts Epic
University of Alabama at Birmingham Birmingham, Alabama Cerner
UC San Diego Health San Diego, California Epic
University of Illinois at Chicago Chicago, Illinois Cerner
University of Texas Faculty Physicians Houston, Texas Allscripts (Enterprise)
Weill Cornell Medicine New York, New York Epic

Table 2 shows overall override statistics by site. Table 2 and subsequent tables present each site’s data by a number instead of name, and the order of the sites is shuffled to preserve anonymity. Consistent with past literature, the overall override rate is very high, at 91%. Two sites required override reasons for all DDI alerts. The other sites made override reasons optional, and users provided them between 8% and 82% of the time. Most sites offered fewer than 10 unique, coded override reason options, but one site offered 100 choices.

Table 2.

Override statistics by site, including anonymized site number and details of alert firing and override rates

Site Alerts Alerts Overridden Overrides With Reasons Unique Override Reasons
1 521 197 495 350 (95) 220 399 (44) 7
2 133 649 122 455 (92) 14 718 (129) 100
3 37 333 24 089 (65) 3979 (17) 6
4 59 295 56 532 (95) 10 329 (18) 6
5 122 659 96 775 (79) 25 692 (27) 6
6 1 434 937 1 335 764 (93) 1 335 764 (100)a 16
7 311 657 273 203 (88) 21 792 (8) 3
8 164 720 141 827 (86) 114 740 (81) 9
9 206 765 186 180 (90) 186 180 (100)a 21
10 104 136 93 610 (90) 77 213 (82) 3
Total 3 096 348 2 825 785 (91) 2 010 806 (71) 177

Values are n (%) unless otherwise indicated.

aOverride reasons were required at these 2 sites.

In total, there were 177 override reasons offered across the 10 sites. Table 3 shows the results of the categorization exercise, with 12 categories identified. Three categories accounted for 78% of all overrides: “will monitor or take precautions,” “not clinically significant,” and “benefit outweighs risk.” The table includes representative verbatim examples from the sites’ CPOE systems for each category, as well as the number of overrides in our sample in each category, the reasons unique to each category, and the number of sites offering at least 1 coded override reason in each category.

Table 3.

Categories of DDI reasons

Category Examples (Verbatim) Overrides Unique Reasons Sites With Reason in Category
Will monitor or take precautions
  • Will monitor patient for interaction

  • Interaction noted

  • Will take precautions

  • Aware of interaction

  • Will follow/monitor

795 990 7 5
Not clinically significant
  • This is not clinically significant

  • This alert is NOT useful

  • Disagree with stated interactions(s)

  • Inaccurate warning

411 782 22 9
Benefit outweighs risk
  • Potential benefit outweighs risk

  • No good alternative

365 734 14 9
Patient tolerated previously
  • Patient tolerated before

  • Medication tolerated by patient

  • Current therapy

117 923 5 5
Dose adjusted
  • Dosage appropriately adjusted

  • Dosing interval appropriately adjusted

  • Have adjusted or will adjust dose

101 593 4 3
Not related to DDIs
  • Patient does not have stated allergy

  • New active cases in same unit

  • Barrett's esophagus

  • Expect improved renal function

  • Clearance wrong due to wrong ht/length

  • Patient is NOT pregnant

84 911 89 7
Not ordering a medication
  • Chart review: no action

  • Entering an historical medication

  • Patient expired

51 570 6 2
Alert is not the recipient’s responsibility
  • Treatment plan requirement

  • Defer to primary physician

  • Defer to pharmacist

  • Spoke with transplant service

52 886 9 4
Agreement, though alert was overridden
  • Order this agent, will stop other drug

  • RPh reviewed-acted

11 969 4 3
See comments
  • See comments

  • Other (free text)

8387 4 3
Order is urgent
  • Deferring to other priorities

  • Emergency

  • The drug is needed urgently

5220 11 4
Error in data
  • Error in data

2841 2 2

DDI: drug-drug interaction; RPh: registered pharmacist.

DISCUSSION

Considerable variation in reasons offered to providers is apparent from our data. Most sites offer relatively few override reasons, though some offer dozens, and 1 site offers 100. The most widely used category of override reason, “will monitor or take precautions,” is only available at half of the participating sites, and no category of reasons is offered at every site. This variation likely stems from a lack of national consensus or standards for override reasons, leaving each site and vendor to determine their own list.

Further, 7 of 10 sites offered users at least 1 coded override reason for DDI alerts that were clearly irrelevant to DDI alerting. For example, many sites allowed users to choose reasons like “patient does not have stated allergy,” “clearance wrong due to wrong ht/length,” or “patient is NOT pregnant,” which apply to other types of medication alerting, but not to DDIs. In discussions with sites, we found that many used the same list of override reasons for all categories of medication alerting (eg, allergy alerting, drug-disease interaction, pregnancy alerting, dose range checking, renal and hepatic dose adjustments). Some sites reported their EHR software does not allow different lists to be used for different alert types, while others simply had not configured different lists.

When the list is too long, instead of picking an accurate reason, users may simply pick a reason at random, and, if a reason is not mandated, users may not provide a reason at all. Prior studies have shown that when mandatory free-text reasons are required, users often enter a space or random characters to move past the screen.29,43 It may similarly be true that with a prespecified list, many users select the top item from the list or a random item so they can move on with the medication order. Thus, one caveat of our results is that users may not always have picked a reason they felt was accurate; indeed, this is certainly true, as seen by the override reasons that were not related to DDIs. Improving the list of override reasons to be shorter and more relevant to alerts may be useful for improving the accuracy of override reasons.

Several categories of override reasons merit special discussion:

  • Will monitor or take precautions: Some override reasons in this category were generic, such as “will monitor,” but others mentioned specific precautions. An example of this is cardiac monitoring when ordering multiple drugs with the potential to prolong the QT interval, or additional monitoring of the international normalized ratio (INR) when prescribing drugs interacting with warfarin. However, these precautions were typically not actionable (only a single site had created actionable precautions, and only for a small number of interactions), which means users must separately order the precaution, increasing the time required and the likelihood of omitting the precaution.

  • “Not clinically significant” and “benefit outweighs risk”: Based on the literature in the reported previously, users are frequently correct when they indicate that a potential interaction is not clinically significant or the benefit of the drug outweighs the risks. These overrides represent a data source for identifying alerts to potentially disable or improve by tailoring. They may also represent an important source of documentation that the ordering provider was aware of the interaction but concluded the benefits outweighed the risks.

  • Patient tolerated previously: Although prior tolerance is not a guarantee of ongoing safety, it is an important indicator. Finding prior tolerance can be difficult, as not all of a patient’s past medications may be documented in the EHR, but it may be possible in certain cases.

  • Dose adjusted: Changing the dose of 1 or both drugs can sometimes mitigate a DDI. Classically, warfarin interacts with a variety of antibiotics, including sulfamethoxazole, leading to an increase in the INR, indicating diminished coagulability and a higher risk of bleeding.44 One option is to avoid these antibiotics in patients taking warfarin, but it may also be appropriate to reduce empirically the dose of warfarin and monitor the INR more frequently.45

  • Alert is not the recipient’s responsibility: This was a diverse and potentially troublesome category, which was most commonly used in situations where multiple people were partially responsible for an order. For example, users might report that the order was required by a pre-existing, institutionally approved treatment plan, or specified by the transplant team but entered by someone else. In other cases, users chose to “defer to primary physician” or “defer to pharmacist” even though they placed (and often, were ultimately responsible for) the order. In these cases, the CPOE system does not necessarily direct those deferrals to the indicated party to acknowledge acceptance of that responsibility, creating further risk. In a CPOE system, certainty about the party responsible for the order is needed.

  • Agreement, though alert was overridden: In this category, users override an alert, but indicate they nonetheless intend to follow it by, for example, discontinuing another drug. There is an inherent risk with this category, as users may fail to take the action they indicate.

Across all the categories described in Table 3, one theme recurred: there is great variation in DDI alert reasons. We found most were not actionable, some included reasons irrelevant to DDI alerting at all, some may delay entry of orders required for safe prescribing of the interacting drugs, and others lack clear communication. This variability impedes accurate data collection and reporting. From these data, it seems clear that, in addition to improving the quality of override reasons, alerts need to be more specific to the drug pair and to the patient. We believe healthcare organizations, in partnership with EHR vendors and medication knowledge–based vendors,46,47 should create more specific, actionable alerts. An ideal alert would:

  1. Be tiered appropriately by severity. For example, severe interactions may be interruptive (eg, clarithromycin and simvastatin, which can cause rhabdomyolysis and acute kidney disease) while less severe interactions could be passive (eg, levothyroxine and calcium, which may reduce levothyroxine absorption if the doses are not separated by time),9,48 or not shown at all. Many overrides are given because the user considers the interaction to not be clinically significant—when appropriate, eliminating these alerts or reducing their severity should reduce the number of overrides.

  2. Display, and take into account, relevant data. As a representative example, DDI alerts related to QT prolongation should display and consider the most recent corrected QT interval, while alerts related to warfarin should show the most recent INR. This could reduce “not clinically significant” and “benefit outweighs risk” overrides by suppressing alerts that, while potentially valuable when only looking at the pair of drugs involved, are no longer relevant once additional data is taken into account.

  3. Allow the user to take specific actions to monitor and reduce the risk of harm directly from the alert notification window. In general, we believe alerting systems should be actionable.49,50 Consider the representative example of DDI alerts related to QT prolongation. These alerts should allow the user to order an electrocardiogram or cardiac telemetry, while alerts related to warfarin should allow the user to order more frequent monitoring of the INR or make appropriate empiric adjustments to the warfarin dose directly from the alert, when such adjustments are supported by evidence. If appropriate precautions are already in place, it would be ideal to suppress the alert; if not, the applicable precautions should display, and be orderable, on the alert screen. Users are already indicating that they will monitor or take follow-up actions by choosing these as override reasons; however, in the systems under study, none of these override reasons actually accomplish an action—they simply log the user’s intent.

  4. Allow the user to cancel any new medications being ordered and also permit discontinuation of existing medications that are part of the interaction. Some systems under study did not allow users to discontinue existing medications in an interaction, so the user is forced to override the alert, indicate that they plan to discontinue the existing drug, and then subsequently remember to actually take that action.

  5. Provide a small number of tailored, accurate override reasons. These reasons should be customized to the exact drug interaction in the alert and should cover all the categories identified in Table 3 that are relevant to DDIs.

  6. Allow the user to document his or her reasoning, as desired, such as with a coded override reason, a free-text comment, or both. This documentation should be stored and readily visible in the medical record and made available to other providers and to CDS developers.

  7. Communicate documented reasoning about DDIs to other members of the care team, such as the nurse or pharmacist. In cases where a user’s override indicates deferral to another party (eg, “defer to pharmacist” or “defer to primary physician”), the CPOE system should require the ordering provider to select the user responsible for dealing with the alert, and then require the responsible party to review and approve the order. Depending on clinical urgency and institutional policies and procedures, this approval might be done before the drug is dispensed.

  8. Allow the user to provide feedback directly to the team responsible for maintaining the local EHR alerting system and the drug interaction knowledge base, so the knowledge base can be continuously improved. The team receiving this feedback should regularly monitor it and respond directly to users about the feedback they provide in a timely fashion.

Satisfying these recommendations may be resource-intensive. Organizations may want to prioritize the most commonly firing, most frequently overridden, and most risky interactions for this treatment, while using basic functionality for others. We believe this would be feasible if starting with a short list of highly actionable DDI pairs, such as the 7 previously described for adults and 19 for pediatric patients.51,52 Further, drug knowledge base vendors should enhance data in their knowledge bases to include alternatives, specific monitoring recommendations and relevant information. This information should be structured and coded (not just present in written monographs), and EHR vendors should be able to use this information to tailor alerts shown to users. Although there would be some cost to add this information to drug knowledge bases and maintain it, doing so at the knowledge base level would be more efficient (and likely more effective) than leaving the work to implementing clinical sites. Additionally, CDS implementers and knowledge base vendors should also focus resources on improving the specificity of DDI alerting more broadly—after all, producing fewer inaccurate alerts reduces the need to override alerts and provide override reasons. Groups such as Leapfrog will need to validate these shorter lists of actionable pairs so health systems are not incentivized to overalert by publicly reported test results.53–55

Our study has several strengths. First, it has a broad sample of U.S.-based hospitals using CPOE with DDI alerting, using a range of the most common CPOE systems. Second, the sample of over 2 million recorded DDI reasons is substantial, allowing for robust analysis of frequency.

Our study also has some limitations. First, we did not analyze free-text override comments. Free-text reasons may contain other categories of overrides not seen in the coded data. We recently conducted a separate study of free-text override reasons for nonmedication alerts at a single site and found them to be highly informative, so this may be a productive line of research in the future.56 Second, we did not receive data from the sites about the drugs that caused the overrides, so we were unable to stratify our analysis by drug type. Third, we did not collect data on the user experience related to override reasons, such as how many reasons were shown on the screen or whether scrolling was required, so we were not able to assess how these user interface issues might have affected override reasons. These data are difficult to collect retrospectively because changes to the user interface over time and differences in the interface across platforms and devices are not consistently recorded. Finally, we did not measure usefulness of the override reasons provided. As described in previously, override reasons may be used to encourage careful consideration by ordering providers, communicate to pharmacists, or document reasoning for future medicolegal need. We do not have outcome data to suggest which reasons are useful in supporting these goals or whether DDI alerts are the best mechanisms to serve these purposes.

CONCLUSION

Organizations, EHR vendors and third-party medication knowledge vendors should improve the options available to their providers who choose to override DDI alerts. These options should be specific to the drugs involved and, where possible, include concrete clinical actions prescribers can take directly from the alert. Furthermore, both EHR vendors and medication knowledge base vendors should enhance the functionality and capabilities of their respective product offerings to facilitate more targeted alerting based on other data in the EHR. Taken together, these improvements should lead to better CDS, safer patient care, and reductions in alert burden for clinicians.

FUNDING

This work was supported by National Library of Medicine of the National Institutes of Health grant number R01LM011966. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

AUTHOR CONTRIBUTIONS

AW wrote the first draft of the manuscript. DSM, SA, and AA performed the analysis. All authors participated in data acquisition and reporting. All authors provided data, contributed to the analysis and made critical revisions for important intellectual content.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

CONFLICT OF INTEREST STATEMENT

None declared.

Supplementary Material

ocz033_Supplementary_Data

APPENDIX

List of coded override reasons aggregated from all 10 sites, by category.

AGREEMENT, THOUGH ALERT WAS OVERRIDDEN

  • Contraindicated (2 sites)

  • Order this agent, will stop other drug (1 site)

  • RPh Reviewed-Acted (1 site)

ALERT IS NOT THE RECIPIENT’S RESPONSIBILITY

  • Agree & reviewed with ordering physician (1 site)

  • Contacted Prescriber & Confirmed Order (1 site)

  • Defer to primary physician (2 sites)

  • Deferred to Pharmacist (1 site)

  • Per protocol (1 site)

  • Spoke with Transplant Service (1 site)

  • Treatment plan requirement (2 sites)

BENEFIT OUTWEIGHS RISK

  • Benefit outweighs risk in this patient (1 site)

  • Benefit outweighs risk (3 sites)

  • Benefit outweighs risk/discussed with pt (1 site)

  • Benefit outweighs the risk (1 site)

  • Benefits outweigh risks (1 site)

  • Intolerance, Relative (1 site)

  • Low risk; appropriate warnings given to patient (1 site)

  • Medication combination needed for this patient (1 site)

  • No good alternative (1 site)

  • No reasonable alternatives for this patient (1 site)

  • Potential benefit outweighs risk (1 site)

  • Reaction does not preclude therapy (1 site)

DOSE ADJUSTED

  • Adjusted dose (1 site)

  • Dosage appropriately adjusted (1 site)

  • Dosing interval appropriately adjusted (1 site)

  • Have adjusted or will adjust dose (1 site)

ERROR IN DATA

  • Data error (2 sites)

NOT CLINICALLY SIGNIFICANT

  • **Not Clinically Relevant (1 site)

  • *Erroneous Alert (1 site)

  • Clinical Judgement (1 site)

  • Disagree with recommendation (2 sites)

  • Disagree with stated interactions(s) (1 site)

  • Does not apply to patient (1 site)

  • Dose Appropriate (1 site)

  • Erroneous (specify) (1 site)

  • Evidence regarding alert is inconclusive (1 site)

  • Inaccurate warning (1 site)

  • Insignificant (1 site)

  • Low risk (1 site)

  • Medically indicated for diagnosis (1 site)

  • Not applicable (3 sites)

  • Patient failed lower dose (1 site)

  • Prescription refill/pt not available (1 site)

  • Reason Reviewed (1 site)

  • This alert is NOT useful (1 site)

  • This is not clinically significant (1 site)

NOT ORDERING a MEDICATION

  • Chart Review: No Action (1 site)

  • Entering an historical medication (1 site)

  • Not Applicable - Alabama Organ Center (1 site)

  • Order already exists (2 sites)

  • Patient expired (1 site)

NOT RELATED TO DDIS

  • Abnormal platelet function (1 site)

  • Adverse effect/reaction not an allergy (1 site)

  • Allergy entered in error (1 site)

  • Allergy history is unreliable. Benefit of treatment outweighs risk (1 site)

  • Allergy information not correct (1 site)

  • Already discussed with Radiologist (1 site)

  • Alternate dosing required (free text) (1 site)

  • Anticoagulation reversed since last INR (1 site)

  • Anti-Platelet Drug therapy (1 site)

  • Any bleed with a high INR (1 site)

  • Assumed Upper GI Bleed (1 site)

  • Barrett's Esophagus (1 site)

  • B-Cell CLL infection prevention (1 site)

  • Bleeding due to thrombocytopenia (1 site)

  • Cardiac disease (1 site)

  • Chronic red blood cell transfusion (1 site)

  • Clearance close enough for safe use (1 site)

  • Clearance wrong due to wrong ht/length (1 site)

  • Clearance wrong due to wrong weight (1 site)

  • Confirmed w pt/EMR no cross sensitivity (1 site)

  • Dermatomyositis (1 site)

  • Desensitization procedure (1 site)

  • Doubtful allergy (1 site)

  • Drug is being used for prophylaxis (1 site)

  • Drug is not dangerous in this trimester (1 site)

  • Duplicate meds necessary for dosing (1 site)

  • Expect improved renal function (1 site)

  • Expect INR to have trended down (1 site)

  • Expect pt has normal renal function (1 site)

  • H & P not completed by order clinician (1 site)

  • High INR from warfarin (1 site)

  • Hyperalimentation to be cancelled (1 site)

  • Indication for antiplatelet therapy (1 site)

  • Insurance information incorrect (1 site)

  • Intracerebral hemorrhage with high INR (1 site)

  • Intracerebral hemorrhage with normal INR (1 site)

  • Maximum doe in the 2 orders is < 4 gms (1 site)

  • Medication allergy rarely a problem with current order (1 site)

  • Medication Intolerance, not true allergy (1 site)

  • Most recent creatinine likely an error (1 site)

  • Multifocal motor neuropathy (1 site)

  • Myasthenia Gravis (1 site)

  • Neurosurgical patient (1 site)

  • New active cases in same unit (1 site)

  • Ongoing/anticipated massive transfusion (1 site)

  • Oral diet to be cancelled (1 site)

  • Patient allergic (2 sites)

  • Patient does not have DM Type II (1 site) [Mismatch]

  • Patient does not have ESRD (1 site)

  • Patient does not have stated allergy (1 site)

  • Patient has hepatitis B (1 site)

  • Patient informed and didn't decline (1 site)

  • patient is getting hemodialysis (1 site)

  • Patient is NOT Pregnant (1 site)

  • Patient not available (2 sites)

  • Patient refused (2 sites)

  • Patient requires a HCT > 36% (1 site)

  • Patient undergoing dialysis (1 site)

  • Patient wants services anyway (1 site)

  • Patient with nausea and vomiting (1 site)

  • Plan to reverse anticoagulation (1 site)

  • Plasmapheresis in a transplant patient (1 site)

  • Poor results on prior application (2 sites)

  • Prior vaccination incorrect (1 site)

  • PRN Medication (1 site)

  • Pt does not have any diagnosis above (1 site)

  • Pt does not have PCOS or DM Type II (1 site)

  • Pt in ICU, system location incorrect (1 site)

  • Pt is a child with no discharge meds (1 site)

  • Rapid blood loss occurring or anticipate (1 site)

  • Readmit to another service (1 site)

  • Recent creatinine normal (1 site)

  • Recent higher PLT count (1 site)

  • Recent PLT transfusion (1 site)

  • Replacement fluid for exchange (1 site)

  • Specimen Obtained (1 site)

  • Steroid Ulcer Prophylaxis (1 site)

  • Stress Ulcer Prophylaxis (1 site)

  • Therapy for loading, now, or stat and then (1 site)

  • Therapy for variable, scaled or tapering schedule (1 site)

  • This HGB is thought to be an error (1 site)

  • Unverified (1 site)

  • Upper GI Bleed (1 site)

  • Will cancel the ketorolac (1 site)

  • Will cancel the NSAID (1 site)

ORDER IS URGENT

  • deferring due to other priorities (1 site)

  • Deferring to other priorities (1 site)

  • Discharge is urgent (1 site)

  • Emergency (1 site)

  • For Procedure (1 site)

  • MD Ordered, Urgent (1 site)

  • Order required for emergency (1 site)

  • Preoperative or bleeding patient (1 site)

  • Pt leaving AMA (1 site)

  • The drug is needed urgently (1 site)

  • Worsening Clinical Condition (1 site)

PATIENT TOLERATED PREVIOUSLY

  • Current Therapy (1 site)

  • Medication tolerated by patient (1 site)

  • Medication/combination known to be tolerated or at home (1 site)

  • Patient tolerated before (1 site)

  • Prev/Now tolerated (1 site)

SEE COMMENTS

  • Free Text Override Reason (1 site)

  • FreeText (1 site)

  • Other (free text) (1 site)

  • See comments (1 site)

WILL MONITOR OR TAKE PRECAUTIONS

  • Aware of interaction, will follow/monitor (1 site)

  • Essential therapy, will take precautions (1 site)

  • Interaction noted, will take precautions (1 site)

  • Premed ordered for adverse side effect (1 site)

  • Will monitor (1 site)

  • Will monitor labs for patient changes (1 site)

  • Will monitor patient for interaction (1 site)

REFERENCES

  • 1. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP.. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA 1997; 2774: 301–6. [PubMed] [Google Scholar]
  • 2. Melmon KL. Preventable drug reactions–causes and cures. N Engl J Med 1971; 28424: 1361–8. [DOI] [PubMed] [Google Scholar]
  • 3. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA 1995; 2741: 29–34. [PubMed] [Google Scholar]
  • 4. Bates DW, Leape LL, Petrycki S.. Incidence and preventability of adverse drug events in hospitalized adults. J Gen Intern Med 1993; 86: 289–94. [DOI] [PubMed] [Google Scholar]
  • 5. Bates DW, Boyle DL, Vander Vliet MB, Schneider J, Leape L.. Relationship between medication errors and adverse drug events. J Gen Intern Med 1995; 104: 199–205. [DOI] [PubMed] [Google Scholar]
  • 6. Lesar TS, Briceland L, Stein DS.. Factors related to errors in medication prescribing. JAMA 1997; 2774: 312–7. [PubMed] [Google Scholar]
  • 7. Kaushal R, Shojania KG, Bates DW.. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 16312: 1409–16. [DOI] [PubMed] [Google Scholar]
  • 8. Kuperman GJ, Bobb A, Payne TH, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 141: 29–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Wright A, Aaron S, Seger DL, Samal L, Schiff GD, Bates DW.. Reduced effectiveness of interruptive drug-drug interaction alerts after conversion to a commercial electronic health record. J Gen Intern Med 2018; 33 (11): 1868–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. van der Sijs H, Aarts J, Vulto A, Berg M.. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 132: 138–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Peterson JF, Bates DW.. Preventable medication errors: identifying and eliminating serious drug interactions. J Am Pharm Assoc (Wash) 2001; 412: 159–60. [DOI] [PubMed] [Google Scholar]
  • 12. Wong A, Rehr C, Seger DL, et al. Evaluation of harm associated with high dose-range clinical decision support overrides in the intensive care unit. Drug Saf 2018. Dec 1 [E-pub ahead of print]. [DOI] [PubMed] [Google Scholar]
  • 13. McCoy AB, Thomas EJ, Krousel-Wood M, Sittig DF.. Clinical decision support alert appropriateness: a review and proposal for improvement. Ochsner J 2014; 142: 195–202. [PMC free article] [PubMed] [Google Scholar]
  • 14. McCoy AB, Waitman LR, Lewis JB, et al. A framework for evaluating the appropriateness of clinical decision support alerts and responses. J Am Med Inform Assoc 2012; 193: 346–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Rehr CA, Wong A, Seger DL, Bates DW.. Determining Inappropriate Medication Alerts from “Inaccurate Warning” Overrides in the Intensive Care Unit. Appl Clin Inform 2018; 092: 268–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Slight SP, Seger DL, Nanji KC, et al. Are we heeding the warning signs? Examining providers’ overrides of computerized drug-drug interaction alerts in primary care. PLoS One 2013; 812: e85071. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS.. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 16321: 2625–31. [DOI] [PubMed] [Google Scholar]
  • 18. Blumenthal D, Tavenner M.. The “meaningful use” regulation for electronic health records. N Engl J Med 2010; 3636: 501–4. [DOI] [PubMed] [Google Scholar]
  • 19. Wright A, Henkin S, Feblowitz J, McCoy AB, Bates DW, Sittig DF.. Early results of the meaningful use program for electronic health records. N Engl J Med 2013; 3688: 779–80. [DOI] [PubMed] [Google Scholar]
  • 20. Phansalkar S, van der Sijs H, Tucker AD, et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 203: 489–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Peters LB, Bahr N, Bodenreider O.. Evaluating drug-drug interaction information in NDF-RT and DrugBank. J Biomed Semantics 2015; 6: 19.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Hoffman S, Podgurski A.. Drug-drug interaction alerts: emphasizing the evidence. Saint Louis Univ J Health Law Policy 2012; 52: 2012-22. [Google Scholar]
  • 23. Barrons R. Evaluation of personal digital assistant software for drug interactions. Am J Health Syst Pharm 2004; 614: 380–5. [DOI] [PubMed] [Google Scholar]
  • 24. Page N, Baysari MT, Westbrook JI.. A systematic review of the effectiveness of interruptive medication prescribing alerts in hospital CPOE systems to change prescriber behavior and improve patient safety. Int J Med Inform 2017; 105: 22–30. [DOI] [PubMed] [Google Scholar]
  • 25. Wong A, Wright A, Seger DL, Amato MG, Fiskio JM, Bates D.. Comparison of overridden medication-related clinical decision support in the intensive care unit between a commercial system and a legacy system. Appl Clin Inform 2017; 083: 866–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. McEvoy DS, Sittig DF, Hickman TT, et al. Variation in high-priority drug-drug interaction alerts across institutions and electronic health records. J Am Med Inform Assoc 2017; 242: 331–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Phansalkar S, Desai AA, Bell D, et al. High-priority drug-drug interactions for use in electronic health records. J Am Med Inform Assoc 2012; 195: 735–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Powers EM, Shiffman RN, Melnick ER, Hickner A, Sharifi M.. Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review. J Am Med Inform Assoc 2018; 2511: 1556–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Chused AE, Kuperman GJ, Stetson PD.. Alert override reasons: a failure to communicate. AMIA Annu Symp Proc 2008; 2008:111–5. [PMC free article] [PubMed] [Google Scholar]
  • 30. Wright A, Ash JS, Aaron S, et al. Best practices for preventing malfunctions in rule-based clinical decision support alerts and reminders: results of a Delphi study. Int J Med Inform 2018; 118: 78–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Hedden H. The Accidental Taxonomist. 2nd ed Medford, NJ: Information Today; 2016. [Google Scholar]
  • 32. Abbas J. Structures for Organizing Knowledge: Exploring Taxonomies, Ontologies, and Other Schemas. New York: Neal-Schuman; 2010. [Google Scholar]
  • 33. Lincoln YS, Guba EG.. Naturalistic Inquiry. Beverly Hills, CA: Sage; 1985. [Google Scholar]
  • 34. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH.. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; 135: 547–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Wright A, Goldberg H, Hongsermeier T, Middleton B.. A description and functional taxonomy of rule-based decision support content at a large integrated delivery network. J Am Med Inform Assoc 2007; 144: 489–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Wright A, Sittig DF, Ash JS, et al. Development and evaluation of a comprehensive clinical decision support taxonomy: comparison of front-end tools in commercial and internally developed electronic health record systems. J Am Med Inform Assoc 2011; 183: 232–42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Dykstra RH, Ash JS, Campbell E, et al. Persistent paper: the myth of “going paperless”. AMIA Annu Symp Proc 2009; 2009: 158–62. [PMC free article] [PubMed] [Google Scholar]
  • 38. Campbell EM, Guappone KP, Sittig DF, Dykstra RH, Ash JS.. Computerized provider order entry adoption: implications for clinical workflow. J Gen Intern Med 2009; 241: 21–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K.. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform 2009; 78 (Suppl 1): S69–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K.. Exploring the unintended consequences of computerized physician order entry. Stud Health Technol Inform 2007; 129 (Pt 1): 198–202. [PubMed] [Google Scholar]
  • 41. Ai A, Wong A, Amato M, Wright A.. Communication failure: analysis of prescribers' use of an internal free-text field on electronic prescriptions. J Am Med Inform Assoc 2018; 256: 709–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Wright A, Ai A, Ash J, et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 255: 496–506. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Grizzle AJ, Mahmood MH, Ko Y, et al. Reasons provided by prescribers when overriding drug-drug interaction alerts. Am J Manag Care 2007; 1310: 573–8. [PubMed] [Google Scholar]
  • 44. Renzi R, Finkbeiner S.. Ciprofloxacin interaction with sodium warfarin: a potentially dangerous side effect. Am J Emerg Med 1991; 96: 551–2. [DOI] [PubMed] [Google Scholar]
  • 45. Ahmed A, Stephens JC, Kaus CA, Fay WP.. Impact of preemptive warfarin dose reduction on anticoagulation after initiation of trimethoprim-sulfamethoxazole or levofloxacin. J Thromb Thrombolysis 2008; 261: 44–8. [DOI] [PubMed] [Google Scholar]
  • 46. Sittig DF, Belmont E, Singh H.. Improving the safety of health information technology requires shared responsibility: It is time we all step up. Healthc (Amst) 2018; 61: 7–12. [DOI] [PubMed] [Google Scholar]
  • 47. Ash JS, Sittig DF, McMullen CK, et al. Studying the vendor perspective on clinical decision support. AMIA Annu Symp Proc 2011; 2011: 80–7. [PMC free article] [PubMed] [Google Scholar]
  • 48. Paterno MD, Maviglia SM, Gorman PN, et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009; 161: 40–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 106: 523–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Kawamoto K, Houlihan CA, Balas EA, Lobach DF.. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 3307494: 765.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51. Classen DC, Phansalkar S, Bates DW.. Critical drug-drug interactions for use in electronic health records systems with computerized physician order entry: review of leading approaches. J Patient Saf 2011; 72: 61–5. [DOI] [PubMed] [Google Scholar]
  • 52. Harper MB, Longhurst CA, McGuire TL, Tarrago R, Desai BR, Patterson A.. Core drug-drug interaction alerts for inclusion in pediatric electronic health records with computerized prescriber order entry. J Patient Saf 2014; 101: 59–63. [DOI] [PubMed] [Google Scholar]
  • 53. Leung AA, Keohane C, Lipsitz S, et al. Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool. J Am Med Inform Assoc 2013; 20 (e1): e85–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC.. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood) 2010; 294: 655–63. [DOI] [PubMed] [Google Scholar]
  • 55. Chaparro JD, Classen DC, Danforth M, Stockwell DC, Longhurst CA.. National trends in safety performance of electronic health record systems in children's hospitals. J Am Med Inform Assoc 2017; 242: 268–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Aaron S, McEvoy DS, Ray S, Hickman T-TT, Wright A.. Cranky comments: detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inform Assoc 2019; 261: 37–43. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

ocz033_Supplementary_Data

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of Oxford University Press

RESOURCES