Abstract
Summary
Clinical decision support (CDS) can improve safety, quality, and cost-effectiveness of patient care, especially when implemented in computerized provider order entry (CPOE) applications. Medication-related decision support logic forms a large component of the CDS logic in any CPOE system. However, organizations wishing to implement CDS must either purchase the computable clinical content or develop it themselves. Content provided by vendors does not always meet local expectations. Most organizations lack the resources to customize the clinical content and the expertise to implement it effectively. In this paper, we describe the recommendations of a national expert panel on two basic medication-related CDS areas, specifically, drug-drug interaction (DDI) checking and duplicate therapy checking. The goals of this study were to define a starter set of medication-related alerts that healthcare organizations can implement in their clinical information systems. We also draw on the experiences of diverse institutions to highlight the realities of implementing medication decision support. These findings represent the experiences of institutions with a long history in the domain of medication decision support, and the hope is that this guidance may improve the feasibility and efficiency CDS adoption across healthcare settings.
Keywords: Clinical decision support, decision support systems, CDS, medication alerting, drug-drug interactions, therapeutic duplication, medical informatics
Introduction
Medication-related content forms a large proportion of the knowledge base of most clinical decision support (CDS) systems. When used in conjunction with a computerized provider order entry system (CPOE) system, medication-related CDS can prevent medication errors that may cause harm to patients and also improve quality and efficiency [1-3]. Further, the ubiquity of medications and the potential of lack of medication-related CDS to cause adverse events make this a crucial part of a clinical information system.
Despite these potential benefits, healthcare organizations have been slow to adopt and implement medication-related CDS. Barriers to adoption, identified by the 2004 Joint CDS Workgroup [4] include the time and labor for development and management of best-practice CDS knowledge, difficulties in achieving consensus about what to display in which clinical situation, and maintaining CDS as changes occur with medications and best practices. Most organizations lack the resources and expertise associated with the large and complex undertaking of developing a CDS knowledge base [5]. This is evident from the fact that development, implementation and subsequent evaluations of CDS systems have been largely limited to a handful of large academic medical centers [3, 6, 7]. Other organizations have elected to purchase their knowledge bases from commercial vendors. In a previous study, Kuperman, et al. identified low specificity of the alerts generated as a major limitation of commercial implementations of CDS. Kuperman, et al. also highlighted the difficulty of being able to adequately tailor the knowledge base [8] which might impede acceptance rate [9]. Further, Weingart, et al. found that over one-third of alerts generated at 5 academic primary care practices, lacked scientific evidence and were not clinically useful [10]. Thus, there is a need to identify a set of clinically significant medication-related decision support rules that can be implemented across healthcare settings.
The goal of this study was to collect both the content and recommendations from a panel of experts with experience in the development and implementation of medication-related decision support. We elected to focus on two types of basic medication decision support: drug-drug interaction (DDI) checking and therapeutic duplication. A DDI is defined as a modification in the effect of a drug when administered with another drug. The effect may be an increase or a decrease in the action of either substance, or it may be an adverse effect that is not normally associated with either drug [11]. The common mode of delivering a DDI is in the form of an alert, typically at order entry, where the knowledge base contains information that the effectiveness or toxicity of one drug is affected by the simultaneous presence of another drug in the body. One example is the co-administration of dextroamphetamine, a CNS stimulant with a monoamine oxidase inhibitor that can potentially cause a fatal hypertensive crisis. We focused on severe DDI rules that providers should be made aware of to prevent possible adverse drug events (ADEs). Duplicate therapy checking consists of algorithms that notify the user if two orders for the same drug or drugs from the same therapeutic class are concomitantly prescribed. For example, if a patient is currently taking captopril and the physician enters an order for enalapril, a therapeutic duplication alert would be fired since both drugs belong to the same therapeutic class of ACE inhibitors. In this area, we focused on drugs that should essentially never need to be administered concurrently.
Many CPOE systems contain knowledge bases with an assortment of alerts for a variety of CDS such as detecting interactions and duplications between drugs. A common way that these methods function is to generate a synchronous alert to a clinical user. Additionally, these knowledge bases vary in their extent of defining the severity of an alert. This is important since clinically insignificant or low severity alerts can cause “alert fatigue,” resulting in clinicians overriding even clinically significant alerts. Two independent investigations at separate institutions found override rates for high severity alerts of over 85% [10, 12, 13]. Additionally, Paterno, et al. found that tiering drug-drug interaction alerts based on the severity of the reaction, i.e. showing only high severity alerts was associated with significantly lower override rates [14]. In another study, van der Sijs, et al. explored whether turning off frequently overridden alerts could reduce alert overload and found that these alerts could not be turned off hospital-wide owing to differences among physicians regarding drug-related knowledge and monitoring practices [15]. Both these studies have highlighted our limited understanding of the domain of medication-related decision support and mechanisms to reach a balance between the effort needed to create a tailored set of decision-support rules and alert overload.
The Office of the National Coordinator for Health Information Technology (ONCHIT), in collaboration with the American Medical Informatics Association developed a roadmap for the implementation of computerized clinical decision support. The committee recommended the identification of key CDS rules and their reformulation to enable adoption in a variety of healthcare settings [16]. In a viewpoint paper, Miller, et al. described not just the need for a U.S. national standard for drug interaction information but also the desired characteristics of such a rule set to promote adoption. These characteristics included the need for a set of drug interaction rules described with generic drug names and in brief, human-readable format, with a computable set of descriptions for the clinical nature of the interactions [17]. Further, van der Sijs, et al. conducted a systematic review to assess the magnitude and reasons for high override rates. In this study, van der Sijs, et al. identified five important factors contributing to useful and appropriate alerts: three related to alert content- sensitivity, specificity, and information content; and two related to alert implementation- consideration of the workflow, and propagating safe and efficient handling of alerts. This study provides insights on the content for a currently implemented and clinically vetted rule set along with perspectives of an expert panel on successful implementation of these alerts [18].
While previous studies have employed similar approaches, several limitations exist in current approaches to identification of a clinically significant set of DDI rules. Wong, et al. evaluated the consistency of listings, severity, and scientific evidence ratings of DDIs involving oral anticancer agents and found the overlap to be only 30% between two commonly used DDI compendia [19]. Low overlap among compendia primarily exists because of the lack of standardization of ratings for consistently evaluating DDIs. Van Roon, et al. described the approach undertaken in The Netherlands by the Working Group on Pharmacotherapy and Drug Information for maintenance of DDI rules. This study described the criteria utilized in an evidence-based procedure for the uniform assessment of drug-drug interactions [20].
The focus of the literature in medication decision support, thus far, has been on the identification of what interventions are most meaningful to implement [5]. This paper goes one step further, explicitly describing a set of tailored alerts and therapeutic classes that have been vetted by a panel of CDS experts, for implementation in clinical information systems. Second, this panel of national experts from diverse organizations reflected on their experiences in implementing medication-related decision support which will provide guidance on how to implement medication decision support.
Methods
To begin to define some of the key issues related to medication-related decision support, we assembled a panel of national CDS experts to derive recommendations on the content and policies related to implementation of medication-related CDS. We focused on two domains: drug-drug interactions and duplicate therapy checking. The intent of this effort was, first, to develop both specific content for these two domains which could represent a starter set of knowledge and second, to describe the experiences of national experts with regard to implementation of medication decision support at their institutions.
Our panel of experts consisted of 7 individuals representing a diverse set of healthcare settings (public and private provider organizations, academic medical centers, and pharmaceutical and medical professional associations), a variety of roles (information technology, healthcare quality, healthcare policy, clinical practice, and research), and a variety of professional backgrounds (medical informaticists, pharmacists, quality experts, and physicians). We chose members in order to develop a panel that would be composed of members with diverse expertise in developing and implementing CDS and not necessarily in clinical medicine. Previous studies that have convened panels for development of CDS rule sets have utilized similar expertise, rather than limiting the discussions to experts in clinical medicine [16, 21]. Panelists were invited via email and received no financial incentives to participate, except travel compensation to attend the meeting.
We used a two-phase approach to gather information from panel members, first an online discussion followed by an in-person meeting. In the first phase, we utilized an online discussion mechanism, using the eRoom software (EMC Corporation, Hopkinton, MA, USA). The discussion was facilitated by two authors (AW, SP) and participants were asked to comment on content related to drug-drug interactions and therapeutic duplication alerts. Second, panelists described how these areas of medication decision support had been implemented at their own institutions. The online discussion took place for a period of approximately 4 months from April to July, 2008.
To begin the discussions, we used the list of drug-drug interactions and therapeutic duplication classes that have been employed at Partners Healthcare, which were tiered into three categories. Our medication knowledge base consists of rules for DDIs and therapeutic duplications that are developed by an in-house team of pharmacists. The pharmacists gather the evidence surrounding any new DDIs or therapeutic duplications and make an assessment, based on the clinical outcome, to determine what level of severity such an interaction would pose. These assessments are then vetted with a group of subject matter experts that consists of practicing clinicians and they determine the final level of an alert. Level 1 consists of the most severe interactions and Level 3 interactions are the least severe. Two criteria drove the choice of the content. First, we wanted to have an initial set of rules that had been validated in the clinical environment. Since these had previously been evaluated by pharmacists and clinicians we did not undertake an evidence-based assessment as part of the research described here. We used a limited set of rules to start the discussions with the panel and vet the content for clinical significance rather than asking the panel to identify each rule that needed to be included in this starter set. The initial set of DDIs consisted of thirty rules- ten Level 1, ten Level 2, and ten level 3 DDI rules. The initial set of therapeutic duplication rules consisted of 15 drug classes and 2 drug class pairs.
Both DDI and therapeutic duplication interactions have created problems in many medication-related CDS implementations with respect to generating excessive alerts leading to alert fatigue [10, 22]. In some implementations, both types of medication-related decision support have even been turned off. Our intent was to identify especially important interactions meriting display.
During the in-person meeting the content and recommendations from the online discussion were revisited and extensively discussed to identify a starter set that could be generalized across healthcare settings. The proceedings of this meeting were audio taped and later transcribed. The clinical content vetted by the expert panel and their discussions are presented here.
Results
The rules defined here form a starter set that the expert panel considered relevant for widespread dissemination of medication decision support in the areas of drug-drug interaction and therapeutic duplication alerts. The panel did not make considerable changes to the suggested set of DDIs and therapeutic duplication rules from the Partners knowledge base; the modifications that were made are detailed below.
From the initial set of DDIs, discussed by the panel, two DDIs related to interactions between anticoagulants were removed and instead added to the list of therapeutic duplications. These consisted of, one Level 1 DDI, between ‘argatroban’ and ‘enoxaparin’ and one Level 3 DDI, between ‘enoxaparin’ and ‘heparin sodium’. The final list consists of a total of 27 interactions: nine Level 1, ten Level 2 and eight Level 3 DDIs. The final set of DDIs and the levels to which they were assigned are presented in ►Table 1.
Table 1.
Medication classes identified by the expert panel for generating drug-drug interaction alerts
|
Level 1 Alerts Hard stop alerts which physician cannot override. |
|
|
Level 2 Alerts Interruptive but the clinician can continue by providing a reason for ordering two interacting drugs |
|
|
Level 3 Alerts Informational and non-interruptive. |
|
The panel identified 16 drug classes for therapeutic duplication checking. In addition, three pairs of drug classes were identified, for which the presence of a drug from each class would trigger an alert. The findings outlined in ►Table 2 represent the final recommendations of the panelists for a starter set of medication decision support rules for therapeutic duplication and DDI checking.
Table 2.
Medication classes identified by the expert panel for generating therapeutic duplication alerts. If a patient has two drugs in any single class an alert is generated. Alerts are also generated if drugs from each of the medication class pairs are concurrently prescribed
|
| Pairs of therapeutic classes |
|
Customization of CDS for the inpatient and outpatient settings
All institutions represented by panel members had some degree of DDI checking available in their CPOE systems, though the discussion revealed a number of differences in the implementations. Sites varied in the level of DDI rules that were being actively used to generate CDS in their CPOE systems. Panelists also discussed whether institutions made a distinction in the rules that were used in the inpatient versus outpatient setting. One institution reported using a filtered list of rules in the inpatient setting which were locally customized from the commercial drug knowledge base. In the outpatient setting, the same institution utilized all the rules from the same vendor’s drug knowledge base. A number of panel members felt that there was more tolerance for receiving low severity alerts in the outpatient setting than in the inpatient setting, although there was no consensus on this point, and some members had strong feelings in the opposite direction. Other institutions used a constrained list of locally customized DDI checking rules in both the inpatient and outpatient settings.
Institutions varied in their strategies for identifying drug pairs selected for alerts in the inpatient and outpatient settings. Panel members agreed that the set of drugs that were known to have high severity interactions with a high degree of confidence were commonly used in both settings. This set included medications that should never be co-prescribed. Many of the variations occurred when a set of drugs with known interactions could possibly be co-prescribed with careful monitoring. The panel suggested that these types of interactions could be excluded from the data set used for alerting in the inpatient setting since patients could more readily be monitored for possible adverse events during their hospitalization. While this is true in most inpatient scenarios, the availability of monitoring may not always imply that it is utilized and patients in the hospital may be just as vulnerable to adverse events as those in the outpatient setting.
Strategies to reduce alert fatigue
The panel discussed mechanisms to reduce alert fatigue caused by DDI and therapeutic duplication alerts. All panelists recognized excessive alerts as an important issue that impeded the effectiveness of medication-related alerts. Some panel members recommended distinguishing between alerts that needed to be seen by the physician versus those that could be directed to the pharmacist, though this was controversial. One institution’s practice was to deliver all possible alerts to the pharmacists who could then determine which alerts needed to be escalated to the physician. Panel members disagreed on whether or not this represents an effective strategy since pharmacists “were not any more immune to alert fatigue than physicians,” and when inundated with alerts, pharmacists may fail to direct clinically significant alerts in need of an intervention to the attention of the physician. All panelists were in agreement that no separate set of DDIs should be created for nurses who should be able to see all the DDIs that the physician received. In addition, our suggestion is to only show the alerts related to drug administration to the nurses and not present these to the physician who may not be aware of the administration regimen. At our institution, pharmacists set up the administration regimen after approving the drug order. DDI alerts that are related to the timing or sequence of administration of the co-prescribed drugs should be shown to the pharmacist or nurse so they are able to act on them. This was also suggested by van der Sijs and colleagues in their review on the magnitude and reasons for high overrides of drug safety alerts [18].
Panelists agreed that stratification of DDI alerts was important since it could decrease the likelihood that clinicians would ignore the high severity alerts. There was also agreement that three levels of stratification – high, moderate, and mild severity should be sufficient. However, institutions used a variety of methods to further stratify alerts. One institution described a rating mechanism that allowed the clinician to see the level of evidence supporting an alert. No other institution had a mechanism for generating an evidence-based rating for further stratification of alerts. Institutions also varied in the latitude provided to physicians in overriding a high severity alert. Some institutions allowed even the highest severity alerts to be overridden by clinicians, provided a reason was documented. In this approach, when one of these alerts was identified, the clinician’s workflow was interrupted, but the alert could be overridden. As a second line of defense, some institutions required that overridden alerts be reviewed by a pharmacist, at least in the inpatient setting, before the interacting drugs were allowed to be dispensed. Other institutions implemented more stringent regulations with “hard stops” for the highest severity alerts, which could not be overridden. This approach is more certain to result in prevention of any of the interacting drugs being prescribed together but can generate resistance among providers if the combination is ever deemed appropriate.
Local customization of commercial drug knowledge bases
Institutions differed in their ability to manipulate the drug knowledge database driving the alerts. Institutions with home-grown drug knowledge databases had maximum flexibility, but there was variability in the ability to customize alerts among institutions using CDS from commercial vendors. Of particular significance, institutions using a commercial vendor lacked the ability to make changes at the drug class level. For example, if an institution chose to activate the interaction between beta-blockers and beta-2-agonists (an interaction resulting in decreased effectiveness of either class of drugs) it would be most efficient to do this at the drug class level instead of manually manipulating each drug pair. While it may be obvious that an institution would desire the ability to customize their drug knowledge database at the drug class level or at the individual pair level, two institutions related their experience with commercial drug knowledge databases which allowed modifications to be made only at the level of individual drug pairs. The limitation was identified, not as the lack of an adequate drug hierarchy or the presence of drug classes in the vendor’s medication terminology, but one related to the contractual agreements between specific implementation of the drug knowledge database in the vendor EMR solutions employed at these institutions.
Panelists noted that while drug classes are an effective mechanism for tailoring, they require careful clinical consideration. Many pharmacological agents belonging to the same drug class display similar therapeutic effects, a principle termed “class effect”. However, many exceptions to this rule exist, such as omeprazole, (belonging to a class of drugs called proton pump inhibitors (PPIs) and curbs gastric acid secretion) which interacts with the antiplatelet agent clopidogrel, while other PPIs do not. In this case, making an assumption that an alert needs to be fired every time a PPI is prescribed in combination with an anti-platelet agent would result in alerts lacking clinical significance.
Inconsistencies between drug knowledge CDS with regards to content pertaining to the drug alerts and the level of severity assigned to these alerts was recognized as a problem across institutions. Lack of agreement on the set of highest severity alerts is an issue not just when medication-related decision support systems are implemented but also when an institution decides to migrate from one drug knowledge CDS to another. The lack of homogeneity in the drug alerting content makes it impossible to maintain customized alerts across migrations. Further, panelists reported that there was lack of agreement among clinicians at their institutions about which alerts were meaningful and the level of severity they should be assigned. This further emphasizes the need for a set of evidence-based, well-tested drug alerts that should be included across institutions.
Panelists highlighted the difficulty experienced in reaching agreement on either development of initial content or customizing commercial content of the initial set of medication-related CDS at their institutions. The clinical resources needed to vet the medication content were enormous, especially if several multi-disciplinary groups and clinical specialties were to participate and agree upon a limited set of drug alerts. The burden of discussing each drug pair that needs to be included in a customized DDI alerting engine is a formidable undertaking for an institution of any size. Future work should focus on validating the most severe alerts that have the potential to cause the most harm if missed by a clinician.
Lack of therapeutic duplication checking across institutions
While the expert panel represented a diverse set of institutions, most did not have therapeutic duplication checking in their CPOE systems. Panelists identified barriers to implementing therapeutic duplication as “alert fatigue” and the perceived overwhelming amount of work required in customizing the vendor knowledge base. Most notable was the discussion on the modification of the list of classes for therapeutic duplication checking. This resulted in the addition of the medication class of anticoagulants and a new drug pair of H2 Blockers and Proton Pump Inhibitors (PPIs). The panel recommended that alerts should be made available for the therapeutic class of anticoagulants considering the severity of consequences in case of inadvertent duplication. Panelists recognized the criticality of a therapeutic duplication resulting from the concomitant use of fractionated or low molecular weight and unfractionated heparin products. Practitioners often switch between these two types of heparin and a provider must be alerted about their inadvertent concomitant use. The ISMP Medication Errors Reporting Program (MERP) has reported at least 3 deaths related to inadvertent duplication of these agents. (23) ISMP has also issued guidelines to prevent the concomitant use of heparin products. A large number of these errors occurred due to inadequate medication reconciliation between transitions of care. This led to a discussion on risks at transitions of care, and panelists recommended that electronic access to a patient’s medication history both from the inpatient and outpatient setting is vital for determining all medications taken by the patient to have a better idea of medications such as opioids that patients were taking. An additional pair of therapeutic classes, H2 blockers and PPIs, was added to the list of therapeutic duplication alerts. Both H2 blockers and PPIs are used to suppress gastric acid secretion, and the use of both in conjunction does not show any added therapeutic benefit.
Discussion
In this study, the expert panel was tasked with developing a set of clinically significant DDIs and therapeutic duplication warnings that could be widely implemented across systems. Additionally, the panel identified implementation issues, related to the creation and maintenance of medication-related decision support. Although all institutions represented had implemented DDI rules, most did not currently include therapeutic duplication checks in their systems due to the excessive occurrence of false positive alerts.
Medical knowledge creation for CDS and sharing represents an arduous task. Previous experiences in medical informatics have informed us of the obstacles encountered in sharing knowledge [24, 25]. The problem of knowledge implementation can be overcome by purchasing CDS software from a commercial vendor. These products contain CDS that may include knowledge on DDIs, minimum and maximum dose alerts, and drug-allergy cross-sensitivity checking [8]. Institutions may choose to incorporate commercial CDS software in their clinical information systems or customize CDS in their “homegrown” clinical information systems. However, the task does not end there. Implementing these “out of the box” CDS solutions can generate an excessive number of alerts [8]. The large frequency of alerts can be attributed to the fact that the CDS driving these alerts have not been sufficiently pruned for clinically insignificant interactions, resulting in overly sensitive drug-allergy interaction checking [26], DDI checking [10, 27], and dose limit checks [27]. Many commercial CDS vendors employ the philosophy of including broad pharmacological classes to trigger alerts which can result in a large volume of low severity alerts, many of which may not be supported by substantial evidence. In order to build an “intelligent” DDI rule base, drug classes should be further modified to identify exceptions in drug members that do not conform to the “class effect”. Additional consideration must be given to identifying drugs that have different pharmacological effects depending on the route of administration. For example, vancomycin when administered orally to treat pseudomembranous colitis, an infection of the colon, is not absorbed into the bloodstream, thus decreasing the likelihood of producing a significant interaction with another drug. However, intravenous administration is needed for the treatment of serious, life-threatening blood and tissue infections, thus putting a patient at risk for a DDI. For therapeutic duplication, consideration of the route of administration is important as well since clinicians do not want to be alerted when topical and systemic forms of the same drug are co-administered. Experts were divided regarding whether or not to include opioids in the list of therapeutic duplication classes. This is a complex issue for a variety of reasons. Multiple opioids may be given for severe chronic pain, often by multiple routes of administration. For example, patients receive transdermal opioid patches supplemented by oral opioids for breakthrough pain. Some panelists felt that alerting a provider that a patient was receiving opioids would be clinically significant in preventing an accidental overdose resulting in potentially serious consequences of respiratory depression or respiratory arrest. However, codified documentation of previous opioid exposure is often lacking in clinical information systems, thus preventing a rule from considering this vital piece of information. Other members of the panel argued that there are numerous situations in which a patient appropriately receives multiple opioids which are appropriate, which would result in overwhelming clinicians with alerts and may not be clinically significant. There was consensus around better use of pain scales and bar coding for medication administration as part of an approach for preventing opiate overdoses.
Customization of CDS is an expensive and resource-intensive process. Various strategies for customization have been employed for tailoring commercial rule bases [9, 28]. Making the list of clinically significant alerts available in an unambiguous form is a first step in this direction. However, actual implementation of such a list will require some level of customization in order to map medication concepts to locally employed medication knowledge bases. Further, such medication concepts would have to be mapped at the correct level of granularity, by taking into consideration formulation and route characteristics, to ensure that specificity of the rule is retained. Also, generalizability of such a list is limited since we did not consider agents used in other countries and have erred on the side of being inclusive. Drugs approved in Europe are sometimes made available in the U.S. as formulary exceptions. The starter set was created as one that is in use at hospitals within our healthcare delivery network therefore it contains medications from our common formulary. However, in order to be implemented, customization would need to take into account representation of locally available agents in the medication knowledge base.
A defined set of medication CDS rules vetted by experts from organizations with experience in medication CDS will make the task of implementing CDS less daunting. We also hope that this defined set of rules will lead to fewer alerts thereby reducing alert fatigue. However, the clinical validity of these rules remains to be tested to assess the sensitivity and specificity of this set of alerts and their ability to prevent dangerous adverse drug events. We identified a broad panel of experts who had previous experience in implementing medication-related CDS at their institutions and vetted a set of DDI alerts and therapeutic classes for duplicate therapy checking. In keeping with Miller’s criteria [17], we have described the drugs as generic names so that institutions have the freedom to incorporate this knowledge depending on the specifications of the formulary used and the medication knowledge base implemented at their institutions.
In summary, the panel identified a broad range of issues that need to be considered for the successful implementation of medication-related CDS, in particular for DDIs and therapeutic duplications, although many of the issues identified apply to other domains as well. A deeper understanding of these issues will enable organizations to more thoroughly examine the medication-related CDS solutions available to them. The expert panel vetted a starter set of medication-related decision support that can be implemented across settings and electronic medical records. This starter set drew upon the valuable experiences of an array of institutions with an exemplary history of CDS development and implementation. The study also highlighted the differences among these institutions and their struggles to achieve the fine balance between over alerting and patient safety. This study attempted to uncover these challenges and the variety of ways in which institutions overcame them.
The starter set would benefit from further validation in other clinical environments and the authors hope that any lessons learned as a result of exercising these recommendations should be reported in the public domain. Equally important to assessing the clinical validity is the implementation of appropriate documentation indicating the reason for overriding a DDI alert. By assessing specific clinical scenarios where it may not be beneficial to present some of these DDI alerts, we can move towards reducing alert fatigue. Further, documentation of the reason for overriding a DDI also becomes critical from the stand point of understanding whether the clinician made an informed judgment to co-prescribe the two potentially harmful medications. This is crucial for accurate measurement of quality measures, such as the National Quality Forum’s Measure 022, that assess the number of patients who are receiving potentially inappropriate medications.
This study has several limitations. We selected leading experts who represented different types of institutions with strong expertise in clinical decision support implementation. However, the findings discussed here are limited to the discussions of the panel and their experiences. Further, we used the set of medication-related rules that are employed at Partners Healthcare to start these discussions. While panel members vetted these alerts based on experience and clinical expertise and had the option to add or delete alerts, the initial alert set had a large influence on the final list of alerts that were accepted by the panel. Future research should focus on actual comparisons of the content representing the most clinically significant alerts implemented across systems and validation of the alert set proposed here.
Conflict of Interest Statement
None of the authors has a conflict of interest with respect to the content of this manuscript.
Human Subjects Protection
This project was reviewed and approved by the Partners HealthCare Institutional Review Board.
Acknowledgements
This study was sponsored by the Center for Education and Research on Therapeutics on Health Information Technology (CERT-HIT) grant [PI: David W. Bates], Grant # U18HS016970 from the Agency for Healthcare Research and Quality (AHRQ). AHRQ was not involved in the design or execution of the project or the decision to publish the results. The authors are grateful for the assistance of Christine Soran and Francine Maloney for transcribing the meeting recordings, and Marianne Zachariah who helped with the preparation of the manuscript and improving its readability through editorial revisions.
References
- 1.Bates DW, Boyle DL, Teich JM. Impact of computerized physician order entry on physician time. Proc Annu Symp Comput Appl Med Care 1994: 996 PMid:7950101 PMCid:2247898 [PMC free article] [PubMed] [Google Scholar]
- 2.Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. Jama 1998; 280(15): 1311-1316 PMid:10428004 PMCid:61372 [DOI] [PubMed] [Google Scholar]
- 3.Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’Luf N, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 6(4): 313-321 PMid:15802474 PMCid:1174880 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Teich JM, Osheroff JA, Pifer EA, Sittig DF, Jenders RA. Clinical decision support in electronic prescribing: recommendations and an action plan: report of the joint clinical decision support workgroup. J Am Med Inform Assoc 2005; 12(4): 365-376 doi:10.1197/jamia.M1822 PMid:17068355 PMCid:2215064 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14(1): 29-40 doi:10.1197/jamia.M2170 PMid:11025783 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr CD, Bates DW. Effects of computerized physician order entry on prescribing practices. Arch Intern Med 2000; 160(18): 2741-2747 doi:10.1001/archinte.160.18.2741 PMid:9292842 PMCid:61254 [DOI] [PubMed] [Google Scholar]
- 7.Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc 1997; 4(5): 364-375 PMid:16622160 PMCid:1513681 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kuperman GJ, Reichley RM, Bailey TC. Using commercial knowledge bases for clinical decision support: opportunities, hurdles, and recommendations. J Am Med Inform Assoc 2006; 13(4): 369-371 doi:10.1197/jamia.M2055 PMid:16221941 PMCid:1380196 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Shah NR, Seger AC, Seger DL, Fiskio JM, Kuperman GJ, Blumenfeld B, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006; 13(1): 5-11 doi:10.1197/jamia.M1868 PMid:14638563 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163(21): 2625-2631 doi:10.1001/archinte.163.21.2625 PMid:12463894 PMCid:2244252 [DOI] [PubMed] [Google Scholar]
- 11.Drug interaction. (n.d.) Mosby’s Dental Dictionary. Retrieved October 23, from Answers.com Web site: <http://www.answers.com/topic/drug-interaction>
- 12.Payne TH, Nichol WP, Hoey P, Savarino J. Characteristics and override rates of order checks in a practitioner order entry system. Proc AMIA Symp 2002: 602-606 PMid:18579840 PMCid:2528033 [PMC free article] [PubMed] [Google Scholar]
- 13.Lin CP, Payne TH, Nichol WP, Hoey PJ, Anderson CL, Gennari JH. Evaluating clinical decision support systems: monitoring CPOE order check override rates in the Department of Veterans Affairs’ Computerized Patient Record System. J Am Med Inform Assoc 2008; 15(5): 620-626 doi:10.1197/jamia.M2453 PMid:18952941 PMCid:2605599 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Paterno MD, Maviglia SM, Gorman PN, Seger DL, Yoshida E, Seger AC, et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009; 16(1): 40-46 doi:10.1197/jamia.M2808 PMid:18436915 PMCid:2442265 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.van der Sijs H, Aarts J, van Gelder T, Berg M, Vulto A. Turning off frequently overridden drug alerts: limited opportunities for doing it safely. J Am Med Inform Assoc 2008; 15(4): 439-448 doi:10.1197/jamia.M2311 PMid:17213487 PMCid:2213467 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc 2007; 14(2): 141-145 doi:10.1197/jamia.M2334 PMid:15905481 PMCid:1174885 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Miller RA, Gardner RM, Johnson KB, Hripcsak G. Clinical decision support and electronic prescribing systems: a time for responsible thought and action. J Am Med Inform Assoc 2005; 12(4): 403-409 doi:10.1197/jamia.M1830 PMid:19033481 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of Drug Safety Alerts in Computerized Physician Order Entry. J Am Med Inform Assoc 2006; 13(2): 138-147 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Wong CM, Ko Y, Chan A. Clinically significant drug-drug interactions between oral anticancer agents and nonanticancer agents: profiling and comparison of two drug compendia. Ann Pharmacother 2008; 42(12): 1737-1748 doi:10.1345/aph.1L255 PMid:16329715 [DOI] [PubMed] [Google Scholar]
- 20.van Roon EN, Flikweert S, le Comte M, Langendijk PN, Kwee-Zuiderwijk WJ, Smits P, et al. Clinical relevance of drug-drug interactions: a structured assessment procedure. Drug Saf 2005; 28(12): 1131-1139 doi:10.2165/00002018-200528120-00007 PMid:19745260 [DOI] [PubMed] [Google Scholar]
- 21.Middleton B. The clinical decision support consortium. Stud Health Technol Inform 2009; 150: 26-30 PMid:19786683 [PubMed] [Google Scholar]
- 22.Weingart SN, Simchowitz B, Shiman L, Brouillard D, Cyrulik A, Davis RB, et al. Clinicians’ assessments of electronic medication safety alerts in ambulatory care. Arch Intern Med 2009; 169(17): 1627-1632 doi:10.1001/archinternmed.2009.300 PMid:9609492 PMCid:61296 [DOI] [PubMed] [Google Scholar]
- 23.Hazard Alert! Action needed to avert fatal errors from concomitant use of heparin products. Institute for Safe Medication Practices (ISMP) Medication Safety Alert [serial on the Internet]. February 21, 2001; 6(4). [Google Scholar]
- 24.Zielstorff RD. Online practice guidelines: issues, obstacles, and future prospects. J Am Med Inform Assoc 1998; 5(3): 227-236 PMid:8130503 PMCid:2248539 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Pryor TA, Hripcsak G. Sharing MLM’s: an experiment between Columbia-Presbyterian and LDS Hospital. Proc Annu Symp Comput Appl Med Care 1993: 399-403 PMid:15298998 PMCid:524628 [PMC free article] [PubMed] [Google Scholar]
- 26.Hsieh TC, Kuperman GJ, Jaggi T, Hojnowski-Diaz P, Fiskio J, Williams DH, et al. Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004; 11(6): 482-491 doi:10.1197/jamia.M1556 PMid:15802481 PMCid:1174882 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Reichley RM, Seaton TL, Resetar E, Micek ST, Scott KL, Fraser VJ, et al. Implementing a commercial rule base as a medication order safety net. J Am Med Inform Assoc 2005; 12(4): 383-389 doi:10.1197/jamia.M1783 PMid:16779381 PMCid:1560631 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Resetar E, Reichley RM, Noirot LA, Dunagan WC, Bailey TC. Customizing a commercial rule base for detecting drug-drug interactions. AMIA Annu Symp Proc 2005: 1094 PMid:9794308 [PMC free article] [PubMed] [Google Scholar]