Abstract
Background: Among the many clinical decision support (CDS) mechanisms available in electronic health record (EHR) systems, dose range checking (DRC) is one of the most impactful safeguard tools integrated within most computerized provider order entry (CPOE) workflows. Unfortunately, improper configurations and lack of resources to maintain and monitor CDS systems can hinder and even disrupt daily clinical operations. Objective: This article seeks to highlight the impact that informatics pharmacists can make by implementing different strategies to decrease nuisance alerts and create clinically meaningful DRC alerts that guide clinicians in their practice. Methods: Following the activation of the DRC application for 3623 medication groupers (ie, generic drugs and all their dosage form variations), informatics pharmacists implemented strategies to monitor DRC alert output and decrease the number of inappropriate alerts. Such strategies included weekly monitoring of alerts, modification of order sentences (including dose, route, and age/weight filters), update to the rule triggering the alerts, and modifications of the preference settings. Results: From July to September 2018, an average of 70 DRC tables were reviewed by informatics pharmacists, reducing the number of overridden DRC alerts to 4796 in the first week of September—a 63% decrease in a 3-month period. Conclusions: By reducing the number of DRC nuisance alerts and improving the clinical content of DRC alerts, informatics pharmacists can contribute to lowering alert fatigue and improving providers’ trust in CDS alerts.
Keywords: dose range checking, clinical decision support, electronic health record system, informatics, alert fatigue
Introduction
Since the early 1960s, health care has been undergoing a digital transformation. 1 This dynamic and continuously evolving process seeks to leverage current and new technological modalities to create safer and more efficient patient-centered workflows. Health care is now mediated through many computer interfaces. 2 Within electronic health record (EHR) systems, health care providers rely on clinical decision support (CDS) mechanisms to safely and efficiently order, process, and administer medications. 3 As defined in the CDS guidebook titled, Improving Medication Use and Outcome With CDS: A Step-by-Step Guide (HIMSS, 2009), a successful CDS must deliver the right information, to the right people, in the right format, through the right channel, at the right point in workflow. 4 Whereas duplicates, drug-drug interactions, and drug-allergy interactions are some of the more commonly recognized CDS alerts, dose range checking (DRC) is one of the most impactful safeguard tools integrated within most computerized provider order entry (CPOE) workflows. 5 Unfortunately, CDS systems do not always improve clinical practice. 6 Improper configurations and lack of resources to maintain and monitor CDS systems can hinder and even disrupt daily clinical operations. The prevalence of high override rates has been attributed to an abundance of clinical alerts leading to greater alert fatigue. 7 Providers experiencing alert fatigue are at greater risk of ignoring potentially crucial notifications or warnings, which can lead to a delay in patient care or even harm to patients. 8 Between January 2005 and June 2010, the US Food and Drug Administration reported 566 deaths related to alert fatigue. The Joint Commission’s (TJC) database also reported 80 alarm-related sentinel events between January 2009 and June 2012. 9 At the time our strategies were implemented, Cerner did not provide any best practice recommendations on how to optimize DRC alerts. This article seeks to highlight the impact that informatics pharmacists can make by implementing different strategies to decrease nuisance alerts and create clinically meaningful DRC alerts that guide clinicians in their practice. The methods and results shared come from actual implementations performed by informatics pharmacists, in collaboration with a team of clinical pharmacists, at Baptist Health South Florida (BHSF).
Methods
Preliminary Clinical Content Review
Baptist Health South Florida, a not-for-profit health care organization comprising 11 hospital facilities and more than 100 outpatient and urgent care centers, utilizes Cerner Millennium as its main EHR system for clinical operations. In July 2018, BHSF created a Pharmacy Informatics department, consisting of 10 informatics pharmacists and 5 clinical business technology consultants, to oversee all aspects of the technologies involved in the medication use process.
In June 2018, following the recommendation of the Leapfrog Hospital Safety Grade, 3623 medication groupers (ie, generic drugs and all their dosage form variations) were activated to fire DRC alerts to end users, including physicians, nurses, and pharmacists. Prior to this activation, only 231 medication groupers were active. However, these groupers were of minimal impact as the active medications were not commonly prescribed in our system. Out of the 3623 activated medication groupers, 623 groupers were selected from a report generated 1 month after the activation of the DRC alerts. The groupers selected represented the top medications triggering DRC alerts. The selected medication groupers were then reviewed by a team of clinical pharmacists responsible for assessing the clinical appropriateness and relevance of the standard content provided by Cerner Multum. Each medication grouper reviewed was assessed independently by 21 pharmacists, with the DRC content organized systematically in a table created in Microsoft Excel. Each DRC table consisted of the medication grouper’s name at the top, followed by the route of administration (eg, oral or intravenous). Each line item in the table required the pharmacist to identify, for the specified age range and renal function, if applicable, appropriate minimum and maximum dose limits for single doses and daily doses. Pharmacists consulted a multitude of resources, including internal policies and guidelines, package inserts, online drug databases such as UpToDate, Micromedex, and NeoFax, and peer-reviewed literature, to determine the DRC values. Although off-label uses were not initially included in the DRC tables, if a recurring prescribing pattern was later identified through DRC alert auditing, the evidence-based off-label dosing would then be included in the table. The process of updating the DRC tables was, and still is, a continuous and dynamic exercise that evolves with the addition of new uses and protocols.
Implementation and Monitoring
Each completed DRC table was then translated into Cerner Millennium’s DRC tool over a period of several months. Each week, the team of clinical pharmacists in charge of performing the clinical review of the DRC tables would submit their updated tables to a team of pharmacists responsible for inputting the data into the DRC tool. A variance of 10% was added to weight-based limits for medications ordered with weight-based dosing to account for variations due to rounding for both adult and pediatric patient populations. This variance was selected based on prior clinical experience, similar variance used in intravenous container overfill, and published literature. 10 Within our health care system, pharmacists are able to round most doses within a 10% limit. The successful implementation of the DRC tool in Cerner Millennium starts with the understanding that the module’s logic is as drug-driven as it is patient-centered. The DRC tables in Cerner not only account for drug-specific information, such as route, drug formulation, and single/daily dose ranges, but they also account for patient-specific data such as age and renal function.
Following the activation of the DRC application for the 3623 medication groupers, informatics pharmacists developed strategies to monitor the DRC alert output and decrease the number of inappropriate alerts. The focus was not only on reducing the total number of alerts triggered but also on improving the quality of the alerts and providing clinically meaningful information. Weekly audits were conducted to identify the alerts firing and the medications triggering the most alerts. Pharmacists within the health system were encouraged to report any DRC alerts considered to be clinically inappropriate as a means to create a dynamic and transparent feedback loop. Immediate attention was given to the medications triggering the greatest number of overridden alerts. Over time, 3 critical interventions were identified to decrease the total number of DRC alerts while improving the quality of the information displayed. It is important to note that these strategies and DRC modifications affected both inpatient and outpatient dose checking within our health system.
Order sentences
The first and most direct opportunity for improvement was determined to be at the order sentence level. Order sentences provide a quick and easy method for selecting multiple order details, such as dose, route, and frequency, for a specific medication all at once with a single mouse click. Order sentences are provided with Cerner Multum content and are further modified or customized by informatics pharmacists based on system protocols. For any medication identified in the top 10 for firing DRC alerts, all available order sentences inside and outside of Power Plans (ie, order sets), both inpatient and outpatient, were reviewed. Modifications included updating the dose, route, and age/weight filters. Alternatively, DRC content was left unchanged if an order sentence was considered to be clinically appropriate based on internal protocols or supporting clinical evidence.
Rule
The DRC alerts generated within the Cerner Millennium interface are driven by the DRC tool in tandem with a rule written in the Discern Expert application. It was identified that the rule was written in such a way that it triggered a DRC alert regardless of order status. Thus, discontinued orders also generated DRC alerts. Such an alert is disruptive as the dose on a discontinued order is no longer clinically relevant. A logic statement was added to the rule to suppress the rule from firing on discontinued orders (Figure 1).
Figure 1.
Discern rule logic statement.
Preference settings
DRC error messages were counted among overridden alerts in our weekly audits as they represent nuisance alerts because the displayed information is neither meaningful nor useful in guiding prescribers in their practice. Furthermore, these alerts fired even if the doses were clinically appropriate. The DRC error messages suppressed specifically relate to technical errors that prevent the DRC tool from performing its function. The general feedback received from providers was that being alerted that the system cannot perform an action does not lead to safer prescribing practices. The types of DRC error messages displayed to end users can be modified at the preference setting level. There are a total of 13 different error messages (Table 1). Each one can be suppressed by changing the default setting from 0 to 1. Five key error messages were identified as not providing enough meaningful information to providers and leading to high override rates. Unfortunately, these preference settings are global and cannot be applied to specific medications or groupers. The first DRC error message suppressed was option 1: “The supplied orderable belongs to multiple groupers.” Medication groupers are assigned by Cerner and are updated through periodic Multum content updates that need to be installed onto the client-specific EHRs. This error message advises end users that the medication ordered is mapped to multiple groupers and therefore Cerner is unable to perform the DRC algorithm as the system cannot determine which grouper to run the DRC rule against. However, as an end user is unable to determine and assign the appropriate grouper, the alert is not useful. Following the suppression of these error messages, our team periodically ran audits to identify any drugs assigned to multiple groupers and remap them to different Cerner Knowledge Index (CKI) values. The second error message suppressed was option 2: “Order dose was missing or invalid.” This error occurs when prescribers enter a dose unit that is inappropriate for the medication (eg, using “mL” for a tablet). Pharmacists verifying these orders also receive a dose incompatibility alert independent of the DRC error message, thereby providing an additional layer of protection. The third error message suppressed was option 4: “The patient’s creatinine clearance was missing or invalid.” This message displays whenever a DRC table contains a row with a different dose range based on creatinine clearance. In some instances, such as onetime orders or on call orders, creatinine clearance may not necessitate renal dose adjustments and therefore the creatinine clearance may not be clinically significant. As this specific alert fires when a creatinine clearance value is missing, regardless of the dose, it fails to provide enough information for the prescriber to determine whether the ordered dose is appropriate or not. In addition, the verifying pharmacist is also expected to independently review renal function at the time of order verification to assess for any possible renal dose adjustments. The fourth error message suppressed was option 5: “The patient’s weight was missing or invalid.” Our system has several mechanisms in place to ensure that patients have documented measured weights prior to medication order entry. Therefore, we decided to suppress this error message to avoid redundancy. However, we recommend leaving this option active in facilities that do not have rules put in place to ensure patients are properly weighed before allowing medication orders to be placed. Finally, option 13, “Unable to convert units,” was suppressed. This error message becomes especially problematic for prescription orders where the dose unit of measure may differ from that used in the inpatient setting. For example, an antibiotic in liquid form ordered in milligrams is logical in the inpatient setting but may be better displayed to a patient in milliliters. Suppressing this error message allows for greater flexibility in the units of measure used for outpatient prescriptions.
Table 1.
DRC Error Message Preference Settings.
| DRC error message | Preference settings |
|---|---|
| First: The supplied orderable belongs to multiple groupers | 1 (suppressed) |
| Second: Order dose was missing or invalid | 1 (suppressed) |
| Third: The continuous dose for this medication cannot be calculated. Rate is zero | 0 |
| Fourth: The patient’s creatinine clearance was missing or invalid | 1 (suppressed) |
| Fifth: The patient’s weight was missing or invalid | 1 (suppressed) |
| Sixth: The patient’s post menstrual age was missing or invalid | 0 |
| Seventh: The diagnosis was missing or invalid | 0 |
| Eighth: The patient’s body surface area was missing or invalid | 0 |
| Ninth: The continuous dose range for this medication cannot be evaluated as this order has a free text rate | 0 |
| Tenth: A strength dose was needed to evaluate the rule, but only a volume dose was provided | 0 |
| Eleventh: The continuous dose range for this medication cannot be evaluated as this medication is not included in every bag | 0 |
| Twelfth: DRC unit not recognized | 0 |
| Thirteenth: Unable to convert units | 1 (suppressed) |
Abbreviation: DRC, dose range checking.
Results
In June 2018, an average of 11 452 DRC alerts were overridden by prescribers per week. The recorded peak occurred between June 25 and July 1, 2018, with 13 183 overridden alerts. The medication with the most overridden alerts was oxycodone-acetaminophen with 600 alerts from June 18 to June 24, 2018 (see Table 2). The majority of the alerts for oxycodone-acetaminophen (accounting for approximately 88% or 527 alerts) were attributed to missing/invalid weight and multiple grouper alerts. These were resolved by customizing the preference settings as described in the “Methods” section.
Table 2.
Top 10 Drugs.
| Top 10 drugs | |
|---|---|
| Medication | Number of overriden alerts |
| Oxycodone-acetaminophen | 600 |
| Multivitamin, prenatal | 448 |
| Albuterol | 336 |
| Oxytocin 20 units | 332 |
| Lorazepam | 304 |
| Lidocaine | 290 |
| Clopidogrel | 237 |
| Potassium chloride | 199 |
| Hydrocortisone topical | 199 |
| Glucose | 192 |
From July to September 2018, an average of 70 DRC tables were reviewed by informatics pharmacists, reducing the number of overridden DRC alerts to 4796 in the first week of September—a 63% decrease in a 3-month period (Figure 2). The data from June 2018 represent a baseline as no intervention was made prior to July 2018.
Figure 2.
Number of overridden DRC alerts in PowerChart.
Abbreviation: DRC, dose range checking.
Whereas the initial strategy to decrease the number of DRC alerts was primarily carried out by systematic DRC table reviews and order sentence updates, the later strategy involved modifying the DRC Error Message preference settings. In January 2019, 14 731 (6.6%) of DRC alerts were overridden out of 224 721 orders. In March 2020, 5893 (3.2%) of DRC alerts were overridden out of 183 754 orders (Table 3). The absolute number of overridden DRC alerts decreased from 14 731 in January 2019 to 5893 in March 2020—a 60% decrease (Figure 3).
Table 3.
Number of Overridden DRC Alerts per Month.
| Month-year | Number of DRC alerts | Number of orders | Ratio in % |
|---|---|---|---|
| January-2019 | 14 731 | 224 721 | 6.6 |
| February-2019 | 11 768 | 190 354 | 6.2 |
| March-2019 | 11 190 | 197 422 | 5.7 |
| April-2019 | 12 028 | 242 282 | 5.0 |
| May-2019 | 8560 | 192 296 | 4.5 |
| June-2019 | 8638 | 194 362 | 4.4 |
| July-2019 | 9997 | 239 576 | 4.2 |
| August-2019 | 7408 | 198 916 | 3.7 |
| September-2019 | 9212 | 252 826 | 3.6 |
| October-2019 | 7569 | 211 590 | 3.6 |
| November-2019 | 7463 | 207 644 | 3.6 |
| December-2019 | 7892 | 211 480 | 3.7 |
| January-2020 | 10 500 | 270 980 | 3.9 |
| February-2020 | 8312 | 218 136 | 3.8 |
| March-2020 | 5893 | 183 754 | 3.2 |
Abbreviation: DRC, dose range checking.
Figure 3.
Number of overridden DRC alerts versus ratio of overridden DRC alerts.
Abbreviation: DRC, dose range checking.
Discussion
Informatics pharmacists have a critical role in reducing alert fatigue and improving the quality of CDS alerts in CPOE systems. With a collaborative approach, the overall rate of overridden DRC alerts, as well as the total number of DRC alerts firing, was reduced drastically in our health system. This was made possible by the systematic review and updating of the DRC tables and by implementing 3 key strategies: reviewing and updating order sentences, optimizing the rule that triggers the DRC alerts, and modifying the DRC Error Message preference settings. As these strategies were being implemented, our team remained in close contact with prescribers and pharmacists, which enabled us to receive direct feedback from the end users whose overall perception of the changes was very positive. Nevertheless, there are several limitations to this study. While our article focuses primarily on reducing the number of overridden DRC alerts, further analysis needs to be conducted on the impact of our interventions on the rate of overridden DRC alerts. Our preliminary data prior to any interventions showed that the rate of overridden DRC alerts was at 52% in June 2018 and decreased to 34.1% in March 2020. However, no correlations were established between the deployment of a specific strategy and its direct impact on the rate of overridden DRC alerts. Further analysis needs to be performed on the impact of alert suppressions on the incidence of adverse events. Data regarding the number of order sentences updated and the number of alerts triggered on discontinued orders were not included in the analysis. Although our team was made aware of any incident reports related to DRC alerts, more research is needed to further capture the effect of our DRC interventions on patient outcomes. As DRC alerts are only one type of CDS alert, additional research needs to be done regarding other opportunities for informatics pharmacists to intervene.
Conclusion
The field of pharmacy informatics is relatively new; yet this interdisciplinary area, which combines an expertise in clinical pharmacy and technology, has proven to play a key role in improving health care workflows by making them more efficient, cost-effective, and safer to patients. By reducing the number of DRC nuisance alerts and improving the quality of their clinical content, informatics pharmacists can contribute to lowering alert fatigue and improving providers’ trust in CDS alerts.
Acknowledgments
The authors wish to extend their sincere appreciation to their colleagues, in the Pharmacy Informatics Department and the Clinical Pharmacists throughout the Baptist Health South Florida System, whose contributions to this project were immensely helpful. The authors owe much gratitude to Dr Madeline Camejo, Vice President of Pharmacy Services, without whom the Pharmacy Informatics Department would not have been created.
Footnotes
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors certify that they have no potential conflicts of interest with respect to the research, authorship, and publication of this article. They have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements) in the subject matter or materials discussed in this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
ORCID iD: Jonathan F. Choukroun 
https://orcid.org/0000-0002-4340-9821
References
- 1. Greenes R. Clinical Decision Support: The Road Ahead. Elsevier; 2007. doi:10.1016/B978-0-12-369377-8.X5000-4. [Google Scholar]
- 2. Evans RS. Electronic health records: then, now, and in the future. Yearb Med Inform. 2016;(suppl 1):S48-S61. doi: 10.15265/IYS-2016-s006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. doi: 10.1038/s41746-020-0221-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Osheroff J. Improving Medication Use and Outcomes with Clinical Decision Support. Chicago, IL: Healthcare Information & Management Systems Society; 2009. [PMC free article] [PubMed] [Google Scholar]
- 5. Boling B, McKibben M, Hingl J, et al. Effectiveness of computerized provider order entry with dose range checking on prescribing errors. J Patient Saf. 2005;1(4):190-194. doi: 10.1097/01.jps.0000215339.03807.fd. [DOI] [Google Scholar]
- 6. Kawamoto K, Houlihan C, Balas B, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765. doi: 10.1136/bmj.38398.500764.8F. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Ancker JS, Edwards A, Nosal S, et al. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017;17:36. doi: 10.1186/s12911-017-0430-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Jones K. Alarm fatigue a top patient safety hazard. CMAJ. 2014;186(3):178. doi: 10.1503/cmaj.109-4696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Citing reports of alarm-related deaths, the Joint Commission issues a sentinel event alert for hospitals to improve medical device alarm safety. ED Manag. 2013;26(6):1-3. [PubMed] [Google Scholar]
- 10. Johnson KB, Lee CK, Spooner SA, Davison CL, Helmke JS, Weinberg ST. Automated dose-rounding recommendations for pediatric medications. Pediatrics. 2011;128(2):e422-e428. doi: 10.1542/peds.2011-0760. [DOI] [PMC free article] [PubMed] [Google Scholar]