Optimizing drug-dose alerts using commercial software throughout an integrated health care system

Abstract

All default electronic health record and drug reference database vendor drug-dose alerting recommendations (single dose, daily dose, dose frequency, and dose duration) were silently turned on in inpatient, outpatient, and emergency department areas for pediatric-only and nonpediatric-only populations. Drug-dose alerts were evaluated during a 3-month period. Drug-dose alerts fired on 12% of orders (104 098/834 911). System-level and drug-specific strategies to decrease drug-dose alerts were analyzed. System-level strategies included: (1) turning off all minimum drug-dosing alerts, (2) turning off all incomplete information drug-dosing alerts, (3) increasing the maximum single-dose drug-dose alert threshold to 125%, (4) increasing the daily dose maximum drug-dose alert threshold to 125%, and (5) increasing the dose frequency drug-dose alert threshold to more than 2 doses per day above initial threshold. Drug-specific strategies included changing drug-specific maximum single and maximum daily drug-dose alerting parameters for the top 22 drug categories by alert frequency. System-level approaches decreased alerting to 5% (46 988/834 911) and drug-specific approaches decreased alerts to 3% (25 455/834 911). Drug-dose alerts varied between care settings and patient populations.

BACKGROUND

Computerized physician order entry (CPOE) presents many opportunities to improve medication-related patient safety. Drug-drug, drug-allergy, drug-diagnosis, and drug-dose alerts are potential areas in which to reduce medication errors and have been identified by LeapFrog and others as CPOE best practices.¹ However, CPOE drug alerts are overridden 49%–96% of the time.² Poorly designed alert systems with low specificity can be an obstacle to patient safety while causing unintended consequences for clinicians.³ Furthermore, barriers to adoption of meaningful clinical decision support include lack of resources and the expertise to develop, implement, and evaluate these systems.⁴ Improving CPOE drug-dose alerts to enhance specificity and sensitivity, reducing false-positive alerts and thereby creating more effective drug-dose alerts, is necessary to realize the full potential of CPOE for patient safety.

To date, few studies have looked at categorizing CPOE drug-dose alerts and described strategies to optimize commercially available CPOE alerts using commercially available drug information in an integrated health care delivery network encompassing different health care settings and different patient populations.⁵ Our study aims were (1) to determine the initial frequency of drug-dose alerts using our commercial electronic health record (EHR) and commercial third-party drug information and (2) to develop and evaluate generalizable strategies to optimize drug-dose alerting.

METHODS

The MetroHealth System (MHS) is a safety-net, integrated, academic health care system affiliated with Case Western Reserve University in Northeast Ohio. MHS includes a 542-bed level-1 trauma center hospital, 4 emergency departments (EDs), and over 2 dozen outpatient clinics. MHS employs over 500 primary-care and specialty-care physicians and over 350 physicians in training. MHS has more than 28 000 inpatient hospitalizations, over 1 million outpatient visits, and over 100 000 ED visits annually. MHS is a Healthcare Information Management and Systems Society Electronic Medical Record Adoption Model Stage 7 health care system in both its hospital and ambulatory settings and a Healthcare Information Management and Systems Society enterprise Davies Award winner.

MHS began using the Epic EHR (EpicCare, Epic Systems, Verona WI, USA), including CPOE in 1999. All visit information is recorded in the EHR, including all orders and prescriptions. All drug-dose alerts were turned on silently, not shown to end users but tracked and analyzed. The Medi-Span^® drug dosing database provided the data for drug-dose decision support. Medi-Span and our EHR vendor provided recommended default settings for drug-dose decision support alerts. The drug-dosing alerts do not have tiers of alerts, such as high, medium, and low risk, as other drug-drug interactions alerts do. These recommended default settings can be adjusted by the customer. The Medi-Span drug dosing database includes data specifying recommended minimum and maximum single dose, daily dose, dose frequency, dose duration, and maximum lifetime dose amounts specific to route of administration, indication, and dose type.

We examined all silent drug-dose alerts during 3 months in 2013. The initial drug-dose alert setup used default EHR system-level settings for drug-dose alerts and the standard drug information vendor values. Each drug-dose alert record included medication name, dosage, route, alert type and percentage difference from the dose rule, patient department, ordering location, ordering provider, and alert ID. We also obtained the total number of medication orders placed each month. Alerts were categorized by type of alert:

• Below minimum daily dose limit

• Below minimum dose frequency

• Exceeding maximum single dose limit

• Exceeding maximum daily dose limit

• Exceeding maximum dose frequency

• Exceeding maximum dose duration

We also categorized drug-dose alerts by care setting (ED, inpatient, or outpatient, based on ordering location) and pediatric-only versus nonpediatric-only, based on department where the order was entered (Table 1).

Table 1.

Drug-dosing alerts by category, care setting, and pediatric-only clinical settings

Drug-dosing alert category	Baseline drug-dose alerts, %	ED, %	IP, %	OP, %	Pediatric, %	Nonpediatric only populations, %
	(n)	(n)	(n)	(n)	(n)	(n)
Below minimum daily dose	24	12	24	40	7	93
	(24 508)	(1684)	(12 922)	(9902)	(1787)	(23 814)
Below minimum frequency	10	7	50	43	5	95
	(10 330)	(718)	(5163)	(4449)	(559)	(9772)
Exceeded maximum duration	5	5	16	79	4	96
	(4972)	(245)	(816)	(3911)	(208)	(4764)
Exceeded maximum frequency	16	17	55	28	5	95
	(16 566)	(2840)	(9143)	(4583)	(634)	(1593)
Exceeded maximum daily dose	23	15	59	26	10	90
	(24 183)	(3662)	(14 177)	(6344)	(2307)	(2187)
Exceeded maximum single dose	23	20	54	26	9	91
	(23 539)	(4594)	(12 760)	(6171)	(2,206)	(21 333)
Total	100	13	53	34	7	93
	(104 098)	(13 743)	(54 981)	(35 371)	(7701)	(96 397)

ED, emergency department; IP, inpatient; OP, outpatient.

The drug-dose data were manually analyzed to determine clinically appropriate, generalizable strategies to reduce clinically false-positive drug-dose alerts. A 2-step approach was used, employing system-level strategies followed by drug-specific approaches. System-level strategies included changing default settings controlling the firing of drug-dose alerts, either turning off whole categories of alerts or changing their firing thresholds. We incrementally increased the thresholds to evaluate the corresponding impact on the percent reduction in alerts. The research team (3 practicing physicians – internist, family medicine physician, and combined internist and pediatrician – and a practicing registered clinical pharmacist) picked final thresholds to balance decrease in presumed false-positive alerts (ie, nonclinically significant dose increases) with potential increase in patient risk (ie, clinically significant dose increases). Individual drugs were identified that comprised a large portion of alerts by volume (any drug causing >1% of all drug-dose alerts). Drug-specific approaches included changes to the drug-dose firing thresholds for specific drugs. These strategies were developed by the authors for clinical relevance and relatively ease of implementation and maintenance.

To estimate the true drug-overdose rate, we identified the number of self-reported drug-dose errors in the patient safety network reporting system throughout our health care system during the study period. Microsoft Excel 2010 was used to calculate and summarize the drug-dosing alerts in each category and produce descriptive statistics. This study did not undergo Institutional Review Board review because the data contained no protected health information and the study was performed for operational purposes.

Results

During the 3-month study period, 834 911 medication orders were entered through CPOE throughout MHS. Silent drug-dose alerts fired on 12% (104 098) of these orders. For comparison, during the same 3-month period, drug-drug interaction and drug-allergy alerts, both turned on in our system for end users to see, fired on 8% (66 206) and 3% (27 464) of medication orders, respectively. Incomplete information alerts indicating possible drug-dose alerts occurred on 11% (91 060) of these orders (75 564 “rule not found” and 15 496 “missing weight”). Examples included missing weights for weight-based medications or instances where rules could not be applied due to the system’s inability to calculate dose, such as titratable intravenous medication indicating a dose range. For alerts with complete information, the types of drug-dosing alerts and their frequency by clinical area and patient population are shown in Table 1.

At the system level, the paramount concern was patient safety, as opposed to drug efficacy, so we turned off all drug-dose minimum alerts. Removing minimum daily dose and minimum frequency alerts removed 24% (24 508) and 10% (10 330) of all alerts.

Maximum single and maximum daily drug-dose alerts accounted for the 2 largest categories of alerts, making up 46% of all the dosing alerts (23% [23 539] and 23% [24 183], respectively). Table 2 depicts the breakdown of maximum drug-dose alerts by overdose magnitude. Based on drug-dose alert analysis and clinical judgment, we increased maximum single and maximum daily drug-dose thresholds to 125%, reducing these alerts by 13% (3131) and 17% (4036), respectively. Changing drug-dose frequency alerts to more than 2 doses per day above the recommended drug-dose frequency reduced these alerts by 61% (10 133).

Table 2.

Frequency of drug-dosing alerts among maximum drug-dose alert categories

Type of Alert	% (n)
Single drug-dose amount alerts
Exceeds maximum single dose limit by 1%–5%	9 (2188)
Exceeds maximum single dose limit by 6%–10%	1 (309)
Exceeds maximum single dose limit by 11%–15%	2 (344)
Exceeds maximum single dose limit by 16%–20%	1 (292)
Exceeds maximum single dose limit by 21%–25%	4 (903)
Exceeds maximum single dose limit by 26%–99%	16 (3838)
Exceeds maximum single dose limit by 100%	37 (8592)
Exceeds maximum single dose limit by >100%	30 (7073)
Subtotal	100 (23 539)
Daily drug-dose amount alerts
Exceeds maximum daily dose limit by 1%–5%	3 (711)
Exceeds maximum daily dose limit by 16%–10%	1 (210)
Exceeds maximum daily dose limit by 11%–15%	4 (852)
Exceeds maximum daily dose limit by 16%–20%	3 (771)
Exceeds maximum daily dose limit by 21%–25%	2 (587)
Exceeds maximum daily dose limit by 26%–99%	32 (7671)
Exceeds maximum daily dose limit by 100%	25 (6162)
Exceeds maximum daily dose limit by >100%	30 (7219)
Subtotal	100 (24 183)
Daily drug-dose frequency alerts
Frequency exceeded by 1 dose/day	44 (7247)
Frequency exceeded by 2 doses/day	17 (2886)
Frequency exceeded >2 doses/day	39 (6433)
Subtotal	100 (16 566)

After system-level configuration changes, individual drugs were identified based on frequency of alerts triggered. We identified the 22 most frequently alerting drugs (any drug that caused 1% or more of total drug-dose alerts after system-level changes occurred), which caused 21% of all maximum single and maximum daily drug-dose alerts (12 461 and 9072, respectively) (Table 3). The top drugs, simethicone and proton-pump inhibitors, had no alerts after drug-specific dose threshold modifications. Single-dose vancomycin had no alerts, and daily dose had 9 alerts after drug-specific configuration changes. Maximum daily and maximum single drug-dose threshold drug-specific modifications reduced these alerts by approximately 21% (21 533).

Table 3.

Optimized individual drug-dose alerts for high drug-dose alerting frequency medications

Drug name	Baseline max single-dose alerts, % (n)	Baseline max daily dose alerts, % (n)	Baseline single-dose limit	Single drug-dose optimization	Baseline daily dose limit	Daily drug-dose optimization
Aluminum and magnesium hydroxide (simethicone)	3 (2992)	1 (1199)	20 mg	30 mg	60 mg	180 mg
Acetaminophen	2 (1966)	1 (1430)	Weight-based	1000 mg	Weight-based	4000 mg
PPI1	1 (1466)	1 (1751)	20 mg	80 mg	20 mg	120 mg
Vancomycin	1 (1128)	0.5 (556)	1000 mg	2000 mg	2000 (weight-based)	3000 mg
Diatrizoate meglumine-sodium (contrast)	0.6 (665)	0 (0)	90 mL	1000 mL	90 mL	1000 mL
Albuterol (inhaler, solutions)	0.6 (598)	0.5 (539)	4 mg or 2 puffs	15 mg or 8 puffs	8 puffs or 30 mg	48 puffs or 288 mg
Prednisolone, prednisone, triamcinolone	0.6 (575)	0.3 (309)	Varied	1000 mg IV or 100 mg PO	Varied	3000 mg IV or 100 mg PO
Lorazepam	0.5 (540)	1 (444)	Weight-based	30 mg	Weight-based	96 mg
Sodium polystyrene	0.4 (376)	0 (2)	15 g	40 g or 30 mmol	60 g	180 g or 30 mmol
Zolpidem	0.3 (336)	0.3 (327)	5 mg	10 mg	5 mg	10 mg
Amoxicillin	0.3 (309)	0.2 (193)	Weight-based	2000 mg	Weight-based	4000 mg
H2 blockers2	0.3 (297)	0.2 (226)	75 mg	300 mg	150 mg	600 mg
Ibuprofen	0.3 (297)	0.5 (566)	Weight-based	800 mg	Weight-based	3600 mg
Insulin	0.3 (294)	0.3 (349)	Varied	100 units	Varied	300 units
Enoxaparin	0.2 (255)	0.2 (166)	40 mg	180 mg	Weight-based	300 mg
Ipratropium	0.2 (253)	0.4 (387)	0.5 mg or 2 puffs	1.5 mg or 4 puffs	2 mg or 12 puffs	6 mg or 24 puffs
Eye/ear drops3	0.1 (151)	0.2 (218)	Varied	Turned off	Varied	Turned off
Statins4	0.1 (138)	0.1 (119)	Drug-based	80 mg	Drug-based	80 mg
Heparin	0.1 (120)	0 (6)	Weight-based	30 000 units	40 000 units	90 000 units
Ferrous sulfate	0.1 (103)	0.05 (47)	Varied	440 mg	Varied	975 mg
Ondansetron	0.05 (55)	0.08 (80)	4 mg	32 mg	12 mg	48 mg
Morphine	0.03 (33)	0.2 (158)	30 mg PO	40 mg PO	30 mg PO or 180 mg IV	80 mg PO or 360 mg IV
Total	12 (12 461)	9 (9072)

¹PPI, proton pump inhibitor (omeprazole, esomeprazole, pantoprazole, lansoprazole)

²H2 blockers (ranitidine, famotidine, cimetidine)

³Eye/ear drops (antibiotics and steroids)

⁴Statins (atrovastatin, pravastatin, rosuvastatin, simvastatin)

PO, oral; IV, intravenous

The 6 system-level interventions resulted in a reduction of the alert rate from 12% to 5% (46 988). Medication-specific changes to the top 22 alerting drugs decreased drug-dose alerts from 5% to 3% (25 455) (Table 4).

Table 4.

Impact of system-level and drug-level drug-dose alert optimization

Drug-dose alert category	Baseline drug-dose alerts, % (n)	Optimization of drug-dose alerts, % (n)	Optimized drug-dose alerts per order	Optimized drug-dose alerts per 100 orders	Decrease in drug-dose alerting, %
System-level drug-dose alerts
Minimum drug-dose daily dose alerts (removed)	24 (24 508)	0 (0)	0	0	100
Minimum drug-dose frequency alerts (removed)	10 (10 330)	0 (0)	0	0	100
Maximum drug-dose duration alerts (removed)	5 (4972)	0 (0)	0	0	100
Maximum drug-dose single-dose alerts (increased to 125% of threshold)	23 (23 539)	42 (19 503)	0.023	2.3	17
Maximum drug-dose daily dose alerts (increased to 125% of threshold)	23 (24 183)	45 (21 052)	0.025	2.5	13
Maximum drug-dose dose frequency alerts (increased to more than 2 doses/day of threshold)	16 (16 566)	14 (6433)	0.008	0.8	61
Subtotal system-level drug-dose alerts	100 (104 098)	100 (46 988)	0.056	5.6	45
Drug-level drug-dose alerts
Maximum drug-dose single-dose alerts (top 22 individual dose adjustment customized)	58 (12 461)	0 (0)a	0a	0a	>>99
Maximum drug-dose daily dose alerts (top 22 individual dose adjustment customized)	42 (9072)	0 (0)a	0a	0a	>99
Subtotal individual drug-dose alerts	100 (21 533)	0 (0)a	0a	0a	>99
Total drug-dose alerts	100 (104 098)	24 (25 455)	0.030	3.0	76

^aApproximate value.

During the 3-month study period, 219 218 outpatient (87%), 26 140 ED (10%), and 7152 inpatient (3%) visits occurred. At baseline, without any drug-dose alert reduction strategies implemented, this led to approximately 0.16 alerts per outpatient, 7.69 alerts per inpatient, and 0.53 alerts per ED patient.

In subpopulation analysis, drug-dose alerts were most prevalent for inpatients, followed by patients in the ED, with the lowest incidence of drug-dose alerts among outpatients. Drug-dose alerts were more prevalent in pediatric-only populations then among other populations (Table 1).

During the study period, 32 drug-dosing errors (24 wrong dose, 7 wrong frequency, and 1 wrong duration) were manually reported through MHS’s patient safety network reporting system, which likely underreports the true number of drug-dose patient safety issues. We analyzed the details of reported errors and found that 3 of those errors would have been intercepted if the drug-dose alerts were displayed to ordering users. Even after our drug-dose optimizations, drug-dose alerts would have been displayed to ordering users in those 3 cases. The other 29 reported drug-dose errors were related to drug administration and not ordering, and so would not have been affected by drug-dose ordering decision support.

DISCUSSION

Out-of-the-box CPOE drug-dosing alerts based on a commercially available EHR and a commercially available drug information reference produces high (∼12%) drug-dose alerting rates across multiple care settings and patient populations. However, approaches that focus on patient safety–related drug-dose alerts that could potentially cause significant patient harm can significantly reduce baseline drug-dose alerts (decreased by almost 80%). Suppressing alerts can be safe for irrelevant alerts and help reduce alert fatigue.⁶ The framework presented here addresses system-level drug-alert configuration changes, as well as drug-specific configuration changes focusing on a limited number of high-drug-dose alerting drugs. These changes can be relatively quickly implemented and are easy to maintain, leveraging the value of a commercial drug information database to provide and maintain comprehensive drug information.⁷

Others have reported a high incidence of drug-dose alerts and their relative ineffectiveness. Our observations are consistent with previous reports showing considerable volumes of dosing alerts.^7–10 Our observations are also consistent with previous reports of top alerting drugs.⁹ Low specificity and sensitivity, alert overload, workflow interruptions, and perceived relevance to improving quality have all been cited as reasons for ignoring drug-dose alerts.² Drug-dose alerts are also complex because of the thousands of drugs involved and the wide variety of dose ranges and issues, such as prn (as needed) medications in short-term situations where the theoretic daily dose would be high but would never be reached in actual daily dose values (such as sublingual nitroglycerin every 5 min).¹¹ Additionally, commercial drug information vendors may be very conservative in their alerting threshold data due to medical-legal reasons as well as to follow market trends.^7,12 Low-specificity alerts can lead to erroneous alert handling.¹³ When optimized, non–drug-dose drug alerting (duplicate drug class, drug-drug, drug-lab, drug-disease, and drug-pregnancy) has achieved high rates of acceptance.¹⁴

Turning off the maximum duration alerts accounted for 5% of all drug-dose alerts (4972/104 098). Most maximum drug-dose duration alerts were for antibiotics, steroids, opioids, and benzodiazepines. While these medications are typically for relatively short-term use, many clinical examples exist where patients are appropriately placed on these medications for chronic use with typically no or minimal adverse effects. For maximum single-dose and maximum daily drug-dose alerts, clinically there is not a discrete cutoff between therapeutic and toxic drug-dose windows, and therapeutic ranges vary by patient for known and unknown factors, with most medications having a therapeutic index (lethal dose for 50% of the population divided by minimum effective dose for 50% of the population) of at least 2 and many times 10 or more for typical patients.¹⁵ Medications with a therapeutic index <2 are already considered narrow therapeutic index medications and typically require additional side effect/toxicity monitoring regardless of the dose being prescribed.¹⁶ Therefore, we felt comfortable that increasing the maximum single-dose and maximum daily drug-dose alerts by 25% would not significantly risk false-negative alerts, but would significantly decrease clinically insignificant false-positive alerts. The 25% increase before alerting also allows for standard “clinically appropriate overdosing” in situations such as weight-based dosing of medications where small rounding up is needed to provide a practical measurement to dose, or in situations where the patient has developed some tolerance to a medication. Although there could be potential patient safety issues with increasing drug-dose frequency alerting by 2 doses per day for certain medications in isolation, all drug-dose alerts are “additive.” Therefore, allowing an increase of 2 doses per day in the drug-dose frequency alert would still cause a maximum daily drug-dose alert to fire if the added 2 doses exceeded the maximum daily drug-dose alert threshold. After making system-level changes, a relatively small number of drugs (<2 dozen) make up a large percentage (46%) of the remaining drug-dose alerts.

We found that in the subcategories below minimum frequency, maximum frequency, maximum daily dose, and single dose, inpatients had a higher incidence of alerts compared to outpatients and the ED across all drug-dose alerts. The subcategories below minimum daily dose and maximum duration alerts were more common in outpatient areas. Our observation is different than that of the Del Beccaro et al.⁵ study in a pediatric population, where they reported a higher percentage of dosing alerts in the outpatient than the inpatient setting.

Our rate of pediatric drug-dose alerts is likely underreported due to our limitation of looking by EHR department. While most pediatric patients are seen in pediatric-only areas, other areas such as the ED and family medicine and internal medicine–pediatrics clinics also see pediatric populations. Del Beccaro et al., in their pediatric population, showed a higher alert rate for dose range (24%) than our rate of 7%.⁵ Some of our top individual drugs with the highest alerting frequency are similar to the top drug alerts reported by Del Beccaro.

The study also demonstrates the absolute value of turning on drug-dose alerts (and all other alerts) silently to end users first. This allows true in silico testing, analysis, and modification of alert algorithms without any impact on end users. As shown here, detailed analysis of silent alert data can relatively quickly lead to modification of alerting parameters that can greatly change alert rates, increasing their effectiveness. Alerts should only be shown to end users after optimization has occurred based on data.

This study has limitations. First of all, we did not have any objective way to assess whether the proposed alerting changes, compared to the baseline alerting, could have led to any increased potential for patient harm (ie, increase false-negative alerting in any clinically significant way). However, the ordering provider is still always responsible for drugs ordered, and the significant decrease in alerts should lead to increased alert acceptance and therefore improve overall patient safety. Additionally, we did not measure the actual acceptance rate of the baseline alerting or the alerting based on the system-level and drug-level strategies described.

CPOE drug-dose alerts using commercial EHRs and commercial drug information can be optimized using a combination of system-level and limited drug-specific approaches that are relatively easy to implement and maintain. These approaches significantly decrease drug-dose alerts while maintaining such alerts for potentially clinically significant drug overdoses, which should increase drug-dose alert acceptance and improve patient safety.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sector.

Competing Interests

The authors have no competing interests to declare.

Contributors

All authors made substantial contributions to the manuscript. SS and DK served as the lead authors, conducting data analysis and leading manuscript preparation and writing. PG and GF provided substantial guidance, feedback, and edits during the research and editing process. All authors approved this work.