Sensitivity and specificity of dosing alerts for dosing errors among hospitalized pediatric patients

Abstract

Objectives

To determine the sensitivity and specificity of a dosing alert system for dosing errors and to compare the sensitivity of a proprietary system with and without institutional customization at a pediatric hospital.

Methods

A retrospective analysis of medication orders, orders causing dosing alerts, reported adverse drug events, and dosing errors during July, 2011 was conducted. Dosing errors with and without alerts were identified and the sensitivity of the system with and without customization was compared.

Results

There were 47 181 inpatient pediatric orders during the studied period; 257 dosing errors were identified (0.54%). The sensitivity of the system for identifying dosing errors was 54.1% (95% CI 47.8% to 60.3%) if customization had not occurred and increased to 60.3% (CI 54.0% to 66.3%) with customization (p=0.02). The sensitivity of the system for underdoses was 49.6% without customization and 60.3% with customization (p=0.01). Specificity of the customized system for dosing errors was 96.2% (CI 96.0% to 96.3%) with a positive predictive value of 8.0% (CI 6.8% to 9.3). All dosing errors had an alert over-ridden by the prescriber and 40.6% of dosing errors with alerts were administered to the patient. The lack of indication-specific dose ranges was the most common reason why an alert did not occur for a dosing error.

Discussion

Advances in dosing alert systems should aim to improve the sensitivity and positive predictive value of the system for dosing errors.

Conclusions

The dosing alert system had a low sensitivity and positive predictive value for dosing errors, but might have prevented dosing errors from reaching patients. Customization increased the sensitivity of the system for dosing errors.

Background

Medication dosing for pediatric patients is a complex process where a clinician individualizes the dose based on a patient's weight, age, organ function, indication, and concomitant disease. Pediatric patients are reported to have a threefold higher potential adverse drug event (ADE) rate than adults.¹ In both the pediatric and adult population, medication prescription has been identified as the most common cause of medication errors and preventable ADEs.^1–3 Dosing errors are the most frequently reported medication error in pediatric patients,⁴ and may be fatal if not corrected before administration.⁵ Errors have been reported to occur in 0.19–2.5% of orders before the implementation of computerized provider order entry (CPOE).^{1 6–8}

Implementation of CPOE and computerized clinical decision support (CDS) systems to improve medication safety in the hospital setting has been increasing across the USA.⁹ This increase has been spurred by the financial incentives offered by the Center for Medicare and Medicaid Services for the implementation of ‘meaningful use’ health information technologies.¹⁰ In a 2008 survey of Children's Hospitals in the USA, 44% of hospitals reported having a dosing CDS functionality.¹¹ Some institutions have designed their own dosing support system or modified a system developed for the adult population to accommodate pediatric dosing support.^{6 12 13} Additionally, medication information database companies have created commercially available pediatric dose ranges for use in electronic health record (EHR) systems. Depending on the EHR provider and the drug information database, the CDS functionalities may or may not be customizable.

The impact of CDS dosing support functionalities on the occurrence of dosing errors and dose-related ADEs varies.¹⁴ Some studies report significant reductions in dosing errors with dosing support independent of CPOE,^{8 15 16} whereas other studies suggest no change or a non-significant increase in dosing errors after the implementation of CPOE with dosing support.^{6 7} Dose, frequency, and wrong dose unit error rates have been reported to occur in 0.27–1.2% of orders after CPOE and CDS dosing support implementation.^6–8

A previous study suggested that an implemented CPOE system with a dosing CDS functionality did not alert providers about 17 of 19 serious dosing errors occurring after implementation.⁶ In addition, it has been suggested that diminished provider response to appropriate alerts may be occurring owing to excessive alerting, also known as alert fatigue.^{17 18} This is of concern for dosing alerts, in particular, as we have previously reported that the majority of dosing alerts may be alerting providers inappropriately.¹⁹ Other studies have also found inaccuracies in commercially available dosing ranges.²⁰ A framework for analyzing alert appropriateness has been used to study kidney injury medication alerts.²¹ While the study using the framework and our previous study¹⁹ effectively analyzed the alerts that occurred, they did not analyze errors and adverse events that occurred without causing a dosing alert, or inappropriate non-alerts. Determining sensitivity and specificity with positive and negative predictive values of dosing alert systems for the occurrence of dosing errors would provide a more complete analysis of dosing alert appropriateness and has not been performed.¹⁴

Objectives

The primary objectives of this study were to determine the sensitivity and specificity of a dosing alert system for dosing errors and to compare the sensitivity of a proprietary dosing alert system with institutional customization with the same system without customization. Secondary objectives were to determine outcomes of dosing alerts, categorize dosing errors by severity, and identify areas for refinement of the dosing alert functionality.

Methods

Description of institution and computerized clinical decision support system

This study was conducted at Nationwide Children's Hospital (NCH), an academic pediatric tertiary referral center with over 350 beds, including neonatal and pediatric intensive care units, an emergency department, and other specialized inpatient units. Our computerized informatics system combined Epic (Epic Systems Corporation, Verona, Wisconsin, USA) with the First Databank (South San Francisco, California, USA) drug information database to provide multiple CDS functionalities, such as medication dosing and infusion calculators, medication allergy alerts, therapeutic duplication alerts, medication interaction alerts, and medication dose range alerts. The CDS dosing alerts were presented to practitioners during the medication ordering and verification process. Practitioners could over-ride each alert and continue with the order as written or discontinue the order before providing a digital signature. Multiple alerts for the same order were presented on one notification screen, but each alert could be individually over-ridden or the order discontinued. Modifications of orders (common with continuous infusions) were also checked by the decision support functionalities.

In the alert notification screen, the dosing alert provided the CDS system calculated dose (single and/or daily), the percentage over or under the CDS dose range, and also the CDS suggested dose range for that medication. Alerts were filtered and not presented to practitioners if the dose was within 10% of the dose range limits, if the alert was for an as needed order exceeding daily dose limits (eg, an every 1 h as needed order in an intensive care unit), and if the alert occurred because the system was unable to convert dosing units for calculation (eg, with topical products and some intravenous fluids). If the system could not find adequate information to determine a dose (eg, weight, if the range was based on weight), an alert also occurred. Medications ordered which did not have associated dose ranges for the patient's age created an alert stating ‘Dose checking cannot be performed’ or ‘Contraindicated based on patient information.’

The proprietary First Databank database provided pediatric specific dose ranges for over 2000 drugs. Practitioners (other than the authors) at NCH customized ranges for 694 of the drugs (central nervous system (CNS) agents, 34.0%, and anti-infective agents, 24.9%, were the top two medication classes with customized ranges). Any NCH customized range took precedence. Dose ranges could be changed based on a medication's dosage form. The minimum and maximum doses could have been written in a weight-based (eg, mg/kg/dose) or fixed-dose format for a one-time dose, single dose for maintenance dosing, or cumulative daily dose for maintenance dosing. Daily dose calculation allowed for frequency and continuous infusion rate determinations. It was possible to specify a dose range by age or weight, gestational age at birth, and route of administration.

Data collection

Reports were generated from pharmacy records that included medication orders and medication dosing alerts, similar to our previous study.¹⁹ In addition, a report was generated including reported ADEs and pharmacist-detected dosing errors (dosing interventions made during order verification or after the medication was administered and documented in the reporting system) occurring during the calendar month of July, 2011. Orders, ADEs, and dosing errors were excluded if they were for outpatients, for patients ≥18 years of age, and for patients involved in a research study. Each medication order which caused a dosing alert to be presented to a practitioner (eg, physician, pharmacist, or nurse), ADE, and dosing error was reviewed, along with the patient's medical record. For each potential dosing error, the medication dose and frequency ordered were assessed for correctness based on the indication-specific dosing recommendations in the Lexi-Comp Pediatric Dosage Handbook²² and any institution-specific dosing recommendations (eg, guidelines and formulary monographs). We chose Lexi-Comp for comparison because it is among the most widely used pediatric dosing reference.

For the purposes of this study, a dosing error was defined as an ordered dose that was outside the Lexi-Comp and institutional dosing recommendations without a clinically justifiable reason (eg, medication titration, pharmacokinetic adjustment), or an ordered dose that was not appropriate for the patient's clinical situation (eg, incorrect dosing for the desired indication or unintentionally starting a patient on a dose different from their home dose). Any orders that did not have dosing recommendations available in Lexi-Comp or institutional recommendations were separately categorized, but not considered dosing errors.

For each dosing error identified, it was determined if an alert occurred (owing to a customized or a non-customized dose range), if an alert would have occurred without customization, how many alerts were over-ridden, if the incorrect medication dose was received by the patient, and if there was patient harm as a result of the dosing error. A medication order is usually entered by a prescriber and then verified by a pharmacist in the hospital before administration. Alerts could be presented to the prescriber, pharmacist, and nurse for a given order. To best approximate medication safety outcomes due to dosing alerts, an order with multiple alerts was only counted as one alerted order. An inappropriate alert over-ride was defined as a dosing error entered into the EHR system with an alert over-ridden by a practitioner. Medications were categorized as having a high potential for adverse events based on the Institute for Safe Medication Practices high alert medication list.²³ Medication classes were determined using the American hospital formulary service pharmacologic-therapeutic classes.²⁴ All dosing errors were categorized using the National Coordinating Council for Medication Errors Reporting and Prevention Medication Error Index. This index categorizes medication errors from A to I based on the severity of the outcome (category A being a situation with the capacity for an error and category I being death as a result of a medication error).²⁵ The first author (JSS) reviewed each order, alert, and/or ADE. Orders with uncertainty were reviewed together with the third author (MCN).

Statistical analysis

Age was summarized by median and range; categorical characteristics were summarized as frequency and percentage. Sensitivity and specificity of the dosing alert system for identifying dosing errors was calculated, with corresponding 95% exact binomial CIs. Sensitivity of the CDS system for dosing errors without customization was compared with the sensitivity with customization by McNemar's test. Significance was set at α=0.05. All statistical tests were completed using SAS software, V.9.2 (SAS Institute Inc, Cary, North Carolina, USA, 2011).

Results

During the study period, 91 963 medication orders were identified from pharmacy records, with 47 181 inpatient pediatric orders included in this analysis (figure 1). Two hundred and fifty-seven medication dosing errors (0.54% of all orders) were identified in 189 patients (48% female, 52% male) with a median age of 6.2 years (range 0–17.8 years). The dosing errors identified most frequently involved anti-infective agents (36.6%), CNS agents (21.4%) and gastrointestinal drugs (12.6%), with 19.8% of errors involving medications with a high risk for adverse events (eg, insulin, morphine, potassium chloride).²³ The majority of dosing errors occurred in specialty (eg, cardiology and pulmonology) units of the hospital (60.3%) and the intensive care units (15.6% in the pediatric, neonatal, and cardiothoracic units), followed by the emergency department (12.8%) and hematology/oncology unit (9.7%). Table 1 provides a detailed analysis of the other characteristics and outcomes of the identified medication dosing errors.

Figure 1.

Medication orders analyzed. ADEs, adverse drug events.

Table 1.

Characteristics and outcomes of medication dosing errors (n=257)

Error type/outcome	Overall (%)	Appropriate alert due to non-customized range (%)	Appropriate alert due to customized range (%)	Inappropriate non-alert due to customized range (%)	Inappropriate non-alert with both ranges (%)
Total dosing errors	257	122	33	17	85
Orders with a correct alert over-ridden*	155	122	33	0	0
Alerts viewed by practitioners	255	197	58	0	0
Potential dosing errors (B)†	179 (69.6)	68 (55.7)	24 (72.7)	15 (88.2)	72 (84.7)
Overdoses (OD)	91 (35.4)	44 (36.1)	7 (21.2)	6 (35.3)	34 (40.0)
Underdoses (UD)	88 (34.2)	24 (19.7)	17 (51.5)	9 (52.9)	38 (44.7)
Tenfold OD‡	8 (3.1)	6 (4.9)	0	0	2 (2.4)
Tenfold UD‡	6 (2.3)	4 (3.3)	2 (6.1)	0	0
High alert‡	39 (15.2)	15 (12.3)	11 (33.3)	2 (11.8)	11 (12.9)
Actual dosing errors (C,D,E)†	78 (30.4)	54 (44.3)	9 (27.3)	2 (11.8)	13 (15.3)
OD	32 (12.5)	21 (17.2)	3 (9.1)	2 (11.8)	6 (7.1)
UD	43 (16.7)	32 (26.2)	6 (18.2)	0	5 (5.9)
Contraindicated	3 (1.2)	1 (0.8)	0	0	2 (2.4)
Tenfold OD or UD‡	0	0	0	0	0
High alert‡§	12 (4.7)	4 (3.3)	5 (15.2)	0	3 (3.5)
Required intervention, no harm‡ (D)†	12 (4.7)	6 (4.9)	0	2 (11.8)	4 (4.7)
Patient harm‡ (E)†	4 (1.6)	2 (1.6)	0	0	2 (2.4)

*A dosing error entered into the system with an alert over-ridden by a practitioner.

†Categorized based on the National Coordination Council for Medication Errors Reporting and Prevention Index.

‡Also included in the OD, UD, or contraindicated categories.

§As defined by the institute for Safe Medication Practices high alert medication list.

Figure 1 shows the breakdown of the order categories. With customized dosing ranges, there were 155 true positive and 1780 false-positive alerts for dosing errors. The sensitivity of the dosing alert system for dosing errors outside the Lexi-Comp and institutional recommendations was 60.3% (CI 54.0% to 66.3%), with a specificity of 96.2% (CI 96.0% to 96.3%) and a positive predictive value of 8.0% (CI 6.8% to 9.3%). Without customization, 54.1% (CI 47.8% to 60.3%) of dosing errors would have caused an alert (sensitivity), compared with 60.3% with customization (p=0.02). This provides a number needed to alert of 16.1, suggesting that for every 16 dosing errors with an alert, one additional alerted order will occur owing to customization.

The sensitivities of customized and non-customized dose ranges for detecting overdoses, 10-fold underdoses, and contraindications were similar. The 10-fold overdoses that did not cause dosing alerts were because the system did not have a dose range for antihemophilic factor (recombinant) and 0.9% sodium chloride. For example, there was no customized or non-customized dose range for 0.9% intravenous sodium chloride administered via continuous infusion. Owing to the absence of this range, an order for 0.9% intravenous sodium chloride was entered as 54 mL/kg continuously instead of 54 mL/h for a 17 kg patient for rehydration (potential 10-fold dosing error) and did not cause an alert. All 10-fold underdoses caused an alert after customization. These alerts would not have occurred if non-customized ranges were used because minimum values were not provided for single dose underdoses. The area in which customization had the most impact was in increasing the percentage of underdose errors with appropriate alerts (49.6% without customization compared with 60.3% with customization, p=0.01) (figure 2). CNS agents comprised 66.7% of the errors that had alerts due of customization, followed by anti-infective agents at 15.2%.

Figure 2.

Comparison of dosing alert system sensitivity with and without customization. ^Overdose was defined as a single or daily dose overdose. ‘Underdose was defined as a single or daily dose underdose. *Comparison of non-customized dosing ranges with those with customization using McNemar's test or Fisher's exact test.

The lack of the dosing rule logic to allow for indication-specific dose ranges was the most common reason for the non-occurrence of dosing alerts for dosing errors. For example, an 8.8 kg female child was ordered methylprednisolone 2 mg/kg/dose intravenously every 6 h for status asthmaticus. This was above the recommended dosing of 1 mg/kg/dose intravenously every 6 h. However, an alert did not occur because an indication-specific range could not be created and the range was broadened to include the 30 mg/kg/day dosing that could be used for anti-inflammatory ‘pulse’ therapy, lupus nephritis or spinal cord injuries.²² Dose ranges that did not match recommendations were the second most common reason for inappropriate non-alerts. For example, a 2-month-old infant born at 35 weeks’ gestation was ordered linezolid 10 mg/kg/dose by mouth every 12 h for treatment of a methicillin-resistant Staphylococcus aureus infection. Dosing frequency for this patient should be every 8 h. An alert did not occur because the daily minimum of the range was 20 mg/kg/day. Patient-specific errors for dosing within recommendations, but inappropriate for the patient, caused the remaining inappropriate non-alerts (table 2). The dosing errors without alerts most frequently involved anti-infective agents (53.9%), CNS agents (16.7%), and gastrointestinal drugs (15.7%).

Table 2.

Reasons for inappropriate non-alerts for dosing errors (n=102 total)

Reason	No (%)
Indication specificity needed	37 (36.3)
Incorrect dose range	36 (35.3)
Dose within normal range, but incorrect based on previous pharmacokinetic evaluation	11 (10.8)
Incorrect weight entered into the medical record	7 (6.9)
Medication reconciliation error	5 (4.9)
Inappropriate medication titration	4 (3.9)
Others	2 (2.0)

The majority (64.7%) of inappropriate non-alerts due to customization occurred because dose ranges were broadened by our institution (table 3). These non-alerts were mainly for anti-infective agents (58.8%). This strategy was used to prevent excessive false-positive alerts for medications with multiple indications and patient-specific dosing adjustments. Seventy per cent of the dosing errors were potential dosing errors that did not reach the patient (category B errors). Dosing alerts occurred in 92 of the category B dosing errors (51.4%) and might have helped to prevent these erroneous doses from being administered (table 1). Most of the dosing errors administered to patients did not cause harm or require an intervention. The inappropriate over-ride rate for dosing errors was 40.6% (63 of 155 errors with appropriate alerts), resulting in two errors that caused patient harm (category E). Additional details about dosing error outcomes are presented in table 1.

Table 3.

Types of dose range customization

	No. orders
Customization causing appropriate alerts	33
Underdoses
Single dose rule added	23 (69.7)
Overdoses
Added appropriate single maximum dose as flat adult dose, instead of a mg/kg dose	4 (12.1)
Changed flat single dose maximum to a weight-based maximum	3 (9.1)
Added single dose maximum for neonatal patients	1 (3.0)
Added daily dose maximum for neonatal patients	1 (3.0)
More conservative single and daily dose rules used	1 (3.0)
Customization causing inappropriate non-alerts	17
Underdoses
Broadened age range or dose range	3 (17.6)
Single dose minimum used without daily dose minimum	2 (11.8)
Neonate specific dose range not created	1 (5.9)
Overdoses
Broadened age range or dose range	8 (47.1)
Dose in mg/kg used as maximum instead of a flat adult dose	2 (11.8)
Neonate specific dose range not created	1 (5.9)

Table 4 provides rates relating to the outcomes associated with dosing alerts. Dosing alerts appropriately alerted practitioners about dosing errors. Although it was not possible to determine if the order was changed because of the alert, the alerts were one of the safety checks which might have prevented the administration of these dosing errors. Inappropriate alert over-rides occurred and erroneous medication orders were administered. In addition, inappropriate non-alerts might have resulted in dosing errors and preventable ADEs. Annualized occurrence estimates are presented in table 4 and provide an estimate of the impact that dosing alerts, customization, alert fatigue, and inappropriate non-alerts might have at our institution each year.

Table 4.

Outcomes associated with the dosing alert system

Outcome evaluated	Occurrence
	Per 100 orders	Annual estimate
Appropriate dose ranges
Medication dosing errors with appropriate alerts	0.33	1860
Potential dosing errors the system helped prevent (B)*	0.19	1104
Dosing errors involving high alert medications† prevented	0.055	312
Tenfold or greater dosing errors prevented	0.025	144
Inappropriate over-rides (alert fatigue)
Incorrect dosing order entry with at least one over-ride (B–E)*	0.33	1860
And an actual dosing error received by the patient (C–E)*	0.13	756
And requiring an intervention (D)*	0.013	72
And causing an adverse event (E)*	0.0042	24
Inappropriate dose ranges (non-alerts)
Overall dosing errors causing an inappropriate non-alert	0.22	1224
Both ranges causing an inappropriate non-alert (B–E)*	0.18	1020
And an actual dosing error received by the patient (C–E)*	0.028	156
And requiring an intervention (D)*	0.0085	48
And causing an adverse event (E)*	0.0042	24
Customization causing an inappropriate non-alert (B–E)*	0.036	204
And an actual dosing error received by the patient (C–E)*	0.0042	24
And requiring an intervention (D)*	0.0021	12

*Categorized based on the National Coordination Council for Medication Errors Reporting and Prevention Index.

†As defined by the institute for Safe Medication Practices high alert medication list.

Discussion

The results of this study suggest that the dosing alert system used at our institution had a low sensitivity and positive predictive value for incorrect doses, despite a high specificity and negative predictive value. Dosing alerts helped to prevent over 90 dosing errors from reaching patients in 1 month, potentially over 1100 annually. Incorrect doses were still administered in 40.6% of orders with appropriate alerts, indicating that alert fatigue may be occurring. This may be a result of the low positive predictive value of the dosing alert system. Customization significantly improved the number of incorrect orders that caused alerts, mainly in the detection of underdoses and dosing errors involving CNS agents. Notably, some 10-fold overdoses, underdoses, and age-related contraindicated medications were ordered without having associated alerts. The potential 10-fold overdoses without alerts were due to a lack of dosing rules for the ordered drug and route. It is important for others to identify medications which do not have dosing rules in order to ensure that 10-fold dosing errors are not entered without causing an alert.

Previous CPOE and CDS implementation studies suggested an overall decrease in medication-related errors after the introduction of combined systems, although analyses of the impact of specific CDS functionalities have shown varying results.^{7 12 26} In addition, some CDS implementation studies were completed using institutionally derived systems and in different countries, potentially limiting their applicability to other institutions.^{7 8 12 13 16} The dosing CDS functionality is important for pediatric institutions because dosing errors are the most commonly reported medication errors in pediatric patients.⁴ Our study used a commercially available decision support database with customization, allowing this study to be applied to other institutions beginning to implement the same pediatric dosing alert functionality.

A framework to evaluate alert appropriateness was recently suggested, although it did not discuss the evaluation of alerts that should have occurred and was validated on a CDS tool providing alerts for patients with acute kidney failure.²¹ This framework may be difficult to apply to dosing alerts presented during the order entry process. Many previous studies lacked an analysis of the sensitivity and specificity of dosing alerts for dosing errors.¹⁴ One study did report that only two of 19 serious dosing errors had associated dosing alerts; however, all alerts were not analyzed.⁶ Our study suggests a higher sensitivity than previously reported,⁶ although it is still lower than desired. Our data might have underestimated the number of undetected errors because some dosing errors that occurred might not have been documented. However, the dosing error rate (0.54%) is consistent with previously reported dosing, frequency, and incorrect dose unit error rates (0.27–1.2%) after CPOE and CDS implementation.^6–8

Low acceptance of dosing recommendations provided before order entry has been reported previously.¹³ Previous studies also gave limited information about responses to dosing alerts and how alerts and responses to alerts affected patient care.^{8 12} A low positive predictive value of dosing alerts for dosing errors, as suggested by our data, could create an environment prone to low acceptance and alert fatigue. Alert fatigue might have allowed two dosing-related preventable ADEs to occur in our analysis (potentially 24 yearly at our institution). These data illustrate the need to refine the dosing alert functionality.

Although our study provides objective results about the outcomes of dosing alerts, there are limitations and unanswered questions for future research. Future research could aim at comparing the sensitivity and specificity of CDS systems at different hospitals or among different drug information databases. Our study design allowed for comparison of sensitivity. However, specificity and positive predictive value might have differed between a customized and non-customized system owing to the broadening of dose ranges by our institution. This strategy can be used to decrease inappropriate alerts, but has the risk of not alerting dosing errors and needs further evaluation. Additionally, we separated medication orders that did not have available dosing recommendations for comparison and we were unable to determine the sensitivity and specificity of the system for this type of order. This should be taken into account when designing future studies as it often occurs in pediatrics and requires further evaluation.

These data demonstrate the ability of dosing alerts to prevent errors and also identify areas for improvement for this CDS functionality. While customization increased the detection of underdoses, there was no significant improvement in the detection of overdoses or 10-fold errors. The lack of indication-specific dose ranges was the main reason why dosing errors did not cause dosing alerts. This problem may not be amendable by institutional customization, and thus information database and EHR companies must collaborate with institutions to make this possible. Incorrect dose ranges were also a common cause of alerts; however, they are amendable by customization. Incorporation of other patient-specific parameters into a dosing alert functionality needs to be explored in order to improve their appropriateness and meaningful use for clinicians (eg, renal function-based dosing adjustments). This would require the ability of a decision support system to use data available in medical records to determine a dose. In adult populations, CDS systems have incorporated renal function and hematologic laboratory values into CDS systems. These interventions led to improved dosing accuracy during renal insufficiency²⁷ and more appropriate use of hematopoietic and anticoagulant agents.²⁸ Our data are consistent with previous reports suggesting that ‘simple rules’ with conditional logic needs to be refined through collaboration to improve CDS functionalities.²⁹

Conclusions

The customized dosing alert system at our institution provided a sensitivity of about 60%, a specificity of 96% and a positive predictive value of 8% for identifying dosing errors. The sensitivity would have been lower (54%) if customization had not occurred. The impact of customization was mainly due to the increased detection of underdosing errors and dosing errors involving CNS agents. Appropriate dosing alerts might have prevented dosing errors from reaching patients, although alert fatigue may have been present. Advancements in alert rule logic should focus on providing indication- and patient-specific dosing support.