Evaluating the accuracy of electronic pediatric drug dosing rules

Abstract

Objective

To determine the accuracy of vendor-supplied dosing eRules for pediatric medication orders. Inaccurate or absent dosing rules can lead to high numbers of false alerts or undetected prescribing errors and may potentially compromise safety in this already vulnerable population.

Materials and methods

7 months of medication orders and alerts from a large pediatric hospital were analyzed. 30 medications were selected for study across 5 age ranges and 5 dosing parameters. The resulting 750 dosing rules from a commercial system formed the study corpus and were examined for accuracy against a gold standard created from traditional clinical resources.

Results

Overall accuracy of the rules in the study corpus was 55.1% when the rules were transformed to fit a priori age ranges. Over a pediatric lifetime, the dosing rules were accurate an average of 57.6% of the days. Dosing rules pertaining to the newborn age range were as accurate as other age ranges on average, but exhibited more variability. Daily frequency dosing parameters showed more accuracy than total daily dose, single dose minimum, or single dose maximum.

Discussion

The accuracy of a vendor-supplied set of dosing eRules is suboptimal when compared with traditional dosing sources, exposing a gap between dosing rules in commercial products and actual prescribing practices by pediatric care providers. More research on vendor-supplied eRules is warranted in order to understand the effects of these products on safe prescribing in children.

Introduction

Clinical decision support (CDS) for medication dosing often relies on vendor-supplied rules integrated within an electronic health record's (EHR's) computerized provider order entry (CPOE) system. These electronic rules (eRules) contain dosing parameter thresholds that can be used to initiate alerts on rule violation. eRules with thresholds inconsistent with common dosing practices can cause over-alerting and alert fatigue, as well as inappropriate dosing of medications.^1–4 Alert fatigue undermines CDS because it leads users to ignore both accurate and inaccurate alerts. In addition, an absence of eRules can lead to undetected prescribing errors. Accurate drug dosing in CDS is particularly important in pediatric settings.

To date, no study has examined the accuracy of vendor-supplied eRules compared to traditional pediatric dosing guidelines found in commonly-used sources, such as authoritative textbooks and online references. These longstanding sources are accepted as the gold standard for pediatric dosing and reflect everyday prescribing behaviors. This study takes the first step towards understanding the magnitude of accuracy of eRules in the pediatric setting.

Background and significance

High numbers of medication-related errors and adverse drug events are known to occur in both inpatient and outpatient settings.^5–9 In one adult study, most preventable adverse drug events occurred during drug ordering.¹⁰ In another large study, 6% of all medication orders had errors. Serious medication errors occurred in 10 of 100 admissions, over half of which were dosing or frequency errors.⁵ Incorrect dosing is also the most common cause of death in the FDA's Adverse Event Reporting System.¹¹ Children are especially vulnerable to dosing-related adverse drug events. Folli found that the majority of medication errors related to dosing were in the most vulnerable children, those <2 years of age or intensive care unit patients.³

Drug dosing accuracy is especially difficult in pediatrics because of many age- and size-specific considerations that add to the complexity of prescribing medications to this population. Pediatric drug dosing is more complex than adult dosing due to weight-based dosing, varying drug metabolism and physiology during development, and the increased off-label use of medications in children.^12–16 Online supplementary appendix 1 lists the more common factors that increase the complexity of prescribing medications to children.

EHRs with CPOE and effective CDS have shown the potential to reduce the risk of drug-related harm, including within the pediatric population.^{4 5 17 18 20–23} Alerts are one common form of CDS used to assist with dosing guidance in CPOE systems. However, the drug-dosing rules used in CDS to trigger alerts generally are not tailored for pediatric use.

We conducted a retrospective, quantitative, cross-sectional study to explore and characterize inaccuracies within a vendor-supplied set of dosing eRules compared to accepted dosing rules from traditional sources. The specific aims of the study were to determine match characteristics between vendor-based dosing eRules and dosing rules from traditional sources, and to determine which medications, age groups, and dosing parameter combinations most closely match traditional dosing guidelines. The Institutional Review Board of Cincinnati Children's Hospital Medical Center (CCHMC) deemed this research exempt from review.

Materials and methods

CCHMC is a 577-bed quaternary care medical center with over 1 million patient encounters per year. CCHMC has a fully-implemented EHR (Epic Systems Corporation, Verona, Wisconsin) with CPOE system that generates roughly 300 000 drug alerts per month. Fifty percent of those alerts are in the drug-dosing category. Medication order alerts are triggered by the CPOE system referencing eRules supplied by one of the industry-leading proprietary drug dosing database vendors.

Creation of the study corpus

All medication alerts and orders from the CCHMC clinical data repository (CDR) from June 1, 2011 through 31 December 2011 were obtained. During this period, there were 1 276 156 unique orders and 1 819 028 unique alerts. Inpatient orders comprised 63.3% of the corpus, with ambulatory orders comprising the remaining 36.7%. All prescribed orders and alerts were included, irrespective of clinical area, provider type, patient type, or location of administration, except as noted below. From these, three groups (the ‘medication groups’) were assembled: the most commonly prescribed medications (‘common medications’), the most frequently alerted medications (‘most alerted medications’), and the medications with the highest rates of alert override (‘medications with highest override rate’).

Orders for intravenous fluids and vaccines were excluded from consideration in the final corpus because the architecture of the CPOE system aggregates these orders into high-level groupings that do not permit deft analysis. Medications with less than 200 orders during the study period were also excluded to remove infrequently ordered medications from consideration.

The 10 most frequent medication formulations from each group were initially selected for study. Some medication formulations were found to belong to more than one group (eg, prednisolone 15 mg/5 mL oral solution in all three groups). Because of this duplication, unique medication formulations were added such that the final corpus had 30 unique medication formulations. By group, it contained the top 14 commonly prescribed medications, the top 13 most alerted medications, and the top 13 medications with most alert overrides (table 1).

Table 1.

Study corpus medication formulations by medication group

	Common medications	Most alerted medications	Medications with highest override rates
Inclusion rationale	Users will require/interact with dosing rules for these drugs frequently	Highly-alerted drugs may predict poor underlying dosing rules	High override rates may indicate poor dosing rules; alerts did not change prescribing behavior
Selection criteria	Highest counts of distinct drug orders over the study period	Drugs with the highest counts of dosing-related alerts	Drugs with the highest override rates (and ordered a minimum of 200 instances during the study)
1	Acetaminophen* 80 mg/0.8 mL PO suspension	Acetaminophen* 80 mg/0.8 mL PO suspension	Paricalcitol IV solution 5 mcg/mL
2	Fentanyl citrate* 0.05 mg/mL injectable solution	Oxymetazoline HCl* 0.05% solution	Ipratropium bromide* 0.02% in solution
3	Ibuprofen* 100 mg/5 mL PO suspension	Acetaminophen* 325 mg PO tablets	Prednisolone sodium phosphate* 15 mg/5 mL PO solution
4	Morphine sulfate 1 mg/mL injectable solution	Fentanyl citrate* 0.05 mg/mL injectable solution	Pentobarbital sodium injection 50 mg/mL
5	Acetaminophen* 325 mg PO tablets	Lidocaine cream 4%	Epoetin alfa 2000 unit/mL injection solution
6	Oxymetazoline HCl* 0.05% solution	Polyethylene glycol 3350 oral powder	Antihemo factor-vwf 1000–2000 unit IV solution
7	Albuterol sulfate (2.5 mg/3 mL) 0.083% Nebulization	Ondansetron HCl* 4 mg/2 mL injectable solution	Fluoxetine HCl 20 mg PO capsules
8	Albuterol sulfate HFA 108 μg/act in aerosol	Prednisolone sodium phosphate* 15 mg/5 mL PO solution	Oxybutynin chloride 5 mg/5 mL PO syrup
9	Ondansetron HCl* 4 mg/2 mL injectable solution	Ciprofloxacin HCl 0.3% ophthalmic solution	Infliximab IV injection 100 mg
10	Bupivacaine HCl 0.25% injectable solution	Ibuprofen* 100 mg/5 mL PO suspension	Meropenem IV for solution 500 mg
11	Ibuprofen* 200 mg PO tabs	Ibuprofen* 200 mg po tabs	Rocuronium bromide IV solution 10 mg/mL
12	Ciprofloxacin-dexamethasone 0.3–0.1% otic suspension	Silver nitrate-potassium nitrate applicator 75–25%	Aspirin chew tablet 81 mg
13	Prednisolone Sodium* Phosphate 15 mg/5 mL PO solution	Ipratropium bromide* 0.02% in solution	Metronidazole tablet 500 mg
14	Cefazolin 100/mL injectable solution

*Denotes medication belongs to more than one medication group.

IV, intravenous; PO, oral.

These 30 medication formulations were then expanded by both dosing parameters and age groups to generate medication/dosing parameter/age group triads. Five medication dosing parameters—total daily dose, single dose minimum, single dose maximum, daily frequency minimum, and daily frequency maximum—were combined with each medication formulation to create five new medication/dosing parameter dyads. Each dyad was further divided by five a priori age ranges, creating medication/dosing parameter/age group triads. The five age groups correlated to ages that are accepted as similar physiologically and developmentally: newborns (0–29 days), infants (30–364 days), toddlers/pre-school children (365–1824 days or 1–5 years), school-age children (1825–4379 days or 5–12 years), and adolescents (4380–6570 days or 12–18 years). This process resulted in 750 medication/dosing parameter/age group triads (a priori rules).

The eRules corresponding to each medication triad were then documented. At times, the age groups in the eRules did not match the a priori age ranges. When needed, additional triads were added to match the age specificity of the eRule. For instance, when the meropenem intravenous solution total daily dose eRule had dosing ranges of 0–7 days and 8–29 days, the dosing rule for the medication triad ‘meropenem total daily dose for newborns’ was replaced by two rules representing the two separate age groups in the eRule. Frequently, however, eRule dosing age ranges were large and spanned several of the a priori age ranges, for example a dosing rule that spanned 0–364 days and overlapped both newborn and infant ranges. Diagnosis-related dosing and rules modified based on creatinine clearance were not considered per se, although the widest possible dosing ranges from all rules available were recorded.

Creation of the gold standard

Five traditional and respected sources were selected for constructing the gold standard medication-dosing guidelines: Harriet Lane Handbook (19th edition),²⁴ PDR.net (Physician's Desk Reference),²⁵ Epocrates Online,²⁶ Micromedex 2.0,²⁷ and Lexi-Comp Online (CCHMC formulary).²⁸ For each medication triad, information from all traditional sources was aggregated into a gold standard rule by finding the most common doses and units among the traditional guidelines. For example, if acetaminophen dosing was 75 mg/kg/day from two sources and 90 mg/kg/day from three sources, then 90 mg/kg/day was selected as the gold standard. If the majority of the sources had no rule available but a corresponding eRule existed, then available rules from the traditional sources were used as the gold standard to encourage matching (bias towards matching).

Dose rule matching

Analysis of the eRules began by comparison of each uncustomized (not locally modified) eRule against its gold standard (see online supplementary appendix 2). eRules were deemed to either match or not match (mismatch). Matches were instances where the gold standard and the eRule had exactly the same values and units (‘rule match’), or where no rule existed for either (‘no rules available’). Mismatches occurred when dosing rule units (mg, mL, mg/kg, etc) were identical but values were not (‘value mismatch’; such as 10 vs 20 mg), when the units of the eRules and gold-standard dosing rules were not equivalent and comparisons could not be made (‘unit mismatch’; such as 10 mg vs 10 mg/kg), when a gold-standard rule could not be constructed but an eRule was present (‘eRule only’), or when no eRules were available but a gold standard rule existed (‘absent eRule’). Value mismatches were further subdivided by their tendency to over-alert or under-alert. For instance, a single dose maximum eRule of 10 mg/kg would over-alert if the gold standard rule was 20 mg/kg and orders between 10–20 mg/kg were placed.

Primary analysis

To evaluate the accuracy of eRules and gold-standard dosing rules in a clinically relevant manner, matching was first analyzed by investigating the quality of matching across the five a priori age ranges. The first step was to map the matched/mismatched corpus dosing rules to the a priori age ranges. Any generated corpus rules that had an overlapping age range with the a priori age range of interest was assigned to, and considered part of, the a priori age group. For example, cefazolin 100 mg/mL injectable solution had two rules that overlapped the newborn period (rule 1 for 0–7 days, rule 2 for 8–364 days). Both rules had to match exactly for the dosing rule to be considered a match across the newborn period. If either one or both rules did not match, the eRule for that age range was considered a mismatch. In effect, this procedure transformed the dosing rules created in earlier steps into rules that fit the a priori age ranges (figure 1). Descriptive statistics of matching were calculated for the corpus by age range, dosing parameter, combinations of medication groups and age range, as well as combinations of medication groups and dosing parameters.

Figure 1.

An example of matching electronic dosing rule (eRule) age ranges to a priori age ranges. In this example, eRule age ranges were mapped to a priori age ranges for the primary analysis. Both the 0–7 day and 8–29 day eRules mapped to the newborn age range, while the 30–4380 day rule mapped to the infant, toddler, and school-age ranges. Examples of primary and secondary analyses are shown. New, newborn; M, match of eRule and gold standard; NM, mismatch of eRule and gold standard.

Secondary analysis

For the secondary analysis, the rate of rule matching across the entire typical pediatric age range was analyzed, without considering the a priori age ranges used in the primary analysis. In this analysis, the proportion of the pediatric lifetime, that is, the first 6570 days (0–18 years) of life, that a given eRule matched the gold standard was calculated (figure 1). For example, if the rules for a medication only matched during the 0–364 days range (but all other age ranges did not match), the match percentage was 364/6570 days, or 5.5%. This analysis was performed to give another sense of how frequently a match between eRules and gold standard rules would occur on any given day of a patient's first 18 years of life, irrespective of a priori age ranges. Rate of matching across all corpus rules, and by dosing parameter and medication group, was calculated.

Gold standard agreement analysis

To evaluate the strength of the constructed gold standard rules, the level of agreement among the five sources was assessed. Evaluation of the agreement between sources on each of the rules, as well as comparisons of rules with eRule matches versus those without eRule matches, was performed.

Dose rounding logic

Some CPOE and eRule vendors include software logic that permits acceptance of a range of dosing around an eRule (often set to ±5% or 10% the value of the eRules) to mitigate over-alerting that may occur due to rounding of doses. Such logic, if enabled, could lead to increased matching rates between the constructed gold standard rule and the eRule. Analysis was performed in this study to evaluate the matching effect of a ±10% allowance.

Results

Primary analysis

A total of 750 pairs of a priori dosing rules were compared. Table 2 displays the aggregate number of matches and proportion of rules that matched for age range and dosing parameter. Each age range and dosing parameter grouping consisted of 150 rules after transformation of the original rules to fit the a priori age ranges. There was a similar proportion of matching across all five age categories (range: 52–57.3%; table 2). The range of matching percent when analyzed by dosing parameter was wider (46–70%; table 2). Of the dosing parameters, daily frequency minimum and daily frequency maximum had the best fit. The mean match of all a priori adjusted dosing rules was 55.1%.

Table 2.

Dosing rule match rates based on a priori age ranges, by age range and dosing parameter

	# Matches	# Rules	Match rate (%)
Age range
Newborns (0–29 days)	81	150	54.0
Infants (30–364 days)	78	150	52.0
Toddlers/pre-school (365–1824 days)	83	150	55.3
School-age children (1825–4379 days)	86	150	57.3
Adolescents (4380–6570 days)	85	150	56.7
Total	413	750	55.1
Dosing parameter
Total daily dose	71	150	47.3
Single dose minimum	69	150	46.0
Single dose maximum	70	150	46.7
Daily frequency minimum	105	150	70.0
Daily frequency maximum	98	150	65.3
Total	413	750	55.1

When analyzed by a combination of medication group and age range, common medications had a higher match rate (66.3%) than the most alerted medications (61.2%) or the medications with the highest override rates (46.5%; table 3). The medications with the highest alert override rates group also showed the largest range, from 32.3% to 55.4%. Newborn common medication dosing rules had the best fit of all age ranges (78.6%), but the worst fit in the medications with the highest override rates group (32.3%).

Table 3.

Dosing rule match rates based on a priori age ranges, by medication group and age range, and medication group and dosing parameter

	# Matches	# Rules	Match rate (%)
Medication group/age range
Common medications
Newborns (0–29 days)	55	70	78.6
Infants (30–364 days)	48	70	68.6
Toddlers/pre-school (365–1824 days)	43	70	61.4
School-age children (1825–4379 days)	44	70	62.9
Adolescents (4380–6570 days)	42	70	60.0
Total	232	350	66.3
Most alerted medications
Newborns (0–29 days)	40	65	61.5
Infants (30–364 days)	35	65	53.9
Toddlers/pre-school (365–1824 days)	38	65	58.5
School-age children (1825–4379 days)	43	65	66.2
Adolescents (4380–6570 days)	43	65	66.2
Total	199	325	61.2
Medications with highest override rates
Newborns (0–29 days)	21	65	32.3
Infants (30–364 days)	27	65	41.5
Toddlers/pre-school (365–1824 days)	36	65	55.4
School-age children (1825–4379 days)	33	65	50.8
Adolescents (4380–6570 days)	34	65	52.3
Total	151	325	46.5
Medication group/dosing parameter
Common medications
Total daily dose	45	70	64.3
Single dose minimum	32	70	45.7
Single dose maximum	40	70	57.1
Daily freq minimum	55	70	78.6
Daily freq maximum	60	70	85.7
Total	232	350	66.3
Most alerted medications
Total daily dose	32	65	49.2
Single dose minimum	36	65	55.4
Single dose maximum	37	65	56.9
Daily freq minimum	50	65	76.9
Daily freq maximum	44	65	67.7
Total	199	325	61.2
Medications with highest override rates
Total daily dose	24	65	36.9
Single dose minimum	31	65	47.7
Single dose maximum	23	65	35.4
Daily freq minimum	39	65	60.0
Daily freq maximum	34	65	52.3
Total	151	325	46.5

Table 3 displays the match rates for dosing rules when analyzed by medication group and dosing parameter. In each medication group, the frequency dosing parameters had a higher match rate than total daily dose, single dose minimum, or single dose maximum.

Secondary analysis

Dosing rule match rates were calculated for the number of days that eRules matched the gold standard rules over the course of a pediatric lifetime. As shown in table 4, the daily frequency parameters had higher match rates (82.1% and 69.1%) than total daily dose, single dose minimum, or single dose maximum rules (50.0%, 45.2%, and 49.4%, respectively). The match rates of dosing rules by medication group and dosing parameter are shown in table 4. The number of matching days for a common medication, when the rule pertained to total daily dose, was 70%. In other words, 70% of the time the total daily dosing eRule for one of these drugs will match the gold standard and the most common traditional dosing guidelines. This translates to 4600 days in the course of a patient's pediatric lifetime that an eRule would be accurate.

Table 4.

Dosing rule match over the pediatric lifetime (0–18 years) by dosing parameter and medication group and dosing parameter

	# Matching/total days	Match rate (%)
Dosing parameter
Total daily dose	3283/6570	50.0
Single dose minimum	2972/6570	45.2
Single dose maximum	3248/6570	49.4
Daily frequency minimum	5393/6570	82.1
Daily frequency maximum	4541/6570	69.1
Mean match rates of all dosing parameters		59.2
Medication group/dosing parameter
Common medications
Total daily dose	4600/6570	70.0
Single dose minimum	2460/6570	37.5
Single dose maximum	3600/6570	54.8
Daily frequency minimum	5184/6570	78.9
Daily frequency maximum	5482/6570	83.4
Mean match rate of common medications		64.9
Most alerted medications
Total daily dose	3176/6570	48.4
Single dose minimum	3002/6570	45.7
Single dose maximum	3568/6570	54.3
Daily frequency minimum	5039/6570	76.7
Daily frequency maximum	4600/6570	70.0
Mean match rate of most alerted medications		59.0
Medications with highest override rates
Total daily dose	2738/6570	41.7
Single dose minimum	3229/6570	49.2
Single dose maximum	2421/6570	36.9
Daily frequency minimum	4648/6570	70.8
Daily frequency maximum	3543/6570	53.9
Mean match rate of medications with most alert overrides		50.5

Again, the highest match rates were found in the daily frequency parameters (table 4). The data also show a higher match rate for the common medications (64.9%) than for the most alerted medication (59.0%) or the medications with highest override rates (50.5%).

Figure 2 shows the proportion of matching and the various causes of mismatching in this analysis. eRule value mismatches that would lead to under-alerting providers to incorrect dosing comprised the largest portion of the mismatches (15.2%), followed by value mismatches that would lead to over-alerting (14.1%). Units of measure that could not be compared (eg, mg vs mg/kg) and scenarios lacking either an eRule or gold standard rule caused many fewer mismatches.

Figure 2.

The proportion of eRules that matched and mismatched the gold standard rules over the course of the pediatric lifetime (0–18 years). A pie chart demonstrating the proportion of matches, as well as mismatches by underlying etiology for the mismatch, for the secondary analysis results.

Gold standard analysis

Figure 3 displays the level of agreement between the gold standard references. A majority consensus (≥3 out of 5 gold standard sources agreed) existed in 76.2% of the rules in the study corpus.

Figure 3.

The agreement level of the gold standard reference rules. Bar graph representation of the amount of traditional dosing source rules that agreed with each other. The x-axis represents the level of agreement among the five gold standard sources while the y-axis represents the percentage of constructed gold standard (GS) rules that fit into that level of agreement. Analysis is subdivided by matching and non-matching GS eRule categories.

Dose rounding logic

Adjustment of the vendor eRules for a ±10% allowance to account for dose rounding logic only allowed one additional rule to match (<0.1% of the rule corpus).

Discussion

This study's primary aim was to evaluate how well one vendor-supplied set of dosing eRules matched common and accepted pediatric prescribing rules. The most notable finding from the study data was the clinically significant inaccuracy in the eRules in all groups. The aggregate match rate was 55.1% for all a priori dosing rules in the primary analysis, indicating that only about 1 out of every 2 eRules exactly matched the dosing rules in the gold standard traditional pediatric dosing rule sources. The secondary analysis of the match rates, without considering age, confirmed these findings, showing that only 57.6% of all pediatric days covered by the eRules were accurate. Low rates like these must be a strong contributor to the high number of false alerts reported in the literature.

The accuracy of eRules did not drastically change when evaluated by age group. Results for newborns, generally considered the most vulnerable age group, were similar to those of all other age groups although the newborn age range eRules accuracy did exhibit greater variability than other age groups in some parameters. Both the best (78.6%, common medications group) and the worst matching rate (32.3% medications with the highest override rates group) for all age ranges occurred in the newborn eRules. We believe several factors are responsible for these observations. Thirty-seven of the 55 newborn rule matches in the common medication group matched because no rules were available in either the eRules or the gold standard. The matching rate would be considerably lower (54.5%) if this subset of ‘matches’ were removed from the calculation. The opposite is true for the medications with the highest override rates group. Only 12 newborn rules in that group matched because no rules exist in either source. From a clinical perspective, it should be noted that most of the group of medications with the highest override rates are not commonly used in neonates, which likely limits the clinical impact of the high mismatch rates. However, other age ranges, where use of the medications is more prevalent, also demonstrated low matching rates for some groups. In these age ranges, low rates are of more clinical significance since they potentially increase the number of false dosing alerts.

Examination of the accuracy of eRules by dosing parameter showed that the daily frequency minimum and maximum eRules were consistently more accurate than total daily dose, single dose minimum, or single dose maximum eRules. This trend persisted when analysis was performed on medication groups and in both the primary and secondary analyses, likely due to the fact that dosing frequencies for many of the medications in the study corpus fall with a limited range, typically 1–4 doses per day. Other dosing group values are often much more variable, thereby increasing the chances of a mismatch.

Examination of the accuracy of eRules by medication group shows that the common medications were more like the gold standard rules than the most alerted medications rules and were much more accurate than the eRules from the medications with the highest override rates group. Explanations for this trend include that common medications are more likely to have formal established dosing rules than the other two medication subgroups. In addition, the medications with the highest override rates tended to be more specialized medications, compared to other groups. This group only required a relatively small number of orders (>200), thereby allowing for inclusion of less frequently prescribed, more-specialized medications. More-specialized medications are likely to have less formal or established dosing rules, which would lead to lower match rates and accuracy of the eRules for this subset.

Construction of the gold standard rules proved to be a difficult task. The analysis of the traditional sources used to create the gold standard reveals a relative heterogeneity of rules. While this may ‘tarnish’ the gold standard a bit, it also reflects the challenging task vendors have to create eRule databases that fit with real-world provider prescribing patterns while not putting patient safety at risk. However, over three-quarters of the gold standard rules created were the product of a majority consensus among the five sources. In cases where there was no clear majority, we made every reasonable attempt to select a gold standard rule that would match with the partner eRule. In short, we actually provided a bias towards matching, yet the study still demonstrated low matching rates.

The allowance for dose rounding logic, which increased the dosing parameter thresholds by ±10%, would theoretically improve matching rates as it expands the acceptable dosing ranges set by the vendor eRules. In this study, however, the matching rates were not appreciably improved by including this allowance, converting only one non-matching gold standard–eRule pair into a matching pair.

Regardless of approach, accuracy discrepancies of the magnitude found in this study highlight several important aspects of both pediatric dosing and use of vendor-supplied eRules to provide CDS for prescribing pediatric medications. First, these findings highlight that the eRules supplied by commercial vendors may not accurately reflect how pediatric providers are prescribing medications in everyday practice. It is possible that vendors of dosing eRules, for a variety of reasons, include only strictly accepted dosing guidelines, such as FDA-approved values, in their products while many medications are used for off-label indications and prior to rigorous pharmacologic study in pediatric populations.¹⁶ If this is true, then eRules should be modified by the institutions implementing them.

Second, the findings of the analysis underscore previously published literature that describes the inherent difficulties in dosing medications in children.^{3 5–9 17 29–31} Unique requirements such as weight-based dosing and other factors increase the heterogeneity of dosing rules seemingly exponentially, making it more difficult for vendors to create products that offer optimal pediatric CDS.

Finally, while the data presented in this report expose a deficiency in medication dosing support in pediatrics, the discrepancies discovered should be viewed as opportunities to improve current systems. Pediatric-specific rules and products should be designed and developed with the issues noted here in mind.

Strengths

Strengths of this study include that the study corpus was based on a large number of medication orders and alerts over 7 months, minimizing the potential for a sampling bias that may have resulted from data collected in a shorter time period. The medication orders and alerts were also derived from multiple practice settings, including inpatient units, outpatient/ambulatory settings, and the emergency department. Inclusion of orders from this variety of settings improves the generalizability of the results. Fourteen types of medication formulations (eg, injectables, suspensions, tablets, creams) were also included in the corpus medications, representing a wide array of formulations. Last, exceptions were made in the study design to encourage dose rule matching when feasible and logical. By doing so, the study attempted to err in a direction that would improve matching rates.

Limitations

The findings from this study were, however, based on data from only one institution and from interrogation of one vendor product. There is also a risk of selection bias mediated through the inclusion criteria chosen for the medication subgroups, with two of the subgroups chosen explicitly because of their potential to generate false alerts. It could be expected that the match rates for dosing rules from these groups would be lower than for commonly prescribed medications. However, those groups are responsible for a large proportion of the false alerts in CCHMC's system and represent targets for future dosing rule improvement. Including those medication groups highlights the disparities that exist and the need for rule reconfiguration.

Finally, intravenous fluids and vaccine dosing rules could not be evaluated due to the configuration of CCHMC's CPOE system.

Future studies

There are ample opportunities to further the work presented in this report. Repetition of the protocol in other healthcare organizations, with eRules from other vendors, using different medication groups, including intravenous fluids and/or vaccines, and examining matching rates based on ordering locations all might yield more information. Unless significant differences from this study are found, however, the next important work must be to determine whether or not quality of clinical care is truly affected by differences in dosing eRules and common ordering practice. Alert fatigue is postulated but not proven in this study, and, if truly present, the magnitude of its effect on clinical care is unknown.

Conclusions

This study evaluated the accuracy of medication dosing rules from a vendor-supplied set of eRules. When compared to a gold standard comprised of rules from traditional dosing sources, low rates of matching were found. The newborn age range was found to have the most variability in matching, demonstrating both high and low rates of congruence in different types of medications. Dosing parameters concerning daily frequencies (minimum and maximum doses per day) consistently matched at higher rates than total daily dose and single dose (minimum and maximum) parameters. Low levels of matching promote false alerts in CPOE systems and may compromise the utility of CDS. This study gives further evidence of the mismatch between eRules and traditional sources, which must be taken into account when CPOE systems are implemented in the pediatric setting. More evaluation of vendor-supplied eRules is required and identified deficiencies need to be addressed if this common form of CDS is intended to minimize prescribing errors and prevent adverse drug events.