Retrospective Evaluation of a Computerized Physician Order Entry Adaptation to Prevent Prescribing Errors in a Pediatric Emergency Department

Abstract

Objective

The goal was to determine the impact on medication prescribing errors of adding a pediatric medication list (quicklist) to a computerized physician order entry system in a pediatric emergency department.

Methods

The quicklist is a drug dosing support tool that targets the most common medications in our clinical setting. We performed a retrospective comparison of orders from 420 randomly selected visits before and after quicklist introduction. Error rates were analyzed with respect to urgency level, physician training level, and patient age. The quicklist was examined for frequency of use and error rates.

Results

The 840 patient visits (420 before intervention and 420 after intervention) generated 724 medication orders, which contained 156 medication prescribing errors (21%). The groups did not differ with respect to urgency level, physician training level, or patient age. There were significant decreases in the rate of errors per 100 visits, from 24 to 13 errors per 100 visits, and in the rate of errors per 100 orders, from 31 to 14 errors per 100 orders. The decrease in the error rates did not vary according to urgency score, age group, or physician training level. The quicklist was used in 30% of the orders in the postintervention group. In this group, the error rate was 1.9 errors per 100 orders when the quicklist was used, compared with 18.3 errors per 100 orders when the list was not used. Errors of wrong formulation, allergy, drug-drug interaction, and rule violations were eliminated.

Conclusion

The introduction of the quicklist was followed by a significant reduction in medication prescribing errors. A list with dosing support for commonly used pediatric medications may help adapt computerized physician order entry systems designed for adults to serve pediatric populations more effectively.

Medication errors are a constant challenge for providers and have potentially serious consequences for patients. Pediatric populations are at unique and significant risks with respect to medical errors.¹ The use of weight-based dosing, off-label drug usage, limited reserves to withstand dosing errors, and inability to communicate with health care personnel to prevent an error or to signal that one has occurred are contributing factors.^2,3 Children have significant pharmacokinetic and pharmacodynamic differences, compared with adults, which make them more susceptible to medication errors. Small calculation errors may translate into large complications, such as a decimal error causing a 10-fold dose increase.⁴ The emergency department (ED) environment presents additional risk because of the emergency nature of the care provided, high patient volume, stress, noise, time pressures, and unfamiliar patients.¹ Therefore, the pediatric ED is at particularly high risk for medication errors. It is estimated that prescribing errors occur in 10% of visits to a pediatric ED.⁵ To date, no studies exist regarding the impact of computerized physician order entry (CPOE) on medication errors in a pediatric ED.

Most adverse drug reactions and medication errors occur at the time of physician prescribing.^6,7 In a pediatric inpatient setting, 74% of medication errors and 79% of adverse drug reactions occurred at the stage of physician ordering, and the most frequent type of medication error was a dosing error.^6,7 CPOE has been shown to reduce rates of medication prescribing errors for hospitalized children.^8,9 However, recent studies have shown little or no effect of some CPOE systems in preventing errors in pediatric populations, and new, computer-related errors sometimes emerge.¹⁰ Despite these shortcomings, CPOE has been supported by the Institute of Medicine, the Leapfrog Group, the Institute for Safe Medication Practices, the American Medical Association, the American Academy of Pediatrics, and others as a safety practice to implement.^11–18

CPOE used in conjunction with decision support has been shown to reduce prescribing errors for hospitalized adults.^13,14,19 Examples of decision support include suggested dosing regimens with age-specific doses, drug-drug interaction warnings, and allergy warnings. A CPOE system with targeted decision support (eg, calculation of creatinine clearance for renal dosing of antibiotics) was shown to be effective for adult patients with renal insufficiency.²⁰ In a study involving 1933 children from 3 health maintenance organizations, electronic prescription ordering without decision support did not lower potential medication dosing error rates for outpatient prescriptions,²¹ which suggests that targeted decision support may be necessary to prevent dosing errors for children. CPOE systems with pediatrics-specific clinical decision support have been suggested as a way to prevent potentially harmful errors for pediatric inpatients.⁷

In an effort to reduce errors, we designed a medication quicklist that provides decision support by supplying pediatric, weight-based doses of formulary-approved drugs for the most commonly prescribed medications in our pediatric ED. The purpose of this study was to determine whether the addition of a medication quicklist to our CPOE system reduced the rate of medication prescribing errors in our pediatric ED.

Methods

Study Design and Study Setting

This was a retrospective cohort study comparing patient visits before and after implementation of a CPOE medication quicklist in an academic, urban, pediatric ED. Our center is a level 1 trauma center, with a physically separate pediatric ED serving an annual volume of ∼30 000 patients. Our staff consists of 7 attending physicians, 4 pediatric emergency medicine fellows, and ∼130 pediatric residents. This study was approved by the institutional review board of Boston University Medical School.

In October 2005, our hospital instituted IBEX (Picis, Wakefield, MA), an Internet browser-based ED information system that integrates patient tracking, physician and nursing documentation, risk management, charge management, prescription writing, discharge instructions, and laboratory, radiology, and medication order entry. IBEX recognizes drug allergies and drug-drug interactions and has order sets for common pathways (eg, pediatric fever and asthma). All IBEX users underwent a required 2-hour training session before being given a password.

We estimated that 420 charts were required in both the preintervention and postintervention groups, on the basis of a significance level of .05 with 80% power to detect a 50% difference in medication error rates before and after the quicklist. A background error rate of 10% before intervention and an average of 1 medication order per chart were assumed.⁵

Quicklist

The medication quicklist was added to IBEX and distributed to faculty members and residents in December 2005. Before distribution of the list, physicians chose medications from a master list of drugs searched for by keyword. Drugs that include the keyword or are closely related appear on the dropdown list, including preparations of medications that do not necessarily appear in our formulary and may not be available in our pharmacy. When selected from the list, the drug name appears in an ordering window with blank fields for dose, unit, route, and frequency. Suggested doses and routes are not available when the search option is used. The dose field requires a numeric entry, and the unit, route, and frequency fields are text boxes. Dropdown lists for all possible units and routes are available. An alternative option is to use an order set that includes commonly used medications for certain situations (eg, morphine sulfate and ketorolac for sickle cell pain crisis). The order sets can default in doses, units, and routes where appropriate (standard adult doses) but do not allow for pediatrics-specific dosing support. After implementation of the quicklist, physicians could choose to use the standard search engine, the order sets for certain situations, or the pediatric quicklist. The system defaults to the most recently used method.

The quicklist was derived by 3 pediatric emergency physicians from our pediatric ED and was checked for accuracy by 2 hospital pediatric pharmacists. The quick-list contains the 75 most commonly prescribed medications in our practice and is hospital formulary compliant. It provides physicians with an alphabetized list of drug names with preparations and concentrations that are stocked in the ED or available in the pharmacy. For example, one option for acetaminophen is listed as follows: “acetaminophen; liquid; 160 mg/5 mL.” When a medication is chosen, the list prompts a suggested weight-based dose, unit, route, and frequency, where appropriate. In the case of the above example, the screen would read, “PO; 15 mg/kg.” The quicklist also has situational information for dosing certain drugs, as for ceftriaxone, “IV/IM; 50 mg/kg or 100 mg/kg for meningitis, maximum 2 g.” The dose, unit, route, and frequency may be overridden by the physician. The patient's weight and allergies are listed on the same screen. The system does not perform a weight-based calculation for the physician. The system contains drug allergy and interaction alerts.

In the postintervention group, to determine whether the quicklist was used, we reviewed the orders for decision support information in a text field called “Notes.” If this field was populated with information (eg, “concentration = 160 mg/5 mL [dose = 15 mg/kg]”), then the medication was known to be derived from the quicklist.

Survey

A 2-question, paper-based survey was given to all pediatric attending physicians and residents who staff the pediatric ED, to determine how often they use the quicklist. The question was, “Do you use the quicklist for medication order entry in the pediatric ED, and if so, how often (all of the time, most of the time, sometimes, rarely, or not at all)?” The survey was circulated at departmental meetings by our chief residents, and physician responses were anonymous.

Inclusion Criteria

A total of 420 charts from randomly chosen days in autumn 2005, 2 months after the introduction of the CPOE system, were reviewed and compared with 420 charts from 1 year later, representing periods before and after implementation of the quicklist. A patient might have been included more than once if they had separate visits within the randomized dates selected. Patient visits were excluded if the patient registered but left the department without being seen by a physician.

Definitions

A medication prescribing error was defined, as in previous research, as any event involving an order that was incomplete, incorrect, or inappropriate.⁸ An example of an incomplete order might be one in which the prescriber left out the route or amount of medication. An incorrect order might involve the wrong dose for the patient's weight, the wrong drug, or the wrong unit for the drug. An inappropriate order might be an instance in which the prescribing physician failed to account for patient-specific information, such as a medication allergy. The definition of an event being a medication prescribing error is not dependent on whether the error did or did not cause harm to a patient. Errors include rule violations where an order is not in compliance with standard hospital practice (for example, using abbreviations or trailing zeros).

Medication prescribing errors were categorized according to error types, that is, wrong dose, wrong unit, wrong frequency, wrong route, wrong drug, allergy, drug interaction, missing information (eg, weight), or rule violation (trailing zeroes or abbreviations). Errors not fitting into one of these categories were designated “other” and included computer-related errors (eg, duplicate orders). A single medication order could have >1 error type.

Detection of Medication Errors

Two medical students underwent a 2-hour training session in medical error detection in which they were trained to recognize error types. Details about errors detected were recorded on a standardized abstraction form. Other data abstracted included patient age, weight, and acuity (as categorized by the triage nurse), type and number of medications prescribed during the visit, and level of training of the prescribing physician. Data were deidentified on entry into the database. One pediatric ED physician reviewed all orders in the study and verified error categorization. A second physician reviewer performed an analysis of 18% of the orders. The consistency between physician raters was excellent, with disagreement in only 2 cases (0.01%).

Statistical Analyses

Demographic characteristics of patient visits before and after introduction of the quicklist were compared according to patient age (0–2, 2–9, 9–14, or 14–21 years), urgency score (high or low), and physician training level (attending physician or resident) by using Fisher's exact test. Patient urgency scores were categorized as high if designated level 1 (emergency) or level 2 (urgent) and as low if designated level 3 (nonurgent) or level 4 (express care). Physician training level was defined as attending physician if a fellow or attending physician-level faculty member prescribed the order and as resident if a resident prescribed the order.

Error rates per 100 patient visits were calculated by dividing the number of errors by the number of ED visits and multiplying the result by 100. A similar calculation was performed for medication orders. We compared error rates before and after implementation of the quicklist by using incidence rate ratios (IRRs), which were measured by using simple Poisson regression analysis. The effects of age, urgency level, and physician training level on error rates were evaluated by using stratified analyses.

To account for clustering according to patient visit, because some patients might have been sampled more than once, we considered models that account for overdispersion. The results were similar and are not reported here.

Results

Demographic Characteristics

A total of 840 patient visits were analyzed during the study period, representing 724 medication orders, 326 before intervention and 398 after intervention. Of the 420 visits before the intervention, 180 visits (43%) had ≥ 1 medication order; of the 420 visits after the intervention, 192 visits (46%) had ≥1 medication order (P = .45). The 2 groups were similar with respect to age distribution, urgency scores, and physician training level (Table 1). The postintervention group had a significantly higher rate of orders per 100 visits (IRR: 1.22; 95% confidence interval [CI]: 1.05–1.41).

Table 1. Characteristics of the Study Population Before and After Pediatric Quicklist Intervention.

	n (%)		P
	Before Quicklist	After Quicklist
Total visits	420	420
Total orders	326	398
Visits with ≥1 order	180 (43)	192 (46)	.45
Visits according to urgency level			.81
High	102 (24)	105 (25)
Low	318 (76)	315 (75)
Visits according to age			.64
0–2 y	64 (15)	75 (18)
2–9 y	112 (27)	117 (28)
9–14 y	49 (12)	49 (11)
14–21 y	195 (46)	179 (43)
Orders according to physician training level			.21
Attending physician	62 (19)	89 (22)
Resident	264 (81)	309 (78)

Error Rates

There were a total of 156 errors identified in the 724 medication orders reviewed in the study (21 errors per 100 orders). Overall, we found significant decreases in the rate of errors per 100 visits, from 24 errors per 100 visits before intervention to 13 errors per 100 visits after intervention (IRR: 0.54; 95% CI: 0.39–0.76), and in the rate of errors per 100 orders, from 31 errors per 100 orders before intervention to 14 errors per 100 orders after intervention (IRR: 0.45; 95% CI: 0.32–0.62) (Table 2). The decrease in the error rates per 100 visits did not vary according to either urgency score or age group. The adjusted IRR was 0.54 (95% CI: 0.39– 0.74). The decrease in the error rates per 100 orders did not vary according to either urgency score or age group. The adjusted IRR was 0.43 (95% CI: 0.31–0.60).

Table 2. Changes in Error Incidence Rates Before and After Pediatric Quicklist Intervention.

	No. of Errors		IRR (95% CI)	P
	Before Quicklist	After Quicklist
Total errors	101	55
Errors per 100 visits	24	13	0.54 (0.39–0.76)	.0003
Errors per 100 orders	31	14	0.45 (0.32–0.62)	<.001
Errors per 100 orders not from quicklist	31	18.28	0.58 (0.42–0.81)	.002
Errors per 100 orders from quicklist	Not applicable	1.87

Error type rates generally were reduced after intervention except for the other category, which increased from 1 to 4 errors, and wrong unit, which increased from 1 to 2 errors; none of the increases was statistically significant. The decrease in error rates could be traced to decreases in wrong dose, wrong frequency, and wrong route error types. Wrong drug error rates also decreased but not statistically significantly. Wrong formulation, allergy, drug-drug interaction, and rule violation errors were eliminated. Wrong unit and missing information error rates did not change appreciably (Table 3).

Table 3. Analysis of Error Types Before and After Pediatric Quicklist Intervention.

Error Type	No. of Errors (No. per 100 Orders)		IRR (95% CI)	P
	Before Quicklist	After Quicklist
Wrong dose	27 (8)	19 (5)a	0.58 (0.32–1.04)	.07
Wrong unit	1 (0.3)	2 (0.5)	1.64 (0.15–18.07)	.69
Wrong frequency	12 (3.7)	1 (0.3)	0.07 (0.01–0.53)	.01
Wrong route	8 (2.4)	2 (0.5)	0.20 (0.04–0.96)	.04
Wrong drug	6 (2)	2 (0.5)	0.27 (0.06–1.35)	.11
Wrong formulation	9 (2.8)	0 (0)		.03
Allergy	2 (0.6)	0 (0)		.32
Drug-drug interaction	1 (0.3)	0 (0)		.61
Rule violation	6 (1.8)	0 (0)		.07
Missing information	28 (9)	25 (6)a	0.73 (0.43–1.25)	.25
Other	1 (0.3)	4 (1)	3.78 (0.36–29.31)	.29

^aOne error from each category occurred when the quicklist was used.

Postintervention Quicklist Use

Ninety-five percent and 91% of medication orders in the preintervention and postintervention groups, respectively, were for medications that exist on the quicklist. The quicklist was confirmed to be used for 30% of the orders (107 of 361 orders) for medications included on the list. There were 2 errors found for medications ordered using the quicklist, namely, wrong dose (dose of ceftriaxone changed from 1800 mg to 1000 mg) and missing information (units missing on an order for albuterol). The rest of the errors in the postintervention group were for medications ordered not using the quicklist and are detailed in Table 3. The error rate when the quicklist was used was 1.87 errors per 100 orders, whereas the error rate when the list was not used was 18.28 errors per 100 orders (IRR: 0.10; 95% CI: 0.02– 0.42) (Table 2).

Training Level of Ordering Physician

Residents ordered 79% of the medications in the study period, with similar proportions in the 2 groups (Table 1). Attending physicians and residents had approximately equal rates of errors per 100 orders (20 and 21 errors per 100 orders, respectively; IRR: 0.94; 95% CI: 0.63–1.40). The change in rates of errors per 100 orders before and after intervention for attending physicians and residents was the same as that for the group as a whole (IRR: 0.45; 95% CI: 0.32–0.62).

Survey

All 11 fellows and attending physicians who prescribed medications during the study period were surveyed. Seven reported using the list all or most of the time, whereas 4 reported using it sometimes or rarely. None reported using it none of the time. Fifty-nine of 132 residents responded to the survey. Four reported using the list all of the time, 13 most of the time, 23 sometimes, 10 rarely, and 20 none of the time. Attending physicians reported using the list more often than residents. In the study, attending physicians accounted for only 21% of total orders but 38% of orders when the quicklist was used.

Discussion

Data are limited on the impact of CPOE on medication errors in the pediatric population, especially in the emergency setting, where pediatric patients are at particularly high risk. Although CPOE reduces medication errors, implementation of these systems can pose challenges. We found an overall reduction in medication prescribing errors of 55% after adapting our pediatric emergency CPOE system by introducing the quicklist. More importantly, the error rate was 10-fold less when medications were ordered by using the quicklist. The list performed equally well across all ages and urgency scores and when used by residents or attending physicians. Our quicklist provides decision support that pertains to the most commonly used drugs in our clinical setting (93% of orders in the study were for drugs on the quicklist), is formulary compliant, has been checked for accuracy by pediatric pharmacists, and is easily accessible within the system. The intervention of a pediatric decision support tool such as the quicklist can help adapt CPOE systems designed for adults to meet the needs of children and adolescents.

The use of the quicklist in this study was relatively low. We were disappointed to find in the survey that only two thirds of the attending physicians in the group used the quicklist most or all of the time and only one half of the residents surveyed used the list sometimes or most of the time. In the postintervention group, only 30% of medications were ordered by using the quicklist. The impact of the quicklist in this subgroup was impressive, with a 10-fold reduction in error rates. Only 2 errors occurred with use of the quicklist. Because a physician can manually override the preset information on the quicklist and enter medications in a free-text box, certain error types are still possible. Studying free-text entry may be an area to pursue to reduce errors made when the quicklist is used. Other ways to improve the use of the quicklist include education detailing medication ordering for physicians and making the quicklist the system default for all pediatric patients.

A limitation of our system was that, throughout the study period, physicians were free to choose from the quicklist, to use order sets, or to search from a master list of drugs by keyword when ordering medications. Physicians tended to select a method by virtue of habit. The relatively limited use of the list might have been attributable to a system upgrade that rendered the quicklist unusable for a few months before the postintervention study period. The list was repaired and redistributed, but many doctors had grown accustomed to using other ordering methods by the time the quicklist was ready to be tested. The delay caused by the system upgrade illustrates the important point that CPOE systems are constantly in flux, subject to manufacturer modifications, local information technology department customizations, and internal changes in hospital formulations.

A second limitation was that error rates in the study decreased from the preintervention group to the postintervention group irrespective of the use of the quicklist. Although the quicklist may be responsible for the majority of our overall improvement in error rates, the improvement may not be solely attributable to the quicklist, because the background rate of errors in our ED seems to be decreasing. We expect that some of the errors were reduced by the physicians learning over time to use the system more effectively. In the postintervention group, however, where the rate of errors when the quicklist was used was 10 times less than that when the quicklist was not used, the declining background rate of errors would have no effect.

A final limitation of our study was that we analyzed only errors that occurred at the time of order entry. It is not known how many of those errors resulted in harm to the patient. We did not investigate whether the errors were detected by other components of the medication-dispensing system (eg, pharmacy or nursing). This approach is similar to other studies that chose to focus on the most common form of medical errors, medication prescribing errors.⁸ CPOE is a medication ordering system, and changes to a CPOE system would be expected to have modest effects on rates of dispensing or administration errors, compared with ordering errors.

Pediatric decision support tools, such as our quicklist, may help adapt CPOE systems designed for adults to address pediatric medication ordering needs more effectively. Pediatric medication ordering requires weight-based dosing calculations, pediatric dosing limits, and the use of liquid formulations.^6,16 Previous research demonstrated that CPOE systems designed for adult populations may not be effective for pediatric patients and may cause new errors to emerge.^10,21 Our pediatric decision support tool significantly reduced medication prescribing errors in our ED, compared with the CPOE system without pediatric decision support.

What's Known on This Subject.

Computerized order entry systems can reduce medication errors. Most studies have investigated systems designed for adult patients.

What This Study Adds.

This study provides a tool to adapt a computerized physician order entry system to help reduce errors for pediatric ED patients.

Abbreviations

CPOE

computerized physician order entry

IRR

incident rate ratio

ED

emergency department

CI

confidence interval