The impact of computerized provider order entry on medication errors in a multispecialty group practice

Abstract

Objective

Computerized provider order entry (CPOE) has been shown to improve patient safety by reducing medication errors and subsequent adverse drug events (ADEs). Studies demonstrating these benefits have been conducted primarily in the inpatient setting, with fewer in the ambulatory setting. The objective was to evaluate the effect of a basic, ambulatory CPOE system on medication errors and associated ADEs.

Design

This quasiexperimental, pretest–post-test study was conducted in a community-based, multispecialty health system not affiliated with an academic medical center. The intervention was a basic CPOE system with limited clinical decision support capabilities.

Measurement

Comparison of prescriptions written before (n=5016 handwritten) to after (n=5153 electronically prescribed) implementation of the CPOE system. The primary outcome was the occurrence of error(s); secondary outcomes were types and severity of errors.

Results

Frequency of errors declined from 18.2% to 8.2%—a reduction in adjusted odds of 70% (OR: 0.30; 95% CI 0.23 to 0.40). The largest reductions were seen in adjusted odds of errors of illegibility (97%), use of inappropriate abbreviations (94%) and missing information (85%). There was a 57% reduction in adjusted odds of errors that did not cause harm (potential ADEs) (OR 0.43; 95% CI 0.38 to 0.49). The reduction in the number of errors that caused harm (preventable ADEs) was not statistically significant, perhaps due to few errors in this category.

Conclusions

A basic CPOE system in a community setting was associated with a significant reduction in medication errors of most types and severity levels.

The Institute of Medicine (IOM) has long been a proponent of electronic health records (EHRs),^{1 2 3 4} making the link between EHR implementation and potential improvements in quality and safety. The vision statement of the National Alliance for Primary Care Informatics endorses the idea that for primary care providers to provide citizens with quality, affordable healthcare, they must be equipped with a fully functional EHR at the point of care.⁵ Research evaluating the impact of EHRs on medication safety has focused largely on evaluation of computerized provider order entry (CPOE) systems, with clinical decision support (CDS) alerts to guide ordering. Several systematic reviews have summarized the benefits of CPOE/CDS systems.^{6 7 8 9 10 11} Collectively, these studies have been conducted primarily in the inpatient settings of academic medical centers, using homegrown systems; most have included CDS alerts to guide ordering. Few have been conducted in the ambulatory setting. Our study addresses some of these gaps, in that we evaluate the impact of implementation of a homegrown, basic CPOE system on medication safety in the ambulatory, community-based setting of a multispecialty, independent medical group.

CPOE systems are computer applications that allow direct, electronic entry of orders for medications, laboratory, radiology, referral, and procedures.¹² CPOE systems for ordering medications are sometimes called electronic prescribing (e-prescribing) systems. CPOE systems are often implemented with clinical decision support (CDS) alerts to guide ordering.¹³ Early research demonstrated the benefits of CPOE/CDS systems in reducing medication errors by as much as 55–86%^{14 15} and subsequent adverse drug events (ADEs), although the latter occur less frequently and are more difficult to identify.^{16 17}

Evidence from several systematic reviews has shown the benefits of CPOE/CDS systems on medication safety.^{6 7 8 9 10 11} The studies included in these reviews overlap, and vary widely in design and results; few include randomized, controlled trials. The heterogeneity of the studies prevented all but one author from conducting a meta-analysis, but these authors found CPOE/CDS systems were associated with a 66% reduction in the odds of an error occurring (OR 0.34; 95% CI 0.22 to 0.52).¹¹ The authors of the most recent review suggest that additional studies of better quality are needed to cover a wide range of clinical and geographic settings, specifically in the ambulatory care setting.¹¹

Six of the studies in these reviews were conducted in the ambulatory setting, and results were mixed.^{18 19 20 21 22 23} Five focused on CDS alerts.^{18 19 20 22 23} The sixth explored the effect of a CPOE system on all types of errors, but results revealed that reductions in errors and preventable ADEs were not significant.²¹ These studies were conducted in academic settings and used homegrown systems.^{18 19 20 21} Thus, generalizability is limited, as most US healthcare is delivered in community-based settings. Further, sample sizes in these studies were small—for example, fewer than 400 CDS events per study arm,¹⁸ fewer than 2.8% of prescriptions electronically prescribed,¹⁹ or 1879 prescriptions reviewed.²¹ Further research is warranted, as investigators estimate that between 11%²⁴ and 28%²⁵ of ADEs that occur in the ambulatory setting are preventable.

We evaluated the effect of a basic CPOE system in the ambulatory setting on medication errors and subsequent ADEs, employing the definition of a medication error and the severity index of the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP)^{26 27} (table 1). We compared pre- to postimplementation errors. We conducted our study at a physician-owned, multispecialty clinic-system in Everett, Washington, located 25 miles north of Seattle. The clinic system is community-based, not affiliated with an academic medical center. It is the largest independent medical group in the State of Washington. In this real-world setting, we evaluated the impact of implementation of a basic CPOE system, with limited CDS alerts, on medication safety. We hypothesized that the CPOE system would be associated with a reduction in errors of all severity levels, and that the types of errors reduced would be those most affected by a basic system (eg, missing information).

Table 1.

Definition of a medication error, error severity index

NCC MERP category	Description of NCCMERP category	ADE category
No error
A	Circumstances or events that have the capacity to cause error
Error, no harm
B	An error occurred, but the medication did not reach the patient	Intercepted potential ADE
C	An error occurred that reached the patient but did not cause patient harm	Non-intercepted potential ADE
D	An error occurred that resulted in the need for increased patient monitoring but no patient harm	Non-intercepted potential ADE
Error, harm
E	An error occurred that resulted in the need for treatment or intervention and caused temporary patient harm	Preventable ADE
F	An error occurred that resulted in initial or prolonged hospitalization and caused temporary patient harm	Preventable ADE
G	An error occurred that resulted in permanent patient harm	Preventable ADE
H	An error occurred that resulted in a near-death event (eg, anaphylaxis, cardiac arrest)	Preventable ADE
Error, death
I	An error occurred that resulted in patient death	Preventable ADE

National Coordinating Council on Medication Error Reporting and Prevention (NCC MERP), definition of a medication error^{26 27}: “A medication error is any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the healthcare professional, patient, or consumer. Such events may be related to professional practice, healthcare products, procedures, and systems, including prescribing*; order communication; product labeling, packaging, and nomenclature; compounding; dispensing; distribution; administration; education; monitoring; and use.” *Our study was limited to prescribing errors.

NCC MERP Severity Index²⁷ and ADE Category Schema⁴⁵.

Methods

Setting

The clinic cares for 250 000 patients, at 14 locations, in 60 clinics. Clinicians admit to the one hospital in the local market. The clinic logs 650 000 ambulatory visits annually, and over 400 providers, approximately equally distributed between primary care and specialty practitioners, prescribe 2.7 million prescriptions, at a retail cost of $140 million (2008). At the time of the study, the clinic owned three retail pharmacies and contracted with 18 health plans, each with unique formularies. Employees of a wholly owned information-systems subsidiary developed the clinic's first EHR in 1995. The EHR contains patient scheduling, chart notes (historical), and laboratory and imaging reports. It can be viewed from the local hospital but is not integrated. Staggered roll out of the CPOE system began in 2003. Each prescriber received a minimum of 30 min of one-on-one training. Skill levels were assessed, and subsequent training was offered as additional one-on-one training, just-in-time training, and site-specific group training. The CPOE system is web-based, uses point-and-click functionality, and integrates e-prescribing into the existing EHR. The CPOE system makes use of the drug database from Multum (Cerner, Denver, Colorado). The Multum database is a comprehensive compendium of drugs, listing each by dosage form, strength, and packaging. It generates new and renewed prescriptions. Prescribers select medications from pull-down menus or from “favorites” lists. Directions can be selected or typed as free-text. During the study, the CPOE system included basic dosing guidance, presented in preference lists, and duplicate therapy checks. When the prescriber entered a child's weight, the system also calculated weight-based, pediatric dosing of drug, strength, and bottle size (if liquid medication). The prescriber could either accept or over-ride the recommendation. Allergy, drug–drug interaction, drug–disease interaction, and laboratory monitoring alerts were added after completion of data collection. Clinic staff can queue prescriptions, but only licensed prescribers can sign and release them. Prescriptions can then be printed or electronically faxed to a pharmacy of the patient's choice. In the course of their usual duties, pharmacists routinely screen for errors, calling prescribers when the need for clarification arises. The dispensing pharmacists continued in this role separate from study activities; these prescriptions were filled once errors were clarified.

Study design

We conducted a quasiexperimental, pretest–post-test study to evaluate prescriptions for errors, comparing the rate, types, and severity of errors that occurred before (handwritten) to after implementation (e-prescribed). Prescriptions were included from all clinic sites and all provider specialties. To facilitate identification and retrieval of prescriptions, we limited the dataset to prescriptions filled at the three onsite retail pharmacies. The number of prescriptions retrieved from each pharmacy reflected the proportion filled at each pharmacy, as a proportion of all prescriptions filled at the three clinic pharmacies combined, during a 12-month period. The CPOE system was implemented at Clinic/pharmacy site A in July 2003. At this site, preimplementation prescriptions were written between March 1 and July 15, 2002; postimplementation prescriptions were e-prescribed between January 14 and July 13, 2004. At clinic sites B, C, and all others, the CPOE system was implemented in July 2004. These sites are served by pharmacies B and C. At these sties, preimplementation prescriptions were written between January 2 and March 4, 2004; postimplementation prescriptions were e-prescribed between July 1, 2005 and April 26, 2006. At each site, preimplementation prescriptions were retrieved in reverse chronologic order (pre-) and chronologic order (post-) until the targeted proportion from each clinic pharmacy was reached. All prescriptions within the stated time frames were evaluated. We evaluated new and renewal prescriptions but excluded prescriptions transferred to/from outside pharmacies, as the transmittal process could cause errors. We also excluded prescriptions for devices and laboratory monitoring supplies.

We limited the evaluation to prescribing errors, creating a list of error types a priori.^{15 16 28 29 30 31 32} These are: inappropriate abbreviations; missing information; illegibility; wrong- directions, strength, drug, dose, dosage form, patient, physician, or route; allergy; drug–drug interaction; drug–disease interaction; therapeutic duplication; contraindication in patients ≥65 years of age; and lack of appropriate laboratory monitoring. We adapted the therapeutic drug class index of the Department of Veterans Affairs,³³ and used Beer's criteria for patients ≥65 years old.³⁴ A team of pharmacists and physicians created a laboratory monitoring guide by listing frequently prescribed medications and corresponding monitoring parameters, specifying appropriate frequency, and target (or normal) ranges.^{35 36 37 38 39 40} All lists were approved by the clinic's pharmacy and therapeutics committee. Using these tools, we created decision rules for evaluation of errors. The University of Washington Human Subjects Committee approved the study under waiver of consent.

Data collection

Between July 27, 2004 and November 18, 2007, two clinical pharmacists independently evaluated 10 169 prescriptions (RNH, KKH). Both were trained by evaluating 100 prescriptions. Their assessments were compared and discrepancies resolved by members of the research team (EBD, JWN, NML, AWF) before full data collection commenced. Decision rules were clarified and codified. These same team members continued to resolve discrepancies throughout the study. Data were recorded on every prescription, whether or not an error was found. We used a computer-based process to conduct the evaluation.⁴¹

Evaluators collected data from three sources: the prescription (handwritten or e-prescribed), clinical information from the EHR, and the prescription as entered into the pharmacy computer system. Because prescriptions were evaluated after their input into the pharmacy dispensing system, comparing the first and third data sources enabled us both to distinguish prescribing errors from errors that occurred during prescription entry into the pharmacy computer system and to consider pharmacist–prescriber prescription clarifications, as these are documented in the pharmacy computer system. To assess for appropriate monitoring, we reviewed clinical data for 6–12 months prior to the date each prescription was written. The duration was driven by disease-state and drug-specific clinical monitoring guidelines. To determine if an error was associated with a subsequent ADE, we reviewed clinical data for 6 months after. We compared inter-rater reliability to the κ statistic.

In addition, in an exploratory effort to identify ADEs caused by medication errors, we independently identified all patients who experienced an inpatient admission within 90 days of each prescription that appeared in the dataset. We defined an ADE as “an injury resulting from a medical intervention related to a drug”.¹⁶ We reviewed the hospital records of patients whose admission, defined by a discharge diagnosis coded using the International Classification of Diseases 9th Revision Clinical Modification system,⁴² could have been medication-induced, regardless of whether we had identified a prescribing error. A list of the implicated ICD-9 codes, and the corresponding medications, is available from the authors. Using the Naranjo algorithm,⁴³ we assessed the probability that the prescription medication could have contributed to the admission.

Statistical analyses

The prescription was the unit of analysis. The primary outcome was whether or not an error occurred; secondary outcomes were error types or severity. The predictor was the presence of the CPOE system. To make unadjusted comparisons, we used the two-sample test of proportions. To make adjusted comparisons of the effect of the CPOE system on the occurrence of an error (yes/no) and on error type (one of 17 types), we first used alternating logistic regression,⁴⁴ accounting for clustering at the geographic clinic site and prescriber (242 prescribers) levels. As the α level for geographic site clustering was not significant, we then clustered solely at the prescriber level and used generalized estimating equations (GEE) with an independent correlation structure, adjusting for geographic site as a fixed effects variable. We also adjusted for prescriber specialty, patient age and gender, therapeutic drug class, and interaction terms between e-prescribing and each covariate. To control for seasonal variations in prescribing we added a dummy variable that represented each season of the year (fall, winter, spring, or summer) prescribed. We accounted for secular trends by specifying the number of weeks between prescribing of the first prescription, and that undergoing review. We created a best-fitting model by retaining cluster variables and covariates with p values <0.05 (Model 1). If at least one in a group of variables was significant, we retained the group. For infrequently occurring errors, we used reduced models to achieve convergence (Models 2 and 3). When comparing error types, we used the Bonferroni correction to adjust for multiple comparisons, and considered p values <0.005 significant.

We created an inverse probability weighting covariate so that the dataset would better reflect the prescribing patterns of all prescriptions written by clinic providers, regardless of whether these prescriptions were filled on site. The weights adjusted for provider specialty and therapeutic drug class, and were stratified by the onsite pharmacy from which each prescription was retrieved. Details about these weights are available from the authors. Based on our pilot data, using an intraclass correlation coefficient of 0.02 and a two-sided α, we calculated 90% power to detect a 20% decrease in errors from a baseline rate of 25%.

For the severity outcome, we collapsed the NCC MERP categories to better reflect errors wherein harm ensued, in the manner of Snyder et al⁴⁵ (table 1). For this comparison, we used a generalized linear latent and mixed effects model (GLLAMM), specified a multinomial logit link, and adjusted for the same covariates as in the GEE model. Analyses were conducted in SAS 9.1 and Stata 10.1.

Results

Patient and prescription characteristics appear in table 2. Each reviewer evaluated 5016 prescriptions pre- and 5153 postimplementation. The κ for inter-rater reliability was 0.62 (93% agreement) at the error-found level, and 0.7 (97% agreement) at the severity level. The frequency of errors declined from 18.2% to 8.2% with use of the CPOE system, an unadjusted reduction of 55%. The adjusted odds of an error occurring postimplementation was 70% lower than pre-implementation (OR: 0.30; 95% CI 0.23 to 0.40; p<0.001) (table 3).

Table 2.

Characteristics of patients and prescriptions

	Pre-CPOE (N=5016)	Post-CPOE (N=5153)
Patient age (≥65 years)	597 (11.9%)	729 (14.2%)***
Female	2887 (57.6%)	3086 (59.9%)*
Prescriber specialty
Internal medicine	1843 (36.7%)	2347 (45.6%)***
Family practice	1255 (25.0%)	1296 (25.2%)
Pediatrics	492 (9.8%)	407 (7.9%)**
Walk-in clinic	475 (9.5%)	345 (6.7%)*
Specialty	836 (16.7%)	646 (12.5%)***
All others	115 (2.3%)	112 (2.2%)
Therapeutic drug class
Antibiotics	1180 (23.5%)	746 (14.5%)***
Antidepressants	257 (5.1%)	296 (5.7%)
Central nervous system agents	402 (8.0%)	568 (11.0%)***
Hormones	278 (5.5%)	370 (7.2%)***
Schedule II–V	1004 (20.0%)	960 (18.6%)
All others	1895 (37.8%)	2213 (43.0%)***
Geographic site
Clinic site A	1420 (28.3%)	1691 (32.8%)***
Clinic site B	1741 (34.7%)	2053 (39.8%)***
Clinic site C	1450 (28.9%)	1087 (21.1%)***
All other clinic sites	405 (8.1%)	322 (6.3%)***

^*p<0.05

^**p<0.005

^***p<0.001 when compared to pre-CPOE.

CPOE, computerized provider order entry.

Table 3.

Impact of the computerized provider order entry (CPOE) system on medication errors

	Pre-CPOE N (%)	Post-CPOE N (%)	Difference N (%); 95% CI for difference (unadjusted)	OR 95% CI (adjusted)‡
Total no of prescriptions reviewed	5016 (49.3%)	5153 (50.7%)	–	–
Total no of prescriptions with one or more errors	911 (18.2%)	423 (8.2%)	488 (10.0%) (8.7% to 11.3%)***	0.30 (0.23 to 0.40)***
Total no of errors	1012	440	–	–
For prescriptions with errors
No of errors per prescription			–	–
1	811	405
2	85	16
3	9	1
4	1	0
For prescriptions with errors			–	–
Mean no of errors per prescription	1.09	1.04

Model 1: Adjusted model contains the following variables: main effects: age (≥65), gender, antibiotics, antidepressants, central nervous system (CNS) agents, hormones, Schedule II–V agents, clinic site A, clinic site B, clinic site C; Interaction terms: CPOE×CNS agents, CPOE×hormones, CPOE×Schedule II–V agents, CPOE×site C.

^***p<0.001.

^‡Generalized estimating equations with independent correlation structure; clustering at the prescriber level; prescription weighting schema applied.

Types of errors are presented in order of decreasing frequency in the preimplementation period (table 4). The greatest reduction in odds occurred with illegibility (97%), followed by inappropriate abbreviations (94%) and information missing (85%) errors. Also significant were reductions in odds of wrong strength (81%), drug–disease interaction (79%), and drug–drug interaction (76%). errors. Non-significant reductions in odds occurred with wrong drug errors (63%), drugs that were contraindicated in patients ≥65 years of age (56%), and wrong directions errors (34%). There was no decrease in the proportion of errors attributed to lack of appropriate laboratory monitoring, therapeutic duplications, or wrong dose. The numbers of errors in remaining categories were small.

Table 4.

Impact of the computerized provider order entry (CPOE) system on medication errors, by error type

Error type‡	Total prescriptions Pre-CPOE N=5016	Total prescriptions Post-CPOE N=5153	Difference N (%); 99.5% CI for difference (unadjusted)	OR (99.5% CI) (adjusted)§
Inappropriate abbreviation	258 (5.1%)	20 (0.4%)	238 (4.7%) (4.1% to 5.4%)***	0.06 (0.02 to 0.27)*** ¶
Information missing	226 (4.5%)	150 (2.9%)	76 (1.6%) (0.9% to 2.3%)***	0.15 (0.04 to 0.51)*** ¶
Illegible prescriptions	142 (2.8%)	2 (<0.1%)	140 (2.8%) (2.3% to 3.3%)***	0.03 (0.01 to 0.06)*** ††
Wrong directions	125 (2.5%)	69 (1.3%)	56 (1.2%) (0.6% to 1.7%)***	0.66 (0.26 to 1.70)¶
Lack of appropriate laboratory monitoring	103 (2.1%)	112 (2.2%)	−9 (−0.1%) (−0.7% to 0.4%)	0.84 (0.33 to 2.20)¶
Contraindication in patients ≥65 years	29 (0.6%)	10 (0.2%)	19 (0.4%) (0.1% to 0.6%)†	0.44 (0.11 to 1.65)††
Drug–disease interaction	28 (0.6%)	9 (0.2%)	19 (0.4%) (0.1% to 0.6%)†	0.21 (0.05 to 0.84)† ††
Drug–drug interaction	25 (0.5%)	13 (0.3%)	12 (0.2%) (<−0.1% to 0.5%)*	0.24 (0.07 to 0.81)*** ‡‡
Wrong strength	19 (0.4%)	7 (0.1%)	12 (0.3%) (<−0.1% to 0.4%)*	0.19 (0.04 to 0.93)† ††
Wrong drug	16 (0.3%)	7 (0.1%)	9 (0.2%) (<−0.01% to 0.4%)	0.37 (0.04 to 3.70)¶
Therapeutic duplication	12 (0.2%)	18 (0.4%)	−6 (−0.2%) (−0.3% to 0.1%)	0.60 (0.05 to 6.49)††
Wrong dose	10 (0.2%)	14 (0.3%)	−4 (<−0.1%) (−0.3% to 0.1%)	1.90 (0.29 to 12.43)††
All other types§§	19 (0.4%)	9 (0.2%)	10 (0.2%) (0.1% to 0.5%)***	0.81 (0.14 to 4.71)††

^*p<0.05

^***p<0.001

^†p<0.005.

^‡Error types sum to more than the number prescriptions with errors due to more than one error type for some prescriptions.

^§Generalized estimating equations with independent correlation structure; clustering at the prescriber level; prescription weighting schema applied.

Adjusted models contain the following variables:

^¶Model 1: Main effects: age (≥65), gender, antibiotics, antidepressants, central nervous system (CNS) agents, hormones, Schedule II–V agents, clinic site A, clinic site B, clinic site C; Interaction terms: CPOE×CNS agents, CPOE×hormones, CPOE×Schedule II–V, CPOE×site C.

^††Model 2: Main effects: age (≥65), gender.

^‡‡Model 3: No additional variables.

^§§All other types=wrong patient, wrong physician, wrong dosage form, wrong route, and drug allergy.

Table 5 summarizes prescriptions by error severity, using collapsed categories that represent the potential to cause an ADE.⁴⁵ E-prescribing was associated with a significant, 57% reduction in the odds of an error occurring that did not cause harm, potential ADEs (levels B–D). Level B errors, which by definition did not reach the patient, decreased from 445 to 84 (8.9% to 1.6%). There was a 49% reduction in odds of errors that caused harm, actual ADEs (levels E and F); this was statistically not significant, perhaps due to the small number of errors in this category. The one level F error occurred preimplementation.

Table 5.

Impact of the computerized provider order entry (CPOE) system on medication errors, by severity

Error severity	Total prescriptions pre-CPOE N=5016	Total prescriptions post-CPOE N=5153	Difference N (%); 95% CI for difference (unadjusted)	OR (99.5% CI) (adjusted)‡
Error severity, by categories
A (potential error; no ADE)	7 (0.1%)	1 (<0.1%)	6 (<0.1%) (<0.1% to 0.2%)*	0.13 (0.02 to 1.07)
B–D (error, no harm; potential ADE)	896 (17.8%)	417 (8.1%)	479 (9.8%) (8.5% to 11.1%)***	0.43 (0.38 to 0.49)***
E and F (error, reached patient-contributed to harm; preventable ADE)	8 (0.2%)	5 (0.1%)	3 (<0.1%) (<−0.1% to 0.2%)	0.58 (0.19 to 1.77)

^*p<0.05

^***p<0.001.

^‡Generalized linear, latent and mixed effects model with adaptive quadrature; multinomial logit model; clustering at prescriber level; no weights applied; no additional variables.

ADE, adverse drug event.

The results of the best-fitting model appear in table 6. In addition to e-prescribing being associated with a 70% reduction in the odds of an error occurring, the results revealed that prescriptions for patients ≥65 (vs <65) years were more likely to be associated with an error, whereas those written for antibiotics were less likely to be associated with an error than other classes. The interaction terms were significant for hormones, Schedule II–V drugs, and Clinic C. We retained the covariates of antidepressants, and the interaction term between e-prescribing and central nervous system (CNS) agents (and therefore CNS agents), as these were significant in the full model. We retained Clinic sites A and B to keep this group together; and female, for convention. In the full model, the occurrence of an error did not differ over provider type, season of the year during which the prescription was written, or the number of weeks since the first prescription was written; thus, these do not appear in table 6.

Table 6.

Impact of the computerized provider order entry (CPOE) system on medication errors, description of multivariate model

	Coefficient (OR) adjusted1 2	95% CI for coefficient (95% CI for OR)‡
Predictor of interest
e-prescribing (eRx)	−1.19 (0.30)	−1.48 to −0.91*** (0.23 to 0.40)
Main effects
Age≥65 years	0.35 (1.42)	0.08 to 0.64** (1.08 to 1.90)
Female	−0.14 (0.87)	−0.30 to 0.03 (0.74 to 1.03)
Therapeutic drug class
Antibiotics	−0.61 (0.54)	−0.88 to −0.34*** (0.41 to 0.71)
Antidepressants	−0.46 (0.63)	−0.97 to 0.06 (0.38 to 1.06)
CNS agents	0.09 (1.09)	−0.43 to 0.60 (0.65 to 1.82)
Hormonal agents	−0.46 (0.63)	−0.89 to −0.03* (0.41 to 0.97)
Schedule II–V agents	0.79 (2.20)	0.47 to 1.10*** (1.60 to 3.00)
Clinic site A	−0.37 (0.69)	−0.67 to −0.06* (0.51 to 0.94)
Clinic site B	−0.06 (0.94)	−0.50 to 0.33 (0.61 to 1.39)
Clinic site C	0.13 (1.14)	−0.26 to 0.52 (0.77 to 1.68)
Interaction terms
CPOE and CNS agents	0.55 (1.73)	−0.15 to 1.25 (0.86 to 3.49)
CPOE and hormonal agents	0.69 (1.99)	0.09 to 1.29* (1.09 to 3.63)
CPOE and Schedule II–V agents	0.52 (1.68)	0.08 to 0.96* (1.08 to 2.61)
CPOE and clinic site C	−0.57 (0.57)	−1.00 to −0.14** (0.37 to 0.87)

^*p<0.05

^**p<0.01

^***p<0.001.

^‡Generalized estimating equations with independent correlation structure; clustering at the prescriber level; prescription weighting schema applied; Adjusted model contains the following variables: Main effects: age (< ≥65), gender, antibiotics, antidepressants, hormones, Schedule II–V agents (referent is all other therapeutic classes), clinic site C (referent is clinic site A); Interaction terms: eRx×hormones, eRx×Schedule II–V agents, eRx×site C.

CNS, central nervous system.

The Appendix (online at http://www.jamia.org/) describes the 14 preventable ADEs (levels E and F) found during prescription evaluation. Four of these were attributed to lack of appropriate laboratory monitoring, three to drug–disease interactions, three to wrong directions, and two to wrong dose. For the exploratory ADE analysis wherein we set out to pair a hospital admission with a prescription in the dataset, we identified 59 prescriptions (0.6%) for which the medication prescribed could be associated with the discharge ICD-9 diagnosis. We reviewed the discharge notes of these patients and applied the Naranjo algorithm.⁴³ In all cases, the score on the Naranjo scale was 2 points, out of a possible total of 13 points—indicating a “possible” association between the medication and admission. Indeed, the only criterion that was met was that of temporal association, and this by study design—each prescription was written prior to admission. With only five of these 59 prescriptions was an error found during the prescription review process; all were of severity level C.

Discussion

Our results suggest that implementation of a basic CPOE system in the ambulatory setting is associated with a significant reduction in medication errors. Consistent with our hypothesis, the types of errors most easily mitigated by a basic CPOE system were reduced. Elimination of illegibility is inherent in the e-prescribing process. Forcing routine functions minimizes the use of inappropriate abbreviations and missing information. Pull-down menus mitigate other “wrongs” (eg, wrong strength). Despite the fact that our CPOE system lacked CDS alerts, reductions in numbers of drug–disease and drug–drug interaction errors were significant, although these occurred infrequently at baseline. Types of errors we would have thought would have occurred frequently with handwritten prescribing (eg, drug–drug interactions) occurred in ≤2% of prescriptions. Perhaps, due to our thorough evaluation methods, our evaluators were able to discern clinically meaningful versus theoretically relevant contraindications, interactions, and allergies. That the CPOE system did not affect the number of inappropriate laboratory monitoring errors suggests that CDS alerts are necessary to further decrease some of the more complex types of errors.

Implementation of the CPOE system was associated with a reduction in the number of errors at each severity level. Importantly, use of the CPOE system was associated with fewer errors that reached the patient (level B). Our model reveals that patients ≥65 years experienced more errors, but these were not associated with e-prescribing.

We used a MEDLINE search from 1997 to December 2008 to identify studies similar to ours. Bates' group was one of the first to demonstrate a significant reduction in serious errors (categories C through I) associated with an inpatient CPOE system, from 10.7 to 4.86 events per 1000 patient days (55% reduction; p<0.01). They noted a reduction in potential ADEs (C and D) from 5.99 to 0.98 events (84% reduction; p=0.002), and in actual ADEs of 4.69 to 3.88 (17% reduction; p=0.37).¹⁴ In a follow-on study, they noted an 86% reduction in serious medication errors (p<0.001), using an advanced CPOE/CDS system.¹⁵ Our results demonstrate a similar overall reduction rate of 55%, although our reductions in potential (9.8%) and actual (0.1%) ADEs were lower. These comparisons are not exact, as Bates' group excluded errors in categories A and B, while we included these. Our 70% reduction in odds of an error is similar to that noted in Shamliyan's meta-analysis.⁹

Of the investigations that have taken place in the ambulatory setting,^{11 18 19 20 21 22 23} only one is comparable.²¹ Gandhi retrospectively compared 1879 prescriptions that were handwritten versus e-prescribed using a basic CPOE system, and noted a reduction in errors (11.0% to 4.3%; p=0.31) and potential ADEs (4.0% to 2.6%; p=0.16). Although their results were not significant, the investigators suggested that CDS alerts could have prevented 97% of errors and 95% of potential ADEs. Our corresponding reductions were significant for errors, 18.0% to 8.2% (levels B–F; p<0.001), and for potential ADEs, 17.8% to 8.1% (B–D; p<0.001). Our rates are higher, but reductions similar in magnitude.

Jha et al found that 1.4% of hospital admissions were due to ADEs and that 28% of these were preventable.⁴⁶ Our rate of preventable ADEs was also <1%. Steele et al's pre-, postevaluation assessed the effect on ordering behavior of drug-laboratory CDS alerts in the ambulatory setting.²³ Applying the Naranjo algorithm,⁴⁵ these investigators noted a trend toward fewer “definite” or “probable” ADEs (4.3% vs 10.3%; p=0.23). Using the Naranjo algorithm we found only “possible” associations between admissions and preventable ADEs, and these were tenuous, at best.

Although evidence suggests that well-designed CPOE/CDS systems can reduce error rates and improve care, most errors do not cause harm. However, even infrequently occurring preventable ADEs are unacceptable. Further, the benefits of CPOE/CDS systems are not limited to clinical outcomes; there are also cost implications. The cost of a preventable ADE in older adults has been conservatively estimated at $1983; a national average cost of $887 million (both 2000 $).⁴⁷ These costs include inpatient stays (62%), emergency department visits (6%), outpatient care and physician fees (28%), and prescribed medications (4%). There are also costs associated with medication errors: prescriptions written without an indication, diseases not being treated, lack of adherence, and staff time spent preventing errors from becoming ADEs. These have yet to be estimated.

Some have noted that improved practitioner performance was associated with automatic prompts¹⁸ and homegrown software^{7 11}; that RR reductions were larger when e-prescriptions were compared with those that were handwritten,¹¹ and when the rate of errors with handwritten prescriptions was >12%.⁹ Others noted greater effects in studies that used a manual chart review to detect errors.¹¹ Our system was homegrown, our baseline error rate of handwritten prescriptions was 18%, and our methods included chart review. Perhaps these factors contributed to the magnitude of our error reduction rate.

Our study describes the benefits of a homegrown CPOE system in a community setting in the Northwest. To our knowledge, this is one of the largest studies conducted to date that evaluates this impact. During the study time frame, no other medication safety initiatives were implemented at any site within the clinic. Our sample size gave us power to detect a reduction in errors, although not in preventable ADEs. We limited our postimplementation analysis to only e-prescriptions. Prescribers still had the option of handwriting prescriptions, although few did, as adoption was rapid, and we allowed a 6-month lag time to achieve stability before evaluating prescriptions written postimplementation. Approximately 10% of prescriptions were handwritten at this juncture; these were written by prescribers across clinic sites and specialties. We could not blind our prescription reviewers as they viewed the actual prescriptions, either handwritten or e-prescribed. To achieve study feasibility, we limited our evaluation to prescriptions filled at the three pharmacies owned by The Everett Clinic. These prescriptions may differ from those filled elsewhere. Our analysis methods included a weighting variable to address this limitation. Ours was not a randomized trial, as is evidenced by some of the differences in characteristics of patients, prescribers, and prescriptions illustrated in table 2. To address this limitation we used analytic methods that accounted for clustering on prescriber, effect modification of therapeutic drug class or site, and observable confounding. Finally, our inpatient chart review methods enabled us to capture information about ADEs, although our methods were not ideal for finding definitive links between medications and subsequent hospital admissions.

Research that further illuminates the patient safety benefits of CPOE/CDS systems is ongoing. Evidence that establishes these benefits is substantial for homegrown systems in the inpatient setting, but is not as robust in the ambulatory setting. Our work contributes to this knowledge base in the latter setting.

Conclusion

The results of our study, which demonstrate the association between implementation of a CPOE system and a reduction in medication errors, indicate that even a basic CPOE system, without CDS alerts, can have a favorable impact on medication safety. That this work was conducted in an independent medical group practice suggests that CPOE implementation is associated with improved medication safety in the real-world setting.