Does Computerized Provider Order Entry Reduce Prescribing Errors for Hospital Inpatients? A Systematic Review

Abstract

Previous reviews have examined evidence of the impact of CPOE on medication errors, but have used highly variable definitions of “error”. We attempted to answer a very focused question, namely, what evidence exists that CPOE systems reduce prescribing errors among hospital inpatients? We identified 13 papers (reporting 12 studies) published between 1998 and 2007. Nine demonstrated a significant reduction in prescribing error rates for all or some drug types. Few studies examined changes in error severity, but minor errors were most often reported as decreasing. Several studies reported increases in the rate of duplicate orders and failures to discontinue drugs, often attributed to inappropriate selection from a dropdown menu or to an inability to view all active medication orders concurrently. The evidence-base reporting the effectiveness of CPOE to reduce prescribing errors is not compelling and is limited by modest study sample sizes and designs. Future studies should include larger samples including multiple sites, controlled study designs, and standardized error and severity reporting. The role of decision support in minimizing severe prescribing error rates also requires investigation.

Introduction

The complexities of medication management pose a significant safety risk for hospitalized patients. Each of the phases of the medication process, namely prescribing, dispensing, administration, and monitoring, provide opportunities for confusion or error. Several studies have shown adverse drug events (ADEs), many of which are preventable, ^1–9 to be common occurrences in inpatients. 1–3,10–12 Medication errors and ADEs most frequently occur at the drug ordering or prescribing stage. ^1–3,13 Prescribing errors occur in 0.3–39.1% ¹⁴ of medication orders for hospital inpatients, and harm due to prescribing errors has been reported in approximately 1% of inpatients. ^1,15 Some of this variation may be explained by differences in study methods, inconsistencies in definitions applied, and the ways in which prescribing rates are calculated. Computerized provider order entry systems (CPOE) have been consistently identified as an important intervention with the potential to reduce prescribing errors and injury, ^16–19 yet the evidence-base for their effectiveness is limited. ²⁰ The CPOE systems automate the medication ordering process and even those without decision support may have advantages over hand-written prescribing in terms of standardization, a full audit trail, legibility, use of approved names, specification of key data fields such as route of administration, and storage and recall of records. ²¹ Many CPOE systems are further enhanced by the integration of clinical decision support systems (CDSS) of varying sophistication to assist in such functions as dose range checking, doses in renal failure, standard dosing schedules for complex but standardized dosing regimens, adherence to prescribing guidelines, identification of duplicate therapy or drug interactions and checking of pertinent biochemical or microbiological test results. ²² Several studies purport to have shown that electronic prescribing with CPOE significantly increases prescribing quality in hospital inpatients, ^21–31 however, attention has also been drawn to new types of errors which have been introduced following CPOE system implementation. 12,21–23,27,28,31,32

Health systems internationally are making large investments in CPOE systems. Arguments in support of such systems often rely upon reference to early seminal evaluation studies. ^33,34 However, such studies are few and predominantly represent results from the United States, and home-grown systems designed by leading hospitals within the United States. ²⁰ Furthermore, some studies of the impact of CPOE have focused on outpatient settings. There has been a proliferation of CPOE systems available commercially but most have not been subject to comprehensive field evaluation, notably of their effectiveness and safety. For health care organizations to make informed decisions about the likely effects of the introduction of these systems, an up-to-date review is required of the latest evidence that is applicable to a broad range of organizations, most of whom are adopting commercial CPOE systems within inpatient settings.

Prescribing Error Rates

A major problem in interpreting results from prescribing error studies is that definitions of “errors” are often ambiguous or not presented at all. ¹⁵ In studies where definitions are given, there are numerous variations. This leads to great difficulty in making valid comparisons of error rates across studies. Methods for measuring prescribing errors among hospitalized patients have also been found to vary considerably; ¹⁴ study designs may be retrospective or prospective, and they may be based on the prescribing process (which identifies all errors including many minor errors which are unlikely to result in patient harm), or on outcome (which focuses on those errors that lead to patient harm). In some studies, prescribing errors are not distinguished from other types of preventable adverse drug events. In several studies, the distinction between a medication error and a prescribing error is unclear. ^21,35,36 In ▶ we define and distinguish some of the most frequently used terms in studies of medication errors.

Figure 1.

Medication error-related terms and definitions used in this paper.

Applying a two-stage Delphi technique, Dean and Barber ¹⁵ derived the definition of a prescribing error as an error resulting from “… a prescribing decision or prescription writing process, [where] there is an unintentional, significant (1) reduction in the probability of treatment being timely and effective, or (2) increase in the risk of harm when compared with generally accepted practice” (p. 235).

▶. illustrates the relationships between ADEs, potential adverse drug events (potential ADEs), adverse drug reactions (ADRs), medication errors, and prescribing errors. Several examples assist in further illustrating the differences and similarities of these categories of medication errors and adverse drug reactions (see Figure 3 available as an online data supplement at http://www.jamia.org).

Figure 2.

Relationship between adverse drug events (ADEs), potential ADEs, medication errors and prescribing errors (modified from Morimoto ⁶³ p. 307).

Previous Reviews of CPOE and Medication Errors in Hospitalized Patients

A literature search of studies of the impact of CPOE in hospitalized patients identified ten reviews, 16,20,37–44 yet none of these focused on the effectiveness of CPOE specifically in reducing prescribing errors. Kaushal et al. ³⁷ (2003) restricted their review to studies evaluating the effect of CPOEs (n = 5) and clinical decision support systems (n = 7) on medication errors and ADEs. Kuperman et al. ¹⁶ (2003) published a review which examined potential benefits and costs and other issues associated with CPOE. They examined a range of endpoints measured in 18 studies. These included five studies on medication errors, ADEs, or adherence to dosing guidelines. Oren et al. ³⁸ (2003) focused on CPOE-related effects on medication errors and ADEs, and outcomes with automated dispensing machines and bar coding. Chaudry et al. ²⁰ (2006) reviewed the impact of health information technology on the quality, efficiency, and costs of medical care, and identified 20 studies which evaluated CPOE and medication errors (n = 15) or medication safety/ADEs (n = 5). A review of medication errors in hospital care published by Murff in 2006 ³⁹ cited ten studies which evaluated CPOE and decision support systems for medication errors, adherence to guidelines, and corollary orders. Corollary orders refer to orders for tests or treatments required to detect, monitor, or ameliorate adverse effects of other tests or treatments. For example, ordering gentamicin should, with few exceptions, trigger a decision to order plasma gentamicin concentrations. Van der Sijs et al. ⁴⁰ (2006) reviewed 17 publications on overriding drug safety alerts during the order entry process and Kuperman et al. ⁴¹ (2007) published a review of medication-related clinical decision support in CPOE systems. In 2008, Eslami et al. ⁴² published a review of 67 medication-related evaluation studies on CPOE and Shamliyan et al. ⁴³ reviewed the evidence of the impact of CPOE on medication errors. The stated objective of the latter review was to examine the association between CPOE and prescribing medication errors. However, not all the outcome measures were prescribing errors, as the twelve identified studies variously examined the association between CPOE and medication errors, prescribing errors, and ADEs, and there appears to have been some confusion as to the actual outcome measures for some individual studies. Ammenwerth et al. ⁴⁴ reviewed 27 studies which evaluated the effect of electronic prescribing on medication errors and ADEs. These included 20 studies in hospital inpatients. Seven of these studies also met all the inclusion criteria for our prescribing error review.

Thus, a limitation of previous reviews is their breadth and failure to answer the fundamental question: does CPOE reduce prescribing errors among hospital inpatients? The objective of this review was to identify and assess only published studies which focused on measuring the effectiveness of CPOE in reducing prescribing errors in an inpatient setting.

Methods

A literature search performed in October 2007 identified studies evaluating the effect of CPOE on prescribing errors. Due to the variation in definitions and terms applied by researchers we used a broad search strategy which incorporated studies of medication errors as well as prescribing errors in hospitalized patients. The search strategy and terms applied to the electronic databases are outlined in Figure 4 (available as an online data supplement at http://www.jamia.org).

We searched Ovid MEDLINE (1950–2007); CINAHL (Nursing and Allied Health) (1982–2007); EMBASE (1974–2007); Journals@Ovid, Inspec via Ovid (1969–2007); International Pharmaceutical Abstract Series via Ovid (1970–2007); Cochrane Database of Systematic Reviews (third Quarter 2007); and the Cochrane Central Register of Controlled Trials (third Quarter 2007). This yielded 954 articles. Additional citations were sourced from reference lists of journal and review articles (n = 45) and a manual search of the Journal of the American Medical Informatics Association (1994–2007; 82 issues) (n = 8) (▶). After initial screening of titles and abstracts, 71 articles were considered for full text review. Review articles, implementation guidelines, user-satisfaction surveys, opinions and letters were excluded. Studies in which the full text was unobtainable (n = 1) were also excluded and searches were limited to articles published in English. Studies that did not include a sample size to allow error rate calculation were also excluded.

Figure 5.

Flow chart showing search outcome.

Two clinical pharmacist reviewers extracted all the data and independently appraised all 71 studies. Disagreement was resolved by discussion and consensus. Studies were evaluated and included only if they reported original data from a study which was conducted in a hospital inpatient setting, if the study design was a pre- and post-CPOE implementation or a comparative (handwritten and CPOE) study, and if one of the main outcome measures was prescribing error rates. Eleven articles met all of these inclusion criteria. ^22–32 In addition, two studies which measured the impact of CPOE on medication errors in hospitalized patients ^21,36 contained sufficient detail to extract the prescribing error results so they were also included. Thus, 13 articles were included in our review.

Of the 58 excluded studies, four were review articles. Four studies were not conducted in an inpatient setting (one was set in each of an outpatient, emergency, or community setting, and one focused on discharge medications). Eighteen studies focused on medication safety in hospital inpatients (12 of which measured the impact of CPOE on medication safety, but none of these included prescribing errors). Twelve studies evaluated various CPOE-related outcomes other than medication errors or prescribing errors. Four CPOE-related discussion papers did not measure any health care variables. The full text of one article could not be located for review, but its title and abstract suggested that it would not have been selected for inclusion. A further ten studies measured the impact of CPOE on medication errors in hospitalized patients. One study was excluded because a sample size was not given. Nine studies were excluded because the prescribing error details were insufficient. Five CPOE-related studies focused on prescribing errors in hospitalized patients were excluded because three were not pre-post CPOE studies or comparative handwritten and CPOE studies, one was not a CPOE study, and one was a case study analysis of prescribing errors. Details of excluded studies may be obtained from the authors.

We separately examined studies of adult patients in ICU and general wards, and pediatric inpatients, as prescribing practices may be different in these settings. Prescribing for pediatric patients involves selection of a medication and its dose, frequency, and formulation, pharmacokinetically and pharmacodynamically suitable for a combination of factors including age, weight or body surface area, and clinical condition. ⁴⁵ The prescriber may need to adjust medication doses according to changes in the patient's body weight. ³ Furthermore, for some “off-label” indications there is often only limited dosing information available. These factors increase the risk of errors, particularly dosing errors. ^3,45 In addition, errors involving the intravenous (IV) route and certain classes of drugs, including anti-infectives, fluids and electrolytes, analgesics and sedatives, have been associated with more frequent prescribing errors in pediatric patients. ³ Similarly, a prescribing error of chemotherapy can have potentially catastrophic consequences due to the high toxicity and narrow therapeutic index of this class of drugs. ⁴⁶

In analyzing the included papers, we focused on changes in error rates and severity, and evidence of any new types of errors generated. We were also cognizant of international differences in prescribing practices. In the UK and Australia, it is practice that a clinician will prescribe medication on the drug chart which will then become the medication administration record (MAR). This process differs in the United States and in Europe, where the clinician will initially prescribe medications, on a “doctor's order sheet” in the United States, or in the patient's medical notes in Europe. Various health professionals are then involved in transcription of the clinician's original order to the MAR. These differences in prescribing practices are a likely source of variation in the frequency and causes of error reported in medication error studies. ⁴⁵ We excluded transcription errors from our definition of prescribing errors, and considered transcription errors to fall under the wider umbrella of medication errors. These types of errors were excluded from this literature review.

Results

Studies of the Effect of CPOE on Prescribing Errors in Hospitalized Patients

Thirteen papers reporting on 12 studies (two articles used the same dataset) met our inclusion criteria and measured the impact of CPOE on prescribing errors in hospital inpatients. These comprised pre- and post-implementation CPOE studies (n = 7), time series (n = 2), cross-sectional (n = 1), crossover (n = 1) and comparative cohort (n = 1) studies. Of these 12 studies, six were conducted in the United States, 24,26,27,29,30,36 three in the UK, 21,22,25,32 two in Europe, ^23,31 and one in Israel. ²⁸

Effect of CPOE on Prescribing Error Rates in Pediatric Hospital Populations

Four studies examined the effect of CPOE with varying levels of clinical decision support on prescribing error rates in pediatric patients 24,26,29,30 (▶). One further study (Mahoney et al.) ²⁷ included both children and adults, but as we were not able to extract the data for the pediatric population this study is not included in ▶.

Table 1.

Table 1 Details of Studies Evaluating the Impact of CPOE on Prescribing Error Rates in Pediatric Hospital Populations

Study (Reference)	Study Design	Study Setting	Study Population	CPOE System	Method of Error Detection	Prescribing Error RatePre-CPOE	Prescribing Error RatePost-CPOE	Outcome—Change in Prescribing Error Rate	Other Outcomes Measured
Potts 29 2004	Prospective cohort study prepost CPOE implementation 2 + 2 mo	Tertiary care academic Children's Hospital (Nashville, United States) 20 bed pediatric critical care unit (PCCU)	13,828 medication orders (n = 514 Patients)	Home grown (some CDSS)	Chart review by designated clinical pharmacist	2662 errors in 6803 orders (39.1%)	110 errors in 7025 orders (1.6%)	95.9% relative reduction (39.1 Versus 1.6%, p < 0.001) in all types of errors associated with medication ordering	Potential ADEs reduced by 40.9% (2.2 versus 1.3%, p < 0.001) PEs reduced by 99.4% (30.1 versus 0.2%, p < 0.001) rule violations (RVs) reduced by 97.9% (6.8 versus 0.1%, p < 0.001)
Cordero 24 2004	Retrospective cohort study prepost CPOE 6 + 6 mo	University Medical Center neonatal intensive care unit (NICU) (Columbus, OH, United States)	111 very low birth weight (VLBW) infants born consecutively within 6 mo before and 100 VLBW infants born within 6 mo after implementation of CPOE	Commercial extensively customized to NICU needs (some CDSS)	Full time NICU dedicated clinical pharmacist pre-data: obtained retrospectively from medical records Post-data: obtained electronically from computerized patient record	At time of admission: 14 of 105 (13%) gentamicin dosages deviated > 10% from recommended dose At time of suspected late-onset sepsis: 2 of 31 (6%) gentamicin dosages incorrect	At time of admission: no medication errors in 92 gentamicin dosages At time of suspected late-onset sepsis: no errors in 28 gentamicin dosages	Error rate in gentamicin dosage reduced to zero
Kim 26 2006	Prospective prepost CPOE study 241 + 296 d	Academic Medical Center (Baltimore, United States) pediatric oncology	1259 pre-CPOE paper pediatric chemotherapy orders (n = 176 patients); 1116 post-CPOE pediatric orders (n = 167 patients)	Commercial (unknown CDSS)	Two-phase daily audit of chemotherapy orders	4.0% incorrect order format (treatment plan), 2.3% incorrect order format (order); 1.1% order and treatment plan nonmatch; 5.8% incorrect calculation	2.6% incorrect order format (treatment plan), 0.06% incorrect order format (order); 6.0% order and treatment plan nonmatch; 0.54% incorrect calculation	4 of 6 high importance steps in the daily chemotherapy ordering process were less likely to be incorrect after CPOE deployment	Daily audit of successful completion of specific steps in treatment plan (confusing audit data analysis results)
Vaidya 30 2006	Simulated test environment, crossover study (handwritten and CPOE)	University Medical Center (Baltimore, United States) pediatric ICU	9 common IV infusions, 234 orders generated by 26 volunteer physicians	Home grown (some CDSS)		170 of 234 (73%) continuous IV infusion orders contained ≥ 1 prescribing errors	10 prescribing errors for 234 continuous IV infusion orders (4.3%)	94.1% relative reduction (73 versus 4.3%, p = 0.0001) in prescribing errors	Handwritten orders: 25% of errors were judged to be “high risk” (→ potential serious ADEs) each prescriber made at least one error CPOE orders: no “high risk” errors

ADE = adverse drug event; CDSS = clinical decision support systems; CPOE = computerized provider order entry; ICU = intensive care unit; IV = intravenous.

In a prospective cohort study pre- and post-CPOE implementation, Potts et al. ²⁹ reviewed the impact of CPOE on prescribing errors in a pediatric critical care unit (PCCU) and reported a significant fall in the overall prescribing error rate from 39.1 to 1.6% pre- and post-CPOE respectively, which represented a 95.9% relative reduction. A further three studies investigated CPOE use in specific prescribing risk areas. Cordero et al. ²⁴ conducted a retrospective cohort study to compare pre- and post-CPOE implementation error rates in gentamicin dosing in very low birth weight infants admitted to a neonatal intensive care unit (NICU) and at the time of suspected late-onset sepsis: pre-CPOE, 13% of gentamicin dosages deviated more than 10% from recommended doses at the time of admission and 6% were incorrect at the time of suspected late-onset sepsis; post-CPOE, no medication errors were found in either group.

In a crossover study (handwritten and CPOE orders) in a simulated test environment, Vaidya et al. ³⁰ showed that a home-grown CPOE system for ordering continuous IV drug infusions in a pediatric intensive care unit (PICU) led to a significant reduction in prescribing error rates from 73 to 4.3% in the handwritten and CPOE groups respectively (relative reduction 94.1%). Kim et al. ²⁶ measured chemotherapy medication order errors in pediatric oncology following the introduction of failure modes and effects analysis (FMEA ^† ) guided CPOE. The results showed that after CPOE was introduced, four of six steps of high importance in a daily chemotherapy ordering process were less likely to be incorrect; in particular daily chemotherapy orders were less likely to involve inappropriate dosing (▶).

Effect of CPOE on Prescribing Error Rates in Adult Hospital Populations

Nine papers reported the effects of CPOE with varying levels of clinical decision support on prescribing error rates in adult patients. 21–23,25,27,28,31,32,36 (See Table 2 available as an online data supplement at http://www.jamia.org.)

Impact of CPOE on Prescribing Errors in Intensive Care Units (ICU)

Three studies investigated the impact of CPOE on the incidence of prescribing errors in adult ICU patients. ^21,23,32 One evaluated CPOE with clinical decision support (Colpaert et al.) ²³ and two without. ^21,32

Colpaert et al. ²³ conducted a controlled cross-sectional trial in two paper-based and one computerized ICU. They showed a significant reduction in overall prescribing error rates in the computerized unit compared with the paper-based units (3.4 and 27.0% respectively, relative reduction 87.4%). The authors identified three groups of medication prescribing errors (PEs): minor PEs (no potential to cause harm); intercepted PEs (potential to cause harm but intercepted in time); and serious PEs (lead to non-intercepted ADEs or potential ADEs). The incidence of total PEs was significantly lower in the computerized unit compared with the paper-based units (3.4 versus 27%) and the incidence of minor PEs was significantly lower in the computerized unit (0.7 versus 18%). The incidence of intercepted PEs and serious PEs were also lower in the computerized unit (0.9 versus 3.8 and 1.8 versus 4.9%, respectively). This was the only study of three conducted in an adult ICU setting involving CPOE with a moderate level of clinical decision support. New errors introduced by CPOE were also identified, in particular an increase in the proportion of duplicate prescriptions (1 out of 331 errors, or 0.3% of errors) in the paper-based unit versus 5 out of 44 errors (11.4%) in the computerized unit) and drug monitoring errors (1 out of 331 errors, 0.3%) and 6 out of 44 errors (13.6%, respectively).

In a prospective pre- and post-CPOE study, Shulmann et al. ²¹ found that CPOE (without clinical decision support) was associated with a significant reduction in the prescribing error rate from 6.6 to 4.7% (relative reduction 28.8%). Most of the errors reported were minor in outcome (72% of the pre-CPOE errors vs. 81% of the post), but two non-intercepted errors with CPOE led to an increased stay or required increased monitoring, and all three of the major intercepted errors (incorrect dose of diamorphine, inappropriate formulation of amphotericin, insufficient monitoring of vancomycin in renal failure) arose with CPOE and could have caused permanent harm or death if they had been administered as prescribed.

Evans et al. ³² conducted a prospective study comparing computer-assisted prescribing and handwritten prescribing for IV fluids, IV drug infusions and intermittent (regular/PRN) drugs. Prescribing error rates remained unchanged for intermittent drugs (10% of orders contained at least one error in both handwritten and computerized prescribing) and prescribing errors increased for IV fluids and infusions (36% of IV fluid and 52.5% of IV infusion handwritten orders contained at least one error, compared with 52% of IV fluid and 68% of IV infusion computerized orders). They also found increased rates of duplicate prescriptions (attributed to the fact that regular, STAT and PRN orders could not be displayed simultaneously) and an increased frequency of failure to discontinue drugs in the computerized prescriptions. However a limitation of this study was a failure of the authors to report data other than as percentages or to conduct any significance testing. The lack of raw numbers also prevents others from performing such analyses.

Impact of CPOE on Prescribing Errors in Adult General Hospital Wards

Five studies investigated the impact of CPOE with varying levels of clinical decision support on prescribing errors in adult general medical or surgical wards. 22,25,27,28,31,36 Dean-Franklin et al ²⁵ conducted a prospective non-random 4 + 4 week pre- and post-CPOE implementation study (3–6 mo before and 6–12 mo after the intervention) in an adult general surgical ward of a teaching hospital, measuring prescribing errors and two other medication related outcomes. The intervention almost halved prescribing errors (3.8 to 2.0% of medication orders, relative reduction 47%, p < 0.001). Donyai et al. ²² studying the effects of electronic prescribing on prescribing errors and pharmacists' interventions (using prescribing error data reported previously by Dean–Franklin et al.), ²⁵ reported that electronic prescribing may prevent many of the more minor errors that pharmacists would have previously identified and corrected. The reduction occurred in errors of prescription writing rather than the prescribing decision. The study further highlighted ten new error types that were specific to electronic prescribing. These mainly involved selection of the incorrect product, dose or frequency from a menu, and inappropriate use or selection of default doses. Two of six major errors identified post-CPOE were considered specific to the default functions of the CPOE.

Voeffray et al. ³¹ conducted a prospective, controlled time series study (15 + 21 mo) in an inpatient and outpatient chemotherapy unit. Their results showed a significant reduction in prescribing error rates from 15 to 0.6% of medication orders per month in the handwritten and CPOE groups respectively (relative reduction 96%). A new category of errors post-CPOE was recognized; the system did not specify the IV fluid in which the chemotherapy drugs were to be prepared if information about the presence of a central venous line was omitted. Additional potential dangers were recognized in that the CPOE system neither corrected unintended discrepancies in prescribing between the outpatient medication regimen and the hospital treatment, nor obliged prescribers to heed computer alerts.

Oliven et al. ²⁸ conducted a prospective cohort study in Israel which compared two medical units, one with handwritten medication orders, the second with CPOE. The study was conducted in parallel on the two departments over six months. Some design features of the study make comparison of the results with others difficult. The stated aim was to compare prescribing errors between the two wards, but the results presented included transcription errors. In the handwritten (HW) ward, drug orders were hand-written by the prescriber and transcribed by nurses, and in the CPOE ward, patient's own medications not available in the hospital were written by hand on the computer-generated printed order. Faults common to handwritten orders (for example, unclear or misspelled drug names requiring clarification from the prescriber, or nonidentification of prescribers) were not included in the error count. The incidence of both drug-related prescribing errors and patient-related prescribing errors were found to be significantly lower in the CPOE ward than in the handwritten ward.

Two of the studies ^27,36 in general hospital wards relied upon voluntary reporting of prescribing errors, a method likely to result in under-reporting of errors. ⁴⁵ As such this methodological weakness limited the confidence which can be placed in the results. Spencer et al. ³⁶ in an observational time series study reported that the number of prescribing errors per discharge was higher after CPOE implementation (increased from 0.015 to 0.019 prescribing errors per discharge). The authors suggested that post-CPOE detection bias could result from health care providers being eager to report problems with a new system. ³⁶

Mahoney et al. ²⁷ conducted a retrospective pre-post CPOE study involving voluntary reporting. Prescribing errors decreased significantly in three out of four monitored categories, specifically drug allergy reporting, excessive dosing and incomplete or unclear orders. A nonsignificant reduction was recorded in therapeutic duplication. Intercepted error reports were sourced primarily from documented clinical pharmacist interventions. Only interventions that were accepted by the prescriber and which subsequently led to a medication order change were included. The primary source of non-intercepted medication error reports was voluntary staff reporting. The data included both prescribing and medication administration errors; the terms prescribing error and medication error were not clearly differentiated; the term medication order was not defined (especially in the context of medication administration and with regard to doses) and the reported error rates were extremely low.

Severity of Post-CPOE Errors

Five studies 21–23,25,30,31 assessed prescribing error severity. Dean–Franklin and Donyai (who used the same dataset) assessed the index of severity of each prescribing error using the method applied by Dean and Barber which uses a scale from 0 (no harm) to 10 (death). ⁴⁸ Shulman et al. defined three classes of severity (minor, moderate, major) but did not define the categories, although the authors stated that the patient outcome from each error was assigned according to an adapted scale, referring to the same work of Dean and Barber. Colpaert used a 0–6 numeric scale where all categories were defined for severity rating. Two studies (Vaidya, ³⁰ Voeffray ³¹ ) did not use a defined severity scale. Vaidya reported that they classed some errors as “high risk” but provided no definition of this category. Voeffray applied a classification of minor and major errors, however the definitions of these categories is inconsistent with established scales such as that applied by Dean et al.

Colpaert reported that pre-CPOE, 60 major prescribing errors were identified in 1224 medication orders (4.9%), and post-CPOE 23 major prescribing errors were found in 1286 orders (1.8%). This represented a significant reduction in the overall proportion of orders containing major prescribing errors. However, pre-CPOE, 60 out of the total 331 errors were rated as of major severity (18%) and post-CPOE 23 out of 44 errors were major (52%). This represents a significant increase in the proportion of errors rated as major, and there was a corresponding significant decrease in the proportion of minor errors post-CPOE.

Shulman ²¹ reported 19 moderate or major errors in 1,036 hand-written orders (1.8%), compared with 22 out of 2,429 CPOE orders (0.9%). This represents a significant decrease in the overall proportion of orders with a moderate/major error. In the handwritten unit, 19 out of the total 69 errors (28%) were rated as moderate/major and in the CPOE Unit 22/117 (19%) errors were moderate/major. This decrease in the proportion of moderate/major errors was not significant.

Dean-Franklin and Donyai reported that there was no statistical difference in the mean severity scores pre- and post-CPOE. Voeffray reported that pre-CPOE, 19% (27 out of 141) of prescribing errors were classified as “major” and 81% (114 out of 141) “minor” pre-CPOE. Post-CPOE, all 6 errors were classed as “minor”. Vaidya, in a simulated test environment, reported 42 out of 170 (25%) handwritten prescribing errors were judged “high risk” and 128/170 (75%) not “high risk”. Post-CPOE no errors were rated as “high risk.”

Discussion

Evidence of the effectiveness of CPOE systems to reduce prescribing errors is limited and the sample sizes and study methods applied reduce the generalisability and strength of evidence. Nearly all studies involved data collection from no more than two wards or units, few used a control group and some relied upon voluntary error data. We identified no study which compared different prescribing systems or included data from more than one organization. Nine of the 12 studies (two studies reported the same data) demonstrated a significant decrease in prescribing error rates (ranging from 29 to 96%). Two of the three studies conducted in adult ICUs reported a reduction in prescribing errors post-CPOE but the early study of Evans et al. ³² found a 30–44% increase in IV prescribing errors for infusion and fluid orders and no change in error rates for regular and PRN orders. All three studies conducted in adult general wards which did not rely upon voluntary error reports found significant reductions in prescribing error rates post-CPOE implementation. The remaining two studies relied upon voluntary reporting. One reported a decrease of 50% in prescribing errors (from 0.34 to 0.17%) while Spencer ³⁶ reported an overall increase from 0.068 to 0.088 prescribing errors per discharge (a 27% overall increase). The low error rates reflect the likely under-reporting of errors which significantly limits the reliance which can be placed on these results.

Of the four studies in pediatric settings only one examined prescribing errors in all drug types, while the others selectively looked at gentamicin, chemotherapy, or IV prescriptions. All showed significant reductions in error rates post-CPOE but Kim et al. ²⁶ found an increase in one of six specific error types in prescribing orders for chemotherapy drugs.

Severity of Post-CPOE Errors

Limited conclusions can be drawn regarding changes in the severity of errors following CPOE. Five of the 12 studies examined error severity, but only two clearly defined their severity categories. Most studies made reference to, or reported, some data to support the claim that CPOE systems are effective at improving the completeness of orders and are effective at reducing these more minor errors. However the question as to whether CPOE is effective at reducing errors of greater severity remains unanswered and should be a focus of future research.

Comparison across studies is difficult given the inconsistency with which results are reported. For example, absolute error rate reduction versus relative change in error rates, and the use of different denominators (including rates per discharge and per prescribing order). Furthermore, studies rarely report consideration of the power of their sample to detect significant changes in serious errors, which is important given their relatively low incidence. A desirable feature of future studies would be the provision of sufficient data to allow calculation of multiple error rates to permit accurate comparison with existing studies. We suggest that all future studies of this type should include as a minimum, the following:

• • definition of prescribing errors

• • absolute error rates pre- and post-CPOE implementation

• • denominators for prescribing error rates including total number of orders

• • proportion of errors by a standardized severity scale in which all categories are defined

• • error rates per severity category using two denominators: (1) total orders, (2) total errors

• • appropriate significance testing

New Types of Errors

New errors introduced by CPOE included the selection of an inappropriate dosage form for a required route (e.g., capsules for intravenous administration), selection of an inappropriate product, ²⁷ incorrect dose, frequency, or formulation from a dropdown menu, ²¹ inappropriate use or selection of default doses ²² and missed drug allergies. ³⁶ Evans et al. ³² found that regular, STAT, and PRN orders could not be displayed simultaneously in the system being evaluated, thus increasing the probability of duplication of orders. Similar increases in duplicate orders post-CPOE were reported in several other studies. ^23,36 Fragmented screen design which prevents prescribers from viewing all orders at once is one CPOE system feature which has been identified as a contributor to such medication error types. ⁴⁹ Increased frequency of failure to discontinue drugs no longer required ³² and increased drug monitoring errors ²³ were also noted. Voeffray et al. ³¹ reported that the chemotherapy unit CPOE system failed to include infusion solution diluents if a central venous line was not listed, and they also reported a very high override rate for drug allergies and high severity drug interactions. A standardized nomenclature of CPOE-related prescribing errors would enhance monitoring of these error types. Current studies provide a useful base from which to commence this process.

The scarcity of high quality studies reporting error severity reduces the extent to which clear conclusions can be drawn about the importance of these new error types. Identifying errors post-CPOE has specific challenges. It may be more difficult to identify emergent new error types that are specific to electronic prescribing, such as entering orders in the wrong patient's record, incorrect default dosing, or inappropriate use of decision support. ^37,50,51 Further, the need to scroll across or down a page, or to view multiple screens sequentially may make duplicated, discontinued, or not ordered prescriptions and the patient's medication history less noticeable. ²² Little discussion occurs in most papers regarding such challenges, or the development of methods to identify new types of CPOE-associated errors.

Reproducible methods are needed to evaluate the impact of interventions designed to reduce prescribing errors. Apart from the work by Ash and Sittig, ^51–55 who are developing tools to avoid or manage the unintended consequences of CPOE implementation and the generation of new kinds of errors, ⁵⁶ limited research has focused on specifically testing methods for identifying new error types which emerge because of these interventions.

Future Work

This review of the evidence-base regarding the effectiveness of e-prescribing systems to reduce prescribing errors, and the severity of those errors, reveals that the amount of evidence is very modest and the quality and generalisability of results is limited. Development of a systematic, valid and efficient way to measure and quantify prescribing errors would be a useful step forward. ⁵⁷ More robust study designs including controlled before and after and randomized controlled trials conducted in diverse health organizations, particularly those using commercial e-prescribing systems are required. ⁴⁴ While Randomized Controlled Studies (RCTs) are considered the gold standard of study design, they are difficult to conduct in the context of CPOE implementation in a clinical setting, as it is often not possible to limit use of the intervention to specific patients. Cluster randomization may be useful in such situations. ⁵⁸

However the lack of controlled trials in this area is also likely to be a reflection of both a lack of investment in such research and the limited ability of researchers to influence system implementation processes in large health care organizations to undertake the most rigorous type of evaluation trial. It would appear that neither research nor health care organizations have allocated resources to evaluating system effectiveness. Nor, for that matter, have large commercial firms invested in rigorous trials to demonstrate that their systems are effective in reducing errors. A stark comparison might be drawn between the quality of evidence on e-prescribing systems and the investment that is made in ensuring that drugs and other medical devices are safe and effective before wide-spread use.

Efforts are underway in the United States to evaluate and certify EHR products at the vendor level. Various methodologies are used to conduct these comparisons, which may be vendor collected or externally evaluated (often on a proprietary basis). On-site evaluations are not usually part of the approach used in these vendor evaluations. Other testing approaches such as the Leapfrog CPOE evaluation methodology ⁵⁹ offer the potential to evaluate many aspects of EHRs in actual use rather than software on the shelf. This approach certifies CPOE as an implemented safe practice in individual hospitals or healthcare delivery organizations. As the decision support in individual organizational CPOE installations can be highly variable, and because of the potential for CPOE to introduce significant errors, such evaluation is highly desirable. ⁴⁹ A certification process will provide a clear definition of product capabilities and compatibilities. It will also ensure interoperability of these products with emerging local and national health information infrastructure. ⁵⁹

Conclusions

Our review attempted to answer one question: Do CPOE systems in hospitals reduce prescribing errors? We identified 12 studies which yielded some evidence of the effectiveness of these systems to reduce prescribing errors. However, the evidence-base is limited by the modest study sample sizes and designs. The identified studies involved data collection from usually no more than two wards or units, few used a control group and two relied upon voluntary error data. Importantly, the review provides clear pointers to categories of new types of CPOE-associated prescribing errors which should be monitored by hospitals embarking on system implementation. Very few studies reported data on the severity of errors. In achieving significant improvements in patient care, attention must be focused on the most severe prescribing errors and how these effectively may be prevented. To date the evidence suggests that CPOE systems, as expected, deal well with improvements in the quality of orders in terms of legibility and completeness. A more standardized approach to future reporting of prescribing error studies is recommended, including the conduct of multisite research and the investigation and comparison of different CPOE systems. Considerably more research is also required to understand how the design of CPOE systems, including specific decision-support features, may be used to make substantial reductions in serious prescribing errors and subsequent patient harm. By narrowing our review we reduced some of the limitations of previous broad reviews in terms of more confidently being able to draw conclusions in relation to our research question. However our review also contains limitations experienced by previous reviews. Central to these is the heterogeneous nature of the studies and their variable quality. This includes the lack of detail regarding the interventions and their features.