ABSTRACT
BACKGROUND
It is uncertain if computerized physician order entry (CPOE) systems are effective at reducing adverse drug event (ADE) rates in community hospitals, where mainly vendor-developed applications are used.
OBJECTIVE
To evaluate the impact of vendor CPOE systems on the frequency of ADEs.
DESIGN AND PATIENTS
Prospective before-and-after study conducted from January 2005 to September 2010 at five Massachusetts community hospitals. Participants were adults admitted during the study period. A total of 2,000 charts were reviewed for orders, medication lists, laboratory reports, admission histories, notes, discharge summaries, and flow sheets.
MAIN MEASURES
The primary outcome measure was the rate of preventable ADEs. Rates of potential ADEs and overall ADEs were secondary outcomes.
KEY RESULTS
The rate of preventable ADEs decreased following implementation (10.6/100 vs. 7.0/100 admissions; p = 0.007) with a similar effect observed at each site. However, the associated decrease in preventable ADEs was balanced against an increase in potential ADEs (44.4/100 vs. 57.5/100 admissions; p < 0.001). We observed a reduction of 34.0% for preventable ADEs, but an increase of 29.5% in potential ADEs following implementation. The overall rate of ADEs increased (14.6/100 vs. 18.7/100 admissions; p = 0.03), which was driven by non-preventable events (4.0/100 vs. 11.7/100 admissions; p < 0.001).
CONCLUSIONS
Adoption of vendor CPOE systems was associated with a decrease in the preventable ADE rate by a third, although the rates of potential ADEs and overall ADEs increased. Our findings support the use of vendor CPOE systems as a means to reduce drug-related injury and harm. The potential ADE rate could be reduced by making refinements to the vendor applications and their associated decision support.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-012-1987-7) contains supplementary material, which is available to authorized users.
KEY WORDS: medication safety, adverse drug events, unintended consequences
INTRODUCTION
Adverse drug events (ADEs) represent an important source of harm among hospitalized patients.1,2 Most preventable ADEs result from errors at the ordering stage.1 Accordingly, interest has emerged in understanding how computerized physician order entry (CPOE) systems can be leveraged to improve patient safety by reducing prescribing errors.2–4 These systems eliminate problems associated with illegible writing, help to structure orders, and may also help to guide management with decision support.
Most studies of CPOE have been done in academic institutions with internally developed systems, demonstrating significant reductions in preventable ADE rates.5,6 These studies have influenced policymakers7 and consumer advocates8,9 to promote the nationwide adoption of CPOE as a safety-driven initiative. However, despite this mandate, CPOE has not yet achieved significant market penetration.9–12 One of the barriers to widespread adoption is the criticism that there is little evidence to inform providers about the benefits of these applications in the community setting, where nearly all CPOE systems are vendor-developed systems.10,13
Therefore, formal evaluation of vendor CPOE systems on patient safety is essential to help accelerate CPOE adoption and to improve the likelihood that it will have the desired impact in all settings.10 We did an initial study to quantify the incidence of ADEs in six community hospitals. In that study, the ADE rate was 15.0/100 admissions, nearly twice that in a prior study in the academic setting,1 and 75% of the ADEs documented were judged preventable compared to 28% in the prior study. Notably, of the preventable ADEs in the community hospital study, 81.5% were judged to be potentially preventable by CPOE.14 Here, we report the follow-up findings after CPOE implementation.
METHODS
We conducted a prospective before-and-after study in five Massachusetts community hospitals that have newly implemented vendor CPOE systems. Although our baseline study reported on six hospitals,14 one of these hospitals chose not to implement CPOE at this time and was therefore excluded. The study was approved by the Partners Healthcare Human Research Committee and by the study site committees.
Study Setting and Participants
We included subjects aged 18 years or older, admitted to any of the participating hospitals during the study period. These hospitals were felt to reasonably represent small-to-medium-sized community hospitals, each with 100–300 inpatient beds; two had house staff, and three did not. At the time of the study, one of the hospitals (“site 3”) had not yet achieved hospital-wide implementation. At that site, while CPOE had been adopted by most medical services, it had not yet been implemented in the emergency, obstetrical, or surgical departments. Accordingly, we limited our study to the hospitalist and consulting medical services at site 3. For the remaining sites, all admitting services were included with the exception of the psychiatric and neonatal services, which were excluded from both phases because they would have required different detection tools.
In all, 200 patient records per hospital were randomly selected using simple random sampling in the pre- and post-phases for detailed review and data abstraction, resulting in a total sample of 2,000 charts. The first phase of this study occurred over a 20-month period (from January 1, 2005 to August 31, 2006); the second phase began 6 months post-implementation at each study site and lasted 6 to 12 months (from October 1, 2008 to September 30, 2010).
Principal Exposure
Each hospital site independently selected and implemented a vendor CPOE system with decision support capabilities. The names of the vendors cannot be published because of contractual nondisclosure agreements between the sites and vendors. Two distinct systems were studied: vendor A (at sites 1, 2, and 3) and vendor B (at sites 4 and 5). The two systems rank among the top five systems in frequency of use in Massachusetts.
Definitions
We defined an “ADE” as any drug-related injury. ADEs were considered preventable if they were due to an error or were preventable by any means available (e.g., anaphylaxis to penicillin in a patient with a known history of penicillin allergy). A non-preventable ADE was any drug-related injury in which there was no error in the medication process (e.g., an allergic reaction in a patient not previously known to have allergies).15 A “medication error” was an error anywhere in the process of prescribing, transcribing, dispensing, administering, or monitoring a drug with no potential for harm or injury (e.g., an order for an oral medication with no route when it is clear that the oral route was intended).16 A “potential ADE” (or “near miss”) was a medication error with the potential to cause harm, but did not actually cause injury to the patient. These may be intercepted (e.g., a 10-fold overdose of levothyroxine caught by a pharmacist) or non-intercepted (e.g., a patient receiving insulin that was intended for another patient, but remaining euglycemic).
Main Outcome Measures and Case Finding
The primary outcome measure was the rate of preventable ADEs. The rates of potential ADEs and overall ADEs were secondary outcomes.
In this study, medication incidents were identified using an adaptation of the trigger tool developed by the Institute for Healthcare Improvement.15,17,18 We reviewed physician orders, medication lists, laboratory reports, admission histories, progress and consultation notes, discharge summaries, and nursing flow sheets. For each trigger found, a detailed description of the incident was extracted for further review. An example of a trigger is the use of sodium polystyrene. Its administration may be in response to an overdose of potassium, a side effect of a medication, or the result of a drug-drug interaction. Additionally, we added “yeast infection related to antibiotics” and “platelet count <50,000 × 106/μl” as triggers, and excluded lidocaine, gentamicin, tobramycin, amikacin, vancomycin, and theophylline levels in our adaptation of the tool.
Data were abstracted and summarized into electronic forms by trained research nurses. Subsequently, each form was independently reviewed by two investigators (AAL, MA, CK, SRS, MC, NK, and BC). Study personnel underwent training using a curriculum developed by the Center of Excellence for Patient Safety Research and Practice at the Brigham and Women’s Hospital. This curriculum was designed to maintain continuity across projects, minimize individual variability, optimize reproducibility in data collection by chart abstractors, and standardize classification of medication errors, potential ADEs, and ADEs by reviewers. This curriculum has been used in a number of previous studies,1,14,19,20 and has been described.15
All reviewers were blinded to hospital site and prescribing physician. First, reviewers classified incidents as ADEs, potential ADEs, or medication errors with no potential for injury; second, ADEs and potential ADEs were rated for severity according to significant (e.g., rash), severe (e.g., two-unit gastrointestinal bleed), life-threatening (e.g., transfer to an intensive care unit), or fatal categories; third, preventability was determined based on clinical judgment. All disagreements in the classification of type, severity, or preventability were resolved by consensus. To assess inter-rater classification agreement, we calculated κ scores for each reviewer pair (based on the initial independent reviews) and summarized them as a weighted average. We had a κ score of 0.89 [95% confidence interval (CI), 0.85 to 0.92] for ADE vs. no ADE, indicating excellent agreement on the type of event. There was good agreement on event severity (κ, 0.59; 95% CI, 0.28 to 0.56) and moderate agreement on preventability (κ, 0.42; 95% CI, 0.28 to 0.56).
Exclusions
We excluded the following incidents after discussion with local informaticians: at sites 1 and 2, CPOE order sets had been intentionally built to omit drug doses for nebulized medications because it was policy for the respiratory therapists to dose such medications. We reasoned that the omission of dosages in these specific cases did not reflect an error. Furthermore, sites 4 and 5 had installed an electronic medication-administration system (eMAR) with barcode technology to intercept errors of drug duplication. As a result, these hospitals designed some order sets with the potential for therapeutic duplication—thus largely relying on the existing eMAR-barcode system to catch potential overdoses. However, a limitation of this particular eMAR-barcode system was its inability to intercept errors when drugs were ordered with dose ranges. Therefore, we excluded incidents routed in therapeutic duplication only if the eMAR-barcode system was able to intercept the error, but included incidents that could not be systematically detected and prevented.
Statistical Analysis
Baseline characteristics between hospitals were compared using Fisher’s exact test and one-way analysis of variance. To facilitate comparisons between sites, rates were expressed as number of events per 100 admissions with 95% CIs. Pre- and post-implementation rates of ADEs and potential ADEs were compared with the McNemar test. To account for hospital effects in the analysis when comparing pre- and post-implementation rates, we developed a fixed-effects model using Poisson regression. To further explore the independent effects of each vendor, a stratified analysis, adjusted by hospital site, was performed to compare average rates of each outcome observed. Finally, we performed a sensitivity analysis by excluding site 3.
RESULTS
A total of 30,161 patients were admitted across the five hospitals during the observation period. The characteristics of the patients admitted to each of the sites varied widely; age, sex, race, and mean length of stay were significantly different (p < 0.001 for all). The mean age of the hospitalized patients was 63.4 years. The mean length of stay was 4.25 days, ranging from 3.25 days at site 2 to 5.16 days at site 1 (Table 1).
Table 1.
Baseline Characteristics of Participating Sites
| All sites | Hospital site | p-value (among all sites) | |||||
|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | |||
| Patient characteristics | |||||||
| No. of patient admissions | 30,161 | 2,826 (9%) | 5,730 (19%) | 4,508 (15%) | 4,992 (17%) | 12,105 (40%) | |
| Age, years—mean (range) | 63.4 (15–107) | 70.3 (15-103) | 58.9 (15–107) | 66.8 (17–106) | 66.9 (18–103) | 62.0 (18–102) | <0.001 |
| 18–44 (%) | 22.0 | 10.4 | 35.4 | 14.6 | 13.1 | 24.3 | <0.001 |
| 45–54 (%) | 11.4 | 10.4 | 10.9 | 11.9 | 13.0 | 11.1 | |
| 55–64 (%) | 12.9 | 11.7 | 10.6 | 15.5 | 14.3 | 12.8 | |
| 65–74 (%) | 14.7 | 15.6 | 11.4 | 15.6 | 17.5 | 14.8 | |
| 75–84 (%) | 21.7 | 26.5 | 17.4 | 22.9 | 24.9 | 20.9 | |
| ≥85 (%) | 17.3 | 25.4 | 14.2 | 19.5 | 17.2 | 16.1 | |
| Sex | |||||||
| Male (%) | 38.7 | 37.2 | 32.8 | 42.0 | 43.7 | 38.5 | <0.001 |
| Female (%) | 61.3 | 62.8 | 67.2 | 58.0 | 56.3 | 61.5 | |
| Race | |||||||
| Caucasian (%) | 88.5 | 91.3 | 88.8 | 95.8 | 85.6 | 85.3 | <0.001 |
| Hispanic (%) | 5.0 | 1.0 | 1.7 | 1.0 | 6.6 | 8.3 | |
| African American (%) | 3.7 | 4.2 | 3.6 | 1.7 | 5.5 | 3.6 | |
| Native American (%) | 0.04 | 0.07 | - | 0.04 | - | 0.07 | |
| Asian (%) | 1.6 | 1.0 | 3.0 | 0.95 | 1.3 | 1.5 | |
| Other (%) | 0.57 | 0.60 | 0.80 | 0.35 | 0.58 | 0.54 | |
| Not recorded (%) | 0.63 | 2.0 | 0.35 | 0.16 | 0.46 | 0.73 | |
| LOS, days—mean (SD)* | 4.25 (4.45) | 5.16 (6.82) | 3.25 (4.18) | 4.13 (4.18) | 4.62 (3.91) | 4.39 (4.07) | <0.001 |
| Hospital characteristics | |||||||
| Housestaff present | - | No | No | No | Yes | Yes | |
| Vendor CPOE system | - | A | A | A | B | B | |
*LOS = length of stay. SD = standard deviation
Overall, more incidents were observed in the post-implementation phase (683 vs. 798 for pre- and post-implementation, respectively; p = 0.003). Accordingly, there was an increase in the rates of overall ADEs (14.6/100 vs. 18.7/100 admissions; p = 0.03) and potential ADEs (44.4/100 vs. 57.5/100 admissions; p < 0.001) following implementation (Table 2). However, the rates of preventable ADEs, the primary study outcome, decreased following CPOE implementation at every site without exception (10.6/100 vs. 7.0/100 admissions; p = 0.007, see Table A, available online). Therefore, there was a 34.0% reduction in the rate of preventable ADEs, despite a 29.5% increase in the potential ADE rate.
Table 2.
Number and Rates of Incidents Pre- and Post-implementation of Computerized Physician Order Entry Systems Among All Sites
| Incident | Total no. (%) | Rate/100 admissions (95% CI) *† | p | ||
|---|---|---|---|---|---|
| Pre | Post | Pre | Post | ||
| ADEs | 146 (21.3) | 187 (23.3) | 14.6 (12.4, 17.1) | 18.7 (16.1, 21.5) | 0.03 |
| Preventable | 106 (72.6) | 70 (37.4) | 10.6 (8.7, 12.7) | 7.0 (5.5, 8.8) | <0.01 |
| Non-preventable | 40 (27.4) | 117 (62.6) | 4.0 (2.9, 5.4) | 11.7 (9.7, 14.0) | <0.01 |
| Potential ADEs | 444 (64.9) | 575 (72.1) | 44.4 (40.4, 48.9) | 57.5 (52.9, 62.3) | <0.01 |
| Intercepted | 46 (10.4) | 79 (13.7) | 4.6 (3.4, 6.1) | 7.9 (6.3, 9.8) | <0.01 |
| Non-intercepted | 398 (89.6) | 496 (86.3) | 39.8 (36.0, 43.8) | 49.6 (45.4, 54.1) | <0.01 |
*Weighted average over all 5 sites. † CI = confidence interval. ADE = adverse drug event
Stratified Analysis
To account for differences in vendor technology, we performed a stratified analysis between vendors A (sites 1, 2, and 3) and B (sites 4 and 5). The effects of CPOE were generally similar for both vendors. There was a decrease in the rates of preventable ADEs (rate reductions of 3.5/100 and 3.7/100 admissions for vendors A and B, respectively), an increase in total ADEs (rate increases by 4.0/100 and 4.3/100 admissions for vendors A and B, respectively), and an increase in potential ADEs (rate increases by 14.3/100 and 11.2/100 admissions for vendors A and B, respectively). Altogether, the estimates from the stratified analysis were highly consistent with the overall study estimates.
Severity of Events
We further analyzed our data based on event severity (Table 3). Among preventable ADEs, only one fatal event was observed, which occurred prior to CPOE implementation. A significant reduction in life-threatening events (p = 0.003) was seen following CPOE implementation. Likewise, the serious and significant preventable ADE rates also decreased, but did not meet statistical significance. Most preventable ADEs were serious in nature. Examples included sedation or confusion from opioids, hemorrhage from supratherapeutic anticoagulation, and hypoglycemia secondary to insulin therapy while fasting.
Table 3.
Severity of Adverse Drug Events and Potential Adverse Drug Events Pre- and Post-implementation of Computerized Physician Order Entry Systems
| Incident | Pre-implementation | Post-implementation | |||
|---|---|---|---|---|---|
| No. (%) | Average rate/100 admissions (95% CI)* | No. (%) | Average rate/100 admissions (95% CI)* | p-value | |
| All ADEs | |||||
| Fatal | 1 (0.68) | 0.1 (0.006,0.44) | 0 | - | NS |
| Life-threatening | 19 (13.0) | 1.9 (1.2,2.9) | 3 (6.5) | 0.30 (0.07,0.78) | 0.003 |
| Serious | 67 (45.9) | 6.7 (5.2,8.4) | 106 (56.7) | 10.6 (8.7,12.7) | 0.003 |
| Significant | 59 (40.4) | 5.9 (4.5,7.5) | 78 (41.8) | 7.8 (6.2,9.7) | 0.11 |
| Total | 146 (100) | 14.6 (12.4,17.1) | 187(100) | 18.7 (16.1,21.5) | 0.03 |
| Preventable ADEs | |||||
| Fatal | 1 (0.94) | 0.1 (0.006,0.44) | 0 | - | NS |
| Life-threatening | 18 (17.0) | 1.8 (1.1,2.8) | 2 (2.9) | 0.20 (0.03,0.62) | 0.003 |
| Serious | 60 (56.6) | 6.0 (4.6,7.6) | 49 (70) | 4.9 (3.8,6.4) | 0.29 |
| Significant | 27 (25.5) | 2.7 (1.8,3.9) | 19 (27.1) | 1.9 (1.2,2.9) | 0.24 |
| Total | 106 (100) | 10.6 (8.7,12.7) | 70(100) | 7.0 (5.5,8.8) | 0.007 |
| Non-preventable ADEs | |||||
| Fatal | 0 | - | 0 | - | - |
| Life-threatening | 1 (2.5) | 0.1 (0.006,0.44) | 1 (0.85) | 0.1 (0.006,0.44) | NS |
| Serious | 7 (17.5) | 0.7 (0.30,1.4) | 57 (48.7) | 5.7 (4.3,7.3) | <0.001 |
| Significant | 32 (80.0) | 3.2 (2.2,4.4) | 59 (50.4) | 5.9 (4.5,7.5) | 0.005 |
| Total | 40(100) | 4.0 (2.9,5.4) | 117(100) | 11.7 (9.7,14.0) | <0.001 |
| All potential ADEs | |||||
| Life-threatening | 18 (4.1) | 1.8 (1.1,2.8) | 43 (7.5) | 4.3 (3.1,5.7) | 0.002 |
| Serious | 275 (61.9) | 27.5 (24.4,30.9) | 367 (63.8) | 36.7 (3.1,40.6) | <0.001 |
| Significant | 151 (34.0) | 15.1 (12.8,17.6) | 165 (28.7) | 16.5 (14.1,19.1) | 0.43 |
| Total | 444(100) | 44.4 (40.4,48.7) | 575(100) | 57.5 (52.9,62.3) | <0.001 |
| Intercepted potential ADEs | |||||
| Life-threatening | 4 (8.7) | 0.4 (0.12,0.92) | 6 (7.6) | 0.6 (0.24,1.2) | 0.53 |
| Serious | 18 (39.1) | 1.8 (1.1,2.8) | 46 (58.2) | 4.6 (3.4,6.1) | <0.001 |
| Significant | 24 (52.2) | 2.4 (1.6,3.5) | 27 (34.2) | 2.7 (1.8,3.9) | 0.67 |
| Total | 46(100) | 4.6 (3.4,6.1) | 79(100) | 7.9 (6.3,9.8) | 0.004 |
| Non-intercepted potential ADEs | |||||
| Life-threatening | 14 (3.5) | 1.4 (0.79,2.3) | 37 (7.5) | 3.7 (2.6,5.0) | 0.002 |
| Serious | 257 (64.6) | 25.7 (22.7,29.0) | 321 (64.7) | 32.1 (28.7,35.7) | 0.008 |
| Significant | 127 (31.9) | 12.7 (10.6,15.0) | 138 (27.8) | 13.8 (11.6.16.2) | 0.50 |
| Total | 398(100) | 39.8 (36.0,43.8) | 496(100) | 49.6 (45.4,54.1) | 0.001 |
*Weighted average over all 5 sites. CI = confidence interval. ADE = adverse drug event
Potential ADEs increased following CPOE implementation; the risk increased for life-threatening (p = 0.002) and serious potential ADEs (p < 0.001). Nearly all the serious and life-threatening potential ADEs were due to therapeutic duplication following CPOE adoption. Of these, most involved excessive dosing of acetaminophen or acetaminophen-containing products. Other examples of life-threatening potential ADEs included overdosing of benzodiazepines, prescribing an antibiotic with a previous history of serious allergy, ordering chemotherapy for the wrong patient, neglecting to provide a stop order for intravenous magnesium sulphate and potassium chloride, and failure to renally dose digoxin in the setting of acute renal failure.
Drug Classes
While sites 1, 3, and 4 experienced an increase in overall ADEs, these rates were driven by non-preventable events. The majority of non-preventable ADEs in the post-implementation phase were related to cardiovascular and analgesic drugs (see Table B, available online).
Sensitivity Analysis
Even upon exclusion of site 3 (which did not achieve hospital-wide implementation), the study findings remained broadly similar. There was a decrease in preventable ADEs (10.5/100 vs. 6.8/100 admissions; p = 0.01) balanced against an increase in non-preventable ADEs (3.9/100 vs. 10.8/100 admissions; p < 0.001). There was likewise an increase in potential ADEs (36.9/100 vs. 63.6/100 admission; p < 0.001).
DISCUSSION
We found that the implementation of vendor CPOE systems in five community hospitals was associated with a reduction of more than a third of all preventable ADEs. The greatest benefit was seen for reducing the frequency of life-threatening and serious preventable ADEs. However, we also identified a significant increase in potential ADEs. Overall, the associated reduction in preventable ADEs following CPOE implementation suggests that vendor-developed applications, which may not have been through as many cycles of refinement as comparable internally developed applications, can still meaningfully reduce the occurrence of drug-related injury and harm. Furthermore, the overall ADE rate actually increased because of an increase in non-preventable ADEs, which, as best as we can determine, was not related to CPOE implementation.
In our study, the following may have contributed to the increase in non-preventable ADEs: physician documentation improved following CPOE, thus facilitating the identification of ADEs; second, over the 5-year study interval, there were changes in prescribing patterns [e.g., a shift towards prescribing hydromorphone over morphine; increased prescribing of beta-blockers and angiotensin-converting enzyme (ACE) inhibitors for coronary artery disease and heart failure]. Accordingly, we found that the increased adoption of these drugs was associated with a consistent increase in harm (e.g., hydromorphone resulting in oversedation; beta-blockers and ACE inhibitors with hypotension and renal dysfunction).
While CPOE convincingly reduces medication errors,5,21 the impact of this technology in reducing preventable ADEs has been more difficult to prove.6,22 Studies of internally developed systems have largely found reductions in ADEs, preventable ADEs, and potential ADEs.19,21,23,24 Only three studies have assessed the effect of commercial CPOE systems with decision support on the occurrence of ADEs and potential ADEs: one, performed in a clinic, showed a reduction in ADEs that did not meet statistical significance25; the remaining two studies, conducted in a pediatric hospital and an adult critical care unit, found statistically significant reductions in ADEs.26,27 We likewise observed similar benefits in the reduction of preventable ADEs with two vendor-based systems following hospital-wide implementation. However, in most other studies, the rate of potential ADEs fell, while it actually increased in this study.
One particular concern expressed by critics of CPOE systems is the wide range of possible outcomes associated with CPOE, with differences in reported benefits and limitations likely resulting from variations in implementation strategies, differences in technology, and use of decision support.5,6,28,29 It cannot be emphasized enough that CPOE technology should be tailored to specific hospital needs according to workflow and co-existing technology. For example, in our study, most potential ADEs were rooted in therapeutic duplication, often resulting in potentially serious or life-threatening overdoses (e.g., in one case, 20.8 g/day of acetaminophen was ordered). Furthermore, this highlights how the introduction of health information technology may sometimes lead to unintended consequences. In this study, some of the hospital-customized standing order sets had the potential of mishandling many sedatives and analgesics (e.g., opioids, non-steroidal anti-inflammatory drugs) through therapeutic duplication. Consequently, potential ADEs indisputably increased post-CPOE implementation. However, these “near misses” did not appear to be the result of software problems inherent to CPOE as designed by the vendor, but rather from order sets designed by local users.
While CPOE is clearly able to improve patient safety, realizing its potential benefits requires appropriate implementation and use.28 All sites can benefit from tracking issues found post-implementation and then making iterative changes. One approach to this evaluation is to use standardized scenarios with simulated patients to determine whether orders are appropriately transmitted and whether decision support tools function as intended. Simulation tools such as the Leapfrog CPOE Evaluation Tool provide specific performance feedback and help to expose potential safety issues among hospitals implementing CPOE.29
While our study provides evidence that vendor CPOE systems are associated with significant reductions in preventable ADE rates, further research is needed to explore the safest and most effective ways to implement vendor technology. Addressing these needs, our research group is currently investigating the impact of varying levels of decision support in vendor CPOE systems on ADE rates, exploring how orders sets should be used and refined, assessing the economic benefits of CPOE adoption, and identifying barriers to successful implementation.
Our study has limitations, the most important being that it was not a randomized controlled trial. During pre-implementation, six hospitals were studied;14 however, one of these hospital chose not to implement CPOE and further declined to participate in the post-implementation portion of this study as a control site. As such, our study had no contemporaneous control group, and the apparent effect could be the result of a secular trend or an intervention other than CPOE. Furthermore, we were unable to adjust for changing case mix during the study period. Therefore, we cannot exclude the possibility of unmeasured confounding resulting in the reduction in preventable ADEs. However, the introduction of CPOE was the main medication safety-oriented intervention during the study interval, thus arguing against major confounding by co-intervention. The large and significant reductions in preventable ADEs also argue against chance being the explanation for our observations. Second, as with previous studies,1,19,23 we reported the rates of ADEs per admission, rather than per medication order. While these rates facilitate comparisons with previously reported estimates, a limitation is that event rates may be impacted if the typical number of medications prescribed changed substantially between the pre- and post-implementation periods. Third, our study is limited by a heterogeneous intervention as two different vendor systems were studied. However, a stratified analysis by vendor did not change our overall findings, and the effect sizes were broadly similar across all sites. Finally, the systems evaluated did not keep records of electronic transactions for potential orders that were aborted when physicians heeded alerts and cancelled their intended orders before submitting. Consequently, the magnitude of benefit of CPOE in reducing preventable ADEs may be even greater than we observed.
In conclusion, vendor CPOE system adoption in five community hospitals was associated with a reduction in more than a third in the preventable ADE rate, thus achieving comparable benefits as internally developed systems. However, the observed benefit was balanced against an increase in potential ADEs. While this study suggests that the preventable ADE rate may fall after CPOE implementation, it also underscores how monitoring for new problems arising from CPOE is crucial to improving overall patient safety.
ELECTRONIC SUPPLEMENTARY MATERIAL
(DOC 98 kb)
Acknowledgements
The Rx Foundation and Commonwealth Fund supported the study. They commented on its design, but were not involved in data collection, data management, analysis, interpretation, or writing of the manuscript. We thank Kathy Zigmont, RN, and Cathy Foskett, RN, for the chart review and data collection at the participating study sites.
Conflict of Interests and Disclosures
Dr. Bates is a coinventor on patent no. 6029138 held by Brigham and Women’s Hospital on the use of decision support software for medical management, licensed to the Medicalis Corporation. He holds a minority equity position in the privately held company Medicalis, which develops web-based decision support for radiology test ordering, and has served as a consultant to Medicalis. He serves on the board for SEA Medical Systems, which makes intravenous pump technology. He serves as an advisor to Calgary Scientific, which makes technologies that enable mobility within electronic health records. He is on the clinical advisory board for Zynx, Inc., which develops evidence-based algorithms, and Patient Safety Systems, which provides a set of approaches to help hospitals improve safety. He consults for Hearst, which develops knowledge resources. He previously served on the board of Care Management International, which is involved in chronic disease management. Dr. Leung is supported by a Clinical Fellowship Award from Alberta Innovates Health Solutions and by a Fellowship Award from the Canadian Institutes for Health Research. He also receives support from the John A. Buchanan Research Chair in General Internal Medicine.
References
- 1.Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA. 1995;274:29–34. doi: 10.1001/jama.1995.03530010043033. [DOI] [PubMed] [Google Scholar]
- 2.Ball MJ, Garets DE, Handler TJ. Leveraging information technology towards enhancing patient care and a culture of safety in the US. Methods Inf Med. 2003;42:503–508. [PubMed] [Google Scholar]
- 3.Milstein A, Galvin RS, Delbanco SF, Salber P, Buck CR., Jr Improving the safety of health care: the leapfrog initiative. Eff Clin Pract. 2000;3:313–316. [PubMed] [Google Scholar]
- 4.Bobb A, Gleason K, Husch M, Feinglass J, Yarnold PR, Noskin GA. The epidemiology of prescribing errors: the potential impact of computerized prescriber order entry. Arch Intern Med. 2004;164:785–792. doi: 10.1001/archinte.164.7.785. [DOI] [PubMed] [Google Scholar]
- 5.Eslami S, Keizer NF, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patients–a systematic review. Int J Med Inform. 2008;77:365–376. doi: 10.1016/j.ijmedinf.2007.10.001. [DOI] [PubMed] [Google Scholar]
- 6.Wolfstadt JI, Gurwitz JH, Field TS, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med. 2008;23:451–458. doi: 10.1007/s11606-008-0504-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kohn LT, Corrigan J, Donaldson MS. Institute of Medicine . Committee on Quality of Health Care in A. To err is human: building a safer health system. Washington, DC: National Academy Press; 2000. [PubMed] [Google Scholar]
- 8.The Leapfrog Group. Fact Sheet: Computerized Physician Order Entry., 2011. (Accessed December 31, 2011, at http://www.leapfroggroup.org/media/file/FactSheet_CPOE.pdf)
- 9.CPOE still a priority, but… Adoption rates vary according to who’s asking. Mod Healthc 2008;38:S5. [PubMed]
- 10.Ford EW, McAlearney AS, Phillips MT, Menachemi N, Rudolph B. Predicting computerized physician order entry system adoption in US hospitals: can the federal mandate be met? Int J Med Inform. 2008;77:539–545. doi: 10.1016/j.ijmedinf.2007.10.009. [DOI] [PubMed] [Google Scholar]
- 11.Yu FB, Menachemi N, Berner ES, Allison JJ, Weissman NW, Houston TK. Full implementation of computerized physician order entry and medication-related quality outcomes: a study of 3364 hospitals. Am J Med Qual. 2009;24:278–286. doi: 10.1177/1062860609333626. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in US hospitals. N Engl J Med. 2009;360:1628–1638. doi: 10.1056/NEJMsa0900592. [DOI] [PubMed] [Google Scholar]
- 13.Kremsdorf R. CPOE: not the first step toward patient safety. Health Manag Technol. 2005;26(66):5. [PubMed] [Google Scholar]
- 14.Hug BL, Witkowski DJ, Sox CM, et al. Adverse drug event rates in six community hospitals and the potential impact of computerized physician order entry for prevention. J Gen Intern Med. 2010;25:31–38. doi: 10.1007/s11606-009-1141-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates DW. Adverse drug events and medication errors: detection and classification methods. Qual Saf Health Care. 2004;13:306–314. doi: 10.1136/qshc.2004.010611. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Bates DW, Boyle DL. Vander Vliet MB, Schneider J, Leape L. Relationship between medication errors and adverse drug events. J Gen Intern Med. 1995;10:199–205. doi: 10.1007/BF02600255. [DOI] [PubMed] [Google Scholar]
- 17.Rozich JD, Haraden CR, Resar RK. Adverse drug event trigger tool: a practical methodology for measuring medication related harm. Qual Saf Health Care. 2003;12:194–200. doi: 10.1136/qhc.12.3.194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Institute for Healthcare Improvement: IHI Trigger Tool for Measuring Adverse Drug Events., 2011. (Accessed December 31, 2011, at http://www.ihi.org/knowledge/Pages/Tools/TriggerToolforMeasuringAdverseDrugEvents.aspx)
- 19.Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280:1311–1316. doi: 10.1001/jama.280.15.1311. [DOI] [PubMed] [Google Scholar]
- 20.Hug BL, Witkowski DJ, Sox CM, et al. Occurrence of adverse, often preventable, events in community hospitals involving nephrotoxic drugs or those excreted by the kidney. Kidney Int. 2009;76:1192–1198. doi: 10.1038/ki.2009.353. [DOI] [PubMed] [Google Scholar]
- 21.Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc. 2008;15:585–600. doi: 10.1197/jamia.M2667. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bates DW. CPOE and clinical decision support in hospitals: getting the benefits. Arch Intern Med. 2010;170:1583–1584. doi: 10.1001/archinternmed.2010.325. [DOI] [PubMed] [Google Scholar]
- 23.Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6:313–321. doi: 10.1136/jamia.1999.00660313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Evans RS, Pestotnik SL, Classen DC, Horn SD, Bass SB, Burke JP. Preventing adverse drug events in hospitalized patients. Ann Pharmacother. 1994;28:523–527. doi: 10.1177/106002809402800417. [DOI] [PubMed] [Google Scholar]
- 25.Steele AW, Eisert S, Witter J, et al. The effect of automated alerts on provider ordering behavior in an outpatient setting. PLoS Med. 2005;2:e255. doi: 10.1371/journal.pmed.0020255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Colpaert K, Claus B, Somers A, Vandewoude K, Robays H, Decruyenaere J. Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial. Crit Care. 2006;10:R21. doi: 10.1186/cc3983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Upperman JS, Staley P, Friend K, et al. The impact of hospitalwide computerized physician order entry on medical errors in a pediatric hospital. J Pediatr Surg. 2005;40:57–59. doi: 10.1016/j.jpedsurg.2004.09.024. [DOI] [PubMed] [Google Scholar]
- 28.Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293:1197–1203. doi: 10.1001/jama.293.10.1197. [DOI] [PubMed] [Google Scholar]
- 29.Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood) 2010;29:655–663. doi: 10.1377/hlthaff.2010.0160. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(DOC 98 kb)