Medication Safety in Two ICUs of a Community Teaching Hospital After EHR Implementation: Sociotechnical and Human Factors Engineering Considerations

Abstract

Objective

To assess the impact of EHR implementation on medication safety in two ICUs.

Methods

Using a prospective pre-post design, we assessed 1,254 consecutive admissions to two ICUs before and after an EHR implementation. Each medication event was evaluated with regard to medication error (error type, medication-management stage) and impact on patient (severity of potential or actual harm).

Results

We identified 4,063 medication-related events either pre- (2,074 events) or post-implementation (1,989 events). Whereas the overall potential for harm due to medication errors decreased post-implementation, only two of the three error rates improved were significantly lower post-implementation. After EHR implementation, we observed reductions in rates of medication errors per admission at the stages of transcription (0.13 to 0, p<.001), dispensing (0.49 to 0.16, p<.001) and administration (0.83 to 0.56, p=.011). Within the ordering stage, four error types decreased post-implementation (orders with omitted information, error-prone abbreviations, illegible orders, failure to renew orders) and four error types increased post-implementation (orders of wrong drug, orders containing a wrong start or stop time, duplicate orders, orders with inappropriate or wrong information). Within the administration stage, we observed a reduction of late administrations, and increases in omitted administrations and incorrect documentation.

Conclusions

EHR implementation in two ICUs was associated with both improvement and worsening in rates of specific error types. Further safety improvements require a nuanced understanding of how various error types are influenced by the technology and the sociotechnical work system of the technology implementation. Recommendations based on human factors engineering principles are provided for reducing medication errors.

INTRODUCTION

Electronic Health Record (EHR) technology has been associated with both improvement and worsening of medication safety [1 2]. Improving this balance is at the core of the 2012 Institute of Medicine report on “Health IT and Patient Safety” [3]: “To protect America’s health, health IT must be designed and used in ways that maximize patient safety while minimizing harm” (page S1). As endorsed in this IOM report and systematic reviews [4 5], understanding the role of EHR in medication safety requires information on specific characteristics of the technology [6–8] and variations in its implementation [9 10], including the sociotechnical work system in which the technology is used [11–13]. The observation that “health IT value is context-dependent” [14] (page 652) mandates that research on EHR technology provide a more nuanced analysis of the effects of specific EHR implementations on medication safety.

Medication safety in intensive care units (ICUs) is a major problem [15–18]. ICUs are environments that interface high patient acuity with complex medication regimens and require rapid, high-stakes diagnostic and therapeutic decisions. EHRs need to be designed and implemented to support the multiple, complex care activities and decisions in this type of environment. Three reviews [10 19 20] indicate that Computerized Provider Order Entry (CPOE), an EHR component, can reduce serious medication errors in ICUs, specifically ordering errors [10 20]. However, an improvement reduction in adverse drug events (ADEs) has not been consistently demonstrated [10 20]. CPOE can eliminate or reduce many medication errors, such as ordering errors (e.g., missing information, illegible prescription, error-prone abbreviations) [21 22]. Alternatively, CPOE and other EHR components may increase errors or introduce new error types [10 20], such as omitted administrations [1] and duplicate orders [22 23]. CPOE implementation in a neurosurgical ICU was followed by an increase in medication errors [24]. The types of medication error following CPOE implementation also changed: omissions and wrong-dose errors increased, and wrong patient selections decreased. CPOE implementation without decision support in one 22-bed ICU in a UK hospital led to a significant decrease in medication orders with errors: from 6.7% to 4.8%. It was also associated with an increase in certain types of errors (dose errors and omission of the required prescription information) and a decrease in missing information regarding dose, unit or frequency. These mixed findings about the impact of EHR technology on medication safety call for a detailed analysis of medication errors in ICUs.

This study’s objective was to assess the impact of EHR technology on medication safety in two ICUs, including specific medication error types at different medication-management stages. An understanding of the sociotechnical work system context of the technology implementation enriches this analysis. Using According to the SEIPS (Systems Engineering Initiative for Patient Safety) model [11 12], we describe how the EHR technology affects is one of many other work system elements; this sociotechnical systems approach helps us to understand the impact of the technology in the context of the rest of the system and the interactions between the technology and the other system elements. We apply the SEIPS model to describe post-implementation changes in the medication-management process. The SEIPS model also informs our discussion of the contributions of EHR technology to medication safety and our human factors engineering recommendations.

METHODS

We collected data on medication safety in two ICUs of a 400-bed major community teaching hospital before and after EHR implementation. We used a prospective pre-post study design to assess the impact of EHR implementation on medication safety. Data were collected ½ to 1 year before EHR implementation and about ½ year after EHR implementation.

Study setting and participants

The hospital was a level-1 trauma center serving a 30-county area in rural Pennsylvania. One ICU was an adult medical/surgical ICU (AICU) with 24 beds at the time of data collection. Full-time intensivists cared for patients, except surgical patients for whom surgeons were primarily responsible with consultative input from the intensivists. The second ICU was an 18-bed cardiac ICU (CICU), which provided intensive cardiovascular care, and post-surgical care after cardiothoracic surgery or solid-organ transplantation; the CICU also functioned as overflow for the AICU. Care for CICU patients was provided by cardiologists working with cardiology fellows or by cardiothoracic and transplant surgeons working with physician assistants.

We assessed a total of 1,254 consecutive ICU patient admissions: 630 admissions between October 2006 and March 2007 (7–12 months before EHR implementation that occurred in October 2007) and 624 admissions between March and June 2008 (5–8 months after EHR implementation). The pre-EHR implementation data collection took 6 months whereas the post-implementation data collection took 4 months as additional nurse data collectors were available to collect data simultaneously in the two ICUs. Six percent of patient admissions had multiple periods in the ICU during the same hospital admission.^* Exclusion criteria were age less than 18 years old, being a prisoner, and being ICU length of stay less than 4 hours [17 25] in an ICU.

The sample size in each ICU was determined through power analyses to detect a 20% decrease in potential preventable ADEs after EHR implementation, considering the medication order volume and average length of stay in each ICU. With an estimated ICC of .03 and an effect size of .10, the estimated sample size was between 220 and 237 patients for each ICU; we decided to enroll about 300 patients as a conservative strategy. Institutional Review Board (IRB) approval was obtained from the University of Wisconsin-Madison and the study site.

Description of the medication-management process and the EHR implementation

The EHR technology included electronic order management (i.e. CPOE, electronic medication administration (eMAR) and an integrated pharmacy system) and physician documentation and was implemented hospital-wide in October 2007 (EpicCare Inpatient Clinical System, Spring 2006 version). Electronic nursing documentation had been implemented in June 2005, i.e. 16 months before the pre-EHR data collection began. Table 1 shows an overview of the medication-management process and identifies work system elements [11 12] that are different pre- and post-implementation. Detailed information about the medication-management process is in Appendix.

Table 1.

Overview of the medication-management process and changes in the sociotechnical work system pre- and post-implementation

Characteristics of work system …			Work system elements that changed significantly between pre- and post-implementation*
… unique to paper process	… common to paper and electronic processes	… unique to electronic process
Providers enter orders into paper chart kept at central ICU location.		Providers enter orders into computer.	Task Technology Physical environment Organization
	Providers are expected to relay STAT (urgent) orders directly to nurse.
Nursing staff or UDC either give carbon copy to ICU pharmacist or send carbon copy to central pharmacy.		Orders are electronically transmitted to pharmacist.	Person Task Technology Physical environment Organization
Pharmacists review all medication orders and enter orders into pharmacy computer system.		Pharmacists review all medication orders in the EHR.	Task Technology
	Computer system performs checks (e.g., drug-allergy).
Nursing staff or UDC transcribe orders onto paper MAR.		Orders are electronically transferred to eMAR.	Person Task Technology Organization
	Pharmacy dispenses medications.
	Nursing staff administer medications.
Nursing staff document administration on paper MAR.		Nursing staff document administration on eMAR.	Technology

^*The sociotechnical work system of the SEIPS model [11 12] includes 5 elements: person, task, technology, physical environment and organization.

Almost all physicians (and outpatient nurses) had been using the full outpatient EHR for at least 18 months prior to the go-live of the inpatient EHR in October 2007. Completion of training on the inpatient EHR was required for all inpatient physicians and nurses for continuation of admitting privileges and employment. The training began with an online, self-paced, competency-based general EHR curriculum. After a physician or nurse completed the online curriculum, they attended four hours of discipline-specific (e.g., cardiology, ED nursing) classes co-led by an education specialist (who led the class through the classroom curriculum) and a representative of the discipline (who helped the class apply the curriculum to their specific work). At-the-elbow trainers and a 24-hour call center were available throughout the hospital during the first month of EHR use. Each department and unit had workers who volunteered to receive special training in order to support their fellow-workers.

Data collection and identification of medication errors and ADEs

The data collection protocol for medication safety events (i.e. medication errors and adverse drug events or ADEs) was based on the protocol of Bates and colleagues [26] and has been described in a previous publication [15]. We categorized medication safety events into:

1. no harm events: medication errors that did not harm patients and had no potential for harm,.

2. potential preventable ADEs: medication errors with potential to harm patients, but did not because errors were intercepted or resilient patients did not experience negative consequences,.

3. actual preventable ADEs: medication errors that harmed patients.

Researchers worked with the IT team, clinical informaticians, pharmacists, nurses and physicians to define what constituted a medication, a medication order, a complete medication order and an ICU order. A review of hospital policies guided these discussions. A standard method for counting medication orders was developed and used for both pre- and post-implementation data collection with consideration for how orders would appear differently on paper and in the EHR. During the post-implementation data collection period, all patient orders were entered and managed electronically, except for chemotherapy orders and intra-operative orders. The research team developed and validated reports using discrete EHR data fields to support the identification of medication safety events. The reports for each patient included: (1) medication orders entered in the last 24 hours, (2) omitted or late medication administrations, (3) lab results with serum or finger-stick glucose values < 70 and INR > 4.9, (4) common medication antidote administration, and (5) missed dose reports.

Data analysis

Beginning with previously published lists [27–31], two researchers (PC, TBW) developed a list of medication error types. The researchers reviewed each event to determine the medication error types and the medication-management stage at which the error occurred. Whenever one researcher was uncertain about the adjudication, both researchers reviewed the data and mutually agreed upon the final categorization. Inter-rater reliability testing on a random sample of 146 events showed high Cohen’s kappa scores (0.97) for categorizing medication error types and stages (99% agreement). Through an iterative process involving about 10 meetings, the researchers then revised the list of error types for each medication-management stage (Table 4).

Two critical-care physicians (RK and SK pre-implementation; SK and MJ post-implementation) independently reviewed events to determine (1) if the patient suffered harm due to a medication, and (2) the severity of actual or potential harm [26]. Two different pairs for a total of three reviewers participated in the pre- and post-implementation adjudication based on their availability; one physician (SK) participated in both pre- and post-implementation adjudication. The three reviewers received the same training that consisted of a didactic session on medication safety and review of about 35 cases with feedback and discussion of the adjudication. The severity of harm was categorized as (1) fatal (for actual ADEs only), (2) life-threatening (e.g., readmission to the ICU, respiratory failure, anaphylaxis, or severe mental-status deterioration), (3) serious (e.g., intestinal bleeding, altered mental status, excessive sedation, acute kidney injury, symptomatic hypotension, allergic reaction less serious than anaphylaxis and more serious than a rash, or fever), and (4) significant (e.g., diarrhea, mild thrombocytopenia, nausea, vomiting or rash.) Inter-rater reliability (Cohen’s kappa) was 0.75 for actual ADE occurrence (97% agreement), 0.44 for the severity of actual ADE-associated harm (64% agreement) and 0.30 for the severity of potential harm (54% agreement). Although low, these reliabilities are typical for these studies [32], justifying a third adjudicator for disagreements occurring between the first two adjudicators. A third physician (MJ pre-implementation; TBW post-implementation) reviewed and resolved disagreements.

The research team recognized that hypoglycemia documentation improved post-EHR implementation. T; this improved documentation and the our ability to use an EHER-based report of hypoglycemia events led to increased detection of these events: 12 pre-EHR and 179 post-implementation events related to hypoglycemia were detected. Therefore, even though these events are clinically important, we decided to exclude them 12 pre-EHR and 179 post-EHR events related to hypoglycemia were excluded from the analyses.

We first report the impact of EHR technology on overall medication safety, then the rates of medication errors at various medication-management stages, and finally the frequencies of specific medication error types. The overall rates of medication errors as well as the rates for each medication-management stage were calculated per admission and per 1,000 patient-days in a manner similar to the approach used by Bates and colleagues.[1 27] We also computed overall rates per 100 medication orders.

RESULTS

Data on admissions, patients and medication orders for the two ICUs are in Table 2. Patients were similar with regard to age, gender, ethnicity, and ICU length of stay pre- and post-EHR implementation. They differed in the total number of medication orders reviewed as well as the number of medication orders per day and per admission. The decrease in the number of medication orders post-EHR implementation was connected to several EHR-related changes. The printed order sheets and electronic order sets were different: printed order sheets had more separate medication orders than electronic order sets available at the time of data collection. In addition, during the pre-EHR phase, orders for certain medications had to be renewed daily; post-implementation, there were fewer orders to review because orders did not have to be renewed daily. The EHR technology facilitated the implementation of this new ordering practice and, therefore, eliminated one error type: failure to renew medications (see Table 4).

Table 2.

Description of patients and medication orders

	Pre	Post	p-value
Number of admissions	630	624	-
Number of patients	610 (AICU: 294, CICU: 322)	599 (AICU: 290, CICU: 316)	-
Patient age in years: mean, ±SD (median)	61.4 ± 15.9 (63)	62.6 ± 16.5 (64)	0.59 [t-test]
Patient gender: % female	43%	44%	0.99 [chi-square test]
Patient ethnicity: % white	95%	93%	0.56 [chi-square test]
ICU length of stay in days: mean ±SD, range (median)	6.6 ± 7.4, 1–67 (4)	6.4 ± 8.1, 1–87 (4)	0.99 [Mann-Whitney test]
Study period in days	191	194	-
Patient-days	4147	4017	-
Medication orders per patient day: mean ±SD (median)	11.9 ± 4.8 (12)	9.6 ± 5.4 (9)	** [Mann-Whitney test]
Medication orders per admission: mean,±SD (median)	72.5 ± 72.1 (49)	52.6 ± 49.9 (38)	** [Mann-Whitney test]

Bold indicates significant pre-post differences that are statistically significant at the p<0.01 (**) or p<0.001 (***) levels.

Medication errors and ADEs

We identified a total of 4,063 medication-related events: 2,074 events pre-implementation and 1,989 events post-implementation (see Figure 1). Sixty-six pre-implementation events and 60 post-implementation events were categorized as non-preventable ADEs because they did not involve medication errors; they were excluded from the analysis. Among the rest of events with medication errors, 632 pre-implementation and 908 post-implementation events had no potential harm. We, therefore, identified 1,333 pre-implementation and 957 post-implementation potential preventable ADEs that did not harm patients but could have, and 43 pre-implementation and 63 post-implementation actual preventable ADEs that harmed patients.

Figure 1.

Breakdown of medication-related events

Statistical analyses comparing the harm distribution for potential preventable ADEs and no harm events (1,965 and 1,865 events pre- and post-implementation respectively) found a statistically significant reduction in level of potential harm (two-ordered multinomial test with permutations; p<.001) (see Figure 1). For instance, in the pre-EHR period, 32% of the events (632 out of 1,965 events) were rated as having no potential harm, whereas in the post-implementation period, this percentage went up to 49% (908 out of 1,865 events). Twelve percent of the pre-EHR potential preventable ADEs and no harm events were life-threatening (241 out of 1,965 events); this percentage dropped to 7% in the post-implementation period (133 out of 1,865 events). The numbers of actual preventable ADEs were small and the differences were not statistically significant.

An example of an actual life-threatening ADE is ventricular arrhythmia from antipsychotic medication administration without appropriate monitoring of QT interval as the dosing of the antipsychotic medication was not modified when the arrhythmia occurred. An example of an actual serious ADE is oversedation related to benzodiazepine overdose. An example of an actual significant ADE is repetitive diarrhea related to an aggressive bowel regimen, which was not put on hold with first loose stool. The fatal ADE was related to a late administration of anticoagulation reversal agent in a patient with severe bleeding.

In the remainder of the paper, we focus on the 2,396290 (1,37633 pre-EHR and 1,020957 post-implementation) potential and actual preventable ADEs.

Antibiotics and electrolyte concentrates were the medications most involved in potential ADEs pre-EHR (25% and 12% respectively) as well as post-implementation (21% and 12% respectively) (data not shown). Medications involved in actual preventable ADEs were varied; e.g., antibiotics (29% pre-EHR and 8% post-implementation), analgesics (7% pre-EHR and 8% post-implementation), and antihypertensives (7% pre-EHR and 10% post-implementation).

Rates of medication errors and ADEs pre- and post-implementation

Table 3 displays data on potential and actual preventable ADEs for the pre-EHR and post-implementation periods. Whereas the rates of potential preventable ADEs per 1,000 patient-days decreased 32% from 329 to 225 (p<0.001), the rates of actual preventable ADEs per 1,000 patient-days did not change significantly (p=0.89). In the post-implementation period, we observed a 28% decrease in potential preventable ADE events (1,333 pre-EHR and 957 post-implementation). For every patient admission, there were 2.12 potential preventable ADE events in the pre-EHR period, and 1.53 potential preventable ADE events in the post-implementation period.

Table 3.

Error rates pre-EHR and post-implementation

	Potential Preventable ADEs			Actual Preventable ADEs
	Pre	Post	p-value~	Pre	Post	p-value~
# errors	1333	957	-	43	63	-
Error rate per admission, mean (SD), range	2.12 (3.27) 0 – 25	1.53 (2.51) 0 – 22	***	0.07 (0.36) 0 – 3	0.10 (0.54) 0 – 8	0.74
Error rate per 1000 patient days, mean (SD), range	329.1 (535.9) 0 – 8333	225.0 (331.7) 0 – 2500	***	13.1 (79.0) 0 – 1000	10.8 (67.4) 0 – 1000	0.89
Error rate per 100 medication order-days, mean (SD), range	4.03 (3.76) 0 – 35	4.48 (4.98) 0 – 50	0.15	3.03 (6.77) 0 – 37	2.22 (1.77) 0 – 8	0.43
Error rates per admission at stages of medication-management process, mean (SD), range
- ordering	0.65 (1.33) 0 – 13	0.79 (1.44) 0 – 10	0.55	0.01 (0.12) 0 – 1	0.02 (0.19) 0 – 3	0.99
- transcription	0.13 (0.45) 0 – 4	0 (0) 0	***	0.01 (0.07) 0 – 1	0 (0) 0	NA
- preparation	0 (0.06) 0 – 1	0 (0.06) 0 – 1	0.99	0 (0) 0	0 (0) 0	NA
- dispensing	0.49 (1.10) 0 – 12	0.16 (0.50) 0 – 5	***	0 (0.04) 0 – 1	0.01 (0.09) 0 – 1	0.99
- administration	0.83 (1.64) 0 – 18	0.56 (1.13) 0 – 12	**	0.04 (0.21) 0 – 2	0.05 (0.28) 0 – 4	0.99
- monitoring	0.01 (0.13) 0 – 2	0.02 (0.14) 0 – 2	0.88	0.01 (0.09) 0 – 1	0.02 (0.14) 0 – 2	0.76

^~p-values of t-tests are adjusted using the Sidak correction for multiple comparisons.

Bold indicates significant pre-post differences that are statistically significant at the p<0.01 (**) or p<0.001 (***) levels.

Because the number of medication orders decreased between the pre-EHR and post-implementation periods (Table 2), we also calculated error rates per 100 medication orders; however, we recognize that this adjustment may be confounded with the effect of the technology (see above description of EHR-related reasons for the decrease in medication orders). We did not find any differences between pre- and post-implementation error rates per 100 medication errors.

In the next step of the analysis, we examined errors at stages of the medication-management process. We used error rates per admission; the adjustment per admission is relevant for all stages of the process. Adjustment per 100 medication orders may be relevant for errors at the ordering stage but not for errors at the administration stage in which adjustment per 100 medication doses may be more appropriate.

Potential and actual preventable ADEs by stage of medication-management process

The analysis of errors across medication-management stages showed that, for potential and actual preventable ADEs, many errors occurred at the ordering stage. These errors did not change significantly after EHR implementation (p=0.55) (Table 3). Ordering errors associated with actual preventable ADEs were also unchanged (p=0.99). At the administration stage, errors associated with potential preventable ADEs decreased from 0.83 to 0.56 errors per admission (p=0.01). For actual preventable ADEs, there was no change (p=0.99). Dispensing errors associated with potential preventable ADEs decreased from 0.49 per admission to 0.16 (p<0.001). There was no change in actual preventable ADEs (p=0.99). Transcription errors disappeared after EHR implementation. Error rates at the preparation and monitoring stages were low and did not change for either potential or actual preventable ADEs.

In the remainder of the analysis, we analyze specific error types, particularly those found at the ordering and administration stages where most errors occurred.

Medication error types for potential preventable ADEs

Table 4 displays the frequencies of specific error types for potential preventable ADEs pre-EHR and post-implementation at various medication-management stages. Actual preventable ADEs are not included in this analysis because of their small numbers. We found several changes in specific error types at the ordering and administration stages. At the ordering stage, error-prone abbreviations and illegible orders disappeared post-implementation (p<0.001 and p=0.02 respectively), and there were fewer orders with omitted information (from 7% to 2%, p<0.001). On the other hand, we observed increases in wrong drug orders (from 2% to 5%, p<0.001), wrong start or stop times orders (from 0.4% to 6%, p<0.001), duplicate orders (from 2% to 11%, p<0.001), and orders with inappropriate or wrong information (from 4% to 11%, p<0.001). At the administration stage, late administrations were significantly reduced (from 27% to 17%, p<0.001), but there were more omitted administrations post-implementation (from 8% to 14%, p<0.001).

Table 4.

Medication error types in potential preventable ADEs pre-EHR and post-implementation

STAGES	ERROR TYPES	Pre-EHR		Post-implementation		p-value~
ORDERING	Order overdose	72	5%	77	8%	0.283
	Order underdose	37	3%	41	4%	0.772
	Omitted information	96	7%	19	2%	**
	Order wrong patient	5	0%	4	0%	1.000
	Order wrong drug	20	2%	49	5%	**
	Order, allergy	10	1%	13	1%	0.991
	Order, drug-drug interaction	1	0%	2	0%	1.000
	Error prone abbreviation	41	3%	0	0%	**
	Illegible order	16	1%	0	0%	*
	Order wrong start or stop times	5	0%	58	6%	**
	Duplicate orders	26	2%	107	11%	**
	Failure to renew order	15	1%	0	0%	*
	Inappropriate or wrong information	55	4%	105	11%	**
	Order-other	11	1%	17	2%	0.704
TRANSCRIPTION	Transcription	84	6%	0	0%	**
PREPARATION	Preparation	3	0%	2	0%	1.000
DISPENSING	Not dispensed or dispensed late	300	23%	96	10%	**
	Dispensing-other	8	1%	6	1%	1.000
ADMINISTRATION	Admin overdose	9	1%	2	0%	0.967
	Admin underdose	3	0%	2	0%	1.000
	Admin wrong patient	1	0%	1	0%	1.000
	Admin wrong drug	1	0%	1	0%	1.000
	Omitted administrations	105	8%	132	14%	**
	Late administrations	363	27%	162	17%	**
	Incorrect documentation	19	1%	32	3%	0.094
	Admin, no order	7	1%	8	1%	1.000
	Admin, allergy	3	0%	0	0%	0.988
	Admin, other	8	1%	10	1%	1.000
MONITORING	Monitoring	9	1%	11	1%	0.999
	TOTAL ERRORS	1,333	100%	957	100%	-

^~p-values of proportional difference tests are adjusted using the Sidak correction to account for multiple comparisons (29 tests).

Bold indicates pre-post differences that are statistically significant at the p<0.05 (*) or p<0.01 (**) levels.

DISCUSSION

In this study of EHR implementation in two ICUs, we found mixed evidence about the overall reduction in potential patient harm due to statistically significant reductions in the number and severity of potential preventable ADEs as well as rates per admission and per 1,000 patient-days, but not for error rate per 100 medication orders. It is important to recognize that the error rate per 100 medication orders may be confounded with the effect of the technology. As described above, several EHR-related reasons explain the decrease in medication orders; for instance, paper order sheets had more separate medication orders as compared to electronic order sets. This may partly explain the lack of statistically significant reduction of error rate per 100 medication orders. Actual preventable ADEs did not show statistically significant differences, at least in part due to their small numbers. We did not observe consistent improvements across medication-management stages or error types. Error rates for potential preventable ADEs were unchanged at the ordering, preparation and monitoring stages. Errors decreased at the transcription, dispensing and administration stages. These changes, however, hide variation in the error types that were affected by the EHR implementation. Therefore, our subsequent analysis of specific error types provides more nuanced information about the impact of EHR technology and opportunities to make its design, implementation and use safer and more effective.

As expected, the EHR technology eliminated error-prone abbreviations, illegible orders and transcription errors, and significantly reduced orders with omitted information as a consequence of structured order entry templates in the CPOE component of the EHR. Other positive changes included significant reductions in medications that were not dispensed or dispensed late, as well as fewer late medication administrations. This reduction in late medication administrations is consistent with other analyses from our larger study on CPOE/EHR implementation in ICUs (http://cqpi.wisc.edu/computerized-provider-order-entry-in-icus.htm) where we found improved timeliness of antibiotic medication administration after EHR implementation [33].

The EHR also led to increases in several types of medication error. In the ordering stage, wrong drug orders, wrong start or stop times orders, duplicate orders, and orders with inappropriate or wrong information increased. In the administration stage, omitted administrations and incorrect documentation of medication administration increased. The increase in orders with wrong start or stop times is consistent with research by Beuscart-Zephir and colleagues [34] who described the challenges of medication scheduling with CPOE implementation. They contrasted the synchronous and iterative nature of communication and coordination pre-CPOE implementation with the asynchronous work organization of physicians and nurses post-CPOE implementation. The CPOE technology became the main coordination mechanism in the medication-management process, which produced challenges in scheduling medications. Specifically CPOE may require physicians to enter administration-scheduling information for which nurses or pharmacists have more contextual knowledge and experience. This may also be relevant for the increase in orders with inappropriate or wrong information. In a previous analysis of our medication safety data [23], we described the duplicate order increase and the various system factors that contribute to those errors, such as poorly designed alerts (e.g., confusing content, high false-positive rate), and unclear roles during interdisciplinary team rounds.

Using the SEIPS model [11 12], we provide information about the potential EHR-related mechanisms that could have contributed either positively or negatively to medication errors (Table 5). We also describe implications and recommendations for human factors engineering-based design and implementation of EHR technology.

Table 5.

Potential Contributions of EHR Technology to Medication Error Types and Recommendations

Changes in Error Types	Possible EHR-Related Mechanisms	Human Factors Engineering (HFE) Implications and Recommendations
ORDERING
orders with omitted information	- EHR medication ordering screens provide guidance for entering all necessary order-related information. - Hard stop – all required information fields must be filled for order to be signed. - Default dose, route, frequency information for medication orders	- The structure of the EHR interface facilitates entry of required information. However, we need to recognize the potential negative consequences of this structure as well as hard stops (see increase in orders with inappropriate or wrong information). - Hard stops may produce questionable information being entered in the EHR as prescribers are trying to accomplish tasks, sometimes under time pressure. - The provision of default values may reduce omitted information, but may be accepted without sufficient thought [39 40].
wrong drug orders	- Medication selection errors due to pick-lists of medications - Wrong patient - Wrong drug for patient	- Relevant contextual information should be provided during ordering (e.g., current list of medications, previously discontinued medications). - Need for human-centered design of pick lists, including presentation of items on pick list (e.g., need for scrolling, most common medications at top of list) and use of TallMan lettering. - Prescribers could see only one patient at a time, but wrong patient ordering still occurred, possibly related to interruptions. - Provision of clinical decision support (e.g., patient with renal failure) may help to reduce wrong drug ordering for patient. - Making picture of patient visible may help during ordering.
orders with error-prone abbreviations	- Design of EHR technology eliminates error-prone abbreviations.	- The EHR design can eliminate errors at the source. - Need to monitor for re-introduction of ambiguous abbreviations.
illegible orders	- EHR technology eliminates handwriting and makes orders legible.	- The EHR design can eliminate errors at the source.
orders with wrong start or stop times	- The medication order includes default start times for all medication orders and stop times for selected medications. Nurses and pharmacists previously specified start times for medication administration. - Default start times were very soon after the order was entered. Physicians did not always change inappropriate start times and accepted default start times even when not appropriate (for example, a new order is placed to decrease a medication dose with a start time within the hour but the medication dose from the prior order was just administered; so the new dose should start when the next dose was due, not immediately).	- Implementation of EHR technology needs to consider the various roles involved in the medication-management process and clarify who in the process should be responsible, for instance, for scheduling medication administrations. Physicians may be responsible for deciding number of days for a medication order. Deciding on specific start or stop times may be assigned to pharmacists and/or nurses. - Making it more obvious when default start/stop times are assigned and changed may help the prescribers in ensuring adequate consideration of these default values.
duplicate orders	- Identical duplicate orders - Duplicate orders with same medication - Duplicate orders with medication of same therapeutic class [23]. Work system factors that contributed to duplicate orders include [23]: - Multiple clinicians entering orders at the same time on the same patient (e.g., during rounds). - Not being able to see relevant context (e.g., recently placed orders) when ordering. - Duplicate order alerts with high false positives leading to alert fatigue and ignoring true alerts. - Unusable duplicate order alerts.	- It is important to design not only the EHR technology, but also the rest of the sociotechnical work system [11 12]. - Proactive risk analysis should be conducted before implementation [37 38]. - After implementation, continued attention needs to be paid to EHR usability and other HFE issues of continuous technology implementation [41 42].
failure to renew orders	- Requirements to renew orders for certain medications were eliminated.	- A consequence of the elimination of renew orders was the continuation of medications for longer than necessary, e.g., intravenous fluids.
orders with inappropriate or wrong information	- This may be related to the decrease in orders with omitted information. A negative consequence of structure in the EHR interface and of hard stops may be inappropriate or wrong information entered. - Some users may use free-text fields rather than the template sections.	- Any functionality of the EHR technology has potential positive and negative consequences. Therefore, proactive risk analysis needs to be performed before the EHR technology is implemented [37 38]. - After EHR implementation, continued attention needs to be paid to the use of the technology; this is in line with a continuous technology implementation process [41 42].
TRANSCRIPTION
transcription errors	- EHR eliminates need for transcription.	- Whenever appropriate, eliminate or automate steps that are potential sources of error.
DISPENSING
dispensing errors	- More efficient and timely flow of information from prescribers to pharmacy system; therefore, pharmacy dispenses medications more quickly.	- Interoperable health information technology can eliminate inefficiency and errors due to paper processing.
ADMINISTRATION
omitted administrations	- Tighter coupling between time order is placed and scheduled first dose administration time. - This may be related to the increase in orders with wrong start times.	- See recommendations about how to reduce orders with wrong start or stop times, including the need to consider the work system and the medication-management process as wholes.
late administrations	- Increased awareness of medication administration times due to design of technology - This may be related to the decrease in dispensing errors, especially late medication dispensing.	- EHR technology may eliminate inefficiency due to paper processing and support more timely medication administration. - Improvement in one part of the process may help improve a subsequent part of the process.
incorrect documentation of medication administration	- Medication administrations recorded in one EHR component (e.g., respiratory therapy notes) may not be recorded in another component (e.g., MAR).	- Eliminate need for duplicate documentation.

Overall, the EHR implementation in two ICUs of a community teaching hospital reduced the level of harm associated with potential and preventable ADEs. Our results go beyond current research by providing information on specific error types at different stages of the medication-management process that are either positively or negatively influenced by the EHR implementation. This list of errors (Table 5) can be useful to healthcare organizations interested in improving medication safety; they can focus on errors that are more likely to occur post-implementation and identify mechanisms for eliminating or mitigating these errors. Healthcare organizations can use proactive risk assessment methods to anticipate and identify potential vulnerabilities and safety concerns before a new technology is implemented.[35 36] Bonnabry and colleagues [37] used FMECA to quantify the potential risk associated with the implementation of CPOE. Our team developed and tested an efficient proactive risk assessment method that can be used before health IT implementation.[38] The error list can also be used by health IT vendors in their quality assurance process to assess their technology and its potential impact on patient safety.

Study limitations

Study limitations include data collection at only two ICUs of a single medical center; therefore, this may limit generalizability of results. However, this study provides useful information about the impact of a commercial EHR system on medication safety, whereas most research has studied homegrown systems [5]. Data were collected between 2006 and 2008 and, therefore, have aged: the EHR technologies have evolved and many new drugs are now being used in ICUs. Therefore, this represents a significant limitation of the study. However, the study, but haveprovides important information about the unique characteristics of error types and the sociotechnical system in which the technology was implemented. T; this allows us to further understand the sociotechnical impact of CPOE/EHR technology on medication safety, in particular in ICUs. Given the increasing attention on EHR safety, our data have important implications for the ways in which the EHR technology can influence either positively or negatively the medication-management process. Because of the pre/post methodology without a control group, we cannot exclude that secular changes (e.g., use of more risky medications, different levels of severity of illness) could contribute to some of the differences seen between the pre- and post-implementation. The study was not powered to show reduction in the rate of actual preventable ADEs.

CONCLUSION

The implementation of EHR technology in two ICUs of a medical center showed mixed evidence for a reduction in overall potential harm related to medication errors. The EHR technology implementation eliminated errors at certain stages of the medication-management process (e.g., transcription) or certain types of errors (e.g., error prone abbreviation, illegible orders), but also led to increases in certain other types of errors (e.g., duplicate ordering errors, omitted administrations). Analysis of specific errors allows a more refined understanding of the impact of EHR technology on medication safety as compared to assessment of overall error rates. Further effort aimed at improving medication safety should focus on improvement in the design and implementation of EHR technology and the associated sociotechnical work system, as well as on the development of other solutions that target the most frequent medication error types still present post-implementation (e.g., duplicate ordering errors, inappropriate or wrong information in order, omitted and late administrations).

Appendix. Description of the medication-management process pre- and post-implementation

Medication management pre-implementation

Orders were entered into a paper chart kept at a central ICU location. Pre-printed order sets were available for post-procedure/surgery orders and were required for all admission, antibiotic, initial insulin and heparin continuous-drip orders. Verbal orders were discouraged and rarely used. Physicians and PAs were expected to relay STAT (urgent) orders directly to a nurse. For medication orders, nursing staff or the unit desk clerk either: 1) gave the carbon copy of the order to the ICU pharmacist on the unit (weekdays) or 2) sent the carbon copy or a photocopied order to the central pharmacy (nights and weekends). Pharmacists reviewed all medication orders for appropriateness, therapy duplication, and required therapeutic substitutions. They entered the orders into the pharmacy computer system (BDM RxTFC ® now Centricity, GE Medical Systems), which performed drug-allergy, drug-drug and drug-food interaction checks. A nurse or unit desk clerk hand transcribed all medication orders onto the paper medication administration record (MAR) from the paper order and pasted the printed medication label (when available) onto the MAR. This MAR transcription was double-checked for accuracy by a nurse. The nurse entered administration times based on hospital standards. A limited number of medications were available for direct dispensing by the ICU nurse via an automated dispensing machine (AccuDose-® ^™, McKesson Corporation, San Francisco, CA). Other medications were prepared and dispensed from the central pharmacy via the tube system or a robot dispenser. STAT medications were prioritized for ICU delivery within one hour of order arrival. Nurses administered medications and recorded the administration time and their initials on the paper MAR kept in the medication room. If a dose of a medication was not available when needed, a nurse sent a paper request form to the pharmacy or called the pharmacy directly. AICU nurses tended to record the time of administration in the MAR as the scheduled administration time unless the time was more than 2 hours different from the scheduled time. CICU nurses documented the actual time of administration to the nearest quarter of the hour. AICU physicians periodically rounded with the AICU pharmacist and reviewed the MAR on rounds. Providers were required to renew antibiotics orders every 5 days and intravenous fluids and continuous drip medications every 3 days. Nurses left sticky notes on the chart to remind the providers about reordering medications.

Medication management post-implementation

Providers entered medication orders directly into the computer system. Most commonly, the provider would enter the first part of the medication name into a search field and then chose a medication, dose form and route from a pick-list of medications generated by the computer. Selecting a medication produced an order screen, many of which had been built with default doses, dosing units, routes and frequencies based on the most common order for the medication. A default dose, route or frequency could be changed by clicking on a labeled button with another potentially appropriate choice or by entering the desired value into a free text field. Dosing units were selected from a drop-down field. The provider could also choose pre-built medication orders from a preference list or an order set. The computer provided a default start time and administration schedule, listing the first dose to be given at the next 5-minute interval of time from the time the order was placed and to continue on the routine administration schedule. All of these fields could be changed, by selecting another available option or by entering free-text administration instructions for the medication or comments about the order. The patient’s allergies appeared in red at the top of the computer screen at all times during ordering. Upon electronic signature, the computer system performed drug-allergy, drug-drug interaction and duplicate checking, and displayed medication alert pop-up messages to the ordering provider based on these checks. Alerts contained: 1) the type of alert (e.g. duplicate order), 2) information about what triggered the alert, 3) the medication order(s) triggering the alert with medication name and route form, and icons to show the existing and new orders and the option to discontinue the order, and 4) a field to enter the reason the alert was overridden if other action was not taken to cancel or modify a medication order. The pharmacist also received medication alerts and could review the provider’s response to an alert. For medication orders with overridden alerts, an icon appeared on the MAR to make other users aware of the override. In addition to computer checking, pharmacists continued to review all medication orders for appropriateness. The pharmacy computer system was integrated with the medication ordering system, so no transcription was required. The medication order populated the integrated eMAR that displayed scheduled administration times. Nurses could record a medication administration time, either by allowing the computer to use the current date/time stamp or by entering the actual medication administration time; this was a change in practice for nurses. The eMAR alerted nurses to overdue medication administrations when they entered a patient record.