Transitioning Between Electronic Health Records: Effects on Ambulatory Prescribing Safety

ABSTRACT

BACKGROUND

Healthcare providers previously using older electronic health records (EHRs) with electronic prescribing (e-prescribing) are transitioning to newer systems to be eligible for federal meaningful use incentives. Little is known about the safety effects of transitioning between systems.

OBJECTIVE

To assess the effect of transitioning between EHR systems on rates and types of prescribing errors, as well as provider perceptions about the effect on prescribing safety.

DESIGN, PARTICIPANTS

Prospective, case study of 17 physicians at an academic-affiliated ambulatory clinic from February 2008 through August 2009. All physicians transitioned from an older EHR with minimal clinical decision support (CDS) for e-prescribing to a newer EHR with more robust CDS.

MAIN MEASUREMENTS

Prescribing errors were identified by standardized prescription and chart review. A novel survey instrument was administered to evaluate provider perceptions about prescribing safety.

KEY RESULTS

We analyzed 1298 prescriptions at baseline, 1331 prescriptions 12 weeks post-implementation, and 1303 prescriptions one year post-implementation. Overall prescribing error rates were highest at baseline (35.7 per 100 prescriptions, 95% confidence interval (CI) 23.2–54.8) and lowest one year post-implementation (12.2 per 100 prescriptions, 95% CI 8.6–17.4) (p < 0.001). Improvement in prescribing safety was mainly a result of reducing inappropriate abbreviation errors. However, rates for non-abbreviation prescribing errors were significantly higher at 12 weeks post-implementation than at baseline (17.7 per 100 prescriptions, 95% CI 9.5–33.0 versus 8.5 per 100 prescriptions, 95% CI 4.6-15.9) (p <0.001) and no different at baseline than one year (10.2 per 100 prescriptions, 95% CI 6.2–18.6) (p = 0.337). Survey results complemented quantitative findings.

CONCLUSIONS

Results from this case study suggest that transitioning between systems, even to those with more robust CDS, may pose important safety threats. Recognizing the challenges associated with transitions and refining CDS within systems may help maximize safety benefits.

INTRODUCTION

Improving healthcare safety is a national priority, and e-prescribing is viewed as an important tool in these efforts^1–2. Research on the ability of e-prescribing to improve safety has predominantly come from the inpatient setting and on locally developed systems created by organizations for their own use^3–5. Fewer studies have been conducted on commercial systems or in the outpatient setting and results have been mixed^6–13. To ensure that federal spending is directed toward effective interventions, it is important to evaluate the effect of e-prescribing on safety in the ambulatory setting, where most prescribing occurs and errors are common^7,12–15.

Use of e-prescribing in the ambulatory setting has been low, although increased use is expected given federal incentives for meaningful EHR use^2,16–20. To demonstrate meaningful use, providers will have to meet certain criteria, including use of e-prescribing²¹. For providers transitioning from paper, commercial EHRs with e-prescribing are likely to be adopted because they are readily available. Some organizations using locally developed systems are also transitioning to commercial systems because locally developed systems, although uniquely customized, require great initiative to maintain and may not follow national standards in interoperability and function^22,23. Healthcare providers may also need to transition to newer versions of existing systems to meet meaningful use requirements.

Implementation of new systems is traditionally challenging and the effect of transitioning between systems on prescribing errors is unknown. Understanding the effects will be informative for those undergoing this type of transition and allow potential safety threats to be better managed^24,25. We therefore conducted this study to examine the effects on prescribing safety of transitioning between two systems in an academic-affiliated ambulatory practice.

METHODS

Study Design

We conducted this prospective study, approved by the institutional review board of Weill Cornell Medical College, of ambulatory care providers using a pre-post design. We analyzed electronic prescriptions on the older system just prior to implementation of the newer system and at twelve weeks and one year post-implementation. We also administered a novel survey to assess consented provider perceptions of the effect on prescribing safety.

Definitions

The Institute of Medicine defines medication errors as any error in the medication process (prescribing, transcribing, dispensing, administering, and monitoring)²⁶. We focused only on prescribing errors as defined in Table 1^7,13,26.

Table 1.

Types of Prescribing Errors

Type of Error	Definition	Example
Prescribing error	Any error in the prescription writing process	Ordering a medication but omitting quantity to be dispensed
Near miss	Potentially harmful errors that were either intercepted or reached the patient but did not cause harm	Ordering amoxicillin for a patient with a known allergy who could not fill the prescription due to pharmacist error detection
Preventable adverse drug event	Injury from a medication as a result of an error	Ordering amoxicillin for a patient with a known allergy who receives the medication and develops anaphylaxis
Rule violation*	Failure to follow strict prescribing rules that were unlikely to result in harm	Failure to write “po” as route for a medication taken only be mouth

As in other published studies, rule violations were not counted as errors^7,13

We use the term e-prescribing to describe only the electronic ordering of medications, regardless of whether prescriptions are printed or electronically transmitted to a pharmacy, as we are focused on the prescribing stage. Finally, we use the term CDS to describe support to improve clinical decision-making related to therapeutic processes of care, for example alerts to identify drug-allergy interactions²⁷.

Background and Setting

We studied faculty providers at an academically-affiliated, hospital-based adult internal medicine ambulatory practice in New York City. All 19 faculty members working 75% time or more and at least two clinic sessions per week were included.

Transition from the Older to Newer System

Providers transitioned from the older to the newer system, whose use was mandatory, in April 2008. The information systems team conducted a large-scale, intensive effort to transition providers, including transferring medication data between systems. All providers were required to attend multiple, 2-hour training sessions prior to go-live. Provider schedules were minimized by 50% for the first two weeks and 75% for the third week of the go-live period. Implementation teams were present on-site during go-live and held weekly sessions for the first month after implementation to answer providers’ questions.

Older System

The older PC-based EHR system was developed by one of the providers at the study site and implemented in 1993. The system had order entry, free-text progress notes, flow sheets, laboratory result viewing, clinical messaging, and scheduling and registration capabilities. Usage for prescribing was near-universal for non-narcotic prescriptions. The only CDS was default formulations (the most commonly used formulation would automatically pre-populate when the medication name was entered). The medication database was managed centrally by the developer but allowed “work-arounds” for free-texting of medications. Prescriptions could not be sent electronically to pharmacies.

Newer System

The newer system is a Certification Committee for Health Information Technology (CCHIT)-certified commercial EHR with e-prescribing²⁸. CCHIT is a federally recognized EHR certification body. In addition to the features offered by the older system, the newer system has additional CDS including alerts for allergies and drug-drug interactions. Prescriptions can be sent electronically to pharmacies. Providers can create preference lists (lists of frequently used orders) and order sets (pre-populated groups of medications). The medication master-file and CDS logic is derived from a third-party supplier.

Data Collection and Review

Prescription Collection

Electronic prescriptions were extracted from each EHR’s database for a two week period in three study intervals: baseline, 12 weeks post-implementation, and one year post-implementation. We chose 12 weeks because it was early after go-live but allowed providers a brief period to acclimate, and one year to ensure a more prolonged period of system use. We excluded prescriptions written by residents. We obtained a minimum of 75 prescriptions on 25 patients per provider, extending data collection beyond two weeks if necessary.

Prescription Review

Two nurse reviewers were trained in an identical manner by R.K. with extensively utilized and standardized methodology^7,29–32. This included review of error definitions, test, and actual cases. Both nurses jointly reviewed cases initially; these cases were then reviewed by R.K to ensure proper classification. Reviews were then conducted separately. Conflicts were resolved through discussion and consensus between R.K. and the nurses. Methodology included error classification and identification of ADE trigger drugs²⁹. We determined inter-rater reliability by having both nurses review the same random sample of fifty prescriptions. Inter-rater agreement for overall error and error type was 1.0, indicating excellent agreement.

Of note, our methodology is primarily designed to detect prescribing errors and rule violations, not near misses and preventable ADEs, which are best detected by chart reviews for every prescription or patient surveys.

Inappropriate Abbreviations

In 2004, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) established a “Do Not Use” list for abbreviations with great potential to cause errors³³. A ban of these was locally instituted. As a result, the older system was modified to automatically change inappropriate abbreviations into acceptable terms on printed prescriptions (for example “QD” printed as “once daily”). However, providers were not required to upgrade to this modified version and we were unable to ascertain from electronic downloads which system version was used and therefore whether the inappropriate abbreviation had been automatically corrected.

Thus we classified all inappropriate abbreviations from the older system as errors. In the newer system, providers were required to override two separate alerts (at ordering and signing) to use an inappropriate abbreviation, however there was no automatic conversion.

Chart Review

The research nurse performed ambulatory chart reviews for suspected near misses or when a drug often used to treat an ADE was prescribed.

Physician Event Review and Classification

Two physicians independently reviewed all suspected near misses and ADEs. Confirmed ADEs and near misses were rated on preventability using a five-point Likert scale and attribution using the Naranjo algorithm³⁴. ADE severity was rated using a four-point Likert scale. Inter-rater agreement for the presence of prescribing errors and near misses was 0.96 and 0.93, indicating excellent agreement.

Survey

We developed a five-page survey based on literature review and consultation with experts in informatics, clinical medicine, and medication safety. Questions focused on physicians’ backgrounds, implementation experiences, and prescribing using the two systems. This confidential survey was administered from January through June 2009 as background for part of a large qualitative study conducted with the same providers³⁵. The survey was piloted with five pediatricians in the same ambulatory care network. We distributed the paper survey and a $10 gift card to eligible physicians. We sent bi-monthly email reminders and visited the site to promote survey completion.

Statistical Analysis

For data management and descriptive statistics, including the survey, we used SAS 9.2 (SAS Institute Inc., Cary, NC). This included comparisons of patient characteristics across the three time points using chi-squared tests for categorical variables and ANOVA for continuous variables. We considered significant findings at the 0.05 level. We used Poisson regression to estimate error rates per 100 prescriptions at the three time points, while adjusting for patient characteristics (age, insurance, and gender). In the model, we compared rates at 12 weeks and 1 year to baseline. We also calculated 95% Poisson confidence intervals for the rates. Using generalized estimating equations, we adjusted for clustering, using provider as the unit of analysis. To reduce the amount of patient clustering, we reviewed a maximum of three prescriptions per patient during data collection. We assumed an independent correlation structure for the Poisson regression. The primary Poisson model was analyzed using Stata 10 (StataCorp, College Station, TX).

RESULTS

Provider Characteristics

Nineteen providers participated in the baseline and 12-week post-implementation data collection periods. Seventeen providers participated at one year (one provider was on maternity leave and one was on sabbatical). Only providers included in all three time points were part of the analysis. Provider characteristics are presented in Table 2.

Table 2.

Healthcare Provider Characteristics

		All Providers (N = 17)
Age, Mean (SD)		46 (8)
Female		7 (41%)
Years since graduation, Mean (SD)		20 (7)
Table 1b: Patient Characteristics
	Baseline†	12 Weeks‡	1 Year§	p-value║
N	646	735	715
Age, Mean (SD)	60 (16)	59 (17)	58 (17)	0.008
Female	426 (66%)	475 (65%)	468 (65%)	0.9
Medicaid or Medicare	284 (44%)	331 (45%)	272 (38%)	0.02

^†Baseline: missing three values for insurance.

^‡12 Weeks: missing 1 value for age, 1 value for gender, and 2 values for insurance.

§1 Year: missing 6 values for insurance.

║ANOVA for continuous variables; CHI squared for categorical variables

Patient Characteristics

We reviewed prescriptions for 646 patients at baseline, 736 patients at 12 weeks post-implementation and 715 patients one year post-implementation (Table 2). There was no significant difference in patient gender across time periods. Patients from the baseline period were older than patients at other time periods.

Rates of Errors

We reviewed 1298 prescriptions at baseline, 1331 prescriptions 12 weeks post-implementation, and 1303 prescriptions one year post-implementation (Table 3). Overall rates of prescribing errors were highest at baseline (35.7 per 100 prescriptions) and significantly lower at 12 weeks (21.1 per 100 prescriptions, p < 0.001) and one year (12.2 per 100 prescriptions, p < 0.001) post-implementation compared to baseline. Inappropriate abbreviation errors were also highest at baseline (24.1 per 100 prescriptions) and significantly lower at 12 weeks (10.6 per 100 prescriptions, p < 0.001) and 1 year (5.9 per 100 prescriptions, p <0.001) post-implementation. Rates of near misses and rule violations did not significantly differ between time periods. No preventable ADEs were detected.

Table 3.

Comparison of Error Rates Between Baseline, 12 Weeks, and 1 Year

	N*				Rates per 100 Prescriptions (95% CI) †
	Total	Baseline	12 Weeks	1 Year	Baseline	12 Weeks	p-value║	1 Year	p-value¶
N	--	--	--	--	1298	1331	--	1303	--
Prescribing errors‡	1038	541	316	181	35.7 (23.2,54.8)	21.1 (14.0,31.9)	<0.001	12.2 (8.6,17.4)	<0.001
Inappropriate abbreviation errors	849	504	223	122	24.1 (15.1,38.5)	10.6 (6.4,17.4)	<0.001	5.9 (4.0,8.9)	<0.001
Non- abbreviation prescribing error§	219	50	106	63	8.5 (4.6,15.9)	17.7 (9.5,33.0)	<0.001	10.8 (6.2,18.6)	0.337
Rule violations	335	106	105	124	45.1 (28.8,70.7)	41.4 (28.2,60.8)	0.627	48.0 (30.1,76.5)	0.763
Near misses	67	25	23	19	8.0 (1.6,39.1)	6.9 (1.7,27.3)	0.555	5.6 (1.2,25.8)	0.146

* Number of prescriptions with errors

† Calculated using Poisson regression, adjusting for clustering and patient characteristics

‡ Excluding rule violations

§ Without errors due to inappropriate abbreviations; excluding rule violations

║ p-value for 12 weeks to baseline comparison

¶ p-value for 1 year to baseline comparison

Types of Prescribing Errors

At all three time periods, inappropriate abbreviations constituted a majority of the prescribing errors (Table 4). Direction and frequency errors were also common. Examples of errors are presented in Table 5. Oral diabetic agents, antibiotics, narcotic analgesics, and topical steroids and antifungals were most frequently associated with prescribing errors. Sumatriptans, phosphodiesterase inhibitors, topical anesthetics and narcotic analgesics were most frequently involved in near misses.

Table 4.

Types of Errors

	Prescribing Errors			Near Misses
	Baseline	12 Weeks	1 Year	Baseline	12 Weeks	1 Year
N	557	338	191	25	23	19
Inappropriate use of abbreviation	506 (91%)	227 (67%)	124 (65%)	--	--	--
Directions error	31 (6%)	74 (22%)	43 (23%)	15 (60%)	6 (26%)	9 (47%)
Frequency error	11(2%)	23 (7%)	7 (4%)	8 (32%)	8 (35%)	7 (37%)
Dose error	3 (1%)	8 (2%)	0 (0%)	2 (8%)	5 (22%)	3 (16%)
Route error	--	--	--	0 (0%)	1 (4%)	0 (0%)
Amount to be dispensed error	2 (0%)	1 (0%)	17 (9%)	--	--	--
Other	4 (1%)	5 (1%)	0 (0%)	0 (0%)	3 (13%)	0 (0%)

Table 5.

Examples of Prescribing Errors and Near Misses

Prescribing Errors
Metformin 500 MG Tab, 1 QD	Use of an inappropriate abbreviation ‘QD’
Lariam® 250 mg, 1 tab weekly for malaria prophylaxis	Incomplete directions, as start of medication and completion of its course needs to be in relation to expected travel time to the malaria endemic region
Near Misses
Imitrex® 100 mg, 1 Tab for Migraine, may repeat	Frequency for medication use is incomplete. No daily maximum or interval time between doses specified
Colchicine 0.6 mg PRN for gout flare	Frequency or maximal allowable daily dose not specified
Viagra® 100 mg, 1 Tab 1 hour prior to sexual intercourse	Maximal daily dose or interval between doses not specified

Rates of Errors Excluding Inappropriate Abbreviations

Because the majority of prescribing errors were inappropriate abbreviations and later versions of the older system automatically corrected these (although we were unable to ascertain whether this correction occurred), we performed a separate analysis excluding inappropriate abbreviations from error counts. With this analysis, the overall prescribing error rate was lowest at baseline and highest 12 weeks post-implementation (8.5 versus 17.7 errors per 100 prescriptions, p < 0.001) (Table 3). Error rates at one year were not significantly different than at baseline (8.5 versus 10.8 errors per 100 prescriptions, p = 0.337).

Survey Results

We achieved a 79% response rate (n = 15) for the survey. Physician satisfaction regarding implementation of the newer system was mixed. Forty seven percent of physicians reported being very or somewhat satisfied with implementation (n = 7), while 40% (n = 6) reported being somewhat or very dissatisfied. Two-thirds of providers (n = 10), however, felt that training on the new system was rarely or never a problem.

With regard to safety, only one-third (n = 5) of physicians felt that the newer system improved safety compared to the older system, despite its more robust CDS. Alert fatigue was also widespread. While 53% of physicians (n = 8) reported that alerts were triggered for at least 50% of prescriptions, nearly two-thirds (n = 9, 60%) reported that alerts were rarely or never useful. In terms of impact on workflow, two thirds of respondents (n = 10) reported that the speed in ordering and refilling medications was much worse or worse with the new system compared with the older system.

DISCUSSION

Our study is the first to our knowledge to quantitatively evaluate the effect on ambulatory prescribing errors of transitioning between e-prescribing systems. Implementation of the new, commercial system led to a significant and progressive decrease in overall rates of prescribing errors, largely by reducing inappropriate abbreviations.

Rates for non-abbreviation prescribing errors, however, were actually highest at 12 weeks post-implementation, suggesting that transitioning between systems may pose potential patient safety threats even for experienced e-prescribers. Previous studies have identified unintended negative consequences from the introduction and use of computerized systems for ordering medications, including the facilitation of errors^36,37. Also in our study, overall error rates for non-abbreviation errors were no different at one year and baseline, despite the fact that the old system had very limited CDS, although due to our small sample size, we may not have been able to detect a significant difference. This is despite a low overall prescribing error rates for non-abbreviation errors compared to rates in other published studies using identical methodology^7,13.

One of the major perceived safety benefits of e-prescribing is CDS to aid with medication ordering³⁸. However, the value of CDS is often limited by providers’ lack of alert acceptance. Multiple studies have shown that providers frequently override alerts because they are perceived as irrelevant^39–42. The inability of EHR systems to effectively present information for CDS purposes may contribute to the mixed safety benefits observed in studies^{1,7–10,12,13}. In our study, for example, the newer system generated two alerts at separate steps in the ordering process to target inappropriate abbreviations. While this led to an immediate reduction in inappropriate abbreviation use, inappropriate abbreviation errors still constituted the majority of prescribing errors, suggesting that the content or the presentation of the alerts had limited effectiveness in modifying prescribing behavior.

In a companion qualitative study that we did with the providers, findings from semi-structured interviews and field observations complement findings in this manuscript³⁵. We found that almost all providers, even though they were experienced e-prescribers, considered the transition difficult. Consistent with our survey findings, most physicians did not view the newer system as improving safety, despite more CDS features, and alert fatigue led to routine overriding of alerts.

Studying the types and frequencies of errors made using e-prescribing systems can allow for targeted improvements in system design, CDS presentation, and implementation. For example, the second most common error we detected was directions errors. Pre-printed templates with clear patient instructions may help eliminate these errors. By determining drug classes most frequently involved in errors, CDS can be developed specifically to reduce these errors. Targeted alerts for certain prescribing errors have been shown to be effective^8–10. In addition, it can also guide calibration of CDS such that there is a higher sensitivity to trigger alerts. Importantly, although the majority of the prescribing errors we detected lacked potential to cause serious harm, these errors can result in inefficiencies (such as pharmacy callbacks) and thus are important to study. Research has shown that pharmacy callbacks are common and often lead to delays in medication dispensing that pose threats to patient safety⁴³.

Limitations

Our study has several limitations. Providers were not blinded to the study’s purpose and may have been extra careful when prescribing, making our results conservative estimates of true error rates. We were also limited by our methodology to comment on near misses and preventable ADEs. However, given previous research demonstrating that 4% of patients experience a preventable ADE, it is likely that near misses and preventable ADEs occurred that we were unable to capture⁷. Additionally, because we were unable to ascertain from electronic downloads whether the older system automatically corrected inappropriate abbreviations, we may have overestimated these errors at baseline.

Our study was conducted in one clinic, limiting generalizability. We studied only two systems, although the commercial system is widely utilized and incorporates many features recommended by an expert panel⁴⁴. We also did not observe providers using both systems, nor do we have data logs tracking the frequency with which alerts were generated or over-ridden. Thus, our ability to comment on usage and usability of the systems is limited and can be derived only from survey data. Finally, due to the small sample sizes for non-abbreviation prescribing errors, we are limited in our ability to detect differences in error rates at different time periods. Future studies should be performed with more providers, multiple systems, and at diverse sites. Longitudinal studies should also be conducted to determine how error rates change over time as iterative refinements are made and providers become more familiar with a new system.

Lessons Learned

With federal incentives promoting meaningful use of certified EHRs, more organizations will likely transition between e-prescribing systems. Our results suggest that transitioning may lead to unintended negative patient safety consequences, particularly early post-transition. This is despite strategies such as pre-transferring medication data between systems, requiring providers to attend mandatory training sessions, and providing on-site support during and after go-live. Additional strategies may therefore be needed to make transitioning safer. For example, providers may need more individualized training and closer follow-up to address prescribing errors in a timelier manner.

Our study also suggests that organizations and vendors may have to better tailor the design and configuration of CDS to achieve greater safety gains. Focusing CDS toward certain types of errors, such as inappropriate abbreviation errors, may be one such strategy. Greater provider education on rates and types of prescribing errors will complement this strategy. Finally, given the rapidly evolving nature of e-prescribing adoption on a national level and the potential safety issues that may arise, it will be important for organizations to monitor safety issues and iteratively refine systems to ensure that adoption actually leads to safer healthcare delivery.