Using an Evidence-Based Approach to EMR Implementation to Optimize Outcomes and Avoid Unintended Consequences

Abstract

Implementation of an electronic medical record (EMR) with computerized physician order entry (CPOE) can provide an important foundation for preventing harm and improving outcomes. Incentivized by the recent economic stimulus initiative, healthcare systems are implementing vendor-based EMR systems at an unprecedented rate. Accumulating evidence suggests that local implementation decisions, rather than the specific EMR product or technology selected, are the primary drivers of the quality improvement performance of these systems. However, limited attention has been paid to effective approaches to EMR implementation. In this case report, we outline the evidence-based approach we used to make EMR implementation decisions in a pragmatic structure intended for replication at other sites.

Implementation of an electronic medical record (EMR) with computerized physician order entry (CPOE) and clinical decision support (CDS) can provide an important foundation for decreasing medication errors and harm.^1–3 However, in a recent simulation-based approach testing the medication-related CDS of CPOE systems at 62 different hospitals, the best performing hospitals represented six different software products, with vendor choice accounting for only 27 percent of observed performance variation.⁴ In this study, local implementation decisions emerged as the most influential driver of EMR-associated medication error reductions.

In the pediatric environment, Han et al. infamously described an increase in mortality associated with the implementation of a commercially available CPOE system in a pediatric intensive care unit (PICU).⁵ Del Beccaro et al. subsequently showed no increase in mortality following implementation of a CPOE system in their PICU,⁶ and most recently, we demonstrated a decrease in hospital-wide mortality following CPOE implementation at Lucile Packard Children’s Hospital (LPCH) at Stanford.⁷ The hospitals in these studies used the same software vendor, again suggesting that local implementation decisions are a critical factor in determining the safety performance of a CPOE system.

Similar to a previously described evidence-based approach to EMR system selection,⁸ the leadership at LPCH employed an evidence-based approach to EMR implementation. We focused on improving outcomes by simultaneously designing the EMR and care processes to achieve the desired benefits. At the same time, we implemented evidence-based recommendations for recognizing, categorizing, and preventing or mitigating unintended consequences.^9–13 The successful implementation of our EMR, at least based on our associated reduction in mortality,⁷ lends credence to these recommendations. In this case report, we review some of the evidence that informed our approach to EMR implementation. We detail our approach using a project management framework to maximize practical value to other organizations.

Background

LPCH is a freestanding quaternary care pediatric and obstetric hospital affiliated with the Stanford University School of Medicine. According to case mix index data, LPCH consistently ranks among the highest acuity of all freestanding children’s hospitals.¹⁴ In 2004, the hospital embarked on a multi-year EMR implementation. Support from the board of directors ensured that the project was sufficiently well resourced.¹⁵ The first phase focused primarily on replacement of a legacy hospital information system with software from a commercial vendor and went live in 2005. The second phase, which focused on implementing CPOE and clinical documentation to achieve improvements in quality, safety, efficiency, regulatory and research outcomes, went live in 2007.

Methods

I. Evidence-Based Planning

The planning phase of any clinical system implementation should be comprehensive. At LPCH, this included staff recruitment, project governance development, recognition and integration of change management concepts, and prioritizing benefits achievement outcomes. Each of these components is further described next.

Project Personnel

Assembling the right project team must have a high priority.^16,17 The team should blend background and experience, local environment knowledge and relevant education. At LPCH, several of our own employees had previous experience implementing CPOE at another site.¹⁸ We supplemented the team with consultants who had experience implementing CPOE in a pediatric environment using the same vendor software at other hospitals. The project also included well-respected subject matter experts from hospital operations. Although this team brought extensive knowledge about clinical workflows, none had informatics backgrounds, so their orientation to the project included a syllabus and journal club where selected fundamental topics in medical informatics were reviewed.^19,20 This forum was sustained throughout the project as a recurring event and contributed to a project-wide interest in using published evidence to guide our implementation decisions.

Project Governance

Input from physicians, nursing, IT, and project management is essential to an optimal clinical system implementation.¹⁶ An ineffective governance structure at the project outset can have negative downstream effects, including increased costs, missed deadlines, and a frustrated team. At LPCH, installing an experienced program director with single-point authority and accountability was a crucial component of the project governance structure. The program director defined the project teams and delegated each team to single leads that were supported by experienced project managers to manage budget, scope, deliverables and timeline. This clarified responsibilities and helped facilitate an effective planning process. Domain input from physicians, operations and technologists is necessary but not sufficient for success; effective project management is a learned and different skill. The program director reported to a steering committee, which included the Chief Operating Officer, Chief Information Officer and the Chief of the Medical Staff.

Change Management

Previous studies have cited the importance of change management in CPOE and EMR rollouts.²¹ At LPCH, we engaged a social psychologist to teach our leaders core concepts of change management.²² In addition, we also undertook two critical initiatives.

Firstly, preprinted, paper-based order sets were rolled out prior to CPOE go-live. The benefits of making these order sets available on paper were numerous, including enhancing medication safety, establishing the process for order set creation and maintenance,²³ and initiating some of the cultural changes necessary to successfully implement CPOE.²⁴ In particular, this provided the opportunity for medical staff to collaborate and agree on a standard of practice for certain procedures ahead of the system implementation.

Secondly, system improvements with beneficial workflow effects were rolled out prior to CPOE. One example was the development of an EMR-integrated physician rounding report directly inspired by the literature,²⁵ and later shown to benefit workflow at LPCH.²⁶ Other initiatives designed to increase enthusiasm and confidence prior to CPOE rollout included integration of an automated wireless lab notification system and electronic forms for manual entry of outside labs.^27,28 All of these literature-guided improvements helped clinicians gain confidence and enthusiasm that the system implementation could provide tangible benefits to care delivery.

Benefits Achievement

To ensure that the benefits of EMR implementation were adequately defined, measured and tracked, an outcomes dashboard was developed prior to the clinical phase implementation. In addition to metrics previously published,²⁹ the dashboard included balancing measures such as mortality and length of stay to ensure that major unintended consequences were not inadvertently overlooked.^9–13 Uniquely, a subset of this dashboard was used for financially incentivizing our vendor in addition to reporting to our hospital board.

II. Evidence-based Design & Build

Initial design and build of over 300 evidence-based electronic order sets was based almost exclusively on the paper order sets described previously. As a result, more effort could be focused on the design and build of CDS rules and alerts, which account for the majority of medication safety benefits realized through CPOE systems.² Unfortunately, these alerts also generate a disproportionately large number of unanticipated consequences, with alert overload a common feature of decision support systems using unmodified commercial rule bases.^12,30,31

At LPCH, we focused time and resources on defining the appropriate balance between over- and under-alerting our clinicians. Inspired by work by Bobb et al.,³² we developed a harm-reduction-based strategy to implementing medication-based CDS while mitigating alert fatigue. Based on potential adverse drug events caused at the prescribing stage, this strategy targets interruptive CDS rules and alerts to those areas of inpatient pediatric medication use with the highest potential for causing harm (see Table 1).^33,34 This approach is consistent with the paradigm shift in patient safety efforts from preventing errors to preventing harm.35

Table 1.

Summary of Harm-Reduction Based Strategy for Mitigating Medication Alert Fatigue

Type of Medication Error	Epidemiology of Harm [REFERENCE NUMBERS]	LPCH Decision
Dose range errors	34% of potential ADE’s (Kaushal)	Dose range checking alerts enabled for all disciplines
	82% of errant orders (Folli)
Known drug allergy	4.3% of potential ADE’s (Kaushal)	Drug allergy checking enabled for all disciplines
	0.4% of errant orders (Folli)
Drug interactions	Not noted (Kaushal)	Major drug-drug interaction alerts enabled for pharmacists only
	1.9% of errant orders (Folli)
Drug and therapeutic class duplication	Not noted (Kaushal or Folli)	Drug duplication alerts enabled for pharmacists only
Missing or wrong weight	0.86% of potential ADE’s (Kaushal)	Custom rule requires documented weight prior to medication order entry

Because workflow-integration has been shown to be the most effective means of delivering decision support, we also focused resources on developing non-interruptive CDS.³⁶ These include integration of external knowledge resources within order sets³⁷ and guided dosing suggestions that appear when an order is first selected.^38,39

III. Evidence-Based System Testing

In addition to the unit and integration testing methods used in most implementations, the project team was inspired by a paper reviewed at our journal club to employ less common testing methods including usability, workflow, simulation and continuity testing.⁴⁰ Although we used direct observation rather than video recordings, we found these testing events to be of great value in improving the system prior to activation, especially as it demonstrated the impact of human factors on the system design.

IV. Evidence-Based Training, Activation and Go-Live Support

The importance of comprehensive training has been well described.¹⁶ Our strategy included a mixture of web-based training and classroom content. “Super users” were identified as non-technical personnel facilitating classes after their own training. Physician training was performed by experienced super user clinicians, so most physicians were trained by physicians.

The approach to system activation was a topic of considerable discussion by project leadership. The group from Ohio State University described how they had activated an entire hospital at once.⁴¹ This big-bang strategy, which is actively promoted by commercial software vendors, had also been successfully employed at other children’s hospitals and was our original intent. However, this approach became a topic of increasing controversy after Han et al. described their “big bang”experiences and one set of experts argued that such a system rollout “goes beyond challenging and borders on the temerarious.”⁴²

Options for incremental activation include rolling out the system by functionality or by geographic unit. Our analysis suggested that the resulting blended paper-electronic system could potentially adversely impact patient safety and that both options would increase overall financial and human resources required for activation support. Given the integrated nature of orders and clinical documentation in the system, we decided that rolling out partial functionality was not a viable option. In the absence of solid data to make this decision, and in the context of financial constraints, we designed an acuity-based activation strategy. We started with a big-bang go-live for over 90 percent of inpatient beds, but spared the 24 beds in our highest acuity patient care units, which were subsequently activated in separate go-lives.⁷ We later published the details of this unique acuity-based activation strategy.⁴³ The importance of providing adequate support post go-live has also been well documented,¹⁶ and we staffed an on-site command center 24 hour/day, 7 days/week for one month after system activation.

V. Evidence-Based Optimization

A recent description of “moving from good to great” in electronic health records distinguished the best sites as those most committed to post go-live optimization.⁴⁴ Recognizing that the clinical transformation journey only begins with implementation, we established a new hospital-based Clinical Informatics department at LPCH with the explicit purpose of continuing system optimization after go-live. This department included nursing and medical staff considered to be both system and workflow experts, as well as clinical data analysts versed in mining clinical data for improvement purposes. Several of our successful endeavors to optimize workflow by leveraging the information system have also been published.^7,26,45–49

Results

Several critical outcome and process metrics were significantly improved as a result of our EMR implementation. For example, LPCH experienced a significant decrease in hospital-wide mortality associated with our EMR implementation.⁷ Other noteworthy metrics included CPOE rates in the pediatric units averaging 97 percent over the first 12 months of go-live, and turnaround times for lab, diagnostic radiology and medications all improving (Figure 1). Though the intent of this paper is to focus on the story of our implementation, these high-level results support the notion that our evidence-based approach to EMR implementation has resulted in substantial improvements of multiple process and outcome metrics.

Figure 1.

Critical Outcome and Process Metrics as a Result of EMR Implementation Of note, we did have one potentially preventable unintended consequence, which was a significant increase in catheter-associated bloodstream infection (CABSI) rates in the neonatal intensive care unit (NICU) associated with the time of our implementation. Subsequent activations in our pediatric and cardiac intensive care units were not associated with CABSI rate changes.

Discussion

This is the first description to our knowledge of an evidence-based approach to EMR implementation. Our experiences using this approach at LPCH resulted in the following lessons learned. First, trained informaticists must be involved with leading all aspects of EMR implementation and optimization.^16,50 Many health IT consultants without professional training are unaware of relevant lessons learned stretching back over 40 years.^51,52 For example, organizations with homegrown EMRs provided inspiration for features we architected into our commercial software,^27,28 and we found literature published from sites with the same commercially-available vendor system to be particularly helpful in guiding our decisions.^5,6,21,25,30 Secondly, understanding the local culture and establishing site-appropriate change management techniques clearly contributed to the successful adoption of our EMR, as reflected by our overall CPOE rates.

Finally, both positive and negative findings in the literature should be used to guide implementation. The project team aggressively focused on mitigating unintended consequences, utilizing evidence-based recommendations for recognizing, categorizing and preventing these issues.^9–13 Nevertheless, we still experienced an unintended consequence manifested as an increase in CABSI rates, which the NICU leadership attributed to a combination of the institutional focus on new workflows combined with an influx of less experienced staff to provide support during the training period. Recognition of this outcome as an unintended consequence allowed us to focus on successfully mitigating it during our subsequent go-lives in the remaining intensive care units.

Conclusion

Since research has shown that variability in outcomes is more related to implementation issues than vendor selection,^4,16 we believe that other hospitals embarking on this journey should consider a similar approach informed by published evidence.