Principles for a Successful Computerized Physician Order Entry Implementation

Abstract

To identify success factors for implementing computerized physician order entry (CPOE), our research team took both a top-down and bottom-up approach and reconciled the results to develop twelve overarching principles to guide implementation. A consensus panel of experts produced ten Considerations with nearly 150 sub-considerations, and a three year project using qualitative methods at multiple successful sites for a grounded theory approach yielded ten general themes with 24 sub-themes. After reconciliation using a meta-matrix approach, twelve Principles, which cluster into groups forming the mnemonic CPOE emerged. Computer technology principles include: temporal concerns; technology and meeting information needs; multidimensional integration; and costs. Personal principles are: value to users and tradeoffs; essential people; and training and support. Organizational principles include: foundational underpinnings; collaborative project management; terms, concepts and connotations; and improvement through evaluation and learning. Finally, Environmental issues include the motivation and context for implementing such systems.

INTRODUCTION

Computerized physician order entry (CPOE) continues to receive attention both because it has been shown to decrease medical errors [1–5], and because it has often been met with resistance on the part of users [6–9]. The National Library of Medicine awarded a three-year research grant to the POE Team (POET) at Oregon Health & Science University to do a field study to identify success factors for implementing CPOE. This was accomplished using two distinctly different approaches: a top-down assessment generated through a consensus conference of thirteen experts held in the spring of 2001; and a bottom-up, grounded theory, approach combining ethnographic and interview methods in the field. While there was considerable overlap and duplication of ideas in these two large data sets, there were also differences. Although each of the two sets of results was useful in its own right, members of POET felt compelled to reconcile them, to establish trustworthiness (the qualitative analog to validation) and to gain new insight.

METHODS

The Top-Down Approach

A two day meeting at the Menucha retreat center near Portland, Oregon involved invited experts from around the world representing multiple stakeholder groups: clinicians, social scientists, information technology implementers, and vendors. The format was designed to stimulate creative discussion and consensus building and included brainstorming and storytelling sessions as well as small group work. All sessions were audiotaped and transcribed.

The data were analyzed qualitatively [10]. All statements from the official typed notes and transcripts were printed out on separate cards (about 500 major statements). Five researchers used a card sort technique to produce ten categories. The conference attendees called them ‘Considerations’ because they should be pondered by those thinking about imp lementing CPOE rather than interpreted as strict guidelines. A consensus statement was generated after several months of online discussion. The list of ten Considerations is given in Table 1. Details are available as a list [at cpoe.org] and in text form [11].

Table 1.

The Ten Themes and Ten Considerations

The Ten Themes (Fieldwork)	The Ten Considerations (Experts)
Separating CPOE from other processes	Motivation for implementing CPOE
Isolating the CPOE process	Outside pressure
Defining the CPOE user	Patient care concerns
The CPOE path
Terms, concepts, connotations	CPOE foundations
Terms & concepts	Vision; Top-level commitment
Connotations	Resources; Trust
	Vendor stability
Context	Costs
Era; Institution; Unit;	Beyond hardware & software
Individual	Initial loss of productivity
Tradeoffs	Workflow & healthcare processes
Benefits	Impact on workflow & communication
Liabilities	Integration with other systems
Conflicts & contradictions	Value to users: decision support systems
Polarization	Alerts & reminders
Incongruities	Order sets; Guidelines
Collaboration & trust	Vision / leadership / people
Multidisciplinary collaboration	Administration
Trust	Physician leaders; Champions
	Clinical support staff
Leaders & bridgers	Technology
Leadership	Customizability
Bridging	Speed; Intuitiveness
	Decision support capability
Organization of information	Management of the project
Task individualization	Planning; Scope
Capture and retrieval of information	Involvement of clinicians
	Metrics for success
Ongoing nature of implementation	Training/support/help at the elbow
Ongoing user involvement	Adequacy & support of support staff/trainers
Ongoing modification
Temporal concerns	Learning/evaluation/improvement
Processing speed	Feedback & evaluation plan
Perception of time	Continuous improvement
Implementation life cycle

The Bottom-Up Approach

Four hospitals with successful CPOE implementations (defined as having over 80% of orders entered this way) were selected for field study based on geography, type of hospital, and length of system use. Two basic methods were used: ethnographic observation and interviews. Observation by research team members with different backgrounds used a common frame of reference and focus. Interviews were primarily oral history interviews of administrators, clinicians, and technology staff. Several focus group interviews were held with house officers and other clinicians.

Researchers typed their own fieldnotes; interview tapes were transcribed by experienced oral history transcriptionists. Data were entered into N5 (formerly QSR NUD*IST, Sage Publications) to assist analysis. The multidisciplinary research team members individually coded and analyzed the transcripts and fieldnotes, then met as a team to agree on overarching themes a total of 33 times. The ten Themes are outlined in Table 2 and have been described in prior papers [12–14].

Table 2.

The Twelve Principles

Mnemonic Grouping	The Twelve Principles
Computer Technology	Temporal concerns Meeting information needs Multidimensional integration Costs
Personal Issues	Value to users and tradeoffs Essential people Training and support
Organizational Issues	Foundational underpinnings Collaborative project management Terms, concepts, & connotations Improvement through evaluation & learning
Environmental Issues	Motivation & context

The Reconciliation

A meta-matrix was developed with the ten Themes along one axis and the ten Considerations along the other, each with all sub-themes listed as well. Meta-matrices are used to organize qualitative data visually for further analysis [15]. This method is especially helpful for analyses done by teams and to merge different qualitative data sets [16]. The meta-relationships are exceedingly complex, and the data behind them are rich and often subtle. While condensing the vast amount of data into categories is useful to illuminate the main points, important details may be excluded. In our process, we identified rows and columns and then asked questions about each cell (is there overlap or not and what is the nature of the overlap?); team members were thus forced to think about the underlying data. When questions could not be answered from memory, the original data were reviewed using the qualitative analysis software.

Three team members who knew the data well “voted” on the amount of overlap in each cell in the matrix. For example, one cell was at the intersection of the Consideration “motivation for implementing POE” and the Theme “context.” Since the motivation might include pressure from outside the hospital and since the context might involve pressure from outside the hospital, each researcher would consider this a strong overlap. There were some strong overlaps, some weak ones, and some areas without overlap. The entire team met to view the matrix with the voting indicated on it and identified strong differences so that they could be further explored.

RESULTS

Table 2 lists twelve areas we called Principles. They represent the combination of the ten Themes from the grounded data and the ten Considerations developed by the experts. Each one is somewhat different from any individual Consideration or Theme. Two Principles emanate from only one of the data sets, but are important and the lack of overlap needs explanation. “Costs” were a Consideration in the data set that resulted from the Menucha consensus conference of experts and did not appear in the data set based on fieldwork, probably because users are not very concerned with cost. “Terms, concepts, and connotations” constituted a Theme in the data set from the fieldwork but were not part of the Considerations outlined by the experts. This is likely because the experts were not fully aware of the importance of language in the interchanges among different factions such as administration, users, and technology staff, whereas the investigators noted it in the field and during interviews. The twelve Principles are briefly summarized below. They are organized in a framework we have used before [17–18] based on a more general framework called the Multiple Perspectives Approach developed by Linstone [19]. We have outlined four aspects of CPOE, the Technical, Personal, Organizational, and Environmental. Coincidentally, by replacing the term Technical with Computer Technology, the aspects create the mnemonic CPOE, which we will use here.

Computer Technology Principles

Principle #1: Temporal concerns

From the user view, the most important CPOE consideration is that it should take no extra time to use it. There are other time elements aside from response time involved: the time it actually takes to place an order and have it carried out and the implementation life cycle are additional concerns.

Principle #2: Technology /meeting information needs

Both the technical aspects of the system and the organization of information are included here. There are several technical details to consider as part of a CPOE implementation. These include strategic considerations, user considerations, and task completion flexibility as well as the quality of the application, from customizability to user friendliness.

Principle #3: Multidimensional integration

Integration was represented in the data in several different ways. From a technology viewpoint, integration of different systems was desired by users for ease of use and timesavings. They wanted seamless access to different systems through CPOE and they especially wanted both inpatient and outpatient orders written this way. They wanted CPOE to be integrated into their workflow so that it did not disrupt their work. Organizational and human integration, such as working on multidisciplinary teams, was strong as well.

Principle #4: Costs

Financial considerations cannot be underestimated: they include not only the cost of hardware and software, but all the costs associated with system implementation. In addition, they must include training and support costs and other ongoing maintenance expenses. There are also hidden costs such as a drop in productivity of each unit as CPOE is initially rolled out.

Personal Principles

Principle #5: Value to users and tradeoffs

While there are benefits to using CPOE, there are also liabilities. Users invariably describe both upsides and downsides when asked about their perceptions. The downsides always involve the time it takes compared to the old way and the rigidity of the systems. The upsides usually involve remote entry of orders and legibility. Useful aspects of the system often include decision support as well. Order sets, groupings of orders that are done together, are another often-cited added benefit.

Principle #6: Essential people

There are two large categories of people who are essential to the successful implementation of CPOE. The first group is the leaders. These include both administrative leaders and clinical leaders. Administrative leaders need to be at the highest levels of the organization, at the chief executive officer or presidential level. Clinical leaders include the chief medical officer and, in academic centers, the department chairs, for support similar to that given at the higher levels. Clinical leaders also include those whose leadership extends to information technology. The Chief Medical Information Officer, an informatician, is a key player who can make or break a system implementation. Opinion leaders, who are respected clinical experts, and champions, who are enthusiastic about the system itself, are also critical. In addition, talented people who speak the languages of both medicine and technology are essential, and there need to be enough of them. These are the staff members who can train, support, and make changes in the system.

Principle #7: Training and support

One of the constant themes identified by the experts at the retreat was the importance of help "at the elbow" at the time of implementation. In addition to the symbolic importance of supporting the users by being present while they are first using the application, intensive support at “go-live” allows the implementation team to directly experience what is and is not working well. Most successful implementations have had more post go-live support than pre-go-live training. Most sites have had 24/7 support for several weeks.

Organizational Principles

Principle #8: Foundational underpinnings

A successful implementation depends on the existence of a firm foundation in organizational terms. Much of this foundation involves aspects of organizational culture that cannot be quickly changed once a decision is made to implement CPOE; rather, CPOE should not be considered if these are not in place. Top-level commitment from administration, both moral support and financial, is mandatory, as is a high level of trust between administration and clinicians. There must be a realistic vision about CPOE and a readiness on everyone’s part to implement it. Leadership must be open to feedback. Both the organization and the vendor need a high degree of stability.

Principle #9: Collaborative project management

A theme that emerged from the field data was “collaboration and trust,” and the Consideration called “management of the project” overlapped with it considerably. This is because one of the most important aspects of project management related to CPOE is teamwork and being able to pull different groups of people together. The groups include multidisciplinary teams of clinicians, leaders, and technology staff. An important component of the project management process, in addition to managing resources and timelines, is assuring that team members treat one another with respect.

Principle #10: Terms, concepts, and connotations

The appropriate uses of language and communication are critical to CPOE implementation success. Collaboration can be improved or undermined, depending on how carefully individuals choose their words. Vocabulary can increase understanding or hinder it. Because multiple clinical disciplines and information technology and administrative groups are involved in CPOE, a common vocabulary with common understanding is needed. In addition, words with negative connotations, especially as they concern another group, need to be eliminated.

Principle #11: Improvement through evaluation and learning

CPOE implementation is an ongoing effort that benefits from continuous improvement. It is important that mechanisms for feedback and modification of the system be in place. The organization should be able to learn from the CPOE implementation project.

Environmental Principles

Principle #12: Motivation and context

It is important to consider the motivation for implementing CPOE because often pressure from outside the hospital or a desire for increased efficiency will motivate administration to want it, but clinicians may remain unmotivated. If, on the other hand, clinicians are highly motivated because they would like decision support capabilities readily available through CPOE, the likelihood of success is greater. Context involves attributes of the institution such as geography, the era during which the system is installed, the kind of unit where it is being implemented, and the types of individuals involved. For example, a Silicon Valley hospital might want to be perceived as being on the cutting edge of technology, the era of managed care may pit administrators against physicians, and emergency room personnel may feel CPOE is inappropriate in acute situations. Motivation and context must be analyzed prior to discussions of implementation so that barriers can be assessed.

CONCLUSIONS

The significance of this research effort lies in its blending of the views of experts in CPOE implementation with those of users and the use of the meta-matrix approach to establish the trustworthiness of the results of two data sets. The resulting twelve Principles are indicative of the complexity of CPOE implementation. Each Principle deserves further research, to characterize more fully the implication of each one on individuals and organizations. The establishment of metrics for measuring success in each area should also be high on the research agenda.

CPOE is beneficial and implementation in large numbers of hospitals is inevitable. The challenge is to anticipate difficulties, to implement smoothly, and to control the organizational upheaval as the organization transforms itself. Successful implementation is possible, but only if the complexity is recognized and skillfully managed.