Understanding Implementation: The Case of a Computerized Physician Order Entry System in a Large Dutch University Medical Center

Abstract

Most studies of the impact of information systems in organizations tend to see the implementation process as a “rollout” of technology, as a technical matter removed from organizational dynamics. There is substantial agreement that the success of implementing information systems is determined by organizational factors. However, it is less clear what these factors are. The authors propose to characterize the introduction of an information system as a process of mutual shaping. As a result, both the technology and the practice supported by the technology are transformed, and specific technical and social outcomes gradually emerge. The authors suggest that insights from social studies of science and technology can help to understand an implementation process. Focusing on three theoretical aspects, the authors argue first that the implementation process should be understood as a thoroughly social process in which both technology and practice are transformed. Second, following Orlikowski's concept of “emergent change,” they suggest that implementing a system is, by its very nature, unpredictable. Third, they argue that success and failure are not dichotomous and static categories, but socially negotiated judgments. Using these insights, the authors have analyzed the implementation of a computerized physician order entry (CPOE) system in a large Dutch university medical center. During the course of this study, the full implementation of CPOE was halted, but the aborted implementation exposed issues on which the authors did not initially focus.

In the traditional system life cycle approach, implementation is seen as a clearly defined phase that comes after the initiation and development phase of a system and before a system is put into actual use. Each phase has clearly defined inputs and outputs, and therefore documentation and signoffs are crucial elements of the life cycle. The phase of implementation encompasses such activities as training, conversion, acceptance testing, and a postimplementation audit.¹ In this view, implementation is understood as a “rollout” in which technology is far removed from its organizational dynamics.

Few studies have been devoted to understanding the actual processes of implementation of information systems in health care, mainly because such studies require access to hospital sites and the following of an implementation over a longer period. Yet it is now generally accepted that the traditional system life cycle is not adequate to understand the process of implementation of information systems.¹ Different authors inside and outside medical informatics have sought to improve on this understanding, and to get a better grasp on the intertwinement of technology and the organization. Kling and Scacchi have proposed the “web of computing” in which the use of an information system is understood in terms of the larger social and technical context in which the information system is embedded.² Medical informatics scholars have sought to understand how information systems are diffused in organizations and the barriers to such processes. Lorenzi and Riley, for example, examined the role of leadership and change management skills to introduce new technologies.³ In a fictitious case presentation of the introduction of a computerized physician order entry (CPOE) system, Ash et al. sought to identify different stakeholders and how their opinions and behavior might influence the adoption of new technology.⁴

We describe findings from a longitudinal study of the implementation of a CPOE system in a large Dutch academic medical center. We were in close contact with this center between 1998 and January 2003. During this time, we collected data through semistructured interviews, observations of staff meetings, and document analysis. Also, we constructed an in-depth overview of events before 1998 through interviews and document analysis. We use this material to further enhance the emerging understanding of the implementation process. We draw on theoretical insights from recent social studies of science and technology.⁵ Contrary to more traditional views, we propose that implementation encompasses the trajectory of introducing an information system from the very first idea that such a system is needed to address perceived organizational problems to the dynamics of use in work practices.⁶

The purpose of this study is to examine the three theoretical aspects to understand the implementation process. First, we elaborate the claim that to understand “implementation,” one needs to focus on the interrelation of the organizational environment and the technology. Kling and Scacchi argue that the social context determines to a large extent how a new technology will be adopted and used by an organization.² They reject the idea that “inherent” characteristics of a technology will determine specific effects in the organization, for example, that increasing the speed of data flows will lead to faster and better decisions. Intent on showing the importance of social context, however, they also tend to underscore the importance of the system's technical properties. Just how an organization will react to an information system, after all, is importantly influenced (not determined, indeed) by the characteristics of that system. As one of us has phrased it, the introduction of information systems in health care practices is a thoroughly social process in which both the technology and the practice are transformed.⁵ We have labeled this perspective the sociotechnical approach.⁷ The term derives from the field of social studies of science and technology, in which researchers aim to understand how technology is shaped as part of “messy” networks that combine technical, social, and economic elements.⁵

The second aspect is the concept of emergent change and the notion of an unpredictable outcome of an implementation process. In the classic system life cycle, each phase forms the input to the next phase. This implies that the introduction of information systems is a linear process with predictable outcomes in the form of deliverables. We argue, however, that implementation processes are typified by contingencies and proceed in a far from linear manner. They are part and parcel of organizational dynamics that, as a result of the complexity of the organizations of which we speak, cannot be foreseen, let alone be predicted.⁸ In addition, the broader context in which these organizations find themselves is in a constant flux. Demographic changes, political pressure, increasing demands for quality care, and new medical technologies alter the conditions in which health care is delivered. Following Orlikowski, we use the concept of “emergent change” to get a grasp on the development of information systems in such circumstances.⁹ This process of “change” never stops; even when the implementation is “formally” finished, users will still shape and craft the information system to fit their particular requirements or interests, often in a way unanticipated by the designers.¹⁰

The third aspect is the concept of fit and the notion that success and failure are not dichotomous and static categories. Rather, “success” and “failure” are socially negotiated judgments, which may vary depending on the moment in the implementation process and the perspective of the stakeholder focused on. In addition, the “success” or “failure” of an information system lies exactly in the interaction between the system's functioning and the organization's needs and working patterns.¹¹ Lorenzi and Riley wrote that a “technically best” system can be brought to its knees by people who do not feel ownership and resist implementation, whereas a “technically mediocre” system may be extremely valued by its users.¹² Often, even apparently clearcut technical shortcomings can be the result of poorly managed development processes.¹¹ We explore how the concept of “fit” can give meaning to successful implementation of an information system.¹³ Southon et al. introduced the concept of fit in health informatics to emphasize the importance of the organizational configuration in managing the transfer and diffusion of technology, specifically organizational strategy, structure, management processes, roles, and skills.¹⁴ Our focus is on the necessity of actively producing fit between work processes and information technology.

Research Methodology

Qualitative research methods are very appropriate to study systems in organizational contexts. Information systems in organizations are complex technologic artifacts because they are shaped by time and continuous change.¹⁵ From 1998, the first author conducted semistructured interviews, attended meetings, and studied documents about the selection, specification, and implementation of the information system, and the evaluation of the CPOE pilots.

The interviewed persons included the two project leaders, the medical and technical coordinators of the implementation team of the CPOE system, and the members of the pilot project that tested the feasibility of order entry in gynecology and pathology. In total, ten persons were interviewed and 15 interviews were conducted (see ▶). Apart from the staff gynecologist and head of pathology, all interviewed persons had various roles during the course of the implementation. Therefore, they could provide us with a unique insight about the full history of the introduction of the system. The interviews lasted between one and two hours. Ongoing contacts have been maintained with a few members of the now-dissolved implementation team who are still active in informatics projects of the medical center. In addition, we observed and took notes of all staff meetings of the pilot project that we followed. We observed the use of the system during the pilot project and occasionally asked the users what they were doing. In the staff meetings, we focused on how the participants were behaving and what they were saying.

Table 1.

List of Interviewed Persons^*

Interview No.	Interviewed Person	When	Topics of Interview
1	Project leader 1	Jan 1999	Goal and setup of CPOE implementation project
2	Project coordinator medical applications	Jan 1999	Goal and setup of CPOE implementation project, envisaged nature of CPOE implementations
3	Project coordinator education and training	Feb 1999	Education and training issues of CPOE system implementation, development of customized user manuals
4	Medical records coordinator clinical surgery and system implementation specialist	Mar 1999	Goal and setup of CPOE system implementation project, setup and expected outcomes of pathology–gynecology pilot project
5	Project leader 2	Mar 1999	Goal of CPOE in medical center, setup of pilot project, collaboration with various clinical departments
6	Chief gynecologist ambulatory clinic	Mar 1999	Goal of CPOE pilot, planning of the pilot project, nature, workflow of pathology order, expectations about CPOE and expected outcomes of the pilot project
7	Chief pathologist	Apr 1999	Expectations of CPOE and pilot, workflow issues in pathology
8	Pathology technician	Apr 1999	Workflow issues of pathology, experience with pilot setup
9	IT administrator department of pathology	May 1999	Setup of pathology pilot, general issues of experience with CPOE system implementation since beginning of project
10	IT administrator department of pathology	May 1999	Clarifications of issues raised in first interview with respect to implementation of the CPOE system
11	Medical records coordinator clinical surgery and system implementation specialist	Jun 1999	Review of pilot project and evaluation of CPOE system implementation project
12	Project coordinator CPOE system	Jul 1999	Evaluation of CPOE system implementation
13	Project leader 1	Jan 2000	Review of aborted implementation and goals of CPOE
14	Project leader 2	Jan 2000	Review of aborted implementation
15	Project coordinator medical applications	Feb 2000	Review of aborted implementation

^*Three interviews were held prior to the implementation pilot project and the issues addressed pertained to the goals of the computerized physician order entry (CPOE) system implementation and the setup of the pilots. The interviews in 1999 addressed the CPOE pilot gynecology–pathology, but general issues were raised as well. The interviews in 2000 were conducted after the presentation of the external evaluation results. The first author remained in contact with the project leader and project coordinator medical applications after 2000.

Finally, we also observed and took notes of the meetings in which the conclusions of the consultancy firm about aborting the implementation were presented. The documents pertaining to this advice and the decision making about the cessation of the implementation were also made available to us.

Notes, interviews, and implementation documents were coded for occurrence of specific key words, but generally the documents available to us were structured so that patterns could be extracted.

Permission to study the documents, interview persons, and attend meetings was obtained from the steering committee of the CPOE system implementation project. The use of patient data was not envisaged and did not take place. Therefore, we did not seek approval of the medical ethics committee of the medical center.

The Case of the Implementation of an Order Entry and Results Reporting System

History of the Implementation

Contrary to the practices of other Dutch university medical centers, a large 953-bed university medical center in the east of the country has always developed its own information systems. In 1988, the hospital concluded that its hospital information system was becoming obsolete and decided that it would rather buy an off-the-shelf product than build a new one again (see ▶ for implementation timeline). As a key feature, the new system would have to be focused on the core business of the hospital: supporting clinical work (rather than only clerical work). A small group of senior staff from the nursing, medicine, clinical laboratories, and information systems departments, the initiators, set out to identify the needs of the hospital and scout for information systems available, especially in the United States.

Figure 1.

Computerized physician order entry (CPOE) system implementation timeline. The timeline indicates key events during the implementation of the system.

The computing infrastructure of the medical center consisted of two IBM mainframe computers under the operating system MVS/XA and the network protocol SNA through which a host of other systems and terminals were connected. On one mainframe resided the information systems in use; the other was used for backup, and development and testing purposes. The medical center had been using IBM systems since the late 1960s.

The university hospital clinical registration system (UZIS) was homegrown, whereas the financial, clerical, and personnel software packages were commercially acquired. UZIS allowed for requesting diagnostic and therapeutic treatments, registering diagnostic and therapeutic interventions, and ordering laboratory tests. In a way, it was a very primitive order entry system.

The existing computing infrastructure and cost considerations constrained the choice of a new information system. An internal report of 1990 described three systems available for the Dutch market, of which the TDS7000 system of the TDS Healthcare Systems Corporation^* was favored. The medical center had always built its own clinical registration system on the existing IBM mainframe infrastructure. It therefore maintained a large development and support group of approximately 100 staff. Yet the center felt that building and maintaining a complete clinical system on its own would be a risky (and potentially very expensive) strategy. Also, changing the technical infrastructure would be costly. The acquisition of the hospital information system that was used by all other university medical centers in The Netherlands and that ran on a different platform was therefore never seriously considered.

The medical center wanted to move toward more clinically oriented systems. All three systems under consideration supported order entry and results reporting. The decision to move toward the TDS7000 system was motivated by site visits in the United States, convincing the participants (including key future users) that the system was the best choice: it had advanced functionalities, its configuration was highly flexible, and it would be feasible to connect to systems already in use because of the network and data communication protocols. With this system, the medical center thought it would become the leading center in patient care information technology, moving far beyond the other university medical centers in the country.

In July 1993, the group proposed to purchase the system, which was primarily a computerized order entry and results reporting system for physicians. The system had been developed in the late 1960s and was the first system specifically designed to have the work of health care professionals as its core orientation rather than being oriented to the support of clerical and financial activities.¹⁶ The advantages listed were less paperwork, more complete orders, fewer transcription errors, and faster availability of results. Indeed, later studies showed that CPOE reduces medication errors and improves patient outcomes.¹⁷ It was expected that reductions of clerical staff could be achieved because of the reduction in paperwork and a more efficient user interface. However, the hospital community was assured that the nature of their work would not seriously change and that there would be no job redundancies. In 1995, after a number of site visits (in the United States and the United Kingdom) and in-house consultations of medical and nursing staff and representatives of the employees, the board of directors signed a contract with the vendor to deliver the system and the accompanying support and implementation procedures. The decision was made to implement and activate the registration functionality first so that UZIS could be phased out as soon as possible and to implement the CPOE functionality after a limited number of pilots. Between 1995 and 1997, work focused on the configuration of the system and training the trainers. Training of all prospective clerical users, including the superusers, started approximately six months before the system became operational. Computer-based training facilities were made available for classroom and individual “walk-in” training. Each prospective user had to pass a computer-based examination before he or she would be issued a password and authorized to use the system. Freeing time for training and education proved to be difficult for busy clerks and caused an extension of the originally planned training period that was to end in the summer of 1997. The hospital community was informed about the progress of implementation through articles in the hospital newsmagazine and special newsletters. These articles were of a human-interest nature because they focused primarily on the personal experiences of individuals in the project. The newsletters carried more factual information such as descriptions of the system and its parts, the implementation plan, the progress, training, and software releases.

The system went live on December 1, 1997. At that moment, only the clerical functions of patient admission, medical procedures registration, and patient scheduling became active hospital-wide. During 1998 and 1999, many corrections and improvements were made to meet the needs of the users and counter serious problems (discussed later in this article). The year 1999 also saw the start of the next phase of the introduction of order entry and results reporting functionality. A few small-scale pilots were conducted to assess the effects and feasibility of fully implementing this functionality. Yet, as we shall see in more detail later, from the summer of 1999, physician resistance against the CPOE system built up significantly. The board of the specialist staff (medisch stafconvent^†) requested an external review of the system. In October 1999, the hospital board hired a consultancy firm to review the system and the project management structure and to make recommendations for the continuation of the project. In February 2000, the consultancy firm advised the board to discontinue the implementation of the system, which was graphically depicted as a Trabant^‡ car with square wheels (see below). In December 2000, the board decided to freeze further deployment of the system and not to implement CPOE. It would allow system modifications only to maintain current functionality for the hospital, minimizing the damage until a new system would be selected and implemented.

The Project Team

A project team was responsible for the implementation of the system (see ▶). This team reported to a steering group, which consisted of key individuals representing the medical departments and the hospital board. The steering group made major decisions about the implementation process, including involvement of clinical departments and the allocation of funding. A medical specialist headed the project team. His main tasks were to manage the project and to ensure the ties with the medical professionals and other stakeholders within the hospital. His position was part-time to allow him to continue his medical practice. A project coordinator, a senior staff member of the information technology department, was responsible for the day-to-day management of the project team and all matters technical. The other members of the team came from the information technology department. The latter focused primarily on “technical” issues such the development of specifications, designing databases, creating a technical infrastructure, designing screens, and educating future users. Each member also addressed a specific domain of applications of the system. Each main domain—medicine, nursing laboratory, radiology, pharmacy, patient registration, and scheduling—had its own task force consisting of members of clinical departments and technical staff. These task forces decided on the way the system would be used and which functionalities would be implemented within their domains.

Figure 2.

The structure of the computerized physician order entry (CPOE) system implementation team. The coordinators of each task force and the project leader formed the implementation team. The project leader was a physician who spent about half the time on the project and was for the other part still active in medicine. The project coordinator, a senior member of the information technology department, was responsible for all technical issues of the implementation and coordinated the members of the implementation team on a day-to-day basis. Each task force was responsible for a particular application domain. The task force “function departments” encompass departments such as radiology, nuclear medicine, pathology, and vascular ultrasound imaging. The task force for pharmacy applications was planned to become active in a later stage. The director of information technology was a member of the Steering Group.

In the beginning, the local project team was mirrored by a team of the vendors as part of their implementation procedures. This was later abandoned because the implementation took more time and effort then expected, and the costs were becoming prohibitive for the hospital. According to the same procedures, key contact persons were appointed at all clinical, ambulatory clinical, and ancillary departments. The original list numbered close to 150 names. The key contact people were usually physicians, nurses, clerical personnel, or technicians who were supposed to be familiar with work procedures in their departments. However, this approach was also abandoned because the number of people involved was too large to manage properly, and it proved to be difficult for these contact people to produce useful information and specifications for the implementation staff. The project team relied mostly on the expertise of the members of the different task forces.

Computerized Physician Order Entry

The core functionality of the system was CPOE, around which the future development of the electronic patient record would be shaped. Our study focused initially on this point. The implementers expected that all physicians would use this application. The people involved in the gynecology–pathology CPOE pilot held the same view. Writing orders, after all, was the professional responsibility of the physician and was not to be delegated to nurses or clerks. User codes, passwords, and electronic authentication would ensure that only physicians could enter orders and that any misuse of the system could be detected through log-on trails. However, this core functionality was never implemented organization-wide and could not be studied in full.

Analysis

The Sociotechnical Reality: The Intertwinement of Organizational Environment and Technology

Existing technologies and organizational arrangements are important factors that determine the introduction of new technologies such as the introduction of CPOE. In the case of the university medical center, the organizational considerations that determined the choice of the system could be summarized as follows.

• The existing IBM technologic infrastructure narrowed the choice. The technology itself did not dictate the choice, but the associated organizational arrangements did. For instance, a large body of information technology staff over a long period had gained expertise to develop and support the IBM infrastructure, including the mainframes and the network. It was simply too costly to retrain them for a new computing infrastructure or to replace them.

• Another factor was the fact that investments had been made in nonclinical IBM mainframe applications such as personnel and billing systems that had to be connected to the new clinical application. Again, it would be a destruction of capital investments and human resources if these systems had to be replaced.

The system had of course to be adapted to the Dutch situation. The most visible adaptation was the translation of the screens into Dutch. However, the more subtle part was the translation of the clerical workflows into pathways within the system. This led to an explosion of the number of screens to reduce the rigidity of the implied workflow. Sometimes this adaptation was difficult to achieve. The system was conceived to be used within the American hospital system, which is primarily oriented to its inpatient functions. The CPOE implementation in El Camino Hospital is a good example of that approach.¹⁸ It proved to be more difficult to adapt the system to the Dutch situation, where medical specialist diagnostic assessment and therapy take place in ambulatory settings (polikliniek, ambulatory clinical department or ambulatory clinic). The ambulatory clinics process a high number of patients per day and form an integral part of the medical center (see ▶). With a referral letter from his or her family physician, a patient usually sees a medical specialist first in the ambulatory clinic. The medical specialist may then decide to admit the patient clinically or continue treatment in an ambulatory mode. Also, after a clinical discharge, a medical specialist would see the patient in the ambulatory clinic for follow-up treatments or checkups. We examine the consequences of that later in this article.

Table 2.

Key Figures of the University Medical Center for the Year 2002^*

Ambulatory clinical department visits	381,011
Clinical admissions	24.494
One-day therapeutic treatments	15,410
Bed occupancy rate	60%
Beds	953
Staff	8,153
Full-time equivalents	5,548

^*Ambulatory clinical departments are an intrinsic part of the Dutch hospital system and are linked to the equivalent clinical departments. Staff physicians and residents see their patients at both places. In the Dutch system patients need to be referred by their family physician to see a medical specialist. A medical specialist will usually see the patient first in the ambulatory clinic and then decide what further action will be needed, including a clinical admission. Therefore, ambulatory clinical departments process a high number of patients. The number of single-day treatments is significant compared with the number of clinical admissions. The high number of staff is indicative of the high load of training requirements.

The screens of the CPOE system were translated into Dutch as ▶ shows. The mainframe screens were emulated in a Microsoft Windows environment. Data could be entered with the keyboard or with the help of a mouse and clicking on selected fields on the screen, a key characteristic of the system. The mouse movements were very coarse. The interface was originally conceived for terminal interaction with a light pen. For adequate user interaction, the screen could hold maximally 24 × 40 characters.

Figure 3.

Dutch translation of the TDS7000 system screen. This menu screen allows doctors to see standing orders and patient data. Under the heading “Patientgegevens” (patient data), the last arrow is a reference to the DRG-like classification scheme (DBCs) that became mandatory in Dutch hospitals as of January 1, 2004. Note the coarseness of the system emulation compared with the finesse of the Windows environment (see Windows task bar).

The implementers of the system assumed that the introduction would not significantly affect the organization for two reasons. First, the introduction of CPOE would build on a functionality that already was agreed on in 1983 and that was, although very limited and primitively, implemented in UZIS. The functionality was expected to remain basically the same. Second, the existing IBM infrastructure would remain familiar to most information technology staff, whereas the Windows emulation would make use easier. The implementation focused at first on translating the clerical procedures of UZIS into the new system. In a later stage, the functionality of real-time physician order entry and results reporting would be added.

In reality, the implementation had a drastic impact on the organization. Soon after the implementation, clerical users found that retrieving and entering patient data took much more time because each screen would allow them to handle only a limited amount of data. For most tasks, many more screens now had to be worked through. Also, when making typing errors, they had to go back in the pathway and redo a part of the transaction or even the whole transaction. Furthermore, they discovered that data they held to be essential vanished after a few screens. In the old system, some data such as the patient ID number remained in view to help navigation through the system.

All this caused a severe slow-down of work processes, which created chaos at the ambulatory clinical department desks: long lines of (often angry) patients waiting to be helped, frustrated physicians, and verbally assaulted secretaries: “Are you so stupid that you can't handle a computer?” [Patient, ambulatory clinic]

The problem could only be remedied by increasing the clerical staffing of the ambulatory clinical departments to handle the patient load. This was quite contrary to the expectation that the introduction of the new system would save on personnel, as was projected in the proposal.

The problems that the users encountered with the system showed that the registration work processes were closely interrelated with UZIS, the old system. The screen of that system always held the patient number on the top line. The users were very proficient with use of command and function keys to interact with the system. A telling example was the following. The laboratory system could not be connected to the system after the “big bang.” UZIS remained functional to allow users to see the data of the laboratory information system. Users of the system also accessed UZIS at a terminal next to the personal computer (PC) and used the patient name and number, visible on that system, to navigate through their own.

The recurrent problems also changed the attitude of those physicians who were at first champions of the system. When they saw that the workload of the clerks had increased and were confronted with the practical consequences of the system in use, they turned into opponents of the system. They started to emphasize that it was not their task or interest to spend so much time behind a computer: “The system requires a doctor to send electronic notes. Doctors don't send notes. They have other people doing that for them.” [Physician, former project leader, CPOE implementation]

Weiner et al. reported a similar response of physicians toward CPOE.¹⁹ In economic terms, CPOE envisages the most highly trained professionals with the greatest opportunity costs to be placed in the data entry role.²⁰ Given the legal distribution of responsibilities, and the expectation that work procedures would be streamlined, the designers in the medical center had embraced this principle. The professionals, however, now started to rally against this idea because it was clear to them that the system would cost them time. What was much less clear to them was what benefit it would bring and how it would fit into their work practices.

Another problem occurred at departments where patients underwent diagnostic or therapeutic treatments. The budgeting of these departments depended on the number of patients treated. When, after some months, the figures requested from the information systems department turned out to be wrong, it took a few months to find the cause. This endangered their proper enumeration, which caused uproar among the medical specialists involved. (In fact, it turned out that the main problems were due primarily to poor instructions on how to enter patient information correctly into the system.) Other small issues further hurt the acceptance of the system, such as a problem with clinical trial patients who erroneously received bills for their experimental treatments.

Our findings emphasize the intertwinement of organizational and technical elements in information technology. The combination of technical considerations (the preexisting IBM infrastructure) and organizational considerations such as “cost containment,” “being clinically oriented,” and “being a leading medical center” together led to the choice for the CPOE system. The system, subsequently, was translated into Dutch and tinkered with to be able to support the typical Dutch “ambulatory clinic” emphasis in hospital work. When put to work in actual practice, the technical features of the system felt restrictive and cumbersome to the users, especially when more and more people had become adjusted to graphic user interfaces. This restrictive and cumbersome “feel,” however, was not just a technologic issue; the screens could have been configured differently (e.g., more easy to use in admitting a patient), but they were not. Also, organizational routines could have been altered to fit the system better, but that was also not tried. Finally, the choice to have the physician in the data entry role was not a technologic necessity; it was an organizational, largely unquestioned, implementation choice.

Understanding the implementation of an information technology, therefore, requires a simultaneous orientation toward both social and technical aspects. Likewise, the account illustrates how it is useless to attempt to determine whether the experienced problems were ultimately “technical” or “human.” Technical design finds its roots in the organizational conditions and arrangements and organizational conditions are changed as a result of technical design, as the example of the clerical users demonstrates.

The Unpredictable Outcome of Implementation Processes: Emergent Change

In 1999, a few pilots were conducted to assess the feasibility of the order entry and results reporting functionality of the system. The main objectives of the pilots were to understand how and which work practices would be influenced by the system, to identify program errors, and to establish the conditions and to plan for a hospital-wide rollout. The conditions were classified in three categories, which were the description of work processes and possible adaptation of the system's functionality, the technical infrastructure, and the level of support and training for users. It was thought that if the conditions in these categories were judged to be adequately met, then the order entry functionality could be activated.

The departments of the hospital were selected according to a judgment about how well work processes were formalized. The department of radiology and nuclear medicine and the department of pathology were considered good candidates for starting pilots because both departments were thought to have well-formalized work routines that would require little adaptation. The clinical department and ambulatory clinical departments were selected through the informal network of the project leader and other team members and at the suggestions of the radiology and pathology departments. There was little preplanning of the pilots. For example, it was agreed that physician order entry would be implemented as a pilot between the departments of gynecology and pathology because during examinations, physicians would usually take tissue specimens and send them to pathology. The number of orders was expected to be sufficient to make a valid assessment of the feasibility of order entry. A plan was made detailing who would be involved in the project. Only a few days before the start of the pilot, the technical staff discovered that the expected refurbishment of the operating theater was canceled and that it was impossible to install a PC because the whole theater would have to be rewired to comply with electrical safety regulations. The pilot then moved to the ambulatory gynecology clinical department and only one doctor was involved. A PC was activated close to the nursing station and a printer was installed for printing labels. Actually, the participants, including the pathologists, the pathology technicians, and the attending gynecologist and nurses, evaluated the outcomes of this pilot very positively.

The connection of the CPOE system with the laboratory information systems of the hospital was considered crucial for laboratory ordering. As in most modern hospital laboratories, the processing of samples and analysis and collection of data are, to a large extent, automated. The selection of a new laboratory information system in the department of clinical chemistry turned out to be very problematic and was only resolved well after the decision to abort CPOE implementation. No connection was made with the new system, and UZIS was still used to see the data on the old laboratory system. The selection process was not a responsibility of the implementation team; it belonged to the laboratory staff who did not want to be involved with the project because they thought that they already had enough problems of their own. It crucially affected the planned implementation.

The examples of the selection process of the pilots and the laboratory information system show that implementation was fraught with uncertainty because it was dependent on the willingness of individuals and departments to participate and on the local contingencies. In a hospital, there is no central line of command that can align departments and projects. The process contributed to the overall unpredictability of the outcomes of the pilots.

Time influenced the implementation process. Because several years passed after the decision to implement a new system, users got acquainted with Windows-based PCs. The interface became “stone age” in appearance: “The characters look like Braille.” [Nurse, dermatology clinic]

Especially due to the pressure from the physicians, the board of directors started to shift their position with regard to the CPOE system. The board did not move toward “owning” the system, but looked for arguments to halt the implementation. Finally, by having an external review, they effectively blamed factors outside of their influence such as the aged character of the system in the context of rapidly developing Windows-based PC systems.

The end result was a “lock-in” situation. The full-blown implementation of the system was aborted, but the system was not put out of use. Nobody was happy with the system, but, on the other hand, it was clear that for the time being, no alternative was available. The system is still in use as a patient registration system, and in due time will be replaced by a new system. Work is now focused on the development of graphic workstations that allow health care professionals to see patient data in a clinically meaningful way.

Designers and implementers devote much effort to controlling the implementation process. Pilots in this perspective are meant to identify shortcomings of the system that can be remedied before a full-blown implementation. The scaling down of the gynecology pilot was not anticipated, as it was the result of contingencies that they could not know. The implementation team did also not realize when the pilots became futile. They did not recognize how adverse experiences with the patient registration portion of the system had negatively affected a major stakeholder group. In the dynamics of implementing the system, the previously mentioned changes were not anticipated and certainly not planned for. Planning occurred as a response to new arising situations, as the gynecology–pathology pilot showed. The focus on systems implementation as a technical strategy allowed different stakeholders to develop their own agenda, as circumstances would dictate. Changes only emerged as a coming together of contingent events and decisions made on an ad hoc basis. As Orlikowski puts it: “emergent change is the realization of a new pattern of organizing in the absence of explicit, a priori intentions.”⁹ Therefore, we would characterize the implementation of an information system in a complex organization as “emergent change.”

“Success” or “Failure”: producing “Fit”

The implementation of an information system in clinical practice is not a linear process with a defined starting point, clearly delineated goals, and readily identifiable stages. By most accounts, the implementation of the system was a failure (although the system is still in place). Looking back, the functionality of order entry and results reporting, which constituted the core of the system, was never activated. However, it would be too simplistic to argue that this failure could have been predicted at the beginning of the project. Similarly, it would be too simplistic to say that a checklist with critical success and failure factors could have prevented the problems. The outcome was rather the result of a series of events and contingencies that were not planned for or whose impact was not anticipated. Decisions that looked reasonable when made can thus become constraining and clearly “wrong” in retrospect. Each step in the process of implementation leads to new and partially unpredictable outcomes that have to be judged in the context of the new situation. In the checklist, the box “appropriate technology” would have been ticked at the time the procurement decision was made; the mainframe technology appeared to be highly problematic only later. At the time of the decision, the choice for the IBM mainframe environment made perfect sense; infrastructures based on PC technology for large organizations were far from mature up to the middle of the 1990s. One may conclude that long lead times for implementation are questionable because objectives, goals, and context (organizational and technical) can change drastically over time, yet it is hard to imagine how such expensive and complex implementation processes could reduce their lead time sufficiently. In complex organizations, large-scale implementations are observed to take a long time. Such long times are even deemed to be necessary for the mutual learning process to develop and implement information systems.^20,21

There is, then, no simple formula for success or failure. The complexity of the sociotechnical networks, and the inherent unpredictability of information system implementation within complex organizations, is simply too great.¹¹ In addition, what counts as success or failure is not clearcut, but the outcome of a social negotiation. Many different definitions of “success” or “failure” can and are used by the involved actors.¹¹ In this case study, the proposal to introduce the CPOE system very much focused on savings. Savings could be found in fewer budgets spent on system development and maintenance and reducing the number of clerical staff. The benefits of medical order entry and results reporting were phrased in technical terms of better readable and more complete orders, but the consequences for medical work were not highlighted. Some studies and reports in the public press have shown that cost benefits of strategic information technology in health care and the services industry in general were not achieved.²² In the discussion about medication errors, physicians do not see themselves as part of the problem.²³ Other professionals in the loop of ordering medication often correct potential errors so that the ordering physician is not confronted with the potential negative outcome of his or her action.²⁴ In a survey of hospitals that have implemented order entry and results reporting systems, Ash et al. found that physicians were using the system in only 15% of these hospitals.²⁵ The remark of one of the leading physicians in the CPOE system implementation project that physicians do not send electronic notes is a telling example.

From the case study, it became apparent that the system did not fit well with registration work practices. In a study about evaluation of medical informatics applications, Kaplan made a plea to use evaluation methodologies based on social interactionist theories.²⁶ In this paper, she suggested that “fit” is a key factor for the successful implementation of information systems in health care. “Fit” has various dimensions such as compatibility of the system with the workflow, with the level of expertise of the users, with the belief system of the user or the organization, and so forth. We agree with Kaplan that fit with work processes constitutes an important explanation for success. However, we find that whether “fit” exists or not is not due to the technology introduced and the practice in which it is to be used. Rather, this “fit” has to be actively produced: the technology and the practice have to be made to fit. An information system has to be adapted to the work practices of the user, but users will have to change their practices as well because of the opportunities and constraints of the technology. To achieve this fit, first of all, requires a thorough understanding of the work practices. Yet an analysis of work practices should not only include an analysis of what people do, but also how they might do it better.²⁷ Often, a proper understanding of a system's functionality can point to the direction of such improvements. In this case, for example, it was a moot point that the hospital did not attempt to restructure work routines so that an order entry system would be able to articulate with the practice much better.⁶ An order entry system becomes truly useful, even for doctors, when standardized care paths and protocolized order sets are made part and parcel of working routines. A deeper understanding of clerical work practices, also, might have led to their reorganization and perhaps could have prevented the increase in the number of clerks. In the current example, because “organizational change” was not planned, measures to remedy organizational impacts were now mainly defensive. Alternatively, proper information technology implementation should always be seen as a process of organizational change and should thus always be oriented toward a redesign of, in this case, professional working patterns.²⁸

Conclusion

In the case study of the implementation of a CPOE system in a large university medical center, we have sought to further our understanding of the process of information systems implementation. We argue, first, that the introduction of the order entry and results reporting system can be properly understood only if we consider the social and technical aspects of the story as highly interrelated. “Implementation” is not a purely “social” process, and it is not determined by or reducible to “technical” issues. It is always and irreducibly both. This makes the task complex for the analyst, because he or she should be able to dive deeply in both the organizational dynamics and the technical details of an implementation account. In addition, it points at the difficulty of managing an implementation; there as well the challenge of “bridging” and integrating the organizational dynamics with the technical (im)possibilities has to be met head on.²⁹

We argue, furthermore, that the implementation process is highly unpredictable. At the time of the decision to implement the system, no one could have foreseen that later experience with PC Windows technology would influence the attitude of users. Despite the fact that the implementation of the system has been “frozen,” continuous changes are made to incorporate new requirements such as the coming of a diagnosis-related group (DRG)-like financing scheme in Dutch hospitals (▶). We saw that the implementation process was influenced by contingencies that were not expected and certainly not planned for. Some were small, for example, the move of the gynecology order entry pilot from the inpatient clinic to the ambulatory clinic. The “feel” of an interface becoming obsolete and the changing position of the medical staff were major changes that had major impact on the acceptance of the system. Unanticipated and unplanned changes are part and parcel of the implementation process and can often manifest themselves in hindsight. Emergent change is a key characteristic of implementing information systems in complex organizations.

We conclude that there is no simple formula for success because of the complexity of the sociotechnical networks and the inherent unpredictability of information system implementation within complex organizations such as the university medical center. Failure of the implementation could not be predicted at the beginning. However, it became apparent that the information system did not fit well with work practices. In our view, “fit” can be seen as a key factor for the successful implementation of information systems. However, fit is not a property that relates to the nature of technology or work practices; it has to be actively produced. Just analyzing work practices to discover how technology might be implemented is not sufficient. Both technology and work practices have to be changed to implement an information system successfully.