The effects of CPOE on ICU workflow: an observational study

Abstract

Computerized physician order entry (CPOE) has had demonstrated benefits in error reduction and guideline adherence, but its implementation has often been complicated by disruptions in established workflow processes. We conducted an observational study of the healthcare team in an intensive care unit after the implementation of mandatory CPOE. We found that policies designed to increase flexibility and safety led to an increased coordination load on the healthcare team, and created opportunities for new sources of error. We attribute this in part to implicit assumptions in the CPOE system design that execution of physician orders is a linear work process. Observational workflow studies are an important tool to understand how to redesign CPOE systems so as to avoid harm and achieve the full potential of benefit for improved patient safety.

INTRODUCTION

Many medical organizations implement computerized physician order entry (CPOE) because of its potential benefits in preventing adverse drug events, increasing access to patient data, and improving clinical guideline adherence[1]. Despite these benefits, some healthcare workers have resisted CPOE because of its disruptive effect on the way they work. These effects are not well understood, and emphasis has been placed on teaching healthcare workers through change management[2]. The integration of new information technology (IT) into an organization necessarily creates sociological as well as technical change[3]. Qualitative studies of IT introduction show the importance of organizational context, such as culture and preferences, to implementation success[4]. Quantitative studies of pre-determined outcome measures give insight into the desired results, but cannot anticipate effects on unmeasured outcomes or intermediate processes [5].

We were interested in understanding local workflow effects, both positive and negative, of introducing a new technology, CPOE. We conducted an observational study of process workflow in an intensive care unit (ICU). Our purpose was to document how an organization adapted its workflow patterns to accommodate new IT, rather than to demonstrate factors which lead to implementation success. Our perspective is that these patterns contain important clinical purposes. We believe that knowledge of these adaptations can inform system redesign to maximize benefit, reduce error, and improve system acceptance.

METHODOLOGY

We use an observational case study method to research the effects of CPOE on healthcare team workflow[6]. This is the appropriate research strategy to understand the dynamics of single settings, when the possible hypotheses are difficult to measure quantitatively and the goal of the exercise is a descriptive understanding of a complex phenomenon. We attempt to generate fresh insight by looking for emergent relationships within the collected observational data, developing theories which fit observed evidence, and relating these insights to previous research. We were concerned with the actual patterns of CPOE use, as well as the perception of how it would be used. This study was part of a larger research effort to develop a computational simulation system for medical processes [7]. One member of the research team (CC) conducted all observations. The researcher is a doctoral student in medical informatics, with some training in the methods of ethnographic research.

We obtained consent to observe the work patterns of 50 individuals on the ICU care team, including the physicians (attendings, fellows, residents, interns, medical students), the nursing staff (day and evening nurses, charge nurses, resource nurses, unit clerks), two pharmacists, and one respiratory therapist (RT). We concentrated on each of these roles for at least 4 hours, with some roles followed for more than 20 hours. A schedule of observations was set up to sample from periods including daytime and nighttime, weekdays, weekends, and holidays. A total of 86 hours of observations were logged.

During each 4–7 hour observation period, the researcher concentrated on one of the roles on the team, except when the roles were meeting in a group (such as during patient rounds). The study subject’s work activities and their duration were recorded at intervals of 1–5 minutes. Interruptions of the work activities, including questions from other clinicians, telephone calls, patient requirements, the subject’s own questions, etc., were recorded. Coordination requirements arising from the activities, including waiting for other team members, were also recorded. Special attention was paid to requirements arising from the use of computers to mediate order writing and fulfillment. We made no effort to measure patient outcomes or other endpoints of treatment.

The study site was a 15-bed, medical/surgical intensive care unit. The physicians and nurses had access to two incompatible computer systems. One system, here called “BedsideSys,” was available only at each patient’s bedside and provided detailed patient information, including data from heart rate and oxygen saturation monitors, intravenous fluids administered, and many other parameters captured electronically or entered at the bedside. Manual data entry was primarily done by nurses. The second system, “HospitalSys,” was available hospital-wide, including at the patient bedside. HospitalSys was a full electronic medical record including progress notes, lab results, and order entry.

The site was unique in that CPOE using HospitalSys had already been introduced to all other wards in the hospital, and had been available (but not used) in the ICU as well. Thus, most of the physicians were already familiar with the system. This minimized the “learning curve” effect of CPOE introduction, limiting most disruptions to changes in workflow patterns. Two months before the study initiation, the team was required to enter all orders using HospitalSys. Previously, the ICU had allowed a parallel paper order system, using paper charts at the patient bedside. At the time of this study, paper orders were no longer permitted, the paper chart was seldom used for progress notes, and the practice of physicians giving verbal orders to the nurses was heavily discouraged.

In keeping with the nature of an observational study, subjects were asked only limited questions aimed at clarification while they were being observed. However, subjects frequently volunteered critiques of the ICU workflow to the researcher, particularly as pertaining to their experience with CPOE. These informal interviews lasted anywhere from a few minutes to a half hour.

The researcher gathered observations using uncoded freehand note taking. At the end of each observation session, the notes were transferred to the computer. At this time, a preliminary coding of the notes was performed, with the goal of refining areas of data collection [8]. In addition, the researcher collected paper artifacts from the days’ work, including patient summaries collated by the team for temporary reference. We also had access to the electronic chart, which was used to verify and clarify observations.

The observational notes were reviewed in an iterative process designed to generate conceptual themes arising from the study. Each observation was coded with one or more concepts. Observations with similar codes were grouped and consolidated if appropriate. An examination of within- and inter- group similarities or differences led to an emergent understanding of the theoretical drivers of observed behavior. After several iterations, a generalized description of each concept was developed, with specific instances to support its validity [6,8]. Simultaneously, an activity model of the workflow was developed using computer simulation software [7]. This proved to be a useful tool in this effort to elucidate the work context of the observations. Certain subjects were invited to comment on the analyses during the study, and a meeting for interested parties was held to give feedback on the results of the study.

RESULTS

We identified three conceptual themes: coordination redundancy, computational interface, and work location.

Coordination redundancy

Within HospitalSys, a physician initiates an order. Typically, if the order is for a medication, the pharmacist verifies the order, and the nurse administers the medication after receiving an order printout from the unit clerk. This work process is illustrated by Figure 1a. A perceived benefit of this system lies in its ability to reduce prescription and transcription errors.

Fig 1.

Order entry workflow

In practice, physicians were often alerted to emergent needs for medication orders by the bedside nurse, who was able to monitor the patient closely. Nurses had previously been able to write verbal orders from the physician, with physician signature later, sometimes after administration of the medication. With CPOE, it was incumbent upon the physician to enter the order as well as sign it. Nurses were previously able to verify the existence of an order by observing the physician during bedside order entry, or by looking into the paper chart.

With CPOE, physicians seldom entered the order at the bedside, so the nurse no longer had this readily available verification. It was inconvenient for the nurse, who interacted primarily with BedsideSys, to monitor HospitalSys. In addition, the paper printout of the order was often delayed in its delivery to the bedside. The bedside nurses assumed responsibility for making sure the verbal order had been written. Thus, nurses frequently conversed with physicians to insure that an order had been entered. These conversations usually took the form of short interruptions of the physicians’ work. In 97 nurse-initiated interruptions recorded during observation of physician workflow, 25% percent concerned whether an order had been entered. Figure 1b illustrates the observed workflow of order entry, with many feedback loops.

HospitalSys contains some mechanisms to address these realistic work processes, for example, by allowing the pharmacist to modify a medication order. Unfortunately, no neat paradigm exists for how these modifications should be introduced. A pharmacist modification often results in a second print-out at the nursing station. It is then incumbent upon the bedside nurse to associate the modified order with the original, physician-initiated order. This process relies upon the nurses to interpret the process by which printouts are linked.

Physicians were allowed the flexibility of entering orders for all patients on the ward, regardless of primary responsibility. This policy was necessary and desirable because of the cross-coverage during call nights. It also facilitated CPOE during group rounds. The unit had a mobile computer, and while the physician with primary responsibility for the patient presented findings to the team, a second physician could enter orders for that patient. One benefit of entering orders during rounds is quicker delivery, since HospitalSys immediately prints the order. However, it was often not possible to enter all the orders for a patient during rounds, since orders are typically generated at the conclusion of one patient presentation, and the group moves to the next patient. An individual order using CPOE took slightly longer to enter than a similar written order, and this problem was compounded by occasional computer network delays.

Physicians developed several methods of coping with this transition during rounds. First, they had informal heuristics regarding which orders should be entered immediately, and which could be entered after rounds. Second, the responsible physician and the second (CPOE) physician held mini-conferences during the transition to clarify the orders needed. Third, some physicians chose to miss the beginning of the next patient presentation in order to put in their orders. Finally, all the physicians took informal notes on the necessary orders, and verified after rounds that the orders had all been entered, and entered correctly. In addition, throughout the day, physicians often entered orders for each other due to convenience.

We found that CPOE created coordination redundancy through increasing the coordination load amongst clinicians and adding verification tasks.

Computational interface

Patient care in the ICU generates many orders, and physicians want to be able to enter these very quickly. The structured data required of CPOE prevents medication errors at the cost of physician time. Of the 19 individual physicians closely monitored in this study, 9 encountered problems with structured data entry during observation, even with the use of quick order templates. Although the templates provided many convenient orders, it relied upon a cognitive model of classifying orders which the physicians did not always share. For example, a simple order of saline was difficult to classify, and a physician, after looking for it under several headings, decided to write a free-text order instead.

The inconvenience of logging into the system, especially since the system’s safety features included a timed logout, caused many physicians to take advantage of each other’s open, logged in sessions, rather than log in themselves. Physicians asked each other to put in orders, or entered orders using another physician’s login, turning the computer over to the logged-in physician when it was necessary to enter a password. Consequently, physicians “signed” orders which they had not entered, or whose purpose may not have been clear to them. These physicians were all cross-covering patients in the ICU so they did not consider it inappropriate to sign each other’s orders. It was not within the scope of this study to measure the frequency of this phenomenon. However, several instances were observed where a physician was questioned about an order they had signed, which they did not recollect. A compounding problem was that it was difficult to see from the computer interface who was logged in. Sometimes physicians did not know they were using each other’s accounts until their password was rejected when they attempted to sign orders. Because of the urgency of providing medical care in the ICU, physicians would often agree to sign the orders entered by a colleague rather than require the colleague to logout, log back in, and re-enter the orders.

We found that minor inconveniences with the computational interface resulted in adaptations to circumvent the desired safety features of the system.

Work location

Geographical distance created workflow disruptions for the entire care team. Physicians who previously wrote orders at the patient bedside in a paper chart were asked to enter these orders electronically. HospitalSys was available on the bedside computers, as well as in two standalone work areas, and one mobile computer. In practice, nurses tended to use the bedside computers. Physicians usually became aware of the need for orders while at the patient bedside, but would choose to travel to the work area to enter the order, so as not to disrupt the nurses’ work. The cognitive load of this burden was frequently exacerbated by interruptions while traveling between the bedside and work area.

Nursing orders were printed out centrally in the ICU, and delay was introduced because the orders had to be delivered to the bedside nurse. Orders for other services, such as RTs, would print out at the department printer located some distance from the ICU. Since they were away from their department for hours at a time, physicians and nurses would have to verbally alert them when their services were needed. Because of geographic distance, the timing of computer orders was out of sync with and often lagged behind the actual work process.

DISCUSSION

Organizations and individuals are resistant to change[2]. Many of the potential benefits of IT may be lost to the perceived costs of a new work practice unless re-design is undertaken to address problems identified in actual practice. The abolition of paper orders appears to be a minor intervention, yet two months after CPOE became mandatory, team members were still engaged in actively explaining the new workflow to each other and modifying their work habits. We found that changing from paper based ordering to electronic ordering modified the workflow through three major mechanisms: coordination redundancy, computational interface, and work location. This observational study did not look at any of the endpoints of care to assess the costs and benefits of switching to CPOE. We find in the short term, however, that CPOE created new sources of coordination work for the care team. While it is possible that the overall impact of the introduction of CPOE was beneficial, these organizational adaptations necessarily contain their own sources of error that must be addressed.

The conceptual model of orders as a linear process, which is the model underlying HospitalSys’ CPOE, assumes that physicians initiate orders and that the orders are processed by nurses, pharmacists, and other health professionals in their respective geographical locations. In practice, we find that many orders are requested by non-physicians, particularly nurses. Because of the urgency of medical care in the ICU setting, orders begin to be executed before all verification steps are complete, and the need for clinicians to be at their geographical printing stations to receive the order introduces delays in the process. HospitalSys CPOE has been in use on other wards of this institution, but what may be adequate for other units may not suit the pressing needs of an ICU. Some improvements may be possible to the computer interface to improve process flow and clarify feedback loops, such as when pharmacists change an order written by a physician. It may be appropriate for the clinical information system to develop new interfaces and process logics for different wards of the hospital. However, a substantial improvement is possible only if we abandon the conceptual model of linear work processes, and begin to imagine how computational systems can support work that is essentially collaborative.

The ICU team has adapted to CPOE through frequent verbal communication regarding orders, and by adding ad hoc verification tasks to check for an order’s existence and correctness. These processes are informal, and thus more prone to error and neglect if the workload becomes heavy. When clinicians rely on each other to confirm an order, they introduce the element of fallible human memory into the process. Only ad hoc methods exist to resolve the discrepancy between a verbal order and its electronic form. Furthermore, although clinicians have been assured that they can rely solely on the electronic form of the order, we observed a strong reliance on paper printouts of the order.

The introduction of new redundancies into the process, such as nurses asking physicians whether an order had been written, are indicative of an organization’s attempts to prevent errors from new perceived sources. Since these behaviors are new, we have little sense of whether they are effective, and what the consequences are if they fail. A recent literature review of health care working conditions indicates that redundancies which lead to increased interruptions may have an adverse effect on the incidence of medical errors [9]. Attempts should be made to monitor and study the new behaviors that arise as an organization adapts to new work practices.

This study has several limitations. It is a study of a single site by a single researcher over a limited duration. Traditionally, case study research has been conducted by individuals, with validity and reliability confirmed through triangulating multiple sources of evidence, both qualitative and quantitative [6]. In this study these included the researcher’s field notes, the electronic documentation in HospitalSys, and paper artifacts generated during clinician work. However, it is impossible to avoid the biases of a single researcher, and the logistical limitations on what that researcher is able to observe. Theories of workflow adaptation can be built from observations at a single site. The generalizability of these theories is ideally confirmed by further research, either by another investigator at the same site, or by investigators at a similar site [8]. Increasing the strength of the theory through repeated validation will assist implementers hoping to apply these findings to their own sites. We hope the findings of this study will contribute to existing interest in using field study methodology to mediate technology adaptation [4]

CONCLUSIONS

We used observational methods to study the systemic effects on workflow of a process improvement designed to increase healthcare quality. We find that organizations adapt to these improvements in unanticipated ways that may introduce their own sources of error. Some of the more problematic adaptations may have straightforward technological solutions. For example, the ability to view HospitalSys using a role (i.e. “physician”) rather than an individual, then requiring individual passwords for signing orders, could reduce inappropriate order signing. More sophisticated methods of viewing the HospitalSys database, such as searching for “all orders written in the last two hours for patients within my responsibility,” may increase the clinician’s ability to find the information in HospitalSys that is relevant to their current work tasks. This could increase the clinician’s reliance on HospitalSys rather than verbal communication for planning and coordinating their work. The introduction of new IT into an organization is certain to change workflow. It is worthwhile to study these effects in order to inform system redesign.