Automation and adaptation: Nurses’ problem-solving behavior following the implementation of bar coded medication administration technology

Abstract

The most common change facing nurses today is new technology, particularly bar coded medication administration technology (BCMA). However, there is a dearth of knowledge on how BCMA alters nursing work. This study investigated how BCMA technology affected nursing work, particularly nurses’ operational problem-solving behavior. Cognitive systems engineering observations and interviews were conducted after the implementation of BCMA in three nursing units of a freestanding pediatric hospital. Problem-solving behavior, associated problems, and goals, were specifically defined and extracted from observed episodes of care. Three broad themes regarding BCMA’s impact on problem solving were identified. First, BCMA allowed nurses to invent new problem-solving behavior to deal with pre-existing problems. Second, BCMA made it difficult or impossible to apply some problem-solving behaviors that were commonly used pre-BCMA, often requiring nurses to use potentially risky workarounds to achieve their goals. Third, BCMA created new problems that nurses were either able to solve using familiar or novel problem-solving behaviors, or unable to solve effectively. Results from this study shed light on hidden hazards and suggest three critical design needs: (1) ecologically valid design; (2) anticipatory control; and (3) basic usability. Principled studies of the actual nature of clinicians’ work, including problem solving, are necessary to uncover hidden hazards and to inform health information technology design and redesign.

“Ultimately, the study of human performance is in one sense or another the study of problem solving.”

– Woods et al., 2007, p. 466

“How do practitioners encounter and deal with ordinary and extraordinary situations? How do they discover relevant domain semantics and learn the underlying domain invariants? How does their cognition work at work, in situ, in vivo? How do they balance demands, shift work, and sacrifice goals? How do they make the brittle work world plastic and resilient? How does technology change all this?”

– Cook, 2005, p.8

1. INTRODUCTION

Operational problems created by the complex, chaotic hospital work environment threaten the safety of healthcare provision (Cook et al., 2000; Gurses & Carayon, 2007). Those problems include production pressure and work overload; non-value added interruptions and competing demands; missing or inadequate medication, supplies, and equipment; and incomplete, ambiguous, or erroneous information (Tucker & Spear, 2006). Some of these problems may be identified by hospitals as latent or active failures to be solved at the organization level, but more often they are simply treated as the “messy” occurrences of everyday work (Berg, 1999). When operational problems are seen as necessary consequences of complex work, their remediation falls to frontline workers, for better or for worse. In hospital settings, nurses are the frontline workers most often observed to engage in operational problem-solving behavior (Tucker et al., 2002; Tucker & Spear, 2006). As in other domains of work, operational problem solving—be it heroic (Reason, 2008) or mundane (Vicente, 1999)—is how nurses are able to accomplish work in a timely, safe, and effective way. As an example of typical problem solving, hospital nurses occasionally “borrow” medication or linen from another patient or unit when the unavailability of those items threatens patient care goals (Eisenhauer et al., 2007; Patterson et al., 2002).

A hallmark of human factors and ergonomics is inquiry into how organizational interventions change the nature of work and how workers adapt following that change (Woods & Dekker, 2000). Given that problem solving is a vital aspect of nursing work, it bears investigating what happens to nurses’ problem-solving behavior following an organizational change. One of the most common and perhaps most impactful changes facing nurses today is new technology. The technology that may represent the most significant change to nursing work is bar coded medication administration (BCMA) systems yet there is a dearth of knowledge on how BCMA alters nursing work (Karsh et al., 2011). Of greatest present relevance, there is no research specifically on the impact of BCMA systems on problem-solving behavior or on how nurses solve new problems introduced by BCMA. It is this study’s objective to fill that research gap at the intersection of automation (BCMA) and adaptation (problem solving).

1.1. Automation – BCMA systems

BCMA systems are commonly recommended for improving hospital patient safety (Institute of Medicine, 2007). In theory, these systems are the last line of defense between a patient and a medication being administered in error (Cescon & Etchells, 2008). Table 1 summarizes the nature of these systems, their prevalence, and some of the key findings from recent BCMA studies.

Table 1.

Basic information about BCMA systems.^a

Goal of BCMA       Ensure the five “rights” of medication administration: right medication, right patient, right dose, right route, right time (Cummings et al., 2005; Neuenschwander et al., 2003) and ensure complete and accurate documentation of the administration process (McRoberts, 2005).

Basic requirements       Machine readable bar code labels that uniquely identify all medications, nursing staff, and patients (Cummings, et al., 2005).

Typical hardware and software configurations       BCMA systems are typically integrated with an electronic medication administration record (eMAR) and electronic nurse documentation (Pedersen & Gumpper, 2008). BCMA hardware generally comes in three varieties: (1) wireless, handheld devices that both read bar codes and have a screen which displays the eMAR, (2) a handheld scanner that is tethered to a wireless computer-on-wheels (COW) that displays the eMAR and allows access to other relevant software, or (3) a tethered scanner attached to a fixed computer station in the patient room.

Cost       Initial capital costs can range from $1 million to $10 million considering all infrastructure, staffing, hardware, software, and training costs (Cummings, et al., 2005).

Typical use process       BCMA use typically follows a common protocol (Carayon et al., 2007; Cummings, et al., 2005; Larrabee & Brown, 2003; McRoberts, 2005; Neuenschwander, et al., 2003). Users scan their own ID badges to access the eMAR or otherwise log into the system. Next, the user acquires the necessary medications, compares the labels on the medications to the data in the eMAR, and scans the medication bar codes. If there is a mismatch between the medications scanned and those currently active in the eMAR, audible and/or visual alerts are triggered. Users should next scan the patient’s ID band, which may also trigger an alert. If the five rights have been confirmed, the user may administer the medication and document the administration. Documentation is typically automated with BCMA systems.

Adoption       The weighted adoption rate among US hospitals was 24% in 2007, up from (Pedersen & Gumpper, 2008) (The 24% included 100% of US Veterans Affairs (VA) hospitals; of the 875 non-VA non-specialty hospitals in the study, 240 [27.4%] had adopted BCMA, though adoption rates varied with hospital size.)

BCMA’s impact on safety       The most rigorous test to date found that non-timing errors were reduced by 41.4%, potential adverse drug events were reduced by 50.8%, and transcription errors were eliminated (Poon et al., 2010).

BCMA’s impact on work       Time spent on specific medication administration tasks changes. One study reported increases in percent of time nurses spent on information retrieval, verifying patient identification, and waiting (e.g., for the computer to operate) and decreases in percent of time nurses spent managing orders and delivering medications (Poon et al., 2008).       The quality of different medication administration tasks changes. One study reported that post-BCMA nurses perceived the process of checking identification to be more accurate, more useful, more consistent, and safer whereas they perceived the process of documenting administration to be less useful, less time efficient, less easy to perform, and no safer than pre-BCMA (Holden et al., 2011).       Nurses execute numerous different workarounds to BCMA protocols because of a variety of system design problems (Carayon, et al., 2007; Koppel, et al., 2008; Miller et al., 2011; Patterson, et al., 2002; van Onzenoort et al., 2008; Vogelsmeier, et al., 2008; Yang et al., 2012). Reported workarounds include duplicating information kept on paper, documenting an administration in BCMA prior to the actual administration, and scanning barcodes not affixed to the patient.

BCMA = bar coded medication administration.

^aWe refer interested readers to more complete discussions of BCMA, the empirical literature, and specific human-computer interaction issues surrounding BCMA use (Karsh, et al., 2011; Koppel, et al., 2008; Patterson et al., 2004; Patterson, et al., 2006).

Table 1 draws a distinction between BCMA’s “impact on safety” (BCMA → outcome) and “impact on work” (BCMA → work structures and processes). This study is concerned with the latter, namely, the impact of BCMA on problem solving. However, most studies of health information technology (IT) are of the impact-on-safety variety. As a consequence, those studies do not capture the mechanisms that mediate between BCMA and safety, a distal outcome. In a prior issue, Holden (2011) argued that from a human factors and ergonomics perspective, health IT’s impact on safety is necessarily mediated by a change in (cognitive) work structures and processes and by healthcare workers’ adaptations in response to such change. Similar contentions have recently appeared in the healthcare community, most notably the conclusions of a 2009 National Research Council report that the mere adoption of health IT is not sufficient to improve healthcare (i.e., BCMA → outcome) but instead “success in this regard will require greater emphasis on providing cognitive support for health care providers” (Stead & Lin, 2009, p.2). The above considerations urge additional investigations of BCMA’s impact on work, of which the present study is one example.

1.2. Adaptation – Problem solving

Operational problem solving (and more broadly, adaptive behavior) is a key component of cognitive work (Mumaw et al., 2000; Patterson, Woods, et al., 2006; Vicente et al., 2001). Hospital healthcare providers, particularly nurses, engage in problem-solving behavior in response to, or in anticipation of problems (Taylor, 1997) (see Figure 1). Here we define a problem as “any occurrence or state that makes goal accomplishment impossible, difficult, or unsatisfying in light of standards for timely and effective performance.” This is a more precise version of Tucker and colleagues’ (2002) definition of a problem as “an undesirable gap between an expected and observed state … that hinders a worker’s ability to complete his or her tasks” (p.124). The problems of present interest are those that result from a mismatch between the demands and the resources within a work system; thus, problems reflect variability in the entire work system, not merely variability in patient conditions. Similar conceptualizations of operational problems in healthcare have been variously named: performance obstacles (Gurses & Carayon, 2007); operational failures (Tucker & Spear, 2006); gaps (Cook, et al., 2000); blocks (Halbesleben et al., 2008); challenges (McAlearney et al., 2007); glitches (Uhlig et al., 2002); and non-routine events (Weinger et al., 2003).

Figure 1.

Problem solving, depicted as a goal-directed response to operational problems.

Problem solving is defined here as “any response to (perceived) problems in which a process or system is manipulated, in order to accomplish some pre-established goal relative to standards for timely and effective performance.” This modifies Tucker and colleagues’ (2002) definition: “identifying and resolving problems that occur in the execution of day-to-day work routines” (p.124). Our definition makes explicit that problem solving is goal-directed behavior. Workers may achieve goals by removing the problem temporarily or permanently (the literal meaning of “solving”), manipulating (e.g., reducing, “drilling through”) the problem, or working around it; the last option is what Figure 1 illustrates. Problem solving can be of first- or second-order variety. First-order problem solving involves “short-term fix” solutions (Tucker & Edmondson, 2003, p.60) or patches that are applied “on the fly” (Koopman & Hoffman, 2003, p.70) to fulfill some more or less immediate goal (Tucker, et al., 2002). Second-order problem solving, in contrast, involves “catching, correcting, and removing underlying causes” of problems and “actively seeking to prevent future occurrences of similar failures” (Tucker & Edmondson, 2003, p.56). Second-order problem solving is important for organizational learning and permanent solutions (Spear & Schmidhofer, 2005; Tucker & Edmondson, 2003; Tucker, et al., 2002). However, it requires resources such as time and a structured system for reporting, analysis, redesign, evaluation, and adjustment; these are often unavailable to nurses. Because our observations revealed almost no second-order problem-solving behaviors, this paper focuses on first-order problem solving.

In this study, we identified cases of problem solving in a pediatric hospital one-year post-implementation of BCMA technology. For each case we sought to identify (1) associated problems, (2) the relevant goals that were or were not met, and (3) whether the problem-solving behavior was affected (e.g., created, altered, hindered, triggered) by the introduction of BCMA. Our research question was:

• What is the nature of problem-solving behavior following BCMA implementation? To understand whether BCMA was responsible for the problem solving observed post-implementation, we (1) compared our observations to comparable observations one year pre-implementation and (2) identified problem-solving behavior deployed in response to BCMA problems.

2. METHOD

2.1. Setting

The study site was a 236-bed, academic, tertiary care, free-standing pediatric hospital in the Midwest US. At the time of data collection, February-March 2008, the hospital had recently completed a facility-wide implementation of BCMA technology. Table 2 further describes the hospital and the BCMA system. The study was approved by both the investigators’ and the hospital’s Institutional Review Boards.

Table 2.

Detail on the study hospital, study units, and bar coded medication administration (BCMA) system.^a

Hospital       Academic, tertiary care, free-standing pediatric community hospital in the Midwest US. 236 beds; annual patient admission of 12,463 in 2008.       In addition to BCMA, hospital information technologies included computerized provider order entry (CPOE), a Pharmacy Information System (PIS), and automated medication-dispensing cabinets managed through the PIS. At the time, orders initiated through CPOE were transcribed by pharmacists into the separate PIS, which automatically fed information into the BCMA system. Nurses could access the original order in the CPOE system as well as enter some (e.g., verbal) orders on behalf of the physician.

Study units       Pediatric intensive care unit (PICU). 36 beds. Multi-specialty unit providing critical care to children (ages birth and beyond) and to adults with unique childhood diseases, managing a spectrum of medical and surgical diseases including trauma, burns, and solid organ transplants.       Hematology/oncology/transplant (HOT) unit. 24 beds. Inpatient services provided to patients (ages infant to 18) with hematologic and oncologic diseases before and after bone marrow transplants.       Medical/surgical (MED) unit. 48 beds. Acute care provided to a mixed population of patients (ages infant to 18) with pre- and post-surgery and medical needs.

BCMA system       Hospital’s stated objectives of implementing BCMA: To improve patient safety by assuring that the correct medication (including pertinent attributes) is administered to the correct patient.       Software: Centricity Pharmacy (GE Healthcare). Software functionality included an overview of scheduled medications, an overview of previously administered medications, verification of the 5 rights of medication administration and alerts for discrepancies, alerts for late medications, medication-related documentation, and verification of employee/login. The electronic medication administration record (eMAR) used by the BCMA system was embedded in and modifiable through the Pharmacy Information System.       Hardware: PICU and HOT used a 4600G handheld gun for scanning. These devices were “dumb” reader input devices interfaced with Centricity Pharmacy software (here, “dumb” means that the scanning device read the barcode without analyzing it; analysis was done within the software). In the PICU the scanning device was attached to a computer-on-wheels and in the HOT unit to a tabletop computer. The MED unit used standalone handheld PDA-style scanners (Symbol Pocket PC model mc 5040) loaded with the software. These PDA-style scanners were also available to other units but were not observed to be used in HOT and were infrequently used in PICU.       Clinical tasks supported: Planning of medication administration, verification of the “5 rights” (right drug, right dose, right patient, right form, right route) and of the right user, documentation of medication administration, and documentation of medication related tasks such as monitoring of blood glucose levels.       Non-clinical tasks supported: Regulatory compliance for documenting controlled substances, administrative oversight of medication policy compliance, and to a lesser degree inventory management (following the study, the hospital also began using the system for charge capture).       Intended end users: Primarily bedside registered nurses (RNs). Nurse interns could use the system but could not sign off on medication administration. Physicians used the BCMA system’s eMAR to view an overview of the medications.       Implementation timeline and end-user training: Roll-out commenced in the HOT unit in December, 2006, and over the next six months was implemented first in the MED unit and then in the PICU. Nurses were trained on the system during in-services. Additionally, some nurses were trained to be super users in order to serve as just-in-time expert assistance on the units.

BCMA = bar coded medication administration; CPOE = computerized provider order entry; PICU = pediatric intensive care unit; HOT = Hematology/oncology/transplant; MED = medical/surgical; eMAR = electronic medication administration record.

^aInformation on the setting is based on conditions during post-BCMA data collection, February–March 2008.

2.2. Analytic framework

We used a cognitive systems engineering approach (Hollnagel & Woods, 1983; Hollnagel & Woods, 2005) to observe and then analyze interactions between domain experts (nurses) and BCMA technology in the process of cognitive work. We made three assumptions:

• problem-solving behavior could be observed during the course of nurses’ work; observed problem-solving behaviors were goal-driven, and operative goals could be discovered or inferred;

• problem-solving behaviors were carried out as a means to deal with current or anticipated problems that could potentially interfere with goal accomplishment.

Problems and problem-solving behavior were defined as above.

2.3. Procedure

Nurses’ work was observed, based on the Work System Analysis method (Karsh & Alper, 2005). A Work System Analysis is a method for observations requiring that data be collected about the people, tasks, technologies, organization, and environment so as to facilitate collection of contextual information during observations. Three researchers (industrial engineers/ergonomists) collected detailed, episodic data in three hospital units, a pediatric intensive care unit, a hematology/oncology unit, and a general medical-surgical unit. In the typical situation, the researcher explained the study during a shift change meeting and then announced his or her intention to observe nurses at work. During the day, the researcher either asked individual nurses for their verbal consent to be observed or nurses volunteered to be observed.

Researchers followed nurses during day shifts only and took handwritten notes on nurses’ actions and comments. Researchers attempted to be unobtrusive and only asked clarifying questions if the opportunity arose. Sometimes, nurses spontaneously volunteered clarifying information or commentary.

Observations totaled 47 hours across 17 unique nurses. We also conducted short interviews with 45 different nurses; during interviews, nurses were provided with preliminary workflow diagrams based on observations and asked to contribute feedback on the process, context about the process (e.g., why certain steps occurred), and any process changes associated with BCMA. Relevant documents and artifacts, such as nursing policies and the paper medication administration record, were collected. (For comparison, we also analyzed 89.5 hours of observations across 79 unique nurses, one year pre-BCMA implementation.)

2.4. Analysis

After data were collected, handwritten notes were transcribed and annotated by each researcher. Researchers grouped notes into discrete episode units describing each instance of problem solving along with relevant quotes and comments. This initial set of episodes included problem solving both related (89 episodes) and unrelated (224) to BCMA. For each episode, the researchers specified the goal blocked by the problem; the problem; the precise actions used to solve the problem; and additional comments, such as the eventual outcome of the episode. By decomposing episodes, the researchers were able to more precisely distinguish problem-solving behavior from behavior that was simply workers doing challenging work. Instances of the latter did not yield clear answers to “what was the precise problem-solving behavior?”, “what problem was addressed?”, and/or “what goal did the problem block?”

Whole episodes, or parts of episodes when applicable, were then subjected to descriptive coding (Miles & Huberman, 1994), wherein categories were generated based on the type of problem-solving behavior observed in the episode. Three researchers conducted the coding independently and met intermittently to discuss, modify, and ultimately come to consensus on the emerging coding scheme. As a validity check, two researchers who did not participate in the initial analyses (authors B.K., an engineering professor, and M.C.S., a pediatric physician), reviewed the analysis; discrepancies were discussed until consensus was reached.

Finally, to address the central research question, the episodes were analyzed for emergent themes about the interaction between BCMA and problem solving.

3. RESULTS

A total of 313 episodes of problem solving were identified, of which 89 were related to BCMA. These 89 were grouped in three broad themes, described below: (1) BCMA facilitated novel problem-solving behavior; (2) BCMA blocked familiar problem-solving behavior; and (3) BCMA introduced new problems. Table 3 contains representative example episodes in each theme, decomposed into corresponding goals, problems, and problem-solving behaviors. Figures 2 – 4 graphically illustrate each theme.

Table 3.

Identified themes and subthemes about BCMA’s impact on problem-solving behavior with example episodes.

Theme/ Subtheme (# contributing episodes)	Example episodes
	Goal	Problem	Problem-solving behavior (and type of behavior/strategy)
1. BCMA facilitated novel problem-solving behavior (13)	Administer right medication at the right time.	Nurse does not know if a medication scheduled to be administered once per shift can be administered yet.	Nurse scans a medication’s barcode to see if the BCMA system accepts the medication, which will tell the nurse whether the medication can be administered at this time. (Electronic solution)
	Quickly access medication information.	Multiple orders in system make quick information seeking difficult.	Nurse sorts medication orders by name in the BCMA software before printing a copy of the medication order list. (Electronic solution)
2. BCMA blocked familiar problem-solving behavior (14)	Administer total of 20mg of a just-ordered medication right away.	Pharmacy will require some time in order to deliver the 20mg dose, and the BCMA system will not allow anything but a single 20mg dose to be administered, even though 2 × 10mg doses are immediately available.	Nurse cannot immediately administer and document a clinically equivalent therapy of 2 × 10mg, a typical solution pre-BCMA. Instead, nurse recruits the help of a pharmacist to create two new 10mg orders in the BCMA system so that 2 × 10mg can be administered as a substitute for a single 20mg dose. The first order is a standing order for 10mg that the nurse can give once right away; it remains in the system until the pharmacist removes it later. The second order is a one-time 10mg order that is discontinued once documented as administered. Pharmacist informs a fellow pharmacist to modify the original 20mg order so that it does not appear as missed (non-administered). (Collaboration, system workaround)
3. BCMA introduced new problems (62)
3a. Nurses responded to new BCMA-problems with familiar problem-solving behavior (29)	Document medication discrepancies in the BCMA system.	Nurse feels uncomfortable using the BCMA system.	Nurse asks a fellow nurse to carry out BCMA documentation tasks in her stead. (Collaboration)
	Plan the day and carry out planned tasks when needed.	There is a lot of information to keep track of and manage throughout the day.	At the start of shift, nurse selects own patients in BCMA system and for each prints the “Timeline” screen from the BCMA software. Nurse then transfers information from this print-out to a gridded sheet of paper that the nurse will refer to and update throughout the day. (Use of paper artifacts, time management)
3b. Nurses responded to new BCMA-related problems by inventing novel problem-solving behavior (13)	Document/reconcile a missed medication administration.	Nurse does not like default options offered in BCMA software for documenting reason for missed medication.	Nurse finds an option to enter a note for why the medication was missed, providing a narrative description that was not possible using the default menu of options.a (Electronic solution)
	Identify in BCMA system which patient will receive medication.	Patient is too far away and the nurse is using a tethered BCMA scanner.	Nurse selects patient manually in BCMA. (System workaround)
3c. Nurses lacked effective problem-solving behavior to address new BCMA-related problems (20)	Administer a medication on time.	In initial attempts, medication does not scan into BCMA.	Nurse tries to scan the medication repeatedly, and then at multiple different computer stations, without success. (Try and try again)
	Administer current dose of scheduled medication, and Avoid taking responsibility for medication missed by nurse on previous shift.	Previously scheduled dose for a medication was not administered by another nurse on the past shift, and BCMA system requires a reason to be documented before another dose can be administered.	After unsuccessful attempt to have pharmacy clear the previous (missed) dose from the BCMA system, nurse does nothing/waits. (Do nothing/wait)

BCMA = bar coded medication administration

^aAlthough the free text note feature was built into the system, for whatever reason the nurse was not aware of it and therefore spent several minutes attempting to document the administration in the standard way (using default menu options) before “discovering” the note feature.

Figure 2.

Theme 1: BCMA permits a new problem solving path for addressing an existing problem.

Figure 4.

Theme 3: BCMA introduces new problems.

3.1. (Theme 1) BCMA facilitated novel problem-solving behavior (Figure 2)

BCMA allowed nurses to invent new problem-solving behavior to deal with pre-existing problems (Figure 2). Most notably, BCMA permitted a new path for information gathering and sensemaking, as shown in the following examples:

• Post-BCMA, nurses could electronically sort medications by name, rather than having to search through a multi-page paper medication administration record (MAR).

• Post-BCMA, nurses used an electronic “timeline” feature to determine precisely when medications were documented as administered.

• Post-BCMA, to determine whether a medication could be presently administered, nurses could simply scan the medication’s barcode to see whether BCMA would “permit” its administration. In effect, nurses could shed some of the labor-intensive computation required pre-BCMA (look up/reconstruct/compute administration history, administration schedule, rules for spacing re-administrations, etc).

3.2. (Theme 2) BCMA blocked familiar problem-solving behavior (Figure 3)

Figure 3.

Theme 2: BCMA blocks a familiar problem-solving path.

Prior to the introduction of BCMA, nurses demonstrated a preference for certain problem-solving behaviors, many of which had developed over time and could be employed to solve a variety of problems. These were consistent with prior research on nurses’ work strategies (Bowers et al., 2001; Eisenhauer, et al., 2007; Hardey et al., 2000) and included the use of paper, team-based problem solving (e.g., engaging a colleague to provide assistance), and the bundling or strategic sequencing of tasks. The introduction of BCMA made it difficult or impossible to apply some of these “familiar problem-solving behaviors” (Figure 3). One case, described in Table 3, involved a nurse needing to administer 20mg of an urgently needed medication (for more detail and analysis, see Holden et al., 2008). In this and other cases of blocked problem solving, we observed that workers often developed ad-hoc solutions: in the 20mg dose case, a nurse received assistance from a pharmacist who had to, in the pharmacist’s words, “play games with the [BCMA] system” (i.e., subvert normative BCMA use) to make the system accept two 10mg doses instead of one of 20mg. This ad-hoc solution was convoluted, unpracticed, and therefore riskier than the familiar solution (Holden, et al., 2008).

3.3. (Theme 3) BCMA introduced new problems (Figure 4)

The BCMA system contributed to new problems that nurses addressed in one of three ways (Figure 4).

3.3.1. (Theme 3a) Nurses responded to new BCMA-problems with familiar problem-solving behavior

When nurses encountered new problems related to BCMA, they often applied behaviors commonly used to solve problems unrelated to BCMA:

• Nurses involved their teammates (especially other nurses) in problem solving, relying on collaboration to solve problems both related and unrelated to BCMA.

• The ubiquitous use of paper (e.g., to plan one’s day, to maintain awareness of patient status) persisted following the introduction of so-called “paperless” BCMA. Paper use became a way to replace or complement BCMA use. Nurses would create portable, lightweight, medication administration schedules by printing the administration “timeline” from the BCMA software or would use paper to document medication administration when the BCMA system was inoperable. Nurses continued to use paper “cheat sheet” artifacts developed pre-BCMA as informal checklists, reference sheets, and repositories of notes of clinical importance.

• Nurses had a variety of familiar strategies for managing their time and supplies, for example, the “bundling” and reprioritizing of tasks or the “borrowing” of medications (Alper et al., 2008). They applied similar strategies to BCMA-related problems. For instance, the medical/surgical unit was using portable, handheld BCMA scanners that were slow to boot up and/or scarce. In response, one nurse turned on the scanner well in advance of needing it to deal with slow boot up or left the scanner on a table outside their patient’s room or in their pockets to deal with scarcity.

3.3.2. (Theme 3b) Nurses responded to new BCMA-related problems by inventing novel problem-solving behavior

New problems created by BCMA sometimes triggered the creation of new problem-solving behavior. Post-BCMA, nurses encountered difficulties documenting medication administration discrepancies such as delayed, missed, changed, or non-scanned medications. The BCMA software offered several pre-set options for documenting such occurrences, but nurses often found them inadequate. Nurses quickly discovered that they could use a text note feature in the system to document actions and reasons pertaining to administration discrepancies.

Most episodes of novel problem-solving behavior applied to BCMA-related problems involved nurses working around the normative process for BCMA use. Sometimes nurses selected patients and medications manually in the BCMA software instead of scanning the respective barcodes. Those workarounds were applied to deal with problems such as the patient being too far away from a tethered BCMA scanner; problems scanning barcodes on oily, wrinkled, or otherwise difficult (if not impossible) to scan packages; or because medication was transferred to or diluted into a non-bar-coded syringe due to the amount of medication to be administered. Some of the workarounds clearly purchased additional risk, for example:

• Scanning a patient’s bar-coded identification label on a piece of paper rather than connected to the patient’s body as per protocol.

• Telling the BCMA system that a medication was administered before it really was, to ensure that it was documented as administered within the allowed window of time.

3.3.3. (Theme 3c) Nurses lacked effective problem-solving behavior to address new BCMA-related problems

At times BCMA introduced new problems to which nurses applied ineffective problem-solving behaviors or for which nurses possessed no acceptable solution. A behavior observed in fifteen episodes was what we termed “try and try again,” wherein a nurse would attempt the same action repeatedly to achieve some goal. For example, nurses were often observed to repeatedly try to scan medication barcodes that would not register with the scanner, e.g., because of reflective surfaces or unevenness of the barcode label. Some barcodes never registered despite repeated attempts. Nurses also “tried again” or “started over” when encountering problems logging onto the BCMA system or BCMA software crashes.

Another unsolved problem was witnessed when a nurse wanted to administer a new dose of a medication but was blocked by BCMA from proceeding. This was because the previous dose of the medication, due during a previous shift, was not documented as administered in the BCMA. The BCMA system first required documentation of the previous dose before the current and clinically indicated dose could be administered. However, the observed nurse did not wish to take responsibility for missed (or undocumented) medication from the previous shift. She attempted unsuccessfully to have a pharmacist remove the original dose in the system then simply waited, not knowing how to proceed.

4. DISCUSSION

Although it is difficult to make a quantitative assessment of relative frequency, problem solving was a commonly observed behavior of pediatric nurses engaged in medication management. This is not unexpected, given that previous research has identified many problems related to medication management (Gurses & Carayon, 2007; Tucker & Spear, 2006). The introduction of BCMA had a variety of effects on problem solving. Neither problems nor problem solving disappeared with the introduction of BCMA, and several new problems and problem-solving behaviors were created as a result, as also reported elsewhere (Koppel et al., 2008; Patterson, Rogers, et al., 2006; Vogelsmeier et al., 2008).

Unlike most health IT research, this study was focused on technology’s impact on work, not on endpoint outcomes such as patient safety. However, because “impact on work” is a broad phenomenon, we sought precision by restricting analyses to one aspect of cognitive work: problem solving. Accordingly, we were able to decompose observed behavioral episodes into pre-defined components (goal, problem, problem-solving behavior), facilitating pattern recognition within a wealth of qualitative data—in this case, the patterns emerged as themes about the intersection of BCMA and problem solving. Further, unlike the increasingly popular studies of unintended consequences of health IT (e.g., Ash et al., 2009; Harrison et al., 2007), ours did not investigate only the undesirable impact of BCMA. Indeed, results showed that the impact of BCMA was mixed—new problems emerged (a phenomenon expected with any intervention) and new problem solving was required, but some (though not all) pre-existing problems were better solved with BCMA.

Present findings contribute in two ways to human factors/ergonomics theory and design of healthcare systems: by shedding light on hidden hazards in complex work and informing the design of health IT.

4.1. Hidden hazards in complex work

Our data show that nurses responded to operational problems with creative problem solving. A hallmark of professionals in complex systems, problem solving provides system resilience and buffers against adverse events but represents extra work to compensate for inadequate systems (Farrington-Darby & Wilson, 2009; Hollnagel et al., 2006; Mumaw, et al., 2000; Roth et al., 2006; Vicente, 1999; Vicente, et al., 2001). The dark side of problem solving is that, when effective, it can mask the existence of problems and the presence of elevated levels of risk despite acceptable outcomes (Karsh, 2011). Fortunately, one gains insight into design problems by systematically studying workers’ compensatory problem-solving behaviors. For example, nurses in this study worked around the BCMA use protocol when using tethered scanners. This was a risky but effective compensatory strategy for solving a problem that could also be solved by procuring longer tethers, adjusting the layout of the room to fit the purchased tethers, or using wireless scanners.

At other times, workers cannot, or do not, compensate for poor design. For example, nurses who were observed having problems logging onto the BCMA system or scanning a barcode repeated their attempts, sometimes dozens of times, until they gave up or succeeded. Even when the required performance was achieved, costs included frustration and potential delays in patient care. Nurses’ perseverance is ultimately good news for patient care, but perhaps bad news for identifying problems (e.g., log-in inefficiencies, scanner problems) as needing a sustainable solution. Again, observing workers’ attempts at problem solving, even suboptimal ones, helps to identify design problems.

Problem solving may sometimes impose as much or more risk than leaving the problem unsolved, as in the case of the 20mg dose. (In that case, leaving the problem unsolved would have meant delaying treatment while the nurse waited for pharmacy to prepare a 20mg dose.) Problem solving may push workers to their performance limits by increasing workload, introducing unfamiliar demands, or shifting work away from well-practiced routines of skill-based or rule-based performance. Problem solving may interrupt nursing work, taking nurses’ time and attention away from patient care (Tucker & Spear, 2006). Finally, problem-solving behaviors may be inelegant, temporary “patch jobs” that may unravel at inopportune times or may contain unknown hazards. Thus, in addition to eliminating problems through design (thus, obviating problem solving), an interesting design challenge is to develop systems that support workers as they problem-solve, allowing “safe resilience” during non-routine performance.

4.2. Problem solving as input into IT design

Understanding why certain problem-solving behaviors are used after IT implementation can be instructive to IT design. For example, in nineteen episodes, nurses used paper artifacts instead of or in addition to BCMA, to solve problems (cf. Hardey, et al., 2000; Saleem et al., 2009). Examining what nurses were not afforded by BCMA but were afforded by paper indicates three design considerations that appeared lacking in BCMA: anticipatory control, ecologically valid design, and basic usability.

4.2.1. Anticipatory control

In control-theoretic terms, nurses’ cognitive work entails (a) setting and adjusting goals (“targeting”), (b) following a goal-directed path (“tracking”), (c) scanning for deviations from the goal-directed path (“monitoring”), and (d) making anticipatory or compensatory corrections (“regulating”) (Hollnagel & Woods, 2005). This control process matches our description of problem-solving behavior as goal-directed adjustments in response to actual or anticipated problems that may cause deviation from a goal-directed path. In complex, time-sensitive systems such as air traffic control or nursing, anticipatory control is especially critical (Rasmussen, 2000). We observed little intentional design to support anticipatory control with BCMA. The administration “timeline” feature was the major exception. Consequently, nurses often used paper artifacts such as one-page “cheat sheets” to anticipate and adjust for future needs. We believe this is because the BCMA was explicitly designed to provide clinical information “at the point of care,” primarily error detection during the act of administration, but largely ignored information needs “before the point of care.” For example, a nurse could not see in the BCMA system non-medication information needed to plan and manage medication tasks: When are my patients’ procedures scheduled? How does this patient react to her medications? What tasks are done and what is left to do? In contrast, seemingly simple pieces of paper (which BCMA was meant to replace) served as ever-present, portable, easily adaptable, information-rich, complex cognitive support tools that enabled nurses to look into the future and achieve anticipatory control (Jones & Nemeth, 2005).

4.2.2. Ecologically valid design

The final design of technology reflects designers’ (sometimes implicit) understanding of the nature of work that the technology will support; this understanding is sometimes ecologically invalid—that is, quite different from the nature of work as it is actually done (Berg, 1999; Carroll & Campbell, 1989; Forsythe, 1996; Karsh et al., 2010; Wears & Berg, 2005; Woods, 1998). Ecologically valid design, or contextual design (Beyer & Holtzblatt, 1998), is design that is based on an accurate understanding of the work. In actual work, for example, nurses need information, both medication-related and non-medication related. In this study, the BCMA system’s design—premised on a potentially over-simplistic medication-centric view of nursing work—provided information about medications but not about patients’ scheduled meal times, blood draws, tests, or procedures, each of which could impact the timing of medications. Other care and comfort goals were not supported at all by BCMA nor were multiple goals that had to be coordinated, for example the care of multiple patients with different needs and schedules. To accomplish these unsupported goals, nurses used paper artifacts that they had designed to match their actual work: information on multiple goals (medications, meals, multiple patients) were combined on unstructured paper documents. Nevertheless, having multiple data sources (both paper and electronic) and the fact that paper artifacts were accessed by nurses but not by others (e.g., pharmacists) may have increased risk.

Another reality of nursing work, including medication management, is its collaborative nature. Consistent with Patterson et al’s (2002) study of BCMA, the BCMA design studied here did not adequately support collaboration. Screens on portable BCMA devices were too small to allow synchronous viewing while desktop BCMA terminals were placed in cramped patient rooms, blocking the involvement of other nurses or even patients (indeed, the BCMA design assumed that patients and their visitors were uninvolved). Help-seeking and -providing behaviors, though ubiquitous, were not supported, and were in fact constrained: for example, many steps are required to “add” then “drop” another nurse’s patient in BCMA. The documentation feature of the BCMA system did not appear designed to communicate information between nurses and other members of the care team (nurses, physicians, assistants, pharmacy, patients): its structured format, red fonts and warnings, and refusal to “move on,” conveyed to nurses that documentation mainly served surveillance and disciplinary, not communication, purposes. Finally, the nurse’s role on the multidisciplinary care team was constrained by BCMA because BCMA assumed by default that any order parameter entered by a physician or added by a pharmacist must be exactly carried out by nurses. In reality, nurses interpret orders, make schedule adjustments to provide timely and optimal care, and use clinical judgment about when to administer medications (Beuscart-Zephir et al., 2005; Institute of Safe Medication Practices, 2010). By enforcing medication parameters such as the interval between re-administering as-needed medications, the studied system conflicted with nurses’ routine exercise of clinical judgment. Although nurses were not necessarily afforded better collaboration through the use of paper artifacts, other studies have described how healthcare workers use non-electronic artifacts such as whiteboards in the emergency department (ED) to facilitate shared situation awareness and communication within distributed, multidisciplinary teams (Xiao, 2005; Xiao et al., 2007).

4.2.3. Usability

Nurses also used paper instead of or in addition to BCMA because BCMA exhibited poor usability: poor fit between the BCMA and the work system elements with which it interacted, from poor software-user fit (e.g., unintuitive options, key functions buried with multiple menus, navigation icons that did not match the behavior of icons in other software) to poor hardware-physical environment fit (e.g., short tethers, keyboards in recessed, unilluminated cabinets). Paper artifacts, more flexible and intuitive, presented fewer usability challenges. Usability-related problems not mitigated by experience and typical training require better design.

4.3. Limitations

Only one hospital and one BCMA system were studied, limiting generalization to other hospitals, settings, and BCMA systems. We were limited in data collection, analysis, and interpretation to only that which was (1) observed/observable, (2) self-reported by nurses during the course of the observation, or (3) explained during an interview or in consultation with clinician colleagues. Due to resource limitations and assurances of confidentiality, we did not ask observed participants to validate our analysis, and instead relied on clinician collaborators to inspect our analysis for external validity. Future research may benefit from further quantifying problem-solving behavior.

Although we identified apparently new problems following the introduction of BCMA, it is possible that some of these were old problems that became more obvious with BCMA. In most cases of new problems, it was also difficult to discern whether the problems were strictly related to poor BCMA design, poor BCMA implementation, or poor integration between BCMA and work process: across cases, all three appeared to be at play.

Our analysis of the studied BCMA’s shortcomings yields suggestions for more anticipatory control and ecologically valid design. Actualizing those suggestions might involve adding information or functionality to the BCMA to support, though not necessarily replace, nursing work. Further inquiry should determine consequences of increased information and functionality, including the potential for decreased reliability and work overload associated with increasingly complex technology (Bainbridge, 1983; Sarter et al., 1997).

Finally, adaptation to and of technology is a phenomenon that unfolds over time and must in the future be studied over many months and even years to truly understand how technologies, processes, people, and organizations are adaptively transformed after the introduction of new IT (DeSanctis & Poole, 1994; Majchrzak et al., 2000). Longitudinal studies must also be designed to understand the organizational routines that predate new IT, because these will influence adaptation (Novak et al., 2012).

5. CONCLUSION

Woods (1998) observes that “practitioners adapt to difficulties” (p.169) and this applies equally to nurses as to pilots, control room operators, or military personnel. Adaptive problem-solving behavior is a double-edged sword: it precludes failures not prevented through design but also obscures those design flaws. The study of problem solving can help identify those flaws and serves as input for design. When change, such as new technology, is being planned, designers can inspect pre-change problem solving to obtain an accurate understanding of work, replacing the inaccurate implicit theories of work typically held by designers and implementers. Following the change, the observation of problem-solving behavior can inform design/redesign. Our own observations of nursing work and the impact of BCMA on problem-solving behavior suggest that design needs to include anticipatory control, ecologically valid design, and IT usability.