Mapping Cognitive Work: The Way Out of Healthcare IT System Failures

Abstract

The failure of automation to improve clinical performance is likely rooted in the design concepts on which IT systems are based. Current systems provide clinicians with specific direction about how to care for individual patients. This is much like the specific, detailed, complicated, and narrow trip route driving directions that can be obtained from various web sites. Daily healthcare work rarely has the certainty that makes such directions useful. Rather than directions, useful healthcare automation is likely to have characteristics of a map. Clinicians could use its depictions of available routes, obstacles, and distances between the current and goal locations in order to choose routes and to track progress toward goals. Such representations are likely to be quite different than those currently incorporated in healthcare automation. We demonstrate the concept of creating maps and using constraints as the basis for the design of healthcare automation.

Introduction

As two of their strategies to improve patient safety, the Institute of Medicine¹ recommended improving access to accurate, timely information, and making relevant information available at point of patient care. Evidence shows IT efforts to respond to real world requirements in healthcare and other sectors are problematic. At least one third of new large information technology (IT) systems in 1994 failed and the remainder fell short of budget and schedule objectives.² Clinical decision support systems’ effect on practitioner performance and patient health remain as inconsistent as they were 15 years ago.³ Systems that are intended to improve on healthcare performance and patient safety are now perceived to create new forms of unintended adverse outcomes.⁴ Heeks, Mundy and Salazar⁵ contend that “many even most health care information systems are failures.” These failures of automation to improve clinical performance are likely rooted in the design concepts on which IT systems are based. This paper describes healthcare cognition and research, and demonstrates the concept of creating maps as the basis for the design of healthcare automation.

Healthcare Information Display

IT systems are often touted as a way to improve patient safety. Attractive human-computer interaction (HCI) features are promised to be implemented once the system has been put into operation. However, the inherent complexity of these new systems and the high cost of integration into existing systems delay the introduction of these desirable features. Once in operation, the need for stability, the pressures of high operational tempo, and the lack of resources make retrofitting high quality HCI features dif cult or impossible. This forces operators to create their own workarounds to compensate for features that never get implemented.

Operator performance in high hazard settings requires the ability to appreciate previous and current states of the system in which they work, and to foresee the implications of previous and current circumstances for what is to follow. In complex systems, these activities rely on the use of one or more displays. Display quality matters, because artifacts shape cognition and collaboration. The way a problem is presented can improve or degrade the cognitive work of operators.⁶

Displays that are suited to cognitive work at both the patient and at the unit level can improve the reliability and efficiency of clinical work, patient safety, and minimize gaps in the continuity of care.⁷ Cognitive artifacts include hard copies of notes, checklists, and status boards, as well as control/display interfaces on individual pieces of electronic equipment, assignment schedules that are shown throughout a facility on computer monitors, and more. These items are an integral part of clinicians’ daily work. Healthcare information displays serve as cognitive artifacts⁸ that influence the delivery of patient care and the coordination of care across and among patients. However, Berg⁹, Heath and Luff,¹⁰ and Ash, et.al,¹¹ among others, contend that IT systems are not suited to clinicians’ cognitive work. For example, clinician decision support systems (CDDS) provide practitioners with specific directions on how to care for individual patients. This is much like the specific, detailed, complicated, and narrow trip route driving directions that can be obtained from web sites such as MapQuest or Expedia. Daily healthcare work rarely has the certainty that makes such directions useful. A different approach to IT design would map a way out of these failures and toward successful support for healthcare applications.

Representations in Healthcare

Clinicians must integrate vast amounts of data in order to make decisions. Typically, IT systems are configured to make as much data available as possible, as reflected in new designs for electronic medical records and telemedicine workstations. Current attempts to manage this array of data try to simplify it by directing attention to what is presumably an optimal path or an optimal set of data. The healthcare work domain is far too complex and uncertain for such an approach.

Rather than specific directions to follow one path to achieve a goal, useful healthcare automation is likely to have characteristics of a diagram, such as a map. Such representations may be quite different than those currently incorporated in healthcare automation. Rather than providing text directions for one way to pursue a goal, a map shows all of the possible routes and allows for these routes to be compared, evaluated, tried out, and used as alternates under different conditions. Using a map of the work domain, clinicians can see available routes, limitations, and distances between the current and goal locations, as well as decide on routes to take and track progress toward goals. A representation is an information ecology, or map, of the work domain that is created to assist the way that clinicians work. Representations can be primarily graphical with alphanumeric elements, or primarily alphanumeric with graphical elements. However it is configured, the fundamental significance of a representation is not in its visual qualities but in how well its visual qualities correspond to elements in the work domain that it is intended to represent what Woods and Hollnagel¹² refer to as its domain semantics.

Skillfully crafted representations facilitate and empower clinician judgment by portraying the relationships among domain semantics, particularly the hard and soft constraints that bound the space of possible solutions. Representations synthesize pertinent elements including scale, relationship, and other aspects of cognitive work that clinicians would otherwise have to combine on their own. They also offer the ability to refine the way that data and information are presented. Effective representations spare operators the task of data synthesis by abstracting the many discrete elements that comprise the work setting at the patient and unit level.¹³

At the patient level, clinical decisions rely on the ability to synthesize many different streams of data in order to perform diagnosis, therapeutic intervention, and monitoring. All rely on direct observation of the patient condition, the patient’s self-report, past/current vital signs, test results, and expert consultant opinions. In addition to primary reviews of data, clinicians also assess data (are they accurate, current, sufficient?), and evaluate prospects (is action indicated, are further data needed, would waiting be prudent?), among other activities. A variety of complex, sophisticated equipment also plays a role, particularly in the intensive care unit (ICU) and emergency department (ED). Patient-level representations can be used to synthesize different types of data through the use of diagrams that leverage human skills such as pattern recognition. Patterns and symbols can be used to support crucial tasks such as the comparison and contrast of various data, and the assessment of trends. Mapping underlying semantics involves representations of relevant variables. These include current and previous patient states, planned medications and procedures and potential interactions. Clinicians can use the array of options in a representation to negociate hard constraints (e.g., the patient has diabetes) and soft constraints (e.g., history and current condition show the patient could tolerate a slightly lower blood pressure that Drug B may induce).

Unit-level representations reflect the newly-developing understanding of technical work.¹⁴ That data spans groups of patients, the current number of patients on the unit, their locations and condition, pending and in-progress diagnosis and treatment, current care providers, and prospective transfer in or out of the unit. In the case of intensive care units, mapping the underlying domain semantics involves representations that support matching resources such as nursing assignments and practitioner attention to a range of current and anticipated demands. In high tempo settings such as the ICU and ED, clinicians develop their own intrinsic mental models of patients and of unit activity. Figure 1 suggests how clinicians rely on discrete, interrelated cognitive artifacts as part of the distributed cognition⁷ that is required to operate in the unit’s uncertain, contingent, tentative and fast-changing circumstances. The panel at left portrays the current circumstance in which clinicians develop their own mental model of each patient and of the unit as a whole. Data sources are separate and vary widely. They include a unit status board showing planned procedures and staff assignments, a monitor showing vital signs telemetry for each patient in the unit, patient chart information, and more. The task of pulling together all of the individual elements of data into a coherent mental model falls to the clinician. Mental models for patients and the unit can differ depending on what artifacts each clinician has seen. The panel at right suggests an approach to support for the cognitive work that clinicians perform that synthesizes data on behalf of the clinicians. The cognitive artifact at right depicts the entire unit based on the past, current and anticipated data that are related to each patient. As data related to the patients change, their representations change. As patient representations change, the representation of the unit changes. In the setting shown at right, clinicians have the opportunity to probe for more particular data related to an individual patient, or to view the unit as a whole. The representations also assist operators in the negotiation of hard constraints (e.g., only six rooms are available) and soft constraints (e.g., that scan can be postponed).

Fig.1.

Cognitive artifacts affect clinician cognitive work

Means to Understand Healthcare Cognition

Patient safety requires methods that get at the actual nature of healthcare as a work domain. They also require action that is informed by that understanding in order to develop effective procedures, hardware and software to assist clinicians’ work performance. Practitioner goals and constraints at work are some of the many factors that comprise the technical work context. Understanding practitioner cognitive work relies on the kind of research activity that is not normally found within the field of healthcare. Cognitive engineering methods¹⁵ such as cognitive task analysis can be used in conjunction with observation in order to map the distributed cognition processes that are related to daily work activity. The naturalistic decision making (NDM) approach¹⁶ has evolved among social scientists and engineers within the past 20 years as the preferred means to human cognitive research. NDM uses stories and mental simulation to capture competing high-level goals in the real world that are under-specified, unstated, varied in their presence, poorly delineated and interactive. NDM employs ethnomethodological techniques in order to better understand how humans in groups dynamically engage the world. Research professionals who perform this type of work in healthcare use methods that include observational study, artifact analysis, workplace studies, schemata analysis, and mental model analysis to understand practitioner cognition at the sharp (operator) end.¹⁷ Each was used to form the foundation of understanding for the following display concept.

An Example of Representation

The coordination of surgical procedure anesthesia assignments for 50–80 cases a day in the outpatient clinic and surgical suite of a major urban teaching hospital relies on the use of cognitive artifacts including the master schedule. Figure 2 shows two versions of the schedule. Clinicians developed the paper version (at left) represents the coordinator’s work domain but ignores time. The version at right illustrates a conceptual prototype for a computer-based version to support anesthesia scheduling. The design draws on the findings from research into schedule development and use that revealed time is a primary aspect of this work setting.¹⁸ It incorporates insights drawn from from methods to develop user-centered systems.¹⁹ Six of sixteen rooms in an in-patent operating room (IOR) unit are shown. Information on each case is displayed in a horizontal bar that is aligned next to the label of the operating room to which it is assigned. A shaded segment follows each procedure to indicate the 45 minute period that is required to clean-up and restock the room. The arrow at top of the display indicates that the time is 0800 on the day of procedures that are being conducted in the IOR unit. Properties of the prototype reflect the schedule coordinator’s domain semantics, including time, clinical staff availability and qualifications, patient volume and characteristics, and room and equipment availability. By using a graphic representation based on time, the coordinator can understand and evaluate relationships among events through time. The prototype supports coordinator needs throughout the day to re-plan, resolve disputes, negotiate, speculate, create trial solutions, anticipate bottlenecks or opportunities, stash resources, husband resources, hedge resources, and bump procedures. Certain information is crucial in order to optimize assignments. This includes knowing when procedures are likely to finish, which procedures can be moved into another room, which opportunities (such as Medicare payment) might be exploited, and where conflicts might occur (such as the overlap shown by the black bar in IOR6). Cases that have already been performed remain on the display in sequence, making it possible to review the entire day’s activities while they are still underway. This proposed approach reveals aspects of schedule management that were hidden in a currently operating computer-supported display that was developed without the kind of research this paper describes.

Fig.2.

Paper copy scheduling display (left) and potential computer-supported version (right). Names are ficticious.

Conclusions

Representations are not simply diagrams, any more than work is simply tasks. Representations are assemblies of relevant information that provide a space for clinicians to succeed within existing constraints and boundaries. This is especially true in instances where work is most difficult, because representations expand the variety of options that are available for clinicians to consider. Representations that are conceived at this level make it possible for workers to examine, compare, and contrast various possible solutions. This includes solutions that violate hard boundaries such as the number of available rooms and soft boundaries such as organizational guidelines, because workers must necessarily make trade-offs in the course of daily work. This use of representations promises to improve clinician performance. Improvement to clinical performance offers a way out of healthcare IT system failure.