Creating Resilient IT: How the Sign-Out Sheet Shows Clinicians Make Healthcare Work

Abstract

Information technology (IT) systems have been described as brittle and prone to automation surprises. Recent report of information system failure, particularly computerized physician order entry (CPOE) systems, shows the result of IT failure in actual practice. Such mismatches with healthcare work requirements necessitate improvement to IT research and development. Efforts to develop successful IT systems for healthcare’s sharp end must incorporate properties that reflect workers’ initiative in respose to domain constraints. Resilience is the feature of some systems that makes it possible for them to respond to sudden, unanticipated demands for performance and return to normal operation quickly, with minimum decrement in performance. Workers create resilience at healthcare’s sharp end by daily confronting constraints and obstacles that need to be surmounted in order to accomplish results. The sign-out sheet is an example of resilience that can be used to guide IT development.

Introduction

Information technology (IT) systems have been developed as a means to improve efficiency and reliability across a spectrum of commercial and institutional applications. Worldwide annual purchases of software and services by organizations and governments have reached an estimated $1 trillion.¹ Even at this significant size, evidence shows that IT efforts to respond to real world requirements are problematic. For example, at least one third of new large information technology (IT) systems in 1994 failed and the remainder fell short of budget and schedule objectives.²

IT has also been employed to improve efficiency and reliability goals for healthcare’s blunt (management) end and has more recently been suggested as a means to improve patient safety at its sharp (operator) end. Through the 1970’s hospital information systems were largely created to handle data that was related to business matters such as billing, supplies, and operations. Intended benefits such as improvements to reliability have more recently been accompanied by unintended . consequences of automating clinical information. Those consequences include clinician over-trust of decision support, the imposition of additional work on clinicians (as they develop workarounds to surmount system shortcomings), and the upset of smooth working relations and communications among care providers.³ Cost considerations also have an effect on IT system development and use. Software systems are often promoted as cost-effective, flexible, and scalable. However, Decruyenaere, et.al ⁴ report that, like other IT systems, the total cost of ownership for ICU information systems is many times more than the initial purchase price. This stems from the unanticipated expenses of installation, use, maintenance, upgrades, changes and disposal over its lifespan. Once IT systems are implemented, pressures to limit costs force IT staff attention to switch exclusively to the support of individual system functions. Cost pressures to limit expenditures compel organizations to leverage existing platforms by making only minor variations in them to serve many applications. For that reason, systems that were developed for one purpose, such as billing, are adapted to serve additional purposes, such as anesthesia assignment scheduling, for which they were never intended.

IT’s Mismatch with Healthcare’s Sharp End

Three kinds of healthcare IT serve as an example: medical records, decision aids, and medical equipment. Electronic versions of medical records attempt to make the large amount of information that they contain useable. Despite these efforts, clinicians find the records to be a poor match for the kinds of cognitive work that they must perform. This mismatch arises from increasing reliance on the medical record to support billing for clinical activity, configuration of records to assist billing and not clinical purposes, difficulty in locating critical information among the vast amount of information that the record contains, and the inability to use the record for important clinical activities such as the comparison of data.

Clinical decision aids sought to help physicians synthesize complex considerations into rule-based guidance on patient care decisions. Berg⁵ describes how previous IT approaches to support clinician cognitive work have attempted to create rule-based aids for patient medical care decisions. Decision support systems need to be constantly monitored to determine whether their suggestions t a particular case. Also, the number of branching points may become so great to accommodate exceptions that the system is impossible to use and maintain.³ Difficulties with this approach demonstrate that decision making under clinical conditions is far more complex and less tractable than proponents of these early systems believed. The effect of clinical decision support systems on practitioner performance and patient health remain as inconsistent as they were 15 years ago.⁶ Relatively few clinical decision support systems (CDSS) are in use after their introduction over 25 years ago.⁷

Medical equipment such as infusion devices increasingly features complex control and display interfaces. Even highly experienced clinicians who have used infusion devices for years get “lost in menuspace” when they perform even the simplest tasks.⁸ Clusters of complex devices occur in acute care, particularly in the intensive care unit (ICU).

Information systems, electronic medical records, decision aids, and equipment all suffer from being what Weiner⁹ termed clumsy automation. Systems that are clumsy, or poorly designed for human use, do not aid but rather impede cognitive work by adding new communication and coordination tasks to an already burdensome workload. Practitioners must develop coping skills including “work-around” procedures in order to adapt to the software system shortcomings. The systems are hard to operate, which induces errors. The net effect is to erode clinical effectiveness and patient care. The failure of clinical IT flows in part from developers’ “disregard for the ways in which people organize their work coupled with a disdain for the ordinary resources on which they rely....”¹⁰ It also results in part from healthcare’s persistent underinvestment in technologies such as IT, as well as the failure to take advantage of engineering-based systems design, analysis, and management tools such as human factors research.¹¹

As the application of IT to healthcare, medical informatics includes components that range from artificial intelligence to decision support systems, electronic patient records, information retrieval, outcomes assessment, and telemedicine. Information and its exchange are crucial at the patient, unit, organization, and environment level. However, cost and political pressures force IT development to minimize the time and effort that is spent to produce a system. Complex IT systems exceed the ability of their creators to understand them and are installed and operated without benefit of testing.

Without testing or risk management, IT developers have no way to know what may go wrong or why.¹ Resulting IT system shortcomings create multiple problems for healthcare. Hospitals have been reported to resist the adoption of IT systems in light of cost-benefit mismatch, rapid obsolescence, and time the systems divert from caring for patients.¹² Systems that are intended to improve on healthcare performance and patient safety are now perceived to create new forms of unintended adverse outcomes.¹³ Heeks, Mundy and Salazar¹⁴ contend that “many even most health care information systems are failures.” The estimated costs of information systems alone for each large hospital are about $50 million, yet the overall benefits and costs of hospital information systems have rarely been assessed. When systems are evaluated, about three quarters are considered to have failed and provide no evidence that they improve the productivity of health professionals.¹⁵

Recent developments in information technology (IT) have made ever greater amounts of data related to patients available to clinicians. Data availability, however, does not equal data utility. In order to be useful, data must be easy to manage so that it supports clinical cognitive work. This simple statement belies the depth and complexity it involves. Misperceptions about IT have substantial consequences for clinical work. Infusion devices and computerized physician order entry (CPOE) systems provide an example. Husch, et.al.¹⁶ contend that dose-related error reduction relies on interfacing infusion devices with other systems dealing with the use of medications such as the electronic medical record, computerized physician order entry, bar code medication administration and pharmacy information system. Such systems may be beneficial, but they can also suffer from difficulties such as being unable to handle marginal conditions that are a regular part of patient care. For example, CPOE relies on a centralized computer system to track and manage the provision of medication. CPOE is intended to create a continuous connection from physician, to pharmacist, to nurse. The approach is intended to reduce causes of medication error by improving the reliability and accuracy of health care system performance. While IT can improve on some difficulties, it can also introduce others. Indeed, Koppel, et.al.¹³ report that clinicians at one major acute care facility perceive their CPOE system to have problems related to data entry, and lack confidence in this clinical IT system’s reliability. Recent studies implicate CPOE systems as a cause of adverse drug events¹⁷, pediatric mortality¹⁸, and erroneous test ordering.¹⁹ Clinicians have responded to the failure of one information source by creating their own, which points the way to successful IT development for sharp end use.

The Sign-Out Sheet

The medical record no longer serves a useful clinical role because it has evolved to primarily serve regulator, medico-legal and billing needs. Clinicians have instead resorted to developing their own informal solution, the sign out sheet, to replace the medical record as their daily patient care reference. Frank, Lawless, and Steinberg²² describe the resident sign-out sheet as a structured report that includes patient-specific information such as patient name, age, gender, room number, and attending physician, a problem list, medications, and allergies. The sheet began as an informal method that residents used to track patients under a clinician’s care, Ram and Block²³ encouraged the use of more structured patient summaries that drew on patient data from existing hospital information systems.

Residents who are assigned to a unit routinely collect essential information on the clinical condition of patients for whom they are responsible. On units that use the sheet, residents update, print out, and distribute copies each morning before the start of rounds. Figure One shows an example of a sign-out sheet from a surgical intensive care unit (SICU). The 8.5” x 11” sheets account for critical information in one or two pages. Pre-printed formats assist writing in essential information such as lab values as they become available. As a cognitive artifact²², the sheet serves as a ready reference for the SICU patient care work domain. It also incorporates the entire distributed cognition²³ in that domain by acounting for every individual who has had, now has, or may have knowledge regarding the patients in the unit. As it embodies the critical issues in this work setting, it also serves as a valuable “way in” to study issues that truly matter here.²⁴

As a hard copy artifact, the sheet can be used to capture critical new information as it becomes available. The sheet serves a variety of functions among a staff that is distributed across time, location, and specialty. For example, it is a means to develop and maintain a consensus among many different departments regarding the current states of patients, and what medical care is to be done in the future. It can also be used for technical work such as planning tests, documenting their results, and determining who may be well enough to transfer out of the unit to a patient ward.

Discussion

Resilience is a feature of some systems that allows them to respond to sudden, unanticipated demands for performance and then to return to their normal operating condition quickly and with a minimum decrement in their performance. The evolution of the sign-out sheet as the primary cognitive artifact in this high stakes care setting is evidence of resilience.²⁵

Systems that have been developed for purposes other than clinical care cannot reflect that domain’s complexity or change as the work demands and worker response require. The evolution of the sign-out sheet demonstrates how workers take action to ensure successful performance of those tasks in the face of obstacles such as the decreasing utility of the medical record. This resilience contrasts with the unchanging (brittle) properties of IT systems such as those described earlier in this paper. Rather than being imported from an external source, the sign-out sheet evolved from a need and information that exists within the ICU work setting. Residents create and maintain the sign-out sheet because it successfully fills a necessary role in their work domain.

ICU activities rely on the acquisition, portrayal and analysis of therapeutic and diagnostic information as an integral part of individual patient care. Equipment and systems that are intended for use by clinicians must necessarily reflect actual clinical practice to be well-suited for use at the sharp end. The efforts that are required to accomplish this are not simple, as this is the most complex and varied work setting that IT has tried to support.²⁶ Support for sharp end cognitive work requires attention to the subtleties and complexities of the real world that are unforgiving in their consequences. Uncertainty and contingency are inherent in the intensive care unit (ICU). The fragile nature of patients who are assigned to the unit can result in rapid changes to their condition with significant consequences. This requires moment-to-monent monitoring as well as well-informed decisions among a team of practitioners throughout a 24-hour work cycle. The daily work of the ICU clinician requires representations that serve as a map of the ever-changing territory of work that must be successfully navigated.²⁷ The sign-out sheet serves this purpose.

Conclusion

The obstacle to the use of IT in healthcare is not computing systems but rather the inability understand the complex operations of health care systems that IT systems are intended to support. The study of real world work requires methods that are designed to understand it. Human factors analysis and system design tools make that understanding possible.¹¹ This type of research, such as artifact analysis of a sign-out sheet, yields insight into what truly aids expertise. Study of cognitive work at the sharp end that combines human factors and clinical expertise will make it possible to develop resilient IT systems that improve, rather than impede, patient care and safety.

Fig. 1.

Portion of two-page Surgical ICU sign-out sheet