Cognitive performance-altering effects of electronic medical records: An application of the human factors paradigm for patient safety

Abstract

According to the human factors paradigm for patient safety, health care work systems and innovations such as electronic medical records do not have direct effects on patient safety. Instead, their effects are contingent on how the clinical work system, whether computerized or not, shapes health care providers' performance of cognitive work processes. An application of the human factors paradigm to interview data from two hospitals in the Midwest United States yielded numerous examples of the performance-altering effects of electronic medical records, electronic clinical documentation, and computerized provider order entry. Findings describe both improvements and decrements in the ease and quality of cognitive performance, both for interviewed clinicians and for their colleagues and patients. Changes in cognitive performance appear to have desirable and undesirable implications for patient safety as well as for quality of care and other important outcomes. Cognitive performance can also be traced to interactions between work system elements, including new technology, allowing for the discovery of problems with “fit” to be addressed through design interventions.

1. Introduction

Patient safety is a pressing global concern, with substantial economic and humanistic implications (Henriksen et al. 2008; World Health Organization 2009). Patient safety is often viewed as the freedom from medical error and accidental harm (e.g., Institute of Medicine 2000; Leape 2009), thus, patient safety goals are often formulated with respect to “the avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of care” (Vincent 2006, p.14). Often, there is recognition that “patient safety is a property that emerges from systems design” (Emanuel et al. 2008, p.7), meaning that the way to achieve patient safety is to change the system (Institute of Medicine 2000). The above error-based view of patient safety is depicted in Figure 1a. A different, though complementary, view is depicted in Figure 1b and can be described as follows (cf. Karsh, Holden et al. 2006):

Figure 1.

Panel (a) depicts how the error-based patient safety paradigm describes patient safety (“A”) as a product of the “system” (“B”) but does not specify the mechanism between the two, thus leaving a “black box” between A and C. In contrast, panel (b) depicts how the human factors (HF) paradigm (b) proposes a performance mechanism to mediate the relationship between work system and outcomes. Accordingly, while the error-based paradigm fails to offer a specific account of how electronic medical records (EMR) systems affect safety outcomes (panel c), the HF paradigm suggests that the effect of EMR is, first, dependent on its interactions with the wider work system, and second, occurs by way of altering the performance processes of health care providers (panel d).

Patient safety is the product of how and how well health care providers (HCPs) perform cognitive work processes. There is no direct way to reduce errors or harm. Instead, errors are reduced when the conditions of work (i.e., the work system) positively shape the way that HCPs perform cognitive work; harm is reduced when the conditions of work allow HCPs to perform well under challenging or disruptive conditions.

The primary goal should not be to reduce error or harm. The goal should be to design work systems that support and enhance work process performance. Error reduction and harm prevention will follow in turn. So will a host of other outcomes, including faster and higher quality care, improved quality of work life for HCPs, and more efficient use of organizational resources.

The above statement builds on a human factors (HF) paradigm for patient safety (Karsh, Holden et al. 2006) in accord with the definition of human factors, or ergonomics, as the discipline concerned with “design and engineering of human-machine systems for the purpose of enhancing human performance” (Dempsey et al. 2000, p.6, emphasis mine). To expand somewhat on Karsh et al's original proposal, I stress the following: the design of work systems must support not only routine cognitive performance (e.g., well illuminated medication rooms that improve visual perception) but also performance under problematic conditions (e.g., access to and discretion over supplies during emergency situations) when human performance is the last arbiter between safety and harm (cf. Hollnagel et al. 2006; Weick and Sutcliffe 2007).

The HF paradigm accepts error and harm reduction as distal but not directly achievable goals. This is why it is necessary to pursue a more proximal goal: that of enhancing cognitive performance.

1.1. Cognitive performance

A straightforward definition of health care provider performance is “the mental, physical, social, and behavioral activities carried out by HCPs toward some (usually patient related) goal” (Karsh, Holden et al. 2006, p.i60); note the similarity of this definition to the definition of “work” as “any physical or mental effort or activity directed toward the production or accomplishment of something” (Hollnagel 1997, p.1171). Performance is goal-driven (Vicente 1999) and in this paper, the primary goals of interest are patients' quality of care and safety. Here, performance is treated as the activities or processes of work, but performance measurement sometimes concerns the outcomes of those activities, instead, such as the product defect rate in manufacturing or the survival rate in a population receiving treatment in health care (see Eddy 1998). One type of performance process measurement in health care is whether and when some process believed to improve patient outcomes was carried out (e.g., administration of aspirin within 24 hours of admission and time to initial reperfusion therapy for heart attack patients) (Silver et al. 2004). For present purposes, the scope of performance is broader, encompassing both the execution of recommended activities within recommended time windows as well as the execution of a much larger set of non-prescribed activities needed to achieve patient care goals; additionally, the way in which performance is carried out is of interest (e.g., how does a physician decide to and then actually order aspirin upon admission).

It can be argued that all performance in health care is cognitive (Hollnagel and Woods 2005), from decision making to consultation to manual laparoscopic suturing, and that the distinction between mental, physical, social, and behavioral aspects of performance is useful but artificial because all four require cognitive processes (Karsh, Holden et al. 2006). Cognitive performance processes can be understood in many ways. A microcognitive lens depicts cognitive performance as consisting of a repeating series of information processing steps such as sensation, perception, central processing, and execution, and narrower constituent processes such as verbal working memory, switching attention between tasks, and the planning and execution of eye movements (Wickens and Flach 1988). Although it is not always evident how microcognitive processes are integrated during the performance of complex and prolonged work such as diagnosis or medication administration, factors that affect individual microcognitive processes are well studied (e.g., studies show that proactive interference is the chief cause of errors in verbal working memory [Postle 2006]). Further, the link between specific microcognitive processes and safety outcomes are straightforward, because psychologists have carefully devised speed and accuracy measures for each process.

In contrast, a macrocognitive lens describes cognition in terms of broad processes such as situation assessment, problem detection, planning, and decision making (Klein et al. 2003). Such a broad perspective makes it easier to see how cognition is not the product of isolated processes in an individual's mind but rather a product of joint activity, between multiple people (e.g., physicians must plan based on their own knowledge but also the knowledge of other physicians and specialists, nurses, and patients) and multiple artifacts (e.g., situation assessment relies on information such as patient vitals and medication history that is stored, arranged, and manipulated in electronic and paper records) often over extended periods of time. Because of the visible role of technology in macrocognition and because health care work contains processes distributed across time and across many agents, the present study adopts a macrocognitive level of analysis, although microcognitive processes are assumed to exist at a lower level of analysis.

Both microcognitive and macrocognitive processes can be seen as operating in the service of maintaining control of health care work, contributing to the setting of goals for system performance outcomes (e.g., acceptable levels of patient health), the monitoring of system performance with respect to those goals (e.g., sensing, perceiving, and generally assessing the patient situation, both performed therapies and how the patient is reacting), the comparison of performance to pre-established goals (e.g., processing collected information on the patient situation and detecting problems), and the regulation of performance (or the adjustment of goals) when goals mismatch (e.g., revising plans and decisions, adjusting therapy) (see Hollnagel and Woods 2005; Karsh, Holden et al. 2006). The design of the health care work system and changes to the system (e.g., new IT) affect how well performance can be controlled. Information systems (manual or computerized), for instance, critically determine clinicians' awareness of the patient situation and, therefore, determine how and when adjustments are made to regulate performance. The inability to make proper adjustments results in the performance of work that is inefficient, difficult, or deviates from pre-established goals. In health care, inefficient or difficult work may jeopardize worker health and unnecessarily deplete organizational resources; performance that deviates from goals means reduced quality of care and the possibility of accidental patient harm.

To sum up, the HF paradigm proposes that the performance of cognitive work mediates between work system design on the one hand, and patient, employee, and organizational outcomes, on the other. The implication is to shift away from a paradigm centered on errors (and by implication, away from counterproductive practices such as counting errors [Dekker 2007; Wears 2008]), and toward a paradigm that seeks to understand work performance and aims to design systems that support performance. The benefits of the HF paradigm over the traditional error-based paradigm can be demonstrated by considering the case of electronic medical records (EMR), a putative lever for patient safety.

1.2. EMR and the human factors paradigm

EMR is “an electronic record of health-related information on an individual that can be created, gathered, managed, and consulted by authorized clinicians and staff within one health care organization” (National Alliance for Health Information Technology 2008, p.6). Often, EMR is defined in terms of its functional components (Chaudhry et al. 2006; Jha et al. 2009; R. H. Miller and Sim 2004), which in addition to “viewing” existing data can include documentation/data entry, electronic order entry/order management, messaging/communication, decision support, and more (Institute of Medicine 2003; R. H. Miller and Sim 2004; Walker et al. 2008). Herein, EMR is conceived of broadly as a package of related functions, with data entry, order entry, and decision support treated as components of that package (Ash and Bates 2005).

Mounting political, financial, and consumer pressures and incentives (Blumenthal and Glaser 2007; Steinbrook 2009; The Leapfrog Group 2008) are moving organizations to implement EMR at increasing rates (Furukawa et al. 2008; Jha et al. 2008; Pedersen and Gumpper 2008). There is some evidence that EMR, especially when equipped with order entry and decision support capabilities, can improve patient safety (see reviews in Parente and McCullough 2009; Shekelle et al. 2006). At the same time, “it is clear as well that computer systems can introduce errors … and may even worsen [patient] outcomes in some instances” (Institute of Medicine 2007).

Those two themes—(1) EMR removes errors/improves patient safety and (2) EMR creates errors/worsens patient safety—are depicted in Figure 1c. It is obvious how they build on an error-based paradigm of patient safety (Figure 1a), in ideology, practice, and in language. Consider, for example, the preference for randomized control trials and pre-post error measures in evaluating EMR and other health IT (Reckmann et al. 2009; Shekelle et al. 2006); a description of a randomized trial using the notation in Figure 1 is “manipulate A, measure changes in C” with the implication that B is never measured (Kaplan 2001). Another example of how an “A→C” error-based paradigm influences thinking about EMR is the newly coined term “e-iatrogenesis,” which can be defined as “error/harm caused (in part) by IT” (Weiner et al. 2007; but see Palmieri et al. 2009 for a more sophisticated treatment of the concept). Although most researchers are well aware that normally EMR does not lead directly to patient safety outcomes (i.e., they are aware that the black box exists and might have some ideas of its content [see, e.g., Kaushal and Bates 2002; Walker and Carayon 2009]), the error-based paradigm does not offer a coherent account of the pathway, or mechanism, between EMR and patient safety. In contrast, applying the performance mechanism from the HF paradigm suggests that the way that EMR improves or worsens outcomes including patient safety depends on how an EMR-enabled work system alters cognitive performance for the better or worse (Figure 1d; see a similar HF-based view, although focusing mostly on altered information and workflow, in Saleem et al. 2009).

There are several ways in which EMR might alter cognitive performance. For example, “use of the EMR improved the legibility and accessibility of progress notes and increased the availability of electronic problem and allergy lists” (R. H. Miller and Sim 2004, p.118) and “use of electronic messaging among providers improved the availability, timeliness, and accuracy of messages and increased completeness of documentation, thus potentially reducing ‘dropped balls’ and safety problems” (p.119). Order entry functionality can improve communication between providers, improve legibility of notes and thus improve providers' ability to make sense of information, and provide a double check on providers' knowledge of drug-specific information (Bates et al. 1998; The Leapfrog Group 2008). EMR and computerized order entry can also worsen cognitive performance, for instance, degrading communication and collaboration, replacing old tasks with ones that are more difficult, and altering providers' interpretation of data (Campbell et al. 2006; Harrison et al. 2007; Horsky et al. 2005; Pirnejad et al. 2008).

Yet, no single study has explicitly identified and described the many different ways in which EMR alters cognitive performance. Nor is there a framework for classifying the performance-altering effects of EMR or any other health IT. Thus, the purpose of the present study was to identify and describe the cognitive performance-altering effects of EMR by analyzing data collected at two US hospitals that implemented EMR. In the process of analyzing those data, a classification framework of the cognitive performance-altering effects of EMR emerged that could be used for future research on EMR and other health IT.

2. Methods

Data were collected by interviewing physicians using EMR with the aim to take advantage of EMR users' insight into their own performance, a powerful yet underappreciated source of information. Interviews can capture information on user-IT interactions in actual clinical environments, whose turbulence, dynamism, and complexity are difficult to simulate in a laboratory during a usability test or laboratory-based task analysis. Interviews also provide insight into things that cannot be easily observed. For example, decision-making or information-gathering processes may be easier to describe than to observe (Ericsson and Simon 1980). As another example, phenomena that are distributed in time and space and that might take months of observations (and good luck) to capture and piece together by an observer can be more readily described by the very expert user who has been using the IT in context for months or years. Information gathered through interviews also has limitations: respondents may not be aware of or able to express important phenomena; the interviewer may bias responses; and interviews are poorly suited to some work metrics, such as patterns of task accomplishment or time-on-task. By using interview methods, cognitive performance was not measured directly, but rather inferred from physicians' accounts of the past and present, possibly introducing measurement error due to inaccurate or incomplete physician accounts or to errors in the interpretation of those accounts.

Twenty interviews, each lasting approximately 60 minutes, were conducted with physicians from two Midwest US hospitals. Information on the two hospitals, on respondents, and on the EMR systems is provided in Table 1. A purposive (non-random/non-probability) theoretical sampling strategy (Strauss and Corbin 1990) was used to recruit participants who could diversify the sample on these attributes: familiarity with health IT; comfort with technology, in general; and age. Participants were not compensated. Interviews were audio recorded, with permission of participants. Interviews at Hospital 1 took place June to October, 2007. Interviews at Hospital 2 took place January to February, 2009. The study was approved by IRBs at the University of Wisconsin-Madison and at both hospitals.

Table 1.

Study sample and electronic medical records system characteristics.

Sample information
Hospital characteristics	400+ bed tertiary care community hospitals in the Midwest US. Both hospitals are nearly identical in the populations they serve, staff training, and type of care provided. Major differences are: Hospital 1's physician staff represents multiple groups whereas Hospital 2's staff is primarily from one physician group; Hospital 2 is faith-based; Hospital 1 is a single, locally governed hospital whereas Hospital 2 is a multi-hospital system governed from outside the region.
Physician participants' specialties (number of participants)	General medicine (10), Anesthesiology (2), Obstetrics/Gynecology (2), Psychiatry (2), Cardiology (1), Orthopedics (1), Ophthalmology (1), Rheumatology (1)
Males / Females	15 / 5
Age
30 – 39	6
40 – 49	7
50 – 59	6
60+	1
Average years in job (range)	15 (4 – 32)
Average years computer experience (range)	20 (10 – 30)
Comfort with general technology (range)a	7.6 (3 – 10)
Comfort with hospital technology (range) a	7.7 (5 – 10)
Electronic medical records (EMR) systems at the time of the interviews
Hospital's stated aim of the system	Hospital 1: Initially to manage business (e.g., billing) with new electronic system and, at the time of the interviews, starting to focus more on improving patient care. Hospital 2: Improved quality of care, improved satisfaction, and improved efficiency.
Type of system	A commercial EMR system. The same, top-ranked national vendor of EMRs provided the system for both hospitalsb The vendor provided the software structure but the hospitals had to fill in much of the content, e.g., developing hospital-specific workflows and templates. Software modifications were made by the vendor, in response to hospital requests.
Type of clinical information managed by the system	Hospital 1: All nursing documentation except nursing notes and care plan; all transcribed medication and non-medication orders except nursing orders; laboratory results; medical imaging results and links to images; dictation transcriptions; reports from ancillary services/third parties; unofficial hand-off reports for some physicians. Hospital 2: All documentation by all providers (except in anesthesiology department); all orders; laboratory results; links to medical images and EKGs; dictation transcriptions; medical records tracking (e.g., deficiencies).
Clinical tasks supported by the system	Hospital 1: Nursing documentation, order transcription, and physician review of clinical data. Hospital 2: All documentation and order entry.
Extent and users of the system	Hospital 1: 4,450 total users, including 552 physician users (excluding residents); 550 residents, interns, and medical students; and 1,052 nurses from all nursing units. Hospital 2: 3,376 total users; 283 physicians, 1,100 nurses
Functionality (for physicians)	Hospital 1: Data repository/retrieval only.c Hospital 2: Data repository/retrieval, computerized clinical documentation, computerized order entry. Both hospitals also had computerized patient admission, transfers, and discharges; patient scheduling; billing; and pharmacy order and dispensing.
Approximate duration of system use at the time of interview	Hospital 1: 3 years. Hospital 2: 9 months (data repository and documentation); 7 months (order entry).

^aOn a 10-point scale (1 = extremely uncomfortable, 10 = extremely comfortable)

^bIn addition, the same vendor provided outpatient EMR systems for physicians' clinics and ten physicians provided information on their outpatient EMR experiences as well

^cTwo physicians at Hospital 1 were in the pilot unit for an order entry module and provided data on the first few weeks of using EMR with order entry functionality

The interviewer (RJH) was trained as a human factors engineer and psychologist and had broad experience and training in work measurement, including qualitative interviewing. The interviewer used a semi-structured interview instrument to elicit a wide range of physician perceptions about using EMR and particular EMR functions (e.g., order entry). Two broad questions elicited a majority of perceptions about performance: “What do you believe are the advantages of using EMR?” and “What do you believe are the disadvantages of using EMR?” The breadth of those questions was appropriate given the intent of the study to identify and describe all possible cognitive performance effects of EMR at the two study sites. Scripted and unscripted follow-up questions and probes elicited further information, as did scripted questions about normative pressures to use EMR, barriers and facilitators to using EMR, and the process of implementation. A summary of results pertaining to this wider set of questions can be found elsewhere (Holden 2010).

Audio recorded interviews were transcribed verbatim before undergoing a descriptive coding process (Miles and Huberman 1994; Strauss and Corbin 1990) intended to identify the themes and nested sub-themes pertaining to EMR-related cognitive performance changes. Themes were treated as behavioral beliefs (i.e., perceived links between using EMR and cognitive performance), as defined by the Theory of Planned Behavior (Ajzen 1991). Codes were assigned to passages of transcribed text and organized using QSR NVivo 8 (Cambridge, MA). The percentage of respondents mentioning a given theme or sub-theme was tabulated.

With respect to quality and validity indicators for qualitative research in health care (Mays and Pope 2000), this study was strongest in reflexivity (the interviewer/analyst carefully attended to his role in shaping the data) and attention to negative cases (deviant data and contradictory data are attended to, not concealed). Although procedures to protect confidentiality prevented the researcher from validating analyses with interview participants, manuscripts were shared with clinician contacts at the two study hospitals with a solicitation for feedback and corrections. In the results, verbatim quotes and the duplication of “in vivo” terms used by respondents provide readers with the ability to assess the correspondence between original data and the analyst's interpretation (Spencer et al. 2003). Space restrictions prevent a full description of data collection and analysis, an important requirement for assessing the quality of qualitative research (Spencer et al. 2003), but a full methodological supplement is available from the author upon request. The study was weakest on the following indicators. Only one theoretic perspective (the HF paradigm) was applied to the data. Triangulation was limited because only one method, interviews, was used and interviews were conducted within one group, attending physicians; even so, the physicians were recruited from two hospitals and differed in sex, age, specialty, and experience.

3. Results

Interviews yielded 20.6 hours of analyzed data and 441 transcribed, single-space pages. Participants identified numerous cognitive performance advantages and disadvantages of using EMR as well as particular EMR features (e.g., order entry, secure messaging). The average interview yielded data coded under about 20 different themes about performance, evenly divided between advantages and disadvantages. The themes are reported in Tables 2 through 6 and themes that came up often are described in more detail below.

Table 2.

Themes and subthemes describing how use of EMR improved the (a) ease and (b) time efficiency of personal cognitive performance

	% Mentioning theme
	Both hospitals	Hospital 1	Hospital 2
(a) IMPROVED EASE OF PERSONAL COGNITIVE PERFORMANCE
Improved ease of accessing information	65%	55%	78%
Easy access to much patient medical history
Easy access to medical references
Easy to find information
Can arrange information in the desired format; collation of information puts important information together; sorting features reduce clutter; using alphabetize patient list function makes it easier to find patients; easy to find information with electronic problem list
Can access information not previously available
Navigation easy because information is all in one place
Can access information from multiple settings, remotely	70%	64%	78%
Can access information from anywhere within the hospital
Can access outpatient information from anywhere (e.g., hospital)
Can check patient information when not at the hospital
Can access information immediately	60%	55%	67%
Can access information without searching for paper chart
Can access much information all at once	45%	36%	56%
Patients for the day can be put together
Accessed information is more legible	45%	27%	67%
Colleagues' electronic outpatient clinical notes are more legible
Can access information concurrently with others	15%	18%	11%
Improved ease of documentation	5%	0%	11%
(b) IMPROVED TIME EFFICIENCY OF COGNITIVE PERSONAL PERFORMANCE
Saves time accessing information	50%	36%	67%
Saves time obtaining test results
Saves time accessing others' notes because not waiting for dictation transcription
Saves time not searching for paper chart
Saves time not waiting for a nurse to look up information for physician
Saves time that would have been spent reading poor handwriting
Saves time entering orders and documentation	30%	18%	44%
Electronic order entry allows fast ordering of medications, especially remotely
Copy and paste feature allows notes to be written quickly
Medication orders are processed faster
Orders are clearer, reducing processing time
Other ways that time is saved	10%	18%	0%
EMR returns to prior state upon log-on, preventing having to start over
Knowing partners' schedules saves time rounding on patients

Table 6.

Themes and subthemes describing how personal use of electronic medical records (EMR) improved or worsened (a) colleagues' and (b) patients' cognitive performance

	% Mentioning theme
	Both hospitals	Hospital 1	Hospital 2
(a) IMPACT ON COLLEAGUES' COGNITIVE PERFORMANCE
Own notes accessed by colleagues (e.g., physicians, nurses) are more legible	30%	0%	67%
Medication orders are more legible for pharmacy	5%	0%	11%
Thought taken out of nurses' work	5%	0%	11%
(b) IMPACT ON PATIENTS' COGNITIVE PERFORMANCE
Outpatients can now access own medical records	10%	0%	22%
Patients have discharge reports that allow them to better care for themselves	5%	0%	11%
Electronic records prevent patients from losing prescription	5%	9%	0%

3.1. Personal performance advantages (Tables 2 and 3)

Table 3.

Themes and subthemes describing how use of EMR improved the quality of personal cognitive performance

	% Mentioning theme
	Both hospitals	Hospital 1	Hospital 2
IMPROVED QUALITY OF PERSONAL COGNITIVE PERFORMANCE
Improved situation awareness and ability to stay informed	50%	27%	78%
More aware of patient condition/needs due to real time viewing of images of X-rays, CT scans, and other results
Better informed/aware when arriving at hospital
Better informed/aware of inpatient visit when in clinic
Improved general awareness of day's workload
Better informed which rooms patients are in
Better informed of who are partners' patients
Less confusion over the meaning of notes due to increased legibility
Improved communication	25%	9%	44%
Improved communication with colleagues
Improved communication with patients
Improved ability to follow trends and status changes	20%	18%	22%
Improved ordering process	15%	0%	33%
More accurate orders
Order sets improve ordering
Improved decision making	15%	18%	11%
Improved ability to see colleagues' thought process	10%	9%	11%
Improved learning by tracking outcomes and errors	5%	0%	11%
Improved flexibility in timing of documentation	5%	0%	11%
System forces behaviors that improve performance	5%	0%	11%

Physicians described how EMR eased personal cognitive performance (Table 2a), particularly the process of accessing information:

“…in order to review the information, um, that's been compiled on a patient, it's a great and wonderful tool. The amount of information I can review and the ease with which I can retrieve it is incredible.”

Finding information became a simpler task due to better organization (“… labs are always at the labs. X-rays are with the X-rays”) as well as software features such as configurable displays and sorting/alphabetizing options. The great breadth of information in the EMR was available in one place and could be consolidated using a summary feature:

“I can quickly get a synopsis of the patient status: his blood pressure, his heart rate, his weight, his—all the things I have to look at—his drugs, his most recent lab studies, his older lab studies if I've forgotten and have to review them.”

The ability to access information “from anywhere at any time” was a commonly cited performance advantage. Remote access reduced physicians' travel both within and to-and-from the hospital and eliminated the need to place phone calls for information. Physicians could also quickly access information in new settings: from home, in a hotel, or in a procedure room (e.g., while waiting for or monitoring another patient).

Information could be accessed immediately, whereas previously “you used to have to wait for a paper chart or you couldn't find the paper chart or it was somewhere sitting on someone's desk that you couldn't locate.” Now, “with it in the computer, it's always there” and “the information is available to more than one person at one time.”

Accessed information was legible, making it easy for physicians to read and make sense of it without “trying to decipher cryptic abbreviations and, um, handwriting that, let's say, is less than optimal,” something that physicians felt greatly affected patient safety.

Although many physicians described how EMR made information retrieval easier, only one physician believed that electronic data entry improved the ease of documentation, particularly due to the ability to automatically import previously documented information, “past history and family history and aspects of social history: those things that don't change.” None of the physicians using computerized order entry mentioned that the system eased the ordering process.

EMR use was believed to improve the time efficiency of clinical work in several ways (Table 2b). This was primarily the case for accessing information. In addition to saving time not searching for paper charts, physicians did not have to wait as long as before to obtain (a) laboratory test results (because of “real-time” uploading of results) or (b) others' documentation (because “you're no longer limited by, let's say, the turnaround time of dictations to happen”).

Another advantage of EMR was that it supported higher quality personal cognitive performance (Table 3). In particular, using EMR improved physicians' situation awareness (Tenney and Pew 2006) and ability to stay informed: “I have a moderately high level of confidence that the data there is accurate and, um, dynamic, that it's, you know, it's going to be changing over time as the clinical situation changes. And, in other words, it's a reliable source of data.”

Because test results (e.g., cholesterol, X-rays) were available in the system soon after the test, physicians were able to be aware of patients' conditions. Physicians could also review inpatient records while seeing patients in the outpatient clinic or at home prior to arriving at the hospital, allowing a physician to “mentally prepare myself for what I'm going to do the next day.” EMR also improved awareness of the more general situation, such as a physician's workload for the day and which patients were assigned to whom. EMR supported better communication. Physicians appreciated that EMR allowed them to send colleagues e-mails or messages within the system containing an entire clinical note or discharge summary, “a pretty handy tool … especially when the doctor's out in the hinter region.” Bi-directional physician-patient communication was made possible by a convenient, secure messaging system. Additionally, face-to-face information transfer from physicians to patients improved:

“It's much easier to do that than it was paper … instead of flipping through the sheets and saying, well, look at this number on several different sheets, you can put it all on one line, you can graph it, you can do whatever format that patients would … find useful.”

At Hospital 2, where EMR was equipped with order entry functionality, the process of making orders was improved, with specific improvements in order accuracy and improved ordering due to order sets (a pre-specified collection of orders that EMR recommends for certain patient conditions; Bobb et al. 2007). Unexpectedly, only three physicians explicitly mentioned how EMR improved decision making, although logically, better access to information potentiates better decision making, as one physician explained:

“…as patients get more complex, the ability to have [an electronic ‘problem list’] in a concise way and quickly be able to access it and then find pertinent pieces of information that help you make integrated decisions is … high.”

3.2. Personal performance disadvantages (Tables 4 and 5)

Table 4.

Themes and subthemes describing how use of EMR worsened the (a) ease and (b) time efficiency of personal cognitive performance

	% Mentioning theme
	Both hospitals	Hospital 1	Hospital 2
(a) PERSONAL COGNITIVE PERFORMANCE MADE MORE DIFFICULT, MORE COMPLEX
Access to information difficult or impossible	70%	73%	67%
Difficult to find information
Information difficult to find due to information overload or variety of options; Information in printed-out version of records is difficult to find; Difficulty finding nurses' notes; Difficulty finding information in physical therapy notes; Difficulty finding scanned-in ophthalmology drawings; Medications put together for discharge are difficult to distinguish
Difficult to access information from other hospital systems
Cannot access old notes when writing new ones
Some information not available electronically (e.g., EKGs, pre-1999 data, progress notes)
No information if physician did not take initiative to make changes in record
New demands and extra steps	55%	45%	67%
Order entry requires many steps, extra steps, numerous clicks
Medication reconciliation is an extra step and creates redundancies
Creating problem list adds work
Entering documentation imposes memory burden
New steps to copy and paste arriving radiology reports
Remote log-in requires extra steps
Patient with multiple problems requires multiple, pre-formatted notes for each problem
System use is burdensome	30%	36%	22%
Remote log-on process is inconvenient
Difficult to make system do what you want it to
Reviewing past notes is cumbersome
Navigating system is difficult
When system times out on its own, current work is not always preserved
Blocks in performance	25%	9%	44%
Allergy warnings require response before order can be completed
If consult was not ordered in system, cannot do consult
Nurses must put in (verbal) order before physician signs off
System stops user when diagnosis does not match
Medicare-allowable orders
System blocks removal of items from record
Data entry difficult	20%	0%	44%
Difficult to obtain desired order
Difficult to enter data and attend to patient concurrently
Increased complexity of work	5%	0%	11%
(b) WORSENED TIME EFFICIENCY OF PERSONAL COGNITIVE PERFORMANCE
Loss of time	80%	64%	100%
Time consuming to do data entry
Time consuming to order medications; to document patient history following new compliance policy; to respond to warnings during medication order entry; to fill out post-operative order sheet
Time consuming to log in and out of system
Log-in from outside hospital takes time because of slowness, multiple steps and connection speed; Within-hospital log-in takes time because system is slow
Time consuming to access data from other hospitals
Time consuming to learn to use system
Time consuming to read notes that provide no additional information because they were copied from before
Takes away patient care time
Takes longer to accomplish tasks than with paper
Having to wait	15%	9%	22%
Waiting to load up computer or software
Waiting for slow software to process
Waiting for nurse to find a way to enter (verbal) order
Waiting for test results to arrive in system

Table 5.

Themes and subthemes describing how use of EMR worsened the quality of personal cognitive performance

	% Mentioning theme
	Both hospitals	Hospital 1	Hospital 2
WORSENED QUALITY OF PERSONAL COGNITIVE PERFORMANCE
Worsened situation awareness and ability to stay informed	45%	36%	56%
Unaware/uninformed because clinical notes less personal and less complete
Copy-and-pasted notes do not provide new information
Difficult to take in gestalt
Difficult to tell trends
Unaware/uninformed when problem list is missing from records
Unaware/uninformed when no indication that tests have been done
At discharge, cannot tell what medications were added or removed during hospitalization
Unclear what was colleagues' thought process (garbage in-garbage out)	35%	9%	67%
EMR use diverts attention from primary task	25%	36%	11%
Worsened communication	10%	0%	22%
Worsened communication between physicians
Reduced (face-to-face) communication with nurses and other hospital staff
Worse communication between physicians and patients' families
Overreliance on possibly incorrect information	10%	0%	22%
Reliance on notes with inaccuracies created and perpetuated by copying and pasting
Unable to flip through EMR looking for information as was possible with paper chart	10%	0%	22%
Reduced attention to detail	5%	0%	11%
Templates do not convey or integrate thought as well as prose	5%	0%	11%

All but one physician (from Hospital 1) also readily described cognitive performance problems. There were numerous examples of how cognitive performance was made more difficult or complex (Table 4a). For example, some information was difficult to find or access. Physicians at Hospital 2, for example, thought that both data retrieval and order entry presented “a serious overload of information in there, which makes it hard to find what you really need.” One physician commented on the overwhelming array of options for ordering medications, leading to “search time issues”: “if I type in the name of the med, I now get a whole list of various ways that med can be done and I have to search through that to find the one I want.” This could be contrasted with the paper-based ordering process, wherein a physician would write out the order as desired, rather than making multiple selections from structured lists.

More generally, information was only accessible to the extent that you “know where to get that data,” explained one physician. She linked this problem to the ability to provide good patient care:

“So an example would be for alcohol-withdrawal protocol, um, as part of that, they, the nurses have to document what's called the CIWA… And so it basically gives you a scale of how much alcohol withdrawal are they really going through. Is it mild, moderate, severe? Um, because when we're, I mean, we're not at the patient's bedside all the time, it's hard to know, well, what happened at 2:00 in the morning.

“… But, um, finding the CIWA score is near impossible, unless you know exactly how to get it. But the nurses are documenting it, and I'd say that a lot of my colleagues, they don't even bother to look, because they don't know how … so they kind of just look at the patient and say, oh, well, you look pretty good today, so I guess it isn't all that bad, sort of thing. Um, even though, maybe last night might have been very bad, uh, and you just don't know it.”

Access difficulties also stemmed from having to switch back and forth between systems carrying information distributed across different care sites, as different hospitals and clinics had separate systems despite using the same vendor. Other information could simply not be accessed, for example, because “you can't see [a prior progress note] at the same time that you're writing the new one,” or because information was entirely missing from the system.

Another difficulty in performance was created by the imposition of “additional work” and “an awful lot of extra steps.” Even though the new demands could be seen as new goal-relevant performance processes—e.g., medication reconciliation: the comparison of medications used by the patient versus medications entered in the system—they were often perceived as barriers to performance or superfluous demands that slowed down physicians' work. A deeper probing of physicians' comments suggested that there were two reasons for perceiving that using EMR created so-called extra work. The first was that the EMR interface was inefficient. A common problem of this sort was the need to click through numerous screens, options, or warnings when switching between systems, retrieving information, or entering orders (“it's only two clicks here and a five clicks here, but when you spend all your life clicking away”). This was especially the case at Hospital 2, where physicians found themselves making numerous selections from the new medication ordering screens, rather than writing out an order directly (see also Koppel et al. 2005): “So you end up having to click through all of these things that I think previously took place behind the scenes. [Before,] the doctor would write, ‘propranolol, 25 milligrams, p.o. b.i.d.,’ and that would, and that's all, and that would take you two seconds to do. Now [you have] to choose propranolol, to find the dose, to find all those times … takes a little bit longer.” Or as another physician summarized, “it's, you know, 10 boxes to click instead of [a] quick written-out thing” (for review of time utilization with EMR versus paper, see, e.g., Pizziferri et al. 2005, and for computerized order entry, see, e.g., Kuperman and Gibson 2003).

The second reason that new EMR related steps in the care delivery process seemed to impose extra work is related to the above comment about how some work “previously took place behind the scenes.” Several respondents were clearly frustrated by having to take on duties that they thought should be done by unit clerks, data entry specialists, pharmacists, and others. There were several emotional comments to that effect, for example “I'm highly resentful of the fact that somebody's using me as a very overqualified typist” and “[I] go through this painful sequence that a monkey can do, over and over again, just to look up like one number on the computer. I mean, it's really, it's very irritating.” In Hospital 2, issues of responsibility for new EMR-related steps came to a head when nurses originally refused to enter physicians' verbal orders into the system. After some time, it was decided that “if you phoned in orders … the nurses would still take them and put them into the computer,” which physicians explained was important when, for example, a physician was in the car. However, the struggle over who would do the extra work remained:

“So there's a surgeon who walks three steps down the hall and takes out his cell phone and calls the nurse. He can't give her the orders verbally, but if he calls them in, she has to put in all the orders for him. And he essentially does all of his orders that way.”

Similar workarounds included writing out orders on paper for nurses to enter or simply having nurses or technicians do the ordering. Clearly, some physicians resented “inheriting” the inefficiencies and extra steps of the EMR software and workflow.

Related, there were numerous comments about how using the system was a burden, difficult to direct (“very difficult to kind of make it do what I think a clinician does”), difficult to navigate, and especially difficult to enter data.

A final type of difficulty was the creation of blocks in performance, such as allergy warnings that physicians now had to address when ordering medications (“you'll have to click through a screen and it'll just keep, it'll give you all these reactions, so you'll see a lot of potential reactions that are not clinically relevant”) or a required sequence of events that had to be followed (“[if] the doc that's requesting [a consult] hasn't actually put in the request … you can't basically do this consult”).

In part due to the above difficulties, using EMR rendered cognitive performance less time efficient (Table 4b). Two common themes emerged on that topic: loss of time and having to wait. Time was lost in several ways:

• “I feel like I spend more time during the day actually logging in and out of [EMR System] than actually filling out an order set, or … seeing the information that I want.”

• “You get a phone message. The nurse takes a message, has to type it all out, sends it to me. I have to respond, type it out, give it back, and sometimes we go back and forth. And so that [takes a lot of time] … whereas before, we used to like verbally communicate, talk back and forth, write a quick note or something…”

• “I can document much faster writing.”

• “…[you are] wasting your time reading something that says the same thing that it said yesterday [because a note was copy-and-pasted].”

Extra waiting occurred when starting the hardware or software, when dealing with hardware or software that is “slow as molasses,” waiting for nurses to do something in the system (“and basically, I had to hang around until they figured it out”), or when certain laboratory results—namely, radiology and EKG reports—were not being entered in a timely way (“there's definitely a lag, and some of these EKG reports, I don't see after the patient goes”).

The quality of cognitive performance was also seen to suffer as a result of using EMR (Table 5). Just as EMR improved situation awareness in some ways, it also reduced situation awareness in several others. First, some physicians felt that “data transmission about the patient is less personal and less complete” and physicians referred to electronic clinical notes as “generic,” “pre-formed,” and “short,” pointing out that “the information may be there, but it is not very well presented … there's no room for commentary by the physicians filling out this computerized form.” Clinical notes could become unwieldy because of the ability to automatically repeat (or “blow in”) information: “because it's easy to blow all this information in, notes become really long and, and a lot of what's in there is not useful to the person reading the note.” Moreover, information could be missing; some of the fragmentation of data made it difficult to take in the gestalt or to see trends; and feedback could be inadequate, preventing physicians from having an accurate, updated sense of the situation.

Another performance problem related to gaining accurate information was that “it's very hard to tell what the physician was thinking now with the way most physicians document.” Some described this as a problem of “garbage in-garbage out,” or, in other words, “the note is only as good as the content in the note. And there's some people who put enough into the note so that I could follow their train of thought, and some do not.” Physicians voiced frustration with notes that were too short or that contained redundant copy-and-pasted information, with the result that “you have no idea what the other person's thinking.”

Physicians also talked about how EMR “diverts your focus away from your primary task,” usually direct patient care. Reasons for diverted attention included information overload (“you can get distracted because there's so much information there [referring to alerts]”), the attention demands of typing (“when I'm typing, I'm not thinking so much about the content than about the process of getting, you know, the characters into the keyboard correctly”), and concurrent demands (“you're focused on, let's say, you're trying to … put in an order, and you have to take care of your patient at the same time”).

Other problems included worsened communication and an overreliance on information that was possibly incorrect, sometimes due to errors created and perpetuated by copying and pasting information (see also Thielke et al. 2007):

“given that, it is much easier to import information into your documentation, um, there's less rigor … to verifying that information. So I've found that errors that end up in the computerized medical record tend to be perpetuated.”

3.3. Effect on others' performance (Table 6)

Not only did physicians relate ways in which EMR improved (or worsened) their own cognitive performance, they also saw effects on colleagues' (Table 6a) and patients' (Table 6b) performance. At Hospital 2, where physicians were entering (mostly) electronic clinical notes and orders, most respondents saw how the increased legibility helped fellow physicians, nurses, and pharmacists to make better sense of notes/orders and to avoid wasting time (e.g., trying to decipher orders or calling to clarify illegible handwriting).

Patients' cognitive performance improved in that outpatients could better care for themselves and comply with orders due to now being able to “access test results or communicate … by e-mail” and with electronic records “patients aren't going to lose the prescription.” One physician talked about the patient performance benefits of the EMR's discharge reports and after-visit summaries:

“…my practice is internal medicine, somewhat elderly practice, and, you know, their ability to take on a lot of information is challenged, you know. And so it's good for them to have something to refer to. And at the end of every visit, there's part of the technology that's called an after-visit summary, and that's really an important routing tool for next steps.”

On the other hand, some physicians saw how EMR functionality worsened (or at least failed to improve) patients' self-care performance. Namely, they felt that discharge notes (1) do not guarantee better-informed patients (“if they don't see it [a particular datum in the note], they don't get it”) and (2) are written for medical professionals, not for lay people:

“The reader needs to be the patient, so that the patient is clear about what they're supposed to be doing at home. But some of that wording is understood that, um, there's other medical personnel reading it … the verbiage needs to be different”

3.4. Comparisons of paper and electronic records

Often, physicians made explicit comparisons between the new EMR and “the paper world,” many of which were already mentioned above, for example, that writing with pen on paper was faster than typing. The bulk of comparisons can be summarized into four major themes, and from each theme, designers of paper and electronic records may extract important design challenges.

First, without exception, EMR was believed to outperform paper records in accessibility because of the omnipresence of electronic records, compared to paper charts that needed to be found and waited on (if someone else was using them). Additionally, as a single, physical document, the chart could not be taken too far away from its typical location for perusal or data entry. The trade-off was the extra time required to load up the EMR software.

Second, electronic and paper systems differed considerably in how they could be manipulated and the flexibility that they allowed. On the one hand, electronic features were thought to allow for better compiling, graphing, and searching of information than was possible with large charts that were sometimes arranged haphazardly and often contained extraneous documents. On the other hand, physicians reported that they lost the ability to manipulate the records like they used to: flipping quickly through the chart, putting two pages side by side, circling important values on paper, and removing extraneous information. The lack of manipulability implied an inflexible electronic system that was difficult to direct, as in this example: A physician reported having to go through the “stupid redundancy” of deleting and immediately re-entering orders for female patients who had just given birth, because the EMR did not know that it was dealing with the same patient who was simply being transferred from one unit (labor and delivery) to another (postpartum).

Third, computerization of data was perceived to reduce the fragmentation caused by paper records. In the paper system, information could be spread across multiple charts or similar information could be in different parts of the same chart because it was entered chronologically and by different people. A physician referring to the paper world of the past said that it was sometimes easier to order a new test rather than searching around to see if one had already been done. At the same time, EMR introduced different problems of fragmentation, because multiple clicks were sometimes necessary to access related information, some previously accessible information (e.g., therapy notes) ended up buried deep in the EMR's information structure, and the EMR imposed a new, less familiar, way of organizing information.

Fourth, paper and electronic records systems were intricately tied to contrasting work practices. Those different practices required physicians to use different skills, assigned to them different responsibilities, and may even have facilitated different local cultures. In the paper world, some information was manually or verbally transmitted: physicians copied information from clipboards to the full chart; nurses passed physicians' instructions to patients; unit clerks conveyed test orders to laboratories; nurses provided physicians with status updates; physicians waited on the dictation service to add information to the chart. With electronic records, information went into the EMR system and could be retrieved directly (or printed for patients). Some physicians perceived that in the paper world, information was not always managed reliably or in a timely manner (e.g., “historically what we tended to do was take an action and record it later at some point”). New forcing functions in the electronic record, for example, the requirement to verify and update the problem list before the EMR software could be closed, were thought to facilitate new information management practices: “It was easier in the past to just kind of let things go, let them fall through the cracks … the piece of paper that the problem list was on would be buried in the chart … Now it's there. It's in your face all the time.” Yet, the new practices of physicians entering information directly brought about by a central, electronic database, required new skills, such as navigating software and typing, with which physicians were less comfortable. As another example, physicians had to learn to specify orders differently; one physician contrasted writing down “KUB” on paper to order an abdominal X-ray and the electronic procedure of selecting from a list of over ten options. Of course, with the electronic system physicians were not doing new work, but rather doing the work that someone else had previously done. In the abdominal X-ray episode, the physician, unsure of which specific X-ray to order, called to the radiology department to ask “I want to order this, how do I do this?” It is probable that the radiology department, upon receiving an order for only “KUB” in the paper world, made the decisions that the physician now had to make in the electronic world. Physicians described how pharmacists would previously fill in details, such as the timing or route of medications, for which physicians were now responsible. In other words, the nature of the medical record determined the responsibilities for certain work, and as described above, physicians sometimes resented shifts in responsibilities. Some physicians perceived that changing procedures and shifts in responsibilities even altered the work culture, for example, at least initially eliminating the implicit assumption that nurses would enter verbal orders on behalf of physicians at Hospital 2.

The above differences between paper and electronic systems suggest at least four considerations for those designing and implementing medical records, paper-based or electronic: omnipresence/access, manipulability/flexibility, integration rather than fragmentation, and awareness of the procedures that are linked to different systems.

4. Discussion

If the impact of EMR on patient safety is mediated by changes in providers' performance (Karsh, Holden et al. 2006), then the many EMR-driven changes in cognitive performance described above provide numerous specific mechanisms by which EMR both improves and worsens patient safety. In addition to confirming several commonly hypothesized performance benefits of EMR (e.g., faster access to information), this study revealed many others. Each performance benefit of EMR described above is a hypothetical mechanism by which EMR improves safety. Likewise, each of the many ways in which performance was found to suffer is a candidate mechanism explaining why some errors persist, new errors surface, and patient safety is worsened when EMR and other health ITs are introduced (e.g., Aarts and Koppel 2009; Campbell et al. 2006; Han et al. 2005; Koppel et al. 2005; McDonald 2006; Weiner et al. 2007).

Present findings parallel those of qualitative research on EMR and CPOE that has identified possibly undesirable consequences of EMR. For example, work by Ash and colleagues has also revealed unintended consequences such as slow-downs in work and ease of use problems, extra burdens and difficulties particularly when entering data, a loss of situation awareness and the big picture, problems due to copying and pasting, system inflexibility creating blocks in performance, and communication problems (Ash et al. 2004; Ash et al. 2009; Campbell et al. 2006). Despite originally focusing on unanticipated, undesirable consequences, Ash et al later presented a broader framework inclusive of consequences that are anticipated and desirable (Ash et al. 2007). In the present study, broad sets of both desirable and undesirable performance consequences of EMR were identified. It cannot be argued from the present findings that EMR is deterministically good or deterministically bad for cognitive performance. Indeed, sometimes the very same EMR properties (e.g., the “copy-paste” function) that improved performance (e.g., saved time) also worsened it (e.g., made it difficult to know what colleagues were thinking because notes became less meaningful). Work by Harrison and colleagues similarly shows how EMR transforms the work system and work patterns, often in unintended ways (Harrison and Koppel 2010; Harrison et al. 2007). In addition, those authors make explicit that change is not unidirectional; instead, clinicians also adapt EMRs and other work system components and use EMR in different ways from what designers intended. One commonly reported adaptation is the use of workaround strategies to deal with blocks in performance (Ash et al. 2004; Koppel et al. 2008; Patterson et al. 2006; Vogelsmeier et al. 2008), a phenomenon found in this study as well (see Section 3.2). Adaptations are ubiquitous because they allow clinicians to cope with existing health care complexity and with new complexity introduced by the integration of IT, but adaptations may also increase risk (Harrison and Koppel 2010; Holden, Alper et al. 2008). Therefore, further research is needed to explore how both IT and clinicians' adaptations result in altered cognitive performance and the implications for patient safety.

This study did not intend to produce an exhaustive account of the cognitive performance effects of all EMR systems, much less of all health ITs. Different implementations of IT in different systems will likely yield different specific performance effects, although the general categories may be similar to the ones here. While physician respondents probably described performance effects that would not be easily discoverable using observations or other methods, respondents may not have mentioned other EMR effects because they occurred outside of their experience or awareness (or because respondents could not or chose not to articulate certain things), a limitation of interview methods. Thus, it is possible that some performance effects of EMR were not captured by this study. (Another issue with using interviews to capture performance changes is that while perceptions/beliefs are important causes of EMR acceptance, resistance, and use behavior [Holden and Karsh 2009, 2010], whether there is also a relationship between perceptions and actual patient safety outcomes is a question requiring further research.)

Even though generalizing from the sample to the population (in the traditional sense) was not an aim of the study, study findings can be used in several other ways. First, they demonstrate how an EMR system has many desirable and undesirable consequences for clinical work performance, rather than merely “improved safety” or “worsened safety.” Cognitive performance effects of EMR are tractable; they can be measured using interviews, among other methods. The present study was limited to interviews, only, but a combination of methods will be needed to triangulate on the phenomenon of cognitive performance changes resulting from EMR implementation (for an excellent example of triangulation in an EMR study, see Carayon et al. 2009). For future research, in addition to using interviews, performance measures might be obtained through surveys, laboratory simulation of IT use (Lin et al. 2001), and by using comprehensive observation methods such as work system analysis (Karsh and Alper 2005) and work domain analysis (Roth 2008). Miller et al (2009) provide an excellent example of the use of work domain analysis to study the effect of electronic and paper-based presentation of clinical information on physicians' and nurses' decision making performance.

A second use of study findings is as building blocks for a theoretic framework that can be applied to other cases of health IT; in that way, the findings may not be “generalizable” but are perhaps “transferrable” (Sjoberg et al. 1991). Indeed, the structure of cognitive performance effects that emerged (i.e., Tables 2 to 6) is an important contribution in that it represents a place to start for building a much-needed theory of how health IT affects cognitive performance. The categories in Tables 2 to 6 can be used as a framework for subsequent interviews, observations, surveys, laboratory/simulation studies, and case study analyses. Third, each cognitive performance effect reported here can be linked to a “system” cause and therefore addressed through work system design, something that is discussed next.

Why study cognitive performance effects when the ultimate goal is safety? First, Figure 1 shows that performance is the path by which safety goals are achieved (or failed) in a work organization. This is how EMR or any other health IT can impact safety—indirectly, by way of performance, not directly. Technology only causes errors or accidental harm directly in rare cases such as the Therac-25 accidents, in which a “bug” in the software and lacking hardware interlocks led a radiation therapy machine to deliver lethal doses of radiation (Casey 1993). But even software bugs can have their effect through cognitive performance: for example, the EMR software upgrade at the Veterans Administration (VA) included bugs that led providers to have inaccurate (or difficult to find) information for decision making and for carrying out procedures (Kuehn 2009). As long as providers use technology to assist them in carrying out cognitive work, rather than the technology doing the work itself, the safety effects of IT in health care will be predominantly routed through cognitive performance.

A second reason to study cognitive performance is that, as depicted in Figure 1 and described in the results, EMR-related performance changes affect not only safety, but also the speed and quality of care (Steele and DeBrow 2008), provider frustration (Sittig et al. 2005), and the efficient utilization of organizational resources (e.g., judicious ordering of tests). Studying performance means studying the causes of those important outcomes, not just of error and harm.

Third, by studying cognitive performance in a newly-computerized work place, one learns about the ways in which workers adapt and make strategic alterations to their environment, including to IT, in order to prevent new IT from disrupting performance and to keep performance effects from compromising patient safety (Harrison et al. 2007; Holden, Alper et al. 2008; Kaplan 1997). Such frequent, adaptive or resilient behavior cannot be captured by studying IT-related changes in error rates or harm and requires more attention to how actual performance of computer-assisted technical work occurs (Patterson et al. in press).

Fourth, the causes of cognitive performance cannot be traced to a single element of the work system, implied in the differences in how “A” is labeled in Figure 1c vs. Figure 1d. Performance is shaped by a very much multifaceted, dynamically interacting work system (Carayon et al. 2006; Karsh, Holden et al. 2006). This means that if one is motivated to understand cognitive performance in an EMR-enabled clinical setting, one will avoid focusing on the EMR in isolation, as it is inconceivable that a technology in isolation will shape complex work performance (see Harrison et al. 2007; Holden, Scanlon et al. 2008). To really understand the causes of performance, one will need to change the unit of analysis from technology/EMR in isolation to the human-technology joint cognitive system/team (Hollnagel and Woods 2005; Klein et al. 2004), the person-machine-context triple (Hoffman et al. 2002), the human-technology-organization triad (Oedewald and Reiman 2007), or to an even a broader interactive sociotechnical system, e.g., of human-task-technology-organization-environment (Carayon 2006; Karsh, Holden et al. 2006; Vincent 2006). Thus, focusing on performance means focusing on the multifaceted, interactive systems that shape performance, rather than on a single non-human element in the system (e.g., a technology like EMR, a tool like a checklist, an organizational element like teamwork training, etc.); this is the real strength of a “systems approach” to safety, not a shift from one isolated system component (human) to another (technology) (Holden 2009; Reason 2000).

One way to make the systems approach a functional and productive part of health IT research and practice is to specify the degree of fit between IT and the other work system elements (e.g., physical environment, task requirements, users' abilities), with the assumption that fit or misfit is the primary cause of performance advantages or disadvantages. IT-work system fit has received much attention in information science research (Goodhue and Thompson 1995; Ivari 1992; Markus and Robey 1983) and is a promising approach for health IT research and practice (Aarts et al. 2004; Ammenwerth et al. 2006; Holden and Karsh 2009, 2010; Holden and Karsh; Kaplan 2001; Karsh, Escoto et al. 2006; Karsh et al. 2009). Working backwards, one could take the performance consequences in Tables 2 to 6, and for each, infer or extract from respondents' comments the interacting system elements that played a part in the performance effect. One could then estimate the degree of fit between the elements and propose design interventions aimed at improving or maintaining fit. For that, HF methods, theory, and expertise (Saleem et al. 2009) will be very valuable, as managing work system interactions in order to achieve improvement goals is arguably the very purpose of human factors (Wilson 2000).

5. Conclusion

Health IT, when “intimately woven into the [work] system” (Perry et al. 2005), has a multitude of effects on how clinicians perform cognitive work, as was shown here with the case of physicians using EMR for patient care. By highlighting cognitive performance, and the performance effects of interventions such as EMR, the human factors paradigm provides a mechanistic account of how work systems and “systems change” improve or worsen patient safety and other key outcomes. An important goal for the patient safety discipline, in general, should be to describe the mechanism by which system design alters patient safety, be that mechanism cognitive performance, as in the present case, or something else, such as resilience/high reliability (Emanuel et al. 2008; Patterson et al. in press), “team play,” (Dekker and Woods 2002) or making systems “lean” (Ballé and Régnier 2007; Spear 2005).