Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Aug 1.
Published in final edited form as: Int J Med Inform. 2015 Apr 15;84(8):578–594. doi: 10.1016/j.ijmedinf.2015.04.002

Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit

Pascale Carayon a,b,*, Tosha B Wetterneck a,b,d, Bashar Alyousef a, Roger L Brown c,d, Randi S Cartmill a, Kerry McGuire a, Peter LT Hoonakker a, Jason Slagle e, Kara S Van Roy a, James M Walker g, Matthew B Weinger e,f, Anping Xie h, Kenneth E Wood i
PMCID: PMC4490834  NIHMSID: NIHMS681166  PMID: 25910685

Abstract

Objective

To assess the impact of EHR technology on the work and workflow of ICU physicians and compare time spent by ICU resident and attending physicians on various tasks before and after EHR implementation.

Design

EHR technology with electronic order management (CPOE, medication administration and pharmacy system) and physician documentation was implemented in October 2007.

Measurement

We collected a total of 289 h of observation pre- and post-EHR implementation. We directly observed the work of residents in three ICUs (adult medical/surgical ICU, pediatric ICU and neonatal ICU) and attending physicians in one ICU (adult medical/surgical ICU).

Results

EHR implementation had an impact on the time distribution of tasks as well as the temporal patterns of tasks. After EHR implementation, both residents and attending physicians spent more of their time on clinical review and documentation (40% and 55% increases, respectively). EHR implementation also affected the frequency of switching between tasks, which increased for residents (from 117 to 154 tasks per hour) but decreased for attendings (from 138 to 106 tasks per hour), and the temporal flow of tasks, in particular around what tasks occurred before and after clinical review and documentation. No changes in the time spent in conversational tasks or the physical care of the patient were observed.

Conclusions

The use of EHR technology has a major impact on ICU physician work (e.g., increased time spent on clinical review and documentation) and workflow (e.g., clinical review and documentation becoming the focal point of many other tasks). Further studies should evaluate the impact of changes in physician work on the quality of care provided.

Keywords: Critical care, Electronic health record, Time study, Physician work, Human factors engineering

1. Introduction

The impact of electronic health record (EHR) technology on physician work can influence their acceptance and use of the technology [1–6]. In particular, physicians have expressed concerns regarding changes in how they spend their time after EHR implementation. EHR technology may create more or new work for physicians [7] such as increased time spent on documentation. This occurs in a context where residents are reporting spending significant time on documentation. For instance, according to a 2006 survey of internal medicine residents [8], about 68% of residents reported spending in excess of 4 h daily on documentation. A recent survey confirms the extensive time spent by hospital physicians in documentation-related activities [9]. Limited research has assessed changes in physician work after EHR implementation [10–12]; the focus of that research has been limited to specific physician tasks (e.g., documentation) [13], and studies are plagued with methodological problems (e.g., small sample size; additional information on time and motion study and sampling can be found in a review of time studies in healthcare [14], or in books by Salvendy [15] and Barnes [10,16]). Additionally, sparse research has assessed the impact of EHR technology on intensive care unit (ICU) physician work [13,17,18]. Caring for complex critically ill patients requires communication and coordination of multiple healthcare team members, and changes in physician work routines could affect their ability to provide safe, high-quality care. Therefore, we postulated a need to better understand the impact of EHR technology on how ICU physicians spend their time on various tasks. Our study systematically examines the impact of EHR technology on the work of resident and attending physicians in the ICU.

1.1. Background

Studies have assessed the impact of various forms of EHR technology on specific physician tasks, such as documentation [13]. This research demonstrates the need to clearly define the EHR technology and its functionalities as these can have varying impact on clinician work, and the need for more comprehensive studies that record data on all tasks performed by physicians. For instance, Overhage and colleagues [19] examined a total of 81 tasks in 11 major categories performed by 34 physicians at 11 primary care internal medicine practices before and after the implementation of a homegrown computerized provider order entry (CPOE) system. Physicians spent slightly more time per patient overall and less time writing orders. In a study of 20 primary care physicians [20] using an adapted task list from that of Overhage and colleagues [19], physicians were found to spend more time on indirect patient care after EHR implementation, such as looking for patient-related information, and reading charts, data or email.

A few studies have examined EHR implementations in hospitals and their impact on physician work. The implementation of an electronic medication management system in an Australian hospital did not lead to any changes in time spent on direct care or medication-related tasks [12]. However, this study was unable to examine the impact of CPOE as it was already implemented at baseline. After the CPOE implementation at Massachusetts General Hospital, interns’ time spent writing orders went from 2.1% to 9% of their total work time and was associated with less time talking and reading [21]. Other studies have documented additional time spent by physicians on the computer after implementation of CPOE in a pediatric emergency department [22], CPOE and electronic nursing documentation in an emergency department [23], and electronic medical records (EMR) in a hospital [24]. A systematic review of research on the impact of EHR on physician work time confirmed that EHR technology tends to increase documentation time [10]. However, little research focuses on ICU physicians [25]. One study examined the time spent by physicians documenting during rounds in a pediatric ICU and an adult ICU, finding that documentation time decreased significantly [26]. This study included residents, attendings and sub-specialty fellows, but did not compare results by type of physician, and focused on documentation-related tasks as opposed to understanding the impact of the technology on all tasks performed by physicians. A second study collected data from five pediatric ICU attending physicians before and after the implementation of an electronic clinical information system, and also focused on documentation time [13]. Whereas time spent on handwritten and electronic documentation was similar, electronic documentation was more detailed, primarily because of the structured data entry process. Whereas the first two studies focus on documentation activities, the third study used a generic list of tasks and evaluated the impact of CPOE implementation among second-and third-year resident physicians rotating in one pediatric ICU with 67 h of observation conducted pre-CPOE and 87 h of observation conducted post-CPOE implementation [17]. Results showed more time spent by physicians interacting with patients, a higher frequency of task switching and more frequent waiting or idle time after CPOE implementation. Our research makes significant contributions to existing research by collecting data from several ICUs at all times and during both weekdays and week-end. In addition, comprehensive information (i.e., not just on documentation tasks) is needed to assess the potentially variable impact of EHR technology on the work of residents compared with attending physicians in the ICU.

While studies have documented the impact of EHR technology on physician work time in various care settings (i.e., primary care, ED, ICU), most studies have methodological weaknesses [10,14]. Many studies rely on subjective assessments of work time [8] or work sampling [13,21]. Continuous data collection on work activities such as time studies or task analysis are more precise methods for measuring time spent on various tasks [10,14,27]. Very few studies capture simultaneous work activities [28–30], which is particularly relevant for assessing the work of ICU physicians who often perform multiple work activities in rapid sequences. Another limitation of existing research is the lack of focus on the distribution of work time across various activities. To more fully understand the impact of EHR implementation on physician work, it is imperative to examine sequential and temporal patterns of work activities in addition to percentages of time spent on activities [17]. In a study of CPOE implementation in a pediatric ICU at the University of Michigan Health System [17], researchers not only examined changes in time utilization (e.g., time spent writing orders on the computer), but also workflow patterns (e.g., task switching and task transition). This promising research method enables us to understand the dynamic changes that occur with EHR implementations. For example, physicians may spend their time differently, but they may also work differently as varied patterns and sequences of activities emerge post-EHR implementation. When conducting this type of research in ICUs, a list of tasks that is representative of the unique work of caring for critical care patients [28,30], instead of generic tasks [17], is necessary; this is what we do in this study.

In this paper, we present data reflective of the impact of EHR technology on how resident and attending physicians spend their time in the ICU on multiple tasks. Our study addresses several conceptual and methodological issues of previous research, in particular assessment of all tasks performed by ICU physicians, and examination of the temporal flow of tasks.

2. Methods

2.1. Study design

This observational study used a prospective pre-post design around EHR implementation. Pre-implementation data were collected from August to November 2006, while post-implementation data were collected from January to March 2008. An EHR technology that included electronic order management (i.e., CPOE, electronic medication administration and a pharmacy system) and physician documentation was implemented hospital-wide in October 2007 (EpicCare Inpatient Clinical System, spring 2006 version). Electronic nursing documentation was implemented in June 2005, i.e., before the pre-EHR data were collected.

2.2. Setting

The study was conducted in three ICUs of a rural 400-bed tertiary-care medical center in the eastern United States. The ICUs were a 24-bed adult medical/surgical ICU, a 38-bed neonatal ICU and an 11-bed pediatric ICU. The medical/surgical ICU was a hybrid model primarily, with medical patients cared for by dedicated intensivists, and surgical patients who were the responsibility of surgeons with consultative input from the intensivists. The PICU and NICU had dedicated pediatric intensivists and neonatologists, respectively.

2.3. Sample

Data collection focused on work done by residents and attending physicians while in the ICU. Therefore, tasks completed outside of the ICU, such as participating in meetings or attending Grand Rounds, were not captured. Physicians did not participate on ‘road trips’ or transport of ICU patients for diagnostic or therapeutic management. Residents, specifically interns and second- and third-year residents who were on rotation in one of the ICUs, were observed in all three units. The number of residents rotating in the ICUs every month was as follows: 8 residents in the adult medical/surgical ICU, 4 residents in the PICU and 4 residents in the NICU. Attendings (intensivists) were observed in the adult medical/surgical ICU only because of resource constraints. In addition, we chose to observe attendings only in the adult medical/surgical ICU because the number of attendings working in the NICU and PICU was very small; at the time of the study, nine intensivists worked in the adult medical/surgical ICU.

2.4. Study procedures

Participants were recruited through informational meetings, printed materials, and communications from leadership. Before commencing an observation period, physicians were approached directly by the researcher, provided additional information about the project, and asked to participate. An effort was made to observe a variety of physicians, including residents in their first, second and third years. We used a purposeful sampling strategy to collect data at varied times of day and night, both on weekdays and weekends, and on all days of the week.

Participation in the study was voluntary. This research received institutional review board approval with a waiver of written informed consent at both the research university and participating hospital. No identifying data were collected for the participating physicians except their resident or attending status. Participating physicians were given information sheets that explained the purpose of the study and described the risks and benefits associated with participation. When patient family members were present during the observation or if the patient was awake and alert, the physician was asked to give the patient/family a brief explanation of the study. The patient or family could refuse to have their physician observed. Observation periods were suspended when physicians were behind closed curtains with their patients or taking personal time (e.g., eating, restroom breaks).

2.5. Data collection

Two trained human factors engineers “shadowed” each physician participant over a continuous period of up to 3 h. The researchers followed a participant at a distance that allowed them to observe the physician’s activities without interfering with natural movement, patient care, and workflow. Conversations with participants were minimized.

2.6. Data collection instrument

Data were collected using a computerized data collection tool developed by Weinger and colleagues [31,32] and adapted for observing ICU physician work [33]. The observers recorded the task or tasks being performed in real time, and these tasks were automatically time-stamped and logged into a data file [30–32]. Observers were able to select multiple tasks occurring at one time, allowing for data collection on simultaneous tasks. Data collection was facilitated by using a stylus on the touch screen of a tablet computer, which permitted observers to enter data while standing or walking. Through software design, task categories could be rapidly, accurately, and reliably selected.

A taxonomy of physician tasks was adapted from the list developed by Overhage and colleagues [19] and revised iteratively through ICU pilot observations and input from physicians and researchers for the ICU setting (see Table 1 for the list of tasks and task categories). The final list of 18 tasks was classified into four categories by a human factors engineer and a critical care physician (PC and KEW), in consultation with another physician (JMW). These categories are (1) direct patient care, (2) care coordination, (3) indirect patient care and (4) non-patient care.

Table 1.

List of ICU physician tasks.

Task categories Tasks Description of tasks
1. Direct patient care 1.1 Physical care of patient Patient assessment, procedures, assisting clinicians, transporting patient, hand hygiene, observing clinicians’ care activities
1.2 Use of monitors and devices Adjusting medical devices, observing monitors and equipment
1.3 Patient conversation Conversing with patient
1.4 Order management including medications Review and documentation of orders and medication administration records
1.5 Clinical review and documentation Review and documentation other than medications and orders, such as reviewing the patient chart, nursing documentation or notes.
2. Care coordination 2.1 Conversation with team physician Conversing with physicians assigned to the same unit, including attendings, fellows, residents, physician assistants, and medical students.
2.2 Conversation with non-team physician Conversing with physicians who are not assigned to the same unit, including consultants, referring physicians and other attendings, fellows, residents, physician assistants and medical students.
2.3 Conversation with nurse Conversing with nurses
2.4 Conversation with other ICU staff Conversing with unit pharmacist, respiratory therapist or unit desk clerk
2.5 Conversation attendance Standing, listening, not actively participating in conversation
2.6 Conversation with patient’s family Conversing with the patient’s family or other patient visitors
3. Indirect patient care 3.1 Conversation with other non-ICU staff Conversing with other ancillary clinical personnel, such as laboratory, radiology, physical therapy, nutritionists or paramedics.
3.2 Administrative review and documentation Review and documentation of other administrative documents such as schedules, educational and research materials, and white board
4. Non patient care 4.1 Conversation with non-clinical staff Conversing with non-clinical staff, such as housekeeping or information technology, or with unidentified people, including those on the other end of a phone conversation.
4.2 Non-clinical manual tasks Other non-clinical manual tasks, such as using office equipment (copier, fax or printer), gathering objects to be carried, and organizing or stapling papers.
4.3 In-between tasks Time spent idle, waiting, in transit, or searching for something.
4.4 Other tasks Tasks not included in any other category, including paging and research.
4.5 Patient room Study participant in patient room; observer unable to enter patient room
Open in a new tab

In the pilot phase, tandem observations with two observers were used as a training tool and were also a means for making further revisions to the software’s user interface, task taxonomy, and observation procedures. Standardized data collection procedures were detailed in a training manual, including where the observers should stand, how to avoid disrupting patient care, and proper use of the job task analysis software. Inter-observer reliability was assessed between pairs of observers, including a human factors engineer and a physician and the two trained human factors engineers on the observation team [33]. Observers began collecting data independently after inter-observer reliability reached the desired goal of at least 80% agreement. Inter-observer reliability was reassessed periodically and found to be stable over time.

2.7. Data analysis

Descriptive data analyses provided the amount of time spent by residents and attendings on the 18 tasks and 4 categories of tasks (Table 1). Data from each of the 77 resident observation periods and 24 attending observation periods were aggregated and the percentage of time spent on each task or category was calculated for each observation period. When multi-tasking occurred during an observation period, the durations of all tasks (including those performed simultaneously) were summed to create a total duration, which was used as the denominator when calculating the percentage of the observation period spent on each task or category. About 12% of the total observation time involved multi-tasking. Comparisons of the pre- and post-implementation means were performed using t-tests for independent samples (our data satisfied the assumptions of the t-test). Please note that means are calculated across observation periods for a particular group (i.e., resident versus attending) for pre-EHR and post-EHR implementation separately. Because of the large number of comparisons and the high likelihood of type I errors, eta squared (η2) was calculated to estimate the proportion of the variability associated with the implementation of EHR technology. A large η2 can be interpreted as indicating that the effect of EHR technology implementation was large [34]. The SPSS© statistical analysis software, version 19, was used to perform the descriptive analyses.

We defined an occurrence as the smallest level of data collection (i.e., the specific time during which a task is observed) and task switching as any change in task type at the occurrence level. To further evaluate the occurrence and frequency of task switching, the transition probability of pairs of tasks occurring in sequence was calculated for each observation period. Due to constraints of this analysis technique, multitasking data were excluded when performing the sequential analyses. When capturing multitasking data, the data collection software required the observer to indicate a “primary” task being performed and a “secondary” task that was simultaneously being performed. The sequential analysis excludes all secondary tasks. In most cases, the primary task was one that was initiated first. The transition probability from one task (predecessor) to another task (successor) was defined as the frequency of this transition divided by the total number of transitions originating from the predecessor [17]. Data were then analyzed by physician groups (resident and attending physicians) and data collection periods (pre- and post-EHR implementation). Mean transition probabilities of pairs of tasks and differences of mean transition probabilities between pre- and post-EHR implementation were calculated for both resident and attending physicians. The mean transition probabilities show the frequency of two tasks being paired together in sequence. For example, a mean transition probability of 0.24 for a specific sequence, such as conversation with team physician to clinical review and documentation, indicates that, in 24% of the times that conversation with a team physician occurs, it is followed by clinical review and documentation. Sequential task analysis was performed using the Generalized Sequential Querier (GSEQ) software, version 4.

3. Results

3.1. Descriptive statistics

A total of 77 observation periods of residents (217 h) and 24 observation periods of attendings (72 h) were completed in the ICUs. Most observation periods were completed on weekdays (68% for residents and 75% for attendings) and during the day (5am–4pm) shift (74% and 62%). About 40% of the observation periods incorporated daily morning or afternoon patient rounds. Resident physicians spent about 15% of their time performing multiple tasks simultaneously; for attending physicians the corresponding percentage was 11%. See Table 2 for additional information on the observation periods.

Table 2.

Description of observation periods.

Residents
Attendings
Pre
Post
Pre
Post
Observation periods Observation hours Observation periods Observation hours Observation periods Observation hours Observation periods Observation hours
Total 40 (100%) 109 (100%) 37 (100%) 108 (100%) 13 (100%) 36 (100%) 11 (100%) 36 (100%)
Weekdays 28 (70%) 76 (70%) 24 (65%) 70 (64%) 10 (77%) 30 (84%) 8 (73%) 26 (71%)
Day shift 32 (80%) 94 (87%) 25 (68%) 78 (72%) 8 (61%) 24 (67%) 7 (64%) 25 (71%)
Daily rounds 15 (38%) 45 (41%) 14 (38%) 50 (46%) 6 (46%) 18 (50%) 5 (45%) 19 (52%)
Open in a new tab

3.2. Distribution of time before and after EHR implementation

As shown in Fig. 1, the time distribution across major task categories changed for both resident and attending physicians when comparing the pre- and post-EHR implementation data.

Fig. 1.

Fig. 1

Open in a new tab

Comparison of time distribution across major task categories for resident and attending physicians pre- and post-EHR implementation.

Before EHR implementation, residents spent 31% of their time on direct patient care, 40% on care coordination, 13% on indirect patient care and 17% on non-patient care; see Table 3. After EHR implementation, the distribution of time changed as follows: 44% on direct patient care (p < 0.001 for pre-post comparison), 35% on care coordination, 6% on indirect patient care (p < 0.001) and 15% on non-patient care. More specifically, residents’ time on clinical review and documentation, considered direct patient care, significantly increased after EHR implementation (18–31%), whereas time spent on administrative review and documentation (12–4%) and in-between tasks (6–4%), considered indirect patient care, decreased. Analysis of the η2 statistics indicated that for residents EHR technology implementation had a relatively large effect on direct patient care, particularly clinical review and documentation; indirect patient care, particularly administrative review and documentation; and in-between tasks. Interestingly, the effect on non-patient care as a category was relatively small.

Table 3.

Time distribution of tasks performed by residents in ICUs before and after EHR implementation.

Pre (n = 40)a
Post (n = 37)a
t-Test (p-value) η2
Mean SD Mean SD
1 Direct patient care 30.8% 12.4% 44.3% 14.8% <0.001*** 0.20
 1.1 Physical care of patient 4.9% 4.7% 4.3% 5.8% 0.621 0.00
 1.2 Use of monitors and devices 1.0% 1.1% 1.2% 2.9% 0.598 0.00
 1.3 Patient conversation 0.6% 1.3% 0.3% 0.5% 0.135 0.03
 1.4 Order management including medications 6.6% 4.3% 8.0% 5.5% 0.219 0.02
 1.5 Clinical review and documentation 17.7% 13.0% 30.5% 15.4% <0.001*** 0.17
2 Care coordination 39.7% 13.3% 34.9% 14.2% 0.131 0.03
 2.1 Conversation with team physician 17.5% 8.6% 16.9% 9.8% 0.776 0.00
 2.2 Conversation with non-team physician 2.1% 4.1% 1.8% 2.9% 0.670 0.00
 2.3 Conversation with nurse 5.7% 5.0% 5.8% 4.4% 0.956 0.00
 2.4 Conversation with other ICU staff 0.8% 1.5% 0.6% 0.9% 0.565 0.00
 2.5 Conversation attendance 12.1% 12.4% 7.1% 9.8% 0.054 0.05
 2.6 Conversation with family 1.4% 2.0% 2.7% 3.8% 0.071 0.04
3 Indirect patient care 12.6% 7.3% 6.3% 6.9% <0.001*** 0.17
 3.1 Conversation with other non-ICU staff 0.6% 1.0% 1.9% 4.5% 0.079 0.04
 3.2 Administrative review and documentation 12.1% 7.2% 4.3% 3.9% <0.001*** 0.31
4 Non-patient care 16.9% 10.3% 14.5% 12.2% 0.358 0.01
 4.1 Non-clinical conversation 1.7% 2.4% 1.2% 1.1% 0.304 0.01
 4.2 Non-clinical manual tasks 2.9% 6.7% 1.1% 1.4% 0.112 0.03
 4.3 In-between tasks 6.4% 2.3% 4.2% 1.9% <0.001*** 0.21
 4.4 Other tasks 4.3% 5.5% 6.1% 10.4% 0.322 0.01
 4.5 Patient room 1.7% 5.1% 1.8% 3.5% 0.919 0.00
Total 100% 100% Wald statisticb
Number of task occurrences per hour 117.0 35.2 153.8 60.5 <0.001*** 0.13
Open in a new tab

Note: The values in this table are observation-specific proportions representing the average percentage of time (or average rate) for all resident observation periods in the pre- or post-EHR implementation period. This is not the same as total proportion or the sum of all task-specific durations across observation periods divided by the total observation time.

***

p < 0.001.

a

Number of observation periods.

b

A one degree chi-square statistic was used to test the hypothesis of difference in Poisson analysis between pre- and post-EHR task occurrences.

Similar to the findings for residents, an increase in time spent on clinical review and documentation was observed for attendings (14–27%) (see Table 4). Time spent by attendings on order management significantly increased after EHR implementation (0.37–3%), but did not significantly change for residents. Analysis of the η2 statistics indicates that for attendings, EHR technology implementation had a relatively large effect on direct patient care, particularly order management, clinical review and documentation and physical care of the patient; moderate effects on care coordination and indirect patient care; and a relatively small effect on non-patient care.

Table 4.

Time distribution of tasks performed by attendings in ICUs before and after EHR implementation.

Pre (n = 13)a
Post (n = 11)a
t-Test η2
Mean SD Mean SD
1 Direct patient care 22.1% 8.7% 33.7% 10.9% 0.008** 0.28
 1.1 Physical care of patient 4.1% 3.3% 2.0% 1.6% 0.051 0.15
 1.2 Use of monitors and devices 2.8% 4.5% 1.6% 1.9% 0.419 0.03
 1.3 Patient conversation 0.6% 0.8% 0.3% 0.5% 0.332 0.04
 1.4 Order management including medications 0.4% 0.9% 3.1% 3.0% 0.015* 0.30
 1.5 Clinical review and documentation 14.1% 9.0% 26.7% 12.0% 0.007** 0.28
2 Care coordination 52.3% 11.7% 43.2% 9.9% 0.054 0.16
 2.1 Conversation with team physician 24.6% 8.6% 26.3% 8.0% 0.625 0.01
 2.2 Conversation with non-team physician 7.3% 5.0% 7.7% 5.9% 0.845 0.00
 2.3 Conversation with nurse 8.2% 6.1% 4.8% 2.8% 0.102 0.12
 2.4 Conversation with other ICU staff 2.3% 2.7% 1.2% 1.1% 0.214 0.06
 2.5 Conversation attendance 7.4% 8.1% 2.3% 3.5% 0.066 0.14
 2.6 Conversation with family 2.6% 4.1% 0.9% 1.4% 0.211 0.07
3 Indirect patient care 9.5% 6.2% 5.5% 4.0% 0.077 0.14
 3.1 Conversation with other non-ICU staff 1.2% 1.4% 0.8% 1.3% 0.499 0.02
 3.2 Administrative review and documentation 8.3% 6.5% 4.7% 2.9% 0.088 0.12
4 Non-patient care 16.1% 6.9% 17.6% 6.2% 0.585 0.01
 4.1 Non-clinical conversation 1.9% 2.1% 3.8% 3.1% 0.102 0.12
 4.2 Non-clinical manual tasks 1.2% 0.8% 0.6% 0.8% 0.076 0.14
 4.3 In-between tasks 6.7% 2.1% 5.3% 2.0% 0.297 0.05
 4.4 Other tasks 4.4% 5.5% 4.0% 4.3% 0.818 0.00
 4.5 Patient room 2.3% 2.5% 4.0% 4.6% 0.309 0.05
Total 100% 100% Wald statisticb
Number of task occurrences per hour 137.8 26.7 105.9 24.7 0.025* 0.29
Open in a new tab

Note: The values in this table are observation-specific proportions representing the average percentage of time (or average rate) for all attending observation periods in the pre- or post-EHR implementation period. This is not the same as total proportion or the sum of all task-specific durations across observation periods divided by the total observation time.

*

p < 0.05.

**

p < 0.01.

a

Number of observation periods.

b

A one degree chi-square statistic was used to test the hypothesis of difference in Poisson analysis between pre- and post-EHR task occurrences.

3.3. Task switching before and after EHR implementation

Notably, the average number of activities that residents performed each hour significantly increased after EHR implementation, from 117 (SD = 35) activities per hour to 154 (SD = 60) activities per hour (p < 0.01) (see Table 3). This was equivalent to 1.95 activities per minute pre-EHR and 2.56 activities per minute post-EHR, or a 31% increase. An opposite trend was found for attending physicians; after EHR implementation the number of task occurrences per hour dropped significantly from 138 (SD = 27) to 106 (SD = 25) (p < 0.01) (see Table 4, Section 4.2). This was equivalent to 2.30 activities per minute pre-EHR and 1.76 activities per minute post-EHR, or a 23% decrease.

3.4. Sequential task analysis before and after EHR implementation

To understand the impact of EHR implementation on the work of ICU physicians, it is important not only to examine the percentages of time spent on the different activities, but also to examine the sequential and temporal patterns of work activities (workflow). The EHR implementation had a major impact on the task of clinical review and documentation for all physicians. Therefore, we focused the sequential task analysis on this specific task. Figs. 2 and 3 show the network of tasks performed by residents and attendings in relation to clinical review and documentation; we then compared the pattern of task sequences before and after EHR implementation.

Fig. 2.

Fig. 2

Open in a new tab

Network of resident tasks around clinical review and documentation pre- and post-EHR implementation. Notes: The number on each arrow going from task A to task B represents the probability that task A precedes task B. The thickness of the arrows varies according to the following four levels of probability: probability less than 0.19; probability more than 0.20 and less than 0.29; probability more than 0.30 and less than 0.39; probability more than 0.40. Transition probabilities below 0.1 are omitted from the diagram.

Fig. 3.

Fig. 3

Open in a new tab

Network of attending tasks around clinical review and documentation pre- and post-EHR implementation. Notes: The number on each arrow going from task A to task B represents the probability that task A precedes task B. The thickness of the arrows varies according to the following four levels of probability: probability less than 0.19; probability more than 0.20 and less than 0.29; probability more than 0.30 and less than 0.39; probability more than 0.40. Transition probabilities below 0.1 are omitted from the diagram.

In each network, the arrows between the clinical review and documentation task and other tasks indicate the relative probability of sequence between two tasks; the direction of the arrow indicates the specific temporal sequence. For example, before EHR implementation, the probability of the sequence conversation with team physician → clinical review and documentation for residents was 0.24, whereas the probability of the reversed sequence was 0.14. Note that the probability of transitions from a specific task sum to 1; thus, in Fig. 2 for residents in the pre-implementation period, the recursive arrow for clinical review and documentation indicates that in 47% of the times that this task occurred, it was followed by another occurrence of clinical review and documentation. Probabilities below 0.1 are omitted from the figures and can be found in the complete transition matrices (see Appendix).

Both before and after EHR implementation, the majority of arrows pointed toward clinical review and documentation: the majority of sequences began with other tasks and ended with clinical review and documentation. A qualitative comparison of the pre-implementation and post-implementation networks for residents showed higher transition probabilities for numerous sequences post-implementation. For example, the transition probability for the sequence order management → clinical review and documentation increased from 0.18 to 0.53. In general, the transition probabilities for the clinical review and documentation task were higher post-EHR implementation; this is partly due to the significant increase in the percentage of time spent on this task. Similar results were found for attendings (see Fig. 3).

4. Discussion

As a form of ‘member checking’, we presented the results of our analysis to groups of (1) medical residents, (2) critical care attendings and physician assistants, (3) nurse managers and medical directors of the ICUs, and (4) the executive team of the participating medical center. Their feedback on the results is incorporated in our discussion of the results.

EHR implementation had a major impact on how residents and attendings spent their time while in the ICU. Both groups of physicians spent more time reviewing and documenting clinical information in the EHR than with paper charts: from 31% to 44% of time for residents (40% increase) and from 22% to 34% of time for attendings (55% increase). Our results confirm that EHR technology has a major impact on physician work [10]; we demonstrated this effect in the ICU. The executive team confirmed receiving numerous reports from physicians indicating an increase in amount of time spent on documentation; medical residents also confirmed this finding during presentation of our study results.

According to the analysis of the η2 statistics, EHR technology implementation had a relatively large effect on direct patient care for both resident and attending physicians. Our data quantified the increased proportion of time spent on clinical review and documentation (one of the tasks in the direct patient care category) on the EHR, which was accompanied by less time on administrative review and documentation and on ‘in-between tasks’. Review and documentation of administrative documents include schedules and educational and research materials, and ‘in-between tasks’ include time spent idle, waiting, in transit or searching for something (see Table 1). Decreased proportion of time spent on administrative review and documentation and on in-between tasks may represent efficiency gains from the use of EHR technology. Electronic information may be more easily available or retrieved; therefore, reducing time needed for administrative review and document. Based on the design and ease of use of the EHR compared to the paper chart and the availability of computers, especially computers on wheels, time spent on ‘in-between tasks’ may also be reduced. EHRs easily allow more than one team member to simultaneously review and document clinical information at the same time on different computer terminals or review around the same terminal versus the paper chart. It is possible that the burden of administrative review and documentation may have been shifted to nurses or unit desk clerks; however, during presentations of our results to ICU physicians and nurses, they did not bring up this issue. Future research should go beyond one particular professional group, and examine the impact of health information technology on team work [35].

The EHR technology did not have the same impact on residents and attendings. Interestingly, we found that attendings spent more time on order management after EHR implementation compared to when there were paper charts but the residents experienced no change. Being a teaching hospital, resident physicians were the major order writers for patient care and there were not any systematic changes in this practice after EHR implementation. However, it is likely that the ability to review previous orders and medication administration records improved with EHR implementation and the increase in time spent on order management by attendings may reflect their oversight of these processes done electronically rather than asking about it in conversation with team members. In a presentation of our results to critical care attendings, they indicated spending more time on the computer and spending less time talking with various people coordinating patient care (see Table 4).

EHR implementation changes not only the time distribution of tasks, but also the temporal patterns of tasks; this was true for both resident and attending physicians but in different ways. Building on the methodology of Zheng and colleagues [17], we examined task switching and task sequences, and we presented new data visualizations to highlight the impact of the EHR technology on ICU physician workflow (see Figs. 2 and 3). After EHR implementation, residents tended to switch tasks more frequently, whereas the reverse was found for attendings. Residents may use the EHR to do work as it comes up. For example, as they have the computer in front of them, they may be able to write orders and review clinical data with more frequent back-and-forth between tasks; this explains the increased probability between order management and clinical review and documentation (see Fig. 2 and Section 3.4). In a presentation to the executive team of the medical center, this process was described as ‘high-speed multi-tasking’ that is likely to occur during rounds. The network analysis shows that many tasks, in particular care coordination tasks (e.g., conversation with team physician), were more likely to occur before clinical review and documentation after EHR implementation. This occurs in a highly interruptive environment where conversations with other physicians, nurses and other people occur frequently; these residents’ care coordination activities occurred 40% of the time pre-EHR and 35% of the time post-EHR. On the other hand, the decrease in task switching for attendings may be related to increased concentration of tasks performed by attendings. Before EHR implementation, attendings may have had more opportunities to be interrupted because of the location of the charts close to the nursing station. After EHR implementation, attendings may have been able to do more work all at once on the same computer without needing to search for charts and in a location away from other team members. We did not collect data on location of tasks; therefore, this explanation is speculative and would require further testing in future research.

Our analysis of task sequences (see Figs. 2 and 3) focused on clinical review and documentation as this is the task most influenced by the EHR technology (see Tables 3 and 4). Not only did ICU physicians spend more time on this task after EHR implementation, it was also the one task where they spent most of their time (30% for residents and 27% for attendings). After EHR implementation, this task also seems to have become the center or focal point around which many other tasks occurred. Numerous tasks were found to frequently precede clinical review and documentation. For instance, conversations between residents and team physicians, or between residents and non-team physicians were often followed by clinical review and documentation; after talking with other physicians, residents were likely to look for information, to place an order, or to document an activity in the EHR. The network analyses (see Figs. 2 and 3) provide useful data visualizations to better understand the impact of EHR technology on ICU physician workflow.

Comparisons between pre- and post-EHR implementation did not show any changes in time spent on conversational tasks, such as communication with nurses and families. Several studies have highlighted communication challenges associated with the use of EHR technology [36,37]; but our study does not show any changes in amount of communication, although the quality of the communication could not be assessed with the observation methodology. Other methods such as questionnaires or conversation analysis are necessary to evaluate the content and quality of communication among ICU clinicians. In another part of our larger study on EHR implementation in ICUs, we conducted a survey of ICU physicians and nurses and found no long-term negative impact of EHR technology on quality of communication such as perceived timeliness and adequacy [6].

4.1. Study limitations

Because data were collected in a single hospital, these findings may not generalize to physician work in other ICUs and hospitals and to other changes in work with other EHR technologies or the same technology given the differences in context. However, our results are important as they show that the technology impacts different groups of physicians (resident versus attending) in different ways and that the technology not only changes the time distribution of tasks, but also the temporal flow of tasks. One limitation of the study design is that many of the same attending physicians were observed in both the pre- and post-implementation periods, but the residents that were observed were far more heterogenous and different in the pre- and post-implementation periods. Because we did not record any identifying data about the participating physicians (see Section 2.4), we were not able to compare the physicians who participated in the pre- and post-EHR implementation periods.

Because data were collected in 2006–2008, it is possible that a similar analysis conducted in a hospital with updated EHR technology could potentially find different results. However, because of the increasing diffusion of EHR technology, it is becoming more difficult to compare physician work with a paper chart to physician work with an EHR. We cannot rule out the possible impact of observer bias. Because the observations were focused on one individual at a time, we are not able to comment on changes in teamwork per se. The lack of a control group in the study design does not allow us to take into account other temporal factors that may have impacted the study, such as changes in organizational culture. However, the extensive observation data collection (total of 289 h) allowed us to examine the impact of EHR technology on ICU physician work in a detailed manner.

4.2. Conclusion

EHR technology has profound impact on the distribution and flow of ICU physician work. We observed some evidence of ‘efficiency gains,’ such as less time spent on administrative review and documentation and ‘in-between’ tasks, for resident physicians but not for attendings. The largest impact of EHR technology was significant increase in time spent on clinical review and documentation by both resident and attending physicians. The sequential task analysis shows that the EHR technology, and in particular clinical review and documentation, becomes a focal point for many tasks.

Our study clearly shows the importance of looking at time distribution and temporal changes in work. Future research should assess the use of EHR technology by ICU physicians during specific time periods such as during ICU rounds. Also, observational studies could better identify the reasons for the increase in resident task switching and its relationship to interruptions or distractions in the environment. Lastly, further study is needed to elucidate whether the increase in physician clinical review and documentation adds to quality of patient care and if not, whether it is the best way to use physician time in the ICU. Our observation methodology could be used for future research of the work of ICU physicians.

Table A1.

Pre-EHR transition matrix (residents, n = 40).

Tasks To
1.1 1.2 1.3 1.4 1.5 2.1 2.2 2.3 2.4 2.5 2.6 3.1 3.2 4.1 4.2 4.3 4.4 4.5
From 1.1 0.0643 0.1342 0.0451 0.0168 0.0327 0.1383 0.0167 0.1071 0.0091 0.1439 0.0380 0.0034 0.0151 0.0051 0.0572 0.1530 0.0040 0.0159
1.2 0.2822 0.0497 0.0289 0.0161 0.0772 0.0733 0.0009 0.1125 0.0026 0.0768 0.0416 0.0019 0.0553 0.0011 0.0290 0.1463 0.0048 0.0000
1.3 0.4487 0.1455 0.0148 0.0000 0.0263 0.0347 0.0000 0.0732 0.0000 0.0234 0.0662 0.0263 0.0175 0.0000 0.0246 0.0687 0.0263 0.0038
1.4 0.0095 0.0056 0.0013 0.0762 0.1839 0.1888 0.0102 0.1337 0.0097 0.1100 0.0133 0.0093 0.0737 0.0096 0.0320 0.1252 0.0081 0.0000
1.5 0.0028 0.0081 0.0000 0.0996 0.2567 0.1392 0.0088 0.0542 0.0054 0.0718 0.0026 0.0050 0.2026 0.0082 0.0240 0.0987 0.0125 0.0000
2.1 0.0303 0.0138 0.0015 0.1135 0.2352 0.0216 0.0166 0.0591 0.0039 0.1342 0.0031 0.0049 0.1300 0.0093 0.0232 0.1746 0.0246 0.0004
2.2 0.0394 0.0046 0.0000 0.0534 0.1872 0.0656 0.0124 0.1311 0.0013 0.1758 0.0012 0.0013 0.0733 0.0061 0.0022 0.2241 0.0211 0.0000
2.3 0.0608 0.0426 0.0120 0.1287 0.1637 0.1027 0.0125 0.0171 0.0163 0.0868 0.0109 0.0125 0.0636 0.0088 0.0231 0.2267 0.0112 0.0000
2.4 0.0063 0.0110 0.0000 0.1519 0.1348 0.1336 0.0000 0.0348 0.0036 0.0053 0.0000 0.0048 0.0393 0.0000 0.0509 0.4070 0.0168 0.0000
2.5 0.0518 0.0159 0.0005 0.0646 0.1521 0.3348 0.0487 0.0592 0.0050 0.0035 0.0128 0.0054 0.0896 0.0036 0.0084 0.1275 0.0161 0.0007
2.6 0.1524 0.0782 0.0359 0.0385 0.0959 0.0213 0.0038 0.1047 0.0000 0.0998 0.0135 0.0000 0.0087 0.0077 0.0231 0.2705 0.0077 0.0000
3.1 0.1212 0.0208 0.0000 0.0536 0.1299 0.1847 0.0057 0.0343 0.0156 0.1031 0.0000 0.0157 0.0625 0.0000 0.0031 0.2244 0.0251 0.0000
3.2 0.0051 0.0074 0.0000 0.0498 0.2911 0.1590 0.0147 0.0523 0.0118 0.1064 0.0056 0.0062 0.0732 0.0250 0.0222 0.1415 0.0288 0.0000
4.1 0.0048 0.0030 0.0111 0.0484 0.1417 0.1217 0.0098 0.0757 0.0278 0.0230 0.0000 0.0000 0.1229 0.0075 0.0331 0.2984 0.0711 0.0000
4.2 0.0905 0.0067 0.0179 0.0786 0.1539 0.0887 0.0029 0.0597 0.0263 0.0420 0.0138 0.0092 0.0787 0.0289 0.0058 0.2905 0.0058 0.0000
4.3 0.0609 0.0240 0.0051 0.0654 0.1040 0.1701 0.0113 0.0945 0.0250 0.0769 0.0178 0.0046 0.0784 0.0266 0.0727 0.1351 0.0260 0.0015
4.4 0.0172 0.0030 0.0000 0.0565 0.1587 0.1029 0.0170 0.0844 0.0048 0.0558 0.0028 0.0031 0.1178 0.0512 0.0240 0.2672 0.0336 0.0000
4.5 0.3750 0.0000 0.1250 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.1250 0.0625 0.0000 0.0000 0.0000 0.0000 0.3125 0.0000 0.0000
Open in a new tab

Table A2.

Post-EHR transition matrix (residents, n = 37).

Tasks To
1.1 1.2 1.3 1.4 1.5 2.1 2.2 2.3 2.4 2.5 2.6 3.1 3.2 4.1 4.2 4.3 4.4 4.5
From 1.1 0.0984 0.1198 0.0143 0.0048 0.0393 0.1238 0.0058 0.1529 0.0087 0.0496 0.0293 0.0133 0.0130 0.0000 0.0455 0.2223 0.0036 0.0265
1.2 0.2229 0.0359 0.0148 0.0278 0.1158 0.0493 0.0028 0.1284 0.0045 0.0504 0.0116 0.0108 0.0212 0.0015 0.0299 0.2567 0.0102 0.0065
1.3 0.2644 0.1568 0.0000 0.0000 0.0000 0.0000 0.0000 0.0641 0.0000 0.0459 0.1552 0.0000 0.0260 0.0000 0.0179 0.2579 0.0000 0.0119
1.4 0.0058 0.0045 0.0000 0.0639 0.5264 0.1373 0.0055 0.0534 0.0038 0.0699 0.0000 0.0119 0.0405 0.0082 0.0095 0.0535 0.0060 0.0000
1.5 0.0073 0.0060 0.0000 0.1226 0.4724 0.1083 0.0110 0.0437 0.0051 0.0662 0.0045 0.0167 0.0564 0.0105 0.0078 0.0427 0.0191 0.0000
2.1 0.0345 0.0119 0.0007 0.1001 0.4033 0.0504 0.0135 0.0609 0.0084 0.1030 0.0025 0.0123 0.0419 0.0048 0.0178 0.1041 0.0304 0.0000
2.2 0.0296 0.0000 0.0000 0.0497 0.3685 0.1491 0.0017 0.0518 0.0000 0.1068 0.0017 0.0071 0.0341 0.0036 0.0000 0.1740 0.0223 0.0000
2.3 0.1075 0.0332 0.0037 0.0704 0.2620 0.1010 0.0055 0.0207 0.0270 0.0471 0.0221 0.0194 0.0404 0.0130 0.0163 0.1895 0.0204 0.0009
2.4 0.0263 0.0339 0.0000 0.0204 0.2172 0.0441 0.0000 0.0776 0.0075 0.0381 0.0000 0.0152 0.0849 0.0000 0.0874 0.3270 0.0205 0.0000
2.5 0.0705 0.0102 0.0010 0.0700 0.2678 0.2443 0.0149 0.0837 0.0260 0.0082 0.0049 0.0429 0.0283 0.0050 0.0158 0.0908 0.0160 0.0000
2.6 0.1928 0.0329 0.0731 0.0000 0.1223 0.0214 0.0083 0.1599 0.0000 0.0249 0.0000 0.0021 0.0240 0.0128 0.0077 0.3126 0.0079 0.0000
3.1 0.0179 0.0100 0.0000 0.0827 0.1543 0.1227 0.0076 0.0892 0.0467 0.0896 0.0080 0.0221 0.1344 0.0037 0.0210 0.1544 0.0368 0.0000
3.2 0.0063 0.0055 0.0029 0.0566 0.3767 0.1805 0.0300 0.0673 0.0098 0.0331 0.0019 0.0221 0.0170 0.0093 0.0304 0.0987 0.0476 0.0046
4.1 0.0000 0.0081 0.0000 0.1045 0.4383 0.1037 0.0244 0.0403 0.0081 0.0267 0.0065 0.0040 0.0259 0.0113 0.0570 0.1193 0.0228 0.0000
4.2 0.1372 0.0275 0.0021 0.0156 0.1768 0.0540 0.0072 0.0571 0.0094 0.0522 0.0161 0.0284 0.0411 0.0218 0.0113 0.2895 0.0472 0.0056
4.3 0.0841 0.0381 0.0040 0.0247 0.1562 0.1576 0.0159 0.1345 0.0227 0.0518 0.0369 0.0135 0.0370 0.0268 0.0740 0.0751 0.0379 0.0098
4.4 0.0101 0.0203 0.0000 0.0758 0.1753 0.1202 0.0242 0.0660 0.0492 0.0414 0.0101 0.0114 0.0927 0.0274 0.0379 0.1993 0.0387 0.0000
4.5 0.2357 0.0262 0.0333 0.0222 0.0000 0.0667 0.0000 0.0857 0.0000 0.0667 0.0000 0.0000 0.0000 0.0000 0.0190 0.4222 0.0222 0.0000
Open in a new tab

Table A3.

Pre-post differences of transition probabilities (residents).

Tasks To
1.1 1.2 1.3 1.4 1.5 2.1 2.2 2.3 2.4 2.5 2.6 3.1 3.2 4.1 4.2 4.3 4.4 4.5
From 1.1 0.0341 −0.0144 −0.0308 −0.0120 0.0066 −0.0145 −0.0109 0.0458 −0.0004 −0.0943 −0.0087 0.0099 −0.0021 −0.0051 −0.0117 0.0693 −0.0004 0.0106
1.2 −0.0593 −0.0138 −0.0141 0.0117 0.0386 −0.0240 0.0019 0.0159 0.0019 −0.0264 −0.0300 0.0089 −0.0341 0.0004 0.0009 0.1104 0.0054 0.0065
1.3 −0.1843 0.0113 −0.0148 0.0000 −0.0263 −0.0347 0.0000 −0.0091 0.0000 0.0225 0.0890 −0.0263 0.0085 0.0000 −0.0067 0.1892 −0.0263 0.0081
1.4 −0.0037 −0.0011 −0.0013 −0.0123 0.3425 −0.0515 −0.0047 −0.0803 −0.0059 −0.0401 −0.0133 0.0026 −0.0332 −0.0014 −0.0225 −0.0717 −0.0021 0.0000
1.5 0.0045 −0.0021 0.0000 0.0230 0.2157 −0.0309 0.0022 −0.0105 −0.0003 −0.0056 0.0019 0.0117 −0.1462 0.0023 −0.0162 −0.0560 0.0066 0.0000
2.1 0.0042 −0.0019 −0.0008 −0.0134 0.1681 0.0288 −0.0031 0.0018 0.0045 −0.0312 −0.0006 0.0074 −0.0881 −0.0045 −0.0054 −0.0705 0.0058 −0.0004
2.2 −0.0098 −0.0046 0.0000 −0.0037 0.1813 0.0835 −0.0107 −0.0794 −0.0013 −0.0690 0.0005 0.0058 −0.0392 −0.0025 −0.0022 −0.0501 0.0012 0.0000
2.3 0.0467 −0.0094 −0.0083 −0.0583 0.0983 −0.0017 −0.0070 0.0036 0.0107 −0.0397 0.0112 0.0069 −0.0232 0.0042 −0.0068 −0.0372 0.0092 0.0009
2.4 0.0200 0.0229 0.0000 −0.1315 0.0824 −0.0895 0.0000 0.0428 0.0039 0.0328 0.0000 0.0104 0.0456 0.0000 0.0365 −0.0800 0.0037 0.0000
2.5 0.0187 −0.0057 0.0005 0.0054 0.1157 −0.0905 −0.0338 0.0245 0.0210 0.0047 −0.0079 0.0375 −0.0613 0.0014 0.0074 −0.0367 −0.0001 −0.0007
2.6 0.0404 −0.0453 0.0372 −0.0385 0.0264 0.0001 0.0045 0.0552 0.0000 −0.0749 −0.0135 0.0021 0.0153 0.0051 −0.0154 0.0421 0.0002 0.0000
3.1 −0.1033 −0.0108 0.0000 0.0291 0.0244 −0.0620 0.0019 0.0549 0.0311 −0.0135 0.0080 0.0064 0.0719 0.0037 0.0179 −0.0700 0.0117 0.0000
3.2 0.0012 −0.0019 0.0029 0.0068 0.0856 0.0215 0.0153 0.0150 −0.0020 −0.0733 −0.0037 0.0159 −0.0562 −0.0157 0.0082 −0.0428 0.0188 0.0046
4.1 −0.0048 0.0051 −0.0111 0.0561 0.2966 −0.0180 0.0146 −0.0354 −0.0197 0.0037 0.0065 0.0040 −0.0970 0.0038 0.0239 −0.1791 −0.0483 0.0000
4.2 0.0467 0.0208 −0.0158 −0.0630 0.0229 −0.0347 0.0043 −0.0026 −0.0169 0.0102 0.0023 0.0192 −0.0376 −0.0071 0.0055 −0.0010 0.0414 0.0056
4.3 0.0232 0.0141 −0.0011 −0.0407 0.0522 −0.0125 0.0046 0.0400 −0.0023 −0.0251 0.0191 0.0089 −0.0414 0.0002 0.0013 −0.0600 0.0119 0.0083
4.4 −0.0071 0.0173 0.0000 0.0193 0.0166 0.0173 0.0072 −0.0184 0.0444 −0.0144 0.0073 0.0083 −0.0251 −0.0238 0.0139 −0.0679 0.0051 0.0000
4.5 −0.1393 0.0262 −0.0917 0.0222 0.0000 0.0667 0.0000 0.0857 0.0000 −0.0583 −0.0625 0.0000 0.0000 0.0000 0.0190 0.1097 0.0222 0.0000
Open in a new tab

Inline graphic ≥ 0.1

Table A4.

Pre-EHR transition matrix (attendings, n = 13).

Tasks To
1.1 1.2 1.3 1.4 1.5 2.1 2.2 2.3 2.4 2.5 2.6 3.1 3.2 4.1 4.2 4.3 4.4 4.5
From 1.1 0.0569 0.1097 0.0644 0.0017 0.0598 0.1627 0.0263 0.1936 0.0282 0.0437 0.0168 0.0104 0.0000 0.0000 0.0267 0.1687 0.0041 0.0264
1.2 0.1422 0.0625 0.0125 0.0000 0.1022 0.1646 0.0444 0.2414 0.0096 0.0336 0.0133 0.0061 0.0119 0.0000 0.0091 0.1330 0.0009 0.0129
1.3 0.2750 0.1114 0.0000 0.0000 0.0000 0.1212 0.0000 0.2697 0.0000 0.0909 0.0606 0.0000 0.0000 0.0000 0.0114 0.0598 0.0000 0.0000
1.4 0.0500 0.0000 0.0000 0.0000 0.1300 0.1000 0.0000 0.2500 0.1000 0.1000 0.0000 0.1000 0.0200 0.0000 0.0000 0.1500 0.0000 0.0000
1.5 0.0083 0.0259 0.0000 0.0168 0.1600 0.2617 0.0444 0.0892 0.0379 0.0734 0.0137 0.0460 0.0718 0.0281 0.0138 0.1022 0.0069 0.0000
2.1 0.0811 0.0220 0.0006 0.0015 0.1939 0.0308 0.0672 0.1243 0.0540 0.1117 0.0097 0.0104 0.0525 0.0060 0.0295 0.1826 0.0164 0.0056
2.2 0.0454 0.0324 0.0011 0.0000 0.0599 0.2403 0.0066 0.0879 0.0274 0.1720 0.0219 0.0068 0.0587 0.0185 0.0070 0.2038 0.0105 0.0000
2.3 0.1267 0.0990 0.0251 0.0058 0.1288 0.1831 0.0497 0.0094 0.0264 0.0475 0.0109 0.0078 0.0354 0.0037 0.0202 0.1927 0.0200 0.0079
2.4 0.0368 0.0288 0.0010 0.0139 0.1149 0.2823 0.0354 0.1242 0.0050 0.0328 0.0033 0.0060 0.0365 0.0139 0.0113 0.2510 0.0030 0.0000
2.5 0.0388 0.0257 0.0049 0.0000 0.1256 0.3467 0.1975 0.1102 0.0268 0.0000 0.0017 0.0065 0.0195 0.0000 0.0067 0.0869 0.0025 0.0000
2.6 0.0829 0.0784 0.0000 0.0000 0.1338 0.0922 0.0604 0.0347 0.0000 0.0182 0.0083 0.0000 0.0227 0.0000 0.0227 0.4456 0.0000 0.0000
3.1 0.0573 0.0048 0.0000 0.0000 0.1886 0.0891 0.0341 0.1015 0.0000 0.1187 0.0000 0.0000 0.0664 0.0000 0.0161 0.3073 0.0161 0.0000
3.2 0.0000 0.0000 0.0154 0.0112 0.1791 0.2277 0.0665 0.1047 0.0640 0.0268 0.0045 0.0178 0.0580 0.0321 0.0383 0.1171 0.0367 0.0000
4.1 0.0083 0.0165 0.0000 0.0000 0.2150 0.1428 0.0443 0.1029 0.0083 0.0000 0.0000 0.0244 0.0152 0.0000 0.0227 0.3478 0.0519 0.0000
4.2 0.1723 0.0045 0.0000 0.0051 0.0880 0.1674 0.0000 0.1022 0.0118 0.0312 0.0033 0.0256 0.0639 0.0257 0.0051 0.2675 0.0070 0.0192
4.3 0.0705 0.0444 0.0062 0.0017 0.0523 0.2215 0.0629 0.1081 0.0469 0.0513 0.0128 0.0196 0.0285 0.0409 0.0732 0.1215 0.0233 0.0145
4.4 0.0048 0.0206 0.0000 0.0000 0.1516 0.1963 0.0371 0.0587 0.0415 0.0275 0.0000 0.0000 0.0206 0.0165 0.0154 0.3765 0.0330 0.0000
4.5 0.2343 0.0000 0.0000 0.0000 0.0000 0.1741 0.0185 0.1528 0.0000 0.0000 0.0185 0.0556 0.0222 0.0000 0.0000 0.3241 0.0000 0.0000
Open in a new tab

Table A5.

Post-EHR transition matrix (attendings, n = 11).

Tasks To
1.1 1.2 1.3 1.4 1.5 2.1 2.2 2.3 2.4 2.5 2.6 3.1 3.2 4.1 4.2 4.3 4.4 4.5
From 1.1 0.0587 0.1278 0.0257 0.0019 0.0250 0.1542 0.0072 0.1678 0.0000 0.0329 0.0056 0.0589 0.0000 0.0000 0.0151 0.3115 0.0000 0.0078
1.2 0.1713 0.1192 0.0100 0.0000 0.1488 0.1203 0.0151 0.1370 0.0050 0.0041 0.0100 0.0172 0.0019 0.0026 0.0083 0.2114 0.0000 0.0182
1.3 0.4619 0.1200 0.0000 0.0000 0.0286 0.0267 0.0286 0.0753 0.0000 0.0333 0.0286 0.0000 0.0000 0.0000 0.0705 0.1267 0.0000 0.0000
1.4 0.0053 0.0000 0.0000 0.1484 0.3206 0.1832 0.0105 0.0358 0.0105 0.0552 0.0000 0.0094 0.0174 0.0359 0.0000 0.1572 0.0106 0.0000
1.5 0.0008 0.0208 0.0010 0.0606 0.2299 0.2897 0.0431 0.0775 0.0264 0.0480 0.0025 0.0095 0.0654 0.0226 0.0079 0.0620 0.0263 0.0062
2.1 0.0194 0.0182 0.0026 0.0379 0.3245 0.0545 0.0677 0.1006 0.0333 0.0443 0.0017 0.0208 0.0473 0.0050 0.0072 0.1783 0.0242 0.0126
2.2 0.0242 0.0202 0.0014 0.0029 0.0980 0.2074 0.0347 0.0562 0.0212 0.1701 0.0000 0.0128 0.0122 0.0000 0.0099 0.2386 0.0673 0.0227
2.3 0.0439 0.0710 0.0045 0.0057 0.2186 0.2213 0.0511 0.0104 0.0209 0.0442 0.0053 0.0159 0.0245 0.0064 0.0059 0.2082 0.0368 0.0056
2.4 0.0038 0.0556 0.0000 0.0178 0.0764 0.1991 0.0616 0.0859 0.0101 0.0442 0.0000 0.0253 0.1069 0.0000 0.0224 0.2724 0.0185 0.0000
2.5 0.1190 0.0101 0.0000 0.0096 0.1002 0.3537 0.1413 0.0724 0.0270 0.0061 0.0000 0.0520 0.0115 0.0065 0.0000 0.0907 0.0000 0.0000
2.6 0.0545 0.0764 0.1182 0.0000 0.1164 0.0182 0.0000 0.1000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.4164 0.0000 0.1000
3.1 0.0107 0.0893 0.0000 0.0089 0.0987 0.2065 0.0071 0.1112 0.0000 0.0653 0.0000 0.0143 0.0893 0.0089 0.0071 0.2451 0.0089 0.0286
3.2 0.0000 0.0000 0.0000 0.0072 0.3186 0.2099 0.0049 0.0397 0.0387 0.0097 0.0000 0.0302 0.0409 0.0252 0.0295 0.1393 0.1062 0.0000
4.1 0.0143 0.0234 0.0000 0.0632 0.2094 0.1668 0.0200 0.0174 0.0450 0.0200 0.0250 0.0000 0.0694 0.0273 0.0143 0.2596 0.0250 0.0000
4.2 0.1006 0.0732 0.0032 0.0000 0.1689 0.0387 0.0238 0.0322 0.1245 0.0000 0.0000 0.0000 0.0712 0.0455 0.0000 0.2618 0.0335 0.0227
4.3 0.0795 0.0562 0.0011 0.0046 0.0964 0.2261 0.0772 0.0872 0.0326 0.0204 0.0024 0.0149 0.0384 0.0414 0.0476 0.1210 0.0353 0.0176
4.4 0.0000 0.0000 0.0000 0.0148 0.2818 0.1072 0.0321 0.0470 0.0083 0.0000 0.0000 0.0041 0.0462 0.1243 0.0482 0.2310 0.0552 0.0000
4.5 0.2042 0.0000 0.0750 0.0000 0.0000 0.2057 0.0750 0.1405 0.0114 0.0000 0.0000 0.0114 0.0000 0.0000 0.0000 0.2769 0.0000 0.0000
Open in a new tab

Table A6.

Pre-post differences of transition probabilities (attendings).

Tasks To
1.1 1.2 1.3 1.4 1.5 2.1 2.2 2.3 2.4 2.5 2.6 3.1 3.2 4.1 4.2 4.3 4.4 4.5
From 1.1 0.0018 0.0181 −0.0386 0.0002 −0.0349 −0.0085 −0.0191 −0.0258 −0.0282 −0.0108 −0.0112 0.0486 0.0000 0.0000 −0.0116 0.1427 −0.0041 −0.0186
1.2 0.0292 0.0567 −0.0025 0.0000 0.0466 −0.0443 −0.0292 −0.1044 −0.0046 −0.0295 −0.0033 0.0111 −0.0101 0.0026 −0.0008 0.0784 −0.0009 0.0053
1.3 0.1869 0.0086 0.0000 0.0000 0.0286 −0.0945 0.0286 −0.1944 0.0000 −0.0576 −0.0320 0.0000 0.0000 0.0000 0.0591 0.0668 0.0000 0.0000
1.4 −0.0447 0.0000 0.0000 0.1484 0.1906 0.0832 0.0105 −0.2142 −0.0895 −0.0448 0.0000 −0.0906 −0.0026 0.0359 0.0000 0.0072 0.0106 0.0000
1.5 −0.0075 −0.0052 0.0010 0.0438 0.0699 0.0280 −0.0013 −0.0117 −0.0115 −0.0253 −0.0113 −0.0365 −0.0065 −0.0055 −0.0059 −0.0402 0.0194 0.0062
2.1 −0.0617 −0.0038 0.0020 0.0364 0.1305 0.0236 0.0004 −0.0237 −0.0208 −0.0674 −0.0080 0.0104 −0.0052 −0.0010 −0.0224 −0.0043 0.0078 0.0070
2.2 −0.0212 −0.0121 0.0004 0.0029 0.0381 −0.0329 0.0282 −0.0317 −0.0062 −0.0019 −0.0219 0.0061 −0.0464 −0.0185 0.0029 0.0348 0.0568 0.0227
2.3 −0.0828 −0.0280 −0.0206 −0.0002 0.0898 0.0381 0.0013 0.0010 −0.0055 −0.0034 −0.0056 0.0081 −0.0109 0.0027 −0.0143 0.0156 0.0168 −0.0023
2.4 −0.0329 0.0268 −0.0010 0.0039 −0.0385 −0.0833 0.0262 −0.0382 0.0051 0.0114 −0.0033 0.0193 0.0704 −0.0139 0.0110 0.0214 0.0155 0.0000
2.5 0.0802 −0.0156 −0.0049 0.0096 −0.0254 0.0070 −0.0562 −0.0378 0.0002 0.0061 −0.0017 0.0455 −0.0081 0.0065 −0.0067 0.0038 −0.0025 0.0000
2.6 −0.0284 −0.0020 0.1182 0.0000 −0.0174 −0.0741 −0.0604 0.0653 0.0000 −0.0182 −0.0083 0.0000 −0.0227 0.0000 −0.0227 −0.0293 0.0000 0.1000
3.1 −0.0466 0.0845 0.0000 0.0089 −0.0899 0.1174 −0.0269 0.0097 0.0000 −0.0534 0.0000 0.0143 0.0229 0.0089 −0.0090 −0.0621 −0.0072 0.0286
3.2 0.0000 0.0000 −0.0154 −0.0040 0.1394 −0.0178 −0.0616 −0.0650 −0.0253 −0.0171 −0.0045 0.0124 −0.0171 −0.0070 −0.0088 0.0223 0.0694 0.0000
4.1 0.0060 0.0069 0.0000 0.0632 −0.0056 0.0240 −0.0243 −0.0855 0.0367 0.0200 0.0250 −0.0244 0.0542 0.0273 −0.0084 −0.0882 −0.0269 0.0000
4.2 −0.0716 0.0686 0.0032 −0.0051 0.0810 −0.1287 0.0238 −0.0700 0.1126 −0.0312 −0.0033 −0.0256 0.0073 0.0198 −0.0051 −0.0058 0.0266 0.0035
4.3 0.0091 0.0119 −0.0051 0.0029 0.0441 0.0046 0.0143 −0.0210 −0.0142 −0.0309 −0.0104 −0.0047 0.0100 0.0006 −0.0256 −0.0006 0.0120 0.0031
4.4 −0.0048 −0.0206 0.0000 0.0148 0.1301 −0.0891 −0.0050 −0.0117 −0.0332 −0.0275 0.0000 0.0041 0.0256 0.1078 0.0328 −0.1455 0.0222 0.0000
4.5 −0.0301 0.0000 0.0750 0.0000 0.0000 0.0316 0.0565 −0.0123 0.0114 0.0000 −0.0185 −0.0442 −0.0222 0.0000 0.0000 −0.0472 0.0000 0.0000
Open in a new tab

Inline graphic ≥ 0.1

Summary points.

What was already known on this topic

  • EHR technology tends to create more or new work for physicians, such as increased time spent on documentation.

  • Little research has examined the effect of EHR implementation on all activities done by physicians, in particular ICU physicians.

What this study added to our knowledge

  • After EHR implementation, both residents and attendings in the ICU spent more time on review and documentation of clinical information.

  • Attendings spent more time on order management after the EHR was implemented, but no similar change was found for residents.

  • EHR implementation caused residents to switch tasks more often. In contrast, attendings switched tasks less frequently after implementation.

Acknowledgments

This research was supported by the Agency for Healthcare Research and Quality (AHRQ), Grant Number: R01 HS15274. Principal Investigator: Pascale Carayon, PhD; co-PI: Kenneth Wood, DO; and by the Clinical and Translational Science Award (CTSA) program, previously through the National Center for Research Resources (NCRR) grant 1UL1RR025011, and now by the National Center for Advancing Translational Sciences (NCATS), grant 9U54TR000021. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The Ph.D. studies of Kerry McGuire and Kara Schultz-Van Roy were funded by an AHRQ T32 training grant (HS000083) [PI: M. Smith].

Appendix

Fig. A1.

Fig. A1

Open in a new tab

Cell plots of mean transition probabilities from and to clinical review and documentation (task 1.5) pre- and post-EHR implementation for residents and attendings.

Footnotes

Conflict of interest

None of the authors have any conflicts of interest related to this research.

Authors’ contributions

PC, TBW, PLTH, JS, JMW, MBW and KEW made substantial contributions to the study conception and design. KM and KSVR contributed significantly to the acquisition of data. PC, BA, RB, RSC, PLTH and AX completed the initial data analysis. All other authors made significant contributions to the data interpretation. PC drafted the manuscript and all authors contributed content and provided feedback. All authors approved the final version of the manuscript. PC accepts direct responsibility for the manuscript.

Contributor Information

Pascale Carayon, Email: carayon@engr.wisc.edu.

Tosha B. Wetterneck, Email: tbw@medicine.wisc.edu.

Bashar Alyousef, Email: bashar.m.alyousef@gmail.com.

Roger L. Brown, Email: rlbrown3@wisc.edu.

Randi S. Cartmill, Email: rcartmill@cqpi.engr.wisc.edu.

Kerry McGuire, Email: kerrymcguire@gmail.com.

Peter L.T. Hoonakker, Email: phoonakker@cqpi.engr.wisc.edu.

Jason Slagle, Email: Jason.slagle@vanderbilt.edu.

Kara S. Van Roy, Email: kkvanroy@gmail.com.

James M. Walker, Email: James.M.Walker@siemens.com.

Matthew B. Weinger, Email: Matt.weinger@vanderbilt.edu.

Anping Xie, Email: axie1@jhmi.edu.

Kenneth E. Wood, Email: kewood@geisinger.edu.

References

  • 1.Holden RJ, Karsh BT. The technology acceptance model: its past and its future in health care. J Biomed Inform. 2010;43(1):159–172. doi: 10.1016/j.jbi.2009.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Holden RJ. Physicians’ beliefs about using EMR and CPOE: in pursuit of a contextualized understanding of health IT use behavior. Int J Med Inform. 2010;79(2):71–80. doi: 10.1016/j.ijmedinf.2009.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Carayon P, Cartmill R, Blosky MA, Brown R, Hackenberg M, Hoonakker P, et al. EHR acceptance by ICU physicians and nurses. In: Albolino S, Bagnara S, Bellandi T, Llaneza J, Rosal G, Tartaglia R, editors. Healthcare Systems Ergonomics and Patient Safety 2011. CRC Press; Boca Raton, FL: 2011. pp. 374–377. [Google Scholar]
  • 4.Yarbrough AK, Smith TB. Technology acceptance among physicians – a new take on TAM. Med Care Res Rev. 2007;64(6):650–672. doi: 10.1177/1077558707305942. [DOI] [PubMed] [Google Scholar]
  • 5.Hoonakker PLT, Carayon P, Brown RL, Cartmill RS, Wetterneck TB, Walker JM. Changes in end-user satisfaction with Computerized Provider Order Entry over time among nurses and providers in intensive care units. J Am Med Inform Assoc. 2012;20:252–259. doi: 10.1136/amiajnl-2012-001114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Hoonakker PLT, Carayon P, Walker JM, Brown RL, Cartmill RS. The effects of Computerized Provider Order Entry implementation on communication in Intensive Care Units. Int J Med Inform. 2013;82(5):e107–e117. doi: 10.1016/j.ijmedinf.2012.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform. 2009;78(Suppl 1):S69–S76. doi: 10.1016/j.ijmedinf.2008.07.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Oxentenko AS, West CP, Popkave C, Weinberger SE, Kolars JC. Time spent on clinical documentation: a survey of internal medicine residents and program directors. Arch Intern Med. 2010;170(4):377–380. doi: 10.1001/archinternmed.2009.534. [DOI] [PubMed] [Google Scholar]
  • 9.Penoyer DA, Cortelyou-Ward KH, Noblin AM, Bullard T, Talbert S, Wilson J, et al. Use of Electronic Health Record documentation by healthcare workers in an acute care hospital system. J Healthc Manag. 2014;59(2):130–144. [PubMed] [Google Scholar]
  • 10.Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12(5):505–516. doi: 10.1197/jamia.M1700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Weinger MB, Herndon OW, Gaba DM. The effect of electronic record keeping and transesophageal echocardiography on task distribution, workload, and vigilance during cardiac anesthesia. Anesthesiology. 1997;87(1):144–155. doi: 10.1097/00000542-199707000-00019. [DOI] [PubMed] [Google Scholar]
  • 12.Westbrook JI, Li L, Georgiou A, Paoloni R, Cullen J. Impact of an electronic medication management system on hospital doctors’ and nurses’ work: a controlled pre–post, time and motion study. J Am Med Inform Assoc. 2013;20(6):1150–1158. doi: 10.1136/amiajnl-2012-001414. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Apkon M, Singhaviranon P. Impact of an electronic information system on physician workflow and data collection in the intensive care unit. Intensive Care Med. 2001;27(1):122–130. doi: 10.1007/s001340000777. [DOI] [PubMed] [Google Scholar]
  • 14.Lopetegui M, Yen PY, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about? J Biomed Inform. 2014;49:292–299. doi: 10.1016/j.jbi.2014.02.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Salvendy G, editor. Handbook of Industrial Engineering. 3. John Wiley & Sons; New York, NY: 2007. [Google Scholar]
  • 16.Barnes RM. Motion and Time Study: Design and Measurement of Work. John Wiley & Sons; New York, NY: 1980. [Google Scholar]
  • 17.Zheng K, Haftel HM, Hirschl RB, O’Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inform Assoc. 2010;17:454–461. doi: 10.1136/jamia.2010.004440. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Miller A, Weinger MB, Buerhaus P, Dietrich MS. Care coordination in intensive care units: communicating across information spaces. Hum Factors. 2010;52:147–161. doi: 10.1177/0018720810369149. [DOI] [PubMed] [Google Scholar]
  • 19.Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: effects on physicians’ time utilization in ambulatory primary care internal medicine practices. J Am Med Inform Assoc. 2001;8:361–371. doi: 10.1136/jamia.2001.0080361. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Pizziferri L, Kittler AF, Volk LA, Honour MM, Gupta S, Wang S, et al. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform. 2005;38(3):176–188. doi: 10.1016/j.jbi.2004.11.009. [DOI] [PubMed] [Google Scholar]
  • 21.Shu K, Boyle D, Spurr C, Horsky J, Heiman H, O’Connor P, et al. Comparison of time spent writing orders on paper with computerized physician order entry. In: Patel V, et al., editors. MEDINFO 2001. IOS Press; Amsterdam, The Netherlands: 2001. pp. 1207–1211. [PubMed] [Google Scholar]
  • 22.Yen K, Shane EL, Pawar SS, Schwendel ND, Zimmanck RJ, Gorelick MH. Time motion study in a pediatric emergency department before and after computer physician order entry. Ann Emerg Med. 2009;53(4):462–468. doi: 10.1016/j.annemergmed.2008.09.018. [DOI] [PubMed] [Google Scholar]
  • 23.Asaro PV, Boxerman SB. Effects of computerized provider order entry and nursing documentation on workflow. Acad Emerg Med. 2008;15(10):908–915. doi: 10.1111/j.1553-2712.2008.00235.x. [DOI] [PubMed] [Google Scholar]
  • 24.Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations: effects on resource utilization. J Am Med Assoc. 1993;269(3):379–383. [PubMed] [Google Scholar]
  • 25.Maslove DM, Rizk N, Lowe HJ. Computerized physician order entry in the critical care environment: a review of current literature. J Intensive Care Med. 2011;26(3):165–171. doi: 10.1177/0885066610387984. [DOI] [PubMed] [Google Scholar]
  • 26.Ballermann MA, Shaw NT, Arbeau KJ, Mayes DC, Gibney RN. Impact of a critical care clinical information system on interruption rates during intensive care nurse and physician documentation tasks. In: Safran C, Reti S, Marin HF, editors. MEDINFO 2010. IOS Press; Amsterdam: 2010. pp. 274–278. [PubMed] [Google Scholar]
  • 27.Finkler SA, Knickman JR, Hendrickson G, Lipkin MJ, Thompson WG. A comparison of work-sampling and time-and-motion techniques for studies in health services research. Health Serv Res. 1993;28(5):577–597. [PMC free article] [PubMed] [Google Scholar]
  • 28.Ballermann M, Shaw N, Mayes D, Gibney RN, Westbrook J. Validation of the Work Observation Method By Activity Timing (WOMBAT) method of conducting time-motion observations in critical care settings: an observational study. BMC Med Inform Decis Mak. 2011;11(1):32. doi: 10.1186/1472-6947-11-32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Tipping MD, Forth VE, O’Leary KJ, Malkenson DM, Magill DB, Englert K, et al. Where did the day go? – a time-motion study of hospitalists. J Hosp Med. 2010;5(6):323–328. doi: 10.1002/jhm.790. [DOI] [PubMed] [Google Scholar]
  • 30.Douglas S, Cartmill R, Brown R, Hoonakker P, Slagle J, Van Roy KS, et al. The work of adult and pediatric intensive care unit nurses. Nurs Res. 2012;62(1):50–58. doi: 10.1097/NNR.0b013e318270714b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Slagle J, Weinger M, Dinh MT, Wertheim V, Williams K. Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology. Anesthesiology. 2002;96(5):1129–1139. doi: 10.1097/00000542-200205000-00016. [DOI] [PubMed] [Google Scholar]
  • 32.Weinger MB, Herndon OW, Zornow MH, Paulus MP, Gaba DM, Dallen LT. An objective methodology for task analysis and workload assessment in anesthesia providers. Anesthesiology. 1994;80(1):77–92. doi: 10.1097/00000542-199401000-00015. [DOI] [PubMed] [Google Scholar]
  • 33.Schultz K, Slagle J, Brown R, Douglas S, Frederick B, Lakhani M, et al. The Human Factors and Ergonomics Society, editor. Development of a job task analysis tool for assessing the work of physicians in the intensive care unit; Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting; Santa Monica, CA: The Human Factors and Ergonomics Society; 2006. pp. 1469–1473. [Google Scholar]
  • 34.Trusty J, Thompson BT, Petrocelli JV. Practical guide for reporting effect size in quantitative research in the journal of counseling & development. J Couns Dev. 2004;82(1):107–110. [Google Scholar]
  • 35.Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. The National Academies Press; Washington, DC: 2012. [PubMed] [Google Scholar]
  • 36.Wu RC, Tran K, Lo V, O’Leary KJ, Morra D, Quan SD, et al. Effects of clinical communication interventions in hospitals: a systematic review of information and communication technology adoptions for improved communication between clinicians. Int J Med Inform. 2012;81(11):723–732. doi: 10.1016/j.ijmedinf.2012.05.014. [DOI] [PubMed] [Google Scholar]
  • 37.Beuscart-Zephir MC, Pelayo S, Anceaux F, Meaux JJ, Degroisse M, Degoulet P. Impact of CPOE on doctor-nurse cooperation for the medication ordering and administration process. Int J Med Inform. 2005;74(7–8):629–641. doi: 10.1016/j.ijmedinf.2005.01.004. [DOI] [PubMed] [Google Scholar]

RESOURCES