Computerized Prescriber Order Entry Implementation in a Physician Assistant–Managed Hematology and Oncology Inpatient Service: Effects on Workflow and Task Switching

The authors found that CPOE implementation did not negatively affect time available for direct patient care and that workflow fragmentation decreased, which is likely beneficial.

Abstract

Purpose:

Little is known about the impact of computerized prescriber order entry (CPOE) systems on inpatient hematology/oncology services. The objective of this study was to quantify the impact of an inpatient CPOE implementation on workflow, with an emphasis on ordering and direct patient care time.

Methods:

We conducted a direct-observation time-and-motion study of the provider team of a hematology/oncology inpatient service both before and after CPOE implementation, characterizing workflow into 60 distinct activity categories. The provider team comprised physician assistants supervised by attending physicians. Results were adjusted to account for variations in the census. We also conducted an analysis of computer logs to assess CPOE system usage.

Results:

Study participants were observed for 228.0 hours over 53 observation sessions. There was little change in the proportion of census-adjusted time spent on ordering (10.2% before v 11.4% after) and on direct patient care (50.7% before v 47.8% after). Workflow fragmentation decreased, with providers spending an average of 131.2 seconds on a continuous task before implementation and 218.3 seconds after (P < .01). An eight-fold decrease in the number of pages was observed during the course of the study.

Conclusion:

CPOE implementation did not negatively affect time available for direct patient care. Workflow fragmentation decreased, which is likely beneficial.

Introduction

The benefits of computerized prescriber order entry (CPOE) are well documented. When implemented properly, these systems can improve the delivery of care by reducing prescriber errors,¹ eliminating lost orders,² reducing ambiguities from illegible handwriting or improperly formed orders,³ providing decision support alerts about dangerous, duplicate, or conflicting orders,⁴ and reducing the turnaround time for order fulfillment.⁵ CPOE is considered a core component of an electronic health record (EHR) system⁶ and is a core function for achieving meaningful use.⁷

Concerns have been raised about CPOE related to unintended consequences that could result in harm,⁸ increased time spent entering orders,^9–11 substantial workflow changes that could lead to dissatisfaction with use of the system,¹¹ questions about the overall benefit of CPOE,¹² and a lower than expected adoption rate.¹³

Other concerns have been raised about the potential impact of CPOE (and EHRs more broadly) on the interaction of clinicians with the care team and patients. Use of these systems has been associated with a decrease in both physician-nurse interactions^14–16 and time spent on direct (ie, face-to-face) patient care.^17,18 This may lead to the perception that patient centeredness is diminished, wherein clinicians spend more time interacting with computers and less with patients.¹⁹

The complexity of hematology/oncology care requires special workflow and safety considerations.^20,21 Although successful CPOE implementations in the oncology setting have been reported,^22–24 little is known about their impact on outcomes related to workflow, such as order-related time from caregivers, time available for direct patient care, and interruptions. To develop a better understanding of these issues, we conducted a workflow analysis of a CPOE implementation in an inpatient hematology/oncology service, including an observational time-and-motion study as well as an analysis of computer access logs. We applied recently developed analytic approaches to quantify the redistribution of clinicians' time before and after CPOE, change in the true flow of the work, and new workflow measures such as workflow fragmentation.²⁵ Workflow fragmentation, or task fragmentation, is the rate of switching between different tasks. A higher frequency of task switching occurs when the continuous time spent on a single task decreases. Higher fragmentation rates can result in an increased mental burden²⁶ and potentially more errors.

Methods

Clinicians and Their Work Environment

We studied a hematology/oncology inpatient service at a large academic medical center staffed by physician assistants (PAs). This non–house staff service was chosen because the same PAs could be observed both before and after the CPOE implementation, which has been a potential limitation of prior studies.^25,27,28 The PAs were the exclusive care providers who performed duties equivalent to those of house staff in other services and were supervised by hospitalist physicians and supported by faculty hematologists and oncologists who used the CPOE system to enter initial chemotherapy orders. The PAs entered all other orders, including those for drugs supporting chemotherapy administration (eg, antiemetics and blood products) as well as orders clarifying chemotherapy administration when questions arose from the nursing and pharmacy staff. Many academic medical centers as well as private practices are adopting nurse practitioners and PAs as primary oncology caregivers. Supervising physicians did not routinely use the electronic systems and were therefore not observed. There were two hospitalist (supervising) physicians during the study period. All patient care was documented in a stable, homegrown EHR in use since 1998; however, it did not support order entry. Before CPOE implementation, all orders were entered on paper. Clinicians used alphanumeric pagers to request a telephone conversation to transmit or clarify information.

CPOE Implementation

In March 2008, a commercial CPOE system (Sunrise Clinical Manager, Version 4.5; Allscripts, Chicago IL) was implemented in the University of Michigan Health System inpatient areas. Before the implementation, the clinical team underwent standardized CPOE training, including hands-on practice sessions. Computers were widely available in care environments, including patient rooms, staff workrooms, and nursing stations, and as mobile computers on wheels.

Workflow Analysis

We conducted a time-and-motion study using a pre-post design during three distinct time periods: 1 month before the CPOE implementation (referred to as T-1), 3 months after implementation (T+3), and 8 months after implementation (T+8). We conducted observations at two postimplementation time periods because there is no consensus regarding the time required to reach intervention maturity.²⁹ Task data were collected by paid trained observers using a tablet computer (Appendix Fig A1, online only).²⁵ The University of Michigan Institutional Review Board approved the study protocol and waived the requirement to obtain informed consent.

Tasks were recorded in 60 activities grouped into six themes and further into 12 categories. The task taxonomy was based on prior clinical workflow analyses (Appendix Table A1, online only).^30,31 The data collection tool supported recording interruptions, including those from pagers, which are potential causes for medical errors.^32,33 Observation sessions included both morning (8am to 12pm) and afternoon (12pm to 4pm) periods on weekdays in approximately 4-hour continuous blocks. Methodologic details are further described in Appendix Table A2 (online only).²⁹ The patient census was recorded for each day of the study period.

Time spent on activities was characterized using two approaches. The first computed aggregated overall time spent on each activity, consistent with prior workflow studies.^30,31 For the second approach, we analyzed the ordered sequence of events to describe and quantify the flow of the work, using a clinical workflow analysis tool.²⁵ This software tool includes a timeline belt visualization of discrete activities and their duration to allow for visually highlighting task switches as well as important measures of workflow. The tool also reports frequently occurring patterns of activity, such as task transitions and common workflow processes, and computes probabilities of task switching. Because the tool can only compare two periods simultaneously, this analysis excluded the T+3 period, based on the assumption that usage patterns would be more stable at T+8 compared with T+3.

For both approaches, we also segmented the analysis to include all of the data and a subset of the data we defined as clinical time, which was the time spent on all patient-related activities, excluding administration (eg, conferences/meetings, talking to colleagues about non–patient-related matters, and so on) and miscellaneous (eg, personal) activities. This was done to account for the variation in inpatient census among the observation periods. In times of low census, the PAs often used down time to accomplish non–patient-related tasks.

Computer Access Logs

To further investigate patterns of computer terminal and CPOE use that could relate to task switching and the potential for workflow interruptions, we examined audit log data from the CPOE system. The logs contained time-stamped user information and unique computer identifiers. Daily log data were trended and compared with the average daily census using linear regression analysis to determine the correlation between the measures and the census.

Results

Provider Activities in Aggregate

We observed the care team for 82.5, 75.0, and 70.5 hours at the T-1, T+3, and T+8 time periods, respectively. A total of 12, 12, and 19 observation sessions were conducted during these three time periods over a span of 2, 4, and 3 weeks, respectively. Patient census means during the observation periods were 11.3, 9.2, and 6.4, respectively. Only once did a clinician ask the observer to stop recording, which lasted for 23.4 minutes. The total numbers of clinical time hours, representing all observations except for administration and miscellaneous activities, were 64.3, 58.5, and 49.8 hours, respectively.

Provider Activity by Theme and Category

Direct patient care constituted the majority of the total time (Fig 1A) and clinical time (Fig 1B). When analyzed by category of activity, the proportion of total time devoted to computer—read increased between the T-1 and T+8 observations (Fig 1C), but the increase was less marked when analyzing only clinical time (Fig 1D). Computer—writing increased with each observation interval.

Figure 1.

Percentage of overall time spent on 60 distinct activities based on the direct-observation time-and-motion study at 1 month before (T−1), 3 months after (T+3), and 8 months after (T+8) computerized prescriber order entry (CPOE) implementation. (A, B) Activities grouped at the theme level, with (A) showing the percentage of overall total time and (B) showing the percentage of clinical time, in which the administration and miscellaneous categories were removed to compensate for variations in the census during the different time periods. (C, D) Activities grouped into 12 categories; (C) includes all activities for overall total time, whereas (D) shows the percentages as a proportion of clinical time only.

Order-Related Activities

Twelve activities related to ordering were aggregated, including writing orders, looking for order forms, and order clarifications (Appendix Table A3, online only). Ordering activities constituted 8.0% of total time before CPOE and 8.2% and 8.1% at time periods T+3 and T+8, respectively. When adjusted for the census, these differences remained small, representing 10.2%, 10.6%, and 11.4% of total clinical time, respectively. After the CPOE implementation, the time spent writing on paper dropped to < 1% of total and clinical time, but it was replaced with a comparable increase in writing orders and filling out forms on the computer, requiring approximately 7.1% of total time (9.1% of clinical time) at observation T+3 and 7.2% of total time (10.2% of clinical time) at observation T+8.

Paging Interruptions

Before the CPOE implementation (T-1), the PAs experienced 1.6 paging interruptions per hour. At the T+3 period, the pages and interruptions per hour dropped to 0.3, and they fell further at T+8 to 0.2 paging interruptions per hour.

Provider Workflow Dynamics

Task fragmentation and duration.

More task fragmentation occurred during the T-1 observations than during the T+8 observations (evident by longer runs of a single color before a task transition represented by a different color; Fig 2). The average time spent on continuous direct patient care activities increased significantly from 134.1 seconds at T-1 to 231.3 seconds at T+8 (P < .01). The average duration of each uninterrupted task also increased, from 131.2 to 218.3 seconds from the T-1 to T+8 periods (P < .01). These differences remained when analyzed using clinical time (138.3 seconds at T-1 v 215.4 seconds at T+8; P < .01).

Figure 2.

A timeline belt visualization showing the decreased fragmentation of workflow before and after computerized prescriber order entry (CPOE) implementation in a hematology/oncology service. Each row (belt) represents an observation session, in which a colored stripe designates a clinical activity in one of six themes. For example, gold represents direct patient care activities such as examining a patient, and blue represents indirect patient activities such as writing on paper forms or writing orders on a computer. The length of each color stripe is proportional to the duration of that activity. Sequential color changes represent cross-category transitions. The set of 12 belts at the top are the pre-CPOE observations, whereas the 19 belts at the bottom are the 8-month post-CPOE observations. Longer observations have been truncated in this figure, but the entire width shown here represents approximately 3.5 hours of observations per belt.

Task switching patterns.

The most frequently occurring two-sequence patterns of activities at T-1 and T+8 are summarized in Table 1. The patterns during both time periods were similar except for a marked increase in the frequency of switching between writing on the computer and other activities, highlighting the dominance that computer data entry had in the post-CPOE phase in terms of workflow change. When considering only clinical time, the difference was even more pronounced; whereas computer—writing was present in two of the six most frequent task transitions during T-1, it was present in all six of the most frequent transitions during T+8. Additionally, with the clinical time analysis, paper—writing was present in two of the six T-1 patterns and none of the T+8 patterns.

Table 1.

Most Frequently Occurring Two-Length Sequential Patterns Before and After CPOE Implementation at T-1 and T+8 Time Periods^*

Two-Length Sequential Patterns T-1 Period, Total Time	Frequency (No./hour)	Two-Length Sequential Patterns T+8 Period, Total Time	Frequency (No./hour)
Talking/rounding → walking/moving	2.30	Talking/rounding → walking/moving	0.99
Walking/moving → talking/rounding	2.20	Computer—writing → talking/rounding	0.96
Paper—writing → walking/moving	0.85	Walking/moving → talking/rounding	0.94
Talking/rounding → patient activity	0.78	Talking/rounding → computer—writing	0.87
Phone → computer—writing	0.78	Computer—read → computer—writing	0.86
Patient activity → talking/rounding	0.72	Computer—writing → computer—read	0.82
Computer—writing → phone	0.70	Talking/rounding → computer—read	0.65
Paper—writing → talking/rounding	0.67	Computer—writing → phone	0.61
Walking/moving → paper—writing	0.65	Phone → computer—writing	0.61

Abbreviation: CPOE, computerized prescriber order entry.

^*Frequency is defined as the average number of transitions occurring per hour. The arrows denote the transition from the activity on the left to the activity on the right.

Computer switching and CPOE system access.

From the CPOE access logs, we calculated the number of daily logins, computer workstation transitions, and unique workstations used by each PA. These are shown along with the average daily census in Appendix Figure A2 (online only). Computer switches measure the number of times a PA changed from one computer workstation to another, whereas unique computers used represents the number of distinct computers that an individual used for CPOE. Computer switches can be considered a proxy for workflow fragmentation and can negatively affect time efficiency. The close correlation between computer transitions and unique computers used (R² = 0.96) suggests that most of the time, the PAs only switched computers when moving to another patient. Because such switches were not modeled as transition of tasks in the workflow analysis, they were not captured as part of the workflow fragmentation measure. The number of logins was highly correlated to the census (R² = 0.98), with a slightly lower correlation between the census and the computer transition and unique computers used (R² = 0.94 for each).

Discussion

Overall, our findings were reassuring. We did not detect significant adverse effects with CPOE implementation regarding workflow, interruptions, task fragmentation, or available direct patient care time. In fact, some of these measures improved. Another recent study of CPOE also reported positive outcomes related to workflow, although the setting did not focus specifically on inpatient hematology/oncology.³⁴

We observed an eight-fold decrease in alphanumeric pages, a common cause of interruptions, after the CPOE system was introduced. Although we were unable to obtain the text of the pages to determine if they were related to clarifications, the decrease can possibly be attributed, in part, to a decreased number of order clarifications resulting from more legible, accurate, and complete orders using computerized order entry. Another study of a CPOE system found a six-fold decrease in order clarifications in a university hospital.³⁵ The CPOE system itself provides a task function to allow for nonurgent communications between providers and staff, and this may also have decreased the need for pages. Because of the limitations of the time-and-motion methodology, we are unable to make additional conclusions about the relationship between CPOE and the pages. It is quite probable that paging frequency is also dependent on the physicians in the service as well as the census and acuity of the patients. Pages are a well-known form of interruption in health care, with potential consequences for patient safety.³⁶ Thus, the reduction in pages observed, regardless of the underlying causes, is likely to be beneficial.

Task fragmentation, as measured by task transitions, also decreased, suggesting that use of the CPOE system allowed clinicians to focus on specific tasks for longer periods of time. This is ideal because increased task fragmentation has been correlated with an increased cognitive load and a potentially concomitant increase in errors.^37–40 This is in contrast to our findings from a similar study conducted in a resident trainee–staffed unit.²⁵ It is unclear what led to the differences, although several possibilities exist. First, the prior study was conducted in a pediatric intensive care unit, which may have different workflow requirements than an inpatient hematology/oncology service. Second, the prior study observed residents as opposed to PAs. Third, the prior study measured the impact of the activation group 3 (or wave 3) implementation of CPOE at our institution. The hematology/oncology service we observed was part of the activation group 4 implementation wave, which occurred approximately 1 year after the pediatric intensive care unit implemented CPOE. Therefore, it is possible that some CPOE workflow issues were optimized in the interval. Finally, although we accounted for the decreased census in the final observation period, the lower census may also have contributed to less workflow fragmentation.

It is reassuring that direct patient care time was not diminished, because such face-to-face time is important for maintaining clinician-patient relationships.^41,42 However, it should be noted that others who have observed a decrease in direct patient care (eg, rounding time) attributable to CPOE concluded that the decrease was a desirable outcome demonstrating improved efficiency.⁴³

Some measures, such as computer logins and computer workstation switches, were highly correlated with the census. This was also evident in the normalized measure of clinical time based on the time-and-motion data, introduced to account for the varying census. This finding suggests that workload and efficiency may be far more influenced by patient load than use of a CPOE system.

The results of our study must be interpreted within the context of its limitations. Although time-and-motion studies are considered one of the best approaches for workflow analysis,⁴⁴ observer bias and the Hawthorne effect (the change in routine behaviors in those who know they are being observed) are potential confounders, despite one recent study suggesting that this may be minimal among health care providers.⁴⁵ We also did not measure satisfaction with the system and are therefore unable to draw conclusions on the relationship between decreased workflow fragmentation and satisfaction. The computer literacy of the PAs was not measured, although it is possible that they were, on average, younger than many attending oncologists and more likely to accept the technology than older individuals.

Another potential limitation of our study was the timing of the observations. The T+3 time period was conducted mid July through mid August, a time when new residents join the hospital services. However, because we chose to observe a nonresident service, this effect should have been minimized. The T+8 time period was conducted in December, and we noted a smaller average daily census compared with the prior observation periods. The daily census can be quite variable, so it is difficult to predict the best months to conduct observations in advance. The average monthly census during the 9 months before and after the CPOE implementation ranged from 5.7 to 12.2, with some summer months having a low census as well. Unrecognized bias may also have influenced our results. For example, we do not know if the patient mix or acuity varied during our observation periods. However, no policy or other administrative changes were made during the study period that would have influenced the complexity or mix of the patients managed in the service.

There were also several strengths of our study. In an attempt to overcome limitations of earlier studies, we focused on the effect of CPOE introduction into an environment in which the electronic documentation portion using an existing EHR system was stable. Additionally, we observed the same care providers before and after CPOE implementation, removing a potential confounder. The attending hematology/oncology physicians themselves routinely rotate on and off the service, and as a result, it would not have been possible to observe the same caregivers before and after. A future study focusing on physicians and the high-risk chemotherapy orders they enter would be of great interest.

It is important to note that our study setting was an inpatient service, where CPOE is most commonly used. Outpatient clinic settings can use similar systems, which are often referred to as e-prescribing. The work environments of inpatient services are often more complex than outpatient clinical environments, the latter usually being more linear, with patients arriving at scheduled intervals and with a lower acuity of medical issues. Therefore, we would exercise caution in extrapolating our inpatient findings to the outpatient setting. However, the methodology we implemented could be used to conduct observations in the outpatient setting as well.

Introduction of CPOE in an inpatient hematology/oncology service may improve some aspects of care, such as reducing task transitions and pages/interruptions. We could not detect differences in order entry time by well-trained PAs before and after the implementation. The census had considerable impacts on many of the workflow measures we analyzed, much more than CPOE itself. The decision to adopt CPOE should be based primarily on the many benefits engendered by the use of such systems.

Appendix

Table A1.

Activity and Analysis Categories and Themes Used for the Study^*

Major Category	Minor Category	Analysis Theme
Computer—read	Medical record, data, laboratory	Indirect patient care: read
	Medical reference	Indirect patient care: other
	Article	Miscellaneous
	E-mail	Miscellaneous
	Web page (nonmedical)	Miscellaneous
	Other	Miscellaneous
	Order clarification	Indirect patient care: read
Computer—writing	Patient note	Indirect patient care: write
	Orders	Indirect patient care: write
	E-mail	Miscellaneous
	Forms	Indirect patient care: write
	Other	Indirect patient care: write
Patient activity	Examine patient	Direct patient care
	Procedure	Direct patient care
	Other	Direct patient care
Paper—read	Medical record, data, laboratory	Indirect patient care: read
	Medical reference	Indirect patient care: other
	Article	Miscellaneous
	Forms	Indirect patient care: read
	Other	Miscellaneous
	Order clarification	Indirect patient care: read
Paper—looking for	Medical record, data, laboratory	Indirect patient care: read
	Radiograph	Indirect patient care: read
	Forms	Indirect patient care: read
	Other	Miscellaneous
Paper—writing	Medical record, data, laboratory	Indirect patient care: write
	Orders	Indirect patient care: write
	Forms	Indirect patient care: write
	Other	Indirect patient care: write
Other—looking for	Patient	Indirect patient care: other
	Colleague	Indirect patient care: other
	Consultant	Indirect patient care: other
	Other	Miscellaneous
Personal	Eating	Miscellaneous
	Idle	Miscellaneous
	Restroom	Miscellaneous
	E-mail	Miscellaneous
	Web surfing	Miscellaneous
	Socializing	Miscellaneous
	Telephone	Miscellaneous
	Other	Miscellaneous
Talking/rounding	With patient/family	Direct patient care
	Colleague/staff for patient	Direct patient care
	Colleague/staff nonpatient	Administration
	Conference/meeting	Administration
	Examining patient	Direct patient care
	Other	Indirect patient care: other
	Order clarification	Indirect patient care: other
Telephone	Patient related	Direct patient care
	Dictating notes	Indirect patient care: other
	Personal	Miscellaneous
	Order clarification	Indirect patient care: other
	Other	Miscellaneous
Waiting	For computer	Indirect patient care: other
	For patient	Indirect patient care: other
	For telephone	Indirect patient care: other
	Other	Indirect patient care: other
Walking/moving	Walking, no talking	Miscellaneous
	Walking/talking for patient	Direct patient care
	Walking/talking nonpatient	Administration
Nonobserved period†	Observer break	Miscellaneous
	Physician request	Miscellaneous
	Patient/family request	Miscellaneous

^*Categories are based on two prior studies.^30,31

^†Nonobserved periods were excluded from the final analysis.

Table A2.

Study-Specific Information Reported in Accordance With the STAMP Checklist^*

Area	Element	STAMP Code	Description
Intervention	Type	INT.1	Implementation of CPOE
	System genre	INT.2	Commercial CPOE system implemented on top of an already existing (since 1998) homegrown EHR system
	Maturity	INT.3	Observations performed 1 month before, 3 months after, and 8 months after CPOE implementation; participants were using CPOE from the time it was implemented
Empirical setting	Institution type	ES.1	Large academic medical center
	Care area	ES.2	Inpatient hematology/oncology service
	Locale	ES.3	Small suburban city (Ann Arbor, MI)
Research design	Protocol	RD.1	Pre-post study
	Duration	RD.2	1-month pre time period was conducted during 12 days of observations over 16-day period 3-month post time period was conducted during 12 days of observations over 32-day period 8-month post time period was conducted during 15 days of observations over 25-day period
	Shift distribution	RD.3	Morning and afternoon shifts were observed in roughly equal proportions
			Most observation sessions were in approximately 4-hour blocks, although some lasted more than 8 hours
	Observation hours	RD.4	Total number of observation hours: 228.0 hours, broken down into: 1-month pre time period: 82.5 hours 3-month post time period: 75.0 hours 8-month post time period: 70.5 hours
Task category	Definition and classification	TC.1	Observations were categorized into 60 distinct tasks, which were grouped into six themes and 12 categories
	Acknowledgement of prior work	TC.2	This study was partly based on work by Overhage et al30 and later refined by Pizziferi et al31; certain analyses were performed based on work by Zheng et al29
	New development	TC.3	Categories initially developed for outpatient setting were modified for inpatient setting, as described in prior work by Zheng et al29
Observer	Size of field team	OBS.1	Two independent observers performed all observations
	Training	OBS.2	Both observers received training on the use of the tablet PC data entry tool as well as practice time observing participants before the start of observation recordings; observers were instructed not to interfere with patient care and to stop observations at the request of patients, families, or clinicians
	Background	OBS.3	Both observers were recent Bachelor of Science graduates in industrial and operations engineering who were in a fellowship working under direction of the study authors as well as Health System Program and Operations Analysis leadership
	Interobserver uniformity	OBS.4	There was no calibration done between the two observers; they conducted observations during nonoverlapping periods
	Continuity	OBS.5	Observer 1 conducted all pre-CPOE observations; observer 2 conducted all post-CPOE observations
	Assignment	OBS.6	Each observer shadowed the three PAs in the clinical service in roughly equal proportions; no specific assignment was done other than scheduling with the PAs
Subject	Size	SUB.1	Three research participants were in the study
	Recruitment and randomization	SUB.2	Participants were not recruited because all PAs in the service were observed
	Continuity	SUB.3	The same three PAs were observed both before and after CPOE implementation
	Background	SUB.4	All three participants were PAs working under direction of a hematology/oncology attending physician; all three were considered computer literate because they had been using a homegrown EHR before CPOE implementation; one was male, and two were female; they had 1.5, 3, and 10 years total experience as PAs, respectively, and 1.5, 1.5, and 2 years experience working in the hematology/oncology service before the study
Data recording	Multitasking	DR.1	Multitasking (eg, talking on telephone while using computer) was not recorded; observers were instructed to record the primary task at any given time
	Nonobserved periods	DR.2	Nonobserved periods were recorded by the observers, but these were short
	Between-task transition	DR.3	Transition periods between tasks were not handled in this study
	Collection tool	DR.4	A customized Microsoft Access database was used to collect data on a tablet PC running Microsoft Windows XP; the Access database was based on and modified from one developed by Pizziferri et al31 and available on the Agency for Healthcare Research and Quality Web site
Data analysis	Definition of key measures	DA.1	Average time spent on activities including ordering activities as well as average continuous time spent on a task were primary measures recorded
	Analytical methods	DA.2	Primary method used was a simple proportion (percentage) of time for each task
Ancillary data	Interruption	AD.1	Interruptions and pages were recorded separately from continuous time tasks
	Interaction	AD.2	Some interactions were recorded via the tool but not the specific individuals with whom interactions took place
	Location	AD.3	Majority of activities occurred in the inpatient ward, but location data during the observations were not recorded

Abbreviations: CPOE, computerized provider order entry; EHR, electronic health record; PA, physician assistant; STAMP, Suggested Time and Motion Procedures.

^*The STAMP checklist is based on work by Zheng et al.²⁹

Table A3.

Time Spent on 12 Order-Related Activities Before and After CPOE Implementation

Activity	Pre-CPOE (T-1)			Post-CPOE (T+3)			Post-CPOE (T+8)
	Time (minutes)	TT (%)*	CT (%)†	Time (minutes)	TT (%)*	CT (%)†	Time (minutes)	TT (%)*	CT (%)†
Computer—read
Medical reference	20.8	0.4	0.6	19.7	0.4	0.6	3.1	0.1	0.1
Order clarification	0.0	0.0	0.0	10.6	0.2	0.3	10.8	0.3	0.4
Computer—writing
Orders	3.3	0.1	< 0.1	319.2	7.1	9.1	304.1	7.2	10.2
Forms	0.5	< 0.1	< 0.1	0.2	< 0.1	< 0.1	0.0	0.0	0.0
Paper—read
Medical reference	1.5	< 0.1	< 0.1	0.0	0.0	0.0	0.0	0.0	0.0
Forms	3.1	0.1	< 0.1	0.0	0.0	0.0	0.0	0.0	0.0
Order clarification	1.3	< 0.1	< 0.1	2.3	< 0.1	< 0.1	1.4	< 0.1	< 0.1
Paper—looking for
Forms	34.4	0.7	0.9	0.0	0.0	0.0	0.0	0.0	0.0
Paper—writing
Orders	249.7	5.0	6.5	14.6	0.3	0.4	20.3	0.5	0.7
Forms	55.5	1.1	1.4	0.0	0.0	0.0	1.2	< 0.1	< 0.1
Talking/rounding
Order clarification	15.7	0.3	0.4	0.0	0.0	0.0	0.0	0.0	0.0
Telephone
Order clarification	9.1	0.2	0.2	4.0	0.1	0.1	0.0	0.0	0.0
Total	394.9	8.0	10.2	370.6	8.2	10.6	340.9	8.1	11.4

Abbreviations: CPOE, computerized provider order entry; CT, clinical time; TT, total time.

^*Includes all analysis themes in denominator.

^†Includes only patient-care analysis themes (excluding administrative and miscellaneous tasks) in denominator to compensate for census differences.

Figure A1.

Screenshot of the data collection tool used for the time-and-motion study. Observers used a Windows Tablet PC for data capture at the point of observation. Results were stored in a Microsoft Access database. Observations were recorded by selecting an activity and then choosing “Add Activity.” Interruptions and pages were recorded by clicking on the corresponding button. Comments were optional. This tool was modified from the system used by Pizziferri et al,³¹ which is available at the Agency for Healthcare Research and Quality Web site.

Figure A2.

Measures of computerized prescriber order entry system use each month based on audit log data for the hematology/oncology service during the 9-month postimplementation period. The average daily census is also shown and correlates closely with the usage measures.

Authors' Disclosures of Potential Conflicts of Interest

Although all authors completed the disclosure declaration, the following author(s) and/or an author's immediate family member(s) indicated a financial or other interest that is relevant to the subject matter under consideration in this article. Certain relationships marked with a “U” are those for which no compensation was received; those relationships marked with a “C” were compensated. For a detailed description of the disclosure categories, or for more information about ASCO's conflict of interest policy, please refer to the Author Disclosure Declaration and the Disclosures of Potential Conflicts of Interest section in Information for Contributors.

Employment or Leadership Position: None Consultant or Advisory Role: None Stock Ownership: Douglas W. Blayney, Google, IBM, Oracle, Qualcomm, Altera, Apple, Autodesk, Citrix, EMC Honoraria: None Research Funding: None Expert Testimony: None Other Remuneration: None

Author Contributions

Conception and design: David A. Hanauer, Kai Zheng, Mary G. Duck, Sung W. Choi, Douglas W. Blayney

Financial support: Douglas W. Blayney

Provision of study materials or patients: Elaine L. Commiskey, Douglas W. Blayney

Collection and assembly of data: David A. Hanauer, Kai Zheng, Elaine L. Commiskey, Mary G. Duck, Douglas W. Blayney

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors