Impact of an electronic medication management system on hospital doctors’ and nurses’ work: a controlled pre–post, time and motion study

Abstract

Objective

To quantify and compare the time doctors and nurses spent on direct patient care, medication-related tasks, and interactions before and after electronic medication management system (eMMS) introduction.

Methods

Controlled pre–post, time and motion study of 129 doctors and nurses for 633.2 h on four wards in a 400-bed hospital in Sydney, Australia. We measured changes in proportions of time on tasks and interactions by period, intervention/control group, and profession.

Results

eMMS was associated with no significant change in proportions of time spent on direct care or medication-related tasks relative to control wards. In the post-period control ward, doctors spent 19.7% (2 h/10 h shift) of their time on direct care and 7.4% (44.4 min/10 h shift) on medication tasks, compared to intervention ward doctors (25.7% (2.6 h/shift; p=0.08) and 8.5% (51 min/shift; p=0.40), respectively). Control ward nurses in the post-period spent 22.1% (1.9 h/8.5 h shift) of their time on direct care and 23.7% on medication tasks compared to intervention ward nurses (26.1% (2.2 h/shift; p=0.23) and 22.6% (1.9 h/shift; p=0.28), respectively). We found intervention ward doctors spent less time alone (p=0.0003) and more time with other doctors (p=0.003) and patients (p=0.009). Nurses on the intervention wards spent less time with doctors following eMMS introduction (p=0.0001).

Conclusions

eMMS introduction did not result in redistribution of time away from direct care or towards medication tasks. Work patterns observed on these intervention wards were associated with previously reported significant reductions in prescribing error rates relative to the control wards.

Background and significance

The extent to which computerized provider order entry (CPOE) systems significantly hinder or assist clinical work efficiency has been a subject of debate and investigation for over 20 years. Yet the evidence base to address this issue remains sparse.^1–3 Zheng and colleagues⁴ recently drew attention to the poor and inconsistent methods used in conducting time and motion studies to measure the impact of systems on work patterns. Controlled studies to measure the effects of CPOE on doctors’ work are rare and we identified no published controlled studies of the impact on hospital nurses’ work.^{1 2 4} Studies in ambulatory care^5–9 and speciality areas in hospitals, such as the ICU,^{10 11} predominate. Despite considerable developments in CPOE systems, the body of evidence regarding the impact of these systems on doctors’ and nurses’ work on general hospital wards is relatively sparse and reliance continues to be placed on results from important early studies.^{5 12 13}

Significant advances have been made in understanding the ways in which systems may disrupt and enhance work patterns and qualitative studies have played a vital role in understanding the complex socio-technical issues associated with integrating information systems into clinical work.^14–19 However, concerns of clinicians regarding the impact that systems have on the efficiency of their work continue to be raised and are a significant barrier to adoption.^20–22 Prior to the imminent introduction of an electronic medication management system (eMMS) (with electronic prescribing and medication administration records (eMAR)), we interviewed 50 hospital clinicians (doctors and nurses) and managers to determine their expectations of system introduction.²³ Work practice change associated with the system was the most strongly and frequently raised issue by all groups.²³

Specific issues often center on the perceived increased time it takes for doctors to prescribe medications and for nurses to perform medication administration using a computer compared to paper medication charts.^{7 22 24} As a result of this, there are concerns that there will be less time for clinicians to spend on direct patient care activities.²⁰ These beliefs are often in contrast to the system benefits promoted to clinicians, namely that information systems will improve efficiency and patient care.^{21 25} Time savings are anticipated by re-distributing time across medication tasks.¹² For example, while it may take longer to type a prescription compared to writing it, the ability to modify existing orders rather than prepare new orders and to use ordersets is expected to save time. Redistribution of time-benefits to others is also anticipated. For example, improving the legibility of orders is expected to reduce nurses’ time spent clarifying orders. However, there are few data available to indicate whether time savings made in a particular sub-task are offset by time losses in other sub-tasks to make medication practice more or less efficient overall. Qualitative studies have revealed that many doctors and nurses report both improved and worsening efficiency.^26–28

Improved sharing of medication information is possible with CPOE, but reduced face-to-face communication between clinicians has also been identified as a potentially negative impact of system introduction, but there is little quantitative evidence to clarify this issue.^{15 29}

We aimed to undertake a large controlled time and motion study of hospital doctors and nurses to measure whether the proportions of time spent in medication tasks, direct care, and communication with each other significantly changed following the introduction of an eMMS with electronic prescribing and eMAR.

Methods

Study design, setting, and sample

We used a controlled before and after study design and used a direct time and motion observational approach. Data were collected from staff on four wards (respiratory, renal/vascular, and two acute geriatric medical wards) in a 400-bed major public hospital in Sydney.

We recruited a sample of 70 nurses and 59 doctors on the wards and collected data as outlined in table 1. Nurses of all classifications were included: registered nurses (RN) (new graduates, RN with 2–4 years of experience, and RN with 5+ years of experience) and clinical nurse specialists (CNS). Approximately 80% participation by eligible nurses was obtained in both the pre- and post-periods.

Table 1.

Details of study samples in each study period

	Nurses	Doctors
Baseline data collection	3 wards 13 July 2005—2 March 2006	4 wards 6 July 2006—21 December 2006
Post-eMMS data collection	2 control and 1 intervention ward 13 August 2008—4 December 2008 Minimum of 9 months after intervention	2 control and 2 intervention wards 18 March 2009—13 July 2009 Minimum of 14 months after intervention
Number of staff observed	70 (30 pre: 8 new graduates, 15 RN, and 7 CNS; 40 post: 9 new graduates, 21 RN and 10 CNS)	59 (20 pre: 7 interns, 8 residents, and 5 registrars; 39 post: 13 interns, 13 residents, and 13 registrars)
Hours observed	276.90 h (pre: 133.17 h; post: 143.73 h)	356.26 h (pre: 150.88 h; post: 205.38 h)

CNS, clinical nurse specialists; eMMS, electronic medication management system; RN, registered nurses.

We sought to include all medical registrars, residents, and interns on the study wards. Physician specialists were excluded as they defer the vast majority of prescribing ordering tasks to more junior staff. A major challenge for the recruitment of doctors was the transitory nature of the medical workforce. For example, in the pre-period we identified a possible study population of 38 doctors of whom nine refused to participate (four because they did not want to be observed and five for a range of reasons such as about to go on leave and pending resignation or transfer). Nine who had agreed to participate were transferred from the study wards prior to observation, leaving 20 who participated. Thus, approximately 69% (20/29) of eligible doctors participated, although 76% had agreed (29/38) to do so. In the post-period we were able to capture more doctors as they rotated into the study wards.

There was a gap of approximately 2.6 years between pre- and post-data collection due to a delay in system implementation on the intervention wards. Post-data collection occurred at least 9 months after system implementation to ensure clinicians were familiar with the system.

For the nurses’ study, two wards acted as controls and one (a geriatric ward) was the intervention ward (the fourth ward was initiating implementation and thus was excluded). For the doctors’ study, two wards acted as controls and two (both geriatric wards) were intervention wards.

Intervention

At baseline all wards used paper medical records and medication charts. All wards had a CPOE system for ordering diagnostic tests and viewing results as well as ordering diets, transport, porters, and allied health consultations. The CPOE system did not allow clinical documentation. In the pre-intervention period, prescribing doctors wrote orders directly onto the medication chart. These charts were then used by nursing staff for medication administration. There was no intermediate transcription step between a prescriber's order and the final medication chart entry, as is the case in some countries. Clinical pharmacists undertake ward rounds and review patients’ medication charts. They will notate or contact the prescribing physician if changes are required. However, this step is not a pre-requisite for nurses to administer medications once a physician has documented an order. This process did not change after eMMS implementation.

On the intervention wards in the post-period, the Cerner Millennium PowerOrders system, allowing electronic medication management functions, was integrated with the hospital's existing CPOE system in November 2007. Prescribers were required to use the system to prescribe medications in the post-period. Prescribing is mainly by menu selection of pre-prepared order sentences which are triggered upon drug selection and which can be modified by the prescriber. ‘Care sets’ allow for a group of related orders to be selected and ordered simultaneously with a single click. Active decision-support at the time consisted of allergy alerts and drug–drug interaction alerts set at the most severe level. During the intervention period, heparin infusions and patient controlled analgesia remained on paper charts. The intervention did not include bedside verification technology, for example, bar coding. Intervention wards had access to both computers on wheels (COWs) and fixed computers (desk PCs) to access the eMMS. The control wards continued to use paper medication charts in the post-period.

Procedures

All nurses and doctors were invited to participate via information sessions followed by a direct approach. Rosters (schedules) from each ward were used to calculate the full-time equivalents for each nurse classification (RN new graduate, RN 2–4 years, RN 5+ years, and CNS) and medical classification (intern, resident, and registrar). The sampling strategy was prepared in order to ensure all hours of the work day and weekdays were sampled proportionately to ensure the sample was representative. Representative sampling was used to determine the number of minutes that participants needed to be observed for each hour of the day for each classification. Following signed consent, nurses and doctors were assigned a study identification number, and demographic information regarding their age, classification, and length of experience was collected. We did not store staff names once a unique identifier was assigned and did not match data for individual staff in the pre- and post-periods. Nurses were given no prior warning of observation periods. Observers randomly allocated nurses to a list for each observation session according to the sampling strategy. If a nurse at the top of the list was not working that day, observers selected the next nurse on the list. A similar approach was used for doctors, but on occasion pre-arranged times were requested as doctors were more mobile and locating them in the hospital could be difficult without this step. Nurses were observed between the hours of 7:00 and 19:00 and doctors from 8:00 to 20:00 on weekdays. Nurses and doctors were observed while both on and off the ward.

The Work Observation Method by Activity Timing (WOMBAT) method was applied.^30–33 This is a technique for undertaking direct observational studies of health professionals. Using software on a handheld computer, observers capture multi-dimensional aspects of work and communication patterns. The WOMBAT tool automatically captures all time data related to tasks and also details interruptions to work and multi-tasking (ie, tasks conducted in parallel). For example, details of tasks conducted in parallel, such as documenting and answering a question from a colleague, could be recorded and time-stamped automatically. Figure 1 provides a screenshot of the data collection tool. The tool can be reconfigured for different task types and study designs. Research teams interested in using this software should contact the corresponding author (JIW).

Figure 1.

Screenshot of Work Observation Method by Activity Timing (WOMBAT) data collection tool.

Information about 10 broad, mutually exclusive work categories was collected. Online supplementary appendix 1 gives the definitions for each of the tasks reported in this paper. This classification was developed following extensive observations and pilot testing.^{34 35} The method has been applied in Australian studies of health professionals^{31 36} and was most recently validated in Canadian studies of intensive care clinicians.^{32 37} The observers shadowed clinicians for an average of 1 h blocks, recording data using the handheld computer. For each task, the data collector recorded with whom the nurse/doctor completed the task, the information tools used, and any interruptions to work (defined as ceasing a task in order to respond to an external stimuli) or tasks completed in parallel (multi-tasking). Tasks are continually recorded and include work, social/personal (including breaks), and activities carried out in transit.

When the participant nurse/doctor engaged with patients, visitors, or other health professionals, the nurse/doctor was asked to introduce the observer and seek permission to continue. Alternatively, the observer would identify themselves. Several dummy observation sessions were undertaken as part of observer training conducted over 2–3 weeks. This also allowed participants to become accustomed to being observed.

Observer training

All eight observers were clinically experienced RNs or medical doctors. Inter-rater reliability tests were performed with two data collectors simultaneously, but independently, observing a clinician and comparing data. Kappa scores³⁸ for task classification were >0.85 throughout data collection, indicating high levels of agreement between observers. On average inter-rater reliability observation sessions lasted 35 min each.

The study was approved by the human research ethics committees of the University of New South Wales and the study hospital.

Statistical analysis

To assess changes in time spent on direct care and medication tasks for each profession, we calculated the proportion of total observed time in each task category by study period (pre/post) and group (control/intervention wards). We further examined medication sub-tasks for doctors and nurses as proportions of the total time spent in medication tasks. 95% CIs of proportions were obtained using the large sample normal approximation. The difference in changes post-eMMS in intervention groups relative to control groups was defined as the differences in the proportions (P) over time between control and intervention groups, that is, (P_{Intervention_post}–P_{Intervention_pre})−(P_{Control_post}–P_{Control_pre}). We compared the difference in changes post-eMMS between intervention and control groups in each task category using the z test for proportions with the level of significance set at p<0.05. Other descriptive statistics for number, average length, and frequency of tasks in each task category were also presented by study period and group.

Information sources (eg, desk PC, COW) used in medication tasks were determined by calculating the proportions of total medication task time undertaken using each source. Time spent in interactions was examined in terms of both proportion of total observation time in professional communication, and the proportion of overall task time completed with others. Data were analyzed using SAS V.9.2.³⁹

Results

Proportion of time spent in direct patient care by doctors and nurses before and after eMMS implementation

On all wards there was no significant change after eMMS introduction in the proportion of time nurses and doctors on the intervention wards spent in direct patient care activities relative to control ward clinicians (p=0.23 and p=0.08, respectively; table 2). Doctors on the control wards in the post-period spent 19.7% (average of 3.1 min/task; 2 h/10 h shift) of their time in direct patient care (excluding medication tasks) and those on the intervention wards 25.7% (average of 2.6 min/task; 2.6 h/10 h shift). Nurses spent 22.1% of their time in direct care activities on the control wards in the post-period and 26.1% on the intervention wards (table 2).

Table 2.

Task time distribution for doctors before (pre) and after (post) electronic medication management system (eMMS) implementation

Task	Group	Period	No. of tasks	Total task time (h)	Mean task time (s)	Frequency of task per hour	% Total* time	95% CI		Diff†	p Value
Nurses
Direct care	Control	Pre	952	20.9	79.0	9.2	20.1	18.0	22.3	3.77	0.23
		Post	769	15.7	73.4	10.9	22.1	20.0	24.3
	Intervention	Pre	257	6.0	83.4	8.8	20.3	15.8	24.9
		Post	886	19.1	77.4	12.2	26.1	23.3	29.0
Medication	Control	Pre	1806	20.8	41.6	17.4	20.1	19.1	21.0	−1.93	0.28
		Post	1264	16.8	47.8	17.9	23.7	21.2	26.2
	Intervention	Pre	532	6.1	41.3	18.2	20.9	19.0	22.7
		Post	1506	16.4	39.3	20.7	22.6	21.2	23.9
Professional	Control	Pre	1525	24.7	58.2	14.7	23.7	20.4	27.1	2.24	0.57
communication		Post	1536	14.2	33.3	21.7	20.1	17.7	22.5
	Intervention	Pre	345	7.1	73.9	11.8	24.2	18.4	30.1
		Post	1536	16.6	38.9	21.1	22.8	19.8	25.7
Doctors
Direct care	Control	Pre	270	14.4	191.5	2.7	14.5	11.4	17.7	4.60	0.08
		Post	378	19.6	186.4	3.8	19.7	17.3	22.1
	Intervention	Pre	248	8.3	120.4	4.8	16.0	13.6	18.3
		Post	562	26.8	172.0	5.4	25.7	23.4	28.0
Medication	Control	Pre	475	6.8	51.6	4.8	6.9	6.2	7.6	0.76	0.4
		Post	541	7.5	49.6	5.4	7.4	6.6	8.3
	Intervention	Pre	272	3.7	49.4	5.2	7.2	6.3	8.1
		Post	707	8.9	45.1	6.7	8.5	7.6	9.5
Professional	Control	Pre	1237	33.1	96.4	12.5	33.5	28.2	38.8	1.72	0.8
communication		Post	1795	36.8	73.8	17.9	36.6	33.4	39.8
	Intervention	Pre	726	17.0	84.2	14.0	32.7	23.7	41.8
		Post	1989	39.5	71.4	19.0	37.6	34.7	40.4

*Tasks which made up the remainder of clinicians’ time were: social/personal time, documentation (other than medication-related), transit time, administration, responding to pages, indirect care, and supervision/education.

†Percentage difference in change over time (pre vs post) between intervention and control groups, that is, Diff=(P_{Intervention_post}–P_{Intervention_pre})−(P_{Control_post}–P_{Control_pre}).

There was an overall significant temporal change in the proportions of time spent in direct care across all wards not associated with eMMS introduction. Over the 2.6 years of the study (the time from pre- to post-data collection) the proportion of time spent in direct patient care significantly increased for doctors from 15.0% (95% CI 12.7% to 17.3%) in the pre-period to 22.6% (95% CI 20.9% to 24.3%) in the post-period (p<0.0001), and for nurses from 20.2% (95% CI 18.2% to 22.1%) in the pre-period to 24.2% (95% CI 22.4% to 26.0%) in the post-period (p=0.003).

Proportion of time spent by doctors and nurses on medication tasks before and after eMMS implementation

Following the introduction of the eMMS on the intervention wards there was no significant change in the proportions of time spent by doctors or nurses on medication-related tasks (table 2) relative to clinicians on the control wards (p=0.4 and p=0.28, respectively). Doctors in the post-period on the control wards spent 7.4% (average of 49.6 s/task; 44.4 min/10 h shift) of their time on medication tasks and those on the intervention wards 8.5% (average of 45.1 s/task; 51 min/10 h shift). Medication tasks consumed a greater proportion of nurses’ overall time taking 23.7% (approximately 2 h in an 8.5 h shift) of their time on the control ward in the post-period and 22.6% (1.9 h/shift) on the intervention wards (table 2).

We examined time distribution for specific types of medication-related tasks to assess if time was redistributed across tasks following eMMS introduction (tables 3 and 4). Both doctors and nurses on the intervention wards significantly increased the proportion of time spent reviewing medications using the eMMS compared to their colleagues on the control wards (p=0.01 and p<0.0001, respectively). However, no overall changes occurred in the time proportions of medication tasks since this increase was offset by (non-significant) decreases in nearly all other medication-related tasks (tables 3 and 4).

Table 3.

Time spent on individual medication tasks by doctors*

Task	Group	Period	No. tasks	Total time in medication task (min)	Mean task time (s)	Frequency of task per hour	% Total medication task time	95% CI		Diff†	p Value
Prescribe	Control	Pre	78	57.6	44.3	11.46	14.1	11.3	16.9	−0.24	0.95
		Post	86	83.2	58.1	11.53	18.6	14.3	22.9
	Intervention	Pre	46	33.2	43.3	12.32	14.8	10.5	19.1
		Post	93	101.2	65.3	10.51	19.1	14.9	23.2
Prepare	Control	Pre	34	45.3	79.9	5.00	11.1	7.5	14.7	−0.54	0.9
		Post	18	37.6	125.2	2.41	8.4	5.3	11.5
	Intervention	Pre	18	25.0	83.3	4.82	11.2	5.4	17.0
		Post	17	42.1	148.4	1.92	7.9	2.7	13.2
Clarify	Control	Pre	41	30.0	43.9	6.03	7.4	5.8	8.9	−3.76	0.3
		Post	17	19.4	68.4	2.28	4.3	2.2	6.5
	Intervention	Pre	29	24.7	51.0	7.77	11.0	6.4	15.6
		Post	12	22.5	112.3	1.36	4.2	0.3	8.2
Discuss	Control	Pre	224	183.0	49.0	32.93	44.8	39.7	50.0	0.10	0.99
		Post	273	168.4	37.0	36.60	37.6	32.3	42.9
	Intervention	Pre	133	100.9	45.5	35.63	45.1	37.6	52.5
		Post	377	201.6	32.1	42.59	38.0	34.0	42.0
Review	Control	Pre	77	66.8	52.1	11.32	16.4	9.4	23.3	11.87	0.01
		Post	100	65.8	39.5	13.41	14.7	11.3	18.1
	Intervention	Pre	34	21.6	38.0	9.11	9.6	6.5	12.7
		Post	151	105.3	41.8	17.06	19.8	15.4	24.2

*Excludes the medication-related tasks of administration, transcribe, and find medication chart, as these categories had fewer than 10 tasks in each period.

†Percentage difference in change over time (pre vs post) between intervention and control groups, that is, Diff=(P_{Intervention_post}–P_{Intervention_pre})−(P_{Control_post}–P_{Control_pre}).

Bold p values represent significance at p<0.01.

Table 4.

Time spent on individual medication tasks by nurses*

Task	Group	Period	No. tasks	Total time in medication tasks (min)	Mean task time (s)	Frequency of task per hour	% Total medication task time	95% CI		Diff†	p Value
Prepare	Control	Pre	529	430.23	48.80	25.38	34.40	31.49	37.30	−6.61	0.171
medication		Post	415	382.58	55.31	24.74	38.02	33.5	42.53
	Intervention	Pre	190	152.93	48.29	31.14	41.77	35.3	48.24
		Post	513	382.57	44.74	31.20	38.78	34.44	43.12
Check	Control	Pre	117	75.65	38.79	5.61	6.05	5.27	6.82	0.65	0.576
medication		Post	54	44.95	49.94	3.22	4.47	3.52	5.42
	Intervention	Pre	32	19.93	37.38	5.24	5.44	3.78	7.11
		Post	58	44.52	46.05	3.53	4.51	3.56	5.47
Administer	Control	Pre	287	271.55	56.77	13.77	21.71	19.12	24.30	−5.43	0.122
medication		Post	194	211.75	65.49	11.57	21.04	17.68	24.4
	Intervention	Pre	99	94.05	57.00	16.22	25.69	21.26	30.12
		Post	196	193.28	59.17	11.92	19.59	16.48	22.7
Chart	Control	Pre	284	141.10	29.81	13.62	11.28	10.19	12.38	−0.41	0.789
medication		Post	147	66.03	26.95	8.76	6.56	5.57	7.56
	Intervention	Pre	111	50.60	27.35	18.19	13.82	11.44	16.2
		Post	190	85.72	27.07	11.55	8.69	7.6	9.78
Discuss	Control	Pre	200	122.40	36.72	9.59	9.79	8.52	11.06	−1.6	0.712
medication		Post	153	133.80	52.47	9.12	13.29	5.07	21.52
	Intervention	Pre	37	18.82	30.51	6.06	5.14	3.84	6.44
		Post	123	69.48	33.89	7.48	7.04	5.91	8.18
Review	Control	Pre	312	151.88	29.21	14.97	12.14	11.19	13.09	9.99	0.00001
medication		Post	118	46.85	23.82	7.03	4.66	4	5.31
	Intervention	Pre	48	21.77	27.21	7.87	5.95	4.45	7.44
		Post	194	83.48	25.82	11.80	8.46	7.14	9.78

*Excludes low frequency tasks of find order, and clarify or order medication, as these categories had fewer than 10 tasks in each period.

†Percentage difference in change over time (pre vs post) between intervention and control groups, that is, Diff=(P_{Intervention_post}–P_{Intervention_pre})−(P_{Control_post}–P_{Control_pre}).

Bold p values represent significance at p<0.01.

Over time doctors and nurses on all wards experienced an increase in the percentage of time devoted to medication tasks. For doctors, it increased from 7.0% (95% CI 6.4% to 7.5%) in the pre-period to 7.9% (95% CI 7.3% to 8.6%) in the post-period (p=0.03), while for nurses, it increased from 20.2% (95% CI 19.4% to 21.1%) in the pre-period to 23.1% (95% CI 21.7% to 24.5%) in the post-period (p=0.001). This increase was not associated with eMMS intervention.

Information sources used during medication tasks

We examined the percentage of medication tasks performed with different information sources before and after eMMS introduction. Nurses on the intervention wards in the post-period significantly reduced their reliance on patients’ paper medical records as part of the medication process. Before eMMS introduction, nurses spent 89.0% (95% CI 81.0% to 97.0%) of medication task time (and 88% of all medication tasks) using a paper record, which declined to only 14.4% (95% CI 12.4% to 16.4%) after eMMS introduction (11% of medication tasks), while there was a much smaller change on the control wards, from 79.6% (95% CI 75.5% to 83.6%) of medication time to 66.9% (95% CI 62.0% to 71.8%). There was a significant relative reduction in the percentage of medication time where paper medical records were used of 61.9% on the intervention wards after eMMS introduction compared to the control wards (p<0.0001). On the intervention wards after eMMS introduction, 52.5% (95% CI 48.9% to 56.0%) of medication tasks involved a COW (63% of medication task time) and 2.5% (95% CI 2.0% to 3.1%) a desk PC (3% of medication task time). No computers were used in the pre-eMMS period and no COWs were available on the control wards in the post-period.

We found no significant change before and after eMMS introduction in the use of paper medical records during medication tasks by intervention ward doctors relative to the control wards (p=0.6). However, intervention ward doctors completed a significantly greater percentage of their medication task time using a desk PC compared to the control ward doctors (27.0% vs 15.2%; p=0.022) and used a COW for 22.0% (95% CI 17.7% to 26.4%) of medication task time. Desk PCs were used on the control wards during medication tasks, for example, to access pathology results from the CPOE system.

Interactions between doctors and nurses and others before and after eMMS implementation

There was no significant change on the intervention wards in the proportion of time nurses and doctors spent in professional communication following the introduction of the eMMS (p=0.57 and p=0.8; table 2). There was also no temporal change in the proportion of time spent on professional communication for either doctors (pre: 33.2% (95% CI 28.6% to 37.9%) vs post: 37.1% (95% CI 35.0% to 39.3%); p=0.13) or nurses (pre: 23.8% (95% CI 20.9% to 26.7%) vs post: 21.4% (95% CI 19.5% to 23.4%); p=0.17). In the post-period, nurses spent about 1.8 h/8.5 h shift of their time on professional communication, while doctors spent about 3.7 h/10 h shift.

However, when we examined task time spent alone and with others, we found that doctors on the intervention wards in the post-period spent significantly more time with patients (6.3% increase; p=0.009), more time with other doctors (21.4% increase; p=0.003), and significantly less time working alone (14.8% decrease; p=0.0003) compared to doctors on the control wards (table 5). Doctors on the intervention wards spent a significantly greater proportion of their time with other doctors, largely due to an increase in the frequency of interactions (tasks per hour) (table 5). Nurses on the intervention wards spent a significantly lower proportion of time working with doctors following eMMS implementation compared to nurses on the control wards (4.2% decrease; p=0.0001; table 6). This was due to both fewer interactions (tasks per hour) and shorter interactions (mean task time) (table 6).

Table 5.

Frequency and proportion of time doctors interacted with others during tasks before (pre) and after (post) electronic medication management system (eMMS) implementation

Doctors interacting with	Group	Period	No. tasks	Mean task time (s)	Frequency of task per hour	% Total task time	95% CI		Diff*	p Value
No one	Control	Pre	868	90.49	8.77	18.40	15.60	21.20	−14.80	0.0003
		Post	1587	75.75	15.79	24.63	21.90	27.36
	Intervention	Pre	552	106.57	10.63	27.44	20.92	33.97
		Post	1696	58.43	16.18	18.87	16.55	21.19
Patient	Control	Pre	556	136.20	5.62	17.74	14.96	20.51	6.32	0.009
		Post	806	131.77	8.02	21.77	19.60	23.93
	Intervention	Pre	368	103.00	7.09	17.68	15.38	19.99
		Post	1210	121.64	11.54	28.03	25.92	30.14
Relative	Control	Pre	121	144.73	1.22	4.10	2.83	5.38	0.99	0.41
		Post	237	136.69	2.36	6.64	5.43	7.85
	Intervention	Pre	60	109.70	1.16	3.07	1.89	4.25
		Post	309	112.10	2.95	6.60	5.58	7.61
Nurse	Control	Pre	420	78.39	4.24	7.71	5.84	9.58	−2.53	0.45
		Post	607	57.16	6.04	7.11	5.63	8.59
	Intervention	Pre	281	87.70	5.41	11.50	5.57	17.42
		Post	691	63.60	6.59	8.37	7.18	9.55
Doctor	Control	Pre	2213	136.27	22.36	70.65	62.62	78.67	21.40	0.003
		Post	2872	105.49	28.57	62.09	57.38	66.80
	Intervention	Pre	1101	108.42	21.21	55.69	45.79	65.58
		Post	3768	95.52	35.94	68.53	64.18	72.88

*Percentage difference in change over time (pre vs post) between intervention and control groups, that is, Diff=(P_{Intervention_post}–P_{Intervention_pre})−(P_{Control_post}–P_{Control_pre}).

Bold p values represent significance at p<0.01.

Table 6.

Frequency and proportion of time nurses interacted with others during tasks before (pre) and after (post) electronic medication management system (eMMS) implementation

Nurses interacting with	Group	Period	No. tasks	Mean task time (s)	Frequency of task per hour	% Total task time	95% CI		Diff*	p Value
No one	Control	Pre	2546	40.63	24.50	24.93	22.34	27.51	−5.23	0.26
		Post	1978	51.12	27.93	35.65	31.35	39.95
	Intervention	Pre	671	44.93	22.93	25.54	19.29	31.79
		Post	1912	47.58	26.22	31.03	26.84	35.21
Patient	Control	Pre	2267	60.14	21.82	32.85	30.65	35.05	1.34	0.67
		Post	1475	60.11	20.83	31.26	29.06	33.47
	Intervention	Pre	678	57.38	23.17	32.96	28.53	37.39
		Post	1492	64.28	20.46	32.71	29.87	35.55
Relative	Control	Pre	345	58.46	3.32	4.86	4.14	5.58	1.28	0.07
		Post	229	56.35	3.23	4.55	3.76	5.34
	Intervention	Pre	78	47.58	2.67	3.14	2.43	3.85
		Post	266	45.28	3.65	4.11	3.6	4.62
Nurse	Control	Pre	2550	80.57	24.54	49.5	44.49	54.52	7.84	0.20
		Post	2007	52.37	28.34	37.06	33.04	41.09
	Intervention	Pre	737	81.14	25.19	50.66	41.59	59.74
		Post	2500	54.02	34.28	46.06	41.92	50.19
Doctor	Control	Pre	318	44.23	3.06	3.39	3.02	3.76	−4.20	0.0001
		Post	217	43.95	3.06	3.36	2.78	3.95
	Intervention	Pre	123	62.02	4.2	6.46	4.56	8.37
		Post	180	36.29	2.47	2.23	1.78	2.67

*Percentage difference in change over time (pre vs post) between intervention and control groups, that is, Diff=(P_{Intervention_post}–P_{Intervention_pre})−(P_{Control_post}–P_{Control_pre}).

Bold p values represent significance at p<0.01.

Discussion

Implementation of the eMMS was not associated with significant changes in the proportions of time doctors and nurses spent on direct patient care or medication-related tasks, relative to their colleagues on the control wards. As the hospital already had in place CPOE for orders other than medications, we were able to specifically examine the impact of the addition of this module on work. The results provide little support for arguments that eMMS use results in redistribution of time away from direct patient care or towards medication tasks.

However, task time redistribution did occur within some specific areas. Examination of medication sub-tasks showed that increases in time spent, for example, reviewing medications in the system, were compensated by reduced time in other areas. Increased time spent reviewing patients’ medication charts may be a potentially positive work practice change for patient care, or indicate that it took longer to scroll through medication charts to find information relative to paper charts. These results suggest that eMMS use may result in a redistribution of time within and across medication-related tasks.

There are limited comparative data, and differences in settings, work task classifications, and methods further hinder comparisons between studies. Early research, such as that by Bates et al¹² published in 1994, showed that following CPOE introduction time spent by doctors ordering increased. They found medical interns significantly increased ordering time from 5.3% to 10.5% (p<0.001) and surgical house officers from 6.4% to 15.5% (p<0.001). However, they also found that the use of ordersets took less time using CPOE compared to paper. In a more recent before and after study of CPOE, Zheng et al⁴⁰ observed medical residents for 68 h in a pediatric intensive care unit. They found no significant change in the time doctors spent in direct care or ordering. Our results from a larger controlled study are consistent with these findings. Further, we provide some of the first such data on nurses’ work patterns following CPOE introduction. CPOE systems and clinicians’ computer skills are likely to have substantially improved over time and both may have contributed to reducing the potential negative impact of CPOE on clinicians’ time.

Intervention ward doctors spent a significantly greater proportion of time with patients and other doctors. Interactions with patients may have increased as a result of the use of COWs at patients’ bedsides, particularly during ward rounds. The increased time with other doctors may have been due to increased use of a shared doctors’ room on wards in which several PCs are located. These hypotheses are consistent with observations from our previous studies of doctors’ use of computing devices on wards with eMMS at this site.⁴¹ We observed that paper medication charts were often reviewed and completed at the central nurses’ work station. However, the eMMS moved these tasks to either the COWs or the PCs in the doctors’ rooms. This physical shifting of work to another location may also have contributed to the observed significant decrease in interactions nurses had with doctors post-eMMS compared to control ward nurses. Interestingly, Zheng et al⁴⁰ also found that doctors spent a greater proportion of time interacting with patients (increasing from 1.18% to 4.05%; p<0.05) after compared to before CPOE implementation. However, as there was no control group in that study, it is not possible to be certain if this was associated with the CPOE system or a temporal change. No reasons for this increase in interactions were suggested by the authors. We found significant temporal changes in the proportion of time both nurses and doctors spent on direct patient care and medication tasks in the post-period. This may reflect the increased complexity and age of hospital patients over time. This finding also starkly demonstrates the importance of controlled studies of work patterns.

Very few previous studies have measured changes in nurses’ time in relation to the introduction of an eMAR system.² A study⁴² on one surgical ward in a UK hospital reported that following the introduction of a closed-loop medication system (comprising e-prescribing, automatic dispensing, bar code identification, and eMAR), medication administration rounds were significantly shorter, but nurses spent a significantly greater proportion of time on medication tasks outside these rounds. That study timed medication rounds and then used work sampling over 20 h to determine time on medication tasks outside rounds, but did not consider time in non-medication tasks such as direct care or professional communication.

Importantly, we are able to link the patterns of work observed in our study with previously published prescribing error rates from these same hospital wards during the same study periods. A review of the medication charts of 1948 patient admissions from these wards before and after eMMS implementation showed prescribing error rates significantly declined with eMMS by 66.1% (from 6.25 errors per admission (95% CI 5.23 to 7.28) to 2.12 (95% CI 1.71 to 2.54); p<0.0001), with no significant change in the prescribing error rates on three control wards.⁴³ Thus doctors on these eMMS wards achieved significant improvements in medication safety while devoting the same proportion of their time to medication and direct patient care tasks compared to doctors on the control wards where no improvement in prescribing error rates was observed.

Time and motion data are very valuable in addressing specific concerns that clinicians have about reduced efficiency and decreases in time spent on patient care associated with CPOE. CPOE systems induce strong emotional reactions⁴⁴ and questions raised about efficiency may be conflated with apprehension about other underlying matters such as shifts in roles and responsibilities which are often less well articulated.^45–47 Gaining an understanding of the dimensions of these concerns requires both quantitative and qualitative studies.⁴⁸ Our results suggest concerns about dramatic reductions in efficiency are unfounded.

Strengths and limitations

Studies of this type in real-world clinical settings are complex and subject to bias from different sources. In relation to published research on this topic, our study has made advances in addressing methodological issues identified as problematic in previous studies.⁴⁰ Our study complies with the criteria outlined in the STAMP guidelines⁴⁰ in relation to observational work study designs. We applied a controlled design which allowed us to account for temporal changes in patterns of work. We worked closely with hospital staff to identify possible confounders. Hospital staff did not identify any significant policy or practice changes which occurred only in the intervention or control wards during this period. It is possible that other factors which we did not measure may have influenced our results. The large sample sizes, a sampling strategy to reflect both staff composition and work at different times of the day, and our ability to account for multi-tasking are particular strengths of the study. Limitations of our study include that we only examined weekday work. The results may thus not be representative of evenings or weekends. We only included four wards, and practices on wards may vary. However, we found that at baseline there were no significant differences in task time distributions for direct care, medication tasks, or professional communication by ward. A further potential limitation of the study was that both intervention wards were acute geriatric wards. However, it is important to note that almost all admissions to these wards were via the emergency department. Also this hospital has no general medicine service and therefore we believe that the findings are likely to be generalizable to general wards. Patients on these wards have a high number of medications and often complex medication regimens. Thus, we believe these wards with heavy prescribing and medication administration loads are good test cases regarding the impact of the eMMS on staff efficiency.

We used a direct observational approach which lends itself to the ‘Hawthorne effect’ whereby participants may change their behavior while being observed. The extended length of our study reduced the chance of sustained behavioral change, particularly on busy hospital wards. Further, observational studies of clinicians in situ have suggested that the extent of behavior change is minimal.^{34 49 50} We had high participation rates for nurses (80%) and doctors (69%). It is possible that doctors and nurses who did not participate were systematically different from the remaining population in terms of the factors under investigation, namely the amount of time that doctors and nurses spend particularly on medication tasks, and that these clinicians’ work was affected in different ways to clinicians in the study. One of the most frequent reasons for not participating in the study was that a doctor was about to go on leave or move to another ward. The large sample sizes guard against the likelihood that this would be a significant form of sampling bias in this study. As indicated in the Methods section, we did not match nurses in the pre- and post-data analysis as ethics approval prohibited retaining individual nurse identification to allow repeated measures analysis.

For doctors we collected data on tasks undertaken beyond their official shift times, but there were insufficient data in this category to assess whether there had been any significant change after eMMS introduction. This is an area that should be considered in future studies. We calculated kappa scores on multiple occasions to test for the reliability of task classification. Methods for applying kappa scores for assessing both task and time concurrently in time and motion studies have not been developed. We are currently developing these methods and such techniques would strengthen future time and motion studies.

Empirical evidence to demonstrate the ideal length of time after the introduction of clinical information systems to assess impact on work practices is lacking. The long time period between pre- and post-data collection in our study is both a potential strength and weakness of the study. These systems can take a long time to bed-down and there have been criticisms that measurements taken too soon after system implementation may not reflect what will become ‘standard’ practice with these systems. However, we also acknowledge that a longer follow-up period may introduce the potential for other significant factors to confound the results. Hence, the importance of applying a controlled design.

Conclusions

This is one of the few controlled studies of the impact of eMMS on hospital clinicians’ task time distribution. The results demonstrate that clinicians’ concerns related to reduced efficiency and time away from direct clinical care as a result of eMMS implementation voiced in interviews and focus groups²³ were not realized. Most importantly, significant improvements were achieved with reductions in prescribing error rates, while maintaining similar patterns of task time distribution relative to colleagues on control wards. However, the implications of reduced interactions between doctors and nurses after eMMS implementation for safe medication administration should be a focus of future investigations.