Errors and electronic prescribing: a controlled laboratory study to examine task complexity and interruption effects

Abstract

Objective

To examine the effect of interruptions and task complexity on error rates when prescribing with computerized provider order entry (CPOE) systems, and to categorize the types of prescribing errors.

Design

Two within-subject factors: task complexity (complex vs simple) and interruption (interruption vs no interruption). Thirty-two hospital doctors used a CPOE system in a computer laboratory to complete four prescribing tasks, half of which were interrupted using a counterbalanced design.

Measurements

Types of prescribing errors, error rate, resumption lag, and task completion time.

Results

Errors in creating and updating electronic medication charts that were measured included failure to enter allergy information; selection of incorrect medication, dose, route, formulation, or frequency of administration from lists and drop-down menus presented by the CPOE system; incorrect entry or omission in entering administration times, start date, and free-text qualifiers; and omissions in prescribing and ceasing medications. When errors occurred, the error rates across the four prescribing tasks ranged from 0.5% (1 incorrect medication selected out of 192 chances for selecting a medication or error opportunities) to 16% (5 failures to enter allergy information out of 32 error opportunities). Any impact of interruptions on prescribing error rates and task completion times was not detected in our experiment. However, complex tasks took significantly longer to complete (F(1, 27)=137.9; p<0.001) and when execution was interrupted they required almost three times longer to resume compared to simple tasks (resumption lag complex=9.6 seconds, SD=5.6; resumption lag simple=3.4 seconds, SD=1.7; t(28)=6.186; p<0.001).

Conclusion

Most electronic prescribing errors found in this study could be described as slips in using the CPOE system to create and update electronic medication charts. Cues available within the user interface may have aided resumption of interrupted tasks making CPOE systems robust to some interruption effects. Further experiments are required to rule out any effect interruption might have on CPOE error rates.

Introduction

New errors are an unintended consequence of using information and communication technologies in complex socio-technical environments such as healthcare.^1–3 Errors arising from the use of computerized provider order entry (CPOE) systems have not been previously examined in a controlled setting. A number of uncontrolled studies have identified new mechanisms for making prescribing errors directly related to the use of CPOE systems for electronic prescribing.^4–6 The range of prescribing errors include selection of the wrong patient; missed drug allergies⁷; incorrect entry or selection of medication, dose, route, formulation, administration time, or frequency^{4 8}; incorrect entry or omission of start date⁴ and free-text qualifiers needed to administer drugs.^{8 9} These errors have been attributed to incorrect selection of information from drop-down menus and inappropriate use of free-text fields, but the primary causal mechanisms behind those actions remains unclear.

New ways of making errors have also been linked to the interruption-prone environment in which clinical tasks are undertaken.^10–12 A comprehensive examination of 7029 CPOE-related medication incidents reported to the US Pharmacopeia Medmarx database found that distractions were reported to be a significant contributing factor, contributing to eight out of ten errors.¹³ Few studies have quantified the effects of interruption on clinical tasks.^{14 15} Using observations of nurses administering medications on hospital wards, a recent study found that each interruption was associated with a 12% increase in procedural failures and a 13% increase in clinical errors.¹⁶ However, little is known about the effects of interruption on prescribing tasks using a CPOE system. A US observational study found that 31% of distractions interrupted clinicians' use of an IT system.¹⁷ The potential for interruptions to disrupt doctors' use of a CPOE system for electronic prescribing tasks and to generate errors has not been previously examined.

We sought to study the effects of interruption and task complexity on electronic prescribing with CPOE. Studies from other domains, such as aviation and clerical office work, suggest that interruption leads to more errors in flight-deck procedures,¹⁸ is associated with a delay resuming, and increases the time to complete computer-based document editing tasks.¹⁹ Furthermore, the disruptive effects of interruption appear to be more pronounced in complex tasks than simple tasks.^{20 21} Based on these previous findings, we sought to test the following hypotheses:

• H1: Interruptions increase the number of prescribing errors when prescribing is the primary task.

• H2: Interruptions increase the time taken to complete primary tasks (task completion time).

• H3: Interruptions are associated with a delay in resuming primary tasks (resumption lag).

• H4: Complex primary tasks are more susceptible to the effects of interruption resulting in higher error rates and longer task resumption times.

Methods

Participants

Thirty-four doctors participated in the study. Junior doctors and resident medical officers who used a CPOE system for electronic prescribing at a 300-bed teaching hospital attached to the University of New South Wales were invited via a call for volunteers advertised on the hospital's notice boards and via a clinician list-server. All junior doctors and resident medical officers are required to complete a formal CPOE training session conducted by the hospital and all participants in this study had complied with this requirement.

Primary prescribing tasks

Four primary prescribing tasks (two simple and two complex) were designed with advice from an expert panel, including four hospital doctors, a clinical pharmacologist and a pharmacist. The design of the prescribing tasks was based upon our observations of doctors' use of CPOE for electronic prescribing within a medical ward of the same hospital.²² Primary tasks were based on hypothetical clinical scenarios representing typical prescribing tasks undertaken by doctors working on a medical ward. The two simple tasks involved creation of an electronic medication chart. For each new chart, doctors were required to locate the patient's record and enter current allergy status before prescribing a required list of medications (table 1). The two complex tasks were based on updating existing electronic medication charts. In response to new therapeutic decisions, this task required a doctor to locate an existing medication chart and replace medications by first ceasing old medications and then adding new ones (table 2).

Table 1.

Simple primary prescribing tasks

Clinical scenario	Paper chart
Scenario A (simple) Mrs Dorothea M Collins is an 84-year-old woman who was admitted early this morning to the hospital (in W ward). She has hypertension, controlled atrial fibrillation and presented with acute pulmonary edema. Her observations are stable, and she doesn't have any chest pain. Following the consultant ward round this morning, you now need to chart her medications. Please use the long hand method (not the ‘Quick List’) to prescribe her medications.	Paper chart for scenario A: Simvastatin 80 mg PO daily: 2200 metoprolol 50 mg PO twice a day: 0800 and 2000 furosemide 40 mg PO twice a day: 0800 and 1200 aspirin 150 mg PO daily: 0800 amiodarone 200 mg PO three times a day: 0800, 1400 and 2000 lisinopril 5 mg PO daily: 2000 NB: allergy status available on paper chart ‘Nil known’.
Scenario B (Simple) Mrs Nancy West is a 76-year-old keen Bridge player (in X ward). She was admitted with a simple fall, which was due to a chest infection. She is normally quite healthy, is much improved, and has been reviewed as fit for discharge home by the ACAT team. The community health team will support her to get her home ASAP and she also has a supportive family network. She also has a groin candidal infection, and oesophageal reflux disease from a hiatus hernia, and is on a statin. Please chart her meds using the longhand method (not the ‘Quick List’).	Paper chart for scenario B: Augmentin Duo Forte (amoxicillin clavulanic acid) 1 tablet, PO twice daily: 0700 and 1700 furosemide 40 mg PO daily: 0800 omeprazole 20 mg PO twice daily: 0800 and 2000 Simvastatin 40 mg PO daily: 2000. clotrimazole (1%) cream topical (comment: to groin area) three times a day: 0800, 1400, and 2200. Paracetamol 1 g PO four times a day: 0600, 1200, 1800, and 2200. NB: allergy status available on paper chart ‘Nil known’.

PO, to be taken orally.

Table 2.

Complex primary prescribing tasks

Clinical scenario	Electronic chart
Scenario C (complex) Mrs Beverly Elizabeth Walker is a 72-year-old woman who has been in hospital for a few days (in Y ward), with a urinary tract infection (UTI). She has been on trimethoprim 300 mg daily. However, she hasn't really improved and last night spiked a fever of 39°C. This is explained by the microbiology results this morning as the E Coli (gram−ve) UTI is resistant to trimethoprim, but sensitive to both ciprofloxacin and ampicillin. The medical registrar has asked you to put her onto intravenous ciprofloxacin 200 mg two times a day. Please review her current medications in MedChart and chart the iv antibiotics. Her BP is low; thus, please stop the atenolol and add PRN sachets of Ural 1 sachet three times a day/PRN. Also she fell earlier and requires analgesia could you add in Endone (oxycodone) 5 mg PO/PRN/max 10 mg per day. Bone density scan was done while in Emergency and geriatricians have asked you to add Fosamax (alendronate)70 mg PO once a week at 0600 starting 1 week from today. She is being transferred to the ward from the emergency department in an hour and also needs a cannula, please! When charting her medications please use the longhand method (not the ‘Quick List’).	Electronic chart for scenario C: citalopram 20 mg PO daily: 0800 temazepam 10 mg, PO/PRN, nocte atenolol 50 mg PO daily atorvastatin 20 mg PO nocte Paracetamol 1 g PO four times a day trimethoprim 300 mg PO daily NB: Allergy status available in electronic chart: allergic to penicillin. Comments: ‘serious rash’.
Scenario D (complex) Mr Arthur Lindsay Foote is a 74-year-old gentleman who presented to hospital (in ward Z) with melaena and new onset dizziness. He had been on Naprosyn SR 1 g daily for 2 months for joint pain and early morning swelling affecting his metacarpophalangeal joints. On examination he has evidence of pallor and synovitis affecting the small joints of his hands. His BP lying down is 130/78 with a pulse of 84. On standing his BP is 100/63 and his pulse is 124. Blood results confirm anemia and as he has ongoing malaena he is transfused with 2 units of packed red cells. During the course of admission he undergoes an endoscopy, which reveals a peptic ulcer. He had been taking ranitidine previously at home, but requires this to be replaced with omeprazole 40 mg daily PO. He also has been reviewed by the rheumatologists who diagnosed him with new onset rheumatoid arthritis. They suggested stopping the Naprosyn SR and adding in folic acid 5 mg PO daily, methotrexate 7.5 mg PO weekly, and prednisolone 10 mg PO daily at 0700 to his medications. Please make the changes addressed above to his current medication chart in MedChart. When charting his medications please use the longhand method (not the 'Quick List').	Electronic chart for scenario D: simvastatin 40 mg PO daily nocte temazepam 10 mg PO daily nocte Paracetamol 1 g PO qid Naprosyn SR 1 g PO daily ranitidine 150 mg PO bd perindopril 5 mg PO daily NB: Allergy status available in electronic chart: nil known.

bd, to be taken twice a day; nocte, to be taken every night; PO, to be taken orally; PRN, to be taken as needed; QID, to be taken four times a day.

Complexity of the primary prescribing tasks was measured by the number of information cues a clinician needs to complete the task.²³ We used GOMS (Goals, Operators, Methods, and Selection rules), a well-recognized cognitive engineering method, to differentiate the relative complexity of prescribing tasks by working memory load requirements.²² A GOMS model of prescribing was used to compute the total, average, and peak number of working memory changes during task execution (simple tasks: total=214 items, average=9.4, peak= 12 vs complex tasks total=333, average=10.9, peak= 16; see Appendix A available as an online data supplement at http://jamia.bmj.com). The correct response to each of the primary prescribing tasks was pre-determined by consensus agreement among the doctors on the expert panel and validated by the pharmacist. Although each clinical scenario was different, the prescribing tasks within each level of complexity were designed to be operationally identical in terms of the number of medications to be prescribed or replaced. For instance, in the simple prescribing tasks, a new medication chart required prescription of six new medications with scheduled doses in addition to entering the patient's allergy status. The complex prescribing task involved updates to an existing electronic medication chart with six medications. Two medications were replaced and two new medications required prescription resulting in a total of eight medications on the chart.

Interrupting tasks

Two interrupting tasks were designed with input from the expert panel. In observations on the hospital wards we found that interruptions to doctors prescribing electronically often involved use of a different clinical application from the same computer terminal. Thus, interrupting tasks that involved use of an electronic medications reference manual (the Australian Medicines Handbook) were designed. Participants were asked to look up the appropriate dosage for the medication described in a hypothetical clinical scenario relating to a different patient (ie, not the same patient as in the primary task) (see Appendix B available as an online data supplement at http://jamia.bmj.com). The tasks required a categorical response for the correct route, dose, interval/duration, and maximum dose. Answers were pre-determined by consensus agreement among the expert panel doctors and validated by the pharmacist. Based on previous experiments, the duration of interrupting tasks was designed to be at least 75 seconds to ensure that participants disengaged from the primary task.²⁴

Experimental design

This study had two within-subject factors: task complexity (simple vs complex) and interruption (interruption vs no interruption) giving four experimental conditions: (a) simple with interruption, (b) simple with no interruption, (c) complex with interruption, and (d) complex with no interruption (figure 1). Each participant was asked to complete four primary prescribing tasks, half of which were interrupted with interruptions evenly distributed between simple and complex tasks (ie, one simple and one complex task was interrupted for each participant). To avoid order effects, the sequence of task presentation was counterbalanced using a balanced Latin square design, which also takes into account the distribution of interruptions, ensuring all four primary prescribing tasks were interrupted equally across all participants.

Figure 1.

Experimental design.

Procedure

Volunteers were asked to attend a 1-hour session at a computer laboratory within the hospital, which provided access to a training version of the CPOE that was set up with four hypothetical patient records. Electronic medication charts were created for the complex primary tasks to update existing electronic medication charts (see Appendix C available as an online data supplement at http://jamia.bmj.com). Patient records and medication charts were reset for each participant.

Instructional phase

Following written informed consent, doctors were asked to complete a short survey about their CPOE use, clinical experience, and provide demographic information (see Appendix D available as an online data supplement at http://jamia.bmj.com). As all participants were experienced in using CPOE and the online medications reference manual no training was provided specifically for the experiment. Participants were informed that the study was not a test of their clinical knowledge or their computer skills and the true objective of the study was not disclosed until the end of the session. However, participants were informed that they may sometimes be required to look up medications-related information using the online medications reference manual, which was available on their computer task bar (Appendix C available as an online data supplement at http://jamia.bmj.com). An investigator showed the participants how to switch back and forth between the primary prescribing task and the medications reference task by clicking on the computer task bar. Participants were instructed that it was important to attend immediately to any requests from the investigator in order to prevent delays in attending to the interruption (ie, by completing current activity related to the primary task).

Primary task assignment

Participants were sequentially presented the primary prescribing tasks in random order according to the Latin square design. Following the procedure used at the time in the hospital's medical wards, participants were provided with paper medication charts to complete the simple prescribing tasks to create new electronic medication charts. No paper charts were provided for the complex tasks as participants were required to make changes to existing electronic charts.

Interrupting task assignment

In order to maintain a uniform interruption position across all primary prescribing tasks, the interrupting task was introduced when a participant began entering the first medication. An investigator (SYWL) initiated the interruption by asking the participant to carry out the interruption task on a paper form. Interruptions were controlled by monitoring participants' computer activity, which was observed at a separate computer workstation using the TechSmith Morae screen capture software that allowed live observation. Participants were instructed to return the interrupting task to the investigator once it was completed and continue with the primary prescribing task.

Data collection

Use of the CPOE and online medications reference manual to complete prescribing and interrupting tasks was recorded using Morae, which also captured a video of all screen activity and an audio recording of all verbalizations. Clinical or technical assistance was not provided. We did not intervene to encourage speed of performance. All participants completed the study within the hour allocated. Ethical approval for the protocol was received from the University of New South Wales and hospital research ethics committees. All participants received two movie vouchers (AUD 20) for an hour of their time to participate in the study.

Outcome measures

Participants' responses to the prescribing tasks with and without interruptions were compared using data extracted from computer printouts of all medication charts from the CPOE system. Prescribing errors were determined by checking participants' responses against correct answers determined by the expert panel and validated by the pharmacist. This process was carried out by one investigator (SYWL) and then double-checked by a second for accuracy (FM). When coders disagreed, the medication chart was re-examined and a consensus code was assigned. The impact of interruptions was determined by examining Morae screen capture videos and audio recordings to calculate the various timing measures. The following measures were examined:

1. Types of prescribing errors

We identified specific errors associated with creating and updating electronic medication charts, such as failure to enter allergy information; selection of incorrect medication, dose, route, formulation, or frequency from lists and drop-down menus presented by the CPOE; incorrect entry or omission in entering administration times, start date, and free-text qualifiers; and omissions in prescribing and ceasing medications (table 3).

Table 3.

Mean error rate (%) by prescribing error type

Prescribing error type	Simple				Complex
	Scenario A		Scenario B		Scenario C		Scenario D
	Errors (error rate, %)	N	Errors (error rate, %)	N	Errors (error rate, %)	N	Errors (error rate, %)	N
Wrong patient*	0	32	0	32
Missed allergy*	5 (16)	32	5 (16)	32
Medication name	0	192	1 (0.5)	192	0	128	0	128
Dose	0	192	1 (0.6)	160	3 (2.3)	128	1 (0.8)	128
Route	0	192	0	160	1 (0.8)	128	0	128
Formulation	0	192	0	192	0	128	0	128
Administration time	9 (2.8)	320	16 (3.8)	416	1 (1)	96	0	128
Frequency	6 (1.0)	192	0	192	5 (3.9)	128	0	128
Medical omission	0	192	0	192	4 (3.1)	128	1 (0.8)	128
Date error	0	192	0	192	6 (4.7)	128	6 (4.7)	128
Qualifier omission‡			4 (12)	32
Cessation error†					5 (8)	64	2 (3)	64
PRN max dose omission†					4 (6)	64
Total	16	1728	27	1792	29	1120	10	1088
Mean error rate, % (SD)	0.93 (1.49)		1.51 (1.64)		2.59 (2.55)		0.92 (1.57)

^*Error type not applicable to the complex tasks.

^†Error type not applicable to the simple tasks and scenario D.

^‡Error type not applicable to scenario A and the complex tasks.

N=number of opportunities for error. For example, six medications are prescribed in scenario A, which is completed by all 32 participants; therefore, creating 192 opportunities for error in selecting the medication name.

2. Error rate

A normalized error rate (equation 1), which takes into account the number of opportunities for making an error (or error opportunities) in executing a particular prescribing task was calculated:^{24 25}

$$\mathrm{Error}\_\mathrm{rate}=\frac{E_j}{E_n}$$ (1)

where, E_j=no. of errors and E_n=no. of error opportunities.

An error opportunity is a ‘chance to make a mistake’.²⁶ The error opportunities we examined were based upon prescribing error types (table 3; wrong patient, missed allergy, wrong medication, etc). For each error type we identified critical subtasks or interactions with the CPOE that could result in an incorrect prescription if incorrectly executed (see Appendix E available as an online data supplement at http://jamia.bmj.com). For example, when prescribing a medication, there is one opportunity to make a mistake in selecting the correct name (item) from the list of medications presented by the CPOE system. In scenario A, which is completed by all 32 participants, six medications are prescribed, presenting 6 times 32 = 192 opportunities for error in selecting a medication name. Thus, error rates were calculated for each prescribing error type and an overall error rate was calculated by summing all errors across all opportunities to make a prescribing error (table 3).

3. Resumption lag (t_rlag)

This is the time taken to re-orient and then restart the primary task after an interruption is over (figure 2). To compute the resumption lag we examined the time elapsed from when a participant clicks back to the CPOE system on their computer task bar (after completing the interruption task using the online medications reference manual) and the first action on screen—that is, key stroke or mouse click from the video log.

Figure 2.

Resumption lag: the time taken to re-orient and resume a primary task at the end of an interruption (adapted from³¹).

4. Task completion time (T_c)

The time taken to complete primary tasks was also calculated using the video log (equation 2):

$$T_c=T_{\mathrm{total}}-T_i-t_{\mathrm{rlag}}$$ (2)

where, T_total = time taken to complete the primary task including any interrupting tasks;

T_i = time taken to complete the interrupting task;

t_rlag = resumption lag.

Statistical analyses

We calculated that 32 primary tasks were required in each arm to detect an increase in errors when interrupted, from 10% to 30% in complex tasks, and from 4% to 10% in simple tasks, with 80% power and p< 0.05.²⁷ Baseline measures for the power calculation were based on a conservative estimate derived from the literature. For simple computer-based tasks, which included reading, counting, comprehension, and selecting tasks, errors were observed in 20% of tasks undertaken at baseline.²⁸ A two-way repeated-measures analysis of variance (ANOVA) was used to assess main effects of interruption and task complexity and an interaction effect of interruption × task complexity. Measures of effect size in AVOVA were examined using partial eta squared (η_p²), which is the proportion of variance in the error rate and task completion time (dependent variables) that is attributable to each effect (independent variables) (η_p²: 0.01= small; 0.06=medium; 0.14=large effect size).²⁷ A one-way repeated measures ANOVA was used to compare error rates by task complexity.

For interrupted tasks, differences in resumption lag between simple and complex tasks were examined using Student t test. The relative magnitude of the difference was examined using η_p², which is the proportion of variance in the resumption lag (dependent variable) that is explained by task complexity (independent variable). All statistical analyses were undertaken using SPSS v17.0 software.

Results

We recruited 34 participants from a cohort of 80 junior doctors and residents who worked at the hospital. The average age of the participants was 27.5 years (SD 3.2) and 59% (N=19) were female. On average, doctors had 2–5 years of clinical experience and reported 6–9 months' experience in using the hospital CPOE for electronic prescribing. Participants indicated having good computer skills in using the CPOE at an average frequency of 2–6 times/week in the last month. Data from two participants were excluded from analysis; one participant's data was lost due to a technical error and a second participant who was interrupted for a considerable length of time because of an emergency event was also excluded.

Impact of interruption on prescribing tasks

We examined the mean error rate for simple and complex tasks with and without interruptions (table 4). The effects of interruption on error rate (F(1, 31)=0.899, p=0.350 (η_p² = 0.028, observed power = 0.151)) and task complexity (F(1, 31)= 3.229, p=0.082 (η_p² = 0.094, observed power = 0.414)) were not statistically significant. The interaction interruption × task complexity also did not reach statistical significance (F(1,31)=0.018, p=0.893 (η_p² = 0.001, observed power = 0.052)).

Table 4.

Error rate and task completion times for simple and complex tasks

Variable	Simple tasks		Complex tasks		Results of ANOVA
	Interrupted	Non-interrupted	Interrupted tasks	Non-interrupted
Mean error rate (N=32) % (SD)	1.0 (1.5)	1.4 (1.6)	1.6 (2.0)	1.9 (2.5)	Interruption: F(1, 31)=0.899, p=0.350 Task complexity: F(1, 31)= 3.229, p=0.082 Interruption × task complexity: F(1, 31)=0.018, p=0.893
Task completion time (N=28*) seconds (SD)	257 (65)	248 (54)	333 (94)	361 (107)	Interruption: F(1, 27)=0.54, p=0.469 Task complexity: F(1, 27)=137.9, p<0.001 Interruption × task complexity: F(1, 27)=1.41, p=0.245

^*Four extreme outliers identified by SPSS were eliminated from the analyses.

Task completion times for simple and complex prescribing tasks with and without interruptions are listed in table 4. Complex tasks took significantly longer to complete than simple tasks (F(1, 27)=137.9, p<0.001 (η_p² = 0.836, observed power = 1.000)). Using η_p² as the measure of association, task complexity accounted for 83.6% of the total variability in the task completion time. Neither the effect of interruption on task completion time (F(1, 27)=0.54, p=0.469 (η_p² = 0.020, observed power = 0.109)) nor the interaction interruption × task complexity reached statistical significance (F(1, 27)=1.41, p=0.245 (η_p² = 0.050, observed power = 0.209)).

Resumption lag was examined for interrupted tasks. Complex tasks required almost three times longer to resume compared to simple tasks (resumption lag_complex=9.6 s, SD=5.6; resumption lag_simple=3.4 s, SD=1.7; t(28)=6.186, p<0.001; η_p²= 0.415).

Types of prescribing errors

To understand types of error associated with CPOE use we pooled data from interrupted and non-interrupted tasks to quantitatively and qualitatively examine error rates for specific types of prescribing errors (table 3). The main errors observed were:

Missed allergy

Failure to enter allergy information into the allergy field accounted for the highest error rate in simple tasks (error rate =16%; N=32).

Medication name

Incorrect selection of a medication from the list presented by the CPOE resulted in prescription of the wrong medication (error rate up to 0.5%; N=192).

Dose

Incorrect selection of a dose from a list presented by the CPOE resulted in prescription of the wrong dose (error rate up to 2.3%; N=128).

Route

Incorrect selection from a CPOE list resulted in prescription of tablets instead of intravenous medication (error rate up to 0.8%; N=128).

Administration time

Failure to change default times for administration of medications was common. The error rate was 2.8% for scenario A (N=320) and 3.8% for scenario B (N=416).

Frequency

Incorrect selection from a CPOE drop-down menu resulted in the wrong frequency (scenario A: error rate=1.0%; N=192). For complex tasks, incorrect selection of the frequency involved PRN doses and a weekly dose (scenario C error rate=3.9%; N=128).

Medication omissions

These involved failure to add new medications when updating an existing electronic medication chart (error rate=0.8% and 3.2% for scenarios C and D, respectively; N=128).

Date error

Incorrect selection of the start date resulted in early commencement of a medication.

Qualifier omission

Omissions in including a text qualifier for administration of a topical medication in scenario B were associated with an error rate of 12% (N=32).

Cessation error

Failure to cease a medication accounted for the highest proportion of errors in scenario C (error rate=8%; N=64). In comparison, the error rate associated with failure to cease a medication in scenario D was 3% (N=64).

PRN max dose omission

A relatively high proportion of errors in scenario C involved errors or omissions in specifying the maximum dose for a PRN medication (error rate=6%; N=64).

Tables 5 and 6 list examples of each prescribing error type. A comparison of error rates by task complexity showed that there was a significant effect (F(29, 3)=4.51, p=0.010 (η_p² = 0.318, observed power = 0.834)). Error rates for complex scenario C were higher than the simple scenario B (error rate_C=2.59%, N=1120; error rate_B=1.51%, N=1792; p=0.007).

Table 5.

Medications associated with errors in simple tasks

Prescribing error type	Examples of errors in simple tasks	Medication type	Potential implications
Missed allergy	Patient allergy status not entered	NA	Nil known allergies in both scenarios; general risk
Medication name	Augmentin Duo prescribed instead of Augmentin Duo Forte	Penicillin antibiotic	Augmentin Duo Forte tablets can be broken in half for elderly patient; unclear if for financial benefit of hospital and patient
Dose	Paracetamol 500 mg prescribed instead of 1 g	Analgesic	Less than prescribed dose; depends on pathology
Administration time	furosemide 40 mg PO twice daily at 0800 and 2000 instead of 1200	Diuretic	Twice daily diuretics generally administered at midday (second dose) to minimize disruption of sleep
	lisinopril 5 mg PO daily at 0800 instead of 2000	ACE inhibitor	Blood pressure lowering drug administered at night to reduce risk of falls
	simvastatin 80 mg PO daily prescribed at 2000 instead of 2200	Statin	Very unusual; motivation unknown
	simvastatin 40 mg prescribed at 2200 instead of 2000
	clotrimazole prescribed at 0600, 1400, and 2000 instead of 2200	Anti-fungal	Resistance/efficacy threat
	Augmentin Duo Forte 1 tablet PO twice daily prescribed at 0800 and 2000 instead of 0700 and 1700	Penicillin antibiotic	Probably no consequences
Frequency	lisinopril prescribed in the morning instead of 2000	ACE inhibitor	Blood pressure lowering drug administered at night to reduce risk of falls
Qualifier omission	Instructions not provided for clotrimazole (1%) cream ‘topical to groin area’	Anti-fungal	Needed to administer the drug appropriately

Table 6.

Medications associated with errors in complex tasks

Prescribing error type	Examples of errors in complex tasks	Medication type	Potential implications
Dosage	Methotrexate 10 mg instead of 7.5 mg PO weekly	Cytotoxic and immunosuppressant	Dosage is highly controlled due to toxicity
	ciprofloxacin 250 mg tablet instead of IV 200 mg	Antibiotic	Wrong dose and route; efficacy threatened
Route	ciprofloxacin 250 mg tablet instead of IV 200 mg	Antibiotic	Wrong dose and route; efficacy threatened
Administration time	Fosamax PO 70 mg once/week 0800 instead of 0600	Bisphosphonate	Administered early to avoid interference of food and other medications with absorption
Frequency	Fosamax 70 mg not specified as once weekly	Bisphosphonate	Potential for overdose
	Ural q5*, q4, q3 h instead of q8 h	Urinary alkaliniser	Maximum dose is up to 4 times daily PRN
	Endone PO 5 mg PRN max 10 mg/day prescribed as q1 min PRN up to 2 doses per day	Opioid analgesic	Not rational
Cessation error	trimethoprim not ceased	Antibiotic	Drug duplication, potential interaction when ciprofloxacin is prescribed
	ranitidine not ceased	Histamine H2-receptor antagonist	Drug duplication when omeprazole is prescribed; irrational therapy
Medical omission	ciprofloxacin not added	Antibiotic	Likely serious consequences
	Ural not added	Uurinary alkaliniser	Comfort; significant
	Endone not added	Opioid analgesic	Inadequate pain relief
	omeprazole not added	Proton pump inhibitor	Symptoms not relieved and potentially dangerous—risk of bleeding.
Date	Fosamax PO 70 mg once/week 0600 starting today rather than 1 week from today	Bisphosphonate	Possible adverse drug reactions
PRN max dose omission	Max dose for Ural not specified	Urinary alkaliniser	Maximum dose is up to 4 times daily

^*Every 5 hours.

Discussion

Main findings and implications

Impact of interruptions

Any impact of interruptions on prescribing errors was not detected in our experiment. One possible reason is that the effect of interruptions on error rate may have been minimized by cues available within the user interface of the CPOE, which may have aided resumption of interrupted tasks. Moreover, the laboratory setting is not a true reflection of cognitive loads in the busy work environment where interruptions have been shown to disrupt other types of clinical tasks resulting in procedural failures and clinical errors.¹⁶ In our experiment, upon task resumption participants had ample time to recall their next action and the task environment provided cues to aid task resumption. This is consistent with observations from other studies of interruptions to computer-based tasks where participants were aided by the screen environment and did not fail to resume an interrupted task.^{20 29} Performance under cognitive load from more demanding competing tasks in a clinical environment may have resulted in a different outcome. Furthermore, interruptions are a complex phenomenon where multiple variables, including the characteristics of primary tasks, the interruptions themselves, and the environment, may influence impact on clinical tasks and errors.³⁰ As interruptions seem to be inherent to clinical work and cannot completely be eliminated it is necessary to understand the circumstances under which interruptions are likely to be dangerous and to design systems that are interruption-tolerant.

While the size of our sample was adequate to detect a difference in resumption lag, the study was not adequately powered to examine the smaller differences in prescribing errors mainly because the error rates we measured were much lower than the initial estimates we used for sample size calculations. Given that total error rates for our prescribing tasks ranged from 0.92–2.59% (N=1088 to N=1120) (table 3), error rate may not be a sensitive enough measure to detect the disruptive effect of interruptions compared to the resumption lag, which has been reliably measured in other laboratory studies.³¹ Observed power to examine the effects of interruption and task complexity and the interaction effect of interruption × task complexity on error rate was less than 0.80, which suggests insufficient power of the experiment rather than absence of an effect of interruption on error rate. Further experiments that are adequately powered to detect small differences in error rate are required to rule out any effect interruption might have on CPOE error rates.

Interruptions did not have any reliable effect on the overall task completion times of simple or complex tasks. A possible explanation is that overall task completion time is a coarse measure and participants could have developed strategies (eg, speeding up task execution) to compensate for the disruptive effect of interruptions throughout the task execution period.³² As a consequence, any disruptive effect of interruption may not have been sufficiently large to be reflected in the overall task completion time. As with error rate, the observed power for the effects of interruption and the interaction effect of interruption × task complexity on task completion time was less than 0.80, which suggests insufficient power of the experiment rather than absence of an effect of interruption on task completion time.

After interruption, complex tasks required almost three times longer to resume compared to simple tasks (resumption lag_complex=9.6 seconds, resumption lag_simple=3.4 seconds). This finding is consistent with Burmistrov and Leonova's ¹⁹ study showing a disruptive effect on complex cognitive operations in a computer-based editing task. However, some caution is needed in interpreting this result as a disparity in the task resumption environment between simple and complex prescribing tasks may have exaggerated the magnitude of the resumption lag. During resumption the computer screen for the simple tasks did not list any medications and the next correct action was to begin prescribing by clicking the ‘Prescribe’ button (Appendix C available as an online data supplement at http://jamia.bmj.com and figure 1). In comparison, the resumption screen for complex task had six existing medications and the next action could relate to any of the six medications (Appendix C: figure 1). Thus the complex resumption screen provided more competing information cues than the simple resumption screen and may have caused participants to spend more time differentiating among competing cues in order to recall their next action. This notion of making the right memory associations with environmental cues upon task resumption is well founded in the activation-based goal memory model,³³ which has been widely used in the study of cognitive processes in interruption.^{24 34 35} Better measurement of the context surrounding interruption may be necessary in future studies.

Types of prescribing errors

We have quantified the types of prescribing errors when using a CPOE system in a controlled setting. Most prescribing errors we observed could be described as slips in using the CPOE system to create and update electronic medication charts, including failure to enter allergy information; selection of incorrect medication, dose, route, formulation, or frequency of administration from lists and drop-down menus presented by the CPOE; incorrect entry or omission in entering administration times, start date, and free-text qualifiers; and omissions in prescribing and ceasing medications. Such slips are errors in the execution or storage phase in an action sequence and they occur with the correct intention from users.^{36 37} Prescribing errors resulting from slips occur not because of deficiency in the clinician's knowledge about prescribing or use of the CPOE but because of human fallibility in executing routine procedures despite having the correct how-to knowledge.

Other CPOE errors in specifying administration times involved changes to default times specified by the system. This highlights a need to train users to double-check administration times for medications such as diuretics and angiotensin-converting enzyme (ACE) inhibitors, which may at times be exceptions to standard times specified in the national inpatient medication chart upon which CPOE default times were based (scenario A: error rate=2.8%, N=320; scenario B: error rate=3.8%, N=416). Specification of PRN medications (error rate=6%, N=64) and weekly medications (error rate=4.7%, N=128) was particularly problematic underlining a need to improve user skills in minimizing mistakes in prescribing these medications.

The consequences of specific types of prescribing errors may be more important than others. For example, the top two error types related to missed allergies and omission of free-text qualifiers. The potential significance of missing allergy information in 10 out of 64 new medication charts is likely to be greater than missing directions for administration in 4 out of 32 prescriptions for a topical medication. The absolute error rate for most other types of prescribing errors ranged from 0.5% (N=192) to 7.8% (N=64), which may have significant consequences for medications such as cytotoxics. Moreover, the impact of prescribing errors could be magnified due to a greater number of opportunities for making such errors in a hospital setting.

Evaluating safe designs

Our method used hypothetical clinical scenarios in a controlled setting. Such an approach provides a quantitative basis to compare different CPOE systems and the design of corrective or preventive strategies to minimize errors. Some of the errors caused by slips can be prevented through improved user interface design and user training. Indeed, two of the problems identified in this experiment have been rectified in the new version of the CPOE deployed at the hospital. Missed allergies that now appear in red text are much more likely to be noticed by a clinician. A monthly calendar has also been introduced to simplify specification of weekly doses.

Comparison with the literature

Few studies have examined the impact of interruptions in healthcare. Task disparity makes direct comparison difficult with other domains, such as aviation, where the effects of interruption have been traditionally studied. The hypotheses (H1, H2) of the current study were not supported despite being derived from studies in the interruption literature. One of the main reasons is that experimental studies have examined the effects of interruption in a range of different tasks. Our hypotheses were based on studies from aviation,¹⁸ production management,^{20 21} and computer-based editing,¹⁹ which are different to prescribing using a CPOE. The difference in task nature might have contributed to the different findings. However, H3 is supported and H4 is partially supported and can be related to existing studies. We found that complex tasks took significantly longer to resume and this finding is consistent with Burmistrov and Leonova¹⁹ who found that interrupting cognitively complex operations in a computer-based editing task resulted in longer resumption lags than interrupting simpler cognitive operations.

Our findings about the mechanisms for CPOE-related error are comparable to other studies of online clinical information systems. As identified in a previous study of online decision support systems using hypothetical scenarios,³⁸ we found that slips in using the CPOE were responsible for the majority of the prescribing errors we identified. The types of errors we have identified are consistent with the mechanisms of prescribing errors found in evaluations of CPOE in clinical settings, including missed drug allergies⁷; incorrect medication, dose, route, formulation, administration time or frequency^{4 8}; and incorrect entry or omission of start date⁴ and free-text qualifiers needed to administer drugs.^{8 9}

Limitations

There are several limitations to the design of the current study. First, the laboratory environment may not be representative of the setting in which a CPOE may be generally used for electronic prescribing. Within a laboratory setting there is a trade-off between obtaining causal relationships among controlled variables and ensuring ecological validity. Therefore, it is possible that participants devoted more time to completing the tasks. Second, the interruptions in our study also related to use of the CPOE and may not have had the same impact of typical interruptions in a clinical setting, which may require a clinician to walk away from their primary task. Third, we used simulated prescribing scenarios for creating and updating electronic medication charts for a specific CPOE system at one teaching hospital attached to the university. Therefore, our results may not be generalizeable to other CPOE systems, which may have a different user interface and use model. Finally, the 32 participants in this study (mean age 27.5 years) were recruited from a cohort of 80 junior doctors and residents employed by the same hospital and may not be representative of other populations of doctors. The prescribing tasks examined are primarily undertaken by this group of doctors as part of their routine clinical duties; therefore, we recruited our participants from the same cohort to minimize bias. As all junior doctors and residents are required to complete a formal CPOE training session conducted by the hospital and routinely use the CPOE system, we would expect the skills of the current participants to be comparable to those of non-participants but this was not measured.

Conclusion

Most electronic prescribing errors found in this study could be described as slips in using the CPOE system to create and update electronic medication charts. Laboratory-based evaluation to measure the types of prescribing error using hypothetical clinical scenarios provides a quantitative basis to compare different CPOE systems and the design of corrective or preventive strategies to minimize errors. Any impact of interruptions on prescribing errors and task completion was not detected in our experiment as the study was underpowered. Cues available within the user interface may have aided resumption of interrupted tasks making CPOE systems robust to some interruption effects. Further experiments that are adequately powered to detect small differences in error rate are needed to rule out any effect interruption might have on CPOE error rates. It is also necessary to understand the circumstances under which interruptions to clinical tasks are likely to be dangerous.