The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals

Abstract

Objectives

To compare the manifestations, mechanisms, and rates of system-related errors associated with two electronic prescribing systems (e-PS). To determine if the rate of system-related prescribing errors is greater than the rate of errors prevented.

Methods

Audit of 629 inpatient admissions at two hospitals in Sydney, Australia using the CSC MedChart and Cerner Millennium e-PS. System related errors were classified by manifestation (eg, wrong dose), mechanism, and severity. A mechanism typology comprised errors made: selecting items from drop-down menus; constructing orders; editing orders; or failing to complete new e-PS tasks. Proportions and rates of errors by manifestation, mechanism, and e-PS were calculated.

Results

42.4% (n=493) of 1164 prescribing errors were system-related (78/100 admissions). This result did not differ by e-PS (MedChart 42.6% (95% CI 39.1 to 46.1); Cerner 41.9% (37.1 to 46.8)). For 13.4% (n=66) of system-related errors there was evidence that the error was detected prior to study audit. 27.4% (n=135) of system-related errors manifested as timing errors and 22.5% (n=111) wrong drug strength errors. Selection errors accounted for 43.4% (34.2/100 admissions), editing errors 21.1% (16.5/100 admissions), and failure to complete new e-PS tasks 32.0% (32.0/100 admissions). MedChart generated more selection errors (OR=4.17; p=0.00002) but fewer new task failures (OR=0.37; p=0.003) relative to the Cerner e-PS. The two systems prevented significantly more errors than they generated (220/100 admissions (95% CI 180 to 261) vs 78 (95% CI 66 to 91)).

Conclusions

System-related errors are frequent, yet few are detected. e-PS require new tasks of prescribers, creating additional cognitive load and error opportunities. Dual classification, by manifestation and mechanism, allowed identification of design features which increase risk and potential solutions. e-PS designs with fewer drop-down menu selections may reduce error risk.

Background and significance

Clinical information systems will facilitate the phasing out of paper-based medication ordering systems over the next decade, raising critical questions for all healthcare professionals, patients, and their carers. To ensure that the inevitable transition to electronic prescribing systems (e-PS) leads to effective new procedures and safe patient outcomes, a clear understanding of any unintended consequences of such change is increasingly important. We need to identify, quantify, and analyze new types of prescribing errors associated with new prescribing practices. We also need to investigate the relative performance of different e-PS in terms of the number and type of prescribing errors associated with their use and, consequently, how user-training and/or design modifications might help mitigate risk. Ultimately, many clinicians may seek to weigh new risks against the intended clinical, administrative, and cost benefits of implementing e-PS. Given the magnitude of the changes facing healthcare systems as clinical information systems are rolled out, the potential for anecdotal evidence and personal experiences to confuse the e-PS debate is significant, further highlighting the need for a robust new knowledge base.

It is now well recognized that clinical information systems have unintended consequences.^1–5 Reports such as the US Institute of Medicine's ‘Health IT and Patient Safety: Building Safer Systems for Better Care’⁶ highlight the challenges. Early studies provide narrative accounts of e-PS-related errors and broad classifications have been proposed.^{1 7} Unintended consequences resulting in errors have been variously labeled ‘system-related,’¹ ‘technology-induced,’⁸ and ‘computer-related.’⁹ A frequent subgroup is system-related errors arising from the use and functionality of e-PS which would be unlikely or unable to occur in paper-based medication ordering systems.^{9 10} Evidence of the nature, extent, and consequences of system-related errors associated with e-PS use is emerging. A systematic review of e-PS effectiveness to reduce prescribing errors showed that of 12 studies between 1998 and 2007, six made mention of new error types.¹¹ These included incorrect selection of order components from drop-down menus, duplicate ordering, and a failure to discontinue drugs due to poor information display. However, there was great variability in the ways these error types were described and limited quantification of their occurrence.

A robust classification of e-PS system-related errors has not yet been developed. Nor has any study systematically applied dual classification to capture both the manifestations and mechanisms of prescribing errors using e-PS to improve our understanding of the relationship between these key dimensions. The current evidence base is not sufficient to alert users or system designers to specific features associated with system-related errors.⁵ For example, there has been no previous study to assess whether e-PS designs which encourage users to select from detailed predefined options versus designs which encourage users to enter information from scratch, reduce error opportunities or result in different types of system-related errors. Such questions are fundamental to designing safer systems. Our aims were to compare the rates, manifestations, and underlying mechanisms associated with system-related prescribing errors arising from the use of two commercial e-PS, and to determine if the rate of system-related errors is greater than the rate of prescribing errors prevented by these e-PS.

Materials and methods

Sample and setting

A clinical audit of medications ordered during 629 inpatient admissions was undertaken at two major teaching hospitals in Sydney, Australia as part of a larger study to assess the effectiveness of e-PS in reducing prescribing errors.¹⁰ Hospital A has 400 beds and Hospital B 326. Each hospital had integrated a different commercial e-PS into their computerized order entry system at least 6 months prior to data collection. Patient charts from three wards: geriatrics (medical and surgical), psychiatry, and cardiology (medical and surgical) were reviewed. Physicians directly enter their medication orders into the e-PS. At each site hospital pharmacists visit wards to review patient charts, and will contact prescribers if they detect prescribing errors. Pharmacists only visit wards on weekdays during business hours, thus not all orders in the hospitals are reviewed by pharmacists. Medication orders do not have to be reviewed by pharmacists prior to medication administration.

Electronic prescribing systems

Cerner Millennium PowerOrders

Prescribing using the Cerner Millennium PowerOrders system predominantly involves prescribers selecting items from a drop-down menu of predefined order sentences triggered on drug selection, which can be edited (figure 1). Predefined order sentences contain details of the drug, strength, dose, and form, for example ‘Paracetamol-codeine 500 mg 8 mg tab, sol.’ ‘Care sets’ allow for a group of related orders to be selected with a single click. Unlisted medications and comments are generated by the prescriber.

Figure 1.

Screenshots of Cerner and MedChart electronic prescribing systems.

CSC MedChart

The prescribing process using CSC MedChart can be performed in three ways: (1) long-hand, where all individual items of prescribing information (eg, dose, route, etc) are entered (via drop-down menus or free text) following the selection of a drug product (figure 1); (2) by using pre-defined order sentences containing all prescription parameters, searchable by category (quick lists) following drug product selection; and (3) by using pre-written order-sets (protocols) allowing for grouped, multiple, pre-written order sentences to be selected with a single click. The majority of clinicians at this site used long-hand prescribing.^{12 13}

While both e-PS provided the capability to select predefined order sentences or to prescribe long-hand (where the user constructed the order sentence from scratch), a central difference was that the Cerner design encouraged prescribers to select from predefined, detailed order sentences, while the MedChart design resulted in a greater emphasis on long-hand prescribing.

In the Cerner system heparin infusions and patient-controlled analgesia remained on paper charts. At the MedChart site, a greater proportion of orders remained on paper charts including: intravenous (IV) fluids, variable dose regimes (such as reducing doses), insulins, oral anticoagulants (warfarin), and epidural or patient-controlled analgesia. Prescribers were required to order an electronic reminder to signal the administration times for these drugs, but the actual orders were located on a paper chart.

Procedures

A daily medication chart audit of all inpatients’ records was performed by three clinical pharmacists independent of the hospitals. When the study pharmacists reviewed medication charts they recorded if there was some indication in the record that an error had been previously identified or corrected, for example, if a hospital pharmacist had made a correction to an order. All such corrected prescribing errors were counted as errors. Data collection occurred on average 18 weeks following e-PS implementation at each site.¹⁰ Inter-rater reliability tests were conducted at multiple times and produced κ scores of 0.82–0.84 for manifestation error classification, indicating high levels of consistency. Severity scores were subject to the following scrutiny. Two pharmacists independently rated potential severity and used consensus approaches when disagreements arose. All serious errors were reviewed by our severity review teams which included nurses, hospital pharmacists, an emergency physician, and clinical pharmacologist, to determine final classification. A random sample of the non-serious prescribing errors was also reviewed in this way. The study was approved by the Human Research Ethics Committees of the University of Sydney and both hospitals.

Error classifications

All prescribing errors were initially classified into procedural or clinical errors and potential error severity was rated on a 5-point scale. Definitions applied have been reported previously.¹⁰ Every prescribing error was further reviewed to assess evidence that the underlying mechanism was ‘system-related,’ defined as ‘Errors where there was a high probability that the functionality or design of the e-PS contributed to the error and there was little possibility that another cause, such as lack of knowledge about the drug, produced the error.’ For example, an order which contained a drug unrelated to the patient's condition, but which was located on a drop-down menu directly above or below a likely drug selection for that patient, was classified as a system-related error. Mechanism classification development initially involved all system-related errors being reviewed by two pharmacists independently. This process sometimes involved attempts to re-create the error in the e-PS to understand the steps in the process which may have led to the error. As new mechanisms were identified these were added to the classification. This process started to reveal the most frequent types of mechanisms leading to system-related errors and definitions for each mechanism type were developed.

The research team, which included a clinical pharmacologist and human factors psychologist, worked with the clinical audit pharmacists to continually refine the classification by reviewing error mechanism categories in relation to specific errors identified. As the classification was refined, this at times required us to go back and re-classify errors to ensure consistency. We started initially with a large number of mechanism categories and gradually collapsed these into meaningful groups. Hospital pharmacists at each of the hospital sites were also used as consultants to confirm our assessments of the system mechanisms. The hospital review committees involved a multi-disciplinary team (eg, nurses, IT staff, representatives from the IT e-PS vendor, doctors). Each hospital review committee reviewed the system-related error categories and specific examples classified under each category. This process was also designed to identify potential remediation strategies. Both hospital committees had the authority to implement recommended changes at the two sites. Thus the classification development process was very iterative and developed using a grounded process from the data. Online supplementary appendix 1 reports the mechanism classification.

Analyses

We calculated the overall and hospital-specific prescribing and system-related error rates per admission and per 100 patient days. Two-sample t tests compared differences between e-PS. The Satterthwaite approximation was adopted when the variances of two samples were not equal. Descriptive analyses of system-related errors by manifestations and mechanisms were conducted. We calculated proportions and rates per 100 admissions of specific categories of system-related errors for each e-PS. Odds ratios (ORs) were used to indicate the association between the manifestation/error mechanism and each e-PS. The OR of a specific error type/mechanism was given as the odds of error occurrence using the MedChart e-PS in ratio to the odds of error occurrence in the Cerner e-PS. The 0.5 values were added as a continuity correction to improve the approximation given zero or small numbers in some categories. Log OR tests were applied to test the significance of association. To adjust for multiple comparisons, the Holm's procedure¹⁴ was adopted. The level of significance was set at 5%. Analyses were performed using SAS.¹⁵ We compared the rates at which these two e-PS reduced prescribing errors, published in our previous study¹⁰ at these same hospitals, to determine if the e-Ps prevented more prescribing errors than they generated.

Results

Of 629 admissions reviewed, 1164 prescribing errors were identified (185/100 admissions). Most were clinical errors (86.5%; 160/100 admissions, n=1007) and 13.5% were procedural (25/100 admissions, n=157). All system-related errors manifested as clinical errors and constituted 42.4% (78/100 admissions, n=493) of total prescribing errors (table 1). Of the 493 system-related prescribing errors identified, 66 (13.4%) were detected by ward staff prior to record audit. Hospital pharmacists identified 34, with other staff identifying the remainder.

Table 1.

Total and system-related prescribing error rates by site

Electronic prescribing system (number of admissions reviewed; total patient days)	Prescribing errors per 100 admissions (95% CI) Errors per 100 patient days; n=number of errors	System-related errors per 100 admissions (95% CI) System-related errors per 100 patient days; n=number of system-related errors	% Of prescribing errors which were system-related (95% CI)
CSC MedChart (n=465 admissions; 4399 patient days)	166 (136 to 195) 17.50; n=770	71 (56 to 85) 7.46; n=328	42.60 (39.10 to 46.09)
Cerner Millennium (n=164 admissions; 2014 patient days)	240 (195 to 286) 19.56; n=394	101 (76 to 125) 8.19; n=165	41.88 (37.10 to 46.75)

The overall proportions of system-related errors were similar at both hospitals (table 1). However, the Cerner site had a higher system-related error rate per 100 admissions (p=0.0036), consistent with the higher overall prescribing error rate at that hospital (p=0.009) (table 1).

System-related prescribing errors by clinical manifestation and severity

For both e-PS the majority of system-related errors manifested as timing errors (n=135) (table 2). The manifestation of system-related errors by e-PS was similar with one exception. Wrong strength errors occurred 13 times (OR=13.34) more frequently with the MedChart system compared to the Cerner system (table 2).

Table 2.

System-related errors by clinical error type and electronic prescribing system

Manifestation	MedChart			Cerner
Clinical error type	n	Per 100 admissions	% Of errors	n	Per 100 admissions	% Of errors	OR†	p Value
Wrong timing	89	19.14	27.1	46	28.05	27.9	0.97	0.9
Wrong strength	106	22.80	32.3	5	3.05	3.0	13.34	0.00001*
Wrong formulation	31	6.67	9.5	11	6.71	6.7	1.46	0.5
Prompt not ordered	28	6.02	8.5	13	7.93	7.9	1.11	0.5
Wrong rate/frequency	25	5.38	7.6	8	4.88	4.8	1.60	0.4
Wrong ancillary information	17	3.66	5.2	11	6.71	6.7	0.81	0.7
Wrong route	1	0.22	0.3	27	16.46	16.4	0.05	0.01
Wrong drug	10	2.15	3.0	14	8.54	8.5	0.39	0.2
Wrong dose unit	14	3.01	4.3	4	2.44	2.4	1.73	0.5
Not indicated	5	1.08	1.5	11	6.71	6.7	0.29	0.2
Wrong dose	0	0	0	11	6.71	6.7	0.09	0.1
Incomplete order	1	0.22	0.3	4	2.44	2.4	0.34	0.4
Duplicated order	1	0.22	0.3	0	0	0	2.04	0.7
Total	328	70.54	100.0	165	100.61	100.0

*Significant after adjustment for multiple comparisons using Holm's procedure, p<0.004.

†OR of errors per 100 admissions, which was defined as the odds of occurrence of a specific error type among all system-related errors using MedChart in ratio to the odds of this error type occurrence using Cerner after adjusting for the patient admissions at the two sites.

Only 11 system-related errors were rated as potentially serious (a score of ≥3 on the 5-point scale). Seven occurred at the Cerner site, representing 3% of system-related errors at that site (4.3/100 admissions (95% CI 0.7 to 7.8)). Four serious errors (1.2%) were identified at the MedChart site (0.9/100 admissions (95% CI 0 to 1.9)). There was no significant difference in serious errors at the two sites (p=0.07). Descriptions of some of these serious errors are listed in table 3. Of the 66 system-related errors detected in the normal course of care, three were rated as serious. The remaining eight serious system-related errors were identified by the study pharmacists.

Table 3.

Examples of serious (severity rating ≥3) system-related errors

Error manifestation category	Error description	Error mechanism	Potential severity rating	Who detected the error
Wrong route	Sodium chloride was ordered as an epidural. The correct route was intravenous infusion. The prescriber selected an order sentence and then edited this sentence and incorrectly selected the route.	Selection error (when editing an order sentence)	3 (moderate)	Nurse
Wrong route	Pantoprazole was ordered to be given intra-articularly. The correct route was an intravenous infusion. The prescriber was constructing an order and in this process selected the incorrect route.	Selection error (when constructing an order sentence)	3 (moderate)	Research pharmacist
Drug not indicated	Warfarin and warfarin target range orders are made in the e-PS. Instead of ceasing an order for warfarin, the prescriber ceased the order for warfarin target range in error. Warfarin dose was given to patient.	Error associated with ancillary information	4 (major)	Research pharmacist
Wrong route	Salbutamol was ordered as an intravenous injection. The correct route was an inhalation. The prescriber selected an order sentence and then proceeded to edit this sentence and made an error in the route.	Selection (when editing an order sentence)	3 (moderate)	Doctor

e-PS, electronic prescribing systems.

Mechanisms of system-related errors

System-related errors were classified according to five broad underlying mechanisms listed in table 4.

Table 4.

Frequency and rates of system-related errors per 100 admissions by mechanism and e-PS

	MedChart e-PS	Cerner e-PS	Total	Comparison between e-PS
Mechanism	Rate/100 admissions (number of errors)	Rate/100 admissions (number of errors)	Rate/100 admissions	OR†	p Value
Selection errors			34.2	4.17	0.00002*
Selection error when ordering, constructing, or editing an order sentence	38.3 (n=178)	22.0 (n=36)
Construction errors			2.7	0.78	0.75254
Error in construction of an order sentence; excludes errors in selection during construction	2.2 (n=10)	4.3 (n=7)
Editing errors			16.5	0.63	0.21465
Error when editing an existing order sentence; excludes selection errors	1.1 (n=5)	9.8 (n=16)
Editing to correct a selection error; excludes selection errors	11.8 (n=55)	17.1(n=28)
New tasks required as a result of e-PS			32.0	0.37	0.00301**
Failure to order a reminder	6.0 (n=28)	12.8 (n=21)
Failure to change default time/date	3.2 (n=15)	17.1 (n=28)
Errors associated with ancillary information	3.0 (n=14)	6.7 (n=11)
System limitation	6.0 (n=23)	6.1 (n=10)
e-PS contains incorrect order sentence	0	4.9 (n=8)
Total errors	70.5 (n=328)	100.6 (n=165)	78.5

*Significant after adjustment for multiple comparisons using Holm's procedure, p<0.0125.

**Significant after adjustment for multiple comparisons using Holm's procedure, p<0.017.

†OR of errors per 100 admissions, which was defined as the odds of error occurrence with a specific mechanism in MedChart in ratio to the odds of error occurrence with this mechanism in Cerner after adjusting for the patient admissions in two sites.

e-PS, electronic prescribing systems.

Selection errors

Selection errors occurred when prescribers made an incorrect selection from a drop-down menu. They were the most frequent mechanism of system-related errors accounting for 43.4% (n=214, 34/100 admissions). Selection errors were four times more frequent with MedChart compared to Cerner (table 4). Selection errors manifested as a range of clinical error types. Figure 2 illustrates a selection error resulting in a wrong drug error. The majority (59.6%, n=106) of MedChart selection errors resulted in wrong strength errors where the dose unit selected was inappropriate for the desired dose (see online supplementary appendix 2). In some instances the dose strength selected was too small, for example, a prescriber selected cyclosporine 50 mg capsule for a patient whose dose was specified as 75 mg. In other cases the dose strength was too large, for example an order of prednisolone 5 mg tab for a desired dose of 4 mg. In further instances the dose required a combination of strengths, for example, oxycodone 80 mg tab for a desired dose of 50 mg (requires a 20 mg+30 mg).

Figure 2.

Example of a selection error (mechanism) made while ordering resulting in a wrong drug error (manifestation). The prescriber selected methylprednisolone acetate for intravenous (IV) administration instead of methylprednisolone sodium succinate (the correct salt for IV use). Methylprednisolone acetate is an oily solution for intra-articular or intramuscular (IM) use, and must NOT be injected intravenously. A warning alert that the two salts of methylprednisolone are available and must be selected correctly appeared not to have been noticed by the prescriber who had to override the alert to continue prescribing.

In the Cerner e-PS, selection errors most frequently resulted in wrong route errors (44.4%, n=16) (see online supplementary appendix 2). Figure 3 illustrates where a prescriber sought to modify an order sentence and mistakenly selected the epidural route.

Figure 3.

Selection error when editing (mechanism) a predefined order sentence resulting in a wrong route error (manifestation). The prescriber modified a predefined order sentence for sodium chloride 0.9% IV infusion, and mistakenly selected the epidural route from the dropdown menu of available routes.

Editing errors

Editing errors accounted for 21.1% of system-related errors and occurred when prescribers modified a predefined order sentence. Editing error rates did not significantly differ by site (table 4). Figure 4 shows an editing error resulting in four clinical errors. In the MedChart system, 98% (n=59) of editing errors resulted in timing errors. In the Cerner system, editing was associated with a range of clinical error types, most frequently wrong route and wrong formulation errors (see online supplementary appendix 2).

Figure 4.

Error when editing (mechanism) a predefined order sentence resulting in four clinical errors: wrong formulation, wrong dose, wrong route, wrong frequency. The prescriber wanted to order nystatin oral drops 1 ml, oral, qid (four times daily), but chose an order sentence with the wrong formulation (the topical cream order sentence sat at the top of a long menu), then modified this order sentence. The resultant order (for 10 ml of the topical cream to be applied every 4 days) contains four errors.

Construction errors

Construction errors occurred when a prescriber formulated a new order sentence and an element was incorrect. For example, a glyceryl trinitrate patch 10 mg/24 h, 1 patch mane (morning), with an order comment ‘50 mg/24 h.’ Construction errors occurred at low but similar frequency at both sites.

New tasks required by the e-PS

We identified four types of tasks that the e-PS required of prescribers not needed with paper charts. In general, these additional tasks were due to limited e-PS functionality and constituted workarounds in response to those limitations. Together these occurred at a rate of 32/100 admissions. Errors in this category were significantly more frequent with Cerner compared to MedChart (table 4).

Failure to order a reminder occurred when a prescriber failed to provide ancillary information required, for example, to provide a reminder that a patient had a dermal patch that required removal. At the time of the study, neither e-PS had an efficient way of recording times for both the administration and removal of dermal patches and each site devised different workarounds.

Failure to correctly change default dates and times was another type of error that would not occur in a paper-based system. These occurred at a higher rate in Cerner (17.1 vs 3.2/100 admissions). For example, the administration time for a dose ordered daily was linked to a first administration default time of 8:00. Therefore, if an antibiotic was ordered at 15:00, the first dose would be scheduled to occur at 8:00 the next day. Thus if the default time was not changed by the prescriber, there was a potential risk of a missed or extra dose being administered.

Errors associated with ancillary information occurred when narrative information provided by the prescriber was not consistent with information in the structured order, for example an order ‘aspirin 100 mg enteric coating tablet’ with instructions ‘dissolve in water.’ This situation also arose when orders were ceased or changed but ancillary information was not altered. This had the potential to lead to confusion. We also identified eight cases at the Cerner site where a predefined order sentence contained an error (table 4).

Comparison of prescribing errors prevented versus those generated by e-PS

In our published, controlled before and after e-PS study,¹⁰ at these two hospitals we found that prescribing errors at Hospital A declined from 625 errors per 100 admissions (95% CI 523 to 728) to 212 (95% CI 171 to 254; p < 0.0001) after the Cerner system was implemented, and at Hospital B from 362 (95% CI 330 to 393) to 146 (95% CI 120 to 173; p<0.0001) after the MedChart system was implemented. Overall, the prescribing errors dropped from 406 per 100 admissions (95% CI 374 to 437) to 185 (95% CI 160 to 210) after the implementation of e-PS. Thus e-PS introduction was associated with a significant overall reduction of 220 prescribing errors per 100 admissions (95% CI 180 to 261; p<0.0001). This result reflects both the prescribing errors the e-PS were effective at preventing plus the new ‘additional’ prescribing errors that they created. The e-PS were overall effective at reducing traditional prescribing errors at a rate of 299/100 admissions (95% CI 263 to 334). However, this effect was diluted with the introduction of new system-related errors which occurred at a rate of 78/100 admissions (95% CI 66 to 91). Thus the e-PS resulted in a net benefit reduction of 220 prescribing errors per 100 admissions.

Discussion

Our research allows us to go some way to begin answering two fundamental questions about e-PS which have not been adequately explored. Firstly, what is the nature and extent of new error types associated with the introduction of e-PS and, secondly, do e-PS prevent more prescribing errors than they generate?

We found system-related errors are frequent, comprising 42% of all prescribing errors (78 per 100 inpatient admissions), at two hospitals with different e-PS. However, only 2.2% were serious errors. A unique aspect of our study is that we are able to show that both e-PS prevented many more prescribing errors than they created at the study hospitals, providing some confidence of the overall benefit of e-PS for patient safety. However, system-related errors significantly reduced the overall effectiveness of e-PS. The high volume of system-related errors adds additional burden on staff to be vigilant, and corrections and clarifications required as a result of these errors may impact the efficiency of the medication process, particularly for nursing and pharmacy staff. Thus the identification and remediation of these errors is central to making e-PS safer and more effective and efficient.

Our findings demonstrate that system-related errors were most frequent when prescribers needed to select information from drop-down menus, edit information within the system, and perform new tasks not previously required. The two e-PS differed in their designs and reliance upon selection and editing of information. For both, error in selection was the most frequent mechanism, associated with 43% of system-related errors. Others have also reported drop-down menu selection as a cause of error^{9 16–18}; our results quantify this conclusion. The MedChart e-PS design, which provided greater opportunities for selection errors, generated these at a rate four times higher than the Cerner e-PS. This suggests that e-PS designed with fewer opportunities for selection errors, that is, by guiding users to select from pre-defined order sentences, may be beneficial. Errors in editing accounted for 21% of system-related errors and did not significantly differ between systems. However, the outcomes of these errors were considerably different. All editing errors using MedChart (except for one) resulted in wrong timing errors, most due to errors when changing default times. Editing errors using Cerner produced a range of clinical errors, most frequently wrong route, formulation, and dose errors. Thus in most instances prescribers using Cerner made errors when editing predefined order sentences, while the MedChart users made editing errors predominantly when changing the default times. The low use of predefined order sentences in MedChart most likely explains the differences in outcomes of the editing errors identified. We found eight predefined order sentences at the Cerner site contained an error. Given the thousands of order sentences which had been written for the system, this represents a small error rate. However, it demonstrates that despite a rigorous internal process of hospital review, errors are possible at this stage and may be difficult to detect. This issue has rarely been raised in papers related to the design and implementation of e-PS.

There are few comparative studies. Walsh et al⁹ applied a similar system-related error definition in a study in a US pediatric hospital but identified only 20 system-related errors (19% of all prescribing errors). However, 35% (n=7) were classed as serious. Our results from two adult teaching hospitals suggest a substantially greater frequency of system-related errors but a smaller percentage of serious errors (2.2%).

Both e-PS generated new tasks for prescribers which have to date received limited attention. Failure to complete these tasks accounted for 32% of all system-related errors. Some of these new tasks related to the limited functionality of the e-PS which resulted in hospitals running hybrid systems where, for example, more complex medications continued to be managed on paper-charts and thus prescribers had to leave alerts within the electronic system. A review of electronic prescribing in the UK National Health Service (NHS) reported that this is a common practice with most UK hospitals found to have taken this approach for variable dose medicines such as warfarin.¹⁹ Thus hospitals intending to implement hybrid systems (eg, with some orders electronic and some on paper) in order to compensate for limitations in system functionality should be aware of the additional risks that this may generate. The inability in the study e-PS to order both placement and removal of patches resulted in workarounds which were not always consistently applied. Failure to pay attention to default times within the e-PS also resulted in many errors and raises the question as to the benefits and risks of using default times.

We found inconsistencies between narrative text and structured order information. Such errors have been identified by others.^20–22 For example, Palchuk et al²¹ reported 16% of 2194 ambulatory orders contained inconsistency errors, 17% of which were rated as potentially severe adverse drug events. Interestingly, Palchuk et al²¹ identified that nearly one third of inconsistency errors were due to prescribers using free-text instructions to request a dose that varied over the day as well as dose escalations or tapers. This suggests their e-PS had limited functionality to support such orders. At the MedChart site it was decided that these types of orders remain on paper until improved e-PS functionality was available, whereas the Cerner site used the e-PS for these orders. This may have contributed to the significantly higher rate of these types of errors at the Cerner site.

Wiener²³ labeled this situation, where automation leads to additional workload and co-ordination demands on users, as ‘clumsy automation.’ The new tasks required by e-PS increase the cognitive load for prescribers.^{18 24} For example, the requirement for prescribers to consider whether default dates and times need to be changed, or to remember to order a reminder if a patient has a paper medication chart, all add to prescribers’ cognitive load and create new opportunities for error. As Sarter et al²⁴ describe, the introduction of complex automation requires users to acquire new skills and ‘to understand input-output relationships to be able to anticipate effects of their own entries’ (p 4). The new e-PS tasks provide examples where prescribers need to not only learn how to use the e-PS but also to understand the implications of using the system in specific ways.²⁴ In the aviation industry the introduction of more sophisticated systems has prompted calls for changes in pilot training, helping users to develop mental models of the way in which systems work.²⁴ As e-PS become more sophisticated it seems likely that similar changes in training will be required to support effective and safe system use.

The study highlights the importance of closely monitoring e-PS outcomes. The majority of errors in this study would have gone undetected without a specific focus on their identification and quantification. This was confirmed by our finding that for only 13% of system-related errors was there some indication in patients’ records that the error had been detected prior to the study audit. While systems for reporting safety issues related to health IT have been advocated,^{6 25} these can only be successful if users are aware of, and report, errors. Studies of adverse event reports demonstrate few system-related errors are reported.²⁶ Thus effective monitoring systems are required, particularly as systems continue to evolve with software upgrades and customization. Doctors who use these systems on a day-to-day basis should be aware of design features and tasks associated with increased risk of generating system-related prescribing errors.

Strengths and limitations

A strength of our study is the systematic review of all medication orders generated by two e-PS to identify system-related prescribing errors and the subsequent dual classification developed and applied. This is an advance over previous attempts to quantify system-related errors which have largely relied upon descriptive accounts of voluntary medication incident reports or qualitative reports of such errors.²⁶ For example, Walsh et al⁹ proposed four categories, three of which were mechanisms (selection errors, keypad entry errors, and incorrect selection of order-sets) and one was a manifestation (duplicate orders). This led to the mechanisms of errors being mixed with the manifestation of such errors, limiting opportunities for understanding the relationship between these key dimensions; hence, the usefulness of our dual classification approach.

A limitation is that some system-related errors are impossible to detect using an audit. For example, a selection error from a drop-down menu which resulted in the prescription of a credible drug given the patient's condition would not have been detected. We also were unable to determine whether other mechanisms, such as a prescriber's inability to view all relevant information on a screen, may have contributed to a system-related error. Our data related to medications generated from a small number of wards and application of our classification on a larger sample would be valuable to understand the generalizability of the findings and classification. We used total admissions and patient days as our denominators. We were unable to obtain data on total orders per patient due to limited study resources. The e-PS reporting systems did not allow us to easily extract this information, but it is expected that this functionality will substantially improve and support the use of these data as a denominator in future studies.

Comparisons between e-PS with different design elements are critical to identify features associated with errors. To date comparative effectiveness studies are rare and those conducted have generally focused on the effectiveness of decision support systems to fire alerts in response to dangerous prescriptions,²⁷ rather than investigation of design features which may increase the risk a prescriber will make an error while prescribing. We investigated two commercial e-PS, both of which are currently in use in hospitals in North America, the UK, and the Asia-Pacific region. While local customization may play a role in some issues identified, our results highlight generic design features and their relationship to system-related errors. Organizational factors play a role in shaping work practices and the relationship of these to system-related errors may be worthy of investigation. Importantly, the dual classification provided data of sufficient detail to allow both hospitals to make changes to their e-PS. For example, calculating the rate of selection errors alone would have been of limited value, but identifying that 60% of selection errors in one e-PS resulted in wrong strength errors provided a specific area for investigation and remediation. Changes were made as a result of this study. The positions of some drop-down menu items were changed, for example to bring frequently used items to the top of long lists, and some default times were changed. The eight errors found in order sentences at the Cerner site were corrected. The vendor of MedChart made software changes, for example, to display pre-defined order sentences (quick lists) for selected drug-products on the first accessible prescribing screen in order to discourage clinicians from long-hand prescribing to reduce the risk of selection errors. Therefore, if the study was repeated today, it would be expected that many of the system-related errors identified would no longer appear and there would be a lower overall rate for both systems.

As e-PS software continues to be modified and upgraded vigilant monitoring is required. As the recent Institute of Medicine report on health IT and patient safety⁶ concluded, a greater understanding of the ways in which systems introduce error requires more research and the application of robust measures of safe practice. Our results provide another building block in this process to secure safer systems.