Effects of training physicians in electronic prescribing in the outpatient setting on clinical, learning and behavioural outcomes: a cluster randomized trial

Abstract

Aims

Electronic prescribing systems may improve medication safety, but only when used appropriately. The effects of task analysis‐based training on clinical, learning and behavioural outcomes were evaluated in the outpatient setting, compared with the usual educational approach.

Methods

This was a multicentre, cluster randomized trial [EDUCATional intervention for IT‐mediated MEDication management (MEDUCATE trial)], with physicians as the unit of analysis. It took place in the outpatient clinics of two academic hospitals. Participants comprised specialists and residents (specialty trainees, in the UK) and their patients. Training took the form of a small‐group session and an e‐learning. The primary outcome was the proportion of medication discrepancies per physician, measured as discrepancies between medications registered by physicians in the electronic prescribing system and those reported by patients. Clinical consequences were estimated by the proportion of patients per physician with at least one missed drug–drug interaction with the potential for causing adverse drug events. A questionnaire assessed physicians' knowledge and skills.

Results

Among 124 participating physicians, primary outcome data for 115 (93%) were available. A total of 1094 patients were included. A mean of 48% of registered medications per physician were discrepant with the medications that their patients reported in both groups (P = 0.14). Due to registration omissions, a mean of 4% of patients per physician had one or more missed drug–drug interactions with the potential to cause a clinically relevant adverse drug event in the intervention group, and 7% in controls (P = 0.11). The percentages of correct answers on the knowledge and skills test were higher in the intervention group (57%) compared with controls (51%; P = 0.01).

Conclusion

The training equipped outpatient physicians with the knowledge and skills for appropriate use of electronic prescribing systems, but had no effect on medication discrepancies.

What is Already Known about this Subject

• In the inpatient setting, incorrect medication overviews in electronic prescribing systems predispose to a higher risk of adverse drug events.

• Classroom‐based training, computer‐based training and feedback seem to be effective methods by which to train physicians in appropriate use of electronic prescribing systems.

• Training content for the appropriate use of electronic prescribing systems was provided by a former task analysis and should contain training in procedural, cognitive and macro‐cognitive tasks.

What this Study Adds

• In the outpatient setting, 48% of registered medications per physician were found to be discrepant with the medications reported by their patients. Due to registration omissions, 4–7% of patients per physician had one or more missed drug–drug interaction, with the potential to cause clinically relevant adverse drug events.

• A task analysis‐based training was found to equip outpatient physicians with the knowledge and skills for appropriate use of electronic prescribing systems.

• This training had no effect on medication discrepancies.

Introduction

Electronic prescribing systems are developed and refined to improve medication safety. An important aim of these systems is to reduce the number of adverse drug events (ADEs). Such systems may reduce the number of ADEs by decreasing the number of administrative medication errors, such as illegible handwriting, errors in the route of administration and dispensing errors, as well as therapeutic medication errors, including unnoticed or inadequately managed drug–drug interactions (DDIs), overlooked contraindications and allergies, under‐ or overdosing, duplicate therapy and undertreatment. However, such a system in itself is not sufficient to eliminate errors, and can even induce new errors 1, 2.

The emphasis on technical system development contrasts with the little attention given to the appropriate use of these systems by physicians, even though the latter is crucial for success 3. This one‐sided approach differs from that in other high‐risk domains, such as aviation. Pilots are trained and tested extensively in theory, using simulators, and in practice, flying aeroplanes under different circumstances, and they are well aware of potential risks and human errors. By contrast, physicians have a much shorter introduction into the control buttons of prescribing systems, and thereafter ‘learning by doing’ limits the potential benefits for medication safety. The outpatient setting is especially challenging because there is limited time available to obtain a complete overview of the patient's medication. Moreover, patients often use multiple medications, prescribed by different physicians and dispensed by more than one pharmacy. Research has shown that incorrect medication overviews in electronic prescribing systems predisposes to a higher risk of ADEs 4.

The optimal strategy for training physicians to use electronic prescribing systems remains unclear 4, 5. A systematic review suggests that a combination of classroom‐based training, computer‐based training and feedback would be appropriate 6. However, the evidence for this is limited and does not provide concrete directions for training content and a didactic approach. Human factor sciences provide a conceptual framework and accompanying methods for analysing and designing training that is focused on improving safety 7. We used a human factors method, a so‐called task‐analysis, to design an educational intervention 8, 9. With this, we aimed to equip physicians with the skills and knowledge necessary for appropriate use of systems for electronic prescribing, and ultimately aimed to increase medication safety. We evaluated the effects of a task analysis‐based educational intervention in the outpatient setting on (the potential consequences of) medication discrepancies and on learning and behavioural outcomes, compared with the usual educational approach. Medication discrepancies were the primary outcome. This was in line with the ‘meaningful use criteria’, stating that recording current medication in electronic prescribing systems is the basis of meaningful use 10.

Methods

Trial design

A two‐arm cluster randomized trial was performed with the objective of demonstrating superiority of the task analysis‐based educational intervention. Physicians were the unit of analysis. Patients visiting a participating physician during the study period were in the same cluster. A cluster design was adopted because the intervention was targeted at the level of physicians, and primary outcomes were measured on the level of patients 11. A waiver from the ethical review board of the two participating hospitals was obtained as the study did not influence patient care and had little impact on patients. The boards of both hospitals granted permission for the study. Full details of the methodology of the study have been described elsewhere 12. The registration number of the study is ISRCTN50890124.

Participants and setting

Eligible physicians worked as specialists and residents (specialty traninees, in the UK) of internal medicine or related specialties (including geriatrics, rheumatology, gastroenterology, cardiology or pulmonology) for at least 4 h a week in the outpatient clinics of one of the two participating academic hospitals in the Netherlands: the University Medical Centre Utrecht (UMCU) and the Erasmus Medical Centre (EMC) Rotterdam. Electronic prescribing systems were available in the two participating hospitals. UMCU used the ChipSoft hospital information system (ChipSoft BV, Amsterdam, the Netherlands), in which prescribing is fully integrated into the electronic health record. EMC used iSoft Medicator (Computer Sciences Corporation, Groningen, the Netherlands), which is partly integrated into the electronic health record. Both electronic prescribing systems were able to store current medication lists and allergies, as well as basic decision support (drug–drug interactions, doses, duplicate orders, contraindications). No formal system or infrastructure was available in either hospitals to exchange data about current medication with general practitioners or community pharmacies.

Written consent was obtained from participating physicians after an oral explanation of the study. Patients were included if they were aged >18 years, and visited one of the participating physicians during the study period. Written consent was requested. Directly after completion of the educational intervention, consecutive patients of the enrolled physicians were invited to participate. At the same time, patients of physicians in the control group were invited to enrol in the study. Patients could only participate once in the study.

Intervention

The task analysis‐based educational intervention consisted of a 1‐h small‐group session, with facilitators introducing and discussing the importance of appropriate use of electronic prescribing systems. In this session, physicians discussed the importance of recording all current medication in the electronic prescribing system, and had the opportunity to discuss perceived challenges and share solutions. E‐learning was introduced to each physician in an individual half‐hour session. Physicians completed the e‐learning modules in their own time and pace, in 2–6 h. The modules were focused on increasing both practical skills (e.g. how to record a prednisone tapering scheme in the system) and cognitive skills (e.g. what to record in the system when patients tell you that they no longer use a certain drug). E‐learning modules were tailored to physicians' needs by allowing the latter to choose their own starting level, and choose to practise at this level or move to the next level 13. Points for continuing medical education were granted if 70% of the e‐learning questions were answered correctly.

E‐learning was developed according to the four‐component instructional design method 14. This allowed for the design of education on the basis of a thorough task analysis. Central to this approach is the expansion of appropriate use from the mere technical act. Full details of the task analysis are described elsewhere 9.

The control group received the ‘usual approach’. This typically consists of an approximately 1‐h introduction into the electronic prescribing system based on exercises in a computer room, with the opportunity to ask questions to a pharmacy technician. This had already taken place before the study period and had not been standardized across hospitals and physicians.

Outcomes

Table 1 describes outcome measures. The primary outcome was the proportion of medication discrepancies per physician, defined as discrepancies between medications registered by physicians in the electronic medication system and medications reported by patients. Medications were compared only in terms of the presence or absence of the active substances; doses, frequencies and administration routes were not taken into account. Patient data were collected by a telephone survey and considered the gold standard for the patient's use of medication. The questionnaire for the telephone survey was derived from the ‘structured medication history’ and a telephone survey based on Gandhi et al. 15, 16.

Table 1.

Description of outcome measures

Primary outcomes: medication discrepancies
Medication discrepancies, proportion a per physician	= number of discrepancies per physician/number of medication records per physician (patient's medication + medication registered but not taken by patient)
Patients with at least one medication discrepancy, proportion per physician	= number of patients per physician with at least one discrepancy/number of patients per physician

Secondary outcomes: missed drug–drug interactions (DDIs)
Missed DDIs, proportion per physician	= number of missed DDIs per physician/number of DDIs per physician
Patients with at least one missed DDI with potential to cause a clinically relevant adverse drug event (ADE), proportion per physician	= number of patients per physician with at least one missed DDI with potential to cause ADE/number of patients per physician

Secondary outcomes: learning and behavioural outcomes
Learning outcomes	= test score for knowledge & skills, and perceived attitude, social norm, self‐efficacy
Determinants of behaviour	= perceived attitude, self‐efficacy and social norms regarding systems for electronic prescribing
Behavioural outcomes	= number of patients from whom physicians obtained a medication history/number of patients per physician
	= number of patients provided with a medication summary/number of patients per physician

^aProportions are given as percentages in the text and the tables

The possible clinical consequences of the identified medication discrepancies were estimated by the proportion of patients per physician with at least one missed drug–drug interaction (DDI) with the potential for causing a clinically relevant ADE. The Dutch clinical guideline for management of DDIs was used to classify potential clinical consequences of DDIs into six levels of severity, from A (potentially resulting in a minor ADE) to F (potentially resulting in a fatal ADE) 17. Severity levels D, E and F are interactions with the potential to cause clinically relevant ADEs.

To measure learning outcomes, we assessed knowledge and skills 1 year after the intervention in both study arms, using an electronic multiple choice test. A test matrix was developed, to ensure content validity and guarantee an even distribution of training content and comparability of questions across hospitals. The 30 questions were pretested by two experts in each hospital. As we tested different types of knowledge (declarative, problem solving, error awareness) in a relatively small number of questions, the calculation of internal consistency with Cronbach's alpha was not applicable 18. A Rit value is the correlation between a individual question score and the total examination score, and reflects the question's capacity to distinguish good from poor performers. Only one of the test's questions had a negative Rit value.

Participants ‘passed’ when 55% of the questions were answered correctly. Scores were corrected for the probability of guessing a correct answer.

Behavioural outcomes were measured by asking patients whether their physician had obtained a medication overview and had provided them with a medication summary. To measure determinants that might have influenced behaviour, physicians completed an electronic survey on their attitudes on electronic prescribing systems, the perceived social norm concerning the appropriate use of electronic prescribing systems, and self‐efficacy 19, 20, 21.

Sample size

The sample size was calculated by assuming that the comparison between groups should be able to detect a difference of at least 10%, with a significance level of 5% (two‐sided), with a control group incidence of discrepancies of 30%. The intraclass correlation coefficient (ICC) was assumed to be 0.1, according to Schnipper et al. 4. A statistical power of >90% is ensured with a sample of 40 physicians per group, with 20 patients per physician. If more physicians can be recruited, fewer patients per physician will be needed, while maintaining power.

Randomization and blinding

After giving consent, physicians were randomly allocated in a 1:1 ratio to the control or intervention group. They were randomized immediately after inclusion by the investigator, with assignments generated by a computer system, provided by an independent research centre. Allocation was sufficiently concealed as assignments for the next allocation were provided at the time of randomization. Randomly permuted blocks, with six clusters per block, were used. Randomization was stratified by hospital.

Patients were blinded to the intervention status of their physician. The nature of the intervention did not allow for blinding physicians for their intervention status. Data cleaning and analysis were blinded to allocation.

Statistical analysis

The analysis followed the intent‐to‐treat principle, with the exception that physicians with no enrolled patients were excluded from the analysis. Clustering was taken into account by using the population average model. SPSS Statistics version 23 (IBM Corp in Armonk, NY, USA) was used for analyses. We used a univariate analysis of variance to analyse the differences in the proportions of discrepancies between study groups. A sensitivity analysis using the binomial model revealed the same results.

Knowledge and skills differences between study groups were analysed using Student's t‐test. In all analyses, P‐values smaller than 0.05 were considered significant. No correction for multiplicity was applied, so all analyses, except for the primary outcome, were considered exploratory.

Results

A total of 206 physicians were invited to participate, of whom 124 (60%) agreed and were included in the study. Recruitment took place between 11 February 2014 and 7 July 2014. The last data were collected on 30 November 2015. The main reason to refuse participation was lack of time.

The included physicians were randomized and 115 were evaluated in the final analysis; two participants withdrew from the study, and seven participants had no enrolled patients (Figure 1.) A total of 1094 patients were included [almost 10 patients per physician (range 1–14)].

Figure 1.

Flow‐chart of participants: enrolment, allocation, follow‐up and analysis. DDI, drug–drug interaction

Physicians in both study arms had a median of 5 years’ experience of using systems for electronic prescribing (Table 2). Prior to the study, all physicians had received a 1‐h introduction in using the electronic prescribing system, usually provided by a pharmacy technician. The included patients used a median of eight medications (range 1–28) in the intervention group and six (range 1–25) in the control group, and one over‐the‐counter (OTC) medication in both study groups (range: intervention, 0–5; control, 0–14).

Table 2.

Baseline characteristics of included physicians and patients

Physician characteristics	Control (n = 61)	Intervention (n = 63)
Age, mean (SD), years	42 (10)	46 (10)
No. (%) of female physicians	30 (50)	33 (52)
Experience with electronic prescribing, mean (SD), years	4.6 (0.7)	4.8 (0.4)
No. (%) of physicians per training status
Residents	27 (44)	19 (30)
Specialists	34 (56)	44 (70)
No. (%) of participants per study site
University Medical Centre Utrecht	32 (52)	34 (54)
Erasmus Medical Centre Rotterdam	29 (48)	29 (46)
No. of patients included per physician, mean (SD)	10 (3.1)	9 (2.6)

Patient characteristics	Control (n = 528)	Intervention (n = 562)
Age, mean (SD), years	57 (17)	53 (15)
No. (%) of female patients	265 (50)	289 (51)
No. of medications per patient, mean (SD)	8.4 (4.3)	7.4 (4.2)
No. of high‐risk medications per patient, mean (SD)	1.7 (1.6)	1.4 (1.5)
No. of OTC medications per patient, mean (SD)	1.0 (1.0)	1.2 (1.2)

OTC, over the counter; SD, standard deviation

Medication discrepancies and potential clinical consequences

A mean of 48% of the registered medications per physician were discrepant with the medications that their patients reported using. These percentages did not differ between study arms (P = 0.14) or study sites (Table 3; Figure 2). This concurs with the observation that 94% of the patients per physician in the intervention group and 96% in the control group had one or more medication discrepancy (P = 0.30). The percentage discrepancies showed a wide range, varying from 9% to 100% for individual physicians. Neither the number of included patients per physician nor the number of medications per patient was related to the percentage discrepancies.

Table 3.

Effect of the educational intervention on medication discrepancies and missed drug–drug interactions

	Control (n = 55)	Intervention (n = 60)	P‐value
Proportion per physician expressed as percentage
Medication discrepancies, mean (SD), %	48 (16)	48 (17)	0.14
Patients with at least one medication discrepancy, mean (SD), %	96 (7)	94 (10)	0.30

	Control (n = 51)	Intervention (n = 54)	P‐value
Missed DDIs, mean (SD), %	28 (28)	25 (27)	0.060
Patients with at least one missed DDI with potential to cause clinically relevant ADE, mean (SD), %	7 (8)	4 (6)	0.11

ADE, adverse drug event; DDI, drug–drug interaction; SD, standard deviation

Figure 2.

Effect of task analysis‐based intervention on medication discrepancies. The box portion of the box plot is defined by two lines at the 25^th percentile and 75^th percentile. The distance between the upper (75^th percentile) and lower (25^th percentile) lines of the box is the inter‐quartile range (IQR). The line inside the box is the median (50^th percentile). The line with a crossbar line that goes out from the box is the box plot whisker. For the upper whisker boundary, it is the largest observation that is less than or equal to the upper edge of the box plus 1,5 times IQR. The small circles are outliers: datapoints outside the whisker boundaries

Of the medication discrepancies, 70% were omissions (i.e. the patient was taking medications not registered in their medication record) and 30% were additions (i.e. the patient did not take medications that were registered in the medication record). Eight per cent of the medication discrepancies concerned high‐risk medications and 30% involved OTC medications. These numbers did not differ between study arms.

Of the drug–drug interactions (DDI), 25% per physician were missed owing to omissions in the electronic medication record in the intervention group, and 28% in the control group (P = 0.06). A mean of 4% of the patients per physician had one or more missed DDIs with the potential to cause a clinically relevant ADE in the intervention group, and 7% in the control group (P = 0.11) (Table 3; Figure 3). For example, a potentially life‐threatening missed DDI was an interaction between renin–angiotensin–aldosterone system inhibitors and potassium‐sparing diuretics, or between medications which both prolong the QT‐time (Table 4).

Figure 3.

Effect of task analysis‐based intervention on missed drug–drug interactions (DDIs) with the potential to cause clinically relevant adverse drug events (ADEs). The box portion of the box plot is defined by two lines at the 25^th percentile and 75^th percentile. The distance between the upper (75^th percentile) and lower (25^th percentile) lines of the box is the inter‐quartile range (IQR). The line inside the box is the median (50^th percentile). The line with a crossbar line that goes out from the box is the box plot whisker. For the upper whisker boundary, it is the largest observation that is less than or equal to the upper edge of the box plus 1,5 times IQR. The small circles are outliers: datapoints outside the whisker boundaries

Table 4.

Learning and behavioural outcomes

Learning outcomes	Control (n = 42)	Intervention (n = 54)	P‐value
Test score, mean (SD), % correctly answered	51 (10)	57 (12)	0.01
Proportion passed participants, mean (SD), %	26 (45)	46 (50)	0.04

Site 1:	Control (n = 23)	Intervention (n = 29)	P‐value
Test score, mean (SD), % correctly answered	56 (10)	63 (12)	0.04
Proportion passed participants, mean (SD), %	48 (50)	66 (50)	0.07

Site 2:	Control (n = 19)	Intervention (n = 24)	P‐value
Test score, mean (SD), % correctly answered	45 (50)	51 (80)	0.02
Proportion passed participants, mean (SD), %	0	25 (40)	0.02

Behavioural outcomes: determinants of behaviour	Control (n = 39)	Intervention (n = 48)	P‐value
Attitude towards effect of electronic prescribing system on:
Patient/physician relationship, % positive attitude	67	80	0.12
Quality of care, % positive attitude	95	96	0.84
Physicians' job satisfaction, % positive attitude	46	57	0.25
General attitude, % positive attitude	84	89	0.32
Perceived social norm to use electronic prescribing systems during consultations:
Perceived social norm for appropriate use, %	48	49	0.98
Self‐efficacy for prescribing with an electronic prescribing system (points on scale 0–100):
A ‘fixed dose’ regimen, mean (SD)	97 (4.6)	97 (5.1)	0.83
A ‘different doses a day’ regimen, mean (SD)	90 (18)	90 (17)	0.94
A tapering scheme, mean (SD)	64 (34)	70 (26)	0.38
Providing a medication summary, mean (SD), points on scale 0–100	78 (31)	90 (18)	0.03

Behavioural outcomes	Control (n = 55)	Intervention (n = 60)	P‐value
Patients from whom a medication history was obtained, proportion per physician, mean (SD), %	55(22)	59(23)	0.20
Patients provided with a medication summary, proportion per physician, mean (SD), %	4(7.7)	10(16)	0.06

SD, standard deviation

Learning and behavioural outcomes

The percentages of correctly answered questions on the test for knowledge and skills were higher in the intervention group (57%) compared with the control group (51%; P = 0.01) (Figure 4). Differences between study sites were remarkable: on one site, 66% passed in the intervention group, compared with 25% on the other site (Table 5).

Figure 4.

Effect of task analysis‐based intervention on knowledge and skills. The box portion of the box plot is defined by two lines at the 25^th percentile and 75^th percentile. The distance between the upper (75^th percentile) and lower (25^th percentile) lines of the box is the inter‐quartile range (IQR). The line inside the box is the median (50^th percentile). The line with a crossbar line that goes out from the box is the box plot whisker. For the upper whisker boundary, it is the largest observation that is less than or equal to the upper edge of the box plus 1,5 times IQR. The small circles are outliers: datapoints outside the whisker boundaries

Table 5.

Frequencies of missed drug–drug interactions in MEDUCATE database

Severity score	Missed interactions	Total interactions in database
F (potentially life threatening)
RAAS inhibitors + potassium sparing diuretics	6	45
QT‐prolongation drug + QT‐prolongation drug	4	5
Potassium + potassium‐sparing diuretics	1	5
E (potential for permanent harm)
MTX + NSAID	4	31
Coumarin + miconazole	3	3
Trimethoprim + RAAS inhibitors/spironolactone	2	6
Simvastatin/atorvastatin + CYP3A4 inhibitors	1	7
SSRI + thiazide	1	4
Statin + colchicine	1	11
D (potential need for hospitalization)
RAAS inhibitors + diuretics	22	103
Diuretics + NSAID	7	20
RAAS inhibitors + NSAID	5	25
Coumarin + omeprazole	5	25
Coumarin + antibiotics	5	11
Coumarin + amiodarone/propafenone	2	11
MTX + antibiotics	1	2
Coumarin + vitamin K	1	1

CYP3A4, cytochrome P450 3A4; NSAID, nonsteroidal anti‐inflammatory drug; MTX, methotrexate; RAAS, renin–angiotensin–aldosterone system; SSRI, selective serotonin reuptake inhibitor

Physicians provided a medication summary to 10% of their patients in the intervention group, compared with 2% in the control group (P = 0.06). The percentage of patients for whom a medication history was actively obtained did not differ between study arms (55% vs. 59%; P = 0.40). Physicians in the intervention group perceived a higher level of self‐efficacy for providing patients with a medication summary (90 points vs. 78 points out of 100; P = 0.03). Other determinants of behaviour were similar for both study groups (Table 4).

Overall satisfaction with the educational intervention was rated as good; 73% of the participants were satisfied with the starting level, 83% were satisfied with the e‐learning structure and 70% actively used the knowledge and skills thereby obtained in their professional practice (Table 6).

Table 6.

Satisfaction and perceptions of the educational intervention

	n = 42
Satisfied with starting level of the e‐learning, %	73
Satisfied with structure of e‐learning, %	83
Practices in training relevant, %	83
Knowledge and skills useful in professional practice, %	70
Consider videos in the e‐learning inspiring	33
Group session considered added value	45
Would recommend training to colleagues	67
Overall satisfaction with group session, mean (SD), on 10‐point scale	6.7
Overall satisfaction with e‐learning, mean (SD), on 10‐point scale	7.4
Overall satisfaction with intervention as a whole, mean (SD), on 10‐point scale	7.3

SD, standard deviation

Discussion

The task analysis‐based educational intervention equipped outpatient physicians with the skills and knowledge to use electronic prescribing systems appropriately, as reflected by higher test scores in favour of the intervention group. However, the ultimate goal to decrease the number of medication discrepancies and their consequences was not reached.

This was the first randomized trial to evaluate the effects of education on medication discrepancies in electronic medication records, and on learning and behaviour, which was an important strength of the study. As our focus was on electronic prescribing as a process and not merely as a technical act, our findings make a contribution to the discussion on how to train physicians in the appropriate use of electronic prescribing systems. However, the result should be interpreted in the light of several limitations. First, actual ADEs could not be assessed. Secondly, patient data were used as the gold standard, and were not triangulated with other sources of information. Thirdly, although the nature of the trial did not allow for the blinding of physicians, they were not fully informed about the trial's outcomes. Lastly, we focused on medication discrepancies because correct registration is the starting point of all other advantages of electronic prescribing. However, other effects of electronic prescribing were not assessed, such as reducing the number of administrative errors, or of overlooked contraindications or allergies.

To understand why medication discrepancies were not influenced by the educational intervention, we need to know which factors influence human, and thus physicians', behaviour 19. According to the theory of planned behaviour and reasoned action, physicians have a higher intention (motivation) for appropriate use of electronic systems when: physicians evaluate the behaviour as positive and important (attitude), they think their significant others want them to use the system appropriately (subjective norm), and they believe they are able to perform this behaviour (self‐efficacy) 20. Physicians also need knowledge and skills, and a facilitating environment. It is therefore hard to understand why the intervention did not decrease the number of medication discrepancies. First, we tried to influence, but probably overestimated, the potential effect of the educational intervention on attitude, perceived social norm and self‐efficacy. We might even have overestimated the effect on knowledge and skills. However, the intervention did appear to have a conclusive effect on learning, as the effect was still measurable after 1 year. A recently published study on prescribing antibiotics also used behavioural sciences to influence physicians’ behaviour 22. These authors found that ‘accountable justification’, whereby physicians had to justify explicitly their decision for prescribing antibiotics, and ‘peer comparison’, whereby physicians’ antibiotic prescribing rates were ranked from highest to lowest within an email, resulted in lower rates of inappropriate antibiotic prescribing 22, 23. These types of intervention are difficult to implement in the domain of appropriate use of electronic prescribing systems. It is impossible to ask for justification for things not done, or to give feedback on something that physicians have omitted. Our results were comparable with those of other studies with a primary focus on training knowledge and skills, in that an influence on skills was observed but the effects on relevant clinical outcomes were difficult to detect 24, 25.

Secondly, the frequency and length of the educational intervention might need to increase, to achieve a greater effect. However, there is a precarious balance between the time investment needed for real learning and the willingness to make, and viability of, this investment, given the other responsibilities physicians have. Thirdly, we may have underestimated the relative contribution of environmental factors on physicians’ behaviour. Unpublished data from the present trial revealed no significant correlations between the proportion of medication discrepancies and physicians' demographics, attitudes, perceived social norm, self‐efficacy, and knowledge and skills. Participating physicians were relatively experienced in electronic prescribing. We hypothesize that the training will be more effective with less experienced physicians but, on the basis of our data, we are not able to substantiate this hypothesis.

The limited length of consultations and the system's interface were probably strong barriers for appropriate use. Finally, intervention effects might have been diluted by uncontrollable factors, such as contamination by physicians in the intervention group inadvertently teaching controls.

The present study highlights the magnitude of the problem of medication discrepancies and missed DDIs. With few exceptions, the medication records of more than 1000 patients, under the care of 115 physicians, showed at least one discrepancy with their actual medication use. In approximately 5% of patients per physician, we detected missed interactions due to registration omissions with the potential for a clinically relevant ADE. This underlies the importance of studies such as this in understanding the cause and solutions for such discrepancies.

Can it be justified not to train physicians in the appropriate use of electronic prescribing systems? This would be tantamount to concluding that electronic prescribing systems are irreparably unintuitive, and that the duration of consultations is so limited that errors will occur regardless of training. In the ideal scenario, we invest in more intuitive electronic prescribing systems, tailored to patients’ characteristics and physicians’ needs; systems facilitating the recording of accurate information about patients’ medication; or in supporting outpatient physicians to obtain a correct medication overview by pharmacy technicians. One of the current initiatives in the Netherlands is a system by which medical data are exchanged electronically between healthcare providers. This takes place via a ‘national switch point’ (NSP), which provides a reference index for routing, identifying, authenticating, authorizing and logging. The NSP can be likened to an air traffic control tower which regulates the exchange of patient data between healthcare providers. This system has the potential further to improve future prescribing. However, even then, physicians will need to accept the importance of increasing medication safety by appropriate use of electronic prescribing systems, and the knowledge and skills to do so. The current intervention may improve this situation by making available real‐life examples of missed DDIs. Paying physicians to use such systems appropriately does not provide them with the necessary knowledge and skills, and undermines the intrinsic intentions for this behaviour 26.

Any success in decreasing the number of medication discrepancies will most likely be due to a combination of educational and environmental factors. With electronic prescribing systems rapidly increasing in number, it will become particularly important for physicians to have the knowledge and skills to use them. The present study showed that the acquisition of knowledge and skills can be achieved by a task analysis‐based educational intervention.

Competing Interests

E.t.B. and F.v.S. had support from the Netherlands Organization for Health Research and Development (ZonMw) for the submitted work. The design, analysis, interpretation and reporting of the study were entirely independent of the funder. The authors declare no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; and no other relationships or activities that could appear to have influenced the submitted work. All authors had full access to all data in the study and had final responsibility for the decision to submit for publication.

We thank all physicians and patients for participating in this study. We thank research assistants for contacting patients to collect data. We thank ZonMw for funding this research. ZonMW is the Netherlands Organisation for Health Research and Development; the main commissioning organisations for ZonMw are the Dutch Ministry of Health and the Netherlands. Organisation for Scientific research.

Contributors

F.v.S. and E.W.M.T.t.B. were responsible for the study concept. F.v.S. and E.W.M.T.t.B., J.E.F.Z.‐v.R, H.K., J.E.C.M.A., C.B.R. and A.C.G.E. designed the study. F.v.S. J.V., I.H.v.S., T.v.G., R.d.M. and E.W.M.T.t.B. carried out data acquisition. F.v.S. carried out data analysis. F.v.S., J.E.F.Z.‐v.R., C.B.R., A.C.G.E. and E.W.M.T.t.B. interpreted the data. F.v.S., J.E.F.Z.‐v.R., C.B.R., A.C.G.E. and E.W.M.T.t.B. drafted the manuscript. F.v.S., J.E.F.Z.‐v.R., J.V., H.K., J.E.C.M.A., I.H.v.S., T.v.G., R.d.M., C.B.R., A.C.G.E. and E.W.M.T.t.B. critically revised the manuscript. J.E.F.Z.‐v.R., C.B.R. and E.W.M.T.t.B. provided supervision. J.V., I.H.v.S., T.v.G. and R.d.M. provided technical support. F.v.S., J.E.F.Z.‐v.R., H.K., C.B.R., A.C.G.E. and E.W.M.T.t.B. obtained funding.

van Stiphout, F. , Zwart‐ van Rijkom, J. E. F. , Versmissen, J. , Koffijberg, H. , Aarts, J. E. C. M. , van der Sijs, I. H. , van Gelder, T. , de Man, R. A. , Roes, C. B. , Egberts, A. C. G. , and ter Braak, E. W. M. T. (2018) Effects of training physicians in electronic prescribing in the outpatient setting on clinical, learning and behavioural outcomes: a cluster randomized trial. Br J Clin Pharmacol, 84: 1187–1197. doi: 10.1111/bcp.13540.