An automated medication system reduces errors in the medication administration process: results from a Danish hospital study

Abstract

Objectives

Improvements in a hospital's medication administration process might reduce the prevalence of medication errors and improve patient safety. The objective of this study was to evaluate the success of an automated medication system in reducing medication administration errors.

Methods

A controlled before-and-after study with follow-up after 4 months was conducted in two Danish haematological hospital wards. The occurrence of administration errors was observed in two 3-week periods. The error rate was calculated by dividing the number of doses with one or more errors by the number of doses (opportunities for errors). Logistic regression was used to assess changes in error rates after implementation of the automated medication system with time, group, and interaction between time and group as independent variables. The estimated parameter for the interaction term was interpreted as the incremental change (‘difference-in-difference’) caused by the new dispensing system.

Results

A total of 697 doses with one or more errors were identified out of 2245 doses. The error rate decreased from 0.35 at baseline to 0.17 at follow-up in the intervention ward and from 0.37 to 0.35 in the control ward. The overall risk of errors was reduced by 57% in the intervention ward compared with the control ward (OR 0.43; 95% CI 0.30 to 0.63).

Conclusions

The automated medication system reduced the error rate of the medication administration process and thus improved patient safety in the medication process.

Introduction

Safe and effective handling of medicine implies that the right patient receives the right medicine in the right dose, at the right time and through the right route. Adverse events associated with medication are one of the largest causes of harm to hospitalised patients. Reviews have suggested that up to 50% of the adverse events in the medication process may be preventable. Thus the medication administration process is an important area for safety improvement.^1–5

A Danish study from 2003 of errors in the medication process found errors in 41% of the observed medication administrations. It was estimated that 20–30% of these errors could potentially have caused adverse events. Implementation of automated technologies in the medication process was suggested as a way to reduce error rates.⁶

A number of technical solutions are available, such as automated dispensing devices that pack medications either as multidose or unit-dose bags for individual patients. The effect of automated drug dispensing has been tested previously, but with inconsistent results. Some studies have found a reduction in the number of medication errors,^{7 8} although a recent review by Tsao et al⁹ concluded that decentralised automated dispensing devices had limited potential to reduce medication errors, and that the impact of such devices is highly institution-specific. This highlights the need for testing automated dispensing systems in different settings.

Barcode-assisted medication administration (BCMA) has been introduced as another solution to improve safety when administering medication. International studies have suggested that patient identification and alignment with the medication administration record can reduce the number of medication administration errors.^10–12

Little is known of the effects on medication administration errors of combining several technologies such as electronic medication administration records (eMARs), automated drug dispensing and BCMA. The objective of this study was to evaluate the effect on rates of error in the medication administration process of implementing an automated medication system (AMS) integrating these three technologies in a Danish hospital department.

Methods

Study design

A prospective, controlled before-and-after study with baseline measurements and follow-up after 4 months was conducted in two Danish haematological university hospital wards from May 2013 to February 2014. An AMS was implemented in one of these wards, while the other ward served as control. A detailed description of the AMS is provided in online supplementary file 1 and the study outline in online supplementary file 2.

During the study period, all registered medication administrations performed by nurses in the participating wards were included. Owing to the technical limitations of the AMS, injectable medicine and liquid formulations had to be excluded from the study.

Data on background characteristics were extracted for two half-year periods to take account of monthly variations: baseline, 1 January to 30 June 2013; follow-up, 1 January to 30 June 2014.

Setting and participants

The hospital pharmacy service was based on ward stock supply, where commonly used medications were kept in the ward's medicine room. Physicians entered prescriptions and changes in prescriptions directly into the eMAR in the ward. The eMAR was used to identify each individual process of medicine administration. The eMAR medication dispensing was carried out by nurses in the medicine room. These procedures were the same in both wards and in both periods of data collection.

Through observational visits it was confirmed that the two participating wards had comparable workflow, medication profile and flow of patients. The two wards were organised under the same departmental management, but had separate ward management and staff. Allocation of wards to either intervention or control was determined by the managers of the haematological department after a brief introduction to the study. Figures 1 and 2 illustrate the medication workflow in the wards at baseline and after implementation of the AMS.

Figure 1.

Usual medication workflow at baseline. eMAR, electronic medication administration record.

Figure 2.

Medication workflow after implementation of the automated medication system (AMS). eMAR, electronic medication administration record; PDA, personal digital assistant.

Characteristics of the participating wards are presented in table 1. The two wards were very similar in size, number of patients, staff and use of medication. However, at follow-up the number of hospitalisation days had decreased and the average length of stay had increased in the intervention ward, resulting in a lower number of admissions.

Table 1.

Background characteristics of the control and intervention ward

Characteristic	Control ward		Intervention ward
	Baseline	Follow-up	Baseline	Follow-up
Number of beds	22	22	21	21
Occupancy rate in the ward	84.6	84.0	88.8	85.8
Number of hospitalisation days	3387	3364	3394	3279
Average length of stay	5.3	5.3	5.2	6.3
Number of admissions	686	674	650	544
Number of full-time employees	48.2	47.3	45.3	45.1
Mean number of tablets per patient per day	14.4	13.7	14.0	14.1

Data were extracted for the following periods: baseline (1 January 2013–30 June 2013); follow-up (1 January 2014–30 June 2014).

Background data were obtained from the hospital administrative system.

Mean number of tablets per patient per day was obtained from observational data (prospective study data).

The error rate in the medication administration process

The primary study outcome was number of errors in the medication administration process. The error rate was determined by dividing the number of doses identified with one or more errors by the number of opportunities for error, corresponding to the total number of doses, in accordance with the work by Allan and Barkers.¹³ Hence, ‘opportunities for errors’ were defined as the sum of any doses given plus any doses prescribed but omitted.

Definition

Errors in the medication administration process were defined as ‘the administration of a dose of medication that deviates from the eMAR prescription, from standard hospital policy or from written procedures’.

Errors in the medication administration process were further divided into:

1. Clinical errors: where the patient did not receive the medication as prescribed in the eMAR

2. Procedural errors: deviations from written procedures or guidelines. Deviations could potentially but not necessarily lead to medication errors.¹⁴

Table 2 outlines the most common types of error from a previous study¹⁵ with the addition of procedural errors identified by hospital policies and local written procedures. Further details about the policies and procedures are described in online supplementary file 3. The AMS was primarily designed to change the medication administration process. Therefore, errors in the process of prescribing medicine were not part of this study.

Table 2.

Definition of identified error types in the medication administration process

Error type	Definition
Clinical errors
Wrong drug	The administered drug was not prescribed in the eMAR
Omission of dose	The prescribed dose of the drug was not administered to the patient
Wrong dose	The administered dose deviated from the prescribed dose
Wrong substitution	The substitution was not generic. A drug was substituted by one with another active ingredient
Wrong administration form	The form of the administered drug deviated from the eMAR prescription
Wrong patient	The administered drugs were given to the wrong patient
Packaging error*	The medication packed by the AMS deviated from the information on the label
Procedural errors
Wrong strength per unit	The strength of the administered drug deviated from the prescription in the eMAR
	For example, 1 tablet of 100 mg was prescribed in the eMAR but 2 tablets of 50 mg were administered by the nurse. If this deviation was not documented in the eMAR it was regarded a procedural error
Wrong administration time	The administration time deviated more than 1 h from the eMAR prescription
Lack of:
Documentation of a substitution	A substitution was made but not documented in the eMAR
Documentation of the administration	The medication was not documented as ‘administered’ in the eMAR
Check of prescription	The nurse did not check the prescription in the eMAR before administering the medicine
Patient ID at the medicine box	The nurse did not label the medicine box with patient ID
Patient ID control	The nurse did not ask for both patient name and civil registration number
Barcode scanning*	The PDA was not used by the staff during administration

*Error type only relevant in the intervention ward at follow-up after implementation of the AMS.

AMS, automated medication system; eMAR, electronic medication administration records; ID, identity; PDA, personal digital assistant.

Data collection

The observation period was 3 weeks before and after implementation of the AMS. Clinical and procedural errors were measured by direct observation of nurses during the medication administration process by three dedicated clinical pharmaconomists with extensive knowledge of the medication used for patients in the haematological department. All three performed observations in both the intervention and control wards in the before and after periods. Medication administration was primarily observed at 08:00, when patients received most of their daily medication. The observers followed a protocol to ensure reliable and valid observations. They were instructed to intervene only if they observed a severe error during the administration process. A 1-day pilot data collection was used to train the observers, to test the protocol, and to align the definition of errors to reduce interobserver variability.

All observations were recorded on paper-based forms and entered into an Excel spreadsheet (Microsoft Office V.2003). These were subsequently compared with the prescribed medication in the eMAR and the written procedures, and any discrepancies were considered errors and categorised according to table 2.

To ensure validity of the entered data, a sample of 80 observations (equally divided by wards and periods) was randomly selected and controlled against the original forms and prescriptions in the eMAR. Only a few discrepancies were identified and adjusted.

Sample size

The number of observations needed to ensure sufficient statistical power was estimated by a power calculation assuming an error rate of 0.22, a power of 80 and an expected reduction in error rate of 30%. This indicated a required sample size of 511 doses in each ward in both data collection periods.

Data analysis

The Excel files were imported into Stata V.12. The datasets remained in their original form, and all changes, recoding and generation of new variables were performed in do-files to ensure transparency. The prevalence of errors was calculated for each defined error type for the intervention and control ward and the two observation periods.

To determine the difference in error rate after implementation of AMS, logistic regression was applied with the presence/absence of error as the dependent variable, and time, group and interaction between time and group as the independent variables. Logistic regression was carried out with the dependent variable coded with zero or more than zero errors per opportunity for error. A statistical significance level of 0.05 was used, and estimates for the OR and 95% CIs for the interaction variable (the ‘difference-in-difference’ estimate) were considered the primary result for the reduction in error rate for the intervention group at follow-up.

In addition, sub-analyses were performed separately for clinical errors and procedural errors to assess the importance of ‘potential clinical errors’ in relation to actual clinical errors.

Procedural errors such as ‘lack of identity (ID) control’, ‘lack of ID on the medicine box’ and ‘lack of personal digital assistant (PDA) verification’ would result in errors for all doses being administered at that specific dosing time. This mutual dependency between the error type and the medicines involved would systematically influence the results. To account for the impact of this potential, bias sensitivity analyses were performed. Further sensitivity analyses were performed to isolate the core effect of the AMS by excluding the manually dispensed doses during follow-up in the intervention ward.

Ethics

This study was registered and approved by the Danish Data Protection Agency (journal No 1-16-02-163-12). According to Danish law, this study did not require ethics approval from the National Committee on Health Research Ethics.

Results

As shown in table 3, a total of 1062 opportunities for errors (512 in the control ward and 550 in the intervention ward) were observed at baseline and 1183 at follow-up (613 in the control ward and 570 in the intervention ward). The number of doses with one or more errors in the medication process was 383 at baseline (191 in the control ward and 192 in the intervention ward) and 314 at follow-up (215 in the control ward and 99 in the intervention ward). The most commonly observed errors in the medication process were lack of ID control, lack of ID on the medicine box, and wrong strength per unit, which were all procedural errors (table 3).

Table 3.

Number of errors within each error type

Error type	Control ward		Intervention ward
	Baseline (N=512)	Follow-up (N=613)	Baseline (N=550)	Follow-up (N=570)
Clinical errors
Wrong drug	0	6	1	2
Omission of dose	1	55	5	3
Wrong dose	9	9	13	4
Wrong substitution	0	0	0	0
Wrong administration form	0	1	2	0
Wrong patient	0	0	0	0
Packaging error*				0
Procedural errors
Wrong strength per unit	51	46	50	1
Wrong administration time	2	0	2	0
Lack of documentation of a substitution	16	14	16	4
Lack of documentation of the administration	14	2	0	0
Lack of prescription check	14	0	49	9
Lack of ID on the medicine box	94	15	98	7
Lack of ID control	133	103	90	66
Lack of barcode scanning*				4

N is the number of observed administered doses. The actual number of errors within each error type is given. In some cases, more than one error per dose was detected.

*Error types only relevant in the intervention ward at follow-up.

ID, identity.

The overall rate of administration errors decreased from 0.35 to 0.17 in the intervention ward and from 0.37 to 0.35 in the control ward (table 4). This resulted in an overall reduced risk of errors of 57% in the intervention ward compared with the control ward: OR 0.43 (0.30 to 0.63). The decrease in overall error rate was statistically significant.

Table 4.

Errors in the medication process, OR and sensitivity analyses for two hospital wards with (intervention) and without (control) an automated medication system

	Control ward				Intervention ward
	Baseline (N=512)		Follow-up (N=613)		Baseline (N=550)		Follow-up (N=570)
	n	Rate	n	Rate	n	Rate	n	Rate	OR	95% CI
Administration errors	191	0.37	215	0.35	192	0.35	99	0.17	0.43	(0.30 to 0.63)
Sub-analysis*
Clinical errors	10	0.02	71	0.12	21	0.04	9	0.02	0.06	(0.02 to 0.17)
Procedural errors	185	0.36	153	0.25	173	0.31	91	0.16	0.7	(0.48 to 1.04)
Sensitivity analysis 1 (exclusion of ‘omission of dose’)
Administration errors excluding ‘omission of dose’	190	0.37	164	0.27	187	0.34	96	0.17	0.64	(0.43 to 0.93)
Clinical errors excluding ‘omission of dose’	9	0.02	16	0.03	16	0.03	6	0.02	0.23	(0.07 to 0.83)
Sensitivity analysis 2 (exclusion of errors types that could systematically influence the results)
Administration errors excluding lack of ID control, ID on the medicine box and barcode scanning	86	0.17	126	0.21	130	0.24	23	0.04	0.11	(0.06 to 0.18)
Procedural errors excluding lack of ID control, ID on the medicine box and barcode scanning	79	0.15	58	0.09	110	0.20	14	0.02	0.18	(0.09 to 0.35)
Sensitivity analysis 3 (exclusion of manually handled medicine at follow-up)†							(N=460)
Administration errors	191	0.37	215	0.35	192	0.35	69	0.15	0.36	(0.24 to 0.54)
Administration errors excluding ‘omission of dose’	190	0.37	164	0.27	187	0.34	66	0.14	0.53	(0.35 to 0.79)
Administration errors excluding lack of ID control, ID on the medicine box and barcode scanning	86	0.17	126	0.21	130	0.24	19	0.04	0.11	(0.06 to 0.19)
Clinical errors	10	0.02	71	0.12	21	0.04	7	0.02	0.06	(0.20 to 0.18)
Clinical errors excluding ‘omission of dose’	9	0.02	16	0.03	16	0.03	4	0.01	0.20	(0.50 to 0.77)
Procedural errors	185	0.36	153	0.25	173	0.31	62	0.13	0.58	(0.38 to 0.87)
Procedural errors excluding lack of ID control, ID on the medicine box and barcode scanning	147	0.29	73	0.12	154	0.28	15	0.03	0.26	(0.14 to 0.48)

*In the sub-analysis, the sum of clinical and procedural errors exceeds the number of administration errors. This is due to a small number of doses with both a clinical and a procedural error (in the control ward: four doses at baseline and nine doses at follow-up; in the intervention ward: two doses at baseline and one dose at follow-up).

†Number of doses that were handled manually at follow-up in the intervention ward was 110, corresponding to 19% of the 570 observed doses (n=460). These were excluded in sensitivity analysis 3 to investigate their potential impact on the final results.

The sub-analysis of clinical and procedural errors separately showed that the clinical error rate decreased from 0.04 to 0.02 in the intervention ward, whereas an increase from 0.02 to 0.12 was seen in the control ward, resulting in a reduced risk of errors of 94%: OR 0.06 (0.02 to 0.17). The rate of procedural errors decreased from 0.31 to 0.16 in the intervention ward and from 0.36 to 0.25 in the control ward. The decrease in procedural error rate was non-significant: OR 0.7 (0.48 to 1.04).

The control ward displayed a high prevalence of the error type ‘omission of dose’ at follow-up (table 2), which could potentially influence the results. In table 4, sensitivity analysis 1 showed that the rate of administration errors decreased from 0.34 to 0.17 in the intervention ward and from 0.37 to 0.27 in the control ward, resulting in an overall reduced risk of errors of 46% in the intervention ward: OR 0.64 (0.43 to 0.93). Excluding ‘omission of dose’ (clinical errors), the analysis showed a reduction in error rate from 0.03 to 0.02 in the intervention ward compared with an increase from 0.02 to 0.03 in the control ward. This corresponded to a reduced risk of 77% of clinical errors: OR 0.23 (0.07 to 0.83).

In sensitivity analysis 2, errors related to identification procedures were excluded, resulting in a reduction in error rate from 0.24 to 0.04 in the intervention ward and an increase from 0.17 to 0.21 in the control ward. This corresponded to an overall reduced risk of errors of 89% in the intervention ward (OR 0.11 (0.06 to 0.18)), supporting the effectiveness of the AMS in reducing the risk of errors. When errors in the identification procedure were excluded from the analysis, the effect of the AMS on procedural errors became statistically significant: OR 0.18 (0.09 to 0.35).

At follow-up, 110/570 of the doses were handled manually. This corresponded to 19% of the doses being dispensed manually because of technical limitations of the AMS. According to sensitivity analysis 3, exclusion of these doses isolated the effect of the AMS and resulted in a further reduction of total number of errors, thus increasing the beneficial effect of the AMS. The exclusion of manually handled doses did not affect the results of clinical errors, but improved the results of procedural errors from non-significant to significant: OR 0.58 (0.38 to 0.87).

To investigate why the AMS did not prevent all clinical errors, an analysis was conducted in which clinical errors were combined with the procedural error type ‘lack of barcode scanning’. This analysis showed that no clinical errors occurred in doses where PDA scanning was used correctly. The number of clinical errors in the intervention ward at follow-up was thus explained by a procedural error (lack of PDA scanning).

Discussion

The AMS effectively reduced the overall risk of errors in the medication administration process by 57% in the intervention ward compared with the control ward.

The reduction of the medication administration error rate found in our study was similar to the findings of other studies investigating the effect of technological interventions. A recent French study by Cousein and colleagues found an overall reduction of medication administration errors of 53% on implementation of an automated distribution system.⁸ A reduction in the medication administration error rate of 58% and 56% was found by De young et al¹⁶ and Helmons et al,¹⁷ respectively, investigating the effect of BCMA.Bonkowski et al¹⁸ found a relative error rate reduction of 80.7% on implementing an integrated electronic medical record with BCMA capacity interventions. However, the AMS used in our study combined prescription control, automated dispensing and BCMA, and it was thus not possible to isolate the effects of the different elements of the system. It is very likely that errors occurring in the dispensing stage (eg, wrong dose or wrong strength per unit) were reduced by the prescription control and automated dispensing, whereas barcode scanning primarily improved the patient identification processes and functioned as an extra control in the administration stage. Altogether the results from our study support the tendency of positive effects of technological interventions in the medication administration process found in previous studies.

While our intervention study was primarily targeted at medicine in tablet form, previous studies have found intravenous therapy to be associated with a higher prevalence of medication errors compared with orally administered medications.^{19 20} It would be relevant to include intravenous medicine in future technological endeavours to reduce medication errors.

Medication administration errors are not always harmful, but are in varying degree found to be associated with adverse events potentially leading to inconvenience, disability or death. ^{6 21–23} Adverse events could thereby result in longer hospitalisation or readmission, thus not only increasing the costs for the patient but also increasing healthcare costs to society.

Strengths and limitations of the study

This study was based on a thoroughly prepared protocol for observations that were performed by the same three observers at baseline and follow-up. We used clear definitions of errors, and our classification of error types was based on previously published studies.^{13 16 24} The participating wards were recruited within the same department, and the observational visits and background characteristics showed the wards to be highly comparable, thus minimising the risk of confounding factors such as staff stress level and differences in medication profile. To account for potential biases and ensure the validity of the final results, thorough statistical analyses were performed and are presented.

Because observations were performed by unblinded pharmaceutical staff, this could possibly have introduced bias, as they could have an interest in positive results of the pharmaceutical technology, thus leading to an overestimation of the effects.

As data were collected by three observers, it may be relevant to compare error rates between observers. However, as we did not record observer ID, we could not include observer as a covariable in the logistic regression.

The study did not use a disguised technique. The ward nursing staff were told that observations were performed to document changes and effects of the AMS on working procedures and medication safety. This could have influenced the results if the nurses made greater efforts during medication administration while being observed. However, this potential bias has previously been found to be of limited importance.⁶

Average length of stay had increased in the intervention ward at follow-up, meaning that fewer patients stayed longer in the ward. This may have resulted in staff becoming more familiar with their patients’ medication and may also have reduced staff stress levels, possibly confounding the results in favour of the AMS.^{25 26} Changes in length of stay were minor and were therefore considered to have little influence on the final results.

The intervention ward was not chosen at random but was selected by the departmental management. It is unclear what motivated the selection, but any unforeseen differences in ward culture, management or other organisational factors may have introduced bias into the study findings. Recruitment of two wards within the same department ensured high comparability, but also exposed the control ward to possible influence from the changes undertaken in the intervention ward.

Surprisingly, the control ward displayed a very high prevalence of ‘omission of dose’ errors at follow-up. This may reflect a lack of care with the prescriptions and failure to update prescriptions in the eMAR by the ward physicians, where ‘old’ prescriptions should be deleted from the eMAR on prescribing of new medication and/or changing the treatment of the patient. If this was the case, the nurses intentionally over-rode the physician's prescription when they believed it was out of date. This was seen in a few cases: for example, prednisolone was prescribed twice to the same patient at the same administration time, and the omission of dose actually prevented the patient from receiving a double dose. In this way, some of these omission errors may have been prescription errors rather than administration errors. To account for the influence of this specific error type, sensitivity analyses were performed excluding ‘omission of dose’. This resulted in a slight decrease in the effectiveness of the AMS, but the difference was still statistically significant.

Implications for practice

The combination of the eMAR, automated dispensing and BCMA appears to be effective in reducing the number of errors in the medication administration process and could be usefully implemented in similar hospital settings.

The effect of technological interventions on patient safety has been investigated in a variety of combinations and settings. A further factor for decision-makers to consider before implementing new initiatives is the cost of the intervention. Only a few studies have evaluated the cost differences between different alternatives for improvement of patient safety in relation to medication errors.^{9 24} Such economic evaluations are highly relevant to decision-making when scarce resources within the healthcare system are being prioritised.

Conclusion

An AMS consisting of technical prescription control, automated dose dispensing and bedside barcode scanning that was implemented in a haematological ward was effective in reducing the error rate in medication administration. This reduction was statistically significant, and sensitivity analyses supported the effect of the AMS.

What this paper adds.

• What is already known on this subject

• Adverse events associated with medication are one of the largest causes of harm to hospitalised patients. The medication administration process is an important area for safety improvement.

• The effect of automated drug dispensing has been tested previously, but with inconsistent results.

• A recent review by Tsao et al concluded that decentralised automated dispensing devices had limited potential to reduce medication errors, and that the impact of such devices is highly institution-specific.

• International studies have suggested that patient identification and alignment with the medication administration record can reduce the number of medication administration errors.

• Little is known of the effects on medication administration errors of combining several technologies such as electronic medication administration records (eMARs), automated drug dispensing, and barcode-assisted medication administration (BCMA).

What this study adds

• This study investigated an intervention where the use of eMARs, automated dispensing and BCMA was combined.

• The combination was effective in reducing the error rate in the medication administration process.

• The automated medication system could be usefully implemented in similar hospital settings to improve patient safety.