Nurse compliance with a protocol for safe injectable medication administration: comparison of two multicentre observational studies

Abstract

Objectives

Medication administration errors with injectable medication have a high risk of causing patient harm. To reduce this risk, all Dutch hospitals implemented a protocol for safe injectable medication administration. Nurse compliance with this protocol was evaluated as low as 19% in 2012. The aim of this second evaluation study was to determine whether nurse compliance had changed over a 4-year period, what factors were associated over time with protocol compliance and which strategies have been implemented by hospitals to increase protocol compliance.

Methods

In this prospective observational study, conducted between November 2015 and September 2016, nurses from 16 Dutch hospitals were directly observed during intravenous medication administration. Protocol compliance was complete if nine protocol proceedings were conducted correctly. Protocol compliance was compared with results from the first evaluation. Multilevel logistic regression analyses were used to assess the associations over time between explanatory variables and complete protocol compliance. Implemented strategies were classified according to the five components of the Systems Engineering Initiative for Patient Safety (SEIPS) model.

Results

A total of 372 intravenous medication administrations were observed. In comparison with 2012, more proceedings per administration were conducted (mean 7.6, 95% CI 7.5 to 7.7 vs mean 7.3, 95% CI 7.3 to 7.4). No significant change was seen in complete protocol compliance (22% in 2016); compliance with the proceedings ‘hand hygiene’ and ‘check by a second nurse’ remained low. In contrast to 2012, the majority of the variance was caused by differences between wards rather than between hospitals. Most implemented improvement strategies targeted the organisation component of the SEIPS model.

Conclusions

Compliance with ‘hand hygiene’ and ‘check by a second nurse’ needs to be further improved in order to increase complete protocol compliance. To do so, interventions focused on nurses and individually tailored to each ward are needed.

Strengths and limitations of this study.

• This is a comprehensive observational study on nurse compliance with the protocol for safe injectable medication administration based on a total of 372 direct observations of intravenous medication administrations within a representative random sample of 16 Dutch hospitals.

• This study provides insight into protocol compliance changes over a 4-year period by a comparison with the results from the first evaluation study conducted in 2011/2012.

• In addition to compliance rates, an overview of implemented hospital strategies was obtained to determine what efforts Dutch hospitals made to improve protocol compliance.

• In this study, medication administration errors and potential harm resulting from these errors were not measured.

Introduction

Injectable medication therapy is considered an essential component of current healthcare delivery. Over 90% of all hospitalised patients receive some form of this therapy.¹ Injectable medication therapy comprises medication that is administered directly into body tissue or the circulatory system.² It includes primarily intravenous medication infusions and injections, but also other administration routes such as subcutaneous and intramuscular injections. The benefits of intravenous medication, such as an immediate therapeutic effect and the possibility to reach therapeutic drug levels in a short period of time, provide at the same time a high risk for patient harm.^{1 3–6} This high risk arises from the fact that errors with intravenous medication are almost irreversible. Errors with intravenous medication occur frequently during hospital admission. The probability of making at least one error at any stage of the intravenous medication process is 73%.⁶ Besides, most errors occur during medication administration. These medication administration errors (MAEs) can be defined as ‘deviations of a drug from a physician’s prescription, the hospital’s policy or the manufacturer’s instructions’.⁷ It is five times more likely that an MAE occurs when intravenous medication is administered compared with non-intravenous medication.⁴

Using a protocol for safe administration of injectable medication contributes to a reduction in medication errors in hospitals.^8–12 In Dutch hospitals, a protocol for safe administration of injectable medication was implemented in 2009 as part of the National Patient Safety programme.¹³ This prevailing protocol contains 35 proceedings for preparing and 25 proceedings for administering injectable medication, and is based on the ‘five rights’ of safe medication administration (right patient, right medication, right dose, right route, right time).³ The goal of the National Patient Safety programme is to achieve 100% compliance with this protocol. In other countries, comparable protocols have been implemented, and protocol steps such as ‘patient identification’ and ‘hand hygiene’ are generally seen as important and included in these protocols.^14–16

Between November 2011 and December 2012, Schilp et al¹⁷ conducted a prospective observational study in 19 Dutch hospitals to evaluate the implementation of the Dutch protocol for safe administration of injectable medication. In total, 2154 intravenous medication administrations by nurses were directly observed, monthly, during a 12-month period, and complete compliance with the protocol was observed in 19% of the observations. The least conducted proceedings were found to be ‘patient identification’, ‘hand hygiene’ and ‘check by a second nurse’. Schilp et al¹⁷ concluded that the implementation of the protocol was inadequate and recommended that more time was needed to increase protocol implementation.

In response to the results of the evaluation study of 2012, Dutch hospitals—supported by the Dutch associations of nurses and hospital pharmacists—proposed follow-up actions to improve protocol compliance—for example, appointing an injectable medication nurse champion, whose responsibility would be to supervise the implementation of the protocol at the hospital and ward levels.¹⁸ In addition, barcode medication administration (BCMA) systems were introduced and increasingly used in Dutch hospitals. A BCMA system enables nurses to scan the barcode on the patient’s wristband and/or medication label to improve compliance with patient identification. Implementation of BCMA systems in hospitals has been associated with a decrease in MAEs.¹⁹ Also, the protocol compliance was a focus of external safety audits by the Dutch Inspectorate of Health Care. Whether these various follow-up actions had impact on nurse compliance with the protocol for safe injectable medication administration is unknown.

Since the most recent evaluation study was conducted 4 years ago, and tracking performance is helpful in determining protocol implementation,¹⁴ we conducted a second prospective observational study to evaluate the current implementation of the protocol for safe injectable medication administration in Dutch hospitals. In addition, we wanted to know which factors are associated over time with complete protocol compliance, since compliance can be influenced by various characteristics (ie, organisational, individual and environmental).^{20 21} Therefore, the aims of this study were (1) to determine whether complete protocol compliance and compliance with individual proceedings have changed compared with the first evaluation study conducted in 2011/2012, (2) to investigate which hospital and administration factors are associated over time with complete protocol compliance and with three individual protocol proceedings as compared with the first evaluation, and (3) to provide an overview of improvement strategies implemented by hospitals to increase protocol compliance.

Methods

Design and setting

For the purpose of this second evaluation, a prospective observational study was conducted in 16 Dutch hospitals from November 2015 to September 2016. These 16 hospitals included one university hospital, six tertiary teaching hospitals and nine general hospitals. The hospitals were randomly selected to participate and originated from the representative (stratified on area and type of hospital) sample of 19 hospitals that participated in the first evaluation in 2011/2012. Of these 19 hospitals, 13 agreed to participate in the second evaluation. To assure a representative measurement for all Dutch hospitals and to gain a sufficient sample size for comparison with the first evaluation, three new hospitals were selected from a new random sample. The main reasons not to participate in the second evaluation were time constraints due to the implementation of a new hospital electronic health record system, and the fact that a similar measurement had recently been conducted by hospital staff. The Strengthening the Reporting of Observational Studies in Epidemiology guideline for reporting observational studies was used to enhance accurate and complete reporting of this study.²²

Participants

Nurses working on three hospital wards—intensive care (IC), internal medicine and (general) surgery—were directly observed during the administration of intravenous medication. These three ward types were considered to be representative of protocol compliance in the whole hospital. All (trainee) nurses involved in the administration of intravenous medication on the study wards were eligible for this study. Verbal consent from the nurses and (wherever possible) the patients was obtained to conduct the observation. Nurse managers of the participating wards were fully informed about the purpose of the study. Nurses were informed about the goal of the observations (correct administration of injectable medication) but not about the specific protocol proceedings being observed, in order to prevent bias (Hawthorne effect).²³ However, nurses could be aware of the observed proceedings on the observation form, since all proceedings follow the current protocol, which is publicly accessible in all hospitals. Participation in the study was voluntary and anonymous for nurses; if a nurse did not want to participate, then he/she was not observed.

Data collection

Data collection was similar to the first evaluation study.¹⁷ In summary, to determine complete protocol compliance and compliance with individual proceedings, direct observations were conducted for patients ≥18 years of age during the intravenous medication rounds from 06:00 to 22:00. Parenteral nutrition, intravenous chemotherapy and acute medications were not observed because for these medications other administration protocols apply. At each hospital, one trained nurse researcher (BS) conducted the observations during two consecutive weekdays. A standardised observation form was used to evaluate performances of the individual proceedings. The form included the nine most important and identifiable administration proceedings from the protocol, predetermined and described by an expert team (table 1). All correctly conducted proceedings were marked on the observation form. Moreover, a minimum of three nurses per ward and a maximum of three administrations per nurse were observed to correct for between-person variation.

Table 1.

Protocol proceedings for administering injectable medication*

Step	Explanation
Check medication	Checking the drug on the basis of a medication list or distribution list
Prepare administration	Preparation of administration: setting pump and speed of injection
Collect materials	Gathering the needed materials and checking the administration label
Patient identification	Identifying the patient either electronically or by checking the name, date of birth, patient number and type of medication
Hand hygiene	Hand disinfection before administration or wearing gloves during administration
Check flow infusion	Checking the intravenous medication line before administering the medication
Check pump mode	Checking or setting the pump mode before administering medication
Check by a second nurse	Having a second nurse check the patient, medication, administration route and administration rate
Sign medication order	As the administrator, signing the medication order

*As published in Schilp et al.¹⁷

To detect a 10% improvement in protocol compliance at a 5% significance level, at least 300 observations were needed during the second evaluation (ß=0.8). This means 20–21 observations per hospital and 6–7 observations per ward. Consequently, only one data collection moment per hospital was needed and planned. During the first evaluation, data were collected during 10 moments (once a month) per hospital to follow process variation over different months and calculate an average compliance rate.

Protocol compliance

The primary outcome was the complete protocol compliance with the Dutch injectable medication protocol. Each observed intravenous medication administration was scored (0–9) and then dichotomised into complete compliance (nine safety proceedings conducted) and incomplete compliance (≤8 safety proceedings conducted).¹⁷ The secondary outcomes were the mean number and percentage of correctly conducted individual proceedings, in particular compliance with ‘patient identification’, ‘hand hygiene’ and ‘check by a second nurse’. These three proceedings were the three least conducted protocol proceedings during the first evaluation.

Factors associated with protocol compliance

To determine factors associated over time with complete protocol compliance and selected individual protocol proceedings, additional variables were registered on the observation form: type of hospital (university, tertiary, general), type of department (general surgery, internal medicine, IC), time of administration (morning (05:00–12:00), afternoon (12:00–18:00) and evening (after 18:00)), type of administration (by intravenous infusion, bolus intravenous injection or intravenous syringe pump) and name and type of medication.

Improvement strategies implemented to increase protocol compliance

To identify improvement strategies implemented by the hospitals, two short interviews were conducted with a quality and safety officer and the head or senior nurse of each ward. During the first interview conducted during the intake, questions regarding the availability of an injectable medication champion, injectable medication education programmes and interruption prevention strategies (ie, do-not-disturb vests) were asked. The second interview followed after the observations and comprised questions regarding the availability and use of information technology to support the injectable medication administration process. In addition, local injectable medication administration protocols were collected to identify other potential improvement strategies. The identified strategies were classified according to the five components of the work system as described in the Systems Engineering Initiative for Patient Safety (SEIPS) model: organisation, technology and tools, person, tasks, and environment.^24–26 The SEIPS model provides a comprehensive theoretical framework for understanding interactions between the components in the work system, processes (eg, protocol compliance) and outcomes (eg, MAEs) in healthcare.²⁷

Data analysis

All results collected on the observation forms were entered in an online database: NETQuestionnaires. Descriptive statistics were used to describe hospital type, ward type, administration time, administration type and medication type. Differences between mean number of conducted protocol proceedings were tested with one-way analysis of variance statistics. Differences in the protocol compliance (complete protocol compliance: yes or no) were tested with χ² statistics.

To assess the associations over time between potential explanatory variables (ie, hospital type, ward type and administration time) and protocol compliance, separate univariate multilevel logistic regression analyses were conducted for four dependent variables: complete protocol compliance (yes/no), patient identification compliance (yes/no), hand hygiene compliance (yes/no) and check by a second nurse compliance (yes/no).²⁸ A three-level multilevel structure was used, whereby the observations were clustered within wards and the wards within hospitals. The explanatory variables were used as independent variables. The fixed effects for the first evaluation were the average value of the intercepts. The fixed effects for the second evaluation were the regression coefficients to the extent that the second evaluation deviated from the first evaluation. In all analyses, a corrected model was used with adjustment for the other two explanatory variables.

In addition, the between-hospital and ward-level variance was split into two elements, one for the first and one for the second evaluation. Also the covariation between both evaluations was modelled at the hospital and ward levels. This resulted in intraclass correlations (ICCs) for each evaluation separately, which indicated whether the relative contribution of the hospital and ward levels differed between both evaluations. Based on the variances and covariance, the correlation between participated wards was calculated.

Descriptive analyses were conducted using IBM SPSS Statistics V.20 and the multilevel analyses using MlwiN V.2.30 (University of Bristol). The multilevel logistic models were calculated using penalised quasi-likelihood second order (or when this failed, first order), with constrained level 1 variance. For all analyses, P values ≤0.05 were considered statistically significant.

Results

In total, 372 intravenous medication administrations were observed, with a range of 20–28 observations per hospital (table 2). Most observations had been conducted at general hospitals (57%), internal medicine (35%) and IC wards (35%), during the afternoon (65%), and of administrations by intravenous infusion (73%).

Table 2.

Descriptive statistics of intravenous medication observations during the two evaluation studies

	First evaluation 2011/2012	Second evaluation 2015/2016
Observations, n	2154	372
Hospitals, n	19	16
Range of observations per hospital, n	70–196	20–28
Type of hospital
University	297 (13.8)	22 (5.9)
Tertiary	750 (34.8)	139 (37.4)
General	1107 (51.4)	211 (56.7)
Type of department
Internal medicine	643 (29.9)	129 (34.7)
(General) surgery	771 (35.8)	112 (30.1)
Intensive care	671 (31.2)	131 (35.2)
Other	69 (3.2)	0 (0)
Administration time
Morning (06:00–12:00)	771 (35.8)	92 (24.7)
Afternoon (12:00–18:00)	1257 (58.4)	243 (65.3)
Evening (after 18:00)	126 (5.8)	37 (9.9)
Type of medication (most common)
Antibiotics	1323 (61.4)	236 (63.4)
Analgesics	167 (7.8)	38 (10.2)
Gastrointestinal medication	178 (8.3)	16 (4.3)
Anaesthetics	27 (1.3)	16 (4.3)
Electrolytes	83 (3.9)	14 (3.8)
Corticosteroids	85 (3.9)	11 (3.0)
Type of administration
By intravenous syringe pump	29 (1.3)	48 (12.9)
By bolus intravenous injection	66 (3.1)	51 (13.7)
By intravenous infusion	2059 (95.6)	273 (73.4)

Data are presented as n (%), unless stated otherwise.

Protocol compliance

Table 3 shows the mean number of correctly conducted protocol proceedings and percentages of intravenous medication administrations with complete protocol compliance during both evaluations. On average, more proceedings per intravenous medication administration were conducted during the second evaluation compared with the first evaluation: 7.6 (95% CI 7.5 to 7.7) vs 7.3 (95% CI 7.3 to 7.4) (P<0.001). However, no significant change was seen in complete protocol compliance during the second evaluation compared with the first evaluation: 22.3% (95% CI 18.1% to 26.5%) vs 19.4% (95% CI 17.7% to 21.1%) (P=0.194).

Table 3.

Comparison of the first and second evaluation study in conducting the complete protocol

	First evaluation 2011/2012	Second evaluation 2015/2016	P value
Conducted proceedings, mean (95% CI)	7.3 (7.3 to 7.4)	7.6 (7.5 to 7.7)	<0.001*
Complete protocol compliance, % (95% CI)	19.4 (17.7 to 21.1)	22.3 (18.1 to 26.5)	0.194†

*Tested by one-way analysis of variance.

†Tested by χ² statistics.

Three proceedings were least often conducted: ‘patient identification’ (80.1%), ‘hand hygiene’ (63.2%) and ‘check by a second nurse’ (47.3%) (figure 1). Compliance rates with the other six proceedings varied between 93% and 100%.

Figure 1.

Compliance percentages with the complete protocol and three individual proceedings within the first (n=2154) and second (n=372) evaluations. Results are presented with 95% CI. Compliance was tested by X² statistics. Compliance with the six other proceedings varied between 93% and 100%, and was significantly increased for ‘prepare administration’, ‘check flow infusion’ and ‘check pump mode’, and significantly decreased for ‘check medication’.

Compliance with ‘patient identification’ improved significantly from 61% (95% CI 58.0% to 62.1%) in the first evaluation to 80% (95% CI 76.1% to 84.2%) in the second evaluation (P<0.001). During the second evaluation, patient identification was conducted in three ways. First, 49% of the nurses identified their patient by a physical check (eg, asking the patient’s name, and/or date of birth, or by checking information on the patient’s wristband). Second, 16% of the nurses identified the patient by using a barcode scanner in addition to the physical check, or by only using a barcode scanner. Third, in 15% of the observations, all on IC wards, the nurse to patient ratio was one nurse per patient. Hence, patient identification was scored as conducted in all these observations.

Compliance with the proceedings ‘hand hygiene’ and ‘check by a second nurse’ remained unchanged. The ‘check by a second nurse’ comprises four subchecks: double-check on ‘right patient’, ‘right medication’, ‘right administration route’ and ‘right administration rate’. During the second evaluation, double-checking the ‘right patient’ (n=255, 69%), ‘administration route’ (n=227, 61%) and ‘administration rate’ (n=177, 48%) was conducted less often compared with double-checking the ‘right medication’ (n=353, 95%).

Factors associated with protocol compliance

The univariate associations over time between three potential explanatory variables (eg, type of hospital, ward type and time of administration) and four dependent variables (complete protocol compliance, compliance with patient identification, compliance with hand hygiene and compliance with check by a second nurse) were investigated. A positive association was found between all three explanatory variables and compliance with ‘patient identification’. Compliance with the proceeding ‘patient identification’ improved significantly over time for all the different administration times (morning, afternoon and evening) (table 4), all the different ward types (IC, internal medicine and (general) surgery) (online supplementary table 1) and in tertiary teaching hospitals (online supplementary table 2). Other investigated hospital and administration-related variables were not associated with complete protocol compliance or compliance with the other two analysed individual proceedings. Furthermore, multilevel analyses showed that the hospital variance became very small and was estimated as 0 (table 4). On the other hand, ward variance increased. For example, 0% (ICC=0.00) of the total variance in the association between ‘patient identification compliance’ and ‘administration time’ can be explained by individual hospitals and 50% (ICC=49.70) by individual wards (table 4). During the first evaluation, opposite results were found, in which the ICCs of hospital variance were high and the ICCs of ward variance were low. In addition, at the ward level, the correlation between the two evaluations was 0.52, indicating that wards having had a high compliance in the first evaluation also had a high compliance in the second evaluation. Vice versa, wards that had a low compliance in the first evaluation also had a low compliance in the second evaluation.

Table 4.

Multilevel analyses of the association between administration time and compliance with the proceeding ‘patient identification’ during the first and second evaluations

	First evaluation 2011/2012		Second evaluation 2015/2016
	n	Estimate (SE)	n	Estimate (SE)
Fixed effects
Patient identification in morning	771	0.19 (0.46)	92	1.97 (0.61)*
Patient identification in afternoon	1257	0.39 (0.45)	243	1.58 (0.53)*
Patient identification in evening	126	0.39 (0.55)	37	1.64 (0.76)*
Random effects
Hospital-level ICC		38.09		0
Hospital-level variance		3.24 (1.21)		0 (0)
Hospital-level covariance and correlation		0 (0); 0
Department-level ICC		23.27		49.70
Department-level variance		1.13 (0.34)		2.40 (0.78)
Department-level covariance and correlation		0.85 (0.46); 0.52

*P<0.05.

ICC, intraclass correlation.

Improvement strategies implemented to increase protocol compliance

Figure 2 shows nine identified strategies implemented by hospitals with the aim to improve compliance with the injectable medication administration protocol. Most strategies were classified according to the SEIPS model as targeting the organisation component (n=3), followed by tasks (n=2) and technology and tools components (n=2). Only one intervention targeted the person and one the environment component.

Figure 2.

Identified strategies implemented by the hospitals during the second evaluation (n=16 hospitals), classified according to the individual components of the Systems Engineering Initiative for Patient Safety model (eg, organisation, technology and tools, person, tasks, and environment). BCMA, barcode medication administration.

Hospitals implemented on average six strategies, ranging between four and nine strategies. Organisation component strategies were appointing an injectable medication champion (15 participating hospitals), conducting internal audits (14 participating hospitals) and having a buddy system in which two nurses double-check their buddies’ intravenous medication administrations (nine participating hospitals). Most appointed injectable medication champions were hospital pharmacists, and the way in which this task was performed varied greatly between hospitals. BCMA systems (eight participating hospitals) and smart pumps (seven participating hospitals) were the implemented tools and technology improvement strategies. Smart pumps are infusion pumps with software that creates a library of medication administration protocols.²⁹ A personal component-related strategy included training and education (eg, e-learning modules and introduction modules) for nurses to enhance their knowledge (16 participating hospitals). Task-related strategies included shifting the tasks of injectable medication preparation from nurses on hospital wards to pharmacy technicians in the (central) hospital pharmacy (11 participating hospitals) and adjusting the timing of the check by a second nurse to the beginning of a shift (10 participating hospitals). Finally, having policy regarding the recognisability of nurses during injectable medication administration (12 participating hospitals) was the only environmental component-related strategy identified. Most combined strategies were training and education, and appointing an injectable medication champion.

Discussion

Compliance with individual proceedings of the Dutch protocol on administering injectable medication has improved over 4 years, but complete protocol compliance did not significantly change. In 19% of the observations in 2011/2012, the protocol was completely conducted, compared with 22% in 2015/2016 (P=0.194). In contrast to the first evaluation study, differences in protocol compliance between wards were greater, and differences between hospitals were smaller. Furthermore, according to the SEIPS model, most improvement strategies targeted the organisation component of the injectable medication administration process.

Compliance with the proceeding ‘patient identification’ increased significantly to an average of 80%. Using a BCMA system to electronically identify patients may have contributed to the higher compliance rate of this proceeding in our study. Taliercio et al³⁰ showed that nurses experience using a BCMA system to identify patients as a major advantage. In our study, a BCMA system was implemented as a strategy in eight (50%) participating hospitals and used in 16% of all observations. Since an increasing number of Dutch hospitals will implement a BCMA system in the next few years and using BCMA will be further integrated in daily nursing practice, we expect that compliance with this proceeding will further increase. A reason for non-compliance with this proceeding can be that nurses believe they know their patients well enough not to ask the patients’ name and date of birth.³¹ Other observational studies on medication administration reported lower compliance rates (33%–80%), but did not specify whether identification was supported by a BCMA system.15 16 32–35

Compliance with the proceeding ‘hand hygiene’ remained unchanged (63%). This may be explained by the lack of improvement strategies specifically targeting hand hygiene compliance in the participating hospitals. The compliance of 63% in our study is comparable with the study of Helder et al,³⁶ which showed a hand disinfection rate during medication administration of 58% after a mutual feedback intervention. Improving hand hygiene remains a challenge in many hospital processes, not only during medication administration. A recent review showed that the overall mean hand hygiene compliance rate after interventions was 57%.³⁷ Huis et al³⁸ explored determinants of hand hygiene improvement strategies and showed that addressing knowledge, awareness, action control and facilitation is not enough to improve hand hygiene compliance. Baseline compliance rates of hand hygiene vary strongly in the literature (20%–60%).³⁹ Also, the increased compliance with hand hygiene appears temporary in most intervention studies. Huis et al³⁸ recommended that social influence, attitude, self-efficacy and attention (person component of SEIPS) should be taken into account in new strategies, and that they should preferably be focused on the whole nursing team.

Compliance with the proceeding ‘check by a second nurse’ also remained unchanged (47%). Of all four subchecks of this proceeding (eg, ‘right patient’, ‘right medication’, ‘right administration route’ and ‘right administration rate’), the subchecks on ‘right patient’ and ‘right medication’ were most often conducted. These subchecks are supported by barcode scanning systems while the subchecks on ‘right administration route’ and ‘right administration rate’ are not. Therefore, for these checks on route and rate of intravenous infusion, a second nurse at the patient’s bedside was necessary. This is a task that depends on nurse capacity and/or workload. In theory, the check by a second nurse for all intravenous medications has become a standard and critical proceeding. Alsulami et al⁴⁰ described that most healthcare professionals prefer the double-check, but that staff shortage can prevent for correctly conducting this proceeding. In practice, we observed that increased workload, indeed, may prevent this standard. Therefore, this proceeding must be prioritised in future studies. In order to facilitate the check by a second nurse, intervention strategies such as adjusting the timing of the check by a second nurse (10 hospitals) and having a buddy system (nine hospitals) have been implemented in the participating hospitals. However, qualitative studies on the check by a second nurse showed that the focus should lie on training and education, automating the proceeding and seeing the check by a second nurse as a method to share opinions.⁴¹

Using the SEIPS model for classifying strategies implemented by the hospitals revealed that most strategies targeted the organisation of the injectable medication administration process. Less strategies targeted the person and environment. This is in contrast with Berdot et al, who showed that most interventions aiming to reduce MAEs targeted technology and tools (eg, automated medication dispensing systems, BCMA systems) and the person (eg, interactive CD-ROM program or simulation-based learning). This can be explained by the fact that Berdot et al⁴² included seven studies, mostly randomised controlled trials, which had MAE rates as outcome measure. Our observational study identified current improvement strategies used in daily practice. Knowing that strategies are most often complex and multifaceted, it is recommended to determine potential barriers prior to implementing a strategy.⁴² These barriers can be found in all SEIPS components. Apparently, Dutch hospitals have been trying to overcome barriers in the injectable medication process by implementing mostly organisational strategies at the hospital level. This is, however, not enough to increase protocol compliance. Since most variations were seen at the ward level, rather than hospital level, future strategies should be tailored to individual wards. It is important to focus these strategies on individuals (eg, nurses, patients, families) and the environment. On the other hand, the protocol itself can also be a focus for discussion. Since two evaluation studies concluded that the implementation of the protocol has not yet been accomplished, it may be necessary to take a critical look at which proceedings are essential and whether the proceedings reflect all SEIPS components.

One of the strengths of this study is that more than 20% of all Dutch hospitals participated in one of the two evaluation studies, 19 during the first evaluation and 16 during the second evaluation. This random and representative sample ensures that the results can be generalised to the Dutch hospital setting. Furthermore, similar observation list, observation procedure and training of researchers were used during both evaluations, and 13 hospitals participated in both evaluations. Therefore, we could compare the two evaluations reliably. However, several uncertainties may have limited the generalisability of our results. First, this second study comprised one data collection moment compared with 10 data collection moments in 2011/2012. As a consequence, the compliance rate reflects one moment in time, compared with an average compliance rate. Nevertheless, we conducted more than the intended 300 observations, and on this basis we think the results reflect current nursing practice. Second, almost all observations (96%) were conducted by one researcher, which could have created error of leniency or severity (ie, rating observations, in particular, positively or negatively).⁴³ However, in our study, using one observer ensured that all administrations were measured in the same way and it appeared that the compliance rates were in line with previous studies. Third, no data about nurse-related characteristics (degree of education and years of experience) and workload-related characteristics (turnover rates, stability of the nursing workforce, stability of the nurse to patient ratio over the years and number of drugs to be dispensed per round per nurse) have been collected. This may have resulted in an incomplete overview of factors associated with protocol compliance. The nurse-related characteristics have not been collected because we used the same observation form as in the first evaluation, which did not include these characteristics. The workload-related characteristics have not been collected because these data appeared too complex and the way these variables are calculated varied per ward and per hospital. Fourth, not all injectable medications were included in the observations, only intravenous medications. Since chemotherapy, and less invasive injectable medication administration routes, such as intramuscular and subcutaneous injections, are increasingly used in hospitals, it would be recommendable to also observe administration of these types of injectable medications in the future. Fifth, the fact that nurses were aware of being observed may have resulted in more compliance. As a consequence, compliance rates could have been overestimated. This so-called Hawthorne effect is a known challenge within observational studies.⁴⁴ To minimise this effect in our study, the researcher was discreet during observations and did not give performance feedback during or after observations. Finally, since the information about implemented improvement strategies was collected during two interviews, it is uncertain how well these strategies are implemented in daily practice on the wards. Therefore, this information provides only a first impression. To be able to determine associations between strategies and protocol compliance, we would recommend to perform a new study aiming to observe the execution of the mentioned strategies on the wards.

In conclusion, our results show that conducting all nine proceedings included in the protocol for safe injectable medication administration by Dutch hospital nurses remains challenging. Importantly, compliance with patient identification during intravenous medication administration has improved and implementing BCMA systems may have contributed to this finding. Therefore, further implementation of BCMA systems in hospitals is recommended. Compliance with ‘hand hygiene’ and ‘check by a second nurse’ needs to be further improved in order to increase complete protocol compliance and reduce the risk of MAEs. To improve compliance with these proceedings, other interventions are needed, preferably focused on nurses, and individually tailored to each ward.