Efficacy of observational hand hygiene audit with targeted feedback on doctors’ hand hygiene compliance: A retrospective time series analysis

Abstract

Background:

Healthcare-associated infection compromises patient safety. Compliance with hand hygiene (HH) guidelines has been shown to be an effective method of reducing infection; however, it remains suboptimal and poorer among doctors compared to other healthcare workers. The aim of this study is to determine the relationship between an individualised observational hand hygiene audit (OHHA) and feedback intervention with observed HH compliance.

Methods:

We used a retrospective interrupted time series design using OHHA data from a five-year period, 2011–2015. OHHA indicated poorer HH compliance among doctors than other healthcare workers in a 345-bed acute private hospital. An increase in orthopaedic surgical site infection prompted additional auditing of the orthopaedic unit further identifying substandard HH compliance among orthopaedic surgeons. In addition to ongoing HH interventions, an individualised hand hygiene audit and feedback intervention focusing on consultant orthopaedic surgeons was implemented. Observed HH compliance improved. The intervention was then extended to include all consultant doctors at the study site. Audit was implemented by trained clinical nurse managers during clinical rounds. Written audit feedback was provided by the infection prevention and control team.

Results:

HH compliance increased significantly among both orthopaedic surgeons and other consultant doctors, P < 0.05.

Conclusion:

An individualised audit and feedback intervention was effective in improving compliance. Incorporation of OHHA with individualised feedback into routine daily practice needs to be considered as a quality improvement opportunity. This study has the potential to inform other audit and feedback interventions to maximise effectiveness and ensure implementation.

Background

Healthcare-associated infection (HCAI) is a worldwide threat to patient safety with prevalence estimated at 6–7% and 10% in high-income (Zarb et al., 2012) and low-income (Allegranzi et al., 2017) countries, respectively. Prevention of HCAI requires a multifactorial approach integrating patient care, infection prevention strategies, performance improvement and appropriate resources with an organisational culture focussed on patient safety (Smiddy et al., 2015). It is believed that that prevention of HCAI is feasible in 50–70% of cases (Umscheid et al., 2011). Compliance with hand hygiene (HH) guidelines has been shown to be an effective method of reducing HCAI (Pittet et al., 2000). However, compliance with recommended HH practices is reported to be suboptimal and poorer among doctors than other groups of healthcare workers (Erasmus et al., 2010).

The World Health Organization (WHO) recommends implementation of observational hand hygiene audit (OHHA) and feedback in addition to system change, training and education, visual reminders and promotion of institutional safety climate, as part of a multimodal strategy to improve hand hygiene in the clinical setting (WHO, 2009a). Implementation of OHHA involves a trained observer monitoring and recording the HH practices of healthcare workers consistent with the WHO ‘5 moments for hand hygiene’ with results presented as a proportion of compliance. The OHHA intervention utilises a standardised implementation methodology (WHO, 2009b) and is carried out by trained auditors who have achieved a Kappa coefficient ⩾ 0.8 on testing, to ensure inter-rate reliability as per the Cochrane Data Collection Checklist (Cochrane EPOC Group, 2002).

OHHA has been adopted internationally and its efficacy has been reported (Staines et al., 2018). However, there is variation across categories of staff, with less improvement reported among doctors than others (Azim et al., 2014). OHHA fits with the audit and feedback family of quality improvement strategies (Ivers et al., 2014a). The delivery of audit and feedback interventions is complex with different elements influencing outcome including, the characteristics of the recipient, the setting, the type of feedback and organisational characteristics (Ivers et al., 2014b). It has been suggested that HH interventions could be improved by customising feedback to recipients’ characteristics, such as professional discipline (Reich et al., 2015).

There has been a focus on hand hygiene improvement in the study facility since 2001. This included training and education, provision of resources and increased signage. OHHA using the WHO audit methodology (WHO, 2009b) commenced in 2010 in the study site. At that time, observed HH compliance for all categories of healthcare workers in the hospital was 43% (95% confidence interval [CI] = 38–47). However, it was noted that compliance among doctors was lower at 10% (95% CI = 4–21). An increase in orthopaedic implant surgical site infection was noted at the end of 2010 prompting an increased focus on OHHA on the orthopaedic unit in January 2011 resulting in implementation of more frequent audit and feedback. By quarter four of 2011 observed HH compliance for all staff on the unit was 81% (95% CI = 71–89); however, orthopaedic surgeons’ observed HH compliance remained poorer at 49% (95% CI = 42, 56). This led to the introduction of monthly OHHA and feedback to the group of orthopaedic surgeons by the infection prevention and control team (IPCT) in October 2011. This feedback took the form of written anonymised, generic monthly reports to the orthopaedic surgeons.

The focus on the surgeons allowed confirmation of poor practice and an impetus to add additional measures. In January 2012, in response to poor compliance and continued lack of improvement, OHHA was augmented with individualised feedback to each orthopaedic surgeon. The decision to proceed with the targeted individualised OHHA approach was made by a multidisciplinary team and was supported by the management team and the orthopaedic surgeon’s users group. The orthopaedic unit’s clinical nurse manager facilitated auditing during the orthopaedic surgeon’s daily clinical rounds. Once individualised OHHA of orthopaedic surgeons was established in the orthopaedic unit it was then extended hospital wide for all consultant doctors in October 2012.

While the value of individualised audit feedback has been reported (Stewardson et al., 2016), the efficacy of individualised feedback to doctors needs to be explored further. The aim of this study is to assess the relationship between an individualised OHHA and feedback intervention with doctors observed HH compliance.

Methods

Study design

A retrospective interrupted time series study design.

Participants and setting

Orthopaedic surgeons (n = 4 in 2012) and consultant doctors (excluding orthopaedic surgeons) (n = 61 in 2012) working at a 345-bed acute private hospital in Ireland were included.

Intervention

The intervention consisted of individualised OHHA with written feedback (Figure 1). Implementation of OHHA was facilitated by the clinical nurse managers in each area during doctors’ clinical rounds. All auditors were trained using the WHO audit tool (WHO, 2009b). Unit-based auditors were not Kappa coefficient (Kraemer, 2015) tested unlike the IPCT who had previously undergone training and Kappa testing as part of a national quality improvement programme. However, the data collected were checked internally by comparing it with the infection prevention and control nurse national audit data which are collected, analysed and reported biannually.

Figure 1.

Individualised OHHA and Feedback Intervention Diagram.

Verbal feedback was provided to individual consultant doctors daily by the clinical nurse managers. Monthly written individual reports were paper-based and posted to doctors by the infection prevention and control team. Cumulative compliance data were also anonymised with personal identifiers and emailed to all doctors. This ensured that all doctors were aware of their own results and the anonymised results of their peers. Quarterly doctor HH reports were emailed to each individual consultant doctor and were also sent to the hospital manager.

The intervention commenced on 2 January 2012 with the orthopaedic surgeons and was extended to all other consultant doctors on 1 October 2012. Data collection for this study completed on 23 December 2015. The intervention is illustrated in Figure 1 and fully described by the TIDier Statement (Hoffmann et al., 2014) in Appendix A.

Control

Pre-intervention OHHA data act as controls and range from 3 January 2011 to 28 September 2012. These data include orthopaedic surgeons and consultant doctors. Some minor changes in the population may have occurred both in the pre- and post-intervention periods due to new appointments and retirements.

Outcome measure

HH compliance is calculated as a percentage of the number of HH opportunities taken (numerator) divided by the number of HH opportunities available (denominator) for orthopaedic surgeons and consultant doctors (WHO, 2009a).

Data analysis: We compared pre- and post-intervention compliance with recommended HH practices using an interrupted time series analysis. A generalised linear model was used to implement segmented regression analysis of the interrupted time series data. Several models were compared and the most parsimonious selected placing the breakpoint before the intervention on 31 December 2011 for orthopaedic surgeons and 31 September 2012 for all other consultant doctors. Results are presented as estimates with 95% CIs. All tests were two-tailed and evaluated at the 0.05 level of significance. Analysis was carried out using STATA data analysis statistical software (StataCorp, 2015).

Ethical approval

Ethical approval for the study was granted by the relevant Ethics Committee.

Results

There were 2686 opportunities monitored among orthopaedic surgeons and 11,265 opportunities monitored among consultant doctors (Table 1). Orthopaedic surgeon HH compliance increased from 59% (95% CI = 36–79) in quarter one of 2011 before the intervention to 100% (95% CI = 90–100) in quarter four of 2015. Consultant doctor HH compliance increased from 55% (95% CI = 42–67) to 94% (95% CI = 92–96) in the same time period (Table 1).

Table 1.

Observed opportunities and percentage compliance with hand hygiene 2011–2015.

Year / Quarter (Q)	Observed opportunities for hand hygiene and % compliance
	Orthopaedic surgeons		Consultant doctors
	Numerator / Denominator	Compliance (% (95% CI))	Numerator / Denominator	Compliance (% (95% CI))
2011 Q1	13/22	59 (36–79)	35/64	55 (42–67)
Q2	No data	-	146/206	71 (64–77)
Q3	2/6	33 (4–72)	149/192	76 (71–83)
Q4	92/187	50 (42–56)	208/316	66 (60–71)
2012 Q1	42/62	68 (55–79)	228/323	70 (65–75)
Q2	9/15	60 (32–84)	292/351	83 (79–87)
Q3	16/30	53 (34–72)	135/227	59 (53–66)
Q4	548/590	93 (90–95)	885/1041	85 (83–87)
2013 Q1	386/386	100 (99–100)*	680/796	85 (83–88)
Q2	311/312	99 (98–100)	744/849	88 (85–90)
Q3	299/306	98 (95–99)	622/761	82 (79–84)
Q4	153/153	100 (98–100)*	1237/1457	85 (83–88)
2014 Q1	102/102	100 (97–100)*	749/867	86 (84–89)
Q2	40/40	100 (93–100)*	400/450	89 (86–92)
Q3	104/104	100 (97–100)*	486/530	91 (89–94)
Q4	135/135	100 (98–100)*	657/692	95 (93–96)
2015 Q1	84/84	100 (96–100)*	447/474	94 (92–96)
Q2	65/65	100 (95–100)*	479/500	96 (94–97)
Q3	51/51	100 (94–100)*	568/611	93 (91–95)
Q4	36/36	100 (92–100)*	526/558	94 (92–96)

^*One-sided 95% confidence interval (CI).

Observed HH compliance improved in both groups over the study period. There was a significant rate of increase in both groups in the pre- and post-intervention periods (P < 0.05) (Table 2, Figure 2).

Table 2.

Quarterly increase in observed hand hygiene compliance before and after individualised OHHA and feedback intervention with 95% confidence intervals.

Category of doctor	Start of individualised OHHA and feedback	Slope before	P value	Slope after	P value	P value of change in slope
Orthopaedic Surgeons	02/01/2012	0.07 (0.03–0.11)	0.02	0.01 (0.00–0.03)	0.00	0.02
Doctors	01/10/2012	0.03 (0.00–0.06)	0.02	0.02 (0.00–0.03)	0.02	0.41

Figure 2.

Hand hygiene interventions and observed compliance.

HH compliance increased significantly per quarter amongst orthopaedic doctors in the pre-intervention period (0.07, P = 0.02). The rate of improvement continued to increase significantly after the intervention (0.01, P = 0.00). The difference in the rate of change per quarter was significant (P = 0.02) (Table 2). HH compliance increased significantly per quarter amongst other consultant doctors in the pre-intervention period (0.03, P = 0.02). The rate of improvement continued to increase significantly after the intervention (0.02, P = 0.02). The difference in the rate of change per quarter was not significant (P = 0.41) (Table 2).

Discussion

Observed HH compliance increased significantly among both groups of doctors included in the study. However, a greater increase was noted among orthopaedic surgeons before the intervention with a significant difference in the change in slope. This could be because this group were improving from lower baseline compliance (Reich et al., 2015). The slower rate of change after the intervention is likely to be due to the ‘ceiling effect’ that is met in the attempt to achieve 100% compliance (Wogalter et al., 1999).

Evidence-based audit and performance feedback is recommended to inform practice improvement (Ivers et al., 2014b). Observational audit as part of multimodal interventions has been shown to improve HH compliance (Rosenbluth et al., 2016). In this study, individualised audit and feedback identified the need for additional education among doctors which was facilitated by the infection prevention and control team. Doctors’ willingness to engage in the HH educational sessions was noted to improve once poor performance was identified and fed back. However, people react differently to interventions therefore providing a targeted individualised approach in conjunction with other quality improvement strategies has been demonstrated to improve not only individual but also organisational compliance (Gould et al., 2018).

Many factors influence healthcare workers’ decision-making regarding HH, including, knowledge, skills, motivational factors and the work environment (Smiddy et al., 2015). Implementation of HH quality improvement interventions in this study demonstrate integration of these factors into practice through provision of education, monitoring, feedback, resources and organisational support. Support from senior management was essential to the implementation of the individualised OHHA intervention. Organisational management collaboration and tailoring of quality improvement HH interventions positively influence implementation (McInnes et al., 2014). Inclusion of both the management team and the orthopaedic surgeons from initiation regarding decisions and implementation of actions to reduce the incidence of surgical site infection was essential to the success of this HH intervention. Provision of HH audit reports to the management team enabled substandard results to be addressed through education, discussion and continued audit (Hysong et al., 2006).

Facilitation of audit during routine clinical practice by staff that were based in clinical areas may have reduced the risk of the Hawthorne effect (McCambridge et al., 2014) and allowed daily auditing on all clinical areas without the need to increase the infection prevention and control resources. The facilitation of auditing by trained clinical nurse managers made this HH improvement intervention feasible. Without the support of the clinical nurse managers it would have been impossible for the IPCT to implement standardised individualised OHHA and feedback for all the consultant doctors consistently. While the use and efficacy of multidisciplinary audit teams has been reported (Son et al., 2011), incorporating OHHA and feedback into routine daily clinical care needs to be explored further.

This individualised OHHA and feedback intervention was introduced to respond to a clinical need with the aim of improving patient safety. Internal validity in this study may have been influenced by the impact of other HH improvement practices which could have resulted in confounding. Changes in the study environment that may have influenced the outcomes included increased availability of alcohol-based hand rub and additional HH education. Communication between the participants in both groups may also have influenced HH behaviour and practice improvement. In addition, it is acknowledged that natural improvements in behaviour occur as professionals develop their clinical skills over time. A large sample size of observed opportunities post intervention strengthens the validity of the findings. Despite the retrospective nature of the study design, the interventions were implemented in a real-time situation to deal with a clinical issue. To support intervention and implementation fidelity a full description of the intervention is available in a TIDier checklist (Hoffmann et al., 2014; Appendix A).

While these study findings are based on a single-centre they are generalisable to other healthcare settings. The use of targeted audit and feedback, and organisational commitment to intervention implementation, is an important consideration for IPCTs. The multidisciplinary approach to surgical site infection resulted in an impetus for overall HH improvement within the hospital and motivated support from orthopaedic surgeons and management. While it would not be feasible for the IPCT to audit every doctor individually, implementation was possible with the support of senior nursing staff. Implementation of individualised OHHA and feedback resulted in significant improvements in doctor’s observed HH performance. A qualitative study establishes the factors which influenced these doctors’ HH (Smiddy et al., 2019). Individualised OHHA and feedback to doctors is to continue in the study site to further evaluate this intervention. The concept of cross discipline or peer auditing should be explored further as a feasible approach to improving practice. This study has the potential to inform other audit and feedback interventions to maximise effectiveness and ensure implementation.