Summary
Background
The influence of the Hawthorne effect on hand hygiene compliance in an intensive care unit was assessed using covert and overt direct observation.
Methods
The observational study was conducted from February to November 2018 in a 24-bed adult intensive care unit in a 243-bed tertiary care hospital, in four periods (P): P-1, February 5-March 3, 29 h (covert) and P-2, March 15-April 16, 33 h (overt), prior to an educational campaign on hand hygiene; and P-3, August 27-September 28, 33 h (covert) and P-4, October 3-November 6, 35 h (overt), after the campaign. Three 20-min observation sessions were performed daily, randomly distributed in the morning, afternoon and evening shifts, including holidays and weekends. Hand hygiene compliance rates observed in Periods 2 and 4 were displayed on an electronic panel installed in the unit. Hand hygiene compliance was assessed according to the World Health Organization “My Five Moments for Hand Hygiene” guidelines.
Results
Before the campaign, the overall hand hygiene compliance rate was 31.95% (340/1064, covert) versus 68.10% (790/1160, overt), and afterwards was 56.11% (615/1096, covert) versus 80.98% (1086/1341, overt). The infection rate was reduced by 22.62% (18.87% versus 14.60%).
Conclusions
The Hawthorne effect and educational campaign markedly influenced compliance with hand hygiene recommendations. The results suggest that combining overt and covert observation methods, including regular feedback on hand hygiene compliance displayed on an electronic panel, may be a valid alternative to increase real hand hygiene compliance rates in hospital practice.
Keywords: Direct observation, Hand hygiene, Hawthorne effect
Introduction
Direct observation of hand hygiene practices in healthcare settings is currently considered the gold standard for monitoring hand hygiene compliance [1]. However, a major limitation of this method is the Hawthorne effect, i.e., subjects who knows they are being observed might behave differently, affecting study outcomes [2]. Although the Hawthorne effect often leads to overestimation of hand hygiene compliance in hospital practice, only a few studies have assessed the extent of the Hawthorne effect by direct observation [3,4]. This study assessed the influence of the Hawthorne effect on hand hygiene compliance in an adult intensive care unit by means of covert and overt observation methods, before and after an educational campaign on hand hygiene.
Methods
Prior to the study period, one closed-format, self-administered questionnaire by the World Health Organization (WHO), the Hand Hygiene Self-Assessment Framework (HHSAF), was applied in our hospital to assess the infrastructure available to promote hand hygiene and practice [5]. Briefly, the HHSAF is structured in five sections with 27 indicators that reflect the five components of the WHO Multimodal Hand Hygiene Improvement Strategy: system change, training and education, evaluation and feedback, reminders in the workplace, and institutional safety climate for hand hygiene. Based on a scoring system, the indicators are assigned values totalling 100 points within each five-element section, for a maximum overall score of 500 points. Based on the score achieved for the five components, the facility is assigned to one of four levels of hand hygiene promotion and practice: 1, inadequate (score 0–125); 2, basic (126–250); 3, intermediate (251–375); or 4, advanced (376–500) [1,5,6].
A prospective direct observational study was conducted from February to November, 2018 in a 24-bed adult intensive care unit (AICU) in a Brazilian 243-bed tertiary care hospital. The AICU is staffed by 18 physicians, 12 nurses, 48 nursing technicians, and 12 other health workers including physiotherapists, laboratory technicians, and radiology technicians. During the study period, data on infection rates and alcohol gel consumption in the AICU were obtained from the hospital infection control committee. The study was approved by the hospital ethics committee.
Direct observation was performed by three of the authors of the present study, who are AICU staff members (SAB and WEF are nurses, and ACSF is a physiotherapist). Before beginning the observational study SAB, WEF, and ACSF were trained in the WHO-recommended method for direct observation, according to the “WHO Hand Hygiene Reference Technical Manual” [7].
The observational study was conducted in four periods: briefly, Period 1, February 5-March 3, 2018, 29h (SAB, covert observer) and Period 2, March 15-April 16, 2018, 33h (SAB, overt observer), which were performed before the educational campaign on hand hygiene; and Period 3, August 27-September 28, 33h (ACSF and WEF, covert observers) and Period 4, October 3-November 6, 2018, 35h (SAB, overt observation), which were performed after the campaign. Observation sessions lasted for 20 min and were performed in three daily sessions, randomly distributed in the morning, afternoon and evening shifts, including holidays and weekends.
Hand hygiene compliance was assessed according to the WHO “My Five Moments for Hand Hygiene” (5MHH) guidelines, which includes the following moments (M): M-1, before touching a patient; M-2, before clean/aseptic procedure; M-3, after body fluid exposure risk; M-4, after touching a patient; and M-5, after touching patient surroundings [1,7]. In general, the observer monitored hand hygiene at 6 of the 24 AICU beds.
Hand hygiene compliance rates observed in Periods 2 and 4 (overt observation) were displayed on an electronic light-emitting diode board (electronic panel) installed in the AICU. The panel was lighted from 7:00 to 21:00 h and displayed hand hygiene compliance rates (general and by professional category) and the message “Save lives − Wash your hands".
The educational campaign on hand hygiene was carried out from August 1–15, 2018, and consisted of lectures to small groups of AICU personnel, to avoid interfering with activities of the unit. A 3-min video on the 6-step hand hygiene technique was shown, combined with a 10-slide session emphasizing the importance of hand hygiene in hospital practice. A WHO pocket leaflet on when and how to conducted hand hygiene was also distributed to each campaign participant.
To determine if there was a significant difference between compliance with hand hygiene recorded by the overt and covert observation methods, the Z statistical test to compare proportions using the program Statistica (version 13.2) was performed. A P value less than 0.05 was considered significant.
Results and discussion
According to HHSAF, our hospital was classified at the intermediate level and received 257.5 of 500 points (i.e., system change, 100 points; training and education, 40 points; evaluation and feedback, 35 points; reminders in the workplace, 42.5 points; institutional safety climate for hand hygiene, 40 points) [5]. This was somewhat surprising, since we were not aware that systematic educational programs to promote hand hygiene practices have been performed at our hospital for the last 15 years. However, the score of 257.5 achieved by our hospital is at the lower limit of the intermediate range (score 251–375) and requires improvement in comparison to other hospitals around the world [6]. For example, a WHO study conducted two global surveys in 2011 and 2015 using the HHSAF. In 2011, 2119 health facilities from 69 countries participated, and in 2015, 807 health facilities from 91 countries participated. The average score of 292.5 in 2011 and 372.9 in 2015 ranked health facilities at the intermediate level (i.e., 251 to 375 points) in relation to promotion and hand hygiene practices [6]. These scores exceed that of our hospital.
A total of 4,661 indications for hand hygiene were recorded during the 130 h (390 sessions lasting 20 min each) of direct observation. Overall hand hygiene compliance rates were: P-1, 31.95% (340/1064; i.e., 340 hand hygiene actions performed versus 1,064 indications for hand hygiene); P-2, 68.10% (790/1160); P-3, 56.11% (615/1096); and P-4, 80.98% (1086/1341) (Table 1).
Table I.
Hand hygiene compliance by the healthcare workers of the adult intensive care unit, in general and according to professional category; hand hygiene indications according to the World Health Organization's “My Five Moments for Hand Hygiene” guidelines; and covert and overt observation before and after the hand hygiene educational campaign
| Hand hygiene indication and professional category | Hand hygiene before the campaign |
P-value∗ | Hand hygiene after the campaign |
P-value∗ | ||
|---|---|---|---|---|---|---|
| P-1 Covert observer | P-2 Overt observer | P-3 Covert observer | P-4 Overt observer | |||
| M-1. Before touching a patient | ||||||
| All | 13.31% (61/458)a | 34.74% (148/426) | 0.0001 | 46.39% (238/513) | 71.87% (391/544) | 0.0001 |
| Physician | 24.32% (18/74) | 38.63% (34/88) | 0.0001 | 51.88% (55/106) | 72.72% (96/132) | 0.0001 |
| Nurse | 26.77% (25/97) | 27.38% (23/84) | 0.7464 | 46.04% (64/139) | 81.81% (108/132) | 0.0001 |
| Nursing technician | 5.77% (13/225) | 37.50% (75/200) | 0.0001 | 41.13% (65/158) | 71.33% (112/157) | 0.0001 |
| Otherb | 8.06% (5/62) | 29.62% (16/54) | 0.0001 | 49.09% (54/110) | 60.97% (75/123) | 0.0001 |
| M-2. Before clean/aseptic procedure | ||||||
| All | 77.77% (14/18) | 100% (26/26) | 0.0001 | 100% (18/18) | 100% (26/26) | 0.9999 |
| Physician | 100% (6/6) | 100% (3/3) | 0.9999 | 100% (9/9) | 100% (17/17) | 0.9999 |
| Nurse | 100% (5/5) | 100% (22/22) | 0.9999 | 100% (9/9) | 100% (9/9) | 0.9999 |
| Nursing technician | 0% (0/3) | 100% (1/1) | 0.0001 | − | − | NA |
| Other | 75% (3/4) | − | NA | − | − | NA |
| M-3. After body fluid exposure risk | ||||||
| All | 100% (7/7) | 85.71% (6/7) | 0.0001 | 100% (12/12) | 100% (6/6) | 0.9999 |
| Physician | − | − | NA | − | − | NA |
| Nurse | 100% (1/1) | − | NA | 100% (1/1) | − | NA |
| Nursing technician | 100% (3/3) | 83.33% (5/6) | 0.0001 | 100% (11/11) | 100% (1/1) | 0.9999 |
| Other | 100% (3/3) | 100% (1/1) | 0.9999 | − | 100% (5/5) | NA |
| M-4. After touching a patient | ||||||
| All | 54.65% (229/419) | 90.24% (546/605) | 0.0001 | 66.44% (295/444) | 90.55% (537/593) | 0.0001 |
| Physician | 66.26% (55/83) | 92.36% (133/144) | 0.0001 | 81.31% (74/91) | 93.54% (116/124) | 0.0001 |
| Nurse | 68.36% (67/98) | 94.44% (119/126) | 0.0001 | 74.13% (86/116) | 89.40% (135/151) | 0.0001 |
| Nursing technician | 40.85% (67/164) | 88.40% (221/250) | 0.0001 | 44.44% (64/144) | 85.85% (170/198) | 0.0001 |
| Other | 54.05% (40/74) | 85.88% (73/85) | 0.0001 | 76.34% (71/93) | 96.66% (116/120) | 0.0001 |
| M-5. After touching patient surroundings | ||||||
| All | 17.90% (29/162) | 66.66% (64/96) | 0.0001 | 47.70% (52/109) | 73.25% (126/172) | 0.0001 |
| Physician | 23.07% (3/13) | 100% (5/5) | 0.0001 | 47.82% (12/23) | 74.35% (29/39) | 0.0001 |
| Nurse | 34.28% (12/35) | 33.33% (3/9) | 0.6361 | 48.64% (18/37) | 76.78% (43/56) | 0.0001 |
| Nursing technician | 9.09% (9/99) | 67.50% (54/80) | 0.0001 | 45.23 (19/42) | 70.83% (51/72) | 0.0001 |
| Other | 33.33% (5/15) | 100% (2/2) | 0.0001 | 42.85% (3/7) | 60% (3/5) | 0.0001 |
P, period; M, moment; −, no observation; NA, not applicable.
∗Significant at P<0.05.
Hand hygiene compliance rate was calculated as percentage (numerator, number of hand hygiene actions performed; denominator, number of hand hygiene opportunities).
Physiotherapists, laboratory technicians, radiology technicians.
The Hawthorne effect, estimated by the difference between overt (P-2) and covert (P-1) observations, revealed a 113% increase in the hand hygiene compliance rate amongst AICU workers before the campaign (31.95% for covert observation versus 68.10% for overt observation), and a 44.32% increase after the campaign (P-3, 56.11% for covert observation versus P-4, 80.98% for overt observation). Overall, with only two exceptions in the nurse category (M-1, 26.77% versus 27.38%, P =0.7464; and M-5, 34.28% versus 33.33%, P = 0,6361), hand hygiene compliance rates differed significantly between covert and overt observations for all moments and professional categories assessed (Table 1). These findings suggest that the Hawthorne effect can be used deliberately to improve compliance with hand hygiene in hospital practice. In this case, it is important for hospitals to establish their own Hawthorne effect and use it to estimate the ‘real’ frequency of hand hygiene [8].
Feedback on hand hygiene adherence is essential to change health worker behaviour and results in increased hand hygiene compliance [1]. For example, one study demonstrated that third-part remote video auditing combined with real-time feedback produced a significant and sustained improvement in hand hygiene in a 17-bed medical intensive care unit. During the 4-month pre-feedback period, the hand hygiene compliance rate was 6.5% versus 81.6% in the 4-month post-feedback period, and 87.9% in the 17-month follow-up period [9]. An interesting aspect of this study was that performance feedback was continuously displayed on electronic boards mounted in the hallway of the unit for continuous viewing by the healthcare workers in the course of their work [9]. This motivated us to use the electronic panel in our study.
Although it is widely recognised that physicians' compliance with hand hygiene is often lower than that of nurses [1], our study showed that during the times when we obtained complete data for all professional categories (i.e., M-1, M-4 and M-5), physicians' level of compliance was similar to that of nurses (Table 1). Amongst other factors, regular feedback on hand hygiene compliance rates displayed on the electronic panel may have contributed to this unusual finding.
The educational campaign lecture on hand hygiene was delivered to groups each comprised of three AICU workers. Seven, nine and 12 lectures were delivered in the morning, afternoon and evening shifts, respectively. A total of 28 lectures were delivered within two weeks, and 84 (93%) AICU workers were involved. Overall compliance according to 5MHH guidelines was highest for M-4 (i.e., after touching a patient) on overt observation (90.24% before the campaign and 90.55% after the campaign) (Table 1). This finding highlights the universal tendency of self-protection of health professionals as described in other studies [1].
During the study period, the infection rate in our AICU was reduced by 22.6% (18.8% before the hand hygiene campaign versus 14.6% afterward), and use alcohol-based hand gel increased 132% (16.93 L versus 39.20 L per 1,000 patient-days) (data not shown). These findings demonstrate the effectiveness of the hand hygiene education campaign.
Our study had the limitations inherent to observational studies, such as observation bias (Hawthorne effect), observer bias, and selection bias. To reduce or eliminate the influence of the Hawthorne effect on covert observation, SAB (observer in P-1 and P-2) was replaced in P-3 by WEF and ASCF. In an attempt to reduce observer bias we sought to make observations in all shifts and on every day of the week during the four periods of the observational study.
Another limitation was the small sample size. Direct observation is estimated to capture only a small number of all hand hygiene opportunities that are occurring simultaneously under the study conditions [8]. For example, one study showed that in a 60-min observation period per day only 0.5–1.7% of the total number of opportunities for hand hygiene was captured [10]. In our study, during the observation periods, we estimated that 447,456 opportunities for hand hygiene occurred, of which we captured 1.04%. In brief, 4661 opportunities were observed at 6 of 24 beds in 130 h; 447,456 opportunities were estimated at 24 beds over 3120 h (i.e., 20min morning+20min afternoon+20min evening = 1 h/day × 130 h × 24 h/day] of activities in the AICU].
Despite these limitations, the direct observation method used in our study is considered the gold standard for hand hygiene compliance in healthcare facilities [1,8]. We believe that our data reflect an overall improvement in compliance with hand hygiene in the AICU during the study period. However, this study must be continued to determine the long-term effectiveness of the intervention.
In conclusion, the Hawthorne effect, associated with overt observation, and the educational campaign had a marked influence on hand hygiene compliance in our adult intensive care unit. The results suggest that combining overt and covert observation methods, including regular feedback on hand hygiene compliance displayed on an electronic panel, may be a valid alternative to increase real hand hygiene compliance rates in hospital practice.
Declaration of Competing Interest
None.
Funding source
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior, Brazil (CAPES); Finance Code 001.
Acknowledgements
We thank the staff of the adult intensive care unit for their cooperation in our study, and Dr Janet W. Reid for revising the English text.
References
- 1.World Health Organization . WHO; Geneva: 2009. WHO guidelines on hand hygiene in health care. First global safety challenge: clean care is safe care. [PubMed] [Google Scholar]
- 2.Chen L.F., Vander Weg M.V., Hofmann D.A., Reisinger H.S. The Hawthorne effect in infection prevention and epidemiology. Infect Control Hosp Epidemiol. 2015;36:1444–1450. doi: 10.1017/ice.2015.216. [DOI] [PubMed] [Google Scholar]
- 3.Eckmanns T., Bessert J., Behnke M., Gastmeier P., Rüden H. Compliance with antiseptic hand rub use in intensive care units: the Hawthorne effect. Infect Control Hosp Epidemiol. 2006;27:931–934. doi: 10.1086/507294. [DOI] [PubMed] [Google Scholar]
- 4.Hagel S., Reischke J., Kesselmeier M., Winning J., Gastmeier P., Brunkhorst F.M. Quantifying the Hawthorne effect in hand hygiene compliance through comparing direct observation with automated hand hygiene monitoring. Infect Control Hosp Epidemiol. 2015;36:957–962. doi: 10.1017/ice.2015.93. [DOI] [PubMed] [Google Scholar]
- 5.World Health Organization . World Health Organization Press; Geneva: 2010. WHO Hand hygiene self-assessment framework 2010. [Google Scholar]
- 6.Kilpatrick C., Tartari E., Gayet-Ageron A., Storr J., Tomczky S., Allegranzi B. Global hand hygiene improvement progress: two surveys using the WHO hand hygiene self-assessment framework. J Hosp Infect. 2018;100:202–206. doi: 10.1016/j.jhin.2018.07.036. [DOI] [PubMed] [Google Scholar]
- 7.World Health Organization . World Health Organization Press; Geneva: 2009. Hand hygiene technical reference manual: to be used by health-care workers, trainers and observers of hand hygiene practices. [Google Scholar]
- 8.Gould D.J., Creedon S., Jeanes A., Drey N.S., Chudleigh J., Moralejo D. Impact of observing hand hygiene in practice and research: a methodological reconsideration. J Hosp Infect. 2017;95:169–174. doi: 10.1016/j.jhin.2016.08.008. [DOI] [PubMed] [Google Scholar]
- 9.Armellino D., Hussain E., Schilling M.E., Senicola W., Eichorn A., Dlugacz Y. Using high-technology to enforce low-technology safety measures: the use of third-party remote video auditing and real-time feedback in healthcare. Clin Infect Dis. 2012;54(1):1–7. doi: 10.1093/cid/cir773. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Fries J., Segre A.M., Thomas G., Herman T., Ellingson K., Polgreen P.M. Monitoring hand hygiene via human observers: how should we be sampling? Infect Control Hosp Epidemiol. 2012;33:689–695. doi: 10.1086/664346. [DOI] [PMC free article] [PubMed] [Google Scholar]