Comparing brief, covert directly-observed hand hygiene compliance monitoring to standard methods: A multicenter cohort study

Hand hygiene (HH) is the cornerstone of infection prevention (IP). Compliance monitoring and feedback are essential tools for hospital IP programs. Direct observation remains the benchmark for assessing compliance, though it is subject to the Hawthorne effect.^1,2 Prior studies have shown that HH compliance collected using covert observers and brief observation periods may reduce this bias.³ The primary aim of this study was to compare rates of HH compliance obtained by covert observers as part of a research study using brief observation periods (i.e. <15 minutes) with rates obtained by hospital IP programs using standard methods.

METHODS

As part of an ongoing, unpublished, multi-center, cluster-randomized trial to assess the efficacy of a direct gloving strategy (a similar strategy previously outlined by Rock et. al⁴), we collected data on HH and glove use for a cohort of healthcare workers (HCW) in adult intensive care units (ICUs) and general pediatric units at three academic medical centers. For the ongoing trial, study units were randomized to a direct gloving policy where HH before non-sterile glove use (e.g. for entry into Contact Precaution (CP) rooms) was not required (intervention) or a policy requiring HH before donning gloves (comparator). Trial outcomes included compliance with expected practices at entry to CP rooms based on the policy associated with study assignment, compliance with HH at room entry for non-CP rooms, and compliance with HH at exit from any room type. All observations from the trial were collected by covert research team members who observed in one location for a maximum of fifteen minutes.³ Data collected as part of the clinical trial was compared to compliance data collected by hospital IP programs at each facility. Hospital IP program HH compliance was collected on entry and exit by covert, non-IP, non-unit-based hospital staff obrvers, who did not limit observations to 15-minute periods and reported as overall compliance (entry/exit combined).

Using a random effects Poisson regression model, compliance rates were compared by collection method (study data collected by study personnel during brief observation periods vs. data collected for hospital IP programs) after adjusting for study month.

RESULTS

Over the 6-month study period, the research team observed 3,243 HH opportunities in 8 units at the 3 participating institutions with an overall rate of HH compliance of 69% for entry to and exit from all rooms combined (2,266 episodes/3243 opportunities). HH compliance reported to hospital IP programs for the same units during the same period was 89% (2,530/2,845).

When controlling for study month, combined HH compliance on entry and exit for all rooms reported by IP programs was 29% higher than that reported by the study’s observers (RR 1.29, 95% CI 1.21 to 1.36). When controlling for study month, combined HH compliance on entry and exit of all rooms reported by IP programs was 8 % higher than that reported by the study’s observers on entry into non-CP rooms and exit from all rooms (Risk Ratio (RR) 1.08, 95% Confidence Interval (CI) 1.01 to 1.14).

DISCUSSION

We compared rates of HH compliance as reported by hospital IP programs with that obtained by research study personnel limiting observations to a maximum of 15 minutes and found higher rates of HH compliance observed by IP programs than among the research team in the same units during the same time period.

Depending on when HH compliance was measured (all rooms or entry into non-CP rooms and exit from all rooms), HH compliance was between 8 and 29% greater when reported by hospital IP programs. We present comparisons of compliance two ways for the following reasons. The study data included over-sampling of CP rooms by design to address the primary study outcome of HH and glove use compliance on entry into CP rooms, whereas the sampling from IP was more likely to reflect the natural proportions of CP rooms within the study units. As prior studies have observed lower compliance with HH before glove use⁵, study data (HH compliance on entry into all rooms) may have underestimated the true unit compliance for all rooms and therefore overestimate the difference between research and hospital IP observations. To account for this, we also reported compliance on entry into non-CP rooms (may have overestimated compliance and thus underestimated the difference seen). We suspect the true difference lies between 8% and 29%; in either case, rates of HH compliance reported by hospital IP programs are consistently higher than that reported by the research team. The results of this study are in accordance with results of prior studies by Kohli and Pan, who found higher rates of HH compliance as assessed by IP programs as compared to covert observers.^6,7 Srigley et al. set to quantify the Hawthorne effect in hand hygiene compliance and reported three-fold high compliance in locations within eyesight of an auditor.²

As illustrated by Figure 1, the magnitude of the difference between the HH compliance rates reported by IP programs and research personnel decreased over the study period. Our study was not designed to explore the reasons for such change, however, we speculate that HCWs may have recognized study personnel over time and adjusted their behavior. Similarly, Chen et al.⁸ found that rates of HH compliance measured by covert observers increased with study duration.

Figure 1.

Infection Prevention-observed compliance at entry and exit in comparison to research personnel. Agency for Healthcare Research and Quality data include hand hygiene compliance on entry into non-contact precaution rooms and exit from any room.

The discrepancy in HH compliance revealed by our study confirms previously published reports of an observation bias when HH observers are known to the HCWs they are observing. Thus, the accuracy and validity of relying on IP program data to evaluate rates of HH compliance must be carefully considered with respect to who, what, when and how measurements are made, particularly when HH compliance rates may be used as incentives for HCWs and for accountability purposes.

This study has several limitations. Study data were extracted from an ongoing clinical trial, which was not specifically designed to assess the aforementioned differences in HH compliance. In addition, the study data included over-sampling of entry into CP rooms. To account for this, data was presented with CP rooms included and excluded. Finally, IP program data was presented to us in aggregate and did not allow for further breakdown at various opportunities such as room entry and exit.

We found that HH compliance rates reported by IP programs were consistently higher than those reported by research personnel, suggesting over estimation of compliance by IP, which may have resulted from a difference in duration of observation or some other unmeasured factor. As IP HH data are increasingly utilized, the results of this study urge caution in over-relying on data derived solely from traditional methods of direct observation, as HH compliance rates acquired from brief observation periods by non-IP staff may be more accurate, particularly when limited to brief observation periods lasting no more than 15 minutes.³

1. HH compliance is commonly monitored by direct observation, which is subject to the Hawethorne bias.

2. Covert research staff observed HH for maximum of 15 min, data compared to Infection Prevention.

3. Here, HH compliance was up to 29% higher as reported by Infection Prevention versus research team.