Assessment of Fidelity in Interventions to Improve Hand Hygiene of Healthcare Workers: A Systematic Review

Abstract

OBJECTIVE

Compliance with hand hygiene in healthcare workers is fundamental to infection prevention yet remains a challenge to sustain. We examined fidelity reporting in interventions to improve hand hygiene compliance, and we assessed 5 measures of intervention fidelity: (1) adherence, (2) exposure or dose, (3) quality of intervention delivery, (4) participant responsiveness, and (5) program differentiation.

DESIGN

Systematic review

METHODS

A librarian performed searches of the literature in PubMed, Cumulative Index to Nursing and Allied Health (CINAHL), Cochrane Library, and Web of Science of material published prior to June 19, 2015. The review protocol was registered in PROSPERO International Prospective Register of Systematic Reviews, and assessment of study quality was conducted for each study reviewed.

RESULTS

A total of 100 studies met the inclusion criteria. Only 8 of these 100 studies reported all 5 measures of intervention fidelity. In addition, 39 of 100 (39%) failed to include at least 3 fidelity measures; 20 of 100 (20%) failed to include 4 measures; 17 of 100 (17%) failed to include 2 measures, while 16 of 100 (16%) of the studies failed to include at least 1 measure of fidelity. Participant responsiveness and adherence to the intervention were the most frequently unreported fidelity measures, while quality of the delivery was the most frequently reported measure.

CONCLUSIONS

Almost all hand hygiene intervention studies failed to report at least 1 fidelity measurement. To facilitate replication and effective implementation, reporting fidelity should be standard practice when describing results of complex behavioral interventions such as hand hygiene.

Healthcare-associated infections (HAIs) continue to cause significant morbidity, mortality, and increased medical costs.^1–3 Each year, approximately 722,000 people in the United States develop an HAI (1 in 25 hospitalized patients), and 75,000 of those affected die. In recent years, HAIs have been recognized as largely preventable, and institutions have undertaken intensive efforts to reduce their occurrence. Hand hygiene (HH) is the cornerstone of infection prevention, and most HAI prevention efforts include HH improvement as a major goal.^4–6 For behavioral interventions such as HH to be effective, fidelity to the intervention is crucial.⁷ A plethora of literature is available on HH interventions; however, fidelity to these interventions is often poorly described. This lack of implementation data reporting poses challenges for institutions seeking to learn from the successes and failures of others, and hinders the examination of intervention effectiveness in real-world healthcare settings.

Fidelity has previously been defined as “the demonstration that an experimental manipulation is conducted as planned.”⁷ Thus, an intervention has demonstrated fidelity if each of its components is delivered to participants without variations. Ensuring fidelity is crucial because it allows for replication, evaluation, comparison, and dissemination of interventions.^7,8 Implementation fidelity of an intervention has 5 domains:⁹ (1) adherence to the program, (2) exposure or dose (ie, the amount of the program delivered), (3) quality of intervention delivery, (4) participant responsiveness, and (5) program differentiation (ie, the presence of distinguishing features of the intervention).

To provide a comprehensive view of the integrity of any intervention, it has been recommended that researchers measure all 5 dimensions⁹ because they are essential for the examination of the causal links between intervention and clinical outcomes.¹⁰ We undertook a systematic review to assess this gap in the literature on HH interventions by examining fidelity reporting in interventions aimed at improving HH compliance.

METHODS

We followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement guidelines in reporting the results of this systematic review.¹¹

Details of the protocol for this systematic review were registered on PROSPERO.

Main Outcome

The main outcome for this systematic review was intervention fidelity. Among the available tools for measuring the fidelity of interventions,¹² the most widely used approach was introduced by Dane and Schneider in 1998.⁹ This tool, which we used in this review, assesses 5 components of fidelity:^9,10

1. Adherence. Adherence measures how well the delivered program elements align with the intervention as planned in the protocol. Thus, it refers to the adherence to an intervention, not HH compliance. Identification of the primary components of each intervention is the first step in assessing adherence fidelity.¹³ These elements are usually embedded in statements that define objectives in published studies. For example, if an educational intervention is planned, researchers determine whether the study assesses whether training actually occurred. This component can be met through surveys of participants, posttests, or independent audits. Likewise, if an intervention aims to give personal alcohol-based hand rubs (ABHR) to healthcare workers (HCWs), adherence assesses whether ABHRs were actually provided.

2. Exposure or dose. This component indicates how much program content reaches participants. In many HH interventions, educational sessions are provided to participants.¹⁴ However, the effectiveness of the intervention may depend on the amount of material presented. Thus, it is important to quantify, for example, the number of sessions completed, their duration, or their intensity.

3. Quality of delivery. Quality refers to the processes and content ideals embedded in an intervention. The quality of an intervention can be assessed in several ways, including direct observations of how the intervention (eg, a survey) is administered, or how self-reporting is conducted.

4. Participant responsiveness. Participant responsiveness measures how engaged participants are in the intervention and how they view this participation. Participants should also be aware of all the intervention components, and it is important to ensure that they feel their opinions are respected. Participant feedback can be assessed through interviews or surveys.

5. Program differentiation. To meet the fidelity criteria for program differentiation, researchers must explain the specific ways in which interventions were conducted. HH interventions often involve several components, such as education, audits, feedback, and administrative leadership engagement. ¹⁵ Regardless of their content, these should all be defined. For example, education might be based on a modification of the World Health Organization’s (WHO’s) 5 Moments of HH or the International Nosocomial Infection Control Consortium (INICC) multidimensional HH approach.^16,17

Briefly, the INICC is an open, non-profit, multicenter network. It conducts HAI surveillance through analysis of standardized data collected voluntarily from its member hospitals. The INICC multidimensional HH approach involves (1) administrative support, (2) supply availability, (3) education and training, (4) reminders in the workplace, (5) process surveillance, and (6) performance feedback.¹⁶

Inclusion and Exclusion Criteria

Studies were included in this systematic review if they (1) assessed interventions to improve HH compliance in a healthcare setting, (2) were implemented in response to an outbreak or ongoing infection control initiatives, and (3) were published in English. Studies were excluded if they (1) did not assess HH, (2) were conducted outside the healthcare setting, or (3) reported HH adherence without reference to a specific intervention.

Information Sources

A health sciences librarian (L.V.) performed searches of the literature in PubMed, Cumulative Index to Nursing and Allied Health (CINAHL), Cochrane library and Web of Science. The bibliographic databases were searched through June 19, 2015. To increase the sensitivity of our search, we also searched for work of published experts in the field. Additional records were identified by reviewing references lists of 2 articles identified from database searches. One was a meta-analysis that assessed adherence to HH,¹⁵ and the other was the World Health Guideline on HH in health care.¹⁸

Search Strategies

In the electronic search of the bibliographic databases, 2 different search strings were used. The PubMed search string was designed as follows: (“implementation fidelity” OR “intervention fidelity” OR “intervention compliance” OR “intervention adherence” OR “treatment fidelity” OR “treatment compliance” OR “treatment adherence” OR program evaluation OR “process evaluation” OR “program integrity” OR “Guideline Adherence”) AND (hand washing OR handwashing OR hand hygiene OR hand disinfection). For the CINAHL, Web of Science, and Cochrane Search databases, the following search string was formulated: (handwashing OR hand washing OR hand hygiene OR hand disinfection) AND (implementation fidelity OR intervention fidelity OR intervention compliance OR intervention adherence OR treatment fidelity OR treatment compliance OR treatment adherence OR program evaluation OR process evaluation OR program integrity).

Data Abstraction

We screened the titles and abstracts of all articles to identify studies meeting our inclusion criteria. A random sample of 10 of the 100 articles (10%) was screened by 2 members of the study team (J.S.M. and A.B.) and the results were compared. There was congruence on 7 of 10 (70%) of the sample. A resolution regarding the 3 articles with inconsistencies was reached by the consensus of the 3 reviewers (J.S.M., A.B., and N.S.). The remaining 90 articles were screened by 1 reviewer (J.S.M.). A data abstraction form containing the following information was used to store the following data for each study: last name of the study’s first author, year of publication, study design, healthcare setting, study location, whether the intervention was bundled, primary outcome, whether the intervention was in response to outbreak, primary type of HH (water and soap or hand rub), the method used to assess fidelity (source), and the format for assessing compliance to HH.

Study Bias Assessment

Assessment of study quality was conducted for each study using a modified version of the Downs and Black quality assessment checklist.¹⁹ This tool assesses reporting, external validity, internal validity, and power. Each of these items was assigned a score of 1 if it was present in the study and zero if it was absent, with a maximum score of 27. Some of the features evaluated by this tool include a clearly described hypothesis, aim, or objective; clearly described main findings; subject or researcher blinding; randomization; whether outcome measures were accurate; and the reliability of intervention compliance. We did not exclude studies on the basis of their bias scores; rather we present these scores to empirically rate study quality. The bias assessment table is provided as online supplementary material.

RESULTS

Study Selection

The search yielded a total of 912 articles (PubMed, 259; CINAHL Plus, excluding MEDLINE, 73; Web of Science, excluding MEDLINE, 170; Cochrane library, 113; author searches, 164; review of references, 133). Of these 912 articles, 120 were duplicates. We excluded these duplicates, which resulted in 792 articles for screening. Of these articles, we excluded 666 studies after title and abstract review because they did not meet the inclusion criteria. This resulted in 126 articles for full article review; of these, 26 were excluded for various reasons (Figure 1). Ultimately, 100 articles were included in the qualitative synthesis. The study selection strategy is outlined in Figure 1, which was adapted from PRISMA.¹¹

FIGURE 1.

Study selection flow diagram, adapted from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement guidelines.

Characteristics of Included Studies

The majority of included studies (n = 89) were quasi-experimental studies; 8 were cluster randomized trials; and 3 were randomized clinical trials, randomized at the individual level. Most were single-center studies (n = 76) and 19 were multisite studies. A total of 34 studies were conducted throughout a healthcare facility, 32 were conducted in the ICU only, and the remainder were conducted in various settings such as general wards, a combination of ICU and general wards, burn units, stepdown units, and ambulatory clinics. In total, 12 of the studies (12%) were bundled interventions combining HH with other interventions (eg, chlorhexidine bathing), but the majority of studies (n = 88) addressed only HH. Only 3 studies were conducted in response to an outbreak; the rest were conducted as part of the ongoing infection control efforts.

HH compliance was the predominant primary outcome in a majority of the studies (n = 8). The remainder of the studies had clinical outcomes, such as incidence of hospital-acquired central venous catheter–related bloodstream infections (CLABSI), ventilator-associated pneumonia (VAP), methicillin-resistant Staphylococcus aureus. Improvement in outcomes (increase of HH adherence or reduction in clinical infections) occurred in 85 of 100 (85%) of the studies. The type of HH used was mainly a combination of ABHR and soap (n = 61), but many studies used ABHR alone (n = 36), and only 3 used soap alone. These results are summarized in Table 1. A complete version of this table is available as supplemental material online.

TABLE 1.

Characteristics of Included Studies

Study Characteristics	No. of Studies (%)
Study design
Quasi-experimental	89 (89)
Cluster randomized	8 (8)
Individual level randomized clinical trials	3 (3)
Locationa
North America, Europe, and Australia	70 (70)
South America	21 (21)
Asia	20 (20)
Africa	5 (5)
Type of hand hygiene used
ABHR and soap	61 (61)
ABHR alone	36 (36)
Soap alone	3 (3)
Outcome studied
HH compliance	88 (88)
Clinical outcomes, HAI	12 (12)
Overall change in outcome
Improved HH compliance	78 (88.6)
Improved clinical outcomes, HAI	7 (58.3)
No. of fidelity measures not reported
1 measure	16 (16)
2 measures	17 (17)
3 measures	39 (39)
4 measures	20 (20)
5 measures	0 (0)
Quality assessment
Mean bias score (SD)	14 (2)

NOTE. HH, hand hygiene; ABHR, alcohol-based hand rubs; HAI, healthcare-associated infections.

^aLocations add up to more than 100 because some studies were conducted in more than 1 country.

Description of Hand Hygiene Interventions

All HH interventions were organized into 1 or more of these components, summarized by Aboumatar et al²⁰ in 2012 as (1) educational approaches, (2) performance measurement and direct staff feedback, (3) communication campaigns, (4) environment optimization, and (5) leadership engagement.

Education was the most commonly included component among the studies (80 of 100; 80%) and included general discussions about HH, its indications, and its clinical importance. The next most frequently included component was comprised interventions incorporating performance measurement and direct feedback (69 of 100; 69%). These components involve providing direct, timely performance feedback to participants, and acknowledging when staff perform or fail to perform HH.

In total, 52 interventions (55%) utilized communication campaigns, including reminder posters on the wards, banners, and other forms of multimedia such as stickers and computer screen savers. In addition, 47 interventions (47%) included environmental optimization. Environmental optimization included improved availability of materials necessary for HH, including their accessibility, and strategic placement in a unit. However, only 21 of 100 interventions (21%) involved leadership engagement. In these studies, hospital leadership actively encouraged staff to become involved in the interventions through performance feedback or compliance incentives.

To assess HH compliance, most studies used direct observations (n = 74; 74%). Some used other innovative means such video surveillance and electronic counting of HH events, while others used a combination of observations, self-reporting, and measurement of product usage.

Risk of Bias Within Studies

All included studies were prospective in design (either experimental or quasi-experimental). However, only 3 studies were randomized controlled trials.^21–23 Based on the Downs and Black tool for bias, the mean score for all the studies was 14 (standard deviation of 2), with a minimum of 8 and a maximum of 19.

Assessment of Fidelity Measures

The 5 elements of fidelity assessed in this systematic review were (1) adherence to the intervention, (2) exposure or dose, (3) quality of intervention delivery, (4) participant responsiveness, and (5) program differentiation.

With the exception of 8 studies, all studies failed to report at least 1 measure of fidelity. The interventions in 3 of these 8 studies ^24,25 applied the WHO multimodal approach (or a modified but similar version of it) and included feedback from the HCWs. The other 5 used the INICC multidimensional HH approach.^{16,17,26–28}

At least 3 fidelity measures were not included in 39 of 100 (39%) of the studies reviewed; 20 of 100 (20%) of studies failed to include 4 measures of fidelity; 17 of 100 (17%) failed to include at least 2 measures, while 16 of 100 (16%) of the studies lacked at least 1 measure of fidelity.

Participant responsiveness and adherence were the fidelity measures most frequently lacking, while quality of the delivery was the fidelity measure most often included. Participant responsiveness was not included in 70 of 100 studies while adherence was not reported in 59 of 100 studies. Quality of the delivery of the intervention was missing in 11 of 100 studies. Dose was not reported in 56 of 100 studies and program differentiation was lacking in 49 of 100 studies.

In interventions that assessed adherence, participants were interviewed or administered surveys asking whether the intervention was implemented. None of the studies used independent observers to assess adherence to the intervention. Exposure or dose was reported by stating the number of sessions times completed or duration or intensity of a given intervention. Most studies that reported quality measured it via observers who used checklists to assess whether the processes and content ideals embedded in an intervention were followed. A few employed self-reports of participants through surveys. All interventions that reported participant responsiveness measured it using interviews or surveys in which participants reported their awareness of the intervention and all its components. Interventions that assessed program differentiation stated the unique features of the interventions. For example, some educational interventions stated that they applied the WHO 5 Moments of HH to teach about HH indications. Others used the CDC guidelines or applied the INICC multidimensional HH approach. A full presentation of these results, including a brief description of the interventions, is shown in Table 2 (available as supplemental material online only).

DISCUSSION

In this review, we found that nearly all HH intervention studies in the literature neglected to collect or report the 5 measures of fidelity.⁹ The WHO recommends that all 5 elements be reported for any multimodal approach.^29,30 The studies that included all measures of fidelity (a minority in our review) used a modified version of this WHO approach or the INICC multidimensional HH approach.¹⁶

Among the 5 elements, quality of program delivery was the most commonly reported measure. This finding suggests that interventions have prioritized measurements related to the HH outcome, possibly to the detriment of other fidelity measures. These neglected elements are crucial for the sustainability of the HH interventions, and not assessing them may partially explain why compliance to HH in healthcare settings remains low³¹ despite the plethora of publications on HH interventions. One such explanation for this paradox is highlighted by our finding that participant responsiveness is the least commonly assessed fidelity measurement. Cooperative compliance of involved HCWs is crucial in ensuring the sustainability of interventions. One study reported that short-term, non-sustained HH compliance is often due to a lack of ongoing reinforcement.³² However, ongoing reinforcement might not be as essential if HCWs were fully engaged in an intervention from its inception.

Reporting of adherence and exposure were similarly uncommon; both were lacking in more than half of the interventions we reviewed. Without measuring adherence, it is difficult to determine to what extent and how an intervention was implemented according to protocol. A lack of exposure data limits study replicability, making it difficult for researchers to duplicate even successfully conducted interventions.

To our knowledge, this is the first systematic review to assess fidelity in HH interventions. A limitation of our review is that we could not rank interventions based on their fidelity measurements because no published studies have investigated which of the 5 elements is most important. While some are more likely to contribute to study replicability (eg, dose, differentiation, and quality), others are more predictive of the sustainability of intervention effects (eg, participant responsiveness and adherence). Ultimately, interventions should strive to include all 5 fidelity components, and future research should fully examine the importance of each element individually.

The limited reporting of fidelity measurements in HH studies, in combination with the relevance of these elements in study replicability and intervention sustainability, makes it essential that the field of infection control prioritize the elements of fidelity in reporting.

In this review, nearly all HH intervention studies in the literature neglected to collect or report the 5 measures of fidelity. This finding has implications for the sustainability of intervention effects and their replication. Future HH interventions should ensure the collection and reporting of fidelity using the established frameworks for doing so.