Making Infection Prevention Education Interactive Can Enhance Knowledge and Improve Outcomes: Results from the Targeted Infection Prevention (TIP) study

Abstract

Objectives

To assess effectiveness of an interactive educational program in increasing knowledge of key infection prevention and control (IPC) principles with emphasis on indwelling device care, hand hygiene and multi-drug resistant organisms (MDROs) among nursing home (NH) healthcare personnel (HCP).

Methods

We conducted a multi-modal randomized-controlled study involving HCP at 12 NHs. Ten comprehensive and interactive modules covered common IPC topics. We compared: a) intervention and control scores to assess differences in pre-test scores as a result of field interventions; b) pre- and post-test scores to assess knowledge gain and c) magnitude of knowledge gain based on job categories.

Results

4,962 tests were returned over the course of the intervention with 389–633 HCP/module. Participants were mostly female certified nursing assistants (CNAs). Score improvement was highest for modules emphasizing hand hygiene, urinary catheter care and MDROs (15.6%, 15.95%, and 22.0%, respectively). After adjusting for cluster study design, knowledge scores were significantly higher after each educational module, suggesting the education delivery method was effective. When compared to CNAs, nursing and rehabilitation personnel scored significantly higher in their knowledge tests.

Conclusion

Our intervention significantly improved IPC knowledge in HCP, especially for those involved in direct patient care. This increase in knowledge along with preemptive barrier precautions and active surveillance has enhanced resident safety by reducing MDROs and infections in high-risk NH residents.

Over 1.5 million people reside in nursing homes (NHs) in the United States, with an additional 3.2 million receiving NH services each year as short-stay, post-acute care residents.¹ Over 2 million infections occur each year in these NH residents, with an estimated 3%–15% acquiring an infection during their NH stay.^1,2 Infection prevention therefore is critical to enhancing resident safety and quality of life as well as reducing healthcare costs.

As compared to acute care, NHs have fewer infection prevention resources, and infection control professionals (ICPs) at the facilities typically have many other responsibilities such as wound care, staff education and management.³ Frontline healthcare personnel (HCP) in this setting also are often not optimally trained in the basic principles of infection prevention and control, resulting in low adherence to hand hygiene protocols and other pathogen and infection prevention practices.^4,5 In addition, there is high staff and leadership turnover, resulting in rapid shifts in programmatic priorities.⁶ Finally, the goals of NHs are inherently different from those of acute care facilities;⁷ NHs provide not only skilled care and rehabilitation, they are also senior living communities serving as their long-term residence.

Educational interventions have been shown to improve knowledge and practice pertaining to infection prevention in both the acute care and NH settings.^8–13 A majority of those educational interventions have generally focused on one aspect of infection prevention, such as hand hygiene,^9–11 antibiotic usage,^{7, 14–16} or evidence-based strategies to prevent a specific type of infection.⁸

The Targeted Infection Program (TIP) study was a multicenter, cluster-randomized trial to test the hypothesis that a multi-modal evidence-based intervention would reduce the prevalence of MDRO colonization and the incidence of device-related infections in NH residents with indwelling devices. The TIP intervention included: 1) preemptive barrier precautions; 2) active surveillance for MDROs and infections with data feedback; 3) NH staff education on key infection prevention practices. The control group NHs continued to practice according to their own infection prevention policies. In a recently published article, we showed that our intervention reduced the overall MDRO prevalence density by 23%, new MRSA acquisitions by 22% and new catheter-associated UTIs (CAUTIs) by 54%.^17–18 This study also showed improved hand hygiene and gown use at intervention sites when compared with control sites.¹⁷

The goal of this study was to then evaluate the incremental impact of a structured interactive educational in-service program on knowledge pertaining to key infection prevention and control principles among HCP in NHs with an emphasis on standard and “enhanced” standard precautions [use of personal protective equipment (PPE) plus medical asepsis]. Our hypothesis was that there would no differences in infection prevention knowledge between the intervention and control site HCP at baseline but that intervention site HCP would have higher post in-service knowledge scores following provision of a comprehensive curriculum of infection prevention and control modules. We were also interested in evaluating differences among disciplines. Understanding these differences would then lead to individualized learning curriculum development.

Methods

Study Design

TIP study is a cluster-randomized, multi-component intervention aimed at reducing the prevalence rates of multidrug-resistant organisms (MDROs), rates of indwelling device-related infections, while also enhancing HCP knowledge of infection control and prevention in the 12 NHs in Southeast Michigan participating in the study.¹⁷ This study was approved by the University of Michigan Institutional Review Board. Six NHs were randomized into the intervention arm of the study and six served as controls. The Targeted Infection Prevention (TIP) Program for NH HCP included: 1) preemptive barrier precautions with glove and gown use for direct care; 2) monthly MDRO and infection surveillance with feedback to the facilities every month; 3) 10 interactive educational modules on infection prevention and control.

Educational Intervention

Our overarching study goals were to design, implement, and evaluate a structured infection control and prevention education program for HCP providing direct clinical care to high-risk older adults in NHs who had indwelling devices (e.g., urinary catheters and feeding tubes). The content of our educational modules and knowledge tests was developed by experts in the field at different levels of practice (physicians, nurses, ICPs, and epidemiologists). The educational modules were presented to HCP at intervention sites through 10 in-services on a broad range of topics including overview of infection prevention practices, hand hygiene, barrier precautions, infection recognition, and care of indwelling devices, with content following evidence-based guidelines. The modules were presented every two to three months over a three-year period, and each module was given one to six times to incorporate HCP working different shifts at each facility. The modules were targeted to the following direct care HCP: certified nursing assistants (CNAs), registered nurses (RNs), and licensed practical nurses (LPNs). NH support services personnel, such as environmental services, food services, and those providing rehabilitative care were also invited to participate.

The 10 educational module topics were selected based on their relevancy to infection prevention and control in NHs (Supplemental Table 1). Based on their content, test questions were created and reviewed for format, consistency, and clarity. The educational in-service sessions were approximately 30 minutes each, incorporated adult learning theory of interactive self-directed learning,¹⁹ and were structured as follows. Five minutes were used at the beginning of the session for a knowledge assessment pre-test (i.e., to determine baseline knowledge). These one-page pretests included 4–10 questions. Modules 1–3 included a 20 minute video on the topic being presented, with the session moderated by research personnel (LM, RO, RAR); for modules 4–10, the format included both a moderated video presentation and an interactive demonstration between our education team and the HCP. These interactive demonstrations included games such as “Jeopardy: Diagnosis of Infection,” urinary catheter care skit, and hand hygiene technique demonstrations, pre- and post-hand hygiene cultures, dance routines, etc (Supplemental Figures 1–6). A knowledge assessment test was given after each module. These knowledge assessment tests also served as part of the educational component (i.e., the post-tests assessed knowledge gains and reinforced the material presented). Each intervention site received a copy of the all pre- and post-tests, as well as the educational videos to educate newly hired HCP or those who were on leave.

Knowledge Assessment and Outcome Measures

At the NH intervention sites, the pre- and post-knowledge tests were numbered so that they were linked with each individual’s responses. Tests were not linked across modules for individual participants. The HCP at the control sites were given a similar test on each session at the same time, but no education was provided, thus no post-test was administered. Our primary objective was to demonstrate overall improvement in knowledge about infection prevention and control among HCP at the intervention sites. We tested the hypotheses that: 1) there would be no differences in the baseline knowledge tests between the intervention and control sites and 2) at the intervention sites, the post in-service knowledge scores would be significantly higher than the baseline knowledge scores.

Statistical Methods

Knowledge test data were imported into SAS 9.3 for analysis. In order to assign a percentage correct score for each test, individual questions were recoded as either right, “1,” or wrong, “0.” Any questions left unanswered were considered missing, and if a whole test was left unanswered, it was not included in the analysis. Knowledge scores were then calculated by summing the recoded answers and dividing by the total number of non-missing answers, resulting in a percentage correct score. Supplemental Table 1 shows the number of questions included from each module for data analysis. Descriptive statistics were pooled across all modules.

Intervention and control site pre-test knowledge scores were compared by module using a two-tailed t-test; intervention site pre-test and post-test scores were compared using a one-tailed paired t-test in order to evaluate overall score improvement by module due to the educational intervention. Using generalized linear mixed models, we evaluated both the differences in baseline knowledge scores between the intervention sites and control sites, and the score improvement due to the educational intervention within the intervention sites. The random effects option allowed us to adjust for respondent characteristics — specifically the facility-level clustering. We also included HCP position as a predictor for score outcome, using CNAs as the baseline comparison. A sensitivity analysis was conducted to adjust for the effect of unanswered questions considered missing.

Results

Study Population

Of the total 4,962 pre- and post-knowledge tests completed and submitted by participants, we received 389 to 633 per module. Across all the modules, 2,004 tests (40%) were from the control sites and 2,958 tests (60%) were from the intervention sites. The majority of participants were females, CNAs or nurses, with more than one year of experience (Table 1). Other categories of HCP included environmental services, food services, rehabilitation, and other miscellaneous personnel.

Table 1.

Participant characteristics by intervention group given in ranges of responses per module for healthcare personnel, 2007–2013, N=4,962

Characteristic*	Control	Intervention
Gender (n = 270–607)
Female	108 – 230	183 – 340
Male	8 – 30	18 – 31
Position (n = 272–610)
CNA	46 – 96	114 – 209
Nurse	49 – 71	59 – 109
Housekeeping	0 – 28	3 – 34
Food Services	0 – 14	1 – 24
Rehabilitation	0 – 34	1 – 29
Other	3 – 47	2 – 21
Years of Experience (n = 170 – 593)
<1	8 – 34	3 – 63
1 – 5	48 – 87	52 – 148
>5	32 – 137	41 – 164

NOTE. Numbers vary based on response rate.

^*Module 4 not included in range as demographic information was not requested.

Pre-In-Service Baseline Knowledge Comparisons between Intervention and Control Sites

To test our first hypothesis, we compared the intervention site pre-test knowledge scores with control site knowledge scores. We anticipated no significant difference in scores between them. Intervention and control sites’ pre-test knowledge scores for a majority of modules (7/10) including “Chain in Infection” (mean difference = 1.3%, P=NS), “Overview of Infection Prevention Programs” (1.2%, P=NS), and “Surveillance Practices” (2.6%, P=NS) were similar (Figure 1). However, intervention sites scored higher on their pre-tests for three modules: “Infection Control Practices: Hand Hygiene” (6.8%, P<0.001), “Standard Precautions and Enhanced Standard Precautions” (7.9%, P<0.001), and “Recognition of Symptoms of Infections” (4.2%, P<0.001), (Figure 1). The content of these modules was reinforced by other components of the intervention (e.g., hand hygiene promotion was concurrent with in-services at intervention sites and intervention sites were asked to use enhanced standard precautions during their care of residents with indwelling devices). Throughout the study, the CNAs and nurses received informational pocket cards pertaining to infection recognition using NH-appropriate definitions for specific infections. Using CNAs as reference and adjusting for facility-level clustering, nurses and rehabilitation personnel had higher pre-test knowledge scores, whereas Environmental Services personnel had lower pre-test scores (Table 2). No score differences were seen between the different amounts of experience.

Figure 1.

Boxplot comparison of 6 control sites and intervention sites nursing home healthcare personnel knowledge survey scores and average score differences, by modules. Bolded p-values are significant at p<0.05 level. Whiskers are the 1^st and 4^th quartiles. The boxes are the 2^nd and 3^rd quartiles with the line indicating the median. The diamond indicates the average. The small dots are outlier points. Number of participants varied by module. HCP, healthcare personnel; Int, intevention group; Cont, control group.

Table 2.

Generalized linear mixed model comparing baseline knowledge score outcome between control and intervention

Variable***	Estimates*	Standard Error	P-value*
Intercept	76.1	1.21	<0.01
Control	0	–	–
Intervention	4.8	1.31	<0.001
CNA	0	–	–
Nurse	6.8	0.61	<0.001
Rehabilitation	4.6	1.28	<0.001
Food services	2.8	1.62	0.08
Other	1.7	1.07	0.11
Environmental services	−4.0	1.28	0.002
<1 yr	0	–	–
1 – 5 yrs	−1.1	0.85	0.21
>5 yrs	0.44	0.84	0.60

^*Estimates are how a particular groups (variable) score percentage would change when compared to the chosen comparison group (where estimates=0) for that category (cont vs. int; position type; years of experience).

^**P-values are significant at P<0.05, in which estimates are significantly higher or lower than comparison group.

^***The position and experience variables is a covariate.

Post-In-Service Effect on Knowledge

To test our second hypothesis, we compared the intervention sites’ post-test knowledge scores to their corresponding pre-test knowledge scores. With this hypothesis, we wanted to test the effectiveness and content of our educational in-services, as well our delivery method. We expected to see a significant improvement in post-test knowledge scores compared to the pre-test knowledge scores. We conducted over 200 in-services across 10 different topics at 6 intervention sites over 36 months. Overall in-service attendance ranged from 211 to 375 HCWs (38.0%–68.2% of nursing staff working that day) per topic.

After Module 1 (the study introduction), there was a significant improvement in scores from pre- to post-tests across all modules (P<0.001), suggesting that both our content and delivery was effective (Figure 2). The score improvement was most dramatic for modules that focused on hand hygiene, enhanced barrier precautions, and MDROs, with a score improvement of 15.6%, 15.9%, and 22.0%, respectively. Using a multivariate linear regression model and after adjusting for HCP type and facility-level clustering, the average post-test knowledge scores for intervention site HCP were significantly higher than pre-test knowledge scores (average 8.5%, P<0.01, Table 3).

Figure 2.

Boxplot comparison of intervention site nursing home healthcare personnel post-test and pre-test knowledge survey scores and average score differences, by module. Bolded p-values are significant at p<0.05 level. Whiskers are the 1^st and 4^th quartiles. The boxes are the 2^nd and 3^rd quartiles with the line indicating the median. The diamond indicates the average. The small dots are outlier points. Number of participants varied by module. NH, nursing home; HCP, healthcare personnel; Post, post-test; Pre, pre-test.

Table 3.

Generalized linear mixed model predicting knowledge score improvement in intervention healthcare personnel from pre-test to post-test.

Variable***	Estimates*	Standard Error	P-value*
Intercept	79.5	1.21	<0.001
Pre-test	0	–	–
Post-test	8.5	0.30	<0.001
CNA	0	–	–
Nurse	5.9	0.55	<0.001
Rehabilitation	6.1	1.48	<0.001
Food services	4.5	1.89	<0.001
Other	2.5	1.32	0.054
Environmental services	−1.12	1.28	0.38
Missing	−2.1	0.60	<0.001

^*Estimates are how a particular groups (variable) score percentage would change when compared to the chosen comparison group (where estimates=0) for that category (cont vs. int; position type).

^**P-values are significant at P<0.05, in which estimates are significantly higher or lower than comparison group.

^***The position variables is a covariate.

Effectiveness of Intervention for Different Disciplines

Multivariate linear regression models were used to evaluate the effect of discipline of HCP on knowledge scores using CNAs as the baseline comparison. Nurses and Rehabilitation care personnel significantly improved scores over baseline by 5.9% (P<0.01) and 6.1% (P<0.01), respectively, whereas Environmental Services personnel experienced less gain compared with the CNAs (−1.12%). There were no significant changes in score in the other HCP groups. This indicates that our educational material targeted to the nursing staff was especially effective in improving their knowledge levels.

Discussion

The purpose of this educational intervention was to enhance HCP knowledge pertaining to infection control and prevention as one component of a multimodal strategy aimed at reducing MDROs and infections in high-risk NH populations. The NH setting is a challenging environment for HCP education due to the constant turnover of personnel and variation in the level of training. In our study, we tested the effectiveness of combining a didactic educational segment with interactive strategies within the same educational module. Our findings demonstrate that the knowledge pertaining to the infection prevention and control topics we presented improved significantly following the educational in-services at the intervention sites compared to the control sites. HCP knowledge gains were most evident for topics such as hand hygiene, urinary catheter care, and MDROs, likely due to the ongoing implementation of other complementary interventions, such as hand hygiene promotional posters and resident-specific signage for enhanced barrier precautions at the intervention sites.

Very few studies have evaluated the effectiveness of an educational intervention in NHs for improving HCP knowledge pertaining to infection prevention practices.^{8–11, 20–23} A small interventional study involving three NHs in Hong Kong was conducted, where 15 residents and 10 HCP received an educational program about infection control with respect to enteral feeding. The intervention consisted of an educational program that lasted for two weeks after the baseline test at week one. The effectiveness of the program was evaluated by giving a post-intervention test at week 8 to both the intervention and control groups. They found a four-point (out of 20) improvement in post-test scores in the intervention group.⁸ Another study conducted in Taiwan across three NHs involving 40 nursing assistants with three educational in-services focused solely on hand hygiene. With knowledge tests taken pre-intervention, and one month and three months post-intervention, the scores of the two post-tests were significantly higher than the pretest.¹¹ Similar studies in acute care have also evaluated the effectiveness of educational intervention in improving infection prevention knowledge.^{9–10, 23–25} Our study adds to this evidence by involving 12 NHs, with an inclusion of multiple disciplines of HCP, over a three-year period, and was built on a comprehensive infection prevention and control curriculum. We show that interactive educational sessions that promote understanding of the content, allowing learners to discover knowledge on their own and making learning task-oriented using interactive tools leads to improved knowledge gains, implementation of knowledge in practice and enhanced patient-level outcomes.^17–19 Furthermore we demonstrate that large scale comprehensive studies can be successfully conducted in NHs over a prolonged period.

Evidence suggests that baseline knowledge as well as knowledge gain after educational intervention varies with job category and perhaps years of experience.^{9–10, 24–26} In our study using CNAs as baseline, nurses and rehabilitation personnel scored better in pre-tests, while environmental services and food services staff scored lower. In a survey of HCPs in France, baseline knowledge about hand hygiene and usefulness of alcohol-based hand hygiene was higher in physicians when compared with other HCPs.⁹ The benefits of traditional educational methods may also vary with job category. In a web-based hand hygiene educational program conducted in an intensive care unit, the improvements in post-test scores were higher for physicians and medical students than for nurses.⁹ In our study we showed difference in knowledge improvement by job category with knowledge gains being the highest for nurses and rehabilitation staff when compared with for CNAs.

Our data show that we need further refinement of our infection prevention and control education to match the knowledge and skills of different providers taking care of older adults. We may need to modify our in-service delivery for the different skill levels and discipline if we want to see a significant knowledge gains for those that serve solely in support services roles. Future studies should focus on evaluating infection prevention and similar education programs that are tailored based on job categories. We did not observe an effect of experience on knowledge scores. This is contrary to previous findings where HCP with more experience have higher scores.⁸ However, an earlier study completed by our research group found no significant differences in knowledge scores by level of experience.²⁷ Future studies should explore this question in greater depth.

We acknowledge a few limitations. First, we had only a baseline assessment at control sites and since there was no intervention, there was no post-test. Thus, we compared the baseline knowledge between control and intervention site HCP to establish baseline comparisons. Second, due to the frequent staff turnover in NHs, our in-services were not linked over time to individual HCP. Thus we could not demonstrate individual HCP knowledge improvement of the entire program over the course of the study. Third, our study was conducted in NH facilities in Southeast Michigan. While our study facilities resemble other U.S. NHs with respect to their ownership status, average number of beds, availability of laboratory and radiology services, and the presence of a person responsible for infection prevention, our results may not be generalizable to other types of long-term care facilities such as assisted living facilities and hospice centers. Intervention and control sites may have differed in terms of employee education level, experience and/or in-service attendance. Lastly, knowledge does not necessarily correlate with behavior however we did report increased hand hygiene and gown use at our intervention sites.¹⁷

Limitations notwithstanding, our study has several key strengths. First, we tested an innovative educational delivery model that combined standardized didactics with interactive hands-on educational strategies. This model has not been previously studied before in the long-term care setting. We believe that our interactive TIP Program can be utilized for several other content areas including but not limited to infection prevention, difficult behaviors in patients with Alzheimer’s, care coordination during transitions of care, and end-of-life discussions. Second, NHs were randomized at the beginning of the study so the intervention and control arms were balanced for a cluster randomized study design. There was also no loss of NHs (clusters) during the study. Third, we created an in-depth infection prevention educational program along with pre- and post-test knowledge tests presented over a three-year period, demonstrating that these programs can be conducted over a prolonged period of time and that the NHs are interested in becoming active partners to improve the care of older adults setting a standard for similar studies in this setting.

Our analysis demonstrates that an educational intervention targeting a multi-disciplinary group of HCP within the NH setting serves to significantly improve their knowledge of infection prevention and control strategies. Ultimately, we hope that this improvement in HCP knowledge will translate into improved infection prevention practices and safer, more effective care of NH residents.