A Multi-State Program to Reduce Catheter-Associated Infections in Intensive Care Units with Elevated Infection Rates

Abstract

BACKGROUND:

Preventing central line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI) remains challenging in intensive care units (ICUs).

OBJECTIVE:

The Agency for Healthcare Research and Quality Safety Program for ICUs aimed to reduce CLABSI and CAUTI in units with elevated rates.

METHODS:

Invited hospitals had at least one adult ICU with elevated CLABSI or CAUTI rates, defined by a positive Cumulative Attributable Difference metric (CAD>0) in the Centers for Disease Control and Prevention’s Targeted Assessment for Prevention Strategy. This externally-facilitated program implemented by a national project team and state hospital associations included on-demand video modules and live webinars reviewing a two-tiered approach for implementing key technical and socio-adaptive factors to prevent catheter infections, using principles and tools based on the Comprehensive Unit-based Safety Program. CLABSI, CAUTI, and catheter use data were collected (pre-intervention 13 months, intervention 12 months). Multilevel negative binomial models assessed changes in catheter-associated infection rates and catheter use.

RESULTS:

Of 366 recruited ICUs from 220 hospitals in 16 states and Puerto Rico for 2 cohorts, 280 ICUs completed the program including infection outcome reporting; 274 ICUs had complete outcome data for analyses. Statistically significant reductions in adjusted infection rates were not observed (CLABSI IRR=0.75, 95% CI=0.52–1.08, p=0.13; CAUTI IRR=0.79, 95% CI=0.59–1.06, p=0.12). Adjusted central line utilization (IRR=0.97, 95% CI=0.93–1.00, p=0.09), and adjusted urinary catheter utilization were unchanged (IRR=0.98, 95% CI=0.95–1.01, p=0.14).

CONCLUSIONS:

This multi-state program targeted ICUs with elevated catheter infection rates, but yielded no statistically significant reduction in CLABSI, CAUTI, or catheter utilization in the first two of six planned cohorts. Improvements in the interventions based on lessons learned from these initial cohorts are being applied to subsequent cohorts.

INTRODUCTION

Central line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI) are morbid and expensive hospital-acquired infections, particularly for patients in the intensive care unit (ICU) who often receive these devices as part of routine care to closely monitor and deliver therapies. ICU patients also commonly have serious medical comorbidities that increase their risk of acquiring drug-resistant catheter-associated infections, as well as complications from antibiotics treating these infections. Due to their high expense and capacity to harm patients, CLABSI and CAUTI have been a major focus of national and international efforts to reduce mortality and morbidity by improving quality of care delivered in hospital settings.^1–5 Multiple Centers for Medicare and Medicaid Services (CMS) programs that impact public reporting of hospital performance and link Medicare payments with hospital rates of CLABSI and CAUTI have been implemented over the past decade.^3,4,6–10

Overall, both CLABSI and CAUTI have decreased nationally in recent years, though there has been notably less success in preventing CAUTI among critically ill patients as well as limited progress in reducing overall central line and urinary catheter days of use for patients who received care in the ICU, despite studies noting opportunities for reducing unnecessary catheters.^5,10–18 In prior national acute-care implementation projects funded by the Agency for Healthcare Research and Quality (AHRQ) – and administered by the Health Research & Educational Trust (HRET), the research and education affiliate of the American Hospital Association – a Comprehensive Unit-based Safety Program (CUSP) has been used to reduce CLABSI and CAUTI. The CLABSI project produced a greater than 40%^19,20 relative reduction in CLABSI in ICUs. The CAUTI project yielded a 32%¹⁴ relative reduction in CAUTI in non-ICUs, but did not produce a significant CAUTI reduction in ICUs. Even in the highly successful CLABSI project, not all ICUs performed equally well. Nationally, CLABSI and CAUTI also remain a persistent challenge for many ICUs, despite numerous ICUs reporting success in published studies.¹⁸ To help ICUs with elevated CLABSI and CAUTI rates, AHRQ initiated the multi-state collaborative described and evaluated in this study, known as the AHRQ Safety Program for ICUs: Preventing CLABSI and CAUTI.²¹ Here we describe this program’s development, implementation, and results for the first two cohorts of this collaborative, which informed important program changes for subsequent cohorts which are in progress.

METHODS

Overview of the Program

The program objective was to reduce CLABSI and CAUTI in ICUs with persistently elevated rates through state or regional consortia in four of ten U.S. Department of Health and Human Services regions of the country as the first phase of a planned nationwide rollout of the project. The program was funded and guided by AHRQ, and led and developed by HRET, the University of Michigan and other members of the National Program Team, including: the American Nurses Association, the Association for Professionals in Infection Control and Epidemiology, the Society of Critical Care Medicine, and the Society of Hospital Medicine. This National Program Team served as coaches and faculty to develop, disseminate, and track program components, as well as garner support for this program within their respective organizations. An additional technical expert panel of experts in patient safety, CLABSI, CAUTI, teamwork, and implementation convened and provided guidance at two separate meetings during the project.

HRET provided centralized coordination and oversight for this program’s implementation via an externally-facilitated model (Figure 1), including facilitation of recruitment, data collection, and educational resource dissemination from state hospital association partners using their pre-existing relationships with participating ICUs. A state hospital association lead in each geographic region was recruited, educated, and provided technical and funding support throughout the program. The state leads were responsible for coordinating the program with the ICUs in their state or region by coordinating with HRET for educational and coaching sessions and holding monthly coaching calls and one state-wide in-person meeting. State leads encouraged ICUs to submit their project data on time, and provided feedback on state and individual hospital performance. In collaboration with program faculty, coaches, and state hospital association leads, hospital ICU staff could also receive support by clinical mentors in their state who provided additional coaching at the local level and mentoring ICUs in implementing their action plan. Puerto Rico and all participating states except 1 had at least one clinical mentor (a nurse or a physician, and ideally a dyad of a nurse and physician).

Figure 1.

External Facilitation and Implementation Model

AHRQ, Agency for Healthcare Research and Quality; ICU, intensive care unit.

*Note: AHRQ provided contract funding, set program objectives and deliverables, provided guidance throughout the project, and coordinated with other federal agencies. AHRQ was not directly responsible for the implementation of the project.

State leads were also expected to perform site visits for up to 50% of their participating hospitals, with the percentage requested by the program varying by state size. Sites were chosen for multiple reasons, including ICU interest in a visit, challenges with the interventions, persistently high CLABSI or CAUTI rates, or had effective practices to share. The visit’s purpose was for state leads and clinical mentors to meet with ICU teams and leadership to strengthen relationships, engage in discussion about infection prevention, and facilitate unit-specific changes by selecting gaps to address by targeted, unit-specific action plans. However, how the state leads identified ICUs for a site visit varied, as well as the visit’s components and data collected.

Program Interventions

To inform this program’s interventions, a systematic literature review¹⁸ was performed to summarize the evidence for practices to reduce CLABSI and CAUTI in the ICU setting. This literature review’s results, along with a previously successful 2-tiered approach implemented in Veterans Affairs hospitals for prioritizing interventions for CAUTI,²² were reviewed with the National Program Team and technical expert panel to inform the intervention. Baseline data for gaps in CLABSI and CAUTI prevention practices specific to the participating units for this project were not available at the time of recruitment or intervention development. Additionally, the intervention was informed by prior experience and resources developed in the Michigan Health and Hospital Association Keystone Centers’ Bladder Bundle Initiative,^23–25 and prior AHRQ-funded national implementation projects^14,19,20,26 for preventing CLABSI and CAUTI that applied the Comprehensive Unit-based Safety Program²⁷ and included a combination of technical and adaptive interventions.

This multi-component intervention addressed both the technical and socio-adaptive components of infection prevention²⁸ and was rolled out in two cohorts. Participating units were provided with an onboarding curriculum (delivered by a combination of live webinars and on-demand webinars) that gave a general overview of CUSP and basic principles units need to understand and implement in order to drive improvement, such as how to assemble their CLABSI and CAUTI prevention teams, the science of safety, developing shared mindsets for infection prevention goals and appropriate catheter use, how to engage senior leadership, and understanding the required resources to support their efforts, and guidance on the data to be collected. Brief, on-demand video modules were developed to address the technical (e.g., alternatives to catheters, aseptic insertion/maintenance, prompting catheter removal when no longer indicated) as well as the socio-adaptive aspects (e.g., strategies to address common barriers in changing clinician behavior, recognizing unit culture challenges and strategies to address them, building successful teams, improving accountability for device use and outcomes, engaging leaders, strategies to sustain change) of CLABSI and CAUTI prevention. These on-demand modules were specifically designed be short in duration and available to view by busy ICU staff when convenient. Monthly “virtual learning group” webinars were available to reinforce the resources and implementation principles provided by national project team members and subject matter experts to address common gaps in both technical and socio-adaptive practices for prevention of CLABSI and CAUTI noted in the systematic review¹⁸ that informed this project, as well as to provide the opportunity for peer-to-peer coaching from participating ICUs. By comparison, the on-demand modules were designed to deliver the “what” of the content, and the virtual learning groups were designed to focus more on the “how” of the intervention, by allowing participants to hear successes, challenges, and strategies employed by their peers for implementing the intervention within their ICUs. Podcasts were developed to support ICU senior leaders, including topics such as why senior leadership matters, and roles and responsibilities of ICU leaders and teams.

Intervention components were prioritized and presented to participating ICUs using a tiered approach. Tier 1 combined technical and socio-adaptive strategies that have been found to be impactful on infection prevention and includes fundamental infection prevention strategies, such as optimizing appropriate use of catheters and ensuring aseptic placement and maintenance care to promote early success. If rates remained high despite implementing Tier 1 recommendations, then units were advised to advance to Tier 2. Tier 2 was a stepwise process that began with a formal reassessment of challenges known as the Guide to Patient Safety (GPS)^29,30 (available at http://www.improvepicc.com/gpsclabsi.html and http://www.catheterout.org/cautigps.html) to inform the selection of other enhanced practices that may be useful as additional interventions, and ending with a root-cause analysis if rates remain elevated. For example, if CLABSI rates remain high after implementing Tier 1 approaches, technological approaches such as antimicrobial-impregnated catheters or daily chlorhexidine bathing could be employed.^31–37 The Tier 2 interventions are not included in Tier 1 because they were judged to be more costly, resource-intensive, or currently supported by less evidence.

The brief on-demand video modules developed for this project introduced the tiered approach to reducing CLABSI and CAUTI in ICUs and included: 1) a brief review of the risk-factors and morbidity associated with unnecessary catheter use, CLABSIs, and CAUTIs; 2) catheter use guidance (including the Centers for Disease Control and Prevention (CDC) Healthcare Infection Control Practices Advisory Committee (HICPAC) consensus-based guidelines for catheter use, the Ann Arbor Criteria for Appropriate Urinary Catheter Use in Hospitalized Medical Patients, and the Michigan Appropriateness Guide for Intravenous Catheters (MAGIC) criteria);^38–41 and 3) evidence-based practices related to “Disrupting the Life Cycle of the Catheter”, with a focus on catheter avoidance or selection, optimal aseptic insertion, maintenance, and prompt removal when no longer clinically appropriate.^18,42 Key technical focal points included proper catheter placement and maintenance, with emphasis on evaluating the necessity for a catheter. Multiple intervention components also addressed several socio-adaptive elements, including how to prioritize use of the potential interventions and resources in the program, and strategies to garner the support of all the team members using CUSP principles, as well as experience in prior implementation projects and clinical work in ICUs by National Program Team members. Principles of CUSP were shared in multiple components of the intervention to provide guidance on improving staff engagement, teamwork, communication, as well as engaging leadership, which can be facilitate units to advocate for additional hospital resources. Finally, the participants were provided education on the data collection and submission processes using brief on-demand modules and coaching calls. Unfortunately, this project was not able to provide financial or personnel resources to the participating units to support purchase of catheter insertion kits, catheter maintenance care supplies, or equipment such as bladder scanners. There were also no additional resources to support additional staff (e.g., nurse assistants to help manage incontinence), or dedicated time for existing staff to perform additional infection prevention activities, such as daily catheter rounds or root causes analyses. Finally, the participants were provided education on the data collection and submission processes using brief on-demand modules and coaching calls.

Program Eligibility and Recruitment

ICUs were identified as eligible for participation if the unit’s hospital met two inclusion criteria. First, the hospital was located in one of the four Health & Human Services (HHS) regions, including 19 states and US territories, selected because these regions had the highest number of hospitals with elevated CLABSI and/or CAUTI rates (defined as a positive cumulative attributable difference (CAD) based on the Targeted Assessment for Prevention (TAP) methodology).^43–45 Second, the hospital had “persistently elevated rates of CLABSI and/or CAUTI,” as defined by a positive CAD, using 1 of 2 data sources: 1) National Healthcare Safety Network (NHSN)⁴⁵ data assessed by the CDC (April 2014-March 2015 for Cohort 1 identification, and January 2015-December 2015 for Cohort 2 identification), or 2) NHSN data publicly reported on the Hospital Compare website from January–December 2014. Veterans Affairs hospitals and pediatric hospitals were excluded.

Two recruitment methods were used. First, eligible hospitals identified through the NHSN data source were sent an initial email from the CDC and one follow-up email informing them about the opportunity to join the AHRQ project and providing a link to the project website where an informational flier and narrated presentation about the project was available and hospital staff could enroll or request additional information; additionally, the CDC called hospitals that were not participating in an NHSN group involved with the Quality Innovation Network-Quality Improvement Organizations contracted by the Centers for Medicare & Medicaid Services to inform them about the opportunity. Second, the list of eligible hospitals identified with Hospital Compare data was shared with the respective state leads, who were advised to encourage each hospital to run its own TAP reports within NHSN, which would identify which ICUs within a hospital might benefit most from participation in the initiative. A supplementary recruitment method was used only for Cohort 2: an informational webinar hosted by HRET highlighting the benefits of the program as well as success stories from an ICU team lead who participated in a prior AHRQ-funded CAUTI-prevention project. In both cohorts, hospitals recruited ICUs that met CAD eligibility criteria, and in some hospitals, the team leader also recruited other ICUs within their hospital that may not have met CAD eligibility criteria. Interested hospitals obtained leadership support for the initiative and identified specific units for the improvements to take place.

Data Collection and Outcomes

The efforts of the collaborative were structured to include 2 different data collection periods: pre-intervention and intervention. The pre-intervention period was 13 months for Cohort 1 and 2. The intervention phase was 11 months with an optional 3 month extension for Cohort 1, and 12 months for Cohort 2. For consistency in the analysis, we used the first month of the 3 month extension of Cohort 1, yielding a 12 month intervention period for analysis for both Cohort 1 and 2. Hospital characteristics collected during the pre-intervention period were obtained from the 2015 American Hospital Association (AHA) Annual Survey of Hospitals. Participating ICUs were encouraged to complete an ICU Assessment Tool, developed specifically for this project, to help the ICU assess its needs in order to develop an action plan. The needs assessment was also used by the national program team and state hospital associations to assist with tailoring coaching and project resources. The assessment included basic unit demographic descriptions such as ICU bed size and ICU type, as well as information about current infection prevention practices and resources used for CLABSI and CAUTI prevention and other strengths and potential barriers including teamwork and communication strategies.

The outcomes were defined by and reported to the NHSN as part of the hospital’s routine surveillance, recommended monthly in this program but required quarterly by NHSN. Primary outcomes were NHSN CLABSI and CAUTI rates calculated as the number of CLABSIs (or CAUTIs) per 1,000 device days. Secondary outcomes were device utilization ratios for both urinary catheters and central venous catheters calculated as the number of device days per 100 patient days. Non-outcome data consisted of demographic data (hospital teaching status, hospital urban/rural location, ICU type, ICU bed size, and ownership), and reported ICU program focus (CLABSI, CAUTI, or both).

Measures of program participation were also summarized, to describe the available data on level of ICU participation in the specific components of the program interventions. These measures included the percentage of units that completed the ICU Assessment Tool at baseline and follow-up, as well as the percentage of units that viewed the various on-line live educational webinars and on-demand modules categorized within 4 levels of participation: 0% webinars/modules viewed, 1–49% viewed, 50–99% viewed, 100% viewed.

Statistical Analysis

Our analyses included participating ICUs that reported CLABSI and CAUTI outcomes and device utilization data. Other variables included hospital characteristic data available from the 2015 AHA Annual Survey of Hospitals⁴⁶ and ICU bed size and type as reported by the ICUs during program recruitment and/or via the ICU Assessment Tool. Descriptive statistics were reported using mean (standard deviation) and N (percent) for unit characteristics for all participating ICUs. Pre-intervention and intervention outcome rates were calculated by aggregating the number of infections and catheter/patient days at each time period over all units. Multilevel negative binomial regression with a random intercept for each ICU was used for modeling all primary (CLABSI rate, CAUTI rate) and secondary (urinary catheter utilization rate, central line utilization rate) outcome measures. This approach included main effects for continuous time, a main effect for an indicator for the start of the intervention period, plus an interaction term with time and an indicator variable for the intervention period. All models were adjusted for teaching status, urban or rural hospital location, type of hospital ownership, ICU bed size, ICU type, and whether or not the ICU focused its program implementation efforts on the relevant infection (CLABSI, CAUTI or both). All characteristic main effects were included simultaneously in the multivariate models. Using the same modeling approach as outlined above, we also conducted sensitivity analyses examining the impact of the intervention over time on the CAUTI, CLABSI, and device utilization outcomes among ICUs with positive CAD values. SAS V9.4 (Cary, NC) and Stata/MP 13.1 (College Station, TX) were used for all analyses.

Study Oversight

This study was deemed exempt from oversight by our institution’s Institutional Review Board. Authors with access to project data signed a confidentiality agreement with HRET. The data analysis plan for the project overall was informed by the Data and Analysis Committee, which was led by HRET and the University of Michigan with input from the National Program Team. Analyses for this publication were prepared and conducted independently by 2 of the authors (DR, MTG).

RESULTS

Hospital and ICU characteristics

Table 1 details the hospital and ICU characteristics. Of 366 recruited adult ICUs from 220 hospitals in 16 states and Puerto Rico (Appendix Figure 1), 280 ICUs completed the program including reporting of infection outcome data for CLABSI and/or CAUTI. Our analysis focuses on the 274 units (171 hospitals) that reported both CLABSI and CAUTI outcome data and had 2015 AHA data, number of beds, and ICU type data available. Similar to participating ICUs in a prior AHRQ collaborative¹⁴ involving both ICUs and non-ICUs, the majority of the ICUs in this collaborative were from urban hospitals (89.8%) and teaching hospitals (70.8%), though less than a quarter were from major teaching hospitals (23.4%). The most common ICU type was Medical/Surgical (77.7%), followed by specialty ICUs including Cardiology and/or Cardiothoracic (12.0%), Trauma/Burn (5.8%), and Neurology and/or Neurosurgery (4.4%). Of ICUs that reported a particular infection focus for this project, the majority (82.6%) focused on both CLABSI and CAUTI.

Table 1.

Hospital and ICU Characteristics

Hospital Characteristics1 (N ICUs, % of 274 units)
Teaching Hospital	194, 70.8%
Major Teaching Hospital	64, 23.4%
Urban Hospital	246, 89.8%
Ownership
Nonprofit, Non-government	150, 54.7%
For profit	47, 17.2%
Government	77, 28.1%
ICU Characteristics2
ICU size in mean number of beds ± SD	16.7 ± 12.4
ICU Type (N units, % of 279 participating units)
Medical/Surgical	213, 77.7%
Cardiology and/or Cardiothoracic	33, 12.0%
Trauma/Burn	16, 5.8%
Neurology and/or Neurosurgery	12, 4.4%
Program Focus, as identified by ICU (N, % of 274)
Focused on CAUTI and CLABSI	185, 67.5%
Focused on CLABSI	202, 73.7%
Focused on CLABSI only	17, 6.2%
Focused on CAUTI	207, 75.5%
Focused on CAUTI only	22, 8.0%
Program focus not specified	50, 18.2%

Abbreviations: CAUTI=Catheter Associated Urinary Tract Infection; CLABSI=Central Line-Associated Bloodstream Infection; ICU=Intensive Care Unit

¹Hospital characteristics are as defined in the American Hospital Association Annual Survey.

²Excluded units from analysis: 1 unit that reported only CAUTI data, 2 units that could not be linked with available AHA survey data, 2 units that were missing data on bed size, and 1 unit that was missing ICU type data.

Changes in CLABSI Rates and Central Line Utilization

The unadjusted CLABSI and central line utilization rates over the continuum of the project are illustrated in Figure 2. Among all participating units reporting CLABSI outcomes, the unadjusted rates of CLABSI decreased by 27.2%, from 1.08 CLABSI per 1,000 central line days at the end of the pre-intervention period to 0.78 per 1,000 central line days at the end of the intervention period. Central line utilization decreased 5.5% from 50% to 47% over the course of the project. CLABSI and central line utilization by ICU type is shared in supplementary online Appendix Figure 2.

Figure 2.

Central Line-Associated Bloodstream Infection (CLABSI) rates and Central Line Utilization Rates, as reported to the National Healthcare Safety Network (NHSN)

In adjusted analyses (Table 2), the CLABSI rate decreased from 1.25 at the end of the pre-intervention period to 1.06 at the end of the intervention period. There was no significant change in CLABSI rates during the pre-intervention period (IRR=0.95, 95% CI=0.77–1.19, p=0.67). During the intervention period, rates decreased 29% (IRR=0.71, 95% CI=0.53–0.96, p=0.03), however the rates did not change significantly over time when comparing the pre-intervention to the intervention period (IRR=0.75, 95% CI=0.52–1.08, p=0.13). After adjustment, central line utilization rates decreased from 50% to 47% during the intervention period. Central line utilization significantly decreased during the pre-intervention period (IRR=0.97, 95% CI=0.94–0.99, p=0.004) as well as during the intervention period (IRR=0.94, 95% CI=0.91–0.96, p<0.001). However, there was not a statistically significant change in central line utilization from the pre-intervention to the intervention period (IRR=0.97, 95% CI=0.93–1.00, p=0.09).

Table 2.

Multilevel Negative Binomial Regression Results

	Adjusted Results*
	IRR (95% CI)	P value
CLABSI Rate
Pre-intervention slope	0.95 (0.77–1.19)	0.67
Shift in rates pre-intervention to intervention	0.90 (0.72–1.11)	0.32
Intervention slope	0.71 (0.53–0.96)	0.03
Intervention slope change	0.75 (0.52–1.08)	0.13
Central Line Utilization Rate
Pre-intervention slope	0.97 (0.94–0.99)	0.004
Shift in rates pre-intervention to intervention	1.01 (0.99–1.03)	0.51
Intervention slope	0.94 (0.91–0.96)	<0.001
Intervention slope change	0.97 (0.93–1.00)	0.09
CAUTI Rate
Pre-intervention slope	0.96 (0.80–1.15)	0.66
Shift in rates pre-intervention to intervention	0.94 (0.79–1.12)	0.48
Intervention slope	0.76 (0.60–0.96)	0.02
Intervention slope change	0.79 (0.59–1.06)	0.12
Urinary Catheter Utilization Rate
Pre-intervention slope	0.95 (0.93–0.97)	<0.001
Shift in rates pre-intervention to intervention	1.00 (0.99–1.02)	0.73
Intervention slope	0.93 (0.91–0.95)	<0.001
Intervention slope change	0.98 (0.95–1.01)	0.14

Abbreviations: CAUTI=Catheter Associated Urinary Tract Infection; CLABSI=Central Line-Associated Bloodstream Infection; ICU=Intensive Care Unit; IRR=Incidence Rate Ratio.

^*All models were adjusted for teaching status, urban or rural hospital location, type of hospital ownership, ICU bed size, ICU type, and whether or not the ICU focused its program implementation efforts on the relevant infection (CLABSI, CAUTI or both).

Changes in CAUTI Rates and Urinary Catheter Utilization

The unadjusted CAUTI and urinary catheter utilization rates over the continuum of the project are illustrated in Figure 3. Among all participating units reporting CAUTI outcomes, the unadjusted rates of CAUTI decreased by 10.7%, from 1.25 CAUTI per 1,000 urinary catheter days at the end of the pre-intervention period to 1.11 per 1,000 urinary catheter days at the end of the intervention period. Urinary catheter utilization decreased 6.7% from 64% to 59% over the course of the project. CAUTI and urinary catheter utilization by ICU type is provided in supplementary online Appendix Figure 3.

Figure 3.

Catheter-Associated Urinary Tract Infection (CAUTI) Rates and Urinary Catheter Utilization Rate, as reported to the National Healthcare Safety Network (NHSN)

In adjusted analyses (Table 2), the CAUTI rate decreased from 1.47 at the end of the pre-intervention period to 1.28 at the end of the intervention. There was no significant change in CAUTI rates during the pre-intervention period (IRR=0.96, 95% CI=0.80–1.15, p=0.66). During the intervention period the rates decreased 24% (IRR=0.76, 95% CI=0.60–0.96, p=0.02), however, the rates did not change significantly over time when comparing the pre-intervention to the intervention period (IRR=0.79, 95% CI=0.59–1.06, p=0.12). Urinary catheter utilization decreased from 63% to 59% during the intervention period. During the pre-intervention period there was a 5% reduction in catheter utilization (IRR=0.95, 95% CI=0.93–0.97, p<.0001). Rates decreased at a nearly identical rate during the intervention period (IRR=0.93, 95% CI=0.91–0.95, p<0.001) as the change from the pre-intervention to the intervention period was a non-statistically significant 2% reduction (IRR=0.98, 95% CI=0.95–1.01, p=0.14).

Sensitivity Analyses

For CLABSI, a total of 146 (53.3%) ICUs had positive CAD values, 111 (40.5%) were negative, and 17 (6.2%) were missing. Among ICUs with positive CAD values, CLABSI rates did not change significantly over time when comparing the pre-intervention to the intervention period (IRR=1.23, 95% CI=0.78–1.95, p=0.38). We did, however, observe a statistically significant reduction in central line utilization from pre- to post-intervention (IRR=0.95, 95% CI=0.90–0.99, p=0.02). For CAUTI, a total of 133 (48.5%) ICUs had positive CAD values, 123 (44.9%) were negative, and 18 (6.6%) were missing. Among ICUs with positive CAD values, CAUTI rates (IRR=0.93, 95% CI=0.64–1.35, p=0.71) and urinary catheter utilization (IRR=0.99, 95% CI=0.96–1.03, p=0.61) did not change significantly over time when comparing the pre-intervention to the intervention period.

Measures of Program Participation

The ICU Assessment Tool was submitted by 247 (90%) of the 274 ICUs at baseline, and 97 (35%) of ICUs at follow-up. All state hospital association leads met the program requirement for performing site visits; unit-specific action plans were submitted by 100 ICUs. Appendix Figure 4 summarizes participation of the ICUs for viewing the modules and webinars; overall, participation by module and webinar viewing varied by educational product, but was higher for those with live and interactive components such as the virtual learning group webinars, as opposed to on-demand only modules.

DISCUSSION

We report the results from the first 2 cohorts of this collaborative for ICUs identified as having elevated CLABSI and CAUTI rates at baseline. Despite expected challenges in recruiting, engaging, and performing site visits for struggling units, this program successfully recruited and engaged a large number and variety of ICUs. This project also rapidly conducted a systematic literature review¹⁸ to inform and tailor educational resources to ICUs. In addition to live educational webinars, the project created on-demand brief video modules in order to be more flexible for ICU clinicians. Despite these efforts, many of the project’s training and coaching resources were underutilized or not accessed by a number of ICUs. We anticipate this may have contributed to the lack of a statistically significant improvement in CLABSI, CAUTI, urinary catheter utilization and central line utilization rates in the intervention period compared to the rate of decline noted in the 12 month pre-intervention period.

This program has three notable features that provide context to these findings that may contribute to why this program did not show the same success in reductions seen in prior collaboratives for CLABSI^19,20 and CAUTI¹⁴. First, this program specifically targeted only ICUs, which are complex hospital units that have not previously shown success in reducing urinary catheter use or CAUTI rates in the prior collaborative¹⁴, and units in which the rate of improvement in CLABSI has slowed in the past few years after earlier successes in CLABSI prevention.^5,14 Second, this program specifically aimed to recruit ICUs that were identified as having elevated CLABSI or CAUTI rates at baseline, as opposed to all-comers recruited in prior AHRQ-funded projects^14,19,20 which could have recruited units that could be described as ‘early adopters’ or ‘early majority’⁴⁷ of technical strategies and behavior changes to reduce catheter-associated infections. To our understanding, unlike the prior collaboratives^14,19,20, this program may also be among the first to recruit hospitals and units by the relatively new CAD metric generated from NHSN data based on the CDC’s TAP strategy which was designed to rank facilities or locations (i.e., units) in order to identify and target areas where the greatest prevention impact could be achieved.⁴⁸ Third, this collaborative obtained and analyzed pre-intervention data for the participating hospitals over a long period (12 months) to allow reliable calculation of the slope of change in CLABSI, CAUTI, and device rates, which permitted a comparison of slope of outcome changes in both the pre-intervention and intervention phases, to account for secular trends in changes in slope over time. In contrast, the prior CLABSI collaborative²⁰ compared the post-intervention outcomes to baseline data that was a weighted average over 4 baseline quarters,and the prior CAUTI collaborative¹⁴ included only 3 months of baseline data for comparison.

Noting the secular trends that were occurring in national CLABSI and CAUTI rates in the pre-intervention period, it is feasible that catheter-infection rates have reduced in recent years to levels for which improvement is becoming increasingly difficult, possibly reaching a ceiling effect. Comparing this collaborative’s 2014–2015 pre-intervention ICU CLABSI rates (1.08 per 1000 central line days) and ICU CAUTI rates (1.25 per 1000 urinary catheter days) to prior collaboratives’ data highlights that ICUs in the current project, though targeted for having excess CLABSI and CAUTI, have much lower rates than prior programs. For example, in the AHRQ On the CUSP: Stop CAUTI project with data from 2011–2013, CAUTI rates at the end of the intervention were 2.50 per 1000 catheter days in ICUs, and 1.52 per 1000 catheter days in non-ICUs.¹⁴ Similarly, in the AHRQ On the CUSP: Stop CLABSI program using 2009–2011 data, CLABSI rates were 1.96 per 1000 catheter days at the project start and 1.15 per 1000 catheter days at project end 16–18 months after implementation.²⁰ However, NHSN rates are also impacted by definition changes occurring between 2011 and 2015, such as a major NHSN CAUTI definition change that removed yeast as qualifying organism in urine culture, which was associated with lower NHSN CAUTI rates.

In a recent systematic review of qualitative studies of healthcare organizations struggling to improve quality,⁴⁹ several domains, such as inadequate infrastructure, that can impede improvements were identified that ICUs in this project may be experiencing more than units in prior projects, for which intervention by on-line and virtual resources, outcome feedback, and the externally-facilitated implementation approach used in this project may be inadequate. For example, this project included some resources for helping units build a team, improve accountability, collaboration, and leadership, and use quality improvement tools such as using data for feedback, though the project did not provide other explicit CUSP-focused training²⁷ as compared to prior projects.^14,20,27 A lesser focus on the adaptive elements of the CUSP model, such as engagement, may have had a bearing on the underutilization of the project’s training and coaching resources. Also, the project could not address common infrastructure barriers such as insufficient staffing and high turnover, often associated with lack of staff resources to internally facilitate and champion the project within the ICU. We also anticipate (but only have limited evidence from site visits and coaching calls for this project to support this) that many of the participating ICUs and their hospitals could have been impacted by “system shocks”⁴⁹ or “disruptive events”,⁵⁰ described as an organization-wide event or change that detracts from day-to-day operations; examples include new electronic health records, mergers or reorganizations, C-suite turnover, and financial difficulties impacting staff and technology resources. Although formal qualitative data collection was not performed for this project, lessons learned from the site visits and coaching calls suggested that major challenges included hospital senior leadership not being engaged, and limited staff resources and time available to focus on this project’s implementation.

We also saw opportunities to tailor the educational and peer coaching resources for ICU type for the future cohorts, and the need to balance the flexibility of on-demand modules for busy clinicians with obtaining team commitment to more scheduled coaching and opportunities to reinforce and expand the education provided in the on-demand educational modules. We also recognized that this program may have required more effort of the ICUs to assess gaps and develop their own individual ICU action plans in contrast to the prior AHRQ-funded programs^14,20 which each implemented a single bundle of a limited number of interventions to prevent CLABSI and CAUTI, respectively, before these types of bundles became standard. Although this program’s approach allowed for and prioritized tailoring the intervention components to the individual ICU’s strengths and needs, this design could potentially be burdensome to ICUs that are struggling at baseline due to suboptimal staffing, resources, and team communication strategies.

Important limitations of this study should be considered. Because it was not a randomized trial, confounding variables could impact the results. For future similar projects, we believe it would be important to consider effectiveness-implementation hybrid trial designs (e.g., stepped wedge cluster randomized trials; sequential, multiple assignment, randomized trials) to assess both the implementation strategy at the unit-level and specific health outcomes. Such design considerations will need to plan for adequate allocation of resources to permit more detailed evaluations of the implementation given collection of implementation metrics is much more resource intensive and not in the usual scope of clinicians in the participating ICUs. Additionally, design decisions should be finalized prior to contract execution, since the trial design impacts contract deliverables such as recruitment strategies and goals, and length of evaluation for the cohorts. Program participation was also voluntary among those invited to participate, so findings may not be generalizable to all ICUs in the United States. The measures of program participation also have important limitations. For example, to be counted as having viewed or participated in each individual webinar or module, an ICU must have had at least one team member access the webinar or module on-line, for any length of time, but the program was unable to track individuals who accessed a webinar by phone without logging into the webinar platform. Also, the reported results may under-represent ICUs use of the on-demand materials, as the results do not include sharing of the materials by means other than by accessing the online material directly; for example, some state leads reported downloading the module slides and distributing them to the ICU teams by email. Also, although we were able to perform a more rigorous assessment of secular trends in the pre-intervention period compared to similar collaboratives given our longer pre-intervention data collection period, we cannot compare the secular trends noted in our program’s participants to non-participants, as we did not have access to the NHSN data for non-participating units in this same time period. Also, although the new CAD metric used to assess eligibility was designed to identify units with the greatest potential for improving infection rates, to our knowledge, this was among the first use of this metric to recruit hospitals and units into an intervention project. It is feasible that although the CAD metric measures and quantitatively ranks opportunities for infection prevention, long-term performance of the CAD metric for intervention prioritization and as a predictor of the likelihood of intervention success remains unclear. Also, although the majority of recruited units had positive CAD values, a large subset of participating units had low or negative CAD values. As such, it is possible that inclusion of these units limited our ability to observe CLABSI and CAUTI reductions. However, we conducted sensitivity analyses only focused on units with positive CAD values and still observed no statistically significant reductions in CLABSI, CAUTI, or urinary catheter utilization, though a significant reduction in central line utilization was observed. Finally, the results presented here from the first 2 cohorts of the project are from a smaller number of units compared to prior intervention efforts^14,20 and may not reflect the final impact of the program across all participating cohorts given the project is ongoing for later cohorts. Still, the intervention slope change coefficients for CAUTI, CLABSI and device utilization rates, while generally in the desired direction of improvement (as better illustrated by our presented 95% confidence intervals), did not meet our criteria for statistical significance.

Despite these limitations, and lack of statistically significant improvements in these first two cohorts, we feel this project has demonstrated important success in rapidly recruiting ICUs and hospitals targeted by baseline performance challenges, developing resources specific for addressing catheter use challenges in ICU settings, as well as encouraging more collaboration between ICUs and state hospital associations which may contribute to sustaining quality improvement work beyond this project. Lessons learned from exit interviews conducted with state leads in this project have been applied to the expansion of this project to other regions of the U.S. which began in September 2017. Given concerns that low participation could be a contributor to lack of statistically significant improvement in outcomes, important modifications to the program materials and approach have been implemented in the ongoing project. First, there is a greater emphasis on adaptive components of the CUSP model to increase engagement and ownership at multiple levels to prevent infections in the ICU. All of the onboarding materials for the expansion project incorporate a more explicit emphasis of CUSP and other adaptive/cultural concepts such as reinforcing the message that this program must be implemented by a team and not solely by the infection preventionist or a single individual in the ICU. Tools were also developed or made available to better support the use and application of CUSP, such as action planning templates to enable easier unit customization of their intervention based on their responses to the ICU Assessment Tool, CLABSI and CAUTI event planning tools to accompany the CUSP Learn from Defects Tool, and new short CUSP training videos and companion audio files of interviews with experts to help ICUs overcome common cultural challenges. Intervention and implementation resources are being augmented to better address the challenge of engaging hospital leadership and to provide coaching and peer support specific to ICU type where possible. These enhancements include requiring a letter of commitment from a hospital senior leader that also includes signatures from the unit lead and quality improvement department head to promote accountability and ease of follow-up, providing additional coaching support through onsite training to the state leads, providing additional resources to state leads to host in-person meetings for their ICUs to increase engagement, peer-to-peer learning and real-time coaching, inviting national program team subject matter experts on site visits to provide extra support to challenged ICUs and to model coaching for state leads, and hosting affinity group webinars to “specialty ICUs” (e.g., neuro ICUs, burn/trauma ICUs) to offer targeted coaching and peer-to-peer learning for ICUs with specific patient populations.