Introducing a Population-Based Outcome Measure to Evaluate the Effect of Interventions to Reduce Catheter-Associated Urinary Tract Infection

Abstract

Background

The catheter-associated urinary tract infection (CAUTI) measure recommended by the National Healthcare Safety Network (NHSN) accounts for the risk of infection in patients with an indwelling urinary catheter, but may not adequately reflect all efforts in enhancing patient safety by reducing urinary catheter use.

Methods

We used computer-based Monte Carlo simulation to compare the NHSN-recommended CAUTI rate (CAUTI per 1,000 catheter days) to the proposed “population CAUTI rate” (CAUTI per 10,000 patient days). We simulated 100 interventions with a wide range of effects on catheter utilization and CAUTI risk among those with catheters, and then compared the two measures pre- and post-intervention across the simulated interventions.

Results

A total of 93 of our 100 simulated interventions yielded reductions in CAUTI; however, in 25 (27%) of the 93 simulations the NHSN CAUTI rate increased after the intervention. In addition, among the 68 simulations in which both the NHSN and the population CAUTI rates decreased, percent decreases in the population CAUTI rate were consistently greater than those in the NHSN rate.

Conclusions

The population CAUTI rate – CAUTIs per 10,000 patient-days – should be calculated along with NHSN rate, particularly in settings where interventions lead to substantial reductions in catheter placement. We suspect this population CAUTI rate may eventually emerge as a primary outcome for hospital-based quality improvement interventions for reducing urinary catheter utilization, especially those focusing on avoiding urinary catheter placement.

INTRODUCTION

Healthcare-associated infections (HAIs) have been linked to significant morbidity and mortality; a urinary tract source accounts for 36% of these infections.¹ Catheter-associated urinary tract infection (CAUTI) acquired during hospitalization is considered a “reasonably preventable” hospital-acquired condition by the Centers for Medicare and Medicaid Services and reimbursement for this condition is denied as of 1 October 2008.^{2, 3} The CAUTI rate is used now as a measure for comparing hospital performance in patient safety, and is included in some mandated state public reporting initiatives and consumer-directed hospital performance information.⁴ Moreover, the United States Department of Health & Human Services Action Plan to prevent HAIs calls for a reduction in the number of symptomatic CAUTI per 1,000 urinary catheter days in the hospital by 25% as a national prevention target.⁵

Despite this increased focus on CAUTI, there are no specific recommendations^6,7 for how data regarding CAUTI should be externally reported to consumers or payers, in contrast to specific recommendations provided for reporting of other HAIs.⁸ Presently, the most common CAUTI metric used is from the Centers for Disease Control and Prevention’s National Healthcare Safety Network (NHSN). The CAUTI rate used by NHSN is calculated by multiplying the number of CAUTI episodes during a period of time by 1,000 and dividing it by the total number of indwelling urinary catheter days during the same period.⁹ While the NHSN CAUTI rate accounts for the risk of infection in patients with an indwelling urinary catheter, it does not account for the risk to the total hospitalized patient population. In addition, it is unclear whether the NHSN CAUTI rate will adequately capture the effect of all interventions to reduce urinary catheter utilization, which are key strategies for preventing CAUTI in hospitalized patients⁶ given the frequent use of urinary catheters without appropriate indications.^10–12 In addition, since the need for a urinary catheter may reflect greater severity of illness, interventions to improve appropriate urinary catheter utilization could even result in a population at higher risk for CAUTI among those patients with a urinary catheter.¹³ With these issues in mind, we compared the current NHSN CAUTI rate to a population-based CAUTI rate using different simulated scenarios related to potential interventions to reduce catheter utilization in order to provide guidance as to the most appropriate measure to assess CAUTI prevention activities.

METHODS

We developed a computer-based Monte Carlo simulation model of CAUTI to compare the NHSN CAUTI rate and the population CAUTI rate across a wide range of hypothetical interventions. Simulation is an appropriate and effective way of generating data for analysis and testing under a large number of possible scenarios, ¹⁴ and has been used in many diverse health-related applications. ¹⁵ The population CAUTI rate was derived by multiplying the number of CAUTI episodes occurring during a period of time by 10,000 and dividing it by the total number of patient-days during the same time period (i.e., CAUTI/10,000 patient days). For each of 100 simulated interventions, 100,000 simulated patients were randomly assigned to be hospitalized pre- or post-intervention, and each patient was then assigned two underlying probabilities: the probability of having a catheter placed, and the probability of acquiring a CAUTI given that a catheter had been placed. Figure 1 shows the structure of the simulation model. Each patient follows one of the twelve paths for each intervention, and is assigned a different probability of CAUTI accordingly.

Figure 1.

The structure of the simulation model

We assumed that the probability of having a catheter placed for any given patient followed a logit model, and depended only on whether the patient was hospitalized prior to or following the intervention. Conservative estimates from previous studies to reduce urinary catheter utilization have shown the pre-intervention mean catheter duration to be approximately five days (range 5–11 days).¹⁶ As such, we simulated seven days of total follow-up for each patient, with the duration of catheterization (for those with a catheter) following a discrete uniform distribution ranging between three and seven days. This model implicitly assumes the probability of catheter placement is constant across time. In the absence of an intervention, the probability of catheter placement was assumed to equal 0.24, which is the 75^th percentile of the catheter utilization ratio in medical inpatient wards reported by the NHSN.⁹ We set pre-intervention utilization at the 75^th percentile so that post-intervention utilization could vary between the 25^th and 75^th percentiles, since catheter utilization was not allowed to increase from pre- to post-intervention in our simulation.

We assumed that the probability of CAUTI given catheter placement also followed a logit model, but one that depended on duration of catheterization as well as whether the patient was hospitalized prior to or following the intervention. The pre-intervention risk of CAUTI among those with catheters was assumed to equal 6.7/1,000 catheter-days, the NHSN-reported pooled mean that was observed in medical inpatient wards. The post-intervention CAUTI risk was then allowed to vary between the NHSN 25^th and 75^th percentiles. Note that CAUTI risk can decrease or increase following intervention, while utilization can only decrease or remain constant. Following Garibaldi et al.,¹⁷ the risk of CAUTI was assumed to increase by 5% with each additional day a catheter remained in place.

Simulated interventions were characterized by their effects on the two patient-level probabilities. These effects were varied in each round of simulations so as to capture a wide range of potential interventions seen in clinical practice. Specifically, the effect on catheter utilization was allowed to vary between a 57% decrease and no effect, and the effect on the risk of CAUTI among those with catheters was allowed to vary between a 50% decrease and a 35% increase. These ranges were chosen so as to allow the post-intervention probabilities of catheter placement and CAUTI (among those with catheters) to vary between the NHSN-reported 25^th and 75^th percentiles of the catheter utilization ratio and NHSN CAUTI rate, respectively.⁹

Although an intervention for CAUTI reduction should not directly increase the risk of CAUTI among those with catheters, our simulation accounted for interventions that could plausibly indirectly increase the measured CAUTI rates. For example, an intervention that reduced catheter placement – but only among those with the lowest risk of CAUTI – would leave only high-risk patients to be catheterized, thereby yielding a potential increase in the observed CAUTI rate among catheterized patients (even if the intervention directly caused a net decrease in the number of CAUTIs among all patients at risk).

RESULTS

A total of 93 of the 100 simulated interventions yielded reductions in the number of CAUTIs and population CAUTI rate; however, in 25 (27%) of the 93 simulations the NHSN CAUTI rate increased after the intervention. All but two of these 25 simulated interventions led to an increase in CAUTI risk among patients with catheters, and therefore represent interventions that decrease catheter utilization for only those patients who are at low risk for CAUTI (Table 1). Furthermore, across the 68 simulations that led to a reduction in both the NHSN and population CAUTI rates, the population CAUTI rate always decreased to a greater extent than the NHSN rate (in fact, as shown in the Appendix, this must be the case if utilization does not increase following intervention).

Table 1.

The 25 simulations where the National Healthcare Safety Network (NHSN) catheter-associated urinary tract infection (CAUTI) rate increases despite a reduction in CAUTIs.

Scenario	% Change in Catheter Use	% Change in CAUTI Risk	% Change in NHSN CAUTI Rate	% Change in Population CAUTI Rate
1	−85	7.8	8.5	−46.2
2	−85	18.9	23.5	−39.1
3	−85	30	28.1	−36.6
4	−75.6	7.8	17.7	−35.4
5	−75.6	18.9	20.3	−35.7
6	−75.6	30	37.5	−26
7	−66.1	7.8	16.5	−32.1
8	−66.1	18.9	28.8	−26.6
9	−66.1	30	30.9	−23.1
10	−56.7	−3.3	2.9	−35.4
11	−56.7	7.8	12.3	−28.4
12	−56.7	18.9	23.3	−22.5
13	−56.7	30	40.7	−9.5
14	−47.2	7.8	11.3	−22.7
15	−47.2	18.9	26.2	−13.9
16	−47.2	30	29.8	−10.4
17	−37.8	−3.3	11.3	−18.2
18	−37.8	7.8	16.8	−13.5
19	−37.8	18.9	23.4	−8.6
20	−37.8	30	30.2	−4.4
21	−28.3	7.8	4.8	−16.5
22	−28.3	18.9	18.3	−4.5
23	−18.9	7.8	6.3	−8.7
24	−18.9	18.9	15.9	−0.9
25	−9.4	7.8	7.4	−1.2

Figure 2 shows the percent change in the population CAUTI (left) and NHSN CAUTI (right) rates, across a wide range of interventions. The two effects characterizing the simulated interventions – the effect on catheter utilization and the effect on the CAUTI risk among those with catheters – are given on the X- and Y-axes, respectively. Each point in each figure therefore corresponds to a different intervention; the color at each point reflects the percent change from pre- to post-intervention in the appropriate measure (red indicates an increase, while blue indicates a decrease). Moreover, darker blues indicate larger percent reductions in the rate studied, while darker reds indicate larger percent increases. White indicates no change in the rate. This is akin to a topographic map, in which color reflects changes in elevation.

Figure 2.

Percent change in population CAUTI Rate and NHSN CAUTI rate across a wide range of interventions

Notes: (i) CAUTI: catheter-associated urinary tract infection; NHSN: National Healthcare Safety Network. (ii) Points A, B, and C each reflect different simulated interventions. (iii) Red areas indicate that, for interventions represented by these areas, the measure increased from pre- to post-intervention, and similarly blue areas indicate that the measure decreased (the magnitude of the increase or decrease is given by the bar in the center in terms of percent change).

To compare the two measures’ performance, we discuss below the simulation results for three specific interventions, labeled intervention A, B, and C, each with varying effects on catheter utilization and CAUTI rate among those with catheters.

Intervention A decreases catheter utilization by 53% (X-axis) but increases the risk of CAUTI among those with catheters (Y-axis) by 21% (holding duration of catheterization constant). This could correspond to an intervention for which catheter placement and/ or utilization is diminished primarily among those who are least susceptible to CAUTI, leaving only those with a high risk of CAUTI to be catheterized. In this case, the intervention yields markedly different effects on CAUTI rates from pre- to post-intervention, with the NHSN CAUTI rate increasing by 20% and the population CAUTI rate decreasing by 36%.

Intervention B decreases the probability of catheter utilization by 38% and decreases the CAUTI rate among those with catheters by 23%. This intervention lies in the middle range of those we investigated. Here the NHSN CAUTI rate decreases by 26% while the population CAUTI rate decreases by 48%.

Intervention C decreases the probability of catheter utilization by only 9%, but decreases the CAUTI rate among those with catheters by 45%. This could correspond to an intervention that ensures the use of aseptic insertion technique and the use of sterile equipment. For this intervention, both measures display roughly the same behavior, with the NHSN CAUTI rate decreasing by 45% and the population CAUTI rate decreasing by 48%.

The horizontal bands for the NHSN CAUTI rate in Figure 2 indicate that the NHSN CAUTI rate gives no information about the effects of interventions on catheter utilization. In particular, the percent change in this measure tracks only the effects on the risk of CAUTI among those with catheters, and is completely insensitive to variation in effects on catheter placement. However, Figure 2 reveals that the bands for the population CAUTI rate are diagonal, indicating that this measure is sensitive to both utilization and risk of CAUTI. The population CAUTI rate, which reflects the number of CAUTIs standardized by total population size (rather than the CAUTI rate among those with catheters), is therefore more nuanced with respect to relaying information about the different potential effects of interventions for CAUTI reduction.

DISCUSSION

To adequately reflect the quality of patient care, measures used to report CAUTI events should ideally capture the effect of quality improvement interventions intended to reduce the use of urinary catheters. This study underscores the importance of the denominator in influencing what an outcome metric is capable of detecting and representing. Our analysis demonstrates that, although a useful and valid metric under certain conditions, the NHSN CAUTI rate may not always detect the impact of important quality improvement efforts targeted at reducing inappropriate urinary catheter utilization. Many of the clinical interventions to reduce urinary catheter utilization have targeted removal of catheters that are no longer needed or unnecessary rather than preventing placement.^16,18–26 Studies have shown that interventions focusing on removing unnecessary catheters (that had been already placed) led to reductions in CAUTI rates,^{18–21,23,24} and resulted in a lower mean duration of catheter utilization.^18,24,25 A study that addressed both avoiding placement of unnecessary catheters and prompt removal of those no longer needed also reported lower duration of catheter utilization and infections per 100 cases.²⁷ Two studies of interventions initiated in the emergency department setting that promoted placement of catheters based on appropriate indications resulted in fewer catheters placed but did not evaluate the impact on CAUTI rates in the hospital setting.^28,29

As our analysis suggests, one of the situations where the NHSN CAUTI rate may not reflect improvements in healthcare quality and patient safety is when interventions are employed to prevent inappropriate catheter placement. Although this type of intervention may result in a significant reduction in the total number of CAUTIs, the NHSN CAUTI rate may increase following the intervention. Furthermore, our results indicate that the NHSN CAUTI rate may underestimate the effect of certain interventions, when compared to the population CAUTI rate, even if both measures show rate reductions. Therefore, surveillance efforts using the NHSN CAUTI rate could lead to the erroneous conclusion that an intervention did not improve outcomes, which may affect hospitals adversely in the environment of pay for performance where top performers may be better compensated.³⁰ As such, the population CAUTI rate may more accurately reflect the magnitude of improvements in CAUTI prevention stemming from quality improvement and patient safety efforts.

In order to better evaluate the effect of an intervention to reduce the risk of CAUTIs, we propose the use of a measure that incorporates the risk to all patients cared for at the hospital. The role of an outcome measure is to accurately reflect the final outcome, in this case the number of CAUTIs. The end goal is to achieve a reduction of total CAUTIs over a period of time for the same population at risk. We suggest using both the NHSN CAUTI rate and the proposed population CAUTI rate, which uses patient-days as the denominator. Measures with patient-days as denominators have been used to evaluate risk of exposure of healthcare workers to bloodborne pathogens,³¹ as an incidence rate for methicillin resistant Staphylococcus aureus infection,³² as well as an acquisition measure for Clostridium difficile in the hospital setting.³³ Such denominators include all patients at risk for exposure and the duration of potential risk, whether they are exposed or not. This more accurately reflects the potential risk, which starts when the patient enters the hospital. As all patients without urinary catheters in place admitted to the hospital are at risk for having a urinary catheter placed, calculating the population CAUTI rate based on patient-days may better reflect both interventions that target appropriate placement and prompt removal of catheters that are no longer needed.

Importantly, the population CAUTI rate accounts for both the information contained within the NHSN CAUTI rate and the catheter utilization ratio (Table 2). The current NHSN CAUTI rate is based on the number of CAUTIs compared to the utilization days of the catheters. This measure, used in conjunction with the device utilization ratio, may suitably reflect process improvement in the intensive care unit setting where the majority of catheters placed initially are appropriate and where patients have similar acuity. In addition, collecting data related to device-days is relatively easy in intensive care units because of the high prevalence of device utilization and few units involved. On the other hand, collection of catheter days is cumbersome outside of the critical care setting and labor intensive unless electronic surveillance is available.³⁴ A logistical advantage of the population CAUTI rate is that one of its components (patient-days) is readily available to individual hospitals. Furthermore, it can easily be calculated from the data currently submitted to NHSN.

Table 2.

The relation between the catheter utilization ratio, the National Healthcare Safety Network (NHSN) catheter-associated urinary tract infection (CAUTI) rate and the proposed population CAUTI Rate

Metric	Calculation
NHSN CAUTI rate	(Total number of CAUTIs/total catheter-days) ×1,000
Catheter utilization ratio	Total catheter-days/ total patient-days
Total patient-days	Total catheter-days/ catheter utilization ratio
Population CAUTI Rate	(Total number of CAUTIs/ total patient-days) ×10,000 = (Total number of CAUTIs/ total catheter-days) ×catheter utilization ratio ×10,000= NHSN CAUTI rate ×catheter utilization ratio ×10

Our findings should be interpreted in the context of the assumptions used to develop our simulation model. For example, we assumed that the simulated interventions do not directly affect the duration of catheterization (among those with catheters placed). Simulation models addressing changes in the duration of use may provide for more comprehensive evaluation of the NHSN CAUTI rate and its relation to the population CAUTI rate. When duration of catheter use is reduced through early removal of catheters that are no longer needed, the NHSN CAUTI rate will vary depending on the risk for patients with continued need of catheterization. Even if the NHSN CAUTI rate decreases, however, it can be shown that the magnitude of this decrease will not match that of the population CAUTI rate, provided utilization does not increase following intervention (see Appendix).

Further, although we did make assumptions regarding the processes by which catheters are placed and CAUTIs occur, we are primarily interested in the relationship between the NHSN CAUTI rate and the population CAUTI rate; this relationship is fixed and only partly dependent upon the realism of the underlying simulation model. The main limitation of our work is that we only explored the case in which the sole difference between the pre- and post-intervention populations is the intervention itself. Thus, when case mix is more than a minor concern – either across hospitals or within a hospital across time – our simulation results may be less relevant.

Importantly, we are suggesting that the population CAUTI rate complement the NHSN CAUTI rate when evaluating hospitals for quality improvement projects to reduce CAUTI. The population CAUTI rate does not evaluate the risk to those with an indwelling urinary catheter; rather, it reflects the risk of CAUTI for patients in a hospital setting regardless of catheter status. The NHSN CAUTI rate will continue to be the best measure to evaluate the risks associated with any breach of the aseptic process when placing and maintaining the urinary catheter.

Our findings have significant implications for choosing outcome measures to evaluate the effect of programs to promote appropriate utilization of urinary catheters in the hospital setting. The population CAUTI rate can be particularly useful in evaluating improvement programs within the same institution; on the other hand, the NHSN CAUTI rate is essential for comparing specific units from different facilities, especially in the intensive care setting.⁸ As states implement projects to reduce CAUTI, the concomitant evaluation of both rates using empirical data will provide better answers on how to use each of the two measures.^35,36

We conclude that both the NHSN CAUTI rate (with catheter-days as the denominator) and the proposed population CAUTI rate (with patient-days as the denominator) are needed to better evaluate the quality improvement processes related to urinary catheter utilization. The NHSN CAUTI rate will continue to be used to evaluate CAUTI in the intensive care or specialty units, and serve as a tool for inter-hospital comparisons. The population CAUTI rate, however, is likely to better reflect within-institution improvement processes, especially those interventions that promote preventing inappropriate urinary catheter placement.

Appendix

Suppose that P₀ and P₁ are the pre- and post-intervention population CAUTI rates, respectively, and that N₀ and N₁ are the pre- and post-intervention NHSN CAUTI rates. Then the percent changes in these rates are given by (P₁ − P₀) / P₀ and (N₁ − N₀) / N₀, respectively.

From Table 2, we have P₀ = 10×N₀U₀ and P₁ = 10×N₁U₁, where U₀ and U₁ are the pre- and post-intervention utilization ratios. If utilization does not increase following intervention (i.e., if U₁ ≤ U₀), we have:

$$({\text{P}}_1-{\text{P}}_0)/{\text{P}}_0=({\text{N}}_1{\text{U}}_1-{\text{N}}_0{\text{U}}_0)/{\text{N}}_0{\text{U}}_0\le ({\text{N}}_1{\text{U}}_0-{\text{N}}_0{\text{U}}_0)/{\text{N}}_0{\text{U}}_0=({\text{N}}_1-{\text{N}}_0)/{\text{N}}_0$$

Therefore the percent change in the population CAUTI rate is necessarily less than (or equal to) the percent change in the NHSN CAUTI rate, as long as utilization does not increase following intervention.