Cost-effectiveness of a quality improvement programme to reduce central line-associated bloodstream infections in intensive care units in the USA

Abstract

Objective

To assess the cost-effectiveness of a multifaceted quality improvement programme focused on reducing central line-associated bloodstream infections in intensive care units.

Design

Cost-effectiveness analysis using a decision tree model to compare programme to non-programme intensive care units.

Setting

USA.

Population

Adult patients in the intensive care unit.

Costs

Economic costs of the programme and of central line-associated bloodstream infections were estimated from the perspective of the hospital and presented in 2013 US dollars.

Main outcome measures

Central line-associated bloodstream infections prevented, deaths averted due to central line-associated bloodstream infections prevented, and incremental cost-effectiveness ratios. Probabilistic sensitivity analysis was performed.

Results

Compared with current practice, the programme is strongly dominant and reduces bloodstream infections and deaths at no additional cost. The probabilistic sensitivity analysis showed that there was an almost 80% probability that the programme reduces bloodstream infections and the infections’ economic costs to hospitals. The opportunity cost of a bloodstream infection to a hospital was the most important model parameter in these analyses.

Conclusions

This multifaceted quality improvement programme, as it is currently implemented by hospitals on an increasingly large scale in the USA, likely reduces the economic costs of central line-associated bloodstream infections for US hospitals. Awareness among hospitals about the programme's benefits should enhance implementation. The programme's implementation has the potential to substantially reduce morbidity, mortality and economic costs associated with central line-associated bloodstream infections.

Strengths and limitations of this study.

• This study was conducted according to best practices in cost-effectiveness analysis and demonstrates that a multifaceted quality improvement programme can reduce the economic costs of central line-associated bloodstream infections for hospitals.

• We used nationally representative data sources to increase generalisability and performed a probabilistic sensitivity analysis to quantify the uncertainty in our cost-effectiveness estimates.

• Owing to data limitations we were unable to assess the impact of patient heterogeneity, such as demographics and clinical characteristics, on baseline risk, treatment effect or resource utilisation. We did not evaluate costs outside the acute hospital setting, such as rehabilitation costs or productivity losses for delays in returning to work.

Introduction

Central line-associated bloodstream infections (CLABSI) are common, expensive to payers and patients, and potentially fatal.^{1 2} Each year, nearly 80 000 Americans develop CLABSIs in intensive care units (ICU), and more than 25 000 of these patients die.³ A single infection can cost payers as much as $56 000, culminating in over $2 billion in related costs per year in the USA.⁴ CLABSIs in ICU patients have an estimated attributable mortality rate of 14–40%, with a prolonged length of stay of 7.5–25 days.^{5 6}

The Keystone ICU project, first launched in Michigan in 2004 and since scaled across the USA, Spain, Peru, Pakistan and the UK, has captured the interest and attention of patients, payers and policymakers for its substantial, sustained and scalable reductions in preventable nosocomial infections. Over 1200 US hospitals are currently participating in this multifaceted quality improvement programme through On the CUSP: Stop BSI, a national collaborative, and many others are likely using checklists and infection prevention programmes in their ICUs as standard practice. The programme has been evaluated through prospective cohort studies,^7–10 retrospective observational studies using claims data,¹¹ and both cluster non-randomised¹² and randomised controlled trials.¹³ When viewed collectively, this evidence suggests that the programme is associated with substantial reductions in CLABSIs and mortality in ICU patients.

In spite of commendable investment in this programme to manage the undesirable consequences of CLABSIs, an important question remains unanswered: compared with current practice, is this programme cost-effective for US hospitals? Reporting of economic data in quality improvement studies is uncommon, and there are few formal cost-effectiveness analyses of quality improvement programmes.^14–16 Similarly, because the estimated gross costs of CLABSIs to the healthcare system are very high, the conclusion that expanding infection control efforts will be cost saving (relative to the costs incurred by expanded efforts) is accepted without rigorous analysis.¹⁷ This paper examines the cost changes and cost-effectiveness of the Keystone ICU project from the perspective of the hospital using nationally representative data sources from the USA.

Methods

Overview of the analysis

We developed a decision tree model to address the choice faced at an individual hospital about implementing the programme (figure 1). The use of a decision tree approach is justified by the short-term progression of CLABSIs. The model assumes that patients do not experience other adverse effects of catheterisation, such as catheter colonisation leading to local infection, hypersensitivity reactions or mechanical complications such as pneumothorax. The Keystone ICU project instead focused on infectious complications, because they are more common, more costly and often fatal.¹⁸ Consistent with other economic evaluations of CLABSIs in the ICU setting, we assumed that the consequences of infection are independent of age, patient disease severity and the causative organism.^{18 19} These assumptions are congruent with the programme itself, which does not discriminate between subgroups of patients based on these factors.

Figure 1.

Decision tree model. Decision tree model depicting programme versus no programme and its effects on outcomes in intensive care unit (ICU) patients. ‘Bloodstream infections’ refers to central line-associated bloodstream infections.

The comparator was current practice as the most realistic alternative faced by organisations seeking to implement the programme. Current practice encompasses on-going or existing activities that might influence the risk of infection among patients, such as the use of anti-infective central venous catheters.

The target population consisted of adult (18 years or older) ICU patients in accordance with studies of the Keystone ICU project and its subsequent replications.¹³ Since costs and benefits only accrue while the programme is implemented, the time frame and analytic horizon are fundamentally the same. We used a time horizon of 5 years. This analysis was performed from the hospital's perspective. Our study aims to address the following question: is implementation of this quality improvement programme to decrease CLABSIs in the ICU a cost-effective approach when compared with current practice in US hospitals?

Quality improvement programme

Details of the programme and its conceptual approach have been described elsewhere.^20–22 In brief, this multifaceted programme employed clinician communication tools, teamwork, and safety culture assessment and improvement tools (known as the Comprehensive Unit-based Safety Program (CUSP)), and a five-item, evidence-based checklist for correctly inserting central venous catheters. The five components of the checklist included using basic hand hygiene, exercising full barrier precautions, cleaning the skin with chlorhexidine, avoiding the femoral site when possible, and removing any unnecessary catheters. A model for translating evidence into practice identified and mitigated local barriers to implementation of the checklist.²² Quarterly infection rates measured at the ICU level were used to monitor progress towards the goal of reducing CLABSIs.

Model inputs

Table 1 shows the key parameters used in the decision tree model, such as probabilities and cost inputs as well as the effectiveness of the programme, which are described in detail in the following sections.

Table 1.

Parameters used in the decision tree model

Parameter	Distribution	Source
Probability of CLABSI*	β: 0.052 (SD 0.0074)	18 25
Death attributable to CLABSI	β: 0.15 (SD 0.056)	6
Incidence rate ratio of programme versus non-programme ICUs	Lognormal based on normal mean 0.19 (SD 0.13)	13
Total cost$
CLABSI (per patient)†	Lognormal based on normal mean $18 793 (SD 5533)	33
Programme (per patient)†	Lognormal based on normal mean $540 (SD 120)	29

*Conditional probability of a CLABSI given exposure to a central venous catheter, assumes standard (non-antimicrobial) catheter.

†Discounted costs presented.

CLABSI, central line-associated bloodstream infection; ICU, intensive care unit.

Risk of CLABSI and death

Estimates of the risk of CLABSI given exposure to a central venous catheter varied from 3.0% to as much as 16.0%.^{23 24} We used a probability estimate of 5.2% for a standard catheter, derived from a meta analysis of 13 randomised controlled trials from a previous economic evaluation.^{18 25} Estimates of the attributable mortality of CLABSI ranged from 14% to 40%.^{5 6 26 27} We used a point estimate of 15%.

Costs

Estimates of payer costs attributable to CLABSI varied widely, from as little as $6000 to over $56 000.^{4 6 18 28} The reasons for this variation can be attributed to the small sample sizes of studies, challenges allocating inpatient costs, perspectives used, the types of cost categories included, and the methodology used for estimation.²⁹ Existing studies have largely focused on financial costs (or hospital charges or payer costs) rather than opportunity costs and so they may incorrectly estimate the economic cost of CLABSIs to hospitals. We considered the economic cost of CLABSIs in terms of the increased length of stay and the variable costs associated with that occurrence.³⁰ Given that a significant amount of hospital costs are fixed in the short run, the economic viability of quality improvement programmes that reduce CLABSIs rests on two things: deploying the bed-days freed by shorter lengths of stay for new admissions and reducing utilisation of medications and supplies. The value of the new admissions (the potential incremental net revenue opportunity per prevented infection) represents the economic cost of infection and accordingly, the potential economic cost avoidance resulting from infection prevention.³¹ Using this approach, we estimated that the discounted cost of a CLABSI was $18 793 (see online appendix tables A and B for details of this calculation).

Start-up costs and recurring costs associated with the quality improvement programme were assigned using an activity-based microcosting of the programme performed in six hospitals in Michigan.²⁹ We assumed that start-up costs occurred in the first year of implementation and did not discount them. Capital items, such as bloodstream infection line carts, were annualised assuming a 5-year useful life and 3% discount rate. We included the opportunity costs of key personnel whose time was committed to the programme even though a hospital may not incur any financial costs related to personnel who are already on staff. We estimated this cost by multiplying each staff person's percentage effort committed to the programme by an estimate of that position's annual compensation. We used the Society of Critical Care Medicine's annual compensation estimates from 2009 for critical care physicians, nurses, pharmacists and respiratory therapists.³² The salary for infection control preventionists was based on the Bureau of Labour Statistics's 2011 Occupational Employment Statistics and we added 30% benefits. Table 2 presents a detailed itemisation of the start-up costs and recurring costs of the programme. Start-up costs were approximately $80 000 and recurring costs were approximately $192 000 per hospital per year. Personnel costs were the largest contributor, comprising 95% of start-up costs and 89% of recurring costs.

Table 2.

Itemisation of programme costs (per hospital)

Cost category	Start-up costs	Recurring costs
Personnel$
Critical care physicians (2 on average per hospital)	$26 004	$71 953
Nurses (8 on average per hospital)	$44 406	$75 306
Respiratory therapists	$4605	$7923
Infection control preventionists	$1981	$7855
Pharmacists	$2725	$7962
Education and Training$
Education and training expenses	$3579
Capital items$
CLABSI line cart/central line insertion cart (annual equivalent cost)	$426	$426
Materials$
Chlorhexidine		$2378
Oral care kits		$6933
Sterile central line dressing kits		$11 555
Total$	$83 725	$192 292

Recurring costs occur each year that the intervention is in place; as such, this total represents the annual recurring cost (not discounted as presented here).

CLABSI, central line-associated bloodstream infection.

We estimated a per patient cost of the programme by deriving an average number of ICU patients per hospital who had central venous catheters (the patients most likely to benefit from the programme's prevention efforts). To calculate this, we first derived a national annual cohort of ICU patients exposed to central venous catheters by multiplying the total annual ICU admissions in the USA by an estimate of the proportion of patients admitted to an ICU that receive central venous catheters.³³ We then divided this group of ICU admissions with central venous catheters by the number of hospitals that reported having adult ICUs in the American Hospital Association Annual Survey of Hospitals,³⁴ yielding an average number of ‘exposed’ patients per hospital. Finally, we divided the total costs of the programme per hospital by the number of patients per hospital to yield an average cost for the programme per patient of $540 (SD, 120). Online appendix table C contains additional details of this calculation, including the uncertainty ranges incorporated into the estimate.

All costs were adjusted to 2013 US dollars using the consumer price index for medical care services. Recurring costs were discounted by 3% annually. In separate sensitivity analyses, we examined the effect of not discounting costs and of discounting costs by 5%.

Effectiveness

We based our estimate of the programme's effectiveness on a cluster randomised controlled trial that found an 81% reduction in CLABSI rates comparing programme ICUs with control ICUs (incidence rate ratio, 0.19; 95% CI 0.06 to 0.57).¹³ We used the CI of this point estimate, a measure of uncertainty in the programme's effectiveness, to derive a SD of the estimate for probabilistic sensitivity analysis. This effectiveness parameter was measured at the ICU level but in our model we assumed that it applied to individual patients. This ecological assumption was necessary because there are no patient level effectiveness estimates available from the Keystone ICU project. The SD of the estimate captures the heterogenic reality that the intervention benefit is not uniform among ICU patients.

To calculate the base case probability of a CLABSI in the programme arm of the decision tree, we multiplied the probability of CLABSI by the incidence rate ratio (0.19) from the trial. We assumed that by the end of the first year of implementation, programme ICUs achieve this reduction. The original ICUs that implemented the programme maintained the reduction for the subsequent 10 years (Sam Watson, written communication, 18 August 2013). We assumed this to be true for the 5-year period used in this evaluation.³⁵

The number of annual ICU admissions requiring catheters, the probability of infection and the attributable mortality parameters were assumed not to change during the 5-year period.

Outcome measures

The two outcomes of interest were CLABSIs prevented and deaths averted (lives saved). We calculated the number of CLABSIs prevented as the difference in the number of CLABSIs between programme and non-programme ICUs. Deaths averted were calculated similarly. We did not discount CLABSIs or deaths in the base case analysis to avoid making the ethically challenging assumption that infections or deaths prevented in the future are worth less than they are in the present. We explored the effect of discounting CLABSIs and deaths by 3% in a sensitivity analysis.

We calculated incremental cost-effectiveness ratios as the additional cost divided by the additional health benefit (CLABSIs prevented or deaths averted) comparing the programme to current practice.

Probabilistic sensitivity analyses

We conducted a probabilistic sensitivity analysis to account for uncertainty in the model's input parameters. We used Monte Carlo simulation to perform 10 000 iterations of the model, simultaneously sampling each parameter from an underlying distribution that reflects the degree of uncertainty in the parameter estimate. Uncertainty in parameter estimates was obtained from the CIs or SDs published with the point estimates. Table 1 presents the modelled distributions, point estimates and SDs for the key model parameters. All analyses were performed using Microsoft Excel (Microsoft Corporation; Redmond, Washington, USA) with @Risk (Palisade Corporation; Ithaca, New York, USA).

Results

Table 3 compares programme and non-programme ICU costs and outcomes for CLABSIs prevented and deaths averted. The quality improvement programme prevents 42 CLABSIs per 1000 patients and averts 6 deaths per 1000 patients at no additional cost in the base case analysis, representing a dominant strategy when compared with current practice.

Table 3.

Comparison of costs and outcomes between programme and non-programme ICUs

	Mean	Median	2.5th–97.5th centile
Non-programme ICU
CLABSIs	52	52	39–66
Deaths	8	8	2–14
Costs*	$987 000	$937 000	$488 000–$1 760 000
Programme ICU
CLABSIs	10	9	3–29
Deaths	2	1	0–5
Costs$*	$738 000	$710 000	$453 000–$1 190 000
Benefit of programme†
CLABSIs prevented	42	42	23–58
Deaths averted	6	6	2–12
Net costs$	−$249 000	−$221 000	−$976 000 to $300 000
Incremental cost-effectiveness ratio (probability)
Cost per CLABSI prevented	Strongly dominant (0.80)‡
Cost per death prevented	Strongly dominant (0.80)‡

Mean, median, 2.5% and 97.5% centile estimates for outputs from probabilistic sensitivity analysis of 10 000 model runs representing uncertainty in epidemiological and economic parameters are reported.

All mean, median, and centile values are expressed per 1000 patients to make the scale easier to interpret. Values have been rounded to three significant digits at most.

*Costs are not presented separately for each outcome (CLABSI and death) because no additional cost was assumed to occur for death; discounted at 3%.

†Benefit of programme determined by subtracting programme ICU estimates from non-programme ICU estimates within the model.

‡Probability that the programme is more effective and less costly than current practice.

CLABSI, central line-associated bloodstream infection; ICU, intensive care unit.

Figure 2 and online supplementary figure S1 show the cost-effectiveness planes comparing the joint density of incremental costs and incremental effects for 10 000 model iterations. Incremental refers to the difference in costs or effects between programme and non-programme ICUs. The X-axis represents the incremental level of effectiveness of an outcome and the Y-axis represents the additional total cost of achieving this outcome. Each data point in the scatterplot represents an estimated incremental cost-effectiveness ratio for the outcome; as such, the scatterplot illustrates the distribution of incremental cost-effectiveness ratios over a sample population. Points falling to the right of the Y-axis demonstrate that the programme is effective for preventing CLABSIs and averting deaths. Points falling above the X-axis represent the additional costs of the programme and points falling below the X-axis represent the economic cost savings from the programme. In figure 2, because 80% of the points fall below the X-axis, there is an 80% probability that the programme reduces bloodstream infections and the infections’ economic costs to hospitals compared with current practice. Figure 2 also demonstrates the presence of few extreme values (outlier incremental cost-effectiveness ratios) in the model, indicating little uncertainty in the estimates of the quality improvement programme's cost-effectiveness compared with current practice. Outliers add variability and uncertainty to the overall cost-effectiveness results and the existence of few such outliers in the cost-effectiveness plane supports the robustness of the programme as a dominant strategy.

Figure 2.

Cost-effectiveness plane for central line-associated bloodstream infections prevented with 95% confidence ellipses. Values on both axes have been multiplied by 1000 to yield incremental costs and effects expressed per 1000 patients to aid interpretation. Incremental refers to the difference in costs or effects between programme and non-programme intensive care units. Cost values in US dollars. Boxes in the plot region display the percentage of the distribution of incremental cost-effectiveness ratios falling above or below $0. The 95% confidence ellipses overlaid on the figure are calculated assuming a bivariate normal distribution and display the uncertainty in the incremental costs-effectiveness ratios.

Online supplementary figures S2 and S3 display tornado diagrams of the results of the probabilistic sensitivity analysis. The opportunity cost of CLABSIs exerted the largest influence on the cost-effectiveness of the programme for preventing CLABSIs and deaths. As such, hospitals implementing the programme should know the opportunity costs they face due to CLABSIs. The opportunity cost is calculated as the potential incremental net revenue opportunity per prevented CLABSI. We provide an example of how to perform this calculation in the online appendix. Discounting CLABSIs or deaths at 3% in addition to costs, does not change the interpretation of our results. Similarly, discounting costs at 0% or 5% does not change the interpretation of our findings (see online appendix table D).

Discussion

This study represents the first formal cost-effectiveness analysis of a nationally implemented quality improvement programme (the Keystone ICU project) in the USA to decrease CLABSIs in critically ill patients. One of the few large scale quality improvement projects to demonstrate long-term sustainability, this programme has the potential to reduce CLABSIs and deaths at no additional cost to US hospitals.

Comparison with other studies

Previous studies highlight the importance and difficulty of developing a business case for quality improvement. A business case exists if healthcare organisations investing in an intervention reap a return on their investment.³⁶ Many prevention initiatives have suffered from a lack of evidence supporting a positive return on investment for hospitals and payers.³⁷ Incentive misalignment results in hospitals incurring costs to implement quality improvement programmes while payers accrue the subsequent cost savings. A financial analysis of a replication of the Keystone ICU project in one tertiary hospital in Hawaii demonstrated that reducing CLABSIs actually resulted in lower profit margins, thus creating a perverse incentive to provide a lower quality of care.³⁸ A different study calculated that, for every CLABSI prevented, the programme costs $5404,²⁹ and suggested that it is cost saving when compared with the cost of an infection (which ranges from $6000 to over $56 000^{4 6 33 39 40}). However, both studies used inaccurate cost analyses that focused on the financial rather than the economic costs of CLABSIs. These studies also did not account for uncertainty in the cost or effect estimates.

Our evaluation offers several improvements to these existing studies. First, whenever possible we used nationally representative data to determine provider salary and compensation costs, so as to increase the ability to generalise our findings. Second, we performed a probabilistic sensitivity analysis to quantify the uncertainty in our cost-effectiveness estimates. Third, we considered the opportunity costs of CLABSIs rather than financial costs. Our estimate of the cost of a CLABSI is based on the foregone hospital revenue that results whenever an infection occurs. From the perspective of the hospital, reducing the cost of CLABSIs is tantamount to reducing this foregone revenue by redeploying ICU beds for new admissions. Finally, we extended the evaluation to consider deaths prevented as an additional outcome because of its interest to clinicians and patients.

Limitations of the study

This evaluation has several limitations. First, the impact of patient heterogeneity, such as demographics and clinical characteristics on baseline risk, treatment effect or resource utilisation was not fully explored.⁴¹ This evaluation sought instead to represent an average ICU experience, in part because the data needed to explore subgroups are not available, but more importantly because the intervention applies to patients irrespective of these differences. It would be impractical—and possibly unethical—to only use this programme in subgroups of patients for whom greater benefit is expected. The programme is based on evidence-based practices for inserting central venous catheters—practices that should apply equally to all patients. Second, we did not evaluate costs outside the acute hospital setting, such as rehabilitation costs or productivity losses for delays returning to work incurred by patients suffering CLABSIs. However, doing so would further support the cost-effectiveness of the programme. We chose to examine costs from the perspective of the hospital because hospitals bear the greatest burden of nosocomial infection costs in the prospective payer system and demonstrating a business case is important for the dissemination of effective quality improvement programmes. Finally, this evaluation did not explore the use of antimicrobial catheters as an comparator because systematic reviews have come to differing conclusions about the extent of their effectiveness in preventing CLABSIs, and many of the trials have been small and of a low quality.^{42 43} In addition, the choice facing hospital decision makers is not necessarily a mutually exclusive choice between the Keystone ICU project or the use of antimicrobial catheters. The results of Keystone ICU project already reflect the use of various types of catheters because the programme itself did not specify catheter type. The parameter estimate for effectiveness of the programme used in this evaluation was derived from a cluster randomised controlled trial. In this setting it can be expected that the utilisation of antimicrobial catheters should be balanced between the intervention and control arms of the trial (though this was not explicitly measured), and the effectiveness estimate is attributable to the quality improvement programme itself. The Keystone ICU project quality improvement programme is also a complex social intervention. Comparing it directly to technology or a device understates its broader effects, which may include reductions in sepsis and ventilator-associated pneumonia or reduced staff turnover resulting from an improved safety culture.²¹ Data for these effects are limited in comparison to the data available for CLABSIs, but recent evidence suggests that the Keystone ICU project significantly reduced rates of ventilator -associated pneumonia in Michigan ICUs.⁹ Inclusion of these additional beneficial effects for the same set of costs would further support the cost-effectiveness of the programme.

Conclusions and implications of study findings

These findings have important implications for healthcare. Broad implementation of the Keystone ICU project in the US healthcare system could substantially reduce the morbidity and mortality associated with CLABSIs and their economic costs to hospitals. Although the Centers for Disease Control and Prevention has demonstrated that significant reductions in CLABSIs are indeed being realised for intensive care patients in the USA,⁴⁴ 800 medium and large hospitals continue to have high CLABSIs rates.⁴⁵ Further dissemination of cost-effective quality improvement programmes is needed. Although our analysis adopted a hospital perspective, payers also stand to benefit from the programme and can support dissemination efforts. Hospitals and payers should partner to reform the incentive structure facing hospitals in order to better support patient safety and quality. Payer support, such as covering or funding some intervention costs and imposing financial penalties on hospitals when patients develop CLABSIs, could encourage uptake and dissemination of the programme. Future evaluations of this quality improvement programme in non-US settings can incorporate country-specific costs or extend the evaluation to consider additional outcomes, such as cases of ventilator-associated pneumonia prevented. As more data and evidence emerges regarding the long-term costs and outcomes for patients with CLABSI and other healthcare associated infections,⁴⁶ future economic evaluations can incorporate this information to gauge the cost-effectiveness of prevention and quality improvement efforts.

To conclude, this multifaceted quality improvement programme, currently being implemented by thousands of hospitals in the USA, likely reduces unnecessary morbidity, mortality and economic costs associated with CLABSIs.