Abstract
Objectives To evaluate the extent to which intensive care units participating in the initial Keystone ICU project sustained reductions in rates of catheter related bloodstream infections.
Design Collaborative cohort study to implement and evaluate interventions to improve patients’ safety.
Setting Intensive care units predominantly in Michigan, USA.
Intervention Conceptual model aimed at improving clinicians’ use of five evidence based recommendations to reduce rates of catheter related bloodstream infections rates, with measurement and feedback of infection rates. During the sustainability period, intensive care unit teams were instructed to integrate this intervention into staff orientation, collect monthly data from hospital infection control staff, and report infection rates to appropriate stakeholders.
Main outcome measures Quarterly rate of catheter related bloodstream infections per 1000 catheter days during the sustainability period (19-36 months after implementation of the intervention).
Results Ninety (87%) of the original 103 intensive care units participated, reporting 1532 intensive care unit months of data and 300 310 catheter days during the sustainability period. The mean and median rates of catheter related bloodstream infection decreased from 7.7 and 2.7 (interquartile range 0.6-4.8) at baseline to 1.3 and 0 (0-2.4) at 16-18 months and to 1.1 and 0 (0.0-1.2) at 34-36 months post-implementation. Multilevel regression analysis showed that incidence rate ratios decreased from 0.68 (95% confidence interval 0.53 to 0.88) at 0-3 months to 0.38 (0.26 to 0.56) at 16-18 months and 0.34 (0.24-0.48) at 34-36 months post-implementation. During the sustainability period, the mean bloodstream infection rate did not significantly change from the initial 18 month post-implementation period (−1%, 95% confidence interval −9% to 7%).
Conclusions The reduced rates of catheter related bloodstream infection achieved in the initial 18 month post-implementation period were sustained for an additional 18 months as participating intensive care units integrated the intervention into practice. Broad use of this intervention with achievement of similar results could substantially reduce the morbidity and costs associated with catheter related bloodstream infections.
Introduction
Catheter related bloodstream infections cause considerable morbidity, mortality, and healthcare costs.1 2 An estimated 82 000 catheter related bloodstream infections and up to 28 000 attributable deaths occur in intensive care units annually,3 and each infection costs about $45 000 (£28 000; €31 000).4 In an ongoing quality improvement project, known as the Michigan Health & Hospital Association (MHA) Keystone ICU project, these infections were substantially reduced in 103 participating intensive care units.5 The median infection rate per 1000 catheter days dropped from 2.7 at baseline to 0 within three months after implementation of an evidence based intervention. Eighteen months after implementation, infection rates had decreased by 66% from baseline. However, whether these initial results were sustained was not known.
Limited evidence assessing the sustainability of quality improvement projects beyond the initial implementation and evaluation period is available.6 7 To evaluate sustainability, a quality improvement project must have an adequate infrastructure to sustain activities beyond its initial phase. After the 18 month post-implementation evaluation period, most hospitals participating in the Keystone ICU project continued to submit data on infection rates. The objective of this study was to evaluate the extent to which intensive care units participating in the initial Keystone ICU project sustained reductions in rates of catheter related bloodstream infection. We hypothesised that the bloodstream infection rate would remain low during the sustainability period relative to baseline.
Methods
Study design
The Keystone ICU project was designed as a prospective cohort study and used a collaborative model involving the Johns Hopkins Quality and Safety Research Group, the MHA Keystone Center for Patient Safety and Quality, and participating hospitals. All hospitals in Michigan with an intensive care unit were invited to participate in the project. To participate, they had to form an intensive care unit team, agree to complete the project work, and send a signed letter of commitment from a senior hospital executive with a list of team members. At a minimum, the team had to include a senior executive, the intensive care unit director and nurse manager, and a physician and nurse in the intensive care units who could both dedicate 20% of their effort to project activities. The initial implementation and evaluation phase was funded by the Agency for Healthcare Research and Quality. During this phase, individual intensive care units implemented two cultural interventions first and then two interventions targeting patients’ safety (in any order).5 The four interventions were implemented at three month intervals. As a result, the intervention targeting catheter related bloodstream infection had a staggered timing for its implementation.
At the end of the initial funding, intensive care unit teams were instructed to sustain the bloodstream infection intervention. They were to maintain a team, integrate the intervention into staff orientation, continue collecting monthly data on catheter related bloodstream infections and catheter days in collaboration with hospital infection control staff, and continue reporting infection rates per 1000 catheter days to appropriate stakeholders. We lacked funding to formally evaluate any intensive care units’ compliance with each of these components. Informally, however, nearly all units reported that they were continuing all of these components. After the initial funding ended, intensive care units paid MHA to continue working with the Johns Hopkins team on new quality improvement interventions not related to catheter related bloodstream infections. Thirteen intensive care units declined further participation specifically because of the MHA fee. Compared with all intensive care units that continued to participate, the 13 units that dropped out of this project were more likely to be at a teaching hospital (93% v 65 %, P=0.04), but they did not vary in the median number of hospital beds (383 v 338, P=0.70) and did not have significantly different rates of catheter related bloodstream infection in the first and final quarters of the initial 18 month post-implementation period or quarterly rates of improvement over that period.
Intervention
We used a conceptual model to increase use of evidence based interventions and improve safety culture. This model has been described elsewhere.8 9 10 The intervention evaluated in this study targeted clinicians’ use of five evidence based recommendations to reduce catheter related bloodstream infections from the Centers for Disease Control and Prevention (CDC).2 5 Four recommendations related to insertion of the catheter: hand washing, using full barrier precautions, cleaning the skin with chlorhexidine, and avoiding the femoral site when possible. The fifth recommendation was to remove unnecessary catheters. Strategies to increase the use of these evidence based recommendations are described elsewhere.5 11 Teams reported all catheter related bloodstream infections and number of days of use of catheters (catheter days). For the remainder of this paper we will use the term bloodstream infection to indicate catheter related bloodstream infection.
Outcomes
The primary outcome was the quarterly rate of bloodstream infections during the 18 month sustainability period. To test sustainability of the intervention, we analysed trends in infection rates for the 18 month initial evaluation period compared with the 18 month sustainability period. Specifically, a total of 103 intensive care units from 67 hospitals contributed data to the initial evaluation period. Some intensive care units did not contribute to all time periods, and 13 of the 103 units declined to participate in the sustainability period, leaving 90 for which we evaluated the primary outcome. All hospitals used standard definitions of catheter related bloodstream infections provided by the CDC.12 Our analysis included hospital teaching status (binary variable) and bed size (continuous variable) to show any effect on distribution of catheter days. These variables came from the American Hospital Association database.
Statistical analysis
We summarised rates of bloodstream infection as medians and interquartile ranges and as means and standard deviations. We used generalised linear latent and mixed models,13 14 with a Poisson distribution, to compare the quarterly bloodstream infection rate from baseline to the end of the initial 16-18 month evaluation period and for the subsequent 18 month sustainability period (from 19-21 months to 33-36 months post-implementation). In the models, we used robust estimation of variance, adjusted for hospital teaching status and bed number, and included two level random effects to account for nested clustering within the data13 15: intensive care units over time and hospitals within regions. We also evaluated whether the relative change in quarterly bloodstream infection rate was different during the initial evaluation period compared with the sustainability period by using a linear spline model, which allowed the relative change in quarterly infection rate to change at the end of the initial 18 month period.16
To assess the sensitivity of our results to missing data and to potential non-reporting bias, we replicated the analysis including only those 43 intensive care units that reported continuous data from baseline to the end of the sustainability period. To assess the sensitivity of our results to a few units that reported quarters with large rates of bloodstream infections (primarily driven by small numbers of catheter days), we replicated the analysis excluding the quarters of data in which catheter days were in the lowest 10% across all quarters. Finally, we replicated our models assuming an over-dispersed Poisson distribution (known as a negative binomial distribution) for the number of bloodstream infections to assess the sensitivity of our results to the Poisson assumption. All P values are two sided; we considered a P value of ≤0.05 to be statistically significant. We used Stata (version 9.1) for all analyses.
Results
During the sustainability period, 90 (87%) of the original 103 intensive care units from 61 (91%) of the original 67 hospitals reported 1532 intensive care unit months of data and 300 310 catheter days. Thirteen units dropped out because they chose not to pay the MHA’s fee. The distribution of total catheter days by hospital teaching status and bed number was consistent between the initial evaluation period and the sustainability period (table 1).
Table 1.
Hospital type/size | Baseline period before implementation of intervention | During implementation of intervention (3 months) | Initial evaluation period (18 months) | Sustainability period (18 months) | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
No of ICUs* | Catheter days (%) | No of ICUs | Catheter days (%) | No of ICUs | Catheter days (%) | No of ICUs | Catheter days (%) | ||||
All hospitals | 55 | 41 506 (100) | 96 | 57 033 (100) | 103 | 300 175 (100) | 90 | 300 310 (100) | |||
Teaching status: | |||||||||||
Teaching | 33 | 29 407 (71) | 66 | 44 752 (78) | 71 | 231 595 (77) | 60 | 230 836 (77) | |||
Non-teaching | 22 | 12 099 (29) | 30 | 12 281 (22) | 32 | 68 580 (23) | 30 | 69 474 (23) | |||
No of beds: | |||||||||||
<200 | 13 | 3 266 (8) | 17 | 3 066 (5) | 19 | 16 349 (5) | 17 | 16 398 (5) | |||
200-299 | 12 | 13 321 (32) | 19 | 10 792 (19) | 23 | 67 970 (23) | 22 | 59 122 (20) | |||
300-399 | 12 | 10 531 (25) | 20 | 12 314 (22) | 20 | 66 765 (22) | 17 | 63 650 (21) | |||
≥400 | 18 | 14 388 (35) | 40 | 30 861 (54) | 41 | 149 091 (50) | 34 | 161 140 (54) |
*Of 103 ICUs participating in initial evaluation, 48 did not contribute baseline data—40 implemented intervention during baseline period of study; 8 did not report baseline data.
Overall, 1394 possible quarters of bloodstream infection data existed for the entire study (baseline data for 55 intensive care units, plus 13 subsequent quarters for the three month implementation period and the 36 month post-implementation period for 103 units) (table 2). Seventy-eight quarters (roughly 5%) of data were lost when the 13 units left the collaboration, and 65 quarters (roughly 5%) of bloodstream infection rates were missing for the remaining units.
Table 2.
Study period | No of ICUs | Median (IQR) No of infections | Median (IQR) catheter days | Infection rate | Incidence rate ratio* (95% CI) | |
---|---|---|---|---|---|---|
Median (IQR) | Mean (SD) | |||||
Baseline | 55 | 2 (1-3) | 551 (220-1091) | 2.7 (0.6-4.8) | 7.7 (28.9) | Reference |
During implementation | 96 | 1 (0-2) | 447 (237-710) | 1.6 (0-4.4) | 2.8 (4.0) | 0.81 (0.61 to 1.08) |
After implementation—initial evaluation period: | ||||||
0-3 months | 95 | 0 (0-2) | 436 (246-771) | 0 (0-3.0) | 2.3 (4.0) | 0.68 (0.53 to 0.88) |
4-6 months | 95 | 0 (0-1) | 460 (228-743) | 0 (0-2.7) | 1.8 (3.2) | 0.62 (0.42 to 0.90) |
7-9 months | 96 | 0 (0-1) | 467 (252-725) | 0 (0-2.0) | 1.4 (2.8) | 0.52 (0.38 to 0.71) |
10-12 months | 95 | 0 (0-1) | 431 (249-743) | 0 (0-2.1) | 1.2 (1.9) | 0.48 (0.33 to 0.70) |
13-15 months | 95 | 0 (0-1) | 404 (158-695) | 0 (0-1.9) | 1.5 (4.0) | 0.48 (0.31 to 0.76) |
16-18 months | 95 | 0 (0-1) | 367 (177-682) | 0 (0-2.4) | 1.3 (2.4) | 0.38 (0.26 to 0.56) |
After implementation—sustainability period: | ||||||
19-21 months | 89 | 0 (0-1) | 399 (230-680) | 0 (0-1.4) | 1.8 (5.2) | 0.34 (0.23 to 0.50) |
22-24 months | 89 | 0 (0-1) | 450 (254-817) | 0 (0-1.6) | 1.4 (3.5) | 0.33 (0.23 to 0.48) |
25-27 months | 88 | 0 (0-1) | 481 (266-769) | 0 (0-2.1) | 1.6 (3.9) | 0.44 (0.34 to 0.57) |
28-30 months | 90 | 0 (0-1) | 479 (253-846) | 0 (0-1.6) | 1.3 (3.7) | 0.40 (0.30 to 0.53) |
31-33 months | 88 | 0 (0-1) | 495 (265-779) | 0 (0-1.1) | 0.9 (1.9) | 0.31 (0.21 to 0.45) |
34-36 months | 85 | 0 (0-1) | 456 (235-787) | 0 (0-1.2) | 1.1 (2.7) | 0.34 (0.24 to 0.48) |
ICU=intensive care units; IQR=interquartile range.
*Calculated with use of generalised linear latent and mixed model,13 with robust variance estimation and random effects to account for clustering of catheter related bloodstream infections within ICUs over time and clustering of hospitals within geographical regions; rates of catheter related bloodstream infections during implementation, initial evaluation, and sustainability periods compared with baseline (pre-implementation) values, adjusted for hospital’s teaching status and number of beds.
Total catheter days were consistent during each quarter in the initial evaluation period and the sustainability period. The mean and median rates of bloodstream infection decreased from 7.7 and 2.7 (interquartile range 0.6-4.8) per 1000 catheter days at baseline to 2.3 and 0 (0.0-3.0) at 0-3 months after implementation of the intervention and to 1.3 and 0 (0-2.4) at 16-18 months after implementation, and they were sustained at 1.1 and 0 (0.0-1.2) at 34-36 months post-implementation (table 2). The multilevel Poisson regression model showed a significant decrease in bloodstream infection rates during all study periods compared with baseline rates; incidence rate ratios decreased from 0.68 (95% confidence interval 0.53 to 0.88) at 0-3 months to 0.38 (0.26 to 0.56) at 16-18 months after implementation, with a sustained improvement of 0.34 (0.24 to 0.48) at 34-36 months (table 2).
From baseline to the end of the initial 18 month evaluation period, the mean rate of bloodstream infection decreased significantly by 12% (95% confidence interval 9% to 15%) per quarter (figure). During the 19-36 month sustainability period, the quarterly bloodstream infection rate did not significantly change from the rate achieved at the end of the initial evaluation period (1% decrease, 95% confidence interval 9% decrease to 7% increase).
Forty-three intensive care units reported data continuously from baseline to the end of the sustainability period. A sensitivity analysis including only these 43 units resulted in similar findings to our main analysis including all units: an estimated 13% (9% to 16%) decrease in the mean rate of bloodstream infection per quarter during the initial evaluation period followed by a less than 1% decrease (4% decrease to 5% increase) per quarter during the sustainability period. The primary results were not substantially changed by exclusion of infection data for quarters in which the total catheter days were in the lowest 10% (<67 days). Finally, the results changed little when we used regression models assuming an over-dispersed Poisson distribution for the number of infections.
Discussion
Our findings show that the markedly reduced rates of bloodstream infection achieved in the initial evaluation period of the Keystone ICU project were sustained for an additional 18 months. The median rate of infection remained at zero for the entire 18 month sustainability period, with a greater than 60% reduction in infection rates from baseline sustained at the end of the 36 month period.
Our project is one of relatively few robustly designed and evaluated large scale quality improvement projects that have shown substantial improvements. Even fewer projects have evaluated the sustainability of such results. Achieving sustainability has been a major challenge for the quality improvement field.17 18 19 The Keystone ICU project coupled a formal model to translate evidence into practice with a comprehensive patient safety intervention to improve culture, educate staff, learn from mistakes, and involve senior leaders.8 10 During interviews, intensive care unit teams noted several factors that were important in sustaining this project, including continued feedback of infection data that the team perceived as valid, improvements in safety culture that occurred as part of the overall Keystone ICU project, an unremitting belief in the preventability of bloodstream infections, involvement of senior leaders who reviewed infection data and provided teams with the resources needed, and a shared goal rather than a competition to reduce infection rates throughout the state of Michigan. Other studies describe similar methods to facilitate sustainability, such as active support from leaders and an infrastructure that supports quality improvement methods, training of staff, involving key internal and external stakeholders,6 20 21 22 23 alignment of project and organisational goals,21 multidisciplinary teams and collaborations,6 and inclusion of change management that also encourages local adaptation and rewards innovation and change.21 Quality improvement initiatives that demonstrate value are more likely to be sustained.21 24 We did not formally evaluate the factors associated with continued improvement. However, the quality improvement model used in the Keystone ICU project and our recommendations to teams at the end of the initial evaluation period to maintain a team, orient new staff, collect monthly data, and report infection rates to appropriate stakeholders target many of the recommendations described in the literature.
Limitations of study
This research has several new limitations not reported in our evaluation of the initial evaluation period.5 Firstly, we cannot evaluate whether the results would have been sustainable if intensive care units were blinded to their infection rates, because our model to change practice included feedback of data so units could evaluate their performance.8 Secondly, we had insufficient funds to formally evaluate mechanisms that contributed to the success of this sustainability, so we cannot evaluate the relative importance during the sustainability phase of periodic meetings between participating units and the Keystone Center. Although these meetings focused on implementing new safety interventions, they may have influenced the sustainability of the bloodstream infection intervention. Also unknown was the impact of Blue Cross Blue Shield of Michigan’s quality payment incentive on a hospital’s decision to continue to participate in the Keystone ICU project. In the first year, hospitals received an incentive payment if they submitted 90% of the required bloodstream infection data. In subsequent years, the incentive payment was based on performance thresholds (statewide infection rate compared against CDC pooled mean) for continued reduction of infection rates. Thirdly, we did not evaluate the use of new technologies, such as impregnated dressings or catheters and chlorhexidine baths, on rates of infection.25 26 27 28
Implications and future research
These findings have important public health consequences. If the multifaceted quality improvement intervention and collaborative model were implemented in all intensive care units across the United States, and the results were similar to those achieved in Michigan, substantial and persistent reductions could be made in the 82 000 infections, 28 000 deaths, and $2.3 billion costs attributed to these infections annually.3 Moreover, the use of this quality improvement model to reduce other complications may further improve quality and reduce costs of care.
A prominent item on the research agenda for quality improvement initiatives is identifying methods that sustain a successful project. Sustainability is often vague and may not be formally separated from the initial implementation and evaluation of the project.22 Although we do not have empirical data to support this statement, both our experience and theoretical reasons support the idea that sustainability should be examined separately from implementation. Weick and Quinn noted that organisations and teams vary in whether they are best suited for episodic or continuous change.29 Moreover, early, mid, and late implementers have differing characteristics and motivations for implementing quality improvement interventions.30 Some implementers focus efforts on short term results, whereas sustainers generally focus on long term results. Teams that are good “implementers” may not be the same as those that are good “sustainers.” More research is needed to better understand when implementation ends and sustaining begins, as well as the attributes of teams that are good implementers and good sustainers.
What is already known on this topic
Bloodstream infections from central venous catheters are preventable to some extent
Sustainability of quality improvement initiatives is a major challenge
What this study adds
A multifaceted quality improvement project can sustain reductions in bloodstream infection rates to 36 months post-implementation
We thank Christine G Holzmueller for her assistance in editing the manuscript. We also thank the Michigan Health & Hospital Association (MHA) Keystone Center and all the intensive care units teams in Michigan (list of participating hospitals in web appendix) for their tremendous efforts. Their leadership and courage in this innovative effort reflects an unrelenting passion and dedication to improve quality and safety for their patients.
Contributors: All the authors were involved in preparing this manuscript and had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. PJP was the principal investigator of the project, was responsible for the overall design and supervision of the study, and wrote the initial draft of the manuscript. All other authors contributed to study design, collecting or analysing data, and redrafting the manuscript. CG was the director of the Keystone Center during the initial evaluation period, and SW was its director during the sustainability period. SW and SB managed collection and quality control of data, and PJP, EC, DJS, and DM provided data cleaning and editing, statistical analysis, and interpretation of results. SB, LL, DT, and JM were part of the Johns Hopkins team that supported the collaborative, and RH, RW, PP, and KS were members of and represented intensive care unit teams in Michigan. SC provided expertise on definitions of catheter related bloodstream infection, measuring infection rates, and interventions to reduce infections. PJP is the guarantor.
Funding: Support for this project, for the period from October 2003 to September 2005, was provided by the Agency for Healthcare Research and Quality (1UC1HS14246) and the MHA. The Agency for Healthcare Research and Quality provided financial support, and the MHA provided support for the biannual statewide meetings. The researchers were independent of the sponsors, and neither sponsor had any influence over the study design; collection, analysis, or interpretation of the data; writing of the manuscript; or decision to submit it for publication.
Competing interests: PJP and CAG received grant support from the Agency for Healthcare Research and Quality, the Robert Wood Johnson Foundation, the National Patient Safety Agency, and the World Health Organization to study and improve quality of care, including catheter related bloodstream infections. They have received lecture fees from various healthcare organisations, and CAG has also received lecture fees from government agencies to speak on quality and patient safety. SC has grant support from Cubist and Astellas, has served as a consultant for Merck, and has been on the advisory boards for Astellas, Forrest, and Cadence. JBS and JAM have grant support from the Robert Wood Johnson Foundation. PP has received lecture fees from Lilly, Merck, Edward Life Sciences, and Sage for various speaking engagements. DMN has had grant and contract support from the National Institutes of Health/National Heart Lung and Blood Institute and a clinician-scientists award from the Canadian Institutes of Health Research.
Ethical approval: The Johns Hopkins University School of Medicine Institutional Review Board reviewed and approved this research.
Data sharing: The dataset and statistical codes are available from the corresponding author.
Cite this as: BMJ 2010;340:c309
Web Extra. Extra material supplied by the author
References
- 1.Burke JP. Infection control—a problem for patient safety. N Engl J Med 2003;348:651-6. [DOI] [PubMed] [Google Scholar]
- 2.Mermel LA. Prevention of intravascular catheter-related infections. Ann Intern Med 2000;132:391-402. [DOI] [PubMed] [Google Scholar]
- 3.Klevens RM, Edwards JR, Richards CL Jr, Horan TC, Gaynes RP, Pollock DA, et al. Estimating health care-associated infections and deaths in US hospitals, 2002. Public Health Rep 2007;122:160-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.O’Grady NP, Alexander M, Dellinger EP, Gerberding JL, Heard SO, Maki DG, et al. Guidelines for the prevention of intravascular catheter-related infections. MMWR 2002;55:1-26. [DOI] [PubMed] [Google Scholar]
- 5.Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006;355:2725-32. [DOI] [PubMed] [Google Scholar]
- 6.Womer RB, Tracy E, Soo-Hoo W, Bickert B, DiTaranto S, Barnsteiner JH, et al. Multidisciplinary systems approach to chemotherapy safety: rebuilding processes and holding the gains. J Clin Oncol 2002;20:4705-12. [DOI] [PubMed] [Google Scholar]
- 7.Neily J, Howard K, Quigley P, Mills PD. One-year follow-up after a collaborative breakthrough series on reducing falls and fall-related injuries. Jt Comm J Qual Patient Saf 2005;31:275-85. [DOI] [PubMed] [Google Scholar]
- 8.Pronovost PJ, Berenholtz SM, Needham DM. Translating evidence into practice: a model for large scale knowledge translation. BMJ 2008;337:963-5. [DOI] [PubMed] [Google Scholar]
- 9.Pronovost PJ, Berenholtz SM, Goeschel C, Thom I, Watson SR, Holzmueller CG, et al. Improving patient safety in intensive care units in Michigan. J Crit Care 2008;23:207-21. [DOI] [PubMed] [Google Scholar]
- 10.Pronovost P, Weast B, Rosenstein B, Sexton B, Holzmueller C, Paine L, et al. Implementing and validating a comprehensive unit-based safety program. J Pat Safety 2005;1:33-40. [Google Scholar]
- 11.Berenholtz SM, Pronovost PJ, Lipsett PA, Hobson B, Earsing K, Farley JE, et al. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med 2004;32:2014-20. [DOI] [PubMed] [Google Scholar]
- 12.National Nosocomial Infections Surveillance System. National nosocomial infections surveillance (NNIS) system report, data summary from January 1992 through June 2004. Am J Infect Control 2004;32:470-85. [DOI] [PubMed] [Google Scholar]
- 13.Rabe-Hesketh S, Skrondal A. Multilevel and longitudinal modeling using Stata. Stata Press, 2005.
- 14.Skrondal A, Rabe-Hesketh S. Generalized latent variable modeling: multilevel, longitudinal, and structural equation models. Chapman & Hall/CRC, 2004.
- 15.Panageas KS, Schrag D, Riedel E, Bach PB, Begg CB. The effect of clustering of outcomes on the association of procedure volume and surgical outcomes. Ann Intern Med 2003;139:658-65. [DOI] [PubMed] [Google Scholar]
- 16.Harrell FE. Regression modeling strategies: with applications to linear models, logistic regression and survival analysis. Springer Science+Business Media, 2001.
- 17.Chin MH, Cook S, Drum ML, Jin L, Guillen M, Humikowski CA, et al. Improving diabetes care in Midwest community health centers with the health disparities collaborative. Diabetes Care 2004;27:2-8. [DOI] [PubMed] [Google Scholar]
- 18.Dietrich AJ, Oxman TE, Williams JW Jr, Kroenke K, Schulbeg HC, Bruce M, et al. Going to scale: re-engineering systems for primary care treatment of depression. Ann Fam Med 2004;2:301-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Berwick DM. Improving patient care: my right knee. Ann Intern Med 2005;142:121-5. [DOI] [PubMed] [Google Scholar]
- 20.Yates GR, Hochman RF, Sayles SM, Stockmeier CA. Sentara Norfolk General Hospital: accelerating improvement by focusing on building a culture of safety. Jt Comm J Qual Saf 2004;30:534-42. [DOI] [PubMed] [Google Scholar]
- 21.Gustafson DH, Sainfort F, Eichler M, Adams L, Bisognano M, Steudel H. Developing and testing a model to predict outcomes of organizational change. Health Serv Res 2003;38:751-76. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82:581-629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lyder CH, Grady J, Mathur D, Petrillo MK, Meehan TP. Preventing pressure ulcers in Connecticut hospitals by using the plan-do-study-act model of quality improvement. Jt Comm J Qual Saf 2004;30:205-14. [DOI] [PubMed] [Google Scholar]
- 24.Alexander JA, Weiner BJ, Metzger ME, Shortell S, Bazzoli G, Hasnain-Wynia R, et al. Sustainability of collaborative capacity in community health partnerships. Med Care Res Rev 2003;60:130-60. [DOI] [PubMed] [Google Scholar]
- 25.Hockenhull JC, Dwan KM, Smith GW, Gamble CL, Boland A, Whalley TJ, et al. The clinical effectiveness of central venous catheters treated with anti-infective agents in preventing catheter-related bloodstream infections: a systematic review. Crit Care Med 2009;37:702-12. [DOI] [PubMed] [Google Scholar]
- 26.Friedman C, Curchoe R, Foster M, Hirji Z, Krystofiak S, Lark RL, et al. APIC/CHICA-Canada infection prevention, control, and epidemiology: professional and practice standards. Am J Infect Control 2008;36:385-9. [DOI] [PubMed] [Google Scholar]
- 27.Climo MW, Sepkowitz KA, Zuccotti G, Fraser VJ, Warren DK, Perl TM, et al. The effect of daily bathing with chlorhexidine on the acquisition of methicillin-resistant Staphylococcus aureus, vancomycin-resistant Enterococcus, and healthcare-associated bloodstream infections: results of a quasi-experimental multicenter trial. Crit Care Med 2009;37:1858-65. [DOI] [PubMed] [Google Scholar]
- 28.Timsit JF, Schwebel C, Bouadma L, Geffroy A, Garrouste-Orgeas M, Pease S, et al. Chlorhexidine-impregnated sponges and less frequent dressing changes for prevention of catheter-related infections in critically ill adults: a randomized controlled trial. JAMA 2009;301:1231-41. [DOI] [PubMed] [Google Scholar]
- 29.Weick KE, Quinn RE. Organizational change and development. Annu Rev Psychol 1999;50:361-86. [DOI] [PubMed] [Google Scholar]
- 30.Rogers EM. Diffusion of innovations. Free Press, 2003.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.