When Antimicrobial Stewardship Isn′t Watching: The Educational Impact of Critical Care Prospective Audit and Feedback

Abstract

Prospective audit and feedback (PAF) is an effective strategy to optimize antimicrobial use in the critical care setting, yet whether skills gained during PAF influence future antimicrobial prescribing is uncertain. This multisite study demonstrates that knowledge learned during PAF is translated and incorporated into the practice of critical care physicians even when not supported by an antimicrobial stewardship program.

Antimicrobials are among the most commonly prescribed medications in hospitals, yet up to 50% of use is inappropriate [1–4]. Antimicrobial overuse has led to increases in antimicrobial resistance with resultant negative impacts on mortality, length of stay (LOS), and healthcare costs [5].

Increasing rates of antimicrobial resistance have led to the development of antimicrobial stewardship programs (ASPs) in an attempt to optimize use [3]. Although antimicrobial stewardship is pertinent to many practice settings, it is particularly relevant in critical care units due to the large volume of antimicrobial use and higher proportion of broad-spectrum agents [6, 7]. Antimicrobial stewardship interventions in critical care settings have resulted in reductions in antimicrobial use, resistance, and costs with neutral or positive effects on mortality, LOS, and readmission rates [8, 9].

Prospective audit of antimicrobial use with direct feedback to the prescriber is one of the core strategies for many effective ASPs [10–13]. Prospective audit and feedback (PAF) is a valuable intervention because each interaction with the clinician provides the opportunity for individualized and relevant education that may be applied to future similar scenarios. However, little is known about the impact of the education received by critical care clinicians during PAF. We hypothesized that skills learned by intensivists during exposure to PAF at one critical care unit would influence antimicrobial prescribing of those same clinicians at another critical care unit that was not actively supported by an ASP.

METHODS

Setting

The Niagara Health System is a 720-bed community teaching hospital with 3 sites offering acute care services including a level III medical-surgical intensive care unit ([ICU] A) and 2 level II medical-surgical ICUs (ICU B and ICU C). Intensive care unit A is a 14-bed closed unit that is staffed by formally trained critical care physicians. Intensive care unit B also provides a closed model of care for 8 ICU beds as well as 6 step-down beds with physician coverage provided by a mix of intensivists and internists. There is considerable cross coverage of physician staffing among the intensivists between ICU A and ICU B. Intensive care unit C is a partially closed unit comprising 8 ICU beds and 8 step-down beds staffed by internal medicine physicians that work solely at that site. Medical patients are under the care of the internist on service while surgeons remain the most responsible physician for surgical postoperative patients admitted to ICU C.

Intervention

Prospective audit and feedback was initiated only in ICU A in July 2013. The ASP team consisted of a clinical pharmacist, who participated in daily ICU multidisciplinary bedside rounds, and an infectious disease-trained ASP physician. Biweekly, all patients who were receiving antimicrobial therapy in ICU A underwent initial assessment by the pharmacist, who then reviewed each case with the ASP physician for appropriateness of therapy. Recommendations to optimize therapy were communicated to the critical care team verbally and via ASP progress notes.

Data Collection

The baseline period included data from July 1, 2012 to June 30, 2013, and the intervention period took place from July 1, 2013 to March 31, 2014.

Antimicrobial Use

Antimicrobial use was measured for all 3 ICUs during the baseline and intervention period and reported in defined daily doses (DDDs) per 1000 patient days monthly. The number of DDDs was determined based on aggregate pharmacy-dispensing data extracted from the pharmacy module of the hospital information system and standardized to determine DDD per 1000 patient days. Antimicrobial use was reported as overall broad-spectrum use that included the sum of DDD for third-generation cephalosporins (ceftriaxone, ceftazidime, cefixime), piperacillin/tazobactam, carbapenems (ertapenem, imipenem, meropenem), and fluoroquinolones (ciprofloxacin, levofloxacin, moxifloxacin) per 1000 patient days. Antipseudomonal antimicrobial use was also reported (sum of DDD for ceftazidime, piperacillin/tazobactam, imipenem, meropenem, ciprofloxacin, gentamicin, tobramycin, and amikacin per 1000 patient days) as was use of select individual antimicrobials.

Antimicrobial stewardship program recommendations were recorded prospectively in themed categories including de-escalation, discontinuation, duration optimization, dose optimization, intravenous to oral step-down, broaden or change empiric coverage, and suggestion for intervention or imaging. Optimization of duration involved setting, shortening, or extending treatment durations as appropriate. Acceptance rate was also recorded.

Patient Characteristics and Clinical Outcomes

Patient characteristics and clinical outcomes data for all 3 units were extracted from prospectively reported data in the Critical Care Information System as part of mandatory provincial reporting to the Ontario Ministry of Health and Long-Term Care [10]. Clinical outcomes data included quarterly LOS and mortality.

Statistical Analysis

Patient characteristics in the pre- and postintervention periods were compared using the Student t test for continuous variables and the χ² test for categorical variables. Antimicrobial use data handling, visualization, and analyses were performed using R version 3.0.2. The primary comparisons were made using Wilcoxon rank-sum tests. We performed 2 statistical tests to assess the appropriateness of this approach. To test for autocorrelation, Box-Pierce tests were performed on the residuals from a simple means model (mean use estimated for each combination of site, antimicrobial group, and intervention period). To test for temporal and seasonal effects, an F test was used to compare the simple means model with a multiple regression discontinuity model. The results of these 2 tests can be found in Supplementary Appendix 1. We adopted an alpha = .05 as the threshold for statistical significance. The data were visualized using density plots produced by ggplot2 (Gaussian smoothing kernel and default bandwidth).

RESULTS

Patient Characteristics

Patient demographics were similar between the 2 time periods for ICU A, B, and C; however, there was a significantly lower proportion of males admitted to ICU A and an increase in average multiple organ dysfunction score in ICU A during the intervention period (see Table 1).

Table 1.

Baseline Characteristics of Critical Care Patients Before and After PAF Implementation at ICU A^a

	ICU A			ICU B			ICU C
Variable	Preintervention Period (n = 1222)	Postintervention Period (n = 1124)	P Value	Preintervention Period (n = 1097)	Postintervention Period (n = 806)	P Value	Preintervention Period (n = 1345)	Postintervention Period (n = 932)	P Value
Average age	65	66	.056	68	68	.449	69	69	.597
Male sex, %	708 (58%)	584 (52%)	<.004	601 (55%)	451 (56%)	.612	743 (55%)	497 (53%)	.367
Medicine admissions, %	1056 (86%)	979 (87%)	.626	1036 (94%)	749 (93%)	.177	1087 (81%)	782 (84%)	.059
Surgical postoperative admission, %	166 (14%)	145 (13%)	.626	61 (6%)	57 (7%)	.179	258 (19%)	150 (16%)	.059
MODSb	2.32	2.95	<.0001	1.65	1.68	.738	1.67	1.85	.071
Mechanical ventilation days/patient	2.18	1.91	.393	1.17	0.95	.522	0.95	0.47	.093
Central line days/patient	4.32	3.68	.081	1.65	1.47	.562	0.95	0.74	.333

Bold values indicate a significant difference between groups.

Abbreviations: ICU, intensive care unit; MODS, multiple organ dysfunction score; PAF, prospective audit and feedback.

^a n = total number of admissions to unit. Note: Preintervention period July 1, 2012 to June 30, 2013; postintervention period July 1, 2013 to March 31, 2014.

^b Multiple organ dysfunction score on admission to unit.

Physician Staffing

Prospective audit and feedback was only implemented at ICU A; however, in ICU B, 159 of 274 (58.0%) days of service were covered by clinicians who had received prior exposure to PAF at ICU A. Physicians who worked at both ICU A and B provided coverage for 214 of 274 (78.1%) days at ICU A and 188 of 274 (68.6%) days at ICU B. The average shift duration was 5.1 days (range, 1–10 days) in ICU A and 3.6 days (range, 1–8 days) in ICU B. Staff worked back and forth between both units, and short, frequent shifts allowed for re-exposure to PAF at ICU A, which provided additional opportunities for education. The physicians at ICU C had no exposure to the ASP team during the intervention period.

Antimicrobial Use

The ASP team reviewed 477 patient cases during the intervention period and made 394 suggestions for optimization of therapy. The critical care team accepted 374 of these recommendations for an acceptance rate of 94.9%. The most common recommendations included duration optimization (40.6%), discontinuation of therapy (21.7%), and de-escalation of therapy (10.7%).

Similar changes in the patterns of antimicrobial use were seen in ICU A and ICU B in the postintervention period after the implementation of PAF in ICU A as shown in Figure 1. Overall broad-spectrum antimicrobial use decreased by 27.1% (P < .001) from 648.40 to 472.64 DDD per 1000 patient days in ICU A and by 21.7% (P = .015) from 400.12 to 313.11 DDD per 1000 patient days in ICU B. There was no change in the use of broad-spectrum antimicrobials in ICU C. There was a significant reduction in the use of antipseudomonal antimicrobials in both ICU A and B where use was decreased by 44.9% (P < .001) from 426.85 to 235.41 DDD per 1000 patients days and 28.7% (P = .003) from 246.85 to 176.04 DDD per 1000 patient days, respectively. There was no difference in antipseudomonal antimicrobial use in ICU C. Changes in commonly used antimicrobials and antimicrobial classes are shown in Supplementary Figure 2. Reductions in antipseudomonal and broad-spectrum antimicrobial use were predominantly related to decreased use of piperacillin/tazobactam, fluoroquinolones, and antipseudomonal carbapenems in ICUs A and B.

Figure 1.

Broad-spectrum and antipseudomonal antimicrobial use in intensive care unit (ICU) A, B, and C pre- and postimplementation of prospective audit and feedback (PAF) at ICU A. Linear lines represent mean antimicrobial use for each time period. Broad-spectrum antimicrobials include ceftriaxone, ceftazidime, cefixime, piperacillin/tazobactam, ertapenem, imipenem, meropenem, ciprofloxacin, levofloxacin, moxifloxacin. Antipseudomonal antimicrobials include ceftazidime, piperacillin/tazobactam, imipenem, meropenem, ciprofloxacin, gentamicin, tobramycin, amikacin. Note: PrePAF implementation July 1, 2012 to June 30 2013; postPAF implementation July 1, 2013 to March 31, 2014. Abbreviation: DDD, defined daily dose.

Clinical Outcomes

In ICU A, average quarterly LOS decreased from 6.26 days to 5.43 days and average quarterly mortality decreased from 13.75% to 8.85%. Average quarterly LOS remained similar during the baseline and intervention period for ICU B (4.31 days and 4.24 days) and ICU C (3.72 and 4.09 days). Average quarterly mortality trended up in the intervention period for ICU B (6.01%–8.15%) and ICU C (6.93%–8.26%).

DISCUSSION

As has been seen in prior studies, the implementation of PAF in ICU A was associated with a statistically significant decrease in broad-spectrum and antipseudomonal antimicrobial use without negatively impacting LOS and mortality [10–12]. However, our study demonstrates a parallel decrease in antimicrobial use in ICU B where formal ASP activities were not initiated but physicians were exposed to PAF in ICU A. There was no change in pattern of use in the control ICU. The change in antimicrobial use in ICU B supports the concept that antimicrobial stewardship principles can be taught through provision of PAF and applied in settings that are not formally supported by an ASP. Each case reviewed with the critical care physician in ICU A during PAF provided an opportunity for individualized, relevant, and practical education.

There are some limitations with our study. First, our study was not designed as a randomized controlled trial, and it is difficult to know whether other factors influenced the changes in antimicrobial use in ICU A and B other than the introduction of ASP. In our analysis, we included an ICU where physicians were not exposed to PAF as a control group to reduce any potential impact this may have had on our study results. Second, prescriber-specific antimicrobial use data were not collected, and as a result we were unable to determine with more certainty that the decrease in antimicrobial use in ICU B was attributed to changes in prescribing behavior among those physicians with ICU A PAF exposure. Third, the short duration of postintervention follow up does not allow a robust analysis of seasonality patterns or whether the impact of ASP education would be sustained in ICU B. However, the seasonality impact should effect all 3 ICUs if present. We did not see a change in the use of broad-spectrum or antipseudomonal antimicrobial use in ICU C, which strongly suggests the absence of a seasonality impact. Fourth, antimicrobial use data were extracted from pharmacy dispensing records and not administration records. As a result, we may have potentially overestimated actual use. However, this same limitation is present among all sites. Lastly, this was a single-center study, and our results may not be generalizable to other organizations due to differences in ICU structure, ASP resources, existing relationships, and culture.

CONCLUSIONS

Our study demonstrates that PAF in the critical care setting can have an impact on physician prescribing behavior beyond that which occurs at the time of the intervention. Knowledge and skills learned during PAF were translated to another critical care practice site that was not formally supported by an ASP. Further studies in different practice settings are required for confirmation of these findings.

Supplementary Data

Supplementary material is available online at Open Forum Infectious Diseases online (http://OpenForumInfectiousDiseases.oxfordjournals.org/).