Comparison of Prior Authorization and Prospective Audit with Feedback for Antimicrobial Stewardship

Abstract

OBJECTIVE

Although prior authorization and prospective audit with feedback are both effective antimicrobial stewardship program (ASP) strategies, the relative impact of these approaches remains unclear. We compared these core ASP strategies at an academic medical center.

DESIGN

Quasi-experimental study.

METHODS

We compared antimicrobial use during the 24 months before and after implementation of an ASP strategy change. The ASP used prior authorization alone during the preintervention period, June 2007 through May 2009. In June 2009, many antimicrobials were unrestricted and prospective audit was implemented for cefepime, piperacillin/tazobactam, and vancomycin, marking the start of the postintervention period, July 2009 through June 2011. All adult inpatients who received more than or equal to 1 dose of an antimicrobial were included. The primary end point was antimicrobial consumption in days of therapy per 1,000 patient-days (DOT/1,000-PD). Secondary end points included length of stay (LOS).

RESULTS

In total, 55,336 patients were included (29,660 preintervention and 25,676 postintervention). During the preintervention period, both total systemic antimicrobial use (−9.75 DOT/1,000-PD per month) and broad-spectrum anti-gram-negative antimicrobial use (−4.00 DOT/1,000-PD) declined. After the introduction of prospective audit with feedback, however, both total antimicrobial use (+9.65 DOT/1,000-PD per month; P < .001) and broad-spectrum anti-gram-negative antimicrobial use (+4.80 DOT/1,000-PD per month; P < .001) increased significantly. Use of cefepime and piperacillin/tazobactam both significantly increased after the intervention (P = .03). Hospital LOS and LOS after first antimicrobial dose also significantly increased after the intervention (P = .016 and .004, respectively).

CONCLUSIONS

Significant increases in antimicrobial consumption and LOS were observed after the change in ASP strategy.

Antimicrobial stewardship programs (ASPs) are multifaceted, interdisciplinary approaches to optimize anti-infective therapy and address emerging resistant pathogens.^1,2 Guidelines for stewardship identify several potential strategies, including 2 core “active” methods: formulary restriction with prior authorization and prospective audit with feedback to prescribers. Prior authorization permits use of select agents after approval from an ASP team member, whereas prospective audit with feedback utilizes postprescriptive reviews conducted by the ASP to recommend changes in agent selection, dosing, or duration of therapy.^2,3

Published literature supports both methods as being effective strategies to decrease antimicrobial exposure, decrease costs, and improve clinical outcomes.^4–9 However, because these 2 ASP methods have not been compared, the most effective approach remains unclear.^2,10 Implementation of ASPs is increasingly common, and widespread adoption of stewardship interventions has been promoted by professional and governmental organizations.^2,11,12 This study aims to compare an ASP based on prior authorization alone to one combining prior authorization and prospective audit with feedback.

METHODS

Setting and Patients

We examined antimicrobial use and length of stay (LOS) before and after a change in ASP approach at the Hospital of the University of Pennsylvania, a 776-bed tertiary care academic medical center in Philadelphia. Approval was obtained from the University of Pennsylvania Institutional Review Board. All inpatients at least 18 years of age who received at least 1 dose of any oral or intravenous antibacterial or antifungal agent were included. To maintain independency of antimicrobial prescribing practices, only the first admission for each individual was included during the study period.

ASP

The hospital ASP was implemented in 1993, using formulary restriction with prior authorization.¹ Prior authorization was provided by infectious diseases–trained clinical pharmacists, infectious diseases fellows, and the infectious diseases consult service, as described previously.¹³ Through May 2009, antimicrobials that were broad spectrum, associated with resistant organisms, associated with serious adverse events, or costly were restricted, requiring prior authorization from the ASP team before dispensing from the pharmacy. This included commonly used broad-spectrum agents such as cefepime and piperacillin/tazobactam as well as antifungals. Vancomycin was available for 72 hours without prior authorization, but further use required ASP approval.

Requests for prior authorization increased over time, many for empiric broad-spectrum antibiotic regimens. To better utilize ASP resources, restriction categories were reassessed. In June 2009, the restriction was lifted for most antimicrobial agents on formulary, including the broad-spectrum agents cefepime, piperacillin/tazobactam, and vancomycin. For these, prospective audit with feedback was also introduced in June 2009. Audits occurred Monday through Friday by 3 infectious diseases–trained clinical pharmacists and included all patients in the hospital with an active order for any of those agents. Suggested modifications, if necessary, were communicated to the covering provider, and acceptance of the recommendations was at their discretion. Prior authorization, conducted in the same manner as before, remained for other broad-spectrum and/or costly agents (including aztreonam, ceftazidime, daptomycin, levofloxacin, linezolid, and meropenem) and for antifungal agents (Figure 1).

FIGURE 1.

Timeline of antimicrobial stewardship program (ASP) strategies for broad-spectrum antimicrobials at the Hospital of the University of Pennsylvania.

For this analysis, the preintervention period contained all patients admitted during the 24 months before the ASP change, June 2007 through May 2009. The postintervention period included admissions during the 24 months after the change, July 2009 through June 2011. June 2009 was excluded from both periods, as this was a transitional time for both prescribers and the ASP.

Outcome Assessment

The primary outcome was antimicrobial doses administered measured by days of therapy (DOT), the preferred metric of antimicrobial use.¹⁴ A single DOT is recorded for each individual antimicrobial administered to a patient on a given day regardless of dose and frequency.¹⁵ A complementary measure of antimicrobial use, length of therapy (LOT), corresponds to each day a patient receives any systemic antimicrobial therapy, regardless of the number of agents or doses.¹⁶ For example, an individual who receives cefepime for 3 days contributes 3 DOT and 3 LOT, whereas a patient receiving both cefepime and vancomycin for 3 days contributes 6 DOT but 3 LOT. Although the primary outcome quantified composite use of all 6 broad-spectrum anti-gram-negative antibiotics on formulary (aztreonam, cefepime, ceftazidime, levofloxacin, piperacillin/tazobactam, and meropenem), subanalyses focused on cefepime and piperacillin/tazobactam, the specific anti-gram-negative agents for which the stewardship approach changed from prior authorization to prospective audit.

Additional end points evaluated included total hospital LOS and LOS after receipt of first dose of systemic antimicrobial, both measured in days for all patients who received at least 1 dose of an antimicrobial. Appropriateness of empiric antibiotic selection for the subset of patients with bloodstream infections (BSIs) caused by gram-negative organisms was also assessed (with the exception of the first year of the preintervention period because microbiology data were not available before May 2008). Empiric antibiotic regimens were evaluated by a blinded infectious diseases–trained clinical pharmacist on the following criteria: (1) the antibiotic(s) were active against the pathogen; (2) active antibiotic(s) were administered within 24 hours of the blood culture; (3) the antibiotic(s) used were of the narrowest spectrum available based on the results of susceptibility testing and the reviewers’ opinion; and (4) in the case of multiple antibiotic agents, redundant or overlapping spectra of activity were minimized.

Control Outcomes

To account for factors that could impact antimicrobial use and LOS other than the change in ASP strategy, we examined 3 additional antimicrobial groups.^17,18 These included (1) all systemic antibiotics and antifungals except the 3 drugs that were prospectively audited (referred to as nonaudited antimicrobials), to uncover trends in total antimicrobial use during the study period; (2) vancomycin, the most commonly used broad-spectrum anti-gram-positive agent at our institution; and (3) all systemic antifungal drugs, to provide a nonequivalent dependent variable. Nonequivalent dependent variables are outcome variables similar to the primary outcome and subject to the same alternative causes of an observed effect but are not directly affected by the intervention.

Data Source

All data were obtained from the Penn Data Store (PDS), a centralized data repository of information from all electronic clinical systems across the Penn Medicine Health System. PDS contains demographics, diagnoses, procedures, laboratory results, medication orders, and doses administered for all inpatient and outpatient visits. For this study, demographic, microbiology, medication orders, and administration records were extracted for inpatients admitted during the 4-year study period.

Statistical Analysis

Patient characteristics were compared between the 2 time periods using the χ² or Fisher exact test for categorical variables and the Wilcoxon rank-sum test for continuous variables. Antimicrobial consumption was evaluated on a monthly basis. All included patients were assigned a month (1–49) according to admission date. DOT, LOT, and LOS for each patient were calculated, totaled for each month, and then standardized to 1,000 patient-days (DOT/1,000-PD, LOT/1,000-PD, and days/1,000-PD, respectively), using total patient-days for all admissions in that month irrespective of antimicrobial administration.¹⁵

Because antimicrobial consumption for a given month is likely related to that in previous months, we used interrupted time series analysis.^17,19 The generalized autoregressive conditional heteroskedasticity (GARCH) model was fitted for all DOT and LOS measures. GARCH allows for the assessment of autocorrelation, volatility, and the influence of an exogenous input, such as the change in ASP method.^20,21

For the treatment of gram-negative BSIs, the frequency of adequate empiric therapy, redundant spectra of activity, and excessively broad therapy were compared in the preintervention and postintervention groups using the χ² test. All analyses were performed with STATA statistical software (ver. 12.1; StataCorp, College Station, TX).

RESULTS

Patient Characteristics

We identified 55,336 patients who received antimicrobial therapy from June 2007 through June 2011: 29,660 patients before the intervention and 25,676 after the intervention. Patients in both groups were similar with regard to baseline demographics collected at admission (Table 1).

TABLE 1.

Patient Characteristics

Characteristic	Preintervention (n = 29,660)	Postintervention (n = 25,676)
Age, median (IQR), years	54 (38–67)	54 (38–67)
Female sex	15,866 (53.5)	13,564 (52.8)
White race	16,700 (56.3)	15,053 (58.6)
β-lactam allergy	2,868 (9.7)	2,675 (10.4)
Admitting service
Medical	9,340 (31.5)	7,692 (30.0)
Surgical	16,420 (55.4)	14,741 (57.4)
Other	3,900 (13.1)	3,243 (12.6)
Admission source
Routine	14,960 (50.4)	12,781 (49.8)
Emergency department	8,459 (28.5)	7,075 (27.6)
Transfer from outside facility	3,624 (12.2)	3,713 (14.5)
Other	2,617 (8.8)	2,107 (8.1)
Risk mortality score
1: Minor risk of mortality	16,353 (55.3)	12,825 (50.4)
2: Moderate risk of mortality	6,285 (21.2)	5,579 (21.9)
3: Major risk of mortality	4,053 (13.7)	4,048 (15.9)
4: Extreme risk of mortality	2,890 (9.8)	3,006 (11.8)
Severity of illness score
1: Minor severity of illness	8,988 (30.4)	6,371 (25.0)
2: Moderate severity of illness	9,867 (33.3)	8,418 (33.1)
3: Major severity of illness	6,770 (22.9)	6,681 (26.2)
4: Extreme severity of illness	3,956 (13.4)	3,988 (15.7)

NOTE. Data are no. (%), unless otherwise indicated. Risk mortality and severity of illness scores are calculated on admission according to All Patient Refined Diagnosis-Related Group. IQR, interquartile range.

Consumption of Broad-Spectrum Anti-Gram-Negative Agents

During the preintervention period, use of broad-spectrum anti-gram-negative antibiotics was declining at a rate of −4.00 DOT/1,000-PD per month. However, during the postintervention period, use increased by 0.80 DOT/1,000-PD per month, indicating that the change in ASP was associated with a slope change of 4.80 DOT/1,000-PD per month (P < .001; Table 2, Figure 2), or an increase of 0.49% per month after the intervention. Similarly, after decreasing during the 2 years before the ASP change, use of cefepime and piperacillin/tazobactam significantly increased following the transition to prospective audit with feedback by 3.21 DOT/1,000-PD per month (P = .003; Table 2, Figure 2). This represented a 1.1% increase in use per month through the postintervention period. The use of the other 4 broad-spectrum anti-gram-negative antibiotics, for which prior authorization was required in both time periods, decreased before the intervention, with no significant increase after the intervention (slope change, 1.62 DOT/1,000-PD per month; P = .100; Table 2, Figure 2).

TABLE 2.

Antimicrobial Consumption over Time

	Slope, DOT/1,000-PD per month (95% CI)
Antimicrobial group	Preintervention	Postintervention	Changea	P
Broad-spectrum anti-gram-negative agents
All agents	−4.00 (−6.11, −1.90)	0.80 (−0.23, 1.83)	4.80 (2.46, 7.14)	<.001
Cefepime and piperacillin/tazobactam	−1.86 (−3.87, 0.14)	1.35 (0.79, 1.91)	3.21 (1.13, 5.30)	.003
Aztreonam, ceftazidime, levofloxacin, and meropenem	−2.15 (−4.04, −0.25)	−0.53 (−0.88, −0.17)	1.62 (−0.31, 3.55)	.100
Other antimicrobial agents
All systemic antimicrobials	−9.75 (−13.69, −5.82)	−0.10 (−1.77, 1.57)	9.65 (5.38, 13.93)	<.001
Vancomycin	−0.46 (−0.86, −0.01)	0.43 (−0.04, 0.89)	0.89 (0.27, 1.50)	.005
Antifungals	−1.77 (−3.07, −0.47)	0.65 (0.21, 1.09)	2.42 (1.05, 3.79)	.001
Nonaudited antimicrobials	−7.43 (−9.98, −4.89)	−1.87 (−3.18, −0.57)	5.56 (2.70, 8.42)	<.001

NOTE. CI, confidence interval; DOT/1,000-PD, days of therapy per 1,000 patient-days.

^aDifference between the preintervention and postintervention slopes.

FIGURE 2.

Trends in consumption of broad-spectrum anti-gram-negative agents (days of therapy per 1,000 patient-days) by month for the 4-year study period.

Consumption of Other Antimicrobial Agents

Overall use of all systemic antimicrobial agents significantly increased after the change in ASP method (P < .001; Table 2, Figure 3). Vancomycin use declined before the intervention but significantly increased after the intervention (P = .005; Table 2, Figure 3). While use of nonaudited antimicrobials also significantly increased after the change in ASP methods (P < .001), the slope during the postintervention period continued to decline at −1.87 DOT/1,000-PD per month (Table 2, Figure 3). The LOT of all systemic antimicrobials declined before the intervention by −2.30 LOT/1,000-PD per month, and, despite a significant increase in slope (P = .029), use continued to decrease after the intervention by −0.33 LOT/1,000-PD per month.

FIGURE 3.

Trends in consumption for all systemic antimicrobials, nonaudited antimicrobials, vancomycin, and antifungal agents (days of therapy per 1,000 patient-days) by month for the 4-year study period.

Antifungal agents, the nonequivalent dependent control variable, followed similar patterns of use as the primary variable. Before the intervention, use of these drugs decreased by −1.77 DOT/1,000-PD per month. After the intervention, use increased by 0.65 DOT/1,000-PD per month, representing a slope change of 2.42 DOT/1,000-PD per month (P = .001; Table 2, Figure 3).

LOS

Before the intervention, total hospital LOS for patients receiving antimicrobials was decreasing at a rate of −1.57 days/1,000-PD per month; however, LOS following the ASP transition increased by 1.94 days/1,000-PD (P = .016). When limiting the analyses to LOS after the first dose of a systemic antimicrobial, we found a similar reversal of trends (−1.42 days/1,000-PD per month before the intervention; +2.30 days/1,000-PD after the intervention; P = .004; Figure 4).

FIGURE 4.

Trends in length of stay for total hospitalization and in length of stay after receipt of antimicrobials by month for the 4-year study period.

Empiric Antibiotic Selection

A total of 488 patients with gram-negative BSIs were identified from the available microbiology data. There were 165 BSI episodes available in 13 months of the preintervention period and 311 in 24 months of the postintervention period. There was no significant difference in the proportion of inappropriate empiric antibiotic regimen selection between the 2 groups (18.2% vs 18.0%; P = .962). Similarly, there were no significant differences in the proportion of redundant antibiotic regimen (11.5% vs 10.3%; P = .681) and in excessively broad antibiotic coverage (42.4% vs 37.3%; P = .275) before and after the intervention.

DISCUSSION

The results of this study represent a formal comparison of the 2 core antimicrobial stewardship methods within a single institution. Our findings suggest that the change from prior authorization to prospective audit with feedback was associated with a significant increase both in use of the affected antimicrobials and in overall use of all antimicrobial agents. Furthermore, broad-spectrum anti-gram-negative agents that still required prior authorization during both time periods continued to decline in use after the change in ASP. Last, the overall change in stewardship approach was associated with a significant increase in hospital LOS.

Prior studies of stewardship interventions have found implementation of either prior authorization or prospective audit with feedback to be effective strategies for decreasing antimicrobial exposure, decreasing costs, and improving clinical outcomes.^4–6,8,9 Elligsen et al⁷ performed a quasi-experimental study to assess the effects of the implementation of prospective audit with feedback in the intensive care units at a tertiary medical center using adequate design and statistical approaches. The ASP was associated with an immediate and sustained decrease in targeted antibiotic consumption as well as in overall antibiotic use. However, this study, like prior stewardship studies, compared drug use in the presence of an ASP to drug use in its absence. One multicentered study, conducted by Cosgrove et al,²² provided the first insight into a possible benefit of an ASP utilizing both active strategies. The intervention, a postprescriptive review overlaid on any ASP already established at the study site, was implemented at 5 tertiary care academic medical centers. Although their findings were heterogeneous, the sites with prior authorization-based ASPs had significant reductions in antimicrobial consumption after the postprescriptive review was implemented, whereas sites without an ASP had significant increases in antimicrobial use after the intervention. These results suggest that an ASP combining both active strategies may be most effective.

LOS as a clinical outcome is a potentially important measure of ASP effectiveness.³ Additionally, LOS is an important factor in healthcare cost analysis.²³ The benefit of ASP efforts in improving LOS has been varied in published studies. Several studies could not establish a significant impact on LOS.^4,5,7 However, Gentry et al²⁴ found a significant reduction in LOS after a change in the ASP program at a Veterans Affairs medical center. Previously, ASP at their hospital relied on prior authorization from an infectious diseases physician for use of restricted or nonformulary antimicrobials beyond 24 hours. Their ASP was changed with the inclusion of an infectious diseases clinical pharmacist who prospectively audited the use of the drugs. While this analysis appears similar to our study, it is important to note that their comparison is between physician- and pharmacist-staffed ASPs. As was demonstrated in another study, these 2 groups differ in stewardship effectiveness.¹³ Our study, however, is a comparison of the 2 active ASP methods used by the same ASP personnel in the same setting. LOS appears to have significantly increased after the change in ASP; however, the mechanism for this finding is unknown.

There are several potential limitations to our study. First, process measures, such as antimicrobial use, are considered inadequate when used alone to evaluate ASP interventions because such outcomes do not demonstrate a direct clinical benefit.¹² Standardized and validated clinical end points have not been established in retrospective studies of anti-infective therapy, a substantial knowledge gap in stewardship research.¹¹ Drug utilization was a relatively easy metric to capture in this first comparison of ASP methods. Additionally, a panel of experts recently recommended DOT as one of 5 quality metrics for ASP evaluation as well as identifying it as a metric for public reporting.¹⁴ However, DOT assesses only antimicrobial doses that were administered to the patient. We were unable to quantify the effect of each ASP method on preventing erroneous or unnecessary therapy.

Another important limitation is that the influence of factors other than the change in ASP cannot be fully excluded. Throughout the 4 years of this study, changes in patient demographics, bacterial resistance rates, or hospital characteristics could have impacted antimicrobial use patterns. Other, more difficult to measure factors could also impact drug use, such as prescribing philosophies or attitudes of the ASP team after liberalization of many antimicrobials. Control variables were used to assess this, in particular, antifungal agents. The increased use of antifungals after the intervention suggests that external factors may have lead to a global trend for increased use of all antimicrobials. However, consumption of the 4 broad-spectrum anti-gram-negative antibiotics that required prior authorization during both time periods continued to decline after the intervention, in contrast to cefepime and piperacillin/tazobactam. Additionally, the use of nonaudited antimicrobials also exhibited a positive change in slope, although the overall trend showed a continued decline in use during the postintervention period. Together, these findings suggest 2 possibilities: (1) antifungals were not an adequate nonequivalent dependent control due factors specific to these drugs or (2) the postintervention increase in use of unrestricted broad-spectrum drugs occurred independently from the increase in antifungal use. The presence of such factors must also be taken into account when interpreting the LOS findings as a possible alternative explanation for the increased stay.

This study observed a significant increase in antimicrobial use after the inclusion of prospective audit with feedback intervention to an ASP based on prior authorization. While other explanations may limit the interpretation of these findings, our results suggest a difference may be present between the 2 core ASP strategies. Further comparative studies focused on treatment-related clinical outcomes and microbiologic sequelae would strengthen the case for one particular antimicrobial stewardship method over others.