Reducing Antimicrobial Use in an Academic Pediatric Institution: Evaluation of the Effectiveness of a Prospective Audit With Real-Time Feedback

Antimicrobial use is decreasing in children's hospitals nationwide. The effect of antimicrobial stewardship at our hospital was compared to that at 43 comparator hospitals, and a greater-than-expected decline in use was found. Clinical factors associated with adherence to stewardship recommendations were explored.

Abstract

Introduction

Antimicrobial use is decreasing across freestanding children's hospitals, predominantly in institutions with antimicrobial stewardship programs (ASPs) in place. A highly effective ASP should effect a greater decrease in use than predicted by existing trends. Antimicrobial stewardship programs depend on clinician adherence to program recommendations, but little is known about factors associated with adherence.

Methods

Parenteral antimicrobial-use data for our institution and 43 additional freestanding children's hospitals were obtained and normalized for patient census. Segmental linear regression was used to compare rates of change of parenteral antimicrobial use before and after ASP implementation. Time-series models were developed to predict use in the absence of intervention. The odds of adherence to ASP recommendations were determined based on provider characteristics and recommendation type.

Results

In the 38 months before ASP implementation, parenteral antimicrobial use was decreasing at our hospital by 3.7%/year, similar to the 3.4%/year found across children's hospitals. The rate of change after implementation of the ASP at our hospital was 11.1%/year, compared to 5.6%/year for other hospitals over the same period. Of 643 interventions, teams adhered with recommendations in 495 cases (77.0%). According to adjusted analysis, primary service was not associated with adherence (P = .356). There was an association between adherence and the role of the clinician receiving a recommendation (P = .009) and the recommendation type (P = .009).

Conclusions

Understanding factors associated with adherence to ASP recommendations can help those who administer such programs to strategize interventions for maximizing efficacy. Our findings reveal the value of a formal ASP in reducing use when controlling for secular trends.

INTRODUCTION

In freestanding children's hospitals, 60% of hospitalized children receive at least 1 dose of an antimicrobial during their admission [1]. There is substantial interhospital variability in both the proportion of hospitalized children treated with antimicrobials and the breadth of spectrum of the agents used [1]. Implementation of an antimicrobial stewardship program (ASP) has proven to prevent medication errors [2], reduce hospital costs [3–7], and reduce rates of infection caused by Clostridium difficile [6, 8] and multidrug-resistant bacterial pathogens [6, 9]. The White House recently released the National Action Plan for Combating Antibiotic-Resistant Bacteria, which instructs the Centers for Medicare and Medicaid Services to add formal stewardship activities to its conditions of participation [10, 11].

There are 2 core strategies used in antimicrobial stewardship: restriction and prospective audit with feedback (PAWF). A recent Cochrane review found these 2 strategies to be similar in effectiveness in reducing exposure of patients to antibiotics, although restrictive policies have a more immediate effect [12]. In children's hospitals, programs that emphasize PAWF have been found to be effective [13–15] and acceptable to clinicians [16]. The success of the PAWF strategy relies on compliance of prescribers with the nonbinding recommendations of the stewardship team.

A recent study using the Pediatric Health Information System (PHIS) database found that overall antimicrobial use, measured as days of therapy per 1000 patient-days (DOT/1000), has been declining in freestanding children's hospitals since 2007 [17]. This decline was occurring in hospitals with or without stewardship programs, but the rate of decline was greater in those with an ASP [17]. The authors of that report pointed out that evaluations of ASPs should account for these secular trends.

The ASP at Monroe Carell Jr. Children's Hospital at Vanderbilt (MCJCHV) was begun in March 2012. We hypothesized that ASP implementation would result in a decline in antimicrobial use and that this decline would exceed that predicted by trends in other freestanding children's hospitals. Furthermore, we hypothesized that ASP implementation would be associated with a decline in pharmacy costs related to antimicrobials. We also sought to identify factors associated with compliance with stewardship interventions, hypothesizing that factors including provider characteristics and ASP intervention type would predict failure to adhere to ASP recommendations.

METHODS

Setting

MCJCHV is a 271-bed freestanding children's hospital in Nashville, Tennessee. There are 100 neonatal intensive care beds and 42 pediatric critical care beds, including a separate cardiac critical care unit. There are active kidney, heart, and stem cell transplantation programs. In 2014, there were 14 615 hospital discharges.

Antimicrobial Stewardship Program

The ASP was implemented fully in March 2012. Since inception, its members have included a pharmacist with infectious diseases (ID) training (0.9 full-time equivalent) and an attending pediatric ID physician (0.3 full-time equivalent). The primary strategy is PAWF. Antimicrobial monitoring is performed with Sentri7 software (Wolters Kluwer Health, Philadelphia, Pennsylvania), which integrates with the electronic medical record system to generate lists of patients who are receiving monitored antimicrobials (Supplementary Table 1, left column). In addition, the monitoring software generates a report of patients with positive blood or cerebrospinal fluid culture results or who are receiving duplicative therapy (eg, piperacillin-tazobactam plus metronidazole). The ASP pharmacist reviews these reports each morning, Monday through Friday, and ordering teams are called with recommendations as needed. The ASP pharmacist most often pages the contact pager listed in the patient's orders, typically the primary team's resident or nurse practitioner. In some instances, the first contact is with an attending or fellow physician for the primary team. The pharmacist can also choose to contact the ordering team's pharmacist, who delivers the recommendation to the providers. The ASP pharmacist and physician review many cases regarding the best course of action before making recommendations. The pharmacist follows up over the next 24 to 48 hours to determine if recommendations were followed. If a recommendation has not been followed and there is concern for imminent patient harm, the ASP physician can elect to call the patient's attending physician to reiterate the recommendations.

Certain antimicrobials are restricted (see Supplementary Table 1, right column) and require approval from the attending ID physician on the clinical service before initiation. Ordering teams must select the approving attending ID physician from a drop-down list in the order-entry system; the selected ID physician receives an e-mail with details of the approval. The prior-authorization requirement is in place 24 hours/day. In selected cases, other subspecialist physicians are permitted to approve specific antimicrobials, as specified in Supplementary Table 1. Antimicrobials are also restricted during times of shortage, as with the intravenous formulation of levofloxacin. All restricted antimicrobials are also monitored.

In addition to these strategies, the ASP actively participates in the development of clinical practice guidelines to standardize the approach to common conditions that are responsible for heavy antimicrobial use.

Intervention Analysis

For each ASP intervention between January 2013 and June 2014, the date of intervention, the patient's primary service, the antimicrobial agent that triggered intervention, the role of the provider who accepted the recommendation, the recommendation type, and adherence were documented in a secure database. These interventions arose via prospective audit and exclude prior-authorization decisions. Recommendations made to the ID service were excluded from analysis because they were uniformly followed. The primary outcome was compliance with the recommendation within 48 hours (adherence). When a compromise was reached or additional clinical information that justified an alternative therapy was provided, the case was recorded as adherent. Exposures included the antimicrobial in question, primary service, the role of the provider who received the recommendation, and the type of recommendation. Primary service was grouped into general medical, including general pediatrics and medical subspecialties except hematology/oncology and stem cell transplant; oncology and stem cell transplant; surgical subspecialties; and critical care, which included the neonatal intensive care, pediatric critical care, and cardiac critical care units. The provider role was identified as attending or fellow, nurse practitioner, resident, or pharmacist. The pharmacist assigned intervention type according to 1 of 13 classifications. Intervention types with fewer than 20 instances were classified as “other” (Supplementary Table 2).

A prespecified multivariable logistic regression model was used to estimate the association of adherence to recommendation with the categorical predictors primary service, provider role, and recommendation type. Separate Wald tests were used to evaluate if each of the predictors was associated with adherence at a significance level of .05. Final results were summarized using adjusted odds ratios (ORs) with 95% confidence intervals (CIs). Analyses were conducted in R 3.1.2 (R Core Team, Vienna, Austria).

Antimicrobial Use

Antimicrobial DOT and monthly census data were obtained from hospitals that reported to the PHIS database between January 2009 and June 2014. The PHIS database collects inpatient data, including billings for medication administration, from 45 not-for-profit children's hospitals throughout the United States [18]. The PHIS database reports were created locally and used as the source of use data for our own institution. Gross use was expressed in DOT, which is equal to the sum of the number of antimicrobials received by a patient on each calendar day of hospitalization [19]. Thus, for example, an infant who receives both ampicillin and gentamicin for 3 days has received 6 DOT. Using this method, an antimicrobial given just before midnight on 1 day and once more the following morning is counted as 2 DOT. Monthly census was expressed in patient-days. The final measure of use, DOT/1000, was calculated by dividing the DOT by (1000 × patient-days) [20]. Antimicrobials were classified as antibiotic, antifungal, or antiviral. “All antimicrobials” included any antimicrobial with intravenous or intramuscular routes of administration; topical and oral agents were excluded. Data for all antimicrobials through June 2014 were available; data for subcategories were available through December 2013. Hence, for each hospital that reported adequate medication-administration data to the PHIS, trends in use of all parenteral antimicrobials or any subclass thereof could be analyzed. Months in which the reported antimicrobial or antibiotic use was less than 250 DOT/1000 or greater than 1000 DOT/1000 were set as missing. For MCJCHV only, monthly parenteral antimicrobial costs, calculated by using charges for doses administered to individual patients, were obtained from administrative databases for the months June 2009 through September 2015. Pre-ASP and post-ASP costs for each class were compared using the Wilcoxon rank-sum test.

To conduct the analysis of antimicrobial use, data were first log-transformed. To assess the effect of ASP implementation at MCJCHV in March 2012, piecewise linear spline regression was used to model the change in use over time, with a knot placed at March 15, 2012. Because the data were log-transformed, the slope of each line was interpreted as percent change in use per year. The slope before March 15, 2012, is denoted the pre-ASP trend, with the slope after March 15, 2012, denoted the post-ASP trend. This analysis was repeated for each included hospital (n = 43) using the same date of March 15, 2012, as the knot. Boxplots were generated to display the distributions of the pre-ASP and post-ASP trends and compare the position of MCJCHV within those distributions during the 2 time frames.

For MCJCHV data only, autoregressive integrated moving average models were used to predict antimicrobial and antibiotic use after March 15, 2012. Holt-Winters double-exponential smoothing was used to allow for periodicity, seasonal trends, and the overall decreasing trend over the observed time period [21, 22]. To visualize the effect of the ASP, we plotted the predicted use if an ASP had not been implemented and the observed use with the ASP implemented over time. Confidence intervals (80% and 95%) for the predictions are included. Analyses were conducted in R 3.1.2 (R Core Team).

This study was deemed exempt as nonhuman subjects research by the institutional review board of Vanderbilt University.

RESULTS

Interventions and Prescriber Adherence

During the study period, 643 ASP interventions were made. The distributions of interventions according to antimicrobial class, primary service, provider role, and intervention type are listed in Table 1. Interventions were made most commonly on anti–methicillin-resistant Staphylococcus aureus agents (33.2%), followed by antipseudomonal beta-lactams (31.4%); no other class accounted for greater than 10% of the total. Vancomycin accounted for 29.8% of all interventions. Critical care services accounted for the majority of interventions (58.4%), followed by general medical services (19.7%). The primary contact was most commonly a resident (41.0%), followed by a nurse practitioner (37.7%). The most common recommendation was to discontinue therapy (30.8%), followed by recommendations to change therapy (19.6%), consult an ID physician (17.9%), and optimize antimicrobial dosing (16.3%).

Table 1.

Distribution of ASP Recommendations and Adjusted Odds Ratios of Adherence According to Primary Service, Prescribing Provider Role, and Recommendation Type

Categorical Predictora	P (for Group)	Adherence
		n (%)	n (%)	OR (95% CI)
Primary service	.356
Critical care		376 (58.4)	275 (73.1)	Reference
Surgical subspecialties		58 (9.0)	41 (70.7)	0.57 (.29–1.14)
General medical		127 (19.7)	105 (82.7)	1.10 (.59–2.05)
Hematology/oncology, stem cell transplant		83 (12.9)	74 (89.2)	1.00 (.34–2.92)
Provider role	.009
Nurse practitioner		243 (37.7)	164 (67.5)	Reference
Resident		264 (40.1)	206 (78.0)	1.87 (1.15–3.02)
Attending or fellow		28 (4.3)	24 (85.7)	2.95 (.94–9.36)
Pharmacist		108 (16.8)	100 (92.6)	4.73 (1.54–14.57)
Recommendation type	.009
Consult with ID physician		115 (17.9)	76 (66.1)	Reference
Change antimicrobial therapy		126 (19.6)	90 (71.4)	1.21 (.67–2.17)
Monitor drug levels		45 (7.0)	33 (73.3)	1.80 (.80–4.04)
Discontinue antimicrobial		198 (30.8)	44 (77.8)	2.01 (1.16–3.48)
Optimize dose		105 (16.3)	90 (85.7)	2.62 (1.30–5.24)
IV-to-PO conversion		20 (3.1)	19 (95.0)	5.78 (.72–46.22)

Abbreviations: ASP, antimicrobial stewardship program; CI, confidence interval; ID, infectious disease; IV, intravenous; OR, odds ratio; PO, per os/oral.

^aThe analysis of each of the 3 variables as a predictor was adjusted for the other 2 variables. The summary P value for each group was calculated using a Wald test.

Prescribers adhered with 495 (77.0%) of 643 interventions. Escalation, in which the ASP physician contacted the service's attending provider, occurred in 15 (2.3%) cases and was successful in 9 (60%) of those cases. Adherence according to exposure group and the ORs of adherence are listed in Table 1. In adjusted analysis, there was no difference in adherence according to clinical service (P = .356). There was a significant difference in adherence among different provider types (P = .009) and among recommendation types (P = .009). Compared with nurse practitioners, pharmacists (OR, 4.73 [95% CI, 1.54–14.57]) and residents (OR, 1.87 [95% CI, 1.15–3.02]) were more likely to adhere to recommendations. When attending and fellow physicians were compared to nurse practitioners, there was no significant difference in the likelihood of adherence (OR, 2.95 [95% CI .94–9.36]). Compared with recommendations to consult the ID physician, recommendations to discontinue an antimicrobial (OR, 2.01 [95% CI, 1.16–3.48]) and to optimize a dose (OR, 2.62 [95% CI, 1.30–5.24]) were significantly more likely to be followed.

Antimicrobial Use and Cost

Mean monthly use of all parenteral antimicrobials, antibiotics, and antivirals was lower in the post-ASP than in the pre-ASP period (Table 2). The autoregressive integrated moving average predictive models for antimicrobial and antibiotic use are shown in Figure 1 (predicted use is indicated by the solid line, and observed used is indicated by the dotted line). Figure 2 displays log-transformed antimicrobial-use data with piecewise linear regression lines for the 2 periods at MCJCHV.

Table 2.

Monthly Parenteral Antimicrobial Use, Normalized to Patient Census

Antimicrobial	Use (Mean [SD]) (DOT/1000)		Months of Post-ASP Observation	P Valuec
	Pre-ASPa	Post-ASPb
All	703 (42.1)	577 (45.6)	28	<.001
Antibiotics	639 (38.1)	527 (43.9)	22	<.001
Antifungals	35 (7.0)	31 (10.0)	22	.077
Antivirals	28 (6.0)	21 (4.0)	22	<.001

Abbreviations: ASP, antimicrobial stewardship program; DOT/1000, days of therapy per 1000 patient-days; SD, standard deviation.

^aThe pre-ASP period was from January 2009 to February 2012 (38 months).

^bThe post-ASP period began in March 2012, and the number of months observed varied according to parameter.

^cWilcoxon rank-sum test.

Figure 1.

Parenteral antimicrobial (A) and antibiotic (B) use at Monroe Carell Jr. Children's Hospital at Vanderbilt. The line marked “Predicted Utilization” represents predicted use based on autoregressive integrated moving average models; the light and dark shaded areas represent 80% and 95% confidence intervals, respectively, for the prediction adjusted for periodicity, seasonal trends, and the overall decreasing trend. The line marked “Actual Utilization” represents actual (observed) use. The y-axis is log-transformed. Abbreviation: DOT/1000, days of therapy per 1000 patient-days.

Figure 2.

Piecewise linear regression of parenteral antimicrobial (A) and antibiotic (B) use at Monroe Carell Jr. Children's Hospital at Vanderbilt in units of days of therapy per 1000 patient-days (DOT/1000), with a knot (vertical dashed line) placed at the time of ASP implementation. Abbreviation: ASP, antimicrobial stewardship program.

Figure 3 displays the pre-ASP and post-ASP slopes for all members of the PHIS for antimicrobials and antibiotics (MCJCHV is indicated by the black dots). The slopes represent percent change per year. There were 30 months across 4 hospitals in the sample in which an individual hospital's reported use was fewer than 250 or greater than 1000 DOT/1000; the data for these hospitals were set to missing because of concerns about reliability. In the pre-ASP period (January 2009 to February 2012), antimicrobial use at MCJCHV was declining at a rate of 3.7%/year, which is similar to the change at the typical PHIS hospital (median decline, 3.4%/year). After ASP implementation at MCJCHV, antimicrobial use declined by 11.1%/year, whereas the median decline for all PHIS hospitals was 5.6%/year. For antibiotics, the rate of change for antimicrobial use decreased from −4.2%/year to −14.4%/year at MCJCHV, compared to a change from −4.6% to −5.6%/year for all PHIS hospitals.

Figure 3.

Distribution of the percent change of parenteral antimicrobial (A) and antibiotic (B) use in all assessed PHIS hospitals before and after the date of ASP implementation at Monroe Carell Jr. Children's Hospital at Vanderbilt (MCJCHV). Solid circles, MCJCHV; bold lines, medians; boxes, 25th and 75th percentiles (vertical lines include the full range); open circles, outliers falling greater than 1.5 times the interquartile range above the 75th percentile or below the 25th percentile. Abbreviations: ASP, antimicrobial stewardship program; PHIS, Pediatric Health Information System.

Antimicrobial costs for all parenteral antimicrobials and for subclasses unadjusted for inpatient census are listed in Table 3. Monthly median expenditures for all parenteral antimicrobials decreased from $136 121 to $93 474, a difference of $42 647/month.

Table 3.

Antimicrobial Costs for Administered Parenteral Antimicrobials During the Pre-ASP and Post-ASP Periods

Antimicrobial	Median Cost (US$/mo)		P Valuea
	Pre-ASP (July 2009–February 2012)	Post-ASP (March 2012–September 2015)
All	136 121	93 474	<.001
Antibiotics	102 096	69 290	<.001
Antifungals	28 331	18 480	<.001
Antivirals	5150	3151	.014

Abbreviation: ASP, antimicrobial stewardship program.

^aWilcoxon rank-sum test.

DISCUSSION

Here, we describe the results of implementation of a new ASP at MCJCHV. The primary strategy of our program is prospective audit, but prior authorization and guideline development have also played important roles.

Prospective audit relies on prescriber acceptance of nonbinding recommendations from the stewardship team. Previous studies of pediatric antimicrobial stewardship have assessed acceptance rates and reported compliance rates that ranged from 78% to 89% [14, 15, 23, 24]. Goldman et al [24] assessed the likelihood of compliance according to antibiotic class, underlying diagnosis, and clinical service line. In the adult literature, compliance has been described as poorer for surgical services than for medical services [25, 26]. Cosgrove et al [26] also found that compliance for interventions aimed at avoiding vancomycin or stopping unnecessary therapy were less successful than other interventions. In sum, the literature on stewardship has provided little guidance on strategies for optimizing compliance with PAWF interventions.

In this study, we evaluated the likelihood of adherence as predicted by recommendation type, provider role, and primary service. Of all recommendation types, teams were least likely to follow a recommendation to consult an ID physician. This finding is counterintuitive, because this recommendation is generally reserved for the most complex cases. It is interesting to note that the recommendation to discontinue an unnecessary antimicrobial was no less likely to be followed than other recommendation types. Recommendations made to a team pharmacist were most likely to be successful, and recommendations made to residents were also more likely to be successful than recommendations made to nurse practitioners.

These results suggest some strategic opportunities for our stewardship team that might apply more widely. A nurse practitioner's ability to effect change within the medical team hierarchy can be limited. Although the stewardship team should always respect this hierarchy, escalation might be helpful in some cases. Team pharmacists in our institution usually review patients and round with their assigned team daily, and prescribers are accustomed to receiving valuable advice regarding medication regimens. A recent qualitative study reported on the value of communicating stewardship recommendations to team pharmacists [27], and our experience provides supporting data. Regarding the type of intervention, a stewardship team should be aware of what types of recommendations are least likely to be followed and have a plan to provide alternative recommendations or escalate the discussion to a higher level as needed.

Our ASP resulted in an immediate and continuing decline in the use of parenteral antimicrobials in all classes. To estimate a more robust counterfactual for antimicrobial consumption, we compared our pre-ASP and post-ASP use trends to those of a large group of children's hospitals via the PHIS database. We confirmed the finding that antimicrobial use has been decreasing across children's hospitals, a trend that has been aided by, but has not depended wholly on, the establishment of formal ASPs [19]. By using these hospitals as a comparator, we found that the program's impact was far greater than would have been expected on the basis of secular trends.

With regard to parenteral antimicrobial costs, we observed a decrease in median pharmacy expenditures of $42 647/month between the pre-ASP and post-ASP eras, despite an increase in inpatient census. When estimating expected costs in the absence of intervention, we could not account for changes in the pricing of antimicrobials over time. Such price changes might include decreases (eg, when a patent expires) and increases (eg, when a brand-name antimicrobial is added to the formulary and its use replaces a generic agent). During this time period, however, meropenem was the only antimicrobial on our formulary to change to generic, and use of this agent was minimal during the entire study period.

There were several limitations to our study. We did not systematically ask providers why they refused to accept stewardship recommendations. The results of previous research have suggested an increase in compliance over time [14, 15], indicating the importance of building relationships to change culture. The outcomes of the adherence assessment would likely vary greatly on the basis of a particular institution's culture and resources, and similar analyses are needed. As with most evaluations of ASPs, we were limited to a quasi-experimental pre-post design. However, use of the PHIS database as a comparator strengthens our conclusions with respect to antimicrobial use. It should be noted that, although the PHIS database has been used for other analyses of antimicrobial use [15, 17], validation of the accuracy of antimicrobial-administration data in this database has not been performed. For cost estimates, we were limited to a simple pre-post design. In addition, the use of very expensive antimicrobials for a small number of patients might drive costs upward disproportionately, even when the use is appropriate. Further limiting the analyses of use and cost was the lack of seasonal uniformity in the pre-ASP and post-ASP periods; the pre-ASP period included a higher proportion of cold-weather months than the post-ASP period. In addition, the end date of the post-ASP period was 6 months later for all antimicrobials than for each subcategory. The use of antimicrobial use as a measurement rests on the assumption that the antimicrobial use that was eliminated was unnecessary. Furthermore, the DOT/1000 unit is the most accepted measure for antimicrobial use, but it cannot detect breadth, toxicity, and cost. We could not assess the effect of the program with respect to patient-specific outcomes.

Our study results elucidate factors associated with adherence to stewardship recommendations. Understanding such factors might help ASPs to maximize efficacy. We also demonstrate the impact of our ASP on antimicrobial use in the context of national trends, which further establishes the value of a formal ASP in the pediatric inpatient setting.

Supplementary Data

Supplementary materials are available at the Journal of the Pediatric Infectious Diseases Society online (http://jpids.oxfordjournals.org). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.