Decreases in Antimicrobial Use Associated With Multihospital Implementation of Electronic Antimicrobial Stewardship Tools

Abstract

Background

Antimicrobial stewards may benefit from comparative data to inform interventions that promote optimal inpatient antimicrobial use.

Methods

Antimicrobial stewards from 8 geographically dispersed Veterans Affairs (VA) inpatient facilities participated in the development of antimicrobial use visualization tools that allowed for comparison to facilities of similar complexity. The visualization tools consisted of an interactive web-based antimicrobial dashboard and, later, a standardized antimicrobial usage report updated at user-selected intervals. Stewards participated in monthly learning collaboratives. The percent change in average monthly antimicrobial use (all antimicrobial agents, anti-methicillin-resistant Staphylococcus aureus [anti-MRSA] agents, and antipseudomonal agents) was analyzed using a pre–post (January 2014–January 2016 vs July 2016–January 2018) design with segmented regression and external comparison with uninvolved control facilities (n = 118).

Results

Intervention sites demonstrated a 2.1% decrease (95% confidence interval [CI], −5.7% to 1.6%) in total antimicrobial use pre–post intervention vs a 2.5% increase (95% CI, 0.8% to 4.1%) in nonintervention sites (absolute difference, 4.6%; P = .025). Anti-MRSA antimicrobial use decreased 11.3% (95% CI, −16.0% to −6.3%) at intervention sites vs a 6.6% decrease (95% CI, −9.1% to −3.9%) at nonintervention sites (absolute difference, 4.7%; P = .092). Antipseudomonal antimicrobial use decreased 3.4% (95% CI, −8.2% to 1.7%) at intervention sites vs a 3.6% increase (95% CI, 0.8% to 6.5%) at nonintervention sites (absolute difference, 7.0%; P = .018).

Conclusions

Comparative data visualization tool use by stewards at 8 VA facilities was associated with significant reductions in overall antimicrobial and antipseudomonal use relative to uninvolved facilities.

Steward participation in implementation and development of comparative antimicrobial use visualization tools and monthly learning collaboratives were associated with reductions in inpatient total and antipseudomonal antimicrobial use at 8 Veterans Affairs (VA) facilities relative to the rest of the VA.

(See the Editorial Commentary by Patel on pages 1177–8.)

Inappropriate antimicrobial prescribing, which accounts for 30%–50% of all use, is a major driver of increased antimicrobial resistance, Clostridioides difficile infection, and other adverse events and unnecessary healthcare costs [1, 2]. Antimicrobial stewardship programs (ASPs) strive to improve antimicrobial use by encouraging evidence-based decisions regarding choice and duration of therapy [3].

Antimicrobial stewards have long lacked the ability to compare their antibiotic usage to either national norms or to comparable facilities. In this regard, the development of standardized antimicrobial administration ratios (SAARs) within the antimicrobial use (AU) option of the National Health Safety Network (NHSN) by the Centers for Disease Control and Prevention (CDC) has been a major advance. These reports provide facility-level measures of days of therapy per 1000 patient-days present (DOT/1000 DP) and utilize indirect standardization techniques to represent antimicrobial use data as observed to expected ratios [4]. However, the NHSN reports do not provide bases on which an institution may compare its antimicrobial use to similar facilities nor demarcate antimicrobial use according to specific diagnoses or across the temporal course of therapy from initiation of empiric therapy through deescalation and subsequent discharge.

To address this information gap, we extended previous projects [5, 6] that extracted inpatient antimicrobial use data from the Veterans Affairs’ (VA’s) Corporate Data Warehouse to develop a suite of interactive graphic tools that provide stewards with in-depth facility-level reports of antibiotic use. Antimicrobial use at the dashboard user’s (eg, steward’s) facility can be compared to all VA facilities or user-selected facilities of similar complexity levels, with plots of the system-wide variability of antimicrobial use. We pilot-tested the usability of these graphic tools and assessed their impact on 3 important antimicrobial use metrics at 8 VA healthcare facilities.

METHODS

Electronic Antimicrobial Graphic Tool Development

Specifying Targeted Infectious Diseases and Creating a Framework for Inpatient Antimicrobial Time Course

We initially constructed antimicrobial use displays according to 2 dimensions: disease and time frame within hospitalization. For the disease dimension, we focused on 3 common conditions: Pneumonia, Urinary tract infection, and Skin/soft tissue infection (PUS). Diagnoses were determined using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) [7] and cross-mapped ICD-10-CM codes [8] for each infectious process at hospital admission and discharge, as identified by a combination of those identified previously in the literature [9] and those identified by finding the descendants of all infections identified in the Systematized Nomenclature of Medicine–Clinical Terms [10].

Antimicrobial prescribing for each PUS diagnosis was classified within a time-based framework that corresponded to important branch points in antimicrobial decision-making that we termed Choice, Change, and Completion (CCC). Choice, the time during which decision-making centers around initial choice of empiric therapy, corresponded to the day of admission (day zero) followed by the next 2 calendar days of hospitalization (days 1 and 2). Change, the time in which antimicrobial therapy can be changed (deescalated) based on microbiologic and other clinical data [11], corresponded to days 3 and 4 of hospitalization. Completion, the time in which antibiotic selection is finalized and length of therapy is determined, corresponded to days 5 and 6 of hospitalization. Admission diagnoses were used to define PUS conditions to be included in Choice, while discharge diagnoses were used to define PUS conditions in Change and Completion. An additional measure termed “duration of total antimicrobial therapy” (DAT) included the entire course (inpatient and outpatient) of antimicrobial treatment if the PUS diagnosis was assigned at admission and discharge; this included the duration of inpatient therapy as well as the days supplied upon discharge. Validation of data capture for the CCC–PUS framework was conducted via chart review at 3 of the intervention sites. At each site, cases for 1 month in which a PUS diagnosis was identified were reviewed to ensure appropriate capture of antimicrobial therapy in each CCC category. This validation uncovered occasional discrepancies that were clarified and refined in our coding.

Interactive Antimicrobial Graphic Tool Development, Implementation, and Evolution

The first iteration of the antimicrobial graphic tools consisted of a web-based dashboard with 3 interactive modules that showed overall trends in antimicrobial DOT/1000 DP comparison of a single facility’s SAARs to other facilities and proportion of patients receiving specific antibiotics at each of the CCC intervals for PUS diagnoses. Stewards had the ability to track their facility’s antimicrobial use (overall, by class of drug, by SAAR category, by individual agent) according to month, quarter, or year stratified by ward type (medical/surgical ward vs intensive care unit). The second module showed the facility’s SAARs on a bar graph compared to other VA facilities that were sharing NHSN AU data, stratified by VA facility complexity [12]. The final module allowed stewards to see the frequency of their facility’s use of any individual antimicrobial agent compared to all other VA facilities on the CCC spectrum for PUS diagnoses in a box-and-whiskers plot, with the ability to stratify according to VA facility complexity and ward type (Figure 1).

Figure 1.

Example of the Choice/Change/Completion box-and-whisker plots. Sample facility’s medical–surgical ward usage of piperacillin-tazobactam for pneumonia is denoted by small square; dotted line represents 50th percentile for all facilities compared; box represents 25th–75th percentile; whiskers represent 5th–95th percentile.

The initial antimicrobial dashboards were implemented between February 2016 and June 2016 at 8 VA facilities recruited by the investigators. Implementation at each site included a visit from study team members, with a kickoff lecture to medical staff in order to promote antimicrobial stewardship. We sought to involve at least 1 physician and 1 pharmacist steward at each site. Stewards were given data-viewing privileges specific to their institution prior to the site visit and were provided with additional instructions during the visit, including how the data could be used to prioritize development of new stewardship interventions.

All 8 sites subsequently underwent qualitative usability assessments of stewards’ interactions with the antimicrobial dashboards via semistructured interviews. Interviews focused on 4 areas: the overall approach to stewardship of each ASP as well as types of stewardship activities, a description of a specific experience using the antimicrobial dashboards, user’s perceived self-efficacy and knowledge regarding the concepts of CCC, and user’s perceptions of usefulness and usability of the dashboards [13].

We held monthly learning collaborative calls with stewards and solicited feedback on how to improve the usability and interpretability of dashboard outputs. Stewards also shared “lessons learned” regarding effective use of the information gleaned from the antimicrobial use displays.

Synthesizing feedback from qualitative interviews and monthly collaboratives, we added several features to visual displays of antimicrobial trends, including facility-specific administration of the following antimicrobial groups: the 5 most commonly prescribed agents at the facility, broad gram-negative rod (GNR), antistaphylococcal, antipseudomonal beta-lactams and anti-methicillin-resistant Staphylococcus aureus (MRSA), fluoroquinolone, and antifungal therapies. Each display combined a line/bar graph of the facility’s quarterly antimicrobial use over a 5-year period on the right side of the screen (with the ability to select any combination of intensive care unit [ICU], medical/surgical ward, and/or community living center (ie, VA nursing home) and a comparator graph on the left that displays aggregate VA-wide usage by selected facility complexity level (Figures 2A–C). Other dashboard tabs allow stewards to compare their facility’s antimicrobial use by SAAR category (Figure 2D), CCC, and DAT for PUS conditions to all high-complexity VA facilities. Furthermore, the Pyramid Analytics (Kirkland, WA) platform allows for exportation of data into Microsoft Excel and graphs into figures that could be downloaded by stewards for presentation or local manipulation.

Figure 2.

Interactive antimicrobial use dashboard examples. A, Overview/overall antimicrobial use (intensive care unit [ICU]). B, Top 5 most utilized agents (ICU). C, Pseudomonal beta-lactams and anti-methicillin-resistant Staphylococcus aureus therapy. D, Facility variation across standardized antimicrobial administration ratio (SAAR) antibiotic groups. The yellow bars represent overall antimicrobial use (corresponding to the scale on the right of each figure). The lines represent antimicrobial use in each Centers for Disease Control and Prevention SAAR group (corresponding to the scale on the left of each figure). Abbreviations: Abx, antibiotic; GNR, gram-negative rod; IM, intramuscular; IV, intravenous; MDRO, multi-drug resistant organism; VA, Veterans Affairs.

Standardized Antimicrobial Use Report Development

In 2017, the interactive platform was supplemented by the development of preprogrammed static reports of antimicrobial use that retained many of the interactive dashboards’ data comparisons. However, the preprogrammed version allowed for updated reports to be automatically sent to stewards at user-defined intervals. When stewards signed up to receive the report, they had the ability to choose the complexity level of facilities to which their site would be compared and the frequency with which and to whom the report is emailed (sample report in the Supplementary Materials).

Analysis of Program Impact on Antimicrobial Use

While stewards were free to choose local interventions to address their most pertinent antimicrobial usage issues, our analysis focused on 3 metrics that we hypothesized would be most affected by stewards’ use of graphic displays: total inpatient use of all antimicrobials, anti-MRSA agents (ceftaroline, dalbavancin, daptomycin, linezolid, oritavancin, quinupristin-dalfopristin, tedizolid, telavancin, intravenous vancomycin), and antipseudomonal agents (amikacin, aztreonam, cefepime, ceftazidime, doripenem, gentamicin, imipenem-cilastatin, meropenem, piperacillin-tazobactam, tobramycin).

For these metrics, antimicrobial usage was calculated per DOT/1000 DP. Change in antimicrobial use over time was assessed with interrupted time series analysis preintervention (January 2014 through January 2016) and postintervention (July 2016 through January 2018), allowing for the 5-month implementation phase in between segments. We used generalized estimation equations with Poisson distribution to estimate percent difference in average monthly antimicrobial use rate between segments as a function of the intervention phase and intervention site indicator. Comparisons across facilities were conducted by aggregating data from the 8 facilities and then analyzing these in relation to aggregated use across all other VA facilities that provide acute care services at an assigned complexity level that had available antimicrobial use data (n = 118).

RESULTS

Steward Insights and Utility Gained From the Program

During monthly collaborative calls, we asked stewards what specific insights and interventions were derived from interrogating the interactive graphic tools (Table 1). Multiple sites focused on high utilization of anti-MRSA and antipseudomonal agents, especially during the Choice treatment phase, and on ICU and surgical wards, prompting consideration of procalcitonin testing and timeout/reminder programs to encourage deescalation. Fluoroquinolone usage and duration of therapy were other themes. One site noted relatively high fluoroquinolone use and durations of therapy that prompted development of order sets to deemphasize fluoroquinolones and creation of urinary antibiograms to assist with nonfluoroquinolone selection for UTI; a follow-up medication use evaluation at that site noted the success of this intervention. Another site that had already transitioned much of its fluoroquinolone and antipseudomonal use to ceftriaxone used the tools to identify opportunities to deescalate to narrower beta-lactams. Stewards also reported using different data reports in informal interactions with stakeholders (hospitalists, intensivists, surgeons, pharmacists, medical trainees), teaching conferences, subspecialty meetings, and committees within their facility (eg, pharmacy and therapeutics, infection control, clinical executive boards). Throughout the postintervention period when outcomes were assessed, participation in the monthly collaborative calls was 83% across all sites (range, 65%–100%). Pharmacists were the primary participants from 3 sites; physicians were primary participants from 2 sites; and pharmacists and physicians participated equally from 3 sites.

Table 1.

Examples of Areas for Potential Improvement Identified by Stewards and Interventions Considered or Developed

Area for Improvement	Intervention
High utilization of anti-methicillin-resistant Staphylococcus aureus and antipseudomonal agents at Choice	Introduction of serial procalcitonin testing for patients with suspected sepsis or lower respiratory tract infection
	Timeout program to encourage deescalation
High fluoroquinolone utilization	Creation and evaluation of treatment pathways and order sets that deemphasize fluoroquinolone use
	Creation of urinary antibiogram to assist in selection of nonfluoroquinolone options
Antipseudomonal agent utilization in SSTI	Pilot program in which providers who use antipseudomonal agents for SSTI are emailed reminders as to the proper indications for their use in SSTI
Excessive duration of therapy	Development of syndrome-specific treatment pathways

Abbreviation: SSTI, skin/soft tissue infection.

Changes in Antimicrobial Usage at Program Sites vs the Rest of the VA

Intervention sites included 7 highly complex sites and 1 less complex site and had a median inpatient bed size of 151 (range, 37–324), with a median ICU census of 14.4 (range, 3.6–24.9) and medical–surgical bed size of 119.5 (range, 37–246). The VA as a whole had a median inpatient bed size of 84 (range, 4–367), with a median ICU census of 8.9 (range, 1.0–26.9) and medical–surgical bed size of 76.5 (range, 4–246). Average monthly antimicrobial use at intervention and nonintervention VA facilities pre and postintervention are shown in Figure 3 with changes summarized in Table 2. Intervention sites averaged a 2.1% decrease (95% confidence interval [CI], −5.7% to 1.6%; P = .2529) in total antimicrobial use, while nonintervention sites averaged a 2.5% increase (95% CI, 0.8% to 4.1%; P = .0026) in use pre vs postintervention. The 4.6% absolute difference in change between intervention and nonintervention sites was statistically significant (P = .025).

Figure 3.

Monthly trends in antimicrobial use at intervention vs control sites. Abbreviations: MDRO, multi-drug resistant organism; MRSA, methicillin-resistant Staphylococcus aureus.

Table 2.

Changes in Average Monthly Antimicrobial Use (Days of Therapy per 1000 Patient-days Present) at Intervention and Nonintervention Veterans Affairs Facilities Pre and Postintervention

	Intervention Sites (n = 8)						Nonintervention Sites (n = 118)
Antimicrobial Use	Pre	95% CI	Post	95% CI	% Change	P Value	Pre	95% CI	Post	95% CI	% Change	P Value	% Difference	P Value
Total	533	474–599	522	471–578	−2.1	.25	548	526–572	562	541–583	+2.5	.0026	−4.6	.025
Anti-methicillin-resistant Staphylococcus aureus	102	79–132	91	71–97	−11.3	<.0001	105	97–113	98	91–105	−6.6	<.0001	−5.2	.092
Antipseudomonal	117	96–143	113	92–139	−3.4	.185	133	123–144	138	128–149	+3.6	.011	−7.0	.018

Abbreviation: CI, confidence interval.

With regard to anti-MRSA antimicrobial use, intervention sites had an average 11.3% (95% CI, −16.0% to −6.3%; P < .0001) decrease and nonintervention sites had an average 6.6% decrease (95% CI, −9.1% to −3.9%; P < .0001) in anti-MRSA antimicrobial use pre vs postintervention; the 4.7% change between the intervention and nonintervention sites showed only a statistical trend for significance (P = .092).

Finally, intervention sites had an average 3.4% (95% CI, −8.2% to 1.7%; P = .185) decrease in antipseudomonal antimicrobial use, while nonintervention sites had an average 3.6% increase (95% CI, 0.8% to 6.5%; P = .011); the 7.0% change between nonintervention and intervention sites was statistically significant (P = .018). We also performed a sensitivity analysis in which we excluded 45 sites of lower complexity from our controls and found nearly identical findings across all 3 outcomes (data not shown).

Variation in Changes in Antimicrobial Use Among Intervention Sites

The variation in changes in total, anti-MRSA, and antipseudomonal use according to individual intervention sites is shown in Figure 4. Changes were largely consistent across sites; however, sites C and H did not observe as consistent reductions in antimicrobial use as the others. Notably, site C had the lowest participation in the monthly collaboratives (65%), and site H experienced the sudden loss of its stewardship pharmacist early in the intervention period.

Figure 4.

Variation among intervention sites in changes in antimicrobial use outcomes. Abbreviations: MRSA, methicillin-resistant Staphylococcus aureus.

DISCUSSION

We developed and deployed interactive and standardized graphic tools at 8 VA sites, allowing stewards to assess facility-level antimicrobial use overall, by drug class, for specific disease conditions, and over the course of therapy. These tools illustrated temporal trends in use and provided detailed comparisons with other similar VA facilities. Despite not proscriptively requiring stewards to focus on specific antimicrobial use policies, we found reductions in overall antimicrobial and antipseudomonal use relative to uninvolved facilities as well as a large absolute decrease in anti-MRSA antimicrobial use.

While we cannot directly attribute the decreases in antimicrobial use to stewards’ use of the antimicrobial graphic tools, we hypothesize that the overall implementation strategy and follow-up served to activate stewards to pursue interventions that focused on the particular needs of their facilities. The inability to easily get data regarding local antimicrobial use patterns has long been recognized as a barrier to effective stewardship [14–16], and providing meaningful standardized metrics to individual facilities across a healthcare system can be challenging [17].

While we did not show a significant decrease in anti-MRSA antibiotic use relative to other sites, anti-MRSA antibiotic use decreased throughout the VA over the time frame of the study compared to overall and antipseudomonal use, despite VA Antimicrobial Stewardship Task Force interventions that targeted both anti-MRSA and antipseudomonal use [18]. We noticed in a prior project that it was easier to show sustained decreases in vancomycin use with a timeout intervention than for piperacillin-tazobactam [19]. It may very well be that, from a stewardship perspective, it is easier to operationalize discontinuation of anti-MRSA therapy (particularly in the VA where there is nasal colonization data that can help guide discontinuation) than antipseudomonal therapy.

Recent efforts to provide antimicrobial use data have focused on raw facility-specific numbers from the AU Option and standardized benchmarking data via SAAR and other observed-to-expected metrics [20] but do not provide comparative data showing interfacility variability of antimicrobial use that may be important in “nudging” stewards to devise interventions targeted to their facility [21]. Rather than applying a “one-size-fits-all” approach to providing actionable metrics for ASPs, we allowed for customization of data receipt and presentation that can support the distinct local needs of any facility.

The most novel aspect of this work was the creation of the CCC framework as a tool to determine where in the typical timeline of treatment a facility may be overly broad in its antimicrobial prescribing patterns. This framework thematically resembles the 4 moments of antibiotic decision-making recently described by Tamma et al: “Does this patient have an infection that requires antibiotics?,” “Have I ordered appropriate cultures before starting antibiotics?,” “A day or more has passed. Can I stop antibiotics?,” and “What duration of antibiotic therapy is needed for this patient’s diagnosis?”[22]. Here, though, we apply a population-based quantitative determination of antimicrobial usage within discrete time frames in which decision-making evolves to allow comparison of these decision points in aggregate across facilities as well as within a facility over time.

We also capture the total duration of antimicrobial therapy prescribed for common infectious syndromes, including antibiotics prescribed at hospital discharge. The postdischarge course may be particularly ripe for antimicrobial stewardship interventions, as highlighted by a recent study of an antimicrobial stewardship intervention to reduce inappropriate fluoroquinolone prescription in 48 Michigan hospitals in which significant reductions in inpatient fluoroquinolone use were offset by twice as many new fluoroquinolone starts after discharge [23].

Limitations of our work include the relatively small number of nonrandomly selected sites involved and the bundling of visual tools with the learning collaborative that does not allow for analysis of the effect of each individual component of the intervention. We also utilized a relatively simple statistical analysis of pre and postintervention antimicrobial utilization in which residual confounding may not have been fully captured. Furthermore, the exact relationship between the amount of antimicrobial use and quality of infectious diseases management is unknown. However, results from multiple VA analyses of antimicrobial utilization for pneumonia and other common infectious conditions indicate that opportunities to reduce excessive antimicrobial use within the VA system remain ample [24–27]. In addition, our CCC paradigm only captures infections present at admission. Antimicrobial utilization for infections acquired after hospital admission are not captured, and their treatment may interfere with our ability to measure antimicrobial use for infections present at hospital admission if the antimicrobial course for the hospital-onset infection overlaps with that of the admission infection.

In summary, while we were able to show temporal improvements in antimicrobial utilization in concert with our intervention, more research is needed on how visual graphics of population-level data can be used to influence prescribing patterns at a systems level. At minimum, our work also lends credence to the role that peer comparison can play in influencing prescribing changes on a facility level (in addition to what has been demonstrated for individual providers [28]). More broadly, we demonstrate the potential value to VA and other large healthcare delivery organizations of providing stewards with robust data on their facility’s antimicrobial utilization. Finally, we hope that the CCC framework we developed in this work can become a useful tool for antimicrobial stewardship clinical and research communities interested in defining opportunities for improved prescribing across the time course of inpatient hospitalization.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Acknowledgments. The authors thank the antimicrobial stewards at the Veterans Affairs (VA) Greater Los Angeles Healthcare System (Maximilian Jahng, PharmD (deceased); Phuong Nguyen, PharmD), the Boise VA Medical Center (Sarah McClain, PharmD; Jefferson Bohan, PharmD; Benjamin Pontefract, PharmD), the VA Salt Lake City Healthcare System (Jesse Sutton, PharmD; Emily Spivak, MD), the Cincinnati VA Medical Center (Victoria Tate, PharmD; Allison Kelly, MD), the William S. Middleton Memorial Veterans Hospital (Paul Lata, PharmD; Christopher Crnich, MD, PhD), the VA Boston Healthcare System (Donald Smith, PharmD; Judith Strymish, MD), the South Texas Veterans Health Care System (Kelly Echevarria, PharmD; Linda Yang, PharmD; Teri Hopkins, PharmD), and the Michael E. DeBakey VA Medical Center (Andrew Hunter, PharmD; Maria Rodriguez-Barradas, MD) for their participation in this project and Michael Fletcher for his administrative support.

Financial support. This work was supported by the VA Health Services Research and Development Service Collaborative Research to Enhance and Advance Transformation and Excellence Initiative, Cognitive Support Informatics for Antimicrobial Stewardship project (CRE 12–313).

Potential conflicts of interest. The authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest.

Presented in part: IDWeek 2016, New Orleans, LA, 26-30 October 2016 (abstract 963) and IDWeek 2017, San Diego, CA, 4-8 October 2017 (abstract 1633).