Improved Antibiotic Prescribing within a Veterans Affairs Primary Care System through a Multifaceted Intervention Centered on Peer Comparison of Overall Antibiotic Prescribing Rates

Reducing inappropriate outpatient antibiotic use is an important national goal. Limited data exist on targeted education and peer comparison of overall antibiotic prescribing rates as an antimicrobial stewardship strategy. Primary care professionals (PCPs) from all seven clinics within our health care system were offered an education session, followed by monthly e-mails with their antibiotic prescribing rate, peer prescribing rates, and a system target.

ABSTRACT

Reducing inappropriate outpatient antibiotic use is an important national goal. Limited data exist on targeted education and peer comparison of overall antibiotic prescribing rates as an antimicrobial stewardship strategy. Primary care professionals (PCPs) from all seven clinics within our health care system were offered an education session, followed by monthly e-mails with their antibiotic prescribing rate, peer prescribing rates, and a system target. A pre-post analysis was conducted to compare prescribing rates during the intervention period (January to June 2017) to a seasonal baseline (January to June 2016) using a regression model. A random sample of prescriptions was reviewed for adherence to consensus guidelines. Educational sessions were attended by 68.5% (50/73) of PCPs. From the baseline to the intervention period, the mean rate of monthly antibiotic prescriptions declined from 76.9 to 49.5 per 1,000 office visits (35.6% reduction [P < 0.001]). Among reviewed cases, unnecessary antibiotic prescribing declined (58.8% [80/136] versus 38.9% [70/180]; 33.9% reduction [P = 0.0006]), and the rate of optimally prescribed antibiotics increased (19.9% [27/136] versus 30% [54/180]; 50.8% increase [P = 0.05]). If an antibiotic was indicated, there were no significant differences in prescribing of guideline-discordant agents (21.4% [12/56] versus 19.1% [21/110] [P = 0.8]) or guideline-concordant agents for a guideline-discordant duration (38.6% [17/44] versus 39.3% [35/89] [P = 1]). There were significant reductions in azithromycin and fluoroquinolone prescriptions (50.9% and 59.4% [P values of <0.001], respectively), but most prescriptions for these agents in the intervention period remained inappropriate. Initial education followed by monthly peer comparison of overall antibiotic prescribing rates reduced total and unnecessary antibiotic prescribing in primary care clinics.

INTRODUCTION

Due to rising antimicrobial resistance and adverse events associated with inappropriate antibiotic use, guidelines recommend (1) and regulatory agencies require (2; https://www.cms.gov/newsroom/fact-sheets/omnibus-burden-reduction-conditions-participation-final-rule-cms-3346-f) that all acute care hospitals and nursing care centers have an antimicrobial stewardship program (ASP) in place. However, the majority of antibiotic usage is in the outpatient setting (3, 4). It is conservatively estimated that at least 30% of antibiotic prescribing nationally at ambulatory care visits is not indicated (5, 6). The rate of inappropriate usage is higher when agent selection and duration of therapy are also considered (7, 8). We reported that almost 50% of antibiotics prescribed in primary care clinics at our VA health care system were not indicated, and an additional 26% of prescriptions were for antibiotics or treatment durations that were inconsistent with guideline recommendations (7). Given findings such as these, increased attention is being given to outpatient antimicrobial stewardship. The National Action Plan for Combating Antibiotic-Resistant Bacteria set a goal of a 50% reduction in inappropriate prescribing by 2020 (9). The U.S. Centers for Disease Control and Prevention (CDC) has reported core elements of outpatient antimicrobial stewardship (10). Most recently, The Joint Commission announced antimicrobial stewardship requirements for ambulatory health care, effective 1 January 2020, which include identifying a leader and taking actions to support an annual goal (11). Nevertheless, more data are needed to better inform outpatient stewardship efforts.

Various models of implementing outpatient antimicrobial stewardship have been evaluated, including clinician education (12, 13), electronic clinical decision support (14), audit and feedback (15–18), and peer comparison (19–21), among others (22–24). To date, in the adult patient population, data exist predominantly for the effect of interventions on acute respiratory tract infections (ARTIs). Stewardship strategies that target feedback to a limited number of conditions and utilize diagnosis codes to identify conditions for which feedback is given can be effective but may fail to capture considerable amounts of antibiotic prescribing (7, 25). Giving regular personal feedback to clinicians on the appropriateness of antibiotic prescribing across all outpatient conditions for which antibiotics are prescribed requires considerable effort and resources. Providing feedback and peer comparison of the overall volume of a clinician’s antibiotic prescribing, without specifically addressing the appropriateness of treatment, is a less resource-intensive approach that may be attractive to programs initiating outpatient stewardship. Data for interventions of this type are limited; large-scale trials have yielded mixed results (26–28).

An ideal outpatient ASP would have a low barrier to implementation, use easily gathered data, decrease inappropriate antibiotic use, and increase guideline concordance for treating common infectious conditions. Our objective was to reduce overall and unnecessary antibiotic prescribing in our VA primary care clinics through initial education, e-mail-based peer comparison of clinicians’ prescription rates, and, after 2 months, the introduction of electronic order sets that incorporate prescribing recommendations.

RESULTS

During the baseline period of January to June 2016, there were 65 primary care professionals (PCPs) caring for 40,734 patients in the VA Pittsburgh health care system (VAPHS). During the intervention period of January to June 2017, there were 73 PCPs caring for 41,191 patients. There were 28,402 office visits during the baseline period and 32,982 office visits during the intervention period. Educational sessions conducted by an infectious diseases (ID) physician (N. R. Shively) on outpatient antibiotic overuse and treatment guidelines for common infections encountered in ambulatory care settings were attended by 68.5% (50/73) of PCPs.

The mean monthly antibiotic prescription rate declined from 76.9 to 49.5 prescriptions per 1,000 office visits from the baseline period to the intervention period (35.6% reduction [P < 0.001]). As a percentage of all prescriptions, the three most commonly prescribed agents during the baseline and intervention periods were azithromycin (25.8% versus 19.6%), amoxicillin-clavulanate (14.1% versus 17.3%), and doxycycline (12.2% versus 17.7%). Significant reductions in prescribing of amoxicillin, amoxicillin-clavulanate, azithromycin, cephalexin, fluoroquinolones, and trimethoprim-sulfamethoxazole were observed, with the largest reductions observed for fluoroquinolones (59.4% reduction [P < 0.0001]) and azithromycin (50.9% reduction [P < 0.001]) (Fig. 1).

FIG 1.

Antibiotic prescriptions per 1,000 patient visits in the baseline period versus the intervention period for the most commonly prescribed antibiotics. The top 5 “other” antibiotics include clindamycin, nitrofurantoin, cefuroxime, penicillin, and erythromycin. NS, not significant.

Among randomly reviewed cases, the most commonly encountered diseases for which an antibiotic was prescribed during both periods were ARTIs (36.8% at baseline versus 35% during the intervention period), skin and soft tissue infections (SSTIs) (14.7% versus 15%), and urinary tract infections (UTIs) (14.7% versus 13.9%). Unnecessary antibiotic prescribing (defined as the use of an antibiotic when treatment was not indicated) declined from 58.8% (80/136) during the baseline period to 38.9% (70/180) during the intervention period (33.9% reduction [P = 0.0006]). A significant reduction in unnecessary antibiotic prescribing was observed for SSTIs, with nonsignificant reductions in chronic obstructive pulmonary disease (COPD) exacerbations, UTIs, and ARTIs (Table 1).

TABLE 1.

Reductions in unnecessary antibiotic prescriptions for the most common conditions, among randomly reviewed cases

Condition	Baseline		Intervention		P value
	No. of antibiotic prescriptions	No. (%) of unnecessary prescriptions	No. of antibiotic prescriptions	No. (%) of unnecessary prescriptions
Acute respiratory tract infection	50	40 (80.0)	63	43 (68.3)	0.2
COPD exacerbation	10	7 (70.0)	10	2 (20.0)	0.07
Skin and soft tissue infection	20	9 (45.0)	27	1 (3.7)	0.0009
Urinary tract infection	20	6 (30.0)	25	3 (12.0)	0.157
Other	36	18 (50.0)	55	21 (38.2)	0.287
Total	136	80 (58.8)	180	70 (38.9)	0.0006

Among cases for which an antibiotic was indicated, no significant differences were observed in prescribing of guideline-discordant agents (21.4% [12/56] versus 19.1% [21/110] [P = 0.8]) or guideline-concordant agents given for a guideline-discordant duration (38.6% [17/44] versus 39.3% [35/89] [P = 1]). Numerical reductions in inappropriate prescribing were seen for most reviewed agents, although azithromycin and ciprofloxacin were frequently prescribed inappropriately in both periods (Table 2). Total antibiotic prescribing and prescribing appropriateness by month during the baseline and intervention periods are displayed in Fig. 2. Optimally prescribed antibiotics (defined as treatment that was both indicated and guideline concordant for the agent and duration) increased from 19.9% (27/136) during the baseline period to 30% (54/180) during the intervention period (50.8% increase [P = 0.05]).

TABLE 2.

Inappropriate prescriptions, by antibiotic^a

Antibiotic	No. (%) of prescriptions				P value
	Baseline		Intervention
	Reviewed	Inappropriate	Reviewed	Inappropriate
Amoxicillin	14	10 (71.4)	19	10 (52.6)	0.31
Amoxicillin-clavulanate	21	14 (66.7)	23	9 (39.1)	0.08
Azithromycin	44	36 (81.8)	32	23 (71.9)	0.4
Cephalexin	8	5 (62.5)	8	2 (25.0)	0.31
Ciprofloxacin	8	7 (87.5)	10	9 (90.0)	1
Doxycycline	15	7 (46.7)	24	10 (41.7)	1
Nitrofurantoin	3	0 (0.0)	4	2 (50.0)	0.43
Trimethoprim-sulfamethoxazole	8	4 (50.0)	7	1 (14.3)	0.28

^aFor this analysis, an antibiotic was considered inappropriate if there was no indication for its usage or if the agent chosen was guideline discordant.

FIG 2.

Antibiotic prescriptions per 1,000 patient visits and percentages of reviewed prescriptions that were unnecessary during the baseline and intervention periods. From the baseline to the intervention period, the mean monthly antibiotic prescription rate declined from 76.9 to 49.5 prescriptions per 1,000 office visits (35.6% reduction [P < 0.001]). Among reviewed cases, unnecessary antibiotic prescribing declined from 58.8% during the baseline period to 38.9% during the intervention period (33.9% reduction [P = 0.0006]). Reviewed prescriptions are a subset of total prescriptions.

In a safety analysis, we reviewed cases of patients who received antibiotics during an admission or emergency department (ED) visit within 30 days of a PCP visit during which no antibiotic was prescribed. There was no difference between baseline and intervention periods in the likelihood that an antibiotic prescribed at the PCP visit would have prevented the subsequent ED visit or admission (Table 3).

TABLE 3.

Likelihood that an antibiotic prescribed at a PCP visit would have prevented a subsequent ED visit or hospital admission^a

Likelihood category	No. (%) of cases		P value
	Baseline	Intervention
Unlikely	57 (89.1)	36 (90.0)	1
Uncertain	2 (3.1)	4 (10.0)	0.2
Might have been helpful	5 (7.8)	0 (0.0)	0.15
Total cases reviewed	64	40

^aCases reviewed were a random subset of patients who received an oral or i.v. antibiotic during an ED visit or admission and had a PCP visit within the prior 30 days at which no antibiotic was prescribed.

DISCUSSION

We demonstrate that a stewardship intervention comprised of clinician education, peer comparison, and electronic order sets significantly improved antibiotic prescribing over a 6-month period in primary care clinics at our VA health care system, including 36% and 34% reductions in overall and unnecessary antibiotic use, respectively, and a 51% increase in optimally prescribed antibiotics. These improvements occurred without evidence of harm to patients. To our knowledge, this is the first study to show that a stewardship strategy built upon feedback to PCPs about their overall use of outpatient antibiotics, without regard to the appropriateness of treatment, is effective in diminishing unnecessary antibiotic usage and optimizing treatment within a health care system. Our approach had a low barrier to implementation and required a relatively small commitment of resources once education was complete and processes were in place. Even though data gathering and e-mail generation were done manually during the intervention period, the endeavor required only ∼4 to 6 h of effort per month. More robust information technology support to automate monthly data collection, analysis, and feedback report generation could reduce this time commitment. Therefore, the intervention described here is an efficient model for health care systems looking to initiate antimicrobial stewardship in primary care settings.

Data on the impact of peer comparisons of overall antibiotic prescribing, rather than the appropriateness of prescriptions for specific conditions, are limited to several national studies. In a randomized controlled trial in the United Kingdom, general practitioners with antibiotic prescribing rates in the top 20% nationally received a feedback letter and a copy of a patient-focused leaflet that enabled backup (delayed) antibiotic prescribing (26). A significant reduction in dispensed antibiotics by 3.3% over 6 months was observed in the feedback group. In contrast, similar randomized controlled trials in Switzerland and Australia did not report significant changes in antibiotic use (27, 28), although the latter study also included drugs other than antibiotics. Several factors may account for differences in the outcomes of these trials and our study. First, our initiative began with in-person educational sessions, which the majority of PCPs attended. Second, feedback e-mails came directly from ID physicians responsible for our system’s ASP, who were known to most PCPs. Third, PCPs were compared to local colleagues. E-mail feedback was deidentified, but it is possible that reputational concerns may have stimulated behavior modification to a greater extent than what was observed with less personalized, national approaches. E-mail may have also led to more engagement than traditional mail. Literature in other fields suggests that peer comparison effectiveness is improved when parties share similar circumstances (29) or a peer network (30), and compliance is increased by proximity to an authority figure (31). Finally, the introduction of electronic order sets after the second month of the intervention may have helped sustain success, although the magnitude of such an effect (if any) is not clear since reductions in overall and unnecessary prescriptions were evident prior to order set introduction.

An increasing body of evidence demonstrates that behavioral interventions can improve patient care and support outpatient antimicrobial stewardship (15–17, 19, 20, 32). Recently, Meeker and colleagues conducted a cluster-randomized trial of 47 primary care practices in Boston, MA, and Los Angeles, CA (19). Two socially motivated behavioral interventions, accountable justification, in which clinicians were required to document the justification for antibiotic prescribing in the electronic health record, and peer comparison, in which clinicians received e-mails comparing their antibiotic prescribing rates to those of “top performers,” led to significantly lower rates of inappropriate antibiotic prescribing for ARTIs. The peer comparison intervention was the only one that led to a durable reduction in antibiotic prescribing, relative to the control, 12 months after the interventions were stopped (21). Limitations of the peer comparison approach utilized in this study were that it focused only on ARTIs (19) and relied upon diagnosis codes in order to define appropriateness. Only 44% of outpatient antibiotic prescriptions (and only 22% for adults ≥65 years old) are for acute respiratory conditions (5). It is unknown whether stewardship interventions aimed at ARTIs may also impact antibiotic prescribing for nonrespiratory conditions, but a broader focus may be necessary to impact total antibiotic prescribing for all conditions. Our group and others have shown that a significant number of antibiotic prescriptions are not associated with a diagnosis code for an infectious disease (7, 25). Therefore, studies based on diagnosis codes fail to account for a sizeable amount of antibiotic usage.

Despite our success in reducing overall and unnecessary antibiotic prescribing and increasing optimized treatment, it is important to acknowledge that antibiotic prescribing remained unnecessary or not optimized in 39% and 70% of instances, respectively. Moreover, there were no improvements in the selection of guideline-recommended agents or treatment durations among patients who received antibiotics. The impact of our intervention may have been underestimated, as our study design for evaluating the appropriateness of antibiotic prescriptions that were written did not allow us to evaluate the appropriate nonprescription of antibiotics. Nevertheless, our experience highlights several areas for future improvement. A priority will be improving antibiotic choice and duration of treatment, as these considerations were major contributors to suboptimal prescribing. Our failure to impact this decision-making may be due to our monthly feedback focus on the overall prescribing rate, without continued reinforcement of agent selection and duration other than in the attached recommendations. Continued use or refinement of electronic order sets may help in this regard. Another obvious area for improvement is in the use of azithromycin and fluoroquinolones. Prescriptions for these agents were significantly reduced by 51% and 59%, respectively, but the vast majority of usage remained inappropriate. Since azithromycin and fluoroquinolones are rarely indicated as agents of choice in outpatient settings, interventions targeted toward these drugs could carry a significant impact.

This study describes a nonrandomized, pre-post analysis of an intervention in a single VA health care system. Our results need to be validated in other settings. Without randomization or a control group, we cannot exclude that the Hawthorne effect or temporal or secular trends contributed to our findings, although we attempted to account for such trends in our regression model. Our approach of not providing direct feedback on the appropriateness of antibiotic prescribing had the advantage of reducing the amount of work needed to establish and maintain our intervention. However, it may have limited our ability to improve compliance with guidelines on antibiotic selection and duration of treatment. We also cannot exclude that the particular characteristics of a clinician’s patient panel, case mix, or practice setting may have justified high rates of antibiotic prescribing compared to other clinicians. Differences in case mix between the VA patient population and other settings could also limit generalizability. Future studies could tailor antibiotic prescribing targets to the characteristics of a given patient panel. As our intervention was multifaceted, we cannot quantify the extent to which each component individually contributed to our results. Education and feedback in our study were delivered by ID physicians, but future projects should investigate if other professionals, such as ID pharmacists, may be similarly effective. Finally, as the present study was not designed to capture antibiotic prescriptions written outside our VA health care system, we were unable to evaluate how this intervention may have influenced patients receiving antibiotic prescriptions from sources other than their VA PCPs.

In conclusion, we show that an easy-to-implement, non-labor-intensive outpatient stewardship intervention that was centered on peer comparison of overall antibiotic prescriptions achieved impressive reductions in total and unnecessary antibiotic usage while increasing optimized treatment, and caused no harm to patients, within VA primary care clinics. Nevertheless, antibiotic prescriptions during the intervention period remained unnecessary in 39% of instances, which leaves much room for further improvement. Other future objectives are to prove that our intervention can improve the choice of antibiotic and duration of treatment and achieve successes that are sustained over the long-term.

MATERIALS AND METHODS

Baseline antibiotic prescribing data were collected and reported previously (7), during which time there were no active antimicrobial stewardship efforts in place. As the data presented here were collected as part of quality improvement activities, VAPHS Institutional Review Board (IRB) approval was waived. The intervention period was from January to June 2017. Given the seasonal variability of outpatient antibiotic prescribing (33, 34), the same months in 2016 were chosen as the baseline for a pre-post analysis. All oral (p.o.) outpatient antibiotic prescriptions written by VAPHS PCPs from our seven clinics were identified as prescribed in the VA electronic health record, the Computerized Patient Record System (CPRS), with prescriptions for >28 days excluded to focus on acute infections. Prescriptions written by medical residents were excluded. The number of patients cited as being cared for by VA PCPs is the sum of all VAPHS clinic panel sizes, excluding patients assigned to resident clinics. The number of patient visits cited is the sum of all office visits to VA PCPs, excluding visits to resident clinics. The antibiotic index was defined as the number of oral antibiotic prescriptions per 1,000 patients per year. For the purposes of feedback, patients were attributed to specific PCPs if they were assigned to their patient panel.

Prior to the intervention period, all PCPs were offered an on-site educational session delivered by an ID physician (N. R. Shively). Topics covered included national data on outpatient antibiotic use and overuse, VAPHS baseline antibiotic prescribing data (7), and a description of what to expect from our project. Antibiotic prescribing guidelines for ARTIs, SSTIs, UTIs, asymptomatic bacteriuria, and COPD exacerbations were reviewed (35–38; http://goldcopd.org/global-strategy-diagnosis-management-prevention-copd-2016/). All PCPs were provided with a handout that included a table of summarized recommendations; this information was also posted in clinic work areas (see the supplemental material).

PCPs were included in the ongoing e-mail-based intervention if they had a panel size of ≥100 patients. The 7 clinics were consolidated into 4 local practice sites; a range of 8 to 18 PCPs per practice site were included in the intervention. During the intervention period, PCPs were e-mailed a monthly comparison of their antibiotic index (based on the prior 3 months of prescribing, presented to PCPs as an annualized number) with the indices of deidentified peers at their practice site as well as a system target (see Fig. S1 and S2 in the supplemental material). The system target antibiotic index was set at 48, which was based on our baseline analysis of antibiotic prescribing volume and appropriateness (7). If a PCP’s antibiotic index was higher than the system target, they were informed that they may be writing unnecessary antibiotic prescriptions, and it was requested that they review the attached antibiotic prescribing recommendations. At the request of several clinicians, late in the second month of the intervention period, electronic order sets corresponding to prescribing recommendations on the handouts were made available to facilitate guideline-concordant prescribing.

Retrospectively, a random sample of approximately 10% of prescriptions during the intervention phase was reviewed and compared to the baseline period for compliance with consensus guidelines. The indication for reviewed antibiotics was based on documentation in the CPRS. It was determined whether the prescribed antibiotic was indicated for the condition documented and whether the prescription was guideline concordant for the agent and duration of therapy. An “unnecessary” antibiotic is one that was given when treatment was not indicated. An “optimally prescribed” antibiotic is one that was indicated and guideline concordant with respect to agent selection and duration. We utilized the same consensus guidelines and procedures for the determination of antibiotic appropriateness as the ones described previously for our baseline analysis (7).

To evaluate for evidence of harm as a result of our intervention, a safety analysis was conducted. Patients were identified who had a PCP visit during which no antibiotic was given within 30 days prior to an ED visit or admission during which an intravenous (i.v.) or p.o. antibiotic was given. A random subset of at least 5% of these cases were evaluated to determine the likelihood that an antibiotic prescribed during the PCP visit could have prevented the subsequent ED visit or admission, with “unlikely,” “uncertain,” and “might have been helpful” as the possible categories. Four authors (N. R. Shively, D. J. Buehrle, C. J. Clancy, and B. K. Decker) participated in this analysis and determined a consensus category for each reviewed case.

Statistical analyses for categorical variables were performed using Fisher’s exact test. To further adjust for seasonality, a regression model was utilized to evaluate the antibiotic prescription rates between the baseline and intervention periods. The number of antibiotic prescriptions per 1,000 patient visits was the dependent variable, with month and intervention as predictors in the model. The usage rate for individual antimicrobial agents was summed per 1,000 visits for each 6-month period and compared as rate ratios. A P value of <0.05 was considered significant.