Feasibility of a Low-Intensity Intervention to Influence Antibiotic Prescribing Rates Use in Outpatient Settings: A Cluster Randomized Controlled Clinical Trial

Abstract

Background

Primary care providers (PCPs) may modify their antibiotic prescription practices if aware of their potentially damaging impact.

Methods

We conducted a cluster randomized controlled trial at 12 Veterans Affairs community-based outpatient clinics. PCPs at clinics randomized to the intervention group received quarterly antibiotic use reports with feedback about antibiotics prescribed for acute respiratory infections and adverse event letters alerting about Clostridioides difficile infection or antibiotic-resistant gram-negative bacteria among their patients. The main outcome, antibiotic prescriptions in primary care visits, was compared in the preintervention (April–September 2020), intervention (October 2020 to September 2021), and postintervention periods (September 2021 to September 2022).

Results

Among 52 PCPs at 6 clinics in the intervention group, 66% (33 of 52) and 54% (28 of 52) received ≥1 antibiotic use report and adverse event letter. In the intervention clinics, the proportion of primary care visits with antibiotic prescription during the preintervention, intervention, and postintervention periods was 1.4% (1088 of 77 697), 1.4% (2051 of 147 858), and 1.3% (1692 of 131 530). In the control clinics, this increased from 1.8% (1560 of 87 897) to 2.1% (3707 of 176 825) and 2.1% (3418 of 162 979), respectively, during the intervention and postintervention periods. The rate of visits with antibiotic prescription did not differ in the preintervention period (odds ratio [95% confidence interval], 1.10 [.87–1.39); P = .43) but did during the intervention (1.30 [1.04–1.62]; P = .022) and postintervention periods (1.38 [1.09–1.74]; P = .007). There were no differences in emergency department visits and hospitalizations.

Conclusions

PCPs from clinics assigned to a low-intensity intervention combining comparative feedback with adverse event notifications had lower antibiotic prescription rates.

In this cluster-randomized controlled trial involving 12 clinics, distributing antibiotic use and adverse event reports to primary care providers was associated with lower antibiotic prescribing rates. This indicates that this relatively low-intensity intervention can support antimicrobial stewardship in outpatient settings.

Clinicians working in primary care are increasingly aware of the growing problem of antibiotic resistance, yet they may not perceive that their own prescribing patterns contribute to this public health crisis. A survey of >1500 primary care providers (PCPs) in the United States found that while >90% acknowledged that inappropriate prescribing contributes to antibiotic resistance, <40% saw it as a problem in their own practices [1]. Delays between antibiotic exposure and adverse events, such as the acquisition of antibiotic-resistant pathogens or Clostridioides difficile infection (CDI), can obscure the consequences of their prescribing choices.

In 2018, Doctor et al [2] conducted a randomized controlled trial to raise awareness among clinicians about the adverse events potentially related to prescribed opioids. Clinicians in the intervention group received letters informing them of accidental overdose deaths among patients to whom they had prescribed opioids. In the subsequent 3 months, opioid prescriptions by clinicians who received the letters decreased by 10%. We inferred that informing primary care clinicians of adverse events potentially related to antibiotics they prescribed could serve as a stewardship intervention in the outpatient setting. To our knowledge, this approach has not been previously studied in a randomized controlled trial. The evaluation of interventions in antimicrobial stewardship at the individual patient level can be constrained by contamination. To avoid this, and for practical reasons, allocation of stewardship intervention at the cluster level is often preferred [3].

We hypothesized that providing PCPs with comparative feedback about their antibiotic prescriptions and notifying them about antibiotic-related adverse events among their patients would reduce antibiotic prescribing rates. To test this hypothesis, we developed a relatively low-intensity intervention consisting of 2 personalized letters distributed quarterly to PCPs working at Veterans Affairs (VA) community-based outpatient clinics (CBOCs). The first letter provided a peer comparison regarding antibiotics prescribed to patients with a diagnosed acute respiratory tract infection (ARI), a well-defined target of antibiotic stewardship efforts since viral pathogens cause ARIs and should not be typically treated with antibiotics [4]. The second letter detailed antibiotic-related adverse events, specifically highlighting CDI and the acquisition of antibiotic-resistant gram-negative bacteria (R-GNB) among the patients in their care, given the potential negative impact of antibiotic use on the microbiota [5]. We describe the intervention as low intensity to reflect the level of engagement required from PCPs and the stewardship resources needed.

In the current study, we describe a cluster randomized controlled trial designed to assess a relatively low-intensity behavioral intervention to promote antimicrobial stewardship in the outpatient setting. The objective was to demonstrate the feasibility and effectiveness of informing practitioners about their antibiotic prescription practices regarding ARIs and antibiotic-related adverse events (eg, CDI and R-GNB) in reducing the use of antibiotics, including for ARIs in VA-CBOCs.

METHODS

Study Design and Data Sources

We conducted a cluster randomized controlled trial at 12 CBOCs affiliated with a large VA medical center (Figure 1). PCPs at clinics randomized to the intervention group received quarterly reports tailored to their practices, while PCPs in the control group did not. The outcome of interest and rate of antibiotic prescriptions were assessed among primary care visits during the preintervention (1 April to 30 September 2020), intervention (1 October 2020 to 30 September 2021), and postintervention periods (1 October 2021 to 30 September 2022). We used the VA Informatics and Computing Infrastructure to access the Veterans Health Administration's corporate data warehouse (CDW) to obtain clinical and administrative data, including antimicrobial use data. The institutional review board at the VA Northeast Ohio Healthcare System approved the study protocol.

Figure 1.

CONSORT flow diagram of cluster randomized controlled trial. Twelve Veterans Affairs (VA)–affiliated community-based outpatient clinics were randomly assigned to receive either the intervention, which was comparative feedback and adverse event notification through the distribution of antibiotic use reports (AURs) and adverse event letters (AELs) to their primary care providers (PCPs), or to serve as controls.

Participants and Randomization

The VA Northeast Ohio Healthcare System includes 12 free-standing community clinics that serve >110 000 veterans annually. The sample consisted of the 12 clinics without further sample size determination. PCPs in this study included physicians, nurse practitioners, and physician assistants without specific inclusion or exclusion criteria. Allocation of the clinics to the intervention and control arms occurred with stratified randomization in which clinics were matched by the size of the total patient panel, and 1 clinic within each pair (1:1) was randomized to the intervention (by B. M. W.). Randomization occurred 3 months before the distribution of the first set of reports. This study did not include factors necessitating patient consent.

The following data obtained from the CDW were used to describe patients and their primary care visits during the 3 periods of the study: age, sex, race, ethnicity, and common comorbid conditions required to calculate the Charlson Comorbidity Index [6]. Patients could have multiple visits throughout the study period; age, comorbid conditions, and the Charlson Comorbidity Index were assessed at the date of the first visit for each patient within each interval of the study period. The list of antibacterial agents was obtained from the CDW, excluding antiviral, antifungal, and antiparasitic agents, and aligned with the National Healthcare Safety Network module on antimicrobial use, as well as the academic detailing reports on ARIs from VA Stewardship Resources.

Development and Implementation of a Behavioral Intervention

The behavioral intervention consisted of issuing 2 reports to providers working in the CBOCs in the intervention group (Supplementary materials, Appendices 1 and 2). The first reports, the antibiotic use reports (AURs), gave comparative feedback to PCPs about antibiotics prescribed for patients with uncomplicated ARIs over the previous 5 quarters. As described elsewhere [7], the reports displayed the number of antibiotics prescribed for an uncomplicated ARI divided by the number of visits for uncomplicated ARIs, as determined by diagnostic codes associated with primary care visits. Complicated ARIs (eg, with underlying chronic respiratory disease) were excluded. Each report compared the rate of antibiotics prescribed for an uncomplicated ARI by the PCP for whom the report was prepared to the average of VA PCPs across all the outpatient clinics included in the study. AURs indicated whether the recipient's rate of antibiotic prescriptions for ARIs was higher, similar, or lower compared to their peers and delineated criteria for being considered a “frontline antibiotic steward” [8].

The second reports, adverse event letters (AELs), alerted PCPs to new detection of CDI or R-GNB among their patients. We defined cases of R-GNB as the incident detection of a newly resistant or more resistant antibiotic susceptibility profile in the following: Pseudomonas aeruginosa, Escherichia coli, Citrobacter koseri, Klebsiella, Enterobacter, Serratia, Proteus, and Providencia. Exclusion criteria were assessed by means of brief chart review: cases involving hospitalized patients, nursing home residents, or persons in hospice care and cases of CDI that the PCP had diagnosed and treated. The AELs reported the date(s) of the adverse event(s) and, for the R-GNB, the species and type of resistance of the cases that met our inclusion and exclusion criteria. All VA prescriptions for antibiotics in the 6 months before the detection of CDI or R-GNB were listed, along with the location of the prescriber (eg, primary care, urgent care, specialty clinic, etc). The AELs were crafted with a user-centered design framework involving PCPs who did not practice at the CBOCs.

During the intervention period, AURs and Adverse Events Letters were distributed quarterly via secure email to PCPs working in the CBOCs in the invention group. The number of ARI visits required for a provider to receive an AUR was selected each quarter to ensure that each intervention site received ≥1 report. We adjusted the minimum number of visits needed accordingly. The threshold decreased over time due to decreases in the number of ARI visits during the early phases of the coronavirus disease 2019 (COVID-19) pandemic, and the lowest threshold was ≥8 ARI visits.

Evaluation

Quantitative Results

The primary quantitative outcome was the rate of antibiotics prescribed at primary care visits associated with diagnoses of uncomplicated ARIs by providers in community clinics in the intervention clinics compared with the control clinics at the cluster level. Another outcome was the rate of antibiotics prescribed at primary care visits by providers in community clinics in the intervention clinics compared with the control clinics. We also summarized the distribution of prescribed antibiotic agents and diagnoses documented in all visits. To assess antibiotic use at VA CBOCs, we used data from the CDW to measure patient encounters, unique patients, and antibiotic prescriptions and to determine the agents prescribed and the International Classification of Diseases, Tenth Revision codes associated with the visit, with particular emphasis in the subset of infectious diseases. Antibiotic indications were classified based on these codes into common infection categories, as described elsewhere (Supplementary Table 1) [9].

The primary outcome was compared between community clinics in the intervention and control arms using a mixed-effects logistic model predicting the presence of an antibiotic at a primary care visit. We considered time as a 3-level categorical variable (preintervention, intervention, and postintervention) and estimated random intercepts for each clinic. The effect of interest was the interaction of time and study arm. Within-time contrasts of the study arm were performed following a detected interaction to estimate the odds ratio (OR) of an antibiotic prescription in the control arm compared with the intervention arm. Additional sensitivity models were estimated: one assessed whether intervention results differed between in-person and telehealth visits, and another incorporated a random effect of the provider. Finally, we summarized acute admissions and emergency department visits within the VA in the week following primary care visits to identify any possible safety consequences of the intervention. All statistical analyses were performed using R software (version 4.0.1) and implementing functions from the lme4 and nlme packages.

Qualitative Results

We conducted an anonymous online survey using REDCap to query providers who received ≥1 AUR about the perceived utility and ease of understanding the content (Supplementary materials, Appendix A). Providers were also invited to participate in semistructured interviews (Supplementary Materials, Appendix B) [10]. The interview guide featured open-ended questions and probes based on participants' responses, ensuring uniform data collection while exploring relevant topics. It was informed by selected domains from the Consolidated Framework for Implementation Research [10]. Qualitative team members were blinded to PCPs' antibiotic prescribing data and debriefed after each interview for quality assurance. Interviews were audio-recorded and transcribed. ATLAS.ti software (version 9, Scientific Software Development) was used for data management, coding, and analysis.

Transcripts were coded and analyzed using deductive and inductive content analysis [11]. Deductive content analysis involved identifying data that fit within a priori categories, with attention to responses specific to interventions and their characteristics. Inductive content analysis involved open and unstructured coding, allowing for discovering new categories and contextual factors and capturing the complexity of clinicians' experiences. Data were aggregated across sites and respondent types and organized using categories describing PCPs' perceptions of intervention, including helpfulness and utility, agreement with recommendations, attitude (including reactance), changes in prescribing behaviors, suggestions for improvement, and unintended consequences.

RESULTS

Participants and Intervention

After randomization, the 6 clinics in the intervention group had 52 PCPs (range, 4–22 at each clinic), and the 6 clinics in the control group had 61 PCPs (range, 6–17). A total of 90 AURs were sent to PCPs in the intervention arm (Table 1), with 63% (33 of 52) receiving ≥1 report and 54% (28 of 52) receiving >1. In addition, 42 AELs were distributed, with 56% (29 of 52) of PCPs receiving ≥1 and 19% (10 of 52) receiving >1. The distribution of reports occurred approximately quarterly.

Table 1.

Distribution of Antibiotic Use Reports and Adverse Event Letters to Primary Care Providers in Clinics Randomized to the Intervention Arm

	PCPs, No. (%)a
	All Clinics (N = 52)	Clinic A (n = 22)	Clinic B (n = 8)	Clinic C (n = 7)	Clinic D (n = 6)	Clinic E (n = 5)	Clinic F (n = 4)
AURs sent, no.	90	32	26	12	5	6	9
PCPs who received any AUR,	33 (63)	12 (55)	8 (100)	5 (71)	2 (33)	3 (60)	3 (75)
PCPs who received >1 AUR	28 (54)	9 (41)	8 (100)	3 (43)	2 (33)	3 (60)	3 (75)
AELs sent, no.	42	15	11	8	4	1	3
PCPs who received any AEL	29 (56)	10 (45)	8 (100)	6 (86)	4 (50)	1 (20)	1 (25)
PCPs who received >1 AEL	10 (19)	4 (18)	2 (25)	2 (29)	1 (17)	0 (0)	1 (25)
Both AUR and AEL
PCPs who received both	24 (46)	8 (47)	8 (100)	20 (71)	9 (17)	1 (20)	1 (25)
PCPs who received >1 AEL and >1 AUR	7 (13)	3 (14)	2 (25)	0 (0)	1 (17)	0 (0)	1 (25)

Abbreviations: AEL, adverse event letters; AUR, antibiotic use report; PCPs, primary care providers.

^aData represent no. (%) of PCPs unless otherwise specified.

Patients treated at clinics randomized to the intervention group were similar to those in the control group (Supplementary Table 2). The most common medical conditions included diabetes mellitus, chronic lung disease, and nondermatologic cancer. The prevalence of the most common medical comorbid conditions was slightly higher in the preintervention than in the intervention and postintervention periods.

Antibiotic Prescriptions

During the study, visit modalities (in person vs telehealth) varied, but ≤5% of visits were associated with infectious disease diagnoses, and ≤1% with ARI diagnoses (Table 2). For the intervention clinics, the percentage of primary care visits with an antibiotic prescription was 1.4% (1088 of 77 697 visits), 1.4% (2051 of 147 858), and 1.3% (1692 of 131 530) across the preintervention, intervention, and postintervention periods, respectively.

Table 2.

Characteristics of Primary Care Visits in Community Clinics Randomized to the Intervention or Control Arm, Stratified by Study Period

Visit Characteristic	Visits, No. (%)a
	Preintervention Period (Apr–Sep 2020)		Intervention Period (Oct 2020 –Sep 2021)		Postintervention Period (Oct 2021–Sep 2022)
	Intervention (n = 77 697)	Control (n = 87 897)	Intervention (n = 147 858)	Control (n = 176 825)	Intervention (n = 131 530)	Control (n = 162 979)
Visit modality
In person	18 614 (24.0)	16 635 (18.9)	72 965 (49.3)	71 812 (40.6)	83 199 (63.3)	84 858 (52.1)
Telehealth	59 083 (76.0)	71 262 (81.1)	74 893 (50.7)	105 013 (59.4)	48 331 (36.7)	78 121 (47.9)
Infectious disease diagnosed
None	74 980 (96.5)	84 856 (96.5)	142 132 (96.1)	169 590 (95.9)	125 248 (95.2)	154 848 (95.0)
ARI	600 (0.8)	662 (0.8)	1115 (0.8)	1445 (0.8)	1292 (1.0)	1672 (1.0)
Other infection	2081 (2.7)	2344 (2.7)	4160 (2.8)	4954 (2.8)	4014 (3.1)	5123 (3.1)
COVID-19	36 (<0.1)	35 (<0.1)	451 (0.3)	836 (0.5)	976 (0.7)	1336 (0.8)
Antibiotic prescriptionb
All visits	1088 (1.4)	1560 (1.8)	2051 (1.4)	3707 (2.1)	1692 (1.3)	3418 (2.1)
ARI visits	140 (23.3)	164 (24.8)	231 (20.7)	435 (30.1)	267 (20.7)	433 (25.9)
VA ED visitc	924 (1.2)	1082 (1.2)	1873 (1.3)	2234 (1.3)	1559 (1.2)	1851 (1.1)
VA hospital admissionc	513 (0.7)	570 (0.7)	1170 (0.8)	1403 (0.8)	855 (0.7)	1043 (0.6)

Abbreviations: ARI, acute respiratory tract infection; COVID-19, coronavirus disease 2019; ED, emergency department; VA, Veterans Affairs.

^aPrimary care visits at community clinics in the intervention and control arm during the study period.

^bAntibiotic prescription at a visit with a diagnostic code for a qualifying respiratory tract infection and no diagnostic codes for other infections.

^cAny VA ED or unplanned hospital admission within 7 days of the outpatient visit.

For control clinics, this proportion increased from 1.8% (1560 of 87 897 visits) in the preintervention period to 2.1% (3707 of 176 825) and 2.1% (3418 of 162 979) in the intervention and postintervention periods, respectively. The rates of visits at which antibiotics were prescribed did not differ significantly between study arms in the preintervention period (model OR comparing control versus intervention groups [95% confidence interval], 1.10 (.87–1.39]; P = .43); Figure 2). During the intervention period, however, PCPs at clinics in the control arm prescribed more antibiotics than those in the intervention arm (model OR [95% confidence interval), 1.30 (1.04–1.62); P = .02); this difference continued through the postintervention period (1.38 [1.09–1.74]; P = .007).

Figure 2.

Rate of primary care visits associated with an antibiotic prescription during the preintervention, intervention, and postintervention study periods. Individual clinics in the intervention (teal) and control (pink) arms are shown as thin lines. Thicker lines represent the mean rate for each arm. The rate of antibiotics prescribed during primary care visits at clinics randomized to the intervention arm was similar to that in the control arm during the preintervention period (P = .43) and lower during the intervention (P = .02) and postintervention (P = .007) periods.

Sensitivity analysis identified significant interactions between modality (in-person vs telehealth), period, and intervention. While no differences were detected between the intervention and control arms in the preintervention period for either modality, the intervention arm effects during both the intervention and postintervention periods were greater among in-person visits than telehealth visits. Moreover, telehealth visits were more common in the control clinics (Table 2).

In the sensitivity analysis in which a random provider effect was added to the model, we observed prescribing ORs consistent with those reported above. Among primary care appointments with an associated ARI diagnosis, the proportions of antibiotic prescriptions during the preintervention, intervention, and postintervention periods were 23.3% (140 of 600), 20.7% (231 of 1115), and 20.7% (267 of 1292), respectively. Among primary care appointments with an associated ARI diagnosis at control clinics, the proportions of antibiotic prescriptions during each study period were 24.8% (164 of 662), 30.1% (435 of 1445), and 25.9% (433 of 1672), respectively.

During the preintervention period, 55% of antibiotic courses (613 of 1111) in the intervention arm and 58% (941 of 1609) in the control arm were prescribed for ≤7 days (Figure 3). In the postintervention period, these figures rose to 63% (1090 of 1722) and 60% (2116 of 3523), respectively. The most commonly prescribed antibiotics were doxycycline, amoxicillin-clavulanate, and azithromycin. Providers in the intervention clinics prescribed more doxycycline and azithromycin while using less amoxicillin-clavulanate and sulfamethoxazole-trimethoprim than providers in the control clinics (Table 3). The rates of admissions and emergency department visits within 7 days of primary care visits were similar across both groups, indicating no unintended negative consequences from the intervention (Table 2).

Figure 3.

Proportion of antibiotic courses stratified by length of therapy during primary care visits at clinics in the intervention (teal) and control (pink) arms during the preintervention, intervention, and postintervention periods.

Table 3.

Antibiotics Prescribed in Association With Primary Care Visits in Community Clinics Randomized to the Intervention or Control Arm, Stratified by Agents and Study Period

Antibiotic Agent	Antibiotic Prescriptions, No. (%)
	Preintervention Period (Apr–Sept 2020)		Intervention Period (Oct 2020–Sep 2021)		Postintervention Period (Oct 2021–Sep 2022)
	Intervention (n = 1111)	Control (n = 1609)	Intervention (n = 2097)	Control (n = 3818)	Intervention (n = 1722)	Control (n = 3523)
Doxycycline	307 (28)	332 (21)	517 (25)	806 (22)	332 (20)	580 (17)
Amoxicillin-clavulanate	147 (14)	292 (19)	323 (16)	781 (21)	290 (17)	789 (23)
Azithromycin	188 (17)	204 (13)	317 (15)	518 (14)	332 (20)	533 (16)
Fluoroquinolones	110 (10)	154 (10)	213 (10)	355 (10)	165 (10)	303 (9)
Amoxicillin	109 (10)	117 (8)	177 (9)	310 (8)	156 (9)	337 (10)
Sulfamethoxazole-trimethoprim	54 (5)	191 (12)	146 (7)	433 (12)	124 (7)	335 (10)
Cephalexin	98 (9)	177 (11)	219 (11)	340 (9)	171 (10)	346 (10)
Nitrofurantoin	55 (5)	58 (4)	100 (5)	130 (4)	88 (5)	174 (5)
Clindamycin	13 (1)	32 (2)	41 (2)	62 (2)	28 (2)	58 (2)
Penicillin	10 (1)	9 (1)	11 (1)	18 (0)	12 (1)	12 (0)
Other	20 (2)	43 (3)	33 (2)	65 (2)	24 (1)	56 (2)

Participant Perceptions of the Behavioral Intervention

In clinics assigned to the intervention group, 33 PCPs received ≥1 AUR and were eligible to receive the online survey, with 13 completing it (39% response rate). Among them, 8 were advanced practice providers, and 9 had >10 years of experience. The survey indicated that 85% found the reports easy to understand and relevant to their practice, and the same percentage wanted to continue receiving them. In addition, 77% of respondents reported that the reports motivated them to become frontline antibiotic stewards.

Seven PCPs also completed semistructured interviews, all receiving ≥1 AUR and AEL. Most interviews lasted approximately 10 minutes (range, 6–28 minutes). The interview provided insights that complemented survey results regarding the providers' experiences with the reports. (Supplementary Table 3). We identified the following themes in the interviews: (1) antibiotics do not treat viral infections; (2) AURs and AELs are useful; (3) focusing on “adverse events” is counterproductive; and (4) content and presentation can be improved. Most respondents acknowledged being aware of antibiotic stewardship principles and the potential negative consequences of antibiotic use. They recalled details from the AURs and found them useful and a “helpful reminder” to reflect on their antibiotic prescribing practices. PCPs appreciated the easy-to-understand information and the graph that allowed them to compare their antibiotic use to others. One suggestion was to avoid using red in the graph, as it has a negative connotation. Other respondents requested more information to guide antibiotic use, including links to authoritative sources.

Most interview respondents did not recall receiving the AELs. Some mentioned that they disregarded these letters because they were not the prescribing clinicians who initiated the antibiotic referenced in the communication. One positive comment noted that the letters contained detailed information about the patients, including the date, specific event, antibiotic, and its consequences, which meant that the respondent “didn’t have to do a lot of searching through the chart to find what they were talking about.” A respondent also noted that the name “adverse event letter” was inaccurate, as the contents of some letters included positive feedback. Some suggested that chart notes or banners might be a more effective means to link adverse events with individual patients, which was also balanced with concerns for alert fatigue. Other suggestions were to use instant messaging via the VA's Microsoft Teams network and noted that the entire primary care team, described as a patient-aligned care team, should be notified.

DISCUSSION

In this cluster-randomized controlled trial to assess the impact of a relatively low-intensity behavioral intervention consisting of 2 types of reports distributed quarterly, we found a sustained difference in the proportion of antibiotic prescriptions written by PCPs between the intervention clinics and the control clinics. During the 6-month preintervention period, which coincided with the early phase of the COVID-19 pandemic, the rates of antibiotic prescriptions were similar in the intervention and control clinics. However, during the intervention and postintervention periods, the proportion of visits with an antibiotic prescription remained steady for clinics in the intervention group and increased for those in the control clinics.

These results suggest that the relatively low-intensity intervention consisting of AURs and Adverse Events Letters effectively influenced the prescribing behavior of PCPs practicing in CBOCs affiliated with a VA medical center. Notably, the intervention may have been effective even if it was delivered to only some providers. The intervention was considered low intensity because it required no direct action from providers. Also, compared with other stewardship interventions studied in cluster randomized controlled trials, the intervention did not include more resource-intensive tools, such as audit and feedback, advanced education, antibiotic restrictions, or involvement of infectious diseases-trained clinicians [4]. The disruption of antibiotic prescription practices due to COVID-19 likely significantly affected our findings. Observations from 108 VA medical centers showed that inpatient antibiotic use increased during the early phases of COVID-19 but subsequently decreased below prepandemic levels [12].

Behavioral nudges use social motivations to influence the prescribing patterns of primary care prescribers [13]. In 2016, Meeker et al [8] described using peer comparison alone and in combination with other interventions to reduce inappropriate antibiotic prescribing for ARIs . Although this approach was effective, antibiotic prescription rates increased somewhat after the intervention stopped, suggesting that peer comparison may require augmentation for a sustained effect [14]. Accordingly, we also sent an Adverse Events Letter to PCPs. These letters were intended to connect an antibiotic exposure and unintended negative consequences in the subsequent days or weeks, namely the acquisition of an R-GNB or CDI. During the brief chart review conducted as part of developing each AEL, we learned that most PCPs were already aware of their patients with CDI, having diagnosed the CDI and prescribed the appropriate treatment. Most patients with a qualifying adverse event received multiple courses of antibiotics during the assessment period. Prescribers included both VA and non-VA clinicians, not just the PCPs. As a result, the AELs may have become general notifications of detected adverse events rather than personalized warnings tied to antibiotics prescribed by the PCPs.

Our study has several significant limitations. It began in the early months of the COVID-19 pandemic when fewer patients sought care for ARIs in primary care clinics. This led to insufficient data to generate an AUR for every PCP each quarter. However, the proportion of patients with diagnostic codes for ARIs and other infections remained similar in the intervention clinics compared to the control clinics.

Throughout the study, clinics in the intervention group had more in-person visits than telehealth visits. In the VA primary care clinics, telehealth was associated with lower rates of antibiotic prescriptions, suggesting that visit types do not explain our findings. Randomization may not have fully addressed the differences between the intervention and control clinics.

This small randomized controlled trial did not evaluate the impact of comparative feedback versus adverse event notification on providers. Our qualitative data suggest that providers recalled the personalized AURs better than the AELs, perhaps because the consequences of antibiotic use were often attributed to others.

Conducted in outpatient clinics within a single VA healthcare system, the study's findings may not be generalizable to other populations, as VA users are predominantly older, white, non-Latino men with a high burden of chronic medical conditions [15, 16]. Moreover, the study relied on administrative data, which has inherent limitations, including difficulty ascertaining whether patients took the prescribed antibiotics or whether prescriptions were filled at non-VA pharmacies without proper documentation.

In conclusion, notwithstanding its limitations, the results of this pilot cluster randomized controlled trial suggest that a relatively low-intensity behavioral intervention, combining comparative feedback and notification about adverse events to providers, can reduce antibiotic prescribing in outpatient clinics. Wider implementation in a postpandemic setting may enhance antibiotic stewardship and help decrease antibiotic-resistant organisms, benefiting public health.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Acknowledgments

Author contributions. R. L. P. J. and F. P. had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Concept and design: B. M. W., U. C. A., L. S., S. L. B., R. L. P. J., and F. P. Acquisition, analysis, or interpretation of data: T. A. B., B. M. W., U. C. A., T. (T.) C., N. M., S. S., C. F., S. Song, K. C. S., R. H., M. H., T. L. W., L. S., S. L. B., R. L. P. J., and F. P. Drafting of the manuscript: T. A. B., B. M. W., and R. L. P. J. Critical revision of the manuscript for important intellectual content: T. A. B., B. M. W., U. C. A., S. Song, R. L. P. J., and F. P. Statistical analysis: T. A. B., B. M. W., and S. Song. Obtained funding: B. M. W., R. L. P. J., and F. P. Administrative, technical, or material support: T. A. B., S. Song, and C. K. Supervision: R. L. P. J. and F. P.

Disclaimer. The findings and conclusions in this document are those of the authors, who are responsible for its content and do not necessarily represent the views of the Veterans Affairs or the US government. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Financial support. This work was supported by Merck (support to R. L. P. J. and F. P.) and by funds and facilities provided by the Geriatric Research Education and Clinical Center at the VA Northeast Ohio Healthcare System, the TECH-GRECC at the VA Pittsburgh Healthcare System, and the Specialty Care Center of Innovation at the VA Northeast Ohio Healthcare System.