Choosing Wisely^™ Interventions to Reduce Antibiotic Overuse in the Safety-Net

Abstract

Objective:

Physician pay-for-performance programs frequently target inappropriate antibiotics. Yet, little is known about pay-for-performance programs’ effects on antibiotic prescribing among safety-net populations at risk for unintended harms from reducing care. We evaluated effects of pay-for-performance-motivated interventions to reduce antibiotic prescriptions for safety-net patients with acute respiratory tract infections (ARTIs).

Study Design:

Interrupted-time-series.

Methods:

A non-randomized trial (5/28/2015–2/1/2018) was conducted at two large academic safety-net hospitals: Los Angeles County+USC (LAC+USC) and Olive View-UCLA (OV-UCLA). In response to California’s 2016 pay-for-performance program to reduce antibiotics for acute bronchitis, five staggered Choosing Wisely^™-based interventions were launched in combination: audit and feedback provided clinicians with case-specific feedback; clinician education included journal clubs/posters/screensavers; suggested alternatives presented clinicians and patients with CDC-recommended non-antibiotic treatments; procalcitonin indicated when bacterial infections were unlikely; public commitment displayed pledges to reduce inappropriate antibiotic prescribing. We also assessed five unintended effects: HEDIS-appropriate prescribing, diagnosis shifting, substituting antibiotics with steroids, increasing antibiotics for ARTIs not penalized by the pay-for-performance program, and inappropriate withholding of antibiotics.

Results:

Among 3,583 consecutive patients with ARTIs, mean antibiotic prescribing rates for ARTIs decreased from 35.9% to 22.9% (OR 0.60 [0.39–0.93]) at LAC+USC; and 48.7% to 27.3% (OR 0.81 [0.70–0.93]) at OV-UCLA after the intervention. HEDIS-inappropriate prescribing rates decreased from 28.9% to 19.7% (OR 0.69 [0.39–1.21]) at LAC+USC; and from 40.9% to 12.5% (OR 0.72 [0.59–0.88]) at OV-UCLA. There was no evidence of unintended consequences.

Conclusions:

These real-world multicomponent interventions responding to pay-for-performance incentives were associated with substantial reductions in antibiotic prescriptions in two safety-net health systems without unintended harms.

Trial Registration:

ClinicalTrials.gov: NCT03464279.

Prècis

This study evaluates the impact of Choosing Wisely^™-based interventions on antibiotic prescribing for viral respiratory tract infections in a real-world safety-net setting.

INTRODUCTION

Inappropriate antibiotics account for 25–50% of all U.S. antibiotic prescriptions,^1–5 and are associated with substantial harm such as antibiotic resistant organisms, which account for over 35,000 excess deaths in the U.S. annually.⁶ Three-quarters of the 266 million outpatient antibiotics prescribed to Americans each year are to treat acute respiratory tract infections (ARTIs) for which antibiotics are indicated in less than half of cases, making ARTIs the single largest opportunity for reducing inappropriate outpatient antibiotics and their associated harms.^4,7–9

Many interventions have successfully reduced inappropriate antibiotic prescribing, but few of these initiatives have been evaluated within safety-net systems.^10–19 This is concerning given that efficacy studies often translate poorly to safety-net systems,^20–22 and quality improvement (QI) and pay-for-performance programs (e.g., 30-day readmissions) have a history of unintentionally penalizing safety-net systems and worsening healthcare disparities.^23–25 It is therefore crucial to understand the effects of complex QI initiatives within safety-net systems in order to prevent unintended harms from befalling disadvantaged and medically-underserved patients.²⁶

In 2016, California Medicaid (Medi-Cal) launched a five-year, $7.5 billion state-wide pay-for-performance QI initiative called Public Hospital Redesign and Incentives in Medi-Cal (PRIME).²⁷ PRIME was one of the largest U.S. state-wide pay-for-performance programs ever implemented,²⁸ and included one aim to reduce inappropriate antibiotic prescriptions for acute bronchitis.²⁹ We evaluated the intended and unintended effects of real-world multicomponent interventions to reduce antibiotic prescribing that arose in response to this large pay-for-performance program.

METHODS

Study Design

This study compared rates of prescribing antibiotics for ARTIs after behavioral interventions at two major academic safety-net medical centers in the nation’s second largest public safety-net healthcare system: Los Angeles County and University of Southern California (LAC+USC) and Olive View-University of California, Los Angeles (OV-UCLA).³⁰ The original design was a quasi-experiment to study the effects of multicomponent interventions at LAC+USC compared to OV-UCLA (control), where to the authors’ knowledge no interventions were planned at the time of study design. However, after data collection the authors learned that PRIME had unexpectedly (and independently of the study team) led OV-UCLA leadership to implement their own interventions to reduce antibiotic prescriptions for acute bronchitis. Given that OV-UCLA was no longer a suitable control site, the study protocol was amended to evaluate the effects of these multicomponent interventions within each site rather than between sites.

The study population and analysis protocol (Supplement 1) were preregistered on ClinicalTrials.gov (NCT03464279).

Settings

The intervention occurred in the urgent care centers of LAC+USC (600-bed hospital in a high density urban area; urgent care with 3 physicians, 10 nurse practitioners, and 2 physician assistants) and OV-UCLA (375-bed hospital with an urban and large rural catchment area; urgent care with ~10 physicians, 1 nurse practitioner, and many rotating trainees). Although LAC+USC and OV-UCLA are both part of the LAC-DHS health system, they function as separate, independent medical centers with their own leadership. Almost half of all LAC-DHS patients’ primary language is not English, 65% identify as Hispanic/Latinx, 65% are Medi-Cal enrollees, and 23% are uninsured.³⁰

Enrollment

The interventions did not require enrollment (Figure 1); they were applied to all urgent care clinicians without direct incentives (financial or otherwise).

Figure 1.

Flow of participants in non-randomized trial of antibiotic prescribing.

Outcome Measures

The primary study outcome was the rate of prescribing antibiotics in an encounter for any ARTI that typically does not require antibiotics (acute bronchitis, non-acute bronchitis, acute bronchiolitis, acute nasopharyngitis, chronic sinusitis, non-strep pharyngitis, acute upper respiratory infection, influenza, viral pneumonia, and cough; eTable1 in Supplement 2). This prespecified outcome was intentionally broader than that of the pay-for-performance program which only measured antibiotics for acute bronchitis (ICD-10 code J20), given that we reasonably expected the interventions to impact antibiotic prescribing for many other non-bacterial ARTIs (e.g., J21/acute bronchiolitis) and to maximize power.

The Healthcare Effectiveness Data and Information Set (HEDIS) criteria were used to stratify antibiotic prescriptions as appropriate (≥1 concomitant diagnosis or comorbidity to justify the antibiotic) or inappropriate (no concomitant diagnoses/comorbidities). Concomitant diagnoses that justified antibiotics were active bacterial infections (e.g., pneumonia) or comorbid conditions (e.g., HIV, COPD) defined by the HEDIS “Competing Diagnosis” and “Comorbid Conditions” value sets.^31,32

Secondary outcomes assessed for unintended consequences: reductions in HEDIS-appropriate prescribing for bacterial respiratory infections (e.g., pneumonia), increased coding for antibiotic-appropriate conditions (i.e., diagnosis shifting),³³ increased steroids for ARTIs or antibiotics for ARTIs not targeted by PRIME (suggesting a substitution of antibiotics for steroids or shifting of antibiotics to similar conditions), and inappropriate withholding of antibiotics upon medical chart review.

Data

Prescribing data, patient and encounter information, and ICD-9/10 codes were extracted from the EHR data repositories of both sites and transferred to UCLA for analysis. Urgent care encounters with a primary ICD code for an eligible ARTI occurring 5/28/2015–2/1/2018 were included. Incidentally, a concurrent EHR upgrade led to missing age, gender, race, and ethnicity data at LAC+USC from 3/1/2017–5/31/2017. Age data were missing from OV-UCLA, but since the average age of the chart review sample from LAC+USC was concordant with the average age of the full LAC+USC population (53 years [SD 15] v. 51 years [SD 16]), the average age of the chart review sample from OV-UCLA (49 years [SD 15]) was used, which is also consistent with publicly available sources.³⁴

Gender and race/ethnicity information were missing in 6% and 10% (respectively) of the population; these patients were included in the analysis.

Interventions

Public Hospital Redesign and Incentives in Medi-Cal (PRIME) was a five year, $7.5 billion pay-for-performance program launched in California in 2016 consisting of 18 QI initiatives.²⁷ One project aimed to reduce antibiotic prescriptions for acute bronchitis (bronchitis encounters with antibiotic prescriptions/all encounters for acute bronchitis; Supplement 1) in accordance with the Choosing Wisely^™ campaign.³⁵ To receive payments, a site had to perform >25^th percentile of all sites and exhibit ≥10% year-over-year gap closure between current performance and the 90^th percentile of performance (or maintain performance ≥90^th percentile if a site had already achieved this benchmark).³⁶ Payments were based upon the performance percentile and the magnitude of gap closure, and were worth hundreds-of-millions of dollars in federal/state funds.²⁹

With the exception of the public commitment intervention below (part of a broader Choosing Wisely^™ effort to reduce medical overuse across LAC-DHS.^37,38), all interventions were hospital-initiated (independently of the study team) in response to the PRIME incentive payments. All interventions remained in-place throughout the entire study period once implemented.

Clinician case-audit feedback³⁹ (LAC+USC 11/21/2016; OV-UCLA 3/28/2017) was an email and in-person intervention and crux of the multicomponent interventions. Urgent care directors delivered one-on-one case-specific feedback to clinicians numerous times throughout the intervention when antibiotic prescribing for acute bronchitis was at-risk of missing the PRIME benchmark reduction. Clinic-level (not clinician-level) performance reports on PRIME across all of LAC-DHS were also disseminated to all clinicians by email multiple times throughout the intervention.

Clinician education (LAC+USC 11/21/2016; OV-UCLA 11/30/2016) was a physical intervention consisting of regular emails from leadership (both sites), two journal clubs (LAC+USC only), and posters/screen savers (OV-UCLA only) based on infographics from the Choosing Wisely^™ campaign to reduce prescribing of antibiotics for ARTIs.³⁵

Suggested alternatives (LAC+USC 10/28/2016; OV-UCLA 11/30/2016) was a physical intervention delivered at both sites through CDC “viral prescription pads” given to urgent care clinicians that listed non-antibiotic treatments for ARTIs (e.g., acetaminophen).⁴⁰ Given that many patients within the LAC-DHS safety-net system present to urgent care seeking antibiotics,⁴¹ clinicians could write these “prescriptions” to assuage this pressure.

Procalcitonin^42–44 (LAC+USC only, 12/5/2016) was a rapid turnaround lab-based intervention. Procalcitonin is a blood marker for bacterial infection, and providers were discouraged from using antibiotics if procalcitonin was <0.25 ug/L.

Public Commitment⁴⁵ (LAC+USC only, 3/20/2017) was an intervention consisting of all urgent care clinicians signing prominently displayed 48”x36” posters pledging to not prescribe unnecessary antibiotics in accordance with Choosing Wisely^™ guidelines (eFigure 1 in Supplement 2).¹⁶

Appropriateness

Antibiotic appropriateness was adjudicated per the HEDIS guidelines for “Avoidance of Antibiotic Treatment in Adults with Bronchitis”^31,32 using an electronic measure (e-measure) that queried EHR data for 1) urgent care encounters for ARTIs, 2) antibiotics prescribed during those encounters, and 3) the presence of competing/comorbid diagnoses that potentially justified an antibiotic.

An independent internal medicine physician and study co-author (R.L.) blindly reviewed a sample of 101 medical charts, of which the majority (~75%) were randomly selected from antibiotic-present encounters in order to focus on measuring overuse (rather than underuse). Objective professional medical society guidelines (e.g., Infectious Diseases Society of America) were used as the gold standard for determining treatment appropriateness (eTable 2 in Supplement 2). Concordance between HEDIS criteria and professional society guidelines were used to calculate sensitivity/specificity of the e-measure. A second general internist and study co-author (J.M.) reconciled cases with unclear adjudication (n=5).

Analysis

We evaluated the percentage of encounters for ARTIs during which an antibiotic was prescribed from 16 months (LAC+USC) or 12 months (OV-UCLA) prior to any intervention components, through 15 months after the first intervention components at both sites.

Given that these pragmatic interventions were implemented in rapid succession, the analysis aggregated all components into a single “intervention” defined by one site-specific start date. Unstructured qualitative interviews with the physician leads of both urgent cares (who were blinded to study results at the time of the interviews) revealed that they intuitively felt that the most substantial component of the interventions was one-on-one clinician case-audit feedback, so the initiation of this component was used as the official intervention start date. One-on-one feedback started on 11/21/2016 at LAC+USC and was among the first elements implemented there, whereas it was among the last rolled out at OV-UCLA on 3/28/2017 (Figure 2).

Figure 2.

Estimated probabilities over time of antibiotic prescribing and diagnosis shifting.

Each panel depicts the estimated probability of the outcome event (Y-axis) at the start of data collection (start of curve), T1 (first vertical dotted line), T2 (second vertical dotted line) and T3 (end of curve). The slopes of the curves connecting the probability point estimates correspond to the odds ratios (ORs), and the changes in slopes from one time period to another correspond to the ratio of ORs (rORs). Thus, change in slopes were interpretable as the change in rate at which the probability of the outcome changed during the given time period compared to the indicated prior time period, assuming that trends in outcome event rates would have paralleled pre-intervention trends in the absence of the intervention (e.g., trends at T2 would have been parallel to trends at T1 had the intervention not occurred).

Time point T1 was the period before any intervention implementation at both sites, T2 was the period after audit and 1:1 feedback (the primary intervention) had been implemented at LAC+USC but not OV-UCLA (though OV-UCLA had other interventions implemented during this time), and T3 was the period after all interventions had been implemented at both sites.

Brackets and upside-down brackets depict within-site statistical significance for OV-UCLA and LAC+USC, respectively, between the time periods corresponding with the start and end of the brackets. Asterisks represent statistical significance by ITS analysis at the following levels: * if P≤.05, ** if P≤.01, *** if P≤.001.The shaded areas around the solid lines represent 95% CIs.

Panels A and B are the primary outcomes of the intervention (A depicts total antibiotic prescribing, and B depicts HEDIS-inappropriate prescribing). Panel C shows HEDIS-appropriate prescribing, and demonstrates no significant changes in appropriate prescribing after intervention implementation at LAC+USC (T2) or OV-UCLA (T3). Pre-intervention trends in any metric (e.g., declines in HEDIS-appropriate prescribing, increases in coding for alternative ICD codes) likely represent secular trends independent from the interventions. Panels D and E are the pre-specified measures of diagnosis shifting to antibiotic-appropriate ICD codes, and show that there were no statistically significant increases in coding for antibiotic-appropriate conditions or comorbidities after the interventions at both sites (there were statistically significant decreases but not increases after the interventions, arguing against any diagnostic gaming to avoid forfeiting the PRIME incentive payments).

We performed two interrupted-time-series (ITS) analyses (between-group and within-group) consisting of segmented logistic regressions on the dependent variable (antibiotic prescribing) clustered by patient, to estimate the odds ratios (ORs) of each outcome during each time period. Time periods (Figure 2) were defined as follows: T1-- before any intervention implementation, T2-- after one-on-one feedback implementation at LAC+USC but not OV-UCLA (though OV-UCLA had other interventions implemented during T2), T3-- after all intervention implementation at both sites. Interactions between time terms and site were used to estimate within-site and between-site differences. Changes in outcomes between time periods were expressed as ratio of ORs (rORs; i.e., T2 OR/T1 OR). For example, a rOR of 0.60 indicated that the odds of antibiotic prescribing fell by 40% from one time period to another. Estimated probabilities of each outcome event over time were also displayed (Figure 2) for ease of interpretation.

Both analyses also were used to assess HEDIS-appropriate/inappropriate prescribing and the pre-registered unintended consequence of diagnosis shifting. Additional unintended consequences (substituting antibiotics with steroids and increasing antibiotic prescriptions for ARTIs not targeted by PRIME) were added post-hoc, so we assessed trends visually and did not include them in the formal ITS analysis.

Two-tailed P<.05 and odds ratios with 95% CIs excluding one were considered significant. Analyses were performed using R v3.6.2 (http://www.r-project.org/).

RESULTS

We identified 3,583 encounters for ARTIs between the two sites (LAC+USC=1,692; OV-UCLA=1,891), 1,380 (38.5%) of which resulted in antibiotic prescriptions. Table 1 depicts patient characteristics.

Table 1.

Patient characteristics.

Characteristic	LAC+USC	OV-UCLA	Total
Mean Age (SD)	49 (16)	49 (15)a	49 (16)
Female, No. (%)	913 (54%)	1,202 (64%)	2,115 (59%)
Missing gender	219 (13%)	0	219 (6%)
Race/Ethnicity, No. (%)
Hispanic/Latinx	1,025 (61%)	1,168 (62%)	2,193 (61%)
Non-Hispanic white	309 (18%)	322 (17%)	631 (18%)
Asian	98 (6%)	149 (8%)	247 (7%)
Black/African American	72 (4%)	72 (4%)	144 (4%)
Not specified/missing	188 (11%)	180 (9%)	368 (10%)
Total	1,692	1,891	3,583

^aAverage age of probability sample as explained in Methods.

Total Antibiotic Prescribing

Both sites experienced absolute decreases in unadjusted mean rates of total antibiotic prescribing for ARTIs from their respective intervention starts to study end: −13.0% at LAC+USC, −21.4% at OV-UCLA (Figure 3). Key odds of prescribing all types of antibiotics are summarized in Figure 4.

Figure 3.

Unadjusted Antibiotic Prescribing for Acute Respiratory Tract Infections Before and After All Interventions at A) LAC+USC and B) OV-UCLA.

The pre-period refers to the time before any interventions were implemented (T1 at both sites—see Figure 2). The post-period refers to the time after the case-audit feedback (the main intervention) was implemented at each site (T2 at LAC+USC and T3 at OV-UCLA). Error bars depict 95% CIs. Per the analysis plan, tests of statistical significance were not performed on unadjusted antibiotic prescribing rates.

Figure 4.

Adjusted Odds Ratios for Antibiotic Prescribing for Acute Respiratory Tract Infections at A) LAC+USC and B) OV-UCLA.

Odds ratios (ORs) during each time period were obtained from the within-group interrupted-time-series analysis consisting of segmented age- and sex-adjusted logistic regressions on the dependent variable (antibiotic prescribing) clustered by patient.

Results were interpretable as the OR of prescribing antibiotics during a post-intervention period divided by the OR of prescribing antibiotics during the period before all interventions (i.e., ratio of ORs). A ratio of ORs (rOR) equal to 0.60 indicated that the odds of prescribing antibiotics decreased by 40% from the pre-intervention period.

All interventions at LAC+USC were implemented during T2 (see Figure 2); Panel A depicts the change in odds of prescribing antibiotics after all interventions were implemented compared to before any interventions were implemented (i.e., T2 v. T1). At OV-UCLA some interventions were implemented during T2, while case-audit feedback (the main intervention) was implemented during T3; Panel B depicts both the change in odds of prescribing antibiotics after some interventions were implemented (post-partial interventions; T2 v. T1) as well as after all interventions were implemented (post-all interventions; T3 v. T1). Thus, LAC+USC had no “post-partial intervention” period.

Asterisks represent statistical significance by ITS analysis at the following levels: * if P≤.05, ** if P≤.01, *** if P≤.001. Error bars depict 95% CIs.

In the ITS within-group analysis, intervention implementation (starting with one-on-one case-audit feedback) at LAC+USC was associated with a significant decrease in the odds of prescribing antibiotics (rOR, 0.60 [95% CI, 0.39–0.93]; P=.022; Figure 4; Figure 2a, T2 v. T1). At OV-UCLA where one-on-one feedback was the last intervention component implemented, the reduction in the odds of prescribing antibiotics was significant when compared to the time period before any intervention components (rOR, 0.81 [0.70–0.93]; P=.003; Figure 4; Figure 2a, T3 v. T1), but not when compared to the time period immediately preceding one-on-one feedback when some intervention components were already in place (rOR, 0.89 [0.57–1.38]; P=.60; Figure 4; Figure 2a, T3 v. T2).

After one-on-one feedback had been implemented at LAC+USC but not OV-UCLA (i.e., T2 v. T1), antibiotic prescribing rates decreased from 35.9% to 22.9% at LAC+USC and from 48.7% to 45.8% at OV-UCLA. The pre-registered between-group ITS analysis revealed this reduction was not significant between sites (ratio of rORs, 0.66 [0.35–1.24]; P=.20). The within-site ORs for each time period and results of all logistic regressions are available in eTable 3 and eTable 4 in Supplement 2, respectively.

Inappropriate Prescribing

Within OV-UCLA, the odds of inappropriately prescribing antibiotics decreased compared to the period before any interventions were implemented (Figure 4; rOR, 0.72 [0.59–0.88]; P=.001; Figure 2b, T3 v. T1). There were no statistically significant changes in these odds at LAC+USC (Figure 4; rOR, 0.69 [0.39–1.21]; P=.20; Figure 2b, T2 v. T1).

Unintended Consequences

Overall, analyses did not suggest any measurable unintended consequences.

Appropriate Prescribing

The estimated probabilities (Figure 2c), unadjusted rates (Figure 3), and adjusted odds (Figure 4; eTable 4) of appropriately prescribing antibiotics for ARTIs did not significantly decline at either site after intervention implementation.

Diagnosis shifting/gaming

In the pre-registered analysis of unintended consequences (eTable 4), the probability of coding for competing diagnoses (Figure 2d) increased before the interventions (T1 at LAC+USC, T2 at OV-UCLA), but decreased immediately after intervention implementation at both sites (T2 at LAC+USC, T3 at OV-UCLA). These probabilities began to increase in T3 at LAC+USC, but did not exceed pre-intervention probabilities. The probability of coding for non-acute bronchitis (Figure 2e) persistently declined after the interventions.

Antibiotic substitution and shifting

There was no sustained increase in prednisone prescribing for ARTIs to suggest a substitution effect, or evidence that antibiotic prescribing was shifted to other ARTIs not targeted by PRIME (eFigure 2a–b in Supplement 2).

Inappropriately withholding antibiotics

Among 101 manually reviewed encounters for ARTIs, 78 resulted in antibiotic prescriptions (77%). All cases of antibiotic withholding (n=23) were deemed appropriate, suggesting no inappropriate withholding of antibiotics. The electronic measure exhibited a sensitivity of 85% and specificity of 76% comparing HEDIS appropriateness to professional society guidelines (medical chart review). The reasons for discordance between the e-measure and chart review are outlined in eTable 5 in Supplement 2. Four out of 35 (11%) HEDIS-appropriate antibiotic prescriptions conflicted with professional society guidelines for antibiotic use.

DISCUSSION

These multicomponent interventions at two safety-net hospitals were associated with large and statistically significant declines in both the rates and the odds of total antibiotic prescribing for ARTIs. Importantly, these reductions were not associated with any of the five potential unintended consequences assessed: 1) reductions in HEDIS-appropriate antibiotics for ARTIs, 2) diagnosis shifting/gaming, 3) substituting antibiotics with steroids, 4) increases in antibiotics for ARTIs not targeted by the pay-for-performance measure, and 5) inappropriate withholding of antibiotics.

The fact that reductions in HEDIS-inappropriate prescribing were only significant in one analysis at OV-UCLA is likely a reflection of limitations in using claims-based quality metrics. Chart review revealed that up to 11% of HEDIS-appropriate antibiotics were discordant with professional society guidelines. For example, prescribing antibiotics for acute bronchitis in someone with well-controlled HIV is generally inappropriate,^46,47 but the HEDIS-inappropriate metric excludes these cases because of the presence of a diagnostic code for HIV. Reductions in these types of inappropriate prescriptions would be captured by the total antibiotic prescribing metric but not the HEDIS-inappropriate metric, so we believe that decreases in total antibiotic prescribing at both sites captures true reductions in inappropriate prescribing. This interpretation is supported by the fact that HEDIS-appropriate prescribing post-intervention held steady at both sites. In the absence of chart-review evidence of inappropriate antibiotic withholding or any other unintended consequences (which is a concern of many pay-for-performance initiatives),^23–25 our findings suggest that both the pay-for-performance program and ensuing Choosing Wisely^™-based interventions were successful in safely reducing antibiotic prescriptions for ARTIs within real-word safety-net settings.

The results of these interventions that appeared temporally associated with audit and feedback compare favorably to other interventions targeting antibiotic prescriptions for ARTIs, of which as few as 35% report clinically relevant reductions in antibiotic prescriptions.⁴⁸ The effect sizes seen in our results (17% absolute reduction) were at the upper limit of those reported in successful multicomponent interventions,^{10,13,14,33,48} and exceeded what are generally seen for single-component education/feedback interventions (average absolute reduction, 7%).⁴⁴ The large effect size exhibited here may be due to a combination of the magnitude of the incentive payments at stake (up to hundreds of millions of dollars over five years)²⁹ and the use of multicomponent interventions which are generally more effective than single-component interventions.^44,48 The costs of implementing such interventions may be offset through pay-for-performance financial incentive programs.

These findings directly address the paucity of studies testing effectiveness of initiatives to reduce antibiotic prescribing for ARTIs among socioeconomically-disadvantaged safety-net populations,^44,48 which is important given historical problems with the generalizability^20,21 and fairness of applying QI programs to safety-net systems.^22–25

Chart review also revealed that the HEDIS measures of appropriateness exhibited reduced real-world specificity owing to providers failing to code correctly (e.g., not coding for pneumonia when all diagnostic criteria were met), highlighting the shortcomings of using administrative claims data in QI outcomes within safety-net settings.⁴⁹

Limitations

This study had several limitations. First, the study protocol had to be amended to include a within-group ITS analysis (see Methods). While this analysis was post-hoc, ITS is one of the most well-established tools to evaluate within-site changes after an intervention,⁵⁰ so would have been the natural pre-specified analysis had the authors known about the OV-UCLA intervention at the time of study design. Second, this pragmatic study assessed the aggregate effects of the multicomponent interventions rather than the precise effects of each component, and was not designed to report data on intervention fidelity. Third, the HEDIS e-measures did not use clinical data to adjust for illness severity when determining appropriateness and were subject to diagnostic coding errors. Fourth, the unintended consequences analyses did not investigate clinical outcomes such as hospital admissions, which should be studied in future research.

CONCLUSIONS

These real-world multicomponent interventions were associated with substantial reductions in antibiotic prescriptions for ARTIs within a large safety-net system. While inappropriate prescribing decreased at only one site, these interventions did not result in any observed unintended consequences—suggesting that pay-for-performance initiatives can motivate interventions to safely reduce antibiotic prescribing within safety-net settings.

Takeaway Points.

• Quality improvement initiatives, particularly pay-for-performance models, have a track record of translating poorly to safety-net systems and unintentionally harming the medically-underserved populations for which they care.

• In a real-world environment within two hospitals in the nation’s second largest public safety-net healthcare system, we observed a 17.2% absolute reduction in total antibiotic prescribing for viral respiratory infections after multicomponent behavioral interventions were implemented in response to a large pay-for-performance initiative based on Choosing Wisely^™ guidelines.

• Our findings suggest that both pay-for-performance programs and Choosing Wisely^™-based interventions can successfully and safely reduce antibiotic overuse among disadvantaged patients.

Data sharing statement:

Aggregated, deidentified data may be provided upon reasonable request to the corresponding author and with express written permission from the Los Angeles County Department of Health Services (LAC-DHS).