A Learning Collaborative to Improve Antibiotic Prescribing in Primary Care Pediatric Practices

Abstract

An American Academy of Pediatrics State Chapter organized a 6-month, mostly online quality improvement (QI) learning collaborative to improve antibiotic prescribing and patient education for upper respiratory infection (URI) and acute otitis media (AOM). Practices submitted data on quality measures at baseline, monthly, and 4 months post-project. Fifty-three clinicians from six independent, private primary care pediatric practices participated. Use of first-line antibiotics for AOM increased from 63.5% at baseline to 80.4% 4-months post-project. Use of safety-net antibiotic prescriptions (SNAP) for AOM increased from 4.5% to 16.9%. Educating patients about management for URI increased from 66.1% to 88.0% and for AOM from 20.4% to 85.6%. Practices maintained high performance for not prescribing antibiotics for URI (94.4% to 96.2%). Leveraging local relationships and national resources, this replicable antibiotic stewardship project engaged independent private practices to improve patient education for URI and AOM and prescribing and use of SNAP for AOM.

Introduction

Antibiotic use is a major driver of antibiotic resistance, a critical global public health threat.¹ In the United States, most antibiotic use occurs in the outpatient setting² and young children have the highest per capita use.³ Annually, over 65 million outpatient antibiotic prescriptions are dispensed to children. At least 29% of antibiotics prescribed to children are unnecessary⁴ and many more are broader spectrum than recommended.⁵ Of antibiotics prescribed to children in doctor’s offices and emergency departments, 31% are for acute otitis media (AOM) and viral upper respiratory infections (URI).5 Adherence to existing guidelines would improve avoidance of antibiotics for URI, use of safety-net antibiotic prescriptions (SNAP), and antibiotic selection for AOM.⁶ Appropriate antibiotic use is key to limiting antibiotic resistance and avoiding their adverse effects, which lead to an estimated 70,000 emergency department visits by children yearly.⁷ Also, emerging data suggest that antibiotic use in young children may increase long-term risk of autoimmune, allergic, and infectious diseases, likely mediated through microbiome disruption.⁸

Antibiotic stewardship aims to optimize antibiotic use. In 2016, the Centers for Disease Control and Prevention (CDC) released the Core Elements of Outpatient Antibiotic Stewardship, providing a framework for implementing antibiotic stewardship in outpatient practices.⁹ The Core Elements recommends evidence-based interventions, including SNAP, commitment posters, clinical decision support and algorithms, and communication tools like viral prescription pads. Studies evaluating implementation of the Core Elements are lacking,¹⁰ and studies of outpatient antibiotic stewardship have been conducted primarily in academic settings or networks that leverage hospital-based programs and infectious disease specialists.¹¹ However, physicians in small independent practices prescribe higher volumes of antibiotics than those in larger group practices,¹² as do those in community practices compared to academic settings.¹³

The American Academy of Pediatrics (AAP) Chapter Quality Network (CQN) designed this quality improvement (QI) project to address antibiotic prescribing in pediatric primary care practices through collaboration with an AAP chapter and a large health insurer and using the Core Elements. CQN’s previous success in effecting improvements in care through learning collaboratives¹⁴ and the evidence underlying the Core Elements guided the design. Partnership with the insurer aimed to leverage the influence of an existing quality incentive payment program. Our objectives were to improve antibiotic prescribing and patient education for AOM and URI in participating practices. Through this report we share our outcomes, lessons learned, and resources to support similar projects in other settings.

Methods

The project was funded by a CDC cooperative agreement with the AAP, which engaged Anthem Blue Cross Blue Shield as an un-funded partner. Anthem’s Enhanced Personal Health Care program (EPHC) offers shared savings to enrolled clinicians who meet quality metrics in a provider performance scorecard and achieve cost savings for the measurement period. The Virginia AAP Chapter leadership recruited practices from among those involved in the EPHC program. For full participation, physicians were offered performance improvement continuing medical education credit, American Board of Pediatrics Maintenance of Certification Part 4 (MOC) credit, and bonus points on their EPHC scorecard, which contributed up to 5% of the year’s score.

In October 2016, the AAP convened two CDC medical epidemiologists, a primary care pediatrician, a QI specialist, two Anthem representatives, CQN staff, and AAP leaders to design the project aims, measurement strategy, key drivers, and interventions. The aims and measures aligned with AAP clinical reports¹⁵ and recommendations. The AAP’s institutional review board determined that the project protocols met federal exemption criteria as QI.

Interventions

The learning collaborative, conducted April-September 2017, was modeled after the Institute for Healthcare Improvement’s Breakthrough Series¹⁶ and supported by the CQN staff, QI specialist, pediatrician, and Chapter Executive Director.

Each practice identified a QI team, including a physician leader, nurse or medical assistant, and practice administrator, to lead improvement activities and participate in webinars and learning sessions. The teams enrolled additional practice clinicians as project participants.

An initial in-person learning session included an overview of appropriate antibiotic prescribing and the Core Elements, review of project measures, and training on QI methods and tools, including SMART (specific, measurable, achievable, relevant, time-bound) aims, the Model for Improvement (Plan-Do-Study-Act [PDSA] cycles),¹⁷ PDSA worksheets, and process mapping. Sample interventions were discussed, including posting practices’ commitment to use antibiotics only when needed,¹⁸ using “viral prescriptions” to guide symptomatic management,¹⁹ and using SNAPs for AOM. Teams developed their own SMART aims, process maps, and initial PDSA plans. Teams took these tools back to their practices to build consensus, train other clinicians, and begin testing. Absent sufficient research-based evidence to guide goal-setting for most measures, we asked practices to set their own goals after receiving baseline performance reports.

Monthly performance data were distributed to practice teams as chapter- and practice-level reports and discussed, along with their PDSA cycles, during 60-minute webinars. Educational and clinical resources, including the CDC’s,²⁰ were shared through a collaboration website. In June and August, online learning sessions included discussion of research on effective interventions and case studies. Between webinars, the QI coach supported teams on PDSA cycles and data collection via email and phone. In discussions on sustaining improvements, the importance of systems-level change, developing and implementing policies, and ongoing measurement and feedback were addressed.

Measures

The overall aim was “From April 2017 to September 2017, we will use QI methods to implement AAP recommendations and CDC Core Elements to achieve measurable improvements in antibiotic prescribing for AOM, avoidance of antibiotic prescribing for URI, and communication practices with patients and caregivers regarding antibiotic prescribing.”

The project measures were:

1. Percent of patients ≥3 months diagnosed with URI and no competing diagnosis who were not prescribed an antibiotic;

2. Percent of patients ≥3 months diagnosed with URI and no competing diagnosis who were provided appropriate education regarding treatment for URI;

3. Percent of patients ≥6 months and <2 years diagnosed with AOM who were prescribed an antibiotic;

4. Percent of patients ≥2 years diagnosed with AOM who were prescribed an antibiotic;

5. Percent of antibiotic prescriptions for patients ≥6 months diagnosed with AOM that were written as a SNAP;

6. Percent of antibiotic prescriptions for patients ≥6 months diagnosed with AOM that were written for a 1^st-line antibiotic (amoxicillin or amoxicillin-clavulanate);

7. Percent of patients ≥6 months diagnosed with AOM who were provided appropriate education regarding symptom management.

Measures 3 and 4 were balancing measures and not targets for improvement.

Data Collection

Practices collected three months of baseline data on each measure for each clinician, though this was optional for the education and SNAP measures because no practices had consistently documented prior performance. Practice teams were provided lists of ICD-10 codes for URI, AOM, and competing diagnoses, query parameters, and instructions for randomizing chart pulls and performing reviews. For each measure, all visits with a relevant diagnosis were eligible except those that also had a competing diagnosis for which antibiotics could be warranted. See Supplement for these resources.

Practice teams entered their baseline, monthly, and post-project (collected 4-months after the project ended) data, obtained through electronic health record (EHR) query or chart review, into the AAP’s Quality Improvement Data Aggregator (QIDA), a web-based data collection and reporting system. For measures collected by EHR query, data on all eligible visits were entered as aggregate numbers for each clinician. For those done by chart review, data from 10 randomly selected eligible visits for both URI and AOM were entered. If fewer than 10 patients were seen for a given diagnosis that month, data for all visits were entered.

Quantitative and qualitative feedback on progress, challenges, and project management were collected through practice-level surveys in June and September. Project leaders reviewed responses to monitor progress and guide project management.

Data Analysis

Run charts were used to illustrate baseline and subsequent monthly data. Robust generalized estimating equations (GEE) with a Poisson distribution and log link function were used to generate relative risk ratios (RRRs) comparing measure rates at each month relative to baseline. For each measure, data were aggregated across three baseline months by summing the numerators and denominators. Analyses, conducted at the physician level, accounted for within-subject dependency. Models included the number of patient visits meeting criteria for each measure as the outcome, with an offset variable equal to the log number of eligible patient visits. Statistical tests were conducted in SAS 9.4 (SAS Institute, Cary, NC).

Results

Seven independent, private primary care pediatric practices in Virginia agreed to participate; one withdrew prior to project start. 53 clinicians from six practices with a total of 16 participating office sites completed the project. On average, 63% of the practices’ clinicians participated (range by practice was 35%−93%; 5–13 clinicians), 79% of whom earned MOC credit. All had participated in the EPHC program for 1–3 years. Table 1 details the practices’ characteristics.

Table 1.

Practice characteristics

	Practice A	Practice B	Practice C	Practice D	Practice E	Practice F	Average across Practices
Total No. of Clinicians	14	11	20	11	13	16	14.2
No of Participating Clinicians*	13	10	7	5	12	6	8.8
Setting	All were Suburban
Payer Mix
Private Insurance, Traditional Fee-for-Service	57%	77%	65%	--	75%	95%	61.5%
Private Insurance, Managed Care (HMO, IPA, PPO, etc.)	40%	3%	30%	95%	9%	--	29.5%
Public Insurance, Fee-for-Service (Medicaid, SCHIP, etc.)	--	11%	--	3%	--	--	2.3%
Public Insurance - Managed Care Medicaid, CHIP	1%	4%	--	--	15%	--	3.3%
TRICARE (military)	--	4%	3%	2%	--	3%	2.0%
Uninsured	2%	1%	2%	--	1%	3%	1.5%

^*Number of clinicians who participated is based on the number of Medical Doctors (MD), Doctors of Osteopathy (DO), Physician Assistants (PA), and Nurse Practitioners (NP) who enrolled and collected data during the project.

Practices planned and implemented improvements using PDSA cycles, guided by learning and sharing during learning sessions and webinars (see Figure 1), which were attended by one to three team members from each practice. Though a few practices struggled to reach consensus, each set performance goals for each measure. Mean goals across practices are reflected in the run-charts.

Figure 1. CQN Judicious Use of Antibiotics Learning Collaberative Structure.

To enable electronic query for all measures, two practices developed drop-down menus or used referral codes to record data not previously documented in their EHRs, including use of SNAPs and provision of education; other practices reported data from electronic query and manual record review. Practices reported baseline data from December 2016 through February 2017, monthly data from April through September 2017, and post-project data from January 2018.

Performance on quality measures

See Figures 2–8 for run charts of aggregate performance across practices on each measure; see the Supplement for practice-level run charts. Table 2 presents results of the statistical analyses.

Figure 2.

Percent of patients with URI (upper respiratory infection) not prescribed an antibiotic, aggregate performance. *Patients ≥3 months and no competing diagnosis.

Figure 8.

Percent of patients with acute otitis media (AOM) who were educated on appropriate treatment for AOM, aggregate performance. *For all patients ≥6 months; education included symptomatic management and when to fill a safety-net antibiotic prescription (SNAP) or follow-up with the office.

Table 2.

RRRs for quality measures during the project period and post-project relative to baseline^a

	% of patients with URI not prescribed an antibiotic	% of patients with URI provided appropriate education	% of patients < 2 years with AOM prescribed an antibioticb	% of patients ≥ 2 years with AOM prescribed an antibioticb	% of antibiotic prescriptions for AOM written as SNAP	% of antibiotic prescriptions for AOM that were for 1st-line antibiotics	% of patients with AOM provided education on symptom management
	RRR (95% CI)	RRR (95% CI)	RRR (95% CI)	RRR (95% CI)	RRR (95% CI)	RRR (95% CI)	RRR (95% CI)
Baselinec	Referent	Referent	Referent	Referent	Referent	Referent	Referent
April 2017	0.99 (0.95–1.04)	1.03 (0.80–1.33)	1.08 (0.97–1.20)	1.10 (1.01–1.20)	2.30 (1.33–3.97)	0.99 (0.88–1.12)	1.48 (1.06–2.07)
May 2017	1.00 (0.95–1.05)	1.09 (0.84–1.40)	1.11 (0.998–1.24)	1.13 (1.04–1.22)	3.42 (2.12–5.51)	1.03 (0.92–1.15)	3.75 (2.93–4.80)
June 2017	1.02 (0.98–1.06)	1.34 (1.05–1.64)	1.01 (0.89–1.14)	1.03 (0.94–1.13)	5.16 (3.25–8.19)	1.12 (0.99–1.27)	4.06 (3.15–5.24)
July 2017	1.02 (0.97–1.07)	1.37 (1.09–1.72)	1.14 (0.98–1.31)	1.13 (1.02–1.25)	4.74 (2.86–7.85)	1.17 (1.02–1.33)	4.04 (3.04–5.35)
August 2017	1.03 (0.98–1.08)	1.34 (1.07–1.67)	1.17 (1.01–1.36)	1.08 (0.97–1.21)	4.77 (2.81–8.11)	1.08 (0.93–1.25)	4.07 (3.02–5.47)
Sept. 2017	1.02 (0.98–1.06)	1.30 (1.09–1.56)	1.16 (1.01–1.34)	1.14 (1.03–1.27)	5.44 (3.34–8.85)	1.25 (1.10–1.42)	4.24 (3.20–5.61)
Post-projectd	1.02 (0.98–1.05)	1.32 (1.11–1.57)	1.08 (0.97–1.20)	1.14 (1.04–1.24)	3.76 (2.35–5.99)	1.27 (1.14–1.41)	4.20 (3.31–5.33)

^aStatistically significant results in bold

^bStatistically significant results not bolded for these measures because they were not targets for improvement

^cBaseline data collected from December 2016-February 2017

^dPost-project data collected in January 2018.

Aggregate performance for the three baseline months on percent of patients with URI not prescribed an antibiotic was 94.4% and the small increase to 96.2% post-project was not significant (RRR 1.02; 95% CI 0.98–1.05). Four practices maintained performance at or near 100%.

Aggregate performance on percent of patients with URI provided appropriate education increased from 66.5% to 88.0% (RRR 1.32; 95% CI 1.11–1.57). Three practices began with performance at or near 100%; one remained at 100% and the others dipped to 98.2% and 88.9%. Two practices that began at or near 0%, due to lack of documentation, increased to 28.8% and 87.9%.

The percent of antibiotic prescriptions for AOM written as SNAP increased from 4.5% to 16.9% post-project (RRR 3.76, 95% CI 2.35–5.99), though the percentage had reached 24.5% at project-end.

The percent of antibiotic prescriptions for AOM written for 1^st-line antibiotics increased from 63.5% to 80.4% (RRR 1.27; 95% CI 1.14–1.41). The percent of patients with AOM provided appropriate education about symptom management increased from 20.4% to 85.6% (RRR 4.20, 95% CI 3.31–5.33). Only 3 practices were regularly documenting such education at baseline (41.9%, 57.0%, and 91.8%). Percentages increased for all practices.

Balancing Measures

The percent of patients with AOM prescribed an antibiotic (including SNAPs) increased for patients ≥2 years (79.3% to 90.2% [RRR 1.14; 95% CI 1.04–1.24]) but did not for patients <2 years (77.1% to 83.1% [RRR 1.08; 95% CI 0.97–1.20]).

Practice-level surveys

Survey responses indicated that education by clinical experts, knowledge of evidence-based guidelines, and awareness of their own performance helped change mindsets, build confidence, and support improvement. Several commented on the difficulty of changing old habits.

Discussion

We report on efforts to improve antibiotic prescribing across independent, private primary care pediatric practices, led by the AAP Virginia Chapter with assistance from AAP CQN staff and CDC antibiotic stewardship experts and with collaboration of a health insurer. Participating practices did not improve from their outstanding baseline performance in not prescribing for URI. The project led to significant and sustained increases in use of SNAPs and first-line antibiotics for AOM and in provision of appropriate education to families regarding URI and AOM – key objectives in reducing unnecessary and inappropriate antibiotic use. This project offers a model for engaging private practices in antibiotic stewardship, independent of academic centers and large health systems.

For AOM, our primary intervention aimed to increase use of SNAP, with the goal of decreasing the number of children taking antibiotics. Evidence suggests that, in most children, non-severe AOM will resolve without antibiotics^21,22 and AAP guidelines recommend offering observation and close follow-up for non-severe unilateral AOM in children 6–23 months of age and for non-severe uni- or bilateral AOM in those >23 months.²³ We observed an unexpected increase of 11% in the proportion of children ≥2 years with AOM who were prescribed an antibiotic. However, between baseline and post-project we also observed a 21.8% reduction in the number of children diagnosed with AOM across both age groups (697 to 545) and a 46.5% increase in the number of children diagnosed with URI (906 to 1327); see Figures 2 and 7 for monthly diagnosis numbers. That finding suggests a shift toward coding less severe illness, for which no antibiotics were prescribed, as URI instead of AOM. Because only half of the practices reported on all visits for each diagnosis (the others reporting on samples thereof), we lacked the data needed to test that hypothesis.

Figure 7.

Percent of prescriptions for acute otitis media (AOM) that were written for a first-line antibiotic, aggregate performance. *For all patients ≥6 months.

Previous studies of patients receiving SNAPs for AOM demonstrated that a minority (31–38%) filled those prescriptions.^24,25,26 It has also been shown that 63% of parents whose children received SNAPs for AOM were willing to treat future episodes with pain medication alone.²¹ These findings suggest that use of SNAPs for AOM can reduce antibiotic use both for the current illness and potentially for future episodes.

Only 2 practices reported PDSA cycles on increasing the proportion of AOM prescriptions written for 1^st-line antibiotics. Nevertheless, overall improvement from 63.5% to 80.4% post-project was achieved. Because we accepted amoxicillin/clavulanate as 1^st-line (rather than adjust for its use because of recent amoxicillin failures), the optimal goal would have been 90%, allowing for 10% of patients reporting allergy to penicillins.⁵

Performance at baseline on educating patients on appropriate treatment of URIs and AOM was low but improved to above 86% by project-end for both, though most practices indicated the improvement was primarily in documentation. Several practices reported valuable consensus-building around how to deliver “appropriate education,” including use of “viral prescriptions,” discussion of SNAPs and when to fill them, and implementing easier EHR documentation.

Anthem participation

Anthem’s interest was to explore metrics for assessing pediatric practice performance. For its EPHC program, Anthem had found few established metrics to be useful for pediatrics due to inadequate volumes at the provider/practice level, high baseline performance, or inapplicability to pediatric care. Anthem wanted to learn about practice-generated data that could support more meaningful scorecards to assess and compare the quality of pediatric practice.

Because EPHC program payments are based on quality metrics and savings, improvements during this project contributed to the next year’s payments if savings were achieved. We could not assess whether or how much Anthem’s involvement contributed to the improvements made by project participants. The small contribution to a complex scorecard and that payments were delayed and dependent on measured savings for the year may have limited incremental motivation for participants. Anthem may adapt a project measure, perhaps use of SNAP, for future EPHC pediatric scorecards.

Limitations and Strengths

The participating practices all delivered primary care in suburban areas of Virginia and their patients were largely insured by commercial plans, limiting generalizability. Such practices may be less concerned about follow-through or loss to follow-up than those in urgent care, retail-based, or lower socioeconomic settings. All practices participated in Anthem’s EPHC program, volunteered for the project, openly shared performance data, and their aggregate baseline performance on prescribing for URI exceeded national averages. These may reflect more desire and willingness to improve quality than practices in general.

The resources available to participants — CDC funding, national expertise, AAP Chapter leadership, insurer engagement — likely exceed those available to most other practices wishing to improve antibiotic stewardship. However, our methods, lessons learned, and resources are available to support and lower the cost of future projects conducted by local or national organizations, practice groups, or individual practices (see Supplement).

Three practices were unable to query needed data from their EHRs, limiting our ability to assess coding shifts suggested by numbers of diagnoses of URI and AOM that changed in opposite directions.

Strengths of the project included that, despite the short project duration, data collected 4 months post-project (in January 2018, twelve months from the mid-baseline month) demonstrated sustained improvements.

Lessons learned

Participants deemed access to clinical expertise, training on practice guidelines, and feedback on performance the most valuable project components. To project leaders, sharing PDSA efforts, reviewing data, and sharing problems and solutions seemed key to practices embracing QI and implementing improvements. The CDC’s educational resources,¹⁸ sometimes customized by practices, were useful in communicating with families about antibiotics. Queryable data elements in some practices’ EHRs enabled measuring performance across all eligible visits, eliminating manual record review and potentially supporting ongoing periodic measurement. Setting project-wide goals would likely have been more effective than having practices set their own. Potential performance-based payment was not a strong driver, perhaps because of the small scorecard contributions and delay in receipt. Collecting data to assess shifts in coding practices may be valuable. And QI in the “real world” introduces variables that can be challenging – one participating practice changed practice organization/ownership and their EHR during the project. The project resources described in Methods and elsewhere and a “change package” developed by the AAP based on this and a related project are compiled in the Supplement to help others implement similar projects.

Conclusions

Improvements in use of 1^st-line antibiotics and SNAP for AOM and in patient education for URI and AOM were achieved through a mostly-virtual learning collaborative that leveraged resources of the AAP and the Virginia Chapter. Local and national organizations (professional, academic, public health) could engage available expertise, existing resources (see Supplement and the CDC’s Core Elements), relationships, clinicians’ motivation, and QI methods to conduct similar projects at low cost to spread antibiotic stewardship to other independent private practices.

Figure 3.

Percent of patients with URI (upper respiratory infection) provided appropriate education on treatment for URI, aggregate performance. *Patients ≥3 months and no competing diagnosis; education included symptomatic treatment, avoidance of antibiotics, and appropriate follow-up.

Figure 4.

Percent of patients ≥6 months and <2 years with acute otitis media (AOM) who were prescribed an antibiotic, aggregate performance. *Include SNAPs (safety-net antibiotic prescription); this was not a quality measure, used to assess overall impact of SNAP use.

Figure 5.

Percent of patients ≥2 years with acute otitis media (AOM) who were prescribed an antibiotic, aggregate performance. *Include SNAPs (safety-net antibiotic prescription); this was not a quality measure, used to assess overall impact of SNAP use.

Figure 6.

Percent of antibiotic prescriptions for acute otitis media (AOM) that were written as a SNAP (safety-net antibiotic prescription), aggregate performance. *For all patients ≥6 months.

Abbreviations:

AAP

American Academy of Pediatrics

AOM

Acute Otitis Media

CDC

Centers for Disease Control and Prevention

CQN

Chapter Quality Network

HER

Electronic Health Record

EPHC

Enhanced Personal Health Care program

MOC

Maintenance of Certification

PDSA

Plan-Do-Study-Act

QI

Quality Improvement

SMART

specific, measurable, achievable, relevant, time-bound

SNAP

Safety-Net Antibiotic Prescription

URI

Upper Respiratory Infection