Sustainability of Interventions to Increase Guideline-Concordant Durations of Antibiotic Therapy for Children with Acute Otitis Media

Abstract

Two low-cost pragmatic interventions (change in the options in the electronic health record; change in the electronic health record plus education plus feedback comparing prescribing with peers) to improve prescribing of guideline-concordant short antibiotic durations for children 2 years and older with uncomplicated acute otitis media were highly effective and results were sustained 18 months after discontinuation of the active components of the interventions.

For children 2 years of age and older with acute otitis media (AOM), short durations of antibiotics (5 days) have similar effectiveness as long durations (10 days) of antibiotics.¹ Thus, national guidelines by the American Academy of Pediatrics recommend that when antibiotics are prescribed, most children in this age group be prescribed a short duration.² Despite this recommendation, we previously reported that more than 95% of prescriptions for children 2 years and older in the US are for 10-day durations, amounting to nearly 41% of antibiotic exposure days that are potentially unnecessary.^3,4 Unnecessarily long antibiotic durations directly harm children by increasing the risk of gastrointestinal adverse drug events¹ and may increase the risk for chronic diseases,⁵ such as juvenile idiopathic arthritis,⁶ later in life.

To address this problem, we implemented 2 low-cost pragmatic interventions to promote prescribing of institutional guideline–concordant short durations (5 days) for children 2 years and older with uncomplicated AOM.⁷ The interventions included an electronic health record (EHR)-only intervention with changes to prescription fields for commonly prescribed antibiotics and a bundled intervention that included prescription field changes along with education for clinicians at 2 sessions and monthly individualized feedback to clinicians on how their prescribed durations compared with their peers for 6 months. Prescription field changes included modifying quick select buttons for duration from 10, 14, and 21 days to 5 and 10 days, adding brief help text summarizing current institutional recommendations for duration, and adding a hyperlink to institutional clinical care pathways. After 6 months, EHR changes were continued, but education sessions and feedback reports were discontinued. Both interventions initially significantly increased prescribing of short durations (49% EHR and 75% bundled), had high fidelity, and were well liked by clinicians. In this evaluation, we aimed to determine the sustainability of each intervention 18 months after clinician education and feedback reports were discontinued.

Methods

This was a quasi-experimental evaluation of the sustainability of 2 pragmatic interventions to increase guideline-concordant antibiotic durations for children 2 years of age and older with uncomplicated AOM. The interventions occurred at Denver Health and Hospital Authority (DH), in Denver, Colorado, from February to August 2020 and included 3 pediatric and 6 family medicine clinics. Details of the interventions, including materials that can be freely used and modified, and initial results were published previously.⁷ During the intervention phase, EHR prescription field changes were implemented and clinicians in the bundled cohort also received 2 education sessions and monthly feedback reports that compared their prescribing with their peers. The EHR-only intervention was implemented in family medicine clinics and the bundled intervention in pediatric clinics. At the start of the postintervention phase, feedback reports were discontinued and no further education sessions were completed. EHR changes remained active. The current analysis includes an evaluation of the interventions over the 3 time periods: preintervention (February 2019-January 2020), intervention (February-August 2020), and postintervention (September 2020-March 2022). We used identical methodology to generate the postintervention cohort and data as was used in the initial publication.⁷

The primary outcome measure was antibiotic prescriptions for children 2 years and older with uncomplicated AOM with an institutional guideline–concordant 5-day duration of therapy. In accordance with the original evaluation, patients aged 2–18 years were included if they had a qualifying International Classification of Diseases, Tenth Revision, code qualifying diagnosis and were prescribed a systemic antibiotic (Table I; available at www.jpeds.com). Patients who had received a systemic antibiotic or had AOM within the previous 30 days, had tympanostomy tubes, or had a competing bacterial diagnosis, such as pneumonia, were excluded. Patients who were prescribed azithromycin or given ceftriaxone also were excluded (<1.5% of patients).

Table I.

International Classification of Diseases, Tenth Revision, codes for acute otitis media

H65.00–.07

H65.191–.199

H66.001–.009

H66.40–.43

H66.90–.93

H67.1–.3

H67.9

Difference-in-differences analysis was used to compare the change in guideline-concordant durations during the preintervention, intervention, and postintervention periods in the EHR-only and bundled intervention groups.⁸ To assess for parallel trends in the preintervention period, we used a placebo difference-in-differences regression using only the preintervention data and did not find a significant preintervention trend (P = .09). To account for potential differences in the patient populations between intervention groups we calculated propensity scores based on age, sex, insurance, race, and ethnicity. Factors included in propensity score calculations were determined a priori. Propensity scores were then used to weight the regression model while controlling for covariates (Table II; available at www.jpeds.com).

Table II.

Adjusted DiD regression estimates

Factors	Estimates	SE	Wald 95% CIs		P value
Adjusted DiD for preintervention vs postintervention
Time	0.518	0.031	0.457	0.580	<.0001
Intervention	−0.044	0.025	−0.094	0.005	.081
Time*intervention	0.262	0.046	0.172	0.352	<.0001
Age	0.009	0.003	0.003	0.014	.002
Race
Black	0.050	0.045	−0.039	0.138	.269
White	−0.005	0.032	−0.068	0.058	.880
Other (ref)
Ethnicity
Hispanic	−0.005	0.028	−0.059	0.050	.872
Insurance
Commercial	0.011	0.090	−0.166	0.188	.904
Public	0.056	0.085	−0.111	0.222	.513
Uninsured	0.053	0.119	−0.181	0.286	.658
Other (ref)
Sex
Female	−0.019	0.021	−0.060	0.022	.369
Adjusted DiD for intervention vs postintervention
Time	0.015	0.051	−0.086	0.115	.776
Intervention	0.219	0.056	0.110	0.328	<.0001
Time*Intervention	−0.001	0.071	−0.141	0.138	.985
Age	−0.008	0.005	−0.017	0.001	.094
Race
Black	0.106	0.078	−0.046	0.258	.172
White	0.037	0.053	−0.067	0.140	.489
Other (ref)
Ethnicity
Hispanic	0.035	0.046	−0.055	0.124	.449
Insurance
Commercial	0.029	0.142	−0.249	0.306	.841
Public	0.105	0.133	−0.157	0.366	.432
Uninsured	0.377	0.194	−0.003	0.756	.052
Other (ref)
Sex
Female	0.008	0.035	−0.060	0.076	.819

DiD, difference in differences.

Balance measures included treatment failure and recurrence. Treatment failure was defined as being prescribed another antibiotic for AOM (same or different) between 3 and 14 days after the initial encounter and recurrence was defined as being prescribed another antibiotic for AOM between 15–30 days and the initial encounter.^7,9,10 Rates were compared between time periods using Fisher exact tests. Significance was defined as alpha = 0.05 using 2-tailed tests. Analyses were conducted in SAS Enterprise Guide 7.1 (SAS Institute).

The project was approved by the Quality Improvement Review Committee of DH, which is authorized by the Colorado Multiple Institute Review Board at the University of Colorado and the DH Ethics Committee and was determined not to be human subjects research, as it constituted program evaluation.

Results

A total of 1376 encounters for AOM over the 3 time periods met the inclusion criteria, of which 649 occurred in clinics that received the bundled intervention and 727 occurred in clinics that received the EHR-only intervention. Overall, most patients identified as Hispanic/Latinx (71.5%) and had public insurance (85.7%; Table III; available at www.jpeds.com). A total of 57 clinicians were included in the EHR-only intervention and 32 in the bundled intervention. In the postintervention period, there were 22 new clinicians in the EHR-only and 7 new clinicians in the bundled intervention clinics; 24 clinicians in the EHR-only intervention and 6 clinicians in the bundled intervention clinics departed before the postintervention phase.

Table III.

Demographic characteristics of eligible patients with AOM during the study periods by intervention

	EHR-only intervention (n = 727)			Bundled intervention (n = 649)
	Preintervention	Intervention	Postintervention	Preintervention	Intervention	Postintervention
Demographic characteristics	n (%) n = 409	n (%) n = 105	n (%) n = 213	n (%) n = 388	n (%) n = 115	n (%) n = 146
Age mean, y (SD)	5.5 (3.9)	6.2 (4.2)	4.9 (3.6)	5.8 (3.8)	6.0 (4.0)	5.1 (3.6)
Sex
Male	200 (48.9)	48 (45.7)	119 (55.9)	194 (50.0)	51 (44.3)	71 (48.6)
Female	209 (51.1)	57 (54.3)	94 (44.1)	194 (50.0)	64 (55.7)	75 (51.4)
Race
Black	54 (13.2)	4 (3.8)	31 (14.6)	45 (11.6)	10 (8.7)	22 (15.1)
White	303 (74.1)	87 (82.9)	161 (75.6)	288 (74.2)	88 (76.5)	102 (69.9)
Other	52 (12.7)	14 (13.3)	21 (9.9)	55 (14.2)	17 (14.8)	22 (15.1)
Ethnicity
Non-Hispanic	123 (30.1)	20 (19.0)	64 (30.0)	105 (27.1)	33 (28.7)	53 (36.3)
Hispanic	286 (69.9)	85 (81.0)	149 (70.0)	283 (72.9)	82 (71.3)	93 (63.7)
Insurance
Commercial	35 (8.6)	12 (11.4)	28 (13.1)	38 (9.8)	14 (12.2)	25 (17.1)
Public	355 (86.8)	89 (84.8)	179 (84.0)	343 (88.4)	98 (85.2)	116 (79.5)
Uninsured	8 (2.0)	2 (1.9)	4 (1.9)	3 (0.8)	0 (0.0)	3 (2.1)
Other	11 (2.7)	2 (1.9)	2 (0.9)	4 (1.0)	3 (2.6)	2 (1.4)

In the unadjusted difference-in-differences model, guideline-concordant prescribing rates among the EHR-only clinics increased from 14.4% during the preintervention period to 66.7% during the postintervention period, an additional 2.9% increase from the intervention period (difference = 52.3%, Table IV). In the clinics with the bundled intervention, guideline-concordant prescribing rates increased from 10.6% during the preintervention period to 87.7% during the postintervention period, an additional 2.5% increase from the intervention period (difference = 77.1%). The unadjusted difference-in-differences estimator attributed an additional 24.9% (P < .01) improvement for the bundled intervention compared with the EHR-only intervention between the pre- and postintervention time periods (Table IV, Figure).

Table IV.

Unadjusted and adjusted analysis of primary outcome measure (5-day duration of therapy) by time period

Interventions	Preintervention	Intervention	Difference	Unadjusted DiD estimate	P value	Adjusted DiD estimate	P value
EHR-only	59/409 (14.4%)	67/105 (63.8%)	49.4%	25.3%	<.01	26.4%	<.01
Bundled	41/388 (10.6%)	98/115 (85.2%)	74.6%
Interventions	Preintervention	Postintervention	Difference	Unadjusted DiD estimate	P value	Adjusted DiD estimate	P value
EHR-only	59/409 (14.4%)	142/213 (66.7%)	52.3%	24.9%	<.01	26.2%	<.01
Bundled	41/388 (10.6%)	128/146 (87.7%)	77.1%
Interventions	Intervention	Postintervention	Difference	Unadjusted DiD estimate	P value	Adjusted DiD estimate	P value
EHR-only	67/105 (63.8%)	142/213 (66.7%)	2.9%	−0.4%	.96	−0.1%	.99
Bundled	98/115 (85.2%)	128/146 (87.7%)	2.5%

DiD, difference in differences.

Figure.

Guideline-concordant antibiotic durations prescribed by intervention and time period.

Using inverse probability weights and controlling for covariates, the adjusted difference-in-differences regression estimated that the bundled intervention improved guideline-concordant prescribing by 26.2% compared with the EHR-only intervention between the pre- and postintervention periods (P < .01). This was nearly identical to the estimator between the preintervention and intervention periods, decreasing only 0.1% (Table IV). Raw data overlaid with estimated rates for the EHR-only and bundled interventions for each time period, based on the regression model, can be seen in the Figure.

The balance measures of treatment failure and recurrence remained uncommon (failure <2% and recurrence <3%) throughout all time periods. These did not significantly differ between either the preintervention and intervention periods or the intervention and postintervention periods (Table V; available at www.jpeds.com).

Table V.

Fisher exact test results for balance measures

	EHR-only intervention (n = 727)
	Preintervention	Postintervention		Intervention	Postintervention
	No. (%), n = 409	No. (%), n = 213	P value	No. (%), n = 105	No. (%), n = 213	P value
Failure	5 (1.2)	4 (1.9)	.50	2 (1.9)	4 (1.9)	1.0
Recurrence	6 (1.5)	2 (0.9)	.72	1 (0.9)	2 (0.9)	1.0
	Bundled intervention (n = 649)
	Preintervention	Postintervention		Intervention	Postintervention
	No. (%) n = 388	No. (%) n = 146	P value	No. (%) n = 115	No. (%) n = 146	P value
Failure	4 (1.0)	2 (1.4)	.67	2 (1.7)	2 (1.4)	1.0
Recurrence	5 (1.3)	3 (2.1)	.46	3 (2.6)	3 (2.1)	1.0

Discussion

Both EHR-only and bundled interventions substantially increased prescribing of guideline-concordant antibiotic durations for AOM, and the change in prescribing was sustained 18 months after discontinuation of clinician education and feedback reports to clinicians. The changes were sustained despite high clinician turnover rates in the clinics. The bundled intervention remained 26% (absolute percentage) more effective than the EHR-only intervention. Over a 2-year evaluation period, treatment failure and recurrence rates were low and did not increase with either intervention.

Antimicrobial stewardship interventions have been shown to reduce the development of antimicrobial resistance, improve patient outcomes, and reduce cost.¹¹ Multifaceted interventions based on the Centers for Disease Control and Prevention Core Elements of Outpatient Stewardship, like the bundled intervention, have consistently shown greater effectiveness than those using only a single modality.¹¹ Un-fortunately, cost and resources required for implementation and sustainability have limited the scalability of interventions. For other interventions, discontinuation of education and/or feedback reports to clinicians has resulted in a return to baseline prescribing.¹² We speculate that there are 3 reasons why the interventions we implemented were highly effective and sustainable with no additional resource usen after the intervention period. First, the interventions aligned with the values of patients. In a parallel project, we found that 86% of parents preferred the shortest duration of antibiotics needed or whatever duration was recommended by the clinician.¹³ Although parents were more reluctant to forgo antibiotics entirely, they strongly preferred shorter durations. This has important implications for other diagnoses such as skin and soft-tissue infections,¹⁴ community-acquired pneumonia,¹⁵ and urinary tract infections,¹⁶ which also may be effectively treated with short durations. Second, the interventions aligned with the preferences and values of clinicians. Optimally, clinicians would not prescribe antibiotics when not needed; however, obtaining clinician buy-in for interventions to reduce immediate antibiotic use can be fraught with challenges. We previously reported that clinicians were more comfortable prescribing short durations than not prescribing an antibiotic or prescribing a different agent. Likely, prescribing shorter durations requires less time educating families than not prescribing an antibiotic or writing a delayed prescription, and clinicians worry less about loss to follow-up for patients who worsen or do not improve. Clinicians in our safety-net health system also worried about inability of families to fill prescriptions written as delayed due to poor access to transportation or pharmacy closures over the weekend. Interventions focused on duration may serve as a bridge to increase clinician buy-in for interventions to reduce immediate antibiotic use. In addition, we were purposeful in our implementation strategy to focus on an overall goal of improving care for patients rather than improving metrics or reducing cost, which harmonized with the goals of clinicians. Third, the interventions aligned with the normal work-flow of clinicians and reduced the time it took to order and document guideline-concordant treatment. EHR changes were built directly into prescription fields and did not require clinicians search for or use specialized EHR tools, such as order sets. Changes could be saved as favorites to simplify future ordering of guideline-concordant antibiotic durations by age. Feedback reports went directly to clinician electronic mail accounts and review of reports was automatically tracked for American Board of Pediatrics Maintenance of Certification credit. Clinicians did not need to access reports on the institution website or spend time finding or documenting review of reports.

Scalability and sustainability of interventions are arguably as important as effectiveness. Although several interventions have been demonstrated to be effective in clinical trials or observational studies, as a field we have not effectively disseminated these interventions. Low-cost, pragmatic interventions that can be implemented across specialties and settings are needed. Such interventions are likely to be more sustainable than costly or complex interventions. Although the bundled intervention was more effective, the EHR-only intervention substantially improved care and could be a pragmatic tool for health systems without the ability to implement the bundled intervention components. The bundled intervention could be scaled with minimal investment by making education sessions virtual, recorded, and open access to other organizations; sharing clinical care pathways between organizations; automating feedback reports and sharing statistical software codes for generation of reports (OASIS code is freely available for this project)¹⁷; and creating large-scale continuing medical education and maintenance of certification programs. The more we collaboratively share effective tools the faster, we are likely to see a decrease in overall antibiotic use.

Patient/parent and clinician input should be included in the design of stewardship programs. Ultimately, the success of interventions depends on clinicians changing their prescribing behavior and patients/parents adhering to treatment plans. If an intervention or implementation strategy is not acceptable to these stakeholders, it is unlikely to be effective or sustainable. Even if clinicians do not prescribe an antibiotic, patients/parents may simply seek care from another health care provider to get the desired medication. There is value in focusing on low-hanging fruit, where we already have consensus on acceptability by patients/parents and clinicians. Short antibiotic durations are one such example.

Strengths of this evaluation include the simplicity of the interventions, ability to assess effectiveness in different specialties, and ability to assess effectiveness and sustainability over a 2-year period. There were also several important limitations. Because the EHR-only intervention was implemented in family medicine clinics and the bundled intervention was implemented in pediatric clinics, we cannot be certain the differences in prescribing between groups was from the interventions rather than from clinic-level factors. We would not expect this difference to affect the sustainability findings, however. Arguably, there may be greater value in including education for family medicine clinicians, compared with pediatric clinicians, because they are challenged with navigating recommendations from numerous professional societies. We relied on EHR data and therefore did not assess accuracy of AOM diagnosis or appropriateness of antibiotic prescriptions or duration. Thus, some patients may have been misclassified as having AOM and it may have been appropriate for some patients to receive an antibiotic duration longer than 5 days. Finally, because this was a single-center evaluation, the results may not be generalizable to other settings. However, the population served by DH is similar to other urban federally qualified healthcare centers.¹⁸

In conclusion, both an EHR-only and bundled intervention were highly effectively in increasing prescribing of guideline-concordant antibiotic durations for AOM and were sustainable 18 months after discontinuation of clinician education and feedback with no additional resource use. Although the bundled intervention had greater effectiveness than the EHR-only intervention, both interventions significantly improved prescribing in a way that was acceptable to parents and clinicians. Scalability of interventions will be vital to improving overall care for children with AOM.

Glossary

AOM

Acute otitis media

DH

Denver Health and Hospital Authority

EHR

Electronic health record