Failure of a Best Practice Alert to Reduce Antibiotic Prescribing Rates for Acute Sinusitis Across an Integrated Health System in the Midwest

Abstract

BACKGROUND:

Antimicrobial resistance is a growing concern, and in recent years, there has been increased interest in ambulatory antimicrobial stewardship. Acute rhinosinusitis (ARS) is one of the most common outpatient diagnoses that results in an antibiotic prescription.

OBJECTIVE:

To determine if a best practice alert (BPA) will affect the percentage of oral antibiotic prescriptions for adults with ARS.

METHODS:

A prospective, pre/post study was initiated to evaluate the percentage of oral antibiotic prescriptions for ARS in 117 primary care clinics in the Midwest. Included in the study results were 16,570 adults who had an office visit for ARS: 8,106 patients from December 1, 2015, to February 28, 2016, were in the pre-intervention group without an active BPA, and 8,464 patients from December 1, 2016, to February 28, 2017, were in the post-intervention group when the BPA was active. The primary outcome was the number of oral antibiotic prescriptions for ARS compared with the number of office visits for ARS in the pre- and postintervention groups.

RESULTS:

The percentage of oral antibiotics prescribed for the pre- and postintervention groups were 94.8% and 94.3%, respectively (P = 0.152). The BPA displayed for 7,780 visits, prompting discontinuation of an antibiotic for 10 (0.1%) visits in the postintervention group.

CONCLUSIONS:

This study suggests that, although an electronic alert may be attractive to facilitate antimicrobial stewardship, it may be ineffective. These results warrant alternative measures to facilitate ambulatory antimicrobial stewardship.

What is already known about this subject

• The majority of antibiotic expenditures are associated with the ambulatory setting, and a modifiable risk for reducing antibiotic resistance is reducing inappropriate antibiotic prescriptions.

• Acute rhinosinusitis (ARS) is a common ambulatory diagnosis that often results in an antibiotic prescription, and treatment guidelines recommend to withhold antibiotics in mild-to-moderate ARS and those with a symptom duration of less than 10 days.

• Best practice alerts (BPAs) are automated alerts within the electronic medical record that help facilitate widespread communication of information to primary care providers and may be used as a tool for an antimicrobial stewardship strategy.

What this study adds

• This study explored the feasibility of using a BPA as an antimicrobial stewardship tool to reduce inappropriate antibiotic prescribing in ARS.

• Results suggest that this BPA was ineffective in reducing inappropriate antimicrobial prescribing in ARS.

• The BPA prompted the removal of an antibiotic prescription during a patient encounter; however, this action was largely dismissed by providers, and prescribing habits of providers did not change throughout the intervention period.

Acute rhinosinusitis (ARS) is a common respiratory illness with over 30 million diagnoses in the United States each year, of which 80% of cases receive antibiotic prescriptions.^1,2 Direct costs associated with the management of sinusitis in the United States surpasses $11 billion annually.³ Although ARS is the fifth most common diagnosis resulting in an antibiotic prescription, clinical guidelines recommend antibiotic therapy only if signs and symptoms are persistent for 10 or more days; are severe for at least 3-4 days (e.g., fever ≥ 39° C and purulent nasal discharge or facial pain); or worsen after initial improvement (double sickening).^3-5

An important modifiable risk factor for antibiotic resistance is reducing the number of inappropriate antibiotic prescriptions.⁶ Over 60% of antibiotic expenditures are associated with the outpatient setting, where overprescribing of antibiotics has been suggested.^7,8 Available data indicate that over 30% of outpatient antibiotic prescriptions in the United States may be inappropriate, and in ARS, it may be closer to 50%.⁸ Given that antibiotic use is a driving factor for antibiotic resistance,⁶ antimicrobial stewardship efforts should target common diagnoses, such as ARS, that often result in antibiotic use.

Reducing inappropriate antibiotic prescriptions for sinusitis represents an opportunity for quality improvement in the era of evidence-based practice. Aligning with recent sinusitis guidelines, a recommendation for reducing low-value care by the American Academy of Family Physicians’ Choosing Wisely is to avoid routine prescribing of antibiotics for acute mild-to-moderate sinusitis.^4,9 The use of best practice alerts (BPAs) is becoming a common strategy to provide real-time clinical decision support and steer clinicians towards providing evidence-based care.¹⁰

Our institution implemented an electronic intervention in an effort to reduce the use of antibiotics for ARS. The objective of this study was to measure the effect of a BPA on the percentage of antibiotic prescribing for ARS.

Methods

This study included office visits for ARS at 117 acute care (defined as urgent care and walk-in), family medicine, and internal medicine clinics that represented 615 providers within an integrated health system in the Midwest (Iowa, Minnesota, North Dakota, and South Dakota). We conducted a quasi-experimental pre/post study that evaluated the proportion of ARS clinic visits resulting in an oral antibiotic prescription.

Patient office visits from December 1, 2015, to February 28, 2016, and December 1, 2016, to February 28, 2017, were included if the patient was aged ≥ 18 years and received a diagnosis of ARS using International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes J01.00, J01.01, J01.10, J01.11, J01.20, J01.21, J01.30, J01.31, J01.40, J01.41, J01.80, J01.81, J01.90, and J01.91. Only the index office visit for each patient was included during the study time frame, and repeat visits were excluded.

Diagnoses used for patient exclusion were adapted from a previous study by Meeker et al. (2016), which evaluated antibiotic prescribing for acute respiratory tract infections.¹¹ Visits were excluded if patients had medical comorbidities, which included any of the following:

1. A concomitant pulmonary illness or diagnosis requiring an antibiotic (ICD-10-CM codes A04.x, A06.2, A09, A25.1, A31.x, A40.9, A41.xx, A46, A53.0, A54.xx, A56.19, A59.xx, A64, A69.20, A74.xx, A78, B58.2, B95.xx, B96.xx, G03.9, G04. xx, G06.0, G61.0, H00.039, H05.039, H61.93, H65.xx, H66. xxx, H73.xxx, H83.09, H92.10, I33.0, I80.9, J03.90, J17, J37.x, J41.x, J42, J43.9, J44.x, J45.xxx, J47.x, J60, J61, J62.x, J63.x, J66.x, J67.x, J68.x, J70.9, J82, J84.xxx, J95.851, J98. xx, J99, K04.x, K05.xx, K11.20, K12.x, K35.xx, K50.90, K57.32, K61.x, K65.x, K68.xx, K75.x, K80.xx, K81.0, K83.0, K91.850, L01.xx, L02.xxx, L03.xxx, L05.xx, L08.xx, M00. xxx, M01.Xxx, M05.10, M34.81, M46.30, M60.009, M86. xxx, M88.9, M89.xxx, M90.xxx, N10, N12, N16, N28.xx, N30.xx, N34.x, N35.111, N37, N39.0, N41.0, N41.x, N43.1, N45.x, N51, N72, N73.x, N75.0, N76.x, N77.1, O23.xxx, O41.xxxx, O86.xx, O91.xxx, O98.919, P25.x, P36.9, P39.x, Q31.x, Q32.x, Q33.x, Q34.x, R09.2, R78.81, S20.95XA, S60. xxxA, T07, T80.219A, T81.4XXA, and Z22.xxx).

2. A diagnosis associated with immunocompromise (ICD-10-CM codes B20, B97.35, C33, C34.xx, C80.x, D49.xx, D70.x, D71, D72.xxx, D76.x, D86.9, I85.xx, K72.xx, K76.x, R75, T86.xxx, Y83.0, Z21, Z85.xxx, Z94.xx, and Z95.3).

3. A diagnosis within the previous 30 days for ARS or other acute respiratory tract infection (ICD-10-CM codes A37.xx, A48.1, B95.3, B96.x, H65.xx, H66.xxx, H73.xxx, H83.09, H92.10, J00, J01.xx, J02.x, J03.xx, J04.xx, J05.xx, J06.x, J11.xx, J12.9, J13, J14, J15.xxx, J18.x J20.9, J21.x, and J36, J40).

A real-time BPA was implemented in the electronic medical record to notify the provider of the Choosing Wisely recommendation when prescribing an antibiotic for a patient diagnosed with ARS during that visit (Figure 1). A diagnosis of sinusitis triggered a pop-up screen with a link to the Choosing Wisely recommendation and a prompt to remove the antibiotic prescription.⁹ Optional justification for continuing the antibiotic order, designated as “acknowledgement reasons,” could be selected in the BPA response field. One of six discrete acknowledgment reasons, as developed by a physician-led clinical decision support committee, could be selected or entered as free text. The alert could be dismissed without requiring the clinician to provide justification for the antibiotic prescription.

FIGURE 1.

Sinusitis Best Practice Alert

The BPA began displaying to providers 6 weeks before the 3-month intervention period during peak winter months (December 2016-February 2017). Data from this period were compared with the same months the previous year (December 2015-February 2016) to minimize any confounding effect of seasonal variation. Data collected included baseline characteristics, oral antibiotics prescribed during office visits, antibiotic recommendation category for ARS (first line, second line, other), and provider responses to the BPA (Table 1). Department types in the study included acute care, family medicine, and internal medicine departments.

TABLE 1.

Patient Characteristics and Antibiotic Prescriptions According to Subgroup

	Pre-intervention Group	Postintervention Group	P Value
Initial visits for ARS during study period	14,566	14,092
Visits excluded due to medical comorbidities	6,460	5,628
Patients eligible for study inclusion	n = 8,106	n = 8,464
Characteristics
Age, years, mean (SD)	44.8 (15.6)	45.6 (15.6)	< 0.001
Male, n (%)	2,955 (36.5)	3,087 (36.5)	0.981
Insurance type, n (%)
Medicare	957 (11.7)	1,063 (12.6)	0.139
State or county subsidized	1,280 (15.8)	1,232 (14.6)	0.027
Private	5,492 (67.8)	5,807 (68.6)	0.237
Self-pay or other	71 (0.9)	48 (0.6)	0.019
Unavailable	306 (3.8)	314 (3.7)	0.825
State, n (%)
Iowa	211 (2.6)	238 (2.8)	0.408
Minnesota	1,657 (20.4)	1,736 (20.5)	0.913
North Dakota	3,714 (45.8)	3,719 (43.9)	0.015
South Dakota	2,524 (31.1)	2,771 (32.7)	0.027
Visit provider type, n (%)
Physician	4,780 (58.9)	4,614 (54.5)	< 0.001
Nurse practitioner	1,744 (21.5)	2,228 (26.3)	< 0.001
Physician assistant	1,552 (19.1)	1,604 (18.9)	0.749
Other	30 (0.4)	18 (0.2)	0.059
Visit department, n (%)
Family medicine	4,556 (56.2)	4,722 (55.8)	0.59
Acute care	3,250 (40.1)	3,449 (40.7)	0.39
Internal medicine	300 (3.7)	293 (3.5)	0.41
Comorbidities, n (%)
Heart failure	29 (0.4)	43 (0.5)	0.142
Ischemic heart disease	178 (2.2)	234 (2.8)	0.019
Hypertension	1,291 (15.9)	1,417 (16.7)	0.156
Diabetes mellitus	447 (5.9)	577 (6.8)	0.014
Depressive disorder	1,000 (12.3)	1,090 (12.9)	0.294
Chronic kidney disease	70 (0.9)	86 (1.0)	0.310
Antibiotic prescriptions by subgroup
All visits, n/total n (%)	7,686/8,106 (94.8)	7,984/8,464 (94.3)	0.152
Age, years, n/total n (%)
18-44	3,939/4,147 (95.0)	3,960/4,194 (94.4)	0.251
45-64	2,900/3,058 (94.8)	3,086/3,247 (95.0)	0.706
65-74	600/640 (93.8)	672/731 (91.9)	0.194
75 or older	247/261 (94.6)	266/292 (91.1)	0.109
Insurance type, n/total n (%)
Medicare	889/957 (92.9)	980/1,063 (92.2)	0.549
State or county subsidized	1,210/1,280 (94.5)	1,159/1,232 (94.1)	0.622
Private	5,224/5,492 (95.1)	5,498/5,807 (94.7)	0.287
Self-pay or other	68/71 (95.8)	47/48 (97.9)	0.525
Unavailable	295/306 (96.4)	300/314 (95.5)	0.585
Visit provider type, n/total n (%)
Physician	4,530/4,780 (94.8)	4,349/4,614 (94.3)	0.275
Nurse practitioner	1,643/1,744 (94.2)	2,080/2,228 (93.4)	0.272
Physician assistant	1,485/1,552 (95.7)	1,538/1,604 (95.9)	0.777
Other	28/30 (93.3)	17/18 (94.4)	0.878
Visit department, n/total n (%)
Acute care	3,183/3,250 (97.9)	3,329/3,449 (96.5)	< 0.001
Family medicine	4,245/4,556 (93.2)	4,403/4,722 (93.2)	0.893
Internal medicine	258/300 (86.0)	252/293 (86.0)	0.998
Comorbidities, n/total n (%)
Heart failure	29/29 (100.0)	39/43 (90.7)	0.091
Ischemic heart disease	168/178 (94.4)	212/234 (90.6)	0.155
Hypertension	1,224/1,291 (94.8)	1,320/1,417 (93.2)	0.071
Diabetes mellitus	445/447 (93.3)	542/577 (93.9)	0.670
Depressive disorder	931/1,000 (93.1)	1,036/1,090 (95.1)	0.059
Chronic kidney disease	66/70 (94.3)	77/86 (89.5)	0.286

ARS = acute rhinosinusitis; SD = standard deviation.

In this study, we categorized first-line antibiotics as amoxicillin or amoxicillin-clavulanate; second-line antibiotics as second- or third-generation cephalosporins, clindamycin, doxycycline, levofloxacin, moxifloxacin, or minocycline; and other antibiotics were those not recommended because of high rates of resistance to S. pneumoniae or lack of appropriate bacterial coverage.^3,4

The primary outcome was the number of oral antibiotic prescriptions for ARS compared with the number of office visits for ARS in the pre- and postintervention groups. The difference in the percentage of antibiotics prescribed between the pre- and postintervention groups was tested using Pearson’s chi-square test. Secondary outcomes included the same evaluation across subgroups and postintervention months (Table 1). The data were analyzed with JMP software, version 12.0.0 (SAS Institute, Cary, NC). A P value of < 0.05 was considered statistically significant.

This work was reviewed by the Sanford Health Institutional Review Board and was determined to be quality improvement rather than human subjects research.

Results

In the pre-intervention group, there were 14,566 unique patient visits, with 8,106 eligible for inclusion. In the postintervention group, there were 14,092 unique patient visits, with 8,464 eligible for inclusion. Results based on baseline characteristics are summarized in Table 1. Antibiotics were prescribed in 94.8% of sinusitis visits in the pre-intervention group, and 94.3% of sinusitis visits in the postintervention group (P = 0.152). The overall percentage of oral antibiotic prescriptions based on department type for acute care, family medicine, and internal medicine were 97.2%, 93.2%, and 86.0%, respectively (P < 0.001).

The BPA displayed for 7,780 visits, prompting discontinuation of an antibiotic in 10 (0.1%) visits in the postintervention group. During the postintervention period, the percentage of oral antibiotics prescribed for the months of December 2016, January 2017, and February 2017 were 94.5%, 94.5%, and 94.1%, respectively (P = 0.749).

First-line antibiotic prescriptions increased from 3,842 (49.9%) in the pre-intervention group to 4,551 (57.0%) in the postintervention group (P < 0.001). The percentage of second-line antibiotic prescriptions was not significantly different, with 1,797 (23.4%) prescriptions in the pre-intervention group compared with 1,834 (22.9%) in the postintervention group (P = 0.436). Other antibiotic prescriptions declined from the pre-intervention group (2,047, 26.6%) compared with the post-intervention group (1,599, 20.0%, P < 0.001).

A multivariable logistic regression analysis was performed to adjust for potential confounding variables related to whether an antibiotic was prescribed during the entire study time frame. The model included the presence of the BPA, age, insurance type, provider type, clinic type, and the following comorbidities: hypertension, heart failure, ischemic heart disease, diabetes mellitus, chronic kidney disease, and depressive disorder. The only independent predictors of an antimicrobial prescription were a visit in the acute care department (P < 0.001) and increasing age as a continuous variable (P < 0.001). Being in the BPA group was not associated with decreased odds of an antimicrobial prescription in the adjusted model (P = 0.125).

Among the subgroups, only the acute care department demonstrated a statistically significant difference in the rate of antimicrobial prescribing (97.9% pre-intervention vs. 96.5% postintervention, P < 0.001). In the age subgroup analyses, patients were divided into 4 age cohorts (18-44 years, 45-64 years, 65-74 years, and 75 years or older), none of which demonstrated statistically significant changes in antimicrobial prescriptions after BPA implementation.

Discussion

Ninety-four percent of patients received an antibiotic if they were seen within our health system for ARS, while national prescribing rates are closer to 80%. Given that an estimated 2%-10% of ARS cases are caused by bacteria,¹² the large percentage of antibiotic prescriptions for ARS suggests probable overprescribing. This BPA was implemented to decrease inappropriate antibiotic prescribing but was largely dismissed by providers, and the percentage of oral antibiotics prescribed for ARS was unaffected.

The majority of office visits for ARS were seen by acute care or family medicine providers, encompassing 96.3% of overall visits in this study, with 3.7% of office visits occurring at internal medicine clinics. The baseline characteristic analyses demonstrated some statistical differences in age, provider type, insurance, and comorbidities (Table 1). However, no differences in antibiotic prescribing were demonstrated in any of these subgroups. The only subgroup found to have a statistical difference in antibiotic prescribing was acute care. Whether the BPA was actually more effective in changing prescribing behavior in the acute care setting is uncertain because of the low absolute decrease in antimicrobial prescriptions (1.4%).

After adjusting for baseline characteristic differences in a multivariable model, the presence of the BPA was not associated with decreased odds of receiving an antimicrobial prescription (Table 1). Our finding related to acute care visits being statistically associated with increased antimicrobial prescriptions over the entire study time frame is pragmatic and likely reflects severity of illness. Primary care specialties have been associated with overall higher rates of antibiotic prescribing, particularly in family medicine compared with internal medicine practices.¹³ Higher prescribing rates within acute care and family medicine departments may be related to pressure from patients, the need to achieve quicker outcomes, or lack of follow-up to ensure symptom resolution, since this visit may not have occurred with the patient’s primary provider.

The percentage of ARS visits that resulted in antibiotic prescriptions by physicians, nurse practitioners, and physician assistants were similar. Nurse practitioners and physician assistants have been associated with higher antibiotic prescribing rates in respiratory tract infections compared with physician-only visits, with minimal difference in the prescribing practices between nurse practitioners and physician assistants.¹⁴ Our results show that ambulatory antimicrobial stewardship efforts should target all providers rather than focusing only on advanced practice providers.

First-line antibiotic prescribing increased 7.1% from the pre-intervention group to the postintervention group, reaching 57%. Second-line antibiotic use did not change between the 2 periods, but use of other antibiotics decreased during the intervention period. In the United States, the prescribing rate of first-line antibiotics for ARS has been reported near 37%, with the majority of patients receiving non–first-line antibiotics.¹⁵ The higher percentage of first-line antibiotic prescribing in our study could be related to its exclusion criteria, which omitted patients with comorbid conditions that may have otherwise elicited more broad-spectrum antibiotic use. The reasons that higher first-line antibiotic use was found in the postintervention group are unclear, since antibiotic selection was not an aim of the intervention. It is possible the BPA coincidentally increased provider vigilance in antibiotic selection, prompting use of narrower-spectrum or guideline-concordant agents.

Although an electronic alert is a common intervention strategy to guide a provider’s practice, it may be judicious to trial and validate its effect before deployment across a health system. When incorporating electronic alerts in the workflow of front-line providers, challenges remain in providing the proper balance of actionable information and dismissal mechanisms.¹⁶ If a BPA is used as an antimicrobial stewardship strategy, consideration should be given to its use as part of a multifaceted intervention rather than a stand-alone tool.

Limitations

This study has limitations that need to be considered. First, the control group was a historical cohort, so the observed intervention effect may be biased by other factors that affected the percentage of antibiotics prescribed. Similar to other investigations of this nature, we were unable to capture how many patients had a true indication for an antibiotic, since the BPA simply served to remind the clinician of best practices. Our proportion of prescribing, however, suggested a substantial degree of inappropriate prescribing in the pre- and postintervention groups.

The study time frame included only a portion of the influenza season. Since data were not collected throughout its continuum (usually regarded as October through March¹⁷), the results of this study cannot be extrapolated to the entire influenza season. The effect of the BPA was evaluated over a short time frame (3 months), and changes in prescribing behavior may have taken longer than the study period was able to detect. However, we evaluated 3 sequential months with the active BPA and did not see a statistically significant change in the proportion of prescribing across those months.

An additional limitation was the geographical area in which the study was conducted. The Midwest has a higher antibiotic prescribing rate than other regions of the United States,⁸ which may have contributed to our study’s high proportion of antibiotic prescribing. However, in regions with lower prescribing rates, there remains an opportunity to reduce inappropriate antibiotic prescribing for ARS.⁸

The ineffectiveness of our BPA could have been due to its workflow disruption, absence of functionality requiring antibiotic justification, and/or lack of provider training regarding the tool. A previous study implemented an electronic alert that required providers to justify antibiotic use in the patient’s medical record, which resulted in a reduced antibiotic prescribing rate in acute viral respiratory tract infections.¹¹ Another recent study combined clinician education with prescribing audit and feedback, which improved guideline-concordant prescribing for common bacterial respiratory tract infections.¹⁸

Multiple factors may have led to the broad dismissal of the BPA by providers. No documentation was required to dismiss the alert nor was the alert preceded by provider education regarding ARS or antimicrobial stewardship. The BPA in this study may have been more effective if the strategy had also incorporated provider feedback, peer comparison, or provider incentives.

Conclusions

Among primary care practices within an integrated health system in the Midwest, the use of a BPA failed to reduce the number of antibiotic prescriptions for ARS. These results support the need for alternative measures to facilitate ambulatory antimicrobial stewardship.