Development of an Electronic Algorithm to Target Outpatient Antimicrobial Stewardship Efforts for Acute Bronchitis and Pharyngitis

Abstract

Background

A major challenge for antibiotic stewardship programs is the lack of accurate and accessible electronic data to target interventions. We developed and validated separate electronic algorithms to identify inappropriate antibiotic use for adult outpatients with bronchitis and pharyngitis.

Methods

We used International Classification of Diseases, 10th Revision, diagnostic codes to identify patient encounters for acute bronchitis and pharyngitis at outpatient practices between 3/15/17 and 3/14/18. Exclusion criteria included immunocompromising conditions, complex chronic conditions, and concurrent infections. We randomly selected 300 eligible subjects each with bronchitis and pharyngitis. Inappropriate antibiotic use based on chart review served as the gold standard for assessment of the electronic algorithm, which was constructed using only data in the electronic data warehouse. Criteria for appropriate prescribing, choice of antibiotic, and duration were based on established guidelines.

Results

Of 300 subjects with bronchitis, 167 (55.7%) received an antibiotic inappropriately based on chart review. The electronic algorithm demonstrated 100% sensitivity and 95.3% specificity for detection of inappropriate prescribing. Of 300 subjects with pharyngitis, 94 (31.3%) had an incorrect prescribing decision. Among 29 subjects with a positive rapid streptococcal antigen test, 27 (93.1%) received an appropriate antibiotic and 29 (100%) received the correct duration. The electronic algorithm demonstrated very high sensitivity and specificity for all outcomes.

Conclusions

Inappropriate antibiotic prescribing for bronchitis and pharyngitis is common. Electronic algorithms for identifying inappropriate prescribing, antibiotic choice, and duration showed excellent test characteristics. These algorithms could be used to efficiently assess prescribing among practices and individual clinicians. Interventions based on these algorithms should be tested in future work.

Antibiotic use is common in outpatient settings, where the equivalent of 836 courses of antibiotics are prescribed for every 1000 persons per year in the United States [1]. However, 30%–50% of these prescriptions are inappropriate, depending on the clinical setting [2–5]. Despite the importance of outpatient antibiotic use in driving antibiotic resistance, developing efficient approaches to effectively tracking inappropriate prescribing in the outpatient setting has proven challenging [6–8].

Developing ways to track antibiotic use using easily available indicators that better identify inappropriate use has several theoretical benefits: (1) improving antibiotic stewardship program (ASP) efficiency by decreasing the work needed to perform manual chart review, (2) prioritizing targets of ASP interventions, (3) assessing the impact of interventions over time, (4) providing relevant metrics to facilities and prescribers, and (5) identifying metrics that can be used by accreditation or public health agencies to benchmark antibiotic use data or to investigate facilities with high rates of antibiotic use [9, 10].

The increasing availability of a broad range of data elements from electronic health records (EHRs) creates the opportunity to develop more actionable metrics using individual indicators or groups of indicators to better identify areas of inappropriate use. In order to utilize data in this way, performance characteristics of candidate electronic indicators need to be systematically evaluated and then operationalized in ways that allow ASPs to perform analyses on an ongoing basis.

The goal of this study was to identify electronic indicators (ie, data elements extractable from electronic data sources without manual chart review) of antibiotic appropriateness for frequently encountered outpatient conditions and validate these indicators against manual chart review. We focused on 2 adult outpatient conditions (ie, pharyngitis, bronchitis) for which antibiotic prescribing is common and often inappropriate [2, 11–14].

METHODS

Study Sites

Adult subjects were included from the University of Pennsylvania Health System (UPHS), which encompasses a large network of outpatient facilities in a variety of geographic and socioeconomic settings. UPHS has 30 primary care practices with both internal medicine and family medicine providers serving a racially and ethnically diverse patient population located in Southeastern Pennsylvania and New Jersey. Within the UPHS network, there are both academic and nonacademic practices, distinguished by whether they engage in teaching of medical students and/or house staff. The same EHR is used for all aspects of clinical care at UPHS outpatient settings (EpicCare, Epic Systems, Inc, Verona, WI, USA). This study was reviewed and approved by the Institutional Review Board of the University of Pennsylvania.

Study Subjects

Adult Bronchitis

Potentially eligible patients were electronically identified as those with an outpatient encounter occurring between March 15, 2017, and March 14, 2018, at which acute bronchitis was identified using specific International Classification of Diseases, 10th Revision (ICD-10), codes (Supplementary Data 1a) [15]. Only encounters designated as an “office visit” were considered eligible. Encounters coded as “telephone visit” or “procedure visit” were ineligible. After identifying all eligible encounters, we excluded patients who had received an ICD-10 code for a “complex chronic condition” at the time of the encounter or within the past year (Supplementary Data 2). These patients were excluded given the difficulty of assessing the appropriateness of antibiotics in patients with these underlying conditions. For the same reason, patients who had an active prescription for an immunocompromising medication at the time of the encounter or within the past year were also excluded (Supplementary Data 3). In addition, subjects were excluded if they carried an ICD-10 diagnosis at the time of the encounter for other conditions that might impact the decision to prescribe antibiotics (eg, chronic obstructive pulmonary disease [COPD], bronchiectasis, chronic bronchitis) (Supplementary Data 4). Subjects were also excluded if they received an ICD-10 code at the encounter that indicated a concurrent common outpatient infection for which antibiotics might be prescribed (Supplementary Data 5). Finally, subjects were excluded if they were assigned an ICD-10 code at the time of the encounter that indicated a suspicion or diagnosis of Bordetella spp. infection (Supplementary Data 6). From all eligible subjects, we randomly selected 150 each from academic practices and nonacademic practices. This number of subjects was chosen to ensure successful completion of manual review of medical records.

Adult Pharyngitis

Potentially eligible subjects were electronically identified as those with an outpatient encounter occurring between March 15, 2017, and March 14, 2018, at which acute pharyngitis was identified using specific ICD-10 codes (Supplementary Data 1b). Consistent with the approach for bronchitis, only those encounters designated as an “office visit” were considered eligible, and subjects with a “complex chronic condition” (Supplementary Data 2), immunocompromising medication use (Supplementary Data 3), and/or a concurrent infection (Supplementary Data 5) were excluded. From all eligible subjects, we randomly selected 150 each from academic practices and nonacademic practices.

Data Collection

Data were collected using 2 separate approaches: (1) manual chart review of the EHR and (2) review of an EHR-based data warehouse (EDW). Manual EHR review was the gold standard for the purposes of analyses. In both approaches, we independently collected all available data on general demographics, type of practice, and type of prescriber. Data were ascertained on antibiotic use in the 30 days before the encounter, diagnosis of bronchitis or pharyngitis (as applicable) in the preceding 30 days, and allergy history. Finally, we ascertained whether an antibiotic was prescribed at the visit, which antibiotics were prescribed, the duration of the prescription, laboratory and radiographic data, and whether justification was documented for the prescribing.

First, data were ascertained via a manual review of the EHR by a highly experienced research coordinator. Data were reviewed for randomly selected subjects by another investigator (E.L.) to confirm accuracy. For bronchitis, we assessed only the decision to prescribe. We did not assess appropriateness of antibiotic choice or duration, as any antibiotic use in our study population was deemed inappropriate (given our exclusion criteria). We determined whether the decision to prescribe was appropriate based on established UPHS antibiotic use guidelines, which state that prescribing for bronchitis is appropriate only if the patient has an established diagnosis of COPD or bronchiectasis that fulfills clinical criteria for antibiotic treatment based on Global Initiative for Obstructive Lung Disease criteria [16]. We did not review patients with bronchitis to whom no antibiotic was prescribed. For pharyngitis, we assessed 3 elements of antibiotic use: (1) the decision to prescribe, (2) the choice of antibiotic, and (3) the duration of the prescription, all based on established institutional guidelines. Specifically, the decision to prescribe an antibiotic was considered appropriate only if a rapid streptococcal antigen test (RSAT) was documented as positive in the medical record. Among subjects who were appropriately prescribed an antibiotic, penicillin and amoxicillin were considered appropriate choices. If the subject had a history of beta-lactam allergy, any antibiotic was considered appropriate. Finally, among subjects for whom the decision to prescribe was appropriate (regardless of the appropriateness of the antibiotic selected), a duration of use of ≤10 days was considered appropriate.

The second approach to data ascertainment was conducted through the EDW using all available data elements including demographics, allergy data, diagnostic codes, prescribing data, and laboratory data. Information in encounter notes was available only via manual EHR review.

Analysis

Bronchitis

For the 300 subjects with bronchitis, we first used the data ascertained from the manual EHR review (gold standard) to describe the characteristics of the selected subjects, the providers, and the clinical settings. We then identified the proportion of subjects for whom an antibiotic was prescribed. Among these subjects, we assessed whether there were any factors (eg, underlying condition, concurrent infection, suspicion of pertussis) that influenced the decision to prescribe an antibiotic but had not been identified in the electronic selection protocol. After accounting for these factors, we determined the final proportion of subjects for whom an antibiotic was prescribed. Given that existing guidelines consider any such antibiotic use inappropriate, this proportion represented the percentage of subjects for whom antibiotic use was inappropriate.

We then used the calculations derived from the manual EHR review to assess test characteristics of an electronic algorithm for identifying inappropriate antibiotic use. This algorithm included both the identification of appropriate subjects using the ICD-10 inclusion and exclusion criteria described previously and identification and evaluation of the antibiotic use itself. The electronic algorithm used only data available in the EDW, recognizing that some nuances of the data (eg, concern regarding pertussis documented only in a progress note) would be available only on manual chart review. We then calculated the sensitivity, specificity, positive predictive value, and negative predictive value of the electronic algorithm in determining inappropriate antibiotic use when compared against the gold standard manual EHR review.

Pharyngitis

For the 300 subjects with pharyngitis, the approach to analysis was similar. We first used the data ascertained from the manual EHR review (gold standard) to describe the characteristics of the selected subjects, the providers, and the clinical settings. Among subjects who received an antibiotic, we assessed whether there were any factors (eg, underlying condition, concurrent infection) that influenced the decision to prescribe an antibiotic but had not been identified in the electronic selection protocol. After accounting for these factors, we determined the final proportion of subjects for whom an antibiotic was prescribed. Antibiotic use was considered appropriate only if an RSAT was positive. In those subjects for whom an antibiotic was appropriately prescribed, we assessed the appropriateness of the antibiotic choice and the duration of therapy as described above.

We then used the calculations of appropriateness for these 3 measures derived from the manual EHR review to assess the test characteristics of an electronic algorithm for identifying inappropriate prescribing, antibiotic choice, and duration in pharyngitis. This algorithm included both the identification of appropriate subjects (eg, those without conditions that would mitigate antibiotic use in pharyngitis) and identification and evaluation of the antibiotic use itself. Following completion of the calculations based on the gold standard manual EHR review above, we repeated these steps using data available in the EDW, recognizing that some nuances of the data (eg, RSAT results documented only in a progress note) would only be available only on manual chart review. These assessments were used to determine the sensitivity, specificity, positive predictive value, and negative predictive value of the electronic algorithm in determining the appropriateness of (1) the decision to prescribe, (2) the antibiotic choice, and (3) the duration of antibiotic use when compared against the gold standard manual EHR review.

RESULTS

Adult Bronchitis

There were 4708 unique encounters with an ICD-10 code for bronchitis. Exclusions included complex chronic conditions (70; 1.5%), immunocompromising drugs (23; 0.5%), complicating conditions (eg, COPD; 116; 2.5%), and concurrent infections (430; 9.1%). Subjects could be excluded for >1 criterion. The most common concurrent infections were sinusitis (n = 348), otitis (n = 63), pharyngitis/tonsillitis (n = 48), and pneumonia (n = 10). Overall, 612 (13%) subjects were excluded.

The baseline characteristics of subjects are noted in Table 1. By design, there were 150 encounters each in academic and nonacademic practices. Overall, 167 (55.7%) subjects were prescribed an antibiotic, all of whom were considered to have received unnecessary antibiotics based on manual chart review. However, in 1 case, there was a concern for pertussis documented in the EHR but not assigned an ICD-10 code and thus not excluded during the electronic selection process. Further, there were 4 subjects in whom there was documentation of an excluding condition including chronic bronchitis (n = 1), emphysema (n = 1), COPD (n = 1), and bronchiectasis (n = 1) without an ICD-10 assigned (and thus not excluded during the electronic selection process). Finally, there was 1 patient for whom an antibiotic was prescribed for an upcoming dental procedure. Taking these factors into consideration (ie, considering antibiotic use appropriate in these cases), there remained 161 (53.7%) subjects who received inappropriate antibiotics.

Table 1.

Characteristics of Adult Study Subjects With Bronchitis ^a

Variable	No. (%)/Median (IQR) (n = 300)
Age, y	60.8 (50.0–68.4)
Female sex	184 (61.3)
Type of practice
Internal medicine	222 (74.0)
Family medicine	78 (26.0)
Type of prescriber
Physician	226 (75.3)
Advanced practice provider	74 (24.7)
Antibiotic allergies
1 antibiotic allergy	61 (20.3)
2 antibiotic allergies	19 (6.3)
3 antibiotic allergies	7 (2.3)
>3 antibiotic allergies	6 (2.0)
Antibiotic allergies (by class—not mutually exclusive)
Penicillins	60 (20.0)
Sulfa drugs	30 (10.0)
Macrolides	12 (4.0)
Tetracyclines	10 (3.3)
Fluoroquinolones	10 (3.3)
Cephalosporins	8 (2.7)
Exposure to antibiotics in past 30 d
1 antibiotic	36 (12.0)
2 antibiotics	3 (1.0)
>2 antibiotics	0 (0)
Antibiotic exposures in past 30 d (by agent)
Azithromycin	20 (6.7)
Amoxicillin	7 (2.3)
Doxycycline	5 (1.7)
Levofloxacin	3 (1.0)
Cefuroxime	3 (1.0)
Others	4 (1.3)
Diagnosis of bronchitis in past 30 d
1 episode	26 (8.7)
2 episodes	4 (1.3)
>2 episodes	0 (0)

Study period: March 15, 2017, to March 14, 2018.

Abbreviations: EHR, electronic health record; IQR, interquartile range.

^aBased on manual EHR review.

When using an EDW-based approach, the algorithm identified 170 subjects in whom antibiotics were inappropriate. Specifically, there were 3 cases in which the EDW approach identified a prescribed antibiotic that was not identified on manual chart review. In addition, the electronic algorithm did not identify the subject with concern for pertussis, nor did it identify 5 subjects with excluding conditions found on manual chart review. The electronic algorithm demonstrated high sensitivity and specificity for detection of inappropriate antibiotic prescribing (Table 2).

Table 2.

Test Characteristics of EHR Electronic Algorithm for Detecting of Inappropriate Prescribing in Bronchitis

		EDW Approach
		Inappropriate (Abx)	Appropriate (No Abx)
EHR manual review approach (gold standard)	Inappropriate	161	0
	Appropriate	9	130
Test characteristics
Sensitivity: 100% (161/161)
Specificity: 93.53% (130/139)
Positive predictive value: 94.71% (161/170)
Negative predictive value: 100% (130/130)

Abbreviations: EDW, EHR-based data warehouse; EHR, electronic health record.

Adult Pharyngitis

There were 7412 unique encounters with an ICD-10 code for pharyngitis. Exclusions included complex chronic condition (70; 0.9%), immunocompromising drug (36; 0.5%), and concurrent infection (601; 8.1%). The most common concurrent infections were sinusitis (n = 423), otitis (n = 158), skin infection (n = 26), and pneumonia (n = 16). Overall, 691 (9.3%) subjects were excluded.

The baseline characteristics of subjects are noted in Table 3. By design, there were 150 encounters each in academic and nonacademic practices. Of 112 (37.3%) subjects who had an RSAT obtained, 29 (25.9%) were positive. Overall, 127 (42.3%) subjects were prescribed an antibiotic on manual EHR review, with 206 (68.7%) determined to have appropriate prescribing. These patients included both those for whom no antibiotic was prescribed (n = 173) and those with an antibiotic prescribed in the setting of a positive RSAT (n = 29). In addition, there were 4 subjects who had other conditions warranting antibiotic use (ie, dental abscess, vaginosis, penile discharge, and rectal abscess). Overall, 94 (31.3%) subjects received antibiotics inappropriately. Among the 29 subjects for whom an antibiotic was correctly prescribed, 27 (93.1%) received an appropriate antibiotic. Of note, there were 3 patients with beta-lactam allergy noted on manual EHR review, but not documented in the EDW. Of the 29 subjects who received antibiotics appropriately, all 29 (100%) received the correct duration.

Table 3.

Characteristics of Adult Study Subjects With Pharyngitis ^a

Variable	No. (%)/Median (IQR) (n = 300)
Age, y	42.0 (30.7–53.6)
Female sex	225 (75.0)
Type of practice
Internal medicine	193 (64.3)
Family medicine	107 (35.7)
Type of prescriber
Physician	207 (69.0)
Advanced practice provider	93 (31.0)
Antibiotic allergies
1 antibiotic allergy	54 (18.0)
2 antibiotic allergies	14 (4.7)
3 antibiotic allergies	6 (2.0)
>3 antibiotic allergies	2 (0.7)
Antibiotic allergies (by class—not mutually exclusive)
Penicillins	42 (14.0)
Sulfa drugs	17 (5.7)
Fluoroquinolones	11 (3.7)
Macrolides	10 (3.3)
Tetracycline	6 (2.0)
Cephalosporins	5 (1.7)
Exposure to antibiotics in past 30 d
1 antibiotic	21 (7.0)
2 antibiotics	2 (0.7)
>2 antibiotics	0 (0)
Antibiotic exposures in past 30 d (by agent)
Azithromycin	7 (2.3)
Amoxicillin	7 (2.3)
Penicillin	3 (1.0)
Metronidazole	2 (0.7)
Others (each = 1)	6 (2.0)
Diagnosis of pharyngitis in past 30 d
1 episode	10 (3.3)
2 episodes	2 (0.7)
>2 episodes	0 (0)

Study period: March 15, 2017, to March 14, 2018.

Abbreviations: EHR, electronic health record; IQR, interquartile range.

^aBased on manual EHR review.

Based on the EDW approach, 127 (42.3%) subjects were identified as having been prescribed an antibiotic (consistent with manual EHR review). Of the 300 total subjects, 200 (66.7%) were determined to have a correct prescribing decision, including those patients for whom no antibiotic was prescribed (n = 173) and those for whom an antibiotic was prescribed in the setting of a positive RSAT (n = 27). Of note, there were 2 subjects for whom an RSAT was identified only on manual EHR review and not from the EDW. The 4 subjects noted on manual EHR review to have other conditions warranting antibiotic use were not identified from the EDW. Among the 27 subjects for which an antibiotic was correctly prescribed based on EDW review, 22 (81.5%) received an appropriate antibiotic. There were an additional 3 patients who were noted to be beta-lactam allergic and were thus considered to have received appropriate therapy. Overall, 25 (92.6%) subjects received the correct antibiotic. Finally, of the 27 subjects who received antibiotics appropriately, 27 (100%) received the correct duration. The test characteristics of the EDW-based algorithm demonstrated very high sensitivity and specificity for each of the outcomes (Table 4A–C).

Table 4A.

Test Characteristics of EHR Electronic Algorithm for Detecting of Inappropriate Prescribing in Pharyngitis

		EDW Approach
		Inappropriate	Appropriate
EHR manual review approach (gold standard)	Inappropriate	94	0
	Appropriate	6	200
Test characteristics
Sensitivity: 100% (94/94)
Specificity: 97.09% (200/206)
Positive predictive value: 94.00% (94/100)
Negative predictive value: 100% (200/200)

Using chart-based designation gold standard for correctly detecting inappropriate therapy. Denominator = all subjects.

Abbreviations: EDW, EHR-based data warehouse; EHR, electronic health record.

Table 4B.

Test Characteristics of EHR Electronic Algorithm for Detecting of Inappropriate Antibiotic Choice in Pharyngitis

		EDW Approach
		Inappropriate	Appropriate
EHR manual review approach (gold standard)	Inappropriate	2	0
	Appropriate	0	25
Test characteristics
Sensitivity: 100% (2/2)
Specificity: 100% (25/25)
Positive predictive value: 100% (2/2)
Negative predictive value: 100% (25/25)

Using chart-based designation gold standard for correctly detecting inappropriate choice of agent. Denominator = all subjects identified by the EHR algorithm as having appropriate prescribing.

Abbreviations: EDW, EHR-based data warehouse; EHR, electronic health record.

Table 4C.

Test Characteristics of EHR Electronic Algorithm for Detecting of Inappropriate Antibiotic Duration in Pharyngitis

		EDW Approach
		Inappropriate	Appropriate
EHR manual review approach (gold standard)	Inappropriate	0	0
	Appropriate	0	27
Test characteristics
Sensitivity: NA (0/0)
Specificity: 100% (27/27)
Positive predictive value: NA (0/0)
Negative predictive value: 100% (27/27)

Using chart-based designation gold standard for correctly detecting inappropriate duration of prescription. Denominator = all subjects identified by EHR algorithm as having appropriate prescribing.

Abbreviations: EDW, EHR-based data warehouse; EHR, electronic health record.

DISCUSSION

The electronic algorithm developed for assessing the decision to prescribe an antibiotic for bronchitis was highly sensitive and specific. Similarly, the electronic algorithm for pharyngitis demonstrated excellent test characteristics when assessing the appropriateness of the decision to prescribe, antibiotic choice, and duration of therapy. Our results demonstrate the potential of the EHR for reducing the workload of ASPs. While some electronically based metrics (eg, overall prescribing rates) are used to track aggregate antibiotic use, they are at best surrogate markers of inappropriate antibiotic use [17]. Therefore, ASP initiatives must utilize time-intensive strategies (eg, medication use evaluations) to qualitatively assess appropriateness of antibiotic use [15]. This historical reliance on manual chart review complicates ASP efforts [8].

The increasing availability of EHRs provides a unique opportunity. The availability of these data elements has the potential to create actionable metrics using individual indicators or groups of indicators to better identify inappropriate use [9, 10]. To utilize these data, performance characteristics of electronic indicators must be systematically evaluated and then operationalized in ways that allow ASPs to perform ongoing analyses. Our work provides an important advance in this domain.

Although our electronic algorithms demonstrated excellent test characteristics for identifying inappropriate antibiotic prescribing, antibiotic choice, and duration, there were 2 scenarios in which results of manual EHR review did not align with the electronic indicators. First, there were several patients for whom information was entered into the encounter note (eg, another condition warranting antibiotics, RSAT results) that was not otherwise documented electronically. Second, there were several subjects for whom a new antibiotic prescription was noted in the EDW but not on manual chart review.

The results suggest that our electronic algorithms for these 2 common outpatient clinical conditions can be used to efficiently and accurately identify instances of inappropriate antibiotic prescribing. These algorithms could provide longitudinal data on patterns of inappropriate antibiotic use and be reported at the level of the health system, the practice, and/or the individual provider to better inform and support ASP targets and initiatives. Also, these data could be used to provide feedback to practices and/or individual providers to allow comparison with peers. As such, the data provided by these algorithms could serve as a foundation for ongoing interventions to improve antibiotic use.

The areas in which discrepancies occurred between the manual EHR review and the electronic algorithm provide important insights into how the algorithms could be improved. Clearer documentation by providers would improve the accuracy of the electronic algorithm. For example, had RSAT results been uniformly documented in the laboratory tests (not just in the progress notes), they would have been identified by the electronic algorithm. Particularly if algorithms are ultimately employed to provide feedback to individual providers, improvements in documentation will ensure that algorithms accurately reflect the appropriateness of prescribing practices.

In some cases, data available only in the encounter note were important in evaluating appropriateness. Particularly for conditions that are more complex than bronchitis and pharyngitis, accurately evaluating appropriateness of prescribing may more frequently require evaluation of text-based data. To this end, evaluating the potential impact of natural language processing approaches may be important in future efforts to develop electronic algorithms for antibiotic use [18, 19]. Consideration should be given to narrative documentation entered using templates that facilitate text-mining efforts for key data elements in the history and physical exam.

This study had several potential limitations. Because our health system uses 1 information system, our electronic algorithm should be further validated using other systems. Although this network included internal medicine and family practice physicians and physician extenders, coding and charting behaviors of clinicians across this network might not be representative of other practices. In addition, it is possible that some clinicians will attempt to avoid antibiotic stewardship scrutiny through diagnostic upcoding. Recognizing that antibiotic prescribing for bronchitis is not indicated, they might use a different code (eg, sinusitis) for which antibiotic use is more often appropriate. Such patients would not have been identified in our cohort. Finally, while our patient population represents considerable diversity, antibiotic use guidelines and prescribing practices may vary across systems and regions. As such, further validation of our algorithms in other settings would be desirable.

In summary, our electronic algorithms for identifying inappropriate prescribing, antibiotic choice, and duration for acute bronchitis and acute pharyngitis were highly accurate. These algorithms could be used to efficiently assess prescribing among practices and individual clinicians to better target ASP initiatives for these 2 conditions. The impact of interventions based on this algorithm should be tested in future work.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.