Development and evaluation of a primary care antimicrobial stewardship program (PC-ASP) in Toronto, Ontario, Canada

Abstract

Background

Effective community-based antimicrobial stewardship programs (ASPs) are needed because 90% of antimicrobials are prescribed in the community. A primary care ASP (PC-ASP) was evaluated for its effectiveness in lowering antibiotic prescriptions for six common infections.

Methods

A multi-faceted educational program was assessed using a before-and-after design in four primary care clinics from 2015 through 2017. The primary outcome was the difference between control and intervention clinics in total antibiotic prescriptions for six common infections before and after the intervention. Secondary outcomes included changes in condition-specific antibiotic use, delayed antibiotic prescriptions, prescriptions exceeding 7 days duration, use of recommended antibiotics, and emergency department visits or hospitalizations within 30 days. Multi-method models adjusting for demographics, case mix, and clustering by physician were used to estimate treatment effects.

Results

Total antibiotic prescriptions in control and intervention clinics did not differ (difference in differences = 1.7%; 95% CI –12.5% to 15.9%), nor did use of delayed prescriptions (–5.2%; 95% CI –24.2% to 13.8%). Prescriptions for longer than 7 days were significantly reduced (–21.3%; 95% CI –42.5% to –0.1%). However, only 781 of 1,777 encounters (44.0%) involved providers who completed the ASP education. Where providers completed the education, delayed prescriptions increased 17.7% (p = 0.06), and prescriptions exceeding 7 days duration declined (–27%; 95% CI –48.3% to –5.6%). Subsequent emergency department visits and hospitalizations did not increase.

Conclusions

PC-ASP effectiveness on antibiotic use was variable. Shorter prescription durations and increased use of delayed prescriptions were adopted by engaged primary care providers.

Introduction

Antibiotic resistance has increased globally, resulting in a call for action from governments and international organizations (1-4). Overuse of antibiotics is a key modifiable driver of resistance. The rate of antibiotic resistance in a country is directly related to the volume of antibiotics consumed there (5). In Canada, efforts to reduce human antibiotic overuse have focused on hospitals and long-term-care institutions (6,7). However, 90% of antibiotics are prescribed in the community (4,8). Most are prescribed for uncomplicated respiratory and urinary tract infections (8, 9), with as many as 30%–50% that are unnecessary (4,10,11). This makes the community setting a critical area for addressing antibiotic overuse.

Two-thirds of community antibiotic prescriptions are written by family doctors and nurse practitioners (8). Whereas antimicrobial stewardship programs (ASPs) in hospitals are common, community clinics lack the recommended infrastructure and funding for ASP activities (12). Key elements of a community ASP have been described (13), but examples of such programs are few, and evaluations have produced inconsistent results (14–16). In Canada, an ongoing community ASP effort in two provinces was associated with reduced prescriptions in one evaluation and reduced costs in another (17,18). However, both studies lacked control groups, which limited inferences about program effectiveness. As a result, additional efforts to identify effective community ASP models are needed.

In 2015, funding was obtained to develop a community-based primary care ASP (PC-ASP). The program was piloted and evaluated in four primary care clinics in Toronto, Ontario, Canada, to assess its feasibility and effectiveness. A qualitative study of participants’ views of the program’s feasibility has previously been reported (19). This article details the development of the PC-ASP and an assessment of its effectiveness on antibiotic prescribing for six common infections in adults.

Methods

Development of the PC-ASP

A multidisciplinary group of clinicians with expertise in hospital antimicrobial stewardship met with a convenience sample of clinicians, pharmacists, and support staff from selected primary care clinics. These community teams agreed to act as ASP champions at their respective sites (20). The group reviewed the hospital ASP to assess which processes were potentially feasible in primary care clinics. Additional interventions shown in randomized trials to reduce antibiotic use in primary care were also reviewed. A final set of interventions was selected to include in a multi-faceted PC-ASP.

Intervention

For this evaluation, the PC-ASP intervention was limited to adults aged 18 years or older and a few common infections, to minimize program complexity and maximize participation. The aim of the program was to reduce antibiotic prescriptions in adults presenting with sore throats (tonsillitis, pharyngitis), acute bronchitis, acute sinusitis, non-specific upper respiratory tract infections (URIs), and acute uncomplicated cystitis. These conditions account for 51% of community antibiotic prescriptions in Canada (8). The multi-faceted intervention included a clinician education program (21,22), patient education materials (23), prescribing decision aids (24–26), antibiotic-specific communication skills scripts for patient engagement (27), and patient advice regarding when to re-consult in cases in which antibiotics were not prescribed (‘safety-netting’) (28). Audit of and feedback on antibiotic prescribing at each intervention clinic was conducted by study team members (AS, WJM), and reports were provided to clinic champions to distribute via email to participants every 2 months (29,30).

The education modules addressed antimicrobial resistance, stewardship, and prescribing issues for each condition, and they provided condition-specific prescribing decision aids, communication scripts, safety-netting advice, options for delayed antibiotic prescriptions and prescription durations (31,32), and patient handouts. The modules took approximately 2 hours to complete and could be completed at the clinician’s convenience. Clinicians’ compliance was assessed by submission of a continuing medical education credit form upon module completion.

Additional optional tasks for non-clinical staff were identified through workflow analyses at each clinic (33). Each clinic chose the interventions considered most feasible in their setting, and participation was voluntary. However, all clinics were asked to complete the education modules and review audit and feedback reports. An initial on-site education session was offered with a reinforcing session after 4 months. Practitioners were aware of their clinic’s participation and the collection of prescribing data. The control site did not receive any intervention.

Study setting and design

The participating clinics comprised a convenience sample of three university-affiliated family medicine clinics that served as intervention clinics and one non-academic control clinic. Two of the intervention clinics provided postgraduate training for family medicine residents, and the third involved medical students only, as did the control clinic. All clinics were multidisciplinary and situated in Toronto or Vaughan. Participants were approximately 67 physicians and 6 nurse practitioners from the intervention clinics and 10 physicians and 3 nurse practitioners from the control site. A quasi-experimental pre- and post-study design with a concurrent control group was used to evaluate the PC-ASP. The intervention was implemented starting June 1, 2016, with prescribing data collected for 1 year before and 1 year after this date.

Data collection

Clinic visits were identified from billing records in the electronic medical record (EMR) system at each clinic. To ensure most eligible visits were identified, a broad set of respiratory and urinary tract codes from the International Classification of Diseases, Ninth Revision (ICD–9; 34) (Appendix A) were searched. Eligible codes were randomly selected from clinics each month. Visits for pregnancy and male urinary tract infections were excluded. Records were examined for visits in the previous 7 days. If the patient had a prior visit with the same diagnosis, this initial visit was selected instead. Follow-up visits after a patient had been first seen elsewhere (eg, walk-in clinic, emergency department) were excluded. A trained abstractor selected visits and extracted data.

Extracted variables included patient age, sex, antibiotic allergies, practice site, encounter date, provider written diagnosis, practitioner type (physician, nurse practitioner, resident), ICD–9 billing code, selected tests (eg, throat swab, urine culture), antibiotic prescriptions (yes, no, delayed), antibiotic name and duration of the prescription, and emergency visits or hospitalizations in the following 30 days. The provider’s diagnosis was compared with the ICD–9 billing code for each visit by one investigator (WJM) using standardized coding rules developed for the study (Appendix B). The final ICD–9 code assigned was adjusted if the provider’s written diagnosis indicated a condition for which an antibiotic might be considered. Otherwise, the original billing code was retained.

Study outcomes

The primary outcome was the difference in overall antibiotic prescriptions between the control and intervention clinics in the year before and after June 2016, for the six main infections combined (URI, pharyngitis, tonsillitis, acute sinusitis, acute bronchitis, and acute uncomplicated cystitis). Secondary outcomes included changes in condition-specific use of antibiotics; delayed antibiotic prescriptions; proportion of prescriptions exceeding 7 days duration; the proportion of first-line antibiotics (as per decision aids); and additional office visits, emergency department visits, or hospitalizations within 30 days.

Statistical analysis

Because baseline prescribing rates for these clinics to inform an accurate sample size determination were lacking, the sample size was based on the number of charts that it was felt practical to review. From the list of all eligible EMR billing codes identified, every seventh chart was randomly selected each month from each clinic for a total of 3,018 visit encounters that were reviewed. Visit and demographic characteristics were described with means, standard deviations, frequencies, and percentages. Differences in these characteristics between intervention clinic and the control clinic populations were determined using t-tests for continuous variables and chi-square or Fisher's exact tests for categorical variables, as appropriate. Crude and adjusted prescribing rate differences were determined for the before-and-after intervention periods, adjusting for differences in patient characteristics, the case mix of conditions, and clustering of patients by prescribers. Adjusted models were determined through mixed-method modelling using the SAS NLMIXED procedure (version 9.4; SAS Institute, Cary, NC, USA). The adjusted difference in differences and 95% confidence intervals between the control and intervention groups before and after the intervention were the main measures used to reflect the intervention effect. The study was approved by the research ethics boards of Sinai Health and University Health Network in Toronto.

Results

Figure 1 shows the number of encounters selected, reasons for exclusion, and final sample of eligible visits. The final sample included 1,212 visits in the year before the intervention and 1,207 in the year after. Of these, 1,823 (75.4%) visits involved the six infections that were the focus of the PC-ASP.

Figure 1:

Selection of the study sample and reasons for exclusion

Table 1 provides a comparison of visit characteristics at the control clinic (n = 313) and the three intervention clinics (n = 899) in the year before the intervention. There were no differences in the proportion of women and men (p = 0.61), but intervention clinic patients were older (mean age 50.4 y) than the control clinic patients (mean age 43.6 y; p < 0.01). Both groups had similar proportions of visits from winter and summer months (p = 0.65). Only intervention clinics had visits involving trainee residents. The mix of infections was similar in both groups (p = 0.065), with fewer cases of sore throats in control clinics (6.4%) than in intervention clinics (9.6%), and more cases of acute bronchitis in control clinics (10.5%) than in intervention clinics (5.3%). Antibiotic allergies were recorded as present in the EMR records of 21.1% of intervention clinic patients and 18.4% of control clinic patients (p = 0.29).

Table 1:

Comparison of primary care patient characteristics in control and intervention clinics in the year before implementation of the PC-ASP intervention (n = 1,212)

Characteristic	No. (%)		p-value
	Control clinic; n = 313	Intervention clinics; n = 899
Sex			0.61
Female	228 (72.8)	640 (71.2)
Male	85 (27.2)	259 (28.8)
Age, y			<0.01
18–49	228 (72.8)	476 (53.0)
50–65	63 (20.1)	219 (24.4)
≥65	22 (7.0)	204 (22.7)
Month seen			0.65
Jan–Mar	86 (27.5)	231 (25.7)
Apr–June	84 (26.8)	220 (24.5)
July–Sept	70 (22.4)	223 (24.8)
Oct–Dec	73 (23.3)	225 (25.0)
Type of practitioner			<0.01
FP	304 (97.1)	584 (65.0)
FP with resident	0 (0)	284 (31.6)
Nurse practitioner	9 (2.9)	31 (3.5)
Conditions*			0.065
URI (460, 464)	89 (28.4)	266 (29.6)
Sinusitis (461)	39 (12.5)	113 (12.6)
Sore throat (462, 463, 034)	20 (6.4)	86 (9.6)
Bronchitis (466)	33 (10.5)	48 (5.3)
Pneumonia (486)	22 (7.0)	57 (6.3)
UTI (595)	48 (15.3)	140 (15.6)
Other†	62 (19.8)	189 (21.0)
Antibiotic allergy reported	66 (21.1)	165 (18.4)	0.29
Antibiotic prescriptions
Crude	127 (40.6)	301 (33.5)	0.02
Case mix adjusted‡	123.8 (39.6)	300.6 (33.4)	<0.01
30-d follow-up
Emergency department or hospitalization	9 (2.9)	20 (2.2)	0.52
Clinical office	132 (42.2)	356 (39.6)	0.42

Note: Percentages may not total 100 because of rounding

^*International Classification of Diseases, Ninth Revision codes: 460, URI; 461, sinusitis; 462, pharyngitis; 463, tonsillitis; 464, laryngitis; 466, acute bronchitis; 486, pneumonia; 595, acute urinary tract infection; 034, strep

^† Includes influenza, 487; cough, 786; viral illness, 079; other urinary, 599

^‡ Adjusted for the mix of infection types (‘conditions’) using mixed-methods model

PC-ASP = Primary care antimicrobial stewardship program; FP = Family practitioner; URI = Upper respiratory tract infection; UTI = Urinary tract infection

Total antibiotic prescriptions for all diagnoses combined were higher at the control clinic (40.6%) than at the intervention clinics (33.5%; p = 0.02) in the year before the intervention. No differences were found between the control and intervention clinics in emergency department visits, hospitalizations or further office visits in 30 days after the initial visit. Factors associated with receiving an antibiotic prescription in the entire sample included being female (p < 0.01), clinic site (p < 0.01), and type of infection (p < 0.01; Table C.1 in Appendix C). The association with female sex was primarily due to having included uncomplicated urinary tract infections (women only) for which antibiotics were frequently prescribed (82.1%). These accounted for 44.5% of antibiotic prescriptions to women (data not shown). There was no association between patient age and antibiotic prescriptions (p = 0.34) or between an antibiotic allergy history and antibiotic prescriptions (p = 0.20).

The changes in antibiotic prescribing outcomes in control and intervention clinics in the year after the intervention compared with the year before, for only the six main PC-ASP infections combined, are shown in Table 2. No significant overall differences were found for changes in total antibiotic prescriptions at intervention and control clinics between baseline and intervention periods, adjusted for age, sex, case mix, and clustering of patients by physicians (overall difference in differences 1.7%; 95% CI –12.5% to 15.9%). Similarly, the difference in use of delayed prescriptions was not significantly different between the groups (–5.2%; 95% CI –24.2% to 13. 8%). Fewer antibiotics were prescribed for longer than 7 days in intervention clinics (–21.4%; 95% CI –42.6% to –0.1%). There was an increase in first-line antibiotic use at intervention clinics, but this was not statistically significant. When the six infections were examined individually, none of the differences achieved statistical significance for any prescribing outcomes (see Table C.2). When any infection was considered, the findings were unchanged (Table C.3).

Table 2:

Comparison of changes in antibiotic prescription outcomes between control and intervention primary care clinics in the year before and year after implementing the PC-ASP, for six infections^* in adults (N = 1,823)

Outcome	Pre-intervention		Post-intervention		Change pre–post
	n/N	Crude %	n/N	Crude %	Crude %	Adjusted %†	Difference in differences (95% CI)	p-value
Total antibiotic prescriptions
Control	109/229	47.6	94/269	34.9	–12.7	–8.3
Intervention	261/653	40.0	246/672	36.6	–3.4	–6.6	+1.7% (–12.5% to 15.9%)	0.81
Delayed antibiotic prescriptions
Control	30/106‡	28.3	35/93	37.6	+9.3	+11.5
Intervention	85/260	32.7	83/244	34.0	+1.3	+6.3	–5.2% (–24.2% to 13.8%)	0.59
Antibiotic prescriptions >7 days duration
Control	32/108	29.6	33/94	35.1	+5.5	+7.4
Intervention	71/259	27.4	36/242	14.9	–12.5	–14.0	–21.4% (–42.6% to –0.1%)	0.05
First-line antibiotic usage§
Control	66/109	60.6	55/94	58.5	–2.1	–0.9
Intervention	188/261	72.0	198/246	80.5	+8.5	+8.4	+9.3% (–9.1% to 27.5%)	0.32

^* Sore throats (tonsillitis, pharyngitis), acute sinusitis, acute bronchitis, upper respiratory infection, and acute cystitis

^† Adjusted for the case mix of conditions, age, sex, and clustering at the physician level

^‡ Denominators differ from total prescriptions in some cases because of missing data

^§ Adjusted for age, sex, and clustering at the physician level

PC-ASP = Primary care antimicrobial stewardship program

At intervention clinics, 781/1,777 (44.0%) visits involved providers who had completed the educational modules. Clinics also implemented different numbers of optional interventions. Clinic 2 held two initial educational meetings; used EMR decision aid support, reminder emails, computer reminders, physician posters, waiting room materials, and support staff activities; and held a 6-month educational reinforcement meeting. At this clinic, 412/582 (70.8%) of visits involved providers who had completed the educational modules. Clinic 4 had one educational pre-intervention meeting, posted decision aids in clinic rooms, printed patient handouts, held a 6-month educational reinforcement meeting, and had 306/631 (48.5%) visits involving providers who completed the education modules. Clinic 3 had no site visits and did not choose additional interventions or reinforcement sessions. They had 63/564 (11.2%) visits involving providers who completed education modules. To assess the effect of clinic and provider compliance, post hoc analyses were conducted of the intervention effectiveness by engagement (high, clinic 2; moderate, clinic 4; low, clinic 3), and by whether education modules had been completed.

The effectiveness of the intervention on prescribing outcomes by clinic engagement is shown in Table C.4. The highly engaged clinic demonstrated a non-significant reduction in total prescriptions compared with the control site for the six infections. Similarly, this clinic demonstrated non-significant increases in delayed antibiotic prescriptions and first-line antibiotic use. The reduction in prescriptions exceeding 7 days was greatest for the highly engaged clinic (p = 0.05). There were no differences in outcomes by clinic engagement when infections were examined individually, although numbers were small (analyses not shown).

Changes in antibiotic prescription outcomes by whether clinicians completed educational modules are shown in Table 3. Although no difference was found in total antibiotic prescriptions, there was a 17.7% absolute increase in use of delayed antibiotic prescriptions by providers completing the education modules (p = 0.06), and a 27.1% decrease in prescriptions issued for longer than 7 days (p = 0.01). There was no difference in first-line antibiotic use (p = 0.56), and few differences reached statistically significance when individual conditions were examined (Table C.5).

Table 3:

Comparison of changes in antibiotic prescription outcomes between control and intervention primary care clinics in the year before and after implementing the PC-ASP for six common infections in adults combined, by completion of PC-ASP education modules

Outcome	Pre-intervention		Post-intervention		Change pre–post
	n/N	Crude %	n/N	Crude %	Crude %	Adjusted* %	Difference in differences (95% CI)	p-value
Total antibiotic prescriptions
Not completed	270/614	44.0	218/611	35.7	–8.3	–8.1
Completed	100/268	37.3	122/330	37.0	0.3	–4.4	+3.8% (–9.3% to 16.8%)	0.57
Delayed antibiotic prescriptions
Not completed	84/267†	31.5	65/216	30.1	–1.4	+1.4
Completed	31/99	31.3	53/121	43.8	+12.5	+19.0	+17.7% (–1.1% to 36.4%)	0.06
Antibiotic prescriptions >7 days duration
Not completed	67/268	25.0	52/217	24.0	–1.0	+1.2
Completed	36/99	36.4	17/119	14.3	–22.1	—25.9	—27.1% (—48.5% to —5.6%)	0.01
First-line antibiotic usage‡
Not completed	178/270	65.9	158/218	72.5	+6.6	+6.3
Completed	76/100	76.0	95/122	77.9	+1.9	+1.2	–5.2% (–22.7% to 12.4%)	0.56

^* Rates adjusted for case mix of eligible conditions, age, sex, and clustering at the physician level

^† Denominators differ from total prescriptions because of missing data

^‡ Rates adjusted for age, sex, and clustering at the physician level

PC-ASP = primary care antimicrobial stewardship program

No significant increase was found in emergency department visits or hospitalizations in the 30 days after an initial office encounter in intervention patients (Table C.6). There was a non-significant trend toward increased office visits in this group (9.8% increase, p = 0.07). There was no clear relationship between clinic engagement or completion of educational modules and additional office visits (Table C.7).

Discussion

The effectiveness of the PC-ASP on antibiotic use was variable. The main consistent change was a decrease in the duration of antibiotic prescriptions. Some changes in other prescribing outcomes were evident in clinics that demonstrated higher levels of engagement with the intervention and where primary care providers had completed the educational component of this multi-faceted intervention.

Overall antibiotic prescriptions were not reduced in this study. A failure to reduce total antibiotic prescriptions has been noted in other studies as well (14,35,36). A Dutch study of education outreach and feedback on prescribing behaviour found no effect on total antibiotic prescriptions or use of recommended antibiotics (35). Similarly, a Spanish study that used one-on-one peer education for a large health region reported fewer inappropriate prescriptions but no change in total antibiotic prescriptions (14). A Scottish study was successful in reducing Clostridium difficile– promoting antibiotics, but there was no reduction in overall antibiotic use (36). This lack of effects on total prescriptions was attributed to insufficient intervention intensity (35), an influenza epidemic (14), and antibiotic substitution (36). In the current study, a lack of provider engagement may have contributed.

A reduction in antibiotic prescriptions exceeding 7 days was evident in clinics that demonstrated high engagement and in which providers had completed the educational modules. The latter also demonstrated an increase in the use of delayed antibiotic prescriptions. This is relevant because only 30% of delayed prescriptions appear to be filled (31). We observed that delayed prescriptions were common even before the study, suggesting that this strategy is readily acceptable to primary care providers. Of note, there was a trend toward increased subsequent office visits to providers in intervention clinics. Whether this was related to greater use of delayed prescriptions requiring follow-up visits is unknown, but it may warrant further study. The reduction in longer-duration antibiotic prescriptions was the only outcome that changed in intervention clinics. Efforts to promote the wider adoption of delayed antibiotic prescriptions and shorter-duration prescriptions seem warranted because these stewardship practices appear to be acceptable to primary care clinicians.

The importance of provider engagement in community ASP efforts has been noted in other studies (37,38). An Israeli study of pediatricians focused on promoting physician engagement through workshops and involvement in developing interventions. Over 4 years, a 9.4% decrease in total antibiotic prescriptions was observed (37). An American study of behavioural interventions for reducing antibiotics in viral respiratory syndrome reported a 5% decrease in prescriptions (38). The study provided $1,200 to physicians and achieved 70% participation. Participation in the present study was voluntary, with no incentives other than continuing medical education credits. This may have contributed to the low engagement by some clinics. Incentives or direct program involvement may be helpful in promoting provider engagement in multi-faceted community ASPs.

Other Canadian and international ASP efforts have also reported mixed effects on prescribing (15,17,39). A trial in academic family medicine walk-in clinics reported a 50% decline in the self-reported intention of patients to use antibiotics after a visit for respiratory infections but did not count delayed prescriptions or dispensed antibiotics, resulting in an unclear impact on total antibiotic prescriptions (39). A Canadian program to promote optimal antimicrobial use reported reduced physician prescribing, but the evaluation lacked a control group (17). A multi-faceted British PC-ASP found no effect on total antibiotic use, except in clinics that complied with the intervention (15). A community stewardship initiative developed by the Public Health Agency of Canada and Choosing Wisely Canada is pending an evaluation of its impact on antibiotic use (40).

Limitations of the current study included low power to determine prescribing changes for individual infections. There was variable intervention compliance with low completion rates of educational modules and few on-site educational sessions by some clinics. Audit and feedback reports, shown to be effective in reducing antibiotic prescribing (29,30), were emailed to prescribers but may not have been reviewed. Some prescribing changes were evident only in post hoc analyses, and results were not corrected for multiple comparisons. Whereas the use of clinical decision aids was central to the PC-ASP intervention, this use occurred in the context of usual provider–patient communication. Other studies with more explicit patient communication or engagement in decision making about antibiotic use have reported larger reductions in antibiotic prescriptions (27,39). This may also have contributed to the limited impact of the PC-ASP intervention. Strengths of the study included incorporating the written diagnosis and not relying solely on billing claims. However, we were not able to definitively assess the appropriateness of each antibiotic prescription. The inclusion of delayed antibiotic prescriptions and duration of prescriptions identified changes that may be overlooked in studies that focus on total prescriptions. Finally, the prescribing rates were adjusted for the differing case mixes of infections in each clinic to more fairly compare clinic prescribing rates (41).

Conclusion

This PC-ASP demonstrated limited effectiveness on antibiotic use, possibly as a result of limited provider engagement, a lack of incentives, or insufficient intervention intensity. Delayed antibiotic prescriptions were used before the study but increased among engaged providers. Reducing the duration of antibiotic prescriptions was also successfully promoted. These strategies may be more likely to be successfully adopted in the community. To implement complex community ASP interventions, more effective strategies to engage community clinicians are likely needed.

Appendix A

International Classification of Diseases, Ninth Revision, billing codes initially selected from clinic electronic medical record billing records:

• 034—Streptococcal sore throat

• 079—Viral infections, unspecified

• 460—Acute nasopharyngitis (common cold)

• 461—Acute sinusitis

• 462—Acute pharyngitis

• 463—Acute tonsillitis

• 464—Acute laryngitis

• 466—Acute bronchitis

• 486—Pneumonia

• 487—Influenza

• 595—Cystitis

• 599—Urinary, unspecified

• 786—Symptoms including respiratory symptoms and other chest symptoms

Appendix B

Coding rules for assigning a final-visit diagnosis:

1. If Bill Dx and Visit Dx are an exact match, code same.

   1. Ex. Bill Dx: 460, Visit Dx: ‘URI’—Final code is 460
2. If Visit Dx contains Bill Dx with extra non-condition words, code with the Bill Dx.

   1. Ex. Bill Dx: 460, Visit Dx: ‘most likely URI’—Final code is 460
   2. If Visit Dx contains the word and another infection condition for which antibiotics are sometimes prescribed (Box B.2), then Dx recoded to new condition: Ex. Bill Dx: 460, Visit Dx: ‘?URI, sinusitis’—Final code is 461 (sinusitis)
3. If Visit Dx contains any other infection words not contained in Bill Dx, then recode Dx to the Visit Dx.

   1. Ex. Bill Dx: 460, Visit Dx: ‘Influenza’ - Final Recode is 487
4. If Visit Dx contains no words of the Bill Dx or one of the conditions, then it is ineligible (Box B.1).

   1. Ex. Bill Dx: 460, Visit Dx: ‘post-nasal drip’ OR ‘halitosis’—Final recode is ‘ineligible’

5. If Bill Dx is 461 and Visit Dx is ‘chronic sinusitis; chronic rhinosinusitis; ?chronic sinusitis,’ then it is ineligible.

6. If Bill Dx listed and only viral infection listed under Visit Dx, retain Bill Dx.

   1. Ex. Bill Dx: 466, Visit Dx: ‘viral’—Final recode is 466

7. If no Visit Dx recorded, then use Bill Dx.

8. If more than one Visit Dx and Bill Dx matches one of the Dx, then use Bill Dx.

   1. Ex. Bill Dx: 595, Visit Dx: ‘cough and UTI’—Final recode is 595
   2. Ex. Bill Dx: 460, Visit Dx: ‘URI and ear infection’—Final recode is 460

Box B.1: Eligible Coded Diagnoses.

Eligible diagnoses URI (460) Sinusitis (461) Pharyngitis (462) Strep (034) Tonsillitis (463) Laryngitis (464) Bronchitis (466) Pneumonia (486) Influenza (487) Lower respiratory tract infection/cough (786) Urinary tract in fection (595) Urinary other (599)

Ineligible diagnoses Asthma (493) Chronic obstructive pulmonary disease (491) Chronic sinusitis (473) Post-nasal drip Pyelonephritis (590) Interstitial cystitis non-matching bill dx and visit dx (eg, 460 and new patient visit)

dx = Diagnosis

Box B.2: Conditions in Which Antibiotics May Sometimes Be Appropriate.

Sinusitis (461) Pharyngitis (462) Tonsillitis (463) Strep (034) Bronchitis (466) Pneumonia (486) Cystitis/urinary tract infection (595) Cough (786)

Appendix C

Table C.1:

Patient and clinic factors associated with an antibiotic prescription, all non–follow-up visits (N = 2,577), unadjusted

Characteristic	n (%)		p-value
	No antibiotic Rx	Antibiotic Rx
Total sample	1,736 (67.4)	841 (32.6)	NA
Period			0.12
Pre-intervention (June 2015–May 2016)	845 (65.9)	437 (34.1)
Post-intervention (June 2016–May 2017)	891 (68.8)	404 (31.2)
Groups			<0.01
Control	421 (62.6)	252 (37.4)
Intervention	1,315 (69.1)	589 (30.9)
Sites			<0.01
Site 1 (Control)	421 (62.6)	252 (37.4)
Site 2	462 (74.5)	158 (25.5)
Site 3	429 (68.0)	202 (32.0)
Site 4	424 (64.9)	229 (35.1)
Age (mean, SD)	48.0 (17.4)	48.7 (16.9)	0.34
Gender			<0.01
Female	1,138 (62.4)	685 (37.6)
Male	598 (79.3)	156 (20.7)
Practitioners			<0.01
Staff physician	1,231 (65.5)	647 (34.5)
Staff with resident	464 (73.1)	171 (26.9)
Nurse practitioner	41 (64.1)	23 (35.9)
Reported antibiotic allergy			0.20
No	1,412 (67.9)	668 (32.1)
Yes	324 (65.2)	173 (34.8)
Diagnostic codes			<0.01
URI (460,464)	670 (91.0)	66 (9.0)
Sinusitis (461)	93 (30.8)	209 (69.2)
Sore throat (462, 462, 034)*	154 (69.4)	68 (30.6)
Bronchitis (466)	129 (68.3)	60 (31.8)
Pneumonia (486)	49 (34.0)	95 (66.0)
Cystitis (595)	67 (17.9)	307 (82.1)
Other (487, 599, 786, 079)†	574 (94.1)	36 (5.9)

Notes: Percentages are based on n for row.

^* Includes pharyngitis (462), tonsillitis (463), and strep (034)

^† Includes influenza (487), cough (786), viral illness (079), other urinary (599)

NA = Not applicable; Rx = Prescription; URI = Upper respiratory infection

Table C.2:

Comparison of changes in antibiotic prescriptions to adults between control and intervention primary care clinics in the year before and after implementing the PC-ASP intervention, by infection type

Condition and outcome	Control, n/N (%)		Intervention, n/N (%)		Change pre–post
	Pre-intervention	Post-intervention	Pre-intervention	Post-intervention	Control, crude % (adjusted %*)	Intervention, crude % (adjusted %*)	Difference in differences (95% CI)	p-value
URI
Total antibiotics	10/89 (11.2)	7/122 (5.7)	32/266 (12.0)	17/259 (6.6)	–5.4 (–5.3)	–5.5 (–5.1)	+0.1% (–9.4% to 9.7%)	0.98
Delayed antibiotic	4/10 (40.0)	5/6 (83.3)	23/32 (71.9)	16/17 (94.1)	+43.3 (+45.1)	+22.2 (+23.2)	–21.9% (–77.3% to 33.5%)	0.43
>7 d	3/9† (33.3)	2/7 (28.6)	12/31 (38.7)	4/16 (25.0)	–4.8 (+32.9)	–13.7 (–34.1)	–67.0% (–175.7% to 41.7%)	0.22
Sinusitis
Total antibiotics	33/39 (84.6)	35/42 (83.3)	69/113 (61.1)	72/108 (66.7)	–1.3 (–5.0)	+5.6 (+10.4)	+15.4% (–6.1% to 36.9%)	0.16
Delayed antibiotics	11/32 (34.4)	11/35 (31.4)	26/69 (37.7)	24/71 (33.8)	–3.0 (–3.4)	–3.9 (–7.1)	–3.7% (–49.4% to 42.0%)	0.87
>7 d	16/33 (48.5)	22/35 (62.9)	21/69 (30.4)	14/72 (19.4)	+14.4 (+19.1)	–11.0 (–10.2)	–29.2% (–68.7% to 10.1%)	0.14
Sore throat
Total antibiotics	7/20 (35.0)	6/36 (16.7)	35/86 (40.7)	20/80 (25.0)	–18.3 (–18.2)	–15.7 (–15.1)	+3.1% (–25.5% to 31.6%)	0.83
Delayed antibiotic	1/7 (14.3)	3/6 (50.0)	18/35 (51.4)	13/20 (65.0)	+35.7 (+29.1)	+13.6 (+16.8)	–12.5% (–66.6% to 41.5%)	0.64
>7 d	6/7 (85.7)	6/6 (100)	24/35 (68.6)	12/20 (60.0)	+14.3 (+7.3)	–8.6 (+4.6)	–2.0% (–57.2% to 53.3%)	0.94
Bronchitis
Total antibiotics	16/33 (48.5)	17/38 (44.7)	11/48 (22.9)	16/70 (22.9)	–3.8% (–0.7)	–0.06% (+0.8)	+1.6% (–28.3% to 31.5%)	0.92
Delayed antibiotic	9/16 (56.3)	11/17 (64.7)	4/11 (36.4)	8/16 (50.0)	+8.5% (+8.9)	+13.6% (+6.2)	–2.7% (–81.2% to 75.8%)	0.94
>7 d	5/16 (31.3)	1/17 (5.9)	2/11 (18.2)	1/16 (6.3)	–25.4% (–20.3)	–11.9% (–3.8)	+16.5% (–23.6% to 56.6%)	0.40
Cystitis
Total antibiotics	43/48 (89.6)	29/31 (93.6)	114/140 (81.4)	121/155 (78.1)	+4.0% (+3.9)	–3.4% (–3.5)	–7.5% (–23.0% to 8.0%)	0.34
Delayed antibiotic	5/41† (12.2)	5/29 (17.2)	14/113 (12.4)	22/120 (18.3)	+5.1% (+4.3)	+5.9% (+6.3)	+1.9% (–18.1% to 22.0%)	0.85
>7 d	2/43 (4.7)	2/29 (6.9)	12/113 (10.6)	5/118 (4.2)	+2.3% (+2.2)	–6.4% (–8.9)	–11.2% (–58.9% to 36.5%)	0.64

^* Rates adjusted for age, sex, and clustering at the physician level

^† Denominators differ from total prescriptions in some cases due to missing data

PC-ASP = primary care antimicrobial stewardship program; URI = Upper respiratory infection

Table C.3:

Comparison of changes in antibiotic prescription outcomes for any infection between control and intervention primary care clinics in the year before and after implementing the PC-ASP (N = 2,419)

Outcome	Pre-intervention		Post-intervention		Change pre–post
	n/N	Crude %	n/N	Crude %	Crude %	Adjusted %*	Difference in differences (95% CI)	p-value
Total antibiotic prescriptions
Control	127/313	40.6	120/329	36.5	–4.1	+0.2
Intervention	301/899	33.5	280/878	31.9	–1.6	–4.2	–4.4% (–17.2% to 8.4%)	0.49
Delayed antibiotic prescriptions
Control	32/123†	26.0	45/118	38.1	+12.1	+15.4
Intervention	92/298	30.9	92/278	33.1	+2.2	+7.0	–8.4% (–25.5% to 8.8%)	0.33
Duration of antibiotic prescriptions
Control	40/126	31.7	47/120	39.2	+7.5	+8.7
Intervention	78/296	26.4	39/275	14.2	–12.2	–11.2	–19.9% (–37.6% to –2.3%)	0.03

^* Adjusted for case mix of conditions, age, sex, and clustering at the physician level

^† Denominators differ from total prescriptions due to missing data

PC-ASP = primary care antimicrobial stewardship program

Table C.4:

Comparison of changes in antibiotic prescription outcomes between control and intervention primary care clinics in the year before and after implementing the PC-ASP for six common infections in adults combined, by site engagement

Outcome and clinic engagement	Pre-intervention		Post-intervention		Change pre–post
	n/N	Crude % rate	n/N	Crude % rate	Crude%	Adjusted%*	Difference in differences (95% CI)	p-value
Total antibiotic prescriptions
Control	109/229	47.6	94/269	34.9	–12.7	–8.5
Low	92/208	44.2	76/194	39.2	–5.0	–5.1	+3.0% (–14.6% to 20.6%)	0.73
Moderate	109/259	42.1	96/250	38.4	–3.7	–5.8	+2.1% (–16.2% to 20.3%)	0.82
High	60/186	32.3	74/228	32.5	+0.2	–10.4	–1.9% (–20.7% to 16.9%)	0.83
Delayed use of prescriptions†
Control	30/106‡	28.3	35/93	37.6	+9.3	+9.9
Low	22/92	23.9	11/75	14.7	–9.2	–12.1	–20.0% (–42.8% to 2.9%)	0.08
Moderate	40/108	37.0	42/95	44.2	+7.2	+9.0	–2.7% (–26.1% to 20.7%)	0.82
High	23/60	38.3	30/74	40.5	+2.2	+13.0	+3.1% (–22.9% to 29.1%)	0.81
Antibiotic prescriptions >7 d duration†
Control	32/108	29.6	33/94	35.1	+5.5	+5.5
Low	19/92	20.7	15/75	20.0	–0.7	+1.6	–3.1% (–34.1% to 27.9%)	0.84
Moderate	26/107	24.3	11/95	11.6	–12.7	–8.3	–14.2% (–37.8% to 9.4%)	0.23
High	26/60	43.3	10/72	13.9	–29.4	–25.4	–30.9% (–62.5% to 0.6%)	0.05
First-line antibiotic usage‡
Control	66/109	60.6	55/94	58.5	–2.1	–0.5
Low	72/92	78.3	62/76	81.6	+3.3	–0.7	+0.01% (–25.6% to 25.6%)	0.99
Moderate	69/109	63.3	72/96	75.0	+11.7	+13.2	+14.4% (–8.2% to 36.9%)	0.20
High	47/60	78.3	64/74	86.5	+8.2	+9.9	+10.5% (–12.9% to 33.8%)	0.37

^* Rates adjusted for case mix of eligible conditions, age, sex, and clustering at the physician level

^† Denominators differ in some cases from total prescriptions due to missing data

^‡ Rates adjusted for age, sex, and clustering at the physician level

PC-ASP = primary care antimicrobial stewardship program; high = Clinic 2; moderate = Clinic 4; low = Clinic 3

Table C.5:

Comparison of changes in antibiotic prescription outcomes for individual infections in adults between control and intervention primary care clinics in the year before and year after implementing the PC-ASP intervention, by module completion

Condition and outcome	Not completed, n/N (%)		Completed, n/N (%)		Change pre–post
	Pre-intervention	Post-intervention	Pre-intervention	Post-intervention	Control Crude % (adjusted %*)	Intervention Crude % (adjusted%*)	Difference in differences (95% CI)	p-value
URI
Total antibiotics	31/233 (13.3)	14/245 (5.7)	11/122 (9.0)	10/136 (7.4)	–7.6 (–7.1)	–1.7 (–1.7)	+5.4% (–3.3% to 14.0%)	0.22
Delayed antibiotic	19/31 (61.3)	11/13 (84.6)	8/11 (72.7)	10/10 (100)	+23.3 (+27.2)	+27.3 (+23.3)	–3.9% (–54.6% to 46.8%)	0.88
>7 d	10/29 (34.5)	3/13 (23.1)	5/11 (45.5)	3/10 (30.0)	–11.4 (+4.1)	–15.5 (–55.1)	–59.2% (–150.3% to 31.9%)	0.20
Sinusitis
Total antibiotics	72/105 (68.6)	74/108 (68.5)	30/47 (63.8)	33/42 (78.6)	–0.05 (+2.0)	+14.7 (+15.5)	+13.5% (–13.2% to 40.2%)	0.32
Delayed antibiotic	24/71† (33.8)	23/73 (31.5)	13/30 (43.3)	12/33 (36.4)	–2.3 (–3.0)	–7.0 (–8.8)	–5.8% (–33.9% to 22.2%)	0.68
>7 d	24/72 (33.3)	32/74 (43.2)	13/30 (43.3)	4/33 (12.1)	+9.9 (+14.8)	–31.2 (–34.0)	–48.8% (–85.2% to-12.3%)	< 0.01
Sore throat
Total antibiotics	26/74 (35.1)	15/77 (19.5)	16/32 (50.0)	11/39 (28.2)	–15.7 (–13.8)	–21.8 (–22.3)	–8.5% (–35.5% to 18.6%)	0.53
Delayed antibiotic	13/26 (50.0)	7/15 (46.7)	6/16 (37.5)	9/11 (81.8)	–3.3 (+0.7)	+44.3 (+51.1)	+50.4% (5.7% to 95.1%)	0.03
>7 d	17/26 (65.4)	10/15 (66.7)	13/16 (81.3)	8/11 (72.7)	+1.3 (+19.7)	–8.5 (–8.4)	–28.1% (–92.5% to 36.4%)	0.38
Bronchitis
Total antibiotics	25/60 (41.7)	22/69 (31.9)	2/21 (9.5)	11/39 (28.2)	–9.8% (–6.2)	+18.7 (+14.0)	+20.1% (–4.6% to 44.9%)	0.11
Delayed antibiotic	12/25 (48.0)	11/22 (50.0)	1/2 (50.0)	8/11 (72.7)	+2.0% (–5.2)	+22.7 (+49.6)	+54.8% (–59.6% to 169.2%)	0.34
>7 d	7/25 (28.0)	2/22 (9.1)	0/2 (0.0)	0/11 (0.0)	–18.9% (–15.7)	0.0 (—)	+15.7% (–11.7% to 43.0%)	0.25
Cystitis
Total antibiotics	116/142 (81.7)	93/112 (83.0)	41/46 (89.1)	57/74 (77.0)	+1.4% (+1.4)	–12.1 (–12.0)	–13.4% (–29.9% to 3.0%)	0.11
Delayed antibiotic	16/114†(14.0)	13/93 (14.0)	3/40 (7.5)	14/56 (25.0)	–0.06% (+0.0)	+17.5 (+0.03)	+0.03% (–69.7% to 69.7%)	0.99
>7 d	9/116 (7.8)	5/93 (5.4)	5/40 (12.5)	2/54 (3.7)	–2.4% (–4.1)	–8.8% (–10.3)	–6.2% (–38.3% to 26.0%)	0.70

^* Rates adjusted for age, sex and clustering at the physician level

^† Denominators differ from total prescriptions in some cases due to missing data

PC-ASP = primary care antimicrobial stewardship program; URI = Urinary tract infection

Table C.6:

Comparison of changes in clinical office follow-up visits and ED visits or hospitalizations within 30 days between control and intervention clinics, before and after implementation of the PC-ASP for six common infections in adults

Outcomes	Pre-intervention		Post-intervention		Change pre–post
	n/N	Crude %	n/N	Crude %	Crude %	Adjusted%*	Difference in differences (95% CI)	p-value
ED visits and hospitalizations
Control	5/229	2.2	5/269	1.9	–0.3	–0.4
Intervention	9/653	1.4	12/672	1.8	+0.4	+0.1	+0.5% (–1.9% to 2.9%)	0.69
Clinical office visit follow-ups
Control	99/229	43.2	94/269	34.9	–8.3	–6.8
Intervention	257/653	39.4	285/672	42.4	+3.0	+3.0	+9.8% (–0.8% to 20.5%)	0.07

^* Rates adjusted for age, sex, case mix of eligible conditions, and clustering at the physician level

ED = Emergency department; PC-ASP = primary care antimicrobial stewardship program

Table C.7:

Comparison of changes in clinical office follow-up visits and emergency room visits or hospitalizations within 30 days between control and intervention clinics before and after implementation of the PC-ASP, by clinic engagement and module completion

Outcome and group	Pre-intervention		Post-intervention		Change pre–post
	n/N	Crude %	n/N	Crude %	Crude %	Adjusted %*	Difference in differences (95% CI)	p-value
Emergency hospitalizations
By clinic (engagement)
Control	5/229	2.2	5/269	1.9	–0.3	–0.5
Low	1/208	0.5	1/194	0.5	+0.03	–0.00	+0.3% (–1.5% to 1.9%)	0.77
Moderate	4/159	1.5	4/250	1.6	+0.06	–0.2	+0.3% (–2.1% to 2.7%)	0.81
High	4/186	2.2	7/228	3.1	+0.9	+0.01	+0.6% (–2.4% to 3.6%)	0.67
By module (completion)
Not completed	9/614	1.5	9/611	1.5	+0.01	–0.2
Completed	5/268	1.9	8/330	2.4	+0.6	+0.01	+0.3% (–2.0% to 2.6%)	0.79
Clinical office follow-up visit
By clinic engagement
Control	99/229	43.2	94/269	34.9	–8.3	–7.6
Low	81/208	38.9	85/194	43.8	+4.9	+6.4	+13.1% (–0.7% to 26.8%)	0.06
Moderate	110/259	42.5	105/250	42.0	–0.5	–0.5	+6.3% (–7.5% to 20.0%)	0.36
High	66/186	35.5	95/228	41.7	+6.2	+4.1	+11.7% (–1.4% to 24.8%)	0.08
By module completion
Not completed	250/614	40.7	232/611	38.0	–2.8	–1.4
Completed	106/268	39.6	147/330	44.6	+5.0	+3.9	+5.3% (–4.5% to 15.2%)	0.29

^*Rates adjusted for age, sex, case mix of eligible conditions, and clustering at the physician level

PC-ASP = primary care antimicrobial stewardship program

Funding Statement

Funding was provided by the Innovation Fund of the Alternative Funding Plan for the Academic Health Sciences Centres of Ontario.

Ethics Approval:

The study was approved by the research ethics boards of Sinai Health and University Health Network in Toronto.

Informed Consent:

N/A

Funding:

Funding was provided by the Innovation Fund of the Alternative Funding Plan for the Academic Health Sciences Centres of Ontario.

Disclosures:

Linda Dresser reports grants from Merck, Avir, and Sunovion outside of the submitted work.

Peer Review:

This manuscript has been peer reviewed.

Animal Studies:

N/A