Interventions to improve antibiotic prescribing practices for hospital inpatients

Abstract

Background

Antibiotic resistance is a major public health problem. Infections caused by multidrug‐resistant bacteria are associated with prolonged hospital stay and death compared with infections caused by susceptible bacteria. Appropriate antibiotic use in hospitals should ensure effective treatment of patients with infection and reduce unnecessary prescriptions. We updated this systematic review to evaluate the impact of interventions to improve antibiotic prescribing to hospital inpatients.

Objectives

To estimate the effectiveness and safety of interventions to improve antibiotic prescribing to hospital inpatients and to investigate the effect of two intervention functions: restriction and enablement.

Search methods

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) (the Cochrane Library), MEDLINE, and Embase. We searched for additional studies using the bibliographies of included articles and personal files. The last search from which records were evaluated and any studies identified incorporated into the review was January 2015.

Selection criteria

We included randomised controlled trials (RCTs) and non‐randomised studies (NRS). We included three non‐randomised study designs to measure behavioural and clinical outcomes and analyse variation in the effects: non‐ randomised trials (NRT), controlled before‐after (CBA) studies and interrupted time series (ITS) studies. For this update we also included three additional NRS designs (case control, cohort, and qualitative studies) to identify unintended consequences. Interventions included any professional or structural interventions as defined by the Cochrane Effective Practice and Organisation of Care Group. We defined restriction as 'using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours)'. We defined enablement as 'increasing means/reducing barriers to increase capability or opportunity'. The main comparison was between intervention and no intervention.

Data collection and analysis

Two review authors extracted data and assessed study risk of bias. We performed meta‐analysis and meta‐regression of RCTs and meta‐regression of ITS studies. We classified behaviour change functions for all interventions in the review, including those studies in the previously published versions. We analysed dichotomous data with a risk difference (RD). We assessed certainty of evidence with GRADE criteria.

Main results

This review includes 221 studies (58 RCTs, and 163 NRS). Most studies were from North America (96) or Europe (87). The remaining studies were from Asia (19), South America (8), Australia (8), and the East Asia (3). Although 62% of RCTs were at a high risk of bias, the results for the main review outcomes were similar when we restricted the analysis to studies at low risk of bias.

More hospital inpatients were treated according to antibiotic prescribing policy with the intervention compared with no intervention based on 29 RCTs of predominantly enablement interventions (RD 15%, 95% confidence interval (CI) 14% to 16%; 23,394 participants; high‐certainty evidence). This represents an increase from 43% to 58% .There were high levels of heterogeneity of effect size but the direction consistently favoured intervention.

The duration of antibiotic treatment decreased by 1.95 days (95% CI 2.22 to 1.67; 14 RCTs; 3318 participants; high‐certainty evidence) from 11.0 days. Information from non‐randomised studies showed interventions to be associated with improvement in prescribing according to antibiotic policy in routine clinical practice, with 70% of interventions being hospital‐wide compared with 31% for RCTs. The risk of death was similar between intervention and control groups (11% in both arms), indicating that antibiotic use can likely be reduced without adversely affecting mortality (RD 0%, 95% CI ‐1% to 0%; 28 RCTs; 15,827 participants; moderate‐certainty evidence). Antibiotic stewardship interventions probably reduce length of stay by 1.12 days (95% CI 0.7 to 1.54 days; 15 RCTs; 3834 participants; moderate‐certainty evidence). One RCT and six NRS raised concerns that restrictive interventions may lead to delay in treatment and negative professional culture because of breakdown in communication and trust between infection specialists and clinical teams (low‐certainty evidence).

Both enablement and restriction were independently associated with increased compliance with antibiotic policies, and enablement enhanced the effect of restrictive interventions (high‐certainty evidence). Enabling interventions that included feedback were probably more effective than those that did not (moderate‐certainty evidence).

There was very low‐certainty evidence about the effect of the interventions on reducing Clostridium difficile infections (median ‐48.6%, interquartile range ‐80.7% to ‐19.2%; 7 studies). This was also the case for resistant gram‐negative bacteria (median ‐12.9%, interquartile range ‐35.3% to 25.2%; 11 studies) and resistant gram‐positive bacteria (median ‐19.3%, interquartile range ‐50.1% to +23.1%; 9 studies). There was too much variance in microbial outcomes to reliably assess the effect of change in antibiotic use.

Heterogeneity of intervention effect on prescribing outcomes

We analysed effect modifiers in 29 RCTs and 91 ITS studies. Enablement and restriction were independently associated with a larger effect size (high‐certainty evidence). Feedback was included in 4 (17%) of 23 RCTs and 20 (47%) of 43 ITS studies of enabling interventions and was associated with greater intervention effect. Enablement was included in 13 (45%) of 29 ITS studies with restrictive interventions and enhanced intervention effect.

Authors' conclusions

We found high‐certainty evidence that interventions are effective in increasing compliance with antibiotic policy and reducing duration of antibiotic treatment. Lower use of antibiotics probably does not increase mortality and likely reduces length of stay. Additional trials comparing antibiotic stewardship with no intervention are unlikely to change our conclusions. Enablement consistently increased the effect of interventions, including those with a restrictive component. Although feedback further increased intervention effect, it was used in only a minority of enabling interventions. Interventions were successful in safely reducing unnecessary antibiotic use in hospitals, despite the fact that the majority did not use the most effective behaviour change techniques. Consequently, effective dissemination of our findings could have considerable health service and policy impact. Future research should instead focus on targeting treatment and assessing other measures of patient safety, assess different stewardship interventions, and explore the barriers and facilitators to implementation. More research is required on unintended consequences of restrictive interventions.

Plain language summary

Improving how physicians working in hospital settings prescribe antibiotics

Review aim

The aim of this Cochrane review was to learn of ways to improve how physicians working in hospital settings prescribe antibiotics. We collected and analysed all relevant studies to answer this question and found 221 studies.

Key messages

The use of an antibiotic policy leads to improved prescribing practices and decreases in the duration of antibiotic treatment.

Interventions that are directed to physicians to improve their antibiotic prescribing practices reduced participant length of stay in hospitals by 1.12 days (based on findings from 15 studies) and did not increase the risk of death (based on findings from 29 studies). Interventions providing advice or feedback to physicians were more effective in improving prescribing practices than those interventions that did not provide this information to physicians. Evidence from seven studies raised concerns that with interventions applying rules to make physicians prescribe properly there were delays in treatment and a breakdown in trust between infection specialists and clinical teams.

What was studied in the review?

Antibiotics are used to treat bacterial infections such as pneumonia. Many bacteria have become resistant to antibiotics over time. Antibiotic resistance is a serious problem for patients and healthcare systems because infections caused by antibiotic‐resistant bacteria can lead to higher rates of death and longer hospital stays. Bacterial resistance often occurs because antibiotics are used when they are not needed. Studies have shown that in about half of cases physicians in hospital are not prescribing antibiotics properly.

We investigated the effectiveness and safety of interventions to help physicians prescribe antibiotics properly and what techniques of behaviour change could influence the success of the interventions.

Key results

We found 221 relevant studies. Ninety‐six studies were from North America. The remaining 125 studies were from Europe (87), Asia (19), South America (8), Australia (8), and East Asia (3). The studies tested interventions that fell broadly into two categories: restrictive techniques, which apply rules to make physicians prescribe properly, and enablement techniques, which provide advice or feedback to help physicians prescribe properly.

We found high‐certainty evidence that interventions lead to more hospital inpatients receiving the appropriate treatment for their condition according to antibiotic prescribing policies. We found moderate‐certainty evidence that interventions reduce the length of hospital stay without increasing patient deaths. Both restriction and enabling techniques were successful in achieving effectiveness of the intervention. We do not need more studies to answer the question of whether these interventions reduce unnecessary antibiotic use, but we do need more research to understand the unintended consequences of the use of restrictive interventions.

Interventions were successful in safely reducing unnecessary antibiotic use in hospitals, despite the fact that the majority did not use a widely adopted behaviour change technique, which is to audit and provide feedback on performance. Effective communication of the review results could have considerable health service and policy impact.

How up‐to‐date is the review?

We searched for studies published up to January 2015.

Summary of findings

Summary of findings for the main comparison. Effects of interventions to improve use of antibiotics on prescribing, clinical outcomes, adverse events, and effect modifiers (heterogeneity).

Patient or population: adults or children undergoing inpatient antibiotic prophylaxis or treatment Settings: mainly high‐income countries (North America or Western Europe) Intervention: any intervention targeting healthcare professionals that aimed to improve antibiotic prescribing to hospital inpatients Comparison: usual care (varied across studies)
Effectiveness: prescribing outcomes from RCTs
Outcomes	Absolute effect*		No of participants (No of studies)	Certainty of the evidence (GRADE)	Comments
	Without intervention	With intervention
Proportion of participants who were treated according to antibiotic prescribing guidelines Follow‐up to end of study	43 per 100	58 per 100	23,394 participants (29 RCTs)	⊕⊕⊕⊕ High	We have graded the certainty of evidence as high because heterogeneity was explained by prespecified effect modifiers (see below). The intervention effect varied between the studies, but the direction of effect was consistent. Restricting the analysis to studies at low risk of bias gave a similar result (RD 11%, 95% CI 10% to 12%).
	Difference: 15 more participants per 100 (95% CI 15 to 23) received appropriate treatment following intervention.
Duration of all antibiotic treatment	11.0 days	9.1 days	3318 participants (14 RCTs)	⊕⊕⊕⊕ High
	Difference: 1.95 fewer days per participant (95% CI 2.22 to 1.67)
Mortality Follow‐up to end of study	11 per 100	11 per 100	15,827 participants 28 (RCTs)	⊕⊕⊕⊝1 Moderate	Mortality and length of stay were measured to determine the impact of reduced antibiotic use on clinical outcomes. The results were similar for studies that targeted antibiotic choice or exposure. Only 1 of the interventions in the RCTs with mortality or length‐of‐stay outcomes had a restrictive component (Singh 2000). This evidence is therefore at high risk of indirectness because 7 studies in the next section of the table (see below) raise concerns about the safety of restrictive interventions. Moreover, the ITS studies showed that restrictive components were included in 42 (34%) of 123 hospital interventions.
	Difference: 0 more deaths per 100 participants (95% CI 1 to 0 fewer)
Mean length of hospital stay per participant	12.9 days	11.8 days	3834 participants 15 (RCTs)	⊕⊕⊕⊝1 Moderate
	Difference: 1.1 fewer days per participant (95% CI 1.5 to 0.7 fewer)
Delay in treatment	Restrictive interventions increased the risk of delay in all 3 studies. The risk to patients resulted in termination of the RCT by the Trial Monitoring Committee.		1 RCT, 2 cohort	⊕⊕⊝⊝2 Low	The evidence from these 7 studies of unintended consequences raises concerns about the directness of the evidence of safety from the 29 RCTs in the previous section of the table (see above).
Negative professional culture	Loss of trust in infection specialists because of failure to record approvals for restricted drugs or provide warning about stopping treatment Misleading or inaccurate information from prescribers in order to meet criteria for restricted drugs. In 1 hospital, misdiagnosis of hospital‐acquired infection was large enough to trigger an outbreak investigation.		1 case control, 2 cohort, 1 qualitative	⊕⊕⊖⊖3 Low
Effect modifiers (heterogeneity) for immediate effect of intervention on prescribing outcomes: impact of behaviour change functions (enablementor restriction) and additional impact of feedback, RCTs and ITS studies. A positive value for Beta means the modifier is associated with increased effect
Effect modifier	Adjusted effect in meta‐regression Beta (95% CI)		Number of studies	Certainty of the evidence (GRADE)	Comments
Enablement	15.12 (8.45 to 21.8)		29 RCTs	⊕⊕⊕⊕ High	The effect of enablement and restriction is similar in the RCTs and ITS studies. Of the 29 RCTs, only 8 (31%) of interventions were hospital‐wide, the majority being in single units. In contrast, 64 (70%) of the interventions in ITS studies were hospital‐wide.
	12.86 (4.11 to 21.6)		91 ITS
Restriction	34.91 (13.52 to 56.29)		29 RCTs	⊕⊕⊕⊕ High
	24.69 (13.74 to 35.64)		91 ITS
Addition of feedback to enablement	10.88 (7.16 to 19.32)		23 RCTs	⊕⊕⊕⊝2 Moderate	Feedback was included in 4 (17%) of 23 RCTs and 20 (47%) of 43 ITS studies with interventions that included enablement. There were not enough interventions with goal setting and action planning to analyse as effect modifiers.
	15.63 (0.56 to 30.70)		43 ITS
Addition of enablement to restriction	38.36 (18.94 to 57.78)		29 ITS	⊕⊕⊖⊖3 Low	Enablement was included in 13 (45%) of 29 ITS studies with restrictive interventions.
*The risk WITHOUT the intervention is based on the median control group risk across studies. The corresponding risk WITH the intervention (and the 95% confidence interval for the difference) is based on the overall relative effect (and its 95% confidence interval). CI: confidence interval; ITS: interrupted time series; RCT: randomised controlled trial; RD: risk difference
GRADE Working Group grades of evidence High certainty: Further research is very unlikely to change our confidence in the estimate of effect. Moderate certainty: Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate. Low certainty: Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate. Very low certainty: We are very uncertain about the estimate.

Details of five GRADE criteria for all outcomes from RCTs are in Appendix 2.

¹We downgraded the evidence to moderate because of indirectness.
 ²We downgraded the evidence because most studies are non‐randomised studies.
 ³We graded the evidence as low because it is all from non‐randomised studies.
 ⁴We graded the evidence as very low because it is all from non‐randomised studies and there was too much heterogeneity for reliable evidence synthesis.

Background

Description of the condition

Antibiotic resistance is a major public health problem. In comparison with infections caused by susceptible bacteria, those caused by multidrug‐resistant bacteria are associated with higher incidences of mortality and prolonged hospital stay (de Kraker 2011). Clostridium difficile infection (CDI) is another manifestation of the collateral damage caused by antimicrobial prescribing (Davey 2010). Such infections are also associated with increased costs resulting from the need to use more expensive antibiotics, prolonged hospital stay (the principal contributor), and expenses related to screening and surveillance, eradication regimens, and consumables (the gloves, gowns, and aprons used to prevent cross‐infection) (de Kraker 2011). The UK 5 Year Antimicrobial Resistance Strategy 2013 to 2018 recognises the importance of reducing inappropriate antibiotic prescribing (Department of Health 2013), the implication being that antibiotic resistance is largely a consequence of the selective pressures of antibiotic usage, and that reducing these pressures by the judicious administration of antibiotics will facilitate a return of susceptible bacteria or, at least, will prevent or slow the pace of the emergence of resistant strains.

At the same time, sepsis is a major cause of avoidable mortality in hospitals, with an estimated 100,000 cases per year in the UK alone (NCEPOD 2015).

Description of the intervention

We included any intervention to improve antibiotic prescribing to hospital inpatients. Antibiotic stewardship has two aims: first, to ensure effective treatment of patients with infection, and second, to minimise collateral damage from antimicrobial use (Davey 2010). Hence the UK Department of Health's Guidance on Antimicrobial Stewardship emphasises the need for urgent treatment of serious infections in addition to minimising unnecessary use of antibiotics (Department of Health 2013). We compared interventions to change professional behaviour with standard practice (no intervention). We classified interventions by their intervention function (Michie 2011). The previous version of this review suggested that restrictive interventions had greater immediate effect on prescribing than interventions that used education or persuasion (Davey 2013). For this update, we identified interventions that were designed to increase enablement, defined as 'increasing means/reducing barriers to increase capability or opportunity' (Michie 2011).

How the intervention might work

In this update of the review we used new data extraction sheets to classify the intervention functions and to identify the behaviour change functions that are used in antimicrobial stewardship interventions (Michie 2013). In particular, we assessed the relative effectiveness of interventions according to how they used enablement and restriction to change behaviour (Michie 2011). We divided the interventions into four groups: enablement without restriction; restriction without enablement; both enablement and restriction; and neither enablement nor restriction.

Why it is important to do this review

This review is an update of Davey 2005 and Davey 2013. It complements a review of interventions to improve prescribing of antibiotics to patients in ambulatory care (Arnold 2005).

Objectives

To estimate the effectiveness and safety of interventions to improve antibiotic prescribing to hospital inpatients and investigate the effect of two intervention functions: restriction and enablement.

Methods

Criteria for considering studies for this review

Types of studies

We included randomised controlled trials (RCTs) and non‐randomised studies (NRS). We included three NRS study designs to measure behavioural and clinical outcomes and analyse variation in the effects: non‐randomised trials (NRT), controlled before‐after (CBA) studies and interrupted time series (ITS) studies. We used Cochrane Effective Practice and Organisation of Care (EPOC) Group eligibility guidance for CBAs and NRTs (EPOC 2016). In addition, for the assessment of unintended consequences, we included three additional NRS designs (case control, cohort, and qualitative studies) to identify additional evidence about long‐term effects and harms of interventions in order to enhance the directness of evidence from RCTs (Schünemann 2013).

Types of participants

Healthcare professionals who prescribe antibiotics to hospital inpatients receiving acute care (including elective inpatient surgery). We excluded interventions targeted at residents in nursing homes or other long‐term healthcare settings.

Types of interventions

We included interventions relevant to improving antibiotic prescribing as outlined in the EPOC taxonomy (EPOC 2015).

1. Audit and feedback defined as any summary of clinical performance of health care over a specified period of time.

2. Education through meetings or distribution of educational materials.

3. Educational outreach through academic detailing or review of individual patients with recommendation for change.

4. Reminders provided verbally, on paper, in the workplace environment (e.g. posters or messages printed on equipment) or on computer.

5. Structural: the influence on antibiotic prescribing of changing from paper to computerised records and of the introduction of new technology for rapid microbiology testing or measurement of inflammatory markers.

In addition, we included the following restrictive interventions: selective reporting of laboratory susceptibilities; formulary restriction; requiring prior authorisation (expert approval) therapeutic substitution; and automatic stop orders.

Enabling interventions were: audit and feedback; educational outreach through review of individual patients with recommendation for change; and circumstantial reminders that were targeted at doctors who were managing specific patients (Table 2). We classified reminders in the form of posters or pocket cards summarising antibiotic policies as environmental restructuring but not as enabling (Table 2). Terms used to describe interventions are described in more detail in the Data extraction and management section.

1. Definition of behaviour change techniques and intervention functions.

Intervention Function	Definition	Intervention components
Education	Increasing knowledge or understanding	Educational meetings; Dissemination of educational materials; Educational outreach
Persuasion	Using communication to induce positive or negative feelings or to stimulate action	Educational outreach by academic detailing or review and recommend change
Restriction	Using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours)	Restrictive
Environmental restructuring	Changing the physical context	Reminders (physical) such as posters, pocket‐size or credit card‐size summaries or on laboratory test reports; Structural (e.g. new laboratory tests or rapid reporting of results)
Enablement	Increasing means/reducing barriers to increase capability or opportunity	Audit and feedback; Decision support through computerised systems or through circumstantial reminders that were triggered by actions or events related to the targeted behaviour; Educational outreach by review and recommend change

We did not consider studies that compared the effectiveness of antibiotic treatments (e.g. intravenous versus oral administration of antibiotics) as eligible for this review.

Types of outcome measures

Primary outcomes

The effect of interventions on antibiotic prescribing measured as either compliance with antibiotic guidelines or policies, the duration of antibiotic treatment, decision to treat, or total duration of treatment. We included studies without reliable or adequate information addressing the primary outcome measure, but we did not use these studies in data synthesis.

Secondary outcomes

Mortality, length of stay, or other clinical outcomes (e.g. surgical‐site infection or acute kidney injury), microbial outcomes (CDI, colonisation or infection with antimicrobial‐resistant bacteria), unintended‐consequences measures (e.g. a delay in start of antibiotic treatment, a change in threshold for diagnosis of hospital‐acquired infection to justify existing prescribing practice). Note that clinical outcomes could be indicators of improved clinical outcomes associated with interventions to increase effective antibiotic treatment, or unintended consequences (e.g. to provide evidence about the safety of interventions to reduce unnecessary antibiotic treatment).

Search methods for identification of studies

Electronic searches

We searched the Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE) for related systematic reviews and the following databases for primary studies without language, publication year, or publication status restrictions in January 2015.

Databases

• Cochrane Central Register of Controlled Trials (CENTRAL; 2015, Issue 1) in the Cochrane Library (searched 22 January 2015)

• MEDLINE (1946 to 19 January 2015) (OvidSP)

• Embase (1947 to 22 January 2015) (OvidSP)

The MEDLINE search strategy was developed by the Cochrane EPOC Group Information Specialist in consultation with the review authors and translated for use in other databases employing appropriate syntax and vocabulary. Results were limited by two methodological filters: the Cochrane Highly Sensitive Search Strategy (sensitivity‐ and precision‐maximising version, 2008 revision) to identify randomised trials (Higgins 2011), and a Cochrane EPOC Group study design filter to identify NRS. Full search strategies are provided in Appendix 1.

Searching other resources

We searched for additional studies using the bibliographies of included articles, personal files, and by contacting experts in the field regarding any unpublished work.

Data collection and analysis

Selection of studies

Two review authors (EB and PD) independently reviewed citations and abstracts retrieved in the search to identify all reports that included original data about interventions to change antibiotic prescribing. If either review author had doubts about eligibility, then both review authors reviewed the full papers. The review authors were not blinded to study author or location. We resolved disagreements by discussion and consensus.

We excluded studies that had no relevant and interpretable data presented or obtainable. We defined 'relevant data' as an intervention that included a change in antibiotic treatment for hospital inpatients and where at least one of the study's reported outcomes was directly attributable to change in antibiotic treatment. We defined 'interpretable data' as follows: CBA, NRT, or RCT designs had to include sufficient data to estimate effect size as change in at least one relevant outcome after the intervention. Interrupted time series studies had to include a clearly defined intervention point.

We did not exclude studies due to high risk of bias.

Data extraction and management

Working in pairs, five review authors (PD, CM, CS, EC, KM) independently performed data abstraction using data extraction sheets including information on: study design, type of intervention (intervention components and functions), presence of controls, type of targeted behaviour, participants, setting, methods (unit of allocation, unit of analysis, study power, methodological risk of bias, consumer involvement), outcomes, and results.

Explanation of terms used to describe interventions

Restriction

We defined restriction as 'using rules to reduce the opportunity to engage in the target behaviour (or increase the target behaviour by reducing the opportunity to engage in competing behaviours)'.

Enablement

We defined enablement as 'increasing means/reducing barriers to increase capability or opportunity'.

Goal setting

We documented the specific prescribing behaviour that was targeted by the intervention (e.g. switch participants from parenteral to oral antibiotics) and how this was incorporated into an aim for the intervention. Was the aim simply a directional change of the target behaviour (e.g. increase or decrease behaviour?), or did the intervention include a specific threshold to be reached (e.g. target behaviour performed more than 95% of the time) or the duration within which the target had to be achieved (e.g. more than 95% reliability within six months)? If the study reported a power calculation, we did not accept this as evidence of a specific threshold unless it was clearly communicated to the professionals who were the targets of the intervention. For example, a power calculation showing that the study could detect a 10% improvement in the targeted behaviour would have to be accompanied by some explicit statement about the intervention aim being at least 10% improvement.

Feedback

We classified interventions as including feedback only if they provided a "summary of clinical performance of healthcare over a specified period of time" (EPOC 2015). We found that some studies did not meet this definition, even though they described their intervention as including feedback in the title (e.g. Elligsen 2012 and Newland 2012) or in the methods (e.g. Palmay 2014). The intervention in these studies was educational outreach by review and recommended change, so the feedback was limited to the individual participants who were reviewed with no feedback about the treatment of other participants over time. In contrast, Buising 2008a is an example of an intervention in which "a formal feedback was provided to units regarding their compliance with the approval system over time" in addition to review and recommend change for individual participants. For studies that met our definition of feedback, we recorded frequency, format (verbal, written, or both) and whether it was delivered by a colleague, supervisor, or somebody external to the clinical team.

Action planning

We documented whether there was a reward for meeting a target, which could be material or social reward (either from self or others) and the use of action plans if the target was not met. Our definition of an action plan was: prompt, detailed planning of performance of the behaviour, which had to include at least one of context, frequency, duration, or intensity. If there was evidence of action planning, we recorded to whom the action plan was tailored (e.g. individual participant or group) and whether participants were involved in developing the action plan.

Intervention components and functions

In the Characteristics of included studies we have listed the intervention components (Types of interventions) and the intervention functions (Michie 2011; Michie 2013). Note that each intervention component may have more than one intervention function. We have presented definitions of intervention functions and their relationship to intervention components in Table 2.

Assessment of the impact of interventions

We have used meta‐analysis to assess the impact of RCTs of interventions and meta‐regression to understand variation in effect estimates for RCTs and ITS studies.

Assessment of risk of bias in included studies

We applied the 2013 EPOC 'Risk of bias' criteria to all papers in the review, including articles in the 2003 review (EPOC 2013). We scored each study for risk of bias as 'low' if all criteria were scored as 'low', 'medium' if one or two criteria were scored as 'unclear' or 'high', and 'high' if more than two criteria were scored as 'unclear' or 'high'.

We applied three additional criteria to studies with microbial outcomes, based on the ORION statement: Guidelines for transparent reporting of outbreak reports and intervention studies of nosocomial infection (Orion Statement; Stone 2007).

1. Case definition: score as 'low' if there is a clear definition either of infection or of colonisation and there were no major changes in laboratory diagnostic methods during the study period.

2. Planned intervention: score as 'low' if the intervention was planned to reduce endemic rates of colonisation or infection and was not implemented in response to an outbreak. Regression to the mean following an outbreak is an important risk of bias for estimates of the effect of interventions in ITS studies of infection (Davey‐Smith 2001; Stone 2007).

3. Other infection control measures: score as 'low' if infection control practices (hand hygiene, gowning, or other personal protection) and isolation or cohorting policies are described and there were no changes coincident with the intervention to change antibiotic prescribing.

We have presented microbial 'Risk of bias' results in the Notes section of the Characteristics of included studies. We have not included them in the 'Risk of bias' tables unless there might also be a risk to prescribing outcomes (e.g. appointment of additional infection control practitioners who might have influenced prescribing).

We assessed risk of bias in case control or cohort studies of unintended consequences with ROBINS‐I: a tool for assessing Risk of Bias in Non‐randomised Studies of Interventions (Sterne 2016). We have reported these 'Risk of bias' assessments in the Notes section of the Characteristics of included studies.

Measures of treatment effect

We assessed the impact of interventions on clinical outcome for studies that provided reliable data about mortality, length of hospital stay, or other clinical outcomes such as acute kidney injury. We did not include clinical outcomes for studies that estimated the impact of their intervention based on modelling (Barlow 2007). We analysed dichotomous data (such as increase in desired practice and mortality) as risk differences and analysed continuous data (such as length of hospital stay) as mean differences.

We critically examined the methods of analysis of ITS data. The preferred method is a statistical comparison of time trends before and after the intervention. If the original paper did not include an analysis of this type, we extracted the data presented in tables or graphs in the original paper and used them to perform new analyses where possible. We used segmented time series regression analysis to estimate the effect of the intervention whilst taking account of time trend and autocorrelation among the observations. We obtained estimates for regression coefficients corresponding to two standardised effect sizes for each study: a change in level and a change in trend before and after the intervention. A change in level was defined as the difference between the observed level at the first intervention time point and that predicted by the pre‐intervention time trend. A change in trend was defined as the difference between post‐ and pre‐intervention slopes (Ramsay 2003). We evaluated the direct effect of the intervention using results reported one month after the start of the intervention. We also reported the level effects at six months, and yearly thereafter when possible. We standardised the results of some ITS studies so that they were on the same scale (per cent change in outcome), thereby facilitating comparisons of different interventions. To do this, we used the change in level and change in slope to estimate the effect size with increasing time after the intervention (one month, six months, one year, etc.) as the per cent change in level at each time point. We did not extrapolate beyond the end of data collection after the intervention. We anticipated that the eligible studies would exhibit significant heterogeneity, due to variations in target clinical behaviours, patient and provider populations, methodological features, characteristics of the interventions, and the contexts in which the interventions were delivered. To address the source of variation in results due to the use of enabling or restrictive interventions, we undertook a random‐effects meta‐regression analysis on study‐level summary effect size at each time point.

We assessed the impact of interventions on microbial outcomes if the study provided reliable data about colonisation or infection with Clostridium difficile or with antibiotic‐resistant bacteria. We did not include microbial outcomes for studies that estimated the future impact of their intervention based on modelling (Paul 2006), or that used clinical definitions of infection that did not distinguish between resistant and sensitive bacteria (Micek 2004; Singh 2000).

Unit of analysis issues

If an RCT did not take into account the effect of clustering in the analysis, we stated this in the 'Risk of bias' assessment. We incorporated consideration of unit of analysis issues as part of the sensitivity analyses.

We estimated intracluster correlation (ICC) for each outcome. The ICCs used reflect that process measures usually have higher ICC than outcome measures and were obtained from the database of ICCs held by the Health Services Research Unit, University of Aberdeen (Health Services Research Unit 2016).

• Prescribing 0.2

• Mortality 0.01

• Length of stay 0.2

Average cluster size (m) = (total number of participants (intervention + control)) ≑ (total number of clusters). Inflation factor = 1 + (m‐1) x ICC. For dichotomous outcomes, we divided events and participants by the inflation factor for intervention and control groups. For continuous outcomes, we multiplied intervention and control standard deviation by the inflation factor.

Dealing with missing data

We have not attempted to account for missing data in the meta‐analysis of RCTs or meta‐regression of ITS studies. For ITS studies, we only analysed effects at a specified time point when data were available, we have not carried forward regression lines beyond the last observation or used regression lines to estimate missing data..

Assessment of heterogeneity

We quantified heterogeneity among studies using the I² statistic and Cochran's Q test (Cochran 1954). The I² statistic quantifies the percentage of the total variation across studies that is due to heterogeneity rather than chance (Higgins 2003); smaller percentages suggest less observed heterogeneity.

Assessment of reporting biases

We assessed publication and selective reporting bias.

Data synthesis

We have analysed the results for RCTs, CBAs, NRT, and ITS studies separately. For the RCT data, we employed a standard meta‐analysis approach using Review Manager 5 for binary (e.g. compliance with guidelines) and continuous (e.g. duration of treatment) outcomes. We analysed the data with a fixed‐effect model (Review Manager 5).

We used Stata 14 for all statistical re‐analyses and meta‐regressions (Stata 2015), and Review Manager 5 for all data synthesis (Review Manager 5).

Subgroup analysis and investigation of heterogeneity

We used meta‐regression to investigate potential effect modifiers. In meta‐regression, the outcome variable is the effect estimate (e.g. a mean difference or a risk difference). The explanatory variables are characteristics of studies that might influence the size of intervention effect (Higgins 2011).

We prespecified four subgroups as explanatory variables for the meta‐regression (Davey 2014):

1. interventions that included enablement versus those that did not;

2. interventions that included restriction versus those that did not;

3. enabling interventions that included feedback versus those that did not;

4. feedback interventions that included goal setting or action planning versus those that did not.

Definitions of these terms can be found in Data extraction and management and Table 2. We expected restriction, enablement, feedback goal setting and action planning to be associated with increased effectiveness of interventions (Ivers 2012).

We included the following three additional variables in the meta‐regression because they might influence the size of intervention effect and explain heterogeneity.

1. Target: choice of antibiotic regimen versus time to first antibiotic dose or exposure to antibiotics, effects possibly greater for interventions targeting choice.

2. Setting: single unit versus multiple wards, effects possibly greater in single unit.

3. Intent: increase effective versus decrease excessive, effects possibly greater with increase effective.

The meta‐regression was performed using standard weighted (by standard error of estimate) linear regression (Higgins 2011).

Sensitivity analysis

We conducted sensitivity analyses by re‐analysing data to investigate the effect of two risks of bias.

1. Lack of adjustment for the effect of clustering in cluster RCTs. We repeated all analyses that included cluster RCTs with adjusted numbers of events and total participants for dichotomous variables and adjusted standard deviation for continuous variables (Analysis 1.2; Analysis 1.5; Analysis 2.2; Analysis 2.5).

2. Overall high risk of bias. We analysed all studies at medium and low risk of bias separately in sensitivity analyses (Analysis 1.3; Analysis 1.6; Analysis 2.3; Analysis 2.6).

1.2. Analysis.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor.

1.5. Analysis.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor.

2.2. Analysis.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor.

2.5. Analysis.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor.

1.3. Analysis.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Dichotomous outcomes, low or medium 'Risk of bias' studies only.

1.6. Analysis.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Continuous outcomes, low or medium 'Risk of bias' studies only.

2.3. Analysis.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 3 Mortality, low or medium 'Risk of bias' RCTs.

2.6. Analysis.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 6 Length of stay, low or medium 'Risk of bias' RCTs only.

Summary of findings

We summarised the findings of the main intervention comparison for the most important outcomes in Table 1. Two review authors independently assessed the certainty of the evidence for each key outcome (high, moderate, low, and very low) using the five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness, and publication bias) (Guyatt 2011). We assessed the following outcomes:

1. compliance with desired practice;

2. duration of antibiotic treatment;

3. mortality;

4. length of hospital stay;

5. delay in treatment;

6. negative professional culture.

We also assessed the evidence from the meta‐regression in terms of the extent to which we believed it helped explain variation of effect. We included the following effect modifiers in our analysis.

1. Enablement (Yes/No)

2. Restriction (Yes/No)

3. Addition of feedback to enablement (Yes/No)

4. Addition of enablement to restriction (Yes/No)

We used the methods and recommendations described in Section 8.5 and Chapter 12 of the Cochrane Handbook for Systematic Reviews of Interventions,Higgins 2011, and the EPOC worksheets (EPOC 2013a). Disagreements on certainty ratings were resolved by discussion, and justification for decisions to down‐ or upgrade the ratings are provided in footnotes in the table and comments made to aid readers' understanding of the review where necessary. We used plain language statements to report these findings in the review. Further details about each of the five GRADE criteria are in Appendix 2.

Evidence from randomised studies started at high certainty and was downgraded according to the five considerations described above. Evidence from non‐randomised studies started at low certainty and was assessed against the same five criteria. We only considered upgrading for non‐randomised evidence in the presence of a large treatment effect, dose response, or where plausible confounding would have reduced the observed effect.

Results

Description of studies

Results of the search

The combined results of all literature searches are described in the study flow diagram (Figure 1).

1.

Figure 1 Study flow diagram.

Included studies

The Characteristics of included studies table lists 221 studies, of which 211 used the following designs to evaluate the intended effect of interventions: 138 ITS studies, 58 RCTs (14 cluster RCTs), 6 CBAs, and 8 NRTs. The remaining 11 studies were designed to identify unintended consequences of interventions and used the following designs: 8 cohort (Connor 2007; Duvoisin 2014; Friedberg 2009; Kanwar 2007; LaRosa 2007; Linkin 2007; Welker 2008; Winters 2010), 1 case control (Calfee 2003), and 1 qualitative (semi‐structured interviews) (Baysari 2013) and 1 ITS (Bell 2014).

Geographical location of study

Ninety‐six studies were from North America. The remaining 125 were from Europe (87, includes Israel), Asia (19), South America (8), Australia (8), and East Asia (3). The number of studies by country (including the countries in four multinational studies) is: Argentina, 1; Australia, 9; Austria, 2; Belgium, 4; Brazil, 4; Canada, 8; China, 6; Colombia, 2; Croatia, 1; Denmark, 3; France, 11; Germany, 12; Greece, 1; Hong Kong, 1; Hungary, 1; India, 1; Indonesia, 1; Israel, 1; Italy, 3; Japan, 1; Korea, 3; Lebanon, 1; Mexico, 1; Netherlands, 11; Norway, 1; Serbia, 1; Singapore, 1; Spain, 5; Sweden, 2; Switzerland, 11; Taiwan, 3; Thailand, 4; Turkey, 1; UK, 22; USA, 89.

Number of hospitals

A total of 178 (79%) studies were conducted in one hospital, 9 studies in 2 hospitals, 18 studies in 3 to 9 hospitals, and 16 studies in 10 or more hospitals.

Deliverer of intervention

Of the 221 interventions, 112 (51%) were designed and delivered by a multidisciplinary team, 54 (24%) by specialist physicians (infectious diseases or microbiology), 35 (16%) by department physicians (e.g. emergency department or critical care), and 20 (9%) by pharmacists.

Funding

Five studies received some funding from manufacturers of drugs or laboratory tests. The remaining 216 studies were funded by government agencies or the participating hospitals. Details are provided in the Characteristics of included studies table.

Power calculations

Details of power calculations are provided in Appendix 3

Excluded studies

We excluded 32 unique studies from the review because they did not contain relevant or interpretable data (Selection of studies). For details of each study, see Characteristics of excluded studies.

Risk of bias in included studies

All 14 CBAs and NRTs were at high risk of bias (Figure 2). High risk of bias was more common in RCTs (36/58, 62%) than in ITS studies (20/138, 14%) (Figure 2). All 51 studies at low risk of bias were ITS studies (Figure 2). Among RCTs, high risk of bias was much more likely in studies with two or fewer hospitals (31/36, 86%) versus three or more hospitals (11/22, 50%). Of the 11 RCTs with two or fewer hospitals with medium risk of bias, nine interventions were circumstantial reminders targeted at doctors who were managing specific patients (Christ‐Crain 2004; Christ‐Crain 2006; Esposito 2011; Kerremans 2009; Lacroix 2014; Lesprit 2013; Long 2014; Senn 2004; Stocker 2010; Strom 2010), so the risks of allocation or contamination bias were relatively low compared with the other RCTs of interventions in one or two hospitals. However, the remaining two RCTs at low risk of bias show that these risks can be minimised for RCTs of review and recommend change interventions in single hospitals (Lesprit 2013; Palmay 2014).

2.

Risk of bias graph: review authors' judgements about each risk of bias item presented as percentages across all included studies.

Blank sections in this graph are due to use of different ROB criteria for CBA, NRT and RCT versus ITS studies

We have presented 'Risk of bias' criteria for the case control and cohort studies of unintended consequences in the Notes section in Characteristics of included studies.. For the nine studies, we assessed the risk of bias as high in two (Calfee 2003; Friedberg 2009), medium in two (Linkin 2007; Welker 2008), and low in five (Connor 2007; Duvoisin 2014; Kanwar 2007; LaRosa 2007; Winters 2010).

Allocation

Most of the RCTs had high risk of selection bias because of problems with concealment of allocation (Figure 2). The RCTs with low risk of selection bias were either cluster RCTs or interventions with circumstantial reminders, for which concealment of allocation is relatively straightforward.

Blinding

Most of the RCTs also had high risk of performance and detection bias because RCTs in single hospitals were often single‐blind and it was difficult to conceal the allocation of participants in these trials (Figure 2).

Incomplete outcome data

The RCTs used data collected specifically for the trial, and all provided convincing evidence about lack of attrition bias. Most of the ITS studies used data from routine systems for prescribing (pharmacy) and microbial (microbiology) outcomes; we assessed these sources as having low risk of attrition bias (Figure 2). Examples of high risk of attrition bias in routine data are changes in the number of participants who did not have serum creatinine measure preoperatively during the study period, which may have biased ascertainment of postoperative kidney injury (Bell 2014), and use of surveillance data about surgical‐site infection that did not include information about infections arising after discharge from hospital (Dua 2014).

Selective reporting

We also assessed routine data systems as being at low risk of reporting bias (Figure 2). Most of the ITS studies used computerised pharmacy systems to measure drug consumption.

Other potential sources of bias

Less than 25% of RCTs provided clear information about baseline outcome; most of these were cluster RCTs (Figure 2). The most common single risk of bias for ITS studies was that the intervention was not independent of other changes (Figure 2). For ITS studies, the main risks of bias were that there were insufficient data to account for seasonal variation or that one or more of the microbial 'Risk of bias' criteria were present (Figure 2).

Effects of interventions

See: Table 1

Studies included in evidence synthesis and 'Summary of findings' tables

Outcomes from 49 (84%) of the 58 RCTs and110 (80%) of the 138 ITS studies were used in at least one meta‐analysis or meta‐regression or are summarised in text or Additional tables. The contribution that each RCT made can be found in Appendix 4. One ITS study contributed data about unintended consequences (Bell 2014). The contribution of 109 ITS studies to meta‐regression of prescribing outcomes is summarised in Appendix 5. Reasons for exclusion of 10 RCTs and 28 ITS studies from evidence synthesis can be found in Appendix 6.

The 10 case control, cohort, or qualitative studies of unintended consequences all contributed evidence about adverse effects.

None of the 6 CBAs or 8 NRTs included evidence about adverse effects of interventions, and there were not enough studies for evidence synthesis.

Intended prescribing outcomes for RCTs and ITS studies included in evidence synthesis

Interventions were targeted at antibiotic treatment for 46 (94%) of 49 RCTs and 101 (92%) of 110 ITS studies. The remaining 11 studies targeted surgical antibiotic prophylaxis (Bell 2014; Dull 2008; Gulmezoglu 2007; Kritchevsky 2008; Meyer 2010; Perez 2003; Schwann 2011; Sun 2011; Van Kasteren 2005; Wax 2007; Weinberg 2001).

For the 148 interventions targeted at antibiotic treatment, the intended outcome of 137 (93%) interventions was to decrease excessive use of antibiotics: 45/46 (98%) RCTs and 93/102 (91%) ITS studies. The only RCT that was primarily intended to increase effective treatment targeted dosing of gentamicin (Burton 1991). Two RCTs with antibiotic choice as the primary outcome did include time to first antibiotic dose for participants with community‐acquired pneumonia as a secondary outcome (Schouten 2007; Yealy 2005). The only other evidence about increasing effective treatment of sepsis came from six ITS studies that aimed to reduce time to first antibiotic dose (Barlow 2007; Hitti 2012; Jobson 2015; Marwick 2013; Volpe 2012; Weiner 2009).

In contrast, reduction in excessive use of antibiotics was the intended outcome of only 3 (25%) of the 12 interventions targeted at surgical antibiotic prophylaxis (Bell 2014; Sun 2011; Van Kasteren 2005). The remaining nine interventions were all intended to increase effective use of antibiotics by increasing the number of participants who received prophylaxis or reducing the time to first antibiotic dose.

Effectiveness and adverse effects of interventions

Effectiveness of interventions in RCTs

Interventions were associated with an increase in compliance with desired practice by 15% (95% confidence interval (CI) 14% to 16%) in 29 RCTs (Analysis 1.1; Figure 3). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 1.2) or risk of bias (Analysis 1.3). Interventions were associated with a reduction in duration of total antibiotic treatment by ‐1.95 days (95% CI ‐2.22 to ‐1.67) in 14 RCTs (Analysis 1.4; Figure 4). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 1.5) or risk of bias (Analysis 1.6).

1.1. Analysis.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Dichotomous outcomes, increase in desired practice.

3.

Forest plot of comparison: 1 Prescribing: RCTs of all interventions to reduce unnecessary prescribing, outcome: 1.1 Dichotomous outcomes, increase in desired practice.

1.4. Analysis.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Continuous outcomes, duration of all antibiotic treatment (days).

4.

Forest plot of comparison: 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 1.4 Continuous outcomes, duration of all antibiotic treatment (days).

In four RCTs the prescribing outcome was the consumption of targeted antibiotics measured in different units (cost, days, or defined daily dose), so results were expressed as standardised mean reduction (Analysis 1.7.).

1.7. Analysis.

Comparison 1 Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD).

Adverse effects of interventions

Evidence from RCTs

Interventions were not associated with any increase in mortality (95% CI 1 to 0 fewer deaths per 100 participants) in 28 RCTs (Analysis 2.1; Figure 5). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 2.2) or risk of bias (Analysis 2.3). Interventions were associated with reduction in length of stay by ‐1.12 days (95% CI ‐1.54 to ‐0.70) in 15 RCTs Analysis 2.4; Figure 6). We obtained similar results in sensitivity analyses for unit of analysis errors (Analysis 2.5) or risk of bias (Analysis 2.6). We found no evidence of a difference in results for interventions that targeted antibiotic exposure (decision to treat or duration of all antibiotic treatment) versus the choice of antibiotic prescribed (Analysis 3.1; Analysis 3.2; Analysis 4.1; Analysis 4.2).

2.1. Analysis.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 1 Mortality, all RCTs.

5.

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.1 Mortality, all RCTs.

2.4. Analysis.

Comparison 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, Outcome 4 Length of stay, all RCTs.

6.

Forest plot of comparison: 2 Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use, outcome: 2.4 Length of stay, all RCTs.

3.1. Analysis.

Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 1 Mortality for trial patients.

3.2. Analysis.

Comparison 3 Adverse effects: Clinical outcomes of interventions targeting antibiotic choice, Outcome 2 Length of stay for trial patients.

4.1. Analysis.

Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 1 Mortality for trial patients.

4.2. Analysis.

Comparison 4 Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure, Outcome 2 Length of stay for trial patients.

One RCT measured clinical outcome as potentially harmful delay in essential treatment (Strom 2010). The outcome was ascertained by the Trial Monitoring Committee, who stopped the trial prematurely when four participants were found to have potentially harmful delay in treatment with trimethoprim‐sulphamethoxazole or warfarin. This was a restrictive intervention intended to prevent interactions between these drugs.

Evidence from NRS

ITS studies

Clinical outcome data were measured as mortality in four ITS studies (Table 3) and length of stay in one ITS study (Table 4). However, we could only calculate 95% CI for three of these studies (Lee 2014; Popovski 2015; Skaer 1993), and the outcome data came from all participants in the hospital rather than just the participants who were the targets of the interventions.

2. Unintended consequences of ITS studies: mortality*.

Study	Prescribing target	Restriction	Design of analysis	Effect estimate	95% CI
Lee 2014	Choice of drug	No	Cohort	Incidence rate ratio 1.1	0.9 to 1.5
Popovski 2015	Choice of drug	No	Cohort	Increase by 1.4%	‐1.2% to 4.1%
Wang 2014	Choice of drug	Yes	ITS, segmented regression	Change in slope ‐0.0172	No data
Yoon 2014	Choice of drug	Yes	Cohort	+0.43 per 1000 OBD	No data

*Mortality was measured in all patients in the hospital rather than just those patients who were the targets of the interventions.

CI: confidence interval
 ITS: interrupted time series
 OBD: occupied bed day

3. Unintended consequences of ITS studies: length of stay*.

Study	Prescribing target	Restrictive	Design of analysis	Effect estimate	95% CI
Mittal 2014	Exposure, % treated	No	Cohort	‐0.5 days	No data
Skaer 1993	Choice of drug	No	Cohort	‐0.1 days	‐0.49 to +0.29

*Length of stay was measured in all patients in the hospital rather than just those patients who were the targets of the interventions.

CI: confidence interval
 ITS: interrupted time series

Three ITS studies reported other clinical outcomes that provided more direct evidence about unintended consequences of the interventions (Table 5). An intervention to promote gentamicin for prophylaxis was intended to reduce risk of CDI but was associated with a large increase in acute kidney injury in the participants undergoing target operations, and as a consequence the antibiotic policy change was reversed (Bell 2014). An intervention designed to shorten time to first antibiotic dose for people with sepsis was not associated with any increase in the time left without being seen for all other participants in the emergency department (Volpe 2012). An intervention to reduce the duration of surgical antibiotic prophylaxis was not associated with increased surgical‐site infection (Van Kasteren 2005).

4. Unintended consequences of ITS studies: other.

Study	Prescribing target	Design of analysis	Effect measure	Effect estimate	95% CI
Bell 2014	Antibiotic choice	ITS, segmented regression	Risk of postoperative acute kidney injury	Increase 98%	93.8% to 94.2%
Van Kasteren 2005	Exposure, duration	Cohort	Surgical‐site infection	Decrease 0.8%	‐2.2% to 0.6%
Volpe 2012	Time to first antibiotic dose	Cohort	Left without being seen rate	Decrease 0.4%	No data

CI: confidence interval
 ITS: interrupted time series

Case control, cohort and qualitative studies

Ten studies investigated unintended consequences of interventions to change antibiotic choice with cohort (n = 8), case control (n = 1), or qualitative case study (n = 1) designs (Table 6).

5. Unintended consequences studies (case control, cohort, or qualitative).

Study	Design	Patients	Intended target	Unintended consequence	Effect estimate	95% CI
Interventions with a restrictive component
Baysari 2013	Qualitative	36 physicians	Reduce unnecessary use of restricted antibiotics	Inaccurate feedback	Not quantified; qualitative study
Calfee 2003	Case control	Not clear		Increase in physician‐based diagnosis of nosocomial infection	No denominator data
Connor 2007	Cohort	120		Failure to warn prescribers about discontinuation	—	—
Duvoisin 2014	Cohort	222	Reduce unnecessary laboratory tests	Delay in TFAD (HR > 1 shows delay less likely in intervention period)	Multivariate HR 1.56	1.17 to 2.07
LaRosa 2007	Cross‐sectional	15,440	Reduce unnecessary use of restricted antibiotics	Orders for restricted antibiotics (% all orders) from 10 to 11 pm vs all other hours	—	—
	Cohort	360		% appropriate orders 10 to 11 pm vs 9 to 10 pm	‐23.7%	‐31.8% to ‐15.5%
Linkin 2007	Cohort	200		Risk of inaccurate information in orders judged inappropriate vs appropriate	OR 2.2	1.0 to 4.4
Winters 2010	Cohort	3251		Risk of 1‐hour delay in TFAD	OR 1.5	1.2 to 1.8
				Risk of 2‐hour delay in TFAD	OR 1.8	1.4 to 2.2
Interventions with no restrictive component
Friedberg 2009	Cohort	13,042	Reduce time to first antibiotic dose for patients with community‐acquired pneumonia	% CAP diagnoses	1% increase	No denominator data
Kanwar 2007	Cohort	518		% correct CAP diagnoses	‐7.9% decrease	‐15.4% to ‐0.4%
Welker 2008	Cohort	548		% correct CAP diagnoses	‐16.0% decrease	‐7.6% to ‐24.4%

CAP: community‐acquired pneumonia
 CI: confidence interval
 HR: hazard ratio
 OR: odds ratio
 TFAD: time to the first antibiotic dose

There was a restrictive component to the intervention in seven studies. One study showed that restriction of laboratory tests of inflammation (C‐reactive protein and white blood cell count) was not associated with an increase in time to first antibiotic dose (Duvoisin 2014). The remaining six studies all revealed unintended consequences of interventions that restricted antibiotic choice by requiring prior approval, as follows.

• Negative professional culture through breakdown in trust and communication (Baysari 2013; Calfee 2003; Connor 2007; Linkin 2007).

• Delay in time to first antibiotic dose (LaRosa 2007; Winters 2010). Evidence of delay in essential treatment was also seen in one RCT (Strom 2010).

In three studies (Friedberg 2009; Kanwar 2007; Welker 2008), the intervention was a national financial incentive in the USA that was intended to reduce time to first antibiotic dose for people admitted to hospital with community‐acquired pneumonia (CAP). In all three studies, the unintended consequence was misdiagnosis of pneumonia, which could lead to an increase in unnecessary antibiotic treatment. In two single‐centre studies, there was a decrease in the percentage of participants with correct diagnosis of CAP based on prespecified criteria (Kanwar 2007; Welker 2008). In contrast, a large, multicentre study reported no evidence of an overall increase in the diagnosis of CAP (Friedberg 2009); however, this study was at high risk of bias.

Explaining heterogeneity in the intended effect of interventions

Meta‐regresson of RCTs

We performed meta‐regression on 29 RCTs with dichotomous prescribing outcomes (Analysis 1.1; Figure 3). Outcomes for all of these trials could be expressed as number of participants where treatment was compliant with policy divided by total participants. We did not perform meta‐regression on 15 RCTs with continuous prescribing outcomes because the outcomes were heterogeneous (Analysis 1.4; Analysis 1.7) and because none of the interventions included restriction or feedback, and only two did not include enablement (Danaher 2009; Kerremans 2008).

Meta‐regression results for 29 RCTs with dichotomous outcomes

In the meta‐regression, enablement, restriction, targeting antibiotic choice versus exposure and high risk of bias were significantly associated with greater intervention effect in univariate analysis, and they all remained significant in multivariate analysis (Figure 7).

7.

Meta‐regression by effect modifier for 29 RCTs. A positive value for Beta indicates enhanced intervention effect. One RCT had both enabling and restrictive components in the intervention (Strom 2010).

Of the 23 RCTs of enabling interventions, four also included feedback (Camins 2009; Schnoor 2010; Schouten 2007; Yealy 2005). All four of these RCTs targeted antibiotic choice, so we have compared their effects with seven RCTs of enabling interventions without feedback that also targeted antibiotic choice. The mean risk difference for interventions with feedback was 19% (95% CI 16% to 22%) (Figure 8) compared with 13% (95% CI 9% to 17%) (Figure 9) for interventions with no feedback. Only two of the feedback RCTs also included action planning (Schouten 2007; Yealy 2005).

8.

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.1 Enablement plus feedback.

9.

Forest plot of comparison 5: RCTs of enablement with and without feedback, outcome: 5.2 Enablement without feedback.

Meta‐regression of ITS studies

Do interventions that involve enablement have greater initial effect?

There were 107 ITS studies with data that could be used for meta‐regression of prescribing outcomes at one, six, or 12 months' postintervention. We used multivariable meta‐regression to identify effect modifiers in 91 ITS studies including data about prescribing at six months' postintervention. As with the RCTs (Figure 7), both enablement and restriction were independently associated with increased effect in ITS studies (Figure 10). Of 29 ITS studies with restrictive interventions, 13 (45%) also had enablement, and this independently enhanced intervention effect (Figure 11). In comparison with interventions targeting antibiotic exposure, those targeting choice were associated with greater effect in RCTs (Figure 7), but not in ITS studies (Figure 10). The number of studies in each category only allowed analysis of the effects of setting in ITS studies (Figure 10), and intention could only be included in meta‐regression of ITS studies of enabling intervention (Figure 12). The limited evidence suggests that intention and setting were not effect modifiers (Figure 7; Figure 10).

10.

Meta‐regression by effect modifiers of intervention for 91 ITS studies. Outcome is effect on prescribing six months' postintervention. There are 16 studies with both enabling and restricting intervention components (Figure 11).

11.

Meta‐regression of prescribing outcome by effect modifiers for 29 ITS studies of interventions that included restriction.

12.

Meta‐regression by effect modifier for 43 ITS studies of interventions that included enablement but not restriction. Outcome is effect on prescribing six months' postintervention. Note that four studies with feedback were not included in this analysis because they also included restriction.

Are interventions that include feedback more effective than those that do not?

Feedback was included in 4 (17%) of 23 RCTs (Figure 8) and 20 (47%) of 43 ITS studies (Figure 12) of enabling interventions that did not include restriction. The intervention was audit and feedback alone in three RCTs and 10 ITS studies. In one RCT and 11 ITS studies, audit and feedback was combined with review and recommend change or circumstantial reminders. Interventions that included feedback were more effective than those that did not. However, there were too few studies with goal setting or action planning to assess their effect in addition to feedback.

There were only two ITS studies with enough data to analyse the effect of adding an additional component to an effective intervention. However, the second intervention component did not include goal setting, feedback, or action planning in either study (Mol 2005; Po 2012)

Summary of interventions for the studies included in meta‐regression

In comparison with RCTs, the ITS studies were more likely to have multiple intervention components: 35 (38%) of 91 ITS studies versus 5 (17%) of 29 RCTs, odds ratio 3.00 (95% CI 1.05 to 8.59) (Table 7). There were also differences in the components for enabling interventions (review and recommend change was included in 53% of ITS studies versus 25% of RCTs) and restrictive interventions (removal of target drugs from clinical areas was included in 34% of ITS studies but in no RCTs) (Table 7). Educational meetings or distribution of educational materials was the most common intervention in studies that did include enablement or restriction (75% of RCTs and 89% of ITS studies) (Table 7).

6. Summary of intervention components for 29 RCTs (Analysis 1.1; Figures 3 and 7) and 91 ITS studies (Figure 10).

Intervention function and components	RCT	ITS
Enablement	24 studies	59 studies
Number of enabling or restrictive intervention components	27	76
Studies with > 1 Enabling intervention component	2 8%*	19 32%*
Audit and feedback	4 17%	24 41%
Computerised decision support	1 4%	3 5%
Circumstantial reminders	16 67%	18 31%
Review and recommend change	6 25%	31 53%
Restriction	2 studies	29 studies
Number of Restrictive intervention components	3	41
Studies with > 1 Restrictive intervention component	1 50%	10 34%
Expert approval	1 50%	18 62%
Compulsory order form	1 50%	7 24%
Removal	0	10 34%
Review and make change	1 50%	6 21%
No Enablement or Restriction	4 studies	18 studies
Number of intervention components	6	25
Studies with > 1 intervention component	2 50%	6 33%
Educational materials or meetings	3 75%	16 89%
Educational outreach (academic detailing)	1 25%	6 33%
Physical reminders	1 25%	2 11%
Structural intervention	1 25%	1 6%

*The denominator for all percentages is the number of studies for each intervention function. One RCT, Strom 2010, and 16 ITS studies (Figure 11) included both enabling and restrictive intervention components.

ITS: interrupted time series
 RCT: randomised controlled trial

Sustainability of intervention effect

Sustainability was assessed in 64 of 91 ITS studies, with prescribing outcome data at both 6 and 12 months' postintervention. Intervention effect was sustained at 12 months' postintervention in 55 (86%) of these studies (95% CI 77% to 94%). There were 13 interventions with neither enablement nor restriction; intervention effect was sustained in 11 (85%) (95% CI 65% to 100%). Consequently, it was unlikely that either enablement or restriction would be associated with greater sustainability. However, the results suggest that restrictive interventions were less likely to have sustained effect if they did not include enablement: 5/8 (62%) versus 12/13 (92%) with enablement, risk difference 30% (95% CI ‐7% to 66%).

Five ITS studies with data about removal of interventions provided additional information about sustainability of interventions (Table 8). Three of these studies also provided data about the effect of the intervention. The intended effect of all interventions was decrease in the use of target antibiotics. Removal of the intervention was associated with increase in the use of target antibiotics in all five studies and, with one exception (Kim 2008), the 95% CI for effect size did not include decrease in use of target antibiotics. Kim 2008 was the only one of these five interventions including enablement by audit and feedback.

7. Data from 5 studies about the effect of removal of interventions. The intended effect of all interventions was reduction in unnecessary antibiotic use.

Study	Intervention function	Intervention effect (95% CI)	Time intervention was in place	Effect of removal (95% CI)
Kallen 2009	Restriction	‐87.5% ‐115.4 to ‐59.7	6 months	398.9% 238.2 to 559.5
Kim 2008	Restriction	‐23.1% ‐53.7 to +7.4	9 months	6.0% ‐23.4 to 35.4
Standiford 2012	Enablement	‐28.6% ‐46.5 to ‐10.6	7 years	31.0% 6.8 to 55.3
Himmelberg 1991	Restriction	No data	“long‐standing”	301.2% 230.9 to 371.5
Skrlin 2011	Restriction		2 years	255.8% 194.7 to 316.9

CI: confidence interval

Microbial outcomes (antibiotic resistance and CDI)

There were 1 CBA and 5 RCTs with microbial outcome data, and these were too heterogeneous for data synthesis (Table 9).

8. Randomised controlled trials with microbial outcomes.

Study	Design	Microbial outcome	Reason not in meta‐analysis
Annane 2013	RCT	Colonisation with MRSA (nasal swab) and GNRB (rectal swabs)	Not comparable with any other RCT
Bouza 2007	RCT	Number of cases of Clostridium difficile	Not in prescribing meta‐analysis
Lesprit 2013	RCT	Secondary infection and/or colonisation with multidrug‐resistant bacteria in the 6 months following randomisation	Not in prescribing meta‐analysis. It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics.
Palmay 2014	RCT	CDI and infection with antibiotic resistant organisms cases/1000 OBD	Not in prescribing meta‐analysis
Singh 2000	RCT	Number of participants with "antimicrobial resistance and/or superinfections" from randomisation until discharge from hospital	It is impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics.

CDI: Clostridium difficile infection
 GNRB: gram‐negative resistant bacteria
 MRSA: methicillin‐resistant Staphylococcus aureus 
 OBD: occupied bed day
 RCT: randomised controlled trial

We performed meta‐regression on 26 ITS studies including reliable data about prescribing outcomes at 6 months and microbial outcomes at 12 months after the intervention (Table 10). Six unplanned interventions (in response to outbreaks) were associated with markedly greater effect on microbial outcomes (Figure 13). When studies were ranked in descending order of effect size for microbial outcome at 12 months, the top five studies were all unplanned interventions (Kim 2008; May 2000; McNulty 1997; Tangdén 2011; Valiquette 2007), with the remaining unplanned intervention ranking 9th (Lautenbach 2003).

9. Microbial outcomes from 26 ITS studies from the prescribing meta‐analysis that include reliable data about prescribing outcomes at 6 months and microbial outcomes at 12 months postintervention.

Prescribing target	Microbial outcome	N	Study ID
Cephalosporins	GNRB	8	Grohs 2014; Kim 2008; Knudsen 2014; Lee 2007; McNulty 1997; Meyer 2009; Petrikkos 2007; Tangdén 2011
	MRSA	1	May 2000
Carbapenems	GNRB	1	Goldstein 2009
Fluoroquinolones	GNRB	3	Cook 2011b; Lafaurie 2012; Willemsen 2010
	MRSA	1	Lafaurie 2012
High‐risk antibiotics	CDI	6	Aldeyab 2012; Chan 2011; Dancer 2013; Fowler 2007; Talpaert 2011; Valiquette 2007
	GNRB	4	Buising 2008a; Chan 2011; Dancer 2013; Liebowitz 2008
	MRSA	6	Aldeyab 2014; Ananda‐Rajah 2010; Chan 2011; Dancer 2013; Fowler 2007; Liebowitz 2008
Total antibiotic use	CDI	2	Cook 2011a; Jump 2012
	MRSA	1	Cook 2011a
Vancomycin	VRE	1	Lautenbach 2003
	Total microbial	34*

*Some studies had more than one microbial outcome, so the total is 34 microbial outcomes from 26 studies.

CDI: Clostridium difficile infection
 GNRB: gram‐negative resistant bacteria
 ITS: interrupted time series
 MRSA: methicillin‐resistant Staphylococcus aureus 
 VRE: vancomycin‐resistant enterococci

13.

Meta‐regression by effect modifiers for 34 microbial outcomes 12 months' postintervention from 26 ITS studies. The bars show the results for unadjusted versus adjusted analyses, the comparison for unplanned interventions is with planned interventions in both the unadjusted and adjusted analysis.

CDI: Clostridium difficile infection
 GPC: infection with antibiotic‐resistant gram‐positive cocci
 GNB: infection with antibiotic‐resistant gram‐negative bacteria

Other infection control: 'Yes' means there were changes to infection control processes during the study period.

In the 20 studies of planned intervention, there were six studies with unclear information about other infection control interventions or changes during the study period (Chan 2011; Grohs 2014; Jump 2012; Liebowitz 2008; Meyer 2009; Petrikkos 2007). We performed meta‐regression on the remaining 14 studies from Table 10 (Figure 14). In contrast with the meta‐regression of all 27 studies (Figure 13), the effects of setting, other infection control interventions, and microbial outcome type were all reversed so that each of these variables was associated with increase in effect size in the 14 studies with planned interventions and details of other infection control interventions (Figure 14).

14.

Meta‐regression by effect modifiers for 20 microbial outcomes 12 months' postintervention from 14 ITS studies of planned interventions that provided details about other infection control changes or interventions.

CDI: Clostridium difficile infection
 GPC: infection with antibiotic‐resistant gram‐positive cocci
 GNB: infection with antibiotic‐resistant gram‐negative bacteria

Other infection control: 'Yes' means there were changes to infection control processes during the study period.

The antibiotic targets for the 20 studies of planned interventions were single antibiotic classes in nine studies (Cook 2011b; Grohs 2014; Knudsen 2014; Lafaurie 2012; Lee 2007; Meyer 2009; Petrikkos 2007; Willemsen 2010; Yoon 2014), high‐risk antibiotics in nine studies (Aldeyab 2012; Aldeyab 2014; Ananda‐Rajah 2010; Buising 2008a; Chan 2011; Dancer 2013; Fowler 2007; Liebowitz 2008; Talpaert 2011), and all antibiotics in the remaining two studies (Cook 2011a; Jump 2012). High‐risk antibiotics were a combination of drugs from more than one class of antibiotic, which were all considered to be high risk for the microbial outcome. The prescribing outcome data reported in these nine studies varied from just one of the high‐risk antibiotics, in Dancer 2013, through individual results for all of the high‐risk antibiotics, in Buising 2008a, Chan 2011, Fowler 2007, and Talpaert 2011, to combined results for all of the high‐risk antibiotics (Aldeyab 2012; Aldeyab 2014; Ananda‐Rajah 2010; Liebowitz 2008).

One study can be used to demonstrate the technical challenges of estimation of intervention effect on microbial outcomes (Dancer 2013). The intervention was addition of complete restriction of ceftriaxone and ciprofloxacin to a pre‐existing multifaceted intervention introduced seven months before restriction and remaining in place throughout the restrictive period (Dancer 2013). We could not analyse the effect of the initial multifaceted intervention because there were no pre‐intervention data about prescribing or microbial outcomes. However, the available data showed CDI was lower by ‐0.143 cases per 1000 occupied bed days per month in the nine months prior to the addition of the restrictive intervention. At the start of the restrictive intervention, CDI rates were already low (1.5 cases per 1000 occupied bed days). After the introduction of restriction, CDI rates continued to decline for five months, and then stabilised at around 0.5 cases per 1000 occupied bed days. These data suggest that the restrictive intervention had no additional effect on the rate of CDI. However, the segmented regression analysis estimated that there was a relative increase of 35.8% in CDI rate 12 months after the restrictive intervention with very wide confidence intervals (from 81.0% decrease to 152.7% increase).

Our review did include one multicentre controlled ITS study comparing CDI rates in six hospitals with antimicrobial stewardship programmes versus four control hospitals (Ostrowsky 2014). We did not include this study in evidence synthesis because neither the interventions nor the prescribing outcomes were standardised across the six hospitals with stewardship programmes. Baseline rates of CDI were only 0.8 cases per 1000 occupied bed days in the intervention and control hospitals before the intervention, and the authors did not report a decrease in aggregate CDI rates either between intervention and non‐intervention groups or within the intervention groups over time (Ostrowsky 2014).

We have not attempted to synthesise microbial outcome data because of the small number of studies, the heterogeneity of intervention targets and prescribing outcomes, and the wide confidence intervals for estimated relative effect. We have focused on the 20 ITS studies of planned interventions and separated the results by microbial outcome type. Interventions were associated with consistent reduction in CDI (median ‐48.6%, interquartile range ‐80.7% to ‐19.2%) but inconsistent effect on resistant gram‐negative bacteria (median ‐12.9%, interquartile range ‐35.3% to 25.2%) and resistant gram‐positive bacteria (median ‐19.3%, interquartile range ‐50.1% to 23.1%). There were too few studies with too much variance in microbial outcomes to reliably assess the relationship between change in antibiotic use and each of the microbial outcomes.

Discussion

Summary of main results

The RCTs provide high‐certainty evidence that interventions are effective in increasing compliance with antibiotic policies and in reducing duration of antibiotic treatment safely, without an increase in mortality. Furthermore, interventions were associated with a reduction in length of stay. The mechanism is not clear, and further investigation is required. However, reducing length of stay is a key organisational objective for most hospitals, so this evidence should be used to prioritise antimicrobial stewardship in hospitals.

Analysis of effect modifiers in RCTs and ITS studies consistently supported the theory that involving enablement increases intervention effect, including those with restrictive components. However, feedback was only used in a minority of enablement interventions, and very few included goal setting or action planning.

Overall completeness and applicability of evidence

The RCTs show that interventions increase compliance with policies or guidelines by 15%, which is a clinically important effect size. However, the result is less impressive when one considers that health professionals’ adherence to prescribing recommendations increased from 43% to 58%, because 58% compliance is probably still far too low. Three studies did achieve 90% compliance with guidelines by making this an explicit goal for the intervention and using action planning to revise interventions until the goal was achieved (Jobson 2015; Volpe 2012; Weinberg 2001).

The ITS studies provided important additional evidence that the results of RCTs regarding effectiveness of interventions can be reproduced in routine practice: 70% of ITS studies reported on hospital‐wide interventions compared with only 31% of RCTs.

Only two ITS studies included data that enabled assessment of the effect of adding an intervention component to an existing intervention (Mol 2005; Po 2012). This is a strong study design that should be more widely used to evaluate these types of interventions.

Safety and unintended consequences of interventions

The main limitation of the RCT evidence regarding safety of reducing unnecessary use was that only two interventions included restriction, and one was stopped early because of delay in the start of treatment (Strom 2010). Two NRS also raised concerns about delay in time to first antibiotic dose associated with restrictive interventions (LaRosa 2007; Winters 2010). Furthermore, four NRS described negative effects of restrictive interventions on professional culture through breakdown in trust and communication (Baysari 2013; Calfee 2003; Connor 2007; Linkin 2007). These NRS used either case control, cohort, or qualitative designs because they required collection of data that were not available in routine clinical systems (Table 6).

The ITS studies provided very little evidence about the safety of interventions because they rely on routine clinical systems for outcome measures, which are currently largely incapable of providing information about specific patients, for example those with infection. Moreover, the range of clinical measures should be extended beyond infection outcomes to include safety indicators such as acute kidney injury (AKI). (Bell 2014). Scotland’s Infection Intelligence Platform was established to improve linkage and availability of routine data (ISD 2016), but research is required to improve timeliness, quality, and relevance of clinical outcome measures and to provide a richer understanding of the unintended consequences of improvement interventions (SISCC 2016). We found only one example of a qualitative study of unintended consequences (Baysari 2013). This is an important study design for investigation of unanticipated consequences of interventions and should be more widely used (Rogers 1995).

Studying the effect of removal of an intervention can be used to provide additional evidence about the outcomes of the original intervention (Walker 2016). This study was from the same group that reported that an intervention that was intended to reduce risk of CDI in people undergoing orthopaedic surgery was associated with an increased risk of postoperative AKI (Bell 2014) (Table 5). The increase in AKI was attributed to change in antibiotic surgical prophylaxis policy from cefuroxime to flucloxacillin and gentamicin. This second study showed reduction in postoperative AKI associated with a change away from flucloxacillin and gentamicin, which provides persuasive additional evidence that gentamicin was responsible for the original increase in postoperative AKI (Walker 2016).

Interventions were consistently associated with reduced length of stay (Analysis 2.4), and the results were similar when analysis was restricted to RCTs at low or medium risk of bias (Analysis 2.6). Measurement of length of stay was intended to provide reassurance about safety of the intervention so that reduction in length of stay is an example of an unanticipated beneficial outcome (Ash 2007; Rogers 1995). We found similar results for interventions that targeted antibiotic choice (Analysis 3.2) or antibiotic exposure (Analysis 4.2). One possible mechanism for reduction in length of stay is that interventions reduced the duration of intravenous antibiotic therapy (Carratala 2012). However, further research is required.

Microbial outcomes

Interventions were consistently associated with reduction in CDI, but less consistently associated with reduction in infection by resistant bacteria. However, intervention effects on microbial outcomes could only be analysed reliably in planned interventions (Figure 13), and our meta‐analysis was limited by four technical challenges.

1. Each study had considerable variance because of the small number of microbial events in each time point.

2. Studies rarely had stable pre‐intervention data, so that extrapolation of the pre‐intervention trend throughout the postintervention phase was probably unreliable.

3. We analysed a single prescribing outcome for each study (even if more were reported). The criteria for selection of the prescribing outcome were determined by the analysis plan for the effect of interventions on prescribing behaviour. However, these criteria may not have been correct for analysis of the relationship between changes in prescribing and microbial outcomes.

4. We could only analyse the relationship between prescribing and microbial outcomes at fixed time points. We chose six and 12 months, respectively, imposing a six‐month time lag for all interventions. However, the time lag will likely vary by prescribing and microbial outcomes, and by intervention context (Vernaz 2008).

Quality of the evidence

We found high‐certainty evidence that interventions increase appropriate use of antibiotics, reduce duration of antibiotic treatment, and shorten hospital stay without increasing the risk of mortality. There was low‐certainty evidence that these interventions can delay treatment and create a negative professional culture (Table 1). High risk of bias was associated with greater intervention effect in RCTs (Figure 7) for the outcome of compliance with desired practice. However, we have presented separate analysis of effects for RCTs at low or medium risk of bias (Analysis 1.3; Analysis 1.6; Analysis 2.3; Analysis 2.6). These analyses provide evidence supporting our decision not to downgrade for risk of bias, since excluding studies at high risk of bias did not substantively change the direction of effect. We did not downgrade for inconsistency since the direction of effect across the studies was consistent, and our meta‐regression provides some explanation for the high levels of statistical heterogeneity between the results of the studies. The certainty of evidence about adverse effects was more variable, with particular concerns about the unintended consequences of restrictive interventions, namely delays in treatment and negative professional culture, for which we have low‐certainty evidence.

The quality of reporting of interventions was poor, which makes it difficult for professionals and clinical teams to reliably implement interventions that have been shown to be useful and for other researchers to replicate or build on research findings (Hoffmann 2014). We found high‐certainty evidence that enablement and restriction both enhanced the effectiveness of interventions. However, we found only moderate‐certainty evidence for the effectiveness of feedback, and there were too few studies with action planning and goal setting to provide any reliable information about the combined effects of these behaviour change techniques.

In the analysis of risk of bias equal weight is given to all criteria (Figure 2). Our results for microbial outcomes clearly showed that the risk of bias from unplanned interventions is much greater than the risk from other infection control interventions (Figure 13; Figure 14).

We found that some NRS study designs provided important additional evidence about intervention effects and sustainability in routine clinical practice (ITS studies) and about unintended consequences (case control, cohort, and qualitative studies). However, we found no useful evidence from CBAs or NRTs and suggest that these study designs should not be included in updates to this review.

Heterogeneity of intervention effect

We found that two intervention functions, enablement and restriction, explained some of the variation in targeted prescribing behaviour. However, we found little evidence that behaviour change theory had been used to design interventions (Charani 2011). There were too few interventions with explicit goals or action planning to include these variables in meta‐regression.

There was no consistent evidence that intervention setting or target explained variation in the effect of interventions (Figure 7; Figure 10)

Potential biases in the review process

Our decision not to use adjusted data for cluster RCTs for the primary analysis could be contested. The consequences of using unadjusted data would be to assign too much weight to cluster studies in the analysis, potentially biasing the effect from our analyses to their results (Higgins 2011). We believe that taking clustering into account is unlikely to impact on the strength of the results in such a way as to change the conclusions of the review. Our sensitivity analyses provide some indirect support for the approach we have undertaken. In comparison to unadjusted results, analyses based on the effective sample sizes calculated from assumed ICCs consistently gave a larger average intervention effect (Analysis 1.1 versus Analysis 1.2; Analysis 2.1 versus Analysis 2.2; Analysis 2.4 versus Analysis 2.5).The increased effect size could be explained by the lower weight assigned to the cluster studies, which tended to have smaller effects than the individually randomised studies.

The electronic literature search did not identify 42 (19%) of the 221 included studies, highlighting some of the challenges in constructing sensitive search terms for reviews of behavioural interventions and the identification of non‐randomised studies. It is possible that additional eligible studies have not been retrieved by the search process we undertook for this review.

We did not find evidence of publication bias in the RCTs, however publication bias is more likely in the ITS studies because the decision to publish may have been made after the analysis of intervention effect.

Agreements and disagreements with other studies or reviews

Agreements

Ivers 2012 included and analysed 140 RCTs that compared any intervention in which audit and feedback was a core, essential component to usual care and evaluated effects on professional practice. The review concluded that interventions were more effective if they also included goal setting and action planning. We were unable to reproduce their analysis because only four of our RCTs included feedback (Figure 8). Although 20 ITS studies included feedback (Figure 12), there were not enough studies with goal setting or action planning for reliable analysis.

Our findings are similar to a previous review that found that behavioural determinants and social norms were not given due consideration in the design and evaluation of interventions to change antibiotic prescribing (Charani 2011).

Sustainability of intervention effect

We found evidence that removal of restriction, in Himmelberg 1991, Kallen 2009, Kim 2008, and Skrlin 2011, or of review and recommend change (enablement, Standiford 2012) was associated with reversal of intervention effect (Table 8). Three previous studies have shown that removal of financial incentives is associated with reversal of intervention effects in primary care (Avery 2012; Dreischulte 2016; Lester 2010). This is an important issue because the attractiveness of interventions will be reduced if improvement resources cannot be moved on to new priorities. Restriction is a relatively low‐cost intervention, but it is worrying that an enabling intervention (review and recommend change) apparently had no sustained effect on clinical teams after being in place for seven years (Standiford 2012). Review and recommend change is a time‐intensive process that was included in 36 (54%) of 67 of the enabling interventions in ITS studies.

Disagreements

A systematic review on current evidence about antimicrobial stewardship objectives reported that "guideline‐adherent empirical therapy was associated with a reduction for mortality (odds ratio 0.65, 95% CI 0.54‐0.80)" (Schuts 2016). Only two of the 39 studies in this review reported an intervention: one was invalid because it was an uncontrolled before‐after study (Garcia 2007), and the other was a CBA (Dean 2006). The remaining 37 studies used case control study or cohort designs to compare the outcomes of participants with and without guideline‐adherent antibiotic treatment, and did not include an intervention to change professional practice. The results of this review are in marked contrast to our analysis of mortality in 11 RCTs targeting antibiotic choice (Analysis 3.1). The most likely explanation for the discrepancy between our results and Schuts 2016 is confounding by indication. It is likely that participants with less complex or severe illness were more likely to receive guideline‐adherent antibiotic treatment and that there was residual confounding after adjustment for available clinical information.

A systematic review on the effect of antibiotic stewardship programmes on CDI reported that interventions were associated with a consistent, significant protective effect (pooled risk ratio for CDI 0.48, 95% CI 0.38 to 0.62) (Feazel 2014). Of the 16 studies included in this systematic review, four were ITS studies that were also included in our review (Elligsen 2012; Fowler 2007; Price 2010; Talpaert 2011), and the remaining 12 studies were either uncontrolled before‐after or inadequate ITS studies. The statistical analysis in this review was not appropriate (Feazel 2014). Calculation of risk ratios for the post‐ versus pre‐intervention periods is an uncontrolled before‐after analysis, which does not provide a reliable estimate of intervention effect.

Additional details about the disagreements with Feazel 2014 and Schuts 2016 can be found in Appendix 7.

Limitations

There are five weaknesses in the current evidence.

1. Evidence of intended effects is unbalanced towards reducing unnecessary treatment (compliance with guidelines that are intended to reduce use of broad‐spectrum antibiotics or shorten duration of treatment). More evidence is required about finessing effective treatment of sepsis without also causing excessive use of antibiotics.

2. The limited evidence regarding adverse effects of restrictive interventions suggests that they can be associated with delay in essential treatment. There is a need for better patient safety outcome measures that can be used in studies of interventions in clinical practice.

3. The majority of the interventions do not use effective behaviour change techniques such as action planning or feedback.

4. Given the critical role of junior doctors in antimicrobial stewardship in hospitals, it is surprising that there is only a single example of an intervention that involved junior doctors in self monitoring and reflection on feedback about their prescribing (Price 2010).

5. Analysis of the impact of interventions on microbial outcomes requires large, multihospital RCTs.

Authors' conclusions

Implications for practice.

Reducing antimicrobial resistance and hospital‐associated infection is a public health priority. Our review shows that antimicrobial stewardship interventions can safely reduce unnecessary antibiotic use in hospitals, despite the fact that the majority of interventions did not use the most effective behaviour change techniques. Consequently, effective dissemination of the review results could have considerable health service and policy impact through greater use of interventions that enhance enablement.

The randomised controlled trials provided high‐certainty evidence that the interventions we have assessed are effective in increasing compliance with guidelines to reduce unnecessary treatment without increasing the risk of mortality. Furthermore, the interventions were associated with reduction in length of stay. The evidence from this review should inform implementation decisions regarding antimicrobial stewardship interventions in hospitals.

In randomised controlled trials and interrupted time series studies, enablement consistently increased the effectiveness of interventions, including restrictive interventions; however, feedback was used in only a minority of enablement interventions, and very few included goal setting or action planning. Antimicrobial management teams might consider using evidence about effective feedback from other clinical settings (Ivers 2012). Training in the design and reporting of behaviour change interventions should be a priority for antimicrobial management teams.

Implications for research.

Given the high certainty of evidence for our primary outcome, we believe that additional trials comparing antibiotic stewardship with no intervention are unlikely to change our conclusions or build on our understanding of the current evidence. Future research should instead focus on measuring clinical outcomes and assessing other measures of patient safety and different stewardship interventions and explore the barriers and facilitators to implementation.

We included 163 NRS but only 11 of these were about unintended consequences. Moreover only one NRS used qualitative methods, which are likely to be required in addition to survey methods for the investigation of unanticipated consequences (Rogers 1995). Future research should make greater use of qualitative methods for investigation of consequences of interventions, for example in process evaluation alongside clinical trials (Grant 2013). Anticipated, undesirable consequences should be regarded as trade‐offs which may need to be accepted in exchange for a greater good (Ash 2007). Future research should examine how decisions are made about the acceptability of trade‐offs (SISCC 2016). The purpose, design, and use of balancing measures in quality and safety improvement has been identified as a priority for research on methods in improvement science (SISCC 2016). Antimicrobial stewardship is an important topic for further research because of the clear competing risks of excessive use of antibiotics and delayed or ineffective treatment of life threatening infection.

Antibiotic stewardship requires clinicians to change their infection control behaviours. Given that the extent to which current antibiotic stewardship programs have incorporated insights and approaches from behavioural science is limited, there is an urgent need to bring together key stakeholders in the design and delivery of stewardship programmes and research experts in improvement and social sciences to develop more impactful stewardship programmes. We propose three key questions, which a Transnational Working Group within the Joint Programming Initiative in Antimicrobial Resistance will address in 2017 (JPIAMR 2016):

1. What behaviour change approaches can be recommended now to optimise hospital stewardship programmes?

2. How can hospital stewardship programmes be designed to maximise implementation across countries?

3. What is the research agenda to optimise efficient implementation of antibiotic stewardship programmes worldwide?

We were unable to perform reliable evidence synthesis on the relationship between prescribing and microbial outcomes with segmented regression analysis of interrupted time series studies from single hospitals. There is an urgent need for co‐ordinated, multicentre research studies.

We found consistent evidence of reduced length of stay as an unanticipated beneficial consequence of interventions that targeted either choice of antibiotic or duration of antibiotic treatment. Further research is required to understand the mechanism for this effect.

What's new

Date	Event	Description
20 February 2017	Amended	The review has been amendment with a correction to the technical settings of the primary analysis.

History

Protocol first published: Issue 1, 2002
 Review first published: Issue 3, 2005

Date	Event	Description
19 January 2015	New citation required and conclusions have changed	The addition of new data to the review has strengthened the conclusions regarding the effect on antibiotic prescribing and mortality. The review shows that there is a reduction in length of hospital stay. The review now has identified that interventions are consistently more effective if they contain enabling components, which provide advice or feedback to help physicians make more informed decisions about their prescribing. However only 10% of interventions used the most effective enabling techniques: goal setting, feedback and action planning. Given the high certainty of evidence for our primary outcome we believe that additional trials comparing antibiotic stewardship with no intervention are unlikely to change our conclusions or build on our understanding of the current evidence. This review includes 221 studies.
19 January 2015	New search has been performed	New searches performed to January 2015 and 132 new studies have been included in the review. New authors: Charis Marwick, Kirsty McNeil, Claire Scott, replacing Lynda Fenelon, Alison Holmes, Phil Wiffen, and Mark Wilcox. Important changes to the methods are inclusion of case control, cohort, or qualitative studies of unintended consequences, new data extraction forms to identify behaviour change techniques in the interventions, and a prespecified subgroup analysis and meta‐regression by behaviour change technique. Cluster non‐randomised controlled trials and randomised controlled trials (RCTs) with fewer than 2 intervention or control sites have been excluded, including 1 non‐randomised controlled trial from the previous version of the review. Results updated, 'Characteristics of included studies' and 'Characteristics of excluded studies' tables re‐written and updated to end of December 2014. Meta‐analysis of RCTs completed prior to meta‐regression of RCTs and interrupted time series studies.
22 November 2014	Amended	Major edits in preparation for next update, 'Characteristics of included studies' table re‐written and updated to end of December 2012.
1 May 2014	Amended	Protocol completely revised.
26 February 2013	New search has been performed	New search, 89 studies found.
26 February 2013	New citation required and conclusions have changed	New search, 89 new studies found.
12 February 2009	Amended	Minor edits, tables modified.
29 July 2008	Amended	Converted to new review format.
28 July 2005	New citation required and conclusions have changed	Substantive amendment

Appendices

Appendix 1. Search strategies

MEDLINE <1946 to Present> and MEDLINE In‐Process & Other Non‐Indexed Citations (Searched 19 January 2015) (OvidSP)

1 (hospital$ and antibiotic?).ti.

2 ((antibiotic? or alamethicin? or amdinocillin? or amdinocillin pivoxil? or amikacin? or amoxicillin? or amoxicillin‐potassium clavulanate combination? or amphotericin? or ampicillin? or anisomycin? or antimycin? or aurodox? or azithromycin? or azlocillin? or aztreonam? or bacitracin? or bacteriocin? or bambermycin? or bongkrekic acid? or brefeldin? or butirosin sulfate? or calcimycin? or candicidin? or capreomycin? or carbenicillin? or carfecillin? or cefaclor? or cefadroxil? or cefamandole? or cefatrizine? or cefazolin? or cefixime? or cefmenoxime? or cefmetazole? or cefonicid? or cefoperazone? or cefotaxime? or cefotetan? or cefotiam? or cefoxitin? or cefsulodin? or ceftazidime? or ceftizoxime? or ceftriaxone? or cefuroxime? or cephacetrile? or cephalexin? or cephaloglycin? or cephaloridine? or cephalosporin? or cephalothin? or cephamycin? or cephapirin? or cephradine? or chloramphenicol? or chlortetracycline? or citrinin? or clarithromycin? or clavulanic acid? or clavulanic acid? or clindamycin? or cloxacillin? or colistin? or cyclacillin? or dactinomycin? or daptomycin? or demeclocycline? or dibekacin? or dicloxacillin? or dihydrostreptomycin sulfate? or diketopiperazine? or distamycin? or doxycycline? or echinomycin? or edeine? or enviomycin? or erythromycin? or erythromycin estolate? or erythromycin ethylsuccinate? or filipin? or floxacillin? or fluoroquinolone? or fosfomycin? or framycetin? or fusidic acid? or gentamicin? or gramicidin? or hygromycin? or imipenem? or josamycin? or kanamycin? or kitasamycin? or lactam? or lasalocid? or leucomycin? or lincomycin? or lincosamide? or lucensomycin? or lymecycline? or mepartricin? or methacycline? or methicillin? or mezlocillin? or mikamycin? or minocycline? or miocamycin? or moxalactam? or mupirocin? or mycobacillin? or nafcillin? or natamycin? or nebramycin? or neomycin? or netilmicin? or netropsin? or nigericin? or nisin? or norfloxacin? or novobiocin? or nystatin? or ofloxacin? or oleandomycin? or oligomycin? or oxacillin? or oxytetracycline? or paromomycin? or penicillanic acid? or penicillic acid? or penicillin?? or piperacillin? or pivampicillin? or polymyxin b? or polymyxin? or pristinamycin? or prodigiosin? or ribostamycin? or rifabutin? or rifamycin? or ristocetin? or rolitetracycline? or roxarsone? or roxithromycin? or rutamycin? or sirolimu? or sisomicin? or spectinomycin? or spiramycin? or streptogramin?? or streptomycin? or streptovaricin? or sulbactam? or sulbenicillin? or sulfamerazine? or sulfamethoxypyridazine? or talampicillin? or teicoplanin? or tetracycline? or thiamphenicol? or thienamycin? or thiostrepton? or ticarcillin? or tobramycin? or troleandomycin? or tunicamycin? or tylosin? or tyrocidine? or tyrothricin? or valinomycin? or vancomycin? or vernamycin? or viomycin? or virginiamycin? or beta‐lactams) adj2 (resistant or resistance)).ti,ab. and (pc.fs. or (preventi$ or best practice? or evidence$ or policy or policies or pathway?).ti,ab,hw. or (guidance or guiding or guide? or guideline? or algorithm? or collaborat$ or computer$ or decision$ or emergency or formulary or guidance or guideline? or icu or impact or initiat$ or intensive care interdisciplin$ or interprofession$ or multidisciplin$ or multi‐disciplin$ or notification? or order entry or pharmacist? or pharmacy or pharmacies or policy or policies or prescrib$ or (quality adj2 (manag$ or improv$ or circle?)) or ((patient? or hospital?) adj2 record?) or reminder? or rotating or rotation or support or team$).ti,ab.)

3 (antibiotic? and (education$ or continuing‐education$ or cme or decision‐making or evidence‐based or ebm or guidance or guideline? or habit? or impact or improper$ or inappropriat$ or influenc$ or intervention? or management or overprescrib$ or overuse or overusing or pattern? or policy or policies or prescribing or prudent$ or stewardship? or rational or unnecessary or "use" or "usage")).ti.

4 (antibiotic? adj4 (education$ or continuing‐education$ or cme or decision‐making or evidence‐based or ebm or guidance or guideline? or habit? or impact or improper$ or inappropriat$ or influenc$ or intervention? or management or overprescrib$ or overuse or overusing or pattern? or policy or policies or prescribing or prudent$ or rational or stewardship or unnecessary or "use" or "usage")).ab.

5 antibiotic?.ti. and evidence‐based.hw.

6 ((antimicrobial? or anti‐microbial? or penicillin?) and (stewardship or guidance or guideline? or policy or policies)).ti.

7 ((antimicrobial? or anti‐microbial? or penicillin?) adj3 (stewardship or guidance or guideline? or policy or policies)).ab.

8 (antibiotic? adj5 (hour? or immediat$ or emergency)).ab. or (antibiotic? and (hour? or immediat$ or emergency)).ti. or (antibiotic? adj3 (rotat$ or timing or time or decision$ or notification or appropriat$)).ab. or (antibiotic? and (rotat$ or timing or time or decision$ or notification or appropriat$)).ti.

9 or/1‐8

10 exp anti‐bacterial agents/

11 antibiotic?.ti,ab.

12 (alamethicin or amdinocillin or amdinocillin pivoxil or amikacin or amoxicillin or amoxicillin‐potassium clavulanate combination or amphotericin or ampicillin or anisomycin or antimycin or aurodox or azithromycin or azlocillin or aztreonam or bacitracin or bacteriocins or bambermycins or bongkrekic acid or brefeldin or butirosin sulfate or calcimycin or candicidin or capreomycin or carbenicillin or carfecillin or cefaclor or cefadroxil or cefamandole or cefatrizine or cefazolin or cefixime or cefmenoxime or cefmetazole or cefonicid or cefoperazone or cefotaxime or cefotetan or cefotiam or cefoxitin or cefsulodin or ceftazidime or ceftizoxime or ceftriaxone or cefuroxime or cephacetrile or cephalexin or cephaloglycin or cephaloridine or cephalosporins or cephalothin or cephamycins or cephapirin or cephradine or chloramphenicol or chlortetracycline or citrinin or clarithromycin or clavulanic acid or clavulanic acids or clindamycin or cloxacillin or colistin or cyclacillin or dactinomycin or daptomycin or demeclocycline or dibekacin or dicloxacillin or dihydrostreptomycin sulfate or diketopiperazines or distamycins or doxycycline or echinomycin or edeine or enviomycin or erythromycin or erythromycin estolate or erythromycin ethylsuccinate or filipin or floxacillin or fluoroquinolones or fosfomycin or framycetin or fusidic acid or gentamicins or gramicidin or hygromycin or imipenem or josamycin or kanamycin or kitasamycin or lactams or lasalocid or leucomycins or lincomycin or lincosamides or lucensomycin or lymecycline or mepartricin or methacycline or methicillin or mezlocillin or mikamycin or minocycline or miocamycin or moxalactam or mupirocin or mycobacillin or nafcillin or natamycin or nebramycin or neomycin or netilmicin or netropsin or nigericin or nisin or norfloxacin or novobiocin or nystatin or ofloxacin or oleandomycin or oligomycins or oxacillin or oxytetracycline or paromomycin or penicillanic acid or penicillic acid or penicillin? or piperacillin or pivampicillin or polymyxin b or polymyxins or pristinamycin or prodigiosin or ribostamycin or rifabutin or rifamycins or ristocetin or rolitetracycline or roxarsone or roxithromycin or rutamycin or sirolimus or sisomicin or spectinomycin or spiramycin or streptogramin? or streptomycin or streptovaricin or sulbactam or sulbenicillin or sulfamerazine or sulfamethoxypyridazine or talampicillin or teicoplanin or tetracycline or thiamphenicol or thienamycins or thiostrepton or ticarcillin or tobramycin or troleandomycin or tunicamycin or tylosin or tyrocidine or tyrothricin or valinomycin or vancomycin or vernamycin or viomycin or virginiamycin or beta‐lactams).ti,ab.

13 (infection control$ or nosocomial$ or cross infection? or hospital acquired infection? or mrsa).ti,ab.

14 methicillin resistan$.ti,ab.

15 aminoglycosides/ or metronidazole/ or anti‐infective agents/ or anti‐infective agents, urinary/

16 or/10‐15

17 (programs or programmes).ti.

18 empiric.ti.

19 (quality adj3 improvement?).ti.

20 (adherence or alert? or benchmark$ or (change adj3 treatment) or computer assist$ or computer support or computeri?ed or clinical decision$ or dosing or education$ or formulary or guidance or guideline? or impact or intervention or justification or methicillan‐resistant or overuse or over‐prescrib$ or overprescrib$ or pathway? or pharmacist? or policy or policies or program or programme or (quality adj3 improv$) or reminder? or resistance or restriction? or rotation? or timing or turnaround or unnecessary).ti.

21 or/17‐20

22 16 and 21

23 22 not 9

24 (randomized controlled trial or controlled clinical trial).pt. or randomized.ab. or placebo.ab. or clinical trials as topic.sh. or randomly.ab. or trial.ti.

25 exp animals/ not humans.sh.

26 43 not 45

27 intervention?.ti. or (intervention? adj6 (clinician? or collaborat$ or community or complex or design$ or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or gp or general practice? or hospital? or impact? or improv$ or individuali?e? or individuali?ing or interdisciplin$ or multicomponent or multi‐component or multidisciplin$ or multi‐disciplin$ or multifacet$ or multi‐facet$ or multimodal$ or multi‐modal$ or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib$ or prescription? or primary care or professional$ or provider? or regulatory or regulatory or tailor$ or target$ or team$ or usual care)).ab.

28 (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab.

29 (hospital$ or patient?).hw. and (study or studies or care or health$ or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw.

30 demonstration project?.ti,ab.

31 (pre‐post or "pre test$" or pretest$ or posttest$ or "post test$" or (pre adj5 post)).ti,ab.

32 (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab.

33 trial.ti. or ((study adj3 aim?) or "our study").ab.

34 (before adj10 (after or during)).ti,ab.

35 ("quasi‐experiment$" or quasiexperiment$ or "quasi random$" or quasirandom$ or "quasi control$" or quasicontrol$ or ((quasi$ or experimental) adj3 (method$ or study or trial or design$))).ti,ab,hw.

36 ("time series" adj2 interrupt$).ti,ab,hw.

37 (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month$ or hour? or day? or "more than")).ab.

38 pilot.ti.

39 pilot projects/ [ml]

40 (clinical trial or controlled clinical trial or multicenter study).pt. [ml]

41 (multicentre or multicenter or multi‐centre or multi‐center).ti.

42 andom$.ti,ab. or controlled.ti.

43 (control adj3 (area or cohort? or compare? or condition or design or group? or intervention? or participant? or study)).ab. not (controlled clinical trial or randomized controlled trial).pt. [ml]

44 "comment on".cm. or review.ti,pt. or randomized controlled trial.pt. [ml]

45 review.ti.

46 (rat or rats or cow or cows or chicken? or horse or horses or mice or mouse or bovine or animal?).ti.

47 exp animals/ not humans.sh.

48 (animal$ not human$).sh,hw.

49 *experimental design/ or *pilot study/ or quasi experimental study/ [em]

50 ("quasi‐experiment$" or quasiexperiment$ or "quasi random$" or quasirandom$ or "quasi control$" or quasicontrol$ or ((quasi$ or experimental) adj3 (method$ or study or trial or design$))).ti,ab.

51 ("time series" adj2 interrupt$).ti,ab.

52 or/26‐43

53 or/44‐48

54 52 not 53

55 9 or 23

56 54 and 55

EMBASE <1996 to 2015 Week 03> (Searched 22 January 2015) (OvidSP)

1 exp *antibiotic agent/

2 (bundle or bundles or education$ or continuing‐education$ or cme or decision‐making or guidance or (guideline? adj2 (adherence or implement$ or complian$ or comply$)) or improper$ or inappropriat$ or incorrect$ or nurse led or overprescrib$ or overuse or overusing or pharmacist initiated or physician? practice? or policy or policies or practice pattern? or (prescribing adj2 (ebm or evidence‐based or habit? or pattern? or practice or practices)) or prudent$ or rational or stewardship or unnecessary or underprescrib$).ti.

3 ("antibiotic use" or "antibiotic usage").ti.

4 (hospital$ and antibiotic?).ti.

5 ((antibiotic? or alamethicin? or amdinocillin? or amdinocillin pivoxil? or amikacin? or amoxicillin? or amoxicillin‐potassium clavulanate combination? or amphotericin? or ampicillin? or anisomycin? or antimycin? or aurodox? or azithromycin? or azlocillin? or aztreonam? or bacitracin? or bacteriocin? or bambermycin? or bongkrekic acid? or brefeldin? or butirosin sulfate? or calcimycin? or candicidin? or capreomycin? or carbenicillin? or carfecillin? or cefaclor? or cefadroxil? or cefamandole? or cefatrizine? or cefazolin? or cefixime? or cefmenoxime? or cefmetazole? or cefonicid? or cefoperazone? or cefotaxime? or cefotetan? or cefotiam? or cefoxitin? or cefsulodin? or ceftazidime? or ceftizoxime? or ceftriaxone? or cefuroxime? or cephacetrile? or cephalexin? or cephaloglycin? or cephaloridine? or cephalosporin? or cephalothin? or cephamycin? or cephapirin? or cephradine? or chloramphenicol? or chlortetracycline? or citrinin? or clarithromycin? or clavulanic acid? or clavulanic acid? or clindamycin? or cloxacillin? or colistin? or cyclacillin? or dactinomycin? or daptomycin? or demeclocycline? or dibekacin? or dicloxacillin? or dihydrostreptomycin sulfate? or diketopiperazine? or distamycin? or doxycycline? or echinomycin? or edeine? or enviomycin? or erythromycin? or erythromycin estolate? or erythromycin ethylsuccinate? or filipin? or floxacillin? or fluoroquinolone? or fosfomycin? or framycetin? or fusidic acid? or gentamicin? or gramicidin? or hygromycin? or imipenem? or josamycin? or kanamycin? or kitasamycin? or lactam? or lasalocid? or leucomycin? or lincomycin? or lincosamide? or lucensomycin? or lymecycline? or mepartricin? or methacycline? or methicillin? or mezlocillin? or mikamycin? or minocycline? or miocamycin? or moxalactam? or mupirocin? or mycobacillin? or nafcillin? or natamycin? or nebramycin? or neomycin? or netilmicin? or netropsin? or nigericin? or nisin? or norfloxacin? or novobiocin? or nystatin? or ofloxacin? or oleandomycin? or oligomycin? or oxacillin? or oxytetracycline? or paromomycin? or penicillanic acid? or penicillic acid? or penicillin?? or piperacillin? or pivampicillin? or polymyxin b? or polymyxin? or pristinamycin? or prodigiosin? or ribostamycin? or rifabutin? or rifamycin? or ristocetin? or rolitetracycline? or roxarsone? or roxithromycin? or rutamycin? or sirolimu? or sisomicin? or spectinomycin? or spiramycin? or streptogramin?? or streptomycin? or streptovaricin? or sulbactam? or sulbenicillin? or sulfamerazine? or sulfamethoxypyridazine? or talampicillin? or teicoplanin? or tetracycline? or thiamphenicol? or thienamycin? or thiostrepton? or ticarcillin? or tobramycin? or troleandomycin? or tunicamycin? or tylosin? or tyrocidine? or tyrothricin? or valinomycin? or vancomycin? or vernamycin? or viomycin? or virginiamycin? or beta‐lactams) adj2 (resistant or resistance) adj10 (best practice? or (chang$ adj (practice or clinical practice)) or evidence‐base? or policy or policies or pathway? or ((treatment or care) adj (algorithm? or pathway? or protocol)) or collaborat$ or computeri?ed or computer‐supported or decision‐mak$ or (support adj decision?) or formulary or guidance or (guideline? adj (adher$ or implement$ or concord$ or comply or complian$)) or interdisciplin$ or interprofession$ or multidisciplin$ or multi‐disciplin$ or notification? or order entry or (pharmacist? adj2 (led or initiat$ or intervention? or participat$)) or policy or policies or (prescrib$ adj (practice? or method? or algorithm? or protocol? or habit?)) or (quality adj (manag$ or improv$ or circle?)) or ((patient? or medical or electronic) adj2 record?) or reminder? or rotating or rotation or team$)).ti,ab.

6 (antibiotic? and (bundle or bundles or education$ or continuing‐education$ or cme or decision‐making or guidance or (guideline? adj2 (adherence or implement$ or complian$ or comply$)) or improper$ or inappropriat$ or incorrect$ or nurse led or overprescrib$ or overuse or overusing or pharmacist initiated or physician? practice? or policy or policies or practice pattern? or (prescribing adj2 (ebm or evidence‐based or habit? or pattern? or practice or practices)) or prudent$ or rational or stewardship or unnecessary or underprescrib$)).ti.

7 (antibiotic? adj3 (bundle or bundles or education$ or continuing‐education$ or cme or decision‐making or guidance or (guideline? adj2 (adherence or implement$ or complian$ or comply$)) or improper$ or inappropriat$ or incorrect$ or nurse led or overprescrib$ or overuse or overusing or pharmacist initiated or physician? practice? or policy or policies or practice pattern? or (prescribing adj2 (ebm or evidence‐based or habit? or pattern? or practice or practices)) or prudent$ or rational or stewardship or unnecessary or underprescrib$)).ab.

8 ((antimicrobial? or anti‐microbial? or penicillin?) and (bundle or bundles or education$ or continuing‐education$ or cme or decision‐making or guidance or (guideline? adj2 (adherence or implement$ or complian$ or comply$)) or improper$ or inappropriat$ or incorrect$ or nurse led or overprescrib$ or overuse or overusing or pharmacist initiated or physician? practice? or policy or policies or practice pattern? or (prescribing adj2 (ebm or evidence‐based or habit? or pattern? or practice or practices)) or prudent$ or rational or stewardship or unnecessary or underprescrib$)).ab. or ((antimicrobial? or anti‐microbial? or penicillin?) and (bundle or bundles or education$ or continuing‐education$ or cme or decision‐making or guidance or (guideline? adj2 (adherence or implement$ or complian$ or comply$)) or improper$ or inappropriat$ or incorrect$ or nurse led or overprescrib$ or overuse or overusing or pharmacist initiated or physician? practice? or policy or policies or practice pattern? or (prescribing adj2 (ebm or evidence‐based or habit? or pattern? or practice or practices)) or prudent$ or rational or stewardship or unnecessary or underprescrib$)).ti.

9 1 and 2

10 or/3‐8

11 9 or 10

12 intervention?.ti. or (intervention? adj6 (clinician? or collaborat$ or community or complex or design$ or doctor? or educational or family doctor? or family physician? or family practitioner? or financial or gp or general practice? or hospital? or impact? or improv$ or individuali?e? or individuali?ing or interdisciplin$ or multicomponent or multi‐component or multidisciplin$ or multi‐disciplin$ or multifacet$ or multi‐facet$ or multimodal$ or multi‐modal$ or personali?e? or personali?ing or pharmacies or pharmacist? or pharmacy or physician? or practitioner? or prescrib$ or prescription? or primary care or professional$ or provider? or regulatory or regulatory or tailor$ or target$ or team$ or usual care)).ab.

13 (pre‐intervention? or preintervention? or "pre intervention?" or post‐intervention? or postintervention? or "post intervention?").ti,ab.

14 (hospital$ or patient?).hw. and (study or studies or care or health$ or practitioner? or provider? or physician? or nurse? or nursing or doctor?).ti,hw.

15 demonstration project?.ti,ab.

16 (pre‐post or "pre test$" or pretest$ or posttest$ or "post test$" or (pre adj5 post)).ti,ab.

17 (pre‐workshop or post‐workshop or (before adj3 workshop) or (after adj3 workshop)).ti,ab.

18 trial.ti. or ((study adj3 aim?) or "our study").ab.

19 (before adj10 (after or during)).ti,ab.

20 (time points adj3 (over or multiple or three or four or five or six or seven or eight or nine or ten or eleven or twelve or month$ or hour? or day? or "more than")).ab.

21 pilot.ti.

22 (multicentre or multicenter or multi‐centre or multi‐center).ti.

23 random$.ti,ab. or controlled.ti.

24 review.ti.

25 or/12‐23

26 25 not 24

27 11 and 26

Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library Issue 1 2015 (Searched 22 January 2015)

#1 antibiotic?:ti,ab,kw

#2 ((antibacterial or anti‐bacterial or antiinfective or anti‐infective) and (agent? or drug?)):ti,ab,kw

#3 ((alamethicin? or amdinocillin? or amdinocillin pivoxil? or amikacin? or amoxicillin? or amoxicillin‐potassium clavulanate combination? or amphotericin? or ampicillin? or anisomycin? or antimycin? or aurodox? or azithromycin? or azlocillin? or aztreonam? or bacitracin? or bacteriocin? or bambermycin? or bongkrekic acid? or brefeldin? or butirosin sulfate? or calcimycin? or candicidin? or capreomycin? or carbenicillin? or carfecillin? or cefaclor? or cefadroxil? or cefamandole? or cefatrizine? or cefazolin? or cefixime? or cefmenoxime? or cefmetazole? or cefonicid? or cefoperazone? or cefotaxime? or cefotetan? or cefotiam? or cefoxitin? or cefsulodin? or ceftazidime? or ceftizoxime? or ceftriaxone? or cefuroxime? or cephacetrile? or cephalexin? or cephaloglycin? or cephaloridine? or cephalosporin? or cephalothin? or cephamycin? or cephapirin? or cephradine? or chloramphenicol? or chlortetracycline? or citrinin? or clarithromycin? or clavulanic acid? or clavulanic acid? or clindamycin? or cloxacillin? or colistin? or cyclacillin? or dactinomycin? or daptomycin? or demeclocycline? or dibekacin? or dicloxacillin? or dihydrostreptomycin sulfate? or diketopiperazine? or distamycin? or doxycycline? or echinomycin? or edeine? or enviomycin? or erythromycin? or erythromycin estolate? or erythromycin ethylsuccinate? or filipin? or floxacillin? or fluoroquinolone? or fosfomycin? or framycetin? or fusidic acid? or gentamicin? or gramicidin? or hygromycin? or imipenem? or josamycin? or kanamycin? or kitasamycin? or lactam? or lasalocid? or leucomycin? or lincomycin? or lincosamide? or lucensomycin? or lymecycline? or mepartricin? or methacycline? or methicillin? or mezlocillin? or mikamycin? or minocycline? or miocamycin? or moxalactam? or mupirocin? or mycobacillin? or nafcillin? or natamycin? or nebramycin? or neomycin? or netilmicin? or netropsin? or nigericin? or nisin? or norfloxacin? or novobiocin? or nystatin? or ofloxacin? or oleandomycin? or oligomycin? or oxacillin? or oxytetracycline? or paromomycin? or penicillanic acid? or penicillic acid? or penicillin?? or piperacillin? or pivampicillin? or polymyxin b? or polymyxin? or pristinamycin? or prodigiosin? or ribostamycin? or rifabutin? or rifamycin? or ristocetin? or rolitetracycline? or roxarsone? or roxithromycin? or rutamycin? or sirolimu? or sisomicin? or spectinomycin? or spiramycin? or streptogramin?? or streptomycin? or streptovaricin? or sulbactam? or sulbenicillin? or sulfamerazine? or sulfamethoxypyridazine? or talampicillin? or teicoplanin? or tetracycline? or thiamphenicol? or thienamycin? or thiostrepton? or ticarcillin? or tobramycin? or troleandomycin? or tunicamycin? or tylosin? or tyrocidine? or tyrothricin? or valinomycin? or vancomycin? or vernamycin? or viomycin? or virginiamycin? or beta‐lactams) and (prescrib$ or resistance or "use" or "usage" or utlii?ation)):ti,ab,kw

#4 ((antibacterial agent? or anti‐bacterial agent?) and (prescrib$ or resistance or "use" or "usage" or utili?ation)):ti,ab,kw

#5 "stewardship":ti,ab,kw

#6 (antibiotic* or antimicrobial*) and (prescrib* or prescrip*):ti,ab,kw

#7 #1 or #2 or #3 or #4 or #5 or #6

Appendix 2. Decisions based on 5 GRADE criteria about quality of evidence from RCTs in 'Summary of findings' table

Outcome prescribing, % compliance with guideline

Criterion	Evidence	Decision
Risk of bias	Effect estimate lower for 15 studies with low/medium risk of bias.	Not serious, 95% confidence interval for effect estimate 10% to 12% in studies at low or medium risk of bias.
Imprecision1	23,394 patients and 3660 events	Not serious
Inconsistency	Chi2 = 367.98, df = 28 (P < 0.00001); I2 = 92%	Not serious, effect size rather than direction (Figure 3). Variation partially explained by prespecified subgroup analysis by intervention function (Figure 7). Direction of effect consistent despite high levels of statistical heterogeneity.
Indirectness	Only 2 RCTs of restrictive interventions (Singh 2000; Strom 2010)	Not serious because this is a concern for safety rather than effectiveness.
Publication bias	Large trials, few commercially sponsored	Not serious

¹Imprecision, optimal information size threshold 862 patients for Δ 10%, control compliance 43%, α 0.05, β 0.2, dropout 10%.

Outcome prescribing, reduction in duration of all antibiotic treatment

Criterion	Evidence	Decision
Risk of bias	Effect estimate greater for 3 studies with low/medium risk of bias (Analysis 1.6).	Not serious
Imprecision1	3318 patients	Not serious, number of patients is > OIS to detect Δ 1 day (3018 patients).
Inconsistency2	All trials: Chi2 = 119.95, df = 13 (P < 0.00001); I2 = 89%	Not serious, most variation is effect size rather than direction (Figure 4).
Indirectness	Not serious for effectiveness	Not serious
Publication bias	Large trials, few commercially sponsored	Not serious

¹Imprecision, OIS is 754 patients for Δ 2 days, standard deviation 9.3 days (highest of the 3 studies contributing > 10% of weight), α 0.05, β 0.8, dropout 10%, and 3018 patients for Δ 1 day.

OIS: optimal information size

Outcome mortality

Criterion	Evidence	Decision
Risk of bias	Effect estimate and confidence interval similar for 8 studies with low/medium risk of bias	Not serious
Imprecision1	17,697 patients and 1587 events This is > OIS for 2% difference in mortality (6726 patients).	Not serious
Inconsistency	Heterogeneity: Chi2 = 16.55, df = 28 (P = 0.96); I2 = 0%	Not serious (Figure 5)
Indirectness	No trials of restrictive interventions. Mortality lower in trials at low/medium risk of bias.	Serious
Publication bias	Large trials, few commercially sponsored	Not serious

¹Imprecision, OIS threshold for patients for non‐inferiority is 6726 patients for a 2% difference in mortality.

OIS: optimal information size

All trials:

Mortality, control 11%, power 80%, dropout 10%

Non‐inferiority criteria	Total number of patients to be recruited
1%	26,900
2%	6726
3%	2988
4%	1682

Outcome length of hospital stay

Criterion	Evidence	Decision
Risk of bias	Effect size only slightly smaller for 6 RCTs at low or medium risk of bias, and the 95% CI did not include increase in length of stay.	Not serious
Imprecision1	3834 patients (> OIS for Δ 1 day but not 0.5 day). The lower bound of CI is reduction by 0.7 days for all RCTs and 0.3 days for RCTs at low or medium risk of bias.	Not serious
Inconsistency	Heterogeneity: Chi2 = 17.32, df = 14 (P = 0.24); I2 = 19%	Not serious, effect size rather than direction (Figure 6)
Indirectness	No trials of restrictive interventions	Serious
Publication bias	Large trials, few commercially sponsored	Not serious

¹Imprecision, OIS is 2014 patients for Δ 1 day and 7640 patients for Δ 0.5 day, standard deviation 7.6 (highest of the 3 studies contributing > 20% of weight), α 0.05, β 0.2, dropout 10%.

CI: confidence interval
 OIS: optimal information size
 RCT: randomised controlled trial

Appendix 3. Details of power calculations for RCTs

Annane 2013

Based on a previous study, the authors estimated that on day 5, ∼85% of control patients would be on antibiotics. They thus calculated that 57 patients in each arm would be needed to detect in a two‐sided test with an 80% probability and a 0.05 type I error, a 25% absolute reduction in the proportion of antibiotic‐treated patients on day 5. They also estimated that 20% of patients would eventually be withdrawn from the study after showing indisputable infection. One hundred and forty patients in total (70 in each arm) would thus be needed.

Bouadma 2010

Power: Assuming a mean of 12 days without antibiotics for the control group, 133 patients per study group would provide 90% power to detect a 3‐day increase in number of days without antibiotics.

Bruins 2005

The sample calculation was based on the difference in mortality of 6.5% as detected by Doern 1994. With 296 patients in each study group in each study period, the study would have power of 80.1% to yield a statistically significant result (α = 0.05, two‐tailed, specific proportions 0.120 vs 0.055).

Christ‐Crain 2004

We designed the trial to enrol 105 patients with completed follow‐up in each group. This number gave the study 95% power to detect a 30% reduction in antibiotic exposure. Assumptions included use of a two‐tailed test, a 5% level of significance, and a standard deviation (SD) of 6 days in both groups.

Christ‐Crain 2006

A study sample of 150 patients in each group gave the study a power of 95% to detect a 30% reduction in antibiotic exposure from 10 to 7 days per patient assuming a two‐tailed test, a 1% level of significance, and a SD of 6 days in both groups. This sample size gave the study a power of 74% to detect a 10% increase in the combined treatment failure and complication rate (from 10% to 20%), using the procalcitonin algorithm with a one‐sided value of 0.05.

Dranitsaris 2001

This study was designed to compare the two cefotaxime groups with the hypothesis that a higher proportion of cefotaxime orders would be within hospital guidelines in the intervention group. Inappropriate antibiotic prescribing has been shown to be as high as 40% (17). By assuming an alpha of 5% (two‐tailed), power of 80%, probability of appropriate prescribing with and without the intervention at 75% and 60% (absolute difference = 15%), respectively, the case sample size for the uncorrected Chi² test in this randomised study was 300, which was then increased by 10% to account for patient dropouts.

Esposito 2011

Pre‐study power calculations (with 90% power) showed that 76 patients in each group were necessary to detect a 15% lower antibiotic use, considering that 100% of children hospitalised for community‐acquired pneumonia were treated with antibiotics and assuming a two‐tailed test and a 5% level of significance. Since we planned to analyse the data in subgroups of mild and severe community‐acquired pneumonia, we doubled the number of patients per group (n = 152). We thus decided to enrol 160 patients in each group to allow for a 5% dropout participant.

Fine 2003

This study was designed with 80% power to detect a 1‐day decrease in length of stay from an assumed baseline of 7.2 days. The sample size was adjusted for the clustering on physician group assuming an average of 3.5 patients per group and an intraclass correlation coefficient of 0.1.

Franz 2004

The sample size calculation was based on the following assumptions: a significance level .05, a power .80, a proportion of initially missed infections of 4% in the interleukin‐8 group and 9% in the standard group, and an equivalence limit of 3%. On the basis of these assumptions, a sample size of 207 patients with infection in each group was required to demonstrate 1‐sided equivalence of the proportions of initially missed infections. Assuming a rate of bacterial infection of 18% in the study population, a total of 1150 patients needed to be enrolled into the study.

Gulmezoglu 2007

We calculated the power using standard formulae for comparison of proportions in a completely randomised design and estimated that with 40 hospitals, we would have 90% power to detect a decrease or an increase in a practice equal to the SD between hospitals, in a one‐sided significance test at 5% level of significance. For example, if the SD of use of episiotomy is 20%, we would be able to detect a decrease in the end‐of‐study rate of use of episiotomy from 70% to 50%. We used a one‐sided significance test because we believed the intervention could only improve the use of evidence‐based practices.

Jensen 2011

The final (adjusted) sample size of 1200 patients was based on an estimated mortality in the standard‐of‐care‐only group of 31.0% and a proposed absolute risk reduction of 7.5%. Detailed sample size considerations are available in the supplemental data (see Supplemental Digital Content 2, links.lww.com/CCM/A257).

Kerremans 2008

It was calculated that 1500 patients were needed to demonstrate a 6% absolute reduction in mortality (power of 80% and a two‐sided alpha of 0.05) from 25% in the control group to 18% in the rapid group (Sample Power, SPSS, Chicago, USA).

Kristoffersen 2009

Pre‐study power calculations (with 90% power) showed that 107 patients in each group were necessary to detect a 20% reduction in antibiotic use (from 10 to 8 days), assuming a two‐tailed test and a 5% level of significance.

Kritchevsky 2008

A priori power calculations determined that 40 hospitals sampling 100 cases per measurement period would give 80% power to detect a 15% difference in the pre–post change between groups in the timing of prophylaxis based on an intraclass correlation coefficient of 0.15, estimated from an earlier study of intensive care unit process improvement (0.05, 2‐tailed test).

Lacroix 2014

Power calculation suggested that 97 patients should be enrolled in each group to give 80% power at the 5% level of significance to detect a 20% difference in antibiotic prescription rate. Taking into account the possibility for lost to follow‐up patients or missing or incomplete results, we considered including 140 patients in each group.

Layios 2012

Assuming a mean stay of 7 days with 50% antibiotic exposure, a study sample of at least 250 patients in each group was deemed necessary to detect a 20% reduction in antibiotic consumption with 95% power at the 5% significance level.

Lesprit 2013

We hypothesized that the intervention might result in a 20% reduction of the duration of hospitalisation. The sample size was estimated based on the results of previous observations performed in our hospital showing that the mean length of hospital stay for patients treated with one of the targeted antibiotics was 15 ± 7 days. To detect a 20% reduction in the length of hospital stay in the intervention group with a type I error of 5% and a type II error of 80%, it was necessary to enrol a total of 506 patients (253 patients in each group).

Long 2014

"Assuming 90% of the patients in the control group would use antibiotics, and anticipating a 15% decrease in antibiotic usage in the procalcitonin (PCT) group, a sample size of 158 patients (79 patients per group) was necessary to detect a significant difference in antibiotic prescription rate between the groups with 80% power and an α error of 0.05. To account for possible loss of patients to follow‐up, we planned to enrol 180 patients." One hundred and eighty eligible patients were randomised to intervention (n = 90) or control (n = 90).

Masia 2008

We hypothesised a difference of at least 15% in defined daily doses of the targeted antibiotics between intervention and control groups based on the results of previous reports. One hundred and forty‐four patients were required in each group to reach 80% power, alpha 0.05, and, within awaited group, standard deviation of 5 days.

Nobre 2008

The trial was designed to enrol at least 66 patients to obtain a power of 90% to detect a 33% (4‐day) difference in the duration of antibiotic therapy for the initial infection between the two groups based on an estimated baseline duration of 12 days.

Oliveira 2013

Sample size calculation was based on data from a previous study, in which the mean duration of antibiotic therapy for the index infection was 8.6 ± 5.0 days among patients treated according to a PCT‐guided protocol, as compared with 10.7 (± 4.0) days in the control group (V. Nobre, unpublished observation, 2008). We thus hypothesised that the duration of the antibiotic therapy in patients treated with a PCT‐guided protocol would be at least 25% shorter than the duration observed in patients treated according to a protocol based on the serum C‐reactive protein levels. We found that 58 patients per group (a total of 116 individuals) would be necessary to demonstrate this difference, with a power of 80% and an alpha error of 5%.

Oosterheert 2005

In the control group, all patients were expected to receive a complete course of antibiotic treatment. On the basis of an expected detection rate of 20% for atypical and viral pathogens in the intervention group and an estimate of the number of possible dropouts, 100 patients would be required to demonstrate a reduction in the use of antibiotic treatment from 100% to 80%.

Paul 2006

The primary outcome measure was % inappropriate treatment, which could only be assessed in patients with microbiologically documented infections. The planned sample of 1500 patients in 15 wards had a power of greater than 99% to detect a 15% reduction in inappropriate antibiotic treatment (from 35% to 20%), for a two‐tailed test, assuming cluster randomisation of wards stratified within three hospitals by a two‐way analysis of variance and a between‐ward variance of 0.0005. We chose a sample size that would allow us to detect a difference even if two wards defaulted. The authors say that "Owing to the grant time limits the trial was stopped before attaining the planned sample size"; they recruited 570 patients for the primary outcome measure instead of the planned 1500.

Schuetz 2009

To define non‐inferiority with regard to the primary combined endpoint, the planning committee agreed on a 7.5% absolute difference as the clinically tolerable upper limit (i.e. at worst the risk of an overall adverse outcome in the PCT group was increased by 7.5%). Based on this non‐inferiority boundary, a minimal sample size of 1002 patients was determined, allowing for an overall adverse outcome rate in the control group of at most 20% and aiming for a power of 90%. Instead of a fixed sample size, we predefined a fixed recruitment period of 18 months with the goal to randomise all eligible patients from the 6 participating hospitals during that period and an extension if fewer than 1002 patients had been recruited. This prospective rule allows for the possibility of a higher number of patients and thus better power for subgroup analyses, while maintaining the integrity of the trial.

Senn 2004

The sample size was estimated according to the Freedman method of sample size estimation under the proportional‐hazards model, on the basis of pre‐study observation. One hundred and thirty‐five patients were required in each group to reach 80% power of demonstrating a 40% increase in the hazard ratio (a difference that would correspond approximately to a 25% reduction in the expected number of antibiotic‐days until modification). For practical reasons, study duration was determined before the beginning of prospective data collection: we chose a five‐month period, which was the estimated time necessary to achieve the calculated sample size. However, the observed effect (14% reduction) was lower than predicted, so the trial was underpowered.

Shehabi 2014

Sample size calculations were derived from the findings of Schuetz in which patients with lower respiratory tract infections treated with a PCT‐based algorithm showed a 35% (29% to 40%) reduction in antibiotic exposure. Assuming a median baseline exposure level of 9 days and a standard deviation of 6 days, with 165 patients per group this study had greater than 90% power to detect a clinically relevant reduction in duration of antibiotic usage of 25% (9.0 versus 6.7 days). As duration of antibiotic usage is unlikely to follow a normal distribution, in accordance with Lehmann this figure was inflated by 15%. To further account for potential dropout or loss to follow‐up (anticipated to be less than 5%), a total of 400 participants were recruited.

Singh 2000

Assuming that the patients in the experimental therapy group would have 10% worse outcome than patients in the standard therapy arm, a sample size of 200 patients (100 in each arm) would detect a difference at 0.05 and power 0.5. Assuming a 20% incidence of development of resistance in the standard therapy group and 5% in the experimental therapy group, a sample size of 176 patients (88 in each group) would be needed for significance at 0.05 and power 0.8.

NB: The study was terminated prematurely because providers caring for patients in the control group were influenced by the favourable results in the intervention group.

Stolz 2007

The trial was designed to demonstrate the persistent superiority of procalcitonin guidance in decreasing antibiotic use up to six months after the index exacerbation. The sample size was calculated from the following assumptions: a 75% use of antibiotics to treat the index exacerbation and an expected absolute reduction of this frequency from 75% to 45% with procalcitonin guidance. Considering an exacerbation rate of 70% within 6 months and 75% antibiotic use in the following exacerbations, a sample size of 186 patients (93 patients per group) was necessary to detect a significant difference in antibiotic use between both groups with a power of 85% and an error of 0.05. Considering a 20% dropout rate after assignment to the study, 223 inclusions were planned.

Stolz 2009

Considering 13 antibiotic‐free days in the control group and 18 antibiotic‐free days in the procalcitonin group, a sample size of 84 patients (42 per group) was necessary to detect a significant difference in antibiotic‐free days alive between both groups with a power of 90% and an error of 0.05 using a two‐tailed test. Assuming 8% lost to follow‐up, we planned the inclusion of 100 participants.

Yealy 2005

The primary outcome was site of treatment rather than the antibiotic process measures. "We estimated that we would need 96 eligible patients per hospital (3072 in total) to achieve 80% power to detect a 12% difference across the intervention groups for the site‐of‐treatment decision among low‐risk patients."

"For the site‐of‐treatment decision, this study achieved greater than 80% power to detect differences of 10% between high‐intensity and moderate‐intensity groups and differences of 12% between high‐intensity and low‐intensity groups according to separate 1‐tailed tests in which the level was 0.025."

Appendix 4. Contribution of 49 RCTs to meta‐analyses and to meta‐regression

Study	MA	MR	Analysis 1.1	Analysis 1.4	Analysis 1.5	Analysis 2.1	Analysis 2.4
Annane 2013	1	1	1	0	0	1	0
Bailey 1997	1	0	0	0	1	1	1
Bouza 2004	1	0	0	1	0	0	0
Bouza 2007	1	0	0	0	0	1	0
Burton 1991	1	1	1	0	0	0	1
Camins 2009	1	1	1	0	0	1	0
Christ‐Crain 2004	1	1	1	1	0	1	1
Christ‐Crain 2006	1	1	1	1	0	1	1
Danaher 2009	1	0	0	1	0	0	0
Ding 2013	1	1	1	1	0	1	0
Dranitsaris 2001	1	1	1	0	0	0	0
Esposito 2011	1	1	1	0	0	0	1
Fine 2003	1	0	0	0	0	1	0
Franz 2004	1	1	1	0	0	0	0
Fraser 1997	1	0	0	0	0	1	0
Gulmezoglu 2007	1	1	1	0	0	0	0
Gums 1999	1	0	0	0	1	1	1
Hochreiter 2009	1	0	0	1	0	1	0
Kerremans 2008	1	0	0	1	0	0	0
Kristofferson 2009	1	0	0	1	0	1	1
Kritchevsky 2008	1	1	1	0	0	0	0
Lacroix 2014	1	1	1	0	0	0	0
Layios 2012	1	0	0	1	0	0	0
Liu 2013	1	0	0	1	0	1	1
Long 2014	1	1	1	0	0	0	0
Maravic‐Stojkovic 2011	1	1	1	0	0	1	1
Masia 2008	1	0	0	0	0	1	0
Micek 2004	1	0	0	1	0	1	1
Oliveira 2013	1	0	0	1	0	1	0
Paul 2006	1	1	1	0	0	1	1
Poehling 2006	1	1	1	0	0	0	0
Qu 2012	1	0	0	1	0	1	1
Schnoor 2010	1	1	1	0	0	1	1
Schouten 2007	1	1	1	0	0	1	0
Schroeder 2009	1	0	0	1	0	1	0
Schuetz 2009	1	1	1	0	0	1	0
Senn 2004	1	1	1	0	0	1	1
Shen 2011	1	0	0	0	1	0	1
Shojania 1998	1	0	0	0	1	0	0
Singh 2000	1	1	1	0	0	1	0
Solomon 2001	1	1	1	0	0	1	0
Stocker 2010	1	1	1	0	0	0	0
Stolz 2007	1	0	0	0	0	1	0
Stolz 2009	1	1	1	0	0	0	0
Strom 2010	1	1	1	0	0	0	0
Trehnholme 1989	1	1	1	0	0	0	0
Walker 1998	1	1	1	0	0	0	0
Wyatt 1998	1	1	1	0	0	0	0
Yealy 2005	1	1	1	0	0	1	0
Totals	49	29	29	14	4	28	15

Appendix 5. Contribution of 109 ITS studies to meta‐regression of prescribing outcomes for intervention effect (n = 107) or removal (n = 5, 2 studies only had data about intervention removal)

	Intervention effect	Intervention removal Table 8	Figure 10	Figure 11	Figure 12
TOTALS	107	5	91	29	43
Study
Abramowitz 1982	1	0	1	0	1
Adachi 1997	1	0	1	0	1
Akenroye 2014	1	0	1	0	1
Aldeyab 2012	1	0	1	1	0
Ananda Rajah 2010	1	0	0	0	0
Ansari 2003	1	0	1	0	1
Avorn 1988	1	0	1	0	1
Bantar 2006	1	0	1	1	0
Barlow 2007	1	0	1	0	1
Bassetti 2009	1	0	1	1	0
Belliveau 1996	1	0	1	1	0
Benson 2014	1	0	1	0	1
Berild 2002	1	0	0	0	0
Borde 2014a	1	0	1	0	1
Borde 2015a	1	0	1	0	1
Borde 2015b	1	0	1	0	1
Bradley 1999	1	0	1	1	0
Buising 2008a	1	0	1	1	0
Buising 2008b	1	0	1	0	1
Bunz 1990	1	0	1	1	0
Buyle 2010	1	0	1	0	1
Chan 2011	1	0	1	1	0
Chan 2014	1	0	1	1	0
Chandy 2014	1	0	1	0	0
Cheng 2009	1	0	1	0	1
Cook 2011	1	0	1	0	1
Cook 2011a	1	0	1	1	0
Cortoos 2011	1	0	1	0	0
Dancer 2013	1	0	1	1	0
Dull 2008	1	0	1	0	1
Elligsen 2012a	1	0	1	0	1
Everitt 1990	1	0	1	1	0
Fitzpatrick 2008	1	0	0	0	0
Fowler 2007	1	0	1	0	1
Fukuda 2014	1	0	1	0	1
Grohs 2014	1	0	0	0	0
Gupta 1989	1	0	1	1	0
Hadi 2008	1	0	1	0	0
Halm 2004	1	0	0	0	0
Hess 1990	1	0	1	0	1
Hitti 2012	1	0	1	0	0
Huber 1982	1	0	0	0	0
Hulgan 2004	1	0	1	0	1
Inaraja 1986	1	0	0	0	0
Jobson 2015	1	0	1	0	1
Jump 2012	1	0	1	0	1
Kallen 2009	1	1	0	0	0
Kim 2008	1	1	1	1	0
Knudsen 2014	1	0	1	0	1
Kumana 2001	1	0	1	0	1
Lafuarie 2012	1	0	1	0	1
Lautenbach 2003	1	0	0	0	0
Lee 1995	1	0	1	0	1
Lee 2007	1	0	0	0	0
Lee 2014	1	0	1	0	1
Liebowitz 2008	1	0	1	0	0
Magedanz 2012	1	0	1	0	1
Marwick 2013	1	0	1	0	1
May 2000	1	0	1	0	0
McElnay 1995	1	0	1	1	0
McGowan 1976	1	0	0	0	0
McNulty 1997	1	0	1	1	0
Mercer 1999	1	0	1	1	0
Meyer 2007	1	0	1	0	0
Meyer 2009	1	0	1	0	0
Meyer 2010	1	0	1	0	0
Mittal 2014	1	0	1	0	1
Mol 2005	1	0	0	0	0
Newland 2012	1	0	1	0	1
Parikh 2014	1	0	1	0	0
Patel 1989	1	0	0	0	0
Perez 2003, Intervention 2	1	0	1	0	1
Peto 2008	1	0	1	1	0
Petrikkos 2007	1	0	1	0	0
Po 2012, Intervention 1	1	0	1	0	0
Popovski 2014	1	0	1	0	0
Price 2010	1	0	1	1	0
Richards 2003	1	0	1	1	0
Ross 2014	1	0	1	0	0
Saizy‐Callaert 2003	1	0	0	0	0
Salama 1996	1	0	1	1	0
Schwann 2011	1	0	1	0	1
Schwartz 2007	1	0	1	0	0
Sirinavin 1998	1	0	0	0	0
Skaer 1993	1	0	1	0	1
Standiford 2012	1	1	1	0	1
Stevenson 1988	1	0	1	0	0
Sun 2011	1	0	1	0	1
Suwangool 1991	1	0	1	1	0
Talpaert 2011	1	0	1	1	0
Tangden 2011	1	0	1	0	0
Toltzis 1998	1	0	1	1	0
Valiquette 2009	1	0	1	0	1
van Kasteren 2005	1	0	1	0	1
Volpe 2012	1	0	1	0	1
Wang 2014	1	0	1	1	0
Wax 2007	1	0	1	0	0
Weinberg 2001	1	0	0	0	0
Weiner 2009	1	0	1	0	1
Wenisch 2014	1	0	1	1	0
Willemsen 2010	1	0	1	0	1
Wilson 1991	1	0	1	0	0
Woodward 1987	1	0	1	1	0
Yeo 2012	1	0	1	0	1
Yong 2010	1	0	1	0	1
Yoon 2014	1	0	1	1	0
Young 1985	1	0	1	1	0

Appendix 6. RCTs and ITS studies not included in any evidence synthesis

Reasons for exclusion of 9 RCTs from prescribing meta‐analysis. Note that these studies had no valid clinical outcome data and so were not included in any meta‐analysis:

Reason	Number	Studies
Prescribing outcome continuous variable with no standard deviation	5	Lesprit 2013; Nobre 2008; Oosterheert 2005; Palmay 2014; Shehabi 2014
Insufficient detail to quantify impact on prescribing outcomes used in the meta‐analyses	4	Bouadma 2010; Farinas 2012; Jensen 2011; Kerremans 2009

Reasons for exclusion of 28 ITS studies from meta‐regression:

16 ITS studies did not include time series data about prescribing outcomes: Aldeyab 2014; Calil 2001; Carling 2003; Charbonneau 2006; Climo 1998; de Champs 1994; Dempsey 1995; Dua 2014; Gerding 1985; Khan 2003; Landman 1999; Lawes 2012; Leverstein‐van Hall 2001; Nuila 2008; Pear 1994; Toltzis 2014. Note that Bell 2014 did not include data about prescribing outcomes but did include valid clinical outcome data (Table 5).

13 ITS studies included time series data about prescribing outcomes but were excluded from meta‐regression for the following reasons:

Study	Reason
Borde 2014b	Only 3 postintervention points and compound outcome (choice and dose) not comparable with other studies
Goldstein 2009	Intervention was substitution of ertapenem for ampicillin‐sulbactam, but there are no ampicillin‐sulbactam data.
Madaras‐Kelly 2006	Effect size reported for segmented regression analysis but no variance.
McLaughlin 2005	Large, unjustified gap between pre‐ and postintervention data
Meyer 1993	Restriction of cephalosporins was in place throughout the study period. The paper reports an outbreak of cephalosporin‐resistant Klebsiella pneumoniae. Following the outbreak "approvals were reduced by 80%", but unclear whether this was because of change in restriction or reduction in requests.
Parienti 2011	Removal of restriction of fluoroquinolone and effect on MRSA, BUT only one data point prior to removal so cannot be re‐analysed.
Ostrowsky 2014	Non‐standardised intervention and prescribing outcomes across multiple hospitals
Pires 2011	"Intervention" was introduction of ertapenem into the formulary with no instruction to use less of anything else.
Pulcini 2011	4 months' pre‐ and postintervention data in 2 weekly time points. Data format not compatible with other studies.
Rattanaumpawan 2011	Removal of restriction only, and there is not enough unnecessary use before de‐restriction to detect change.
Richardson 2000	Not truly 3 pre‐intervention time points, and time intervals irregular.
Uçkay 2009	Comparison is between the deliverer of the same intervention (infectious disease physicians with and without infection control training). No pre‐intervention data
van Hees 2008	Large, unjustified gap between pre‐ and postintervention data

Appendix 7. Details of disagreements with other reviews

A systematic review on current evidence about antimicrobial stewardship objectives reported that "guideline‐adherent empirical therapy was associated with a relative risk reduction for mortality of 35% (odds ratio 0.65, 95% CI 0.54‐0.80)" (Schuts 2016). This analysis was based on 39 studies, of which 19 were identified by our literature search. We have reviewed the 20 studies that were not identified by our literature search. Only two of the 39 studies in this review reported an intervention, and both were identified by our literature review: one was invalid because it was an uncontrolled before‐after study (Garcia 2007), and one controlled before‐after study (CBA) is in our 'Characteristics of included studies' table (Dean 2006). The remaining 27 studies used case control or cohort designs to compare the outcomes of patients with and without guideline‐adherent antibiotic treatment, and did not include an intervention to change professional practice. The results of this review are in marked contrast to our analysis of mortality in 11 randomised controlled trials targeting antibiotic choice (Analysis 3.1). The aim of these interventions was to increase adherence with antibiotic guidelines for the antibiotic or route of administration. We have presented results as risk differences (Figure 8), but the odds ratio for mortality in these 11 randomised controlled trials is 0.96 (95% confidence interval (CI) 0.82 to 1.13). The most likely explanation for the discrepancy between our results and those of Schuts 2016 is confounding by indication. It is likely that patients with less complex or severe illness were more likely to receive guideline‐adherent antibiotic treatment and that there was residual confounding after adjustment for available clinical information. The only valid intervention study in the analysis by Schuts 2016 was a CBA. This study compared outcomes for community‐acquired pneumonia (CAP) for patients in 16 hospitals that had implemented a policy based on national guidelines with 19 control hospitals from the same state (Dean 2006). The CAP policy included several important elements in addition to antibiotic choice, such as antibiotic administration in the outpatient or emergency department before admission to hospital; administration of enoxaparin; and early ambulation of hospital inpatients. This study did not include any measures of process compliance, so it is unclear whether there is any relationship between mortality and adherence with the antibiotics recommended in the CAP policy.

A systematic review on the effect of antibiotic stewardship programmes on Clostridium difficile infection (CDI) reported that interventions were associated with a consistent, significant protective effect (pooled risk ratio for CDI 0.48, 95% CI 0.38 to 0.62) (Feazel 2014). This analysis was based on 16 studies, of which 10 were identified by our literature search. We have reviewed the six studies that were not identified by our literature search. Of the 16 studies included in this systematic review, four were interrupted time series (ITS) studies that we have included in our review (Elligsen 2012; Fowler 2007; Price 2010; Talpaert 2011); the remaining 12 studies were either uncontrolled before‐after or inadequate ITS studies. Elligsen 2012 only has reliable data about prescribing outcomes; CDI data are in the form of an inadequate CBA with aggregated before and after data from one intervention and one control site. The statistical analysis in this review, Feazel 2014, was not appropriate for the three ITS studies included in our review (Fowler 2007; Price 2010; Talpaert 2011). Calculation of risk ratios for the post‐ versus pre‐intervention periods is an uncontrolled before‐after analysis, which does not provide a reliable estimate of intervention effect. This is most clearly demonstrated by the results of one study (Price 2010), in which CDIs were declining pre‐intervention by ‐0.04 cases per 1000 occupied bed days per month (95% CI ‐0.08 to ‐0.01; P = 0.03). Postintervention CDI continued to decline at a slightly greater rate, but our estimate of the intervention effect was only a 10% reduction at 12 months (95% CI 85% reduction to 65% increase). In the systematic review (Feazel 2014), the reported risk ratio in the post‐ versus pre‐intervention phase was 0.52 (95% CI 0.44 to 0.61), but this result is mainly attributable to a steady decline in CDI over the entire study period rather than to any intervention effect.

Data and analyses

Comparison 1. Effectiveness: Prescribing outcomes from RCTs of interventions to reduce unnecessary antibiotic use.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Dichotomous outcomes, increase in desired practice	29	23394	Risk Difference (M‐H, Fixed, 95% CI)	0.15 [0.14, 0.16]
2 Dichotomous outcomes, all RCTs with results of cluster RCTs adjusted by inflation factor	29	5802	Risk Difference (M‐H, Fixed, 95% CI)	0.17 [0.15, 0.19]
3 Dichotomous outcomes, low or medium 'Risk of bias' studies only	15	13086	Risk Difference (M‐H, Fixed, 95% CI)	0.11 [0.10, 0.12]
4 Continuous outcomes, duration of all antibiotic treatment (days)	14	3318	Mean Difference (IV, Fixed, 95% CI)	‐1.95 [‐2.22, ‐1.67]
5 Continuous outcomes, duration of all antibiotic treatment with results of cluster RCTs adjusted by inflation factor	14	3318	Mean Difference (IV, Fixed, 95% CI)	‐1.95 [‐2.23, ‐1.67]
6 Continuous outcomes, low or medium 'Risk of bias' studies only	3	755	Mean Difference (IV, Fixed, 95% CI)	‐3.06 [‐3.76, ‐2.37]
7 Continuous outcome, consumption of targeted antibiotic only, standardised mean reduction (original outcome cost, days or DDD)	4	1053	Std. Mean Difference (IV, Fixed, 95% CI)	‐0.25 [‐0.37, ‐0.13]

Comparison 2. Adverse effects: Clinical outcomes from RCTs of interventions to reduce unnecessary antibiotic use.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Mortality, all RCTs	28	15827	Risk Difference (M‐H, Fixed, 95% CI)	‐0.00 [‐0.01, 0.00]
2 Mortality, all RCTs with results of cluster RCTs adjusted by inflation factor	28	8332	Risk Difference (M‐H, Fixed, 95% CI)	‐0.01 [‐0.02, 0.01]
3 Mortality, low or medium 'Risk of bias' RCTs	8	6249	Risk Difference (M‐H, Fixed, 95% CI)	‐0.00 [‐0.02, 0.01]
4 Length of stay, all RCTs	15	3834	Mean Difference (IV, Fixed, 95% CI)	‐1.12 [‐1.54, ‐0.70]
5 Length of stay, all RCTs with results of cluster RCTs adjusted by inflation factor	15	3834	Mean Difference (IV, Fixed, 95% CI)	‐1.22 [‐1.68, ‐0.76]
6 Length of stay, low or medium 'Risk of bias' RCTs only	6	1731	Mean Difference (IV, Fixed, 95% CI)	‐0.85 [‐1.38, ‐0.32]

Comparison 3. Adverse effects: Clinical outcomes of interventions targeting antibiotic choice.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Mortality for trial patients	11	7658	Risk Difference (M‐H, Fixed, 95% CI)	‐0.00 [‐0.02, 0.01]
2 Length of stay for trial patients	7	2276	Mean Difference (IV, Fixed, 95% CI)	‐1.50 [‐2.16, ‐0.83]

Comparison 4. Adverse effects: Clinical outcomes of interventions targeting antibiotic exposure.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Mortality for trial patients	18	9173	Risk Difference (M‐H, Fixed, 95% CI)	‐0.00 [‐0.01, 0.01]
2 Length of stay for trial patients	8	1558	Mean Difference (IV, Fixed, 95% CI)	‐0.87 [‐1.42, ‐0.33]

Comparison 5. Modifiers of intended effect: Comparison of enabling interventions with and without feedback.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Enablement with feedback	4	3747	Risk Difference (M‐H, Fixed, 95% CI)	0.19 [0.16, 0.22]
2 Enablement without feedback	7	1827	Risk Difference (M‐H, Fixed, 95% CI)	0.13 [0.09, 0.17]

5.1. Analysis.

Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 1 Enablement with feedback.

5.2. Analysis.

Comparison 5 Modifiers of intended effect: Comparison of enabling interventions with and without feedback, Outcome 2 Enablement without feedback.

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Abramowitz 1982.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all adult patients in the hospital CLINICAL PROBLEM: receiving treatment with target antibiotics SETTING: single university hospital in the USA
Interventions	FORMAT, Interventions: educational meetings with dissemination of materials; audit and feedback; educational outreach by review and recommend change Intervention Functions: education; enablement; persuasion DELIVERER: pharmacist COMPARISON: 9 months' pre‐intervention. Usual care DESIRED CHANGE: reduce inappropriate
Outcomes	PRESCRIBING: Choice: decrease in use of cefoxitin and cefamandole COST: total cost of 6 target antibiotics (calculated from data in Tables 1 and 2)
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Not stated.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper (comparison of means, uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine pharmacy systems database.
Free of other bias (ITS) ?	Low risk	Price of target antibiotics constant over the study period.

Adachi 1997.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: single hospital in the USA
Interventions	FORMAT, Interventions: dissemination of educational materials; educational outreach by review and recommend change; reminders (physical ‐ newsletter) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: reduce vancomycin prescribing and increase appropriate use of vancomycin COST: valid financial savings
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	> 1 year data pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper (comparison of means, uncontrolled before and after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Unclear risk	Not clear, no information about changes in price of vancomycin over the study period.

Akenroye 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all paediatricians and nurses in the ED PARTICIPANTS: all children with bronchiolitis CLINICAL PROBLEM: acute bronchiolitis presenting to a paediatric ED SETTING: 1 university hospital in the USA
Interventions	FORMAT, Interventions: audit and feedback; dissemination of educational materials; educational outreach by review and recommend change; reminders (physical ‐ posters and email) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: departmental physicians, nurses, and managers COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: exposure, % children treated with antibiotics CLINICAL: balancing, % admission rate, % return ED visit rate, ED length of stay (minutes) FINANCIAL: total cost per patient. No data about the intervention cost.
Notes	FINANCIAL SUPPORT: Funding: Boston Children’s Hospital Department of Medicine Quality Improvement Publication (QIPub) grant. Competing Interest: none declared ADDITIONAL INFORMATION: care pathway is in a supplementary online file
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Electronic outcome data
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Electronic outcome data
Incomplete outcome data addressed (ITS) ?	Low risk	Electronic outcome data
Free of selected reporting (ITS) ?	Low risk	Electronic outcome data
Free of other bias (ITS) ?	Low risk	> 1 year of data pre‐ and postintervention

Aldeyab 2012.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all adult patients in the hospital CLINICAL PROBLEM: patients requiring therapeutic or prophylactic antibiotics SETTING: 1 university hospital in the UK
Interventions	FORMAT, Interventions: audit and feedback; restrictive ‐ expert approval Intervention Functions: enablement, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of target antibiotics in DDD/100 OBD MICROBIAL: Clostridium difficile infections/100 OBD
Notes	FINANCIAL SUPPORT: Funding: Deanship of Scientific Research (DSR), King Abdulaziz University, Jeddah grant no. 7‐968‐D1432. Competing interest: none declared ADDITIONAL DATA: restriction policy is described in detail in an additional online file for this paper and in Conlon 2011. Microbial Risk of Bias: LOW, case definition Low, planned intervention Low, other infection control Low
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Changes in CDI screening policy and cleaning policy occurred between Phases 1 and 2 (Figure 1).
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Electronic data from pharmacy and microbiology
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Electronic data from pharmacy and microbiology
Incomplete outcome data addressed (ITS) ?	Low risk	Electronic data from pharmacy and microbiology
Free of selected reporting (ITS) ?	Low risk	Electronic data from pharmacy and microbiology
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Aldeyab 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all adult patients in the hospital CLINICAL PROBLEM: patients requiring therapeutic or prophylactic antibiotics SETTING: 1 university hospital in the UK
Interventions	FORMAT: same as in Aldeyab 2012; this article provides additional microbial outcome data for impact on MRSA infections DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: same as in Aldeyab 2012 MICROBIAL: MRSA infections/100 OBD
Notes	FINANCIAL SUPPORT: same as in Aldeyab 2012 ADDITIONAL DATA: restriction policy is described in detail in an additional online file for Aldeyab 2012 and in Conlon 2011 (additional studies) Microbial Risk of Bias: LOW, case definition Low, planned intervention Low, other infection control Low
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Data and segmented regression model of alcohol‐based hand rub included as a proxy measure for infection control practices.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Electronic data from microbiology
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Electronic data from microbiology
Incomplete outcome data addressed (ITS) ?	Low risk	Electronic data from microbiology
Free of selected reporting (ITS) ?	Low risk	Electronic data from microbiology
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Ananda‐Rajah 2010.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the medical‐surgical ICU PARTICIPANTS: all patients in the ICU CLINICAL PROBLEM: reduction in use of broad‐spectrum antibiotics considered high risk for selection of MRSA SETTING: 1 university hospital in Australia
Interventions	FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of broad‐spectrum antibiotics in DDD/1000 OBD MICROBIAL: MRSA bacteraemia rate
Notes	FINANCIAL SUPPORT: none declared. Competing Interest: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH, case definition Low, planned intervention Low, other infection control High. Infection control interventions close to antibiotic stewardship interventions clearly documented in Figure 1.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Other changes are clearly documented in Figure 1. This includes an outbreak of Acinetobacter infection co‐incident with the stewardship intervention, which resulted in appointment of 2 infection control practitioners and associated interventions. The additional staff could have influenced prescribing outcome.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention is point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Pharmacy and microbiology routine data
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Pharmacy and microbiology routine data
Incomplete outcome data addressed (ITS) ?	Low risk	Pharmacy and microbiology routine data
Free of selected reporting (ITS) ?	Low risk	Pharmacy and microbiology routine data
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Annane 2013.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in participating ICUs PARTICIPANTS: all patients in the ICUs with sepsis. Over a 3‐year period, 62/1250 screened patients were eligible for the study, of whom 31 were randomised to each arm CLINICAL PROBLEM: sepsis SETTING: 8 hospitals in France
Interventions	FORMAT, Interventions: structural ‐ rapid testing of PCT with decision support algorithm Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 140 participants in total (70 in each arm) would be needed (details in Appendix 3)
Outcomes	PRESCRIBING: exposure, % receiving antibiotics at day 5 CLINICAL: mortality, length of ICU stay, length of hospital stay MICROBIAL: colonisation with MRSA (nasal swab) and GNRB (rectal swabs)
Notes	FINANCIAL SUPPORT: Funding: commercial, Thermo Fisher B.R.A.H.M.S. France, a subsidiary of the maker of the PCT assay used in this study. Competing interests: none declared ADDITIONAL INFORMATION: supplementary online file has PCT algorithm, authors provided full study protocol (in French) Microbial Risk of Bias: MEDIUM (no data about infection control)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated
Allocation concealment (selection bias)	Low risk	Central allocation
Blinding (performance bias and detection bias) All outcomes	High risk	PCT levels not reported on control participants.
Incomplete outcome data (attrition bias) All outcomes	Low risk	No participants lost to follow‐up.
Selective reporting (reporting bias)	Low risk	No participants lost to follow‐up.
Other bias	High risk	Study stopped prematurely because of low recruitment.
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT levels not reported on control participants.
Baseline characteristics similar?	Low risk	Table 1

Ansari 2003.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: antibiotics dispensed to hospital wards for administration for therapy or prophylaxis SETTING: 1 university hospital in the UK
Interventions	FORMAT, Interventions: educational meetings; dissemination of educational materials; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: total use of Alert Antibiotics in DDD/1000 OBD FINANCIAL: cost of antibiotics adjusted for changes in price over the 4‐year study period. Cost of the Alert Antibiotic Monitoring intervention and of the setup and analysis of the ward antimicrobial supply database (Table 3)
Notes	FINANCIAL SUPPORT: no financial support. Competing Interests: none declared ADDITIONAL DATA: email response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	"In 2000, the Antibiotic Subcommittee of Tayside University Hospitals Trust devised an Alert Antibiotic Policy to reduce inappropriate use of key antibiotics, targeted because they should be reserved for infections caused by organisms that are resistant to first line antimicrobials." There were no other changes in local or national policy likely to influence use of Alert Antibiotics.
Analysed appropriately (ITS) ?	Low risk	Done in original paper: segmented regression analysis with adjustment for autocorrelation and seasonality.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	"The aim of this study was to use routine data from the pharmacy stock control computer to evaluate this intervention". Sources and methods of data collection were the same before and after the intervention.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	"After evaluation of the intervention according to patient records and its shortcomings, we decided to use the pharmacy stock data. During the 4 year period of analysis no restriction policy for dispensing the Alert Antibiotics was implemented by the hospital pharmacy, therefore the pharmacy data about dispensed Alert Antibiotics would provide us with the best available independent indicator for evaluation of the intervention."
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	"Correcting for autocorrelation avoids underestimating standard errors and overestimated significance of the effects of an intervention. For estimating seasonal autocorrelation, the autoregression model needs to evaluate correlations between error terms separated by multiples of 12 months. Accounting for seasonally correlated errors usually requires at least 24 monthly data points." Data about cost of antibiotics adjusted for price changes during study period.

Avorn 1988.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians at 1 teaching hospital PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: patients receiving therapy with cefazolin, clindamycin, or metronidazole SETTING: a 460‐bed teaching hospital in the USA
Interventions	FORMAT, Interventions: educational meetings; dissemination of educational materials; reminders ‐ circumstantial (order form triggered by receiving target antibiotic) and physical (posters) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: inappropriate dosing intervals of cefazolin, clindamycin, and metronidazole FINANCIAL: estimated annual expenditure on the 3 drugs
Notes	FINANCIAL SUPPORT: Fund for Cooperative Innovation of Blue Cross of Massachusetts and the Massachusetts Hospital Association. Competing interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	No price changes in the target antibiotics during the study period.
Analysed appropriately (ITS) ?	Low risk	Done in original paper: segmented regression analysis.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Bailey 1997.

Methods	STUDY DESIGN: RCT stratified by type of infection Risk of Bias: HIGH
Participants	PROVIDERS: all physicians at 2 teaching hospitals, excluding ICUs PARTICIPANTS: a total of 102 inpatients, 51 intervention and 51 control CLINICAL PROBLEM: patients receiving IV ABs for at least 3 days, but excluded if in ICU or with uncontrolled infection or close to discharge SETTING: 2 tertiary‐care teaching hospitals in USA
Interventions	FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: none reported
Outcomes	PRESCRIBING: patients switched from parenteral to oral antibiotics or discontinuation of 1 or more antibiotics and mean IV antibiotic days COST: mean antibiotic costs CLINICAL: 30‐day re‐admission (total and infection‐related) and in‐hospital mortality
Notes	FINANCIAL SUPPORT: Funding: Department of Pharmacy. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Physicians of patients considered candidates for intervention were randomised to be either contacted by the clinical pharmacist ... or to be observed"
Allocation concealment (selection bias)	Unclear risk	Not stated
Blinding (performance bias and detection bias) All outcomes	Unclear risk	Not stated
Incomplete outcome data (attrition bias) All outcomes	Low risk	No problems found.
Selective reporting (reporting bias)	Low risk	No problems found.
Other bias	High risk	No power calculation. Prices of antibiotics unlikely to change over the 6‐month study period.
Baseline Outcomes similar?	Unclear risk	Not stated
Free of contamination?	Unclear risk	Not stated
Baseline characteristics similar?	Low risk	See Table 1 in study.

Bantar 2006.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: IV antibiotics, restriction applied to carbapenems SETTING: a single university hospital in Argentina. Total use was compared for > 2 years before and after the intervention
Interventions	FORMAT, Intervention 1: educational outreach by review and recommend change; restrictive ‐ compulsory order form Intervention Functions: education, enablement, persuasion, restriction Intervention 2: unavailability of antibiotics during a national financial crisis DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive (choice)
Outcomes	PRESCRIBING: use of all IV antibiotics and carbapenems in DDD/1000 OBD CLINICAL: all‐cause inpatient mortality
Notes	FUNDING: none. Competing Interests: 2 authors declared conflicts of interest for speaker and advisory board fees ADDITIONAL INFORMATION: no response from authors
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Intervention 1 was independent of other changes. The "crisis" (following the intervention) was a national economic crisis and will be reported separately in the review.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data were obtained from pharmacy systems.
Knowledge of the allocation adequately prevented(ITS)?	High risk	Prescribing data were processed by the investigators to convert grams to DDD and identify only IV antibiotics.
Incomplete outcome data addressed (ITS) ?	Low risk	Routine pharmacy data
Free of selected reporting (ITS) ?	Unclear risk	Processing of data has potential for selective outcome reporting.
Free of other bias (ITS) ?	Low risk	3 years' data pre‐ and 2 years' data postintervention

Barlow 2007.

Methods	STUDY DESIGN: Controlled ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients presenting with pneumonia were recruited prospectively CLINICAL PROBLEM: adults with community‐acquired pneumonia SETTING: 2 acute university hospitals in Scotland
Interventions	FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials; reminders ‐ physical by posters and email Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: control hospital with no intervention DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Choice: % appropriate antibiotics within 4 h of admission COST: cost‐effectiveness, intervention cost, and estimated impact on mortality
Notes	FINANCIAL SUPPORT: Funding: NHS Education Scotland and Chief Scientist Office, Scotland. Competing Interests: none declared ADDITIONAL DATA: email response from authors with additional information about intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk
Knowledge of the allocation adequately prevented(ITS)?	High risk
Incomplete outcome data addressed (ITS) ?	Low risk
Free of selected reporting (ITS) ?	Low risk
Free of other bias (ITS) ?	High risk

Bassetti 2009.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the ICU (mixed medical/surgical) PARTICIPANTS: all patients in the ICU CLINICAL PROBLEM: requiring empirical antibiotic therapy SETTING: 1 university hospital in Italy
Interventions	FORMAT, Interventions: educational outreach by review and recommend change; restrictive ‐ compulsory order form Intervention Functions: education, enablement, persuasion, restriction DELIVERER: specialist physicians (ID) COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of cephalosporins in DDD/1000 OBD MICROBIAL: MRSA
Notes	FINANCIAL SUPPORT: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: LOW
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk	Routine pharmacy data
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine pharmacy data
Incomplete outcome data addressed (ITS) ?	Low risk	Routine pharmacy data
Free of selected reporting (ITS) ?	Low risk	Routine pharmacy data
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention. Microbial Risk of Bias: case defintion done, planned intervention done, other infection control measures done.

Baysari 2013.

Methods	STUDY DESIGN: unintended consequences, qualitative Risk of Bias: not assessed (qualitative study)
Participants	PROVIDERS: 36 physicians PARTICIPANTS: patients receiving antibiotic treatment CLINICAL PROBLEM: patients receiving antibiotics that the hospital policy designated as requiring approval SETTING: 1 hospital in Australia
Interventions	FORMAT, Intervention: audit and feedback; restriction by prior approval Intervention Functions: enablement, persuasion, restriction DELIVERER: AMT DESIRED CHANGE: decrease excessive
Outcomes	UNINTENDED CONSEQUENCES: problems with antibiotic policy and approval process identified through semi‐structured interviews with prescribers who had received feedback letters
Notes	FINANCIAL SUPPORT: Funding: St Vincent’s Clinic Foundation Research Grant, annual Grant #3 and National Health and Medical Research Council program grant #568612. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data about the antibiotic policy and feedback

Bell 2014.

Methods	STUDY DESIGN: unintended consequences, ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in general, gynaecological, orthopaedic, urological, and vascular surgery wards PARTICIPANTS: 12,883 patients undergoing elective surgery CLINICAL PROBLEM: risk of postoperative AKI following policy change to gentamicin for prophylaxis SETTING: 1 hospital in the UK
Interventions	FORMAT: Interventions: audit and feedback; educational meetings; dissemination of antibiotic policy; reminders (physical ‐ posters in operating theatres) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, the policy was intended to reduce Clostridium difficile infection
Outcomes	UNINTENDED CONSEQUENCES: % postoperative AKI before and after antibiotic policy change
Notes	FINANCIAL SUPPORT: Funding: Scottish Government Healthcare Associated Infection Task Force. Competing Interests: none declared ADDITIONAL DATA: email response from authors but no additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Data from laboratory computer system (serum creatinine)
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from laboratory computer system
Incomplete outcome data addressed (ITS) ?	High risk	Completeness of pre‐ and postoperative creatinine data presented in full for all services (Table 2). There was a significant increase in testing after policy change in gynaecology.
Free of selected reporting (ITS) ?	Low risk	Data from laboratory computer system
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Belliveau 1996.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: patients receiving vancomycin therapy SETTING: 1 university hospital in the USA
Interventions	FORMAT, Interventions: educational meetings; dissemination of educational materials; educational outreach by academic detailing; reminders (physical ‐ posters and newsletter); restrictive ‐ expert approval Intervention Functions: education, environmental restructuring, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: vancomycin doses/1000 OBD
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	> 12 months' pre‐ and postrestriction data
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper (comparison of means with t‐test, uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention. Outcome data were collected from all participants.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Benson 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients receiving therapeutic antibiotics SETTING: 1 university hospital in the USA
Interventions	FORMAT, Interventions: audit and feedback; educational outreach by academic detailing Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: antibiotic cost per patient day
Notes	FINANCIAL SUPPORT: none. Competing Interests; none declared ADDITIONAL INFORMATION: no response from author
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Electronic data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Electronic data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Electronic data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Electronic data from pharmacy computer
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Berild 2002.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: physicians (paediatricians) in the hospital PARTICIPANTS: all paediatric patients in the hospital CLINICAL PROBLEM: children with infections requiring antibiotic therapy SETTING: 1 paediatric university hospital in Norway
Interventions	FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials Intervention Functions: education, enablement DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: total antibiotic usage and usage of 5 specific groups of antibiotics in DDD/100 OBD
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Done, 3 years' pre‐intervention and 2 years' postintervention data
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper (run charts, Figure 1, with no statistical analysis).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	Changes in antibiotic price were documented with their contribution to reduction in cost over the study period (Table 1 in study).

Borde 2014a.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians and pharmacists in the Medical Service PARTICIPANTS: all adult patients in the Medical Service CLINICAL PROBLEM: patients receiving antibiotics SETTING: 1 university hospital in Germany
Interventions	FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials; educational outreach by review and recommend change; reminders ‐ circumstantial, on rounds Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, aim was to reduce use of 3rd‐generation cephalosporins and fluoroquinolones by 30% in 12 months
Outcomes	PRESCRIBING: Choice: drug use measured in RDD/100 OBD FINANCIAL: cost of intervention and impact on prescribing cost
Notes	FINANCIAL SUPPORT: Funding: internal funds from the Department of Medicine and Federal Ministry of Health (BMG grant IIA5‐2011‐2511FSB340). Competing Interests: none declared
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Prescribing data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Prescribing data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Prescribing data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Prescribing data from pharmacy computer
Free of other bias (ITS) ?	Low risk	> 24 months' data pre‐ and postintervention

Borde 2014b.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients with Staphylococcus aureus bacteraemia CLINICAL PROBLEM: compliance with a bundle of indicators of effective treatment and investigation SETTING: 1 community hospital in Southern Germany
Interventions	FORMAT, Interventions: dissemination of educational materials; reminders ‐ circumstantial, on microbiology reports for positive blood cultures Intervention Functions: education, enablement, environmental restructuring DELIVERER: ID physician COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Choice: average score per participant, with 0.5 points for each of 4 prescribing indicators, maximum score 2.0 per participant CLINICAL: not valid (mean mortality in pre‐ and postintervention phases)
Notes	FINANCIAL SUPPORT: Funding: internal funds from the Department of Medicine and Federal Ministry of Health (BMG grant IIA5‐2011‐2511FSB340). Competing Interests: none declared ADDITIONAL DATA: the original paper reports average scores per participant for compliance, with 5 bundle elements of which only 2 were about antibiotic prescribing (Figure 2). The authors provided us with additional data about scores for the 2 prescribing elements in the bundle.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	High risk	Prescribing outcomes were collected by the investigators.
Knowledge of the allocation adequately prevented(ITS)?	High risk	Prescribing outcomes were collected by the investigators.
Incomplete outcome data addressed (ITS) ?	Unclear risk	Data are presented as % compliance per quarter, but it is not clear whether complete data were collected from all participants.
Free of selected reporting (ITS) ?	Unclear risk	Data are presented as % compliance per quarter, but it is not clear whether complete data were collected from all participants.
Free of other bias (ITS) ?	High risk	Only 9 months' data postintervention

Borde 2015a.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians caring for medical emergency patients PARTICIPANTS: all medical patients in the ED CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: 1 university hospital in Germany
Interventions	FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials; educational outreach by review and recommend change; reminders ‐ circumstantial, on rounds Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, aim was to reduce use of 3rd‐generation cephalosporins by 20% in 12 to 24 months
Outcomes	PRESCRIBING: Choice: drug use measured in RDD/100 OBD
Notes	FINANCIAL SUPPORT: Funding: internal funds from the Department of Medicine and Federal Ministry of Health (BMG grant IIA5‐2011‐2511FSB340). Competing Interests: none declared
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy computer
Free of other bias (ITS) ?	Low risk	> 24 months' data pre‐ and postintervention

Borde 2015b.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians and pharmacists in the Medical Service PARTICIPANTS: all adult patients in the Medical Service CLINICAL PROBLEM: patients receiving antibiotics SETTING: 1 200‐bed community hospital
Interventions	FORMAT, Interventions: educational meetings; dissemination of educational materials; educational outreach by review and recommend change in ICU and for bacteraemic patients in other wards; reminders ‐ circumstantial, on rounds Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, aim was to reduce use of 3rd‐generation cephalosporins and fluoroquinolones by 30% in 12 months
Outcomes	PRESCRIBING: Choice: target drug use measured in RDD/100 OBD. Exposure: impact on total anti‐infective use was measured
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy computer
Free of other bias (ITS) ?	Low risk	> 12 months' data pre‐ and postintervention

Bouadma 2010.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the ICU PARTICIPANTS: all patients in the ICU, 311 randomised to intervention and 319 to control CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: 5 hospitals in France, 4 university and 1 general
Interventions	FORMAT, Interventions: reminders ‐ circumstantial; structural ‐ procalcitonin testing with decision support by treatment algorithm Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physicians (Anaesthesiology and Intensive Care) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 133 participants per study group (details in Appendix 3)
Outcomes	PRESCRIBING: Exposure: days of antibiotic exposure per 1000 patient days CLINICAL: primary outcome measure 28‐day mortality, also 60‐day mortality, length of ICU stay, and length of hospital stay
Notes	FINANCIAL SUPPORT: Funding: Assistance Publique‐Hopitaux de Paris, France and B.R.A.H.M.S, Germany. Competing Interests: 4 authors declared conflicts of interest from several pharmaceutical companies ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer‐generated randomisation sequence
Allocation concealment (selection bias)	Low risk	Assignment concealed before allocation.
Blinding (performance bias and detection bias) All outcomes	High risk	Assignment not concealed postallocation.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome data reported on 98% of participants in control and intervention groups.
Selective reporting (reporting bias)	Low risk	Outcome data reported fully on all included participants.
Other bias	High risk	Patients assigned to the trial were < 50% of all patients receiving antibiotics (630/1315).
Baseline Outcomes similar?	High risk	No data
Free of contamination?	Low risk	Procalcitonin only reported on intervention participants.
Baseline characteristics similar?	Low risk	ITS

Bouza 2004.

Methods	STUDY DESIGN: RCT Risk of bias: HIGH
Participants	PROVIDERS: ICU staff PARTICIPANTS: 297 patients with bloodstream infection in hospital, 109 control and 188 intervention CLINICAL PROBLEM: bacteraemia/fungaemia (bloodstream infection) SETTING: 1 university hospital in Spain
Interventions	FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: microbiologists (specialist physicians) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: no information about sample size
Outcomes	PRESCRIBING: Choice: proportion of days on which adequate treatment received CLINICAL: Intended: length of stay, mortality
Notes	FINANCIAL SUPPORT: Funding: Red Española de Investigación de Patología Infecciosa (REIPI C03‐14) and Fondo de Investigaciones Sanitarias of Spain (FIS 02‐1049). Competing Interest: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"We randomly classified the patients ... into 3 different group by means of a computer assisted random list"
Allocation concealment (selection bias)	Unclear risk	No information
Blinding (performance bias and detection bias) All outcomes	High risk	Not possible with this study design
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not stated
Selective reporting (reporting bias)	Unclear risk	Not stated
Other bias	High risk	Not done, adequate prescription was defined by 7 criteria, some of which required clinical judgement. The reliability of the primary outcome measure was not assessed.
Baseline Outcomes similar?	Unclear risk	Not stated
Free of contamination?	High risk	All doctors in the hospital were distributed across all 3 study groups.
Baseline characteristics similar?	Unclear risk	Not stated

Bouza 2007.

Methods	STUDY DESIGN: RCT Risk of bias: HIGH
Participants	PROVIDERS: ICU staff PARTICIPANTS: 250 patients in the adult ICU, 167 intervention and 83 control CLINICAL PROBLEM: ventilator‐associated pneumonia with bacteria identified on gram stain of first tracheal aspirate SETTING: single general, teaching, and referral hospital in Spain
Interventions	FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: microbiologists (specialist physicians) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: no information about sample size
Outcomes	PRESCRIBING: Exposure: mean days of therapy MICROBIAL: Clostridium difficile infection CLINICAL:Balancing: median days of fever and mechanical ventilation FINANCIAL: cost of antibiotics
Notes	Microbial Risk of Bias HIGH: no case definition, no details of other infection control measures ADDITIONAL DATA: no response from authors to request for additional data FINANCIAL SUPPORT: Red Española de Investigación de Patología Infecciosas (REIPI) and Fondo de Investigación Sanitaria (FIS). The Spanish Ministry of Health (BEFI BF03/00237, to M.V.T.). Competing Interest: none declared
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated
Allocation concealment (selection bias)	Low risk	Computer generated
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all participants.
Selective reporting (reporting bias)	Unclear risk	No primary outcome measure identified. Defined daily dose of antibiotic therapy free from selective reporting, but other outcomes (e.g. % adequate days of antibiotic therapy) were not.
Other bias	High risk	High microbial risk of bias
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	ETEST results only available for intervention group.
Baseline characteristics similar?	Low risk	Table 1

Bradley 1999.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: physicians in an adult haematology unit PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: adult patients receiving treatment for haematological malignancy SETTING: adult haematology unit in a university hospital in the UK
Interventions	FORMAT, Interventions: restrictive Intervention Functions: restriction by removal DELIVERER: specialist physician (microbiologist) COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of 4 principal IV antibiotics in patient days per month MICROBIAL: probability of remaining free of colonisation by GRE by weeks of exposure on the ward from date of first admission
Notes	FINANCIAL SUPPORT: Funding: commercial, Wyeth Pharmaceuticals. Competing Interest: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Only 4 months' pre‐intervention data, so secular changes possible.
Analysed appropriately (ITS) ?	Low risk	Done in original paper: Kaplan‐Meier plot and log rank test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, screening protocol was the same throughout the study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, screening protocol was the same throughout the study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, screening protocol was the same throughout the study period.
Free of selected reporting (ITS) ?	Low risk	Done, screening protocol was the same throughout the study period.
Free of other bias (ITS) ?	Low risk	Microbiology Risk of Bias Criteria: Case definition: DONE, colonisation by screening; Planned intervention: DONE; Other infection control, isolation, and IC practices: DONE, same throughout study.

Bruins 2005.

Methods	STUDY DESIGN: NRT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: 1833 patients with bacterial infection in hospital in 3 study periods. Period 1: 294 intervention, 320 control; Period 2: 303 intervention, 317 control; Period 3: 308 intervention, 328 control CLINICAL PROBLEM: inappropriate antibiotic therapy SETTING: 1 university hospital in the Netherlands
Interventions	FORMAT, Interventions: structural ‐ rapid microbiology laboratory testing Intervention Functions: environmental restructuring DELIVERER: specialist physician (microbiologist) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: yes, 296 participants in each study arm (details in Appendix 3)
Outcomes	PRESCRIBING: % of participants who receive appropriate treatment in first 48 h. Turnaround times for microbiology tests and results CLINICAL: intended clinical outcomes, total hospital mortality rate and length of hospital stay COST: valid financial savings
Notes	FINANCIAL SUPPORT: Funding: commercial, bioMerieux and Stichting Zorg op Regionale ´ Grondslag (ZORG). Competing Interest: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Quasi‐randomised. "Patients were randomised on the basis of the sum of the day and month of their date of birth ... even numbers assigned to the control group ... odd number to the intervention group"
Allocation concealment (selection bias)	High risk	Allocation not concealed.
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk
Selective reporting (reporting bias)	Low risk
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	Rapid reports only received by intervention group.
Baseline characteristics similar?	Low risk	Table 1

Buising 2008a.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: use of restricted antibiotics: cephalosporins, carbapenems, quinolones, glycopeptides, and aminoglycosides SETTING: 1 university hospital in Australia
Interventions	FORMAT, Interventions: audit and feedback; educational outreach by review and recommend change; restrictive ‐ expert approval and removal Intervention Functions: education, enablement, environmental, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of restricted antibiotics in DDD/1000 OBD MICROBIAL: ARGNB (Escherichia coli,Pseudomonas aeruginosa); ARGPB (MRSA)
Notes	FINANCIAL SUPPORT: Funding: National Health and Medical Research Council of Australia; Biotechnology Innovation Fund from the Commonwealth Government of Australia; Melbourne Health. Competing Interest: none declared Microbial Risk of Bias: LOW (case definition, planned intervention, and other infection control measures all low risk)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis is point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology systems
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology systems
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology systems
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology systems
Free of other bias (ITS) ?	Low risk	5 years' pre‐ and 2 years' postintervention data

Buising 2008b.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the ED PARTICIPANTS: all patients with community‐acquired pneumonia in the ED CLINICAL PROBLEM: rate of empiric antibiotic prescribing that was concordant with recommendations SETTING: 1 university hospital in Australia
Interventions	FORMAT, Interventions 1: educational outreach by academic detailing; reminders ‐ physical, posters Intervention Functions: education, environmental restructuring, persuasion Intervention 2: structural ‐ computerised decision support Intervention Functions: enablement, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % prescribing concordant with recommendation
Notes	FINANCIAL SUPPORT: Funding: National Health and Medical Research Council of Australia. Competing Interest: none declared ADDITIONAL DATA: email response from authors with further details about intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Junior staff who were targets of the intervention rotated every 3 months.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data collection was identical in all 3 phases.
Knowledge of the allocation adequately prevented(ITS)?	High risk	The nurse and physicians who collected data were not blinded to allocation.
Incomplete outcome data addressed (ITS) ?	Low risk	Data were collected from all eligible participants.
Free of selected reporting (ITS) ?	Low risk	Data were collected from all eligible participants. The accuracy of data collection was checked in a 5% sample of participants.
Free of other bias (ITS) ?	High risk	1 year of data in pre‐ and Intervention 1 time series, but only 6 months' data for Intervention 2

Bunz 1990.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: receiving metronidazole SETTING: single university hospital in Canada
Interventions	FORMAT, Interventions: educational meetings; dissemination of educational materials; reminders ‐ circumstantial, on rounds; restrictive ‐ review and make change Intervention Functions: education, enablement, restriction DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % doses of metronidazole prescribed 12‐hourly
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Although the pre‐ and postintervention phases were only a 6‐month period, data from 1 year prior to the intervention were used to control for any seasonal variation in prescribing patterns.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: run charts with no statistical analysis.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, the analysis included all prescriptions for metronidazole.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Burton 1991.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: 147 receiving aminoglycosides CLINICAL PROBLEM: patients receiving IV aminoglycosides SETTING: 1 hospital in the USA
Interventions	FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: reduce inappropriate POWER CALCULATION: no information about sample size
Outcomes	PRESCRIBING: Choice: aminoglycoside dosing and serum concentration CLINICAL: Intended: length of stay
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Random numbers table used to assign 9 of 17 house staff teams to the intervention group. Patients allocated to intervention or control groups based on house staff team to which they were admitted. The other 8 teams were assigned as control groups"
Allocation concealment (selection bias)	Unclear risk	Not stated but unlikely: 9 house staff teams were in the intervention group, 8 control, groups swapped over after 4 months.
Blinding (performance bias and detection bias) All outcomes	High risk	"Blinding as to patient status was not performed"
Incomplete outcome data (attrition bias) All outcomes	Low risk	No problems found.
Selective reporting (reporting bias)	Low risk	No problems found.
Other bias	High risk	Unit of analysis error for length of stay. This was a cluster RCT, but length of stay was analysed at participant level.
Baseline Outcomes similar?	Unclear risk	Not measured before interventions.
Free of contamination?	High risk	Not done, 9 house staff teams were in the intervention group, 8 control, groups swapped over after 4 months.
Baseline characteristics similar?	Low risk	See Table 2 in paper.

Buyle 2010.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients receiving IV fluoroquinolones CLINICAL PROBLEM: switch from IV fluoroquinolones to oral SETTING: 1 hospital in Belgium
Interventions	FORMAT, Interventions: educational meetings; dissemination of guideline; reminders ‐ circumstantial and physical (pre‐printed note placed in patient notes when the patient fulfilled criteria for IV to oral switch). NB: the circumstantial reminder was only implemented on some wards (abdominal surgery, gastro‐enterology, and plastic surgery) over 2 months, and there are no reliable data to estimate the effect of this component. Intervention Functions: education, enablement (only for the circumstantial reminder), environmental restructuring (only for the circumstantial reminder) DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate
Outcomes	PRESCRIBING: Choice: % IV/(IV + oral) fluoroquinolone usage
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interest: none declared ADDITIONAL DATA: email from authors with further information about the intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA.
Incomplete outcome data addressed (ITS) ?	Low risk	Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA.
Free of selected reporting (ITS) ?	Low risk	Data for ITS from pharmacy computer (Figure 1). Other data in Tables 2 and 3 not valid, UBA.
Free of other bias (ITS) ?	Low risk	21 months' pre‐ and 24 months' postintervention

Calfee 2003.

Methods	STUDY DESIGN: unintended consequences, case control Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in adult medical and surgical units PARTICIPANTS: all patients in the units CLINICAL PROBLEM: use of targeted antibiotics (3rd‐generation cephalosporins, piperacillin/tazobactam, aztreonam, carbapenems, parenteral clindamycin, oral and parenteral vancomycin, parenteral fluoroquinolones and macrolides, and fluconazole) SETTING: 1 hospital in the USA
Interventions	FORMAT, Interventions: restrictive by review and make change, automatic stop order for prescriptions not meeting policy indications Intervention Functions: restriction DELIVERER: AMT COMPARISON: case control study DESIRED CHANGE: decrease excessive use of targeted drugs
Outcomes	UNINTENDED CONSEQUENCES: proportion of nosocomial infections reported solely on the basis of a treating physician’s diagnosis during the endemic and epidemic periods (Table 1)
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding unlikely 2. Selection of participants into the study: Unclear, insufficient detail about selection of cases for the endemic and epidemic period 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention, and unaffected by knowledge of the outcome or risk of the outcome 4. Deviation from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Unclear, outcomes are reported as % with no numerator or denominator data 6. Measurement of outcome: High, outcome measure objective, but outcome assessors were aware of the intervention status, and the study does not report the actual number of cases 7. Selection of the reported result: High, reported effect selected from multiple measurements within the outcome domain FINANCIAL SUPPORT:Funding: no information. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data

Calil 2001.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: staff in a neonatal unit PARTICIPANTS: all patients in the neonatal care unit CLINICAL PROBLEM: requiring neonatal care SETTING: 1 neonatal care unit in a university hospital in Brazil
Interventions	FORMAT: no valid prescribing data. Restrictive DELIVERER: specialist physician COMPARISON: usual carer DESIRED CHANGE: decrease exessive
Outcomes	MICROBIAL: monthly incidence of Enterobacter cloacae infections
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	More than 1 year of data before and after intervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with logistic regression analysis of relationship between antibiotic prescribing and resistance.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Unclear risk	Not clear, no information about changes in sampling or testing protocol over study period.
Free of other bias (ITS) ?	High risk	Not done. Microbial Risk of Bias Criteria: Case definition: infection, monthly infections with E cloacae; Unplanned intervention: other infection control measures: barrier precautions, isolation of participants, and personal IC procedures fully described and same in both phases.

Camins 2009.

Methods	STUDY DESIGN: cluster RCT, service level Risk of Bias: HIGH
Participants	PROVIDERS: all internal medicine teams in the hospital PARTICIPANTS: 784 patients prescribed antibiotics in the hospital (390 intervention, 394 control), 12 clusters (internal medicine teams) CLINICAL PROBLEM: patients receiving therapeutic piperacillin‐tazobactam, levofloxacin, or vancomycin SETTING: 1 hospital in the USA
Interventions	Interventions: audit and feedback; dissemination of guidelines; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease in excessive treatment POWER CALCULATION: assuming a baseline proportion of inappropriate use for target antimicrobials of 35% (with inappropriate‐use data based on preliminary‐usage data from Grady Memorial Hospital), review of at least 330 antimicrobial prescriptions in each arm would allow for detection of a 10% reduction in inappropriate antimicrobial use.
Outcomes	PRESCRIBING: Choice: % appropriate
Notes	FINANCIAL SUPPORT: Emory Medical Care Foundation; National Institutes of Health (UL1RR024992 to BCC, K12 RR017643 to MDK and HMB, K23 AI054371 to MDK, and UL1 RR025008 to HMB). Competing Interests: BCC reports was on the speakers’ bureau for Wyeth Pharmaceuticals. All other authors report no conflicts of interest. ADDITIONAL DATA: no additional data requested
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Each month, 6 internal medicine teams were randomly assigned to the intervention arm, and 6 teams were randomly assigned to the control group by means of a random number list.
Allocation concealment (selection bias)	High risk	Not concealed
Blinding (performance bias and detection bias) All outcomes	High risk	Not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all participants.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all participants.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	High risk	Doctors randomised to intervention were in the same hospital as control doctors.
Baseline characteristics similar?	Low risk

Carling 2003.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: NOT CLEAR SETTING: 1 community teaching hospital in the USA
Interventions	FORMAT: no valid prescribing data. Educational outreach ‐ review and recommend change; educational meetings with dissemination of educational materials DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: prevalence of Clostridium difficile, ceftazidime‐resistant Enterobacteriaceae, and MRSA FINANCIAL: cost of intervention
Notes	FINANCIAL SUPPORT: Funding: institutional support. Competing Interests: none declared ADDITIONAL DATA: no additional data requested
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	3 years' pre‐intervention data
Analysed appropriately (ITS) ?	Low risk	Done in original paper: regression analysis with adjustment for autocorrelation. Analysis repeated by review team because of incomplete reporting of results.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Unclear risk	Not clear, no information about changes in sampling or testing protocol over study period.
Free of other bias (ITS) ?	Low risk	VRE isolation unlikely to have influenced C difficile or resistant gram‐negative bacteria. Microbial Risk of Bias Criteria: Planned intervention: DONE Implementation of antimicrobial management team in response to increase in use of target drugs. Case definition: DONE for C difficile infection (diarrhoea and toxin positive) or infection with clinical isolates of gram‐negative bacteria resistant to ceftazidime, or MRSA (CDC definition of nosocomial infection). Other infection control measures: DONE For C difficile contact precautions and procedures for cleansing equipment and patient care areas remained unchanged. Other infection control processes are not described in detail but may have changed during the study period (e.g. VRE isolation introduced after intervention). Data about VRE infections NOT RELIABLE: There were no cases in the pre‐intervention phase and none in the first 3 years postintervention, but there was an outbreak in the 4th and 5th postintervention years caused by admission of patients from other hospitals who were colonised with VRE.

Chan 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients CLINICAL PROBLEM: use of restricted antibiotics (amikacin, 3rd‐ and 4th‐generation cephalosporins, carbapenems, fluoroquinolones, glycopeptides, and piperacillin/tazobactam) SETTING: 1 university hospital in Taiwan
Interventions	FORMAT, Interventions: educational outreach by review and recommend change; restrictive ‐ expert approval required plus review and make change Intervention Functions: education, enablement, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: DDD/1000 OBD of restricted antibiotics MICROBIAL: isolation rates Clostridium difficile, MRSA, and multidrug‐resistant gram‐negative bacteria
Notes	ADDITIONAL DATA: no response from authors to request for additional data FINANCIAL SUPPORT: Funding: Chang Gung Memorial Hospital (Taoyuan, Taiwan) (grant CMRPG340236). Competing Interests: none declared Microbial Risk of Bias: HIGH (case definition clear, planned intervention but no data about infection control)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	States in discussion that biggest limitation was lack of external controls, but that is common to all ITS studies.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Unclear risk	DDD data from pharmacy computer, the same pre‐ and postintervention
Knowledge of the allocation adequately prevented(ITS)?	Low risk	DDD data from pharmacy computer, the same pre‐ and postintervention
Incomplete outcome data addressed (ITS) ?	Low risk	DDD data from pharmacy computer, the same pre‐ and postintervention
Free of selected reporting (ITS) ?	Low risk	DDD data from pharmacy computer, the same pre‐ and postintervention
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Chan 2015.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients requring vancomycin CLINICAL PROBLEM: patients requiring more than 2 doses of vancomycin treatment SETTING: 1 university hospital in the USA
Interventions	FORMAT Interventions: restrictive ‐ expert approval Intervention Functions: restriction DELIVERER: AMT COMPARISON: pre‐existing antimicrobial stewardship programme with audit and feedback. No valid data about impact of this programme (UBA). DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of vancomycin in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk	Routine data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Routine data from pharmacy computer
Free of other bias (ITS) ?	Low risk	21 months' pre‐ and 51 months' postintervention data

Chandy 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving antibiotics CLINICAL PROBLEM: total antibiotic use in the hospital SETTING: 1 university hospital in India
Interventions	FORMAT Interventions: dissemination of educational materials (guidelines) Intervention Functions: education DELIVERER: AMT COMPARISON: pre‐dissemination DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: total antibiotic use in DDD/100 OBD
Notes	FINANCIAL SUPPORT: none. Competing Interests: none declared ADDITIONAL INFORMATION: authors provided additional detail about the antibiotic policy and confirmed that feedback was not used in this intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk	Routine data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Routine data from pharmacy computer
Free of other bias (ITS) ?	Low risk	> 18 months' data pre‐ and postintervention

Charbonneau 2006.

Methods	STUDY DESIGN: Controlled ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients who qualified for fluoroquinolone therapy CLINICAL PROBLEM: infection with MRSA SETTING: 1 university hospital in France
Interventions	FORMAT: no valid prescribing data. Restriction, educational meetings, and dissemination of educational materials DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: reduction of MRSA infections
Notes	FINANCIAL SUPPORT: Funding: Programme Hospitalier de Recherche Clinique. Competing Interests: none declared ADDITIONAL DATA: no request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	1 year post‐ and 2 years' pre‐intervention data, so secular changes unlikely. Infection control protocols were unchanged pre‐ and postintervention.
Analysed appropriately (ITS) ?	Low risk	Done in original paper: the study is analysed as a CBA adjusting for confounders and slope and level. The ITS analyses are correct, but the results are not well reported.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Unclear risk	Not clear, no information about changes in sampling or testing protocol over study period.
Free of other bias (ITS) ?	Low risk	Microbial Risk of Bias Criteria: Planned intervention: DONE Case definition: DONE clear case definition of clinical infection: "A new case was defined as a case in a patient with no previous history of MRSA or ESBL‐EB colonization or infection who was infected with MRSA or ESBL‐EB no less than 48 h after hospital admission." Other infection control measures: DONE "The measures recommended by French national guidelines for the prevention of nosocomial infections were implemented in the 4 study hospitals several years before the study began"

Cheng 2009.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving IV antibiotics CLINICAL PROBLEM: reduce inappropriate prescribing of broad‐spectrum IV antibiotics in hospital inpatients SETTING: 1 university hospital in China
Interventions	FORMAT, Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of targeted antibiotics in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Does not mention other changes apart from preceding Antimicrobial Stewardship Programme.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Routine pharmacy data used for outcome, so assume complete.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine pharmacy data used for outcome, so assume complete.
Incomplete outcome data addressed (ITS) ?	Low risk	Routine pharmacy data used for outcome, so assume complete.
Free of selected reporting (ITS) ?	Low risk
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Christ‐Crain 2004.

Methods	STUDY DESIGN: cluster RCT Risk of bias: MEDIUM
Participants	PROVIDERS: physicians in hospital PARTICIPANTS: 234 patients (124 intervention, 119 control), 16 clusters (weeks randomly assigned to either standard or procalcitonin) CLINICAL PROBLEM: suspected  lower respiratory tract infection SETTING: 1 university hospital in Switzerland
Interventions	FORMAT, Interventions: dissemination of educational materials; reminders ‐ circumstantial and physical (procalcitonin algorithm) triggered by prescribing antibiotics; structural Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 105 participants in each group
Outcomes	PRESCRIBING: Choice: relative risk of antibiotic exposure measured in percentage and patient‐days CLINICAL: Balancing: length of stay; mortality
Notes	FINANCIAL SUPPORT: Funding: B.R.A.H.M.S (Hennigsdorf, Germany) and Orgenium Laboratories (Turku, Finland) provided assay material and partial support of this investigator‐initiated project. Freiwillige Akademische Gesellschaft Basel, Switzerland; internal from the Department of Internal Medicine and the Divisions of Endocrinology and Pneumology. Competing Interests: BM served as consultant and received payments from B.R.A.H.M.S (the manufacturer of procalcitonin assays) to attend meetings related to the trial and for travel expenses, speaking engagements, or research ADDITIONAL DATA: email response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"We randomly assigned eligible patients ... according to a computer generated week wise randomisation scheme"
Allocation concealment (selection bias)	Unclear risk	"We randomly assigned eligible patients either standard antimicrobial therapy (standard group) or procalcitonin‐guided antimicrobial treatment (procalcitonin group) according to a computer‐generated week wise randomisation scheme". No information about concealment of allocation
Blinding (performance bias and detection bias) All outcomes	Low risk	"Single blinded intervention trial"
Incomplete outcome data (attrition bias) All outcomes	Low risk	Antibiotic data from all treated participants
Selective reporting (reporting bias)	Low risk	Objective outcome measure in all participants
Other bias	Low risk	No other apparent biases found.
Baseline Outcomes similar?	Unclear risk	Not stated
Free of contamination?	Low risk	Although same doctors treated participants in non‐intervention weeks, they did not have data about procalcitonin results.
Baseline characteristics similar?	Low risk	Done, Tables 1 and 2 in the original paper

Christ‐Crain 2006.

Methods	STUDY DESIGN: RCT Risk of bias: MEDIUM
Participants	PROVIDERS: physicians in hospital PARTICIPANTS: 302 patients (151 intervention, 151 control) CLINICAL PROBLEM: suspected community‐acquired pneumonia SETTING: 1 university hospital in Switzerland
Interventions	FORMAT, Interventions: dissemination of educational materials; reminders ‐ circumstantial and physical (procalcitonin algorithm) triggered by prescribing antibiotics; structural Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 150 participants in each group
Outcomes	PRESCRIBING: Choice: relative risk of antibiotic exposure, total antibiotic use. Duration of antibiotic course CLINICAL: Balancing: mortality and length of hospital stay FINANCIAL: total antibiotic cost and cost per patient
Notes	FINANCIAL SUPPORT: Funding: B.R.A.H.M.S (Hennigsdorf, Germany), Pfizer (Schweiz AG), and Mepha (Schweiz AG) was used for assay material and salaries of technical personnel; internal from Departments of Internal Medicine and Emergency Medicine, the Stiftung Forschung Infektionskrankheiten (SFI), and Departments of Endocrinology and Pulmonary Medicine, University Hospital Basel, Switzerland. Competing Interests: 2 authors received payments from B.R.A.H.M.S AG, the manufacturer of the procalcitonin assay. ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Patients were randomly assigned to one of the two groups by sealed opaque envelopes", no information about generation of randomisation sequence
Allocation concealment (selection bias)	Low risk	"Sealed opaque envelopes"
Blinding (performance bias and detection bias) All outcomes	Low risk
Incomplete outcome data (attrition bias) All outcomes	Low risk
Selective reporting (reporting bias)	Low risk
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	Same doctors in the intervention and control weeks, but they did not have access to procalcitonin results in the control weeks.
Baseline characteristics similar?	Low risk	Done, Table 1 in the original paper

Chu 2003.

Methods	STUDY DESIGN: CBA Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: patients with clinical problem CLINICAL PROBLEM: community‐acquired pneumonia SETTING: a total of 36 (20 intervention, 16 control), non‐university community hospitals in USA
Interventions	FORMAT, Interventions: audit and feedback; educational meetings; dissemination of educational materials ‐ pack including guideline and literature review Intervention Functions: education, enablement DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Choice: process measures sputum and blood cultures within 4 hours, antibiotics within 4 hours, first antibiotic in emergency room CLINICAL: Intended: mortality and LOS
Notes	FINANCIAL SUPPORT: Funding: contract 500‐99‐P619 "Utilization and Quality Control Peer Review Organization for the State of Oklahoma" from the Centers for Medicare and Medicaid Services. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Control cohort study (CBA)
Allocation concealment (selection bias)	High risk	Control cohort study (CBA)
Blinding (performance bias and detection bias) All outcomes	High risk	Control cohort study (CBA)
Incomplete outcome data (attrition bias) All outcomes	Low risk	Objective primary outcome collected on all participants.
Selective reporting (reporting bias)	Low risk	Objective primary outcome collected on all participants.
Other bias	Low risk	No other apparent biases found.
Baseline Outcomes similar?	Low risk	Tables 1 and 2
Free of contamination?	Low risk	Intervention and control were at different sites.
Baseline characteristics similar?	Low risk	Tables 3 and 4

Clerc 2014.

Methods	STUDY DESIGN: NRT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: "We planned to include around 100 patients in the intervention group". No power calculation provided. Recruited 106 intervention and 91 control participants. CLINICAL PROBLEM: first episode of Staphylococcus aureus bacteraemia SETTING: 1 university hospital in Switzerland
Interventions	FORMAT, Interventions: structural ‐ rapid laboratory testing for meticillin resistance Intervention Functions: environmental restructuring DELIVERER: specialist physician (ID and Microbiology) COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % compliance with guideline recommended use of vancomycin
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL INFORMATION: the authors confirmed that this intervention did not include feedback to participants
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Odd versus even hospital number
Allocation concealment (selection bias)	Unclear risk	"Mode of allocation was concealed from the clinicians", but unclear how this was achieved.
Blinding (performance bias and detection bias) All outcomes	High risk	Open study
Incomplete outcome data (attrition bias) All outcomes	Low risk	Primary outcome reported on all participants.
Selective reporting (reporting bias)	Unclear risk	Primary outcome reported on all participants. Authors did a secondary analysis excluding participants with penicillin allergy, but this was not prespecified.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Unclear risk	Clinicians received results verbally and electronically, so it is likely that they were aware of the intervention, which may have influenced their management of other participants.
Baseline characteristics similar?	Low risk	Table 1

Climo 1998.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: all patients in the hospital SETTING: a 703‐bed tertiary‐care university hospital in the USA
Interventions	FORMAT: no reliable prescribing data. Restriction by expert approval DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: cases of Clostridium difficile‐associated diarrhoea per quarter (ITS data). Prevalence of clindamycin‐resistant C difficile
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Done, infection control measures fully described and same in both phases.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Unclear risk	Not clear, no information about changes in sampling or testing protocol over study period
Free of other bias (ITS) ?	High risk	NOT DONE Microbial Risk of Bias Criteria: Planned intervention: NOT DONE; Case definition: DONE infection, diarrhoea, and toxin positive Other infection control measures: DONE barrier precautions, isolation of participants with C difficile‐associated diarrhoea, and personal IC procedures fully described and same in both phases.

Connor 2007.

Methods	STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW
Participants	PROVIDERS: all physicians prescribing vancomycin PARTICIPANTS: 120 patients with vancomycin prescription approved for only 72 hours CLINICAL PROBLEM: interruption of vancomycin treatment SETTING: 1 hospital in the USA
Interventions	FORMAT, Interventions: reminders (circumstantial and physical) stickers in medical records on day 3 warning of impending stop order; restrictive: stop order if approval not obtained Intervention Functions: enablement, environmental restructuring, restriction DELIVERER: AMT COMPARISON: participants with and without sticker DESIRED CHANGE: decrease excessive
Outcomes	UNINTENDED CONSEQUENCES: interruption of vancomycin treatment
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding unlikely 2. Selection of participants into the study: Low, selection into the study unrelated to intervention (sticker in notes) or outcome 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome 4. Deviations from intended interventions: Low, the study was designed to detect intervention failure (no warning sticker) 5. Missing data: Low, outcome data and intervention status complete on all 120 participants 6. Measurement of outcome: Low, outcome measure objective and unaffected by intervention status 7. Selection of the reported result: Low, reported effect predefined FINANCIAL SUPPORT: Funding: none. Competing Interests: EL received research support from Merck Pharmaceuticals and Ortho‐McNeil Pharmaceuticals. All other authors reported no conflicts of interest.

Cook 2011a.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving antibiotics CLINICAL PROBLEM: use of all prophylactic and therapeutic antibiotics SETTING: 1 university hospital in the USA
Interventions	FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: total use of all antibiotics in DDD/1000 OBD MICROBIAL: Clostridium difficile and MRSA infections/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: PPC is a member of the speakers’ bureau of Pfizer, Astellas, and Merck. PPC has received research funding from GlaxoSmithKline, Merck, Gilead, Pfizer, and Bristol‐Myers Squibb. All other authors have none to declare. ADDITIONAL DATA: email response from authors with additional data about intervention Microbial Risk of Bias HIGH: case definition low; planned intervention, other infection control high ‐ new policy for screening and isolation of MRSA introduced just before prescribing intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Routine data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Routine data from pharmacy computer
Free of other bias (ITS) ?	Low risk	2 years' pre‐ and postintervention data

Cook 2011b.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving antibiotics CLINICAL PROBLEM: patients receiving ciprofloxacin for treatment of any infection SETTING: 1 university hospital in the USA
Interventions	FORMAT, Intervention 1 component: educational outreach by review and recommend change Intervention 1 functions: education, enablement, persuasion Intervention 2 component: restrictive by expert approval Intervention 2 function: restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of ciprofloxacin in DDD/1000 OBD MICROBIAL: infections with ARGNB ‐ % carbapenem resistant Pseudomonas aeruginosa
Notes	FINANCIAL SUPPORT: Funding: commercial, grant from Merck & Co., Inc. Competing Interests: PC is a member of the speakers’ bureau of Merck and Astellas. He has received research support from Merck, Gilead, and Pfizer. ADDITIONAL DATA: email response from authors with additional data about intervention Microbial Risk of Bias HIGH case definition low; planned intervention, other infection control high ‐ change in screening and isolation for MRSA just before prescribing intervention may have impacted on transmission of P aeruginosa
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Routine data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Routine data from pharmacy computer
Free of other bias (ITS) ?	Low risk	5 years' pre‐ and 4 years' postintervention data

Cortoos 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all adult patients with community‐acquired pneumonia CLINICAL PROBLEM: compliance with guideline for community‐acquired pneumonia SETTING: 1 university hospital in the Netherlands
Interventions	FORMAT: Intervention 1: dissemination of educational materials Intervention 1 function: education Intervention 2: reminders ‐ physical, questionnaire about guideline compliance, distributed once Intervention 2 functions: environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % guideline compliance
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL INFORMATION: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	This and all other ROB criteria are for interventions 1 and 2 only. Intervention 3 and 4 could not be evaluated because they are too close together and also coincided with an influenza epidemic. Neither intervention 3 nor intervention 4 meets the EPOC minimum criteria for ITS. There are insufficient data to adjust for seasonal effects, and the target condition (pneumonia) has large seasonal variation.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	High risk	Data collection was different in the postintervention phase (see below).
Knowledge of the allocation adequately prevented(ITS)?	Unclear risk	Compliance to therapy was assessed with a "computerised algorithm". However, the criteria for guideline adherence presented in the supplementary materials (Table S2) would require chart review, unless the hospitals had very sophisticated electronic patient records, which is not stated. The fact that patients were excluded because of "incomplete files" suggests that chart review was required, so knowledge of the allocated interventions could not be adequately prevented.
Incomplete outcome data addressed (ITS) ?	Unclear risk	The 477 included participants had complete data for assessment of outcomes. 5 patients were excluded because of incomplete patient records.
Free of selected reporting (ITS) ?	Low risk
Free of other bias (ITS) ?	Unclear risk	Insufficient data to account for seasonal effects. Although information about guideline compliance is reported for 2 hospitals, the ITS in Figure 1 only has data from 1 hospital (UZL). The data for the second hospital (ZOL) are actually a UBA and have been excluded.

Danaher 2009.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 52 patients (14 intervention, 38 control) CLINICAL PROBLEM: excessive prescribing of antibiotics SETTING: 1 military teaching hospital in USA
Interventions	FORMAT, Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion FORMAT: Persuasive: educational outreach ‐ review and recommend change DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: "Since this was an explanatory study, no a priori estimates of effect size were available to perform power and sample size calculations." The goal was to have 180 participants in the trial.
Outcomes	PRESCRIBING: Choice: antibiotic use (DDDs and days of treatment) CLINICAL: Balancing: clinical outcomes, mortality, and re‐admission
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated
Allocation concealment (selection bias)	Unclear risk	21 of 73 patients considered for enrolment were excluded, but it is not clear if this was pre‐ or postrandomisation. The number of participants in the study group was 14, versus 38 in the control group, with no justification for the unequal numbers.
Blinding (performance bias and detection bias) All outcomes	Low risk	Primary outcome data from pharmacy computer
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all participants.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all participants.
Other bias	High risk	Aim was to enrol 180 patients, but only 72 patients were identified, and 21 of them were excluded.
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	High risk	Education intervention with study and control in same hospital
Baseline characteristics similar?	Low risk

Dancer 2013.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: requiring antibiotic prophylaxis or treatment SETTING: 1 district general hospital in the UK
Interventions	FORMAT: Interventions: restrictive Intervention Functions: restriction by removal from all wards except for ED and ICU and by therapeutic substitution ("empirical prescription of ceftriaxone and ciprofloxacin for systemic sepsis and surgical prophylaxis was changed to amoxicillin, gentamicin and metronidazole") DELIVERER: AMT COMPARISON: multifaceted intervention introduced 7 months before restriction and remaining in place throughout restrictive period. Components: audit and feedback; educational outreach by review and recommend change; educational meetings and reminders on microbiology reports. There is only 2 months' data before the multifaceted intervention, so it is not possible to estimate its effect. DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of cephalosporins and ciprofloxacin in DDD/1000 OBD MICROBIAL: CDI, MRSA, and resistant gram‐negative bacteria
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: SD received financial support for attending conferences by Janssen‐Cilag, Pfizer, and Novartis ADDITIONAL DATA: authors provided additional details about the intervention, including information about regular feedback to participants that was not in the original paper Microbial Risk of Bias LOW case definition Low, planned intervention Low, other infection control practices Low
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	9 months' data pre‐restriction includes an additional persuasive intervention 7 months' pre‐restriction; effect cannot be assessed because of insufficient pre‐intervention data.
Analysed appropriately (ITS) ?	Low risk	Analysed by correlation and time‐lag modelling, but re‐analysed as segmented regression analysis.
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk	Routine data from microbiology and pharmacy computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data from microbiology and pharmacy computers
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data from microbiology and pharmacy computers
Free of selected reporting (ITS) ?	Low risk	Routine data from microbiology and pharmacy computers
Free of other bias (ITS) ?	High risk	Only 9 months' pre‐intervention data

de Champs 1994.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: physicians on a paediatric ICU PARTICIPANTS: all patients on paediatric ICU CLINICAL PROBLEM: neonates requiring intensive care including empirical antibiotic treatment SETTING: paediatric ICU in a university hospital in France
Interventions	FORMAT: No valid prescribing data. Restrictive: change in antibiotic policy from gentamicin to amikacin DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: monthly incidence of infection with multiresistant Enterobacter cloacae
Notes	FINANCIAL SUPPORT: Funding: grant from D.R.E.D. (Direction de la Recherche et des Etudes Doctorales). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Only 7 months' pre‐intervention data, so secular/seasonal changes possible. Very complex case definition with no information about how this was applied reliably across the pre‐ and postintervention periods.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Unclear risk	Case definition included clinical interpretation.
Knowledge of the allocation adequately prevented(ITS)?	Unclear risk	NOT CLEAR because of case definition
Incomplete outcome data addressed (ITS) ?	Unclear risk	Availability of all data required for the case definition not documented.
Free of selected reporting (ITS) ?	Unclear risk	Not clear, no information about changes in sampling or testing protocol over study period.
Free of other bias (ITS) ?	High risk	Microbial outcome risk of bias: Unplanned intervention: implementation of change in response to emergence of gentamicin‐resistant E cloacae; Case definition:infection from clinical or screening isolates combined with 7 clinical criteria and 5 additional laboratory criteria assessed by a resident paediatrician and a consultant microbiologist and verified by a consultant paediatrician. Reliability of this outcome measure not clear. Other infection control measures: well documented, no changes during the study period

Dean 2001.

Methods	STUDY DESIGN: CBA Risk of Bias: HIGH
Participants	PROVIDERS: all inpatient and outpatient services in the state of Utah PARTICIPANTS: 22,985 Medicare beneficiaries aged 65 or older with 28,661 episodes of community‐acquired pneumonia, of which 7719 were hospitalised CLINICAL PROBLEM: community‐acquired pneumonia SETTING: 23 hospitals (and 60 outpatient clinics), all in Utah, USA
Interventions	FORMAT: no valid prescribing data. Reminder; educational outreach ‐ academic detailing; and educational meetings or dissemination of educational material DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	CLINICAL: Intended: 30‐day mortality and length of stay
Notes	FINANCIAL SUPPORT: supported by HealthInsight and Intermountain Healthcare. The analyses upon which this publication is based were performed under contract number 500 –96‐P604, entitled “Utilization and Quality Control Peer Review Organization for the State of Utah”, sponsored by the Health Care Financing Administration (HCFA), Department of Health and Human Services. This article is a direct result of the Health Care Quality Improvement Program initiated by HCFA, which has encouraged identification of quality improvement projects derived from analysis of patterns of care, and therefore required no special funding on the part of the contractor. Conflict of interest: no information ADDITIONAL DATA: authors provided additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	CBA
Allocation concealment (selection bias)	High risk	CBA
Blinding (performance bias and detection bias) All outcomes	High risk	CBA
Incomplete outcome data (attrition bias) All outcomes	Low risk	Objective outcome measure collected on all participants.
Selective reporting (reporting bias)	Low risk	Objective outcome measure collected on all participants.
Other bias	Low risk	No other apparent biases found.
Baseline Outcomes similar?	Low risk	Table 1
Free of contamination?	Unclear risk	NOT CLEAR, some hospitals had both intervention and control physicians. Intermountain Healthcare provides 50% of regional health care delivery in Utah. In rural IHC hospitals, 90% of pneumonia admissions were cared for by IHC‐affiliated physicians, whereas in urban IHC hospitals only 44% of pneumonia admissions were cared for by IHC‐affiliated physicians. Non‐affiliated physicians caring for patients at IHC hospitals may have been influenced by guideline implementation at these hospitals.
Baseline characteristics similar?	Low risk	Table 1

Dean 2006.

Methods	STUDY DESIGN: CBA Risk of bias: HIGH
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: a total of 17,728 patients aged 66 years or older CLINICAL PROBLEM: admitted with community‐acquired pneumonia SETTING: 35 hospitals in Utah, USA (16 from Intermountain Healthcare and 19 from other systems)
Interventions	FORMAT: no valid prescribing data. Reminder; educational outreach by academic detailing; and educational meetings with dissemination of educational materials DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	CLINICAL: Intended: 30‐day mortality, LOS, and 30‐day re‐admission
Notes	FINANCIAL SUPPORT: this study was funded by the Deseret Foundation and HealthInsight, Salt Lake City. The authors have no relevant conflicts of interest to report. ADDITIONAL DATA: authors provided additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	CBA
Allocation concealment (selection bias)	High risk	CBA
Blinding (performance bias and detection bias) All outcomes	High risk	CBA
Incomplete outcome data (attrition bias) All outcomes	Low risk	Electronic record linkage used.
Selective reporting (reporting bias)	Low risk	30‐day mortality was primary outcome.
Other bias	Low risk	Objective primary outcome measure
Baseline Outcomes similar?	Low risk	Table 3
Free of contamination?	Low risk	NOT CLEAR, some hospitals had both intervention and control physicians. The 100,000 annual inpatient admissions of Intermountain Healthcare represent almost one‐half of Utah hospital admissions. Intermountain Healthcare has an employed physician group and several non‐Medicare health maintenance organisation insurance plans, but many non‐employed physicians and non‐health maintenance organisation patients also utilise its facilities. Non‐affiliated physicians caring for patients at Intermountain Healthcare hospitals may have been influenced by guideline implementation at these hospitals.
Baseline characteristics similar?	Unclear risk	Not stated

Dempsey 1995.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: patients with nursing home‐acquired pneumonia SETTING: 1 hospital in the USA
Interventions	FORMAT: no valid prescribing data. Audit and feedback; reminders; and educational meetings with dissemination of educational materials DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	CLINICAL: Intended: length of stay FINANCIAL: charge per case of nursing home‐acquired pneumonia
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	< 1 year data pre‐ and postintervention, so seasonal trends cannot be excluded.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Patient administration system
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Patient administration system
Incomplete outcome data addressed (ITS) ?	Unclear risk	No explicit statement about complete follow‐up
Free of selected reporting (ITS) ?	Low risk
Free of other bias (ITS) ?	Low risk

Ding 2013.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians PARTICIPANTS: 78 patients with acute exacerbations of idiopathic pulmonary fibrosis (39 intervention, 39 control) CLINICAL PROBLEM: management of acute exacerbations SETTING: 1 hospital in China
Interventions	FORMAT, Interventions: structural ‐ introduction of procalcitonin testing with decision support algorithm Intervention Functions: enablement, environmental restructuring DELIVERER: respiratory physicians COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Exposure: % participants treated and duration of antibiotic treatment CLINICAL: Balancing: mortality, length of stay, duration of mechanical ventilation
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response to request from authors
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer‐generated numbers
Allocation concealment (selection bias)	Low risk	Computer‐generated numbers
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all participants.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all participants.
Other bias	High risk	No power calculation
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	Procalcitonin results only available for intervention participants.
Baseline characteristics similar?	Low risk	Table 1

Dranitsaris 2001.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: physicians assigned to the 7 services PARTICIPANTS: 309 patients with clinical problem (162 intervention, 147 control) CLINICAL PROBLEM: adult patients with infections requiring IV cefotaxime SETTING: 2 hospitals in Canada
Interventions	FORMAT: Interventions: educational outreach ‐ review and recommend change Intervention Functions: enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 330 participants, 165 in each group. Details in Appendix 3
Outcomes	PRESCRIBING: Choice: percentage of cefotaxime prescriptions that were consistent with guideline for both indication and dosage SECONDARY: mean duration of therapy and cost per treatment course
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Randomised on a one to one basis via a computer generated list"
Allocation concealment (selection bias)	Low risk	Randomisations carried out in central pharmacy and "telephone on a consecutive basis".
Blinding (performance bias and detection bias) All outcomes	High risk	Not done, acknowledged as a limitation by authors on page 179.
Incomplete outcome data (attrition bias) All outcomes	Low risk	See Table 3; all participants included
Selective reporting (reporting bias)	Low risk	See Table 3; all participants included
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	High risk	Control participants were managed by the same physicians as intervention participants.
Baseline characteristics similar?	Low risk	Table 1

Dua 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians involved in vascular surgery PARTICIPANTS: all patients undergoing vascular surgery CLINICAL PROBLEM: surgical‐site infection following vascular surgery SETTING: USA, multiple hospitals (stratified, random sample of 20% of all non‐federal inpatient hospital admissions throughout the USA)
Interventions	FORMAT: no valid prescribing data. Surgical Care Improvement Project core measures with financial incentives implemented in 2006 DELIVERER: specialist phsicians COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: no data CLINICAL: inpatient surgical‐site infection
Notes	FINANCIAL SUPPORT: no funding. Competing Interests: none declared ADDIDIONAL DATA: authors responded to request but had no additional relevant data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	No data about antibiotic prescribing
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Unclear risk	Outcome relied on ICD discharge coding to identify surgical‐site infection, may have been influenced by financial incentives to meet SCIP targets.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Electronic outcome data
Incomplete outcome data addressed (ITS) ?	High risk	Outcome data were restricted to inpatient coding, but most surgical‐site infections likely to present postdischarge.
Free of selected reporting (ITS) ?	Low risk	Electronic outcome data
Free of other bias (ITS) ?	Low risk	> 1 year of pre‐ and postintervention data

Dull 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians, pharmacists, and nurses in surgical department PARTICIPANTS: all patients undergoing elective surgery CLINICAL PROBLEM: choice, timing, and duration of antibiotic prophylaxis SETTING: 7 hospitals in the USA
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of educational materials; educational outreach by academic detailing; reminders (physical, posters, intranet, and faxes to physicians) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: % participants with prophylaxis discontinued within 24 h of surgery
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Electronic outcome data
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Electronic outcome data
Incomplete outcome data addressed (ITS) ?	Low risk	Electronic outcome data
Free of selected reporting (ITS) ?	Low risk	Electronic outcome data
Free of other bias (ITS) ?	High risk	10 months' pre‐ and 12 months' postintervention data

Duvoisin 2014.

Methods	STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW
Participants	PROVIDERS: all physicians PARTICIPANTS: 222 infants with early‐onset neonatal sepsis CLINICAL PROBLEM: early onset sepsis SETTING: 1 hospital in Switzerland
Interventions	FORMAT, Interventions: restrictive by review and make change targeted at ordering of CBC and CRP tests Intervention Functions: restriction DELIVERER: specialist physician (ID) COMPARISON: usual care DESIRED CHANGE: decrease excessive use of diagnostic tests
Outcomes	UNINTENDED CONSEQUENCES: time to first antibiotic dose and complications (requirement for catecholamine treatment and/or mechanical ventilation, meningitis, or death)
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding unlikely 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome 4. Deviations from intended interventions: Low, the study demonstrated large reduction in CBC (30%) and CRP (91%) 5. Missing data: Low, outcome data and intervention status complete on all 222 participants 6. Measurement of outcome: Low, outcome measure objective and unaffected by intervention status 7. Selection of the reported result: Low, reported outcomes predefined and measured from routine data FINANCIAL SUPPORT: Funding: SICPA Foundation and the Société Académique Vaudoise. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data about intervention

Elligsen 2012.

Methods	STUDY DESIGN: CITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the critical care team PARTICIPANTS: all critical care patients in the hospital CLINICAL PROBLEM: decrease use of broad‐spectrum antibiotics in critical care patients SETTING: 1 tertiary‐care centre in Ontario, Canada
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of targeted broad‐spectrum antibiotics (days of therapy/1000 OBD)
Notes	FINANCIAL SUPPORT: Funding: Canadian Institutes of Health Research, Ontario Ministry of Health, and Long Term Care Academic Health Services Centre Innovation Award. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Done. October to August both pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Done, point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	High risk	No, the intervention was open to all participants and prescribers, difficult to conceal.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, Figures 1 and 2
Free of other bias (ITS) ?	Low risk	Done, no other apparent biases

Esposito 2011.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all paediatric physicians PARTICIPANTS: 319 children with pneumonia were enrolled and randomly assigned 1:1 to the treatment groups, but, as consent was withdrawn during the study in 9 cases (5 intervention and 4 control), the final analysis was based on 310 children (155 intervention and control) CLINICAL PROBLEM: children hospitalised with community‐acquired pneumonia SETTING: 1 university hospital in Italy
Interventions	FORMAT: Interventions: structural ‐ rapid testing for procalcitonin and decision support algorithm Intervention Functions: enablement, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 76 participants in each group. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: % started on antibiotics and % children treated for > 10 days CLINICAL: length of stay, duration of fever, antibiotic adverse effects
Notes	FINANCIAL SUPPORT: Funding: Italian Ministry of Health (Bando Giovani Ricercatori 2007). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated
Allocation concealment (selection bias)	Low risk	Sealed envelopes
Blinding (performance bias and detection bias) All outcomes	Low risk	PCT levels only reported on intervention participants.
Incomplete outcome data (attrition bias) All outcomes	Low risk	319 randomised, consent was withdrawn during the study in 9 cases (3% participants, 5 in the PCT group and 4 in the control group). Outcomes reported on all participants (Tables 2‐3). All 310 children came to the planned follow‐up visits.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all 310 participants (Tables 2‐3). All 310 children came to the planned follow‐up visits.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT levels only reported on intervention participants.
Baseline characteristics similar?	Low risk	Table 1

Everitt 1990.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: physicians in Obstetrics & Gynaecology PARTICIPANTS: patients (women) with clinical problem CLINICAL PROBLEM: Caesarean section SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (circumstantial, on the structured order form for intravenous antibiotics, which was triggered for every order for IV antibiotics); restriction by expert approval and by removal Intervention Functions: education, enablement, environmental restructuring, restriction DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: relative use of cefazolin or cefoxitin in Caesarean sections that received < 5 g of either drug perioperatively FINANCIAL: estimated financial savings
Notes	FINANCIAL SUPPORT: Funding: Beth Israel Hospital, Boston, Massachusetts and Fund for Cooperative Innovation of Blue Cross of Massachusetts and the Massachusetts Hospital Association. Competing Interests: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Only 9 months pre‐intervention data, so secular/seasonal changes possible.
Analysed appropriately (ITS) ?	Low risk	Done in original paper, segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	Antibiotic costs adjusted to 1986 prices over the whole study period.

Farinas 2012.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 1185 patients receiving at least 3 days of IV antibiotics (571 intervention, 614 control) CLINICAL PROBLEM: adherence to recommendations for change of therapy SETTING: 1 university hospital in Spain
Interventions	FORMAT: no valid prescribing outcome data. Educational outreach (review and recommend change) DELIVERER: specialist (ID) physicians COMPARISON: usual care DESIRED CHANGE: increase appropriate antibiotic treatment SAMPLE SIZE: 571 intervention, 614 control POWER CALCULATION: no power calculation. No adjustment for intracluster correlation
Outcomes	PRESCRIBING: Choice but no valid outcome data (% adherence with recommendations, but no data about antibiotic use in terms of choice, route, or duration of treatment) CLINICAL: Balancing: length of stay and % treatment failure
Notes	FINANCIAL SUPPORT: Funding: Fondo de Investigaciones Sanitarias (FIS PI06/90094), and Instituto de Formación e Investigación Marqués de Valdecilla (IFIMAV) (API 06/03). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer
Allocation concealment (selection bias)	High risk	Randomisation stratified by clinical units, not blinded. Participants were randomised by groups (stratified randomisation by clinical units) to intervention or non‐intervention using the EPIDAT 3.1 programme (Dirección Xeral de Saúde Pública, Xunta de Galicia & Organización Panamericana de la Salud. Santiago de Compostela, Coruña, Spain, 2003).
Blinding (performance bias and detection bias) All outcomes	High risk	Randomisation not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all participants.
Selective reporting (reporting bias)	Unclear risk	The primary outcome (clinical failure) was complex and not entirely objective.
Other bias	High risk	Unit of analysis error, no adjustment for intracluster correlation
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	19 participants in the control group were excluded because they had ID consultation.
Baseline characteristics similar?	Low risk	Table 1

Fine 2003.

Methods	STUDY DESIGN: cluster RCT, service level Risk of bias: HIGH
Participants	PROVIDERS: all physicians in hospitals PARTICIPANTS: 608 patients with community‐acquired pneumonia (263 intervention, 325 control), 7 clusters (sites) CLINICAL PROBLEM: duration of IV antibiotic therapy and LOS SETTING: 7 nonprofit hospitals in Pittsburgh, Pennsylvania, USA
Interventions	FORMAT: Interventions: dissemination of educational materials, educational outreach by review and recommend change; reminders (circumstantial, physical, detail sheets in physician notes for patients with community‐acquired pneumonia and verbal, telephone calls); restrictive; structural Intervention Functions: education, enablement, environmental restructuring, persuasion, restriction DELIVERER: AMT  COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 600 participants in total. Details in Appendix 3
Outcomes	PRESCRIBING: Choice: duration of IV antibiotic therapy CLINICAL: intended clinical outcomes, mortality, re‐admission
Notes	FINANCIAL SUPPORT: Funding: Agency for Healthcare Research and Quality and the National Institute of Allergy and Infectious Diseases (HS08282), Robert Wood Johnson Foundation. Competing Interests: no statement ADDITIONAL DATA: authors provided additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Physician groups were randomly assigned after stratification for practice type, group size, and patient volume, but details not clear.
Allocation concealment (selection bias)	High risk
Blinding (performance bias and detection bias) All outcomes	Low risk
Incomplete outcome data (attrition bias) All outcomes	Low risk
Selective reporting (reporting bias)	Low risk
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data about LOS prior to intervention
Free of contamination?	Low risk
Baseline characteristics similar?	Low risk

Fitzpatrick 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: prescribing of cefuroxime and quinolones SETTING: 1 hospital in the UK
Interventions	FORMAT: Interventions: dissemination of guideline Intervention Functions: education DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of cefuroxime and ciprofloxacin (DDD/Finished Consultant Episode ratio)
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	No mention of any other changes, although little information given.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Done. Intervention point was clear.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done. Outcomes are objective.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done. Figures 1 and 2
Free of other bias (ITS) ?	Low risk	Done. No other bias apparent.

Fowler 2007.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: physicians in the hospital PARTICIPANTS: patients 80 years and older CLINICAL PROBLEM: Clostridium difficile infection in the elderly SETTING: 3 acute medical wards for the elderly in 1 university hospital in the UK
Interventions	FORMAT: Interventions: audit and feedback, dissemination of guideline; reminders (physical, laminated pocket version of guideline) Intervention Functions: education, enablement, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate
Outcomes	PRESCRIBING: Choice: monthly use of target antibiotics MICROBIAL: monthly count of cases of CDI
Notes	FINANCIAL SUPPORT: no funding. Competing Interests: none declared ADDITIONAL DATA: email response from authors but no additional data Microbial Risk of Bias LOW: Planned intervention: Low Case definition: Low, National definition. Other infection control measures: Low
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Ongoing audit and feedback
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Done. Point of analysis is point of the intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	High risk	No, not possible
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems so unlikely to be incomplete.
Free of selected reporting (ITS) ?	Low risk	Done, Figures 3 and 4
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Franz 2004.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: physicians in neonatal units PARTICIPANTS: 1291 neonates < 72 hours of age were randomised (656 intervention, 635 control) CLINICAL PROBLEM: suspected bacterial infection SETTING: 8 centres in 5 countries (Australia, Austria, Belgium, Germany, Sweden)
Interventions	FORMAT, Interventions: dissemination of guideline; structural, introduction of testing for C‐reactive protein and interleukin‐8 with decision support algorithm Intervention Functions: education, enablement, environmental restructuring DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, total of 1150 participants. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: number of newborn infants who received antibiotic therapy
Notes	FINANCIAL SUPPORT: Funding: grant P.575 from the Center for Applied Clinical Studies of the University of Ulm and Swedish Research Council. DPC (Los Angeles, CA) provided the Immulite automated analysers and the kits for determination of IL‐8 and sponsored the initial meeting of the investigators. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Randomly assigned to 1 or 2 diagnostic algorithms using sealed opaque envelopes"
Allocation concealment (selection bias)	Low risk
Blinding (performance bias and detection bias) All outcomes	Low risk	Done, IL‐8 results were only provided to physicians in the intervention group.
Incomplete outcome data (attrition bias) All outcomes	Low risk
Selective reporting (reporting bias)	Low risk
Other bias	Low risk
Baseline Outcomes similar?	High risk	No data
Free of contamination?	Low risk
Baseline characteristics similar?	Low risk

Fraser 1997.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: medical, surgery, intensive care, haematology, and oncology PARTICIPANTS: patients with the clinical problem CLINICAL PROBLEM: adult inpatients receiving 1 or more of 10 designated parenteral antibiotics for 3 or more consecutive days SETTING: single teaching hospital in the USA
Interventions	FORMAT: Interventions: educational outreach by review and recommend change; reminders (circumstantial, physical, written suggestions placed in the notes of participants receiving IV antibiotics) Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive SAMPLE SIZE CALCULATION: no information
Outcomes	PRESCRIBING: Choice: days receiving IV antibiotic therapy per participant, DDDs of IV antibiotics per participant. Antibiotic charges (USD) per participant CLINICAL: Balancing: clinical response at 3 days after completion of antibiotics; retreatment with antibiotics within 7 days; inpatient mortality; re‐admission within 30 days of discharge FINANCIAL: savings on drug costs in USD
Notes	FINANCIAL SUPPORT: Funding: commercial (Bayer Pharmaceuticals) and the Maine Medical Center Research Committee. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Patients randomised ... using an unblocked computer generated random number table"
Allocation concealment (selection bias)	High risk	Not possible; "The patient population was assigned to 1 of 4 medical service groups based on where they were treated at randomizations"
Blinding (performance bias and detection bias) All outcomes	High risk	Not possible
Incomplete outcome data (attrition bias) All outcomes	Low risk	For primary outcomes, not secondary
Selective reporting (reporting bias)	Low risk	Based on microbial response and other clinical parameters
Other bias	Low risk	No problems noted.
Baseline Outcomes similar?	Unclear risk	No information about baseline outcomes pretrial in the allocated groups.
Free of contamination?	High risk	Doctors likely to have cared for participants in all groups.
Baseline characteristics similar?	Low risk	Table 1

Fridkin 2002.

Methods	STUDY DESIGN: CBA Risk of Bias: HIGH
Participants	PROVIDERS: a total of 50 ICUs located in 20 hospitals PARTICIPANTS: patients in the ICU CLINICAL PROBLEM: vancomycin use, prevalence of VRE SETTING: hospitals in the USA participating in the ICU surveillance component of National Nosocomial Infection Surveillance
Interventions	FORMAT: 5 interventions were used by 3 to 19 hospitals (some hospitals used more than 1). 3 interventions were hospital‐wide and 2 were unit‐specific. Hospital‐wide interventions (22 ICUs) Intervention 1: educational meetings with dissemination of educational materials, 9 ICUs. Intervention function: education. Intervention 2: audit and feedback, 19 ICUs. Intervention function: enablement. Intervention 3: restriction, 3 ICUs. Intervention function: restriction. Unit‐specific interventions (11 ICUs) Intervention 4: educational meetings with dissemination of educational materials. Intervention function: education. Intervention 5: restriction, 3 ICUs. Intervention function: restriction. DELIVERER: AMT COMPARISON: national benchmark data DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: DDDs of vancomycin MICROBIAL: percentages of VRE and MRSA
Notes	FINANCIAL SUPPORT: Funding: CDC Emerging Infections Program. Competing Interests: no information ADDITIONAL DATA: email response from authors but no additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	CBA ‐ not randomised
Allocation concealment (selection bias)	High risk	CBA ‐ not randomised
Blinding (performance bias and detection bias) All outcomes	High risk	CBA, allocation not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	Not clear: “Susceptibility reports from isolates obtained as part of infection‐control surveillance were excluded.” Criteria for exclusion of isolates are not described and may not have been consistent across all hospitals.
Selective reporting (reporting bias)	Low risk	Not clear: “Susceptibility reports from isolates obtained as part of infection‐control surveillance were excluded.” Criteria for exclusion of isolates are not described and could have led to reporting bias.
Other bias	Unclear risk	NOT CLEAR Microbial Risk of Bias Criteria: Case definition: percentage VRE or percentage MRSA in clinical isolates; Planned intervention: DONE; Other infection control isolation: NOT CLEAR; IC practices: NOT CLEAR Data were collected about infection control changes in response to feedback of data, but the paper does not report any results.
Baseline Outcomes similar?	Unclear risk	Not stated
Free of contamination?	Low risk	Interventions were at different hospitals from control sites.
Baseline characteristics similar?	Unclear risk	Not stated

Friedberg 2009.

Methods	STUDY DESIGN: unintended consequences, cohort study Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in EDs PARTICIPANTS: 13,042 adult patients CLINICAL PROBLEM: presenting with respiratory symptoms SETTING: 385 hospitals in the USA
Interventions	FORMAT, Interventions: audit and feedback, public reporting of antibiotic timing measure as 1 of 10 national quality indicators; financial, institution incentive Intervention Functions: enablement, incentive DELIVERER: Hospital Quality Alliance COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	UNINTENDED CONSEQUENCES: rates of pneumonia diagnosis, antibiotic use, and waiting times to see a physician
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Unclear, analysis says it was adjusted for confounding of the effect of intervention but insufficient detail 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement classification of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Deviations from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Unclear, outcome data reported as % with no numerator/denominator (Table 2) 6. Measurement of outcome: High, the effect estimate is based on regression analysis of annual data for 3 years pre‐ and 2 years postintervention (i.e. only 2 postintervention time points). The authors say that "based on the NHAMCS sample, there were an estimated 40 million (95% confidence interval, 39 to 42 million) ED visits to hospitals by adults with respiratory symptoms between 2001 and 2005." In Table 1, around 10% of these patients had a diagnosis of CAP, so they were not short of data! They should surely have split their data into more time points. 7. Selection of the reported result: Low, single analysis of the intervention‐outcome relationship FINANCIAL SUPPORT:Funding: Primary Care Teaching and Education Fund (internal), Health Resources and Services Administration, and Agency for Healthcare Research and Quality. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data

Fukuda 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all adult inpatients receiving target antibiotics for 14 days or more CLINICAL PROBLEM: de‐escalation of treatment in patients who received carbapenems, cephalosporins, or quinolones for at least 14 days SETTING: 1 community hospital in Japan
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive and decrease cost
Outcomes	PRESCRIBING: Choice: cost of target antibiotics (USD/1000 OBD)
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Outcome data from pharmacy computer
Free of other bias (ITS) ?	High risk	Only 6 month pre‐intervention data, so cannot adjust for seasonal effects.

Gerding 1985.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all prescribers in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: requiring aminoglycoside treatment SETTING: 1 Veterans Administration hospital in the USA. UBA data about resistance from 14 other similar hospitals
Interventions	FORMAT: no valid prescribing data. Restrictive. DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: resistance to gentamicin and aminoglycoside use
Notes	FINANCIAL SUPPORT: Funding: commercial, Bristol Laboratories and the Veterans Administration. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: MEDIUM, case definition Low, planned intervention Low, other infection control Unclear, no information
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Only 4 months' pre‐intervention data, so secular/seasonal changes possible. No information about infection control measures.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analyses was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Routine data
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data
Free of selected reporting (ITS) ?	Low risk	Routine data
Free of other bias (ITS) ?	Unclear risk	NOT CLEARMicrobial Outcome Risk of Bias: Planned intervention: DONE Implementation in response to emergence of gentamicin resistance over the previous 5 years; Case definition: DONE Infection from clinical isolates; Other infection control measures: NOT CLEAR, no information provided.

Goldstein 2009.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians PARTICIPANTS: all adult patients in the hospital CLINICAL PROBLEM: patients requiring IV antibiotics SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: dissemination of formulary Intervention Function: education After 9 months there was an additional restrictive intervention (autosubstitution of ampicillin sulbactam by ertapenem), but this was not targeted at imipenem use, and no data are provided about prescribing or microbial outcomes for ampicillin sulbactam. DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: imipenem use in DDD MICROBIAL: % susceptibility to imipenem in clinical isolates of Pseudomonas aeruginosa
Notes	FINANCIAL SUPPORT: Funding: commercial Merck (manufacturers of ertapenem). Competing Interests: Ellie JC Goldstein is on the advisory boards of Merck, is in the speakers' bureau of Merck, and received research support from Merck; Shuang Lu is employed by Merck Research Laboratories and may own stock or stock options. Anne R Meibohm was formerly employed by Merck Research Laboratories and may own stock or stock options. ADDITIONAL DATA: email response from authors to request for additional data Microbial Risk of Bias LOW: case definition low risk, planned intervention low risk, other infection control measures low risk, no change
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	High risk	Only 6 months' pre‐intervention data for intervention 1 and 9 months' for intervention 2

Grohs 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians PARTICIPANTS: all adult patients CLINICAL PROBLEM: requiring treatment with IV 3rd‐generation cephalosporin SETTING: 1 university hospital in France
Interventions	FORMAT: Intervention: distribution of antibiotic policy Intervention Function: education DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: ceftriaxone use in DDD/1000 OBD MICROBIAL: number of participants carrying high level AmpC beta‐lactamase
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition low, planned intervention low, other infection control measures unclear (no data)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from microbiology and pharmacy computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from microbiology and pharmacy computers
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from microbiology and pharmacy computers
Free of selected reporting (ITS) ?	Low risk	Outcome data from microbiology and pharmacy computers
Free of other bias (ITS) ?	High risk	Short time series, annual data with only 5 pre‐ and 7 postintervention data points

Gulmezoglu 2007.

Methods	STUDY DESIGN: cluster RCT, hospital level Risk of Bias: HIGH
Participants	PROVIDERS: obstetric teams PARTICIPANTS: 1000 consecutively delivered women in obstetric units, 40 clusters (hospitals) CLINICAL PROBLEM: women undergoing Caesarean section SETTING: 22 hospitals in Mexico City and 18 in Thailand
Interventions	FORMAT: Interventions: educational meetings and dissemination of brochures; reminders (physical, posters and brochures) Intervention Functions: education, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective POWER CALCULATION: yes, 40 hospitals. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: % women receiving antibiotic prophylaxis for Caesarean section
Notes	FINANCIAL SUPPORT: Funding: UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP). Competing Interests: 4 authors were editors of The WHO Reproductive Health Library since its inception in 1997 to date of publication. ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Random number generator used (detailed in other article).
Allocation concealment (selection bias)	Low risk	Allocation by hospital
Blinding (performance bias and detection bias) All outcomes	Unclear risk	No mention of this
Incomplete outcome data (attrition bias) All outcomes	Low risk	Field workers collected from hospital data and were able to consult mothers for any missing data.
Selective reporting (reporting bias)	Low risk	Field workers collected from hospital data and were able to consult mothers for any missing data.
Other bias	High risk	End of study in Thai control hospital was conducted at a later stage due to other healthcare‐related activities going on.
Baseline Outcomes similar?	Unclear risk	Appear to be different but unclear
Free of contamination?	Low risk	Allocation by hospital
Baseline characteristics similar?	Unclear risk	No data

Gums 1999.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: a total of 272 patients CLINICAL PROBLEM: patients receiving inappropriate antibiotic therapy judged on culture results, risk of toxicity or drug interaction, drug cost, and duration of treatment SETTING: single 275‐bed community hospital in the USA
Interventions	FORMAT: Interventions: educational outreach by review and recommend change; reminders (circumstantial and physical, placed in notes of patients who were receiving antibiotics) Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no justification provided for the sample size
Outcomes	PRESCRIBING: Choice: cost of antibiotic therapy CLINICAL: Balancing: length of stay FINANCIAL: charges for antibiotics, laboratory and radiology services, total patient charges. Implementation cost based on days per week required for Pharmacy and Infectious Diseases staff.
Notes	FINANCIAL SUPPORT: no information ADDITIONAL INFORMATION: no response from authors
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Method of sequence generation not clear; "eligible patients were blindly randomised to the intervention or control group"
Allocation concealment (selection bias)	High risk	Not possible to conceal allocation because all intervention participants had a consultation, whereas no control participants did.
Blinding (performance bias and detection bias) All outcomes	Unclear risk	Unclear, despite objective primary outcome measure (LOS), it is not clearly stated that record linkage was without knowledge of allocation.
Incomplete outcome data (attrition bias) All outcomes	Low risk	No problems found, data were analysed from 93% of randomised participants.
Selective reporting (reporting bias)	Low risk	No problems found.
Other bias	Low risk	No other apparent biases found.
Baseline Outcomes similar?	Low risk	Done for primary outcome
Free of contamination?	Low risk	Participants were randomised to receive a consultation from an ID specialist (intervention) or no consultation (control), so no contamination likely.
Baseline characteristics similar?	Low risk	Done, Table 1 of the original paper

Gupta 1989.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: patients with clinical problem CLINICAL PROBLEM: patients receiving cefazolin SETTING: 1 university hospital in Canada
Interventions	FORMAT: Interventions: dissemination of memo; reminders (physical, newsletter); restrictive by review and make change Intervention Functions: education, environmental restructuring, persuasion, restriction DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % of cefazolin doses prescribed at < 8‐hour intervals
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Only 3 months' pre‐intervention data, so secular/seasonal changes possible.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper, Х2 test on mean before‐after.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Hadi 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: residents in internal medicine department PARTICIPANTS: patients with clinical problem CLINICAL PROBLEM: antibiotics use in patients with a fever SETTING: 5 wards in internal medicine department of teaching hospital in Indonesia
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by academic detailing; reminders (physical, pocket book version of guideline) Intervention Functions: education, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate
Outcomes	PRESCRIBING: Exposure: % patients treated and total antibiotic consumption (DDD/100 patient days)
Notes	FINANCIAL SUPPORT: Funding: Royal Netherlands Academy of Arts and Sciences, Scientific Programme Indonesia‐Netherlands (SPIN). Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	No, seasonal variation
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Done, point of analysis is point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data was collected by trained data collectors.
Knowledge of the allocation adequately prevented(ITS)?	High risk	No, blinding was not possible.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, states they assured completeness of data by collecting while patients were still in the department.
Free of selected reporting (ITS) ?	Low risk	Done, Figure 2
Free of other bias (ITS) ?	Low risk	Done, all biases addressed.

Halm 2004.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospitals PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: adults with community‐acquired pneumonia SETTING: 4 university hospitals, New York, USA
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (circumstantial and physical, on computer order system for antibiotics and pocket version of guideline) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: percentage of patients treated with guideline‐recommended antibiotics
Notes	FINANCIAL SUPPORT: Funding: Mount Sinai‐New York University Health System, the North Shore‐Long Island Jewish Health System, and the Robert Wood Johnson Foundation Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	NOT DONE, subjective outcome measure, not blinded
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with χ2 test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data collection same pre‐ and postintervention.
Knowledge of the allocation adequately prevented(ITS)?	High risk	NOT DONE, subjective outcome measure, not blinded
Incomplete outcome data addressed (ITS) ?	Unclear risk	Not stated whether outcome data collected on all participants.
Free of selected reporting (ITS) ?	Unclear risk	Not stated whether outcome data collected on all participants.
Free of other bias (ITS) ?	High risk	NOT DONE, the only reliable data for analysis are about compliance with the antibiotic policy, which was 80% at baseline. Serious risk of ceiling effect.

Hess 1990.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: receiving cefazolin therapy SETTING: a 719‐bed tertiary‐care medical centre in the USA
Interventions	FORMAT: Interventions: dissemination of guideline; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: cefazolin expenditure per patient day FINANCIAL: savings in drug costs
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	12 months' data pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper, no statistical analysis, and only comparison was between mean (uncontrolled) before and after.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Unclear risk	On page 588 the authors state that "a proportion of these savings can be attributed to a decrease in acquisition cost", but they do not say how much.

Himmelberg 1991.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: physicians in the hospital PARTICIPANTS: patients in the hospital CLINICAL PROBLEM: patients receiving restricted antibiotics SETTING: a tertiary‐care teaching hospital in the USA
Interventions	FORMAT: Interventions: restrictive, removal of restriction Intervention Functions: restriction DELIVERER: specialist physician COMPARISON: 6 months in the restriction period were compared with 6 months after restriction was lifted. DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: number of courses and cost of restricted drugs FINANCIAL: cost of drugs
Notes	FINANCIAL SUPPORT:Funding: commercial, Pfizer Roerig and the Upjohn companies. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Data collected in same months in 2 consecutive years.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Hitti 2012.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the ED PARTICIPANTS: all patients with sepsis in the ED CLINICAL PROBLEM: sepsis SETTING: 1 hospital in Beirut, Lebanon
Interventions	FORMAT, Interventions: structural Intervention Functions: environmental restructuring, antibiotics required for sepsis treatment were stored in an Automated Dispensing Cabinet in the ED instead of having to be ordered from pharmacy DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: time to first antibiotic dose in minutes measured both from arrival in the ED and from ordering the antibiotic
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention is point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Retrospective data collection using the same methods throughout
Knowledge of the allocation adequately prevented(ITS)?	High risk	Data were collected from case records, and allocation was not concealed.
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data reported on all 110 included participants.
Free of selected reporting (ITS) ?	Low risk	Exclusion rates similar pre‐ (13/69) and post‐ (11/65) intervention.
Free of other bias (ITS) ?	High risk	Data only collected for 7 months pre‐ and 8 months postintervention, so secular trends possible.

Hochreiter 2009.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in ICU PARTICIPANTS: all patients with the clinical problem CLINICAL PROBLEM: duration of antibiotic therapy in 110 patients with suspected bacterial infections (57 intervention, 53 control) SETTING: surgical intensive care ward in 1 hospital in Germany
Interventions	FORMAT, Interventions: reminders (circumstantial and physical, procalcitonin‐based decision support algorithm); structural (introduction of procalcitonin testing) Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: department (ICU) physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information provided
Outcomes	PRESCRIBING: Exposure: duration of all antibiotic therapy in days
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: SS has served as consultant and received payments from B.R.A.H.M.S AG for speaking engagements. All other authors declare no conflicts of interest. ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No explanation of randomisation process
Allocation concealment (selection bias)	Unclear risk	Details of allocation process not provided.
Blinding (performance bias and detection bias) All outcomes	High risk	Open trial
Incomplete outcome data (attrition bias) All outcomes	Low risk	Done, Table 1 and text regarding excluded patients
Selective reporting (reporting bias)	Unclear risk	No explicit statement, so selective outcome reporting is possible.
Other bias	Low risk	Done, all biases addressed.
Baseline Outcomes similar?	Unclear risk	No baseline outcome measurement
Free of contamination?	Low risk	Done, procalcitonin results not available for controls.
Baseline characteristics similar?	Low risk	Done, mainly similar (IC days slightly different)

Huber 1982.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: appropriateness of inpatient prescribing of cephalexin SETTING: 1 university hospital in the USA
Interventions	FORMAT, Interventions: restrictive by expert approval and removal Intervention Functions: restriction DELIVERER: pharmacists COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: cephalexin dosing units
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	> 2 years' data pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: no statistical analysis of time series, presented as chart.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Hulgan 2004.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: physicians in the hospital PATIENTS: all patients with clinical problem CLINICAL PROBLEM: use of IV and oral quinolones SETTING: university hospital in the USA
Interventions	FORMAT, Interventions: reminders (circumstantial and physical, computerised decision support system integrated into an existing provider order entry system) Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive use of IV quinolones
Outcomes	PRESCRIBING: Choice: number of orders for oral quinolone FINANCIAL: savings on drug costs in USD
Notes	FINANCIAL SUPPORT: Funding: NIH Training Grant T32 AI 07474‐08 and Vanderbilt Clinical Research Scholar Award K12 RR17697 (TH). Competing Interests: DAT and RAM receive authorship royalties through Vanderbilt University from the commercial distribution of WizOrder. STR has received consulting fees from McKesson Information Solutions, which has licensed WizOrder for commercial distribution. None of the other authors has related disclosures or potential conflicts of interest. ADDITIONAL DATA: email response from authors with additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Objective outcome measure
Analysed appropriately (ITS) ?	Low risk	Done in original paper: segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was increase in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	1 year of data pre‐ and postintervention
Free of other bias (ITS) ?	Low risk	Objective primary outcome, cost analysis adjusted to 2003 prices.

Inaraja 1986.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving antibiotics CLINICAL PROBLEM: patients receiving antibiotics SETTING: 1 447‐bed university hospital in Spain
Interventions	Interventions: educational outreach by review and recommend change; restrictive antibiotic policy but mode of restriction not clear Intervention Functions: enablement, persuasion, restriction DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: cephalosporin use measured with costs as a percentage of cephalosporins plus penicillins plus aminoglycosides
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Only 12 months' data (9 months' pre‐ and 3 months' postintervention), so cannot control for seasonal effects.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Jensen 2011.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in ICUs PARTICIPANTS: All adult patients in ICUs for > 24 hrs CLINICAL PROBLEM: suspected sepsis SETTING: 9 multidisciplinary ICUs across Denmark
Interventions	FORMAT:Interventions: reminders (circumstantial and physical, drug‐escalation algorithm and intensified diagnostics based on daily procalcitonin measurements); structural (rapid procalcitonin testing) Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: deparmental physicians (ICU) COMPARISON: usual care DESIRED CHANGE: increase effective SAMPLE SIZE: yes, total 1200 participants. Details in Appendix 3 1200 participants were randomised and included in the analysis.
Outcomes	PRESCRIBING: Choice: time to first antibiotic dose; number (%) ICU days spent with at least 3 antibiotics CLINICAL: intended 28‐day mortality; unintended (balancing) days in ICU; relative risk of renal impairment
Notes	FINANCIAL SUPPORT: Funding: Danish State, the Lundbeck Foundation, the Toyota Foundation, the A.P. Møller Foundation, the Horboe Foundation, and the Capitol Region of Denmark. Competing Interests: Dr. Jensen received speaker fee and travel reimbursement from B.R.A.H.M.S Diagnostica and an unrestricted grant from the organisation for sample transport and analysis. The remaining authors have not disclosed any potential conflicts of interest. ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Randomisation was performed 1:1 using a computerised algorithm created by the database manager.
Allocation concealment (selection bias)	Low risk	Investigators were masked to assignment before randomisation. Concealed block size, pre‐stratified for site of recruitment, initial Acute Physiology and Chronic Health Evaluation, and age (entered in an encrypted screening form in a password‐protected website)
Blinding (performance bias and detection bias) All outcomes	Low risk	Investigators, treating physicians, and the co‐ordinator were unaware of outcomes during the study.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all randomised participants.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all randomised participants.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT measures only reported for intervention participants.
Baseline characteristics similar?	Low risk	Table 1

Jobson 2015.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians and nurses in the paediatric ED PARTICIPANTS: all children with central lines CLINICAL PROBLEM: time to first antibiotic dose in children with fever SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: audit and feedback at individual and group level; educational meetings, dissemination of educational materials; educational outreach by academic detailing at individual and group level; reminders (circumstantial (on electronic health record), physical (cards attached to computers, weekly email newsletter), and verbal); structural (placing antibiotics in front‐line Pyxis stock) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: departmental physicians COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Choice: % of participants receiving first antibiotic dose within 60 minutes
Notes	FINANCIAL SUPPORT: Funding: no external. Competing Interests: none declared ADDITIONAL DATA: authors provided additional data about intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Statistical process control chart
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Primary outcome was time to first antibiotic dose from patient administration system.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Primary outcome was time to first antibiotic dose from patient administration system.
Incomplete outcome data addressed (ITS) ?	Low risk	Primary outcome was time to first antibiotic dose from patient administration system.
Free of selected reporting (ITS) ?	Low risk	Primary outcome was time to first antibiotic dose from patient administration system.
Free of other bias (ITS) ?	High risk	Only 8 months' pre‐intervention data, so seasonal effects cannot be excluded.

Jump 2012.

Methods	STUDY DESIGN: CITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians and nurses in the hospitals PARTICIPANTS: all patients in the hospitals CLINICAL PROBLEM: patients requiring antibiotics or with suspected Clostridium difficile infection SETTING: 1 long‐term care facility (intervention) and 1 hospital (control) in the USA
Interventions	FORMAT: Interventions: audit and feedback; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: days of therapy with all antibiotics/1000 OBD MICROBIAL: +ve C difficile tests per 1000 OBD
Notes	FINANCIAL SUPPORT: Funding: National Institutes of Health (grants R03‐AG040722 to RLPJ, K23‐DK087919 to PED, and R01‐AI063517 to RAB), Veterans Affairs Merit Review Program, Veterans Integrated Service Network 10 Geriatric Research Education and Clinical Center (VISN 10 GRECC). Competing Interests: RLPJ reports that she has consulted for GOJO and Pfizer and has received grant support ViroPharma. RAB reports that he has consulted for AstraZeneca and has received grant support from AstraZeneca, Ribx, Pfizer, and Steris. CJD reports that he has consulted for BioK, Optimer, and GOJO and has received grant support from ViroPharma, Merck, and Pfizer. All other authors report no conflicts of interest. ADDITIONAL DATA: email with additional data; further information about the intervention is given in Jump 2013. Microbial Risk of Bias: HIGH. Case definition low; Planned intervention low; Other infection control high, no data about infection control other than that the intervention also increased isolation of participants with C difficile infection. Moreover, the intervention discouraged repeat testing of participants with known C difficile infection, which may have biased the microbial outcome.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Kallen 2009.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring therapeutic antibiotics SETTING: 1 community hospital in the USA
Interventions	FORMAT: Intervention: restrictive by removal from all wards Intervention Function: restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease use of fluoroquinolones in order to contain an outbreak of Clostridium difficile infection
Outcomes	PRESCRIBING: Choice: use of fluoroquinolones, DDD/100 OBD MICROBIAL: C difficile infections
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared Microbial Risk of Bias: HIGH, case definition yes, planned intervention no (part of response to outbreak), other infection control measures no (several important changes made at the same time as prescribing intervention)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	No, as this was during an outbreak
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy computer
Free of other bias (ITS) ?	High risk	< 1 year data postintervention, fluoroquinolones reintroduced

Kanwar 2007.

Methods	STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the ED PARTICIPANTS: 518 adult patients CLINICAL PROBLEM: hospital admission diagnosis of CAP SETTING: 1 hospital in the USA
Interventions	FORMAT, Interventions: audit and feedback; financial, institution incentive Intervention Functions: enablement, incentive DELIVERER: Blue Cross‐Blue Shield of Michigan incentive program COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	UNINTENDED CONSEQUENCES: confirmation of admission diagnosis by chest X‐ray, mean antibiotic administration per patient admitted with CAP
Notes	NRSI RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement class of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status reported on all 518 patients 6. Measurement of outcome: Low, outcome measure objective and measured from patient administration system 7. Selection of the reported result: Low, single, prespecified analysis of the intervention‐outcome relationship FINANCIAL SUPPORT: Funding, none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data

Kerremans 2008.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: 1498 patients with bacterial infections (746 intervention, 752 control) CLINICAL PROBLEM: antibiotic use in adult patients with bacterial infections SETTING: 1 university hospital in the Netherlands
Interventions	FORMAT: Intervention: structural (rapid microbiology laboratory testing) Intervention Functions: environmental restructuring DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 1500 participants in total. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: total antibiotic use (average DDDs per patient) CLINICAL: Intended: mortality
Notes	FINANCIAL SUPPORT: Funding: Dutch Association of University Hospitals (‘VAZ‐Doelmatigheidproject’ no. 99207). bioMerieux provided additional funding through an unrestricted grant. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Done, computer‐generated randomisation
Allocation concealment (selection bias)	High risk	No, states concealment was impossible.
Blinding (performance bias and detection bias) All outcomes	High risk	No formal blinding attempted.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Done, Figure 1
Selective reporting (reporting bias)	Low risk	Done, all outcomes reported.
Other bias	Low risk	No other apparent issues
Baseline Outcomes similar?	Unclear risk	No baseline measurement of outcome
Free of contamination?	Low risk	Done
Baseline characteristics similar?	Low risk	Done, Table 1

Kerremans 2009.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: 211 patients with positive blood cultures (93 intervention, 108 control) CLINICAL PROBLEM: antibiotic use in adult patients with bacterial infections SETTING: 1 tertiary‐care university medical centre in the Netherlands
Interventions	FORMAT: Intervention: structural ‐ other (out‐of‐hours blood culture incubator intended to reduce laboratory turnaround time) Intervention Function: environmental restructuring DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: increase effective POWER CALCULATION: no information. In the Discussion, the authors say "our sample size was too small to study the impacts of time to positivity (Gram stain), identification, and susceptibility testing separately on outcome".
Outcomes	PRESCRIBING: Choice: time to first antibiotic regimen change CLINICAL: Intended: mortality and length of stay
Notes	FINANCIAL SUPPORT: Funding: Becton Dickinson provided the outside BACTEC incubator used in this study. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer‐generated list by independent epidemiologist
Allocation concealment (selection bias)	High risk	Allocation not concealed
Blinding (performance bias and detection bias) All outcomes	Unclear risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	1 episode of missing data in each arm of study
Selective reporting (reporting bias)	Low risk	Complete outcomes reported.
Other bias	Low risk
Baseline Outcomes similar?	Low risk	States no significant differences at baseline.
Free of contamination?	Low risk	Rapid reporting only occurred for intervention participants.
Baseline characteristics similar?	Low risk	States no significant differences at baseline.

Khan 2003.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: Clostridium difficile‐associated diarrhoea SETTING: an 800‐bed non‐teaching hospital in the UK
Interventions	FORMAT: no valid prescribing data. Restriction with educational meetings and dissemination of guideline. DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: incidence of C difficile‐associated diarrhoea
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	> 1 year data in each of the 3 phases
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: no statistical analysis, mean cases per quarter compared between periods.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done: "The standard operating procedure for selection and processing stool specimens did not change over the study period. All stool specimens from inpatients with liquid or bloody diarrhoea and those receiving antibiotic therapy were tested for C. difficile toxin. C. difficile toxin was detected by cytotoxic activity on a fibroblast cell line, with specific neutralization by Clostridium sordelli antiserum"
Free of other bias (ITS) ?	High risk	NOT DONE for the intervention that was intended to reduce C difficile infection in Phase 3 Microbial Outcome Risk of Bias: Planned intervention: NOT DONE for unplanned intervention Phase 3 Case definition: DONE C difficile infection; all stool specimens from inpatients with liquid or bloody diarrhoea and those receiving antibiotic therapy were tested for C difficile toxin. Other infection control measures: DONE, well described and same in all 3 phases

Kim 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving therapeutic antibiotics CLINICAL PROBLEM: outbreak of ESBL infections SETTING: 1 hospital in Korea
Interventions	FORMAT: Interventions: audit and feedback; restrictive by expert approval Intervention Functions: enablement, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, use of cephalosporins to contain outbreak of ESBL
Outcomes	PRESCRIBING: use of cephalosporins (DDD/1000 OBD) MICROBIAL: isolates of ESBL and new patients with ESBL infection
Notes	FINANCIAL SUPPORT: Funding: City of Seoul grant #10920 and KICOS project grant (Battelle Institute, Korea University). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias HIGH. Case definition Low, planned intervention High (response to outbreak of ESBL), other infection control Unclear (no detail, and authors state that they did not take this into account in their analysis)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Not in original paper but re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk

Knudsen 2014.

Methods	STUDY DESIGN: CITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians, nurses, and pharmacists in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: the intervention was intended to reduce infections caused by ESBL‐ and AmpC‐producing gram‐negative bacteria SETTING: 1 university hospital (intervention) and 4 additional hospitals (control) in Denmark
Interventions	FORMAT: Interventions: audit and feedback; educational meetings; dissemination of guidelines; educational outreach by review and recommend change; reminders (physical, intranet and pocket guidelines; circumstantial, verbal by pharmacy technicians and infection control nurses) Intervention Functions: education, enablement, environmental restructuring, persuasion. The intervention also included the same components targeted at infection control measures (hand hygiene and isolation). DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: cefuroxime use in DDD/1000 OBD MICROBIAL: cases per 1000 OBD per month
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: the authors provided multiple additional files of information about the intervention, including examples of the feedback newsletters (in Danish). Microbial Risk of Bias: LOW, case definition low, planned intervention low, other infection control measures low
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	The antimicrobial stewardship intervention was simultaneous with an intervention to improve infection control practice (personal protection and isolation).
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computuers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from pharmacy and microbiology computuers
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computuers
Free of selected reporting (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computuers
Free of other bias (ITS) ?	Low risk	> 1 year of data pre‐ and postintervention. Microbial risk of bias low

Kristoffersen 2009.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 210 patients with suspected lower respiratory tract infection (103 intervention, 107 control) CLINICAL PROBLEM: antibiotic consumption and length of stay in patients with suspected lower respiratory tract infections SETTING: 3 hospitals in Denmark
Interventions	FORMAT: Interventions: dissemination of guideline; reminders (circumstantial and physical, decision support algorithm triggered by PCT test result); structural, introduction of PCT testing Intervention Functions: education, enablement, environmental restructuring DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 107 participants in each group. Details in Appendix 3
Outcomes	PRESCRIBING: Choice and exposure: antibiotics prescribed and duration of antibiotic treatment CLINICAL: Balancing: length of stay and mortality
Notes	FINANCIAL SUPPORT: Funding: Danish Medical Research Council and the Danish Lung Association Study ID: NCT00415753, 271‐05‐0765. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer generated
Allocation concealment (selection bias)	Low risk	Concealed until PCT test results available
Blinding (performance bias and detection bias) All outcomes	Low risk	Objective outcome measure: length of stay from routine data system
Incomplete outcome data (attrition bias) All outcomes	Low risk	States that 3 patients died, 2 in PCT and 1 in control
Selective reporting (reporting bias)	Low risk	Objective outcome measure: length of stay from routine data system
Other bias	Low risk	Adequately powered
Baseline Outcomes similar?	Unclear risk	No beseline outcome measures
Free of contamination?	Low risk	PCT results only available for intervention participants.
Baseline characteristics similar?	Low risk	Mostly similar apart from those with cancer (7 in PCT and 0 in control), although this was adjusted for using sensitivity analysis.

Kritchevsky 2008.

Methods	STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM
Participants	PROVIDERS: physicians responsible for antimicrobial prophylaxis PARTICIPANTS: patients undergoing cardiac surgery, hip and knee replacements, and hysterectomy, 44 clusters (hospitals) CLINICAL PROBLEM: Preoperative antimicrobial prophylaxis SETTING: 44 acute care hospitals in the USA
Interventions	FORMAT: Interventions: educational meetings with dissemination of guideline; educational outreach by academic detailing Intervention Functions: education, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective POWER CALCULATION: yes, 40 hospitals sampling 100 cases per measurement period. Details in Appendix 3
Outcomes	PRESCRIBING: Choice and exposure: 5 performance measures of antimicrobial prophylaxis (timing, receipt, duration, selection, and single preoperative dose)
Notes	FINANCIAL SUPPORT: Funding: grant R01 HS11331‐01A1 from the Agency for Healthcare Research and Quality and Centers for Disease Control and Prevention. Competing Interests: none declared ADDITIONAL DATA: authors provided additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computerised random number generator
Allocation concealment (selection bias)	Low risk	By institution
Blinding (performance bias and detection bias) All outcomes	Unclear risk	Does not say if it was blinded or not
Incomplete outcome data (attrition bias) All outcomes	Low risk	Trained data collectors, completeness assured by project staff.
Selective reporting (reporting bias)	Low risk	All outcomes reported.
Other bias	High risk	High risk of selection bias, as hospitals nominated themselves to be included into the study.
Baseline Outcomes similar?	Low risk	See Table 3
Free of contamination?	Low risk	By institution
Baseline characteristics similar?	Low risk	See Table 2

Kumana 2001.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PATIENTS: all patients in the hospital CLINICAL PROBLEM: patients receiving glycopeptides (teicoplanin or vancomycin) SETTING: 1 hospital in Hong Kong
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines Intervention Functions: education, enablement DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: DDD per month of glycopeptides CLINICAL: Balancing: cohort study of patients who died following Staphylococcus aureus bacteraemia
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Done, 32 months' pre‐ and 11 months' postintervention, so secular or seasonal effects unlikely.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before and after) with χ2 test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	11 months' postintervention data, 32 months' pre‐intervention data
Free of other bias (ITS) ?	Low risk	Reliable primary outcome

Lacroix 2014.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: 30 physicians PARTICIPANTS: 271 children with fever (131 intervention, 140 control) CLINICAL PROBLEM: fever without source SETTING: 1 university hospital in Switzerland
Interventions	FORMAT: Interventions: reminders (circumstantial and physical, decision support lab score derived from PCT, C‐reactive protein, and urine dipstick); structural, introduction of PCT testing Intervention Functions: enablement, environmental lab score derived from PCT, C‐reactive protein, and urine dipstick DELIVERER: departmental physicians COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 140 participants taking into account dropouts. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: % patients receiving antibiotics CLINICAL: re‐admission and time to clinical resolution
Notes	FINANCIAL SUPPORT: Funding: commercial, bioMérieux for data management, statistical analysis, and loan of the procalcitonin assay. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Excel‐generated random numbers table
Allocation concealment (selection bias)	Low risk	Sealed envelopes
Blinding (performance bias and detection bias) All outcomes	Low risk	Outcome measured from routine data.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Incomplete outcome data on 3 of 134 control and 4 of 140 intervention children.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all remaining children.
Other bias	Low risk	The trial ended after completion of a sufficient number of children at the expected timing.
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	No lab score released for control children.
Baseline characteristics similar?	Low risk	Table 2

Lafaurie 2012.

Methods	STUDY DESIGN: CITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving antibiotics CLINICAL PROBLEM: use of fluoroquinolones SETTING: 1 university hospital in France (intervention) with control data from 700 hospitals in the Coordinating Centres for Nosocomial Infection Control
Interventions	FORMAT: Interventions: audit and feedback; educational meeting with dissemination of guideline; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: fluoroquinolone use in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH, case definition Low, planned intervention Low, other infection control High, increase in use of alcohol‐based handrub throughout study period
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Low risk for prescribing outcome
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of anaysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	> 1 year of data pre‐ and postintervention

Landgren 1988.

Methods	STUDY DESIGN: CBA Risk of Bias: HIGH
Participants	PROVIDERS: all surgeons at the hospitals PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: patients receiving surgical antibiotic prophylaxis SETTING: 12 hospitals in Australia
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; educational outreach by academic detailing; reminders (physical, posters) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: pharmacist COMPARISON: 6 hospitals were used as control in year 1, then intervention and control hospitals were crossed over in year 2 DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice and exposure: appropriate duration and timing of prophylaxis FINANCIAL: drug cost savings in AUD
Notes	FINANCIAL SUPPORT: Funding: Commonwealth Department of Health. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	CBA; "hospitals were paired being matched as far as possible for type size and surgical load"
Allocation concealment (selection bias)	High risk	Not done, CBA
Blinding (performance bias and detection bias) All outcomes	High risk	Not stated; all hospitals in same Australian state, CBA so not possible
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	No statement
Selective reporting (reporting bias)	Low risk	Objective primary outcome measure on all patients
Other bias	Low risk	No other apparent biases found.
Baseline Outcomes similar?	Low risk	Done, pre‐intervention data for primary outcome similar in intervention and control hospitals.
Free of contamination?	Low risk	Intervention and control sites were different hospitals.
Baseline characteristics similar?	Unclear risk	Only information is about characteristics of hospital (teaching, rural, etc.), no data about case mix and unlikely to change over study period.

Landman 1999.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: university hospital in the USA
Interventions	FORMAT: no valid prescribing outcome data. Restriction. DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: Incidence (new cases per 1000 discharges per month) of ceftazidime‐resistant Klebsiella pneumoniae, MRSA, and cefotaxime‐resistant Acinetobacter species (ITS data)
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias MEDIUM: case definition: Low; planned intervention: Low; infection control practices: High. At the start of the intervention, contact precautions were changed to include patients with Clostridium difficile infection.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Reliable primary outcome
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with t‐test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Unclear risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Unclear risk	Not clear, no information about protocols for clinical sampling or testing
Free of other bias (ITS) ?	High risk	Change in infection control practices at start of intervention

LaRosa 2007.

Methods	STUDY DESIGN: unintended consequences, cross‐sectional and cohort study Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 15,440 patients (cross‐sectional) and 360 patients (cohort) CLINICAL PROBLEM: receiving restricted antibiotics SETTING: 1 hospital in the USA
Interventions	FORMAT, Interventions: restrictive by prior approval Intervention Functions: restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	UNINTENDED CONSEQUENCES: delay in ordering of restricted antibiotics
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status complete for both cross‐sectional and cohort study 6. Measurement of outcome: Low, outcome measures objective and ascertained from patient administration system 7. Selection of the reported result: Low, single analysis of prespecified outcomes FINANCIAL SUPPORT: Funding: Centers for Education and Research on Therapeutics grant (U18‐HS10399) from the Agency for Healthcare Research and Quality (AHRQ), the Mentored Patient‐Oriented Research Career Development Award (K23‐AI‐060887‐01) of the NIH from the National Institute of Allergy and Infectious Diseases, Public Health Service grant (DK‐02987‐01) of the NIH, and an Improving Patient Safety Through Reduction in Medication Errors grant (P01‐HS11530‐01) from the AHRQ. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data

Lautenbach 2003.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians at the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: requiring antibiotic treatment SETTING: 1 university hospital in the USA
Interventions	FORMAT: Intervention: restrictive by expert approval, not clear if there was also removal Intervention Functions: restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: vancomycin use in DDD per 1000 patient days MICROBIAL: proportion of enterococci resistant to vancomycin
Notes	FINANCIAL SUPPORT: Funding: Public Health Service (grant DK‐02987‐01) of the National Institutes of Health (to EL). This study was also supported in part by an Agency for Healthcare Research and Quality Centers for Education and Research on Therapeutics co‐operative agreement (U18‐HS10399). Competing Interests: no information ADDITIONAL DATA: authors provided additional prescribing data to enable segmented regression analysis Microbial Risk of Bias HIGH: Case definition: Low. Planned intervention: High: unplanned intervention in response to emergence of VRE over the previous 3 years. Other infection control measures: Low
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk
Knowledge of the allocation adequately prevented(ITS)?	Low risk
Incomplete outcome data addressed (ITS) ?	Low risk
Free of selected reporting (ITS) ?	Low risk
Free of other bias (ITS) ?	High risk	Microbial outcome risk of bias: HIGH .

Lawes 2012.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: Staphylococcus aureus bacteraemia and use of antibiotics considered to be high risk for Clostridium difficile infection SETTING: 1 university hospital in the UK
Interventions	FORMAT: Interventions: dissemination of new antibiotic policy 3 months before the structural intervention; restrictive: the new antibiotic policy included requirement for expert approval; structural: introduction of universal screening for MRSA Intervention Functions: education, environmental restructuring, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: no valid data for re‐analysis in the paper, but the authors' ARIMA time series analysis includes the effect of the change in antibiotic policy on the microbial outcomes MICROBIAL: S aureus bacteraemias, MRSA, and MSSA
Notes	FINANCIAL SUPPORT: Funding: Scottish government Health Directorate. Competing Interests: IG has received personal and grant financial support from companies manufacturing diagnostics and therapeutics for MRSA. BE has received grant financial support from Novartis. Other authors: none ADDITIONAL DATA: authors provided additional data Microbial Risk of Bias MEDIUM: case definition High, MRSA screening introduced at the same time as change in antibiotic policy, planned intervention Low, other infection control Low for isolation and personal infection control
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	The change in antibiotic policy was 9 months after the introduction of MRSA screening. The authors' analysis suggests an independent effect from the policy change.
Analysed appropriately (ITS) ?	Low risk	ARIMA time series analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Routine patient administration systems
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine patient administration systems
Incomplete outcome data addressed (ITS) ?	Low risk	Routine patient administration systems
Free of selected reporting (ITS) ?	Low risk	Routine patient administration systems
Free of other bias (ITS) ?	Low risk	Other microbial ROB criteria low

Layios 2012.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the ICUs PARTICIPANTS: 389 patients in the ICUs for > 48 h and with PCT measured (211 intervention, 178 control) CLINICAL PROBLEM: duration of antibiotic treatment SETTING: 5 ICUs in 1 university hospital in Belgium
Interventions	FORMAT: Interventions: reminders (circumstantial and physical, decision support algorithm triggered by PCT test result); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: specialist physician (ICU and respiratory) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 250 participants in each group. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: antibiotic consumption as % ICU days and DDD/100 OBD CLINICAL: mortality, length of ICU stay, days on ventilator
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"patients were prospectively randomized", but no information about how
Allocation concealment (selection bias)	Unclear risk	"patients were prospectively randomized", but no information about how
Blinding (performance bias and detection bias) All outcomes	Low risk	Procalcitonin only reported for intervention participants.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all participants.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all participants.
Other bias	High risk	Study did not achieve recruitment required by power calculation.
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	Procalcitonin only reported for intervention participants.
Baseline characteristics similar?	Low risk	Table 1

Lee 1995.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: physicians PATIENTS: a total of 480 patients reviewed during study period CLINICAL PROBLEM: patients receiving ceftriaxone SETTING: a hospital in the USA
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (circumstantial and physical, letters sent to physicians when intervention needed plus posters) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: grams of ceftriaxone and cefotaxime FINANCIAL: cost of intervention (0.5 FTE ID physician and savings on drug costs)
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy computer
Free of other bias (ITS) ?	High risk	> 1 year data pre‐ and postintervention, but only 4 postintervention time points (quarterly data)

Lee 2007.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all staff in the hospital PARTICIPANTS: all patients receiving cephalosporins CLINICAL PROBLEM: high endemic rate of ESBL infections SETTING: 1 university children's hospital in Korea
Interventions	FORMAT: Intervention: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: specialist physicians (paediatric ID) COMPARISON: pre‐intervention DESIRED CHANGE: decrease in use of extended‐spectrum cephalosporins to reduce ESBL infections
Outcomes	PRESCRIBING: Choice: days on target antibiotics/1000 OBD MICROBIAL: ESBL strains as % total isolates
Notes	FINANCIAL SUPPORT: Funding: Wyeth Research. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias LOW: case definition yes, planned intervention yes, stable ESBL for 3 years pre‐intervention, other infection control yes
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Infection control policies unchanged throughout (page 631).
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Decrease
Unlikely to affect data collection (ITS) ?	Low risk	Data about prescribing and microbial outcomes were from routine, electronic data systems.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data about prescribing and microbial outcomes were from routine, electronic data systems.
Incomplete outcome data addressed (ITS) ?	Low risk	Data about prescribing and microbial outcomes were from routine, electronic data systems.
Free of selected reporting (ITS) ?	Low risk	Data about prescribing and microbial outcomes were from routine, electronic data systems.
Free of other bias (ITS) ?	Low risk	4 years' data pre‐ and 3 years' data postintervention, so can account for temporal trends.

Lee 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the units PARTICIPANTS: all patients in the units CLINICAL PROBLEM: requiring therapeutic antibiotics SETTING: internal medicine (2 units) at 1 university hospital in Canada
Interventions	FORMAT: Interventions: audit and feedback; educational meetings (monthly with residents) with dissemination of educational materials Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: DDD/1000 OBD of target antibiotics FINANCIAL: intervention cost and savings (cost of all antibiotics)
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: authors provided additional data about the intervention and for the meta‐regression
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcomes were measured from electronic pharmacy data.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcomes were measured from electronic pharmacy data.
Incomplete outcome data addressed (ITS) ?	Low risk	Outcomes were measured from electronic pharmacy data.
Free of selected reporting (ITS) ?	Low risk	Outcomes were measured from electronic pharmacy data.
Free of other bias (ITS) ?	Low risk	> 12 months' data pre‐ and postintervention

Lesprit 2013.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in medical and surgical wards PARTICIPANTS: 753 patients receiving antibiotics (376 intervention, 377 control) CLINICAL PROBLEM: duration of treatment in patients receiving 1 of the targeted antibiotics for at least 3 days SETTING: 1 university hospital in France
Interventions	FORMAT: Intervention: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 253 participants in each group. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: duration of all antibiotic treatment CLINICAL: Balancing: mortality, ICU admission, new course of antibiotic treatment, length of stay FINANCIAL: intervention cost and savings (supplementary file) MICROBIAL: secondary infection and/or colonisation with multidrug‐resistant bacteria in the 6 months following randomisation
Notes	FINANCIAL SUPPORT:Funding: none. Competing Interests: none declared ADDITIONAL DATA: supplementary file online with data about financial and microbial outcomes, no response from authors to request for additional data Microbial Risk of Bias: case defintion low, planned intervention low, other infection control unclear, no information
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Eligible patients were allocated to either the intervention or the control group using a computer‐generated randomisation list, which was maintained independently of the IDP.
Allocation concealment (selection bias)	Low risk	Concealment of the allocation was maintained, as the physician in charge of the patient and the IDP were involved only after randomisation.
Blinding (performance bias and detection bias) All outcomes	High risk	Blinding was not possible.
Incomplete outcome data (attrition bias) All outcomes	Low risk	No participants were lost to follow‐up.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all participants.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	IDP only visited intervention participants.
Baseline characteristics similar?	Low risk	Table 1

Leverstein‐van Hall 2001.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: Departments of Neurology and Neurosurgery PARTICIPANTS: all patients in the departments CLINICAL PROBLEM: colonisation with gentamicin‐resistant Enterobacteriaceae SETTING: 1 858‐bed university hospital in the Netherlands
Interventions	FORMAT: no valid prescribing data, restriction by expert approval and removal DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: prevalence of gentamicin‐resistant Enterobacteriaceae in weekly screening stool swabs
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	NOT DONE, major changes in infection control 4 weeks before the antibiotic restriction. Separate effect cannot be estimated because no screening before change in infection control.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: no statistical analysis, time series data presented as run chart.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Screening protocol was the same pre‐ and postintervention.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Screening protocol was the same pre‐ and postintervention.
Incomplete outcome data addressed (ITS) ?	Unclear risk	NOT CLEAR, no explicit statement about complete screening samples for all participants
Free of selected reporting (ITS) ?	Unclear risk	NOT CLEAR, no explicit statement about complete screening samples for all participants
Free of other bias (ITS) ?	High risk	Microbial Outcome Risk of Bias Criteria: Case definition: DONE colonisation by screening; Planned intervention: NOT DONE, in response to increase in GRE; Other infection control practices: NOT DONE changes 4 weeks before antibiotic restriction; Isolation: isolation of gentamicin‐resistantEnterobacteriaceae‐positive patients in either side‐rooms or cohorted with other positive patients; IC practices: increase in education plus several new hygiene practices: disposable washing gloves, elbow‐directed soap dispensers; new room‐cleaning protocol. Hygiene was emphasised and more stringent barrier precautions.

Liebowitz 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: incidences of MRSA SETTING: 1 general hospital in the UK
Interventions	FORMAT: Intervention: educational meetings with dissemination of guideline; reminders, verbal on rounds Intervention Function: education, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate
Outcomes	PRESCRIBING: Choice: DDDs per 1000 OBD each month MICROBIAL: Episodes of MRSA blood isolates per 1000 OBD each month
Notes	FINANCIAL SUPPORT: Funding: unrestricted educational grant from Wyeth. Competing Interests: LDL received honoraria for lectures from Bayer and Bard. ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB HIGH; case definition Low, planned intervention Low, other infection control High, no information about infection control other than screening for MRSA
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	18 months' pre‐ and 15 months' postintervention data
Analysed appropriately (ITS) ?	Low risk	Segmented regresssion analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis is point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Pharmacy data used pre‐ and postintervention.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Pharmacy data used pre‐ and postintervention.
Incomplete outcome data addressed (ITS) ?	Low risk	Pharmacy data used pre‐ and postintervention.
Free of selected reporting (ITS) ?	Low risk	Pharmacy data used pre‐ and postintervention.
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Linkin 2007.

Methods	STUDY DESIGN: unintended consequences, cohort study Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 200 patients CLINICAL PROBLEM: requests for restricted antibiotic to the Antimicrobial Stewardship Program SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: restrictive Intervention Functions: restriction DELIVERER: AMT COMPARISON: patients with appropriate vs inappropriate requests DESIRED CHANGE: decrease excessive
Outcomes	UNINTENDED CONSEQUENCES: accuracy of laboratory and clinical information provided in calls to the Antimicrobial Stewardship Program
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, the effects of inaccurate communication and each of the potential confounders on the risk of inappropriate antimicrobial recommendations were evaluated in bivariable analyses 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, antimicrobial recommendations were evaluated for appropriateness by a 3‐person panel of infectious diseases experts blinded to the accuracy of information communicated during the Antimicrobial Stewardship Program call 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: High, panelists could not agree on appropriateness of treatment for 37 patients. Outcome data complete for the 163 included patients 6. Measurement of outcome: Low, outcome measures objective and ascertained from patient administration system 7. Selection of the reported result: High, multiple secondary analyses were performed using the main study outcome FINANCIAL SUPPORT: Funding: National Institutes of Health, Agency for Healthcare Research and Quality, and University of Pennsylvania. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data

Liu 2013.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: Department of Emergency Medicine, ICU staff PARTICIPANTS: adults (age > 18) with sepsis CLINICAL PROBLEM: sepsis without 7 exclusion criteria (cultures positive with Pseudomonas aeruginosa,Acinetobacter baumannii,Mycobacterium tuberculosis or any fungi, viral or parasitic infection, chronic localised inflammation, antibacterial therapy for > 48 h, immunosuppression, cancer, or refusal to consent) SETTING: ICU in 1 university hospital in China
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm triggered by measurement of PCT); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Exposure: duration of all antibiotic treatment CLINICAL: Balancing: 28‐day mortality, length of hospital stay, length of ICU stay, recurrence within 28 days
Notes	Translated from Chinese FUNDING: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Random number table method
Allocation concealment (selection bias)	Unclear risk	No information about concealment
Blinding (performance bias and detection bias) All outcomes	Unclear risk	No information about blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome reported on all participants.
Selective reporting (reporting bias)	Low risk	Outcome reported on all participants.
Other bias	High risk	The study had 7 exclusion criteria that are not all clearly defined, so there is a high risk of selection bias, especially as allocation was probably not concealed.
Baseline Outcomes similar?	Unclear risk	No data about baseline outcomes
Free of contamination?	Low risk	PCT results only for intervention participants.
Baseline characteristics similar?	Low risk	Table 1, age, gender, APACHE score, comorbidities

Long 2014.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians PARTICIPANTS: 216 consecutive patients hospitalised with exacerbations of acute asthma CLINICAL PROBLEM: antibiotic treatment of acute asthma SETTING: 1 university hospital in China
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm triggered by measurement of PCT); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physicians (Internal and Geriatric Medicine) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 90 participants per group. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: % treated with antibiotics CLINICAL: Balancing: length of hospital stay; clinical, laboratory, and spirometry outcomes at discharge; and results of spirometry at the 12‐month follow‐up examination, as well as the results of the Asthma Control Test
Notes	FINANCIAL SUPPORT: Funding: Shanghai Fifth People’s Hospital Science Foundation and Minhang District Natural Science Foundation of Shanghai. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Allocation to either intervention was conducted according to computer‐generated random numbers produced by an independent statistician."
Allocation concealment (selection bias)	Low risk	"After randomization, an opaque, sealed, sequentially numbered envelope containing the PCT or control protocol was prepared for each subject according to group assignment"
Blinding (performance bias and detection bias) All outcomes	High risk	Blinding was not possible.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all 180 randomised participants.
Selective reporting (reporting bias)	Low risk	Antibiotic use reported for all 180 participants.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	Procalcitonin only reported on intervention participants.
Baseline characteristics similar?	Low risk	Tables 1 and 2

Madaras‐Kelly 2006.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all prescribers and staff PATIENTS: all inpatients CLINICAL PROBLEM: patients receiving antibiotic treatment and patients with MRSA infections SETTING: university‐affiliated veterans hospital in the USA
Interventions	FORMAT: Interventions: educational meetings, in‐service training sessions with dissemination of guideline; reminders (circumstantial, electronic, triggered by prescribing target drugs) Intervention Functions: education, enablement, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: change in use of levofloxacin, ciprofloxacin, and other antibiotics MICROBIAL: MRSA infection rate (number/1000 OBD)
Notes	FINANCIAL SUPPORT: This article is the result of work supported with resources and the use of facilities at the Boise Veterans Affairs Medical Center, and is partially funded by an unrestricted educational grant from Wyeth Pharmaceuticals. Conflict of interest: no information ADDITIONAL DATA: email response from authors but no additional data Microbial ROB: MEDIUM. Case definition Low, Planned intervention Low, Other infection control High, prescribing intervention coincident with infection control interventions
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Data collected for 11 months postintervention. Season included as a variable in the model, and summer found to be associated with lower MRSA infection rate. Coincident with infection control intervention for norovirus outbreak, infection control variables included in the model and significantly associated with lower MRSA rate.
Analysed appropriately (ITS) ?	Low risk	Done in original paper: segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Unclear risk	Not clear, no information about protocols for sampling or testing for MRSA over the study period
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Objective data about MRSA
Incomplete outcome data addressed (ITS) ?	Low risk	Identification of cases was the same in the pre‐ and postintervention phases.
Free of selected reporting (ITS) ?	Low risk	In addition to the primary outcome of MRSA infections, the figure shows percentage of MRSA for all Staphylococcus aureus isolates with a reduction coincident with the intervention.
Free of other bias (ITS) ?	High risk	NOT DONE data are MRSA infection rates in 6‐month time periods based on very small numbers of cases (80 cases in 3½ years). Microbial Outcome Risk of Bias: Case definition: MRSA infection. Screening for nosocomial infections was performed through daily review of hospital admissions and discharges, intravenous antibiotic use by patients admitted to the emergency department, and laboratory reports with case confirmation by review of medical records. “An infection was assumed to be caused by MRSA if cultures of blood, intravenous line, sputum, urine, tissue, or stool obtained at the time of symptom development yielded MRSA.” Planned intervention: YES. Intervention introduced in July 2003 in response to May 2003 SHEA recommendations that institutions where MRSA is endemic should consider limiting the use of broad‐spectrum antibiotics, especially fluoroquinolones. Other infection control: NOT DONE. Antibiotic intervention coincident with environmental decontamination and hand hygiene campaign because of norovirus outbreak. Data about some infection control variables showed no change after start of intervention.

Magedanz 2012.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: physicians in the hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: antibiotic use in cardiology hospital, primary target fluorquinolone use SETTING: 1 cardiology hospital in Brazil
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: Intervention 1 ID physician (2 h per day), Intervention 2 AMT (physician plus pharmacist, 4 h per day) COMPARISON: usual care DESIRED CHANGE: reduce inappropriate
Outcomes	PRESCRIBING: Choice: monthly consumption (DDDs/100 OBD) of antibiotics, primary target fluoroquinolones FINANCIAL: hours of time to implement the intervention
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	States in discussion that most changes not related to any other external factor.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention is point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from electronic pharmacy records
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from electronic pharmacy records
Incomplete outcome data addressed (ITS) ?	Low risk	Data from electronic pharmacy records
Free of selected reporting (ITS) ?	Low risk	Data from electronic pharmacy records
Free of other bias (ITS) ?	Low risk	> 12 months' data in each of the 3 study phases

Maravic‐Stojkovic 2011.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in cardiac surgery PARTICIPANTS: 205 patients undergoing cardiac surgery CLINICAL PROBLEM: antibiotic treatment after surgery SETTING: 1 university hospital in Serbia
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm triggered by measurement of PCT); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physicians (ICU and cardiology) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: unclear, target effect size decrease from 45% of antibiotic use in the standard group to 22% in the procalcitonin group, but no data about sample size
Outcomes	PRESCRIBING: Exposure: % treated with antibiotics CLINICAL: ICU stays, hospital stay, rehospitalisation, incidence of infections, severe non‐infection complications, and mortality rate with 1‐year follow‐up FINANCIAL: cost of antibiotics and PCT tests
Notes	FINANCIAL SUPPORT: no information provided ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer‐generated randomisation scheme
Allocation concealment (selection bias)	Unclear risk	No information about concealment
Blinding (performance bias and detection bias) All outcomes	High risk	Blinding not possible
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all 205 participants.
Selective reporting (reporting bias)	Low risk	Antibiotic treatment reported on all participants.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT only measured on intervention participants.
Baseline characteristics similar?	Low risk	Tables 1 and 2

Marwick 2013.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in medical and surgical wards PARTICIPANTS: all patients in medical and surgical wards CLINICAL PROBLEM: suspected sepsis (systemic inflammatory response and clinical suspicion of infection) SETTING: 1 university hospital in Scotland
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; reminders (physical, posters in the wards and monthly email to doctors) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Choice: time to first antibiotic dose
Notes	FINANCIAL SUPPORT: Funding: Scottish Government Chief Scientist Office (CSO) Clinical Academic Training Fellowship (CAF/07/06). Competing Interests: salary costs for 2 investigators from CSO, no others declared ADDITIONAL DATA: email response from authors to request for additional data with additional detail from a PhD thesis
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	There was a national intervention (Scottish Patient Safety Program) that included reducing time to rescue of deteriorating patients throughout the pre‐ and postintervention phases.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Objective primary outcome measure (time to first antibiotic dose) collected by single person (CM).
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Objective primary outcome measure (time to first antibiotic dose) collected by single person (CM).
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data collected on all participants.
Free of selected reporting (ITS) ?	Low risk	Outcome data collected on all participants.
Free of other bias (ITS) ?	Low risk	Data collected over winter months in pre‐ and postintervention period.

Masia 2008.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 278 patients receiving antibiotics, 146 intervention, 132 control CLINICAL PROBLEM: prescription of target antibiotics SETTING: 1 university hospital in Spain
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 140 participants in each group
Outcomes	PRESCRIBING: Choice: use of target drugs in DDD CLINICAL: length of stay, mortality, re‐admissions
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Eligible prescriptions were allocated daily to either the intervention or the control group using a computer‐generated randomisation list.
Allocation concealment (selection bias)	Low risk	Concealment of allocation was pharmacy controlled. Instruction in allocation concealment was provided.
Blinding (performance bias and detection bias) All outcomes	High risk	Blinding was not possible.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome data reported on all randomised participants.
Selective reporting (reporting bias)	Low risk	Outcome data reported on all randomised participants.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	High risk	The authors say: "To minimize contamination bias, that is, any change in antibiotic prescription practice in the control group, only the infectious diseases physicians and hospital pharmacists were informed about the implementation of the program." However, they were placing written recommendations in case notes for intervention patients, and physicians caring for those patients would also be caring for control patients.
Baseline characteristics similar?	Low risk	Table 1

May 2000.

Methods	STUDY DESIGN: Controlled ITS Risk of bias: MEDIUM
Participants	PROVIDERS: staff of Trauma & Burns ICU (TBICU), Medical ICU (MICU), and Surgical ICU (SICU) PATIENTS: all patients in these ICUs CLINICAL PROBLEM: adults needing intensive care SETTING: single > 500‐bed university hospital in the USA
Interventions	FORMAT: Intervention: dissemination of guideline Intervention Function: education DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of vancomycin, 3rd‐generation cephalosporins, and piperacillin tazobactam per 1000 patient days MICROBIAL: MRSA infections and VRE infections per 1000 patient days
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB HIGH: Case definition: Low. Planned intervention: High for intervention ward (response to increasing VRE in previous 2 years). However, steady increase not an outbreak and VRE data presented for other wards with no intervention. Other infection control: High, no information about isolation or infection control practices before or after the intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Only 9 months' data pre‐intervention, so secular/seasonal effects possible. No information about infection control practices before or after the intervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: χ2 test, uncontrolled before‐after with Poisson regression analysis of VRE rates.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, objective outcome measure
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Objective outcome measure, VRE infections
Incomplete outcome data addressed (ITS) ?	Low risk	Done, objective outcome measure
Free of selected reporting (ITS) ?	Unclear risk	Not clear, no information about protocol for sampling or testing over study period
Free of other bias (ITS) ?	Low risk	>1 year of data pre‐ and post‐intervention

McElnay 1995.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients in hospital CLINICAL PROBLEM: all patients receiving antibiotics SETTING: 370‐bed District General Hospital in the UK
Interventions	FORMAT: Interventions: educational meetings and dissemination of new antibiotic policy; educational outreach by academic detailing, "education of junior medical staff on the rationale behind the antibiotic selection was also carried out by clinical pharmacists" (p208); restrictive by compulsory order form and removal Intervention Functions: education, persuasion, restriction DELIVERER: department physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: dosage units of target antibiotic FINANCIAL: expenditure on antibiotics
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	12 months' data pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	Antibiotic costs were adjusted to 1989 prices.

McGowan 1976.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: single university hospital in USA
Interventions	FORMAT: Intervention: restrictive by expert approval and probably by review and make change Intervention Function: restriction DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: grams of chloramphenicol (thousands), data are also presented for other drugs (ampicillin, nafcillin, and cloxacillin)
Notes	FINANCIAL SUPPORT: Funding: grants 5R01‐A1‐23, 2T01‐AJ‐08, and IT01‐Ai‐447 from the National Institute of Allergy and Infectious Diseases. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Data over 8 years, 4 years pre‐ and 4 years postintervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

McLaughlin 2005.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: staff from 12 medical wards PATIENTS: all patients in the wards CLINICAL PROBLEM: adults requiring IV antibiotic therapy SETTING: single university hospital in the UK
Interventions	FORMAT: Interventions: educational meetings with dissemination of protocol for IV to oral switch; educational outreach by academic detailing; reminders (circumstantial, sticker in charts of patients receiving IV antibiotics and physical, posters in wards and at nursing stations) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: modification of existing management (faster switch from IV to oral administration of antibiotics)
Outcomes	PRESCRIBING: Choice: appropriateness of timing of IV to oral switch
Notes	FINANCIAL SUPPORT: Funding: Greater Glasgow Health Board. Competing Interests: no information ADDITIONAL DATA: authors provided additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Not done, data were only collected for 4 weeks before and after the intervention, so secular changes could have accounted for any differences.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with χ2 test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Unclear risk	Not stated
Knowledge of the allocation adequately prevented(ITS)?	High risk
Incomplete outcome data addressed (ITS) ?	Unclear risk	No information about reliability or completeness of primary outcome
Free of selected reporting (ITS) ?	Unclear risk	No information about reliability or completeness of primary outcome
Free of other bias (ITS) ?	High risk	Only 4 weekly time points pre‐ and postintervention

McNulty 1997.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the elderly care unit PARTICIPANTS: all patients in the elderly care unit CLINICAL PROBLEM: Clostridium difficile in the elderly care unit SETTING: elderly care unit in 1 District General Hospital (non‐teaching) in the UK
Interventions	FORMAT: Interventions: dissemination of new antibiotic policy; restrictive by removal and by review and make change Intervention Functions: education, enablement, environmental restructuring, persuasion, restriction DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: monthly cost of cefuroxime (ITS data) MICROBIAL: cases of CDI per month (ITS data)
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB HIGH: Case definition: Low, CDI, definition unchanged during the study periods. Unplanned intervention: High, antibiotic restriction was implemented in response to increasing cases of CDI in the preceding 7 months despite increased infection control. Other infection control measures: High, changes to environmental cleaning and reminders about hand hygiene implemented 3 months before the start of intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk
Analysed appropriately (ITS) ?	Low risk
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk
Knowledge of the allocation adequately prevented(ITS)?	Low risk
Incomplete outcome data addressed (ITS) ?	Low risk
Free of selected reporting (ITS) ?	Low risk
Free of other bias (ITS) ?	Low risk

Mercer 1999.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: physicians PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: patients receiving ceftriaxone SETTING: a 360‐bed community hospital in the USA
Interventions	FORMAT: Interventions: dissemination of guidelines; educational outreach by academic detailing; educational outreach by review and recommend change; reminders (physical, posters in clinical areas); restrictive by compulsory order form, expert approval required, removal and review and make change Intervention Functions: education, environmental restructuring, restriction DELIVERER: specialist physician (ID) COMPARISON: usual care DESIRED CHANGE: reduction in established management (reduction in antibiotic costs)
Outcomes	PRESCRIBING: Choice: cost of antibiotics (USD) as an indicator of use COSTS: cost of antibiotics
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Full year before and after
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	Antibiotic costs were adjusted to 1995 prices and excluded ancillary or administrative charges.

Meyer 1993.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients receiving antibiotics SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: restrictive by expert approval required Intervention Functions: restriction COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of ceftazidime, imipenem, and ceftriaxone reported as number of approvals for these drugs MICROBIAL: incidence of ceftazidime‐resistant Klebsiella pneumoniae as the rate per 1000 average daily census
Notes	FINANCIAL SUPPORT: Funding: BMA Medical Foundation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB: HIGH Case definition Low, Unplanned intervention High, Other infection control High
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Infection control intervention simultaneous with antibiotic intervention. 14 months' pre‐ and 11 months' postintervention, so secular change unlikely.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: run chart with no statistical analysis.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	High risk	Pre‐intervention data were incomplete.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period. Criteria for sampling and testing were unchanged over the study period.
Free of other bias (ITS) ?	High risk	NOT DONE. Microbial Outcome Risk of Bias Criteria: Planned intervention: NOT DONE, unplanned intervention. Case definition: DONE, microbial outcome was colonisation by surveillance screening. Clinical infection was diagnosed by CDC definition but not used as an outcome. Infection or colonisation by case note review. Other infection control measures: NOT DONE, barrier precautions were instituted on colonised and infected patients at the same time that ceftazidime restriction was implemented.

Meyer 2007.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: physicians in the neurosurgical ICU PARTICIPANTS: patients with pneumonia CLINICAL PROBLEM: antibiotic treatment for pneumonia in neurosurgical ICU SETTING: neurosurgical ICU in 1 hospital in Germany
Interventions	FORMAT: Interventions: educational meeting with neurosurgeons and dissemination of guideline Intervention Functions: education DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, in the new guideline the duration of antibiotic therapy for nosocomial pneumonia was reduced from 14 to 7 days, while for community‐acquired pneumonia the period fell from 10 to 5 days
Outcomes	PRESCRIBING: Exposure: total antibiotic use and cost/1000 OBD FINANCIAL: changes in total antibiotic cost
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome from pharmacy database pre‐ and postintervention.
Knowledge of the allocation adequately prevented(ITS)?	Unclear risk	No mention of blinding
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome from pharmacy database pre‐ and postintervention.
Free of selected reporting (ITS) ?	Low risk	Outcome from pharmacy database pre‐ and postintervention.
Free of other bias (ITS) ?	Low risk	> 1 year of data pre‐ and postintervention

Meyer 2009.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all patients in an adult surgical ICU PARTICIPANTS: all staff in the ICU CLINICAL PROBLEM: use of 3rd‐generation cephalosporins for treatment and prophylaxis of specific infections plus duration of prophylaxis for fractures SETTING: 1 surgical ICU in a teaching hospital in Germany
Interventions	FORMAT: Interventions: dissemination of guidelines and educational meetings in departments of surgery and anaesthesiology Intervention Functions: education DELIVERER: multidisciplinary AMT COMPARISON: pre‐intervention outcomes DESIRED CHANGE: reduction in use of cephalosporins and resistance in gram‐negative bacteria
Outcomes	PRESCRIBING: Choice: use of cephalosporins in DDD/1000 OBD MICROBIAL: resistance to cephalosporins and piperacillin in gram‐negative bacteria isolated from clinical and surveillance cultures
Notes	FINANCIAL SUPPORT: Funding: Federal Ministry of Education and Research (01Kl 9907). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial ROB: HIGH Case definition Low; Planned intervention Low; Other infection control Unclear, no clear information about isolation or personal‐protection policies
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	Data for > 2 years' pre‐ and postintervention, so secular trends accounted for.

Meyer 2010.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the ICU PARTICIPANTS: all patients with clinical problem CLINICAL PROBLEM: reducing length of antibiotic prophylaxis for cerebrospinal shunts SETTING: ICU department in 1 teaching hospital in Germany
Interventions	FORMAT: Intervention: educational meeting and dissemination of new policy. In autumn 2003, a comprehensive teaching session on antibiotic prophylaxis in cerebrospinal shunts was organised by the infection control and neurosurgery teams. This resulted in a revised recommendation of single‐shot prophylaxis with cefuroxime for shunt catheters, beginning in January 2004. Prior to implementation of this recommendation, cefuroxime was administered for the whole duration of external cerebrospinal fluid drainage, which could be up to 2 to 3 weeks. Intervention Functions: education, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, shorten duration of prophylaxis
Outcomes	PRESCRIBING: Exposure: total antibiotic use in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: Federal Ministry of Education and Research (01Kl 9907). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Says they could not control for changes over time and that an antimicrobial stewardship programme was implemented
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharamacy computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharamacy computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharamacy computers
Free of selected reporting (ITS) ?	Low risk	Data from pharamacy computers
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Micek 2004.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: ICU physicians PATIENTS: 302 adults in the ICU (154 intervention, 148 control) CLINICAL PROBLEM: VAP requiring antibiotics SETTING: single ICU in a teaching hospital in the USA
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Exposure: duration of all antibiotic therapy
Notes	FINANCIAL SUPPORT: Funding: part commercial, Barnes‐Jewish Hospital Foundation and an unrestricted grant from Elan Pharmaceuticals. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Patients were randomly assigned", but no details of how the sequence was generated
Allocation concealment (selection bias)	Unclear risk	Not stated
Blinding (performance bias and detection bias) All outcomes	High risk	Not possible
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome data were missing from 4 (2.6%) patients in the intervention group and 8 (5.4%) patients in the control group.
Selective reporting (reporting bias)	Low risk	Done, outcomes were obtained from routine data systems.
Other bias	High risk	The policy was only implemented at weekends or on holidays when 1 of the 2 investigators was available in the hospital.
Baseline Outcomes similar?	Unclear risk	No data about duration of therapy before the intervention
Free of contamination?	High risk	Physicians managing patients in the control group would have seen reminders for the intervention group.
Baseline characteristics similar?	Low risk	Table 1

Mittal 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the Department of Paediatrics PARTICIPANTS: all children < 2 years old with bronchiolitis CLINICAL PROBLEM: antibiotic use as part of a new Clinical Practice Guideline to improve management of bronchiolitis SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: audit and feedback, educational meeting with dissemination of guideline; reminders (verbal (on rounds, so may have been circumstantial) and physical (pocket‐size guideline, screensavers)) Intervention Functions: education, enablement, persuasion DELIVERER: departmental physicians (paediatrics and respiratory) COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: % treated with antibiotics CLINICAL: length of stay, re‐admission
Notes	FINANCIAL SUPPORT: Funding: no external. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Antibiotic use was 1 of 10 recommendations in the guideline; the other 9 would have impacted on clinical outcomes.
Analysed appropriately (ITS) ?	Low risk	Statistical process control charts
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	All outcome data from hospital patient administration system
Knowledge of the allocation adequately prevented(ITS)?	Low risk	All outcome data from hospital patient administration system
Incomplete outcome data addressed (ITS) ?	Low risk	All outcome data from hospital patient administration system
Free of selected reporting (ITS) ?	Low risk	All outcome data from hospital patient administration system
Free of other bias (ITS) ?	Low risk	Data collected over 3 winters, 1 pre‐ and 2 postintervention.

Mol 2005.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: physicians in the Department of Internal Medicine PATIENTS: all patients in the wards CLINICAL PROBLEM: receiving antibiotic therapy SETTING: 1 university hospital in the Netherlands
Interventions	FORMAT: 1st Intervention: audit and feedback; educational meetings with dissemination of guideline 1st Intervention Functions: education, enablement 2nd Intervention: audit and feedback; educational meetings with dissemination of guideline; educational outreach by academic detailing 2nd Intervention Functions: education, enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % compliance with guideline; antibiotic cost FINANCIAL: antibiotic cost
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Done in original paper: segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was increase in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Data collection method was same throughout study.
Knowledge of the allocation adequately prevented(ITS)?	High risk	Subjective outcome without blinded assessment
Incomplete outcome data addressed (ITS) ?	Unclear risk	Not stated whether compliance was assessed in all patients.
Free of selected reporting (ITS) ?	Unclear risk	Not stated whether compliance was assessed in all patients.
Free of other bias (ITS) ?	Low risk	The kappa value for the primary outcome measure was 0.71, which is below the level set by EPOC, but for the reasons given in the text we feel is adequate for assessment of compliance with an antibiotic guideline. Drug costs were adjusted to April 2001 prices.

Newland 2012.

Methods	STUDY DESIGN: CITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients in children's hospital CLINICAL PROBLEM: inappropriate use of antimicrobials; a group of broad‐spectrum, or “select”, antibiotics 2 calendar days after they were initiated by the clinician SETTING: 1 children's hospital in the USA (intervention) with data from 25 similar hospitals of the Child Health Corporation of America as control
Interventions	FORMAT: Interventions: educational outreach by review and recommend change. NB the authors describe their intervention as "audit and feedback", but there was no feedback of data over time about progress to goal, just review with feedback about individual patients Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: total antibiotic use (days of therapy/1000 patient days)
Notes	FINANCIAL SUPPORT: Funding: Agency for Healtcare Quality and Reseach (grant U18‐HS10399). Competing Interests: none declared ADDITIONAL DATA: authors provided additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Unclear, there were some infection control initiatives running.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Yes, point of analysis is point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Yes, data collection was the same pre‐ and postintervention.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Yes, objective outcomes
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data, so could assume complete.
Free of selected reporting (ITS) ?	Low risk	Yes, all relevant outcomes reported.
Free of other bias (ITS) ?	Low risk	Yes, all biases addressed.

Nobre 2008.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the ICU PARTICIPANTS: 282 patients with suspected sepsis, 79 randomised (39 intervention, 40 control) CLINICAL PROBLEM: duration of antibiotic treatment in patients with sepsis SETTING: 1 ICU in 1 university hospital in Switzerland
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physician (ICU) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, a total of at least 66 participants. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: duration of treatment in days CLINICAL: Balancing: mortality, relapse of infection, length of ICU stay, length of hospital stay
Notes	FINANCIAL SUPPORT: Funding: commercial B.R.A.H.M.S AG (USD 50,000). Competing Interests: 2 authors received speaker honoraria from B.R.A.H.M.S AG. ADDITIONAL DATA: online supplementary file with addtional infromation about stopping rules in PCT group. No response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The randomisation was performed using a computer‐based random number generation.
Allocation concealment (selection bias)	Low risk	Allocation was issued using opaque, sealed, numbered envelopes.
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk
Selective reporting (reporting bias)	High risk	8/39 (20%) patients excluded from intervention versus 3/40 (7%) from control; 4 patients excluded from intervention for "complicated infections", which is likely to have biased the results on duration of antibiotic treatment.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT only measured for intervention group.
Baseline characteristics similar?	Low risk	Table 1

Nuila 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in hospital PARTICIPANTS: all patients receiving antibiotics CLINICAL PROBLEM: reduce cases of Clostridium difficile‐associated disease in hospital by restricting use of parenteral antibiotics SETTING: 1 teaching hospital in the USA
Interventions	FORMAT: no valid prescribing data. Restriction and educational outreach ‐ review and recommend change DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce inappropriate
Outcomes	MICROBIAL: incidence of Clostridium difficile‐associated disease
Notes	FINANCIAL SUPPORT: Funding: Merit Review Funding and Department of Veterans Affairs. Competing Interests: none declared ADDITIONAL DATA: email from authors but no additional data Microbial ROB: MEDIUM: Case definition Low, Planned intervention Low, Other infection control High
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	MRSA control programme introduced simultaneously.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Routine data from microbiology computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data from microbiology computer
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data from microbiology computer
Free of selected reporting (ITS) ?	Low risk	Routine data from microbiology computer
Free of other bias (ITS) ?	High risk	Only 6 months' data postintervention Microbial Outcome Risk of Bias Criteria: Case definition: DONE, CDC definition of C difficile. Planned intervention: DONE. Other infection control measures: NOT DONE, MRSA control programme introduced simultaneously.

Oliveira 2013.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians PARTICIPANTS: 355 ICU patients assessed for inclusion, 94 patients randomised CLINICAL PROBLEM: 94 patients with suspected sepsis randomised (49 intervention, 45 control) SETTING: 1 university hospital ICU in Brazil
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: specialist physician (Infectious Diseases) COMPARISON: usual care, patients monitored with C‐reactive protein DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 58 participants per group. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: duration of treatment in days CLINICAL: mortality, recurrence of infection, ICU length of stay, hospital length of stay, nosocomial infection
Notes	FINANCIAL SUPPORT: Funding: Minas Gerais Research Foundation (Fundação de Amparo à Pesquisa do Estado de Minas Gerais). Competing Interests: 1 author received payment for lectures from bioMérieux. No others declared. ADDITIONAL DATA: online Microsoft Word document with additional information about the criteria for stopping antibiotics, no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Randomisation was performed using a table of computer‐generated random numbers.
Allocation concealment (selection bias)	Low risk	Sealed, opaque envelopes were used for the randomisation.
Blinding (performance bias and detection bias) All outcomes	High risk	Not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	1 patient excluded from intervention and 2 from control. Outcomes measured on all other randomised participants.
Selective reporting (reporting bias)	Low risk	Duration of antibiotics measured from patient administration system.
Other bias	Unclear risk	"Patients showing reduction in SOFA and no sign of active infection were to receive no more than 7 days of antibiotic therapy. We used the biomarker‐guided protocols to further reduce this duration (i.e., to less than seven days)". This suggests that the ID physicians imposed a ceiling of 7 days' treatment for these patients for both intervention and control groups. Study did not achieve required recruitment.
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT only measured for intervention group.
Baseline characteristics similar?	Low risk	Table 1

Oosterheert 2005.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: hospital physicians PATIENTS: inpatients with LRTI, 107 randomised (55 intervention, 52 control) CLINICAL PROBLEM: admitted to hospital for treatment of LRTI SETTING: 2 Dutch hospitals
Interventions	FORMAT: Interventions: educational meetings; dissemination of written information about study procedures, test characteristics discussed and results from previous studies; structural, rapid laboratory testing (PCR) for viral and atypical bacterial pathogens Intervention Functions: education, environmental restructuring DELIVERER: specialist physician (Medical Microbiology) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, a total of 100 patients. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: % patients treated CLINICAL: mortality, median duration of antibiotic treatment FINANCIAL: cost of hospitalisation, all diagnostic and treatment costs
Notes	FINANCIAL SUPPORT: Funding: Association of Academic Hospitals and the Dutch Health Insurance Council (grant 01233). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Patients were randomly allocated ... by means of a computer generated table"
Allocation concealment (selection bias)	High risk	Allocation by investigators
Blinding (performance bias and detection bias) All outcomes	High risk	Investigators were not blinded to patient randomisation.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Reported on all 107 patients
Selective reporting (reporting bias)	Low risk
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	Test data only reported for intervention patients.
Baseline characteristics similar?	High risk	"slightly more patients in the intervention group had received previous antibiotic treatment ": 42% vs 23%, which is not "slighly more"

Ostrowsky 2014.

Methods	STUDY DESIGN: CITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospitals PARTICIPANTS: all patients in the hospitals CLINICAL PROBLEM: reduce use of antibiotics considered high risk for Clostridium difficile infection SETTING: 10 hospitals in the USA, 6 intervention and 4 control
Interventions	FORMAT: Interventions: educational meetings (6 hospitals), dissemination of algorithms (3 hospitals), educational outreach by review and recommend change (2 hospitals), restrictive automatic stop order (1 hospital), unspecified "hospital wide restriction" (3 hospitals). NB the authors describe the intervention in 2 hospitals as "audit and feedback", but there was no feedback of data over time about progress to goal, just review with feedback about individual patients. Intervention Functions: education, enablement, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive. Each intervention hospital did a case control study to identify high‐risk antibiotics; these were piperacillin tazobactam (6 hospitals), fluoroquinolones (5 hospitals), or cefepime (2 hospitals).
Outcomes	PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD and in days of therapy MICROBIAL: C difficile infection (cases per 10,000 OBD)
Notes	FINANCIAL SUPPORT: Funding: Agency for Healthcare Research and Quality; US Department of Health and Human Services. Competing Interests: none declared ADDITIONAL DATA: email response from authors with additional data about the intervention used by each of the 6 intervention hospitals
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Outcome data from pharmacy and microbiology computers
Free of other bias (ITS) ?	High risk	Intervention targets and intervention design were different in each of the 6 hospitals. Microbial ROB MEDIUM: case definition low, planned intervention low, other infection control UNCLEAR

Ozkaya 2009.

Methods	STUDY DESIGN: NRT Risk of Bias: HIGH
Participants	PROVIDERS: all staff in the ED PARTICIPANTS: all children with influenza‐like illness CLINICAL PROBLEM: reduction in antibiotic prescribing for influenza SETTING: 1 university hospital in Turkey
Interventions	FORMAT: Intervention: structural, rapid laboratory test for influenza Intervention Function: environmental restructuring DELIVERER: specialist physicians, Department of Paediatrics COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: % children prescribed antibiotics
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: email response from authors but no additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Does not say how groups were allocated
Allocation concealment (selection bias)	Unclear risk	Says there was blinding but unclear who was blinded.
Blinding (performance bias and detection bias) All outcomes	Unclear risk	Says there was blinding but unclear who was blinded.
Incomplete outcome data (attrition bias) All outcomes	Low risk	All included
Selective reporting (reporting bias)	Low risk	Yes, all outcomes reported.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No baseline outcome data
Free of contamination?	High risk	Within same ward
Baseline characteristics similar?	Low risk	Yes, Table 1

Palmay 2014.

Methods	STUDY DESIGN: cluster RCT, stepped wedge, service level Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in 6 hospital services PARTICIPANTS: all patients in 6 hospital services, 6 clusters (services) CLINICAL PROBLEM: use of targeted antibiotics (carbapenems (ertapenem, meropenem), piperacillin‐tazobactam, 3rd‐generation cephalosporins (ceftazidime, ceftriaxone), fluoroquinolones (ciprofloxacin, levofloxacin, moxifloxacin), and intravenous vancomycin) SETTING: 1 university hospital in Canada, 6 services: Neurosurgery, Orthopaedics, Nephrology, General Internal Medicine, Cardiology, General Surgery/Trauma
Interventions	FORMAT: Interventions: educational outreach by review and recommend change; reminders (circumstantial, physical, written recommendation on each patient reviewed) NB the authors describe their intervention as "audit and feedback", but there was no feedback of data over time about progress to goal, just review with feedback about individual patients. Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Choice: use of target antibiotics in days of therapy/1000 OBD MICROBIAL: Clostridium difficile infection and infection with antibiotic‐resistant organisms FINANCIAL: time required to implement the intervention in critical‐care wards is described in Elligson 2012a.
Notes	FINANCIAL SUPPORT: Funding: Ontario Ministry of Health and Canadian Institutes of Health Research. Competing Interests: none declared ADDITIONAL DATA: email response with additional details about the intervention from authors including Elligson 2012a describing the design and cost of implementing the intervention in critical‐care wards Microbial Risk of Bias: MEDIUM: case definition low, unplanned intervention low, other infection control UNCLEAR
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	The order of implementation of the intervention on the 6 clinical services was determined by random number generation performed by a statistician uninvolved in daily stewardship activities.
Allocation concealment (selection bias)	Low risk	Following a 6‐month control period during which none of the services received antimicrobial stewardship (1 May 2010 to 31 October 2011), the intervention was introduced to each additional service at 1‐month intervals, beginning on 1 November 2010. By 1 April 2011, clinical rollout was complete.
Blinding (performance bias and detection bias) All outcomes	High risk	Blinding was not possible.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Prescribing outcome data were from pharmacy computer.
Selective reporting (reporting bias)	Low risk	Prescribing outcome data were from pharmacy computer.
Other bias	Low risk	Unit of analysis was service, and clustering was included in the model. "Negative binomial regression, accounting for clustering at the level of service using random effects as well as for secular and seasonal trends, was used to compare overall targeted antimicrobial utilization in the control and intervention periods for the analysis involving patients qualifying for the stewardship intervention as well as the analysis of all admitted patients. The unit of analysis was each service’s mean monthly targeted days of therapy count. The covariates included in these multivariable models were study period, study month (as a continuous variable), and season"
Baseline Outcomes similar?	Low risk	Table 4
Free of contamination?	High risk	Contamination could have occurred during the rollout of intervention over 6 months.
Baseline characteristics similar?	Low risk

Parienti 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving antibiotics in the hospital CLINICAL PROBLEM: use of fluoroquinolones; the aim of the study was to assess the effect of removing restriction SETTING: 1 university hospital in France
Interventions	FORMAT: no reliable prescribing data. The intervention was removal of restriction, but only 1 prescribing outcome data point during restriction and 3 after restriction lifted. DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: monthly MRSA rate (%)
Notes	FINANCIAL SUPPORT:Funding: Centre Hospitalier Universitaire de Caen and the French Health Ministry (Programme Hospitalier de Recherche Clinique National). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	MRSA data from microbiology computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	MRSA data from microbiology computer
Incomplete outcome data addressed (ITS) ?	Low risk	MRSA data from microbiology computer
Free of selected reporting (ITS) ?	Low risk	MRSA data from microbiology computer
Free of other bias (ITS) ?	Low risk	MICROBIAL RISK OF BIAS: case defiinition Low, planned intervention Low, other infection control Low, use of alchohol‐based hand rub (ABHR) unchanged during period of fluoroquinolone restriction (2001‐2) and for 3 years after restriction lifted (2003‐5). Data are also presented for a further 6 years of increased use of ABHR (2006‐11).

Parikh 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all paediatric physicians in the hospitals PARTICIPANTS: children aged 28 days to 2 years CLINICAL PROBLEM: antibiotic use in children with a primary diagnosis of acute bronchiolitis SETTING: 41 hospitals in the USA
Interventions	FORMAT: Intervention: publication of American Academy of Pediatricians (AAP) bronchiolitis guidelines Intervention Functions: education but no information about how the guidelines were disseminated DELIVERER: departmental physicians (Pediatrics) COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: % children treated with antibiotics
Notes	FINANCIAL SUPPORT: Funding: Academic Pediatric Association. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data. AAP 2006 Bronchiolitis Guidelines downloaded.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from patient administration systems
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from patient administration systems
Incomplete outcome data addressed (ITS) ?	Low risk	Data from patient administration systems
Free of selected reporting (ITS) ?	Low risk	Data from patient administration systems
Free of other bias (ITS) ?	Low risk	Monthly data points for 20 months' pre‐ and 60 months' postintervention. "Guideline published in October 2006. Study phases: preguideline (November 2004 to March 2005), postguideline early (November 2007 to March 2008), and postguideline late (November 2011 to March 2012). These time periods were selected for the unadjusted analysis because they represent 3 bronchiolitis seasons, before and after guideline publication; the 2006 to 2007 season was not included because this is the year the guideline was published and was a period of distribution and assimilation. For the adjusted segmented regression analysis, publication of the guidelines, October 2006, was considered the event point."

Patel 1989.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: single hospital in the UK
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change; reminders (physical and verbal, posters and intervention promoted at weekly ward meetings) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: expenditure on oral co‐amoxiclav
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Only 5 months' pre‐intervention data, so secular changes possible.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Paul 2006.

Methods	STUDY DESIGN: cluster RCT, service level Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PATIENTS: all patients in the 3 hospitals (intervention 8 wards with 1245 patients, 297 with microbiologically documented infections; control 7 wards with 1081 patients, 273 with microbiologically documented infections), 15 clusters (wards) CLINICAL PROBLEM: antibiotic prescribing SETTING: 3 hospitals in 3 countries: Israel, Germany, and Italy
Interventions	FORMAT: Intervention: reminders (circumstantial, triggered by prescription of antibiotics); structural, computer decision support system Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 1500 patients with microbiologically documented infections. Details in Appendix 3
Outcomes	PRESCRIBING: Choice: appropriate antibiotic treatments COST: Costs, which included the estimated ecological cost of inappropriate antibiotic treatment CLINICAL: Balancing: length of stay, 30‐day mortality
Notes	FINANCIAL SUPPORT: Funding: EU Fifth Framework, Information Society Technologies, IST‐9999‐11459. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Wards were randomly allocated ... by drawing a random code from a closed opaque box"
Allocation concealment (selection bias)	High risk	Allocation could not be concealed.
Blinding (performance bias and detection bias) All outcomes	Low risk	Primary outcome was measured by the CDSS.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all patients.
Selective reporting (reporting bias)	Low risk	The primary outcome was objective, based on whether or not the prescriber selected one of the CDSS top 3 recommendations.
Other bias	High risk	Trial was underpowered for the primary outcome measure. Adjustment of drug costs for changes in prices not necessary because the intervention lasted only 6 months.
Baseline Outcomes similar?	Low risk	Table 1, cohort study before trial
Free of contamination?	Low risk	Only intervention wards had CDSS.
Baseline characteristics similar?	Low risk

Pear 1994.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: single university hospital in the USA
Interventions	FORMAT: restrictive, no valid prescribing data DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	MICROBIAL: cases of CDAD per month (ITS data). Prevalence of clindamycin‐resistant Clostridium difficile
Notes	FINANCIAL SUPPORT: none
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Enough data to account for seasonal variation, and infection control measures did not change over study period.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: run chart with no statistical analysis.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	High risk	Not done, the method of detection of C difficile toxin changed from cell culture assay in the first 4 years of the study to a latex test in the final year (5 months after the start of clindamycin restriction).
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	High risk	Not done, change in method of testing for C difficile during the study period (see case definition).
Free of other bias (ITS) ?	High risk	Microbial Outcome Risk of Bias Criteria: Case definition: NOT DONE Infection: diarrhoea with positive assay for C difficile cytotoxin and antibiotic therapy within the previous 60 days. However, the method of detection of toxin changed from cell culture assay in the first 4 years of the study to a latex test in the final year (5 months after the start of clindamycin restriction). Planned intervention: NOT DONE Response to an outbreak of CDAD starting 12 months before restriction. Other infection control, isolation, and IC practices: DONE Infection control measures were identical in the year before and after the start of clindamycin restriction. Hospital staff education and increased availability of gloves and improvement of environmental hygiene were implemented a year before restriction of clindamycin with no apparent impact on the frequency of new cases.

Perez 2003.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: physicians, surgeons, paediatricians, obstetricians‐gynaecologists, and intensivists PARTICIPANTS: adults and children with normal renal function CLINICAL PROBLEM: inappropriate prescribing of antibiotics (specifically in relation to intervals between doses of aminoglycosides and 1st‐ and 3rd‐generation cephalosporins for Intervention 1 and timing of surgical prophylaxis for Intervention 2) SETTING: university hospital in Colombia
Interventions	FORMAT: Interventions:Intervention 1: reminders (posters, not circumstantial); educational meetings and dissemination of guidelines; restrictive by expert approval. Intervention 2: reminder (circumstantial, on blood pressure cuffs in operating theatre); educational meetings and dissemination of guidelines Intervention Functions:Intervention 1: education, environmental restructuring, persuasion. Intervention 2: education, enablement, environmental restructuring, persuasion DELIVERER: pharmacists COMPARISON: usual care DESIRED CHANGE: Intervention 1: increase effective; Intervention 2: decrease excessive
Outcomes	PRESCRIBING: Choice: reduction in incidence of incorrect antibiotic prescriptions (dosing intervals and timing of surgical prophylaxis).
Notes	FINANCIAL SUPPORT: Funding: International Clinical Epidemiology Network (INCLEN, grant #1004‐97‐6501) and by Pontificia Universidad Javeriana (grant #12‐24‐01‐ 31). Competing Interests: no information
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	> 1 year data pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Done in original paper: ARIMA analysis, selected in preference to segmented regression analysis because of nonlinear outcome data.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Peto 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the surgical ICU PARTICIPANTS: adult patients in surgical ICU (excluding general surgical and medical) CLINICAL PROBLEM: excessive antibiotic use SETTING: surgical ICU in a university hospital in Hungary
Interventions	FORMAT: Interventions: educational outreach by review and recommend change; restrictive by expert approval Intervention Functions: enablement, persuasion, restriction DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: total antibiotic consumption (DDD per 100 patient days)
Notes	FINANCIAL SUPPORT: no information
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Routine data from pharmacy
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Routine data from pharmacy
Incomplete outcome data addressed (ITS) ?	Low risk	Routine data from pharmacy
Free of selected reporting (ITS) ?	Low risk	Routine data from pharmacy
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Petrikkos 2007.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients requiring antibiotics CLINICAL PROBLEM: decrease use of cephalosporins SETTING: 1 university hospital in Greece
Interventions	FORMAT: Intervention: restrictive by expert approval Intervention Function: restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of cephalosporins in DDD/100 OBD MICROBIAL: % ESBL‐producing Klebsiella pneumoniae and Escherichia coli
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition Low, planned intervention Low, other infection control Unclear, no data about other infection control measures
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Unclear risk	1 year (6 x 2‐monthly time points) pre‐ and postintervention

Pires 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all prescribers in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients receiving carbapenems SETTING: 1 teaching hospital in Brazil
Interventions	FORMAT: Intervention: restriction by removal from availability in the hospital Intervention Function: restriction DELIVERER: Infection Control Committee COMPARISON: pre‐intervention DESIRED CHANGE: reduction in use of targeted carbapenems and in resistance
Outcomes	PRESCRIBING: Choice: use of target antibiotics MICROBIAL: carbapenem resistance in Pseudomonas aeruginosa
Notes	FINANCIAL SUPPORT: Funding: Fundo de Incentivo a Pesquisa e Eventos, Hospital de Clinicas de Port Alegre. Competing Interests: none declared
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	No outbreak or other changes coincident with intervention
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	Data for 18 months' pre‐ and 3 years' postintervention

Po 2012.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: reduce linezolid use SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions:Intervention 1: educational mettings or dissemination of educational materials. Intervention 2: reminders, structural, circumstantial ‐ computerised physician order entry system (CPOE) and educational meetings or dissemination of educational materials Intervention Functions:Intervention 1: education. Intervention 2: education, enablement, environmental restructuring DELIVERER: specialist physician COMPARISON: usual care DESIRED CHANGE: reduce inappropriate
Outcomes	PRESCRIBING: Choice: linezolid use (DDD per 1000 patient days)
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Yes, reports on all likely influencing interventions.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Yes, the point of analysis is the point of the intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Yes, pharmacy data used both pre‐ and postintervention.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome is objective.
Incomplete outcome data addressed (ITS) ?	Low risk	Yes, pharmacy data, so should be complete.
Free of selected reporting (ITS) ?	Low risk	Yes, all outcomes reported.
Free of other bias (ITS) ?	High risk	< 1 year data for phases 1 and 2

Poehling 2006.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: doctors in the ED PARTICIPANTS: children with influenza‐like illness CLINICAL PROBLEM: decrease antibiotic prescribing for influenza SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: structural, rapid influenza testing Intervention Functions: environmental restructuring DELIVERER: specialist physicians, Department of Pediatrics COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: % children treated
Notes	FINANCIAL SUPPORT: Funding: New Vaccine Surveillance Network and Robert Wood Johnson Generalist Physicians Faculty Scholars Program. Competing Interests: none declared
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Random number generator
Allocation concealment (selection bias)	Unclear risk	Does not say, but possibly not due to nature of study
Blinding (performance bias and detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	From records, outcomes on all included children
Selective reporting (reporting bias)	Low risk	From records, outcomes on all included children
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No baseline outcomes taken.
Free of contamination?	Low risk	Influenza testing only on children in intervention group
Baseline characteristics similar?	Low risk	Table 1

Popovski 2015.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients with intra‐abdominal infections CLINICAL PROBLEM: decrease use of ciprofloxacin for empirical treatment SETTING: 1 university hospital in Canada
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; reminders (physical, posters, and on intranet) Intervention Functions: education, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of ciprofloxacin in DDD/1000 OBD CLINICAL: mortality, re‐admission (cohort data)
Notes	FINANCIAL SUPPORT: Funding: commercial Merck, Pfizer, Astellas, and the Medbuy Corporation. Hamilton Health Sciences Foundation (Jack Hirsh Fellowship). Competing Interests: 1 author received honoraria from Merck and Astellas for lectures. All other authors: none to declare ADDITIONAL DATA: email response from authors with guideline and additional data about the intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Outcome data from pharmacy computer
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Price 2010.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: requiring antibiotic treatment or prophylaxis SETTING: 1 university hospital in the UK
Interventions	FORMAT: Interventions: dissemination of guidelines; restrictive by removal and expert approval Intervention Functions: education, restriction Note that the published paper says: "The policy was widely disseminated in the hospital but no specific measures were put in place to enforce compliance". However, the antibiotic policy provided by the authors says: “Cephalosporins and fluoroquinolones. These agents will NOT be ward stock on any general medical or surgical wards – continuation of therapy beyond 24 hours (in Medicine) and single dose prophylaxis (in Surgery) requires consultant review, prescription by consultant and discussion with Micro ID”. DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of cephalosporins and quinolones (combined) in DDD/1000 OBD MICROBIAL: Clostridium difficile infection
Notes	FINANCIAL SUPPORT: Funding: part commercial, Optimer Pharmaceuticals and US Department of Veterans Affairs. Competing Interests: 1 author declared multiple commercial sources of research funding and held patents relevant to C difficile infection licensed to ViroPharma ADDITIONAL INFORMATION: authors provided the 2008 version of the hospital antibiotic policy, which included details about the restrictions on use of target drugs Microbial Risk of Bias MEDIUM: case definition yes, planned intervention yes, infection control no (a cohorting ward was introduced at the same time as the antibiotic policy)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	For prescrbiing outcome
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Pharmacy computer
Free of other bias (ITS) ?	High risk	1 year data pre‐ and postintervention Microbial Risk of Bias: cohorting introduced at the same time as prescribing intervention.

Pulcini 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the Medical ICU PARTICIPANTS: all patients in the ICU CLINICAL PROBLEM: receiving antibiotics for 24 h to 96 h SETTING: 1 university hospital in France
Interventions	FORMAT: Interventions: audit and feedback; educational outreach by academic detailing; reminders (physical, circumstantial, stickers placed in notes of patients receiving target antibiotics) Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: departmental physician (ICU consultant) COMPARISON: usual care DESIRED CHANGE: increase appropriate
Outcomes	PRESCRIBING: Choice: % appropriate treatment
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL INFORMATION: email from authors with additional information about intervention. The intervention design is described in more detail in Pulcini 2008.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk
Unlikely to affect data collection (ITS) ?	Low risk	Primary outcome was appropriateness of treatment at 24 to 96 hours, which was the same in pre‐ and postintervention period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Dual data entry, the ICU consultant was blinded to study period, although the ID physician was not.
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data on all participants
Free of selected reporting (ITS) ?	Low risk	No evidence of selective reporting
Free of other bias (ITS) ?	High risk	< 1 year of data (25 weeks) in the pre‐ and postintervention phases

Qu 2012.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the ICU PARTICIPANTS: 71 patients with with confirmed severe acute pancreatitis CLINICAL PROBLEM: PCT for guiding duration of antibiotic therapy SETTING: 1 hospital in China
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: department physician (ICU) COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: duration of all antibiotic treatment CLINICAL: Balancing: mortality and length of stay FINANCIAL: cost of hospitalisation, but no information about cost of intervention
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Says it was randomised, but no further information
Allocation concealment (selection bias)	Unclear risk	No information
Blinding (performance bias and detection bias) All outcomes	Unclear risk	No mention of blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes on all 71 participants
Selective reporting (reporting bias)	Low risk
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No information
Free of contamination?	Low risk	PCT results only reported for intervention participants.
Baseline characteristics similar?	Low risk	Yes, Table 1

Rattanaumpawan 2010.

Methods	STUDY DESIGN: NRT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 953 hospitalised adults (1028 prescriptions) CLINICAL PROBLEM: receiving treatment with piperacillin/tazobactam, imipenem, and meropenem SETTING: 1 hospital in Thailand
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduce excessive
Outcomes	PRESCRIBING: Choice: use of target antibiotics CLINICAL: Balancing: mortality, length of stay FINANCIAL: cost of target antibiotics and all antibiotics. No information about cost of intervention
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: email from authors but no additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	By hospital number, even number in last digit received intervention
Allocation concealment (selection bias)	High risk	Not concealed
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Complete data reported.
Selective reporting (reporting bias)	High risk	Some outcomes (favourable clinical outcome, death from infection) subject to selective outcome reporting. No discussion of why there was a significant difference for death because of infection but no difference in the % of patients alive on discharge from hospital.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	Baseline frequency of inappropriate treatment was 50% in January 2007, but no information about risk of inappropriate treatment in the control group in August 2007.
Free of contamination?	High risk	Invervention and control participants were in the same hospital, and physicians were likely to have patients in both groups.
Baseline characteristics similar?	Low risk	Table 1

Rattanaumpawan 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: adult patients receiving antibiotics CLINICAL PROBLEM: unnecessary double coverage for infection with anaerobic bacteria SETTING: 1 hospital in Thailand
Interventions	FORMAT: Interventions: restrictive by prior approval, the intervention was removal of this restriction Intervention Functions: restriction DELIVERER: AMT COMPARISON: unnecessary treatment before and after removal of the restriction DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: cumulative incidence of unnecessary treatment in DDD/admission
Notes	FINANCIAL SUPPORT: Funding: National Institutes of Health grant K24‐AI080942. Competing Interests: 1 author received research support from Merck, Ortho‐McNeil, Cubist, and AstraZeneca. ADDITIONAL DATA: email response from authors but no additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	1 year of data pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	High risk	Primary outcome (unnecessary DACT) not objective.
Knowledge of the allocation adequately prevented(ITS)?	High risk	Primary outcome (unnecessary DACT) not objective.
Incomplete outcome data addressed (ITS) ?	High risk	Figure 1 includes 4 months with no unnecessary DACT, but it is not clear whether this was because there was no DACT or because all DACT was necessary. With the exception of July 2008, these months had relatively high use of ampicillin/sulbactam and metronidazole, so suggests they missed some DACT patients.
Free of selected reporting (ITS) ?	Unclear risk	Not clear if outcome was reported on all patients.
Free of other bias (ITS) ?	Low risk

Richards 2003.

Methods	STUDY DESIGN:  ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients except ICU, ER, ID CLINICAL PROBLEM: receiving treatment with target antibiotics SETTING: 1 university hospital in Australia
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; reminders (circumstantial and physical, on computer order form when prescribing antibiotics); restrictive by compulsory order form, expert approval, and removal Intervention Functions: education, enablement, environmental restructuring, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: reduction in established management (reduction in use of target drugs)
Outcomes	PRESCRIBING: Choice: Primary: use of cefotaxime or ceftriaxone Secondary: use of other antibiotics: gentamicin, benzyl penicillin, carbapenems, piperacillin, ticarcillin, and ciprofloxacin FINANCIAL: cost of intervention
Notes	FINANCIAL SUPPORT: Funding: Royal Melbourne Hospital. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	8 months data pre‐intervention, 15 months postintervention, not enough to adjust for seasonal variation
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with Kruskal‐Wallis test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Richardson 2000.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital. Number, age, and time since qualification NOT CLEAR. 3 intensive care units, 3 general medical, and 1 general surgical PARTICIPANTS: a total of 618 episodes of vancomycin use (220 pre‐ and 398 postintervention). Number of patients, age, gender, and ethnicity NOT CLEAR. CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: single tertiary‐care teaching hospital in the USA with 150 acute care and 90 long‐term care beds
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: enablement, persuasion COMPARISON: data for 3 months in the previous year (April, August, and January) DESIRED CHANGE: decrease excessive (reduction in inappropriate use of vancomycin with the aim of reducing prevalence of VRE infections)
Outcomes	PRESCRIBING: Choice: % episodes of vancomycin use deemed inappropriate
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Data only collected for 3 months pre‐ and 6 months postintervention, so secular/seasonal changes possible.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Unclear risk	The reliability of the assessment of appropriate vancomycin use was not reported.
Knowledge of the allocation adequately prevented(ITS)?	High risk	Retrospective assessment of appropriateness without concealment of study phase.
Incomplete outcome data addressed (ITS) ?	High risk	Assessment of appropriateness from retrospective assessment of all patients treated in 1 month but only done every 4 to 6 months.
Free of selected reporting (ITS) ?	Unclear risk	Not clear, data were only collected intermittently.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Ross 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all paediatricians in the hospitals PARTICIPANTS: children with community‐acquired pneumonia CLINICAL PROBLEM: increase use of guideline‐recommended antibiotics SETTING: 38 hospitals in the USA
Interventions	FORMAT: Intervention: dissemination of national guidelines Intervention Function: education DELIVERER: specialist physicians COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % patients treated with guideline‐recommended antibiotics
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: 1 author has received research funding from Merck and Cubist and has served as a consultant for Merck, Pfizer, Astellas Pharma, and Cubist, and 3 authors have received research funding from Pfizer. ADDITIONAL DATA: email from authors with no additional data. Paediatric infectious diseases guidelines available online (cid.oxfordjournals.org/content/early/2011/08/30/cid.cir531.full)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from Pediatric Health Information System
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from Pediatric Health Information System
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from Pediatric Health Information System
Free of selected reporting (ITS) ?	Low risk	Outcome data from Pediatric Health Information System
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Saizy‐Callaert 2003.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: single 600‐bed university hospital in France
Interventions	FORMAT: Interventions: educational meetings and dissemination of protocol; reminders (physical, pocket‐size guideline); restrictive by compulsory order form and expert approval Intervention Functions: education, environmental restructuring, restriction COMPARISON: data for 3 years after implementation of the programme DESIRED CHANGE: reduce excessive
Outcomes	PRESCRIBING: Choice: anti‐infective expenditure per hospital patient
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	4 years' data pre‐ and 3 years' data postintervention, so enough data to account for seasonal change
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with Fisher's exact test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	High risk	There is no information about change in price of antibiotics over the study period.
Free of selected reporting (ITS) ?	Unclear risk	The intervention was targeted at specific antibiotics, but no information is provided about their use or cost.
Free of other bias (ITS) ?	Unclear risk	No adjustment of antibiotic costs for change in price, so change in price of antibiotics (rather than change in use) over the study period may have been responsible for reduction in cost per patient over the study period. No data about number of admissions pre‐intervention.

Salama 1996.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: requiring antibiotic therapy SETTING: 1 university hospital in Canada
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines; educational outreach by academic detailing; reminders (circumstantial, physical, and verbal: newsletters, posters, pocket charts, educational rounds, and triggered by prescribing of target drugs); reminders (physical); restrictive by compulsory order form plus automatic 3‐day stop order for all antibiotics and review and make change (therapeutic substitution of selected drugs) Intervention Functions: education, enablement, environmental restructuring, persuasion, restriction DESIRED CHANGE: reduction in vancomycin and ceftazidime use
Outcomes	PRESCRIBING: Choice: vancomycin and ceftazidime use in units, antibiotic cost as a percentage of total drug cost
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	> 12 months' data pre‐ and postintervention, enough to account for seasonal change
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Schnoor 2010.

Methods	STUDY DESIGN: cluster RCT, hospital level Risk of Bias: HIGH
Participants	PROVIDERS: doctors managing patients with community‐acquired pneumonia PARTICIPANTS: 623 patients with community‐acquired pneumonia (275 intervention, 348 control), 8 clusters (hospitals) CLINICAL PROBLEM: community‐acquired pneumonia SETTING: 8 hospitals in Germany
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guideline; reminders (physical, posters, electronic and pocket versions of guideline) Intervention Functions: education, enablement, environmental restructuring DESIRED CHANGE: increase in compliance of initial treatment with guideline recommendation and decrease in duration of treatment POWER CALCULATION: no information
Outcomes	PRESCRIBING: Choice: % guideline compliant for initial treatment CLINICAL: Balancing: mortality, length of stay
Notes	FINANCIAL SUPPORT: Funding: German Medical Assembly grant 06‐69 and German Federal Ministry of Education and Research grant 01K10103‐105. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Computer, by hospital
Allocation concealment (selection bias)	Low risk
Blinding (performance bias and detection bias) All outcomes	Unclear risk	Not clear who collected outcome data or whether they were blinded to allocation
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	All outcome data given as %, so unclear if some patients were missing.
Selective reporting (reporting bias)	Unclear risk	No details of data collection
Other bias	High risk	Intervention period (1 April 2007 to 29 February 2008) was different than control period (1 September 2006 to 28 February 2007).
Baseline Outcomes similar?	High risk	Duration of inpatient antibiotic at baseline was appropriate in only 47% intervention (versus 57% control).
Free of contamination?	Low risk	Randomised by site
Baseline characteristics similar?	High risk	75% inpatients in control group versus 50% for intervention, also fewer CURB 0 and more CURB 3

Schouten 2007.

Methods	STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians PARTICIPANTS: 827 patients with lower respiratory tract infection (before intervention, 212 intervention, 166 control; after intervention, 276 intervention, 166 control). 6 clusters (hospitals) CLINICAL PROBLEM: patients with lower respiratory tract infection SETTING: 6 hospitals in the Netherlands
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guideline; educational outreach by academic detailing; reminders (physical, desktop on computers, and pocket card) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive (choice and streamlining) and increase effective (timeliness) POWER CALCULATION: no information
Outcomes	PRESCRIBING: Choice: % patients compliant with guideline for selected drug, timing (within 4 h of presentation), switching from IV to oral and streamlining CLINICAL: Balancing: mortality, length of stay
Notes	FINANCIAL SUPPORT: Funding: The Netherlands Organisation for Health Research and Development (Zon/Mw; 2300.0024). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Blinded researcher coin flip, hospital level
Allocation concealment (selection bias)	Low risk	Allocation at hospital level
Blinding (performance bias and detection bias) All outcomes	Unclear risk	No information
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome data for all patients
Selective reporting (reporting bias)	Unclear risk	All relevant outcomes reported.
Other bias	Low risk
Baseline Outcomes similar?	Low risk	Table 3, also pair‐matched clusters for important variables
Free of contamination?	Low risk	Allocation at hospital level
Baseline characteristics similar?	Low risk	No clinically relevant differences

Schroeder 2009.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the ICU PARTICIPANTS: all patients with sepsis in the ICU CLINICAL PROBLEM: receiving antibiotics for suspected intra‐abdominal sepsis SETTING: 1 university hospital in Germany
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physicians COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Exposure: duration of antibiotic treatment (days) CLINICAL: Balancing: length of hospital stay
Notes	FINANCIAL SUPPORT: Funding: none declared. Competing Interests: 1 author had speaking engagements for B.R.A.H.M.S AG. ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No information
Allocation concealment (selection bias)	Unclear risk	No information
Blinding (performance bias and detection bias) All outcomes	High risk	Not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all patients.
Selective reporting (reporting bias)	Low risk	Outcomes reported on all patients.
Other bias	High risk	Only 27 of 125 screened patients were randomised.
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT only measured for intervention patients.
Baseline characteristics similar?	Low risk	Table 1

Schuetz 2009.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians PARTICIPANTS: 1381 patients with lower respiratory tract infection randomised (687 intervention, 694 control), 6 clusters (hospitals) CLINICAL PROBLEM: lower respiratory tract infection SETTING: 6 hospitals in Switzerland
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physician (respiratory) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 1002 participants total. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: % patients treated
Notes	FINANCIAL SUPPORT: Funding: grant SNF 3200BO‐116177/1 from the Swiss National Science Foundation and contributions from santésuisse and the Gottfried und Julia Bangerter‐Rhyner Foundation and participating hospitals. B.R.A.H.M.S Inc, the major manufacturer of the procalcitonin assay, provided all assay‐related material, Kryptor machines if not already available on site, and kits and maintenance required for 10,000 measurements related to the study. Competing Interests: 3 authors received support from B.R.A.H.M.S Inc to attend meetings and fulfil speaking engagements, and 1 author served as a consultant and received research support from B.R.A.H.M.S Inc. ADDITIONAL DATA: authors provided additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Prespecified, computer‐generated randomisation list
Allocation concealment (selection bias)	Low risk	Centralised, password‐protected website
Blinding (performance bias and detection bias) All outcomes	Low risk	Password‐protected website with instructions for PCT and control groups
Incomplete outcome data (attrition bias) All outcomes	Low risk	Only 1 of 1381 patients lost to follow‐up; 16 (2%) patients in PCT group and 6 (1%) patients in control group withdrew.
Selective reporting (reporting bias)	Low risk	> 95% surviving patients completed 30‐day interview.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No baseline outcome data
Free of contamination?	High risk	The study was conducted in 6 hospitals, but patients in each hospital were in both intervention and control groups.
Baseline characteristics similar?	Low risk	Table 1

Schwann 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians PARTICIPANTS: all patients undergoing elective surgery requiring antibiotic prophylaxis CLINICAL PROBLEM: timing of first dose of antibiotic SETTING: 1 hospital network in the USA
Interventions	FORMAT: Interventions: reminders (circumstantial and physical, point‐of care electronic prompt (triggered by operating room admission)); reminders (physical); restrictive; structural Intervention Functions: enablement, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: % patients with first dose administered within 1 hour of incision CLINICAL: Intended: surgical‐site infection
Notes	FINANCIAL SUPPORT: Funding: Lehigh Valley Hospital Network and Allentown Anesthesia Associates. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Statistical process control charts
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Electronic data for prescribing
Knowledge of the allocation adequately prevented(ITS)?	Unclear risk	Infection control personnel were blinded for assessment of wound infection, unsure about compliance data.
Incomplete outcome data addressed (ITS) ?	Low risk	Electronic data for prescribing
Free of selected reporting (ITS) ?	Low risk	Electronic data for prescribing
Free of other bias (ITS) ?	Low risk	1 year data pre‐ and postintervention

Schwartz 2007.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the long‐term care facility PARTICIPANTS: all patients receiving antibiotics CLINICAL PROBLEM: patients receiving antimicrobials SETTING: 1 hospital
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines and treatment algorithms; reminders (physical, pocket guidelines) The guideline has 16 algorithms for management of clinical problems (fever, leukocytosis, confusion, diarrhoea) and common infections in older people. Intervention Functions: education, environmental restructuring DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: antibiotic days/100 OBD
Notes	FINANCIAL SUPPORT: Funding: Chicago Antimicrobial Resistance Project, Centers for Disease Control and Prevention (U50/ CCU515853). Competing Interests: no information ADDITIONAL DATA: email response with the guideline. The guidelines are supposed to be available online, but the link does not work.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Days of antimicrobial use calculated automatically by pharmacy computer.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Days of antimicrobial use calculated automatically by pharmacy computer.
Incomplete outcome data addressed (ITS) ?	Unclear risk	1 and 2 January 2000 start data were censored, but this was reported and would have little impact on the other 48 data points.
Free of selected reporting (ITS) ?	Unclear risk	Days of antimicrobial use calculated automatically by pharmacy computer.
Free of other bias (ITS) ?	High risk	Only 10 months' pre‐intervention data, so secular trends could not be addressed.

Senn 2004.

Methods	STUDY DESIGN: RCT Risk of bias: MEDIUM
Participants	PROVIDERS: residents on medical and surgical wards PATIENTS: 251 patients were recruited, 126 intervention and 125 control CLINICAL PROBLEM: adult patients receiving IV antibiotics for 3 to 4 days with no modification since starting treatment SETTING: single 800‐bed university hospital in Switzerland. Data collected over 5 months. POWER CALCULATION: yes, 135 patients in each group, but the trial was underpowered because the observed effect was lower than predicted. Details in Appendix 3
Interventions	FORMAT: Interventions: dissemination of questionnaire about guidelines; reminders (circumstantial and physical, questionnaire mailed to the resident in charge of patients who were receiving IV antibiotic treatment. The questionnaire asked 3 questions regarding possible adaptation of antibiotic therapy on day 3 or 4, and was collected 24 hours later. If the resident had not yet completed it at that time, he/she was reminded once to do so.) Intervention Functions: education, enablement, environmental restructuring DELIVERER: AMT COMPARISON: control patients with no intervention DESIRED CHANGE: reduction in established management (reduction in duration of IV therapy) TIMING: intervention at the point of decision making (potential modification 3 to 4 days after start of antibiotics)
Outcomes	PRESCRIBING: Choice: % of patients discontinuing IV antibiotics and hazard ratio adjusted for patients' Karnofsky functional index
Notes	FINANCIAL SUPPORT: Funding: Quality Improvement Committee of the Lausanne University Hospital and grant 32–63128.00 of the Swiss National Science Foundation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"Patients allocated ... by using a computer generated randomizations list"
Allocation concealment (selection bias)	Low risk	"Concealment of allocation was achieved as the physician in charge of the patient was involved after randomizations"
Blinding (performance bias and detection bias) All outcomes	High risk	"This was a randomised, controlled, open trial"
Incomplete outcome data (attrition bias) All outcomes	Low risk	Primary outcome measure (duration of IV antibiotics) collected on all patients. Only 70% of questionnaires returned for the intervention group, which could account for the intervention effect being lower than expected. However, this did not affect outcome assessment.
Selective reporting (reporting bias)	Low risk	Complete primary outcome data
Other bias	High risk	The study was underpowered.
Baseline Outcomes similar?	Low risk	Pre‐study group, data collected for 2 months before intervention to estimate the magnitude of possible observation bias (Figure 2).
Free of contamination?	Low risk	The pre‐intervention group data were comparable to the control group, suggesting minimal observation bias.
Baseline characteristics similar?	Low risk	Presented in Table 1

Shehabi 2014.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the ICU PARTICIPANTS: 400 patients; 6 withdrew consent, leaving 196 in the intervention and 198 in the control group CLINICAL PROBLEM: patients with suspected sepsis and likely to receive antibiotics/remain in the ICU for at least 24 h SETTING: 11 university hospitals in Australia
Interventions	Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physicians (ICU) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, 165 participants per group. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: duration of antibiotic treatment (days) CLINICAL: mortality, re‐admission, and length of hospital stay
Notes	FINANCIAL SUPPORT: Funding: Intensive Care Foundation of Australia and New Zealand. Material support was provided by Roche Diagnostics, Thermo Fisher Scientific, and bioMérieux. Roche Diagnostics and Thermo Fisher Scientific provided additional unrestricted grant funding. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Patients were variable block randomised 1:1 via a secured central study website into either a PCT‐guided (PCT) group or clinician‐guided (standard care) group. Randomisation was stratified according to the presence of septic shock (defined by the receipt of inotropes and/or any vasopressors within the previous 24 hours).
Allocation concealment (selection bias)	Low risk	See above
Blinding (performance bias and detection bias) All outcomes	High risk	Single‐blind
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on 196/200 intervention and 198/200 control participants.
Selective reporting (reporting bias)	Low risk
Other bias	Low risk	1567 patients screened, but 1167 excluded; full details of how many patients met each of the exclusion criteria (Figure 1)
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT only reported for intervention participants.
Baseline characteristics similar?	Low risk	Table 1

Shen 2011.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians on 2 respiratory wards PARTICIPANTS: all patients on the wards CLINICAL PROBLEM: receiving antibiotics for respiratory infection SETTING: 1 hospital
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Choice: score on 6 indicators of inappropriate antibiotic use: indication, choice, dosage, dosing schedule, duration, conversion CLINICAL: Balancing: length of stay FINANCIAL: cost (mean, SD) of antibiotics and total patient costs
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Says it was randomised, no further information.
Allocation concealment (selection bias)	High risk	Patients from 2 wards were randomised, and there is no information about allocation concealment.
Blinding (performance bias and detection bias) All outcomes	Low risk	"At the end of the study, a blinded coordinating investigator recorded the patients’ data"
Incomplete outcome data (attrition bias) All outcomes	Low risk	All outcomes reported.
Selective reporting (reporting bias)	Low risk	All outcomes reported.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No information
Free of contamination?	High risk	Intervention and control patients were on both wards.
Baseline characteristics similar?	Low risk	Table 1

Shojania 1998.

Methods	STUDY DESIGN: RCT with nested ITS analysis (Figures 3 and 4) Risk of Bias: HIGH
Participants	PROVIDERS: unit of randomisation ‐ 396 physicians in 7 specialties. Non‐physicians (nurses or pharmacists) who were authorised to enter orders that required eventual signing off by physicians were also randomised. PARTICIPANTS: There were 5536 episodes of care in 1798 patients. CLINICAL PROBLEM: receiving vancomycin treatment SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: dissemination of guideline; reminders (circumstantial, delivered through computer screen at the time of physician order entry and after 72 hours of therapy). The reminder required prescribers to produce a response: when someone would enter an order for intravenous vancomycin, a pop‐up screen would appear and display the appropriate indications for vancomycin use, which was a checkbox list of indications based on CDC guidelines. Users had to pick a reason or enter free text under 'other' in order to proceed. Intervention Functions: education, enablement, environmental restructuring DELIVERER: AMT COMPARISON: no reminder. ITS analysis used 9 months' pre‐intervention data. DESIRED CHANGE: reduction of established management (reduction in use of vancomycin) POWER CALCULATION: no information
Outcomes	PRESCRIBING: Choice: initiation and renewal of vancomycin therapy. Duration of vancomycin therapy on a per‐prescriber basis. Total use of vancomycin in the hospital. FINANCIAL: estimated savings
Notes	FINANCIAL SUPPORT: Funding: grant R01‐HS08927 from the Agency for Healthcare Policy and Research. Competing Interests: no information ADDITIONAL DATA: email from authors with additional details about the intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"The study was a randomised controlled trial"; no details on how randomisation sequence was generated
Allocation concealment (selection bias)	High risk	States "possible that physicians in the control group could learn of the intervention from physicians in the study group"
Blinding (performance bias and detection bias) All outcomes	High risk	Not done
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not clear for primary outcome
Selective reporting (reporting bias)	Low risk	Based on numbers of vancomycin orders
Other bias	Low risk	No issues noted.
Baseline Outcomes similar?	Unclear risk	No information about pre‐intervention vancomycin use
Free of contamination?	High risk	States "possible that physicians in the control group could learn of the intervention from physicians in the study group"
Baseline characteristics similar?	Low risk	Table 1

Singh 2000.

Methods	STUDY DESIGN: RCT, allocation by patient Risk of Bias: HIGH
Participants	PROVIDERS: all physicians on 1 ICU PARTICIPANTS: 81 episodes of care (39 intervention, 42 control) CLINICAL PROBLEM: suspected ventilator‐associated pneumonia with low CPIS SETTING: 1 hospital in the USA
Interventions	FORMAT: Intervention: restrictive by expert approval and review and make change Intervention Function: restriction DELIVERER: ID physician COMPARISON: choice, number, and duration of antibiotics at the discretion of the care providers DESIRED CHANGE: reduction of established management (reduction in duration of antibiotic treatment) POWER CALCULATION: yes, 88 patients per group. The study was terminated early. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: total duration of all antibiotic treatment CLINICAL: Balancing: mortality, length of ICU stay MICROBIAL: number of patients with "antimicrobial resistance and/or superinfections" from randomisation until hospital discharge FINANCIAL: total costs of care for patients with CPIS < 6 at 3 days and no extrapulmonary infections
Notes	FINANCIAL SUPPORT: Funding: Bayer Corporation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	"Patients were randomized to either the control group or experimental group"; no information about how randomisation sequence was generated
Allocation concealment (selection bias)	Unclear risk	Not stated
Blinding (performance bias and detection bias) All outcomes	High risk	Page 509: "Because the study was not blinded, physicians and care providers could see the results"
Incomplete outcome data (attrition bias) All outcomes	Low risk	Most outcomes are reported for 78 (96%) episodes of care; antimicrobial resistance and superinfection in 74 (91%) of episodes.
Selective reporting (reporting bias)	Low risk	No problems found.
Other bias	High risk	Microbial Risk of Bias HIGH. Case definition for microbial outcome NOT CLEAR: "Follow‐up respiratory cultures or cultures from clinical specimens performed 7 to 28 d after initiation of antibiotics were evaluated to assess the emergence of antimicrobial resistance or superinfections. Emergence of resistance was defined as the detection of new antimicrobial resistance pattern in the old or previously isolated organism. Superinfection was defined as the detection of the following organisms not present at study entry: Acinetobacter species, Serratia marcescens, Pseudomonas aeruginosa, Stenotrophomonas maltophilia, Enterobacter species, Citrobacter species, methicillin‐resistant Staphylococcus aureus (MRSA), Enterococcus species, and Candida species." It is therefore impossible to assess the impact of the intervention on colonisation or infection with bacteria resistant to specific antibiotics. Infection control NOT CLEAR. Planned intervention YES
Baseline Outcomes similar?	Unclear risk	Not stated, no information about pre‐intervention duration of antibiotic treatment
Free of contamination?	Unclear risk	Not stated
Baseline characteristics similar?	Low risk	See Table 1 in study.

Sirinavin 1998.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring treatment with imipenem vancomycin or injectable ciprofloxacin SETTING: 1 hospital in Thailand
Interventions	FORMAT: Interventions: educational meetings with dissemination of antimicrobial order form; educational outreach by review and recommend change of cases of inappropriate prescribing by ID consultant; restrictive by compulsory order form Intervention Functions: education, enablement, persuasion, restriction Figure 2 suggests that expenditure increased sharply in the final year of the study when ID consultant review ceased. DELIVERER: specialist physician (ID) COMPARISON: data for 4 years' pre‐restriction DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: restricted drugs cost in million THB/200,000 OBD
Notes	FINANCIAL SUPPORT: Funding: Ramathibodi Research Fund. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	4 years' data pre‐ and postintervention
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: run charts with no statistical analysis.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	High risk	NOT DONE, there is no information about change in price of antibiotics over the study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Unclear risk	NOT CLEAR, no adjustment of antibiotic costs for change in price, so change in price of antibiotics (rather than change in use) over the study period may have been responsible for some of the change in cost. Data were not adjusted for number of admissions or occupied bed days.

Skaer 1993.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: physicians (numbers not clear) PATIENTS: all patients in the hospital CLINICAL PROBLEM: adult patients receiving imipenem treatment SETTING: 1 hospital in the USA
Interventions	FORMAT: Intervention: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: pharmacist COMPARISON: usual care in the pre‐intervention phase DESIRED CHANGE: reduce excessive
Outcomes	PRESCRIBING: Choice: Monthly use (doses) of imipenem CLINICAL: cohort data about length of stay and hospital charges for patients with a primary diagnosis of infection
Notes	FINANCIAL SUPPORT: Funding: Washington State University College of Pharmacy and Pullman Memorial Hospital. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk
Knowledge of the allocation adequately prevented(ITS)?	Low risk
Incomplete outcome data addressed (ITS) ?	Low risk
Free of selected reporting (ITS) ?	Low risk
Free of other bias (ITS) ?	Low risk	Yes for primary outcome but fatally flawed (UBA) for secondary outcomes.

Skrlin 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: use of ceftriaxone following removal of restriction SETTING: 1 hospital in Croatia
Interventions	FORMAT: Intervention: restrictive, removal of restriction by expert approval Intervention Function: restriction DELIVERER: AMT COMPARISON: removal of restriction versus restriction DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of ceftriaxone in DDD/1000 OBD MICROBIAL: number of ESBL‐producing strains/1000 OBD
Notes	FINANCIAL SUPPORT: Funding none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: LOW case definition Low, planned intervention Low, other infection control Low
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	24 months' data pre‐ and postintervention

Solomon 2001.

Methods	STUDY DESIGN: cluster RCT, service level Risk of Bias: HIGH
Participants	PROVIDERS: 17 Internal Medicine services randomly assigned to intervention (9 services) or control (8 services) PARTICIPANTS: a total of 4500 patients admitted during the baseline and study periods, of whom 260 patients received 278 unnecessary prescriptions for the target drugs; 17 clusters (services) CLINICAL PROBLEM: patients receiving ceftazidime or levofloxacin. SETTING: 1 hospital in the USA POWER CALCULATION: no information. The methods say that the statistical model adjusted for clustering, but no results are given (see risk of bias).
Interventions	FORMAT: Interventions: educational meetings with dissemination of policy for necessary use; educational outreach by review and recommend change, either verbal (face to face or telephone) or by email Intervention Functions: education, enablement, persuasion COMPARISON: randomly assigned control services DESIRED CHANGE: reduce excessive POWER CALCULATION: no information. Note from Statistician: The study adjusted for some clustering, but possibly only in the repeated measures, not in the hospitals. Just using the results from Table 2, I do not get the P value that they state in the table using a unit of analysis error approach. This suggests to me that they are adjusting for "things". I therefore think on balance that it is probably OK to use the results.
Outcomes	PRESCRIBING: Choice: % patients with target antibiotics discontinued. Exposure: % patients with all antibiotics discontinued CLINICAL: inpatient mortality, transfer to ICU, length of stay, and re‐admission within 30 days of discharge FINANCIAL: estimated annual cost of the intervention
Notes	FINANCIAL SUPPORT: Funding: Brigham and Women’s Hospital and Arthritis Foundation Investigator Award. Competing Interests: no information ADDITIONAL DATA: email from authors with information about the intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"We assigned services to intervention or control status using a blocked randomization design"
Allocation concealment (selection bias)	Unclear risk	Not concealed
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Figure 2 and text give %, no denominator.
Selective reporting (reporting bias)	Unclear risk	Figure 2 and text give %, no denominator.
Other bias	Unclear risk	The methods say: "To estimate the relative reduction in unnecessary use of target antibiotics in the intervention group, we used a fixedeffects model (PROC GENMOD in SAS statistical software).20 This model used a log‐linear link function, assumed a Poisson distribution, and accounted for overdispersion. Experimental group assignment (intervention or control) was the independent variable of interest, the individual service was considered a class effect, and covariates included level of baseline prescribing and time, modeled as both a linear and categorical effect. The interaction between assignment and time was also assessed. We further considered a linear randomeffects model to account for variation between services (PROC MIXED in SAS statistical software)20; the results of this analysis were similar to those found in the fixed‐effects models with respect to the level of statistical significance, and only the fixedeffects model results are presented." However, no model outputs are given in the results (only point estimates), and the discussion says only: "This significant effect of the intervention remained after adjusting for baseline prescribing, clustering of repeated measures within a given service, and duration of the intervention."
Baseline Outcomes similar?	Low risk	Figures 1 and 2
Free of contamination?	High risk	The services were in the same hospital.
Baseline characteristics similar?	Low risk	Table 1

Standiford 2012.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: cost of animicrobials SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care pre‐intervention and impact of removal of the intervention (2 years) DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: quarterly cost of all antimicrobials CLINICAL: Balancing: cohort data for mortality, length of stay, and unplanned re‐admission. The DRG case mix index was monitored to ensure that changes in outcomes were not related to this index.
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	No information is given about changes in drug pricing over the 12 years of data collection, which is likely to have changed the outcome measure. In addition, there were changes in pharmacy data systems after the intervention, but the timing is clearly documented in Figure 1.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data were from the Pharmacy Administration and were independent from the AMT.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data were from the Pharmacy Administration and were independent from the AMT.
Incomplete outcome data addressed (ITS) ?	Low risk	Data were from the Pharmacy Administration and were independent from the AMT.
Free of selected reporting (ITS) ?	Low risk	Data were from the Pharmacy Administration and were independent from the AMT.
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Stevenson 1988.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: receiving antibiotics SETTING: 1 hospital in the UK
Interventions	FORMAT: Interventions: dissemination of antibiotic policy Intervention Function: education DELIVERER: pharmacist COMPARISON: 10 quarters (30 months) pre‐intervention DESIRED CHANGE: reduce excessive
Outcomes	PRESCRIBING and FINANCIAL: Choice: average cost of antibiotics per patient. Prices were indexed to 1980.
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	2 years' data pre‐ and postintervention, enough to account for seasonal effects
Analysed appropriately (ITS) ?	Low risk	Done in original paper: regression analysis testing for structural break associated with intervention.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, drug costs were adjusted to 1980 prices.
Free of other bias (ITS) ?	Low risk	Drug costs were adjusted to 1980 prices and adjusted for number of discharges or deaths.

Stocker 2010.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the neonatal ICU PARTICIPANTS: 121 neonates (60 intervention, 61 control) CLINICAL PROBLEM: suspected sepsis SETTING: 1 hospital in Switzerland
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physician (Paediatrics) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: unclear. The trial was designed to obtain a power of 90% to detect a 30% difference between the 2 groups in the duration of antibiotic therapy, with an estimated standard deviation of 50%. Sample size: no information
Outcomes	PRESCRIBING: Exposure: duration, % treated > 72 h
Notes	FINANCIAL SUPPORT: Funding: commercial B.R.A.H.M.S Diagnostica (Berlin, Germany) provided the testing kits for PCT. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Randomised using assignment cards in envelopes
Allocation concealment (selection bias)	Low risk	Randomised using assignment cards in envelopes
Blinding (performance bias and detection bias) All outcomes	High risk	Not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes on all patients
Selective reporting (reporting bias)	Low risk	Outcomes on all patients
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	PCT results only reported for intervention patients.
Baseline characteristics similar?	Low risk	Table 1

Stolz 2007.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in Internal Medicine PARTICIPANTS: all patients hospitalised with exacerbations of chronic obstructive pulmonary disease; 288 screened, 226 randomised (113 intervention, 113 control) CLINICAL PROBLEM: use of therapeutic antibiotics SETTING: 1 hospital
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: departmental physician (Respiratory) COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION: yes, total 186 participants. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: % antibiotic use for the exacerbation and in the subsequent 6 months CLINICAL: Balancing: length of stay, death, symptom scores
Notes	FINANCIAL SUPPORT: Funding: part commercial University Hospital Basel. B.R.A.H.M.S provided procalcitonin assays for this investigator‐driven study. Competing Interests: 1 author served as consultant and received payments from B.R.A.H.M.S to attend meetings and for travel expenses, speaking engagements, or research. ADDITIONAL DATA: email response from authors with additional information about intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	No details: "Patients satisfying the entry criteria were randomly assigned to one of two groups at the time of admission to the emergency department "
Allocation concealment (selection bias)	Unclear risk	No details
Blinding (performance bias and detection bias) All outcomes	High risk	SIngle‐blind
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome data reported on all patients.
Selective reporting (reporting bias)	High risk	11 (10%) patients excluded from intervention and 7 (6%) from control group for "absence of COPD according to GOLD", but this should have occurred pre‐randomisation.
Other bias	Low risk
Baseline Outcomes similar?	High risk	No data
Free of contamination?	Low risk	PCT only reported for intervention patients.
Baseline characteristics similar?	Low risk	Table 1

Stolz 2009.

Methods	STUDY DESIGN: RCT Risk of Bias: MEDIUM
Participants	PROVIDERS: all staff in adult ICUs PARTICIPANTS: 101 patients with VAP (51 intervention, 50 control) CLINICAL PROBLEM: receiving antibiotics for VAP SETTING: 3 university hospitals in Switzerland and the USA
Interventions	FORMAT: Interventions: reminders (circumstantial, decision support algorithm with each PCT test); structural, introduction of PCT testing Intervention Functions: enablement, environmental restructuring DELIVERER: respiratory physicians COMPARISON: usual care DESIRED CHANGE: decrease excessive POWER CALCULATION; yes, 84 participants total. Details in Appendix 3
Outcomes	PRESCRIBING: Exposure: duration of antibiotic treatment CLINICAL: Balancing: mortality, hospital length of stay
Notes	FINANCIAL SUPPORT: Funding: Swiss National Foundation, Margarete und Walter Liechtenstein Foundation, Freiwillige Akademische Gesellschaft Basel, Will Rogers Foundation, and participating hospitals. B.R.A.H.M.S AG funded assay material and logistics. Competing Interests: not clear. The published paper says that a statement of interest for the study itself is available but the web address provided online and in print does not work. ADDITIONAL DATA: email response from authors with additional information about intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Block size 20 envelopes
Allocation concealment (selection bias)	Low risk	Sealed, opaque envelopes
Blinding (performance bias and detection bias) All outcomes	High risk	Primary outcome measure required collection of data from case noes by investigators.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Text shows that primary outcome was reported for all 101 randomised patients.
Selective reporting (reporting bias)	Low risk	Text shows that primary outcome was reported for all 101 randomised patients.
Other bias	Low risk	Multivariate analysis to adjust primary outcome for age, microbiology and centre effect
Baseline Outcomes similar?	Unclear risk	No data about baseline outcomes
Free of contamination?	Low risk	Procalcitonin only measured for intervention patients.
Baseline characteristics similar?	Low risk	Table 1

Strom 2010.

Methods	STUDY DESIGN: cluster RCT, professional level Risk of Bias: MEDIUM
Participants	PROVIDERS: A total of 1971 clinicians were assigned to either an intervention group receiving a nearly hard‐stop alert or a control group receiving the standard practice. PARTICIPANTS: 342 patients receiving warfarin and trimethoprim‐sulfamethoxazole (194 intervention, 148 control), 1971 clusters (physicians) CLINICAL PROBLEM: reduce risk of interaction between warfarin and trimethoprim‐sulfamethoxazole SETTING: 2 hospitals in the USA POWER CALCULATION: "It is generally accepted that randomization of at least 100 subjects will produce balance between the study groups and, of course, the present sample size is much larger than this."
Interventions	FORMAT: Interventions: reminder (circumstantial) and restrictive by compulsory electronic order form that would not allow concomitant orders of warfarin and trimethoprim‐sulfamethoxazole. The only exception allowed by the order form was the indication of Pneumocystis carinii pneumonia prophylaxis. Expert approval was allowed for other patients when discussed with pharmacy. Intervention Functions: enablement, environmental restructuring, restriction DELIVERER: pharmacist COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: the proportion of desired responses (i.e. not reordering the alert‐triggering drug within 10 minutes of firing) CLINICAL: Balancing: 2 potential adverse outcomes of the computerised hard‐stop alert were monitored and reported to the Institutional Review Board. The first was a delay in obtaining trimethoprim‐sulfamethoxazole when the practitioner believed that an infection was best treated with trimethoprim‐sulfamethoxazole and when the potential warfarin interaction was judged less important than the need for the antibiotic. The second was unintentional warfarin cessation in a patient previously undergoing long‐term warfarin therapy. The study therefore also assessed the incidence of warfarin cessation on the day when an order of trimethoprim‐sulfamethoxazole was attempted in a patient already receiving warfarin.
Notes	FINANCIAL SUPPORT: Funding: University of Pennsylvania Health System and Agency for Healthcare Research and Quality. Competing Interests: none declared ADDITIONAL DATA: email response from authors with additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Number randomisation
Allocation concealment (selection bias)	Low risk	"Each medical practitioner has a unique access code to use the electronic ordering system, and the order system menu can be varied by individual user. In addition, we wanted to keep each practitioner in the same study group for the duration of the study to minimize contamination between the 2 groups. However, there is the possibility"
Blinding (performance bias and detection bias) All outcomes	High risk	Not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome reported on all patients, determined electronically.
Selective reporting (reporting bias)	Low risk	Outcome reported on all patients, determined electronically.
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No information
Free of contamination?	Low risk	"We attempted to reduce contamination by trying to complete this study as rapidly as possible. It was initially planned to last 7 months but had to be terminated early."
Baseline characteristics similar?	Low risk	Table 1

Sun 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all cardiac surgeons and other professionals PARTICIPANTS: all patients undergoing coronary artery bypass surgery CLINICAL PROBLEM: improve reliability of administration of prophylaxis (first dose within 1 h of incision and duration not > 24 h) SETTING: 1 hospital in Taiwan
Interventions	FORMAT: Interventions: audit and feedback; educational meetings with dissemination of guidelines and evidence base Intervention Functions: education, enablement DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice and exposure: time to first antibiotic dose, % of prophylaxis ≤ 24 h
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	From Taiwan Quality Improvement Project database
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from Taiwan Quality Improvement Project database.
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome reported on all patients pre‐ and postintervention.
Free of selected reporting (ITS) ?	Low risk	Objective outcomes from Taiwan Quality Improvement Project database
Free of other bias (ITS) ?	High risk	< 1 year data pre‐ and postintervention

Suwangool 1991.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the Department of Medicine PARTICIPANTS: all patients in the Department of Medicine CLINICAL PROBLEM: inappropriate antibiotic prescribing SETTING: single university hospital in Thailand
Interventions	FORMAT: Interventions: dissemination of guidelines; restrictive by expert approval Intervention Functions: education, restriction DELIVERER: AMT COMPARISON: 6 months' data pre‐intervention DESIRED CHANGE: reduce excessive (cost)
Outcomes	PRESRIBING: Choice: monthly cost of target antibiotics CLINICAL: cohort data about mortality
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy computer
Free of other bias (ITS) ?	Unclear risk	< 1 year data pre‐ and postintervention. During the 18‐month study period, no adjustment was made to antibiotic costs for changes in prices, so changes in cost may have been due to changes in price as well as use.

Talpaert 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients receiving antibiotics for prophylaxis or treatment. The intervention targeted fluoroquinolones, cephalosporins, clindamycin, amoxicillin, and co‐amoxiclav, as they were considered to be "high risk" for Clostridium difficile infection SETTING: 1 hospital
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change; reminders (verbal (on rounds) and physical (laminated pocket cards and posters)); restrictive by removal of target drugs from clinical areas Intervention Functions: education, enablement, environmental restructuring, persuasion restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD MICROBIAL: monthly cases of C difficile infection
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: 1 author was paid lecture fees and provided sponsorship to attend conferences by pharmaceutical companies unrelated to this study. ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias LOW: case definition Low (new cases), planned intervention Low, other infection control Low, fully reported in ORION format
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Change in site ‐ moved to another building
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	From electronic records, so unlikely
Knowledge of the allocation adequately prevented(ITS)?	Low risk	From electronic records, so unlikely
Incomplete outcome data addressed (ITS) ?	Low risk	From electronic records, so unlikely
Free of selected reporting (ITS) ?	Low risk	From electronic records, so unlikely
Free of other bias (ITS) ?	Low risk	1 year data pre‐ and postintervention

Tangdén 2011.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving therapeutic antibiotics CLINICAL PROBLEM: aim (i) to reduce the consumption of 2nd‐ and 3rd‐generation cephalosporins; and (ii) to avoid increased prescription of fluoroquinolones and carbapenems. SETTING: 1 hospital in Sweden
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by academic detailing Intervention Functions: education, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of target drugs in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: no external. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH, Case definition Low, Unplanned intervention High (outbreak), Other infection control High. "In August 2006, the hospital director organized a steering group (SG) with the assignment to implement the necessary measures to contain the outbreak, including reinforcement of hygienic measures, such as hand disinfection, use of disposable gloves and aprons, and isolation of patients colonized or infected with ESBL‐KP.14 In addition to hygienic measures, the SG decided to perform an antibiotic intervention."
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	DDD from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	DDD from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	DDD from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	DDD from pharmacy computer
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention

Toltzis 1998.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the mixed medical and surgical paediatric ICU PARTICIPANTS: all patients in the paediatric ICU CLINICAL PROBLEM: patients requiring antibiotic treatment SETTING: a paediatric ICU in 1 hospital in the USA
Interventions	FORMAT: Intervention: restrictive, probably by expert approval ("Prohibition of ceftazidime use unless the patient's microbiological results indicated that the drug was necessary for cure.") Intervention Function: restriction DELIVERER: specialist (ID) physician COMPARISON: 7 months' data before the start of the intervention DESIRED CHANGE: reduce excessive
Outcomes	PRESCRIBING: Choice: ceftazidime use in doses
Notes	FINANCIAL SUPPORT: Funding: grant HD31323‐02 from the National Institutes of Health. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	NOT CLEAR, data for 7 months pre‐ and 12 months postintervention, not enough to adjust for seasonal variation
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after) with χ2 test.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	High risk	< 1 year data pre‐intervention

Toltzis 2002.

Methods	STUDY DESIGN: NRT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians (paediatricians) on the ICU PARTICIPANTS: all neonates in the ICU CLINICAL PROBLEM: neonates with proven or suspected infections caused by gram‐negative bacteria SETTING: 1 neonatal ICU in 1 hospital in the USA
Interventions	FORMAT: no valid prescribing data. Restrictive by removal, monthly rotation of the antibiotic regimen used for empirical prescribing of patients with proven or suspected gram‐negative infections DELIVERER: specialist physician (ICU) COMPARISON: standard practice DESIRED CHANGE: reduce excessive (colonisation with multiresistant bacteria)
Outcomes	MICROBIAL: incidence of colonisation with multiantibiotic‐resistant aerobic gram‐negative bacilli
Notes	FINANCIAL SUPPORT: Funding: grant HD 31323‐05 from the National Institutes of Health Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: MEDIUM Case definition Low, Planned intervention Low, Other infection control Unclear
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	NRT with monthly rotation of regimens
Allocation concealment (selection bias)	High risk	Not possible with this study design
Blinding (performance bias and detection bias) All outcomes	High risk	Not possible with this study design
Incomplete outcome data (attrition bias) All outcomes	Unclear risk	Not stated whether screening samples obtained from all patients
Selective reporting (reporting bias)	Unclear risk	Not stated whether screening samples obtained from all patients
Other bias	Unclear risk	NOT CLEAR Microbial Outcome Risk of Bias Criteria Case definition: DONE Colonisation by screening. "For the purpose of this study, an 'antibiotic‐resistant Gram‐negative organism' was defined as any Gram‐negative bacillus resistant to gentamicin, piperacillin‐tazobactam, or ceftazidime. Pharyngeal and rectal swab specimens were obtained on all infants every Monday, Wednesday, and Friday". Planned intervention: DONE; Other infection control, Isolation: IC practices: NOT CLEAR Not described, but it is reasonable to assume that they were the same for the intervention and control groups due to the controlled clinical trial design.
Baseline Outcomes similar?	Unclear risk	Not stated
Free of contamination?	Unclear risk	Not stated, but doctors likely to have been managing patients in more than 1 study phase.
Baseline characteristics similar?	Low risk	Results, paragraph 1

Toltzis 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all paediatric surgeons and anaesthetists PARTICIPANTS: all children undergoing surgery CLINICAL PROBLEM: increase % of patients receiving antibiotic prophylaxis within 1 hour of incision as 1 of 3 components of a bundle of care SETTING: 8 paediatric hospitals in the USA
Interventions	FORMAT: no valid prescribing data. Audit and feedback DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	CLINICAL: surgical‐site infection rate per 100 procedures
Notes	FINANCIAL SUPPORT: Funding: Ohio Business Roundtable, the Cardinal Health Foundation, and the Ohio Children’s Hospital Association. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Administration of antibiotics within 1 hour was 1 component of the bundle; the other 2 were avoiding shaving and encouraging use of clorhexidine for disinfection. In addition, 9 months after the intervention began an additional antibiotic element was added to encourage administration of an additional dose for operations lasting more than 3 hours.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Patient administration systems and routine collection of surgical‐site infection data by each hospital's infection prevention teams
Knowledge of the allocation adequately prevented(ITS)?	High risk	Infection prevention teams were not prevented from knowing about allocation.
Incomplete outcome data addressed (ITS) ?	Low risk
Free of selected reporting (ITS) ?	Low risk	Data reported for all months when operations took place
Free of other bias (ITS) ?	High risk	Only 8 months' pre‐intervention data

Trenholme 1989.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 226 patients (110 intervention, 116 control) CLINICAL PROBLEM: patients with bacteraemia SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: educational outreach by review and recommend change (intervention and control); structural, rapid processing and reporting of antimicrobial susceptibility tests (intervention only) Intervention Functions: enablement, environmental restructuring, persuasion DESIRED CHANGE: reduce excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Choice: % changes in therapy in response to recommendation FINANCIAL: savings in drug costs
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Not stated; "the organism from the patient was randomly assigned"
Allocation concealment (selection bias)	Unclear risk	Not stated
Blinding (performance bias and detection bias) All outcomes	Unclear risk	Not stated as blind
Incomplete outcome data (attrition bias) All outcomes	Low risk	Table 2 reports primary outcome for all 226 randomised patients.
Selective reporting (reporting bias)	Low risk	Table 2 reports primary outcome for all 226 randomised patients.
Other bias	Low risk	No other apparent biases found.
Baseline Outcomes similar?	Unclear risk	No information about recommendations for changes in therapy before the intervention
Free of contamination?	Unclear risk	Likely to be contamination as doctors managing control patients would receive advice on intervention patients.
Baseline characteristics similar?	Unclear risk	No information

Uçkay 2009.

Methods	STUDY DESIGN: ITS Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in one orthopaedic unit PARTICIPANTS: all patients in one orthopaedic unit CLINICAL PROBLEM: suspected bone and joint infection SETTING: 1 hospital in Switzerland
Interventions	FORMAT: Interventions: educational outreach by review and recommend change. The intervention is reported in 2 phases, the 1st delivered by "Dedicated ID specialist and one internist" and the 2nd delivered by "ID specialist with experience in Infection Control". Intervention Functions: enablement, persuasion DELIVERER: specialist (ID) physician COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of IV and oral antibiotics in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: none.Competing Interests: none declared. ADDITIONAL DATA: email response from authors with additional data about the intervention
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Insufficient information to assess
Analysed appropriately (ITS) ?	Low risk	Time series analysis with ARIMA modelling
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Unclear risk	Outcome data were from routine pharmacy systems.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data were from routine pharmacy systems.
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data were from routine pharmacy systems.
Free of selected reporting (ITS) ?	High risk	The information in Table 1 does not include total antibiotic use or cost, so cannot be used to support the claims made in the paper.
Free of other bias (ITS) ?	High risk	< 1 year pre‐intervention data Insufficient information to assess. In particular, it is not clear what difference the "ID specialist with experience in Infection Control" would make compared with "Dedicated ID specialist and one internist".

Valiquette 2007.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: receiving therapeutic or prophylactic antibiotics SETTING: 1 hospital in Canada
Interventions	FORMAT: Interventions: educational meetings with dissemination of guideline and letter; educational outreach by review and recommend change; reminders (physical, pocket‐size guideline) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice and exposure: use of individual targeted drugs in DDD/1000 OBD; use of all antibiotics in DDD/1000 OBD MICROBIAL: Clostridium difficile infections/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: National Foundation for Infectious Diseases. Competing Interests: 1 author has been on the speakers’ bureau for Wyeth; served on advisory boards for Wyeth and Cubist; and received grants from Wyeth, Genzyme, and Arpida. 1 author has been on the speakers’ bureau for Wyeth Canada; served on advisory boards for Bayer, Wyeth, ViroPharma, and Acambis; and received grants from Genzyme. ADDITIONAL DATA: email from authors but no additional data Microbial Risk of Bias: HIGH Case definition Low. Planned intervention High, response to epidemic of infection caused by high‐virulence strain. Other infection control High, the rate of CDI was already declining in response to infection control intervention when antimicrobial intervention began.
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Antimicrobial intervention followed an infection control intervention, so it is not possible to assess the independent impact on C difficile infection. Moreover, the infection control intervention could have been responsible for some or all of the reduction in total antibiotic use.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	High risk	Microbial Risk of Bias HIGH > 1 year of data pre‐intervention

van Hees 2008.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the Departments of Internal Medicine, Gastroenterology, Surgery, Urology, and Pulmonary Diseases PARTICIPANTS: all patients in the same departments CLINICAL PROBLEM: patients receiving ciprofloxacin SETTING: 1 university hospital in the Netherlands
Interventions	FORMAT: Interventions: educational meetings; educational outreach by review and recommend change Intervention Functions: education, enablement, persuasion DELIVERER: specialist physicians (microbiologists) COMPARISON: usual care DESIRED CHANGE: decrease excessive (reduce unnecessary ciprofloxacin)
Outcomes	PRESCRIBING: Choice: use of ciprofloxacin in prescriptions/1000 OBD
Notes	FINANCIAL SUPPORT:Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data for primary outcome measure were from pharmacy computer.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data for primary outcome measure were from pharmacy computer.
Incomplete outcome data addressed (ITS) ?	Low risk	Data for primary outcome measure were from pharmacy computer.
Free of selected reporting (ITS) ?	Low risk	Data for primary outcome measure were from pharmacy computer.
Free of other bias (ITS) ?	High risk	Only 3 months' pre‐ and 6 months' postintervention data, so cannot be adjusted for seasonal trends.

Van Kasteren 2005.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospitals PATIENTS: all patients undergoing elective surgery CLINICAL PROBLEM: surgical prophylaxis across 4 surgical disciplines SETTING: 14 hospitals in the Netherlands
Interventions	FORMAT, Interventions: audit and feedback; educational meetings with dissemination of guidelines Intervention Functions: education, enablement DELIVERER: AMT COMPARISON: pre‐intervention periods DESIRED CHANGE: reduce excessive duration of surgical prophylaxis
Outcomes	PRESCRIBING: Exposure: total antibiotic use in DDD/100 procedures CLINICAL: Balancing: cohort data on surgical‐site infections
Notes	FINANCIAL SUPPORT: Funding: Netherlands Organisation for Health Research and Development (ZonMw). Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Only 6 months' pre‐ and postintervention data, and the model was not adjusted for seasonal trends.
Analysed appropriately (ITS) ?	Low risk	Done in original paper: segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period. Change in price unlikely to be a problem because only 6 months' data pre‐ and postintervention.

Volpe 2012.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the ED PARTICIPANTS: all patients with fever and suspected neutropenia CLINICAL PROBLEM: fever and suspected neutropenia SETTING: 1 university paediatric hospital in the USA
Interventions	FORMAT: Interventions: audit and feedback with action planning; educational meetings with dissemination of care algorithm and forms to facilitate care; educational outreach by academic detailing; reminders (circumstantial, root‐cause analysis of individual cases not meeting goal); reminders (physical, posters, email, and verbal, during rounds) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective, reduce time to first antibiotic dose
Outcomes	PRESCRIBING: Choice: time (minutes) to first antbiotic dose BALANCING MEASURE OF UNINTENDED CONSEQUENCES: "For balancing measures during the improvement period, we chose to follow the timeliness of first b‐agonist treatment of patients with asthma and the left without being seen (LWBS) rate."
Notes	FINANCIAL SUPPORT:Funding: no external funding. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Statistical process control chart
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from patient administration system
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from patient administration system
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from patient administration system
Free of selected reporting (ITS) ?	Low risk	Outcome data from patient administration system
Free of other bias (ITS) ?	Low risk	> 12 months' data pre‐ and postintervention

Walker 1998.

Methods	STUDY DESIGN: RCT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 50 patients (25 intervention, 25 control) CLINICAL PROBLEM: duration of IV antibiotics for patients with community‐acquired pneumonia SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: educational outreach by review and recommend change Intervention Functions: enablement, persuasion A written recommendation to change from IV ceftriaxone to an oral regimen was placed in each patient's prescription chart by the pharmacist. Direct contact with prescribers was not possible "because the medical staff in community hospitals have a large variation in the hours in which they make rounds" and "the physician is frequently busy, phone calls usually involve multiple pharmacists". DELIVERER: pharmacist COMPARISON: standard practice (no intervention) DESIRED CHANGE: reduce excessive POWER CALCULATION: no information
Outcomes	PRESCRIBING: Choice: number of patients changed to oral antibiotic therapy CLINICAL: Balancing: re‐admissions (total and for pneumonia)
Notes	FINANCIAL SUPPORT: Funding: commercial, Pharmacia and Upjohn. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"A list of random numbers was generated from Sigmastat version 1.0 statistical software"
Allocation concealment (selection bias)	Unclear risk	Not stated, but open label, so unlikely to be concealed
Blinding (performance bias and detection bias) All outcomes	High risk	"Open label"
Incomplete outcome data (attrition bias) All outcomes	Low risk	No problems found.
Selective reporting (reporting bias)	Low risk	No problems found.
Other bias	Low risk	No other apparent biases found.
Baseline Outcomes similar?	Unclear risk	Not stated
Free of contamination?	Unclear risk	Not stated
Baseline characteristics similar?	Low risk	See Table 1 in paper

Wang 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in 16 adult ICUs PARTICIPANTS: all patients in the ICUs CLINICAL PROBLEM: use of target antibiotics in patients with positive blood cultures SETTING: 1 University hospital in Taiwan
Interventions	FORMAT: Intervention: educational outreach by review and recommned change; restrictive by expert approval Intervention Functions: education, enablement, persuasion, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive, reduce cost of antimicrobials by reducing unnecessary use
Outcomes	PRESCRIBING: Choice: primary outcome is cost of all antimicrobials (Figure 4G). Also reports impact on use of 7 target antibacterials and use of antifungals in DDD/1000 OBD. CLINICAL: Balancing: mortality, ICU re‐admission (segmented regression analysis)
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Outcome data from pharmacy computer
Free of other bias (ITS) ?	High risk	> 12 months' data pre‐ and postintervention. However, no adjustment of primary outcome for changes in drug pricing over the 5 years of the study.

Wax 2007.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all anaesthetists in the hospital PARTICIPANTS: all patients undergoing elective surgery CLINICAL PROBLEM: time to first dose for antibiotic prophylaxis SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: reminders (physical, electronic on screen during all surgical procedures, not just those requiring prophylaxis) Intervention Functions: education, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Choice: % patients with first dose within 1 hour of incision
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcome data from electronic patient record, Anaesthesia Information Management System (AIMS)
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcome data from electronic patient record (AIMS)
Incomplete outcome data addressed (ITS) ?	Low risk	Outcome data from electronic patient record (AIMS)
Free of selected reporting (ITS) ?	Low risk	Outcome data from electronic patient record (AIMS)
Free of other bias (ITS) ?	High risk	Only 6 months' data pre‐intervention

Weinberg 2001.

Methods	STUDY DESIGN: controlled ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: operating theatre teams at participating hospitals PARTICIPANGS: low‐income women needing C‐section CLINICAL PROBLEM: infection after C‐section SETTING: 2 hospitals in Colombia
Interventions	FORMAT: Interventions: audit and feedback in the form of run charts for the 2 key process measures (secondary outcomes) with data collected and displayed by the clinical teams; dissemination of flow charts with revised system for administration of prophylactic antibiotics Intervention Functions: education, enablement DELIVERER: obstetric teams, doctors and nurses COMPARISON: physician choice about antibiotic and timing DESIRED CHANGE: reduce infection after C‐section TIMING: before clinical decision making; the intervention was continued for 2 years
Outcomes	PRESCRIBING: Choice: percentage of women who received prophylaxis; percentage who received prophylaxis within 1 hour CLINICAL: Intended: SSI rate per 100 C‐sections
Notes	INSTRUCTIONS: action plan provided, specific target but no specified time for target to be achieved FINANCIAL SUPPORT: Funding: International Society for Infectious Diseases, Paul Schliesman Memorial Traveling Fellowship, and the Von L. Meyer Award. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	Data collection method was the same pre‐ and postintervention.
Analysed appropriately (ITS) ?	Low risk	Done in original paper: segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Data collection method was the same pre‐ and postintervention.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Prescribing outcome data were from electronic systems.
Incomplete outcome data addressed (ITS) ?	Low risk	For prescribing outcome. Not stated whether SSI was evaluated in all patients
Free of selected reporting (ITS) ?	Low risk	For prescribing outcome. Not stated whether SSI was evaluated in all patients
Free of other bias (ITS) ?	High risk	< 1 year of data in each of the 3 study phases

Weiner 2009.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all attending emergency physicans, physician assistants, and emergency nurses PARTICIPANTS: all patients with community‐acquired pneumonia CLINICAL PROBLEM: time to first antibiotic dose SETTING: 1 university hospital in the USA
Interventions	FORMAT: Interventions: audit and feedback; reminders (physical, electronic ‐ weekly emails) Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: departmental nurse administrator COMPARISON: usual care DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Choice: mean time to first antibiotic dose (minutes)
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed; our analysis questions the authors' conclusion that the intervention was effective.
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	TFAD from patient administration system
Knowledge of the allocation adequately prevented(ITS)?	Low risk	TFAD from patient administration system
Incomplete outcome data addressed (ITS) ?	Low risk	TFAD from patient administration system, outcome reported on all included patients.
Free of selected reporting (ITS) ?	Low risk	"Patients were excluded if the time of antibiotic administration was not documented in the electronic medical record, if the patient was documented as having received antibiotics within 48 hours prior to arrival, or if the patient was referred from another facility or clinic with a known diagnosis of pneumonia." Exclusion rate in pre‐intervention period (37/281, 13%) similar to intervention period (40/342, 12%).
Free of other bias (ITS) ?	High risk	Only 11 months' data pre‐ and postintervention

Weiss 2013.

Methods	STUDY DESIGN: cluster NRT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the ICU PARTICIPANTS: all patients in the ICU CLINICAL PROBLEM: receiving antibiotic treatment SETTING: 1 University hospital in the USA
Interventions	FORMAT: Intervention: reminder, verbal (on rounds) based on a scripted electronic checklist of issues to discuss about antibiotics Intervention Functions: environmental restructuring, persuasion DELIVERER: departmental physicians COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: duration of empiric antibiotic treatment before narrowing choice, % patient days on which empiric antibiotics were used. Exposure: duration of all antibiotic treatment CLINICAL: Balancing: mortality (total, standardised mortality ratio, and adjusted odds of death), length of hospital stay, length of ICU stay
Notes	FINANCIAL SUPPORT: National Heart, Lung, and Blood Institute (T32HL076139‐07) and Parker B. Francis Fellowship to CHW. Dr Weiss has received funding from the National Institutes of Health. Drs Sung and Rho received a travel award to present a research abstract at American Thoracic Society conference in May 2012 from Northwestern University. Dr Wunderink is a board member for Pfizer and has consulted for Crucell (now Johnson & Johnson), Trius, AstraZeneca, and GlaxoSmithKline. He has received grant support from bioMérieux and payment for lectures from the American Thoracic Society. The remaining authors have not disclosed any potential conflicts of interest. ADDITIONAL DATA: online supplementary data for this article and further details of intervention in Weiss 2011. No response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Coin toss to allocate 1 medical team to intervention and 1 to control
Allocation concealment (selection bias)	High risk	No concealment
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcomes reported on all patients.
Selective reporting (reporting bias)	High risk	No information about inter‐rater reliability of primary outcome measure, which was not objective: "empirical antibiotics were defined as any antimicrobial agent administered without culture‐documented infection".
Other bias	High risk	Unit of analysis error, no adjustment for clustering
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	High risk	Intervention and control teams worked on the same ICU.
Baseline characteristics similar?	Low risk	Table 1

Welker 2008.

Methods	STUDY DESIGN: unintended consequences, cohort study Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the ED PARTICIPANTS: 548 patients with an admission diagnosis of community‐acquired pneumonia CLINICAL PROBLEM: hospital admission diagnosis of community‐acquired pneumonia SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: audit and feedback; financial, institution incentive Intervention Functions: enablement, incentive DELIVERER: departmental physicians (ED) COMPARISON: usual care (before introduction of core quality measure of 4 hours' time to first antibiotic dose) DESIRED CHANGE: increase effective
Outcomes	UNINTENDED CONSEQUENCES: accuracy of admission diagnosis, antibiotic‐associated adverse drug events
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status complete in all 548 patients 6. Measurement of outcome: High, outcome measures not objective, and investigators were not blinded to intervention status 7. Selection of the reported result: Low, single analysis of prespecified outcomes FINANCIAL SUPPORT: Funding: commercial: Pfizer, US Pharmaceutical Corporation. Competing Interests: none declared. ADDITIONAL DATA: no response from authors to request for additional data

Wenisch 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients receiving moxifloxacin SETTING: 1 university hospital in Austria
Interventions	FORMAT: Intervention: educational meetings; restrictive by compulsory order form Intervention Functions: education, restriction DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of moxifloxacin in DDD
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: Low for case definition, planned intervention, and other infection control
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Re‐analysed
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	High risk	< 12 months' data in the pre‐intervention (5 months) and postintervention (7 months) phases Microbial Risk of Bias LOW: case definition Low, planned intervention Low, other infection control Low

Willemsen 2010.

Methods	STUDY DESIGN: ITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving therapeutic antibiotics CLINICAL PROBLEM: decrease use of ciprofloxacin SETTING: 1 hospital in the Netherlands
Interventions	FORMAT: Interventions: educational meetings with dissemination of guidelines; educational outreach by review and recommend change; reminders (physical, newsletter and on all microbiology reports saying that ciprofloxacin should be prescribed on strict indications only) Intervention Functions: education, enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: prescribed daily doses of ciprofloxacin (IV and oral) MICROBIAL: % quionolone‐resistant gram‐negative clinical isolates
Notes	FINANCIAL SUPPORT: Funding: Amphia Hospital, Breda/Oosterhout, Netherlands. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: LOW Case definition infection with quionolone‐resistant gram‐negative bacteria, Planned intervention Low, Other infection control Low, no changes (information in Discussion)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Outcomes from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Outcomes from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Outcomes from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Outcomes from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	1 year data pre‐ and postintervention

Wilson 1991.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients receiving amoxicillin or pivampicillin SETTING: 3 hospitals in the UK
Interventions	FORMAT: Intervention: dissemination of newsletter to all prescribers Intervention Function: education DELIVERER: pharmacists COMPARISON: 5 months before introduction of the newsletter DESIRED CHANGE: reduce excessive
Outcomes	PRESCRIBING: Choice: use of amoxicillin and pivampicillin
Notes	FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	Only 5 months' pre‐intervention data. Even with 26 months' postintervention data, could still be secular changes.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: run chart with no statistical analysis.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Winters 2010.

Methods	STUDY DESIGN: unintended consequences, cohort study Risk of Bias: LOW
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 3251 patients receiving antibiotics CLINICAL PROBLEM: time to first antibiotic dose SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: restrictive by prior approval Intervention Functions: restriction DELIVERER: AMT COMPARISON: usual care, 10 restricted vs 15 unrestricted antibiotics; daytime (8 am to 10 pm) when prior approval is required vs nighttime (10 pm to 8 am) when the first dose of all antimicrobials was exempted DESIRED CHANGE: decrease excessive
Outcomes	UNINTENDED CONSEQUENCES: delays of > 1 hour or > 2 hours in TFAD
Notes	ROBINS‐I RISK OF BIAS CRITERIA: 1. Confounding: Low, confounding of the effect of intervention unlikely in this study 2. Selection of participants into the study: Low, selection into the study unrelated to intervention or outcome 3. Measurement of interventions: Low, intervention status well defined, recorded at the time of intervention and unaffected by knowledge of the outcome or risk of the outcome 4. Departures from intended interventions: Low, no switches to other interventions or evidence of intervention failure 5. Missing data: Low, outcome data and intervention status complete in all 3251 patients 6. Measurement of outcome: Low, outcome measures objective and ascertained from patient administration system 7. Selection of the reported result: Low, single analysis of prespecified outcomes FINANCIAL SUPPORT: no information ADDITIONAL DATA: no response from authors to request for additional data

Wishaupt 2011.

Methods	STUDY DESIGN: NRT Risk of Bias: HIGH
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: 614 children < 12 years old (309 intervention, 305 control) CLINICAL PROBLEM: acute respiratory infections (NB only 2/3 of randomised patients admitted to hospital) SETTING: 1 hospital in the Netherlands
Interventions	FORMAT: Intervention: structural, rapid reporting of microbiology results Intervention Function: environmental restructuring DELIVERER: specialist physician (Microbiology) COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Exposure: % treated with antibiotics and duration if treated CLINICAL: Intended: length of stay
Notes	FINANCIAL SUPPORT:Funding: Research Activity Committee of the Reinier de Graaf Hospital (project 620604). Competing Interests: no information ADDITIONAL DATA: email response and additional files (protocol) from authors
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	By lab number
Allocation concealment (selection bias)	High risk	Not concealed
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	States missing information was retrieved from records
Selective reporting (reporting bias)	Low risk	Outcomes on all patients
Other bias	Low risk
Baseline Outcomes similar?	Unclear risk	No information
Free of contamination?	Low risk	Rapid reporting for intervention only
Baseline characteristics similar?	Low risk	Table 1

Woodward 1987.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: inpatient prescribing of all antibiotics SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: educational meetings; restrictive by expert approval, automatic stop order after 72 hours' treatment, and by removal from formulary Intervention Functions: education, restriction DELIVERER: specialist physician (ID) DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING and FINANCIAL: total antibiotic costs and average antibiotic cost per day
Notes	FINANCIAL SUPPORT: Funding: administration of Barnes Hospital. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk	25 months' pre‐ and 17 months' postintervention data
Analysed appropriately (ITS) ?	Low risk	Done in original paper: ordinary least squares regression analysis adjusting for pre‐existing time trends, re‐analysis with segmented regression performed for the purposes of comparison of effect size with other studies in the review.
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Unclear risk	The abstract states: "Even after some cost increases (not significant) in new and other antibiotics, the program saved $1.33 per antibiotic day", but it is not clear whether the analysis was adjusted for changes in the price of antibiotics during the 3½‐year study period.

Wyatt 1998.

Methods	STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM
Participants	PROVIDERS: a total of 25 hospitals, 13 control and 12 intervention, targeting 2 providers (lead obstetrician and senior midwife manager) in each hospital PARTICIPANTS: 1318 episodes of care in 1318 patients, 25 clusters (hospitals) CLINICAL PROBLEM: administration of prophylactic antibiotics to women undergoing Caesarean section. The intervention also targeted 3 other care processes. SETTING: 25 district general (non‐teaching) hospitals POWER CALCULATION: As only 25 obstetric units were available for randomisation, and accurate baseline figures for the rates and variability of the 4 marker clinical practices were not available, sample size calculation was not carried out.
Interventions	FORMAT: educational meeting with dissemination of guideline and slides COMPARISON: 13 control hospitals with no intervention DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Exposure: % women that received antibiotic prophylaxis
Notes	FINANCIAL SUPPORT: Funding: regional research implementation initiatives of the North Thames and South Thames regional health authorities; the Imperial Cancer Research Fund; and North Staffordshire Hospital Trust. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Obstetric units were allocated to intervention or control group by the toss of a coin.
Allocation concealment (selection bias)	Low risk	To eliminate bias during data collection at follow‐up by a second research midwife, and to allow blinded assessment of guideline quality, the allocation was concealed from everyone except JCW, DGA, RJ, and the first research midwife.
Blinding (performance bias and detection bias) All outcomes	Low risk	To eliminate bias during data collection at follow‐up by a second research midwife, and to allow blinded assessment of guideline quality, the allocation was concealed from everyone except JCW, DGA, RJ, and the first research midwife.
Incomplete outcome data (attrition bias) All outcomes	Low risk	"No unit was excluded after randomisation, all intervention units participated in the visits, and data on clinical practices were available for all units, although smaller numbers of case notes were obtainable than planned for steroid usage"
Selective reporting (reporting bias)	Low risk	See above
Other bias	Low risk	"To reduce the impact of ceiling effects, the proportion of cases in which clinicians failed to carry out each clinical practice was recorded for each obstetric unit at baseline and follow up, and then baseline to follow up ratios were computed to yield the risk ratio for failure to implement each practice in each unit."
Baseline Outcomes similar?	Unclear risk	"Accurate baseline figures for the rates and variability of the four marker clinical practices were not available"
Free of contamination?	Low risk	Randomisation by units that were located in different hospitals
Baseline characteristics similar?	Low risk	"Despite randomisation there were baseline differences in two of the four clinical practices" (use of ventouse and use of polyglycolic acid sutures). "There were no other baseline differences." (includes antibiotic prophylaxis)

Yealy 2005.

Methods	STUDY DESIGN: cluster RCT, hospital level Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the ED PARTICIPANTS: 2075 patients admitted from ED (849 intervention, 1227 control), 32 clusters (EDs) CLINICAL PROBLEM: community‐acquired pneumonia SETTING: 32 EDs in the USA
Interventions	FORMAT: low‐intensity (control, 8 hospitals); moderate‐intensity (12 hospitals); and high‐intensity (12 hospitals) interventions Low‐intensity intervention: audit and feedback of baseline data; dissemination of guidelines Low‐intensity invervention functions: education, enablement Moderate‐intensity intervention: same as low intensity, but with additional on‐site educational meeting before patient enrolment Moderate‐intensity intervention additional function: education High‐intensity intervention: same as moderate with additional audit and feedback of data about management of individual patients within a week of enrolment plus 2 monthly feedback of group performance data; educational outreach through academic detailing with Plan Do Study Act cycles to discuss actions to be taken in response to group performance data High‐intensity intervention additional functions: education, enablement, persuasion DELIVERER: departmental physicians COMPARISON: usual care DESIRED CHANGE: increase effective: 4 process measures including time to first antibiotic dose POWER CALCULATION: Primary outcome was site of treatment rather than the antibiotic process measures. "We estimated that we would need 96 eligible patients per hospital (3072 in total) to achieve 80% power to detect a 12% difference across the intervention groups for the site‐of‐treatment decision among low‐risk patients." "For the site‐of‐treatment decision, this study achieved greater than 80% power to detect differences of 10% between high‐intensity and moderate‐intensity groups and differences of 12% between high‐intensity and low‐intensity groups according to separate 1‐tailed tests in which the level was 0.025."
Outcomes	PRESCRIBING: Choice: time to first antibiotic dose and choice compliant with guideline CLINICAL: Intended: mortality and medical complications
Notes	INSTRUCTIONS: action plan provided, no explicit target FINANCIAL SUPPORT: Funding: Agency for Healthcare Research and Quality (grant number R01 HS10049). National Institute of Allergy and Infectious Diseases (grant number K24 AI001769). Competing Interests: 1 author received consultancies, honoraria or grants from Genesoft Pharmaceuticals, Zynx Health Corporation, Healthcare Communications Inc., Stephen Lynn Klein, Kellogg Grants, and Pfizer Inc. ADDITIONAL DATA: email response from authors to request for additional data with care pathway, slide sets, order sheets, and protocol (Yealy 2004)
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	"After stratifying emergency departments by state, teaching status, and annual volume, our statistician randomly assigned these departments to low‐intensity, moderate‐intensity, and high‐intensity guideline implementation strategies in the ratio of 2:3:3, respectively"
Allocation concealment (selection bias)	Low risk
Blinding (performance bias and detection bias) All outcomes	High risk	Not blinded
Incomplete outcome data (attrition bias) All outcomes	Low risk	Incomplete chart review on only 19 (0.6%) of 3219 patients
Selective reporting (reporting bias)	Low risk
Other bias	Low risk	"The target sample size included an adjustment of 30% to account for the clustering of patients within providers."
Baseline Outcomes similar?	Unclear risk	No data
Free of contamination?	Low risk	Cluster RCT
Baseline characteristics similar?	Low risk	Demographic characteristics differed between eligible patients who were and were not enrolled. Moreover, authors observed some imbalances in levels of illness severity across the intervention groups; however, their analyses of the site of treatment were performed separately for low‐risk and higher‐risk patients, and their multivariable analyses were not sensitive to the few imbalances that were observed at baseline.

Yeo 2012.

Methods	STUDY DESIGN: CITS Risk of Bias: LOW
Participants	PROVIDERS: all physicians PARTICIPANTS: all patients receiving therapeutic antibiotics CLINICAL PROBLEM: use of all carbapenems (ertapenem, imipenem, and meropenem), 3rd‐ and 4th‐generation cephalosporins (ceftriaxone, ceftazidime, and cefepime), piperacillin/tazobactam, and vancomycin SETTING: 1 cancer hospital in Singapore
Interventions	FORMAT: Interventions: audit and feedback; educational outreach by review and recommend change Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: 1 author received research funding and speaker’s honoraria from Pfizer, AstraZeneca, Janssen‐Cilag, and Merck Sharp & Dohme. ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Low risk
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Prescribing outcome in DDD from pharmacy computer
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Prescribing outcome in DDD from pharmacy computer
Incomplete outcome data addressed (ITS) ?	Low risk	Prescribing outcome in DDD from pharmacy computer
Free of selected reporting (ITS) ?	Low risk	Prescribing outcome in DDD from pharmacy computer
Free of other bias (ITS) ?	Low risk	Same 11 months of data (Aug‐Jun) in consecutive years pre‐ and postintervention

Yong 2010.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients receiving therapeutic antibiotics CLINICAL PROBLEM: use of broad‐spectrum antibiotics (3rd‐ and 4th‐generation cephalosporins, aminoglycosides, antipseudomonal penicillins, carbapenems, fluoroquinolones) SETTING: 1 hospital in Australia
Interventions	FORMAT: Interventions: structural, computerised decision support system Intervention Functions: enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of broad‐spectrum antibiotics in DDD/1000 OBD
Notes	FINANCIAL SUPPORT: Funding: Victorian Department of Human Services Quality Branch and Australian Commonwealth Biotechnology Information Fund, which funded the development of Guidance DS. Competing Interests: none declared ADDITIONAL DATA: email from authors with additional data about intervention (Richards 2003; Thursky 2006) Microbial Risk of Bias: MEDIUM (Other infection control High)
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Acinetobacter outbreak during intervention period resulting in hand hygiene and staff education interventions. Also see Table 4.
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	> 1 year data pre‐ and postintervention, so low risk for prescribing outcome Microbial Risk of Bias: MEDIUM Case definition Low, % susceptibility of Pseudomonas isolates, Planned intervention Low for outcome (outbreak was of Acinetobacter), Other infection control High, enhanced during prescribing intervention

Yoon 2014.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: requiring therapeutic antibiotics and receiving carbapenems SETTING: 1 university hospital in Korea, same hospital as Kim 2008
Interventions	FORMAT: Intervention 1: restrictive by expert approval (same intervention format as Kim 2008) Intervention 1 functions: restriction Intervention 2: addition of reminders (circumstantial, electronic triggered by computerised antibiotic order, the system is described in more detail in Kim 2008) Intervention 2 functions: enablement, environmental restructuring, persuasion DELIVERER: AMT COMPARISON: usual care DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of carbapenems in DDD/1000 OBD MICROBIAL: infections with CRAB (carbapenem‐resistant Acinetobacter baumanii)/1000 OBD CLINICAL: Balancing measures of adverse effects, all‐cause mortality
Notes	FINANCIAL SUPPORT: Funding: commercial, Merck Sharp & Dohme. Competing Interests: supported by Merck Sharp & Dohme Corp. ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition Low, planned intervention Low, other infection control High, ICU cleaning intervention during Phase 3
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	High risk	Intensive environmental cleaning implemented in 2012 in ICU, which was intended to reduce infections with CRAB (microbial outcome).
Analysed appropriately (ITS) ?	Low risk	Segmented regression analysis
Shape of effect pre‐specified (ITS) ?	Low risk	Point of intervention was point of analysis.
Unlikely to affect data collection (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Data from pharmacy and microbiology computers
Incomplete outcome data addressed (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of selected reporting (ITS) ?	Low risk	Data from pharmacy and microbiology computers
Free of other bias (ITS) ?	Low risk	> 1 year data in each study phase

Young 1985.

Methods	STUDY DESIGN: ITS Risk of Bias: MEDIUM
Participants	PROVIDERS: all physicians in the hospital PARTICIPANTS: all patients in the hospital CLINICAL PROBLEM: patients requiring aminoglycoside antibiotic treatment SETTING: 1 hospital in the USA
Interventions	FORMAT: Interventions: restrictive by review and make change (substitution of amikacin for gentamicin) and expert approval from the Infectious Diseases Division Intervention Function: restriction DELIVERER: pharmacist DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: gentamicin usage as a percentage of total aminoglycoside usage
Notes	FINANCIAL SUPPORT: Funding: Veterans Adminstration and Bristol‐Myers Squibb. Competing Interests: none declared, but Bristol‐Myers Squibb was the manufacturer of amikacin ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Intervention independent (ITS) ?	Unclear risk	3 months' data before, 15 months' during, and 22 months' after the restriction. Not enough data to adjust for seasonal variation.
Analysed appropriately (ITS) ?	Low risk	Re‐analysed. Not done in original paper: comparison of means (uncontrolled before‐after).
Shape of effect pre‐specified (ITS) ?	Low risk	Done, intended effect was decrease in primary outcome, and point of analysis was point of intervention.
Unlikely to affect data collection (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Knowledge of the allocation adequately prevented(ITS)?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Incomplete outcome data addressed (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of selected reporting (ITS) ?	Low risk	Done, data were from routine systems and unlikely to change over study period.
Free of other bias (ITS) ?	Low risk	No other apparent biases found.

Yu 2014.

Methods	STUDY DESIGN: CBA Risk of Bias: HIGH
Participants	PROVIDERS: all physicians PARTICIPANTS: all patients CLINICAL PROBLEM: patients receiving therapeutic antibiotics SETTING: 5 hospitals in an integrated healthcare system in the USA
Interventions	FORMAT: Intervention: educational outreach through review and recommend change in 2 hospitals Intervention Functions: enablement, persuasion DELIVERER: AMT COMPARISON: usual care in 3 hospitals DESIRED CHANGE: decrease excessive
Outcomes	PRESCRIBING: Choice: use of target antibiotics in DDD/1000 OBD CLINICAL: hospital standardised mortality ratio MICROBIAL: Clostridium difficile infection rates FINANCIAL: total and direct acquisitional cost of targeted antimicrobials
Notes	FINANCIAL SUPPORT: Funding: none. Competing Interests: none declared ADDITIONAL DATA: no response from authors to request for additional data Microbial Risk of Bias: HIGH case definition Not Clear, planned intervention Low, other infection control measures Not Clear
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Study sites selected from baseline antimicrobial use.
Allocation concealment (selection bias)	High risk	No concealment
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Data from pharmacy computer
Selective reporting (reporting bias)	Low risk	Data from pharmacy computer
Other bias	Low risk
Baseline Outcomes similar?	High risk	Table 2
Free of contamination?	Low risk	Intervention and control sites different hospitals
Baseline characteristics similar?	High risk	Several potentially important differences between intervention and control sites

Zanetti 2003.

Methods	STUDY DESIGN: NRT Risk of Bias: HIGH
Participants	PROVIDERS: all surgeons in the hospital PARTICIPANTS: 331 patients undergoing cardiac surgery CLINICAL PROBLEM: additional dose of antibiotic prophylaxis for operations that lasted more than 4 hours SETTING: 1 hospital in the USA
Interventions	FORMAT: Intervention: dissemination of guideline; reminder (circumstantial, electronic, automated intra‐operative alert) Intervention Functions: education, enablement, environmental restructuring, persuasion COMPARISON: control group plus 480 patients from the 6 months before the study period DESIRED CHANGE: increase effective
Outcomes	PRESCRIBING: Exposure: % patients who received additional intra‐operative antibiotics CLINICAL: Intended: wound infection rate
Notes	FINANCIAL SUPPORT: Funding: Centers for Disease Control and Prevention cooperative agreement, UR8/CCU115079, University Hospital of Lausanne, and the Leenaards Foundation. Competing Interests: no information ADDITIONAL DATA: no response from authors to request for additional data
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	High risk	Based on a case number assigned to every surgical procedure performed in the hospital, independent of the study itself
Allocation concealment (selection bias)	High risk	No concealment
Blinding (performance bias and detection bias) All outcomes	High risk	No blinding
Incomplete outcome data (attrition bias) All outcomes	Low risk	Outcome on all 273 patients
Selective reporting (reporting bias)	Low risk	Outcome on all 273 patients
Other bias	Low risk	Outcome on all 273 patients
Baseline Outcomes similar?	Low risk	Cohort data before start of trial
Free of contamination?	High risk	Control patients were operated on by the same surgeons, and the reminder for intervention patients is likely to have increased awareness of the need for additional doses.
Baseline characteristics similar?	Low risk	Table 1

AB: antibiotic
 AKI: acute kidney injury
 AMT: multidisciplinary antibiotic management team
 APACHE: Acute Physiology and Chronic Health Evaluation
 ARGNB: antibiotic‐resistant gram‐negative bacilli
 ARGPB: antibiotic‐resistant gram‐positive bacilli
 ARIMA: autoregressive integrated moving average
 ASP: Antimicrobial Stewardship Program
 BCT: behaviour change technique
 CAP: community‐acquired pneumonia
 CBA: controlled before‐after study
 CBC: complete blood count
 CDAD: Clostridium difficile‐associated diarrhoea
 CDC: Centers for Disease Control and Prevention
 CDI: Clostridium difficile infection
 CDSS: clinical decision support system
 CI: confidence interval
 CITS: comparative interrupted time series
 CPIS: clinical pulmonary infection score
 CRP: C‐reactive protein
 C‐section: Caesarean section
 DACT: double anaerobic coverage therapy
 DDD: defined daily dose
 DRG: diagnosis‐related group
 ED: emergency department
 EPOC: Effective Practice and Organisation of Care
 ER: emergency room
 ESBL‐EB: extended‐spectrum beta‐lactamase‐producing Enterobacteriaceae
 FTE: full‐time equivalent
 GRE: glycopeptide‐resistant enterococci
 IC: infectious control
 ICD: International Classification of Diseases
 ICU: intensive care unit
 ID: infectious diseases
 IDP: infectious diseases physician
 IHC: Intermountain Healthcare
 IL‐8: interleukin‐8
 ITS: interrupted time series
 IQR: interquartile range
 IV: intravenous
 LOS: length of stay
 MRSA: methicillin‐resistant Staphylococcus aureus
 MSSA: methicillin‐sensitive Staphylococcus aureus
 LRTI: lower respiratory tract infection
 MICU: medical intensive care unit
 NHAP: nursing home‐acquired pneumonia
 NIH: National Institutes of Health
 NRT: non‐randomised (controlled) trial
 NRSI: non‐randomised studies of interventions
 OBD: occupied bed day
 OR: odds ratio
 PA: parenteral antibiotics
 PCR: polymerase chain reaction
 PCT: procalcitonin
 RCT: randomised controlled trial
 RCOG: Royal College of Obstetricians and Gynaecologists
 RDD: recommended daily doses
 ROB: risk of bias
 ROBINS‐I: risk of bias in non‐randomised studies of interventions
 RR: risk ratio
 SCIP: Surgical Care Improvement Project
 SD: standard deviation
 SE: standard error
 SHEA: Society for Healthcare Epidemiology of America
 SICU: surgical intensive care unit
 SNF: skilled nursing facilities
 SSI: surgical‐site infection
 TFAD: time to first antibiotic dose
 TREAT: computerised decision support system for antibiotic treatment
 UBA: uncontrolled before‐after study
 VAP: ventilator‐associated pneumonia
 VRE: vancomycin‐resistant enterococci

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Ahronheim 2000	RCT with no relevant data. Antibiotics were only part of a complex care plan for 6% of participants in the intervention group, and the outcome data do not include information about the effect of the intervention on antibiotic prescribing.
Bruno‐Murtha 2005	ITS of antibiotic cycling with no interpretable data because there are no pre‐cycling data. Only provides data for 4 phases of cycling.
Burke 1997	ITS with no interpretable data. 2 different interventions (education, then restriction via order form) with 3 points before the education intervention and 3 after, but the restriction intervention started after the 4th point.
Cook 2006	ITS with no interpretable data because no clearly defined point in time at which the intervention started.
Crist 1987	NRT with no interpretable data. Unacceptable allocation bias ("the allocation of a patient to a particular group was determined by the attending physician").
Cunningham 2008	ITS with no relevant data. The only valid outcome data are about compliance with a guideline about generic documentation of prescription rather than any specific antibiotic prescribing outcome. The data about time to first antibiotic dose are UBA.
Dellinger 2005	ITS with no interpretable data because no clearly defined point in time at which the intervention started. Only 4 data points for antibiotic use, and the intervention included multiple components in addition to antibiotic use, so even if an intervention effect could be calculated reliably it could not be attributed to change in antibiotic prescribing.
Destache 1990	RCT with no interpretable data because of incomplete and selective reporting of outcome data. The primary outcome measure was length of stay, but 32% of participants in the intervention group were excluded because they had prolonged length of stay.
Ehrenkranz 1992	RCT with no interpretable data. Only report data for participants whose physicians followed recommendations.
Ehrenkranz 1993	RCT with no interpretable data. Only report data for participants whose physicians followed recommendations.
Evans 1994	NRT with no interpretable data. The first part compared the drugs that the Antibiotic Consultant programme recommended, with the drugs actually prescribed by physicians. Data from the second part are presented in an uninterpretable format, with the denominator as cultures, not participants or physicians.
Foy 2004	Cluster RCT with no relevant data. Intervention targeted 5 care processes for women having an abortion. Only 1 included antibiotic prescribing within a composite (antibiotic prophylaxis or screening for lower genital tract organisms). Effect of intervention on prescribing cannot be estimated.
Garcia‐San Miguel 2014	Cluster RCT with no interpretable data. The study included 9 hospitals with 32 hospitalisation units (wards). Patients were included if they had drugs dispensed from an electronic system. Baseline: Jan‐June 2003 baseline, no intervention 1. Jan‐June 2004, intervention in half of the wards that were randomised in each hospital 2. Jan‐June 2005, cross‐over, intervention in wards that were randomised to control in Period 2 There is no description of the randomisation process. The primary outcome measure was adherence to recommendations; text on page 658 says they do not present data about mortality or re‐admission, but that appears to be what is in Figure 4. Figure 4: legend (and text) says it is about DDD and cost of drugs, but labelling says it is mortality and re‐admission. We asked authors to clarify and provide valid outcome data but received no reply.
Gerding 1991	ITS with no interpretable data. Describes 10 years of experience with aminoglycoside cycling, but the intervention periods cannot be mapped onto the outcome data about prescribing or resistance.
Kolar 1999	ITS with no interpretable data due to inadequate control for the effect of other interventions (infection control measures; see detailed critique by Monnet 2000).
Lan 2003	ITS with unacceptable missing data and inappropriate statistical analysis. There are 3 monthly data points pre‐intervention, then a gap in colonisation data for 3 months at the start of the intervention period followed by 3 monthly data points from months 4 to 6 of the intervention phase.
Lee 2004	ITS with no interpretable data. There were no isolates of ESBL‐Klebsiella pneumoniae in the last 3 months of the intervention phase, but no data are provided about the number of specimens screened. Appropriate statistical analysis in original paper not done (averages pre‐ and postintervention with χ2 and Fisher's exact test). Re‐analysis not possible because there are only 2 reliable data points in the postintervention phase.
MacCosbe 1985	RCT with no interpretable data. Only 29% of randomised doctors were followed up, and recommendations were only made in 6% of the intervention group.
Marrie 2000	Cluster RCT with no relevant data. Antibiotic prescribing was only 1 component of a care pathway, results for impact on antibiotic prescribing and its contribution to outcome not reported separately.
Martin 2005	ITS with no interpretable data. No antibiotic data pre‐intervention, only data about MRSA; this information is uninterpretable without information about pre‐intervention antibiotic prescribing.
McGregor 2006	RCT with no interpretable data. Statistical analysis of primary outcome measure (antibiotic costs) not done, and re‐analysis not possible from the data presented.
Nagao 2010	ITS with no interpretable data. Figure 1 reports the number of participants with inappropriate antibiotic use, consultations, significant laboratory test results, and total number of blood cultures obtained. However, the number of participants in each category is not clear in the figure. We asked the authors for raw data but they were unable to provide this information.
Naughton 2001	Cluster RCT in 10 skilled nursing facilities. The intention was to increase use of IV antibiotics for severe pneumonia. The comparison was between the same intervention delivered by a multidisciplinary team (intervention) versus a physician (control). There was no difference in the intervention effect, but the study provides no reliable evidence of intervention effect (UBA data in all 10 skilled nursing facilities).
Pastel 1992	NRT in 1 hospital, no interpretable data because no protection against contamination and unreliable primary outcome measure.
Ronning 1998	RCT with no relevant data. Not primarily an intervention on antibiotic therapy, compared stroke unit versus general medical ward.
Sanazaro 1978	NRT with no relevant data. Antibiotic prescribing was only 1 of 3 components of a care pathway, results for impact on antibiotic prescribing and its contribution to outcome not reported separately.
Takahashi 2010	ITS with no interpretable data. The only time series data (Figures 2 and 3) are MRSA and Pseudomonas aeruginosa infections. The paper claims that a prophylaxis intervention in early 2007 was responsible for reduction in P aeruginosa and MRSA infections, whereas the figures clearly show the reduction happened between July and December 2006. The paper does not include valid data about prescribing outcomes, and the authors were unable to provide these data.
Thomas 2002	CBA in 64 hospitals, no interpretable data because no clear point in time for the intervention.
Tiley 2003	ITS with no interpretable data. Multiple interventions are described without clear definition of intervention points.
Tsiata 2001	RCT with no interpretable data. These are provider interventions, but allocation was by patient randomisation. The unequal numbers of patients in each group (134 Group A, 141 Group B, and 105 Group C) and the differences in baseline characteristics indicate unacceptable allocation bias.
Van Loon 2005	ITS with no interpretable data about the impact of antibiotic cycling on resistance because there are no pre‐cycling data.
Wahlstrom 2003	RCT with no relevant data. Antibiotics included in the indicators for treatment of hospitalised cases of pneumonia (compliance with policy, dose and duration) and diarrhoea (no use of antibiotics without bacterial identification), but no separate data are presented for these outcomes. The only data provided are mean scores on a single composite indicator for each condition.

CBA: controlled before‐after study
 DDD: defined daily dose
 ESBL: extended‐spectrum beta‐lactamase
 ITS: interrupted time series
 MRSA: methicillin‐resistant Staphylococcus aureus 
 NRT: non‐randomised trial
 RCT: randomised controlled trial
 UBA: uncontrolled before‐after study

Differences between protocol and review

The protocol was completely revised for this update of the review. The most notable changes to the original protocol used for the first version of the review are as follows.

1. We amended the main outcome of interest to reflect desired change in practice. This fits better with the overall objective of the review relating to appropriate prescription in order to provide evidence of better targeting of antibiotic prescribing.

2. We changed the measure of effect from risk ratios to risk differences to better convey the intervention effect in absolute terms.

3. We adjusted for the effect of clustering in sensitivity analyses, as we had not considered this aspect of trial design in the previous version of the review.

Contributions of authors

Peter Davey (Clinical Pharmacologist) wrote the protocol; assisted with the literature search; reviewed all intervention studies for risk of bias using Cochrane Effective Practice and Organisation of Care (EPOC) Group methodology; contributed to re‐analysis of data from interrupted time series (ITS) studies and meta‐regression of ITS studies and randomised controlled trials (RCTs); wrote the first draft of the review and was responsible for final decisions about included studies; contributed to EPOC check sheets, data extraction, and GRADE assessment of certainty of evidence.
 
 Charis Marwick (Infectious Diseases Physician) re‐analysed all of the ITS studies and performed meta‐regression of ITS studies and RCTs with an analysis plan written by Craig Ramsay (Statistician); was a member of the review writing group; and contributed to EPOC check sheets, data extraction, and GRADE assessment of certainty of evidence.

Claire Scott (Psychologist) managed the review; set up the database; was a member of the review writing group; and contributed to EPOC check sheets, data extraction, and GRADE assessment of certainty of evidence.

Esmita Charani (Pharmacist) and Kirsty McNeil (Medical Student) were members of the review writing group and contributed to EPOC check sheets, data extraction, and GRADE assessment of certainty of evidence.

Erwin Brown (Medical Microbiologist) initiated the review in 2000 and for this update handsearched bibliographies of individual papers for additional references; screened titles and abstracts; and reviewed all papers to identify those that reported the results of an intervention to change antibiotic prescribing.

Ian Gould (Medical Microbiologist) reviewed papers for microbial risk of bias and was a member of the review writing group.
 
 Craig Ramsay (Statistician) wrote the analysis plan for re‐analysis of ITS studies and meta‐regression of ITS studies and RCTs.

Susan Michie (Psychologist) advised on the design of data extraction for behaviour change techniques and the analysis of intervention functions; was a member of the review writing group; and contributed to GRADE assessment of certainty of evidence.

Sources of support

Internal sources

• Aberdeen Royal Infirmary, Aberdeen, Scotland, UK.

• Imperial College, London, England, UK.

• University of Dundee, Dundee, Scotland, UK.

• University of Aberdeen, UK.

• UK Cochrane Centre, UK.

• University College London, UK.

External sources

• British Society for Antimicrobial Chemotherapy, UK.

• Chief Scientist Office for Scotland, UK.

  Major grant CZH4861

Declarations of interest

Peter Davey is an author of four of the included studies. Charis Marwick is an author of two of the included studies. Ian Gould is an author of one of the included studies. Craig Ramsay is an author of one of the included studies. Other review authors completed data extractions for these studies. The institutions of the following authors received funding from the Chief Scientist Office that helped to support the conduct of this review: Peter Davey, Charis Marwick, Esmita Charani.

Peter Davey, none other than as indicated above.
 Charis Marwick, none other than as indicated above.
 Claire Scott, none other than as indicated above.
 Esmita Charani, none other than as indicated above.
 Kirsty McNeil, none other than as indicated above.
 Susan Michie, none other than as indicated above.
 Erwin Brown, none other than as indicated above.
 Ian Gould, none other than as indicated above.
 Craig Ramsay, none other than as indicated above.

Edited (no change to conclusions)