Feedback of Antibiotic Prescribing in Primary Care (FAPPC) trial: results of a real-world cluster randomized controlled trial in Scotland, UK

Abstract

Objectives

To evaluate the effect of general practice-level prescribing feedback on antibiotic prescribing in a real-world pragmatic cluster randomized controlled trial

Methods

Three hundred and forty general practices in four territorial Health Boards in NHS Scotland were randomized in Quarter 1, 2016 to receive four quarterly antibiotic-prescribing feedback reports or not, from Quarter 2, 2016 to Quarter 1, 2017. Reports included different clinical topics, benchmarking against national and health board rates, and behavioural messaging with improvement actions. The primary outcome was total antibiotic prescribing rate. There were 16 secondary prescribing outcomes and 5 hospital admission outcomes (potential adverse effects of reduced prescribing). The main evaluation timepoint was 1 year after the final report (Quarter 1, 2018), with an additional evaluation in the quarter after the final report (Quarter 2, 2017). Routine administrative NHS data were used to generate the feedback reports and analyse the effects.

Results

Total antibiotic prescribing rates were lower at the main evaluation timepoint in both intervention (1.83 versus baseline 1.93 prescriptions/1000 patients/day) and control (1.90 versus baseline 1.98) practices, with no evidence of intervention effect [adjusted rate ratio (ARR) 0.98 (95% CI 0.94–1.02; P = 0.35)]. At the additional timepoint, adjusted total antibiotic prescribing rates were 1.67 and 1.73 prescriptions/1000 patients/day, with evidence of a small intervention effect, ARR 0.99 (0.98–1.00; P = 0.03).

Conclusions

This well-designed, practice-level antibiotic-prescribing feedback had limited evidence of additional effects in the context of decreasing antibiotic prescribing and an established national stewardship programme.

Introduction

Antibiotic use in humans is a key driver of emerging antibiotic resistance. Antimicrobial stewardship interventions and programmes have been widely implemented to reduce inappropriate use of antibiotics. The Scottish Antimicrobial Prescribing Group (SAPG), established in 2008, provides a national framework for antimicrobial stewardship.¹ SAPG-coordinated stewardship initiatives, delivered locally by NHS health board antimicrobial management teams and prescribing support teams, have been associated with considerable reductions in primary care antibiotic prescribing but increasing targets for reductions in antibiotic prescribing within the UK continue to challenge.²

Feedback of practice is a common component of healthcare improvement interventions and feedback of prescribing rates can be facilitated by the availability of routine electronic data capture in many contexts.³ In UK primary care feedback identifying high-risk prescribing, such as combinations of chronic medications associated with increased risk of gastrointestinal bleeding or acute kidney injury, has been very effective in randomized controlled trials.^4–6 Antibiotics are almost always prescribed acutely for short courses with associated risks more distant from the acute prescription. Feedback of antibiotic prescribing has had more mixed effects internationally, with limited effectiveness in reducing total antibiotic prescribing.^7–13 However, a large-scale general practice trial (1581 practices randomized) in England targeted the 20% of practices with the highest prescribing rates in each local area and achieved an 3.3% reduction in intervention practices, an estimated 73 406 fewer antibiotic items dispensed.¹⁴ Low-cost interventions that can be delivered at scale can thus have a large overall impact despite a relatively small absolute effect.

A systematic review of audit and feedback as a healthcare improvement strategy reported variable effects but feedback may be more effective when it includes both explicit targets and an action plan.¹⁵ Primary care prescribing feedback trials have included intervention arms with and without behaviour change components, including action planning, and reported increased effectiveness when this was included.^6,16,17 There are limited antibiotic prescribing trials using this approach in the literature but it was effective in a primary care dental study in Scotland.¹⁷

SAPG provided reports on antibiotic prescribing including a range of quality indicators at national and Health Board level for the 14 regional NHS Health Boards in Scotland since soon after its inception in 2008 and these have been generated using the Prescribing Information System (PIS) since 2009.¹⁸ Prescribing support teams within Health Boards could also utilize the PIS to generate general practice-level reports to support engagement with prescribers on antibiotic use. This combination of national and bespoke local reporting approaches is not conducive to evaluating the effect on prescribing practice. The Feedback on Antibiotic Prescribing in Primary Care (FAPPC) trial reported here involved SAPG providing feedback directly to practices, with action planning, for the first time. A randomized design was applied to facilitate evaluation and inform decisions on continuation and national roll-out.

The aim of the FAPPC trial was to evaluate the effect in Scottish primary care of actionable, practice-level antibiotic prescribing feedback on rates of primary care antibiotic prescribing. A secondary aim was to examine changes in hospital admissions with infection, a potential unintended consequence of change in prescribing practice.

Methods

Study design

The design was a two-arm cluster randomized controlled trial with general practice as the unit of randomization and analysis. The trial was highly pragmatic, and embedded in existing information and stewardship systems, to rigorously evaluate the effect of an NHS-led intervention.

Participants

All primary care general medical practices (‘practices’) located in 4 of the 14 territorial NHS Health Boards in Scotland were eligible, except practices: with <250 registered patients (typically very unusual practices e.g. serving homeless or very remote populations); with missing list size, age, gender or deprivation categories, and/or missing prescribing data required for stratification, and/or which ceased to exist or merged with another practice (in different arm) during the trial. Data analysed included all patients registered with each practice. All eligible practices were randomized in Quarter 1, 2016 to receive the intervention or not, with no requirement for active recruitment or consent.

Intervention

The intervention consisted of four feedback reports containing each practice’s quarterly antibiotic prescribing rates, with comparison to Health Board and national benchmarks, at 25th percentiles. Practice and benchmarking rates were presented as quarterly time series for the 4 years prior to the report issue date. The reports incorporated behaviour change techniques associated with increased effectiveness of feedback,¹⁵ and hospital-based antibiotic prescribing interventions.¹⁹ These included providing repeated feedback from a credible source (authoritative NHS organization) and clear guidance on expected behaviour with target-setting (local and national benchmarks). Reports also included educational information and links to resources produced by SAPG and the Royal College of General Practitioners (see Supplementary data, available at JAC Online).

Reports were delivered by e-mail from an NHS National Services Scotland (NSS) e-mail address to all intervention practices, with a cover letter signed by the Scottish Government’s Chief Medical Officer and the Chair of the Scottish Antimicrobial Prescribing Group (see Supplementary data). Control practices did not receive intervention feedback reports but links to the educational resources are freely available via the Scottish Antimicrobial Prescribing Group website,¹ and practices in both arms received continuing national and local antimicrobial stewardship interventions, which may have included locally produced prescribing feedback. Such feedback could not have included the national benchmarking data, and we are not aware of any that used the clinical themes and behavioural messaging that were key components of FAPPC feedback reports. No practices were aware this was a randomized trial and only practices open for the duration of the trial were included in analysis. The only potential source of contamination was individual GPs moving between intervention and control practices during the trial, but it is unlikely that this would significantly impact on overall results (no data were available on GP movement).

The data used to generate the reports are held within the NHS NSS Prescribing Information System (PIS), which is used for reimbursement of pharmacies for all dispensed NHS primary care prescriptions.¹⁸ Antibiotic ‘prescriptions’ in feedback reports and trial outcome analyses were defined as dispensed prescribed items for any systemic drugs in the British National Formulary chapter 5.1 (Antibacterials), excluding 5.1.9 (drugs for tuberculosis) and 5.1.10 (drugs for leprosy).

The first report was distributed in calendar Quarter 2 of 2016 (Figure 1). Every report contained rates of total antibiotic prescribing (prescriptions per 1000 registered patients per day), with each report then including a different subset of antibiotic prescribing. The subject for each report was agreed by SAPG and targeted areas of high antibiotic use in primary care and/or specific national stewardship priorities at that time: Report 1—children and older people; Report 2—treatment and prophylaxis of urinary tract infections; Report 3—broad-spectrum antibiotics associated with increased risk of Clostridioides difficile infection; and Report 4—treatment for skin and soft tissue infections (Table 1).

Figure 1.

FAPPC trial timeline: baseline measurement, feedback reports and outcome analysis.

Table 1.

Antibiotic prescribing measures and rates included in each feedback report

Report date and topic	Measure	Intended direction of change	Rate
1. Quarter 2, 2016: children and older people	All antibiotics in all age groups	Decrease	Number of prescriptions per 1000 registered patients (in age group) per day
	All antibiotics, 0–4 years
	All antibiotics, 5–64 years
	All antibiotics, >64 years
2. Quarter 3, 2016: treatment or prevention of UTI	All antibiotics in all age groups	Decrease	Number of prescriptions per 1000 registered patients per day
	Trimethoprim 3 day prescriptions	Increase	Percentage of all trimethoprim prescriptions dispensed to adult females that were for a 3 day course
	Prophylaxis against UTI	Decrease	Number of patients aged ≥16 years, per 1000 registered, with six or more prescriptions for trimethoprim, nitrofurantoin, ciprofloxacin or cefalexin in the previous 12 monthsa
3. Quarter 4, 2016: broad-spectrum antibiotics with increased risk of C. difficile	All antibiotics in all age groups	Decrease	Number of prescriptions per 1000 registered patients per day
	Cephalosporins
	Co-amoxiclav
	Fluoroquinolones
4. Quarter 1, 2017: treatment of SSTI	All antibiotics in all age groups	Decrease	Number of prescriptions per 1000 registered patients per day
	Flucloxacillin prescribed at recommended dose	Increase	Prescriptions for flucloxacillin 500 mg capsules as a percentage of all flucloxacillin capsule prescriptions dispensed for adultsb
	Prescriptions for acne or rosacea	Decrease	Number of patients, per 1000 registered patients, with four or more prescriptions for oxytetracycline or lymecycline in the previous 12 monthsc

^aUTI prophylaxis is advised for 6 months maximum and six or more prescriptions in 12 months may indicate prolonged prophylaxis.

^bRepresents the proportion of prescriptions that are at the recommended dose of 500 mg (250 mg capsules may indicate subtherapeutic dosing in adults).

^cAntibiotic prescriptions for acne or rosacea should be reviewed at 6–8 weeks and given for a maximum 8 months. Four or more prescriptions in 12 months may indicate prolonged treatment.

Outcome measures

The primary outcome was the quarterly rate of antibiotic prescriptions per 1000 registered patients per day, using the practice list size in that quarter as the denominator. There were 21 secondary outcome measures. The main evaluation timepoint, specified at trial registration, was 1 year after the final feedback report (Quarter 1, 2018). An additional evaluation timepoint in the calendar quarter immediately following the final feedback report (Quarter 2, 2017), specified after registration but before data extraction, aimed to determine whether there might be transient effects that were not sustained.

Secondary antibiotic outcome measures included rates of prescriptions in three age groups, and prescriptions of 10 individual and 3 groups of antibiotic drugs (grouped by most common clinical indication), all per 1000 registered patients per day, and all specified at trial registration (Box S1 and Table 3).

Table 3.

Adjusted rates and rate ratios for prescribing outcomes (per 1000 registered patients per day) and hospital admission outcomes (per 10 000 patients per quarter, aggregated at Health Board level) at the main analysis timepoint—1 year after the last feedback

	Adjusted rates		Intervention effect
	Intervention	Control	ARR (95% CI)
Primary outcome
All antibiotic prescriptions	1.83	1.90	0.98 (0.94–1.02)
Secondary prescribing outcomes
Antibiotics for patients aged 0–4 yearsa	0.78	0.74	1.05 (0.58–1.19)
Antibiotics for patients aged 5–64 yearsa	0.53	0.52	1.01 (0.97–1.05)
Antibiotics for patients aged ≥65 yearsa	1.29	1.31	0.98 (0.95–1.01)
Amoxicillin	0.52	0.56	0.96 (0.92–1.02)
Phenoxymethylpenicillin	0.12	0.12	1.05 (0.98–1.12)
Flucloxacillin	0.18	0.19	0.93 (0.89–0.99)
Co-amoxiclav	0.06	0.06	1.03 (0.95–1.14)
Doxycycline	0.19	0.20	0.98 (0.92–1.05)
Clarithromycin	0.12	0.13	0.93 (0.84–1.02)
Trimethoprim	0.20	0.19	1.01 (0.95 to 1.06)
Nitrofurantoin	0.12	0.12	0.97 (0.92–1.03)
Ciprofloxacin	0.05	0.05	1.03 (0.94–1.12)
Cefalexin	0.05	0.05	0.99 (0.89–1.10)
Antibiotics commonly used for RTIsb	0.82	0.87	0.98 (0.93–1.03)
Antibiotics commonly used for UTIsc	0.47	0.48	1.00 (0.95–1.04)
Antibiotics commonly used for long-term skin infectionsd	0.09	0.09	1.00 (0.93–1.06)
Hospital admission outcomes
Mastoiditis, peritonsillar abscess, pneumonia or COPD	7.4	7.6	0.97 (0.91–1.02)
Cellulitis or erysipelas	2.3	2.3	0.99 (0.88–1.12)
UTI	2.9	2.8	1.05 (0.85 to 1.32)
Sepsis	4.0	4.2	0.97 (0.87 to 1.08)
Composite of all above infections	14.0	14.4	0.98 (0.93–1.02)

Prescribing analyses used negative binomial regression and admission analyses used Poisson regression, except UTI, which used negative binomial.

^aAdjusted only for baseline rate of antibiotic prescriptions in that age group. All other prescribing analyses were adjusted for practices’ strata for age, deprivation, urban/rural classification and list size, and practices’ baseline prescribing rate for that outcome. Admission outcomes were adjusted only for the baseline rate of those admissions.

^bIncludes amoxicillin, doxycycline, phenoxymethylpencillin.

^cIncludes trimethoprim, nitrofurantoin, ciprofloxacin, cefalexin, co-amoxiclav.

^dIncludes oxytetracycline, lymecycline, minocycline.

Hospital admissions were assessed to detect whether any change in antibiotic prescribing in primary care was associated with change in secondary care presentations with complicated bacterial infections. Outcome measures included rates per 10 000 registered patients per quarter with hospital admissions with four types of bacterial infection [respiratory tract infection (RTI), skin/soft tissue infection (SSTI), urinary tract infection (UTI), sepsis], and a composite measure including all four types (Box S1 and Table 3). The RTI admission outcome was specified at trial registration. The other three infection types and the composite measure were specified after registration but prior to data extraction. Data on all NHS hospital admissions in Scotland are held in the Scottish Morbidity Record 01 (SMR01)²⁰ dataset, hosted by NHS NSS. Data on patients registered with trial practices with admissions with International Classification of Diseases, Tenth Revision (ICD-10)²¹ codes indicating relevant bacterial infections were extracted (Table S1).

Sample size calculation

The sample size calculation used Scottish national data from 2014 and was based on the primary outcome. It assumed a mean practice list size of 5600 patients and a mean baseline antibiotic prescribing rate of 2.05 per 1000 patients per day. There was very large between-practice variation (from 0.19 to 8.35 prescriptions/1000 residents/day), which complicated accurate sample size calculation. An estimated 183 practices per arm were required to detect a 7.5% difference between arms at the 5% significance level, with 80% power. Four territorial Health Boards, including a total of 391 practices, were selected.

Randomization and blinding

Intervention allocation was performed by the Bespoke Services Division, within the Information Services Division of NHS NSS. The randomization used stratified random sampling within each Health Board, with strata based on the mean age, deprivation quintile (Scottish Index of Multiple Deprivation)²² and rurality index (Scottish Government Urban/Rural classification)²³ of registered patients in each practice, and on quintiles of practice list size and total dispensing volume (prescriptions/1000 patients/quarter). Strata that contained only one practice were grouped together, within each Health Board, before randomization. Computer-generated simple random sampling (using CSPLAN function in IBM SPSS Statistics) was applied within each stratum.

The NSS teams allocating practices to intervention or control arms and preparing the reports could not be blinded to practices’ allocation. Practices in both arms were unaware that prescribing feedback was being delivered as a randomized trial. The analysis of trial outcomes was at the University of Dundee and was completely independent to NSS. The statistician was blinded to practice allocation until the analysis was complete.

Statistical analysis

Analyses for all outcomes examined intervention effects 1 year after the final feedback report (Quarter 1, 2018), and at an additional timepoint in the quarter immediately after the final report (Quarter 2, 2017). Analyses were ITT and included all practices that existed until the end of the study. Analyses were at cluster (practice) level, the same as the unit of randomization.

Prescribing outcomes were analysed using negative binomial regression models, an extension of Poisson regression that accommodates over-dispersion. Models included the log of the number of patients per practice as an offset and Health Board as a random variable and were adjusted for practice strata used in randomization (mean patient age, rurality and deprivation indices, and practice list size), and for the baseline prescribing rate (rate in Quarter 1, 2016) for that outcome where possible. Hospital admission outcomes were analysed used Poisson regression with negative binomial extension if required. Due to the small numbers of admissions per practice, data were aggregated at Health Board level, and rates were adjusted for the baseline rate of each outcome only.

Analyses used R (sample size calculation), IBM SPSS Statistics (randomization) and STATA 15 (outcome analysis). A template for intervention description and replication (TIDieR) checklist and CONSORT checklist for reporting a cluster randomized trial have been completed (Tables S2 and S3).

Ethics and registration

This study was reviewed by the East of Scotland Research Ethics Service and NHS Tayside Research Governance, who deemed it service evaluation that did not need ethics committee review.

Trial registration: ISRCTN70810031; https://doi.org/10.1186/ISRCTN70810031.

Results

Randomization and baseline characteristics

Of a total 391 practices in the four Health Boards at the time of randomization, 340 practices were eligible. One hundred and eighty-one practices were randomized to receive the intervention and 159 to normal practice (imbalance resulting from the large number of small strata, with randomization done separately in each stratum). Nine (5.6%) intervention and five (3.1%) control practices were lost to follow-up post-randomization, due to practice closures, with 326 practices analysed (Figure 2). The baseline characteristics of practices, and their patients, were similar in intervention and control groups (Table 2). Health Board had no evidence of effect in initial models so was excluded. The baseline antibiotic prescribing rate was 1.93 prescriptions/1000 patients/day in intervention practices versus 1.98 in control practices.

Figure 2.

Consort diagram of practices in the FAPPC trial.

Table 2.

Baseline characteristics of practices included in analysis

Practice characteristics	Intervention 172 practices	Control 154 practices
List size
Mean (SD)	6244 (3554)	6284 (3428)
Dispensing
Yes	17 (9.9)	16 (10.4)
No	155 (90.1)	138 (89.6)
Health Board
A	28 (16.3)	25 (16.2)
B	43 (25.0)	41 (26.6)
C	41 (23.8)	38 (24.7)
D	60 (34.9)	50 (32.5)
Practice contract type
General Medical Services (17J)	145 (84.3)	133 (86.4)
Other (17C or 2C)	26 (15.7)	21 (13.6)
Urban/rural
Urban areas	105 (61.0)	90 (58.4)
Accessible small towns and rural areas	28 (16.3)	24 (15.6)
Remote small towns and rural areas	13 (7.6)	16 (10.4)
Very remote small towns and rural areas	26 (15.1)	24 (15.6)
Gender of registered patients
Male	543 653 (50.6)	490 466 (50.7)
Female	530 285 (49.4)	477 232 (49.3)
Age group of registered patients (years)
0–4	53 971 (5.0)	50 565 (5.2)
5–14	109 826 (10.2)	103 760 (10.7)
15–24	127 571 (11.9)	109 055 (11.3)
25–44	300 355 (28.0)	252 515 (26.1)
45–64	296 389 (27.6)	273 770 (28.3)
65–74	104 666 (9.7)	100 425 (10.4)
75–84	60 089 (5.6)	57 572 (5.9)
≥85	21 071 (2.0)	20 036 (2.1)
Deprivation (quintiles of SIMD score)
Q1 (most deprived)	201 891 (18.8)	177 057 (18.3)
Q2	237 050 (22.1)	236 439 (24.5)
Q3	231 229 (21.6)	188 651 (19.5)
Q4	191 210 (17.8)	175 350 (18.1)
Q5 (most affluent)	208 984 (19.5)	188 077 (19.4)
Missing	2563 (0.2)	2356 (0.2)
Baseline antibiotic prescribing rate
Items per 1000 patients per day in Quarter 1, 2016	1.93	1.98

Values are numbers (percentages) unless stated otherwise. Q, quintile.

Primary outcome

Prior to the study start, total antibiotic prescribing rates were decreasing in both study arms and this downward trend continued during the study period (Figure S1). There was no evidence of intervention effect on total antibiotic prescribing at the main analysis timepoint (Quarter 1, 2018), with adjusted rate ratio (ARR) for intervention versus control of 0.98 (0.94–1.02; P = 0.35) (Table 3). At the additional analysis timepoint (Quarter 2, 2017) there was evidence of a small intervention effect on total antibiotic prescribing. Adjusted post-intervention rates were 1.67 and 1.73 prescriptions/1000 patients/day in intervention and control practices, respectively, with ARR 0.99 (0.98–1.00; P = 0.03) (Table S4).

Secondary prescribing outcomes

There was no evidence of intervention effect on most secondary antibiotic prescribing outcomes, at either evaluation timepoint, with small changes in both directions that would not be clinically meaningful given the relatively small numbers of prescriptions affected (Table 3 and Table S4). There were changes in the intended direction at the additional analysis timepoint for prescribing of any antibiotics for patients aged 65 years and over [ARR 0.97 (0.94–1.00)], who have relatively high rates of prescriptions, and amoxicillin in all age groups [ARR 0.95 (0.91–1.00)], which is the most commonly prescribed antibiotic in primary care in the UK (Table S4).

Hospital admissions

There was no evidence of intervention effects on hospital admissions with complicated respiratory tract infections, skin infections, urinary tract infections, sepsis, or the composite of these four groups, at either evaluation timepoint (Table 3 and Table S4).

Discussion

Summary of main findings

In this pragmatic real-world randomized controlled trial of practice-level antibiotic prescribing feedback compared with usual practice, in the context of sustained antimicrobial stewardship activity and falling antibiotic prescribing, there was no evidence of effect on the primary outcome (overall antibiotic prescribing) at the main analysis timepoint (1 year after the last feedback report). At the additional evaluation timepoint (the quarter following the last feedback report) there was evidence of an effect on the primary outcome, and potentially clinically useful effects on two secondary prescribing outcomes (decreases in all antibiotic prescribing for patients aged ≥65 years and amoxicillin prescribing). However, these effects were not sustained at 1 year post intervention and were observed in the context of multiple secondary outcomes analysed. There was no evidence of effect on infection-related hospital admissions.

Strengths and weaknesses

The main strength is the pragmatic randomized trial design involving a large number of general practices, conducted as part of NHS improvement work within a pre-existing nationally coordinated antimicrobial stewardship programme, the type of intervention that could be deployed at scale in routine care. Prescribers in trial practices were unaware that they were in a trial and the statistician analysing the trial outcomes was blinded to practice allocation until analysis was complete. The intervention incorporated the majority of ‘best practices’ recommended by the authors of the Cochrane feedback review (Table 4).^15,24 The feedback was valid, recent, about the team’s own behaviour, and repeated over time. The reports came from a trusted source and included comparative data. The behaviour is amenable to feedback,^14,17 the recipients can generate improvement, and the 25^th percentile benchmark provided a performance target. Goal setting, with multiple elements of best practice, is more complicated with some elements not feasible for every practice and/or individuals in a large-scale intervention, but FAPPC incorporated those elements that were feasible (Table 4). Finally, clear action plans were included. Thus, the intervention incorporated at least 10 of 13 best practices recommended as active ingredients of feedback interventions.²⁴

Table 4.

Best practices when designing audit and feedback interventions recommended by Ivers et al. [24] and the extent to which the FAPPC intervention incorporated these

Recommended best practices [24]	FAPPC incorporation
Data are valid	Yes
Data are based on recent performance	Yes
Data are about the individual/team’s own behaviour(s)	Yes
Audit cycles are repeated, with new data presented over time	Yes
Presentation is multimodal including either text and talking or text and graphical materials	No—reports contained graphs of individual practice data and generic explanatory text so not multimodal performance feedback.
Delivery comes from a trusted source	Yes—cover letter signed by the Chair of SAPG and the Scottish Chief Medical Officer.
Feedback includes comparison data with relevant others	Yes—local Health Board and Scottish national.
Targeted behaviour is likely to be amenable to feedback	Yes
Recipients are capable and responsible for improvement	Yes
The target performance is provided	Yes—the 25th percentile rate was presented as a benchmark, an implicit target to meet or better.
Goals set for the target behaviour are aligned with personal and organizational priorities	Partly—reducing antibiotic prescribing is a national organizational priority but may not be a priority for individual practices or prescribers.
Goals for target behaviour are specific, measurable, achievable, relevant, time-bound	Partly—the target is specific and measurable, should be achievable (since has been met by 25% of practices), is relevant to those setting the targets, but with no time specified in the report.
A clear action plan is provided when discrepancies are evident	Yes

One potential explanation for the null result (for the primary outcome at the main timepoint) is that this feedback added little to existing stewardship interventions that had been applied over the previous 14 years, with steady and substantial reductions in antibiotic prescribing. Total primary care antibiotic use in Scotland had already reduced by 11%, from 2.2 to 2.0 prescriptions per 1000 population per day, between 2012 and 2016, when this trial started.²⁵ However, there was evidence of effect immediately after the last feedback that was not sustained once the feedback stopped (Table S4, Table 3 and Figure S1). There is virtually no published evidence on whether impacts are sustained, despite considerable literature on feedback interventions, and such analyses should take changes over time into account. Although the early intervention effect we observed for the primary outcome was small [ARR 0.99 (0.98 to 1.00)] it would equate to a clinically meaningful reduction in the annual number of antibiotic prescriptions nationwide and contribute to progress towards targets for reduction. There were no data collected on engagement with existing stewardship interventions, and there will be practice-level variation in prioritization and available resource. However, it is unlikely that this was systematically different between intervention and control practices. Another potential explanation is that the sample size calculation used an optimistic effect size, albeit one within the range of observed effects of feedback¹⁵ and a published trial,¹⁴ and that the number of ineligible practices was somewhat higher than expected. However, the observed CIs around the primary outcome effect estimate [ARR 0.98 (0.94–1.02)] are not consistent with a type 2 error missing a large intervention effect. Another methodological weakness was that the intervention and control arms were unbalanced in number, but the baseline characteristics of practices and their registered patients were balanced so this is unlikely to have affected the result.

Comparison with other work

The most directly comparable published randomized trial,¹⁴ which had a statistically significant effect of similar magnitude to the non-significant difference in our study (3% reduction), was conducted in England in 2014. At that time, the stewardship programme was less well established without the background of falling primary care prescribing. In addition, that intervention targeted the highest prescribing 20% of practices¹⁴ and poor baseline performance is a predictor of larger feedback effect size.¹⁵ In a trial of antibiotic prescribing feedback in general dental practice in Scotland, there was a 5.7% reduction in intervention practices. Behaviour change messages increased the effect but health board comparator data and an additional round of feedback did not,¹⁷ contrary to findings and recommendations from the Cochrane group.^15,24 Antibiotic stewardship in dentistry is relatively recent in Scotland,²⁶ and there was not a comparable background of falling prescribing rates. We anticipated that a mature stewardship environment might enhance receptiveness to feedback compared with other settings but, conversely, it may have meant that additional improvements are more challenging.

Internationally, large-scale primary care trials of prescribing feedback aiming to reduce total antibiotic use have typically had limited effects,^7–9,11,12 although one intervention found that added ‘behavioural impact optimization’ had marginal benefit.⁸ An Irish intervention achieved a 2%–3% reduction in total prescribing,¹⁰ a similar effect size to the larger English study targeting the highest prescribing practices,¹⁴ but it only included a small number of volunteer practices. Prescribing feedback trials aiming to improve the choice of antibiotic for specific clinical indications in primary care, typically UTI,^13,27,28 RTI^29–31 or both,^32,33 report larger effect sizes of up to 20%.²⁷ However, such trials that also measured total antibiotic use found no effect or an unintended increase.^13,32 Appropriate prescribing for specific clinical indications was the topic of some FAPPC feedback reports but the outcomes measured total prescriptions in each category. We did not evaluate dose or duration of prescriptions so there may have been undetected improvements in appropriate prescribing.

Primary care feedback trials of high-risk prescribing other than antibiotics have typically reported much larger effect sizes, in the region of 30%–40% reductions in targeted prescribing.^4–6 Those interventions targeted long-term prescriptions that confer risks in combination and/or in specific patients and the desired action is that prescribers review and revise chronic prescriptions. This is quite different to antibiotic feedback, where the desired action is to reduce future acute prescribing, which is plausibly more challenging to change.

Implications for policy and practice

A more targeted approach may have better return on investment, for example targeting the highest prescribers. Post hoc analysis of subgroups of practices in FAPPC, stratified by baseline total prescribing rate, did not indicate a clear difference but the behavioural messaging in our reports was not designed to target high prescribers, in contrast to that in other work.¹⁴ This approach would need more evidence in the Scottish context to support investment and roll-out. The null result for the primary trial outcome raises questions around the resource supporting untested interventions in NHS practice improvement but multifaceted interventions are more effective in healthcare improvement than single interventions. The SAPG improvement programme embodies a long-term, real-world, multifaceted intervention and demonstrating effectiveness of one individual component may remain challenging. Initially after this trial follow-up period ended, SAPG produced and disseminated similar quarterly reports nationally, but this has subsequently reduced to annually.

Unanswered questions for future research

In this real-world, resource-constrained trial it was not possible to conduct a parallel process evaluation to capture practice response to the feedback or record baseline engagement with SAPG’s improvement programme. Process evaluation to understand and explain effect size and variation should be included in future evaluations, if at all possible. Electronic prescribing data in Scotland are becoming more accessible and available within a shorter time frame. The feasibility and acceptability, and then effectiveness, of more frequent feedback incorporating near real-time data merits investigation. The set-up of such feedback would likely be more resource intense, which needs to be considered in evaluation of effect. Finally, this work was carried out before the COVID-19 pandemic, which has had a major impact on patient access to primary healthcare and to prescribing behaviour. Prescribing feedback trials in this context require specific design considerations.

Conclusions

This well-designed, real-world, practice-level antibiotic prescribing feedback had minimal additional effects in the context of decreasing antibiotic prescribing and an established national stewardship programme. Designing and implementing effective feedback interventions remains a priority to support challenging targets for reductions in antibiotic prescribing.

Funding

This work was undertaken as part of the SAPG work programme, which is funded by Scottish Government.

Transparency declarations

None to declare.

Supplementary data

Box S1, Tables S1–S4 and Figure S1 are available as Supplementary data at JAC Online.