Improving antibiotic prescribing quality in out-of-hours primary care: a mixed-methods study using participatory action research

Abstract

Background

During out-of-hours (OOH) primary care, GPs overprescribe antibiotics for respiratory tract infections (RTIs). Many interventions have been shown to improve antibiotic prescribing quality, but their implementation in practice remains difficult. Participatory action research (PAR) aims to explore, implement and evaluate change in practice with an active involvement of local stakeholders, while generating knowledge through experience.

Objectives

To evaluate whether PAR improves antibiotic prescribing quality for RTIs during OOH primary care and simultaneously identify the pivotal lessons learned.

Methods

A mixed-methods study with a PAR approach in three OOH GP cooperatives (GPCs). Each GPC co-created a multifaceted intervention focusing on improving antibiotic use for RTIs through plan-do-study-act cycles. We quantified antibiotic prescribing quality indicators and formulated the lessons learned from a qualitative process analysis.

Results

Interventions were chosen with the GPs and adapted to be context-relevant. The willingness to work on quality and engagement of local stakeholders led to ownership of the project, but was time-consuming. In one GPC, antibiotic prescribing significantly decreased for tonsillitis, bronchitis, otitis media and acute upper RTI. In all three GPCs, use of guideline-recommended antibiotics for otitis media significantly increased.

Conclusions

Implementing multifaceted interventions through PAR can lower total and increase guideline-recommended antibiotic prescribing for RTIs in OOH primary care. Co-creating interventions with GPs to suit local needs is feasible, but reaching all GPs targeted is challenging.

Introduction

Antibiotic overconsumption and the use of broad-spectrum antibiotics are the main drivers for antibiotic resistance, and therefore one of the major concerns in modern medicine worldwide.^1–3 Most antibiotics are consumed in the community and prescribed by GPs for respiratory tract infections (RTIs).^4,5 In Belgium, every GP is required to provide out-of-hours (OOH, e.g. weekends and bank holidays) primary care, typically at a GP cooperative (GPC). These GPCs offer a unique advantage in that they provide access to a diverse group of GPs operating in the same setting, making it feasible to implement interventions on a larger scale. Interestingly, GPs’ antibiotic prescribing rates during OOH care closely mirror those during their regular office hours in individual practices. RTIs are the most commonly diagnosed conditions in OOH,⁶ often leading to antibiotic prescriptions.⁷ Given the high frequency of patients presenting with respiratory infections, these GPCs offer ample opportunities for implementing and assessing interventions. All of these factors collectively make GPCs an intriguing context for studying and enhancing the appropriate use of antibiotics. Furthermore, antibiotic prescribing in Belgium ranks high compared with the European average, making additional efforts to improve antibiotic prescribing quality crucial.^8–11

Many interventions to improve antibiotic prescribing quality have been developed, studied and have proven to be (cost-)effective.^12–15 However, sustained implementation and improvement in daily practice remains a big societal issue.¹⁶ To address this gap, we investigated a participatory action research (PAR) project to improve antibiotic prescribing quality, which we have named the BAbAR project (Better AntiBiotic prescribing through participatory Action Research).¹⁷

PAR is a type of implementation research that uses mixed methods to describe, interpret and explain a period of inquiry, which specifically focuses on reflection, action and change in social situations together with participants.^18–20 PAR is a bottom-up, democratic approach not only to improve, but also to understand and learn from daily practice.^21,22 It simultaneously creates scientific and social knowledge by using multiple qualitative and quantitative methods in close collaboration with local stakeholders who are considered as co-researchers. The explicit use and evidence of PAR to improve antibiotic prescribing is limited, but promising.²³ Typically, PAR works through different phases in an iterative process of planning, action and reflection.^21,24–26

Here, we aim to evaluate the effect of using a PAR approach during OOH care in three GPCs on antibiotic prescribing quality for RTIs and to describe the lessons learned from this process.²⁷

Methods

Ethics

The study was approved by the Ethics Committee of the Antwerp University Hospital/University of Antwerp (reference number 17/08/089) and registered on ClinicalTrials.gov (NCT03082521). Verbal informed consent, after receiving written information, or written informed consent was obtained from all participants.

Study context

In Belgium, OOH primary care during bank holidays and weekends is organized in large-scale GPCs in most regions. GPs work in a rotation system at the GPC for OOH consultations and home visits. The number of shifts per GP varies per region and is on average about 15 shifts per year of approximately 12 h each. Patients have open access to the GPC, without any form of triage (at the time of the study). Patients only pay a small out-of-pocket contribution. Three intervention GPCs (GPC 1, GPC 2, GPC 3, each covering a certain region) participated. They were chosen based on their accessibility, the researchers’ familiarity with the GPCs and willingness of the GPC itself to participate in the study. Additionally, three GPCs were selected as a control group based on their data completeness and availability at the time in the database iCAREdata.²⁸ Setting up the collaborations and explorative discussions started in 2018. Interventions were implemented from September 2019 until the end of February 2020, and stopped prematurely because of the COVID-19 pandemic. Key differences between the three regions are summarized in Table 1.

Table 1.

Characteristics, set-up and process of the PAR project in the three regions

	GPC 1	GPC 2	GPC 3
Urban/rural	Urban	Rural/urban	Rural
Number of GPs	185	275	137
Patients living in the area	185 839	251 289	140 095
Co-researchers	GPC board & manager Master’s student medicine Local GP trainee	GPC board & manager Quality commission GPC chairman	GPC board & manager GP board member Local GP trainee
Exploratory phase
Timing	2017–19	May 2018	Oct 2018
Actions	Prescribing feedback GPC board Co-creation meetings Exploratory work: Video observations—(elicitation) interviews (2017–18)	Prescribing feedback GPC manager and chairman Quality commission Co-creation meetings	Prescribing feedback GPC board Co-creation meetings
Initiated by	Academic researcher and local GP trainee	GPC chairman & quality commission	GP board member and local GP trainee
Facilitating change phase
Timing	September 2019–February 2020 (6 months)	September 2019–February 2020 (6 months)	September 2019–February 2020 (6 months)
Start of the interventions	Prescribing feedback at GPC level, compared with other GPCs  E-mail  Interactive co-creation sessions  Newsletter	Prescribing feedback at GPC level, compared with other GPCs  Newsletter  Interactive co-creation sessions	Prescribing feedback at GPC level, compared with other GPCs  Newsletter  Interactive co-creation sessions
Initial focus of intervention	Otitis media: reduction and choice of antibiotics	Otitis media: reduction and choice of antibiotics Tonsillitis: reduction of antibiotics Bronchitis: reduction and choice of antibiotics	Diverticulitis RTI Sick children reduction and choice of antibiotics
Interventions	Access to e-learning Desk stand with information Reminder e-mails Patient leaflets Poster	Pop-ups in EHR Local medical-pharmaceutical interdisciplinary meetings Patient leaflets Poster	CRP POCT with 1 to 1 training and guidelines Access to e-learning Patient leaflets Poster
Aim	Knowledge, enhance patient communication, raise awareness	Knowledge, enhance patient communication, raise awareness	Knowledge, decision support, enhance patient communication, raise awareness
PDSA cycles and evaluations	2 cycles Interviews with GPs & prescribing feedback	2 cycles Interviews with GPs & prescribing feedback	2 cycles Interviews with GPs & prescribing feedback
Adaptations after PDSA cycle	Focus of the intervention: promoting the newly published antibiotic guidelines: Choice of antibiotic Visibility of materials Implementation of pop-ups in EHR (adopted from GPC2)	Visibility of materials Choice to continue with same focus	Implementing newly published antibiotic guidelines Adaptation of posters
Evaluation phase
Evaluation	In-depth interview with local key person & online questionnaire with board and management	In-depth interview with local key person & online questionnaire with board and management	In-depth interview with local key person & online questionnaire with board and management
Response to COVID-19 pandemic	Study on hold: OOH care reformed to triage hubs	Study on hold: OOH care reformed to triage hubs	Availability of CRP POCT in the triage hubs, but little used

Study design

This study used mixed methods, combining quantitative and qualitative data, in a PAR approach with local stakeholders as co-researchers, to improve the quality of antibiotic prescribing in OOH primary care.¹⁷ Figure 1 shows the exploratory phase, with an assessment of the context, readiness and barriers for change; a facilitating change phase that used plan-do-study-act (PDSA) cycles to implement the chosen interventions in real practice; and an evaluation phase. In this paper we focus on the implementation and evaluation. Results from the exploratory phase are discussed elsewhere.^7,29–31 To assess the effect of the interventions, a quantitative before-and-after study design was used. To evaluate the process we used qualitative framework analysis.

Figure 1.

Overview of the phases during the BAbAR project and description of the PAR set-up and data collection. Facilitating change and evaluating phases are discussed in this paper.

Each participating GPC put together its own multifaceted intervention. Multiple effective interventions were offered, from which each GPC could choose depending on their needs and subsequently they were tailored to their setting/context. The interventions used in this study were drawn from previously published trials of successful interventions.

Data collection and analysis

Quantitative data

Pseudonymized routine clinical data from the three GPCs, as well as pooled data from three control GPCs (where no interventions took place), automatically extracted from reports of the consultations, and collected in the database iCAREdata,²⁸ were used to calculate validated antibiotic prescribing quality indicators (APQIs)²⁷. The APQIs include the proportion of patients receiving an antibiotic and, among those receiving antibiotics, the proportion of patients receiving the recommended antibiotic for six different indications (ICPC-2 code³²): acute upper RTI (R74), sinusitis (R75), tonsillitis (R76), bronchitis (R78), pneumonia (R81) and acute otitis media (H71). Prescribing data were allocated to one of three time periods, i.e. the 6 months between September and February in 2017–18 (Period 1), 2018–19 (Period 2) and 2019–20 (Period 3; intervention period). First the proportion of patients receiving (recommended) antibiotics was calculated from the pooled data of all three intervention GPCs for all indications, and from the pooled data from all three control GPCs for all indications to investigate differences between the three periods for the intervention and control GPCs, respectively. Next, these APQIs were calculated for the three intervention GPCs separately, for each of the six different indications separately, as well as for each possible combination between GPC and indication. Data were missing for Period 2 in GPC 3. Therefore, APQIs for GPC 3 were calculated from data between September 2017 and December 2017 for Period 1 and from data between September 2019 and December 2019 for Period 3 (intervention period). All home visits were excluded from the data.

Depending on the sample size, a chi-squared or Fisher exact test was used to compare APQIs between the three time periods. In the case of a significant difference between the three time periods, one-sided post hoc tests were performed to locate the differences. For GPC 3, only a comparison between Period 1 and Period 3 (intervention period) from September to December could be made due to missing data in 2018.

RStudio (R version 4.2.2) was used for data analysis.

Qualitative data

We used different types of qualitative data. We used informal sources to gather information during the process of developing, implementing and evaluating the interventions, such as field notes, informal discussions, a timeline and stakeholder feedback. We also used more formal data collection: (i) developing phase: six co-creation sessions (two sessions per GPC with 6–15 GPs per session) with GPs and/or the management of the GPCs before the implementation to deliver feedback on antibiotic prescribing practices and co-create the multifaceted interventions; (ii) implementation phase (during PDSA cycles): 26 semi-structured interviews with participating GPs; and (iii) evaluating phase: three semi-structured interviews with the main co-researchers (GPs) and a questionnaire with the GPC board members with open questions and scores, informed by the literature and the normalization process theory.^33,34 The online questionnaire was completed by 12 persons in total from the three GPCs’ boards.

Semi-structured interviews of a convenience sample of GPs were done by telephone, audio recorded and transcribed verbatim. The interview guide and questionnaires are provided as Supplementary data, available at JAC-AMR Online). We analysed the interviews using a deductive thematic framework method.^35,36 We used the assessment guide of Waterman et al.²⁰ as a theoretical framework to interpret all the data and to understand how our PAR approach enabled or disabled the implementation, embedding and integration of quality improvement of antibiotic prescribing in OOH primary care. Coding, indexing and summarizing was done by Y.B. and O.V. for the 26 interviews with GPs, and at different intervals discussed with A.C. Interviews of the co-researchers, written reports of the co-creation sessions, the questionnaires, field notes, reflexive diary and timeline were analysed by A.C. and discussed with S.A. Excel was used to aid data management.

Table 1 shows an overview of the timing, the chosen interventions, the focus of the interventions, the aims, the PDSA cycles and consequent adaptations of the interventions and the evaluation per GPC.

Results

A total of 579 GPs were active during OOH care in the three intervention GPCs. During the 6 month intervention period, 752 consultations for RTIs were done in GPC 1, 1998 in GPC 2 and 665 in GPC 3 (Table 1) and 2624 in the control GPCs.

Chosen interventions

The chosen interventions of GPC 1 focused on GP education through the availability of a web-based e-learning package that was installed on the GPC computers, and a summary of the otitis media guideline printed on a desk reminder. An existing patient leaflet (TARGET Treating Your Infection leaflet),³⁷ with information on the duration of infections, on safety-netting advice and, if appropriate, on delayed antibiotic prescribing was translated and adapted to meet the needs expressed by the GPCs. This leaflet was co-created by GPCs 1 and 3 and adopted by GPC 2 because of its usability and attractiveness.

GPC 2 chose to implement pop-ups in the electronic health record (EHR) with advice to withhold antibiotics, and if chosen otherwise, to provide prescribing guidance when diagnosing otitis, tonsillitis or bronchitis. They co-designed the content of the pop-ups and the cost for implementing this system was covered by the GPC itself. Besides that, they organized local interdisciplinary meetings with pharmacists.

GPC 3 chose to implement C-reactive protein (CRP) point-of-care testing (POCT) as their main intervention, of which they themselves bore the financial cost, combined with a poster providing the current antibiotic prescribing and CRP POCT use guidelines.

Effect of the PAR project as a whole on antibiotic prescribing quality

Proportion of patients prescribed antibiotics

Pooled data

Considering the outcome based on the pooled data from all three intervention regions, the proportion of patients prescribed an antibiotic for any indication was significantly lower in the intervention period (Period 3), where 41.2% were prescribed antibiotics, compared with 50.1% in Period 1 [−8.9% (95% CI: −6.5% to −11.3%)] and 48.0% in Period 2 [−6.8% (95% CI: −4.3% to −9.4%)] (Table 2). No differences in the proportion of patients prescribed an antibiotic for any indication were found between the three study periods in the control regions.

Table 2.

Total number of patients, and proportion of patients receiving antibiotics for Periods 1, 2 and 3 in intervention and control GPC

		GPC 1		GPC 2		GPC 3a		Intervention		Control
		N	%	N	%	N	%	N	%	N	%
R74 acute upper RTI	Period 1	427	13.35	963	39.67	339	23.89	1729	30.08	1005	15.52
	Period 2	467	7.92b	1057	39.74	/	/	1524	29.99	1020	16.96
	Period 3	428	10.05	1044	25.29b,c	295	17.97b	1767	20.38b,c	1454	18.29
R75 sinusitis	Period 1	59	32.20	203	57.14	71	70.42	333	55.55	181	37.57
	Period 2	77	45.45	205	52.68	/	/	282	50.71	167	38.32
	Period 3	55	38.18	186	49.46	53	60.38	294	49.32	181	35.36
R76 tonsillitis	Period 1	79	70.89	244	87.30	65	84.62	388	83.51	191	68.06
	Period 2	77	77.92	249	89.96	/	/	326	87.12	221	75.11
	Period 3	103	53.40b,c	213	75.12 b,c	122	86.07	438	73.06b,c	367	77.66
R78 bronchitis	Period 1	60	60.00	257	82.10	88	82.95	405	79.01	142	58.45
	Period 2	56	53.57	188	75.53	/	/	244	70.49b	156	68.59
	Period 3	57	56.14	159	66.67 b,c	67	71.64	283	65.73b	179	55.31c
R81 pneumonia	Period 1	10	60.00	71	70.42	5	80.00	86	69.77	31	83.87
	Period 2	7	100.00	46	84.78	/	/	53	86.79	23	95.65
	Period 3	15	93.33	62	74.19	26	92.31	103	81.55	53	83.02
H71 acute otitis media	Period 1	29	41.38	289	71.63	23	69.57	341	68.92	170	47.65
	Period 2	101	52.48	278	69.42	/	/	379	64.91	244	50.00
	Period 3	92	47.83	318	58.81 b,c	99	71.72	509	59.34b	373	56.30
All RTIs	Period 1	664	28.01	2027	58.16	591	47.21	3282	50.09	1720	31.63
	Period 2	785	28.28	2023	55.66	/	/	2808	48.01	1831	35.72
	Period 3	750	27.87	1982	43.14b,c	662	50.30	3394	41.16b,c	2607	37.13

Period 1 = September 2017—February 2018; Period 2 = September 2018—February 2019; Period 3 = September 2019—February 2020 = intervention period.

Bold: specific focus on this indication per GPC.

^aNo data were available from January 2018 until February 2019. Percentages were calculated from data between September 2017 and December 2017 for Period 1 and from data between September 2019 and December 2019 for Period 3. Percentages could not be calculated for Period 2 due to missing data.

^bSignificantly lower percentage of antibiotics prescribed compared with Period 1 at a 5% significance level.

^cSignificantly lower percentage of antibiotics prescribed compared with Period 2 at a 5% significance level.

By GPC and indication

Table 2 also shows the total numbers of patients as well as the proportions of patients prescribed antibiotics for Periods 1, 2 and 3 in each of the intervention GPCs separately, both for any indication and per indication. In GPC 1, no differences in the proportion of patients prescribed an antibiotic for any indication were found between the three study periods. However, the proportion of patients prescribed an antibiotic for acute upper RTI was significantly lower in Period 2, where 7.9% were prescribed an antibiotic, compared with 13.4% in Period 1 [−5.4% (95% CI: −1.2% to −9.7%)]. The proportion of patients prescribed an antibiotic for tonsillitis was significantly lower in the intervention period, where 53.4% were prescribed antibiotics compared with 70.9% in Period 1 [−17.5% (95% CI: −2.5% to −32,5%)] and 77.9% in Period 2 [−24.5% (95% CI: −10.0% to −39.0%)].

In GPC 2, the proportion of patients prescribed an antibiotic for any indication was significantly lower in the intervention period, where 43.1% were prescribed antibiotics, compared with 58.2% in Period 1 [−15.0% (95% CI: −11.9% to −18.1%)] and 55.7% in Period 2 [−12.5% (95% CI: −9.4% to −15.6%)]. More specifically, the proportion of patients prescribed an antibiotic was significantly lower in the intervention period compared with Periods 1 and 2 for acute upper RTI, tonsillitis, acute bronchitis and acute otitis media. For acute upper RTI, 25.3% were prescribed antibiotics in the intervention period compared with 39.7% in Period 1 [−14.4% (95% CI: −10.2% to −18.5%)] and 39.7% in Period 2 [−14.5% (95% CI: −10.4% to −18.5%)]. For tonsillitis, 75.1% were prescribed antibiotics in the intervention period compared with 87.3% in Period 1 [−12.2% (95% CI: −4.9% to −19.8%)] and 90.0% in Period 2 [−14.8% (95% CI: −7.5% to −22.2%)]. For bronchitis, 66.7% were prescribed antibiotics in the intervention period compared with 82.1% in Period 1 [−15.4% (95% CI: −6.2% to −24.6%)] and 75.5% in Period 2 [−8.9% (95% CI: 1.3% to −19.0%)]. For acute otitis media, 58.8% were prescribed antibiotics in the intervention period compared with 71.6% in Period 1 [−12.8% (95% CI: −5.0% to −20.7%)] and 69.4% in Period 2 [−10.6% (95% CI: −2.6% to −18.6%)].

In GPC 3, the proportion of patients prescribed an antibiotic for any indication did not differ between periods, but the proportion of patients prescribed an antibiotic for acute upper RTI was significantly lower in Period 3, where 18.0% were prescribed antibiotics, compared with 23.9% in Period 1 [−5.9% (95% CI: −12.6% to 0.7%)].

In the control GPCs, the proportion of patients prescribed an antibiotic for bronchitis was significantly lower in Period 3, where 55.3% were prescribed antibiotics, compared with 68.6% in Period 2 [−13.3% (95% CI: −2.4% to −24.2%)].

Proportion of patients prescribed guideline-recommended antibiotics

Pooled data

Considering the pooled data from all three intervention regions, the proportion of patients prescribed a guideline-recommended antibiotic for any indication was significantly higher in the intervention period and in Period 2, where 70.3% and 68.3% were prescribed a guideline-recommended antibiotic, respectively, compared with 61.3% in Period 1 [+9.0% (95% CI: +5.6% to +12.5%) and +7.1% (95% CI: +3.6% to +10.6%), respectively] (Table 3). In the control GPCs, the proportion of patients prescribed a guideline recommended antibiotic for any indication was also significantly higher in the intervention period and in Period 2, where 74.4% and 72.2% were prescribed a guideline-recommended antibiotic, respectively, compared with 66.9% in Period 1 [+7.5% (95% CI: +2.5% to +12.4%) and +5.3% (95% CI: −0.1% to +10.7%), respectively].

Table 3.

Total number of patients prescribed an antibiotic, and proportion of patients prescribed the recommended antibiotic for Periods 1, 2 and 3 in intervention and control GPC

		GPC 1		GPC 2		GPC 3a		Intervention		Control
		N	%	N	%	N	%	N	%	N	%
R74 acute upper RTI	Period 1	57	54.39	382	63.61	81	66.67	520	63.08	156	64.74
	Period 2	37	67.57	420	71.90b	/	/	457	71.55b	173	73.41
	Period 3	43	53.49	264	68.18	53	60.38	360	65.28	266	72.93
R75 sinusitis	Period 1	19	57.89	116	44.83	50	40.00	185	44.86	68	45.59
	Period 2	35	60.00	108	48.15	/	/	143	51.05	64	54.69
	Period 3	21	47.62	92	44.57	32	50.00	145	46.21	64	57.81
R76 tonsillitis	Period 1	56	78.57	213	78.87	55	74.55	324	78.09	130	76.92
	Period 2	60	76.67	224	79.46	/	/	284	78.87	166	85.54
	Period 3	55	76.36	160	81.25	105	83.81	320	81.25	285	84.21
R78 bronchitis	Period 1	36	50.00	211	36.49	73	46.58	320	40.31	83	56.63
	Period 2	30	63.33	142	40.14	/	/	172	44.19	107	47.66
	Period 3	32	53.13	106	50.94 b	48	39.58	186	48.39	99	53.54
R81 pneumonia	Period 1	6	100.00	50	42.00	4	0.00	60	45.00	26	57.69
	Period 2	7	42.86	39	41.03	/	/	46	41.30	22	63.63
	Period 3	14	71.43	46	58.70	24	54.17	84	59.52	44	43.18
H71 acute otitis media	Period 1	12	75.00	207	82.13	16	50.00	235	79.57	81	86.42
	Period 2	53	75.47	193	83.94	/	/	246	82.11	122	84.43
	Period 3	44	93.18 c	187	95.19b,c	71	85.92b	302	92.72b,c	210	84.29
All RTIs	Period 1	186	63.98	1179	62.00	279	56.27	1644	61.25	544	66.91
	Period 2	222	69.37	1126	68.12b	/	/	1348	68.32b	654	72.17b
	Period 3	209	68.42	855	71.35b	333	68.77 b	1397	70.29b	968	74.38b

Period 1 = September 2017—February 2018; Period 2 = September 2018—February 2019; Period 3 = September 2019—February 2020 = intervention period.

Bold: specific focus on this indication per GPC.

^aNo data were available from January 2018 until February 2019. Percentages were calculated from data between September 2017 and December 2017 for Period 1 and from data between September 2019 and December 2019 for Period 3. Percentages could not be calculated for Period 2 due to missing data.

^bSignificantly higher percentage of guideline-recommended antibiotics, among the antibiotics prescribed, compared with Period 1 at a 5% significance level.

^cSignificantly higher percentage of guideline-recommended antibiotics, among the antibiotics prescribed, compared with Period 2 at a 5% significance level.

By GPC and indication

Table 3 also shows the total numbers of patients receiving antibiotics and the proportions of patients receiving guideline-recommended antibiotics for Periods 1, 2 and 3, in each of the intervention GPCs separately, both for any indication and per indication.

In GPC 1, the proportion of patients prescribed a guideline-recommended antibiotic for any indication did not differ between periods, but the prescription of guideline-recommended antibiotics for acute otitis media was significantly higher in the intervention period, where 93.2% were prescribed the guideline-recommended antibiotic, compared with period 2, where 75.5% were prescribed the guideline-recommended antibiotic [+17.7% (95% CI: +1.3% to +32.4%)].

In GPC 2, the proportion of patients prescribed a guideline-recommended antibiotic for any indication was significantly higher in the intervention period and in Period 2, where 71.4% and 68.1% were prescribed a guideline-recommended antibiotic, respectively, compared with 62.0% in Period 1 [+9.3% (95% CI: +5.1% to +13.6%) and +6.1% (95% CI: +2.1% to +10.1%), respectively]. A significant increase was found in the proportion of patients prescribed the guideline-recommended antibiotic for acute upper RTI in Period 2, where 71.9% were prescribed the guideline-recommended antibiotic, compared with 63.6% in Period 1 [+8.3% (95% CI: +1.6% to +15.0%)]. A significant increase was found in the proportion of patients prescribed the guideline-recommended antibiotic for bronchitis in the intervention period, where 50.9% were prescribed the guideline-recommended antibiotic, compared with 36.5% in Period 1 [+14.5% (95% CI: +2.2% to +26.7%)]. Lastly, a significant increase was observed in the prescription of guideline-recommended antibiotics for acute otitis media, where 95.2% of the patients received the guideline-recommended antibiotic, compared with 82.1% in Period 1 [+13.1% (95% CI: +6.5% to +19.6%)] and 83.9% in Period 2 [+11.3% (95% CI: +4.7% to 17.8%)].

In GPC 3, the proportion of patients prescribed a guideline-recommended antibiotic for any indication was significantly higher in the intervention period, where 68.8% were prescribed a guideline-recommended antibiotic, compared with 56.3% in Period 1 [+12.5% (95% CI: +4.5% to +20.5%)]. Further, a significant increase was found in the proportion of patients prescribed the guideline-recommended antibiotic for acute otitis media in the intervention period, where 85.9% was prescribed the guideline-recommended antibiotic, compared with 50.0% in Period 1 [+35.9% (95% CI: +10.8% to +61.0%)].

In the control GPCs, no significant differences were found between the proportion of patients prescribed a guideline-recommended antibiotic during the study periods per indication.

The absolute differences and comparisons of observed proportions of patients receiving antibiotics or receiving guideline-recommended antibiotics between the three periods can be found in Tables S2.1 and S2.2.

Evaluation of the PAR approach to improve antibiotic prescribing behaviour

We evaluated the use of the PAR approach in this project with the help of eight pivotal factors of PAR discussed in the assessment guide from Waterman et al.:²⁰ (1) participation; (2) key persons; (3) action researcher–participant relationship; (4) real-world focus; (5) resources; (6) research methods; (7) project process and management: (7.1) responsiveness and flexibility; (7.2) feedback mechanisms; (7.3) evaluation; and (8) knowledge.

Table 4 shows the characteristics of the participants in this qualitative process evaluation.

Table 4.

Characteristics of the participants in the process evaluation

Evaluation during intervention: telephone interviews
Total number of participating general practitioners
GPC 1	10
GPC 2	5
GPC 3	11
Age (years)
Mean (SD)	38.7 (12.4)
Median (min–max)	34 (24–59)
Gender distribution
Male	11
Female	15
Duration of the interviews (min)
Mean	12
Range (min–max)	8–19
Evaluation after intervention: online questionnaires
Questionnaires with key persons within the GPC
Number of participants
GPC 1	5
GPC 2	4
GPC 3	3
Main function of key persons
GPC board member/GP	3
Commission Member GPC quality/GP	3
Study facilitator BAbAR & GP	3
GPC manager	3
Evaluation after intervention: in-depth interviews with key organizer/co-researcher within each GPC
Number of participants	3 (1/GPC)
Duration of the interviews (min)
Mean (SD)	30 (6.4)
Range (min–max)	25–37

Participation

In the exploratory phase and the co-creation sessions, a diverse group (age, gender, type of practice) of GPs from all GPCs participated, and were willing to think about the needs, problems and the set-up of the interventions.

All GPs involved acknowledged that improving antibiotic prescribing quality in OOH is an important and relevant theme and a uniform GPC policy could support their decision not to prescribe antibiotics. There was legitimate concern that only the ‘conscious prescribers’ would take an interest in the ongoing interventions. E-mail reminders during the project were implemented to raise awareness of the ongoing project, but its impact is unclear.

The participating GPCs were committed and engaged in creating awareness and a climate to work on the quality of care, by making time to discuss prescribing feedback, proposing solutions and communicating about it to their GPs.

A lot of infections pass at the GPC. Many antibiotics are therefore prescribed here. Generalizing (antibiotic) policy and encouraging all doctors to prescribe correctly is, in my opinion, certainly one of the duties of the GPC. This benefits the patient and general health care. (Questionnaire, KP 12, female, GPC 3)

Key persons

In GPC 1, the study was not initiated by the GPC itself, and the leadership came mostly from the academic partner.

In GPCs 2 and 3, the project was initiated by local GPs, active in the GPC managing board, who wanted to work on antibiotic prescribing quality. They had a strong sense of ownership of the project and a personal engagement to strive for improvement. Furthermore, it is worth noting that GPC 2 was the sole GPC with a quality committee of multiple GPs, dedicated to enhancing the quality of care within the GPC. This distinction presents a potential rationale for their performance in terms of keeping the project on the agenda, motivating peers and improving antibiotic prescribing behaviour compared with the other GPCs. GPs from GPC 3 applied for external funding, which enabled them to become paid co-researchers and fund the intervention.

‘Any financial support I can get is nice. This financial support makes the difference between having to work for free after hours or actually during working hours. Feeling that what interests you is also a matter of social importance gives me motivation. In addition, I am given a chance to conduct research for the first time.’ (Field notes: funding application, KP 11, female, GPC 3)

Action researcher–participant relationship

Key persons from the participating GPCs were considered co-researchers by involving them in all steps of the project from designing to dissemination of findings.

The in-depth interviews with these key persons showed that the PAR process was considered interesting, empowering and beneficial to themselves (personal development) and to their GPC (working on quality of care), which led to a feeling of ownership and active engagement with the project. Their involvement led to a more comprehensive and contextualized understanding of the process.

Key persons highlighted the importance of the scientific input of the academic team to provide prescribing feedback, to help address problems and co-create an intervention strategy. However, although the aim was to achieve a democratic and empowering approach, it remained difficult to reach all GPs working at the GPCs.

The support of the board and the academic partner was a strength in this project. The tools we have developed such as flyers, pop-ups, feedback information, was really strong. The difficulty is that we still have to reach individual doctors. That’s a weakness of course. It’s not overnight. It is still the doctor who decides whether to prescribe an antibiotic. (Interview, KP 6, male, GPC 2)

The main researcher (A.C.) played an insider’s as well as an outsider’s role, as she worked previously in GPC 1 as a GP. From her notes in the reflexive diary she noticed that her background as a GP made it easier to engage other GPs in the project and understand the OOH context in which GPs work. A pitfall was that her own perceptions and experiences could influence the data analysis.

Real-world focus

The context of antibiotic prescribing during OOH care was well explored in the exploratory phase of the project through interviews, video observations and prescribing analysis.^7,29,30

When developing the interventions, we strived to stay as close as possible to the GPs’ needs. For example, GPs wanted to provide the patient with a leaflet to take home that is informative, attractive, with limited text, and available in multiple languages, to support their communication when not prescribing antibiotics. So an existing leaflet was adapted for this purpose and digital translations in 19 languages were provided.

There are so many population groups, especially here, and communication is so difficult, it’s a group that gets more antibiotics than usual, out of pure misery. And there are so many different languages here, it doesn’t help to translate it in just two languages. (Focus group, GP, male, GPC 1)

In the exploratory phase, GPs indicated that they were afraid of the occurrence of complications when not prescribing. Therefore, safety-netting advice was included in the leaflet to support the communication. Yet, it was indicated by GPs that the leaflet was often given without further explanation. Besides, often the leaflet and the prescribing feedback were not even noticed by the GPs in the consultation room during the shift. The lack of time during a busy OOH shift was often mentioned as a reason for not using the provided interventions. Nevertheless, the pop-up notifications integrated into the EHR system used by GPC 2, and subsequently adopted by GPC 1 towards the end of the project, could not be ignored. Hence, these pop-up notifications might have had a more pronounced impact on prescribing behaviour in terms of effectiveness in GPC 2.

The sustainability of the interventions was a concern for all the key persons within the GPCs, for which a sustained effort and continuous reminders were considered necessary.

The onset of the COVID-19 pandemic changed the work of the GPs considerably. The focus was no longer on whether or not to prescribe antibiotics, but on whether or not there was a COVID-19 infection and safely delivering triage, causing the interventions to be halted.

Resources

Initially, there were no resources to fund the interventions. The GPCs primarily used low-cost effective interventions, including the utilization of pre-existing interventions to avoid development expenses. Printing costs were covered by the research team, while the substantial acquisition cost of the CRP POCT device was secured through an application for external funding by the respective GPC (3). Additionally, GPC 2, implementing pop-ups in the GPCs’ EHR, independently covered the developing costs, showing their commitment to the project and willingness to invest in enhancing quality of care. Thus, the financial burden for the interventions was predominantly shouldered by the GPCs, highlighting the project’s prospects for sustainability and scalability.

Key persons explained that internal support by the GPC board and external support by for example an academic partner, easy and quick access to prescribing feedback, the necessary budget and a local champion who can move it forward with enough time, are essential for the sustainability of the implementation.

‘It’s hard to find shoulders that want to carry everything. It is always the same shoulders.’ (Interview, KP11, female, GPC 3)

Research methods

We used a mixed-methods approach, using quantitative and multiple qualitative data collection methods. Throughout the data collection, the different stakeholders (co-researchers, managers, boards, GPs) were repeatedly involved at several stages. Data were continuously collected through field notes, which fed the analysis and collected knowledge that was necessary for the ‘real-world’ focus.

Project process and management:

Responsiveness and flexibility

Rapid adjustments could be made during the course of the study when necessary. During the interviews, for example, we noticed that GPs were insufficiently informed about the ongoing project. Reminder e-mails were implemented very quickly to keep them engaged.

Feedback on their antibiotic prescribing quality could be given quickly because of the quasi real-time data collection of iCAREdata and a comprehensive review was delivered quarterly to each GPC on their progress.

Best practices were shared and adopted by the other GPCs. For example, the pop-ups designed and implemented in the EHR by GPC 2 were adopted by GPC 1, and have been made available to GPCs not involved in this study.

Due to the COVID-19 pandemic, the project was terminated prematurely. Only two PDSA cycles were completed, and it was not possible to make all necessary adaptations (for example, extra training on how to communicate safety-netting) to enhance uptake of the interventions and to engage more GPs.

Feedback mechanisms

Qualitative results of the exploratory phase^29,30 and antibiotic prescribing feedback, using APQIs²⁷ from each GPC benchmarked against other GPCs were discussed in the co-creation groups. All GPs received a feedback summary by e-mail.

How come we do so much worse here than those other GPCs, those prescription numbers really need to improve. (Notes from meeting GPC 2)

Three-monthly feedback of the APQI values of their targeted indications was provided for every GPC through a newsletter. New actions and goals were identified and refined for a new PDSA cycle after every feedback by the main key persons of the GPCs and the research team (Table 1). GPCs 1 and 3 utilized a distinct newsletter exclusively for the project, whereas at GPC 2 the information was integrated into the pre-existing GP newsletter, which may have already served as an effective means of communication among the GPs and the GPC. This feedback played a pivotal role in ensuring that all stakeholders remained informed about the project’s expectations, objectives and ongoing progress.

It is noteworthy that the most pronounced improvements in antibiotic prescribing behaviour were observed for the indications specifically targeted by the respective interventions. For instance, GPC 1 focused its efforts on enhancing prescribing behaviour for acute otitis media, while GPC 2 aimed at reducing prescriptions for tonsillitis and simultaneously enhancing prescribing practices for bronchitis and acute otitis media. In contrast, GPC 3 adopted a more general approach, addressing a broader range of RTIs.

Evaluation

The project was evaluated at different stages, using qualitative interviews with the different stakeholders (GPs, key persons, GPC management) to capture their views and experiences of the project and to see what adjustments were needed. Also, APQI values were assessed, before during and after the study.

Knowledge

Parts of the intervention focused on improving the knowledge on appropriate antibiotic prescribing, for example by providing prescribing guidelines and an e-learning tool, which focuses on increasing self-efficacy in making ‘no antibiotic prescribing’ decisions and communicating them effectively. GPs indicated that the mere presence of the study material and the ongoing project led to a change in awareness of their prescribing quality, although on the other hand some GPs were not aware enough of the ongoing interventions.

Using this PAR framework as a theoretical background, we hoped not only to describe the implementation but also to explain the process on how and why changes did or did not occur.

Discussion

The BAbAR project using a PAR approach to improve the antibiotic prescribing quality in OOH primary care was prematurely stopped by the COVID-19 pandemic. Nevertheless, in one GPC, antibiotic prescribing significantly decreased for tonsillitis, bronchitis, otitis media and acute upper RTI, and in all three GPCs, use of guideline-recommended antibiotics for otitis media significantly increased. The PAR approach was time-intensive and involved a high degree of investment on behalf of the (co-)researchers and the different stakeholders. The project and the interventions provided knowledge to the GPs, but above all led to greater awareness of GPs’ own prescribing behaviour. However, it remained difficult to engage the large and varied group of GPs and improve their prescribing quality.

Possible hypotheses for the positive effect in one GPC are: a small-scale stepwise approach with clear goals, more OOH shifts per GP (i.e. more exposure to the interventions), strong key persons who communicate regularly to the whole GP group, worse antibiotic prescribing quality at the start and the effect of the pop-ups in the EHR. The positive effect could also be explained by the availability of the largest real-life dataset in this GPC and therefore there could potentially be more power to detect differences. In all GPCs, the guideline-recommended antibiotic was more frequently prescribed for otitis media.

Since the project ended, the GPC boards have continued to prioritize on antibiotic prescribing quality. The pop-ups remain implemented, they are eager to receive prescribing feedback and the CRP POCT got a permanent place in one of the GPCs. The interventions chosen together with the local stakeholders were adapted to the context and considered relevant, but not always feasible as different barriers were identified. The PAR project itself led to lessons learned when setting up these kinds of quality improvement projects. The use of mixed methods led to rich data.³⁸ However, the integration of quantitative and qualitative data was limited.

Local prescribing feedback and a PAR approach was seen as empowering and meaningful. Key persons of each GPC were considered co-designers and co-researchers, and these were indispensable for the success we saw within the time frame of the project. For the sustainability of this type of project, the necessary long-term commitment of local key persons and sufficient time investment is crucial, together with the necessary financial resources.

PAR

The active involvement of local stakeholders when setting up interventions is considered good practice in implementation science and is more and more applied in antimicrobial stewardship studies.^39,40 In our study, much emphasis was placed on the participatory approach. The explicit use of PAR with all key features to improve antibiotic prescribing quality is still limited in primary care.^23,41 Collaboration and building trusting partnerships with local co-researchers fosters ownership,⁴² which also has been shown in a Vietnamese PAR study on antibiotic use in the community.⁴³ An insider perspective has the potential to produce more useful research questions and data that are more valid for real-world practice.⁴⁴ Because local practitioners ‘live’ the problems, they are in the best position to develop the best solutions, to assess acceptability and feasibility, and to provide ideas to improve sustainability.⁴⁵ Furthermore, it is considered more and more important to also address the emotional, cognitive and social factors that potentially influence antibiotic prescribing,⁴⁶ which can be achieved with a PAR approach.

Strengths and limitations

Implementation of evidence-based practice using PAR is a promising approach.^41,47 To the best of our knowledge, this is the first study explicitly using a PAR approach to improve antibiotic prescribing quality in an OOH primary care setting. We held on to the PAR paradigm in all stages of the research from set-up to implementation and evaluation with a strong involvement of the local stakeholders being actual co-researchers and even co-authors in this manuscript. Although we mostly complied with the original study protocol,¹⁷ PAR is a dynamic process and the study was responsive to continuous adaptations. By giving a rich description of the variation in content, context and application of the interventions, we aimed for optimal transferability.⁴⁸ Throughout the project we used both methods and researchers’ triangulation to gain a deep understanding.⁴⁹

A limitation of this study is that we possibly predominantly collected positive responses during the co-creation groups and interviews because of the participation of motivated GPs. The telephone interviews with GPs, evaluating the use of the interventions after their OOH shift, were brief and might have lacked some depth.

A patient representation group was involved in developing the patient materials, but we lacked patient involvement in further stages of the research.

We do not know the effect of our interventions on GPs’ prescribing habits outside the OOH context, nor how and to what extent all of the provided interventions were used by the different GPs and GPCs.

By using the same time period (September–February for GPCs 1 and 2; September–December for GPC 3) and a contemporary pooled control group to compare prescribing quality, we have mitigated potential seasonal and external effects. Compared with a randomized controlled trial, the used approach in our study makes it difficult to establish a causal relationship between the intervention and the outcome and results will not be exactly replicable.

With regard to the prescribing changes, due to the shorter than planned study duration and the large number of GPs who only work a limited number of shifts, the outcomes must be interpreted with caution. Behaviour change processes take time, and it’s likely the full potential of the approach was not achieved during the 6 month intervention period. It is plausible that some GPs may have participated more actively compared with others; however, our analysis does not focus on these variances at the individual physician level.

Using routinely collected EHR data ensured a high grade of completeness of data. However, the APQIs do not cover all codes that could be linked to an antibiotic prescription.⁷ The data of 2018 for GPC 3 were unavailable, and there were no data analysed after the stopping of the intervention due to the COVID-19 pandemic.⁵⁰ The low number of patient contacts in some GPCs for some APQIs might have resulted in a lack of power to detect an actual effect of the interventions on the antibiotic prescribing quality for some RTIs, in particular on guideline-recommended antibiotic prescribing.

Data of three non-involved GPCs were pooled as a control group. Differences between the intervention and control GPCs, such as differences in patient demographics or in other factors that might influence antibiotic prescribing, were not taken into account because the analysis was not based on patient-level data.

Implications for practice and future research

Implementation and evaluation was still ongoing at the start of the COVD-19 pandemic. Belgian OOH primary care refocused and reorganized itself to safely provide triage, urgent care and COVID-19 testing. Determinants influencing antibiotic prescribing changed rapidly, and consequences at this point for antibiotic quality and resistance in the long term are difficult to predict. Data show a reduction in the proportion of patients prescribed an antibiotic in OOH primary care during the pandemic.⁵¹

Most GPs in Belgium participate in local quality circles or peer-review groups several times a year. These fixed groups consists of 10–20 GPs who discuss quality-of-care topics. The advantage over a GPC is that there is often a bond of trust and that it is smaller scale. However, a single intervention integrated in the group’s normal working procedure did not have a significant effect on the quality of antibiotic prescribing.⁵² An empowering bottom-up PAR approach has not yet been studied.

We provided an overview of the facilitators and barriers, which could serve as recommendations for using a PAR approach in other countries or other settings (Table 5).

Table 5.

Strengths and barriers of the pivotal factors of PAR learned during the BAbAR project to improve antibiotic prescribing behaviour during OOH primary care

Pivotal factors
1. Participation	Strengths	In 2 out of 3 GPCs, the project was bottom-up initiated by local GPs
		Participants were engaged early in the process
		Participation of stakeholders was voluntary
	Barriers	In 1 of the 3 GPCs, the study was initiated by the study team, so there were limited local key persons
		Only committed GPs participated in the exploratory, co-creation and evaluation phase
		Level of participation, ownership of the project and motivation for change differed between the different GPCs
2. Key persons	Strengths	Great involvement and commitment of a number of key persons
		Were considered as co-researchers, and involved in every stage of the research
		Provided understanding of the context
	Barriers	Limited number of key persons
		Requires a great commitment
3. Action researcher–participant relationship	Strengths	Insider-outsider role of the main researcher increases credibility
		Positive relationship between researcher and key persons
	Barriers	Varying levels of commitment between the different GPCs and/or GPs
4. Real-world focus	Strengths	The exploratory phase of this project analysed in depth the context
		Use of quality indicators to describe and identify the current state
		Material adapted for target population (e.g. leaflet in multiple languages)
		Materials containing solutions for difficulties experienced by GPs during the consultation (e.g. safety-netting, explaining duration of infection, leaflet to provide instead of a prescription…)
		Cross-pollination of ideas and solutions over the different GPCs
	Barriers	Time-consuming due to mixed-methods approach
		The sudden onset of COVID-19 stopped the project
		Practical problems (e.g. availability of materials, busy shifts)
5. Resources	Strengths	Budget raised by both the GPC and the academic partner
	Barriers	Limited budget
		Time-consuming project
6.Research methods	Strengths	Mixed-methods approach to gain a rich description
		Exploring the problem could be considered as an intervention already by exposing the problem
	Barriers	Data processing takes time
7. Project process and management
Responsiveness and flexibility	Strengths	Rapid and flexible feedback and adjustments
	Barriers	Limited PDSA cycles
Feedback mechanisms	Strengths	Access and automatic extraction of data from the EHR
		Use of standardized quality indicators
	Barriers	Manually processing the data to provide tailored feedback
		Possible errors in the data due to registration faults
Evaluation	Strengths	During and after implementation phase
		Mixed methods
	Barriers	Difficult evaluation of outcomes because of large group of different GPs (limited contact with the intervention) and shorter study period than foreseen
8. Knowledge	Strengths	Contribution to knowledge at GP level, but moreover contribution to knowledge at the level of using PAR in this context

Conclusions

To improve antibiotic prescribing quality in OOH primary care, tailoring evidence-based interventions, co-designed with local stakeholders and using a PAR approach, is feasible and can be effective, but is messy. A cross-pollination of ideas developed at different GPCs led to exchange and adoption of best practices. Engaging a large and diverse group of GPs remains challenging. PAR addresses the need to understand the local context and build trustful relationships, which are critical for implementing interventions.

Funding

This work was supported by the Faculty of Medicine and Health sciences of the University of Antwerp through a PhD fellowship.

Transparency declarations

None to declare.

Author contributions

A.C. led the project, with supervision by S.C., R.R., H.P. and S.A. Together they designed the study set-up. S.T., O.V, A.T. and Y.B. were involved as local co-researchers in setting up the study, data collection and analysis. A.C. was involved in data collection and analysis. H.V. was responsible for statistical analysis. A.C., S.C., S.A. and H.P. interpreted the data. A.C. wrote, and led the manuscript development and revisions. All authors read, revised and approved the final manuscript

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Supplementary data

Interview guide/questionnaires and Tables S2.1 and S2.2 are available as Supplementary data at JAC-AMR Online.