Defining appropriate antibiotic prescribing in primary care: A modified Delphi panel approach

Abstract

Background

Antimicrobial overuse contributes to antimicrobial resistance. In the ambulatory setting, where more than 90% of antibiotics are dispensed, there are no Canadian benchmarks for appropriate use. This study aims to define the expected appropriate outpatient antibiotic prescribing rates for three age groups (<2, 2–18, >18 years) using a modified Delphi method.

Methods

We developed an online questionnaire to solicit from a multidisciplinary panel (community–academic family physicians, adult–paediatric infectious disease physicians, and antimicrobial stewardship pharmacists) what percentage of 23 common clinical conditions would appropriately be treated with systemic antibiotics followed with in-person meetings to achieve 100% consensus.

Results

The panelists reached consensus for one condition online and 22 conditions face-to-face, which took an average of 2.6 rounds of discussion per condition (range, min–max 1–5). The consensus for appropriate systemic antibiotic prescribing rates were, for pneumonia, pyelonephritis, non-purulent skin and soft tissue infections (SSTI), other bacterial infections, and reproductive tract infections, 100%; urinary tract infections, 95%–100%; prostatitis, 95%; epididymo-orchitis, 85%–88%; chronic obstructive pulmonary disease, 50%; purulent SSTI, 35%–50%; otitis media, 30%–40%; pharyngitis, 18%–40%; acute sinusitis, 18%–20%; chronic sinusitis, 14%; bronchitis, 5%–8%; gastroenteritis, 4%–5%; dental infections, 4%; eye infections, 1%; otitis externa, 0%–1%; and asthma, common cold, influenza, and other non-bacterial infections (0%). (Note that some differed by age group.)

Conclusions

This study resulted in expert consensus for defined levels of appropriate antibiotic prescribing across a broad set of outpatient conditions. These results can be applied to community antimicrobial stewardship initiatives to investigate the level of inappropriate use and set targets to optimize antibiotic use.

Antimicrobial overuse is the most important driver of antimicrobial resistance, and antimicrobial resistance is an emerging global public health crisis (1). Antimicrobial stewardship programs have been developed to optimize the appropriate use of antibiotics so that these lifesaving medications continue to be effective for future generations (2). Antimicrobial stewardship programs are an Accreditation Canada Required Organizational Practice for hospitals (3); however, more than 90% of antibiotic use in Canada occurs outside hospital settings (4).

The extent of unnecessary antibiotic use in primary care in Canada is not well known; however, studies from the United States have shown that 30%–50% of antibiotic courses are prescribed unnecessarily (5,6). However, US data may not be directly applicable to Canadian primary care settings. Moreover, the assumptions used to define US benchmarks were not derived from a rigorous process, such as a modified Delphi method. The modified Delphi technique is a scientific research technique to build consensus. Setting realistic targets for interventions to reduce and optimize antibiotic use through antimicrobial stewardship initiatives is essential. To ascertain the quantity of unnecessary antibiotic use, localized benchmarks are needed specific to primary care settings; however, evidence necessary to inform Canadian benchmarks is currently lacking (1).

The objective of this study was to use a modified Delphi approach to rigorously establish expected appropriate outpatient antibiotic prescribing rates for 23 clinical conditions in the Canadian primary care setting.

Method

Rigorous application of scientific research techniques to achieve consensus, such as the Delphi Panel methodology, allows for an unbiased survey of experts in a high-quality, evidence-based and scientific manner when guidelines are insufficient or not available (7–10). In the traditional Delphi technique, participants complete self-administered questionnaires, and the research team sends back the collated results and feedback for additional rounds of review until consensus is reached. This process is completely anonymous; there are no direct interactions among participants (8). The traditional approach is not always feasible to implement because it can be a lengthy process and does not provide the opportunity for real-time discussion. The modified Delphi method remains a structured and iterative process that systematically collects informed judgement through repeated rounds of anonymous voting to achieve expert consensus when uncertainty may exist because little or no definitive evidence is available (7). Unlike the traditional approach, the modified Delphi method provides panelists with opportunities to discuss judgement while building consensus. Therefore, this approach has increasingly been used in various areas of medicine to identify quality indicators for diagnosis and treatment recommendations (11–15).

We sent the invitation to participate in this study in April 2018, sent the link to the online survey via email in August 2018, and held three in-person meetings from August to October 2018 in Toronto.

Panel preparation and setting

This modified Delphi method incorporated a nominal group technique to allow in-person discussions and shorten the time requirement while minimizing bias from influential participants by retaining some degree of anonymity. The multidisciplinary panel consisted of a purposive sample of nine local experts: one pediatric and three adult infectious disease physicians, three family physicians, and two antimicrobial stewardship pharmacists. Five experts practiced clinically in a community hospital or clinic, and four primarily practiced in academic centres. All nine experts agreed to participate and declared no conflicts of interest. In-person meetings were held 1 month apart for a maximum of 3 hours per meeting to avoid fatigue. Anonymous input was solicited from each panel member and kept confidential to minimize bias and peer pressure. There was no monetary incentive to participate, and panelists understood that the process would be repeated until consensus was reached for all conditions.

We a priori selected 23 clinical conditions to be adjudicated by the expert panel. We chose conditions that could be identified using administrative data to inform future studies and establish the appropriateness of antibiotic use with administrative datasets. An evidence summary table was created using available evidence and guidelines for the 23 clinical conditions, including information such as expected probability of spontaneous resolution of the condition, proportions of each condition that are bacterial versus viral in origin, and diagnostic challenges. This table was distributed to the expert panel to facilitate the discussion (Supplemental Appendix 1). The objective was to determine appropriate systemic antibiotic prescribing rates for each condition on the basis of scientific evidence, medical advice, and the health care provider’s clinical experience. These expert-informed appropriate prescribing rates were expected to be used in future studies to assess community prescribing as reflected in administrative billing claims, so the experts were instructed not to consider factors such as diagnostic accuracy but to assume the recorded billing claim accurately represented the physician’s most likely clinical diagnosis of the patient at the time of assessment. Only the use of oral systemic antibiotics was considered.

This modified Delphi method consisted of one round of online questionnaire sent to panelists via email followed by as many rounds as needed during face-to-face meetings to reach 100% consensus (see Figure 1). This study was designed to be utilized in the Canadian primary care context at the population level to estimate appropriate antibiotic prescribing rate, thus, we aimed to include most commonly seen clinical conditions in primary care where the appropriateness of antibiotic prescribing could be assessed. (Supplemental Appendix 2).

Figure 1:

Flowchart illustrating the modified Delphi steps taken to reach 100% consensus for all 23 conditions

Modified Delphi round 1: email questionnaire

An online survey was developed to assess a total of 69 items—23 conditions stratified by three age group (<2 y, 2–18 y, >18 y) to further differentiate the potential variability in prescribing for conditions that may be age dependent. The survey was distributed to panel experts with a summary table of existing evidence from the literature. The survey repeated the same question structure with different age group and condition combinations (e.g., ‘What is the appropriate antibiotic prescribing rate for children less than two years of age with the common cold?’). Panel experts were asked to provide a specific point-estimate percentage of appropriate antibiotic prescribing for each item (see Supplementary Appendix 3 for a screenshot of the survey). The facilitator collated the results, and the mode of each result was used to start the face-to-face discussion with the most commonly reported appropriateness percentage.

Modified Delphi rounds: face-to-face discussions and anonymous voting

Items for which consensus was reached during round 1 were removed from the pool, and those for which consensus was not reached were discussed during face-to-face meetings with a facilitator. Panel experts were asked to anonymously provide an appropriate percentage of antibiotic prescribing after the group discussion. The mean of these results was calculated and presented to the panelists for an anonymous vote to agree or disagree with this mean result as the appropriateness benchmark. The voting was done via online polling and used the same question structure with different condition and age group combinations (e.g., ‘For the common cold in children less than two years, will you accept an appropriate antibiotic prescribing rate of X%?’). This process was repeated until 100% consensus was reached among panel experts for all conditions.

Statistical analysis

Once all conditions met unanimous consensus, the 69 final percentages were presented descriptively as point estimates. The range of values at each voting round was presented graphically. The final point estimates for all conditions were further organized into tiers because previous studies have used similar tier classification systems to organize clinical conditions into groupings of whether antibiotics are generally indicated, sometimes indicated, or generally not indicated (5,6).

Previous studies have not used a rigorous method to define these categories; therefore, we created a revised tier system based on this modified Delphi process:

1. tier 1: always indicated (expected prescribing rate of 100%),

2. tier 2a: frequently indicated (expected prescribing rate of 51%–99%),

3. tier 2b: sometimes indicated (expected prescribing rate of 21%–50%),

4. tier 2c: rarely indicated (expected prescribing rate 1%–20%), and

5. tier 3: never indicated (expected prescribing rate of 0%).

Results

We successfully defined appropriate systemic antibiotic prescribing rates for all 23 clinical conditions with 100% consensus reached. An average of 2.6 rounds was required to reach consensus for 69 items (23 conditions × 3 age groups) with a minimum of one and a maximum of five rounds. Figure 2 shows the spread of percentages collected from the first round of survey results before in-person discussion. Some conditions had a much larger range than others. For example, purulent skin and soft tissue infection for adults ranged initially from 0% to 90%, but the panel was able to reach consensus after three rounds of in-person discussion. Figure 3 graphically illustrates the consensus process for two of the conditions (graphs for the other conditions are available in Supplemental Appendix 4). The graphs consistently show that iterative rounds of in-person discussion successfully built consensus among the panelists as the spread of values became narrower and narrower and a final value was ultimately arrived at.

Figure 2:

Spread of percentages collected from panelists during the first round of survey responses for adults before in-person discussion sorted by mean

Note: Each dot represents a percentage given by a panelist

* Includes typhoid fever, tuberculosis, pertussis, and septicemia

† Includes herpes simplex, herpes zoster, infectious mononucleosis, warts, other viral illnesses, ring worm, candidiasis, stomatitis, and serous otitis media

COPDE = Chronic obstructive pulmonary disease exacerbation; SSTI = Skin and soft tissue infection

Figure 3:

Illustration of consensus building through iterative rounds of in-person discussion using 2 of 66 items discussed in person

Note: Size of diamond is proportional to the number of responses

Pyelonephritis across all three age groups (i.e., 3 of 69 items) was the only condition for which consensus was reached during the initial round of survey questionnaires. The remaining 66 items were discussed during face-to-face meetings over 3 half-days. Consensus was reached on the majority (82%; 54 of 66) of the conditions in two rounds of discussions, with consensus on more than half (35 of 66) being reached after one round. Antibiotics were considered to be always appropriate for five conditions for all age groups: non-purulent skin and soft tissue infections (SSTIs), pyelonephritis, pneumonia, reproductive tract infections, and other bacterial infections. For the common cold, influenza, asthma, and other nonbacterial infections, antibiotics were considered to never be appropriate. The complete list of results for all conditions by the three age groups is provided in Table 1. Table 2 groups the 23 conditions into tiers on the basis of how often antibiotics are indicated. Most conditions fall into the same tier regardless of age group except urinary tract infections, otitis externa, and pharyngitis.

Table 1:

Expected appropriate oral antibiotic prescribing rates in a Canadian primary care setting for 23 common clinical conditions stratified in descending order by 3 age groups

	Appropriate prescribing rate (%) for each age group
Conditions	<2 y	2–18 y	>18 y
Non-purulent skin and soft tissue infection	100	100	100
Pyelonephritis	100	100	100
Pneumonia	100	100	100
Reproductive tract infections	100	100	100
Other bacterial infections*	100	100	100
Urinary tract infections	100	100	95
Prostatitis	—	—	95
Epididymo-orchitis	—	85	88
Chronic obstructive pulmonary disease exacerbation	—	—	50
Purulent skin and soft tissue infection	50	50	30
Acute otitis media	40†	30	—
Acute pharyngitis	18	40	28
Acute sinusitis	20	20	18
Chronic sinusitis	14	14	14
Acute bronchitis	5	8	8
Gastroenteritis	4	4	5
Dental conditions	4	4	4
Eye infections	1	1	1
Otitis externa	0	0	1
Asthma	0	0	0
Common cold	0	0	0
Influenza	0	0	0
Other non-bacterial infections‡	0	0	0

Note: Dash indicates not applicable

* Includes typhoid fever, tuberculosis, pertussis, and septicemia

† Aged 6 mo–2 y; for children aged <6 mo, antibiotics were considered appropriate for this condition 100% of the time

‡ Includes herpes simplex, herpes zoster, infectious mononucleosis, warts, other viral illnesses, ringworm, candidiasis, stomatitis, and serous otitis media

Table 2:

Tiers of the 23 clinical condition by level of appropriate antibiotic prescribing in percentages

	Tier 2
Tier 1: Always indicated (100%)	2a: Frequently indicated (51–99%)	2b: Sometimes indicated (21–50%)	2c: Rarely indicated (1–20%)	Tier 3: Never indicated (0%)
Pneumonia Pyelonephritis N on-purulent SSTI Other bacterial infections* Reproductive tract infections Urinary tract infections (aged ≤18 y)	Urinary tract infections (aged >18 y) Prostatitis Epididymo-orchitis	COPDE Purulent SSTI Otitis media Pharyngitis (aged ≥2 y)	Acute sinusitis Chronic sinusitis Acute bronchitis Gastroenteritis Dental conditions Eye infection Otitis externa (aged >18 y) Pharyngitis (aged <2 y)	Asthma Common cold Influenza Other non-bacterial infections† Otitis externa (aged ≤18 y)

* Includes typhoid fever, tuberculosis, pertussis, and septicemia

† Includes herpes simplex, herpes zoster, infectious mononucleosis, warts, other viral illnesses, ring worm, candidiasis, stomatitis, and serous otitis media

SSTI = Skin and soft tissue infections; COPDE = Chronic obstructive pulmonary disease exacerbations

Certain conditions in selected age groups were determined by consensus to be not applicable either because of insufficient evidence to define appropriateness rates or because the condition did not occur in sufficient numbers in that age group. These conditions included acute otitis media for adults aged older than 18 years, prostatitis and chronic obstructive pulmonary disorder exacerbation for children aged younger than 18 years, and epididymo-orchitis for children aged younger than 2 years.

Discussion

Using a modified Delphi method, we defined appropriate systemic antibiotic prescribing rates for 23 clinical conditions across three age groups in a Canadian primary care setting. We used a robust methodology of expert consensus to organize these conditions so that we could estimate the extent of unnecessary antibiotic use in primary care. Conducting antimicrobial stewardship in the community to optimize antibiotic use is a national priority (1). Localized assessment of unnecessary antibiotic use is essential because of the large regional variability in antibiotic use, and it is likely much of this variability is driven by differences in inappropriate use (16,17). These results can be used to establish realistic and safe targets for reduction in community antibiotic use.

The results of our study are designed to be used in the Canadian primary care context at the population level to estimate appropriate antibiotic prescribing rates. In this context, our study has a number of advantages. We attempted to include the most common clinical conditions seen in a primary care setting in which antibiotics may be prescribed and that could be ascertained through physician billing claims. A number of common conditions have been previously validated with administrative data (18–20). Moreover, we stratified our experts’ opinions by clinically relevant age groupings.

A study in the United States estimated appropriate antibiotic prescribing for ambulatory care visits separated by age groups of 0–19, 20–64, and 65 years and older (6). Although it is challenging to compare specific percentages with results stratified by different age groups, the tier system used by the US study produced results similar to ours with some differences. The differences were that we categorized antibiotics as rarely indicated for acute bronchitis and as never indicated and rarely indicated, respectively, for adults and children with otitis externa, whereas the US study categorized antibiotics as never indicated for bronchitis and maybe indicated for otitis externa (6). The US study estimated appropriateness on the basis of national guidelines and with regions with lowest prescribing rates as targets (6).

A strength of our study is that we derived our tier classification using a more rigorous modified Delphi method. Two previous studies have derived ideal antibiotic prescribing rates for UK (21) and European (22) primary care settings. The appropriateness rates in those studies are shown and compared with the current consensus-defined Canadian appropriateness rates in Table 3. Smith et al (21) used an expert elicitation consensus methodology to quantify estimates with a range of uncertainty. They used this process only for conditions that were deemed to ‘sometimes require antibiotics’ on the basis of existing guidelines. The estimates were elicited through one-on-one scripted interviews. The European Surveillance of Antimicrobial Consumption generated an antibiotic prescribing quality indicator. They also used a modified Delphi method that included 62 experts over two meetings. All three groups produced similar estimates for conditions such as urinary tract infections (UTIs), acute exacerbation of chronic obstructive pulmonary disease, acute bronchitis, acute sinusitis, and gastroenteritis. There were notable differences for otitis media and acute pharyngitis; our expert panel had slightly higher appropriate rates for a similar age group. It is important to note that different guideline consultation and condition groupings can contribute to the differences in results between ideal prescribing rates among studies. Supplementary Appendix 2 lists the definitions of the conditions our study refers to. The decisions made by our expert panel were based on the physician’s assessment during the time of patient visit and not necessarily from a confirmed diagnosis. For example, physicians are usually recommended to initiate treatment for suspected UTI (23), so our panel’s decision was based on an office visit with symptoms, not on the final results of a diagnostic test such as urinalysis or urine culture. Smith et al and Adriaenssens et al only developed rates for a selected number of conditions, thus limiting comparisons with our study. In addition, these groups did not use a methodology to reach 100% consensus and either reported averages with interquartile ranges (21) or presented only a range of estimates (22).

Table 3:

Comparison of ideal or appropriate prescribing rates among this study, Smith et al, and Adrianssens et al

Condition (age category)	Current study (Canada), %	Smith et al. (21) (United Kingdom) % (IQR)	Adrianssens et al. (22) (European), %, range
Urinary tract infection (adult)	95	75 (61–86)	80–100
COPDE (adult)	50	54 (31–78)	—
Acute otitis media (6–24 mo)	40	19 (9–33)	—
Acute otitis media (≥2 y)	30	—	0–20
Acute bronchitis (adults)	8	13 (6–22)	0–30
Acute sinusitis (adults)	18	11 (5–18)	0–20
Acute pharyngitis (≥2 y)	40	13 (7–22)	0–20
Gastroenteritis (≥2 y)	4	9 (4–16)	—

Note: Dash indicates not applicable

COPDE = Chronic obstructive pulmonary disease exacerbation

This study has some limitations. Although efforts were made to maintain anonymity and minimize peer influence, it is possible that panelists experienced exhaustion or group pressure for conformity during in-person meetings. However, the discussion allowed panelists to explore reasons for disagreements and were encouraged to reassess, modify, and develop second opinions. Voting decisions remained anonymous, to the extent possible, to minimize bias. It has been shown that participants prefer personal communication over computer-mediated interaction, and direct interaction can achieve coherence without losing effectiveness in a non-hierarchical meeting (24). Balanced participation from all panelists is the key strength of a consensus method (9).

One of the notable challenges of a face-to-face meeting is the potential of having a dominant participant who drives the discussion and unduly influences the group decisions (8). The facilitator in this study, a research coordinator with both qualitative and quantitative research experience but no clinical background, was able to remain neutral and moderate a balanced discussion among the panelists. Realizing that consensus on a specific percentage may be difficult to reach, the facilitator emphasized the intent of finding a percentage that the panelists could accept knowing that it might not be the most desired value but was reasonable and within range.

We felt that a point estimate would have the greatest applicability to the field in terms of estimating unnecessary antibiotic prescribing rates in primary care; however, we recognize the degree of subjectivity involved and that there is likely a range of clinically acceptable prescribing rates. These point estimates may not be applicable to all patient populations for whom antibiotic prescribing recommendations may differ (e.g., immune-compromised patients).

Last, our panel consisted of experts from diverse clinical backgrounds, but for logistical reasons they were not geographically representative of Canada. Clinical experience and antibiotic resistance rates may differ across geographical regions (e.g., Indigenous populations on the reserve), and these results should be interpreted within the context of local treatment guidelines.

Conclusion

This study defined appropriate primary care systemic antibiotic prescribing rates for 23 clinical conditions using a rigorous modified Delphi method. Applying the results to antimicrobial stewardship initiatives creates the possibility to determine the level of unnecessary use and set targets to optimize outpatient antibiotic use.

Funding Statement

Funding for this article was provided by the Physician Services Incorporated Foundation.

Competing Interests:

Dr Schwartz reports grants from Physician Services Incorporated Foundation during the conduct of the study.

Ethics Approval:

N/A

Informed Consent:

N/A

Registry and the Registration No. of the Study/Trial:

N/A

Animal Studies:

N/A

Funding:

Funding for this article was provided by the Physician Services Incorporated Foundation.

Peer Review:

This article has been peer reviewed.