Metrics for evaluating antibiotic use and prescribing in outpatient settings

Abstract

Antimicrobial stewardship interventions in outpatient settings are diverse and a variety of outcomes have been used to evaluate these efforts. This narrative review describes, compares and provides specific examples of antibiotic use and other prescribing measures to help antimicrobial stewards better understand, interpret and implement metrics for this setting. A variety of data have been used including those generated from drug sales, prescribing and dispensing activities, however data generated closest to when an individual patient consumes an antibiotic is usually more accurate for estimating antibiotic use. Availability of data is often dependent on context such as information technology infrastructure and the healthcare system under consideration. While there is no ideal antibiotic use or prescribing metric for evaluating antimicrobial stewardship activities in the outpatient setting, the intervention of interest and available data sources are important factors. Common metrics for estimating antimicrobial use include DDD per 1000 inhabitants per day (DID) and days of therapy per 1000 inhabitants/day (DOTID). Other prescribing metrics such as antibiotic prescribing rate (APR), proportion of prescriptions containing an antibiotic, proportion of prolonged antibiotic courses prescribed, estimated appropriate APR and quality indicators are used to assess specific aspects of antimicrobial prescribing behaviour such as initiation, selection, duration and appropriateness. Understanding the context of prescribing practices helps to ensure feasibility and relevance when implementing metrics and targets for improvement in the outpatient setting.

Introduction

Antimicrobial stewardship interventions in outpatient settings are diverse, ranging from broad public awareness campaigns and health system level policy changes to specific primary care interventions such as educational workshops and audit and feedback directed at individual prescribers.¹ Since the goals of antimicrobial stewardship efforts are to optimize appropriate antibiotic prescribing while minimizing unnecessary antibiotic exposure and reduce harms, a variety of metrics have been used to evaluate outpatient antimicrobial stewardship programmes. A systematic review evaluating the impact of outpatient stewardship programmes on outcomes found that prescribing outcomes (antibiotic use or prescribing rate, antibiotic selection, duration and guideline concordance) were frequently included, but that most studies were not designed to measure other outcomes (patient-centred and antimicrobial resistance).² It has also been suggested that outpatient antimicrobial stewardship programmes should at a minimum track overall antibiotic use and other prescribing outcomes such as prescribing for specific infections and agent selection, acknowledging that benchmarking and tracking such metrics is challenging in this setting.³

The objective of this review is to describe, compare and contrast antibiotic use and other prescribing metrics to help antimicrobial stewards better understand, interpret and implement antimicrobial use and other prescribing metrics in the outpatient setting as antimicrobial stewardship efforts in this setting grow. As there are advantages and drawbacks to each metric, as well as a lack of standardization, the nature of interventions being evaluated and the available data sources are important factors when deciding between specific metrics. Table 1 provides a summary of strengths and limitations as well as examples of how these metrics have been used in the literature.

Table 1.

Examples of outpatient antibiotic use and prescribing metrics

Metric	Description	Advantages/limitations	Examples of use
Antibiotic use
DID	Represents the volume of antibiotics used (measured in DDDs) at a population level.	Advantages: Can be calculated using drug purchasing data or dispensing data. Suitable for comparison across regions/countries and over time as DDD is a technical unit of measurement independent of price, currencies, package size and strength.7 Limitations: WHO-assigned conversion factor may not always reflect dosing most commonly used in practice. DDD underestimates AMU in individuals requiring reduced doses (e.g. renal insufficiency) and in the paediatric population. Availability of population-level data often lags in time and may not be practical to guide ASP efforts within a practice.	Evaluation of a public information campaign on APRs for outpatients (regional level).4 $\mathrm{DID}=\frac{\mathrm{Total} \mathrm{defined} \mathrm{daily} \mathrm{dose}}{1000 \mathrm{inhabitants} \mathrm{per} \mathrm{day}}$
DOTID	Represents total number of days of antibiotic exposure at a population level.	Advantages: More accurate estimate of exposure than DDD/DID in paediatric population and individuals requiring reduced doses. Limitations: Requires comprehensive population-level AMU data including information about duration of therapy (e.g. duration of therapy prescribed, days supplied).	Nationwide study of outpatient antimicrobial utilization patterns for children (country level).11 $\mathrm{DOT}/\mathrm{PID}=\frac{\mathrm{DOT} \mathrm{of} \mathrm{all} \mathrm{antibiotics} \mathrm{dispensed}}{1000 \mathrm{paediatric} \mathrm{inhabitants} \mathrm{per} \mathrm{day}}$
Prescribing
APR	A measure of antibiotic initiation relative to a population or sub-population, patient years or patient visits.	Advantages: Can be calculated using prescribing or dispensing data. Useful for personalized data and peer-comparison between prescribers. APR may be a more clinically relevant metric to influence behaviour change in prescribers than measures of total antibiotic use such as DID. Limitations: Does not address appropriateness of antibiotic selection or duration. If adjusted for patient characteristics, access to demographic and/or clinical data is required. Does not address accuracy of diagnosis. APR is a point estimate of appropriateness across a cohort of patients and does not evaluate appropriateness for individual patients.	Evaluation of an education strategy targeting physicians and pharmacists on antibiotic prescribing (provincial level).17 $\mathrm{APR}=\frac{\mathrm{Total} \mathrm{outpatient} \mathrm{antibiotic} \mathrm{prescriptions}}{1000 \mathrm{population}}$ Evaluation of providing social norm feedback to high prescribers of antibiotics on unnecessary prescribing (practice level). STAR-PU adjusts for patient characteristics in a practice (e.g. age, group, gender).16 $\mathrm{Adjusted} \mathrm{APR}=\frac{\mathrm{Antibiotic} \mathrm{items} \mathrm{dispensed}}{1000 \mathrm{weighted} \mathrm{population}}$
Proportion of prescriptions containing an antibiotic	Number of antibiotic prescriptions divided by the total number of prescriptions.	Advantages: Can be determined using prescription information only. Limitations: May not accurately reflect volume of visits or patient complexity. Subject to variability in overall prescribing volume, which may not be an accurate reflection of patient volume.	Assessment of long-term effect of an interactive educational seminar on antibiotic prescriptions (prescriber level).20 $\mathrm{\%}=\frac{\mathrm{Antibiotic} \mathrm{prescriptions}}{\mathrm{Total} \mathrm{prescriptions}}$
Proportion of prolonged antibiotic course prescribed	Describes duration of therapy; number of antibiotic prescriptions >specific number of days divided by total number of antibiotic prescriptions.	Advantages: Can be determined using prescription information only. Prolonged durations represent an important component of inappropriate antibiotic prescribing. Prescribers’ preference for prolonged antibiotic duration does not generally overlap with initiation. Limitations: Does not address initiation or selection.	Evaluation of inter-physician variability in prescribed antibiotic treatment durations (prescriber level).21 $\mathrm{\%}=\frac{\mathrm{Antibiotic} \mathrm{courses}>8 \mathrm{days}}{\mathrm{Total} \mathrm{number} \mathrm{of} \mathrm{antibiotic} \mathrm{courses}}$
Appropriateness of antibiotic prescribing	Assesses adherence of antibiotic prescribing to recommendations, guidelines or other criteria; may include initiation, selection, dosing and/or duration.	Advantages: Optimal metric to evaluate quality of prescribing. Limitations: Challenging and labour-intensive to measure; may require granular clinical information that may not be readily available. Difficult to compare across regions/practices due to differences in resistance patterns and prescribing guidelines. Assumes guidelines or criteria being used to evaluate appropriateness are the gold standard. Requires that guidelines be available for conditions being evaluated.	Evaluation of behavioural interventions on inappropriate antibiotic prescribing during ambulatory visits for acute RTI (practice level).27 $\mathrm{APR}=\frac{\mathrm{Antibiotic} \mathrm{prescriptions} \mathrm{for} \mathrm{RTI}}{\mathrm{Total} \mathrm{visits} \mathrm{for} \mathrm{RTI}}$ Evaluation of intervention to improve quality of antimicrobial prescribing for urinary tract infections (practice level).22 % appropriate $=\frac{\mathrm{Prescriptions} \mathrm{for} \mathrm{first}‐\mathrm{line} \mathrm{antibiotics} \mathrm{for} \mathrm{UTI}}{\mathrm{Total} \mathrm{antibiotic} \mathrm{prescriptions} \mathrm{for} \mathrm{UTI}}$
Estimated appropriate APR	Benchmark estimates considered to be an appropriate, ideal APR for target conditions.	Advantages: Established using a consensus methodology; often based on rates derived from literature, guidelines and/or administrative data. Useful for regional or large group comparisons of antibiotic prescribing. Does not require detailed review of individual patient records. Limitations: Requires linkage of prescribing or dispensing information to a clinical condition or diagnosis most often through administrative data. Does not address accuracy of diagnosis. APR is a point estimate of appropriateness across a cohort of patients and does not evaluate appropriateness for individual patients.	Estimation of proportions of antibiotic use that may be inappropriate among US ambulatory care visits. Diagnosis-specific appropriate APR estimates based on expert review of national guidelines and regional APRs from national surveys.28 Evaluation of appropriateness of antibiotic prescribing for different clinical conditions in primary care clinics in the UK. Expert elicitation used to generate benchmark estimates of ideal APR for conditions where antibiotic prescribing was classified as ‘sometimes appropriate’.29 Estimation of unnecessary APRs in a Canadian primary care setting.31 Expected appropriate APR for 23 conditions defined using modified Delphi method.30
QI for antibiotic use	Specific and measurable elements of practice that can be used to assess the quality of AMU.	Advantages: Facilitates comparisons between practices, regions or countries. Established using a consensus methodology. Limitations: Requires comprehensive effort to determine; may require linking of antibiotic prescription data with diagnosis and patient-level data depending on QI.	Development of valid drug-specific QIs for outpatient antibiotic use in Europe, derived from ESAC data (e.g. consumption of fluoroquinolones expressed as a percentage of total consumption of all antibacterial agents).32 Development of QIs for diagnosis and treatment of RTI in general practice derived from literature review using modified Delphi method (e.g. number of patients aged >2 years with less than 3 days of symptoms of acute otitis media treated with antibiotics).33

AMU, antimicrobial use; ASP, antimicrobial stewardship programme; STAR-PU, specific therapeutic group age-sex related prescribing units; RTI, respiratory tract infections; ESAC, European Surveillance of Antimicrobial Consumption.

Available sources of data

Antibiotic use and other prescribing metrics are typically based on drug sales (or purchasing), prescription or dispensing data, and availability of these data is often tied to factors such as information technology infrastructure and features of the healthcare system (i.e. how healthcare is delivered and funded) under consideration. It would be expected that data that are generated closest to when a patient consumes an antibiotic will more accurately estimate antibiotic use. For inpatient (e.g. hospital) and long-term care settings, actual consumption is documented using medication administration records. However, in the outpatient setting, there is no overarching mechanism to capture this information and, as a result, antibiotic use is estimated based on drug sales, prescription and/or dispensing data.

Antibiotic sales data from wholesale distributors or pharmacies serve as an overall proxy for antimicrobial exposure in the community. Prescription and dispensing data from community pharmacies, insurance claims databases or medical records can provide more granular information about antibiotic utilization (e.g. doses supplied, between-class comparisons). Prescription and dispensing data are imperfect and can potentially overestimate or underestimate use depending on the context. For example, prescription data can overestimate use because not all patients will end up filling their prescriptions. Dispensing data can also overestimate use if patients fill their prescriptions but do not adhere to therapy. However dispensing data could also underestimate use if there is an alternate supply (e.g. over the counter, drug samples) or if they are incomplete (e.g. prescriptions paid for by patients not included in insurance claims).

Antibiotic use metrics

Defined daily dose per 1000 inhabitants per day (DID) is widely used to evaluate the impact of broad antimicrobial stewardship interventions such as public education campaigns.^4–6 Endorsed by WHO as the international standard for drug utilization monitoring, DDD is a common measure of antimicrobial utilization for tracking antibiotic use in inpatient and outpatient settings. DDD is ‘the assumed average maintenance dose per day for a drug used for its main indication in adults’ and is calculated by converting antimicrobial grams used by a drug-specific WHO-assigned DDD conversion factor.⁷ The main advantage of DDD is that it facilitates comparison across regions and countries over time despite differences in cost, pack size, duration and dose and can be calculated using drug purchasing or dispensing data. However, the WHO-assigned conversion factor may not always reflect dosing most commonly used in practice, particularly for individuals requiring reduced dosing such as children and those with renal impairment. Population denominators are normally based on census information or the WHO mid-year population, which is an estimate of the resident population on 1 July of each year.^8,9

Days of therapy per 1000 inhabitants/day (DOTID) is another metric used to estimate antibiotic use at a population level. Days of therapy (DOT), also called antibiotic days or antimicrobial days, refers to the number of days that a patient receives an antimicrobial agent. When normalized by 1000 inhabitants per day, DOTID represents the total number of days of antibiotics at a population level. This metric is more accurate at estimating antimicrobial exposure than DDD in some populations because it does not depend on dose. However, it is important to note that when used to describe antibiotic use in hospital settings, DOT is typically based on administered doses, whereas in the outpatient setting it is still only an estimate of patient consumption unless data sources such as patient diaries or surveys are incorporated.¹⁰ Information about intended duration of therapy such as the DOT from prescription data or days supplied from dispensing data is therefore used to determine DOT in outpatient settings.¹¹

Antibiotic prescribing metrics

Antibiotic prescribing rate (APR) is a measure of antibiotic initiation relative to a population or sub-population, patient years or patient visits. Choosing patient population or patient years as a denominator may be more appropriate if trying to evaluate APR broadly (i.e. at a national/regional level), especially if outpatients are able to obtain antibiotics from multiple settings (i.e. urgent care, general practice) and/or from different types of prescribers (i.e. physician and non-physician). At a local level, selecting patient visits as a denominator may be desirable if attempting to evaluate APR for an individual or group of prescribers within a practice specifically related to patient encounters for certain infectious conditions. For example, APR is often calculated for antibiotics of interest or targeted conditions such as respiratory tract infections where prescription or dispensing data can be linked to information about indication, diagnosis or reason for healthcare visits. APR is commonly used to evaluate interventions to change prescriber behaviour including healthcare system policy and funding changes, peer-comparison and education programmes.^12–17 Although this metric may lack adjustment for patient complexity, APR has been shown to be strongly linked to inappropriate or unnecessary prescribing.¹⁸ There is substantial inter-physician variability in prescribing that is not explained by differences in patient populations or practice characteristics, suggesting APR is a useful proxy for antibiotic appropriateness.¹⁹

Proportion of prescriptions containing an antibiotic is an alternative measure of antibiotic initiation that is based on the total number of prescriptions rather than a measure of patient population or visits. Like APR, this metric has been used to evaluate interventions aimed at influencing antibiotic initiation by prescribers. However, important limitations are that the numerator and denominator may be highly correlated and it does not adjust for patient volume, patient complexity or a prescriber’s tendency to prescribe other medications, which could compromise its accuracy.²⁰

Proportion of prolonged antibiotic courses prescribed is used to describe duration of therapy, such as the number of antibiotic prescriptions greater than a specified number of days. Inclusion of this metric in assessment of individual prescriber tendencies has demonstrated that prescribers’ preference for prolonged antibiotic duration does not generally overlap with initiation, therefore it can be used together with APR or proportion of prescriptions containing an antibiotic to enable more comprehensive evaluation of prescribing behaviour.²¹

Appropriateness of antimicrobial prescribing refers to concordance of prescribing to recommendations, guidelines or other criteria and usually encompasses multiple facets of prescribing (e.g. combination of initiation, selection, dosing and/or duration). While often considered to be an optimal metric for evaluating the quality of antimicrobial prescribing, it is labour-intensive to determine and may require granular clinical information that is difficult to access, making inclusion of this metric more feasible where there exists a mature information technology infrastructure or for periodic audits for a defined timeframe.^22–24 It can also be challenging to compare across regions/practices due to differences in resistance patterns and prescribing guidelines, and also assumes that guidelines themselves being used to evaluate appropriateness are the accepted gold standard.

To partially address some of these challenges, appropriateness has also been evaluated using the following mutually exclusive categories: optimal (i.e. use of antibiotics is aligned with guidelines), adequate (i.e. use of antibiotics is reasonable and mostly aligned with guidelines), suboptimal (i.e. use of antibiotics for established infections where selection, dose or route can be improved), inappropriate (i.e. established infections where pathogen is resistant, or antimicrobial is not recommended) and unnecessary (i.e. use of antibiotics for non-infectious or non-bacterial syndromes).^25,26 The latter category has been the focus of outpatient stewardship interventions aimed at reducing antibiotic prescribing for acute respiratory conditions (e.g. acute bronchitis, influenza) that are considered conditions or diagnoses where antibiotics are unnecessary.²⁷ Table 2 provides examples of how this framework can be used to evaluate appropriateness of outpatient antibiotic prescribing for uncomplicated cystitis.

Table 2.

Evaluating appropriateness of antimicrobial prescribing^25,26

Category		Description	Examples for uncomplicated cystitis and asymptomatic bacteriuria
Appropriate	optimal	Prescription aligns with guidelines in terms of agent, dose, route and duration.	Nitrofurantoin 100 mg orally twice daily × 5 days.
	adequate	Prescription aligns mostly with guidelines and is considered reasonable, but selection, dose, route or duration are less than optimal.	Trimethoprim/sulfamethoxazole 160 mg/800 mg orally twice daily × 5 days (i.e. duration could be shorter at 3 days, nitrofurantoin may be recommended as first-line therapy).
Inappropriate	suboptimal	Prescription is considered unreasonable due to excessively broad spectrum, higher than necessary dose, prolonged duration or lack of de-escalation (route or spectrum) when indicated.	Ciprofloxacin 500 mg orally twice daily × 3 days (i.e. spectrum too broad).
	inadequate	Prescription is deemed not adequate to treat or prevent infection. The antibiotic selection is a mismatch with cultured or suspected pathogen, route or dose will not provide adequate concentration at site of infection.	Amoxicillin 500 mg orally three times daily empirically (i.e. amoxicillin has inadequate activity against most urinary pathogens).
	unnecessary	The indication does not require any antimicrobial therapy.	Nitrofurantoin 100 mg orally twice daily for asymptomatic bacteriuria in a non-pregnant patient (i.e. antibiotics not required).

Alternatively, the use of APR and quality indicators are additional methods for evaluating appropriateness of prescribing, especially as it pertains to a large population of interest or for multiple points in time.

Estimated appropriate APRs are point estimates that approximate an appropriate, ideal APR for target conditions and are often established using consensus methodology and based on a combination of prescribing rates derived from literature, guidelines, billing or clinical data.^28–30 For example, in a study of US ambulatory care visits, the regional rates of visits with antibiotics prescribed were used to determine APRs for sinusitis and otitis media (e.g. 27% and 20% respectively for adults <64 years of age).²⁸ Similar to appropriateness of antibiotic prescribing, in order to facilitate benchmarking, conditions are commonly grouped as antibiotic appropriate (i.e. conditions for which antibiotics are almost always indicated), sometimes appropriate (i.e. conditions for which antibiotics may be indicated) and antibiotic inappropriate (i.e. conditions for which antibiotics are almost never indicated). Estimated appropriate APRs for these categories are then used as benchmarks for evaluating observed APRs at a population, practice or prescriber level.^28,29,31 While estimated appropriate APRs do not address the accuracy of diagnosis, nor do they evaluate appropriateness of prescriptions for individual patients, a key advantage is that they do not require detailed review of clinical records and can therefore be more efficiently used for a larger population.

Quality indicators (QIs) for antibiotic use refer to specific and measurable elements of practice that can be used to assess the quality of antimicrobial prescribing from different perspectives at an aggregate level. QIs can be specific to a condition, antibiotic agent, antibiotic prescribing decision or diagnostic process or can be general and capture overall antibiotic use. For example, the proportion of all antibiotics prescribed that are fluoroquinolones is a QI specific to an antibiotic class of interest. QIs are commonly established using a consensus methodology and aim to facilitate comparison of various aspects of antibiotic use and prescribing between practices, regions or countries.^32–34

Implementation considerations

In addition to factors already highlighted, the intended and unintended impacts of antimicrobial stewardship interventions, baseline antibiotic use and expected degree of change are key considerations when implementing metrics. Recent evidence indicates that, in the outpatient setting, the most common indications for which antibiotics are prescribed inappropriately include the common cold, bronchitis and sinusitis, however this may vary by practice setting.³¹ It should be noted that interventions to improve prescribing for targeted conditions for which antibiotics are overprescribed may result in ‘diagnosis shifting’ where clinician diagnosis patterns shift away from targeted antibiotic inappropriate conditions (e.g. acute bronchitis) to other, antibiotic appropriate conditions (e.g. pneumonia) in order to justify antibiotic prescribing.³⁵ Similarly, interventions to improve appropriate prescribing of specific antibiotics may result in unintentional shifts to non-preferred antibiotics, a phenomenon often known as ‘squeezing the balloon’.³⁶ Recognizing the potential for these unintended impacts, it is important to evaluate overall antibiotic use and prescribing outcomes in addition to tracking these measures only for targeted conditions, individual antibiotics or class of antibiotics alone.^35,36 While not discussed in this review, other metrics pertaining to clinical patient outcomes and balancing measures such as repeat visits, hospital or emergency department visits and adverse drug events should also be considered for inclusion. Finally, establishing a numerical or percentage target for improvement is crucial and selection should be informed by baseline antibiotic use or appropriateness and the extent to which this can potentially be improved. This will vary substantially by region and context, however studies have observed that 25% to 30% of antibiotics prescribed in outpatient settings are unnecessary, which can help determine feasible targets.^28,31

Conclusions

This review provides an overview and examples of antibiotic use and prescribing metrics that are commonly used to evaluate outpatient antimicrobial stewardship initiatives, each with unique advantages and limitations. Since no ideal metric exists, antimicrobial stewards should take into consideration availability of data, labour intensity and the relevance of specific metrics to the particular aspect of antibiotic use or prescribing they are seeking to monitor or change.