Benefit–harm analysis of azithromycin for the prevention of acute exacerbations of chronic obstructive pulmonary disease

Abstract

Background

Low-dose oral azithromycin therapy is recommended as a preventive treatment for acute exacerbations of COPD. However, the overall benefit–harm balance of this treatment has not been well studied.

Methods

A probabilistic Markov model of COPD was created to simulate the course of COPD over 20 years. The model was populated with evidence from the literature and dedicated data analysis. The benefit of azithromycin was modelled as a reduction in exacerbation rates. Adverse events, including cardiovascular events, hearing loss, gastrointestinal symptoms and antimicrobial resistance (leading to a gradual decline in the effectiveness of azithromycin), were considered. All outcomes were assigned a health-related utility weight to estimate the overall net change in the quality-adjusted life year (QALY) associated with the use of azithromycin.

Results

In patients with a positive exacerbation history, azithromycin resulted in a net QALY gain of 17.9 per 100 patients (99.8% probability of expected QALY gain) over 20 years. The net benefit increased to 21.8 QALYs per 100 patients (99.9% probability of expected QALY gain) among the ‘frequent exacerbator’ subgroup. Azithromycin was not net beneficial among those without any moderate/severe exacerbations in the previous year. Findings were robust against series of sensitivity, scenario and threshold analyses.

Conclusions

Long-term therapy with azithromycin confers a net benefit to ex-smoker patients with COPD with a recent history of exacerbations and an even larger benefit to those who are frequent exacerbators.

Key messages.

What is the key question?

• What is the benefit–harm balance of maintenance azithromycin therapy in patients with COPD, and which subgroups benefit the most from it?

What is the bottom line?

• Daily maintenance azithromycin therapy is very likely net-beneficial over 20 years among patients with COPD who are former smokers and have a recent history of exacerbations, especially those who tend to exacerbate frequently.

Why read on?

• Contemporary guidelines express concerns about adverse effects of long-term maintenance azithromycin therapy; our analysis shows that guideline-recommended azithromycin therapy is very likely to be net beneficial, a finding that remained robust to several assumptions regarding the long-term effectiveness and adverse event risks of this treatment.

Introduction

COPD is a common inflammatory lung disorder that is characterised by persistent airflow limitation and periods of acute worsening of respiratory symptoms, called exacerbations.¹ With 3.23 million deaths reported in 2019 alone,² COPD is a leading cause of morbidity and mortality around the world.¹ COPD exacerbations are a major cause of medical hospital admissions across many jurisdictions.³

Prevention of exacerbations is a major goal in COPD management. Pharmacotherapy is a central component of such prevention. A large randomised controlled trial (RCT) involving 1142 subjects has shown that daily use of low-dose azithromycin therapy, a broad-spectrum antibiotic with immunomodulatory properties, reduces the rate of exacerbations by 27%.⁴ Based on such results, the use of maintenance azithromycin is currently recommended for patients who continue to exacerbate despite being on maximal inhaled therapies. However, daily use of azithromycin is associated with side effects, including antibiotic resistance,⁵ impaired hearing,⁴ cardiovascular (CV) events⁶ and gastrointestinal (GI) symptoms,⁵ and there have been concerns about whether the risk of such adverse events outweighs the benefit of azithromycin over the long term.⁵ In light of such concerns, an objective evaluation of the benefits and harms of azithromycin is warranted.

To the best of our knowledge, the benefit–harm balance of azithromycin has not been objectively studied in COPD. A benefit–harm analysis is a method for quantitative assessment of the overall value of a treatment based on consideration of its benefits and side-effects. This framework provides an objective and transparent mechanism for combining the various outcomes of treatment into a single ‘net benefit’ metric that can inform clinical decision-making.⁷ The primary objective of this study was to perform a probabilistic benefit–harm analysis of long-term, low-dose daily azithromycin for the prevention of exacerbations in patients with moderate to severe COPD. Since COPD is a heterogeneous disease, the net benefit of azithromycin can vary according to patient characteristics (specifically based on their exacerbation history). Therefore, our secondary objective was to identify patient subgroups that are most likely to receive a net benefit from azithromycin.

Methods

We conducted a probabilistic model-based benefit–harm analysis of prophylactic azithromycin (250 mg/d) to prevent COPD exacerbations among patients who are already on maximum inhaled therapy. The benefit of azithromycin (exacerbation reduction) and its harm (adverse events) were transformed into quality-of-life weights to generate a single metric of net changes in quality-adjusted life years (QALYs).

The baseline time horizon of the study was 20 years and outcomes were discounted at 3% per year (both parameters were subjected to sensitivity analysis).⁸ The overall characteristics of the population were adapted from the MACRO study, the largest and most conclusive RCT of prophylactic azithromycin in COPD.⁴ In line with this study, patients with moderate to severe COPD (defined as Global Initiative for Chronic Obstructive Lung Disease (GOLD) severity grades II–IV) with at least one moderate or severe exacerbation in the previous year were considered. Following MACRO’s inclusion criteria, we also assumed that patients with corrected QT (QTc) interval prolongation are excluded from treatment, and so are patients with existing hearing impairment or concomitant asthma. Because the majority of MACRO patients were from the USA, for consistency, we prioritised US-based studies for evidence synthesis.

Model

We created a probabilistic discrete-time Markov model to simulate the natural history of COPD and the effectiveness of azithromycin and its side effects. The core states of the model were based on GOLD severity grades II–IV. These states were chosen because there are robust data on the rate of transition across GOLD grades over time and on their relationship with exacerbations.^{9 10} We further subdivided each state into ‘no long-term adverse effects’ and ‘long-term adverse effects’ to model the possible long-lasting side effects of azithromycin. The cycle length of the model was 1 year. A schematic illustration of the model is provided in figure 1. Input parameters are provided in table 1.

Figure 1.

Schematic illustration of the model. GOLD¹ defines COPD as the ratio of FEV1 to FVC of less than 0.7. The severity stages of the disease used in the model are defined as GOLD II (moderate COPD): 50%≤FEV1<80% of a predicted reference value for a healthy individual; GOLD III (severe COPD): 30%≤FEV1<50% predicted; and GOLD IV (very severe COPD): FEV1<30% predicted. In the states with long-term adverse events, the exacerbation rate for each GOLD grade was the same as that reported by Hoogendoorn et al. ¹⁰. Exacerbations tend to decrease in states without long-term adverse event because of the treatment effect. FEV1, forced expiratory volume in one second; FVC, forced vital capacity; GOLD, Global Initiative for Chronic Obstructive Lung Disease; COPD, chronic obstructive pulmonary disease.

Table 1.

Input parameters of the model per person-year

Parameter	Base value			Probability distribution			Source
	GOLD II	GOLD III	GOLD IV	GOLD II	GOLD III	GOLD IV
Initial distributions	0.26	0.4	0.34	NA	NA	NA	Albert et al 4
Baseline age		65			NA		Albert et al 4
Transition probability (95% CI)							Hoogendoorn et al 9
Ex-smokers	0.034 (0.033 to 0.035)	0.030 (0.028 to 0.031)	NA	Beta (5703, 162 046)	Beta (1372, 44 377)	NA
Current smokers	0.037 (0.036 to 0.038)	0.031 (0.029 to 0.032)	NA	Beta (6206, 161 543)	Beta (1418, 44 332)	NA
Exacerbation rate in the reference group (95% CI)							Hoogendoorn et al 10
Total exacerbations	1.17 (0.93 to 1.50)	1.61 (1.51 to 1.74)	2.10 (1.51 to 2.94)	Lognormal (0.16, 0.13)	Lognormal (0.48, 0.04)	Lognormal (0.74, 0.17)
Severe exacerbations	0.16 (0.07 to 0.33)	0.22 (0.20 to 0.23)	0.28 (0.14 to 0.63)	Lognormal (−1.83, 0.37)	Lognormal (−1.51, 0.02)	Lognormal (−1.27, 0.40)
Rate ratio of exacerbation (total and severe) by 12-month history pattern (95% CI)							ECLIPSE study32 (online supplemental appendix 4)
No exacerbations in the first year	Total: 0.16 (0.14 to 0.17)			Total: normal (0.160, 0.007)
	Severe: 0.16 (0.12 to 0.20)			Severe: normal (0.16, 0.02)
≥1 moderate/severe exacerbations in the first year	1 (Reference)			NA
≥2 moderate or ≥1 severe exacerbation in the first year	Total: 1.25 (1.16 to 1.33)			Normal (1.25, 0.04)
	Severe: 1.36 (1.17 to 1.58)			Normal (1.36, 0.10)
≥2 moderate/severe exacerbations in the first year	Total: 1.30 (1.20 to 1.40)			Normal (1.30, 0.05)
	Severe: 1.26 (1.07 to 1.48)			Normal (1.26, 0.10)
Relative risk of exacerbation in the treatment group compared with the control group (95% CI)
Total	0.73 (0.63 to 0.84)			Lognormal (−0.31, 0.07)			Albert et al 4
Ex-smokers	0.65 (0.55 to 0.77)			Lognormal (−0.43, 0.08)			Han et al 25
Current smokers	0.99 (0.71 to 1.38)			Lognormal (−0.01, 0.17)			Han et al 25
Decline in treatment effect	RR0 exp (– k * (year – 1)) RR0: first-year effect k=0.22				Fixed		Pomares et al 13 (online supplemental appendix 1)
Rate of mortality due to exacerbations (95% CI)	0.156 (0.109 to 0.203)			Beta (35.6, 192.4)			Hoogendoorn et al 11
Hazard ratio of cardiovascular death in the first 5 days (95% CI)	2.88 (1.79 to 4.63)			Lognormal (1.06, 0.24)			Ray et al 6
Relative risk of hearing loss due to azithromycin (95% CI)	1.168 (1.030 to 1.325)			Lognormal (0.15, 0.06)			Li et al 5
Annual incidence of hearing loss (95% CI)	0.023 (0.011 to 0.035)			Normal (0.023, 0.006)			Lin et al 17 (online supplemental appendix 2)
Prevalence of GI symptoms in the population	0.332 (0.326 to 0.333)			Beta (28 945, 58 768)			Almario et al 18
Relative risk of GI symptoms due to azithromycin (95% CI)	1.187 (0.761 to 1.849)			Lognormal (0.17, 0.22)			Li et al 5
Baseline utility (EQ-5D) (95% CI)	0.787 (0.771 to 0.802)	0.750 (0.731 to 0.768)	0.647 (0.598 to 0.695)	Beta (2251, 609.4)	Beta (1666, 555.5)	Beta (245.7, 134.1)	Rutten-van Mölken et al 33
Decrease in utility due to exacerbations (95% CI)							Sadatsafavi et al 34 (online supplemental appendix 3)
Mild and moderate	0.015 (0.002 to 0.040)	0.049 (0.020 to 0.090)	0.049 (0.020 to 0.090)	Beta (2.08, 132.50)	Beta (7.19, 140.30)	Beta (7.19, 140.30)
Severe and very severe	0.068 (0.035 to 0.110)	0.065 (0.030 to 0.100)	0.065 (0.030 to 0.100)	Beta (11.92, 162.60)	Beta (12.7, 182.1)	Beta (12.7, 182.1)
Decrease in utility due to hearing loss (95% CI)	0.187 (0.167 to 0.207)			Beta (6.92, 30.08)			NICE Guideline21
Utility improvement due to hearing aids (95% CI)	0.060 (0.044 to 0.073)			Normal (0.060, 0.006)			Barton et al 35
Decrease in utility due to the GI symptoms (95% CI)	0.026 (0.024 to 0.028)			Beta (66 343, 2 475 535)			Sullivan and Ghushchyan36 (online supplemental appendix 3)
Discount rate		3%			NA		Sanders et al 8

ECLIPSE, Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints; EQ-5D, EuroQol 5-Dimensional; GI, gastrointestinal; GOLD, Global Initiative for Chronic Obstructive Lung Disease; NA, not applicable; NICE, National Institute for Health and Care Excellence.

Natural history of COPD

The probabilities of transitioning from one GOLD grade to the next were based on the pooled analysis by Hoogendoorn et al.⁹ Progression in severity grades was assumed to be independent of the azithromycin treatment status because there is no evidence that azithromycin has a direct impact on lung function decline over time. Background mortality rates were taken from US life tables. Mortality directly related to exacerbations was exclusively modelled for severe and very severe events.¹¹

Exacerbations were modelled as events, with annual rates modelled as a function of the underlying GOLD grades.¹⁰ We chose the ‘event-based’ definition of exacerbations when extracting the related parameters from the literature and included moderate (requiring outpatient care or the initiation of antibiotics or systemic corticosteroids) and severe (requiring inpatient care) exacerbations.

Treatment effectiveness

Based on the MACRO study, a relative risk of 0.73 (0.63–0.84) was modelled for the effect of azithromycin.⁴ In a sensitivity analysis, the effect size reported by a Cochrane systematic review that included other macrolides was considered.¹²

Treatment adverse events

We performed a scoping literature review to find the adverse events of azithromycin that are prevalent or consequential enough to materially affect its net benefit and to estimate the frequency of these events. All studies that evaluated the safety of azithromycin (in patients with COPD or other conditions) were considered. Our review identified the following adverse events: antimicrobial resistance, CV toxicity, hearing impairment and GI symptoms.

For antimicrobial resistance, the vast majority of studies compared the colonisation rate of macrolide-resistant bacteria in azithromycin users with that in control subjects.¹² A Cochrane systematic review reported that due to the variations in methodologies, the authors could not pool data on the colonisation rate and stated that there was insufficient evidence to predict how colonisation rates would affect the resistance patterns in the community.¹² Most empirical studies of preventive azithromycin have been short in duration. The longest study to date showed that azithromycin remained effective for at least 2 years; however, there was a 34% decline in its efficacy by year 3.¹³ As such, in the base–case analysis, we modelled a gradual decline in the relative efficacy of azithromycin. We modelled a negative exponential decline in relative risk reduction for exacerbation that matched the observed decline in the third year of the aforementioned study (see online supplemental appendix 1).¹³ This assumption was subjected to threshold analysis.

The major concern regarding CV toxicity of azithromycin is QT-interval prolongation and its associated sudden cardiac death.^{6 14} A large US-based study demonstrated that in the first 5 days of exposure, the mortality risk increased by 288%,⁶ but the risk was undetectable after 6 days.¹⁵ In contrast, another study did not find an association between azithromycin and CV death.¹⁶ To account for these findings, in the base–case analysis, we modelled an increase in mortality (due to sudden cardiac arrhythmia) immediately after initiation of therapy but no change in the mortality risk afterwards. This is overall a conservative assumption (against azithromycin), given the exclusion of patients with prolonged QTc interval in our study. Alternative assumptions were evaluated in the sensitivity and threshold analyses.

The added risk of hearing loss due to azithromycin was modelled on a relative scale as reported by Li et al,⁵ applied to the background incidence of hearing loss taken from a population-based US study (see online supplemental appendix 2).¹⁷ We assumed that the occurrence of new hearing impairment would result in the discontinuation of treatment and require the patients to wear hearing aids for the rest of their lives.

GI symptoms are known to be one of the adverse events of antibiotics¹² and have been reported in almost all of the trials studying the effect of azithromycin. Vomiting, abdominal pain, diarrhoea and decreased appetite were the most frequently recorded GI symptoms in patients who used azithromycin.⁵ The background rate of GI symptoms in the general population was taken from a large (n=71 812) US study,¹⁸ and the additional risk due to the treatment was derived from a systematic review by Li et al on the adverse events of azithromycin.⁵ The GI symptoms were considered temporary and did not result in the discontinuation of treatment.

Health state utility values (utilities)

To calculate QALYs, each model state was assigned a utility value representing the average health-related quality of life of patients in that state. Short-term events (eg, exacerbations and GI symptoms) were modelled by an instantaneous drop in QALY. We used EuroQol 5-Dimensional (EQ-5D) utilities as the reference values,¹⁹ given the large body of evidence regarding the quality of life across GOLD grades and the effect of exacerbations on the quality of life of patients measured using this instrument. See online supplemental appendix 3 for more details about how utilities were modelled.

However, it is noted that EQ-5D lacks sufficient sensitivity to detect reductions in quality of life due to hearing loss.²⁰ Therefore, we followed the approach adopted by the National Institute for Health and Care Excellence in its health technology assessment of early versus delayed management of hearing loss in adults,²¹ and used the Health Utility Index (HUI)-3 instrument to capture changes in utility from hearing loss.²² In a sensitivity analysis, we used EQ-5D utilities instead of HUI-3 for impaired hearing.

Analysis

The analysis was fully probabilistic, incorporating uncertainty in the input parameters to estimate net benefits through Monte Carlo simulation. Probability distributions were assigned based on the type of the parameter and the level of uncertainty (eg, width of CI). The primary benefit–harm metric was the net QALY gain from the use of azithromycin. Azithromycin was deemed to be net beneficial if the (average) expected value of QALY gained was positive. The probability of a positive gain in QALY was also quantified. Because the QALY gain is a random variable due to uncertainty in evidence, in line with previous studies,²³ we further required the probability of obtaining a positive QALY gain to be above 60% for azithromycin to be considered net beneficial. We did not report p values because in benefit–harm analysis, it is the overall expected change in the outcome of interest, independent of the level of statistical significance, that is relevant.²⁴

Subgroup analysis

We investigated the benefit–harm of azithromycin among patients with different histories of exacerbations. We used individual-level data from the Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints (ECLIPSE) cohort to calculate rate multipliers for each unique pattern of exacerbation in the previous 12 months (see online supplemental appendix 4). Furthermore, the potential benefit of treatment stratified by smoking status was explored using different effect sizes for former and current smokers, based on a subgroup analysis of the MACRO study.²⁵

Threshold and sensitivity analyses

We performed separate threshold analyses to identify the threshold for each parameter at which the expected net benefit crossed zero. In addition, sensitivity analyses were conducted on CV death risk, treatment effect size, disutility due to hearing loss, discount rates, time horizon and adherence rates. Regarding adherence, given that parameters were derived from studies that used intention-to-treat analysis, our analysis incorporated loss in adherence over the treatment period. However, given that adherence to treatment in the community is generally lower than that in clinical studies, we also modelled the effect of permanent discontinuation of treatment (above and beyond treatment discontinuation in underlying RCTs) in a sensitivity analysis (online supplemental appendix 5).

All analyses were performed in R V.4.0.1. The open-source model and the analytical code are available online (https://github.com/safaahmadian/AZT_HarmBenefitAnalysis).

Results

Table 2 provides the expected value of outcomes over 20 years for the main analysis, standardised to a cohort of 100 patients to facilitate interpretations. Among the primary target population (patients with COPD with GOLD grade ≥II with at least one moderate/severe exacerbation in the past 12 months), the treatment increased their QALYs by 17.9 per 100 patients over the given time horizon. The probability of positive expected QALY gain was 99.8%.

Table 2.

Outcomes of the probabilistic analysis over 20 years for 100 patients with COPD

	No azithromycin (95% CI)	Azithromycin (95% CI)	Difference (95% CI)
Total exacerbations (n)	2047 (1635 to 2567)	1931 (1545 to 2435)	−116 (−176 to −63)
Severe or very severe exacerbations (n)	280 (200 to 397)	265 (190 to 375)	−15 (−25 to −8)
Cumulative incidence of hearing loss (n)	24 (13 to 34)	28 (15 to 40)	4 (1 to 7)
Average episodes of GI symptoms (n)	19 (17 to 21)	23 (15 to 33)	4 (−3 to 13)
Mortality due to exacerbation (n)	40 (28 to 54)	38 (26 to 52)	−2 (−3 to −1)
Other mortality (Including CV death) (n)	34 (28 to 39)	35 (29 to 40)	1 (0 to 2)
Life years	1170 (1011 to 1303)	1204 (1053 to 1328)	34 (18 to 51)
Total QALYs	555.5 (482.7 to 624.4)	573.5 (504.7 to 637.8)	17.9 (6.2 to 30.0)

Results are reported for 100 patients.

GI, gastrointestinal; QALY, quality-adjusted life year.

Subgroup analysis

The use of azithromycin in current smokers was associated with a net QALY loss of 4.3 per 100 patients over 20 years, and azithromycin was only 36.8% likely to be net beneficial. On the other hand, in ex-smokers, the expected net QALY gain was 25.3 per 100 patients over 20 years, and the probability of positive QALY gain was 99.9%.

Results of subgroup analysis by exacerbation history are provided in figure 2. Among patients with a negative exacerbation history in the previous 12 months, azithromycin was associated with an expected net QALY loss of 0.4 per 100 patients and a 45.7% chance of being net-beneficial, which did not meet the prespecified >60% probability criterion. Conversely, patients who had two or more moderate or at least one severe exacerbation in the previous year (the GOLD definition of a frequent exacerbator) derived the most benefit from azithromycin (expected net QALY of 21.8 per 100 patients, probability positive benefit=99.9%).

Figure 2.

Probability distribution of net QALY gain according to patients’ exacerbation history in the past 12 months. (A) Patients with no exacerbation; (B) patients with ≥1 moderate/severe exacerbations (the reference population); (C) patients with ≥2 moderate/severe exacerbations; (D) patients with ≥2 moderate or ≥1 severe exacerbations in the previous 12 months. QALY, quality-adjusted life years.

Threshold and sensitivity analyses

Figure 3 provides the results of the threshold analyses. In all figures, the solid line represents the threshold value at which the expected net benefit crosses zero. The value used in the base-case analysis is also highlighted by the dashed line for comparison.

Figure 3.

Results of threshold analyses. Each point on each graph is based on running the model for 1000 simulations.^† The relative risk of exacerbation in the treatment group shows the treatment effect, ^‡ The higher values of k in the equation, the faster the decline in the effectiveness of azithromycin over time. CV, cardiovascular; GI, gastrointestinal; GOLD, Global Initiative for Chronic Obstructive Lung Disease; QALY, quality-adjusted life years.

Preventive azithromycin was net beneficial as long as it was associated with a risk reduction of 5% or more for exacerbations (figure 3A). Azithromycin remained net beneficial even if exacerbations were assumed not to cause a reduction in quality of life (figure 3B) or mortality (figure 3C), reflecting respectively, the survival benefit and gain in quality of life of preventing exacerbations. The net benefit of azithromycin approached zero only when the decline in the effectiveness of azithromycin was so high that azithromycin was only effective in the first year of treatment (figure 3D). Conversely, the net benefit was almost two times higher than the base-case value if the treatment effect did not wane over 20 years. Similarly, the assumption that the excess CV death risk persisted for 1 year did not change the direction of the expected net benefit (figure 3E). The relative risk of hearing loss and GI symptoms should be 1.9 and 3.5 times higher than the base-case value, respectively, to result in a zero net benefit (figure 3F, H). Likewise, the reduction in quality of life due to hearing loss and GI symptoms should be implausibly high (0.87/year and 0.27/year, respectively) to nullify the net benefit gain (figure 3G, I).

Results of the sensitivity analyses are provided in the online supplemental appendix and show that the overall findings were robust to changes in several key assumptions in the analyses (no increased CV death risk, alternative source for treatment effect, smaller disutility due to hearing loss, discount rates of 0% and 5%, time horizon of 1 and 35 years, and explicit modelling of non-adherence to azithromycin).

Discussion

We demonstrated that in patients with GOLD grade II or higher who have a positive exacerbation history despite using maximal inhaled therapies, show a normal QTc interval on baseline ECG and do not have comorbid asthma or existing hearing impairment, the addition of azithromycin is net beneficial. The probability of azithromycin conferring net benefit in such patients exceeded 90%. The expected gain in QALY (0.179) is within the range of QALY gain estimated for inhaled therapies from economic evaluations of such therapies (eg, 0.137 for triple vs double therapy and 0.131 between different dual therapies).^{26 27} Our subgroup analyses showed that the benefit is mostly concentrated in patients who are not current smokers and who have had a positive exacerbation history in the previous 12 months. The GOLD management strategy,¹ the joint European Respiratory Society / American Thoracic Society (ATS/ERS),²⁸ and the statement by the American College of Chest Physicians/Canadian Thoracic Society²⁹ all recommend the use of azithromycin in patients who continue to exacerbate despite optimal inhaled therapies, but have also expressed concern about the adverse effects of maintenance azithromycin therapy. Our quantitative benefit–harm analysis provides further support for this recommendation, and our sensitivity and threshold analyses should lessen the concerns about the long-term balance of harms and benefits of this therapy. Importantly, given the high likelihood of net benefit of azithromycin in patients with only one moderate/severe exacerbation in the previous year (who also meet all the other inclusion criteria previously mentioned), our results support extending the eligibility criteria of ‘frequent exacerbators’ to include patients with any history of moderate/severe exacerbation in the previous year. It is noted that our results represent the benefit–harm balance of maintenance azithromycin therapy as evaluated in landmark clinical trials (eg, MACRO) and as recommended by contemporary disease management strategies. However, the manner in which azithromycin is prescribed in the ‘real world’ (eg, target population and adherence levels) might be different from guideline recommendations, and the actual net benefit of azithromycin can be affected by such departures from recommended usage.

Multiple sensitivity and threshold analyses provided assurances that our results were robust to a range of variations in the benefits and harms of azithromycin, including declining benefits of azithromycin on exacerbation risk over time, its potential to induce adverse events, and its impact on quality of life. Of particular concern among experts has been the durability of the net benefits of azithromycin over time, given the risk of antimicrobial resistance with its long-term use.⁵ Some evidence suggests that treatment effect declines modestly over 3 years,¹³ but to the best of our knowledge, empirical evidence beyond this period does not exist. Importantly, our results were robust to the assumption of diminishing treatment effects. These important findings indicate that concerns about the potentially diminishing effectiveness of azithromycin should not preclude its use in patients who still exacerbate despite optimal inhaled therapy.

To the best of our knowledge, this is the first quantitative benefit–harm analysis of maintenance azithromycin in COPD. Despite the relevance of quantitative benefit–harm analysis for evidence-inform decision-making, we are aware of only one benefit–harm analysis in the context of COPD that evaluated roflumilast.³⁰ Similar to our study, the authors collated data from multiple sources and combined multiple aspects of treatment using weights that reflected their clinical burden to derive a scalar net benefit index. They found that roflumilast was generally net harmful; the probability of it being net beneficial was >60% only among patients whose baseline risk of severe exacerbations was >22%/year. A difference between this study and ours is that the formerly assigned weights to different outcomes were based on expert opinion, whereas we used preference-based weights that gave rise to QALYs.

Our study has several strengths. The use of a probabilistic model enabled us to combine evidence from disparate sources to calculate a single index for the net benefit and to extrapolate the results to a sufficiently long time-horizon. By using health utility values as the ‘currency’ for quantifying harm and benefit, we were able to combine various aspects of benefit and harms and to generate a singular estimate of net benefit. Access to individual-level data from ECLIPSE enabled us to provide estimates of net benefit for subgroups based on their exacerbation history. By properly incorporating uncertainty in the evidence, we were able to make probabilistic statements about the likelihood of benefit, which showed that the evidence, while uncertain, points towards a high likelihood of net benefit in patients with a positive exacerbation history. This was further confirmed through multiple threshold and sensitivity analyses.

The limitations of this study should also be acknowledged. Given that in practice, azithromycin may be given for many years, the short-term follow-up time of RCTs provided a truncated picture of the overall benefits (or harms) of this treatment. As such, we had to extrapolate beyond the follow-up time of the longest empirical studies, which requires assumptions about the durability of benefits and harms. However, our sensitivity and threshold analyses showed that the conclusions do not change across a wide range of assumptions on the long-term trajectories of outcomes; in particular, our results remained robust against assumptions on the waning effect of azithromycin and the duration of heightened CV death risk. On the other hand, the effect of antimicrobial resistance in the community was not considered in this study and requires further investigation. We used utility weights from different sources as there was not a single study that consistently provided all the values required for our analysis, although we tried to reduce heterogeneity by prioritising values from the USA. Still, differences in the utility values for different model parameters might be affected by the differences in study design. Lastly, azithromycin is not the only choice of therapy in patients who exacerbate while on maximal inhaled therapy. However, because of an insufficient amount of direct evidence, we could not include other treatments (eg, roflumilast) in this analysis.

The model projected that azithromycin is not net beneficial in current smokers. The major source of evidence for the effect modification of smoking is a post hoc analysis of MACRO,²⁵ which may have been underpowered for this subgroup analysis (MACRO’s sample size was based on identifying the main effect) and susceptible to a chance finding from multiple hypothesis testing. Observational studies have suggested that azithromycin is effective in reducing exacerbations in both current and ex-smokers, though the benefits were smaller in current than ex-smokers.³¹ We were unable to use these data because of the possibility of confounding and bias, and in general, we did not find a properly designed quasi-experimental study that sufficiently adjusted for potential confounding variables. Studying the effectiveness of azithromycin in current smokers thus should be prioritised to generate high quality evidence to revisit this question. Further, we did not model the potential long-lasting effect of exacerbations (eg, drop in lung function), as we consider evidence on this aspect to still be controversial. However, we note that modelling such indirect treatment effect would lead to even greater benefits of the treatment and make our results even more favourable.

In summary, this study shows that azithromycin is very likely to be net beneficial in patients with COPD with a positive history of exacerbations. While the ATS/ERS statement has made a conditional statement on the use of azithromycin in this patient subgroup and considered the quality of evidence to be low, our results suggest that this benefit is robust to many alternative assumptions. Future research should evaluate the effectiveness of azithromycin in current smokers. Quantitative benefit–harm analysis should also be considered for many other interventions in COPD for which the benefits and harms are only subjectively considered by expert panels.

Data availability statement

The computer code generating all the results presented in this work is available from https://github.com/safaahmadian/AZT_HarmBenefitAnalysis. One section of the code (the analysis of ECLIPSE data) is based on data that were obtained under license and the data cannot be shared.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Ethics approval was obtained from The University of British Columbia’s Human Ethics Board (H15-02037).