The Effect of Mass Azithromycin Distribution on Childhood Mortality: Beliefs and Estimates of Efficacy

Abstract

A cluster-randomized trial demonstrated that mass oral azithromycin distribution reduced childhood mortality 49.6% (Trachoma Amelioration in Northern Amhara [TANA]). The relative risk of childhood mortality was then estimated using two approaches: an expert survey and a Bayesian analysis. The survey asked public health experts to estimate the true effect of mass azithromycin distribution on childhood mortality. The Bayesian estimation used the TANA study's results and prior estimates of the efficacy of other effective population-level interventions. The experts believed mass azithromycin reduces childhood mortality (relative risk = 0.83, 95% credible intervals [CrI] = 0.70–1.00). The Bayesian analysis estimated a relative risk of 0.71 (95% CrI = 0.39–0.93). Both estimates suggest that azithromycin may have a true mortality benefit, though of a smaller magnitude than found in the single available trial. Prior information about nonantibiotic, population-level interventions may have informed the expert's opinions. Additional trials are needed to confirm a mortality benefit from mass azithromycin.

Introduction

In 2013, nearly 6.3 million children died before the age of five worldwide.¹ The overwhelming majority of these deaths occur in developing countries, with sub-Saharan Africa accounting for nearly 50% of under five deaths.¹ The Trachoma Amelioration in Northern Amhara (TANA) study found that distribution of azithromycin to entire communities to treat and prevent blinding trachoma significantly reduced childhood mortality.^2,3 Some degree of mortality reduction is plausible, since the antibiotic has activity against the three most common causes of childhood mortality in the developing world⁴: malaria,^5,6 diarrhea,⁷ and respiratory infections.^6,8 Several other population-level interventions, such as vaccination,⁹ nutrient supplementation,¹⁰ and exposure limitation have been shown to reduce childhood mortality in the developing world, but with more modest reductions.

The Bill & Melinda Gates Foundation hosted a meeting in 2011 to discuss azithromycin as a means to reduce childhood mortality in sub-Saharan Africa and other parts of the developing world. Attendees were internationally recognized experts in epidemiology, biostatistics, malaria, respiratory diseases, diarrhea, demography, vitamin A, and trachoma. At the meeting, we distributed a survey to those present on the efficacy of azithromycin mass drug administration (MDA) for reducing childhood mortality and possible mechanisms. Separately, we assessed the efficacy of other population-level interventions to use as a prior in a Bayesian analysis of the TANA study.

Materials and Methods

On April 11, 2011, we administered a paper questionnaire (Supplemental Appendix 1) to those in attendance at a Bill & Melinda Gates Foundation-sponsored meeting entitled “Mortality Reduction after Oral Azithromycin: Study Design Convening.” The anonymous questionnaire addressed two issues: beliefs on the effect size of azithromycin on childhood mortality and the mechanism of such an effect were it to exist. Experts were presented the following scenario: children aged 6–60 months in communities in rural sub-Saharan Africa were provided with 3 years of biannual azithromycin treatment or placebo. A range of possible outcomes was provided, and respondents were instructed to choose the most likely effect on mortality along with the most extreme effects they would consider plausible (specifically, the bounds beyond which they would expect only a 5% chance of finding the true effect size). The input from each individual was then fit to a skew-normal distribution,¹¹ and individual distributions were pooled by taking the arithmetic mean with equal weight given to each response. For simplicity, the pooled distribution will be termed the survey distribution. To evaluate the plausibility of proposed mechanisms for any presumed effect, we provided a list of five possible causes of mortality: malaria, respiratory disease, diarrhea, noninfectious causes, or other causes. Respondents ranked the importance from 1 (most likely) to 5 (least likely). Responses were evaluated by Wilcoxon signed-rank test. The University of California, San Francisco Committee on Human Research certified the survey as exempt from Institutional Review Board review.

In an effort to assess the range of efficacy seen with other mass interventions, searches of the Cochrane database were performed for combinations of “child,” “children,” or “childhood” with “death” or “mortality.” To be included, a Cochrane Review needed to describe a community-wide intervention that was provided without regard for signs or symptoms and not associated with childbirth, and contained at least five trials with childhood mortality data. For every qualifying Cochrane Review, we converted the estimated childhood mortality results of each study into a log odds ratio and standard error. Using these parameters, we created an arithmetic mean of all studies, weighted by the inverse of the study variance. For simplicity, the resulting distribution will be termed the empirical prior.

We used the empirical prior and the likelihood from the negative binomial regression from the TANA study to estimate a Bayesian posterior.² Probabilities of benefit were determined by calculating the area corresponding to a relative risk < 1 for each of the figures. We interpreted all distributions as expressing degrees of belief with appropriate centralized 95% credible intervals (CrI).¹² All analyses were performed using Mathematica 7.0 (Wolfram Research, Champaign, IL).

Results

Twenty-eight of 29 (97%) experts present at the Bill & Melinda Gates Foundation meeting fully completed and returned the questionnaire. The survey distribution had a mean relative risk of childhood mortality with biannual azithromycin of 0.83 (95% CrI = 0.70–1.00, Figure 1A). The distribution suggests that experts believed that the probability that azithromycin reduced childhood mortality was 94%.

Figure 1.

Comparison of estimates of the relative risk of childhood mortality. In A–D, the x-axis represents the relative risk of childhood mortality. The y-axis is a unit-less density measure. A depicts the survey distribution, a probability density function created by pooling 28 skew-normal distributions fit to the questionnaire responses. The pooled function (light gray, dashed outline) is shown along with scaled-down representations of individual responses (darker gray, grey outline). B shows the Bayesian empirical prior distribution, which was calculated from meta-analysis of Cochrane reviews. The pooled function (dark gray, black outline) is shown along with scaled-down representations of the seven interventions (gray, gray outline). C shows the Bayesian analysis, which contains the empirical distribution as the prior probability (dark gray, black outline), the results of the Trachoma Amelioration in northern Amhara (TANA) study as the likelihood (light gray, dashed outline), and the resulting posterior probability (gray, dashed outline). D compares the survey distribution (light gray, dashed outline) to the Bayesian posterior (gray, dashed outline).

The average rankings for the mechanisms of mortality reduction, in order of importance, were 1.64 for respiratory infection, 2.00 for diarrhea, 2.97 for malaria, 3.96 for other infectious, and 4.65 for noninfectious. All pairwise comparisons differed significantly (P < 0.05) except for respiratory infection and diarrhea (P = 0.19).

The initial screening of the Cochrane Library yielded 237 unique reviews (accessed June 6, 2011). On inspection, 22 of these reported childhood mortality and were population-based mass treatment interventions. Of those, seven contained mortality data from at least five studies and were included. The interventions were oral iron supplementation for anemia,¹³ rotavirus vaccination,¹⁴ vitamin A supplementation,¹⁰ chemoprophylaxis for malaria,¹⁵ injected cholera vaccination,¹⁶ insecticide-treated bed nets,¹⁷ and pneumococcal conjugate vaccines.¹⁸

Results from the meta-analyses are shown in Figure 1B. The empirical prior has a mean of 0.86 and a standard deviation of 0.22. From this prior distribution, the probability that a population-based intervention had a benefit for childhood mortality was 85%.

The Bayesian analysis is depicted in Figure 1C. The resulting posterior distribution has a mean relative risk of 0.71 (95% CrI = 0.39–0.93). The posterior distribution suggests that the probability that azithromycin lowers childhood mortality is 99.2%. Figure 1D provides a visual comparison of the estimates from the survey distribution to the Bayesian posterior.

Discussion

Here we present the results from a survey of a group of experts regarding the efficacy of azithromycin to reduce childhood mortality in sub-Saharan Africa. This investigation was prompted by the results of the TANA clinical trial for trachoma in Ethiopia, which found a large and significant mortality reduction with azithromycin compared with a group receiving no treatment (relative risk = 0.496).² The results of the survey, performed after the completion of TANA, suggest that experts believe azithromycin has a true mortality benefit. However, they believe that this benefit is more modest (mean relative risk = 0.83) than the results of the TANA trial (relative risk = 0.496). Experts believe that if there were truly a reduction in mortality, the mechanism would most likely be from a reduction in respiratory infections or diarrhea; this is consistent with a previous report.¹⁹

There are several explanations for the discrepancy between the trial outcome and expert belief. TANA used no placebo, the sample size was relatively small (82 total deaths out of 18,415 total children aged 1–9), and the resulting confidence intervals were wide.² Small trials with significant results tend to overestimate the effect size, for the simple reason that they could have only achieved significance with a large effect. The observed mortality reduction may be affected by some combination of systematic and random errors; experts may have taken these factors into account. In addition, the meta-analysis showed that population-level interventions have typically produced more modest mortality reductions. Experts may be unknowingly performing an informal Bayesian-type of analysis using prior beliefs similar to the empirical distribution presented here. In fact, more than one survey respondent commented that they thought that azithromycin could certainly have an effect on mortality, but they did not believe the magnitude of the effect reported in the TANA study since other mass interventions had not been shown to be nearly that effective.

For this Bayesian analysis, an empirical prior probability was created using indirect evidence. When used appropriately, indirect evidence can increase the accuracy of estimates, particularly when direct evidence is rare and indirect evidence is plentiful.²⁰ Here, we estimated the prior probability of the effect on childhood mortality of a population-level intervention, which is thought to work at the individual level. The resulting Bayesian analysis suggested that, given both the indirect evidence and the results from Porco and others, the relative risk of childhood mortality in communities given mass azithromycin is 0.71, with a peak density at 0.80. Similar to the survey results, this suggests a definite mortality benefit from mass oral azithromycin distribution, but of a lesser magnitude than the outcome of TANA.

Even if future studies increase confidence that mass distribution of azithromycin provides a mortality benefit for children, the interventions will remain controversial in the absence of endemic trachoma. Some members of the medical community might be comfortable distributing azithromycin if the mechanism for the mortality reduction was clear. Others might consider side effects, costs, and the impact of antibiotic resistance to outweigh any benefit.²¹ Several recent trachoma studies investigated childhood mortality as a secondary outcome (Partnership for Rapid Elimination of Trachoma [PRET] and Tripartite International Research for the Elimination of Trachoma [TIRET])^22,23; however, these studies are neither designed nor adequately powered to detect anything but a very large treatment effect. The analyses presented here suggest the need for a large randomized controlled trial to definitively resolve the question of the effect of azithromycin on mortality. Mortality Reduction after Oral Azithromycin (MORDOR, Bill & Melinda Gates Foundation, no. OP1032340) is a placebo-controlled trial in Niger, Malawi, and Tanzania designed to detect a 10% reduction in mortality because of mass azithromycin distribution, which is in line with what experts anticipate and with what other population-level interventions have proven to offer based on these analyses. If proven effective as an intervention, then further work would be necessary to assess efficacy when integrated with other community programs such as distributions of vitamin A or albendazole, or vaccinations for Streptococcus pneumonia, Haemophilus influenza, and/or rotavirus. Although it would be unprecedented to treat an entire population with systemic antibiotics to reduce mortality, that should not dissuade further study.