Bayesian meta‐analysis of phase 3 results of aducanumab, lecanemab, donanemab, and high‐dose gantenerumab in prodromal and mild Alzheimer's disease

Abstract

INTRODUCTION

Phase 3 trials using the anti‐amyloid antibodies aducanumab, lecanemab, donanemab, and high‐dose gantenerumab in prodromal and mild Alzheimer's disease dementia were heterogeneous in respect to statistical significance of effects. However, heterogeneity of results has not yet directly be quantified.

METHODS

We used Bayesian random effects meta‐analysis to quantify evidence for or against a treatment effect, and assessed the size of the effect and its heterogeneity. Data were extracted from published studies where available and Web based data reports, assuming a Gaussian data generation process.

RESULTS

We found moderate evidence in favor of a treatment effect (Bayes factor = 13.2). The effect was moderate to small with −0.33 (95% credible interval −0.54 to −0.10) points on the Clinical Dementia Rating – Sum of Boxes (CDR‐SB) scale. The heterogeneity parameter was low to moderate with 0.21 (0.04 to 0.45) CDR‐SB points.

DISCUSSION

Heterogeneity across studies was moderate despite some trials reaching statistical significance, while others did not. This suggests that the negative aducanumab and gantenerumab trials are in full agreement with the expected effect sizes.

1. INTRODUCTION

The field of disease‐modifying treatment of Alzheimer's disease (AD) has received increased attention after the United States Food and Drug Administration (FDA) granted accelerated approval to the anti‐amloyid antibodies aducanumab and lecanemab, in June 2021 and January 2023, respectively, and traditional approval to lecanemab in July 2023. However, these new generation anti‐amyloid antibodies achieved seemingly conflicting statistical evidence regarding their treatment effects in their recently completed phase 3 trials. Of aducanumab's phase 3 trials, one was positive and one was negative ¹ ; lecanemab's single phase 3 trial was positive, ² and high‐dose gantenerumab's trials were both negative (as reported at the CTAD conference in November 2022), and donanemab's phase 3 trial was positive. ³ This heterogeneity of evidence has been attributed to a range of factors, among them different binding properties of the antibodies, ⁴ differential targeting of amyloid clearance mechanisms, including phagocytosis, disruption of Aβ aggregation, and peripheral sink, ⁵ , ⁶ different routes of administration, and heterogeneous study populations, some of them potentially including rapid progressors¹. However, this heterogeneity of evidence is determined by the fact that “significant” and “non‐significant” p‐values are frequently considered to be at odds with one another, thus constituting conflicting evidence. This view limits the scope of scientific discussion to a mutually exclusive binary. For decades, many authors have pointed out misconceptions of the p‐value that lead to erroneous conclusions, nicely summarized in ref. 7. One misconception is that “Studies with p‐values on opposite sides of .05 are conflicting”, ⁷ page 137. It would be more conducive to scientific discussion if evidence were conceptualized as a continuum reaching from strong support against the hypothesis under consideration to strong support for the hypothesis under consideration. The Bayesian framework allows direct estimation of evidence in favor or against the null and the alternative hypothesis and can overcome the fallacy of assuming different outcomes in presence of meaningless differences in p‐values on opposite sides of 0.05.

Bayesian random effects meta‐analysis allows assessing both the mean effect across studies and between study heterogeneity with the posterior distribution providing an estimate of the most likely values of the mean and heterogeneity parameters given the data. Additionally, the Bayesian framework allows direct quantification of evidence in favor of or against an effect on a continuous basis. ⁸

We derived the recently published effect size estimates from the phase 3 data of the anti‐amyloid antibodies aducanumab, lecanemab, gantenerumab, and donanemab, and applied Bayesian random‐effect meta‐analysis with three aims: first, to assess whether there was an overall positive effect; second, establish this effect's strength and heterogeneity across studies; and third, the likely range of the effect in a potential future study with these or similar antibodies. From an immunological perspective, the decision to only include gantenerumab, aducanumab, lecanemab, and donanemab was based on previous studies showing a distinct binding profile of these so called “new generation” antibodies compared with first generation antibodies, such as solanezumab, bapineuzumab, and crenezumab. ⁴ , ⁹ , ¹⁰ , ¹¹ In addition, understanding the effect sizes, both clinically and statistically, for the newly FDA approved drugs and the strength and heterogeneity of the clinical data are questions that arise often from clinicians.

RESEARCH IN CONTEXT

1. Systematic review: We reviewed the literature using PubMed, meeting abstracts, and presentations. Several meta‐analyses have examined the efficacy of anti‐amyloid antibodies in Alzheimer's disease (AD), but few directly quantified the evidence for or against an effect and the heterogeneity between studies.

2. Interpretation: The results suggest a small overall effect for the new generation antibodies. Heterogeneity between studies was low, although results were on different sides of a significant p‐value. Thus, the negative studies are also in full agreement with the expected effect sizes.

3. Future directions: The findings suggest a prioritization of future lines of research, including definition of patient‐relevant outcomes, lowering burden of treatment, and testing combination treatments targeting several pathophysiological domains of AD. Our findings also indicate that use of appropriate statistical approaches in a Bayesian framework provide quantitative evidence for or against an effect and direct estimates of the heterogeneity of results given the data.

2. METHODS

2.1. Data generation

Case numbers, mean rates of CDR decline, and corresponding standard errors (SE) for placebo and treatment, respectively, were derived from:

1. tab. 2 in ref. 1 for aducanumab. The standard error was given in ref. 1 for the placebo groups, but not for the other groups. Therefore, the SE were derived from tab. 9 (p. 177) for the EMERGE study, and tab. 15 (p. 190) for the ENGAGE study in the summary statistics of Biogen's aducanumab trial, available here: https://www.fda.gov/media/143502/download.

2. fig. 2 in ref. 2 for lecanemab.

3. The Figure on slide 20 of the presentation of the gantenerumab topline results on CTAD 2022.

4. tab. 2 of ref. 3 for donanemab. Of note, the donanemab results had not been available at the original submission, but have been added during the review process.

Based on the derived SE and the number of cases (n), we determined the corresponding standard deviations (SD) of the sampling distribution according to:

$$SD=SE\ast \sqrt{n}.$$

This model assumes a Gaussian data generation process.

Results for aducanumab were published separately for the high‐dose and the low‐dose groups in both the peer‐reviewed paper ¹ and the FDA files, in agreement with the primary analysis plan. Consequently, we treated groups as separate studies, since we did not have access to the primary data to pool effects across doses. For our endpoint, we focused exclusively on the Clinical Dementia Rating – Sum of Boxes (CDR‐SB), ¹² as this measure was the primary endpoint for antibodies aducanumab, lecanemab, and gantenerumab and a pre‐specified secondary endpoint for donanemab.

Further details on the data generation and tests of its consistency are reported in the Supplementary Section.

2.1.1. Descriptive Statistics

Table 1 shows estimated mean values, SE, SD, and number of cases.

TABLE 1.

Extracted study data

Study	ΔCDR a placebo	ΔCDR a treatment	SE b (ΔCDR placebo)	SE b (ΔCDR treatment)	N c placebo	N c treatment
EMhd	1.74	1.35	0.12	0.12	548	547
EMld	1.74	1.47	0.12	0.12	548	543
ENGhd	1.56	1.59	0.11	0.11	545	555
ENGld	1.56	1.38	0.11	0.11	545	547
Clar	1.66	1.21	0.07	0.07	875	859
GRAD‐I	3.67	3.36	0.29	0.29	485	499
GRAD‐II	3.01	2.82	0.28	0.28	477	498
TRAIL‐2	2.33	1.66	0.09	0.09	794	838

Abbreviations: CDR‐SB, Clinical Dementia Rating – Sum of Boxes.

^a ΔCDR is change of CDR‐SB from baseline in the ITT populations based on linear mixed effect models.

^b SE is standard error for mean change of CDR‐SB from baseline in the ITT populations based on linear mixed effect models.

^c n is number of cases for the ITT populations.

2.2. Statistical analysis

The three different aims outlined above were addressed by three separate analyses:

First, we quantified evidence in favor of the hypothesis that an anti‐amyloid antibody treatment results in negative changes of the CDR‐SB score by calculating the Bayes factor BF₁₀ using Bayesian random‐effects meta‐analysis ¹³ across the eight studies (counting aducanumab high dose and low dose as separate studies) in the software JASP 0.16.4 with a weakly informed normally distributed prior with mean 0 and standard deviation of 1. Here, a BF₁₀ >3 is considered to provide moderate evidence in favor of the alternative hypothesis of a treatment effect compared to the null hypothesis.

Second, after establishing evidence of a treatment effect, we used a Bayesian random‐effects meta‐analysis with the package “bayesmeta” in R in ref. 14 to determine estimates for the effects of treatment on rates of CDR‐SB change. This analysis provided us with posterior effects and 95% credible intervals for the following three measures:

1. The mean treatment effect size (score points of the CDR‐SB);

2. The prediction value, indicating the expected effect for a new study using the same or similar antibodies;

3. The heterogeneity parameter tau (τ), quantifying the amount of between‐study variation that is, not explained by the fixed effect. The parameter τ equals the variance of the random effect minus the within‐study sampling variance so that if τ increases the variance of the random effect also increases, which means that the random effect estimate becomes less precise.

We chose a weakly informed normally distributed prior with mean 0 and SD of 1 for the treatment effect size. This prior allows for conflicting evidence. The prior for the heterogeneity parameter τ was chosen as a weakly informed half‐normal distribution with a standard deviation of 0.5, consistent with a previous systematic review of heterogeneity estimates across a large range of meta‐analyses from the Cochrane library. ¹⁵ In a sensitivity analysis, we compared effects when using a non‐informative Jeffreys prior for heterogeneity, and a value of 4 for the prior of the standard deviation of the effect estimate.

Third, to account for imprecision in deriving SE and subsequent SD estimates from the graphs, we conducted a sensitivity analysis in which we repeated the analyses with 0.5 times the SD added to the SD estimates.

As an additional analysis, we conducted a meta‐regression analysis of the effect of change in the positron emission tomography (PET) centiloid values by treatment on differences in CDR‐SB scores as described in ref. 16.

3. RESULTS

The Bayesian random‐effects meta‐analysis yielded a BF₁₀ of 13.1, providing evidence in favor of a treatment effect across studies. A sequential analysis of the posterior model probabilities (Figure 1) suggests that with study eight, that is, TRAILBLAZER‐ALZ2, the posterior probability becomes dominant for the random effect model, that is, non‐zero mean effect and larger than zero heterogeneity. The posterior model probability provided by sequential analyses must always be consistent with every single individual study incorporated into the sequential analysis. Our sequential plot of the Bayes factor (Figure S1) hints at moderate heterogeneity: study 1, the high‐dose EMERGE trial, provides no evidence in either direction yet, the low‐dose EMERGE and the Clarity trial bring the evidence close to moderately in favor of the treatment effect, the GRADUATE and TRAILBLAZER‐ALZ2 studies tilt the evidence firmly in favor of moderate to strong evidence. The only non‐supportive evidence – high‐dose ENGAGE – does not reach convincing levels of evidence against the treatment effect, suggesting that its evidence is not in serious conflict with the remaining studies.

FIGURE 1.

Sequential plot of the posterior model probabilities, the posterior probability for each of four hypotheses is displayed as a function of the number of studies included in the analysis. The hypothesis are: Fixed H0, the fixed‐effect null hypothesis; mean effect = 0, heterogeneity = 0; Fixed H1, the fixed‐effect alternative hypothesis; mean effect ≠ 0, heterogeneity = 0; Random H0, the random‐effect null hypothesis; mean effect = 0, heterogeneity > 0; Random H1, the random‐effect alternative hypothesis; mean effect ≠ 0, heterogeneity > 0, As evidence accumulates from studies, the posterior probability is dominant for the random effect alternative hypothesis, indicating non‐zero mean effect and larger than zero heterogeneity

The forest plot across the eight studies is shown in Figure 2. The high‐dose aducanumab EMERGE, lecanemab and donanemab trials provided evidence in favor of the expected treatment effect, and the mean of the posterior distribution of effects across all eight studies favored treatment with a small effect of −0.33 (95% credible interval −0.54 to −0.10) CDR‐SB points. Notably, its credible interval did not include zero, despite gantenerumab's widely heterogeneous effects. The 95% credible interval surrounding the predicted effect obtained in a potential future study with a similar antibody ranged from −0.88 to 0.25 points on the CDR‐SB scale, indicating that treatment effect sizes of 0 and greater than 0 are both plausible in future clinical trials. A treatment effect size of 0 would mean that the treatment is ineffective without causing harm to the participants, while an effect size greater than 0 would indicate that the treatment speeds up the course of AD. The heterogeneity parameter τ of 0.21 CDR‐SB points (95% credible interval 0.04 to 0.45) is slightly higher than half the size of the mean effect (0.33 CDR‐SB), suggesting that heterogeneity was moderate. When looking at the joint distribution of the effect μ and the heterogeneity parameter τ in Figure 3, one can see that the effect estimates exclude zero even at relatively high values of τ.

FIGURE 2.

Forest plot, The forest plot features the direct and indirect estimates of treatment effects across the cohorts. The direct or quoted estimates are the parameter estimates based only on the effects in the particular study, while the indirect or shrinkage estimates are the estimates shrunk to the mean of all studies, taking into account the information from all other studies for the single estimate. Negative numbers favor treatment. Estimates are based on Bayesian random effect meta‐analysis models with weakly informative priors (normal (mean = 0, standard deviation = 1)) and heterogeneity priors (half‐normal (scale = 0.5)). 95% CI, 95% credible interval, AduEMERGEhd, aducanumab EMERGE trial, high dose; AduEMERGEld, aducanumab EMERGE trial, low dose; AduENGhd, aducanumab ENGAGE trial, high dose; AduENGld, aducanumab ENGAGE trial, low dose; Gant1, gantenerumab Graduate I study; Gant2, gantenerumab Graduate II study; Donanamab, donanemab TRAILBLAZER‐ALZ 2 study, Parameter estimates are given together with their 95% credible intervals. The “mean” effect size is given in score points of the CDR‐SB. The “prediction” value indicates the expected mean effect for a new study using the same or similar antibodies. The “heterogeneity” parameter tau (τ) quantifies the amount of between‐study variation that is, not explained by the fixed effect. CDR‐SB, Clinical Dementia Rating – Sum of Boxes; CI, confidence interval

FIGURE 3.

Joint posterior distribution of the heterogeneity and effect size estimates, plot of the joint posterior density of heterogeneity (τ) and mean effect (μ). Red lines trace the contours of constant density corresponding to approximate 2D 50%, 90%, 95%, and 99% credible regions. Horizontal green lines indicate medians and 95% credible intervals for heterogeneity parameter τ. The vertical green lines indicate medians and 95% credible intervals for the mean effect μ. They are based on the estimates from the posterior distribution of each parameter, corresponding to the average effects and credible intervals plotted in the forest plot. Blue lines show the conditional mean effect μ as a function of the heterogeneity τ (solid line) along with conditional 95% confidence bounds (dashed blue lines). This illustrates that only for values of the heterogeneity parameter τ at the upper end of the credible interval (most right horizontal dashed green line), the 95% credible interval of the mean effect begins to include zero.

When using a standard deviation of 4 instead of 1 for the effect prior and a non‐informative Jeffreys prior instead of the half‐normal distribution for the heterogeneity parameter, the effects were essentially unchanged (Figure S2). When we chose priors between 0.8 and −0.8 for the mean effect with an SD of 2, the results remained largely unchanged as well, underscoring the robustness of the findings.

Repeating the analyses with 0.5 times the SD added to the SD estimates, affected the numerical results mildly without changing the fundamental conclusions: the mean treatment effect was −0.33, a slightly wider credible interval of −0.58 to −0.03, and a heterogeneity estimate of 0.16 (0.0 to 0.41). The results for a prior centered at a small effect of −0.4 with a standard deviation of 2 are shown in Figure S3.

Finally, in the meta‐regression analysis the effect was −0.0061 [95% credible interval = −0.00086 to −0.0032] CDR‐SB score points less per one centiloid less amyloid PET signal with a heterogeneity estimate τ = 0.12 [0.00 to 0.31] (Figure S4).

4. DISCUSSION

Our Bayesian meta‐analysis suggests that overall, there was a moderate to small effect of ‐0.33 points less decline on the CDR‐SB and that heterogeneity between the studies was moderate. This effect size can explain why some of the trials fell on the “statistically significant” side of the p‐value and some did not, ¹ , ² even if one disregards potential biological ⁴ or pharmacological factors for efficacy. It is a misunderstanding of the p‐value to assume that a positive trial contradicts a negative trial because one is significant and the other is not. ⁷ A continuous approach can help overcome this misconception: in a frequentist framework, the use of exact p‐values rather than the dichotomization according to some threshold; in a Bayesian framework, estimating the posterior distribution of effect sizes and use of Bayes factors to quantify evidence for and against the null hypothesis.

Our analysis extends previous work ¹⁷ that found strong evidence for no effect of treatment in a Bayesian meta‐analysis across six trials with the antibodies solanezumab, bapineuzumab and one low‐dose trial of gantenerumab. Here, we provide moderate evidence supporting a treatment effect. Our results also extend a previous meta‐analysis that showed evidence against an effect based on the two aducanumab phase 3 trials alone. ¹⁸ Our results are compatible with the results of a meta‐analysis considering the high‐dose results of aducanumab and the previous phase 2 data for lecanemab and donanemab, yielding a mean effect of −0.24 CDR‐SB points. ¹⁹ The notion of a small effect is also compatible with previous frequentist meta‐analyses. ¹⁹ , ²⁰ , ²¹

Given the observed data, the 95% credible interval for the average effect was between −0.54 and −0.10 points on the CDR‐SB scale. When taking into account the actual observed decline of 1.57 to 3.67 points in the CDR‐SB over 18 months, the average effect corresponds to an average reduction in CDR‐SB rates of 9% to 21% across all studies compared with progression rates on placebo. This effect relates to a limited time window, where the key question will be if such effect persists when treatment extends beyond 18 months. Furthermore, these effects are below the minimal clinically important difference in CDR‐SB for mild cognitive impairment (MCI) and mild AD cases, based both on clinician judgement and a statistical criterion of half the baseline's SD. ²² These estimates of the minimal clinically important difference have been challenged because it was thought that a difference of about −0.5 points on the CDR‐SB score at 18 months may lead to a very significant effect for patients if it persists or even increases with longer treatment duration. ²³ While this would be true, such a persisting effect remains to be shown. Demonstration of disease modification would make enduring effects on cognitive outcomes more likely. Use of delayed start design and engagement of downstream markers by treatment have been proposed to demonstrate disease modifying effects. ²⁴

In our data, the effect of a new study with these or similar antibodies was predicted to fall between ‐0.88 to 0.25 points on the CDR‐SB scale. The heterogeneity estimate was moderate, despite studies differing in terms of statistical significance. Of note, the effect, albeit becoming very small, would still exclude zero even at more extreme estimates for the heterogeneity parameter as illustrated in Figure 2. These findings underscore that the negative trials ENGAGE, and Graduate I and II do not represent outliers but are in full agreement with the effects that have to be expected for such antibodies. Therefore, the evidence of all trials should be considered when investigating the efficacy, potential side effects, and patient and health care burden of an anti‐amyloid treatment. These results are encouraging, as they suggest that some real, albeit small effect, may underlie the recently published positive studies. At the same time, it is challenging because it suggests that the new‐generation antibodies achieve only small effect sizes, which alone are not sufficient to achieve a breakthrough in the disease‐modifying treatment of AD.

Consistent with a previous instrumental variable meta‐analysis that showed on average 0.3 points improvement in the Mini‐Mental State Examination (MMSE) score per 0.1 standardized uptake value ratio (SUVR) unit change in amyloid PET SUVR across 14 trials, ²⁰ we found that one centiloid less amyloid PET signal corresponded with −0.061 less worsening in CDR‐SB score, fully accounting for the overall treatment effect.

We would like to address the following limitations. First, we did not have access to the original data, so the standard deviation estimates were based on SE derived from charts, which resulted in some imprecision. To account for this, a sensitivity analysis was performed in which the SD estimates were increased by 50% without this changing the main results. Second, we treated the low‐dose and high‐dose aducanumab trials as separate studies. The reason was that we did not have access to the data pooled across doses. The paper ¹ and the FDA's summary statistics of Biogen's aducanumab trial only reported results split according to high‐ and low‐dose groups, consistent with the primary analysis plan for EMERGE and ENGAGE. Third, a limitation of meta‐analyses without access to the primary data are necessary assumptions on underlying data generation processes. Here, we assumed a Gaussian model for deriving standard error and standard deviation estimates. Fourth, we only considered phase 3 trial results, which differ from some previous studies that conducted meta‐analyses on a mixture of phase 2 and phase 3 trial outcomes. ¹⁹ , ²⁵ , ²⁶ Our focus was the degree of heterogeneity of the phase 3 trials that were conducted for purpose of regulatory approval, while aiming to assess whether the statistically non‐significant phase 3 trials of aducanumab and gantenerumab were outliers or fell within the expected range of findings. Adding data from phase 2 trials for aducanumab and donanemab would likely not have altered the overall results, as suggested by the data of, ¹⁹ but was outside of the scope of this study. Finally, a meta‐analysis can provide evidence regarding the existence and size of an effect, but it cannot establish the causes of any effect. Thus, other factors besides a true biological effect may contribute to the estimated effect. One example is possible bias from treatment‐specific adverse events, such as ARIAs or local reactions, which may contribute some unblinding of caregivers and raters assessing the CDR‐SB.

Future lines of research have already been defined but need to be prioritized. These include the development and use of primary endpoints that detect patient‐relevant outcomes. The CDR‐SB simultaneously measures cognitive and functional impairments, ²⁷ making it more relevant from a patient perspective than purely cognitive scales such as ADAScog. Nevertheless, the CDR‐SB may be less sensitive to patient‐relevant changes compared with more meaningful markers of everyday functioning that may become available through home‐based digital assessments, ²⁸ but remain to be validated for this purpose. ²⁹ Another line of research is investigating alternative routes of administration for treatments to reduce patient burden and healthcare costs, such as subcutaneous injection instead of intravenous infusion. ³⁰ Finally, targeting the amyloid cascade alone may not be enough to alter the course of AD. ³¹ A major requirement to establish the effect of any disease‐modifying treatment in AD and any other disease is the application of appropriate study designs ²⁴ and statistical approaches. Simply considering whether the results of a study fall on one side or the other of a significant p‐value ⁷ is not sufficient to derive maximum information from the results of a trial. ⁸ The Bayesian approach to probability helps interpret results in the context of all available studies. It provides quantitative evidence for or against an effect and direct estimates of the parameters of interest such as effect sizes and their actual distribution given the data. ¹⁷ , ³²

CONFLICT OF INTEREST STATEMENT

Stefan Teipel participated on scientific advisory boards of Roche Pharma AG, Biogen, and Grifols SA, and received lecture fees from Eisai. Anna G.M. Temp and Michael W. Lutz have no conflicts of interest. Author disclosures are available in the Supporting Information.

CONSENT STATEMENT

Consent of human subjects was not necessary since only aggregated metadata were entered into the meta‐analysis.

Supporting information

Supporting Information

Supporting Information

Teipel S, Temp AGM, Lutz MW. Bayesian meta‐analysis of phase 3 results of aducanumab, lecanemab, donanemab, and high‐dose gantenerumab in prodromal and mild Alzheimer's disease. Alzheimer's Dement. 2024;10:e12454. 10.1002/trc2.12454

DATA AVAILABILITY STATEMENT

The JASP file underlying the Bayes factor analysis and Figure 3 and the R code to generate the data and plot Figures 1 and 2 can be obtained without restrictions from the Open Science Framework at: https://osf.io/8tjda.