Estimated effects of amyloid reduction on cognitive change: A Bayesian update across a range of priors

Abstract

INTRODUCTION

The results of the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 trials have rekindled discussion on the impact of amyloid‐targeting drugs. We use a Bayesian approach to quantify how rational observers would have updated their prior beliefs based on new trial results.

METHODS

We used publicly available data from the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 trials to estimate the effect of reducing amyloid on the clinical dementia rating scale, sum of boxes (CDR‐SB) score. A range of prior positions were then updated according to Bayes’ theorem using these estimates.

RESULTS

After updating with new trial data, a wide range of starting positions resulted in credible intervals that did not include no effect of amyloid reduction on CDR‐SB score.

DISCUSSION

For a range of starting beliefs and assuming the veracity of the underlying data, rational observers would conclude there is a small benefit of amyloid reductions on cognition. This benefit must be weighed against opportunity cost and side‐effect risk.

Highlights

• The results of recent trials of amyloid‐targeting drugs have rekindled discussion on the impact of amyloid reductions achieved with amyloid‐targeting drugs on cognition.

• Prior to the announcement of trial results, beliefs about the effects of altering amyloid levels varied.

• For a range of starting beliefs, one would conclude there is a small benefit of amyloid reductions due to amyloid‐targeting drugs on cognition.

• The perceived value of individual drugs must balance the magnitude of this benefit against opportunity cost and risk of side effects.

1. INTRODUCTION

The results of the CLARITY‐AD trial and recent results of TRAILBLAZER‐ALZ 2 trial have rekindled discussion and debate on the value of reducing brain amyloid levels with amyloid‐targeting drugs. ¹ , ² , ³ , ⁴ , ⁵ Previous meta‐analyses of randomized trials of amyloid‐targeting drugs suggested that the effects of amyloid‐removal on cognitive outcomes were close to null, ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ and these studies continue to be cited and referenced as evidence that an overall effect of this mechanism is likely to be negligible. Some have interpreted the small statistically significant effects reported in CLARITY‐AD and TRAILBLAZER‐ALZ 2 as unconvincing. While some cite concerns regarding publication bias and other design issues, ¹² others cite a lack of clinical relevance of the reported effect size. ¹³ Common counterarguments include that newer drugs target specific amyloid species that may confer benefits that earlier drug classes did not. That is, they can achieve greater reductions in the relevant species, which is not necessarily captured by a simple measure of overall change in amyloid. ¹⁴ , ¹⁵ , ¹⁶ In addition, it has been suggested that there are non‐linear effects such that cognitive benefits are only realized with the larger amyloid reductions produced by these newer medications. ¹⁷

Bayesian statistics, formalized from the early to mid‐twentieth century, ¹⁸ , ¹⁹ , ²⁰ introduced the concept of a rational observer, which describes an individual who will constrain and update their beliefs according to the rules of probability. ²¹ Rational observers may have radically different starting beliefs, but, as increasing evidence accumulates, rational observers ultimately converge on the truth. In such a model, beliefs described with a prior probability distribution are updated with new evidence in order to obtain a posterior probability distribution that describes updated beliefs. ²⁰

Here, we consider how rational observers with very different beliefs about the effect of reducing amyloid on cognition, which may arise due to differences in beliefs about the pathogenesis of Alzheimer's disease (AD), ² , ³ , ⁴ publication bias, ²² or the trustworthiness of pharmaceutical companies (eg, in reporting deaths ²³ ), would have responded to the recently announced results from trials of lecanemab (CLARITY‐AD in December of 2022), gantenerumab (GRADUATE I and II in December of 2022), and donanemab (TRAILBLAZER‐ALZ 2 in July 2023). The goal of this analysis is to systematically describe how beliefs should have been updated when combining recent evidence with a range of starting beliefs about amyloid reduction.

2. METHODS

Summary statistics from the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 trials were used to obtain estimates of the clinical dementia rating scale, sum of boxes (CDR‐SB) score change per 10‐centiloid reduction for lecanemab, gantenerumab (GRADUATE I and II were considered together), and donanemab, respectively. Since only aggregated data were available and presented inconsistently across these trials, we used previously developed instrumental variable (IV) methods to estimate these quantities. ⁶ , ²⁴ These IV methods were developed to use aggregated data to obtain a standardized effect of amyloid removal on cognitive change across trials with heterogeneous amyloid removal. ²⁵ Given the large sample sizes, probability densities from the normal distribution were used to calcuated the likelihood. CDR‐SB scores are on an 18‐point scale with increases in CDR‐SB scores indicative of cognitive worsening. Summary statistics used to produce these estimates and associated standard errors are given in Supplemental Methods. ¹ , ²⁶ Trials report the CDR‐SB score for the full sample, while amyloid change is given for a positron emission tomography (PET) subsample for CLARITY‐AD and GRADUATE I and II. If the PET subsample were randomly selected from all trial participants, cognitive change per centiloid estimates would not be biased (precluding loss to follow‐up, which could affect all change measures). However, participation in such substudies was voluntary. However, to determine bias, substudy participation would have to be associated with future amyloid removal, and it seems unlikely that this would present a significant bias. Standard errors in the overall estimate reflect that the standard error for the amyloid removal estimate is larger than it would be if the whole sample had undergone PET. Participants in TRAILBLAZER‐ALZ underwent follow‐up PET scans to monitor amyloid removal progress, with slightly more participants receiving a final PET scan than completing the final CDR‐SB assessment.

RESEARCH IN CONTEXT

1. Systematic review: The results of the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 trials have rekindled discussion in the scientific literature on the effect on cognition of reducing the amyloid burden using amyloid‐targeting drugs. We surveyed the literature to ascertain a range of plausible beliefs represented in the scientific literature on the effect of amyloid reductions on cognition prior to the announcement of these trial results. We consider how rational individuals representing this range of positions would have responded to these trial results.

2. Interpretation: We found that, for a range of starting beliefs and assuming the veracity of the underlying data, rational individuals would conclude there is a small benefit of amyloid reductions due to amyloid‐targeting drugs on cognition.

3. Future directions: Any benefits of amyloid‐targeting drugs must be weighed against opportunity cost and side‐effect risk.

We consider four initial positions for beliefs about the effect of reducing amyloid on cognition: a position based on the results of previous meta‐analyses, ¹¹ , ²⁵ a position based on an assumption of significant publication bias that ascribes equal probability to benefit and harm, a position based on putative harm, and a position based solely on studies of monoclonal antibodies. ²⁵ Each position was given a normally distributed prior, with equal variances for all four positions. The mean and variance of meta‐analysis informed prior is subjective but roughly based on the frequentist confidence interval for CDR‐SB score change per 0.1 SUVR given in the 2022 update in Ackley et al. ²⁵ and assuming approximate transforms to centiloids for florbetapir. ²⁷ That is, the mean change difference and upper and lower confidence limits for this change were scaled by a factor of 0.554. ²⁷ The means of all other priors are intended to reflect a range of subjective starting positions. Table 1 describes the rationales for these priors and gives the means of each prior and symmetric intervals within which 95% of the probability mass of the prior distribution is located. This interval is a credible interval since we are explicitly ascribing belief to these priors. Additional priors with a range of means and variances were used in sensitivity analyses to validate the results.

TABLE 1.

Description of prior positions and mean and 95% credible interval for effect of a 10‐centiloid reduction on CDR‐SB score change for each of the four main starting positions. All priors have the same variance.

Position	Rationale	Prior mean and 95% credible interval
A. Meta‐analysis informed	A prior meta‐analysis gave a point estimate corresponding to a small benefit (based on Ackley 2021). We use the confidence interval from this meta‐analysis as the basis for our prior.	−0.028 (−0.072, 0.015)
B. Publication bias	Skeptics may believe prior meta‐analyses were too optimistic, given the potential for publication bias and numerous trials registered in clinicaltrials.gov with no results ever reported. We adopt a prior with the same variance as in the meta‐analysis‐informed case, centered at no effect and ranging from small harm to small benefit.	0.000 (−0.043, 0.043)
C. Putative harm	Given concerns about the side effects of amyloid‐targeting therapies, some individuals may believe such therapies are harmful. We adopt a credible interval with the same variance as in the meta‐analysis‐informed case, with uncertainty ranging from very small to moderate harms.	0.055 (0.012, 0.099)
D. Based on monoclonal antibody (MAB) drugs	Some researchers prefer meta‐analyses restricted to data from select trials of monoclonal antibody drugs, which indicate a larger benefit than the comprehensive meta‐analysis previously considered. We adopt a posterior with the same variance as in the meta‐analysis‐informed case, with uncertainty ranging from very small to moderate benefits.	−0.055 (−0.099, −0.012)

The four prior positions were updated using Bayes' theorem using the IV estimates for the effect of a 10‐centiloid reduction on the CDR‐SB score from the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 trials. We used the fact that the normal distribution is a conjugate prior for a normally distributed likelihood function with a fixed variance in order to have a closed‐form solution for the posterior distribution. This gives an analytic solution for the posterior distribution, which is also normal. ²⁸ The likelihood was considered to be normally distributed with mean given by the effect per 10 centiloids estimated from the trials and variance. Since the t distribution does not have a conjugate prior, ²⁹ the variance was assumed to be precisely known and given by a variance obtained from IV estimates. This is analogous to using z versus t statistics in frequentist statistics, and, given the size of the trials, can be considered a very good approximation. The prior position was updated with both the CLARITY‐AD trial and the GRADUATE I and II trials together, as well as with CLARITY‐AD trial and the GRADUATE I and II trials individually.

As sensitivity analyses, we consider 33 additional starting priors, ranging from effects of −0.1 to 0.1 CDR‐SB points per 10‐centiloid reduction. We also evaluated alternate standard deviations—both narrower and wider by a factor of two than that of the main analysis—for this range of prior means. These priors are specified in Table S1. Finally, additional posteriors, which include additional data from the CREAD trial, are included in Table S2.

Finally, we calculate prior mean thresholds required to have a posterior that produces a credible interval with an upper limit of zero for three standard deviations: the standard deviation used in the main analysis and two additional standard deviations narrower and wider by a factor of two than that of the main analysis. We refer to these as the main‐analysis, narrow, and wide standard deviations. All analysis was performed using R version 4.2.1. Since this study used only publicly available, aggregated summary statistics, it is not research involving human subjects.

3. RESULTS

The IV estimates for the effect of a 10‐centiloid reduction on CDR‐SB scores are −0.076 (standard error, 0.019) for CLARITY‐AD (lecanemab), −0.043 (standard error, 0.021) for GRADUATE I and II (gantenerumab), and −0.081 (standard error, 0.015) for TRAILBLAZER‐ALZ 2 (donanemab). Means and associated standard errors were used to define the likelihood used to update the starting priors. Posterior means and associated credible intervals for updates of the four main priors with estimates from CLARITY‐AD and GRADUATE I and II estimates are shown in Table 2. Prior and posterior distributions are shown in Figure 1. After updating with only the CLARITY‐AD estimate, the mean of the posteriors varied from a benefit of 0.020 to 0.067 CDR‐SB points per 10‐centiloid reduction, with only the 95% credible interval for the putative harm position including no effect. After updating with only the GRADUATE I and II estimate, the mean of the posteriors varied from a harm of 0.003 to a benefit of 0.049, with 95% credible intervals for the publication bias and putative harm positions including no effect. After updating with only the TRAILBLAZER‐ALZ 2 estimate, the mean of posteriors ranged from a benefit of 0.039 to 0.073, and no 95% credible intervals contained no effect. After updating with data from all trials, the mean of the posteriors varied from a benefit of 0.048 to 0.068, and no 95% credible intervals contained no effect. Table S1 and Figure S1 give posterior means and associated credible intervals for updates of additional priors. In prior‐mean‐threshold calculations, we find that prior means of at least 0.23, 0.05, and 0.94 CDR‐SB points per 10 centiloids for the main‐analysis, narrow, and wide standard deviations, respectively, are required to produce credible intervals with an upper limit of no effect when updating with data from all trials.

TABLE 2.

Prior and posterior means and credible intervals for effect of a 10‐centiloid reduction on CDR‐SB score for each of the four main starting positions.

Prior description	Prior	Posterior CLARITY‐AD only	Posterior GRADUATE I and II only	Posterior TRAILBLAZER‐ALZ 2 only	Posterior CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2
A. Meta‐analysis informed	−0.028 (−0.072, 0.015)	−0.056 (−0.084, −0.027)	−0.036 (−0.066, −0.006)	−0.065 (−0.089, −0.041)	−0.063 (−0.081, −0.045)
B. Publication bias	0.000 (−0.043, 0.043)	−0.044 (−0.072, −0.015)	−0.023 (−0.052, 0.007)	−0.056 (−0.080, −0.032)	−0.058 (−0.076, −0.040)
C. Putative harm	0.055 (0.012, 0.099)	−0.020 (−0.048, 0.008)	0.003 (−0.027, 0.033)	−0.039 (−0.063, −0.015)	−0.048 (−0.067, −0.030)
D. Based on MAB	−0.055 (−0.099, −0.012)	−0.067 (−0.096, −0.039)	−0.049 (−0.078, −0.019)	−0.073 (−0.097, −0.049)	−0.068 (−0.086, −0.050)

Note. Posterior columns give means and credible intervals for posteriors updated with just the CLARITY‐AD estimate, posteroirs updated with just the GRAUDATE I and II estimate, and posteriors updated with the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 estimates. All prior and posterior distributions are normally distributed, and the interval is constructed to be symmetric about the mean and contain 95% of the probability mass.

Abbreviation: MAB, monoclonal antibodies.

FIGURE 1.

Prior and posterior distributions for effect of a 10‐centiloid reduction on CDR‐SB score for the four main starting positions. (A–D) correspond to the four starting positions outlined in Tables 1 and 2. Each panel shows the prior, a posterior updated with just CLARITY‐AD estimates (“CL”), a posterior updated with just GRAUDATE I and II estimates (“GR”), a posterior updated with just TRAILBLAZER‐ALZ 2 estimates (“TR”), and a final posterior (“Posterior”) that updates with all trial data. Mean ($\mu$) and standard deviation ($\sigma$) of the prior are given in the panel label. MAB: monoclonal antibodies.

4. DISCUSSION

After updating with recent findings, all starting positions resulted in credible intervals that did not include no effect of amyloid reduction on CDR‐SB score change. Although the most divergent starting positions differed by 0.110, the means of the posteriors differed by only 0.020 CDR‐SB points per 10‐centiloid reduction. Therefore, for a wide range of starting beliefs, rational observers with those beliefs should conclude—if inferences are drawn only from the available trial evidence—that within the time frame of the trials, there is a small benefit for cognition of amyloid reductions due to amyloid‐targeting drugs. However, determining whether an individual amyloid‐targeting drug has value requires consideration of not only the average effect but whether the drug produces meaningful changes for an individual, the probability and severity of side effects, and the opportunity cost to the individual and to the healthcare system.

Many have argued that monoclonal antibody drugs or drugs that effectively target protofibrils are more effective than older small‐molecule drugs to explain why prior trials failed. ¹⁴ , ¹⁵ , ¹⁶ However, these arguments are no longer necessary to justify beliefs that amyloid‐targeting drugs are effective. This is because, with the addition of current evidence, pooling across all drugs for which there are data available indicates overall benefit. That is, for a wide range of starting beliefs, including those that equally weight past and current evidence, updated beliefs should assign low probabilities to no effect or cognitive harm of amyloid reduction. The new posterior point estimate and credible interval are consistent with previous meta‐analysis results showing a small benefit across trials with confidence intervals that included the null because, without the new trial data, estimates were less precise. ⁶ Further, pooling across all drugs is a more conservative assumption, thus making that assumption and finding overall benefit strengthens arguments for overall benefit.

Rational observers should also conclude that large benefits, such as those exceeding 0.1 CDR‐SB points per 10‐centiloid reduction, are also implausible. The most optimistic starting beliefs, based on the results of the EMERGE and ENGAGE trials of aducanumab, also preclude large benefits. For context, individuals with a diagnosis of symptomatic AD decline by an average of one to two CDR‐SB points per year, depending on severity. ³⁰ In addition, although the GRADUATE I and II trials did not meet their primary endpoint, ³¹ , ³² the point estimates were similar to the meta‐analyzed effect estimate. That is, while gantenerumab's effects in GRADUATE I and II appear to be smaller than those of CLARITY‐AD, in the context of prior evidence, they do support a small overall benefit of amyloid‐targeting drugs.

Our analysis has important limitations. First, the selection of priors is subjective. However, the meta‐analysis‐informed prior reflects the estimated CDR‐SB score change associated with an approximately 10‐centiloid reduction based on a previously published frequentist meta‐analysis. ²⁵ Like the meta‐analysis point estimate, the mean of this prior indicates a small benefit, but this benefit is smaller than the point estimates reported in the ENGAGE, CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 studies. Also, like the meta‐analysis confidence interval, the 95% credible interval includes no effect and ranges from very small cognitive harm to no effect to small cognitive benefit. Furthermore, while other reasonable selections for priors would be possible for this and each of the other positions, the majority of priors, including priors that indicate significant harm, nonetheless indicate an overall benefit. An additional limitation of the meta‐analysis‐informed prior is that it was based on a post hoc conversion to centiloids that may not adequately capture variances between studies, with the true variance being either larger or smaller than the calculated variance. Analyses based on a wider range of 33 alternative priors are given in the supplement, including priors with both wider and narrow variances.

Second, a longstanding criticism of Bayesian statistics is that different priors can produce dramatic effects on calculated posteriors. However, different rational observers with different starting positions should nonetheless converge on a single value corresponding to the true parameter with repeated experimentation. Typically, convergence would not be expected with just three new trials. However, we find that recent trials were much more informative than prior trials due to the large effects on amyloid reduction and the relatively large sample sizes. As a result, in our analyses, we find that for a range of starting positions, including some that would not be well justified by prior data, rational observers should conclude there is overall benefit for amyloid removal.

A final limitation is that priors outside the range of those we considered or violations of other necessary assumptions may imply that observers may conclude that amyloid‐targeting drugs confer no benefit, despite the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 results. While we assume the veracity of the reported data, skeptics may believe that the CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 trials were poorly designed or had significant flaws leading to biased estimates. This might include loss of blinding, lack of assessment of disease modification, broken randomization, differential loss to follow‐up or mortality, or frank malfeasance and data fabrication. ¹ , ¹² , ³³ , ³⁴ It is beyond the scope of this article to assess whether data quality concerns are plausible or have been adequately addressed (eg, sensitivity analyses in supplemental information of ¹ ). Current skepticism is fueled by financial conflicts of interest, lack of long‐term follow‐up, and lack of data sharing. ²² Providing extra evidence (ie, data sharing) and consideration would help the field come to consensus. ³³

Individuals have a range of beliefs on the effect of amyloid removal on cognition. The paper represents an attempt to ascertain whether consensus on whether amyloid removal reduces cognitive decline is reasonable given the diversity of starting opinions and based on the available trial data to date. Our results would indicate that amyloid removal appears to confer small benefit based on available evidence, but those results do not pertain to the value of any particular drug. Debates will certainly continue regarding whether these drugs should be approved or reimbursed and in which populations, as such decisions require not only evidence of an effect but the clinical relevance of an effect, safety concerns, and other factors. Moreover, individual decisions about whether to take an approved drug will additionally involve consideration of the associated opportunity costs. Specifically, questions remain regarding safety and concurrent anticoagulant usage, ²³ the generalizability of findings to Black and other populations with increased vascular risk, ³⁵ costs of the drug and infrastructure required for MRI safety monitoring, ³⁶ whether apparent benefits are due to unblinding, ³³ and long‐term outcomes. ³⁷ , ³⁸ Further, overstating the theoretical confirmation or therapeutic advances associated with the apparent efficacy of these drugs may have negative consequences for scientific progress in the field. ³⁹

The results of this study are not surprising in the context of prior frequentist meta‐analyses. In Ackley et al. the confidence interval for the overall effect of amyloid reduction on cognitive change in 2021 included both small effects and no effect of amyloid reduction on cognitive change (0.05, 95% CI: (−0.032, 0.13) Mini‐Mental State Examination points per 0.1 SUVr). However, that meta‐analysis included a calculation that suggested one compelling trial could shift the overall estimate to statistical significance. A meta‐analysis by Pang et al. replicating Ackley et al. included data from the phase II PRIME trial of aducanumab, one of the most optimistic trials of the benefits of amyloid removal to date (cognitive change per SUVR change), and obtained an estimate that indicated overall benefit. ²⁴ Other analyses did not estimate a cognitive benefit per amyloid change but indicated either null effects or small benefits were plausible. ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ None of these meta‐analyses include the most recent data from CLARITY‐AD, ¹ GRADUATE I and II (unpublished), and TRAILBLAZER‐ALZ 2. ⁵

Some researchers argue, based on qualitative assessment of the trials ⁴⁰ and animal models, that amyloid removal has non‐linear effects on cognition, with benefits only becoming apparent once a large fraction of amyloid has been removed. ¹⁷ We also did not consider other quantities that might be scaling factors preferable to amyloid removal, such as residual amyloid remaining after treatment. Furthermore, since amyloid removal may operate via its downstream effects, drugs that remove amyloid faster might be expected to show greater benefit over an 18‐month timeframe. ¹⁷ Thus, reporting a per 10‐centiloid decrease could be viewed as an inaccurate linearity assumption or an oversimplification of the treatment definition. However, we note that CLARITY‐AD, GRADUATE I and II, and TRAILBLAZER‐ALZ 2 removed a similar amount of amyloid, so our estimates pertain to the specific ranges of amyloid in these trials.

This quantitative approach allows for a determination of rational positions given a range of prior starting beliefs. Current debates should focus on upstream issues such as data and analysis quality and downstream issues such as whether minimum clinically meaningful differences have been achieved, safety, infrastructure, and generalizability of results. Based on available evidence, it seems reasonable to believe that amyloid reduction is associated with decreased measures of cognitive decline but provides at most small cognitive benefit on the timescale of these trials.

CONFLICT OF INTEREST STATEMENT

The authors declare that they have no relevant financial disclosures. Author disclosures are available in the supporting information.

CONSENT STATEMENT

Since this study used only publicly available, aggregated summary statistics, it is not research on human subjects.

Supporting information

Supporting Information

Supporting Information

Supporting Information

Ackley SF, Wang J, Chen R, Power MC, Allen IE, Glymour MM. Estimated effects of amyloid reduction on cognitive change: A Bayesian update across a range of priors. Alzheimer's Dement. 2024;20:1149–1155. 10.1002/alz.13470