Comment on “Effect of reduction in brain amyloid levels on change in cognitive and functional decline in randomized clinical trials: An updated instrumental variable meta-analysis”

To the Editor

We read with great interest Pang et al.’s article “Effect of reduction in brain amyloid levels on change in cognitive and functional decline in randomized clinical trials: An updated instrumental variable (IV) meta-analysis.” Meta-analyses of trials of amyloid-targeting drugs are valuable because they can quantify the effect of drug-induced changes in amyloid on cognition and meta-analytic results should shape interpretations of future trials of medications with similar biological mechanisms. As authors of the original Ackley et al. instrumental variable meta-analysis that publicized this method,(1) we are pleased that researchers at Biogen have updated and extended our original work. When considering only the same trials we used, but with updated and corrected values, Pang et al. replicated our finding: amyloid reduction was non-significantly associated with a small slowing of MMSE decline. Pang et al.’s analyses also incorporated data from new trials and internal Biogen reports, and considered additional cognitive outcomes. Their final results indicated small, statistically significant benefits of amyloid reduction on cognitive change. The confidence intervals in the Pang et al analysis included our estimated effects and precluded large benefits of amyloid reduction within the trial follow-up period.

Repeated failures of anti-amyloid medications are often cited as evidence against the amyloid cascade hypothesis of Alzheimer’s Disease (e.g., (2)). As discussed in our original manuscript, many trials of earlier drugs removed very little amyloid and thus are not particularly influential in the meta-analyzed IV estimate. Therefore, with ongoing large trials of drugs that effectively and rapidly remove amyloid, incorporating new and updated data into pooled estimates is an important endeavor. Pang et al.’s article makes a substantive contribution to our understanding by updating results including new and additional trials of anti-amyloid antibody drugs. We note that their updated estimate is contained within our confidence interval.

However, Pang et al.’s analysis faced many of the limitations of our own paper arising from incomplete data availability for the underlying trials. Fewer than half of prior trials of amyloid-targeting therapies ever publicly reported results on both change in amyloid-PET SUVr and change in a cognitive measure. It is almost certain that these trials did not achieve cognitive benefit,(3) but their impact on amyloid reduction is unclear. If the unreported trials did not change amyloid, they are largely irrelevant for the meta-analyzed effect estimate; if the unreported trials reduced amyloid but did not affect cognition, including them in the meta-analysis would reduce the estimated benefit of amyloid reduction.

Any pooled estimates hinge on quality and reliability of the underlying data. Neither Ackley et al. nor Pang et al. were able to use centiloid-based measures of amyloid change and neither adopted a random-effects meta-analysis, which would mitigate concerns that trials used different radiotracers to estimate change in amyloid. The current meta-analyzed estimates are substantially influenced by data from Biogen on lecanamab (BAN2401) and aducanumab. The termination of aducanumab trials and subsequent post-hoc analyses that diverge from the pre-specified analyses,(4) as well as important protocol changes in the trial of lecanemab,(5) suggest these aggregated data may not be perfectly reliable. Further, the aggregated data are often reported inconsistently across data sources. In some cases, results were reported not as the raw, group-specific means, but as adjusted estimates from imprecisely described regression models. Regression-based adjustments were sometimes necessary to address protocol changes in the trials, but unreported decisions about the regression model specification may have important consequences for the change estimates in the meta-analysis.

Ackley et al. developed an ad-hoc IV method to estimate the IV using only aggregated data–a method also implemented in Pang et al. A similar meta-analysis with individual-level data would have major advantages. Using individual-level data would permit assessment of heterogeneity in response to amyloid reduction, so we could determine whether some patients respond to amyloid reduction while others do not. Individual-level data would also permit evaluation of a non-linear dose-response relationship between amyloid-reduction and cognition, in case a minimum threshold of amyloid reduction is necessary to achieve any cognitive benefit. Individual-level data could be used to evaluate differential loss to follow up due to side effects such as ARIA,(6) a selection process that could bias results. IV analyses with individual-level data require fewer assumptions. Privacy concerns with sharing individual level data for these analyses are minimal, because only information on randomization group, amyloid change, cognition, and hypothesized modifiers such as APOE-ε4 status would be needed.

In light of the limitations of the data underlying Ackely et al. and Pang et al. we urge caution in interpreting p-values. The estimates in Pang et al. are calculated assuming no publication bias, non-informative loss to follow-up, consistency across radiotracers, and homogeneity in effect across drugs. Given that all of these biasing processes are likely, even this small estimated benefit in Pang et al. may be too optimistic.

Good science requires that methods and estimates be updated and improved upon. We are pleased to see the continued use of instrumental variable meta-analysis to update estimates of the effect of amyloid reduction on cognitive change. Given the importance of settling questions not only around the efficacy of specific drugs, but also around Alzheimer’s disease biology, we hope that other scientists will continue to update these pooled estimates, as more and better data becomes available. This goal–and the pursuit of effective treatments for Alzheimer’s disease–would be best served if pharmaceutical companies made individual-level raw trial data available to independent researchers.