Effects of intensive lifestyle changes on the progression of mild cognitive impairment or early dementia due to Alzheimer’s disease: the need for rigor

Abstract

We consider the recent publication by Ornish and colleagues and the rigor expected for interventional clinical trials. We contend that lifestyle intervention trials should strive for the same rigor as drug trials and highlight opportunities to improve rigor in this example, particularly in design, data analysis, and publication of results for this and other lifestyle intervention studies.

We read with interest the recent publication in Alzheimer’s Research and Therapy by Ornish et al. [1] This article communicated the results of a randomized controlled trial of “intensive lifestyle changes” in patients with Mild Cognitive Impairment. The article sparked extensive media coverage, including interviews of participants in the trial on CNN. This demonstrates the keen interest in dementia research and the immediate impact trial publications can have on public awareness of possible interventions.

The media coverage hailed this as a study that demonstrated lifestyle changes could “improve Alzheimer’s symptoms for some” (https://www.cnn.com/2024/06/07/health/alzheimers-dementia-ornish-lifestyle-wellness/index.html), yet we are concerned that such conclusions could be premature, particularly given some of the methodological limitations of the study. Lifestyle interventions have received substantial media attention since the publication of the FINGER study [2], the Lancet Commission report on lifestyle risk factors [3], and the launching of the Worldwide Finger Initiative to test lifestyle interventions in more than 19 global regions [4]. Thankfully, the first of these studies have begun to publish protocol papers [5–7] and have strived to conduct rigorous studies that ensure results are as free from bias as possible. This standard must be held in all studies but particularly interventional research. The trial by Ornish and colleagues demonstrates important opportunities to increase rigor in trials of lifestyle interventions.

First, trials should be designed with the goal of reducing bias. The gold standard design to reduce bias is the randomized double-blind placebo-controlled trial. Ornish and colleague indicate that such a design is not possible with lifestyle interventions and instead opted for a wait list control in which those randomized to non-intervention were informed they would have access to the intervention program “free of charge” after the 20-week intervention period was concluded. Indeed, it is difficult to blind someone to whether they are physically exercising. Other studies, however, have introduced controls such as stretching and toning [8] that control for the not insignificant effects of these routine interactions, often in groups and with expert facilitators. Moreover, true controls were entirely feasible for other aspects of the intervention, such as providing meals that failed to achieve the food quality standards indicated by the investigators as essential in their intervention, and placebo controls for the eight dietary supplements they included. Without these controls, the trial results are highly susceptible to placebo effect and other sources of bias, including investigator bias when completing the outcome measures, which notably included subjective assessments.

Second, trials of lifestyle interventions require careful planning for statistical power and prespecified statistical analysis plans. Such power calculations are essential to ensuring that participants are not needlessly put at risk by enrolling them in trials doomed to be underpowered and unable to test the hypothesis under study [9]. Ornish and colleagues indicate that prespecified power calculations informed the need for a trial of no less than 100 patients, but that slow recruitment and the COVID-19 pandemic forced closure of the trial early, after just 51 participants were randomized. This would seem to place the trial at risk for being drastically under-powered, though it is impossible to assess this because the approach to determining power for the stated co-primary outcomes is never described. The authors also strayed from trial convention in their approach to data analysis. As indicated in the supplementary material (the primary paper did not include information about statistical analyses), the authors used one-tailed level 0.05 tests of their primary outcomes, a departure from the more typical two-tailed level 0.05 tests (or one-tailed level 0.025 tests) that leave open the possibility of an intervention resulting in benefit or harm. A more conservative approach to protect against erroneously concluding intervention benefit would have been to set the familywise alpha level to be 0.025 (half of the oft used 0.05), with this total type I error probability partitioned across the four co-primary outcomes. Another common approach would have been to implement a closed testing procedure with co-primary endpoints ranked in testing order, but this was not done either. Instead, the authors used the more liberal one-sided level 0.05 for the test of each outcome, which was achieved for three out of four outcomes, but not a fourth (and none would have achieved the more rigorous alpha level). Lastly, the supplementary material indicates that testing of three of the co-primary outcomes was based upon the Mann-Whitney-Wilcoxon rank-sum test, yet the manuscript reports differences in estimated means along with these p-values. The Mann-Whitney-Wilcoxon rank-sum is not, however, a test of means [10] and therefore the statistical inference presented does not correspond to the scientific estimates given in the paper.

Finally, just as with drug trials, manuscripts reporting results from trials of lifestyle interventions should adhere to CONSORT reporting guidelines (https://www.equator-network.org/reporting-guidelines/consort/). This includes publishing the trial protocol and the prespecified statistical analysis plan with the primary paper. It also includes a checklist that ensures rigor and reproducibility. Reproducibility is particularly challenging when developing more “personalized approaches” to interventions, but if the goal is to advance the field toward meaningful therapies, this will be paramount. In addition to the other elements described above, many of which are indicated in the CONSORT guidelines, we also would have liked to see a more complete description of the eligibility requirements and reasons for ineligibility for the 1300 participants considered for the trial but deemed ineligible; information about who implemented the randomization scheme and how; and clearer information about how the specific trial was funded and potential conflicts of interest for the authors.

In conclusion, lifestyle interventions are prominent and promising interventions for cognitive disorders that warrant investigation. Yet, absent the same requirements for rigor and reproducibility as are applied in drug and other interventional trials, the field is at risk for misinformation resulting from biased studies.

Author contributions

JDG and DLG contributed equally to the design, drafting, and editing of this manuscript.

Funding

This work was supported by NIA AG066519.

Data availability

No datasets were generated or analysed during the current study.

Declarations

Ethics approval and consent to participate

Publish were not required for this communication.

Competing interests

The authors declare no competing interests.