Perspective: A Critical Look at the Ancillary Age-Related Eye Disease Study 2: Nutrition and Cognitive Function Results in Older Individuals with Age-Related Macular Degeneration^,

Abstract

A large body of literature suggests that the dietary carotenoids lutein and zeaxanthin and long-chain polyunsaturated fatty acids such as docosahexaenoic acid are related to improved cognitive function across the life span. A recent report by the Age-Related Eye Disease Study (AREDS) group appears to contradict the general findings of others in the field. In this review, we look critically at the methods, study designs, and analysis techniques used in the larger body of literature and compare them with the recent AREDS reports.

As the world’s population ages, the number of adults predicted to suffer from cognitive decline and dementia is expected to increase dramatically (1). Alzheimer disease (AD)³, the most common form of dementia, shares several features with other central neurodegenerative diseases, such as age-related macular degeneration (AMD). Kaarniranta et al. (2) referred to AMD as “Alzheimer’s of the eye,” with good reason; AD and AMD share many etiologic and histopathologic similarities, such as extracellular deposits containing β-amyloid. Oxidation and inflammation appear to be key to both diseases, and neither seems particularly amenable to late-stage treatments. This is 1 reason why prevention, especially aimed at reducing oxidative and inflammatory stress, is often regarded as our most promising approach to these illnesses.

Over the past few decades, the evidentiary basis for this approach has become clear. Importantly, such evidence does not come only from the field of epidemiology; rather, it represents a confluence of data ranging from cellular studies (3), animal models (4), and human clinical studies that are based on methods such as neuroimaging (5). Such data suggest that early lifestyle interventions, especially improving the diet, can dramatically lower the risk of degenerative diseases that affect the central nervous system. For example, Mares et al. (6) studied the lifestyle characteristics of older women (55–74 y) over a period of 6 y as part of the Carotenoids in Age-Related Eye Disease Study. The women who had a “good diet,” defined as the highest quintile in the Healthy Eating Index, had a 46% lower probability of developing early AMD. That number jumped to 71% when higher levels of physical activity and nonsmokers were included. Feart et al. (7) followed older subjects over a period of 10 y and found that subjects with the highest serum lutein concentrations had a significantly lower risk of all-cause dementia and AD.

A general reading of the literature on diet, AMD, and AD, however, shows that the results are actually quite mixed. Most of the results that are based on laboratory studies including animal and cellular models are consistent in showing the risks posed by oxidative/inflammatory stress and a protective role of diet. In contrast, the epidemiologic results are inconsistent; some indicated a protective role of a good diet (many from longitudinal analyses), whereas others showed null results. This pattern of mixed results was similar to those that were reported when originally studying the association between smoking and AD; some studies found that smoking actually reduced AD risk (8). This unlikely result was likely due to errors in experimental design, as reviewed by Kukull (9).

Smoking behavior is relatively easy to associate with disease risk because people usually tend to remember when they started smoking and have relatively stable smoking habits, especially if they are moderate or heavy smokers (10). Long-term dietary intake, on the other hand, is very difficult to accurately quantify, making the study of dietary effects on chronic disease even more challenging. Longitudinal studies of dietary intake and degenerative diseases that occur at the end of life tend, for obvious reasons, to assess individuals who are older, usually old enough that the disease of interest will have some chance of occurring during the study period (11–15). Such individuals have likely had covert disease for many years before the study intervention [some believe, for example, that many chronic diseases begin in utero and reflect the accumulation of damage over a lifetime (16)]. Often these diseases are as multifactorial as aging itself, and usually only a very small number of lifestyle factors are explored. Furthermore, because the number of subjects studied tends to be very high, the measures themselves tend to be crude and short. All of these challenges characterize recent results from the second Age-Related Eye Disease Study (AREDS2) trial and highlight the limitations of these types of trials.

AREDS2

The Age-Related Eye Disease Study (AREDS), launched > 20 y ago by the National Eye Institute, was the first large-scale study on nutrition that the institute had initiated [typically, the NIH spends ∼5% of its total budget on nutritional research (17)]. The study found that a supplement containing a mix of vitamins and minerals (vitamins C and E, β-carotene, copper, and zinc) reduced progression to later stages of AMD by ∼25% (18). Consequently, recommending the AREDS supplement has become standard practice by many eye care providers for older individuals who show early signs of macular degeneration. The supplement had some shortcomings, however. For instance, it included β-carotene, the only commercially available carotenoid at the time. Albanes et al. (19) showed an increased risk of some forms of cancer, especially lung cancer in current smokers, with β-carotene supplementation. Because smoking was known even at that time to be a risk factor for AMD [reports of smoking as a risk factor date back to the 1970s, e.g., (20)], the inclusion of β-carotene in the original supplement was a source of concern. In addition to concerns about β-carotene, a subsequent analysis (1) also showed that some individuals (21) (∼19% of the sample, specifically with the CFH genotype) responded negatively to the addition of zinc to the supplement.

In 2006, a second clinical trial was launched, termed AREDS2 (22–24). The goal of this trial was to investigate whether changing the original supplement would modulate the effects. Specifically, the macular carotenoids lutein and zeaxanthin were added to the supplement (eliminating β-carotene for some arms of the study), the long-chain PUFAS (LC-PUFAs) DHA and EPA were added to the supplement, and amounts of zinc were altered. These variations to the supplement were tested in 4 separate arms: original AREDS, AREDS + lutein and zeaxanthin (LZ), AREDS + DHA and EPA, and AREDS + LZ and LC-PUFAs. Zinc reduction and β-carotene elimination were also tested within these arms (2). One unusual aspect of the study was that the sample was maintained on the original AREDS supplement (on the basis of the “standard of care” argument; hence, it was considered unethical to have a true placebo control who did not receive the supplement). The basic conclusion of AREDS2 was that, other than eliminating β-carotene, adding additional nutrients did not cause a further reduction in risk in the primary analysis (secondary and subgroup analyses did find an ∼10% improvement with LZ) (3).

In 2015, an ancillary study of AREDS2 was published that used the same complex design with “cognition” as the outcome variable (24). This study was initiated in part on the basis of findings presented in an AREDS report (25), which suggested that those with AMD who suffered from poor visual acuity were at increased risk of cognitive decline. To make this association, AREDS participants were given an in-clinic cognitive function battery. For the AREDS2 ancillary study, the in-clinic battery was converted to a telephone battery (26) and administered to a subset of the AREDS2 sample. The AREDS2 ancillary study concluded that, “Among older persons with AMD, oral supplementation with LC-PUFAs or lutein/zeaxanthin had no statistically significant effect on cognitive function.” Because this result appears to contradict many others in the literature, it is worth examining in greater detail.

Experimental Design

The authors described this ancillary study as a “randomized, double-masked, placebo-controlled, 2 × 2 factorial trial” (26). They also stated that “All participants were also given varying combinations of vitamins C, E, β-carotene, and zinc.” It is typically the case that in a placebo-controlled trial, the placebo group serves as an untreated control who is given a true placebo, which has no inherent therapeutic effect. In drug trials, patients who receive a placebo actually receive a true placebo and are naive to the substance being tested against the placebo. True placebo-controlled designs are rare in nutrition trials, because the placebo group is rarely naive to the substance being tested in the active treatment. In the case of the AREDS2 trial, for example, even if there were a true placebo group against which the AREDS formulation + LZ and LC-PUFAs could be compared, individuals randomly assigned to that group would still have had exposure to vitamin C, vitamin E, zinc, and the other components of the supplement in their normal diet. With the exception of this issue, however, it is possible to otherwise have a true placebo group (a group that is untreated outside of usual dietary consumption), at least within the confines of the study. In the case of this study, a true placebo control would have been necessary to determine whether LZ and/or LC-PUFAs improved cognitive function or prevented cognitive decline (2 separate but related research questions). This design was, however, not used in the AREDS2 or the ancillary study.

The AREDS2 is not the only trial that has combined a placebo with another treatment. There are examples, almost entirely from pharmacy, in which a placebo is given with a drug and then compared with that same drug combined with other drugs that are thought to modulate the effects of the drug in question. The use of the term “placebo-controlled” in this case can often be misleading, because the “placebo” group is not serving as an untreated control.

The goal of this type of design is, simply, almost always to improve upon a formulation. This appears to have been a primary goal of the AREDS2. Unlike AREDS2, however, when this is the primary goal the studies are typically funded by the companies that make, and profit from, the formulations.

An argument in favor of the “additive placebo” design is that the treatment in question represents the “standard of care.” Hence, withholding that treatment would represent a risk. If, however, the goal of AREDS2 was to study the role of LZ and/or LC-PUFAs in cognition, it is unclear why they would select a sample who had eye disease and therefore needed to be taking a supplement that would preclude a true control group. In other words, the standard-of-care argument works for AMD but not cognition, which was the focus of this result. In both cases (studying AMD and cognition, presumably with hopes of preventing decline), convenience sampling of this sort seriously reduces the generalizability of a study.

Study Population

Given the above issues with experimental design, the AREDS2 ancillary cognition study can, at best, conclude that additive nutrition is not more effective than the original nutrient combination, which the study authors already showed to be effective at reducing risk of AMD (18), a central nervous system disease related to AD (2, 27–31). Unless additional caveats are made, however, this conclusion seems highly constrained, which leads to questions about the study population itself.

First, the authors described the study population as “well nourished.” Because those with the lowest intakes of foods such as fruit and vegetables and oily fish [for review, see (32)] tend to be at the highest risk of cognitive decline, the AREDS2 population seemed least likely to benefit from a nutritional intervention. Moreover, all of the participants were, as stated previously, also taking a mixed antioxidant supplement containing some of the very antioxidants known to be depleted in those at risk of cognitive decline (33), such as vitamins C and E.

Second, the study population also had AMD. Although some studies [e.g., (34)] found links between age-related eye disease and cognitive functional decline [including the original AREDS trial (25)], there is a difference between cognitive decline and dementia, and not all studies have shown a link between AMD and AD [e.g., (35)]. Because both diseases share risk factors, however, selecting individuals with a largely untreatable, progressive central nervous system disease (AMD) that 1) closely mirrors AD in terms of lifestyle risk factors and 2) is related to cognitive decline also appears ill-advised. For example, Nolan et al. (36) found that supplementing patients with AD with macular carotenoids did not improve cognitive function. Studies that focused on older adults at risk of, but with no signs of, dementia found consistent relations between cognitive function and LZ status (37–42). Of course, for the result to be valid even for this highly selective sample, the measures of cognitive function and decline must also be valid.

Measuring Cognition

Cognition is a relatively wide set of mental abilities that include processes related to memory, knowledge, problem solving, judgment, evaluation, comprehension, language, etc. There is an equally vast array of methods for measuring cognitive function, which range from atomistic assessments of cognitive fundamentals, such as processing speed, to complex questionnaire assessments of executive function, to functional neuroimaging of cognitive tasks. These tests are all designed to illuminate different aspects of cognition and are geared toward specific groups. For example, some are simply created to provide quick screening tools for dementia (e.g., phone screens that can be used to recruit subjects for a study); others are aimed at comprehensive evaluation of healthy individuals. Although these assessments have some similarities, they are obviously not the same. Staging decline, for example, assumes “normal” is a category, and data are interpreted as the ways in which those at risk deviate from normal. Measuring “function” assumes a change (in either direction) in previous functioning that meets statistical criteria but may or may not reflect a deviation from normal.

In AREDS2, a battery of tests that are commonly used to screen for cognitive impairment (e.g., the Telephone Interview of Cognitive Status, a corollary of the Mini-Mental State Examination) were used. These assessments, however, are necessarily short. For example, in AREDS2, all of the testing, including screening for hearing impairment and depression, occurred “over a period of 30 min.” In other words, some very complex aspects of cognitive function were tested, on average, in just a few minutes over the telephone. This is obviously different from traditional cognitive testing, which is designed to be relatively exhaustive, contains scales for internal consistency, etc.

This is likely why many of the individual tests did not correlate well with the more careful testing that was done within the clinic. For example, the uncorrected clinic compared with telephone correlations were only modest for individual tests such as the Telephone Interview of Cognitive Status (r = 0.44 compared with in-clinic Mini-Mental State Examination; 95% CI: 0.40, 0.49) and the digits backward test (r = 0.35; 95% CI: 0.31, 0.40) (26). Test-retest correlations should be high, of course, because they represent, for instance, letter fluency simply predicting letter fluency. Low test-retest correlations, however, are important to consider when evaluating outcomes. It is unlikely, for instance, that diet would predict the ability to count digits backward because counting digits backward on the phone explained only 12% of the variance in counting digits backward in the clinic.

The main outcome used in AREDS2 was the composite of all the tests administered. This value, uncorrected, correlated highly with the clinic assessment (r = 0.77; 95% CI: 0.74, 0.79)—in fact, higher than all of the individual cognitive tests except for letter fluency (0.79; 95% CI: 0.77, 0.81). This higher correlation was due to adding tests into the composite that were not direct tests of cognition.

As noted, one reason why large epidemiologic studies use such coarse assessments is that they have to be relatively quick. When you are testing many thousands of subjects, more careful testing becomes prohibitive. Is it better, however, to conduct a 10-min phone interview in thousands or more extensive testing (e.g., cognitive neuroimaging) in a smaller number? The AREDS2 authors argued that “eating foods rather than taking any specific supplement may have an effect.” We agree, but of course one could never do such a study using the AREDS2 design. One goal of using large samples is to obtain results that are generalizable to a larger population. If, however, one preselects the sample (e.g., primarily white elderly individuals with visual disability), then such generalization is gone. For example, the authors noted that “it is possible that these supplements were started too late in the aging process since the mean age of the study population at baseline was 72.7 y.” These types of limitations, however, are easy to anticipate and avoid (i.e., pick younger samples). Convenience sampling is only convenient if it does not lead to erroneous conclusions. Furthermore, very large samples can make even very small effects (from a clinical perspective) significant.

Some questions do not lend themselves to such designs. AREDS2 is certainly a classic example. AREDS2 authors noted that their results provide “a more definitive result showing the effects of oral nutritional supplementation on cognition.” Perhaps a better framing of this definitive result would be that adding LC-PUFAs to the AREDS supplement did not change performance on an abbreviated test of cognition administered to well-nourished elderly patients with AMD.