Genetic Testing in Persons with Age-Related Macular Degeneration and the Use of the AREDS Supplements; To Test or Not to Test?

The controversy surrounding the use of genetic testing to guide the treatment of persons with age-related macular degeneration (AMD) continues. In 2001, the results of the Age-Related Eye Disease Study (AREDS), a placebo-controlled trial, demonstrated that oral supplementation with a combination of antioxidant vitamins and zinc reduced the risk of progression to late AMD by 25% in persons with intermediate AMD in at least one eye.¹ Klein et al evaluated the influence of the genotypes complement factor H (CFH) (Y402H, rs1061170) and LOC387715/age-related maculopathy susceptibility 2 (ARMS2) (A69S, rs10490924) on the response to treatment with AREDS supplements (combination of antioxidants and zinc), zinc alone, or antioxidants alone in 876 AREDS participants who had available DNA and who were at high risk of developing advanced AMD.² Although there was a possible interaction between CFH genotype and treatment, Klein et al concluded that the AREDS supplements were associated with a general reduction in the risk of developing late AMD in all genotype groups compared with placebo, and neither antioxidant alone nor zinc alone was superior to the antioxidant and zinc combination in any of the genetic groups examined.

Awh and his colleagues created a genetic test to evaluate CFH and ARMS2 genes, and performed retrospective analyses of AREDS subgroups (n=989).³ They claimed that treatment with the AREDS supplements should be tailored according to the patient’s genotype, suggesting the need to genotype all patients taking the AREDS supplements. The AREDS investigators compared response to treatment in individuals with different genotype configurations in a larger group of AREDS participants (n=1,237) and failed to find statistically significant differences in response to treatment with AREDS supplement.⁴ In this current issue, Awh et al have further refined their genetic subgroups based on outcome, and furthered their claim that AREDS supplements can be harmful to individuals with certain genotypes.⁵

Are these findings by Awh et al true associations or are they the result of chance, selection bias, or some other confounder? Our request for the identification codes of the AREDS participants in their analyses was turned down. Since the DNA and data used by Awh et al. originated from our AREDS dataset, we have reconstructed their sample – which represents only a subset of AREDS participants for whom genetic information is available. Based on when and how the DNA were requested, we are confident of the identification codes for 893 (90%) of the 989 participants used in their analyses, which we verified by finding similar progression rates to late AMD and similar risk ratios for each of the supplements in each of their genetic risk groups.

We agree with Awh et al, that the ultimate test of the validity of their study is a replication sample.⁵ Thus, it is fortuitous that Awh and colleagues had access to only a portion of the AREDS patients with available DNA. We were able to assemble a validation cohort from the remaining patients (n=526) and this cohort is referred here as the “residual cohort”. If the findings from Awh’s recent report are correct, the results of the analysis from this residual cohort will likely be in the same direction (either beneficial or harmful), and on average, of the same magnitude as those published by Awh et al, validating their analysis. However, if Awh’s results were generated by selection bias and not true associations, the results would be different, likely regressing to the overall mean differences observed in the AREDS primary study results.

Results

As previously published, we genotyped CFH rs412852 and rs3766405, and ARMS2 c.372_815del443ins54 in our study cohort. Figure 1 demonstrates the results of the analyses of Awh’s subgroup and the residual cohort, stratified for each of the genotypic groupings suggested by Awh et al. Striking differences are displayed in the various genotypic groups between the Awh subgroup and the residual cohort. In the genotypic group with 0 or 1 CFH risk alleles and no ARMS2 risk alleles (Figure 1a), only the antioxidants alone were beneficial in Awh’s analyses while in the residual cohort, the results showed a marked beneficial treatment effect of the AREDS supplements and a smaller beneficial effect by zinc, similar to that of the overall results of AREDS. For the group with 2 CFH risk alleles and no ARMS2 risk alleles (Figure 1b), Awh’s analysis revealed about a threefold increase in harmful effects for those assigned to either zinc or the AREDS supplements. However, the residual cohort analysis showed a beneficial effect of AREDS supplements and a general regression to the mean, rather than in the direction of the Awh’s analyses. The zinc group also regressed to the mean in the residual cohort. In the group with 0 or 1 CFH risk alleles and 1 or 2 ARMS2 risk alleles (Figure 1c), the results were similar in both studies. In those with 2 CFH risk alleles and 1 or 2 ARMS2 risk alleles (Figure 1d), while the Awh analyses barely showed a beneficial treatment effect in any of the components, the residual cohort demonstrated the beneficial effects of the AREDS supplements. In fact, for all four of Awh’s genotypic groups, the combination of antioxidants and zinc was found to be beneficial and the treatment of choice in the residual cohort.

Figure 1.

shows the Hazard Ratios for the progression to late age-related macular degeneration for each of the components of the Age-Related Eye Disease Study (AREDS) supplement: antioxidant, zinc, combination of antioxidant and zinc vs. placebo in the analyses reported by Awh et al⁵ and the results of the replication of the analyses in the residual cohort of AREDS participants. These are stratified by the genotype treatment groups defined by the Awh et al⁵ publication. Figure 1a–c are the four genotype treatment groups. Note the differences seen in Figures 1a and 1b where beneficial effect of the combination of antioxidant and zinc is greater in the analyses conducted in the residual cohort. The harmful effects of genotype treatment group 2 (figure 1b) in the Awh analyses were not replicated in the analyses conducted on the residual cohort. Figure 1e is stratified by astrological signs, using outcomes for the basis of the subgrouping. Using the Awh cohort, we found a harmful effect of zinc in those born under the birth signs of Aries and Cancer. The analysis in the residual cohort demonstrated a beneficial effect with zinc, demonstrating the important of replication in studies of genetic associations.

Discussion

How do we explain the differences in these results? The major problem relates to how the genotypic subgroups are selected. Subgroup analyses are difficult to interpret even when the subgroups are prespecified, but the subgroup definitions analyzed by Awh et al were guided by the same data used to evaluate their effectiveness. Thus, the genetic subgroups in the Awh report were not prespecified, and were chosen from nine different genetic subgroups they created based on CFH and ARMS2 genes. These nine subgroups were then organized based on rates of progression to late AMD. Because of this built-in bias, statistical testing and p-values as reported by Awh et al are difficult to interpret. Finding statistically significant differences is almost a certainty when there are multiple groups (hence multiple comparisons) and when subgroup selection is based on outcome. The fallacy of attributing associations to subgroups derived in this manner has been well described. An example that is easily understood is to divide the population into subgroups by astrological sign. One can then select those with the most beneficial treatment effects and those with the least beneficial effects, easily demonstrating “statistical significance” by comparing the two groups without accounting for multiple testing or for the lack of a pre-specified comparison. Using Awh’s cohort of patients, those born under the sign of Aries or Cancer were “harmed” by treatment with zinc (Figure 1e). However, the residual cohort which was not subjected to selection bias prior to analyses, showed no harmful effect of zinc for these astrological signs (Figure 1e).

These data strongly suggest and we believe that the conclusions from the Awh report are not correct. The results could not be replicated with the analysis using the residual cohort. Because of the multiplicity of genetic subgroups, the large number of potential comparisons, and the lack of prespecified hypothesis, it is not unusual that initial associations with outcome cannot be replicated. Early reports of genetic influence on the response to anti-vascular endothelial growth factor therapy were promising suggesting a genetic association with response to intravitreal therapy but this too did not pass the crucial test of replication.

Recommendations

The combination of antioxidants and zinc, as found in both the AREDS and AREDS2 supplements, remains the only proven beneficial formulation regardless of genotype, with no apparent indication for treatment with either antioxidants or zinc alone. Genetic testing is NOT recommended for initiating or determining the appropriateness of the AREDS formulation. One should not deprive patients of a therapy that has been proven to have significant public health impact on the basis of a statistically flawed, not replicated retrospective analysis of existing data.