Evaluating candidate genes in common epilepsies and the nature of evidence

Summary

Very few genetic associations for idiopathic epilepsy have been replicated and this has tempered enthusiasm for the results of genetic studies in epilepsy. What are the reasons for lack of replication? While type 1 error, population stratification, and multiple testing have been discussed extensively, the importance of genetic heterogeneity has been relatively neglected. In the first part of this review, we explore the sources of genetic heterogeneity and their importance for epilepsy genetic studies. In the second part, we review alternatives to the simple law of replication, revisiting Bradford Hill's guidelines for evidence of causality. A coherence perspective is applied to three examples. We conclude that adopting the perspective of integrating coherent and consistent evidence from different experimental approaches is a more appropriate requirement for proceeding to functional studies.

Introduction

Over 50 genetic associations with various idiopathic epilepsy syndromes have now been reported, and the number is increasing rapidly. However, the fact that most associations have not been replicated makes the importance and credibility of any given association report difficult to evaluate. Which findings, and what kinds of findings, are most likely to be “real,” and, even among those that may be “real,” what should be the criteria for deciding which to pursue with difficult, expensive, and time-consuming biological studies?

The purpose of the current work is to explore not only the reasons for replication failure, but to suggest a framework that puts association studies in perspective. In the first section, we discuss why genetic associations in epilepsy have been replicated less often than might be expected. In the second section, we go beyond replication and examine other commonly used epidemiological criteria that could help in judging genetic evidence, and in determining which results deserve the greatest attention. We discuss specific examples in the third section.

Failure to replicate

Very few putative epilepsy susceptibility genes have been replicated, but the same holds true for other complex diseases (Ioannidis et al., 2001). Failure to replicate is most often attributed to multiple testing and type I error considerations (i.e., the belief that the original finding was a false positive) (Tan et al., 2004). While previous reviews of the subject have emphasized type I error, population stratification, and sample size as correctable methodological errors (Tan et al., 2004; Durner et al., 2006), our thesis is that genetic heterogeneity is just as important a design factor for current and future epilepsy genetic studies. If not properly addressed, then genetic heterogeneity will continue to confound study designs despite enhanced sample sizes, new technologies (e.g., genome-wide association studies), and methods for correcting for false positives (e.g., population stratification correction methods and multiple testing corrections).

Variation of risk genotype between populations

Genetic heterogeneity exists when a single disease phenotype can be caused by independent genetic loci. Genetic heterogeneity (interfamilial) is an important confounder in population-based studies of epilepsy, in both association and linkage studies (Vieland, 2001). Sometimes there are no phenotypic features to distinguish different genetic forms, but often differences that may exist remain unexplored. For example, generalized tonic–clonic seizures (GTCS) that occur on awakening may be inherited differently from GTCS that occur at random times of the day (Durner et al., 1999). These distinguishing features can be used to stratify datasets to guide gene searches. Genetic heterogeneity may occur within or between different subpopulations. For example, evidence for linkage to EJM1 in juvenile myoclonic epilepsy (JME) is limited to European origin families and not found in Hispanic origin or African-American families (Weissbecker et al., 1991; Durner et al., 1991; Sander et al., 1997; Greenberg et al., 2000). Another JME locus at 6p12 was found in families from Belize and Mexico (Liu et al., 1995) discussed later in this paper (Suzuki et al., 2004). In a third example, GTCS are linked to markers on chromosome 10 in families from India, but no such linkage peak exists in corresponding European datasets (Puranam et al., 2005). This geographic diversity of findings is increasingly recognized as the norm in international studies of complex disease, and likely reflects differing genetic selection pressures on subpopulations in different environments. In epilepsy, genetic heterogeneity also reflects the multiple contributions to the functional integrity of neural circuits that regulate cortical excitability.

Variation in risk alleles between populations (Stratification)

Differing allele frequencies and distributions between populations is one reason why a gene may exert a major effect in one population but may appear to have a much lower risk in another population, and may not even be reproduced in independent studies. For example, the risk of myocardial infarction and prostate cancer differs between European Americans and African-Americans because the same risk alleles are distributed differentially (Amundadottir et al., 2006; Helgadottir et al., 2006). The problem of population stratification in epilepsy studies is more fully discussed in a recent publication (Durner et al., 2006).

Variation in genotype because of ascertainment

The current paradigm in epilepsy genetics holds that epilepsies are channelopathies. Channelopathies were discovered in epilepsy by studying densely affected pedigrees, an ascertainment strategy that biases toward autosomal dominant forms of disease, and toward intrafamilial genetic heterogeneity, that is multiple genetic disease forms within the same pedigree (Durner et al., 1992; Pal and Greenberg, 2002). Since the common forms of epilepsy are not Mendelian, should we expect to find channelopathies being “necessary” for common idiopathic epilepsies?

Phenotypic differences in rare pedigrees, compared to the typical forms of epilepsy, again offer clues to underlying genetic heterogeneity. Such clues are found in age of onset, seizure severity, or CNS involvement. For example, autosomal dominant JME has an earlier onset than typical JME, as early as 8 or even 5 years of age (Cossette et al., 2002). Unusually high seizure frequency is seen in a pedigree with autosomal dominant rolandic epilepsy; affected individuals also exhibited speech and oral dyspraxia and cognitive deficits, out of proportion to that seen in typical rolandic epilepsy (RE) (Scheffer et al., 1995). Similarly, an autosomal recessive form of RE cosegregates with exercise-induced dystonia and writer's cramp, features that are not associated with typical RE (Guerrini et al., 1999). These are unusual phenotypic features. Their very unusualness cast some doubt on whether the genes identified in rare pedigrees play any causative role in typical forms of idiopathic epilepsy.

Implications

These examples indicate that a simple “law of replication” to discount or validate genetic association studies seems an incomplete and overly simplistic way of weighing evidence. Genetic heterogeneity, as well as population stratification, type 1 error etc, may explain why certain reported susceptibility alleles do not appear to increase risk in different subpopulations or in subtly different phenotypes. It is therefore misleading to seek “universal” associations for candidate epilepsy genes. These issues are of considerable importance not only for current association and linkage studies, but also for future genome-wide association studies (GWA), which have a requirement for large sample size (Hirschhorn and Daly, 2005). The pressure to meet these sample size criteria may compromise the capacity to assess or control for genetic heterogeneity, effectively reducing the power of the GWA approach.

Evaluating evidence

Are there other considerations, beyond simple replications of allelic association that can assist in determining which associations we take “to the next step”?

Beyond replication

Consistent replication is one of a handful of criteria that, either explicitly or implicitly, has been used to judge causal relationships in epidemiological investigations over the last half century. The medical statistician Austin Bradford Hill presented a widely used set of guidelines in 1965 (Hill, 1965). Table 1 categorizes Hill's “criteria” according to their relevance to complex disease genetics.

Table 1.

Guidelines to assess causation suggested by Bradford Hill, 1965, and their relevance for discriminating quality of evidence for association studies in complex disease genetics (Hill, 1965)

Useful discriminator	Poor discriminator	Not relevant*
Consistency	Biological plausibility	Temporal sequence
Absence of confounding	Strength of association	Biologic gradient
Coherence	Analogy
(Experimental reversibility)	Specificity

^*The concept of temporal sequence is redundant because the existence of the genome sequence always predates the onset of disease. At the simplest level, a biologic gradient predicts greater disease effect with larger doses of the risk factor. However, the validity of biologic gradient depends entirely on the genetic model, but in general terms is not a useful discriminator between true and false associations. Two more criteria, biological plausibility and analogy are often used to justify the selection of candidate genes, but they have limited utility for discriminating between true and false associations. Biological plausibility merely serves to highlight our ignorance of biology. For example, the identification of cystatin B as the susceptibility gene for Unverricht-Lundborg syndrome, a progressive myoclonic epilepsy, was greeted with disbelief in 1991 (Lehesjoki et al., 1991). Only a decade later do we understand its role in CNS apoptosis (Lieuallen et al., 2001). Analogy might be taken to refer to the analogous effect of mutations in homologous genes in other species. Although naturally occurring and experimental feline, canine and rodent genetic epilepsies have been described, they are usually monogenic and so far homologous genetic susceptibilities have not consistently been found in humans (Noebels, 2003). Lastly, the demonstration that the phenotype can be normalized by (genetically) correcting the mutation, ie experimental reversibility, could be viewed as the ultimate “proof-of-concept” for involvement in disease. However, this usually comes at a late stage of investigation.

Let us start with consistency or replication because consistency is currently the sine qua non of genetic epidemiology. While simple replication of genetic associations may be problematic, one must also remember that not all associations which one sets out to replicate are born equal. Allelic associations may, on the one hand, follow a prior linkage study, defining a small area of the genome in which we are virtually certain a disease gene exists, or on the other hand, they may be candidate gene explorations or the result of a whole genome association study. A positive allelic association found in an area of prior linkage indicates stronger evidence that the association is “real” and important than if there was no prior linkage evidence in that region of the genome (Roeder et al., 2006). Simultaneously, however, alternative explanations for replication (or lack thereof) also need to be considered and excluded, that is the presence of confounders (Table 2).

Table 2.

Evaluating genetic association studies

(a) Have the authors employed an accepted, valid definition of the phenotype adequate enough to reproduce the study?

(b) Have the authors measured the phenotype appropriately?

(c) Were suitable controls (matched at least for age, sex, race) used for comparison of allele frequency?

(d) What was the probability of replication, given the sample size of the replication study and the strength of association in the original study?

(e) Were markers of comparable location and informativeness used in this study?

(f) Was there consideration of possible genetic heterogeneity based on symptoms or race?

Genetic association studies introduce their own set of confounders and design limitations besides heterogeneity. These include the information content of the marker examined, the strength of the presumed linkage disequilibrium (LD) between the marker allele of interest and the disease allele, and population stratification (mentioned earlier). Even if all these factors are taken care of, there is still the possibility of genotyping error, an explanation given recently for retracting evidence of replication in idiopathic generalized epilepsy (IGE) (Makoff et al., 2005).

Most importantly, one must assume that two independent studies employ identical phenotypes, and that heterogeneity is similarly distributed in both samples. Thus it becomes almost impossible for readers, on the basis of strength of association alone, to judge the comparability of two studies that investigate association between a given phenotype and a given gene variant. Confounders need to be evaluated in tandem. Without the proper control of confounders, an initial association cannot be discounted.

Thus, difficulty in controlling for differences across studies makes one question the dogma of accepting only independent replication of association as proof. Yet, most genes identified in idiopathic epilepsies are at an early stage of investigation, where the field is demanding replication studies of reported associations before going forward. For many monogenic neurological disorders, the idea of seeking simple replication has been transcended by coherent evidence from different types of biological studies. For example, the involvement of MECP2 in Rett syndrome has been proven through linkage analysis, mutation analysis, expression analysis, and a mouse model (Ellison et al., 1992; Amir et al., 1999; Moretti et al., 2006) and the broader phenotype of MECP2 associated mutations subsequently probed (showing why specificity is not necessarily useful in determining the veracity of a disease-allele association). An emphasis on coherence might serve us well in genetic studies of common complex epilepsies as it has in monogenic disorders.

Coherence of evidence

Genetic studies in epilepsy have taken various experimental approaches as their starting point—linkage, association, and mutation analyses are the three most common. Linkage studies test for concordant inheritance, within pedigrees, of a phenotype (disease) and alleles at a marker locus. Linkage can be helpful in telling us that there are loci near a marker that exert a strong effect on the inheritance of a disease, and are “necessary,” if not sufficient, for disease expression (Greenberg, 1993). However, linkage is less helpful for identifying the precise genes which confer disease susceptibility in the population, or for locating loci that are neither necessary nor sufficient for disease expression, so-called susceptibility loci (Greenberg, 1993).

Association studies test for excess cooccurrence of disease and a marker (genetic variant) in a population. Linkage is often seen without association, because classic linkage analysis requires the existence of linkage equilibrium between markers, that is that there is no LD between markers. That will almost always be the case unless the marker is actually a part of the disease-causing locus or the marker is located somewhere in the genome with extensive LD (e.g., the human leukocyte antigen (HLA) region or in an LD block). Association can be detected without linkage, for example when an allele accounts for only a minor proportion of the inherited trait variance, meaning that even though the allele is more frequent in affected individuals, it does not predict disease status within a pedigree. Thus an association, even if replicated several times, does not necessarily imply that the allele variant has a strong causative role in the disease. A good example of this is the association, and absence of linkage, between HLA-DR3 and autoimmune thyroid disease (Roman et al., 1992).

Another technique is mutation screening. Mutation screening analyses can substantiate the biological plausibility of a specific gene variant in causing the disease of interest or a related phenotype (i.e., that the mutation is sufficient to cause the disease). Mutation screening, as commonly applied, does not often show the presence of mutations in all affected members of the pedigree, nor does such analysis usually show the absence of mutations in unaffected members (Haug et al., 2003). Such awkward incompatibilities are often attributed to nonpenetrance or phenocopies, but these are not biological explanations as much as statements of our ignorance of the true mechanisms. Equally, they bring into question the causative role of these mutations, especially since these mutations are not commonly found in population based samples (as opposed to rare pedigrees). In the absence of identifying a mutation or set of mutations that consistently appear in patients, it is difficult to evaluate the contribution of the gene, in which the mutation appears, to disease expression (see below for an example).

It is obvious that linkage, association, and mutation analyses can play complementary parts in establishing the causative role of a gene variant for a given phenotype. Moreover, evidence from one approach needs to be interpreted for coherence in the context of evidence from other approaches. However, even coherence has limitations imposed on it by genetic heterogeneity and other confounding factors. Thus, a mutation discovered by one group may not be confirmed by another group because the patient sample is different. These problems are abolished when different approaches are applied to the same sample. A few examples illustrate the application of the coherence perspective.

Example 1

The EFHC1 gene for JME (Suzuki et al., 2004). The linkage and mutation analysis for this locus are convincing, but association evidence is lacking. The 6p12-p11 locus was first found by linkage analysis in Mexican families, and confirmed in an independent Dutch sample (Liu et al., 1996; Bai et al., 2002; Pinto et al., 2004). Using the original data, 44 families of JME probands, linked to 6p12-p11 markers, were then screened for mutations. Seventeen genes in the linked region were excluded and the candidate gene EFHC1 was chosen for further analysis. Four different nonsynonymous mutations were found in EFHC1, but only in six of the 44 families. Within these six families, all epilepsy or EEG trait-affected individuals carried the mutation, but so did 11 unaffected individuals. These particular mutations were not observed in 382 healthy controls. EFHC1 coding mutations reduced apoptosis in mouse neurons, establishing plausibility for the epileptogenic potential of EFHC1 mutations (Suzuki et al., 2004). These mutations were not found in the Dutch sample (Pinto et al., 2006). In this example, linkage analysis within the original sample is convincing, is replicated, and is buttressed by functional studies. However, the mutation analysis leaves unanswered questions—why do unaffected family members carry the mutations while 38/44 (86%) of the families do not segregate any EFHC1 mutation? And how do the mutation data fit with the linkage data—can mutations in six families produce a significant lod score? Perhaps other mutations are responsible, and might be reproducible in the Dutch sample, or are there different mutations in Dutch and Mexican samples segregating to the same linkage region? Perhaps one way to break the deadlock of mutation evidence is to try a third approach—to demonstrate association between EFHC1 mutations, or nearby genetic marker variants, in population JME cases and controls.

Example 2

Electrophysiological experiments have suggested a role for T-type calcium channels in the maintenance of a stable thalamocortical oscillatory network, perturbations in which can lead to idiopathic generalized seizures. Several investigators have therefore sought evidence in favor of a role for calcium T-type channel mutations in IGE. Evidence for a role of CACNA1A in IGE has not been convincing (Chioza et al., 2001). The approach taken was to simply start with mutation analysis, without demonstrating a major effect of the gene on the phenotype. CACNA1A was selected as a candidate gene because mutations in its mouse homolog are associated with behavioral arrest seizures and ataxia in tottering and leaner mice (Fletcher et al., 1996). Linkage of IGE to CACNA1A loci has not been demonstrated and mutations in human CACNA1A do not lead to a seizure phenotype. Nineteen alleles from five CACNA1A markers were tested for association in 367 patients with all subtypes of IGE. One allele was significantly associated with IGE (Chioza et al., 2001). This finding has not been replicated in an independent German sample, even in IGE subtypes (Sander et al., 2002). Thus there is only a single unreplicated association, with no evidence for a major genetic effect of CACNA1A on any IGE subtype, and no biological evidence for a seizure phenotype. Given the lack of either replication or coherent support, it is perhaps more rewarding to turn attention to other members of the T-type channel family, such as CACNG3 (Everett et al., 2007).

Example 3

BRD2 is a major susceptibility gene for JME at the EJM1 locus at chromosome 6p21 (Pal et al., 2003). The EJM1 locus was discovered through whole genome linkage analysis of strictly defined JME and confirmed in three separate family collections (Greenberg et al., 1988; Durner et al., 1991; Weissbecker et al., 1991; Sander et al., 1997; Greenberg et al., 2000). Genetic heterogeneity at EJM1 has been noted, and seems to be explained by racial origin: linkage evidence is only present in European origin families (Sander et al., 1997; Greenberg et al., 2000). An association study of single nucleotide polymorphism (SNP) markers and haplotypes across the entire EJM1 region led to the identification of BRD2 as the JME susceptibility gene at the EJM1 locus (Pal et al., 2003). Population stratification was excluded by repeating the case control study with family-based association methods. Subsequently, association with BRD2 markers has been replicated in English and Irish JME samples, but not in south Indian or Australian samples (Cavalleri et al., 2007). BRD2 markers were also associated with the photoparoxysmal response in German IGE patients (Lorenz et al., 2006). BRD2 is therefore a gene with major effect on the JME phenotype, and allelic variants confer increased risk both within families and in population studies. However, mutational and/or functional analyses are still necessary to demonstrate an effect on epileptogenesis.

Conclusions

Independent replication of association remains an important criterion in judging evidence, yet the absence of replication may not necessarily invalidate an original finding. Simple replication of association, (or for that matter linkage or mutation), by an independent group, after accounting for genetic heterogeneity and all possible confounders, might be a near impossibility. However, it is still possible to prioritize candidates for functional studies in the absence of independent replication. We suggest that we adopt a perspective integrating results from different experimental methods, rather than place one over another in importance or insisting only on replication. We believe that coherence of experimental methods is a more informative approach to simple replication, because it forces us to evaluate different kinds of experimental data in the context of findings from other approaches. Because of the difficulty of replication, coherence may sometimes be limited to data from a single research group or patient sample. Difficulty in replication should not deter investigators from moving forward towards functional studies if they find coherence in their own data.