Gene-by-environment interaction studies have not properly controlled for potential confounders: The problem and the (simple) solution

Abstract

Candidate gene-by-environment (G×E) interaction research tests the hypothesis that the effects of some environmental variable (e.g., childhood maltreatment) on some outcome measure (e.g., depression) depend on a particular genetic polymorphism. Because this research is inherently non-experimental, investigators have been rightly concerned that detected interactions could be driven by confounders (e.g., ethnicity, gender, age, socioeconomic status, etc.) rather than by the specified genetic or environmental variables per se. In an attempt to eliminate such alternative explanations for detected G×E interactions, investigators routinely enter the potential confounders as covariates in general linear models. However, this practice does not control for the effects these variables might have on the G×E interaction. Rather, to properly control for confounders, researchers need to enter the covariate-by-environment and the covariate-by-gene interaction terms in the same model that tests the G×E term. In this manuscript, I demonstrate this point analytically and show that the practice of improperly controlling for covariates is the norm in the G×E interaction literature to date. Thus, many alternative explanations for G×E findings that investigators had thought were eliminated have not been.

Candidate gene-by-environment interaction (G×E) studies test the hypothesis that the effect of some environmental variable (e.g., childhood maltreatment) on some outcome measure (e.g., depression) depends on a particular (“candidate”) genetic polymorphism. This research area has been a hot topic in genetics, with hundreds of publications reporting positive G×E discoveries over the last 15 years, but there has been increasing skepticism about the validity of many of these findings (1–6). This skepticism is based on a number of substantive and statistical concerns: (a) a low replication rate among attempted direct replications of G×E findings; (b) the possibility that G×E findings capitalized on chance from among many unreported analyses; (c) a publication bias toward positive findings; (d) small sample sizes that exacerbate the already low statistical power for detecting interactions (7) which, counter-intuitively, increases the false positive rate; and (e) the low prior probability that a specified environmental variable interacts with a specified candidate gene polymorphism. These concerns have led some researchers to suggest that the false positive rate (8)—the proportion of significant “discoveries” that are actually false—in the G×E literature is very high, well above the nominal type-I error rate of .05 (1, 6). In essence, skeptics are concerned that the lessons learned from high-powered genome-wide association studies, which failed to corroborate previous candidate gene findings (9–13), will apply equally to G×E findings once large genome-wide interaction studies (14) are performed. In response to such concerns, at least two journals, Behavior Genetics (15) and Journal of Abnormal Child Psychology (16), have recently published policies outlining stricter criteria that must be met before manuscripts reporting candidate gene main effects or interactions will be considered for review.

The current manuscript focuses on an additional statistical problem that appears pervasive in the G×E literature. Namely, potential confounders have not been properly controlled for in the statistical models used to test G×E effects. Typically, G×E studies enter three variables—the genetic polymorphism (e.g., using a dummy or effects coding), the environmental variable, and the product of these two variables (testing the G×E effect)—into a regression equation to predict some outcome measure. However, there are often variables such as ethnicity, gender, age, socioeconomic status, education, IQ, and so forth that investigators wish to eliminate as possible alternative explanations for any G×E finding. Investigators typically enter these variables into the regression equation as covariates to “control” for their potential confounding effects on the interaction of interest. However, while entering these covariates does control for their potentially confounding influences on the main effects of the genotype and the environment, it does nothing to control for the potential confounding influences these variables might have on the interaction term. Rather, to properly control for potential confounders, investigators need to enter all the covariate-by-environment and the covariate-by-gene interaction terms in the same model that tests the gene-by-environment interaction term. Note that all simple effects and interaction effects between the covariates and the genetic and environmental variables must be entered. So, for example, to control for ethnicity and gender, investigators need to enter six terms (ethnicity, gender, ethnicity-by-gene, ethnicity-by-environment, gender-by-gene, and gender-by-environment) along with the original terms (gene, environment, and G×E). The G×E term would then be properly adjusted for the potential confounding effects of these covariates.

This general point concerning proper covariate adjustment for interactions has been made before with respect to personality (17) and social psychological (18) research, but it does not appear to be in circulation in the genetics field, as evident from the literature review below. Here, I demonstrate this problem analytically, discuss three example studies that have not properly controlled for covariates and how the conclusions of these studies might be misleading, and show that improper control for covariates is widespread in the G×E literature.

Quantification of bias when improperly controlling for covariates in G×E studies

The quantification of the bias that occurs in the interaction term in the presence of improperly modeled covariates has been derived under simplifying assumptions by Yzerbyt, Muller, and Judd (18), and so here I merely translate their conclusions to a G×E framework and refer the interested reader to their article. For simplicity, let G_i be the effects-coded (−1, 0, +1 for the aa, Aa, and AA alleles, arbitrarily coded) genetic variable where p(a)=p(A)=.5, E_i be a normally distributed, standardized environmental variable, and C1_i be a mean centered covariate of interest (e.g., an ancestry score from a principal components analysis of the identity-by-state matrix) that is correlated (confounded) with either G_i or E_i. The substantive conclusions of what follows do not depend on these distributional assumptions, but the assumptions simplify the math. For the derivations below, let us first assume that C1_i is confounded with G_i. In a properly specified model, the dependant variable, Y_i, is therefore a function of these variables and error:

$$Y_i={\beta }_0+{\beta }_G{G}_i+{\beta }_E{E}_i+{\beta }_{C1}C{1}_i+{\beta }_{G\times E}{G}_i{E}_i+{\beta }_{C1\times E}C{1}_i{E}_i+{\epsilon }_i$$ (1)

where G_iE_i is the product of the genetic and environmental term and C1_iE_i is the product of the covariate and environmental term.

Notice that when C1_i is confounded with G_i, either G_i or C1_i might interact with E_i, and thus both the G_iE_i and the C1_iE_i term must be included in the properly specified model. This allows for the possibility that it is the covariate that interacts with the hypothesized environmental moderator rather than, or in addition to, the genetic polymorphism interacting with the hypothesized environmental moderator. If C1 is ethnicity, for example, one can imagine that individuals of a certain ethnic background are more sensitive to the environmental variable than individuals of another ethnic background. This could easily occur due to, e.g., cultural differences in reporting of environmental adversity, such that the more ‘sensitive’ ethnicity only reports environmental adversity when it is more severe and harmful. To the degree that there are genotype frequency differences between ethnicities, G_iE_i will be confounded with C1_iE_i. Alternatively, if C1 is socioeconomic status, subtle stratification not captured by self-report ethnicity, or a gene-environment correlation, may cause a relationship between the genetic polymorphism and C1, again leading to the G_iE_i term being confounded with the C1_iE_i term. In either case, the model in Equation (1) will properly control for such alternative explanations, and β_G×E will be estimated correctly in the presence of β_C1×E.

However, assume that the investigators control for the covariate in the typical way by estimating only its main effect, and fit the following model:

$$Y_i={\beta }_{0*}+{\beta }_{G*}{G}_i+{\beta }_{E*}{E}_i+{\beta }_{C1*}C{1}_i+{\beta }_{G\times E*}{G}_i{E}_i+{\epsilon }_i$$ (2)

The bias in the G×E term can then be quantified as the difference between β_G×E (the unbiased estimate from Model 1) and β_G×E* (the biased estimate from Model 2). In this case:

$${\beta }_{G\times E*}={\beta }_{G\times E}+{\beta }_{C1\times E}\frac{{\sigma }_{C1,G}}{{\sigma }_G^2}$$ (3)

and β_G×E* is biased as a function of ${\beta }_{C1\times E}\frac{{\sigma }_{C1,G}}{{\sigma }_G^2}$. Note that β_C1 does not affect the bias; controlling for the main effect of the covariate does nothing to control for the covariate’s effect on the interaction. It is therefore possible that some or all of the estimated G×E effect in a model that “controls” for only the main effect of a covariate is due to the interaction between the covariate and the environmental term rather than the G×E effect itself. A similar situation occurs if the covariate is correlated with the environmental variable and interacts with the genetic polymorphism. For example, the effect of the genetic polymorphism may depend on ethnic or socioeconomic background rather than on the hypothesized environmental moderator. In this case, β_G×E* is biased as a function of ${\beta }_{G\times C1}\frac{{\sigma }_{C1,E}}{{\sigma }_E^2}$. Thus, to properly control for all the potential ways k covariates might confound the G×E effect of interest, investigators should fit the following model:

$$Y_i={\beta }_0+{\beta }_G{G}_i+{\beta }_E{E}_i+\sum _k{\beta }_k{C}_{ki}+\sum _k{\beta }_{G\times Ck}{G}_{i}C{k}_i+\sum _k{\beta }_{Ck\times E}C{k}_i{E}_i+{\beta }_{G\times E}{G}_i{E}_i+{\epsilon }_i$$ (4)

When the signs of the ${\beta }_{C1\times E}\frac{{\sigma }_{C1,G}}{{\sigma }_G^2}$ or the ${\beta }_{G\times C1}\frac{{\sigma }_{C1,E}}{{\sigma }_E^2}$ terms are opposite the sign of the β_G×E term, properly controlling for covariates can increase power to detect true G×E interactions. However, when those terms are of the same sign as the β_G×E term, properly controlling for covariates will weaken evidence for apparent G×E interactions.

The G×E term will be biased in Model 2 when (a) the covariate is related to the genetic variable and the covariate-by-environment interaction coefficient is nonzero or (b) the covariate is related to environmental variable and the covariate-by-gene interaction coefficient is nonzero. Nevertheless, the decision of whether to include or drop covariates along with their interaction terms in a model should be based on theory, not on statistical significance. As demonstrated via simulation by Yzerbyt et al. (18), dropping non-significant covariate interaction terms can seriously inflate the type-I error rate of the G×E term. Terms that are non-significant can still share enough variance with the G×E term to change conclusions about its significance.

Finally, it should be noted that even if a G×E result ‘disappears’ after properly controlling for covariates, this does not necessarily mean that the original G×E hypothesis was wrong. For example, the genetic polymorphism might cause changes in the covariate which in turn moderates the environmental variable, in which case the covariate is a mediating mechanism by which the gene moderates the environmental variable (19). That said, this possibility applies to all models that statistically control for covariates in regression, and the traditional interpretation of ‘disappearing’ effects after controlling for a covariate is that the true causal pathway is ambiguous and alternative (confounding) explanations cannot be ruled out. That said, in some cases, a particular causal pathway can be discarded as impossible or unlikely. In such cases, investigators can be more definitive about ruling out certain hypotheses. For example, changes at a genetic polymorphism will not lead to changes in ethnicity, and so a G×E hypothesis can be safely discarded if it is mediated by an ethnicity-by-environment interaction.

Three examples of misspecified models in the G×E literature

I briefly review three highly-cited examples from the G×E literature where investigators improperly attempted to control for covariates in their regression models. The purpose is not to draw attention to these studies per se, nor to suggest that they are particularly egregious examples of this practice; as shown below, no G×E study reviewed here properly controlled for covariates. Rather, the purpose is to better illustrate the problem with examples representative of the field, and to allow the reader to gauge the plausibility (or implausibility) of alternative explanations that could have been tested had investigators properly controlled for covariates.

Kaufman et al., 2004: A gene-by-environment or an ethnicity-by-environment interaction?

Using a mixed-ethnicity sample (32% African American, 22% biracial, and 46% non-African American; n=104), Kaufman et al. (20) reported results showing that the depressogenic effect of a repeat polymorphism (short/long [s/l]) at the serotonin-transporter- linked polymorphic region (5HTTLPR) depended on childhood maltreatment and on social support. I focus first on the two-way 5HTTLPR-by-maltreatment interaction they describe. The investigators included ethnicity (the ancestral proportion score), age, and gender in the regression equation “given the relevance of these potential confounding variables in interpreting the study results,” (p. 17318). The test of the 5HTTLPR-by-maltreatment interaction was significant (p=.007), and this effect was primarily due to maltreated individuals with the s/s allele having significantly higher depression scores. However, as noted by the investigators, African Americans have a significantly higher frequency of the long repeat allele compared to non-African Americans. If, due to cultural norms, maltreated African Americans are less likely to report depression than maltreated non-African Americans, some or all of the detected G×E interaction may have been due to ethnicity moderating the effect of maltreatment. Somewhat less plausibly, it is also possible that the effect of 5HTTLPR on depression depends on ethnicity. If African Americans in the sample had different rates of maltreatment, a 5HTTLPR-by-ethnicity interaction might also have caused the apparent 5HTTLPR-by-maltreatment interaction. Because the authors failed to include the environment-by-ethnicity and gene-by-ethnicity interaction terms, these alternative explanations for their findings cannot be ruled out.

Kaufman et al. (20) reported their 5HTTLPR-by-maltreatment interaction in a model that also tested a two-way 5HTTLPR-by-social support interaction (ns) and a three-way 5HTTLPR-by-maltreatment-by-social support interaction (p=.0001). This raises two issues. First, in models testing three-way interactions, investigators must include not only all relevant two-way covariate-by-gene and covariate-by-environment interactions, but must also include all relevant three-way interactions involving the covariate. With small sample sizes, this can eat up a relatively large number of available degrees of freedom, but it is necessary if investigators wish to eliminate these covariates as explanations for their interaction results.

Second, it is difficult and potentially misleading to interpret two-way interactions in the presence of three-way interactions. In such a model, the lower-order two-way interactions become conditional interactions, and the regression betas and p-values are interpreted as the predicted two-way interactions when the other (omitted) variable is coded as 0 (21). For example, the 5HTTLRP-by-maltreatment interaction reported by Kaufman et al. (2004) is the predicted effect of this interaction when social support is at 0. Whether “0” is meaningful (e.g., the average level of social support) or not (e.g., outside the range of the data) is essential for interpreting the lower-order interactions (the exact same issue applies to “main” effects in the context of interactions). Because the authors do not mention their final coding scheme for social support, it is not possible to know whether the reported significant two-way interaction is meaningful, although in interpreting their results above, it was assumed that the authors centered social support so that the two-way 5HTTLRP-by-maltreatment effect is the interaction predicted to occur among those at average levels of social support.

Caspi et al., 2005: The effect of COMT on psychosis risk depends on adolescent cannabis use, but is cannabis the true moderator?

In a sample of 803 Caucasian individuals, Caspi et al. (22) found that adolescent-onset cannabis use interacted with a SNP in the Catechol-O-Methyltransferase (COMT) gene to significantly predict several related adult psychotic symptoms. The investigators attempted to rule out the hypothesis that early cannabis use was a gateway to using amphetamines and hallucinogens, which in turn were the true moderators of the COMT polymorphism. They did this by including amphetamine/hallucinogen usage as a covariate in the model, which unsurprisingly (see Equation 3) had little effect on the COMT-by-cannabis use interaction. However, given that there is a relationship between early cannabis use and later usage of ‘harder’ drugs (23), it is possible that the observed interaction had little to do with cannabis use, but rather was driven by or was partially mediated by a COMT-by-hallucinogen/amphetamine interaction. Similarly, Caspi et al. (22) attempted to eliminate the counter-explanation that the COMT-by-cannabis interaction was driven by conduct disorder by including conduct disorder as a covariate, which again had little effect on the interaction result. However, given the relationship between cannabis use and conduct disorder reported by the investigators, it is also possible that the observed interaction was caused by COMT effects differing by level of conduct disorder. In other words, despite attempts to show the specificity of the interaction by controlling for covariates, their findings do not provide convincing evidence that adolescent cannabis use per se moderated the effect of COMT. Finally, if COMT itself is related either to hallucinogen/amphetamine usage or to conduct disorder (due to a passive or evocative gene-environment correlation or to subtle stratification effects), then it is possible that there is no G×E interaction here at all. Rather, the effect of conduct disorder (or hallucinogen/amphetamine usage, SES, IQ, etc.) on psychosis might depend on cannabis usage, and the apparent G×E interaction may have actually been caused by a covariate-by-cannabis interaction.

Cicchetti et al., 2007: A gene-by-environment or a gender-by-environment interaction?

Using a mixed-gender (54% male) sample of 267 individuals, Cicchetti et al. (24) found that a repeat polymorphism in the X-linked monoamine oxidase A (MAOA) gene interacted with childhood maltreatment to predict depression. The investigators coded “high activity” of the gene as having more than 3.5 repeats (63% allele frequency) and “low activity” as fewer than 3.5 repeats. Because males have only one copy of the gene, coding the genetic variable for males was straight-forward, but it was unclear how to code heterozygous (high/low) females, who were therefore excluded. However, this coding strategy probably induced a relationship between MAOA activity and gender. The proportion of males was ~.63 for the high activity allele and ~.37 for the low activity allele. However, for females these proportions were ~.63²=.40 for the high activity and ~.37²=.14 for the low activity alleles (assuming Hardy-Weinberg equilibrium). Thus, females were probably over-represented in the high activity group: there were ~1.7 times more males in the high vs. low activity groups but ~2.9 times more females in the high vs. low activity groups. Investigators controlled for the main effects of gender and ethnicity, but not for their interactions with MAOA activity or childhood maltreatment. Therefore, a potential alternative to their findings is that the effects of maltreatment depend on gender, which presented itself as a MAOA-by-maltreatment interaction in their results. Last, this study also used a mixed-ethnicity sample of African Americans, European Americans, and Hispanics, and given large differences in MAOA allele frequencies between ethnicities (25), it is also possible that the observed interaction was driven by an ethnicity-by-maltreatment or an ethnicity-by-COMT interaction.

Literature review

To understand the extent of improper usage of covariates in G×E studies, I selected all (47) novel G×E studies that were identified in the Duncan & Keller (2011) review of the first ten years of candidate G×E studies in psychiatry. Novel studies (first reports of a given G×E finding) were selected because replication attempts were likely to employ the same model used in the original report, and therefore would provide redundant information about typical practices for controlling covariates. Studies were coded according to the following criteria: (a) whether they reported significant G×E findings or not; (b) whether the investigators properly controlled for covariates by including all relevant covariate-by-gene and covariate-by-environment interactions; and (c) whether the sample was ethnically heterogeneous or not. Of the 47 studies, 45 (96%) reported significant G×E results (Table 1). This high rate, when compared to the lower rate (27%) of positive results among replication attempts (not shown), is probably symptomatic of publication bias (1). As shown in Table 1, of the 41 studies that attempted to statistically control for potential confounders by including them as covariates in linear models, none used the properly specified model. Assuming that this sample of studies is representative of studies in the wider G×E literature, it is likely that almost all published G×E findings that have attempted to statistically control for covariates have done so improperly, and thus alternative explanations for these findings cannot be ruled out.

Table 1.

First Author	Year	Ref	Sig GxE?	Eth Het?	Proper Control?
Amstadter	2009	28	Y	Y	N
Aslund	2009	29	Y	Y	N/A
Bakermans-Kranenburg	2006	31	Y	U	N/A
Bau	2000	32	Y	Y	N
Bet	2009	33	Y	N	N
Binder	2008	34	Y	Y	N
Blomeyer	2008	35	Y	N	N
Bradley	2008	36	Y	Y	N
Caspi	2002	37	Y	N	N
Caspi	2003	27	Y	N	N
Caspi	2005	22	Y	N	N
Chotai	2003	38	Y	U	N
Covault	2007	39	Y	N	N
Dick	2006	26	Y	N	N
Fox	2008	40	Y	U	N
Gacek	2008	41	N	N	N
Gervai	2007	42	Y	Y	N
Grabe	2005	43	Y	N	N
Grabe	2009	44	Y	N	N
Haeffel	2008	45	Y	N	N/A
Henquet	2009	46	Y	U	N
Jokela	2007	47	Y	U	N
Jokela	2007	48	Y	U	N
Jokela	2007	49	Y	U	N
Jokela	2007	50	Y	U	N
Kahn	2003	51	Y	Y	N
Keltikangas-Järvinen	2004	52	Y	U	N
Koenen	2009	53	Y	Y	N
Lahti	2005	54	Y	U	N
Lasky-Su	2007	55	Y	U	N
Laucht	2007	56	Y	N	N
Nobile	2007	57	Y	N	N
Nobile	2009	58	Y	N	N
Ozkaragoz	2000	59	Y	N	N/A
Perroud	2008	60	Y	N	N
Racine	2009	61	N	N	N
Retz	2008	62	Y	N	N
Seeger	2004	63	Y	N	N/A
Stein	2008	64	Y	N	N
Stevens	2009	65	Y	N	N
Sun	2008	66	Y	N	N/A
Todd	2007	67	Y	Y	N
van Winkel	2008	68	Y	N	N
Vanyukov	2007	69	Y	N	N
Waldman	2007	30	Y	Y	N
Xu	2009	70	Y	N	N
Yen	2008	71	Y	N	N

Note. Sig. GxE? = whether the primary G×E hypothesis was statistically significant; Eth. Het? = whether the sample was ethnically heterogeneous [U=unknown]; Proper Control? = whether the investigators included all relevant covariate-by-gene and covariate-by-environment interactions in the model testing G×E [N/A=Not Applicable because investigators did not attempt to control for any covariates].

Because allele frequencies in the candidate genes typically investigated in G×E studies often differ across ethnicities, an ethnicity-by-environment interaction is a particularly plausible alternative explanation for G×E findings from ethnically heterogeneous samples. Of the 47 studies, 26 used an ethnically homogeneous sample, 10 used an ethnically heterogeneous sample, and 11 did not provide information about ethnicity. Most but not all of those studies that failed to provide information about the ethnic compositions of their samples were conducted in Europe and presumably used ethnically homogeneous samples. Thus, stratification is a possible alternative explanation for around one-fifth of these G×E results.

Finally, although no study included all relevant covariate-by-environment or covariate-by-gene terms to control for the effects of the covariates on G×E interactions, it should be noted that several studies conducted follow-up analyses that went at least partway toward eliminating certain covariates as alternative explanations for the G×E findings. Dick et al. (26) tested gender-by-gene and age-by-gene interactions in separate models that did not include the G×E term and found they were not significant. This procedure does make it less likely that the two covariates investigated are responsible for the reported G×E interaction. However, it did not include the covariate-by-gene or covariate-by-environment interactions in the primary model, and as noted above, even non-significant covariate interaction terms can substantively change conclusions about the interaction of interest. Caspi et al (27) stratified their sample by MAOA genotype and noted that the interaction held in each subsample. This is a highly conservative control for MAOA and essentially eliminates it as a potential confounder of their G×E finding, but the investigators did not control for their other covariate (gender) in the same way and did not control for any other potential confounders. Similarly, Amstadter et al. (28) and Ǻslund et al. (29) restricted follow-up analyses to ethnically homogeneous subsamples and found similar results to their original ones, eliminating stratification as a possible alternative explanation to their findings. However, the datasets were not similarly stratified on other covariates. Finally, the study by Waldman (30) was the only one of those investigated that included several covariate-by-gene (in this case) interaction terms in the primary model to eliminate several alternative explanations for the G×E finding, but the study failed to control for other covariates such as ethnicity in a similar way.

Conclusions

Because G×E research is inherently non-experimental (even if the environmental variable is manipulated, the genetic variable cannot be), it is essential that investigators control for potential confounders in order to eliminate alternative explanations for G×E results. Unfortunately, it appears that virtually no G×E studies to date have appropriately controlled for covariates. This is not to say that previously published G×E findings are necessarily wrong; properly controlling for confounders would not have changed conclusions in some cases and may have even strengthened them in others. However, the point is that it is unknown how often G×E conclusions would have changed with properly specified models, and this is cause for concern.

There are at least two related potential objections to the recommendation to include all relevant covariate-by-gene and covariate-by-environment interactions to models estimating a G×E term. The first has to do with overfitting: with so many terms, it may be unrealistically hopeful to obtain precise estimates of all the covariate interaction terms, especially if sample sizes are small. However, the purpose of including covariate interaction terms is not to estimate their effects per se, but rather to control for their effects on the G×E term of interest. The second potential objection is that, with a large number of interaction terms included in the model, multicolinearity may degrade evidence for the G×E term. However, this is entirely the point. To the degree interaction terms containing covariates are correlated with the G×E term, alternative explanations for the observed G×E interaction are possible. Moreover, inclusion of covariate interaction terms in a model tested on the full dataset is a much more statistically powerful approach for controlling potential confounders than is splitting the data by covariates and testing the G×E term in each subset of the data. Finally, investigators should be assuaged by the fact that if covariate interaction terms have no true relationship with the G×E term, the G×E interaction estimate typically changes very little, and is as likely to be strengthened as weakened by proper inclusion of covariate interaction terms.

The recommendations of this manuscript extend to future genome-wide interaction studies as well. For such studies, it is not sufficient to control for stratification, site, platform, and plate effects as is done in traditional (main effect) genome-wide studies. Rather, all relevant covariate-by-gene and covariate-by-environment interactions must also be included in the model to eliminate artifactual genome-wide signals that may otherwise swamp what are likely to be small true G×E signals.

In summary, G×E research has generated much excitement over the past decade. Findings from the field suggest an appealing possibility: genes are not destiny—their effects depend on environmental context. This may often be true, but to date, the field has not convincingly demonstrated that any particular G×E finding is robust. This is not only because investigators have failed to properly specify covariates in their models, but also because sample sizes have typically been small, the appropriateness of multiple testing corrections has been difficult to verify, and the unpublished ‘file drawer’ of negative findings may be large. These issues have led to an erosion of confidence in published G×E findings. This confidence will increase as investigators, reviewers, and editors acknowledge these issues and take steps to rectify them.