Assessing Gene-Environment Interactions for Common and Rare Variants with Binary Traits Using Gene-Trait Similarity Regression

Abstract

Accounting for gene–environment (G×E) interactions in complex trait association studies can facilitate our understanding of genetic heterogeneity under different environmental exposures, improve the ability to discover susceptible genes that exhibit little marginal effect, provide insight into the biological mechanisms of complex diseases, help to identify high-risk subgroups in the population, and uncover hidden heritability. However, significant G×E interactions can be difficult to find. The sample sizes required for sufficient power to detect association are much larger than those needed for genetic main effects, and interactions are sensitive to misspecification of the main-effects model. These issues are exacerbated when working with binary phenotypes and rare variants, which bear less information on association. In this work, we present a similarity-based regression method for evaluating G×E interactions for rare variants with binary traits. The proposed model aggregates the genetic and G×E information across markers, using genetic similarity, thus increasing the ability to detect G×E signals. The model has a random effects interpretation, which leads to robustness against main-effect misspecifications when evaluating G×E interactions. We construct score tests to examine G×E interactions and a computationally efficient EM algorithm to estimate the nuisance variance components. Using simulations and data applications, we show that the proposed method is a flexible and powerful tool to study the G×E effect in common or rare variant studies with binary traits.

HUMAN complex traits have a multifactor etiology that involves the interplay between genetic susceptibility and environmental exposures. Studies of gene–environment (G×E) interactions can facilitate our understanding of genetic heterogeneity under different environmental exposures (Kraft et al. 2007; Van Os and Rutten 2009), help to identify high-risk subgroups in the population (Murcray et al. 2009), provide insight into the biological mechanisms of complex diseases (Thomas 2010), and improve the ability to discover susceptible genes that interact with other factors but exhibit little marginal effect (Thomas 2010). However, finding significant G×E interactions is not an easy task. Model misspecification, inconsistent definitions of the environmental variable, and insufficient sample sizes are just a few of the issues that often lead to low power and nonreproducible findings in G×E studies (Mechanic et al. 2012; Jiao et al. 2013; Winham and Biernacka 2013). In particular, the sample size needed to detect a G×E effect is usually four times larger than that needed to detect a main effect of similar magnitude (Thomas 2011). Thus, researchers need a robust, powerful G×E test to generate reproducible findings.

Conventionally, researchers search for significant genetic or G×E associations, using single-SNP methods, e.g., the Kraft 2-d.f. test (Kraft et al. 2007) or the simultaneous test of Dai (Dai et al. 2012). More complex methods (e.g., Mukherjee and Chatterjee 2008; Murcray et al. 2009; Sohns et al. 2013) aim to retain the advantages from both the case-only test (high power but sensitive to G–E correlations) and the standard case–control G×E test (low power but robust to G–E correlations). Despite the many efforts to improve single-SNP G×E tests, issues remain; e.g., a large proportion of trait heritability remains unexplained (Manolio et al. 2009) due to false positive and/or false negative findings.

Inflated false positive rates arise when the model used to screen for G×E interactions does not correctly reflect the true underlying genetic (G) and environmental (E) effects (Voorman et al. 2011; Lin et al. 2013; Wang et al. 2013). To address this issue, Voorman et al. (2011) suggested a model-robust estimate of the variance, and Lin et al. (2013) and Wang et al. (2013) suggested a random-effect model to capture the genetic main effect. The false negative (underpower) issues can be addressed by evaluating G×E effects on a set of markers, e.g., on genes, linkage disequilibrium (LD) blocks, or pathways (Tzeng et al. 2011; Lin et al. 2013). Marker–set G×E analysis can improve power by aggregating effects across markers. Such accumulation methods account for LD among markers and reduce the total number of tests to be performed. The improved power is particularly crucial for common variants with subtle individual effects and for rare variants with sparse occurrence (Sham and Cherny 2011). In addition, operating at a gene/pathway level helps increase reproducibility (Sohns et al. 2013).

Several G×E marker-set methods are available to study associations with common variants, where the major task is to avoid a large number of parameters for modeling G, E, and G×E variables. One of the first proposed G×E marker-set methods was Tukey’s 1-d.f. test (Chatterjee et al. 2006), which made significant progress toward fully understanding complex diseases. However, this method makes the often incorrect assumption that a SNP’s interaction effect is proportional to its marginal genetic effect (Winham and Biernacka 2013). Other commonly adopted G×E marker-set methods include minimum P-value (min-P) methods and weighted burden methods, where weights can be obtained from the principal components (PCs) of the SNP genotypes (Winham and Biernacka 2013) or from the G–E correlation (Jiao et al. 2013). In particular, Jiao et al. (2013) showed that the correlation between G and E can serve as an informative indicator for G×E interactions and that incorporating G–E correlations as weights can increase the signal-to-noise ratio in a G×E marker set while avoiding permutations. However, these observations are valid only when the true G–E correlation is in the same direction as the G×E interaction (Jiao et al. 2013). Fan and Lo (2013) proposed a model-free approach based on a summation of partitions to evaluate the interaction effects for rare variants. However, their method evaluates only the combined effect of G and G×E, not the separated effects. Recently, Lin et al. (2013) proposed a generalized linear mixed-effect model (GLMM) for G×E interactions for binary and continuous traits and showed it has superior power and robustness over min-P methods. A similar method, similarity regression (SimReg), proposed by Tzeng et al. (2011) to study marker-set G×E for continuous traits, was shown to be connected to linear mixed-effect models.

In this article, we extend the SimReg G×E framework established in Tzeng et al. (2011) to binary traits with common or rare variants. SimReg, which is inspired by Haseman–Elston regression for linkage analysis (Haseman and Elston 1972; Elston et al. 2000) and haplotype similarity tests for regional association (Tzeng et al. 2003; Beckmann et al. 2005), uses a regression model to correlate trait similarity with genetic similarity across multiple loci and to account for covariates. SimReg has been shown to perform well for common and rare variants (Tzeng et al. 2011). However, unlike similarity-based testing for the genetic main effect (Tzeng et al. 2009) or for G×E with quantitative traits (Tzeng et al. 2011), G×E tests with binary traits have several challenges associated with computation and estimation. In particular, G×E tests require the estimation of nuisance parameters to capture the main effects. Estimating these parameters requires high-dimensional integration and the inversion of a high-dimensional similarity matrix. For quantitative G×E tests, this estimation can be sidestepped using the normality of the phenotype, but no such useful properties exist for binary G×E tests. To overcome these challenges, Lin et al. (2013) proposed using ridge regression to estimate the nuisance main effects, selecting the tuning parameter using generalized cross validation.

In our work, we develop an EM algorithm to approximate the integration and we alleviate the computational burden of maximum-likelihood estimation (MLE) by performing a low-rank approximation of the similarity matrix. We show that the SimReg coefficient can be expressed as a variance component of a working GLMM, which facilitates the derivation of a test statistic and unifies SimReg with other random-effect-based methods (e.g., Lin et al. 2013; Wang et al. 2013). The proposed SimReg method can incorporate covariates and uses a permutation-free procedure to evaluate G×E effects. In addition, the proposed method extends the model from linear effects (e.g., Jiao et al. 2013; Lin et al. 2013) to other complex effects by selecting appropriate similarity metrics, and it avoids the need to select tuning parameters. Unlike current robust marker-set G×E methods that focus on common variant analysis, we investigate the performance of the proposed G×E strategy with rare and common variants. We evaluate the validity and power of the proposed method using simulation studies and illustrate the utility of the proposed method via two data applications: one studies the interactions between PLA2G7 and physical activity on obesity, using Cohorte Lausannoise (CoLaus) sequencing data, and a second assesses the effect modifier role of body mass index (BMI) on the association between TCF7L2 and type 2 diabetes, using the Wellcome Trust Case Control Consortium data.

Materials and Methods

Gene–trait similarity regression for G×E effects

Let $Y_i$ be the binary disease indicator for individual i ($i=1,\dots ,n$); i.e., $Y_i=1$ if individual i has the disease of interest and $Y_i=0$ otherwise. Let $G_i^m$ be the minor allele count for individual i at locus $m (m=1,\dots ,M)$, let $X_{Ei}$ be a $1\times K_E$ vector of environmental factors, and let $X_{Ci}$ be the $1\times K_C$ vector of confounders. The full covariate vector is $X_i=(1,X_{Ci},X_{Ei})$ with dimension $1\times (1+K_C+K_E)$. All covariates are standardized to have a mean of 0 and a variance of 1. For illustration, we consider the case where $K_E=1$, but it is straightforward to extend the proposed work to $K_E>1$.

We quantify the trait similarity for a pair of individuals i and j, $T_{ij}$, as the weighted sample covariance between their disease statuses; i.e., $T_{ij}=\{{\omega }_i(Y_i-{\mu }_i^0)\}\{{\omega }_j(Y_j-{\mu }_j^0)\}$, where ${\mu }_i^0=E(Y_i|X_i)$ is the subject-specific trait mean accounting for covariate $X_i$ but assuming no genetic effects and ${\omega }_i$ is a weight accounting for the fact that the $Y_i$’s have difference variances (Tzeng et al. 2009). From this definition, the expected trait similarity $E\left(T_{ij}X\right)={\omega }_i{\omega }_j\times E\{(Y_i-{\mu }_i^0)(Y_j-{\mu }_j^0)\}$ is the covariance of $Y_i$ and $Y_j$ with weights ${\omega }_i{\omega }_j$. For binary traits, we assume a logistic model, ${\mu }_i^0=e^{X_i\gamma }/(1+e^{X_i\gamma })$, where γ is the coefficient vector of the covariate $X_i$ and ${\omega }_i={\mu }_i^0\left(1-{\mu }_i^0\right)$ is the optimal weight for the logistic model (Tzeng et al. 2009).

Genetic similarity is calculated as the weighted sum of single-marker similarities; i.e., $S_{ij}={\displaystyle {\sum }_{m=1}^M{w}_{m}s(G_i^m,G_j^m),}$ where $s(G_i^m,G_j^m)$ is the genetic similarity at marker m and $w_m$ is the weight. There are several choices for $s(G_i^m,G_j^m)$ (e.g., Wessel and Schork 2006; Schaid 2010a); a popular one is the identity-by-state (IBS) metric: $s_{\text{IBS}}=2-|G_i^m-G_j^m|$. Weights $w_m$ are typically based on allele frequencies, the degree of evolutionary conservation, or the functionality of the variants (Wessel and Schork 2006; Price et al. 2010; Schaid 2010a,b). For example, one can use the minor allele frequency (MAF) of marker m, denoted by $q_m$, to up-weight similarities that are contributed by rare variants: e.g., $w_m={\left(1-q_m\right)}^{24}$ (Wu et al. 2011) can be used to target rare variants only, or a moderate weight $w_m=q_m^{-3/4}$ (Pongpanich et al. 2012) can be used to promote similarities attributed to rare alleles while retaining the contributions from common variants.

The proposed G×E gene–trait similarity regression model is

$$E\left(T_{ij}|X,S\right)=a+b\times X_{Ei}{X}_{Ej}+c\times S_{ij}+d\times S_{ij}\times X_{Ei}{X}_{Ej},i\ne j.$$ (1)

Because $T_{ij}$ incorporates baseline covariate information, model (1) does not contain an intercept or an $X_{Ei}{X}_{Ej}$ interaction covariate term (i.e., $a=b=0)$ (Tzeng et al. 2011). Using model (1), one can assess the G×E interaction by testing $H_0^{GE}:d=0$, or one can perform a joint test for the genetic main effect and G×E interactions simultaneously by testing $H_0^{\text{Joint}}:c=d=0$. The joint test is recommended if either the genetic heterogeneity or the G×E interaction mechanism is unknown (Kraft et al. 2007; Tzeng et al. 2011).

Score test for G×E effects and joint effects

Following a similar procedure to that found in Tzeng et al. (2009), we connect the similarity regression to a working GLMM to derive the score test. Consider the following GLMM,

$$g(\mu )=X\gamma +h_G+h_{GE},$$ (2)

where $\mu =\left({\mu }_1,\dots ,{\mu }_n\right)$ is a vector of conditional means ${\mu }_i=E\left(Y_i|X,h_G,h_{GE}\right)$ and $g(.)$ is a link function. Here, we consider a logit link $\text{g}({\mu }_i)=\log \{{\mu }_i/(1-{\mu }_i)\}$.. Vectors $h_{G\left(n\times 1\right)}=\left(h_{G1},\mathrm{...},h_{Gn}\right)$ and $h_{GE\left(n\times 1\right)}=\left(h_{GE1},\mathrm{...},h_{GEn}\right)$ contain the subject-specific genetic main effect and G×E interaction, respectively. Assume $h_G$ and $h_{GE}$ are random effects; i.e., $h_G\sim N\left(0,{\tau }_G{S}_G\right)$ and $h_{GE}\sim N\left(0,{\tau }_{GE}{S}_{GE}\right)$ with $S_G=\left\{S_{ij}\right\}$, $S_{GE}=D{S}_{G}D$, and $D=\text{diag}\left\{X_{Ei}\right\}$. Then, the marginal covariance of $Y_i$ and $Y_j$ in this working model is

$$\text{cov}\left(Y_i,Y_j\right)\approx {\left\{g^{\prime }\left({\mu }_i^0\right)g^{\prime }\left({\mu }_j^0\right)\right\}}^{-1}\times \left\{{\tau }_G{S}_{ij}+{\tau }_{GE}{X}_{Ei}{X}_{Ej}{S}_{ij}\right\},$$

where $g^{\prime }\left(\mu \right)=\partial g\left(\mu \right)/\partial \mu$ (see Appendix A). Recall the expected trait similarity is $E\left(T_{ij}|X\right)={\omega }_i{\omega }_j\times \text{cov}(Y_i,Y_j)$. Therefore,

$$E\left(T_{ij}|X\right)\approx {\omega }_i{\omega }_j\times {\left\{g^{\prime }\left({\mu }_i^0\right)g^{\prime }\left({\mu }_j^0\right)\right\}}^{-1}\times \left\{{\tau }_G\times S_{ij}+{\tau }_{\mathrm{GE}}\times X_{Ei}{X}_{Ej}{S}_{ij}\right\}$$

$$={\tau }_G\times S_{ij}+{\tau }_{\mathrm{GE}}\times X_{Ei}{X}_{Ej}{S}_{ij},$$

where ${\omega }_i=g^{\prime }\left({\mu }_i^0\right)=1/{\mu }_i^0(1-{\mu }_i^0)$. In other words, we can examine $H_0^{GE}:d=0$ and $H_0^{\text{Joint}}:c=d=0$ of model (1) by testing $H_0^{GE}:{\tau }_{GE}=0$ and $H_0^{\text{Joint}}:{\tau }_G={\tau }_{GE}=0$ in model (2), respectively.

To derive the score test statistics, we rewrite model (2) as

$$g(\mu )=X\gamma +Z_{G}b+Z_{GE}{b}_{GE},$$ (3)

where $b\sim N(0,{\tau }_G{I}_{L\times L})$, $b_{GE}\sim N(0,{\tau }_{GE}{I}_{L\times L})$, L is the rank of matrix $S_G$, and $Z_G$ is a $n\times L$ matrix satisfying $Z_G{Z}_G^T=S_G$. Matrix $Z_{GE}$ is defined in the same manner as $Z_G$, and $Z_{GE}=D{Z}_G$ because $S_{GE}=D{S}_{G}D$. Following Zhang and Lin (2003), the score statistic to examine the G×E effect (i.e., testing $H_0^{GE}:{\tau }_{GE}=0)$ can be calculated as

$$\begin{array}{l}{U}_{GE}\\ =\frac{1}{2}\{{\text{(}{y}_1^W-X\text{γ})}^T{V}_1^{-1}{S}_{GE}{V}_1^{-1}\left(y_1^W-X\text{γ}\right)-tr\left(P_1{S}_{GE}\right)\}{|}_{{\tau }_G=\widehat{{\tau }_G},{\tau }_{GE}=0,\gamma =\widehat{\gamma }},\end{array}$$

where $y_1^W=X\widehat{\gamma }+Z_G\widehat{b}+{\Delta }_G(y-{\widehat{\mu }}^G)$ is the working vector in model (3) under $H_0^{GE}:{\tau }_{GE}=0;$ ${\mu }^G=E\left(YX,b\right)=g^{-1}(X\gamma +Z_{G}b);$ ${\Delta }_G=\text{diag}\{g^{\prime }({\mu }_i^{\text{G}})\}$ with $g^{\prime }\left({\mu }_i\right)=1/\{{\mu }_i(1-{\mu }_i)\}$, and ${\mu }_i^G$ is the ith entry of ${\mu }^G;$ $\widehat{{\tau }_G}$ and $\widehat{\gamma }$ are the MLEs for ${\tau }_G$ and γ under $H_0^{GE}$, respectively; $V_1=W_G^{-1}+{\tau }_G{S}_G$ with $W_G=\text{diag}\{{\mu }_i^G(1-{\mu }_i^G)\}$, and $P_1=V_1^{-1}-V_1^{-1}X{(X^T{V}_1^{-1}X)}^{-1}{X}^T{V}_1^{-1}$. As noted in the literature (Zhang and Lin 2003; Tzeng and Zhang 2007), the second term, $tr(P_1{S}_{GE})$, is the mean of the first term and its variability is small compared to the first term. Thus, we derive our test statistic using only the first term; i.e.,

$$T_{GE}=\frac{1}{2}\left\{{(y_1^W-X\text{γ})}^T{V}_1^{-1}{S}_{GE}{V}_1^{-1}\left(y_1^W-X\text{γ}\right)\right\}{|}_{{\tau }_G=\widehat{{\tau }_G},{\tau }_{GE}=0, \gamma =\widehat{\gamma }}.$$

We propose an EM algorithm in Appendix B to obtain the MLEs for ${\tau }_G$ and γ.

In a similar manner, the score statistic under $H_0^{\text{Joint}}:{\tau }_G={\tau }_{GE}=0$ can be obtained as

$$U_{\text{Joint}}=\frac{1}{2}\{{(y_0^W-X\gamma )}^T{V}_0^{-1}\left(S_{GE}+S_G\right)V_0^{-1}\left(y_0^W-X\gamma \right)$$

$$-tr[P_0\left(S_{GE}+S_G\right)]\}{|}_{{\tau }_G=0, {\tau }_{GE}=0, \gamma =\widetilde{\text{γ}}},$$

and we define the test statistic of the joint effect as

$$T_{\text{Joint}}=\frac{1}{2}\{{(y_0^W-X\gamma )}^T{V}_0^{-1}(S_{GE}+S_G)V_0^{-1}(y_0^W-X\gamma )\}{|}_{{\tau }_G=0, {\tau }_{GE}=0, \gamma =\widetilde{\text{γ}}},$$

where $y_0^W=X\tilde{\gamma }+\Delta \left(y-\widehat{{\mu }^0}\right)$ is the working vector under $H_0^{\text{joint}}:{\tau }_G={\tau }_{GE}=0$. Here, ${\mu }^0=E\left(YX\right)=g^{-1}\left(X\gamma \right)$, $V_0=W_0^{-1}$, $W_0=\text{diag}\left\{{\mu }_i^0\left(1-{\mu }_i^0\right)\right\}$, $P_0=V_0^{-1}-V_0^{-1}X{(X^T{V}_0^{-1}X)}^{-1}{X}^T{V}_0^{-1}$, and $\tilde{\gamma }$ is the MLE for γ under $H_0^{\text{Joint}}$.

We show in Appendix C that $T_{GE}$ and $T_{\text{Joint}}$ follow a weighted ${\chi }^2$-distribution asymptotically under $H_0^{GE}$and $H_0^{\text{Joint}}$, respectively. P-values can then be calculated numerically using moment-matching approximations (Duchesne and Lafaye de Micheaux 2010).

Low-rank approximation of ${\mathit{S}}_{\mathit{G}}$ for computational and statistical efficiency

The calculation of the G×E test statistic involves the inversion of matrices $V_1$ and $S_{GE}$, both of dimension $n\times n$. When n is large (e.g., >5k), direct inversion of these matrices can be computationally intensive, and the inversion must be performed at every EM iteration to obtain main-effect term b (see Appendix B). To reduce the computational intensity and to facilitate the inversion of these matrices, we consider a low-rank approximation of $S_G$. The low-rank approximation has been used in the literature to improve power when the number of markers increases and when more noise is incorporated into $S_G$ (Cai et al. 2011). Previous works (Cai et al. 2011; Tzeng and Zhang 2007; Tzeng et al. 2011) indicate that $S_G$ is a positive semidefinite matrix, for which there are a few dominant eigenvalues. Assume that${\lambda }_1\ge {\lambda }_2\ge \cdots \ge {\lambda }_{\tilde{L}},\tilde{L}\le L$, are the leading eigenvalues that explain the majority of the variance of $S_G$ [i.e., ${\Sigma }_{\ell \text{=1}}^{\widehat{L}}{\text{λ}}_{\ell }/{\Sigma }_{\ell =1}^L{\text{λ}}_{\ell }\ge p$ for some $p\in (0,1]$] and have corresponding eigenvectors $e_1,e_2,\cdots ,e_{\tilde{L}}$. Then, we approximate $Z_G$ by $\widetilde{Z_G}\equiv \left[\sqrt{{\lambda }_1}{e}_1,\dots ,\sqrt{{\lambda }_{\tilde{L}}}{e}_{\tilde{L}}\right]$. For an appropriate choice of p (e.g., $p=0.90~0.99)$, $\widetilde{S_G}=\widetilde{Z_G}{\widetilde{Z_G}}^T$ contains most of the information from $S_G$. Especially with rare variant data, $\tilde{L}$ is usually $<L$, and the computation is more straightforward.

Miao (2009) indicated that the potential bias caused by a low-rank approximation can be minimized if a high percentage of the variation of $S_G$ can be retained. In our explorations, we found that selecting too small a p did not affect the test size but did lead to power loss because too much genetic information is discarded. We also found that the power loss with a large p (e.g., p = 0.99) was negligible but could stabilize the numerical calculation and boost computational efficiency. The improvement when $p=0.99$ occurs because $S_G$ has many eigenvalues that are near zero. Using a p slightly <1 removes a large number of near-zero eigenvalues, which stabilizes the numerical computations, shortens the computational time, and yields a type I error rate close to the nominal level (Table 1).

Table 1. Type I error rates of SimReg tests with vs. without low-rank approximation in rare-variant (RV) simulations.

% variance retained in ${\mathit{S}}_{\mathit{G}}$ (denoted by p)	Case–control sampling	Random sampling
Joint test $(a_G,a_{GE})=(0,0)$
p = 100%	0.052 (0.0070)a	0.025 (0.0049)
p = 99%	0.052 (0.0070)	0.052 (0.0070)
G×E test $(a_G,a_{GE})=(0,0)$
p = 100%	0.036 (0.0059)	0.024 (0.0048)
p = 99%	0.047 (0.0067)	0.043 (0.0064)
G×E test $(a_G,a_{GE})=(0.02,0)$
p = 99% with 20 causal G SNPs	0.064 (0.0077)	0.046 (0.0066)
p = 99% with 40 causal G SNPs	0.049 (0.0068)	0.046 (0.0066)
p = 99% with 60 causal G SNPs	0.045 (0.0066)	0.043 (0.0064)
p = 99% with 80 causal G SNPs	0.050 (0.0069)	0.042 (0.0063)
p = 99% with 100 causal G SNPs	0.041 (0.0063)	0.045 (0.0066)

The corresponding standard errors (SEs) are shown in parentheses. The values in italics are those whose 95% confidence intervals (i.e., rate $\pm 1.96\times \text{SE}$) fall below the nominal level. The results are based on 1000 replications. $a_G$ and $a_{GE}$ are the group PARs of the genetic main effect and the G×E effect, respectively.

^aStandard errors of the type I error rates.

Simulation studies

To investigate the performance of the proposed SimReg G×E method, we conducted simulation studies. The first simulation focuses on rare-variant (RV) analysis using sequence data, and the second simulation focuses on common-variant (CV) analysis using HapMap data. The simulation data and code are available from the Dryad Digital Repository (http://datadryad.org/) at http://doi.org/10.5061/dryad.742gv (i.e., Dryad data identifier:doi:10.5061/dryad.742gv).

RV simulations:

We obtained 10,000 haplotypes for a 1-Mb region simulated by COSI (Schaffner et al. 2005) according to a coalescent model where the LD pattern and population history mimicked those of the European population. We selected the first 100 rare loci [i.e., minor allele frequency (MAF) <5%] for further analyses. We randomly drew 2 haplotypes with replacement from the 10,000 to form each subject’s genotype. We generated the binary phenotype from a Bernoulli $({\pi }_i)$ distribution, where ${\pi }_i=e^{{\eta }_i}/\left(1+e^{{\eta }_i}\right)$, ${\eta }_{\mathit{i}}={\gamma }_0+{\mathit{X}}_{\mathit{Ei}}{\gamma }_{\mathit{E}}+{\displaystyle {\sum }_{\mathit{r}=1}^R{\mathit{G}}_{\mathit{ri}}{\gamma }_{\mathit{G}}^{\mathit{r}}}+{\displaystyle {\sum }_{\mathit{r}=1}^R{\mathit{G}}_{\mathit{ri}}{\mathit{X}}_{\mathit{Ei}}{\gamma }_{\mathit{GE}}^{\mathit{r}}}$, R is the number of causal loci, and $G_{ri}$ is the number of rare alleles at causal locus r, $1\le r\le R$. While we varied the value of R, we controlled the population attributable risk (PAR) at $a_G$ and $a_{GE}$ for the genetic main effect and G×E effect, respectively (Madsen and Browning 2009). Given $a_G$, $a_{GE}$, and R, we calculate ${\gamma }_G^r$ and ${\gamma }_{GE}^r$ using ${\gamma }_G^r=\log \left\{\left(a_G/R\right)/\left((1-a_G/R)\times q_r\right)+1\right\}$ and ${\gamma }_{GxE}^r=\log \left\{\left(a_{GE}/R\right)/\left((1-{\alpha }_{GE}/R)\times q_r\right)+1\right\}$ (Madsen and Browning 2009), where $r=1,\mathrm{...},R$, and $q_r$ is the MAF for the rth locus based on the 10,000 haplotypes. We considered both case–control sampling with 750 cases and 750 controls and random sampling with sample size 1500 and prevalence rate 0.3.

In the type I error analysis, we set $(a_G,a_{GE})=(0,0)$ for the joint test and considered $(a_G,a_{GE})=(0,0)$ and $\left(0.02,0\right)$ for the G×E test. Because the burden-based tests are sensitive to the misspecification of the main-effect model (Voorman et al. 2011), we set a weak main-effect PAR so that the burden-based tests can still serve as a valid benchmark. We performed 10,000 replicates per scenario. In the power analysis, we set $(a_G,a_{GE})=(0.02,0.1)$ for both the G×E test and the joint test and considered $R=20, 40, 60, 80$, and 100. We performed 500 replicates per scenario. In all analyses, the 100 loci were included in the association tests.

SimReg’s performance was compared to GESAT (Lin et al. 2013) and a burden-based G×E test. GESAT is a GLMM-based G×E test that is closely connected to SimReg: from the GLMM representation in model (2), we see that SimReg assumes $h_{GE}\sim N\left(0, {\tau }_{GE}{S}_{GE}\right)$, where $S_{GE}$ (calculated through the similarity kernel) determines how the G×E effects are modeled. In contrast, GESAT assumes a linear effect on $h_{GE}$, i.e., $h_{GE}=X_{GE}{\beta }_{GE}$ with ${\beta }_{GE}\sim N\left(0, {\tau }_{GE}I\right)$, which is equivalent to setting $S_{GE}=X_{GE}{X}_{GE}^T$ (i.e., a linear kernel with $w_m=1$).

For SimReg, we used the weighted IBS kernel with weight $w_m={\left(1-q_m\right)}^{24}$. For GESAT, we used R code provided by the authors with the default settings to perform G×E tests (the code does not support joint tests). For the burden-based G×E test, we first summarize the marker-set information of subject i, using the number of rare variants in the set, referred to as mutation burden. Then, we fit a logistic model, $\text{logit}P(Y_i=1|X_i,G)={\beta }_0+X_{Ei}{\beta }_E+\widetilde{G_i}{\beta }_G+\widetilde{G_i}{X}_{Ei}{\beta }_{GE}$, where $\widetilde{G_i}$ is the mutation burden for subject i. Under this model, the G×E effect can be detected by testing $H_0:{\beta }_{GE}=0$, and the joint effect can be detected by testing $H_0:{\beta }_G={\beta }_{GE}=0$.

CV simulations:

We obtained 234 phased haplotypes of gene TCF7L2 from chromosome 10 of the Utah residents with ancestry from northern and western Europe (CEU) samples in HapMap 3. We focused our analysis on the 29 typed SNPs genotyped in the Wellcome Trust Case Control Consortium (WTCCC) analysis (Wellcome Trust Case Control Consortium 2007). The MAFs of these 29 SNPs ranged from 0.0085 to 0.48. We randomly drew 2 haplotypes with replacement from the 234 phased haplotypes to form an individual genotype. We assumed that 2 of the 29 SNPs were causal and simulated the binary phenotype of individual i from a Bernoulli $({\pi }_i)$ distribution, where ${\pi }_i=e^{{\eta }_i}/\left(1+e^{{\eta }_i}\right)$, ${\eta }_i={\gamma }_0+X_{Ei}{\gamma }_E+G_i^1{\gamma }_G^1+G_i^2{\gamma }_G^2+G_{1i}{X}_{Ei}{\gamma }_{GE}^1+G_{2i}{X}_{Ei}{\gamma }_{GE}^2$, and $G_i^r$ is the number of minor alleles at the causal locus $r=1,2$. We generated the ith individual’s environmental covariate, $X_{Ei}$, from a $N(0,6)$ distribution and set ${\gamma }_0=-2.5$, ${\gamma }_E=\text{log}(1.5)=0.4055$. As in the RV simulations, we considered case–control sampling (with 750 cases and 750 controls) and random sampling (with sample size 1500 and prevalence rate 0.3).

In the type I error analysis, we set ${\gamma }_{GE}={\gamma }_G^1={\gamma }_G^2=0$ for the joint test. For the G×E test, we set ${\gamma }_{GE}=0$ and considered ${\gamma }_G^1={\gamma }_G^2=0$ and ${\gamma }_G^1={\gamma }_G^2=1/2\times \log \left(1.2\right)=0.0912$. We considered five pairs of causal SNPs (i.e., ${\gamma }_G^r>0$) with different MAFs as shown in Table 3. We performed 1000 replicates per scenario. In the power analysis, we set ${\gamma }_G^1={\gamma }_G^2=1/2\times \log \left(1.2\right)=0.0912$ and ${\gamma }_{GE}^1={\gamma }_{GE}^2=1/2\times \log \left(1.055\right)=0.0268$ for both the G×E test and the joint test. We considered all possible pairs of causal SNPs for a total of $\left(\begin{array}{c}29\\ 2\end{array}\right)$ = 406 scenarios. We performed 100 replicates per scenario. To mimic the typical CV analysis, we excluded the 2 causal SNPs and analyzed the other 27 SNPs only in the association tests. For SimReg, we set the locus-specific weight $w_m=1$. We compared the proposed SimReg method to GESAT and the single-SNP minimum P-value method (referred to as min-P). For the min-P method, we fitted the model $\text{logit}P(Y_i=1|X_i,G)={\delta }_0+X_{Ei}{\delta }_E+G_i^m{\delta }_G+G_i^m{X}_{Ei}{\delta }_{GE}$ for each SNP m to obtain the P-values of the G×E test (i.e., testing $H_0:{\delta }_{GE}=0$) and the joint test (i.e., testing $H_0:{\delta }_{GE}={\delta }_G=0$). For a given test (e.g., the G×E test), we took the minimum of the 27 G×E P-values and calculated the adjusted P-value as $1-{\left\{1-\text{min }P\text{-value}\right\}}^{k_{\text{eff}}}$, where $k_{\text{eff}}$ is the effective number of independent tests obtained using the method of Moskvina and Schmidt (2008).

Table 3. Type I error rates of the G×E test and the joint test for common-variant (CV) simulations.

Effect size considered	MAFs of the causal SNPs	SimReg	min-P	GESAT
Joint test $({\gamma }_G^r={\gamma }_{GE}^r=0)$	NA	0.044 (0.0065)a	0.060 (0.0075)	NA
G×E test $({\gamma }_{GE}^r=0)$
${\gamma }_G^r=0$	NA	0.037 (0.0060)	0.054 (0.0072)	0.036 (0.0059)
${\gamma }_G^r=0.0912$	0.009, 0.094	0.042 (0.0068)	0.040 (0.0062)	0.053 (0.0071)
${\gamma }_G^r=0.0912$	0.009, 0.1966	0.040 (0.0062)	0.043 (0.0064)	0.055 (0.0072)
${\gamma }_G^r=0.0912$	0.094, 0.1966	0.040 (0.0062)	0.045 (0.0066)	0.070 (0.0081)
${\gamma }_G^r=0.0912$	0.1966, 0.2222	0.047 (0.0067)	0.044 (0.0065)	0.051 (0.0070)
${\gamma }_G^r=0.0912$	0.2991, 0.4188	0.049 (0.0068)	0.050 (0.0069)	0.054 (0.0072)

The corresponding standard errors (SEs) are shown in parentheses. The values in italics/boldface type are those whose 95% confidence intervals (i.e., rate ± 1.96 × SE) fall below/above the nominal level. The results were obtained based on 1000 replications, and ${\gamma }_G^r$ and ${\gamma }_{GE}^r$ are the effect sizes of the causal SNPs for the main effect and the G×E effect, respectively.

^aStandard errors of the type I error rates.

Results and Discussion

Simulation studies

Results of type I error analyses (Table 1, Table 2, and Table 3):

Table 2. Type I error rates of the G×E test and the joint test for rare-variant (RV) simulations.

Nominal level	SimRega	Burden-based	GESAT
Joint test $(a_G,a_{GE})=(0,0)$
0.05	0.0504 (0.0022)b	0.0511 (0.0022)	NA
0.01	0.0093 (0.0010)	0.0110 (0.0010)	NA
0.005	0.0047 (0.0007)	0.0056 (0.0007)	NA
0.001	0.0010 (0.0003)	0.0011 (0.0003)	NA
G×E test $(a_G,a_{GE})=(0,0)$
0.05	0.0496 (0.0022)	0.0523 (0.0022)	0.05090 (0.0024)
0.01	0.0085 (0.0009)	0.0104 (0.0010)	0.0119 (0.0011)
0.005	0.0038 (0.0006)	0.0044 (0.0007)	0.0050 (0.0007)
0.001	0.0007 (0.0026)	0.0008 (0.0003)	0.0007 (0.0003)
G×E test $(a_G,a_{GE})=\left(0.02,0\right)$c
0.05	0.0473 (0.0021)	0.0482 (0.0021)	0.0602 (0.0024)
0.01	0.0099 (0.0010)	0.0112 (0.0011)	0.0119 (0.0011)
0.005	0.0052 (0.0007)	0.0055 (0.0007)	0.0062 (0.0008)
0.001	0.0014 (0.0004)	0.0010 (0.0003)	0.0009 (0.0003)

Data were generated using a case–control design. The corresponding standard errors (SEs) are shown in parentheses. The values in italics/boldface type are those whose 95% confidence intervals (i.e., rate ± 1.96 × SE) fall below/above the nominal level. $a_G$ and $a_{GE}$ are the group PARs of the genetic main effect and the G×E effect, respectively. The results were obtained based on 10,000 replications.

^aUsing p (the proportion of variation explained by the leading eigenvalues in $S_G$) = 0.99.

^bStandard errors of the type I error rates.

^cAssuming 40 SNPs with causal main (G) effect.

The type I errors for the G×E test and the joint test are shown in Table 1 and Table 2 for RV simulations and Table 3 for CV simulations. From Table 1, we see that SimReg can have conservative type I errors when using P = 100%, which can be alleviated by using P = 99%. Table 2 shows that SimReg, burden-based, and GESAT methods all have type I error rates around the nominal level in RV analyses. Table 3 shows that SimReg, min-P, and GESAT all have type I error rates around the nominal level in the CV analyses.

Results of RV power analyses (Figure 1):

Figure 1.

Power of G×E and joint tests for rare-variant simulations. The powers of SimReg, burden-based, and GESAT tests are represented by the solid (─), dashed (- - -), and dotted (⋯) lines, respectively. GESAT is performed only under case–control studies. The results were obtained based on 500 replicates.

The power results for a main-effect group PAR ($a_G)$ of 0.02 and a G×E group PAR ($a_{GE}$) of 0.1 are shown in Figure 1. For the G×E tests and the joint tests, SimReg has higher power than the burden-based test and GESAT (G×E test only) across different numbers of causal SNPs and different study designs. GESAT has the lowest power for the G×E test. Because we assumed a linear G×E effect in the simulation, the power loss may be attributable to the unweighted similarity (i.e., $w_m=1)$, which resulted in an overall similarity score dominated by less-frequent over rare variants and led to little variations among individual pairs.

We note that for both the SimReg and burden-based tests, the power of the joint test is slightly less than the power of the G×E test. It is likely that this is caused by the weak main-effect signal in the simulation: the majority of the simulated data sets had significant G×E effects but negligible genetic main effects. Consequently, compared to the G×E test statistic, the joint test statistic may have incorporated additional noise from the G test statistic, which can result in power loss. We also observe that the power loss in the joint test appears to be larger for SimReg than for the burden-based tests because the degrees of freedom (d.f.) of a SimReg test spent on the G effect tend to be higher than those of a burden-based test. However, the power of SimReg is still higher than that of the burden-based test, and the additional d.f. consumed by SimReg (compared to the burden-based test) ensure robustness against between-locus etiological heterogeneities (Pongpanich et al. 2012) as well as against model misspecifications.

Results of CV power analyses (Figure 2):

Figure 2.

Power of G×E and joint tests for common-variant simulations. The side-by-side boxplots show the powers of the proposed SimReg method, the minimum P-value method, and GESAT. A total of $\left(\begin{array}{c}29\\ 2\end{array}\right)=406$ scenarios were considered (i.e., letting each SNP pair of the 29 SNPs be causal), with 100 replicates per scenario. The 406 scenarios were classified into three groups based on the LD pattern between the 2 causal SNPs and the remaining 27 SNPs. The red “X” in each boxplot represents the average power for each LD group.

To present the power results of the $\left(\begin{array}{c}29\\ 2\end{array}\right)$ = 406 scenarios, we grouped the scenarios into three categories based on the LD structure between the causal SNPs and the analyzed SNPs. The three LD groups, i.e., the lower one-third (low LD), the middle one-third (medium LD), and the top one-third (high LD), are defined based on the average of 54 $R^2$ values, where each value is the $R^2$ between a causal SNP (2 in total) and an analyzed SNP (27 in total). We present side-by-side boxplots of the power of SimReg, min-P, and GESAT (for G×E tests) as well as the mean power value in Figure 2. We observe that when the LD is lower, the power of all methods is lower. This is expected because under low-LD scenarios the markers contain less information about the 2 causal loci. For the G×E test (Figure 2, top), SimReg and GESAT have very similar power, as expected because both methods set $w_m=1$. The powers of SimReg and min-P are similar when LD is low. As the LD increases, SimReg starts to have power improvement over min-P. The difference becomes more obvious when LD is high. For the joint test (Figure 2, bottom), the relative power of SimReg vs. min-P is similar to what was observed for the G×E tests. Furthermore, the relative performance between SimReg and min-P for binary traits is similar to what was observed for quantitative traits (Tzeng et al. 2011).

Data Applications

Analysis of gene-by-physical activity effect on obesity, using CoLaus samples:

We used Sanger sequence data of the PLA2G7 gene for 1961 subjects from the CoLaus (Song et al. 2012) and studied PLA2G7’s association with the levels of lipoprotein-associated phospholipase A2 (Lp-PLA2). The CoLaus study of Firmann et al. (2008) is a population-based study to assess the risk factors of cardiovascular disease (CVD) in Caucasian residents of Lausanne, Switzerland aged 35–75 years. PLA2G7 encodes Lp-PLA2, and the elevated plasma levels of Lp-PLA2 activity have been shown to be associated with increased risk of coronary heart disease (Thompson et al. 2010). We imputed sporadic missing genotypes, using the MaCH software package (Li et al. 2010), and obtained a total of 100 SNPs with MAF < 0.05 (range from 0.000255 to 0.029).

The genetic influence of PLA2G7 on the body mass affected by exercise has been reported in the literature (Wootton et al. 2007; Detopoulou et al. 2009). The potential modulating effect of PLA2G7 on arachidonic acid was hypothesized to be related to the association between the PLA2G7 variants and a reduced risk of coronary artery disease (Ninio et al. 2004; Wootton et al. 2007). Using PLA2G7 as a positive control, we investigated the potential interaction between physical activity and genetic variants on BMI. We defined obesity as BMI > 30 and evaluated the effects of PLA2G7 (G), physical activity (E), and G×E interactions on obesity. We considered three methods: SimReg, GESAT, and the burden-based test. In all analyses, we adjusted for age, sex, ethnic background (five PCs), smoking status, and alcohol consumption. For SimReg, we used weight $w_m={\left(1-q_m\right)}^{24}$ and a low-rank approximation with $\text{p}=0.99;$ the resulting P-values of the joint test and the G×E test were $1.46\times {10}^{-3}$ and $1.05\mathit{ }\times {10}^{-3}$, respectively, which suggested that PLA2G7 may affect the influence of physical activity on obesity. GESAT, which set $w_m=1$, yielded a G×E P-value of 0.637. These results are not unexpected given the simulation results; i.e., the unweighted similarity scores did not have power to detect rare variants because the contribution from rarer variants may be overwhelmed by the less rare variants during collapsing. The P-values of the burden-based tests were 0.013 for the joint test and $3.84\times {10}^{-3}$ for the G×E test, which are larger than SimReg P-values but give the same significant conclusions as SimReg. The results agree with the observation from the RV simulations that the proposed method is more powerful in detecting G×E effects.

Analysis of TCF7L2-by-BMI effect on type 2 diabetes, using WTCCC samples:

The data were obtained from the type 2 diabetes (T2D) case–control study conducted by the WTCCC (Wellcome Trust Case Control Consortium 2007). The controls were samples from the 1958 British Birth Cohort. The case samples were collected from various sites across the United Kingdom to be comparable to the controls. The genotyping was conducted on an Affymetrix 500K chip. Previous genome-wide association studies (Timpson et al. 2009) have indicated an interaction between TCF7L2 and BMI on T2D. Treating this TCF7L2×BMI effect on T2D as a true positive, we evaluated the performance of the proposed SimReg test (with weight $w_m=1$) and compared to GESAT and the min-P test.

We fitted a model where the response variable is the T2D status and the explanatory variables include the 29 SNPs in TCF7L2, BMI, TCF7L2×BMI, and sex. After applying sample and SNP quality control filters to remove substantial missing data, the data set contained 1913 cases and 1455 controls. We first performed the joint test and obtained a P-value of $1.81\times {10}^{-10}$ for SimReg and $1.39\times {10}^{-9}$ for min-P. The gene-level P-value of min-P is obtained as $1-{\left(1-\underset{1\le \ell \le 29}{\min }P{\text{-val}}_{\ell }\right)}^{K_{\text{eff}}}$, where $K_{\text{eff}}=19.8$ is the effective number of independent tests for TCF7L2 estimated by Moskvina and Schmidt (2008). The P-values of the G×E tests are $4.05\times {10}^{-5}$ for SimReg, $6.74\times {10}^{-6}$ for GESAT, and $2.72\times {10}^{-3}$ for min-P (adjusted P-value). The difference between SimReg and GESAT P-values can be attributed to the different choices of kernels (e.g., IBS kernel for SimReg vs. linear kernel for GESAT) and the different algorithm to estimate the nuisance main effects (e.g., EM algorithm vs. ridge penalization). The relatively large P-values of min-P suggest that there may be multiple moderate-effect loci in TCF7L2 contributing to the T2D risk, as opposed to a few strong-effect loci. The magnitude of the P-value difference in the joint tests was relatively small compared to the P-value difference in G×E tests, suggesting a strong main effect of TCF7L2 on T2D as shown in the literature (Helgason et al. 2007; Scott et al. 2007).

Conclusion

In this article we proposed a marker-set method based on similarity regression to examine G×E effects for binary traits and showed it is computationally feasible, powerful, and applicable to both common and rare variants. By demonstrating the equivalence of our gene-similarity regression model to a GLMM framework, we showed that SimReg is robust against model misspecification, like other random-effects-based approaches (e.g., Lin et al. 2013). However, because the structure of $S_{GE}$ is atypical, one cannot apply the general score test of GLMM as implemented in existing statistical software because it often yields invalid estimates of ${\tau }_G$ (e.g., negative values). We developed an EM algorithm to address the challenges associated with estimation and computation encountered in GLMM model fitting. The C code that implements the proposed joint and G×E tests is available at http://www4.stat.ncsu.edu/~jytzeng/software_simreg.php. We demonstrated the utility of SimReg in rare variant G×E analysis. We also found that for RVs, the low-rank approximation to the main-effect similarity matrix ($S_G$) is necessary to avoid an overconservative type I error rate.

One possible strategy to apply the proposed SimReg tests is to start with a joint test to detect the overall association induced by the G main effect or the G×E effects. A screening by joint tests may lead to increased flexibility and power to detect a signal because some genes can exhibit negligible marginal effects but strong effects among particular exposure groups (Kraft et al. 2007; Thomas 2010). If the joint test is rejected, a G×E test can then be used to identify whether the effects of the genetic variables are modified by the environmental variables.

One can view the SimReg framework as an implementation of a class of models for modeling $h_{GE}$, which includes GESAT as a special case. In SimReg, one can determine how the G×E effect is modeled by specifying a certain similarity metric, e.g., linear kernel, IBS kernel, or quadratic kernel, as well as by imposing variant-specific weights when collapsing the information across markers. If a linear kernel is used with $w_m=1$, the SimReg G×E test is equivalent to GESAT. However, one subtle difference is that SimReg uses an EM algorithm to estimate the nuisance main effects, whereas GESAT uses a penalized method. Another remark concerns the role of the variant-specific weight based on MAFs. As we observed in the numerical studies, although the unweighted similarity performed satisfactorily in CV analyses, it has little power in RV analyses. This is because the sum of unweighted similarity scores would be dominated by information from nonrare events. Consequently, when rare variants are studied, the multimarker similarity scores would exhibit little variation. The MAF-based weights in essence perform a soft thresholding to downweight or diminish the contribution of less-frequent or common variants in the multimarker similarity score.

The rationale of a collapsing analysis is to detect the amplified effects of rare variants in aggregate. Experience from main-effect testing suggests that variance component-based tests such as SimReg would have better power than burden-based tests if genetic effects vary radically across variants or if many null variants exist in the set (Pongpanich et al. 2012; Lee et al. 2014). However, the presence of many null variants can still unfavorably affect the test performance. For main-effect collapsing tests, efforts have been made to boost power when the signal sparsity is low by adaptively focusing on the subsets enriched with causal variants (e.g., Barnett 2014; Pan et al. 2014). Their extensions to G×E tests will be helpful to further optimize the power to detect G×E effects.

In this work we focused on examining the G×E interaction effect for a single environmental factor. However, a similar model involving multiple G×E interaction effects could be fitted. This method could be easily extended to test for gene–gene interaction in cases where one gene is suspected to interplay with other genes.

Appendix A: Marginal Trait Covariance $\text{cov}({\mathit{Y}}_{\mathit{i}},{\mathit{Y}}_{\mathit{j}})$

Define $h_i=h_{Gi}+h_{GEi}$. Under GLMM (2),

$$\text{cov}\left(Y_i,Y_j\right)$$

$$={\text{cov}}_h\left\{E(Y_i\right|X,h),E(Y_j|X,h)\}+E_h\{\text{cov}(Y_i,Y_j|X,h)\}$$

$$={\text{cov}}_h\left\{E(Y_i|X,h),E(Y_j|X,h)\right\}$$

$$(\because \hspace{0.17em}\text{conditional independence of}\hspace{0.17em}{Y}_i\hspace{0.17em}\text{and}\hspace{0.17em}{Y}_j)$$

$$={\text{cov}}_h\left\{g^{-1}\left(X_i\gamma +h_i\right),g^{-1}\left(X_j\gamma +h_j\right)\right\}$$

$$\approx {\text{cov}}_h\left\{\begin{array}{c}\left[g^{-1}\left(X_i\gamma +E{h}_i\right)+\left[{\frac{\partial g^{-1}\left(X_i\gamma +h_i\right)}{\partial h_i}|}_{h_i=E{h}_i}\right]\left(h_i-E{h}_i\right)\right],\\ \left[g^{-1}\left(X_j\gamma +E{h}_j\right)+\left[{\frac{\partial g^{-1}\left(X_j\gamma +h_j\right)}{\partial h_j}|}_{h_j=E{h}_j}\right](h_j-E{h}_j)\right]\end{array}\right\}$$

[by taking the first-order Taylor expansion of $g^{-1}\left(X_i\gamma +h_i\right)$ with respect to $h_i$ around $E{h}_i=0$]

$$\begin{array}{l}={\text{cov}}_h\{\left[g^{-1}\left(X_i\gamma \right)+\left[{\frac{\partial g^{-1}\left(X_i\gamma +h_i\right)}{\partial h_i}|}_{h_i=0}\right]\times h_i\right],\\ \left[g^{-1}\left(X_j\gamma \right)+\left[{\frac{\partial g^{-1}\left(X_j\gamma +h_j\right)}{\partial h_j}|}_{h_j=0}\right]\times h_j\right]\}\end{array}$$

$$=\left[{\frac{\partial g^{-1}\left(X_i\gamma +h_i\right)}{\partial h_i}|}_{h_i=0}\right]\times \left[{\frac{\partial g^{-1}\left(X_j\gamma +h_j\right)}{\partial h_j}|}_{h_j=0}\right]\times \text{cov}\left(h_i,h_j\right)$$

$$=\left[{\frac{\partial g^{-1}\left(X_i\gamma +h_i\right)}{\partial h_i}|}_{h_i=0}\right]\times \left[{\frac{\partial g^{-1}\left(X_j\gamma +h_j\right)}{\partial h_j}|}_{h_j=0}\right]\times \text{cov}\left(h_{Gi}+h_{GEi},h_{Gj}+h_{GEj}\right)$$

$$={\left\{\frac{\partial g({\mu }_i^0)}{{\mu }_i^0}\right\}}^{-1}\times {\left\{\frac{\partial g({\mu }_j^0)}{{\mu }_j^0}\right\}}^{-1}\times \left\{cov(h_{Gi},h_{Gj})+cov(h_{GEi},h_{GEj})\right\}$$

$$={\left\{g^{\prime }\left({\mu }_i^0\right)g^{\prime }\left({\mu }_j^0\right)\right\}}^{-1}\times \left\{{\tau }_G{S}_{ij}+{\tau }_{GE}{X}_{Ei}{X}_{Ej}{S}_{ij}\right\},$$

Appendix B: EM Algorithm to Estimate ${\mathit{\tau }}_{\mathit{G}}$ and σ in the SimReg G×E Test

Under the null hypothesis $H_0^{GE}:{\tau }_{GE}=0$, model (3) becomes $g\left(\mu \right)=X\gamma +Z_{G}b$ with $b\sim N(0,{\tau }_G{I}_{L\times L })$. Let $Y=(Y_1,\dots ,Y_n)$ be the vector of binary traits, and let $\theta =(\gamma ,{\tau }_G)$ be the parameter vector. We consider an expectation-maximization algorithm based on observed data Y and missing data b. Let $\log f(Y,b;\theta )$ be the complete data log-likelihood. In the expectation step (E-step), we compute $Q(\theta |{\theta }^{(t)})$ as

$$\begin{array}{l}Q(\theta |{\theta }^{(t)})=E\{\log f(Y,b;\theta )|Y,{\theta }^{(t)}\}\hfill \\ =E\{\log f(Y|b;\theta )|Y,{\theta }^{(t)}\}+E\{\log f(b;\theta )|Y,{\theta }^{(t)}\},\hfill \end{array}$$

because $f(Y,b;\theta )=f(Y|b;\theta )f(b;\theta )$. For the first term, we have

$$\mathit{E}\left\{\log \hspace{0.17em}\mathit{f}\left(\mathit{Y}|\mathit{b};\theta \right)|\mathit{Y};{\theta }^{\left(\mathit{t}\right)}\right\}={\displaystyle \sum _{\mathit{i}=1}^n\mathit{E}\left\{{\mathit{Y}}_{\mathit{i}}\log {\mu }_{\mathit{i}}+\left(1-{\mathit{Y}}_{\mathit{i}}\right)\log \left(1-{\mu }_{\mathit{i}}\right)|\mathit{Y},{\theta }^{\left(\mathit{t}\right)}\right\}}.$$ (B1)

For the second term, note that

$$\begin{array}{l}\log f(b;\theta )=\log f(b;{\tau }_G)\hfill \\ =\log \left\{{(2\pi )}^{-\left(L/2\right)}\right|{\tau }_G{I}_L{|}^{-\left(1/2\right)}\exp \{-\frac{1}{2}{b}^T{({\tau }_G{I}_L)}^{-1}b\}\}\hfill \\ =-\frac{L}{2}\log 2\pi -\frac{L}{2}\log {\tau }_G-\frac{b^{T}b}{2{\tau }_G},\hfill \end{array}$$

where $\left|{\tau }_G{I}_L\right|={\tau }_G^L$. Therefore,

$$\begin{array}{l}E\left\{\log f\left(b;\theta \right)\right|Y,{\theta }^{(t)}\}=E\{(-\frac{L}{2}\log 2\pi -\frac{L}{2}\log {\tau }_G-\frac{b^{T}b}{2{\tau }_G})|Y,{\theta }^{(t)}\}\hfill \\ =-\frac{L}{2}\log 2\pi -\frac{L}{2}\log {\tau }_G-\frac{E\left(b^{T}b|Y,{\theta }^{\left(t\right)}\right)}{2{\tau }_G}.\hfill \end{array}$$ (B2)

By expressing the complete-data log-likelihood in two parts, the fixed effect γ occurs only in the first term $E\{\log f(Y|b,{\theta }^{\left(t\right)})\}$ and variance component ${\tau }_G$ occurs only in the second term, $E\{\log f(b;\theta )|Y,{\theta }^{(t)}\}$.. Thus, the maximization steps for obtaining ${\widehat{{\tau }_G}}^{(t+1)}$ and ${\widehat{\gamma }}^{(t+1)}$ can be discussed separately.

Maximization step for obtaining ${\widehat{{\mathit{\tau }}_{\mathit{G}}}}^{(\mathit{t}+1)}$

To obtain ${\widehat{{\tau }_G}}^{(t+1)}$, we can focus on $E\{\log f(b;\theta )|Y,{\theta }^{(t)}\}$. We take the derivative of (B2) with respect to ${\tau }_G$ and get

$$\frac{\partial E\{\log f(b;\theta )|Y,{\theta }^{\left(t\right)}\}}{\partial {\tau }_G}=-\frac{L}{2{\tau }_G}+\frac{E(b^{T}b|Y,{\theta }^{(t)})}{2{\tau }_G^2}.$$

Setting this equal to zero, we get

$$\begin{array}{l}{\widehat{{\tau }_G}}^{(t+1)}=\frac{E(b^{T}b|Y,{\theta }^{(t)})}{L}\hfill \\ =\frac{1}{L}\left[b^{\left(t\right)T}{b}^{\left(t\right)}+\text{trace}\left({\Sigma }^{\left(t\right)}\right)\right].\hfill \end{array}$$ (B3)

Equation B3 follows because $(b|Y,{\theta }^{(t)})\sim N(b^{(t)},{\Sigma }^{(t)})$ approximately. To derive this approximation, we first reexpress $f(Y,b)$ as $f\left(Y|b\right)f(b)$, i.e., a product of a Gaussian kernel and some function of Y. Finally, because $f\left(Y,b\right)=f\left(b|Y\right)f(Y)$, we have $f\left(b|Y\right)\dot{\sim }N$. We provide the details in the next subsection.

Derivation of $\mathit{f}(\mathit{b}|\mathit{Y})$ as well as its mean ${\mathit{b}}^{(\mathit{t})}$ and variance ${\mathrm{\Sigma }}^{(\mathit{t})}$

$$\begin{array}{l}f(Y,b;{\theta }^{(t)})=f(Y|b;{\theta }^{\left(t\right)})f(b;{\theta }^{\left(t\right)})\hfill \\ ={\displaystyle \prod }_{i=1}^n\{{\mu }_i^{Y_i}{(1-{\mu }_i)}^{1-Y_i}{(2\pi )}^{-\left(L/2\right)}-{\tau }_G^{-\left(L/2\right)}\exp (-\frac{b^{T}b}{2{\tau }_G})\}\hfill \\ =\exp \{{\displaystyle \sum }_{i=1}^n[Y_i\log {\mu }_i+(1-Y_i)\log (1-{\mu }_i)]-\frac{L}{2}\log 2\pi -\frac{L}{2}\log {\tau }_G-\frac{b^{T}b}{2{\tau }_G}\}\hfill \\ =\exp \{h(b)\},\hfill \end{array}$$

where

$$\mathit{h}\left(\mathit{b}\right)={\displaystyle \sum _{\mathit{i}=1}^n\left[{\mathit{Y}}_{\mathit{i}}\log {\mu }_{\mathit{i}}+\left(1-{\mathit{Y}}_{\mathit{i}}\right)\log \left(1-{\mu }_{\mathit{i}}\right)\right]-\frac{\mathit{L}}{2}\text{log}2\pi -\frac{\mathit{L}}{2}\text{log}{\tau }_{\mathit{G}}-\frac{{\mathit{b}}^{\mathit{T}}\mathit{b}}{2{\tau }_{\mathit{G}}}}.$$ (B4)

Let $b^{(t)}$ be the value that maximizes $h(b);$ i.e., $h^{\prime }\left(b^{\left(t\right)}\right)=0$. By a Taylor expansion of $h(b)$ with respect to b around $b^{(t)}$, we have

$$\begin{array}{l}h(b)\approx h(b^{(t)})+h^{\prime }(b^{(t)})(b-b^{(t)})+\frac{1}{2}{(b-b^{(t)})}^T{h}^{″}(b^{(t)})(b-b^{(t)})\hfill \\ =h(b^{(t)})+\frac{1}{2}{(b-b^{(t)})}^T{h}^{″}(b^{(t)})(b-b^{(t)}).\hfill \end{array}$$

Therefore, the complete data log-likelihood can be approximated by

$$\begin{array}{l}f(Y,b;{\theta }^{(t)})\approx \exp \{h(b^{(t)})+\frac{1}{2}{(b-b^{(t)})}^T{h}^{″}(b^{(t)})(b-b^{(t)})\}\hfill \\ =\exp \{h(b^{(t)})\}\exp \{\frac{1}{2}{(b-b^{(t)})}^T{h}^{″}(b^{(t)})(b-b^{(t)})\}.\hfill \end{array}$$ (B5)

In Equation B5, $\exp \{-\left(1/2\right){(b-b^{(t)})}^T[-h^{″}(b^{(t)})](b-b^{(t)})\}$ is a Gaussian kernel with $-h^{″}(b^{(t)})={[{\Sigma }^{(t)}]}^{-1}$. Thus, the conditional distribution of $(b|Y;{\theta }^{(t)})$ approximately follows a multivariate normal distribution with mean vector $b^{(t)}$ and variance–covariance matrix ${\Sigma }^{(t)}={[-h^{″}(b^{(t)})]}^{-1}$.

Next we calculate $h^{\prime }(b)$ and $h^{″}\left(b\right)$. In Equation B4, we rewrite ${\mu }_i$ as ${\mu }_i\left(b\right)$ to emphasize that it is a function of b; i.e., ${\mu }_i\left(b\right)=\exp (X_i\gamma +Z_{i}b)/\left(1+\exp (X_i\gamma +Z_{i}b)\right)$ with $Z_{i(1\times L)}$, the ith row of matrix $Z_G$. Note that ${\dot{\mu }}_i{\left(b\right)}_{L\times 1}\equiv \partial {\mu }_i\left(b\right)/\partial b=Z_i^T\left(\exp (X_i\gamma +Z_{i}b)\right)/{\left\{1+\exp (X_i\gamma +Z_{i}b)\right\}}^2=Z_i^T{\mu }_i\left(b\right)\left\{1-{\mu }_i\left(b\right)\right\}$. Then

$$\begin{array}{l}{h}^{\prime }\left(b\right)=\frac{\partial h(b)}{\partial b}={\displaystyle \sum }_{i=1}^n\{Y_i\times \frac{{\dot{\mu }}_i\left(b\right)}{{\mu }_i(b)}+\left(1-Y_i\right)\times \frac{-{\dot{\mu }}_i\left(b\right)}{1-{\mu }_i(b)}\}-\frac{b}{{\tau }_G}\hfill \\ ={\displaystyle \sum }_{i=1}^n\{Z_i^T{Y}_i\left\{1-{\mu }_i\left(b\right)\right\}-Z_i^T\left(1-Y_i\right){\mu }_i(b)\}-\frac{b}{{\tau }_G}\hfill \\ ={\displaystyle \sum }_{i=1}^n\{Z_i^T{Y}_i-Z_i^T{\mu }_i\left(b\right)\}-\frac{b}{{\tau }_G}\hfill \\ =Z^{T}Y-Z^T\mu \left(b\right)-\frac{b}{{\tau }_G}\hfill \\ =Z^T(Y-\mu \left(b\right))-\frac{b}{{\tau }_G},\hfill \end{array}$$

where $\mu \left(b\right)={({\mu }_1\left(b\right),{\mu }_2\left(b\right),\mathrm{...},{\mu }_n\left(b\right))}^T$, and

$$\begin{array}{l}{h}^{″}\left(b\right)=\frac{\partial h^{\prime }(b)}{\partial b^T}={\displaystyle \sum }_{i=1}^n\{Z_i^T{Y}_i-Z_i^T{\dot{\mu }}_i\left(b\right)\}-\frac{1}{{\tau }_G}{I}_L\hfill \\ =-{\displaystyle \sum }_{i=1}^n{\mu }_i\left(b\right)\left\{1-{\mu }_i\left(b\right)\right\}{Z}_i^T{Z}_i-\frac{1}{{\tau }_G}{I}_L\hfill \\ =-Z^{T}W(b)Z-\frac{1}{\tau }{I}_L\hfill \\ =-(Z^{T}W(b)Z+\frac{1}{\tau }{I}_L),\hfill \end{array}$$

where $W(b)=\text{diag}[{\mu }_i\left(b\right)\left\{1-{\mu }_i\left(b\right)\right\}]$.

Finally, we obtain $b^{(t)}$, i.e., the maximizer of $h(b)$. First, we rewrite $b^{(t)}$ as $b_t;$ then we apply the Newton–Raphson method and obtain the iterative estimator of $b_t$ as

$$\begin{array}{l}{b}_t^{(k+1)}=b_t^{(k)}-{[h^{″}(b_t^{(k)})]}^{-1}{h}^{\prime }(b_t^{(k)})\hfill \\ =b_t^{(k)}+{[Z^{T}W\left(b_t^{\left(k\right)}\right)Z+\frac{1}{{\tau }_G}{I}_L]}^{-1}\left[Z^T\left\{Y-\mu \left(b_t^{\left(k\right)}\right)\right\}-\frac{b_t^{\left(k\right)}}{{\tau }_G}\right],\hfill \end{array}$$

which depends on ${\tau }_G$ and γ, and we set ${\tau }_G={\widehat{{\tau }_G}}^{(t)}$ and $\gamma ={\gamma }^{(t)}$. The maximizer, $b^{(t)}$, is obtained at each iteration until it converges, i.e., until the difference $\left|b_t^{(k+1)}-b_t^{(k)}\right|$ falls below a prespecified threshold, e.g., ${10}^{-7}$. We denote the maximizer as $b_t^{(\mathrm{\infty })}$ and also set $b^{(t)}=b_t^{(\mathrm{\infty })}$.

Maximization step for obtaining ${\widehat{\mathit{\gamma }}}^{(\mathit{t}+1)}$

To obtain ${\widehat{\gamma }}^{(t+1)}$, we focus on the first term of $Q(\theta |{\theta }^{(t)});$ i.e.,

$$\mathit{E}\left\{\text{log}\hspace{0.17em}\mathit{f}\left(\mathit{Y}|\mathit{b};\theta \right)|\mathit{Y};{\theta }^{\left(\mathit{t}\right)}\right\}={\displaystyle \sum _{\mathit{i}=1}^n\mathit{E}\left\{{\mathit{Y}}_{\mathit{i}}\hspace{0.17em}\text{log}\hspace{0.17em}{\mu }_{\mathit{i}}+\left(1-{\mathit{Y}}_{\mathit{i}}\right)\text{log}\left(1-{\mu }_{\mathit{i}}\right)|\mathit{Y},{\theta }^{\left(\mathit{t}\right)}\right\}}\equiv \mathit{d}\left(\gamma \right).$$

We rewrite ${\mu }_i$ as ${\mu }_i\left(\gamma \right)$ here to emphasize that it is a function of γ; i.e., ${\mu }_i\left(\gamma \right)=\exp (X_i\gamma +Z_{i}b)/\left(1+\exp (X_i\gamma +Z_{i}b)\right)$. We have that ${\dot{\mu }}_i(\gamma )\equiv \partial {\mu }_i\left(\gamma \right)/\partial \gamma =X_i^T\left(\exp (X_i\gamma +Z_{i}b)/{\left\{1+\exp (X_i\gamma +Z_{i}b)\right\}}^2\right)=X_i^T{\mu }_i\left(\gamma \right)\left\{1-{\mu }_i\left(\gamma \right)\right\}$.. Then

$$\begin{array}{l}{d}^{\prime }\left(\gamma \right)=\frac{\partial d(\gamma )}{\partial \gamma }=\frac{\partial {\sum }_{i=1}^{n}E\{Y_i\log {\mu }_i+(1-Y_i)\log (1-{\mu }_i)|Y,{\theta }^{(t)}\}}{\partial \gamma }\hfill \\ ={\displaystyle \sum }_{i=1}^{n}E\{X_i^T{Y}_i\frac{{\dot{\mu }}_i\left(\gamma \right)}{{\mu }_i\left(\gamma \right)}+X_i^T\left(1-Y_i\right)\frac{-{\dot{\mu }}_i\left(\gamma \right)}{1-{\mu }_i(\gamma )}\}\hfill \\ ={\displaystyle \sum }_{i=1}^{n}E\{X_i^T{Y}_i\left\{1-{\mu }_i\left(\gamma \right)\right\}-X_i^T\left(1-Y_i\right){\mu }_i(\gamma )\}\hfill \\ ={\displaystyle \sum }_{i=1}^n{X}_i^T(Y_i-{\mu }_i\left(\gamma \right))\hfill \\ =X^T\left(Y-\mu (\gamma )\right),\hfill \end{array}$$

where μ = (μ₁(γ), μ₂(γ), … , μ_n(γ))^T = μ(b), and

$$\begin{array}{l}{d}^{″}\left(\gamma \right)=\frac{\partial d^{\prime }(\gamma )}{\partial {\gamma }^T}={\displaystyle \sum }_{i=1}^n{X}_i^T(Y_i-{\dot{\mu }}_i\left(\gamma \right))\hfill \\ =-{\displaystyle \sum }_{i=1}^n{\mu }_i\left(\gamma \right)\left\{1-{\mu }_i(\text{γ})\right\}{X}_i^T{X}_i\hfill \\ =-X^{T}W(\gamma )X.\hfill \end{array}$$

Recall that $W\left(\gamma \right)=\text{diag}\left\{{\displaystyle {\sum }_{i=1}^n{\mu }_i}\left(\gamma \right)\left\{1-{\mu }_i(\text{γ})\right\}\right\}=\text{diag}\left\{{\displaystyle {\sum }_{i=1}^n{\mu }_i}\left(b\right)\left\{1-{\mu }_i(b)\right\}\right\}=W(b)$. Using the first and second derivatives of $d(\gamma )$, the estimator of ${\gamma }^{(t+1)}$, rewritten as ${\gamma }_{t+1}$, at the $(k+1)$th iteration, is given by

$$\begin{array}{l}{\gamma }_{t+1}^{(k+1)}={\gamma }_{t+1}^{(k)}-{[d^{″}({\gamma }_{t+1}^{(k)})]}^{-1}{d}^{\prime }({\gamma }_{t+1}^{(k)})\hfill \\ ={\gamma }_{t+1}^{(k)}+{[X^{T}W\left({\gamma }_{t+1}^{\left(k\right)}\right)X]}^{-1}{X}^T\left(Y-\mu \left({\gamma }_{t+1}^{\left(k\right)}\right)\right),\hfill \end{array}$$

which depends on ${\tau }_G$ and b. We set ${\tau }_G={\widehat{{\tau }_G}}^{(t)}$ and $b=b^{(t)}$. Then ${\gamma }^{(t+1)}={\gamma }_{t+1}^{(\mathrm{\infty })}$.

Putting it all together, at iteration $t+1$ we have following estimators:

• ${\widehat{{\tau }_G}}^{(t+1)}=-\left(1/r\right)\left[b^{\left(t\right)T}{b}^{\left(t\right)}+\text{trace}\left({\Sigma }^{\left(t\right)}\right)\right]$, where $b^{(t)}=b_t^{(\mathrm{\infty })}$ and $b_t^{\left(k+1\right)}=b_t^{\left(k\right)}+{[Z^{T}W(b_t^{(k)})Z+\left(1/{\tau }_G\right)I_L]}^{-1}[Z^T(Y-\mu (b_t^{(k)})-b_t^{(k)}/{\tau }_G]$.

• ${\gamma }^{(t+1)}={\gamma }_{t+1}^{(\mathrm{\infty })}$ and ${\gamma }_{t+1}^{\left(k+1\right)}={\gamma }_{\text{t}+1}^{\left(k\right)}+{[X^T\mu ({\gamma }_{t+1}^{\left(k\right)})X]}^{-1}{X}^T(Y-\mu ({\gamma }_{t+1}^{\left(k\right)}))$.

Appendix C: Asymptotic Distributions of the Score Test Statistics

Recall that $T_{GE}=\left(1/2\right)\left\{{(y_1^W-X\widehat{\gamma })}^T{V}_1^{-1}{S}_{GE}{V}_1^{-1}\left(y_1^W-X\widehat{\gamma }\right)\right\}{|}_{{\tau }_G=\widehat{{\tau }_G}, {\tau }_{GE}=0}$. Because $\widehat{\gamma }={(X^T{V_0}^{-1}X)}^{-1}{X}^T{V_0}^{-1}{Y_1}^W$, we have

$$\begin{array}{l}{y}_1^W-X\widehat{\gamma }=[I_n-X{(X^T{V}_1^{-1}X)}^{-1}{X}^T{V}_1^{-1}](y_1^W-X\gamma )\hfill \\ =K_1(y_1^W-X\gamma ),\hfill \end{array}$$

where $K_1=[I_n-X{(X^T{V}_1^{-1}X)}^{-1}{X}^T{V}_1^{-1}]$.. Therefore, $T_{GE}$ can be rewritten as

$$\begin{array}{l}{T}_{GE}=\frac{1}{2}\{{(y_1^W-X\gamma )}^T{K}_1^T{V}_1^{-1}{S}_{GE}{V}_1^{-1}{K}_1(y_1^W-X\gamma )\}\hfill \\ =\frac{1}{2}\{{(y_1^W-X\gamma )}^T{V}_1^{-1/2}{V}_1^{1/2}{K}_1^T{V}_1^{-1}{S}_{GE}{V}_1^{-1}{K}_1{V}_1^{1/2}{V}_1^{-1/2}(y_1^W-X\gamma )\}\hfill \\ =\frac{1}{2}\{{\widetilde{y_1^W}}^T{A}_1 \widetilde{y_1^W}\},\hfill \end{array}$$ (C1)

where $\widetilde{y_1^W}=V_1^{-1/2}(y_1^W-X\gamma )$, and $A_1=V_1^{1/2}{K}_1^T{V}_1^{-1}{S}_{GE}{V}_1^{-1}{K}_1{V}_1^{1/2}$. In addition, the working vector $y_1^W$ has mean $X\gamma$ and variance $V_1$ (Zhang and Lin 2003), and thus $\widetilde{y_1^W}$ has mean 0 and variance $I_{n\times n}$.

Let ${\eta }_i^1$, $i=1,\dots ,L$, denote the nonzero eigenvalues of matrix $A_1$ and let ${\mathbf{\nu }}_i^1$ denote the corresponding eigenvectors. Then, $T_{GE}={\Sigma }_{i=1}^L{\eta }_i^1{({{\mathit{\nu }}_i^1}^T\widetilde{y_1^W})}^2={\Sigma }_{i=1}^L{\eta }_i^1{(Z_i)}^2$, where $Z_i\dot{\sim }N(0,1)$. Therefore, $T_{GE}$ can be approximated by a weighted sum of ${\chi }^2$-distributions ${\Sigma }_{i=1}^L\widehat{{\eta }_i^1}{\chi }_{i(1)}^2$. By a similar derivation, the distribution of $T_{\text{joint}}$ can be approximated by ${\Sigma }_{i=1}^L\widehat{{\eta }_i^0}{\chi }_{i(1)}^2$, where the ${\eta }_i^0$’s are the nonzero eigenvalues of matrix $A_0=V_0^{1/2}{K}_0^T{V}_0^{-1}(S_G+S_{GE})V_0^{-1}{K}_0{V}_0^{1/2}$, with $K_0=\left[I_n-X{\left(X^T{V}_0^{-1}X\right)}^{-1}{X}^T{V}_0^{-1}\right]$.