Multivariate partial linear varying coefficients model for gene‐environment interactions with multiple longitudinal traits

Abstract

Correlated phenotypes often share common genetic determinants. Thus, a multi‐trait analysis can potentially increase association power and help in understanding pleiotropic effect. When multiple traits are jointly measured over time, the correlation information between multivariate longitudinal responses can help to gain power in association analysis, and the longitudinal traits can provide insights on the dynamic gene effect over time. In this work, we propose a multivariate partially linear varying coefficients model to identify genetic variants with their effects potentially modified by environmental factors. We derive a testing framework to jointly test the association of genetic factors and illustrated with a bivariate phenotypic trait, while taking the time varying genetic effects into account. We extend the quadratic inference functions to deal with the longitudinal correlations and used penalized splines for the approximation of nonparametric coefficient functions. Theoretical results such as consistency and asymptotic normality of the estimates are established. The performance of the testing procedure is evaluated through Monte Carlo simulation studies. The utility of the method is demonstrated with a real data set from the Twin Study of Hormones and Behavior across the menstrual cycle project, in which single nucleotide polymorphisms associated with emotional eating behavior are identified.

1. INTRODUCTION

Cross‐sectional disease traits have been the primary focus in genetic association studies. Given the improved power to identify disease genes with phenotypic data measured over time, longitudinal designs are becoming popular in genetic association studies. ¹ , ² , ³ , ⁴ Most statistical methods developed so far focus on a single outcome of interest. When multiple outcomes are measured over time, for example, multiple measures of heart function in a longitudinal study of cardiac function, methods focusing on just a single outcome over time may not provide a complete picture of cardiac function.

In genetics, the phenomenon that a single gene or locus influences more than one trait is known as pleiotropy. ⁵ , ⁶ Genetic pleiotropy plays a crucial role in many complex diseases. One of the most well‐known examples is the phenylketonuria (PKU) disease. ⁷ The conventional approach to identify genetic pleiotropic effects on multiple traits is to test the association between a gene and each trait individually and then determine whether the genetic effect is significantly associated with more than one trait. The disadvantages of this approach, such as the inflation in the family wise Type I error and incomplete information in individual tests compared to a combined analysis for multiple traits, have been discussed in some studies. ⁵ Therefore, a joint genetic association test on multiple traits is more desirable to control the family wise Type I error and enhance the power of tests.

In real life, timing is a very important factor in the development of a disease. Genetic effects on a disease trait vary during the life span of an individual. The function of a gene depends largely on when it turns on and off, which could show a temporal pattern. In order to capture the dynamic effect of a gene on a disease trait over time, it is natural to model the dynamic effect as a potential (nonlinear) function over time. Considering multiple longitudinal traits, we proposed the following partially linear varying coefficients model,

$$y_{li}(t_{ij})={\beta }_{0l}(t_{ij})+{\beta }_{1l}(t_{ij})G_i+{\mathit{\alpha }}_l^T{\mathbf{Z}}_{ij}+{ϵ}_{li}(t_{ij}),l=1,\dots ,L,i=1,\dots ,N,j=1,\dots ,n_i,$$ (1)

where $y_{lij}=y_{li}(t_{ij})$ is the response variable which measures the $l$th phenotype on the $i$th subject at the $j$th time point; ${\mathbf{Z}}_{ij}$ is a $p$‐dimensional vector of covariates, which can be either time dependent or independent; $G_i$ denotes the time‐invariant genetic variable within subject; ${\beta }_{0l}(·)$ and ${\beta }_{1l}(·)$ are unknown functions; and the stacked error vector ${\mathit{ϵ}}_i={({\mathit{ϵ}}_{1i}^T,\dots ,{\mathit{ϵ}}_{Li}^T)}^T$ with ${\mathit{ϵ}}_{li}={({ϵ}_{li1},\dots ,{ϵ}_{li{n}_i})}^T$ is assumed to have mean zero and covariance ${\mathbf{\sum }}_i$. Models for multivariate longitudinal traits are necessarily complex, because they must consider different types of correlations for each independent subject: correlation between measurements for the same trait at different time points, correlation between measurements at the same time point on different traits, and correlation between measurements at different time points and on different traits. With the stacked error vector ${\mathit{ϵ}}_i$, its covariance matrix ${\mathbf{\sum }}_i$ carries all of these correlations.

If we use a time‐varying environmental factor $X_{ij}$ instead of $t_{ij}$ in the model, that is,

$$y_{lij}={\beta }_{0l}(X_{ij})+{\beta }_{1l}(X_{ij})G_i+{\mathit{\alpha }}_l^T{\mathbf{Z}}_{ij}+{ϵ}_{lij},$$

then the model can be used for jointly modeling nonlinear gene‐environment (G$\times$E) interactions for multiple longitudinal traits. In the model, one can assess the influence of $X$ on $G$ to affect multiple responses $Y$. Models for nonlinear G$\times$E interactions have been studied. ⁸ , ⁹

Qu and Li applied the method of quadratic inference functions (QIF) to the varying coefficients models for longitudinal data. ¹⁰ One important advantage is that the QIF method only requires correct specification of the mean structure and does not require any likelihood or approximation of the likelihood in hypothesis testing. In addition, when the working correlation structure is misspecified, QIF is more efficient than the generalized estimation equation (GEE) approach. Another advantage of the QIF approach is that the inference function has an asymptotic form, which provides a model selection criteria similar to AIC and BIC. It also allows us to test whether coefficients are significantly time‐varying based on the asymptotic results.

Rochon analyzed bivariate longitudinal data for discrete and continuous outcomes by using generalized estimating equations, which did not utilize the nice property of the QIF. ¹¹ Cho applied QIF for multivariate longitudinal data with generalized linear models, which is not adequate to consider nonlinear effects as in varying coefficient models. ¹² Using random effects for modeling longitudinal data is another very popular and common way. ¹³ Proudfoot and his coauthors modeled the longitudinal data using random effects then combines multiple outcomes together again similar to the generalized estimating equations. ¹⁴ Recently, Zhao and his coauthors proposed a joint penalized quasi‐likelihood modeling based on splines for multivariate longitudinal data using random effects with applications to HIV‐1 RNA load levels and CD4 cell counts. ¹⁵ Hector and Song investigated a distributed quadratic inference function framework to jointly estimate regression parameters from multiple heterogeneous data sets with correlated responses. ¹⁶

With the nice property of QIF to deal with complicated correlated data focusing on a univariate longitudinal response, ¹⁰ in this article, we consider to generalize it to partial linear varying coefficient models with multivariate longitudinal responses. ¹⁷ , ¹⁸ The purpose of this article is to develop a powerful joint testing procedure using QIF for model (1). If the correlation between the longitudinal outcomes is reasonably high, we aim to show that the joint test has higher power than the marginal tests to detect the signal like the genetic effect in model (1). We first use splines to approximate the nonparametric functions in the model, ¹⁹ followed by penalized estimation to avoid over fitting. Then we develop a 2‐step testing procedure to have a joint test for the interaction effect on multiple outcomes based on the QIF approach, followed by separate test of marginal effect on each outcome if the overall null is rejected. In cross‐sectional studies, Wu et al. ²⁰ developed a multivariate partially linear varying coefficient model to detect G$\times$E interactions with multiple traits. Their method can select genetic variants with pleiotropic effects incorporating either the homogeneity (ie, pleiotropy) or heterogeneity (ie, no pleiotropy) assumptions. However, their approach cannot provide uncertainty quantification for the selected variables. Generalizing their method to multivariate longitudinal data is worth further studying.

This article is organized as follows. We state our proposed model in Section 2.1, and generalize the QIF method to the multivariate longitudinal responses in Section 2.2. Estimation procedure and asymptotical properties of estimators are provided in Section 2.3. A theorem for the general goodness‐of‐fit test via QIF is established in Section 2.4, based on which we propose a 2‐step testing procedure. We assess the finite sample performance of the proposed procedure with Monte Carlo simulation in Section 3 and illustrate the proposed methodology by the analysis of an emotional eating behavior study in Section 4. Conclusions and discussion are made in Section 5. Proofs are rendered to Appendix.

2. STATISTICAL METHODS

2.1. A joint multivariate partial linear model

In multivariate longitudinal studies, suppose $y_{lij}$ is the $l$th continuous outcome collected on the $i$th observation at time point $t_{ij}$, where $l$ = 1, $\dots$ , $L$, $i$ = 1, $\dots$ , $N$, $j$ = 1, $\dots$ , $n_i$. The joint partially linear varying coefficient models are defined as

$$y_{lij}=y_{li}(t_{ij})={\beta }_{0l}(t_{ij})+{\beta }_{1l}(t_{ij})G_i+{\mathit{\alpha }}_l^T{\mathbf{Z}}_{ij}+{ϵ}_{lij},$$

where $G_i$ is the single nucleotide polymorphism (SNP) variable which does not depend on time and other types of measurement; ${\mathit{Z}}_{ij}$ is a $p$‐dimensional covariate vector, which can be either time‐dependent or time‐independent; to accommodate the correlation between multiple responses, we stack the error terms ${ϵ}_{lij}$ together into a long vector

$${\mathit{ϵ}}_i=\left(\begin{array}{c}{\mathit{ϵ}}_{1i}\\ ⋮\\ {\mathit{ϵ}}_{Li}\end{array}\right)\text{where}{\mathit{ϵ}}_{li}=\left(\begin{array}{c}{ϵ}_{li1}\\ ⋮\\ {ϵ}_{li{n}_i}\end{array}\right).$$

We assume ${\mathit{ϵ}}_i$ mean 0 with covariance $\mathbf{\sum }$, which carries three different association information: the within‐subject correlation across different time points, the between‐subject correlation at the same time point and the between‐subject correlation across different time points; ${\beta }_{0l}(·)$ and ${\beta }_{1l}(·)$ are unknown nonparametric smooth functions, representing the main time effect and time dependent genetic effect respectively. To illustrate the idea, in the following we demonstrate the methods assuming $L$=2. For the situation where there are more than two traits ($L>2$), the technique can be easily extended.

2.2. Quadratic inference function

To construct the objective function using the QIF approach, we first approximate the unknown functions ${\beta }_{01}$, ${\beta }_{11}$, ${\beta }_{02}$, and ${\beta }_{12}$ by a $q$‐degree truncated power spline basis, that is,

$${\beta }_{sl}(t)\approx {\mathbf{B}}_{sl}{(t)}^T{\mathit{\gamma }}_{sl},\text{for}s=0,1\text{and}l=1,2,$$ (2)

where ${\mathbf{B}}_{sl}(t)={(1,t,t^2,\dots ,t^{q_{sl}},{(t-{\kappa }_{sl,1})}_{+}^{q_{sl}},\dots ,{(t-{\kappa }_{sl,K_{sl}})}_{+}^{q_{sl}})}^T$ is a truncated power spline basis with degree $q_{sl}$ and $K_{sl}$ knots ${\kappa }_{sl,1},\dots ,{\kappa }_{sl,K_{sl}}$. ${\mathit{\gamma }}_{sl}$ is a $(q_{sl}+K_{sl}+1)$‐dimensional vector of spline coefficients.

Under the GEE framework, we solve

$$\sum _{i=1}^N{\dot{\mathit{\mu }}}_i^T{\mathbf{v}}_i^{-1}({\mathbf{y}}_i-{\mathit{\mu }}_i)=0,$$ (3)

where ${\mathbf{y}}_i={({\mathbf{y}}_{1i}^T,{\mathbf{y}}_{2i}^T)}^T$, ${\mathbf{y}}_{li}={(y_{li1},\dots ,y_{li{n}_i})}^T$; ${\mathit{\mu }}_i=E({\mathbf{y}}_i)$ is the mean function and ${\dot{\mathit{\mu }}}_i$ is the first derivative of ${\mathit{\mu }}_i$ with respect to the parameters; ${\mathbf{v}}_i$ is the covariance matrix of ${\mathbf{y}}_i$ and can be decomposed as ${\mathbf{v}}_i={\mathbf{A}}_i^{1/2}\mathbf{R}(\mathit{\rho }){\mathbf{A}}_i^{1/2}$ with ${\mathbf{A}}_i$ being a diagonal matrix of marginal variances and $\mathbf{R}(\mathit{\rho })$ being a working correlation matrix with nuisance parameters $\mathit{\rho }$. To avoid the estimation of $\mathit{\rho }$, QIF approach considers the inverse of the correlation matrix $\mathbf{R}$ as a linear combination of several known basis matrices in a form

$${\mathbf{R}}^{-1}\approx a_1{\mathbf{M}}_1+a_2{\mathbf{M}}_2+\cdots +a_h{\mathbf{M}}_h,$$ (4)

where ${\mathbf{M}}_1$ is the identity matrix and ${\mathbf{M}}_2,\dots ,{\mathbf{M}}_h$ are symmetric basis matrices. As discussed in the existing literature, ¹⁰ , ¹² the choice of the basis for the inverse of the correlation matrix plays an important role. Suppose $\mathbf{\Gamma }$ as the within‐subject correlation structure and $\omega$ as between‐subject correlation coefficient, that is, the working correlation structure can be expressed as the Kronecker product (tensor product) $\mathbf{R}=\mathbf{\Omega }\otimes \mathbf{\Gamma }$ with $\mathbf{\Omega }$ as the $2\times 2$ symmetric matrix with 1 on the diagonal and $\omega$ elsewhere. The inverse of the Kronecker product is ${\mathbf{R}}^{-1}={\mathbf{\Omega }}^{-1}\otimes {\mathbf{\Gamma }}^{-1}=({\gamma }_0\mathit{I}+{\gamma }_1\mathit{W})\otimes {\mathbf{\Gamma }}^{-1}$ with $\mathit{W}$ as a $2\times 2$ symmetric matrix with 0 on the diagonal and 1 elsewhere, and $\mathit{I}$ as the identity matrix with compatible dimension. So if the basis matrix for the inverse of the within‐subject correlation ${\mathbf{\Gamma }}^{-1}$ is given by ${\mathit{U}}_1(=\mathit{I}),{\mathit{U}}_2,\dots ,{\mathit{U}}_k$, then we have the bases $\mathit{M}$'s as $\{\mathit{I}\otimes {\mathit{U}}_j,\mathit{W}\otimes {\mathit{U}}_j:1\le j\le k\}$. For exchangeable working correlation, we can set $k=2$ and ${\mathbf{U}}_2$ has 0 on the diagonal and 1 elsewhere. If the working correlation is AR(1), we can set $k=2$ and ${\mathbf{U}}_2$ to have 1 on its two subdiagonals and 0 elsewhere. Following QIF approach, we define the estimation function as

$${\bar{g}}_N(\mathit{\theta })=\frac{1}{N}\sum _{i=1}^N{g}_i(\mathit{\theta })=\frac{1}{N}\left[\begin{array}{c}\sum _{i=1}^N{\dot{\mathit{\mu }}}_i^T{\mathbf{A}}_i^{-1/2}{\mathbf{M}}_1{\mathbf{A}}_i^{-1/2}({\mathbf{y}}_i-{\mathit{\mu }}_i)\\ ⋮\\ \sum _{i=1}^N{\dot{\mathit{\mu }}}_i^T{\mathbf{A}}_i^{-1/2}{\mathbf{M}}_h{\mathbf{A}}_i^{-1/2}({\mathbf{y}}_i-{\mathit{\mu }}_i)\end{array}\right].$$ (5)

Using the spline approximation, the mean function ${\mathit{\mu }}_i$ can be written as

$${\mathit{\mu }}_i(\mathit{\theta })=\left[\begin{array}{c}{\mathit{\mu }}_{1i}(\mathit{\theta })\\ {\mathit{\mu }}_{2i}(\mathit{\theta })\end{array}\right]=\left[\begin{array}{c}{\mu }_{1i1}(\mathit{\theta })\\ ⋮\\ {\mu }_{1i{n}_i}(\mathit{\theta })\\ {\mu }_{2i1}(\mathit{\theta })\\ ⋮\\ {\mu }_{2i{n}_i}(\mathit{\theta })\end{array}\right]=\left[\begin{array}{c}{\mathbf{B}}_{01}^T(t_{i1}){\mathit{\gamma }}_{01}+{\mathbf{B}}_{11}^T(t_{i1}){\mathit{\gamma }}_{11}{G}_i+{\mathit{\alpha }}_1^T{\mathbf{Z}}_{i1}\\ ⋮\\ {\mathbf{B}}_{01}^T(t_{i{n}_i}){\mathit{\gamma }}_{01}+{\mathbf{B}}_{11}^T(t_{i{n}_i}){\mathit{\gamma }}_{11}{G}_i+{\mathit{\alpha }}_1^T{\mathbf{Z}}_{i{n}_i}\\ {\mathbf{B}}_{02}^T(t_{i1}){\mathit{\gamma }}_{02}+{\mathbf{B}}_{12}^T(t_{i1}){\mathit{\gamma }}_{12}{G}_i+{\mathit{\alpha }}_2^T{\mathbf{Z}}_{i1}\\ ⋮\\ {\mathbf{B}}_{02}^T(t_{i{n}_i}){\mathit{\gamma }}_{02}+{\mathbf{B}}_{12}^T(t_{i{n}_i}){\mathit{\gamma }}_{12}{G}_i+{\mathit{\alpha }}_2^T{\mathbf{Z}}_{i{n}_i}\end{array}\right],$$

and the first derivative of ${\mathit{\mu }}_i$ is given as,

$${\dot{\mathit{\mu }}}_i=\left[\begin{array}{cccccc}{\mathbf{B}}_{01}^T(t_{i1})& {\mathbf{B}}_{11}^T(t_{i1})G_i& {\mathbf{Z}}_{i1}& 0& 0& 0\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ {\mathbf{B}}_{01}^T(t_{i{n}_i})& {\mathbf{B}}_{11}^T(t_{i{n}_i})G_i& {\mathbf{Z}}_{i{n}_i}& 0& 0& 0\\ 0& 0& 0& {\mathbf{B}}_{02}^T(t_{i1})& {\mathbf{B}}_{12}^T(t_{i1})G_i& {\mathbf{Z}}_{i1}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ 0& 0& 0& {\mathbf{B}}_{02}^T(t_{i{n}_i})& {\mathbf{B}}_{12}^T(t_{i{n}_i})G_i& {\mathbf{Z}}_{i{n}_i}\end{array}\right],$$

where $\mathit{\theta }={({\mathit{\gamma }}_{01}^T,{\mathit{\gamma }}_{11}^T,{\mathit{\alpha }}_1^T,{\mathit{\gamma }}_{02}^T,{\mathit{\gamma }}_{12}^T,{\mathit{\alpha }}_2^T)}^T$.

Setting each component in (5) to be zero will result in more equations than unknown parameters. Following the idea of generalized method of moments, ²¹ the QIF method is defined as

$$Q_N(\mathit{\theta })=N{\bar{g}}_N^T{\bar{C}}_N^{-1}{\bar{g}}_N,$$ (6)

where ${\bar{C}}_N=\frac{1}{N}{\sum }_{i=1}^N{g}_i{g}_i^T$ is a consistent estimator for $\mathrm{var}(g_i)$. Minimizing the objective function (6) provides the estimation of the parameters.

2.3. Estimation procedure via penalized QIF

The estimation of the parameters can be obtained through minimizing the objective function, that is,

$$\widehat{\mathit{\theta }}=\arg \underset{\mathit{\theta }}{\min }{Q}_N(\mathit{\theta }).$$

To avoid over‐fitting, we can define a penalized QIF in a form

$$N^{-1}{Q}_N(\mathit{\theta })+\lambda {\mathit{\theta }}^T\mathbf{D}\mathit{\theta },$$ (7)

where $\mathbf{D}$ is a diagonal matrix with 1 if the corresponding parameter is the spline coefficient associated with knots, and 0 otherwise. Minimizing the penalized QIF provides

$$\widehat{\mathit{\theta }}=\arg \underset{\mathit{\theta }}{\min }(N^{-1}{Q}_N(\mathit{\theta })+\lambda {\mathit{\theta }}^T\mathbf{D}\mathit{\theta }).$$ (8)

To estimate the tuning parameter $\lambda$, we can extend the generalized cross‐validation ¹⁰ , ²² , ²³ to the penalized QIF and define the generalized cross‐validation statistic as

$$\text{GCV}(\lambda )=\frac{N^{-1}{Q}_N}{{(1-N^{-1}df)}^2}$$

with the effective degree of freedom

$$\text{df}=\text{tr}\left[{({\ddot{Q}}_N+2N\lambda D)}^{-1}{\ddot{Q}}_N\right],$$

where ${\ddot{Q}}_N$ is the second derivative of $Q_N$. The optimized tuning parameter $\lambda$ is given as

$$\widehat{\lambda }=\arg \underset{\lambda }{\min }\text{GCV}(\lambda ).$$

To establish the asymptotic properties for the penalized quadratic inference function estimators with fixed knots, we assume ${\mathit{\theta }}_0$ to be the parameter satisfying $E_{{\mathit{\theta }}_0}(g_i)=0$. Similar theoretical results are provided in Qu and Li. ¹⁰ Following their idea and extend those results to the estimators in our model, we get the strong consistency of the resulting estimators in Theorem 1. The $\sqrt{N}$‐consistency and asymptotic normality of the estimators are given in Theorem 2 .

Theorem 1

Suppose conditions (A1)‐(A6) in the Appendix hold and the smoothing parameter ${\lambda }_N=o(1)$ , then the estimator $\widehat{\mathit{\theta }}$ , which is obtained by minimizing the penalized quadratic function in ( 7 ), exists and converges to ${\mathit{\theta }}_0$ almost surely.

Theorem 2

Suppose conditions (A1)‐(A6) in the Appendix hold and the smoothing parameter ${\lambda }_N=o(N^{-1/2})$ , then the estimator $\widehat{\mathit{\theta }}$ obtained by minimizing the penalized quadratic function in ( 7 ) is asymptotically normally distributed with the limiting distribution,

$$\sqrt{N}(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)\underset{}{\overset{d}{\to }}N(\mathbf{0},{({\mathbf{G}}_0^T{\mathbf{C}}_0^{-1}{\mathbf{G}}_0)}^{-1}),$$

where the calculation of ${\mathbf{G}}_0$ defined in (A6) and ${\mathbf{C}}_0$ defined in (A5) can be found in the Appendix.

2.4. A two‐step hypothesis testing procedure

Compared to GEE, an advantage of the QIF approach is that QIF provides a goodness‐of‐fit test without estimating the second moment parameters. Suppose that the $d$‐dimensional parameter vector $\mathit{\gamma }$ is partitioned into $(\mathit{\psi },\mathit{\zeta })$, where $\mathit{\psi }$ is the parameter of interest with dimension $d_1$, and $\mathit{\zeta }$ is a nuisance parameter with dimension $d_2=d-d_1$. If we are interested in testing

$$H_0:\mathit{\psi }={\mathit{\psi }}_0,$$

then the test statistic

$$Q_N({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})-Q_N(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }}),$$

follows an asymptotically chi‐square distribution with $d_1$ degrees of freedom as from Qu and her coauthors work cited below. ²⁴

Theorem 3

Suppose that all required regularity conditions are satisfied and

$\mathit{\psi }$

has dimension

$d_1$

. Under the null hypothesis,

$Q_N({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})-Q_N(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})$

is asymptotically chi‐square distributed with

$d_1$

degrees of freedom, where

$$\tilde{\mathit{\zeta }}=\arg \min Q_N({\mathit{\psi }}_0,\mathit{\zeta }),(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})=\arg \min Q_N(\mathit{\psi },\mathit{\zeta }).$$ (9)

In Model (1), it is of interest to test whether the genetic effects on multiple traits are significant or not. Based on Theorem 3, we develop a 2‐step testing procedure for testing the significance of the varying coefficient functions. In the first step, the joint test is performed to see whether a genetic factor has a significant effect on at least one longitudinal trait. If the testing result in the first step is significant, we then further conduct the marginal test in the second step to assess if the genetic effect is significant on both traits or just one trait. The first step is a joint test of significance followed by a marginal test to assess individual significance. For associated multiple traits with reasonably strong correlation, the joint test is more powerful than the marginal tests, which is empirically verified in our simulation studies.

2.4.1. Step 1: Joint test

First, we are interested in testing whether the genetic factor $G$ has an effect on at least one longitudinal trait. The hypothesis is stated as

$$H_0:{\beta }_{11}(·)={\beta }_{12}(·)=0\text{v.s.}{H}_1:{\beta }_{11}(·)\ne 0\text{or}{\beta }_{12}(·)\ne 0.$$

This can be handled through the truncated power spline approximation of the nonparametric functions stated in (2). In particular, testing this hypothesis is equivalent to test the following null hypothesis

$$H_0:{\mathit{\gamma }}_{11}={\mathit{\gamma }}_{12}=\mathbf{0}.$$

According to Theorem 3, we can construct a test statistic

$$T_N=Q_N(\tilde{\mathit{\theta }})-Q_N(\widehat{\mathit{\theta }}),$$

where

$$\tilde{\mathit{\theta }}={\text{arg min}}_{{\mathit{\gamma }}_{11}={\mathit{\gamma }}_{12}=\mathbf{0}}{Q}_N\left\{{\mathit{\gamma }}_{01},{\mathit{\gamma }}_{11},{\mathit{\alpha }}_1,{\mathit{\gamma }}_{02},{\mathit{\gamma }}_{12},{\mathit{\alpha }}_2|{\mathbf{y}}_1,{\mathbf{y}}_2,\mathbf{G},\mathbf{Z}\right\},$$

and

$$\widehat{\mathit{\theta }}=\text{arg min}{Q}_N\left\{{\mathit{\gamma }}_{01},{\mathit{\gamma }}_{11},{\mathit{\alpha }}_1,{\mathit{\gamma }}_{02},{\mathit{\gamma }}_{12},{\mathit{\alpha }}_2|{\mathbf{y}}_1,{\mathbf{y}}_2,\mathbf{G},\mathbf{Z}\right\}.$$

The test statistic $T_N$ has an asymptotic ${\chi }^2$ distribution with the degrees of freedom equal the number of constraints under $H_0$, according to Theorem 3.

2.4.2. Step 2: Marginal tests

From the joint test, if there exists a significant genetic effect on at least one longitudinal trait, then we can further test the marginal effects, that is,

$$H_{0l}:{\beta }_{1l}(·)=0\text{v.s.}{H}_1:{\beta }_{1l}(·)\ne 0,l=1,2.$$

Based on (2), this is equivalent to test $H_{01}:{\mathit{\gamma }}_{11}=\mathbf{0}$ and $H_{02}:{\mathit{\gamma }}_{12}=\mathbf{0}$, separately.

For testing $H_{01}:{\mathit{\gamma }}_{11}=\mathbf{0}$, we use test statistic $T_{N1}=Q_N({\tilde{\mathit{\gamma }}}_{01},\mathbf{0},{\tilde{\mathit{\alpha }}}_1)-Q_N({\widehat{\mathit{\gamma }}}_{01},{\widehat{\mathit{\gamma }}}_{11},{\widehat{\mathit{\alpha }}}_1)$, where

$$({\tilde{\mathit{\gamma }}}_{01},\mathbf{0},{\tilde{\mathit{\alpha }}}_1)={\text{arg min}}_{{\mathit{\gamma }}_{11}=\mathbf{0}}{Q}_N\left\{{\mathit{\gamma }}_{01},{\mathit{\gamma }}_{11},{\mathit{\alpha }}_1|{\mathbf{y}}_1,\mathbf{G},\mathbf{Z}\right\},$$

and

$$({\widehat{\mathit{\gamma }}}_{01},{\widehat{\mathit{\gamma }}}_{11},{\widehat{\mathit{\alpha }}}_1)=\text{arg min}{Q}_N\left\{{\mathit{\gamma }}_{01},{\mathit{\gamma }}_{11},{\mathit{\alpha }}_1|{\mathbf{y}}_1,\mathbf{G},\mathbf{Z}\right\}.$$

Similarly, we can construct a test statistic $T_{N2}=Q_N({\tilde{\mathit{\gamma }}}_{02},\mathbf{0},{\tilde{\mathit{\alpha }}}_2)-Q_N({\widehat{\mathit{\gamma }}}_{02},{\widehat{\mathit{\gamma }}}_{12},{\widehat{\mathit{\alpha }}}_2)$ for testing $H_{02}:{\mathit{\gamma }}_{12}=\mathbf{0}$, where

$$({\tilde{\mathit{\gamma }}}_{02},\mathbf{0},{\tilde{\mathit{\alpha }}}_2)={\text{arg min}}_{{\mathit{\gamma }}_{12}=\mathbf{0}}{Q}_N\left\{{\mathit{\gamma }}_{02},{\mathit{\gamma }}_{12},{\mathit{\alpha }}_2|{\mathbf{y}}_2,\mathbf{G},\mathbf{Z}\right\},$$

and

$$({\widehat{\mathit{\gamma }}}_{02},{\widehat{\mathit{\gamma }}}_{12},{\widehat{\mathit{\alpha }}}_2)=\text{arg min}{Q}_N\left\{{\mathit{\gamma }}_{01},{\mathit{\gamma }}_{12},{\mathit{\alpha }}_2|{\mathbf{y}}_2,\mathbf{G},\mathbf{Z}\right\}.$$

The asymptotic distribution of the test statistics $T_{N1}$ and $T_{N2}$ can be obtained from Theorem 3.

3. SIMULATION STUDIES

3.1. Simulation setup

In this section, the finite sample performance of the proposed method is evaluated through Monte Carlo simulation studies. Two continuous longitudinal responses are generated from the models

$$\begin{array}{ll}\left\{\begin{array}{c}{y}_{1ij}=y_{1i}(t_{ij})={\beta }_{01}(t_{ij})+{\beta }_{11}(t_{ij})G_i+{\alpha }_1{Z}_i+{ϵ}_{1ij},\\ y_{2ij}=y_{2i}(t_{ij})={\beta }_{02}(t_{ij})+{\beta }_{12}(t_{ij})G_i+{\alpha }_2{Z}_i+{ϵ}_{2ij},\end{array}\right.& \end{array}$$

where ${\beta }_{01}(t_{ij})=0.5\cos (2\pi t_{ij})$, ${\beta }_{11}=\sin (\pi (t_{ij}-0.2))$, ${\beta }_{02}(t_{ij})=\sin (\pi t_{ij})-0.5$, ${\beta }_{12}(t_{ij})=\cos (\pi t_{ij}-0.8)$, ${\alpha }_1=0.2$ and ${\alpha }_2=0.3$. We generate the same number of time points $n$ for each individual ${\mathbf{t}}_i=(t_{i1},\dots ,t_{in})$ from a uniform distribution $U(0,1)$. The time independent predictor variable $Z_i$ is also generated from $U(0,1)$. We set the minor allele frequency (MAF) for $G_i$ as $p_A$ and assume Hardy‐Weinberg equilibrium. Three different SNP genotypes $AA$, $Aa$, and $aa$ are simulated from a multinomial distribution with frequencies $p_A^2$, $2p_A(1-p_A)$ and ${(1-p_A)}^2$, respectively. In this simulation study, we vary $p_A\in \{0.1,0.3,0.5\}$ to investigate the effect of minor allele frequency. Variable $G$ takes value {0,1,2} corresponding to genotypes $\{aa,Aa,AA\}$, following an additive model. We assume ${ϵ}_{1ij}$ and ${ϵ}_{2ij}$ are jointly normally distributed as

$$\left(\begin{array}{c}{\mathit{ϵ}}_{1i}\\ {\mathit{ϵ}}_{2i}\end{array}\right)\mathbb{\backsim }N\left[\begin{array}{c}\left(\begin{array}{c}\mathbf{0}\\ \mathbf{0}\end{array}\right),\left(\begin{array}{cc}{\sigma }_1^2{\sum }_{11}& {\sigma }_1{\sigma }_2{\sum }_{12}\\ {\sigma }_1{\sigma }_2{\sum }_{12}& {\sigma }_2^2{\sum }_{22}\end{array}\right)\end{array}\right].$$

We set the marginal variances ${\sigma }_1^2$ = ${\sigma }_2^2$ = 0.1. The true correlation structure of ${\sum }_{11}$ and ${\sum }_{22}$ are both exchangeable with the structure

$$\left(\begin{array}{ccccc}1& \rho & \rho & \cdots & \rho \\ \rho & 1& \rho & \cdots & \rho \\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \rho & \rho & \rho & \cdots & 1\end{array}\right)$$

with ${\rho }_1={\rho }_2=0.5$. And for ${\sum }_{12}$ we choose

$$\left(\begin{array}{ccccc}\tau & {\rho }_{12}& {\rho }_{12}& \cdots & {\rho }_{12}\\ {\rho }_{12}& \tau & {\rho }_{12}& \cdots & {\rho }_{12}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ {\rho }_{12}& {\rho }_{12}& {\rho }_{12}& \cdots & \tau \end{array}\right)$$

with ${\rho }_{12}=0.2$ as the between‐subject correlation across different time points. We vary the between‐subject correlation at the same time point $\tau =corr({ϵ}_{1ij},{ϵ}_{2ij})$ to investigate the power gain for the joint test.

We draw 1000 data sets with sample size $N=200,500$ and time points $n_i=n=10$, in order to compare the performances of our proposed method under different sample sizes. We set ${\mathbf{M}}_1$ to be the identity matrix and ${\mathbf{M}}_2$ to be 1 on subdiagonals and 0 elsewhere, that is, AR(1) working correlation. An important issue for the model selection is to decide whether the spline model (2) is adequate for further penalization by (8). In the following simulations, we use quartic splines with the number of knots taken to be the largest integer not greater than 0.6 $\times N^{1/5}$ as suggested in Tian and his colleagues work. ²⁵

3.2. Estimation performance

We use the asymptotic normality in Theorem 2 to construct the Wald type confidence interval for parameters ${\alpha }_1$ and ${\alpha }_2$. Table 1 summarizes the empirical coverage probability (CP) in percentage and the average length (AL) of the confidence intervals at $95\%$ confidence level based on 1000 simulation replicates. As we can see from the table, the CPs are close to the nominal level 95%. When the sample size gets larger, the ALs are shorter and the CPs are closer to 95%.

TABLE 1.

Empirical coverage probability (%) and average length of confidence intervals for ${\alpha }_l$, $l=1,2$

	$N=200$		$N=500$
	CP	AL	CP	AL
${\alpha }_1$	93.2	0.078	94.8	0.050
${\alpha }_2$	93.7	0.078	95.6	0.050

Next, we consider the estimation performance of the nonparametric functions ${\beta }_{sl}(t)$ for $s=0,1,l=1,2$. In Figure 1, the plots are from the case with sample size $N=200$ and $p_A=0.1$ (for other situations, please refer to the Appendix). For each function, the red solid line is the true function, and the three blue dashed lines correspond to the average of the estimated functions from 1000 simulation replicates in the middle and the 95% pointwise confidence bands with the standard error calculated from the standard deviation of 1000 replicates. The estimation is quite accurate with low sample size and MAF. As the MAF or sample increases, the estimation performance improves (see Appendix ). We also use the asymptotic normality in Theorem 2 to construct the pointwise confidence intervals. Table 2 summarized the empirical coverage probability (CP) in percentage and the average length (AL) (in parentheses) of the CIs for ${\beta }_{sl}(t)$ at $t=0.2,0.4,0.6$, and 0.8 for sample size $N=500$ and $N=1000$. The CPs are all close to the nominal level 95% and the ALs are shorter under a larger sample size.

FIGURE 1.

The estimation of nonparametric functions ${\beta }_{sl}(·)$ for $s=0,1,l=1,2$ with $N$ = 200 and $p_A=0.1$. In each panel, the red solid line is the true function, and the three blue dashed lines correspond to the estimated function in the middle and the 95% pointwise confidence bands

TABLE 2.

Empirical coverage probability (%) and average length of pointwise confidence intervals (in parentheses) for ${\beta }_{sl}(t)$ at $t=0.2,0.4,0.6$, and 0.8

$t$	$N$	Intercept ${\beta }_{01}(t)$	Slope ${\beta }_{11}(t)$	Intercept ${\beta }_{02}(t)$	Slope ${\beta }_{12}(t)$
0.2	500	92.2 (0.115)	92.6 (0.073)	92.9 (0.115)	92.7 (0.073)
	1000	91.7 (0.082)	93.3 (0.052)	92.9 (0.082)	92.9 (0.052)
0.4	500	90.5 (0.109)	91.9 (0.069)	92.4 (0.108)	92.8 (0.068)
	1000	92.3 (0.077)	93.7 (0.049)	94.4 (0.077)	93.1 (0.049)
0.6	500	91.0 (0.111)	91.0 (0.070)	92.3 (0.110)	93.1 (0.070)
	1000	94.0 (0.079)	93.8 (0.050)	95.1 (0.079)	94.4 (0.050)
0.8	500	91.5 (0.113)	93.1 (0.072)	92.9 (0.113)	94.1 (0.071)
	1000	91.5 (0.081)	94.2 (0.051)	94.8 (0.080)	94.6 (0.051)

3.3. Testing performance

We propose a two‐step hypothesis testing procedure to detect the genetic effects on multiple traits. With the joint test, higher power is expected with correlated traits than the marginal tests. We would like to evaluate how much we can gain in power when the correlation between multiple traits increases. This is done by varying the correlation coefficient $\tau$ at the same time point for the two simulated traits.

We evaluate the performance of the joint test under the null hypothesis $H_0:{\beta }_{11}(·)={\beta }_{12}(·)=0$. Power is evaluated under a sequence of alternative models with different values of $\delta$, which is denoted by $H_1:{\beta }_{11}(·)=\delta {\beta }_{11}(·)\text{and}{\beta }_{12}(·)=\delta {\beta }_{12}(·)$. The performance of the marginal tests for the nonparametric functions corresponding to different traits is evaluated under the two null hypotheses $H_{01}:{\beta }_{11}(·)=0$ and $H_{02}:{\beta }_{12}(·)=0$ respectively. For each test, power is evaluated under a sequence of alternative models, denoted by $H_{al}:{\beta }_{1l}(·)=\delta {\beta }_{1l}(·),l=1,2$, correspondingly.

Figure 2 shows the power comparison between joint test and marginal tests under different correlation coefficient $\tau$ varying from 0.1 through 0.6. Each panel corresponds to the results with one $\tau$ value and displays the comparison of the three power curves, with the empirical size (when $\delta$ = 0) and power at different $\delta (>0)$ at the significance level 0.05 and sample size $N=200$. Similar pattern can be observed for larger sample size $N=500$. As expected, the Type I error is closer to 0.05 and the power increase as the signal $\delta$ increases for every power curve. When $\tau$ is small (low correlation), we do not see much power gain of the joint test compared to the marginal tests. As $\tau$ increases (correlation between the two traits increases), we observe higher power of the joint test (starting at $\tau =0.4$). This shows that the joint test is more powerful than the marginal tests for moderate or high correlation between traits. We also conducted simulations to evaluate the impact of between‐subject correlation across different time points on the testing power. We observed similar results as the one by varying the between‐subject correlation at the same time point. Due to space limit, the results were rendered in the supplemental file.

FIGURE 2.

The power comparison between the joint test and marginal tests under different correlation coefficient $\tau$ from 0.1 to 0.6 with sample size $N=200$. The exact empirical sizes are given in Table 3

MAF also plays a major role for the inference performance of an association test in general. For the proposed method, the power increases as the MAF $p_A$ increases from 0.1 to 0.5. This is in align with the general conception. In particular, there is a big power improvement as $p_A$ increases from 0.1 to 0.3 as shown in Figure 3.

FIGURE 3.

The power comparison of the joint test under different minor allele frequencies ($P_A=0.1,0.3,0.5$) and different sample sizes ($N=200,500$)

TABLE 3.

Empirical size for the joint and marginal tests under different correlation coefficient $\tau$ from 0.1 to 0.6 with sample size $N=200$

	$\tau =0.1$	$\tau =0.2$	$\tau =0.3$	$\tau =0.4$	$\tau =0.5$	$\tau =0.6$
Joint	0.040	0.040	0.037	0.036	0.034	0.026
Marginal 1	0.038	0.036	0.038	0.039	0.039	0.038
Marginal 2	0.034	0.036	0.039	0.037	0.038	0.037

4. REAL DATA APPLICATION

We applied the proposed multivariate partially linear varying coefficients model and the two‐step hypothesis testing procedure to the Twin Study of Hormones and Behavior across the Menstrual Cycle project ²⁶ from the Michigan State University Twin Registry (MSUTR). ²⁷ , ²⁸ , ²⁹ The goal of the study was to examine associations between changes in estradiol and progesterone levels and emotional eating across the menstrual cycle. Emotional eating was measured with the Dutch Eating Behavior Questionnaire (DEBQ) and negative affect was measured with the Negative Affect scale from the Positive and Negative Affect Schedule (PANAS). The DEBQ assesses the tendency to eat in response to negative emotions while PANAS is used to measure negative emotional states like sadness and anxiety.

In this study, we wanted to examine how genes respond to the hormone change (eg, estrogen) to affect emotional eating measured by DEBQ and PANAS. Since body mass index (BMI) is an important covariate for the study, we included it in the linear component of the model. Although the original study contains twins data, we only included one of the twins in each family in the analysis to make the samples independent. Measurements for each participant were collected for 45 consecutive days, which then were grouped into eight menstrual cycle phases, that is, ovulatory phase (1), transition ovulatory to midluteal (2), midluteal phase (3), transition midluteal to premenstrual (4), premenstrual phase including the first day of menstrual cycle (5), remaining days of menstrual cycle, part of follicular phase (6), follicular phase (7) and transition follicular to ovulatory phase (8). They were grouped into these phases based on profiles of changes in estrogen and progesterone across the cycle. ³⁰ Data that belong to the same phase were averaged to get a phase‐level measure. All individuals were aligned according to the 8 phases for further analysis.

To demonstrate the utility of the method, here we focused on a candidate gene, nuclear receptor coactivator 7 (NCoA7). This gene codes for an estrogen receptor‐associated protein which plays an important role in the cellular response to estrogen. After removing SNPs with MAF $<0.05$, we had 12 SNPs measured on 327 participants for further analysis.

We consider the partially linear varying coefficient model with the two longitudinal traits, namely DEBQ and PANAS, with the form

$$y_{ij}^D={\beta }_0^D(X_{ij})+{\beta }_1^D(X_{ij})G_i+{\alpha }^D{Z}_{ij}+{ϵ}_{ij}^D,$$

$$y_{ij}^P={\beta }_0^P(X_{ij})+{\beta }_1^P(X_{ij})G_i+{\alpha }^P{Z}_{ij}+{ϵ}_{ij}^P.$$

For the $i$th individual measured at menstrual cycle phase $j$, the two longitudinal traits are denoted as $y_{ij}^D$ and $y_{ij}^P$ for DEBQ and PANAS, respectively. One phase dependent covariate BMI is denoted as $Z_{ij}$. $X_{ij}$ refers to the hormone estradiol level, which is standardized to range between 0 and 1 by $\mathrm{\Phi }((E-{\bar{X}}_E)/S_E)$ where $E$ is the original estrogen level, ${\bar{X}}_E$ and $S_E$ are the sample mean and standard deviation of $E$, and $\mathrm{\Phi }$ is the cumulative distribution function of a standard normal. $G_i$ represents the SNP variable and the 12 SNPs were analyzed separately.

We aimed to test if an SNP is associated with the two traits with its effect modified by the estrogen hormone level, that is,

$$H_0:{\beta }_1^D(·)={\beta }_1^P(·)=0v.s.H_1:{\beta }_1^D(·)\ne 0\text{or}{\beta }_1^P(·)\ne 0.$$

We applied the quadratic splines and the exchangeable working correlation structure for this real data analysis. After the Bonferroni correction for the 12 SNPs, we found three SNPs, rs584032, rs6911452, and rs9401855, are significant with the joint test. Table 4 lists the results. The joint test results are all more significant than the marginal tests. The between‐trait correlations between DEBQ and PANAS at the same cycle phase are shown in Figure 4, which shows a quite strong correlation at different phases ranging from 0.36 to 0.61. This explains why the joint test shows stronger significance than the marginal tests. Figure 4 shows the detailed correlation information about the three components: within‐trait correlation, between‐trait correlation at the same time points and across different time points.

TABLE 4.

The test results of the 3 significant SNPs with their rs numbers, the alleles (minor allele shows with bold font), the MAF, and the $P$‐values for the joint test (denoted as $P_{joint}$) and the two marginal tests (denoted as $P_{DEBQ}$ and $P_{PANAS}$)

SNP	Alleles	MAF	$P_{joint}$	$P_{DEBQ}$	$P_{PANAS}$
rs584032	T/A	0.176	2.390e‐4	3.472e‐2	1.937e‐3
rs6911452	A/G	0.089	8.225e‐4	3.491e‐3	4.627e‐3
rs9401855	A/G	0.129	3.758e‐4	5.449e‐2	1.108e‐3

FIGURE 4.

The correlation information including within‐trait correlation, between‐trait correlation at the same and across different cycle phases. The x‐axis and y‐axis represent the 8 cycle phases for the two variables DEBQ and PANAS, respectively

Figure 5 shows the estimated nonparametric coefficient functions for both responses DEBQ and PANAS, with SNP rs9401855 as an example. The point‐wise 95% confidence bands cover a large part of the zero line for the DEBQ, which is consistent with its $P$‐value .0545 from the marginal test. And for PANAS, in the central region, the zero line is outside of the 95% point‐wise confidence bands, which is also consistent with the marginal $P$‐value .0011. The result shows that this SNP interacts with estrogen hormone and only affects PANAS, but not DEBQ. The negative coefficients show that estrogen hormone negatively impacts PANAS. Individuals carrying the GG genotype are more likely to experience negative affective states such as sadness and anxiety, compared to those carrying one or no G allele. From the slightly quadratic effect curve, it can be seen that the negative impact peaks around phase 5‐6, that is, the Premenstrual phase including the first day of menstrual cycle (5) to Remaining days of menstrual cycle, part of follicular phase (6), while less negative impact is observed at the beginning and the end of the eight phased cycle (ie, during the ovulatory phase).

FIGURE 5.

The estimated intercept and slope functions for DEBQ and PANAS from the joint model (red solid curve) and their point‐wise 95% confidence bands (dashed curve)

5. DISCUSSION

Joint analysis of multiple correlated traits can potentially improve the power to identify genetic variants associated with complex traits. However, association analysis focusing on multiple longitudinal traits has not be well studied. Method on G$\times$E interaction with multiple traits under a longitudinal design is even rare. In this article, we proposed a joint multivariate varying coefficient modeling approach to accommodate correlated longitudinal traits and proposed a testing procedure to identify genetic variants associated with multiple longitudinal traits with their effects modified by some environmental factors. By modeling the environmental effect with a nonparametric function, one can estimate the dynamic changing effect of $G$ on $Y$ over the changing values of $X$. The nonparametric function is flexible in the sense that the function is determined by the data without assuming a parametric structure. Both simulation and real data analysis demonstrate the utility of the proposed method.

One difficulty in jointly modeling multiple longitudinal traits is to model the complex correlation structure. For each subject, we should consider correlation between measurements for the same trait at different time points, correlation between measurements at the same time point on different traits, and correlation between measurements at different time points and on different traits. We applied the QIF approach in estimation and testing procedures. There are several advantages for QIF approach. First, the QIF approach only requires correct specification of the mean structure and does not require any joint likelihood in hypothesis testing. Second, it avoids estimating the nuisance correlation structure parameters by assuming that the inverse of working correlation matrix can be approximated by a linear combination of several known basis matrices. Third, when the working correlation structure is misspecified, the QIF is more efficient than the GEE approach. Fourth, the inference function of the QIF approach has an explicit asymptotic form, which provides a model selection criteria and allows us to test whether coefficients are significant or time varying based on the asymptotic results. It is worth mentioning that missing completely at random (MCAR) is assumed in this work, which is a common assumption under the QIF framework when dealing with missing data. ³¹

In the real application, we investigated association of SNPs in a candidate gene with two longitudinal traits DBEQ and PANAS. Although the data were regrouped into eight phases, they still carry the temporal information and can be treated like longitudinal data. The results show that three SNPs passed the Bonferroni threshold with the joint test and the $P$‐values of the joint test are smaller than the individual marginal test. This shows the relative advantage of the joint test. As shown in the simulation study, the joint test can achieve power gain when the traits are correlated. Therefore, it is essential to assess the correlations between traits when fitting multiple traits jointly and conducting joint testing.

Our method was demonstrated with two traits. The method can be extended to multiple longitudinal traits with $L>2$, although the computational cost might increase. In addition, our method is not restricted to a longitudinal study. It also applies to other studies where multiple traits can be measured over a linear scale. For example, in a pharmacogenetic study, multiple drug responses (eg, blood pressure and heart rate) can be measured over different dosage of a drug treatment. The proposed model can be fitted to assess how genes respond to the increasing dosage levels to affect the drug responses. For another example, in a brain imaging genetic study, brain activities in different brain regions can be measured over a spatial scale and can be treated as multiple traits. One can fit the proposed model to understand how genes affect brain activities over a spatial scale.

Supporting information

Data S1: Supporting Information

PROOFS OF THEOREMS

1.

To establish the asymptotic properties for the estimator of $\mathit{\theta }$, we need the following regularity conditions.

• (A1) $\{n_i\}$ is a bounded sequence of integers.
• (A2)The parameter space ${\mathrm{\Omega }}_{\mathit{\theta }}$ is compact and ${\mathit{\theta }}_0$ is an interior point of ${\mathrm{\Omega }}_{\mathit{\theta }}$.
• (A3)The parameter $\mathit{\theta }$ is identified, that is, there is a unique ${\mathit{\theta }}_0\in {\mathrm{\Omega }}_{\mathit{\theta }}$ such that the first moment assumption $E[g_i({\mathit{\theta }}_0)]=0$ holds for $i=1,\dots ,N$, and $E[g_i(\mathit{\theta })]$ is continuous.
• (A4) $E[g(\mathit{\theta })]$ is continuous in $\mathit{\theta }$.
• (A5) ${\bar{C}}_N(\widehat{\mathit{\theta }})=\frac{1}{N}{\sum }_{i=1}^N{g}_i(\widehat{\mathit{\theta }})g_i{(\widehat{\mathit{\theta }})}^T$ converges almost surely to ${\mathbf{C}}_0$, which is a constant and invertible matrix.
• (A6)The first derivative of ${\bar{g}}_N$ exists and is continuous. $\frac{\partial {\bar{g}}_N}{\partial \mathit{\theta }}(\widehat{\mathit{\theta }})$ converges in probability to ${\mathbf{G}}_0$ if $\widehat{\mathit{\theta }}$ converges in probability to ${\mathit{\theta }}_0$.

Proof of Theorem 1

$\widehat{\mathit{\theta }}$ exists because (7) has zero as a lower bound and the global minimum exists. To prove the consistency, first, the estimator $\widehat{\mathit{\theta }}$ is obtained by minimizing (7), then we have

$$\frac{1}{N}{Q}_N(\widehat{\mathit{\theta }})+{\lambda }_N{\widehat{\mathit{\theta }}}^{T}D\widehat{\mathit{\theta }}\le \frac{1}{N}{Q}_N({\mathit{\theta }}_0)+{\lambda }_N{\mathit{\theta }}_0^{T}D{\mathit{\theta }}_0.$$ (A1)

Since

$$\frac{1}{N}{Q}_N({\mathit{\theta }}_0)={\bar{g}}_N^T({\mathit{\theta }}_0){\bar{C}}_N^{-1}({\mathit{\theta }}_0){\bar{g}}_N({\mathit{\theta }}_0)=o(1),$$

by the strong law of large number and (A5), and ${\lambda }_N=o(1)$,

$$\frac{1}{N}{Q}_N({\mathit{\theta }}_0)+{\lambda }_N{\mathit{\theta }}_0^{T}D{\mathit{\theta }}_0\underset{}{\overset{a.s.}{\to }}0.$$

Thus, we can obtain from (A1) that

$$\frac{1}{N}{Q}_N(\widehat{\mathit{\theta }})={\bar{g}}_N^T(\widehat{\mathit{\theta }}){\bar{C}}_N^{-1}(\widehat{\mathit{\theta }}){\bar{g}}_N(\widehat{\mathit{\theta }})\underset{}{\overset{a.s.}{\to }}0.$$ (A2)

Since the parameter space ${\mathrm{\Omega }}_{\mathit{\theta }}$ is compact, by Glivenko‐Cantelli theorem,

$$\underset{\theta \in {\mathrm{\Omega }}_{\mathit{\theta }}}{\sup }\left|{\bar{g}}_N(\mathit{\theta })-E[g(\mathit{\theta })]\right|\underset{}{\overset{a.s.}{\to }}0.$$

Hence, by (A5) and the continuity mapping theorem,

$$\left|{\bar{g}}_N^T(\widehat{\mathit{\theta }}){\bar{C}}_N^{-1}(\widehat{\mathit{\theta }}){\bar{g}}_N(\widehat{\mathit{\theta }})-E{[g(\widehat{\mathit{\theta }})]}^T{\mathbf{C}}_0^{-1}E[g(\widehat{\mathit{\theta }})]\right|\underset{}{\overset{a.s.}{\to }}0.$$

Combined with (A2), we get

$$E{[g(\widehat{\mathit{\theta }})]}^T{\mathbf{C}}_0^{-1}E[g(\widehat{\mathit{\theta }})]\underset{}{\overset{a.s.}{\to }}0.$$ (A3)

Suppose $\widehat{\mathit{\theta }}$ is not a strong consistent estimator of $\mathit{\theta }$, then there exists a neighborhood of the true parameter ${\mathit{\theta }}_0$, say $U$, such that $\widehat{\mathit{\theta }}\in U^c$. Since $E{[g(\mathit{\theta })]}^T{\mathbf{C}}_0^{-1}E[g(\mathit{\theta })]$ is a continuous function and $U^c$ is compact, there exists a point ${\mathit{\theta }}^{\ast }\in U^c$ such that

$$E{[g({\mathit{\theta }}^{\ast })]}^T{\mathbf{C}}_0^{-1}E[g({\mathit{\theta }}^{\ast })],$$

achieve its minimum in $U^c$. By the identification of $\mathit{\theta }$ in (A3), there is a unique ${\mathit{\theta }}_0\in {\mathrm{\Omega }}_{\theta }$ satisfying $E[g({\mathit{\theta }}_0)]=0$, and we have

$$E{[g({\mathit{\theta }}^{\ast })]}^T{\mathbf{C}}_0^{-1}E[g({\mathit{\theta }}^{\ast })]>0,$$

which contradicts (A3). Hence, $\widehat{\mathit{\theta }}$ is a consistent estimator of $\mathit{\theta }$.

Proof of Theorem 2

The estimate of $\mathit{\theta }$ satisfies

$$\frac{1}{N}\frac{\partial Q_N}{\partial \mathit{\theta }}(\widehat{\mathit{\theta }})+2{\lambda }_{N}D\widehat{\mathit{\theta }}=0.$$

By Taylor's expansion, we obtain

$$\frac{1}{N}\frac{\partial Q_N}{\partial \mathit{\theta }}({\mathit{\theta }}_0)+2{\lambda }_{N}D{\mathit{\theta }}_0+\left(\frac{1}{N}\frac{{\partial }^2{Q}_N}{\partial {\mathit{\theta }}^2}(\tilde{\mathit{\theta }})+2{\lambda }_{N}D\right)(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)=0,$$

where $\tilde{\mathit{\theta }}$ is some value between $\widehat{\mathit{\theta }}$ and ${\mathit{\theta }}_0$. Thus, we can have

$$\widehat{\mathit{\theta }}-{\mathit{\theta }}_0=-{\left(\frac{1}{N}\frac{{\partial }^2{Q}_N}{\partial {\mathit{\theta }}^2}(\tilde{\mathit{\theta }})+2{\lambda }_{N}D\right)}^{-1}\left(\frac{1}{N}\frac{\partial Q_N}{\partial \mathit{\theta }}({\mathit{\theta }}_0)+2{\lambda }_{N}D{\mathit{\theta }}_0\right).$$ (A4)

Since $\widehat{\mathit{\theta }}$ converges to ${\mathit{\theta }}_0$ in probability and $\tilde{\mathit{\theta }}$ is between $\widehat{\mathit{\theta }}$ and ${\mathit{\theta }}_0$, by (A5) and (A6) we can get

$$\begin{array}{llll}\frac{1}{N}\frac{{\partial }^2{Q}_N}{\partial {\mathit{\theta }}^2}(\tilde{\mathit{\theta }})& =2{\frac{\partial {\bar{g}}_N}{\partial \mathit{\theta }}}^T(\tilde{\mathit{\theta }}){\bar{C}}_N^{-1}(\tilde{\mathit{\theta }})\frac{\partial {\bar{g}}_N}{\partial \mathit{\theta }}(\tilde{\mathit{\theta }})+o_p(1)& & \\ & \underset{}{\overset{p}{\to }}2{\mathbf{G}}_0^T{\mathbf{C}}_0^{-1}{\mathbf{G}}_0.& & \end{array}$$

When ${\lambda }_N=o(N^{-1/2})$,

$${\left(\frac{1}{N}\frac{{\partial }^2{Q}_N}{\partial {\mathit{\theta }}^2}(\tilde{\mathit{\theta }})+2{\lambda }_{N}D\right)}^{-1}=\frac{1}{2}{({\mathbf{G}}_0^T{\mathbf{C}}_0^{-1}{\mathbf{G}}_0)}^{-1}+o_p(N^{-1/2}).$$

Similarly, since

$$\frac{1}{N}\frac{\partial Q_N}{\partial \mathit{\theta }}({\mathit{\theta }}_0)={\frac{\partial {\bar{g}}_N}{\partial \mathit{\theta }}}^T({\mathit{\theta }}_0){\bar{C}}_N^{-1}({\mathit{\theta }}_0){\bar{g}}_N({\mathit{\theta }}_0),$$

and ${\lambda }_N=o(N^{-1/2})$, we have

$$\frac{1}{N}\frac{\partial Q_N}{\partial \mathit{\theta }}({\mathit{\theta }}_0)+2{\lambda }_{N}D{\mathit{\theta }}_0={\mathbf{G}}_0^T{\mathbf{C}}_0^{-1}{\bar{g}}_N({\mathit{\theta }}_0)+o(N^{-1/2}).$$

Therefore, (A4) can be written as

$$\sqrt{N}(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)=-\sqrt{N}{({\mathbf{G}}_0^T{\mathbf{C}}_0^{-1}{\mathbf{G}}_0)}^{-1}{\mathbf{G}}_0^T{\mathbf{C}}_0^{-1}{\bar{g}}_N({\mathit{\theta }}_0)+o_p(1).$$ (A5)

By Central Limit Theorem,

$$\sqrt{N}{\bar{g}}_N({\mathit{\theta }}_0)\underset{}{\overset{d}{\to }}N(\mathbf{0},{\mathbf{C}}_0).$$ (A6)

Using (A5) and (A6), we obtain

$$\sqrt{N}(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)\underset{}{\overset{d}{\to }}N(\mathbf{0},{({\mathbf{G}}_0^T{\mathbf{C}}_0^{-1}{\mathbf{G}}_0)}^{-1}).$$

Proof of Theorem 3

By Taylor's expansion,

$$Q({\mathit{\psi }}_0,{\mathit{\zeta }}_0)-Q(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})={\left(\begin{array}{c}{\mathit{\psi }}_0-\widehat{\mathit{\psi }}\\ {\mathit{\zeta }}_0-\widehat{\mathit{\zeta }}\end{array}\right)}^T\dot{Q}(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})+\frac{1}{2}{\left(\begin{array}{c}{\mathit{\psi }}_0-\widehat{\mathit{\psi }}\\ {\mathit{\zeta }}_0-\widehat{\mathit{\zeta }}\end{array}\right)}^T\ddot{Q}({\mathit{\psi }}^{\ast },{\mathit{\zeta }}^{\ast })\left(\begin{array}{c}{\mathit{\psi }}_0-\widehat{\mathit{\psi }}\\ {\mathit{\zeta }}_0-\widehat{\mathit{\zeta }}\end{array}\right),$$

where $({\mathit{\psi }}^{\ast },{\mathit{\zeta }}^{\ast })$ is some value between $({\mathit{\psi }}_0,{\mathit{\zeta }}_0)$ and $(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})$. We can also obtain from Taylor's expansion that

$$Q({\mathit{\psi }}_0,{\mathit{\zeta }}_0)-Q({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})={({\mathit{\zeta }}_0-\tilde{\mathit{\zeta }})}^T{\dot{Q}}_{\zeta }({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})+\frac{1}{2}{({\mathit{\zeta }}_0-\tilde{\mathit{\zeta }})}^T{\ddot{Q}}_{\zeta \zeta }({\mathit{\psi }}_0,{\mathit{\zeta }}^{\ast \ast })({\mathit{\zeta }}_0-\tilde{\mathit{\zeta }}),$$

where ${\mathit{\zeta }}^{\ast \ast }$ is between ${\mathit{\zeta }}_0$ and $\tilde{\mathit{\zeta }}$. From conditions in (9), we have

$$\dot{Q}(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})=0\text{and}\dot{Q}({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})=0.$$

Hence

$$Q({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})-Q(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})=\frac{1}{2}{\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)}^T\ddot{Q}({\mathit{\psi }}^{\ast },{\mathit{\zeta }}^{\ast })\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)-\frac{1}{2}{\left(\begin{array}{c}\mathbf{0}\\ \tilde{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)}^T\ddot{Q}({\mathit{\psi }}_0,{\mathit{\zeta }}^{\ast \ast })\left(\begin{array}{c}\mathbf{0}\\ \tilde{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right).$$

If we expand ${\dot{Q}}_{\zeta }({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})$ about ${\mathit{\zeta }}_0$, and ${\dot{Q}}_{\zeta }(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})$ about $({\mathit{\psi }}_0,{\mathit{\zeta }}_0)$, we obtain

$$0={\dot{Q}}_{\zeta }({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})={\dot{Q}}_{\zeta }({\mathit{\psi }}_0,{\mathit{\zeta }}_0)+{\ddot{Q}}_{\zeta \zeta }(\tilde{\mathit{\zeta }}-{\mathit{\zeta }}_0)+O_p(N^{-\frac{1}{2}}),$$

$$0={\dot{Q}}_{\zeta }(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})={\dot{Q}}_{\zeta }({\mathit{\psi }}_0,{\mathit{\zeta }}_0)+{\ddot{Q}}_{\zeta \psi }(\widehat{\mathit{\psi }}-{\mathit{\psi }}_0)+{\ddot{Q}}_{\zeta \zeta }(\widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0)+O_p(N^{-\frac{1}{2}}).$$

The above two equations give us

$$\tilde{\mathit{\zeta }}-{\mathit{\zeta }}_0={\ddot{Q}}_{\zeta \zeta }^{-1}{\ddot{Q}}_{\zeta \psi }(\widehat{\mathit{\psi }}-\mathit{\psi })+(\widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0)+O_p(N^{-\frac{1}{2}}),$$

which can be written as

$$\left(\begin{array}{c}\mathbf{0}\\ \tilde{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)=\left(\begin{array}{cc}\mathbf{0}& \mathbf{0}\\ {\ddot{Q}}_{\zeta \zeta }^{-1}{\ddot{Q}}_{\zeta \psi }& I\end{array}\right)\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right).$$

Then $Q({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})-Q(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})$ can be written as

$${\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)}^T\left\{\ddot{Q}({\mathit{\psi }}^{\ast },{\mathit{\zeta }}^{\ast })-{\left(\begin{array}{cc}\mathbf{0}& \mathbf{0}\\ {\ddot{Q}}_{\zeta \zeta }^{-1}{\ddot{Q}}_{\zeta \psi }& I\end{array}\right)}^T\ddot{Q}({\mathit{\psi }}_0,{\mathit{\zeta }}^{\ast \ast })\left(\begin{array}{cc}\mathbf{0}& \mathbf{0}\\ {\ddot{Q}}_{\zeta \zeta }^{-1}{\ddot{Q}}_{\zeta \psi }& I\end{array}\right)\right\}\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)+O_p(N^{\frac{1}{2}}),$$

which is asymptotically equivalent to

$$\begin{array}{llll}& {\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)}^T\left\{\left(\begin{array}{cc}{J}_{\psi \psi }& J_{\psi \zeta }\\ J_{\zeta \psi }& J_{\zeta \zeta }\end{array}\right)-{\left(\begin{array}{cc}\mathbf{0}& \mathbf{0}\\ J_{\zeta \zeta }^{-1}{J}_{\zeta \psi }& I\end{array}\right)}^T\ddot{Q}({\mathit{\psi }}_0,{\mathit{\zeta }}^{\ast \ast })\left(\begin{array}{cc}\mathbf{0}& \mathbf{0}\\ J_{\zeta \zeta }^{-1}{J}_{\zeta \psi }& I\end{array}\right)\right\}\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)& & \\ & ={(\widehat{\mathit{\psi }}-{\mathit{\psi }}_0)}^T(J_{\psi \psi }-J_{\psi \zeta }{J}_{\zeta \zeta }^{-1}{J}_{\zeta \psi })(\widehat{\mathit{\psi }}-{\mathit{\psi }}_0).& & \end{array}$$

By theorem 3.2 in Hansen, ²¹

$$\left(\begin{array}{c}\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\\ \widehat{\mathit{\zeta }}-{\mathit{\zeta }}_0\end{array}\right)\underset{}{\overset{d}{\to }}{N}_d\left\{\left(\begin{array}{c}\mathbf{0}\\ \mathbf{0}\end{array}\right),{\left(\begin{array}{cc}{J}_{\psi \psi }& J_{\psi \zeta }\\ J_{\zeta \psi }& J_{\zeta \zeta }\end{array}\right)}^{-1}\right\}.$$

Therefore,

$$\widehat{\mathit{\psi }}-{\mathit{\psi }}_0\underset{}{\overset{d}{\to }}{N}_{d1}\left(0,{(J_{\psi \psi }-J_{\psi \zeta }{J}_{\zeta \zeta }^{-1}{J}_{\zeta \psi })}^{-1}\right),$$

thus, $Q({\mathit{\psi }}_0,\tilde{\mathit{\zeta }})-Q(\widehat{\mathit{\psi }},\widehat{\mathit{\zeta }})$ follows ${\chi }_{d_1}^2$ asymptotically.

Wang H, Zhang J, Klump KL, Alexandra Burt S, Cui Y. Multivariate partial linear varying coefficients model for gene‐environment interactions with multiple longitudinal traits. Statistics in Medicine. 2022;41(19):3643–3660. doi: 10.1002/sim.9440

DATA AVAILABILITY STATEMENT

Research data used for real data analysis are from a different group and are not shared. R code used to implement the method can be downloaded at https://github.com/Honglang/MPLVC.