Inference for high-dimensional linear mixed-effects models: A quasi-likelihood approach

Abstract

Linear mixed-effects models are widely used in analyzing clustered or repeated measures data. We propose a quasi-likelihood approach for estimation and inference of the unknown parameters in linear mixed-effects models with high-dimensional fixed effects. The proposed method is applicable to general settings where the dimension of the random effects and the cluster sizes are possibly large. Regarding the fixed effects, we provide rate optimal estimators and valid inference procedures that do not rely on the structural information of the variance components. We also study the estimation of variance components with high-dimensional fixed effects in general settings. The algorithms are easy to implement and computationally fast. The proposed methods are assessed in various simulation settings and are applied to a real study regarding the associations between body mass index and genetic polymorphic markers in a heterogeneous stock mice population.

1. Introduction

The results of scientific experiments are often subject to environmental effects as experimental units can be grouped and settled in diverse environments, where the observations within the same group can be dependent as a cluster. Clustered data commonly arise in many fields, such as biology, genetics, and economics. Linear mixed-effects models provide a flexible tool for analyzing such clustered data, which include repeated measures data, longitudinal data, and multilevel data (Pinheiro and Bates 2000; Goldstein 2011). The linear mixed-effects models incorporate both the fixed and random effects, where the random effects induce correlations among the observations within each cluster and accommodate the cluster structure. In many genomic and economic studies, the dimension of the covariates can be large and possibly much larger than the sample size. A variety of statistical models and approaches have been proposed and studied for analyzing high-dimensional data. However, most of them are restricted to dealing with independent observations, such as linear models and generalized linear models. Statistical inference for high-dimensional linear mixed-effects models remains to be a challenging problem. In this work, we consider estimation and inference of unknown parameters in high-dimensional mixed-effects models.

For ease of presentation, we use the setting for clustered data to present a linear mixed-effects model. For repeated measurement data, the repeated measures form a cluster. Let i = 1, …, n be the cluster indices. For the i-th cluster, we have a response vector $y_i\in {ℝ}^{m_i}$, a design matrix for the fixed effects $X^i\in {ℝ}^{m_i\times p}$, and a design matrix for the random effects $Z^i\in {ℝ}^{m_i\times q}$, where m_i is the size of the i-th cluster. A linear mixed-effects model (Laird and Ware 1982) can be written as

$$y_i=X^i{\beta }^{*}+Z^i{\gamma }_i+{ϵ}_i,i=1,\dots ,n,$$ (1)

where ${\beta }^{*}\in {ℝ}^p$ is the vector of the fixed effects, ${\gamma }_i\in {ℝ}^q$ is the vector of the random effects of the i-th cluster, and ${ϵ}_i\in {ℝ}^{m_i}$ is the noise vector of the i-th cluster. For i = 1, …, n, we assume γ_i and ϵ_i are independently distributed with mean zero and variance $\Psi \in {ℝ}^{q\times q}$ and ${\sigma }_e^2{I}_{m_i}$, respectively. Detailed assumptions are given in Sections 2 and 3.

Much existing literature on linear mixed-effects models assumes that the number of random effects q and cluster sizes m_i are fixed. Without special emphasis, we say a fixed-dimensional setting if p, q, and ${\left\{m_i\right\}}_{i=1}^n$ are all fixed numbers, and a high-dimensional setting if p is large and possibly much larger than N, where $N=\sum _{i=1}^n{m}_i$ is the total sample size. We refer to γ_i and ϵ_i as the random components.

1.1. Related literature

In the fixed-dimensional setting, many methods have been proposed to jointly estimate the fixed effects and variance parameters. We refer to Gumedze and Dunne (2011) for a comprehensive review. Among them, the maximum likelihood estimators (MLEs) and restricted MLEs are most popular for estimation and inference in linear mixed-effects models. Restricted MLEs can produce unbiased estimators of the variance components in the low-dimensional setting but it is not applicable in the high-dimensional setting. Furthermore, these likelihood-based estimators rely heavily on the normality assumptions of the random components. Computationally, maximizing the likelihood can generally lead to a nonconvex optimization problem that typically has multiple local maxima. Hence, the performance of likelihood-based methods lacks of guarantees in real applications.

As an alternative, Sun et al. (2007) proposed moment estimators of the fixed effects and variance parameters for a random effect varying-coefficient model. Peng and Lu (2012) considered such moment estimators for fixed-dimensional linear mixed-effects models. Their proposed estimators have closed-form solutions and are computationally efficient. The consistency and asymptotic normality of these estimators are justified under certain conditions in the fixed-dimensional setting. Ahmn et al. (2012) proposed another moment-based method for the estimation and selection of the variance components of the random effects in the fixed-dimensional setting. This method works especially well when the number of the random effects is as large as the cluster sizes, i.e. m₁ = … = m_n = q.

For inference of variance components in the fixed-dimensional setting, the likelihood ratio, score, and Wald tests (Stram and Lee 1994; Lin 1997; Verbeke and Molenberghs 2003; Demidenko 2004) are broadly used. However, when testing the existence of the random effects, the asymptotic distribution of the likelihood ratio is usually a mixture of chi-square distributions (Miller 1977; Self and Liang 1987). Since these methods are based on the MLEs or restricted MLEs as initial estimators, they also suffer from the drawbacks of likelihood-based methods discussed above.

In the high-dimensional setting, the problems are much more challenging. Assuming fixed cluster sizes, Schelldorfer et al. (2011) analyzed the rate of convergence for the global maximizer of the ℓ₁-penalized likelihood with fixed designs. As mentioned before, the analysis for the global optimum may not apply to the realizations due to the existence of local maxima. Fan and Li (2012) studied the fixed effects and random effects selection in a high-dimensional linear mixed-effects model when the cluster sizes are balanced, i.e. (max_i m_i) / (min_i m_i) < ∞. The selection consistency requires minimum signal strength conditions regarding the fixed effects and the random effects. Bradic et al. (2019) considered testing a single coefficient of the fixed effects in the high-dimensional linear mixed-effects models with fixed cluster sizes, fixed number of random effects, and sub-Gaussian designs. The theoretical analyses in all three aforementioned papers require the positive definiteness of the covariance matrix of the random effects. This condition takes prior knowledge on the existence of the random effects and can be hard to fulfill in applications. Moreover, the optimal convergence rate of parameter estimation remains unknown. In fact, estimators of fixed effects in Schelldorfer et al. (2011) and Bradic et al. (2019) may not be rate optimal for estimation according to our analysis. Finally, estimation and inference of the variance components in the high-dimensional setting remain largely unknown.

The problems of estimation and inference of the fixed effects in linear mixed-effects models are related to high-dimensional linear models. Many penalized methods have been proposed for prediction, estimation, and variable selection in high-dimensional linear models; see, for example, Tibshirani (1996); Fan and Li (2001); Zou (2006); Candes and Tao (2007); Meinshausen and Bühlmann (2010); Zhang (2010). Statistical inference on a low-dimensional component of a high-dimensional regression vector has been considered and studied in linear models and generalized linear models with “debiased” estimators (Zhang and Zhang 2014; van de Geer et al. 2014; Javanmard and Montanari 2014), and the minimaxity and adaptivity of confidence intervals have been studied in Cai and Guo (2017) and Cai et al. (2020). The idea of debiasing has also been studied and extended to solve other statistical problems, such as statistical inference in Cox models (Fang et al. 2017), simultaneous inference (Zhang and Cheng 2017; Dezeure et al. 2017), and semi-supervised inference (Cai and Guo 2020).

1.2. Our contributions

In this paper, we develop a simple but powerful method for inference of the unknown parameters in high-dimensional linear mixed-effects models. Our method is applicable to the settings where the number of random effects can possibly be large and the cluster sizes can be either fixed or growing, balanced or unbalanced. The proposed method is easy to implement and the optimization in each step is either analytic or convex.

Based on a proxy of the true covariance matrix, we develop a penalized quasi-likelihood approach for fixed effects estimation. The proposed estimator is minimax rate optimal under general conditions. We further develop a debiased estimator for hypothesis testing and construction of confidence intervals for the fixed effects. The proposed estimator does not require normality assumptions or the structural assumptions on the variance components. We further apply the idea of quasi-likelihood to estimate the variance components and prove its optimality under certain conditions.

Our analysis provides a novel insight for understanding and simplifying the linear mixed-effects models by approximating the true unknown covariance matrix of the random components with some simple proxy matrices. In this way, one separates the tasks of estimating the fixed effects and variance components and avoids the nuisance parameters in each optimization step. This improves the computational efficiency and simplifies the theoretical analysis.

1.3. Notation

Throughout the paper, we use i to index the i-th cluster and k to index the k-th observation in each cluster. Let y, γ, ϵ, and X be obtained by stacking vectors y_i, γ_i, ϵ_i, and matrices Xⁱ underneath each other, respectively. Let $Z\in {ℝ}^{N\times (nq)}$ be a block diagonal matrix with the i-th block being Zⁱ. Let ${\Sigma }_{\theta }^i=Z^i\Psi Z^i+{\sigma }_e^2{I}_{m_i}$ and ${\Sigma }_{\theta }\in {ℝ}^{N\times N}$ be a block diagonal matrix with the i-th block being ${\Sigma }_{\theta }^i$. Let ${\Sigma }_z^i={\left(Z^i\right)}^{\top }{Z}^i/m_i$ and ${\Sigma }_{z,x}^i={\left(Z^i\right)}^{\top }{X}^i/m_i$, i = 1, …, n. For a random variable $u\in ℝ$, define its sub-Gaussian norm as $\Vert u{\Vert }_{{\psi }_2}=\underset{l\ge 1}{\text{sup}}{l}^{-1/2}{\mathbb{E}}^{1/l}\left[|u{|}^l\right]$. We refer to $\Vert u{\Vert }_{{\psi }_2,Z}=\underset{l\ge 1}{\text{sup}}{l}^{-1/2}{\mathbb{E}}^{1/l}\left[|u{|}^l\mid Z\right]$ as the conditional sub-Gaussian norm of u. For a random vector $U\in {ℝ}^{n_0}$, define its sub-Gaussian norm as $\Vert U{\Vert }_{{\psi }_2}=\underset{\Vert v{\Vert }_2=1,v\in {ℝ}^{n_0}}{\text{sup}}\Vert \langle U,v\rangle {\Vert }_{{\psi }_2}$. Define it conditional sub-Gaussian norm as $\Vert U{\Vert }_{{\psi }_2,Z}=\underset{\Vert v{\Vert }_2=1,v\in {ℝ}^{n_0}}{\text{sup}}\Vert \langle U,v\rangle {\Vert }_{{\psi }_2,Z}$.

Let $A\in {ℝ}^{n_0\times n_0}$ be a symmetric matrix. A ⪰ 0 means that A is semi-positive definite and A ≻ 0 means that A is positive definite. Let Λ_max (A) and Λ_min (A) denote the largest and smallest eigenvalues of A, respectively. Let ∥A∥₂ denote Λ_max(A). Let $\Vert A{\Vert }_F^2=\text{Tr}\left(A^{\top }A\right)$, where Tr(A) is the trace of matrix A. Let c, c₀, c₁, …, C, C₀, C₁, … denote some generic positive constants that can vary in different statements.

1.4. Organization of the paper

The rest of the paper is organized as follows. Section 2 introduces the idea of quasi-likelihood and a procedure for the fixed effects inference. Section 3 provides a theoretical analysis for the inference procedures proposed in Section 2. Section 4 introduces estimators for the variance components and provides upper and lower bounds. Numerical performance of the proposed methods is investigated in Section 5 in various simulation settings. The proposed methods are applied in Section 6 to analyze a real study on the associations between the body mass index and genetic variants in a stock mice population where the cage effect is modeled as a random effect. A discussion is given in Section 7. Proofs and more numerical results are provided in the Supplementary Materials.

2. Inference for fixed effects: the method

In many applications of the linear mixed-effects models, inference of the fixed effects is of main interest. In this section, we present our method for fixed effects inference and describe its motivations. We assume that the vector of fixed effects β* is sparse such that ∥β*∥₀ ≤ s with s unknown. We consider model (1) where p, s, and q can grow and p can be much larger than N. The cluster sizes ${\left\{m_i\right\}}_{i=1}^n$ can be either fixed or grow with n.

2.1. Motivations of the proposed method

For fixed effects estimation in model (1), the main challenges are posed by the high-dimensionality of the fixed effects and the clustered structure of the observations. Before developing a new method, it is helpful to understand the new challenges posed by the cluster structures in model (1) in terms of estimation and inference. For this purpose, we study the consequences of mis-specifying a linear mixed-effects model as a standard linear model.

Applying Lasso (Tibshirani 1996) directly to the observations generated from (1), we analyze

$${\widehat{\beta }}^{(lm)}=\underset{b\in {ℝ}^p}{\text{argmin}}\left\{\frac{1}{2N}\Vert y-Xb{\Vert }_2^2+{\lambda }^{(lm)}\Vert b{\Vert }_1\right\}$$ (2)

for some tuning parameter λ^(lm) > 0. In a typical analysis of the Lasso, the convergence rate of ${\widehat{\beta }}^{(lm)}$ depends on the restricted isometries of the sample covariance matrix, X^⊤X / N, the sparsity of the true coefficients, and the so-called “empirical process” part of the problem, ∥X^⊤(y – Xβ*) / N∥_∞. It is known that for linear models with row-wise independent sub-Gaussian (X, y), the “empirical process” part is of order $\sqrt{\text{log}p/N}$, which gives the optimal convergence rate in ℓ₂-norm. In the following proposition, we study the size of “empirical process” part when the true model is (1).

Proposition 2.1 (The rate of Lasso for linear mixed-effects models).

Suppose that the responses y_i are generated with respect to model (1) and each row of X is independently generated with covariance matrix Σ_x|z conditioning on Z. Then for any fixed j ∈ {1, …, p},

$$\mathbb{E}\left[{\left|\frac{1}{N}{X}_{.,j}^{\top }(Z\mathit{\gamma }+\mathit{ϵ})\right|}^2\mid Z\right]=\frac{{\left({\Sigma }_{x\mid z}\right)}_{j,j}{\sigma }_e^2}{N}+\frac{{\left({\Sigma }_{x\mid z}\right)}_{j,j}\sum _{i=1}^n{m}_i\text{Tr}\left(\Psi {\Sigma }_z^i\right)}{N^2}+\frac{\sum _{i=1}^n{m}_i^2{\Vert {\Psi }^{1/2}\mathbb{E}\left[{\sum }_{z,x}^i|Z\right]\Vert }_2^2}{N^2}.$$ (3)

If Ψ is positive definite and ${\left\{m_i\right\}}_{i=1}^n$ have bounded diagonal elements, then the second term on the right hand side of (3) is ≍ q / N. If it further holds that $\mathbb{E}\left[{\Sigma }_{z,x}^i|Z\right]\ne 0$, i.e. X and Z are correlated, then the third term can be ≳ min_1≤i≤n m_i / N. That is, the Lasso may not be rate optimal for clustered data if either q grows, or, ${\left\{m_i\right\}}_{i=1}^n$ grow and X and Z are correlated. On the other hand, if the q and $m_i^{’}\text{s}$ are all constant, it is not hard to prove that the original Lasso is still rate optimal for model (1).

Therefore, proper methods need to be developed for high-dimensional linear mixed-effects models under general conditions on q and ${\left\{m_i\right\}}_{i=1}^n$. The main challenge comes from the correlation among observations induced by the random effects. For the i-th block, the covariance of the random components is ${\Sigma }_{{\theta }^{*}}^i=Z^i\Psi {\left(Z^i\right)}^{\top }+{\sigma }_e^2{I}_{m_i}$, which involves unknown parameters. We consider a proxy of ${\Sigma }_{{\theta }^{*}}^i$ as

$${\Sigma }_a^i=a{Z}^i{\left(Z^i\right)}^{\top }+I_{m_i}$$

with some predetermined constant a > 0. The following proposition shows that this approximation is valid up to some scaling constant. Let ${\Sigma }_a\in {ℝ}^{N\times N}$ be the block diagonal matrix with the i-th block being ${\Sigma }_a^i$.

Proposition 2.2.

If Ψ is positive definite, then for any constant a > 0,

$$\text{min}\left\{\frac{1}{{\sigma }_e^2},\frac{a}{{\Lambda }_{\text{max}}(\Psi )}\right\}{\Sigma }_a^{-1}\hspace{1em}{\Sigma }_{{\theta }^{*}}^{-1}\hspace{1em}\text{max}\left\{\frac{1}{{\sigma }_e^2},\frac{a}{{\Lambda }_{\text{min}}(\Psi )}\right\}{\Sigma }_a^{-1}.$$

Therefore, if Ψ has positive and bounded eigenvalues, ${\Sigma }_{{\theta }^{*}}^{-1}$ and ${\Sigma }_a^{-1}$ are of the same rate and only differ by constants. This property of ${\Sigma }_a^{-1}$ is crucial to achieve the general results in this work. A broader class of proxy matrices have been considered in Fan and Li (2012) for variable selection and in Bradic et al. (2019) for hypothesis testing, which include ${\Sigma }_a^{-1}$ as a special case. As reviewed in Section 1.1, afore-mentioned two papers considered relatively restrictive scenarios in terms of group sizes and the dimension of the random effects. It is not clear whether the desired property proved in Proposition 2.2 holds for the general class of proxy matrices.

2.2. The quasi-likelihood approach

We consider a quasi-likelihood approach which replaces ${\Sigma }_{{\theta }^{*}}^i$ with ${\Sigma }_a^i$ for some constant a > 0 in the likelihood function for Gaussian mixed-effects models. Specifically, let X_a and y_a denote the transformed observations such that $\left(X_a,y_a\right)=\left({\Sigma }_a^{-1/2}X,{\Sigma }_a^{-1/2}y\right)$.

First, we estimate the fixed effects via the Lasso based on the transformed data. For some fixed a > 0, define

$$\widehat{\beta }=\underset{\beta \in {ℝ}^p}{\text{argmin}}\left\{\frac{1}{2\text{Tr}\left({\Sigma }_a^{-1}\right)}{\Vert y_a-X_a\beta \Vert }_2^2+\lambda \Vert \beta {\Vert }_1\right\}$$ (4)

for some tuning parameter λ > 0. The quantity $\text{Tr}\left({\Sigma }_a^{-1}\right)$ can be viewed as the effective sample size in the current problem and its magnitude is studied in Remark 3.1. The choice of a will be studied theoretically in Section 3 and numerically in Section 5.

Given the task of making inference for ${\beta }_j^{*}$, we propose the following debiased estimator. For $\widehat{\beta }$ defined in (4),

$${\widehat{\beta }}_j^{(db)}={\widehat{\beta }}_j+\frac{{\widehat{w}}_j^{\top }\left(y_a-X_a\widehat{\beta }\right)}{{\widehat{w}}_j^{\top }{\left(X_a\right)}_{.,j}},$$ (5)

where ${\widehat{w}}_j\in {ℝ}^N$ can be viewed as a correction score. It can be computed via another Lasso regression (Zhang and Zhang 2014; van de Geer et al. 2014) or via a quadratic optimization (Zhang and Zhang 2014; Javanmard and Montanari 2014). For computational convenience, we consider the Lasso approach based on the transformed data. Define the correction score ${\widehat{w}}_j={\left(X_a\right)}_{.,j}-{\left(X_a\right)}_{.,-j}{\widehat{\kappa }}_j$, where

$${\widehat{\kappa }}_j=\underset{{\kappa }_j\in {ℝ}^{p-1}}{\text{argmin}}\left\{\frac{1}{2\text{Tr}\left({\Sigma }_a^{-1}\right)}\left||{\left(X_a\right)}_{.,j}-{\left(X_a\right)}_{.,-j}{\kappa }_j\right|{|}_2^2+{\lambda }_j{\Vert {\kappa }_j\Vert }_1\right\},$$ (6)

for some tuning parameter λ_j > 0. A two-sided 100× (1 – α)% confidence interval for ${\beta }_j^{*}$ can be constructed as

$${\widehat{\beta }}_j^{(db)}\pm z_{\alpha /2}\sqrt{{\widehat{V}}_j},$$ (7)

where z_τ is the τ-th quantile of a standard normal distribution and ${\widehat{V}}_j$ is an estimator of the variance of ${\widehat{\beta }}_j^{(db)}$. We propose to use the following empirical variance estimate

$${\widehat{V}}_j=\frac{\sum _{i=1}^n{\left[{\left({\widehat{w}}_j^i\right)}^{\top }\left(y_a^i-X_a^i\widehat{\beta }\right)\right]}^2}{{\left({\widehat{w}}_j^{\top }{\left(X_a\right)}_{.,j}\right)}^2},$$ (8)

where $\widehat{\beta }$ is the initial Lasso estimator (4), ${\widehat{w}}_j^i\in {ℝ}^{m_i}$ is the i-th sub-vector of ${\widehat{w}}_j$ such that ${\widehat{w}}_j={\left({\left({\widehat{w}}_j^1\right)}^{\top },\dots ,{\left({\widehat{w}}_j^n\right)}^{\top }\right)}^{\top }$, and $y_a^i$ is the i-th sub-vector of y_a. The idea of empirical variance estimator has been considered in Bühlmann and van de Geer (2015) to deal with the misspecified linear models. The format of (8) is however different from the one for linear models because it is an average over n groups rather than N observations. In this work, ${\widehat{V}}_j$ serves as a convenient alternative to the limiting distribution-based variance estimators. In fact, the limiting distribution of ${\widehat{\beta }}_j^{(db)}$ involves nuisance parameters coming from the complicated variance components. By using the empirical residuals of the transformed data, we bypass the estimation of the nuisance parameters.

3. Inference for fixed effects: theoretical guarantees

In this section, we provide theoretical guarantees for the estimators described in Section 2.2. We first detail our assumptions.

Condition 3.1 (Sub-Gaussian random components).

The random noises ϵ_i,k, i = 1, …, n, k = 1, …, m_i, are independent with mean zero and variance $0<{\sigma }_e^2<K_0<\infty$. The sub-Gaussian norms of ϵ_i,k, are upper bounded by K₀. The random effects ${\gamma }_i\in {ℝ}^q$, i = 1, …, n, are independent with mean zero and covariance Ψ K₁I_q for some positive constant K₁. For i = 1, …, n, ϵ_i and γ_i are independent of each other and are independent of (Xⁱ, Zⁱ). The sub-Gaussian norms of ${\Sigma }_{{\theta }^{*}}^{-1/2}(Z\mathit{\gamma }+\mathit{ϵ})$ are upper bounded by K₀.

Condition 3.1 assumes sub-Gaussian random components while that classical linear mixed-effects models always assume Gaussian random components (Pinheiro and Bates 2000). Hence, Condition 3.1 is less restrictive and is more robust to model misspecifications than the classical assumptions. In addition, we do not require Ψ to be strictly positive definite. A scenario of singular Ψ is that some components of the random effects do not exists such that some diagonal elements of Ψ are zero.

Regarding the conditions on the designs, the estimation and inference in the linear mixed-effects models are usually conditioning on Z in order to maintain the cluster structure. Schelldorfer et al. (2011) and Fan and Li (2012) assume both X and Z are fixed. Jiang et al. (2016) considers estimation and inference in a misspecified linear model when both X and Z are random. Bradic et al. (2019) assumes X is sub-Gaussian with mean zero and Z is fixed, which implies that X and Z are independent. In the current work, we consider random designs satisfying the following condition.

Condition 3.2 (Sub-Gaussian [INEQ-START).

X conditioning on Z] Conditioning on Z, each row of X is independent with mean zero and covariance matrix Σ_x|z such that 0 < K_* ≤ Λ_min (Σ_x|z) ≤ Λ_max (Σ_x|z) ≤ K* < ∞. Conditioning on Z, the conditional sub-Gaussian norms of $X_{k,.}^i$ are upper bounded by K₀.

In Condition 3.2, we assume sub-Gaussian X and Z have mean independence, i.e., $\mathbb{E}[X\mid Z]=0$ for simplicity. This is slightly weaker than assuming X and Z are mutually independent and it holds when Z is deterministic including the random intercept model. In Section 3.4, we study the performance of our proposal when $\mathbb{E}[X\mid Z]\ne 0$.

3.1. Fixed effects estimation

In this subsection, we analyze the theoretical performance of (4) under Conditions 3.1 and 3.2. Define

$${\lambda }_a^{*}=\frac{\sqrt{\text{Tr}\left({\Sigma }_a^{-1}{\Sigma }_{{\theta }^{*}}{\Sigma }_a^{-1}\right)\text{log}p}}{\text{Tr}\left({\Sigma }_a^{-1}\right)}.$$

Lemma 3.1 (Fixed effects estimation with quasi-likelihood based Lasso).

Assume that Conditions 3.1 and 3.2 hold true. There exists a constant c₁ such that for $\lambda \ge c_1{\lambda }_a^{*}$ and $\text{Tr}\left({\Sigma }_a^{-1}\right)\gg s\text{log}p$, we have with probability at least 1 − 2exp(−log p),

$$\left||\widehat{\beta }-{\beta }^{*}\right|{|}_1\le C_{1}s\lambda ,\left||\widehat{\beta }-{\beta }^{*}\right|{|}_2\le C_2\sqrt{s}\lambda ,\text{and}\frac{1}{\text{Tr}\left({\Sigma }_a^{-1}\right)}\left||X_a\left(\widehat{\beta }-{\beta }^{*}\right)\right|{|}_2^2\le C_{3}s{\lambda }^2$$ (9)

for some positive constants C₁, C₂, and C₃. Moreover, for any a > 0,

$${\lambda }_a^{*}\le \sqrt{\frac{\left({\Lambda }_{\text{max}}(\Psi )/a+{\sigma }_e^2\right)\text{log}p}{\text{Tr}\left({\Sigma }_a^{-1}\right)}}.$$

Remark 3.1.

For any a ≥ 0,

$$\sum _{i=1}^n\text{max}\left\{m_i-q,0\right\}\le \text{Tr}\left({\Sigma }_a^{-1}\right)\le N.$$

Lemma 3.1 provides upper bounds for the prediction error and the estimation errors in ℓ₁-norm and ℓ₂-norm. By setting a to be a positive constant, the ℓ₂-error of $\widehat{\beta }$ is of order $\sqrt{s\text{log}p/\text{Tr}\left({\Sigma }_a^{-1}\right)}$. Remark 3.1 studies the magnitude of the effective sample size $\text{Tr}\left({\Sigma }_a^{-1}\right)$. As pointed out by a reviewer, in the case of equal group sizes and q / m ≤ c₀ < 1, $\text{Tr}\left({\Sigma }_a^{-1}\right)\asymp N$, i.e. the convergence rates are the same as the rates in linear models. Revoking Proposition 2.1, it shows that $\widehat{\beta }$ has a faster convergence rate than ${\widehat{\beta }}^{(lm)}$ in the regime that q grows but remains relatively small to m.

The results of Lemma 3.1 hold for any positive constant a. Different choices of a can affect the constants in the upper bound and the empirical performance of the method. To understand the optimal choice of a, we prove the following remark.

Remark 3.2 (Effect of a).

Suppose Ψ = η*I_q. For any given n, p, q, ${\left\{m_i\right\}}_{i=1}^n$, $a={\eta }^{*}/{\sigma }_e^2$ minimizes ${\lambda }_a^{*}$ for a∈ (0, ∞). If it further holds that η* ≠ 0 and q < max_i≤n m_i, then ${\lambda }_0^{*}>{\lambda }_a^{*}$ for any $a\in \left(0,{\lambda }_a^{*}\right]$.

Remark 3.2 gives the optimal choice of a for Ψ = η*I_q. In this case, setting $a={\eta }^{*}/{\sigma }_e^2$ minimizes ${\lambda }_a^{*}$ and hence minimizes the upper bound on the estimation errors when other parameters and constants are fixed. The optimal choice of a is intuitive as it mimics the MLE procedure. Furthermore, when the random effects exist and q < max_i≤n m_i, then setting a = 0 is strictly worse than the proposed quasi-likelihood approach with $a\in \left(0,{\lambda }_a^{*}\right]$. We mention that the condition q < max_i≤n m_i is sufficient but not necessary. This remark sheds lights on the choice of a in general settings as any semi-positive definite Ψ can be upper and lower bounded by diagonal matrices. From the optimization perspective, we treat a as a tuning parameter in the optimization (4). In Section 5, we carefully examine the effect of a on estimation accuracy in numerical experiments.

3.2. Rate optimality of the proposed estimator

In this subsection, we study the minimax optimality of proposed estimator for the fixed effects. We consider

$$X_{k,.}^i\mid Z{~}_{i.i.d.}N\left(0,{\Sigma }_x\right),{\gamma }_i{~}_{i.i.d.}N(0,\Psi )\text{and}{ϵ}_{i,k}{~}_{i.i.d.}N\left(0,{\sigma }_e^2\right),$$ (10)

k = 1, …, m_i, i, = 1, …, n. Consider the following parameter space

$$\begin{gathered} \Xi (s,Z)=\{v=\left({\beta }^{*},\Psi ,{\sigma }_e^2,{\Sigma }_x,Z\right):{\Vert {\beta }^{*}\Vert }_0\le s,0<{\sigma }_e^2\le K_0,0\Psi K_1, \\ 1/K^{*}\le {\Lambda }_{\text{min}}\left({\Sigma }_x\right)\le {\Lambda }_{\text{max}}\left({\Sigma }_x\right)\le K^{*}<\infty \}, \end{gathered}$$ (11)

where K* ≥ 1. We see that (10) and (11) define a special case of Conditions 3.1 and 3.2. We prove the minimax optimal rate of convergence in ℓ₂-norm with respect to Ξ(s).

Theorem 3.1. (Minimax lower bounds for estimating the fixed effects).

Suppose that (1) and (10) are true. If $s\le c\text{min}\left\{\text{Tr}\left({\Sigma }_a^{-1}\right)/\text{log}p,p^{\nu }\right\}$ for 0 < ν < 1 / 2 and c > 0, then there exists some constant c₁ > 0 such that for any fixed a > 0,

$$\underset{\widehat{\beta }}{\text{inf}}\underset{v\in \Xi (s,Z)}{\text{sup}}{\mathbb{P}}_v\left({\Vert \widehat{\beta }-{\beta }^{*}\Vert }_2^2\ge c_1\frac{s\text{log}\left(p/s^2\right)}{\text{Tr}\left({\Sigma }_a^{-1}\right)}\mid Z\right)\ge \frac{1}{4}.$$

Together with (9), this shows that $\widehat{\beta }$ is minimax rate optimal in ℓ-error in the parameter space Ξ(s, Z). In the proof, we use the minimax optimality of ℓ₁-penalized MLE, which has ${\Sigma }_{{\theta }^{*}}^{-1}$ as the weighting matrix and use Proposition 2.2 to show the equivalence of MLE and proposed estimator in term of convergence rate.

3.3. Statistical inference of the fixed effects

Debiased estimators can be used for statistical inference of linear combinations of regression coefficients in high-dimensional linear models (Zhang and Zhang 2014; van de Geer et al. 2014; Javanmard and Montanari 2014). Under certain conditions, the debiased estimators are asymptotically normal and can be used to construct confidence interval with optimal lengths (Cai and Guo 2017). To make inference of ${\beta }_j^{*}$, we consider the debiased estimator proposed in (5). Let H_j be the support of ${\left({\Sigma }_{x\mid z}^{-1}\right)}_{.,j}$.

Theorem 3.2. (Asymptotic normality of the debiased estimator).

Assume Conditions 3.1 and 3.2. Let $\lambda \wedge {\lambda }_j\ge c_1\sqrt{\text{log}p/\text{Tr}\left({\Sigma }_a^{-1}\right)}$ with a large enough constant c₁. For ${\widehat{\beta }}_j^{(db)}$ defined in (5), if ${(s\text{log}p)}^2\vee \text{log}n{\text{max}}_i{m}_i\ll \text{Tr}\left({\Sigma }_a^{-1}\right){\Lambda }_{\text{min}}\left({\Sigma }_a^{-1/2}{\Sigma }_{{\theta }^{*}}{\Sigma }_a^{-1/2}\right)$ and $\left|H_j\right|\text{log}p\ll \text{Tr}\left({\Sigma }_a^{-1}\right)$, then it holds that

$$V_j^{-1/2}\left({\widehat{\beta }}_j^{(db)}-{\beta }_j^{*}\right)=R_j+o_P(1),$$

where $R_j\stackrel{D}{\to }N(0,1)$ for

$$V_j=\frac{{\widehat{w}}_j^{\top }{\Sigma }_a^{-1/2}{\Sigma }_{{\theta }^{*}}{\Sigma }_a^{-1/2}{\widehat{w}}_j}{{\left\{{\widehat{w}}_j^{\top }{\left(X_a\right)}_{.,j}\right\}}^2}.$$

The magnitude of V_j satisfies

$$V_j=\frac{{\left({\Sigma }_{x\mid z}^{-1}\right)}_{j,j}\text{Tr}\left({\Sigma }_a^{-1}{\Sigma }_{{\theta }^{*}}{\Sigma }_a^{-1}\right)}{{\text{Tr}}^2\left({\Sigma }_a^{-1}\right)}\left(1+o_P(1)\right).$$ (12)

Theorem 3.2 shows the asymptotic normality of the proposed debiased estimator under the given conditions. The convergence rate of ${\widehat{\beta }}_j^{(db)}$ is $V_j^{1/2}$ with magnitude provided in (12). Remark 3.2 helps to understand the effect of a on the inference results. As Σ_x|z is positive definite, V_j is proportional to ${\left({\lambda }_a^{*}\right)}^2$ for any given p. Hence, using the debiased Lasso for linear models, i.e., setting a = 0, can lead to large V_j and low power in statistical inference. We verify these arguments numerically in Section 5. The sparsity of ${\left({\Sigma }_{x\mid z}^{-1}\right)}_{.,j}$ guarantees that ${\widehat{\kappa }}_j$ converges to its probabilistic limit so that the central limit theorem can be justified. When Ψ is positive definite, the sample size condition for asymptotic normality is ${(s\text{log}p)}^2\vee \text{log}n{\text{max}}_i{m}_i\vee \left|H_j\right|\text{log}p\ll \text{Tr}\left({\Sigma }_a^{-1}\right)$. When Ψ is singular, it is sufficient to require ${(s\text{log}p)}^2{\text{max}}_i{m}_i\vee \text{log}n{\text{max}}_i{m}_i^2\vee \left|H_j\right|\text{log}p\ll \text{Tr}\left({\Sigma }_a^{-1}\right)$.

Theorem 3.2 is related to the results in some other works. When there is no random effect components, i.e. Z = 0, the conditions and conclusions of Theorem 3.2 recover the conditions and conclusions for the debiased Lasso in linear models, say, in Theorem 2.4 of van de Geer et al. (2014). In comparison to BCG19 of Bradic et al. (2019), the limiting distribution of their test statistic under the null hypothesis does not require sparse regression coefficients but requires $\left|H_j\right|=o(\sqrt{n}/\text{log}p/\text{log}n)$, using our notations. The power analysis for their test statistic requires $\text{max}\left\{s,\left|H_j\right|\right\}=o(\sqrt{n}/\text{log}p/\text{log}n)$. In comparison, the sample size condition in our work (in the fixed q and m scenario) is $s=o(\sqrt{n}/\text{log}p)$ and | H_j | = o(n / log p), which is weaker. More importantly, the realization of ${\widehat{\beta }}_j^{(db)}$ does not rely on the null hypothesis and hence can be directly used to construct confidence intervals. We examine the empirical performance of these two different approaches in Section 5.

If one has a consistent estimator of V_j, the confidence intervals of the fixed effects can be constructed based on the limiting distribution of ${\widehat{\beta }}_j^{(db)}$. However, a plug-in estimate of V_j would require the knowledge of the structures of variance components and extra efforts on estimation. In the following, we show that an empirical estimator of V_j, ${\widehat{V}}_j$ defined in (8), is consistent under mild conditions. Let $c_n=\text{log}n{\text{max}}_i{m}_i/{\Lambda }_{\text{min}}\left({\Sigma }_a^{-1/2}{\Sigma }_{{\theta }^{*}}{\Sigma }_a^{-1/2}\right)$.

Lemma 3.2 (Convergence rate of the variance estimator).

Under the conditions of Theorem 3.2, for ${\widehat{V}}_j$ defined in (8),

$$\left|{\widehat{V}}_j/V_j-1\right|=O_P\left(c_n^{1/2}{\text{Tr}}^{-1/2}\left({\Sigma }_a^{-1}\right)+c_n\frac{s\log p}{\text{Tr}\left({\Sigma }_a^{-1}\right)}\right).$$

Lemma 3.2 implies that the proposed variance estimator is consistent if $c_n=o\left({\text{Tr}}^{1/2}\left({\Sigma }_a^{-1}\right)\right)$ and the conditions of Theorem 3.2 hold ture. The quantity c_n is to account for the correlations within clusters. The proposed ${\widehat{V}}_j$ is robust in the sense that it does not rely on the specific structure of the variance components and is consistent under mild conditions. Based on Theorem 3.2 and Lemma 3.2, hypothesis testing and constructing confidence intervals are both achievable. The asymptotic validness of the proposed confidence interval (7) is guaranteed.

We conclude this section with a further comment on the benefits of using the quasi-likelihood. In fact, even if we have consistent estimators of the variance parameters, say $\widehat{\theta }$, using proxy matrix Σ_a to compute the debiased estimator is still favorable over using ${\Sigma }_{\widehat{\theta }}$. First, using $\widehat{\theta }$ can bring the complex dependence structure to R_j as the correction score would also depend on the random components. This makes it difficult to justify the asymptotic normality of R_j. Second, ${\Sigma }_{\widehat{\theta }}^{-1}$ may not approximate ${\Sigma }_{{\theta }^{*}}^{-1}$ well enough in the sense that the magnitude of the error in $\widehat{\theta }$ can be larger than the magnitude of the bias of the debiased Lasso estimator. As a result, the sample size condition for the asymptotic normality may be impaired.

3.4. Results for possibly correlated X and Z

In this subsection, we consider a relaxed version of Condition 3.2 that allows for correlation between X and Z.

Condition 3.3.

Conditioning on Z, each row of X is independently distributed with covariance matrix Σ_x|z such that 0 < K_* ≤ Λ_min (Σ_x|z) ≤ Λ_max (Σ_x|z) ≤ K* < ∞. Conditioning on Z, the conditional sub-Gaussian norms of $X_{k,.}^i$ are upper bounded by K₀. Moreover, ${\text{max}}_{1\le j\le p}{\Vert \mathbb{E}\left[X_{.,j}\mid Z\right]\Vert }_2^2\le c_1\text{Tr}\left({\Sigma }_a^{-1}\right)$ for some large enough c₁.

Revoking that ${\Vert X_{.,j}\Vert }_2^2\le CN$ and Remark 3.1, a sufficient condition for the last statement to hold is q / (min_i m_i) ≤ c₀ <1.

Lemma 3.3 (Fixed effects estimation with Lasso).

Assume that Conditions 3.1 and 3.3 hold true. There exist large enough constants c₁ and c₂ such that for $\lambda \ge c_1\sqrt{\left({\sigma }_e^2+K_1/a\right)\text{log}p/\text{Tr}\left({\Sigma }_a^{-1}\right)}$ and $\text{Tr}\left({\Sigma }_a^{-1}\right)\gg s\text{log}p$, we have with probability at least 1 − 2exp(−c₂ log p),

$$\left||\widehat{\beta }-{\beta }^{*}\right|{|}_1\le C_{1}s\lambda ,\left||\widehat{\beta }-{\beta }^{*}\right|{|}_2\le C_2\sqrt{s}\lambda ,\text{and}\frac{1}{\text{Tr}\left({\Sigma }_a^{-1}\right)}\left||X_a\left(\widehat{\beta }-{\beta }^{*}\right)\right|{|}_2^2\le C_{3}s{\lambda }^2$$ (13)

for some large enough constants C₁, C₂, and C₃.

Under Condition 3.3, for any constant a > 0, the effective sample size for the proposed approach is still of order $\text{Tr}\left({\Sigma }_a^{-1}\right)$. However, we may not have a clear understanding of the optimal choice of a under current conditions but a can still be chosen by cross-validation in applications.

For inference of the fixed effects, one issue caused by the correlation between X and Z is that the limit of ${\widehat{\kappa }}_j$ in (6) can depend on a and its sparsity is hard to guarantee. If its limit is sparse indeed, then the central limit theory of Theorem 3.2 still holds. If its limit is not sparse, then one may consider the debiasing scheme for linear models with the initial estimator computed by the quasi-likelihood approach. We show in the Supplementary Materials (Theorem A) that our proposed debiased estimator with a = 0 in (5) and (6) is robust to the correlation between X and Z. However, its asymptotic normality requires stronger sample size conditions when Ψ is positive definite and it can have significantly wider confidence intervals and hence lower statistical power. We examine this method in Section 5 numerically.

4. Variance components estimation

In this section, we consider estimating the unknown parameters of variance components. While fixed effects inference can be of main interest in many problems, estimation of variance components can provide insights into the structure of the data. As far as we know, this problem has not been studied in existence of high-dimensional fixed effects. We will demonstrate that the idea of quasi-likelihood approach can be applied to estimating the variance components in high-dimensional linear mixed-effects models.

We parameterize Ψ as follows. Let ${\eta }^{*}\in {ℝ}^d$ be the true parameter such that

$$\Psi ={\Psi }_{{\eta }^{*}}=\sum _{j=1}^d{\eta }_j^{*}{G}_j\in {ℝ}^{q\times q},$$ (14)

where G₁, … G_d are symmetric basis matrices that are linearly independent in the sense that

$$\sum _{j=1}^d{c}_j{G}_j=0\text{iff}{c}_1=\dots =c_d=0.$$ (15)

The dimension d is allowed to grow to infinity. The structure of ${\Psi }_{{\eta }^{*}}$ (14) incorporates most commonly used models in applications, such as the random intercept model and the models used in twin or family studies (Wang et al. 2011). One should note that any symmetric Ψ can be represented via (14) with η* being the vector of its upper diagonal elements. Without loss of generality, we assume the basis matrices have a constant scale, i.e. max_{1≤j ≤d} ∥G_j∥₂ ≤ C < ∞.

4.1. Estimating the variance components

A widely used approach for estimating the variance components is the Gaussian maximum likelihood method. However, this approach is highly restricted to the Gaussian assumptions. We consider a different approach that deals with sub-Gaussian random components. We first split the data into two folds: Let I₁ ∪ =I₂ = [n], I₁ ∩ I₂ = ∅, and | I₁ |≈| I₂ |≈ n / 2. Let ${\widehat{\beta }}^{(2)}$ be an initial estimate of β* with the second half of the data ${\left\{X^i,Z^i,y_i\right\}}_{i\in I_2}$. We compute the residuals $\widehat{r_i}=y_i-X_i{\widehat{\beta }}^{(2)}$ for i ∈ I₁ and estimate ${\sigma }_e^2$ via

$${\sigma }_e^2=\frac{1}{\sum _{i\in I_1}\text{Tr}\left(P_{Z^i}^{\perp }\right)}\sum _{i\in I_1}{\Vert P_{Z^i}^{\perp }{\widehat{r}}_i\Vert }_2^2.$$ (16)

Next, we estimate η* via

$$\widehat{\eta }=\underset{\eta \in {ℝ}^d}{\text{argmin}}\sum _{i\in I_1}\left||{\left({\Sigma }_a^i\right)}^{-1/2}\left({\widehat{r}}_i{\widehat{r}}_i^{\top }-Z^i{\Psi }_{\eta }{\left(Z^i\right)}^{\top }-{\sigma }_e^2{I}_{n_i}\right){\left({\Sigma }_a^i\right)}^{-1/2}\right|{|}_F^2,$$ (17)

where constant K ≥ K₂ and ${\sigma }_e^2$ is obtained via (16).

The rationale of (16) is that the observations $P_{Z^i}^{\perp }\left(y_i-X^i{\beta }^{*}\right)$ have covariance matrix ${\sigma }_e^2{P}_{Z^i}^{\perp }$ which only involves the target parameter ${\sigma }_e^2$. Replacing β* with its quasi-likelihood estimate gives (16). This estimator is meaningful only when $\sum _{i\in I_1}\text{Tr}\left(P_{Z^i}^{\perp }\right)>0$, i.e. $\sum _{i\in I_1}{m}_i\text{max}\left\{0,1-q/m_i\right\}>0$. The rationale of (17) comes from the MLE. One can check that the derivative of the target function in (17) would be the score function with respect to η if we replace Σ_a with the MLE estimate of ${\Sigma }_{{\theta }^{*}}$. Different from the MLE, we estimate ${\sigma }_e^2$ and η* separately. This is because a joint estimation of η* and ${\sigma }_e^2$ may have poor performance. The reason is that, loosely speaking, the observed data involves N independent observations of the random noise and n independent observations of random effects. When N ≫ n, the convergence rate for estimating ${\sigma }_e^2$ and η* can have different magnitudes and a joint estimation can lead to ill-positioned Hessian matrix and non-sharp convergence rate. The sample splitting is for technical reasons and it is for proving that the estimation error of $\widehat{\eta }$ is independent of the error of the fixed-effects estimation.

Computationally, ${\sigma }_e^2$ in (16) is a one-step estimator and (17) involves a convex optimization, which can be easily implemented. On the other hand, sample splitting can lead to sub-optimal finite sample performance and it is worthwhile to perform a cross-fitting step. That is, one can run another round of (16) and (17) with samples in the two folds switched and report the average of two estimates as the final estimate.

4.2. Upper bound analysis

In this subsection, we analyze the proposed estimator of the variance components. Let $D_G\in {ℝ}^{d\times d}$ be such that

$${\left\{D_G\right\}}_{j,k}=\text{Tr}\left(G_j{G}_k\right).$$ (18)

The matrix D_G only depends on the pre-specified basis and Λ_min (D_G) > 0 as G_j, j = 1, …, d, are linearly independent.

Theorem 4.1. (Convergence rate of variance components estimates)

Assume that Conditions 3.1 and 3.2 hold and $\sum _{i\in I_2}\text{Tr}\left({\left({\Sigma }_a^i\right)}^{-1}\right)\gg s\text{log}p$. Then

$$|{\sigma }_e^2-{\sigma }_e^2|=O_P\left({\left(\sum _{i\in I_1}\text{max}{m}_i\left\{0,1-q/m_i\right\}\right)}^{-1/2}+\frac{s\text{log}p}{\sum _{i\in I_2}\text{Tr}\left({\left({\sum }_a^i\right)}^{-1}\right)}\right).$$

If further n ≥ c₁ log d for some large enough c₁, ${\text{min}}_{1\le i\le n}{\Lambda }_{\text{min}}\left({\Sigma }_z^i\right)\ge c_0/m_i>0$, and 0 < c₁ ≤ Λ_min (D_G) ≤ Λ_max (D_G) ≤ c₂ < ∞, then

$${\Vert \widehat{\eta }-{\eta }^{*}\Vert }_2=O_P\left(\sqrt{\frac{d\text{log}d}{n}}\right).$$

The convergence rate of ${\sigma }_e^2$ depends on the effective sample size in I₁ as well as the estimation error of ${\widehat{\beta }}^{(2)}$. In comparison to the rate of variance estimation in linear models (Verzelen 2012), the current result replaces the total sample size with the effective sample size. On the other hand, $\widehat{\eta }$ has the typical parametric rate when there are d unknown parameters and n independent observations of random effects. The estimation error of $\widehat{\eta }$ is independent of the error of the fixed effects estimation.

In terms of conditions, D_G depends on pre-specified basis matrices and it eigenvalues are positive and bounded in many cases. Consider the class of basis matrices where $G_{j,k}\in {ℝ}^{q\times q}$ such that (G_{j, k})_{l, q} = (G_{j, k})_{q, l} = 1 if l = j, q = k and (G_{j, k})_{l, q} = 0 otherwise. For any 1 ≤ d ≤ q(q + 1) / 2, it is easy to check that in this case Λ_min (D_G) =Λ_max (D_G) =1. For independent sub-Gaussian $Z_{k,.}^i$, ${\text{min}}_{1\le i\le n}{\Lambda }_{\text{min}}\left({\Sigma }_z^i\right)\ge c_0$ with high probability if q log n ≪ min_1≤i≤m m_i. To summarize, the estimators proposed in Section 4.1 are mainly for the scenario where q ≤ c₀ min_1≤i≤n m_i for sufficiently small c₀.

4.3. Rate optimality of estimating variance components

Now we turn to study the minimax lower bound for estimating the variance parameters. We consider the random components satisfying (10) and parameter space Ξ(s, Z) (11).

Theorem 4.2. (Minimax lower bounds for estimation of variance components.)

Suppose that (1) and (10) are true. If $s\le c\text{min}\left\{\text{Tr}\left({\Sigma }_a^{-1}\right)/\text{log}p,p^{\nu }\right\}$ for 0 < ν < 1 / 2 and c > 0 for some c₀ > 0, then there exists some constants c₁ − c₃ > 0 such that

$$\underset{{\sigma }_e^2}{\inf }\underset{v\in \Xi (s,Z)}{\text{sup}}{\mathbb{P}}_v\left(|{\sigma }_e^2-{\sigma }_e^2|\ge c_1{\text{Tr}}^{-1/2}\left({\Sigma }_a^{-1}\right)+c_2\frac{s\text{log}\left(p/s^2\right)}{\text{Tr}\left({\Sigma }_a^{-1}\right)}\mid Z\right)\ge \frac{1}{4}$$

If further Λ_max (D_G) ≤ C < ∞,

$$\underset{\widehat{\eta }}{\text{inf}}\underset{\mathit{v}\in \Xi (s,Z)}{\text{sup}}{\mathbb{P}}_{\mathit{v}}\left({‖\widehat{\eta }-{\eta }^{*}‖}_2\ge c_3{n}^{-1/2}\mid Z\right)\ge \frac{1}{4}.$$

Theorem 4.2 and Theorem 4.1 together imply that ${\sigma }_e^2$ is rate optimal in Ξ(s, Z) under the conditions of Theorem 4.2 if when $\text{Tr}\left({\Sigma }_a^{-1}\right)\asymp \sum _{i=1}^n{m}_i\text{max}\left\{0,1-q/m_i\right\}$. As explained after Lemma 3.1, in the case where group sizes are equal and q / m ≤ c₀ <1, $\sum _{i=1}^n\text{max}\left\{0,m_i-q\right\}\asymp \text{Tr}\left({\Sigma }_a^{-1}\right)\asymp N$. Moreover, $\widehat{\eta }$ is rate optimal when d is finite. When d grows, regularized estimators of η* can have smaller estimation error than $\widehat{\eta }$, similar to the famous Stein’s phenomenon.

5. Simulation results

In this section, we present simulation results to evaluate the empirical performance of the proposed methods and compare it with some related methods. We examine the effect of a on estimation and inference of the fixed effects.

We generate data as follows. We set N = 144 and p = 300. Each row of (X, Z) are i.i.d. generated from a normal distribution with mean zero and covariance such that Σ_x = I_p, Σ_z = I_q, and (Σ_{x, z})_{k, j} = ρ^j for 1≤ j, k ≤ q and (Σ_{x, z})_k,. = 0 for k > q. That is, the correlation between X_j and Z is nonzero if j ≤ p and is 0 if j > q. The random noises are i.i.d. generated via ${ϵ}_i~N\left(0,0.25I_{m_i}\right)$ and the random effects are i.i.d. generated via γ_i ~ N(0, Ψ). We consider q ∈ {2, 8, 14}. The matrix Ψ will be specified later. The responses y are generated via model (1) with s = 5 and β_1:5 = (1, 0.5, 0.2, 0.1, 0.05)^⊤ and equal cluster sizes, i.e. m₁ =…= m_n = m. Each setting is replicated with 300 independent Monte Carlo simulations.

5.1. Statistical inference for fixed effects

We first examine the empirical performance of the proposed confidence intervals (7) and hypothesis testing based on ${\widehat{\beta }}_j^{(db)}$. We consider two covariance matrices of random effects, a “positive definite Ψ” where Ψ =_{j, k} 0.56^|j−k| for 1 ≤ j, k ≤ q, and a “singular Ψ” with a diagonal Ψ where Ψ =_{j, j} 0.56 for 1≤ j ≤ q / 2 and Ψ_{j, j} = 0 otherwise. For the proposed method, we first choose a by cross-validation using the error criteria ${\Vert y-X\widehat{\beta }(a)\Vert }_2^2$, where $\widehat{\beta }(a)$ the the proposed estimate associated with a specific a. The tuning parameter λ is chosen as ${\sigma }^{(\mathit{\text{init}})}\sqrt{2\text{log}p/N}$, where σ^(init) is computed via the scaled-Lasso (Sun and Zhang 2012) with observations {X_a, y_a}. For computing ${\widehat{\beta }}_j^{(db)}$, the tuning parameters λ_j are set to be ${\sigma }_x\sqrt{2\text{log}p/N}$, where σ_x is computed via the scaled-Lasso with observations {(X_a)_{.,− j},(X_a)_{., j}}. The tuning parameters for BCG19 are chosen as in Section 5 of Bradic et al. (2019).

We see from Table 1 that the coverage probabilities of the proposed confidence intervals are close to the nominal level in most scenarios. It shows that the proposed method is robust to large m and q and singular Ψ. We see that the confidence intervals have shorter lengths when m increases. This is because when q is fixed and m grows, the effective sample size $\text{Tr}\left({\Sigma }_a^{-1}\right)$ increases and the proposed estimators have smaller estimation errors. See Table 1 in the Supplementary Materials for details. When q grows and m is fixed, the effective sample size $\text{Tr}\left({\Sigma }_a^{-1}\right)$ is smaller and the proposed estimators have larger estimation errors. The results for ρ = 0.2 are reported in the Table 2 of the Supplementary Materials.

Table 1.

95%-confidence intervals given by the proposed approach with positive definite Ψ and singular Ψ when ρ = 0.“cov(0.5)” and “cov(0)” denote the coverage probabilities for β_j = 0.5 and β_j = 0, respectively. “sd(0.5)” and “ sd(0)” denote the standard deviations for β_j = 0.5 and β_j = 0, respectively.

q	m	Positive definite Ψ				Singular Ψ
		cov(0.5)	cov(0)	sd(0.5)	sd(0)	cov(0.5)	cov(0)	sd(0.5)	sd(0)
2	4	0.940	0.957	0.068	0.062	0.953	0.943	0.062	0.056
	8	0.943	0.967	0.063	0.053	0.938	0.981	0.058	0.049
	12	0.943	0.967	0.061	0.049	0.967	0.948	0.059	0.047
8	4	0.943	0.960	0.195	0.177	0.957	0.943	0.128	0.111
	8	0.960	0.940	0.148	0.123	0.933	0.919	0.105	0.088
	12	0.943	0.973	0.106	0.083	0.957	0.976	0.085	0.066
14	4	0.937	0.953	0.276	0.264	0.976	0.948	0.173	0.153
	8	0.937	0.947	0.243	0.217	0.924	0.929	0.158	0.132
	12	0.933	0.950	0.202	0.166	0.981	0.924	0.148	0.112

In Table 2, we report the type-I error and power of our proposed method and those of BCG19. The computational time for our proposal is around 8s per experiment and that for BCG19 is around 20s per experiment. Ideally, the rejection rate for the true null should be close to 5% and the rejection rates for ${\beta }_j^{*}\in \{1,0.5,0.2\}$ should be larger than 5%. We see that both our proposal and BCG19 are effective at controlling the type-I error. However, BCG19 is less powerful than our proposal when q is large and Ψ is positive definite. When Ψ is singular, two methods have comparable performance in most scenarios.

Table 2.

The rejection rate for testing $H_0:{\beta }_j^{*}=0$ at 95% level for ${\beta }_j^{*}\in \{1,0.5,0.2,0\}$ with positive definite Ψ (p.d.) and singular Ψ when ρ = 0.

Ψ	q	m	Proposed				BCG19
			1	0.5	0.2	0	1	0.5	0.2	0
p.d.	2	4	1	1	0.793	0.043	1	1	0.627	0.043
		8	1	1	0.880	0.033	1	1	0.850	0.040
		12	1	1	0.940	0.033	1	1	0.936	0.040
	8	4	0.997	0.713	0.163	0.040	0.987	0.593	0.150	0.067
		8	1	0.923	0.243	0.060	0.987	0.747	0.173	0.060
		12	1	1	0.423	0.027	1	0.840	0.207	0.030
	14	4	0.943	0.437	0.117	0.047	0.867	0.327	0.123	0.037
		8	0.967	0.487	0.113	0.053	0.927	0.397	0.107	0.057
		12	0.993	0.610	0.157	0.050	0.930	0.477	0.150	0.060
singular	2	4	1	1	0.895	0.057	1	1	0.900	0.029
		8	1	1	0.952	0.020	1	1	0.943	0.024
		12	1	1	0.914	0.052	1	1	0.927	0.057
	8	4	1	0.995	0.362	0.057	1	0.976	0.371	0.052
		8	1	0.986	0.438	0.071	1	0.986	0.438	0.062
		12	1	1	0.638	0.024	1	1	0.567	0.047
	14	4	1	1	0.638	0.024	1	1	0.567	0.048
		8	1	0.895	0.228	0.043	1	0.890	0.233	0.052
		12	1	0.895	0.229	0.067	1	0.890	0.233	0.052

Table 3 demonstrates the effect of a on estimation and inference of the fixed effects. The results for singular Ψ are reported in the Supplementary Materials. We see that choosing a = 0 can lead to large estimation errors and significantly wider confidence intervals. This implies that the Lasso for linear models is less accurate than our proposed methods with a > 0. In all the scenarios of (q, m), we see that the estimation error first decreases as a increases and then increases as a increases. This phenomenon agrees with Remark 3.2. For the inference results, the proposed confidence interval has the desired coverage probabilities as long as a is not too large. We see that setting a = 0 has coverage probabilities close to the nominal level but the confidence intervals are significantly wider than setting a > 0. This implies that using the linear debiased Lasso can lead to low power in hypothesis testing for mixed-effects models.

Table 3.

Effect of different a on sum of squared error (SSE) for estimating β* and on the accuracy of confidence intervals with positive definite Ψ and ρ = 0. “cov(0.5)” and “cov(0)” denote the coverage probabilities for β_j = 0.5 and β_j = 0, respectively. “sd(0.5)” and “sd(0)” denote the standard deviations for β_j = 0.5 and β_j = 0, respectively.

(q, m)	a	SSE	$\text{Tr}\left({\Sigma }_a^{-1}\right)$	cov(0.5)	cov(0)	sd(0.5)	sd(0)
(2,4)	0	0.321	144.0	0.948	0.967	0.138	0.122
	2	0.134	87.3	0.962	0.962	0.074	0.065
	4	0.113	81.4	0.933	0.952	0.070	0.062
	8	0.111	77.7	0.933	0.957	0.068	0.062
	16	0.103	75.2	0.900	0.967	0.065	0.060
	32	0.105	73.8	0.890	0.933	0.065	0.060
(8,8)	0	0.753	144.0	0.943	0.943	0.261	0.235
	2	0.338	34.5	0.952	0.962	0.150	0.122
	4	0.342	26.2	0.948	0.933	0.148	0.121
	8	0.349	19.6	0.943	0.967	0.144	0.119
	16	0.350	14.8	0.910	0.962	0.143	0.116
	32	0.402	10.9	0.895	0.967	0.142	0.119
(14,12)	0	0.961	144.0	0.948	0.967	0.344	0.316
	2	0.531	19.9	0.981	0.948	0.211	0.167
	4	0.501	12.9	0.938	0.967	0.199	0.162
	8	0.515	8.1	0.938	0.971	0.200	0.167
	16	0.504	5.0	0.948	0.957	0.200	0.161
	32	0.551	2.9	0.905	0.967	0.190	0.155

5.2. Estimating variance components

In this subsection, we consider estimating variance components with the proposed method. The true fixed effects and data generation steps are the same as in Section 5.1. We use the whole data to estimate ${\sigma }_e^2$ and η*. We set ${\sigma }_e^2=0.25$. We first consider diagonal Ψ with d = 2. The basis matrices are set to be

$$G_1=\left(\begin{array}{cc}{I}_{q/2}& 0\\ 0& 0\end{array}\right)\text{and}{G}_2=\left(\begin{array}{cc}0& 0\\ 0& I_{q/2}\end{array}\right).$$

For diagonal Ψ, η* = (0.56, 0.56)^⊤. For singular Ψ, η* = (0.56, 0)^⊤. Table 4 shows the mean absolute errors of ${\sigma }_e^2$ ($\text{mae}.{\sigma }_e^2$), ${\eta }_1^{*}$ (mae.η₁), and ${\eta }_2^{*}$ (mae.η₂). A scenario with relatively large d is reported in the Supplementary Materials (Table 4).

Table 4.

Estimation of the variance components with the proposed method for positive definite and singular Ψ when ρ = 0.

m	q	Positive definite Ψ			Singular Ψ
		$\text{mae}.{\sigma }_e^2$	mae.η1	mae.η2	$\text{mae}.{\sigma }_e^2$	mae.η1	mae.η2
4	2	0.115	0.206	0.207	0.076	0.150	0.050
8	2	0.091	0.212	0.249	0.070	0.171	0.020
	4	0.122	0.164	0.166	0.071	0.136	0.027
12	2	0.087	0.268	0.245	0.076	0.197	0.015
	6	0.126	0.163	0.160	0.078	0.116	0.019

6. Application to a genome-wide association study in a mouse population

We apply the proposed method to estimate the effects of genetic variants on the body mass index (BMI) in a heterogenous stock mice population generated by the Welcome Trust Centre for Human Genetics http://gscan.well.ox.ac.uk. The data is available in R package “BGLR” (Perez and de los Campos 2014). The dataset consists of 1,814 mice, each genotyped over 10,346 polymorphic markers (SNPs) and has been used for genome-wide genetic association studies of multiple traits (Shifman et al. 2006; Valdar et al. 2006). This mice population consists of 8 liters and were housed in 523 different cages, each including a different number of mice. The distribution of cage density is in Figure 1. We are interested in identifying the genetic variants that are associated with the BMI phenotype. The measurements of BMI are transformed as described in Valdar et al. (2006) so that the data distribution is close to normal. In many mice experiments, cages often contribute significant environmental effects to the phenotypes such as BMI and mice in the same cage tend to be correlated in their phenotype measurements. It is therefore important to account for such cage effect in genetic association studies and the linear mixed-effects model can be employed.

Fig. 1.

Cage density of the stock mice population. The average density of cages with at least two individuals is 3.70.

In the current analysis, we incorporate the effect of cages as a single random effect and consider the following model

$$Y_{i,k}={\beta }_0+\sum _{j=1}^{10346}{\beta }_j{X}_{j,k}^i+{\tau }_{1}ag{e}_{i,k}+{\tau }_2\mathit{\text{gende}}{r}_{i,k}+{\gamma }_i+{ϵ}_{i,k},$$

where Y_{i, k} is the BMI of the kth mouse in the ith cage, $X_{j,k}^i$, is the numerical genotype at the genetic variant j for the ith mouse in cage k, β₀ and β_j are the regression coefficients corresponding to the intercept and genetic variants, τ₁ and τ₂ are the regression coefficients for age and gender, γ_i is the cage-specific random effect for the i-th cage. For cages with only one individual, we only fit the fixed effects.

The fixed effects are estimated via a weighted Lasso. To mitigate the relatively high correlation among the design, we first compute ridge regression estimates of the fixed effects, say ${\widehat{\beta }}^{(rr)}$, with tuning parameter chosen by cross-validation and use normalized ${\left\{1/\left|{\widehat{\beta }}_j^{(rr)}\right|\right\}}_{j=1}^p$ (sum up equal to p) as the weights for the Lasso estimates. The regression coefficient $\widehat{\beta }$ is obtained by fitting (4) with tuning parameter $\lambda =0.655\times \sqrt{2\text{log}p/N}$, where 0.655 is the noise level estimated by the scaled Lasso. In terms of statistical inference, we compute the debiased Lasso estimates of the fixed effect via (5) and their variances according to (8). According to cross-validation, we set a = 2.

6.1. Identification of BMI associated genetic variants

We control the false discovery rate (FDR) at 5% using the procedure proposed in Xia et al. (2018). Our method identifies 14 covariates with p-value threshold 6.7 × 10⁻⁵. The QQ plot of the z-scores of all the covariates is given in the left panel of Figure 2. It shows some deviation from standard normal density at both tails, indicating that some variants can be associated with BMI. Some of the genetic variants identified are in or near the genes known to be associated with body growth, body size, metabolism or obesity. For example, SNP rs13478535 is a variant in Auts2 gene, which has been shown to be related to with either low birth weight or small stature mice (Gao et al. 2014). SNP rs13481413 is one of the genetic variants in gene Immp2l, which is associated with food intake and body weight (Han et al. 2013). cAMP response element binding protein (Crebbp) has been postulated to play an important role downstream of the melanocortin-4 receptor and may affect other pathways that are implicated in the regulation of body weight (Chiappini et al. 2011).

Fig. 2.

The normal QQ-plots of the z-scores of the debiased Lasso estimators with proposed approach (left) and the debiased Lasso estimates with a = 0 (right) for 10348 fixed effects. The straight reference line passes the first and third quantiles of the z-scores.

We also consider applying the proposed procedure with a = 0. This is equivalent to applying the Lasso to fit the linear model to without considering the random cage effects. The tuning parameters are chosen in the same way as above. Only gender is selected as nonzero at FDR level 0.05. This is possibly due to the model misspecification and larger variances of the debiased Lasso estimators. The QQ-plot of the z-scores based on the debiased Lasso estimation of the linear model (right panel of Figure 2) shows that the z-scores clearly deviate from the standard normal distribution. These results indicate that the proposed estimation and inference methods for the linear mixed-effects model indeed provide an effective way of identifying important genetic variants associated with BMI in mice.

6.2. Evaluation of cage effect

For estimating the variance components, we only use the clusters with at least two observations. The estimated variance of the random effects is 0.202 and the estimated variance of the noise is 0.209. We compute the standard error of the estimated variance of the random effects assuming that the random components are normally distributed. The estimated standard deviation is 0.018, which indicates a strong cage effect.

7. Discussion

The present paper considers estimation and inference of unknown parameters in a high-dimensional linear mixed-effects model. Optimal rate of convergence for estimation was established and rate-optimal estimators were developed. The proposed methods have general applicability in modeling repeated measures and longitudinal data, especially when the cluster sizes are large or heterogeneous. The desirable properties of the proposed estimators are mainly due to the proper approximations of the unknown oracle weighting matrix ${\Sigma }_{{\theta }^{*}}$. Our proposed estimation procedure is computationally efficient and does not require strong distributional assumptions on the random effects and error distributions.

The proposed methods have important applications in large-scale genetic association studies in humans, including both family-based studies where the kinship coefficients can be used to specify the random effects and population cohort studies where the random effects can be used to adjust for population stratification (Yang et al. 2014). Instead of considering one genetic variant at a time as in typical mixed-effects models in genetic association studies (Yang et al. 2014), our model considers all the variants jointly. We expect gain in power in detecting phenotype-associated genetic variants by allowing for flexible random effects and by considering all genetic variants jointly using high-dimensional mixed-effects models studied in this paper.

Table 5.

Selected covariates at FDR level 0.05. 13 SNPs and gender are selected at FDR ≤ 0.05 and their z-scores are reported. The genes where the SNPs are located are presented when they are available.

SNP	Gene	z-score	SNP	Gene	z-score
rs13476390		−5.11	rs13482464		−4.01
rs13478535	Auts2	−4.42	rs4152477	Crebbp	4.17
rs3023058	Srrm3	4.22	rs6251709	Csnka2ip	4.49
rs13480072		−4.52	rs6185805	Mtcl1	−4.22
rs13481413	Immp2l	−4.22	gnf18.028.738		−4.34
rs13481961		6.27	gnfX.070.167		−4.24
rs4139535		4.51	gender		19.63