Statistical significance in high-dimensional linear mixed models

Abstract

This paper concerns the development of an inferential framework for high-dimensional linear mixed effect models. These are suitable models, for instance, when we have n repeated measurements for M subjects. We consider a scenario where the number of fixed effects p is large (and may be larger than M), but the number of random effects q is small. Our framework is inspired by a recent line of work that proposes de-biasing penalized estimators to perform inference for high-dimensional linear models with fixed effects only. In particular, we demonstrate how to correct a ‘naive’ ridge estimator in extension of work by Bühlmann (2013) to build asymptotically valid confidence intervals for mixed effect models. We validate our theoretical results with numerical experiments, in which we show our method outperforms those that fail to account for correlation induced by the random effects. For a practical demonstration we consider a riboflavin production dataset that exhibits group structure, and show that conclusions drawn using our method are consistent with those obtained on a similar dataset without group structure.

1. Introduction

Modern statistical problems are increasingly high-dimensional, with the number of covariates p potentially vastly exceeding the sample size N. This is due in part to technological advances that facilitate data collection. For instance, we are now able to measure the expression of many genes in a given specimen at little cost. However, it often remains expensive to have many replicates/species to experiment on, resulting in N ≪ p.

Fortunately, significant progress has been made in developing rigorous statistical tools for tackling such problems. While earlier work largely targeted point estimation and/or variable selection, recent years have seen a number of proposals on how to also assign uncertainty, statistical significance and confidence in high-dimensional models. This is of great practical importance, particularly when interpretation of parameters and variables is of key priority.

Early attempts are highly varied in their approach. Stability selection was proposed by Meinshausen and Bühlmann (2010) as a generic method for controlling the expected number of false positive selections; with improvements given by Shah and Samworth (2013). Sample splitting, where a first subsample is used to screen, and a second subsample is used to perform inference (Wasserman and Roeder, 2009; Meinshausen et al., 2009) has also been explored. Taking an alternative approach, Lockhart et al. (2014), Tibshirani et al. (2014) and Lee et al. (2016) build a framework for conditional inference for high-dimensional linear models, i.e., conduct inference given some covariates have been selected.

In this paper, we propose an unconditional inferential framework for high-dimensional linear mixed effect models, with the goal of testing null hypotheses of the form

$$H_{0,G}:{\beta }_j^{\ast }=0\text{for all}j\in G$$ (1.1)

where ${\beta }^{\ast }\in {\mathbb{R}}^p$ is the vector of fixed effect regression coefficients, and G may be any subset of {1,…,p}. Of particular interest is the case G = {j}, i.e., testing if a single fixed effect coefficient ${\beta }_j^{\ast }$ is zero. A related goal is to construct confidence intervals for ${\beta }_j^{\ast }$, j = 1,…,p. This problem arises naturally in many settings, as observations are rarely independent. A prime example is the analysis of longitudinal data, which is highly prevalent in clinical studies. In such settings mixed effect models are a natural extension of linear models for modeling data exhibiting group-structured dependence.

Our framework is inspired by a recent line of work that proposes de-biasing penalized estimators as an approach to inference for high-dimensional linear models with fixed effects only. There, the limiting distribution of the modified estimator is tractable and, thus, can be used to construct approximate p-values and confidence intervals. For example, in high-dimensional linear regression, Zhang and Zhang (2014), van de Geer et al. (2014), and Javanmard and Montanari (2014) suggest de-sparsifying the lasso: starting with the biased lasso estimator, the authors ‘invert’ the corresponding Karuhn-Kush-Tucker (KKT) optimality conditions to form an estimator that is approximately unbiased for β* and normally distributed. By construction, the de-biased estimator can then be used to derive confidence intervals and p-values. Ning and Liu (2017) extended this strategy by developing a score test for inference in penalized M-estimators.

Our proposed method bears strongest resemblance to Bühlmann (2013). Developed for high-dimensional linear models, the framework of Bühlmann (2013) is similar to those put forth by Zhang and Zhang (2014), van de Geer et al. (2014), and Javanmard and Montanari (2014), except it uses ridge estimation as a starting point. While the overall framework we consider is similar, there are important differences in the specifics on how to correct — or rather, approximately correct — for the bias in the ridge estimator, and how to compute an approximation of the limiting distribution of the de-biased estimator, to construct p-values and confidence intervals for elements in β*. As will be evident later, these differences are direct results of having to cope with dependencies induced by the random effects in the linear mixed effect model. The naive treatment of ignoring the dependencies, as we demonstrate in numerical examples, leads to poor practical performance (particularly, when inverting estimator to obtain confidence intervals, the confidence intervals have insufficient coverage). We address this issue by introducing a two-stage procedure that yields consistent estimates of the parameters that determine these dependencies. While we describe a ridge-based framework, the methodology could be extended to make use of other high-dimensional estimators as the starting point for constructing a de-biased estimator.

Our decision to use a ridge estimator is based on simulation findings for standard linear models showing that while asymptotically optimal, confidence intervals from ℓ₁-based de-biasing (Zhang and Zhang, 2014; van de Geer et al., 2014; Javanmard and Montanari, 2014) tend to have coverage problems in finite samples. Yu et al. (2018) similarly noticed that confidence intervals based on a de-biased ℓ₁-estimator for high-dimensional Cox model had poorer than theoretical coverage in practice. Although its theoretical justification is similar, the ridge-based method of Bühlmann (2013) yields better finite-sample error control.

Our paper is organized as follows. The remainder of this section provides a brief overview of the subsequent notation. Section 2 makes explicit the form of the high-dimensional linear mixed effect model we are working with. In Section 3, we describe the details of our method: specifically, how it builds upon Bühlmann (2013) to accommodate dependence within groups induced by the random effects. We also present theory, along with the required assumptions, to justify it. Numerical experiments can be found in Section 4, followed by a practical application of the method in Section 5. We conclude with a discussion and elaborate on potential extensions in Section 6. Proofs are collected in the Appendix.

Notation

Matrices are written in upper-case bold-face and their entries in corresponding lower-case. So a_jk is the (j, k)th entry of matrix $\mathbf{A}\in {\mathbb{R}}^{n_1\times n_2}$. For j ∈ {1,…,n₂} and J ⊆ {1,…,n₂}, a_j and A_J denote the jth column of A and the column-wise concatenation of columns in A indexed by the set J, respectively. The ith row of A is denoted a⁽ⁱ⁾. For r ∈ [1, ∞], the ℓ_r norm of a vector $u\in {\mathbb{R}}^n$ is $\Vert u{\Vert }_r=({\sum }_{i=1}^p\mid u_i{\mid }^r{)}^{1∕r}$, and the induced norm of a matrix $\mathbf{A}\in {\mathbb{R}}^{n_1\times n_2}$ is ${⫴\mathbf{A}⫴}_r=\text{sup}\{{⫴\mathbf{A}x⫴}_r:x\in {\mathbb{R}}^{n_2},\Vert x{\Vert }_r=1\}$. With this notation, ⫼A⫼₂ is the spectral norm, ⫼A⫼₁ the maximum absolute column sum of the matrix, and ⫼A⫼_∞ the maximum absolute row sum of the matrix. We use ∥A∥_r to denote the ℓ_r norm of the vectorization of A.

The projection of ${\mathbb{R}}^{n_2}$ onto the linear space generated by the rows of A is denoted P_A = A(A^TA)⁻A^T, where A⁻ is the Moore-Penrose inverse of A. For square matrices A₁ and A₂ of the same dimensions, A₁ ≤ A₂ indicates that A₂ – A₁ is positive semi-definite.

For real-valued functions g₁(x) and g₂(x) defined on (0, ∞), we write g₁(x) ≲ g₂(x) if there is a constant c ∈ (0, ∞) such that g₁(x) ≤ cg₂(x), and g₁(x) ≳ g₂(x) if instead g₁(x) ≥ cg₂(x). We write g₁(x) ≍ g₂(x) if both g₁(x) ≲ g₂(x) and g₁(x) ≳ g₂(x). Then, g₁(x) = o(g₂(x)) if g₁(x)/g₂(x) → 0 as x → ∞, and g₁(x) = O(g₂(x)) if there is a c ∈ (0, ∞) such that ∣g₁(x)∣ ≤ cg₂(x) for all x large enough. The latter relations also apply when x is a vector, where x → ∞ is interpreted elementwise. Finally, if $X\in \mathbb{R}$ is a random variable and $a\in \mathbb{R}$ is some constant, we write ∣X – a∣ = o_P(1) if X converges to a in probability, i.e., X →_p a.

2. The linear mixed effect model

Consider M groups of observations of sizes n₁,…,n_M. Let m = 1,…, M be group indices, and let i = 1,…,n_m index the observations within group m. Let N be the total number of observations, so $N={\sum }_{m=1}^M{n}_m$. We may later assume, without loss of generality, that n_m = n for all groups, or that, N = nM. The proposed framework allows for non-uniform group sizes with minor adjustments, so long as the group sizes are of the same order.

For group m ∈ {1,…,M}, we observe the response vector $y_m\in {\mathbb{R}}^n$, generated as

$$y_m={\mathbf{X}}_m{\beta }^{\ast }+{\mathbf{Z}}_m{v}_m+{ϵ}_m,m=1,\dots ,M$$ (2.1)

with

1. ${\beta }^{\ast }\in {\mathbb{R}}^p$, an unknown vector of fixed regression coefficients;

2. $v_m\in {\mathbb{R}}^q$, m = 1,…,M vectors of group-specific random effects, with $v_m\underset{i.i.d.}{\sim }\mathcal{N}(0,{\Psi }^{\ast })$, Ψ* an unknown q × q positive definite covariance matrix;

3. errors ${ϵ}_m\underset{i.i.d.}{\sim }\mathcal{N}(0,{\sigma }^{\ast 2}{\mathbf{I}}_{n\times n})$ for unknown σ*², which are independent of v₁,…,v_M; and

4. ${\mathbf{X}}_m\in {\mathbb{R}}^{n\times p}$ and ${\mathbf{Z}}_m\in {\mathbb{R}}^{k\times q}$ known design matrices.

By construction, β* represents effects shared across groups while v_m, m = 1,…,M, represent group-specific deviations. It will be convenient to write the model more compactly. Define vectors $y=[y_1^T,\dots ,y_M^T{]}^T$, $v=[v_1^T,\dots ,v_M^T{]}^T$, $ϵ=[{ϵ}_1^T,\dots ,{ϵ}_M^T{]}^T$, a stacked matrix $\mathbf{X}=[{\mathbf{X}}_1^T,\dots ,{\mathbf{X}}_M^T{]}^T$, and Z = diag(Z₁,…,Z_M). Then we can write (2.1) as

$$y=\mathbf{X}{\beta }^{\ast }+\mathbf{Z}v+ϵ.$$ (2.2)

Marginalizing out the random effects yields

$$y\sim \mathcal{N}(\mathbf{X}{\beta }^{\ast },\mathbf{V}({\sigma }^{\ast 2},{\Psi }^{\ast }))\text{with}\mathbf{V}({\sigma }^{\ast 2},{\Psi }^{\ast }))={\sigma }^{\ast 2}{\mathbf{I}}_{N\times N}+\mathbf{Z}{\Psi }^{\ast }){\mathbf{Z}}^T,$$ (2.3)

where Ψ*^(B) = I_M×M ⊗ Ψ*. This implies that V(σ*², Ψ*) is block-diagonal and observations belonging to different groups are independent. Thus, the inclusion of random effects only induces dependencies between observations belonging to the same group. We will be primarily working with the marginal form (2.3) in subsequent sections.

We study the presented model under the following assumptions:

1. High dimensions: We allow p, the number of fixed regression coefficients, to be possibly much larger than N. On the other hand, q, the number of random effect variables, is assumed to be of constant order, or at least smaller than n.

2. Sparsity of β*: We assume β* to be sparse in the sense that most of its elements are zero: a more precise specification on the level of sparsity required is detailed in Section 3.2.

3. Structure of Ψ*: Our paper primarily considers the scenario of Ψ* = τ*²I_q×q. However, our method, and corresponding theoretical results, can be extended to accommodate the more general scenario of Ψ* = D* where D* is a diagonal q × q matrix.

4. Standardization of design matrices: The design matrices X and Z are assumed fixed and standardized with $\Vert x_j{\Vert }_2^2=N$ for j ∈ {1,…,p} and $\Vert z_j{\Vert }_2^2=n$ for j ∈ {1,…,qM}.

3. A ridge-based inferential framework

We would like to test null hypotheses of the form (1.1), i.e., $H_{0,G}:{\beta }_j^{\ast }=0$ for all j ∈ G, for subsets G ⊂ {1,…,p}, and construct confidence intervals for ${\beta }_j^{\ast }$. This section formally introduces our inferential framework. We first describe its foundation, the de-biased ridge estimator, and show how it can be used to accomplish these tasks. We then detail how to assemble the components needed to construct this de-biased ridge estimator and approximate its limiting distribution. Theoretical justification of our approach is provided along the way.

3.1. A de-biased ridge estimator

As in Bühlmann (2013), our starting point is the ridge estimator given by

$$\widehat{\beta }=\text{arg}\underset{\beta \in {\mathbb{R}}^p}{\text{min}}\Vert y-\mathbf{X}\beta {\Vert }_2^2∕N+\lambda \Vert \beta {\Vert }_2^2.$$ (3.1)

This estimator is natural in models with homoscedastic and uncorrelated errors but in the linear mixed effect model, the random effects results in correlation. We thus refer to $\widehat{\beta }$ from (3.1) as the ‘naive’ ridge estimator. The estimator has a simple closed form expression,

$$\widehat{\beta }=N^{-1}{\left(\widehat{\Sigma }+\lambda {\mathbf{I}}_{p\times p}\right)}^{-1}{\mathbf{X}}^T\mathbf{Y},$$ (3.2)

where $\widehat{\Sigma }={\mathbf{X}}^T\mathbf{X}∕N$. It is straightforward to show that the ridge estimator is normally distributed with covariance matrix, multiplied by a factor of N,

$${\Omega }^{\ast }=(\widehat{\Sigma }+\lambda {\mathbf{I}}_{p\times p}{)}^{-1}{\mathbf{X}}^T\mathbf{V}({\sigma }^{\ast 2},{\tau }^{\ast 2})\mathbf{X}(\widehat{\Sigma }+\lambda {\mathbf{I}}_{p\times p}{)}^{-1}∕N.$$ (3.3)

As in Bühlmann (2013), we assume that the diagonal entries of ${\Omega }^{\ast }=({\omega }_{jk}^{\ast })$ satisfy

$${\omega }_{\text{min}}^{\ast }\equiv \underset{j\in \{1,\dots ,p\}}{\text{min}}{\omega }_{jj}^{\ast }>0.$$ (3.4)

Likewise, we do not require (3.4) to be bounded away from 0 as a function of N or p. This condition, in fact, is fairly mild; it is only violated under special kinds of design matrices. To illustrate, define R ≡ rank(X) and let X = QDΓ^T be the singular value decomposition with left singular vectors $\mathbf{Q}\in {\mathbb{R}}^{N\times N}$ satisfying Q^TQ = I_N×N, $\mathbf{D}\in {\mathbb{R}}^{N\times N}$ a diagonal matrix with entries s₁ ≥ … ≥ s_N (i.e., singular values of X), and right singular vectors $\Gamma \in {\mathbb{R}}^{p\times N}$ satisfying Γ^TΓ = I_N×N. Let ν_min(A) and ν_max(A) be the smallest and largest eigenvalue of any square matrix A, respectively. We can then show the following.

Lemma 1. Condition (3.4) holds if and only if X ≠ 0 and

$$\underset{j\in \{1,\dots ,p\}}{\text{min}}\underset{k\in \{1,\dots ,N\},s_k\ne 0}{\max }{\Gamma }_{jk}^2>0.$$ (3.5)

In the high-dimensional case with R ≤ N < p, the parameter β* is not identifiable: many vectors $\theta \in {\mathbb{R}}^p$ satisfy Xβ* = Xθ. A natural parameter to consider, as noted in Shao and Deng (2012), is θ* = P_X^Tβ* = X^T(XX^T)⁻Xβ* = ΓΓ^Tβ*, the projection of β* onto the linear space generated by the rows of X. As it turns out, under condition (3.4), or equivalently (3.5), the ridge estimator $\widehat{\beta }$ is a reasonable proxy for θ* when λ is sufficiently small.

Proposition 2. Suppose that λ > 0 and (3.4), or equivalently, (3.5), holds. Then, under our linear mixed effect model from Section 2, the ridge estimator (3.2) satisfies

$$\begin{gathered} \underset{j\in \{1,\dots ,p\}}{\text{max}}\mid \mathbb{E}\left[{\widehat{\beta }}_j\right]-{\theta }_j^{\ast }\mid \le \lambda \Vert {\theta }^{\ast }{\Vert }_2{\nu }_{\text{min},+}{\left(\widehat{\Sigma }\right)}^{-1}, \\ \underset{j\in \{1,\dots ,p\}}{\text{min}}\text{Var}\left[{\widehat{\beta }}_j\right]\ge N{\omega }_{\text{min}}^{\ast } \end{gathered}$$

where ${\nu }_{\text{min},+}(\widehat{\Sigma })$ refers to the smallest non-zero eigenvalue of $\widehat{\Sigma }$.

Proposition 2, which is proven in the Appendix, implies that the bias in estimating θ* with $\widehat{\beta }$ is small when λ > 0 is sufficiently small. We explicitly quantify how small λ needs to be for the estimation bias to be smaller than the standard error of $\widehat{\beta }$.

Corollary 3. Suppose that the ridge penalty parameter λ > 0 is chosen such that $\frac{\lambda }{\sqrt{{\omega }_{\text{min}}^{\ast }}}\le \frac{{\nu }_{\text{min},+}(\widehat{\Sigma })}{\sqrt{N}\Vert {\theta }^{\ast }{\Vert }_2}$, and that condition (3.4), or equivalently, (3.5) holds. Then,

$$\underset{j\in \{1,\dots ,p\}}{\text{max}}\mid \mathbb{E}\left[{\widehat{\beta }}_j\right]-{\theta }_j^{\ast }\mid \le \underset{j\in \{1,\dots ,p\}}{\text{min}}\sqrt{\text{Var}\left[{\widehat{\beta }}_j\right]}.$$

Our interest, however, lies in β*, not θ*. Thus, for $\widehat{\beta }$ to be useful, we need to adjust $\widehat{\beta }$ for the projection bias $B_j={\theta }_j^{\ast }-{\beta }_j^{\ast }$. By definition of θ*, one observes that

$$B_j=({\mathbf{P}}_{{\mathbf{X}}^T}{\beta }^{\ast }{)}_j-{\beta }_j^{\ast }=({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}{\beta }_j^{\ast }-{\beta }_j^{\ast }+\sum _{k\ne j}({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}{\beta }_k^{\ast },$$ (3.6)

which, under the null hypothesis H_0,j : ${\beta }_j^{\ast }=0$, becomes,

$$B_{H_0,j}=\sum _{k\ne j}({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}{\beta }_k^{\ast }.$$ (3.7)

The quantity can be approximated by

$${\widehat{B}}_{H_0,j}=\sum _{k\ne j}({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}{\widehat{\beta }}_k^{\text{init}}.$$ (3.8)

where ${\widehat{\beta }}^{\text{init}}$ is a consistent initial estimator of β* (and consistency occurs under additional assumptions). Consider then the corrected ridge estimator ${\widehat{\beta }}_j^{\text{corr}}$ as a statistic for testing H_0,j:

$${\widehat{\beta }}_j^{\text{corr}}={\widehat{\beta }}_j-{\widehat{B}}_{H_0,j}={\widehat{\beta }}_j-\sum _{k\ne j}({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}{\widehat{\beta }}_k^{\text{init}}.$$ (3.9)

Assuming that $\underset{j\in \{1,\dots ,p\}}{\text{min}}{\omega }_{\text{min}}^{\ast }>0$, we can write

$${\widehat{\beta }}_j^{\text{corr}}=W_j+{\gamma }_j,$$

where

$$\begin{gathered} {\gamma }_j=({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}{\beta }_j^{\ast }-\sum _{k\ne j}({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}\left({\widehat{\beta }}_k^{\text{init}}-{\beta }_k^{\ast }\right)+{\delta }_j, \\ {\delta }_j={\delta }_j(\lambda )=\mathbb{E}\left[{\widehat{\beta }}_j\right]-{\theta }_j^{\ast }. \end{gathered}$$

A rearrangement of the above set of equations yields

$$\frac{{\widehat{\beta }}_j^{\text{corr}}}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}}-{\beta }_j^{\ast }=\frac{W_j}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}}-\sum _{k\ne j}\frac{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}}({\widehat{\beta }}_k^{\text{init}}-{\beta }_k^{\ast })+\frac{{\delta }_j}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}}.$$ (3.10)

Then, from model (2.3), it follows that

$$W_1,\dots ,W_p\sim \mathcal{N}(0,{\Omega }^{\ast }∕N).$$ (3.11)

The normalizing factors needed to bring the W_j to N(0, 1) scale are given by ${\kappa }_j={\kappa }_j(N,p)=\sqrt{N∕{\omega }_{jj}^{\ast }}$. The proof is straightforward.

Theorem 4. Suppose we choose the ridge penalty parameter λ > 0 such that

$$\lambda {\left({\omega }_{\text{min}}^{\ast }\right)}^{-1∕2}=o\left({\nu }_{\text{min},+}(\widehat{\Sigma })∕\left(N^{1∕2}\Vert {\theta }^{\ast }{\Vert }_2\right)\right),(N,p\to \infty ),$$ (3.12)

and assume that for our choice of ${\widehat{\beta }}^{init}$, there exist constants C_j = C_j(N, p) such that

$$\mathbb{P}\left[\bigcap _{j=1}^p\left\{\mid {\kappa }_j(N,p)\sum _{k\ne j}({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}\left({\widehat{\beta }}_k^{init}-{\beta }_k^{\ast }\right)\mid \le C_j(N,p)\right\}\right]\to 1(N,p\to \infty ).$$ (3.13)

Then, under the null hypothesis, H_0,j, for all w > 0,

$$\underset{N,p\to \infty }{\text{lim}\text{sup}}\mathbb{P}\left[\mid {\kappa }_j{\widehat{\beta }}_j^{corr}\mid >w\right]-\mathbb{P}\left[\mid \tilde{W}\mid +C_j>w\right]\le 0,$$ (3.14)

where $\tilde{W}\sim N(0,1)$. In addition, for any sequence of subsets G_p ⊆ {1,…,p}, if H_0,Gp is true, then for any w > 0,

$$\underset{N,p\to \infty }{\text{lim}\text{sup}}\mathbb{P}\left[\underset{j\in G_p}{\max }\mid {\kappa }_j{\widehat{\beta }}_j^{corr}\mid >w\right]-\mathbb{P}\left[\underset{j\in G_p}{\max }\left(\mid \tilde{W}\mid +C_j\right)>w\right]\le 0.$$ (3.15)

In subsequent sections, we identify specific scalings of N and p such that Theorem 4 becomes applicable. Based on the asymptotic distributions in Theorem 4, we can construct p-values for testing H_0,G, G ⊆ {1,…,p}. For testing the individual null hypothesis H_0,j, we define the p-value for the two-sided alternative as

$${\varrho }_j=2(1-\Phi (({\kappa }_j\mid {\widehat{\beta }}_j^{\text{corr}}\mid -C_j{)}_{+})),$$ (3.16)

where Φ is the standard normal distribution function. For testing the group null hypothesis H_0,G, ∣G∣ > 1, we define the p-value as

$${\varrho }_G=1-\mathbb{P}\left[\underset{j\in G}{\max }({\kappa }_j\mid W_j\mid +C_j)\le \underset{j\in G}{\max }{\kappa }_j\mid {\widehat{\beta }}_j^{\text{corr}}\mid \right],$$ (3.17)

where W₁,…,W_p are as in (3.11). From Theorem 4, we can derive the following corollary.

Corollary 5. Under the conditions in Theorem 4, for any α ∈ (0, 1), the following statements hold:

$$\begin{gathered} \underset{N,p\to \infty }{\text{lim}\text{sup}}\mathbb{P}[{\varrho }_j\le \alpha ]-\alpha \le 0if{H}_{0,j}istrue, \\ \underset{N,p\to \infty }{\text{lim}\text{sup}}\mathbb{P}[{\varrho }_G\le \alpha ]-\alpha \le 0if{H}_{0,G}istrue. \end{gathered}$$

3.2. Consistent estimation of variance parameters

As presented, the de-biased ridge framework depends on the values of the unknown parameters σ*² an τ*². We employ a two-step approach to consistent estimation of these parameters.

1. Let $S=\{j:{\beta }_j^{\ast }\ne 0\}$ be the support of β*, with cardinality d = ∣S∣. We use the Lasso estimator ${\widehat{\beta }}^L=\text{arg}\underset{\beta \in {\mathbb{R}}^p}{\text{min}}\Vert y-\mathbf{X}\beta {\Vert }_2^2∕N+2{\lambda }_L\Vert \beta {\Vert }_1$ with an appropriate choice of tuning parameter λ_L to identify an initial guess of the elements (i.e., indices) in S. We define $\widehat{S}=\{j:{\widehat{\beta }}_j^L\ne 0\}$ as our guess for the support S. By properties of the Lasso, $\mid \widehat{S}\mid \le N$, although, in general, $\widehat{S}$ may not be a good estimate of S.

2. Working with the (potentially misspecified) random effects model

   $$y={\mathbf{X}}_{\widehat{S}}{\beta }_{\widehat{S}}^{\ast }+\mathbf{Z}b+ϵ,$$ (3.18)

   we apply Henderson’s Method III (Henderson, 1953) to form estimates ${\widehat{\sigma }}^2$ and ${\widehat{\tau }}^2$. Henderson’s Method III is particularly tractable theoretically and enables us to study consistency in the scenario where (3.18) is actually misspecified, i.e., $\mid S\setminus \widehat{S}\mid >0$. For a discussion of Henderson’s methods and the appeals of Method III, see (Searle, 1968).

In recent years Henderson’s methods have largely been supplanted by alternatives such as restricted maximum likelihood (REML) for variance component estimation (Harville, 1977); it is customary to refer to variances of random effects as variance components. We thus provide a brief overview of what Henderson’s Method III entails. Consider, first, the low-dimensional model (2.3) with p < N. To simplify the notation in the following explanation, we momentarily define $\tilde{\mathbf{X}}=[\mathbf{X}\mathbf{Z}]$. By not distinguishing between fixed and random effects, the idea behind Henderson’s methods is to match the differences in the reductions in the sum-of-squares between sub-models of (2.3) to its expected value, not unlike a method-of-moments approach. To elaborate, in fitting (2.3) to data y, the reduction in the sum of squares is

$$\mathcal{R}(\beta ,v)=y^T{\mathbf{P}}_{\tilde{\mathbf{X}}}y.$$ (3.19)

Likewise, the decrease in the sum of squares due to fitting the reduced model y = Xβ + ϵ is

$$\mathcal{R}(\beta )=y^T{\mathbf{P}}_{\mathbf{X}}y.$$ (3.20)

The expected difference in the reductions $\mathcal{R}(v\mid \beta )\equiv \mathcal{R}(\beta ,v)-\mathcal{R}(\beta )$ is

$$\mathbb{E}[\mathcal{R}(v\mid \beta )]={\tau }^{\ast 2}\text{tr}({\mathbf{Z}}^T[{\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\mathbf{X}}]\mathbf{Z})+{\sigma }^{\ast 2}\left[\text{rank}\left(\tilde{\mathbf{X}}\right)-\text{rank}(\mathbf{X})\right].$$ (3.21)

Moreover,

$$\mathbb{E}[y^{T}y-R(\beta ,v)]={\sigma }^{\ast 2}\left[N-\text{rank}\left(\tilde{\mathbf{X}}\right)\right].$$ (3.22)

Together, (3.21) and (3.22), when matching theoretical expectations to empirical averages, form a triangular system of linear equations, from which we derive ${\widehat{\sigma }}^2$ and ${\widehat{\tau }}^2$. We find

$${\widehat{\sigma }}^2=\frac{y^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\tilde{\mathbf{X}}})y}{N-\text{rank}\left(\tilde{\mathbf{X}}\right)},$$ (3.23)

$${\widehat{\tau }}^2=\frac{y^T({\mathbf{P}}_{\tilde{\mathbf{X}}}-{\mathbf{P}}_{\mathbf{X}})y-{\widehat{\sigma }}^2\left[\text{rank}\left(\tilde{\mathbf{X}}\right)-\text{rank}(\mathbf{X})\right]}{\text{tr}({\mathbf{Z}}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\mathbf{X}})\mathbf{Z})}.$$ (3.24)

It is straightforward to see that the ${\widehat{\sigma }}^2$ and ${\widehat{\tau }}^2$ generated from (3.23) and (3.24) are unbiased, presuming that the true model is y = Xβ + Zv + ϵ. For consistency, some additional assumptions are needed, which we will discuss later in this section.

Returning to our two-step procedure and high-dimensional setup, Step 1 identifies a candidate low-dimensional sub-model, which is used in Step 2 to obtain variance component estimates. We do not require the candidate model to encompass the truth; however, λ_L should be such that $\widehat{S}$, from Step 1, reliably captures the indices of the ‘strong’ signals in β*. The idea is that missing ‘weak’ signals only negligibly affect the accuracy of ${\widehat{\sigma }}^2$ and ${\widehat{\tau }}^2$ in Step 2. We now show that this two-step procedure yields consistent estimators ${\widehat{\sigma }}^2$ and ${\widehat{\tau }}^2$ in the setting where N → ∞ to (specifically, n is fixed, but the number of groups M → ∞) and d² logp/M = o(1), provided some additional technical assumptions hold. From here on, this will also be the scaling assumed for Theorem 4, as well as Corollary 5. We first present the assumptions necessary for consistency and then formally state the theorem.

For ξ > 1, define the cone

$$\mathcal{C}(\xi ,S)=\{u\in {\mathbb{R}}^p:\Vert u_{S^c}{\Vert }_1\le \xi \Vert u_S{\Vert }_1\}.$$ (3.25)

Assumption 1. For some constant ξ > 1,

$$\zeta \equiv \text{inf}\left\{\frac{\Vert \widehat{\Sigma }u{\Vert }_{\infty }}{\Vert u_A{\Vert }_{\infty }}:u\in {\mathcal{C}}_{-}(\xi ,S),\mid A\setminus S\mid \le p\right\}\gtrsim 1$$ (3.26)

with ${\mathcal{C}}_{-}(\xi ,S)\equiv \{u:u\in \mathcal{C}(\xi ,S),u_j{\Sigma }_{j,}.u\le 0\forall j\notin S\}$ the sign-restricted version of (3.25).

The quantity ζ in (3.26) is defined more generally in Ye and Zhang (2010), where it is termed a sign-restricted cone invertibility factor (SCIF). We have the following lemma.

Lemma 6. Suppose Assumption 1 holds, and let λ_L be defined by (A.2) (or 3.28) for some small ε > 0 and ξ as in Assumption 1. If u* ≤ λ_L(ξ − 1)/(ξ + 1), then

$$\Vert {\widehat{\beta }}^L-{\beta }^{\ast }{\Vert }_{\infty }\le \frac{{\lambda }_L+u^{\ast }}{\zeta }\le \frac{2\xi {\lambda }_L}{(\xi +1)\zeta }.$$ (3.27)

In the proof of Lemma 6 (provided in the Appendix), SCIF naturally appears when deriving an upper bound for $\Vert {\widehat{\beta }}^L-{\beta }^{\ast }{\Vert }_{\infty }$. Lemma 6 assumes that Assumption 1 is satisfied, and that λ_L in Step 1 is chosen such that

$${\lambda }_L=\frac{(\xi +1)}{(\xi -1)}\sqrt{\frac{2({\sigma }^{\ast 2}+{\tau }^{\ast 2}qn)(\log p-\log (\epsilon ∕2))}{N}}\asymp \sqrt{\frac{\log p}{qM}}=o(1),$$ (3.28)

with ξ as in Assumption 1. It then establishes that

$$\Vert {\widehat{\beta }}^L-{\beta }^{\ast }{\Vert }_{\infty }\le 2\xi {\lambda }_L∕\zeta (\xi +1)=o(1),$$

with probability exceeding 1 – ε, where ε > 0 can be taken arbitrarily small. A direct implication is that if the lemma’s conditions are satisfied, $S\setminus \widehat{S}$ only includes indices corresponding to ‘weak’ signals in β* of magnitude less than 4ξλ_L/ζ(ξ + 1) = o(1) with close to certainty, which is part of what Step 1 sets out to achieve.

Assumption 2. There exists an integer N′ ≲ d such that for the same constant ξ > 1 as in Assumption 1,

$$\frac{d{\xi }^2}{{\psi }^2(\xi ,S)}<\frac{N^{\prime }}{{\psi }_{+}(N^{\prime },S)},$$ (3.29)

where

$$\psi (\xi ,S)=\text{min}\left\{\frac{d^{1∕2}\Vert \mathbf{X}u{\Vert }_2}{N^{1∕2}\Vert u_S{\Vert }_1}:u\in \mathcal{C}(\xi ,S),u\ne 0\right\}$$ (3.30)

and ${\psi }_{+}(N^{\prime },S)=\underset{\mathcal{A}\cap S=\varnothing ,\mid A\mid \le N^{\prime }}{\max }{\nu }_{\text{min}}\left(\frac{{\mathbf{X}}_{\mathcal{A}}^T{\mathbf{X}}_{\mathcal{A}}}{N}\right)$ is the sparse upper eigenvalue of models disjoint with S.

Assumption 2 is needed to control the number of false positive selections in $\widehat{S}$ from Step 1. In particular, we have

Lemma 7. Suppose that Assumption 2 holds, and λ_L is defined according to (3.28). In the event that u* ≤ λ_L(ξ − 1)/(ξ + 1), $\mid \widehat{S}\setminus S\mid <N^{\prime }$.

Put simply, Lemma 7 claims that under Assumption 2 and our choice of λ_L from (3.28), the total number of false selections in Step 1 is bounded by N′, with probability exceeding 1 – ε. The proof is provided in the Appendix.

Assumption 3. Let $\check{\mathbf{X}}$ be formed by joining any N′ columns in X with ${\beta }_j^{\ast }=0$ to the d support columns in X. For the same N′ as in Assumption 2,

$$\text{rank}([{\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\check{\mathbf{X}}}]\mathbf{Z})=\text{rank}(\mathbf{Z})=qM,$$ (3.31)

$${\mathbf{Z}}^T[{\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\check{\mathbf{X}}}]\mathbf{Z}\gtrsim {\mathbf{I}}_{qM\times qM},$$ (3.32)

and the qM singular values of $[{\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\check{\mathbf{X}}}]\mathbf{Z},s_1,\dots ,s_{qM}$, satisfy

$$\frac{\mid \{i:s_i\ne 0\}\mid }{{\left({\sum }_{i=1}^{qM}{s}_i^2\right)}^2}=o(1).$$ (3.33)

By (3.31) in Assumption 3, the fixed data matrix Z has full column rank, and no column vector of Z can be represented as a linear combination of the column vectors of any ‘feasible’ ${\mathbf{X}}_{\widehat{S}}$ assuming that λ_L is chosen according to (3.28). After all, N′ + d is the upper bound on the number of selected fixed effects with probability exceeding 1 – ε (Lemma 7). Additionally, by (3.32), the sum of the squared perpendicular distances between each column vector in Z and its projection onto the linear subspace spanned by the column vectors of feasible ${\mathbf{X}}_{\widehat{S}}$ matrices is at least on the order of qM (substantial, given there are qM columns in Z). The latter half of Assumption 3 requires all columns of $({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\check{\mathbf{X}}})\mathbf{Z}$ are ‘close’ to being linearly independent from one another and ‘contribute equally’ to its rank. In particular, note that (3.33) is satisfied if

$$c_1<\frac{s_j}{s_k}<c_2\text{for}j\ne k\text{and some constants}{c}_1,c_2>0.$$ (3.34)

It is thus clear that (3.31) and (3.32) imply that random effects must not be confounded with any ‘feasible’ set of fixed effects (from Step 1) while (3.33) implies that the random effects are not confounded from one another. Analogous conditions were shown to be necessary to prove consistency of REML estimators in Jiang (1996).

Assumption 4. For any j ∈ S such that $\mid {\beta }_j^{\ast }\mid <4\xi {\lambda }_L∕\zeta (\xi +1)$, with λ_L defined as in (3.28), $\Vert {\Gamma }_{\tilde{\mathbf{X}}}{x}_j{\Vert }_{\infty }\asymp 1$. Here, ${\Gamma }_{\tilde{\mathbf{X}}}{\mathbf{D}}_{\tilde{\mathbf{X}}}{\Gamma }_{\tilde{\mathbf{X}}}^T$ is the eigen-decomposition of $\tilde{\mathbf{X}}{\left({\tilde{\mathbf{X}}}^T\tilde{\mathbf{X}}\right)}^{-}{\tilde{\mathbf{X}}}^T$ (defined for this Assumption) with $\tilde{\mathbf{X}}=[\check{\mathbf{X}}\mathbf{Z}]$, where $\check{\mathbf{X}}$ is formed by joining any N′ columns in X with ${\beta }_j^{\ast }=0$ to the d – 1 support (excluding j) columns in X. The N′ referenced here is the same as in Assumptions 2 and 3.

Assumption 4 requires that covariates corresponding to weak (but non-zero) signals in β* (for which we cannot quantify a bound on the probability they are to be included in ${\mathbf{X}}_{\widehat{S}}$) are not too strongly correlated to covariates in ${\mathbf{X}}_{\widehat{S}}$ nor covariates associated with the random effects. This somewhat resembles the irrepresentability conditions needed for model selection consistency in Lasso–see, e.g., Zhao and Yu (2006). However, the two assumptions are very different: Aside from differences in the quantities involved, a key difference is that the irrepresentability condition requires a very stringent upper bound on non-confounding between fixed effects, whereas Assumption 4 only requires boundedness. As shown in the numerical experiments in the Appendix, as the number of covariates and sparsity of the model vary, Assumption 4 is very likely to be satisfied with even small bounds, whereas the irrepresentability condition is increasingly less likely to hold.

We can now state our main result on consistency of variance component estimators, which validates our two-step procedure.

Theorem 8. Consider N, p → ∞ with n fixed, M → ∞. Furthermore, suppose p → ∞ with d²q log p/M = o(1). Suppose Assumptions 1-4 are satisfied and Λ_L is chosen according to (3.28) with ε ∝ 1/p. Then, ${\widehat{\sigma }}^2$ and ${\widehat{\tau }}^2$ are consistent for σ*² and τ*², respectively, i.e.,

$$\mid {\widehat{\sigma }}^2-{\sigma }^{\ast 2}\mid =\mid {\widehat{\tau }}^2-{\tau }^{\ast 2}\mid =o_P(1)(N,p\to \infty ).$$ (3.35)

Because $\mid {\widehat{\sigma }}^2-{\sigma }^{\ast 2}\mid$ and $\mid {\widehat{\tau }}^2-{\tau }^{\ast 2}\mid$ are both o_P(1), we can use ${\widehat{\sigma }}^2$ and ${\widehat{\tau }}^2$ as plug-in values for σ*² and τ*², respectively. From there, we can form a consistent estimator of Ω* and normalizing constants κ_j.

For practical applications, REML can be used as a substitute for Henderson’s Method III for Step 2. Theory for REML would be a possible avenue for further explorations.

3.3. An initial estimator for β* and our choice of C_j

To form ${\widehat{\beta }}^{\text{init}}$, we consider the ordinary least-squares (OLS) fit restricted to $\widehat{S}$, i.e.,

$${\widehat{\beta }}^{\text{init}}=\text{arg}\underset{\beta \in {\mathbb{R}}^p:{\beta }_{{\widehat{S}}^c}=0}{\text{min}}\Vert y-\mathbf{X}\beta {\Vert }_2^2.$$ (3.36)

We proceed to demonstrate that the error ${\widehat{\beta }}^{\text{init}}-{\beta }^{\ast }$ is o(1) in ∓₁ norm.

Assumption 5. For the same N′ as in Assumptions 2, 3, 4, the sparse lower eigenvalue for models containing S of cardinality smaller than d + N′ is constant and greater than 0,

$${\psi }_{-}(N^{\prime },S)=\underset{\mathcal{A}\supset S,\mid \mathcal{A}\setminus S\mid \le N^{\prime }}{\text{min}}{\nu }_{\text{min}}\left(\frac{{\mathbf{X}}_{\mathcal{A}}^T{\mathbf{X}}_{\mathcal{A}}}{N}\right)\gtrsim 1,$$

Assumption 5, in conjunction with previous assumptions and choice of Λ_L (3.28), can be used to control the ℓ₁ norm of the estimation error ${\widehat{\beta }}^{\text{init}}-{\beta }^{\ast }$.

Theorem 9. Suppose Assumptions 1-5 hold. Under the same conditions as in Theorem 8, for some universal constant C > 0,

$$\Vert {\widehat{\beta }}^{init}-{\beta }^{\ast }{\Vert }_1\le Cd\sqrt{\frac{q\log p}{M}}$$ (3.37)

with probability converging to 1 as N, p → ∞.

Theorem 9 implies that we have the following crude bound, based on Hölder’s inequality,

$$\begin{gathered} \mid {\kappa }_j\sum _{k\ne j}({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}\left({\widehat{\beta }}_k^{\text{init}}-{\beta }_k^{\ast }\right)\mid \le {\kappa }_j\underset{k\ne j}{\max }\mid ({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}\mid \Vert {\widehat{\beta }}^{\text{init}}-{\beta }^{\ast }{\Vert }_1 \\ \le {\kappa }_j\underset{k\ne j}{\max }\mid ({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}\mid Cd{\lambda }_L. \end{gathered}$$ (3.38)

The following corollary is a direct consequence of the crude bound (3.38).

Corollary 10. Suppose the conditions in Theorem 9 are satisfied, and that d, the sparsity of β*, satisfies d ≤ C⁻¹ (M/(q log p))^η, with C as in Theorem 9 and η ∈ (0, 1/2). Then,

$$C_j=\underset{k\ne j}{\max }\mid {\kappa }_j({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jk}\mid {\left(\frac{q\log p}{M}\right)}^{1∕2-\eta }$$ (3.39)

satisfies condition (3.13) in Theorem 4.

4. Numerical experiments

4.1. A practical choice for λ_L

In practical applications, we run into the issue of not being able to set λ_L according to (3.28), as it involves knowing τ* and σ*. However, we can derive a (slightly ad-hoc) approximation of what λ_L should be. Upon closer examination of the proof of Lemma 11, we can substitute the term σ*² + τ*²qn with ν_max(V(σ*, τ*)) = σ*² + τ*²ν_max(Z^TZ). The latter can be approximated according to the following procedure, assuming that the ratio τ*/σ* is not too small:

1. Apply scaled lasso (Sun and Zhang, 2012) to obtain an initial ‘average’ noise estimate. The solution to the scaled lasso problem is characterized by

   $$({\widehat{\beta }}^{\text{scaled}},{\widehat{\sigma }}^{\text{scaled}})\in \text{arg}\underset{\beta ,\sigma }{\text{min}}\frac{\Vert y-\mathbf{X}\beta {\Vert }_2^2}{2N\sigma }+\frac{\sigma }{2}+{\lambda }_{\text{univ}}\Vert \beta {\Vert }_1$$ (4.1)

   with ${\lambda }_{\text{univ}}=\sqrt{2\log p∕N}$.

2. Take ${\lambda }_L={\widehat{\sigma }}^{\text{scaled}}{\lambda }_{\text{univ}}{\rho }_Z$ with

$${\rho }_Z=\sqrt{\frac{{\nu }_{\max }({\mathbf{Z}}^T\mathbf{Z})}{\text{tr}({\mathbf{Z}}^T\mathbf{Z})∕N}}.$$ (4.2)

We provide a heuristic justification. Ignoring the finer details involved in the theory, for the scaled lasso, $({\widehat{\sigma }}^{\text{scaled}}{)}^2$ serves as a good approximation for $\Vert {ϵ}^{\ast }{\Vert }_2^2∕N$, where we have defined ϵ* = y – Xβ*. In linear models, ϵ* holds i.i.d. observations drawn from a N(0, σ*²) distribution. By the law of large numbers, $\Vert {ϵ}^{\ast }{\Vert }_2^2∕N$ converges to σ*² for large N. Under a heteroskedastic error model, with ϵ* independent and ${ϵ}_i^{\ast }\sim N(0,{\sigma }_i^{\ast 2})$, we can match $\Vert {ϵ}^{\ast }{\Vert }_2^2∕N$ to its expectation, which is given by ${\sum }_{i=1}^N{\sigma }_i^{\ast 2}∕N$, so $({\widehat{\sigma }}^{\text{scaled}}{)}^2$ can be used to approximate the ‘average’ noise level. If ϵ* ~ N (0, V(σ*, τ*)), then using a similar expectation matching argument, we can expect $({\widehat{\sigma }}^{\text{scaled}}{)}^2$ to act as a surrogate for

$${\sigma }^{\ast 2}+\frac{{\tau }^{\ast 2}\text{tr}(\mathbf{Z}{\mathbf{Z}}^T)}{N},$$ (4.3)

which follows from the fact that ∥Γϵ*∥₂ = ∥ϵ*∥₂ for any N × N orthogonal matrix Γ (overloading Γ from (3.5)). What we actually need is σ*² + τ*²ν_max(Z^TZ). Then in the scenario where ratio τ*²/σ*² is not too small, ρ_Z from (4.2) should give us a choice of λ_L that is close to the desired one from (3.28). Our choice of λ_L is constructed according to the above procedure for all subsequent numerical experiments.

4.2. A look into p-values

Denote the ‘unblocked’ version of Z as Z_u; i.e., Z_u is a N × q matrix formed by row-wise concatenating the M diagonal blocks in Z. We generate data from model (2.1) according to following schemes, setting M = 25 and n = 6:

• (M1) For p ∈ {300, 600}, q ∈ {1, 2}, we construct [X Z_u] from N i.i.d. realizations from a $\mathcal{N}(0,{\Phi }^{\ast })$ distribution with Φ* = {ϖ_jk} a (p + q) × (p + q) matrix with ${\varphi }_{jk}^{\ast }={0.2}^{\mid j-k\mid }$. X and Z (the ‘blocked’ version) are then normalized such that $\Vert x_j{\Vert }_2^2=N$ and $\Vert z_j{\Vert }_2^2=n$ for all j. For b ∈ {0.5, 1}, we set the p-dimensional vector of fixed regression coefficients to

  $$\beta =[b,\dots ,b,0,\dots ,0],$$

  where, the first d ∈ {5, 10} entries of β are nonzero. The variance parameters σ* and τ* are set to 0.5 and 1 respectively.

• (M2) Same as (M1) except with Φ* = I_(p+q)×(p+q).

The numerical experiments are setup similarly to those in Bühlmann (2013) and Schelldorfer et al. (2011). We set the ridge penalty parameter λ to 1/N for all experiments. Additionally, we set C_j according to Corollary 10 with η = 0.005.

We first consider null hypotheses of the form

$$H_{0,j}:{\beta }_j=0.$$ (4.4)

We consider decision rules based on a significance level α = 0.05, i.e., we reject H_0,j if the event E_j = {ϱ_j ≤ 0.05} occurs, where ϱ_j is as defined in (3.16). Following Bühlmann (2013), we evaluate the performance of the tests based on the type I error, averaged over the non-support indices,

$$\text{Avg. type I error}=(p-d{)}^{-1}\sum _{j\in S^c}\widehat{\mathbb{P}}(E_j),$$ (4.5)

and the power, averaged over the support indices,

$$\text{Avg. power}=d^{-1}\sum _{j\in S}\widehat{\mathbb{P}}(E_j),$$ (4.6)

where $\widehat{\mathbb{P}}$ denotes the empirical probability over 1000 simulations. The results, presented in Figure 1, suggest that type I error is well-controlled for all combinations of p, q, b and d for the two different models. Power is high in most scenarios, but appears to vary with the aforementioned quantities, noticeably decreasing with b. However, this is to be expected.

Figure 1:

Average power vs. average type I error for testing groups of coefficients under the two models for different combinations of p, q, b and d.

We also consider null hypotheses of the form

$$H_{0,G}:{\beta }_j=0\text{for all}j\in G.$$ (4.7)

with G taken either to be {1,…,100} (G1), or {101,…,200} (G2). By construction, the hypothesis H_0,G1 should be accepted while H_0,G2 rejected. We consider decision rules based on a significance level α = 0.05 and reject H_0,G if the event E_G = {ϱ_G ≤ 0.05} occurs, with ϱ_G defined in (3.17). To evaluate the performance of these tests, we consider type I error and power, which can be represented by $\widehat{\mathbb{P}}(E_{G2})$ and $\widehat{\mathbb{P}}(E_{G1})$, respectively, where again, $\widehat{\mathbb{P}}$ denotes the empirical probability over 1000 simulations. Figure 2 visualizes the results.

Figure 2:

Average power vs. average type I error for testing groups of coefficients under the two models for different combinations of p, q, b and d.

4.3. Comparisons with existing methods

In this section, we conduct a short numerical example to examine whether one could ‘naively’ apply inferential procedures for high-dimensional linear models to obtain inference for parameters in mixed models.

Consider Model (M1) from Section 4.2 in the instance of p = 300 and q = 1. Let β* = [0.05, 2, 4, 3, 0.1, 0,…, 0]. We compare our method against

1. ridge-based inference procedure of Bühlmann (2013), which is an analogue of our method developed for high-dimensional linear models;

2. lasso-based inference procedure of van de Geer et al. (2014), which entailes de-sparsifying a lasso estimator.

The differences are fairly evident when comparing confidence interval coverage. For any α ∈ (0, 1), define ${\mathbb{Q}}_{\alpha }[W_j]$ as the α-th quantile of the distribution of W_j. Under the conditions of Theorem 4, if the assumed model is correct, (3.11) suggests that confidence intervals of the form

$$\left[\frac{{\widehat{\beta }}_j^{\text{corr}}}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}}-\frac{{\mathbb{Q}}_{1-\alpha ∕2}[W_j]+C_j}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}},\frac{{\widehat{\beta }}_j^{\text{corr}}}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}}+\frac{{\mathbb{Q}}_{1-\alpha ∕2}[W_j]+C_j}{({\mathbf{P}}_{{\mathbf{X}}^T}{)}_{jj}}\right]$$

should guarantee coverage of at least (1 – α)%. Rather than setting C_j according to Corollary 10, we set them to be the same as the ‘C_j-analogues’ from Bühlmann (2013), to make the two methods comparable. Our choice of C_j are larger than theirs, so if anything, this ad-hoc decision provides Bühlmann (2013)’s method an unfair advantage. In Figure 3, we examine 95% confidence interval coverage for the three methods, based on the above modifications.

Figure 3:

Confidence interval coverage for ${\beta }_j^{\ast }$, j = 1,…,p; target coverage is 95% (with 1000 simulations, the standard deviation is ~0.69%). Here, lmm () refers to our method; ridge () to the method of Bühlmann (2013); and lasso () to the method of van de Geer et al. (2014).

Overall, our method, which accounts for random effects, performs best at attaining the target guaranteed coverage across all ${\beta }_j^{\ast }’\mathrm{s}$, compared to the methods proposed in Bühlmann (2013) and van de Geer et al. (2014). While Bühlmann (2013)’s method does come close, coverage falls short at 16 indices: minimum coverage achieved was 92.9% (with 1000 simulations, this is a statistically significance difference from 0.95). At initial glance it appears that the lasso-based method from van de Geer et al. (2014) performs quite well; however, a closer examination of the results reveals otherwise. Specifically, the lasso-based method does very poorly over some of the active coefficients, as made evident in Table 1.

Table 1:

Confidence interval coverage for signals ${\beta }_j^{\ast }$, j = 1,…, 5; target coverage is 95%.

	Our method	Bühlmann (2013)	van de Geer et al. (2014)
${\beta }_1^{\ast }$	0.977	0.974	0.994
${\beta }_2^{\ast }$	0.973	0.963	0.865
${\beta }_3^{\ast }$	0.969	0.971	0.782
${\beta }_4^{\ast }$	0.971	0.972	0.886
${\beta }_5^{\ast }$	0.983	0.993	1.000

5. An application to riboflavin production data

In this section, we apply our proposed methodology to data on riboflavin (vitamin B₂) production by Bacillus subtilis. The data is made publicly available by Bühlmann et al. (2014); the original data was provided by DSM (Switzerland). The dataset, referenced as riboflavinGrouped, has M = 28 specimens measured at two to six time points, resulting in N = 111 observations in total. For each specimen at each time point, we record a single real valued response variable, the log-transformed riboflavin production rate, as well as the expression levels of p = 4088 genes. We are interested in identifying which gene is significantly correlated with riboflavin production.

To account for correlations induced by repeated measurements, a natural model to consider is the random intercept model, in which we assume that

$$y_m={\mathbf{X}}_m{\beta }^{\ast }+v_m+{ϵ}_m,$$ (5.1)

with v_m, m = 1,…, M i.i.d. with v_m ~ N(0, τ*²), and ϵ_m, m = 1,…, M, independent with ϵ_m ~ N(0,σ*²I_nm×nm), and generated independently of v₁,… v_m. Note that (5.1) can be represented by (2.1) with the Z_m’s taken to be column vectors of 1s of lengths n_m. Most of the theoretical results assume the n_m’s are equal, but it is straightforward to show the results hold so long as n_m are on the same order of magnitude, as they are here.

We apply our proposed framework and compute the marginal p-values for testing ${\beta }_j^{\ast }=0$. Controlling the family-wise error rate (FWER) at 5%, via a simple Bonferroni correction, we find a single significant gene in riboflavin production: YXLD-at. This result matches previous findings by Javanmard and Montanari (2014) and Meinshausen et al. (2009) using an homogeneous dataset with N = 71 samples provided by the same source (riboflavin in Bühlmann et al. (2014)). Like us, Meinshausen et al. (2009) makes a single discovery, YXLD-at, while Javanmard and Montanari (2014) also labels YXLE-at as significant. The method of Bühlmann (2013), on the other hand, makes no discoveries.

6. Discussion

We presented a new framework for constructing asymptotically valid p-values and confidence intervals for the fixed effects in high-dimensional linear mixed effect models. It entails de-biasing a ‘naive’ ridge estimator, whose asymptotic distribution we can approximate sufficiently well if the number of independent groups of observations M scales at least with d²q log p. Simulation studies in high-dimensional suggest that our method provides good control of type-I error. It also provides good results for a riboflavin dataset with group structure, where we confirmed results obtained in earlier work based on a homogeneous dataset from the same source (Javanmard and Montanari, 2014; Meinshausen et al., 2009).

Several extensions to our methodology would be of interest for future work. First, our proposal for selecting the tuning parameter λ_L relies on the assumption that τ*²/σ*² is not too small. Although it appears to work well in practice, one could also consider an iterative scheme that repeatedly updates λ_L based on the resultant estimates of σ*² and τ*²: this can be readily implemented in practice but may be difficult to validate theoretically. Second, here we required the number of random effects q to be quite small (treated as constant in the theory). This assumption can be relaxed by, e.g., taking Ψ* to be a general diagonal matrix, i.e., ${\Psi }^{\ast }=\text{diag}({\tau }_1^{\ast 2},\dots ,{\tau }_q^{\ast 2})$, and assuming that a small number of ${\tau }_j^{\ast 2}’\mathrm{s}$ are nonzero, i.e., cardinality of $T\equiv \{j:{\tau }_j^{\ast 2}\ne 0\}$ is small, less than n. Then, instead of screening for fixed effects in Step 1, we can screen for both fixed and random effects by incorporating a double penalization scheme as in Li et al. (2018). This way, in Step 2, both $\mid \widehat{S}\mid$ and $\mid \widehat{T}\mid$ are small, and we can apply Henderson’s method III as before.

A few other details should also be discussed for completeness. First, multiple testing can be handled using the Westfall-Young procedure of Bühlmann (2013). This multiple testing adjustment, which strongly controls the family-wise error rate, can directly be used in conjunction with our method for generating p-values for the individual hypothesis tests. Second, the ridge-based framework of Bühlmann (2013), which is a basis for our method, is known to not have optimal power. Bühlmann (2013) shows that the detection rate may be larger than N^−1/2, whereas, under certain conditions, the detection limit for the de-biased lasso approach of Zhang and Zhang (2014) is in the N^−1/2 range. A possible extension of our work is to build a lasso-based inferential framework for high-dimensional linear mixed effect models. In fact, as suggested in the Introduction, our methods can be adapted to other high-dimensional estimators; and ridge is just an example. From van de Geer et al. (2014), we can obtain asymptotically optimal inference for linear fixed effect models—i.e., for y = Xβ* + ϵ with N observations and ϵ_i i.i.d N(0, σ*²)—by leveraging the fact that the Lasso estimator with non-negative penalty parameter λ, $\widehat{\beta }(\lambda )$, can be rewritten as

$$\widehat{\beta }(\lambda )-{\beta }^{\ast }+\lambda \widehat{\Theta }\widehat{ı}=\lambda \widehat{\Theta }{\mathbf{X}}^Tϵ∕N-\Delta ∕\sqrt{N},\text{where}\Delta ≔\sqrt{N}(\widehat{\Theta }\widehat{\Sigma }-{\mathbf{I}}_{p\times p})(\widehat{\beta }(\lambda )-{\beta }^{\ast })$$

by inverting the KKT conditions, with $\widehat{ı}$ arising from the subdifferential of ∥β∥₁. Taking $\widehat{\Theta }$ to be a reasonably good approximation of an inverse of $\widehat{\Sigma }$, the Δ term becomes asymptotically negligible, and we can use the normality of ϵ to develop asymptotically valid tests and confidence intervals for β*. (The scaled lasso furnishes a consistent estimator of σ*².) Extending this approach to the linear mixed-effect setup (per Section 2) requires meeting the challenge that the ϵ_i are no longer i.i.d., which could be addressed using the methods of Section 3.2.

A. Appendix

A.1. Proof of Results in Section 3

A.1.1. Proof of Lemma 1

It is straightforward to show that Ω* can be lower bounded as

$${\Omega }^{\ast }\ge c(\widehat{\Sigma }+\lambda {\mathbf{I}}_{p\times p}{)}^{-1}\widehat{\Sigma }(\widehat{\Sigma }+\lambda {\mathbf{I}}_{p\times p}{)}^{-1}\equiv {\tilde{\Omega }}^{\ast },$$

for some c satisfying 0 < c < ν_min (V(σ*², τ*²)). Since σ*² is positive, ν_min (V(σ*², τ*²)) > 0. Note that ${\tilde{\Omega }}^{\ast }$ can alternatively be written as

$${\tilde{\Omega }}^{\ast }=\Gamma \text{diag}\left(\frac{s_1^2}{(s_1^2+\lambda {)}^2},\dots ,\frac{s_N^2}{(s_N^2+\lambda {)}^2}\right){\Gamma }^T,$$

which, in turn, implies that

$${\tilde{\omega }}_{\text{min}}^{\ast }=\underset{j\in \{1,\dots ,p\}}{\text{min}}\sum _{k=1}^N\frac{s_k^2}{(s_k^2+\lambda {)}^2}{\Gamma }_{jk}^2,$$

and the claim follows.

A.1.2. Proof of Proposition 2

This was proven in Shao and Deng (2012) (see proof of their Theorem 1). Define Γ = [Γ′ (Γ)_⊥]; Γ′ is orthogonal, i.e., Γ′^TΓ′ = Γ′Γ′^T = I_p×p . By definition (3.2), we have

$$\begin{gathered} \mathbb{E}[\widehat{\beta }]-{\theta }^{\ast }=\frac{1}{N}(\widehat{\Sigma }+\lambda {\mathbf{I}}_{p\times p}{)}^{-1}{\mathbf{X}}^T\mathbf{X}{\theta }^{\ast }-{\theta }^{\ast } \\ =-({\lambda }^{-1}{N}^{-1}{\mathbf{X}}^T\mathbf{X}+{\mathbf{I}}_{p\times p}{)}^{-1}{\theta }^{\ast } \\ =-{\Gamma }^{\prime }({\lambda }^{-1}{N}^{-1}{\Gamma }^{\prime T}{\mathbf{X}}^T\mathbf{X}{\Gamma }^{\prime }+{\mathbf{I}}_{p\times p}{)}^{-1}{\Gamma }^{\prime T}\Gamma {\Gamma }^T{\theta }^{\ast } \\ -\Gamma ({\lambda }^{-1}{N}^{-1}{\mathbf{D}}^2+{\mathbf{I}}_{R\times R}{)}^{-1}{\Gamma }^T{\theta }^{\ast }. \end{gathered}$$

Observing that the diagonal entries to D are positive, one obtains

$$({\lambda }^{-1}{N}^{-1}{\mathbf{D}}^2+{\mathbf{I}}_{R\times R}{)}^{-1}\underset{¯}{\prec }\frac{{\lambda }^{-1}∕{\nu }_{\text{min},+}(\widehat{\Sigma })}{1+{\lambda }^{-1}∕{\nu }_{\text{min},+}(\widehat{\Sigma })}{\mathbf{I}}_{R\times R},$$ (A.1)

which, combined with the fact that Γ^TΓ = I_R×R, we obtain

$$\underset{j\in \{1,\dots ,p\}}{\max }\mid \mathbb{E}[{\widehat{\beta }}_j]-{\theta }_j^{\ast }\mid \le \lambda \Vert {\theta }^{\ast }{\Vert }_2{\nu }_{\text{min},+}(\widehat{\Sigma }{)}^{-1},$$

as desired. The bound on the variance follows directly from (3.3).

A.2. Proof of Theorems 4, 8 and 9

We first establish Theorem 4, which follows directly from Proposition 2.

A.2.1. Proof of Theorem 4

It follows from Proposition 2 that

$$\underset{j}{\max }{\kappa }_j\mid {\delta }_j\mid =\underset{j}{\max }{\kappa }_j\mid \mathbb{E}[{\widehat{\beta }}_j]-{\theta }_j^{\ast }\mid \le \frac{\lambda \Vert {\theta }^{\ast }{\Vert }_2{\nu }_{\text{min},+}{\left(\widehat{\Sigma }\right)}^{-1}}{N^{-1∕2}{\omega }_{jj}^{\ast 1∕2}}\le \frac{\lambda \Vert {\theta }^{\ast }{\Vert }_2{\nu }_{\text{min},+}{\left(\widehat{\Sigma }\right)}^{-1}}{N^{-1∕2}{\omega }_{\text{min}}^{\ast 1∕2}},$$

which, due to our choice of the ridge penalty parameter λ > 0 in (3.12), is o(1) as N, p α ∞. The claim now follows from (3.10) and the assumption given by (3.13).

Because there is an overlap in the lemmas used to prove Theorems 8 and 9, we present them together. Define u* = ∥X^T(y – Xβ*)∥_∞/N.

Lemma 11.Let

$${\lambda }_L=\frac{(\xi +1)}{(\xi -1)}\sqrt{\frac{2({\sigma }^{\ast 2}+{\tau }^{\ast 2}qn)(\log (p)-\log (\epsilon ∕2))}{N}}.$$ (A.2)

Under the model given by (2.3), the event u* ≤ λ_L(ξ – 1)/(ξ + 1) occurs with probability greater than 1 – ε.

Proof. Define $u_j=x_j^T(y-\mathbf{X}{\beta }^{\ast })∕N$. Then u* = max_j∣u_j∣. Under model (2.3), we observe that,

$$u_j\sim N(0,x_j^T\mathbf{V}({\sigma }^{\ast 2},{\tau }^{\ast 2})x_j)$$

It follows from the Gaussianity of u_j (in fact, sub-Gaussianity would suffice) that

$$\mathbb{P}[\mid u_j\mid >{\lambda }_L(\xi -1)∕(\xi +1)]\le 2\text{exp}\left\{-\frac{{\lambda }_L^2(\xi -1{)}^2∕(\xi +1{)}^2}{2x_j^T\mathbf{V}({\sigma }^{\ast 2},{\tau }^{\ast 2})x_j}\right\}\le 2\text{exp}\left\{-\frac{{\lambda }_L^2(\xi -1{)}^2∕(\xi +1{)}^2}{2N{v}_{\max }(\mathbf{V}({\sigma }^{\ast 2},{\tau }^{\ast 2}))}\right\}\le \frac{\epsilon }{p}.$$ (A.3)

The second inequality follows from the fact that the columns of X are standardized such that $\Vert x_j{\Vert }_2^2=N\forall j$. For the third inequality, recall that the columns of Z are standardized such that $\Vert z_j{\Vert }_2^2=n\forall j$, which implies that the largest eigenvalue of V(σ*², τ*²), the true covariance of y, satisfies ν_max(V(σ*, τ*)) ≤ σ*² + τ*²qn. The third inequality in (A.3) is obtained by plugging in our choice of λ_L (3.28). Employing a union bound, we then have

$$\mathbb{P}[u^{\ast }\le {\lambda }_L(\xi -1)∕(\xi +1)]\ge 1-\sum _{j=1}^p\mathbb{P}[\mid u_j\mid >{\lambda }_L(\xi -1)∕(\xi +1)]\ge 1-\epsilon ,$$

This is our desired result.

A.2.2. Proof of Lemma 6

We use arguments similar to those employed in the proof of Theorem 3 in Ye and Zhang (2010). Suppose that u* ≤ λ_L. Define $h={\widehat{\beta }}^L-{\beta }^{\ast }$. The Karuhn-Kush-TUcker (KKT) optimality conditions for Lasso is given by

$$\{\begin{array}{cc}\frac{x_j^T(y-\mathbf{X}{\widehat{\beta }}^L)}{N}={\lambda }_L\text{sign}({\widehat{\beta }}_j^L),\hfill & {\widehat{\beta }}_j^L\ne 0,\hfill \\ \frac{x_j^T(y-\mathbf{X}{\widehat{\beta }}^L)}{N}\in {\lambda }_L[-1,+1],\hfill & {\widehat{\beta }}_j^L=0.\hfill \end{array}\phantom{\}}$$

With some rearrangement, the KKT conditions can be rewritten as

$$\frac{{\mathbf{X}}^T(y-\mathbf{X}{\beta }^{\ast })-\widehat{\Sigma }h}{N}={\lambda }_L\widehat{ı}$$ (A.4)

with $\widehat{ı}\in {\mathbb{R}}^p$ and ${ı}_j=\text{sign}({\widehat{\beta }}_j^L)$ if $j\in \widehat{S}$ and ι_j ∈ [−1, +1] otherwise: the subdifferential which arises from ∥β∥₁. Rearranging (A.4) and observing that $\text{sign}({\widehat{\beta }}_j^L)=\text{sign}(h_j)$ for j ∉ S yields

$$h^{\prime T}\widehat{\Sigma }h\le (u^{\ast }+{\lambda }_L)\Vert h_S^{\prime }{\Vert }_1+(u^{\ast }-{\lambda }_L)\Vert h_{S^c}^{\prime }{\Vert }_1$$

for all vectors h’ with $\text{sign}(h_{S^c}^{\prime })=\text{sign}(h_{S^c})$. If we take h’ = h, one can see that $h\in \mathcal{C}(\xi ,S)$:

$$\begin{gathered} 0\le h^T\widehat{\Sigma }h\le (u^{\ast }+{\lambda }_L)\Vert h_S^{\prime }{\Vert }_1+(u^{\ast }-{\lambda }_L)\Vert h_{S^c}^{\prime }{\Vert }_1 \\ \Rightarrow \Vert h_{S^c}^{\prime }{\Vert }_1\le \frac{(u^{\ast }+{\lambda }_L)}{({\lambda }_L-u^{\ast })}\Vert h_S^{\prime }{\Vert }_1\le \xi \Vert h_S^{\prime }{\Vert }_1. \end{gathered}$$

On the other hand, setting h′ to be any vector so that for some j ∈ S^c, $h_j^{\prime }=h_j$ and 0 elsewhere gives

$$h_j{\widehat{\Sigma }}_{j,}.h\le (u^{\ast }-{\lambda }_L)\mid h_j\mid \le 0,$$

which implies that $h\in {\mathcal{C}}_{-}(\xi ,S)$. The KKT conditions (A.4) also tell us that

$$\Vert \widehat{\Sigma }h{\Vert }_{\infty }\le u^{\ast }+{\lambda }_L,$$

which, when combined with the definition of ζ (3.26) yields

$$\Vert h{\Vert }_{\infty }\le \frac{u^{\ast }+{\lambda }_L}{\zeta },$$

which is the desired result. In the event that u* ≤ λ_L(ξ – 1)/(ξ + 1), we have

$$\Vert h{\Vert }_{\infty }\le \frac{2{\lambda }_L}{\zeta (\xi +1)}.$$

A.2.3. Proof of Lemma 7

The proof is adapted from that of Theorem 3 in Sun and Zhang (2012). By construction, ${\widehat{\beta }}^L$ satisfies the KKT conditions from (A.4) which implies that

$$\begin{gathered} \frac{\mid x_j^{T}X({\widehat{\beta }}^L-{\beta }^{\ast })\mid }{N}=\frac{\mid x_j^T(y-\mathbf{X}{\widehat{\beta }}^L-ϵ)\mid }{N} \\ \ge \frac{\mid x_j^T(y-\mathbf{X}{\widehat{\beta }}^L-ϵ)\mid }{N}-\frac{\mid x_j^Tϵ\mid }{N} \\ \ge {\lambda }_L-u^{\ast }. \end{gathered}$$

For $\mathcal{A}\subseteq \widehat{S}\setminus S$, such that $\mid \mathcal{A}\mid \le N^{\prime }$, the previous inequality implies

$$\begin{gathered} ({\lambda }_L-u^{\ast }{)}^2\mid \mathcal{A}\mid \le \frac{{\sum }_{j\in A}\mid x_j^T\mathbf{X}({\widehat{\beta }}^L-{\beta }^{\ast }){\mid }^2}{N^2} \\ =\frac{{\sum }_{j\in A}(\mathbf{X}h{)}^T{x}_j{x}_j^T(\mathbf{X}h)}{N^2}\le \frac{{\kappa }_{+}(N^{\prime },S)\Vert \mathbf{X}h{\Vert }_2^2}{N}. \end{gathered}$$ (A.5)

Going back to the KKT conditions in (A.4), we have, for arbitrary $h^{\prime }\in {\mathbb{R}}^p$,

$$\frac{(\mathbf{X}{\widehat{\beta }}^L-\mathbf{X}{h}^{\prime }{)}^T\mathbf{X}h}{N}\le {\lambda }_L(\Vert h^{\prime }{\Vert }_1-\Vert {\widehat{\beta }}^L{\Vert }_1)+u^{\ast }\Vert h^{\prime }-{\widehat{\beta }}^L{\Vert }_1,$$

which, when combined with the fact that

$$2(\mathbf{X}{\widehat{\beta }}^L-\mathbf{X}{h}^{\prime }{)}^T\mathbf{X}h=\Vert \mathbf{X}{\widehat{\beta }}^L-\mathbf{X}{h}^{\prime }{\Vert }_2^2+\Vert \mathbf{X}h{\Vert }_2^2-\Vert \mathbf{X}{\beta }^{\ast }-\mathbf{X}{h}^{\prime }{\Vert }_2^2$$

gives the inequality

$$\begin{gathered} \frac{\Vert \mathbf{X}h{\Vert }_2^2}{N}\le {\lambda }_L(\Vert {\beta }^{\ast }{\Vert }_1-\Vert {\widehat{\beta }}^L{\Vert }_1)+u^{\ast }\Vert h{\Vert }_1 \\ \le ({\lambda }_L+u^{\ast })\Vert h_S{\Vert }_1. \end{gathered}$$ (A.6)

Thus, h lies in the cone in (3.25) in the event that u* ≤ λ_L(ξ – 1)/(ξ + 1) (by noting that the left-hand side is lower bounded by 0). By definition of κ(ξ, S) from (3.30),

$$\frac{\Vert \mathbf{X}h{\Vert }_2^2}{N}\le \frac{({\lambda }_L+u^{\ast }{)}^{2}d}{{\kappa }^2(\xi ,S)},$$

which, when combined with (A.5) implies

$$\mid \mathcal{A}\mid \le \frac{{\kappa }_{+}(N^{\prime },S){\xi }^{2}d}{{\kappa }^2(\xi ,S)}<N^{\prime },$$

by Assumption 2.

A.2.4. Proof of Theorem 8

Suppose that u* ≤ λ_L(ξ − 1)/(ξ + 1). Then by Lemmas 6 and 7 and the referenced assumptions within, we have

$$\Vert {\widehat{\beta }}^L-{\beta }^{\ast }{\Vert }_{\infty }\le \frac{2\xi {\lambda }_L}{(\xi +1)\zeta }\Rightarrow \mid {\beta }_j^{\ast }\mid \le \frac{4\xi {\lambda }_L}{(\xi +1)\zeta }\text{for all}j\in S\setminus \widehat{S},$$ (A.7)

$$\mid \widehat{S}\setminus S\mid \le N^{\prime }\Rightarrow \mid \widehat{S}\mid \le N^{\prime }+d\lesssim d.$$ (A.8)

Denote $\widehat{\mathbf{X}}=[{\mathbf{X}}_{\widehat{S}}\mathbf{Z}]$. Under candidate model (3.18), our variance component estimators (via Henderson’s Method III) are given by

$$\begin{gathered} {\widehat{\sigma }}^2=\frac{y^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})y}{N-\text{rank}(\widehat{\mathbf{X}})}, \\ {\widehat{\tau }}^2=\frac{y^T\left({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}\right)y-{\widehat{\sigma }}^2[\text{rank}(\widehat{\mathbf{X}})-\text{rank}({\mathbf{X}}_{\widehat{S}})]}{\text{tr}\left[{\mathbf{Z}}^T\left({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}\right)\mathbf{Z}\right]}. \end{gathered}$$

See (3.23) and (3.24).

Consider the more interesting scenario where $\mid S\setminus \widehat{S}\mid >0$. If S is contained within $\widehat{S}$, then it is straightforward to show that variance component estimators are consistent (the true model is a submodel of the proposed one). We first prove, under the given assumptions, that $\mid {\widehat{\sigma }}^2-{\sigma }^{\ast 2}\mid =o_P(1)$. Write $S_O=S\setminus \widehat{S}$, ‘O’ for omitted. Then,

$$\begin{gathered} {\widehat{\sigma }}^2=\frac{(\widehat{\mathbf{X}}{\beta }_{\widehat{S}}^{\ast }+{\mathbf{X}}_{S^O}{\beta }_{S^O}^{\ast }+ϵ{)}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})(\widehat{\mathbf{X}}{\beta }_{\widehat{S}}+{\mathbf{X}}_{S^O}{\beta }_{S^O}^{\ast }+ϵ)}{N-\text{rank}(\widehat{\mathbf{X}})} \\ =\frac{{\beta }_{S^O}^{\ast T}{\mathbf{X}}_{S^O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}}){\mathbf{X}}_{S^O}{\beta }_{S^O}^{\ast }}{N-\text{rank}(\widehat{\mathbf{X}})}+\frac{2{\beta }_{S^O}^{\ast T}{\mathbf{X}}_{S^O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}+\frac{{ϵ}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}. \end{gathered}$$ (A.9)

We proceed to show that the three parts to (A.9) satisfy

$$\mid \frac{{ϵ}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}-{\sigma }^{\ast 2}\mid =o_P(1),$$ (A.10)

$$\mid \frac{2{\beta }_{S^O}^{\ast T}{\mathbf{X}}_{S^O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}\mid =o_P(1),$$ (A.11)

$$\text{and}\frac{{\beta }_{S^O}^{\ast T}{\mathbf{X}}_{S^O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}}){\mathbf{X}}_{S^O}{\beta }_{S^O}^{\ast }}{N-\text{rank}(\widehat{\mathbf{X}})}=o(1),$$ (A.12)

which would suggest that ${\widehat{\sigma }}^2$ is indeed consistent for σ*². Note that the term in (A.12) is $\text{Bias}({\widehat{\sigma }}^2)=\mathbb{E}[{\widehat{\sigma }}^2]-{\sigma }^{\ast 2}$.

1. Proving (A.11): Let ${\Gamma }_{{\widehat{\mathbf{X}}}_{\perp }}{\mathbf{D}}_{{\widehat{\mathbf{X}}}_{\perp }}{\Gamma }_{{\widehat{\mathbf{X}}}_{\perp }}^T$ represent the eigendecomposition of ${\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}}$. We note that the latter is idempotent, implying that the diagonal matrix ${\mathbf{D}}_{{\widehat{\mathbf{X}}}_{\perp }}$, which is of rank $N-\text{rank}(\widehat{\mathbf{X}})$, has only 0 and 1s as its eigenvalues. It is straightforward to show that

   $$\begin{gathered} \mathbb{E}\left[\frac{2{\beta }_{S^O}^{\ast T}{\mathbf{X}}_{S^O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}\right]=0\text{and} \\ \text{Var}\left[\frac{2{\beta }_{S^O}^{\ast T}{\mathbf{X}}_{S^O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}})ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}\right]=\frac{4{\sigma }^{\ast 2}{\beta }_{S^O}^{\ast T}{\mathbf{X}}_{S^O}^T{\Gamma }_{{\widehat{\mathbf{X}}}_{\perp }}{\mathbf{D}}_{{\widehat{\mathbf{X}}}_{\perp }}{\Gamma }_{{\widehat{\mathbf{X}}}_{\perp }}^T{\mathbf{X}}_{S^O}{\beta }_{S^O}^{\ast }}{[N-\text{rank}(\widehat{\mathbf{X}}){]}^2} \\ \le \frac{4{\sigma }^{\ast 2}\Vert {\Gamma }_{{\widehat{\mathbf{X}}}_{\perp }}^T{\mathbf{X}}_{S^O}{\beta }_{S^O}^{\ast }{\Vert }_{\infty }^2}{N-\text{rank}(\widehat{\mathbf{X}})} \\ \stackrel{\prec }{\sim }\frac{4{\sigma }^{\ast 2}\Vert {\Gamma }_{{\widehat{\mathbf{X}}}_{\perp }}^T{\mathbf{X}}_{S^O}{\Vert }_{\infty }^2}{N-\text{rank}(\widehat{\mathbf{X}})}\times \frac{d^{2}q\log p}{M}=o(1). \end{gathered}$$

   Statement (A.11) then follows from Chebyshev’s inequality.

2. Proving (A.10): Let ${\chi }_i^2(1)$, $i=1,\dots ,N-\text{rank}(\widehat{\mathbf{X}})$, be i.i.d random variables following a χ² distribution with 1 degree of freedom. Observe that,

   $$\frac{{ϵ}^T[{\mathbf{I}}_{N\times N}-{\mathbf{P}}_{\widehat{\mathbf{X}}}]ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}{=}_d\frac{{ϵ}^T{\mathbf{D}}_{{\widehat{\mathbf{X}}}_{\perp }}ϵ}{N-\text{rank}(\widehat{\mathbf{X}})}{=}_d\frac{{\sigma }^{\ast 2}{\sum }_{i=1}^{N-\text{rank}(\widehat{\mathbf{X}})}{\chi }_i^2(1)}{N-\text{rank}(\widehat{\mathbf{X}})}$$ (A.13)
   and

   $$\mid \frac{{\sigma }^{\ast 2}{\sum }_{i=1}^{N-\text{rank}(\widehat{\mathbf{X}})}{\chi }_i^2(1)}{N-\text{rank}(\widehat{\mathbf{X}})}-{\sigma }^{\ast 2}\mid =o_P(1).$$ (A.14)

   Here, (A.14) follows from Lemma 7 and the fact that N ≿ N′ + d + qM, which implies that $N-\text{rank}(\widehat{\mathbf{X}})\to \infty$ as M α ∞. Applying the Strong Law of Large Numbers (SLLN) for i.i.d. random variables, we arrive at (A.10).

3. Proving (A.12): We observe that

   $$\begin{gathered} \mid \text{Bias}({\widehat{\sigma }}^2)\mid \le \Vert {\Gamma }_{{\widehat{\mathbf{X}}}_{\perp }}^T{\mathbf{X}}_{S_O}{\Vert }_{\infty }^2\times \Vert {\beta }_{S_O}^{\ast }{\Vert }_{\infty }^2\times d^2 \\ \lesssim \frac{d^{2}q\log (p)}{M}=o(1), \end{gathered}$$

   and we have completed our proof that ${\widehat{\sigma }}^2$ is consistent under the stated assumptions.

We now demonstrate that the same claim holds for ${\widehat{\tau }}^2$. Expanding out y, we obtain, after some algebraic manipulation,

$$\begin{gathered} {\widehat{\tau }}^2=\frac{{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}){\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}+\frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}+\frac{{ϵ}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })} \\ +\frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}+\frac{v^T{\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}+\frac{2v^T{\mathbf{Z}}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })} \\ -\frac{{\sigma }^{\ast 2}[\text{rank}(\widehat{\mathbf{X}})-\text{rank}({\mathbf{X}}_{\widehat{S}})]}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}-\frac{\text{Bias}({\widehat{\sigma }}^2)[\text{rank}(\widehat{\mathbf{X}})-\text{rank}({\mathbf{X}}_{\widehat{S}})]}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}, \end{gathered}$$ (A.15)

where we have defined ${\mathbf{Z}}^{\prime }=({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}$. We set out to prove that the terms in (A.15) satisfy

$$\mid \frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\mid =o_P(1),$$ (A.16)

$$\mid \frac{{ϵ}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}-\frac{{\sigma }^{\ast 2}[\text{rank}(\widehat{\mathbf{X}})-\text{rank}({\mathbf{X}}_{\widehat{S}})]}{\text{tr}[{\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }]}\mid =o_P(1),$$ (A.17)

$$\mid \frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\mid =o_P(1),$$ (A.18)

$$\mid \frac{v^T{\mathbf{Z}}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}-{\tau }^{\ast 2}\mid =o_P(1),$$ (A.19)

$$\mid \frac{2v^T{\mathbf{Z}}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\mid =o_P(1),$$ (A.20)

and

$$\frac{{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}){\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}-\frac{\text{Bias}({\widehat{\sigma }}^2)[\text{rank}(\widehat{\mathbf{X}})-\text{rank}({\mathbf{X}}_{\widehat{S}})]}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}=o(1).$$ (A.21)

Let ${\mathbf{Q}}_{{\mathbf{Z}}^{\prime }}{\mathbf{D}}_{{\mathbf{Z}}^{\prime }}{\Gamma }_{{\mathbf{Z}}^{\prime }}^T$ represent the singular value decomposition of Z′, with Q_Z′ and Γ_Z′ of dimensions N × qM and qM × qM, respectively. Additionally, write the eigendecompositions of ${\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}$ and ${\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}$ as ${\Gamma }_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}{\mathbf{D}}_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}{\Gamma }_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}^T$ and ${\Gamma }_{{\mathbf{X}}_{\widehat{S}}\perp }{\mathbf{D}}_{{\mathbf{X}}_{\widehat{S}}\perp }{\Gamma }_{{\mathbf{X}}_{\widehat{S}}\perp }^T$, respectively. Note that (A.19) and (A.21) make up $\text{Bias}[{\widehat{\tau }}^2]=\mathbb{E}[{\widehat{\tau }}^2]-{\tau }^{\ast 2}$.

To avoid repetition, some of the proofs are presented in abbreviated form.

1. Proving (A.16): Clearly,

   $$\begin{gathered} \mathbb{E}\left[\frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=0 \\ \text{Var}\left[\frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=\frac{4{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}){\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }}{\text{tr}[{\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }{]}^2} \\ =\frac{4{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T{\Gamma }_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}{\mathbf{D}}_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}{\Gamma }_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}^T{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }}{\text{tr}[{\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }{]}^2} \\ \le \frac{4\times d^2\times qM\times \Vert {\Gamma }_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}{\mathbf{X}}_{S_O}{\Vert }_{\infty }\times \Vert {\beta }_{S_O}^{\ast }{\Vert }_{\infty }^2}{\text{tr}[{\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }{]}^2} \\ =o(1), \end{gathered}$$

   following from (3.32) in Assumption 3 and Assumption 4.

2. Proving (A.17): Orthogonality of ${\Gamma }_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}$ implies that ${ϵ}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ{=}_d{ϵ}^T{\mathbf{D}}_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}ϵ$, so

   $$\mathbb{E}\left[\frac{{ϵ}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=\mathbb{E}\left[\frac{{ϵ}^T{\mathbf{D}}_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=\frac{{\sigma }^{\ast 2}[\text{rank}(\widehat{\mathbf{X}})-\text{rank}({\mathbf{X}}_{\widehat{S}})]}{\text{tr}[{\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }]}$$
   and, using properties of quadratic forms, we have

   $$\begin{gathered} \text{Var}\left[\frac{{ϵ}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=\text{Var}\left[\frac{{ϵ}^T{\mathbf{D}}_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}}ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right] \\ =\frac{2{\sigma }^{\ast 4}\text{rank}({\mathbf{D}}_{\widehat{\mathbf{X}}\perp {\mathbf{X}}_{\widehat{S}}})}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }{)}^2} \\ =\frac{2{\sigma }^{\ast 4}\text{rank}({\mathbf{Z}}^{\prime })}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }{)}^2}\lesssim \frac{1}{M}=o(1), \end{gathered}$$

   the latter relation following from (3.33) in Assumption 3. This proves (A.17).
3. Proving (A.18): Proof is similar to that of (A.16), as we note that

   $$\begin{gathered} \mathbb{E}\left[\frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=0, \\ \text{Var}\left[\frac{2{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=\frac{4{\tau }^{\ast 2}{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T{\mathbf{Q}}_{{\mathbf{Z}}^{\prime }}{\mathbf{D}}_{{\mathbf{Z}}^{\prime }}^2{\mathbf{Q}}_{{\mathbf{Z}}^{\prime }}^T{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }{)}^2} \\ \le \frac{4{\tau }^{\ast 2}\text{tr}({\mathbf{Z}}^{\prime T}\mathbf{Z})\Vert {\mathbf{Q}}_{{\mathbf{Z}}^{\prime }}^T{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }{\Vert }_{\infty }^2}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime }{)}^2} \\ =o(1), \end{gathered}$$

   having applied Assumptions 3 and 4 here.
4. Proving (A.19): As for (A.17), using again properties of quadratic forms, we can show that

   $$\begin{gathered} \mathbb{E}\left[\frac{v^T{\mathbf{Z}}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]={\tau }^{\ast 2}, \\ \text{Var}\left[\frac{v^T{\mathbf{Z}}^T({\mathbf{I}}_{N\times N}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}})\mathbf{Z}v}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}\right]=\frac{2{\tau }^{\ast 4}\text{tr}({\mathbf{D}}_{{\mathbf{Z}}^{\prime }}^4)}{\text{tr}({\mathbf{D}}_{{\mathbf{Z}}^{\prime }}^2{)}^2}=o(1), \end{gathered}$$

   the last relation the result of (3.33) from Assumption 3.
5. Proving (A.20): We can rewrite

   $$\frac{2v^T{\mathbf{Z}}^{\prime T}ϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}=\frac{2v^T{\Gamma }_{{\mathbf{Z}}^{\prime }}{\mathbf{D}}_{{\mathbf{Z}}^{\prime }}{\mathbf{Q}}_{{\mathbf{Z}}^{\prime }}^Tϵ}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}{=}_d\frac{{\sigma }^{\ast }{\tau }^{\ast }{\sum }_{i=1}^{qM}{s}_i{B}_i}{{\sum }_{i=1}^{qM}{s}_i^2}.$$

   where B_i, i = 1,… rank(Z′) are random variables formed as the product of two independent N(0, 1) random variables. Then,

   $$\text{Var}\left(\sum _{i=1}^{qM}{s}_i{B}_i\right)=\sum _{i=1}^{qM}{s}_i^2,$$

   which implies that

   $$\text{Var}\left(\frac{{\sigma }^{\ast }{\tau }^{\ast }{\sum }_{i=1}^{qM}{s}_i{B}_i}{{\sum }_{i=1}^{qM}{s}_i^2}\right)=\frac{1}{{\sum }_{i=1}^{qM}{s}_i^2}$$

   and since ${\sum }_{i=1}^{qM}{s}_i^2\asymp qM$ by (3.32) in Assumption 3, we have proven our claim (A.20).

6. Proving (A.21): By the definition of $\text{Bias}[{\widehat{\tau }}^2]$, it is clear that

   $$\begin{gathered} \mid \text{Bias}({\widehat{\tau }}^2)\mid \le \frac{{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T({\mathbf{P}}_{\widehat{\mathbf{X}}}-{\mathbf{P}}_{{\mathbf{X}}_{\widehat{S}}}){\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}+\frac{\mid \text{Bias}({\widehat{\sigma }}^2)\mid [\text{rank}(\widehat{\mathbf{X}})-\text{rank}({\mathbf{X}}_{\widehat{S}})]}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })} \\ \le \frac{{\beta }_{S_O}^{\ast T}{\mathbf{X}}_{S_O}^T{\mathbf{P}}_{\mathbf{Z}}{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}+\frac{qM\times \mid \text{Bias}({\widehat{\sigma }}^2)\mid }{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })} \\ \le \frac{qM\times d\times \Vert {\Gamma }_{\mathbf{Z}}{\mathbf{X}}_{S^O}{\Vert }_{\infty }^2\times \Vert {\beta }_{S_O}^{\ast }{\Vert }_{\infty }^2}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })}+\frac{qM\times \text{Bias}({\widehat{\sigma }}^2)}{\text{tr}({\mathbf{Z}}^{\prime T}{\mathbf{Z}}^{\prime })} \\ \lesssim \frac{d^{2}q\log (p)}{M}=o(1), \end{gathered}$$

   where the second last relation follows from the proven claim that $\text{Bias}({\widehat{\sigma }}^2)$ is o(1), (3.32) in Assumption 3 and Assumption 4.

Since the event u* ≤ λ_L(ξ − 1)/(ξ + 1) occurs with probability greater than 1 – 1/p → 1 as p → ∞,

$$\begin{gathered} \mid {\widehat{\sigma }}^2-{\sigma }^{\ast 2}\mid =o_P(1), \\ \mid {\widehat{\tau }}^2-{\tau }^{\ast 2}\mid =o_P(1) \end{gathered}$$

as claimed.

A.2.5. Proof of Theorem 9

Suppose that u* ≤ λ_L(ξ − 1)/(ξ + 1), and write $S_O=S\setminus \widehat{S}$. The OLS fit ${\widehat{\beta }}^{\text{init}}$ has a simple closed-form expression:

$$\begin{gathered} {\widehat{\beta }}_{\widehat{S}}^{\text{init}}=({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^{T}y \\ ={\beta }_{\widehat{S}}^{\ast }+({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }+({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T(y-\mathbf{X}{\beta }^{\ast }). \end{gathered}$$

and ${\widehat{\beta }}_{{\widehat{S}}^c}^{\text{init}}=0$. Thus, by triangle inequality,

$$\Vert {\widehat{\beta }}_{\widehat{S}}^{\text{init}}-{\beta }_{\widehat{S}}^{\ast }{\Vert }_1\le \Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }{\Vert }_1+\Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T(y-\mathbf{X}{\beta }^{\ast }){\Vert }_1.$$ (A.22)

We proceed by first bounding the first term on the right-hand side of (A.22). By Assumption 5,

$$\begin{gathered} \Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\Vert }_2\le \sqrt{{\nu }_{\max }[({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}]} \\ =\sqrt{{\nu }_{\max }[({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}]} \\ =\sqrt{\frac{1}{N}{\nu }_{\max }\left[({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}∕N{)}^{-1}\right]} \\ =\sqrt{\frac{1}{N}∕{\nu }_{\text{min}}\left({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}∕N\right)}\le \frac{1}{\sqrt{N{\psi }_{-}(N^{\prime },S)}}\lesssim \frac{1}{\sqrt{N}}. \end{gathered}$$

This, in turn, implies that

$$\begin{gathered} \Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }{\Vert }_1\le \sqrt{N^{\prime }+d}\Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }{\Vert }_2 \\ \le \sqrt{N^{\prime }+d}\Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\Vert }_2\Vert {\mathbf{X}}_{S_O}{\beta }_{S_O}^{\ast }{\Vert }_2 \\ \le \sqrt{N^{\prime }+d}\Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T{\Vert }_2\Vert {\mathbf{X}}_{S_O}{\Vert }_F\Vert {\beta }_{S_O}^{\ast }{\Vert }_{\infty } \\ \le \sqrt{\frac{(N^{\prime }+\mid S\mid )d}{{\psi }_{-}(N^{\prime },S)}}\frac{2\xi {\lambda }_L}{(\xi +1)\zeta }\lesssim \sqrt{\frac{d^{2}q\log (p)}{M}}=o(1), \end{gathered}$$

where the last relation follows from Assumption 2.

We proceed to bound the second component on the right-hand side of (A.22). We observe that

$$\begin{gathered} \Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^Tϵ{\Vert }_1\le \sqrt{N^{\prime }+d}\Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}{)}^{-1}{\mathbf{X}}_{\widehat{S}}^T(y-\mathbf{X}{\beta }^{\ast }){\Vert }_2 \\ \le \sqrt{N^{\prime }+d}{\Vert ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}∕N{)}^{-1}\frac{{\mathbf{X}}_{\widehat{S}}^T(y-\mathbf{X}{\beta }^{\ast })}{N}\Vert }_2 \\ \le \sqrt{N^{\prime }+d}{\Vert \mid ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}∕N{)}^{-1}\mid \Vert }_2{\Vert \frac{{\mathbf{X}}_{\widehat{S}}^T(y-\mathbf{X}{\beta }^{\ast })}{N}\Vert }_2 \\ \le (N^{\prime }+d){\Vert \mid ({\mathbf{X}}_{\widehat{S}}^T{\mathbf{X}}_{\widehat{S}}∕N{)}^{-1}\mid \Vert }_2{\Vert \frac{{\mathbf{X}}_{\widehat{S}}^T(y-\mathbf{X}{\beta }^{\ast })}{N}\Vert }_{\infty } \\ \le \frac{N^{\prime }+d}{{\kappa }_{-}(N^{\prime },S)}{\lambda }_L\lesssim \sqrt{\frac{d^{2}q\log (p)}{M}}=o(1). \end{gathered}$$

From Lemma 6,

$$\Vert {\widehat{\beta }}_{S_O}^{\text{init}}-{\beta }_{S_O}^{\ast }{\Vert }_{\infty }=\Vert {\beta }_{S_O}^{\ast }{\Vert }_{\infty }\lesssim \sqrt{\frac{q\log (p)}{M}},$$

which implies that

$$\Vert {\widehat{\beta }}_{S_O}^{\text{init}}-{\beta }_{S_O}^{\ast }{\Vert }_1\lesssim \sqrt{\frac{d^{2}q\log (p)}{M}}=o(1).$$

By Lemma 11, the event u* ≤ λ_L(ξ − 1)/(ξ + 1) occurs with probability exceeding 1 – 1/p. Combined, we obtain the desired result.

A.3. Empirical Evaluation of Assumption 4

To assess the stringency of Assumption 4 compared to the irrepresentability condition (Zhao and Yu, 2006), we conduct a simulation study similar to Zhao and Yu (2006), but customized to our mixed linear model setting.

Consider the model

$$y=X{\beta }^{\ast }+Z\nu +\epsilon ,$$

with $X\in {\mathbb{R}}^{nM\times p}$ and $Z\in {\mathbb{R}}^{nM\times qM}$, $\beta \in {\mathbb{R}}^p$, $\nu \in {\mathbb{R}}^p$ and $\epsilon \in {\mathbb{R}}^{nM}$. We consider q = 2 random effects, M = 25 groups, and n = 20 samples within each group. Among the p fixed effect covariates, we set d to have nonzero coefficients and the rest to have zero coefficients. More specifically, we set ${\beta }^{\ast }=(\underset{d}{\underbrace{1,\dots ,1}},\underset{p-d}{\underbrace{0,\dots ,0}}{)}^{\mathsf{T}}$.

To assess the stringency of the two assumptions, in each of B = 1000 simulation replications, we randomly generate design matrices X and Z jointly as [X, Z_u] ~_iid N(0, Σ), where Z_u is the un-blocked version of Z. The covariance matrix Σ is generated from a Wishart(p+q, I_p+q) distribution, and X and Z are scaled such that $\Vert x_j{\Vert }_2^2=nM$ and $\Vert z_j{\Vert }_2^2=n$. Following Zhao and Yu (2006), we consider p = 2^k for k ∈ {3, 4,…, 8}, and set d = tp/8 for t ∈ {1, 2,…, 8}.

Let A* be the index of the true active set (hence ∣A*∣ = d). Further, let S ⊂ (A*)^c with ∣S∣ = min(p – d, p) be a random subset of variables with zero coefficients. For j ∈ A*, let $\tilde{X}=[X_{A^{\ast }\setminus \{j\}},X_S,Z]$ be the augmented design matrix, and denote $\widehat{\Sigma }={\widehat{X}}^{\mathsf{T}}\widehat{X}∕N$.

With the above notations, the irrepresentability condition is satisfied if T_IR = max_j∈A* T_IR,j < 1, where

$$T_{\text{IR},j}=\Vert {\widehat{\Sigma }}_{(A^{\ast }\setminus \{j\}{)}^c,A^{\ast }\setminus \{j\}}{\left({\widehat{\Sigma }}_{(A^{\ast }\setminus \{j\}{)}^c,A^{\ast }\setminus \{j\}}\right)}^{-1}\text{sign}({\beta }_A^{\ast }\setminus \{j\}){\Vert }_{\infty }.$$

Assumption 4 involves a related quantity, $T_{4,j}=\Vert {\Gamma }_{\tilde{X}}{x}_j{\Vert }_{\infty }$, where ${\Gamma }_{\tilde{X}}{D}_{\tilde{X}}{\Gamma }_{\tilde{X}}^{\mathsf{T}}$ is the eigen-decomposition of $\tilde{X}{\left(\tilde{X}{\tilde{X}}^{\mathsf{T}}\right)}^{-1}{\tilde{X}}^{\mathsf{T}}$. However, this assumption is satisfied if T₄ ≡ max_j∈A* T_4,j = O(1). Thus, to satisfy Assumption 4, we need a constant C, not dependent on N and p, such that T₄ < C. While C can be any large but fixed constant, in this simulation we consider a moderate value of C = 5.

The proportion of simulated data sets, where the irrepresentability assumption and Assumption 4 are satisfied are shown in Tables 2 and 3, respectively. As in Zhao and Yu (2006), the results in Table 2 indicate that the irrepresentability assumption can be stringent, especially as the dimension p and the number of nonzero coefficients d increase. In contrast, the results in Table 3 suggest that, for C = 5, Assumption 4 is much more likely to hold. Moreover, the proportion of cases for which this assumption holds does not change with p or d. While the appropriate choice of C is generally unknown, the results in this simulation suggest that even with moderate values (in this case C = 5) Assumption 4 is likely satisfied.

Table 2:

Proportions of cases satisfying the irrepresentability condition (Zhao and Yu, 2006).

TIR < 1	p = 8	p = 16	p = 32	p = 64	p = 128	p = 256
d = p/8	1	1	0.975	0.823	0.327	0.022
d = 2p/8	1	0.735	0.283	0.013	0	0
d = 3p/8	0.736	0.231	0.014	0	0	0
d = 4p/8	0.356	0.062	0.001	0	0	0
d = 5p/8	0.244	0.024	0	0	0	0
d = 6p/8	0.208	0.015	0	0	0	0
d = 7p/8	0.199	0.012	0	0	0	0

Table 3:

Proportions of cases satisfying Assumption 4.

T4 < 5	p = 8	p = 16	p = 32	p = 64	p = 128	p = 256
d = p/8	0.998	0.999	0.997	0.996	0.994	0.99
d = 2p/8	1	1	0.996	0.995	0.99	0.982
d = 3p/8	0.998	0.999	0.994	0.992	0.985	0.973
d = 4p/8	0.999	0.998	0.991	0.995	0.984	0.954
d = 5p/8	0.996	0.995	0.993	0.989	0.976	0.938
d = 6p/8	1	0.997	0.993	0.987	0.967	0.93
d = 7p/8	0.996	0.995	0.995	0.988	0.956	0.927