Biclustering analysis of functionals via penalized fusion

Abstract

In biomedical data analysis, clustering is commonly conducted. Biclustering analysis conducts clustering in both the sample and covariate dimensions and can more comprehensively describe data heterogeneity. In most of the existing biclustering analyses, scalar measurements are considered. In this study, motivated by time-course gene expression data and other examples, we take the “natural next step” and consider the biclustering analysis of functionals under which, for each covariate of each sample, a function (to be exact, its values at discrete measurement points) is present. We develop a doubly penalized fusion approach, which includes a smoothness penalty for estimating functionals and, more importantly, a fusion penalty for clustering. Statistical properties are rigorously established, providing the proposed approach a strong ground. We also develop an effective ADMM algorithm and accompanying R code. Numerical analysis, including simulations, comparisons, and the analysis of two time-course gene expression data, demonstrates the practical effectiveness of the proposed approach.

1. Introduction

In biomedical data analysis, clustering has been routinely conducted. The clustering of samples can assist better understanding sample heterogeneity, and the clustering of covariates can identify those that behave similarly across samples and then, for example, improve our understanding of covariate functionalities. Clustering can also serve as the basis of other analysis, for example, regression. Biclustering analysis has also been developed, identifying clustering structures in both sample and covariate dimensions. It includes sample- and covariate-clustering as special cases and, in a sense, can be more comprehensive. For generic reviews of techniques, theories, and applications of clustering, we refer to [19,46].

This study has been partly motivated by the analysis of gene expression data, for which sample- and covariate-clustering as well as biclustering have been extensively conducted [21,45]. Most gene expression studies generate “snapshot” values. Unlike some types of omics measurements, gene expression values can be time-dependent, and the temporal trends of gene expressions can have important biological implications [16]. Accordingly, time-course gene expression studies have been conducted, generating multiple measurements at different time points for each gene of each sample. In the analysis of time-course gene expression data, besides simple statistics, functional data analysis (FDA) techniques, have been adopted and shown as powerful [12].

FDA deals with data samples that consist of curves or other infinite-dimensional data objects. Over the last two decades, we have witnessed significant developments in its theory, method, computation, and application. For systematic reviews, we refer to [2,15,23,40]. In FDA, clustering analysis has been of particular interest. A popular approach projects functional data into a finite-dimensional space and then applies existing clustering methods. For example, Abraham et al. [1] conduct B-spline expansions, and clusters the estimated coefficients using a k-means algorithm. Peng and Müller [30] develop a distance for sparse functional data, and apply a k-means algorithm to functional principle component analysis (PCA) scores. Other approaches, such as Bayesian [37], subspace [3,9,10], and model-based [18,20], have also been developed. We refer to [17,40] for surveys on functional data clustering. Most works in this area, however, have focused on either sample- or covariate-clustering.

For biclustering analysis (of gene expression and other types of data), in this article, we take the “natural next step” and consider the scenario where for each covariate of each sample, a function or its realizations at discrete time points are available. We note that, although this study has been partly motivated by gene expression data and some of the discussions are focused on such data, the considered data scenario and proposed technique can have applications far beyond such data. For example, in biomedical studies, many biomarkers measured in blood tests vary across time, and their values can be obtained from medical records. In financial studies, many measures of a company, for example size and stock price, vary across time. As such, our investigation can have broad applications.

There is a vast literature on biclustering analysis with scalar measurements. Directly applying such techniques to the present problem will involve either treating functional measurements as scalars and then computing distances (between covariates and samples) – which may be ineffective by not sufficiently accounting for the functional nature of data, or first estimating functionals and then computing distances between the estimates – which may also encounter challenges when a large number of functionals need to be jointly estimated. Our literature review suggests that there are also a handful recent biclustering methods designed for functional (especially including longitudinal) data. For example, Slimen et al. [35] propose a biclustering method for multivariate functional data based on the Gaussian latent block model (LBM) using the first functional PCA scores. Bouveyron et al. [4] develop an extension of the Gaussian LBM by modeling the whole set of functional PCA scores. In another work [28], a biclustering method with a plaid model is extended to three-dimensional data arrays, of which multivariate longitudinal data is a special case.

For the biclustering analysis of functionals, in this article, we develop a penalized fusion based approach. More specifically, a nonparametric model is assumed for each covariate of each sample, allowing for sufficient flexibility in modeling. A doubly penalization technique is adopted, which includes a smoothness penalty to regulate nonparametric estimation. The most significant advancement is the second, fusion penalty, which “transforms” clustering in both sample and covariate dimensions to a penalized estimation problem. Statistical and numerical investigations are conducted, providing the proposed approach a solid ground. This study may complement and advance from the existing ones in multiple aspects. Compared to direct applications of biclustering methods for scalars (that either directly compute distances without functional estimation or estimate functionals separately), the proposed approach can more effectively accommodate the functional nature of data or generate more effective estimation. This is because it “combines” clustering and estimation, and as such, estimation only needs to be conducted for clusters as opposed to individual covariates, potentially leading to a smaller number of parameters and hence more effective estimation. Compared to some of the existing biclustering methods for functionals, such as [4,35], the proposed approach has a much easier way of determining the number of clusters. In addition, unlike [4,35], it does not make stringent distributional assumptions (for example, normality). Meanwhile, rigorous theoretical investigations are conducted beyond methodological developments, granting the proposed approach a stronger statistical basis. It also advances from the clustering of functional covariate effects (assuming homogeneous samples) by simultaneously examining sample heterogeneity, thus being more comprehensive. Additionally, this study may also advance and enrich the penalized fusion technique. Clustering via penalized fusion has been pioneered in [8] and other studies. Compared to alternative clustering techniques, it is more recent and has notable statistical and numerical advantages [44]. Compared to the existing penalized fusion based clustering, this study differs by conducting biclustering and by having unknown parameters generated from the basis expansion of functionals. Last but not least, this study also provides a practically useful and new way of analyzing time-course gene expression data (and other data with similar characteristics).

The remainder of this article is organized as follows: Section 2 introduces the new biclustering approach via penalized fusion and develops an effective computational algorithm. Statistical properties are established to provide our method a strong theoretical support. Simulation studies and the analysis of two time-course expression data are conducted in Sections 3 and 4, respectively. Section 5 concludes with a brief discussion. The proofs of the main results are presented in Appendix A.

2. Methods

2.1. Data and model settings

For the j ∈ {1, …, q}th covariate of sample i ∈ {1, …, N }, denote ${\mathbf{Y}}_{i,j}={\left(Y_{i,j,1},\dots ,Y_{i,j,n_{i,j}}\right)}^{\top }$ as the ordered measurements (ordered by time for time-course gene expression data), which are the discrete realizations of an unknown underlying functional. Further denote ${\mathbf{Y}}_i={\left({\mathbf{Y}}_{i,1}^{\top },\dots ,{\mathbf{Y}}_{i,q}^{\top }\right)}^{\top }$, $\mathbf{Y}={\left({\mathbf{Y}}_1^{\top },\dots ,{\mathbf{Y}}_N^{\top }\right)}^{\top }$ and $n={\sum }_{i=1}^N{\sum }_{j=1}^q{n}_{i,j}$. Under the biclustering analysis framework, assume that data can be “decomposed” into K_r sample (row) groups and K_c covariate (column) groups. Note that advancing from many existing approaches, the numbers of two dimensional groups are not pre-specified. Denote $t_{i,j,m}^{\prime }s\in \mathcal{T}=[0,1]$ as the observed time points. If (sample i, covariate j) belongs to the k_r th sample group and the k_c th covariate group, then

$$Y_{i,j,m}=g_{\left(k_r,k_c\right)}\left(t_{i,j,m}\right)+{ϵ}_{i,j,m},$$ (1)

where $g_{\left(k_r,k_c\right)}(t)$ is the unknown mean function, and ${ϵ}_{i,j,m}^{\prime }s$ are the random errors with mean zero.

For estimation, we adopt the basis expansion technique. Specifically, denote U_p(t) = (U_1,p(t), …, U_p,p(t))^⊤ as the collection of p rescaled basis functions. In the literature, there are extensive studies on choosing the form and number of basis functions [32], which will not be reiterated here. In our numerical study, we adopt B-spline basis functions of order d = 3. Let g_i,j(t) be the unknown mean function for the jth covariate of the ith sample, then we have

$$g_{i,j}(t)\approx U_p^{\top }(t){\mathit{\beta }}_{i,j},$$

where β_i,j = (β_i,j,1, …,β_i,j,p)^⊤ is the vector of unknown coefficients. Further denote ${\mathbf{U}}_{i,j}={\left(U_p\left(t_{i,j,1}\right),\dots ,U_p\left(t_{i,j,n_{i,j}}\right)\right)}^{\top }$. For estimation (without clustering), consider the objective function

$$Q(\mathit{\beta })=\frac{1}{2}\Vert \mathbf{Y}-\mathbf{U}\beta {\Vert }_2^2+\frac{1}{2}{\gamma }_1{\mathit{\beta }}^{\top }\mathbf{M}\beta =\frac{1}{2}\sum _{i=1}^N\sum _{j=1}^q\left({\Vert {\mathbf{Y}}_{i,j}-{\mathbf{U}}_{i,j}{\mathit{\beta }}_{i,j}\Vert }_2^2+{\gamma }_1{\mathit{\beta }}_{i,j}^{\top }\mathbf{D}{\mathit{\beta }}_{i,j}\right),$$ (2)

where U = diag(U_1,1, …, U_1,q, …, U_N,q), $\mathit{\beta }={\left({\mathit{\beta }}_{1,1}^{\top },\dots ,{\mathit{\beta }}_{1,q}^{\top },\dots ,{\mathit{\beta }}_{N,q}^{\top }\right)}^{\top }$, M = diag(D, …, D), D = δ^⊤ δ, δ is a (p − 2) × p matrix representing the second order differential operator, and γ₁ is a non-negative tuning parameter. In this objective function, the first term is the lack-of-fit, and the penalty term controls the smoothness of estimation.

2.2. Biclustering via penalized fusion

Under the clustering via penalized fusion framework, two samples (covariates) belong to the same cluster if and only if they have the same regression coefficients. As such, clustering amounts to determining whether two samples (covariates) have the same estimated coefficients. For samples i₁, i₂ ∈ {1, …, N}, denote ${\mathit{\beta }}_{i_1}^{(r)}$, ${\mathit{\beta }}_{i_2}^{(r)}$ as the length p × q vectors of coefficients. For covariates j₁, j₂ ∈ {1, …, q}, denote ${\mathit{\beta }}_{j_1}^{(c)}$, ${\mathit{\beta }}_{j_2}^{(c)}$ as the length p × N vectors of coefficients. For estimating β and hence determining the clustering structure, we propose minimizing the objective function:

$$L(\mathit{\beta })=Q(\mathit{\beta })+\sum _{1\le i_1<i_2\le N}{p}_{\tau }\left({\Vert {\mathit{\beta }}_{i_1}^{(r)}-{\mathit{\beta }}_{i_2}^{(r)}\Vert }_2,{\gamma }_2\right)+\sum _{1\le j_1<j_2\le q}{p}_{\tau }\left({\Vert {\mathit{\beta }}_{j_1}^{(c)}-{\mathit{\beta }}_{j_2}^{(c)}\Vert }_2,{(N/q)}^{1/2}{\gamma }_2\right).$$ (3)

Here p_τ (, ) is a penalty function, τ is a regularization parameter, ∥ · ∥₂ is the ℓ₂ norm, and γ₂ is a data-dependent tuning parameter. (N/q)^1/2 is added to make the two penalties comparable. In our numerical study, we adopt MCP [47], that is, $p_{\tau }(t,\gamma )=\gamma {\int }_0^t(1-x/(\tau \gamma ){)}_{+}dx$ with τ > 1. Here (x)₊ = x if x > 0, and (x)₊ = 0 otherwise. Note that SCAD [14] and some other penalties are also applicable. Denote the estimator as $\widehat{\mathit{\beta }}$. Let $\left\{{\widehat{\mathit{\alpha }}}_1^{(r)},\dots ,{\widehat{\mathit{\alpha }}}_{{\widehat{K}}_r}^{(r)}\right\}$ be the distinct values of ${\widehat{\mathit{\beta }}}_i^{(r)}’\text{s}$. Similarly, let $\left\{{\widehat{\mathit{\alpha }}}_1^{(c)},\dots ,{\widehat{\mathit{\alpha }}}_{{\widehat{K}}_c}^{(c)}\right\}$ be the distinct values of ${\widehat{\mathit{\beta }}}_j^{(c)}’\text{s}$. We can then obtain the block structure of $\widehat{\mathit{\beta }}$ by $\left\{{\widehat{\mathit{\alpha }}}_{1,1}^{(r,c)},\dots ,{\widehat{\mathit{\alpha }}}_{{\widehat{K}}_r,{\widehat{K}}_c}^{(r,c)}\right\}$, which are the distinct values of ${\widehat{\mathit{\beta }}}_{i,j}$, and set ${\widehat{K}}_b={\widehat{K}}_r\times {\widehat{K}}_c$.

In (3), penalty is imposed to the norms of all pairwise differences to promote equality, as in “standard” penalized fusion [8]. Here it is noted that, as in [8], since there is no information on the order of samples/covariates, all pairwise differences are taken, which differs from, for example, fused Lasso and other fused penalizations. Different from [8], as clustering needs to be conducted in both the sample and covariate dimensions, two fusion penalties are imposed, promoting equality in two directions. It is also noted that each specific coefficient shows up in three different penalties. As to be shown below, with properly chosen tunings, there is not an over penalization problem. In addition, it is not rare to have a parameter involved in two or more penalties [7].

The proposed approach involves two tunings, which have “ordinary” implications, with one controlling smoothness and the other determining the structure of clustering. One possibility is to conduct a two-dimensional grid search. Here we adopt the alternative proposed in [48], which has two steps and a lower computational cost. In particular, in the first step, we set γ₂ = 0 and select the optimal γ₁ by minimizing:

$$\mathrm{BIC}\left({\gamma }_1\right)=\sum _{i=1}^N\sum _{j=1}^q\left\{\log \left(\frac{{\Vert {\mathbf{Y}}_{i,j}-{\widehat{\mathit{g}}}_{i,j}\Vert }_2^2}{n_{i,j}}\right)+\frac{\log \left(n_{i,j}\right)}{n_{i,j}}{\mathrm{df}}_{i,j}\right\},$$

where ${\text{df}}_{i,j}=\mathrm{trace}\left\{{\mathbf{U}}_{i,j}{\left({\mathbf{U}}_{i,j}^{\top }{\mathbf{U}}_{i,j}+{\gamma }_1\mathbf{D}\right)}^{-1}{\mathbf{U}}_{i,j}^{\top }\right\}$ and ${\widehat{\mathit{g}}}_{i,j}={\left({\widehat{\text{g}}}_{i,j}\left(t_{i,j,1}\right),\dots ,{\widehat{g}}_{i,j}\left(t_{i,j,n_{i,j}}\right)\right)}^{\top }$ with ${\widehat{g}}_{i,j}(t)=U_p^{\top }(t){\widehat{\mathit{\beta }}}_{i,j}$.

In the second step, we fix the value of γ₁ at the optimal and select γ₂ by minimizing

$$\mathrm{BIC}\left({\gamma }_2\right)=\log \left(\frac{\Vert \mathbf{Y}-\widehat{\mathit{g}}{\Vert }_2^2}{Nq}\right)+\frac{\log (Nq)}{Nq}\text{df},$$

where $\text{df}=\left({\widehat{K}}_r{\widehat{K}}_c/Nq\right){\sum }_{i=1}^N{\sum }_{j=1}^q{\text{df}}_{i,j}$ and $\widehat{\mathit{g}}={\left({\widehat{\mathit{g}}}_{1,1}^{\top },\dots ,{\widehat{\mathit{g}}}_{N,q}^{\top }\right)}^{\top }$.

2.3. Computation

We develop an effective algorithm based on the ADMM technique. Specifically, we first reformulate (3) as

$$\arg \min Q(\mathit{\beta })+\sum _{\delta \in {\Delta }^{(r)}}{p}_{\tau }\left({\Vert {\mathit{\eta }}_{\delta }^{(r)}\Vert }_2,{\gamma }_2\right)+\sum _{\delta \in {\Delta }^{(c)}}{p}_{\tau }\left({\Vert {\mathit{\eta }}_{\delta }^{(c)}\Vert }_2,{(N/q)}^{1/2}{\gamma }_2\right),$$

$$\text{subject}\text{to}{\mathit{\beta }}_{i_1}^{(r)}-{\mathit{\beta }}_{i_2}^{(r)}={\eta }_{\delta }^{(r)},{\mathit{\beta }}_{j_1}^{(c)}-{\mathit{\beta }}_{j_2}^{(c)}={\eta }_{\delta }^{(c)},$$

where Δ^(r) = {δ = (i₁, i₂) : 1 ≤ i₁ < i₂ ≤ N} and Δ^(c) = {δ = (j₁, j₂) : 1 ≤ j₁ < j₂ ≤ q}. Optimizing the constrained objective function is equivalent to optimizing the augmented Lagrangian function:

$$\begin{array}{l}{L}_{\theta }\left(\mathit{\beta },{\mathbf{H}}_r,{\mathbf{H}}_c,{\mathit{\Lambda }}_r,{\mathit{\Lambda }}_c\right)=\frac{1}{2}\Vert \mathbf{Y}-\mathbf{U}\beta {\Vert }_2^2+\frac{1}{2}{\gamma }_1{\mathit{\beta }}^{\top }\mathbf{M}\beta +\sum _{\delta \in {\Delta }^{(r)}}{p}_{\tau }\left({\Vert {\mathit{\eta }}_{\delta }^{(r)}\Vert }_2,{\gamma }_2\right)+\sum _{\delta \in {\Delta }^{(r)}}{\lambda }_{\delta }^{(r)\top }\left({\mathit{\eta }}_{\delta }^{(r)}-{\mathit{\beta }}_{i_1}^{(r)}+{\mathit{\beta }}_{i_2}^{(r)}\right)\\ +\frac{\theta }{2}\sum _{\delta \in {\Delta }^{(r)}}{\Vert {\mathit{\eta }}_{\delta }^{(r)}-{\mathit{\beta }}_{i_1}^{(r)}+{\mathit{\beta }}_{i_2}^{(r)}\Vert }_2^2+\sum _{\delta \in {\Delta }^{(c)}}{p}_{\tau }\left({\Vert {\mathit{\eta }}_{\delta }^{(c)}\Vert }_2,{(N/q)}^{1/2}{\gamma }_2\right)\\ +\sum _{\delta \in {\Delta }^{(c)}}{\lambda }_{\delta }^{(c)\top }\left({\mathit{\eta }}_{\delta }^{(c)}-{\mathit{\beta }}_{j_1}^{(c)}+{\mathit{\beta }}_{j_2}^{(c)}\right)+\frac{\theta }{2}\sum _{\delta \in {\Delta }^{(c)}}{\Vert {\mathit{\eta }}_{\delta }^{(c)}-{\mathit{\beta }}_{j_1}^{(c)}+{\mathit{\beta }}_{j_2}^{(c)}\Vert }_2^2,\end{array}$$ (4)

where θ is a small positive constant, ${\mathbf{H}}_r=\left({\mathit{\eta }}_{(1,2)}^{(r)},\dots ,{\mathit{\eta }}_{(N-1,N)}^{(r)}\right)$, ${\mathbf{H}}_c=\left({\mathit{\eta }}_{(1,2)}^{(c)},\dots ,{\mathit{\eta }}_{(q-1,q)}^{(c)}\right)$, ${\mathit{\Lambda }}_r=\left({\mathit{\lambda }}_{(1,2)}^{(r)},\dots ,{\mathit{\lambda }}_{(N-1,N)}^{(r)}\right)$, and ${\mathit{\Lambda }}_c=\left({\mathit{\lambda }}_{(1,2)}^{(c)},\dots ,{\mathit{\lambda }}_{(q-1,q)}^{(c)}\right)$. Here we introduce the dual variables ${\mathit{\lambda }}_{\delta }^{(r)}$ and ${\mathit{\lambda }}_{\delta }^{(c)}$ corresponding to the pair δ in Δ^(r) and Δ^(c), and the cardinality of Δ^(r) and Δ^(c) are denoted by |Δ^(r)| and |Δ^(c)|.

We consider an iterative algorithm, where the updates in step m + 1 are:

$$\begin{array}{l}{\mathit{\beta }}^{(m+1)}=\underset{\mathit{\beta }}{\arg \min }{L}_{\theta }\left(\mathit{\beta },{\mathbf{H}}_r^{(m)},{\mathbf{H}}_c^{(m)},{\mathit{\Lambda }}_r^{(m)},{\mathit{\Lambda }}_c^{(m)}\right),{\mathbf{H}}_r^{(m+1)}=\underset{{\mathbf{H}}_r}{\arg \min }{L}_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_r,{\mathit{\Lambda }}_r^{(m)}\right),\\ {\mathbf{H}}_c^{(m+1)}=\underset{{\mathbf{H}}_c}{\arg \min }{L}_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_c,{\mathit{\Lambda }}_c^{(m)}\right),{\mathit{\lambda }}_{\delta }^{(r)(m+1)}={\mathit{\lambda }}_{\delta }^{(r)(m)}+\theta \left({\mathit{\eta }}_{\delta }^{(r)(m+1)}-{\mathit{\beta }}_{i_1}^{(r)(m+1)}+{\mathit{\beta }}_{i_2}^{(r)(m+1)}\right),\delta \in {\Delta }^{(r)},\\ {\mathit{\lambda }}_{\delta }^{(c)(m+1)}={\mathit{\lambda }}_{\delta }^{(c)(m)}+\theta \left({\mathit{\eta }}_{\delta }^{(c)(m+1)}-{\mathit{\beta }}_{j_1}^{(c)(m+1)}+{\mathit{\beta }}_{j_2}^{(c)(m+1)}\right),\delta \in {\Delta }^{(c)}.\end{array}$$ (5)

More specifically, when optimizing over β, we consider

$$f(\mathit{\beta })=\frac{1}{2}\Vert \mathbf{Y}-\mathbf{U}\beta {\Vert }_2^2+\frac{1}{2}{\gamma }_1{\mathit{\beta }}^{\top }\mathbf{M}\beta +\frac{\theta }{2}\left(\sum _{\delta \in {\Delta }^{(r)}}{\Vert {\tilde{\mathit{\eta }}}_{\delta }^{(r)(m)}-{\mathbf{B}}_{\delta }^{(r)}\mathit{\beta }\Vert }_2^2+\sum _{\delta \in {\Delta }^{(c)}}{\Vert {\tilde{\mathit{\eta }}}_{\delta }^{(c)(m)}-{\mathbf{B}}_{\delta }^{(c)}\mathit{\beta }\Vert }_2^2\right),$$ (6)

where ${\tilde{\mathit{\eta }}}_{\delta }^{(r)}={\mathit{\eta }}_{\delta }^{(r)}+\frac{1}{\theta }{\mathit{\lambda }}_{\delta }^{(r)}$, ${\tilde{\mathit{\eta }}}_{\delta }^{(c)}={\mathit{\eta }}_{\delta }^{(c)}+\frac{1}{\theta }{\mathit{\lambda }}_{\delta }^{(c)}$, ${\mathbf{B}}_{\delta }^{(r)}={\left({\mathit{e}}_{i_1}^{(r)}-{\mathit{e}}_{i_2}^{(r)}\right)}^{\top }\otimes {\mathbf{I}}_{qp}$, ${\mathbf{B}}_{\delta }^{(c)}={\mathbf{I}}_N\otimes \left[{\left({\mathit{e}}_{j_1}^{(c)}-{\mathit{e}}_{j_2}^{(c)}\right)}^{\top }\otimes {\mathbf{I}}_p\right]$, ${\mathit{e}}_i^{(r)}$ is an N × 1 zero vector except that its ith element is 1, ${\mathit{e}}_i^{(c)}$ is a q × 1 zero vector except that its jth element is 1, ⊗ is the Kronecker product, and Ip is a p × p identity matrix. Denote ${\mathbf{B}}_r={\left({\mathbf{B}}_{(1,2)}^{(r)\top },\dots ,{\mathbf{B}}_{(N-1,N)}^{(r)\top }\right)}^{\top }$, ${\mathbf{B}}_c={\left({\mathbf{B}}_{(1,2)}^{(c)\top },\dots ,{\mathbf{B}}_{(q-1,q)}^{(c)\top }\right)}^{\top }$, ${\tilde{\mathbf{H}}}_r=\left({\tilde{\mathit{\eta }}}_{(1,2)}^{(r)},\dots ,{\tilde{\mathit{\eta }}}_{(N-1,N)}^{(r)}\right)$, and ${\tilde{\mathbf{H}}}_c=\left({\tilde{\mathit{\eta }}}_{(1,2)}^{(c)},\dots ,{\tilde{\mathit{\eta }}}_{(q-1,q)}^{(c)}\right)$. Then the update for β is

$${\mathit{\beta }}^{(m+1)}={\left({\mathbf{U}}^{\top }\mathbf{U}+{\gamma }_1\mathbf{M}+\theta {\mathbf{B}}_r^{\top }{\mathbf{B}}_r+\theta {\mathbf{B}}_c^{\top }{\mathbf{B}}_c\right)}^{-1}\left({\mathbf{U}}^{\top }\mathbf{Y}+\theta {\mathbf{B}}_r^{\top }\mathrm{vec}\left({\tilde{\mathbf{H}}}_r^{(m)}\right)+\theta {\mathbf{B}}_c^{\top }\mathrm{vec}\left({\tilde{\mathbf{H}}}_c^{(m)}\right)\right),$$ (7)

where vec(Z) is the vectorization of matrix Z by columns.

For H_r, we consider

$$f\left({\mathit{\eta }}_{\delta }^{(r)}\right)=p_{\tau }\left({\Vert {\mathit{\eta }}_{\delta }^{(r)}\Vert }_2,{\gamma }_2\right)+\frac{\theta }{2}{\Vert {\mathit{\eta }}_{\delta }^{(r)}-{\mathit{\beta }}_{i_1}^{(r)(m+1)}+{\mathit{\beta }}_{i_2}^{(r)(m+1)}+{\mathit{\lambda }}_{\delta }^{(r)(m)}/\theta \Vert }_2^2.$$ (8)

Denote ${\mathit{z}}_{\delta }^{(r)(m+1)}={\mathit{\beta }}_{i_1}^{(r)(m+1)}-{\mathit{\beta }}_{i_2}^{(r)(m+1)}-{\mathit{\lambda }}_{\delta }^{(r)(m)}/\theta$. With the KKT conditions of (8), we can get a closed form solution of H_r:

$${\mathit{\eta }}_{\delta }^{(r)(m+1)}=\{\begin{array}{ll}{\mathit{z}}_{\delta }^{(r)(m+1)},\hfill & \text{if}{\Vert {\mathit{z}}_{\delta }^{(r)(m+1)}\Vert }_2\ge \tau {\gamma }_2,\hfill \\ \frac{\tau \theta }{\tau \theta -1}{\left(1-\frac{{\gamma }_2/\theta }{{\Vert {\mathit{z}}_{\delta }^{(r)(m+1)}\Vert }_2}\right)}_{+}{\mathit{z}}_{\delta }^{(r)(m+1)},\hfill & \text{if}{\Vert {\mathit{z}}_{\delta }^{(r)(m+1)}\Vert }_2<\tau {\gamma }_2.\hfill \end{array}$$ (9)

Similarly, denote ${\mathit{z}}_{\delta }^{(c)(m+1)}={\mathit{\beta }}_{j_1}^{(c)(m+1)}-{\mathit{\beta }}_{j_2}^{(c)(m+1)}-{\mathit{\lambda }}_{\delta }^{(c)(m)}/\theta$, and we can get a closed form solution of H_c:

$${\mathit{\eta }}_{\delta }^{(c)(m+1)}=\{\begin{array}{ll}{\mathit{z}}_{\delta }^{(c)(m+1)},\hfill & \text{if}{\Vert {\mathit{z}}_{\delta }^{(c)(m+1)}\Vert }_2\ge {(N/q)}^{1/2}\tau {\gamma }_2,\hfill \\ \frac{\tau \theta }{\tau \theta -1}{\left(1-\frac{{(N/q)}^{1/2}{\gamma }_2/\theta }{{\Vert {\mathit{z}}_{\delta }^{(c)(m+1)}\Vert }_2}\right)}_{+}{\mathit{z}}_{\delta }^{(c)(m+1)},\hfill & \text{if}{\Vert {\mathit{z}}_{\delta }^{(c)(m+1)}\Vert }_2<{(N/q)}^{1/2}\tau {\gamma }_2.\hfill \end{array}$$ (10)

Consider the initial values ${\mathit{\beta }}^{(0)}={\left({\mathbf{U}}^{\top }\mathbf{U}+{\gamma }_1\mathbf{M}\right)}^{-1}{\mathbf{U}}^{\top }\mathbf{Y}$,${\mathit{\eta }}_{\delta }^{(r)(0)}={\mathit{\beta }}_{i_1}^{(r)(0)}-{\mathit{\beta }}_{i_2}^{(r)(0)}$, and ${\mathit{\eta }}_{\delta }^{(c)(0)}={\mathit{\beta }}_{j_1}^{(c)(0)}-{\mathit{\beta }}_{j_2}^{(c)(0)}$, and ${\mathit{\Lambda }}_r^{(0)}$ and ${\mathit{\Lambda }}_c^{(0)}$ are set as zero. The ADMM based algorithm is summarized in Algorithm 1.

Algorithm 1.

Input:

Response vector Y, basis expansion design matrix U, and difference matrix M;

Tuning parameters γ1 and γ2. Specific to MCP, regularization parameter τ;

Output:

Coefficient vector β, splitting variables Hr and Hc, and dual variables Λr and Λc;

1: repeat

2: for m = 0, 1, 2 … do

3: Update β by (7).

4: Update Hr by (9).

5: Update Hc by (10).

6: Update Λr and Λc by (5).

7: end for

8: until the stopping criteria are met, which are set as ${\Vert {\mathit{r}}_r^{(m+1)}\Vert }_2\le {ϵ}_1^{pri}$, ${\Vert {\mathit{r}}_c^{(m+1)}\Vert }_2\le {ϵ}_2^{pri}$, ${\Vert {\mathit{s}}_r^{(m+1)}\Vert }_2\le {ϵ}_1^{dual}$, and ${\Vert {\mathit{s}}_c^{(m+1)}\Vert }_2\le {ϵ}_2^{dual}$ in our numerical study.

Proposition 1.

Denote the two primal residuals as ${\mathbf{r}}_r^{(m+1)}={\mathbf{B}}_r{\mathit{\beta }}^{(m+1)}-vec\left({\mathbf{H}}_r^{(m+1)}\right)$ and ${\mathit{r}}_c^{(m+1)}={\mathbf{B}}_c{\mathit{\beta }}^{(m+1)}-vec\left({\mathbf{H}}_c^{(m+1)}\right)$, and the two dual residuals as ${\mathit{s}}_r^{(m+1)}=\theta {\mathbf{B}}_r^{\top }\left[vec\left({\mathbf{H}}_r^{(m+1)}\right)-vec\left({\mathbf{H}}_r^{(m)}\right)\right]$ and ${\mathit{s}}_c^{(m+1)}=\theta {\mathbf{B}}_c^{\top }\left[vec\left({\mathbf{H}}_c^{(m+1)}\right)-vec\left({\mathbf{H}}_c^{(m)}\right)\right]$. Then

$$\underset{m\to \infty }{\lim }{\Vert {\mathit{r}}_r^{(m+1)}\Vert }_2^2=0,\underset{m\to \infty }{\lim }{\Vert {\mathit{r}}_c^{(m+1)}\Vert }_2^2=0,\underset{m\to \infty }{\lim }{\Vert {\mathit{s}}_r^{(m+1)}+{\mathit{s}}_c^{(m+1)}\Vert }_2^2=0.$$

This result establishes convergence of the proposed algorithm. In numerical analysis, we stop the algorithm and conclude convergence when ${\Vert {\mathit{r}}_r^{(m+1)}\Vert }_2\le {ϵ}_1^{pri}$, ${\Vert {\mathit{r}}_C^{(m+1)}\Vert }_2\le {ϵ}_2^{pri}$, ${\Vert {\mathit{s}}_r^{(m+1)}\Vert }_2\le {ϵ}_1^{dual}$ and ${\Vert {\mathit{s}}_c^{(m+1)}\Vert }_2\le {ϵ}_2^{dual}$. Following [5], we set the tolerance parameters as follows:

$$\begin{array}{l}{ϵ}_1^{pri}=\sqrt{\left|{\Delta }^{(r)}\right|pq}{ϵ}^{abs}+{ϵ}^{rel}\max \left\{{\Vert {\mathbf{B}}_r{\mathit{\beta }}^{(m+1)}\Vert }_2,{\Vert \mathrm{vec}\left({\mathbf{H}}_r^{(m+1)}\right)\Vert }_2\right\},\\ {ϵ}_2^{pri}=\sqrt{\left|{\Delta }^{(c)}\right|pN}{ϵ}^{abs}+{ϵ}^{rel}\max \left\{{\Vert {\mathbf{B}}_c{\mathit{\beta }}^{(m+1)}\Vert }_2,{\Vert \mathrm{vec}\left({\mathbf{H}}_c^{(m+1)}\right)\Vert }_2\right\},\\ {ϵ}_1^{dual}=\sqrt{Nqp}{ϵ}^{abs}+{ϵ}^{rel}{\Vert {\mathbf{B}}_r^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m+1)}\right)\Vert }_2,{ϵ}_2^{dual}=\sqrt{Nqp}{ϵ}^{abs}+{ϵ}^{rel}{\Vert {\mathbf{B}}_c^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m+1)}\right)\Vert }_2.\end{array}$$ (11)

Here ϵ^abs and ϵ^rel are predetermined small values, for example 10⁻³. In all of our numerical analysis, convergence is satisfactorily achieved within a small to moderate number of iterations. The code and example are publicly available at https://github.com/ruiqwy/Biclustering.

2.4. Statistical properties

For a vector $\mathit{z}={\left(z_1,\dots ,z_s\right)}^{\top }\in {ℝ}^s$, let $\Vert \mathit{z}{\Vert }_{\infty }={\max }_{1\le l\le s}\left|z_l\right|$. For a matrix Z_s×h, let $\Vert \mathbf{Z}{\Vert }_2={\max }_{\mathit{v}\in {ℝ}^h,\Vert \mathit{v}{\Vert }_2=1}\Vert \mathbf{Z}v{\Vert }_2$ and $\Vert \mathbf{Z}{\Vert }_{\infty }={\max }_{1\le i\le s}{\sum }_{j=1}^h\left|Z_{i,j}\right|$. For any two sequences of real numbers {a_n} ≥ 1 and {b_n} ≥ 1, denote b_n ≪ a_n if b_n/a_n = o(1). Let r be a positive integer, v ∈ (0, 1], and κ = r + v > 1.5. Let $\mathcal{H}$ be the collection of functions g on $\mathcal{T}=[0,1]$, where the rth derivative g^(r) exists and satisfies the Lipschitz condition with order v:

$$\left|g^{(r)}\left(z_1\right)-g^{(r)}\left(z_2\right)\right|\le C{\left|z_1-z_2\right|}^v,0\le z_1,z_2\le 1,$$

and C is a positive constant.

Define the following collections of index sets for clustering memberships: ${\mathcal{G}}^{(r)}=\left({\mathcal{G}}_1^{(r)},\dots ,{\mathcal{G}}_{K_r}^{(r)}\right)$ for samples, ${\mathcal{G}}^{(c)}=\left({\mathcal{G}}_1^{(c)},\dots ,{\mathcal{G}}_{K_c}^{(c)}\right)$ for covariates, and ${\mathcal{G}}^{(r,c)}=\left({\mathcal{G}}_{1,1}^{(r,c)},\dots ,{\mathcal{G}}_{k_r,k_c}^{(r,c)},\dots ,{\mathcal{G}}_{K_r,K_c}^{(r,c)}\right)$ for both samples and covariates. Define ${\mathcal{M}}_G=\{\mathit{\beta }\in {ℝ}^{Nqp}:{\mathit{\beta }}_{i_1,j_1}={\mathit{\beta }}_{i_2,j_2},\text{for}\text{any}\left(i_1,j_1\right),\left(i_2,j_2\right)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)},1\le k_r\le K_r,1\le k_c\le K_c\}$. Let $\left|{\mathcal{G}}_{k_r}^{(r)}\right|$, $\left|{\mathcal{G}}_{k_c}^{(c)}\right|$, and $\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|$ be the sizes of ${\mathcal{G}}_{k_r}^{(r)}$, ${\mathcal{G}}_{k_c}^{(c)}$, and ${\mathcal{G}}_{k_r,k_c}^{(r,c)}$, respectively. Further define $\left|{\mathcal{G}}_{min}^{(r)}\right|={\min }_{1\le k_r\le K_r}\left|{\mathcal{G}}_{k_r}^{(r)}\right|$, $\left|{\mathcal{G}}_{min}^{(c)}\right|={\min }_{1\le k_c\le K_c}\left|{\mathcal{G}}_{k_c}^{(c)}\right|$ and $\left|{\mathcal{G}}_{min}^{(r,c)}\right|=\left|{\mathcal{G}}_{min}^{(r)}\right|\times \left|{\mathcal{G}}_{min}^{(c)}\right|\cdot \left|{\mathcal{G}}_{max}^{(r,c)}\right|$ can be defined accordingly. Let ρ(t) = γ⁻¹p_τ(t, γ). Assume the following conditions.

(C1) $g_{k_r,k_c}\in \mathcal{H}$ for all k_r ∈ {1, …, K_r}, k_c ∈ {1, …, K_c}, and ${\left|{\mathcal{G}}_{max}^{(r,c)}\right|}^{1/(2\kappa )}\ll p\ll {\left|{\mathcal{G}}_{min}^{(r,c)}\right|}^{1/3}$.

(C2) The distribution of t_i,j,m’s, i ∈ {1, …, N}, j ∈ {1, …, q}, m ∈ {1, …, n_i,j} follows a density function f_T, which is absolutely continuous. There exist constants c₁ and C₁ such that $0<c_1\le {\min }_{t\in \mathcal{T}}{f}_T(t)\le {\max }_{t\in \mathcal{T}}{f}_T(t)\le C_1<\infty$.

(C3) n_i,j’s are uniformly bounded for all i ∈ {1, …, N}, j ∈ {1, …, q}.

(C4) p_τ (t, γ) is symmetric, non-decreasing, and concave in t for t ∈ [0, ∞]. There exists a constant 0 < a < ∞ such that ρ(t) is a constant for all t ≥ aγ, and ρ(0) = 0. ρ^′(t) exists and is continuous except for a finite number of t and ρ^′(0+) = 1.

(C5) Let ${ϵ}_{i,j}={\left({ϵ}_{i,j,1},\dots ,{ϵ}_{i,j,n_{i,j}}\right)}^{\top }$, where ϵ_i,j,m’s are independent across (i, j) (among different individual observational vectors) and correlated across m (within the same (i, j)). Furthermore, there exist F > 0 and c₂ > 0, such that for all i ∈ {1, …, N} and j ∈ {1, …, q},

$$\text{E}\left(\exp \left\{F{\left|n_{i,j}^{-1}{ϵ}_{i,j}^{\top }{ϵ}_{i,j}\right|}^{1/2}\right\}\right)\le c_2.$$

Similar conditions have been assumed in the literature. The first condition in (C1) ensures that the Hölder’s condition is satisfied [36]. The second condition in (C1) pertains to the growth rate of the number of internal knots, in a way similar to [25] and [24]. Condition (C2) assumes the boundedness of the density function, similarly to [48] and others. Conditions similar to (C3) have been commonly made. In the analysis of high-dimensional data, conditions similar to (C4) have been common, and it is easy to verify that MCP and SCAD satisfy (C4). Condition (C5) gives the boundedness condition for the error terms, and a similar condition can be found in [11].

When the true clustering structure is known, the oracle estimator for β can be defined as

$${\widehat{\mathit{\beta }}}^{or}=\underset{\mathit{\beta }\in {\mathcal{M}}_G}{\arg \min }\frac{1}{2}\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}}\left\{{\Vert {\mathbf{Y}}_{i,j}-{\mathbf{U}}_{i,j}{\mathit{\beta }}_{i,j}\Vert }_2^2+{\gamma }_1{\mathit{\beta }}_{i,j}^{\top }\mathbf{D}{\mathit{\beta }}_{i,j}\right\},$$

where ${\widehat{\mathit{g}}}_{\left(k_r,k_c\right)}^{or}$ is defined as the oracle estimator of $g_{\left(k_r,k_c\right)}$ based on ${\widehat{\mathit{\beta }}}^{or}$. Let β* be the underlying true coefficient vector and $g_{\left(k_r,k_c\right)}^{*}$ be the true value of $g_{\left(k_r,k_c\right)}$. For any L²-integrable function g, denote $\Vert g\Vert ={\left({\int }_{t\in \mathcal{T}}{g}^2(t)f_T(t)dt\right)}^{1/2}$.

Theorem 1.

Assume that (C1)–(C5) hold. If ${\gamma }_1=o\left({\left|{\mathcal{G}}_{min}^{(r,c)}\right|}^{-1/2}\right)$ and $p\log (Nq)\ll \left|{\mathcal{G}}_{min}^{(r,c)}\right|$, then with probability at least 1 − 3K_rK_cp/(Nq),

$$\underset{1\le i\le N,1\le j\le q}{\sup }{\Vert {\widehat{\mathit{\beta }}}_{i,j}^{or}-{\mathit{\beta }}_{i,j}^{*}\Vert }_2\le \psi ,\underset{1\le k_r\le K_r,1\le k_c\le K_c}{\sup }\Vert {\widehat{g}}_{\left(k_r,k_c\right)}^{0r}-g_{\left(k_r,k_c\right)}^{*}\Vert \le \psi ,$$

where $\psi =C^{*}{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}$, and C* is a large constant.

This theorem establishes consistency of the oracle estimates with a high probability. Denote $b={\min }_{\left(k_r,k_c\right)\ne \left(k_r^{\prime },k_c^{\prime }\right)}\Vert g_{\left(k_r,k_c\right)}^{*}-g_{\left(k_r^{\prime },k_c^{\prime }\right)}^{*}\Vert$. We can further establish the following result.

Theorem 2.

Assume that (C1)–(C5) and conditions in Theorem 1 hold. If $b\gg {\gamma }_2{\left|{\mathcal{G}}_{min}^{(c)}\right|}^{-1/2}$, $b\gg {(N/q)}^{1/2}{\gamma }_2{\left|{\mathcal{G}}_{min}^{(r)}\right|}^{-1/2}$, and ${\gamma }_2\gg {(pq)}^{1/2}\log (Nq)/\min \left\{\left|{\mathcal{G}}_{min}^{(r)}\right|,\left|{\mathcal{G}}_{min}^{(c)}\right|\right\}$, then there exists a local minimizer $\widehat{\mathit{\beta }}$ of L(β) satisfying

$$P\left(\widehat{\mathit{\beta }}={\widehat{\mathit{\beta }}}^{or}\right)\to 1asN,q\to \infty .$$

This theorem establishes that the oracle estimator is a local minimizer of the objective function with a high probability. The estimation consistency along with the separateness of the true functions can lead to the clustering consistency.

3. Simulation

We conduct simulation to assess performance of the proposed approach and gauge against the following alternatives: (a) the bKmeans method [1], which first fits each curve using B-splines and then clusters the estimated coefficients using the k-means technique by rows and columns, (b) the funHDDC method [33], which has been developed for multivariate functional data clustering based on latent mixture models. It has been applied to longitudinal data, and (c) the funLBM method [4], which has been developed for functional data biclustering based on latent block models. Here we note that the proposed and funLBM methods conduct biclustering directly, whereas the bKmeans and funHDDC methods have been originally designed for one-way clustering–hence they are applied twice to achieve both row and column clusterings. In addition, the funHDDC and funLBM methods are not directly applicable to functional data with unequal measurements. We apply imputation [26] to tackle this problem. As discussed in Section 1, biclustering methods for functional data are very limited. It is possible to modify other existing one-way functional clustering methods to achieve biclustering, however, this demands additional methodological developments. The three alternatives considered here have been chosen because of their closely related frameworks and competitive performance.

In evaluation, we examine both clustering and estimation accuracy. Specifically, when examining clustering accuracy, we consider the estimated numbers of row clusters ${\widehat{K}}_r$, column clusters ${\widehat{K}}_c$, and biclusters ${\widehat{K}}_b$. In addition, we use the Rand index and adjusted Rand index to assess the accuracy of clustering, including RI_r and ARI_r for row clustering, RI_c and ARI_c for column clustering, and RI_b and ARI_b for biclustering. The Rand index is defined by RI = (TP + TN)/(TP + FP + FN + TN), where for example TP is the true positive count, defined as the number of sample pairs from the same cluster and assigned to the same cluster, and the other counts can be defined accordingly. As the Rand index tends to be large even under random clusterings, we also examine the adjusted Rand index defined as ARI = (RI − E(RI))/(max(RI) − E(RI)), which can partly correct this problem. To evaluate estimation accuracy, we examine the integrated squared error (ISE) defined as

$$\text{ISE}=\frac{1}{n}\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}}\sum _{m=1}^{n_{ij}}{\left\{g_{\left(k_r,k_c\right)}\left(t_{i,j,m}\right)-{\widehat{g}}_{i,j}\left(t_{i,j,m}\right)\right\}}^2.$$

We consider a total of K_b = 9 biclusters, which are formed by K_r = 3 sample (row) clusters and K_c = 3 covariate (column) clusters. $Y_{i,j,m}=g_{\left(k_r,k_c\right)}\left(t_{i,j,m}\right)+{ϵ}_{i,j,m}$ with t_i,j,m’s, m ∈ {1, …, 10}, equally spaced on [0, 1]. The nine true functional forms are g_(1,1)(t) = cos(2πt), g_(2,1)(t) = 1 − 2exp(−6t), g_(3,1)(t) = −1.5t, g_(1,2)(t) = 1 + sin(2πt), g_(2,2)(t) = 2t², g_(3,2)(t) = t + 1, g_(1,3)(t) = 2(sin(2πt) + cos(2πt)), g_(2,3)(t) = 1 + t³, and $g_{(3,3)}(t)=2\sqrt{t}+1$. They are also graphically presented in Fig. 1. To better mimic real data, we allow a certain proportion (ζ) of the curves from each bicluster to have 20% missing measurements. When implementing the proposed approach, we choose smoothing splines with the number of internal knots J = 3. We also fix θ = 1 and τ = 3. In what follows, under Examples 1 and 2, N > q, whereas under Example 3, N = q. Under Examples 1–3, the random errors are independent, whereas under Example 4, they are correlated. Note that under Examples 1–4, simulation results are calculated based on automatic cluster selection. Example 5 is designed to investigate the performance of these methods when the numbers of clusters are correctly prespecified. A total of 100 replicates are simulated under each setting.

Fig. 1.

Example 1: Curves of observed data (black dotted), estimated (blue solid) by the proposed method, and true (red solid) functions with (a) N = 30 and (b) N = 90 for one replicate. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Example 1.

N = 30, 60, and 90. q = 9. The clusters are balanced, with each row cluster containing N/3 samples and each column cluster containing q/3 covariates. ζ = 0.3. The random errors are iid $\mathcal{N}\left(0,{0.6}^2\right)$.

Example 2.

The settings are the same as in Example 1, except that the clusters are unbalanced. The row clusters have sizes 1:2:3, and the column clusters have sizes 2:3:4.

Example 3.

Set (N, q) = (30, 30), (39, 39), (45, 45), ζ = 0.3 and 0.4. The rest are the same as in Example 1.

Example 4.

The settings are similar to those under Example 1. The random errors are correlated with an AR(1) correlation structure, where AR stands for auto-correlation. Consider AR coefficient ϕ = 0.2 and 0.8, representing weak and strong correlations.

Example 5.

The settings are the same as those in Example 1. The difference is that the numbers of clusters are correctly prespecified instead of being selected by the BIC criterion.

Results for Example 1 are presented in Figs. 1 and 2 as well as Tables 1 and 2. More specifically, in Fig. 1, we show the true functions for all clusters as well as sample observed data and estimated functions. In Table 1, we summarize the numbers of identified row and column clusters as well as biclusters. In Table 2, we summarize the Rand and adjusted Rand index values. In Fig. 2, we present the boxplots of ISE (note that different panels have different ranges for the Y-axis). Results for Examples 2–5 are presented in the Supplementary section. Although different examples have different numerical results, overall, the advantage of the proposed approach is clearly observed. Consider for example Table 1 with N = 30. The proposed approach has the mean number of row clusters 2.83, compared to 2.76, 2.63, and 4.66 of the three alternatives. When N = 90, the proposed approach has the mean number of biclusters 8.71, compared to 3.51, 6.33, and 10.68 of the three alternatives. The improved clustering accuracy is further proved by the Rand index values in Table 2. For example with N = 90, the adjusted Rand index value for biclustering with the proposed approach is 0.964, compared to 0.358, 0.686, and 0.764 with the three alternatives. Fig. 2 shows that as N increases, estimation accuracy of the proposed approach (and two alternatives) increases. Under all three N values, the proposed approach has significantly smaller ISE values. Moreover, comparing the results of Example 5 with Example 1, we observe similar performance and that the proposed approach still performs better when the numbers of clusters are correctly prespecified.

Fig. 2.

Example 1: Boxplots of ISE with (a) the proposed method, (b) bKmeans, (c) funHDDC, and (d) funLBM.

Table 2.

Example 1: Mean and standard error (shown in parentheses) of RI_r. ARI_r. RI_c. ARI_c. RI_b. and ARI_b based on 100 replicates.

N	Method	RIr	ARIr	RIc	ARIc	RIb	ARIb
30	Proposed	0.940 (0.189)	0.911 (0.278)	0.936 (0.203)	0.910 (0.279)	0.927 (0.238)	0.909 (0.281)
	bKmeans	0.860 (0.173)	0.740 (0.290)	0.296 (0.163)	0.052 (0.194)	0.673 (0.174)	0.307 (0.167)
	funHDDC	0.744 (0.031)	0.493 (0.074)	0.940 (0.107)	0.880 (0.215)	0.889 (0.051)	0.598 (0.120)
	funLBM	0.913 (0.053)	0.786 (0.109)	0.913 (0.064)	0.746 (0.153)	0.951 (0.029)	0.708 (0.113)
60	Proposed	0.966 (0.138)	0.947 (0.208)	0.963 (0.152)	0.945 (0.212)	0.959 (0.177)	0.943 (0.216)
	bKmeans	0.887 (0.132)	0.780 (0.248)	0.316 (0.195)	0.077 (0.239)	0.704 (0.142)	0.339 (0.191)
	funHDDC	0.767 (0.021)	0.546 (0.049)	0.998 (0.025)	0.995 (0.050)	0.922 (0.014)	0.692 (0.044)
	funLBM	0.918 (0.110)	0.828 (0.221)	0.929 (0.119)	0.840 (0.257)	0.953 (0.052)	0.796 (0.198)
90	Proposed	0.978 (0.117)	0.966 (0.176)	0.975 (0.131)	0.965 (0.178)	0.971 (0.154)	0.964 (0.180)
	bKmeans	0.886 (0.134)	0.778 (0.251)	0.342 (0.226)	0.109 (0.279)	0.709 (0.152)	0.358 (0.227)
	funHDDC	0.769 (0.017)	0.551 (0.040)	0.990 (0.049)	0.980 (0.098)	0.919 (0.025)	0.686 (0.061)
	funLBM	0.909 (0.121)	0.813 (0.241)	0.908 (0.130)	0.793 (0.276)	0.944 (0.056)	0.764 (0.210)

4. Applications

Here we analyze two time-course gene expression data. Although in a sense the data characteristics are similar, the two data analyses may serve different purposes. In particular, the first dataset is “older”, which has been analyzed multiple times in the literature, and has a clearer sample clustering structure. In contrast, the second dataset is more recent, and its analysis may lead to a higher practical impact.

4.1. T-cell data

This data has been generated in a study of T-cell activation [31]. It is publicly available in the R package longitudinal (http://www.strimmerlab.org/software/longitudinal/) and contains two subsets: tcell.10 and tcell.34. The first subset contains measurements for 10 samples and 58 genes at 10 unequally spaced time points, t ∈ {0, 2, 4, 6, 8, 18, 24, 32, 48, 72}, whereas the second subset contains measurements for 34 samples and the same genes at the same time points. In [31], the distinctions between the two subsets have been noted, and they have been combined for analysis. Prior to analysis, we conduct data processing, including gene expression normalization using the method developed in [29] and linearly transforming the observed times to [0, 1], and set the knots at 0.06, 0.2, and 0.4 as well as the order as 3.

The proposed approach identifies two sample clusters, with sizes 10 and 34, which exactly match the original subset structure. The distinctions of the samples in the two subsets have been noted in [31]. As such, they are supposed to belong to different clusters. In this sense, our “finding”, although as expected, is re-assuring. In addition, eight gene clusters are identified, among which there are four trivial clusters with sizes one. The four non-trivial clusters have sizes 27, 18, 5, and 4. Detailed information on the gene clusters is available from the authors. The eight non-trivial biclusters are presented in Fig. 3. Biclusters 1–4 correspond to tcell.10, and the rest correspond to tcell.34. It is observed that the estimated functions clearly differ across biclusters. The observed temporal trends are highly similar to those reported in [28], which provides support to the validity of our approach.

Fig. 3.

Analysis of T-cell data: Curves of observed data (black dotted) and estimated functions (blue solid) for the eight non-trivial bicluster, as well as yellow points indicating the estimated values at t ∈ {0, 2, 4, 6, 8, 18, 24, 32, 48, 72} by the proposed method. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

The three alternatives are also applied. The bKmeans approach identifies three sample clusters (with sizes 10, 17, and 17) and four gene clusters (with sizes 9, 15, 19, and 15). Compared to the proposed approach, the adjusted Rand index values are 0.441 (sample), 0.619 (gene), and 0.430 (bicluster). The funHDDC approach identifies two sample clusters (with sizes 10 and 34) and two gene clusters (with sizes 9 and 49). Compared to the proposed approach, the adjusted Rand index values are 1.000 (sample), 0.286 (gene), and 0.452 (bicluster). The funLBM approach identifies two sample clusters (with sizes 10 and 34) and six gene clusters (with sizes 9, 4, 12, 5, 18, and 10). Compared to the proposed approach, the adjusted Rand index values are 1.000 (sample), 0.586 (gene), and 0.646 (bicluster). Unlike for the simulated data, it is difficult to objectively evaluate the accuracy of clustering. However, for the proposed approach, the matching with the original sample distinction and published findings can provide a strong support, which is not shared by the alternatives.

4.2. Vaccine data

This data is generated in a relative recent study [43] and available at GEO with identifier GSE124533. The study settings have been described in detail in [43]. Briefly, it concerns with the time course of whole blood gene expressions, and the samples are healthy adults residing in an inpatient unit. The samples have been randomized into three protocols (305 A, 305B and 305C). Within each protocol, samples have been randomized to receive immunization via either vaccine or saline placebo. The treatments have been referred to as YFV and VZV (under 305 A), HBV1 and HBV3 (under 305B), and TIV and ATIV (under 305C). In this experiment, gene expression levels are measured at t ∈ {1, 2, 3, 4, 5, 7, 14, 21, 28} days after immunization. A total of 43 genes have been studied, which are selected from two gene modules defined in the published literature [6,22]. Prior to analysis, gene expression normalization, rescaling of the time points (to the unit interval), and the selection of knots order are conducted in a similar way as in the previous data analysis.

Two sets of analysis are conducted. In the first set, we focus on the samples under 305 A, which contain 20 samples treated with VZV and 20 with YFV. In the second set, we pool all 122 samples from the three protocols. We note that although the gene time courses have been analyzed in [43], there is insufficient attention to clustering. Complementary to the existing literature, our clustering analysis can potentially review sample heterogeneity as well as coordination among genes.

Results for the first set of analysis are presented in Fig. 5, where we observe two sample clusters and two gene clusters, leading to four biclusters. Here the two sample clusters match the VZV and YFV experimental conditions, which provides support to the validity of our analysis. The two gene clusters contain 27 and 16 members, respectively, which are very close to the module structure. Fig. 5 shows that the temporal trends of the four clusters differ significantly, with the level of variation and position of “peak” varying significantly. The observed trends are similar to those reported in [43]. We also refer to [43] for phamarcodynamic interpretations of the findings.

In the second set of analysis, we identify four sample clusters, with sizes 96, 5, 20, and 1, respectively. In what follows, we focus on the non-trivial clusters. Clusters 1 and 2 contain samples treated with VZV, HBV1, HBV3, ATIV, and TIV, and cluster 3 contains samples treated with YFV. In the original publication, there has been little attention to sample similarity/difference across protocols. Our analysis may suggest the significant difference between YFV and other treatments as well as the relative similarity of the five treatments (YFV excluded). Our analysis leads to two gene clusters, with sizes 25 and 18, respectively. This structure is again very similar to the module structure. The overall six non-trivial biclusters are shown in Fig. 4, where we observe significant across-cluster differences. Among the six patterns, biclusters 5 and 6 are similar to those observed in the first set of analysis, where biclusters 1–4 are relatively different.

Fig. 4.

Analysis of vaccine data with samples under all three protocols: Curves of observed data (black dotted) and estimated functions (blue solid) for non-trivial clusters, as well as yellow points indicating the estimated values at t ∈ {1, 2, 3, 4, 5, 7, 14, 21, 28} by the proposed method. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

The three alternatives are also applied. The bKmeans approach identifies three sample clusters (with sizes 20, 27, and 75) and two gene clusters (with sizes 26 and 17). Compared to the proposed approach, the adjusted Rand index values are 0.551 (sample), 0.907 (gene), and 0.666 (bicluster). The funHDDC approach identifies two sample clusters (with sizes 20 and 102) and three gene clusters (with sizes 26, 12 and 5). Compared to the proposed approach, the adjusted Rand index values are 0.819 (sample), 0.774 (gene), and 0.758 (bicluster). The funLBM approach identifies four sample clusters (with sizes 20, 39, 24 and 39) and two gene clusters (with sizes 20, 23). Compared to the proposed approach, the adjusted Rand index values are 0.276 (sample), 0.818 (gene), and 0.386 (bicluster).

5. Discussion

In this article, we have conducted the biclustering analysis when functions (to be exact, their realizations at discrete time points), as opposed to scalars, are present. The data structure fits time-course gene expression and other experiments. The analysis objective is considerably more complex than the biclustering analysis of scalars and one-way clustering of functions. We have developed a novel approach based on the penalized fusion technique. Methodologically, it differs significantly from the existing biclustering and fusion approaches. Theoretically, it has the much desired consistency property, making it advantageous over some of the existing alternatives that do not have theoretical support. Numerically, it has generated more accurate clustering and estimation in simulation and led to different findings in data analysis.

In our estimation, we have adopted the penalized smoothing technique. An alternative, which may be computationally simpler, is to take fewer basis functions, with which we can eliminate the smoothness penalty. Theoretically and numerically, we expect similar performance. The fusion technique involves pairwise differences/penalties, which may incur higher computational cost when N and/or q are large. In our simulation, we have considered moderate values, which match our data analysis. It will be of interest to develop computationally more scalable approaches/algorithms, for example via model averaging. This is beyond our scope and will be postponed to the future. In data analysis, findings with certain support have been made. In the literature, most existing studies are on the “static” functionalities of genes. It will be important to further understand the dynamics of gene expressions and more solidly interpret the findings.

Table 1.

Example 1: Mean, median, and standard error (SE) of ${\widehat{K}}_r$, ${\widehat{K}}_c$, and ${\widehat{K}}_b$ as defined in Section 2, as well as the percentage of identifying the corresponding true numbers based on 100 replicates.

N	Method	${\widehat{K}}_r$				${\widehat{K}}_c$				${\widehat{K}}_b$
		Mean	Median	SE	Per	Mean	Median	SE	Per	Mean	Median	SE	Per
30	Proposed	2.83	3.00	0.53	0.90	2.83	3.00	0.53	0.90	8.29	9.00	2.18	0.90
	bKmeans	2.76	3.00	0.64	0.66	1.13	1.00	0.46	0.05	3.09	3.00	1.34	0.03
	funHDDC	2.63	2.00	0.86	0.28	2.76	3.00	0.43	0.76	7.27	6.00	2.70	0.21
	funLBM	4.66	5.00	0.64	0.09	4.43	5.00	0.83	0.22	20.88	25.00	5.31	0.09
60	Proposed	2.91	3.00	0.43	0.93	2.90	3.00	0.41	0.94	8.61	9.00	1.74	0.93
	bKmeans	2.86	3.00	0.57	0.66	1.18	1.00	0.54	0.07	3.43	3.00	1.97	0.05
	funHDDC	2.20	2.00	0.64	0.04	2.99	3.00	0.10	0.99	6.58	6.00	1.92	0.04
	funLBM	3.42	3.00	0.64	0.66	3.24	3.00	0.55	0.82	11.15	9.00	3.31	0.55
90	Proposed	2.93	3.00	0.36	0.96	2.93	3.00	0.36	0.96	8.71	9.00	1.45	0.96
	bKmeans	2.83	3.00	0.51	0.74	1.23	1.00	0.58	0.08	3.51	3.00	1.87	0.08
	funHDDC	2.14	2.00	0.38	0.12	2.96	3.00	0.20	0.96	6.33	6.00	1.17	0.11
	funLBM	3.25	3.00	0.46	0.76	3.30	3.00	0.54	0.74	10.68	9.00	2.03	0.52

Appendix A. Proofs

Proof of Proposition 1.

By the definitions of ${\mathbf{H}}_r^{(m+1)}$ and ${\mathbf{H}}_c^{(m+1)}$, for any H_r and H_c, we have

$$L_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_r^{(m+1)},{\mathbf{H}}_c^{(m+1)},{\mathit{\Lambda }}_r^{(m)},{\mathit{\Lambda }}_c^{(m)}\right)\le L_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_r,{\mathbf{H}}_c,{\mathit{\Lambda }}_r^{(m)},{\mathit{\Lambda }}_c^{(m)}\right).$$

Fig. 5.

Analysis of vaccine data with samples under 305A: Curves of observed data (black dotted) and estimated functions (blue solid), as well as yellow points indicating the estimated values at t ∈ {1, 2, 3, 4, 5, 7, 14, 21, 28} by the proposed method. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Let $\Xi \left({\mathit{\beta }}^{(m+1)}\right)=\left\{\left({\mathbf{H}}_r,{\mathbf{H}}_c\right):{\mathbf{B}}_r{\mathit{\beta }}^{(m+1)}-\mathrm{vec}\left({\mathbf{H}}_r\right)=\mathbf{0},{\mathbf{B}}_c{\mathit{\beta }}^{(m+1)}-\mathrm{vec}\left({\mathbf{H}}_c\right)=\mathbf{0}\right\}$ and $\mathbf{P}={\sum }_{\delta \in {\Delta }^{(r)}}{p}_{\tau }\left({\Vert {\mathit{\eta }}_{\delta }^{(r)}\Vert }_2,{\gamma }_2\right)+{\sum }_{\delta \in {\Delta }^{(c)}}{p}_{\tau }\left({\Vert {\mathit{\eta }}_{\delta }^{(c)}\Vert }_2,{(N/q)}^{1/2}{\gamma }_2\right)$. We can define

$$f^{(m+1)}=\underset{\Xi \left({\mathit{\beta }}^{(m+1)}\right)}{\inf }\left\{\frac{1}{2}{\Vert \mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{(m+1)}\Vert }_2^2+\frac{1}{2}{\gamma }_1{\mathit{\beta }}^{(m+1)\top }\mathbf{M}{\mathit{\beta }}^{(m+1)}+\mathbf{P}\right\}=\underset{\Xi \left({\mathit{\beta }}^{(m+1)}\right)}{\inf }{L}_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_r,{\mathbf{H}}_c,{\mathit{\Lambda }}_r^{(m)},{\mathit{\Lambda }}_c^{(m)}\right),$$

and then $L_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_r^{(m+1)},{\mathbf{H}}_c^{(m+1)},{\mathit{\Lambda }}_r^{(m)},{\mathit{\Lambda }}_c^{(m)}\right)\le f^{(m+1)}$.

For any integer n, we have $\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m+n-1)}\right)=\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m)}\right)+\theta {\sum }_{i=1}^{n-1}\left[\mathrm{vec}\left({\mathbf{H}}_r^{(m+i)}\right)-{\mathbf{B}}_r{\mathit{\beta }}^{(m+i)}\right]$ and $\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m+n-1)}\right)=\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m)}\right)+\theta {\sum }_{i=1}^{n-1}\left[\mathrm{vec}\left({\mathbf{H}}_c^{(m+i)}\right)-{\mathbf{B}}_c{\mathit{\beta }}^{(m+i)}\right]$, and then

$$\begin{array}{l}{L}_{\theta }\left({\mathit{\beta }}^{(m+n)},{\mathbf{H}}_r^{(m+n)},{\mathbf{H}}_c^{(m+n)},{\mathit{\Lambda }}_r^{(m+n-1)},{\mathit{\Lambda }}_c^{(m+n-1)}\right)\\ =\frac{1}{2}{\Vert \mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{(m+n)}\Vert }_2^2+\frac{1}{2}{\gamma }_1{\mathit{\beta }}^{(m+n)\top }\mathbf{M}{\mathit{\beta }}^{(m+n)}+\mathbf{P}+{\left\{\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m)}\right)+\theta \sum _{i=1}^{n-1}\left[\mathrm{vec}\left({\mathbf{H}}_r^{(m+i)}\right)-{\mathbf{B}}_r{\mathit{\beta }}^{(m+i)}\right]\right\}}^{\top }\\ \times \left[\mathrm{Vec}\left({\mathbf{H}}_r^{(m+n)}\right)-{\mathbf{B}}_r{\mathit{\beta }}^{(m+n)}\right]\\ +{\left\{\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m)}\right)+\theta \sum _{i=1}^{n-1}\left[\mathrm{vec}\left({\mathbf{H}}_c^{(m+i)}\right)-{\mathbf{B}}_c{\mathit{\beta }}^{(m+i)}\right]\right\}}^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_c^{(m+n)}\right)-{\mathbf{B}}_c{\mathit{\beta }}^{(m+n)}\right]+\frac{\theta }{2}{\Vert \mathrm{vec}\left({\mathbf{H}}_r^{(m+n)}-{\mathbf{B}}_r{\mathit{\beta }}^{(m+n)}\right)\Vert }_2^2\\ +\frac{\theta }{2}{\Vert \mathrm{vec}\left({\mathbf{H}}_c^{(m+n)}-{\mathbf{B}}_c{\mathit{\beta }}^{(m+n)}\right)\Vert }_2^2\le f^{(m+n)}.\end{array}$$

Since the augmented Lagrangian function L_θ(β, H_r, H_c, Λ_rΛ_c) is differentiable with respect to β and is convex with respect to each ${\mathit{\eta }}_{\delta }^{(r)}$ and ${\mathit{\eta }}_{\delta }^{(c)}$. By Theorem 4.1 of [38], there exists a limit point of $\left({\mathit{\beta }}^{(m)},{\mathbf{H}}_r^{(m)},{\mathbf{H}}_c^{(m)}\right)$, denoted by $\left({\mathit{\beta }}^{*},{\mathbf{H}}_r^{*},{\mathbf{H}}_c^{*}\right)$. Then we have

$$f^{*}=\underset{m\to \infty }{\lim }{f}^{(m+1)}=\underset{m\to \infty }{\lim }{f}^{(m+n)}=\underset{\Xi \left({\mathit{\beta }}^{*}\right)}{\inf }\left\{\frac{1}{2}{\Vert \mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{*}\Vert }_2^2+\frac{1}{2}{\gamma }_1{\mathit{\beta }}^{*\top }\mathbf{M}{\mathit{\beta }}^{*}+\mathbf{P}\right\}$$

For all t ≥ 0, we have

$$\begin{array}{l}\underset{m\to \infty }{\lim }{L}_{\theta }\left({\mathit{\beta }}^{(m+n)},{\mathbf{H}}_r^{(m+n)},{\mathbf{H}}_c^{(m+n)},{\mathit{\Lambda }}_r^{(m+n-1)},{\mathit{\Lambda }}_c^{(m+n-1)}\right)\\ =\frac{1}{2}{\Vert \mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{*}\Vert }_2^2+\frac{1}{2}{\gamma }_1{\mathit{\beta }}^{*\top }\mathbf{M}{\mathit{\beta }}^{*}+\mathbf{P}+\underset{m\to \infty }{\lim }\mathrm{vec}{\left({\mathit{\Lambda }}_r^{(m)}\right)}^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_r^{*}\right)-{\mathbf{B}}_r{\mathit{\beta }}^{*}\right]+\left(n-\frac{1}{2}\right)\theta {\Vert \mathrm{vec}\left({\mathbf{H}}_r^{*}\right)-{\mathbf{B}}_r{\mathit{\beta }}^{*}\Vert }_2^2\\ +\underset{m\to \infty }{\lim }\mathrm{vec}{\left({\mathit{\Lambda }}_c^{(m)}\right)}^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_c^{*}\right)-{\mathbf{B}}_c{\mathit{\beta }}^{*}\right]+\left(n-\frac{1}{2}\right)\theta {\Vert \mathrm{vec}\left({\mathbf{H}}_c^{*}\right)-{\mathbf{B}}_c{\mathit{\beta }}^{*}\Vert }_2^2\le f^{*}.\end{array}$$

Thus

$$\underset{m\to \infty }{\lim }{\Vert {\mathit{r}}_r^{(m+1)}\Vert }_2^2={\Vert {\mathbf{B}}_r{\mathit{\beta }}^{*}-\mathrm{vec}\left({\mathbf{H}}_r^{*}\right)\Vert }_2^2=0,\underset{m\to \infty }{\lim }{\Vert {\mathit{r}}_c^{(m+1)}\Vert }_2^2={\Vert {\mathbf{B}}_c{\mathit{\beta }}^{*}-\mathrm{vec}\left({\mathbf{H}}_c^{*}\right)\Vert }_2^2=0.$$

Besides, by the definition of β^(m+1), we have that

$$\begin{array}{l}\partial L_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_r^{(m+1)},{\mathbf{H}}_c^{(m+1)},{\mathit{\Lambda }}_r^{(m)},{\mathit{\Lambda }}_c^{(m)}\right)/\partial \mathit{\beta }\\ =-{\mathbf{U}}^{\top }\left(\mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{(m+1)}\right)+{\gamma }_1\mathbf{M}{\mathit{\beta }}^{(m+1)}-\theta {\mathbf{B}}_r^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_r^{(m)}\right)+\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m)}\right)/\theta -{\mathbf{B}}_r{\mathit{\beta }}^{(m+1)}\right]\\ -\theta {\mathbf{B}}_c^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_c^{(m)}\right)+\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m)}\right)/\theta -{\mathbf{B}}_c{\mathit{\beta }}^{(m+1)}\right]\\ =-{\mathbf{U}}^{\top }\left(\mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{(m+1)}\right)+{\gamma }_1\mathbf{M}{\mathit{\beta }}^{(m+1)}-{\mathbf{B}}_r^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m)}\right)-\theta {\mathbf{B}}_r^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_r^{(m)}\right)-{\mathbf{B}}_r{\mathit{\beta }}^{(m+1)}\right]\\ -{\mathbf{B}}_c^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m)}\right)-\theta {\mathbf{B}}_c^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_c^{(m)}\right)-{\mathbf{B}}_c{\mathit{\beta }}^{(m+1)}\right]\\ =-{\mathbf{U}}^{\top }\left(\mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{(m+1)}\right)+{\gamma }_1\mathbf{M}{\mathit{\beta }}^{(m+1)}-{\mathbf{B}}_r^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m+1)}\right)+\theta {\mathbf{B}}_r^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_r^{(m+1)}\right)-\mathrm{vec}\left({\mathbf{H}}_r^{(m)}\right)\right]\\ -{\mathbf{B}}_c^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m+1)}\right)+\theta {\mathbf{B}}_c^{\top }\left[\mathrm{vec}\left({\mathbf{H}}_c^{(m+1)}\right)-\mathrm{vec}\left({\mathbf{H}}_c^{(m)}\right)\right]=0.\end{array}$$

Then we can obtain

$${\mathit{s}}_r^{(m+1)}+{\mathit{s}}_c^{(m+1)}={\mathbf{U}}^{\top }\left(\mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{(m+1)}\right)-{\gamma }_1\mathbf{M}{\mathit{\beta }}^{(m+1)}+{\mathbf{B}}_r^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m+1)}\right)+{\mathbf{B}}_c^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m+1)}\right).$$

By ${\Vert {\mathbf{B}}_r{\mathit{\beta }}^{*}-\mathrm{vec}\left({\mathbf{H}}_r^{*}\right)\Vert }_2^2=0$ and ${\Vert {\mathbf{B}}_c{\mathit{\beta }}^{*}-\mathrm{vec}\left({\mathbf{H}}_c^{*}\right)\Vert }_2^2=0$, we have

$$\begin{array}{l}\underset{m\to \infty }{\lim }\partial L_{\theta }\left({\mathit{\beta }}^{(m+1)},{\mathbf{H}}_r^{(m+1)},{\mathbf{H}}_c^{(m+1)},{\mathit{\Lambda }}_r^{(m)},{\mathit{\Lambda }}_c^{(m)}\right)/\partial \mathit{\beta }\\ =-{\mathbf{U}}^{\top }\left(\mathbf{Y}-\mathbf{U}{\mathit{\beta }}^{(m+1)}\right)+{\gamma }_1\mathbf{M}{\mathit{\beta }}^{(m+1)}-{\mathbf{B}}_r^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_r^{(m+1)}\right)-{\mathbf{B}}_c^{\top }\mathrm{vec}\left({\mathit{\Lambda }}_c^{(m+1)}\right)=\mathbf{0}.\end{array}$$

Therefore ${\lim }_{m\to \infty }{\mathit{s}}_r^{(m+1)}+{\mathit{s}}_c^{(m+1)}=\mathbf{0}$. □

Let $\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|={\sum }_{(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}}{n}_{i,j}$ and $n_m={\max }_{i\in \{1,\dots ,N\},j\in \{1,\dots ,q\}}{n}_{i,j}<\infty$. Then $\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|\le \left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|\le n_m\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|$. Denote the number of internal knots as J and then J = p − d. Recall that $b={\min }_{\left(k_r,k_c\right)\ne \left(k_r^{\prime },k_c^{\prime }\right)}\Vert g_{\left(k_r,k_c\right)}^{*}-g_{\left(k_r^{\prime },k_c^{\prime }\right)}^{*}\Vert$.

Lemma 1.

Under Condition (C1), there exists a spline approximation ${\mathit{\alpha }}_{k_r,k_c}^{*\top }{U}_p(t)$ of the true function $g_{\left(k_r,k_c\right)}^{*}(t)$ for k_r ∈ {1, …, K_r} and k_c ∈ {1, …, K_c}, such that

$$\underset{t\in \mathcal{T}}{\sup }\left|g_{\left(k_r,k_c\right)}^{*}(t)-{\mathit{\alpha }}_{k_r,k_c}^{*\top }{U}_p(t)\right|=O\left(J^{-\kappa }\right).$$

Proof.

Lemma 1 follows from Corollary 6.21 of [34]. This lemma has been used in a number of studies that involve spline expansion [25,42]. We omit the proof here. □

Lemma 2. Under Conditions (C1)–(C3) and b ≫ J^−κ, there exists a constant C₂ > 0 such that for all $\left(k_r,k_c\right)\ne \left(k_r^{\prime },k_c^{\prime }\right)$, such that

$${\Vert {\alpha }_{k_r,k_c}^{*}-{\alpha }_{k_r^{\prime },k_c^{\prime }}^{*}\Vert }_2\ge \frac{1}{2}{C}_2^{-1/2}b,$$

when N and q are sufficiently large.

Proof.

By the triangular inequality, we have

$$\Vert {\left({\alpha }_{k_r,k_c}^{*}-{\alpha }_{k_r^{\prime },k_c^{\prime }}^{*}\right)}^{\top }{U}_p\Vert \ge \Vert g_{\left(k_r,k_c\right)}^{*}-g_{\left(k_r^{\prime },k_c^{\prime }\right)}^{*}\Vert -\Vert g_{\left(k_r,k_c\right)}^{*}-{\alpha }_{k_r,k_c}^{*\top }{U}_p\Vert -\Vert g_{\left(k_r^{\prime },k_c^{\prime }\right)}^{*}-{\alpha }_{k_r^{\prime },k_c^{\prime }}^{*\top }{U}_p\Vert .$$ (A.1)

Besides, by Theorem 5.4.2 of [13], Condition (C2), and the definition of the rescaled B-spline basis, for any vector ${\mathit{\alpha }}_{p\times 1}^{\prime }$, there exists a constant C₂ > 0 such that

$${\Vert {\mathit{\alpha }}^{\prime \top }{U}_p\Vert }^2\le C_2{\Vert {\mathit{\alpha }}^{\prime }\Vert }_2^2.$$ (A.2)

Combining (A.1), (A.2), and Lemma 1, we have

$$\begin{array}{l}{\Vert {\alpha }_{k_r,k_c}^{*}-{\alpha }_{k_r^{\prime },k_c^{\prime }}^{*}\Vert }_2\ge C_2^{-1/2}\left\{\Vert g_{\left(k_r,k_c\right)}^{*}-g_{\left(k_r^{\prime },k_c^{\prime }\right)}^{*}\Vert -\Vert g_{\left(k_r,k_c\right)}^{*}-{\alpha }_{k_r,k_c}^{*\top }{U}_p\Vert -\Vert g_{\left(k_r,k_c\right)}^{*}-{\alpha }_{k_r,k_c}^{*\top }{U}_p\Vert \right\}\\ \ge C_2^{-1/2}\left(b-2M_2{J}^{-\kappa }\right)>C_2^{-1/2}\left(b-2\times \frac{1}{4}b\right)=\frac{1}{2}{C}_2^{-1/2}b,\end{array}$$

where the third inequality is obtained when N and q are sufficiently large since b ≫ J^−κ. □

Lemma 3

(Bernstein’s Inequality, Lemma 2.2.11 in [39]). For independent random variables Y₁, …, Y_n with means 0 and $E{\left|Y_i\right|}^m\le m!M^{m-2}{v}_i/2$ for some constants M, v_i, and every m ≥ 2,

$$P\left(\left|Y_1+\cdots +Y_n\right|>x\right)\le 2\exp \left\{-\frac{1}{2}\frac{x^2}{v+Mx}\right\},$$

where v = v₁ + · · · + v_n.

Proof of Theorem 1.

Given ${\widehat{\mathit{\beta }}}^{or}\in {\mathcal{M}}_G$, when the true block memberships ${\mathcal{G}}_{1,1}^{(r,c)},\dots ,G_{K_r,K_c}^{(r,c)}$ are known, the oracle estimators for all β_i,j’s are the same if $(i.j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}$. Thus we can explore the properties of ${\widehat{\mathit{\beta }}}^{or}$ by examining the properties of the oracle common coefficient vector ${\widehat{\mathit{\alpha }}}^{or}={\left({\widehat{\mathit{\alpha }}}_{1,1}^{or\top },\dots ,{\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or\top },\dots ,{\widehat{\mathit{\alpha }}}_{K_r,K_c}^{or\top }\right)}^{\top }$, which is defined as

$${\widehat{\mathit{\alpha }}}^{or}=\arg \underset{\mathit{\alpha }}{\min }\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}{\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right),$$

and

$${\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)=\frac{1}{2}{\Vert {\mathbf{Y}}_{\left(k_r,k_c\right)}-{\mathbf{U}}_{\left(k_r,k_c\right)}{\mathit{\alpha }}_{k_r,k_c}\Vert }_2^2+{\gamma }_1\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|{\mathit{\alpha }}_{k_r,k_c}^{\top }\mathbf{D}{\mathit{\alpha }}_{k_r,k_c},$$

where ${\mathbf{Y}}_{\left(k_r,k_c\right)}=\mathrm{vec}\left\{Y_{i,j},(i,j)\in {\mathcal{G}}_{k_r,k_c}\right\}$, ${\mathbf{U}}_{\left(k_r,k_c\right)}={\left({\mathbf{U}}_{i,j}^{\top },(i,j)\in {\mathcal{G}}_{k_r,k_c}\right)}^{\top }$. The corresponding true B-spline coefficient vector is denoted by ${\mathit{\alpha }}^{*}={\left({\mathit{\alpha }}_{1,1}^{*\top },\dots ,{\mathit{\alpha }}_{k_r,k_c}^{*\top },\dots ,{\mathit{\alpha }}_{K_r,K_c}^{*\top }\right)}^{\top }$. Note that

$${\frac{\partial {\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)}{\partial {\mathit{\alpha }}_{k_r,k_c}}|}_{{\mathit{\alpha }}_{k_r,k_c}={\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}}-{\frac{\partial {\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)}{\partial {\mathit{\alpha }}_{k_r,k_c}}|}_{{\mathit{\alpha }}_{k_r,k_c}={\mathit{\alpha }}_{k_r,k_c}^{*}}={\frac{\partial {\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)}{\partial {\mathit{\alpha }}_{k_r,k_c}\partial {\mathit{\alpha }}_{k_r,k_c}^{\top }}|}_{{\mathit{\alpha }}_{k_r,k_c}={\overline{\mathit{\alpha }}}_{k_r,k_c}}\left({\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}-{\mathit{\alpha }}_{k_r,k_c}^{*}\right),$$

where ${\overline{\mathit{\alpha }}}_{k_r,k_c}$ is between ${\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}$ and ${\mathit{\alpha }}_{k_r,k_c}^{*}$. Then we have

$${\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}-{\mathit{\alpha }}_{k_r,k_c}^{*}=-{{\left({\frac{\partial {\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)}{\partial {\mathit{\alpha }}_{k_r,k_c}\partial {\mathit{\alpha }}_{k_r,k_c}^{\top }}|}_{{\mathit{\alpha }}_{k_r,k_c}={\overline{\mathit{\alpha }}}_{k_r,k_c}}\right)}^{-1}\frac{\partial {\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)}{\partial {\mathit{\alpha }}_{k_r,k_c}}|}_{{\mathit{\alpha }}_{k_r,k_c}={\mathit{\alpha }}_{k_r,k_c}^{*}}.$$

Hence

$${\Vert {\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}-{\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2\le {\Vert \left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|{\left({\frac{\partial {\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)}{\partial {\mathit{\alpha }}_{k_r,k_c}\partial {\mathit{\alpha }}_{k_r,k_c}^{\top }}|}_{{\mathit{\alpha }}_{k_r,k_c}={\overline{\mathit{\alpha }}}_{k_r,k_c}}\right)}^{-1}\Vert }_2{\Vert {{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}^{-1}\frac{\partial {\widehat{L}}^{or}\left({\mathit{\alpha }}_{k_r,k_c}\right)}{\partial {\mathit{\alpha }}_{k_r,k_c}}|}_{{\mathit{\alpha }}_{k_r,k_c}={\mathit{\alpha }}_{k_r,k_c}^{*}}\Vert }_2:=A_{k_r,k_c}^{(1)}\times A_{k_r,k_c}^{(2)}.$$ (A.3)

By Lemma A.8 of [41], Conditions (C1) and (C2), we can derive that there exists a constant C₃ > 0 such that for any 1 ≤ k_r ≤ K_r, 1 ≤ k_c ≤ K_c,

$$P\left(A_{k_r,k_c}^{(1)}\le C_3\right)=P\left({\Vert \frac{{\mathbf{U}}_{\left(k_r,k_c\right)}^{\top }{\mathbf{U}}_{\left(k_r,k_c\right)}}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}+\frac{{\gamma }_1\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|\mathbf{D}}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}\Vert }_2\le C_3\right)\ge 1-p/(Nq).$$ (A.4)

Besides, note that

$$\begin{array}{l}{A}_{k_r,k_c}^{(2)}={\Vert -\frac{{\mathbf{U}}_{\left(k_r,k_c\right)}^{\top }}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}\left({\mathbf{Y}}_{\left(k_r,k_c\right)}-{\mathit{g}}_{\left(k_r,k_c\right)}^{*}+{\mathit{g}}_{\left(k_r,k_c\right)}^{*}-{\mathbf{U}}_{\left(k_r,k_c\right)}{\mathit{\alpha }}_{k_r,k_c}^{*}\right)+{\gamma }_1\frac{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}\mathbf{D}{\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2\\ \le {\Vert \frac{{\mathbf{U}}_{\left(k_r,k_c\right)}^{\top }}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}{ϵ}_{k_r,k_c}\Vert }_2+{\Vert \frac{{\mathbf{U}}_{\left(k_r,k_c\right)}^{\top }}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}\left({\mathit{g}}_{\left(k_r,k_c\right)}^{*}-{\mathbf{U}}_{\left(k_r,k_c\right)}{\mathit{\alpha }}_{k_r,k_c}^{*}\right)\Vert }_2+{\Vert {\gamma }_1\frac{\mid {\mathcal{G}}_{k_r,k_c}^{(r,c)}}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}\mathbf{D}{\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2:=B_{k_r,k_c}^{(1)}+B_{k_r,k_c}^{(2)}+B_{k_r,k_c}^{(3)}.\end{array}$$ (A.5)

Since the rescaled B-spline values are finite, there exists constant M₁ > 0 such that U_l,p(t) ≤ M₁ for l ∈ {1, …, p}. Let U_(i,j)·l denote the lth column of U_(i,j), and we verify the condition of Lemma 3 by Condition (C5)

$$\text{E}{\left|{\mathbf{U}}_{(i,j)\cdot l}^{\top }{ϵ}_{i,j}\right|}^m\le \text{E}\left({\left|{\mathbf{U}}_{(i,j)\cdot l}^{\top }{\mathbf{U}}_{(i,j)\cdot l}\right|}^{m/2}\cdot {\left|{ϵ}_{i,j}^{\top }{ϵ}_{i,j}\right|}^{m/2}\right)\le {\left(F^{-1}{M}_1\right)}^{m}m!\text{E}\left(\exp \left\{F{\left|n_{i,j}^{-1}{ϵ}_{i,j}^{\top }{ϵ}_{i,j}\right|}^{1/2}\right\}\right)\le {\left(F^{-1}{M}_1\right)}^{m}m!c_2.$$

Applying Lemma 3, we have

$$P\left(\left|\sum _{(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}}{\mathbf{U}}_{(i,j)\cdot l}^{\top }{ϵ}_{i,j}\right|>x\right)\le 2\exp \left\{-\frac{1}{2}\frac{x^2}{v+F^{-1}{M}_{1}x}\right\},$$ (A.6)

where $v={\sum }_{(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}}{v}_{i,j}$ and $v_{i,j}=2F^{-2}{M}_1^2{c}_2$.

Let ${\mathbf{U}}_{\left(k_r,k_c\right)\cdot l}$ denote the lth column of ${\mathbf{U}}_{\left(k_r,k_c\right)}$. For some constant 0 < C < ∞, combining Condition (C5) and (A.6), we have

$$\begin{array}{l}P\left({\Vert {\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}^{-1}{\mathbf{U}}_{\left(k_r,k_c\right)}^{\top }{ϵ}_{k_r,k_c}\Vert }_{\infty }>C{F}^{-1}{M}_1{\left(\log (Nq)/\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|\right)}^{1/2}\right)\\ \le \sum _{l=1}^{p}P\left(\left|{\mathbf{U}}_{\left(k_r,k_c\right)\cdot l}^{\top }{ϵ}_{k_r,k_c}\right|>C{F}^{-1}{M}_1{\left(\log (Nq)\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|\right)}^{1/2}\right)\\ =\sum _{l=1}^{p}P\left(\left|\sum _{(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}}{\mathbf{U}}_{(i,j)\cdot l}^{\top }{ϵ}_{i,j}\right|>C{F}^{-1}{M}_1{\left(\log (Nq)\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|\right)}^{1/2}\right)\\ \le 2p\exp \left\{-\frac{1}{2}\frac{C^2{F}^{-2}{M}_1^2\left(\log (Nq)\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|\right)}{2F^{-2}{M}_1^2{c}_2\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|+C{F}^{-2}{M}_1^2{\left(\log (Nq)\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|\right)}^{1/2}}\right\}\le 2p\exp \{-\log (Nq)\}\le 2p/Nq.\end{array}$$

Hence, we have that with probability at least 1 − 2p/(Nq),

$$B_{k_r,k_c}^{(1)}\le C{F}^{-1}{M}_1{\left(p\log (Nq)/\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|\right)}^{1/2}.$$ (A.7)

By Lemma 1, there exists a constant M₂ > 0 such that

$$B_{k_r,k_c}^{(2)}\le p^{1/2}{\Vert \frac{{\mathbf{U}}_{\left(k_r,k_c\right)}^{\top }}{\left|{\mathcal{G}}_{k_r,k_c}^{(r)*}\right|}\left({\mathit{g}}_{\left(k_r,k_c\right)}^{*}-{\mathbf{U}}_{\left(k_r,k_c\right)}{\mathit{\alpha }}_{k_r,k_c}^{*}\right)\Vert }_{\infty }\le p^{1/2}{\Vert \frac{{\mathbf{U}}_{\left(k_r,k_c\right)}^{\top }}{\left|{\mathcal{G}}_{k_r,k_c)}^{(r,c)*}\right|}\Vert }_{\infty }{\Vert \left({\mathit{g}}_{\left(k_r,k_c\right)}^{*}-{\mathbf{U}}_{\left(k_r,k_c\right)}{\mathit{\alpha }}_{k_r,k_c}^{*}\right)\Vert }_{\infty }\le M_1{M}_2{p}^{1/2}{J}^{-\kappa }.$$ (A.8)

In addition,

$$B_{k_r,k_c}^{(3)}\le {\gamma }_1\frac{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)*}\right|}{\Vert {\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2\Vert \mathbf{D}{\Vert }_2\le p^{1/2}{\gamma }_1{\Vert {\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_{\infty }\Vert \mathbf{D}{\Vert }_2.$$ (A.9)

Thus by (A.5), (A.7), (A.8), and (A.9), for any 1 ≤ k_r ≤ K_r, 1 ≤ k_c ≤ K_c, with probability at least 1 − 2p/(Nq),

$$A_{k_r,k_c}^{(2)}\le C{F}^{-1}{M}_1{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}+M_1{M}_2{p}^{1/2}{J}^{-\kappa }+\underset{k_r,k_c}{\max }{\Vert {\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_{\infty }\Vert \mathbf{D}{\Vert }_2{\gamma }_1{p}^{1/2}.$$

By Condition (C1) and ${\gamma }_1=o\left({\left|{\mathcal{G}}_{min}^{(r,c)}\right|}^{-1/2}\right)$, when N and q are sufficiently large, we have

$$p^{1/2}{J}^{-\kappa }\ll {\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2},p^{1/2}{\gamma }_1\ll {\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}.$$

Hence, for any 1 ≤ k_r ≤ K_r, 1 ≤ k_c ≤ K_c, with probability at least 1 − 2p/(Nq),

$$A_{k_r,k_c}^{(2)}\le C_4{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2},$$

where C₄ is a large constant. Together with (A.3) and (A.4), for any 1 ≤ k_r ≤ K_r, 1 ≤ k_c ≤ K_c,

$$\begin{array}{l}P\left({\Vert {\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}-{\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2\le C_3{C}_4{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}\right)\ge 1-P\left(A_{k_r,k_c}^{(1)}>C_3\right)-P\left(A_{k_r,k_c}^{(2)}>C_4{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}\right)\\ \ge 1-3p/(Nq).\end{array}$$

By the Bonferroni’s inequality, we have

$$\begin{array}{l}P\left(\underset{1\le k_r\le K_r,1\le k_c\le K_c}{\sup }{\Vert {\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}-{\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2\le C_3{C}_4{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}\right)\\ \ge 1-\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}P\left({\Vert {\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}-{\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2>C_3{C}_4{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}\right)\ge 1-3K_r{K}_{c}p/(Nq).\end{array}$$

By Lemma 1 and (A.2), we have

$$\begin{array}{l}\Vert {\widehat{g}}_{\left(k_r,k_c\right)}^{or}-g_{\left(k_r,k_c\right)}^{*}\Vert =\Vert {\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or\top }{U}_p-{\mathit{\alpha }}_{k_r,k_c}^{*\top }{U}_p+{\mathit{\alpha }}_{k_r,k_c}^{*\top }{U}_p-{\mathit{g}}_{\left(k_r,k_c\right)}^{*}\Vert \le \Vert {\left({\widehat{\mathit{\alpha }}}_{k_r,k_c}^{or}-{\mathit{\alpha }}_{k_r,k_c}^{*}\right)}^{\top }{U}_p\Vert +\Vert {\mathit{\alpha }}_{k_r,k_c}^{*\top }{U}_p-{\mathit{g}}_{\left(k_r,k_c\right)}^{*}\Vert \\ \le C_2^{1/2}{C}_3{C}_4{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}+M_2{J}^{-\kappa }\le \left(C_2^{1/2}{C}_3{C}_4+M_2/2\right){\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}\\ =C^{*}{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2},\end{array}$$

where $C^{*}=\max \left\{C_3{C}_4,C_2^{1/2}{C}_3{C}_4+M_2/2\right\}$. That is,

$$P\left(\underset{1\le k_r\le K_r,1\le k_c\le K_c}{\sup }\Vert {\widehat{g}}_{\left(k_r,k_c\right)}^{or}-g_{\left(k_r,k_c\right)}^{*}\Vert \le \psi \right)\ge 1-3K_r{K}_{c}p/(Nq),$$

where $\psi =C^{*}{\left(p\log (Nq)/\left|{\mathcal{G}}_{min}^{(r,c)}\right|\right)}^{1/2}$. □

Proof of Theorem 2.

Let ${\rho }_1(t)={\gamma }_2^{-1}{p}_{\tau }\left(t,{\gamma }_2\right)$ and ${\rho }_2(t)={\left({(N/q)}^{1/2}{\gamma }_2\right)}^{-1}{p}_{\tau }\left(t,{(N/q)}^{1/2}{\gamma }_2\right)$. Define

$$Q(\mathit{\beta })=\frac{1}{2}\sum _{i=1}^N\sum _{j=1}^q\left({\Vert {\mathbf{Y}}_{i,j}-{\mathbf{U}}_{i,j}{\mathit{\beta }}_{i,j}\Vert }_2^2+{\gamma }_1{\mathit{\beta }}_{i,j}^{\top }\mathbf{D}{\mathit{\beta }}_{i,j}\right),$$

$$Pen(\mathit{\beta })={\gamma }_2\sum _{\left(i_1,i_2\right)\in {\Delta }^{(r)}}{\rho }_1\left({\Vert {\mathit{\beta }}_{i_1}^{(r)}-{\mathit{\beta }}_{i_2}^{(r)}\Vert }_2\right)+{(N/q)}^{1/2}{\gamma }_2\sum _{\left(j_1,j_2\right)\in {\Delta }^{(c)}}{\rho }_2\left({\Vert {\mathit{\beta }}_{j_1}^{(c)}-{\mathit{\beta }}_{j_2}^{(c)}\Vert }_2\right),$$

$$Q^{\mathcal{G}}(\mathit{\alpha })=\frac{1}{2}\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\left({\Vert {\mathbf{Y}}_{\left(k_r,k_c\right)}-{\mathbf{U}}_{\left(k_r,k_c\right)}{\mathit{\alpha }}_{k_r,k_c}\Vert }_2^2+{\gamma }_1\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|{\mathit{\alpha }}_{k_r,k_c}^{\top }\mathbf{D}{\mathit{\alpha }}_{k_r,k_c}\right),$$

$$Pe{n}^{\mathcal{G}}(\mathit{\alpha })={\gamma }_2\sum _{k_r<k_r^{\prime }}\left|{\mathcal{G}}_{k_r}^{(r)}\right|\left|{\mathcal{G}}_{k_r^{\prime }}^{(r)}\right|{\rho }_1\left({\Vert {\mathit{\alpha }}_{k_r}^{(r)}-{\mathit{\alpha }}_{k_r^{\prime }}^{(r)}\Vert }_2\right)+{(N/q)}^{1/2}{\gamma }_2\sum _{k_c<k_c^{\prime }}\left|{\mathcal{G}}_{k_c}^{(c)}\right|\left|{\mathcal{G}}_{k_c^{\prime }}^{(c)}\right|{\rho }_2\left({\Vert {\mathit{\alpha }}_{k_c}^{(c)}-{\mathit{\alpha }}_{k_c^{\prime }}^{(c)}\Vert }_2\right),$$

where ${\mathit{\alpha }}_{k_r}^{(r)}={\left({\mathit{\alpha }}_{k_r,1}^{(r)\top },\dots ,{\mathit{\alpha }}_{k_r,q}^{(r)\top }\right)}^{\top }$ with ${\mathit{\alpha }}_{k_r,j}^{(r)}={\mathit{\alpha }}_{k_r,k}$ if $j\in {\mathcal{G}}_k^{(c)}$, ${\mathit{\alpha }}_{k_c}^{(c)}={\left({\mathit{\alpha }}_{1,k_c}^{(c)\top },\dots ,{\mathit{\alpha }}_{N,k_c}^{(c)\top }\right)}^{\top }$ with ${\mathit{\alpha }}_{i,k_C}^{(c)}={\mathit{\alpha }}_{k,k_c}$ if $i\in {\mathcal{G}}_k^{(r)}$. Let $L(\mathit{\beta })=Q(\mathit{\beta })+Pen(\mathit{\beta })$, $L^{\mathcal{G}}(\mathit{\alpha })=Q^{\mathcal{G}}(\mathit{\alpha })+Pe{n}^{\mathcal{G}}(\mathit{\alpha })$

We define two mappings, $\tilde{T}:{\mathcal{M}}_{\mathcal{G}}\to {\tilde{\mathcal{M}}}_{\mathcal{G}}$ and $\widehat{T}:{ℝ}^{Nqp}\to {\widehat{\mathcal{M}}}_{\mathcal{G}}$, and the two subspaces are defined by

$${\tilde{\mathcal{M}}}_{\mathcal{G}}=\{\mathit{\alpha }\in {ℝ}^{K_r{K}_{c}p}:{\mathit{\alpha }}_{k_r,k_c}={\mathit{\beta }}_{i,j},\text{for}\text{any}(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)},1\le k_r\le K_r,1\le k_c\le K_c\},$$

$${\widehat{\mathcal{M}}}_{\mathcal{G}}=\left\{\mathit{\alpha }\in {ℝ}^{K_r{K}_{c}p}:{\mathit{\alpha }}_{k_r,k_c}={\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|}^{-1}\sum _{(i,j)\in {\mathcal{G}}_{k_r,k_c}^{(r,c)}}{\mathit{\beta }}_{i,j},1\le k_r\le K_r,1\le k_c\le K_c\right\}.$$

For every $\mathit{\beta }\in {\mathcal{M}}_{\mathcal{G}}$, we have $Pen(\mathit{\beta })=Pe{n}^{\mathcal{G}}(\tilde{T}(\mathit{\beta }))$, and for every $\mathit{\alpha }\in {\tilde{\mathcal{M}}}_{\mathcal{G}}$, we have $Pen\left({\tilde{T}}^{-1}(\mathit{\alpha })\right)=Pe{n}^{\mathcal{G}}(\mathit{\alpha })$. Hence

$$L(\mathit{\beta })=L^{\mathcal{G}}(\tilde{T}(\mathit{\beta })),L^{\mathcal{G}}(\mathit{\alpha })=L\left({\tilde{T}}^{-1}(\mathit{\alpha })\right).$$ (A.10)

Consider the neighborhood of β*:

$$\Theta =\left\{\mathit{\beta }\in {ℝ}^{Nqp}:\underset{1\le i\le N,1\le j\le q}{\sup }{\Vert {\mathit{\beta }}_{i,j}-{\mathit{\beta }}_{i,j}^{*}\Vert }_2\le \psi \right\}.$$

By the result in Theorem 1, there is an event E₁ such that on E₁,

$$\underset{1\le i\le N,1\le j\le q}{\sup }{\Vert {\widehat{\mathit{\beta }}}_{i,j}^{or}-{\mathit{\beta }}_{i,j}^{*}\Vert }_2\le \psi ,$$

and $P\left(E_1^C\right)\le 3K_r{K}_{c}p/(Nq)$. Hence ${\widehat{\mathit{\beta }}}^{or}\in \Theta$ on E₁. For any $\mathit{\beta }\in {ℝ}^{Nqp}$, let $\tilde{\mathit{\beta }}={\tilde{T}}^{-1}(\widehat{T}(\mathit{\beta }))$. Inspired by [27], we show that ${\widehat{\mathit{\beta }}}^{or}$ is a strictly local minimizer of objective function (3) with probability tending to 1 through the following two steps:

1. (i) On E₁, $L(\tilde{\mathit{\beta }})>L\left({\widehat{\mathit{\beta }}}^{or}\right)$ for any β ∈ Θ and $\tilde{\mathit{\beta }}\ne {\widehat{\mathit{\beta }}}^{or}$.

2. (ii) There is an event E₂ such that $P\left(E_2^C\right)\le c_2/(Nq)$. On E₁ ∩ E₂, there is a neighborhood of ${\widehat{\mathit{\beta }}}^{or}$, denoted by Θ^′, such that $L(\mathit{\beta })\ge L(\tilde{\mathit{\beta }})$ for any β ∈ Θ^′ ∩ Θ for sufficiently large N and q.

Therefore, by the results in (i) and (ii), we have $L(\mathit{\beta })>L\left({\widehat{\mathit{\beta }}}^{or}\right)$ for any β ∈ Θ^′ ∩ Θ and $\tilde{\mathit{\beta }}\ne {\widehat{\mathit{\beta }}}^{or}$, so that ${\widehat{\mathit{\beta }}}^{or}$ is a strictly local minimizer of L(β) on E₁ ∩ E₂ with P(E₁ ∩ E₂) ≥ 1 − 3K_rK_pp/(Nq) − c₂/(Nq) for sufficiently large N and q.

Firstly, we prove the result in (i). Let $\widehat{T}(\mathit{\beta })=\mathit{\alpha }={\left({\mathit{\alpha }}_{1,1}^{\top },\dots ,{\mathit{\alpha }}_{K_r,K_C}^{\top }\right)}^{\top }$ and ${\mathit{\alpha }}_{k_r}^{(r)*}={\left({\mathit{\beta }}_{i,1}^{(r)*\top },\dots ,{\mathit{\beta }}_{i,q}^{(r)*\top }\right)}^{\top }$ for $i\in {\mathcal{G}}_{k_r}^{(r)}$. Since

$${\Vert {\mathit{\alpha }}_{k_r}^{(r)}-{\mathit{\alpha }}_{k_r^{\prime }}^{(r)}\Vert }_2\ge {\Vert {\mathit{\alpha }}_{k_r}^{(r)*}-{\mathit{\alpha }}_{k_r^{\prime }}^{(r)*}\Vert }_2-2\underset{1\le k_r\le K_r}{\sup }{\Vert {\mathit{\alpha }}_{k_r}^{(r)}-{\mathit{\alpha }}_{k_r}^{(r)*}\Vert }_2,$$

and

$$\begin{gathered} \underset{1\le k_r\le K_r}{\sup }{\Vert {\mathit{\alpha }}_{k_r}^{(r)}-{\mathit{\alpha }}_{k_r}^{(r)*}\Vert }_2^2=\underset{1\le k_r\le K_r}{\sup }\left\{\sum _{k_c=1}^{K_c}\left|{\mathcal{G}}_{k_c}^{(c)}\right|\cdot {\Vert \sum _{i\in {\mathcal{G}}_{k_r}^{(r)}}\sum _{j\in {\mathcal{G}}_{k_c}^{(c)}}{\mathit{\beta }}_{i,j}/\left(\left|{\mathcal{G}}_{k_r}^{(r)}\Vert {\mathcal{G}}_{k_c}^{(c)}\right|\right)-{\mathit{\alpha }}_{k_r,k_c}^{*}\Vert }_2^2\right\} \\ \le \underset{1\le k_r\le K_r}{\sup }{\left|{\mathcal{G}}_{k_r}^{(r)}\right|}^{-1}\sum _{k_c=1}^{K_c}\sum _{i\in {\mathcal{G}}_{k_r}^{(r)}}\sum _{j\in {\mathcal{G}}_{k_c}^{(c)}}{\Vert {\mathit{\beta }}_{i,j}-{\mathit{\beta }}_{i,j}^{*}\Vert }_2^2\le q\underset{1\le i\le N,1\le j\le q}{\sup }{\Vert {\mathit{\beta }}_{i,j}-{\mathit{\beta }}_{i,j}^{*}\Vert }_2^2 \end{gathered}$$ (A.11)

by Lemma 2, for any $k_r\ne k_r^{\prime }$

$$\begin{array}{l}{\Vert {\mathit{\alpha }}_{k_r}^{(r)}-{\mathit{\alpha }}_{k_r^{\prime }}^{(r)}\Vert }_2\ge \frac{1}{2}{\left|{\mathcal{G}}_{min}^{(c)}\right|}^{1/2}{C}_2^{-1/2}b-2q^{1/2}\underset{1\le i\le N,1\le j\le q}{\sup }{\Vert {\mathit{\beta }}_{i,j}-{\mathit{\beta }}_{i,j}^{*}\Vert }_2\\ \ge \frac{1}{2}{\left|{\mathcal{G}}_{min}^{(c)}\right|}^{1/2}{C}_2^{-1/2}b-2q^{1/2}{C}_3\psi >a{\gamma }_2.\end{array}$$

The last inequality follows from the assumption that ${\left|{\mathcal{G}}_{min}^{(c)}\right|}^{1/2}b\gg {\gamma }_2\gg {(pq)}^{1/2}\log (Nq)/\min \left\{\left|{\mathcal{G}}_{min}^{(r)}\right|,\left|{\mathcal{G}}_{min}^{(c)}\right|\right\}\gg q^{1/2}\psi$ Similarly, for any $k_c\ne k_c^{\prime }$, we have

$$\begin{array}{l}{\Vert {\mathit{\alpha }}_{k_c}^{(c)}-{\mathit{\alpha }}_{k_c^{\prime }}^{(c)}\Vert }_2\ge \frac{1}{2}{\left|{\mathcal{G}}_{min}^{(r)}\right|}^{1/2}{C}_2^{-1/2}b-2N^{1/2}\underset{1\le i\le N,1\le j\le q}{\sup }{\Vert {\mathit{\beta }}_{i,j}-{\mathit{\beta }}_{i,j}^{*}\Vert }_2\\ \ge \frac{1}{2}{\left|{\mathcal{G}}_{min}^{(r)}\right|}^{1/2}{C}_2^{-1/2}b-2N^{1/2}{C}_3\psi >a{\left(N/q\right)}^{1/2}{\gamma }_2.\end{array}$$

Hence by Condition (C4), $Pe{n}^{\mathcal{G}}(\widehat{T}(\mathit{\beta }))=C_{pen}$, a constant, and hence $L^{\mathcal{G}}(\widehat{T}(\mathit{\beta }))=Q^{\mathcal{G}}(\widehat{T}(\mathit{\beta }))+C_{pen}$ for all β ∈ Θ. Since ${\widehat{\mathit{\alpha }}}^{or}$ is the unique global minimizer of $Q^{\mathcal{G}}(\mathit{\alpha })$, $Q^{\mathcal{G}}(\widehat{T}(\mathit{\beta }))>Q^{\mathcal{G}}\left({\widehat{\mathit{\alpha }}}^{or}\right)$ for all $\widehat{T}(\mathit{\beta })\ne {\widehat{\mathit{\alpha }}}^{or}$, and thus $L^{\mathcal{G}}(\widehat{T}(\mathit{\beta }))>L^{\mathcal{G}}\left({\widehat{\mathit{\alpha }}}^{or}\right)$ for all $\widehat{T}(\mathit{\beta })\ne {\widehat{\mathit{\alpha }}}^{or}$. By (A.10), we have $L^{\mathcal{G}}(\widehat{T}(\mathit{\beta }))=L(\tilde{\mathit{\beta }})$ and $L^{\mathcal{G}}\left({\widehat{\mathit{\alpha }}}^{or}\right)=L\left({\widehat{\mathit{\beta }}}^{or}\right)$. Therefore $L(\tilde{\mathit{\beta }})>L\left({\widehat{\mathit{\beta }}}^{or}\right)$ for all $\tilde{\mathit{\beta }}\ne {\widehat{\mathit{\beta }}}^{or}$, and the result (i) is proved.

Next we prove result (ii). For a positive sequence ν_n, let

$${\Theta }^{\prime }=\left\{\mathit{\beta }\in {ℝ}^{Nqp}:\underset{1\le i\le N}{\sup }{\Vert {\mathit{\beta }}_i^{(r)}-{\widehat{\mathit{\beta }}}_i^{(r)or}\Vert }_2\le v_n,\underset{1\le j\le q}{\sup }{\Vert {\mathit{\beta }}_j^{(c)}-{\widehat{\mathit{\beta }}}_j^{(c)or}\Vert }_2\le v_n\right\},$$

$$Pe{n}^r(\mathit{\beta })={\gamma }_2\sum _{\left(i_1,i_2\right)\in {\Delta }^{(r)}}{\rho }_1\left({\Vert {\mathit{\beta }}_{i_1}^{(r)}-{\mathit{\beta }}_{i_2}^{(r)}\Vert }_2\right),Pe{n}^c(\mathit{\beta })={(N/q)}^{1/2}{\gamma }_2\sum _{\left(j_1,j_2\right)\in {\Delta }^{(c)}}{\rho }_2\left({\Vert {\mathit{\beta }}_{j_1}^{(c)}-{\mathit{\beta }}_{j_2}^{(c)}\Vert }_2\right),$$

and Pen(β) = Pen^r(β) + Pen^c(β). For β ∈ Θ^′ ∩ Θ, by Taylor’s expansion, we have

$$L(\mathit{\beta })-L(\tilde{\mathit{\beta }})={\Omega }_1+{\Omega }_2+{\Omega }_3,$$ (A.12)

where

$${\Omega }_1=\sum _{i=1}^N\sum _{j=1}^q{\left[-{\mathbf{U}}_{i,j}^{\top }\left({\mathbf{Y}}_{i,j}-{\mathbf{U}}_{i,j}{\overline{\mathit{\beta }}}_{i,j}\right)+{\gamma }_1\mathbf{D}{\overline{\mathit{\beta }}}_{i,j}\right]}^{\top }\left({\mathit{\beta }}_{i,j}-{\tilde{\mathit{\beta }}}_{i,j}\right),$$

$${\Omega }_2=\sum _{i=1}^N{\left({\frac{\partial Pe{n}^r(\overline{\mathit{\beta }})}{\partial {\mathit{\beta }}_i^{(r)}}|}_{{\mathit{\beta }}_i^{(r)}={\overline{\mathit{\beta }}}_i^{(r)}}\right)}^{\top }\left({\mathit{\beta }}_i^{(r)}-{\tilde{\mathit{\beta }}}_i^{(r)}\right),{\Omega }_3=\sum _{j=1}^q{\left({\frac{\partial Pe{n}^c(\overline{\mathit{\beta }})}{\partial {\mathit{\beta }}_j^{(c)}}|}_{{\mathit{\beta }}_j^{(c)}={\overline{\mathit{\beta }}}_j^{(c)}}\right)}^{\top }\left({\mathit{\beta }}_j^{(c)}-{\tilde{\mathit{\beta }}}_j^{(c)}\right),$$

with $\overline{\mathit{\beta }}={\left({\overline{\mathit{\beta }}}_{1,1}^{\top },\dots ,{\overline{\mathit{\beta }}}_{N,q}^{\top }\right)}^{\top }$ and ${\overline{\mathit{\beta }}}_{i,j}=s{\mathit{\beta }}_{i,j}+(1-s){\tilde{\mathit{\beta }}}_{i,j}$ for some s ∈ (0, 1).

Firstly, we have

$$\begin{array}{l}{\Omega }_2={\gamma }_2\sum _{i_1<i_2}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right){\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2^{-1}{\left({\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\right)}^{\top }\left({\mathit{\beta }}_{i_1}^{(r)}-{\tilde{\mathit{\beta }}}_{i_1}^{(r)}\right)\\ +{\gamma }_2\sum _{i_1>i_2}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right){\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2^{-1}{\left({\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\right)}^{\top }\left({\mathit{\beta }}_{i_1}^{(r)}-{\tilde{\mathit{\beta }}}_{i_1}^{(r)}\right)\\ ={\gamma }_2\sum _{i_1<i_2}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right){\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2^{-1}{\left({\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\right)}^{\top }\left[\left({\mathit{\beta }}_{i_1}^{(r)}-{\tilde{\mathit{\beta }}}_{i_1}^{(r)}\right)-\left({\beta }_{i_2}^{(r)}-{\tilde{\mathit{\beta }}}_{i_2}^{(r)}\right)\right].\end{array}$$

When i1, $i_2\in {\mathcal{G}}_{k_r}^{(r)}$, ${\tilde{\mathit{\beta }}}_{i_1}={\tilde{\mathit{\beta }}}_{i_2}$. Thus

$$\begin{array}{l}{\Omega }_2={\gamma }_2\sum _{k_r=1}^{K_r}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)},i_1<i_2}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right){\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2^{-1}{\left({\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\right)}^{\top }\left({\mathit{\beta }}_{i_1}^{(r)}-{\mathit{\beta }}_{i_2}^{(r)}\right)\\ +{\gamma }_2\sum _{k_r<k_r^{\prime }{i}_1\in {\mathcal{G}}_{k_r}^{(r)}{i}_2\in {\mathcal{G}}_{k_r^{\prime }}^{(r)}}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right){\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2^{-1}{\left({\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\right)}^{\top }\left[\left({\mathit{\beta }}_{i_1}^{(r)}-{\tilde{\mathit{\beta }}}_{i_1}^{(r)}\right)-\left({\mathit{\beta }}_{i_2}^{(r)}-{\tilde{\mathit{\beta }}}_{i_2}^{(r)}\right)\right].\end{array}$$

As shown in Theorem 1, ${\sup }_i{\Vert {\tilde{\mathit{\beta }}}_i^{(r)}-{\mathit{\beta }}_i^{(r)*}\Vert }_2^2={\sup }_{k_r}{\Vert {\mathit{\alpha }}_{k_r}^{(r)}-{\mathit{\alpha }}_{k_r}^{(r)*}\Vert }_2^2\le q{\psi }^2$. Since ${\overline{\mathit{\beta }}}_i^{(r)}=s{\mathit{\beta }}_i^{(r)}+(1-s){\tilde{\mathit{\beta }}}_i^{(r)}$${\sup }_i{\Vert {\overline{\mathit{\beta }}}_i^{(r)}-{\mathit{\beta }}_i^{(r)*}\Vert }_2\le s{q}^{1/2}\psi +(1-s)q^{1/2}\psi =q^{1/2}\psi$. For $k_r\ne k_r^{\prime }$, $i_1\in {\mathcal{G}}_{k_r}^{(r)}$, $i_2\in {\mathcal{G}}_{k_r^{\prime }}^{(r)}$, we have

$${\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\ge \underset{i_1\in {\mathcal{G}}_{k_r}^{(r)},i_2\in {\mathcal{G}}_{k_r^{\prime }}^{(r)}}{\min }{\Vert {\mathit{\beta }}_{i_1}^{(r)*}-{\mathit{\beta }}_{i_2}^{(r)*}\Vert }_2-2\underset{i}{\max }{\Vert {\overline{\mathit{\beta }}}_i^{(r)}-{\mathit{\beta }}_i^{(r)*}\Vert }_2\ge \frac{1}{2}{\left|{\mathcal{G}}_{min}^{(c)}\right|}^{1/2}{C}_2^{-1/2}b-2q^{1/2}\psi >a{\gamma }_2,$$

and thus ${\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right)=0$. Therefore,

$$\begin{array}{l}{\Omega }_2={\gamma }_2\sum _{k_r=1}^{K_r}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)},i_1<i_2}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right){\Vert {\mathit{\beta }}_{i_1}^{(r)}-{\mathit{\beta }}_{i_2}^{(r)}\Vert }_2\\ \ge {\gamma }_2\sum _{k_r=1}^{K_r}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)},i_1<i_2}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right)q^{-1/2}\sum _{j=1}^q{\Vert {\mathit{\beta }}_{i_1,j}-{\mathit{\beta }}_{i_2,j}\Vert }_2\\ ={\gamma }_2{q}^{-1/2}\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)},i_1<i_2}\sum _{j\in {\mathcal{G}}_{k_c}^{(c)}}{\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right){\Vert {\mathit{\beta }}_{i_1,j}-{\mathit{\beta }}_{i_2,j}\Vert }_2.\end{array}$$

Similarly to (A.11), ${\sup }_i{\Vert {\tilde{\mathit{\beta }}}_i^{(r)}-{\widehat{\mathit{\beta }}}_i^{(r)or}\Vert }_2\le v_n$ and ${\sup }_i{\Vert {\mathit{\beta }}_i^{(r)}-{\widehat{\mathit{\beta }}}_i^{(r)or}\Vert }_2\le v_n$. Then we have

$$\underset{i_1<i_2}{\sup }{\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\le 2\underset{i}{\sup }{\Vert {\overline{\mathit{\beta }}}_i^{(r)}-{\tilde{\mathit{\beta }}}_i^{(r)}\Vert }_2\le 2\underset{i}{\sup }{\Vert {\mathit{\beta }}_i^{(r)}-{\tilde{\mathit{\beta }}}_i^{(r)}\Vert }_2\le 2\left(\underset{i}{\sup }{\Vert {\mathit{\beta }}_i^{(r)}-{\widehat{\mathit{\beta }}}_i^{(r)or}\Vert }_2+\underset{i}{\sup }{\Vert {\tilde{\mathit{\beta }}}_i^{(r)}-{\widehat{\mathit{\beta }}}_i^{(r)or}\Vert }_2\right)\le 4v_n.$$

Hence ${\rho }_1^{\prime }\left({\Vert {\overline{\mathit{\beta }}}_{i_1}^{(r)}-{\overline{\mathit{\beta }}}_{i_2}^{(r)}\Vert }_2\right)\ge {\rho }_1^{\prime }\left(4v_n\right)$ by the concavity of ρ(·). As a result,

$${\Omega }_2\ge {\gamma }_2{q}^{-1/2}\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)},i_1<i_2}\sum _{j\in {\mathcal{G}}_{k_c}^{(c)}}{\rho }_1^{\prime }\left(4v_n\right){\Vert {\mathit{\beta }}_{i_1,j}-{\mathit{\beta }}_{i_2,j}\Vert }_2.$$ (A.13)

Next we consider Ω₃. Similarly to the derivation of (A.13), we can derive

$${\Omega }_3\ge {\gamma }_2{q}^{-1/2}\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{j_1,j_2\in {\mathcal{G}}_{k_c}^{(c)}{j}_1<j_2}\sum _{i\in {\mathcal{G}}_{k_r}^{(r)}}{\rho }_2^{\prime }\left(4v_n\right){\Vert {\mathit{\beta }}_{i,j_1}-{\mathit{\beta }}_{i,j_2}\Vert }_2.$$ (A.14)

Lastly for Ω₁, we have

$${\Omega }_1=-\sum _{i=1}^N\sum _{j=1}^q{\mathit{w}}_{i,j}^{\top }\left({\mathit{\beta }}_{i,j}-{\tilde{\mathit{\beta }}}_{i,j}\right)=-\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)}}\sum _{j_1,j_2\in {\mathcal{G}}_{k_c}^{(c)}}\frac{{\mathit{w}}_{i_1,j_1}^{\top }\left({\mathit{\beta }}_{i_1,j_1}-{\mathit{\beta }}_{i_2,j_2}\right)}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|},$$

and

$$\begin{array}{l}\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)}}\sum _{j_1,j_2\in {\mathcal{G}}_{k_c}^{(c)}}\frac{\left|{\mathit{w}}_{i_1,j_1}^{\top }\left({\mathit{\beta }}_{i_1,j_1}-{\mathit{\beta }}_{i_2,j_2}\right)\right|}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|}\le \underset{i,j}{\sup }{\Vert {\mathit{w}}_{i,j}\Vert }_2\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)}}\sum _{j_1,j_2\in {\mathcal{G}}_{k_c}^{(c)}}\frac{{\Vert {\mathit{\beta }}_{i_1,j_1}-{\mathit{\beta }}_{i_2,j_2}\Vert }_2}{\left|{\mathcal{G}}_{k_r,k_c}^{(r,c)}\right|}\\ \le 2\underset{i,j}{\sup }{\Vert {\mathit{w}}_{i,j}\Vert }_2\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)},i_1<i_2}\sum _{j\in {\mathcal{G}}_{k_c}^{(c)}}\frac{{\Vert {\mathit{\beta }}_{i_1,j}-{\mathit{\beta }}_{i_2,j}\Vert }_2}{\left|{\mathcal{G}}_{k_r}^{(r)}\right|}+2\underset{i,j}{\sup }{\Vert {\mathit{w}}_{i,j}\Vert }_2\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{j_1,j_2\in {\mathcal{G}}_{k_c}^{(c)}{j}_1<j_2}\sum _{i\in {\mathcal{G}}_{k_r}^{(r)}}\frac{{\Vert {\mathit{\beta }}_{i,j_1}-{\mathit{\beta }}_{i,j_2}\Vert }_2}{\left|{\mathcal{G}}_{k_c}^{(c)}\right|},\end{array}$$

where ${\mathit{w}}_{i,j}={\mathbf{U}}_{i,j}^{\top }\left({\mathbf{Y}}_{i,j}-{\mathbf{U}}_{i,j}{\overline{\mathit{\beta }}}_{i,j}\right)-{\gamma }_1\mathbf{D}{\overline{\mathit{\beta }}}_{i,j}$. Note that

$$\underset{i,j}{\sup }{\Vert {\mathit{w}}_{i,j}\Vert }_2\le \underset{i,j}{\sup }{\Vert {\mathbf{U}}_{i,j}^{\top }\left({\mathit{g}}_{i,j}^{*}-{\mathbf{U}}_{i,j}{\mathit{\beta }}_{i,j}^{*}\right)\Vert }_2+\underset{i,j}{\sup }{\Vert \left({\mathbf{U}}_{i,j}^{\top }{\mathbf{U}}_{i,j}+{\gamma }_1\mathbf{D}\right)\left({\mathit{\beta }}_{i,j}^{*}-{\overline{\mathit{\beta }}}_{i,j}\right)\Vert }_2+\underset{i,j}{\sup }{\Vert {\gamma }_1\mathbf{D}{\mathit{\beta }}_{i,j}^{*}\Vert }_2+\underset{i,j}{\sup }{\Vert {\mathbf{U}}_{i,j}^{\top }{ϵ}_{i,j}\Vert }_2.$$

By Lemma 1 ${\sup }_{i,j}{\Vert {\mathbf{U}}_{i,j}^{\top }\left({\mathit{g}}_{i,j}^{*}-{\mathbf{U}}_{i,j}{\mathit{\beta }}_{i,j}^{*}\right)\Vert }_2\le n_m{M}_1{M}_2{p}^{1/2}{J}^{-\kappa }$. Moreover, ${\sup }_{i,j}{\Vert \left({\mathbf{U}}_{i,j}^{\top }{\mathbf{U}}_{i,j}+{\gamma }_1\mathbf{D}\right)\left({\mathit{\beta }}_{i,j}^{*}-{\overline{\mathit{\beta }}}_{i,j}\right)\Vert }_2\le (n_m^{1/2}{p}^{1/2}{M}_1+{\gamma }_1\Vert \mathbf{D}{\Vert }_2)\psi ,{\sup }_{i,j}{\Vert {\gamma }_1\mathbf{D}{\mathit{\beta }}_{i,j}^{*}\Vert }_2\le p^{1/2}{\gamma }_1\Vert \mathbf{D}{\Vert }_2{\Vert {\mathit{\beta }}^{*}\Vert }_{\infty }$. With the Bonferroni’s inequality, Markov’s inequality, and Condition (C5), we have

$$\begin{array}{l}P\left(\underset{i,j}{\sup }{\Vert {\mathbf{U}}_{(i,j)}^{\top }{ϵ}_{i,j}\Vert }_2>2n_{i,j}{F}^{-1}{M}_1{p}^{1/2}\log (Nq)\right)\le \sum _{i=1}^N\sum _{j=1}^{q}P\left({\Vert {\mathbf{U}}_{(i,j)}^{\top }{ϵ}_{i,j}\Vert }_2>2n_{i,j}{F}^{-1}{M}_1{p}^{1/2}\log (Nq)\right)\\ \le \sum _{i=1}^N\sum _{j=1}^{q}P\left(F{n}_{i,j}^{-1/2}{\Vert {ϵ}_{i,j}\Vert }_2>2\log (Nq)\right)\le c_2/(Nq).\end{array}$$

Together with Conditions (C1) and (C3), we have

$$\underset{i,j}{\sup }{\Vert {\mathit{w}}_{i,j}\Vert }_2=O\left(p^{1/2}\log (Nq)\right)$$ (A.15)

holds with probability at least 1 − c₂/(Nq). Let ν_n = o(1), then ${\rho }_1^{\prime }\left(4v_n\right)\to 1$ and ${\rho }_2^{\prime }\left(4v_n\right)\to 1$. Since ${\gamma }_2\gg {(pq)}^{1/2}\log (Nq)/\min \left\{\left|{\mathcal{G}}_{min}^{(r)}\right|,\left|{\mathcal{G}}_{min}^{(c)}\right|\right\}$, then by (A.12)–(A.15)

$$\begin{gathered} L(\mathit{\beta })-L(\tilde{\mathit{\beta }})={\Omega }_1+{\Omega }_2+{\Omega }_3\ge \sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{i_1,i_2\in {\mathcal{G}}_{k_r}^{(r)},i_1<i_2}\sum _{j\in {\mathcal{G}}_{k_c}^{(c)}}\left[{\gamma }_2{q}^{-1/2}{\rho }_1^{\prime }\left(4v_n\right)-\frac{2\underset{i,j}{\sup }{\Vert {\mathit{w}}_{i,j}\Vert }_2}{\left|{\mathcal{G}}_{k_r}^{(r)}\right|}\right]{\Vert {\mathit{\beta }}_{i_1,j}-{\mathit{\beta }}_{i_2,j}\Vert }_2 \\ +\sum _{k_r=1}^{K_r}\sum _{k_c=1}^{K_c}\sum _{j_1,j_2\in {\mathcal{G}}_{k_c}^{(c)}{j}_1<j_2}\sum _{i\in {\mathcal{G}}_{k_r}^{(r)}}[{\gamma }_2{q}^{-1/2}{\rho }_2^{\prime }\left(4v_n\right)-\frac{2\underset{i,j}{\sup }{\Vert {\mathit{w}}_{i,j}\Vert }_2}{\left|{\mathcal{G}}_{k_c}^{(c)}\right|}]{\Vert {\mathit{\beta }}_{i,j_1}-{\mathit{\beta }}_{i,j_2}\Vert }_2\ge 0 \end{gathered}$$

holds with probability at least 1 − c₂/(Nq), which completes the proof of result (ii). □