Estimation of multiple networks with common structures in heterogeneous subgroups

Abstract

Network estimation has been a critical component of high-dimensional data analysis and can provide an understanding of the underlying complex dependence structures. Among the existing studies, Gaussian graphical models have been highly popular. However, they still have limitations due to the homogeneous distribution assumption and the fact that they are only applicable to small-scale data. For example, cancers have various levels of unknown heterogeneity, and biological networks, which include thousands of molecular components, often differ across subgroups while also sharing some commonalities. In this article, we propose a new joint estimation approach for multiple networks with unknown sample heterogeneity, by decomposing the Gaussian graphical model (GGM) into a collection of sparse regression problems. A reparameterization technique and a composite minimax concave penalty are introduced to effectively accommodate the specific and common information across the networks of multiple subgroups, making the proposed estimator significantly advancing from the existing heterogeneity network analysis based on the regularized likelihood of GGM directly and enjoying scale-invariant, tuning-insensitive, and optimization convexity properties. The proposed analysis can be effectively realized using parallel computing. The estimation and selection consistency properties are rigorously established. The proposed approach allows the theoretical studies to focus on independent network estimation only and has the significant advantage of being both theoretically and computationally applicable to large-scale data. Extensive numerical experiments with simulated data and the TCGA breast cancer data demonstrate the prominent performance of the proposed approach in both subgroup and network identifications.

1. Introduction

Many modern applications often involve analyzing a network structure for a set of high-dimensional variables. For example, biological networks specific to disease contexts explore the patterns of association in molecular data (e.g., genes, proteins, etc.) and play a critical role in understanding the underlying biological progresses [4, 16, 30]. Several statistical methods have been developed for network estimation. Among them, Gaussian graphical models have been widely employed, where the precision matrix describes the conditional dependencies between variables [8, 38]. To capture sparsity patterns in the precision matrix, one strategy is to conduct estimation based on the penalized likelihood function [1, 12]. Another strategy is to reduce the estimation of the precision matrix to a collection of sparse regression problems [26], which is not only easy to optimize but also more amenable to theoretical analysis. Recent methodological developments include CLIME [6], Tiger [22], SCIO [24], and scaled-Lasso [34], among which Tiger is computationally the most efficient due to its square root loss function and asymptotic tuning-free property.

Despite considerable successes, these studies are limited to homogeneous analysis and assume that high-dimensional data has only a single network. In practical applications, it is common that observed data comes from multiple subgroups and has heterogeneous dependencies. For example, in biomedical studies, cancer heterogeneity has received extensive attention, and the biological networks often differ across subtypes of the same cancer while also having certain commonalities. To accommodate both the specific and common information among multiple networks, based on the Gaussian graphical models, significant progresses have been made recently using the group regularization techniques, including the more popular likelihood-based strategies [2, 7, 14, 28] and relatively limited investigations from a sparse linear regression perspective in a column-by-column fashion [25].

However, all the aforementioned approaches rely on prior known subgroup memberships, which are usually not available for practical high-dimensional data with complex and unknown group structures. To accommodate the unknown heterogeneity, the Gaussian mixture model serves as a suitable choice for simultaneously conducting subgroup and network identifications. To this end, [13] develops a joint graphical lasso penalty on multiple precision matrices to extract both homogeneous and heterogeneous information across all subgroups. [15] takes a further step, developing an efficient ECM algorithm and establishing non-asymptotic statistical properties for the estimated networks. Recently, [29] develops a penalized fusion method for heterogeneity analysis based on the Gaussian mixture model, which has the advantage of being capable of automatically determining the number of subgroups. The aforementioned approaches are based on likelihood regularization strategies and are often only applicable to small-scale data with dimensions less than 100.

In this study, we develop a new joint estimation approach for multiple networks with unknown sample heterogeneity. Based on the Gaussian mixture model, we propose estimating multiple precision matrices in a column-by-column manner with a reparameterization technique and introduce a composite minimax concave penalty (MCP) to effectively accommodate both specific and common structures of networks. This is significantly different from the existing heterogeneous network estimation publications [15, 29] (which directly maximize a penalized likelihood with respect to large matrices) and has the advantage of automatically imposing an adaptive penalty on the parameters, resulting in scale-invariant estimators and potential tuning-insensitive properties. This study is much more than an extension of the homogeneous column-wise Gaussian graphical model [22, 25]. Specifically, significantly advancing from [22] which is based on the square root loss function, the proposed approach is based on ordinary square loss, enjoying simplicity and achieving optimization convexity. In addition, significantly advancing from [25], the proposed approach can provide equivalent estimates under scaling in mixture models, which is much more important than in homogeneous analysis. The proposed approach allows theoretical studies to focus on independent network estimation only (disregarding the diagonal elements of precision matrices) and can take advantage of parallel computation to achieve more efficiency, making it theoretically and computationally feasible for large-scale data. Overall, this study provides a practically useful tool for the joint estimation research of multiple networks from heterogeneous subgroups.

2. Methods

Suppose that there are $n$ independent subjects, which come from $K$ subgroups. For the ith subject, denote ${\mathit{x}}_i={\left(x_{i,1},\dots ,x_{i,p}\right)}^{\mathrm{\top }}$ as the p-dimensional vector of predictor measurements and $C_i\in \{1,\dots ,K\}$ as the subgroup assignment. Assume that

$$x_i\mid C_i=k\sim {𝒩}_p\left({\mathit{\mu }}_k,{\Omega }_k^{-1}\right),$$ (1)

where ${\mathit{\mu }}_k={\left({\mu }_{k,1},\dots ,{\mu }_{k,p}\right)}^{\mathrm{\top }}$ and ${\mathbf{\Omega }}_k={\left({\omega }_{k,\ell j}\right)}_{p\times p}$ are the mean vector and precision matrix for the kth subgroup, respectively. This Gaussian graphical model has been popular in existing network analysis publications. For the kth subgroup, the relationships between the predictors in the network are measured using the subgroup-specific precision matrix ${\mathbf{\Omega }}_k$, which describes the conditional independence between any two predictors given the rest. Specifically, if ${\omega }_{k,\ell j}\ne 0$, there is a connection between predictors $\ell$ and $j$ for the kth subgroup, and otherwise if ${\omega }_{k,\ell j}=0$.

Based on (1), when $C_i=k$, the conditional distribution of $x_{i,j}$ given ${\mathit{x}}_{i,\backslash j}={\left\{x_{i,\ell }:\ell \ne j\right\}}_{(p-1)\times 1}$ is

$$x_{i,j}\mid {\mathit{x}}_{i,\backslash j};C_i=k\sim 𝒩\left({\mu }_{k,j}-\sum _{\ell \ne j}{\omega }_{k,j\ell }/{\omega }_{k,jj}\left(x_{i,\ell }-{\mu }_{k,\ell }\right),1/{\omega }_{k,jj}\right).$$ (2)

For $\ell \ne j$, define

$${\ddot{\beta }}_{k,j,\ell }:=-{\omega }_{k,j\ell }/{\omega }_{k,jj},{\ddot{\beta }}_{k,j,j}:={\mu }_{k,j}-\sum _{\ell \ne j}{\ddot{\beta }}_{k,j,\ell }{\mu }_{k,\ell },\text{and}{\sigma }_{k,j}^2:=1/{\omega }_{k,jj}.$$ (3)

Then (2) can be rewritten as:

$$x_{i,j}\mid {\mathit{x}}_{i,\backslash j};C_i=k\sim 𝒩\left({\ddot{\beta }}_{k,j,j}+{\mathit{x}}_{i,\backslash j}^{\top }{\ddot{\mathit{\beta }}}_{k,j,\backslash j},{\sigma }_{k,j}^2\right),$$ (4)

where ${\ddot{\mathit{\beta }}}_{k,j,\backslash j}={\left\{{\ddot{\beta }}_{k,j,\ell }:\ell \ne j\right\}}_{(p-1)\times 1}$. Based on (3), estimation of ${\mathbf{\Omega }}_k$’s can be achieved by considering the conditional distribution (4) for $j\in \{1,\dots ,p\}$, where ${\omega }_{k,j\ell }\ne 0$ corresponds to ${\ddot{\beta }}_{k,j,\ell }\ne 0$ for $\ell \ne j$.

In practice, the subgroup assignments $C_i$’s are not always observed. To accommodate the unknown sample heterogeneity, we consider the following conditional Gaussian mixture model:

$$x_{i,j}\mid {\mathit{x}}_{i,\backslash j}\sim \sum _{k=1}^K{\pi }_k𝒩\left({\ddot{\beta }}_{k,j,j}+{\mathit{x}}_{i,\backslash j}^{\top }{\ddot{\mathit{\beta }}}_{k,j,\backslash j},{\sigma }_{k,j}^2\right),$$

where ${\pi }_k=\mathit{Pr}\left(C_i=k\right)$ is the probability that a subject belongs to the $k$th subgroup with ${\sum }_{k=1}^K{\pi }_k=1$. For more effective heterogeneous network estimation while accommodating the common structure across the $K$ networks, we consider reparameterization for each $k\in \{1,\dots ,K\}$ and $j,\ell \in \{1,\dots ,p\}$:

$${\tau }_{k,j}=1/{\sigma }_{k,j},{\beta }_{k,j,\ell }={\ddot{\beta }}_{k,j,\ell }/{\sigma }_{k,j},$$ (5)

and propose the following penalized objective function for each $j\in \{1,\dots ,p\}$:

$$L_n\left({\theta }_j\right)=\sum _{i=1}^n\text{ln}\left\{\sum _{k=1}^K{\pi }_{k}g\left(x_{i,j};x_{i,\backslash j}^{\top }{\beta }_{k,j,\backslash j}+{\beta }_{k,j,j},{\tau }_{k,j}\right)\right\}-n\sum _{\ell \ne j}\rho \left\{\sum _{k=1}^K\rho \left(\left|{\beta }_{k,j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\},$$ (6)

where ${\mathit{\theta }}_j=\mathrm{v}\mathrm{e}\mathrm{c}\left({\pi }_1,\dots ,{\pi }_K,{\mathit{\alpha }}_j\right)$ is the vector of the overall unknown parameters in the jth subproblem with ${\mathit{\alpha }}_j=\mathrm{vec}\left({\tau }_{1,j},\dots ,{\tau }_{K,j},{\mathit{\beta }}_j\right)$ and ${\mathit{\beta }}_j=\mathrm{vec}\left({\beta }_{1,j,j},{\mathit{\beta }}_{1,j,\backslash j},\dots ,{\beta }_{K,j,j},{\mathit{\beta }}_{K,j,\backslash j}\right),g\left(x_{i,j};{\mathit{x}}_{i,\backslash j}^{\mathrm{\top }}{\mathit{\beta }}_{k,j,\backslash j}+{\beta }_{k,j,j},{\tau }_{k,j}\right)=\mathrm{l}\mathrm{n}{\tau }_{k,j}-\left({\tau }_{k,j}{x}_{i,j}\right.{\left.-{\beta }_{k,j,j}-{\mathit{x}}_{i,\backslash j}^{\mathrm{\top }}{\mathit{\beta }}_{k,j,\backslash j}\right)}^2/2$, and $\rho (v;\lambda ,\gamma )=\lambda {\int }_0^v{\left\{1-(\lambda \gamma {)}^{-1}x\right\}}_{+}dx$ is the minimax concave penalty (MCP) with tuning parameter $\lambda$ and regularization parameter $\gamma$. Here, following the existing literature [3], the tuning parameter of the outer penalty is chosen to be $K\lambda \gamma /2$, ensuring that the group-level penalty reaches a maximum if and only if each of its components is at its maximum.

The proposed estimation procedure has been motivated by the following considerations. In (6), the first term is a reparameterized version of the log-likelihood function based on the distribution of $x_{i,j}$ given $x_{i,\backslash j}$. Different from the studies that estimate the precision matrix based on the joint distribution of ${\mathit{x}}_i$ directly, the proposed conditional distribution-based strategy can reduce the estimation problem of a large matrix to a collection of sparse linear regression problems, enjoying not only great computational efficiency but also satisfactory theoretical properties. Specifically, as demonstrated in [33], the scale-invariance considerations and theoretical results indicate the necessity of taking penalty levels proportional to noise levels in high-dimensional problems. Thus, taking advantage of the column-wise strategy, we utilize reparameterization to automatically scale the penalty to the noise level and make the proposed estimator scale-invariant [32, 42]. Different from the existing heterogeneous network analysis studies [13, 15, 29], such a strategy achieves “adaptive adjustment” of the coefficients based on the data characteristics, resulting in a more accurate estimation. The proposed model is closely related to the scaled sparse linear regression [33], which does not require any knowledge of noise and has comparable theoretical properties to studies under known noise. In contrast, penalizing ${\ddot{\beta }}_{k,j,\ell }$’s directly on the basis of (4) leads to an indirect effect on the estimation of the scale parameters ${\sigma }_{k,j}$’s and non-convex optimization, which may result in serious consequences in mixture model analysis. Additionally, such a reparameterization strategy enjoys the advantage that the identification and estimation results are insensitive to the choice of tuning parameters, which simplifies the tuning procedure. In the second term, we apply the composite MCP [3] to capture heterogeneity and homogeneity across all $K$ subgroups. According to the relationship between ${\beta }_{k,j,\ell }$ and ${\omega }_{k,j\ell }$, the outer MCP encourages the $K$ estimated precision matrices to share a similar pattern of sparsity to accommodate the potential common network structure across the subgroups, where ${\beta }_{1,j,\ell },\dots ,{\beta }_{K,j,\ell }$ will be simultaneously shrunk to zero or not. On the other hand, the inner MCP imposed on ${\beta }_{k,j,\ell }$’s amounts to further sparsing each precision matrix to accommodate the specific network structure across the subgroups. Here, different from the sparse group lasso penalty commonly used in the literature [13, 15], the adopted composite MCP improves accuracy and interpretability of the model by allowing important relationships to have large coefficients without overshrinking. As our primary goal is to estimate subgroup-specific precision matrices and investigate heterogeneous networks, the intercept terms ${\beta }_{k,j,j}$’s are not subject to penalization. The model can be easily extended to accommodate sparse learning for the subgroup means as well.

2.1. Computation

We develop an Expectation Maximization (EM) algorithm to optimize objective function (6). Specifically, we introduce a latent variable $z_{ki}=I\left\{C_i=k\right\}$, where $I\{\cdot \}$ is the indicator function, and then consider the following complete objective function:

$$Q_n\left({\mathit{\theta }}_j\right)=\sum _{i=1}^n\sum _{k=1}^K{z}_{ki}\text{ln}\left\{{\pi }_{k}g\left(x_{i,j};{\mathit{x}}_{i,\backslash j}^{\top }{\mathit{\beta }}_{k,j,\backslash j}+{\beta }_{k,j,j},{\tau }_{k,j}\right)\right\}-n\sum _{\ell \ne j}\rho \left\{\sum _{k=1}^K\rho \left(\left|{\beta }_{k,j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\}.$$ (7)

Denote $\mathbf{\Psi }=\mathrm{vec}\left({\mathit{\mu }}_1,\dots ,{\mathit{\mu }}_K,{\mathbf{\Omega }}_1,\dots ,{\mathbf{\Omega }}_K,{\pi }_1,\dots ,{\pi }_K\right)$ and $\mathit{\alpha }=\mathrm{v}\mathrm{e}\mathrm{c}\left({\mathit{\alpha }}_1,\dots ,{\mathit{\alpha }}_p\right)$. The computation proceeds with the following steps.

1. Initialization: Set iteration time $t=0$. For each $k\in \{1,\dots ,K\}$, initialize ${\pi }_k^{\left(0\right)}=1/K$ and ${\mathbf{\Omega }}_k^{\left(0\right)}={\mathit{I}}_p$, where ${\mathit{I}}_p$ is an identity matrix of dimension $p$. Randomly divide the subjects into $K$ subgroups of an equal size. For each $k\in \{1,\dots ,K\}$, take the mean of each subgroup as the initial value ${\mathit{\mu }}_k^{\left(0\right)}$.

2. In the $t$th E step, the conditional expectation of (7) with respect to ${\mathbf{\Psi }}^{(t-1)}$ obtained at the $(t-1)$th iteration is

   $${\text{E}}_{{\Psi }^{(t-1)}}\left\{Q_n\left({\theta }_j\right)\right\}=\sum _{i=1}^n\sum _{k=1}^K{\rho }_{ki}^{(t)}\text{ln}\left\{{\pi }_{k}g\left(x_{i,j};x_{i,\backslash j}^{\top }{\beta }_{k,j,\backslash j}+{\beta }_{k,j,j},{\tau }_{k,j}\right)\right\}-n\sum _{\ell \ne j}\rho \left\{\sum _{k=1}^K\rho \left(\left|{\beta }_{k,j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\}$$ (8)

   for $j\in \{1,\dots ,p\}$. Here ${\rho }_{ki}^{\left(t\right)}=Pr\left(z_{ki}=1\mid {\mathit{x}}_i;{\mathbf{\Psi }}^{(t-1)}\right)={\pi }_k^{(t-1)}{f}_k\left({\mathit{x}}_i;{\mathit{\mu }}_k^{(t-1)},{\mathbf{\Omega }}_k^{(t-1)}\right)/\left\{{\sum }_{\ell =1}^K{\pi }_{\ell }^{(t-1)}{f}_{\ell }\left({\mathit{x}}_i;{\mathit{\mu }}_l^{(t-1)},{\mathbf{\Omega }}_{\ell }^{(t-1)}\right)\right\}$ with $f_k$ being a multivariate Gaussian density function with a subgroup-specific mean vector ${\mathit{\mu }}_k^{(t-1)}$ and precision matrix ${\mathbf{\Omega }}_k^{(t-1)}$, where ${\mathit{\mu }}_k^{(t-1)}$ and ${\mathbf{\Omega }}_k^{(t-1)}$ are computed based on the estimation of $\mathit{\alpha }$ at the $(t-1)$th iteration and equations (3) and (5) (see the supplementary materials for more details). Here, with the consideration of the symmetry of ${\mathbf{\Omega }}_k$’s, following [22], we compute ${\omega }_{k,\ell j}^{(t-1)}=\left({\omega }_{k,\ell j}^{(t-1)}+{\omega }_{k,j\ell }^{(t-1)}\right)/2$. To further ensure the positive definiteness of the precision matrices, a small diagonal matrix is added to ${\mathbf{\Omega }}_k$’s [25].
3. In the tth M step,

   • Maximize (8) with respect to ${\pi }_k$ and compute ${\pi }_k^{\left(t\right)}={\sum }_{i=1}^n{\rho }_{ki}^{\left(t\right)}/n$.
   • Maximize (8) with respect to ${\mathit{\alpha }}_j$. For each $j\in \{1,\dots ,p\}$, the following two steps are carried out sequentially,
     (1) With the other parameters fixed, optimizing $Q_n\left({\mathit{\theta }}_j\right)$ with respect to ${\tau }_{k,j}$ for each $k\in \{1,\dots ,K\}$ yields

     $${\tau }_{k,j}^{(t)}=\text{arg}\underset{{\tau }_{k,j}}{\text{min}}\left\{\frac{{\sum }_{i=1}^n{\rho }_{ki}^{(t)}{\left({\tau }_{k,j}{x}_{i,j}-{\beta }_{k,j,j}^{(t-1)}-{\mathit{x}}_{i,\backslash j}^{\top }{\mathit{\beta }}_{k,j,\backslash j}^{(t-1)}\right)}^2}{2}-\sum _{i=1}^n{\rho }_{ki}^{(t)}\text{ln}{\tau }_{k,j}\right\}=\frac{{\tilde{b}}_k^{(t)}+\sqrt{{\left({\tilde{b}}_k^{(t)}\right)}^2+4n{\tilde{a}}_k^{(t)}{\pi }_k^{(t)}}}{2{\tilde{a}}_k^{(t)}},$$

     where ${\tilde{a}}_k^{\left(t\right)}={\sum }_{i=1}^n{\rho }_{ki}^{\left(t\right)}{x}_{i,j}^2$ and ${\tilde{b}}_k^{\left(t\right)}={\sum }_{i=1}^n{\rho }_{ki}^{\left(t\right)}\left({\mathit{x}}_{i,\backslash j}^{\mathrm{\top }}{\mathit{\beta }}_{k,j,\backslash j}^{(t-1)}+{\beta }_{k,j,j}^{(t-1)}\right)x_{i,j}$.
     (2) With the other parameters fixed, optimizing $Q_n\left({\mathit{\theta }}_j\right)$ with respect to ${\mathit{\beta }}_j$ yields that

     $${\mathit{\beta }}_j^{(t)}=\text{arg}\underset{{\mathit{\beta }}_j}{\text{min}}\left[\sum _{i=1}^n\sum _{k=1}^K\frac{{\rho }_{ki}^{(t)}{\left({\tau }_{k,j}^{(t)}{x}_{i,j}-{\beta }_{k,j,j}-{\mathit{x}}_{i,\backslash j}^{\top }{\mathit{\beta }}_{k,j,\backslash j}\right)}^2}{2}+n\sum _{\ell \ne j}\rho \left\{\sum _{k=1}^K\rho \left(\left|{\beta }_{k,j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\}\right],$$ (9)

     which is a weighted linear regression problem with composite MCP and can be realized using the R package grpreg [3].

By iterating the E and M steps, convergence is achieved usually within a moderate number of iterations, which is concluded in our numerical study if ${\sum }_{k=1}^K\left\{{∥{\mathit{\mu }}_k^{\left(t\right)}-{\mathit{\mu }}_k^{(t-1)}∥}_2/{∥{\mathit{\mu }}_k^{\left(t\right)}∥}_2+{∥{\mathbf{\Omega }}_k^{\left(t\right)}-{\mathbf{\Omega }}_k^{(t-1)}∥}_F/{∥{\mathbf{\Omega }}_k^{\left(t\right)}∥}_F\right\}\le 0.01$ with ${∥{\mathbf{\Omega }}_k^{\left(t\right)}∥}_F=\sqrt{{\sum }_{i,j}{\left({\omega }_{k,ij}^{\left(t\right)}\right)}^2}$ being the Frobenius norm of ${\mathbf{\Omega }}_k^{\left(t\right)}$. To improve performance of the proposed EM algorithm, following published literature [42], we consider multiple random initializations of subjects’ subgroup memberships and choose the estimate with the smallest AIC defined below as the final estimate.

Following [3], we set $\gamma =3$ in the proposed algorithm. For $\lambda$, following [7], we adopt the AIC criterion defined as $-2{\sum }_{i=1}^n\mathrm{l}\mathrm{n}\left\{{\sum }_{k=1}^K{\hat{\pi }}_k{f}_k\left({\mathit{x}}_i;{\stackrel{\mathit{ˆ}}{\mathit{\mu }}}_k,{\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k\right)\right\}+{\sum }_{k=1}^{K}2{\hat{s}}_k$, where ${\hat{\pi }}_k$’s, ${\stackrel{\mathit{ˆ}}{\mathit{\mu }}}_k$’s, and ${\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k$’s are the final updates from the proposed algorithm, and ${\hat{s}}_k=\#\left\{(\ell ,j):{\hat{\omega }}_{k,\ell j}\ne \mathrm{0,1}\le \ell <j\le p\right\}$. Given fixed tuning parameters, in each iteration of the proposed EM algorithm, for each $j\in \{1,\dots ,p\}$, updating the regression parameters ${\mathit{\beta }}_j$ costs at most $O\left\{nK\right(p-1\left)\right\}$ operations, resulting in a complete cost up to $O\left\{nKp\right(p-1\left)\right\}$ operations. In comparison, the existing popular SCAN approach [15], which conducts heterogeneous network analysis based on the Gaussian mixture model and EM algorithm, has a computational complexity of $O\left\{nK{p}^2+K{p}^3\right\}$ in each iteration of the EM algorithm. The proposed approach is computationally more feasible for large datasets with $p>n$. Since the optimization of (8) can be realized separately for $j\in \{1,\dots ,p\}$, we develop a parallel computing strategy for the EM algorithm to further improve efficiency. The R package MultiNet that implements the proposed approach in a parallel manner is available at https://github.com/mengyunwu2020/MultiNet.

3. Statistical Properties

For a set $𝒮$, denote ${𝒮}^c$ and $\left|𝒮\right|$ as its complement and cardinality, respectively. For a vector $\mathit{v}$, denote ${\mathit{v}}_{𝒮}$ as the component of $\mathit{v}$ indexed by $𝒮$. For a matrix $\mathit{M}={\left(M_{ij}\right)}_{p_1\times p_2}$, denote ${\mathit{M}}_{{𝒮}_1,{𝒮}_2}$ as the submatrix of $\mathit{M}$ indexed by ${𝒮}_1$ and ${𝒮}_2$ and $\parallel \mathit{M}{\parallel }_{\mathrm{\infty }}={\mathrm{m}\mathrm{a}\mathrm{x}}_{i\in \left\{1,\dots ,p_1\right\}}{\sum }_{j=1}^{p_2}\left|M_{ij}\right|$ as the maximum induced norm of $\mathit{M}$.

Denote ${\mathbf{\Omega }}_k^0$ as the true precision matrix for the $k$th subgroup and ${\mathit{E}}_k^0=\left\{(\ell ,j):1\le \ell \ne j\le p,{\omega }_{k,\ell j}^0\ne 0\right\}$ as the set of the nonzero off-diagonal elements of ${\mathbf{\Omega }}_k^0$. For the jth subproblem, let the vector of the true parameters be ${\mathit{\theta }}_j^0=\mathrm{vec}\left({\pi }_1^0,\dots ,{\pi }_K^0,{\tau }_{1,j}^0,\dots ,{\tau }_{K,j}^0,{\beta }_{1,j,j}^0,\dots ,{\beta }_{K,j,j}^0,{\mathit{\beta }}_{1,j,\backslash j}^0,\dots ,{\mathit{\beta }}_{K,j,\backslash j}^0\right)$, and denote $C_j=\left\{q:{\theta }_{j,q}^0\ne 0\right\}$ as the non-zero element set, where ${\theta }_{j,q}^0$ is the $q$th element of ${\mathit{\theta }}_j^0$. In addition, for the $j$th subproblem and $k$th subgroup, denote ${𝒮}_{j,k}=\left\{\ell :{\beta }_{k,j,\ell }^0\ne 0\right.$ and $\left.\ell \ne j\right\}=\left\{\ell :{\omega }_{k,j\ell }^0\ne 0\right.$ and $\left.\ell \ne j\right\}$. Let $s_0={\mathrm{m}\mathrm{a}\mathrm{x}}_{j\in \{1,\dots ,p\},k\in \{1,\dots ,K\}}\left|{𝒮}_{j,k}\right|$.

Let ${\mathit{\theta }}_{j,C_j}^{\mathrm{*}}=\mathrm{v}\mathrm{e}\mathrm{c}\left({\pi }_1^{\mathrm{*}},\dots ,{\pi }_K^{\mathrm{*}},{\tau }_{1,j}^{\mathrm{*}},\dots ,{\tau }_{K,j}^{\mathrm{*}},{\beta }_{1,j,j}^{\mathrm{*}},\dots ,{\beta }_{K,j,j}^{\mathrm{*}},{\mathit{\beta }}_{1,j,{𝒮}_{j,1}}^{\mathrm{*}},\dots ,{\mathit{\beta }}_{K,j,{𝒮}_{j,K}}^{\mathrm{*}}\right)$ be the maximizer of the oracle counterpart of objective function (6) defined as:

$${\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)=\sum _{i=1}^n\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)=\sum _{i=1}^n\text{ln}\left\{\sum _{k=1}^K{\pi }_{k}g\left(x_{i,j};{\mathit{x}}_{i,{𝒮}_{j,k}}^{\top }{\mathit{\beta }}_{k,j,{𝒮}_{j,k}}+{\beta }_{k,j,j},{\tau }_{k,j}\right)\right\},$$

where $f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)={\sum }_{k=1}^K{\pi }_{k}g\left(x_{i,j};{\mathit{x}}_{i,{𝒮}_{j,k}^{\mathrm{\top }}}^{\mathrm{\top }}{\mathit{\beta }}_{k,j,{𝒮}_{j,k}}+{\beta }_{k,j,j},{\tau }_{k,j}\right)$.

To establish the theoretical results of the proposed approach in terms of identification and estimation, we need the following technical assumptions.

Assumption 1. The probability density function $f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)$ has a common support and is identifiable with respect to ${\mathit{\theta }}_j$. Moreover, the first and second derivatives of $\mathrm{l}\mathrm{n}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)$ satisfy that, for any $q,\ell \in \{1,\cdots ,K(p+2\left)\right\}$,

$$\begin{gathered} E\left\{{\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)}{\partial {\theta }_{j,q}}|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}\right\}=0, \\ E\left\{{\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)}{\partial {\theta }_{j,q}}\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)}{\partial {\theta }_{j,\ell }}|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}\right\}=E\left\{-{\frac{{\partial }^2\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)}{\partial {\theta }_{j,q}\partial {\theta }_{j,\ell }}|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}\right\}. \end{gathered}$$

Assumption 2. The Fisher information matrix

$$ℐ\left({\mathit{\theta }}_{j,C_j}\right)=E\left[\left\{\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}\right\}{\left\{\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}\right\}}^{\top }\right]$$

is finite and positive definite at ${\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0$. Furthermore, there exist finite constants $m_1$ and $m_2$ such that for any $q,\ell \in C_j$,

$$E\left\{{\frac{{\partial }^2\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{{\left(\partial {\theta }_{j,q}\right)}^2}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}\right\}<m_1,E\left[{{\left\{\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\theta }_{j,\ell }}\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\theta }_{j,q}}\right\}}^2|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}\right]<m_2.$$

Assumption 3. There exists an open set ${𝒩}_0$ containing the true parameter ${\mathit{\theta }}_j^0$ and satisfies that, for almost all ${\mathit{v}}_i=\mathrm{vec}\left(x_{i,j},{\mathit{x}}_{i,\backslash j}\right)$ and ${\mathit{\theta }}_j\in {𝒩}_0$, the density $f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)$ admits all third derivatives with respect to ${\mathit{\theta }}_j$. Moreover, there exist integrable functions $M_1\left({\mathit{v}}_i\right)$ and $M_2\left({\mathit{v}}_i\right)$, such that for any ${\mathit{\theta }}_j\in {𝒩}_0$ and $q,\ell ,m\in \{1,\cdots ,K(p+2\left)\right\}$,

$$\left|\frac{{\partial }^2}{\partial {\theta }_{j,q}\partial {\theta }_{j,\ell }}\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)\right|\le M_1\left({\mathit{v}}_i\right),\left|\frac{{\partial }^3}{\partial {\theta }_{j,q}\partial {\theta }_{j,\ell }\partial {\theta }_{j,m}}\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)\right|\le M_2\left({\mathit{v}}_i\right).$$

Assumption 4. For some constant $b\in (\mathrm{0,1}/4),K=O\left(n^b\right)$ and $s_0=o\left(n^{1/4-b}\right)$.

Assumption 5. $K{s}_0/\left(\sqrt{n}{\lambda }^2\right)\to 0$ and $n^{a/2-1/2}\sqrt{\mathrm{l}\mathrm{n}n}/{\lambda }^2\to 0$, when $n\to \mathrm{\infty }$, where $a\in (\mathrm{0,1}/2)$.

Assumption 6. ${\mathrm{m}\mathrm{i}\mathrm{n}}_{k\in \{1,\dots ,K\},j\in \{1,\dots ,p\}}\left\{\left|{\beta }_{k,j,\ell }^0\right|:\ell \in {𝒮}_{j,k}\right\}>\gamma \lambda$, and ${\theta }_{j,q}^0$ is bounded for each $j\in \{1,\dots ,p\}$ and $q\in$ $\{1,\dots ,K(p+2\left)\right\}$.

Assumption 7. $\mathrm{l}\mathrm{n}\left(p\right)=O\left(n^a\right)$.

Assumptions 1–3 impose conditions on the mixture distribution and have been commonly assumed in the field of mixture modeling [18, 20, 42]. Assumption 4 makes a constraint on the sparsity parameter $s_0$ and allows the number of subgroups $K$ to grow slowly with $n$. It suggests that, for each sub-regression problem, as long as the number of non-zero parameters (i.e., $\left(s_{0}K\right)$) is equal, the required sample size $n$ is the same. Similar conditions have also been assumed in published multiple network analysis studies, such as Condition (C1) and assumptions in Proposition 1 in [5]. Assumption 5 restricts the rate of $\lambda$ relative to the sample size. A similar condition is often assumed in high-dimensional studies [9, 42]. Assumption 6 imposes constraints on the true parameters, where the first subcondition restricts the rate by which the nonzero coefficients can be distinguished from zero, and the other one restricts the true parameters to a bounded range, which are also considered in the literature [9, 15]. Assumption 7 allows the dimension $p$ to grow exponentially.

Theorem 1. Under Assumptions 1–4, there exists a strict local maximizer ${\mathit{\theta }}_{j,C_j}^{\mathrm{*}}$ of ${\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)$ such that $∥{\mathit{\theta }}_{j,C_j}^{\mathrm{*}}-{\mathit{\theta }}_{j,C_j}^0∥=O_p\left(\sqrt{K{s}_0/n}\right)$.

Theorem 2. Under Assumptions 1–7, the oracle estimator ${\tilde{\mathit{\theta }}}_j$ with ${\tilde{\mathit{\theta }}}_{j,C_j}={\mathit{\theta }}_{j,C_j}^{\mathrm{*}}$ and ${\tilde{\mathit{\theta }}}_{j,C_j^c}=0$ is a strict local maximizer of $L\left({\mathit{\theta }}_j\right)$ with probability tending to 1.

Theorem 3. Under Assumptions 1–7, there exist a strict local maximizer ${\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_j$ of $L_n\left({\mathit{\theta }}_j\right)$ for each $j\in \{1,\dots ,p\}$, where for each $k\in \{1,\dots ,K\}$, the corresponding estimated precision matrix ${\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k$ obtained from (5) and the resulting edge set ${\stackrel{\mathit{ˆ}}{\mathit{E}}}_k=\left\{(\ell ,j):1\le \ell \ne j\le p\mathit{and}{\hat{\omega }}_{k,\ell j}\ne 0\right\}$ satisfy that

1. ${\mathrm{m}\mathrm{a}\mathrm{x}}_{k,j,\ell }\left|{\hat{\omega }}_{k,j\ell }-{\omega }_{k,j\ell }^0\right|=O_p\left(\sqrt{K{s}_0/n}\right),{\sum }_{k=1}^K{∥{\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k-{\mathbf{\Omega }}_k^0∥}_{\mathrm{\infty }}=O_p\left(\sqrt{K^3{s}_0^3/n}\right)$, and ${\sum }_{k=1}^K{∥{\left({\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k-{\mathbf{\Omega }}_k^0\right)}_{𝒜}∥}_F=O_p\left(\sqrt{K^3{s}^2/n}\right)$,

2. with probability tending to 1, ${\stackrel{\mathit{ˆ}}{\mathit{E}}}_k={\mathit{E}}_k^0$.

Here, $𝒜=\{(\ell ,j):1\le \ell \ne j\le p\}$ is the off-diagonal index set for the p × p precision matrix, and $s={\max }_{k\in \{1\dots K\}}|{\mathit{E}}_k^0|$ is the sparsity parameter of the true precision matrices.

The proofs of Theorems 1, 2, and 3 are provided in Section 7. In our study, we reduce the estimation problem of a large matrix to a collection of sparse linear regression problems. For each regression problem, we examine the theoretical properties based on the oracle estimator. A similar strategy has been considered in published high-dimensional data analysis studies, such as [9, 17, 42]. Theorem 1 establishes the estimation consistency of the oracle estimator ${\mathit{\theta }}_{j,C_j}^{\mathrm{*}}$, where the true sparsity structure is known and thus the number of unknown parameters is in the same order as $K{s}_0$. Theorem 2 then demonstrates that the proposed estimator ${\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_j$ is asymptotically as efficient as the oracle one. In Theorem 3, we further study the estimation errors of the precision matrices under the elementwise sup-norm and maximum induced norm, as well as that of the off-diagonal elements under the Frobenius norm, which are useful for the graph recovery from the precision matrices.

Remark 1. As the spectral norm is dominated by the maximum induced norm, the estimation error of the proposed estimators under the spectral norm also follows a rate of $O_p\left(\sqrt{K^3{s}_0^3/n}\right)$, which is important for the consistency of the eigenvalue and eigenvector estimates and can be further used to analyze theoretical properties of downstream inferences.

Remark 2. Considering the goal of network estimation and taking advantage of the column-wise strategy, the proposed theoretical study can focus on the off-diagonal elements of ${\mathbf{\Omega }}_k$’s. Under the assumption $\mathrm{l}\mathrm{n}\left(p\right)=O\left(n^a\right)$, the accuracy of the off-diagonal elements $\left(𝒜\right)$ can reach ${\sum }_{k=1}^K{∥{\left({\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k-{\mathbf{\Omega }}_k^0\right)}_{𝒜}∥}_F=O_p\left(\sqrt{K^3{s}^2/n}\right)$, which depends on the sparsity parameter of the true precision matrices, making the proposed approach applicable to large-scale datasets. In contrast, the existing heterogeneous network analysis approaches, including [13, 15, 29], rely heavily on the estimation properties of the precision matrices as a whole, which requires consideration of the estimation error of the diagonal elements and needs the assumption $p\mathrm{l}\mathrm{n}\left(p\right)=o\left(n\right)$.

4. Simulation

We first consider $p=100$ and the number of subgroups $K\in \{\mathrm{2,3},4\}$. Two settings for the subgroup sizes are considered, where the first one is a balanced design with 200 subjects in each subgroup, and the second one is an imbalanced design with subgroup sizes being $\left(\mathrm{150,200}\right),\left(\mathrm{150,200,250}\right)$, and $\left(\mathrm{155,185,215,245}\right)$ for $K\in \{\mathrm{2,3},4\}$, respectively. The networks for the $K$ subgroups are generated as follows. First, following [7], we simulate each network with ten unconnected subnetworks and consider three settings S1-S3 with different levels of similarity across the subgroups. Specifically, under settings S1 and S2, there are two and five subnetworks where all the subgroups share the same sparsity structures, respectively. Under setting S3, for $K=2$, eight of the ten subnetworks have the same sparsity structures in all the subgroups; and for $K=3$ and 4, there are five subnetworks with the same sparsity structures in all the subgroups, and another three subnetworks with the same sparsity structures shared only by the first two subgroups. Second, for each subnetwork, we consider three types of network structure: Power-law network, for which the degree distribution follows a power law; Nearest-neighbor network, where $p/10$ points are first randomly generated on a unit square, and based on the calculated $p/10\times (p/10-1)/2$ pairwise distances, 2 nearest neighbors of each point are found and connected; and Erdös-Rényi network, where the edge between each pair of nodes is added independently with probability 0.2.

For each $k\in \{1,\dots ,K\}$, the observations of the $k$th subgroup are generated from ${𝒩}_p\left({\mathit{\mu }}_k,{\mathbf{\Omega }}_k^{-1}\right)$, where the first four elements of ${\mathit{\mu }}_k$ are non-zero and the rest $p-4$ elements are all zero (details are provided in the supplementary materials, see Appendix), and ${\mathbf{\Omega }}_k$’s are generated based on the networks. Specifically, following [22], for the $k$th network, we generate an adjacency matrix ${\mathit{A}}_k$ whose non-zero off-diagonal entries (corresponding to edges) are generated from Uniform $\left(\right[-0.6,-0.3]\cup [\mathrm{0.3,0.6}\left]\right)$ and the diagonal entries are set to be 0. Then, ${\mathbf{\Omega }}_k$ is constructed as ${\mathbf{\Omega }}_k=\mathit{D}\left[{\mathit{A}}_k+\left\{\left|{\lambda }_{\mathrm{m}\mathrm{i}\mathrm{n}}\left({\mathit{A}}_k\right)\right|+0.2\right\}{\mathit{I}}_p\right]\mathit{D}$, where $\mathit{D}$ is a diagonal matrix whose diagonal elements are $\left({\mathbf{1}}_5,3{\mathbf{1}}_5,{\mathbf{1}}_5,3{\mathbf{1}}_5,\dots ,{\mathbf{1}}_5,3{\mathbf{1}}_5\right)$ with ${\mathbf{1}}_d$ being a $d$-dimensional vector of all ones, and ${\lambda }_{\mathrm{m}\mathrm{i}\mathrm{n}}\left({\mathit{A}}_k\right)$ is the smallest eigenvalue of ${\mathit{A}}_k$.

In addition to the proposed approach, five alternatives are also considered. SCAN, which is a heterogeneous network analysis approach based on the penalized log-likelihood of the Gaussian mixture model and the EM-ADMM algorithm [15]. Tiger, which is a single network analysis method via column-wise linear regressions and exploits Lasso based on a square root loss function for sparse network estimation [22]. True+Tiger, True+JGL, and True+JSEM, which apply Tiger, JGL, and JSEM for multiple network estimation based on the true subgroup memberships of subjects. Specifically, Tiger is conducted separately for each subgroup. JGL is the likelihood-based joint graph Lasso for multiple network estimation with known subgroup memberships [7]. JSEM conducts a joint analysis of multiple networks using neighborhood selection based on the group lasso penalty [25]. Both SCAN and True+JGL can be realized using the R package JGL, and Tiger and True+Tiger can be realized using the $R$ package huge. The $R$ code implementing True+JSEM can be downloaded from https://github.com/drjingma/JSEM. Among these approaches, SCAN achieves subgroup identification and multiple network estimation simultaneously, as well as accommodating common structures among networks. It is the most direct competitor for the proposed approach. In contrast, the other alternatives were originally developed for homogeneous network analysis or heterogeneous network analysis with known subgroup memberships. Tiger, True+Tiger, and True+JSEM estimate the sparse precision matrices in a column-by-column fashion. The first two perform sparse regression based on the square root loss function, but they either ignore the differences or the commonalities across subgroups. True+JSEM utilizes the squared loss function and does not consider the sparsity structure within each subgroup.

To evaluate the performance of different approaches, we consider the following measures: (1) Clustering error (CE) for evaluating subgroup identification performance, which calculates the distance between the estimated and true subgroup memberships $\hat{\phi }$ and $\phi$, defined as $\mathrm{C}\mathrm{E}={\left(C_n^2\right)}^{-1}\left|\left\{(i,j):I\left\{\hat{\phi }\left({\mathit{x}}_i\right)=\hat{\phi }\left({\mathit{x}}_j\right)\right\}\ne I\left\{\phi \left({\mathit{x}}_i\right)=\phi \left({\mathit{x}}_j\right)\right\},i<j\right\}\right|;$ (2) Precision matrix square error (PME) for measuring estimation performance, defined as PME $={\sum }_{k=1}^K{∥{\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k-{\mathbf{\Omega }}_k^0∥}_F/K$; (3) True and false positive rates (TPR and FPR) for evaluating network identification performance, defined as ${\mathrm{T}\mathrm{P}\mathrm{R}=\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0\right\}\right]/K$ and $\mathrm{F}\mathrm{P}\mathrm{R}={\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0\right\}\right]/K$.

For each scenario, 100 replicates are conducted. For a fair comparison, for all approaches, the true value of $K$ is considered. The results under the three types of network structure for $K=3$ are shown in Tables 1–3, respectively. The rest of the results for $K=2$ and $K=4$ are provided in the supplementary materials. It can be observed that the proposed approach has superior or competitive performance compared to SCAN in terms of subgroup identification accuracy under all the simulation scenarios. The improvement is more significant under the scenarios with more complex network structures (e.g. power-law network), lower levels of subgroup differences (e.g. S3), or a more imbalanced sample design. For example, for $K=3$ and the power-law network (Table 1), under the scenario with setting S3 and an imbalanced design, the median CEs are 0.000 (proposed) and 0.166 (SCAN), respectively. Additionally, the proposed approach also performs better in network estimation and identification accuracy. It is able to identify most TPs while keeping FPs much lower than the alternatives. For example, for $K=3$ and the Erdös-Rényi network (Table 3), under the scenario with setting S2 and a balanced design, the proposed approach has $\left(\text{PME, TPR, FPR}\right)=\left(\mathrm{16.751,\; 0.987,\; 0.039}\right)$, while $\left(\mathrm{26.975,\; 0.772,\; 0.076}\right)$ for SCAN, $\left(\mathrm{31.025,\; 0.789,\; 0.169}\right)$ for Tiger, $\left(\mathrm{21.104,\; 0.870,\; 0.075}\right)$ for True+Tiger, $\left(\mathrm{20.795,\; 0.901,\; 0.184}\right)$ for True+JGL, and $\left(\mathrm{16.403,\; 0.965,\; 0.087}\right)$ for True+JSEM.

Table 1:

Simulation results under the scenarios with the power-law networks and $K=3$. In each cell, we show the median (median absolute deviation) of the CE, PME, TPR and FPR values based on 100 replicates. $\mathrm{C}\mathrm{E}={\left(C_n^2\right)}^{-1}\left|\left\{(i,j):I\left\{\hat{\phi }\left({\mathit{x}}_i\right)=\hat{\phi }\left({\mathit{x}}_j\right)\right\}\ne I\left\{\phi \left({\mathit{x}}_i\right)=\phi \left({\mathit{x}}_j\right)\right\},i<j\right\}\right|,\mathrm{P}\mathrm{M}\mathrm{E}={\sum }_{k=1}^K{∥{\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k-{\mathbf{\Omega }}_k^0∥}_F/K$, $\mathrm{TPR}={\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0\right\}\right]/K$, and $\mathrm{FPR}={\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0\right\}\right]/K$.

Approach	CE	PME	TPR	FPR
S1 with the balanced design
proposed	$0.000\left(0.000\right)$	$20.377\left(1.058\right)$	$0.961\left(0.004\right)$	$0.037\left(0.001\right)$
SCAN	$0.001\left(0.001\right)$	$32.988\left(0.207\right)$	$0.757\left(0.006\right)$	$0.066\left(0.002\right)$
Tiger	-	$36.751\left(0.268\right)$	$0.763\left(0.016\right)$	$0.193\left(0.004\right)$
True+Tiger	-	$28.146\left(0.608\right)$	$0.832\left(0.008\right)$	$0.067\left(0.002\right)$
True+JGL	-	$27.372\left(0.539\right)$	$0.888\left(0.008\right)$	$0.181\left(0.003\right)$
True+JSEM	-	$22.451\left(0.635\right)$	$0.943\left(0.006\right)$	$0.084\left(0.003\right)$
S2 with the balanced design
proposed	$0.000\left(0.000\right)$	$19.884\left(1.528\right)$	$0.957\left(0.010\right)$	$0.036\left(0.002\right)$
SCAN	$0.000\left(0.000\right)$	$28.831\left(0.359\right)$	$0.791\left(0.009\right)$	$0.061\left(0.002\right)$
Tiger	-	$34.823\left(0.302\right)$	$0.779\left(0.012\right)$	$0.183\left(0.006\right)$
True+Tiger	-	$25.802\left(0.289\right)$	$0.848\left(0.005\right)$	$0.064\left(0.001\right)$
True+JGL	-	$25.367\left(0.453\right)$	$0.910\left(0.004\right)$	$0.172\left(0.003\right)$
True+JSEM	-	$20.182\left(0.549\right)$	$0.960\left(0.005\right)$	$0.070\left(0.007\right)$
S3 with the balanced design
proposed	$0.000\left(0.000\right)$	$21.495\left(0.770\right)$	$0.963\left(0.006\right)$	$0.037\left(0.001\right)$
SCAN	$0.000\left(0.000\right)$	$30.745\left(0.258\right)$	$0.778\left(0.008\right)$	$0.058\left(0.001\right)$
Tiger	-	$36.745\left(0.320\right)$	$0.739\left(0.012\right)$	$0.189\left(0.005\right)$
True+Tiger	-	$28.153\left(0.319\right)$	$0.839\left(0.006\right)$	$0.063\left(0.001\right)$
True+JGL	-	$26.963\left(0.559\right)$	$0.907\left(0.008\right)$	$0.168\left(0.004\right)$
True+JSEM	-	$19.588\left(0.497\right)$	$0.973\left(0.005\right)$	$0.068\left(0.004\right)$
S1 with the imbalanced design
proposed	$0.000\left(0.000\right)$	$22.823\left(1.167\right)$	$0.952\left(0.007\right)$	$0.037\left(0.001\right)$
SCAN	$0.002\left(0.002\right)$	$35.378\left(0.475\right)$	$0.739\left(0.007\right)$	$0.058\left(0.002\right)$
Tiger	-	$38.486\left(0.219\right)$	$0.756\left(0.011\right)$	$0.190\left(0.005\right)$
True+Tiger	-	$29.594\left(0.574\right)$	$0.837\left(0.005\right)$	$0.064\left(0.001\right)$
True+JGL	-	$28.999\left(0.487\right)$	$0.900\left(0.007\right)$	$0.173\left(0.002\right)$
True+JSEM	-	$24.722\left(0.767\right)$	$0.940\left(0.007\right)$	$0.079\left(0.006\right)$
S2 with the imbalanced design
proposed	$0.000\left(0.000\right)$	$21.155\left(1.522\right)$	$0.949\left(0.009\right)$	$0.037\left(0.001\right)$
SCAN	$0.086\left(0.084\right)$	$31.318\left(2.751\right)$	$0.766\left(0.025\right)$	$0.060\left(0.005\right)$
Tiger	-	$34.915\left(0.306\right)$	$0.796\left(0.012\right)$	$0.187\left(0.005\right)$
True+Tiger	-	$26.661\left(0.522\right)$	$0.846\left(0.005\right)$	$0.067\left(0.001\right)$
True+JGL	-	$26.007\left(0.662\right)$	$0.906\left(0.006\right)$	$0.173\left(0.003\right)$
True+JSEM	-	$20.472\left(0.552\right)$	$0.959\left(0.004\right)$	$0.068\left(0.012\right)$
S3 with the imbalanced design
proposed	$0.000\left(0.000\right)$	$22.736\left(2.614\right)$	$0.954\left(0.009\right)$	$0.038\left(0.002\right)$
SCAN	$0.166\left(0.004\right)$	$34.468\left(4.191\right)$	$0.753\left(0.063\right)$	$0.063\left(0.009\right)$
Tiger	-	$37.710\left(0.241\right)$	$0.744\left(0.010\right)$	$0.180\left(0.004\right)$
True+Tiger	-	$28.955\left(0.590\right)$	$0.837\left(0.006\right)$	$0.065\left(0.002\right)$
True+JGL	-	$28.095\left(0.539\right)$	$0.908\left(0.010\right)$	$0.173\left(0.004\right)$
True+JSEM	-	$20.138\left(0.685\right)$	$0.977\left(0.007\right)$	$0.068\left(0.003\right)$

Table 3:

Simulation results under the scenarios with the Erdös-Rényi networks and $K=3$. In each cell, we show the median (median absolute deviation) of the CE, PME, TPR and FPR values based on 100 replicates. $\mathrm{C}\mathrm{E}={\left(C_n^2\right)}^{-1}\left|\left\{(i,j):I\left\{\hat{\phi }\left({\mathit{x}}_i\right)=\hat{\phi }\left({\mathit{x}}_j\right)\right\}\ne I\left\{\phi \left({\mathit{x}}_i\right)=\phi \left({\mathit{x}}_j\right)\right\},i<j\right\}\right|,\mathrm{P}\mathrm{M}\mathrm{E}={\sum }_{k=1}^K{∥{\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k-{\mathbf{\Omega }}_k^0∥}_F/K$, ${\mathrm{T}\mathrm{P}\mathrm{R}=\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0\right\}\right]/K$, and ${\mathrm{F}\mathrm{P}\mathrm{R}=\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0\right\}\right]/K$.

Approach	CE	PME	TPR	FPR
S1 with the balanced design
proposed	$0.002\left(0.002\right)$	$16.805\left(1.169\right)$	$0.988\left(0.004\right)$	$0.040\left(0.001\right)$
SCAN	$0.004\left(0.002\right)$	$26.831\left(1.337\right)$	$0.779\left(0.016\right)$	$0.076\left(0.004\right)$
Tiger	-	$31.927\left(0.190\right)$	$0.797\left(0.015\right)$	$0.176\left(0.005\right)$
True+Tiger	-	$17.662\left(0.593\right)$	$0.918\left(0.008\right)$	$0.073\left(0.001\right)$
True+JGL	-	$18.989\left(0.619\right)$	$0.918\left(0.008\right)$	$0.181\left(0.003\right)$
True+JSEM	-	$17.261\left(0.455\right)$	$0.945\left(0.008\right)$	$0.098\left(0.003\right)$
S2 with the balanced design
proposed	$0.002\left(0.002\right)$	$16.751\left(0.906\right)$	$0.987\left(0.004\right)$	$0.039\left(0.002\right)$
SCAN	$0.004\left(0.002\right)$	$26.975\left(0.412\right)$	$0.772\left(0.007\right)$	$0.076\left(0.001\right)$
Tiger	-	$31.025\left(0.220\right)$	$0.789\left(0.017\right)$	$0.169\left(0.005\right)$
True+Tiger	-	$21.104\left(0.451\right)$	$0.870\left(0.005\right)$	$0.075\left(0.001\right)$
True+JGL	-	$20.795\left(0.460\right)$	$0.901\left(0.020\right)$	$0.184\left(0.021\right)$
True+JSEM	-	$16.403\left(0.586\right)$	$0.965\left(0.009\right)$	$0.087\left(0.002\right)$
S3 with the balanced design
proposed	$0.002\left(0.002\right)$	$14.699\left(0.628\right)$	$0.996\left(0.004\right)$	$0.039\left(0.002\right)$
SCAN	$0.006\left(0.006\right)$	$17.155\left(2.467\right)$	$0.768\left(0.027\right)$	$0.061\left(0.008\right)$
Tiger	-	$20.312\left(0.181\right)$	$0.796\left(0.012\right)$	$0.186\left(0.006\right)$
True+Tiger	-	$17.670\left(0.467\right)$	$0.919\left(0.008\right)$	$0.074\left(0.001\right)$
True+JGL	-	$17.271\left(0.459\right)$	$0.930\left(0.010\right)$	$0.158\left(0.005\right)$
True+JSEM	-	$15.435\left(0.433\right)$	$0.971\left(0.004\right)$	$0.098\left(0.003\right)$
S1 with the imbalanced design
proposed	$0.004\left(0.002\right)$	$16.979\left(1.322\right)$	$0.986\left(0.006\right)$	$0.038\left(0.003\right)$
SCAN	$0.004\left(0.002\right)$	$26.194\left(0.787\right)$	$0.786\left(0.008\right)$	$0.076\left(0.002\right)$
Tiger	-	$31.834\left(0.169\right)$	$0.797\left(0.012\right)$	$0.175\left(0.004\right)$
True+Tiger	-	$18.193\left(0.564\right)$	$0.918\left(0.005\right)$	$0.077\left(0.002\right)$
True+JGL	-	$19.442\left(0.488\right)$	$0.914\left(0.008\right)$	$0.184\left(0.003\right)$
True+JSEM	-	$17.479\left(0.499\right)$	$0.942\left(0.008\right)$	$0.097\left(0.002\right)$
S2 with the imbalanced design
proposed	$0.002\left(0.002\right)$	$18.227\left(1.626\right)$	$0.983\left(0.007\right)$	$0.039\left(0.003\right)$
SCAN	$0.004\left(0.002\right)$	$27.080\left(0.547\right)$	$0.764\left(0.010\right)$	$0.076\left(0.002\right)$
Tiger	-	$31.129\left(0.221\right)$	$0.797\left(0.009\right)$	$0.168\left(0.004\right)$
True+Tiger	-	$21.627\left(0.477\right)$	$0.870\left(0.005\right)$	$0.077\left(0.002\right)$
True+JGL	-	$21.139\left(0.567\right)$	$0.893\left(0.013\right)$	$0.182\left(0.008\right)$
True+JSEM	-	$17.005\left(0.579\right)$	$0.962\left(0.007\right)$	$0.088\left(0.003\right)$
S3 with the imbalanced design
proposed	$0.002\left(0.002\right)$	$14.946\left(0.959\right)$	$0.992\left(0.004\right)$	$0.039\left(0.003\right)$
SCAN	$0.008\left(0.004\right)$	$25.161\left(0.934\right)$	$0.791\left(0.016\right)$	$0.079\left(0.003\right)$
Tiger	-	$32.592\left(0.195\right)$	$0.794\left(0.018\right)$	$0.159\left(0.006\right)$
True+Tiger	-	$18.603\left(0.628\right)$	$0.919\left(0.008\right)$	$0.078\left(0.001\right)$
True+JGL	-	$18.169\left(0.440\right)$	$0.931\left(0.009\right)$	$0.161\left(0.004\right)$
True+JSEM	-	$15.791\left(0.668\right)$	$0.972\left(0.004\right)$	$0.099\left(0.002\right)$

In general, under scenarios with a smaller number of subgroups $(K=2)$, performance of all approaches is significantly improved, with the proposed approach still performing the best and having better performance with increasing similarity across networks (from S1 to S3). The superiority of the proposed approach becomes more prominent under scenarios with a larger number of subgroups, suggesting the validity of the proposed strategy under complex situations. Tiger tends to perform the worst as it ignores heterogeneity. Comparatively, SCAN can simultaneously conduct subgroup identification and multiple network estimation, thus having better performance. Under ideal scenarios with true subgroup memberships, True+Tiger, True+JGL, and True+JSEM perform the second best, with True+JSEM being the most prominent as it conducts joint network analysis with an effective sparse linear programming perspective. In most cases, benefiting from the satisfactory scale-invariant property, the proposed approach, although with unknown sample heterogeneity, can exhibit more accurate network identification than these methods depending on true subgroup memberships. In summary, our approach can effectively capture both shared and unique network structures in heterogeneous network data across diverse scenarios with various degrees of within-group similarity, network structures, and number of subgroups.

In addition to the above analyses, we investigate scenarios where the predictors are higher-dimensional, with $p=500$ and $p=\mathrm{1,000}$. The detailed settings and results (Table S7) are provided in the supplementary materials, where the simulated networks for $p=500$ are denser than those for $p=\mathrm{1,000}$. As it is not computationally feasible to conduct analysis with SCAN for scenarios with $p=\mathrm{1,000}$, the corresponding results are not available. For larger-scale data, the performance of those regularized likelihood-based approaches such as SCAN and True+JGL decays as expected, especially for SCAN, whose performance gets dramatically worse and cannot afford the computational cost in the case of $p=\mathrm{1,000}$. However, the network analysis approaches based on column-wise linear regressions continue to maintain their satisfactory performance. Under the scenarios with a denser network and high dimension $(p=500)$, the proposed approach exhibits slightly inferior performance compared to True+JSEM, as it is more difficult to accurately identify subgroup memberships for subjects, resulting in reduced network identification accuracy. Under the scenario with sparser networks $(p=\mathrm{1,000})$, although with a high dimension, the proposed approach still behaves more favorably. Even when the networks exhibit a higher degree of similarity (S3), the TPR of the proposed approach drops slightly due to the misclassification of subjects, but it still has a smaller number of false positives for the networks.

To gain a deeper insight into the benefit of considering common structures in different networks, we compare the proposed approach with an ad-hoc approach called “ad-hoc MCP” that estimates each network separately using only MCP based on the identified subgroups with the proposed approach. The comparison results under the scenario with $K=3$ are provided in Supplementary Table S8, which demonstrates that the proposed approach has superior identification and estimation performance compared to the ad-hoc MCP method. Furthermore, we take one replicate under the scenario with setting S3, a balanced design, and the power-law network as an example, and provide the heatmaps of the true sparse structures and estimated results with the proposed approach and the ad-hoc MCP approach in Supplementary Figs. S1 and S2. It is observed that the proposed approach can more accurately identify the true sparse structures and more effectively accommodate the common structures across different networks.

4.1. Computer time

To examine the computational superiority of the proposed approach, we conduct simulations with various values of $(n,p,K)$, which are all implemented on a computer with an Intel Core i7 processor and 24 GB of RAM. The computer time results of the proposed approach and the alternatives with fixed tuning parameters are reported in Supplementary Table S9. In general, analysis with the proposed approach is observed to take more time than Tiger and those methods based on known subgroup memberships, which is due to the fact that our approach aims to perform both subgroup and multiple network identifications, while the others either only analyze a single network or rely on prior subgroup information. Compared to the most direct competitor SCAN, which is also based on the Gaussian mixture model and EM algorithm, the proposed approach is significantly faster, especially for large-scale data. For example, under the scenario with $n=600,p=500$, and $K=3$, the average computer time is 1794.367 (proposed) and 5169.790 (SCAN) seconds. When the dimension increases to 1,000, SCAN is computationally infeasible (more than 24 hours) even with a sample size of $n=300$. However, the proposed algorithm is still computationally affordable, with the computer time being around 2.6 hours when $n=\mathrm{1,000}$.

The computational cost of the proposed approach can be significantly reduced by using parallel computing, benefiting from the column-wise strategy. For $K=3$, we compare the computational cost of the single-thread $c_{\mathit{\text{single}}}$ and its parallel version $c_{\mathit{\text{parrallel}}}$ under the scenarios with various sample sizes, dimensions, and number of cores. The cost ratio $r_c=c_{\mathit{\text{parrallel}}}/c_{\mathit{\text{single}}}$ as a function of dimension is provided in Supplementary Fig. S3. As can be observed, the computer time of the proposed approach speeds up over the single-thread version of the algorithm, especially for high-dimensional settings and the use of multi-core processors. The corresponding costs of the proposed analysis with eight cores are also provided in Supplementary Table S9. For high-dimensional data, the proposed approach can sometimes be as efficient as the True+JGL and True+JSEM methods. For example, under the scenario with $n=300$, $p=1000$, and $K=2$, the average computer time is 172.217 seconds (proposed with parallel), 678.119 seconds (True+JGL), and 553.785 seconds (True+JSEM).

4.2. Tuning-insensitive regularization path

We further numerically examine the tuning-insensitive properties of the proposed approach. Motivated by [22, 33], for finite samples, we consider the tuning parameter with the form $\zeta \sqrt{\mathrm{l}\mathrm{n}\left\{K\right(p-1\left)\right\}/n}$, where $\zeta$ is a positive constant independent of all unknown parameters, and introduce the graph recovery accuracy: Accuracy = TPR – FPR. Taking the simulation scenario under the power-law network and setting S1 as an example, we examine the values of Accuracy and PME as a function of $\zeta$ for the proposed approach, as well as SCAN and True+JSEM, in Fig. 1. It is observed that for the proposed approach, the regularization paths are flat without a significant change when $\zeta$ lies in the range of $\left(\mathrm{0.4,0.7}\right)$, which suggests that the proposed approach is empirically insensitive to the tuning parameter $\lambda$. In contrast, for SCAN and True+JSEM, a larger range of $\left({\zeta }_1,{\zeta }_2\right)$ (or $\zeta$) is required in the search of the optimal value, and the paths present more irregular changes, which indicates more sensitivity of SCAN and True+JSEM to the choice of tuning parameters. In general, the proposed approach is easier to implement than SCAN and True+JSEM to find reasonable tuning parameters.

Fig. 1:

Column one: Values of Accuracy and PME as a function of $\zeta$ for the proposed approach; Columns two and three: Values of Accuracy and PME as a function of ${\zeta }_1$ for the individual-level sparsity parameter which has form ${\zeta }_1\sqrt{\mathrm{l}\mathrm{n}\left\{K\right(p-1\left)\right\}/n}$, and ${\zeta }_2$ for the group-level sparsity parameter which has form ${\zeta }_2\sqrt{\mathrm{l}\mathrm{n}(p-1)/n}$ for the SCAN approach; Column four: Values of Accuracy and PME as a function of $\zeta$ for the group-level sparsity parameter which has form $\zeta \sqrt{\mathrm{l}\mathrm{n}(p-1)/n}$ for the True+JSEM approach, under the scenario with the power-law network and setting S1.

5. Data Analysis

We focus on reconstructing gene networks for cancer, which is an important task for better understanding the underlying biological processes. Specifically, we analyze breast cancer data from The Cancer Genome Atlas (TCGA), where the underlying heterogeneity has been posing an increasing public health concern. The mRNA gene expression measurements are considered, which are downloaded from the TCGA website using the R package cgdsr. In total, 1, 100 breast cancer subjects are available with 18, 506 gene expression measurements. As the number of connected genes is not expected to be large, to improve stability as well as reduce computational cost, we conduct prescreening, which has been a common technique in published studies. Specifically, following [25], we focus on the genes in the Wnt signaling, oxidative phosphorylation, Mtor signaling, and citrate cycle tca cycle pathways, which have been demonstrated to play an important role in all cancer types in the literature. This results in 316 genes for downstream analysis.

In practice, the number of subgroups $K$ is usually unknown. In this study, we adopt the gap statistic using the $\mathrm{R}$ package $\mathrm{N}\mathrm{b}\mathrm{C}\mathrm{l}\mathrm{u}\mathrm{s}\mathrm{t}$ to select the optimal value of $K$. Given the candidate set $K\in \{\mathrm{2,3},\mathrm{4,5},\mathrm{6,7},8\}$, with the gap statistic, the proposed approach identifies two subgroups with group sizes 805 (subgroup 1) and 295 (subgroup 2). In addition, 1,171 and 666 edges among 309 and 289 genes are discovered for the two subgroups, respectively, and 222 common edges across the two subgroups are identified. The graphical representation of the identified networks is provided in Fig. 2.

Fig. 2:

Data analysis: gene networks for the two subgroups identified with the proposed approach. The black lines represent the common edges shared by the two subgroups, and the blue lines represent the specific edges for each subgroup.

To gain more insight into the identified networks, we examine the related genes’ functional and biological connections by conducting Gene Ontology (GO) enrichment analysis, which is implemented using DAVID 2021 [31]. Our analysis first suggests that those genes involved in the common edges are functionally and biologically connected with certain significantly enriched GO terms. For example, in one common subnetwork, genes ATP6V1A, ATP5F1A, ATP6V1B2, and NDUFS1 are enriched with the ATP metabolic process (GO:0046034, P-value: $7.37\times {10}^{-3}$), which has been demonstrated to be closely linked to the selective killing of respiratory competent cancer cells that are critical for tumor progression, including breast cancer [21]. In addition, in this subnetwork, genes ROCK1, ROCK2, and RAC1 are enriched with regulation of stress fiber assembly (GO:0051492, P-value: $1.06\times {10}^{-3}$), and studies have shown that stress fiber-mediated cellular stiffness can promote tumor growth in the precancerous stage, including breast cancer [35]. Genes NDUFA11, COX6A1, RPS6KA3, FRAT1, STK11, SOX17, RPS6KA1, EP300, and ATP6V1E1 are enriched with protein binding function (GO:0005515, P-value: $3.14\times {10}^{-12}$), which has been reported to play an important role in cancer treatment, and in fact several proteins have been shown to be effective in delivering to tumor sites [37]. Genes GSK3B, PRKAA1, CHD8, TCF7, LRP6, NKD2, SOX17, DVL1, and DVL3 are enriched with the Wnt signaling pathway (GO:0016055, P-value: $1.63\times {10}^{-21}$), which has been reported to be activated in over half of breast cancer patients and plays an important role in triple negative breast cancer development [39]. Moreover, genes COX7B, NDUFA11, COX4I1, NDUFA10, COX6A1, COX7C, COX8A, NDUFC2, NDUFC1, and SDHD are enriched with mitochondrial inner membrane (GO:0005743, P-value: $1.39\times {10}^{-58}$), which has been reported to be essential in the regulation of cancer cell migration and invasion [36], including breast cancer [41]. The GO enrichment analysis also finds that the genes involved in the specific edges of different subgroups are associated with some distinct significantly enriched GO terms. For example, genes WNT5A, TSC2, ULK1, AXIN2, CSNK1E, and TP53 are enriched with the protein localization process (GO:0008104, P-value: $3.58\times {10}^{-3}$) in subgroup 1, which has been reported to be implicated in the pathogenesis of human diseases, including breast cancer, and therapeutic strategies targeting protein localization have been conceptualized as promising for the treatment of a variety of human diseases [19]. In addition, genes LEF1, PSEN1, DKK1, and LRP6 are enriched with embryonic limb morphogenesis process (GO:0030326, P-value: 0.02) in subgroup 2, where some embryonic genes are significantly upregulated in estrogen receptor negative breast cancer [43]. These biologically sensible findings provide support for the validity of the proposed network identification analysis.

Analyses are also conducted using the alternative approaches with $K=2$. Here, as the true subgroup memberships are not available, for comparative analyses under known subgroup memberships as well as indirect support for the subgroup identification results, we follow [25] and consider three clinical breast cancer subgroups, namely ER+, ER−, and other unevaluated cases, based on whether the subjects have estrogen receptors. In our data analysis, the sizes of these three subgroups are 812, 238, and 50, respectively. The multiple network estimation procedure using the alternatives is performed based on the ER+ and ER− subgroups. The summary comparison results of the heterogeneity analysis and network analysis are reported in Tables S10 and S11 of the supplementary materials, where the numbers of subjects in each subgroup and edges in each network identified by the different approaches and their overlaps are provided. Here, the subgroups identified using the different approaches are matched by correlation, and the different approaches lead to different subgroup memberships. For ER+ and ER−, we present the proportions of subjects that are identified in different subgroups using the proposed and SCAN approaches in Fig. S4 of the supplementary materials. It is observed that the two approaches can discriminate the ER+ and ER− subgroups effectively, with the proposed approach demonstrating more advantageous results. The P-value of the Chi-square test for the proposed approach is $6.08\times {10}^{-69}$. Additionally, by matching the identified subgroups with ER status, the CE of the proposed approach is 0.29, compared to 0.37 for SCAN. These results suggest the effectiveness of the proposed analysis.

In Table S11 of the supplementary materials, we observe that the different approaches identify a moderate number of overlapping edges. To indirectly support the network identification results, following [11], we adopt a resampling strategy and compute the negative log-likelihood statistic (NLS) to evaluate “prediction accuracy”, with a smaller value indicating a better performance. Specifically, we first randomly divide the data into a training and a testing set, estimate the parameters on the training set, and finally compute the NLS using the testing set. Based on 100 resamplings, the proposed approach has an average NLS value of 77449.60, compared to 78278.28 for SCAN, 99115.53 for Tiger, 97528.56 for ER+Tiger, 97385.31 for ER+JGL, and 96463.68 for ER+JSEM, with the proposed approach having competitive prediction accuracy.

6. Discussion

Network analysis of heterogeneous subgroups is still a wide-open problem in various research fields. In this article, we have proposed a new joint estimation approach for multiple networks based on the multivariate Gaussian mixture modeling. Different from the existing regularized likelihood-based approaches, which are usually only applicable to small-scale data, the proposed approach provides a more effective and useful tool for estimating multiple precision matrices by solving a collection of simpler sparse regression subproblems. Specifically, based on the Gaussian graphical model, we have proposed a reparameterized mixture of regression models with composite MCP to effectively accommodate both similarities and differences across undiscovered distinct subgroups. Such a strategy enjoys the scale-invariant and tuning-insensitive properties and can be solved in parallel to largely reduce computational cost. The theoretical properties of the proposed approach are investigated, indicating that the proposed estimator enjoys the oracle property. A number of numerical experiments demonstrate the superior performance of the proposed approach in terms of subgroup and network identification accuracy. The application to a TCGA data exploits different gene networks for breast cancer and rediscovers biologically sensible gene relationships associated with the heterogeneous subgroups.

In the proposed approach, we have adopted the composite MCP to accommodate the common and specific structures among different networks. Other penalties, such as $\rho \left({∥\mathrm{v}\mathrm{e}\mathrm{c}\left({\beta }_{1,j,\ell },\dots ,{\beta }_{K,j,\ell }\right)∥}_2;{\lambda }_1,{\gamma }_1\right)+{\sum }_{k=1}^K\rho \left(\left|{\beta }_{k,j,\ell }\right|;{\lambda }_2,{\gamma }_2\right)$ (sparse group MCP) can also achieve identification at both the group and individual levels and may have satisfactory statistical and numerical properties. Both composite MCP and sparse group MCP have been adopted in published studies, such as [3, 17] for composite MCP, and [23] for sparse group MCP. It is expected that performance of these two penalties may depend on the underlying model, data settings, and other factors. We adopt the composite MCP, as it has been popular in the literature and leads to satisfactory numerical performance. In future works, some other penalties, including this sparse group MCP, can be further investigated. In our data analysis, the TCGA breast cancer data has been analyzed, which has often been used for Gaussian distribution-based network analysis [27, 40]. We have followed these studies and conducted analysis without data transformation or normalization, and some results with important biological implications have been found. We acknowledge the complexity of gene expression data with potential non-Gaussian properties. It would be of interest to implement robust techniques like the $t$ distribution mixture model or some other nonparanormal mixture graphical models to further study non-Gaussian distributed data. We have mainly focused on the identification of gene networks using expression data. Many other biological measurements, such as mutation and DNA methylation, can be further used for a better understanding of the mechanisms of cancer.

7. Technical details

Proof of Theorem 1. Recall that ${\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)={\sum }_{i=1}^n\mathrm{l}\mathrm{n}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)$. Let ${\delta }_n=\sqrt{K{s}_0/n}$ and $\mathit{h}$ be a $s_j$-dimensional vector, where $s_j=3K+{\sum }_{k=1}^K\left|{𝒮}_{j,k}\right|$ is the nonsparsity size for the $j$th subproblem. It suffices to show that ${\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}^0+\right.\left.{\delta }_n\mathit{h}\right)<{\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}^0\right)$ everywhere on the boundary $\{\mathit{h}:\parallel \mathit{h}\parallel =C\}$, where $C$ is a sufficiently large positive constant. By Taylor expansion, we have:

$$\begin{gathered} {\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}^0+{\delta }_n\mathit{h}\right)-{\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}^0\right) \\ ={\delta }_n{\mathit{h}}^{\top }\left\{{\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}\right\}+\frac{1}{2}{\delta }_n^2{\mathit{h}}^{\top }\left\{{\frac{{\partial }^2{\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{{\partial }^2{\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}\right\}\mathit{h}+{\frac{{\delta }_n^3}{6}\sum _{q,\ell ,m\in C_j}\frac{{\partial }^3{\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{\partial {\theta }_{j,q}\partial {\theta }_{j,\ell }\partial {\theta }_{j,m}}|}_{{\mathit{\theta }}_{j,C_j}={\stackrel{\circ }{\mathit{\theta }}}_{j,C_j}}{h}_q{h}_{\ell }{h}_m \\ :=I+II+III, \end{gathered}$$

where ${\stackrel{\circ }{\mathit{\theta }}}_{j,C_j}$ lies on the line segment connecting ${\mathit{\theta }}_{j,C_j}^0+{\delta }_n\mathit{h}$ and ${\mathit{\theta }}_{j,C_j}^0$.

For $I$, by Assumption 1, as $n\to \mathrm{\infty }$,

$$\begin{gathered} {\frac{1}{\sqrt{n}}\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}=\sqrt{n}\left\{{\frac{1}{n}\sum _{i=1}^n\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}\right\} \\ =\sqrt{n}[{\frac{1}{n}\sum _{i=1}^n\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}-E\left\{{\frac{\partial \text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^{\mathbf{0}}}\right\}\to 𝒩\left(0,ℐ\left({\mathit{\theta }}_{j,C_j}^0\right)\right). \end{gathered}$$

Thus,

$${\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{\partial {\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}=O_p(\sqrt{n}).$$

Then, it is easy to see that

$$\left|I\right|\le O_p\left(n{\delta }_n^2\right)\parallel \mathit{h}\parallel .$$

For $II$,

$$II=-\frac{1}{2}n{\delta }_n^2{\mathit{h}}^{\top }ℐ\left({\mathit{\theta }}_{j,C_j}^0\right)\mathit{h}+\frac{1}{2}n{\delta }_n^2{\mathit{h}}^{\top }\left\{{\frac{1}{n}\frac{{\partial }^2{\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{{\partial }^2{\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}+ℐ\left({\mathit{\theta }}_{j,C_j}^0\right)\right\}\mathit{h}.$$

Following similar arguments as in the proof of Lemma 8 in [10], with Assumptions 1–2, we have:

$$\Vert {\frac{1}{n}\frac{{\partial }^2{\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{{\partial }^2{\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0}+ℐ\left({\mathit{\theta }}_{j,C_j}^0\right)\Vert =o_p\left(1/s_j\right).$$

Therefore,

$$II=-\frac{1}{2}n{\delta }_n^2{\mathit{h}}^{\top }ℐ\left({\mathit{\theta }}_{j,C_j}^0\right)\mathit{h}+\frac{1}{2}n{\delta }_n^2\parallel \mathit{h}{\parallel }^2\times o_p(1).$$

For III, by Assumption 3 and applying Cauchy-Schwartz inequality, we have:

$$\begin{gathered} \left|III\right|={\frac{{\delta }_n^3}{6}{\left|\sum _{q,\ell ,m\in C_j}\frac{{\partial }^3{\tilde{L}}_n\left({\mathit{\theta }}_{j,C_j}\right)}{\partial {\theta }_{j,q}\partial {\theta }_{j,\ell }\partial {\theta }_{j,m}}\right|}_{{\mathit{\theta }}_{j,C_j}={\stackrel{\circ }{\mathit{\theta }}}_{j,C_j}}{h}_q{h}_{\ell }{h}_m\left|=\frac{{\delta }_n^3}{6}\right|\sum _{q,\ell ,m\in C_j}\sum _{i=1}^n\frac{{\partial }^3\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_{j,C_j}\right)}{\partial {\theta }_{j,q}\partial {\theta }_{j,\ell }\partial {\theta }_{j,m}}|}_{{\mathit{\theta }}_{j,C_j}={\stackrel{\circ }{\mathit{\theta }}}_{j,C_j}}{h}_q{h}_{\ell }{h}_m| \\ \le \frac{{\delta }_n^3}{6}\sum _{i=1}^n{\left\{\sum _{q,\ell ,m\in C_j}{M}_2^2\left(v_i\right)\right\}}^{1/2}\parallel \mathit{h}{\parallel }^3=O_p\left(s_j^{3/2}{\delta }_n\right)\times n{\delta }_n^2\times \parallel \mathit{h}{\parallel }^2. \end{gathered}$$

By Assumption 4, for $b\in (\mathrm{0,1}/4),K=O\left(n^b\right)$ and $s_0=o\left(n^{1/4-b}\right)$, then ${\left(K{s}_0\right)}^2=o\left(n^{1/2}\right)$. Thus,

$$III=o_p\left(n{\delta }_n^2\right)\parallel \mathit{h}{\parallel }^2.$$

Due to the positive definiteness of the Fisher information matrix $ℐ\left({\mathit{\theta }}_{j,C_j}\right)$ at ${\mathit{\theta }}_{j,C_j}={\mathit{\theta }}_{j,C_j}^0$, for a sufficiently large $C$, the quadratic function $-n{\delta }_n^2{\mathit{h}}^{\mathrm{\top }}ℐ\left({\mathit{\theta }}_{j,C_j}^0\right)\mathit{h}/2$ in $II$ dominates $I$ and $III$. This completes the proof.

Proof of Theorem 2. To prove that ${\tilde{\mathit{\theta }}}_j$ is a local maximizer of $L_n\left({\mathit{\theta }}_j\right)$, we consider ${\mathit{\theta }}_j$ in a small neighborhood of ${\tilde{\mathit{\theta }}}_j$ such that $∥{\mathit{\theta }}_j-{\tilde{\mathit{\theta }}}_j∥=O\left(n^{a/2-1/2}\sqrt{\mathrm{l}\mathrm{n}n/\left(Kp\right)}+s_0\sqrt{K/\left(pn\right)}\right)$, and let ${\underset{\_}{\mathit{\theta }}}_j$ have ${\underset{\_}{\mathit{\theta }}}_j{C}_j={\mathit{\theta }}_j{C}_j$ and ${\underset{\_}{\mathit{\theta }}}_j{C}_j^c=\mathbf{0}$. According to Theorem 1, we have $L_n\left({\underset{\_}{\mathit{\theta }}}_j\right)<L_n\left({\tilde{\mathit{\theta }}}_j\right)$. Hence it suffices to show $L_n\left({\underset{\_}{\mathit{\theta }}}_j\right)>L_n\left({\mathit{\theta }}_j\right)$. Notice that

$$\begin{gathered} L_n\left({\mathit{\theta }}_j\right)-L_n\left({\underset{\_}{\mathit{\theta }}}_j\right)={\tilde{L}}_n\left({\mathit{\theta }}_j\right)-{\tilde{L}}_n\left({\underset{\_}{\mathit{\theta }}}_j\right)-n\sum _{\ell \ne j}\left[\rho \left\{\sum _{k=1}^K\rho \left(\left|{\beta }_{k,j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\}-\rho \left\{\sum _{k=1}^K\rho \left(\left|{\underset{\_}{\beta }}_{k,j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\}\right] \\ :=L_{n1}+L_{n2}. \end{gathered}$$

For $L_{n1}$, we have:

$$L_{n1}={\tilde{L}}_n\left({\mathit{\theta }}_j\right)-{\tilde{L}}_n\left({\underset{\_}{\mathit{\theta }}}_j\right)={\sum _k\sum _{\ell \in {𝒮}_{j,k}^c}\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}|}_{{\mathit{\theta }}_j={\overline{\mathit{\theta }}}_j}{\beta }_{k,j,\ell }=\sum _k\sum _{\ell \in {𝒮}_{j,k}^c}\left\{{\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}|}_{{\mathit{\theta }}_j={\tilde{\mathit{\theta }}}_j}+{{\left({\overline{\mathit{\theta }}}_j-{\tilde{\mathit{\theta }}}_j\right)}^{\top }\frac{{\partial }^2{\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }\partial {\mathit{\theta }}_j}|}_{{\mathit{\theta }}_j={\stackrel{\prime }{\mathit{\theta }}}_j}\right\}{\beta }_{k,j,\ell },$$

where ${\overline{\mathit{\theta }}}_j$ lies on the line segment connecting ${\mathit{\theta }}_j$ and ${\underset{\_}{\mathit{\theta }}}_j$, and ${\stackrel{\prime }{\mathit{\theta }}}_j$ lies on the line segment connecting ${\overline{\mathit{\theta }}}_j$ and ${\tilde{\mathit{\theta }}}_j$.

First consider ${\left.\left\{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)/\partial {\beta }_{k,j,\ell }\right\}\right|}_{{\mathit{\theta }}_j={\tilde{\mathit{\theta }}}_j}$. With a second order Taylor expansion, for $\ell \in {𝒮}_{j,k}^c$, we have:

$$\begin{gathered} {\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}|}_{{\mathit{\theta }}_j={\tilde{\mathit{\theta }}}_j}={\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}+{{\left({\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\right)}^{\top }\frac{{\partial }^2{\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }\partial {\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}+{{\left({\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\right)}^{\top }\frac{{\partial }^3{\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }{\partial }^2{\mathit{\theta }}_{j,C_j}}|}_{{\mathit{\theta }}_j={\stackrel{`}{\mathit{\theta }}}_j}\left({\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\right) \\ :=I+II+III, \end{gathered}$$ (10)

where ${\stackrel{`}{\mathit{\theta }}}_j$ lies on the line segment connecting ${\tilde{\mathit{\theta }}}_j$ and ${\mathit{\theta }}_j^0$.

For $I$ in (10), define the following event:

$${\text{Ω}}_1=\left\{\underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}{\left|\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}\right|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}|\le {\zeta }_n\sqrt{n}\right\},$$

where ${\zeta }_n=n^{a/2}\sqrt{\mathrm{l}\mathrm{n}n}$ with $a\in (\mathrm{0,1}/2)$. By Assumption 3 and $\mathrm{l}\mathrm{n}\left(p\right)=O\left(n^a\right)$ in Assumption 7, together with Bernstein’s inequality, we can obtain that, when $n\to \mathrm{\infty }$, there exists $\kappa >0$ such that:

$$\begin{gathered} \mathit{Pr}\left({\text{Ω}}_1\right)=1-\mathit{\text{Pr}}\left\{\underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}{\left|\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}\right|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}|>{\zeta }_n\sqrt{n}\right\}\ge 1-\sum _{k=1}^K\sum _{\ell \in {𝒮}_{j,k}^c}\mathit{\text{Pr}}\left\{{\left|\frac{1}{\sqrt{n}}\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}\right|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}|>{\zeta }_n\right\} \\ \ge 1-2\left(Kp-\sum _{k=1}^K\left|{𝒮}_{j,k}\right|\right)\text{exp}\left(-\frac{{\zeta }_n^2}{2\kappa }\right)\ge 1-2Kp\text{exp}\left(-\frac{{\zeta }_n^2}{2\kappa }\right)\to 1. \end{gathered}$$

Thus, with probability tending to 1,

$$\underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}{\left|\frac{\partial {\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}\right|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}|=O\left(n^{a/2+1/2}\sqrt{\text{ln}n}\right).$$

For $II$ in (10), applying Assumption 3 and Cauchy-Schwartz inequality yields that:

$$\begin{gathered} {\underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}{\left|{\left({\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\right)}^{\top }\frac{{\partial }^2{\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }\partial {\mathit{\theta }}_{j,C_j}}\right|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}\left|\le \underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}\sum _{i=1}^n\right|\sum _{m\in C_j}\frac{{\partial }^2\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }\partial {\theta }_{j,m}}|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}\left({\tilde{\theta }}_{j,m}-{\theta }_{j,m}^0\right)| \\ \le \underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}\sum _{i=1}^n[\sum _{m\in C_j}{\left\{{\frac{{\partial }^2\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }\partial {\theta }_{j,m}}|}_{{\mathit{\theta }}_j={\mathit{\theta }}_j^0}\right\}}^{\frac{1}{2}}\Vert {\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\Vert \\ \le \sum _{i=1}^n{\left[s_j{\left\{M_1\left(v_i\right)\right\}}^2\right]}^{\frac{1}{2}}\Vert {\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\Vert =O_p\left(K\sqrt{n}{s}_0\right). \end{gathered}$$ (11)

Similarly, for III in (10), we have:

$$\begin{gathered} \underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}{\left|{\left({\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\right)}^{\top }\frac{{\partial }^3{\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }{\partial }^2{\mathit{\theta }}_{j,C_j}}\right|}_{{\mathit{\theta }}_j={\stackrel{`}{\mathit{\theta }}}_j}\left({\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\right)| \\ \le \underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}\sum _{i=1}^n{\left[\sum _{q,m\in C_j}{\left\{{\frac{{\partial }^3\text{ln}f\left(x_{i,j};{\mathit{x}}_{i,\backslash j},{\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }\partial {\mathit{\theta }}_{j,q}\partial {\mathit{\theta }}_{j,m}}|}_{{\mathit{\theta }}_j={\stackrel{`}{\mathit{\theta }}}_j}\right\}}^2\right]}^{\frac{1}{2}}{\Vert {\tilde{\mathit{\theta }}}_{j,C_j}-{\mathit{\theta }}_{j,C_j}^0\Vert }^2\le \sum _{i=1}^n{\left[s_j^2{\left\{M_2\left(v_i\right)\right\}}^2\right]}^{\frac{1}{2}}{\Vert {\tilde{\theta }}_{j,C_j}-{\theta }_{j,C_j}^0\Vert }^2 \\ =o_p\left(K\sqrt{n}{s}_0\right). \end{gathered}$$

Then, we have:

$$\underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}{\left|\frac{\partial L_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }}\right|}_{{\mathit{\theta }}_j=\tilde{\mathit{\theta }}}|\le O_p\left(n^{a/2+1/2}\sqrt{\text{ln}n}\right)+O_p\left(K\sqrt{n}{s}_0\right)+o_p\left(K\sqrt{n}{s}_0\right).$$

Moreover, using similar discussion as that in (11), we can obtain that:

$${\underset{k\in \{1,\dots ,K\}}{\text{max}}\underset{\ell \in {𝒮}_{j,k}^c}{\text{max}}{\left({\overline{\mathit{\theta }}}_j-{\tilde{\mathit{\theta }}}_j\right)}^{\top }\frac{{\partial }^2{\tilde{L}}_n\left({\mathit{\theta }}_j\right)}{\partial {\beta }_{k,j,\ell }\partial {\mathit{\theta }}_j}|}_{{\mathit{\theta }}_j={\dot{\mathit{\theta }}}_j}\le \sum _{i=1}^n{\left[K(p+2){\left\{M_1\left(v_i\right)\right\}}^2\right]}^{\frac{1}{2}}\Vert {\overline{\mathit{\theta }}}_j-{\tilde{\mathit{\theta }}}_j\Vert =O(n\sqrt{Kp})\Vert {\overline{\mathit{\theta }}}_j-{\tilde{\mathit{\theta }}}_j\Vert .$$

Together with $∥{\stackrel{-}{\mathit{\theta }}}_j-{\tilde{\mathit{\theta }}}_j∥\le ∥{\mathit{\theta }}_j-{\tilde{\mathit{\theta }}}_j∥=O\left(n^{a/2-1/2}\sqrt{\mathrm{l}\mathrm{n}n/\left(Kp\right)}+s_0\sqrt{K/\left(pn\right)}\right)$, we obtain that:

$$L_{n1}={\tilde{L}}_n\left({\mathit{\theta }}_j\right)-{\tilde{L}}_n\left({\underset{\_}{\mathit{\theta }}}_j\right)=\sum _k\sum _{\ell \in {𝒮}_{j,k}^c}\left\{O_p\left(n^{a/2+1/2}\sqrt{\text{ln}n}\right)+O_p\left(K\sqrt{n}{s}_0\right)\right\}{\beta }_{k,j,\ell }.$$ (12)

For $L_{n2}$, let $\dot{\rho }(\beta ;\lambda ,\gamma )$ be the derivative of $\rho (\beta ;\lambda ,\gamma )$ with respect to $\beta$, more specifically,

$$\dot{\rho }(\beta ;\lambda ,\gamma )=\left(\lambda -\frac{\beta }{\gamma }\right)I\{\beta \le \lambda \gamma \}.$$

By Taylor expansion, we have:

$$L_{n2}=-n\sum _{k=1}^K\sum _{\ell \in {𝒮}_{j,k}^c}\dot{\rho }\left\{\sum _{k^{\text{'}}=1}^K\rho \left(\left|{\breve{\beta }}_{k^{\text{'}},j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\}\dot{\rho }\left(\left|{\breve{\beta }}_{k,j,\ell }\right|;\lambda ,\gamma \right)\left|{\beta }_{k,j,\ell }\right|,$$ (13)

where $\breve{\mathit{\beta }}$ lies between $\underset{\_}{\mathit{\beta }}$ and $\mathit{\beta }$. Result (13) also depends on the following fact: by Assumption $6,{\breve{\beta }}_{k,j,\ell }>\gamma \lambda$, and thus $\dot{\rho }\left(\left|{\breve{\beta }}_{k,j,\ell }\right|;\lambda ,\gamma \right)=0$ for $\ell \in {𝒮}_{j,k}$.

Note that when $\ell \in {𝒮}_{j,k}^c$, when $n\to \mathrm{\infty }$,

$$\dot{\rho }\left(\left|{\breve{\beta }}_{k,j,\ell }\right|;\lambda ,\gamma \right)\to \lambda ,$$

and

$$\sum _{k^{\text{'}}=1}^K\rho \left(\left|{\breve{\beta }}_{k^{\text{'}},j,\ell }\right|;\lambda ,\gamma \right)=\sum _{k^{\text{'}}\ne k}^K\rho \left(\left|{\breve{\beta }}_{k^{\text{'}},j,\ell }\right|;\lambda ,\gamma \right)+\rho \left(\left|{\breve{\beta }}_{k,j,\ell }\right|;\lambda ,\gamma \right)\to m_{j,\ell }\frac{\gamma {\lambda }^2}{2},$$

where $m_{j,\ell }=\#\left\{k^{\mathrm{\text{'}}}:{\beta }_{k^{\mathrm{\text{'}}},j,\ell }^0\ne 0\right\}<K$ is the number of the nonzero elements in $\left\{{\beta }_{1,j,\ell },\dots ,{\beta }_{K,j,\ell }\right\}$. Therefore, for $\ell \in {𝒮}_{j,k}^c$, when $n\to \mathrm{\infty }$,

$$\dot{\rho }\left\{\sum _{k^{\text{'}}=1}^K\rho \left(\left|{\breve{\beta }}_{k^{\text{'}},j,\ell }\right|;\lambda ,\gamma \right);\lambda ,\frac{K\lambda \gamma }{2}\right\}\dot{\rho }\left(\left|{\breve{\beta }}_{k,j,\ell }\right|;\lambda ,\gamma \right)\to \left(1-\frac{m_{j,\ell }}{K}\right){\lambda }^2.$$ (14)

Following similar arguments as in the proof of Theorem 1 in [9], and combining with the results of (12) and (14), we can obtain that when $n\to \mathrm{\infty }$,

$$L_n\left({\mathit{\theta }}_j\right)-L_n\left({\underset{\_}{\mathit{\theta }}}_j\right)=L_{n1}+L_{n2}\to \sum _k\sum _{\ell \in {𝒮}_{j,k}^c}\left\{O_p\left(n^{a/2+1/2}\sqrt{\text{ln}n}\right)+O_p\left(K{s}_0\sqrt{n}\right)\right\}{\beta }_{k,j,\ell }-n\left(1-\frac{m_{j,\ell }}{K}\right){\lambda }^2\left|{\beta }_{k,j,\ell }\right|.$$

By Assumption 5, $n^{a/2-1/2}\sqrt{\mathrm{l}\mathrm{n}n}/{\lambda }^2\to 0$ and $K{s}_0/\left(\sqrt{n}{\lambda }^2\right)\to 0$ when $n\to \mathrm{\infty }$, we can prove that $L_n\left({\mathit{\theta }}_j\right)-L_n\left({\underset{\_}{\mathit{\theta }}}_j\right)<0$, which indicates that ${\tilde{\mathit{\theta }}}_j$ is a local maximizer of $L_n\left({\mathit{\theta }}_j\right)$.

Proof of Theorem 3. From Theorem 2, we know that there exists a strict local maximizer ${\stackrel{\mathit{ˆ}}{\mathit{\theta }}}_j$ of $L_n\left({\mathit{\theta }}_j\right)$, which is asymptotically as efficient as ${\tilde{\mathit{\theta }}}_j$. Next we consider the properties of the estimators ${\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k$’s obtained from (5) and the resulting edge set ${\stackrel{\mathit{ˆ}}{\mathit{E}}}_k=\left\{(\ell ,j):1\le \ell \ne j\le p\right.$, and $\left.{\hat{\omega }}_{k,\ell j}\ne 0\right\}$. Recall that

$${\beta }_{k,j,\ell }=-{\omega }_{k,j\ell }/\sqrt{{\omega }_{k,jj}},{\tau }_{k,j}=\sqrt{{\omega }_{k,jj}}.$$

Thus,

$${\omega }_{k,j\ell }=-{\beta }_{k,j,\ell }{\tau }_{k,j},{\omega }_{k,jj}={\tau }_{k,j}^2.$$

Together with the result in Theorem 2, it’s obvious that with probability tending to 1, ${\stackrel{\mathit{ˆ}}{\mathit{E}}}_k={\mathit{E}}_k^0$. Furthermore, since $\left|{\hat{\tau }}_{k,j}-{\tau }_{k,j}^0\right|\le O_p\left(\sqrt{K{s}_0/n}\right)$ and ${\tau }_{k,j}^0$ is bounded by Assumption 6, $\left|{\hat{\tau }}_{k,j}/{\tau }_{k,j}^0-1\right|\le O_p\left(\sqrt{K{s}_0/n}\right)$. Thus for the diagonal element ${\omega }_{k,jj}$, we have:

$$\left|{\hat{\omega }}_{k,jj}-{\omega }_{k,jj}^0\right|=\left|{\hat{\tau }}_{k,j}^2-{\left({\tau }_{k,j}^0\right)}^2\right|=\left|{\hat{\tau }}_{k,j}-{\tau }_{k,j}^0\right|\left({\hat{\tau }}_{k,j}+{\tau }_{k,j}^0\right)=O_p\left\{\sqrt{K{s}_0/n}\left(\underset{k}{\text{max}}\underset{j}{\text{max}}{\tau }_{k,j}^0\right)\right\}=O_p\left(\sqrt{K{s}_0/n}\right).$$

Together with $\left|{\hat{\beta }}_{k,j,\ell }-{\beta }_{k,j,\ell }^0\right|\le O_p\left(\sqrt{K{s}_0/n}\right)$, for the off-diagonal element ${\omega }_{k,j\ell }$ with $j\ne \ell$, we have:

$$\begin{gathered} \left|{\hat{\omega }}_{k,j\ell }-{\omega }_{k,j\ell }^0\right|=\left|{\hat{\beta }}_{k,j,\ell }{\hat{\tau }}_{k,j}-{\beta }_{k,j,\ell }^0{\tau }_{k,j}^0\right|\le \left(\left|{\hat{\beta }}_{k,j,\ell }{\hat{\tau }}_{k,j}-{\beta }_{k,j,\ell }^0{\hat{\tau }}_{k,j}\right|+\left|{\beta }_{k,j,\ell }^0{\hat{\tau }}_{k,j}-{\beta }_{k,j,\ell }^0{\tau }_{k,j}^0\right|\right) \\ \le {\hat{\tau }}_{k,j}\left|{\hat{\beta }}_{k,j,\ell }-{\beta }_{k,j,\ell }^0\right|+\left|\frac{{\hat{\tau }}_{k,j}}{{\tau }_{k,j}^0}-1\right|\left|{\omega }_{k,j\ell }^0\right|=O_p\left(\sqrt{K{s}_0/n}\right). \end{gathered}$$

Thus, we conclude that ${\mathrm{m}\mathrm{a}\mathrm{x}}_{k,j,\ell }\left|{\hat{\omega }}_{k,j\ell }-{\omega }_{k,j\ell }^0\right|=O_p\left(\sqrt{K{s}_0/n}\right)$.

Similarly,

$$\begin{gathered} \sum _{k=1}^K{\Vert {\hat{\Omega }}_k-{\Omega }_k^0\Vert }_{\infty }=\sum _{k=1}^K\underset{j}{\text{max}}\left\{\sum _{\ell =1}^p\left|{\hat{\omega }}_{k,j\ell }-{\omega }_{k,j\ell }^0\right|\right\}=\sum _{k=1}^K\underset{j}{\text{max}}\left\{\sum _{\ell \ne j}\left|{\hat{\beta }}_{k,j,\ell }{\hat{\tau }}_{k,j}-{\beta }_{k,j,\ell }^0{\tau }_{k,j}^0\right|+\left|{\hat{\tau }}_{k,j}^2-{\left({\tau }_{k,j}^0\right)}^2\right|\right\} \\ \le \sum _{k=1}^K\underset{j}{\text{max}}\left\{\sum _{\ell \ne j}\left(\left|{\hat{\beta }}_{k,j,\ell }{\hat{\tau }}_{k,j}-{\beta }_{k,j,\ell }^0{\hat{\tau }}_{k,j}\right|+\left|{\beta }_{k,j,\ell }^0{\hat{\tau }}_{k,j}-{\beta }_{k,j,\ell }^0{\tau }_{k,j}^0\right|\right)+\left|{\hat{\tau }}_{k,j}-{\tau }_{k,j}^0\right|\left({\hat{\tau }}_{k,j}+{\tau }_{k,j}^0\right)\right\} \\ \le \sum _{k=1}^K\underset{j}{\text{max}}\left\{\left|{\hat{\tau }}_{k,j}-{\tau }_{k,j}^0\right|\sum _{\ell \ne j}\left|{\beta }_{k,j,\ell }^0\right|+{\hat{\tau }}_{k,j}\sum _{\ell \ne j}\left|{\hat{\beta }}_{k,j,\ell }-{\beta }_{k,j,\ell }^0\right|+\left|{\hat{\tau }}_{k,j}-{\tau }_{k,j}^0\right|\left({\hat{\tau }}_{k,j}+{\tau }_{k,j}^0\right)\right\} \\ \le O_p\left(\sqrt{K{s}_0/n}\underset{j}{\text{max}}\sum _{k=1}^K\sum _{\ell \ne j}\left|{\beta }_{k,j,\ell }^0\right|+K{s}_0/\sqrt{n}+\sqrt{K^3{s}_0/n}\right)=O_p\left(\sqrt{K^3{s}_0^3/n}\right). \end{gathered}$$

Besides,

$$\begin{gathered} \sum _{k=1}^K{\Vert {\left({\hat{\Omega }}_k-{\Omega }_k^0\right)}_{𝒜}\Vert }_F=\sum _{k=1}^K{\left\{\sum _{j=1}^p\sum _{\ell \ne j}{\left({\hat{\omega }}_{k,j\ell }-{\omega }_{k,j\ell }^0\right)}^2\right\}}^{1/2}=\sum _{k=1}^K{\left\{\sum _{j=1}^p\sum _{\ell \ne j}{\left({\hat{\beta }}_{k,j,\ell }{\hat{\tau }}_{k,j}-{\beta }_{k,j,\ell }^0{\tau }_{k,j}^0\right)}^2\right\}}^{1/2} \\ \le \sum _{k=1}^K{\left\{2\sum _{j=1}^p{\left({\hat{\tau }}_{k,j}-{\tau }_{k,j}^0\right)}^2\sum _{\ell \ne j}{\left({\beta }_{k,j,\ell }^0\right)}^2+2\sum _{j=1}^p{\hat{\tau }}_{k,j}^2\sum _{\ell \ne j}{\left({\hat{\beta }}_{k,j,\ell }-{\beta }_{k,j,\ell }^0\right)}^2\right\}}^{1/2} \\ \le \sum _{k=1}^K{\left\{2\sum _{j=1}^p{\left(\frac{{\hat{\tau }}_{k,j}}{{\tau }_{k,j}^0}-1\right)}^2\sum _{\ell \ne j}{\left({\omega }_{k,j\ell }^0\right)}^2+2\sum _{j=1}^p{\hat{\tau }}_{k,j}^2\sum _{\ell \ne j}{\left({\hat{\beta }}_{k,j,\ell }-{\beta }_{k,j,\ell }^0\right)}^2\right\}}^{1/2} \\ \le O_p\left(\sqrt{K{s}_0/n}\right)\sum _{k=1}^K{\left\{\sum _{j=1}^p\sum _{\ell \ne j}{\left({\omega }_{k,j\ell }^0\right)}^2\right\}}^{1/2}+O_p\left[\sum _{k=1}^K{\left\{\sum _{j=1}^p\sum _{\ell \ne j}{\left({\hat{\beta }}_{k,j,\ell }-{\beta }_{k,j,\ell }^0\right)}^2\right\}}^{1/2}\right]\le O_p\left(\sqrt{K^3{s}_{0}s/n}\right)\le O_p\left(\sqrt{K^3{s}^2/n}\right). \end{gathered}$$

Table 2:

Simulation results under the scenarios with the nearest-neighbor networks and $K=3$. In each cell, we show the median (median absolute deviation) of the CE, PME, TPR and FPR values based on 100 replicates. $\mathrm{C}\mathrm{E}={\left(C_n^2\right)}^{-1}\left|\left\{(i,j):I\left\{\hat{\phi }\left({\mathit{x}}_i\right)=\hat{\phi }\left({\mathit{x}}_j\right)\right\}\ne I\left\{\phi \left({\mathit{x}}_i\right)=\phi \left({\mathit{x}}_j\right)\right\},i<j\right\}\right|,\mathrm{P}\mathrm{M}\mathrm{E}={\sum }_{k=1}^K{∥{\stackrel{\mathbf{ˆ}}{\mathbf{\Omega }}}_k-{\mathbf{\Omega }}_k^0∥}_F/K$, $\mathrm{TPR}={\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0\ne 0\right\}\right]/K$, and $\mathrm{FPR}={\sum }_{k=1}^K\left[{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0,{\hat{\omega }}_{k,\ell j}\ne 0\right\}/{\sum }_{\ell <j}I\left\{{\omega }_{k,\ell j}^0=0\right\}\right]/K$.

Approach	CE	PME	TPR	FPR
S1 with the balanced design
proposed	$0.000\left(0.000\right)$	$18.171\left(0.524\right)$	$0.982\left(0.005\right)$	$0.037\left(0.002\right)$
SCAN	$0.000\left(0.000\right)$	$34.619\left(0.392\right)$	$0.710\left(0.008\right)$	$0.076\left(0.001\right)$
Tiger	-	$37.508\left(0.207\right)$	$0.817\left(0.016\right)$	$0.184\left(0.004\right)$
True+Tiger	-	$22.575\left(0.616\right)$	$0.862\left(0.007\right)$	$0.068\left(0.001\right)$
True+JGL	-	$22.317\left(0.562\right)$	$0.904\left(0.008\right)$	$0.178\left(0.004\right)$
True+JSEM	-	$18.468\left(0.498\right)$	$0.964\left(0.005\right)$	$0.084\left(0.003\right)$
S2 with the balanced design
proposed	$0.000\left(0.000\right)$	$17.441\left(0.515\right)$	$0.987\left(0.003\right)$	$0.037\left(0.001\right)$
SCAN	$0.002\left(0.002\right)$	$36.546\left(0.699\right)$	$0.713\left(0.008\right)$	$0.073\left(0.002\right)$
Tiger	-	$40.345\left(0.245\right)$	$0.825\left(0.013\right)$	$0.183\left(0.004\right)$
True+Tiger	-	$23.529\left(0.654\right)$	$0.867\left(0.006\right)$	$0.066\left(0.001\right)$
True+JGL	-	$21.983\left(0.497\right)$	$0.925\left(0.016\right)$	$0.190\left(0.020\right)$
True+JSEM	-	$17.508\left(0.603\right)$	$0.980\left(0.005\right)$	$0.071\left(0.003\right)$
S3 with the balanced design
proposed	$0.000\left(0.000\right)$	$17.637\left(0.507\right)$	$0.990\left(0.003\right)$	$0.037\left(0.002\right)$
SCAN	$0.002\left(0.002\right)$	$34.577\left(0.543\right)$	$0.739\left(0.005\right)$	$0.078\left(0.001\right)$
Tiger	-	$38.052\left(0.304\right)$	$0.808\left(0.015\right)$	$0.192\left(0.006\right)$
True+Tiger	-	$24.609\left(0.612\right)$	$0.852\left(0.008\right)$	$0.072\left(0.001\right)$
True+JGL	-	$22.986\left(0.589\right)$	$0.919\left(0.014\right)$	$0.215\left(0.008\right)$
True+JSEM	-	$19.003\left(0.477\right)$	$0.957\left(0.005\right)$	$0.073\left(0.003\right)$
S1 with the imbalanced design
proposed	$0.000\left(0.000\right)$	$18.582\left(0.864\right)$	$0.977\left(0.003\right)$	$0.037\left(0.002\right)$
SCAN	$0.000\left(0.000\right)$	$33.991\left(0.492\right)$	$0.710\left(0.008\right)$	$0.075\left(0.001\right)$
Tiger	-	$37.336\left(0.196\right)$	$0.824\left(0.010\right)$	$0.181\left(0.004\right)$
True+Tiger	-	$23.089\left(0.447\right)$	$0.862\left(0.005\right)$	$0.071\left(0.001\right)$
True+JGL	-	$22.730\left(0.562\right)$	$0.907\left(0.006\right)$	$0.179\left(0.002\right)$
True+JSEM	-	$18.693\left(0.413\right)$	$0.962\left(0.005\right)$	$0.083\left(0.004\right)$
S2 with the imbalanced design
proposed	$0.000\left(0.000\right)$	$17.526\left(0.909\right)$	$0.980\left(0.008\right)$	$0.038\left(0.002\right)$
SCAN	$0.002\left(0.002\right)$	$35.934\left(0.459\right)$	$0.718\left(0.005\right)$	$0.072\left(0.002\right)$
Tiger	-	$40.089\left(0.176\right)$	$0.817\left(0.008\right)$	$0.178\left(0.003\right)$
True+Tiger	-	$24.006\left(0.455\right)$	$0.863\left(0.007\right)$	$0.071\left(0.002\right)$
True+JGL	-	$22.817\left(0.562\right)$	$0.919\left(0.013\right)$	$0.176\left(0.005\right)$
True+JSEM	-	$17.546\left(0.663\right)$	$0.980\left(0.003\right)$	$0.071\left(0.004\right)$
S3 with the imbalanced design
proposed	$0.000\left(0.000\right)$	$17.688\left(0.609\right)$	$0.987\left(0.003\right)$	$0.038\left(0.001\right)$
SCAN	$0.002\left(0.002\right)$	$34.453\left(0.414\right)$	$0.739\left(0.008\right)$	$0.076\left(0.002\right)$
Tiger	-	$38.031\left(0.239\right)$	$0.824\left(0.012\right)$	$0.192\left(0.005\right)$
True+Tiger	-	$25.031\left(0.467\right)$	$0.857\left(0.005\right)$	$0.076\left(0.002\right)$
True+JGL	-	$23.854\left(0.716\right)$	$0.921\left(0.010\right)$	$0.219\left(0.005\right)$
True+JSEM	-	$19.367\left(0.456\right)$	$0.957\left(0.005\right)$	$0.074\left(0.004\right)$