ABSTRACT
Network deconvolution (ND) is a method to reconstruct a direct-effect network describing direct (or conditional) effects (or associations) between any two nodes from a given network depicting total (or marginal) effects (or associations). Its key idea is that, in a directed graph, a total effect can be decomposed into the sum of a direct and an indirect effects, with the latter further decomposed as the sum of various products of direct effects. This yields a simple closed-form solution for the direct-effect network, facilitating its important applications to distinguish direct and indirect effects. Despite its application to undirected graphs, it is not well known why the method works, leaving it with skepticism. We first clarify the implicit linear model assumption underlying ND, then derive a surprisingly simple result on the equivalence between ND and use of precision matrices, offering insightful justification and interpretation for the application of ND to undirected graphs. We also establish a formal result to characterize the effect of scaling a total-effect graph. Finally, leveraging large-scale genome-wide association study data, we show a novel application of ND to contrast marginal versus conditional genetic correlations between body height and risk of coronary artery disease; the results align with an inferred causal directed graph using ND. We conclude that ND is a promising approach with its easy and wide applicability to both directed and undirected graphs.
Keywords: direct-effect network, directed graph, Gaussian graphical model; marginal correlation, partial correlation, total-effect graph
1. INTRODUCTION
Networks, also called graphs exchangeably, either directed or undirected, depicting complex relationships among variables, have emerged as an important tool in biology, medicine, economics and social sciences, engineering, and other fields. Although a so-called total-effect graph describing a total effect of any node on the other (for a directed graph), or a marginal association between any pair of nodes (for an undirected graph) (Becker et al., 2023) is often easier to obtain, it consists of both direct and indirect effects, while it is of great interest to disentangle the two types of effects (Kang et al., 2015). Partly because it is often impossible to do so experimentally, one has to rely on computational approaches. In our motivating example, given that body height is marginally associated with the risk of coronary artery disease (CAD) and also with other risk factors of CAD, a natural question is whether height is a direct risk factor of CAD (after conditioning on or adjusting for other risk factors). See Lin et al. (2023) for another example on inferring direct versus indirect risk factors causal for cardiovascular disease, which has immediate implications for intervention. Hence it is a common problem to reconstruct a direct-effect graph (depicting direct/conditional effects/associations) from an observed total-effect graph. Network deconvolution (ND) was proposed as a general approach for such a purpose (Feizi et al., 2013), garnering more than 319 citations by a Google Scholar search as of January 2024; more example applications are detailed in Web Appendix A. Briefly, ND decomposes a total-effect network as the sum of a direct-effect network and an indirect network, the latter of which is further decomposed as the sum of an infinite power series of the direct-effect network. However, as to be explained later, its justification depends on linear (causal) structural models for directed graphs, while it remains largely unclear or even controversial why/whether it can be applied to undirected graphs. This may be concerning in spite of its proposed applications to undirected graphs (Feizi et al., 2013); in particular, it is natural to ask what is obtained at the end of its application. This problem has been pointed out by others (Pachter, 2014), along with a simulation study to raise skepticism about its performance, especially compared with the use of (regularized) covariance/precision matrices and Gaussian graphical models (GGMs) (Pachter, 2014). On the other hand, Alipanahi and Frey (2013) stated that applying ND to a correlation matrix resulted in the corresponding negative precision matrix (on the off-diagonal elements), offering some insights and strong support for ND. Unfortunately, this result does not seem to be widely known, and importantly, neither its condition nor derivation was given, which motivated the current work. Furthermore, we discuss the connections of ND to other methods, such as a shrinkage estimator of covariance matrix (Schäfer and Strimmer, 2005), the graphical lasso (Friedman et al., 2008), and some nonparametric rank-based methods (Xue and Zou, 2012; Liu et al., 2012); we also establish a formal result for scaling an undirected total-effect graph as recommended in Feizi et al. (2013), which is found to be closely related to a shrinkage estimator of covariance matrix.
In summary, we aim to offer some insights on the application of ND to undirected (association) graphs, formally showing the connection between ND and the approach of using (marginal) correlation and precision matrices (ie, use of marginal and partial correlations) for undirected graphs. We start by introducing ND and its application on directed graphs in Section 2.1, then present a derivation of the connection to support ND’s application to undirected graphs in Section 2.2. For illustration, some simple numerical examples are given in Section 3.1. Then, by leveraging large-scale genome-wide association study (GWAS) summary data, we consider an application to infer the relationship between CAD and body height in Section 3.2. We first confirm the well-established marginal association between CAD and height, then show that an application of ND suggests no direct association after conditioning on two other risk factors, low-density lipoprotein (LDL) cholesterol and pulse pressure. For comparison, we also apply a published method to the same data to infer a causal directed graph, reaching the same conclusion. We further clarify the effects of scaling an undirected graph, discuss its connection with the regularized estimator of a direct-effect network, and compare their numerical performance with high-dimensional data in Section 4. We end the paper with a short discussion section.
2. NETWORK DECONVOLUTION
2.1. Application of ND on directed graphs
The idea of ND stems from a directed graph among a set of variables/nodes of interest, say 
, where the total effect from one node to another can be decomposed into its direct effect and possible indirect effects through other nodes (see Figure 1A in Feizi et al. (2013)). Accordingly, Feizi et al. (2013) start by modeling an observed total effect in a total-effect network 
as the sum of the direct effect given by the direct-effect network 
and many indirect effects via paths of increasing lengths in 
, 
, 
as follows:
 |
(1) |
A simple and intuitive justification of Equation 1 is available only for directed graphs. Specifically, the total effect 
from 
to 
is decomposed into the direct effect 
from 
to 
, and the indirect effect 
from 
to 
passing through any single node 
in 
, and the indirect effect 
from 
to 
passing through any two nodes 
and 
in 
, and so on. See a simple illustrative example in Lin et al. (2023). We clarify that the derivation depends on an implicit assumption of linear (causal) structural equation models: for any 
, if there is a direct effect from 
to 
, we have
 |
(2) |
where any 
if and only if there is a direct effect of 
on 
, and 
is a random error. The direct-effect model induces a total-effect model:
 |
(3) |
where 
is a random error. We assume that, without loss of generality, each variable 
is already standardized to have mean 0 and variance 1, so no intercept is needed.
Then, under the assumption that the spectral radius of 
is smaller than 1, the infinite series summation in Equation 1 will converge to 
(Young, 1981); and we have a closed-form solution:
 |
(4) |
It is noted that for a directed acyclic graph (DAG) 
, we have 
for any integer 
, thus Equation 1 always converges. Furthermore, 
is nilpotent with all eigenvalues equal to 0 (Fieller, 2018), implying that its spectral radius is always smaller than 1.
A sufficient condition for the identifiability of model Equation 2 is that the model is recursive, that is, if we write Equation 2 in matrix form 
, then 
is lower triangular and 
is diagonal (ie, errors are uncorrelated) (Bollen, 1989). In general, achieving identifiability requires some conditions or use of interventions or instrumental variables; for example, see a formal result in Section 2.1 of Li et al. (2023) for DAGs. A more general discussion is beyond the scope of this paper; we refer the reader to Bollen (1989); Drton et al. (2011); Foygel et al. (2012); Drton and Weihs (2016). Furthermore, if a total-effect graph 
is identifiable (or consistently estimable), then under suitable conditions, by the continuous mapping theorem, ND can be applied to achieve the same for the direct-effect graph 
.
2.2. Application of ND on undirected graphs
An important and intriguing question is whether ND can be applied to undirected graphs, and if so, why, considering its justification seems to depend on directed graphs. Feizi et al. (2013) proposed to use the Pearson correlation between each pair of nodes to measure the total effects in 
, with the diagonal set to zero. Alipanahi and Frey (2013) stated, but without showing its derivation or conditions, that the resulting 
of applying ND to such a 
is the corresponding negative precision matrix (on the off-diagonal elements). Here we offer a formal derivation, along with its (implicit) assumptions.
We take a perhaps trivial but intuitive approach by converting an undirected graph into a directed graph: each edge in the former can be regarded as a bi-directional edge with the same effect size in either direction in the latter. Treating Equation 3 as (marginal) regression models for association analysis, instead of causal models, we retain the symmetry property of 
: 
for any 
(because each variable 
is standardized to have mean 0 and variance 1). Then, we can apply ND as before for a directed graph. A natural question is what exactly is the resulting 
. In particular, we’d like to know whether/how 
is related to the regression coefficients in the joint regression models: for any 
,
 |
(5) |
where 
gives conditional association between 
and 
, and 
is a random error.
The following proposition provides a surprisingly simple answer.
Proposition 1
Let
be the invertible correlation matrix among the set of variables of interest. If
, then
, where
is the precision matrix.
Proof.
Substituting
in Equation 4, we have
This proposition gives us an insight on ND: when 
is symmetric, consisting of marginal correlations (in the off-diagonal and of 0’s in the diagonal), the resulting 
encodes exactly the same structure of the recovered direct-effect (or conditional association) graph as that by partial correlations. To be specific, denote the precision matrix 
, and the direct-effect network 
, then we have
 |
where 
is the partial correlation between 
and 
given the remaining variables. This result is in agreement with the statement in Alipanahi and Frey (2013), though they did not specify the relationship between the diagonals. It is well known that 
in Equation 5 (Lauritzen, 1996), hence Proposition 1 suggests that, starting from marginal regression models Equation 3, applying ND recovers the structure (ie, non-zero regression coefficients) of the corresponding joint regression models Equation 5, as does covariance selection. Furthermore, under the normality assumption as in the GGM, the (direct-effect) graph structure corresponds to non-zero elements of the precision matrix (or equivalently, 
): there is an edge between 
and 
if and only if 
, indicating conditional dependence between 
and 
(Lauritzen, 1996).
Remark 1
Although the original derivation of ND requires the assumption that the spectral radius of the direct-effect graph
is smaller than one (to ensure the convergence of Equation 1), Proposition 1 says that applying Equation 4 on
will always return the negative precision matrix (except for the diagonal) regardless of this assumption. In particular, when
has some eigenvalues smaller than −1, resulting in a spectral radius greater than 1, the precision matrix
is still positive definite.
Remark 2
Proposition 1 holds regardless of whether an estimated (eg, sample) correlation matrix or a true/population correlation matrix is used for
. This implies that, when a shrinkage estimator of the correlation matrix is used in ND (Schäfer and Strimmer, 2005), the resulting
will coincide with the network structure induced by the corresponding partial correlation matrix; the latter of which was used in the simulation in Pachter (2014). And in fact, as we will show later in Section 4.1 and Section 4.2, linearly scaling the total-effect matrix before applying ND is equivalent to using a shrinkage estimator of the correlation matrix in ND. Furthermore, if the inverse of the GLASSO estimator of the precision matrix (with a non-zero penalty parameter) is used as
(which is invertible) (Friedman et al., 2008), ND will yield
leading to the same network structure induced by the GLASSO estimate of the precision matrix. These results may facilitate the use of ND for high-dimensional data, which is explored in Section 4.
Remark 3
Instead of using Pearson’s correlation coefficient to measure marginal associations in
, one may consider a nonparametric rank-based correlation coefficient, that is, Spearman’s rho or Kendall’s tau, which can be (asymptotically) transformed to Pearson’s correlation coefficient (Kendall, 1948; Kruskal, 1958). Furthermore, under the nonparanormal assumption on the data distribution, one can still consistently estimate the precision matrix even for high-dimensional data while obtaining the same interpretation of a GGM (ie, the equivalence between the independence and no conditional association represented in the precision matrix) (Xue and Zou, 2012; Liu et al., 2012). These results imply that, if one of the two rank-based correlation coefficients is suitably transformed and then used in
, ND will yield
, where
is asymptotically a precision matrix, which can be interpreted as for a GGM under the nonparanormal assumption on the data. These results may facilitate the use of ND for non-Gaussian data, which is explored in Section 4.3.
Remark 4
In the above application of ND, it may appear strange to set
, instead of
. This recipe was suggested without any justification by Feizi et al. (2013), while a justification is offered here. On the other hand, it is known now that setting
does not work for directed graphs with cycles (Lin et al., 2023).
To summarize, ND can be applied to an undirected total-effect graph 
(with off-diagonals as marginal correlations and diagonals as 0), yielding a direct-effect graph 
equal to the negative precision matrix (except on diagonals).
3. EXAMPLES
3.1. Simple numerical examples
In this section, we present two simple examples to illustrate Proposition 1, and in passing, compare ND on directed graphs and undirected graphs. We first show an example where the same direct-effect graph structure is shared by both directed and undirected graphs. Consider a true (causal) direct graph shown in Figure 1A, then we apply ND on an undirected total-effect graph using the true correlations as follows:
FIGURE 1.
Examples: true causal graphs among 3 standardized variables 
. Panel (A): 
is a mediator. Panel (B): 
is a collider.
 |
(6) |
And by inverting the correlation matrix, we obtain the precision matrix, from which we can easily verify Proposition 1 holds:
 |
We also apply ND on a directed total-effect graph as follows:
 |
(7) |
We can see that in the resulting 
in both Equations 7 and 6, there is no direct link between 
and 
, which aligns with the true graph shown in Figure 1A.
Next, we show an example where the structures of the resulting direct-effect graphs can be different when ND is applied to a directed graph and its corresponding undirected (association) graph. Consider the true causal graph shown in Figure 1B, we apply ND on the undirected and directed total-effect graphs in Equations 8 and 9, respectively:
 |
(8) |
 |
(9) |
In this example, the resulting 
in Equation 8 suggests a direct edge between 
and 
while the one in Equation 9 does not. Nonetheless, it is easy to verify that 
and 
.
3.2. A real data example
We present two applications of ND to investigate the relationship between height and CAD using genome-wide association study (GWAS) summary statistics. It is well-established that shorter stature is associated with a higher risk of CAD in large observational studies (Collaboration, 2012), but the underlying mechanism still remains unclear. At the same time, shorter stature is associated with several well-known risk factors for CAD. For instance, greater height was associated with lower LDL cholesterol levels (Oh et al., 2019), which is a significant causal risk factor for CAD (Karalis, 2009). Additionally, taller individuals were found to have lower pulse pressure (PP, defined as the difference between systolic and diastolic blood pressure) (Bourgeois et al., 2017), while higher pulse pressure has been identified as an important risk factor for CAD (Franklin et al., 1999). Langenberg et al. (2003) also hypothesized that the association between height and CAD was partly mediated by increased PP. We therefore would like to disentangle the direct (genetic) dependency among height, LDL, pulse pressure, and CAD, which can be accomplished by application of ND. We used publicly available large-scale GWAS summary datasets for height, CAD, LDL, and PP in our analysis (Web Appendix D).
In the first application, we applied ND on an undirected genetic correlation graph. A trait can be decomposed into genetic and environmental components, and a (marginal) genetic correlation measures the correlation of the genetic components of two traits, which has been widely used to study the overall genetic similarity between two traits; see a brief introduction on genetic correlation in Web Appendix B, and Van Rheenen et al. (2019) for a comprehensive review. Notably, such a genetic correlation can be estimated through two separate GWAS datasets, one for each trait; that is, we do not need to have a GWAS dataset including two or more traits. As for other types of correlations, a conditional (or partial) genetic correlation can be similarly defined and inferred using multiple regression or a precision matrix (Werme et al., 2022); here we used ND. Specifically, we estimated a (marginal) genetic correlation graph by estimating its each element 
(and the corresponding 
-value) for two traits i and j using LD-score regression (LDSC) (Bulik-Sullivan et al., 2015). LDSC assumes a polygenic model and estimates the genetic correlation considering effects from genome-wide SNPs. The number of SNPs used in estimating genetic correlation for each pair of traits is provided in Web Table 1. To infer the partial genetic correlation graph, we adopted a parametric bootstrap scheme. We first generated 500 bootstrap samples 
, where the covariance matrix among the 6 genetic correlation estimates 
was estimated via the block-jackknife method implemented in LDSC. Then we applied ND to each bootstrap sample to obtain 
for 
. A 
-value for each entry of 
(to test for the presence of the edge) was calculated element-wise by comparing a Z-statistic to the standard normal distribution; the Z-statistic was calculated using the bootstrap estimates 
and the bootstrap standard errors.
TABLE 1.
MCC results for different graph structures and sample sizes.
| Structure (Sparsity) | N | ND | ND-FK | ND-CV | ND-SS | GLASSO |
|---|---|---|---|---|---|---|
| Random (0.05) | 20 | NA | 0.29 | 0.30 | 0.30 | 0.22 |
| Random (0.05) | 200 | 0.63 | 0.54 | 0.64 | 0.68 | 0.65 |
| Random (0.05) | 500 | 0.79 | 0.54 | 0.74 | 0.77 | 0.75 |
| Random (0.2) | 20 | NA | 0.16 | 0.18 | 0.18 | 0.11 |
| Random (0.2) | 200 | 0.44 | 0.26 | 0.56 | 0.55 | 0.57 |
| Random (0.2) | 500 | 0.68 | 0.21 | 0.69 | 0.69 | 0.67 |
| Random (0.4) | 20 | NA | 0.08 | 0.11 | 0.11 | 0.06 |
| Random (0.4) | 200 | 0.36 | 0.10 | 0.47 | 0.47 | 0.45 |
| Random (0.4) | 500 | 0.60 | 0.07 | 0.61 | 0.61 | 0.53 |
| Circular | 20 | NA | 0.30 | 0.48 | 0.47 | 0.62 |
| Circular | 200 | 0.98 | 0.25 | 0.89 | 0.86 | 0.76 |
| Circular | 500 | 0.96 | 0.22 | 0.93 | 0.93 | 0.73 |
Figure 2 shows the estimated marginal genetic correlation graph (left) and partial genetic correlation graph (right); only statistically significant correlations/edges are shown with a Bonferroni-corrected 
-value threshold of 
. In Figure 2A, consistent with the literature, we observed a significant negative genetic correlation between height and CAD, as well as negative genetic correlations between height and the other two CAD risk factors, LDL and PP, which suggested that genetic factors contributing to greater height were associated with a lower risk of developing CAD, lower levels of LDL and PP. In Figure 2B, conditioning on LDL and PP led to a substantial reduction in the genetic correlation between height and CAD, which was no longer statistically significant. This suggested that the shared genetic component between height and CAD was largely driven by LDL and PP. One possible mechanism leading to this observation is that LDL and PP act as mediators between height and CAD, instead of confounders, given that it is unlikely for LDL or PP to directly affect one’s height.
FIGURE 2.
The estimated marginal genetic correlation graph (Panel A) and its corresponding partial/conditional genetic correlation graph from ND (Panel B). Blue edges represent positive effects and purple edges represent negative effects. Only significant edges identified at the Bonferroni-adjusted significance level of 0.05/6 are shown.
For comparison, in the second application we first inferred a directed causal graph for total effects of each trait on others before applying ND for a direct-effect graph. The total causal effect from one trait to the other can be estimated by Mendelian randomization (MR). MR is an instrumental variable (IV) method that uses genetic variants associated with the exposure as IVs to infer the causal relationship of an exposure on an outcome; see a brief introduction on MR in Web Appendix C, and Burgess et al. (2019) for a comprehensive review. In this application, we took the approach of Graph-MRcML Lin et al. (2023): each (causal directed) total-effect 
was estimated by applying MR with trait i as the exposure and trait j as the outcome, then the direct-effect graph was estimated by applying ND to the total-effect graph 
. Statistical inference was again conducted using a resampling approach; see Lin et al. (2023) for details.
The estimated directed total-effect graph (left) and direct-effect graph (right) are shown in Figure 3. A Bonferroni-corrected 
-value threshold of 
was used. We observed significant negative total effects of height on CAD, LDL, and PP in Figure 3A. After adjusting for LDL and PP, the direct effect of height on CAD was not significant anymore in Figure 3B.
FIGURE 3.
The estimated directed total-effect graph (Panel A) and corresponding direct-effect graph from ND (Panel B). Blue edges represent positive effects and purple edges represent negative effects. Only significant edges identified at the Bonferroni-adjusted significance level of 0.05/12 are shown.
It is notable that the resulting direct-effect graphs obtained from applying ND on the undirected (Figure 2B) and directed graphs (Figure 3B) are in good agreement; for convenience, we call a marginal (or partial) association as a total (or direct) effect (in an undirected graph). For example, first, consistently, the link between height and CAD was not significant in either of the two direct-effect graphs (after conditioning on the other two CAD risk factors, LDL and PP). This scenario was similar to our first numerical example, Figure 1A, where LDL and PP served as the mediators between height and CAD. This finding suggested that the association between shorter stature and an increased risk of CAD may be largely explained by the mediating effects through an adverse lipid profile and high pulse pressure. Our findings align well with previous evidence. For example, using 180 height-associated variants, Nelson et al. (2015) found that the link between genetically determined shorter height and an increased risk of CAD was partly mediated by LDL. Furthermore, in individuals of shorter height, reflected waves tend to arrive earlier and during systole, leading to increased central pressure augmentation and higher pulse pressure (O’Rourke and Nichols, 1998). Increased pulse pressure elevation was shown to induce endothelial dysfunction (Ryan et al., 1995), a type of non-obstructive CAD. Second, we also found that the partial genetic correlation between LDL and PP was nominally significant in the undirected direct-effect graph (with 
-value=0.03, edge not shown in Figure 2B) even though their total/marginal genetic correlation was absent with 
-value=0.4. This observation may be attributed to CAD acting as a collider between LDL and PP (ie, LDL 
CAD 
PP), as illustrated in Figure 3B. As demonstrated in our second numerical example (Figure 1B), conditioning on a collider leads to a non-zero partial correlation. Finally, it is important to keep in mind that differences between the two methods we used, namely LDSC and MR, for estimating the undirected and directed total-effect graphs will generally contribute to discrepancies between the undirected and directed direct-effect graphs. For example, LDSC used genome-wide genetic variants to infer the undirected genetic correlation graph, while MR only used a few significant genetic variants under strong IV assumptions to infer the directed total causal graph. Nonetheless, despite differences of the two methods and their assumptions in estimating the directed and undirected total-effect graphs while the same ND was equally applied to obtain their corresponding direct-effect graphs, the overall agreements in this application lend strong support to the conclusions.
4. ND ON HIGH-DIMENSIONAL DATA
4.1. Effects of scaling an undirected total-effect graph
As discussed in Section 2.1, one crucial assumption of ND is that the spectral radius of the direct-effect graph 
is smaller than 1 to ensure that the right-hand side of Equation 1 converges and Equation 4 holds. Unfortunately, this assumption may not always hold in practice, and Feizi et al. (2013) proposed to linearly scale the total-effect graph 
before applying ND (Equation 4) to the scaled total-effect graph, yielding 
with 
. Feizi et al. (2013) further suggested 
, where 
, 
and 
are the largest positive and smallest negative eigenvalues of 
. While such a scaling can guarantee that the spectral radius of the resulting direct-effect matrix (
) is less than one, it remains controversial as different scaling factors 
(or equivalently 
in their recipe) will lead to different direct-effect graphs. More importantly, what exactly is the resulting 
of applying ND to a scaled total-effect graph 
? The following proposition gives an answer to this intriguing question when 
is an undirected total-effect graph and constructed based on the Pearson correlation for each pair of nodes as discussed in Section 2.2. We will provide a data-driven approach to estimate the scaling factor in Section 4.2.
Proposition 2
Let
be a positive scaling factor, and
be the scaled total-effect graph. If the unscaled total-effect graph
, then applying ND to
results in direct-effect graph
;
is the precision matrix of variables
, where
,
. That is,
, where
.
Proof.
Denote the eigendecomposition of the covariance matrix as
, where
gives the eigenvalues. Substituting the scaled total-effect graph
in Equation 4, we have
where
.
Proposition 2 provides both an interpretation of and a justification for applying ND on a scaled total-effect graph consisting of marginal Pearson’s correlations: the resulting direct-effect graph encodes the structure of conditional dependence among a set of transformed variables 
, which however may be different from that of the original variables 
. This phenomenon is similar to that in measurement error models (Strauss, 1981; Fuller, 2009). Intuitively, adding noises to the original variables (by scaling a total-effect graph 
) is expected to make it statistically more difficult to infer the structure of the corresponding 
or the conditional dependence matrix of the original variables 
. Another perspective on the scaling factor in high-dimensional problems is discussed next in Section 4.2.
Proposition 2 also clearly shows why scaling with an 
small enough can always yield a direct-effect graph 
with its spectral radius less than 1; see also a formal proof in the supplementary of Feizi et al. (2013). Any eigenvalue of 
is in the form of 
, which tends to 0 as 
. In contrast, any eigenvalue of 
from an unscaled 
is in the form of 
, whose absolute value can be equal to or larger than 1 if 
(or if 
, which however is unlikely because 
is an eigenvalue of the covariance matrix 
that is expected to be semi-positive definite). Finally, it is noted that the assumption of the spectral radius less than 1 can be regarded as a working assumption, which is not required for applying ND with Equation 4, at least for undirected graphs as pointed out in Remark 1. For example, even when any eigenvalue of 
is smaller than −1, the corresponding precision matrix 
is still positive definite and statistically meaningful. In other words, it is not really necessary to scale an undirected total-effect graph in ND (as to be confirmed in numerical studies Section 4.3). On the other hand, due to the lack of a result analogous to Proposition 1 for directed graphs, it remains open whether the assumption of the spectral radius less than 1 is necessary for ND when applied to directed graphs.
4.2. Scaled ND as a shrinkage estimator
In Section 4.1, we introduce the scaled ND as its use of 
, where 
is the scaled total-effect graph as proposed in Feizi et al. (2013). From Proposition 2, it turns out that the resulting 
gives the same conditional dependence structure as the inverse of the ridge-penalized covariance/correlation matrix 
, where 
is shrunk towards the identify matrix 
. Schäfer and Strimmer (2005) have derived an analytical form of 
(see Equation 10 therein) such that the mean squared error of 
is minimized. On the other hand, under the Gaussian assumption, Warton (2008) has shown that this ridge estimator of the correlation matrix is the maximum penalized likelihood estimator with a penalty proportional to 
(see Theorem 1 therein). This is equivalent to penalizing the sum of eigenvalues of 
- when the sample size is small, the sample estimate 
may be ill-conditioned, and the eigenvalues of 
may approach 
.
Motivated by the aforementioned connection between the scaled ND (applied to the correlation matrix) and the regularized correlation matrix, we provide a heuristic interpretation of scaling in ND in general. First, we can write Equation 1 as 
. Then given an estimate of the total graph 
, the ND estimator of direct graph Equation 4 can be obtained by minimizing the loss function 
, assuming 
is invertible. Furthermore, the scaled ND solution 
can be obtained by minimizing the following penalized loss function:
 |
(10) |
where 
. The first penalty term in Eqation 10 
shrinks the Frobienius norm of 
, which is an upper bound of the spectral radius of 
, that is 
. Note that 
is the Frobenius inner product between two matrices, which measures their similarity. Thus the second penalty term in Equation 10 can be viewed as shrinking the dissimilarity between 
and 
. In practice, we can use cross-validation to estimate the tuning parameter 
such that it minimizes cross-validated risk.
 |
where 
is the estimated total graph based on the kth fold data, 
is the estimated direct graph from minimizing Equation 10 using all other data but the kth fold, and 
is the corresponding predicted total graph. Estimation of 
can be implemented using numerical optimization.
4.3. Numerical results
In this section, we conduct simulation studies to compare the performance of the variations of ND and GLASSO in estimating conditional dependency. First, we used the R package BDgraph (Mohammadi and Wit, 2019) to generate Gaussian data from the following network structures:
A random network: the edge sets were randomly generated from a Bernoulli distribution with different sparsity levels (0.05, 0.2 ,0.4), and the precision matrices were drawn from a G-Wishart distribution with default parameters in BDgraph;
A circular network: the (unscaled) precision matrix is tridiagonal with
and 
.
The number of nodes/variables p was set to be 50, and the sample sizes N varied from 
. The following methods were compared:
ND: ND without scaling, that is,
;ND-FK: ND with scaling where the scaling factor is suggested by Equation 12 in the supplementary note of Feizi et al. (2013) to ensure the spectral radius of
smaller than 1;ND-CV: ND with scaling where the scaling factor is estimated by 2-fold cross validation;
ND-SS: ND with scaling where the scaling factor is the analytical solution in Schäfer and Strimmer (2005);
GLASSO: GLASSO with the tuning parameter selected by Extended Bayesian Information Criteria (EBIC) (Foygel and Drton, 2010). This is implemented in qgraph::EBICglasso in R with default parameters.
Unlike GLASSO, ND and scaled-ND do not perform variable selection automatically. Here, we inferred the underlying direct-effect graph using false discovery rate multiple testing procedure. Specifically, we performed a nonparametric bootstrap to compute the P-value for each element in the estimated direct-effect graph, and corrected for multiple testing using q-value implemented in the R package qvalue. The FDR cutoff was set to be 0.1. Accuracy of edge identification was assessed by the average Matthews correlation coefficient (MCC) over 100 replications.
As shown in Table 1, when 
, ND failed to run as the sample correlation matrix was not positive definite. In the random network structure, ND-CV, ND-SS, and GLASSO performed similarly, and had higher MCC than non-scaled ND when the sample size was medium 
. In the circular network structure, GLASSO had higher MCC than ND methods when sample size 
; however, when sample size increased, ND, ND-CV, and ND-SS had higher MCC than GLASSO. We want to highlight one observation that ND-FK did not perform well with the lowest MCC. In both graph structures, we found that the spectral radius of 
was oftentimes greater than 1. ND-FK used the scaling factor proposed in Feizi et al. (2013), which was approximately 0.5 on average across 100 replicates in all scenarios, to guarantee that the spectral radius of 
must be smaller than 1, which was not necessary for the analysis of GGM as suggested by Proposition 1. Adding too much unnecessary noise will distort the conditional dependency, resulting in the poor performance of ND-FK, especially when the sample size was large (
). On the other hand, the scaling factors used in ND-CV increased as the sample size increased, and approached to 1 when 
. This is expected because, with a small sample size, there was large uncertainty in estimating the total-effect network, then a small 
should be chosen (ie, larger penalty); conversely, with a large sample size, less penalty was required (ie, 
closer to 1). This observation was also consistent with that in Dr. Lion Pachter’s blog (Pachter, 2014), where they also conducted a GGM simulation study and found that ND-FK did not perform well and even worse than the non-scaled ND. Lastly, in terms of computational time, ND took approximately 3 seconds on average to run with 1000 bootstrap samples, while GLASSO took approximately 1 second on average to run with a search over 100 lambdas value (ie, the tuning parameter).
Next, we investigated the performance of ND on non-Gaussian data. We first used BDgraph to generate Gaussian data from a random graph or a circular graph as above, then used the exponential of the generated data as our observed data. We considered 
variables and sample size N from 
. The sparsity level of the random graph was varied from 
. In addition to the previous 5 methods with Pearson correlation as input, we further applied ND with Kendall’s tau correlation, and GLASSO for non-paranormal data implemented in the huge R package (Zhao et al., 2012).
As shown in Table 2, in general, ND using Kendall’s tau had higher MCC than their Pearson correlation counterparts, with ND-K and ND-CV-K performing best overall. Moreover, ND methods with Kendall’s tau correlation (except for ND-FK-K in the circular graph structure) had higher MCC than GLASSO-npn.
TABLE 2.
MCC results for different graph structures and sample sizes with non-Gaussian data. Suffix “-P” indicates Pearson correlation, and suffix “-K” indicates Kendall’s tau correlation.
| Structure | N | ND-P | ND-FK-P | ND-CV-P | ND-SS-P | GLASSO-P | ND-K | ND-FK-K | ND-CV-K | ND-SS-K | GLASSO-npn |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Random (0.05) | 50 | 0.04 | 0.45 | 0.44 | 0.46 | 0.21 | 0.48 | 0.49 | 0.51 | 0.53 | 0.03 |
| Random (0.05) | 200 | 0.63 | 0.58 | 0.56 | 0.57 | 0.44 | 0.73 | 0.68 | 0.68 | 0.67 | 0.60 |
| Random (0.05) | 500 | 0.70 | 0.64 | 0.64 | 0.65 | 0.59 | 0.76 | 0.73 | 0.73 | 0.72 | 0.59 |
| Random (0.1) | 50 | 0.01 | 0.43 | 0.42 | 0.43 | 0.22 | 0.51 | 0.51 | 0.52 | 0.52 | 0.02 |
| Random (0.1) | 200 | 0.55 | 0.52 | 0.54 | 0.55 | 0.40 | 0.69 | 0.59 | 0.64 | 0.61 | 0.51 |
| Random (0.1) | 500 | 0.64 | 0.55 | 0.61 | 0.62 | 0.50 | 0.69 | 0.59 | 0.65 | 0.62 | 0.50 |
| Random (0.2) | 50 | 0.01 | 0.34 | 0.35 | 0.35 | 0.19 | 0.41 | 0.41 | 0.46 | 0.44 | 0.03 |
| Random (0.2) | 200 | 0.46 | 0.40 | 0.48 | 0.48 | 0.30 | 0.61 | 0.42 | 0.57 | 0.51 | 0.41 |
| Random (0.2) | 500 | 0.55 | 0.38 | 0.53 | 0.53 | 0.37 | 0.58 | 0.39 | 0.55 | 0.50 | 0.44 |
| Circular | 50 | 0.00 | 0.38 | 0.08 | 0.33 | −0.10 | 0.92 | 0.39 | 0.90 | 0.70 | 0.39 |
| Circular | 200 | 0.14 | 0.31 | 0.31 | 0.43 | −0.12 | 0.71 | 0.16 | 0.67 | 0.52 | 0.40 |
| Circular | 500 | 0.37 | 0.25 | 0.43 | 0.47 | −0.09 | 0.55 | 0.05 | 0.54 | 0.46 | 0.41 |
5. DISCUSSION
ND is a simple and intuitive algorithm to disentangle direct and indirect effects/associations that has been more widely applied in bioinformatics, though its connection to and comparison with other graphical models remains largely open. On the other hand, although ND was motivated and justified by its application to directed graphs, a justification for its use on undirected graphs is not well known. Here we have established an equivalence of the end product of ND to the partial correlation/precision matrix when ND is applied to a marginal correlation matrix, thus offering a strong support for the use of ND on undirected graphs. As a motivating example, we have applied ND on an undirected genetic correlation graph to infer a partial genetic correlation graph, which was recently advocated as a useful tool in genetics but estimated via a precision matrix by others (Werme et al., 2022; Yang et al., 2024). We have also clarified the implicit assumption of ND on linear models (with a linear effect of each node on the other), as implied by marginal and partial Pearson’s correlations. Note that the above equivalence result does not depend on the normality assumption; if indeed the normality assumption holds for each node in the undirected graph, the absence of a total (or direct) (linear) effect/association between any two nodes can be interpreted as marginal (or conditional) independence as in a GGM. We have further established a formal result to show the close connection between scaling a total-effect graph (as a marginal correlation matrix) before applying ND as recommended in the original ND paper and a shrinkage covariance estimator, which, to our best knowledge, is new. Furthermore, we have proposed an estimator of the scaling factor (as a tuning parameter) and demonstrated its good performance when applied to high-dimensional data.
There are some limitations of the results presented here. It is unclear what ND yields if the true model is non-linear, or if other metrics for marginal association such as mutual information are used. We have restricted our discussion of undirected (total-effect) graphs to that with the use of Pearson’s correlation to measure any marginal association. However, due to its connection with other nonparametric rank-based methods as discussed in Remark 3 and the good performance of applying ND (with or without scaling) with Kendall’s tau on non-Gaussian data in our simulations, we expect ND to be a promising approach. Future investigation is warranted. Finally, we have mainly focused on the application of ND on an undirected correlation graph, and its application on directed graphs needs further investigation.
We note that, in the final stage of the preparation of this manuscript, we became aware of an unpublished Master’s thesis at Radboud University Nijmegen (Gerritse, 2018), which studied the same problem with some overlapping contents. To our best knowledge, both Gerritse (2018) and Alipanahi and Frey (2013) are not widely known (and certainly were unknown to us until a few months ago before writing this paper); the current paper reinforces some of their main points with its own new contributions.
Supplementary Material
Web Appendices, Tables referenced in Section 3.2, and R code are available with this paper at the Biometrics website on Oxford Academic. Code for the real data application can also be found on https://github.com/ZhaotongL/ND_paper.
Acknowledgement
We thank the reviewers and editors for many helpful and constructive comments.
Contributor Information
Zhaotong Lin, Division of Biostatistics and Health Data Science, University of Minnesota, Minneapolis, MN 55455, United States; Department of Statistics, Florida State University, Tallahassee, FL 32306, United States.
Isaac Pan, Department of Mathematics and Statistics, Pomona College, Claremont, CA 91711, United States.
Wei Pan, Division of Biostatistics and Health Data Science, University of Minnesota, Minneapolis, MN 55455, United States.
FUNDING
This work was supported by NIH grants R01 AG065636, R01 AG069895, RF1 AG067924, U01 AG073079, and R01 AG074858, and by the Minnesota Supercomputing Institute at the University of Minnesota.
CONFLICT OF INTEREST
None declared.
DATA AVAILABILITY
The genome-wide association study summary statistics for height, coronary artery disease, low-density lipoprotein cholesterol and pulse pressure that support the findings in this paper were downloaded from https://gwas.mrcieu.ac.uk/ with ID ieu-a-89, ebi-a-GCST005195, ebi-a-GCST002222, and ebi-a-GCST90018970 respectively. The processed data are also available at https://doi.org/10.6084/m9.figshare.26410951.v1.
References
- Alipanahi B., Frey B. J. (2013). Network cleanup. Nature Biotechnology, 31, 714–715. [DOI] [PubMed] [Google Scholar]
- Becker M., Nassar H., Espinosa C., Stelzer I. A., Feyaerts D., Berson E. et al. (2023). Large-scale correlation network construction for unraveling the coordination of complex biological systems. Nature Computational Science, 3, 346–359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bollen K. A. (1989). Structural Equation Models with Observed Variables, New York: John Wiley & Sons, Ltd. [Google Scholar]
- Bourgeois B., Watts K., Thomas D. M., Carmichael O., Hu F. B., Heo M. et al. (2017). Associations between height and blood pressure in the United States population. Medicine, 96, e9233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bulik-Sullivan B., Finucane H. K., Anttila V., Gusev A., Day F. R., Loh P.-R. et al. (2015). An atlas of genetic correlations across human diseases and traits. Nature Genetics, 47, 1236–1241. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burgess S., Smith G. D., Davies N. M., Dudbridge F., Gill D., Glymour M. M. et al. (2019). Guidelines for performing Mendelian randomization investigations: update for summer 2023. Wellcome open research, 4, 186. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Collaboration E. R. F. (2012). Adult height and the risk of cause-specific death and vascular morbidity in 1 million people: individual participant meta-analysis. International Journal of Epidemiology, 41, 1419–1433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Drton M., Foygel R., Sullivant S. (2011). Global identifiability of linear structural equation models. The Annals of Statistics, 39, 865–886. [Google Scholar]
- Drton M., Weihs L. (2016). Generic identifiability of linear structural equation models by ancestor decomposition. Scandinavian Journal of Statistics, 43, 1035–1045. [Google Scholar]
- Feizi S., Marbach D., Médard M., Kellis M. (2013). Network deconvolution as a general method to distinguish direct dependencies in networks. Nature Biotechnology, 31, 726–733. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fieller N. (2018). Basics of Matrix Algebra for Statistics with R, Boca Raton: Chapman and Hall/CRC. [Google Scholar]
- Foygel R., Draisma J., Drton M. (2012). Half-trek criterion for generic identifiability of linear structural equation models. The Annals of Statistics, 40, 1682–1713. [Google Scholar]
- Foygel R., Drton M. (2010). Extended Bayesian information criteria for Gaussian graphical models. Advances in Neural Information Processing Systems, 23, 604–612. [Google Scholar]
- Franklin S. S., Khan S. A., Wong N. D., Larson M. G., Levy D. (1999). Is pulse pressure useful in predicting risk for coronary heart disease? The Framingham heart study. Circulation, 100, 354–360. [DOI] [PubMed] [Google Scholar]
- Friedman J., Hastie T., Tibshirani R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9, 432–441. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fuller W. A. (2009). Measurement Error Models, New York: John Wiley & Sons. [Google Scholar]
- Gerritse E. (2018). Network deconvolution: an analysis. Master’s thesis (Supervisor: Johannes Textor; Second Reader: Tom Claassen) at Radboud University Nijmegen, [Accessed 29 November 2023]. [Google Scholar]
- Kang T., Moore R., Li Y., Sontag E., Bleris L. (2015). Discriminating direct and indirect connectivities in biological networks. Proceedings of the National Academy of Sciences, 112, 12893–12898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Karalis D. G. (2009). Intensive lowering of low-density lipoprotein cholesterol levels for primary prevention of coronary artery disease. Mayo Clinic Proceedings, 84, 345–352. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kendall M. G. (1948). Rank Correlation Methods, London: Charles Griffin and Co. Ltd [Google Scholar]
- Kruskal W. H. (1958). Ordinal measures of association. Journal of the American Statistical Association, 53, 814–861. [Google Scholar]
- Langenberg C., Hardy R., Kuh D., Wadsworth M. E. (2003). Influence of height, leg and trunk length on pulse pressure, systolic and diastolic blood pressure. Journal of Hypertension, 21, 537–543. [DOI] [PubMed] [Google Scholar]
- Lauritzen S. L. (1996). Graphical Models, vol. 17, Oxford: Clarendon Press. [Google Scholar]
- Li C., Shen X., Pan W. (2023). Inference for a large directed acyclic graph with unspecified interventions. Journal of Machine Learning Research, 24, 1–48. [PMC free article] [PubMed] [Google Scholar]
- Lin Z., Xue H., Pan W. (2023). Combining mendelian randomization and network deconvolution for inference of causal networks with gwas summary data. PLoS Genetics, 19, e1010762. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu H., Han F., Yuan M., Lafferty J., Wasserman L. (2012). High-dimensional semiparametric Gaussian copula graphical models. The Annals of Statistics, 40, 2293–2326. [Google Scholar]
- Mohammadi R., Wit E. C. (2019). BDgraph: an R package for Bayesian structure learning in graphical models. Journal of Statistical Software, 89, 1–30. [Google Scholar]
- Nelson C. P., Hamby S. E., Saleheen D., Hopewell J. C., Zeng L., Assimes T. L. et al. (2015). Genetically determined height and coronary artery disease. New England Journal of Medicine, 372, 1608–1618. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Oh N.-K., Song Y.-M., Kim S.-H., Park M. J. (2019). Short stature is associated with increased risk of dyslipidemia in korean adolescents and adults. Scientific Reports, 9, 14090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O’Rourke M., Nichols W. W. (1998). Wave reflections, London: Arnold/Oxford University Press. [Google Scholar]
- Pachter L. (2014). The network nonsense of Manolis Kellis. [Accessed 05 July 2023].
- Ryan S. M., Waack B. J., Weno B. L., Heistad D. D. (1995). Increases in pulse pressure impair acetylcholine-induced vascular relaxation. American Journal of Physiology-Heart and Circulatory Physiology, 268, H359–H363. [DOI] [PubMed] [Google Scholar]
- Schäfer J., Strimmer K. (2005). A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical Applications in Genetics and Molecular Biology, 4,Article32. [DOI] [PubMed] [Google Scholar]
- Strauss D. (1981). Testing partial correlations when the third variable is measured with error. Educational and Psychological Measurement, 41, 349–358. [Google Scholar]
- Van Rheenen W., Peyrot W. J., Schork A. J., Lee S. H., Wray N. R. (2019). Genetic correlations of polygenic disease traits: from theory to practice. Nature Reviews Genetics, 20, 567–581. [DOI] [PubMed] [Google Scholar]
- Warton D. I. (2008). Penalized normal likelihood and ridge regularization of correlation and covariance matrices. Journal of the American Statistical Association, 103, 340–349. [Google Scholar]
- Werme J., van der Sluis S., Posthuma D., de Leeuw C. A. (2022). An integrated framework for local genetic correlation analysis. Nature Genetics, 54, 274–282. [DOI] [PubMed] [Google Scholar]
- Xue L., Zou H. (2012). Regularized rank-based estimation of high-dimensional nonparanormal graphical models. The Annals of Statistics, 40, 2541–2571. [Google Scholar]
- Yang Y., Lorincz-Comi N., Zhu X. (2024). Estimation of the genetic gaussian network using GWAS summary data. arXiv:2401.07767, preprint: not peer reviewed. [DOI] [PMC free article] [PubMed]
- Young N. (1981). The rate of convergence of a matrix power series. Linear Algebra and its Applications, 35, 261–278. [Google Scholar]
- Zhao T., Liu H., Roeder K., Lafferty J., Wasserman L. (2012). The huge package for high-dimensional undirected graph estimation in R. The Journal of Machine Learning Research, 13, 1059–1062. [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Web Appendices, Tables referenced in Section 3.2, and R code are available with this paper at the Biometrics website on Oxford Academic. Code for the real data application can also be found on https://github.com/ZhaotongL/ND_paper.
Data Availability Statement
The genome-wide association study summary statistics for height, coronary artery disease, low-density lipoprotein cholesterol and pulse pressure that support the findings in this paper were downloaded from https://gwas.mrcieu.ac.uk/ with ID ieu-a-89, ebi-a-GCST005195, ebi-a-GCST002222, and ebi-a-GCST90018970 respectively. The processed data are also available at https://doi.org/10.6084/m9.figshare.26410951.v1.