Skip to main content
Journal of Applied Statistics logoLink to Journal of Applied Statistics
. 2021 Feb 8;50(3):495–511. doi: 10.1080/02664763.2021.1884847

A practical two-sample test for weighted random graphs

Mingao Yuan 1,CONTACT, Qian Wen 1
PMCID: PMC9930797  PMID: 36819081

Abstract

Network (graph) data analysis is a popular research topic in statistics and machine learning. In application, one is frequently confronted with graph two-sample hypothesis testing where the goal is to test the difference between two graph populations. Several statistical tests have been devised for this purpose in the context of binary graphs. However, many of the practical networks are weighted and existing procedures cannot be directly applied to weighted graphs. In this paper, we study the weighted graph two-sample hypothesis testing problem and propose a practical test statistic. We prove that the proposed test statistic converges in distribution to the standard normal distribution under the null hypothesis and analyze its power theoretically. The simulation study shows that the proposed test has satisfactory performance and it substantially outperforms the existing counterpart in the binary graph case. A real data application is provided to illustrate the method.

Keywords: Two-sample hypothesis test, random graph, weighted graph

1. Introduction

A graph or network G=(V,E) is a mathematical model that consists of a set V of nodes (vertices) and a set E of edges. In the past decades, it has been widely used to represent a variety of systems in various regimes [7,8,10,19,20]. For instance, in social networks, a node denotes an individual and an edge represents the interaction between two individuals [10]; in brain graphs, a node may be a neural unit and the functional link between two units forms an edge [15]; in co-authorship networks, the authors of a collection of articles are the nodes and an edge is defined to be the co-authorship of two authors [20]. Due to the widespread applications, network data analysis has drawn a lot of attentions in both statistical and machine learning communities [1,4–6,11,18,24]. Most of the existing literature focus on mining a single network, such as community detection [1,4–6], global testing of the community structures [6,11,18,24] and so on. In practice, a number of graphs from multiple populations may be available. For example, in the 1000 Functional Connectomes Project, 1093 fMRI (weighted) networks were collected from subjects located in 24 communities [12]; to study the relation between Alzheimer's disease and a functional disconnection of distant brain areas, dozens of functional connectivity (weighted) networks from patients and control subjects were constructed [21]. In this case, a natural and fundamental question is to test the difference between two graph populations, known as graph two-sample hypothesis testing.

There are a few literature dealing with the graph two-sample hypothesis testing problem [12–14,22]. Specifically, [12] firstly investigated this problem and proposed a χ2-type test. In [22], the authors developed a kernel-based test statistic for random dot product graph models. Under a more general setting, [13] studied the graph two-sample test from a minimax testing perspective and proposed testing procedures based on graph distance such as Frobenius norm or operator norm. The threshold of the test statistics in [13] could be calculated by concentration inequalities, which usually makes the test very conservative [14]. To overcome this issue, [14] derived the asymptotic distribution of the test statistic and proposed practical test methods that outperform existing methods.

In practice, most of the graphs are weighted [2,3,12,21,23]. The testing procedures in [9,13,14,22] are designed under the context of binary (unweighted) graphs and the tests cannot be directly applied to weighted graphs (see Section 3 for an example). Consequently, before using these tests, one has to artificially convert weighted graphs into binary graphs, which can result in a loss of information [2,3,23]. Motivated by the Tfro test in [14], we propose a powerful test statistic for weighted graph two-sample hypothesis testing. Under the null hypothesis, the proposed test statistic converges in distribution to the standard normal distribution and the power of the test is theoretically characterized. Simulation study shows that the test can achieve high power and it substantially outperforms its counterpart in the binary graph case. Besides, we apply the proposed test to a real data.

The rest of the paper is organized as follows. In Section 2, we formally state the weighted graph two-sample hypothesis testing problem and present the theoretical results. In Section 3, we present the simulation study results and real data application. The proof of main result is deferred to Section 4.

2. Weighted graph two-sample hypothesis test

For convenience, let XF represent random variable X follows distribution F and let Bern(r) denote the Bernoulli distribution with success probability r.

Let V={1,2,,n} be a vertex (node) set and G=(V,E) denote an undirected graph on V with edge set E. The adjacency matrix of graph G is a symmetric matrix A{0,1}n×n such that Aij=1 if (i,j)E and 0 otherwise. The graph G is binary or unweighted, since Aij only records the existence of an edge. If AijBern(pij),0pij1, then the graph G is called an inhomogeneous random graph(inhomogeneous Erdös-Rényi graph). Let μ=(μij)1i<jn be a sequence of real numbers and Q=(Qij)1i<jn be a sequence of distributions defined on a bounded interval, where each Qij is uniquely parametrized by its mean value μij. A weighted random graph G=(V,Q,μ) is defined as follows. For nodes i, j,

Aij=Aji,AijQij(μij),1i<jn,

Aii=0 (i=1,2,,n) and Aij is independent of Akl if {i,j}{k,l}. If Qij(μij)=Bern(μij), then G=(V,Q,μ) is just the inhomogeneous random graph [13,14].

Given i.i.d. graph sample G1,,GmG1=(V,Q,μ1) and i.i.d. graph sample H1,,HmG2=(V,Q,μ2), we are interested in the weighted graph two-sample hypothesis testing problem

H0:G1=G2,H1:G1G2. (1)

Let AGk and AHk be the adjacency matrix of graph Gk and Hk respectively. Then AGk,ijQij(μ1,ij) and AHk,ijQij(μ2,ij) for 1i<jn. Consequently, (1) is equivalent to the following hypothesis test

H0:μ1=μ2,H1:μ1μ2.

In the binary graph case ( Qij(μij)=Bern(μij),1i<jn), several testing procedures for (1) are available in the literature. For m and small n, a χ2-type test was proposed in [12]. For m = 1 and n, under the random dot product model, a nonparametric test statistic was developed in [22], and a test based on eigenvalues of adjacency matrix under the inhomogeneous random graph could be found in [14]. A more practical case is small m (m2) and n. In this case, a test called Tfro was proposed in [13] and its asymptotic behavior was studied in [14]. Recently, [9] proposed a test statistic based on the largest eigenvalue of a Wigner matrix.

In this work, we study (1) for a broad class of distributions Q and focus on the regime m2 and n. The sample size m could be either fixed or tend to infinity along with n.

To define the test statistic, the two samples Gk,Hk,(1km) are randomly partitioned into two parts, denoted as Gk,Hk,(1km/2) and Gk,Hk,(m/2<km) with a little notation abuse. Let sn2=1i<jnTij2, where

Tij=km2(AGk,ijAHk,ij)k>m2(AGk,ijAHk,ij).

We propose the following test statistic for (1):

Tn=1i<jnTijsn. (2)

Let σij2=Var(AGk,ij) and ηij=E(AGk,ijAHk,ij)4 under H0. The asymptotic distribution of Tn is given in the following theorem.

Theorem 2.1

Suppose

n=o(1i<jnσij4),1i<jnσij8(1i<jnσij4)2=o(1),1i<jnσij4ηijm(1i<jnσij4)2=o(1),1i<jnηij2m2(1i<jnσij4)2=o(1). (3)

Then under H0, Tn converges in distribution to N(0,1), the standard normal distribution, as n.

Theorem 2.1 states that the limiting distribution of Tn under the null hypothesis is N(0,1). Given type one error α, reject H0 if |Tn|>Z(1α2) where Z(1α2) is the 100(1α2)% quantile of the standard normal distribution.

Condition (3) could be simplified in the binary case. Suppose Qij(μij)=Bern(μij) and μ1,ij=μ2,ij=μij1δ for some δ(0,1) under H0. Then ηij2σij22μij. In this case, condition (3) reduces to n=o(μF2). Here μF denotes the Frobenius norm of matrix μ. To see this, let C be a generic constant, then

0.5δ2μF2=δ21i<jnμij21i<jnσij41i<jnμij2=0.5μF2, (4)
1i<jnσij8(1i<jnσij4)2C1i<jnμij4(1i<jnμij2)2C1i<jnμij2(1i<jnμij2)2=C1μF20, (5)
1i<jnσij4ηijm(1i<jnσij4)2C1i<jnμij2m(1i<jnμij2)2=C1mμF20, (6)
1i<jnηij2m2(1i<jnσij4)2C1i<jnμij2m2(1i<jnμij2)2=C1m2μF20. (7)

If n=o(μF2), then (3) holds by (4),(5)–(7).

In the following, we analyze the power of the proposed test statistic. Let σ1,ij2=Var(AGk,ij), σ2,ij2=Var(AHk,ij) under H1 and

Vij=σ1,ij2+σ2,ij2+(μ1,ijμ2,ij)2,λn=m1i<jn(μ1,ijμ2,ij)221i<jnVij2.

Theorem 2.2

Suppose n=o(m1i<jnVij2). Then under H1, Tn=λn+OP(1).

According to Theorem 2.2, the power of the test goes to one if λn, as n. The expression of λn explicitly characterizes the effect of sample size m and the mean and variance of edge weight on the power of the test statistic.

In the following, let us restrict Theorem 2.2 to binary graphs to see when the test could achieve high power. Suppose Qij(μt,ij)=Bern(μt,ij), μt,ij0, and μ1,ij/μ2,ijτ( τ>0) for t = 1, 2. Then

Vij=μ1,ij(1μ1,ij)+μ2,ij(1μ2,ij)+(μ1,ijμ2,ij)2=(μ1,ij+μ2,ij)(1+o(1)).

In this case, n=o(m1i<jnVij2) requires n=o(mμ1+μ2F2). Besides,

λn=m1i<jn(μ1,ijμ2,ij)221i<jn(μ1,ij+μ2,ij)2(1+o(1))=(1+o(1))mμ1μ2F222μ1+μ2F. (8)

For fixed μ1 and μ2, as the sample size m increases, the power increases. As μ1μ2F2 gets larger, the power gets higher when μ1+μ2F2 and m are held constant.

Remark 1

The quantity λn in Theorem 2.2 completely characterizes the power of our test. For binary graphs, the sparsity may increase or decrease the power, dependent on the model settup. To see this, we consider two scenarios below.

  1. Suppose μ1,ij=τan for a constant τ>0 and μ2,ij=an with an=o(1), 1i<jn. By (8), it follows that
    λn=mnan(τ1)24(τ+1)[1+o(1)].
    For fixed sample size m and the number of nodes n, the power of our test statistic declines as the networks get sparser (smaller an).
  2. Suppose μ1,ij=an+bn and μ2,ij=anbn with an=o(1) and bn=o(1), 1i<jn. Then by equation (8), one has
    λn=mn2bn2an[1+o(1)].
    The ratio bn2an controls the power, if the sample size m and the number of nodes n are held constant. Model 1: an=n0.6n, bn=an, then bn2an=1 and λn=mn2[1+o(1)]. Model 2: an=n0.7n, bn=anlogn, then bn2an=1logn and λn=mn2logn[1+o(1)]. Clearly, Model 1 is sparser than Model 2 but our test achieves higher power under Model 1 than Model 2 based on Theorem 2.2.

Remark 2

Recall that the Tfro test in [14] is defined as

Tfro=1i<jnTijtn,

where

tn2=1i<jnkm2(AGk,ij+AHk,ij)k>m2(AGk,ij+AHk,ij).

The difference between Tn and Tfro lies in the difference between sn and tn. Note that sn2 in Tn is proved to be a consistent estimator of the variance of 1i<jnTij under H0 for a broad class of distributions Q, while tn2 may not be a consistent estimator of the variance. To see this,

τn2=E[tn2]=1i<jnm2μij2.

By the proof of Theorem 2.1, we have sn2=(1+op(1))σn2 and

σn2=E[sn2]=1i<jnm2σij4.

Here σn2 is the variance of 1i<jnTij. For any distribution Q with τn2(1+o(1))σn2, the test statistic Tfro will fail, since tn2 is not a consistent estimator of σn2 in this case. For example, let Qij(μij) be the beta distribution Beta(α,β). Then for 1i<jn,

μij=αα+β,σij2=αβ(α+β)2(α+β+1),σij2μij=β(α+β)(α+β+1).

When α and β are fixed constants, τn2(1+o(1))σn2. In this case, Tfro doesn't work.

For Qij(μij)=Bern(μij), σij2=μij(1μij). If μij1ϵ with ϵ(0,1), then τn2(1+o(1))σn2. In this case, Tfro will fail. On the contrary, when μij=o(1), σij2=(1+o(1))μij and the test Tfro may be valid. The simulation results in Section 3 are consistent with the above findings.

3. Simulation and real data

In this section, we evaluate the finite sample performance of the proposed test Tn and compare it with the test Tfro in [14] by simulation. Besides, we apply our test method to a real data.

3.1. Simulation

Throughout this simulation, we set the nominal type one error α to be 0.05. The empirical size and power are obtained by repeating the experiment 1000 times. We take n = 10, 30, 50, 100, 200, 300 and m = 2, 4, 14.

In the first simulation, we generate weights from beta distribution. Specifically, we generate G1,,GmG1=(V,Q,μ1) with Qij(μ1,ij)=Beta(a,b) for 1i<jn/2 or n/2<i<jn and Qij(μ1,ij)=Beta(c,d) for 1in/2<jn. Denote the graph model as G1(Beta(a,b),Beta(c,d)). For a fixed constant ϵ ( ϵ0), we generate the second sample H1,,HmG2=(V,Q,μ2), with Qij(μ2,ij)=Beta(a+ϵ,b+ϵ) for 1i<jn/2 or n/2<i<jn and Qij(μ2,ij)=Beta(c+ϵ,d+ϵ) for 1in/2<jn. Denote the graph model as G2(Beta(a+ϵ,b+ϵ),Beta(c+ϵ,d+ϵ)).

Note that the constant ϵ ( ϵ0) characterizes the difference between μ2,ij and μ1,ij with fixed a, b, c, d, since for Beta(a+ϵ,b+ϵ), the mean is equal to

μ(ϵ)=a+ϵ1a+b+2ϵ2.

Clearly μ(ϵ) is an increasing function of ϵ (ϵ0) and larger ϵ implies larger difference in the means and consequently the power of the test Tn is supposed to increase.

We take a = 2, b = 3, c = 1, d = 3 and a = 9, b = 3, c = 3, d = 2 to yield right-skewed and left-skewed beta distributions respectively. The simulation results are summarized in Tables 1 and 2, where the sizes (powers) are reported in column(s) with ϵ=0 ( ϵ>0). The sizes and powers of Tfro are all zeros, which indicates this test (designed for binary graphs) does not apply to weighted graph (see Remark 2 for explanation). On the contrary, all the sizes of the proposed test Tn are close to 0.05, which implies the null distribution is valid even for small networks (small n) and small sample sizes (small m). Besides, the power can approach one, this shows the consistency of the proposed test Tn. The parameter ϵ, n, m have significant influence on the powers. As any one of them increases with the rest held constant, the power of Tn gets higher.

Table 1.

Simulated size and power with graphs generated from G1(Beta(2,3),Beta(1,3)) and G2(Beta(2+ϵ,3+ϵ),Beta(1+ϵ,3+ϵ)).

n(m=2) Method ϵ=0 (size) ϵ=0.3 (power) ϵ=0.5 (power) ϵ=0.7 (power)
10   0.000 0.000 0.000 0.000
30   0.000 0.000 0.000 0.000
50   0.000 0.000 0.000 0.000
100 Tfro 0.000 0.000 0.000 0.000
200   0.000 0.000 0.000 0.000
300   0.000 0.000 0.000 0.000
10   0.046 0.052 0.060 0.069
30   0.043 0.065 0.085 0.123
50   0.049 0.079 0.093 0.203
100   0.052 0.089 0.248 0.604
200 Tn 0.045 0.199 0.754 0.995
300   0.055 0.383 0.968 1.000
n(m=4) Method ϵ=0 (size) ϵ=0.3 (power) ϵ=0.5 (power) ϵ=0.7 (power)
10   0.000 0.000 0.000 0.000
30   0.000 0.000 0.000 0.000
50   0.000 0.000 0.000 0.000
100 Tfro 0.000 0.000 0.000 0.000
200   0.000 0.000 0.000 0.000
300   0.000 0.000 0.000 0.000
10   0.049 0.058 0.065 0.069
30   0.048 0.067 0.134 0.237
50   0.048 0.088 0.251 0.576
100 Tn 0.056 0.209 0.757 0.992
200   0.047 0.594 0.999 1.000
300   0.058 0.907 1.000 1.000
n(m=14) Method ϵ=0 (size) ϵ=0.3 (power) ϵ=0.5 (power) ϵ=0.7 (power)
10   0.000 0.000 0.000 0.000
30   0.000 0.000 0.000 0.000
50   0.000 0.000 0.000 0.000
100 Tfro 0.000 0.000 0.000 0.006
200   0.000 0.000 0.538 1.000
300   0.000 0.000 1.000 1.000
10   0.044 0.056 0.115 0.244
30   0.048 0.193 0.707 0.980
50   0.047 0.460 0.989 1.000
100 Tn 0.044 0.960 1.000 1.000
200   0.041 1.000 1.000 1.000
300   0.044 1.000 1.000 1.000

Table 2.

Simulated size and power with graphs generated from G1(Beta(9,3),Beta(3,2)), G2(Beta(9+ϵ,3+ϵ),Beta(3+ϵ,2+ϵ)).

n(m=2) Method ϵ=0 (size) ϵ=0.5 (power) ϵ=0.7 (power) ϵ=0.9 (power)
10   0.000 0.000 0.000 0.000
30   0.000 0.000 0.000 0.000
50   0.000 0.000 0.000 0.000
100 Tfro 0.000 0.000 0.000 0.000
200   0.000 0.000 0.000 0.000
300   0.000 0.000 0.000 0.000
10   0.043 0.051 0.055 0.057
30   0.050 0.056 0.061 0.065
50   0.047 0.062 0.076 0.077
100 Tn 0.050 0.058 0.093 0.205
200   0.049 0.136 0.304 0.611
300   0.057 0.231 0.591 0.926
n(m=4) Method ϵ=0 (size) ϵ=0.5 (power) ϵ=0.7 (power) ϵ=0.9 (power)
10   0.000 0.000 0.000 0.000
30   0.000 0.000 0.000 0.000
50   0.000 0.000 0.000 0.000
100 Tfro 0.000 0.000 0.000 0.000
200   0.000 0.000 0.000 0.000
300   0.000 0.000 0.000 0.000
10   0.047 0.053 0.056 0.057
30   0.045 0.059 0.065 0.075
50   0.055 0.063 0.110 0.181
100 Tn 0.053 0.113 0.294 0.602
200   0.041 0.357 0.834 0.989
300   0.046 0.674 0.988 1.000
n(m=14) Method ϵ=0 (size) ϵ=0.3 (power) ϵ=0.5 (power) ϵ=0.7 (power)
10   0.000 0.000 0.000 0.000
30   0.000 0.000 0.000 0.000
50   0.000 0.000 0.000 0.000
100 Tfro 0.000 0.000 0.000 0.000
200   0.000 0.000 0.000 0.000
300   0.000 0.000 0.000 0.000
10   0.044 0.060 0.064 0.077
30   0.048 0.070 0.136 0.290
50   0.051 0.082 0.286 0.667
100 Tn 0.049 0.187 0.784 0.998
200   0.051 0.578 1.000 1.000
300   0.045 0.902 1.000 1.000

In the second simulation, we generate binary graphs to compare the performance of Tn and Tfro. Specifically, we generate G1,,GmG1=(V,Q,μ1) with Qij(μ1,ij)=Bern(a) for 1i<jn/2 or n/2<i<jn and Qij(μ1,ij)=Bern(b) for 1in/2<jn. Denote the graph model as G1(Bern(a),Bern(b)). For a constant ϵ ( ϵ0), the second sample H1,,Hm are generated from G2=(V,Q,μ2), with Qij(μ2,ij)=Bern(a+ϵ) for 1i<jn/2 or n/2<i<jn and Qij(μ2,ij)=Bern(b+ϵ) for 1in/2<jn. Denote the graph model as G2(Bern(a+ϵ),Bern(b+ϵ)).

We take a = 0.05, b = 0.01, a = 0.1, b = 0.05 and a = 0.5, b = 0.5 to yield sparse, moderately sparse and dense networks, respectively. Tables 35 summarize the simulation results, where the sizes (powers) are reported in column(s) with ϵ=0 ( ϵ>0). When a = 0.05, b = 0.01 and a = 0.1, b = 0.05, the networks are too sparse so that the denominators of Tn and Tfro may be zeros for smaller n. Consequently, Tn and Tfro may not be available and we denote them as NA in Tables 3 and 4.

Table 3.

Simulated size and power with graphs generated from G1(Bern(0.05),Bern(0.01)) and G2(Bern(0.05+ϵ),Bern(0.01+ϵ)).

n(m=2) Method ϵ=0 (size) ϵ=0.03 (power) ϵ=0.05 (power) ϵ=0.07 (power)
10   NA NA NA NA
30   NA NA NA NA
50   NA 0.038 0.094 0.236
100 Tfro 0.043 0.088 0.295 0.754
200   0.044 0.244 0.880 1.000
300   0.031 0.503 0.997 1.000
10   NA NA NA NA
30   NA NA NA NA
50   NA 0.050 0.114 0.280
100 Tn 0.052 0.099 0.343 0.791
200   0.048 0.283 0.903 1.000
300   0.044 0.548 0.998 1.000
n(m=4) Method ϵ=0 (size) ϵ=0.03 (power) ϵ=0.05 (power) ϵ=0.07 (power)
10   NA NA NA NA
30   0.032 0.058 0.132 0.342
50   0.043 0.078 0.314 0.750
100 Tfro 0.035 0.231 0.868 1.000
200   0.049 0.740 1.000 1.000
300   0.041 0.976 1.000 1.000
10   NA NA NA NA
30   0.043 0.064 0.143 0.368
50   0.047 0.094 0.344 0.767
100 Tn 0.050 0.259 0.885 1.000
200   0.058 0.773 1.000 1.000
300   0.051 0.986 1.000 1.000
n(m=14) Method ϵ=0 (size) ϵ=0.03 (power) ϵ=0.05 (power) ϵ=0.07 (power)
10   NA 0.069 0.189 0.392
30   0.038 0.263 0.872 1.000
50   0.049 0.613 1.000 1.000
100 Tfro 0.040 0.995 1.000 1.000
200   0.048 1.000 1.000 1.000
300   0.038 1.000 1.000 1.000
10   NA 0.060 0.146 0.335
30   0.048 0.277 0.858 0.998
50   0.055 0.622 1.000 1.000
100 Tn 0.055 0.996 1.000 1.000
200   0.060 1.000 1.000 1.000
300   0.049 1.000 1.000 1.000

Table 5.

Simulated size and power with graphs generated from G1(Bern(0.5),Bern(0.4)) and G2(Bern(0.5+ϵ),Bern(0.4+ϵ)).

n(m=2) Method ϵ=0 (size) ϵ=0.07 (power) ϵ=0.10 (power) ϵ=0.12 (power)
10   0.000 0.000 0.000 0.000
30   0.000 0.000 0.000 0.002
50   0.000 0.000 0.002 0.001
100 Tfro 0.000 0.000 0.005 0.022
200   0.000 0.006 0.131 0.523
300   0.001 0.032 0.615 0.983
10   0.041 0.043 0.047 0.051
30   0.048 0.054 0.077 0.101
50   0.054 0.065 0.104 0.182
100 Tn 0.051 0.109 0.290 0.551
200   0.050 0.298 0.797 0.981
300   0.042 0.550 0.989 1.000
n(m=4) Method ϵ=0 (size) ϵ=0.07 (power) ϵ=0.10 (power) ϵ=0.12 (power)
10   0.001 0.001 0.001 0.002
30   0.001 0.001 0.002 0.002
50   0.001 0.002 0.003 0.030
100 Tfro 0.000 0.007 0.139 0.502
200   0.000 0.162 0.973 1.000
300   0.000 0.625 1.000 1.000
10   0.044 0.063 0.064 0.066
30   0.045 0.066 0.124 0.232
50   0.058 0.112 0.255 0.518
100 Tn 0.053 0.269 0.793 0.965
200   0.047 0.787 1.000 1.000
300   0.044 0.983 1.000 1.000
n(m=14) Method ϵ=0 (size) ϵ=0.07 (power) ϵ=0.10 (power) ϵ=0.12 (power)
10   0.000 0.001 0.002 0.014
30   0.000 0.013 0.174 0.559
50   0.000 0.083 0.812 0.997
100 Tfro 0.000 0.834 1.000 1.000
200   0.032 0.998 1.000 1.000
300   0.032 1.000 1.000 1.000
10   0.052 0.075 0.120 0.189
30   0.040 0.273 0.741 0.955
50   0.040 0.648 0.990 1.000
100 Tn 0.055 0.997 1.000 1.000
200   0.049 0.998 1.000 1.000
300   0.050 1.000 1.000 1.000

Table 4.

Simulated size and power with graphs generated from G1(Bern(0.1),Bern(0.05)) and G2(Bern(0.1+ϵ),Bern(0.05+ϵ)).

n(m=2) Method ϵ=0 (size) ϵ=0.03 (power) ϵ=0.05 (power) ϵ=0.07 (power)
10   NA NA NA NA
30   0.025 0.031 0.036 0.054
50   0.031 0.033 0.039 0.101
100   0.030 0.038 0.123 0.311
200 Tfro 0.034 0.085 0.401 0.891
300   0.046 0.157 0.756 0.998
10   NA NA NA NA
30   0.043 0.055 0.060 0.083
50   0.049 0.050 0.069 0.131
100   0.053 0.063 0.172 0.403
200 Tn 0.049 0.123 0.483 0.933
300   0.046 0.204 0.818 0.998
n(m=4) Method ϵ=0 (size) ϵ=0.03 (power) ϵ=0.05 (power) ϵ=0.07 (power)
10   NA NA NA NA
30   0.034 0.040 0.056 0.148
50   0.030 0.037 0.120 0.328
100   0.040 0.074 0.402 0.895
200 Tfro 0.031 0.277 0.962 1.000
300   0.036 0.531 1.000 1.000
10   NA NA NA NA
30   0.044 0.056 0.072 0.185
50   0.049 0.053 0.165 0.392
100   0.053 0.114 0.486 0.922
200 Tn 0.049 0.339 0.976 1.000
300   0.048 0.606 1.000 1.000
n(m=14) Method ϵ=0 (size) ϵ=0.03 (power) ϵ=0.05 (power) ϵ=0.07 (power)
10   0.039 0.044 0.075 0.156
30   0.031 0.091 0.432 0.886
50   0.035 0.199 0.874 1.000
100 Tfro 0.022 0.717 1.000 1.000
200   0.023 0.995 1.000 1.000
300   0.026 1.000 1.000 1.000
10   0.046 0.049 0.092 0.159
30   0.055 0.120 0.480 0.889
50   0.048 0.257 0.889 1.000
100 Tn 0.046 0.765 1.000 1.000
200   0.040 0.997 1.000 1.000
300   0.046 1.000 1.000 1.000

The sizes of Tn fluctuate around 0.05 and the pattern of powers resemble that in Tables 1 and 2. Since the networks are binary, the test Tfro is applicable. For denser networks, the test seems to be pretty conservative since almost all the sizes are less than 0.04 in Table 4 and almost all the sizes are zeros in Table 5 (see Remark 2 for explanation). This fact undermines its power significantly. On the contrary, the proposed test Tn has satisfactory power and outperforms Tfro substantially. For sparser networks in Table 3, the sizes of Tfro are closer to 0.05 and has powers close to that of Tn. This simulation shows the advantage of the proposed test Tn over Tfro under the setting of binary graphs.

3.2. Real data application

In this section, we consider applying the proposed method to a real life data that can be downloaded from a public database (http://fcon_1000.projects.nitrc.org/indi/retro/cobre.html). This data set contains Raw anatomical and functional scans from 146 subjects (72 patients with schizophrenia and 74 healthy controls). After a series of processes done by [16], only 124 subjects (70 patients with schizophrenia and 54 healthy controls) were kept. In their study, 263 brain regions of interests were chosen as nodes and connectivity between nodes were measured by the edge weights that represent the Fisher-transformed correlation between the fMRI time series of the nodes after passing to ranks [17].

As the healthy group (54 networks) and patient group (70 networks) have different sample sizes, our test statistic is not directly applicable. We adopt the following two methods to solve this issue. The first way is to randomly sample 16 networks from the 54 networks in the health group and unite them with the 54 networks of health group to yield 70 samples. Then the healthy group and patient group have equal sample sizes and we can calculate the test statistics. This process is repeated 100 times and the five-number summary of the test statistics is presented in Table 6. The second method is to randomly sample 54 networks from the schizophrenia patient group and then calculate the test statistics based on the sampled 54 networks and the 54 networks in the healthy group. The random sampling procedure is repeated 100 times and the five-number summary of test statistics is presented in Table 7. All the calculated test statistics Tn and Tfro are much larger than 1.96, which leads to the same conclusion that the patient population significantly differs from the healthy population at significance level α=0.05. Moreover, the proposed test statistic Tn is almost twice of Tfro, implying that our test is more powerful to detect the population difference.

Table 6.

Repeat sampling 16 networks from 54 networks in the healthy group.

Method Min. 1st Qu. Median 3rd Qu. Max.
Tn 43.52 50.33 53.62 57.05 62.39
Tfro 22.27 25.93 27.43 29.45 32.34

Table 7.

Repeat sampling 54 networks from 70 networks in patient group.

Method Min. 1st Qu. Median 3rd Qu. Max.
Tn 27.81 35.45 37.55 39.83 47.57
Tfro 12.33 15.09 15.88 16.95 20.53

The computation of the proposed test statistic requires randomly splitting two samples into two groups. In order to evaluate the effect of the random splitting on the proposed test, we randomly sample 54 networks from the patient group, denoted as Gk,(1k54). Let Hk,(1k54) be the 54 networks in the healthy group. Consider Gk,Hk,(1k54) as the two samples. We randomly partition the two samples into two groups, denoted as G~k,H~k,(1k27) and G~k,H~k,(27<k54) and then compute the test statistics Tn and Tfro. This procedure is repeated 100 times and the five-number summary of the 100 calculated test statistics are recorded in Table 8. The same conclusion could be drawn based on the 100 statistics, implying that random splitting does not significantly affect the proposed method.

Table 8.

Random splitting of two samples Gk,Hk,1k54.

Method Min. 1st Qu. Median 3rd Qu. Max.
Tn 28.25 35.28 38.19 40.95 46.33
Tfro 12.30 15.12 16.12 17.18 19.21

Additionally, to compare the performance of Tn and Tfro in binary graph setting, we artificially transform the weighted graphs to binary graphs by thresholding as follows. For a given threshold τ, if the absolute value of an edge weight is greater (smaller) than τ, then the edge is transformed to 1 (0). Smaller (larger) τ yields denser (sparser) networks. We take the threshold values τ{0.01,0.03,0.1,0.3,0.5,0.7,0.9}. For each τ, we calculate the test statistics as in Table 6 and the results are summarized in Table 9. The threshold τ dramatically affects the conclusion. For 0.1τ0.7, both Tn and Tfro reject the null hypothesis H0 that the two network populations are the same, with Tn more powerful than Tfro in most cases. However, for τ=0.01 (denser networks), Tn rejects the null hypothesis H0, while Tfro fails to reject H0. This analysis outlines the importance to develop testing procedures for weighted networks, as artificial transforming weighted networks to unweighted networks may lead to contradictory conclusions.

Table 9.

Transforming weighted graphs to unweighted graphs with different threshold τ.

Method Min. 1st Qu. Median 3rd Qu. Max.
Tn  ( τ=0.01) 9.70 16.40 19.61 22.10 31.88
Tfro ( τ=0.01) 0.42 0.70 0.83 0.93 1.31
Tn  ( τ=0.03) 11.91 17.32 21.20 24.90 33.23
Tfro ( τ=0.03) 1.54 2.17 2.62 3.05 3.95
Tn  ( τ=0.1) 11.86 19.66 23.01 25.78 38.03
Tfro ( τ=0.1) 4.88 7.81 9.01 9.93 14.10
Tn  ( τ=0.3) 14.06 26.37 29.64 32.42 41.33
Tfro ( τ=0.3) 11.81 20.90 23.20 25.34 32.16
Tn  ( τ=0.5) 11.86 16.57 18.19 19.21 24.43
Tfro ( τ=0.5) 10.14 13.81 15.10 16.14 20.14
Tn  ( τ=0.7) 5.60 6.93 7.32 7.93 9.37
Tfro ( τ=0.7) 6.55 8.02 8.53 9.08 10.60
Tn  ( τ=0.9) 0.47 1.79 2.19 2.52 3.36
Tfro ( τ=0.9) 0.81 2.54 2.89 3.55 5.02

4. Proof of main results

Proof. Proof of Theorem 2.1 —

We employ the Lindeberg Central Limit Theorem to prove theorem 2.1.

Firstly, note that under H0, we have

σn2=E[sn2]=1i<jnE[Tij2]=1i<jnE[km2(AGk,ijAHk,ij)]2E[k>m2(AGk,ijAHk,ij)]2=1i<jnkm2E(AGk,ijAHk,ij)2k>m2E(AGk,ijAHk,ij)2=1i<jnm2σij4,

Next, we verify the Lindeberg condition. By Cauchy-Schwarz inequality and Markov inequality, it follows that for any ϵ>0,

ETij2I[|Tij|>ϵσn]ETij4P[|Tij|>ϵσn]ETij4ETij4ϵ4σn4=ETij4ϵ2σn2.

Notice that

ETij4=k1,k2,k3,k4m2E(AGk1,ijAHk1,ij)(AGk2,ijAHk2,ij)(AGk3,ijAHk3,ij)×(AGk4,ijAHk4,ij)k1,k2,k3,k4>m2E(AGk1,ijAHk1,ij)(AGk2,ijAHk2,ij)×(AGk3,ijAHk3,ij)(AGk4,ijAHk4,ij).

Since for distinct k1, k2, k3,k4{1,2,,m},

E[(AGk1,ijAHk1,ij)3(AGk2,ijAHk2,ij)]=0,E[(AGk1,ijAHk1,ij)2(AGk2,ijAHk2,ij)(AGk3,ijAHk3,ij)]=0,E[(AGk1,ijAHk1,ij)(AGk2,ijAHk2,ij)(AGk3,ijAHk3,ij)(AGk4,ijAHk4,ij)]=0.

As a result, it follows

k1,k2,k3,k4m2E(AGk1,ijAHk1,ij)(AGk2,ijAHk2,ij)(AGk3,ijAHk3,ij)(AGk4,ijAHk4,ij)=k1=k2k3=k4m2E(AGk1,ijAHk1,ij)2(AGk3,ijAHk3,ij)2+k1=k3k2=k4m2E(AGk1,ijAHk1,ij)2(AGk2,ijAHk2,ij)2+k1=k4k2=k3m2E(AGk1,ijAHk1,ij)2(AGk2,ijAHk2,ij)2+k1=k4=k2=k3m2E(AGk1,ijAHk1,ij)4=k1=k4=k2=k3m2E(AGk1,ijAHk1,ij)4+3k1k2m2E(AGk1,ijAHk1,ij)2×(AGk2,ijAHk2,ij)2

Then we have

ETij4=[km2E(AGk,ijAHk,ij)4+3k1k2m2E(AGk1,ijAHk1,ij)2(AGk2,ijAHk2,ij)2]×[k>m2E(AGk,ijAHk,ij)4+3k1k2>m2E(AGk1,ijAHk1,ij)2(AGk2,ijAHk2,ij)2]=(m2σij4+3mηij)2=m4σij8+6m3σij4ηij+9m2ηij2,

where ηij=E(AGk,ijAHk,ij)4. Hence, by condition (3), it follows that

1σn21i<jnETij2I[|Tij|>ϵσn]1ϵ2σn41i<jnETij4m41i<jnσij8+6m31i<jnσij4ηij+9m21i<jnηij2ϵ2m4(1i<jnσij4)20.

By the Lindeberg Central Limit Theorem, we conclude that σn11i<jnTij converges in distribution to N(0,1).

Finally, we prove Tn converges in distribution to N(0,1) by proving that sn2=(1+op(1))σn2. Note that for i<j and k<l,

E[(Tij2m2σij4)(Tkl2m2σkl4)]=0,if{i,j}{k,l}.

Consequently, one has

E[sn2σn2]2=E[1i<jn(Tij2m2σij4)]2=1i<jnE[(Tij2m2σij4)]2=O(n2m4),

Hence, sn2=σn2+Op(n2m2). If n2m2=o(σn2), then sn2=(1+op(1))σn2, which implies Tn converges in distribution to N(0,1) by Slutsky's theorem.

Proof. Proof of Theorem 2.2 —

Under H1, we have

Λij=E[Tij]=E[km2(AGk,ijAHk,ij)]E[k>m2(AGk,ijAHk,ij)]=km2E(AGk,ijAHk,ij)k>m2E(AGk,ijAHk,ij)=m24(μ1,ijμ2,ij)2,

and

Vij=E(AGk,ijAHk,ij)2=E(AGk,ijμ1,ij+μ1,ijμ2,ij+μ2,ijAHk,ij)2=E(AGk,ijμ1,ij)2+E(μ1,ijμ2,ij)2+E(μ2,ijAHk,ij)2=σ1,ij2+σ2,ij2+(μ1,ijμ2,ij)2.

Then

σ1,n2=Esn2=m241i<jnVij2,

and

E[sn2σ1,n2]2=E[1i<jn(Tij2m24Vij2)]2=1i<jnE[(Tij2m24Vij2)]2=O(n2m4).

Hence, sn2=σ1,n2(1+oP(1)) if nm=o(m41i<jnVij2). Note that

E[1i<jn(TijΛij)]2=1i<jnE(TijΛij)2σ1,n2.

As a result, under H1, the test statistic is decomposed as

Tn=1i<jn(TijΛij)sn+1i<jnΛijsn=σ1,nsn(1i<jn(TijΛij)σ1,n+1i<jnΛijσ1,n)=1i<jnΛijσ1,n+OP(1).

Acknowledgments

The authors are grateful to the Editor, the Associate Editor and Referees for helpful comments that significantly improved this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

References

  • 1.Amini A., Chen A., and Bickel P., Pseudo-likelihood methods for community detection in large sparse networks, Ann. Stat. 41 (2013), pp. 2097–2122. [Google Scholar]
  • 2.Aicher C., The weighted stochastic block model, Applied Mathematics Graduate Theses and diss., 50, 2014.
  • 3.Aicher C., Jacob A., and Clauset A., Learning latent block structure in weighted networks, J. Complex. Netw. 3 (2015), pp. 221–248. [Google Scholar]
  • 4.Abbe E., Community detection and stochastic block models: recent developments, J. Mach. Learn. Res. 18 (2018), pp. 1–86. [Google Scholar]
  • 5.Agarwal S., Branson K., and Belongie S., Higher order learning with graphs, Proc. Int. Conf. Mach. Learn. (2006), pp. 17–24. [Google Scholar]
  • 6.Bickel P.J. and Sarkar P., Hypothesis testing for automated community detection in networks, J. R. Stat. Soc. Ser. B 78 (2016), pp. 253–273. [Google Scholar]
  • 7.Chen J. and Yuan B., Detecting functional modules in the yeast protein-protein interaction network, Bioinformatics 22 (2006), pp. 2283–2290. [DOI] [PubMed] [Google Scholar]
  • 8.Costa L.F., Oliveira Jr O.N., Travieso G., Rodrigues F.A., Villas Boas P.R., Antiqueira L., Viana M.P., and Correa Rocha L.E., Analyzing and modeling real-world phenomena with complex networks: A survey of applications, Adv. Phys. 60 (2011), pp. 329–412. [Google Scholar]
  • 9.Chen L., Zhou J., and Lin L., Hypothesis testing for populations of networks, preprint. Available at https://arxiv.org/pdf/1911.03783.pdf
  • 10.Fortunato S., Community detection in graphs, Phys. Rep. 486 (2010), pp. 75–174. [Google Scholar]
  • 11.Gao C. and Lafferty J., Testing for global network structure using small subgraph statistics, preprint (2017). Available at https://arxiv.org/pdf/1710.00862.pdf
  • 12.Ginestet C.E., Li J., Balanchandran P., Rosenberg S. and Kolaczyk E.D., Hypothesis testing for network data in functional neuroimaging, Ann. Appl. Stat. 11 (2017), pp. 725–750. [Google Scholar]
  • 13.Ghoshdastidar D., Two-sample hypothesis testing for inhomogeneous random graphs, preprint (2019). Available at arXiv:1707.00833.
  • 14.Ghoshdastidar D. and Luxburg V.U., Practical methods for graph two-sample testing, NIPS 2018 (2018), pp. 3019–3028. Montreal, Canada, 2018. [Google Scholar]
  • 15.Garcia J., Ashourvan A., Muldoon S., Vettel J. and Bassett D., Applications of community detection techniques to brain graphs: Algorithmic considerations and implications for neural function, Proc. IEEE Inst. Electr. Electron. Eng. 106 (2018), pp. 846–867. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Relin A., Jess D., Kessler D., Levina E. and Taylor S.F., Network classification with applications to brain connectomics, Ann. Appl. Stat. 13 (2013), pp. 1648–1677. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Relion J.A. and Kessler D., Graphclass: Network classification. R package version 1.1. (2019).
  • 18.Lei J., A goodness-of-fit test for stochastic block models, Ann. Stat. 44 (2016), pp. 401–424. [Google Scholar]
  • 19.Ma'ayan A., Introduction to network analysis in systems biology, Sci. Signal, 4 (2011), pp. tr5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Newman M.E.J., Coauthorship networks and patterns of scientific collaboration, PNAS 101 (2004), pp. 5200–505. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Stam C.J., Jones B.F., Nolte G., Breakspear M. and Scheltens P., Small-world networks and functional connectivity in Alzheimer's disease, Cereb. Cort. 17 (2007), pp. 92–99. [DOI] [PubMed] [Google Scholar]
  • 22.Tang M., Athreya A., Sussman D.L., Lyzinski V., and Priebe C.E., A nonparametric two-sample hypothesis testing problem for random graphs, Bernoulli 23 (2017), pp. 1599–1630. [Google Scholar]
  • 23.Thomas A.C. and Blitzstein J.K., Valued ties tell fewer lies: Why not to dichotomize network edges with thresholds, preprint (2011). Available at arXiv:1101.0788.
  • 24.Yuan M. and Nan Y., Test dense subgraphs in sparse uniform hypergraph, Commun. Stat. Theory Methods (2020) DOI: 10.1080/03610926.2020.1723637. [DOI] [Google Scholar]

Articles from Journal of Applied Statistics are provided here courtesy of Taylor & Francis

RESOURCES