Robust Multi-Network Clustering via Joint Cross-Domain Cluster Alignment

Abstract

Network clustering is an important problem that has recently drawn a lot of attentions. Most existing work focuses on clustering nodes within a single network. In many applications, however, there exist multiple related networks, in which each network may be constructed from a different domain and instances in one domain may be related to instances in other domains. In this paper, we propose a robust algorithm, MCA, for multi-network clustering that takes into account cross-domain relationships between instances. MCA has several advantages over the existing single network clustering methods. First, it is able to detect associations between clusters from different domains, which, however, is not addressed by any existing methods. Second, it achieves more consistent clustering results on multiple networks by leveraging the duality between clustering individual networks and inferring cross-network cluster alignment. Finally, it provides a multi-network clustering solution that is more robust to noise and errors. We perform extensive experiments on a variety of real and synthetic networks to demonstrate the effectiveness and efficiency of MCA.

I. Introduction

Networks (or graphs) are widely used in representing relationships between instances, in which each node corresponds to an instance and each edge depicts the relationship between a pair of instances. Network clustering (or graph clustering) [1]–[3] has become an effective means in discovering modules formed by closely related instances in such networks, which may in turn reveal functional structure of the networks. Recently, the attention has moved from clustering in a single homogeneous network (built on instances from one domain) to joint clustering on multiple heterogeneous networks (from different but related domains), due to obvious reasons: integrating information from different but related domains not only may help to resolve ambiguity and inconsistency in clustering outcome, but also may discover and leverage strong associations between clusters from different domains. Consequently, these multi-view network clustering methods [3], [4] are able to substantially improve the clustering accuracy. For example, millions of genetic variants on human genome have been reported to be disease related, most of which are in the form of single nucleotide polymorphism (SNP). These SNPs do not function independently. Instead, a set of SNPs may play joint roles in a disease. Such interactions between SNPs can be modeled by a SNP interaction network. Fig. 1 shows an exemplar SNP interaction network ${\mathcal{G}}_1$ of 17 SNPs on the left, in which nodes are SNPs and weighted edges represent interactions between SNPs. Even though the underlying biological processes are complex and only partially solved, it is well established that SNPs may alter the expression levels of related genes which may in turn have a cascading effect to other genes, e.g., in the same biological pathways [5]. The interactions between genes can be measured by correlations of gene expressions and represented by a gene interaction network. Fig. 1 shows an exemplar gene interaction network of 20 genes on the right, in which nodes are genes and weighted edges represent interactions between genes. These two networks are heavily related because of the (complicated) relationships between SNPs and genes, as demonstrated in many expression quantitative trait loci (eQTL) studies. These cross-domain relationships are represented by dotted edges between SNPs and genes in Fig. 1. The strength of such relationship is coded by the edge weight. It is evident that a joint analysis becomes essential in these related domains.

Fig. 1.

An exemplar SNP interaction network and gene interaction network in an eQTL study

Despite the success of previous approaches in network clustering, they still suffer from two common limitations. First, existing methods usually assume that information collected in different domains are for the same set of instances. Thus, the cross-domain instance relationships are strictly one-to-one correspondence. This assumption may not hold in many applications. More often than not, data instances (e.g., SNPs) in one domain may be related to multiple instances (e.g., genes) in another domain. Methods that can account for many-to-many cross-domain relationships are in need [6]. Second, existing approaches tend to focus on network clustering and ignore any associations that may be exhibited between clusters from different domains. However, “alignment” between clusters from multiple domains may provide a more comprehensive depiction of the whole system. For example, a cluster of SNPs may jointly regulate the expressions of a cluster of genes, which may be revealed by cluster level associations. Fig. 1 shows 2 SNP clusters: A (including SNPs {1, 2, 3, 4}) and B (including SNPs {12, 13, 14, 16}), and 3 gene clusters: C (including genes {a, b, c, d}), D (including genes {p, q, r, s}) and E (including genes {i, j, k, m}). As summarized in Table I, SNP cluster A is strongly associated with gene cluster C and SNP cluster B is strongly associated with gene cluster D. Gene cluster E is not strongly associated with any SNP cluster. Although we are given cross-domain associations at the instance level, it is still nontrivial to discover cross-domain associations at the cluster level, especially in the presence of noise. Our goal is to discover such strong associations between pairs of clusters from different domains simultaneously when we perform network clustering.

TABLE I.

Cross-domain associations between SNP clusters and gene clusters in Fig. 1

association	1st cluster pair	2nd cluster pair
SNP interaction network ${\mathcal{G}}_1$	{12, 13, 14, 16}	{1, 2, 3, 4}
gene interaction network ${\mathcal{G}}_2$	{p, q, r, s}	{a, b, c, d }

In this paper, we propose a robust approach, MCA (Multi-network Clustering via cluster Alignment), to detect network clusters in multiple domains and their cross-domain associations. In addition to the advantages discussed above, the duality between clustering in individual networks and inferring cross-network cluster alignment enables mutual reinforcement when both tasks are performed simultaneously. As a result, MCA can effectively filter noise and resolve ambiguities in individual networks, and achieve much higher accuracy in detecting network clusters and their cross-domain associations. It also employs a sparsity regularizer on the cluster alignment to provide additional robustness to noise in the prior cross-domain (instance-level) relationships.

Our contributions are summarized as follows.

• To the best of our knowledge, little prior work has studied the problem of cross-domain cluster association detection. In this paper, we propose and investigate this novel problem under the multi-domain setting. The problem is essential to a wide range of applications.

• We develop a framework, MCA, based on nonnegative matrix tri-factorization to simultaneously cluster instances within each domain and reveal the associations between clusters from different domains. Clustering and cluster association discovery could mutually enhance each other. We provide rigorous theoretical analysis of MCA in terms of its correctness, convergence and complexity.

• We evaluate MCA by extensive experiments on both synthetic and real datasets. The experimental results demonstrate that MCA is superior to existing approaches in both clustering accuracy and cluster association accuracy.

II. Related Work

To the best of our knowledge, this is the first work to “align” clusters across multiple domains. Existing work on network clustering primarily focused on clustering in a single network [7], [9]. In [9], the authors pioneered a graph partitioning algorithm using normalized cut. Spectral clustering has gained popularity as an efficient clustering algorithms in recent years [11]. It is simple to implement since its computation burden is primarily on the computation of eigenvectors which has been studied in-depth in numerical analysis. In [7], the authors proposed a framework to detect communities in a network based on modularity. Some other multi-domain graph (network) clustering methods [3] focus on improving the clustering accuracy within each domain utilizing information from other domains. The cross-domain instance relationships are only used to enhance the clustering result within each domain. They do not capture associations between clusters across multiple domains. Multi-domain data are inherently heterogenous. Networks constructed from multiple related domains can be transformed into a heterogenous information network, on which clustering may be performed [12]. However, this pioneer work focused on ranking-based clustering which ranks clusters on a pre-specified target type (domain). This is different from our goal of performing clustering within and cross domains. In addition, some methods on co-clustering also make use of Nonnegative Matrix Tri-Factorization (NMTF) and graph regularizer. Co-clustering [13], DNMTF [14] and RCC [15] were originally designed to improve the clustering accuracy on documents by clustering rows and columns of a term-document matrix simultaneously [16]. They usually pay special attentions to the duality of clustering of terms and clustering of documents. The cross-domain cluster associations are not explicitly considered by co-clustering methods, even though some of these methods may be adapted to derive information about cross-domain cluster associations. They are either incapable of handling multi-network setting or sensitive to noise, since they were not designed for network clustering.

III. Multi-Network Clustering via Cross-Domain Cluster Alignment

In this section, we discuss the problem definition and our proposed algorithm MCA.

A. Problem Definition

Suppose that we have N domains $\{{\mathcal{D}}_1,{\mathcal{D}}_2,\dots ,{\mathcal{D}}_N\}$. Instances and their relationships within each domain are represented by a network ${\mathcal{G}}_p$, (1 ≤ p ≤ N). Let A_p be affinity/adjacency matrix ${\mathcal{G}}_p$. It is possible that some instances in domain ${\mathcal{D}}_p\left(1\le p\le N\right)$ may be related to some instances in domain ${\mathcal{D}}_q\left(1\le q\le N,p\ne q\right)$. These cross-domain relationships between instances can be represented by a matrix W_pq. Important notations are listed in Table II. More often than not, the matrix W_pq is derived from prior knowledge and may be incomplete and noisy. Our goal is to integrate these cross-network instance relationships into the task of multi-network clustering and infer cross-network associations between clusters. We formulate this problem as an optimization problem that generates clustering and cluster associations simultaneously. We now discuss them in detail.

TABLE II.

Summary of symbols and their meanings

Symbol	Description
N	The number of domains
${\mathcal{D}}_p$	The p-th domain
np	The number of instances in ${\mathcal{D}}_p$
kp	The number of clusters in ${\mathcal{D}}_p$
${\mathcal{G}}_p$	The network representing relationship among instances in ${\mathcal{D}}_p$
Ap	The affinity/adjacency matrix of ${\mathcal{G}}_p$
Wpq	The cross-domain relationship between instances from ${\mathcal{D}}_p$ and ${\mathcal{D}}_q$
Hp	The cluster assignment matrix in ${\mathcal{D}}_p$
Spq	The cross-domain alignment matrix between ${\mathcal{D}}_p$ and ${\mathcal{D}}_q$

For simplicity, we begin with 2 domains. For clustering in domain ${\mathcal{D}}_1$, We want to minimize the following objective function

$${\displaystyle \sum _{ij}{\mathbf{A}}_{1_{ij}}}\Vert {\mathbf{H}}_{1_i.}-{\mathbf{H}}_{1_j.}{\Vert }_2^2,s.t.{\mathbf{H}}_1^T{\mathbf{H}}_1=\mathbf{I},{\mathbf{H}}_1\ge 0$$

where H₁ is the cluster assignment matrix in domain ${\mathcal{D}}_1$ and ${\mathbf{H}}_{1_i·}$ means the i-th row of matrix H₁. ${\mathbf{H}}_{1_{i,k}}$ can be viewed as the probability that the i-th instance in domain ${\mathcal{D}}_1$ belongs to the k-th cluster of this domain. A similar objective function can be applied to clustering in domain ${\mathcal{D}}_2$.

In order to capture the cross-network cluster associations, we adopt the co-clustering strategy that minimizes the following objective function

$$\begin{array}{r}\Vert {\mathbf{W}}_{12}-{\mathbf{H}}_1{\mathbf{S}}_{12}{\mathbf{H}}_2^T{\Vert }_F^2+{\eta }_1\Vert {\mathbf{S}}_{12}|{|}_1,\\ s.t.{\mathbf{H}}_1^T{\mathbf{H}}_1=\mathbf{I},{\mathbf{H}}_2^T{\mathbf{H}}_2=\mathbf{I},{\mathbf{H}}_1\ge \mathbf{0},{\mathbf{H}}_2\ge \mathbf{0}\end{array}$$

This objective function is the Sparse Nonnegative Matrix Tri-Factorization. With orthogonality constraints on H₁ and H₂, it is equivalent to running K-means co-clustering on W₁₂ [13]. S₁₂ is the cross-domain alignment matrix, depicting the alignment of the clusters from the two domains. Because W₁₂ may contain noise, we employ the ℓ₁-norm on S₁₂ to suppress the influence of any inconsistencies in W₁₂.

Combining together the above two parts, we have the following optimization problem

$$\begin{array}{r}\mathit{min}\Vert {\mathbf{W}}_{12}-{\mathbf{H}}_1{\mathbf{S}}_{12}{\mathbf{H}}_2^T{\Vert }_F^2+{\alpha }_1{\displaystyle \sum _{ij}{\mathbf{A}}_{1_{ij}}}\Vert {\mathbf{H}}_{1_i.}-{\mathbf{H}}_{1_j.}{\Vert }_2^2\\ +{\alpha }_2{\displaystyle \sum _{ij}{\mathrm{A}}_{2_{ij}}\Vert {\mathbf{H}}_{2_i.}}-{\mathbf{H}}_{2_j.}{\Vert }_2^2+{\eta }_1\Vert \mathbf{S}|{|}_1\\ s.t.{\mathbf{H}}_1^T{\mathbf{H}}_1=\mathbf{I},{\mathbf{H}}_2^T{\mathbf{H}}_2=\mathbf{I},{\mathbf{H}}_1\ge \mathbf{0},{\mathbf{H}}_2\ge \mathbf{0}\end{array}$$ (1)

where α₁, α₂ and η₁ are parameters that balance different between terms, whose values can be determined via cross validation.

By simplifying it, we obtain

$$\begin{array}{r}\underset{{\mathbf{H}}_1\ge \mathbf{0},{\mathbf{H}}_2\ge \mathbf{0},\mathbf{S}\ge \mathbf{0}}{\min }\Vert {\mathbf{W}}_{12}-{\mathbf{H}}_1{\mathbf{S}}_{12}{\mathbf{H}}_2^T{\Vert }_F^2+{\alpha }_1\text{Tr}({\mathbf{H}}_1^T{\mathbf{\Theta }}_1{\mathbf{H}}_1)\\ +{\alpha }_2\text{Tr}({\mathbf{H}}_2^T{\mathbf{\Theta }}_2{\mathbf{H}}_2)+{\eta }_1{\Vert \mathbf{S}\Vert }_1\\ s.t.{\mathbf{H}}_1^T{\mathbf{H}}_1=\mathbf{I},{\mathbf{H}}_2^T{\mathbf{H}}_2=\mathbf{I}\end{array}$$ (2)

where Θ₁ and Θ₂ are the Laplacian Matrices of ${\mathcal{G}}_1$ and ${\mathcal{G}}_2$, respectively.

Now we come back to the multi-domain case. Let W_pq be the matrix defining instance level relationships between domain ${\mathcal{D}}_p$ and domain ${\mathcal{D}}_q$. Here we assume ${\mathbf{W}}_{pq}={\mathbf{W}}_{qp}^T$. Similarly, we use S_pq to represent the cross-domain cluster alignment matrix between domain ${\mathcal{D}}_p$ and domain ${\mathcal{D}}_q$ and we have ${\mathbf{S}}_{pq}={\mathbf{S}}_{qp}^T$. The optimization problem in Eq. (2) can be naturally extended to the following multi-domain case.

$$\begin{array}{r}\underset{{\mathbf{H}}_p\ge \mathbf{0},{\mathbf{S}}_{pq}\ge \mathbf{0},\forall p\ne q}{\min }{\displaystyle \sum _{p\ne q}\Vert {\mathbf{W}}_{pq}-{\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\Vert }_F^2}+\\ {\displaystyle \sum _p{\alpha }_p\text{Tr}\left({\mathbf{H}}_p^T{\mathbf{\Theta }}_p{\mathbf{H}}_p\right)+}\\ {\displaystyle \sum _{p\ne q}{\eta }_{pq}{\Vert {\mathbf{S}}_{pq}\Vert }_1},s.t.\forall p,{\mathbf{H}}_p^T{\mathbf{H}}_p=\mathbf{I}\end{array}$$ (3)

B. Learning Algorithm

In this section, we present the learning algorithm, MCA, to solve the optimization problem in Eq. (3). Since the objective function is not jointly convex with respect to all variables, we adopt an alternating optimization scheme. Specifically, each time we optimize the objective with respective to one variable while fixing others. The following two theorems set the foundation for our algorithm. Their correctness and convergence are guaranteed, which will be proven later.

Theorem 1

While fixing other variables, the objective function in Eq. (3) will monotonically decrease every time we update S_pq according to Eq. (4) until convergence.

$${\mathbf{S}}_{pq}\leftarrow {\mathbf{S}}_{pq}\circ \sqrt{\frac{\left({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q\right)}{\left({\mathbf{H}}_p^T{\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q\right)+\frac{1}{2}{\eta }_{pq}}}$$ (4)

Theorem 2

While fixing other variables, the objective function in Eq. (3) will monotonically decrease every time we update H_p according to Eq. (5) until convergence.

$${\mathbf{H}}_p\leftarrow {\mathbf{H}}_p\circ \sqrt{\frac{{\displaystyle {\sum }_{q\ne p}\left({\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T\right)+{\alpha }_p\left({\mathbf{\Theta }}_p^{-}{\mathbf{H}}_p\right)}}{{\mathbf{H}}_p{\mathbf{H}}_p^T\left({\displaystyle {\sum }_{q\ne p}\left({\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T\right)+{\alpha }_p\left({\mathbf{\Theta }}_p^{-}{\mathbf{H}}_p\right)}\right)}}$$ (5)

where ${\mathbf{\Theta }}_p^{-}$ is the negative part of Θ_p, i.e., ${\mathbf{\Theta }}_{p_{ij}}^{-}=(|{\mathbf{\Theta }}_{p_{ij}}|-{\mathbf{\Theta }}_{p_{ij}})/2$.

Note that ∘, $\frac{(\cdot )}{(\cdot )}$ and $\sqrt{(\cdot )}$ are element-wise multiplication, division and square root, respectively. Based on Theorems 1 and 2, we develop an iterative updating algorithm summarized in Algorithm 1.

C. Correctness Analysis

In this section, we give the correctness analysis of the updating rules in Theorem 1, according to the Karush-Kuhn-Tucker (KKT) condition. The proof of updating rules for Theorem 2 is similar and hence omitted here.

Define the Lagrangian function with respect to S_pq as

$$L({\mathbf{S}}_{pq})=-2\text{Tr}({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T)+\text{Tr}({\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q{\mathbf{S}}_{pq}^T{\mathbf{H}}_p^T)+{\eta }_{pq}{\mathbf{S}}_{pq}-\text{Tr}({\mathbf{\Lambda }}_{pq}{\mathbf{S}}_{pq}^T)$$ (6)

where Λ_pq is a symmetric matrix whose entries are the lagrangian multipliers. Note that ‖S_pq‖₁ becomes S_pq in Eq. (6) because S_pq ≥ 0. We also omit all constant terms with respect to S_pq in Eq.(6). The same tricks are used in the following analysis.

The partial derivative with respect to S_ij is

$$\frac{\partial L({\mathbf{S}}_{pq})}{\partial {\mathbf{S}}_{pq}}=-2({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q)+2({\mathbf{H}}_p^T{\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q)+{\eta }_{pq}-{\mathbf{\Lambda }}_{pq}$$ (7)

From the optimality condition $\frac{\partial L({\mathbf{S}}_{pq})}{\partial {\mathbf{S}}_{pq}}=0$,

$${\mathbf{\Lambda }}_{pq}=-2{\left({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q\right)}_{ij}+2\left({\mathbf{H}}_p^T{\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q\right)+{\eta }_{pq}$$ (8)

The KKT complementarity condition for the nonnegativity of S_ij is

$${\mathbf{\Lambda }}_{pq}{\mathbf{S}}_{pq}=0$$ (9)

Combining with Eq. (8), the KKT complementarity condition becomes

$$(-2{({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q)}_{ij}+2({\mathbf{H}}_p^T{\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q)+{\eta }_{pq})\circ {\mathbf{S}}_{pq}=0$$ (10)

According to Eq. (4), the update rule for S is

$${\mathbf{S}}_{pq}\leftarrow {\mathbf{S}}_{pq}\circ \sqrt{\frac{\left({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q\right)}{\left({\mathbf{H}}_p^T{\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q\right)+\frac{1}{2}{\eta }_{pq}}}$$

At convergence, S_pq at the left-hand side and right-hand side should be equal. Then, via simple derivation, we can verify that the update rule for S_pq in Eq. (4) satisfies the KKT complementarity condition in Eq. (10).

D. Convergence Analysis

In this subsection, we prove the guarantee of convergence using an auxiliary function [17].

Definition 1

[17] Z(h, h′) is an auxiliary function for f(h) if the conditions

$$Z(h,h\prime )\ge f\left(h\right),Z\left(h,h\right)=f\left(h\right)$$ (11)

are satisfied for any given h, h′.

Lemma 1

If Z is an auxiliary function for f, then f is non-increasing under the update [17]

$$h^{\left(t+1\right)}=\mathit{arg}\underset{h}{min}Z(h,h^{\left(t\right)})$$ (12)

Proof

f(h^(t+1)) ≤ Z(h^(t+1), h^(t)) ≤ Z(h^(t), h^(t)) = f(h^(t)).   ■

The auxiliary function with respect to S is

$$\begin{array}{r}Z(S_{pq},S_{pq}^{\prime })=-2{\displaystyle \sum _{i,j}{({\mathbf{H}}_p^T\mathbf{W}{\mathbf{H}}_q)}_{ij}{S}_{p{q}_{ij}}^{\prime }}(1+\mathit{log}\frac{{\mathbf{S}}_{p{q}_{ij}}}{S_{p{q}_{ij}}^{\prime }})\\ +{\displaystyle \sum _{i,j}\frac{{({\mathbf{H}}_p^T{\mathbf{H}}_p{S}_{pq}^{\prime }{\mathbf{H}}_q^T{\mathbf{H}}_q)}_{ij}{\mathbf{S}}_{p{q}_{ij}}^2}{S_{p{q}_{ij}}^{\prime }}}+{\eta }_{pq}{\displaystyle \sum _{i,j}\frac{{\mathbf{S}}_{p{q}_{ij}}^2+{\mathbf{S}}_{p{q}_{ij}}^{\prime 2}}{2{\mathbf{S}}_{p{q}_{ij}}^{\prime 2}}}\end{array}$$ (13)

$$\begin{array}{r}\frac{\partial Z}{\partial {\mathbf{S}}_{p{q}_{ij}}}=-2{\left({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q\right)}_{ij}\frac{{\mathbf{S}}_{p{q}_{ij}}^{\prime }}{{\mathbf{S}}_{p{q}_{ij}}}+\\ 2\frac{{\left({\mathbf{H}}_p^T{\mathbf{H}}_p{\mathbf{S}}_{pq}^{\prime }{\mathbf{H}}_q^T{\mathbf{H}}_q\right)}_{ij}}{{\mathbf{S}}_{p{q}_{ij}}^{\prime }}{\mathbf{S}}_{p{q}_{ij}}+{\eta }_{pq}\frac{{\mathbf{S}}_{p{q}_{ij}}}{{\mathbf{S}}_{p{q}_{ij}}^{\prime }}\end{array}$$ (14)

Letting $\frac{\partial Z}{\partial {\mathbf{S}}_{p{q}_{ij}}}=0$, we obtain

$${\mathbf{S}}_{p{q}_{ij}}={\mathbf{S}}_{p{q}_{ij}}^{\prime }\sqrt{\frac{\left({\mathbf{H}}_p^T{\mathbf{W}}_{pq}{\mathbf{H}}_q\right)ij}{{\left({\mathbf{H}}_p^T{\mathbf{H}}_p{\mathbf{S}}_{pq}^{\prime }{\mathbf{H}}_q^T{\mathbf{H}}_q\right)}_{ij}+\frac{1}{2}{\eta }_{pq}}}$$ (15)

Similarly, the auxiliary function with respect to H_p is

$$\begin{array}{r}Z({\mathbf{H}}_p,{\mathbf{H}}_p^{\prime })=-2{\displaystyle \sum _{q\ne p}{\displaystyle \sum _{i,j}{({\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T)}_{ij}{\mathbf{H}}_{p_{ij}}^{\prime }(1+\mathit{log}\frac{{\mathbf{H}}_{p_{ij}}}{{\mathbf{H}}_{p_{ij}}^{\prime }})+}}\\ {\displaystyle \sum _{q\ne p}{\displaystyle \sum _{i,j}\frac{{({\mathbf{H}}_p^{\prime }{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q{\mathbf{S}}_{pq}^T)}_{ij}{\mathbf{H}}_{p_{ij}}^2}{{\mathbf{H}}_{p_{ij}}^{\prime }}+{\alpha }_p{\displaystyle \sum _{i,j}\frac{{({\mathbf{\Theta }}_p^{+}{\mathbf{H}}_p^{\prime })}_{ij}{\mathbf{H}}_{p_{ij}}^2}{{\mathbf{H}}_{p_{ij}}^{\prime }}}}}\\ -{\alpha }_p{\displaystyle \sum _{i,j,k}{\mathbf{\Theta }}_{p_{ki}}^{-}}{\mathbf{H}}_{p_{kj}}^{\prime }{\mathbf{H}}_{p_{ij}}^{\prime }(1+\mathit{log}\frac{{\mathbf{H}}_{p_{kj}}{\mathbf{H}}_{p_{ij}}}{{\mathbf{H}}_{p_{kj}}^{\prime }{\mathbf{H}}_{p_{ij}}^{\prime }})\\ +{\displaystyle \sum _{i,j}\frac{{({\mathbf{H}}_p^{\prime }{\mathbf{\Lambda }}_p)}_{ij}{\mathbf{H}}_{p_{ij}}^2}{{\mathbf{H}}_{p_{ij}}^{\prime }}-\text{Tr}({\mathbf{\Lambda }}_p)}\end{array}$$ (16)

$$\begin{array}{r}\frac{\partial Z}{\partial {\mathbf{H}}_{p_{ij}}}=-2{\displaystyle \sum _{q\ne p}{\left({\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T\right)}_{ij}}\frac{{\mathbf{H}}_{p_{ij}}^{\prime }}{{\mathbf{H}}_{p_{ij}}}+\\ 2{\displaystyle \sum _{q\ne p}\frac{{\left({\mathbf{H}}_p^{\prime }{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q{\mathbf{S}}_{pq}^T\right)}_{ij}{\mathbf{H}}_{p_{ij}}}{{\mathbf{H}}_{p_{ij}}^{\prime }}}+2{\alpha }_p\frac{{\left({\mathbf{\Theta }}_p^{+}{\mathbf{H}}_p^{\prime }\right)}_{ij}{\mathbf{H}}_{p_{ij}}}{{\mathbf{H}}_{p_{ij}}^{\prime }}\\ -2{\alpha }_p\frac{{\left({\mathbf{\Theta }}_p^{-}{\mathbf{H}}_p^{\prime }\right)}_{ij}{\mathbf{H}}_{p_{ij}}^{\prime }}{{\mathbf{H}}_{p_{ij}}}+2\frac{{\left({\mathbf{H}}_p^{\prime }{\mathbf{\Lambda }}_p\right)}_{\mathit{ij}}{\mathbf{H}}_{p_{ij}}}{{\mathbf{H}}_{p_{ij}}^{\prime }}\end{array}$$ (17)

Letting $\frac{\partial Z}{\partial {\mathbf{H}}_{p_{ij}}}=0$, we obtain

$${\mathbf{H}}_{p_{ij}}={\mathbf{H}}_{p_{ij}}^{\prime }\sqrt{\frac{{\displaystyle {\sum }_{q\ne p}{\left({\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T\right)}_{ij}}+{\alpha }_p{\left({\mathbf{\Theta }}_p^{-}{\mathbf{H}}_p^{\prime }\right)}_{ij}}{{\displaystyle {\sum }_{q\ne p}{\left({\mathbf{H}}_p^{\prime }{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q{\mathbf{S}}_{pq}^T\right)}_{ij}}+{\alpha }_p{\left({\mathbf{\Theta }}_p^{+}{\mathbf{H}}_p^{\prime }\right)}_{ij}+{\left({\mathbf{H}}_p^{\prime }{\mathbf{\Lambda }}_p\right)}_{ij}}}$$ (18)

In order to determine Λ_p, we have

$$\frac{\partial L}{\partial {\mathbf{H}}_p}=-2{\displaystyle \sum _{q\ne p}{\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T}+2{\displaystyle \sum _{q\ne p}{\mathbf{H}}_p{\mathbf{S}}_{pq}{\mathbf{H}}_q^T{\mathbf{H}}_q{\mathbf{S}}_{pq}^T}+2{\alpha }_p{\mathbf{\Theta }}_p{\mathbf{H}}_p+2{\mathbf{H}}_p{\mathbf{\Lambda }}_p$$ (19)

Letting $\frac{\partial L}{\partial {\mathbf{H}}_p}=0$, we can solve Λ_p.

After submitting Λ_p, we obtain

$${\mathbf{H}}_{p_{ij}}={\mathbf{H}}_{p_{ij}}^{\prime }\sqrt{\frac{{\displaystyle {\sum }_{q\ne p}{\left({\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T\right)}_{ij}+{\alpha }_p{\left({\mathbf{\Theta }}_p^{-}{\mathbf{H}}_p^{\prime }\right)}_{ij}}}{{\left({\mathbf{H}}_p^{\prime }{\mathbf{H}}_p^{\prime T}\right({\displaystyle {\sum }_{q\ne p}{\mathbf{W}}_{pq}{\mathbf{H}}_q{\mathbf{S}}_{pq}^T+}{\alpha }_p{\mathbf{\Theta }}_p^{-}{\mathbf{H}}_p^{\prime }\left)\right)}_{ij}}}$$ (20)

E. Complexity Analysis

With proper order of multiplication, updating S and H once require $\mathcal{O}({\tilde{n}}^2\tilde{k}+\tilde{n}{\tilde{k}}^2+{\tilde{k}}^3)$ and $\mathcal{O}(N\tilde{n}{\tilde{k}}^2+{\tilde{n}}^2\tilde{k})$, where $\tilde{n}={max}_p\left\{n_p\right\}$ is the largest number of instances among all domains and k = max_p{k_p} is the largest number of clusters among all domains. If the number of iterations is Iter, the overall time complexity is $O(\mathit{Iter}(\mathit{ñ}{\tilde{k}}^2+{\mathit{ñ}}^2\tilde{k}+{\tilde{k}}^3+N\mathit{ñ}{\tilde{k}}^2))$. In practice, the number of instances is much larger than the number of clusters in a domain, which leads to $\tilde{n}\gg \tilde{k}$. In this scenario, the overall time complexity of MCA can be simplified to $\mathcal{O}(\mathit{Iter}({\mathit{ñ}}^2\tilde{k}+N\mathit{ñ}{\tilde{k}}^2))$.

IV. Experimental Results

In this section, we evaluate the performance of MCA on both synthetic and real datasets. To the best of our knowledge, there is no previous method that was specifically designed to discover cluster associations. Some co-clustering methods might be adapted to infer cluster associations. We compare our method MCA with three well-known co-clustering methods–Nonnegative Matrix Tri-Factorization proposed by Chris Ding (denoted as NMTF_Chris in our paper) [13], Graph Dual Regularization Non-negative Matrix Tri-Factorization (DNMTF) [14] and Robust Co-Clustering (RCC) [15]. The parameters of each algorithm are tuned using a 5-fold cross validation. Other co-clustering methods are not compared because they are either not suitable for network clustering or do not impose nonnegativity on association matrix S. The nonnegativity constraint on S is essential to ensure the result interpretability in this problem setting.

A. Evaluation Metrics

We evaluate our results in two perspectives: clustering accuracy within each domain and cluster association accuracy across domains.

1) Clustering Accuracy

We use the widely used normalized mutual information (MI) metric to evaluate the clustering accuracy in each domain. For any domain $\mathcal{D}$, assume that $\mathcal{C}=\left\{c_i,i=1,2,\cdots ,\widehat{k}\right\}$ is the clustering result where c_i is the i-th cluster. Let $\mathcal{T}=\left\{t_i,i=1,2,\cdots k\right\}$ be the ground truth where t_i is the i-th cluster. The normalized MI is defined as

$$\widehat{M}I\left(\mathcal{C},\mathcal{T}\right)=\frac{MI\left(\mathcal{C},\mathcal{T}\right)}{\mathit{max}\left(H\left(\mathcal{C}\right),H\left(\mathcal{T}\right)\right)}$$ (21)

where $H\left(\mathcal{C}\right)$ and $H\left(\mathcal{T}\right)$ are the entropies for clustering $\mathcal{C}$ and $\mathcal{T}$ and $MI\left(\mathcal{C},\mathcal{T}\right)$ is the mutual information between $\mathcal{C}$ And $\mathcal{T}$.

$$MI(\mathcal{C},\mathcal{T})={\displaystyle \sum _{c_i\in \mathcal{C},t_j\in \mathcal{T}}p(c_i,t_j)\text{ln}\frac{p(c_i,t_j)}{p(c_i)p(t_j)}}$$ (22)

where p(c_i) is the percentage of instances contained in c_i and p(c_i, t_j) is the percentage of instances contained in the intersection of c_i and t_j.

2) Cluster Association Accuracy

To evaluate the cross-network cluster association accuracy, we propose a new metric, Clustering Association Metric (CAM). For simplicity, we consider the case where there are only two domains. Assume that we discover $\widehat{h}$ pairs of cluster associations $\{c_j,c_j^{\prime }\},1\le j\le \widehat{h}$, where c_j is a cluster in domain ${\mathcal{D}}_1$ and $c_j^{\prime }$ is its corresponding cluster in domain ${\mathcal{D}}_2$. Also assume that the ground-truth contains h pairs of cluster associations $\{t_i,t_i^{\prime }\},1\le i\le h$, where t_i is a cluster in domain ${\mathcal{D}}_1$ and $t_j^{\prime }$ is its corresponding cluster in domain ${\mathcal{D}}_2$. Then the CAM is defined by the following equation.

$$\mathit{\text{CAM}}=\frac{1}{h}{\displaystyle \sum _{i=1}^h\underset{1\le j\le \widehat{h}}{\mathit{\text{max}}}}\frac{|(t_i\cup t_i^{\prime })\cap (c_j\cup c_j^{\prime })|}{|(t_i\cup t_i^{\prime })\cup (c_j\cup c_j^{\prime })|}$$ (23)

where ∪ is the set union and ∩ is the set intersection. Here, in order to measure the similarity between an inferred cluster association $\{c_j,c_j^{\prime }\}$ and the ground-truth $\{t_i,t_i^{\prime }\}$, we use $\frac{|(t_i\cup t_i^{\prime })\cap (c_j\cup c_j^{\prime })|}{|(t_i\cup t_i^{\prime })\cup (c_j\cup c_j^{\prime })|}$ to evaluate the degree of overlap between $t_i\cup t_i^{\prime }$ and $c_j\cup c_j^{\prime }$. For each ground-truth association pair $\{t_i,t_i^{\prime }\}$, we get the maximal value of $\frac{|(t_i\cup t_i^{\prime })\cap (c_j\cup c_j^{\prime })|}{|(t_i\cup t_i^{\prime })\cup (c_j\cup c_j^{\prime })|},\forall j=1,\cdots ,\widehat{h}$. The CAM is the average of the maximal values for all ground-truth pairs.

B. Simulation Study

We constructed a simulation study on the two synthetic networks ${\mathcal{G}}_1$, ${\mathcal{G}}_2$ in Fig. 1. We compare our method MCA with DNMTF, NMTF_Chris and RCC with respect to robustness to varying levels of noise in the cross-domain instance-level relationship matrix W₁₂. Fig. 2 shows that MCA achieves much higher clustering accuracy than all existing methods at all noise levels. Fig. 3 demonstrates the clear advantage of MCA over existing methods in capturing cross-domain cluster associations.

Fig. 2.

Clustering accuracy as a function of increasing percentage of noise in W₁₂ on simulated data.

Fig. 3.

Cluster association accuracy as a function of increasing percentage of noise in W₁₂ on simulated data.

C. DBLP Dataset

We also evaluate our method MCA using a labeled DBLP dataset [18], [19]. The dataset consists of papers and authors from 4 research areas: Database (DB), Artificial Intelligence (AI), Data Mining (DM) and Information Retrieval (IR). It contains 20 conferences and 4057 authors. These conferences are listed by area in Table III and the author distribution by area is shown in Table IV. We use ${\mathcal{D}}_1$ to denote the author domain and ${\mathcal{D}}_2$ to denote the conference domain. In ${\mathcal{D}}_1$, the network ${\mathcal{G}}_1$ represents the co-authorship. Each entry $A_{1_{ij}}$ in the affinity matrix A₁ of ${\mathcal{G}}_1$ is the number of papers coauthored by the i-th and j-th authors. The affinity matrix A₂ of ${\mathcal{G}}_2$ represents the similarities between the topics covered by two conferences. To compute it, we first construct the term-conference matrix F, in which each entry F_ij is the number of occurrences of the i-th term in the titles of papers published in the j-th conference. Thus each column F_j of the matrix can be viewed as a feature vector describing the j-th conference. The similarity score of two conferences j and j′ can be computed as $\frac{{\mathbf{F}}_j\cdot {\mathbf{F}}_{j\prime }}{\Vert {\mathbf{F}}_j\Vert \Vert {\mathbf{F}}_{j\prime }\Vert }$. W₁₂ represents the relationships between authors and conferences, in which each entry denotes the number of papers that an author published in a given conference. A snapshot of the DBLP network used in our experiment can be seen in Figure 6.

TABLE III.

List of conferences from each research area in the DBLP dataset

DB	AI	DM	IR
PODS	AAAI	KDD	SIGIR
SIGMOD	ICML	ICDM	WWW
VLDB	IJCAI	SDM	WSDM
EDBT	CVPR	PKDD	ECIR
ICDE	ECML	PAKDD	CIKM

TABLE IV.

Number of authors from each research area in the DBLP dataset

	DB	AI	DM	IR
number	1197	1109	745	1006
percentage	29.5%	27.3%	18.4%	24.8%

Fig. 6.

A snapshot of the real DBLP network. We only display edges whose weights are above some threshold.

To compare the robustness of different methods, we introduce noise by randomly shuffling a certain percentage of the entries in W₁₂. Fig. 4 shows that noise in the prior knowledge on cross-domain relationships does not affect the clustering accuracy of MCA in the conference domain at all, and only lowers the accuracy of MCA in the author domain modestly when W₁₂ is dominated by noise. Fig. 5 shows that the accuracy of the inferred cross domain cluster associations also only drops modestly for MCA when the noise level is very high. In contrast, we observe that all other methods are far more sensitive to noise, among which NMTF_Chris performs noticeably better than ONMTF and RCC.

Fig. 4.

Normalized Mutual Information with respect to different noise levels on the DBLP dataset.

Fig. 5.

Cluster association accuracy with respect to different noise levels on the DBLP dataset.

To better understand how these methods perform, we list the top 4 associations between conference clusters and author clusters returned by MCA and NMTF_Chris in Table V when the noise level is set to 30%. We do not list the results by DNMTF and RCC because they return only a single cluster that includes all conferences in the conference domain. This is obviously not what we desire. From Table V, we observe that MCA produces the correct clustering result in the conference domain. The conference cluster in each of the top 4 pairs corresponds to a distinct research area. However, NMTF_Chris makes many mistakes. It splits the conferences from the Database area into two clusters. The third and fourth conference clusters are mixtures of conferences from different areas. In the author domain, for each author cluster, the percentage of authors from each of the 4 research areas is also shown in Table V. Each author cluster returned by MCA is primarily dominated by authors from one research area, as indicated by the largest percentage highlighted in bold in each column, which perfectly matches the area suggested by the associated conference cluster. For example, consider the 1st pair of conference cluster and author cluster returned by MCA. The conference cluster includes PODS, SIGMOD, VLDB, ICDE and EDBT, all of which are Database conferences. 94.2% of authors in the author cluster also come from the Database area. MCA correctly infers this association between the author cluster and conference cluster. We can make same observation on the remaining cluster pairs by MCA in Table V. It demonstrates that MCA can discover meaningful associations between clusters from different domains. From Table V, we also observe that some conference clusters and author clusters discovered by NMTF_Chris represent a mixture of multiple research areas. To further quantify this observation, for the 4 author clusters, we compute the KL-divergence between author’s research area distributions of each pair of author clusters. A KL-divergence of 1 indicates that authors in the two clusters are from two distinct areas. A KL-divergence of 0 indicates that the two author clusters have identical research area distributions and thus are not distinguishable from each other. We use dark color for small KL-divergence and light color for large KL-divergence in Fig. 7. A diagonal entry depicts the KL-divergence of an author cluster to itself which is always 0. Off-diagonal entries correspond to KL-divergence of two author clusters — the larger the KL-divergence, the better the clustering result. We observe that the 4 clusters by MCA all have large KL-divergence to each other but the first two clusters by NMTF_Chris have small KL-divergence. This proves that MCA is more robust to random noise.

TABLE V.

The results on DBLP by MCA and NMTF_Chris at 30% noise level

			pair No. 1	pair No. 2	pair No. 3	pair No. 4
MCA	conference		PODS SIGMOD VLDB ICDE EDBT	KDD ICDM SDM PKDD PAKDD	AAAI ICML IJCAI CVPR ECML	SIGIR WWW WSDM ECIR CIKM
	author	DB	0.943	0.134	0.038	0.293
		DM	0.036	0.835	0.151	0.088
		AI	0.007	0.000	0.698	0.00
		IR	0.014	0.031	0.113	0.619
NMTF_Chris	conference		VLDB ICDE EDBT	PODS SIGMOD	AAAI ICML IJCAI ECML PKDD PAKDD	SIGIR WWW WSDM ECIR CIKM KDD ICDM SDM CVPR
	author	DB	0.736	0.868	0.035	0.084
		DM	0.236	0.088	0.292	0.361
		AI	0.000	0.022	0.602	0.042
		IR	0.028	0.022	0.071	0.513

Fig. 7.

The pairwise KL-divergence between research area distributions of author clusters in Table V.

Duplicate Names

It is possible that some authors may have the same name in the DBLP database. Publications by these authors might be mistakenly associated with other authors with the same name. In order to evaluate the robustness of our method in this context, we first randomly pick a certain percentage of authors and then randomly pair them up. We “pretend” that the two authors in each pair use the same name and thus their publications are not distinguishable. We replace the two corresponding row vectors in W₁₂ by their average vector. Since only a small percentage of authors may have this issue, we only test the robustness for up to 30% of duplicate names. Fig. 8 shows that MCA can successfully resolve the ambiguity and thus its clustering accuracies and cluster association accuracy are not impacted. NMTF_Chris is the second best, having comparable accuracies in cluster association and author clustering, but failing short on the conference clustering accuracy.

Fig. 8.

Results on the DBLP dataset with duplicate names.

D. Yeast eQTL Dataset

Expression quantitative trait loci (eQTL) mapping is the process of identifying single nucleotide polymorphisms (SNPs) that play important role in the expression of genes. It has been widely used to dissect genetic basis of complex traits [5]. Traditionally, associations between individual expression traits and SNPs are assessed separately [20]. Since genes in the same biological pathways are often coregulated and may share a common genetic basis [21], it is crucial to understand how multiple modesetly-associated SNPs interact to influence the phenotypes [22]. To answer this question, several approaches have been proposed to study the joint effect of multiple SNPs by testing the association between a set of SNPs and a gene expression trait [23]–[26]. Despite their success, these methods have two common limitations. First, only the association between a set of SNPs and a single expression trait is studied. Therefore, they overlook the joint effect of a set of SNPs on the activities of a set of genes, which may act and interact with each other to achieve certain biological function. Second, the SNP sets used in these methods are usually taken from known biological pathways, which are far from being complete. These methods cannot identify unknown associations between SNP sets or gene sets. To better elucidate the genetic basis of gene expression and understand the underlying biology pathways, it is highly desirable to develop methods that can automatically infer association between a group of SNPs and a group of genes. The process of identifying such associations is referred to as group-wise eQTL mapping, to distinguish it from the individual eQTL mapping [6] process that identifies associations between individual SNPs and genes. The MCA method proposed in this paper is suitable for group-wise eQTL mapping.

We compare MCA with NMTF_Chris, DNMTF and RCC on a yeast eQTL dataset [27]. This dataset originally includes expression profiles of 6229 genes and genotype profiles of 2956 SNPs. After preprocessing (e.g., removing missing values), the dataset is reduced to 1017 SNPs and 4474 genes expression profiles.

We denote the SNP domain as ${\mathcal{D}}_1$ and the gene domain as ${\mathcal{D}}_2$, respectively. The SNP interaction network ${\mathcal{G}}_1$ is generated as in [28]. The gene interaction network ${\mathcal{G}}_2$ is constructed by computing the Pearson’s correlation of the expression levels of each pair of genes.

$$r_{XY}=\frac{{\displaystyle {\sum }_i\left(X_i-\overline{X}\right)\left(Y_i-\overline{Y}\right)}}{\sqrt{{\displaystyle {\sum }_i{\left(X_i-\overline{X}\right)}^2}}\sqrt{{\displaystyle {\sum }_i{\left(Y_i-\overline{Y}\right)}^2}}}$$ (24)

where X and Y are vectors representing the expression profiles of the two genes. X_i and Y_i are the ith components of X and Y, respectively. From Eq. (24), the value of Pearson’s correlation r ranges from −1 to 1, where 1 means that two genes are completely positively correlated and −1 means that they are completely negatively correlated. The edge between genes X and Y in the gene interaction network is weighted by |r_XY |. The association matrix W₁₂ is given by the association tests between individual SNPs and individual genes using PLINK [29].

Gene Ontology Enrichment Analysis

Since there is no ground-truth in the Yeast eQTL dataset, we cannot measure the clustering accuracy and cluster association accuracy directly. Here, we evaluate the quality of our result by the Gene Ontology Enrichment Analysis (GOEA) [30]. For each inferred gene cluster c_i, we identify the most significantly enriched Gene Ontology categories [31]. The significance (p-value) is determined by the Fisher’s exact test. The raw p-values are further calibrated to correct for the multiple testing problem [32]. To compute calibrated p-values for c_i, we perform a randomization test, wherein we apply the same test to 1000 randomly created gene sets that have the same number of genes as in c_i. In order to evaluate the clusters in the SNP domain, we first need to map the SNPs in a cluster to their nearest genes on the genome, and then apply the standard procedure of GOEA on the set of genes to compute a p-value. In Fig. 9, clusters are arranged in ascending order of their p-values. We consider the clusters with p-value less than 0.05 to be significant. The numbers of significant gene and SNP clusters are listed in Table VI. Not surprisingly, MCA can identify more significant clusters in both gene and SNP domains than the competitors.

Fig. 9.

Gene ontology enrichment analysis on the yeast eQTL data.

TABLE VI.

The number of significantly enriched clusters measured by GOEA

	MCA	NMTF_Chris	DNMTF	RCC
gene	36	31	23	26
SNP	99	98	82	62

E. Gene Disease Dataset

We further evaluate our algorithm MCA on a gene disease network dataset [33]. The dataset contains 590 disease phenotypes in 20 disease classes and 7997 genes in 200 gene pathways. There are two domains: gene domain ${\mathcal{D}}_1$ and disease phenotype domain ${\mathcal{D}}_2$. ${\mathcal{G}}_1$ represents the “functional” relationships between genes which are measured by interactions between the proteins transcribed from the genes, because most genes “perform” their functions through their transcribed proteins. This protein-protein interaction network can be obtained from HPRD [34]. The relationships among phenotypes are represented by a phenotype similarity network ${\mathcal{G}}_2$, which is obtained from [35]. It is an undirected network with vertices representing OMIM [36] disease phenotypes and edges (with weights between 0 and 1) representing the similarities between phenotypes measured by their co-occurrences in clinical synopsis records. The associations between disease phenotypes and genes are also available in OMIM. We evaluate the clustering accuracy in each domain using the normalized MI discussed in Section IV-A. The first row in Table VII is the normalized MI in the phenotype domain and the second row is the normalized MI in the gene domain. As we can see, MCA is again the winner.

TABLE VII.

Results on the gene disease network

method	MCA	NMTF_Chris	DNMTF	RCC
Normalized MI_pheno	0.19	0.13	0.14	0.15
Normalized MI_gene	0.05	0.02	0.05	0.04

F. Performance Evaluation

In this section, we study the run-time performance of MCA, measured by the number of iterations before converging to a local optima. Table VIII summarizes the network size and the number of iterations upon convergence on difference data sets. We observe that MCA can converge within a reasonable number of iterations even for large networks. As expected, the number of iterations will increase as the network size increases. Usually, several hundreds of iterations are needed before convergence, but the actual running time is fast. Table IX shows the time used by different methods to convergence on the DBLP dataset. All methods except RCC run very fast. We can conclude that MCA can achieve much better accuracy without entailing more computation time.

TABLE VIII.

Number of iterations to converge

dataset	size of network ${\mathcal{G}}_1$	size of network ${\mathcal{G}}_2$	number of iterations
synthetic	17	20	24
DBLP	441	20	80
eQTL	1017	4474	741
Gene-disease	3619	366	144

TABLE IX.

Amount of time to converge on DBLP

method	MCA	NMTF_Chris	DNMTF	RCC
time cost (second)	0.8	0.4	0.4	11.5

V. CONCLUSION

In this paper, we propose a novel algorithm, MCA, for network clustering across multiple related domains. By leveraging the duality between single network clustering and inferring cross-network cluster alignment, MCA well incorporates any prior knowledge on cross-network instance relationships into multi-network clustering. The algorithm is robust to noise and is capable of detecting cross-domain associations between clusters, which, was never addressed in previous study. Extensive experiments on both synthetic and several real datasets demonstrate the effectiveness and efficiency of MCA and its advantages over existing methods.