Layer reconstruction and missing link prediction of multilayer network with a maximum a posteriori estimation

Abstract

From social networks to biological networks, different types of interactions among the same set of nodes characterize distinct layers, which are termed multilayer networks. Within a multilayer network, some layers, confirmed through different experiments, could be structurally similar and interdependent. In this paper, we propose a maximum a posteriori based method to study and reconstruct the structure of a target layer in a multilayer network. Nodes within the target layer are characterized by vectors, which are employed to compute edge weights. Further, to detect structurally similar layers, we propose a method for comparing networks based on the eigenvector centrality. Using similar layers, we obtain the parameters of the conjugate prior. With this maximum a posteriori algorithm, we can reconstruct the target layer and predict missing links. We test the method on two real multilayer networks, and the results show that the maximum a posteriori estimation is promising in reconstructing the target layer even when a large number of links is missing.

I. INTRODUCTION

Network science has been widely used in different areas, such as information diffusion, infectious disease spread, and gene co-expression analysis. Through network analysis, one can study the relations among nodes, and the robustness of a system [1, 2]. For example, epidemiologists can predict the number of people infected by COVID-19 through epidemic analysis. Then, they can provide advice to policymakers at an early stage to curb the spreading of the disease [3]. By constructing co-expression networks, biologists can discover crucial genes (nodes) by simply choosing genes with high degree centralities, closeness centralities, or eigenvector centralities. In a typical protein network, five to seven layers are considered to represent different types of molecular interactions [4, 5], including proteolysis, genetic interaction, co-expression, etc. Such multilayer networks are obtained through biological experiments, which could be expensive and time-consuming. A layer can be particularly important but also incomplete, with many missing links. A critical research goal is to reconstruct this important layer, called the target layer, without performing the expensive experiments, but by exploiting all information embedded in the other layers. In other words, we can estimate the target layer through existing layers [4, 6–9]. After constructing the target layer, researchers can devote restricted resources to the detection of edges with high probabilities.

Various methods have been proposed to reconstruct networks and predict missing links, and most of them are based on generative models [4, 6, 7, 10–24]. A generative model reconstructs the network topology by fitting a stochastic network model [7, 13, 14, 25], and uses a maximum-likelihood estimation (MLE) algorithm to find the optimal parameters that can describe the network. In [13, 14], the authors presented a degree-correlated stochastic block model to reconstruct a single layer network. Edges are computed through the tensor product of node vectors, and the entries of the node vectors are the degrees of the nodes in each community. Authors in paper [7] extended the single layer stochastic block model to multilayer networks and used it to predict missing links and detect overlapping communities. The authors tried all the layer combinations to find the layers that can improve the maximum likelihood. However, when there are many layers, it is burdensome to try all the layer combinations to find the interdependent layers. The authors validated the algorithm through two real multilayer networks by hiding 20% of links and non-links.

In multilayer networks, there could be structurally similar layers. Therefore, it is possible to take advantage of similar layers to help reconstruct the target layer. We propose comparing the target layer with the remaining layers if the target layer is partially known. In the literature, multiple methods have been proposed to compare networks [26–30]. The authors in [27, 28] present a method called DeltaCon. The DeltaCon method compares the affinity scores of every pair of nodes in two networks. The method is very sensitive to changes in the number of edges, and the removal of edges results in a significant change in the distance. Papers [29, 30] review and compare some network-comparing methods, including vertex/edge overlapping, vertex/edge vector similarity, and the SimHash algorithm. The vertex/edge overlapping method applies the rule that two graphs are similar if they share many vertices and edges. According to the analysis in [29], the drawbacks of this method are that it is not sensitive to changes in high-quality vertices, topology, and properties of networks. The vertex/edge vector similarity method compares the node/edge weight vectors of two networks. The drawback of this method is that it is not sensitive to changes in the topology and other properties of networks. To take advantage of the features of networks, the SimHash algorithm is introduced to compare networks. The PageRank [31] together with edges are used as network features in SimHash algorithm to compare web page networks.

In this paper, we propose a maximum a posteriori (MAP) based-method for target layer reconstruction as well as for link prediction. The MLE algorithm and entropy-related approaches must depend on the known information of the target layer. Consequently, the reconstruction is significantly affected by the available information of the target layer. In the MAP algorithm, the layers that are similar to the target layer will be considered to compute the parameters of the conjugate prior. Experimental results show that the MAP algorithm provides more consistent results than the MLE method. The first contribution of this paper is to discover an incomplete target layer by computing its edges through a dot product of node vectors. The optimal entries of node vectors are obtained by maximizing the posterior probabilities of the stochastic model. In our experiments, we find that the model accuracy can be improved if we increase the dimension of node vectors, but the return is diminishing for large vector dimensions. Another contribution is that we introduce the eigenvector centrality-based SimHash algorithm to detect structurally similar layers (interdependent layers). The eigenvector centralities of nodes are extracted as network features, which allow us to recover the structure of networks, as shown in the experimental results. In this work, we assume that the number of edges between any pair of nodes follows Poisson distribution [7, 13]. Hence, the Gamma distribution will be the conjugate prior for the Poisson distribution [32]. We compute the parameters of the conjugate prior through the adjacency matrices of similar layers, and the contributions of the similar layers are weighted by their similarities. The number of edges between each pair of nodes is calculated as the dot product of the node vectors. In our experiments, we show that similar layers are critical in improving the robustness of link predictions.

The paper is organized as follows. In section two, we first introduce the MAP method on target layer reconstruction. Next, we propose the eigenvector centrality-based SimHash algorithm to find structurally similar layers. Then, we propose the process for identifying parameters of the conjugate prior under different circumstances. In section three, we first evaluate the eigenvector centrality based SimHash algorithm on two real multilayer networks. Then, we evaluate the MAP algorithm-based target layer reconstruction on the two real networks and compare the differences between the MLE algorithm and the MAP algorithm. We conclude the paper in section four.

II. LAYER RECONSTRUCTION IN MULTILAYER NETWORKS

A. Maximum a posteriori based stochastic model

In this section, we define the stochastic model for both directed and undirected multilayer networks. The adjacency matrix of the target layer is denoted by A. The goal of this reconstruction is to estimate A, given a partial knowledge of the target layer and of other layers in the multilayer network. To reconstruct the target layer, a set of parameters is needed to describe the model, which we denote as θ. Based on Bayes’ theorem, the posterior probability of θ is

$$P(\theta \mid A)=\frac{P(A\mid \theta )P(\theta )}{P(A)},$$ (1)

where P(θ | A) is the posterior probability of θ, P(A | θ) is the likelihood of A under θ, P(θ) is the prior probability of θ and P(A) is the marginal likelihood that contains all the information of the network. Since P(A) is a constant, P(θ | A) is proportional to the product of P(A | θ) and P(θ). Therefore, we have

$$P(\theta \mid A)\propto P(A\mid \theta )P(\theta ).$$ (2)

For any pair of nodes in the network, we use E_ij to denote the expected number of links (which could be fractional) between node i and node j. In unweighted networks, the entries of the adjacency matrix are denoted by 0 or 1. Here, since the entries E_ij are real numbers, we can interpret network A as a weighted network.

Before we substitute any parameters into expression (2), we make the following assumptions. The links in the target layer are independent and identically distributed. In other words, the number of edges between node i and node j does not affect the relation between node i and node k. Further, we assume the number of links between any pair of nodes is extracted from a Poisson distribution, i.e., $P\left(A_{ij}\mid E_{ij}\right)=\frac{e^{-E_{ij}}{\left(E_{ij}\right)}^{A_{ij}}}{A_{ij}!}$. We can rewrite expression (2) after substituting E_ij and A_ij as

$$P(\theta \mid A)\propto \prod _{i,j}\frac{e^{-E_{ij}}{\left(E_{ij}\right)}^{A_{ij}}}{A_{ij}!}P\left(E_{ij}\right).$$ (3)

In the MLE algorithm, the prior probability P(E_ij) can be neglected since it is a constant. In the MAP algorithm, we need to specify the prior distribution of P(E_ij). The conjugate prior distribution for the Poisson distribution is the Gamma distribution

$$\begin{gathered} P\left(E_{ij}\right)=\frac{{\beta }_{ij}^{{\alpha }_{ij}}}{\Gamma \left({\alpha }_{ij}\right)}{E}_{ij}^{{\alpha }_{ij}-1}{e}^{-{\beta }_{ij}{E}_{ij}} \\ \propto E_{ij}^{{\alpha }_{ij}-1}{e}^{-{\beta }_{ij}{E}_{ij}}, \end{gathered}$$ (4)

where α_ij, β_ij and Γ(α_ij) are the shape parameter, the scale parameter, and the Gamma function of the Gamma distribution, respectively. In section II C, we introduce a procedure to determine α_ij and β_ij through the layers with high similarities. Substituting the conjugate Gamma distribution into expression (3), we have

$$P(\theta \mid A)\propto \prod _{i,j}{e}^{-(\beta +1)E_{ij}}{\left(E_{ij}\right)}^{A_{ij}+\alpha -1}.$$ (5)

Note that we have left out constant terms.

The problem now has been simplified to finding the parameters E_ij that can maximize the posterior probability. However, an expression for E_ij has not been specified. In this work, we compute the links through node vectors. The nodes in the target layer are represented by vectors. The expected number of links E_ij can be computed by

$$E_{ij}=\sum _{z=1}^K{s}_{iz}{t}_{jz},$$ (6)

where s_iz and t_jz are respectively the zth entry of node i’s vector and node j’s vector. Here, we use s and t to denote source and target nodes. K is the dimension of the vector. Some MLE related works use tensor factorization to decompose the links, and the dimension of the tensor is interpreted as the number of overlapping communities. As a result, there should be an optimal number of communities that can maximize the estimation accuracy. However, this cannot be rigorously achieved and the tensor factorization can be simplified to the dot product according to our analysis in appendix A.

Expression (5) is still intractable after substituting E_ij with equation (6). We take the logarithmic form of expression (5), which gives

$$\begin{gathered} L(\theta \mid A)=\sum _{i,j}\left[\left(A_{ij}+{\alpha }_{ij}-1\right)\text{log}{E}_{ij}-\left({\beta }_{ij}+1\right)E_{ij}\right] \\ =\sum _{i,j}[\left(A_{ij}+{\alpha }_{ij}-1\right)\text{log}\sum _z^K{s}_{iz}{t}_{jz} \\ -\left({\beta }_{ij}+1\right)\sum _z^K{s}_{iz}{t}_{jz}], \end{gathered}$$ (7)

where L(θ | A) is the log posterior.

To find the maximized posterior for expression (7), we apply the Jensen’s inequality $\text{log}\overline{x}\ge \overline{\text{log}x}$, which gives

$$\begin{gathered} \text{log}\sum _z^K{s}_{iz}{t}_{jz}=\text{log}\sum _z^K{q}_{ijz}\frac{s_{iz}{t}_{jz}}{q_{ijz}} \\ \ge \sum _z^K{q}_{ijz}\text{log}\frac{s_{iz}{t}_{jz}}{q_{ijz}} \\ =\sum _z^K{q}_{ijz}\left(\text{log}{s}_{iz}{t}_{jz}-\text{log}{q}_{ijz}\right). \end{gathered}$$ (8)

The equality is satisfied when

$$q_{ijz}=\frac{s_{iz}{t}_{jz}}{\sum _z{s}_{iz}{t}_{jz}}.$$ (9)

After substituting expression (8) and equation (9) into equation (7), equation (7) can be simplified to

$$\begin{gathered} L(\theta \mid D)=\sum _{i,j,z}[\left(A_{ij}+{\alpha }_{ij}-1\right)q_{ijz}\text{log}{s}_{iz}{t}_{jz} \\ -\left({\beta }_{ij}+1\right)s_{iz}{t}_{jz}]. \end{gathered}$$ (10)

Taking the derivative of equation (10) and equating to zero, we obtain the values of s_iz and t_iz, the optimal node vectors that maximize the posterior probability:

$$s_{iz}=\frac{{\sum }_j\left(A_{ij}+{\alpha }_{ij}-1\right)q_{ijz}}{{\sum }_j\left({\beta }_{ij}+1\right)t_{jz}},$$ (11)

$$t_{jz}=\frac{{\sum }_i\left(A_{ij}+{\alpha }_{ij}-1\right)q_{ijz}}{{\sum }_i\left({\beta }_{ij}+1\right)s_{iz}}.$$ (12)

The procedure to obtain the optimized s_iz and t_jz is to assign random initial values for s_iz and t_jz, then update equation (9), (11), and (12) iteratively until equation (7) converges. However, before applying the above iteration, we need to identify α_ij and β_ij, which are introduced in section II B and section II C.

B. Similarity and layer comparison

In section II A, we have detailed the procedure to compute the optimal node vectors by maximizing the posterior probability. The parameters α_ij and β_ij for the prior distribution are required to perform the posterior probability maximization. We propose to compute α_ij and β_ij through the layers of the multilayer network that are similar to the target layer. Keep in mind that there are missing links in the target layer, and the percentage of missing links is not known at all. Therefore, the primary factor for an effective network-comparing method is that the method must not be significantly affected by the percentage of missing links.

Networks can be characterized by multiple types of centralities, such as, degree centrality, eigenvector centrality, closeness centrality. The degree centrality measures the importance of a node by capturing the number of links the node has, while the eigenvector centrality can be regarded as an extension of degree centrality in which node’s importance is also affected by its neighbors’ importance. The closeness centrality of a node is the average length of shortest paths between the node and all other nodes. One or more of the centralities can describe the features of a network. To compare the target layer with the other layers, we can compare the features of the layers. For our purpose, the eigenvector centrality is selected as the network feature, and is used in the SimHash algorithm to compute similarities.

The SimHash algorithm works as follows [29, 30]. The feature of a network can be expressed as a set of token-value pairs {(v_i : w_i)}, where v_i is a node and w_i is its measure under the feature, for example its eigenvector centrality. Note that the target layer and the layer to be compared have the same set of tokens. In cryptography, any messages can be encrypted to a unique binary number (digest). Similarly, we can represent each token with a unique binary number with ϕ bits (2^ϕ > the number of v_i). For each binary number (digest), we map every 1 to w_i, and 0 to −w_i. Thus, each token is mapped to a weighted digest with ϕ digits. To obtain the weighted digest of the network, we sum up all the weighted digests of the tokens. Note that there is no carry in the summation.

To measure the similarities between the target layer and the other layers, we can compare the digests of the layers. A simple way to compare the digests is to convert the weighted digests to binary digests. The binary digest of a network can be obtained by setting positive digits to 1 and negative digits to 0. The similarity between the target layer m and any other layer r can be measured by

$${\mu }_{m,r}=1-\frac{\mathit{\text{Hamming}}\left(H_m^d,H_r^d\right)}{\varphi },$$ (13)

where μ_m,r is the similarity between layer m and r, $H_m^d$ is the binary digest of the target layer, and $H_r^d$ is the binary digest of the layer to be compared, respectively.

An alternative way to obtain the similarities is by computing the Pearson correlation coefficient of the weighted digests.

The estimation results based on the Pearson correlation are shown in appendix B. In this work, we measure the similarity between the target layer and the other layers through the binary-based digests. The influence of the bit number is discussed in appendix C.

C. Identify the parameters of the Gamma distribution prior

Finally, we introduce the procedure to determine the parameters α_ij and β_ij of the conjugate prior. We discuss this problem in two cases.

In the first case, we assume the structure of the target layer is partially known, i.e., the entries of the adjacency matrix are partially known. In this case, we apply the layer comparison method introduced above, and the L′ layers with highest similarities are considered to identify the parameters of conjugate priors. The parameters of the Gamma distribution prior are computed as

$$\{\begin{array}{l}{\alpha }_{ij}=\mathit{\text{max}}\left(\sum _{r=1}^{L^{\prime }}{\mu }_{m,r}{A}_{ij}^r,1\right)\hfill \\ {\beta }_{ij}=\mathit{\text{max}}\left(\sum _{r=1}^{L^{\prime }}{\mu }_{m,r},\frac{\sum _{r=1}^{L^{\prime }}{\mu }_{m,r}}{\sum _{r=1}^{L^{\prime }}{\mu }_{m,r}{A}_{ij}^r}\right),\hfill \end{array}$$ (14)

where m denotes the target layer, μ_m,r is the similarity between the target layer m and any layer r. $A_{ij}^r$ is the adjacency matrix of layer r in L′. In equations (11) and (12), since A_ij could be zero, if α_ij is less than one, we obtain a negative s_ij. Thus, we need to limit the range of α_ij. If ${\sum }_{r=1}^{L^{\prime }}{\mu }_{m,r}{A}_{ij}^r\in (0,1)$, we set α_ij = 1, and set β_ij to $\frac{{\sum }_{r=1}^{L^{\prime }}{\mu }_{m,r}}{{\sum }_{r=1}^{L^{\prime }}{\mu }_{m,r}{A}_{ij}^r}$ to maintain the means of the Gamma distribution prior unchanged. If ${\sum }_{r=1}^{L^{\prime }}{\mu }_{m,r}{A}_{ij}^r=0$, we will set α_ij = 1, and set β_ij equal to a large number to ensure the MAP algorithm converges.

A special case is the structure of the target layer is not known at all. In this case, the comparison between the target layer and the other layers is not feasible. In this case, an alternative and heuristic way to compute the parameters of conjugate prior can be based on the functionally similar layers. If available, we can use additional published networks and data as a new multilayer network, in which layers are functionally similar. Thereafter, we can apply the proposed MAP algorithm to compute the parameters of conjugate prior through this new multilayer network. The similarities between the target layer and the functionally layers cannot be determined through any network comparison algorithm. Thus, we assume the similarities are all ones. We assign the parameters of the Gamma distribution as

$$\{\begin{array}{l}{\alpha }_{ij}={\sum }_{r=1}^{L^{\prime }}{A}_{ij}^r\hfill \\ {\beta }_{ij}=L^{\prime }.\hfill \end{array}$$ (15)

Similarly, if ${\sum }_{r=1}^L{A}_{ij}^r<1$, we will set α_ij = 1, and set β_ij to a large number to make the MAP algorithm converge.

In this scenario, we do not have any structural information regarding the target layer. If we set A_ij = 0 in equations (11) and (12), this equivalent to assuming zero presences of all the edges in the target layer. To avoid this issue, we take the entries of A_ij as the ratio of α_ij and β_ij, i.e., A_ij = α_ij/β_ij. The entries are assigned as the average presences over the other layers.

III. EXPERIMENTAL VALIDATION

In this section, we evaluate the method introduced in section II. First, the SimHash algorithm is applied on two real networks, in which different percentages of links are removed uniformly at random. Second, the effectiveness of the MAP algorithm for the two real multilayer networks is evaluated, and the MAP algorithm is compared to the MLE algorithm.

The first real network we use to evaluate our proposed MAP algorithm is the FAO (Food and Agriculture Organization) trade network [33]. The FAO multilayer network is composed of 364 layers, and each layer represents a product trading among 214 countries. A link is detected between two nodes in a layer if there is trading of the corresponding product between the two countries. We show three layers of the FAO network in Fig. 1 through the KiNG software [34]. Since the layers in the FAO network are not ordered in any particular way, in the experiments, we only perform the evaluation by assuming that the target layer is one of the the first nine layers of the FAO network. Note that all the 363 remaining layers are always used in our experiments for detecting the similar layers.

FIG. 1.

Three layers of the FAO (a) network and HVR (b) network. Nodes in different layers share the same plane coordinates.

The second real network (HVR network) we use to evaluate the proposed MAP algorithm has nine layers and 307 nodes [35], which represent malaria parasite genes. The nine layers correspond to nine highly variable regions on the genes themselves. An edge is detected if two genes share an exact match of significant length in the highly variable region. We show three layers of the HVR network in Fig. 1.

A. Numerical results on layer comparison

The evaluation of the SimHash algorithm is performed on both the FAO network and the HVR network. The binary digest of each layer is obtained through the SimHash algorithm based on the eigenvetor centrality of the nodes. The similarity of any two layers can be computed through equation (13). In Fig. 2, each layer is respectively set as the target layer, and the similarities between the target layer and all the other layers are shown in interquartile ranges (IQR). For the FAO network, we only show the results of the first nine layers as target layers, and each IQR bar contains 363 similarity values. The similarities obtained through the SimHash algorithm are between 0 and 1. A zero similarity means the two layers have totally opposite eigenvector centrality distribution, while for 0.5 similarity the two layers are independent. A similarity approaching one means the two layers are similar. In the FAO network, the similarities are between 0.5 and 1, which indicates there are no layers with totally opposite eigenvector centrality distribution. In the HVR network, most of the layers are independent, since the similarities are all less than 0.8, except layer 7 and layer 9.

FIG. 2.

Layer comparison of the FAO network (a) and HVR network (b). (a) shows the similarities between the target layer and all the other 363 layers in the FAO network. (b) shows the similarities between the target layer and the other eight layers in the HVR network. Note that each similarity value is averaged over 10 runs.

The network-comparing method must be able to discover structurally similar layers even if there are missing links in the network. To this end, we uniformly at random remove 20%, 40%, 60% and 80% of edges from the target layer to generate incomplete networks and compare the incomplete networks with the target network. In Fig. 3(a), each layer of the FAO network (first nine layers) and HVR network is set as target network respectively, and we randomly remove 20%, 40%, 60% and 80% of edges from the target layer. The similarities between the incomplete networks with removed edges and the target layer are computed. From the top panels, we observe that the similarities are greater than 0.8 even with 80% of edges randomly removed.

FIG. 3.

Eigenvector centrality-based SimHash algorithm. Panel (a) shows the comparison between the first nine layers of the FAO network and their incomplete counterparts, where 20%, 40%, 60% and 80% edges are removed uniformly at random from the target layer. Panel (b) shows the same experiment on the HVR network. Panel (c) compares the reduced networks (the first nine layers of the FAO network) with the five most similar layers. The mean of the similarities is shown in the panel. Similarly, 20%, 40%, 60% and 80% edges are removed uniformly at random from the target layer. Panel (d) shows the same experiment on the HVR network. Note that each similarity value is averaged over 10 runs.

There are always differences between the target layer and similar layers. In the second experiment, we show that missing links in the target layers do not affect the similarity significantly between the target layer and its similar layers. For each of the target layers, we choose five most similar layers from all the other layers, we then randomly remove 20%, 40%, 60% and 80% of edges from the target layer to generate incomplete networks. The incomplete networks are compared to the five similar layers, and we can obtain five similarities for different removal percentages. In the bottom panels, we show the average of the five similarities. The results for the first nine layers of the FAO network are shown in Fig. 3(c). We observe that the similarities decrease slightly even when 80% of edges are randomly removed. For the HVR network, where the layers are heterogeneous, though we randomly remove different percentages of links, the similarities are still maintained at low levels.

B. Validation of layer reconstruction and link prediction

The MAP method we have introduced in this work has the goal of layer reconstruction and missing link estimation. The similarity between the target layer and other layers can be obtained through the SimHash algorithm as introduced above. In this part, we show the effectiveness of the MAP algorithm, and compare the differences between the MAP algorithm and the MLE algorithm performance. Other methods such as entropy-based approaches, are equivalent to the MLE algorithm and are discussed in appendix D.

Here, we use the receiver-operator characteristic (ROC) curve and the area under the curve (AUC) to evaluate the effectiveness of our method. The model is perfect when the AUC is approaching one, and 0.5 means the model guesses the edge weights randomly.

a. The dimension of node vectors.

The dimension of node vectors is a critical factor for the MAP algorithm. In experiments, we find that there is no such number of communities [36–40] that can maximize the estimation accuracy. For both the FAO network and HVR network, we remove 40% of edges from the target layer to generate incomplete networks. The five layers with highest SimHash similarities are used to compute the parameters of the conjugate prior. The target layer is reconstructed through both the MLE algorithm and the MAP algorithm. In Fig. 4, we can observe an increased estimation accuracy with respect to the increment of the number of dimensions, though the gain diminishes for dimensions greater than 40. In the following experiments, we use a dimension of 50 which balances running time and estimation accuracy. Apart from the dimension of node vectors, the estimation accuracy is also affected by the number of similar layers. The influence of the number of similar layers on the estimation accuracy is analyzed in appendix E.

FIG. 4.

AUC vs. the dimension of node vectors. Panel (a) shows the MLE method on the first nine layers of the FAO network, where each layer is set as the target layer respectively. Panel (b) shows the results of MAP method when the five layers with highest similarities are adopted to compute the parameters of the conjugate prior. Panel (c) shows the results of the MLE method on the HVR network. Panel (d) is the results of MAP method on the HVR network. Note that the results are averaged over 10 runs of cross-validations.

b. Comparison between the MLE algorithm and the MAP algorithm.

Both the MLE algorithm and the MAP algorithm have their own benefits and disadvantages. The major difference between the MAP and MLE methods is that the MAP method can incorporate prior information (other similar layers), while the MLE relies on the available information of the target layer solely. The comparison between the MLE algorithm and the MAP algorithm is performed on both the FAO network (first nine layers) and the HVR network. The two algorithms are implemented on incomplete layers, which are generated by randomly removing 20%, 40%, 60%, 80% and 100% of edges from the two networks. In Fig. 5(a) and Fig. 6(a), we can see that the estimation based on the MLE algorithm is significantly affected by the missing links. However, the robustness of the estimation is greatly improved after we adopt structurally similar layers to reconstruct the target layer, as shown in Fig. 5(b). On the contrary, the estimation accuracy deteriorates if we adopt layers with heterogeneous structures to reconstruct the target layer, which is shown in Fig. 6(b).

FIG. 5.

Comparison of the MLE and MAP methods on the FAO network. Panel (a) shows results of the MLE method. Panel (b) shows the results of the MAP method. Results are averaged over 10 runs of cross-validations.

FIG. 6.

Comparison of the MLE and MAP methods on the HVR network. Panel (a) shows results of the MLE method. Panel (b) shows the results of the MAP method. Results are averaged over 10 runs of cross-validations.

Intuitively, the MLE method reconstructs the target layer through the known information of the target layer itself. As a result, the estimation accuracy is related to the available information, i.e., the percentage of known links. In the MAP algorithm, the estimation is not only affected by the known information of the target layer but also the similar layers. In real applications, if the percent of missing links is less than 20%, it is recommended to use the MLE algorithm, since it provides more accurate results than the MAP algorithm. Conversely, the MAP algorithm is the better choice if researchers are unaware of the percentage of missing links.

IV. CONCLUSION AND FUTURE WORKS

In this paper, we present a novel MAP estimation-based algorithm for target layer reconstruction in multilayer networks. In multilayer networks, some layers are structurally similar; thus, we can take advantage of the similar layers to reconstruct the target layer. In section II, we first derive the maximum a posteriori estimation for target layer reconstruction in multilayer network. Second, the eigenvector centrality-based SimHash algorithm is introduced to detect structurally similar layers. The SimHash algorithm compares network features, thus it is not affected by the missing links. Third, we introduce two scenarios to obtain the parameters of the Gamma conjugate prior. In the first case, where the target layer is partially known, the SimHash algorithm is adopted to detect structurally similar layers. In the second case, where the target layer is not known at all, functionally similar layers is used to compute the parameters of the conjugate prior. In section III, we first show that the eigenvector centrality-based SimHash algorithm is able to return consistent similarity levels for different percentages of missing links. Then, we show that the estimation accuracy can be improved by increasing the number of dimensions of node vectors, and the gain is diminishing for dimensions greater than 40 for the two networks. We find that with a great number of similar layers, we can obtain more consistent estimation results. Finally, the MLE method and MAP method are compared on two real networks. The experimental results suggest that if there are less than 20% of missing links, the MLE method has better performance. However, if the percentage of missing links is 40% or more, the MAP method returns results that are more consistent.

However, there are still some limitations to the MAP method we present. The first limitation is that the missing links in the target layer are required to be removed uniformly at random. The similarities are obtained through network feature comparison, which means the structure of the target layer needs to be maintained. Targeted removal of links will change the structure of the network. The second limitation concerns the unknown relation between the vector dimension and network size. In our numerical experiments, we test multiple dimensions and adopt a dimension of 50, which balances running time and estimation accuracy.

Recent results in [41–44] present some methods to estimate the eigenvector centralities based on nodal data without requiring the network structure. Recall that SimHash algorithm is based on the eigenvector centrality obtained from the target layer. Therefore, estimating eigenvector centrality without constructing network is a good alternative for future work.

The MAP algorithm we present to reconstruct a target layer in a multilayer network shows promising results when we can identify the similarities between the target layer and other layers. The experimental results show that the estimations of our MAP method are less likely to be affected by missing links, which is not known in real applications. Therefore, our MAP method can be used to direct experiments, especially when there is no information about the target layer.

Appendix A: tensor factorization

In [7, 13, 14], nodes are factorized by membership vectors. The dimension of membership vectors is interpreted as the number of overlapping communities. In addition, each edge is computed through the tensor product of the membership vectors. The number of communities is obtained by maximizing the likelihood. Vectorization is also used in some natural language processing algorithms [45–49] in which words are embedded as vectors to preserve their relations to contexts. However, the dimension of word vectors does not have any semantic meaning, rather the choice of the vector dimension is first affected by the data set size. According to [46, 49], larger dimensions can improve model accuracy, but the gain diminishes for vectors larger than 200 dimensions. The choice of dimension is also related to the available resources, and it is better to reduce the dimension as long as the choice does not affect the estimation accuracy substantially. In fact, the tensor factorization can be simplified to dot product, we prove this point as follows.

We assume the number of edges between any two nodes is the tensor product of two node vectors and the control matrix, i.e., E_ij = Σs_ikt_jlw_kl. w_kl (k is not equal to l) is the parameter that controls the edges from any source node i (in community k) to any target node j (in community l). Then, we have

$$\begin{gathered} E_{ij}=\sum s_{ik}{t}_{jl}{w}_{kl} \\ =s_{ik}{t}_{jk}{w}_{kk}+s_{ik}{t}_{jl}{w}_{kl}+s_{il}{t}_{jl}{w}_{ll} \\ +\left(\sum s_{im}{t}_{jn}{w}_{mn}-s_{ik}{t}_{jk}{w}_{kk}-s_{ik}{t}_{jl}{w}_{kl}-s_{il}{t}_{jl}{w}_{ll}\right). \end{gathered}$$ (A1)

In equation A1, we denote the sum of the first three terms as $E_{ij}^{kl}$. Then, we can incorporate w_kk into s_ik, and denote it as $s_{ik}^{\prime }=s_{ik}{w}_{kk}$. Similarly, $s_{il}^{\prime }=s_{il}{w}_{ll}$. Therefore, we have

$$\begin{gathered} E_{ij}^{kl}=s_{ik}^{\prime }{t}_{jk}+s_{ik}^{\prime }{t}_{jl}\frac{w_{kl}}{w_{kk}}+s_{il}^{\prime }{t}_{jl} \\ =s_{ik}^{\prime }{t}_{jk}+\left(s_{ik}^{\prime }\frac{w_{kl}}{w_{kk}}+s_{il}^{\prime }\right)t_{jl}. \end{gathered}$$ (A2)

Since nodes between different communities are loosely connected, we have w_kl < w_kk, and we assume ϵ = w_kl/w_kk. Then, equation A2 can be written as

$$\begin{gathered} E_{ij}^{kl}=s_{ik}^{\prime }{t}_{jk}+\left(ϵs_{ik}^{\prime }+s_{il}^{\prime }\right)t_{jl} \\ =s_{ik}^{\prime }{t}_{jk}+s_{il}^{″}{t}_{jl}, \end{gathered}$$ (A3)

where $s_{il}^{″}=ϵs_{ik}^{\prime }+s_{il}^{\prime }$. For every inter-community edge originating from node i in community k, we can incorporate it into community l by incrementing ϵs_ik to $s_{il}^{\prime }$. Edges can be factorized through the dot product of node vectors. Therefore, the tensor factorization is equivalent to the dot product.

Appendix B: Estimation based on the Pearson correlation

With the SimHash algorithm, the weighted digest of each layer can be obtained. The similarity between any two layers can be measured by computing the Pearson correlation of the weighted digests. In Fig. B.1, we show the estimation results based on the Pearson correlation coefficient. We observe that the estimation is as consistent as the results based on the Hamming distance. Therefore, the correlation can be an alternative for the Hamming distance.

FIG. B.1.

The estimation results based on the Pearson correlation of the weighted digest.

Appendix C: The choice of the number of bits

The choice of ϕ affects the resolution and the consistency of the similarity. If the number of tokens is large, it is recommended to employ large ϕ. In our work, the sizes of the two networks are not too large, so we adopt ϕ = 512 bits. In Fig. C.1, we validate this choice on the FAO network. We set each of the first nine layers of the FAO network as the target network (original network), and randomly remove 20%, 40%, 60% and 80% of edges from the target network to generate incomplete networks. We then compare the incomplete networks with the original network. In the experiments, we adopt ϕ = 16, 64, 256, and 512 digits, respectively. We can observe that the similarities obtained with 256 digits are close to that obtained with 512 digits, while the similarities obtained with 16 and 64 digits vary remarkably.

FIG. C.1.

Comparison of varying the number of digits (based on the Hamming distance). The four panels show the results based on 20% (a), 40% (b), 60% (c), and 80% (d) of edges randomly removed.

Appendix D: Comparison between the MLE algorithm and entropy-based approaches

In section III B, we compared the MAP algorithm with the MLE algorithm. In fact, the entropy-based approaches are equivalent to the MLE algorithm. In the following, we prove that the MLE algorithm is equivalent to the entropy-based approaches.

Given a network G, we assume the number of nodes is fixed, and the edge weights are variables. Thus, we can define the entropy of a network in terms of edge weights as

$$\begin{gathered} H(G)=-\sum _G\sum _u{p}^U(u)\text{log}{p}^U(u) \\ =\sum _G\sum _u{p}^U(u)\text{log}\frac{1}{p^U(u)}, \end{gathered}$$ (D1)

where U is any edge in the network G, u is the weight of edge U. p^U(u) is the precise probability of edge U with weight u. We assume for any edge U, it can take n_U values. If the n_U values follow uniform distribution, then we have $p^U(u)=\frac{1}{n_U}$. In this case, the network has the largest entropy, which means the network is totally uncertain. On the other hand, for any edge U, if there is a value u_s with p^U(u_s) = 1, the entropy of the network will be zero, which indicates all the edge weights in the network are known.

In our problem, the goal is to find a model to reconstruct the target layer, thus, we need a set of parameters to describe the model. If the model is close to the true network, the entropy of the model is minimized. Hence, the problem is transformed to finding the parameters of a model with minimum entropy, i.e., reducing uncertainty.

Equation D1 can be written in expectation form

$$H(p)=\sum _G{E}_{U~p^U(u)}\left(\text{log}\frac{1}{p^U(u)}\right).$$ (D2)

Consider the Kullback-Leibler divergence (KL divergence). We assume θ is the parameter set that can describe the model, the probability of edge U with weight u is q^U(u; θ). Thus, the relative entropy is

$$\begin{gathered} D_{kl}(p\Vert q)=\sum _G\sum _u{p}^U(u)\text{log}\frac{p^U(u)}{q^U(u;\theta )} \\ =\sum _G{E}_{U~p^U(u)}\left(\text{log}\frac{p^U(u)}{q^U(u;\theta )}\right) \\ =\sum _G{E}_{U~p^U(u)}\left(\text{log}\frac{1}{q^U(u;\theta )}\right) \\ -\sum _G{E}_{U~p^U(u)}\left(\text{log}\frac{1}{p^U(u)}\right). \end{gathered}$$ (D3)

The second term in the right-hand side is equation D2 and the first term in the right-hand side is the cross entropy, which is

$$H(p,q)=\sum _G{E}_{U~p^U(u)}\text{log}\frac{1}{q^U(u;\theta )}.$$ (D4)

Recall that our problem is to reconstruct the target layer and equation D2 is the entropy of the target layer. Equation D2 is determined by the data (network) solely. Thus, the problem can be simplified to minimizing the cross entropy, i.e., minimizing equation D4.

Then, we consider the MLE method. The MLE algorithm is to find the parameter set θ that most likely fit a given set of data (network) D. We skip some intermediate steps and use the logarithm form directly as following

$$\begin{gathered} \theta ={\mathit{\text{argmax}}}_{\theta }P(G\mid \theta ) \\ ={\mathit{\text{argmax}}}_{\theta }\sum _G\sum _u\text{log}{p}^U(u;\theta ). \end{gathered}$$ (D5)

p^U(u; θ) is the model with parameter set θ to describe the true data D. We assume that edge U can take n_U values and the n_U values follow the uniform distribution, then we have $p_{MLE}^U(u)=\frac{1}{n_U}$. Introducing $p_{MLE}^U$ to equation D5 does not change the results, we have

$$\begin{gathered} \theta ={\mathit{\text{argmax}}}_{\theta }\sum _G\sum _u\frac{1}{n_U}\text{log}{p}^U(u;\theta ) \\ ={\mathit{\text{argmax}}}_{\theta }\sum _G{E}_{u~p_{MLE}^U(u)}\left(\text{log}{p}^U(u;\theta )\right). \end{gathered}$$ (D6)

Our goal is still to find a set of parameters that can reconstruct the layer. Then, we can generalize this problem by replacing $p_{MLE}^U(u)$ with p^U(u), and replacing p^U(u; θ) with q^U(u; θ). The replacements can be regarded as taking real data (network) into equation D7. We have

$$\begin{gathered} \theta ={\mathit{\text{argmax}}}_{\theta }\sum _G{E}_{u~p^U(u)}\text{log}{q}^U(u;\theta ) \\ ={\mathit{\text{argmax}}}_{\theta }\sum _G-E_{u~p^U(u)}\text{log}{q}^U(u;\theta ) \\ ={\mathit{\text{argmax}}}_{\theta }\sum _G{E}_{u~p^U(u)}\text{log}\frac{1}{q^U(u;\theta )}. \end{gathered}$$ (D7)

Thus, the MLE algorithm is equivalent to minimizing the cross entropy.

Appendix E: The number of similar layers on the estimation accuracy

If more similar layers are employed to compute the parameters of the conjugate prior, the influence of similar layers on the reconstruction will be promoted. Consequently, the influence of the known part of the target layer will be down weighted. On the contrary, if we trust the known part of the target layer, we can employ less similar layers to compute the parameters of the conjugate prior. The experimental results are shown in Fig. E.1, top 3, 5, 10, and 20 similar layers are adopted to compute the parameters of the conjugate prior. We observe that the estimation based on 10 and 20 similar layers are robust with respect to the missing links, while the estimation based on three and five similar layers are influenced by the missing links significantly.

FIG. E.1.

The number of similar layers on the estimation results. 20% (a), 40% (b), 60% (c), and 80% (d) of edges are randomly removed from the target layer. Different numbers of similar layers are used to compute the parameters of the conjugate prior. The experiments are conducted on the first nine layers of the FAO network. Each AUC is averaged over 10 runs of cross-validations.