Progresses and challenges in link prediction

Summary

Link prediction is a paradigmatic problem in network science, which aims at estimating the existence likelihoods of nonobserved links, based on known topology. After a brief introduction of the standard problem and evaluation metrics of link prediction, this review will summarize representative progresses about local similarity indices, link predictability, network embedding, matrix completion, ensemble learning, and some others, mainly extracted from related publications in the last decade. Finally, this review will outline some long-standing challenges for future studies.

Graphical abstract

Computer science; Network; Network topology

Introduction

Network is a natural and powerful tool to characterize a huge number of social, biological, and information systems that consist of interacting elements, and network science is currently one of the most active interdisciplinary research domains (Barabasi, 2016; Newman, 2018). Link prediction is a paradigmatic problem in network science that attempts to uncover missing links or predict future links (Lü and Zhou, 2011), which has already found many theoretical and practical applications, such as reconstruction of networks (Squartini et al., 2018; Peixoto, 2018), evaluation of evolving models (Wang et al., 2012; Zhang et al., 2015), inference of biological interactions (Csermely et al., 2013; Ding et al., 2014), online recommendation of friends and products (Aiello et al., 2012b; Lü et al., 2012), and so on.

Thanks to a few pioneering works (Liben-Nowell and Kleinberg, 2007; Clauset et al., 2008; Zhou et al., 2009; Guimerà and Sales-Pardo, 2009), link prediction has been one of the most active research domains in network science. Early contributions were already summarized by a well-known survey article (Lü and Zhou, 2011), and this review will first define the standard problem and discuss some well-known evaluation metrics and then introduce most representative achievements in the last decade (mostly published after (Lü and Zhou, 2011)), including local similarity indices, link predictability, network embedding, matrix completion, ensemble learning, and some others. Lastly, this review will show limitations of existing studies as well as open challenges for future studies.

Evaluation

Consider a simple network $G(V,E)$, where V and E are sets of nodes and links, the directionalities and weights of links are ignored, and multiple links and self-connections are not allowed. We assume that there are some missing links or future links in the set of nonobserved links $U\backslash E$, where U is the universal set containing all $\left|V\right|\left(\left|V\right|-1\right)/2$ potential links. The task of link prediction is to find out those missing or future links. To test the algorithm’s accuracy, the observed link, E, is divided into two parts: the training set $E^T$ is treated as known information, while the probe set $E^P$ is used for algorithm evaluation, and no information in $E^P$ is allowed to be used for prediction. The majority of known studies applied “random division”, namely $E^P$ is randomly drawn from E. In the case of predicting future links, “temporal division” is usually adopted where $E^P$ contains most recently appeared links (Lichtenwalter et al., 2010). In some real networks, missing links have different topological features from observed links. For example, missing links are more likely to be associated with low-degree nodes since interactions between hubs are easy to be known. In such situations, we may apply “biased division” to ensure that $E^P$ is consisted of links with similar topological features to missing links (Zhu et al., 2012).

Performance evaluation metrics can be roughly divided into two categories: threshold-dependent metrics (e.g., fixed threshold accuracy) and threshold-independent metric (e.g., area under threshold curve). Precision and recall are the two most widely used metrics in the former category. Precision is defined as the ratio of relevant items selected to the number of items selected. That is to say, if we take the top-L links as the predicted ones, among which $L_r$ links are correctly predicted; then, the Precision equals $L_r/L$. Recall is defined as the ratio of relevant items selected to the total number of relevant items, say $L_r/\left|E^P\right|$. An obvious drawback of threshold-dependent metrics is that we generally do not have a reasonable way to determine the threshold, like the number of predicted links L or the threshold score for the existence of links. A widely adopted way is setting $L=\left|E^P\right|$, at which precision = recall (Lü and Zhou, 2011; Liben-Nowell and Kleinberg, 2007). Although $\left|E^P\right|$ is generally unknown, an experiential and reasonable setting is $\left|E^P\right|=0.1\left|E\right|$ because 10% of links in the probe set are usually enough for us to get statistical solid results while the removal of 10% of links will probably not destroy the structural features of the target network (Lü et al., 2015).

Some studies argued that a single value might not well reflect the performance of a predictor (Lichtenwalter et al., 2010; Yang et al., 2015). Therefore, robust evaluation based on threshold-dependent metrics should cover a range of thresholds (e.g., by varying L), which is actually close to the consideration of threshold curves. The precision–recall (PR) curve (Yang et al., 2015) and receiver operating characteristic (ROC) curve (Hanely and McNeil, 1982) are frequently considered in the literature. The former shows precision with respect to recall at all thresholds and the latter represents performance trade-off between true positives and false positives at different thresholds. We say algorithm X is strictly better than algorithm Y only if X’s threshold curve completely dominates Y’s curve (mathematically speaking, a curve A dominates another curve B in a space is B is always equal or below curve A (Provost et al., 1998)). Davis and Goadrich (2006) have proved that a curve dominates in ROC space if and only if it dominates in PR space.

As the condition of domination is too rigid, we usually use areas under threshold curves as evaluation metrics. The area under the ROC curve (AUC) can be interpreted as the probability that a randomly chosen link in $E^P$ is assigned a higher existence likelihood than a randomly chosen link in $U\backslash E$. If all likelihoods are generated from an independent and identical distribution, the AUC value should be about 0.5. Therefore, the degree to which the value exceeds 0.5 indicates how better the algorithm performs than pure chance. Thus far, AUC is the most frequently used metric in link prediction, probably because it is highly interpretable, easy to be interpolated, and of good visualization. Meanwhile, readers should be aware of some remarkable disadvantages of AUC, for example, AUC is inadequate to evaluate the early retrieval performance which is critical in real applications especially for many biological scenarios (Saito and Rehmsmeier, 2015), and AUC will give misleadingly overhigh score to algorithms that can successfully rank many negatives in the bottom while this ability is less significant in imbalanced learning (Yang et al., 2015; Lichtenwalter and Chawla, 2012). A typical viewpoint in the early studies is that AUC is suitable for imbalanced learning because AUC is not sensitive to the ratio of positives to negatives and thus can reflect an algorithm’s ability that is independent to the data distribution (Fawcett, 2006). However, recent perspective in machine learning and big data is that talking about the performance or ability of a classification algorithm without specified datasets is meaningless so that this previous advantage gets increasing criticisms (Yang et al., 2015; Lichtenwalter and Chawla, 2012). Hand argued that AUC is fundamentally incoherent because AUC uses different misclassification cost distributions for different classifiers (Hand 2009). Hand’s criticism is deep, but AUC indeed measures relative ranks instead of absolute loss and is irrelevant to misclassification cost so that to dissect AUC in the narration involving misclassification cost may be unfair. To overcome the above disadvantages, scientists have proposed a number of alternatives of AUC, such as H measure (Hand 2009), concentrated ROC (Swamidass et al., 2010), and normalized discounted cumulative gain (Wang et al., 2013). At the same time, the area under the precision-recall curve (AUPR) becomes increasingly popular, especially for biological studies. Though the AUPR score is less interpretable than AUC, each point in the PR curve has an explicit meaning, and the absolute accuracy metrics (e.g., precision and recall) are usually closer to practical requirements than relative ranks.

In summary, empirical comparisons and systematic analyses about evaluation metrics for link prediction are important tasks in this stage because many publications use AUC as the sole metric, while an ongoing empirical study (by Y.-L. Lee and T. Zhou, unpublished) shows that in about 1/3 pairwise comparisons, AUC and AUPR give different ranks of algorithms, and a recent large-scale experimental study also shows inconsistent results by AUC and AUPR (Muscoloni and Cannistraci, 2021). Before a comprehensive and explicit picture obtained, my suggestion is that we have to at least simultaneously report both AUC and AUPR, and only if an algorithm can obviously beat another one in both metrics for a network, we can say the former performs better in this case.

Local similarity indices

A similarity-based algorithm will assign a similarity score to each nonobserved link, and the one with a higher score is of a larger likelihood to be a missing link. Liben-Nowell and Kleinberg (2007) indicated that a very simple index named “common neighbor” (CN), say

$$S_{xy}^{CN}=\left|{\Gamma }_x\cap {\Gamma }_y\right|,$$ (Equation 1)

with ${\Gamma }_x$ and ${\Gamma }_y$ being sets of neighbors of nodes x and y, performing very well in link prediction for social networks. Zhou et al. (2009) proposed the “resource allocation” (RA) index via weakening the weights of large-degree common neighbors, namely

$$S_{xy}^{RA}=\sum _{z\in {\Gamma }_x\cap {\Gamma }_y}\frac{1}{k_z},$$ (Equation 2)

where $k_z$ is the degree of node z. The simplicity, elegance, and good performance of CN, RA, and some other alternatives (Lü and Zhou, 2011) lead to increasing attention on local similarity indices.

In the recent decade, probably, the most impressive achievement on local similarity indices is the proposal of the local community paradigm (Cannistraci et al., 2013a), which suggests that two nodes are more likely to link together if their common neighbors are densely connected. Accordingly, Cannistraci et al. (2013a) proposed the CAR index where the CN index is multiplied by the number of observed links between common neighbors, as follows:

$$S_{xy}^{CAR}=S_{xy}^{CN}\cdot \sum _{z\in {\Gamma }_x\cap {\Gamma }_y}\frac{\left|{\gamma }_z\right|}{2},$$ (Equation 3)

where ${\gamma }_z$ is the subset of z’s neighbors that are also common neighbors of x and y. Analogously, RA index can be improved by accounting for the local community paradigm as follows:

$$S_{xy}^{CRA}=\sum _{z\in {\Gamma }_x\cap {\Gamma }_y}\frac{\left|{\gamma }_z\right|}{k_z}.$$ (Equation 4)

By integrating the idea of Hebbian learning rule, the above index is further extended and renamed as Cannistraci-Hebb (CH) index (Muscoloni et al., 2018):

$$S_{xy}^{CH}=\sum _{z\in {\Gamma }_x\cap {\Gamma }_y}\frac{1+k_z^{\left(i\right)}}{1+k_z^{\left(e\right)}},$$ (Equation 5)

where $k_z^{\left(i\right)}$ is the internal degree of z, say the number of z’s neighbors that are also in ${\Gamma }_x\cap {\Gamma }_y$, and $k_z^{\left(e\right)}$ is the external degree of z, say the number of z’s neighbors that are not in ${\Gamma }_x\cap {\Gamma }_y\cap \{x,y\}$. The core idea of CH index is to consider the negative impacts of external local community links (see Muscoloni et al. (2020) for more CH indices according to the core idea). Extensive empirical analyses (Muscoloni et al., 2020; Zhou et al., 2021) indicated that the introduction of local community paradigm and Hebbian learning rule could considerably improve the performance of routine local similarity indices.

In most known studies, the presence of many 2-hop paths between a pair of nodes is considered to be the strongest evidence indicating the existence of a corresponding missing link or future link. Although in local path index (Lü et al., 2009) and Katz index (Katz, 1953) longer paths are taken into account, they are considered to be less significant than 2-hop paths. Surprisingly, some recent works have argued that 3-hop-based similarity indices perform better than 2-hop-based indices. Pech et al. (2019) assumed that the existence likelihood of a link is a linear sum of all its neighbors’ contributions. After some algebra, Pech et al. (2019) obtained a global similarity index called linear optimization (LO) index, as follows:

$$S^{LO}=\alpha A{(\alpha A^{T}A+I)}^{-1}{A}^{T}A=\alpha A^3-{\alpha }^2{A}^5+{\alpha }^3{A}^7-{\alpha }^4{A}^9+\cdots ,$$ (Equation 6)

where A and I are adjacency matrix and identity matrix, respectively. Clearly, the number of 3-hop paths $A^3$ can be interpreted as a degenerated index of LO. Indeed, Daminelli et al. (2015) have already applied 3-hop-based indices to predict missing links in bipartite networks, while almost no one at that time has tried 3-hop-based indices on unipartite networks. Kovács et al. (2019) noted the bipartite nature of the protein-protein interaction networks (not fully bipartite, but of high bipartivity (Holme et al., 2003)) and independently proposed a degree-normalized index (called L3 index) based on 3-hop paths as follows:

$$S_{xy}^{L3}=\sum _{u,v}\frac{a_{xu}{a}_{uv}{a}_{vy}}{\sqrt{k_u{k}_v}}.$$ Equation (7)

They showed its advantage compared with 2-hop-based indices in predicting protein-protein interactions. Muscoloni et al. (2018) further proposed a theory that generalized 2-hop-based indices to n-hop-based indices with n > 2 and demonstrated the superiority of 3-hop-based indices over 2-hop-based indices on protein-protein interaction networks, world trade networks, and food webs. For example, in their framework (Muscoloni et al., 2018), the n-hop-based RA index reads as follows:

$$S_{xy}^{RA\left(n\right)}=\sum _{z_1,z_2,\cdots ,z_{n-1}\in \mathbb{L}\left(n\right)}\frac{1}{\sqrt[n-1]{k_{z_1}{k}_{z_2}\cdots k_{z_{n-1}}}},$$ (Equation 8)

where $\mathbb{L}\left(n\right)$ is the set of all n-hop simple paths connecting x and y, and $z_1,z_2,\cdots ,z_{n-1}$ are the intermediate nodes on the considered path. Accordingly, the L3 index is exactly the same to the 3-hop-based RA index.

We have implemented extensive experiments on 137 real networks (Zhou et al., 2021), suggesting that (i) 3-hop-based indices outperform 2-hop-based indices subject to AUC, while 3-hop-based and 2-hop-based indices are competitive on precision; (ii) CH indices perform the best among all considered candidates; and (iii) 3-hop-based indices are more suitable for disassortative networks with lower densities and lower average clustering coefficient. Furthermore, we have showed that a hybrid of 2-hop-based and 3-hop-based indices via collaborative filtering techniques can result in overall better performance (Lee and Zhou, 2021).

Link predictability

Quantifying link predictability of a network allows us to evaluate link prediction algorithms for this network and to see whether there is still a large space to improve the current prediction accuracy. Lü et al. (2015) raised a hypothesis that missing links are difficult to predict if their addition causes huge structural changes, and thus, a network is highly predictable if the removal or addition of a set of randomly selected links does not significantly change structural features of this network. Denote A the adjacency matrix of a simple network $G(V,E)$ and $\mathrm{\Delta }A$ the adjacency matrix corresponding to a set of randomly selected links $\mathrm{\Delta }E$ from E. After the removal of $\mathrm{\Delta }E$, the remaining network $G^R$ is also a simple network so that the corresponding adjacency matrix, $A^R=A-\mathrm{\Delta }A$, can be diagonalized as follows:

$$A^R=\sum _{k=1}^N{\lambda }_k{x}_k{x}_k^T,$$ (Equation 9)

where $N=\left|V\right|$ and ${\lambda }_k$ and $x_k$ are the kth eigenvalue and corresponding orthogonal and normalized eigenvector of $A^R$, respectively. Considering $\mathrm{\Delta }E$ as a perturbation to $A^R$, which results in an updated eigenvalue ${\lambda }_k+\mathrm{\Delta }{\lambda }_k$ and a corresponding eigenvector $x_k+\mathrm{\Delta }{x}_k$, then we have the following equation:

$$\left(A^R+\mathrm{\Delta }\text{A}\right)\left(x_k+\mathrm{\Delta }{x}_k\right)=\left({\lambda }_k+\mathrm{\Delta }{\lambda }_k\right)\left(x_k+\mathrm{\Delta }{x}_k\right).$$ (Equation 10)

Similar to the process to get the expectation value of the first-order perturbation Hamiltonian, we neglect the second-order small terms and the changes of eigenvectors and then obtain the following equation:

$$\mathrm{\Delta }{\lambda }_k\approx \frac{x_k^T\mathrm{\Delta }A{x}_k}{x_k^T{x}_k},$$ (Equation 11)

as well as the perturbed matrix:

$$\tilde{A}=\sum _{k=1}^N\left({\lambda }_k+\mathrm{\Delta }{\lambda }_k\right)x_k{x}_k^T,$$ (Equation 12)

which can be considered as the linear approximation of A if the expansion is only based on $A^R$. If the perturbation does not significantly change the structural features, the eigenvectors of $A^R$ and those of A should be almost the same, and thus, $\tilde{A}$ should be very close to A according to Equation (12). We rank all links in $U\backslash E^R$ in a descending order according to their values in $\tilde{A}$ and select the top-L links to form the set $E^L$, where $L=\left|\mathrm{\Delta }E\right|$. Links in $E^R$ and $E^L$ constitute the perturbed network, and if this network is close to G (because $\tilde{A}$ is close to A), $E^L$ should be close to $\mathrm{\Delta }E$. Therefore, Lü et al. (2015) finally proposed an index called structural consistency to measure the inherent difficulty in link prediction as follows:

$${\sigma }_c=\frac{|E^L\cap \mathrm{\Delta }E|}{\left|\mathrm{\Delta }E\right|}.$$ Equation (13)

The above perturbation method can also be applied to predict missing links, and the resulted “structural perturbation method” (SPM) is still one of the most accurate methods till far (Muscoloni and Cannistraci, 2021).

Koutra et al. (2015) found that the major part of a seemingly complicated real network can be represented by a few elemental substructures like cliques, stars, chains, bipartite cores, and so on. Inspired by this study, Xian et al. (2020) claimed that a network is more regular and thus more predictable if it can be well represented by a small number of subnetworks. To reduce the tremendous complexity caused by countless subnetworks, they further set a strong restriction that candidate subnetworks are ego networks of all nodes, where ego network (also called egocentric network) is a subnetwork induced by a central node (known as the ego) and all other nodes directly connected to the ego (called alters), see (Wang et al., 2016a) for example. Obviously, the ego network of node i can be represented by the ith row or ith column of the adjacency matrix A, and if a network can be perfectly represented by all ego networks, there exists a matrix $Z\in {\mathbb{R}}^{N\times N}$ such that A = AZ. Intuitively, if a network G is very regular, the corresponding representation Z should have three properties: (i) G can be well represented by its ego networks so that AZ is close to A; (ii) G can be well represented by a small number of ego networks so that Z is of low rank since the redundant ego networks correspond to zero rows in Z; and (iii) each ego network of G can be represented by a very few other ego networks so that Z is sparse. Accordingly, the best representation matrix $Z^{\ast }$ can be obtained by solving the following optimization problem:

$$\underset{Z}{\min }\text{rank}\left(Z\right)+\alpha \Vert Z\Vert +\beta \Vert A-AZ\Vert ,$$ (Equation 14)

where $\alpha$ and $\beta$ are tradeoff parameters. Based on $Z^{\ast }$, Xian et al. (2020) proposed an ad hoc index named structural regularity index, as follows:

$${\sigma }_r=\frac{1}{\sqrt{\frac{n-r}{n}}\sqrt{\frac{\tau }{nr}}},$$ (Equation 15)

where r is the rank of $Z^{\ast }$, $\tau$ is the number of zero entries in $Z^{\ast }$, $\frac{n-r}{n}$ denotes the proportion of identical ego networks, and $\frac{\tau }{nr}$ characterizes the density of zero entries of the reduced echelon form of $Z^{\ast }$. Clearly, a lower r and a larger $\tau$ will result in a smaller ${\sigma }_r$, corresponding to a more predictable network.

Xian et al. (2020) suggested that the structural regularity corresponds to redundant information in the adjacency matrix, which can be characterized by a low-rank and sparse representation matrix. Sun et al. (2020) proposed a more direct method to measure such redundancy. Their train of thought is that a more predictable network contains more structural redundancy and thus can be compressed by a shorter binary string. As the shortest possible compression length can be calculated by a lossless compression algorithm, they used the obtained normalized shortest compression length of a network to quantify its structure predictability.

To validate their methods, Lü et al. (2015) and Xian et al. (2020) tested on many real networks about whether ${\sigma }_c$ and ${\sigma }_r$ are strongly correlated with prediction accuracies of a few well-known algorithms. This is rough because any algorithm cannot stand for the theoretical best algorithm. Sun et al. (2020) adopted an improved method that uses the best performance among a number of known algorithms for each tested network to approximate the performance of the theoretically best predicting algorithm. Garcia-Perez et al. (2020) analyzed the ensemble of simple networks, where each can be constructed by generating a link between any node pair i and j with a known linking probability $p_{ij}$. For such theoretical benchmark, the best possible algorithm is to rank unobserved links with largest linking probabilities in the top positions and the theoretical limitation of precision can be easily obtained. They showed that if the size of $\mathrm{\Delta }A$ is too small in compared with A, the evaluation of predictability by ${\sigma }_c$ is less accurate.

Structural consistence, structural regularity, and compression length are all ad hoc methods. They can be used to probe the intrinsic difficulty in link prediction but cannot mathematically formulate the limitation of prediction. Mathematically speaking, there could be a God algorithm that correctly predicts all missing links, except for those indistinguishable from nonexistent links. A link (i, j) is indistinguishable from another link (u, v) if and only if there is a certain automorphism f such that f(i) = u and f(j) = v, or f(i) = v and f(j) = u. This extremely rigid definition from automorphism-based symmetry makes virtually all real networks have predictability very close to 1, which is indeed meaningless in practice. Using synthetic networks with known prediction limitation is a potentially promising way to evaluate predictors as well as indices for predictability (Garcia-Perez et al., 2020; Muscoloni and Cannistraci, 2018b), but the results may be irrelevant to real-world networks.

All above studies target static networks, while a considerable fraction of real networks are time varying (named as temporal networks) (Holme and Saramäki, 2012). Temporal networks are usually more predictable since one can utilize both topological and temporal patterns. Ignoring topological correlations, the randomness, and thus predictability of a time series can be quantified by the entropy rate (Xu et al., 2019). Tang et al. (2020) listed weights of all possible links as an expanded vector with dimension $N^2$ (self-connections are allowed, directionalities are considered), and thus, the evolution of a temporal network can be fully described by a matrix $M\in {\mathbb{R}}^{N^2\times T}$, where T is the number of snapshots under consideration. After $M$, the evolution of a temporal network can be treated as a stochastic vector process, and how to measure the predictability of temporal networks is transformed to a solved problem based on the generalized Lempel-Ziv algorithm (Kontoyiannis et al., 1998). An obvious defect is that the vector dimension is too big, resulting in huge computational complexity. Tang et al. (2020) thus replaced $M$ by a much smaller matrix where only links occurring $\ge 10\text{\%}$ of snapshots are taken into consideration. An intrinsic weakness of Lempel-Ziv algorithm is that it tends to overestimate the predictability, and thus, in many situations the estimated values are very close to 1 (Xu et al., 2019; Song et al., 2010). Tang et al. (2020) proposed a clever method that compares the predictability of the target network with the corresponding null network, and thus, the normalized predictability is able to characterize the topological-temporal regularity in addition to the least predictable one.

Network embedding

A network embedding algorithm will produce a function $g:V\to {\mathbb{R}}^d$ with $d\ll N$ so that every node is represented by a low-dimensional vector (Cui et al., 2018). Then, the existence likelihood of a nonobserved link (u, v) can be estimated by the inner product, the cosine similarity, the Euclidean distance, or the geometrical shortest path of the two learned vectors $g\left(u\right)$ and $g\left(v\right)$ (Cui et al., 2018; Cannistraci et al., 2013b). Early methods cannot handle large-scale networks because they usually rely on solving the leading eigenvectors (Tenenbaum et al., 2000; Roweis and Saul, 2000).

Mikolov et al. (2013a, 2013b) proposed a language embedding algorithm named SkipGram that represents every word in a given vocabulary by a low-dimensional vector. Such representation can be obtained by maximizing the co-occurrence probability among words appearing within a window t in a sentence, via some stochastic gradient descent methods. Based on SkipGram, Perozzi et al. (2014) proposed the so-called DeepWalk algorithm, where nodes and truncated random walks are treated as words and sentences. Grover and Leskovec (2016) proposed the node2vec algorithm that learns the low-dimensional representation by maximizing the likelihood of preserving neighborhoods of nodes. Grover and Leskovec argued that the choice of neighborhoods plays a critical role in determining the quality of the representation. Therefore, instead of simple definitions of the neighborhood of an arbitrary node u, such as nodes with distance no more than a threshold to u (like the breadth-first search) and nodes sampled from a random walk starting from u (like the depth-first search), they utilized a flexible neighborhood sampling strategy by biased random walks, which smoothly interpolates between breadth-first search and depth-first search. Considering a random walk that just traversed link (z, v) and now resides at node v, the transition probability from v to any v’s immediate neighbor x is ${\pi }_{vx}/{\sum }_{y\in {\text{Γ}}_v}{\pi }_{vy}$. The node2vec algorithm sets the unnormalized probability as follows:

$${\pi }_{vx}={\alpha }_{pq}\left(z,x\right)\cdot w_{vx},$$ (Equation 16)

where $w_{vx}$ is the weight of link (v, x) and

$${\alpha }_{pq}\left(z,x\right)=\{\begin{array}{cc}1/p,& d_{zx}=0\\ 1,& d_{zx}=1\\ 1/q,& d_{zx}=2\end{array}.$$ (Equation 17)

Obviously, the sampling strategy in DeepWalk is a special case of node2vec with p = 1 and q = 1. By tuning p and q, node2vec can achieve better performance than DeepWalk in link prediction.

Tang et al. (2016) argued that DeepWalk lacks a clear objective function tailored for network embedding and proposed the LINE algorithm that learns node representations on the basis of a carefully designed objective function that preserves both the first-order and second-order proximity. The first-order proximity is captured by the observed links and thus can be formulated as follows:

$$O_1=-\sum _{\left(u,v\right)\in E}{w}_{uv}\log p_1\left(u,v\right),$$ (Equation 18)

where $w_{uv}$ is the weight of the observed link (u, v) and

$$p_1\left(u,v\right)=\frac{1}{1+\text{exp}\left[-g\left(u\right)\cdot g\left(v\right)\right]}$$ (Equation 19)

describes the likelihood of the existence of (u, v) given the embedding $g$. Of course, one can adopt other alternatives of Equation (19). The second-order proximity assumes that nodes sharing many connections to other nodes are similar to each other. Accordingly, each node is also treated as a specific context and nodes with similar distributions over contexts are assumed to be similar. Then, the second-order proximity can be characterized by the objective function:

$$O_2=-\sum _{\left(u,v\right)\in E}{w}_{uv}\log p_2\left(u|v\right),$$ (Equation 20)

where $p_2\left(u|v\right)$ denotes the probability that node v will generate a context u, namely

$$p_2\left(u|v\right)=\frac{\text{exp}\left[g^{\prime }\left(u\right)\cdot g\left(v\right)\right]}{{\sum }_{z\in V}\text{exp}\left[g^{\prime }\left(z\right)\cdot g\left(v\right)\right]},$$ (Equation 21)

with $g^{\prime }\left(u\right)$ being the context representation of u. Clearly, $O_2$ is naturally suitable for directed networks. By minimizing $O_1$ and $O_2$, LINE learns two kinds of node representations that, respectively, preserve the first-order and second-order proximity and takes their concatenation as the final representation.

In addition to DeepWalk, LINE, and node2vec, other well-known network embedding algorithms that have been applied in link prediction include DNGR (Cao et al., 2016), SDNE (Wang et al., 2016b), HOPE (Ou et al., 2016), GraphGAN (Wang et al., 2018), and so on. On the one hand, embedding is currently a very hot topic in network science and thought to be a promising method for link prediction. On the other hand, some very recent empirical studies (Muscoloni et al., 2020; Mara et al., 2020; Ghasemian et al., 2020) involving more than a thousand networks showed negative evidence that network embedding algorithms perform worse than some elaborately designed mechanistic algorithms. This is not a bad news because one can expect that some link prediction algorithms will enlighten and energize researchers in network embedding and thus make contributions to other aspects of network analyses like community detection, classification, and visualization.

Another notable embedding method is based on the hyperbolic network model (Krioukov et al., 2010; Papadopoulos et al., 2012), where each node is represented by only two coordinates (i.e., $d=2$) in a hyperbolic disk. The hyperbolic network models are very simple yet can reproduce many topological characteristics of real networks, such as sparsity, scale-free degree distribution, clustering, small-world property, community structure, self-similarity, and so on (Krioukov et al., 2010; Papadopoulos et al., 2012; Muscoloni and Cannistraci, 2018a). Papadopoulos et al., (2014) applied the hyperbolic model to the Internet at Autonomous Systems (AS) level and showed better performance than traditional methods (e.g., CN index and Katz index) in predicting missing links associated with low-degree nodes (see again (Zhu et al., 2012) for biased sampling preferring low-degree nodes). Wang et al. (2016c) proposed a link prediction algorithm for networks with community structure based on hyperbolic embedding, showing good performance for community networks with power-law degree distributions. Alessandro and Cannistraci proposed the so-called nonuniform popularity-similarity-optimization (nPSO) model as a generative model to grow random networks embedded in the hyperbolic space (Muscoloni and Cannistraci, 2018a), and they leveraged the nPSO model as a synthetic benchmark for link prediction algorithms, showing that the algorithm only accounting for hyperbolic distance does not perform well at the presence of communities (Muscoloni and Cannistraci, 2018b). They further proposed a variant of geometric embedding named minimum curvilinear automata (MCA) (Muscoloni and Cannistraci, 2018c), whose link prediction accuracy is higher than the simple hyperbolic distance (Muscoloni and Cannistraci, 2018b) but lower than coalescent embedding in hyperbolic space (Muscoloni et al., 2017). Kitsak et al. (2020) proposed an embedding algorithm named HYPERLINK in hyperbolic space, whose goal is to obtain more accurate prediction of missing links, so that HYPERLINK performs better than previous hyperbolic embedding algorithms (most of these algorithms are designed to reproduce topological features, not to predict missing links). HYPERLINK is often competitive to other well-known link predictors, and it is in particular good at predicting missing links that are really hard to predict.

Matrix completion

Matrix completion aims to reconstruct a target matrix, given a subset of known entries. Since links can be fully conveyed by the adjacency matrix A, it is natural to regard link prediction as a matrix completion task. Denote $E^k$ the set of node pairs corresponding to known entries in A that can be utilized in the matrix completion task. In most studies, $E^k=E^T$, while we should be aware of that $E^k$ can also contain some known absent links. The matrix completion problem can be formulated in line with supervised learning, as follows:

$$\underset{\vartheta }{\min }\frac{1}{\left|E^k\right|}\sum _{(i,j)\in E^k}\ell \left[a_{ij},{\tilde{a}}_{ij}\left(\vartheta \right)\right]+\Omega \left(\vartheta \right),$$ (Equation 22)

where $\vartheta$ is the parameter vector, ${\tilde{a}}_{ij}$ is the predicted value of the model, $\ell (\cdot ,\cdot )$ is a loss function, and $\Omega$ is a regularization term preventing overfitting. The most frequently used loss functions are squared loss $\ell \left(a,b\right)={(a-b)}^2$ and logistic loss $\ell \left(a,b\right)=\log \left(1+e^{-ab}\right)$.

Matrix factorization is a very popular method for matrix completion, which has already achieved great success in a closely related domain, the design of recommender systems (Koren et al., 2009). We consider a simple network with symmetry A and assume $\tilde{A}$ can be approximately factorized as $\tilde{A}\approx U{U}^T$ with $U\in {\mathbb{R}}^{N\times d}$ and $d\ll N$; then, we need to solve the following optimization problem:

$$\underset{U}{\min }\frac{1}{\left|E^k\right|}\sum _{(i,j)\in E^k}\ell \left(a_{ij},u_i^T{u}_j\right)+\Omega \left(U\right),$$ (Equation 23)

where $u_i$ and $u_j$ are the ith and jth rows of U., respectively Notice that $u_i^T$ is the transpose of $u_i$ not the ith row of $U^T$. Though without topological interpretation, $u_i$ can be treated as a lower-dimensional representation of node i, and matrix factorization can also be considered as a kind of matrix embedding algorithms (Qiu et al., 2019). If we adopt the squared loss function and the Forbenius norm for $\Omega$, then we get a specific optimization problem:

$$\underset{U}{\min }\frac{1}{\left|E^k\right|}\sum _{(i,j)\in E^k}{\left(a_{ij}-u_i^T{u}_j\right)}^2+\lambda {\Vert U\Vert }_F^2,$$ (Equation 24)

where $\lambda$ is a tradeoff parameter. Menon and Elkan (2011) suggested that one can directly optimize for AUC on the training set, given some known absent links. Accordingly, the objective function Equation (23) can be rewritten in terms of AUC as follows:

$$\underset{U}{\min }\frac{1}{\left|E_{+}^k\right|\left|E_{-}^k\right|}\sum _{\left(i,j\right)\in E_{+}^k,(x,y)\in E_{-}^k}\ell \left(1,u_i^T{u}_j-u_x^T{u}_y\right)+\Omega \left(U\right),$$ (Equation 25)

where $E_{+}^k$ and $E_{-}^k$ are sets of known present and known absent links, respectively. Cleary, $E_{+}^k\cap E_{-}^k=\varnothing$ and $E_{+}^k\cup E_{-}^k=E^k$. Menon and Elkan (2011) showed that the usage of AUC-based loss function can improve AUC value by around 10% comparing with the routine loss function like Equation (24).

The factorization in Equation (24) is easy to be extended to directed networks (Menon and Elkan, 2011), bipartite networks (Natarajan and Dhillon, 2014), temporal networks (Ma et al., 2018), and so on. For example, if A is asymmetry, we can replace $\tilde{A}\approx U{U}^T$ by $\tilde{A}\approx U\Lambda U^T$ with $\Lambda \in {\mathbb{R}}^{d\times d}$, and thus, Equation (24) can be extended to the following:

$$\underset{U,\Lambda }{\min }\frac{1}{\left|E^k\right|}\sum _{(i,j)\in E^k}{\left(a_{ij}-u_i^T\Lambda u_j\right)}^2+\frac{{\lambda }_U}{2}{\Vert U\Vert }_F^2+\frac{{\lambda }_{\Lambda }}{2}{\Vert \Lambda \Vert }_F^2,$$ (Equation 26)

Analogously, for bipartite networks like gene-disease associations (Natarajan and Dhillon, 2014) and user-product purchases (Shang et al., 2010), if $A\in {\mathbb{R}}^{M\times N}$, then we can replace $\tilde{A}\approx U{U}^T$ by $\tilde{A}\approx W{H}^T$, where $W\in {\mathbb{R}}^{M\times d}$ and $H\in {\mathbb{R}}^{N\times d}$. Accordingly, we get the optimization problem for bipartite networks as follows:

$$\underset{W,H}{\min }\frac{1}{\left|E^k\right|}\sum _{(i,j)\in E^k}{\left(a_{ij}-w_i^T{h}_j\right)}^2+\frac{{\lambda }_W}{2}{\Vert W\Vert }_F^2+\frac{{\lambda }_H}{2}{\Vert H\Vert }_F^2,$$ (Equation 27)

where $w_i$ and $h_j$ are the ith and jth rows of W and H, respectively. More details can be found in Menon and Elkan (2011), Natarajan and Dhillon (2014), and Ma et al. (2018).

The explicit features of nodes, such as tags associated with users and products (Zhang et al., 2011), can also be incorporated in the matrix factorization framework. Menon and Elkan (2011) suggested a direct combination of explicit features and latent features learned from the observed topology. Denoting $x_i\in {\mathbb{R}}^s$ the vector of explicit features of node i, the predicted values ${\tilde{a}}_{ij}$ in Equation (22) are then replaced by

$${\tilde{a}}_{ij}=u_i^T{u}_j+v^T{x}_i+v^T{x}_j,$$ (Equation 28)

where $v\in {\mathbb{R}}^s$ is a vector of parameters. Experiments showed that the incorporation can considerably improve the prediction accuracy (Menon and Elkan, 2011). Jain and Dhillon (2013) proposed a so-called inductive matrix completion (IMC) algorithm that uses explicit features to reduce the computational complexity. In IMC, the predicted value can be expressed as ${\tilde{a}}_{ij}=x_i^{T}Q{Q}^T{x}_j,$ where $x_i\in {\mathbb{R}}^s$ is the vector of i’s explicit features, and $Q\in {\mathbb{R}}^{s\times t}$ is a low-rank matrix with small t, which describes the latent relationships between explicit features and topological structure. Q can be learned from observed links by the following optimization problem:

$$\underset{Q}{\min }\frac{1}{\left|E^k\right|}\sum _{(i,j)\in E^k}{\left(a_{ij}-x_i^{T}Q{Q}^T{x}_j\right)}^2+\lambda {\Vert Q\Vert }_F^2.$$ (Equation 29)

Notice that, in Equation (24), the number of parameters to be learned is Nd, while in Equation (29), we only need to learn st parameters. The numbers of latent features (d and t) could be more or less the same as they are largely dependent on the topological structure, while the number of nodes N is usually much larger than the number of explicit features s. Therefore, the computational complexity of IMC should be much lower than brute-force factorization methods. The original IMC is proposed for bipartite networks (Jain and Dhillon, 2013), which has already found successful applications in the design of recommender systems (Jain and Dhillon, 2013) and the prediction of gene- and RNA-disease associations (Natarajan and Dhillon, 2014; Lu et al., 2018; Chen et al., 2018).

Pech et al. (2017) argued that low rank is the most critical property in matrix completion. They assumed that the observed network can be decomposed into two parts as $A=A_B+A_E$, where $A_B$ is called the backbone preserving the network organization pattern, and $A_E$ is a noise matrix, in which positive and negative entries are spurious and missing links, respectively. Pech et al. (2017) considered only two simple properties: the low rank of $A_B$ and the sparsity of $A_E$. Accordingly, $A_B$ and $A_E$ can be determined by solving the following optimization problem:

$$\underset{A_B,A_E}{\min }\text{rank}\left(A_B\right)+\lambda {\Vert A_E\Vert }_0\text{subject to}A=A_B+A_E,$$ (Equation 30)

where ${A_E}_0$ is the $l_0$-norm counting the number of nonzero entries of $A_E$. The predicted links can be obtained by sorting entries in $A_B$ that correspond to zero entries in A. This method is straightforwardly named as low rank (LR) algorithm. Although being simple, LR performs better than well-known similarity-based algorithms reported in Zhou et al. (2009) and Cannistraci et al. (2013a), hierarchical structure model (Clauset et al., 2008), and stochastic block model (Guimerà and Sales-Pardo, 2009), while slightly worse than LOOP (Pan et al., 2016), structural perturbation model (Lü et al., 2015), and Cannistraci-Hebb automata (Muscoloni et al., 2020).

Ensemble learning

In an early survey (Lü and Zhou, 2011), we noticed the low stability of individual link predictors and thus suggested ensemble learning as a powerful tool to integrate them. Ensemble learning is a popular method in machine learning, which constructs and integrates a number of individual predictors to achieve better algorithmic performance (Zhou, 2012). Roughly speaking, ensemble learning techniques can be divided into two classes: the “parallel ensemble” where individual predictors do not strongly depend on each other and can be implemented simultaneously (e.g., bagging (Breiman, 1996) and random forests (Breiman, 2001)) and the “sequential ensemble” where the integration of individual predictors has to be implemented in a sequential way (e.g., boosting (Freund and Schapire, 1997) and stacking (Wolpert, 1992)). In the following, we will, respectively, introduce how parallel ensemble and sequential ensemble are applied to link prediction.

Given an observed network, an individual link predictor will produce a rank of all unobserved links. Pujari and Kanawati (2012) proposed an aggregation approach on ranks resulted from individual algorithms. If there are R ranks produced by R individual predictors, an unobserved link e’s Borda count $B_k\left(e\right)$ in the kth rank can be defined as the number of links ranked ahead of e (there are many variants of Borda count and here we use the simplest one). Pujaji and Kanawati used a weighted aggregation to obtain the final score of any unobserved link e, as follows:

$$B\left(e\right)=\sum _{k=1}^R{w}_k{B}_k\left(e\right),$$ (Equation 31)

where $w_k$ is set to be proportional to the precision of the kth predictor trained by the observed network. Clearly, smaller $B\left(e\right)$ indicates higher existence likelihood. In addition to rank aggregation, a similar weighting technique can also be applied in integrating likelihood scores. If every unobserved link is assigned a score (higher score indicates higher existence likelihood) by each predictor, then the final score of any unobserved link e can be defined in a weighted form as follows:

$$S\left(e\right)=\sum _{k=1}^R{w}_k{S}_k\left(e\right),$$ (Equation 32)

where $S_k\left(e\right)$ is the score from the kth predictor. Different from rank aggregation, $S_k(\cdot )(k=1,2,\cdots ,R)$ should be normalized before the weighted sum to ensure scores from different predictors are comparable. An alternative aggregation method is the ordered weighted averaging (OWA) (Yager, 1988), where the R predictors are ordered according to their importance to the final prediction, as $S^{\left(1\right)},S^{\left(2\right)},\cdots ,S^{\left(R\right)}$, and then the final score of any unobserved link e is

$$S\left(e\right)=\sum _{k=1}^R{w}_k{S}^{\left(k\right)}\left(e\right),$$ (Equation 33)

where ${\sum }_{k=1}^R{w}_k=1$ and $w_1\ge w_2\ge \cdots \ge w_R\ge 0$. Without prior information, the most usual way to determine the weights is using the maximum entropy method, which maximizes $-{\sum }_{k=1}^R{w}_k\ln w_k$ subject to ${\sum }_{k=1}^R{w}_k=1$ and $\eta =\frac{1}{R-1}{\sum }_{k=1}^{R-1}(R-k)w_k$, where $\eta$ is a tunable parameter measuring the extent to which the ensemble (33) is like an or operation. If $\eta$ is very large, then $w_1\approx 1$ and $w_k\approx 0(k\ge 2)$, that is to say, only the first predictor works. He et al. (2015) applied OWA to aggregate nine local similarity indices. These indices are ordered according to their normalized values (irrelevant to their qualities), which is essentially unreasonable. Therefore, although He et al. (2015) reported considerable improvement, later experiments (Wu et al., 2019; Zhang et al., 2020) indicated that the method in He et al. (2015) does not work well because the position of a predictor is irrelevant to its quality. In contrast, if the order is relevant to the predictors’ qualities (e.g., according to their precisions trained by the target network), OWA will bring in remarkable improvement compared with individual predictors (Wu et al., 2019). As some link prediction algorithms scale worse than $\mathbb{O}\left(N\right)$, Duan et al. (2017) argued that to solve smaller problems multiple times is more efficient than to solve a single large problem. They considered a latent factor model (similar to the one described by Equation (23), with complexity $\mathbb{O}\left(N{d}^2\right)$) and developed several ways for the bagging decomposition, such as bagging with random nodes together with their immediate neighbors and bagging preferring dense components. They showed that those bagging techniques can largely reduce computational complexity without sacrificing prediction accuracy. Considering the family of stochastic block models (Guimerà and Sales-Pardo, 2009), Valles-Catala et al. (2018) showed that the integration (via Markov Chain Monte Carlo sampling according to Bayesian rules) of individually less plausible models can result in higher predictive performance than the single most plausible model.

Boosting is a typical sequential ensemble algorithm that trains a base learner from initial training set and adjusts weights of instances (the wrongly predicted instances will be enhanced while the easy-to-be-predicted instances will lose weights) in the training set to train the next learner. Such operation will continue until reaching some preseted conditions. The most representative boosting algorithm is AdaBoost (Freund and Schapire, 1997), which is originally designed for binary classification and thus can be directly applied in link prediction. AdaBoost is an additive model as

$$H\left(x\right)=\sum _{t=1}^T{\alpha }_t{h}_t\left(x\right),$$ (Equation 34)

where $h_t$ is the tth base learner, ${\alpha }_t$ is a scalar coefficient, and T is a preseted terminal time. H aims to minimize the expected value of an exponential loss function:

$$\ell \left(H,W\right)={\mathbb{E}}_{x\sim W}\left[e^{-f\left(x\right)H\left(x\right)}\right],$$ (Equation 35)

where $f\left(x\right)$ denotes the true class of the instance x and W is the original weight distribution. Without specific requirements, we usually set $W\left(x\right)=1/m$ for every instance x where m is the number of instances. We set $W_1=W$ and learn $h_1$ from $W_1$, and then, ${\alpha }_1$ is determined by minimizing $\ell \left({\alpha }_1{h}_1,W_1\right)$. The weight of an instance x in the second step is updated as follows:

$$W_2\left(x\right)=\{\begin{array}{ll}\frac{1}{Z_2}{W}_1\left(x\right)e^{-{\alpha }_1},\hfill & h_1=f\left(x\right)\hfill \\ \frac{1}{Z_2}{W}_1\left(x\right)e^{{\alpha }_1},\hfill & h_1\ne f\left(x\right)\hfill \end{array},$$ Equation (36)

where $Z_2$ is the normalization factor. Obviously, if the instance x can be correctly classified, its weight will decrease; otherwise, its weight will increase. Such process iterates until reaching the terminal time T. When applying AdaBoost in link prediction, we need to be aware of the following three issues. (i) The base learner should be sensitive to $W_t$ so that we cannot use unsupervised algorithms or supervised algorithms insensitive to $W_t$. (ii) In addition to positive instances (observed links), negative instances should be sampled from unobserved links. Though it introduces some noise, the influence is ignorable if the network is sparse. (iii) The negative instances should be undersampled to keep the data balanced. Comar et al. (2011) proposed the so-called LinkBoost algorithm, which is an extension of AdaBoost to link prediction with a typical matrix factorization model being the base learner. Instead of undersampling negative instances, they suggest a cost-sensitive loss function which penalizes the misclassifying links as nonlinks about N times heavier than misclassifying nonlinks as links. They further considered a degree-sensitive loss function that penalizes more for misclassification of links between low-degree nodes than high-degree nodes.

Stacking (Wolpert, 1992) is another powerful approach in sequential ensemble. It trains a group of primary learners from the initial training set and uses the outputs of primary learners as input features to train the secondary learner that provides the final prediction. If both primary learners (i.e., input features) and training instances are directly generated by the same training set, the risk of overfitting will be very high. Therefore, the original training set D, usually containing similar numbers of positive and negative instances for data balance, is divided into J sets with same size as $D_1,D_2,\cdots ,D_J$. Denoting $h_r^{\left(j\right)}$ the primary learner using the rth algorithm and trained from the jth fold of the training set $\overline{D_j}$ = D\D_j, for each instance $x_i\in D_j$, its feature vector is $z_i=(z_{i1},z_{i2},\cdots ,z_{iR})$, where $z_{ir}=h_r^{\left(j\right)}\left(x_i\right)$ and R is the number of primary algorithms. This J-fold division ensures all features of any instance x are obtained by primary learners trained without x. Some scientists have already used similar techniques (e.g., using various regressions to integrate results from primary predictors and other features (Zhang et al., 2020; Zhang, 2017; Fire et al., 2013)), but they are not aware of stacking model and did not employ any measures to avoid overfitting. Li et al. (2020) proposed a stacking model for link prediction, which use logistic regression and XGBoost to learn 4 similarity indices. Their method is inspiring, but they only considered 4 primary predictors and tested on two very small networks with some experimental results (e.g., the AUC values of CN index) far different from well-known results, and thus, the reported results and conclusion are questionable. Ghasemian et al. (2020) proposed a stacking model that considers 203 primary link predictors on 550 disparate networks. Using a standard supervised random forest algorithm (Breiman, 2001) as the secondary learner, Ghasemian et al. (2020) argued that the stacking model is remarkably superior to individual predictors for real networks and can approach to the theoretical optima for synthetic networks with known highest prediction accuracies. In addition, they showed that social networks are more predictable than biological and technological networks. However, a recent large-scale experiment (Muscoloni and Cannistraci, 2021) suggested that the above stacking model does not perform better than SPM (Lü et al., 2015) and Cannistraci-Hebb automata (Muscoloni et al., 2020). Wu et al. (2019) proposed an alternative sequential ensemble strategy called network reconstruction, which reconstructs network via one link prediction algorithm and predicts missing links by another prediction algorithm.

Discussion

In this review, to improve the readability, we classify representative works in the last decade into five groups. Of course, some novel and interesting methods, such as evolutionary algorithm (Bliss et al., 2014), ant colony approach (Sherkat et al., 2015), structural Hamiltonian analysis (Pan et al., 2016), and leading eigenvector control (Lee et al., 2021), do not belong to any of the above groups, and readers are encouraged to read other recent surveys (Wang et al., 2015; Martínez et al., 2016; Kumar et al., 2020) as complements of the present review.

Computational complexities of some representative algorithms mentioned in this perspective are reported in Table 1. Those algorithms are roughly categorized into three classes according to their computational complexities: the ones of “high” scalability can be applied to large-scale networks with millions of nodes, the ones of “medium” scalability can be applied to mid-sized networks with tens to hundreds of thousands of nodes, and the ones of “low” scalability can only deal with small networks with up to a few thousands of nodes using a common desktop computer. Given the size and sparsity of the target network, as well as the computational power, this table helps readers in finding suitable algorithms.

Table 1.

Computational complexities of some representative link prediction algorithms

Algorithm	References	Complexity	Scalability
CN	Liben-Nowell and Kleinberg (2007)	$\mathbb{O}\left(N{k}^2\right)$	High
RA	Zhou et al. (2009)	$\mathbb{O}\left(N{k}^2\right)$	High
LP	Zhou et al. (2009)	$\mathbb{O}\left(N{k}^3\right)$	Medium
CH	Muscoloni et al. (2018)	$\mathbb{O}\left(N{k}^3\right)$	Medium
L3	Kovács et al. (2019)	$\mathbb{O}\left(N{k}^3\right)$	Medium
SPM	Lü et al. (2015)	$\mathbb{O}\left(d{N}^2\right)$	Medium
DeepWalk	Perozzi et al. (2014)	$\mathbb{O}\left(\gamma N\log N\right)$	Medium
LINE	Tang et al. (2016)	$\mathbb{O}\left(tNk\right)$	Medium
NetSMF	Qiu et al. (2019)	$\mathbb{O}\left(N{d}^2\right)$	Medium
IMC	Jain and Dhillon (2013)	$\mathbb{O}\left(Nk{d}^2\right)$	Medium
LR	Pech et al. (2017)	$\mathbb{O}\left(d{N}^2\right)$	Medium
HSM	Clauset et al. (2008)	$\mathbb{O}\left(e^N\right)$	Low
SBM	Guimerà and Sales-Pardo (2009)	$\mathbb{O}\left(e^N\right)$	Low
LOOP	Pan et al. (2016)	$\mathbb{O}\left(N^3\right)$	Low

$N$ is the number of nodes, $k$ is the average degree, $\gamma$ is usually a large number depending on the number of random walks, the length of each walk, and the implemented deep learning model, $t$ is the number of iterations, and $d$ is the representation dimension or the preseted rank for matrix factorization and low-rank decomposition.

Very recently, a notable issue is the applications of neural networks in link prediction, which may be partially facilitated by the dramatic advances of deep learning techniques. Zhang and Chen (2017) trained a fully connected neural network on the adjacency matrices of enclosing subgraphs (with a fixed size) of target links. They applied a variant of the Weisfeiler-Lehman algorithm to determine the order of nodes in each adjacency matrix, ensuring that nodes with closer distances to the target link are ranked in higher positions. Zhang and Chen (2018) further proposed a novel framework based on graph neural networks, which can learn multiple types of information, including general structural features and latent and explicit node features. In this framework, a node’s order in the enclosing subgraph can be determined only by its closeness to the target link and the subgraph size can be flexible. Wang et al. (2020) directly represented the adjacency matrix of a network as an image and then learned hierarchical feature representations by training generative adversarial networks. Some preliminary experimental results suggested that the performance of those methods (Zhang and Chen, 2017, 2018; Wang et al., 2020) is highly competitive to many other state-of-the-art algorithms. Despite of the promising results, at present, features and models are simply pieced together without intrinsic connections. The above pioneering works (Zhang and Chen, 2017, 2018; Wang et al., 2020) provide a good start but we still need in-depth and comprehensive analyses to push forward related studies.

Although most link prediction algorithms only account for structural information, attributes of nodes (e.g., expression levels of genes (Natarajan and Dhillon, 2014) and tags of citation and social networks (Zhang et al., 2011; Wang et al., 2019)) can be utilized to improve the prediction performance. It is easy to treat attributes as independent information additional to structural features and work out a method that directly combines the two, while what is lacking but valuable is to uncover nontrivial relationship between attributes and structural roles and then design more meaningful algorithms. Beyond explicit attributes, we should also pay attention to dynamical information. It is known to us that limited time series obtained from some dynamical processes can be used to reconstruct network topology (Shen et al., 2014) while even a small fraction of missing links in modeling dynamical processes can lead to remarkable biases (Nicolaou and Motter, 2020); however, studies about how to make use of the correlations between topology and dynamics to predict missing links or how to take advantage of link prediction algorithms to improve estimates of dynamical parameters are rare.

By an elaborately designed model, Gu et al. (2017) showed that there is no ground truth in ranking influential spreaders even with a given dynamics. Peel et al. (2017) proved that there is no ground truth and no free lunch for community detection. The latter implies that no detection algorithm can be optimal on all inputs. Fortunately, we have ground truth in link prediction; however, extensive experiments (Ghasemian et al., 2020) also implicate that no known link predictor performs best or worst across all inputs. If link prediction is a no-free-lunch problem, then no single algorithm performs better or worse than any other when applied to all possible inputs. It raises a question that whether the study on prediction algorithms is valuable. The answer is of course YES (Guimerà, 2020) because we actually have free lunches as what we are interested in, the real networks, have far different statistics from those of all possible networks. As Ghasemian et al. (2020) argued that the ensemble models are usually superior to individual algorithms, a related question is whether the study on individual algorithm is valuable. The answer is still YES. Firstly, a recent large-scale experimental study (Muscoloni and Cannistraci, 2021) indicated that the performance of the stacking model is worse than elaborately designed individual algorithms, like SPM (Lü et al., 2015) and Cannistraci-Hebb automata (Muscoloni et al., 2020). Secondly, an individual algorithm could be highly cost-effective for its competitive performance and low complexity in time and space. Above all, individual algorithms, especially the mechanistic algorithms, may provide significant insights about network organization and evolution. In some real applications like friend recommendation, predictions with explanations are more acceptable (Barbieri et al., 2014), which cannot be obtained by ensemble learning. In addition, an alogical reason is that some elegant individual models (e.g., HSM (Clauset et al., 2008), SBM (Guimerà and Sales-Pardo, 2009), SPM (Lü et al., 2015), HYPERLINK (Kitsak et al., 2020), etc.) bring us inimitable esthetic perception that cannot be experienced elsewhere.

Along with fruitful algorithms proposed recently, the design of novel and effective algorithms for general networks is increasingly hard. We expect a larger fraction of algorithms in the future studies will be designed for networks of particular types (e.g., directed networks (Zhang et al., 2013), weighted networks (Zhao et al., 2015), multilayer networks (Bacco et al., 2017), temporal networks (Bu et al., 2019), hypergraphs and bipartite networks (Daminelli et al., 2015; Benson et al., 2018), networks with negative links (Leskovec et al., 2010; Tang et al., 2015), etc.) and networks with domain knowledge (e.g., drug-target interactions (Wu et al., 2018), disease-associated relations (Zeng et al., 2018), protein-protein interactions (Kovács et al., 2019; Lei and Ruan, 2013), criminal networks (Berlusconi et al., 2016), citation networks (Liu et al., 2019), academic social networks (Kong et al., 2019), knowledge graphs (Nickel et al., 2015), etc.). We should take serious consideration about properties and requirements of target networks and domains in the algorithm design, instead of straightforward (and thus less valuable) extensions of general algorithms. For example, if we attempt to recommend friends in an online social network based on link prediction (Aiello et al., 2012b), we need to consider how to explain our recommendations to improve the acceptance rate (Barbieri et al., 2014), how to use the acceptance/rejection information to promote the prediction accuracy (Wu et al., 2013), and how to avoid recommending bots to real users (Aiello et al., 2012a). These considerations will bring fresh challenges in link prediction.

Early studies often compare a very few algorithms on several small networks according to one or two metrics. Recent large-scale experiments (Mara et al., 2020; Ghasemian et al., 2020; Muscoloni et al., 2020; Muscoloni and Cannistraci, 2021; Zhou et al., 2021) indicated that the above methodology may result in misleading conclusions. Future studies ought to implement systematic analyses involving more synthetic and real networks, benchmarks, state-of-the-art algorithms, and metrics. Researchers can find benchmark datasets for networks from Open Graph Benchmark (OGB, ogb.stanford.edu), Pajek (mrvar.fdv.uni-lj.si/pajek), and Link Prediction Benchmarks (LPB, www.domedata.cn/LPB). If relevant results cannot be published in an article with limited space, they should be made public (better together with data and codes) in some accessible websites like GitHub, OGB, and LPB.

Lastly, we would like to emphasize that the soul of a network lies in its links instead of nodes; otherwise, we should pay more attention on set theory rather than graph theory. Therefore, in network science, link prediction is a paradigmatic and fundamental problem with long attractivity and vitality. Beyond an algorithm predicting missing and future links, link prediction is also a powerful analyzing tool, which has already been utilized in evaluating and inferring network evolving mechanisms (Wang et al., 2012; Zhang et al., 2015; Zhang, 2017), testing the privacy-protection algorithms (as an attaching method) (Xian et al., 2021), evaluating and designing network embedding algorithms (Dehghan-Kooshkghazi et al., 2021; Gu et al., 2021), and so on. Though the last decade has witnessed plentiful and substantial achievements, the study of link prediction is just unfolding and more efforts are required toward a full picture of how links do emerge and vanish.