Visibility graph for time series prediction and image classification: a review

Abstract

The analysis of time series and images is significant across different fields due to their widespread applications. In the past few decades, many approaches have been developed, including data-driven artificial intelligence methods, mechanism-driven physical methods, and hybrid mechanism and data-driven models. Complex networks have been used to model numerous complex systems due to its characteristics, including time series prediction and image classification. In order to map time series and images into complex networks, many visibility graph algorithms have been developed, such as horizontal visibility graph, limited penetrable visibility graph, multiplex visibility graph, and image visibility graph. The family of visibility graph algorithms will construct different types of complex networks, including (un-) weighted, (un-) directed, and (single-) multi-layered networks, thereby focusing on different kinds of properties. Different types of visibility graph algorithms will be reviewed in this paper. Through exploring the topological structure and information in the network based on statistical physics, the property of time series and images can be discovered. In order to forecast (multivariate) time series, several variations of local random walk algorithms and different information fusion approaches are applied to measure the similarity between nodes in the network. Different forecasting frameworks are also proposed to consider the information in the time series based on the similarity. In order to classify the image, several machine learning models (such as support vector machine and linear discriminant) are used to classify images based on global features, local features, and multiplex features. Through various simulations on a variety of datasets, researchers have found that the visibility graph algorithm outperformed existing algorithms, both in time series prediction and image classification. Clearly, complex networks are closely connected with time series and images by visibility graph algorithms, rendering complex networks to be an important tool for understanding the characteristics of time series and images. Finally, we conclude in the last section with future outlooks for the visibility graph.

Introduction

With the growth of the Internet and mobile technology, the amount of data faced by researchers is ever-increasing [1, 2]. Describing big data and the complicated relationships between data has become an open problem that still needs to be solved. Artificial intelligence and machine learning algorithms [3, 4], due to their data-driven characteristics, have become one of the most popular algorithms in various fields because they can discover the complicated information behind big data and serve as a black-box model to address problems that could not be solved in the past. In the field of data mining, time series and image analysis are indispensable topics, attracting researchers in various fields [5, 6], including computer science [7], economics [8], environment [9], and biology [10]. Researchers have proposed various algorithms to promote the development and progress of this field [11–13], including data-driven models, mechanism-driven models, and hybrid mechanism and data-driven models. For example, a long short-term memory network with a chaotic Henry gas solubility optimization model was developed to forecast the crude oil time series [14], a nonlinear hybrid model was applied to analyze and predict the wind speed data collected from five wind farms in the Marmara region, Turkey [15], a nonlinear autoregressive network with exogenous inputs is applied to analyze the impact of environmental stresses on the infection of COVID-19 [16], recurrent neural networks based on the dual-stage two-phase model were proposed to address long-term multivariate time series prediction [17], different divergence and entropy are applied to analyze the multiscale structure and irreversibility in financial time series [18, 19], and a hybrid model consisting of 2D curvelet transform and deep learning technique was developed to diagnose COVID-19 disease based on chest X-ray images [20].

A complex system includes many components that are related to each other. Complex systems have been used to address practical problems [21, 22], including social media, modern cities, human brains, infrastructure systems, and economic organizations. Complex systems are often modeled by complex networks. Nodes and edges in the network usually represent individuals (components) and connections between them in the system [23, 24], respectively. Complex networks have been widely applied, for instance, identifying influential individuals in the spread [25, 26], analyzing the dynamics of the ecosystem [27–30], studying synchronization control [31, 32], and discovering properties of complex systems [33, 34]. Interdisciplinary research has also been carried out to strengthen the ability of complex networks to solve problems in combination with other disciplines, including entropy [35, 36], fuzzy theory [37, 38], game theory [39, 40], machine learning [41], evidence theory [42–44], and information fusion [45–47]. Complexity measure and uncertainty measure are very important for time series, which can be measured by entropy [48, 49]. Belief entropy has the efficiency to model uncertainty of mass function [50, 51], which is used in time series analysis [52]. Several algorithms have contributed to describing the complicated information and features hidden in time series and images. However, when the complexity of the system increases, the state of the art will not be able to describe the property of time series and images properly due to their own limitations. Complex networks, as one of the most effective interdisciplinary methods for describing complex systems, have been used to analyze time series and images [53, 54]. Various algorithms have been developed to construct complex networks from time series [55, 56], including proximity networks, visibility graphs, and transition networks. The visibility graph (VG) algorithm [57–59] is an important algorithm that constructs complex networks with corresponding topological structures based on the dynamic property of the univariate/multivariate time series, which has been widely used.

In the network generated by the visibility graph algorithm, whether nodes are connected is determined by the visibility criterion between data points in the time series. In detail, the time series will be displayed through the histogram. If the straight “visibility” line that connects two data points does not intersect any intermediate data height [57], two data points in the time series can see each other from the top of the bar, resulting in the connectivity between two nodes in the natural visibility graph. Luque et al. [60] then developed the horizontal visibility graph. The novelty of this algorithm is that two nodes in the network will be connected if a horizontal line that connects corresponding data points in the time series does not intersect any intermediate data height. Other types of visibility graphs [61] have been proposed to obtain the network with different structure based on the feature of time series, such as (un-) weighted, (un-) directed, and (single-) multi-layered networks. Numerous properties in the time series can be revealed by the family of visibility graphs, for example, estimating the Hurst exponent of fractal stochastic processes [62, 63], proving the relationship between the power-law degree distribution and fractality in series [57], analyzing multifractal properties of time series [64], and measuring the irreversibility of real-valued time series [65–67]. In addition, it has been applied in different fields to address practical problems, including studying the dynamics of a passive scalar plume [68], planning long-voyage routes [69], analyzing electroencephalogram signals [70], extracting hidden information in coupled timessss series [71, 72], and aggregating data in complex systems [73].

Recently, the family of visibility graph algorithms has been applied to forecast the time series by analyzing the structural information of the network [74]. After the time series $(t_i,y_i)$, $t_i=1,2,\dots ,T$ is converted into a complex network, the similarity between nodes is measured by the local random walk algorithm, and the most similar node to the last node ($(t_T,y_T)$ in time series) is used to forecast the data point at $T+1$. Apart from the local random walk, there are many algorithms for identifying the similarity between nodes in the network [75], for example, the similarity can be measured by the relative entropy based on the local network [76], and the local random walk approach was improved to be applied in the visibility graph [77]. Fuzzy logic has been applied to determine weight parameters of different sources of information, thereby considering more information in the network to improve the prediction accuracy [78]. The information considered in prediction models includes the information in the last data point ($(t_T,y_T)$) [74], the relationship between $(t_T,y_T)$ and its most similar node(s) [79], and the adjacent information [80]. The weight parameters of the prediction model can also be determined by the distance between data points [79, 81]. The induced-ordered weighted averaging aggregation (IOWA) operation and network-based multiple time-frequency spaces were applied to forecast time series to improve the accuracy [82]. In order to validate the performance of these visibility graph-based prediction models, both in-sample forecasting and out-of-sample frameworks have been applied. After studying the time series, the image visibility graph [83] has been proposed to map images into complex networks to analyze properties of images, including filters, pattern recognition, and texture classification. The global features, local features, and multiplex features can be extracted from images via the image visibility graph [83]. Notably, the visibility patch can accurately describe the local information of images, thereby embedding the image in a low-dimensional space. The image visibility graph can be further used to classify images into different categories by combining with several machine learning models. Compared with traditional artificial intelligence models, including VGG16, VGG19, and convolutional neural network (CNN), the best classification accuracy obtained by image visibility graph algorithms [83–85] is higher, illustrating the effectiveness of the visibility graph algorithm.

Here, we have reviewed the methods to map time series and images into complex networks, mainly the visibility graph algorithm and its various variants. In addition, we have also reviewed the time series forecasting and image classification methods based on the visibility graph algorithm. These constitute the framework of this review paper. Different kinds of complex network construction methods are first introduced in Sect. 2. The visibility graph and its improved forms are then introduced in Sect. 3, for example, the horizontal visibility graph and multiplex visibility graph are applied to analyze univariate time series and multivariate time series (Sect. 3.1 to Sect. 3.4), respectively. A variety of other improved visibility graph algorithms with different visibility criteria and time complexity is introduced in Sect. 3.5. Analyzing different types and characteristics of time series by the visibility graph are discussed in Sect. 3.6. After mapping the time series into the complex network, the time series can be predicted through the local random walk model and other information fusion methods (Sect. 4), including fuzzy logic, distance-based method, and IOWA operation. Construction Cost Index and several criteria are used to compare the accuracy obtained by VG-based algorithms with other approaches (simple moving average approach). The image visibility graph and its applications in feature extraction, filters, pattern recognition, and texture classification are introduced in Sect. 5. The conclusions and future outlooks are discussed in Sect. 6.

Network-based prediction approach

Given a time series $(t_i,y_i)$, where $t_i=1,2,\dots ,T$ and T is the length, Zhang et al. [86] developed an algorithm to map the pseudoperiodic time series into a complex network. $(t_i,y_i)$ is first divided into $\left|N\right|$ disjoint cycles $\left\{C_1,,,C_2,,,\dots ,,,C_{\left|N\right|}\right\}$ based on the local minimum or local maximum value in the time series. Each node in the network will represent a cycle $C_i$. There are two ways to determine whether two nodes are connected in this approach: (1) required distance $\theta$ and (2) linear correlation coefficient $\rho$. The distance between two nodes ($C_i$ and $C_j$) is defined as follows,

$$\begin{array}{c}\hfill D_{\mathit{ij}}=\underset{l=0,1,\dots ,l_j-l_i}{min}\frac{1}{l_i}\sum _{k=1}^{l_i}∥X_k-Y_{k+l}∥,\end{array}$$ 1

where $l_i$ and $l_j$ are the length of $C_i$ and $C_j$, respectively. It is important to note that $l_j$ should be larger than $l_i$ in this model. $X_k$ and $Y_k$ are the kth point of $C_i$ and $C_j$, respectively. If the distance $D_{\mathit{ij}}$ between two nodes is smaller than $\theta$ (for the toy model) or $\rho$ (for the experimental time series), $C_i$ and $C_j$ will be connected in the complex network. The steps of this mapping model is given in Algorithm 1.

Xu et al. [87] then developed a new model to construct the network from the time series that can describe several characteristics of continues dynamics (such as periodic, chaotic, and periodic with noise) based on the distribution of sub-networks, and more details about this work can be found in Ref. [87]. The steps are shown below:

• Step (1)

  Obtain the time series $(t_i,y_i)$, where $t_i=1,2,\dots ,T$.

• Step (2)

  The series is embedded in a phase space [88], and each state in the phase space is represented as a node in the network.

• Step (3)

  Connect each node to its $\tau$ nearest neighbor nodes to generate the complex network.

For this method, the temporal separation of these eligible neighbor nodes should be larger than the average period of the series to avoid inhabiting the same “strand” [89]. There will be $\tau$ new nearest neighbor nodes for each node, regardless of whether the node has been connected. In addition, there are no multiple links between each pair of nodes. Therefore, each node has an average of $2\tau$ edges connected to neighbor nodes ($〈k〉=2\tau$). Selecting $\tau$ nearest neighbor nodes in this method is more flexible than setting the distance threshold, which can obtain an adjustable threshold based on the density of phase space. They also suggest that this approach works best when $\tau$ is $\left[3,,,8\right]$ and $\left[3,,,4\right]$ for maps and flows, respectively [87].

Lacasa et al. [57] proposed the visibility algorithm to construct the complex network from the time series based on the visibility criterion. An example time series and corresponding visibility graph (VG) are given in Fig. 1a. In this algorithm, each point will be represented by a node in the same order in the network. For any two data points $(t_i,y_i)$ and $(t_j,y_j)$, node i and node j will be connected if visibility exits between data points in time series, that is, any point $(t_k,y_k)$ between them meets the condition,

$$\begin{array}{c}\hfill y_k<y_j+(y_i-y_j)\frac{t_j-t_k}{t_j-t_i}.\end{array}$$ 2

Therefore, the visibility graph has the following three properties,

1. Undirected The edges between nodes have no direction, which is different from the sequential time series.

2. Connected There is no isolated node in the network as the visibility exists between every pair of neighbor data points.

3. Invariant under affine transformations The rescaling and translation of horizontal and vertical axes do not affect the visibility criterion and the structure of the network.

The data in the time series is sequential, and the property of the time series is mainly analyzed from the historical data. The visibility between data points in the time series can be expressed as the connection between nodes, and the properties in the series are also inherited by the network. In detail, periodic series, random series, and fractal series are mapped to regular network, random network, and scale-free network, respectively. The periodic series and regular network are shown in Fig. 1a. The degree distributions that can indicate the property of random and scale-free networks constructed by corresponding time series can be found in Fig. 2. As discussed in ref [57], a natural bridge can be built between the time series analysis and network theory through the visibility graph.

Fig. 1.

An example time series $(t_i,y_i)$ ($T=16$) and corresponding VG, HVG, and LPVG. Solid and dashed line in c indicate edges generated by the VG algorithm and new edges under limited penetrable distance $\lambda =1$, respectively. Image generated based on Refs. [57, 60, 90]

Fig. 2.

The example of random and fractal time series with corresponding degree distribution of random and scale-free networks. The fractal time series is a Brownian motion with parameter 0.2. Image generated based on Ref. [57]

Visibility graph

Due to the importance of the visibility graph, the family of the visibility graph is introduced in this section. Different types of networks can be constructed based on different visibility criteria, including (un-) weighted, (un-) directed, and (single-) multi-layered network. Therefore, this section is the foundation for time series prediction and image classification.

Horizontal visibility graph

Different from the visibility graph, node i and node j will be connected in the horizontal visibility graph (HVG) if all nodes between them ($t_i<t_k<t_j$) meet the following geometrical criterion [60],

$$\begin{array}{c}\hfill y_k<y_i,y_k<y_j.\end{array}$$ 3

The criterion can also be rewritten as

$$\begin{array}{c}\hfill y_k<inf\left\{y_i,y_j\right\}.\end{array}$$ 4

Due to the strict mapping rule of HVG, HVG is a sub-network of VG when they are constructed by the same time series. In addition, the average degree of nodes in HVG is smaller than the average degree of nodes in VG, so the relationship between nodes is limited to the local structural property. An example of HVG is shown in Fig. 1b. In addition to the last two properties of VG, the horizontal visibility graph has three additional properties,

1. Reversible and irreversible properties of the converting Information loss in the series caused by the binary adjacency matrix in unweighted networks can be avoided by using weighted networks to represent the time series.

2. Directed and undirected properties of the converting Undirected networks are constructed by this algorithm, but directed networks can also be constructed by distinguishing the ingoing degree $d_{\mathrm{in}}$ and outgoing degree $d_{\mathrm{out}}$.

3. Geometric criteria difference HVG has “less visibility” than VG due to its strict geometric criteria, but it does not affect qualitative features of the network.

Based on the characteristic of the HVG, correlated stochastic, uncorrelated, and chaotic processes can be easily classified by the HVG algorithm [91]. Apart from the undirected networks, the directed links (from time $t_i$ to time $t_j>t_i$) are introduced in the network by the directed VG and HVG algorithms [65], which can show the time-reversal asymmetry of the series. The degree $d(t_i)$ of node at time $t_i$ is defined as the sum of the ingoing degree $d_{\mathrm{in}}(t_i)$ and outgoing degree $d_{\mathrm{out}}(t_i)$, that is, $d(t_i)=d_{\mathrm{in}}(t_i)+d_{\mathrm{out}}(t_i)$. The irreversibility of the time series can be measured by the Kullback–Leibler divergence between in-degree and out-degree distribution in the directed HVG [65].

Limited penetrable visibility graph

Based on the framework of VG, the limited penetrable visibility graph (LPVG) [90] was proposed to consider more links when mapping the time series into a complex network. The limited penetrable distance $\lambda$ is defined in this method. Two nodes (node i and node j) will be connected if there are only $n<\lambda$ nodes (node k) that do not fulfill the condition in Eq. (2), which means there are less than $\lambda$ intermediate data points $(t_k,y_k)$ are

$$\begin{array}{c}\hfill y_k>y_j+(y_i-y_j)\frac{t_j-t_k}{t_j-t_i}.\end{array}$$ 5

The LPVG has the same three properties as the VG, and each node has at least $2\lambda +1$ edges. In addition, LPVG has stronger connectivity compared to VG and HVG, resulting in a better anti-noise ability. However, some nodes that are useful for prediction are excluded in the LPVG, resulting in the loss of information and inaccurate predictions. An example of LPVG algorithm ($\lambda =1$) is given in Fig. 1c: Solid lines represent edges generated by the VG algorithm, and dashed lines represent new edges. The LPVG will degenerate to the VG when $\lambda =0$ (no new edges in Fig. 1a).

The limited penetrable horizontal visibility graph (LPHVG) [92] is similar to LPVG. The difference is that there should be less than $\lambda$ intermediate data $(t_k,y_k)$ are

$$\begin{array}{c}\hfill y_k>inf\left\{y_i,y_j\right\}.\end{array}$$ 6

The LPHVG will also be equivalent to HVG when $\lambda =0$ (no new edges).

The directed-limited penetrable horizontal visibility graph (DLPHVG) and the image-limited penetrable horizontal visibility graph (ILPHVG), which are named as the LPHVG family, have also been developed [93]. Numerical experiments have been taken to investigate topological properties of these networks, and they can be used to show the irreversibility of time series and discriminate noise from chaos. The sequential motifs in the LPVF can extract the hidden information in different flow patterns [94].

Multiscale limited penetrable horizontal visibility graph

In order to investigate the time series from the multiscale perspective, the multiscale limited penetrable horizontal visibility graph (MLPHVG) [95] has been proposed. The steps of this algorithm are shown below:

• Step (1)

  Obtain the time series $(t_i,y_i)$, where $t_i=1,2,\dots ,T$.

• Step (2)
  Define the temporal scale and obtain the coarse-grained time series $(t_i,y_i^{\left[\kappa \right]})$,

  $$\begin{array}{c}\hfill y_j^{\left[\kappa \right]}=\frac{{\sum }_{i=(j-1)\kappa +1}^{j\kappa }{y}_i}{\kappa },1\le j\le T/\kappa ,\end{array}$$ 7

  where $\kappa$ is the scale factor.
• Step (3)

  The complex network is then constructed from the coarse-grained time series based on the LPHVG algorithm [92] with different scales $\kappa$.

Multiplex visibility graph

Since physical, economic, and biological phenomena can be more effectively and accurately described by the multivariate time series, there have been numerous technologies and approaches to analyze the multidimensional signal. From the visibility graph perspective, the multiplex visibility graph (MVG) can map the multidimensional time series into multi-layer networks, thereby extracting information in the time series by analyzing the topological structural characteristics of the multi-layer network [96].

For a multivariate time series (M-dimension), $\mathrm{x}\left(t_i\right)$ is given as $\left\{x^{\left[1\right]}\left(t_i\right),x^{\left[2\right]}\left(t_i\right),\dots ,x^{\left[M\right]}\left(t_i\right)\right\}\in {\Re }^M$ for $\forall 0<t_i<T$. A M-layer multiplex network can be constructed, and each layer (e.g., layer $\xi$) can be constructed by the time series of $x^{\left[\xi \right]}\left(t_i\right)$ based on the VG algorithm. It is important to note that other types of visibility graph can also be applied to define each layer of the multiplex graph. One 3-layer multiplex network under the undirected HVG is constructed in Fig. 3 as an example. For the M-layer multiplex network, the topological structure is defined by the vector of adjacency matrices $\left\{A^{\left[1\right]},A^{\left[2\right]},\dots ,A^{\left[M\right]}\right\}$, where $A^{\left[\xi \right]}=\left\{a_{\mathit{ij}}^{\left[\xi \right]}\right\}$ represent the adjacency matrix of layer $\xi$, and $a_{\mathit{ij}}^{\left[\xi \right]}$ shows whether node i and node j are connected in layer $\xi$ [97]. Therefore, the information of the multivariate time series can be analyzed by the structure and characteristics of the multiplex network.

Fig. 3.

Mapping a multivariate time series ${\left\{x^{\left[\xi \right]}\left(t_i\right)\right\}}_{\xi =1}^3$ into a 3-layer multiplex network by the MVG algorithm under the HVG geometrical criterion. Each layer ($L_{\alpha }$) of the multiplex network is constructed by $x^{\left[\alpha \right]}\left(t_i\right)$ in the multivariate time series by the HVG algorithm. Image generated based on Ref. [96]

In order to investigate the information shared by different variables of the high-dimensional system, two measures are developed in the multiplex network. The average edge overlap is defined to show the abundance of single edges across layers,

$$\begin{array}{c}\hfill \alpha =\frac{{\sum }_i{\sum }_{j>1}{\sum }_{\xi }{a}_{\mathit{ij}}^{\left[\xi \right]}}{M{\sum }_i{\sum }_{j>i}\left(1-{\delta }_{0,{\sum }_{\xi }{a}_{\mathit{ij}}^{\left[\xi \right]}}\right)},\end{array}$$ 8

where $\alpha \in \left[1,/,M,,,1\right]$, and two cases are discussed below.

1. $\alpha =1/M$: Only $a_{\mathit{ij}}^{\left[\xi \right]}=1$, and $a_{\mathit{ij}}^{\left[\zeta \right]}=0$ for $\forall \zeta \ne \xi$.

2. $\alpha =1$: The networks in M layers are the same.

The interlayer mutual information is then developed to investigate the interlayer correlation between the degree of the same node in different layers,

$$\begin{array}{c}\hfill I_{\xi ,\zeta }=\sum _{k^{\left[\xi \right]}}\sum _{k^{\left[\zeta \right]}}P(k^{\left[\xi \right]},k^{\left[\zeta \right]})log\frac{P(k^{\left[\xi \right]},k^{\left[\zeta \right]})}{P(k^{\left[\xi \right]})P(k^{\left[\zeta \right]})},\end{array}$$ 9

where $P(k^{\left[\xi \right]})$ and $P(k^{\left[\zeta \right]})$ are the degree distribution in layer $\xi$ and $\zeta$, respectively, and $P(k^{\left[\xi \right]},k^{\left[\zeta \right]})$ represents the joint probability [96]. $I={〈I_{\xi ,\zeta }〉}_{\xi ,\zeta }$ can be then obtained to describe the typical amount of information flow.

Vector visibility graph (VVG) was also proposed to map multivariate time series into directed complex networks and study structures of stock markets [98]. The principled and systematic procedure that constructs the network from a multivariate time series was proposed by Kramer et al. [99], which was applied to analyze the signal during an epileptic seizure. The network can also be constructed by the small-shuffle surrogate approach from the multivariate time series [100], thereby showing the relationship of irregular fluctuations between two series. In order to study the nonlinear dynamic characteristic of gas–liquid flows, Gao et al. [101] have developed a new approach to map multivariate time series to a directed weighted complex network, which can investigate the transitions of different flow patterns. They then constructed a multiscale complex network based on the multivariate time series [102] and studied the coupled dynamical characteristics in the network by using a new clustering coefficient entropy in the gas–liquid two-phase flow experiment [103].

Others

The time complexity of the VG algorithm [57] is $\mathrm{O}(n^2)$, thus it needs a long time to map long time series to a complex network. In order to reduce the time complexity, Lan et al. [104] developed a fast transform VG algorithm based on the degree centrality of each node. They proposed the “divide & conquer” algorithm to find nodes from large degree to small degree and divide the time series into multiple segments to identify the connectivity between nodes, thereby achieving a shorter amount of time $\mathrm{O}(nlogn)$. Weighted complex networks can also be constructed from time series [105] based on the improved VG algorithm. The weight of links is determined by the induced ordered averaging aggregation operator and visibility graph aggregation operator, which can consider both the importance of nodes in the network and time decay in time series. Different dynamics in time series can construct different structures of the directed weighted network [106] because of orbits in the chaotic attractor.

The sequential visibility-graph motif [107] is a computationally efficient and highly informative feature, which can classify dynamics properties with noise, thereby being used to study the deterministic and stochastic dynamics in the time series. The parametric natural visibility graph (PNVG) algorithm will consider the view angle apart from the original features in VG [108], which can study the property that cannot be identified by the VG. When there is a window slide moving in the time series, the time-dependent limited penetrable visibility graph (TDLPVG) algorithm has been developed, which can analyze the statistical properties of the time series [109]. The parametric modified limited penetrable visibility graph (PMLPVG) is then improved based on the LPVG [110], so that it shows the dynamic properties of time series and enhances the reasonability of the penetrable visibility, which has been tested in different real-world time series. Due to the unavoidability of noise in time series, the improved power of scale-freeness of VG (PSVG) is developed to measure the fractality of time series by analyzing the fractality in different scales [111]. The time series can also be analyzed based on the recurrence network [112–114]. There are still many other types of visibility graphs with different features and geometrical criteria, which can be used to analyze different time series.

Application

Due to the effectiveness of the VG algorithm, the family of VG algorithms has been applied in different fields to address practical problems, such as financial time series [115–117], electroencephalogram (EEG) signal [70, 118], traffic data [119, 120], earthquake time series [121, 122], and market dataset [123, 124].

In order to make the time series smoothing, the degree distribution in the VG that shows the importance of nodes is applied to determine the weight of aggregation, and the visibility graph averaging (VGA) aggregation operator is developed to aggregate several time series [125]. Inspired by the VGA aggregation operator, the visibility graph power averaging aggregation operator [126] is then proposed based on a new support function that takes into account the relationship and correlation between values. The ordered visibility graph weighted averaging aggregation operator [127] that takes into account the information of order and argument values is then developed to determine the weight to aggregate time series effectively. Xu et al. [128] linearly combined weights obtained by the visibility graph averaging operator and induced ordered weighted aggregation operator, thereby smoothing time series and avoiding information loss. The visibility graph averaging aggregation operator has also been used on the topic of dynamic data fusion, which is a data-driven approach to consider past data credibility without subjective parameters setting [73]. The HVG can be applied to analyze chaotic signals polluted with white noise [65], thereby showing the irreversibility of series with noise. The noise from chaos can also be distinguished by the HVG algorithm [129]. The nonstationary time series can also be analyzed by the variant of VG algorithm [109, 130]. Through the application in the exchange rate series, the original, shuffled, and detrended series can be analyzed by the VG algorithm [63].

The visibility graph algorithm is also applied to study the feature of EEG signals, including the diagnosis of Alzheimer’s disease [131], sleep stages classification [118], epileptic seizure detection [132, 133], and alcoholism identification [134]. Apart from the EEG signal, the seismicity sequences can also be analyzed by the VG [135], and they found the power-law distribution of magnitude intensity. In addition, the time series of energy dissipation rates in turbulence [136] can be investigated by the visibility graph. The statistical characteristic of the network is then studied by the degree distribution and the box-covering algorithm. The fractional Brownian motions and multifractal random walks are then analyzed by the visibility graph [137], and they found the degree of nodes in the VG follows power-law distributions, which is influenced by the temporal correlation of the time series. Lacasa et al. [62] also mapped the fractional Brownian motion into a scale-free network to study the relationship between the degree distribution and Hurst parameter H, thereby investigating long-range dependence in this kind of time series. The synchronization between time series has been measured by the visibility graph similarity (VGS) approach [138]. The VGS indicates the correlation between the degree sequence of different systems, thereby showing the similarity of dynamics between two coupled systems (temporal synchronization).

Time series prediction

In order to forecast the time series, many algorithms have been proposed. For example, the linear autoregressive model was proposed based on a linear function [139]. The local linear predictor [140] is then developed which can achieve a similar performance to the global linear predictor. A transformer-based model [141] was proposed to predict long sequence time-series, which has a better performance than existing methods. Several time series prediction methods based on deep learning [142] were reviewed. We will also review the time series prediction models based on the visibility graph algorithm. This method analyzes the property of time series from the structural information of the complex network, which is a new perspective. To accurately forecast the time series from the visibility graph perspective, the similarity between nodes needs to be evaluated first to develop the prediction model. In the complex network, the similarity between nodes can be evaluated by numerous approaches [75], including maximum likelihood models, probabilistic methods, and structure-based algorithms.

Zhang et al. [74] first developed a model based on the visibility graph and local random walk in the network to forecast time series. Given a time series $(t_i,y_i)$, where $t_i=1,2,\dots ,T$, the network is constructed by the VG algorithm [57]. The topological structure can be represented by the adjacency matrix $A_{\left|N\right|\times \left|N\right|}$, where $a_{\mathit{ij}}$ shows whether node i is connected to node j. $k_i={\sum }_j{a}_{\mathit{ij}}$ is the degree of node i. Note that T in the time series equals to $\left|N\right|$ in the network. The similarity between the last node at time T (corresponding to $(t_T,y_T)$) and other nodes is evaluated by the local random walk approach [77]. Specifically, the transition probability that a random walker at node i walks to node j each step is represented by,

$$\begin{array}{c}\hfill P_{\mathit{ij}}=a_{\mathit{ij}}/k_i,\end{array}$$ 10

where $P_{\mathit{ij}}$ is an element in the probability matrix $P_{\left|N\right|\times \left|N\right|}$. Hence, the probability that a walker reaches node j from node i after t steps is defined by,

$$\begin{array}{c}\hfill {\pi }_i(t)=P^T{\pi }_i(t-1),\end{array}$$ 11

where ${\pi }_i$ is an all-zero vector ($\left|N\right|\times 1$) except the ith element is 1. The similarity between nodes based on the local random walk approach is then obtained by,

$$\begin{array}{c}\hfill S_{\mathit{ij}}(t)=\frac{k_i}{2\left|E\right|}{\pi }_{\mathit{ij}}(t)+\frac{k_j}{2\left|E\right|}{\pi }_{\mathit{ji}}(t).\end{array}$$ 12

Note that t is the time that the walker needs in the network, which is not related to the time series. In addition, $S_{\mathit{ij}}$ is a symmetric matrix ($S_{\mathit{ij}}=S_{\mathit{ji}}$). In order to prevent this walker from going away from both node i and node j, the random walker should walk in the local network rather than the entire network, resulting in a higher similarity,

$$\begin{array}{c}\hfill S_{\mathit{ij}}=\sum _{l=1}^t{S}_{\mathit{ij}}(l).\end{array}$$ 13

After identifying the most similar node $(t_S,y_S)$ to the last node $(t_T,y_T)$ at time T, the time series $(t_{T+1},{\overline{y}}_{T+1})$ at the next time $T+1$ is obtained by,

$$\begin{array}{c}\hfill {\overline{y}}_{T+1}=\frac{y_T-y_S}{t_T-t_S}(t_{T+1}-t_T)+y_T.\end{array}$$ 14

The framework of this method is shown in Fig. 4a. When the value at $T+1$ is predicted, a new network with $T+1$ nodes will be constructed by the VG algorithm, thereby predicting the value at the next time $T+2$. This is called the one-step-ahead prediction because it only predicts the value at the next time. A multi-step-ahead prediction approach has also been proposed to increase the applicability of the model [74].

Fig. 4.

The forecasting frameworks of different approaches. Image generated based on Refs. [74, 78, 80]

This model was further improved by fuzzy logic [78]. After identifying the most similar node $(t_S,y_S)$ to the last node $(t_T,y_T)$, the distance between two nodes is determined by,

$$\begin{array}{c}\hfill d_{\mathit{ST}}=\left|t_T-t_S\right|,\end{array}$$ 15

and the fuzzy variable is obtained by,

$$\begin{array}{c}\hfill r={log}_2{d}_{\mathit{ST}}.\end{array}$$ 16

Three factors are then considered to design the fuzzy rule in this method, and they are:

1. Range of visibility Some old but still visible data is not conducive to the current prediction.

2. Level of visibility Several fuzzy sets are used to describe the level of visibility.

3. Effect of visibility The predicted data $(t_{T+1},{\widehat{y}}_{T+1})$ is determined by (a) the direct effect from $(t_T,y_T)$ and (b) the indirect effect from the initial forecasting result $(t_{T+1},{\overline{y}}_{T+1})$.

Therefore, the weighting parameters of the direct effect $w_1$ and indirect effect $w_2$ are determined by the fuzzy rule,

$$\begin{array}{c}\hfill w_1,w_2=\frac{{\sum }_{l=1}^M{Q}^l{\mu }_{\mathit{pl}}(r)}{{\sum }_{l=1}^M{\mu }_{\mathit{pl}}(r)}.\end{array}$$ 17

The specific introduction about the fuzzy rule can be found in Ref. [78]. Finally, the predicted value can be obtained by two kinds of effect,

$$\begin{array}{c}\hfill {\widehat{y}}_{T+1}=w_1\times y_T+w_2\times {\overline{y}}_{T+1}.\end{array}$$ 18

The predicted value ${\widehat{y}}_{T+1}$ obtained by considering two kinds of information can give a more accurate prediction, and this method has high flexibility and predictability. The framework of this method is shown in Fig. 4b.

When $(t_S,y_S)$ is farther away from $(t_T,y_T)$, it will reveal more historical information of the time series, allowing this point to be more important. Hence, a new approach has been proposed [79] that determines the weighting parameters by the distance between nodes,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill w_1& =\frac{d_{T(T+1)}}{d_{S(T+1)}},\hfill \\ \hfill w_2& =\frac{d_{\mathit{ST}}}{d_{S(T+1)}}.\hfill \end{array}\end{array}$$ 19

The predicted value ${\widehat{y}}_{T+1}$ is also obtained by Eq. (18) with new weighting parameters.

Mao and Xiao [80] then developed a model that combines the linear approximation model and the adjacent prediction approach. Specifically, the impact of $(t_T,y_T)$ on $(t_{T+1},y_{T+1})$ is similar to the impact of $(t_S,y_S)$ on $(t_{S+1},y_{S+1})$ due to the similarity between $(t_T,y_T)$ and $(t_S,y_S)$, and thus, the adjacent prediction approach is defined by,

$$\begin{array}{c}\hfill {\tilde{y}}_{T+1}=\frac{y_{S+1}-y_S}{t_{S+1}-t_S}(t_{T+1}-t_T)+y_T.\end{array}$$ 20

The linear approximation prediction model is given in Eq. (14), and the weighting parameters are obtained by the distance between nodes (Eq. (19)). Therefore, the final prediction result ${\widehat{y}}_{T+1}$ is obtained by,

$$\begin{array}{c}\hfill {\widehat{y}}_{T+1}=w_1\times {\tilde{y}}_{T+1}+w_2\times {\overline{y}}_{T+1}.\end{array}$$ 21

The framework of this method is shown in Fig. 4c.

After constructing the complex network from the fast transform algorithm [104], Zhao et al. [81] proposed a method to consider the impact of all nodes that are visible to the last node $(t_T,y_T)$. The set of nodes that are visible to $(t_T,y_T)$ is denoted by $t^{\ast }=\left\{{t_i}^{\ast }\right\},i\in R$. If there is only one visible node in $t^{\ast }$, the initial predicted value based on the visible node $t_S={t_i}^{\ast }$ can be obtained by Eq. (20), and the revised predictive value is obtained by,

$$\begin{array}{c}\hfill y_{T+1}^{\left[i^{\ast }\right]}=\frac{y_T+{\tilde{y}}_{T+1}}{2}.\end{array}$$ 22

However, the time series is usually complicated. Therefore, there are more than one visible nodes in $t^{\ast }$ in general. Hence, the effect of all visible nodes on future nodes should be considered, and the final predicted value ${\widehat{y}}_{T+1}$ is defined by,

$$\begin{array}{c}\hfill {\widehat{y}}_{T+1}=\sum _{i^{\ast }}{w}_{\left[i^{\ast }\right]}{y}_{T+1}^{\left[i^{\ast }\right]},\end{array}$$ 23

where $w_{\left[i\right]}$ is the weighting parameter of the visible node at ${t_i}^{\ast }$ that is defined by,

$$\begin{array}{c}\hfill w_{\left[i^{\ast }\right]}=\frac{k_{i^{\ast }}/d_{i^{\ast }T}}{\sum _{i^{\ast }}{k}_{i^{\ast }}/d_{i^{\ast }T}}.\end{array}$$ 24

where $k_{i^{\ast }}$ is the degree of node, $d_{i^{\ast }T}$ is the distance obtained by Eq. (15). Therefore, this approach considers more historical information in the time series, resulting in a comprehensive prediction. Recently, Hu and Xiao [143] have found that the directed VG algorithm still perform well after adding observational noise.

Local random walk has been applied in these methods to predict the potential link, but it leads to a high time complexity. A fast algorithm based on Markov chain was then developed to forecast the time series [144]. The similarity between nodes is obtained by the stationary distribution of the probability transfer matrix,

$$\begin{array}{c}\hfill {V_{\mathit{Ti}}}^x(i)=P^x{V}_0,\end{array}$$ 25

where $V_0$ is the probability distribution of the initial nodes, P is the transfer matrix, and ${V_{\mathit{Ti}}}^x(i)$ describes the similarity between the last node $(t_T,y_T)$ and other nodes i. This method needs x iteration for $\left|V^{x+1}-V^x\right|<{10}^{-5}$. In addition, this method considers more historical information of the time series, and top k most similar nodes (the top k maximum value in V) are then selected to forecast the time series,

$$\begin{array}{c}\hfill {\widehat{y}}_{T+1}=\sum _{S=1}^k\frac{V_S}{\sum _{S=1}^k{V}_S}\left(\frac{y_T-y_S}{t_T-t_S}(t_{T+1}-t_T)+y_T\right),\\ \hfill \end{array}$$ 26

where weighting parameters are determined by the similarity degree. The value of k is determined by the trend of the time series:

1. $k=2$: The time series has a downward or upward trend.

2. $k=4$: It is a periodic time series.

Liu et al. [82] then developed a fuzzy interval model to forecast hydrological and financial tie series. In their method, the time series needs to be decomposed and reconstructed first, the predicted values in different time-frequency spaces are then aggregated to obtain the final result. The steps of this approach are shown below:

1. The time series $(t_i,y_i)$ ($t_i=1,2,\dots ,T$) is decomposed into different intrinsic mode functions (IMFs),

   $$\begin{array}{c}\hfill y_i=\sum _{j=1}^n{c_i}^{\left[j\right]}+{r_i}^{\left[n\right]},\end{array}$$

   where ${r_i}^{\left[n\right]}$ is the residue after T IMFs are extracted. The detail settings can be found in Ref. [82].

2. The time series can be then reconstructed by the residue of different time-frequency spaces.

3. For each reconstructed part, the time series will be used to construct the VG (Eq. (2)), and the similarity between nodes can be measured by the local random walk approach (Eqs. (12) and (13)).

4. The predicted value for each reconstructed time series can be obtained by Eq. (14).

5. The golden-rule-based representation function [145] is then used to determine the membership of the forecasting interval for each time-frequency space.

6. The final predicted value can be obtained by aggregating the values in different time-frequency space via the induced ordered weighted averaging aggregation operation [146].

The Construction Cost Index (CCI)1 describes the average price of several factors in the construction industry from January 1990 to July 2014, rendering people to estimate costs and make budgets in advance, thus attracting researchers’ interests. These models based on the visibility graph are used to forecast the CCI dataset. Using the approach in Ref. [74], predicted values and actual values of CCI data are shown in Fig. 5a. In the entire time span, predicted values are similar to real values, thereby showing the performance of VG-based forecasting models. The constructed VGs in different steps ($T=50$, $T=100$, and $T=200$) are shown in Fig. 5b–d, respectively. The last node at time T and its most similar node are represented by the red node and blue node, respectively. Several criteria are then used to evaluate the performance of the predictability, including mean absolute difference (MAD), mean absolute percentage error (MAPE), root mean square error (RMSE), and normalized root mean squared error (NRMSE),

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathrm{MAD}& =\frac{1}{n}\sum _{i=1}^n\left|{\widehat{y}}_i-y_i\right|,\hfill \\ \hfill \mathrm{MAPE}& =\frac{1}{n}\sum _{i=1}^n\frac{\left|{\widehat{y}}_i-y_i\right|}{y_i}\times 100,\hfill \\ \hfill \mathrm{RMSE}& =\sqrt{\frac{1}{n}\sum _{i=1}^n{\left|{\widehat{y}}_i-y_i\right|}^2},\hfill \\ \hfill \mathrm{NRMSE}& =\frac{\sqrt{\frac{1}{n}\sum _{i=1}^n{\left|{\widehat{y}}_i-y_i\right|}^2}}{y_{max}-y_{min}}\times 100,\hfill \end{array}\end{array}$$ 27

where $y_i$ is the actual value, ${\widehat{y}}_i$ is the predicted value, n is the number of data. The simple moving average ($k=1$) [147] is applied as the comparison method. The results of the comparison method and several VG-based approaches (in-sample forecasting) are given in Table 1. As observed from the results, these VG-based models can better forecast CCI time series with lower errors.

Fig. 5.

Predicting CCI data by the approach in Ref. [74] and corresponding VGs with different numbers of nodes. a Real and predicted values of CCI data. b Constructed VG with 50 nodes ($T=50$). c Constructed VG with 100 nodes ($T=100$). d Constructed VG with 200 nodes ($T=200$). Red node and blue node mean the last node at time T and its most similar node. Image generated based on Ref. [74]. (Color figure online)

Table 1.

Errors of different approaches in CCI dataset (in-sample forecasting)

Reference	MAD	MAPE(%)	RMSE	NRMSE(%)
Ref. [147]	21.59	0.3110	32.73	0.6350
Ref. [74]	20.05	0.2900	–	–
Ref. [78]
Phase 1	20.0	0.2889	29.3	0.5690
Phase 2	19.6	0.2845	29.1	0.5659
Ref. [79]	19.30	0.2797	28.16	0.5462
Ref. [81]	19.92	0.2882	29.06	0.5600
Ref. [144]	19.88	0.2899	28.06	0.5455

Data obtained from Refs. [74, 78, 79, 81, 144]

A sliding window over data has also been applied to validate the forecasting via the out-of-sample prediction. The algorithm of the out-of-sample prediction is shown in Algorithm 2.

Two approaches are used to make out-of-sample predictions, and the results with different lengths of the sliding window L are shown in Table 2. When the length of the sliding window L becomes longer, the MAPE is also larger. In addition, the MAPE obtained by the approach in Ref. [144] is lower than the MAPE obtained by the model in Ref. [78]. The average values of MAPE under $L=3,6,12$ are then used to evaluate the performance of different methods. Two VG-based approaches are compared with Holt ES, Holt-Winter ES, ARIMA, Seasonal ARIMA, and official ENR data. The results (Table 3) show that the VG-based approaches achieve the lowest MAPE and have better performance.

Table 2.

MAPE of two VG-based prediction models with different values of L in CCI dataset (out-of-sample forecasting)

Reference	$L=3$ (%)	$L=6$ (%)	$L=12$ (%)
Zhang et al. [78]	0.48	0.75	1.27
Liu et al. [144]	0.45	0.62	0.84

Data obtained from Refs. [78, 144]

Table 3.

The average value of MAPE under $L=3,6,12$ obtained by different approaches in CCI dataset (out-of-sample forecasting)

Method	Holt ES	Holt-Winter ES	ARIMA	Seasonal ARIMA	ENR	Zhang et al. [78]	Liu et al. [144]
MAPE	1.03%	1.00%	1.29%	1.06%	1.22%	0.83%	0.64%

Data obtained from Refs. [78, 144]

Image classification

Apart from one-dimensional time series, the family of VG algorithms can also be applied to two-dimensional images [83], including image processing and classification. In the network, each node represents a pixel in the image, and the connection between nodes is derived from the image visibility graph (IVG) algorithm. Specifically, a $N\times N$ image is represented by its pixel matrix $I_{N\times N}\in R$, and there will be $N^2$ nodes in corresponding IVG. Followed by the IVG algorithm, two nodes ij and $i^{\prime }{j}^{\prime }$ will be connected if they fulfill both conditions:

1. $i=i^{\prime }$ or $j=j^{\prime }$ or $\left(i=i^{\prime }+p\right)\wedge \left(j=j^{\prime }+p\right)$ for any integer p.

2. $I_{\mathit{ij}}$ and $I_{i^{\prime }{j}^{\prime }}$ are connected in the VG.

If the second condition is that $I_{\mathit{ij}}$ and $I_{i^{\prime }{j}^{\prime }}$ are connected in the HVG, the constructed network will be the image horizontal visibility graph (IHVG). Connections in both IVG and IHVG have specific directions (rows, columns, and diagonals) in the image according to the first condition. When the pixel matrix $I_{3\times 3}$ is given as

$$\begin{array}{c}\hfill I_{3\times 3}=\left[\begin{array}{ccc}105& 90& 100\\ 50& 95& 90\\ 90& 10& 40\end{array}\right],\end{array}$$

the constructed IHVGs with and without the lattice structure are shown in Fig. 6.

Fig. 6.

Constructed IHVGs with different structures. a Constructed IHVG with the lattice structure. b Constructed IHVG without the lattice structure. Image generated based on Ref. [84]

The family of IVG algorithms can be applied as image filters. For a given image $I_{N\times N}$, the visibility filter obtained by the IVG and IHVG are defined as $F_{\mathit{VG}}{(I)}_{N\times N}$ and $F_{\mathit{HVG}}{(I)}_{N\times N}$, respectively. Filters ($F_{\mathit{VG}}{(I)}_{\mathit{ij}}$ and $F_{\mathit{HVG}}{(I)}_{\mathit{ij}}$) are defined by the property of node in the constructed network, such as k-filter based on the degree, C-filter based on the clustering coefficient, and $K_{\mathit{nn}}$-filter based on the degree–degree [148]. Specifically, the clustering coefficient needs to be rescaled in grayscale intensity $\left\{0,,,255\right\}$ in the C-filter. Several standard grayscale images and their IVG k-filter are shown in Fig. 7.

Fig. 7.

Several standard grayscale images and their IVG k-filter. Image generated based on Ref. [83]

More importantly, the family of IVG algorithms can be used in pattern recognition and texture classification based on three kinds of features.

1. Multiplex features It is usually applied to deal with several images at once, such as RBG images. These images will construct an image multiplex visibility graph (IMVG). The features in the multilayer network can be described by three types of descriptors, including the intra-layer descriptors, inter-layer descriptors, and intrinsically multiplex descriptors. This kind of feature needs further exploration because multilayer networks are still not widely used in images.

2. Global features It is defined by the topological structural information of the entire network, such as the degree distribution and clustering distribution. Other statistical characteristics of networks can also be applied here.

3. Local features It is mainly defined by the visibility patches (VPs) in the network, with order p, the visibility patch contains $p^2$ nodes ${\left\{\mathit{ij}\right\}}_{i=r_1,j=r_2}^{r_1+p-1,r_2+p-1}$ for $1\le r_1\le N-p$ and $1\le r_2\le N-p$. In the network, the lowest order is $p=3$ because the structure of all VPs will be the same when the order is $p=2$. The VP with order p can be represented by a $2p+2+q$-dimensional vector,

   $$\begin{array}{c}\hfill \left[r_1,r_2,\dots ,r_p,c_1,c_2,\dots ,c_p,d_1,d_2,l_1,l_2,\dots ,l_q\right],\end{array}$$

   where $q=4(p-2)$. Specifically, there are p rows, p columns, main diagonal and anti-diagonal of the patch, and q motifs of lower orders of off-diagonals and off-anti-diagonals.

An example of VP with order $p=3$ and its label are shown in Fig. 8. When two outer nodes are connected, the label is 1, otherwise the label is 0 (Fig. 8). There are total $2^{(2p+2)}=2^8=256$ kinds of patches with order $p=3$ in the network. The vector of the patch shown in Fig. 8 is $\left[1,,,0,,,0,,,1,,,0,,,0,,,1,,,0\right]$, and thus, the label can be obtained by,

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill L& =\sum _{i=1}^{2p+2+q}{V}_i{2}^{2p+2+q-i}=\sum _{i=1}^8{V}_i{2}^{8-i}\hfill \\ \hfill & =r_1{2}^7+\dots +d_2{2}^0+1=147.\hfill \end{array}\end{array}$$

Hence, VP can be used to represent the local feature of the network, and the number of VP with different labels (VP distribution) can be further used to classify images with different properties.

Fig. 8.

An example of obtaining the label of VP with order $p=3$. Image generated based on Ref. [83]

After collecting these three types of features of images, the principal component analysis (PCA) is used to reduce the number of features, thereby avoiding overfitting. Different kinds of support vector machine (SVM) algorithms are then applied to classify images into different categories.

The colored Brodatz dataset [151] has 12 classes and is widely applied to analyze texture and understand scanned images. Several IVG-based approaches and comparison methods based on artificial intelligence are used to classify images into different categories in this dataset (Table 4). The global and local multiplex features in the IVG and IHVG are both used to represent and classify images [83], the best classification accuracy obtained from all classifiers in the MATLAB Classification Learner is 99.8%. The texture classification based on image visibility graph (TCIVG) [84] considers the degree distribution of IVG and IHVG to classify images, and the best classification accuracy of this approach (91.4%) is obtained by applying quadratic discriminant in the IVG without lattice. Images are then divided into multiple segments by the matching intensity for image visibility graphs (MIIVG) [85], and the structure is described by the reference patterns and matching intensity. The best classification accuracy of this approach is 99.7%, and it is obtained by linear discriminant in MIIVG-1 and quadratic discriminant in MIIVG-2. The best classification accuracy of three comparison methods (CNN, VGG 16, and VGG19) in this dataset is 99.2%, 99.6%, and 99.7%, respectively. Therefore, IVG-based algorithms outcompete other comparison methods based on artificial intelligence on image classification.

Table 4.

The best classification accuracy obtained by different IVG-based models and comparison methods based on artificial intelligence in the colored Brodatz dataset

Method	Classifier	Accuracy (%)
IVG & IHVG [83]	Support vector machine	99.8
TCIVG [84]	Quadratic discriminant	91.4
MIIVG [85]	Linear & quadratic discriminant	99.7
CNN [149]	Support vector machine	99.2
VGG16 [150]	Linear discriminant	99.6
VGG19 [150]	Linear discriminant	99.7

Data obtained from [83–85]

IVG has also been applied as a parameter-free model to explore the local and global structure of signatures, thereby verifying signatures [152]. This approach can achieve higher accuracy than other classic state-of-the-art models. Pessa et al. [153] then proposed an approach to map images to ordinal networks by determining the structure of networks from the symbolization process. They have shown that this model can be used to identify image symmetries and classify textures in real-world datasets. In addition, their method is robust against noise, resulting in higher classification accuracy.

Conclusions and future outlooks

Analyzing time series and images from the perspective of complex networks is an innovative and effective method to address interdisciplinary problems. The characteristic of time series and images, including nonlinear dynamics, chaos, irreversibility, and local information of images, can be analyzed through the topological structure of complex networks. After combining artificial intelligence and information fusion models, the visibility graph can be applied to forecast time series and classify images, which can achieve a good performance. In this review paper, the visibility graph that can map time series and images to complex networks is discussed. Different kinds of rules that construct complex networks from time series are introduced in Sect. 2, where the visibility graph is the focus of this review paper. Hence, the family of visibility graphs is then introduced in Sect. 3 to construct different types of networks, including (un-) weighted, (un-) directed, and (single-) multi-layered networks. Specifically, the application of visibility graph algorithms in different fields is discussed in Sect. 3.6. The visibility graph can be combined with the local random walk algorithm and other information fusion models to forecast financial time series with in-sample forecasting and out-of-sample forecasting (Sect. 4). Through the comparison in the CCI dataset, the VG-based models are shown to achieve a better performance. The image visibility graph algorithm is then introduced in Sect. 5 to map images to complex networks, and images can be classified by considering three types of features extracted from the image visibility graph with the help of the support vector machine and other machine learning models. By considering the multiplex, global, and local features in the complex network, the IVG-based algorithms outcompete other comparison methods based on artificial intelligence in the colored Brodatz dataset.

We now highlight several research directions in this field that are of theoretical and practical value:

1. The family of visibility graph algorithms has been applied to many fields to solve various applied and engineering problems (as discussed in our main text). However, researchers have not developed corresponding visibility graph algorithms based on intrinsic features (such as period, dynamic characteristics, and local information) of different types of time series (including finance, earthquake, and EEG signal), but still use the same algorithm to solve problems from various disciplines. That is to say, existing visibility graph algorithms are not developed and improved based on the background of the problem statement. Therefore, algorithms for constructing complex networks based on the feature of time series still need further exploration to discover time series characteristics more accurately.

2. In complex networks, there are many algorithms to measure the similarity between nodes, such as common neighbors, SimRank, and superposed random walk. However, only the local random walk model is used in the existing time series prediction models. Can other models that consider more structural information about the time series be proposed to forecast the time series then? How can weights in these models be determined more reasonably?

3. The visibility graph algorithms have been proposed to address problems on the topic of time series and images. Can it be further extended to solve audio and video data to increase the scope of application of the visibility graph? Since the network-based analysis model can describe complicated relationships between data, it may provide a new perspective for analyzing the ever-increasing high-dimensional and large-scale datasets.

4. The visibility graph algorithms can be more closely integrated with artificial intelligence models. The data-driven machine learning model can mine the universal laws and features hidden behind the data. The visibility graph algorithm studies the visibility between the data through the network, thereby providing a new research perspective. Hybrid models will not only improve the accuracy of time series prediction and image classification, but also reduce the dimensionality of complicated time series and images dataset, thus revealing more features.

Data availability

All data generated or analyzed during this study are included in this published article.

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.