Optimal synchronization of directed complex networks

Abstract

We study optimal synchronization of networks of coupled phase oscillators. We extend previous theory for optimizing the synchronization properties of undirected networks to the important case of directed networks. We derive a generalized synchrony alignment function that encodes the interplay between the network structure and the oscillators' natural frequencies and serves as an objective measure for the network's degree of synchronization. Using the generalized synchrony alignment function, we show that a network's synchronization properties can be systematically optimized. This framework also allows us to study the properties of synchrony-optimized networks, and in particular, investigate the role of directed network properties such as nodal in- and out-degrees. For instance, we find that in optimally rewired networks, the heterogeneity of the in-degree distribution roughly matches the heterogeneity of the natural frequency distribution, but no such relationship emerges for out-degrees. We also observe that a network's synchronization properties are promoted by a strong correlation between the nodal in-degrees and the natural frequencies of oscillators, whereas the relationship between the nodal out-degrees and the natural frequencies has comparatively little effect. This result is supported by our theory, which indicates that synchronization is promoted by a strong alignment of the natural frequencies with the left singular vectors corresponding to the largest singular values of the Laplacian matrix.

Synchronization is vital to the functionality of many natural and engineered systems,^1–3 including cardiac pacemaker cells,⁴ circadian rhythms,⁵ Josephson junction arrays,⁶ and power grids.⁷ This has generated considerable interest in optimizing the synchronization properties of networks.^8–13 In a recent publication, we developed a theoretical framework for optimizing the synchronization properties of undirected networks of heterogeneous oscillators.¹³ Here, we extend this theory to the important case of directed networks and derive a generalized synchrony alignment function (SAF) that can be used to systematically optimize a network's synchronization properties. Furthermore, this approach allows us to examine which structural properties promote synchronization in directed networks. Potential applications include systems where strong synchronization is essential for efficient functionality, including cardiac electrophysiology,^14,15 synthetic cell engineering,¹⁶ and power grid dynamics.^17–20

I. INTRODUCTION

The tendency for large groups of individual units to reach consensus despite having heterogeneous dynamical properties has served as strong motivation for scientists to study synchronization of coupled dynamical systems.^1,2 A paradigmatic model for studying the synchronization and the emergence of collective behavior was developed by Kuramoto,²¹ who showed that under appropriate conditions, the dynamics of N oscillators can be reduced to the evolution of N phases, θ_i, for $i=1,\dots ,N$. When placed on a network that indicates the oscillators' interaction, the phases evolve according to

$${\dot{\theta }}_i={\omega }_i+K\sum _{j=1}^N{A}_{\mathit{i}\mathit{j}}{H}_{\mathit{i}\mathit{j}}({\theta }_j-{\theta }_i),$$ (1)

where ω_i is the natural frequency of oscillator i, K is the global coupling strength, $H_{\mathit{i}\mathit{j}}(\theta )$ is a $2\pi$-periodic coupling function, and A is the adjacency matrix that encodes the network structure such that A_ij = 1 if a link exists from node j to node i. In many cases, the topology of the network is assumed to be undirected, so that $A=A^T$; however, here we will consider the more general case of a possibly directed network topology.²² Furthermore, to ensure for the possibility of synchronization from Eq. (1), we assume that the coupling frustration²³ is sufficiently small, $|H_{\mathit{i}\mathit{j}}(0)/\sqrt{2}{H}_{\mathit{i}\mathit{j}}^{\prime }(0)|\ll 1$.

Extensive research has demonstrated that the interplay between dynamics and network structure has nonlinear effects on the synchronization of a network. For instance, different networks can give rise to different synchronization patterns,^24–29 and at the same time synchronization can be utilized to analyze the properties of a given network.^30,31 The macroscopic synchronization dynamics of Eq. (1) is typically quantified using the classical Kuramoto order parameter r defined by the complex number

$$r{e}^{i\psi }=\frac{1}{N}\sum _{j=1}^N{e}^{i{\theta }_j},$$ (2)

which represents the centroid of phases $\left\{{\theta }_i\right\}$ after mapping them onto the complex unit circle. In particular, r ranges between 0 and 1, representing completely incoherent and perfectly synchronized states, respectively, with intermediate values representing partially synchronized states. In a recent publication,¹³ we developed a theoretical framework for optimizing the synchronization properties of a given undirected network, as defined by maximizing the order parameter r. In particular, we derived the synchrony alignment function (SAF), a functional that encodes the interplay between the local dynamical properties (i.e., the oscillators' natural frequencies $\left\{{\omega }_i\right\}$) and the network structure (via the eigenvalues and eigenvectors of the network Laplacian matrix). We showed that the SAF can be used to systematically optimize the synchronization properties of a network under a wide variety of constraints.

As in the study of other network-coupled dynamical processes, extending analysis for undirected networks to directed networks represents a non-trivial hurdle for theoretical and practical progresses.^32–34 In the case of identical oscillators, our understanding of synchronization on directed networks is well-developed through the framework of Master Stability Functions,³⁵ and in fact, directed coupling can be utilized to achieve improved and moreover optimized synchronization, although sometimes at the expense of reduced robustness.^36–38

In this paper, we extend our previous results¹³ for optimizing synchronization of heterogeneous oscillators in undirected networks to the case of directed networks. We derive a generalized SAF and demonstrate its utility with several examples. In particular, we show that the generalized SAF can be used to optimize synchronization under several constraints: (i) choosing the oscillators' natural frequencies for a given network; (ii) arranging a set of pre-chosen natural frequencies on a given network; and (iii) building a network for a given set of natural frequencies. We emphasize that our approach allows for efficient optimization of a network's synchronization properties based on objective measures, not heuristics. Furthermore, the generalized SAF approach allows us to investigate the dynamical and topological properties of synchrony-optimized networks. For example, we study the directed network properties of nodal in- and out-degrees and find that synchronization is promoted by a strong, positive correlation between the nodal in-degrees and magnitude of the natural frequencies; however, the relationship between the nodal out-degrees and the natural frequencies has comparatively little effect on synchronization. We additionally observe that synchronization is enhanced by a negative correlation between the natural frequencies of neighboring oscillators. These results extend the previous research^8–13 that observed similar correlations to promote synchronization, but only considered the case of undirected networks.

The remainder of this paper is organized as follows: In Sec. II, we present our theoretical framework and derive the generalized SAF. In Sec. III, we provide several examples of optimizing synchronization using the generalized SAF. In Sec. IV, we study the effects that directed network properties have on optimal synchronization and the effect that optimizing synchronization has on directed network structure. In Sec. V, we study the relationship between structural and dynamical properties of synchrony-optimized networks. In Sec. VI, we conclude with a summary and discussion of our results.

II. DERIVATION OF THE GENERALIZED SYNCHRONY ALIGNMENT FUNCTION (SAF)

The derivation of the generalized SAF begins as in Ref. 13, by considering the dynamics of Eq. (1) in a state of strong synchronization, i.e., $r\approx 1$. For a generic network, such a state can be obtained in several ways, typically by either (i) sufficiently increasing the coupling strength K or (ii) sufficiently decreasing the spread or standard deviation of the natural frequencies. In fact, up to a rescaling of time, these two actions are equivalent. In this regime, the oscillators become strongly clustered about the mean phase ψ such that $|{\theta }_j-{\theta }_i|\ll 1$ for all $1\le i,j\le N$, and thus Eq. (1) can be linearized to

$${\dot{\theta }}_i\approx {\tilde{\omega }}_i-KH\prime (0)\sum _{j=1}^N{L}_{\mathit{i}\mathit{j}}{\theta }_j,$$ (3)

where we have assumed for simplicity that each coupling function is the same, i.e., $H_{\mathit{i}\mathit{j}}(\theta )=H(\theta )$ for all $1\le i,j\le N$, we have defined an effective natural frequency

$${\tilde{\omega }}_i={\omega }_i+KH(0)k_i^{\text{in}},$$ (4)

L is the Laplacian matrix defined for directed networks with entries defined

$$L_{\mathit{i}\mathit{j}}={\delta }_{\mathit{i}\mathit{j}}{k}_i^{\text{in}}-A_{\mathit{i}\mathit{j}},$$ (5)

where δ_ij is the Kronecker delta, and the nodal in- and out-degrees are given by

$$k_i^{\text{in}}=\sum _{j=1}^N{A}_{\mathit{i}\mathit{j}},\hspace{1em}{k}_i^{\text{out}}=\sum _{j=1}^N{A}_{\mathit{j}\mathit{i}}.$$ (6)

In the vector form, Eq. (3) can be more conveniently rewritten as

$$\dot{\mathit{\theta }}=\tilde{\mathit{\omega }}-KH\prime (0)L\mathit{\theta },$$ (7)

where $\mathit{\theta }={[{\theta }_1,\dots ,{\theta }_N]}^T$ and $\tilde{\mathit{\omega }}={[{\tilde{\omega }}_1,\dots ,{\tilde{\omega }}_N]}^T$.

We now search for a phase-locked solution of Eq. (7), $\dot{\mathit{\theta }}=\Omega 1$, where Ω represents the collective frequency of the networks. To find $\mathit{\omega }$ and Ω, we require the Moore-Penrose pseudoinverse $L^{†}$ that satisfies $L{L}^{†}L=L$ and $L^{†}L{L}^{†}=L^{†}$.³⁹ In the case of directed networks where $L^T\ne L$, the formulation for the pseudoinverse requires the singular value decomposition of L

$$L=U\Sigma V^T,$$ (8)

which is defined by the set of 2N equations

$$L{\mathit{v}}^i={\sigma }_i{\mathit{u}}^i,\hspace{1em}{L}^T{\mathit{u}}^i={\sigma }_i{\mathit{v}}^i.$$ (9)

In particular, the left and right singular vectors ${\mathit{u}}^i$ and ${\mathit{v}}^i$ populate the columns of U and V, respectively, and the singular values σ_i populate the diagonal matrix $\Sigma =\text{diag}({\sigma }_1,\dots ,{\sigma }_N)$. The singular vectors are normalized such that each set ${\left\{{\mathit{u}}^i\right\}}_{i=1}^N$ and ${\left\{{\mathit{v}}^i\right\}}_{i=1}^N$ forms an orthonormal basis for ${ℝ}^N$ and U and V are orthogonal matrices. Importantly, the singular values σ_i are all real and non-negative.⁴⁰ Since each row of L sums to zero, the first singular value ${\sigma }^1$ is precisely zero and corresponds to right singular vector ${\mathit{v}}^1={[1,\dots ,1]}^T$. If the network is strongly connected, i.e., any node can be reached from any other node, then all other singular values are positive and can be ordered $0={\sigma }_1<{\sigma }_2\le \dots \le {\sigma }_N$. Finally, defining ${\Sigma }^{†}=\text{diag}(0,{\sigma }_2^{-1},\dots ,{\sigma }_N^{-1})$, the pseudoinverse $L^{†}$ is given by

$$L^{†}=V{\Sigma }^{†}{U}^T=\sum _{j=2}^N\frac{{\mathit{v}}^j{\mathit{u}}^{\mathit{j}\mathit{T}}}{{\sigma }_j}.$$ (10)

The formulation of the pseudoinverse $L^{†}$ in terms of the singular value decomposition in the case of a directed network has a number of implications on the collective dynamics of networks. First, the collective frequency Ω of the synchronized population is not necessarily equal to the mean $⟨\tilde{\omega }⟩$, but rather is given by a weighted average of the entries of $\tilde{\mathit{\omega }}$.³¹ Specifically, the weights are given by the entries of the first left singular vector ${\mathit{u}}^1$

$$\Omega =\frac{⟨{\mathit{u}}^1,\tilde{\mathit{\omega }}⟩}{⟨{\mathit{u}}^1,\mathbf{1}⟩}=\frac{{\sum }_i{u}_i^1{\tilde{\omega }}_i}{{\sum }_i{u}_i^1},$$ (11)

where $⟨\mathit{x},\mathit{y}⟩={\mathit{x}}^T\mathit{y}={\sum }_i{x}_i{y}_i$ denotes the inner product. Returning to the dynamics of Eq. (7), we enter the rotating frame ${\theta }_i\mapsto {\theta }_i+\Omega t$ and apply the pseudoinverse $L^{†}$ to find the steady-state solution,

$${\mathit{\theta }}^{*}=\frac{L^{†}\tilde{\mathit{\omega }}}{KH\prime \left(0\right)}.$$ (12)

To evaluate the order parameter r for a state given by Eq. (12), we note that with a suitable shift in initial conditions the mean phase ψ of the population can be set to zero. Thus, the order parameter can be expanded to

$$r\approx 1-\frac{\Vert {\mathit{\theta }}^{*}{\Vert }^2}{2N},$$ (13)

where $\Vert ·\Vert$ denotes the Euclidean norm. Finally, noting that the squared norm can be evaluated by $\Vert x{\Vert }^2=⟨x,x⟩$, we arrive at

$$r\approx 1-\frac{J(\tilde{\mathit{\omega }},L)}{2K^2{H}^{\prime 2}\left(0\right)},$$ (14)

where $J(·,·)$ is the generalized synchrony alignment function defined as

$$J(\tilde{\mathit{\omega }},L)=\frac{1}{N}\sum _{j=2}^N\frac{{⟨{\mathit{u}}^j,\tilde{\mathit{\omega }}⟩}^2}{{\sigma }_j^2}.$$ (15)

Equations (14) and (15) serve as an objective measure of the degree of synchronization for a given network. Importantly, Eq. (14) implies that maximizing the order parameter r is equivalent to minimizing the generalized SAF, thus providing a framework for optimizing the synchronization properties of a directed network. Specifically, the SAF encodes the interplay between the local oscillator dynamics $\tilde{\mathit{\omega }}$ and the network structure, here represented by the left singular vectors ${\mathit{u}}^j$ and singular values σ_j, and can be used to optimize synchronization analytically and/or algorithmically, depending on the given constraints. In Section III, we will demonstrate the utility of the generalized SAF in optimizing synchronization in directed networks for several classes of constraints.

Before demonstrating the utility of the generalized SAF via concrete examples, a few remarks are in order. The SAF $J(\tilde{\mathit{\omega }},L)$ for a directed network given by Eq. (15) represents the generalization from the undirected case. The analysis presented here for directed networks includes undirected networks as a special case: For undirected networks where $L^T=L$, the singular values are precisely the eigenvalues of L, and the left and right singular vectors become equal and correspond to the eigenvectors of L. In the directed case, the left singular vectors and singular values play equivalent roles as the eigenvectors and eigenvalues in the undirected case; they determine the alignment of the local oscillator dynamics $\tilde{\mathit{\omega }}$ with the network structure. We note that the effective natural frequencies $\tilde{\mathit{\omega }}$ depend not only on the natural frequencies $\mathit{\omega }$ but also on the coupling strength K, nodal in-degree ${\mathit{k}}^{\text{in}}$, and coupling function via H(0). Therefore, optimization of synchronization using the SAF can generally depend on the coupling strength. However, if the oscillator coupling is not frustrated, i.e., $H(0)=0$, then the system simplifies, $\tilde{\mathit{\omega }}=\mathit{\omega }$, and optimization via the SAF is independent of coupling strength.

III. OPTIMIZING NETWORK SYNCHRONIZATION

We now demonstrate with several examples the utility of the generalized SAF for optimizing the synchronization properties of directed networks. We will study three classes of constrained optimization problems: (i) allocation of natural frequencies given a fixed network (Sec. III A); (ii) arrangement of a set of pre-chosen natural frequencies given a fixed network (Sec. III B); and (iii) construction of a network for a set of pre-chosen natural frequencies (Sec. III C). In all cases, we will consider natural frequency vectors with mean zero, $⟨\omega ⟩=\frac{1}{N}{\sum }_{j=1}^N{\omega }_i=0$, and fixed, nonzero standard deviation, ${\sigma }^2=\frac{1}{N}{\sum }_{j=1}^N{({\omega }_i-⟨\omega ⟩)}^2>0$, to avoid a trivial solution.

For all numerical experiments, we will consider two network models: Erdős-Rényi (ER) random networks⁴⁵ that are constructed such that a directed link $j\to i$ exists with probability p and networks with scale-free (SF) degree distributions $P(k^{\text{in}})\propto {(k^{\text{in}})}^{-\gamma },\hspace{0.17em}P(k^{\text{out}})\propto {(k^{\text{out}})}^{-\gamma }$ with enforced minimum degree $k^{\text{in}},k^{\text{out}}\ge k_0$ constructed using the configuration model.⁴⁶ The mean degree of ER and SF networks is given by $⟨k^{\text{in}}⟩=⟨k^{\text{out}}⟩=p(N-1)$ and $(\gamma -1)k_0/(\gamma -2)$, respectively. Furthermore, we will consider as an illustrative example Kuramoto-type coupling, i.e., $H(\theta )=\text{sin}(\theta )$. Note that in this case, the coupling is not frustrated [i.e., $H(0)=0$] so that the effective natural frequencies of the linearized system are precisely the natural frequencies, $\tilde{\mathit{\omega }}=\mathit{\omega }$ [see Eq. (4)]. One benefit of this choice is that because $\tilde{\mathit{\omega }}$ is independent of the coupling strength K, so is the optimization of the SAF. However, we stress that our results can be applied from other choices of coupling function, for instance, Sakaguchi-Kuramoto coupling⁴¹ as well as coupling with higher harmonics.^42–44

A. Oscillator allocation

We consider first the case of oscillator allocation. We assume that an underlying network structure is given, and we can freely choose the natural frequency ω_i at each node. In this case, the optimal choice of $\mathit{\omega }$ can be found analytically. By considering the expansion $\mathit{\omega }={\sum }_{j=1}^N{\alpha }_j{\mathit{u}}^j$ and substituting it into Eq. (15), we find that $J(\mathit{\omega },L)$ can be minimized by placing as much weight as possible into the N-th singular vector coefficient α_N. However, since the collection of left singular vectors ${\mathit{u}}^j$ tends to have non-zero mean, an additional shift is required to ensure that $⟨\omega ⟩=0$. It is straightforward to show that the optimal choice of natural frequencies is precisely

$$\mathit{\omega }=\pm \sigma \sqrt{N}\left({\mathit{u}}^N-\frac{⟨u^N⟩}{⟨u^1⟩}{\mathit{u}}^1\right)/\sqrt{1-\frac{{⟨u^N⟩}^2}{{⟨u^1⟩}^2}},$$ (16)

where $⟨u^l⟩=\frac{1}{N}{\sum }_{j=1}^N{u}_j^l$ represents the mean entry of the $l\text{th}$ left singular vector. Thus, the optimal choice of frequencies is proportional to the dominant left singular vector ${\mathit{u}}^N$ with a shift to retain a mean of zero.

We now demonstrate the effectiveness of the optimal solution given by Eq. (16) in comparison to a random allocation of natural frequencies. Specifically, we consider both ER and SF networks of size N = 100 with mean degree $⟨k⟩=4$ (for SF networks, we let γ = 3) with (i) the optimal choice given in Eq. (16) with σ = 1 and (ii) a set of natural frequencies drawn randomly from the standard normal distribution. We consider 50 networks each of type ER and SF, and for each network we simulate Eq. (1) over a range of coupling strengths for both the optimal and random set of natural frequencies. In Fig. 1(a), we plot the resulting synchronization profiles r vs K (averaged over the 50 network realizations), indicating optimal and random natural frequency results in blue circles and red triangles, respectively. We denote results using ER and SF networks with filled and unfilled markers, respectively. The optimal choice of natural frequencies clearly outperforms the random allocation over the entire range of coupling strength, leading to strong synchronization (i.e., $r\approx 1$) even for very small K.

FIG. 1.

Optimal synchronization of directed networks: Synchronization profiles r vs K for (a) oscillator allocation, (b) oscillator arrangement, and (c) network construction for normally distributed natural frequencies. Erdős-Rényi (ER) and scale-free (SF) networks of size N = 1000 in (a) and (b) and N = 500 in (c) with mean degree $⟨k⟩=4$ and γ = 3 are considered. Results represent an average over 50 network realizations of each network type. Results for the optimal and random networks are plotted in blue circles and red triangles, respectively, and ER and SF networks are indicated with filled and unfilled markers, respectively.

B. Oscillator arrangement

Next, we consider oscillator arrangement. Here, we assume that an underlying network structure is given, but rather than choosing natural frequencies freely (i.e., as in Sec. III A), we are given a set of pre-chosen natural frequencies $\left\{{\omega }_i\right\}$ that must be arranged on the network. In general, the optimal solution depends on the particular network structure and set of natural frequencies given, making analytical treatment impossible. Specifically, since there are $N!$ possible arrangements, it is infeasible to conduct an exhaustive search for the best arrangement even for a moderately sized network. We thus develop a computationally feasible solution for minimizing the SAF.

We propose here a simple greedy algorithm to produce an approximation of the optimal solution. Initially, we arrange the set of natural frequencies (here chosen from the standard normal distribution) randomly on the network. Then, in each step, we propose a switch of a single pair of oscillators, ${\omega }_i\leftrightarrow {\omega }_j$, obtain a new frequency vector $\mathit{\omega }\prime$, and compute the new SAF $J(\mathit{\omega }\prime ,L)$. If the new SAF is less than the previous SAF, $J(\mathit{\omega }\prime ,L)<J(\mathit{\omega },L)$, then the switch is accepted, and otherwise it is rejected. This process is repeated until a chosen number S of switches is proposed. We note that the initial arrangement of oscillators needs not be random, and in fact, we can start closer to an optimal arrangement by setting an initial arrangement that aligns closely with one or more of the dominant left singular vectors of L, saving a significant number of proposed switches.

We demonstrate the power of this approach by simulating the dynamics of Eq. (1) on ER and SF networks before and after the natural frequencies are rearranged using the algorithm above. We consider again networks of size N = 1000 with mean degree $⟨k⟩=4$ and γ = 3, first arranging normally distributed natural frequencies randomly, then rearranging them with $S={10}^5$ proposed switches. We plot the resulting synchronization profiles in Fig. 1(b), indicating the random and rearranged natural frequency results with blue circles and red triangles, respectively. We again denote ER and SF networks with filled and unfilled markers. As in the case of oscillator allocation, the system with rearranged natural frequencies significantly outperforms the system with randomly arranged natural frequencies. Comparing to the results of optimal allocation that are shown in Fig. 1(a), in the case of pre-chosen natural frequencies the abrupt transition to synchronization occurs at a larger coupling strength (i.e., $K\approx 0.3$) and the order parameters are generally smaller. Nevertheless, as in the case of optimal arrangement, the networks with optimally arranged frequencies significantly outperform those with randomly arranged frequencies.

C. Network construction

We now consider the problem of network construction. We assume that a set of pre-chosen natural frequencies is given, and a network must be build to best synchronize the oscillators. We will assume that a fixed number M of directed links can be made. As in the case of oscillator arrangement, the problem sensitively depends on the particular set of natural frequencies given. Furthermore, there is a combinatorially large number of possible networks without copied or self links that can be built, $(\begin{array}{c}N(N-1)\\ M\end{array})$, which makes a full search unfeasible. Thus, we again proceed algorithmically.

We implement an accept-reject algorithm that is initialized with a random network. We begin by constructing a random network with M links around the pre-chosen natural frequencies, which we ensure to be strongly connected. Next, we propose link rewiring: uniformly at random, we select a link $j\to i$ to delete, and we replace it with a new link between the two previously disconnected nodes $j\prime \to i\prime$, which are also selected uniformly at random. This yields a new directed Laplacian matrix, $L\prime$, and new SAF, $J(\mathit{\omega },L\prime )$. If the new SAF is less than the previous SAF, $J(\mathit{\omega },L\prime )<J(\mathit{\omega },L)$, then the link replacement is accepted, and otherwise it is rejected. As before, this process is repeated until S proposed rewirings are considered.

We highlight the effectiveness of this method by simulating the dynamics of Eq. (1) on networks before and after the iterative rewiring process. We initialize the rewiring algorithm with both ER and SF networks of size N = 500 and mean degree $⟨k⟩=4$ and γ = 3 with normally distributed frequencies and propose $S={10}^4$ link rewirings. We plot the resulting synchronization profiles in Fig. 1(c), indicating the random and rewired network results with blue circles and red triangles, respectively. We denote the results using initialized ER and SF networks with filled and unfilled markers, respectively. As expected, the networks that are rewired to minimize the SAF outperform the initial networks. Similar to the oscillator arrangement case, the transition to synchronization occurs at a larger coupling strength [i.e., $K\approx 0.3$] than in the oscillator allocation case, and it remains abrupt. We note that since the rewiring algorithm is greedy, in principle, the initial network structure may have an effect on the final network. However, we find that initializing the algorithm with ER and SF networks yields very similar results, which suggests that the network initialization does not have a significant effect on the outcome of the rewiring process.

IV. EFFECT OF STRUCTURAL PROPERTIES OF DIRECTED NETWORKS

Having demonstrated the effectiveness of the generalized SAF for optimizing a network's synchronization properties, we next investigate the role of directed network structure in the optimization of synchronization. In particular, we consider two general questions. First, what are the effects of various directed network structures on optimal synchronization? Second, what are the effects of optimal synchronization on directed network structures? We address these questions below in Secs. IV A and IV B, respectively.

A. Degree assortativity

We begin by investigating the effects that directed network properties have on optimal synchronization. Perhaps the property that differentiates directed from undirected networks is the characterization of each node by two degrees rather than one (specifically, the in- and out-degrees $k_i^{\text{in}}$ and $k_i^{\text{out}}$ for a given node i). Thus, a given directed network can easily be classified with a particular degree assortativity, measuring the correlation between in- and out-degrees at each node in the network. This can be measured with the degree assortativity coefficient,⁴⁷ i.e., the Pearson correlation coefficient, defined as

$$c_{k^{\text{in}},k^{\text{out}}}=\frac{{\sum }_i(k_i^{\text{in}}-⟨k⟩)(k_i^{\text{out}}-⟨k⟩)}{\sqrt{\left[{\sum }_i{(k_i^{\text{in}}-⟨k⟩)}^2\right]\left[{\sum }_i{(k_i^{\text{out}}-⟨k⟩)}^2\right]}},$$ (17)

where $-1\le c_{k^{\text{in}},k^{\text{out}}}\le 1$ such that $c_{k^{\text{in}},k^{\text{out}}}>0$ and $c_{k^{\text{in}},k^{\text{out}}}<0$ indicate a positive and negative correlation, respectively, and $c_{k^{\text{in}},k^{\text{out}}}\approx 0$ indicates no correlation. The results presented in Sec. III and Fig. 1 used random networks with no correlations between in- and out-degrees. This begs the question: What is the effect of degree assortativity on optimal synchronization?

We investigate the effects of assortative and disassortative degree correlations on optimal synchronization in the cases of oscillator allocation and arrangement. In particular, given the sequence of in- and out-degrees for a particular network, we construct two new networks using in- and out-degree pairs set to maximize and minimize the degree assortativity. Maximum assortativity is obtained by matching the maximum in-degree with the maximum out degree, etc., and maximum disassortativity is obtained by matching the maximum in-degree with the minimum out-degree, etc. Beginning with ER networks of size N = 1000 with mean degree $⟨k⟩=4$, we plot the synchronization profiles r vs K of the original network (dashed curves) and its assortative (blue circles) and disassortative (red triangles) rewirings after optimally allocating and arranging frequencies in Figs. 2(a) and 2(b), respectively. The results represent an average over 50 network realizations, and in the case of oscillator arrangement, the frequencies are normally distributed. We repeat these simulations on SF networks of size N = 1000 with $⟨k⟩=4$ and γ = 3, plotting the results from optimal allocation and arrangement in Figs. 3(a) and 3(b), respectively.

FIG. 2.

Effect of degree assortativity (ER networks): Synchronization profiles r vs K for optimally (a) allocated and (b) arranged frequencies on ER networks with N = 1000, $⟨k⟩=4$ (dashed curves) as well as these networks rewired to be assortative with $c_{k^{\text{in}},k^{\text{out}}}=0.99$ (blue circles) and disassortative with $c_{k^{\text{in}},k^{\text{out}}}=-0.92$ (red triangles). Results represent an average over 50 network realizations, and the frequencies are normally distributed for (b). Uncorrelated results are plotted with a dashed curve.

FIG. 3.

Effect of degree assortativity (SF networks): Synchronization profiles r vs K for optimally (a) allocated and (b) arranged frequencies on SF networks with N = 1000, $⟨k⟩=4$, γ = 3 (dashed curves) as well as these networks rewired to be assortative with $c_{k^{\text{in}},k^{\text{out}}}=0.94$ (blue circles) and disassortative with $c_{k^{\text{in}},k^{\text{out}}}=-0.19$ (red triangles). Results represent an average over 50 network realizations, and the frequencies are normally distributed for (b). Uncorrelated results are plotted with a dashed curve.

We note first that, while there is a difference between optimal synchronization in assortative and disassortative networks, the optimization methods presented above are effective in both cases. Interestingly, we find that the effect of degree assortativity on synchrony optimization differs between the cases of optimal allocation and optimal arrangement. In particular, we observe that in both ER and SF networks when oscillators are optimally allocated, disassortative networks tend to outperform assortative networks [see Figs. 2(a) and 3(a)]. However, when oscillators are optimally arranged, assortative networks tend to outperform disassortative networks [see Figs. 2(b) and 3(b)]. Thus, we conclude that positive/negative degree assortativity in directed networks does not necessarily imply improved or diminished synchronization properties and depends on the nature of the constraints on the optimization problem. However, we do point out that these tendencies do not appear to depend much on network heterogeneity, since the results for ER and SF networks are similar.

B. Evolution of network structure

Next, we investigate the effect that optimization has on a given directed network's structure. In particular, we consider the case of network rewiring and study the evolution of the degree distribution P(k) for both in- and out degrees $k^{\text{in}}$ and $k^{\text{out}}$. We proceed as follows. We initialize an ER network of size N = 500 with $⟨k⟩=4$ and a set of pre-chosen natural frequencies. After noting the initial degree distributions of $k^{\text{in}}$ and $k^{\text{out}}$, we apply the rewiring algorithm described in Sec. III C for $S=5\times {10}^4$ proposed rewirings and study the resulting degree distributions of $k^{\text{in}}$ and $k^{\text{out}}$. Importantly, we consider two classes of natural frequencies. First, we consider frequencies drawn from the standard normal distribution (as used above), representing a homogeneous collection of frequencies. Second, we consider frequencies drawn from a much broader, heavy-tailed distribution described by

$$P(\omega )=\{\begin{array}{cc}(\beta -1)/2\beta {\omega }_0\hfill & \text{if}\hspace{0.17em}|\omega |<{\omega }_0,\\ ((\beta -1){\omega }_0^{\beta -1}/2\beta {\omega }_0)|\omega {|}^{-\beta }& \text{if}\hspace{0.17em}|\omega |\ge {\omega }_0,\end{array}$$ (18)

which represents a symmetric (continuous and piecewise-smooth) distribution on $ℝ$ that is uniform for $|\omega |\le {\omega }_0$ and has a power-law decay with exponent β for $|\omega |>{\omega }_0$. We choose $\beta =2.5$ and ${\omega }_0=1$.

We present results for this experiment in Fig. 4, plotting the degree distributions $P(k^{\text{in}})$ and $P(k^{\text{out}})$ for the initial networks in panel (a) in blue circles and red triangles, respectively, and the resulting distributions after rewiring for optimal synchronization for the cases of normal and heavy-tailed frequencies in panels (b) and (c), respectively. Results represent an average over 50 network realizations. We emphasize that the initial networks used for the cases of normal vs heavy-tailed frequencies are identical, and thus the difference in the distributions after rewiring for optimality is due solely to the effect that the different frequency distributions have on the rewiring process. Noting that results are plotted in log-log format, both the in- and out-degree distributions of the initial networks are thin and in fact, they remain thin after rewiring for the case of normal frequencies. However, in the case of heavy-tailed frequencies, we observe that while the out-degree distribution remains thin, the in-degree distribution becomes very wide, indicating the emergence of nodes with very large in-degree compared with the rest of the network, or hubs.

FIG. 4.

Effect of optimization on directed network structure: Degree distributions P(k) of networks (a) before and (b) and (c) after rewiring for optimal synchronization. In (b), the natural frequencies are normally distributed, while in (c) the frequency distribution is heavy-tailed as defined in Eq. (18). The initial network is ER with N = 500 and $⟨k⟩=4$. Results represent an average over 50 network realizations.

These results shed some light on the role of directedness of networks in optimal synchronization. In the case of undirected networks, we found¹³ in a similar rewiring experiment that the heterogeneity of the degree distribution of an optimized network roughly matches the heterogeneity of the frequency distribution—heavy-tailed frequency distributions give rise to heavy-tailed degree distributions, and thin frequency distributions give rise to thin degree distributions. The results for directed networks provide an interesting contrast to this phenomenon. In particular, the heterogeneity of the in-degree distribution matches the heterogeneity of the frequency distribution, but no such relationship is observed for the out-degree distribution. This suggests that in the case of directed networks, in-degrees play an important role in optimizing synchronization properties, while out-degrees are much less significant. As we will see below, the role of in- and out-degrees extends to the correlations we observe between the degrees and frequencies.

V. EFFECT OF INTERPLAY BETWEEN STRUCTURAL AND DYNAMICAL PROPERTIES

Having investigated the effects of directed network structure on optimal synchronization, and vice versa, we now consider the relationship between the structural and dynamical properties of optimal networks. In particular, what relationships between dynamical and structural properties promote synchronization in directed networks? In our previous work,¹³ we observed two general properties that are common to undirected synchrony-optimized networks: (i) a strong positive correlation between an oscillator's nodal degree and its natural frequency and (ii) a strong negative correlation between an oscillator's natural frequency and the average natural frequencies of its network neighbors. These results are consistent with those found to promote global synchronization in other studies;^8–12 however in each of these cases, only undirected networks were considered. This leaves a significant gap in our understanding of which directed network properties promote synchronization.

In Sec. IV B, we observed that in-degrees play a particularly important role in synchrony optimization compared with the out-degrees. This begs the question: How does the relationship between in-degrees and frequencies compare to the relationship between out-degrees and frequencies? To address this question, we study the system properties of synchrony-optimized networks resulting from the optimization of the general SAF. In Sec. V A, we study correlations between oscillators' natural frequencies and their nodal in- and out-degrees. In Sec. V B, we study the correlations between the natural frequencies of neighboring oscillators.

A. Degree-frequency correlations

We begin our investigation by considering correlations between each oscillator's nodal in- and out-degree, $k_i^{\text{in}}$ and $k_i^{\text{out}}$, with their respective natural frequency in absolute value, $\left|{\omega }_i\right|$. We present the results for synchrony-optimized networks arising for the optimization problem of rearranging a set of pre-chosen frequencies on a given network (see Sec. III B); however, we emphasize that these results are in close agreement with our study of networks resulting for the other optimization problems discussed in Sec. III.

In Fig. 5, we show scatter plots of $\left|{\omega }_i\right|$ versus (a) $k_i^{\text{in}}$ and (b) $k_i^{\text{out}}$. Black dashed lines indicate the least squares line of best fit for each case. We show the results, for example, ER network of size N = 1000 with mean degree $⟨k⟩=4$ and natural frequencies that are drawn from the standard normal distribution. The natural frequencies are arranged on the network by following the algorithm described in Sec. III B with $S={10}^5$ proposed switches. By comparing Figs. 5(a) and 5(b), we observe contrasting relationships that natural frequencies have with in-degrees and out-degrees. In particular, a strong positive relationship is clear between natural frequencies and in-degrees, but no such relationship is clear between natural frequencies and out-degrees. To support this observation, we calculate the assortativity coefficient between the respective quantities,

$$c_{k,|\omega |}=\frac{{\sum }_i(k_i-⟨k⟩)\left(\right|{\omega }_i|-⟨|\omega |⟩)}{\sqrt{\left[{\sum }_i{(k_i-⟨k⟩)}^2\right]\left[{\sum }_i{\left(\right|{\omega }_i|-⟨|\omega |⟩)}^2\right]}}.$$ (19)

We find that in our particular example, $c_{k^{\text{in}},|\omega |}=0.8530$ and $c_{k^{\text{out}},|\omega |}=0.0726$.

FIG. 5.

Degree-frequency correlations: For an ER network of size N = 1000 with $⟨k⟩=4$ and normally distributed natural frequencies, the natural frequency $\left|{\omega }_i\right|$ in magnitude vs (a) the nodal in-degree $k_i^{\text{in}}$ and (b) nodal out-degree $k_i^{\text{out}}$. The least squares line of best fit is plotted in dashed black, and the Pearson correlation coefficients for the results in panels (a) and (b) are $c_{k^{\text{in}},|\omega |}=0.8530$ and $c_{k^{\text{out}},|\omega |}=0.0726$, respectively.

B. Frequency-frequency correlations

Next, we consider the relationship between natural frequencies of neighboring oscillators. We note that in the directed case, this relationship is more nuanced than in the undirected case, and therefore we consider the relationships between a given oscillator's natural frequency and the mean natural frequency of its neighbors (i) along in-coming links and (ii) along out-going links. To investigate these relationships, we define the mean neighboring frequencies, respectively, as

$${⟨\omega ⟩}_i^{\text{in}}=\frac{1}{k_i^{\text{in}}}\sum _{j=1}^N{A}_{\mathit{i}\mathit{j}}{\omega }_j,\hspace{1em}{⟨\omega ⟩}_i^{\text{out}}=\frac{1}{k_i^{\text{out}}}\sum _{j=1}^N{A}_{\mathit{j}\mathit{i}}{\omega }_j.$$ (20)

Physically, the mean in-frequency for oscillator i, ${⟨\omega ⟩}_i^{\text{in}}$, represents the mean frequency of all the oscillators that influence oscillator i, while the mean out-frequency for oscillator i, ${⟨\omega ⟩}_i^{\text{in}}$, represents the mean frequency of all the oscillators that are influenced by oscillator i.

Using the same network as in the previous example, we plot ${⟨\omega ⟩}_i^{\text{in}}$ and ${⟨\omega ⟩}_i^{\text{out}}$ in Figs. 6(a) and 6(b), respectively, against the natural frequency ω_i. In both panels, one can observe a negative relationship. However, the two cases differ in their structure—note the strong “bow-tie”-like structure in Fig. 6(a). This bow-tie structure is reminiscent of the frequency-frequency relationship that we previously observed for undirected networks.¹³ To quantify this structure, we count the fraction of pairs $({\omega }_i,{⟨\omega ⟩}_i)$ that fall into each quadrant, which we indicate in the respective corners of Figs. 6(a) and 6(b). This allows us to quantify the proportion of oscillators whose natural frequency shares their sign with the frequencies opposite incoming and outgoing links. Approximately 20% of the natural frequencies share a sign with their respective mean in-frequency, while 30% share a sign with their respective mean out-frequency. This finding suggests that the relationship between each natural frequency and its respective mean in-frequency has a more significant effect on synchronization than the relationship with its respective mean out-frequency.

FIG. 6.

Frequency-frequency correlations: For an ER network of size N = 1000 with $⟨k⟩=4$ and normally distributed natural frequencies, the mean (a) in-frequency ${⟨\omega ⟩}_i^{\text{in}}$ and (b) out-frequency ${⟨\omega ⟩}_i^{\text{out}}$ vs the oscillator's natural frequency ω_i. Note the “bow-tie” structure in panel (a). Numbers in each corner indicate the fraction of $({\omega }_i,{⟨\omega ⟩}_i)$ pairs that fall in each respective quadrant.

C. Evolution of correlations

Finally, we investigate the evolution of a network's properties during the optimization process. Again, we restrict our attention to oscillator arrangement, as discussed in Sec. III B, and we thus consider the evolution of a network through a process of switching the natural frequencies. Throughout this evolution, we will consider the correlation between both in- and out-degrees and natural frequencies, measured by the assortativity coefficient in Eq. (19), as well as frequency-frequency correlations measured by

$$c_{\omega ,\omega }=\frac{{\sum }_{i,j}{A}_{\mathit{i}\mathit{j}}({\omega }_i-⟨\omega ⟩)({\omega }_j-⟨\omega ⟩)}{\sqrt{\left({\sum }_{i,j}{A}_{\mathit{i}\mathit{j}}{({\omega }_i-⟨\omega ⟩)}^2\right)\left({\sum }_{i,j}{A}_{\mathit{i}\mathit{j}}{({\omega }_j-⟨\omega ⟩)}^2\right)}},$$ (21)

which in this case quantifies the assortativity between neighboring natural frequencies.

In Fig. 7, we plot the results obtained from optimally rearranging natural frequencies drawn from a standard normal distribution on an ER network of size N = 1000 with mean degree $⟨k⟩=4$ over the course of $S=2\times {10}^4$ proposed switches. For context, we plot the evolution of the SAF $J(\mathit{\omega },L)$ in Fig. 7(a). In Fig. 7(b), we plot the correlation coefficients $c_{k^{\text{in}},|\omega |},\hspace{0.17em}{c}_{k^{\text{out}},|\omega |}$, and $c_{\omega ,\omega }$ in blue, red, and green, respectively. (We note that these also align from top to bottom, respectively.) Results represent an average over 50 network realizations. For comparison, results for a single realization are plotted as gray curves.

FIG. 7.

Evolution of correlations: Averaged over 50 ER networks of size N = 1000 with $⟨k⟩=4$ and normally distributed natural frequencies, the evolution of (a) the SAF $J(\mathit{\omega },L)$ and (b) the correlation coefficients $c_{k^{\text{in}},|\omega |}$ (top, blue), $c_{k^{\text{out}},|\omega |}$ (middle, red), and $c_{\omega ,\omega }$ (bottom, green) as natural frequency switches are proposed. Gray curves indicate results for a single realization.

Our main observation is that as more switches are proposed, we see a rapid shift in the SAF, $c_{k^{\text{in}},|\omega |}$, and $c_{\omega ,\omega }$, but only a slight shift in $c_{k^{\text{out}},|\omega |}$. As the network's synchronization properties improve, the correlation coefficients $c_{k^{\text{in}},|\omega |}$ and $c_{\omega ,\omega }$ quickly saturate to approximately 0.7 and −0.3, respectively. In contrast, $c_{k^{\text{out}},|\omega |}$ remains small in comparison but is on average positive. These findings are consistent with the numerical results presented in Secs. V A and V B on the degree-frequency correlations and frequency-frequency correlations observed in optimally rewired networks.

VI. DISCUSSION

In this paper, we have developed a framework for optimizing the synchronization properties of directed networks of coupled oscillators. Our results can be regarded as an extension to the previous work that focused on the case of undirected networks.¹³ Our main theoretical result is the derivation of the generalized synchrony alignment function given by Eq. (15), which we have shown can be used to systematically optimize the synchronization properties of networks under several constraints, including oscillator allocation (Sec. III A), oscillator arrangement (Sec. III B), and network construction (Sec. III C). We emphasize that this approach is efficient, does not require large-scale simulation of dynamics [i.e., either Eq. (1) or Eq. (3)], and is based on an objective measure of synchronization (i.e., a perturbation analysis of the Kuramoto order parameter) and not heuristics.

The generalized SAF approach presented here shows that the synchronization properties of a general directed network depend on the alignments of the natural frequency vector with the left singular vectors of the Laplacian, weighted appropriately by the left singular values. In particular, stronger synchronization is attained as the natural frequency vector becomes aligned with dominant singular vectors, i.e., those associated with larger singular values. This is a natural generalization from the undirected case, where it was found that stronger alignments of the natural frequency vector with the more dominant eigenvectors of the Laplacian promote synchronization. We emphasize that our approach is designed to maximize the Kuramoto order parameter in the regime where strong synchronization may occur, but point out that even for relatively small coupling strengths the approach works remarkably well. We also point out that the method is not designed to minimize the critical coupling strength associated to the onset of synchronization. Furthermore, we hypothesize that our results may be applied to better understand the effect that failures and other perturbations have on synchronization in complex networks.²⁰

To provide further insight into the mechanisms that enhance synchronization and the role of directedness, we investigated the effects that directed network properties have on optimal synchronization and the effects that optimization has on directed network structures. In undirected networks, it has been observed that after rewiring for optimal synchronization, the heterogeneity of the degree distribution roughly matches the heterogeneity of the frequency distribution. We have found a contrasting phenomenon for directed networks: after rewiring for optimal synchronization, the heterogeneity of the in-degree distribution roughly matches the heterogeneity of the frequency distribution; no such relationship was found for the out-degrees. We have also studied the relationship between structural and dynamical properties of optimized networks. In the undirected case, it is well known that a positive correlation between degrees and natural frequencies and a negative correlation between neighboring oscillators' frequencies promote synchronization. We have shown that these relationships become more nuanced in the directed case; synchronization is promoted by a positive correlation between the nodal in-degree of an oscillator and its respective natural frequency, while the relationship between the nodal out-degree of an oscillator and its respective natural frequency has comparatively little effect. We also studied correlations between the natural frequencies of neighboring oscillators and found that directed synchrony-optimized networks display a stronger relationship between each natural frequency and the mean natural frequency of its neighbors along in-coming links than its neighbors along out-going links. Taken together, these results indicate that in-degrees play a significant role in the synchronization properties of directed networks, while out-degrees are much less significant.