Pattern Formation over Multigraphs

Abstract

Two of the most common pattern formation mechanisms are Turing-patterning in reaction-diffusion systems and lateral inhibition of neighboring cells. In this paper, we introduce a broad dynamical model of interconnected modules to study the emergence of patterns, with the above mentioned two mechanisms as special cases. Our results do not restrict the number of modules or their complexity, allow multiple layers of communication channels with possibly different interconnection structure, and do not assume symmetric connections between two connected modules. Leveraging only the static input/output properties of the subsystems and the spectral properties of the interconnection matrices, we characterize the stability of the homogeneous fixed points as well as sufficient conditions for the emergence of spatially non-homogeneous patterns. To obtain these results, we rely on properties of the graphs together with tools from monotone systems theory. As application examples, we consider patterning in neural networks, in reaction-diffusion systems, and contagion processes over random graphs.

1 Introduction

Spatial pattern formation plays a fundamental role in the development of complex self-organized systems, such as multi-cellular organisms [1], [2]. The vast majority of theoretical results about the emergence of patterns focus on diffusion-driven instabilities, the so-called Turing-patterning in both biological [3], [4], [5], [6], [7] and abiological systems [8], [9]. However, patterning is also faciliated by mechanisms without any diffusible molecules. For instance, Turing patterns can appear in systems made of immobile agents as a result of differential growth [10], or alternatively, in the case of lateral inhibition in the Notch pathway where neighboring cells inhibit each other from converging on the same fate [1], [11], [12], [13], [14]. Thus, there is growing attention targeted at understanding pattern formation mechanisms other than Turing-patterning.

Studies of patterning either focus on the continuous case with partial differential equations, or consider network analogues: interconnected dynamical systems where nodes represent systems and edges stand for interconnections (e.g., ecological metapopulations [15], [16], spreading of infections over transportation networks [17], [18], [19], diffusively coupled chemical reactors or cells [11], [20], [21], [22], [23]). Since the high-dimension of the resulting problem renders the analysis difficult, studies so far mainly considered small networks comprising only a few nodes, and resorted to numerical simulations in the case of large-scale networks [11], [13], [22], [23], [24].

To characterize pattern formation in large-scale networks, we view the network as the interconnection of input/output models [25], [26]. Inputs and outputs correspond to, for instance, the concentration of species used for communication among cells, and the interconnection structure is encoded with directed and weighted graphs P⁽¹⁾, …, P^(m), one for each layer of communication channels. Nodes represent modules (e.g., cells in reaction-diffusion systems), edges stand for connection between two nodes, and weights represent the strength of this connection. In addition to studying the stability of the homogeneous fixed points, this formulation allows us to characterize sufficient conditions for the emergence of spatially non-homogeneous patterns irrespective of network complexity.

Recent efforts focusing on pattern formation over directed graphs [27], [28] and multigraphs [28], [29], [30], [31] share a common characteristic: the matrices describing coupling among nodes are the Laplacian operators associated with the network structures, yielding P^(k) with zero row-sum. While this assumption is appropriate in the context of diffusion-driven instabilities, it does not permit the study of pattern formation propelled by mechanisms without diffusible molecules, e.g., in the case of lateral inhibition P^(k) is row-stochastic (studied in [25], [26] when P^(k) are identical and symmetric).

Considering the above, the novelty of this paper is that it focuses on pattern formation over large-scale multigraphs, with directed edges, and without restriction on the row-sum of P^(k). Therefore, our results (1) apply to networks where communication among nodes occurs over multiple layers of channels, with possibly different interconnection structures (e.g., two nodes can be connected in one layer and disconnected in another); (2) allow for asymmetric communication among nodes (e.g., one node can have an effect on another without any effect from the other); and (3) capture a wide array of mechanisms leading to patterning (diffusion-driven instability and lateral inhibition both emerge as special cases). Additionally, while the above studies (except for [26]) only focus on the instability of the homogeneous fixed points, we also reveal sufficient conditions for the emergence of non-homogeneous patterns and characterize their location relative to the homogeneous fixed points. Although our main motivation is understanding pattern formation in cellular systems, our results characterize the emergence of patterns in networked systems over directed multigraphs without restrictions to biological systems.

This paper is organized as follows. We first present the mathematical model considered for studying the emergence of patterns over directed multigraphs, together with the main questions of the paper and with the notation and technical assumptions. After focusing on the existence of the homogeneous fixed points, we consider pattern formation over networks with a single layer of communication channels, then generalize the results to the case of multigraphs with multiple layers of communication channels. Finally, we illustrate the implications of the results through application examples considering neural networks, reaction-diffusion systems, and contagion processes over random graphs.

2 Mathematical Model and Notation

Consider a network of identical dynamical systems i = 1, 2, …, N, each described by the model

$$\begin{gathered} {\stackrel{.}{x}}_i=f(x_i,u_i), \\ {y}_i=h(x_i), \end{gathered}$$ (1)

where x_i ∈ ℝⁿ denotes the state of system i, and u_i ∈ ℝ^m and y_i ∈ ℝ^m represent the input and output of this system, respectively, where $u_i={(\begin{array}{ccc}{u}_i^{(1)}& \dots & u_i^{(m)}\end{array})}^T$ and $y_i={(\begin{array}{ccc}{y}_i^{(1)}& \dots & y_i^{(m)}\end{array})}^T$. Introduce x, u and y as the concatenations of x_i, u_i and y_i for i = 1, 2, …, N, respectively, and define $u^{(k)}:={(\begin{array}{ccc}{u}_1^{(k)}& \dots & u_N^{(k)}\end{array})}^T$ and $y^{(k)}:={(\begin{array}{ccc}{y}_1^{(k)}& \dots & y_N^{(k)}\end{array})}^T$.

We consider interactions among subsystems of the form

$$u^{(k)}=P^{(k)}{y}^{(k)}k=1,\dots ,m,$$ (2)

where the entry $p_{i,j}^{(k)}$ of the matrix P^(k) ∈ ℝ^N×N represents the strength of the effect of subsystem j on subsystem i through the k^th channel. Therefore, we represent the network by a set of m directed and connected graphs (V,P^(k)), where V and P^(k) denote the set of vertices and the weighted adjacency matrix (assumed to be irreducible), respectively.

In this paper, we study the fixed points of (1)–(2). When does the homogenous fixed points become unstable, setting the stage for patterning? When do spatially non-homogenous patterns emerge? By grouping modules that share the same fate, is it possible to reduce the complexity of the analysis?

While addressing these questions, we consider various subsets of the following main assumptions:

• (A1)both f(·, ·) and h(·) are continuously differentiable;
• (A2)for all u ∈ ℝ^m the set of equations 0 = f(x, u) has a solution denoted by x =: S(u), in which case we define T(u) := h(S(u));
• (A3) ${\frac{\partial f(x,u)}{\partial x}|}_{(S(u),u)}$ is Hurwitz for all u ∈ ℝ^m;
• (A4)the maps S : ℝ^m → ℝⁿ and T : ℝ^m → ℛ^m are continuously differentiable;
• (A5)T(·) is bounded and $\frac{\partial T(u)}{\partial u}$ is sign-stable (i.e., with T⁽ⁱ⁾(·) denoting the i^th entry of T(·) we have that $\frac{\partial T^{(i)}(u)}{\partial u^{(j)}}\le 0$ or $\frac{\partial T^{(i)}(u)}{\partial u^{(j)}}\ge 0$ for all u and i, j = 1, …, m);
• (A6)P^(k)1_N = p^(k)1_N for some p^(k) ∈ ℝ for k = 1, …, m (constant row-sum);
• (A7)P⁽¹⁾, …, P^(m) commute and are diagonalizable (when m > 1).

Throughout the paper, we drop the superscript when m = 1. Furthermore, let e_i denote the i^th unit vector and write M ⪯ 0 and M ⪰ 0 to denote that all entries of M are non-positive and non-negative, respectively. Finally, ρ(M) and s(M) denote the spectral radius and the largest real part of the eigenvalues of M, respectively, whereas diag(v) defines the diagonal matrix composed of the elements of the vector v.

3 Results

Before studying the emergence of patterns, we focus on the existence of homogeneous fixed points. To this end, we first present an input/output formulation for studying the fixed points of (1)–(2), which will be used throughout the paper.

Lemma 1

Assume that (A2) holds. If $u_i^{(k)}$ for i = 1, …, N and k = 1, …, m satisfy

$$\left(\begin{array}{c}{u}_1^{(k)}\\ ⋮\\ u_N^{(k)}\end{array}\right)=P^{(k)}\left(\begin{array}{c}{T}^{(k)}(u_1)\\ ⋮\\ T^{(k)}(u_N)\end{array}\right),$$ (3)

then x_i = S(u_i) is a fixed point of (1)–(2). Conversely, if S(·) in (A2) is unique and x is a fixed point of (1)–(2) then the corresponding u^(k) from (2) satisfies (3).

Proof

Follows from the definition of T(·) in (A2).

Lemma 2

Provided (A2), (A4), (A5) and (A6), ∃x₀ ∈ ℝⁿ such that x = 1_N ⊗ x₀ is a fixed point of (1)–(2).

Proof

It is sufficient to show that ∃u₀ ∈ ℝ^m such that $u^{(k)}={\mathbf{1}}_N(e_k^T{u}_0)$ satisfies (3), as then x_i = x₀ = S(u₀) is a fixed point of (1)–(2) from Lemma 1. If u₀ satisfies

$$\left(\begin{array}{c}{u}_0^{(1)}\\ ⋮\\ u_0^{(m)}\end{array}\right)=\left(\begin{array}{c}{p}^{(1)}{T}^{(1)}(u_0)\\ ⋮\\ p^{(m)}{T}^{(m)}(u_0)\end{array}\right)$$ (4)

with p^(k) ∈ ℝ from (A6) then $u^{(k)}={\mathbf{1}}_N(e_k^T{u}_0)$ satisfies (3). Therefore, in what follows we prove that ∃u₀ ∈ ℝ^m such that u₀ satisfies (4).

Since T^(k)(u_i) is bounded from (A5), we have |p^(k)T^(k)(·)| ≤ b^(k) for some b^(k) ≥ 0. Therefore, it follows from (A4) that the function

$$F(·):=\left(\begin{array}{c}{p}^{(1)}{T}^{(1)}(·)\\ ⋮\\ p^{(m)}{T}^{(m)}(·)\end{array}\right)$$

is a continuous mapping of the compact convex set ℬ := [−b⁽¹⁾, b⁽¹⁾] × … [−b^(m), b^(m)] into itself (i.e., F : ℬ → ℬ). Invoking the Brouwer fixed-point theorem [32] we conclude that there exists u₀ ∈ ℬ such that F(u₀) = u₀, therefore, u₀ satisfies (4).

In what follows, we assume that the conditions of Lemma 2 are met, thus (1)–(2) has a homogeneous fixed point of the form x = 1_N ⊗ x₀ for some x₀ ∈ ℝⁿ, and let u₀ ∈ ℝ^m denote the corresponding value of u_i for i = 1, …, N.

3.1 Patterning with a Single Layer of Communcation

In this section, we focus on the case when m = 1 in (2), thus we drop the superscript.

3.1.1 Stability of the Homogeneous Fixed Points

In the following theorem, we derive a sufficient condition for the instability of the homogeneous fixed point x = 1_N ⊗ x₀ relying only on the input/output function T(·) and on the eigenvalues of P (necessary conditions are presented in Lemma 1 in the Appendix when the linearization of (1)–(2) is also available).

Theorem 1

Assume that (A1), (A2) and (A3) hold. The fixed point x = 1_N⊗x₀ of (1)–(2) is unstable if P has a real eigenvalue λ_i such that

$$1-T^{\prime }(u_0){\lambda }_i<0.$$ (5)

Proof

With

$$\begin{gathered} A:={\frac{\partial f(x_i,u_i)}{\partial x_i}|}_{(x_0,u_0)}, \\ B:={\frac{\partial f(x_i,u_i)}{\partial u_i}|}_{(x_0,u_0)}, \\ C:={\frac{\partial h(x_i)}{\partial x_i}|}_{x_0}, \end{gathered}$$ (6)

it is sufficient to show that (5) implies that A+λ_iBC has an eigenvalue with positive real part according to Lemma 1 in the Appendix.

Since from [25] we obtain that T′(u₀) = −CA⁻¹B, (5) is equivalent to the condition 1 + λ_iCA⁻¹B < 0. From Sylvester’s determinant theorem it follows that

$${(-1)}^{n}det(A)det(1+{\lambda }_{i}C{A}^{-1}B)={(-1)}^{n}det(A+{\lambda }_{i}BC).$$

Claim 2 in [26] yields that (−1)ⁿ det(A) > 0 since A is Hurwitz by (A3), thus we obtain that (−1)ⁿ det(A+λ_iBC) < 0. From this we conclude that A + λ_iBC has a positive real eigenvalue invoking Claim 2 in [26].

3.1.2 Emergence of Patterns

Next, we study the emergence of patterns. To this end, we rely on results from the theory of monotone systems together with the notion of balanced partitioning of graphs [33]. To simplify notation, consider M ∈ ℝ^N×N and introduce

$$\mathrm{\Psi }(M):=M-\sum _{i=1}^N\text{diag}(e_i)M\text{diag}(e_i),$$

which is the same as M except it has zeros in the diagonal.

Definition 1

The graph 𝒢 = (V,W) is balanced if there is a partition of its set of nodes V into V₁ and V₂ such that all positive edges connect nodes within V₁ or V₂, and negative edges connect nodes between V₁ and V₂. Furthermore, define the bipartition vector b := (b₁ … b_N)^T such that b_i = (−1)^k if node i belongs to V_k (k = 1, 2).

Theorem 2

Provided (A2), (A4) and (A5), assume that the graph with irreducible adjacency matrix Ψ(PT′(u₀)) is balanced with bipartition vector b. Introduce u^* := 1_Nu₀ and the cone 𝒦 = {u : S(u − u^*) ⪰ 0} where S = diag(b). If

$$1-{\lambda }_i{T}^{\prime }(u_0)<0$$ (7)

for some real eigenvalue λ_i of P, then both sets u^* ± 𝒦 contain a point u ≠ u^* such that x_i = T(u_i) is a fixed point of (1)–(2).

Proof

Introduce the auxiliary dynamical system

$$\stackrel{.}{u}=-u+P\left(\begin{array}{c}T(u_1)\\ ⋮\\ T(u_N)\end{array}\right)$$ (8)

and note that the fixed points of (8) are identical to the solutions of (3) when m = 1. Therefore, the fixed points of (8) are fixed points of (1)–(2) from Lemma 1. Therefore, in the rest of the proof we focus on the fixed points of (8).

First, introduce the coordinate transformation w := Su and note that S = S⁻¹, yielding

$$\stackrel{.}{w}=-w+SP\left(\begin{array}{c}T(b_1{w}_1)\\ ⋮\\ T(b_N{w}_N)\end{array}\right)=:F(w).$$ (9)

The Jacobian of (8) is given by J(u) := −I + PΔ(u) where Δ(u) := diag(T′(u₁) … T′(u_N)), so that the Jacobian of (9) is given by DF(w) := SJ(Sw)S. We next show that (9) is cooperative by proving that DF(w) is Metzler for all w ∈ ℝ^N. To this end, note that since the graph with irreducible adjacency matrix Ψ(PT′(u₀)) is balanced with bipartition vector b, so is Ψ(PΔ(u)) from (A5). Therefore, we have that b_ib_jp_i,jT′(u_j) ≥ 0 for i ≠ j and i, j = 1, …, N. It then follows that DF(w) is Metzler for all w ∈ ℝ^N, thus (9) is cooperative.

Second, we focus on the bounded forward invariant set 𝒱 in Lemma 2 in the Appendix. From (A5) we have that ∃T̄ > 0 such that |T(·)| ≤ T̄. With this, introduce w̄ := max(|u₀|, ||P||₁ T̄) and the set 𝒲 := [− w̄, w̄]^N. We next show that 𝒱 := S𝒲 is forward invariant for (8). To see this, note that from (9) we obtain that

$${\stackrel{.}{w}}_i=-w_i+b_i\sum _{j=1}^N{p}_{i,j}T(b_j{w}_j),$$

where $\mid b_i{\sum }_{j=1}^N{p}_{i,j}T(b_j{w}_j)\mid \le {\Vert P\Vert }_1\overline{T}$. This yields F_w(w̄1_N) ≤ 0 and F_w(−w̄1_N) ≥ 0 from (9). Given that (9) is cooperative, thus monotone with respect to the standard orthant cone ${ℝ}_{\ge 0}^N$, we conclude that the hypercube 𝒲 is forward invariant and it contains the equilibrium point Su^* as w̄ ≥ |u₀|. Therefore, 𝒱 = S𝒲 is bounded, forward invariant and it contains u^*.

Third, note that S⁻¹J(u^*)S = −I + S⁻¹PT′(u₀)S. We already proved above that D := S⁻¹PT′ (u₀)S is Metzler. Let d_i,j denote the entries of D (i, j = 1, …, N), introduce d := min_i d_i,i and Q := D − dI together with α := 1 − d. With this, we obtain that S⁻¹J(u^*)S = −αI + Q such that Q ⪰ 0 is irreducible (since P is). Therefore, to invoke Lemma 2 in the Appendix, all there is left to show is that ρ(Q) > α.

To this end, note that the eigenvalues of D are λ_jT′(u₀) and since D is Metzler from above, we invoke Corollary 4.3.2 in [34] to conclude that s(D) = λ_iT′(u₀) for some i such that λ_i is real. Therefore, the condition in (7) yields that s(D) > 1. Furthermore, from Q = D − dI it follows that s(Q) = s(D) − d, and since α = 1 − d we obtain that s(Q) > α. Finally, from the Perron-Frobenius theorem (Theorem 4.3.1 in [34]) we have that ρ(Q) = s(Q), thus ρ(Q) > α.

Now we can invoke Lemma 2 in the Appendix with 𝒱, S and Q defined above to conclude that both sets u^* ± 𝒦 contain a fixed point u ≠ u^* of (8). This is equivalent to having solutions u of (3) in both sets u^* ± 𝒦 different from u^*. Finally, we conclude from Lemma 2 that such solutions u = (u₁ … u_N)^T yield fixed points x = (T(u₁) … T(u_N))^T of (1)–(2).

3.1.3 Patterns with Groups

Finally, we search for equilibrium points of (1)–(2) in which subsystems are grouped into classes O₁, …, O_r such that x_i = x_j if i, j ∈ O_k. Such a solution yields patterns in which subsystems of the same class have identical steady states, and it reduces the complexity of the analysis by decreasing the dimension of the problem. To find these solutions, we rely on the notion of equitable partitions of graphs [33].

Definition 2

For a weighted and directed graph (V,P) with adjacency matrix P, a partition π of the vertex set V into classes O₁, …, O_r is said to be equitable if there exist p̄_i,j for i, j = 1, …, r such that

$${\overline{p}}_{i,j}=\sum _{v\in O_j}{p}_{u,v}\forall u\in O_i.$$ (10)

Let the reduced adjacency matrix P̄ ∈ ℝ^r×r be formed by the entries p̄_i,j.

Theorem 3

Provided (A2), (A4) and (A5), let π be an equitable partition of the vertices V of the graph (V,P) into classes O₁, …, O_r and let P̄ denote the resulting reduced adjacency matrix. Assume that Ψ(P̄T′(u₀)) is irreducible and balanced with bipartition vector b̄. Introduce u^* := 1_ru₀ and the cone 𝒦 = {u : S(u − u^*) ⪰ 0} where S = diag(b). If

$$1-{\overline{\lambda }}_i{T}^{\prime }(u_0)<0,$$ (11)

for some real eigenvalue λ̄_i of P̄ then both sets u^* ±𝒦 contain a point ū ≠ u^* such that x_i = T(ū_j) for i ∈ O_j is a fixed point of (1)–(2).

Proof

Consider the reduced set of equations

$$\left(\begin{array}{c}{\overline{u}}_1\\ ⋮\\ {\overline{u}}_r\end{array}\right)=\overline{P}\left(\begin{array}{c}T({\overline{u}}_1)\\ ⋮\\ T({\overline{u}}_r)\end{array}\right).$$ (12)

Following the same steps as in the proof of Theorem 2, we conclude that (12) has equilibrium points ū ≠ u^* in both sets u^*±𝒦. Exploiting the fact that π is an equitable partition of (V,P), a solution ū of (12) also defines a solution u of (3) in which u_i = ū_j for all i ∈ O_j, so that x_i = T(ū_j) for i ∈ O_j, concluding the proof.

3.2 Patterning with Multiple Layers of Communication

We now focus on the case of multiple layers of communication channels (m = 2, 3, … ). In what follows, we assume that (A7) holds, thus from Theorem 1.3.19. in [35] it follows that there exists an invertible matrix W such that $W^{-1}{P}^{(k)}W=\text{diag}(\begin{array}{ccc}{\lambda }_1^{(k)}& \dots & {\lambda }_N^{(k)}\end{array})=:{\mathrm{\Lambda }}^{(k)}$ for k = 1, …, m, and define

$${\mathrm{\Lambda }}_i:=\left[\begin{array}{ccc}{\lambda }_i^{(1)}& & \\ & \ddots & \\ & & {\lambda }_i^{(m)}\end{array}\right]i=1,\dots ,N.$$ (13)

As the proofs of theorems presented in this chapter are similar to those of the previous one for a single channel of communication, we provide them in the Appendix for the sake of brevity.

3.2.1 Stability of the Homogeneous Fixed Points

We first derive a sufficient condition for the instability of the homogeneous fixed point x = 1_N ⊗ x₀ relying only on the input/output function T(·) and the eigenvalues of P^(k) for k = 1, …, m (necessary conditions are presented in Lemma 3 in the Appendix when the linearization of (1)–(2) is also available).

Theorem 4

Assume that (A1), (A2), (A3), (A4) and (A7) hold. The fixed point x = 1_N ⊗ x₀ of (1)–(2) is unstable if

$$det(I_m-{\mathrm{\Lambda }}_i{T}^{\prime }(u_0))<0$$ (14)

for some real Λ_i defined in (13).

3.2.2 Emergence of Patterns

We next study the emergence of patterns and their relationship with the homogeneous fixed point x = 1_N ⊗ x₀.

Theorem 5

Assume that (A2), (A4), (A5) and (A7) hold. Furthermore, assume that the graphs with adjacency matrices P⁽¹⁾, …, P^(m) are balanced with the same bipartition vector b, and T′(u₀) is also balanced with bipartition vector q. Introduce u^* := 1_N ⊗ u₀ and the cone 𝒦 = {u : S(u − u^*) ⪰ 0} where S = diag(b) ⊗ diag(q). If for some i we have that

$$\underset{i}{max}s({\mathrm{\Lambda }}_i{T}^{\prime }(u_0))>1,$$ (15)

then both sets u^* ± 𝒦 contain a point u ≠ u^* such that x_i = T(u_i) is a fixed point of (1)–(2).

3.2.3 Patterns with Groups

Finally, we search for equilibrium points of (1)–(2) in which subsystems are grouped into classes O₁, …, O_r such that x_i = x_j if i, j ∈ O_k. Let P̄^(k)ℝ^r×r denote the reduced adjacency matrices with eigenvalues ${\overline{\lambda }}_i^{(k)}(i=1,\dots ,r,k=1,\dots ,m)$, and define ${\overline{\mathrm{\Lambda }}}_i:=\text{diag}(\begin{array}{ccc}{\overline{\lambda }}_i^{(1)}& \dots & {\overline{\lambda }}_i^{(m)}\end{array})$.

Theorem 6

Assume that (A2), (A4), (A5) and (A7) hold. Let π be an equitable partition of the vertices of the graphs with adjacency matrices P⁽¹⁾, …, P^(m) into classes O₁, …, O_r for all k = 1, …, m. Furthermore assume that the graphs with adjacency matrices P̄⁽¹⁾, …, P̄^(m) are balanced with the same bipartition vector b, and T′(u₀) is also balanced with bipartition vector q. Introduce u^* := 1_r ⊗ u₀ and the cone 𝒦 = {ū : S(ū − u^*) ⪰ 0} where S = (1_rb^T) ⊗ (1_mq^T). If for some i we have that

$$\underset{i}{max}s({\overline{\mathrm{\Lambda }}}_i{T}^{\prime }(u_0))>1,$$

then both sets u^* ± 𝒦 contain a point ū ≠ u^* such that x_i = T(ū_j) for i ∈ O_j is a fixed point of (1)–(2).

4 Application examples

We next focus on the emergence of patterns in neural networks, in reaction-diffusion systems, and in contagion processes over random networks to demonstrate how our results can be employed when studying pattern formation over networks of the form (1)–(2).

4.1 Pattern Formation in Neural Networks

Before illustrating how our results can be applied to study pattern formation over multigraphs, we first focus on pattern formation over single graphs here. In particular, consider the interconnection of N leaky integrate-and-fire neurons [36], described by

$$\begin{gathered} {\stackrel{.}{x}}_i=-a{x}_i+g(u_i), \\ {y}_i=x_i, \\ u=Py \end{gathered}$$ (16)

with a > 0 and where g(·) is an increasing function such that g(0) = 0. In what follows, we consider

$$g(u_i)=G\frac{exp\left(\frac{2\mu }{G}{u}_i\right)-1}{exp\left(\frac{2\mu }{G}{u}_i\right)+1}$$ (17)

to model the saturated nature of the interconnection channels. For simplicity, we focus on the case when P1_N = 0_N so that the origin is a fixed point of (16), thus x₀ = u₀ = 0.

From Lemma 4 in the Appendix it follows that the origin is globally asymptotically stable if −a + μω_i has negative real part for i = 1, …, N, where ω_i denotes the eigenvalues of the matrix P_abs where [P_abs]_i,j:= |p_i,j |. As a concrete example, introduce the asymmetric interconnection matrix

$$P=\left[\begin{array}{cccc}1& -1& & \\ & \ddots & \ddots & \\ & & \ddots & -1\\ -1& & & 1\end{array}\right],$$ (18)

describing the interconnection of N neurons in a ring structure such that each neuron activates itself and inhibits its neighbor on the right-hand side, and assume that N is even. Lemma 4 in the Appendix then yields that the origin is a globally asymptotically stable fixed point of (16)–(18) if μ < a/2 (left panel in Fig. 1).

Fig. 1.

Pattern formation in neural networks. The origin is globally asymptotically stable when μ < a/2 (left panel, simulation parameters: N = 50, a = 1, G = 1, μ = 0.4). A unique alternating steady state pattern emerges when μ > a/2 (right panel, simulation parameters: N = 50, a = 1, G = 1, μ = 0.4).

Building on the results presented in Section III.A, we first show that an alternating pattern (right panel in Fig. 1) of the form x₁ = −x₂ = x₃ = …= x_N−1 = −x_N ≠ 0 emerges when μ > a/2. Then we prove that this pattern is unique (up to rotation along the ring) and stable.

Invoking Theorem 3, we first prove that (16)–(18) has two fixed points other than the origin: one such that x_2k−1 ≥ 0 and x_2k ≤ 0; and another such that x_2k−1 ≤ 0 and x_2k ≥ 0 for k = 1, …, N/2. To this end, note first that the partition π of the vertices into O₁ = {1, 3, …, N − 1} and O₂ = {2, 4, …, N} is equitable. The eigenvalues of the corresponding reduced adjacency matrix

$$\overline{P}=\left[\begin{array}{cc}1& -1\\ -1& 1\end{array}\right]$$

are λ̄₁ = 0 and λ̄₂ = 2. Second, the irreducible matrix

$$\mathrm{\Psi }(\overline{P}{T}^{\prime }(u_0))=\left[\begin{array}{cc}0& -\raisebox{1ex}{$\mu $}\!\left/ \!\raisebox{-1ex}{$a$}\right.\\ -\raisebox{1ex}{$\mu $}\!\left/ \!\raisebox{-1ex}{$a$}\right.& 0\end{array}\right]$$

is balanced with bipartition vector b̄ = (1 −1)^T. Third, since u₀ = 0, we obtain that u^*:= 1_ru₀ is the origin, yielding the cone 𝒦:= {ū: diag(b̄)(ū − u^*) ⪰ 0} = {ū: ū₁ ≥ 0, ū₂ ≤ 0}. Therefore, from Theorem 3 it follows that if μ > a/2 then there exist ū₁ ≥ 0 and ū₂ ≤ 0 not simultaneously zero such that for i = 1, …, N both

$$x_i=\{\begin{array}{ll}T({\overline{u}}_1)\hfill & i\text{even}\hfill \\ T({\overline{u}}_2)\hfill & i\text{odd}\hfill \end{array}$$ (19)

and

$$x_i=\{\begin{array}{ll}T({\overline{u}}_1)\hfill & i\text{odd}\hfill \\ T({\overline{u}}_2)\hfill & i\text{even}\hfill \end{array}$$ (20)

are fixed points of (16)–(18), where T(ū₁) ≥ 0 and T(ū₂) ≤ 0 are not simultaneously zero.

Next, we prove that apart from the origin, the only fixed points of (16)–(18) are those in (19)–(20). To this end, consider first the unique solution x^* > 0 of

$$x^{\ast }=\frac{G}{a}\frac{exp(\frac{4\mu }{G}{x}^{\ast })-1}{exp(\frac{4\mu }{G}{x}^{\ast })+1}$$ (21)

and note that with ū₁:= 2x^* > 0 and ū₂:= −2x^* < 0 we have x^* = T(ū₁) > 0 and −x^* = T(ū₂) < 0. Since |g(u_i)| < G we have that ẋ_i can only be zero if |x_i| < G/a, therefore, what is left to show is that if x_i ∈ (0, x^*) or x_i ∈ (x^*,G/a) then x can not be a fixed point of (16)–(18). To this end, note that

$$x_{i+1}=x_i+\frac{G}{2\mu }ln\left(\frac{\frac{G}{a}-x_i}{\frac{G}{a}+x_i}\right)=:M(x_i),$$ (22)

so that for x to be a fixed point we must have that

$$x_i=(\underset{N}{\underbrace{M\circ M\circ \cdots \circ M}})(x_i).$$

From (22) it follows that |M(x^*)| = x^* and we obtain that |M(x_i)| < x_i for |x_i| < x^* and |M(x_i)| > x_i for x^* < |x_i| < G/a. Therefore, we must have |x_i| = x^* for x to be a fixed point, and then from (22) it follows that either x_i = (−1)ⁱx^* or x_i = (−1)ⁱ⁺¹x^* for i = 1, 2, …, N.

Finally, we show that the alternating patterns in (19)–(20) are stable fixed points of (16)–(18). To this end, define α:= 2μ/a and v:= 2ax^*/G such that from (21) we have that v = (e^2αv − 1)/(e^2αv + 1), and let v = V (α) denote its positive solution. Define h(α):= 2αe^{αV (α)}/[(e^{2αV (α)}+1)²], yielding g′(ū₁) = g′ (ū₂) = g′ (±2x^*) = ah(α). Since 0 < h(α) < 0.5 for α > 1 (verified numerically), this then implies that with P̃:= ah(α)P its spectral radius ρ(P̃) is such that ρ(P̃) = ah(α)ρ(P) < a. Lemma 5 in the Appendix then yields that the alternating patterns in (19)–(20) are stable fixed point of (16)–(18).

4.2 Pattern Formation in Reaction-Diffusion Networks

We first briefly illustrate the idea underlying diffusion-driven pattern formation considering the system

$$\begin{gathered} {\stackrel{.}{x}}_{i,1}=a{x}_{i,1}+b{x}_{i,2}, \\ {\stackrel{.}{x}}_{i,2}=-c{x}_{i,1}-d{x}_{i,2}+g(u_i), \\ {y}_i=x_{i,2}, \\ u=Py, \end{gathered}$$ (23)

such that a, b, c, d > 0, g(0) = 0 and g′ (·) > 0; for instance, see (17). The unconnected systems (P = 0) are stable at the origin if ad − bc < 0 and a < d, but can become unstable when interconnected. As a concrete example, consider the interconnection matrix

$$P=\left[\begin{array}{cccc}-2& 1& & 1\\ 1& \ddots & \ddots & \\ & \ddots & \ddots & 1\\ 1& & 1& -2\end{array}\right].$$ (24)

From Lemma 4 in the Appendix it follows that the origin is stable if g′ (0) < (a − d)/4, and conversely, it becomes unstable when g′ (0) > (a − d)/4. As previously, we can study the emergence of various patterns invoking Theorem 3. For instance, the partition π of the vertices into O₁ = {1, 3, …, N−1} and O₂ = {2, 4, …, N} is equitable, so that following a similar reasoning as in the previous example, we expect the emergence of an alternating pattern, verified in Fig. 2. The stability of various patterns can be analyzed using Lemma 5 in the Appenidx, as demonstrated in the previous example.

Fig. 2.

Pattern formation in reaction-diffusion networks. Once g′ (0) > (a − d)/4, the origin becomes unstable and patterns emerge, for instance, the alternating pattern in which subsequent nodes have alternating fates along a ring (simulation parameters: a = G = μ = 1, b = c = d = 2, N = 20.)

Having illustrated diffusion-driven pattern formation on the idealized system in (23), we next focus on the Brusselator [37], a standard biochemical model for Turing pattern formation (other models, such as that of Gierer and Meinhardt [4], of Schnakenberg [38], and of Thomas [39] can be treated similarly). We demonstrate how our results can be used in the case of multigraphs, and illustrate that considering different interconnection structures for different species can decrease the often prohibitively large difference in diffusion coefficients required for pattern formation.

Consider first the non-dimensional model of an isolated Brusselator given by

$$\begin{gathered} {\stackrel{.}{x}}_{i,1}=1-(1+b)x_{i,1}+a{x}_{i,1}^2{x}_{i,2}, \\ {\stackrel{.}{x}}_{i,2}=b{x}_{i,1}-a{x}_{i,1}^2{x}_{i,2}, \end{gathered}$$ (25)

where a, b > 0. It follows from linearization about the steady state x = (1, b/a)^T that the unique fixed point is stable if b < a + 1.

Upon interconnection, the dynamics of the network become

$$\begin{gathered} {\stackrel{.}{x}}_{i,1}=1-(1+b)x_{i,1}+a{x}_{i,1}^2{x}_{i,2}+g_1(u_i^{(1)}), \\ {\stackrel{.}{x}}_{i,2}=b{x}_{i,1}-a{x}_{i,1}^2{x}_{i,2}+g_2(u_i^{(2)}), \\ {y}_{i,j}=x_{i,j}j=1,2, \\ {u}^{(j)}=P^{(j)}{y}^{(j)}j=1,2 \end{gathered}$$ (26)

with g_i(·) from (17). Since g_i(0) = 0 and P⁽ⁱ⁾1_N = 0_N, we obtain u₀ = (0, 0)^T at the homogeneous steady state. Next, we study the stability of this fixed point.

Define ${\omega }_i^{(j)}:=-g_j^{\prime }(0){\lambda }_i^{(j)}$. Then, from (26) Lemma 6 in the Appendix it follows that x = 1_N ⊗ x₀ is unstable if for some i ∈ {1, …, N} we have

$$b>1+a\frac{1+{\omega }_i^{(1)}}{{\omega }_i^{(2)}}+{\omega }_i^{(1)},$$ (27)

and stable otherwise. Since $g_j^{\prime }(0)>0$ and ${\lambda }_i^{(1)}\le 0$, we have that ${\omega }_i^{(j)}\ge 0$ for j = 1, 2 and i = 1, …, N. For instance, consider the case when P⁽¹⁾ = P⁽²⁾ = P with P from (24), so that ${\lambda }_N^{(1)}={\lambda }_N^{(2)}=-4$. Considering the parameters a = 1, b = 1.8, and μ₂/μ₁ = 21 from [40], we need to have μ₁ > 0.0174 to satisfy (27). When μ₁ < 0.0174, the homogeneous steady state is stable (Fig. 3, top panels), conversely, a pattern emerges when μ₁ becomes greater than 0.0174 (Fig. 3, bottom panels).

Fig. 3.

Emergence of patterns with N = 20 Brusselators interconnected in a ring structure. When μ₁ < 0.0174, the homogeneous steady state is stable (μ₁ = 0.015 in the top panels). When μ₁ becomes greater than 0.0174, the homogeneous steady state becomes unstable and a pattern emerges: an alternating one in this particular simulation (μ₁ = 0.02 in the bottom panels).

Next, we illustrate how we can leverage different interconnection structures for different species to decrease the often prohibitively large difference in diffusion coefficients required for pattern formation. To this end, note first that (27) reveals that b > 1 is necessary for x = 1_N ⊗ x₀ to become unstable, so that in what follows we consider this to be the case. Assume that (27) is satisfied for some i, and introduce $\epsilon :={\omega }_i^{(2)}/{\omega }_i^{(1)}$. Then from (27) it follows that we must have $\epsilon >a{(1+\sqrt{b})}^2/{(1-b)}^2=:{\epsilon }^{\ast }$ for x = 1_N ⊗ x₀ to become unstable. If the interconnection matrices are identical, i.e., P⁽¹⁾ = P⁽²⁾, then ${\lambda }_i^{(1)}={\lambda }_i^{(2)}$ for i = 1, 2, …, N, yielding ε^* = μ₂/μ₁. Since ε^* > 1, we have that μ₂ > μ₁, thus the diffusibility of the inhibitor x_i,2 dominates that of the activator x_i,1, hence Turing instability is often referred to as “local activation with long range inhibition.”

Since ε^* = μ₂/μ₁ can be much greater than 1, especially when b ≈ 1, this often presents experimental challenges in synthetic biology. To overcome this obstacle, it is possible to artificially decrease the diffusion coefficient of the activator by introducing a third molecule binding to the activator, thus rendering it effectively inactive [41]. Instead of adding this molecule to all the physical channels among nodes, we can further decrease ε by adding this molecule only to a few select physical channels, thus introducing different interconnection matrices for activator and inhibitor (i.e., P⁽¹⁾ ≠ P⁽²⁾), yielding different eigenvalues ${\lambda }_i^{(1)}$ and ${\lambda }_i^{(2)}$ for i = 1, …, N.

To illustrate this, let ${\lambda }_N^{(j)}$ denote the smallest (largest in absolute value) eigenvalue of P^(j) for j = 1, 2, and assume that (27) holds for i = N, thus the fixed point is unstable. This means that $({\mu }_2{\lambda }_N^{(2)})/({\mu }_1{\lambda }_N^{(1)})>{\epsilon }^{\ast }$, so that if ${\lambda }_N^{(2)}>{\lambda }_N^{(1)}$ then it is sufficient to have ${\mu }_2/{\mu }_1>{\epsilon }^{\ast }{\lambda }_N^{(1)}/{\lambda }_N^{(2)}=:\stackrel{\sim }{\epsilon }$ such that ε̃< ε^*. For instance, consider first the case when P⁽¹⁾ = P⁽²⁾ = P with P from (24), so that ${\lambda }_N^{(1)}={\lambda }_N^{(2)}=-4$. Next, rewiring the network for the activators x_i,1 such that the corresponding interconnection matrix becomes P⁽¹⁾:= ([P⁽²⁾]² + 6P⁽²⁾)/3 yields ${\lambda }_N^{(1)}=-3$ (as N → ∞). This means that ε̃ = 0.75ε^*, thus the required difference between diffusion coefficients is now reduced by 25%.

4.3 Contagion Processes over Random Graphs

Finally, we discuss pattern formation in connected random networks, illustrated considering contagion processes. Here we focus on two of the most common network models: Erdos-Renyi random graphs [42] and scale-free networks generated by preferential attachment [43] (other network models can be anayzed similarly). Considering the symmetric and irreducible Laplacian matrix P associated with the network structure, we have that ρ(P) = s(P) = λ_max > 0 where λ_max is the dominant eigenvalue of P (Theorem 4.3.1 in [34]), a quantity that plays a central role in pattern formation.

In Fig. 4 we display the dominant eigenvalue λ_max of P for various network sizes as a function of density. According to these results, λ_max is heavily influenced by how the random network is generated: it is significantly larger in the case of scale-free networks than in Erdos- Renyi random graphs. To illustrate what this means in the context of pattern formation, we next fix the network size (N = 200) and consider two networks generated by the above two methods with the same density (5%). The dominant eigenvalue λ_max of the Laplacian matrices associated with these two networks differ significantly: ${\lambda }_{max}^{\text{SF}}\approx 50$ and ${\lambda }_{max}^{\text{ER}}\approx 20$ for scale-free networks and Erdos-Renyi random graphs, respectively, in accordance with the results depicted in Fig. 4.

Fig. 4.

The dominant egienvalue λ_max of the Laplacian associated with the network structure is greater in scale-free networks (SF) than in Erdos-Renyi random graphs (ER) of the same density.

As for nodal dynamics, we consider one of the standard compartmental models in epidemiology: the SIS model given by

$$\begin{gathered} {\stackrel{.}{x}}_{i,1}=\alpha -\beta x_{i,1}{x}_{i,2}+\gamma x_{i,2}-\delta x_{i,1}+g(u_i), \\ {\stackrel{.}{x}}_{i,2}=\alpha +\beta x_{i,1}{x}_{i,2}-\gamma x_{i,2}-\delta x_{i,2}, \end{gathered}$$ (28)

for i = 1, …, N, where α, β, γ, and δ are the birth, infection, revocery, and death rates, respectively [44]. In this model, x_i,1 and x_i,2 denote the number of susceptible and infected (e.g., with flu) agents in population i, respectively.

From (17) we obtain that g(u_i) ≡ 0 when μ = 0, thus the nodes in the network are effectively not interconnected, so that trajectories of (28) converge to the homogeneous fixed point x = 1_N ⊗ x₀ where x₀ = (x_0,1, x_0,2)^T. In the simulations we consider α = β = 1 and γ = δ = 10, yielding x_0,1 = x_0,2 = 1. According to Theorem 2, once (7) is satisfied, the homogeneous fixed point becomes unstable, thus giving rise to patterning. With g(u_i) from (17) we obtain that T′(u₀) = μ/11 with α = β = 1 and γ = δ = 10. Therefore, since ${\lambda }_{max}^{\text{ER}}<{\lambda }_{max}^{\text{SF}}$, from (7) we expect no patterning when $\mu \in [0,\raisebox{1ex}{$11$}\!\left/ \!\raisebox{-1ex}{${\lambda }_{max}^{\text{SF}}$}\right.)\approx [0,0.22)$, patterning only in the scale-free network for $\mu \in (\raisebox{1ex}{$11$}\!\left/ \!\raisebox{-1ex}{${\lambda }_{max}^{\text{SF}}$}\right.,\raisebox{1ex}{$11$}\!\left/ \!\raisebox{-1ex}{${\lambda }_{max}^{\text{ER}}$}\right.)\approx (0.22,0.55)$, and patterning in both networks when $\mu >\raisebox{1ex}{$11$}\!\left/ \!\raisebox{-1ex}{${\lambda }_{max}^{\text{ER}}$}\right.\approx 0.55$, verified in Fig. 5.

Fig. 5.

Pattern formation with SIS nodal dynamics over an Erdos-Renyi random graph (top panels) and over a scale-free random network generated by preferential attachment (bottom panels). Both networks have N = 200 nodes and density of 5%. When μ is sufficiently small such that (7) is not satisifed, both the population size x_i,1+x_i,2 for each node and the infected fraction x_i,2/(x_i,1+x_i,2) of the population are the same for all nodes. Once μ is greater than the critical threshold μ^* ∝ ¹/_λmax such that (7) is satisfied, the homogeneous fixed point becomes unstable and patterns emerge. Since λ_max is greater in scale-free networks than in Erdos-Renyi random graphs (Fig. 4), the critical threshold μ^* is smaller for the former. Simulation parameters: α = β = 1, γ = δ = G = 10, together with μ = 0.1 (black), μ = 0.4 (dashed light grey), and μ = 0.7 (solid dark grey).

Finally, to interpret the above results, note that P is symmetric, thus λ_max = σ_max where σ_max is the largest singular value of P. From this, the induced 2-norm of P is ||P||₂ = max_||z||2=1 ||Pz||₂ = σ_max = λ_max = ρ(P). Let d_max denote the maximum vertex degree and assume that the vertex degree of node i is d_max. Let z^* ∈ ℝ_N be such that z_j = 1/d_max if p_i,j = 1 (nodes i and j are neighbors) and zero otherwise, so that ||z||₂ = 1. From this we obtain that ρ(P) = ||P||₂ ≥ d_max. The maximum vertex degree is usually (much) greater in the case of preferential attachment than in case of Erdos-Renyi random graphs (with the same density), in turn providing a (much) greater lower bound for the spectral radius ρ(P), elucidating the simulation results in Fig. 4, and as a result, those in Fig. 5.

5 Discussion

In this paper, we presented analytical results for pattern formation in large-scale networks. Our results are applicable to multigraphs having multiple layers of communication channels with possibly different interconnection structure for these layers, and we allow for asymmetric connections between nodes. Furthermore, by placing no restriction on the row-sum of the interconnection matrices, our results apply to a wide set of mechanisms leading to patterning (e.g., diffusion-driven instability and lateral inhibition). Therefore, the results presented here significantly advance our understanding of patterning in a general setting by overcoming the dimensional constraints of earlier studies.

By relying only on the static input/output characteristic of each module and the algebraic properties of the interconnection matrices, we first characterized the stability of the homogeneous fixed points. Following this, we provided sufficient conditions for the emergence of non-homogeneous patterns for balanced graphs and demonstrated that equitable partitions provide templates for patterns such that modules share the same fate within partitions. Finally, we illustrated our results in the context of neural networks, of reaction-diffusion systems, and of contagion processes over random graphs.

Within our formulation, we allowed for directed edges between nodes, different topologies for different layers of communication channels, and we placed no restriction on the row-sum of the interconnection matrices. Because of this general formulation, the practical applications of our results span many fields where the emergence of patterns is a central topic of interest. For instance, we demonstrated that considering different interconnection structures for different species in reaction-diffusion networks can be leveraged to decrease the often prohibitively large difference in diffusion coefficients required for pattern formation. This result can provide insight when studying self-organization in developmental biology [45] as well as guide the design of population-level behavior in synthetic biology [46]. Additionally, we illustrated not only that random networks with the same density can behave drastically differently in terms of patterning, but more importantly, that this difference can be understood by focusing on the dominant eigenvalue of the interconnection matrix. In addition to being relevant when focusing on the spreading of infectious diseases and the behavior of multiple connected habitats in ecology [47], this result is especially well-suited for studying social networks to understand, for instance, racial segregation in sociology [48] and the polarization of opinions in political science [49]. Finally, our results apply equally to different mechanisms underlying pattern formation (diffusion, averaging, lateral inhibition, etc.), both for the diffusion of matter (zero row-sum due to conservation laws [27], [28], [29], [30], [31]) and for the diffusion of opinions (non-zero row-sum due to lack of conservation laws [25], [26]), further broadening the applicability of our results and their practical implications.

The two most limiting assumptions of the results presented here concern the interconnection structure. First and foremost, to leverage results from monotone systems theory to conclude the emergence of non-homogeneous steady state patterns we relied on the fact that the graphs representing the connections among modules are balanced. To overcome this limitation, a generalization to a larger class of graphs needs to be developed. Second, in the case of multiple layers of communication channels, we assumed that the set of interconnection matrices P⁽¹⁾, …, P^(m) commute. This allowed us to simultaneously diagonalize these matrices, thus significantly decrease the difficulty of analyzing the emergence of patterns. While this assumption is not overly restrictive in biological systems, where the interconnection channels are often identical for all species, it might not be the case in other contexts. For instance, in social networks different types of ties can have fundamentally dissimilar underlying interconnection structure. While a set of commuting adjacency matrices allow for complex interconnection structures, to broaden the applicability of our results this assumption needs to be weakened.

Biographies

Andras Gyorgy received M.S. degrees both in Electrical Engineering and in Biomedical Engineering in 2009 and 2011, respectively, both from the Budapest University of Technology and Economics, Hungary. He then received the Ph.D. degree from the Massachusetts Institute of Technology in 2016. He is currently a postdoctoral researcher at the University of California, Berkeley. His research is in dynamical systems and control theory with applications to biological networks in synthetic and systems biology.

Murat Arcak received the B.S. degree from the Bogazici University, Istanbul, in 1996 and the M.S. and Ph.D. degrees from the University of California, Santa Barbara, in 1997 and 2000, respectively. He is currently a professor at U.C. Berkeley in the Electrical Engineering and Computer Sciences Department. His research is in dynamical systems and control theory with applications to synthetic biology, multi-agent systems, and transportation. He received the Donald P. Eckman Award from the American Automatic Control Council in 2006, the Control and Systems Theory Prize from the Society for Industrial and Applied Mathematics (SIAM) in 2007, and the Antonio Ruberti Young Researcher Prize from the IEEE Control Systems Society in 2014. He is a member of SIAM and a fellow of IEEE.