Local synchronization of one-to-one coupled neural networks with discontinuous activations

Abstract

In this paper, local synchronization is considered for coupled delayed neural networks with discontinuous activation functions. Under the framework of Filippov solution and in the sense of generalized derivative, a novel sufficient condition is obtained to ensure the synchronization based on the Lyapunov exponent and the detailed analysis in Danca (Int J Bifurcat Chaos 12(8):1813–1826, 2002; Chaos Solitons Fractals 22:605–612, 2004). Simulation results are given to illustrate the theoretical results.

Introduction

It is well known that differential equations with discontinuous terms on the right-hand side occur in many real problems and are widely used as simplified mathematical models of physical systems although the classical solutions of such systems may not exist (Danca 2002). Sometimes physical laws are expressed by discontinuous functions, examples include dry friction, impacting machines, power circuits, switching in electronic circuits and many others (Danca 2004).

Delayed neural networks, as the special complex networks, have also been found to exhibit complex and unpredictable behaviors including stable equilibria, periodic oscillations, bifurcation and chaotic attractors (Cao 2001; Liang et al. 2008; Liu and Cao 2009). But all these papers are based on the assumption that the activation functions are continuous. In fact, the neural networks with discontinuous activation functions are more important, which frequently arise in practice (Forti and Nistri 2003; Forti et al. 2005). For the discontinuous system, it has been an old research topic for decades. Several types of solutions have been available such as Caratheodory solutions, Filippov solutions and sample-and-hold ones (Cortés 2008). Recently, the Filippov solutions have been utilized as a feasible approach in the field of mathematics and control for discontinuous dynamical systems (Cortés 2008; Filippov 1988; Forti and Nistri 2003; Forti et al. 2005; Huang et al. 2009; Liu and Cao 2009; Lu and Chen 2006, 2008).

On the other hand, synchronization means two or more systems which are either chaotic or periodic share a common dynamical behavior. It has been shown that this common behavior can be induced by coupling or by external forcing. As we all know, chaotic systems exhibit sensitive dependence on initial conditions. Because of this property, chaotic systems are difficult to be synchronized or controlled. Since the pioneer work of Pecora and Carrol (1990), chaotic synchronization has become a hot topic in nonlinear dynamics owing to their theoretical significance and potential applications.

For a dynamical system with discontinuous vector field, there have been a few works about the synchronization issue of discontinuous dynamical systems (Danca 2002, 2004). To the best of our knowledge, however, there are few works about the synchronization of discontinuous delayed neural networks. Motivated by the above discussions, in this paper we will consider the local synchronization of delayed neural networks with discontinuous activations based on Lyapunov exponents (Fujisaka and Yamada 1983). Firstly, under the framework of Filippov, the existence of solutions for such discontinuous neural systems can be guaranteed. Secondly, a sequence of smooth functional differential systems (approximation systems) are constructed and prove that the solutions of these systems converge to a solution of discontinuous system via the matrix measure. Finally, some sufficient criteria are derived to guarantee the local synchronization of the one-to-one coupled discontinuous neural networks by means of the Lyapunov exponents.

The rest of the paper is organized as follows. “Model formulation and preliminaries” gives some preliminaries. “Existence of Filippov solutions and approximation” discusses the existence of Filippov solutions for the discontinuous neural networks and constructs a sequence of smooth approximation systems. “Lyapunov exponents and local synchronization of coupled neural networks” presents some sufficient conditions for synchronization of the delayed coupled system. In “Illustrative examples”, simulation results aiming at substantiating the theoretical analysis are reported. Main conclusions are presented in “Conclusions”.

Model formulation and preliminaries

In this paper, we consider the following neural networks described by the system

1

where is the state vector associated with the neurons; is an n × n constant diagonal matrix with d_i > 0, i = 1, 2, ... , n; A = (a_ij)_n×n and B = (b_ij)_n×n are the connection weight matrix and the delayed connection weight matrix, respectively; is a diagonal mapping where f_i, i = 1, 2, ... , n, represents the neuron input-output activation; τ is a constant delay and I is a constant input.

Definition 1 Class of functions: we call if for all i = 1, 2, ..., n, f_i(·) satisfies:

1. f_i(·) is continuously differentiable, except on a finite set of isolated points where the right and left limits and exist. Let I_k be open subsets of for k = 1, 2, ..., h such that

2. Except on the isolated points {ρⁱ_k}, f_i(·) is Lipschitz continuous, i.e., there exist constants K⁻_i and K⁺_i, i = 1, 2, ..., n, such that

   2

   Since the classical notion of derivatives at the discontinuity points of f_i cannot be used here, a new concept of derivative is required.

Definition 2 Let then f is said to be generalized differentiable at if the following limit exists and is finite

3

and, is called as the generalized derivative (generalized Jacobi matrix ) of f at x*. Further, we say that f is generalized differentiable on if it is so at every and denoted this class of functions by

Example Let us consider

with ρ_k = 0. and Then, for x* = 0, one has While for

does not exist, and g(x) is not generalized differentiable at the point 0.In the following, we apply the framework of Filippov in discussing the solution of delayed neural networks (1).

Definition 3 Suppose Then is called as a set-valued map from if for each point x of a set there corresponds a nonempty set A set-valued map F with nonempty values is said to be upper-semi-continuous at x₀ ∈ E if, for any open set N containing F(x₀), there exists a neighborhood M of x₀ such that F(M) ⊂ N. F(x) is said to have a closed (convex, compact) image if for each x ∈ E, F(x) is closed (convex, compact).

Now we introduce the concept of Filippov solution. Consider the following system

4

where f(·) is not continuous.

Definition 4 A set-valued map is defined as

5

where K(E) is the closure of the convex hull of set and μ(N) is Lebesgue measure of set N. A solution in the sense of Filippov (1988) of the Cauchy problem for Eq. 4 with initial condition x(0) = x₀ is an absolutely continuous function x(t), t ∈ [0, T], which satisfies x(0) = x₀ and differential inclusion:

6

Now we denote

where We extend the concept of the Filippov solution to the differential Eq. 1 as follows:

Definition 5 A function is a solution (in the sense of Filippov) of the discontinuous system (1) on [−τ, T), if:

1. x is continuous on [−τ, T) and absolutely continuous on [0,T);

2. x(t) satisfies

   7

   Or equivalently,
3. there exists a measurable function such that for a.e. t ∈ [−τ, T) and

   8

   where the single-valued function α is the so-called measurable selection of the function which approximates in some neighborhood of

It is obvious that the set-valued map has nonempty compact convex values. Furthermore, it is upper-semi-continuous (Aubin and Cellina 1984) and hence it is measurable. Here, we remark that all the set-valued functions obtained by Filippov regularization applied to functions verify the above several properties. Hence, by the measurable selection theorem (Aubin and Frankowska 1990), if x(t) is a solution of (1), then there exists a measurable function such that for a.e. t ∈ [0, +∞), the Eq. 8 is true.

Definition 6 For any continuous function and any measurable function such that for a.e. s ∈ [−τ, 0], an absolute continuous function x(t) = x(t, θ, ψ) associated with a measurable function α(t) is said to be a solution of the Cauchy problem for system (1) on [0,T) (T might be ∞) with initial value (θ(s), ψ(s)), s ∈ [−τ, 0], if

9

Definition 7 The matrix measure of a real square matrix A = (a_ij)_n×n is as follows:

where is an induced matrix norm on is the identity matrix, and p = 1, 2, ∞.

When the matrix norm we can obtain the matrix measure

Existence of Filippov solutions and approximation

In this section, we will prove that under some conditions, systems (9) exist solutions globally in the sense of Filippov.

Theorem 1 Suppose thatsatisfies a growth condition (g.c.): there exist constantsK₁, K₂ ≥ 0 with

10

Then, there exists at least one solution of system (1) in the sense of Eq. 9.

Proof Based on the detailed discussions in “Model formulation and preliminaries", the set-valued map is upper-semi-continuous with nonempty compact convex values, the local existence of a solution x(t) of (9) can be guaranteed (Filippov 1988). In Forti et al. (2005), the solution’s local existence was considered by step-by-step construction.Denote By (10), for a.e. t ∈ [0, +∞), one has

11

where It follows that

By the Gronwall inequality, one has

Hence, since x(t) remains bounded for positive times, it is defined on [0, +∞).

Remark 1 In Forti et al. (2005), the global existence of Filippov solutions was obtained by the Lyapunov method. In Lu and Chen (2006, 2008), the authors got the same results by constructing high-gain systems. In Huang et al. (2009); Xue and Yu (2007), the existence of period solutions have been guaranteed for the discontinuous sysytems by the famous Dugundji and Granas Theorem.

Next, we will construct a sequence of functional differential systems (approximation systems) with continuously differentiable nonlinear functions and prove that the solutions of these systems converge to a solution of system (1).

Let be the set of discontinuous points of f_i(·). Pick a strictly decreasing sequence with such that holds for any k₁ ≠ k₂ and where Define functions as follows:

where is continuously differentiable satisfying

12

13

For example, we can pick where and have to be determined from the above two equations.

Thus, we obtain the following smooth system:

14

which will be proved as an approximation of the initial discontinuous system (1).

From the construction, the condition (10) and Definition 1, there exist constants and such that

15

and

16

As pointed out in Hale (1977), for the smooth function g^m with every fixed m, the system

has a unique solution x^m(t) on [0, +∞).

Following, we will point out that the solution sequence of system sequence (14) converges to the solution of original system (1). To prove our results, the following two lemmas are needed.

Lemma 1 (Huang et al. 2009). Suppose that functiony(t) is non-negative whent ∈ (t₀, + ∞) and satisfies the following

wherea, b, σ are positive constants, anda > b. Then we have the following inequality

whereandris the unique positive solution of

Here the upper-right Dini derivativeD⁺y(t) is defined aswhereh→0⁺means thathapproaches 0 from the right-hand side.

Lemma 2 (Lu and Chen 2006, 2008). The solution sequenceof system sequence (14) will converge to the solution of original system (1), if the solutionsare uniformly bounded.

Theorem 2 The solution sequenceof approximation system (14) converges to the solution of original system (1) if for the constantK₁in the growth condition (g.c.), the following inequality holds:

17

Proof From Lemma 2, we only need to prove the solutions of system (14) are uniformly bounded. Consider the following positive radially unbounded Lyapunov-Krasovskii functional candidate for model (14) as

18

Calculating the upper right-hand derivative of V(t) along the positive half trajectory of Eq. 14, we have

It follows from (15) that

Let By Lemma 1, it follows that

19

where r is the unique positive solution of

Based on (19), it is obviously that the solutions {x^m(t)} of the systems (14) will be located in one bounded set, this completes the proof.

Remark 2 In Lu and Chen (2006, 2008), the authors used the Arzela-Ascoli lemma and the diagonal selection principle to get the convergence of {x^m(t)}, and then obtained the the convergence of based on the Mazur’s convexity theorem. At last, they obtained the the convergence of measurable function sequence {γ^m(t)} and then got the existence of a solution for the discontinuous system finally. The key points lie in the uniform boundedness of solution sequence {x^m(t)}, which could be ensured by Theorem 2 in this paper.

Remark 3 Matrix measure can have positive as well as negative values, whereas a norm can assume only nonnegative values. It is sign-sensitive in that μ(− A) ≠ μ(A) in general, whereas Because of these special properties, the results based on matrix measure usually are more precise and less restrictive. In our paper, the uniform boundedness can be ensured by means of matrix measure which indicates that such approach is really effective.

Lyapunov exponents and local synchronization of coupled neural networks

For the the continuous dynamical system based on the Oseledec’s theorem (Oseledec 1968), the Lyapunov exponents are σ_i are the eigenvalues of J, for x₀ ranging over where can be replaced by to guarantee the existence of the Lyapunov exponents; J is the Jacobi matrix evaluated at the initial value x(t) satisfying variational equations and ɛ(t) is the distance function between the two trajectories starting from x₀ and x₀ + ɛ(0), respectively, with initial conditions ɛ(t₀) = I generally.

Now, for the discontinuous system (1) with the Jacobi matrix J(x) is not defined at but, by Definitions 1 and 2, the generalized Jacobi matrix can be used.

In this section, we consider the synchronization issue of coupled delayed neural networks consisting of two identical networks:

20

where c > 0 is the coupling strength.

Let G(x(t)) =  −Dx(t) + Af(x(t)) + Bf(x(t − τ)) + I(t) with f is a continuous function, then the one-to-one coupled delayed neural networks (20) change to be

21

The variational equations for the synchronized trajectory z(t) = x(t) − y(t) = 0 of (21) are:

22

where J₁(t) is the Jacobi matrix of G(t) evaluated along the synchronized trajectory x(t) = y(t).

Subtracting in (21) that

23

where J₂(t) is the Jacobi matrix of the flow z(t).

Based on the fundamental results of the linear stability. The spectrum of Lyapunov exponents of Eq. 23, given by the characteristic equation, can be divided into two parts: λ¹ = {λ₁, λ₂, ..., λ_n} associated with the evolution on the invariant (synchronization) manifold x(t) = y(t), and the remaining part describing the evolution transverse to the above manifold and exactly describing the distance between the state and the synchronization manifold x(t) = y(t). Let denote the largest in the transverse space, The negativity of guarantees that all the transverse eigenmodes are stable. In other words, the coupled system (20) is locally synchronized if From (23), we know that So, if the largest Lyapunov exponent λ_m verifies c > λ_m/2, then local synchronization can be realized.

Hence, for the continuous function f, the following Lemma can ensure the synchronization of one-to-one coupled neural networks:

Lemma 3 (Danca2002). Let λ_mbe the largest Lyapunov exponent of the continuous dynamical systemAssume one-to-one coupling (20): Ifc > λ_m/2, then the coupled systems satisfy local synchronization.

When f is discontinuous but the above Lemma can be utilized to realize the synchronization of one-to-one coupled discontinuous neural networks based on the generalized Jacobi matrix.

Let be the set of discontinuous points of f_i(·). By Definitions 1 and 2, defining

and q(x) = (q₁(x₁), q₂(x₂), ..., q_n(x_n))^T. From construction, we can see that q(x) is continuously differentiable on and f’s generalized Jacobi matrix

Hence, the Lyapunov exponents of the following smooth system:

24

are just the ones of discontinuous neural network (1) in the sense of generalized derivative.

Theorem 3 Let λ_mbe the largest Lyapunov exponent of system (24). Assumec > λ_m/2 in the one-to-one coupling (20), then the coupled discontinuous systems (20) satisfy local synchronization.

Proof Straightforward from Lemma 3 and Definition 2.

Remark 4 When the discontinuous system possesses chaotic behavior, the class function f could ensure that the chaotic behavior still remain chaotic when the trajectories cross the discontinuous surface. In addition, the concept of generalized derivative is introduced, which seems that it is possible to find the Lyapunov exponents and synchronize two identical discontinuous systems which having chaotic motion. Hence, the activation function f need to be of class in this paper.

Illustrative examples

Example 1 Consider the coupled discontinuous delayed neural networks consisting of two identical networks (20) with

where the activation function is defined as ; x(t) = [x₁(t), x₂(t)]^T, y(t) = [y₁(t), y₂(t)]^T are the state vectors. When a = 0, the neural network model (1) is actually chaotic (Lu 2002) as shown in Fig. 1. When a ≠ 0, the model (1) is a discontinuous neural network. But for the activation function by utilizing the properties of generalized derivative and based on the detailed discussion in Danca (2002, 2004), the chaotic behavior could remain chaotic when the trajectories cross the discontinuous surface (due to the discontinuous points as shown in Fig. 2 with a =  −0.025.

Fig. 1.

Phase trajectories of model (1) with a = 0

Fig. 2.

Phase trajectories of model (1) with a =  −0.025

Using the one-to-one synchronization Theorem 3, we obtained the result illustrated in Figs. 3 and 4 with the initial condition x(t) = (0.2, 0.3)^T,  y(t) = (0.4, 0.6)^T,  ∀ t ∈ [ − 1, 0]. If we choose c = 0.2, after small time, the systems become synchronized.

Fig. 3.

Synchronization of two identical discontinuous networks modeled (20)

Fig. 4.

The synchronization error of the state variables e_i = y_i(t) − x_i(t), i = 1, 2

Example 2 In Example 1, pick the connection weight matrix A and the delayed connection weight matrix B as and respectively. Let a = −0.03, utilizing the properties of generalized derivative and based on the detailed discussion in Danca (2002, 2004), the chaotic behavior (Lu 2002) could remain chaotic when the trajectories cross the discontinuous surface. Using the one-to-one synchronization Theorem 3 again, we obtained the result illustrated in Figs. 5 and 6 with the initial condition x(t) = (0.2, 0.3)^T,  y(t) = (0.4, 0.6)^T,  ∀ t ∈ [−1, 0]. If we choose c = 0.7, after small time, the systems become synchronized.

Fig. 5.

Synchronization of two identical discontinuous networks modeled (20)

Fig. 6.

The synchronization error of the state variables e_i = y_i(t) − x_i(t), i = 1, 2

Conclusions

This paper has discussed the local synchronization of delayed neural networks with discontinuous activations. Sufficient conditions have been derived for local synchronization of such systems based on the Lyapunov exponents. In the sense of Filippov solution and generalized derivative, coupled chaotic synchronization of discontinuous delayed neural networks has been considered. The obtained results are novel since there are few works about the synchronization of delayed discontinuous systems. Finally, two numerical examples have been given to illustrate the usefulness of our results.