Solving general convex quadratic multi-objective optimization problems via a projection neurodynamic model

Abstract

A neural network model is constructed to solve convex quadratic multi-objective programming problem (CQMPP). The CQMPP is first converted into an equivalent single-objective convex quadratic programming problem by the mean of the weighted sum method, where the Pareto optimal solution (POS) are given by diversifying values of weights. Then, for given various values weights, multiple projection neural networks are employded to search for Pareto optimal solutions. Based on employing Lyapunov theory, the proposed neural network approach is established to be stable in the sense of Lyapunov and it is globally convergent to an exact optimal solution of the single-objective problem. The simulation results also show that the presented model is feasible and efficient.

Introduction

The CQMPP is considered as

$$\begin{array}{c}\hfill \text{minimize}f(x)=(f_1(x),f_2(x),...,f_K(x))\end{array}$$ 1

         subject to

$$\begin{array}{cc}\hfill & x\in S=\{x\in \Omega \mid Ex-g\le 0,Ax-b=0\},\hfill \end{array}$$ 2

where the objective functions are of the form

$$\begin{array}{c}\hfill f_k(x)=\frac{1}{2}{x}^T{Q}_{k}x+c_k^{T}x,\end{array}$$

$\Omega =\{x\in {\mathbb{R}}^n,l_i\le x_i\le h_i\text{for}i=1,2,...,n\},$ and some $h_i$ (or $-l_j$) could be $+\infty ,$ $Q_k\in {\mathbb{R}}^{n\times n}$ is a real symmetric positive semidefinite matrix, $E\in {\mathbb{R}}^{m\times n},$ $g\in {\mathbb{R}}^m$, $A\in {\mathbb{R}}^{l\times n},$ $\text{rank}(A)=l,(0\le l<n)$ and $b\in {\mathbb{R}}^l.$ It is clear that functions $f_k(k=1,...,K)$ are convex and twice continuous differentiable on open convex set $\mathcal{E}\subseteq {\mathbb{R}}^n$ including $\Omega .$ Throughout this paper, we assume The K objectives are conflicted with each other. Therefore, the target of CQMPP is to achieve a set of efficient solutions that are called Pareto set.

Multi-objective optimization problem has become an important research area for both scientists and researchers. In the multi-objective optimization problem, multi-objective functions need to be optimized simultaneously. In the case of multi-objectives, there may not necessarily exist a solution that is best with respect to all objectives because of confliction among objectives. Therefore, there usually exist a set of solutions for the multi-objective case which cannot simply be compared with each other. For such solutions, called no-dominated solutions or POS’s, no improvement is possible in any objective function without scarifying at least one of the other objective functions (Miettinen 2002; Copado-Mendez et al. 2014; Oberdiecka and Pistikopoulos 2016; Mian et al. 2015; Hwang and Masud 1979). For an introduction to multi-objective optimization we refer the reader to Chinchuluun and Pardalos (2007), Ehrgott (2005) and Ehrgott and Wiecek (2005). For more some recent new works and surveys of recent developments on multi-objective optimization problems one can see (Jiang et al. 2022; Pan et al. 2022; Zhang et al. 2022; Lai et al. 2022; Hu et al. 2022; Kabgani and Soleimani-damaneh 2022; Ghalavand et al. 2021; Shavazipour et al. 2021; Dominguez-Rios et al. 2021; Groetzner and Werner 2022).

The multi-objective problem is convex if all its objective functions are convex and its feasible set is convex. There are several methods for solving these kinds of problems. For instance, for multi-objective linear programming (MOLP) problems, researchers have developed a variety of methods for generating the efficient set or the non-dominated set such as multi-objective simplex methods, interior point methods, and objective space methods. Multi-objective simplex methods and interior point methods work in variable space to find efficient solutions, see the references in Ehrgott and Wiecek (2005). Benson in Benson (1998) has studied an outer approximation method to find the extended feasible set in objective space (and thereby the non-dominated set) of an MOLP. In Ehrgott et al. (2007) and Shao and Ehrgott (2008) some improvements for this algorithm have been suggested. Krichen et al. in Krichen et al. (2012) have considered a new method based on the concept of adjacencies between efficient extreme points for generating the entire efficient set for MOLPs in the outcome space. Matejas and Peric in Matejas and Peric (2014) have suggested a new iterative method based on the principles of game theory for solving MOLPs with an arbitrary number of decision makers. Antunes et al. in Antunes et al. (2016) have studied five multiobjective interactive methods for MOLP problems, which are representative of different strategies of search. Lu et al. in Lu et al. (2020) have solved an equivalent mixed integer programming problem to compute an optimal solution to the MOLP problems. Shao and Ehrgott in Shao and Ehrgott (2008) introduced an approximation version of Benson’s algorithm to sandwich the extended feasible set of an MOLP with an outer approximation and an inner approximation.

A more general state of MOLP is CQMPP as stated in (1)–(2). Problems of type (1)–(2) arise in many different applications, such as engineering, economics and biological systems (see Miettinen 2002; Copado-Mendez et al. 2014; Oberdiecka and Pistikopoulos 2016; Mian et al. 2015). One of the most well known strategies to obtain a Pareto point of (1)–(2) is the $\epsilon$-constraint method (Miettinen 2002). However in this strategy the solution of the CQMPP problem depends on the values of certain parameter, namely ${\epsilon }_j$. Hence, while many researchers consider the iterative solution of the resulting optimization problems for different parameter values (Capitanescu et al. 2015) and (Antipova et al. 2015), some attention has been given to the explicit calculation of the entire Pareto front via parametric programming, which solves optimization problems as a function and for a range of certain parameters. In Gass and Saaty (1955) and Yuf and Zeleny (1976), authors considered the case of linear cost functions, and in Papalexandri and Dimkou (1998) the case of a mixed-integer nonlinear multi-objective optimization problem was addressed. The case of quadratic cost functions was treated in Goh and Yang (1996) and Ghaffari-Hadigheh et al. (2010), although either only conceptually or for the case where the quadratic part remains constant. In Oberdiecka and Pistikopoulos (2016) authors introduced an algorithm for the approximate explicit solution of multi-objective optimization problems with general convex quadratic cost functions and linear constraints by multi-parametric programming. In Ehrgott et al. (2011), Ehrgott et al. extended the algorithm in Shao and Ehrgott (2008) to approximately solve CQMPP. De Santis and Eichfelder in De Santis and Eichfelder (2021) presented a branch-and-bound algorithm for minimizing multiple convex quadratic objective functions over integer variables. Shang and Yu in Shang and Yu (2011) presented a constraint shifting combined homotopy method for solving CQMPP with both equality and inequality constraints. Luquea et. al in Luquea et al. (2010) described an interactive procedural algorithm for CQMPP based upon the Tchebycheff method, Wierzbicki’s reference point approach, and the procedure of Michalowski and Szapiro. Zerfa and Chergui in Zerfa and Chergui (2022) presented a branch-and-bound based algorithm to generate all non-dominated points for a multiobjective integer programming problem with convex quadratic objective functions and linear constraints.

In the past decades, neural networks have been successfully applied to various fields such as signal processing, pattern recognition, fixed-point computations, control and other scientific areas. Mathematically, the neural dynamical approach converts an optimization problem into a dynamical system or an system of ordinary differential equations so that their equilibrium point corresponds to the optimal solution of the optimization problem. Then the optimal solution can be obtained by tracking the solution trajectory of the designed dynamical system based on hardware and software implementation. The literature contains many studies on neural networks for solving real-time optimization problems, (see Kennedy and Chua 1988; Rodriguez-Vazquez et al. 1990; Friesz et al. 1994; Gao 2004; Gao et al. 2005; Nazemi and Effati 2013; Nazemi 2012, 2013, 2011; Nazemi and Omidi 2012; Yang and Cao 2008; Xia 2004; Xia and Wang 2000; Nikseresht and Nazemi 2019, 2018; Karbasi et al. 2021; Feizi and Nazemi 2021; Karbasi et al. 2020; Nazemi and Sabeghi 2019; Nazemi and Mortezaee 2019; Nazemi 2019; Arjmandzadeh et al. 2019). Although each of these studies work successfully for a specific kind of optimization problems, to the best of our knowledge, there exist only one neural network approach that solves CQMPP (see Rizk-Allah and Abo-Sinna 2017 and Abo-Sinna and Rizk-Allah 2018). However, the structure of the proposed model is rather complicated and further simplification can be achieved. Thus, proposing an efficient neural network for solving CQMPP’s with good stability properties and convergence results is very necessary and meaningful. The main contributions of this study are as follows:

• (i)In the presented model, the primal and dual problems can be solved simultaneously.
• (ii)By the means of the weighted sum method, POSs can be obtained through using different weights.
• (iii)The possible convergence of the new network to the optimal solution when ${\sum }_{k=1}^K{w}_k{Q}_k$ is positive semidefinite, which is the limitation of the models in Nazemi and Effati (2013) and Nazemi (2012).

Motivated by the former discussions, in this paper, we consider a neural network approach for solving CQMPP’s. Firstly, the CQMPP is transformed to a single-objective convex quadratic programming by the means of weighted sum method. Based on the Karush-Kuhn-Tucker (KKT) optimality conditions and to obtain a set of pareto optimal solutions, a collective neural networks related to various values of weights are constructed. The equilibrium point of the neural network is proved to be equivalent to the KKT point of the convex programming. The existence and uniqueness of an equilibrium point of the neural network will be also analyzed. By constructing a suitable Lyapunov function, for an arbitrary initial point, the studied neural network is proved to be global stable is the sense of Lyapunov and is capable to obtain the optimal solutions for the obtained nonlinear programming with different values of weights. Further, the POSs of the CQMPP are generated by utilizing different weights.

This paper is organized as follows. In Sect. 2, some preliminaries of the multi-objective optimization are described. In Sect. 3, a collaborative neurodynamic strategy is addressed. A comparison of the suggested model with some previous neural networks are stated in Sect. 4. In Sect. 5, the stability of the equilibrium point and the convergence of the optimal solution are investigated. In Sect. 6, several illustrative examples and simulation results are given to show the feasibility of the raised model. Conclusions are given in Sect. 7.

Preliminaries of multi-objective optimization

This section provides the necessary mathematical background for multi-objective optimization problems (Miettinen 2002; Chankong and Haimes 1983).

Definition 1

The natural ordering cone is defined as follows:

$$\begin{array}{c}\hfill {\mathbb{R}}_{\geqq }^K=\{x\in {\mathbb{R}}^K:x_k\ge 0,k=1,2,...,K\}.\end{array}$$

For any $y,\overline{y}\in {\mathbb{R}}^K:$

$y<\overline{y}$ $⟺$ $y_k<\overline{y_k}$ for all $k=1,2,...,K,$

$y\leqq \overline{y}$ $⟺$ $y_k⩽\overline{y_k}$ for all $k=1,2,...,K,$

$y\le \overline{y}$ $⟺$ $y_k⩽\overline{y_k}$ for all $k=1,2,...,K$ and $y\ne \overline{y}.$

Due to the conflict of the objective functions, it is often not possible to find a single solution what would be optimal for all the objectives simultaneously. As a result, the solution of problem (1)–(2) is established by the concept of Pareto optimality, which refers to the solution that none of the objective functions can be enhanced without detriment to at least one of the other objective functions. A formal definition of Pareto optimality is given in the following.

Definition 2

A feasible solution $x^{\ast }\in S$ is called

• (i)an efficient (a Pareto optimal) solution for problem (1)–(2), if there is no other $x\in S$ such that $f(x)\le f(x^{\ast }).$
• (ii)a weakly efficient (a weakly Pareto optimal) solution for problem (1)–(2), if there is no other $x\in S$ such that $f(x)<f(x^{\ast }).$

In order to acquire Pareto optimal solutions, one widely used approach is the weighted sum method (Abd El-Waheda et al. 2010) and (Miettinen 2002), which transforms the multiple objectives into a single one by multiplying each objective with a weight. Given a weighting vector $w={(w_1,w_2,\dots ,w_K)}^{\top },$ problem (1)–(2) is converted into

$$\begin{array}{c}\hfill \text{minimize}\sum _{k=1}^K{w}_k{f}_k(x)\end{array}$$ 3

         subject to

$$\begin{array}{c}\hfill x\in S,w\in W=\{w\in {\mathbb{R}}^K\mid w_k\ge 0,\sum _{k=1}^K{w}_k=1\}.\end{array}$$ 4

Theorem 1

Miettinen (2002) If $x^{\ast }\in S$ is an optimal solution of the weighting problem (3)–(4) where either $w_k>0,k=1,\dots ,K$, or $x^{\ast }$ is a unique optimal solution, then $x^{\ast }$ is a Pareto optimal solution of the multi-objective program (1)–(2).

Theorem 2

Miettinen (2002) Let the multi-objecive optimization problem (1)–(2) be convex. If $x^{\ast }\in S$ is Pareto optimal, then there exists a weighting vector $w\in W$ which is a solution to the weighting problem (3)–(4).

The following assumptions hold throughout of this paper.

Assumption 1

The minimizer of each objective function $f_k$ exists.

Assumption 2

A finite solution of (3)–(4) satisfies the Slater condition (Avriel 1976), i.e., for a given $w\in W,$ there exists a $x^0\in {\mathbb{R}}^n$ such that

$$\begin{array}{c}\hfill E{x}^0-g<0,A{x}^0-b=0.\end{array}$$

Assumption 3

The $\{E_j^T|j=1,\dots ,m\}\cup \{A_q^T|q=1,\dots ,l\}$ are linear independent, where $E={\left(E_1,,,E_2,,,\dots ,,,E_m\right)}^T$ and $A={\left(A_1,,,A_2,,,\dots ,,,A_l\right)}^T$.

Assumption 4

In all parts, we assume $w\in W.$

The neural network model

In this section, we will construct a neural network for solving (3)–(4). First, we give a sufficient and necessary condition for the optimal solution of (3)–(4), which is the theoretical foundation for us to design the network for it.

Theorem 3

Bazaraa et al. (1993) $x^{\ast }\in S$ is an optimal solution of (3)–(4) if and only if there exist ${\lambda }^{\ast }\in {\mathbb{R}}^m$ and ${\mu }^{\ast }\in {\mathbb{R}}^l$ such that ${({x^{\ast }}^T,{{\lambda }^{\ast }}^T,{{\mu }^{\ast }}^T)}^T$ satisfies the following KKT system

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\lambda }^{\ast }\ge 0,E{x}^{\ast }-g\le 0,{{\lambda }^{\ast }}^T(E{x}^{\ast }-g)=0,\hfill \\ {\sum }_{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)+E^T{\lambda }^{\ast }+A^T{\mu }^{\ast }=0,\hfill \\ A{x}^{\ast }=b,w\in W.\hfill \end{array}\right)\end{array}$$ 5

In order to gain a projection neural network, we state the KKT conditions (5) in variational inequality form.

Theorem 4

Gao (2004) $x^{\ast }\in S$ is an optimal solution of (3)–(4), if and only if there exists $u^{\ast }={(u_1^{\ast },u_2^{\ast },\cdots ,u_m^{\ast })}^T\in {\mathbb{R}}^m$ and ${\mu }^{\ast }={(v_1^{\ast },v_2^{\ast },\cdots ,v_l^{\ast })}^T\in {\mathbb{R}}^l$ such that ${({x^{\ast }}^T,{u^{\ast }}^T,{v^{\ast }}^T)}^T$ satisfies the following KKT conditions:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{(x-x^{\ast })}^T({\sum }_{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)+E^T{u}^{\ast }-A^T{v}^{\ast })\ge 0,\forall x\in \Omega ,\hfill \\ u_j^{\ast }{(E{x}^{\ast }-g)}_j=0,u_j^{\ast }\ge 0,{(E{x}^{\ast }-g)}_j\le 0,j=1,...,m,A{x}^{\ast }-b=0.\hfill \end{array}\right)\end{array}$$ 6

Lemma 1

Youshen and Wang (2004) $x^{\ast }$ is an optimal solution of (3)–(4) if and only if there exists $u^{\ast }={(u_1^{\ast },u_2^{\ast },\cdots ,u_m^{\ast })}^T\in {\mathbb{R}}^m$ and $v^{\ast }={(v_1^{\ast },v_2^{\ast },\cdots ,v_l^{\ast })}^T\in {\mathbb{R}}^l$ such that:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{x}^{\ast }-P_{\Omega }[x^{\ast }-{\sum }_{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)-E^T{u}^{\ast }+A^T{v}^{\ast }]=0,\hfill \\ u^{\ast }-{[u^{\ast }+(E{x}^{\ast }-g)]}^{+}=0,A{x}^{\ast }-b=0,\hfill \end{array}\right)\end{array}$$ 7

where $P_{\Omega }(x)=[{(x_1)}_{l_1}^1,{(x_2)}_{l_2}^2,\dots ,{(x_n)}_{l_n}^n]\in {\mathbb{R}}^n,{(x_i)}_{l_i}^i=min\{h_i,max\{x_i,h_i\}\}$ and ${(x_i)}^{+}=max\{0,x_i\}.$

Lemma (1) indicates that the optimal solution of (3)–(4) can be obtained by solving (7).

Lemma 2

Kinderlehrer and Stampacchia (2000) Assume that the set $\Omega \subset {\mathbb{R}}^n$ is a closed convex set. Then

$$\begin{array}{cc}& {(V-P_{\Omega }(V))}^T(P_{\Omega }(V)-U)\ge 0,V\in {\mathbb{R}}^n,U\in \Omega ,\hfill \end{array}$$ 8

$$\begin{array}{cc}& \Vert P_{\Omega }(U)-P_{\Omega }(V)\Vert \le \Vert U-V\Vert ,U,V\in {\mathbb{R}}^n.\hfill \end{array}$$ 9

A basic property of the projection mapping on a closed convex set is stated in Lemma (2).

For simplicity, we denote

$$\begin{array}{cc}& z(t)={(x^T,u^T,v^T)}^T\in {\mathbb{R}}^{n+m+l},\hfill \\ \hfill & D=\Omega \times {\mathbb{R}}^{m+l},\hfill \\ \hfill & \overline{u}={[u+(Ex-g)]}^{+},\hfill \\ \hfill & \overline{x}=P_{\Omega }[x-\sum _{k=1}^K{w}_k(Q_{k}x+c_k)-E^{T}u+A^T(v-Ax+b)],\hfill \\ \hfill & D^{\ast }=\{z\in {\mathbb{R}}^{n+m+l}|z\text{solves}(7).\}\hfill \end{array}$$

Based on the above results and by an extension of the neural network in Gao (2004), we introduce the following architecture to solve (3)–(4):

$$\begin{array}{c}\hfill \frac{\mathit{dz}}{\mathit{dt}}=\frac{d}{\mathit{dt}}\left(\begin{array}{c}x\\ u\\ v\end{array}\right)=-\tau \left(\begin{array}{c}2(x-\overline{x})\\ u-\overline{u}\\ Ax-b\end{array}\right),\end{array}$$ 10

here $\tau$ is a scale parameter and indicates the convergence rate of the neural network (10). For simplicity of our analysis, we let $\tau =1.$

Let $D^e$ be the equilibrium point set of the neural network (10). From Lemma 1 we have that $x\in D^e$ is an equilibrium point of the neural network (10) if and only if $x\in D^{\ast }$ is an optimal solution of (3)–(4). That is, $D^{\ast }=D^e$.

In the following, we give a more technical description of the proposed neural network algorithm whose framework is shown in Algorithm 1. The procedures start with choosing an arbitrary initial vector. Then iterative procedures are controlled by the principal while-loop of Algorithm 1. In each iteration, calculate the right side of system (10), update initial vector, calculation of error and stopping criteria, must be scheduled. The pseudo code that explains the implementation of the presented algorithm is stated as follows:

Remark 1

By using the weighted sum method, a single–objective optimization problem of (1)–(2) can be formulated as (3)–(4), in which the scalar objective (3) is summation of weighted original objective functions. A system consisting of multiple neural networks is developed to cooperatively seek Pareto optimal solutions of the multi-objective optimization problem (1)–(2). Each neural network is required to be run for a fixed given weight $w\in W.$ In fact for $w\in W,$ each neural network optimizes ${\sum }_{k=1}^K{w}_k{f}_k(x)$ such that $x\in S,$ while seeks for a point of Pareto efficient frontier (Pareto optimal solution). Thus, for a given arbitrary value $w^i\in W$ and solving the single–objective problem as ith sub–problem, ith neural network with weight $w^i\in W$ is as

$$\begin{array}{c}\hfill \frac{d{z}^i}{\mathit{dt}}=\frac{d}{\mathit{dt}}\left(\begin{array}{c}{x}^i\\ u^i\\ v^i\end{array}\right)=-\left(\begin{array}{c}2(x^i-{\overline{x}}^i)\\ u^i-{\overline{u}}^i\\ A{x}^i-b\end{array}\right),\end{array}$$

where

$$\begin{array}{cc}& {\overline{u}}^i={[u^i+(E{x}^i-g)]}^{+},\hfill \\ \hfill & {\overline{x}}^i=P_{\Omega }[x^i-\sum _{k=1}^K{w}_k^i(Q_k{x}^i+c_k)-E^T{u}^i+A^T(v^i-A{x}^i+b)],\hfill \\ \hfill \end{array}$$

is adopted and updated to optimize the ith sub–problem. Each neural network is supplied with a $w^i\in W$ and it generates an equilibrium point after performing precise local search. That is, given a set of weights $w=\{w^1,w^2,\cdots ,w^{N_i}\}$ where $w^i\in W$ and $N_i$ is the total number of employed neural networks. The neural networks will output a set of equilibrium points $z=\{z^1,z^2,\cdots ,z^{\mathit{Ni}}\}.$ Necessary and sufficient conditions are satisfied to guarantee the convergence of the collaborative neurodynamic system to a Pareto optimal solution. Finally, Pareto optimal set is constructed by using different values of weights.

Stability analysis

In the section, we will study some stability and convergence properties for network (10). We now establish our main results as follows.

Theorem 5

For any initial point $z(t_0)={(x{(t_0)}^T,u{(t_0)}^T,v{(t_0)}^T)}^T\in {\mathbb{R}}^{n+m+l}$, there exists a unique continuous solution $z(t)={(x{(t)}^T,u{(t)}^T,v{(t)}^T)}^T$ for system (10). Moreover, $x(t)\in \Omega$ and $u(t)\ge 0$, provided that $x(t_0)\in \Omega$ and $u(t_0)\ge 0$.

Proof

The projection mappings $P_{\Omega }(.)$ and ${(.)}^{+}$ are non-expansive. Since $(Q_{k}x+c_k),k=1,2,\dots ,K$ and ${(Ex-g)}_j,j=1,2,\dots ,m$ are continuously differentiable on an open convex set $\mathcal{E}\subseteq {\mathbb{R}}^n$ including $\Omega$, then $x-P_{\Omega }[x-{\sum }_{k=1}^K{w}_k(Q_{k}x+c_k)-E^{T}u+A^T(v-Ax+b)],$ $u-{[u+(Ex-g)]}^{+}$ and $Ax-b$ are locally Lipschitz continuous. According to the local existence of ordinary differential equations, the initial value problem of the system (10) has a unique solution of (10) on $[t_0,T)$ as is its maximal interval of existence. Moreover, it can be seen in the proof of Theorem 4.4 that the solution z(t) is bounded and thus $T\to +\infty .$

Let the initial point $x_0=x(t_0)\in \Omega$ and $u_0=u(t_0)\ge 0.$ Since

$$\begin{array}{c}\hfill \left\{\begin{array}{c}\frac{\mathit{dx}}{\mathit{dt}}+x=P_{\Omega }[x-{\sum }_{k=1}^K{w}_k(Q_{k}x+c_k)-E^T\overline{u}-A^T(v-Ax+b)],\hfill \\ \frac{\mathit{du}}{\mathit{dt}}+u={[u+(Ex-g)]}^{+}.\hfill \end{array}\right)\end{array}$$ 11

We have

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\int }_{t_0}^t(\frac{\mathit{dx}}{\mathit{dt}}+x)e^{t}dt={\int }_{t_0}^t{P}_{\Omega }[x-{\sum }_{k=1}^K{w}_k(Q_{k}x+c_k)-E^T\overline{u}-A^T(v-Ax+b)]e^{t}dt,\hfill \\ {\int }_{t_0}^t(\frac{\mathit{du}}{\mathit{dt}}+u)e^{t}dt={\int }_{t_0}^t{[u+(Ex-g)]}^{+}{e}^{t}dt.\hfill \end{array}\right)\end{array}$$ 12

Then

$$\begin{array}{c}\hfill \left\{\begin{array}{c}x(t)=e^{-(t-t_0)}{x}_0+e^{-t}{\int }_{t_0}^t{e}^t{P}_{\Omega }[x-{\sum }_{k=1}^K{w}_k(Q_{k}x+c_k)-E^T\overline{u}-A^T(v-Ax+b)]dt,\hfill \\ u(t)=e^{-(t-t_0)}{u}_0+e^{-t}{\int }_{t_0}^t{e}^t{[u+(Ex-g)]}^{+}dt.\hfill \end{array}\right)\end{array}$$ 13

By the integration mean value theorem, we have:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}x(t)=e^{-(t-t_0)}{x}_0+(1-e^{-(t-t_0)})P_{\Omega }[\widehat{x}-{\sum }_{k=1}^K{w}_k(Q_k\widehat{x}+c_k)-E^T\widehat{u}-A^T(\widehat{v}-A\widehat{x}+b)],\hfill \\ u(t)=e^{-(t-t_0)}{u}_0+(1-e^{-(t-t_0)}){[\widehat{u}+E\widehat{x}-g]}^{+}.\hfill \end{array}\right)\end{array}$$ 14

It follows that $x(t)\in \Omega$ and $u(t)\ge 0$ from $x(t_0)\in \Omega$ and $u(t_0)\ge 0$. This completes the proof. $\square$

Remark 2

The results of Theorem 5 indicate that neural network (10) is well defined. In the proof of Theorem 5, we assume ${(x_0^T,u_0^T,v_0^T)}^T\in \Omega \times {\mathbb{R}}_{+}^m\times {\mathbb{R}}^l,$ that is to say, the initial point should be feasible. But if ${(x_0^T,u_0^T,v_0^T)}^T{)}^T\notin \Omega \times {\mathbb{R}}_{+}^m\times {\mathbb{R}}^l$, as $t\to +\infty$, ${(x{(t,x_0)}^T,u{(t,x_0)}^T,v{(t,v_0)}^T)}^T$ will eventually approach $\Omega \times {\mathbb{R}}_{+}^m\times {\mathbb{R}}^l$ by (14). So there exists a moment $t_0$,$t>t_0$ such that ${(x{(t,x_0)}^T,u{(t,x_0)}^T,v{(t,v_0)}^T)}^T$ can be regarded as a feasible point (or approximate) and ${(x{(t,x_0)}^T,u{(t,x_0)}^T,v{(t,v_0)}^T)}^T$ will stay in $\Omega \times {\mathbb{R}}_{+}^m\times {\mathbb{R}}^l$ from then on.

Lemma 3

Let ${({x^{\ast }}^T,{u^{\ast }}^T,{v^{\ast }}^T)}^T\in {\Omega }^e$ be an equilibrium point of (10) and

$$\begin{array}{c}\hfill V(z,z^{\ast })=\psi (z)-\psi (z^{\ast })-(z-z^{\ast })\mathrm{\nabla }\psi (z^{\ast })+\frac{1}{2}{\Vert z-z^{\ast }\Vert }^2,\end{array}$$ 15

where

$$\begin{array}{cc}& \psi (z)=\sum _{k=1}^K{w}_k{f}_k(x)+\frac{1}{2}{(\Vert \overline{u}\Vert }^2{+\Vert v-Ax+b\Vert }^2).\hfill \end{array}$$ 16

Then

• i)$V(z,z^{\ast })$ is convex on D and continuously differentiable on $E\times {\mathbb{R}}^{m+l}.$
• ii)
  $$\begin{array}{c}\hfill V(z,z^{\ast })\ge \frac{1}{2}{\Vert z-z^{\ast }\Vert }^2,\forall z\in D.\end{array}$$ 17

Proof

• i)It is easy to verify that ${\Vert v-Ax+b\Vert }^2$ is differentially convex on ${\mathbb{R}}^{n+l}.$ From Bazaraa et al. (1993), ${\Vert \overline{u}\Vert }^2$ is a convex differentiable on $\Omega \times {\mathbb{R}}^m$ and continuously differentiable on $\mathcal{E}\times {\mathbb{R}}^m.$ By the differentiable convexity of $f_k(x),k=1,2,\dots ,K,$ we see that $\psi (z)$ in (16) is also convex on D and continuously differentiable on $E\times {\mathbb{R}}^m\times {\mathbb{R}}^l.$ Thus i) is achieved.
• ii)
  Using the former analysis of i), it follows that

  $$\begin{array}{c}\hfill \psi (z)\ge \psi (z^{\ast })+{(z-z^{\ast })}^T\mathrm{\nabla }\psi (z^{\ast }),\forall z\in D.\end{array}$$

  By (15) we have

  $$\begin{array}{c}\hfill V(z,z^{\ast })\ge \frac{1}{2}{\Vert z-z^{\ast }\Vert }^2,\forall z\in D.\end{array}$$ 18

$\square$

The results in Lemma (3) pave a means for us to study the dynamics of neural network (10). In particular, the network (10) has the following basic property.

Theorem 6

The neural network (10) is stable in the sense of Lyapunov. Moreover, the solution trajectory of neural network (10) is extendable to the global existence.

Proof

By theorem 5, $\forall {(x_0^T,u_0^T,v_0^T)}^T\in \Omega \times {\mathbb{R}}_{+}^m\times {\mathbb{R}}^l$, there exists a unique continuous solution ${(x{(t)}^T,u{(t)}^T,v{(t)}^T)}^T\subseteq \Omega \times {\mathbb{R}}_{+}^m\times {\mathbb{R}}^l$ for system (10).

Define a Lyapunov function as (15). Noting that ${\Vert \overline{u}\Vert }^2={\Sigma }_{j=1}^m{[{(u_j+{(Ex-g)}_j)}^{+}]}^2$ and

$$\begin{array}{cc}\hfill & {[{(u_j+{(Ex-g)}_j)}^{+}]}^2\hfill \\ \hfill & =\left\{\begin{array}{cc}{[(u_j+{(Ex-g)}_j)]}^2,\hfill & u_j+{(Ex-g)}_j\ge 0,\hfill \\ 0,\hfill & o.w.\hfill \end{array}\right)\hfill \end{array}$$ 19

We have

$$\begin{array}{c}\hfill {\mathrm{\nabla }\Vert \overline{u}\Vert }^2=\mathrm{\nabla }(\sum _{j=1}^m{[{(u_j+{(Ex-g)}_j)}^{+}]}^2)=\left(\begin{array}{c}2E^T\overline{u}\\ 2\overline{u}\end{array}\right).\end{array}$$ 20

Using i) in Lemma (3) and (15) we get

$$\begin{array}{cc}\hfill & \mathrm{\nabla }\psi (z)=\hfill \\ \hfill & \left(\begin{array}{c}\sum _{k=1}^K{w}_k(Q_{k}x+c_k)+E^T\overline{u}-A^T(v-Ax+b)\\ \overline{u}\\ v-Ax+b\end{array}\right),\hfill \end{array}$$

and

$$\begin{array}{c}\hfill \mathrm{\nabla }V(z,z^{\ast })=z-z^{\ast }+\mathrm{\nabla }\psi (z)-\mathrm{\nabla }\psi (z^{\ast }).\end{array}$$

Thus, by (7),(10) and $\overline{u}-u=(Ex-g)+{(-u-(Ex-g))}^{+},$ $\forall z\in D$ one has

$$\begin{array}{cc}& \frac{dV(z,z^{\ast })}{\mathit{dt}}={\displaystyle \frac{\partial V(z,z^{\ast })}{\partial x}}{\displaystyle \frac{\mathit{dx}}{\mathit{dt}}}+{\displaystyle \frac{\partial V(z,z^{\ast })}{\partial u}}{\displaystyle \frac{\mathit{du}}{\mathit{dt}}}\hfill \\ \hfill & +{\displaystyle \frac{\partial V(z,z^{\ast })}{\partial v}}{\displaystyle \frac{\mathit{dv}}{\mathit{dt}}}\hfill \\ \hfill & =2{(x-\overline{x})}^T[x-x^{\ast }-\sum _{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)\hfill \\ \hfill & -E^T{u}^{\ast }+A^T{v}^{\ast }\hfill \\ \hfill & +\sum _{k=1}^K{w}_k(Q_{k}x+c_k)+E^T\overline{u}-A^T(v-Ax+b)]\hfill \\ \hfill & {-\Vert Ax-b\Vert }^2\hfill \\ \hfill & +2{(v-v^{\ast })}^T(Ax-b)+{(u-\overline{u})}^T(u-2u^{\ast }+\overline{u})\hfill \\ \hfill & =2{(\overline{x}-x^{\ast })}^T[x-\overline{x}+\sum _{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)+E^T{u}^{\ast }\hfill \\ \hfill & -A^T{v}^{\ast }-\sum _{k=1}^K{w}_k(Q_{k}x+c_k)-E^T\overline{u}\hfill \\ \hfill & +A^T{(v-Ax+b)]+2\Vert x-\overline{x}\Vert }^2\hfill \\ \hfill & +2{(x-x^{\ast })}^T[\sum _{k=1}^K{w}_k(Q_{k}x+c_k)+E^T\overline{u}\hfill \\ \hfill & -\sum _{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)-E^T{u}^{\ast }]\hfill \\ \hfill & {+\Vert Ax-b\Vert }^2+{\Vert u-\overline{u}\Vert }^2+2{(\overline{u}-u^{\ast })}^T\hfill \\ \hfill & [-(Ex-g)-{(-(Ex-g)-u)}^{+}]\hfill \\ \hfill & =2{(\overline{x}-x^{\ast })}^T[x-\overline{x}-\sum _{k=1}^K{w}_k(Q_{k}x+c_k)-E^T\overline{u}\hfill \\ \hfill & +A^T(v-Ax+b)]\hfill \\ \hfill & +2{(\overline{x}-x^{\ast })}^T[\sum _{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)+E^T{u}^{\ast }-A^T{v}^{\ast }]\hfill \\ \hfill & {+2\Vert x-\overline{x}\Vert }^2\hfill \\ \hfill & {+\Vert u-\overline{u}\Vert }^2+{\Vert Ax-b\Vert }^2+2{(x-x^{\ast })}^T\hfill \\ \hfill & [\sum _{k=1}^K{w}_k(Q_{k}x+c_k)-\sum _{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)]\hfill \\ \hfill & +2{\overline{u}}^T{[-(Ex-g)-(-(Ex-g)-u)}^{+}\hfill \\ \hfill & +E^T(x-x^{\ast })]\hfill \\ \hfill & -2{u^{\ast }}^T{[-(Ex-g)-(-(Ex-g)-u)}^{+}\hfill \\ \hfill & +E^T(x-x^{\ast })]\hfill \\ \hfill & \ge {2\Vert x-\overline{x}\Vert }^2+2{(x-x^{\ast })}^T[\sum _{k=1}^K{w}_k(Q_{k}x+c_k)\hfill \\ \hfill & -\sum _{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k)]-2{\overline{u}}^T(E{x}^{\ast }-g)\hfill \\ \hfill & {+\Vert u-\overline{u}\Vert }^2{+\Vert Ax-b\Vert }^2+2{\overline{u}}^T[(E{x}^{\ast }-g)\hfill \\ \hfill & -(Ex-g)+E^T(x-x^{\ast })]\hfill \\ \hfill & -2{u^{\ast }}^T[(E{x}^{\ast }-g)-(Ex-g)+E^T(x-x^{\ast })],\hfill \end{array}$$ 21

In the above calculations we used

$$\begin{array}{cc}\hfill & {[{(-u-(Ex-g))}^{+}]}^T\overline{u}=0,\hfill \\ \hfill & {(E{x}^{\ast }-g)}^T{u}^{\ast }=0,\hfill \\ \hfill & {[{(-u-(Ex-g))}^{+}]}^T{u}^{\ast }\ge 0,\hfill \\ \hfill & {(E{x}^{\ast }-g)}^T\overline{u}\le 0.\hfill \end{array}$$

Moreover, in the inequality of Lemma (2), let $V=x-{\sum }_{k=1}^K{w}_k(Q_{k}x+c_k)-E^T\overline{u}+A^T(v-Ax+b)$ and $U=x^{\ast }$, we obtain:

$$\begin{array}{cc}\hfill & \left(x,-,\sum _{k=1}^K,w_k,(Q_{k}x+c_k),-,E^T,\overline{u}\right)\hfill \\ \hfill & {\left(+,,,A^T,(v-Ax+b),-,\overline{x}\right)}^T(\overline{x}-x^{\ast })\ge 0.\hfill \end{array}$$ 22

From the differentiable convexities of $f_k(x),k=1,2,...,K,$ $\forall x\in \Omega$, we have

$$\begin{array}{c}\hfill [\sum _{k=1}^K{w}_k(Q_{k}x+c_k)-\sum _{k=1}^K{w}_k(Q_k{x}^{\ast }+c_k){]}^T(x-x^{\ast })\ge 0,\end{array}$$ 23

Substituting (6), (22) and (23) into (21), $\forall t\ge t_0$, one has

$$\begin{array}{cc}\hfill & \frac{dV(z,z^{\ast })}{\mathit{dt}}\le {-2\Vert x(t)-\overline{x}(t)\Vert }^2-{\Vert Ax(t)-b\Vert }^2\hfill \\ \hfill & {-\Vert u(t)-\overline{u}(t)\Vert }^2\le 0.\hfill \end{array}$$ 24

This means neural network (10) is globally stable in the sense of Lyapunov.

From (24) we know that V is a non-increasing function with respect to t, so

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{2}}{\Vert z(t)-z^{\ast }\Vert }^2\le V(z,z^{\ast })\le V(z(t_0),z^{\ast }),\forall t_0\le t\le T,\end{array}$$ 25

This shows the state trajectory of neural network (10) is bounded. Thus $T=+\infty$. $\square$

Theorem 7

The solution trajectory of the neural network (10) is globally convergent to an optimal point of the problem (3)–(4). In particular, neural network (10) is asymptotically stable when $D^{\ast }$ has only a point.

Proof

From (17), the level set $L(z_0)=\{z\in D|V(z,z^{\ast })\le V(z(t_0),z^{\ast })\}$ is bounded. Thus, there exists a convergent subsequence

$$\begin{array}{c}\hfill \{z(t_k)|t_0<t_1<...<t_k<t_{k+1}\},\text{and}{t}_k\to \infty \text{as}k\to \infty ,\end{array}$$

such that

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}z(t_k)=\overline{z},\end{array}$$ 26

where $\overline{z}$ satisfies

$$\begin{array}{c}\hfill \frac{dV(z,z^{\ast })}{\mathit{dt}}=0,\end{array}$$

which indicates that $\overline{z}$ is an $\omega$-limit point of $\{z(t)|t\ge t_0\}.$ Using the LaSalle Invariant Set Theorem (Miller and Michel 1982), one has that $\{z(t)\to M\}$ as $t\to \infty ,$ where M is the largest invariant set in $\Lambda =\{z(t)|\frac{dV(z,z^{\ast })}{\mathit{dt}}=0\}.$ From (10) and (24), it follows that $\frac{\mathit{dx}}{\mathit{dt}}=\frac{\mathit{du}}{\mathit{dt}}=\frac{\mathit{dv}}{\mathit{dt}}=0\iff \frac{dV(z,z^{\ast })}{\mathit{dt}}=0.$ Thus $\overline{z}\in D^{\ast }$ by $M\subseteq \Lambda \subseteq D^{\ast }.$

Substituting $z^{\ast }=\widehat{z}$ in (15), similar to the analysis above, we can prove that the function $V[z(t),\widehat{z}]$ is monotonically nonincreasing in $[t_0,+\infty )$, and

$$\begin{array}{c}\hfill V(z,\widehat{z})\ge {\Vert z-\widehat{z}\Vert }^2/2,\forall z\in D.\end{array}$$

From the continuity of $V(z,\widehat{z})$ and (26), $\forall \epsilon >0$, there exist a $t_q$ such that

$$\begin{array}{c}\hfill V[z(t),\widehat{z}]\le \frac{1}{2}{\epsilon }^2,whent\ge t_q.\end{array}$$ 27

Thus

$$\begin{array}{cc}\hfill & \Vert z(t)-\widehat{z}\Vert \le \sqrt{2V[z(t),\widehat{z}]}\le \sqrt{2V[z(t_q),\widehat{z}]}\le \epsilon ,\hfill \\ \hfill & whent\ge t_q.\hfill \end{array}$$

That is,${lim}_{k\to \infty }z(t)=\widehat{z}$. So, the solution trajectory of the neural network (10) is globally convergent to an equilibrium point $\widehat{z}$ is also an optimal solution of (3)–(4). In particular, if $D^{\ast }=\{z^{\ast }\}$, then for each $x_0\in \Omega$, v free in sign, and $u_0\ge 0$, the solution z with the initial point $z(t_0)=z^0$ will approach $z^{\ast }$ by the analysis above. That is, the neural network (10) is globally asymptotically stable. $\square$

Comparing with some existing models

In order to see how well the presented neural network (10) can be applied to solve (3)–(4), we compare it with some existing neural network models.

First, let us consider the following problem

$$\begin{array}{cc}\hfill & min\sum _{k=1}^K{w}_k{f}_k(x)\hfill \end{array}$$ 28

$$\begin{array}{cc}\hfill & \mathrm{s}.\mathrm{t}.Ex-g\le 0,\hfill \end{array}$$ 29

$$\begin{array}{cc}\hfill & Ax=b,x\in \Omega ,\hfill \end{array}$$ 30

where $f_k(x)=\frac{1}{2}{x}^T{Q}_{k}x+c_k^{T}x,$ $w\in W,x\in {\mathbb{R}}^n,$ $\Omega =\{x\in {\mathbb{R}}^n:l_i\le x_i\le h_i,i=1,2,\dots ,n\},$ and some $h_i$ (or $-l_j$) could be $+\infty ,$ $Q_k\in {\mathbb{R}}^{n\times n}$ is a real symmetric positive semidefinite matrix, $E\in {\mathbb{R}}^{m\times n},$ $g\in {\mathbb{R}}^m$, $A\in {\mathbb{R}}^{l\times n},$ $\text{rank}(A)=l,(0\le l<n)$ and $b\in {\mathbb{R}}^l.$ By extending the globally projected dynamic systems (Friesz et al. 1994) to constrained optimization problems by Xia in Xia (2004), a neural network can be utilized for solving (28)–(30) as

$$\begin{array}{cc}\hfill & \frac{\mathit{dx}}{\mathit{dt}}=\left(P_{\Omega },(x-,\left(\sum _{k=1}^K,w_k,(Q_{k}x+c_k),+,E^T,\lambda ,-,A^T,\mu )\right),-,x\right),\hfill \end{array}$$ 31

$$\begin{array}{cc}\hfill & \frac{d\lambda }{\mathit{dt}}=\left(-,\lambda ,+,{(\lambda +Ex-g)}^{+}\right),\hfill \end{array}$$ 32

$$\begin{array}{cc}\hfill & \frac{d\mu }{\mathit{dt}}=-Ax+b,\hfill \end{array}$$ 33

where $\Omega \subset {\mathbb{R}}^n$ is a closed convex set and $P_{\Omega }:{\mathbb{R}}^n\to \Omega$ is a projection operator (Gao 2004) defined by

$$\begin{array}{c}\hfill \begin{array}{c}{P}_{\Omega }(x_i)=\left\{\begin{array}{cc}{l}_i,\hfill & x_i<l_i,\hfill \\ x_i,\hfill & l_i\le x_i\le h_i,\hfill \\ h_i,\hfill & x_i>h_i.\hfill \end{array}\right)\end{array}\end{array}$$

It is well known that the asymptotical stability of the dynamic model (31) - (33) for a monotone and a symmetric mapping could not be guaranteed (Gao et al. 2005)and (Xia and Wang 2000). Thus, the system described by (31) – (33) cannot solve problem (28) – (30) in some cases. For instance, one can see Example 1 in this manuscript.

The proposed model in Nazemi and Effati (2013) can be used for constructing the following neural network for solving (3)–(4) as

$$\begin{array}{cc}\hfill & \frac{d(y(t))}{\mathit{dt}}=-\mathrm{\nabla }{E}_2(y(t)),\hfill \end{array}$$ 34

$$\begin{array}{cc}\hfill & y(0)=y_0,y(t)={(x(t),\lambda (t),\mu (t))}^T,\hfill \end{array}$$ 35

where

$$\begin{array}{cc}\hfill E_2(y)=& \frac{1}{2}{\Vert \rho (y)\Vert }^2,\hfill \end{array}$$ 36

$$\rho \left(y\right)=\left[\begin{array}{c}\sum _{k=1}^K{w}_k(Q_{k}x+c_k)+A^T\mu +E^T\lambda \hfill \\ {\varphi }_{\mathit{FB}}^{\epsilon }\left(\lambda ,-(Ex-g)\right)\hfill \\ b-Ax\hfill \end{array}\right],$$ 37

and

$${\varphi }_{\mathit{FB}}^{\epsilon }a,b)=(a+b)-\sqrt{a^2+b^2+\epsilon },\epsilon \to 0_{+}.$$

Using (Nazemi 2012), under the condition that $f_k(x)(k=1,...,K)$ is strictly convex, a lagrangian network model can be constructed for solving (3)–(4) as

$$\begin{array}{cc}\hfill & \frac{\mathit{dy}}{\mathit{dt}}=\Phi (y),\hfill \end{array}$$ 38

$$\begin{array}{cc}\hfill & y(t_0)=y_0,y(t)={(x(t),\lambda (t),\mu (t))}^T,\lambda (t_0)>0,\hfill \end{array}$$ 39

where

$$\begin{array}{c}\hfill \Phi (y)=\left[\begin{array}{c}-\left({\sum }_{k=1}^K,w_k,(Q_{k}x+c_k),+,\frac{1}{2},E^T,{\lambda }^2,+,A^T,\mu \right)\\ \text{diag}({\lambda }_1,\dots ,{\lambda }_m)(Ex-g)\\ Ax-b\end{array}\right].\end{array}$$

It should be noted that the convergence of the models (34)–(35) and (38)–(39) under the condition that ${\sum }_{k=1}^K{w}_k{Q}_k$ is positive definite matrix is only guaranteed. Thus, these models can not solve linear programming problems which limits their application. Moreover, the neural network (10) can converge to the optimal solution under the condition of positive semidefinite matrix ${\sum }_{k=1}^K{w}_k{Q}_k.$ For instance, one can see Example 6.1 in this manuscript.

Numerical simulations

In order to clarify the effectiveness of the proposed neural network (10), in this section, we test several examples. The simulation is conducted on Matlab R2022.

Example 1

Matejas and Peric (2014) Let $f_1(x):=50x_1+100x_2+17.5x_3,f_2(x):=50x_1+50x_2+100x_3,f_3(x):=20x_1+50x_2+100x_3$ and $f_4(x):=25x_1+75x_2+12x_3.$ Considering

$$minimizef(x):=(f_1(x),f_2(x),f_3(x),f_4(x))subjecttox\in S=\left\{\begin{array}{c}x\in {\mathbb{R}}^3|12x_1+17x_2\le 1400,\hfill \\ 3x_1+9x_2+8x_3\le 1000,\hfill \\ 10x_1+13x_2+15x_3\le 1750,\hfill \\ 6x_1+16x_3\le 1325,\hfill \\ 12x_2+7x_3\le 900,\hfill \\ 9.5x_1+9.5x_2+4x_3\le 1075,\hfill \\ x_1,x_2,x_3\ge 0.\hfill \end{array}\right\}$$ 40

All simulation results show that the state trajectories of the model (10) with

${(w_1,w_2,w_3,w_4)}^T={(0.3,0.3,0.3,0.1)}^T$ converge to the unique optimal solution $x^{\ast }={(45.22,49.61,43.52)}^T.$ Fig. 1 depicts that the trajectories of the network (10) converge to the optimal solution of the related single-objective problem.

Fig. 1.

Transient behavior of ${(x_1,x_2,x_3)}^T$ of the neural network (10) in Example 1

To make a comparison, we solve the MOLP problem by using neural network (31) - (33) shown in Fig. 2. It is clear that this model is not stable. The MOLP problem is also solved by using the model in (38) - (39). From Fig. 3, we see that this model is not suitable for solving the MOLP problem, too.

Fig. 2.

Divergent behavior of the system (31)–(33) in Example 1

Fig. 3.

Divergent behavior of the system (38) - (39) in Example 1

Example 2

Abo-Sinna and Rizk-Allah (2018)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \text{minimize}& f(x):=({(x_1-1)}^2+{(x_2-1)}^2,{(x_1-2)}^2+{(x_2-3)}^2)\hfill \\ \hfill & \mathrm{subject}\mathrm{to}\hfill \\ \hfill & x\in S=\left\{\begin{array}{c}x\in {\mathbb{R}}^2\mid x_1+2x_2-10\le 0,\\ x_1\ge 0,\\ 0\le x_2\le 4.\end{array}\right\}\hfill \end{array}\end{array}$$ 41

Table 1 shows results for different values of ${(w_1,w_2)}^T.$ All simulation results show that the state trajectories of the model (10) converge to the optimal solution $x^{\ast }$ by choosing different weight values ${(w_1,w_2)}^T.$ For example, Fig. 4 shows that the trajectories of the network (10) with 5 different initial points and weight ${(w_1,w_2)}^T={(0.65,0.35)}^T$ converge to the optimal solution of the related single-objective problem. The generated Pareto fronts using the established neural network approach is presented in Fig. 5.

Table 1.

Results for some values of ${(w_1,w_2)}^T$ in Example 2

Weight	Optimal solution		Objective function value
${(w_1,w_2)}^T$	$x_1^{\ast }$	$x_2^{\ast }$	$f_1$	$f_2$	$w_1{f}_1+w_2{f}_2$
${(0,1)}^T$	1.999999825	3.000000641	5.000002214	6.69E-13	6.69E-13
${(0.1,0.9)}^T$	1.899998993	2.800000072	4.049998447	0.050000173	0.45
${(0.2,0.8)}^T$	1.800000014	2.600000773	3.200002496	0.199999376	0.8
${(0.3,0.7)}^T$	1.699998518	2.400000889	2.450000415	0.449999822	1.05
${(0.4,0.6)}^T$	1.599998815	2.19999806	1.799993921	0.800004052	1.2
${(0.5,0.5)}^T$	1.499999145	1.999998341	1.249995827	1.250004173	1.25
${(0.6,0.4)}^T$	1.399999185	1.799998667	0.799997216	1.800004177	1.2
${(0.7,0.3)}^T$	1.29999882	1.600000145	0.449999466	2.450001247	1.05
${(0.8,0.2)}^T$	1.200001354	1.399999501	0.200000143	3.19999943	0.8
${(0.9,0.1)}^T$	1.100001094	1.2000013	0.050000739	4.04999335	0.45
${(1,0)}^T$	0.999999646	1.000001578	2.62E-12	4.999994397	2.62E-12

Fig. 4.

Transient behaviors of ${(x_1,x_2)}^T$ with $w_1=0.65,w_2=0.35$ for 5 different initial points of the neural network (10) in Example 2

Fig. 5.

Obtained POSs of the problem (41) using the neural network (10) in Example 2

Example 3

Ehrgott et al. (2011)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \text{minimize}& f(x):=({(x_1-1)}^2+{(x_2-1)}^2,{(x_1-2)}^2\hfill \\ \hfill & +{(x_2-3)}^2,{(x_1-4)}^2+{(x_2-2)}^2)\hfill \\ \hfill & \mathrm{subject}\mathrm{to}\hfill \\ \hfill & x\in S=\left\{\begin{array}{c}x\in {\mathbb{R}}^2|x_1+2x_2\le 10,\\ 0\le x_1\le 10,\\ 0\le x_2\le 4.\end{array}\right\}\hfill \end{array}\end{array}$$ 42

Table 2 depicts results for different values of ${(w_1,w_2,w_3)}^T.$ Fig. 6 shows the convergence behavior of x(t) based on the network (10) with 25 random initial points. Figure 7 illustrates that the feature of the proposed approach is contained in the convergent to Pareto fronts of the problem (42) by adopting different values of ${(w_1,w_2,w_3)}^T.$

Table 2.

Results for some values of ${(w_1,w_2,w_3)}^T$ in Example 3

Weight	Optimal solution		Objective function value
${(w_1,w_2,w_3)}^T$	$x_1^{\ast }$	$x_2^{\ast }$	$f_1$	$f_2$	$f_3$	$w_1{f}_1+w_2{f}_2+w_3{f}_3$
${(0.2,0.4,0.4)}^T$	2.6000022	2.199995	3.9999949	1.0000106	2.0000122	2
${(0.4,0.2,0.4)}^T$	2.3999969	1.8000035	2.5999969	1.5999892	2.5999793	2.4
${(0.4,0.4,0.2)}^T$	1.9999954	1.9999954	1.9999815	1.0000091	3.9999909	2
${(0.4,0.3,0.3)}^T$	2.199998	1.8999956	2.2499873	1.2500087	3.2500134	2.25
${(0.3,0.4,0.3)}^T$	2.2999970	2.0999956	2.8999826	0.9000061	2.8999882	2.1
${(0.3,0.3,0.4)}^T$	2.5000026	2.0000012	3.2500102	1.2500003	2.2499903	2.25
${(0.6,0.3,0.1)}^T$	1.5999985	1.7000049	0.8500051	1.8499883	5.8500030	1.65
${(0.1,0.6,0.3)}^T$	2.5000058	2.499999	4.5000142	0.5000067	2.5000206	1.5
${(0.1,0.3,0.6)}^T$	3.0999935	2.1999959	5.8499631	1.8499922	0.8500102	1.65
${(0.8,0.1,0.1)}^T$	1.3999980	1.2999987	0.2499976	3.2500068	7.2499837	1.25
${(0.1,0.8,0.1)}^T$	2.1000027	2.6999939	4.0999852	0.1000041	4.1000223	0.9
${(0.1,0.1,0.8)}^T$	3.4999962	2.0000035	7.2499877	3.2499814	0.2499981	1.25
${(0.6,0.2,0.2)}^T$	1.7999984	1.5999966	0.9999932	2.0000103	5.0000183	2

Fig. 6.

Transient behaviors of ${(x_1,x_2)}^T$ with $w_1=0.2,w_2=0.45,w_3=0.35$ for 25 different initial points of the neural network (10) in Example 3

Fig. 7.

Pareto front of the problem (42) using (10) in Example 3

Example 4

Oberdiecka and Pistikopoulos (2016)

$$\begin{array}{c}\hfill \begin{array}{cc}& minf(x):=(f_1(x),f_2(x),f_3(x))\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.Ax\le b\hfill \\ \hfill & x\in {\mathbb{R}}^2,\hfill \end{array}\end{array}$$ 43

where

$$\begin{array}{cc}\hfill f_1(x)=& x^T\left[\begin{array}{cc}2.5& 0\\ 0& 7.5\end{array}\right]x+{\left[\begin{array}{c}3\\ 0\end{array}\right]}^{T}x,\hfill \\ \hfill f_2(x)=& x^T\left[\begin{array}{cc}3.3& 0\\ 0& 8.5\end{array}\right]x+{\left[\begin{array}{c}1\\ -1\end{array}\right]}^{T}x-1,\hfill \\ \hfill f_3(x)=& x^T\left[\begin{array}{cc}3.5& 0\\ 0& 0.25\end{array}\right]x+2,\hfill \\ \hfill A=& \left[\begin{array}{cc}4& -3\\ 0& -3\\ -4& 2\\ 6& 0\\ -6& -2\\ -9& -1\end{array}\right],b=\left[\begin{array}{c}20\\ 14\\ 8\\ 20\\ 39\\ 17\end{array}\right]\hfill \end{array}$$

Table 3 depicts results for different values of ${(w_1,w_2,w_3)}^T.$ Fig. 8 shows the convergence behavior of x(t) based on the proposed neural network with 11 random initial points. Figure 9 demonstrates that the feature of the proposed approach is contained in the convergent to Pareto fronts of the problem (43) by adopting different values of ${(w_1,w_2,w_3)}^T.$

Table 3.

Results for some values of ${(w_1,w_2,w_3)}^T$ in Example 4

weight	optimal solution		objective function value
${(w_1,w_2,w_3)}^T$	$x_1^{\ast }$	$x_2^{\ast }$	$f_1$	$f_2$	$f_3$	$w_1{f}_1+w_2{f}_2+w_3{f}_3$
${(0.2,0.4,0.4)}^T$	$-$0.1552789	0.0400000	$-$0.39355	$-$1.10211	2.08479	0.31436024
${(0.4,0.2,0.4)}^T$	$-$0.2287582	0.0208333	$-$0.55219	$-$1.07321	2.18326	0.43778594
${(0.4,0.4,0.2)}^T$	$-$0.2649000	0.0310083	$-$0.61205	$-$1.05616	2.24584	$-$0.2181220
${(0.4,0.3,0.3)}^T$	$-$0.2467110	0.0266666	$-$0.58263	$-$1.06647	2.21320	0.11096710
${(0.3,0.4,0.3)}^T$	$-$0.2083334	0.0349344	$-$0.50734	$-$1.08966	2.15221	0.05759643
${(0.3,0.3,0.4)}^T$	$-$0.1910823	0.0306122	$-$0.47493	$-$1.09323	2.12802	0.38075848
${(0.6,0.3,0.1)}^T$	$-$0.3697149	0.0211998	$-$0.76405	$-$0.93602	2.47852	$-$0.49138443
${(0.1,0.6,0.3)}^T$	$-$0.1371947	0.0506329	$-$0.34530	$-$1.10392	2.06651	$-$0.07692767
${(0.1,0.3,0.6)}^T$	$-$0.0898201	0.0434781	$-$0.23511	$-$1.09060	2.02870	0.866532153
${(0.8,0.1,0.1)}^T$	$-$0.4664179	0.0072732	$-$0.85499	$-$0.75534	2.76142	$-$0.48338602
${(0.1,0.8,0.1)}^T$	$-$0.1697531	0.0528075	$-$0.41630	$-$1.10376	2.10155	$-$0.71448631
${(0.1,0.1,0.8)}^T$	$-$0.0591715	0.0277364	$-$0.16299	$-$1.06881	2.01244	1.48677679
${(0.6,0.2,0.2)}^T$	$-$0.3496514	0.0160000	$-$0.74139	$-$0.96003	2.42796	$-$0.15125035

Fig. 8.

Transient behaviors of ${(x_1,x_2)}^T$ with $w_1=0.1,w_2=0.4,w_3=0.5$ for 11 different initial points of the neural network (10) in Example 4

Fig. 9.

Pareto front of the problem (43) in Example 4

To end this section, we answer two natural questions: what are the practical and computational advantages of the network (10), compared to existing generally available algorithms for CQMPP’s? Are there advantages of our network compared to the existing ones? To answer these, we summarize what we have observed from numerical experiments and theoretical results as below.

• We compare framework (10) with some existing models in Sect. 5. At the first glance, we see that the difference of the numerical performance is very marginal by testing an optimization example.

• From Theorem 7, we observe that the solution converges with any initial point. Thus, changing initial points may not have much effect for our neural network model. In fact, our model is globally convergent to the optimal solution.

• The suggested neural network (10) can be implemented without a penalty parameter and can be convergent to an exact solution of the CQMPP’s.

Conclusion

In this paper we established a neural network model for CQMPPs. The provided neural network has been shown to be Lyapunov stable and converge globally to the related single-objective of the CQMPP. The convergence of the neural network has been validated with the simulation results of the CQMPP examples. The studied neural network does not also require any adjustable parameter and its structure is simple. Thus, the model is more suitable to be implemented in hardware. Moreover, since CQMPP problem has wide applications, the new model and the obtained results in this paper are significant in both theory and application. The CQMPP simulation results describe the effectiveness of the suggested framework.

Funding

The authors have not disclosed any funding.

Data Availability

Enquiries about data availability should be directed to the authors.

Declarations

Conflict of interest

Both of the authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with animals performed by any of the authors.