Partial Exactness for the Penalty Function of Biconvex Programming

Abstract

Biconvex programming (or inequality constrained biconvex optimization) is an important model in solving many engineering optimization problems in areas like machine learning and signal and information processing. In this paper, the partial exactness of the partial optimum for the penalty function of biconvex programming is studied. The penalty function is partially exact if the partial Karush–Kuhn–Tucker (KKT) condition is true. The sufficient and necessary partially local stability condition used to determine whether the penalty function is partially exact for a partial optimum solution is also proven. Based on the penalty function, an algorithm is presented for finding a partial optimum solution to an inequality constrained biconvex optimization, and its convergence is proven under some conditions.

1. Introduction

Multi-convex programming is a non-convex optimization problem [1,2], where biconvex programming is a special case. It is ubiquitous nowadays in fields such as control [3,4,5], machine learning [6,7], signal and information processing [8,9], communication [10,11], and also NP-hard problems [12]. The existing research on multi-convex programming mainly solves some very special models [12,13,14,15,16,17,18,19]. These studies all give specific methods for each special model. When problems change, new methods need to be studied. Biconvex programming is a simple case of multi-convex programming, and it is also studied in research [14] as a special case. Therefore, it is of great significance to study the algorithm of biconvex programming for solving practical engineering optimization problems.

Bilinear programming is the simplest case of biconvex programming and is the earliest and most studied as per [14]. There are two main ways to solve bilinear programming. One is the simplex algorithm based on sub-problems, and the other is the alternating direction method. For example, Liang and Bai [15] and Hajinezhad and Shi [16] both proposed the alternating direction method of multipliers (ADMM) algorithm for two special bilinear programming problems, where the extended Lagrangian penalty function uses a square penalty. Furthermore, Charkhgard et al. [17] presented a multi-linear programming algorithm using the linear programming algorithm.

In 2007, Gorski et al. [14] reviewed the development of the theory and algorithms of biconvex optimization. The biconvex functions have some good properties that are similar to convex functions, such as the biconvex separation theorem and an equivalence between local optimal solutions and stationary points. The biconvex programming algorithm is based on the idea that the solution to alternating sub-problems can converge to the stationary point of the original problem. For example, in 2015, Li et al. [18] studied an alternating convex search method to solve a stationary point problem of biconvex programming. In 2016, Shah et al. [19] presented an alternating search method with a square penalty function to solve biconvex programming, which had great effect in image recognition. Al-Khayyaltt and Falk [20] discussed an algorithm of biconvex programming using the idea of branch-and-bound.

In short, based on the above, the mainstream method used to solve biconvex programming is an alternative to subproblem solving, because the bi-convexity of biconvex programming guarantees the convergence of alternative search methods and effectually cuts down the scale of problem solving via decomposition calculation. Hence, in view of the large scale of biconvex programming(BCP), alternative subproblem solving will be the main method adopted in further research regarding biconvex programming.

In this paper we consider biconvex programming with inequality constraints as follows:

$$\begin{array}{ccc}\hfill (\mathrm{BCP})& min& h({\mathit{u}}_1,{\mathit{u}}_2)\hfill \\ & \mathrm{s}.\mathrm{t}.& g_i({\mathit{u}}_1,{\mathit{u}}_2)\le 0,i=1,2,\cdots ,m,\hfill \end{array}$$

where $h,g_i:R^{n_1}\times R^{n_2}\to R(i=1,2,\cdots ,m)$ are biconvex if $h({\mathit{u}}_1,{\mathit{u}}_2)$ and $g_i({\mathit{u}}_1,{\mathit{u}}_2)$ are convex in ${\mathit{u}}_1\in R^{n_1}$ for every fixed ${\mathit{u}}_2$ and in ${\mathit{u}}_2\in R^{n_2}$ for every fixed ${\mathit{u}}_1$. Let $\mathit{u}=({\mathit{u}}_1,{\mathit{u}}_2)$ and $I=\{1,2,\cdots ,m\}$. The feasible set of (BCP) is denoted by:

$$\mathbf{U}=\{({\mathit{u}}_1,{\mathit{u}}_2)\in R^{n_1}\times R^{n_2}\mid g_i({\mathit{u}}_1,{\mathit{u}}_2)\le 0,i\in I\}.$$

Let

$$\mathbf{U}({\mathit{u}}_2)=\{{\mathit{u}}_1\in R^{n_1}\mid g_i({\mathit{u}}_1,{\mathit{u}}_2)\le 0,i\in I\}.$$

When ${\mathit{u}}_2$ is fixed, define the suboptimal problem:

$$\begin{array}{ccc}\hfill (\mathrm{BCP})({\mathit{u}}_2)& min& h({\mathit{u}}_1,{\mathit{u}}_2)\hfill \\ & \mathrm{s}.\mathrm{t}.& {\mathit{u}}_1\in \mathbf{U}({\mathit{u}}_2).\hfill \end{array}$$

Let

$$\mathbf{U}({\mathit{u}}_1)=\{{\mathit{u}}_2\in R^{n_2}\mid g_i({\mathit{u}}_1,{\mathit{u}}_2)\le 0,i\in I\}.$$

When ${\mathit{u}}_1$ is fixed, define the suboptimal problem:

$$\begin{array}{ccc}\hfill (\mathrm{BCP})({\mathit{u}}_1)& min& h({\mathit{u}}_1,{\mathit{u}}_2)\hfill \\ & \mathrm{s}.\mathrm{t}.& {\mathit{u}}_2\in \mathbf{U}({\mathit{u}}_1).\hfill \end{array}$$

Many practical engineering optimization problems can be transformed into biconvex programming, for example the two-cardinality sparse convex optimization problem and the nonconvex quadratic programming problem.

It is well known that an effective method to solve constrained optimization is by penalty function [21]. Its main idea is to transform a constrained optimization problem into a sequence of unconstrained optimization subproblems that are easier to solve. When the penalty function is not exact, many calculations are needed to solve unconstrained optimization subproblems so as to obtain an approximate solution to inequality constrained optimization. For example, it is proven that the exact penalty function method for solving constrained optimization is very efficient, as first proposed by Zangwill (1967) in [22]. In theory, if the penalty function is exact, to obtain an optimal solution to a constrained optimization problem only one unconstrained optimization subproblem is solved. Hence, the exact penalty function algorithm takes less time than the inexact penalty function algorithm. Additionally, this approach has recently received great attention from both theoretical and practical arenas. Many studies were later presented based on the exact penalty function algorithm, such as Rosenberg (1986) [23] and Di Pillo (1986) [24]. In addition, it is essential to determine the exactness of a penalty function under the stability condition [25,26]. Hence, this paper mainly focuses on the relationship between the partial exactness of the penalty function and the partial stability of biconvex programming. On the other hand, there exist other approaches to reduce constrained optimization problems to unconstrained ones; for example, the index method presented in the monograph [27] or the method for computable boundaries presented in [28]. This shows the significance of the study of the partial exact penalty function.

In order to ensure exactness, we propose the following penalty function:

$$H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )=h({\mathit{u}}_1,{\mathit{u}}_2)+\rho \sum _{i\in I}max{\{g_i({\mathit{u}}_1,{\mathit{u}}_2),0\}}^p,$$

where penalty parameter $\rho >0$ and $0<p\le 1$. By the definition of exactness in [29], a penalty function is exact at $p=0.5$, but is not exact at $p=1$, such that $\underset{x\in R^1}{min}f(x)=xs.t.x^2\le 0$. Hence, the exactness of an optimization problem depends on the structure of the problem. In this paper, we will study the exactness of a more extensive penalty function for biconvex programming than the one presented in [30].

The remainder of the paper is organized as follows. In Section 2, for a partial optimum solution, the partial exactness of the penalty function is proven under the partial Karush–Kuhn–Tucker (KKT) condition or the partial stableness condition. An algorithm is presented to find out a partial optimum solution to (BCP) with convergence.

2. Partial Exactness and a Penalty Function for (BCP)

According to Gorski et al. [14], defining the partial optimum of (BCP) is very meaningful. The concept of the partial optimum of (BCP) is given as follows:

Definition 1.

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathit{U}$. If:

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\le h({\mathit{u}}_1,{\mathit{u}}_2^{*}),\forall {\mathit{u}}_1\in \mathit{U}({\mathit{u}}_2^{*}),\end{array}$$ (1)

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\le h({\mathit{u}}_1^{*},{\mathit{u}}_2),\forall {\mathit{u}}_2\in \mathit{U}({\mathit{u}}_1^{*}),\end{array}$$ (2)

then $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is called a partial optimum of (BCP). A partial optimum $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ of (BCP) means that ${\mathit{u}}_1^{*}$ is an optimal solution to (BCP)(${\mathit{u}}_2^{*}$) and ${\mathit{u}}_2^{*}$ is an optimal solution to (BCP)(${\mathit{u}}_1^{*}$).

Next, let us give the equivalence of a partial optimum of (BCP) to a partial KKT point under some conditions.

Let $h,g_i:R^{n_1}\times R^{n_2}\to R,i\in I=\{1,2,\cdots ,m\}$ be biconvex and differentiable. Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathbf{U}$. If there are ${\alpha }_i(i=1,2,\cdots ,m)$ such that:

$$\begin{array}{c}\hfill \nabla h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+\sum _{i=1}^m{\alpha }_i\nabla g_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=0,\end{array}$$ (3)

$$\begin{array}{c}\hfill {\alpha }_i{g}_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=0,{\alpha }_i\ge 0,i=1,2,\cdots ,m,\end{array}$$ (4)

then $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a KKT point of (BCP).

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathbf{U}$. If there are ${\alpha }_i^{(1)},{\alpha }_i^{(2)}(i=1,2,\cdots ,m)$ such that:

$$\begin{array}{c}\hfill {\nabla }_{{\mathit{u}}_1}h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+\sum _{i=1}^m{\alpha }_i^{(1)}{\nabla }_{{\mathit{u}}_1}{g}_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=0,\end{array}$$ (5)

$$\begin{array}{c}\hfill {\nabla }_{{\mathit{u}}_2}h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+\sum _{i=1}^m{\alpha }_i^{(2)}{\nabla }_{{\mathit{u}}_2}{g}_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=0,\end{array}$$ (6)

$$\begin{array}{c}\hfill {\alpha }_i^{(1)}{g}_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=0,{\alpha }_i^{(2)}{g}_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=0,{\alpha }_i^1\ge 0,{\alpha }_i^2\ge 0,i=1,2,\cdots ,m,\end{array}$$ (7)

then $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a partial KKT point of (BCP).

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathbf{U}$. The constraint of (BCP) is called a partial Slater constraint qualification at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$, if there is $({\overline{\mathit{u}}}_1,{\overline{\mathit{u}}}_2)\in R^{n_1}\times R^{n_2}$ such that:

$$g_i({\mathit{u}}_1^{*},{\overline{\mathit{u}}}_2)<0,g_i({\overline{\mathit{u}}}_1,{\mathit{u}}_2^{*})<0,i=1,2,\cdots ,m.$$

In fact, if the Slater constraint qualification is satisfied for a convex programming, an optimal solution of the convex programming is equal to a KKT condition. For biconvex programming, we have the results in [28] as follows.

Theorem 1.

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathit{u}$. If (BCP) is satisfied with partial Slater constraint qualification at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$, then $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a partial optimum of (BCP) if and only if $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a partial KKT point of (BCP).

Corollary 1.

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathit{u}$ be a partial optimum of (BCP). If (BCP) is satisfied with partial Slater constraint qualification at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$, then $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a KKT point of (BCP) if and only if (5), (6), and (7) hold with ${\alpha }_i^1={\alpha }_i^2,i=1,2,\cdots ,m$.

Example 1.

Let the biconvex programming:

$$\begin{array}{ccc}\hfill (BCP1)min& & h(x,y)=xy\hfill \\ \hfill s.t.& & g_1(x,y)=1-x-y\le 0,\hfill \end{array}$$

where $h,g_1:R\times R\to R$. For $\forall \theta \in (0,1)$, it is clear that $(x,y)=(\theta ,1-\theta )$ is the partial KKT point and the partial optimum of (BCP1). If $y=1-x$, then $h(x,y)=x-x^2$. We have $h(x,y)\to -\infty$ as $x\to \infty$. Hence, a local optimal solution to (BCP1) cannot be solved, such as in $(x,y)=(\frac{1}{2},\frac{1}{2})$.

Example 1 means that if there is no optimal solution to biconvex programming, there may exist a partial KKT point or a partial optimum. It is obvious that an optimal solution to biconvex programming is the partial optimum. For biconvex programming, there is a partial optimum even if an optimal solution to biconvex programming is not obtained. Example 1 further indicates that the partial optimum is very important to biconvex programming.

Let $\rho >0$ be given. Consider the following optimization problem:

$$(\mathrm{BCP}({\mathit{u}}_2;\rho ))min{H}_p({\mathit{u}}_1,{\mathit{u}}_2,\rho ),\mathrm{s}.\mathrm{t}.{\mathit{u}}_1\in R^{n_1}.$$

where ${\mathit{u}}_1$ is a decision variable when ${\mathit{u}}_2$ is fixed, and

$$(\mathrm{BCP}({\mathit{u}}_1;\rho ))min{H}_p({\mathit{u}}_1,{\mathit{u}}_2,\rho ),\mathrm{s}.\mathrm{t}.{\mathit{u}}_2\in R^{n_2}.$$

where ${\mathit{u}}_2$ is a decision variable when ${\mathit{u}}_1$ is fixed.

Definition 2.

Let $({\overline{\mathit{u}}}_1^{*},{\overline{\mathit{u}}}_2^{*})\in R^{n_1}\times R^{n_2}$. If:

$$\begin{array}{c}\hfill H_p({\overline{\mathit{u}}}_1^{*},{\overline{\mathit{u}}}_2^{*},\rho )\le H_p({\mathit{u}}_1,{\overline{\mathit{u}}}_2^{*},\rho ),\forall {\mathit{u}}_1\in R^{n_1},\end{array}$$ (8)

$$\begin{array}{c}\hfill H_p({\overline{\mathit{u}}}_1^{*},{\overline{\mathit{u}}}_2^{*},\rho )\le H_p({\overline{\mathit{u}}}_1^{*},{\mathit{u}}_2,\rho ),\forall {\mathit{u}}_2\in R^{n_2},\end{array}$$ (9)

then $({\overline{\mathit{u}}}_1^{*},{\overline{\mathit{u}}}_2^{*})$ is called a partial optimum of $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$.

Next, we define the concept of a partially exact penalty function for biconvex programming as follows.

Definition 3.

(1) Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ be a partial optimum of (BCP). If there is a ${\rho }^{\prime }$ such that $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a partial optimum of $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ for $\forall \rho >{\rho }^{\prime }$, then $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is called a partially exact penalty function.

(2) Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ be a partial optimum of $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$. If $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ of $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is a partial optimum of (BCP), then ρ is called a partially exact value of the penalty parameter.

Example 2 below shows the partial exactness of a penalty function under different p values.

Example 2.

Let the biconvex programming:

$$\begin{array}{ccc}\hfill (BCP2)min& & h(x,y)=xy+x+y\hfill \\ \hfill s.t.& & x^2+y^2\le 0.\hfill \end{array}$$

It is clear that $(0,0)$ is a partial optimum of (BCP2). The penalty function for (BCP2) is defined by:

$$H_p(x,y;\rho )=xy+x+y+\rho max{\{x^2+y^2,0\}}^p.$$

When $p=1$, a partial optimum of $\underset{x,y\in R}{min}{H}_1(x,y;\rho )$ is $(x(\rho ),y(\rho ))=(-\frac{1}{2\rho +1},-\frac{1}{2\rho +1})$. Letting $\rho \to +\infty$, we have $(x(\rho ),y(\rho ))\to (0,0)$. Hence, $H_1(x,y,\rho )$ is not a partially exact penalty function. When $p=0.5$, we have:

$$H_{0.5}(x,y;\rho )=xy+x+y+\rho \sqrt{x^2+y^2}.$$

A partial optimum of $\underset{x,y\in R}{min}{H}_{0.5}(x,y;\rho )$ is $(0,0)$ for $\rho \ge 0$. Hence, $H_{0.5}(x,y,\rho )$ is a partially exact penalty function for $\rho \ge 1$. Example 2 means that the partial exactness of penalty function depends on the parameter p.

Example 3.

Let the biconvex programming:

$$\begin{array}{ccc}\hfill (BCP3)min& & h(x,y)=xy\hfill \\ \hfill s.t.& & xy\ge 1.\hfill \end{array}$$

It is easy to verify that $(x^{*},y^{*})={(\theta ,\frac{1}{\theta })}^T$ is an optimal solution to (BCP3) for any $\theta \ne 0$. Thus, $(x^{*},y^{*})={(\theta ,\frac{1}{\theta })}^T$ is a partial optimum of (BCP3) too. The example illustrates that all partial optimums of (BCP) may be optimal solutions.

$(x^{*},y^{*})={(1,1)}^T$ is a partial optimum of (BCP3). The square penalty function for (BCP3) is defined by:

$$H_2(x,y;\rho )=xy+\rho max{\{0,1-xy\}}^2.$$

It is easy to check that $(x^{*},y^{*})=(\frac{1}{\delta }(1-\frac{1}{2\rho }),\delta )$ is a partial optimum of $H_2(x,y,\rho )$ for $\delta \ne 0$ and $\rho \ge 1$. It is easy to check that $H_2(x,y;\rho )$ is not partially exact.

For $0<p\le 1$, a penalty function for (BCP3) is defined by:

$$H_p(x,y;\rho )=xy+\rho max{\{0,1-xy\}}^p.$$

We have that $(x^{*},y^{*})=(1,1)$ is a partial optimum of $H_p(x,y;\rho )$ for $\rho >1$. It is easy to check that $H_p(x,y;\rho )$ is partially exact for $\rho >1$. We easily check that $(1,1)$ is a KKT point.

Example 3 illustrates that the partially exact penalty function for a partial optimum can be as good as traditional exact penalty functions.

We prove the similarity of the partially exact penalty result to [28].

Theorem 2.

Let $h({\mathit{u}}_1,{\mathit{u}}_2)\ge 0,\forall ({\mathit{u}}_1,{\mathit{u}}_2)\in R^{n_1}\times R^{n_2}$. Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathit{U}$ be a partial optimum of (BCP). If $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a KKT partial point of (BCP), i.e., there are ${\alpha }_i^{(1)},{\alpha }_i^{(2)}(i=1,2,\cdots ,m)$ such that (5), (6), and (7) are true, then $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is a partially exact penalty function for $\rho >{\rho }^{*}$, where:

$${\rho }^{*}=max\{h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),{\alpha }_i^{(1)},{\alpha }_i^{(2)}|i=1,2,\cdots ,m\}.$$

Proof. 

If $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a partial KKT point of (BCP), there are ${\alpha }_i^{(1)}$ and ${\alpha }_i^{(2)}(i=1,2,\cdots ,m)$ such that (5), (6) and (7) are true. Because $h({\mathit{u}}_1,{\mathit{u}}_2),g_i({\mathit{u}}_1,{\mathit{u}}_2)(\forall i\in I)$ are biconvex functions for $\forall {\mathit{u}}_1\in R^{n_1}$ and $\forall {\mathit{u}}_2\in R^{n_2}$, we have:

$$\begin{array}{ccc}\hfill h({\mathit{u}}_1,{\mathit{u}}_2^{*})& \ge & h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+{\nabla }_{{\mathit{u}}_1}h{({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})}^T({\mathit{u}}_1-{\mathit{u}}_1^{*}),\hfill \end{array}$$ (10)

$$\begin{array}{ccc}\hfill g_i({\mathit{u}}_1,{\mathit{u}}_2^{*})& \ge & g_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+{\nabla }_{{\mathit{u}}_1}{g}_i{({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})}^T({\mathit{u}}_1-{\mathit{u}}_1^{*}),i\in I,\hfill \end{array}$$ (11)

$$\begin{array}{ccc}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2)& \ge & h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+{\nabla }_{{\mathit{u}}_2}h{({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})}^T({\mathit{u}}_2-{\mathit{u}}_2^{*}),\hfill \end{array}$$ (12)

$$\begin{array}{ccc}\hfill g_i({\mathit{u}}_1^{*},{\mathit{u}}_2)& \ge & g_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+{\nabla }_{{\mathit{u}}_2}{g}_i{({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})}^T({\mathit{u}}_2-{\mathit{u}}_2^{*}),i\in I.\hfill \end{array}$$ (13)

By (5), (7), (10), and (11), we have:

$$\begin{array}{ccc}\hfill h({\mathit{u}}_1,{\mathit{u}}_2^{*})& \ge & h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-\sum _{i=1}^m{\alpha }_i^{(1)}{\nabla }_{{\mathit{u}}_1}{g}_i{({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})}^T({\mathit{u}}_1-{\mathit{u}}_1^{*})\hfill \\ & \ge & h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})+\sum _{i=1}^m{\alpha }_i^{(1)}(g_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-g_i({\mathit{u}}_1,{\mathit{u}}_2^{*}))\hfill \\ & =& h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-\sum _{i=1}^m{\alpha }_i^{(1)}{g}_i({\mathit{u}}_1,{\mathit{u}}_2^{*}).\hfill \end{array}$$ (14)

Let $P(t)=max{\{t,0\}}^p,t\in R$, and $0<p\le 1$. Let us take $\rho >{\rho }^{*}$ with ${\rho }^{*}=max\{h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),$${\alpha }_i^{(1)},{\alpha }_i^{(2)}|i=1,2,\cdots ,m\}.$ If there is a $g_i({\mathit{u}}_1,{\mathit{u}}_2^{*})>1$, then $P(g_i({\mathit{u}}_1,{\mathit{u}}_2^{*}))>1$. We have:

$$\begin{array}{c}\hfill h({\mathit{u}}_1,{\mathit{u}}_2^{*})+\rho \sum _{i=1}^{m}P(g_i({\mathit{u}}_1,{\mathit{u}}_2^{*}))\ge {\rho }^{*}\ge h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),\end{array}$$

i.e.,

$$H_p({\mathit{u}}_1,{\mathit{u}}_2^{*},\rho )\ge H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho ).$$

Otherwise, if $g_i({\mathit{u}}_1,{\mathit{u}}_2^{*})\le 1$ for $\forall i\in I$, from (14) we have:

$$\begin{array}{ccc}\hfill h({\mathit{u}}_1,{\mathit{u}}_2^{*})+\rho \sum _{i=1}^{m}P(g_i({\mathit{u}}_1,{\mathit{u}}_2^{*}))& \ge & h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\hfill \\ & & +\sum _{i=1}^m(\rho max{\{g_i({\mathit{u}}_1,{\mathit{u}}_2^{*}),0\}}^p-{\alpha }_i^{(1)}{g}_i({\mathit{u}}_1,{\mathit{u}}_2^{*})).\hfill \end{array}$$

Hence, for $\rho >{\rho }^{*}$, we have:

$$\begin{array}{c}\hfill H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )\le H_p({\mathit{u}}_1,{\mathit{u}}_2^{*},\rho ).\end{array}$$ (15)

Similarly, for $\rho >{\rho }^{*}$, we have:

$$\begin{array}{c}\hfill H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )\le H_p({\mathit{u}}_1^{*},{\mathit{u}}_2,\rho ).\end{array}$$

By Definition 2, we have that $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is a partially exact penalty function for $\rho >{\rho }^{*}$.

□

Note. If there is a number A such that $h({\mathit{u}}_1,{\mathit{u}}_2)\ge A$ for any $({\mathit{u}}_1,{\mathit{u}}_2)\in R^{n_1}\times R^{n_2}$, then the conclusion of Theorem 2 holds too. If we let $h_1({\mathit{u}}_1,{\mathit{u}}_2)=h({\mathit{u}}_1,{\mathit{u}}_2)-A$, we have $h_1({\mathit{u}}_1,{\mathit{u}}_2)\ge 0$ on $R^{n_1}\times R^{n_2}$. It is clear that the problem

$$\begin{array}{ccc}\hfill {(\mathrm{BCP})}_1& min& h_1({\mathit{u}}_1,{\mathit{u}}_2)\hfill \\ & \mathrm{s}.\mathrm{t}.& g_i({\mathit{u}}_1,{\mathit{u}}_2)\le 0,i=1,2,\cdots ,m,\hfill \end{array}$$

is equal to the problem (BCP). Particularly, when $p=1$, we have the following conclusion.

Theorem 3.

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\in \mathit{U}$ be a partial optimum of (BCP). If $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a KKT partial point of (BCP), i.e., there are ${\alpha }_i^{(1)}$ and ${\alpha }_i^{(2)}(i=1,2,\cdots ,m)$ such that (5), (6), and (7) are true, then $H_1({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is a partially exact penalty function with $p=1$ for $\rho >{\rho }^{*}$, where:

$${\rho }^{*}=max\{{\alpha }_i^{(1)},{\alpha }_i^{(2)}|i=1,2,\cdots ,m\}.$$

Theorem 3 is consistent with Theorem 2 in [28].

Similar to that for a constrained penalty function presented in [25,26], the concept of stability for a penalty function of (BCP) is defined. Let ${\mathit{s}}_1=(s_1^1,s_2^1,\cdots ,s_m^1)$ and:

$$\mathbf{U}({\mathit{u}}_2,{\mathit{s}}_1)=\{{\mathit{u}}_1\in R^{n_1}\mid g_i({\mathit{u}}_1,{\mathit{u}}_2)\le s_i^1,i\in I\}.$$

When ${\mathit{u}}_2$ is fixed, define a perturbed problem:

$$\begin{array}{ccc}\hfill (\mathrm{BCP})({\mathit{u}}_2,{\mathit{s}}_1)& min& h({\mathit{u}}_1,{\mathit{u}}_2)\hfill \\ & \mathrm{s}.\mathrm{t}.& {\mathit{u}}_1\in \mathbf{U}({\mathit{u}}_2,{\mathit{s}}_1).\hfill \end{array}$$

Let ${\mathit{s}}_2=(s_1^2,s_2^2,\cdots ,s_m^2)$ and:

$$\mathbf{U}({\mathit{u}}_1,{\mathit{s}}_2)=\{{\mathit{u}}_2\in R^{n_2}\mid g_i({\mathit{u}}_1,{\mathit{u}}_2)\le s_i^2,i\in I\}.$$

When ${\mathit{u}}_1$ is fixed, define a perturbed problem:

$$\begin{array}{ccc}\hfill (\mathrm{BCP})({\mathit{u}}_1,{\mathit{s}}_2)& min& h({\mathit{u}}_1,{\mathit{u}}_2)\hfill \\ & \mathrm{s}.\mathrm{t}.& {\mathit{u}}_2\in \mathbf{U}({\mathit{u}}_1,{\mathit{s}}_2).\hfill \end{array}$$

Definition 4.

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ be a partial optimum of (BCP), and ${\mathit{u}}_{1{\mathit{s}}_1}^{*}$ and ${\mathit{u}}_{2{\mathit{s}}_2}^{*}$ be optimal solutions to (BCP)(${\mathit{u}}_2^{*},{\mathit{s}}_1$) and (BCP)(${\mathit{u}}_1^{*},{\mathit{s}}_2$), respectively, for any ${\mathit{s}}_1,{\mathit{s}}_2\in R^m$. If there is a ${\rho }^{\prime }>0$ such that:

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_{1{\mathit{s}}_1}^{*},{\mathit{u}}_2^{*})\le \rho |{\mathit{s}}_1|,\forall \rho >{\rho }^{\prime },\end{array}$$ (16)

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2}^{*})\le \rho |{\mathit{s}}_2|,\forall \rho >{\rho }^{\prime },\end{array}$$ (17)

where $|{\mathit{s}}_1|={\displaystyle \sum _{i=1}^m}P(s_i^1)$ and $|{\mathit{s}}_2|={\displaystyle \sum _{i=1}^m}P(s_i^2)$, then $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is partially stable at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$. Furthermore, if there are a $\gamma >0$ and a ${\rho }^{\prime }>0$ such that (16) and (17) hold for $|{\mathit{s}}_1|\le \gamma$ and $|{\mathit{s}}_2|\le \gamma$, then $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is partially locally stable at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$.

Theorem 4.

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ be a partial optimum of (BCP). If $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is partially stable, then $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is a partially exact penalty function at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$.

Proof. 

Let us prove that $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is a partially exact penalty function when $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is partially stable at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$. Suppose that $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is not a partially exact penalty function. According to the definition of partial stability, for any ${\mathit{s}}_1,{\mathit{s}}_2$, we obtain that there is a ${\rho }^{\prime }$ satisfying that:

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_{1{\mathit{s}}_1}^{*},{\mathit{u}}_2^{*})\le \rho |{\mathit{s}}_1|,\forall \rho >{\rho }^{\prime },\end{array}$$ (18)

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_2^{*},{\mathit{u}}_{2{\mathit{s}}_2}^{*})\le \rho |{\mathit{s}}_2|,\forall \rho >{\rho }^{\prime }.\end{array}$$ (19)

Then, there always exists some $\rho >{\rho }^{\prime }$ such that $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is not a partial optimum of $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$, i.e., there is some $({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{\prime })$ such that:

$$H_p({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*},\rho )<H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )=h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),$$

$$H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime },\rho )<H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )=h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}).$$

Thus,

$$h({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*})+\rho \sum _{i\in I}P(g_i({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*}))<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),$$

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime })+\rho \sum _{i\in I}P(g_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime }))<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}).$$

Suppose that ${\mathit{u}}_1^{\prime }\in \mathbf{U}({\mathit{u}}_2^{*}),{\mathit{u}}_2^{\prime }\in \mathbf{U}({\mathit{u}}_1^{*})$. We have:

$$h({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*})<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),$$

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime })<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}).$$

This implies that $h({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*})<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})<h({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*})$ and $h({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime })<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime })$, which shows that $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is not a partial optimum of (BCP). A contradiction occurs. Hence, ${\mathit{u}}_1^{\prime }\in \mathbf{U}({\mathit{u}}_2^{*})$ and ${\mathit{u}}_2^{\prime }\in \mathbf{U}({\mathit{u}}_1^{*})$ do not hold, and $\sum _{i\in I}P(g_i({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*}))>0$ or $\sum _{i\in I}P(g_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime }))>0$.

Let ${\mathit{s}}_1^{\prime }={(s_1^{1^{\prime }},s_2^{1^{\prime }},\dots ,s_m^{1^{\prime }})}^{\top }$ with $s_i^{1^{\prime }}=g_i({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*})$ and ${\mathit{s}}_{2^{\prime }}={(s_1^{2^{\prime }},s_2^{2^{\prime }},\dots ,s_m^{2^{\prime }})}^{\top }$ with $s_i^2=g_i({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime })$, $i=1,2,\cdots ,m$, and ${\mathit{u}}_{1{\mathit{s}}_1^{\prime }}^{*}$ and ${\mathit{u}}_{2{\mathit{s}}_2^{\prime }}^{*}$ be optimal solutions to (BCP)(${\mathit{u}}_2^{*},{\mathit{s}}_1^{\prime }$) and (BCP)(${\mathit{u}}_1^{*},{\mathit{s}}_2^{\prime }$), respectively. Then, $h({\mathit{u}}_{1{\mathit{s}}_1^{\prime }}^{*},{\mathit{u}}_2^{*})\le h({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*})$ and $h({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2^{\prime }}^{*})\le h({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime })$. Thus,

$$h({\mathit{u}}_{1{\mathit{s}}_1^{\prime }}^{*},{\mathit{u}}_2^{*})\le h({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*}),$$

$$h({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2^{\prime }}^{*})\le h({\mathit{u}}_1^{*},{\mathbf{x}}_2^{\prime }).$$

Therefore,

$$\begin{array}{ccc}\hfill h({\mathit{u}}_{1{\mathit{s}}_1^{\prime }}^{*},{\mathit{u}}_2^{*})+\rho \sum _{i\in I}P(s_i^{1^{\prime }})& \le & h({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*})+\rho \sum _{i\in I}P(s_i^{1^{\prime }})\hfill \\ & =& H_p({\mathit{u}}_1^{\prime },{\mathit{u}}_2^{*},\rho )<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),\hfill \end{array}$$

and:

$$\begin{array}{ccc}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2^{\prime }}^{*})+\rho \sum _{i\in I}P(s_i^{2^{\prime }})& \le & h({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime })+\rho \sum _{i\in I}P(s_i^{2^{\prime }})\hfill \\ & =& H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{\prime },\rho )<h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*}),\hfill \end{array}$$

which shows that:

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2^{\prime }}^{*})>\rho |{\mathit{s}}_1^{\prime }|,$$

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_{1{\mathit{s}}_1^{\prime }}^{*},{\mathit{u}}_2^{*})>\rho |{\mathit{s}}_2^{\prime }|,$$

where $|{\mathit{s}}_1^{\prime }|=\sum _{i\in I}P(s_i^{1^{\prime }})$ and $|{\mathit{s}}_2^{\prime }|=\sum _{i\in I}P(s_i^{2^{\prime }})$. These inequalities contradict (18) and (19). Hence, that $H_p(x,y,\rho )$ is not partially stable yields a contradiction with the assumption, which proves that $H_p(x,y,\rho )$ is a partially exact penalty function. □

Theorem 5.

Let $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ be a partial optimum of (BCP). If $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is a partially exact penalty function at $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$, then $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is partially locally stable. In particular, for $p=1$, $H_1({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is partially stable.

Proof. 

Let us prove that $H_p(\mathbf{x},\mathbf{y},\rho )$ is partially locally stable when $H_p(\mathbf{x},\mathbf{y},\rho )$ is a partially exact penalty function. According to the definition of a partially exact penalty function, if $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a partial optimum of (BCP), there always exist some $\rho >{\rho }^{\prime }$ such that:

$$\begin{array}{c}\hfill H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )\le H_p({\mathit{u}}_1,{\mathit{u}}_2^{*},\rho ),\forall {\mathit{u}}_1\in R^{n_1},\end{array}$$ (20)

$$\begin{array}{c}\hfill H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )\le H_p({\mathit{u}}_1^{*},{\mathit{u}}_2,\rho ),\forall {\mathit{u}}_2\in R^{n_2}.\end{array}$$ (21)

Let ${\mathit{u}}_{1{\mathit{s}}_1}^{*}$ and ${\mathit{u}}_{2{\mathit{s}}_2}^{*}$ be optimal solutions to (BCP)(${\mathit{u}}_2^{*},{\mathit{s}}_1$) and (BCP)(${\mathit{u}}_1^{*},{\mathit{s}}_2$), respectively, for any ${\mathit{s}}_1,{\mathit{s}}_2\in R^m$. By (20), and (21), we have:

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )\le H_p({\mathit{u}}_{1{\mathit{s}}_1}^{*},{\mathit{u}}_2^{*},\rho ),\end{array}$$ (22)

$$\begin{array}{c}\hfill h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})=H_p({\mathit{u}}_1^{*},{\mathit{u}}_2^{*},\rho )\le H_p({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2}^{*},\rho ).\end{array}$$ (23)

It is clear that for $s_i^1\le 1,s_i^2\le 1(i\in I)$, we have:

$$\begin{array}{c}\hfill P(g_i({\mathit{u}}_{1{\mathit{s}}_1}^{*},{\mathit{u}}_2^{*}))\le P(s_i^1),P(g_i({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2}^{*}))\le P(s_i^2),i\in I.\end{array}$$ (24)

From (22), (23), and (24), for $|{\mathit{s}}_1|\le m$ and $|{\mathit{s}}_2|\le m$:

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_{1{\mathit{s}}_1}^{*},{\mathit{u}}_2^{*})\le \rho |{\mathit{s}}_1|,\forall \rho >{\rho }^{\prime },$$

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})-h({\mathit{u}}_1^{*},{\mathit{u}}_{2{\mathit{s}}_2}^{*})\le \rho |{\mathit{s}}_2|,\forall \rho >{\rho }^{\prime }.$$

It follows from the definition that $H_p(x,y,\rho )$ is partially locally stable. □

Theorem 4 and Theorem 5 mean that the stability condition is sufficient for the partial exact penalty function, but the necessary condition of the partial exact penalty function is partially locally stable, as shown in the following example.

Example 4.

Let the perturbed problem (BCP3), and its (BCP3($x,s_1$)) and (BCP3($y,s_2$)):

$$\begin{array}{ccc}\hfill (BCP3)(x,s_1)\underset{y}{min}& & h(x,y)=xy\hfill \\ \hfill s.t.& & 1-xy\le s_1,\hfill \\ \hfill (BCP3)(y,s_2)\underset{x}{min}& & h(x,y)=xy\hfill \\ \hfill s.t.& & 1-xy\le s_2.\hfill \end{array}$$

It is clear that $(x^{*},y^{*})=(1,1)$ is a partial optimum of (MCP3) and its objective function value is 1. $x_{s_1}^{*}=1-s_1$ and $y_{s_2}^{*}=1-s_2$ are optimal solutions to (BCP3)($1,s_1$) and (BCP3)($1,s_2$), respectively, for any $s_1,s_2\in R$. Let $P(t)=max{\{t,0\}}^p$ and $s_1,s_2\le 1$ with $0<p<1$. For $\rho >1$,

$$h(x^{*},y^{*})-h(x^{*},y_{s_1}^{*})=1-(1-s_1)\le \rho P(s_1),$$

$$h(x^{*},y^{*})-h(x_{s_1}^{*},y^{*})=1-(1-s_2)\le \rho P(s_2),$$

then $H_p(x,y,\rho )$ is partially locally stable. When $p=1$, $H_1(x,y,\rho )$ is partially stable.

3. Partial Optimum Penalty Function Algorithm for (BCP)

Now, we present an algorithm to solve a partial optimum of (BCP) by solving the penalty function problem BCP($\rho )$ as follows.

$$\begin{array}{ccc}\hfill BCP(\rho )& min& H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )s.t.({\mathit{u}}_1,{\mathit{u}}_2)\in R^{n_1}\times R^{n_2}.\hfill \end{array}$$

Based on the above results, the Partial Optimum Penalty Function Algorithm (Algorithm 1) was designed to compute a partial optimum of (BCP).

We prove the convergence of the Algorithm 1 in Theorem 6. Let:

$$S(L,h)=\{({\mathit{u}}_1,{\mathit{u}}_2)\mid L\ge h({\mathit{u}}_1,{\mathit{u}}_2),\}$$

be a level set. If for any given $L>0$, $S(L,h)$ is bounded, then $S(L,h)$ is called bounded.

Algorithm 1: POPFA Algorithm

Step 1: Choose ${\rho }_1>0,N>1,0<p\le 1$ and $k=1$. Step 2: Solve $({\mathit{u}}_1^k,{\mathit{u}}_2^k)$ to be a partial optimum of $$min{H}_p({\mathit{u}}_1,{\mathit{u}}_2,{\rho }_k)s.t.({\mathit{u}}_1,{\mathit{u}}_2)\in R^{n_1}\times R^{n_2}.$$ Step 3: If $({\mathit{u}}_1^k,{\mathit{u}}_2^k)$ is feasible to (BCP), stop and $({\mathit{u}}_1^k,{\mathit{u}}_2^k)$ is a partial optimum of (BCP). Otherwise, ${\rho }_{k+1}=N{\rho }_k,k=:k+1$ and go to Step 2.

Theorem 6.

Let $\{({\mathit{u}}_1^k,{\mathit{u}}_2^k)\}$ be the sequence generated by the Algorithm 1 and $h(·)$ be continuous.

(i) If $\{({\mathit{u}}_1^k,{\mathit{u}}_2^k)\}(k=1,2,\cdots ,\overline{k})$ is a finite sequence (i.e., the Algorithm 1 stops at the $\overline{k}$-th iteration), then $({\mathit{u}}_1^{\overline{k}},{\mathit{u}}_2^{\overline{k}})$ is a partial optimum of (BCP).

(ii) Let $\{({\mathit{u}}_1^k,{\mathit{u}}_2^k)\}$ be an infinite sequence, sequence $\{H_p({\mathit{u}}_1^k,{\mathit{u}}_2^k,{\rho }_k)\}$ be bounded and the level set $S(L,h)$ be bounded. Then $\{({\mathit{u}}_1^k,{\mathit{u}}_2^k)\}$ is bounded and any limit point $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ of it is a partial optimum of (BCP).

Proof. 

(i) The conclusion is clear.

(ii) By the Algorithm 1, since $\{H_p({\mathit{u}}_1^k,{\mathit{u}}_2^k,{\rho }_k)\}$ is bounded as $k\to +\infty$, there must be some $L>0$ such that:

$$\begin{array}{ccc}\hfill L& >& H_p({\mathit{u}}_1^k,{\mathit{u}}_2^k,{\rho }_k)\hfill \\ & \ge & h({\mathit{u}}_1^k,{\mathit{u}}_2^k)+{\rho }_k\sum _{i=1}^{m}P(g_i({\mathit{u}}_1^k,{\mathit{u}}_2^k))\hfill \\ & \ge & h({\mathit{u}}_1^k,{\mathit{u}}_2^k).\hfill \end{array}$$ (25)

Since h is continuous and the level set $S(L,h)$ is bounded and closed, $\{({\mathit{u}}_1^k,{\mathit{u}}_2^k)\}$ and $\{h({\mathit{u}}_1^k,{\mathit{u}}_2^k)\}$ are bounded. Without loss of generality, suppose $({\mathit{u}}_1^k,{\mathit{u}}_2^k)\to ({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$. Hence, there is an $A>0$ such that $h({\mathit{u}}_1^k,{\mathit{u}}_2^k)>-A$. From (25), we have:

$$\begin{array}{c}\hfill \sum _{i=1}^{m}P(g_i({\mathit{u}}_1^k,{\mathit{u}}_2^k))\le \frac{1}{{\rho }_k}(L-h({\mathit{u}}_1^k,{\mathit{u}}_2^k))<\frac{L+A}{{\rho }_k}.\end{array}$$

We have ${\displaystyle \sum _{i=1}^m}P(g_i({\mathit{u}}_1^k,{\mathit{u}}_2^k))\to 0$ as ${\rho }_k\to +\infty$. Hence, $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a feasible solution to (BCP).

Let any ${\overline{\mathit{u}}}_1\in \mathbf{U}({\mathit{u}}_2^{*})$ and ${\overline{\mathit{u}}}_2\in \mathbf{U}({\mathit{u}}_1^{*})$. Since $({\mathit{u}}_1^k,{\mathit{u}}_2^k)$ is a partial optimum of

$$min{H}_p({\mathit{u}}_1,{\mathit{u}}_2,{\rho }_k)s.t.({\mathit{u}}_1,{\mathit{u}}_2)\in R^{n_1}\times R^{n_2},$$

we have:

$$h({\mathit{u}}_1^k,{\mathit{u}}_2^k)\le H_p({\mathit{u}}_1^k,{\mathit{u}}_2^k,{\rho }_k)\le H_p({\overline{\mathit{u}}}_1,{\mathit{u}}_2^k,{\rho }_k),$$

$$h({\mathit{u}}_1^k,{\mathit{u}}_2^k)\le H_p({\mathit{u}}_1^k,{\mathit{u}}_2^k,{\rho }_k)\le H_p({\mathit{u}}_1^k,{\overline{\mathit{u}}}_2,{\rho }_k).$$

Let $k\to +\infty$, and the above inequations are:

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\le h({\overline{\mathit{u}}}_1,{\mathit{u}}_2^{*}),$$

$$h({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})\le h({\mathit{u}}_1^{*},{\overline{\mathit{u}}}_2).$$

Hence, $({\mathit{u}}_1^{*},{\mathit{u}}_2^{*})$ is a partial optimum of (BCP). □

Theorem 6 means that the Algorithm 1 has good convergence in theory. From Theorem 4, when the penalty function is partially stable, the Algorithm 1 solves a single unconstrained optimization problem for the smaller penalty parameter $\rho$. Since the penalty function $H_p({\mathit{u}}_1,{\mathit{u}}_2,\rho )$ is nonsmooth for $0<p<1$, it is necessary to smooth the constrained nonsmooth term to design an effective algorithm. Therefore, the smoothing algorithm of a partial exact penalty function is worthy of further study. In fact, for $p=1$, we have published a paper regarding a smoothing partially exact penalty function algorithm [30], where two numerical examples show the proposed algorithm is effective for biconvex programming.

4. Conclusions

In this paper, we studied the partial optimum solution to biconvex programming using the penalty function, which is partially exact. The form of this penalty function is more general than that in [28]. We proved that the partial exactness of the penalty function for biconvex programming is equivalent to the partial KKT condition, and we proved that the partial exactness of the penalty function for biconvex programming is equivalent to partially local stability. Based on the penalty function, the Algorithm 1 was theoretically presented to solve a partial optimum solution to biconvex programming. The convergence of the algorithm was also proven. The Algorithm 1 may solve a partial optimum solution to biconvex programming under the smaller penalty parameter. In the future, we may study the smoothing problem of the partial exact penalty function and its algorithm.

Author Contributions

Formal analysis and methodology, M.J.; investigation and methodology, Z.M.; methodology, R.S. All authors have read and agreed to the published version of the manuscript.

Funding

This work is supported by the National Natural Science Foundation of China (grant No.11871434) and the Natural Science Foundation of Zhejiang Province (grant No.LY18A010031).

Conflicts of Interest

The authors declare no conflict of interest.