New generalized variable stepsizes of the CQ algorithm for solving the split feasibility problem

Abstract

Variable stepsize methods are effective for various modified CQ algorithms to solve the split feasibility problem (SFP). The purpose of this paper is first to introduce two new simpler variable stepsizes of the CQ algorithm. Then two new generalized variable stepsizes which can cover the former ones are also proposed in real Hilbert spaces. And then, two more general KM (Krasnosel’skii-Mann)-CQ algorithms are also presented. Several weak and strong convergence properties are established. Moreover, some numerical experiments have been taken to illustrate the performance of the proposed stepsizes and algorithms.

Introduction

Since the CQ algorithm for solving the split feasibility problem (SFP) was proposed [1] in order to get better convergence speed, much attention has been paid to improve the variable stepsize of CQ algorithm.

Let $H_1$ and $H_2$ be real Hilbert spaces, C and Q be nonempty closed convex subsets of $H_1$ and $H_2$, respectively, and $A:H_1\to H_2$ be a bounded linear operator. In this setting, the SFP [2] is formulated as finding a point x̂ with the requirement

$$\hat{x}\in C\text{and}A\hat{x}\in Q.$$ 1.1

Denote the set of solutions for the SFP by $\mathrm{\Gamma }=C\cap A^{-1}(Q)$, and define $f:H_1\to R$ by

$$f(x)=\frac{1}{2}{\parallel (I-P_Q)Ax\parallel }^2,$$

we see the function $f(x)$ is convex and continuously differentiable.

In [1, 3], Byrne proposed his CQ algorithm; it generates a sequence $\{x_n\}$ via the recursion

$$x_{n+1}=P_C(I-{\tau }_n{A}^{\ast }(I-P_Q)A{x}_n),n\ge 0,$$ 1.2

where initial $x_0\in H_1$ and ${\tau }_n\in (0,2/{\parallel A\parallel }^2)$, $P_C$ and $P_Q$ are the orthogonal projections onto C and Q, respectively.

Considering that the projection onto the nonempty closed convex sets C and Q might be hard to be implemented, Yang [4] and Xu [5] showed that if C and Q are level sets of convex functions, it just needs to project onto half-spaces $C_n$ and $Q_n$, thus the so-called relaxed CQ algorithm is reduced to the following formula:

$$x_{n+1}=P_{C_n}(I-{\tau }_n{A}^{\ast }(I-P_{Q_n})A{x}_n),n\ge 0.$$ 1.3

In (1.3), the projections $P_{C_n}$ and $P_{Q_n}$ have closed-form expressions, thus they are easily to be computed. Define $f_n:H_1\to \mathrm{R}$ by

$$f_n(x)=\frac{1}{2}{\parallel (I-P_{Q_n})Ax\parallel }^2,$$

then the convex objective $f_n$ is also differentiable and has a Lipschitz gradient given by

$$\mathrm{\nabla }{f}_n(x)=A^{\ast }(I-P_{Q_n})Ax.$$

Noting that when applying (1.2) and (1.3) to solve the practical problems such as signal processing and image reconstruction, which can be covered by the SFP, it is hard to avoid that a fixed stepsize related to the norm of A sometimes affects convergence of the algorithms. Therefore, in order not to compute the matrix inverse and the largest eigenvalue of the matrix $A^{T}A$ and have a sufficient decrease of the objective function at each iteration, people have invented various algorithms with a variable or self-adaptive stepsize. Since Qu [6] presented a searching method by adopting Armijo-like search, many similar methods have been proposed, such as [7–12] etc. However, through these methods, self-adaptive stepsize at each iteration can be achieved, most formats of them are becoming more complex, it is difficult to apply them to some practical problems, and this needs considerable time complexity, especially for the large-scale setting and sparse problem.

On the other hand, another way to construct the variable stepsize without calculating matrix norm was proposed by Yang in [13], that is,

$${\tau }_n:=\frac{{\rho }_n}{\parallel \mathrm{\nabla }f(x_n)\parallel },$$ 1.4

where $\{{\rho }_n\}$ is a sequence of positive real numbers such that

$$\sum _{n=0}^{\mathrm{\infty }}{\rho }_n=\mathrm{\infty },\sum _{n=0}^{\mathrm{\infty }}{\rho }_n^2<\mathrm{\infty }.$$

at the same time, the additional conditions that Q is a bounded subset and A is a full column rank matrix are required. Wang et al. [14] applied (1.4) to solve the SFP. Afterwards, in order to remove the two additional conditions, López et al. [15] introduced another choice of the stepsize sequence $\{{\tau }_n\}$ as follows:

$${\tau }_n:=\frac{{\rho }_n{f}_n(x_n)}{{\parallel \mathrm{\nabla }{f}_n(x_n)\parallel }^2},$$ 1.5

where $\{{\rho }_n\}$ is chosen in the interval $(0,4)$. Furthermore, in paper [16] we were aware that if $\mathrm{\nabla }{f}_n(x_n)=0$ in (1.5) for some $n\ge 1$, the corresponding algorithms in [15] have to be terminated. In this case, $x_n$ may not be in C and is not necessarily a solution of SFP (1.1). With this observation, we have introduced a stepsize sequence $\{{\tau }_n\}$ as follows:

$${\tau }_n:=\frac{{\rho }_n{f}_n(x_n)}{{(\parallel \mathrm{\nabla }{f}_n(x_n)\parallel +{\omega }_n)}^2},$$ 1.6

where $\{{\rho }_n\}$ is also chosen in the interval $(0,4)$ and $\{{\omega }_n\}$ is a sequence of positive numbers in $(0,1)$. This choice of the stepsize sequence makes the associated algorithms never terminate unless the solution of SFP has been found. However, there exists inconvenience dealing with the choice of the parameter ${\omega }_n$. Only when it is a small number, a similar convergence speed as adopting (1.5) can be guaranteed. After observing many experiments, the order of magnitude for ${\omega }_n$ usually about less than 10⁻⁵ can satisfy that, it closely relates with the bit of computer and the float precision of the calculation software.

In order to improve these and avoid the calculations of $f_n(x)$ or $\mathrm{\nabla }{f}_n(x_n)$, in this paper, we firstly propose two simpler choices of a stepsize as follows:

$${\tau }_n:=\frac{{\rho }_n{\parallel x_n-{\overline{x}}_n\parallel }^2}{{\parallel A{x}_n-A{\overline{x}}_n\parallel }^2}$$ 1.7

and

$${\tau }_n:=\frac{{\rho }_n{\parallel {\overline{x}}_n\parallel }^2}{{\parallel A{\overline{x}}_n\parallel }^2},$$ 1.8

where $\{{\rho }_n\}$ is chosen in the interval $(0,2)$, $x_n\ne {\overline{x}}_n$, and ${\overline{x}}_n\ne 0$. The advantages of our choices (1.7) and (1.8) lie in facts that they not only possess simpler formats easily to be calculated and implemented in practice, but also have significantly faster convergence speed, especially in a large-scale setting and sparse problem. Secondly, we present two generalized variable stepsizes as follows:

$${\tau }_n:=\frac{{\rho }_n{\parallel r(x_n)\parallel }^2}{{\parallel Ar(x_n)\parallel }^2}$$ 1.9

or

$${\tau }_n:=\frac{{\rho }_n{\parallel p(x_n)\parallel }^2}{{\parallel A^{T}p(x_n)\parallel }^2},$$ 1.10

where $\{{\rho }_n\}$ is chosen in the interval $(0,2)$, and $r(x_n)\ne 0$ and $p(x_n)\ne 0$. Consequently, stepsizes (1.5)-(1.8) are the special cases of (1.9) or (1.10), and many similar stepsize formats can be obtained from them.

Recently, Yao et al. [17] have applied (1.5) to an improved CQ algorithm with a generalized Halpern iteration. In paper [18], we have modified the relative parameters with satisfactory conditions. Then, in this paper, we combine the iterations in [17, 18] with the KM-CQ iterations in [19, 20]. We propose two more general KM-CQ algorithms with the generalized variable stepsize (1.9) or (1.10), and they can be used to approach the minimum norm solution of the SFP that solves special variational inequalities.

The rest of this paper is organized as follows. Section 2 reviews some propositions and known lemmas. Section 3 gives two modified CQ algorithms with simpler variable stepsizes and shows the weak convergence. Section 4 presents two general KM-CQ algorithms with the generalized variable stepsizes and proves the strong convergence. In Section 5, we include numerical experiments to testify the better performance of the proposed stepsizes and algorithms with typical problems of signal processing and image restoration. Finally, Section 6 gives some conclusions and further research aim.

Preliminaries

Let H be a real Hilbert space with the inner product $\langle \cdot ,\cdot \rangle$ and the norm $\parallel \cdot \parallel$, respectively. Let C be a nonempty closed convex subset of H. We will use the following notations:

• ‘→’ stands for strong convergence;

• ‘⇀’ stands for weak convergence;

• I stands for the identity mapping on H.

Recall that a mapping $T:C\to H$ is nonexpansive iff $\parallel Tx-Ty\parallel \le \parallel x-y\parallel$ for all $x,y\in C$.

A mapping $\psi :C\to H$ is said to be δ-contractive iff there exists a constant $\delta \in [0,1)$ such that

$$\parallel \psi (x)-\psi (y)\parallel \le \delta \parallel x-y\parallel$$

for all $x,y\in C$.

Recall also that the nearest point projection from H onto C, denoted by $P_C$, assigns to each $x\in H$ the unique point $P_{C}x\in C$ with the property $\parallel x-P_{C}x\parallel \le \parallel x-y\parallel$, $\mathrm{\forall }y\in C$. We collect the basic properties of $P_C$ as follows.

Proposition 2.1

[6, 21]

(${\mathrm{p}}_1$)

$\langle x-P_{C}x,y-P_{C}x\rangle \le 0$ for all $x\in H$, $y\in C$;

(${\mathrm{p}}_2$)

${\parallel P_{C}x-P_{C}y\parallel }^2\le \langle P_{C}x-P_{C}y,x-y\rangle$ for every $x,y\in H$;

(${\mathrm{p}}_3$)

${\parallel P_{C}x-P_{C}y\parallel }^2\le {\parallel x-y\parallel }^2-{\parallel (I-P_C)x-(I-P_C)y\parallel }^2$ for all $x,y\in H$;

(${\mathrm{p}}_4$)

$\langle (I-P_C)x-(I-P_C)y,x-y\rangle \ge {\parallel (I-P_C)x-(I-P_C)y\parallel }^2$ for all $x,y\in H$;

(${\mathrm{p}}_5$)

${\parallel P_{C}x-z\parallel }^2\le {\parallel x-z\parallel }^2-{\parallel P_{C}x-x\parallel }^2$ for all $x\in H$, $z\in C$.

In a Hilbert space H, the next following facts are well known.

Proposition 2.2

$\mathrm{\forall }x,y\in H$, $\mathrm{\forall }t\in \mathrm{R}$,

• (i) ${\parallel x\pm y\parallel }^2={\parallel x\parallel }^2\pm 2\langle x,y\rangle +{\parallel y\parallel }^2$;
• (ii) ${\parallel tx+(1-t)y\parallel }^2=t{\parallel x\parallel }^2+(1-t){\parallel y\parallel }^2-t(1-t){\parallel x-y\parallel }^2$.

For the SFP, we assume that the following conditions are satisfied in a Hilbert space [5]:

(i) The solution set of the SFP is nonempty.

(ii) The set C is given by

$$C=\{x\in H_1|c(x)\le 0\},$$

where $c:H_1\to \mathrm{R}$ is a convex function and C is nonempty.

The set Q is given by

$$Q=\{y\in H_2|q(y)\le 0\},$$

where $q:H_2\to \mathrm{R}$ is a convex function and Q is nonempty.

(iii) Assume that both c and q are subdifferentiable on $H_1$ and $H_2$, respectively. The subdifferentials are defined as follows:

$$\partial c(x)=\{\xi \in H_1|c(z)\ge c(x)+\langle \xi ,z-x\rangle \mathit{\text{ for all }}z\in H_1\}\ne \mathrm{\varnothing },$$

for all $x\in C$, and

$$\partial q(y)=\{\eta \in H_2|q(u)\ge q(y)+\langle \eta ,u-y\rangle \mathit{\text{ for all }}u\in H_2\}\ne \mathrm{\varnothing },$$

for all $y\in Q$.

(iv) We also assume that ∂c and ∂q are bounded on bounded sets. The set $C_n$ is given by

$$C_n=\{x\in H_1|c(x_n)+\langle {\xi }_n,x-x_n\rangle \le 0\},$$

where ${\xi }_n\in \partial c(x_n)$. The set $Q_n$ is constructed by

$$Q_n=\{y\in H_2|q(A{x}_n)+\langle {\eta }_n,y-A{x}_n\rangle \le 0\},$$

where ${\eta }_n\in \partial q(A{x}_n)$.

It is easily seen that $C\subseteq C_n$ and $Q\subseteq Q_n$ for all n.

The relaxed CQ algorithm [4] can be seen as a special case of the classical gradient projection method (GPM). To see this, we can consider the following convex minimization problem:

$$\underset{x\in C_n}{min}{f}_n(x).$$ 2.1

It is well known that $\hat{x}\in C_n$ is a solution of problem (2.1) if and only if

$$\langle \mathrm{\nabla }{f}_n(\hat{x}),x-\hat{x}\rangle \ge 0,\mathrm{\forall }x\in C_n.$$ 2.2

Also, we know that (2.2) holds true if and only if

$$\hat{x}=P_{C_n}(I-\tau \mathrm{\nabla }{f}_n)\hat{x},\mathrm{\forall }\tau >0.$$

Therefore, we use the GPM to solve the SFP, for any $x_0\in H_1$,

$$x_{n+1}=P_{C_n}[x_n-{\tau }_n\mathrm{\nabla }{f}_n(x_n)],n\ge 0,$$ 2.3

where ${\tau }_n\in (0,2/L)$, while L is the Lipschitz constant of $\mathrm{\nabla }{f}_n$. Noting that $L={\parallel A\parallel }^2$, we see that (2.3) is exactly the relaxed CQ algorithm (1.3) .

Proposition 2.3

see [22]

Let $\{{\alpha }_n\}$ be a sequence of nonnegative real numbers such that

$${\alpha }_{n+1}\le (1-t_n){\alpha }_n+t_n{b}_n,n\ge 1,$$

where $\{t_n\}$ is a sequence in $(0,1)$ and $b_n\in \mathbb{R}$ such that

• (i) ${\sum }_{n=1}^{\mathrm{\infty }}{t}_n=\mathrm{\infty }$;
• (ii) ${\overline{lim}}_n{b}_n\le 0$ or ${\sum }_{n=1}^{\mathrm{\infty }}|t_n{b}_n|<\mathrm{\infty }$.

Then ${\alpha }_n\to 0$.

Proposition 2.4

[23, 24]

For some countable index set Λ, we denote by ${\ell }_p={\ell }_p(\mathrm{\Lambda })$, $1\le p\le \mathrm{\infty }$. Soft-thresholding leads to the unique minimizer of a functional combining ${\ell }_2$ and $l_1$-norms,

$${\mathrm{S}}_{\mu }(a)=arg\underset{x\in {\ell }_2(\mathrm{\Lambda })}{min}({\parallel x-a\parallel }^2+2\mu {\parallel x\parallel }_1),$$ 2.4

where μ is a certain positive number and ${\mathrm{S}}_{\mu }$ is the soft-thresholding operation defined by ${\mathrm{S}}_{\mu }{(a)}_i=S_{\mu }(a_i)$, $i\in \mathrm{\Lambda }$, with

$$S_{\mu }(z)=\{\begin{array}{cc}z-\mu ,\hfill & z>\mu ,\hfill \\ 0,\hfill & |z|\le \mu ,\hfill \\ z+\mu ,\hfill & z<-\mu .\hfill \end{array}$$

For the ${\ell }_1$-ball $B_R=\{x\in {\ell }_2:{\parallel x\parallel }_1\le R\}$ with $R:={\parallel x^{\ast }\parallel }_1$, where $x^{\ast }\in {\ell }_2$ is the solution of problem (2.4), we replace the thresholding with the projection ${\mathrm{P}}_{B_R}$, and with a slight abuse of notation, we denote ${\mathrm{P}}_{B_R}$ by ${\mathrm{P}}_R$; then we introduce two properties of ${\ell }_2$-projections onto ${\ell }_1$-balls [25].

Lemma 2.1

For any $a\in {\ell }_2$ and for $\mu >0$, ${\parallel {\mathrm{S}}_{\mu }(a)\parallel }_1$ is a piecewise linear, continuous, decreasing function of μ; moreover, if $a\in {\ell }_1$, then ${\parallel {\mathrm{S}}_0(a)\parallel }_1={\parallel a\parallel }_1$ and ${\parallel {\mathrm{S}}_{\mu }(a)\parallel }_1=0$ for $\mu \ge {max}_i|a_i|$.

Lemma 2.2

If ${\parallel a\parallel }_1>R$, then the ${\ell }_2$ projection of a on the ${\ell }_1$-ball with radius R is given by ${\mathrm{P}}_R(a)={\mathrm{S}}_{\mu }(a)$, where μ (depending on a and R) is chosen such that ${\parallel {\mathrm{S}}_{\mu }(a)\parallel }_1=R$. If ${\parallel a\parallel }_1\le R$, then ${\mathrm{P}}_R(a)={\mathrm{S}}_0(a)=a$.

Next, we discuss a method to compute μ.

Proposition 2.5

[25]

For any $a\in \mathrm{\Omega }\subseteq {\ell }_2$, $dim(\mathrm{\Omega })=n$, sort the absolute values of the components of a by descending order, obtaining the rearranged sequence ${(a_i^{\ast })}_{i=1,\dots ,n}$. Then we perform a search to find k such that

$${\parallel {\mathrm{S}}_{a_k^{\ast }}(a)\parallel }_1=\sum _{i=1}^{k-1}(a_i^{\ast }-a_k^{\ast })\le R<\sum _{i=1}^k(a_i^{\ast }-a_{k+1}^{\ast })={\parallel {\mathrm{S}}_{a_{k+1}^{\ast }}(a)\parallel }_1$$

or equivalently,

$${\parallel {\mathrm{S}}_{a_k^{\ast }}(a)\parallel }_1=\sum _{i=1}^{k-1}i(a_i^{\ast }-a_{i+1}^{\ast })\le R<\sum _{i=1}^{k}i(a_i^{\ast }-a_{i+1}^{\ast })={\parallel {\mathrm{S}}_{a_{k+1}^{\ast }}(a)\parallel }_1.$$

Set $\nu :=k^{-1}(R-{\parallel {\mathrm{S}}_{a_k^{\ast }}(a)\parallel }_1)$, and $\mu :=a_k^{\ast }+\nu$, then

$$\begin{array}{rl}{\parallel S_{\mu }(a)\parallel }_1& =\sum _{i\in \mathrm{\Omega }}max(|a_i|-\mu ,0)=\sum _{i=1}^k(a_i^{\ast }-\mu )\\ & =\sum _{i=1}^{k-1}(a_i^{\ast }-a_k^{\ast })+k\nu ={\parallel S_{a_k^{\ast }}(a)\parallel }_1+k\nu =R.\end{array}$$

CQ algorithms with two simpler variable stepsizes

In this section, two simpler variable stepsizes are proposed below. The advantages of the two stepsizes, comparing with (1.5) and (1.6), are that neither prior information about the matrix norm A nor any other conditions on Q and A are required.

A simpler variable stepsize for CQ algorithm

We propose a new and simpler variable stepsize method for solving the feasibility problem. The algorithm is presented as follows.

Algorithm 3.1

For any initial data $x_0\in H_1$, $u\in H_1$ and $u\ne 0$, assume that the nth iterate $x_n$ has been constructed; then we compute the $(n+1)$ th iterate $x_{n+1}$ via the formula

$${\overline{x}}_n=P_{C_n}(t_{n}u+(1-t_n)x_n),$$ 3.1

$$x_{n+1}=P_{C_n}(x_n-{\tau }_n\mathrm{\nabla }{f}_n(x_n)),$$ 3.2

where

$${\tau }_n:=\frac{{\rho }_n{\parallel x_n-{\overline{x}}_n\parallel }^2}{{\parallel A{x}_n-A{\overline{x}}_n\parallel }^2},$$ 3.3

with $\{{\rho }_n\}\subset (0,2)$ and $\{t_n\}\subset (0,1)$. If $x_{n+1}=x_n$ or $A{x}_n=A{\overline{x}}_n$ for some $n\ge 0$, then $x_n$ is a solution of SFP (1.1) and the iteration stops; otherwise, we set $n:=n+1$ and go to (3.1) to compute the next iterate $x_{n+2}$.

Remark 3.1

We can easily approximate the upper bound λ of the eigenvalue interval to the symmetric matrix $A^{T}A$ from [1, 3], thus for any $x_{n\ge 0}$, we can obtain ${\tau }_n\in (0,2/\lambda )\subset (0,2/L)$, where L is the largest eigenvalue of $A^{T}A$.

Now we prove the convergence property of Algorithm 3.1.

Theorem 3.1

If $\mathrm{\Gamma }\ne \mathrm{\varnothing }$ and ${\underset{\_}{lim}}_n{\tau }_n(2-\lambda {\tau }_n)\ge \sigma >0$, the sequence $\{x_n\}$ generated by Algorithm  3.1 converges weakly to a solution of SFP (1.1).

Proof

Let $x^{\ast }$ be a solution of SFP, since $C\subseteq C_n$, $Q\subseteq Q_n$, thus $x^{\ast }=P_C(x^{\ast })=P_{C_n}(x^{\ast })$ and $A{x}^{\ast }=P_Q(A{x}^{\ast })=P_{Q_n}(A{x}^{\ast })$. It shows that $x^{\ast }\in C_n$ and $\mathrm{\nabla }{f}_n(x^{\ast })=0$ for all $n=0,1,\dots$ , using (3.2) and (${\mathrm{p}}_4$), we have

$$\begin{array}{rcl}{\parallel x_{n+1}-x^{\ast }\parallel }^2& =& {\parallel P_{C_n}(x_n-{\tau }_n\mathrm{\nabla }{f}_n(x_n))-x^{\ast }\parallel }^2\\ & \le & {\parallel (x_n-x^{\ast })-{\tau }_n\mathrm{\nabla }{f}_n(x_n)\parallel }^2\\ & =& {\parallel x_n-x^{\ast }\parallel }^2+{\tau }_n^2{\parallel \mathrm{\nabla }{f}_n(x_n)\parallel }^2-2{\tau }_n\langle x_n-x^{\ast },\mathrm{\nabla }{f}_n(x_n)\rangle \\ & =& {\parallel x_n-x^{\ast }\parallel }^2+{\tau }_n^2{\parallel \mathrm{\nabla }{f}_n(x_n)\parallel }^2\\ & & -2{\tau }_n\langle x_n-x^{\ast },\mathrm{\nabla }{f}_n(x_n)-\mathrm{\nabla }{f}_n\left(x^{\ast }\right)\rangle \\ & \le & {\parallel x_n-x^{\ast }\parallel }^2+{\tau }_n^{2}L{\parallel (I-P_{Q_n})A{x}_n\parallel }^2\\ & & -2{\tau }_n\langle A{x}_n-A{x}^{\ast },(I-P_{Q_n})A{x}_n-(I-P_{Q_n})A{x}^{\ast }\rangle \\ & \le & {\parallel x_n-x^{\ast }\parallel }^2+{\tau }_n^{2}L{\parallel (I-P_{Q_n})A{x}_n\parallel }^2\\ & & -2{\tau }_n{\parallel (I-P_{Q_n})A{x}_n-(I-P_{Q_n})A{x}^{\ast }\parallel }^2\\ & =& {\parallel x_n-x^{\ast }\parallel }^2+({\tau }_n^{2}L-2{\tau }_n){\parallel (I-P_{Q_n})A{x}_n\parallel }^2.\end{array}$$ 3.4

Combining with Remark 3.1, we know ${\tau }_n^{2}L-2{\tau }_n<0$, thus it implies that the sequence $\{{\parallel x_n-x^{\ast }\parallel }^2\}$ is monotonically decreasing and hence $\{x_n\}$ is bounded. Consequently, from (3.4) we get

$$\underset{n\to \mathrm{\infty }}{lim}{\parallel (I-P_{Q_n})A{x}_n\parallel }^2=0.$$ 3.5

Assume that x̄ is an accumulation point of $\{x_n\}$ and $\{x_{n_i}\}\to \overline{x}$, $\{x_{n_i}\}$ is a subsequence of $\{x_n\}$. Then from (3.5) it follows

$$\underset{n_i\to \mathrm{\infty }}{lim}{\parallel (I-P_{Q_{n_i}})A{x}_{n_i}\parallel }^2=0.$$ 3.6

Next we show $\overline{x}\in \mathrm{\Gamma }$.

Firstly, we show $\overline{x}\in C$. We prove it from two cases.

Case 1. ${lim}_{n_i\to \mathrm{\infty }}{x}_{n_i+1}\ne \overline{x}$. Without loss of generality, we may assume ${lim}_{n_i\to \mathrm{\infty }}{x}_{n_i+1}=\tilde{x}\ne \overline{x}$. Set $z_{n_i}=x_{n_i}-{\tau }_{n_i}\mathrm{\nabla }{f}_{n_i}(x_{n_i})$, so $x_{n_i+1}=P_{C_{n_i}}(z_{n_i})$ is the solution of the following programming:

$$min\frac{1}{2}{\parallel z-z_{n_i}\parallel }^2\text{s.t. }c(x_{n_i})+\langle {\xi }_{n_i},z-x_{n_i}\rangle \le 0.$$

By the Kuhn-Tucker condition, there exists a nonnegative number ${\nu }_{n_i}$ such that

$$x_{n_i+1}-z_{n_i}+{\nu }_{n_i}{\xi }_{n_i}=0,$$ 3.7

$${\nu }_{n_i}(c(x_{n_i})+\langle {\xi }_{n_i},x_{n_i+1}-x_{n_i}\rangle )=0.$$ 3.8

If there exist infinite $n_i$ such that ${\nu }_{n_i}=0$ or ${\xi }_{n_i}=0$, from (3.7) and (3.8) it leads to $\tilde{x}=\overline{x}$, so the contradiction happens. Therefore, ${\nu }_{n_i}>0$ and ${\xi }_{n_i}\ne 0$ for sufficiently large $n_i$. We go on to divide the discussion into two cases.

(1) If $inf\{\parallel {\xi }_{n_i}\parallel \}>0$, we may assume ${\xi }_{n_i}\to \overline{\xi }\ne 0$. Then $\overline{\xi }\in \partial c(\overline{x})$ by the lower semicontinuity of $\partial c(x)$. From (3.7) we have ${\nu }_{n_i}\to \langle \overline{\xi },\tilde{x}-\overline{x}\rangle /{\parallel \overline{\xi }\parallel }^2\stackrel{\mathrm{\Delta }}{=}\overline{\nu }$. Thus we obtain from (3.7) and (3.8)

$$\tilde{x}-\overline{x}+\overline{\nu }\overline{\xi }=0,$$ 3.9

$$\overline{\nu }(c(\overline{x})+\langle \overline{\xi },\tilde{x}-\overline{x}\rangle )=0.$$ 3.10

They mean that $\tilde{x}=P_{C(\overline{x})}(\overline{x})$.

Therefore, $\overline{x}\ne P_{C(\overline{x})}(\overline{x})$ by assumption. From (${\mathrm{p}}_5$) we have

$${\parallel \tilde{x}-x^{\ast }\parallel }^2\le {\parallel \overline{x}-x^{\ast }\parallel }^2-{\parallel \overline{x}-\tilde{x}\parallel }^2<{\parallel \overline{x}-x^{\ast }\parallel }^2.$$

Since $\{{\parallel x_n-x^{\ast }\parallel }^2\}$ is decreasing, both x̄ and x̃ are the accumulation points of $\{x_n\}$, then ${\parallel \tilde{x}-x^{\ast }\parallel }^2={\parallel \overline{x}-x^{\ast }\parallel }^2$. It is a contradiction, so this case will not occur.

(2) If $inf\{\parallel {\xi }_{n_i}\parallel \}=0$, then we get the lower semicontinuity of $\partial c(x)$. Since $c(x)$ is convex, then x̄ is a minimizer of $c(x)$ over $H_1$. Since $c(x^{\ast })\le 0$, then $c(\overline{x})\le c(x^{\ast })\le 0$. So $\overline{x}\in C$.

Case 2. ${lim}_{n_i\to \mathrm{\infty }}{x}_{n_i+1}=\overline{x}$. As in Case 1, one has (3.7) and (3.8) by the Kuhn-Tucker condition. If there exist infinite $n_i$ s.t. ${\nu }_{n_i}=0$ or ${\xi }_{n_i}=0$, then we have $P_{C_{n_i}}(z_{n_i})=z_{n_i}$, so $c(x_{n_i})+\langle {\xi }_{n_i},z_{n_i}-x_{n_i}\rangle \le 0$. Since

$$\underset{n_i\to \mathrm{\infty }}{lim}(z_{n_i}-x_{n_i})=\underset{n_i\to \mathrm{\infty }}{lim}{\tau }_{n_i}{A}^T(P_{Q_{n_i}}-I)A{x}_{n_i}=0,$$

then $c(\overline{x})\le 0$. Therefore, we have $\overline{x}\in C$.

Assume ${\nu }_{n_i}>0$ and ${\xi }_{n_i}\ne 0$ for sufficiently large $n_i$. If $inf\{\parallel {\xi }_{n_i}\parallel \}=0$, such as above, it follows $\overline{x}\in C$. If $inf\{\parallel {\xi }_{n_i}\parallel \}>0$, similar to Case 1(1), it leads to $\overline{x}=P_{C(\overline{x})}(\overline{x})$, which implies $c(\overline{x})+\langle \overline{\xi },\overline{x}-\overline{x}\rangle \le 0$. So $\overline{x}\in C$.

In summary, we can conclude $\overline{x}\in C$.

Secondly, we need to show $A\overline{x}\in Q$. From (3.5) we have

$$\underset{n_i\to \mathrm{\infty }}{lim}{\parallel (I-P_{Q_{n_i}})A{x}_{n_i}\parallel }^2=0.$$ 3.11

Since $P_{Q_{n_i}}(A{x}_{n_i})\in Q_{n_i}$, we have

$$q(A{x}_{n_i})+\langle {\eta }_{n_i},P_{Q_{n_i}}(A{x}_{n_i})-A{x}_{n_i}\rangle \le 0.$$

Moreover, limiting the inequality and taking account of (3.11), we obtain that

$$q(A\overline{x})\le 0,$$

that is, $A\overline{x}\in Q$.

Therefore x̄ is a solution of SFP. Thus we may replace $x^{\ast }$ in (3.4) with x̄, and get $\{\parallel x_n-\overline{x}\parallel \}$ is convergent. Because there exists a subsequence $\{\parallel x_{n_i}-\overline{x}\parallel \}$ convergent to 0, then $x_n\to \overline{x}$ as $n\to \mathrm{\infty }$. □

The other simpler variable stepsize for CQ algorithm

In this part, we introduce the other simpler choice of the stepsize ${\tau }_n$, which also is a variable stepsize to CQ algorithm. Either combining with the relaxed CQ algorithm [4], we have the next algorithm.

Algorithm 3.2

Choose the initial data $\mathrm{\forall }{x}_0\in H_1$, for $u\in H_1$ and $u\ne 0$. Assume that the nth iterate $x_n$ has been constructed; then we compute the $(n+1)$ th iterate $x_{n+1}$ via the formula

$${\overline{x}}_n=P_{C_n}(t_{n}u+(1-t_n)x_n),$$ 3.12

$$x_{n+1}=P_{C_n}(x_n-{\tau }_n\mathrm{\nabla }{f}_n(x_n)),$$ 3.13

where

$${\tau }_n:=\frac{{\rho }_n{\parallel {\overline{x}}_n\parallel }^2}{{\parallel A{\overline{x}}_n\parallel }^2},$$ 3.14

with $\{{\rho }_n\}\subset (0,2)$ and $\{t_n\}\subset (0,1)$. If $x_{n+1}=x_n$, then stop and $x_n$ is a solution of SFP (1.1); otherwise, back to (3.12) and continue to compute $x_{n+2}$.

Obviously, (3.14) is also consistent with Remark 3.1. Thus, similar to the proof of Theorem 3.1, we can deduce that Algorithm 3.2 converges weakly to a solution of SFP (1.1).

Two general KM-CQ algorithms with generalized variable stepsize

In this section, we integrate the variable stepsizes from (1.5) to (1.8) and obtain a variable stepsize that can cover them. After that, we apply it to improve the algorithms presented in [17] and [18] and construct two algorithms for approximating some solution of (1.1).

Let $\psi :C\to H_1$ be a δ-contraction with $\delta \in (0,1)$, let $r:H_1\to H_1\mathrm{\setminus }\mathrm{\Theta }$ and $q:H_1\to H_2\mathrm{\setminus }\mathrm{\Theta }$ be nonzero operators, where Θ denotes the zero point.

A generalized variable stepsize for a general KM-CQ algorithm

The next recursion not only possesses a more generalized adaptive descent step, but it also can be implemented easily by the relaxed method.

Algorithm 4.1

Choose the initial data $x_0\in H_1$ arbitrarily. Assume that the nth iterate $x_n$ has been constructed; then we compute the $(n+1)$ th iterate $x_{n+1}$ via the formula

$$x_{n+1}=(1-{\beta }_n)x_n+{\beta }_n{P}_{C_n}[{\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n],$$ 4.1

where $U_n{x}_n=(I-{\tau }_n{A}^T(I-P_{Q_n})A)x_n$, ${\tau }_n:=\frac{{\rho }_n{\parallel r(x_n)\parallel }^2}{{\parallel Ar(x_n)\parallel }^2}$ or ${\tau }_n:=\frac{{\rho }_n{\parallel p(x_n)\parallel }^2}{{\parallel A^{T}p(x_n)\parallel }^2}$, $\{{\rho }_n\}\subset (0,2)$, $\{t_n\}\subset (0,1)$, $\{{\alpha }_n\}$ and $\{{\beta }_n\}$ are two real sequences in $[0,1]$. If $x_{n+1}=x_n$ for some $n\ge 0$, then $x_n$ is a solution of SFP (1.1) and the iteration stops; otherwise, continue to compute $x_{n+2}$.

Theorem 4.1

Suppose that the SFP is consistent, that is, $\mathrm{\Gamma }=C\cap A^{-1}(Q)\ne \mathrm{\varnothing }$, ${\underset{\_}{lim}}_n{\tau }_n(2-\lambda {\tau }_n)\ge \sigma >0$. Assume that the sequences $\{{\alpha }_n\}$ and $\{{\beta }_n\}$ satisfy the following conditions:

($C_1$)

${lim}_{n\to \mathrm{\infty }}{\alpha }_n=0$ and ${\sum }_{n=1}^{\mathrm{\infty }}{\alpha }_n=\mathrm{\infty }$;

($C_2$)

$0<{\underset{\_}{lim}}_n{\beta }_n$.

Then $\{x_n\}$ defined by (4.1) converges strongly to $x^{\ast }=P_{\mathrm{\Gamma }}\psi x^{\ast }$, which solves the following variational inequality:

$$\langle (\psi -I)x^{\ast },y-x^{\ast }\rangle \le 0,\mathrm{\forall }y\in \mathrm{\Gamma }.$$ 4.2

Proof

Since $P_{\mathrm{\Gamma }}:H_1\to \mathrm{\Gamma }\subset C$ is nonexpansive and $\psi :C\to H_1$ is δ-contractive, therefore, we have $P_{\mathrm{\Gamma }}\psi :C\to C$ is a contraction with $\delta \in (0,1)$. By the Banach contractive mapping principle, there exists a unique $x^{\ast }\in C$ such that $x^{\ast }=P_{\mathrm{\Gamma }}\psi x^{\ast }$. By virtue of (${\mathrm{p}}_1$), we see that (4.2) holds true.

By virtue of $x^{\ast }$ being a solution of the SFP, $x^{\ast }\in C\cap A^{-1}(Q)$, and $C\subseteq C_n$, $Q\subseteq Q_n$, then $x^{\ast }=P_C(x^{\ast })=P_{C_n}(x^{\ast })$ and $A{x}^{\ast }=P_Q(A{x}^{\ast })=P_{Q_n}(A{x}^{\ast })$. From (${\mathrm{p}}_4$) we have

$$\begin{array}{rcl}{\parallel U_n{x}_n-x^{\ast }\parallel }^2& =& {\parallel x_n-{\tau }_n\mathrm{\nabla }{f}_n(x_n)-x^{\ast }\parallel }^2\\ & =& {\parallel x_n-x^{\ast }\parallel }^2+{\parallel {\tau }_n\mathrm{\nabla }{f}_n(x_n)\parallel }^2-2{\tau }_n\langle x_n-x^{\ast },\mathrm{\nabla }{f}_n(x_n)\rangle \\ & =& {\parallel x_n-x^{\ast }\parallel }^2+{\tau }_n^2{\parallel \mathrm{\nabla }{f}_n(x_n)\parallel }^2\\ & & -2{\tau }_n\langle A{x}_n-A{x}^{\ast },(I-P_{Q_n})A{x}_n-(I-P_{Q_n})A{x}^{\ast }\rangle \\ & \le & {\parallel x_n-x^{\ast }\parallel }^2+{\tau }_n^{2}L{\parallel (I-P_{Q_n})A{x}_n\parallel }^2\\ & & -2{\tau }_n{\parallel (I-P_{Q_n})A{x}_n\parallel }^2\\ & =& {\parallel x_n-x^{\ast }\parallel }^2+({\tau }_n^{2}L-2{\tau }_n){\parallel (I-P_{Q_n})A{x}_n\parallel }^2.\end{array}$$ 4.3

Obviously, ${\tau }_n\in (0,2/L)$ from Remark 3.1, then ${\tau }_n^{2}L-2{\tau }_n<0$, in particular, we obtain

$$\parallel U_n{x}_n-x^{\ast }\parallel \le \parallel x_n-x^{\ast }\parallel .$$ 4.4

At this point, we can establish the boundedness of $\{x_n\}$. To see this, using (4.1) we have

$$\begin{array}{c}\parallel x_{n+1}-x^{\ast }\parallel \hfill \\ =\parallel (1-{\beta }_n)x_n+{\beta }_n{P}_{C_n}[{\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n]-x^{\ast }\parallel \hfill \\ \le (1-{\beta }_n)\parallel x_n-x^{\ast }\parallel +{\beta }_n\parallel P_{C_n}[{\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n]-x^{\ast }\parallel \hfill \\ \le (1-{\beta }_n)\parallel x_n-x^{\ast }\parallel +{\beta }_n\parallel {\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n-x^{\ast }\parallel \hfill \\ =(1-{\beta }_n)\parallel x_n-x^{\ast }\parallel \hfill \\ +{\beta }_n\parallel {\alpha }_n(\psi (x_n)-\psi \left(x^{\ast }\right))+(1-{\alpha }_n)(U_n{x}_n-x^{\ast })+{\alpha }_n\psi \left(x^{\ast }\right)-{\alpha }_n{x}^{\ast }\parallel \hfill \\ \le (1-{\beta }_n)\parallel x_n-x^{\ast }\parallel +{\alpha }_n{\beta }_n\parallel \psi (x_n)-\psi \left(x^{\ast }\right)\parallel +(1-{\alpha }_n){\beta }_n\parallel U_n{x}_n-x^{\ast }\parallel \hfill \\ +{\alpha }_n{\beta }_n\parallel \psi \left(x^{\ast }\right)-x^{\ast }\parallel \hfill \\ \le (1-(1-\delta ){\alpha }_n{\beta }_n)\parallel x_n-x^{\ast }\parallel +{\alpha }_n{\beta }_n\parallel \psi \left(x^{\ast }\right)-x^{\ast }\parallel \hfill \\ =(1-(1-\delta ){\alpha }_n{\beta }_n)\parallel x_n-x^{\ast }\parallel +(1-\delta ){\alpha }_n{\beta }_n\frac{\parallel \psi (x^{\ast })-x^{\ast }\parallel }{1-\delta }\hfill \\ \le max\{\parallel x_0-x^{\ast }\parallel ,\frac{\parallel \psi (x^{\ast })-x^{\ast }\parallel }{1-\delta }\}=M,\hfill \end{array}$$

for all $n\ge 0$, which indicates $\{x_n\}$ is bounded. Set $z_n=P_{C_n}[{\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n]$, thus $\{z_n\}$ is also bounded.

Next, we prove $x_n\to x^{\ast }$ ($n\to \mathrm{\infty }$). By virtue of (4.1), Proposition 2.1(${\mathrm{p}}_2$) and (4.4), we have

$$\begin{array}{rl}& {\parallel x_{n+1}-x^{\ast }\parallel }^2\\ & ={\parallel (1-{\beta }_n)x_n+{\beta }_n{P}_{C_n}[{\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n]-x^{\ast }\parallel }^2\\ & \le (1-{\beta }_n){\parallel x_n-x^{\ast }\parallel }^2+{\beta }_n{\parallel P_{C_n}[{\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n]-x^{\ast }\parallel }^2,\end{array}$$ 4.5

where we set

$$\begin{array}{c}{\parallel P_{C_n}[{\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n]-x^{\ast }\parallel }^2\hfill \\ ={\parallel P_{C_n}[w_n]-x^{\ast }\parallel }^2\hfill \\ =\langle P_{C_n}[w_n]-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle \hfill \\ =\langle P_{C_n}[w_n]-w_n,P_{C_n}[w_n]-x^{\ast }\rangle +\langle w_n-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle ,\hfill \end{array}$$

since $\langle P_{C_n}[w_n]-w_n,P_{C_n}[w_n]-x^{\ast }\rangle \le 0$, we have

$$\begin{array}{c}{\parallel P_{C_n}[w_n]-x^{\ast }\parallel }^2\hfill \\ \le \langle w_n-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle \hfill \\ =\langle {\alpha }_n\psi (x_n)+(1-{\alpha }_n)U_n{x}_n-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle \hfill \\ =\langle {\alpha }_n(\psi (x_n)-\psi \left(x^{\ast }\right))+(1-{\alpha }_n)(U_n{x}_n-x^{\ast })+{\alpha }_n(\psi \left(x^{\ast }\right)-x^{\ast }),P_{C_n}[w_n]-x^{\ast }\rangle \hfill \\ \le ({\alpha }_n\parallel \psi (x_n)-\psi \left(x^{\ast }\right)\parallel +(1-{\alpha }_n)\parallel U_n{x}_n-x^{\ast }\parallel )\parallel P_{C_n}[w_n]-x^{\ast }\parallel \hfill \\ +{\alpha }_n\langle \psi \left(x^{\ast }\right)-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle \hfill \\ \le (1-(1-\delta ){\alpha }_n)\parallel x_n-x^{\ast }\parallel \parallel P_{C_n}[w_n]-x^{\ast }\parallel +{\alpha }_n\langle \psi \left(x^{\ast }\right)-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle \hfill \\ \le \frac{1-(1-\delta ){\alpha }_n}{2}{\parallel x_n-x^{\ast }\parallel }^2+\frac{1}{2}{\parallel P_{C_n}[w_n]-x^{\ast }\parallel }^2\hfill \\ +{\alpha }_n\langle \psi \left(x^{\ast }\right)-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle .\hfill \end{array}$$

Therefore,

$${\parallel P_{C_n}[w_n]-x^{\ast }\parallel }^2\le (1-(1-\delta ){\alpha }_n){\parallel x_n-x^{\ast }\parallel }^2+2{\alpha }_n\langle \psi \left(x^{\ast }\right)-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle .$$ 4.6

Substituting (4.6) into (4.5) can yield

$$\begin{array}{rl}& {\parallel x_{n+1}-x^{\ast }\parallel }^2\\ & \le (1-{\beta }_n){\parallel x_n-x^{\ast }\parallel }^2\\ & +(1-(1-\delta ){\alpha }_n){\beta }_n{\parallel x_n-x^{\ast }\parallel }^2+2{\alpha }_n{\beta }_n\langle \psi \left(x^{\ast }\right)-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle \\ & \le (1-(1-\delta ){\alpha }_n{\beta }_n){\parallel x_n-x^{\ast }\parallel }^2+(1-\delta ){\alpha }_n{\beta }_n\frac{2}{1-\delta }\langle \psi \left(x^{\ast }\right)-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle .\end{array}$$ 4.7

Since $x^{\ast }\in C\subseteq C_n$, $P_{C_n}:H_1\to C\subseteq C_n$ and $\psi :C\subseteq C_n\to H_1$, then $P_{C_n}\psi :C_n\to C_n$, $x^{\ast }=P_{C_n}\psi x^{\ast }$.

Due to the property of the projection (${\mathrm{p}}_1$) in Proposition 2.1,

$$\begin{array}{rl}& \underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}sup}\langle \psi \left(x^{\ast }\right)-x^{\ast },P_{C_n}[w_n]-x^{\ast }\rangle \\ & =\underset{P_{C_n}[w_n]\in C_n}{max}\langle \psi \left(x^{\ast }\right)-P_{C_n}\psi \left(x^{\ast }\right),P_{C_n}[w_n]-P_{C_n}\psi \left(x^{\ast }\right)\rangle \le 0.\end{array}$$ 4.8

Applying (4.8) and Proposition 2.3 to (4.7), we deduce that $x_n\to x^{\ast }$.

Assume that x̂ is an accumulation point of $\{x_n\}$ and $x_{n_i}\to \hat{x}$, where ${\{x_{n_i}\}}_{i=1}^{\mathrm{\infty }}$ is a subsequence of $\{x_n\}$. Next we will prove that x̂ is a solution of SFP.

As $x_n\to x^{\ast }$, we know $x_n\rightharpoonup x^{\ast }$, that is,

$$\underset{n\to \mathrm{\infty }}{lim}\parallel x_{n+1}-x_n\parallel =0.$$ 4.9

Therefore, limit (4.4), we can obtain

$$\underset{n\to \mathrm{\infty }}{lim}\parallel U_n(x_n)-x^{\ast }\parallel =0.$$

Then, limit (4.3), we get

$$\underset{n\to \mathrm{\infty }}{lim}\parallel (I-P_{Q_n})A{x}_n\parallel =0.$$ 4.10

On the one hand, we show $\hat{x}\in C$.

Notice that $x_{n_i}\to \hat{x}$ and $x_{n_{i+1}}-x_{n_i}\to 0$ ($i\to \mathrm{\infty }$). Since $x_{n_{i+1}}\in C_{n_i}$, then by virtue of the definition of $C_{n_i}$, we have

$$c(x_{n_i})+\langle {\xi }_{n_i},x_{n_{i+1}}-x_{n_i}\rangle \le 0,\mathrm{\forall }i=1,2,\dots ,$$

taking the limit and using (4.9), we obtain that

$$c(\hat{x})\le 0.$$

Hence, we get $\hat{x}\in C$.

On the other hand, we need to show $A\hat{x}\in Q$.

Since $P_{Q_{n_i}}(A{x}_{n_i})\in Q_{n_i}$, we have

$$q(A{x}_{n_i})+\langle {\eta }_{n_i},P_{Q_{n_i}}(A{x}_{n_i})-A{x}_{n_i}\rangle \le 0,$$

taking $n_i\to \mathrm{\infty }$, by virtue of (4.10), we deduce that

$$q(A\hat{x})\le 0,$$

that is, $A\hat{x}\in Q$.

Therefore, x̂ is a solution of SFP.

Thus we may replace $x^{\ast }$ in (4.7) with x̂ and get $\{\parallel x_n-\hat{x}\parallel \}$ is convergent. Because there exists a subsequence $\{\parallel x_{n_i}-\hat{x}\parallel \}$ convergent to 0, then $x_n\to \hat{x}$ as $n\to \mathrm{\infty }$. □

The other extended algorithm

Let $h:C\to H_1$ be a κ-contraction. Let $B:H_1\to H_1$ be a self-adjoint strongly positive bounded linear operator with coefficient $\lambda \in (0,1)$, for $\mathrm{\forall }x\in H_1$, there exists $\langle Bx,x\rangle \ge \lambda {\parallel x\parallel }^2$. Take a constant σ such that $0<\sigma \kappa <\lambda$.

As B is self-adjoint, we have$\parallel B\parallel ={sup}_{\parallel x\parallel =1}\langle Bx,x\rangle$. $I-B$ is also self-adjoint, then

$$\begin{array}{rcl}\parallel I-B\parallel & =& \underset{\parallel x\parallel =1}{sup}\langle (I-B)x,x\rangle =\underset{\parallel x\parallel =1}{sup}\{{\parallel x\parallel }^2-\langle Bx,x\rangle \}\\ & \le & \underset{\parallel x\parallel =1}{sup}\{(1-\lambda ){\parallel x\parallel }^2\}\le 1-\lambda .\end{array}$$

In (4.1), we set $\psi (x)=\sigma h(x)+(I-B)U_{n}x$, thus

$$\begin{array}{rcl}\parallel \psi (x)-\psi (y)\parallel & \le & \sigma \kappa \parallel x-y\parallel +\parallel I-B\parallel \parallel x-y\parallel \\ & \le & (\sigma \kappa +1-\lambda )\parallel x-y\parallel ,\end{array}$$

for $\mathrm{\forall }x,y\in H_1$, we know that $\sigma \kappa +1-\lambda \in (0,1)$, $\psi :C\to H_1$ is still a contraction. Accordingly, we have the following extended algorithm that is a special case of Algorithm 4.1.

Algorithm 4.2

Choose the initial data $x_0\in H_1$ arbitrarily. Assume that the nth iterate $x_n$ has been constructed; then we compute the $(n+1)$ th iterate $x_{n+1}$ via the formula

$$x_{n+1}=(1-{\beta }_n)x_n+{\beta }_n{P}_{C_n}[{\alpha }_n\sigma h(x_n)+(I-{\alpha }_{n}B)U_n{x}_n],$$ 4.11

where $U_n{x}_n=(I-{\tau }_n{A}^T(I-P_{Q_n})A)x_n$, ${\tau }_n:=\frac{{\rho }_n{\parallel r(x_n)\parallel }^2}{{\parallel Ar(x_n)\parallel }^2}$ or ${\tau }_n:=\frac{{\rho }_n{\parallel p(x_n)\parallel }^2}{{\parallel A^{T}p(x_n)\parallel }^2}$, ${\tau }_n\in (0,2/L)$, $\{{\rho }_n\}\subset (0,2)$, $\{t_n\}\subset (0,1)$, $\{{\alpha }_n\}$ and $\{{\beta }_n\}$ are two real sequences in $[0,1]$. If $x_{n+1}=x_n$, then stop and $x_n$ is a solution of SFP (1.1); otherwise, continue to compute $x_{n+2}$.

Theorem 4.2

Suppose that the SFP is consistent, that is, $\mathrm{\Gamma }=C\cap A^{-1}(Q)\ne \mathrm{\varnothing }$, ${\underset{\_}{lim}}_n{\tau }_n(2-\lambda {\tau }_n)\ge \sigma >0$, assume that the sequences $\{{\alpha }_n\}$ and $\{{\beta }_n\}$ satisfy the following conditions:

($C_1$)

${lim}_{n\to \mathrm{\infty }}{\alpha }_n=0$ and ${\sum }_{n=1}^{\mathrm{\infty }}{\alpha }_n=\mathrm{\infty }$;

($C_2$)

$0<{\underset{\_}{lim}}_n{\beta }_n$.

Then $\{x_n\}$ defined by (4.11) converges strongly to $x^{\ast }=P_{\mathrm{\Gamma }}[\sigma h(x^{\ast })+(I-B)U_n{x}^{\ast }]$, which solves the following variational inequality:

$$\langle \sigma h\left(x^{\ast }\right)-B\left(x^{\ast }\right),y-x^{\ast }\rangle \le 0,\mathrm{\forall }y\in \mathrm{\Gamma }.$$

Numerical experiments and results

This section considers two numerical experiments to illustrate the performance of the above proposed variable stepsizes in CQ algorithm. Firstly, we see that a great amount of problems in signal and image processing can be seen as estimating $x\in {\mathrm{R}}^N$ from the linear observation model

$$y=Ax+\epsilon ,$$ 5.1

where $y\in {\mathrm{R}}^M$ is the observed or measured data with noisy ε. $A:{\mathrm{R}}^N\to {\mathrm{R}}^M$ denotes the bounded linear observation or measurement operator. Sometimes, the range of A may not be closed in most inverse problems, therefore, if A is ill-conditioned, the problem will be ill-posed.

If x is a sparse expansion, finding the solutions of (5.1) can be seen as finding a solution to the least-square problem

$$\underset{x\in {\mathrm{R}}^N}{min}\frac{1}{2}{\parallel y-Ax\parallel }^2\text{subject to }{\parallel x\parallel }_1<t$$ 5.2

for any real number $t>0$.

Problem (5.2) is a particular case of SFP (1.1) where $C=\{x\in {\mathrm{R}}^N:{\parallel x\parallel }_1\le t\}$ and $Q=\{y\}$, i.e., find ${\parallel x\parallel }_1\le t$ such that $Ax=y$. Therefore, CQ algorithm can be applied to solve (5.2). From Propositions 2.4 and 2.5 the projection onto C can be easily computed [15], while Lemmas 2.1 and 2.2 show the special situation of Proposition 2.4.

Next, following the experiments in [15, 26], we choose two particular problems, i.e., the compressed sensing and the image deconvolution, which are covered by (5.1). The experiments compare the performances of the proposed stepsizes of the CQ algorithm in this paper with the stepsizes in [15] and [16].

Compressed sensing

We consider a typical compressed sensing model, where a sparse signal recovery problem with a signal $x\in {\mathrm{R}}^N$, and $N=2^{12}$. This original signal x contains only $m=50$ spikes with amplitude ±1, and the spikes are located at random, see the top of Figure 1. x is being reconstructed from $M=2^{10}$ measurements, thus A is a $M\times N$ matrix randomly obtained with independent samples of a orthonormalized standard Gaussian distribution, and the noisy ε is with variance ${\sigma }_{\epsilon }^2={10}^{-4}$. To (5.2), we also set t=50.

Figure 1.

Compressed sensing problem, from top to bottom: original signal, results of CQ algorithm with stepsizes ( 1.5 ), ( 1.6 ), ( 3.3 ) and ( 3.14 ), the last is KM-CQ algorithm with stepsize ( 3.3 ).

For stepsizes (1.5) and (1.6) in [15] and [16], respectively, we still consider the stepsize with constant $\rho =2$, ${\omega }_n={(n+2)}^{-5}$, while for stepsizes (3.3) and (3.14), we set $\rho =1$. For (3.1) and (3.12), we set $t=0.1$ and $u=rand(N,1)\in (0,1)$. For (4.1) we use (3.1) and (3.3), and set ${\alpha }_n={(n+2)}^{-1}$, ${\beta }_n=1-{(n+2)}^{-1}$ and $\psi \equiv 0$ as its special case. All the processes are started with the initial signal $x_0=0$ and finished with the stop rule

$$\parallel x_{n+1}-x_n\parallel /\parallel x_n\parallel <{10}^{-3}.$$

We also calculated the mean squared error (MSE) for the results

$$\mathit{MSE}=(1/N)\parallel x^{\ast }-x\parallel ,$$

where $x^{\ast }$ is an estimated signal of x.

The simulated results of different algorithms with different steps can be seen in Figure 1 and Table 1. Algorithms 3.1 and 3.2 have less iteration steps and smaller MSE, especially for Algorithm 3.1. Thus, we see that stepsizes (3.3) and (3.14) not only have simper formats than before, but also can make CQ algorithms have faster iteration and better restored precision.

Table 1.

Results of different stepsizes and algorithms for Figure  1

Algorithms	MSE	n	CPU time (s)
CQ with (1.5)	2.8021e−005	66	0.8504
CQ with (1.6)	2.844e−005	66	0.8579
3.1	5.1714e−006	25	0.6858
3.2	1.5253e−006	64	1.5685

Image deconvolution

In this subsection, we apply the CQ algorithms in the paper to image deconvolution. The observation model can also be described as (5.1), we wish to estimate an original image x from an observation y, while matrix A represents the observation operator, and ε is a sample of a zero-mean white Gaussian field of variance σ. For the 2D image deconvolution problem, A is a block-circulant matrix with circulant blocks [27]. We stress that the goal of these experiments is not to assess the restored precision of the algorithms, but to apply the algorithms in paper to solve this particular SFP, then compare the iterative speed and restored precision of the proposed stepsizes against the CQ algorithms.

According to papers [26, 27], we also take the well-known Cameraman image. In the experiments, we employ Haar wavelets, and the blur point spread functions are uniform blur with size $9\times 9$, $h_{ij}={(1+i^2+j^2)}^{-1}$, for $i,j=-4,\dots ,4$ and for $i,j=-7,\dots ,7$. The noise variance is ${\sigma }^2=0.308$, 2 and 8, respectively. We have $N=M={256}^2$, then the block-circulant matrix A can be constructed by the blur point spread functions, and A may be very ill-conditioned. Set all the threshold values $\mu =0.25$, t is the sum of all the pixel values in the original image. Moreover, we use $y=\mathit{IFFT}(\mathit{FFT}(A).\ast \mathit{FFT}(x))+\epsilon$ to obtain the observation, where FFT is the fast Fourier transform, IFFT is the inverse fast Fourier transform. Other settings in the above stepsizes and algorithms are the same as in 4.1. We set the initial image $x_0=0$ and also follow the stop rule $\parallel x_{n+1}-x_n\parallel /\parallel x_n\parallel <{10}^{-3}$.

The results of iteration steps, CPU time and the SNR improvements are presented in Table 2. It also testifies that the proposed stepsizes and algorithms in this paper can give better performance.

Table 2.

Results for the restorations of different stepsizes and algorithms

Blur kernel	${\mathit{\sigma }}^{\mathbf{2}}$	Algorithms	SNR (dB)	n	CPU time (s)
9 × 9 uniform	0.308	CQ with (1.5)	16.1802	52	3.3883
		CQ with (1.6)	16.1722	48	3.1636
		3.1	14.5266	19	1.3847
		3.2	14.5265	19	1.3199
		4.1 with (3.3)	14.2464	34	2.4280
$h_{ij}={(1+i^2+j^2)}^{-1}$ for i,j = −4,…,4	2	CQ with (1.6)	22.6184	19	1.3007
		CQ with (1.5)	22.4329	17	1.1718
		3.1	19.8401	14	1.0746
		3.2	19.8401	14	1.0266
		4.1 with (3.3)	19.4912	33	2.3306
$h_{ij}={(1+i^2+j^2)}^{-1}$ for i,j = −7,…,7	8	CQ with (1.6)	12.7305	39	2.6739
		CQ with (1.5)	14.6363	42	2.8604
		3.1	19.3561	18	1.3390
		3.2	19.3560	18	1.3057
		4.1 with (3.3)	18.6140	33	2.3628

Conclusions and discussion

In this paper, we have proposed two simpler variable stepsizes for the CQ algorithm. Compared with the other related variable stepsizes, they also need not to compute the largest eigenvalue of A and can be calculated easily. Furthermore, we also presented a more general KM-CQ algorithm with generalized variable stepsizes. As a special case, we deduced another general format. Obviously, both the general algorithms with the generalized variable stepsizes can solve the SFP and some special variational inequality problem better. The corresponding weak and strong convergence properties have also been established. In the experiments, through the compressed sensing and image deconvolution models, we compare the proposed stepsizes with the former ones, the results obtained from the proposed stepsizes and algorithms appear to be significantly better.

We should notice that the values of parameter ${\rho }_n$ are fixed in the above experiments. Actually, a different value of ${\rho }_n$ can also affect the convergence speed of the algorithms. Therefore, our future work is to find the method to choose a self-adaptive sequence $\{{\rho }_n\}$.