Inducing strong convergence into the asymptotic behaviour of proximal splitting algorithms in Hilbert spaces

Abstract

Proximal splitting algorithms for monotone inclusions (and convex optimization problems) in Hilbert spaces share the common feature to guarantee for the generated sequences in general weak convergence to a solution. In order to achieve strong convergence, one usually needs to impose more restrictive properties for the involved operators, like strong monotonicity (respectively, strong convexity for optimization problems). In this paper, we propose a modified Krasnosel'skiĭ–Mann algorithm in connection with the determination of a fixed point of a nonexpansive mapping and show strong convergence of the iteratively generated sequence to the minimal norm solution of the problem. Relying on this, we derive a forward–backward and a Douglas–Rachford algorithm, both endowed with Tikhonov regularization terms, which generate iterates that strongly converge to the minimal norm solution of the set of zeros of the sum of two maximally monotone operators. Furthermore, we formulate strong convergent primal–dual algorithms of forward–backward and Douglas–Rachford-type for highly structured monotone inclusion problems involving parallel-sums and compositions with linear operators. The resulting iterative schemes are particularized to the solving of convex minimization problems. The theoretical results are illustrated by numerical experiments on the split feasibility problem in infinite dimensional spaces.

1. Introduction and preliminaries

Let $\mathcal{H}$ be a real Hilbert space endowed with inner product $⟨\cdot ,\cdot ⟩$ and associated norm $\parallel \cdot \parallel =\sqrt{⟨\cdot ,\cdot ⟩}$. Let $T:\mathcal{H}\to \mathcal{H}$ be a nonexpansive mapping, that is $\parallel Tx-Ty\parallel \le \parallel x-y\parallel$ for all $x,y\in \mathcal{H}$. One of the most popular iterative methods for finding a fixed point of the operator T is the Krasnosel'skiĭ–Mann algorithm

$$x_{n+1}=x_n+{\lambda }_n\left(T{x}_n-x_n\right)\mathrm{\forall }n\ge 0,$$ (1)

where $x_0\in \mathcal{H}$ is arbitrary and $({\lambda }_n{)}_{n\ge 0}$ is a sequence of nonnegative real numbers. Provided $\mathrm{Fix}T=\{x\in \mathcal{H}:Tx=x\}\ne \mathrm{\varnothing }$, one can show under mild conditions imposed on $({\lambda }_n{)}_{n\ge 0}$, that the sequence $(x_n{)}_{n\ge 0}$ converges weakly to an element in $\mathrm{Fix}T$ (see for instance [6]).

The applications and the impact of this fundamental result go beyond the usual fixed point theory, representing in fact the starting point for the derivation of algorithms and related convergence statements in connection with the solving of monotone inclusions. In this context, we mention the classical forward–backward algorithm for determining a zero of the sum of a set-valued maximally monotone operator and a single-valued and cocoercive one and the Douglas–Rachford algorithm for determining a zero of the sum of two set-valued maximally monotone operators. The paradigms behind these classical methods can be transferred to the solving of convex optimization problems, too (see [6]).

The iterative algorithms mentioned above share the common property that the generated sequences converge weakly to a solution of the problem under investigation. However, for applications where infinite dimensional functional spaces are involved, weak convergence is not satisfactory. In order to achieve strong convergence, one usually needs to impose more restrictive properties for the involved operators, like strong monotonicity when considering monotone inclusions and strong convexity when solving optimization problems. Since there evidently are applications for which these stronger properties are not fulfilled, the interest of the applied mathematics community in developing algorithms which generate iterates that strongly convergence is justified.

We mention in this sense the Halpern algorithm and its numerous variants designed for finding a fixed point of a nonexpansive mapping (see for instance [13]). In the context of solving monotone inclusions, we mention the proximal-Tikhonov algorithm

$$x_{n+1}={\left(\mathrm{Id}+{\lambda }_n(A+{\mu }_n\mathrm{Id})\right)}^{-1}\left(x_n\right)\mathrm{\forall }n\ge 0,$$

where $A:\mathcal{H}\backslash rightrightarrows\mathcal{H}$ is a maximally monotone operator, $\mathrm{Id}$ is the identity operator on $\mathcal{H}$ and $({\lambda }_n{)}_{n\ge 0}$ and $({\mu }_n{)}_{n\ge 0}$ are sequences of nonnegative real numbers. Under mild conditions imposed on $({\lambda }_n{)}_{n\ge 0}$ and $({\mu }_n{)}_{n\ge 0}$, one can prove strong convergence of $(x_n{)}_{n\ge 0}$ to the minimal norm solution of the set of zeros of A (see [19,27]). It is important to emphasize the Tikhonov regularization terms $({\mu }_n\mathrm{Id}{)}_{n\ge 0}$ in the above scheme, which actually enforces the strong convergence property. In the absence of the regularization term, the above numerical scheme becomes the classical proximal algorithm for determining a zero of the operator A, for which in general only weak convergence can be proved (see [23]). For more theoretical results concerning Tikhonov regularization and more motivational arguments for using such techniques, especially for optimization problems, we refer the reader to Attouch's paper [2]. For other techniques and tools in order to achieve strong convergence we mention also the works of Haugazeau [18] and [5].

In this article, we will first introduce and investigate a modified Krasnosel'skiĭ–Mann algorithm with relaxation parameters, having the outstanding property that it generates a sequence of iterates which converges strongly to the minimal norm solution of the fixed points set of a nonexpansive mapping. In contrast to [13,20,27] (see also the references therein), where the techniques and tools used have their roots in fixed point theory results for contractions, our convergence statements follow more directly. Relying on this, we derive a forward–backward and a Douglas–Rachford algorithm, both endowed with Tikhonov regularization terms, which generate iterates that strongly converge to the minimal norm solution of the set of zeros of the sum of two maximally monotone operators. The resulting iterative schemes are particularized to the minimization of the sum of two convex functions.

Furthermore, we deal with complexly structured monotone inclusions where parallel-sums and compositions with linear operators are involved. By making use of modern primal–dual techniques (see [10,12,15,17,28] and also [8,9]), we derive strongly convergent numerical schemes of forward–backward and Douglas–Rachford-type, both involving Tikhonov regularization terms and having the remarkable property that all the operators are evaluated separately. Moreover, the designed algorithms solve both the structured monotone inclusion problem and its dual monotone inclusion problem in the sense of Attouch-Théra (see [3]). When particularized to convex optimization problems, this means the concomitantly solving of a primal problem and its Fenchel dual one. For other types of primal–dual algorithms with strong convergence properties and their applications we refer the reader to [1,4]. Finally, in the last section we carry out numerical experiments on the split feasibility problem in infinite dimensional Hilbert spaces which illustrate the potential of the algorithm endowed with Tikhonov regularization terms.

In the remaining of this section, we recall some results which will play a decisive role in the convergence analysis of the proposed algorithms. The following result is related to the convergence of a sequence satisfying a sharp quasi-Fejér monotonicity property and follows as a direct consequence of [27, Lemma 2.5].

Lemma 1

Let $(a_n{)}_{n\ge 0}$ be a sequence of nonnegative real numbers satisfying the inequality

$$a_{n+1}\le (1-{\theta }_n)a_n+{\theta }_n{b}_n+{ϵ}_n\mathrm{\forall }n\ge 0,$$

where

1. $0\le {\theta }_n\le 1$ for all $n\ge 0$ and $\sum _{n\ge 0}{\theta }_n=+\mathrm{\infty };$

2. $\underset{n\to +\mathrm{\infty }}{lim sup}{b}_n\le 0;$

3. ${ϵ}_n\ge 0$ for all $n\ge 0$ and $\sum _{n\ge 0}{ϵ}_n<+\mathrm{\infty }$.

Then the sequence $(a_n{)}_{n\ge 0}$ converges to 0.

We close this section with a result that is a consequence of the demiclosedness principle (see [6, Corollary 4.18]) and it will be used in the proof of Theorem 3, which is the main result of this paper.

Lemma 2

Let $T:\mathcal{H}\to \mathcal{H}$ be a nonexpansive operator and let $(x_n{)}_{n\ge 0}$ be a sequence in $\mathcal{H}$ and $x\in \mathcal{H}$ be such that $w-\underset{n\to +\mathrm{\infty }}{lim}{x}_n=x$ and $(T{x}_n-x_n{)}_{n\ge 0}$ converges strongly to 0 as $n\to +\mathrm{\infty }$. Then $x\in \mathrm{Fix}T$.

2. A strongly convergent Krasnosel'skiĭ–Mann algorithm

In order to induce strong convergence into the asymptotic behaviour of the Krasnosel'skiĭ–Mann algorithm for determining a fixed point of a nonexpansive mapping $T:\mathcal{H}\to \mathcal{H}$, we propose the following modified version of it:

$$x_{n+1}={\beta }_n{x}_n+{\lambda }_n\left(T\left({\beta }_n{x}_n\right)-{\beta }_n{x}_n\right)\mathrm{\forall }n\ge 0,$$ (2)

where $x_0\in \mathcal{H}$ is the starting point and $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ suitably chosen sequences of positive numbers. In the proof of the theorem below, we denote by ${\mathrm{proj}}_C:\mathcal{H}\to C,{\mathrm{proj}}_C\left(x\right)={\mathrm{argmin}}_{c\in C}\parallel x-c\parallel$, the projection operator onto the nonempty closed convex set $C\subseteq \mathcal{H}$. We notice that for a nonexpansive mapping $T:\mathcal{H}\to \mathcal{H}$, its set of fixed points $\mathrm{Fix}T$ is closed and convex (see [6, Corollary 4.15]).

Theorem 3

Let $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ be real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$.

Consider the iterative scheme (2) with and arbitrary starting point $x_0\in \mathcal{H}$ and a nonexpansive mapping $T:\mathcal{H}\to \mathcal{H}$ fulfilling $\mathrm{Fix}T\ne \mathrm{\varnothing }$. Then $(x_n{)}_{n\ge 0}$ converges strongly to ${\mathrm{proj}}_{\mathrm{Fix}T}\left(0\right)$.

Proof.

For the beginning, we prove that $(x_n{)}_{n\ge 0}$ is bounded. Let $x\in \mathrm{Fix}T$. Due to the nonexpansiveness of T, we have for any $n\ge 0$:

$$\begin{array}{rl}\parallel x_{n+1}-x\parallel & =\parallel (1-{\lambda }_n)({\beta }_n{x}_n-x)+{\lambda }_n\left(T\left({\beta }_n{x}_n\right)-Tx\right)\parallel \\ & \le (1-{\lambda }_n)\parallel {\beta }_n{x}_n-x\parallel +{\lambda }_n\parallel T\left({\beta }_n{x}_n\right)-Tx\parallel \\ & \le \parallel {\beta }_n{x}_n-x\parallel \\ & =\parallel {\beta }_n(x_n-x)+({\beta }_n-1)x\parallel \\ & \le {\beta }_n\parallel x_n-x\parallel +(1-{\beta }_n)\parallel x\parallel .\end{array}$$

A simple induction leads to the inequality

$$\parallel x_n-x\parallel \le max\{\parallel x_0-x\parallel ,\parallel x\parallel \}\mathrm{\forall }n\ge 0,$$

hence $(x_n{)}_{n\ge 0}$ is bounded.

We claim that

$$\parallel x_{n+1}-x_n\parallel \to 0\text{as}n\to +\mathrm{\infty }.$$ (3)

Indeed, by taking into account that T is nonexpansive and that $(x_n{)}_{n\ge 0}$ is bounded, we obtain for any $n\ge 1$ the following estimates:

$$\begin{array}{rl}\parallel x_{n+1}-x_n\parallel & =\parallel (1-{\lambda }_n){\beta }_n{x}_n-(1-{\lambda }_{n-1}){\beta }_{n-1}{x}_{n-1}+{\lambda }_{n}T\left({\beta }_n{x}_n\right)-{\lambda }_{n-1}T\left({\beta }_{n-1}{x}_{n-1}\right)\parallel \\ & \le \parallel (1-{\lambda }_n)({\beta }_n{x}_n-{\beta }_{n-1}{x}_{n-1})+({\lambda }_{n-1}-{\lambda }_n){\beta }_{n-1}{x}_{n-1}\parallel \\ & +\parallel {\lambda }_n\left(T\left({\beta }_n{x}_n\right)-T\left({\beta }_{n-1}{x}_{n-1}\right)\right)+({\lambda }_n-{\lambda }_{n-1})T\left({\beta }_{n-1}{x}_{n-1}\right)\parallel \\ & \le \parallel {\beta }_n{x}_n-{\beta }_{n-1}{x}_{n-1}\parallel +|{\lambda }_n-{\lambda }_{n-1}|C_1,\end{array}$$

where $C_1>0$.

Further, we derive for any $n\ge 1$:

$$\begin{array}{rl}\parallel x_{n+1}-x_n\parallel & \le \parallel {\beta }_n(x_n-x_{n-1})+({\beta }_n-{\beta }_{n-1})x_{n-1}\parallel +|{\lambda }_n-{\lambda }_{n-1}|C_1\\ & \le {\beta }_n\parallel x_n-x_{n-1}\parallel +|{\beta }_n-{\beta }_{n-1}|C_2+|{\lambda }_n-{\lambda }_{n-1}|C_1,\end{array}$$

where $C_2>0$. Statement (3) is a consequence of Lemma 1, for $a_n:=\parallel x_n-x_{n-1}\parallel ,b_n:=0,{ϵ}_n:=|{\beta }_n-{\beta }_{n-1}|C_2+|{\lambda }_n-{\lambda }_{n-1}|C_1$ and ${\theta }_n:=1-{\beta }_n,n\ge 1$.

In the following we prove that

$$\parallel x_n-T{x}_n\parallel \to 0\text{as}n\to +\mathrm{\infty }.$$ (4)

For any $n\ge 0$ we have the following inequalities:

$$\begin{array}{rl}\parallel x_n-T{x}_n\parallel & \le \parallel x_{n+1}-x_n\parallel +\parallel x_{n+1}-T{x}_n\parallel \\ & =\parallel x_{n+1}-x_n\parallel +\parallel (1-{\lambda }_n)({\beta }_n{x}_n-T{x}_n)+{\lambda }_n\left(T\left({\beta }_n{x}_n\right)-T{x}_n\right)\parallel \\ & \le \parallel x_{n+1}-x_n\parallel +(1-{\lambda }_n)\parallel {\beta }_n{x}_n-T{x}_n\parallel +{\lambda }_n\parallel {\beta }_n{x}_n-x_n\parallel \\ & \le \parallel x_{n+1}-x_n\parallel +(1-{\lambda }_n)\parallel {\beta }_n{x}_n-{\beta }_{n}T{x}_n\parallel \\ & +(1-{\lambda }_n)\parallel {\beta }_{n}T{x}_n-T{x}_n\parallel +{\lambda }_n(1-{\beta }_n)\parallel x_n\parallel \\ & \le \parallel x_{n+1}-x_n\parallel +(1-{\lambda }_n)\parallel x_n-T{x}_n\parallel \\ & +(1-{\lambda }_n)(1-{\beta }_n)\parallel T{x}_n\parallel +{\lambda }_n(1-{\beta }_n)\parallel x_n\parallel .\end{array}$$

From here we deduce that for any $n\ge 0$:

$${\lambda }_n\parallel x_n-T{x}_n\parallel \le \parallel x_{n+1}-x_n\parallel +(1-{\lambda }_n)(1-{\beta }_n)\parallel T{x}_n\parallel +{\lambda }_n(1-{\beta }_n)\parallel x_n\parallel .$$

Taking into account that $(x_n{)}_{n\ge 0}$ is bounded, (3) and the properties of the sequences involved, we derive from the last inequality that (4) holds.

In what follows we show that $(x_n{)}_{n\ge 0}$ actually converges strongly to ${\mathrm{proj}}_{\mathrm{Fix}T}\left(0\right):=\overline{x}$. Since T is nonexpansive, we have for any $n\ge 0$:

$$\begin{array}{rl}\parallel x_{n+1}-\overline{x}\parallel & =\parallel (1-{\lambda }_n)({\beta }_n{x}_n-\overline{x})+{\lambda }_n\left(T\left({\beta }_n{x}_n\right)-T\overline{x}\right)\parallel \\ & \le (1-{\lambda }_n)\parallel {\beta }_n{x}_n-\overline{x}\parallel +{\lambda }_n\parallel T\left({\beta }_n{x}_n\right)-T\overline{x}\parallel \\ & \le \parallel {\beta }_n{x}_n-\overline{x}\parallel .\end{array}$$

Hence,

$$\begin{array}{rl}\parallel x_{n+1}-\overline{x}{\parallel }^2& \le \parallel {\beta }_n{x}_n-\overline{x}{\parallel }^2\\ & =\parallel {\beta }_n(x_n-\overline{x})+({\beta }_n-1)\overline{x}{\parallel }^2\\ & ={\beta }_n^2\parallel x_n-\overline{x}{\parallel }^2+2{\beta }_n(1-{\beta }_n)⟨-\overline{x},x_n-\overline{x}⟩+(1-{\beta }_n{)}^2\parallel \overline{x}{\parallel }^2\\ & \le {\beta }_n\parallel x_n-\overline{x}{\parallel }^2+(1-{\beta }_n)\left(2{\beta }_n⟨-\overline{x},x_n-\overline{x}⟩+(1-{\beta }_n)\parallel \overline{x}{\parallel }^2\right)\mathrm{\forall }n\ge 0.\end{array}$$ (5)

Next we show that

$$\underset{n\to +\mathrm{\infty }}{lim sup}⟨-\overline{x},x_n-\overline{x}⟩\le 0.$$ (6)

Assuming the contrary, there would exist a positive real number l and a subsequence $(x_{k_j}{)}_{j\ge 0}$ such that

$$⟨-\overline{x},x_{k_j}-\overline{x}⟩\ge l>0\mathrm{\forall }j\ge 0.$$

Due to the boundedness of the sequence $(x_n{)}_{n\ge 0}$, we can assume without losing the generality that $(x_{k_j}{)}_{j\ge 0}$ weakly converges to an element $y\in \mathcal{H}$. According to Lemma 2, by taking into consideration (4), it follows that $y\in \mathrm{Fix}T$. From this and the variational characterization of the projection we easily derive

$$\underset{j\to +\mathrm{\infty }}{lim}⟨-\overline{x},x_{k_j}-\overline{x}⟩=⟨-\overline{x},y-\overline{x}⟩\le 0,$$

which leads to a contradiction. This shows that (6) holds. Thus

$$\underset{n\to +\mathrm{\infty }}{lim sup}\left(2{\beta }_n⟨-\overline{x},x_n-\overline{x}⟩+(1-{\beta }_n)\parallel \overline{x}{\parallel }^2\right)\le 0.$$

A direct application of Lemma 1 to (5), for $a_n:=\parallel x_n-\overline{x}{\parallel }^2,b_n:=2{\beta }_n⟨-\overline{x},x_n-\overline{x}⟩+(1-{\beta }_n)\parallel \overline{x}{\parallel }^2,{ϵ}_n:=0$ and ${\theta }_n:=1-{\beta }_n,n\ge 0,$ delivers the desired conclusion.

Remark 4

Condition (ii) in the previous theorem is satisfied by every monotonically increasing (and, in consequence, convergent) sequence $({\lambda }_n{)}_{n\ge 0}\subseteq (0,1]$ and also by every monotonically decreasing (and, in consequence, convergent) sequence $({\lambda }_n{)}_{n\ge 0}\subseteq (0,1]$ having as limit a positive number.

Condition (i) in the previous theorem is satisfied by every monotonically increasing sequence $({\beta }_n{)}_{n\ge 0}\subseteq (0,1]$ which fulfils $\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1$ and $\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$, as it is for instance the sequence with ${\beta }_0\in (0,\frac{1}{2})$ and ${\beta }_n=1-1/(n+1)$ for any $n\ge 1$.

An immediate consequence of Theorem 3 is the following corollary, which proposes an iterative scheme that finds a minimal norm solution of the set of fixed points of an averaged operator. Let $\alpha \in (0,1)$ be fixed. We say that $R:\mathcal{H}\to \mathcal{H}$ is an α-averaged operator if there exists a nonexpansive operator $T:\mathcal{H}\to \mathcal{H}$ such that $R=(1-\alpha )\mathrm{Id}+\alpha T$. It is obvious that α-averaged operators are also nonexpansive. The $\frac{1}{2}$-averaged operators are nothing else than the firmly nonexpansive ones and form the most important representatives of this class. For properties and insights into these families of operators we refer the reader to [6]. The following result will play in the next section a determinant role in the converge analysis of the forward–backward method endowed with Tikhonov regularization term.

Corollary 5

Consider the iterative scheme

$$x_{n+1}={\beta }_n{x}_n+{\lambda }_n\left(R\left({\beta }_n{x}_n\right)-{\beta }_n{x}_n\right)\mathrm{\forall }n\ge 0,$$ (7)

with $x_0\in \mathcal{H}$ as starting point, $R:\mathcal{H}\to \mathcal{H}$ an α-averaged operator, for $\alpha \in (0,1),$ such that $\mathrm{Fix}R\ne \mathrm{\varnothing }$ and $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le 1/\alpha$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$. Then $(x_n{)}_{n\ge 0}$ converges strongly to ${\mathrm{proj}}_{\mathrm{Fix}R}\left(0\right)$.

Proof.

Since R is α-averaged, there exists a nonexpansive operator $T:\mathcal{H}\to \mathcal{H}$ such that $R=(1-\alpha )\mathrm{Id}+\alpha T$. The conclusion follows from Theorem 3, by taking into account that (7) is equivalent to

$$x_{n+1}={\beta }_n{x}_n+\alpha {\lambda }_n\left(T\left({\beta }_n{x}_n\right)-{\beta }_n{x}_n\right)\mathrm{\forall }n\ge 0$$

and that $\mathrm{Fix}R=\mathrm{Fix}T$.

3. A forward–backward algorithm with Tikhonov regularization term

This section is dedicated to the formulation and convergence analysis of a forward–backward algorithm with Tikhonov regularization terms, which generates a sequence of iterates that converges strongly to the minimal norm solution of the set of zeros of the sum of two maximally monotone operators, one of them being single-valued.

For readers' convenience, we recall some standard notions and results in monotone operator theory which will be used in the following (see also [6,7,26]). For an arbitrary set-valued operator $A:\mathcal{H}\backslash rightrightarrows\mathcal{H}$ we denote by $\mathrm{Gr}A=\left\{\right(x,u)\in \mathcal{H}\times \mathcal{H}:u\in Ax\}$ its graph. Then $A^{-1}:\mathcal{H}\backslash rightrightarrows\mathcal{H}$, which is the operator with $\mathrm{Gr}{A}^{-1}=\left\{\right(x,u)\in \mathcal{H}\times \mathcal{H}:x\in Au\}$, denotes the inverse operator of A. We use also the notation $\mathrm{zer}A=\{x\in \mathcal{H}:0\in Ax\}$ for the set of zeros of A. We say that A is monotone, if $⟨x-y,u-v⟩\ge 0$ for all $(x,u),(y,v)\in \mathrm{Gr}A$. A monotone operator A is said to be maximally monotone, if there exists no proper monotone extension of the graph of A on $\mathcal{H}\times \mathcal{H}$. The resolvent of A, $J_A:\mathcal{H}\backslash rightrightarrows\mathcal{H}$, is defined by

$$J_A=(\mathrm{Id}+A{)}^{-1},$$

where $\mathrm{Id}:\mathcal{H}\to \mathcal{H},\mathrm{Id}\left(x\right)=x$ for all $x\in \mathcal{H}$, is the identity operator on $\mathcal{H}$. Moreover, if A is maximally monotone, then $J_A:\mathcal{H}\to \mathcal{H}$ is single-valued and maximally monotone (see [6, Proposition 23.7 and Corollary 23.10]).

Let $\gamma >0$. We say that $B:\mathcal{H}\to \mathcal{H}$ is γ-cocoercive, if $⟨x-y,Bx-By⟩\ge \gamma \parallel Bx-By{\parallel }^2$ for all $x,y\in \mathcal{H}$.

The following technical result (see [21, Theorem 3(b)] and [16, Proposition 2.4]) gives an expression for the averaged parameter of the composition of two averaged operators. We refer also to [6, Proposition 4.32] for other results of this type.

Proposition 6

Let $T_i:\mathcal{H}\to \mathcal{H}$ be ${\alpha }_i$-averaged, where ${\alpha }_i\in (0,1),$i=1,2. Then the composition $T_1\circ T_2$ is α-averaged, where

$$\alpha =\frac{{\alpha }_1+{\alpha }_2-2{\alpha }_1{\alpha }_2}{1-{\alpha }_1{\alpha }_2}\in (0,1).$$

Theorem 7

Let $A:\mathcal{H}\backslash rightrightarrows\mathcal{H}$ be a maximally monotone operator and $B:\mathcal{H}\to \mathcal{H}$ a β-cocoercive operator, for $\beta >0,$ such that $\mathrm{zer}(A+B)\ne \mathrm{\varnothing }$. Let $\gamma \in (0,2\beta ]$. Consider the iterative scheme

$$x_{n+1}=(1-{\lambda }_n){\beta }_n{x}_n+{\lambda }_n{J}_{\gamma A}\left({\beta }_n{x}_n-\gamma B\left({\beta }_n{x}_n\right)\right)\mathrm{\forall }n\ge 0,$$ (8)

with $x_0\in \mathcal{H}$ as starting point and $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le (4\beta -\gamma )/2\beta$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$.

Then $(x_n{)}_{n\ge 0}$ converges strongly to ${\mathrm{proj}}_{\mathrm{zer}(A+B)}\left(0\right)$.

Proof.

It is immediate that the iterative scheme (8) can be written in the form

$$x_{n+1}={\beta }_n{x}_n+{\lambda }_n\left(T\left({\beta }_n{x}_n\right)-{\beta }_n{x}_n\right)\mathrm{\forall }n\ge 0,$$

where $T=J_{\gamma A}\circ (\mathrm{Id}-\gamma B)$.

We consider two cases. The first one is when $\gamma \in (0,2\beta )$.

According to [6, Corollary 23.8 and Remark 4.24(iii)], $J_{\gamma A}$ is $\frac{1}{2}$-cocoercive. Moreover, by [6, Proposition 4.33], $\mathrm{Id}-\gamma B$ is $\gamma /2\beta$-averaged. Combining this with Proposition 6, we derive that T is $2\beta /(4\beta -\gamma )$-averaged. The statement follows now from Corollary 5, by noticing that $\mathrm{Fix}T=\mathrm{zer}(A+B)$ (see [6, Proposition 25.1(iv)]).

The second case is when $\gamma =2\beta$. The cocoercivity of B implies that $\mathrm{Id}-\gamma B$ is nonexpansive, hence the operator $T=J_{\gamma A}\circ (\mathrm{Id}-\gamma B)$ is nonexpansive, too, the conclusion follows in this situation from Theorem 3.

Remark 8

The choice ${\lambda }_n=1$ for any $n\ge 0$ in the previous theorem leads to the iterative scheme

$$x_{n+1}=J_{\gamma A}\left({\beta }_n{x}_n-\gamma B\left({\beta }_n{x}_n\right)\right)\mathrm{\forall }n\ge 0,$$

which further becomes in case B=0

$$x_{n+1}=J_{\gamma A}\left({\beta }_n{x}_n\right)\mathrm{\forall }n\ge 0.$$

This last relation can be equivalently written as

$$x_n\in \frac{1}{{\beta }_n}{x}_{n+1}+\frac{\gamma }{{\beta }_n}A{x}_{n+1}=\left(\mathrm{Id}+{ϵ}_n\mathrm{Id}+\frac{\gamma }{{\beta }_n}A\right)\left(x_{n+1}\right),$$

where ${ϵ}_n\mathrm{Id}$ (with ${ϵ}_n:=1/{\beta }_n-1>0$ and $\underset{n\to +\mathrm{\infty }}{lim}{ϵ}_n=0$) represents the Tikhonov regularization term, which enforces the strong convergence of the sequence $(x_n{)}_{n\ge 0}$ to the minimal norm solution. For other types of Tikhonov-like methods for monotone inclusion problems we refer the reader to [19,25,27] and the references therein.

In the remaining of this section we turn our attention to the solving of optimization problems of the form

$$\underset{x\in \mathcal{H}}{min}\left\{f\right(x)+g(x\left)\right\},$$ (9)

where $f:\mathcal{H}\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ is a proper, convex and lower semicontinuous function and $g:\mathcal{H}\to \mathbb{R}$ is a convex and Fréchet differentiable function with $1/\beta$-Lipschitz continuous gradient, for $\beta >0$.

For a proper, convex and lower semicontinuous function $f:\mathcal{H}\to \mathbb{R}\cup \{+\mathrm{\infty }\}$, its (convex) subdifferential at $x\in \mathcal{H}$ is defined as

$$\mathrm{\partial }f\left(x\right)=\{u\in \mathcal{H}:f(y)\ge f(x)+⟨u,y-x⟩\mathrm{\forall }y\in \mathcal{H}\},$$

for $x\in \mathcal{H}$ with $f\left(x\right)=+\mathrm{\infty }$ and as $\mathrm{\partial }f\left(x\right)=\mathrm{\varnothing }$, otherwise. When seen as a set-valued mapping, the convex subdifferential is a maximally monotone operator (see [22]) and its resolvent is given by $J_{\mathrm{\partial }f}={\mathrm{prox}}_f$ (see [6]), where ${\mathrm{prox}}_f:\mathcal{H}\to \mathcal{H}$,

$${\mathrm{prox}}_f\left(x\right)=\underset{y\in \mathcal{H}}{\mathrm{argmin}}\left\{f\left(y\right)+\frac{1}{2}\parallel y-x{\parallel }^2\right\},$$ (10)

denotes the proximal operator of f.

Corollary 9

Let $f:\mathcal{H}\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ be a proper, convex and lower semicontinuous function and $g:\mathcal{H}\to \mathbb{R}$ a convex and Fréchet differentiable function with $1/\beta$-Lipschitz continuous gradient, for $\beta >0,$ such that ${\mathrm{argmin}}_{x\in \mathcal{H}}\left\{f\right(x)+g(x\left)\right\}\ne \mathrm{\varnothing }$. Let $\gamma \in (0,2\beta ]$. Consider the iterative scheme

$$x_{n+1}=(1-{\lambda }_n){\beta }_n{x}_n+{\lambda }_n{\mathrm{prox}}_{\gamma f}\left({\beta }_n{x}_n-\gamma \mathrm{\nabla }g\left({\beta }_n{x}_n\right)\right)\mathrm{\forall }n\ge 0,$$ (11)

with $x_0\in \mathcal{H}$ as starting point and $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le (4\beta -\gamma )/2\beta$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$.

Then $(x_n{)}_{n\ge 0}$ converges strongly to the minimal norm solution of (9).

Proof.

The statement is a direct consequence of Theorem 7, by choosing $A:=\mathrm{\partial }f$ and $B:=\mathrm{\nabla }g$ and by taking into account that

$$\mathrm{zer}(\mathrm{\partial }f+\mathrm{\nabla }g)=\underset{x\in \mathcal{H}}{\mathrm{argmin}}\left\{f\right(x)+g(x\left)\right\}$$

and the fact that $\mathrm{\nabla }g$ is β-cocoercive due to the Baillon–Haddad Theorem (see [6, Corollary 18.16]).

4. A Douglas–Rachford algorithm with Tikhonov regularization term

In this section we derive from the Krasnosel'skiĭ–Mann algorithm formulated in Section 2 an iterative scheme of Douglas–Rachford-type, which generates sequences that strongly converge to a zero of of the sum of two set-valued maximally monotone operators.

In what follows, we denote by $R_A=2J_A-\mathrm{Id}$ the reflected resolvent of a maximally monotone operator $A:\mathcal{H}\backslash rightrightarrows\mathcal{H}$.

Theorem 10

Let $A,B:\mathcal{H}\backslash rightrightarrows\mathcal{H}$ be two maximally monotone operators such that $\mathrm{zer}(A+B)\ne \mathrm{\varnothing }$ and $\gamma >0$. Consider the following iterative scheme:

$$(\mathrm{\forall }n\ge 0)\left\{\begin{array}{l}{y}_n=J_{\gamma B}\left({\beta }_n{x}_n\right)\\ z_n=J_{\gamma A}(2y_n-{\beta }_n{x}_n)\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(z_n-y_n)\end{array}\right.$$

with $x_0\in \mathcal{H}$ as starting point and $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le 2$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$.

Then the following statements are true:

1. $(x_n{)}_{n\ge 0}$ converges strongly to $\overline{x}:={\mathrm{proj}}_{\mathrm{Fix}{R}_{\gamma A}{R}_{\gamma B}}\left(0\right)$ as $n\to +\mathrm{\infty };$

2. $(y_n{)}_{n\ge 0}$ and $(z_n{)}_{n\ge 0}$ converge strongly to $J_{\gamma B}\left(\overline{x}\right)\in \mathrm{zer}(A+B)$ as $n\to +\mathrm{\infty }$.

Proof.

Taking into account the iteration rules and the definition of the reflected resolvent, the iterative scheme in the enunciation of the theorem can be equivalently written as

$$\begin{array}{rl}{x}_{n+1}& ={\beta }_n{x}_n+{\lambda }_n\left[J_{\gamma A}\circ (2J_{\gamma B}-\mathrm{Id})\left({\beta }_n{x}_n\right)-J_{\gamma B}\left({\beta }_n{x}_n\right)\right]\\ & ={\beta }_n{x}_n+{\lambda }_n\left[\left(\frac{\mathrm{Id}+R_{\gamma A}}{2}\circ R_{\gamma B}\right)\left({\beta }_n{x}_n\right)-\frac{\mathrm{Id}+R_{\gamma B}}{2}\left({\beta }_n{x}_n\right)\right]\\ & ={\beta }_n{x}_n+\frac{{\lambda }_n}{2}\left(T\left({\beta }_n{x}_n\right)-{\beta }_n{x}_n\right)\mathrm{\forall }n\ge 0,\end{array}$$ (12)

where $T:=R_{\gamma A}\circ R_{\gamma B}:\mathcal{H}\to \mathcal{H}$ is a nonexpansive operator (see [6, Corollary 23.10(ii)]). From [6, Proposition 25.1(ii)] we have $\mathrm{zer}(A+B)=J_{\gamma B}\left(\mathrm{Fix}T\right)$, hence $\mathrm{Fix}T\ne \mathrm{\varnothing }$. By applying Theorem 3, we obtain that $(x_n{)}_{n\ge 0}$ converges strongly to $\overline{x}:={\mathrm{proj}}_{\mathrm{Fix}T}\left(0\right)$ as $n\to +\mathrm{\infty }$, hence (a) holds.

Further, by taking into account the definition of the sequence $(y_n{)}_{n\ge 0}$ and the continuity of the resolvent operator, we obtain that $(y_n{)}_{n\ge 0}$ converges strongly to $J_{\gamma B}\overline{x}\in \mathrm{zer}(A+B)$ as $n\to +\mathrm{\infty }$. Finally, by taking the limit in the recursive formula of the sequence $(x_n{)}_{n\ge 0}$, we obtain that $z_n-y_n$ converges strongly to 0 as $n\to +\mathrm{\infty }$, thus (b) holds, too.

Remark 11

The classical Douglas–Rachford method, which reads

$$(\mathrm{\forall }k\ge 0)\left\{\begin{array}{l}{y}_n=J_{\gamma B}\left(x_n\right)\\ z_n=J_{\gamma A}(2y_n-x_n)\\ x_{n+1}=x_n+{\lambda }_n(z_n-y_n),\end{array}\right.$$

produces sequences for which in general only weak convergence to a zero of the A+B can be proved (see e.g. [6, Theorem 25.6]). In order to ensure strong convergence, one usually needs to impose restrictive conditions on the monotone operators involved, like uniform monotonicity (which is a generalization of strong monotonicity). This is not the case for the iterative scheme stated in Theorem 10, where we are able to guarantee strong convergence in the very general situation of maximally monotone operators.

Further, we look at optimization problems of the form

$$\underset{x\in \mathcal{H}}{min}\left\{f\right(x)+g(x\left)\right\},$$ (13)

where $f,g:\mathcal{H}\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ are proper, convex and lower semicontinuous functions. We denote by

$$\text{dom }f=\{x\in \mathcal{H}:f(x)<+\mathrm{\infty }\}$$

the effective domain of the function f.

In order to proceed, we need the following notion. For $S\subseteq \mathcal{H}$ a convex set, we denote by

$$\text{sqri }S:=\{x\in S:{\cup }_{\lambda >0}\lambda (S-x\left)\text{is a closed linear subspace of}\mathcal{H}\right\}$$

its strong quasi-relative interior. Notice that we always have $\mathrm{int}S\subseteq \mathrm{sqri}S$ (in general this inclusion may be strict). If $\mathcal{H}$ is finite-dimensional, then $\mathrm{sqri}S$ coincides with $\mathrm{ri}S$, the relative interior of S, which is the interior of S with respect to its affine hull. The notion of strong quasi-relative interior belongs to the class of generalized interiority notions and plays an important role in the formulation of regularity conditions, which are needed in convex optimization in order to guarantee duality results and also subdifferential sum formulas. The one considered in the next result is the so-called Attouch-Brézis regularity condition. We refer to [6,7,26,29] for more interiority notions and their impact on the duality theory.

Corollary 12

Let $f,g:\mathcal{H}\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ be proper, convex and lower semicontinuous functions such that ${\mathrm{argmin}}_{x\in \mathcal{H}}\left\{f\right(x)+g(x\left)\right\}\ne \mathrm{\varnothing }$ and $0\in \mathrm{sqri}(\mathrm{dom}f-\mathrm{dom}g)$ and $\gamma >0$. Consider the following iterative scheme:

$$(\mathrm{\forall }n\ge 0)\left\{\begin{array}{l}{y}_n={\mathrm{prox}}_{\gamma g}\left({\beta }_n{x}_n\right)\\ z_n={\mathrm{prox}}_{\gamma f}(2y_n-{\beta }_n{x}_n)\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(z_n-y_n)\end{array}\right.$$

with $x_0\in \mathcal{H}$ as starting point and $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le 2$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$.

Then the following statements are true:

1. $(x_n{)}_{n\ge 0}$ converges strongly to $\overline{x}:={\mathrm{proj}}_{\mathrm{Fix}T}\left(0\right)$ as $n\to +\mathrm{\infty },$ where $T=(2{\mathrm{prox}}_{\gamma f}-\mathrm{Id})\circ (2{\mathrm{prox}}_{\gamma g}-\mathrm{Id});$

2. $(y_n{)}_{n\ge 0}$ and $(z_n{)}_{n\ge 0}$ converge strongly to ${\mathrm{prox}}_{\gamma g}\left(\overline{x}\right)\in {\mathrm{argmin}}_{x\in \mathcal{H}}\left\{f\right(x)+g(x\left)\right\}$ as $n\to +\mathrm{\infty }$.

Proof.

The result is a direct consequence of Theorem 10 for $A=\mathrm{\partial }f$ and $B=\mathrm{\partial }g$ and by noticing that the regularity condition $0\in \mathrm{sqri}(\text{dom }f-\text{dom }g)$ ensures the relation (see [6, Proposition 7.2])

$$\mathrm{zer}(\mathrm{\partial }f+\mathrm{\partial }g)=\underset{x\in \mathcal{H}}{\mathrm{argmin}}\left\{f\right(x)+g(x\left)\right\}.$$

5. Strongly convergent primal–dual algorithms

The aim of this section is to induce strong convergence in the nowadays so popular primal–dual algorithms designed for solving highly structured monotone inclusions involving parallel-sums and compositions with linear operators.

5.1. A primal–dual algorithm of forward–backward-type with Tikhonov regularization terms

In this subsection, the following monotone inclusion problem will be in the focus of our investigations.

Problem 13

Let $A:\mathcal{H}\backslash rightrightarrows\mathcal{H}$ be a maximally monotone operator and $C:\mathcal{H}\to \mathcal{H}$ a μ-cocoercive operator, for some $\mu >0$. Let m be a strictly positive integer and for any $i=1,\dots ,m,$ let ${\mathcal{G}}_i$ be a real Hilbert space, $B_i,D_i:{\mathcal{G}}_i\backslash rightrightarrows{\mathcal{G}}_i$ be maximally monotone operators such that $D_i$ are ${\nu }_i$-strongly monotone, for some ${\nu }_i>0,$ and $L_i:\mathcal{H}\to {\mathcal{G}}_i$ be a nonzero linear continuous operator. The problem is to solve the primal inclusion

$$\text{find }\overline{x}\in \mathcal{H}\text{such that}0\in A\overline{x}+\sum _{i=1}^m{L}_i^{\ast }\left(B_i◻D_i\right)\left(L_i\overline{x}\right)+C\overline{x},$$ (14)

together with the dual inclusion of Attouch-Théra type $($see [3,15,28])

$$\text{ find }{\overline{v}}_1\in {\mathcal{G}}_1,\dots ,{\overline{v}}_m\in {\mathcal{G}}_m\text{such that }\mathrm{\exists }x\in \mathcal{H}:\left\{\begin{array}{l}-\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\in Ax+Cx\\ {\overline{v}}_i\in \left(B_i◻D_i\right)\left(L_{i}x\right),i=1,\dots ,m.\end{array}\right.$$ (15)

Some of the notations used above are to be specified. The operator $L_i^{\ast }:{\mathcal{G}}_i\to \mathcal{H}$, defined via $⟨L_{i}x,y⟩=⟨x,L_i^{\ast }y⟩$ for all $x\in \mathcal{H}$ and all $y\in {\mathcal{G}}_i$, denotes the adjoint of the linear continuous operator $L_i:\mathcal{H}\to {\mathcal{G}}_i$, for $i=1,\dots ,m$. We say that $D_i:{\mathcal{G}}_i\backslash rightrightarrows{\mathcal{G}}_i$ is ${\nu }_i$-strongly monotone, for some ${\nu }_i>0$, if $⟨x-y,u-v⟩\ge {\nu }_i\parallel x-y{\parallel }^2$ for all $(x,u),(y,v)\in \mathrm{Gr}{D}_i,i=1,\dots ,m.$ The parallel sum of the set-valued operators $B_i,D_i:{\mathcal{G}}_i\backslash rightrightarrows{\mathcal{G}}_i$ is defined as $B_i◻D_i:{\mathcal{G}}_i\backslash rightrightarrows{\mathcal{G}}_i,B_i◻D_i=(B_i^{-1}+D_i^{-1}{)}^{-1}$, for $i=1,\dots ,m$.

We say that $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)\in \mathcal{H}\times$${\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ is a primal–dual solution to Problem 13, if

$$-\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\in A\overline{x}+C\overline{x}\text{and}{\overline{v}}_i\in \left(B_i◻D_i\right)\left(L_i\overline{x}\right),i=1,\dots ,m.$$ (16)

It is easy to see that, if $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)\in \mathcal{H}\times$${\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ is a primal–dual solution to Problem 13, then $\overline{x}$ is a solution to (14) and $({\overline{v}}_1,\dots ,{\overline{v}}_m)\in$${\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ is a solution to (15). Moreover, if $\overline{x}\in \mathcal{H}$ is a solution to (14), then there exists $({\overline{v}}_1,\dots ,{\overline{v}}_m)\in$${\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ such that $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ is a primal–dual solution to Problem 13 and, if $({\overline{v}}_1,\dots ,{\overline{v}}_m)\in$${\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ is a solution to (15), then there exists $\overline{x}\in \mathcal{H}$ such that $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ is a primal–dual solution to Problem 13.

Theorem 14

In Problem 13, suppose that

$$0\in \mathrm{ran}\left(A+\sum _{i=1}^m{L}_i^{\ast }\circ \left(B_i◻D_i\right)\circ L_i+C\right).$$ (17)

Let τ and ${\sigma }_i,$$i=1,\dots ,m,$ be strictly positive numbers such that

$$2\cdot min\{{\tau }^{-1},{\sigma }_1^{-1},\dots ,{\sigma }_m^{-1}\}\cdot min\{\mu ,{\nu }_1,\dots ,{\nu }_m\}\left(1-\sqrt{\tau \sum _{i=1}^m{\sigma }_i\parallel L_i{\parallel }^2}\right)\ge 1.$$

Let be the starting point $(x_0,v_{1,0},\dots ,v_{m,0})\in \mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ and set:

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{p}_n=J_{\tau A}\left[{\beta }_n{x}_n-\tau \left({\beta }_n\sum _{i=1}^m{L}_i^{\ast }{v}_{i,n}+C\left({\beta }_n{x}_n\right)\right)\right]\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(p_n-{\beta }_n{x}_n)\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}{q}_{i,n}=J_{{\sigma }_i{B}_i^{-1}}\left[{\beta }_n{v}_{i,n}+{\sigma }_i\left(L_i(2p_n-{\beta }_n{x}_n)-D_i^{-1}\left({\beta }_n{v}_{i,n}\right)\right)\right]\\ v_{i,n+1}={\beta }_n{v}_{i,n}+{\lambda }_n(q_{i,n}-{\beta }_n{v}_{i,n})\end{array}\end{array}$$ (18)

where $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ are real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le (4\beta \rho -1)/2\beta \rho$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty },$

for

$$\beta =min\{\mu ,{\nu }_1,\dots ,{\nu }_m\}$$

and

$$\rho =min\{{\tau }^{-1},{\sigma }_1^{-1},\dots ,{\sigma }_m^{-1}\}\left(1-\sqrt{\tau \sum _{i=1}^m{\sigma }_i\parallel L_i{\parallel }^2}\right).$$

Then there exists a primal–dual solution $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ to Problem 13 such that the sequence of primal–dual iterates $(x_n,v_{1,n},\dots ,v_{m,n})$ converges strongly to $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ as $n\to +\mathrm{\infty }$.

Remark 15

1. Since $D_i:{\mathcal{G}}_i\backslash rightrightarrows{\mathcal{G}}_i$ is ${\nu }_i$-strongly monotone, we have that $D_i^{-1}:{\mathcal{G}}_i\to {\mathcal{G}}_i$ is ${\nu }_i$-cocoercive, for $i=1,\dots ,m$.

2. The resolvent of the inverse operator of a maximally monotone operator $M:\mathcal{H}\backslash rightrightarrows\mathcal{H}$ can be computed as follows (see [6]):

$$\mathrm{Id}=J_{\gamma M}+\gamma J_{{\gamma }^{-1}{M}^{-1}}\circ {\gamma }^{-1}\mathrm{Id}.$$ (19)

Proof.

The idea is to apply Theorem 7 in an appropriate product space under the use of appropriate renorming techniques (see [28]). We consider the Hilbert space $\mathit{K}=\mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ endowed with inner product and associated norm defined, for $(x,v_1,\dots ,v_m)$, $(y,q_1,\dots ,q_m)\in \mathit{K}$, via

$$\begin{array}{rl}⟨(x,v_1,\dots ,v_m),(y,q_1,\dots ,q_m){⟩}_{\mathit{K}}& =⟨x,y{⟩}_{\mathcal{H}}+\sum _{i=1}^m⟨v_i,q_i{⟩}_{{\mathcal{G}}_i}\\ \mathrm{and}\parallel (x,v_1,\dots ,v_m){\parallel }_{\mathit{K}}& =\sqrt{\parallel x{\parallel }_{\mathcal{H}}^2+\sum _{i=1}^m\parallel v_i{\parallel }_{{\mathcal{G}}_i}^2},\end{array}$$ (20)

respectively. Furthermore, we consider the set-valued operator

$$\mathit{M}:\mathit{K}\backslash rightrightarrows\mathit{K},(x,v_1,\dots ,v_m)\mapsto (Ax,B_1^{-1}{v}_1,\dots ,B_m^{-1}{v}_m),$$

which is maximally monotone, since A and $B_i$, $i=1,\dots ,m,$ are maximally monotone (see [6, Propositions 20.22 and 20.23]), and the linear continuous operator

$$\mathit{S}:\mathit{K}\to \mathit{K},(x,v_1,\dots ,v_m)\mapsto \left(\sum _{i=1}^m{L}_i^{\ast }{v}_i,-L_{1}x,\dots ,-L_{m}x\right),$$

which is skew-symmetric (i.e. ${\mathit{S}}^{\ast }=-\mathit{S}$) and hence maximally monotone (see [6, Example 20.30]). We also consider the single-valued operator

$$\mathit{Q}:\mathit{K}\to \mathit{K},(x,v_1,\dots ,v_m)\mapsto \left(Cx,D_1^{-1}{v}_1,\dots ,D_m^{-1}{v}_m\right),$$

which is once again maximally monotone, since C and $D_i$ are maximally monotone for $i=1,\dots ,m$. Therefore, since $\text{dom }\mathit{S}=\mathit{K}$, $\mathit{M}+\mathit{S}$ is maximally monotone (see [6, Corollary 24.4(i)]). According to [28, page 672]

$$\mathit{Q}\text{ is }\beta -\text{cocoercive}.$$

Further, one can easily verify that (17) is equivalent to $\mathrm{zer}(\mathit{M}+\mathit{S}+\mathit{Q})\ne \varnothing$ and (see also [15, page 317])

$$\begin{array}{rl}& (x,v_1,\dots ,v_m)\in \text{zer }\left(\mathit{M}+\mathit{S}+\mathit{Q}\right)\\ & \iff (x,v_1,\dots ,v_m)\text{ is a primal--dual solution to Problem }13.\end{array}$$ (21)

We also introduce the linear continuous operator

$$\mathit{V}:\mathit{K}\to \mathit{K},(x,v_1,\dots ,v_m)\mapsto \left(\frac{x}{\tau }-\sum _{i=1}^m{L}_i^{\ast }{v}_i,\frac{v_1}{{\sigma }_1}-L_{1}x,\dots ,\frac{v_m}{{\sigma }_m}-L_{m}x\right),$$

which is self-adjoint and ρ-strongly positive (see [28]), namely, the following inequality holds

$$⟨\mathit{x},\mathit{V}\mathit{x}{⟩}_{\mathit{K}}\ge \rho \parallel \mathit{x}{\parallel }_{\mathit{K}}^2\mathrm{\forall }\mathit{x}\in \mathit{K}.$$

Therefore, its inverse operator ${\mathit{V}}^{-1}$ exists and it fulfils $\parallel {\mathit{V}}^{-1}\parallel \le 1/\rho$.

The algorithmic scheme (18) in the statement of the theorem can be written by using this notations as

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{\beta }_n\left({\tau }^{-1}{x}_n-\sum _{i=1}^m{L}_i^{\ast }{v}_{i,n}\right)-{\tau }^{-1}{p}_n+\sum _{i=1}^m{L}_i^{\ast }{q}_{i,n}-C\left({\beta }_n{x}_n\right)\in A{p}_n+\sum _{i=1}^m{L}_i^{\ast }{q}_{i,n}\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(p_n-{\beta }_n{x}_n)\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}{\beta }_n({\sigma }_i^{-1}{v}_{i,n}-L_i{x}_n)-{\sigma }_i^{-1}{q}_{i,n}+L_i{p}_n-D_i^{-1}\left({\beta }_n{v}_{i,n}\right)\in B_i^{-1}\left(q_{i,n}\right)-L_i{p}_n\\ v_{i,n+1}={\beta }_n{v}_{i,n}+{\lambda }_n(q_{i,n}-{\beta }_n{v}_{i,n}),\end{array}\end{array}$$ (22)

By introducing the sequences

$${\mathit{x}}_n=(x_n,v_{1,n},\dots ,v_{m,n}),\text{and}{\mathit{y}}_n=(p_n,q_{1,n},\dots ,q_{m,n})\mathrm{\forall }n\ge 0,$$

the scheme (22) can equivalently be written in the form

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{\beta }_n\mathit{V}\left({\mathit{x}}_n\right)-\mathit{V}\left({\mathit{y}}_n\right)-\mathit{Q}\left({\beta }_n{\mathit{x}}_n\right)\in \left(\mathit{M}+\mathit{S}\right)\left({\mathit{y}}_n\right)\\ {\mathit{x}}_{n+1}={\beta }_n{\mathit{x}}_n+{\lambda }_n\left({\mathit{y}}_n-{\beta }_n{\mathit{x}}_n\right).\end{array}$$ (23)

Furthermore, we have for any $n\ge 0$

$$\begin{array}{rl}& {\beta }_n\mathit{V}\left({\mathit{x}}_n\right)-\mathit{V}\left({\mathit{y}}_n\right)-\mathit{Q}\left({\beta }_n{\mathit{x}}_n\right)\in \left(\mathit{M}+\mathit{S}\right)\left({\mathit{y}}_n\right)\\ & \iff \left({\beta }_n\mathit{V}-\mathit{Q}\circ \left({\beta }_n\mathrm{Id}\right)\right)\left({\mathit{x}}_n\right)\in (\mathit{M}+\mathit{S}+\mathit{V})\left({\mathit{y}}_n\right)\\ & \iff {\mathit{y}}_n=(\mathit{M}+\mathit{S}+\mathit{V}{)}^{-1}\left({\beta }_n\mathit{V}-\mathit{Q}\circ \left({\beta }_n\mathrm{Id}\right)\right)({\mathit{x}}_n)\\ & \iff {\mathit{y}}_n={\left(\mathrm{Id}+{\mathit{V}}^{-1}(\mathit{M}+\mathit{S})\right)}^{-1}\left({\beta }_n\mathrm{Id}-{\mathit{V}}^{-1}\circ \mathit{Q}\circ \left({\beta }_n\mathrm{Id}\right)\right)\left({\mathit{x}}_n\right)\\ & \iff {\mathit{y}}_n=J_{\mathit{A}}\left({\beta }_n{\mathit{x}}_n-\mathit{B}\left({\beta }_n{\mathit{x}}_n\right)\right),\end{array}$$

where

$$\mathit{A}:={\mathit{V}}^{-1}\left(\mathit{M}+\mathit{S}\right)\text{and}\mathit{B}:={\mathit{V}}^{-1}\mathit{Q}.$$ (24)

Let ${\mathit{K}}_{\mathit{V}}$ be the Hilbert space with inner product and norm defined, for $\mathit{x},\mathit{y}\in \mathit{K}$, by

$$⟨\mathit{x},\mathit{y}{⟩}_{{\mathit{K}}_{\mathit{V}}}=⟨\mathit{x},\mathit{V}\mathit{y}{⟩}_{\mathit{K}}\mathrm{and}\parallel \mathit{x}{\parallel }_{{\mathit{K}}_{\mathit{V}}}=\sqrt{⟨\mathit{x},\mathit{V}\mathit{x}{⟩}_{\mathit{K}}},$$ (25)

respectively. As the set-valued operators $\mathit{M}+\mathit{S}$ and $\mathit{Q}$ are maximally monotone on $\mathit{K}$, the operators $\mathit{A}$ and $\mathit{B}$ are maximally monotone on ${\mathit{K}}_{\mathit{V}}$ (see also [28]). Furthermore, $\mathit{B}$ is $\beta \rho$-cocoercive on ${\mathit{K}}_{\mathit{V}}$. Moreover, since $\mathit{V}$ is self-adjoint and ρ-strongly positive, weak and strong convergence in ${\mathit{K}}_{\mathit{V}}$ are equivalent with weak and strong convergence in $\mathit{K}$, respectively.

Taking this into account, it follows that (23) becomes

$$\left(\mathrm{\forall }n\ge 0\right){\mathit{x}}_{n+1}={\beta }_n{\mathit{x}}_n+{\lambda }_n\left(J_{\mathit{A}}\left({\beta }_n{\mathit{x}}_n-\mathit{B}\left({\beta }_n{\mathit{x}}_n\right)\right)-{\beta }_n{\mathit{x}}_n\right),$$

which is the algorithm presented in Theorem 7 for determining the zeros of $\mathit{A}+\mathit{B}$ in case $\gamma =1$. However, we have

$$\mathrm{zer}(\mathit{A}+\mathit{B})=\mathrm{zer}\left({\mathit{V}}^{-1}\left(\mathit{M}+\mathit{S}+\mathit{Q}\right)\right)=\mathrm{zer}(\mathit{M}+\mathit{S}+\mathit{Q}).$$

According to Theorem 7, ${\mathit{x}}_n$ converges strongly to ${\mathrm{proj}}_{\mathrm{zer}(\mathit{A}+\mathit{B})}(0,0,\dots ,0)$ in the space ${\mathit{K}}_{\mathit{V}}$ as $n\to +\mathrm{\infty }$ and the conclusion follows from (21).

In the remaining of this subsection we investigate the convergence property of the algorithm (18) in the context of simultaneously solving complexly structured convex optimization problems and their Fenchel duals. The problem under investigation is the following one.

Problem 16

Let $f\in \mathrm{\Gamma }\left(\mathcal{H}\right)$ and $h:\mathcal{H}\to \mathbb{R}$ be a convex and differentiable function with a ${\mu }^{-1}$-Lipschitz continuous gradient, for some $\mu >0$. Let m be a strictly positive integer and for $i=1,\dots ,m,$ let ${\mathcal{G}}_i$ be a real Hilbert space, $g_i,l_i\in \mathrm{\Gamma }\left({\mathcal{G}}_i\right)$ such that $l_i$ is ${\nu }_i$-strongly convex, for some ${\nu }_i>0$ and $L_i:\mathcal{H}\to {\mathcal{G}}_i$ a nonzero linear continuous operator. Consider the convex optimization problem

$$\underset{x\in \mathcal{H}}{inf}\left\{f\left(x\right)+\sum _{i=1}^m\left(g_i◻l_i\right)\left(L_{i}x\right)+h\left(x\right)\right\}$$ (26)

and its Fenchel-type dual problem

$$\underset{v_i\in {\mathcal{G}}_i,i=1,\dots ,m}{sup}\left\{-\left(f^{\ast }◻h^{\ast }\right)\left(-\sum _{i=1}^m{L}_i^{\ast }{v}_i\right)-\sum _{i=1}^m\left(g_i^{\ast }\left(v_i\right)+l_i^{\ast }\left(v_i\right)\right)\right\}.$$ (27)

We denote by $\mathrm{\Gamma }\left(\mathcal{H}\right)$ the set of proper, convex and lower semicontinuous functions defined on $\mathcal{H}$ with values in the extended real line $\mathbb{R}\cup \{+\mathrm{\infty }\}$. The conjugate of a function f is $f^{\ast }:\mathcal{H}\to \overline{\mathbb{R}}$, $f^{\ast }\left(p\right)=sup\{⟨p,x⟩-f(x):x\in \mathcal{H}\}$ for all $p\in \mathcal{H}$. Moreover, if $f\in \mathrm{\Gamma }\left(\mathcal{H}\right)$, then $f^{\ast }\in \mathrm{\Gamma }\left(\mathcal{H}\right)$, as well, and $(\mathrm{\partial }f{)}^{-1}=\mathrm{\partial }{f}^{\ast }$. Finally, having two proper functions $f,g:\mathcal{H}\to \overline{\mathbb{R}}$, their infimal convolution is defined by $f◻g:\mathcal{H}\to \overline{\mathbb{R}}$, $\left(f◻g\right)\left(x\right)=\underset{y\in \mathcal{H}}{inf}\left\{f\right(y)+g(x-y\left)\right\}$ for all $x\in \mathcal{H}$.

Corollary 17

In Problem 16, suppose that

$$0\in \mathrm{ran}\left(\mathrm{\partial }f+\sum _{i=1}^m{L}_i^{\ast }\circ \left(\mathrm{\partial }{g}_i◻\mathrm{\partial }{l}_i\right)\circ L_i+\mathrm{\nabla }h\right).$$ (28)

Let τ and ${\sigma }_i,$$i=1,\dots ,m,$ be strictly positive numbers such that

$$2\cdot min\{{\tau }^{-1},{\sigma }_1^{-1},\dots ,{\sigma }_m^{-1}\}\cdot min\{\mu ,{\nu }_1,\dots ,{\nu }_m\}\left(1-\sqrt{\tau \sum _{i=1}^m{\sigma }_i\parallel L_i{\parallel }^2}\right)\ge 1.$$

Let be the starting point $(x_0,v_{1,0},\dots ,v_{m,0})\in \mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ and set:

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{p}_n={\mathrm{prox}}_{\tau f}\left[{\beta }_n{x}_n-\tau \left({\beta }_n\sum _{i=1}^m{L}_i^{\ast }{v}_{i,n}+\mathrm{\nabla }h\left({\beta }_n{x}_n\right)\right)\right]\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(p_n-{\beta }_n{x}_n)\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}{q}_{i,n}={\mathrm{prox}}_{{\sigma }_i{g}_i^{\ast }}\left[{\beta }_n{v}_{i,n}+{\sigma }_i\left(L_i(2p_n-{\beta }_n{x}_n)-\mathrm{\nabla }{l}_i^{\ast }\left({\beta }_n{v}_{i,n}\right)\right)\right]\\ v_{i,n+1}={\beta }_n{v}_{i,n}+{\lambda }_n(q_{i,n}-{\beta }_n{v}_{i,n}),\end{array}\end{array}$$ (29)

where $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ are real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le (4\beta \rho -1)/2\beta \rho$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty },$

for

$$\beta :=min\{\mu ,{\nu }_1,\dots ,{\nu }_m\}$$

and

$$\rho :=min\{{\tau }^{-1},{\sigma }_1^{-1},\dots ,{\sigma }_m^{-1}\}\left(1-\sqrt{\tau \sum _{i=1}^m{\sigma }_i\parallel L_i{\parallel }^2}\right).$$

Then there exists $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)\in \mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ such that $(x_n,v_{1,n},\dots ,v_{m,n})$ converges strongly to $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ as $n\to +\mathrm{\infty }$ and $\overline{x}$ is an optimal solution of the problem (26), $({\overline{v}}_1,\dots ,{\overline{v}}_m)$ is an optimal solution of (27) and the optimal objective values of the two optimization problems coincide.

Remark 18

The proximal-point operator of the conjugate function can be computed via the Moreau's decomposition formula

$${\mathrm{prox}}_{\gamma f}+\gamma {\mathrm{prox}}_{\left(1/\gamma \right)f^{\ast }}\circ {\gamma }^{-1}\mathrm{Id}=\mathrm{Id},$$ (30)

which is valid for $\gamma >0$ and $f\in \mathrm{\Gamma }\left(\mathcal{H}\right)$ (see [6]).

Proof.

Consider the maximal monotone operators

$$A=\mathrm{\partial }f,C=\mathrm{\nabla }h,B_i=\mathrm{\partial }{g}_i\text{and}{D}_i=\mathrm{\partial }{l}_i,i=1,\dots ,m.$$

The Baillon–Haddad Theorem (see [6, Corollary 18.16]) ensures that C is μ-cocoercive. Since $l_i$ is ${\nu }_i$-strongly convex, $D_i$ is ${\nu }_i$-strongly monotone, for $i=1,\dots ,m.$ According to [6, Proposition 17.10, Theorem 18.15], $D_i^{-1}=\mathrm{\nabla }{l}_i^{\ast }$ is a monotone and ${\nu }_i^{-1}$-Lipschitz continuous operator for $i=1,\dots ,m$. The strong convexity of the functions $l_i$ guarantees that $g_i◻l_i\in \mathrm{\Gamma }\left({\mathcal{G}}_i\right)$ (see [6, Corollary 11.16, Proposition 12.14]) and $\mathrm{\partial }\left(g_i◻l_i\right)=\mathrm{\partial }{g}_i◻\mathrm{\partial }{l}_i,i=1,\dots ,m$, (see [6, Proposition 24.27]).

Hence, the monotone inclusion problem (14) reads

$$\text{find }\overline{x}\in \mathcal{H}\text{such that}0\in \mathrm{\partial }f\left(\overline{x}\right)+\sum _{i=1}^m{L}_i^{\ast }\left(\mathrm{\partial }{g}_i◻\mathrm{\partial }{l}_i\right)\left(L_i\overline{x}\right)+\mathrm{\nabla }h\left(\overline{x}\right),$$ (31)

while the dual monotone inclusion problem (15) reads

$$\text{ find }{\overline{v}}_1\in {\mathcal{G}}_1,\dots ,{\overline{v}}_m\in {\mathcal{G}}_m\text{such that }\mathrm{\exists }x\in \mathcal{H}:\left\{\begin{array}{l}-\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\in \mathrm{\partial }f\left(x\right)+\mathrm{\nabla }h\left(x\right)\\ {\overline{v}}_i\in \left(\mathrm{\partial }{g}_i◻\mathrm{\partial }{l}_i\right)\left(L_{i}x\right),i=1,\dots ,m.\end{array}\right.$$ (32)

One can see that if $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)\in \mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ is a primal–dual solution to (31)–(32), namely,

$$-\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\in \mathrm{\partial }f\left(\overline{x}\right)+\mathrm{\nabla }h\left(\overline{x}\right)\text{and}{\overline{v}}_i\in \left(\mathrm{\partial }{g}_i◻\mathrm{\partial }{l}_i\right)\left(L_i\overline{x}\right),i=1,\dots ,m,$$ (33)

then $\overline{x}$ is an optimal solution of the problem (26), $({\overline{v}}_1,\dots ,{\overline{v}}_m)$ is an optimal solution of (27) and the optimal objective values of the two problems coincide. Notice that (33) is nothing else than the system of optimality conditions for the primal–dual pair of convex optimization problems (26)–(27).

The conclusion follows now from Theorem 14.

Remark 19

(i) The relation (28) in the above theorem is fulfilled if the primal problem (26) has an optimal solution $\overline{x}\in \mathcal{H}$ and a suitable regularity condition holds. Under these auspices there exists an optimal solution to (27) $({\overline{v}}_1,\dots ,{\overline{v}}_m)\in {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$, such that $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ satisfies the optimality conditions (33) and, consequently, (28) holds.

(ii) Further, let us discuss some conditions ensuring the existence of a primal optimal solution. Suppose that the primal problem (26) is feasible, which means that its optimal objective value is not identical $+\mathrm{\infty }$. The existence of optimal solutions for (26) is guaranteed if, for instance, f+h is coercive (that is $\underset{\parallel x\parallel \to \mathrm{\infty }}{lim}(f+h)\left(x\right)=+\mathrm{\infty }$) and for all $i=1,\dots ,m$, $g_i$ is bounded from below. Indeed, under these circumstances, the objective function of (26) is coercive (use also [6, Corollary 11.16 and Proposition 12.14] to show that for all $i=1,\dots ,m$, $g_i◻l_i$ is bounded from below and $g_i◻l_i\in \mathrm{\Gamma }\left({\mathcal{G}}_i\right)$) and the statement follows via [6, Corollary 11.15]. On the other hand, if f+h is strongly convex, then the objective function of (26) is strongly convex, too, thus (26) has a unique optimal solution (see [6, Corollary 11.16]).

(iii) We discuss at this point a suitable regularity condition as mentioned at item (i) above. Since $\mathrm{dom}\left(g_i◻l_i\right)=\mathrm{dom}{g}_i+\mathrm{dom}{l}_i,i=1,\dots ,m$, one can use to this end the regularity condition of interiority-type (see also [15])

$$(0,\dots ,0)\in \mathrm{sqri}\left(\prod _{i=1}^m(\text{dom }{g}_i+\text{dom }{l}_i)-\left\{\right(L_{1}x,\dots ,L_{m}x):x\in \text{dom }f\}\right).$$ (34)

This is fulfilled provided that one of the following conditions is verified (see [15, Proposition 4.3]):

1. $\mathrm{dom}{g}_i+\mathrm{dom}{l}_i={\mathcal{G}}_i$, $i=1,\dots ,m$;

2. $\mathcal{H}$ and ${\mathcal{G}}_i$ are finite-dimensional and there exists $x\in \mathrm{ri}\mathrm{dom}f$ such that $L_{i}x-r_i\in \mathrm{ri}\mathrm{dom}{g}_i+\mathrm{ri}\mathrm{dom}{l}_i$, $i=1,\dots ,m$.

5.2. A primal–dual algorithm of Douglas–Rachford-type with Tikhonov regularization terms

The problem that we investigate in this section reads as follows.

Problem 20

Let $A:\mathcal{H}\backslash rightrightarrows\mathcal{H}$ be a maximally monotone operator. Let m be a strictly positive integer and for any $i=1,\dots ,m,$ let ${\mathcal{G}}_i$ be a real Hilbert space, $B_i,D_i:{\mathcal{G}}_i\backslash rightrightarrows{\mathcal{G}}_i$ be maximally monotone operators and $L_i:\mathcal{H}\to$${\mathcal{G}}_i$ a nonzero linear continuous operator. The problem is to solve the primal inclusion

$$\text{find }\overline{x}\in \mathcal{H}\text{such that }0\in A\overline{x}+\sum _{i=1}^m{L}_i^{\ast }\left(B_i◻D_i\right)\left(L_i\overline{x}\right)$$ (35)

together with the dual inclusion

$$\text{find }{\overline{v}}_1\in {\mathcal{G}}_1,\dots ,{\overline{v}}_m\in {\mathcal{G}}_m\text{such that }(\mathrm{\exists }x\in \mathcal{H})\left\{\begin{array}{l}-\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\in Ax\\ {\overline{v}}_i\in \left(B_i◻D_i\right)\left(L_{i}x\right),i=1,\dots ,m.\end{array}\right.$$ (36)

Different to Problem 13, the operators $D_i,i=1,\dots ,m$ are general maximally monotone operators, thus they will have to be addressed through their resolvents. This is why in this context a primal–dual algorithm relying on the Douglas–Rachford paradigm is more appropriate.

Theorem 21

In Problem 20, suppose that

$$0\in \mathrm{ran}\left(A+\sum _{i=1}^m{L}_i^{\ast }\circ \left(B_i◻D_i\right)\circ L_i\right).$$ (37)

Let $\tau ,{\sigma }_i>0,$$i=1,\dots ,m,$ be strictly positive numbers such that

$$\tau \sum _{i=1}^m{\sigma }_i\parallel L_i{\parallel }^2<4.$$

Let be the starting point $(x_0,v_{1,0},\dots ,v_{m,0})\in \mathcal{H}\times {\mathcal{G}}_1\cdots \times {\mathcal{G}}_m$ and set:

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{p}_{1,n}=J_{\tau A}\left({\beta }_n{x}_n-\frac{\tau }{2}{\beta }_n\sum _{i=1}^m{L}_i^{\ast }{v}_{i,n}\right)\\ w_{1,n}=2p_{1,n}-{\beta }_n{x}_n\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}{p}_{2,i,n}=J_{{\sigma }_i{B}_i^{-1}}\left({\beta }_n{v}_{i,n}+\frac{{\sigma }_i}{2}{L}_i{w}_{1,n}\right)\\ w_{2,i,n}=2p_{2,i,n}-{\beta }_n{v}_{i,n}\end{array}\\ z_{1,n}=w_{1,n}-\frac{\tau }{2}\sum _{i=1}^m{L}_i^{\ast }{w}_{2,i,n}\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(z_{1,n}-p_{1,n})\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}{z}_{2,i,n}=J_{{\sigma }_i{D}_i^{-1}}\left(w_{2,i,n}+\frac{{\sigma }_i}{2}{L}_i(2z_{1,n}-w_{1,n})\right)\\ v_{i,n+1}={\beta }_n{v}_{i,n}+{\lambda }_n(z_{2,i,n}-p_{2,i,n}),\end{array}\end{array}$$ (38)

where $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le 2$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$.

Then there exists an element $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)\in \mathcal{H}\times {\mathcal{G}}_1\cdots \times {\mathcal{G}}_m$ such that the following statements are true:

1. by setting

   $$\begin{array}{rl}{\overline{p}}_1& =J_{\tau A}\left(\overline{x}-\frac{\tau }{2}\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\right),\\ {\overline{p}}_{2,i}& =J_{{\sigma }_i{B}_i^{-1}}\left({\overline{v}}_i+\frac{{\sigma }_i}{2}{L}_i(2{\overline{p}}_1-\overline{x})\right),i=1,\dots ,m,\end{array}$$

   the element $({\overline{p}}_1,{\overline{p}}_{2,1},\dots ,{\overline{p}}_{2,m})\in \mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ is a primal–dual solution to Problem 20;

2. $(x_n,v_{1,n},\dots ,v_{m,n})$ converges strongly to $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ as $n\to +\mathrm{\infty };$

3. $(p_{1,n},p_{2,1,n},\dots ,p_{2,m,n})$ and $(z_{1,n},z_{2,1,n},\dots ,z_{2,m,n})$ converge strongly to $({\overline{p}}_1,{\overline{p}}_{2,1},\dots ,{\overline{p}}_{2,m})$ as $n\to +\mathrm{\infty }$.

Proof.

For the proof we use Theorem 10 (see also [8]) in the same setting as in the proof of Theorem 14, namely, by considering $\mathit{K}=\mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ endowed with inner product and associated norm defined in (20). Furthermore, we consider again the maximally monotone operator

$$\mathit{M}:\mathit{K}\backslash rightrightarrows\mathit{K},(x,v_1,\dots ,v_m)\mapsto (Ax,B_1^{-1}{v}_1,\dots ,B_m^{-1}{v}_m),$$

the linear continuous skew-symmetric operator

$$\mathit{S}:\mathit{K}\to \mathit{K},(x,v_1,\dots ,v_m)\mapsto \left(\sum _{i=1}^m{L}_i^{\ast }{v}_i,-L_{1}x,\dots ,-L_{m}x\right)$$

and the (this time not necessarily single-valued) maximally monotone operator

$$\mathit{Q}:\mathit{K}\backslash rightrightarrows\mathit{K},(x,v_1,\dots ,v_m)\mapsto \left(0,D_1^{-1}{v}_1,\dots ,D_m^{-1}{v}_m\right).$$

Since $\mathrm{dom}\mathit{S}=\mathit{K}$, both $\frac{1}{2}\mathit{S}+\mathit{Q}$ and $\frac{1}{2}\mathit{S}+\mathit{M}$ are maximally monotone (see [6, Corollary 24.4(i)]). Furthermore,

$$\left(37\right)\iff \mathrm{zer}\left(\mathit{M}+\mathit{S}+\mathit{Q}\right)\ne \varnothing$$

and

$$\begin{array}{rl}& (x,v_1,\dots ,v_m)\in \mathrm{zer}\left(\mathit{M}+\mathit{S}+\mathit{Q}\right)\\ & \iff (x,v_1,\dots ,v_m)\text{ is a primal--dual solution to Problem }20.\end{array}$$ (39)

We introduce the linear continuous operator

$$\mathit{V}:\mathit{K}\to \mathit{K},(x,v_1,\dots ,v_m)\mapsto \left(\frac{x}{\tau }-\frac{1}{2}\sum _{i=1}^m{L}_i^{\ast }{v}_i,\frac{v_1}{{\sigma }_1}-\frac{1}{2}{L}_{1}x,\dots ,\frac{v_m}{{\sigma }_m}-\frac{1}{2}{L}_{m}x\right),$$

which is self-adjoint and ρ-strongly positive (see also [8]), for

$$\rho :=\left(1-\frac{1}{2}\sqrt{\tau \sum _{i=1}^m{\sigma }_i\parallel L_i{\parallel }^2}\right)min\left\{\frac{1}{\tau },\frac{1}{{\sigma }_1},\dots ,\frac{1}{{\sigma }_m}\right\}>0,$$

namely, the following inequality holds

$$⟨\mathit{x},\mathit{V}\mathit{x}{⟩}_{\mathit{K}}\ge \rho \parallel \mathit{x}{\parallel }_{\mathit{K}}^2\mathrm{\forall }\mathit{x}\in \mathit{K}.$$

Therefore, its inverse operator ${\mathit{V}}^{-1}$ exists and it fulfils $\parallel {\mathit{V}}^{-1}\parallel \le 1/\rho$.

The algorithmic scheme (38) is equivalent to

$$(\mathrm{\forall }n\ge 0)⌊\begin{array}{l}\frac{{\beta }_n{x}_n-p_{1,n}}{\tau }-\frac{1}{2}\sum _{i=1}^m{L}_i^{\ast }({\beta }_n{v}_{i,n}-p_{2,i,n})\in \frac{1}{2}\sum _{i=1}^m{L}_i^{\ast }{p}_{2,i,n}+A{p}_{1,n}\\ w_{1,n}=2p_{1,n}-{\beta }_n{x}_n\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}\frac{{\beta }_n{v}_{i,n}-p_{2,i,n}}{{\sigma }_i}-\frac{1}{2}{L}_i({\beta }_n{x}_n-p_{1,n})\in -\frac{1}{2}{L}_i{p}_{1,n}+B_i^{-1}{p}_{2,i,n}\\ w_{2,i,n}=2p_{2,i,n}-{\beta }_n{v}_{i,n}\end{array}\\ \frac{w_{1,n}-z_{1,n}}{\tau }-\frac{1}{2}\sum _{i=1}^m{L}_i^{\ast }{w}_{2,i,n}=0\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(z_{1,n}-p_{1,n})\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}\frac{w_{2,i,n}-z_{2,i,n}}{{\sigma }_i}-\frac{1}{2}{L}_i(w_{1,n}-z_{1,n})\in -\frac{1}{2}{L}_i{z}_{1,n}+D_i^{-1}{z}_{2,i,n}\\ v_{i,n+1}={\beta }_n{v}_{i,n}+{\lambda }_n(z_{2,i,n}-p_{2,i,n}).\end{array}\end{array}$$ (40)

By considering for any $n\ge 0$ the notations

$${\mathit{x}}_n=(x_n,v_{1,n},\dots ,v_{m,n}),{\mathit{y}}_n=(p_{1,n},p_{2,1,n},\dots ,p_{2,m,n})\text{and}{\mathit{z}}_n=(z_{1,n},z_{2,1,n},\dots ,z_{2,m,n}),$$

the iterative scheme (40) can be written as

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}\mathit{V}({\beta }_n{\mathit{x}}_n-{\mathit{y}}_n)\in \left(\frac{1}{2}\mathit{S}+\mathit{M}\right){\mathit{y}}_n\\ \mathit{V}(2{\mathit{y}}_n-{\beta }_n{\mathit{x}}_n-{\mathit{z}}_n)\in \left(\frac{1}{2}\mathit{S}+\mathit{Q}\right){\mathit{z}}_n\\ {\mathit{x}}_{n+1}={\beta }_n{\mathit{x}}_n+{\lambda }_n\left({\mathit{z}}_n-{\mathit{y}}_n\right),\end{array}$$ (41)

which is further equivalent to

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{\mathit{y}}_n={\left(\mathrm{Id}+{\mathit{V}}^{-1}(\frac{1}{2}\mathit{S}+\mathit{M})\right)}^{-1}\left({\beta }_n{\mathit{x}}_n\right)\\ {\mathit{z}}_n={\left(\mathrm{Id}+{\mathit{V}}^{-1}(\frac{1}{2}\mathit{S}+\mathit{Q})\right)}^{-1}\left(2{\mathit{y}}_n-{\beta }_n{\mathit{x}}_n\right)\\ {\mathit{x}}_{n+1}={\beta }_n{\mathit{x}}_n+{\lambda }_n\left({\mathit{z}}_n-{\mathit{y}}_n\right).\end{array}$$ (42)

Let ${\mathit{K}}_{\mathit{V}}$ be the Hilbert space with inner product and norm defined, for $\mathit{x},\mathit{y}\in \mathit{K}$, via

$$⟨\mathit{x},\mathit{y}{⟩}_{{\mathit{K}}_{\mathit{V}}}=⟨\mathit{x},\mathit{V}\mathit{y}{⟩}_{\mathit{K}}\mathrm{and}\parallel \mathit{x}{\parallel }_{{\mathit{K}}_{\mathit{V}}}=\sqrt{⟨\mathit{x},\mathit{V}\mathit{x}{⟩}_{\mathit{K}}},$$ (43)

respectively. As the set-valued operators $\frac{1}{2}\mathit{S}+\mathit{M}$ and $\frac{1}{2}\mathit{S}+\mathit{Q}$ are maximally monotone on $\mathit{K}$, the operators

$$\mathit{B}:={\mathit{V}}^{-1}\left(\frac{1}{2}\mathit{S}+\mathit{M}\right)\text{and}\mathit{A}:={\mathit{V}}^{-1}\left(\frac{1}{2}\mathit{S}+\mathit{Q}\right)$$ (44)

are maximally monotone on ${\mathit{K}}_{\mathit{V}}$. Furthermore, since $\mathit{V}$ is self-adjoint and ρ-strongly positive, strong convergence in ${\mathit{K}}_{\mathit{V}}$ is equivalent with strong convergence in $\mathit{K}$.

Taking this into account, (42) becomes

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{\mathit{y}}_n=J_{\mathit{B}}\left({\beta }_n{\mathit{x}}_n\right)\\ {\mathit{z}}_n=J_{\mathit{A}}\left(2{\mathit{y}}_n-{\beta }_n{\mathit{x}}_n\right)\\ {\mathit{x}}_{n+1}={\beta }_n{\mathit{x}}_n+{\lambda }_n\left({\mathit{z}}_n-{\mathit{y}}_n\right),\end{array}$$ (45)

which is the Douglas–Rachford algorithm formulated in Theorem 10 in case $\gamma =1$ for determining the zeros of $\mathit{A}+\mathit{B}$. It is easy to see that

$$\mathrm{zer}(\mathit{A}+\mathit{B})=\mathrm{zer}\left({\mathit{V}}^{-1}\left(\mathit{M}+\mathit{S}+\mathit{Q}\right)\right)=\mathrm{zer}(\mathit{M}+\mathit{S}+\mathit{Q}).$$

By Theorem 10(a), there exists $\overline{\mathit{x}}=(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)\in \mathrm{Fix}\left(R_{\mathit{A}}{R}_{\mathit{B}}\right)$, such that $J_{\mathit{B}}\overline{\mathit{x}}\in \mathrm{zer}(\mathit{A}+\mathit{B})=\mathrm{zer}(\mathit{M}+\mathit{S}+\mathit{Q})$. The claim follows from Theorem 10, (39) and by writing $J_{\mathit{B}}\overline{\mathit{x}}$ in terms of the resolvents of the operators involved in the expression of $\mathit{B}$.

We close this section by considering the variational case.

Problem 22

Let $f\in \mathrm{\Gamma }\left(\mathcal{H}\right),$m be a strictly positive integer and for $i=1,\dots ,m,$ let ${\mathcal{G}}_i$ be a real Hilbert space, $g_i,l_i\in \mathrm{\Gamma }\left({\mathcal{G}}_i\right)$ and $L_i:\mathcal{H}\to {\mathcal{G}}_i$ a nonzero linear continuous operator. Consider the convex optimization problem

$$\underset{x\in \mathcal{H}}{inf}\left\{f\left(x\right)+\sum _{i=1}^m\left(g_i◻l_i\right)\left(L_{i}x\right)\right\}$$ (46)

and its conjugate dual problem

$$\underset{(v_1,\dots ,v_m)\in {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m}{sup}\left\{-f^{\ast }\left(-\sum _{i=1}^m{L}_i^{\ast }{v}_i\right)-\sum _{i=1}^m\left(g_i^{\ast }\left(v_i\right)+l_i^{\ast }\left(v_i\right)\right)\right\}.$$ (47)

Corollary 23

In Problem 22, suppose that

$$0\in \mathrm{ran}\left(\mathrm{\partial }f+\sum _{i=1}^m{L}_i^{\ast }\circ \left(\mathrm{\partial }{g}_i◻\mathrm{\partial }{l}_i\right)\circ L_i\right).$$ (48)

Let $\tau ,{\sigma }_i>0,$$i=1,\dots ,m,$ be strictly positive numbers such that

$$\tau \sum _{i=1}^m{\sigma }_i\parallel L_i{\parallel }^2<4.$$

Let be the starting point $(x_0,v_{1,0},\dots ,v_{m,0})\in \mathcal{H}\times {\mathcal{G}}_1\cdots \times {\mathcal{G}}_m$ and set:

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{p}_{1,n}={\mathrm{prox}}_{\tau f}\left({\beta }_n{x}_n-\frac{\tau }{2}{\beta }_n\sum _{i=1}^m{L}_i^{\ast }{v}_{i,n}\right)\\ w_{1,n}=2p_{1,n}-{\beta }_n{x}_n\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}{p}_{2,i,n}={\mathrm{prox}}_{{\sigma }_i{g}_i^{\ast }}\left({\beta }_n{v}_{i,n}+\frac{{\sigma }_i}{2}{L}_i{w}_{1,n}\right)\\ w_{2,i,n}=2p_{2,i,n}-{\beta }_n{v}_{i,n}\end{array}\\ z_{1,n}=w_{1,n}-\frac{\tau }{2}\sum _{i=1}^m{L}_i^{\ast }{w}_{2,i,n}\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(z_{1,n}-p_{1,n})\\ \text{For }i=1,\dots ,m\\ ⌊\begin{array}{l}{z}_{2,i,n}={\mathrm{prox}}_{{\sigma }_i{l}_i^{\ast }}\left(w_{2,i,n}+\frac{{\sigma }_i}{2}{L}_i(2z_{1,n}-w_{1,n})\right)\\ v_{i,n+1}={\beta }_n{v}_{i,n}+{\lambda }_n(z_{2,i,n}-p_{2,i,n}),\end{array}\end{array}$$ (49)

where $({\lambda }_n{)}_{n\ge 0}$ and $({\beta }_n{)}_{n\ge 0}$ are real sequences satisfying the conditions:

1. $0<{\beta }_n\le 1$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim}{\beta }_n=1,$$\sum _{n\ge 0}(1-{\beta }_n)=+\mathrm{\infty }$ and $\sum _{n\ge 1}|{\beta }_n-{\beta }_{n-1}|<+\mathrm{\infty };$

2. $0<{\lambda }_n\le 2$ for any $n\ge 0,$$\underset{n\to +\mathrm{\infty }}{lim inf}{\lambda }_n>0$ and $\sum _{n\ge 1}|{\lambda }_n-{\lambda }_{n-1}|<+\mathrm{\infty }$.

Then there exists an element $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)\in \mathcal{H}\times {\mathcal{G}}_1\cdots \times {\mathcal{G}}_m$ such that the following statements are true:

1. by setting

   $$\begin{array}{rl}{\overline{p}}_1& ={\mathrm{prox}}_{\tau f}\left(\overline{x}-\frac{\tau }{2}\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\right),\\ {\overline{p}}_{2,i}& ={\mathrm{prox}}_{{\sigma }_i{g}_i^{\ast }}\left({\overline{v}}_i+\frac{{\sigma }_i}{2}{L}_i(2{\overline{p}}_1-\overline{x})\right),i=1,\dots ,m,\end{array}$$

   the element $({\overline{p}}_1,{\overline{p}}_{2,1},\dots ,{\overline{p}}_{2,m})\in \mathcal{H}\times {\mathcal{G}}_1\times \cdots \times {\mathcal{G}}_m$ is a primal–dual solution to Problem 20, namely,

   $$-\sum _{i=1}^m{L}_i^{\ast }{\overline{v}}_i\in \mathrm{\partial }f\left(\overline{x}\right)\text{and}{\overline{v}}_i\in \left(\mathrm{\partial }{g}_i◻\mathrm{\partial }{l}_i\right)\left(L_i\overline{x}\right),i=1,\dots ,m,$$ (50)

   hence ${\overline{p}}_1$ is an optimal solution to (46) and $({\overline{p}}_{2,1},\dots ,{\overline{p}}_{2,m})$ is an optimal solution to (47);

2. $(x_n,v_{1,n},\dots ,v_{m,n})$ converges strongly to $(\overline{x},{\overline{v}}_1,\dots ,{\overline{v}}_m)$ as $n\to +\mathrm{\infty };$

3. $(p_{1,n},p_{2,1,n},\dots ,p_{2,m,n})$ and $(z_{1,n},z_{2,1,n},\dots ,z_{2,m,n})$ converge strongly to $({\overline{p}}_1,{\overline{p}}_{2,1},\dots ,{\overline{p}}_{2,m})$ as $n\to +\mathrm{\infty }$.

Remark 24

The hypothesis (48) in Theorem 23 is fulfilled, if the primal problem (46) has an optimal solution, the regularity condition (34) holds and

$$0\in \mathrm{sqri}(\text{dom }{g}_i^{\ast }-\text{dom }{l}_i^{\ast })\text{for}i=1,\dots ,m.$$

According to [6, Proposition 15.7], the latter also guarantees that $g_i◻l_i\in \mathrm{\Gamma }\left({\mathcal{G}}_i\right)$, $i=1,\dots ,m$.

6. Numerical experiments: applications to the split feasibility problem

Let $\mathcal{H}$ and $\mathcal{G}$ be real Hilbert spaces and $L:\mathcal{H}\to \mathcal{G}$ a bounded linear operator. Let C and Q be nonempty, closed and convex subsets of $\mathcal{H}$ and $\mathcal{G}$, respectively. The split feasibility problem (SFP) searches a point $x\in \mathcal{H}$ with the property

$$x\in C\mathrm{and}Lx\in Q.$$ (51)

The (SFP) was originally introduced by Censor and Elfving [11] for solving inverse problems in the context of phase retrieval, medical image reconstruction and intensity modulated radiation therapy.

We show that the strongly convergent primal–dual algorithms which we have investigated in Section 5 are excellently suited to solve the (SFP), especially in the case when infinite dimensional Hilbert spaces are involved. For this purpose, we note that problem (51) can be written equivalently in the form

$$\underset{x\in \mathcal{H}}{min}\left\{{\delta }_C\left(x\right)+{\delta }_Q\left(Lx\right)\right\},$$ (52)

where

$${\delta }_S\left(x\right):=\left\{\begin{array}{ll}0,& \text{if }x\in S\\ +\mathrm{\infty },& \mathrm{else}\end{array}\right.$$

denotes the indicator function of a subset S of a Hilbert space. Another alternative way to write problem (51) is as the minimization problem

$$\underset{x\in \mathcal{H}}{min}\left\{\frac{1}{2}{d}_C^2\left(x\right)+{\delta }_Q\left(Lx\right)\right\},$$ (53)

where $d_C\left(x\right)=\underset{u\in \mathcal{C}}{inf}\parallel x-u\parallel$ denotes the distance of the point $x\in \mathcal{H}$ to the set C. Both optimization problems are special instances of Problem 16, consequently, we will solve them by making use of the algorithm stated in Corollary 17.

The optimization problem (52) can be stated in the framework of Problem 16 by taking $f={\delta }_C$, m=1, $g_1={\delta }_Q$, $l_1={\delta }_{\left\{0\right\}}$, $L_1=L$ and h=0. The iterative scheme stated in Corollary 17 reads:

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{p}_n=P_C({\beta }_n{x}_n-\tau {\beta }_n{L}^{\ast }{v}_n)\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(p_n-{\beta }_n{x}_n)\\ q_n={\beta }_n{v}_n+\sigma L(2p_n-{\beta }_n{x}_n)-\sigma P_Q({\sigma }^{-1}{\beta }_n{v}_n+L(2p_n-{\beta }_n{x}_n\left)\right)\\ v_{n+1}={\beta }_n{v}_n+{\lambda }_n(q_n-{\beta }_n{v}_n).\end{array}$$ (54)

The optimization problem (52) can be stated in the framework of Problem 16 by taking f=0, m=1, $g_1={\delta }_Q$, $l_1={\delta }_{\left\{0\right\}}$, $L_1=L$ and $h=\frac{1}{2}{d}_C^2$. Noticing that $\mathrm{\nabla }\left(\frac{1}{2}{d}_C^2\right)=\mathrm{Id}-P_C$, the iterative scheme stated in Corollary 17 reads:

$$\left(\mathrm{\forall }n\ge 0\right)⌊\begin{array}{l}{p}_n={\beta }_n{x}_n-\tau ({\beta }_n{L}^{\ast }{v}_n+{\beta }_n{x}_n-P_C({\beta }_n{x}_n\left)\right)\\ x_{n+1}={\beta }_n{x}_n+{\lambda }_n(p_n-{\beta }_n{x}_n)\\ q_n={\beta }_n{v}_n+\sigma L(2p_n-{\beta }_n{x}_n)-\sigma P_Q({\sigma }^{-1}{\beta }_n{v}_n+L(2p_n-{\beta }_n{x}_n\left)\right)\\ v_{n+1}={\beta }_n{v}_n+{\lambda }_n(q_n-{\beta }_n{v}_n).\end{array}$$ (55)

For the numerical example we consider the following setup: let $\mathcal{H}=\mathcal{G}=L^2\left(\right[0,2\pi \left]\right):=\{f:[0,2\pi ]\to \mathbb{R}:{\int }_0^{2\pi }|f(t){|}^2\mathrm{d}t<+\mathrm{\infty }\}$ equipped with the scalar product $⟨f,g⟩:={\int }_0^{2\pi }f\left(t\right)g\left(t\right)\mathrm{d}t$ and the associated norm $\parallel f\parallel :=\left({\int }_0^{2\pi }|f\right(t){|}^2\mathrm{d}t{)}^{1/2}$ for all $f,g\in L^2\left(\right[0,2\pi \left]\right)$. The sets

$$C:=\left\{x\in L^2\left(\right[0,2\pi \left]\right):{\int }_0^{2\pi }x\left(t\right)\mathrm{d}t\le 1\right\}$$

and

$$Q:=\left\{x\in L^2\left(\right[0,2\pi \left]\right):{\int }_0^{2\pi }|x\left(t\right)-\sin \left(t\right){|}^2\mathrm{d}t\le 16\right\}$$

are nonempty, closed and convex subsets of $L^2\left(\right[0,2\pi \left]\right)$. Notice that $C=\{x\in L^2([0,2\pi ]):⟨x,u⟩\le 1\}$ and $Q=\{x\in L^2([0,2\pi ]):\parallel x-f\parallel \le 4\}$, where $u:[0,2\pi ]\to \mathbb{R},u\left(t\right)=1$ for all $t\in [0,2\pi ]$, and $f:[0,2\pi ]\to \mathbb{R},f\left(t\right)=\sin t$ for all $t\in [0,2\pi ]$. We define the linear continuous operator $L:L^2\left(\right[0,2\pi \left]\right)\to L^2\left(\right[0,2\pi \left]\right)$ as

$$\left(Lx\right)\left(t\right):=\left({\int }_0^{2\pi }x\left(s\right)\mathrm{d}s\right)\cdot t.$$

Let $x,y\in L^2\left(\right[0,2\pi \left]\right)$. Then

$$⟨Lx,y⟩={\int }_0^{2\pi }Lx\left(t\right)y\left(t\right)\mathrm{d}t={\int }_0^{2\pi }x\left(s\right){\int }_0^{2\pi }ty\left(t\right)\mathrm{d}t\mathrm{d}s,$$

and therefore

$$\left(L^{\ast }y\right)\left(t\right)={\int }_0^{2\pi }sy\left(s\right)\mathrm{d}s.$$

Furthermore, by using the Cauchy-Schwarz inequality yields

$$\begin{array}{rl}\parallel L{\parallel }^2& =\underset{\parallel x\parallel =1}{sup}{\int }_0^{2\pi }Tx(t{)}^2\mathrm{d}t=\underset{\parallel x\parallel =1}{sup}{\int }_0^{2\pi }{t}^2\mathrm{d}t{\left({\int }_0^{2\pi }x\left(s\right)\mathrm{d}s\right)}^2\\ & \le \underset{\parallel x\parallel =1}{sup}\frac{8}{3}{\pi }^3{\int }_0^{2\pi }x(s{)}^2\mathrm{d}s\cdot 2\pi =\frac{16}{3}{\pi }^4.\end{array}$$

The projection onto the set C can be computed as (see [6, Example 28.16])

$$P_C\left(x\right)=\left\{\begin{array}{ll}\frac{1-{\int }_0^{2\pi }x\left(t\right)\mathrm{d}t}{2\pi }+x,& \mathrm{if}{\int }_0^{2\pi }x\left(t\right)\mathrm{d}t>1\\ x,& \mathrm{else}.\end{array}\right.$$

On the other hand $P_Q$ is given by [6, Example 28.10]

$$P_Q\left(x\right)=\left\{\begin{array}{ll}\sin +\frac{4(x-\sin )}{\left({\int }_0^{2\pi }|x\right(t)-\sin (t){|}^2\mathrm{d}t{)}^{1/2}},& \mathrm{if}{\int }_0^{2\pi }|x\left(t\right)-\sin \left(t\right){|}^2\mathrm{d}t>16\\ x,& \mathrm{else}.\end{array}\right.$$

We implemented the algorithms (54) and (55) in MATLAB used symbolic computation for generating the sequences of iterates. One can easily notice that, in this particular setting, the split feasibility problem and, consequently, both addressed optimization problems are solvable. The numerical experiments confirmed that the primal–dual algorithms involving Tikhonov regularization terms outperform the ones without Tikhonov regularization terms, which correspond to the case when ${\beta }_n=1$ for every $n\ge 0$ and for which is known that they weakly convergence to a primal–dual solution of the corresponding KKT system of optimality conditions (Tables 1 and 2).

Table 1. Comparison of the variants without and with Tikhonov regularization terms of the primal–dual algorithm (54), for different starting values and step sizes $\tau =0.1$ and $\sigma =0.01$.

		Number of iterations
$x_0$	$v_0$	${\beta }_n=1$	${\beta }_n=1-\frac{1}{n+1}$
$\frac{t^2}{10}$	$\frac{t^2}{10}$	13	1
$\frac{t^2}{10}$	$\frac{1}{2}{\mathrm{e}}^t$	20	11
$\frac{t^2}{10}$	${\mathrm{e}}^t+\frac{t^2}{24}$	21	12
$\frac{1}{2}{\mathrm{e}}^t$	$\frac{t^2}{10}$	> 150	11
$\frac{1}{2}{\mathrm{e}}^t$	$\frac{1}{2}{\mathrm{e}}^t$	20	12
$\frac{1}{2}{\mathrm{e}}^t$	${\mathrm{e}}^t+\frac{t^2}{24}$	21	13
${\mathrm{e}}^t+\frac{t^2}{24}$	$\frac{t^2}{10}$	> 150	15
${\mathrm{e}}^t+\frac{t^2}{24}$	$\frac{1}{2}{\mathrm{e}}^t$	20	13
${\mathrm{e}}^t+\frac{t^2}{24}$	${\mathrm{e}}^t+\frac{t^2}{24}$	21	13

Table 2. Comparison of the variants without and with Tikhonov regularization terms of the primal–dual algorithm (55), for different starting values and step sizes $\tau =0.1$ and $\sigma =0.01$.

		Number of iterations
$x_0$	$v_0$	${\beta }_n=1$	${\beta }_n=1-\frac{1}{n+1}$
$\frac{t^2}{10}$	$\frac{t^2}{10}$	24	1
$\frac{t^2}{10}$	$\frac{1}{2}{\mathrm{e}}^t$	46	10
$\frac{t^2}{10}$	${\mathrm{e}}^t+\frac{t^2}{24}$	46	10
$\frac{1}{2}{\mathrm{e}}^t$	$\frac{t^2}{10}$	30	6
$\frac{1}{2}{\mathrm{e}}^t$	$\frac{1}{2}{\mathrm{e}}^t$	24	11
$\frac{1}{2}{\mathrm{e}}^t$	${\mathrm{e}}^t+\frac{t^2}{24}$	35	21
${\mathrm{e}}^t+\frac{t^2}{24}$	$\frac{t^2}{10}$	32	6
${\mathrm{e}}^t+\frac{t^2}{24}$	$\frac{1}{2}{\mathrm{e}}^t$	36	12
${\mathrm{e}}^t+\frac{t^2}{24}$	${\mathrm{e}}^t+\frac{t^2}{24}$	24	11

In the two tables above we present the numbers of iterations needed by the two algorithms to approach a solution of the split feasibility problem $\left(SFP\right)$. We consider as stopping criterion

$$E\left(x_n\right):=\frac{1}{2}\parallel P_C\left(x_n\right)-x_n{\parallel }^2+\frac{1}{2}\parallel P_Q\left(L{x}_n\right)-L{x}_n{\parallel }^2\le {10}^{-3},$$

while taking as relaxation variables ${\lambda }_n=0.4$ for every $n\ge 0$ and as Tikhonov regularization parameters ${\beta }_n:=1-1/(n+1)$ for every $n\ge 0$, respectively, ${\beta }_n=1$ for every $n\ge 0$ for the variants without Tikhonov regularization terms.

7. Further work

We point out some directions of research related to proximal methods with Tikhonov regularization terms, which merit to be addressed starting from the investigations made in this paper:

1. To consider in the numerical algorithms of type forward–backward and Douglas–Rachford proposed in this paper dynamic step sizes, which are known to increase the flexibility of the algorithms. This can be for instance done by formulating first a Krasnosel'skiĭ–Mann–type algorithm for determining an element in the intersection of the sets of fixed points of a family of nonexpansive operators $(T_k{)}_{k\ge 0}$. Suitable choices of the operators in this family, in the spirit of the investigations we made in the Sections 3 and 4, can lead to iterative algorithms with dynamic step sizes (see [14]).

2. To employ in the iterative schemes discussed in this article inertial and memory effects, which are known to contribute to the acceleration of the convergence behaviour of the algorithms.

3. To translate the proposed numerical methods to a framework that goes beyond the Hilbert space setting considered in this article, in order to allow applications where functional spaces are involved. We refer the reader to [24,25,27] and the references therein for tools and techniques which allow to prove convergence statements in Banach spaces with some particular underlying geometric structures.

Funding Statement

Research of Radu Ioan Boţ was partially supported by FWF (Austrian Science Fund), project I 2419-N32. Research of Ernö Robert Csetnek was supported by FWF (Austrian Science Fund), projects M 1682-N25 and P 29809-N32. Research of Dennis Meier was partially supported by FWF (Austrian Science Fund), project I 2419-N32, and by the Doctoral Programme Vienna Graduate School on Computational Optimization (VGSCO), project W1260-N35.

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Radu Ioan Boţ  http://orcid.org/0000-0002-4469-314X