On the String Averaging Method for Sparse Common Fixed Points Problems

Abstract

We study the common fixed point problem for the class of directed operators. This class is important because many commonly used nonlinear operators in convex optimization belong to it. We propose a definition of sparseness of a family of operators and investigate a string-averaging algorithmic scheme that favorably handles the common fixed points problem when the family of operators is sparse. The convex feasibility problem is treated as a special case and a new subgradient projections algorithmic scheme is obtained.

1 Introduction

Given a finite family of operators ${\left\{T_i\right\}}_{i=1}^m$ acting on the Euclidean space Rⁿ with Fix T_i ≠ ∅, i = 1, 2, …, m, the common fixed point problem is to find a point

$$x^{\ast }\in {\cap }_{i=1}^m\mathrm{Fix}{T}_i,$$ (1)

where Fix T_i is the fixed points set of T_i. In this paper we study the common fixed point problem for sparse directed operators. We use the term directed operators for operators in the ℑ-class of operators as defined and investigated by Bauschke and Combettes in [3] and by Combettes in [18]. Additionally, we focus on sparse operators and, for that purpose, we give a definition of sparseness of a family of operators.

The significance of working with this class stems from the fact that many commonly used types of nonlinear operators arising in convex optimization are directed operators (see, e.g., [3]) and, when developing algorithms for the problem (1) for such operators, we take advantage of their sparsity, whenever it exists.

The algorithms that are in use to find a common fixed point can be, from their structural view point, sequential, when only one operator at a time is used in each iteration, or simultaneous (parallel), when all operators in the given family are used in each iteration. There are algorithmic schemes which encompass sequential and simultaneous properties. These are the, so called, string-averaging [9] and block-iterative projections (BIP) [1], schemes, see also [15]. It turns out that the sequential and the simultaneous algorithms are special cases of the string-averaging and of the BIP algorithmic schemes.

Our objective here is to propose and study a string-averaging algorithmic scheme that enables component-wise weighting. Our work is a theoretical development aimed at gauging how far can the notions of sparsity, component-weighting and algorithmic string-averaging be expanded to cover the common fixed point problem for directed operators. The origins lie in [11] where a simultaneous projection algorithm, called component averaging (CAV), for systems of linear equations, that uses component-wise weighting was proposed. Such weighting enables, as shown and demonstrated experimentally on problems of image reconstruction from projections in [11], significant and valuable acceleration of the early algorithmic iterations due to the high sparsity of the system matrix appearing there. A block-iterative version of CAV, named BICAV, was introduced later in [12]. Full mathematical analyses of these methods, as well as their companion algorithms for linear inequalities, were presented by Censor and Elfving [10] and by Jiang and Wang [25]. In Section 2 we present preliminary material on directed operators and discuss some of their particular cases. In Section 3 we develop and study our string-averaging algorithmic scheme. In Section 4 we consider, as a special case, the convex feasibility problem and apply our algorithm from Section 3 using subgradient projectors.

1.1 Earlier work

The string-averaging algorithmic scheme has attracted attention recently and further work on it has been reported since its presentation in [9]. In [14] we investigated the behavior of string-averaging algorithms for inconsistent convex feasibility problems. In Bauschke, Matoušková and Reich [4] string-averaging was studied in Hilbert space. In Crombez [19, 20] the string-averaging algorithmic paradigm is used to find common fixed points of certain paracon-tractive operators in Hilbert space. In Bilbao-Castro, Carazo, García and Fernández [6], an implementation of the string-averaging method to electron microscopy is reported. Butnariu, Davidi, Herman and Kazantsev [7] call a certain class of string-averaging methods the Amalgamated Projection Method and show its stable behavior under summable perturbations. The iterative procedure studied in Butnariu, Reich and Zaslavski [8, Sections 6 and 7] is also a particular case of the string-averaging method. In Rhee [27] the string-averaging scheme is applied to a problem in approximation theory.

The notion of sparseness is very well understood and used for matrices and, from there, the road to sparseness of the Jacobian (or generalized Jacobian) matrix as an indicator of sparseness of nonlinear operators is short, see, e.g., Betts and Frank [5]. Our definition of sparseness of operators does not require differentiability (or subdifferentiability) and generalizes those previous notions.

2 Directed operators

We recall the definitions and results on directed operators and their properties as they appear in Bauschke and Combettes [3, Proposition 2.4] and Combettes [18], which are also sources for further references on the subject. Let 〈x,y〉 and ∥x∥ be the Euclidean inner product and norm, respectively, in Rⁿ.

Given x,y ∈ Rⁿ we denote the half-space

$$H(x,y)≔\{u\in R^n\mid \langle u-y,x-y\rangle \le 0\}.$$ (2)

Definition 1 An operator T : Rⁿ → Rⁿ is called directed if

$$\mathrm{Fix}T\subseteq H(x,T\left(x\right)),\mathit{for}\mathit{all}x\in R^n,$$ (3)

or, equivalently,

$$\mathit{if}z\in \mathrm{Fix}T\mathit{then}\langle T\left(x\right)-x,T\left(x\right)-z\rangle \le 0,\mathit{for}\mathit{all}x\in R^n.$$ (4)

The class of directed operators is denoted by ℑ. Bauschke and Combettes [3] defined the directed operators (although without using this name) and showed (see [3, Proposition 2.4]) (i) that the set of all fixed points of a directed operator T with nonempty FixT is closed and convex because

$$\mathrm{Fix}T=\bigcap _{x\in R^n}H(x,T\left(x\right)),$$ (5)

and (ii) that the following holds

$$\text{If}T\in \Im \text{then}I+\lambda (T-I)\in \Im ,\text{for all}\lambda \in [0,1],$$ (6)

where I is the identity operator. The localization of fixed points is discussed in [23, pages 43-44]. In particular, it is shown there that a firmly nonexpansive operator, namely, an operator N : Rⁿ → Rⁿ that fulfills

$${\parallel N\left(x\right)-N\left(y\right)\parallel }^2\le \langle N\left(x\right)-N\left(y\right),x-y\rangle ,\text{for all}x,y\in R^n,$$ (7)

satisfies (5) and is, therefore, a directed operator. The class of directed operators, includes additionally, according to [3, Proposition 2.3], among others, the resolvents of a maximal monotone operators, the orthogonal projections and the subgradient projectors (see Example 7 below). Note that every directed operator belongs to the class of operators ℱ⁰, defined by Crombez [21, p. 161], whose elements are called elsewhere quasi-nonexpansive or paracontracting operators.

The following definition of a closed operator will be required.

Definition 2 An operator T : Rⁿ → Rⁿ is said to be closed at y ∈ Rⁿ if for every x̄ ∈ Rⁿ and every sequence ${\left\{x^k\right\}}_{k=0}^{\infty }$ in Rⁿ, such that, lim_k→∞x^k = x̄ and lim_k→∞T(x^k) = y, we have T(x̄) = y.

For instance, the orthogonal projection onto a closed convex set is everywhere a closed operator, due to its continuity.

Remark 3 [18] If T : Rⁿ → Rⁿ is nonexpansive, then T − I is closed on Rⁿ.

Consider a finite family T_i : Rⁿ → Rⁿ, i = 1, 2, …, m, of operators. In sequential algorithms for solving the common fixed point problem the order by which the operators are chosen for the iterations is determined by a control sequence of indices ${\left\{i\left(k\right)\right\}}_{k=0}^{\infty }$, see, e.g., [15, Definition 5.1.1].

Definition 4 (i) Cyclic control. A control sequence is cyclic if i(k) = k mod m + 1, where m is the number of operators in the common fixed point problem.

(ii) Almost cyclic control. ${\left\{i\left(k\right)\right\}}_{k=0}^{\infty }$ is almost cyclic on {1, 2, …, m}, if 1 ≤ i(k) ≤ m for all k ≥ 0, and there exists an integer c ≥ m (called the almost cyclicality constant), such that, for all k ≥ 0, {1, 2, …, m} ⊆ {i(k + 1), i(k + 2), …, i(k + c)}.

The notions “cyclic” and “almost cyclic” are sometimes also called “periodic” and “quasi-periodic”, respectively, see, e.g., [22].

Given a finite family T_i : Rⁿ → Rⁿ, i = 1, 2, …, m, of directed operators with a nonempty intersection of their fixed points sets, such that T_i − I are closed at 0, for every i ∈ {1, 2, …, m}. The following algorithm for finding a common fixed point of such a family is a special case of [18, Algorithm 6.1]. We will use it in the sequel.

Algorithm 5 Almost Cyclic Sequential Algorithm (ACSA) for solving common fixed point problem

Initialization: x⁰ ∈ Rⁿ is an arbitrary starting point.

Iterative Step: Given x^k, compute x^k+1 by

$$x^{k+1}=x^k+{\lambda }_k(T_{i\left(k\right)}\left(x^k\right)-x^k).$$ (8)

Control: ${\left\{i\left(k\right)\right\}}_{k=0}^{\infty }$ is almost cyclic on {1, 2, …, m}.

Relaxation parameters: ${\left\{{\lambda }_k\right\}}_{k=0}^{\infty }$ are confined to the interval [0,2].

The convergence theorem for Algorithm 5 for a finite family of directed operators is as follows.

Theorem 6 Let ${\left\{T_i\right\}}_{i=1}^m$ be a finite family of directed operators T_i : Rⁿ → Rⁿ, which satisfies

• (i) $\Omega ≔{\cap }_{i=1}^m$ Fix T_i is nonempty, and

• (ii) T_i − I are closed at 0, for every i ∈ {1, 2, …, m}.

Then any sequence ${\left\{x^k\right\}}_{k=0}^{\infty }$, generated by Algorithm 5, converges to a point in Ω.

Proof. This follows as a special case of [18, Theorem 6.6 (i)]. ■

In the next definition and lemma we recall the notion of the subgradient projector and show that this operator satisfies condition (ii) of Theorem 6.

Definition 7 See, e.g., [3, Proposition 2.3(iv)]. Let f : Rⁿ → R be a convex function such that the level-set F := {x ∈ Rⁿ | f(x) ≤ 0} is nonempty. The operator

$${\Pi }_F\left(y\right)≔\{\begin{array}{cc}y-\frac{f\left(y\right)}{{\parallel q\parallel }^2}q,\hfill & \mathit{if}f\left(y\right)>0,\hfill \\ y,\hfill & \mathit{if}f\left(y\right)\le 0,\hfill \end{array}\phantom{\}}$$ (9)

where q is a selection from the subdifferential set ∂f(y) of f at y, is called a subgradient projector relative to f.

Lemma 8 Let f : Rⁿ → R be a convex function, let y ∈ Rⁿ and assume that the level-set F ≠ ∅. For any q ∈ ∂f(y), define the closed convex set

$$L=L_f(y,q)≔\{x\in R^n\mid f\left(y\right)+\langle q,x-y\rangle \le 0\}.$$ (10)

Then the following hold:

• (i) F ⊆ L. If q ≠ 0 then L is a half-space, otherwise L = Rⁿ.

• (ii) Denoting by P_L(y) the orthogonal projection of y onto L,

  $$P_L\left(y\right)={\Pi }_F\left(y\right).$$ (11)

• (iii) P_L − I is closed at 0.

Proof. For (i) and (ii) see, e.g., [2, Lemma 7.3]. (iii) Denote Ψ = P_L − I. Take any x̄ ∈ Rⁿ and any sequence ${\left\{x^k\right\}}_{k=0}^{\infty }$ in Rⁿ, such that, lim_k→∞x^k = x̄ and lim_k→∞ :(x^k) = 0. Since f is convex, its subdifferential is uniformly bounded on bounded sets, see, e.g., [2, Corollary 7.9]. Using this and the continuity of f we obtain, from (9), that f(x̄) = 0, and, therefore, Ψ(x̄) = 0. ■

3 The new string averaging algorithmic scheme

We study here a particular modification of the string averaging paradigm, adapted to handle the common fixed point problem for sparse directed operators.

3.1 The string averaging prototypical scheme

The string averaging prototypical scheme is defined as follows. Let the string S_p, for p = 1, 2, …, t, be a finite, nonempty ordered subset of elements taken from {1, 2, …, m} of the form

$$S_p≔\left\{i_1^p,i_2^p,\dots ,i_{\gamma \left(p\right)}^p\right\}.$$ (12)

The length γ(p) of the string S_p is the number of its elements. We do not require that the strings ${\left\{S_p\right\}}_{p=1}^t$ should be disjoint. Suppose that there is a set Q ⊆ Rⁿ such that there are operators V1, V2, …, V_m mapping Q into Q and an operator V which maps Q^t = Q × Q × … × Q into Q. Then the string averaging prototypical scheme is as follow.

Algorithm 9 The string averaging prototypical algorithmic scheme [9]

Initialization: x⁰ ∈ Q is an arbitrary starting point.

Iterative Step: Given the current iterate x^k,

• (i) calculate, for all p = 1, 2, … , t,

  $$M_p\left(x^k\right)≔V_{i_{\gamma \left(p\right)}^p}\dots V_{i_2^p}{V}_{i_1^p}\left(x^k\right),$$ (13)
• (ii) and then calculate,

  $$x^{k+1}=V(M_1\left(x^k\right),M_2\left(x^k\right),\dots ,M_t\left(x^k\right)).$$ (14)

For every p = 1, 2, … t, this algorithmic scheme applies to x^k successively the operators whose indices belong to the p-th string. This can be done in parallel for all strings and then the operator V maps all end-points onto the next iterate x^k+1. This is indeed an algorithm provided that the operators ${\left\{V_i\right\}}_{i=1}^m$ and V all have algorithmic implementations. In this framework we get a sequential algorithm by the choice t = 1 and S1 = {1, 2, …, m} and a simultaneous algorithm by the choice t = m and S_p = {p} , p = 1, 2, …, t.

In our new algorithmic scheme we assume that a finite family of directed operators (see Definition 1) ${\left\{T_i\right\}}_{i=1}^m$ is given with ${\cap }_{i=1}^m\mathrm{Fix}{T}_i\ne \varnothing$. After applying the operators ${\left\{T_i\right\}}_{i=1}^m$ along strings, the end-points will be averaged not by taking a plain convex combination but by doing a, so called, component-averaging step. The component averaging principle, introduced for linear systems in [11], [12], is a useful tool for handling sparseness in the linear case.

3.2 Sparseness of operators and the new algorithm

To define sparseness of the set of operators ${\left\{T_i\right\}}_{i=1}^m$ we need to speak about zeros of the vectors x − T_i(x).

Definition 10 Let T : Rⁿ → Rⁿ be a directed operator. If (x − T(x))_j = 0, for all x ∉ FixT then j is called a void of T and we write j = voidT.

For every i ∈ {1, 2, …, m} define the following sets

$$Z_i≔\{(i,j)\mid 1\le j\le n,j=\text{void}{T}_i\},$$ (15)

i.e., Z_i contains all the pairs (i, j), such that (x−T_i(x))_j = 0, for all x ∉ FixT_i.

Definition 11 The family of directed operators ${\left\{T_i\right\}}_{i=1}^m$ will be called sparse if the set $Z≔{\cap }_{i=1}^m{Z}_i$ is nonempty and contains many elements.

Remark 12 The word “many” in Definition 11 is not well-defined. The more pairs (i,j) are contained in Z the higher is the sparseness of the family. It is of some interest to note that sparseness of matrices was considered as early as in 1971. Wilkinson [28, p. 191] refers to it by saying: “We shall refer to a matrix as dense if the percentage of zero elements or its distribution is such as to make it uneconomic to take advantage of their presence”. Obviously, denseness is meant here as an opposite of sparseness.

Denote by I_j, 1 ≤ j ≤ n, the set of indices of strings that contain an index of an operator T_i for which (i, j) ∉ Z_i, i.e.,

$$I_j≔\{p\mid 1\le p\le t,(i,j)\notin Z_i\text{for some}i\in S_p\}$$ (16)

and let s_j = |I_j| (the cardinality of I_j). Equivalently,

$$I_j=\{p\mid 1\le p\le t,j\ne \text{void}{T}_i\text{for some}i\in S_p\}.$$ (17)

Definition 13 [24, Definition 1] The component-wise string averaging operator relative to the family of strings S := {S₁, S₂, …, S_t} is a mapping CA_S : R^n×t → Rⁿ, defined as follows. For x¹, x²,…, x^t ∈ Rⁿ,

$${\left({\mathit{CA}}_S(x^1,x^2,\dots ,x^t)\right)}_j≔(1∕s_j)\underset{p\in I_j}{\Sigma }{x}_j^p,\mathit{for}\mathit{all}1\le j\le n,$$ (18)

where $x_j^p$ is the j-th component of x^p, for 1 ≤ p ≤ t.

Our new scheme performs sequential steps within each of the strings of the family S and merges the resulting end-points by the component-wise string averaging operator (18) as follows.

Algorithm 14

Initialization: x⁰ ∈ Rⁿ is an arbitrary starting point and define an integer constant N, such that N ≥ m.

Iterative step: Given x^k, compute x^k+1 as follows:

• (i) For every 1 ≤ p ≤ t (possibly in parallel): Execute a finite number, not exceeding N, of iterative steps of the form (8), on the operators {T_i}_i∈Sp of the p-th string and denote the resulting end-points by ${\left\{{\stackrel{-}{x}}^p\right\}}_{p=1}^t$.

• (ii) Apply

  $$x^{k+1}={\mathit{CA}}_S({\stackrel{-}{x}}^1,{\stackrel{-}{x}}^2,\dots ,{\stackrel{-}{x}}^t).$$ (19)

3.3 Convergence

For the proof of convergence of Algorithm 14 we need the following construction. From the family ${\left\{T_i\right\}}_{i=1}^m$ of directed operators in Rⁿ we construct another family of directed operators in a higher-dimensional space R^s and a family of strings for those operators. For the new operators and new strings, the operators belonging to different strings do not share any common variables. Therefore, the parallel processing of the strings in Rⁿ in (i) of Algorithm 14 is equivalent to performing sequential ACSA iterations on the new directed operators in R^s. Moreover, using ideas of Pierra's [26] formalization, we show that the component-wise string averaging step in (ii) of Algorithm 14 is equivalent to an orthogonal projection onto a certain subspace of R^s. Inspired by the construction in [24], this is done as follows.

We represent each I_j is explicitly as

$$I_j=\{p_{j,1},p_{j,2,},\dots ,p_{j,s_j}\},$$ (20)

which defines each double-indexed p in an obvious way. Let R^s be the s-dimensional Euclidean space, where $s={\Sigma }_{j=1}^n{s}_j$, and denote the components of each y ∈R^s by

$$\mathbf{y}=(\underset{s_1\text{elements}}{\underbrace{y_{1,1},y_{1,2},\dots ,y_{1,s_1}}},\dots ,\underset{s_n\text{elements}}{\underbrace{y_{n,1},y_{n,2},\dots ,y_{n,s_n}}})={\left\{y_{j,\ell }\right\}}_{j=1,\ell =1}^{n,s_j}.$$ (21)

Define a linear mapping

$$\delta :R^n\to R^s,\text{by}\delta \left(x\right)=\delta (x_1,x_2,\dots ,x_n)≔\mathbf{y},$$ (22)

where y_j,pj,1 = y_j,pj,2 = … = y_j,pj,sj = x_j for j = 1, 2, …, n. Let D be the range of δ, i.e.,

$$\mathbf{D}≔\left\{\mathbf{d}\in R^s\mid \mathbf{d}=(\underset{s_1\text{times}}{\underbrace{d_1,d_1,\dots ,d_1}},\dots ,\underset{s_n\text{times}}{\underbrace{d_n,d_n,\dots ,d_n}})\right\},$$ (23)

which is a subspace of R^s. Define γ new operators where $\gamma ={\Sigma }_{p=1}^t\gamma \left(p\right)$ in the following manner: For each p, ${\left\{i_w^p\right\}}_{w=1}^{\gamma \left(p\right)}$ are the indices of the operators T_i that are included in the string S_p, see (12). To each pair $(i_w^p,p)$ we attach a new operator ${\mathbf{T}}_{i_w^p,p}:R^s\to R^s$, defined by

$${\mathbf{T}}_{i_w^p,p}\left(\mathbf{y}\right)≔U_p\left(T_{i_w^p}\left({\Pi }_p\left(\mathbf{y}\right)\right)\right),$$ (24)

where the operators in the right-hand side of (24) are defined as follows. Π_p : R^s → Rⁿ, 1 ≤ p ≤ t, is defined component-wise for each 1 ≤, j ≤ n as

$${\left({\Pi }_p\left(\mathbf{y}\right)\right)}_j≔\{\begin{array}{cc}{y}_{j,l},\hfill & \text{if}{p}_{j,l}=p,\text{for some}\ell \in \{1,2,\dots ,s_j\},\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}\phantom{\}}$$ (25)

$T_{i_w^p}$ is the w-th directed operator in the string S_p and U_p : Rⁿ → R^s, 1 ≤ p ≤ t, is defined component-wise for each 1 ≤ j ≤ n and 1 ≤ ℓ ≤ s_j as

$${\left(U_p\left(x\right)\right)}_{j,\ell }=\{\begin{array}{cc}{x}_j,\hfill & \text{if}{p}_{j,\ell }=p,\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}\phantom{\}}$$ (26)

The new operators ${\mathbf{T}}_{i_w^p,p}$ have fixed point sets Fix ${\mathbf{T}}_{i_w^p,p}\subset R^s$. Each string $S_p=\left\{i_1^p,i_2^p,\dots ,i_{\gamma \left(p\right)}^p\right\}$ in Rⁿ gives rise to a string

$${\mathbf{S}}_p=\left\{(i_1^p,p),(i_2^p,p),\dots ,(i_{\gamma \left(p\right)}^p,p)\right\}$$ (27)

of the same length in R^s. Note, that operators ${\mathbf{T}}_{i_w^p,p}$ that belong to different strings in the family of strings ${\left\{S_p\right\}}_{p=1}^t$ do not have a common variable which is not a void.

Lemma 15 Every operator ${\mathbf{T}}_{i_w^p,p}$, 1 ≤ p ≤ t, $1\le i_w^p\le \gamma \left(p\right)$ is a directed operator and ${\mathbf{T}}_{i_w^p,p}-\mathbf{I}$ is closed at 0, where I is the identity operator in R^s.

Proof. If $\mathbf{z}\in \mathrm{Fix}{\mathbf{T}}_{i_w^p,p}$ then z ∈ Im U_p, the image set of U_p. Moreover, then also ${\Pi }_p\left(\mathbf{z}\right)=z^{\ast }\in \mathrm{Fix}{T}_{i_w^p}$. For every x ∈ R^s we have

$$\begin{array}{cc}\hfill & \langle {\mathbf{T}}_{i_w^p,p}\left(\mathbf{x}\right)-\mathbf{x},{\mathbf{T}}_{i_w^p,p}\left(\mathbf{x}\right)-\mathbf{z}\rangle \hfill \\ \hfill & =\langle {\Pi }_p\left({\mathbf{T}}_{i_w^p,p}\left(\mathbf{x}\right)\right)-{\Pi }_p\left(\mathbf{x}\right),{\Pi }_p\left({\mathbf{T}}_{i_w^p,p}\left(\mathbf{x}\right)\right)-{\Pi }_p\left(\mathbf{z}\right)\rangle \hfill \\ \hfill & \langle T_{i_w^p}\left({\Pi }_p\left(\mathbf{x}\right)\right)-{\Pi }_p\left(\mathbf{x}\right),T_{i_w^p}\left({\Pi }_p\left(\mathbf{x}\right)\right)-z^{\ast }\rangle \le 0,\hfill \end{array}$$ (28)

since the operator $T_{i_w^p}$ is directed, therefore, (28) implies that ${\mathbf{T}}_{i_w^p,p}$ is also directed. Next, we show that ${\mathbf{T}}_{i_w^p,p}-\mathbf{I}$ is closed at 0. Let ${\left\{{\mathbf{x}}^k\right\}}_{k=1}^{\infty }$ be a sequence in R^s, such that lim_k→∞ x^k = x̄ and ${\lim }_{k\to \infty }\left({\mathbf{T}}_{i_w^p,p}\left({\mathbf{x}}^k\right)-{\mathbf{x}}^k\right)=0$. Since the operator Π_p is continuous, we obtain

$$\underset{k\to \infty }{\lim }{\Pi }_p\left({\mathbf{x}}^k\right)={\Pi }_p\left(\stackrel{-}{\mathbf{x}}\right),$$ (29)

and

$$\underset{k\to \infty }{\lim }\left(T_{i_w^p}\left({\Pi }_p\left({\mathbf{x}}^k\right)\right)-{\Pi }_p\left({\mathbf{x}}^k\right)\right)=0.$$ (30)

The operator $T_{i_w^p}-I$ is closed at zero and, therefore,

$$T_{i_w^p}\left({\Pi }_p\left(\stackrel{-}{\mathbf{x}}\right)\right)={\Pi }_p\left(\stackrel{-}{\mathbf{x}}\right).$$ (31)

Applying U_p to both sides of (31), we obtain

$${\mathbf{T}}_{i_w^p,p}\left(\stackrel{-}{\mathbf{x}}\right)=U_p\left({\Pi }_p\left(\stackrel{-}{\mathbf{x}}\right)\right).$$ (32)

From

$$\stackrel{-}{\mathbf{x}}=\underset{k\to \infty }{\lim }{\mathbf{T}}_{i_w^p,p}\left({\mathbf{x}}^k\right)=U_p\cdot \left(\underset{k\to \infty }{\lim }{T}_{i_w^p}\left({\Pi }_p\left({\mathbf{x}}^k\right)\right)\right),$$ (33)

follows that x̄ ∈ Im U_p and therefore U_p(Π_p(x̄) = x̄. Then, from (32) one has that ${\mathbf{T}}_{i_w^p,p}\left(\stackrel{-}{\mathbf{x}}\right)=\stackrel{-}{\mathbf{x}}$ from which the closedness of ${\mathbf{T}}_{i_w^p,p}-\mathbf{I}$ follows. ■

Define the set

$$\mathbf{C}={\cap }_{p=1}^t{,}_{w=1}^{\gamma \left(p\right)}\mathrm{Fix}{\mathbf{T}}_{i_w^p,p}.$$ (34)

The mapping δ : Rⁿ → D is a one-to-one mapping. Therefore, in the space R^s, we can reformulate the problem (1) as

$$\text{Find}\mathbf{y}\in \mathbf{C}\cap \mathbf{D}.$$ (35)

This means that

$$x\in {\cap }_{i=1}^m\mathrm{Fix}{T}_i\text{if and only if}\mathbf{y}=\delta \left(x\right)\in \mathbf{C}\cap \mathbf{D},$$ (36)

and, hence, the m–sets problem (1) is reduced to the 2-sets problem (35), which involves only a vector subspace and a convex set.

Next we present the alternative formulation of the Algorithm 14 in which the operations are performed in R^s.

Algorithm 16

Initialization:

• (i) x⁰ ∈ Rⁿ is arbitrary and define an integer constant N, such that N ≥ m.

• (ii) y⁰ = δ(x⁰) is the initial vector in R^s.

Iterative step: Given y^k, compute y^k+1 via:

• (i) In R^s, for every 1 ≤ p ≤ t (possibly in parallel): Execute a finite number, not exceeding N, of iterative steps of the form (8) on the operators ${\left\{{\mathbf{T}}_{i_w^p,p}\right\}}_{w=1}^{\gamma \left(p\right)}$ of the p-th string and denote the resulting end-points by ${\left\{{\stackrel{-}{\mathbf{y}}}^p\right\}}_{p=1}^t$.

• (ii) Apply CA_S in R^s as follows. For 1 ≤ j ≤ n, set

  $$y_{j,1}^{k+1}=\dots =y_{j,s_j}^{k+1}=\frac{1}{s_j}\left(\underset{\ell =1}{\overset{s_j}{\Sigma }}{\stackrel{-}{y}}_{j,\ell }^{p_{j,\ell }}\right).$$ (37)

• (iii) Denote ${\mathbf{y}}^{k+1}≔(y_{1,1}^{k+1},y_{1,2}^{k+1},\dots ,y_{1,s_1}^{k+1},\dots ,y_{n,1}^{k+1},y_{n,2}^{k+1},\dots ,y_{n,s_n}^{k+1})$.

The following lemma shows that the averaging operation in the iterative step (ii) of Algorithm 16 is the orthogonal projection onto the subspace D.

Lemma 17 Let y =(y_1,1, y_1,2, …, y_1,s1, …, y_n,1, y_n,2, …, y_n,sn) ∈ R^s, then

$$P_{\mathbf{D}}\left(\mathbf{y}\right)=\delta (\frac{1}{s_1}\underset{{\ell }_1=1}{\overset{s_1}{\Sigma }}{y}_{1,{\ell }_1},\frac{1}{s_2}\underset{{\ell }_2=1}{\overset{s_2}{\Sigma }}{y}_{2,{\ell }_2},\dots ,\frac{1}{s_n}\underset{{\ell }_n=1}{\overset{s_n}{\Sigma }}{y}_{n,{\ell }_n}).$$ (38)

Proof. Using the definition of the orthogonal projection we obtain

$$\begin{array}{cc}\hfill {\parallel P_{\mathbf{D}}\left(\mathbf{y}\right)-\mathbf{y}\parallel }^2& =\min \{{\parallel \delta \left(d\right)-\mathbf{y}\parallel }^2\mid d\in R^n\}\hfill \\ \hfill & =\min \left\{\underset{j=1}{\overset{n}{\Sigma }}\underset{\ell =1}{\overset{s_j}{\Sigma }}{|d_j-y_{j,\ell }|}^2\mid d\in R^n\right\}.\hfill \end{array}$$ (39)

The minimum is obtained when the gradient is equal to zero,

$$\frac{\partial }{\partial d_j}\left(\underset{j=1}{\overset{n}{\Sigma }}\underset{\ell =1}{\overset{s_j}{\Sigma }}{|d_j-y_{j,\ell }|}^2\right)=0,\text{for all}j=1,2,\dots n.$$ (40)

Then,

$$\underset{\ell =1}{\overset{s_j}{\Sigma }}(d_j-y_{j,\ell })=0,\text{for all}j=1,2,\dots n,$$ (41)

and

$$d_j=\frac{1}{s_j}\underset{\ell =1}{\overset{s_j}{\Sigma }}{y}_{j,\ell },\text{for all}j=1,2,\dots n,$$ (42)

and the proof is complete. ■

Now we are ready to prove our main convergence result.

Theorem 18 ${\cap }_{i=1}^m\mathrm{Fix}{T}_i\ne \varnothing$ then any sequence ${\left\{x^k\right\}}_{k=0}^{\infty }$, generated by the Algorithm 14, converges to a solution of (1).

Proof. The consistency assumption on the problem (1) implies that (35) is also consistent. Moreover, Lemma 28 guarantees that all the operators are directed and that ${\mathbf{T}}_{i_w^p,p}-\mathbf{I}$, 1 ≤ w ≤ γ(p), 1 ≤ p ≤ t and P_D − I are closed at 0. The Algorithm 16 can be executed in R^s in parallel or sequentially, since the strings do not contain any common non-void variables. Therefore, from Theorem 6 follows convergence to a common fixed point of the operators ${\mathbf{T}}_{i_w^p,p}$, 1 ≤ w ≤ γ(p), 1 ≤ p ≤ t, and P_D and the proof is complete. ■

4 Special case: The convex feasibility problem

The convex feasibility problem (CFP) is to find a point x* in the intersection C of m closed convex subsets C₁, C₂, …, C_m ⊆ Rⁿ. Each C_i is expressed as

$$C_i=\{x\in R^n\mid f_i\left(x\right)\le 0\},$$ (43)

where f_i : Rⁿ → R is a convex function, so the CFP requires a solution of the system of convex inequalities

$$f_i\left(x\right)\le 0,i=1,2,\dots ,m.$$ (44)

The convex feasibility problem is a special case of the common fixed point problem, where the directed operators are the subgradient projectors relative to f_i (see, Example 7 and Lemma 8 above).

In a recent paper by Gordon and Gordon [24] a new parallel “Component-Averaged Row Projections (CARP)” method for the solution of large sparse linear systems was introduced. It proceeds by dividing the equations into nonempty, not necessarily disjoint, sets (strings), performing Kaczmarz row projections within the strings, and merging the results by component-averaging operations to form the next iterate. As shown in [24], using orthogonal projections onto convex sets, this method and its convergence proof also apply to the consistent nonlinear CFP.

In contrast, when applied to a CFP, our Algorithm 14 gives rise to a method which is structurally similar to CARP but uses subgradient projections instead of orthogonal projections. This is, of course, a development that might be very useful for CFPs with nonlinear convex sets for which each orthogonal projection mandates an inner-loop of distance optimization. We use now our results from Section 3 to present a string-averaging algorithm with component-wise averaging for a sparse CFP.

Sparseness of the nonlinear system (44) can be defined in compliance with Definitions 10 and 11 by speaking about zeros of the subgradients of the functions f_i and to do so we use the next definition.

Definition 19 Let f_i : Rⁿ → R, i = 1, 2, …, m, be convex functions. For any x ∈ Rⁿ, the m × n matrix $Q\left(x\right)={\left(q_{\mathit{ij}}\right)}_{i=1}^m{,}_{j=1}^n$ is called a generalized Jacobian of the family of functions ${\left\{f_i\right\}}_{i=1}^m$ at the point x if $q_{\mathit{ij}}\equiv q_j^i$, for all i and all j, for some $q^i={\left(q_j^i\right)}_{j=1}^n$ such that qⁱ ∈ ∂f_i(x).

This definition coincides in our case with the Clarke's generalized Jacobian, see [16] and [17]. A generalized Jacobian Q(x) of the functions in (44) is not unique because of the possibility to fill it up with different subgradients from each subdifferential set. In case all f_i are differentiable the generalized Jacobian reduces to the usual Jacobian.

We define for every i ∈ {1, 2, …, m} the following sets

$$Z_i≔\{(i,j)\mid 1\le j\le n,f_i\left(x\right)\text{is independent of}{x}_j\text{for all}x\in R^n\}.$$ (45)

A mapping F : Rⁿ → R^m given by $F\left(x\right)={\left\{f_i\left(x\right)\right\}}_{i=1}^m$ will be called sparse if some of its component functions f_i do not depend on some of their variables x_j which means that $Z={\cap }_{i=1}^m{Z}_i\ne \varnothing$. The more pairs (i,j) are contained in Z the higher is the sparseness of the mapping F.

Next we recall the cyclic subgradient projections (CSP) method for the CFP (studied in [13]) which is a special version of the ACSA algorithm (Algorithm 5).

Algorithm 20 Cyclic Subgradient Projections (CSP)

Initialization: x⁰ ∈ Rⁿ is arbitrary.

Iterative step:

$$x^{k+1}≔\{\begin{array}{cc}{x}^k-{\lambda }_k\frac{f_{i\left(k\right)}\left(x^k\right)}{{\parallel q^k\parallel }^2}{q}^k,\hfill & \mathit{if}{f}_{i\left(k\right)}\left(x^k\right)>0,\hfill \\ x^k,\hfill & \mathit{if}{f}_{i\left(k\right)}\left(x^k\right)\le 0,\hfill \end{array}\phantom{\}}$$ (46)

where q^k ∈ ∂f_i(k) (x^k) is a subgradient of f_i(k) at the point x^k.

Relaxation parameters: ${\left\{{\lambda }_k\right\}}_{k=0}^{\infty }$ are confined to the interval [ε, 2 − ε], where ε > 0.

Control: Almost cyclic on {1, 2, …, m}.

According to our scheme the algorithm for solving the CFP performs CSP steps within the strings and merges the results by the C A_S(x₁, x₂, …, x_t) component-averaging operation.

Algorithm 21

Initialization: x⁰ ∈ Rⁿ is arbitrary and define an integer constant N, such that N ≥ m.

Iterative step: Given x^k, compute x^k+1 via:

• (i) For every 1 ≤ p ≤ t (possibly in parallel): Execute a finite number, not exceeding N, of CSP steps on the inequalities of the p-th string S_p and denote the resulting point by ${\left\{{\stackrel{-}{\mathbf{x}}}^p\right\}}_{p=1}^t$.

• (ii) Apply

  $$x^{k+1}={\mathit{CA}}_S({\stackrel{-}{x}}^1,{\stackrel{-}{x}}^2,\dots ,{\stackrel{-}{x}}^n).$$ (47)