Perturbation-resilient block-iterative projection methods with application to image reconstruction from projections

Abstract

A block-iterative projection algorithm for solving the consistent convex feasibility problem in a finite-dimensional Euclidean space that is resilient to bounded and summable perturbations (in the sense that convergence to a feasible point is retained even if such perturbations are introduced in each iterative step of the algorithm) is proposed. This resilience can be used to steer the iterative process towards a feasible point that is superior in the sense of some functional on the points in the Euclidean space having a small value. The potential usefulness of this is illustrated in image reconstruction from projections, using both total variation and negative entropy as the functional.

1. Introduction

We discuss block-iterative projection (BIP) methods for the convex feasibility problem that can be stated as: find a point in the nonempty intersection of a finite family of closed convex subsets of a Euclidean space. This is a fundamental problem in many areas of mathematics and the physical sciences; see Combettes (1993, 1996) and references therein. It has been used to model significant real-world problems in many fields, including image reconstruction from projections (Herman, 1980), radiation therapy treatment planning (Censor et al., 1988), and crystallography (Marks et al., 1999), to name but a few.

Projection methods are iterative algorithms that use projections onto sets while relying on the general principle that when a family of (usually closed and convex) sets is present then projections onto the individual sets are easier to perform than projections onto other sets (e.g., the intersection of the sets) that are derived from the given individual sets. Projection algorithms have various algorithmic structures, of which some are particularly suitable for parallel computing, and they have desirable convergence properties and/or good initial behavior (see, e.g., Bauschke and Borwein 1996 and Censor and Zenios 1997).

In this paper, we focus our attention on BIP methods. These are algorithms that in an iterative step first project the current iterate simultaneously onto the sets of some subfamily (called a block) of the whole family of sets and then take a convex combination of the resulting points as the next iterate. This block-iterative algorithmic scheme encompasses as special cases the sequential (row-action) algorithmic structure as well as the fully simultaneous algorithmic structure, but many additional in-between structures are permitted and convergence to a solution of the convex feasibility problem is guaranteed under reasonable conditions.

The specific question that we ask is: Are BIP methods resilient to perturbations of the iterates occurring during the iterative process? This question is important for two reasons. First, because numerical errors and system model noise may cause at every iterative step deviations from the ideal (mathematical) projection point. Second, because we can use the flexibility that such perturbations allow to create block-iterative algorithms that will not only solve a convex feasibility problem but will steer the iterates toward a feasible point that has some desirable properties.

The answer that we give here is affirmative. We show that the convergence to a solution of a BIP algorithm for a consistent convex feasibility problem is retained even if in each iterative step the current iterate is perturbed into x^k + γ_kw^k. For this to happen, we need the perturbations to be bounded and summable; i.e., they are such that the sequence ${\left\{{\underset{\_}{\mathit{w}}}^k\right\}}_{k=0}^{\infty }$ is bounded and the real parameters γ_k obey ${\sum }_{k=0}^{\infty }{\gamma }_k<\infty$.

Our analysis that proves that BIP methods are resilient to perturbations is based on the construction of another convex feasibility problem in an appropriately chosen product space, and on using for it an amalgamated projection method. In Butnariu et al. (2007), we recently proved the convergence under summable perturbations for such methods and in this paper we utilize that result for our product space construction that allows us to extend it to BIP methods.

The proved resilience of our algorithm is used to steer the iterative process towards a feasible point that is superior in the sense of some functional on the points in the Euclidean space having a small value. The potential usefulness of this is demonstrated in image reconstruction from projections, using total variation (TV) as the functional. We confirm, using our algorithm, the validity of previously reported results obtained using alternative TV-minimizing algorithms: when a physically unrealistic mathematical idealization of data collection in the image reconstruction from projections problem is employed (one in which consistency is assured by simulating data collection using the same model as used in turning the image reconstruction problem into a convex feasibility problem), then remarkably high-quality reconstructions can be obtained in very underdetermined situations by TV-minimization (as opposed to, say, norm-minimization or entropy-maximization). We also illustrate that even when applied to data that have been collected in a physically realistic manner, the result produced using the TV-minimizing perturbations appears to be superior to that produced without them.

Section 2 provides the necessary background by restating the previously published perturbation-resilient amalgamated projection method. Our new perturbation-resilient block-iterative algorithm is specified in Section 3, together with a proof of its convergence under perturbations. Our proof is based on reduction, using the methodology of product spaces, to the result presented in Section 2. In Section 4, the notion of superiorization is introduced; it is the method of steering a perturbation-resilient algorithm towards the minimizer of a given functional by an appropriate selection of the perturbations. The potential usefulness of this is illustrated in Section 5, by showing the applicability of our approach to image reconstruction from projections with TV as the given functional. The examples that are given in this section are all linear; more nonlinear problems can probably benefit from the superiorization methodology proposed here, but are outside the scope of this paper. Finally, we present a discussion of our approach in Section 6.

2. Background

Let J be a positive integer. Throughout this paper, we use bold italic letters for points in the vector space ℝ^J and regular italic letters for real numbers in ℝ. For x ∈ ℝ^J, we use the notation

$$\mathit{x}=\left(\begin{array}{c}{x}_1\\ ⋮\\ x_J\end{array}\right),x_j\in \mathbb{R}\text{for}1\le j\le J.$$ (1)

For any x ∈ ℝ^J and for any nonempty subset M of ℝ^J the distance between the point x and the set M is defined by

$$d(\mathit{x},M)=inf\{||\mathit{x}-\mathit{y}||\mid \mathit{y}\in M\},$$ (2)

where $\left|\right|\mathit{x}-\mathit{y}\left|\right|=\sqrt{{\sum }_{j=1}^J{(x_j-y_j)}^2}$ denotes the norm of x − y. If M is closed and convex, then there is one, and only one, y ∈ M such that

$$\left|\right|\mathit{x}-\mathit{y}\left|\right|=d(\mathit{x},M).$$ (3)

This y is called the projection of x onto M and is denoted by P_Mx.

For the rest of this section, we consider the following consistent convex feasibility problem: given I closed and convex subsets C₁, …, C_I of ℝ^J such that the set

$$C=\underset{i=1}{\overset{I}{\cap }}{C}_i,$$ (4)

is nonempty, find a point x in C. Such points x are called feasible.

An index vector is a nonempty ordered set t = (t₁, …, t_N), where N is an arbitrary positive integer, whose elements t_n are in the set {1, …, I}. For an index vector t, we define the composite operator

$$P[t];=P_{C_{t_N}}\cdots P_{C_{t_1}}.$$ (5)

A finite set Ω of index vectors is called fit if, for each i ∈ {1, …, I}, there exists t = (t₁, …, t_N)∈ Ω such that t_n = i for some n ∈ {1, …, N}. If Ω is a fit set of index vectors, then a function ω: Ω → ℝ₊₊ = (0, ∞) is called a fit weight function if Σ_t∈Ω ω (t) = 1. A pair (Ω, ω) consisting of a fit set of index vectors and a fit weight function defined on it is called an amalgamator. For each amalgamator (Ω, ω), we define the operator P: ℝ^J → ℝ^J by

$$\mathbf{P}\mathit{x}:=\sum _{t\in \mathrm{\Omega }}\omega (t)P[t]\mathit{x}.$$ (6)

Theorem 1

If {β_k}_{k∈ ℕ} is a sequence of positive real numbers such that ${\sum }_{k=0}^{\infty }{\beta }_k<\infty$ and {v^k}_{k∈ ℕ} is a bounded sequence of points in ℝ^J, then for any amalgamator (Ω, ω) and any x⁰ ∈ ℝ^J, the sequence x⁰, x¹, x², … generated by

$${\mathit{x}}^{k+1}=\mathbf{P}({\mathit{x}}^k+{\beta }_k{\mathit{v}}^k),\forall k\in \mathbb{N},$$ (7)

converges, and its limit is in C.

Proof

See Butnariu et al. (2007).

We refer to a procedure defined by (7) as a perturbation-resilient amalgamated projection method. Amalgamated projection methods without perturbations (i.e., with β_k = 0, for all k ∈ ℕ in (7)) are string averaging methods as introduced in Censor et al. (2001a), and further studied in Crombez (2002, 2004), Bauschke et al. (2004), Bilbao-Castro et al. (2003), and Censor and Tom (2003).

3. A Perturbation-resilient block-iterative algorithm

In this section, we propose a perturbation-resilient block-iterative algorithm for solving convex feasibility problems. Our convergence analysis shows that the perturbation-resilient block-iterative algorithm converges to a solution of the consistent convex feasibility problem as long as the perturbations are bounded and summable. Since the proof of convergence of the new algorithm will be by reduction to Theorem 1 applied in a different space, we need to introduce extra notation. Let L and Z be positive integers. Throughout this paper, we use underlined bold italic letters for points in ℝ^L. Our new convex feasibility problem is stated as: given Z closed and convex subsets D₁, …, D_Z of ℝ^L, such that the set

$$D=\underset{z=1}{\overset{Z}{\cap }}{D}_z$$ (8)

is nonempty, find a point x in D.

Given an ordered set S = (S₁, … S_R), such that S_r ⊆ ℝ^L, for 1≤ r ≤ R, we set J = RL and define the product set ℙS as the set of all

$$\mathit{x}=\left(\begin{array}{c}{x}_1\\ ⋮\\ x_L\\ ⋮\\ x_{(R-1)L+1}\\ ⋮\\ x_{RL}\end{array}\right)\in {\mathbb{R}}^J,$$ (9)

such that, for 1 ≤ r ≤ R,

$$\left(\begin{array}{c}{x}_{(r-1)L+1}\\ ⋮\\ x_{rL}\end{array}\right)\in S_r.$$ (10)

Under these circumstances, we use the notations

$${\underset{\_}{\mathit{x}}}_r=\left(\begin{array}{c}{x}_{(r-1)L+1}\\ ⋮\\ x_{rL}\end{array}\right)\text{and}\mathit{x}=\left(\begin{array}{c}{\underset{\_}{\mathit{x}}}_1\\ ⋮\\ {\underset{\_}{\mathit{x}}}_R\end{array}\right),$$ (11)

and we refer to ℝ^J(=ℝ^RL) as the product space. Note that for x, y ∈ ℝ^J,

$${\left|\right|\mathit{x}-\mathit{y}\left|\right|}^2=\sum _{r=1}^R{\Vert {\underset{\_}{\mathit{x}}}_r-{\underset{\_}{\mathit{y}}}_r\Vert }^2.$$ (12)

It is well known (and easy to prove) that if every element S_r of S = (S₁, …, S_R) is a closed convex subset of ℝ^L, then ℙS is a closed convex subset of ℝ^J. Furthermore, according to Censor and Zenios (1997, Lemma 5.9.2),

$$P_{\mathbb{P}S}\left(\begin{array}{c}{\underset{\_}{\mathit{x}}}_1\\ ⋮\\ {\underset{\_}{\mathit{x}}}_R\end{array}\right)=\left(\begin{array}{c}{P}_{S_1}{\underset{\_}{\mathit{x}}}_1\\ ⋮\\ P_{S_R}{\underset{\_}{\mathit{x}}}_R\end{array}\right).$$ (13)

One more piece of terminology that we need is the following. The canonical mapping δ: ℝ^L → ℝ^J is defined by: For any x ∈ ℝ^L, δ (x): = x, with x₁ = · · · = x_R = x (here we made use of the notation of (11)).

We now return to the convex feasibility problem that is stated in the first paragraph of this section. For 1 ≤ u ≤ U let B_u be an ordered set (b_u,1, …, b_u,|Bu|) of elements of {1, …, Z} (|B_u| denotes the cardinality of B_u). We call such a B_u a block. We define the composite operator Q: ℝ^L → ℝ^L as

$$\mathit{Q}:=Q_U\cdots Q_1,$$ (14)

where for x ∈ ℝ^L and 1 ≤ u ≤ U,

$$Q_u\underset{\_}{\mathit{x}}:=\frac{1}{R}\sum _{z\in B_u}{P}_{D_z}\underset{\_}{\mathit{x}}+\frac{R-\mid B_u\mid }{R}\underset{\_}{\mathit{x}},$$ (15)

and

$$R:=max\{\mid B_u\mid \mid 1\le u\le U\}.$$ (16)

An iterative procedure based on x^k+1 = Qx^k is a member of the family of BIP methods (Censor et al., 2001b, 2007; Aharoni and Censor, 1989; Aleyner and Reich, 2008; Eggermont et al., 1981; Elfving, 1980).

Theorem 2

Let L, Z, U, ${\{B_u\}}_{u=1}^U$ and Q be as defined above. Let λ be a real number such that 0 < λ ≤ 1, {γ_k}_{k∈ ℕ} be a sequence of positive real numbers such that ${\sum }_{k=0}^{\infty }{\gamma }_k<\infty$, {w^k}k∈ ℕ be a bounded sequence of points in ℝ^L, and x⁰ ∈ ℝ^L. If $\{1,\dots ,Z\}={\cup }_{u=1}^U{B}_u$, then the sequence x⁰, x¹, x², … generated by the iterative procedure

$${\underset{\_}{\mathit{x}}}^{k+1}=\lambda \mathit{Q}({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k)+(1-\lambda )({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k),\forall k\in \mathbb{N},$$ (17)

converges, and its limit is in D.

Proof

Our proof utilizes Theorem 1 by showing that the procedure in (17) is a perturbation-resilient amalgamated projection method in a product space. (Pierra (1984) was the first to use convergence results of sequential algorithms in a product space to prove the convergence of simultaneous algorithms; see also Censor and Tom, 2003 and Censor and Zenios, 1997, Section 5.9.) We present the proof for the case λ < 1, and afterward indicate how it can be altered for the case λ = 1.

We define a convex feasibility problem with I = U+2 convex sets in ℝ^J for J = RL and a perturbation-resilient amalgamated projection method. For 1 ≤ u ≤ U,

$$C_u:=\mathbb{P}(D_{b_{u,1}},\dots ,D_{b_{u,\mid B_u\mid }},{\mathbb{R}}^L,\dots ,{\mathbb{R}}^L),$$ (18)

with R − |B_u| copies of ℝ^L at the end of (18). Further,

$$C_{U+1}:=\{\mathit{\delta }(\underset{\_}{\mathit{x}})\mid \underset{\_}{\mathit{x}}\in {\mathbb{R}}^L\},$$ (19)

$$C_{U+2}:={\mathbb{R}}^J.$$ (20)

We now specify P_Cu, for 1 ≤ u ≤ U+2, by making use of (13). For 1 ≤ u ≤ U,

$$P_{C_u}\left(\begin{array}{c}{\underset{\_}{\mathit{x}}}_1\\ ⋮\\ {\underset{\_}{\mathit{x}}}_{\mid B_u\mid }\\ {\underset{\_}{\mathit{x}}}_{\mid B_u\mid +1}\\ ⋮\\ {\underset{\_}{\mathit{x}}}_R\end{array}\right)=\left(\begin{array}{c}{P}_{D_{b_{u,1}}}{\underset{\_}{\mathit{x}}}_1\\ ⋮\\ P_{D_{b_{u,\mid B_u\mid }}}{\underset{\_}{\mathit{x}}}_{\mid B_u\mid }\\ {\underset{\_}{\mathit{x}}}_{\mid B_u\mid +1}\\ ⋮\\ {\underset{\_}{\mathit{x}}}_R\end{array}\right).$$ (21)

The set C_U+1 is called the diagonal convex (Pierra, 1984) or the diagonal subset (Censor and Tom, 2003) and

$$P_{C_{U+1}}\mathit{x}=\mathit{\delta }\left(\frac{1}{R}\sum _{r=1}^R{\underset{\_}{\mathit{x}}}_r\right)$$ (22)

(for a proof see Pierra, 1984). Clearly,

$$P_{C_{U+2}}\mathit{x}=\mathit{x}.$$ (23)

Next, we show that $C={\cap }_{u=1}^{U+2}{C}_u$ is nonempty. By the assumption of the consistent convex feasibility problem of this section, $D={\cap }_{z=1}^Z{D}_z$ is nonempty. It is easy to see that if x ∈ D then δ (x) ∈ C. From this follows that C₁, …, C_I satisfy the conditions of the convex feasibility problem of the last section and hence Theorem 1 applies.

To show that Theorem 1 implies Theorem 2, we need to make some specifications. We let

$$\mathrm{\Omega }:=\{(1,U+1,\dots ,U,U+1),(U+2)\}.$$ (24)

(To be precise, in the first of these two index vectors every second element is U+1 and these are preceded by 1, …, U, respectively.) This Ω is fit because, for each i ∈ {1, …, U, U + 1, U + 2 = I}, there exist (t₁, …, t_N) ∈ Ω such that t_n = i for some n ∈ {1, …, N}. Let the weight of the first of the two index vectors in Ω be λ and the weight of the second one be 1 − λ. The resulting weight function ω is fit since each weight is in ℝ₊₊ (this is where one needs λ < 1) and Σ_t∈Ω ω (t) = 1. We also set β_k = γ_k and v^k = δ (w^k), for all k ∈ ℕ, and x⁰ = δ (x⁰):

Claim

Given the specifications of the previous paragraph, the sequence x⁰, x¹, x², … produced by (7) of Theorem 1 has the property that, for k ∈ ℕ, x^k = δ (x^k); where x⁰, x¹, x², … is the sequence produced by (17) of Theorem 2.

If this claim is true, then Theorem 2 follows as we now show. By Theorem 1, x⁰, x¹, x², … converges to an x* ∈ C: Since x* ∈ C_U+1, it is of the form x* = δ (x*), for some x* ∈ ℝ^L. By the Claim, x^k = δ (x^k), for k ∈ ℕ. By (12),

$${\left|\right|{\underset{\_}{\mathit{x}}}^k-{\underset{\_}{\mathit{x}}}^{\ast }\left|\right|}^2\le {\left|\right|{\mathit{x}}^k-{\mathit{x}}^{\ast }\left|\right|}^2,$$ (25)

and since the right-hand side converges to zero, so must the left-hand side, and hence x⁰, x¹, x², … converges to x*. Since x* = δ (x*) ∈ C_u, for 1 ≤ u ≤ U we have, by (18), that x* ∈ D_bu,m for 1 ≤ u ≤ U and 1 ≤ m ≤ |B_u|. Since $\{1,\dots ,Z\}={\cup }_{u=1}^U{B}_u$ and $D={\cap }_{z=1}^Z{D}_z$, it must be so that x* ∈ D.

We next prove the Claim by induction on k. The Claim is clearly true for k = 0 since x⁰ = δ (x⁰): Let us now assume that x^k = δ (x^k) is true for some k ≥ 0 and prove that it follows that

$${\mathit{x}}^{k+1}=\mathit{\delta }({\underset{\_}{\mathit{x}}}^{k+1}).$$

Using (5)–(7), (23) and (24) with 0< λ <1, we obtain

$${\mathit{x}}^{k+1}=\lambda P_{C_{U+1}}{P}_{C_U}\cdots P_{C_{U+1}}{P}_{C_1}(x^k+{\beta }_k{\mathit{v}}^k)+(1-\lambda )({\mathit{x}}^k+{\beta }_k{\mathit{v}}^k).$$ (26)

By the induction hypothesis and the assumptions that β_k = γ_k and v^k = δ (w^k) we obtain

$${\mathit{x}}^{k+1}=\lambda P_{C_{U+1}}{P}_{C_U}\cdots P_{C_{U+1}}{P}_{C_1}(\mathit{\delta }({\underset{\_}{\mathit{x}}}^k)+{\gamma }_k\mathit{\delta }({\underset{\_}{\mathit{w}}}^k))+(1-\lambda )(\mathit{\delta }({\underset{\_}{\mathit{x}}}^k)+{\gamma }_k\mathit{\delta }({\underset{\_}{\mathit{w}}}^k)).$$ (27)

By (14) and (17) and the obvious linearity of δ we obtain

$$\mathit{\delta }({\underset{\_}{\mathit{x}}}^{k+1})=\lambda \mathit{\delta }(Q_U\cdots Q_1({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k))+(1-\lambda )(\mathit{\delta }({\underset{\_}{\mathit{x}}}^k)+{\gamma }_k\mathit{\delta }({\underset{\_}{\mathit{w}}}^k)).$$ (28)

Since the second terms of (27) and (28) are identical, the proof that x^k+1 = δ (x^k+1) is complete if we can show that

$$P_{C_{U+1}}{P}_{C_U}\cdots P_{C_{U+1}}{P}_{C_1}\mathit{\delta }({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k)=\mathit{\delta }(Q_U\cdots Q_1({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k)).$$ (29)

In fact, we show by induction on u the stronger statement that

$$P_{C_{U+1}}{P}_{C_u}\cdots P_{C_{U+1}}{P}_{C_1}\mathit{\delta }({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k)=\mathit{\delta }(Q_u\cdots Q_1({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k)),$$ (30)

for 0 ≤ u ≤ U. This is clearly true for u = 0. Let us now assume that it is true for a u such that 0 ≤ u ≤ U, and show that it is also true for u + 1.

Let y abbreviate Q_u · · · Q₁ (x^k + γ_kw^k). By the induction hypothesis, (30), (21), (22) and (15),

$$\begin{array}{l}{P}_{C_{U+1}}{P}_{C_{u+1}}{P}_{C_{U+1}}{P}_{C_u}\cdots P_{C_{U+1}}{P}_{C_1}\mathit{\delta }({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k)=P_{C_{U+1}}{P}_{C_{u+1}}\mathit{\delta }(\underset{\_}{\mathit{y}})\\ =P_{C_{U+1}}\left(\begin{array}{c}{P}_{D_{b_{u+1,1}}}\underset{\_}{\mathit{y}}\\ ⋮\\ P_{D_{b_{u+1,}\mid B_{u+1}\mid }}\underset{\_}{\mathit{y}}\\ \underset{\_}{\mathit{y}}\\ ⋮\\ \underset{\_}{\mathit{y}}\end{array}\right)=\mathit{\delta }\left(\frac{1}{R}\sum _{z\in B_{u+1}}{P}_{D_z}\underset{\_}{\mathit{y}}+\frac{R-\mid B_{u+1}\mid }{R}\underset{\_}{\mathit{y}}\right)\\ =\mathit{\delta }\left(Q_{u+1}\underset{\_}{\mathit{y}}\right)=\mathit{\delta }(Q_{u+1}{Q}_u\cdots Q_1({\underset{\_}{\mathit{x}}}^k+{\gamma }_k{\underset{\_}{\mathit{w}}}^k)).\end{array}$$ (31)

This completes the proof for the case λ < 1.

For the special case where λ = 1, we alter Ω to include the single index vector

$$(1,U+1,\dots ,U,U+1),$$ (32)

with weight 1. The rest of the proof is just a simpler version of the proof for the case λ < 1.

4. A heuristic approach to superiorization

Theorem 2 guarantees the convergence of the perturbation-resilient block-iterative algorithm (17) to a feasible point when the perturbations are bounded and summable. We make use of this property to steer the iterates towards the minimizer of a given convex function φ, i.e., towards the x ∈ ℝ^L that provides the solution of the problem

$$\text{minimize}\phi (\underset{\_}{\mathit{x}}),\text{subject}\text{to}\underset{\_}{\mathit{x}}\in D.$$ (33)

The heuristic provided below is not guaranteed to achieve actual convergence to the minimizer x. However, as demonstrated by examples in the next section, it proceeds so that the value of the given function tends to be appropriately reduced and yet convergence to a feasible point is not compromised. This allows us to do superiorization, by which we mean the production of a superior solution (just as optimization produces an optimal solution) subject to given (convex) constraints. The idea is that the ability to perturb the original algorithm without losing convergence to a feasible point allows us to steer the algorithm towards a feasible point that is superior, according to some criterion, to the one to which we would get without the perturbations.

Consider a convex function φ: ℝ^L → ℝ that has a minimizer over the set D. For any k ∈ ℕ, let ρ^k ∈ ∂φ(x^k) be a subgradient of φ at x^k, and define

$${\underset{\_}{\mathit{w}}}^k:=\{\begin{array}{ll}-\frac{{\rho }^k}{\left|\right|{\rho }^k\left|\right|,}\hfill & \text{if}{\rho }^k\ne 0,\hfill \\ 0,\hfill & \text{if}{\rho }^k=0.\hfill \end{array}$$ (34)

Clearly, the sequence {w^k}_k∈ℕ defined by (34) is bounded. Therefore, by Theorem 2, for any summable sequence of positive real numbers {γ_k}_k∈ℕ, the sequence {x^k}_k∈ℕ generated according to (17) converges to a point in D.

In our implementation, we use the following methodology for generating the real numbers {γ_k}_k∈ℕ. We define, for any x ∈ ℝ^L, the proximity function

$$\text{Res}(\underset{\_}{\mathit{x}}):=\sqrt{\sum _{z=1}^Z{(d(\underset{\_}{\mathit{x}},D_z))}^2}.$$ (35)

It can be seen that x ∈ D if, and only if, Res (x) = 0. Furthermore, if Res(x) > 0, then its size indicates how badly x violates the given collection {D₁, …, D_Z} of constraints. An approximate solution x to the convex optimization problem (33) should have a small value of Res (x) and should aim at finding, among all x with similar (or smaller) value of Res (x), an x for which φ (x) is small relative to the others.

Guided by this principle, we generate {γ_k}_k∈ℕ as follows. We initialize γ to be an arbitrary positive number, which we denote by γ_{− 1} = 1. (We have always used γ₋₁ = 1.) In the process of the iterative step from x^k to x^k+1, we also update the value of γ, which is (in the notation of (17)) γ_{k − 1} at the beginning of the iterative step and γ_k at its end. This updating is done according to the pseudocode below (in which w^k is defined by (34)). We terminate the iterative process when we find an x^k such that Res(x^k) < ε, where ε is a user-specified small positive number. The complete superiorization algorithm consists of (17) with the w^k defined by (34) and the γ_k defined by the following pseudocode that makes use of (35).

1:  logic = true
2:  while (logic)
3:   y = xk + γwk
4:   if (φ (y) ≤ φ (xk)
5:    then
6:      xk+1 = λQy + (1 − λ)y
7:      if (Res(xk+1) < Res (xk))
8:       then logic = false
9:       else γ = γ/2
10:    else γ = γ/2

What this algorithm performs is a steering process towards a small value of φ (see Step 4 of the pseudocode), while attempting to maintain the convergence to the feasible region, as guaranteed by Theorem 2 for a proper choice of the sequence {{γ_k}_k∈ℕ (see Step 7 of the pseudocode). The idea of using such a heuristic with a perturbation-resilient iterative method was first introduced in Butnariu et al. (2007) in the context of amalgamated projection methods. The resulting point of the iterative process depends on the starting point x⁰ and the choice of ε. In the next section, we give several illustrations of the new approach using the above algorithm, in which a functional is reduced and a solution point is obtained with a value of φ that is in all but one of the cases nearer to the minimum than the value for the phantom (i.e., the test image).

5. Results

We illustrate our superiorization approach in an area of image processing, namely tomographic reconstruction of images that are not uniquely determined from the available data, with the help of a convex functional φ that assigns to each image a number that indicates, in some sense, the “undesirability” of the image.

Figure 1(a) shows a phantom that is a 243 × 243 digitized image (thus L = 59,049), representing a cross-section of a human head; cf. Herman (1980). (We use this phantom in three out of the four experiments reported in this section.) The components of x represent the average X-ray attenuations within the 59,049 pixels, each of which is of size 0.0752 × 0.0752 (the assumed unit of length is 1 cm). The values of these components range from 0 to 0.5639; for display purposes any value below 0.204 is shown as black (gray value 0) and any value above 0.21675 is shown as white (gray value 255), with a linear mapping of the x-component values into gray values in between (this is true for all the figures throughout this paper).

Fig. 1.

Illustration of reconstructions obtained by our block-iterative algorithm from 82 views, with hyperplane constraints and stopping criterion of Res (x^k) <0.05. (a) Head phantom for which data were collected. (b) Norm-minimizing reconstruction (no perturbations). (c) Reconstruction with TV-superiorizing perturbations. (d) Reconstruction with entropy-superiorizing perturbations.

In the first experiment, data were collected by calculating line integrals through the digitized image for 82 sets of equally spaced parallel lines. Each such line integral gives rise to a linear equation in the components of x; the set of x that is consistent with such a line integral is a hyperplane in ℝ^L. These 82 sets are composed of 60 equally spaced views with 3° increments between consecutive directions and an additional 22 views with angle directions that make the large tumor on the left-hand side of the phantom in Fig. 1(a) a ghost (i.e., its ray-sums are zero for those 22 directions; Herman and Davidi, 2008). The vector x that represents the phantom itself lies in the intersection of all the hyperplanes that are associated with these directions. In this experiment, we used measurements for 25,452 lines, making our problem very much underdetermined. (The intersection of all the hyperplanes is an at least 33,597-dimensional subspace of ℝ^59,049.) In the terminology of Section 3, Z = 25,452 and, for 1 ≤ z ≤ Z, D_z is one of the hyperplanes. It is typical in practice (and this is what we will be doing here) that each block B_u consists of all the indices z associated with the measurements taken in a particular direction (and therefore $\{1,\dots ,Z\}={\cup }_{u=1}^U{B}_u$ with U = 82). We now describe the two convex functions φ that we have chosen for the first experiment.

Many researchers in image processing have been advocating the use of TV, e.g., Butnariu et al. (2007), Combettes and Pesquet (2004), Malgouyres (2002), Persson et al. (2001), and Rudin et al. (1992). For a G × H image q whose pixels are denoted by q_g,h (1 ≤ g ≤ G, 1 ≤ h ≤ H), the TV of q is

Fig. 1. Illustration of reconstructions obtained by our block-iterative algorithm from 82 views, with hyperplane constraints and stopping criterion of Res{x^k} <0.05. (a) Head phantom for which data were collected. (b) Norm-minimizing reconstruction (no perturbations). (c) Reconstruction with TV-superiorizing perturbations. (d) Reconstruction with entropy-superiorizing perturbations.

$$TV(q):=\sum _{g=1}^{G-1}\sum _{h=1}^{H-1}\sqrt{{(q_{g+1,h}-q_{g,h})}^2+{(q_{g,h+1}-q_{g,h})}^2}.$$ (36)

By mapping q into a (G × H = L)-dimensional point x (by stacking into a single column all the columns of q), this definition gives rise to a functional φ that can be used in our superiorization algorithm described in the previous section.

Our second choice for the convex function φ to be used in this experiment is based on the maximum entropy formalism, which is a general scientific approach with whole books devoted to it (e.g., Levine and Tribus, 1979). The suggestion that it be used for image reconstruction first appeared in the open literature in Gordon et al. (1970). It has been extensively used in the related field of digital image restoration (see, e.g., Andrews and Hunt, 1977). An algorithm that finds the maximum entropy solution of a consistent system of equations with nonnegativity constraints is the multiplicative algebraic reconstruction technique (MART), as was proved in Lent (1977). MART is actively used in various applications (see, e.g., Konovalov et al., 2008; Worth and Nickels, 2008). For images q that satisfy q_g,h ≥ 0, for 1 ≤ g ≤ G, 1 ≤ h ≤ H, with at least one value strictly greater than 0, the negative-entropy of q is

$$\mathit{negative}-\mathit{entropy}(q):=\sum _{g=1}^G\sum _{h=1}^H\left(\frac{q_{g,h}}{Q}\right)ln\left(\frac{q_{g,h}}{Q}\right),$$ (37)

where Q is a constant provided to us by estimating the sum of the q_{g, h} from the measured data (see the end of Section 6.4 of Herman, 1980) for a discussion as to why we may assume in image reconstruction from projections that this estimate is extremely accurate). We use negative entropy, rather than entropy, to define φ, since our algorithm is written for minimizing a function. Nevertheless, in reporting on our experiments we will be giving the value of the entropy (which is minus the value provided by (37)), thus a higher value will indicate a more desirable solution according to the maximum entropy principle. We need to point out that the previously specified algorithm needs to be altered for entropy maximization, due to the fact that in that case φ is defined only on the nonnegative orthant. For example, there is nothing to prevent Step 3 of the pseudocode to produce a y that is not in the domain of φ and this would make it impossible to execute Step 4. To avoid such difficulties, we put in additional conditions controlling the flow of the algorithm so that the φ based on negative entropy need never be evaluated for an argument outside the nonnegative orthant. We omit the technical details.

We applied our block-iterative algorithm (17) to the specified data set for the two choices of φ given above. The relaxation parameter for all reconstructions was set to λ = 1 and all were stopped when Res (x^k) < 0.05. Figure 1(b) shows the reconstruction when no perturbations are introduced and Fig. 1(c) shows the reconstruction when TV-superiorizing perturbations are present, set as specified in the previous section. Both runs started with x⁰ being the zero point (i.e., all its components are 0). Note that Res (x⁰) = 330.208, demonstrating that the value 0.05 for the stopping criterion is small in our context. Since the convex sets are hyperplanes, the algorithm without perturbations (i.e., γ_k = 0, for k ∈ ℕ) converges to the feasible point with minimal norm (see Herman, 1980). Clearly, the reconstruction in Fig. 1(c) is visually superior to the reconstruction in Fig. 1(b). As a numerical measure, the norm of the difference between the reconstruction without perturbations in Fig. 1(b) and the phantom in Fig. 1(a) is 3.764, while the norm of the difference between the reconstruction with the perturbations in Fig. 1(c) and the phantom is 0.157, making it 24 times smaller than that without perturbations. Moreover, the TV of the image in Fig. 1(c) is 421.568, which is near to (and is actually less than) the TV of the phantom in Fig. 1(a) (450.594), whereas the TV of the image in Fig. 1(b) is 1270.240.

We further report on the results of the reconstruction when the perturbations are aimed at maximizing the entropy as defined by (37). For this run, x⁰ was set to be the point that is also the initial point for MART (Lent, 1977) for which each component is either zero or another constant, selected based on the measurements so that the total density is the Q of (37). Figure 1(d) presents the resulting image for the chosen stopping criterion. If we compare the two superiorizing reconstruction results (Fig. 1(c) and (d)), the TV-superiorizing reconstruction appears clearly preferable. However, it is worth noting that the entropy-superiorizing algorithm behaves as advertised: a superior result relative to the function of (37) is provided by the entropy-superiorization algorithm. In fact, we ran MART (as implemented in the software package SNARK05; Carvalho et al.) on the data of this experiment and obtained an entropy of 10.305. The entropy of the image in Fig. 1(d) is 10.307, which is higher than that obtained by MART for the same data and the same stopping criterion, indicating that our algorithm performs better in the superiorization sense. However, looking at the image produced by TV-superiorization (Fig. 1(c)) we see that, for this data set, the TV-minimization criterion characterizes the phantom better than the maximum entropy criterion.

As a further illustration, we report on results of an experiment in which in addition to the Z hyperplanes used in the previous experiment, we also have a convex set corresponding to nonnegativity constraints on the pixel values; i.e., the set

$$D_{Z+1}=\{\underset{\_}{\mathit{x}}\mid x_l\ge 0,\text{for}1\le l\le L\}.$$ (38)

It is easy to see that if y = P_DZ+1 x, then y_l = max(x_l, 0), for 1 ≤ l ≤ L. In the exact formulation of our algorithm for this case, we introduce in addition to the U blocks used in the previous case one more block B_U+1 that contains R copies of the index Z + 1. From (15) it follows that, for all x ∈ ℝ^L, Q_U+1 x = P_DZ+1 x, which implies that the implementation of the operator Q_U+1 is trivial. Our actual algorithm for taking care of nonnegativity (in addition to the hyperplane constraints) uses instead of (14) the operator

$$\mathbf{Q}:=Q_{U+1}{Q}_U\cdots Q_{U+1}{Q}_1,$$ (39)

which means that in the sequence of the blocks of the convex sets, each B_u (i.e., each block of hyperplanes, as defined for the previous experiment) is followed by a projection onto D_Z+1.

We compared the performance of our block-iterative algorithm without and with perturbations for the TV functional (36). As in the previous experiment, we started both reconstructions with x⁰ being the zero point, set λ = 1, and set the algorithms to stop at Res(x^k) < 0.05. The results of the reconstructions are given in Fig. 2. Note again the superiority of the reconstruction of the TV-superiorizing algorithm with perturbations present (Fig. 2(b)) as compared with the one produced without perturbations (Fig. 2(a)). The norm of the difference between the reconstruction without perturbations in Fig. 2(a) and the phantom is more than eight times greater than that between the reconstruction with the perturbations in Fig. 2(b) and the phantom (1.254 and 0.143, respectively). The use of the nonnegativity constraint (38) improved the quality of the reconstructions in both cases, although only slightly in the TV-superiorizing case. The TV for the image on the right in Fig. 2 is 421.185, while for that on the left it is 667.926.

Fig. 2.

Illustration of reconstructions obtained by our block-iterative algorithm from 82 views, with hyperplane and nonnegativity constraints and stopping criterion of Res (x^k) < 0.05. (a) Reconstruction when no perturbations are present. (b) Reconstruction with TV-superiorizing perturbations.

Until now, all the reconstructions were from data sets in which the line integrals were calculated exactly based on a digitized phantom. The reconstruction algorithms that we used were in fact developed using the assumption that this is indeed the nature of the data. However, real data in applications of image reconstruction from projections will not be such. A basic reason is that the object from which the data are collected will not be a digitized one (i.e., there will be variations even within pixels). Also, the detectors used in real instruments for collecting data will have a width and so, even if they were otherwise perfect, they could not be used for measuring line integrals exactly. In addition, measurements are stochastic in nature; in computerized tomography (CT), for example, the line integrals are estimated by the use of a, by necessity, finite number of X-ray photons, resulting in statistical noise in these estimates. There are further sources of discrepancy between the assumed mathematical model and physically collected data, such as the presence of scattered X-ray photons corrupting the readings by the individual detectors (see Herman, 1980).

In the third experiment, we demonstrate our block-iterative algorithm when the data are realistic from these points of view. Calculations of the line integrals were based on a geometrical description of the head phantom rather than on its digitization. In order to simulate the width of the detector, for each line for which the algorithm assumes that the data had been collected, we introduce 10 additional lines (five on both sides) with spacing a/11 between them, where a (= 0.0752 in our case) is the distance between parallel lines along which data are assumed to have been collected in the mathematical formulation. In addition, statistical noise and scatter were introduced at the levels found in real CT scanners. (The software SNARK05 (Carvalho et al.), which we used for all our experiments, allows us to simulate this kind of data collection, the details of which are explained in Herman (1980).) Data were generated for 360 sets of equally spaced parallel lines (each line gives rise to a hyperplane in ℝ^59,049), with 0.5° increments between consecutive directions. Similarly to our previous experiments, blocks were formed using all measurements taken from a particular direction (making U = 360), and an additional constraint of the form (38) was utilized in the same manner as in the second experiment.

The reconstructions from such realistic data, both without and with TV-superiorizing perturbations, are shown in Fig. 3. The stopping criterion in both cases was Res (x^k) < 3.75, which is reasonable since for this noisy data set the value of Res for the phantom is actually higher (3.97, to be exact). This means in particular that, due to the discrepancy between the actual mode of data collection and the mathematical model that is assumed for it, the phantom no longer lies in the intersection of all the constraints that our algorithm is designed to satisfy simultaneously. Nevertheless, both reconstructions seem to capture the essential features of the phantom, with the reconstruction on the right (obtained using TV-superiorizing perturbations) somewhat more pleasing to the eye than the reconstruction on the left; in particular, it seems to be smoother. This is reflected by the norms of the differences between the reconstructions and the phantom; they are 1.914 and 1.877, respectively, for Fig. 3(a) and Fig. 3(b). Comparing the TVs of the two reconstructed images, we note that the image on the right has a much lower TV (440.411) than the image on the left (518.779). (We point out again that the TV of the phantom is 450.594.) Matching the results of the previous two experiments indicates that the use of TV-superiorizing perturbations steers the iterates towards an image with a lower TV, which appears to be a desirable feature when attempting to reconstruct an object of the type shown in Fig. 1(a).

Fig. 3.

Illustration of reconstructions obtained by our block-iterative algorithm from 360 views of physically realistic projection data, with hyperplane and nonnegativity constraints and stopping criterion of Res (x^k) <3.75.

(a) Reconstruction when no perturbations are present. (b) Reconstruction with TV-superiorizing perturbations.

In all the examples above, when the algorithm stops it provides an image with a value of the functional φ less than that of the phantom. The following example shows that this is not always so. Figure 4(a) shows a head phantom for which data were collected. The only difference between the head phantom in Fig. 1(a) and this one is that in Fig. 1(a) the phantom includes local inhomogeneities; i.e., each of the 243 × 243 pixels was multiplied by a sample from a normal distribution with mean 1 and a standard deviation that is appropriate to produce the variability found in actual CT scans of the human head. For the current example we decreased the number of views to 22 and calculated ideal line integrals through the digitized head phantom (similarly to the way we collected the data for the first experiment at the beginning of the section). This gave rise to 6,914 hyperplanes, thus making the problem even more underdetermined than in our previous examples. Here U = 22 and Z = 6,914, and so we have almost an order of magnitude fewer linear equations than unknowns. We again ran our algorithm starting from the zero point (Res (x⁰) = 171.557 in this case) and chose the stopping criterion of Res (x^k) <0.05. The TV of the phantom in Fig. 4(a) is 421.735. Since the system is consistent, the phantom in Fig. 4(a) lies in the intersection of all the constraints. Figure 4(b) shows the result of the TV-superiorizing when perturbations are present. The TV of that image (427.372) is higher than the TV of the phantom. Even though the reconstruction is visually poor, the superiority of it is seen by comparing it with the reconstruction when no perturbations are used in Fig. 4(c) and when the perturbations are aimed at maximizing the entropy (Fig. 4(d)). As a numerical measure, the norm of the difference between the reconstruction with perturbations in Fig. 4(b) and the phantom in Fig. 4(a) is 1.684, which is less than a fifth of the norm of the difference between Fig. 4(c) and the phantom and less than half of the norm of the difference between Fig. 4(d) and the phantom.

Fig. 4.

Illustration of reconstructions obtained by our block-iterative algorithm using consistent data with 22 views, hyperplane constraints and stopping criterion of Res (x^k) <0.05. (a) Head phantom for which data were collected.

(b) Reconstruction with TV-superiorizing perturbations. (c) Reconstruction when no perturbations are present.

(d) Reconstruction with entropy-superiorizing perturbations.

Up to this point we have been concentrating on the quality of our algorithm from the point of view of it achieving its aim of providing a point for which the values of both Res and the specified φ are small, without paying any attention to the computational costs involved. It happens to be the case that the timing of our algorithm can be greatly improved under certain circumstances. The reason for this is that the number R in (15) is large; it is 345 for our examples. From this it follows that x^k+1 = Qx^k is practically the same as x^k and so it takes quite a number of iterations to reach the stopping criterion. Fortunately, under many reasonable circumstances (e.g. if all the convex sets are hyperplanes), it is possible to replace (15) by an alternative, but similar, formula and achieve a more than an order of magnitude speed up while retaining the limiting convergence property stated in Theorem 2. We will publish the exact formulation, proof and illustration of this claim in a forthcoming paper; here we note only that similar approaches have been published already for algorithms without perturbations (e.g., in Censor et al., 2007). To indicate the great potential that exists for speeding up the block-iterative algorithm used in this section we note that it required 10,400 iterations to reach the very stringent stopping criterion that resulted in Fig. 1 (we note that essentially the same image would have been obtained if we stopped much earlier), but when we used instead the perturbed fully sequential algorithm of Butnariu et al. (2007), which is the same as the algorithm described in this paper with all blocks being of size 1, then the stopping criterion was reached in only 20 iterations. This is not necessarily a good thing from the point of view of superiorization, since perturbations are made only at the end of each iteration and so the value of the φ is likely to be much higher if the algorithm stops after relatively few iterations. In practice one would use an underrelaxed version of the fully sequential algorithm (i.e., a smallish λ in (17)), which would slow it down to some extent. Also, it is often the case that special hardware allows us to implement a block-iterative algorithm so that one iteration of it requires less time than one iteration of its fully sequential version. For these reasons, the speeded up versions of our perturbed block-iterative algorithm have the potential of being in practice advantageous.

6. Discussion

In this section, we discuss the nature of our approach to solving our problem bearing in mind the existence of alternative approaches. We give reasons why feasibility problems of various kinds are looked at from the viewpoint of projection methods. Projections onto sets are used in a wide variety of methods in optimization theory, but not every method that uses projections really belongs to the class of projection methods. As mentioned earlier, projection methods are iterative algorithms that use projections onto individual sets, relying on the general principle that when a family of (usually closed and convex) sets is present then projections onto the given individual sets are easier to perform than projections onto other sets (intersections, image sets under some transformation, etc.) that are derived from the given individual sets.

The main advantage of projection methods, which makes them successful in real-world applications, is computational. They commonly have the ability to handle huge-size problems of dimensions beyond which other, more sophisticated currently available, methods cease to be efficient. This is so because the building bricks of a projection algorithm are the projections onto the given individual sets (assumed and actually easy to perform) and the algorithmic structure is either sequential or simultaneous (or in-between, i.e., block-iterative, as discussed in the current paper). Sequential algorithmic structures cater for the row-action approach (see Section 1.3 on classification of parallel algorithms in Censor and Zenios, 1997) while simultaneous algorithmic structures favor parallel computing platforms (see, e.g., Censor et al., 2001b).

In the problems discussed in the previous section, the number of unknowns was 59,049. In the examples given in Herman and Chen (2008) (a paper devoted to radiation therapy planning), the number of unknowns was 515 and the feasible constraint set consisted of 128,668 pairs of linear inequality constraints. In four of the six cases reported there, the projection method called “ART3+” found a feasible point in less than 3 s, and in the remaining two cases a feasible point was found in less than 34 s. These times are for a standard PC, using an Intel Xeon 1.7 GHz processor and 1 G RAM. The problem in Herman and Chen (2008) is actually small compared with some of the other applications for which projection methods have been successfully used. In Carazo et al. (2006) (a paper devoted to reconstruction from electron microscopic projections), there are examples in which 16,777,216 unknowns are to be recovered from 4,587,520 measurements (each giving an approximate linear equality) and others in which 884,436 unknowns are to be recovered from 92,160,000 measurements. Projection methods (similar to the algorithm presented in this paper, but without perturbations) were used in Carazo et al. (2006) to handle such large problems in a reasonable time.

As is true for all methodologies, projection methods are not necessarily the approach of choice in all applications. However, in the important applications mentioned in this paper (bio-medicine and image processing), projection methods work well and have been used successfully for a long time. A mathematical reason for this is that for these problems the angles between hyperplanes or half-spaces, represented by the linear equalities or linear inequalities, are in general large due to the high sparsity of the system matrix.

In a recent paper, Gould (2008) wrote, referring to projection methods for convex feasibility problems: “Unfortunately, particularly given the large literature which might make one think otherwise, numerical tests indicate that in general none of the variants considered are especially effective or competitive with more sophisticated alternatives.” Gould also stated there: “We do not want to suggest here that successive and simultaneous projection methods are not useful, since in particular they appear to have been applied successfully for many years in medical imaging, radiation therapy planning and signal processing [references]. But our experience suggests that despite the large literature devoted to theoretical analysis, they should not be considered as the method of choice for a given application without further strong empirical evidence to support such a claim.” The largest problem tried in Gould (2008) has 22,275 unknowns and 6630 linear constraints and, for that problem, all methods reported in Gould (2008) needed 42 s or more to reach the stopping point used there on an 3.06 GHz Dell Precision 650 Workstation, which should be nearly twice as fast as the computer used in Herman and Chen (2008). Comparing this with the results from Herman and Chen (2008) quoted above indicates that Gould’s criticism of projection methods may very well be invalid for the applications mentioned in the previous paragraph.

Projection methods are not restricted to convex feasibility problems. The same general principle of reaching a goal associated with a family of sets by performing projections onto individual members of the family works also when solving certain optimization problems (see, e.g., Censor and Zenios, 1997). Another problem that is related to the convex feasibility problem is the best approximation problem of finding the projection of a given point onto the nonempty intersection of a family of closed convex subsets (see, e.g., Deutsch, 2001). The convex sets commonly represent mathematical constraints obtained from the modeling of a real-world problem. In the convex feasibility approach any point in the intersection is an acceptable solution to the real-world problem, whereas the best approximation formulation is usually appropriate if some initial point has been obtained from modeling and computational efforts that did not take into account the constraints, and now we wish to incorporate them by seeking a point in the intersection of the convex sets that is closest to the initial point. Iterative projection algorithms for finding a projection of a point onto the intersection of sets are more complicated then algorithms for finding just any feasible point in the intersection because they must have, in their iterative steps, some built-in “memory” mechanism to remember the original point whose projection is sought after. The sequential or parallel algorithms of Dykstra (see Deutsch, 2001), of Haugazeau (see Bauschke and Combettes, 2001), and of Halpern-Lions-Wittmann-Bauschke (see Bauschke, 1996) employ variants of such memory mechanisms. It is well-known that the convex feasibility problem is a special case of the common fixed points problem. In that problem a common fixed point of a family of operators, not necessarily projectors, is sought. This is a fundamental problem in fixed point theory and there is a large, and ever growing, body of literature available on theoretical issues of generalizations and convergence associated with projection methods in that mathematical field as well as in optimization oriented papers. In our paper, we have not considered the generalizations and extensions discussed in this paragraph, but it appears to us that perturbation-resilient BIP methods are likely to play a role in their future developments.