Accelerated Edge-Preserving Image Restoration Without Boundary Artifacts

Abstract

To reduce blur in noisy images, regularized image restoration methods have been proposed that use non-quadratic regularizers (like l₁ regularization or total-variation) that suppress noise while preserving edges in the image. Most of these methods assume a circulant blur (periodic convolution with a blurring kernel) that can lead to wraparound artifacts along the boundaries of the image due to the implied periodicity of the circulant model. Using a non-circulant model could prevent these artifacts at the cost of increased computational complexity.

In this work we propose to use a circulant blur model combined with a masking operator that prevents wraparound artifacts. The resulting model is non-circulant, so we propose an efficient algorithm using variable splitting and augmented Lagrangian (AL) strategies. Our variable splitting scheme, when combined with the AL framework and alternating minimization, leads to simple linear systems that can be solved non-iteratively using FFTs, eliminating the need for more expensive CG-type solvers. The proposed method can also efficiently tackle a variety of convex regularizers including edge-preserving (e.g., total-variation) and sparsity promoting (e.g., l₁ norm) regularizers. Simulation results show fast convergence of the proposed method, along with improved image quality at the boundaries where the circulant model is inaccurate.

I. Introduction

IMAGE restoration is a well studied problem and there are many methods for deblurring and denoising. Usually image restoration is treated as an optimization problem where the restored image is obtained by minimizing a cost function consisting of a data fidelity term and a regularization term. The data fidelity term ensures good fit of the blur model to the measurements, and the regularizer ensures stability of the solution and incorporates smoothness to suppress noise. A quadratic data fidelity term is often used, based on the additive zero-mean Gaussian noise model. For the regularizer, using a quadratic term can lead to over-smoothing. Recently there is increasing interest in non-quadratic regularizers, especially edge-preserving ones like total variation (TV) [1]–[3] and sparsity promoting ones like l₁ regularization [4], [5].

Most existing restoration methods make simplifying assumptions concerning the system model and the most common one is the use of a circulant blurring model [3]–[10] because it facilitates FFT-based optimization (e.g., non-iterative matrix inversions [6], [10]). Despite its popularity, the purely circulant model is inaccurate, since it implies an unrealistic periodic extension at the image boundaries that can lead to severe reconstruction artifacts [11].

In this work we focus on a more realistic non-circulant blurring model that is shift-invariant within the region of interest and propose an efficient algorithm for image restoration with non-quadratic regularization. Our model is equivalent to that proposed in [11], and similarly to [12] we focus on edge-preserving regularizers instead of the quadratic regularizer used in [11]. However, unlike [11], [12] which treat boundaries explicitly via a low-dimensional auxiliary variable, our approach is based on an elegant formulation that requires no preprocessing of the data or explicit treatment of the unknown, extrapolated boundaries. In addition, the methods in [11], [12] require CG-type solvers to optimize the associated auxiliary variable that is avoided in our proposed algorithm based on the AL framework with variable splitting.

Our formulation combines a circulant blur model with a masking operator to prevent wraparound artifacts. Then we use a specific variable splitting strategy that decouples the circulant blur and the mask. When combined with the AL framework and alternating minimization, our splitting leads to an iterative algorithm with simple update steps that can be implemented non-iteratively in closed-form. We present numerical results that illustrate the improved quality of reconstructed images using a non-circulant model and also the improved convergence speed of our proposed algorithm compared to other state-ofthe-art methods that can be used to tackle the non-circulant reconstruction problem.

II. Problem Formulation

Regularized image restoration can be approached in two main ways. One is the analysis formulation [13], where the objective is to obtain an estimate of the true image $\widehat{\mathrm{x}}$, and the other is the synthesis formulation [4], where the objective is to estimate a set of transform coefficients $\widehat{\mathbf{w}}$ and obtain the reconstructed image through a transform as $\widehat{\mathbf{x}}=\mathbf{W}\widehat{\mathbf{w}}$. In this work, we focus on the analysis formulation. Our approach can be extended to the synthesis one using techniques similar to that proposed in [6].

For image restoration, we consider the following analysis formulation, where the image estimate $\widehat{\mathrm{x}}$ is obtained by minimizing a cost function

$$\widehat{\mathbf{x}}=\underset{\mathbf{x}}{\text{argmin}}\left\{\Psi \left(\mathbf{x}\right)\triangleq J\left(\mathbf{x}\right)+\lambda \Phi \left(\mathbf{Rx}\right)\right\},$$ (1)

J(·) is the data fidelity term, Φ(·) is a regularizer function, R is a sparsifying transform (e.g., wavelet frames or finite differences) and λ is the regularization parameter.

A. Data Fidelity

To design a restoration algorithm one must make some modeling assumptions. Often deblurring algorithms are developed assuming a circulant blur model represented by $\check{\mathbf{A}}$ and the data fidelity term is modeled as [3]–[10]:

$$J\left(\check{\mathbf{x}}\right)\triangleq {\Vert \mathbf{y}-\check{\mathbf{A}}\check{\mathbf{x}}\Vert }_2^2$$ (2)

where y is the observed (M × 1) vectorized blurred and noisy image, $\check{\mathbf{x}}$ is the vectorized (M × 1) image to be reconstructed, and $\check{\mathbf{A}}$ is an (M × M) circulant blurring matrix. Even though this model is very popular in the image restoration literature [3]–[10], it is inaccurate since the assumption of circulant blur rarely, if ever, applies in practice.

Reconstructing images under the assumption of a purely circulant model can lead to severe artifacts due to discontinuities at the boundaries caused by the periodic extension of the image [11]. Simple approaches like zero-padding or replicated boundary extension do not resolve this issue since they do not eliminate the discontinuity at the boundaries. As suggested in [11] data pre-processing techniques like replicated boundary extension combined with edge-tapering can reduce, but not completely eliminate the artifacts. This will be illustrated in the experiments section (§IV). A typical data preprocessing approach yields an augmented (N × 1, with N > M)¹ data vector

$$\tilde{\mathbf{y}}\triangleq \text{edgetaper}\left\{\text{replicate}\right(\mathbf{y}\left)\right\},$$ (3)

where replicate corresponds to replicated boundary extension, e.g., for 1D y[k] = y[n] for k > n (implemented using Matlab's padarray function with the `replicate' option), and edgetaper corresponds to edge-tapering, i.e., blurring the boundaries of the image using the known PSF to eliminate discontinuities caused by periodic extension [11, §I] (implemented using Matlab's edgetaper function). A corresponding data fidelity term is given by:

$$J\left(\mathbf{x}\right)\triangleq {\Vert \tilde{\mathbf{y}}-\mathbf{Ax}\Vert }_2^2,$$ (4)

where x is the (N × 1) image to be reconstructed, and A is an (N × N) circulant blurring matrix.

Instead of the circulant model (which uses periodic boundary conditions), an alternative is to use reflexive boundary conditions along with DCT, as suggested in [14]. In this case the blurring matrix has the Toeplitz-plus-Hankel form and can be diagonalized using the DCT. A corresponding data fidelity term is given by:

$$J\left(\check{\mathbf{x}}\right)\triangleq {\Vert \mathbf{y}-{\mathbf{A}}_R\check{\mathbf{x}}\Vert }_2^2,$$ (5)

where $\check{\mathbf{x}}$ is the (M × 1) image to be reconstructed, and A_R is an (M × M) Toeplitz-plus-Hankel blurring matrix. This DCT method requires the PSF to be symmetric [11], [14], which may not hold for some applications e.g., motion blur. In cases of non-symmetric PSFs this method can only be used to find preconditioners that facilitate iterative solvers like CG [14].

To eliminate boundary artifacts, the methods in [11], [12] use a data fidelity term

$$J\left(\mathbf{x}\right)\triangleq {\Vert \mathbf{y}-\tilde{\mathbf{A}}\mathbf{x}\Vert }_2^2,$$ (6)

with an (M × N) non-circulant blurring matrix $\tilde{\mathbf{A}}$. Then, the non-circulant matrix $\tilde{\mathbf{A}}$ is augmented with additional rows (using a low dimensional auxiliary matrix a) to create a circulant (N × N) system matrix A. Instead of tackling the non-circulant Hessian ${\tilde{\mathbf{A}}}^{\prime }\tilde{\mathbf{A}}$ directly, it is rewritten as:

$${\tilde{\mathbf{A}}}^{\prime }\tilde{\mathbf{A}}={\mathbf{A}}^{\prime }\mathbf{A}-{\mathbf{a}}^{\prime }\mathbf{a},$$ (7)

consisting of a block-circulant (with circulant blocks) Hessian A′A and a low rank component a′a. Inverting the non-circulant Hessian ${\tilde{\mathbf{A}}}^{\prime }\tilde{\mathbf{A}}$ (along with a suitable block-circulant regularization matrix) is then carried out using the Sherman–Morrison Matrix Inversion Lemma (MIL) [11, Eq. (5)–(7)]. The MIL separates the two terms in the RHS of (7) and involves inverting the circulant part (i.e., A′A with the regularization term) and a low-dimensional matrix involving the auxiliary matrix a. The circulant component is inverted using FFTs and inversion of the low dimensional component can be performed with an iterative algorithm (e.g., CG).

In this work, similarly to [11], [12], we consider a more realistic non-circulant model, that is shift-invariant within the region of interest, but avoids the assumption of periodic end conditions. However, instead of using the manipulation in (7), we introduce a masking operator that eliminates the wraparound artifacts at the boundaries caused by periodic convolution. With our proposed model the data fidelity term becomes

$$J\left(\mathbf{x}\right)\triangleq {\Vert \mathbf{y}-\mathbf{TAx}\Vert }_2^2,$$ (8)

where T is an (M × N) masking matrix that truncates the circular wraparound at the boundaries, A is an (N × N) circulant matrix, and x is an (N × 1) vector as in (4) and (6)². The matrix T is a truncated identity matrix resulting from the removal of rows corresponding to boundary pixels, and thus, T′T is an (N × N) diagonal matrix with 0s and 1s. Even though the model in (8) is shift-invariant within the image, the masking operator makes the overall system model TA shift-variant. To efficiently handle this shift-variant model, we propose to use a variable splitting scheme that decouples T and A and in turn allows the use of FFT-based computations as described in §III-B.

B. Regularizer

The algorithms discussed in this paper can tackle a general class of convex regularizers Φ(·) in (1), but we will focus on edge-preserving and sparsity promoting regularizers like Total Variation (TV) and analysis l₁ regularization, respectively, for brevity.

• 1)
  l₁ regularization or discrete anisotropic total-variation:

  $$\Phi \left(\mathbf{Rx}\right)={\Vert \mathbf{Rx}\Vert }_1,$$ (9)

  where R = W is a wavelet frame, typically excluding the approximation level, or $\mathbf{R}=\mathbf{C}\triangleq {\left[{\mathbf{C}}_1^{\prime }{\mathbf{C}}_2^{\prime }\right]}^{\prime }$ is a matrix of horizontal and vertical finite differences.
• 2)
  Discrete isotropic total-variation

  $$\Phi \left(\mathbf{Cx}\right)=\sum _{n=1}^N\sqrt{{\mid {\left[{\mathbf{C}}_1\mathbf{x}\right]}_n\mid }^2+{\mid {\left[{\mathbf{C}}_2\mathbf{x}\right]}_n\mid }^2}.$$ (10)

To reduce computational complexity and allow FFT-based non-iterative solving of linear systems, the sparsifying matrix R (wavelet frame or finite differences) is assumed to have periodic end conditions such that R′R is circulant.

III. Proposed Model-Based Restoration

In this work, our goal is to estimate the image x by minimizing the following cost function that we call problem P1:

$$\widehat{\mathbf{x}}=\underset{\mathbf{x}}{\text{argmin}}\left\{\Psi \left(\mathbf{x}\right)\triangleq \frac{1}{2}{\Vert \mathbf{y}-\mathbf{TAx}\Vert }_2^2+\lambda \Phi \left(\mathbf{Rx}\right)\right\}.$$ (11)

Minimizing (11) is a non-trivial optimization problem. Existing methods include non-linear conjugate gradient (NCG) [15], iterative shrinkage/thresholding (ISTA) [4], [16], (M)FISTA [8], [17], and variable splitting/ADMM (Alternating Direction Method of Multipliers) algorithms [6], [10], [18]–[20]. Some of these methods, e.g., (M)FISTA, SALSA [6], Split-Bregman (SB) [20], and FTVd [18] are computationally efficient when used with the data model in (2) since they exploit the circulant nature of A in (2). However, a straightforward application of these methods to the non-circulant model in (8) may increase their computation time since the inner sub-probems of these algorithms may no longer admit explicit closed-form updates. We discuss this in detail in §III-A and also provide experimental evidence in §IV-B. To handle the non-circulant model in (8) we propose a specific variable splitting strategy [6], [21] presented in §III-B. We then use the AL framework and alternating minimization to obtain an iterative algorithm that exploits the structures of T and A and thereby, converges faster.

A. Existing Restoration Algorithms

1) Non-linear Conjugate Gradient (NCG)

Using NCG to solve P1 requires computing the gradient of Ψ(x). This is problematic for TV or l₁ norm regularizers that use the non-smooth absolute value function. To alleviate this problem the common approach is to use a rounding parameter to approximate the absolute value function as

$$\mid x\mid =\sqrt{{\mid x\mid }^2+\epsilon ,}$$ (12)

where ε is the rounding parameter. NCG also needs a line search method for which we can use the technique proposed in [15, Sec. IV].

The disadvantages of NCG are slow convergence, and perhaps, the fact that it does not converge to a solution of P1, due to the approximation in (12).

2) (MF)ISTA

ISTA was first introduced for restoration problems with synthesis-type priors [4], [16] and it was later generalized to analysis-type priors [8], [17]. (M)FISTA is an improvement on ISTA that converges faster to a solution of P1. The general methodology of both ISTA and (M)FISTA is to convert the original problem P1 into a denoising problem that can be solved non-iteratively for l₁-synthesis priors or iteratively for analysis priors. The difference between ISTA and (M)FISTA is the use of a specific two-step update in (M)FISTA [8, Eq. (4.1)–(4.3)] that accelerates convergence [8, Thm. 4.4]. For analysis regularizers, the denoising step cannot be performed in a single-step, in which case one can use a Chambolle-type algorithm as in [7, Eq. (5)–(7)].

Even though MFISTA converges faster than the simpler ISTA, variable-splitting/ADMM algorithms [6], [10], [21] have been developed that exhibit faster convergence.

3) Variable-Splitting/ADMM Algorithms [6], [18], [20]

The main idea of these algorithms is to break down the original problem P1 into smaller tasks by introducing appropriate auxiliary constraint variables. The resulting minimization sub-problems are decoupled and easier to solve compared to the original minimization problem P1. There are several different ways to choose the splitting variables that lead to a variety of such variable-splitting based algorithms [6], [10], [21]. These algorithms have been shown to converge faster than MFISTA [6]. The algorithm we developed for the non-circulant image restoration problem is based on the variable-splitting ADMM framework presented in [10], [21], [22].

a) SALSA [6]

One approach for solving P1 is to split the regularization term by introducing an auxiliary variable u = x. The constrained problem is formulated as:

$$\underset{\mathbf{x},\mathbf{u}}{\text{min}}\left\{\Psi (\mathbf{x},\mathbf{u})\triangleq \frac{1}{2}{\Vert \mathbf{y}-\mathbf{TAx}\Vert }_2^2+\lambda \Phi \left(\mathbf{Ru}\right)\right\}\text{s.t.}\mathbf{u}=\mathbf{x},$$ (13)

and the associated AL function is

$$\mathcal{L}(\mathbf{x},\mathbf{u},\mu ,\eta )=\Psi (\mathbf{x},\mathbf{u})+\frac{\mu }{2}{\Vert \mathbf{u}-\mathbf{x}-\eta \Vert }_2^2,$$ (14)

where η is linearly related to the Lagrange multiplier for the constraint in (13) and μ > 0 is an AL penalty parameter [6], [10], [21], [22].

This formulation leads to SALSA [6], where (13) is solved with the following alternating minimization scheme:

$${\mathbf{x}}^{(k+1)}=\underset{\mathbf{x}}{\text{argmin}}\left\{\frac{1}{2}{\Vert \mathbf{y}-\mathbf{TAx}\Vert }_2^2+\frac{\mu }{2}{\Vert {\mathbf{u}}^{\left(k\right)}-\mathbf{x}-{\eta }^{\left(k\right)}\Vert }_2^2\right\}$$ (15)

$${\mathbf{u}}^{(k+1)}=\underset{\mathbf{u}}{\text{argmin}}\left\{\lambda \Phi \left(\mathbf{Ru}\right)+\frac{\mu }{2}{\Vert \mathbf{u}-{\mathbf{x}}^{(k+1)}-{\eta }^{\left(k\right)}\Vert }_2^2\right\}$$ (16)

$${\eta }^{(k+1)}={\eta }^{\left(k\right)}-{\mathbf{u}}^{(k+1)}+{\mathbf{x}}^{(k+1)}.$$ (17)

Since the cost function (15) is quadratic, the minimization with respect to x can be expressed in closed form as:

$${\mathbf{x}}^{(k+1)}={\mathbf{H}}_{\mu ,1}^{-1}[{\mathbf{A}}^{\prime }{\mathbf{T}}^{\prime }\mathbf{y}+\mu ({\mathbf{u}}^{(k+1)}-{\eta }_1^{\left(k\right)}\left)\right],$$ (18)

where

$${\mathbf{H}}_{\mu ,1}\triangleq {\mathbf{A}}^{\prime }{\mathbf{T}}^{\prime }\mathbf{T}\mathbf{A}+\mu \mathbf{I}.$$ (19)

For the purely circulant model in (2), T = I and one can invert H_μ,1 using FFTs. However, for the more realistic model in (8), the Hessian H_μ,1 has no exploitable structure and finding a minimizer non-iteratively can be computationally intensive for large N. Instead, one can apply a few PCG iterations using the circulant preconditioner M₁ ≜ A′ A + μI⁻¹ along with warm starting find an approximate solution.

The minimization with respect to u in (16) can be implemented non-iteratively for certain synthesis-type regularizers. However, for analysis-type regularizers (e.g., involving finite differences or wavelet frames), it has to be performed iteratively. Following the implementation of SALSA [6], we used the Chambolle-type algorithm [7, Eq. (6)–(7)] for the update of u. Despite the approximate update steps, SALSA can be shown to converge to a solution of P1 [6], [23].

b) Split-Bregman Algorithm [20]

An alternative approach is to split the regularization term by introducing the auxiliary variable v = Rx. In this case the constrained problem is formulated as

$$\underset{\mathbf{x},\mathbf{v}}{\text{min}}\left\{\Psi (\mathbf{x},\mathbf{v})\triangleq \frac{1}{2}{\Vert \mathbf{y}-\mathbf{TAx}\Vert }_2^2+\lambda \Phi \left(\mathbf{v}\right)\right\}\text{s.t.}\mathbf{v}=\mathbf{Rx},$$ (20)

and the associated AL function is

$$\mathcal{L}(\mathbf{x},\mathbf{v},\mu ,\eta )=\Psi (\mathbf{x},\mathbf{v})+\frac{\mu }{2}{\Vert \mathbf{v}-\mathbf{Rx}-\eta \Vert }_2^2,$$ (21)

where η is related to the Lagrange multiplier for the constraint in (20).

The splitting in (20) is similar to that in the SB [20] and FTVd [18] algorithms, and the problem is again solved using the AL framework and by performing the following minimizations alternatively:

$${\mathbf{x}}^{(k+1)}=\underset{\mathbf{x}}{\text{argmin}}\left\{\frac{1}{2}{\Vert \mathbf{y}-\mathbf{TAx}\Vert }_2^2+\frac{\mu }{2}{\Vert {\mathbf{v}}^{\left(k\right)}-\mathbf{Rx}-{\eta }^{\left(k\right)}\Vert }_2^2\right\}$$ (22)

$${\mathbf{v}}^{(k+1)}=\underset{\mathbf{v}}{\text{argmin}}\left\{\lambda \Phi \left(\mathbf{v}\right)+\frac{\mu }{2}{\Vert \mathbf{v}-{\mathbf{Rx}}^{(k+1)}-{\eta }^{\left(k\right)}\Vert }_2^2\right\}$$ (23)

$${\eta }^{(k+1)}={\eta }^{\left(k\right)}-{\mathbf{v}}^{(k+1)}+{\mathbf{Rx}}^{(k+1)}.$$ (24)

As in SALSA, the minimization with respect to x in (22) has a closed-form solution since the associated cost function is quadratic:

$${\mathbf{x}}^{(k+1)}={\mathbf{H}}_{\mu ,2}^{-1}\left[{\mathbf{A}}^{\prime }{\mathbf{T}}^{\prime }\mathbf{y}+\mu {\mathbf{R}}^{\prime }\left({\mathbf{v}}^{(k+1)}-{\eta }^{\left(k\right)}\right)\right],$$ (25)

where

$${\mathbf{H}}_{\mu ,2}\triangleq {\mathbf{A}}^{\prime }{\mathbf{T}}^{\prime }\mathbf{TA}+\mu {\mathbf{R}}^{\prime }\mathbf{R}.$$ (26)

Again in this case, for the model in (8), the Hessian matrix H_μ,2 has no exploitable structure and solving (25) can be computationally intensive for large N. As in SALSA, one can apply a few PCG iterations with warm starting and a circulant preconditioner M₂ ≜ (A′ A + μR′R)⁻¹

The minimization with respect to v in (23) can be performed non-iteratively for several synthesis- and analysis-type regularizers, including TV and ι₁ norm of wavelet coefficients, using a soft thresholding/shrinkage rule:

$${\mathbf{v}}^{(k+1)}=\text{shrink}\left\{{\mathbf{Rx}}^{\left(k\right)}+{\eta }^{\left(k\right)},\frac{\lambda }{\mu }\right\}.$$ (27)

For the bilateral TV and ι₁ regularization the shrinkage operator is element-wise, whereas for isotropic TV a vector shrinkage rule can be used, as in [18, §2.1].

A reviewer suggested a variation of the SB algorithm, formulated by using the method in [11] for the updates of x in (22). Although it is more complicated than applying CG to (25), this formulation has the benefit of solving a much smaller problem using CG iterations only for the boundaries and updates the rest of the image with the use of FFTs. The method is explained in detail in [11] and also in the alternating minimization scheme of [12]. For this method one could apply CG iterations for the boundaries only at the first few outer iterations of the algorithm to reduce computation time. Such a scheme would not lead to a solution of P1 in (11), but can yield images with improved quality compared to any pre-processing method. Depending on the noise level and type of blur the restored image could even be practically indistinguishable from the actual solution of P1. However, there is no way to know in advance for how many steps should the CG sub-iterations be applied to achieve restoration with sufficiently reduced artifacts. For the purpose of convergence speed comparisons we will refer to this variation as the Split-Bregman-MIL (SB-MIL) algorithm. This name was used since the method uses the Matrix Inversion Lemma (MIL) to decouple the circulant part from the boundaries and efficiently perform the update for x.

Compared to SALSA, the benefit of SB and SB-MIL is the single-step update for v that avoids the inner iterations of the Chambolle-type [7, Eq. (6)–(7)] algorithm required in SALSA.

B. Proposed Algorithm: ADMM-P2

Even though both SALSA and SB-based algorithms (e.g., SB [20] and SB-MIL studied here) decouple the regularization term, their main caveat is the need for (P)CG iterations to obtain the update of x ((18) and (25)) or a low-dimension vector corresponding to the boundaries in SB-MIL. These inner iterations can increase convergence time. Our method alleviates this problem by introducing a second auxiliary variable u₀ = Ax. This additional splitting is similar to those used in [21, Sec. IV–B] for MRI, [22, Sec. III] for CT, and [10] for image restoration, although in [10] the goal is to separate the entire data-fidelity term from the data model, whereas in our case the additional splitting u₀ separates the blur and the masking operator inside the data-fidelity term in (11). The resulting constrained problem P2 is given by:

$$\begin{array}{cc}\hfill & \underset{\mathbf{x},{\mathbf{u}}_0,{\mathbf{u}}_1}{\text{min}}\left\{\Psi ({\mathbf{u}}_0,{\mathbf{u}}_1)\triangleq \frac{1}{2}{\Vert \mathbf{y}-{\mathbf{Tu}}_0\Vert }_2^2+\lambda \Phi \left({\mathbf{u}}_1\right)\right\}\hfill \\ \hfill & \text{s}.\text{t}.{\mathbf{u}}_0=\mathbf{Ax}\text{and}{\mathbf{u}}_1=\mathbf{Rx}.\hfill \end{array}$$ (28)

where A is the N × N circulant blurring matrix and R is an R × N sparsifying transform matrix. We can now rewrite P2 in a more concise form as follows:

$$\underset{\mathbf{x},\mathbf{w}}{\text{min}}\Psi \left(\mathbf{w}\right)\text{s}.\text{t}.\mathbf{w}=\mathbf{Dx}$$ (29)

where

$$\mathbf{x}\triangleq \left[\begin{array}{c}\hfill {\mathbf{u}}_0\hfill \\ \hfill {\mathbf{u}}_1\hfill \end{array}\right],\mathbf{D}\triangleq \left[\begin{array}{c}\hfill \mathbf{A}\hfill \\ \hfill \mathbf{R}\hfill \end{array}\right].$$ (30)

From this formulation it is evident that the constraint matrix D is full column rank provided that the null spaces of A′ A and R′R intersect only trivially, which is usually true in image restoration problems (low-pass nature of A and high-pass nature of R). This requirement does not impose serious restrictions on R for deblurring problems, i.e., it is not necessary for R to be full column rank, which can be the case with commonly used sparsifying transforms, e.g., finite differences, wavelets (excluding the approximation level). The importance of D being full column rank relates to the hypothesis of a theorem due to Eckstein and Bertsekas [23, Thm. 8], summarized below, that guarantees convergence of ADMM type algorithms.

To solve the problem P2 with the method of multipliers we construct the Augmented Lagrangian (AL) function, which can be expressed as:

$$\mathcal{L}(\mathbf{x},\mathbf{w},\gamma )=\Psi \left(\mathbf{w}\right)+\frac{\mu }{2}{\Vert \mathbf{w}-\mathbf{Dx}-\eta \Vert }_{\Lambda }^2,$$ (31)

where μ is the AL penalty parameter, the parameter η relates to the Lagrange multiplier, and Λ is a symmetric positive definite weighting matrix defined as

$$\Lambda \triangleq \left[\begin{array}{cc}\hfill {\mathbf{I}}_N\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill \nu {\mathbf{I}}_R\hfill \end{array}\right],$$ (32)

where ν > 0.

To solve P1 in (28) we apply an alternating minimization scheme, which at the kth iteration, leads to the following updates

$${\mathbf{x}}^{(k+1)}=\underset{\mathbf{x}}{\text{argmin}}{\Vert {\mathbf{W}}^{\left(k\right)}-\mathbf{Dx}-{\eta }^{\left(k\right)}\Vert }_{\Lambda }^2$$ (33)

$${\mathbf{w}}^{(k+1)}=\underset{\mathbf{w}}{\text{argmin}}\Psi \left(\mathbf{w}\right)+\frac{\mu }{2}{\Vert \mathbf{W}-{\mathbf{Dx}}^{(k+1)}-{\eta }^{\left(k\right)}\Vert }_{\Lambda }^2$$ (34)

$${\eta }^{(k+1)}={\eta }^{\left(k\right)}-{\mathbf{w}}^{(k+1)}+{\mathbf{Dx}}^{(k+1)}.$$ (35)

Although the theorem [23, Thm. 8] allows for inexact updates i.e., ${\epsilon }_{\mathbf{x}}^k\triangleq {\Vert {\mathbf{x}}^{\left(k\right)}-{\mathbf{x}}_{\ast }^{\left(k\right)}\Vert }_2>0$ and ${\epsilon }_{\mathbf{w}}^k\triangleq {\Vert {\mathbf{w}}^{\left(k\right)}-{\mathbf{w}}_{\ast }^{\left(k\right)}\Vert }_2>0$ where (${\mathbf{x}}_{\ast }^{\left(k\right)}$ and ${\mathbf{w}}_{\ast }^{\left(k\right)}$) are solutions of (33) and (34) respectively, our formulation leads to closed-form updates for (33)–(34), that can be implemented non-iteratively, i.e., ${\epsilon }_{\mathbf{x}}^k={\epsilon }_{\mathbf{w}}^k=0$ (up to numerical round-off errors). We review below the theorem of Eckstein and Bertsekas [23, Thm. 8] that is useful for analyzing the convergence of (33)–(35).

Theorem 1

Consider (28) where Ψ(·) is closed, proper and convex and D has full column rank. Let ${\eta }^{\left(0\right)}\in {\mathcal{R}}^{N+R}$, μ > 0, ${\Sigma }_k{\epsilon }_{\mathbf{x}}^k<\infty$, and ${\Sigma }_k{\epsilon }_{\mathbf{w}}^k<\infty$. If (28) has a solution (x*, w*) then the sequence of updates {(x^(k),w^(k))}_k generated by (33)–(35) converges to(x*,w*). If (28) has no solution then at least on of the sequence {(x^(k),w^(k))}_k or {η^(k)}_k diverges.

Even though the theorem of Eckstein and Bertsekas [23, Thm. 8] uses an AL function with Λ = I, we can still apply the theorem to (31) through a simple variable transformation [22, Footnote 3].

Using the structure of Ψ(w) and D, we can reformulate the AL function from (31) as

$$\mathcal{L}(\mathbf{x},\mathbf{u},\mu ,\nu ,\eta )=\Psi ({\mathbf{u}}_0,{\mathbf{u}}_1)+\frac{\mu }{2}{\Vert {\mathbf{u}}_0-\mathbf{Ax}-{\eta }_0\Vert }_2^2+\frac{\mu \nu }{2}{\Vert {\mathbf{u}}_1-\mathbf{Rx}-{\eta }_1\Vert }_2^2,$$ (36)

where η₀ and η₁ are related to the Lagrange multipliers for the constraints in (28). Using this form of the AL function, the alternating minimization scheme at the kth step can be expressed as:

$${\mathbf{u}}_0^{(k+1)}=\underset{{\mathbf{u}}_0}{\text{argmin}}\left\{\frac{1}{2}{\Vert \mathbf{y}-{\mathbf{Tu}}_0\Vert }_2^2+\frac{\mu }{2}{\Vert {\mathbf{u}}_0-{\mathbf{Ax}}^{\left(k\right)}-{\eta }_0^{\left(k\right)}\Vert }_2^2\right\}$$ (37)

$${\mathbf{u}}_1^{(k+1)}=\underset{{\mathbf{u}}_1}{\text{argmin}}\left\{\lambda \Phi \left({\mathbf{u}}_1\right)+\frac{\mu \nu }{2}{\Vert {\mathbf{u}}_1-{\mathbf{Rx}}^{\left(k\right)}-{\eta }_1^{\left(k\right)}\Vert }_2^2\right\}$$ (38)

$${\mathbf{x}}^{(k+1)}=\underset{\mathbf{x}}{\text{argmin}}\left\{\frac{\mu }{2}{\Vert {\mathbf{u}}_0^{(k+1)}-\mathbf{Ax}-{\eta }_0^{\left(k\right)}\Vert }_2^2+\frac{\mu \nu }{2}{\Vert {\mathbf{u}}_1^{(k+1)}-\mathbf{Rx}-{\eta }_1^{\left(k\right)}\Vert }_2^2\right\}$$ (39)

$${\eta }_0^{(k+1)}={\eta }_0^{\left(k\right)}-{\mathbf{u}}_0^{(k+1)}+{\mathbf{Ax}}^{(k+1)}$$ (40)

$${\eta }_1^{(k+1)}={\eta }_1^{\left(k\right)}-{\mathbf{u}}_1^{(k+1)}+{\mathbf{Rx}}^{(k+1)}.$$ (41)

The minimization with respect to u₀ in (37) is straightforward since the corresponding cost function is quadratic. The closed-form solution is expressed as

$${\mathbf{u}}_0^{(k+1)}={({\mathbf{T}}^{\prime }\mathbf{T}+\mu {\mathbf{I}}_N)}^{-1}\left[{\mathbf{T}}^{\prime }\mathbf{y}+\mu \left({\mathbf{Ax}}^{\left(k\right)}+{\eta }_0^{\left(k\right)}\right)\right],$$ (42)

and can be easily computed in a single-step update since it only requires inversion of the diagonal matrix (T′T + μI_N).

The minimization with respect to u₁ in (38) can be performed non-iteratively using a (vector) soft thresholding/shrinkage rule similar to (27), using a threshold value $\frac{\lambda }{\mu \nu }$ instead of $\frac{\lambda }{\mu }$ used in (27).

Finally the minimization with respect to x in (39) is also straightforward due to the corresponding quadratic cost function. The closed form of the update is

$${\mathbf{x}}^{(k+1)}={\mathbf{H}}_{\nu }^{-1}\left[{\mathbf{A}}^{\prime }\left({\mathbf{u}}_0^{(k+1)}-{\eta }_0^{\left(k\right)}\right)+\nu {\mathbf{R}}^{\prime }\left({\mathbf{u}}_1^{(k+1)}-{\eta }_1^{\left(k\right)}\right)\right],$$ (43)

where

$${\mathbf{H}}_{\nu }\triangleq {\mathbf{A}}^{\prime }\mathbf{A}+\nu {\mathbf{R}}^{\prime }\mathbf{R}.$$ (44)

We perform multiplication by ${\mathbf{H}}_{\nu }^{-1}$ in (43) efficiently using FFTs since the blurring matrix A is circulant and the sparsifying operator R (frame or differencing matrix) consists of circulant blocks (periodic end conditions) such that R′R is circulant. Thus, the update of x can be computed non-iteratively.

Combining the above expressions for the updates, we now summarize the proposed AL algorithm for solving P2.

ADMM-P2: AL algorithm for P2.

1: Select x(0), ν > 0, and μ > 0

2: Precompute T′y

3: Set , and k = 0

4: repeat

5:  Obtain using (42)

6:  Obtain using (27)

7:  Obtain x(k + 1) using (43)

8:

9:

10:  k = k + 1

11: until stop criterion is met

Unlike SB-based algorithms (including SB-MIL) and SALSA for solving (11), all the steps of ADMM-P2 are single-step updates due to the extra splitting that decouples the circulant blur operator A from the masking operator T. The experiments in §IV-B show that the non-iterative updates of ADMM-P2 help achieve convergence in less time than other methods.

C. AL Parameter Selection

SALSA and ADMM-P2 belong to the general class of ADMM algorithms. Thus, the ADMM convergence theorem of Eckstein and Bertsekas [23, Thm. 8] applies in these cases and the algorithms are guaranteed to converge to a solution of the original problem P1, even when the inner minimization steps are not performed exactly. The SB-based algorithms are also convergent, although the convergence theory is different from that of ADMM [19], [20].

SALSA, ADMM-P2, SB, and SB-MIL are guaranteed to converge regardless of the choice of AL parameters μ and ν [23]. However, the choice of AL parameters affects the convergence speed of these algorithms. We found experimentally that the best AL parameters for fast convergence depend on the regularization parameter λ and the maximum intensity (x_max) of the blurred image, similarly to the choice proposed in [9]. The best parameters can also depend on the spread of eigenvalues of A′A and R′R

For SALSA we found that a choice of μ = 2⁷λ/x_max works well for several different choices of regularization parameter λ.

For SB and SB-MIL, we found experimentally that the choice of the AL parameter μ also depends on μ_min ≜ argmin_μ k(A′A + μR′R), where k(·) is the condition number. As a rule of thumb we choose the AL parameter as μ = 2⁸λμ_min/x_max.

For ADMM-P2, we found experimentally that the best product μν depends on λ, x_max and ν_min ≜ argmin_ν k(A′A + νR′R). Thus, for fast convergence we choose the AL parameters to achieve μν = 2⁸λν_min/x_max. Also, we found that we can fix the parameter μ to a value μ₀ = 2⁻⁴ and then choose the parameter ν as: ν = 2⁸λν_min/μ₀x_max.

The above rules for the parameter selection work well when the blurring system matrix A is scaled such that the maximum eigenvalue of A′A is equal to 1.

IV. Experiments

The first part of the experiments compares the quality of the reconstructed images obtained using the proposed non-circulant (8) and the purely circulant (2) reconstruction models. The second part compares the convergence speed of the proposed ADMM-P2 algorithm to that of NCG, ISTA, MFISTA, SALSA, SB, and SB-MIL for restoration using the proposed non-circulant model.

A. Non-Circulant Versus Circulant Restoration

For all our experiments we used the 256×256 cameraman image as the true, noise-free image, and following the experimental setting in [8] we scaled it to the range[0, 1] (Figure 1a). For the blurring kernel we used a uniform 9×9 blur, a uniform 15×15 blur and a 15 pixel straight motion blur at a 30× angle. For this image and blur sizes the valid part of the convolution is the 248×248 (for 9×9 blur) or 242×242 (for 15×15 blur) central region and the data simulated with Matlab's conv2 function using the `valid' option are of this size.

Fig. 1.

Experiment 4 using 15 × 15 uniform blur: Restoration results using isotropic TV regularization. (a) Cameraman true image (242 × 242), scaled to the range[0, 1]. (b) Blurred and noisy image from Matlab's `valid' conv2 operation. (c) Restored image from purely circulant model (2). (d) Restored image from purely circulant model with data pre-processing (4). (e) Restored image using reflexive boundary conditions and DCT (5). (f) Restored image from proposed non-circulant model (8).

To assess the quality of our proposed non-circulant reconstruction method we compared it to reconstructions from

• a purely circulant model (2) without data pre-processing,

• a purely circulant model (2) where we used data pre-processing with boundary replication combined with edge tapering,

• and reflexive end conditions and DCT (5).

When using the non-circulant model in (8) or the purely circulant model with data pre-processing in (4), the reconstructed images have original 256×256 size and then they are truncated to match the data size. When reconstructing with the purely circulant model without pre-processing in (2) or with the DCT method in (5), the reconstructed image has the same size as the data, so no truncation is necessary.

For our experiments, we used three types of regularizers, namely isotropic and anisotropic Total Variation, and l₁ norm of the coefficients of the undecimated 2-level Haar wavelet transform, excluding the approximation level.

We performed three sets of experiments with different levels of Blurred Signal-to-Noise Ratio (BSNR ≜ 10 log₁₀var(Ax) /σ²)) [3] and we chose the regularization parameter λ to achieve low Normalized Root Mean Square (NRMS) error for a given BSNR level. The experimental setup is as follows

• 1)20 dB BSNR (σ² = 5.03 × 10⁻⁴) with λ = 2⁻¹⁰,
• 2)30 dB BSNR (σ² = 5.03 × 10⁻⁵) with λ = 2⁻¹²,
• 3)40 dB BSNR (σ² = 5.03 × 10⁻⁶) with λ = 2⁻¹⁵,
• 4)50 dB BSNR (σ² = 5.03 × 10⁻⁷) with λ = 2⁻¹⁷,

where σ² is the Gaussian noise variance. We quantified the reconstruction quality by computing the Improvement in Signal-to-Noise Ratio (ISNR) between the restored image $\widehat{\mathbf{x}}$ and the true image x_true. The ISNR was measured in the 248 × 248 or 242 × 242 (depending on the blur size) central region that corresponds to the valid part of the convolution, since this was the actual size of the input data vector y. Any extrapolated values (when reconstructing with the non-circulant model) are not used in the ISNR calculations and not shown in any of the figures.

Figure 1 shows the restored images from experiment 4 (50dB BSNR) with uniform 15 × 15 blur using isotropic TV regularization. Fig. 1c shows that the restored image using the purely circulant model in (2) exhibits severe ringing artifacts, similar to those in the results of [11], that are not confined to the boundaries of the image. When data pre-processing is used, the ringing artifacts are significantly reduced, but not completely removed (Fig. 1d). These artifacts can be more severe and lead to more degradation when the strength of the regularizer decreases as seen in the quantitative results in Table I. When reflexive boundary conditions and the DCT are used, the artifacts are again significantly reduced but clearly visible in high BSNR cases. Finally, we see that the reconstructed images from the proposed non-circulant model in (8) are free of any ringing artifacts and the reconstructed image closely resembles the true noise-free image (Fig. 1f).

TABLE I.

ISNR in dB for all experiments.

Blur Type	9 × 9 Unif.				15 × 15 Unif.				15 Pix. Mot.
Reg. Type	Non-Circ.	Circ.	Circ*	Refl.	Non-Circ.	Circ.	Circ*	Refl.	Non-Circ.	Circ.	Circ*	Refl.
Experiment 1: BSNR = 20 dB
Wavelet l1	3.7	−0.9	3.6	3.6	3.4	−2.7	3.3	3.4	4.1	−1.3	4.1	N/A
TVaniso	3.5	−1.2	3.4	3.4	3.7	−2.9	3.6	3.7	4.3	−1.5	4.3	N/A
TViso	3.6	−1.4	3.6	3.4	4.0	−3.1	3.9	3.7	4.5	−1.7	4.4	N/A
Experiment 2: BSNR = 30 dB
Wavelet l1	5.7	−3.3	5.6	5.7	4.9	−5.3	4.7	4.8	7.1	−4.4	6.7	N/A
TVaniso	5.4	−3.5	5.4	5.4	5.1	−5.6	4.8	5.1	7.1	−4.6	6.7	N/A
TViso	5.8	−3.7	5.7	5.4	5.4	−5.8	5.1	5.1	7.4	−4.8	6.8	N/A
Experiment 3: BSNR = 40 dB
Wavelet l1	8.6	−8.1	7.9	8.2	7.2	−12.8	6.4	7.1	11.6	−9.2	8.8	N/A
TVaniso	8.4	−8.4	7.5	7.9	7.2	−13.5	6.2	6.9	11.4	−10.2	8.1	N/A
TViso	8.3	−8.6	7.3	7.9	7.4	−13.8	6.3	6.9	11.4	−10.6	7.7	N/A
Experiment 4: BSNR = 50 dB
Wavelet l1	11.8	−10.5	9.0	10.6	9.8	−15.0	6.4	9.3	15.7	−13.2	8.1	N/A
TVaniso	11.5	−10.4	8.3	10.1	9.7	−15.2	3.1	9.3	15.0	−13.5	6.6	N/A
TViso	11.6	−10.6	8.2	10.1	9.9	−15.3	2.4	9.3	14.7	−13.8	5.8	N/A

^*The denotes reconstruction using the purely circulant model with data pre-processing (4). For the case of motion blur the results for reflexive end conditions are not presented since the method is not appropriate for non symmetric PSF.

Figure 2 shows the restored images from experiment 2 (30dB BSNR) with 15 pixel straight motion blur using isotropic TV regularization. Fig. 2c shows that the restored image using the purely circulant model in (2) exhibits severe ringing artifacts, similar to those in Fig. 1c. When data pre-processing is used (Fig. 2d) the ringing artifacts are reduced and are less obvious compared to Fig. 1d because of the high regularization parameter for low BSNR. When reflexive boundary conditions and the DCT are used (Fig. 1e) the method fails to produce accurate results since the PSF is non-symmetric [11], [14]. Finally, we see that the reconstructed images from the proposed non-circulant model in (8) are free of any ringing artifacts and the reconstructed image closely resembles the true noise-free image (Fig. 2f).

Fig. 2.

Experiment 3 using 15 pixel motion blur: Restoration results using isotropic TV regularization. (a) Cameraman true image (242 × 242), scaled to the range[0, 1]. (b) Blurred and noisy image from Matlab's `valid' conv2 operation. (c) Restored image from purely circulant model (2). (d) Restored image from purely circulant model with data pre-processing (4). (e) Restored image using reflexive boundary conditions and DCT (5). For this case, the results are inaccurate since this method is not applicable for non-symmetric PSF [11], [14]. Also the image had to be scaled to fit the dynamic range of the rest of the reconstructed images. (f) Restored image from proposed non-circulant model (8).

Figure 3 shows the restored images from experiment 1 (20dB BSNR) with with uniform 9×9 blur using isotropic TV regularization. The restored image using the purely circulant model (Fig. 3c) exhibits severe ringing artifacts as in the previous cases. However, the reconstructed images using data pre-processing (Fig. 3d) and using reflexive boundary conditions and DCT (Fig. 3e) look very similar to the reconstructed images from the non-circulant model (Fig. 3f). This happens because the strong regularizer used to suppress noise over-smooths the image and thus the reconstruction artifacts are smoothed out as well making the images look almost identical.

Fig. 3.

Experiment 1 using 9 × 9 uniform blur: Restoration results using isotropic TV regularization. (a) Cameraman true image (248 × 248), scaled to the range[0, 1]. (b) Blurred and noisy image from Matlab's `valid' conv2 operation. (c) Restored image from purely circulant model (2). (d) Restored image from purely circulant model with data pre-processing (4). (e) Restored image using reflexive boundary conditions and DCT (5). (f) Restored image from proposed non-circulant model (8).

Table I gives the quantitative results in terms ISNR for all experiments. In all cases, the ISNR of reconstructions based on the non-circulant model are higher than that of the reconstructions based on the purely circulant model without data pre-processing. In the case of reconstruction with the circulant model and data preprocessing or with reflexive boundary conditions and DCT, we can see that for lower SNR, where stronger regularization is required, the smoothing of the regularizer can suppress the ringing artifacts leading to reconstruction ISNR comparable to that of the proposed non-circulant model reconstruction. However, in higher SNR regimes, where less smoothing is required, the artifacts from the circulant model reconstruction become more prominent and the non-circulant reconstruction shows significantly improved reconstruction quality.

B. Convergence Speed Comparison

For the convergence speed experiments we used the setting from Experiment 3 (i.e., 40 dB BSNR, σ² = 5.03 × 10⁻⁶ with λ = 2⁻¹⁵) with 9 × 9 uniform blur. We compared the convergence speed of the following algorithms that were all implemented in Matlab.

• NCG-L with L line-search sub-iterations [15];

• ISTA [7];

• MFISTA-M with M sub-iterations of the Chambolle-type algorithm [7, Eq. (6)–(7)];

• SALSA-N-M with N CG sub-iterations and M sub-iterations of [7, Eq. (6)–(7)];

• SB-N with N CG sub-iterations;

• SB-MIL-N with N CG sub-iterations;

• ADMM-P2 (proposed).

For these experiments we focused on algorithms that solve P1 in (11). Thus, methods that use approximations in the data-fit term, i.e., circulant reconstruction (with or without data pre-processing (2) and (4)) and reflexive end conditions with DCT (5), were not included, since they would obviously not converge to a solution of (11). NCG is an exception since the approximation affects the regularization and not the data-fit term.

For NCG we chose the rounding parameter to be ε = 10⁻⁶, which yielded good convergence speed without compromising too much the resulting solution. We also used L = 2, and 5 line-search iterations. For MFISTA, we used M = 1, 4, 10, and 20 iterations of [7, Eq. (6)–(7)]. For SALSA we used N = 1, 4, and 10 CG iterations and M = 1, 4, 10, and 15 iterations of [7, Eq. (6)–(7)]. Finally, for SB and SB-MIL we used N = 1, 4, and 10 CG iterations. The CG-solvers inside SALSA and SB were terminated when ${\Vert {\mathbf{x}}^{(k+1)}\Vert }_2^2{\alpha }^{\left(k\right)}{\Vert {\mathbf{d}}^{\left(k\right)}\Vert }_2^2<\delta$, where x^(k+1) is the new update, d^(k) is the search direction vector, α^(k) is the step size after the kth iteration, and δ is a threshold chosen as δ = 10⁻⁶. For all AL-based algorithms we chose the parameters μ and ν using the rules described in §III-C, with λ = 2⁻¹⁵, x_max = 1, and μ₀ = 2⁻⁴.

All the experiments were conducted on a PC with a dual quad-core 2.6GHz Intel Xeon processor. Table II shows the per-iteration time of each algorithm measured in milliseconds.

TABLE II.

Outer iteration time of competing algorithms measured in milliseconds. L is the number of line-search iterations, M is the number of Chambolle-type iterations and N is the number of CG iterations.

	TV	Wavelet l1 norm
NCG	36 + 7L	95 + 15L
ISTA	34	159
MFISTA	17 + 19M	57 + 135M
SALSA	20 + 19M + 23N	22 + 135M + 24N
SB	39 + 31N	100 + 30N
SB-MIL	40 + 13N	102 + 14N
ADMM-P2	38	90

Since the goal of the restoration problem is to find a solution to the original problem P1, we quantified the speed of convergence as the normalized l₂ distance between the estimate at iteration k (x^(k)), and the limit x^(∞) (that represents a solution of P1) given by

$${\xi }^{\left(k\right)}=10{\text{log}}_{10}\left(\frac{{\Vert {\mathbf{x}}^{\left(k\right)}-{\mathbf{x}}^{\left(\infty \right)}\Vert }_2^2}{{\Vert {\mathbf{x}}^{\left(\infty \right)}\Vert }_2^2}\right).$$

The limit x^(∞) was obtained by running 10⁵ iterations of MFISTA-20, since MFISTA converges to a solution of P1 and also it would not give an unfair advantage to the proposed ADMM algorithm. Since the computational load per-iteration of each algorithm varies, we evaluated ξ^(k) as a function of both iteration number and algorithm run-time. For all algorithms we used x⁽⁰⁾ = A′T′y as the initial guess. Table III shows the time and iteration number required from each algorithm to reach a −50dB error ξ^(k). The threshold of −50dB was chosen as a benchmark since below this point there are practically no visual differences between the reconstructed image x^(k) and the converged image x^(∞).

TABLE III.

Time and iteration number required for every algorithm to reach −50dB error compared to the converged image x^(∞).

	TV		Wavelet l1
	Time	Iter. #	Time	Iter. #
NCG-5	56.1	298	155.6	1897
ISTA	559.1	15000	885.0	8000
MFISTA-1	18.1	355	46.8	323
MFISTA-4	44.1	407	161.8	367
SALSA-4-1	11.7	88	15.4	71
SALSA-4-4	13.6	71	29.6	60
SALSA-4-10	24.3	74	66.2	61
SB-4	12.5	82	21.9	103
SB-MIL-4	4.9	80	10.7	100
ADMM-P2	2.8	107	5.8	109

Figure 4 shows the convergence rate ξ^(k) in terms of iteration and run-time for experiments involving TV and wavelet l₁ norm regularizers. The AL based algorithms (SALSA, SB, SB-MIL and ADMM-P2) converge significantly faster than the rest for all cases.

Fig. 4.

Experiment 3: Algorithm convergence speed results with convergence measured in terms of drop in NRMSE in dB, ξ^(k), between the estimated image $\widehat{\mathbf{x}}$ and the converged image x^(∞). Left column (a,c) is convergence per iteration, and right column (b,d) is convergence in run-time. First row (a,b) is from TV regularizer, and second row (c,d) is from wavelet l₁ norm regularizer.

In terms of convergence rate per iteration, SALSA has a slight advantage over SB, SB-MIL and ADMM-P2 when we use enough inner Chambolle-type iterations. However, since the inner iterations can be computationally expensive, especially for the analysis l₁ regularization with a wavelet frame, its run-time is significantly higher. In terms of run-time, ADMM-P2 is the fastest algorithm with a speed-up of about 2 times compared to the Split-Bregman-MIL algorithm, which is the closest competitor.

V. Discussion

As seen in Figure 1, image restoration with a circulant model can lead to severe image distortion, when applied to data with realistic boundaries. The ringing artifacts due to the discontinuity at the boundaries are not localized and even existing methods for data-preprocessing [11] either cannot fully suppress these artifacts and/or may be applicably only in specific cases like symmetric PSFs.

Our proposed method showed significant improvement of the restored images compared to the standard method (that uses an unrealistic, purely circulant blur model), when applied to data with more realistic boundaries. Our results reinforce the importance of using reconstruction models that do not make any specific assumptions about boundary extension in the data, and also show that the effects of model mismatch at the boundaries (although often ignored) can be severe in terms of image quality degradation. In the case of reflexive boundary conditions the artifacts can be significantly reduced, but the applicability of this method is only limited to symmetric PSFs [11], [14], whereas our more general model based approach can handle efficiently any type of PSF. In addition, the formulation of our model, even though similar to the one used in [11], [12], leads to a more elegant approach of the non-circulant reconstruction problem that does not require any data pre-processing and estimation of the extrapolated image boundaries.

Our proposed ADMM-P2 algorithm, using an additional splitting variable, can efficiently handle the additional complexity introduced by the masking operator without the need for costly CG iterations for the inner sub-problems. The fact that our algorithm uses only non-iterative updates enhances its efficiency even when compared to more sophisticated approaches like SB-MIL. In addition the formulation is more straightforward compared to SB-MIL and allows for easier implementation. The caveat of tuning one additional AL parameter can be easily alleviated by using an empirical method like the one presented in §III-C. Our ADMM-P2 algorithm, being specifically designed for non-circulant deblurring problems, exhibits improved performance compared to existing state-of-the-art methods [6], [17], [20]. Finally, our ADMM-P2 algorithm can be easily extended to three-dimensional problems, where it could be potentially useful in 3D microscopy deconvolution [24], and also accommodate different likelihood models (e.g., Poisson noise) [9].

After finalizing our work, we became aware of a related recent arXiv submission [25].

Biographies

Antonios Matakos (S'04) received the Engineering Diploma in Electrical Engineering from Aristotle University of Thessaloniki, Greece, in 2006 and the M.S.E. also in Electrical Engineering from the University of Michigan, Ann Arbor MI, in 2009. Currently he is a Ph.D. candidate in EE:Systems at the University of Michigan.

His current research interests include statistical image processing and model based, iterative reconstruction methods for MRI.

Sathish Ramani (S'08–M'09) received the M. Sc. degree in physics from the Sri Sathya Sai Institute of Higher Learning, Puttaparthy, India, the M. Sc. degree in electrical engineering from the Indian Institute of Science, Bangalore, India, and the Ph. D. degree in electrical engineering from Ecole Poly-technique Fdrale de Lausanne, Lausanne, Switzerland, in 2000, 2005, and 2009, respectively.

He was a Swiss National Science Foundation Post-Doctoral Fellow with the Electrical Engineering and Computer Science Department, Systems Group, University of Michigan, Ann Arbor, from 2009 to 2010, where he is currently a Research Fellow. His current research interests include splines, interpolation, variational methods in image processing, and optimization algorithms for biomedical imaging applications.

Dr. Ramani was a recipient of a Best Student Paper Award from the IEEE International Conference on Acoustics, Speech, and Signal Processing in 2006.

Jeffrey A. Fessler (F'06) received the BSEE degree from Purdue University in 1985, the MSEE degree from Stanford University in 1986, and the M.S. degree in Statistics from Stanford University in 1989. From 1985 to 1988 he was a National Science Foundation Graduate Fellow at Stanford, where he earned a Ph.D. in electrical engineering in 1990. He has worked at the University of Michigan since then. From 1991 to 1992 he was a Department of Energy Alexander Hollaender Post-Doctoral Fellow in the Division of Nuclear Medicine. From 1993 to 1995 he was an Assistant Professor in Nuclear Medicine and the Bioengineering Program. He is now a Professor in the Departments of Electrical Engineering and Computer Science, Radiology, and Biomedical Engineering. He is a Fellow of the IEEE, for contributions to the theory and practice of image reconstruction. He received the Francois Erbsmann award for his IPMI93 presentation. He serves as an associate editor for IEEE Transactions on Medical Imaging and is a past associate editor for the IEEE Transactions on Image Processing and the IEEE Signal Processing Letters. He was co-chair of the 1997 SPIE conference on Image Reconstruction and Restoration, technical program co-chair of the 2002 IEEE International Symposium on Biomedical Imaging (ISBI), and was general chair of ISBI 2007. He serves as the Chair of the Steering Committee of the IEEE Transactions on Medical Imaging, and as Chair of the ISBI Steering Committee. He served as Associate Chair of his Department from 2006–2008. His research interests are in statistical aspects of imaging problems, and he has supervised doctoral research in PET, SPECT, X-ray CT, MRI, and optical imaging problems.