Directional TGV-Based Image Restoration under Poisson Noise

Abstract

We are interested in the restoration of noisy and blurry images where the texture mainly follows a single direction (i.e., directional images). Problems of this type arise, for example, in microscopy or computed tomography for carbon or glass fibres. In order to deal with these problems, the Directional Total Generalized Variation (DTGV) was developed by Kongskov et al. in 2017 and 2019, in the case of impulse and Gaussian noise. In this article we focus on images corrupted by Poisson noise, extending the DTGV regularization to image restoration models where the data fitting term is the generalized Kullback–Leibler divergence. We also propose a technique for the identification of the main texture direction, which improves upon the techniques used in the aforementioned work about DTGV. We solve the problem by an ADMM algorithm with proven convergence and subproblems that can be solved exactly at a low computational cost. Numerical results on both phantom and real images demonstrate the effectiveness of our approach.

1. Introduction

Poisson noise appears in processes where digital images are obtained by the count of particles (generally photons). This is the case of X-ray computed tomography, positron emission tomography, confocal and fluorescence microscopy and optical/infrared astronomical imaging, to name just a few applications (see, e.g., [1] and the references therein). In this case, the object to be restored can be represented as a vector $\mathbf{u}\in {\mathbb{R}}^n$ and the data can be assumed to be a vector $\mathbf{b}\in {\mathbb{N}}_0^n$, whose entries $b_j$ are sampled from n independent Poisson random variables $B_j$ with probability

$$P(B_j=b_j)=\frac{e^{-{(A\mathbf{u}+\gamma )}_j}{(A\mathbf{u}+\gamma )}_j^{b_j}}{b_j!}.$$

The matrix $A=\left(a_{ij}\right)\in {\mathbb{R}}^{n\times n}$ models the observation mechanism of the imaging system and the following standard assumptions are made:

$$a_{ij}\ge 0 \mathrm{for} \mathrm{all} i,j,\sum _{i=1}^n{a}_{ij}=1 \mathrm{for} \mathrm{all} j.$$ (1)

The vector $\gamma \in {\mathbb{R}}^n$, with $\gamma >0$, models the background radiation detected by the sensors.

By applying a maximum-likelihood approach [1,2], we can estimate $\mathbf{u}$ by minimizing the Kullback–Leibler (KL) divergence of $A\mathbf{u}+\gamma$ from $\mathbf{b}$:

$$D_{KL}(A\mathbf{u}+\gamma ,\mathbf{b})=\sum _{i=1}^n\left(b_{i}ln\frac{b_i}{{[A\mathbf{u}+\gamma ]}_i}+{[A\mathbf{u}+\gamma ]}_i-b_i\right),$$ (2)

where we set

$$b_{i}ln\frac{b_i}{{[A\mathbf{u}+\gamma ]}_i}=0\mathrm{if}{b}_i=0.$$

Regularization is usually introduced in (2) to deal with the ill-conditioning of this problem. The Total Variation (TV) regularization [3] has been widely used in this context, because it preserves edges and is able to smooth flat areas of the image. However, since it may produce staircase artifacts, other TV-based regularizers have been proposed. For example, the Total Generalized Variation (TGV) has been proposed and applied in [4,5,6,7] to overcome the staircasing effect while keeping the ability of identifying edges. On the other hand, to improve the quality of restoration for directional images, the Directional TV (DTV) regularization has been considered in [8], in the discrete setting. In [9,10], a regularizer combining DTV and TGV, named Directional TGV (DTGV), has been successfully applied to directional images affected by impulse and Gaussian noise.

Given an image $\mathbf{u}\in {\mathbb{R}}^n$, the discrete second-order Directional TGV of $\mathbf{u}$ is defined as

$${\mathrm{DTGV}}^2\left(\mathbf{u}\right)=\underset{\mathbf{w}\in {\mathbb{R}}^{2n}}{min}{\alpha }_0{∥\tilde{\nabla }\mathbf{u}-\mathbf{w}∥}_{2,1|{\mathbb{R}}^{2n}}+{\alpha }_1{∥\tilde{\mathcal{E}}\mathbf{w}∥}_{2,1|{\mathbb{R}}^{4n}},$$ (3)

where $\mathbf{w}\in {\mathbb{R}}^{2n}$, $\tilde{\nabla }\in {\mathbb{R}}^{2n\times n}$ and $\tilde{\mathcal{E}}\in {\mathbb{R}}^{4n\times 2n}$ are the discrete directional gradient operator and the directional symmetrized derivative, respectively, and ${\alpha }_0,{\alpha }_1\in (0,+\infty )$. For any vector $\mathbf{v}\in {\mathbb{R}}^{2n}$ we set

$${\parallel \mathbf{v}\parallel }_{2,1|{\mathbb{R}}^{2n}}=\sum _{j=1}^n\sqrt{v_j^2+v_{n+j}^2},$$ (4)

and for any vector $\mathbf{y}\in {\mathbb{R}}^{4n}$ we set

$${\parallel \mathbf{y}\parallel }_{2,1|{\mathbb{R}}^{4n}}=\sum _{j=1}^n\sqrt{y_j^2+y_{n+j}^2+y_{2n+j}^2+y_{3n+j}^2}.$$ (5)

Given an angle $\theta \in [-\pi ,\pi ]$ and a scaling parameter $a>0$, we have that the discrete directional gradient operator has the form

$$\tilde{\nabla }=\left[\begin{array}{c}{D}_{\theta }\\ D_{{\theta }^{\perp }}\end{array}\right]=\left[\begin{array}{c}cos\left(\theta \right)D_H+sin\left(\theta \right)D_V\\ a\left(-sin\left(\theta \right)D_H+cos\left(\theta \right)D_V\right)\end{array}\right],$$

where $D_{\theta },D_{{\theta }^{\perp }}\in {\mathbb{R}}^{n\times n}$ represent the forward finite-difference operators along the directions determined by $\theta$ and ${\theta }^{\perp }=\theta +\frac{\pi }{2}$, respectively, and $D_H,D_V\in {\mathbb{R}}^{n\times n}$ represent the forward finite-difference operators along the horizontal and the vertical direction, respectively. Moreover, the directional symmetrized derivative is defined in block-wise form as

$$\tilde{\mathcal{E}}=\left[\begin{array}{cc}{D}_{\theta }& 0\\ \frac{1}{2}{D}_{{\theta }^{\perp }}& \frac{1}{2}{D}_{\theta }\\ \frac{1}{2}{D}_{{\theta }^{\perp }}& \frac{1}{2}{D}_{\theta }\\ 0& D_{{\theta }^{\perp }}\end{array}\right].$$

It is worth noting that, by fixing $\theta =0$ and $a=1$, we have $D_{\theta }=D_H$ and $D_{{\theta }^{\perp }}=D_V$, and the operators $\tilde{\nabla }$ and $\tilde{\mathcal{E}}$ define the TGV${}^2$ regularization [4].

We observe that the definition of both the matrix A and the finite difference operators $D_H$ and $D_V$ depend on the choice of boundary conditions. We make the following assumption.

Assumption 1.

We assume that periodic boundary conditions are considered for A, $D_H$ and $D_V$. Therefore, those matrices are Block Circulant with Circulant Blocks (BCCB).

In this work we focus on directional images affected by Poisson noise, with the aim of assessing the behaviour of DTGV in this case. Besides extending the use of DTGV to Poisson noise, we introduce a novel technique for estimating the main direction of the image, which appears to be more efficient than the techniques applied in [9,10]. We solve the resulting optimization problem by using a customized version of the Alternating Direction Method of Multipliers (ADMM). We note that all the ADMM subproblems can be solved exactly at a low cost, thanks also to the use of FFTs, and that the method has proven convergence. Finally, we show the effectiveness of our approach on a set of test images, corrupted by out-of-focus and Gaussian blurs and noise with different signal-to-noise ratios. In particular, the KL-DTGV model of our problem is described in Section 2 and the technique for estimating the main direction is presented in Section 3. A detailed description of the ADMM version used for the minimization is given in Section 4 and the results of the numerical experiments are discussed in Section 5. Conclusions are given in Section 6.

Throughout this work we denote matrices with uppercase lightface letters, vectors with lowercase boldface letters and scalars with lowercase lightface letters. All the vectors are column vectors. Given a vector $\mathbf{v}$, we use $v_i$ or ${\left(\mathbf{v}\right)}_i$ to denote its i-th entry. We use ${\mathbb{R}}_{+}$ to indicate the set of real nonnegative numbers and $\parallel ·\parallel$ to indicate the two-norm. For brevity, given any vectors $\mathbf{v}$ and $\mathbf{w}$ we use the notation $(\mathbf{v},\mathbf{w})$ instead of ${\left[{\mathbf{v}}^{\top }{\mathbf{w}}^{\top }\right]}^{\top }$. Likewise, given any scalars v and w, we use $(v,w)$ to indicate the vector ${\left[vw\right]}^{\top }$. We also use the notation $({\left[\mathbf{v}\right]}_1,{\left[\mathbf{v}\right]}_2)$ to highlight the subvectors ${\left[\mathbf{v}\right]}_1$ and ${\left[\mathbf{v}\right]}_2$ forming the vector $\mathbf{v}$. Finally, by writing $\mathbf{v}>0$ we mean that all the entries of $\mathbf{v}$ are nonnegative and at least one of them is positive.

2. The KL-DTGV${}^2$ Model

We briefly describe the KL-DTGV${}^2$ model for the restoration of directional images corrupted by Poisson noise. Let $\mathbf{b}\in {\mathbb{R}}^n$ be the observed image. We want to recover the original image by minimizing a combination of the KL divergence (2) and the DTGV${}^2$ regularizer (3), i.e., by solving the optimization problem

$$\begin{array}{cc}\underset{\mathbf{u},\mathbf{w}}{min}\hfill & {\displaystyle \lambda D_{KL}(A\mathbf{u}+\gamma ,\mathbf{b})+{\alpha }_0{∥\tilde{\nabla }\mathbf{u}-\mathbf{w}∥}_{2,1|{\mathbb{R}}^{2n}}+{\alpha }_1{∥\tilde{\mathcal{E}}\mathbf{w}∥}_{2,1|{\mathbb{R}}^{4n}}}\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & \mathbf{u}\ge 0,\hfill \end{array}$$ (6)

where $\mathbf{u}\in {\mathbb{R}}^n$, $A\in {\mathbb{R}}^{n\times n}$, $\gamma ,\mathbf{b}\in {\mathbb{R}}^n$, $\mathbf{w}\in {\mathbb{R}}^{2n}$, and $\tilde{\nabla }\in {\mathbb{R}}^{2n\times n}$ and $\tilde{\mathcal{E}}\in {\mathbb{R}}^{4n\times 2n}$ are the linear operators defining the DTGV${}^2$ regularization. The parameters $\lambda \in (0,+\infty )$ and ${\alpha }_0,{\alpha }_1\in (0,1)$ determine the balance between the KL data fidelity term and the two components of the regularization term.

We note that problem (6) is a nonsmooth convex optimization problem because of the properties of the KL divergence (see, e.g., [11]) and the DTGV operator (see, e.g., [10]).

3. Efficient Estimation of the Image Direction

An essential ingredient in the DTGV regularization is the estimation of the angle $\theta$ representing the image texture direction. In [10], an estimation algorithm based on the one in [12] is proposed, whose basic idea is to compute a pixelwise direction estimate and then $\theta$ as the average of that estimate. In [9], which focuses on impulse noise removal, a more efficient and robust algorithm for estimating the direction is presented, based on the Fourier transform. The main idea behind this algorithm is to exploit the fact that two-dimensional Fourier basis functions can be seen as images with one-directional patterns. However, despite being very efficient from a computational viewpoint, this technique does not appear to be fully reliable in our tests on Poissonian images (see Section 5.1). Therefore, we propose a different approach for estimating the direction, based on classical tools of image processing: the Sobel filter [13] and the Hough transform [14,15].

Our technique is based on the idea that if an image has a one-directional structure, i.e., its main pattern consists of stripes, then the edges of the image mainly consist of lines going in the direction of the stripes. The first stage of the proposed algorithm uses the Sobel filter to determine the edges of the noisy and blurry image. Then, the Hough transform is applied to the edge image in order to detect the lines. The Hough transform is based on the idea that each straight line can be identified by a pair $(r,\eta )$ where r is the distance of the line from the origin, and $\eta$ is the angle between the x axis and the segment connecting the origin with its orthogonal projection on the line. The output of the transform is a matrix in which each entry is associated with a pair $(r,\eta )$, i.e., with a straight line in the image, and its value is the sum of the values in the pixels that are on the line. Hence, the elements with the highest value in the Hough transform indicate the lines that are most likely to be present in the input image. Because of its definition, the Hough transform tends to overestimate diagonal lines in rectangular images (diagonal lines through the central part of the image contain the largest number of pixels); therefore, before computing the transform we apply a mask to the edge image, considering only the pixels inside the largest circle centered in the center of the image. After the Hough transform has been applied, we compute the square of the two-norm of each column of the matrix resulting from the transform, to determine a score for each angle from $-{90}^{\circ }$ to ${90}^{\circ }$. Intuitively, the score for each angle is related to the number of lines with that particular inclination which have been detected in the image. Finally, we set the direction estimate $\theta \in [-\pi ,\pi ]$ as

$$\theta =\left\{\begin{array}{cc}{\displaystyle \frac{90-{\eta }_{max}}{180}\pi ,}\hfill & {\eta }_{max}\ge 0,\hfill \\ {\displaystyle \frac{-90-{\eta }_{max}}{180}\pi ,}\hfill & {\eta }_{max}<0.\hfill \end{array}\right.$$

where ${\eta }_{max}$ is the value of $\eta$ corresponding to the maximum score. A pseudocode for the estimation algorithm is provided in Algorithm 1 and an example of the algorithm workflow is given in Figure 1.

Algorithm 1 Direction estimation.

1:Use the Sobel operator to obtain the image $\mathbf{e}$ of the edges of the noisy and blurry image $\mathbf{b}$. 2:Apply a disk mask to cut out some diagonal edges in $\mathbf{e}$, obtaining a new edge image $\tilde{\mathbf{e}}$ (Figure 1b). 3:Compute the Hough transform $h\left(\tilde{\mathbf{e}}\right)$ (Figure 1c). 4:Set ${\eta }_{max}$ as the value of $\eta$ corresponding to the column of $h\left(\tilde{\mathbf{e}}\right)$ with maximum 2-norm. (Figure 1d) 5:Set $\theta =\{\begin{array}{cc}\frac{90-{\eta }_{max}}{180}\pi ,& {\eta }_{max}\ge 0,\\ \frac{-90-{\eta }_{max}}{180}\pi ,\hfill & {\eta }_{max}<0.\hfill \end{array}$ (yellow line in Figure 1a)

Figure 1.

Workflow of Algorithm 1 on a random directional image.

4. ADMM for Minimizing the KL-DTGV${}^{\mathbf{2}}$ Model

Although problem (6) is a bound-constrained convex optimization problem, the nondifferentiability of the DTGV${}^2$ regularizer does not allow its solution by classical optimization methods for smooth problems, such as gradient methods (see [16,17,18] and the references therein). However, the problem can be solved by methods based on splitting techniques, such as [19,20,21,22,23]. Here we solve (6) by the Alternating Direction Method of Multipliers (ADMM) [20]. To this end, we first reformulate the problem as follows:

$$\begin{array}{cc}\underset{\mathbf{u},\mathbf{w},{\mathbf{z}}_1,{\mathbf{z}}_2,{\mathbf{z}}_3,{\mathbf{z}}_4}{min}\hfill & {\displaystyle \lambda D_{KL}({\mathbf{z}}_1+\gamma ,\mathbf{b})+{\alpha }_0\parallel {\mathbf{z}}_2{\parallel }_{2,1|{\mathbb{R}}^{2n}}+{\alpha }_1{\parallel {\mathbf{z}}_3\parallel }_{2,1|{\mathbb{R}}^{4n}}+{\chi }_{{\mathbb{R}}_{+}^n}\left({\mathbf{z}}_4\right)}\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & {\mathbf{z}}_1=A\mathbf{u},\hfill \\ \hfill & {\mathbf{z}}_2=\tilde{\nabla }\mathbf{u}-\mathbf{w},\hfill \\ \hfill & {\mathbf{z}}_3=\tilde{\mathcal{E}}\mathbf{w},\hfill \\ \hfill & {\mathbf{z}}_4=\mathbf{u},\hfill \end{array}$$ (7)

where ${\mathbf{z}}_1\in {\mathbb{R}}^n$, ${\mathbf{z}}_2\in {\mathbb{R}}^{2n}$, ${\mathbf{z}}_3\in {\mathbb{R}}^{4n}$, ${\mathbf{z}}_4\in {\mathbb{R}}^n$, and ${\chi }_{{\mathbb{R}}_{+}^n}\left({\mathbf{z}}_4\right)$ is the characteristic function of the nonnegative orthant in ${\mathbb{R}}^n$. A similar splitting has been used in [24] for TV-based deblurring of Poissonian images. By introducing the auxiliary variables $\mathbf{x}=(\mathbf{u},\mathbf{w})$ and $\mathbf{z}=({\mathbf{z}}_1,{\mathbf{z}}_2,{\mathbf{z}}_3,{\mathbf{z}}_4)$ we can further reformulate the KL-DTGV${}^2$ problem as

$$\begin{array}{cc}\underset{\mathbf{x},\mathbf{z}}{min}\hfill & {\displaystyle F_1\left(\mathbf{x}\right)+F_2\left(\mathbf{z}\right)}\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & H\mathbf{x}+G\mathbf{z}=0,\hfill \end{array}$$ (8)

where we set

$$F_1\left(\mathbf{x}\right)=0,F_2\left(\mathbf{z}\right)=\lambda D_{KL}({\mathbf{z}}_1+\gamma ,\mathbf{b})+{\alpha }_0\parallel {\mathbf{z}}_2{\parallel }_{2,1|{\mathbb{R}}^{2n}}+{\alpha }_1{\parallel {\mathbf{z}}_3\parallel }_{2,1|{\mathbb{R}}^{4n}}+{\chi }_{{\mathbb{R}}_{+}^n}\left({\mathbf{z}}_4\right),$$ (9)

and we define the matrices $H\in {\mathbb{R}}^{8n\times 3n}$ and $G\in {\mathbb{R}}^{8n\times 8n}$ as

$$H=\left[\begin{array}{cc}\hfill A& \hfill 0\\ \hfill \tilde{\nabla }& \hfill -I_{2n}\\ \hfill 0& \hfill \tilde{\mathcal{E}}\\ \hfill I_n& \hfill 0\end{array}\right],G=\left[\begin{array}{cccc}\hfill -I_n& \hfill 0& \hfill 0& \hfill 0\\ \hfill 0& \hfill -I_{2n}& \hfill 0& \hfill 0\\ \hfill 0& \hfill 0& \hfill -I_{4n}& \hfill 0\\ \hfill 0& \hfill 0& \hfill 0& \hfill -I_n\end{array}\right].$$ (10)

We consider the Lagrangian function associated with problem (8),

$$\mathcal{L}(\mathbf{x},\mathbf{z},\mathbf{\xi })=F_1\left(\mathbf{x}\right)+F_2\left(\mathbf{z}\right)+{\mathbf{\xi }}^{\top }\left(H\mathbf{x}+G\mathbf{z}\right),$$ (11)

where $\mathbf{\xi }\in {\mathbb{R}}^{8n}$ is a vector of Lagrange multipliers, and then the augmented Lagrangian function

$${\mathcal{L}}_A(\mathbf{x},\mathbf{z},\mathbf{\xi };\rho )=F_1\left(\mathbf{x}\right)+F_2\left(\mathbf{z}\right)+{\mathbf{\xi }}^{\top }\left(H\mathbf{x}+G\mathbf{z}\right)+\frac{\rho }{2}{∥H\mathbf{x}+G\mathbf{z}∥}_2^2,$$ (12)

where $\rho >0$.

Now we are ready to introduce the ADMM method for the solution of problem (8). Let ${\mathbf{x}}^0\in {\mathbb{R}}^{3n}$, ${\mathbf{z}}^0\in {\mathbb{R}}^{8n}$, ${\mathbf{\xi }}^0\in {\mathbb{R}}^{8n}$. At each step $k>0$ the ADMM method computes the new iterate $\left({\mathbf{x}}^{k+1},{\mathbf{z}}^{k+1},{\mathbf{\xi }}^{k+1}\right)$ as follows:

$$\begin{array}{cc}\hfill {\mathbf{x}}^{k+1}& {\displaystyle =arg\underset{\mathbf{x}\in {\mathbb{R}}^{3n}}{min}{\mathcal{L}}_A(\mathbf{x},{\mathbf{z}}^k,{\mathbf{\xi }}^k;\rho ),}\hfill \\ \hfill {\mathbf{z}}^{k+1}& {\displaystyle =arg\underset{\mathbf{z}\in {\mathbb{R}}^{8n}}{min}{\mathcal{L}}_A({\mathbf{x}}^{k+1},\mathbf{z},{\mathbf{\xi }}^k;\rho ),}\hfill \\ \hfill {\mathbf{\xi }}^{k+1}& {\displaystyle ={\mathbf{\xi }}^k+\rho \left(H{\mathbf{x}}^{k+1}+G{\mathbf{z}}^{k+1}\right).}\hfill \end{array}$$ (13)

Note that the functions $F_1\left(\mathbf{x}\right)$ and $F_2\left(\mathbf{z}\right)$ in (8) are closed, proper and convex. Moreover, the matrices H and G defined in (10) are such that $G=-I_{8n}$ and H has full rank. Hence, the convergence of the method defined by (13) can be proved by applying a classical convergence result from the seminal paper by Eckstein and Bertsekas [25] (Theorem 8), which we report in a form that can be immediately applied to our reformulation of the problem.

Theorem 1.

Let us consider a problem of the form (8) where $F_1\left(\mathbf{x}\right)$ and $F_2\left(\mathbf{z}\right)$ are closed, proper and convex functions and H has full rank. Let ${\mathbf{x}}^0\in {\mathbb{R}}^{3n}$, ${\mathbf{z}}^0\in {\mathbb{R}}^{8n}$, ${\mathbf{\xi }}^0\in {\mathbb{R}}^{8n}$, and $\rho >0$. Suppose $\left\{{\epsilon }_k\right\},\left\{{\nu }_k\right\}\subset {\mathbb{R}}_{+}$ are summable sequences such that for all k

$$\begin{array}{ccc}& & ∥{\mathbf{x}}^{k+1}-arg\underset{\mathbf{x}\in {\mathbb{R}}^{3n}}{min}{\mathcal{L}}_A(\mathbf{x},{\mathbf{z}}^k,{\mathbf{\xi }}^k;\rho )∥\le {\epsilon }_k,\hfill \\ & & ∥{\mathbf{z}}^{k+1}-arg\underset{\mathbf{z}\in {\mathbb{R}}^{8n}}{min}{\mathcal{L}}_A({\mathbf{x}}^{k+1},\mathbf{z},{\mathbf{\xi }}^k;\rho )∥\le {\nu }_k,\hfill \\ & & {\mathbf{\xi }}^{k+1}={\mathbf{\xi }}^k+\rho \left(H{\mathbf{x}}^{k+1}+G{\mathbf{z}}^{k+1}\right).\hfill \end{array}$$

If there exists a saddle point $({\mathbf{x}}^{*},{\mathbf{z}}^{*},{\mathbf{\xi }}^{*})$ of $\mathcal{L}(\mathbf{x},\mathbf{z},\mathbf{\xi })$, then ${\mathbf{x}}^k\to {\mathbf{x}}^{*}$, ${\mathbf{z}}^k\to {\mathbf{z}}^{*}$ and ${\mathbf{\xi }}^k\to {\mathbf{\xi }}^{*}$. If such saddle point does not exist, then at least one of the sequences $\left\{{\mathbf{z}}^k\right\}$ or $\left\{{\mathbf{\xi }}^k\right\}$ is unbounded.

Since we are dealing with linear constraints, we can recast (13) in a more convenient form, by observing that the linear term in (12) can be included in the quadratic one. By introducing the vector of scaled Lagrange multipliers ${\mu }^k=\frac{1}{\rho }{\mathbf{\xi }}^k$, the ADMM method becomes

$$\begin{array}{ccc}\hfill {\mathbf{x}}^{k+1}& =& {\displaystyle arg\underset{\mathbf{x}\in {\mathbb{R}}^{3n}}{min}\frac{\rho }{2}{∥H\mathbf{x}-{\mathbf{z}}^k+{\mu }^k∥}_2^2,}\hfill \end{array}$$ (14)

$$\begin{array}{ccc}\hfill {\mathbf{z}}^{k+1}& =& {\displaystyle arg\underset{\mathbf{z}\in {\mathbb{R}}^{8n}}{min}{F}_2\left(\mathbf{z}\right)+\frac{\rho }{2}{∥H{\mathbf{x}}^{k+1}-\mathbf{z}+{\mu }^k∥}_2^2,}\hfill \end{array}$$ (15)

$$\begin{array}{ccc}\hfill {\mu }^{k+1}& =& {\displaystyle {\mu }^k+H{\mathbf{x}}^{k+1}+G{\mathbf{z}}^{k+1}.}\hfill \end{array}$$ (16)

In the next sections we show how the solutions to subproblems (14) and (15) can be computed exactly with a small computational effort.

4.1. Solving the Subproblem in $\mathbf{x}$

Problem (14) is an overdetermined least squares problem, since H is a tall-and-skinny matrix with full rank. Hence, its solution can be computed by solving the normal equations system

$$H^{\top }H\mathbf{x}=H^{\top }{\mathbf{v}}_x^k,$$ (17)

where we set ${\mathbf{v}}_x^k={\mathbf{z}}^k-{\mu }^k$. Starting from the definition of H given in (10), we have

$$\begin{array}{ccc}\hfill H^{\top }H& =& \left[\begin{array}{cc}{I}_n+A^{\top }A+{\tilde{\nabla }}^{\top }\tilde{\nabla }& -{\tilde{\nabla }}^{\top }\\ -\tilde{\nabla }& I_{2n}+{\tilde{\mathcal{E}}}^{\top }\tilde{\mathcal{E}}\end{array}\right]=\hfill \\ & =& \left[\begin{array}{ccc}{I}_n+A^{\top }A+{\tilde{\nabla }}^{\top }\tilde{\nabla }& -D_{\theta }^{\top }& -D_{{\theta }^{\perp }}^{\top }\\ -D_{\theta }& I_n+D_{\theta }^{\top }{D}_{\theta }+\frac{1}{2}{D}_{{\theta }^{\perp }}^{\top }{D}_{{\theta }^{\perp }}& \frac{1}{2}{D}_{{\theta }^{\perp }}^{\top }{D}_{\theta }\\ -D_{{\theta }^{\perp }}& \frac{1}{2}{D}_{\theta }^{\top }{D}_{{\theta }^{\perp }}& I_n+\frac{1}{2}{D}_{\theta }^{\top }{D}_{\theta }+D_{{\theta }^{\perp }}^{\top }{D}_{{\theta }^{\perp }}\end{array}\right].\hfill \end{array}$$

System (17) may be quite large and expensive, also for relatively small images. However, as pointed out in Assumption 1, A, $D_{\theta }$ and $D_{{\theta }^{\perp }}$ have a BCCB structure, hence all the blocks of $H^{\top }H$ maintain that structure. By recalling that BCCB matrices can be diagonalized by means of two-dimensional Discrete Fourier Transforms (DFTs), we show how the solution to (17) can be computed expeditiously.

Let $\mathcal{F}\in {\mathbb{C}}^{n\times n}$ be the matrix representing the two-dimensional DFT operator, and let ${\mathcal{F}}^{*}$ denote its inverse, i.e., its adjoint. We can write $H^{\top }H$ as

$$H^{\top }H=\left[\begin{array}{ccc}{\mathcal{F}}^{*}& 0& 0\\ 0& {\mathcal{F}}^{*}& 0\\ 0& 0& {\mathcal{F}}^{*}\end{array}\right]\left[\begin{array}{ccc}\mathsf{\Gamma }& -{\mathsf{\Delta }}_{\theta }^{*}& -{\mathsf{\Delta }}_{{\theta }^{\perp }}^{*}\\ -{\mathsf{\Delta }}_{\theta }& {\mathsf{\Phi }}_{11}& {\mathsf{\Phi }}_{12}\\ -{\mathsf{\Delta }}_{{\theta }^{\perp }}& {\mathsf{\Phi }}_{21}& {\mathsf{\Phi }}_{22}\end{array}\right]\left[\begin{array}{ccc}\mathcal{F}& 0& 0\\ 0& \mathcal{F}& 0\\ 0& 0& \mathcal{F}\end{array}\right],$$ (18)

where each block of the central matrix is the diagonal complex matrix associated with the corresponding block in $H^{\top }H$, and ${\mathsf{\Delta }}_{\theta }^{*},{\mathsf{\Delta }}_{{\theta }^{\perp }}^{*}$ denote the (diagonal) adjoint matrices of ${\mathsf{\Delta }}_{\theta },{\mathsf{\Delta }}_{{\theta }^{\perp }}$. By (18) and the definition of $\mathbf{x}$, we can reformulate (17) as

$$\left[\begin{array}{ccc}\mathsf{\Gamma }& -{\mathsf{\Delta }}_{\theta }^{*}& -{\mathsf{\Delta }}_{{\theta }^{\perp }}^{*}\\ -{\mathsf{\Delta }}_{\theta }& {\mathsf{\Phi }}_{11}& {\mathsf{\Phi }}_{12}\\ -{\mathsf{\Delta }}_{{\theta }^{\perp }}& {\mathsf{\Phi }}_{21}& {\mathsf{\Phi }}_{22}\end{array}\right]\left[\begin{array}{c}\mathcal{F}\mathbf{u}\\ \mathcal{F}{\mathbf{w}}_1\\ \mathcal{F}{\mathbf{w}}_2\end{array}\right]=\left[\begin{array}{c}\mathcal{F}{\left[H^{\top }{\mathbf{v}}_x^k\right]}_1\\ \mathcal{F}{\left[H^{\top }{\mathbf{v}}_x^k\right]}_2\\ \mathcal{F}{\left[H^{\top }{\mathbf{v}}_x^k\right]}_3\end{array}\right],$$ (19)

where we split $\mathbf{w}$ and ${\mathbf{v}}_x^k$ in two and three blocks of size n, respectively.

Now we recall a result about the inversion of block matrices. Suppose that a square matrix M is partitioned into four blocks, i.e.,

$$M=\left[\begin{array}{cc}{M}_{11}& M_{12}\\ M_{21}& M_{22}\end{array}\right];$$

then, if $M_{11}$ and $M_{22}$ are invertible, we have

$$\begin{array}{cc}\hfill M^{-1}=& {\left[\begin{array}{cc}{M}_{11}& M_{12}\\ M_{21}& M_{22}\end{array}\right]}^{-1}\hfill \\ \hfill =& \left[\begin{array}{cc}{\left(M_{11}-M_{12}{M}_{22}^{-1}{M}_{21}\right)}^{-1}& 0\\ 0& {\left(M_{22}-M_{21}{M}_{11}^{-1}{M}_{12}\right)}^{-1}\end{array}\right]\left[\begin{array}{cc}I& -M_{12}{M}_{22}^{-1}\\ -M_{21}{M}_{11}^{-1}& I\end{array}\right].\hfill \end{array}$$ (20)

By applying (20) to the matrix consisting of the second and third block rows and columns of the matrix in (19), which we denote $\mathsf{\Phi }$, we get

$$\begin{array}{cc}\hfill {\mathsf{\Phi }}^{-1}=& {\left[\begin{array}{cc}{\mathsf{\Phi }}_{11}& {\mathsf{\Phi }}_{12}\\ {\mathsf{\Phi }}_{21}& {\mathsf{\Phi }}_{22}\end{array}\right]}^{-1}\hfill \\ \hfill =& \left[\begin{array}{cc}{\left({\mathsf{\Phi }}_{11}-{\mathsf{\Phi }}_{12}{\mathsf{\Phi }}_{22}^{-1}{\mathsf{\Phi }}_{21}\right)}^{-1}& -{\left({\mathsf{\Phi }}_{11}-{\mathsf{\Phi }}_{12}{\mathsf{\Phi }}_{22}^{-1}{\mathsf{\Phi }}_{21}\right)}^{-1}{\mathsf{\Phi }}_{12}{\mathsf{\Phi }}_{22}^{-1}\\ -{\left({\mathsf{\Phi }}_{22}-{\mathsf{\Phi }}_{21}{\mathsf{\Phi }}_{11}^{-1}{\mathsf{\Phi }}_{12}\right)}^{-1}{\mathsf{\Phi }}_{21}{\mathsf{\Phi }}_{11}^{-1}& {\left({\mathsf{\Phi }}_{22}-{\mathsf{\Phi }}_{21}{\mathsf{\Phi }}_{11}^{-1}{\mathsf{\Phi }}_{12}\right)}^{-1}\end{array}\right].\hfill \end{array}$$ (21)

To simplify the notation we set

$$\mathsf{\Psi }=\left[\begin{array}{cc}{\mathsf{\Psi }}_{11}& {\mathsf{\Psi }}_{12}\\ {\mathsf{\Psi }}_{21}& {\mathsf{\Psi }}_{22}\end{array}\right]={\mathsf{\Phi }}^{-1},$$ (22)

and observe that the matrices ${\mathsf{\Psi }}_{ij}\in {\mathbb{C}}^{n\times n}$ are diagonal. Letting ${\mathsf{\Delta }}^{*}=\left[{\mathsf{\Delta }}_{\theta }^{*}{\mathsf{\Delta }}_{{\theta }^{\perp }}^{*}\right]$, applying the inversion formula (20) to the whole matrix in (19), and using (21) and (22), we get

$${\left[\begin{array}{cc}\mathsf{\Gamma }& -{\mathsf{\Delta }}^{*}\\ -\mathsf{\Delta }& \mathsf{\Phi }\end{array}\right]}^{-1}=\left[\begin{array}{cc}{\mathsf{\Xi }}^{-1}& 0\\ 0& {\mathsf{\Omega }}^{-1}\end{array}\right]\left[\begin{array}{cc}{I}_n& -{\mathsf{\Delta }}^{*}\mathsf{\Psi }\\ -\mathsf{\Delta }{\mathsf{\Gamma }}^{-1}& I_{2n}\end{array}\right],$$ (23)

where

$$\begin{array}{cc}\hfill \mathsf{\Xi }& =\mathsf{\Gamma }-{\mathsf{\Delta }}^{*}\mathsf{\Psi }\mathsf{\Delta }=\mathsf{\Gamma }-\left[{\mathsf{\Delta }}_{\theta }^{*}{\mathsf{\Delta }}_{{\theta }^{\perp }}^{*}\right]\left[\begin{array}{cc}{\mathsf{\Psi }}_{11}& {\mathsf{\Psi }}_{12}\\ {\mathsf{\Psi }}_{21}& {\mathsf{\Psi }}_{22}\end{array}\right]\left[\begin{array}{c}{\mathsf{\Delta }}_{\theta }\\ {\mathsf{\Delta }}_{{\theta }^{*}}\end{array}\right],\hfill \\ \hfill \mathsf{\Omega }& =\mathsf{\Phi }-\mathsf{\Delta }{\mathsf{\Gamma }}^{-1}{\mathsf{\Delta }}^{*}=\left[\begin{array}{cc}{\mathsf{\Phi }}_{11}& {\mathsf{\Phi }}_{12}\\ {\mathsf{\Phi }}_{21}& {\mathsf{\Phi }}_{22}\end{array}\right]-\left[\begin{array}{c}{\mathsf{\Delta }}_{\theta }\\ {\mathsf{\Delta }}_{{\theta }^{\perp }}\end{array}\right]{\mathsf{\Gamma }}^{-1}\left[{\mathsf{\Delta }}_{\theta }^{*}{\mathsf{\Delta }}_{{\theta }^{\perp }}^{*}\right].\hfill \end{array}$$

We note that $\mathsf{\Xi }\in {\mathbb{C}}^{n\times n}$ is diagonal (and its inversion is straightforward), while $\mathsf{\Omega }\in {\mathbb{C}}^{2n\times 2n}$ has a $2\times 2$ block structure with blocks that are diagonal matrices belonging to ${\mathbb{C}}^{n\times n}$. Thus, we can compute $\mathsf{{\rm Y}}={\mathsf{\Omega }}^{-1}$ by applying (20):

$$\begin{array}{cc}\hfill \mathsf{{\rm Y}}& =\left[\begin{array}{cc}{\mathsf{{\rm Y}}}_{11}& {\mathsf{{\rm Y}}}_{12}\\ {\mathsf{{\rm Y}}}_{21}& {\mathsf{{\rm Y}}}_{22}\end{array}\right]={\left[\begin{array}{cc}{\mathsf{\Omega }}_{11}& {\mathsf{\Omega }}_{12}\\ {\mathsf{\Omega }}_{21}& {\mathsf{\Omega }}_{22}\end{array}\right]}^{-1}\hfill \\ \hfill & =\left[\begin{array}{cc}{\left({\mathsf{\Omega }}_{11}-{\mathsf{\Omega }}_{12}{\mathsf{\Omega }}_{22}^{-1}{\mathsf{\Omega }}_{21}\right)}^{-1}& -{\left({\mathsf{\Omega }}_{11}-{\mathsf{\Omega }}_{12}{\mathsf{\Omega }}_{22}^{-1}{\mathsf{\Omega }}_{21}\right)}^{-1}{\mathsf{\Omega }}_{12}{\mathsf{\Omega }}_{22}^{-1}\\ -{\left({\mathsf{\Omega }}_{22}-{\mathsf{\Omega }}_{21}{\mathsf{\Omega }}_{11}^{-1}{\mathsf{\Omega }}_{12}\right)}^{-1}{\mathsf{\Omega }}_{21}{\mathsf{\Omega }}_{11}^{-1}& {\left({\mathsf{\Omega }}_{22}-{\mathsf{\Omega }}_{21}{\mathsf{\Omega }}_{11}^{-1}{\mathsf{\Omega }}_{12}\right)}^{-1}\end{array}\right]\hfill \end{array}$$ (24)

Summing up, by (19), (23) and (24), the solution to (17) can be obtained by computing

$$\left[\begin{array}{c}{\mathbf{y}}_1\\ {\mathbf{y}}_2\\ {\mathbf{y}}_3\end{array}\right]=\left[\begin{array}{cc}{\mathsf{\Xi }}^{-1}& 0\\ 0& \mathsf{{\rm Y}}\end{array}\right]\left[\begin{array}{cc}{I}_n& -{\mathsf{\Delta }}^{\top }\mathsf{\Psi }\\ -\mathsf{\Delta }{\mathsf{\Gamma }}^{-1}& I_{2n}\end{array}\right]\left[\begin{array}{c}\mathcal{F}{\left[H^{\top }{\mathbf{v}}_x^k\right]}_1\\ \mathcal{F}{\left[H^{\top }{\mathbf{v}}_x^k\right]}_2\\ \mathcal{F}{\left[H^{\top }{\mathbf{v}}_x^k\right]}_3\end{array}\right],$$ (25)

and setting

$${\mathbf{u}}^{k+1}={\mathcal{F}}^{*}{\mathbf{y}}_1,{\mathbf{w}}_1^{k+1}={\mathcal{F}}^{*}{\mathbf{y}}_2,{\mathbf{w}}_2^{k+1}={\mathcal{F}}^{*}{\mathbf{y}}_3.$$ (26)

Remark 1.

The only quantity in (25) that varies at each iteration is ${\mathbf{v}}_x^k$. Hence, the matrices Δ, Γ, Ψ, ${\mathsf{\Xi }}^{-1}$, and Υ can be computed only once before the ADMM method starts. This means that the overall cost of the exact solution of (14) at each iteration reduces to six two-dimensional DFTs and two matrix–vector products involving two $3\times 3$ block matrices with diagonal blocks of dimension n.

4.2. Solving the Subproblem in $\mathbf{z}$

By looking at the form of $F_2\left(\mathbf{z}\right)$–see (9)–and by defining the vector ${\mathbf{v}}_z^k=H{\mathbf{x}}^{k+1}+{\mu }^k$, we see that problem (15) can be split into the four problems

$$\begin{array}{ccc}\hfill {\mathbf{z}}_1^{k+1}& =& {\displaystyle arg\underset{{\mathbf{z}}_1\in {\mathbb{R}}^n}{min}\lambda D_{KL}({\mathbf{z}}_1+\gamma ,\mathbf{b})+\frac{\rho }{2}{∥{\mathbf{z}}_1-{\left[{\mathbf{v}}_z^k\right]}_1∥}_2^2,}\hfill \end{array}$$ (27)

$$\begin{array}{ccc}\hfill {\mathbf{z}}_2^{k+1}& =& {\displaystyle arg\underset{{\mathbf{z}}_2\in {\mathbb{R}}^{2n}}{min}{\alpha }_0{\parallel {\mathbf{z}}_2\parallel }_{2,1|{\mathbb{R}}^{2n}}+\frac{\rho }{2}{∥{\mathbf{z}}_2-{\left[{\mathbf{v}}_z^k\right]}_2∥}_2^2,}\hfill \end{array}$$ (28)

$$\begin{array}{ccc}\hfill {\mathbf{z}}_3^{k+1}& =& {\displaystyle arg\underset{{\mathbf{z}}_3\in {\mathbb{R}}^{4n}}{min}{\alpha }_1{\parallel {\mathbf{z}}_3\parallel }_{2,1|{\mathbb{R}}^{4n}}+\frac{\rho }{2}{∥{\mathbf{z}}_3-{\left[{\mathbf{v}}_z^k\right]}_3∥}_2^2,}\hfill \end{array}$$ (29)

$$\begin{array}{ccc}\hfill {\mathbf{z}}_4^{k+1}& =& {\displaystyle arg\underset{{\mathbf{z}}_4\in {\mathbb{R}}^n}{min}{\chi }_{{\mathbb{R}}_{+}^n}\left({\mathbf{z}}_4\right)+\frac{\rho }{2}{∥{\mathbf{z}}_4-{\left[{\mathbf{v}}_z^k\right]}_4∥}_2^2,}\hfill \end{array}$$ (30)

where ${\mathbf{v}}_z^k=({\left[{\mathbf{v}}_z^k\right]}_1,{\left[{\mathbf{v}}_z^k\right]}_2,{\left[{\mathbf{v}}_z^k\right]}_3,{\left[{\mathbf{v}}_z^k\right]}_4)$, with ${\left[{\mathbf{v}}_z^k\right]}_1\in {\mathbb{R}}^n$, ${\left[{\mathbf{v}}_z^k\right]}_2\in {\mathbb{R}}^{2n}$,${\left[{\mathbf{v}}_z^k\right]}_3\in {\mathbb{R}}^{4n}$, and ${\left[{\mathbf{v}}_z^k\right]}_4\in {\mathbb{R}}^n$. Now we focus on the solution of the four subproblems.

4.2.1. Update of ${\mathbf{z}}_1$

By the form of the Kullback–Leibler divergence in (2), the minimization problem (27) is equivalent to

$$\underset{{\mathbf{z}}_1\in {\mathbb{R}}^n}{min}\lambda \sum _{i=1}^n\left(b_{i}ln\frac{b_i}{{\left({\mathbf{z}}_1\right)}_i+{\gamma }_i}+{\left({\mathbf{z}}_1\right)}_i+{\gamma }_i-b_i\right)+\frac{\rho }{2}\sum _{i=1}^n{\left({\left({\mathbf{z}}_1\right)}_i-d_i\right)}^2,$$ (31)

where we set $\mathbf{d}={\left[{\mathbf{v}}_z^k\right]}_1$ to ease the notation. From (31) it is clear that the problem in ${\mathbf{z}}_1$ can be split into n problems of the form

$$\underset{z\in \mathbb{R}}{min}\lambda \left(-bln(z+\gamma )+z\right)+\frac{\rho }{2}{\left(z-d\right)}^2.$$ (32)

Since the objective function of this problem is strictly convex, its solution can be determined by setting the gradient equal to zero, i.e., by solving

$$\lambda \left(-\frac{b}{z+\gamma }+1\right)+\rho \left(z-d\right)=0,$$

which leads to the quadratic equation

$$z^2+\left(\frac{\lambda }{\rho }+\gamma -d\right)z-\frac{\lambda }{\rho }\left(\frac{\rho }{\lambda }\gamma d-\gamma +b\right)=0.$$ (33)

Since, by looking at the domain of the objective function in (32), $z+\gamma$ has to be strictly positive, we set each entry of ${\mathbf{z}}_1^{k+1}$ as the largest solution of the corresponding quadratic Equation (33).

4.2.2. Update of ${\mathbf{z}}_2$ and ${\mathbf{z}}_3$

The minimization problems (28) and (29) correspond to the computation of the proximal operators of the functions $f\left({\mathbf{z}}_2\right)=\frac{{\alpha }_0}{\rho }{\parallel {\mathbf{z}}_2\parallel }_{2,1|{\mathbb{R}}^{2n}}$ and $g\left({\mathbf{z}}_3\right)=\frac{{\alpha }_1}{\rho }{\parallel {\mathbf{z}}_3\parallel }_{2,1|{\mathbb{R}}^{4n}}$, respectively.

By the definitions given in (4) and (5), we see that the two (2,1)-norms correspond to the sum of two-norms of vectors in ${\mathbb{R}}^2$ and ${\mathbb{R}}^4$, respectively. This means that the computation of both the proximal operators can be split into the computation of n proximal operators of functions that are scaled two-norms in either ${\mathbb{R}}^2$ or ${\mathbb{R}}^4$.

The proximal operator of the function $f\left(\mathbf{y}\right)=c\parallel \mathbf{y}\parallel$, $c>0$, at a vector $\mathbf{d}$ is

$${prox}_{c\parallel ·\parallel }\left(\mathbf{d}\right)=arg\underset{\mathbf{y}}{min}c\parallel \mathbf{y}\parallel +\frac{1}{2}{\parallel \mathbf{y}-\mathbf{d}\parallel }^2.$$

It can be shown (see, e.g., [26] [Chapter 6]) that

$${prox}_{c\parallel ·\parallel }\left(\mathbf{d}\right)=\left(1-\frac{c}{max\left\{\parallel \mathbf{d}\parallel ,c\right\}}\right)\mathbf{d}=max\left\{\frac{\parallel \mathbf{d}\parallel -c}{\parallel \mathbf{d}\parallel },0\right\}\mathbf{d}.$$ (34)

Hence, for the update of ${\mathbf{z}}_2$ we proceed as follows. By setting $\mathbf{d}={\left[{\mathbf{v}}_z^k\right]}_2$ and $c=\frac{{\alpha }_0}{\rho }$, for each $i=1,\dots ,n$ we have

$$\left({\left({\mathbf{z}}_2^{k+1}\right)}_i,{\left({\mathbf{z}}_2^{k+1}\right)}_{n+i}\right)={prox}_{c\parallel ·\parallel }\left((d_i,d_{n+i})\right).$$

To update ${\mathbf{z}}_3$, we set $\mathbf{d}={\left[{\mathbf{v}}_z^k\right]}_3$ and $c=\frac{{\alpha }_1}{\rho }$ and compute

$$\left({\left({\mathbf{z}}_3^{k+1}\right)}_i,{\left({\mathbf{z}}_3^{k+1}\right)}_{n+i},{\left({\mathbf{z}}_3^{k+1}\right)}_{2n+i},{\left({\mathbf{z}}_3^{k+1}\right)}_{3n+i}\right)={prox}_{c\parallel ·\parallel }\left((d_i,d_{n+i},d_{2n+i},d_{3n+i})\right).$$

4.2.3. Update of ${\mathbf{z}}_4$

It is straightforward to verify that the update of ${\mathbf{z}}_4$ in (30) can be obtained as

$${\mathbf{z}}_4^{k+1}={\mathsf{\Pi }}_{{\mathbb{R}}_{+}^n}\left({\left[{\mathbf{v}}_z^k\right]}_4\right),$$

where ${\mathsf{\Pi }}_{{\mathbb{R}}_{+}^n}$ is the Euclidean projection onto the nonnegative orthant in ${\mathbb{R}}^n$.

4.3. Summary of the ADMM Method

For the sake of clarity, in Algorithm 2 we sketch the ADMM version for solving problem (7).

In many image restoration applications, a reasonably good starting guess for $\mathbf{u}$ is often available. For example, if A represents a blur operator, a common choice is to set ${\mathbf{u}}^0$ equal to the the noisy and blurry image. We make this choice for ${\mathbf{u}}^0$. By numerical experiments we also verified that once $\mathbf{x}=(\mathbf{u},\mathbf{w})$ has been initialized, it is convenient to set ${\mathbf{u}}^1={\mathbf{u}}^0$, ${\mathbf{w}}_1^1={\mathbf{w}}_1^0$ and ${\mathbf{w}}_2^1={\mathbf{w}}_2^0$ and to shift the order of the updates in the ADMM scheme (14)–(16), so that a “more effective” initialization of $\mathbf{z}$ and $\mu$ is performed. We see from line 9 of Algorithm 2 that the algorithm stops when the relative change in the restored image $\mathbf{u}$ goes below a certain threshold $tol\in (0,1)$ or a maximum number of iterations $k_{max}$ is reached. Finally, we note that for the case of the KL-TGV${}^2$ model, corresponding to $\theta =0$ and $a=1$, we have that $D_{\theta }=D_H$ and $D_{{\theta }^{\perp }}=D_V$; hence, we use the initialization ${\mathbf{w}}_1^0=D_H{\mathbf{u}}^0$ and ${\mathbf{w}}_2^0=D_V{\mathbf{u}}^0$.

Algorithm 2 ADMM for problem (7).

1:Let ${\mathbf{u}}^0\in {\mathbb{R}}^n$, ${\mathbf{w}}_1^0=D_{\theta }{\mathbf{u}}^0$, ${\mathbf{w}}_2^0=D_{{\theta }^{\perp }}{\mathbf{u}}^0$, ${\mu }^0=0$, ${\mathbf{z}}^0=0$, $\lambda ,\rho \in (0,+\infty )$, ${\alpha }_0,{\alpha }_1\in (0,1)$  2:Compute matrices $\mathsf{\Delta }$, $\mathsf{\Gamma }$, $\mathsf{\Psi }$, ${\mathsf{\Xi }}^{-1}$, and $\mathsf{{\rm Y}}$ as specified in Section 4.1  3:Let $k=0$, ${\mathbf{u}}^1={\mathbf{u}}^0$, ${\mathbf{w}}^1={\mathbf{w}}^0$, $stop=false$, $tol\in (0,1)$, $k_{max}\in \mathbb{N}$  4:while $\mathrm{not}stop\mathrm{and}k\le k_{max}$ do  5: Compute ${\mathbf{z}}^{k+1}$ by solving the four subproblems (27)–(30)  6: Compute ${\mu }^{k+1}$ as in (16)  7: $k=k+1$  8: Compute ${\mathbf{u}}^{k+1}$, ${\mathbf{w}}_1^{k+1}$ and ${\mathbf{w}}_2^{k+1}$ by (25) and (26)  9: Set $stop=\left(\parallel {\mathbf{u}}^{k+1}-{\mathbf{u}}^k\parallel <tol\parallel {\mathbf{u}}^k\parallel \right)$ 10:end while

5. Numerical Results

All the experiments were carried out using MATLAB R2018a on a 3.50 GHz Intel Xeon E3 with 16 GB of RAM and Windows operating system. In this section, we first illustrate the effectiveness of Algorithm 1 for the estimation of the image direction by comparing it with the one given in [9] and by analysing its sensitivity to the degradation in the image to be restored. Then, we present numerical experiments that demonstrate the improvement of the KL-DTGV${}^2$ model upon the KL-TGV${}^2$ model for the restoration of directional images corrupted by Poisson noise.

Four directional images named phantom ($512\times 512$), grass ($375\times 600$), leaves ($203\times 300$) and carbon ($247\times 300$) were used in the experiments. The first image is a piecewise affine fibre phantom image obtained with the fibre_phantom_pa MATLAB function available from http://www2.compute.dtu.dk/~pcha/HDtomo/ (accessed on 20 September 2020). The second and third images represent grass and veins of leaves, respectively, which naturally exhibit a directional structure. The last image is a Scanning Electron Microscope (SEM) image of carbon fibres. The images are shown in Figure 2, Figure 3, Figure 4 and Figure 5.

Figure 2.

Test problem phantom: original and corrupted images. The yellow dash-dotted line indicates the direction estimated by Algorithm 1 and the red dashed line the direction estimated by the method in [9].

Figure 3.

Test problem grass: original and corrupted images. The yellow dash-dotted line indicates the direction estimated by Algorithm 1 and the red dashed line the direction estimated by the method in [9].

Figure 4.

Test problem leaves: original and corrupted images. The yellow dash-dotted line indicates the direction estimated by Algorithm 1 and the red dashed line the direction estimated by the method in [9].

Figure 5.

Test problem carbon: original and corrupted images. The yellow dash-dotted line indicates the direction estimated by Algorithm 1 and the red dashed line the direction estimated by the method in [9].

To simulate experimental data, each reference image was convolved with two PSFs, one corresponding to a Gaussian blur with variance 2, generated by the psfGauss function from [27], and the other corresponding to an out-of-focus blur with radius 5, obtained with the function fspecial from the MATLAB Image Processing Toolbox. To take into account the existence of some background emission, a constant term $\gamma$ equal to ${10}^{-10}$ was added to all pixels of the blurry image. The resulting image was corrupted by Poisson noise, using the MATLAB function imnoise. The intensities of the original images were pre-scaled to get noisy and blurry images with Signal to Noise Ratio (SNR) equal to 43 and 37 dB. We recall that in the case of Poisson noise, which affects the photon counting process, the SNR is estimated as [28]

$$\mathrm{SNR}=10{log}_{10}\left(\frac{N_{\mathrm{exact}}}{\sqrt{N_{\mathrm{exact}}+N_{\mathrm{background}}}}\right),$$

where $N_{\mathrm{exact}}$ and $N_{\mathrm{background}}$ are the total number of photons in the image to be recovered and in the background term, respectively. Finally, the corrupted images were scaled to have their maximum intensity values equal to 1. For each test problem, the noisy and blurry images are shown in Figure 2, Figure 3, Figure 4 and Figure 5.

5.1. Direction Estimation

In Figure 2, Figure 3, Figure 4 and Figure 5 we compare Algorithm 1 with the algorithm proposed in [9], showing that Algorithm 1 always correctly estimates the main direction of the four test images. We also test the robustness of our algorithm with respect to noise and blur. In Figure 6 we show the estimated main direction of the phantom image corrupted by Poisson noise with SNR $=35,37,39,41,43$ dB and out-of-focus blurs with radius R $=5,7,9$. In only one case (SNR $=35$, R $=7$) Algorithm 1 fails, returning as estimate the orthogonal direction, i.e., the direction corresponding to the large black line and the background color gradient. Finally, we test Algorithm 1 on a phantom image with vertical, horizontal and diagonal main directions corresponding to $\theta =0,90,45$. The results, in Figure 7, show that our algorithm is not sensitive to the specific directional structure of the image.

Figure 6.

Direction estimation for phantom with SNR $=35,37,39,41,43$ dB (from left to right) and out-of-focus blur with radius R $=5,7,9$ (from top to bottom).

Figure 7.

Direction estimation for phantom with SNR $=37,43$ (top, bottom) and out-of-focus blur with radius R $=5$.

5.2. Image Deblurring

We compare the quality of the restorations obtained by using the DTGV${}^2$ and TGV${}^2$ regularizers and ADMM for the solution of both models. In all the tests, the value of the penalty parameter was set as $\rho =10$ and the value of the stopping threshold as $tol={10}^{-4}$. A maximum number of $k_{\max }=500$ iterations was allowed. By following [9,10], the weight parameters of DTGV were chosen as ${\alpha }_0=\beta$ and ${\alpha }_1=(1-\beta )$ with $\beta =2/3$. For each test problem, the value of the regularization parameter $\lambda$ was tuned by a trial-and-error strategy. This strategy consisted in running ADMM with initial guess ${\mathbf{u}}^0=\mathbf{b}$ several times on each test image, varying the value of $\lambda$ at each execution. For all the runs the stopping criterion for ADMM and the values of ${\alpha }_0$, ${\alpha }_1$ and $\rho$ were the same as described above. The value of $\lambda$ yielding the smallest Root Mean Square Error (RMSE) at the last iteration was chosen as the “optimal” value.

The numerical results are summarized in Table 1, where the RMSE, the Improved Signal to Noise Ratio (ISNR) [29], and the structural similarity (SSIM) index [30] are used to give a quantitative evaluation of the quality of the restorations. As a measure of the computational cost, the number of iterations and the time in seconds are reported. Table 1 also shows, for each test problem, the values of the regularization parameter $\lambda$. The restored images are shown in Figure 8, Figure 9, Figure 10 and Figure 11. For the carbon test problem, Figure 12 shows the error images, i.e., the images obtained as the absolute difference between the original image and the restored one. The values of the pixels of the error images have been scaled in the range $[m,M]$ where m and M are the minimum and maximum pixel value of the DTGV${}^2$ and TGV${}^2$ error images.

Table 1.

Numerical results for the test problems.

Blur	SNR	Model	$\mathit{\lambda }$	RMSE	ISNR	MSSIM	Iters	Time
phantom
Out-of-focus	43	DTGV	57.5	2.2558 $\times {10}^{-2}$	9.5472	9.3007 $\times {10}^{-1}$	86	10.95
		TGV	275	2.8043 $\times {10}^{-2}$	7.6568	8.9887 $\times {10}^{-1}$	89	11.33
	37	DTGV	3.25	3.7573 $\times {10}^{-2}$	7.4431	8.5823 $\times {10}^{-1}$	122	15.45
		TGV	22.5	4.1719 $\times {10}^{-2}$	6.5339	8.4061 $\times {10}^{-1}$	52	6.64
Gaussian	43	DTGV	25	1.5530 $\times {10}^{-2}$	9.1966	9.7829 $\times {10}^{-1}$	56	7.17
		TGV	100	1.8100 $\times {10}^{-2}$	7.8667	9.7200 $\times {10}^{-1}$	45	5.76
	37	DTGV	3	2.5498 $\times {10}^{-2}$	9.0841	9.2994 $\times {10}^{-1}$	90	11.41
		TGV	17.5	3.0674 $\times {10}^{-2}$	7.4788	9.0199 $\times {10}^{-1}$	53	6.76
grass
Out-of-focus	43	DTGV	60	3.6313 $\times {10}^{-2}$	7.7364	8.7262 $\times {10}^{-1}$	136	15.55
		TGV	550	3.6575 $\times {10}^{-2}$	7.6738	8.7188 $\times {10}^{-1}$	179	20.39
	37	DTGV	50	5.6164 $\times {10}^{-2}$	4.7390	7.6165 $\times {10}^{-1}$	160	18.56
		TGV	55	5.7604 $\times {10}^{-2}$	4.5191	7.4566 $\times {10}^{-1}$	72	8.31
Gaussian	43	DTGV	65	2.9883 $\times {10}^{-2}$	6.3343	9.2764 $\times {10}^{-1}$	106	12.08
		TGV	650	3.0814 $\times {10}^{-2}$	6.0676	9.2523 $\times {10}^{-1}$	136	15.48
	37	DTGV	5.5	4.2274 $\times {10}^{-2}$	4.7973	8.5615 $\times {10}^{-1}$	98	11.13
		TGV	35	4.3936 $\times {10}^{-2}$	4.4624	8.4795 $\times {10}^{-1}$	54	6.18
leaves
Out-of-focus	43	DTGV	125	6.2767 $\times {10}^{-2}$	7.4978	8.2099 $\times {10}^{-1}$	251	31.18
		TGV	1100	8.2397 $\times {10}^{-2}$	5.1342	7.1557 $\times {10}^{-1}$	435	53.74
	37	DTGV	12.5	9.5597 $\times {10}^{-2}$	4.1497	6.3065 $\times {10}^{-1}$	257	31.87
		TGV	90	1.1874 $\times {10}^{-1}$	2.2665	4.3294 $\times {10}^{-1}$	113	14.03
Gaussian	43	DTGV	150	7.3332 $\times {10}^{-2}$	4.8675	7.7456 $\times {10}^{-1}$	236	29.13
		TGV	1750	8.0857 $\times {10}^{-2}$	4.0190	7.3001 $\times {10}^{-1}$	380	46.77
	37	DTGV	12.5	9.0999 $\times {10}^{-2}$	3.3907	6.6469 $\times {10}^{-1}$	148	18.36
		TGV	100	1.0308 $\times {10}^{-1}$	2.3081	5.6534 $\times {10}^{-1}$	103	12.85
carbon
Out-of-focus	43	DTGV	150	1.8360 $\times {10}^{-2}$	1.2830 $\times {10}^1$	9.4734 $\times {10}^{-1}$	331	13.78
		TGV	850	2.3825 $\times {10}^{-2}$	1.0567 $\times {10}^1$	9.3671 $\times {10}^{-1}$	233	9.73
	37	DTGV	20	3.1682 $\times {10}^{-2}$	8.2416	8.6294 $\times {10}^{-1}$	171	7.07
		TGV	150	3.8840 $\times {10}^{-2}$	6.4723	8.2237 $\times {10}^{-1}$	155	6.55
Gaussian	43	DTGV	250	2.0453 $\times {10}^{-2}$	8.6178	9.5974 $\times {10}^{-1}$	305	12.53
		TGV	950	2.4839 $\times {10}^{-2}$	6.9302	9.5698 $\times {10}^{-1}$	171	7.12
	37	DTGV	15	2.7995 $\times {10}^{-2}$	6.2017	9.3007 $\times {10}^{-1}$	128	5.36
		TGV	150	3.3061 $\times {10}^{-2}$	4.7572	8.9690 $\times {10}^{-1}$	118	4.73

Figure 8.

Test problem phantom: images restored with DTGV${}^2$ (top) and TGV${}^2$ (bottom).

Figure 9.

Test problem grass: images restored with DTGV${}^2$ (top) and TGV${}^2$ (bottom).

Figure 10.

Test problem leaves: images restored with DTGV${}^2$ (top) and TGV${}^2$ (bottom).

Figure 11.

Test problem carbon: images restored with DTGV${}^2$ (top) and TGV${}^2$ (bottom).

Figure 12.

Test problem carbon: difference images with DTGV${}^2$ (top) and TGV${}^2$ (bottom).

From the results, it is evident that the DTGV${}^2$ model outperforms the TGV${}^2$ one in terms of quality of the restoration. A visual inspection of the figures shows that the DTGV${}^2$ regularization is very effective in removing the noise, while for high noise levels the TGV${}^2$ reconstructions still exhibit noise artifacts. Finally, by observing the “Iters” column of the table, we can conclude that, on average, the TGV${}^2$ regularization requires less ADMM iterations to achieve a relative change in the restoration that is below the fixed threshold. However, the computational time per iteration is very small and also ADMM for the KL-DGTV${}^2$ regularization is efficient.

Finally, to illustrate the behaviour of ADMM, in Figure 13 we plot the RMSE history for the carbon test problem. A similar RMSE behaviour has been observed in all the numerical experiments.

Figure 13.

Test problem carbon: RMSE history for the KL-DGTV${}^2$ (continuous line) and KL-TGV${}^2$ (dashed line) models.

6. Conclusions

We dealt with the use of the Directional TGV regularization in the case of directional images corrupted by Poisson noise. We presented the KL-DTGV${}^2$ model and introduced a two-block ADMM version for its minimization. Finally, we proposed an effective strategy for the estimation of the main direction of the image. Our numerical experiments show that for Poisson noise the DTGV${}^2$ regularization provides superior restoration performance compared with the standard TGV${}^2$ regularization, thus remarking the importance of taking into account the texture structure of the image. A crucial ingredient for the success of the model was the proposed direction estimation strategy, which proved to be more reliable than those proposed in the literature.

Possible future work includes the use of space-variant regularization terms and the analysis of automatic strategies for the selection of the regularization parameters.

Author Contributions

All authors have contributed equally to this work. All authors have read and agreed to the published version of the manuscript.

Funding

This research was partially supported by the Istituto Nazionale di Alta Matematica, Gruppo Nazionale per il Calcolo Scientifico (INdAM-GNCS). D. di Serafino and M. Viola were also supported by the V:ALERE Program of the University of Campania “L. Vanvitelli”.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.