A New Design in Iterative Image Deblurring for Improved Robustness and Performance

Abstract

In many applications, image deblurring is a pre-requisite to improve the sharpness of an image before it can be further processed. Iterative methods are widely used for deblurring images but care must be taken to ensure that the iterative process is robust, meaning that the process does not diverge and reaches the solution reasonably fast, two goals that sometimes compete against each other. In practice, it remains challenging to choose parameters for the iterative process to be robust. We propose a new approach consisting of relaxed initialization and pixel-wise updates of the step size for iterative methods to achieve robustness. The first novel design of the approach is to modify the initialization of existing iterative methods to stop a noise term from being propagated throughout the iterative process. The second novel design is the introduction of a vectorized step size that is adaptively determined through the iteration to achieve higher stability and accuracy in the whole iterative process. The vectorized step size aims to update each pixel of an image individually, instead of updating all the pixels by the same factor. In this work, we implemented the above designs based on the Landweber method to test and demonstrate the new approach. Test results showed that the new approach can deblur images from noisy observations and achieve a low mean squared error with a more robust performance.

1. Introduction

The need for image deblurring is widely encountered as a pre-requisite in many applications, ranging from computer vision and pattern recognition to machine intelligence and morphologic quantification [1, 2, 3, 4, 5]. Often a blurred image needs to be restored to improve its sharpness before further steps, such as segmentation, detection, and classification, can be taken [6]. In computed tomography [7], electrical-capacitance and -impedance tomography [8, 9], remote sensing [10, 11], video surveillance [12], face recognition [13], and other applications, image deblurring amounts essentially to solving an ill-posed problem. Solutions for the ill-posed problem can be categorized into direct methods via some types of regularization [14, 15, 16, 17] and iterative methods via some updating and stopping rules [18, 19, 20, 21]. For both types of methods, challenges remain on how to make the deblurring process robust. In this work we focus on iterative methods by designing a new approach to improve the robustness of the iterative process.

There are several requirements for iterative methods to function successfully. First, the iterative process must be robust in terms of remaining non-divergent. To prevent divergence, we need to set the step size appropriately but the problem is that we usually do not know the right range to choose a step size. A step size that is set overly too small will make the iterative process very slow to converge while a large step size will lead to divergence. In this work we designed a technique to update the step size at each iteration, instead of keeping it fixed throughout all the iterations. Specifically, we updated the step size as a vector that is of the same size as an input image arranged in the lexicographic manner, rather than as a scalar as most existing techniques do. In other words, each pixel of the image is updated by its own step size, instead of by a one-size-fit-all step size. Second, as we will show in this paper, when the observed images are noisy, the noise term is often propagated through the iterative process in existing iterative methods. To address this problem, we designed a relaxed initialization setup to stop the noise term from propagating through the iterative process. Together, the two new designs result in a more robust iterative process for image deblurring.

To test and demonstrate the performance of the new approach, we chose the Landweber method as the underlying iterative technique [22] because it is a widely used technique and many existing techniques are modifications of the standard Landweber method [23, 24, 25, 26, 27, 28, 29]. Like many other iterative techniques, the performance of the Landweber method depends on several factors. First, an accurate estimation or knowledge about the underlying blurring function is needed for the method to achieve high performance. In this work we assume that the blurring function in the form of a point spread function (PSF) is known a priori. Second, the presence of noise may affect the performance of the method to a large extent, particularly, when noise may propagate from one iteration to the next. Third, the maintenance of stability or convergence of the iterative process is a key factor to ensure that the results are meaningful. In this work, we implemented our new design on the Landweber method to suppress the propagation of measurement noise and to maintain a higher degree of convergence through the iterations. We tested and compared the new approach with the standard Landweber method and showed that the new approach can perform much better in terms of obtaining deblurred images with less error and higher stability. We note that, though we used the Landweber method as the testbed in this work, the proposed approach may be adapted to other iterative methods in terms of incorporating the two new designs into the underlying methods.

The paper is organized as follows. Section 2 reviews the signal model and uses the Landweber method to point out how the iterative process is subject to the presence of noise and unstable iterations. Section 3 presents our new approach and the motivation behind each design. Section 4 provides test results and comparisons between the new approach and the Landweber method. Discussion and conclusions are provided in Section 5.

2. The Image Model and Existing Iterative Method

Here we present the model of image deblurring and the Landweber method first. We then discuss two factors that affect the performance of the Landweber method in practice. Given a noisy observation g and a known PSF A, we consider the solution to the following problem

$$g=Af+w$$ (1)

where f is the true image to be found and w is the measurement noise that is usually assumed to have a Gaussian distribution N (0, σ²). In matrix-vector notation, g, f, and w are column vectors of length L, and A is a matrix of L × L. The PSF A generally acts as a low-pass filter that makes the underlying image f blurry. To find a solution $\tilde{f}$ that minimizes the squared error (SE)

$$E={(g-A\tilde{f})}^T(g-A\tilde{f})$$ (2)

where the superscript T stands for transpose, the Landweber method tries to find $\tilde{f}$ iteratively by searching in the direction of negative gradient of Eq. (2). The standard Landweber method can be set up in two steps,

$$f^{[1]}=g+\beta A^T(g-Ag)$$ (3)

$$f^{[n]}=f^{[n-1]}+\beta A^T\left(g-A{f}^{[n-1]}\right)$$ (4)

for n = 2, 3, …, where the superscript [n] stands for the n-th iteration and β is a positive constant. Eq. (3) is the initialization of the method and Eq. (4) is the iterative process that stops when the change between two consecutive iterations is less than a preset threshold. To ensure the stability of the Landweber method, β must satisfy

$$\beta \in \left(0,\frac{2}{{\lambda }_{\text{max}}^2}\right)$$ (5)

where λ_max is the largest singular value of A. Usually A has a large condition number, defined as the ratio of its largest singular value over the smallest singular value. According to Eq. (3), the initialization of the Landweber method, when we replace g by Af + w, becomes

$$f^{[1]}=(Af+w)+\beta A^T((Af+w)-A(Af+w)).$$ (6)

The first factor that affects the performance of the Landweber method is the measurement noise, which may propagate through the iterations, particularly when SNR is relatively high. This can be seen if we rewrite Eq. (6) as

$$f^{[1]}=Af+\underset{\uparrow }{w}+\beta \left(A^{T}Af+A^{T}w-A^{T}AAf-A^{T}Aw\right)$$ (7)

From Eq. (7) we note that the first noise term w, pointed by the arrow, is not reduced by any low-pass filtering functions while the second and third w’s are filtered by A^T and A^T A, respectively. For the convenience of discussion, we call the first noise term w as the un-filtered noise. In addition of being un-filtered, this noise term w will propagate to the next iteration as seen next. We can rewrite Eq. (7) as

$$f^{[1]}=Af+\underset{\uparrow }{w}+\beta \left(A^{T}Af+A^{T}w-A^{T}AAf-A^{T}Aw\right)$$ (8)

$$=\underset{\uparrow }{w}+\left[Af+\beta \left(A^{T}Af+A^{T}w-A^{T}AAf-A^{T}Aw\right)\right]$$ (9)

$$\equiv \underset{\uparrow }{w}+H(A,f,W)$$ (10)

where in Eq. (10) we use H(A, f, w) to represent [Af + β(A^T Af + A^T w − A^T AAf − A^T Aw)] of Eq. (9). Using Eq. (10) to obtain f^[2] we will have

$$f^{[2]}=f^{[1]}+\beta A^T\left(g-A{f}^{[1]}\right)$$ (11)

$$=\underset{\uparrow }{w}+H(A,f,w)+\beta A^T(g-A(w+H(A,f,w))).$$ (12)

From Eqs. (7) and (12), we can deduce that for all the sequential iterations the un-filtered w will remain un-processed and propagate to the next iteration. Therefore, a logical solution is to stop the propagation of the un-filtered w. We will describe our approach to this problem in the next section.

The second factor that affects the Landweber method in practice is how to determine an appropriate β whereas a β that is too large will cause divergence and a β that is too small will result in slow convergence. The second term on the right-hand side of Eq. (4), βA^T (g − Af^[n]), can be considered as the step size of the iteration. More precisely, we can consider β as the step size because it is the only term that can be set by a user. We next use an example to illustrate how β may determine the convergence of the iterative process. For the purpose of illustration, we chose a 1D signal f as shown in Figure 1(a) as the first example. The length L of the signal was 100. The blurred noisy observation is shown in Figure 1(b), with an SNR of 27 dB. At first, we set β to 0.20 and used the standard Landweber method to solve for $\tilde{f}$. The result is shown in Figure 1(c), from which we can see that the restored $\tilde{f}$ approximated the true f well. As a comparison, we next set β to 0.23 and repeated the standard Landweber method. The result is shown in Figure 1(d), which does not resemble the true f, indicating that the Landweber method diverged, even though the SNR of 27 dB was considerably high.

Figure 1:

(a) True signal f. (b) Observed g with noise at SNR of 27 dB. (c) Restored signal by the Landweber method with β = 0.20. (d) Restored signal by the Landweber method with β = 0.23 as the process diverged.

To confirm whether the Landweber method converged or diverged in the above cases, we plotted MSE (mean squared error) in Figure 2. Here MSE is defined as

$$\text{MSE}=\frac{{\Vert f^{[n]}-f\Vert }_2^2}{L}$$ (13)

where L is the length or number of pixels of the signal. The MSEs corresponding to Figures 1(c) and (d) are shown in Figures 2(a) and (b), respectively. Indeed, we can observe that, when β was 0.20, the MSE decreased with the number of iterations. However, when β was increased to 0.23, the MSE increased significantly with the number of iterations, pointing to divergence of the method. Though MSE measurement confirmed that a large β will cause the Landweber method to diverge, as we expected, it does not offer an insight into whether all the pixels of f^[n] contributed equally to the divergence. So we defined, at each iteration n, a pixel-wise error matrix E between f^[n] and the true signal f such that

$$E(i,n)={\left(f^{[n]}(i)-f(i)\right)}^2$$ (14)

for i = 1, …, L and n = 1, 2, … where the superscript n stands for the n-th iteration. In our example, because f had 100 pixels and we ran the Landweber method for 60 iterations, we produced a 3D surface plot of 100 × 60 showing the behavior of E at each iteration. Figure 2(c) shows E corresponding to β = 0.20. From the surface plot we can make two observations. First, as n incremented, the pixel-wise error E decreased overall, indicating that the Landweber method converged at β = 0.20. Second, not all the pixels of f^[n] contributed equally to the MSE. From the surface plot we note that certain pixels, in this case the pixels located in the middle of each f^[n], had larger pixel-wise error at the early iterations. As n incremented, the pixel-wise error of these pixels reduced. For comparison, in Figure 2(d) we showed the surface plot of E corresponding to β = 0.23. From this figure we can observe that not all the pixels contributed to the divergence of the Landweber method. It can been seen that only the pixels located in the middle of each f^[n] contributed significantly to the divergence when n became large. The above observation indicates that using a scalar step size β to apply to all the pixels of f^[n−1] may not be optimal as not all of pixels contribute equally to the divergence of the iterations. If we can calculate E we can explore its behavior through the iteration to avoid divergence.

Figure 2:

(a) MSE of Figure 1(c). (b) MSE of Figure 1(d). (c) Surface plot of pixel-wise error matrix E for β = 0.20. (d) Surface plot of pixel-wise error matrix E for β = 0.23.

In reality, because we do not know f, we cannot calculate error matrix E. However, we can calculate the difference between two consecutive iterative results f^[n] and f^[n−1] and look for clues for convergence or divergence from this difference. So in replacement for E, we defined, at each iteration n, a pixel-wise difference matrix D between f^[n] and f^[n−1]

$$D(i,n)={\left(f^{[n]}(i)-f^{[n-1]}(i)\right)}^2$$ (15)

for i = 1, …, L and n = 2, 3, …. Figures 3(a) and (b) show the surface plots of the D matrices corresponding to β of 0.20 and 0.23, respectively. When we compare Figures 2(c) and (d) to Figures 3(a) and (b), respectively, we observe that, although the plots have different numerical values, the shapes of the plots are very similar in both the convergent and divergent cases. In other words, from Figures 3(a) and (b), and Figure 3(b) in particular, we can observe that not all the pixels contributed equally to the divergence of the Landweber method and it makes sense to set β, instead of as a scalar, as a vector of the same size of the underlying image arranged in the lexicographic manner to adjust each pixel individually. This motivated us to look for an alternative scheme such that β can be a vectorial step size to apply pixel-wise on f^[n].

Figure 3:

(a) Surface plot of pixel-wise difference matrix D for β = 0.20. (b) Surface plot of pixel-wise difference matrix D for β = 0.23.

3. Proposed Approach

We propose two designs to the standard Landweber method and similar iterative methods to stop the propagation of the un-filtered noise term and to improve the stability of the iterative process. The first design is to relax the initialization of the iterative process. The second design is to introduce an adaptive vectorial pixel-wise step size for the iterative process.

Using the standard Landweber method as the underlying technique, the steps of our proposed approach are as follows:

$$f^{[0]}=g$$ (16)

$$f^{[1]}=\beta A^T(g-Ag)$$ (17)

$$\text{FOR}n=2,3,\dots$$ (18)

$$h=\frac{{\left(f^{[n-1]}-f^{[n-2]}\right)}^2}{\Vert f^{[n-1]}-f^{[n-2]}{\Vert }^2}$$ (19)

$${\Theta }^{[n-1]}=\beta *{\left(1+e^h\right)}^{-1}$$ (20)

$$f^{[n]}=f^{[n-1]}+{\Theta }^{[n-1]}*\left[A^T\left(g-A{f}^{[n-1]}\right)\right]$$ (21)

The iteration stops when the difference between two consecutive f^[n] and f^[n−1] is lower than a small threshold. Eq. (19) represents a vectorial operation. It computes h, which is a vector storing the inter-iteration changes and is used to calculate the adaptive pixel-wise updates of the step size. In Eqs. (20) and (21) * stands for point-wise multiplication. Comparing Eq. (17) with Eq. (3), the new approach removes the un-filtered noise term w from the initialization but still uses the scalar step size β in the iterative process. If we replace g by Af + w in Eq. (17) we obtain

$$f^{[1]}=\beta A^T(Af+w-A(Af+w)).$$ (22)

We can see that there is no un-filtered noise term w in f^[1] and, what’s more, no noise term will propagate to the next iteration without being filtered by A^T and A. For example, we can obtain f^[2] as

$$f^{[2]}=f^{[1]}+\beta A^T\left(g-A{f}^{[1]}\right)$$ (23)

$$=\beta A^T(Af+w-A(Af+w))+\beta A^T\left(g-A{f}^{[1]}\right)$$ (24)

It is clear from the above equation that there is no un-filtered w in the iteration.

In the new approach, Θ^[n−1] is a column vector of length L that functions as pixel-wise step size. Here we note that the term inside the square brackets of Eq. (21) is a column vector of length L. The effect of Θ^[n] is similar to that of β of Eq. (17) but has a vectorial form of ${\Theta }^{[n]}={\left[{\theta }^{[n]}{}_1,{\theta }^{[n]}{}_2,\dots ,{\theta }^{[n]}{}_L\right]}^T$ where each element will be determined adaptively starting at iteration 2. The essence of the second modification is to convert the scalar step size β to a vectorial step size Θ of which, at iteration n, each element is inversely dependent on the difference between f^[n−1] and f^[n−2]. The effect of building Θ in this way is to penalize the pixels that have a large change from one iteration to the next. As can be seen from Figures 3(b) and 2(d), such pixels contribute more to the instability of the Landweber method. It is, therefore, logical to apply smaller step size θ’s to them, e.g., by multiplying (1 + e^h)⁻¹ in Eq. (20). In other words, Eq. (20) adaptively assigns large θ’s to pixels that change less or moderately from the previous iterations but smaller θ’s to pixels that tend to have large changes from the previous iterations to slow them down.

4. Results

At first, we used the new approach to repeat the deblurring example of Figure 1(b) with a β of 0.23. At β = 0.23, as shown in Figure 1(d) and Figure 2(b), the standard Landweber method diverged and generated a large MSE. For comparison, the restored signal and MSE given by the new approach at β = 0.23 were shown in Figure 4. The restored signal resembled the true signal closely and maintained convergence. Its final MSE at iteration 60 was 0.031. We then increased β to 0.40 in the new approach and plotted its results in Figures 4(c) and (d). The MSE at iteration 60 was 0.029. It can be seen that, even with a larger β, the new approach was able to obtain a stable output and converge to a good result. Comparing Figures 4(b) with (d), it is interesting to note that at a larger β = 0.40 the MSEs of the new approachs at the first several iterations were higher than the MSEs corresponding to β = 0.23, but as the iterations progress, the MSEs at β = 0.40 quickly decreased. When the iterations stopped at 60, the final MSEs corresponding to β = 0.40 and 0.23 were almost identical.

Figure 4:

(a) Result given by the new approach to restore the observation of Figure 1(b) with β = 0.23. Note the restore signal (red) resembled the true signal (blue). (b) MSE of (a) with a final MSE of 0.031 at iteration 60. (c) Result given by the new approach at β = 0.40. (d) MSE of (c) with a final MSE of 0.029 at iteration 60.

We then applied the new approach to deblur images and compared its results with those given by the Landweber method. As the first example, we applied the new approach on a phantom image, as shown in Figure 5(a). Its noisy blurry observation is shown in Figure 5(b) with an SNR of 6.5 dB. In this work we define SNR as

$$\text{SNR}=10\text{log}\frac{{\Vert S\Vert }_2^2/K}{{\sigma }_w^2}(\text{dB})$$ (25)

where S is the signal, K is the number of pixels in S, and ${\sigma }_w^2$ is the variance of the noise. We used the standard Landweber method with a β of 0.2 to deblur the image. The result is shown in Figure 5(c), which can be seen to have only moderately deblurred the image. The MSE is shown in Figure 5(d) and the final MSE was 0.175. The deblurred image given by the new approach is shown in Figure 5(e), and we can see that the result is much cleaner and resembles more closely the true image. The MSE is shown in Figure 5(f) and the final MSE was 0.09. Comparing Figures 5(c) with (e) we note that the result given by the new approach resembles more closely the original image, particularly for the weak object located at the lower right corner, while the result from the standard Landweber method contained a large amount of noise and the weak object was not highly visible. It is interesting to note that, from the MSE plots, the new approach initially had a large MSE (Figure 5(f)) but its MSE reduced quickly. When the iteration stopped, the new approach had a much lower MSE of 0.09 of Figure 5(f), as compared to the final MSE of 0.175 of Figure 5(e). The reason for the initial large error of the new approach is that the initialization of the new approach aims to prevent the propagation of the noise term ω rather than minimizing the squared error ${(g-A\tilde{f})}^T(g-A\tilde{f})$ from the very beginning. As we reasoned in Eqs. (7–12), this new initialization, though generates a larger error at iterations 0 and 1, achieves smaller errors in the late iterations.

Figure 5:

(a) A phantom image f. (b) The noisy observation g of which the SNR was 6.5 dB. (c) Image restored by the standard Landweber method at β = 0.2. (d) MSE corresponding to (c) at each iteration. The final MSE was 0.175. (e) Image restored by the new approach with the same β. (f) MSE corresponding to (e) at each iteration. The final MSE was 0.09.

Next, we tested the new approach on some real test images. The original test image “boat” is shown in Figure 6(a). A small amount of white Gaussian noise was added to a blurred version of the original, as shown in Figure 6(b). We applied the standard Landweber method with β of 0.1 to deblur the image. The deblurred result and the MSE are shown in Figures 6(c) and (d), respectively. The deblurring result and the MSE given by the new approach, using the same β, is shown in Figure 6(e) and (f), respectively. We can observe that the output of the new approach was sharper than the result of the Landweber method. Two MSE plots again revealed that, although the new approach had higher MSE at the beginning, it achieved a much lower MSE overall when the iterations stopped. To check on the effect of relaxed initialization on the overall performance of the new approach, we tentatively switched from the relaxed initialization to the standard initialization step of Eq. (3) while we maintained the adaptive vectorial step size to deblur the same image. The deblurring result and the MSE are shown in Figure 7. We observed that using the standard initialization with the adaptive vectorial step size improved upon the Landweber method, as seen in Figures 6(c) and (d), but did not achieve the performance as good as we observed in Figures 6(e) and (f). In fact, the final MSE of 259 in Figure 7(b) was much higher than the final MSE of 178 given by the new approach in Figure 6(f).

Figure 6:

(a) The original “boat” image. (b) Noisy blurred image. SNR was 19 dB. (c) Deblurred image by the standard Landweber method. (d) MSE of (c). The final MSE was 297. (e) Deblurred image by the new approach. (f) MSE of (e). The final MSE was 178.

Figure 7:

(a) Deblurred image by the new approach by using the standard initialization of Eq. (3) instead of the relaxed initialization. (b) MSE of (a). The final MSE was 259.

Next, we compared the performance of the new approach and the Landweber method on a second test image. The original test image “bridge” is shown in Figure 8(a). The noisy blurred image, with an SNR of 10 dB, is shown in Figure 8(b). Comparing the deblurred image given by the Landweber method, Figure 8(c), with that given by the new approach, Figure 8(e), we can observe that the new approach gave a better result. Comparison between Figures 8(d) and (f) confirmed that the new approach had a much lower MSE of 519 than the MSE of 1632 of the Landweber method, which actually started to diverge at around iteration 50. To test how the new approach performed for different β, we first doubled the value of β from 0.1 used in Figure 8(b) to 0.2 and then applied the new approach and the Landweber method. Figure 9 demonstrated that that the Landweber method actually diverged and produced a final MSE of 2037. Even the minimum MSE achieved by the Landweber method at iteration 22 was 1418, higher than the final MSE given by the new approach, which was 674. We then reduced β to 0.05 and compared the two methods. Figure 10 shows the resulting deblurred images and the MSEs. At a smaller β the Landweber method started to diverge at a later iteration, Figure 10(b) but the minimum MSE of 1404 it achieved was still significantly higher than the MSE of 539 produced by the new approach.

Figure 8:

(a) The original “bridge” image. (b) Noisy blurred image. SNR was 10 dB. (c) Deblurred image by the Landweber method with a β of 0.1. (d) MSE of (c). The final MSE was 1632. (e) Deblurred image by the new approach. (f) MSE of (e). The final MSE was 519.

Figure 9:

(a) Deblurred image by the Landweber method with a β of 0.2. (b) MSE of (a). The final MSE was 2037. (c) Deblurred image by the new approach. (d) MSE of (c). The final MSE was 674.

Figure 10:

(a) Deblurred image by the Landweber method with a β of 0.05. (b) MSE of (a). The final MSE was 1404. (c) Deblurred image by the new approach. (d) MSE of (c). The final MSE was 539.

We conducted more comparisons using other images and different PSFs, A₁ and A₂, of which A₁ has a moderate smoothing effect with a condition number of 3.85 × 10⁶ and A₂ has more severe smoothing effect and a larger condition number. Table 1 shows the comparison between the Landweber method and the new approach on test image “Barbara” for 200 iterations in different scenarios. Scenarios 1 and 2 show the results of using PSF A₁ at SNRs of 18 and 8 dB. In scenario 1 the new approach had a lower MSE than the Landweber method. In scenario 2, when the SNR was reduced almost by half to 8 dB, the Landweber method started to diverge while the new approach still converged, producing a lower MSE. In scenarios 3 to 5, we kept the SNR the same but changed β from 0.2 to 0.1 and then to 0.05 to evaluate how the two methods compared with different β’s. The comparison shows that the Landweber method started to diverge in scenarios 3 and 4. In all three scenarios the new approach converged and had lower MSEs. In scenarios 6 and 7, we applied a smoother PSF A₂ to blur the original image and tested the two methods under different SNRs. The new approach performed better in each scenario. Figure 11 shows a representative example comparison between the two methods in scenario 1. Using test image “Barbara” we then evaluated how the new approach behaved for a fixed β but different SNRs. Figure 12 shows the MSEs of the new approach, in blue, and that of the Landweber method, in red, for β = 0.05 at SNRs of 20 dB, 14 dB, 10 dB, and 8 dB. It can seen that, though the new approach had larger MSEs at the initial iterations, its results quickly outperformed that of the Landweber method. When SNRs were lower at 10 and 8 dB, the Landweber method started to diverge, while the new approach was still convergent, indicating that the new approach has a larger range of stability in terms of selecting β than the Landweber method. We repeated the above comparison by doubling β to 0.1 and reproduced the MSEs in Figure 13. It is interesting to note that the Landweber method started to diverge even when the SNR was high.

Table 1:

Comparison of deblurring test image “Barbara” by the Landweber method and the new approach in MSE. † means that the iteration started to diverge.

Scenario	1	2	3	4	5	6	7
PSF	A1	A1	A1	A1	A1	A2	A2
SNR (dB)	18	8	4	4	4	19	9
β	0.1	0.1	0.2	0.1	0.05	0.1	0.1
Landweber method (MSE)	507	2157†	620	5457†	4080†	489	1582†
New approach (MSE)	371	753	404	1518	1116	398	530

Figure 11:

(a) Test image “Barbara”. (b) Blurred noisy observation with an SNR of 19 dB. (c) Deblurred image by the Landweber method. (d) MSE of (c). (e) Deblurred image by the new approach. (f) MSE of (e).

Figure 12:

(a-d) The MSEs of deblurred test image “Barbara” given by the new algorithm (blue) and the Landweber method (red) under SNRs of 20, 14, 10, and 8 dB, respectively, and β = 0.05.

Figure 13:

(a-d) The MSEs of deblurred test image “Barbara” given by the new algorithm (blue) and the Landweber method (red) under SNRs of 20, 14, 10, and 8 dB, respectively, and β = 0.1

We also tested the new approach on image “saturn”, Figure 14(a), and show the results in Figure 14. The SNR of the noisy observation was 13 dB, Figure 14(b). From the deblurred images by the Landweber method, Figure 14(c), and the new approach, Figure 14(d), at β of 0.05 for both methods, we can observe that the new approach generated much cleaner and sharper results. For instance, the three bright spots located to the left of the original image can be clearly observed in Figure 14(d) but less so in Figure 14(c). The final MSEs of the Landweber method and the new approach were 375 and 69, respectively.

Figure 14:

(a) The original image “saturn”. (b) The noisy blurred observation of (a). (c) Deblurred image by the Landweber method. (d) Deblurred by the new approach. (e) MSE of (c). (f) MSE of (d).

Tables 2, 3, and 4 report the results when we repeated the comparisons for test images “house”, “cameraman”, and “Lena”. In all these comparisons the new approach generated results with lower MSEs and, in most cases when the Landweber method started to diverge, the new approach still converged.

Table 2:

Comparison of deblurring test image “House” by the Landweber method and the new approach in MSE. † means that the iteration started to diverge.

Scenario	1	2	3	4	5	6	7
PSF	A1	A1	A1	A1	A1	A2	A2
SNR (dB)	19	10	5	5	5	21	11
β	0.1	0.1	0.2	0.1	0.05	0.1	0.1
Landweber method (MSE)	265	1920†	8586†	5257†	3864†	241	1341
new approach (MSE)	144	533	2864†	1311	1050	164	195

Table 3:

Comparison of deblurring test image “Cameraman” by the Landweber method and the new approach in MSE. † means that the iteration started to diverge.

Scenario	1	2	3	4	5	6	7
PSF	A1	A1	A1	A1	A1	A2	A2
SNR (dB)	18	9	4	4	4	20	10
β	0.1	0.1	0.2	0.1	0.05	0.1	0.1
Landweber method (MSE)	423	2088	8729†	5417†	4043†	503	1607
new approach (MSE)	332	722	3027†	1502	1195	448	581

Table 4:

Comparison of deblurring test image “Lena” by the Landweber method and the new approach in MSE. † means that the iteration started to diverge.

Scenario	1	2	3	4	5	6	7
PSF	A1	A1	A1	A1	A1	A2	A2
SNR (dB)	18	9	4	4	4	20	10
β	0.1	0.1	0.2	0.1	0.05	0.1	0.1
Landweber method (MSE)	248†	1899†	8550†	5200†	3819†	227	1325†
new approach (MSE)	117	499	2808†	1263†	912	134	266

We compared the speed of computations for the new approach and the standard method and found that the two implementations require approximately the same time to complete the same number of iterations. However, considering the fact that the new approach can achieve the same deblurring result in terms of MSE at earlier iterations, the new approach may require less time to complete in practice. In summary, our tests demonstrated that the new approach deblurred noisy images successfully and achieved lower MSEs than the standard Landweber method under a variety of conditions.

5. Discussion and Conclusions

The essence of this work is the two new designs aims to improve the robustness of iterative methods for image deblurring. The first design, relaxed initialization, aims to stop the propagation of the un-filtered noise in the iterative process. The relaxed initialization of the new approach allows deblurring of images with relatively low SNR, a distinct advantage in real world applications. The adaptive pixel-wise updates of the step size enable the new approach to accommodate a large range of selection for the initial step size yet still be able to converge to results with lower MSEs. The MSE analysis confirms that, while the new approach generally produces a high MSE at the beginning, its MSE quickly decreases and settles to a much lower MSE than the standard Landweber method.

While in this work we used the Landweber method as the underlying iterative algorithm and compared the new approach against the Landweber method, we note that, as many iterative deblurring algorithms have the similar initialization and a scalar step size in iterations, the new approach can be integrated into other iterative methods. In future work, we will test the new design by extending it to other iterative methods and compare the performance of the underlying iterative methods with and without relaxed initialization and pixel-wise step size update. We will also explore different designs of the pixel-wise update for achieving more robust and accurate performance in image restoration. As we discussed in the results section, the extra computational cost of pixel-wise updates of the step size in each iteration is minimal. And because the new approach can achieve better results at fewer iterations, the new approach may actually take up less time than the underlying algorithms it is built on.

In implementation, the new approach does not introduce extra parameters and in many cases allows a large range of β to choose from. The new approach is particularly powerful when SNR is relatively low and the image is moderately to severely blurred. The standard Landweber method can perform well when the amount of noise is low and blurriness is not severe. We note that the calculation of the adaptive vectorial step size in Eq. (20) is not the only choice. Other designs to determine the adaptive vectorial step size is possible and may depend on the underlying image deblurring problems that one encounters. So far, in this work, we tested the idea of using the changes of each point of the previous two iterations to build vector Θ in Eq. (19). This design of Θ is independent of PSF A, but it is probable that there exists better designs of Θ to achieve the goals of faster converge, higher robustness, and lower MSEs. Also in future work we will investigate the interaction among A, the changes of f during iterations, and the design of Θ to determine whether an optimal Θ can be derived from analysis of the structure and other features of A to achieve the above goals.

In this work we assumed that the PSF of the blurring effect is known beforehand. When the PSF is not known a priori, a blind deconvolution scheme is needed to estimate both the PSF and the underlying image [30, 31, 32, 33, 34]. It is possible to extend the idea of relaxed initialization and pixel-wise adjustment of the weight factor to blind deconvolution.

Highlights.

• A new approach is designed to improve the robustness of iterative processes for image deblurring

• A relaxed initiation of iterative processes is designed to prevent the propagation of the noise term in the observed images

• A pixel-wise update of the step size for the iterative process is introduced to prevent divergence in computation

• The new method can restore blurred images when the signal-to-noise ratio is low

Author Biographies

Taihao Li, PhD, received his PhD in information system engineering from University of Tokushima, Japan, in 2006. During 2006–2011, he was a postdoc researcher at Harvard University. He is now a professor with College of Medical Instruments, Shanghai University of Medicine & Health Sciences, China. His research interests include affective computing, image processing, and artificial intelligence.

Huai Chen, MD, is a Professor at Department of Radiology of the First Affiliated Hospital of Guangzhou Medical University. His research interests are on radiology research, including quality of image and application of computational methods on radiology data.

Min Zhang, PhD, is a post-doctoral fellow at the Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School. Her research interests are in different geometry, image processing, machine learning, and bioinformatics.

Xinhua Cao, PhD, is an Instructor of Radiology at Harvard Medical School, Nuclear Medicine Information System Application Manager at Boston Children’s Hospital. His interests include research and development of advanced algorithms and computational tools to improve the efficiency and productivity of clinical practice and research in nuclear medicine and radiology.

Shupeng Liu, PhD, is an Associate Professor in Biomedical Engineering at School of Communication and Information Engineering, Shanghai University. His research interests focus on the biomedical applications research of nanosciences, information and electronics, such as cell and pharmaceutical spectra analysis, optical probe, and biomedical devices development.

Shunren Xia, PhD, is a Professor at the Department of Biomedical Engineering, Zhejiang University, Hangzhou, China. Dr. Shunren Xia received PhD from Southeast University in 1992. He is the author or coauthor of more than 160 papers in medical imaging and image processing. His research interests include machine learning, image processing, medical imaging, computer aided diagnosis on medical images.

Geoffrey Young, MD, is an Assistant Professor of Radiology at Harvard Medical School, Associate Neuroradiologist at Brigham and Women’s Hospital and the Dana Farber Cancer Institute in Boston MA. His research interests include advanced MR methods of tumor assessment including perfusion imaging, diffusion imaging, quantitative MR, and machine learning.

Xiaoyin Xu, PhD, is an Associate Professor at the Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School. His research interests are in computational science and image processing.