Adaptive continuation based smooth $l_0$-norm approximation for compressed sensing MR image reconstruction

Abstract.

Purpose

There are a number of algorithms for smooth $l_0$-norm (SL0) approximation. In most of the cases, sparsity level of the reconstructed signal is controlled by using a decreasing sequence of the modulation parameter values. However, predefined decreasing sequences of the modulation parameter values cannot produce optimal sparsity or best reconstruction performance, because the best choice of the parameter values is often data-dependent and dynamically changes in each iteration.

Approach

We propose an adaptive compressed sensing magnetic resonance image reconstruction using the SL0 approximation method. The SL0 approach typically involves one-step gradient descent of the SL0 approximating function parameterized with a modulation parameter, followed by a projection step onto the feasible solution set. Since the best choice of the parameter values is often data-dependent and dynamically changes in each iteration, it is preferable to adaptively control the rate of decrease of the parameter values. In order to achieve this, we solve two subproblems in an alternating manner. One is a sparse regularization-based subproblem, which is solved with a precomputed value of the parameter, and the second subproblem is the estimation of the parameter itself using a root finding technique.

Results

The advantage of this approach in terms of speed and accuracy is illustrated using a compressed sensing magnetic resonance image reconstruction problem and compared with constant scale factor continuation based SL0-norm and adaptive continuation based $l_1$-norm minimization approaches. The proposed adaptive estimation is found to be at least twofold faster than automated parameter estimation based iterative shrinkage-thresholding algorithm in terms of CPU time, on an average improvement of reconstruction performance 15% in terms of normalized mean squared error.

Conclusions

An adaptive continuation-based SL0 algorithm is presented, with a potential application to compressed sensing (CS)-based MR image reconstruction. It is a data-dependent adaptive continuation method and eliminates the problem of searching for appropriate constant scale factor values to be used in the CS reconstruction of different types of MRI data.

1. Introduction

Sparse signal reconstruction in underdetermined systems has gained wide attention in a variety of applications, such as compressed sensing (CS), image inpainting, image super-resolution, and image deconvolution.^1–4 The main task in the sparse reconstruction problem is to find the sparsest solution of the underdetermined system of linear equations. Mathematically, one needs to solve the $l_0$-norm minimization to obtain the sparsest solution.^3,5,6 The $l_0$-norm represents the total number of non-zero elements in the solution. But the $l_0$-norm function is highly discontinuous and non-differentiable, which makes the $l_0$-norm problem hard to solve. Besides, a common goal in all sparse reconstruction problems is to minimize the $l_0$-norm under a set of linear constraints, wherein the number of observations is less than the dimension of the desired solution. Because there exist a large number of permutations for a sparse signal of length $N$ to be $K$-sparse, the computational complexity required to find the optimal permutation is very high.⁷ Despite the fact that an optimal solution of the sparse reconstruction problem is NP-hard,⁵ there exist a large number of methods that provide an approximate solution to solve the problem. These methods can be categorized into combinatorial methods:^8–10 $l_1$-norm regularization methods,^11–15 $l_p$-norm $(0<p<1)$ regularization methods, and smooth $l_0$-norm (SL0) approximation methods.¹⁶ The motivation for this work follows from the findings in Refs. 16 and 17 that bring out the merits of the SL0 approach for addressing the problem of reducing the burden of $l_0$ search using a gradient descent algorithm with increased robustness against noise and faster than methods based on $l_1$-norm minimization.

The basic idea in the SL0 approach is to approximate the $l_0$-norm by a suitable continuous function that approximates the Kronecker delta function with an additional constraint that the value of the function becomes close to one when the magnitudes of the sparse coefficients lie below the value pre-assigned to a parameter $\sigma$ and then minimize it using the gradient descent method.^16,17 The quality of such an approximation can be controlled by fixing the value of the parameter $\sigma$. For small values of $\sigma$, the function is highly nonsmooth and hence the minimization often does not result in the true solution. Therefore, the minimization is often accomplished using a “decreasing” sequence of the $\sigma$ values in every iteration, which is similar to the continuation trick used in the iterative shrinkage-thresholding algorithm (ISTA) based approach. The reconstruction performance of ISTA and its variants^11,13,18,19 depends on the value of the regularization parameter $\beta$ that balances data fidelity and sparsity. To enforce sparsity, ISTA iterations are usually performed using larger values of $\beta$ initially and continuously reduced by a factor $0<\mu <1$ to achieve rapid convergence. The rate of convergence depends on the initial value of ${\beta }^{(0)}$ and the scale factor $\mu$, whereas the reconstruction quality depends on the final value of ${\beta }^{(f)}$. The optimal values of $\beta$ and $\mu$ are data-dependent and change dynamically in each iteration.

Similar to ISTA based continuation, the idea of continuation applied to the SL0 method consists of continuously applying a set of gradient projections whereby the solution is iteratively updated by one-step gradient descent of the SL0 approximating function (gradient step) using an initial high value of $\sigma$, followed by projecting the result onto the admissible solution set (projection step) consisting of a number of sub-iterations. Each block consisting of gradient descent update steps followed by the projection steps with $L$ sub-iterations can be treated as an intermediate optimization step. For each step, the parameter value that minimizes the approximating function in the previous iteration acts as the initial parameter value for the optimizer in the current iteration. The number of sub-iterations for the intermediate optimizer can be kept small due to the incremental changes in $\sigma$ over the main iterations and the fact that there is no stringent requirement of an early accurate minimizer. Proceeding in this fashion, it is possible to track the global minima without getting trapped into the local minima.

Until now, there is no theoretical justification for determining how much “gradual” should the $\sigma$ values decrease, and it remains an open problem for investigation. This is in some ways implicitly related to finding any sequence that guarantees escaping from local minima for the Gaussian family of SL0 functions. Although the continuation based SL0 approximation is faster, the final reconstruction error is parameter dependent. Application to MRI compressed sensing reveals the significance of choosing the proper value of ${\sigma }_0$ and the rate of decrease of $\sigma$, as both influence the final reconstruction error and the speed of convergence in a data dependent manner. These issues, mathematically difficult but essential for robust reconstruction and faster convergence, are currently addressed by adaptively estimating $\sigma$ in the form of two coupled sub-problems. We exploit the idea that distance between gradient descent updates of successive iterations should be minimized to attain the global minimum. To achieve this, we solve two subproblems alternatively. One is a sparse regularization-based subproblem, which is solved with a precomputed $\sigma$, and the second subproblem is the estimation of $\sigma$ using a root finding technique. Implementation of the proposed method consists of two loops: the “outer” loop is the loop in which $\sigma$ is estimated, and the “inner” loop is the one in which SL0 function is iteratively minimized for the chosen value of $\sigma$.

To evaluate the performance of the proposed method, we have carried out a number of experiments with different types of MR images and also compared them with state-of-the-art methods. From the experimental results, it is observed that the proposed method shows better performance in terms of reconstruction quality due to data-dependent automated adaptive parameter selection and less computational time since it requires a smaller number of outer iterations that involve simple matrix and vector operations.

The rest of the paper is organized as follows. Section 2 summarizes a brief overview of CS based MR image reconstruction and automated iterative shrinkage-thresholding based approach followed by continuation based SL0-norm minimization method and the proposed adaptive SL0-norm minimization technique. Section 3 includes the experimental evaluation with various MR datasets and comparative analysis. Finally, the conclusion of the work is given in Sec. 4.

2. Theory

2.1. CS MRI Reconstruction Model

In this paper, we demonstrate the SL0 minimization based CS MR image reconstruction. MRI is one of the widely used medical imaging modalities. Radiologists frequently preferred MRI over other imaging techniques due to the non-ionizing radiation and high contrast soft-tissue imaging capability. However, it has a fundamental limitation of slow imaging, which restricts its applications. The application of CS in MRI significantly reduces the data acquisition time by acquiring a smaller number of $k$-space samples.^20,21 The compressively sensed $k$-space data $K_u\in {\mathbb{C}}^{(M\times 1)}$ of the MR image $U\in {\mathbb{R}}^{(M\times 1)}$ is given as

$$K_u=\mathrm{\Phi }FU=F_{u}U,$$ (1)

where $\mathrm{\Phi }\in {\mathbb{R}}^{(M\times N)}$ is a binary matrix prepared by selecting the rows from an $M\times N$ identity matrix according to the undersampling pattern; $F\in {\mathbb{C}}^{(N\times N)}$ is the forward Fourier transform operator, and $F_u\in {\mathbb{C}}^{(M\times N)}$ represents the undersampled Fourier operator corresponding to the undersampling scheme and $M\ll N$. In the noiseless scenario, the reconstruction problem can be formulated as a non-convex optimization problem:

$$\underset{U}{\min }{\Vert U\Vert }_0\text{subject to}{K}_u=F_{u}U,$$ (2)

where $\Vert {\Vert }_0$ is the $l_0$-norm that is defined as the number of non-zero elements. Practically, MR images are not sparse in the spatial domain. Nevertheless, these images are highly compressible in the transform domains, such as the wavelet domain. Therefore, the problem defined in Eq. (2) can now be represented as

$$\underset{U}{\min }{\Vert \mathrm{\Psi }U\Vert }_0\text{subject to}{\Vert K_u-F_{u}U\Vert }_2^2\le \epsilon ,$$ (3)

where $\mathrm{\Psi }\in {\mathbb{C}}^{(N\times N)}$ represents the sparsifying transform operator, $\epsilon$ is a small positive constant that represents an acceptable noise level. The term ${\Vert \mathrm{\Psi }U\Vert }_0$ denotes the sparsity of the underlying image in the transform domain and ${\Vert K_u-F_{u}U\Vert }_2^2$ is a data fidelity term. Since $M\ll N$, the above problem is an underdetermined system of linear equations, and there exist an infinite number of solutions. Based on the sparse nature of the wavelet coefficients, the optimal solution can be identified as the sparsest set of wavelet coefficients that satisfy Eq. (3). Since the optimization problem defined in Eq. (3) is NP-hard, there is no direct way to find the solution. Existing methods either resort to greedy approaches that exploit the relationship between sparsity and incoherence or rely on approximating the cost function by a representative surrogate model. Among the latter, $l_1$-norm based approximation is the most common wherein the minimization of the $l_1$-norm based approximating function can be easily implemented using ISTA.

2.2. Iterative Shrinkage-Thresholding-Based $l_1$-Norm Minimization

The unconstrained version of the equivalent $l_1$-norm regularization form of the optimization problem in Eq. (3) can be written as

$$\underset{U}{\min }\frac{1}{2}{\Vert K_u-F_u{\mathrm{\Psi }}^T\omega \Vert }_2^2+\beta {\Vert \omega \Vert }_1,$$ (4)

where $\omega =\mathrm{\Psi }U$ represents the sparse coefficients of the image in the transform domain; ${\Vert ·\Vert }_1$ is $l_1$-norm; $\beta$ is a small positive regularization parameter to balance between the data fidelity and sparsity terms. Iterative thresholding-based methods use a shrinkage function:

$$\begin{gathered} {\omega }^{(k+1)}={\text{soft}}_{\beta }({\omega }^{(k)}) \\ =\{\begin{array}{ll}({\omega }^{(k)}{)}_i-\beta \frac{({\omega }^{(k)}{)}_i}{|({\omega }^{(k)}{)}_i|},& \text{if}|({\omega }^{(k)}{)}_i|>\beta \\ 0& \text{if}|({\omega }^{(k)}{)}_i|<\beta \end{array}, \end{gathered}$$ (5)

to obtain the desired solution by element-wise soft-thresholding the coefficients iteratively. The parameter $\beta$ controls the level of sparsity in the solution. The optimal value of $\beta$ is data-dependent and it is difficult to predict. A higher value of $\beta$ leads to a sparser solution, which may not be the optimal solution. On the other hand, a smaller value of $\beta$ also degrades performance, known as the “cold” starting point. The continuation-based approach to accelerate convergence speed is also well established. Here, $\beta$ is initialized to a larger value and reduced using a constant scale factor in succeeding iterations. Although it substantially improves the convergence speed, both the initial $\beta$ value and the scale factor need to be predetermined. To address this issue, Mathew and Paul²² introduced an automated parameter selection-based ISTA, where an intermediate step optimization is introduced to maximize the $l_2$-norm of the gradient descent update.²³ The implementation involves solving two sub-problems in an alternating fashion. The first, sub-problem involves sparse regularization using previously computed ${\beta }^{(k-1)}$, and the second sub-problem estimates the value of ${\beta }^{(k)}$ to be used in the succeeding iteration. Algorithmic steps of the automated parameter estimation based iterative shrinkage-thresholding algorithm (AISTA) are summarized in Algorithm 1.

Algorithm 1.

Automated iterative shrinkage-thresholding algorithm.

1. initialization: ${\beta }^0=0.01$, $k=1$, $tol=1\times {10}^{-4}$

2. Input: $\mathrm{\Psi },{\mathbf{F}}_u,{\mathbf{K}}_u$

3. ${\omega }^0\leftarrow \mathrm{\Psi }{\mathbf{F}}_u^T{\mathbf{K}}_u$

4. while $\tau \le tol$ do

5.  ${\widehat{\omega }}^{(k)}\leftarrow {\omega }^{(k-1)}+\mathrm{\Psi }{F}_u^T(K_u-F_u{\mathrm{\Psi }}^T{\omega }^{(k-1)})$

6.  ${\omega }^{(k)}\leftarrow sof{t}_{{\beta }^{(k-1)}}({\widehat{\omega }}^{(k)})$

7.  ${\beta }^{(k)}\leftarrow \underset{\beta }{\max }\left\{\frac{1}{2}{\Vert {\omega }^{(k)}+\mathrm{\Psi }{F}_u^T(K_u-F_u{\mathrm{\Psi }}^T{\omega }^{(k)})\Vert }_2^2\right\}$

8.  $\tau \leftarrow \frac{\Vert {\widehat{\omega }}^{(k)}-{\widehat{\omega }}^{(k-1)}{\Vert }_2}{\Vert {\widehat{\omega }}^{(k-1)}{\Vert }_2}$

9.  $k\leftarrow k+1$

10. end while

2.3. Continuation Based Smooth $l_0$-Norm Minimization

The main idea of SL0 is to approximate the non-smooth $l_0$-norm with a differentiable function.^16,17 This smooth approximation approaches the exact $l_0$-norm function as $\sigma \to 0$. The SL0 algorithm iteratively uses gradient-projection steps; first, it computes the gradient descent step on the SL0 function (gradient step), and then projects the result onto the feasible set (projection step). It can be observed that large enough coefficients incur no shrinkage, as in thresholding, whereas small enough coefficients are enforced toward zero to promote sparsity.^24,25

The $l_0$-norm minimization problem defined in Eq. (3) can be rewritten as an unconstrained optimization problem as

$$\underset{\omega }{\min }\frac{1}{2}{\Vert K_u-F_u{\mathrm{\Psi }}^T\omega \Vert }_2^2+\alpha \Vert \omega {\Vert }_0,$$ (6)

where $\alpha$ is a regularization parameter.¹⁶ Although a direct form of $l_0$-norm minimization is NP-hard, the approximation of the discontinuous $l_0$-norm by a continuous function, commonly known as the smooth $l_0$-norm (SL0) approximation, provides a convenient way of representing the $l_0$-norm prior in Eq. (6). With an appropriate choice for the value of the modulation parameter $\sigma$ that determines the smoothness of the function, the $l_0$-norm of the sparse coefficients in Eq. (6) can be represented as

$${\Vert \omega \Vert }_0\approx \underset{\sigma \to 0}{\lim }{F}_{\sigma }(\omega )=\underset{\sigma \to 0}{\lim }\sum _{i=1}^N{f}_{\sigma }({\omega }_i),$$ (7)

where ${\omega }_i$ is the $i$’th element of $\omega$. Using the Gauss function $f_{\sigma }({\omega }_i)=1-\exp (-\frac{{\omega }_i^2}{2{\sigma }^2})$, the limiting values closely approximate the $l_0$-norm as shown in Fig. 1.

Fig. 1.

Plot of $f_{\sigma }({\omega }_i)$ for different values of $\sigma$ and comparison with $l_0$- and $l_1$- norms.

As $f_{\sigma }({\omega }_i)$ is smooth and differentiable, a gradient descent update step can be easily deduced making use of the closed form expression for the gradient:

$${\nabla }_{\sigma }{F}_{\sigma }(\omega )=[{\omega }_1\exp (-{\omega }_1^2/2{\sigma }^2),{\omega }_2\exp (-{\omega }_2^2/2{\sigma }^2),\cdots ,{\omega }_N\exp (-{\omega }_N^2/2{\sigma }^2)].$$ (8)

Using this, the optimization problem is implemented in two steps. These include a Landweber update $\widehat{\omega }$ to maintain the consistency in the Fourier domain, followed by a second gradient descent update that promotes sparsity using SL0 approximation:

$$\omega =\widehat{\omega }-\alpha \nabla F_{\sigma }(\widehat{\omega }),$$ (9)

$$\widehat{\omega }=\omega +\mathrm{\Psi }{F}_u^T(K_u-F_u{\mathrm{\Psi }}^T\omega ).$$ (10)

In order to enhance the convergence speed, the value of $\sigma$ is reduced in each iteration by a constant scale factor $\mu$; i.e., ${\sigma }^{(k)}=\mu {\sigma }^{(k-1)}$, where initially, i.e., $k=1$, we start with a predetermined ${\sigma }^0$, and in each iteration value of $\sigma$ is updated by a fixed scale factor $\mu$. All existing SL0 methods use empirically determined values of ${\sigma }^0$ and $\mu$. The value of $\sigma$ determines the level of the sparsity of the signal to be reconstructed. A large $\sigma$ value leads to the sparest solution by neglecting smaller signal coefficients. On the other hand, a small value of $\sigma$ needs infinite time to reconstruct a sparse signal. Similarly, $\mu$ determines the convergence speed. Similar to ISTA, the best possible choice for the values of ${\sigma }^0$ and $\mu$ are data-dependent and hence difficult to predetermine. Empirically, it is observed that the preferred choice of $\mu$ values to speed up convergence changes in each iteration. So, a predetermined constant valued $\mu$ cannot give the best performance in terms of speed and accuracy. This is addressed using the proposed ACSL0 algorithm.

2.4. Adaptive Continuation based Smooth $l_0$-Norm Minimization

For convergence, the $l_2$-norm of the difference between solutions of two successive iterations $E^{(k)}≔{\omega }^{(k)}-{\omega }^{(k+1)}$ decreases with iterations. Equivalently from Eq. (10), the difference between the intermediate gradient descent solutions ${\widehat{E}}^{(k)}≔{\widehat{\omega }}^{(k)}-{\widehat{\omega }}^{(k+1)}$ can be expressed as

$${\widehat{E}}^{(k)}=E^{(k)}-{\mathrm{\Phi }}^T\mathrm{\Phi }(E^{(k)}),$$ (11)

where $\mathrm{\Phi }\stackrel{\mathrm{\Delta }}{=}{F}_u{\mathrm{\Psi }}^T$. To ensure convergence, therefore, $\underset{k\to \infty }{\lim }\Vert E^{(k)}\Vert \to 0$ would imply $\underset{k\to \infty }{\lim }\Vert {\widehat{E}}^{(k)}\Vert \to 0$ from Eq. (11). For a fixed $\sigma$, Eqs. (9) and (10) are repeated $L$ times. With $L$ subiterations,

$${\Vert {\widehat{E}}^{(k)}\Vert }_2=\Vert {\widehat{\omega }}^{(k)}-{(1+\mathcal{W})}^L(1-{\partial }_{\sigma }{)}^L({\widehat{\omega }}^{(k)}){\Vert }_2,$$ (12)

where $\mathcal{W}(·)\stackrel{\mathrm{\Delta }}{=}{\mathrm{\Phi }}^T{K}_u-{\mathrm{\Phi }}^T\mathrm{\Phi }(·)$ and ${\partial }_{\sigma }(·)\stackrel{\mathrm{\Delta }}{=}\alpha \nabla F_{\sigma }(·)$.

In adaptive continuation, for any intermediate step $k$ to $k+1$, the value of ${\sigma }^{(k)}$ selected should minimize $\Vert {\widehat{E}}^{(k)}{\Vert }_2$. Using reverse triangular inequality, lower limit for ${\sigma }^{(k)}$ can be obtained such that

$$\begin{gathered} \underset{{\sigma }^{(k)}}{\min }\Vert {\widehat{\omega }}^{(k)}-(1+\mathcal{W}{)}^L(1-{\partial }_{\sigma }{)}^L({\widehat{\omega }}^{(k)}){\Vert }_2\ge \\ \underset{{\sigma }^{(k)}}{\min }(|\Vert {\widehat{\omega }}^{(k)}{\Vert }_2-\Vert (1+\mathcal{W}{)}^L(1-{\partial }_{\sigma }{)}^L({\widehat{\omega }}^{(k)}){\Vert }_2|). \end{gathered}$$ (13)

Since RHS of the above inequality for obtaining the lower bound for ${\sigma }^{(k)}$ is $\underset{{\sigma }^{(k)}}{\min }(\Vert {\widehat{\omega }}^{(k)}{\Vert }_2-\Vert (1+\mathcal{W}{)}^L(1-{\partial }_{\sigma }{)}^L({\widehat{\omega }}^{(k)}){\Vert }_2)$, and ${\widehat{\omega }}^{(k)}$ is already computed and independent of ${\sigma }^{(k)}$, the $l_2$-norm prior for SL0-minimization problem now becomes

$$J(\sigma )=-\Vert (1+\mathcal{W}{)}^L(1-{\partial }_{\sigma }{)}^L({\widehat{\omega }}^{(k)}){\Vert }_2.$$ (14)

After inclusion of the above $l_2$-norm prior in the SL0-minimization problem of Eq. (6), the composite optimization problem can be expressed as

$$\underset{\omega ,\sigma }{\min }\frac{1}{2}\Vert K_u-F_u{\mathrm{\Psi }}^T\omega {\Vert }_2^2+\alpha F_{\sigma }(\omega )+J(\sigma ).$$ (15)

The optimization problem in Eq. (15) can be efficiently solved by alternatively minimizing two subproblems. Subproblem-1 involves the SL0-norm regularization problem implemented using the previously computed value of $\sigma$ and subproblem-2 estimates the $\sigma$ value to be used for SL0 approximation in the next iteration given the $\widehat{\omega }$ obtained from subproblem-1. Here, subproblem 1 is solved using Eqs. (9) and (10) with precomputed value of $\sigma$. Subproblem-2 can be solved by estimating the root of ${\nabla }_{\sigma }J(\sigma )=0$.

Algorithmic steps of the adaptive continuation-based SL0 minimization are summarized in Algorithm 2. The modulation parameter $\sigma$ determines the quality of approximation: the larger $\sigma$ is, the smoother $F_{\sigma }$ is but the worse the approximation to $l_0$; and vice versa. One can obtain the sparsest solution when $\sigma \to 0$. It consists of two loops: the “outer” loop is the loop in which $\sigma$ is estimated, and the “inner” loop is the one in which $F_{\sigma }$ is iteratively minimized for the chosen value of $\sigma$. The parameter $\tilde{\alpha }$ in steps 6 and 13 of the algorithm is obtained as $\tilde{\alpha }=\alpha ({\sigma }^{(k)}{)}^2$.

Algorithm 2.

Adaptive continuation based SL0-norm minimization.

1. Initialization: $\alpha =2,{\sigma }_{\min }=0.01,L=4,k=1,tol=1\times {10}^{-4}$

2. Input: $\mathbf{\Psi },{\mathbf{F}}_u,{\mathbf{K}}_u$

3. ${\sigma }^{(0)}\leftarrow c\Vert \mathrm{\Psi }{F}_u^T{K}_u{\Vert }_{\infty }$

4. ${\widehat{\omega }}^{(0)}\leftarrow \mathbf{\Psi }{\mathbf{F}}_u^T{\mathbf{K}}_u$

5.  $$\begin{gathered} \delta \leftarrow [{\widehat{\omega }}_1^{(0)}\exp (-({\widehat{\omega }}_1^{(0)}{)}^2/2({\sigma }^{(0)}{)}^2), \\ {\widehat{\omega }}_2^{(0)}\exp (-({\widehat{\omega }}_2^{(0)}{)}^2/2({\sigma }^{(0)}{)}^2),\cdots , \\ {\widehat{\omega }}_N^{(0)}\exp (-({\widehat{\omega }}_N^{(0)}{)}^2/2({\sigma }^{(0)}{)}^2)] \end{gathered}$$

6. ${\omega }^{(0)}\leftarrow {\widehat{\omega }}^{(0)}-\tilde{\alpha }\delta$

7. while $\tau \le tol$ do

8.  ${\tilde{\omega }}^{(0)}\leftarrow {\widehat{\omega }}^{(k-1)}$

9.  for $l=1:L$

10.   $$\begin{gathered} \delta \leftarrow [{\tilde{\omega }}_1^{(l-1)}\exp (-({\tilde{\omega }}_1^{(l-1)}{)}^2/2({\sigma }^{(k-1)}{)}^2), \\ {\tilde{\omega }}_2^{(l-1)}\exp (-({\tilde{\omega }}_2^{(l-1)}{)}^2/2({\sigma }^{(k-1)}{)}^2),\cdots , \\ {\tilde{\omega }}_N^{(l-1)}\exp (-({\tilde{\omega }}_N^{(l-1)}{)}^2/2({\sigma }^{(k-1)}{)}^2)] \end{gathered}$$

11.   ${\overline{\omega }}^{(l)}\leftarrow {\tilde{\omega }}^{(l-1)}-\tilde{\alpha }\delta$

12.   ${\tilde{\omega }}^l\leftarrow {\bar{\omega }}^{(l)}+\mathrm{\Psi }{F}_u^T(K_u-F_u{\mathrm{\Psi }}^T{\bar{\omega }}^{(l)})$

13.  end while

14.  ${\widehat{\omega }}^{(k)}\leftarrow {\tilde{\omega }}^{(L)}$

15.  ${\omega }^{(k)}\leftarrow {\overline{\omega }}^{(L)}$

16.  $$\begin{gathered} {\sigma }^{(k)}\leftarrow \max \{{\sigma }_{\min },\sigma \text{obtained}\text{by}\text{solving} \\ {\nabla }_{\sigma }J({\sigma }^{(k)})=0\} \end{gathered}$$

17.  $\tau \leftarrow \frac{\Vert {\omega }^{(k)}-{\omega }^{(k-1)}{\Vert }_2}{\Vert {\omega }^{(k-1)}{\Vert }_2}$

18.  $k\leftarrow k+1$

19. end while

20. Output: ${\overline{\omega }}^{(L)}$

3. Results and Discussion

All experimental simulations were carried out in the MATLAB environment on a PC equipped with Intel i5 CPU and 16 GB of memory running the Windows 10 operating system. Experimental performance was evaluated using both local and standard publicly available datasets. We have considered five volunteer datasets acquired using a GE 3.0T MR system from Sree Chitra Tirunal Institute for Medical Sciences & Technology, Trivandrum, India²⁶ with parameter settings as shown in Table 1. In addition, we have considered two publicly available datasets.²⁷ Both experimentally collected and publicaly shared data are first retrospectively undersampled using variable density undersampling masks.²¹ Reconstruction performance is evaluated in terms of normalized mean square error (NMSE). Continuation is stopped when the relative change in $l_2$-norm of the reconstructed image in two successive iterations is less than a predefined tolerance level.

Table 1.

Different types of datasets and corresponding parameter settings.

	Parameter settings
Dataset	Sequence	No. of channel	$k$-space matrix size	TR/TE (ms)	Slice thickness (mm)	FOV (mm)
FLAIR brain I	FLAIR spin echo	6	512 × 416 × 6	9000/89	5.0	240
FLAIR brain II	FLAIR spin echo	16	224 × 256 × 16	9000/89	5.0	240
T2 weighted spine I	T2-weighted spin echo	4	450 × 448 × 4	5520/98	3.0	230
T2 weighted spine II	T2-weighted spin echo	14	691 × 768 × 14	4500/98	3.0	400
T2 WEIGHTED BRAIN	T2-weighted spin echo	6	243 × 320 × 6	5500/92	5.0	230

3.1. Varying Scale Factor Continuation Based Smooth L0 Minimization

In the varying scale factor continuation method, we search for a best scale factor in the range $[0.01,1.0]$ that corresponds to the peak of $-J({\sigma }^{(k)})$. Figure 2(a) shows normalized plots of $-J({\sigma }^{(k)})$ for four successive iterations, each with $L=4$ subiterations. Wavelet coefficients are computed using undersampled T2 weighted brain image. In each iteration, the $\sigma$-value corresponding to the peak of $-J({\sigma }^{(k)})$ is used to update the SL0 function in the next iteration. Figure 2(b) shows monotonically decreasing nature of $\sigma$ values corresponding to the peak of each normalized plot. In the adaptive continuation method, the peak $\sigma$-value in each iteration is obtained using a root finding method instead of the brute-force search procedure.

Fig. 2.

(a) Normalized plots of $-J({\sigma }^{(k)})$ for four successive iterations, each with $L=4$ subiterations. Wavelet coefficients are computed using undersampled T2 weighted brain image. (b) Monotonically decreasing nature of $\sigma$ values corresponding to the peak of each normalized plot.

The significance of selecting ${\sigma }^{(k)}$ value corresponding to the peak of the $l_2$-norm prior, i.e., $J({\sigma }^{(k)})$ is numerically shown in Fig. 3. Figure 3(a) shows normalized plots of $-J({\sigma }^{(k)})$ for a targeted search window. The green, blue, and red markers indicate the peak, pre-peak, and post-peak values of the $l_2$-norm prior, respectively. Figure 3(b) shows plot of NMSE versus iteration number for images reconstructed using the ${\sigma }^{(k)}$ values corresponding to the peak, pre-peak, and post-peak values of the $l_2$-norm prior. It is seen that the choice of ${\sigma }^{(k)}$ corresponding to the peak of the $l_2$-norm prior leads to lower NMSE.

Fig. 3.

(a) Normalized plots of $-J({\sigma }^{(k)})$ for a targeted search window. The green, blue, and red markers indicate the peak, pre-peak, and post-peak values of the $l_2$-norm prior, respectively. (b) Plot of NMSE versus iteration number for images reconstructed using the ${\sigma }^{(k)}$ values corresponding to the peak, pre-peak, and post-peak values of the $l_2$-norm prior.

3.2. Comparison Between Constant Scale Factor Based SL0 and Adaptive Continuation Based SL0 Minimization

The initial value of $\sigma$ is first obtained using ${\sigma }^{(0)}=c_1\Vert \mathrm{\Psi }{F}_u^{T}y{\Vert }_{\infty }$. In case of constant scale factor based SL0, the subsequent $\sigma$ values are updated according to the corresponding $\mu$, i.e., ${\sigma }^{(k)}=\mu \times {\sigma }^{(k-1)}$. In ACSL0, the value of $\sigma$ is updated by solving ${\nabla }_{\sigma }J({\sigma }^{(k)})=0$. Figure 4 shows the iteration-wise $\sigma$ values and the corresponding NMSE values obtained using constant scale factor based SL0 and ACSL0.

Fig. 4.

(a) Plot of iteration-wise ${\sigma }^{(k)}$ values using constant scale factor $\mu =0.3,0.5,0.7$ and ACSL0. (b) Plot of NMSE versus iteration number for images reconstructed using constant SL0 and ACSL0.

3.3. Comparison of Performance

For comparison, we have used the same stopping criterion. The performance of ACSL0 minimization is compared with the state-of-the-art continuation-based methods both qualitatively and quantitatively. Quantitative comparison based on the reconstruction accuracy is included in Table 2. The images reconstructed using different continuation-based methods: SpaRSA,²⁸ exponential rule based,²⁹ AISTA,²² and ACSL0 are shown in Fig. 5. The difference images clearly indicate the superior performance of ACSL0 in terms of preserving structural details and edges. The computational cost for convergence of these algorithms is empirically measured in terms of CPU time in seconds. We used MATLAB’s built-in function “cputime” for measuring computational time. Each value shown in Table 2 is an average of 150 runs for different datasets. Among other continuation based methods, the performance of AISTA is closer to ACSL0, albeit the longer reconstruction time as indicated.

Table 2.

Performance comparison in terms of NMSE and CPU time for five different datasets.

Dataset method	SpaRSA (NMSE/CPU time)	Exponential rule (NMSE/ CPU time)	AISTA (NMSE/ CPU time)	Proposed (NMSE/ CPU time)
FLAIR brain I	0.1158/24.84	0.1164/22.17	0.1148/72.28	0.1122/23.67
FLAIR brain II	0.1030/25.64	0.1025/24.23	0.1016/66.40	0.0950/18.65
T2 weighted spine II	0.0852/28.37	0.0824/33.70	0.0387/82.85	0.0227/38.93
T2 weighted brain	0.1096/26.06	0.1093/ 23.21	0.1082/74.26	0.1045/20.82
FSE knee	0.0961/25.23	0.0939/ 22.60	0.0916/71.12	0.0872/21.32

Fig. 5.

Left to right panels in the first row show T2 weighted brain images reconstructed using SpaRSA, exponential rule based, AISTA and ACSL0. Corresponding difference images are shown in the second row. Similarly, the reconstructed and corresponding difference images using FLAIR brain II and T2 weighted spine I datasets third to firth rows.

4. Conclusion

An adaptive continuation-based SL0 algorithm is presented, with a potential application to CS-based MR image reconstruction. It is a data-dependent adaptive continuation method and eliminates the problem of searching for appropriate constant scale factor values to be used in the CS reconstruction of different types of MRI data listed in Table 1. The current form of adaptive estimation is found to be at least twofold faster than AISTA and provides superior reconstruction performance in terms of NMSE. This is ascribed to the form of adaptive estimation of the modulation parameter in each iteration, instead of adopting an ad hoc specification of the scale factor.

Biographies

Sumit Datta received his BTech degree in electronics and communication engineering from the National Institute of Technology Agartala, India, in 2011, and his MTech and PhD degrees in electronics and communication engineering from Tezpur University, India, in 2014 and 2019, respectively. Currently, he is an assistant professor in the School of Electronic Systems and Automation at Digital University Kerala (former IIITM-Kerala), India. Before joining DUK, he was with the Department of Electronics and Electrical Engineering, Indian Institute of Technology Guwahati, India, as a post-doctoral fellow. His research interests include biomedical signal/image processing, compressed sensing MRI, super-resolution, and medical image analysis using deep learning. He is a senior member of IEEE.

Joseph Suresh Paul graduated in electrical engineering with distinction from the University of Calicut, Kerala, India, in 1987 and received his master’s (1992) and doctoral degree (2000) in electrical engineering from the Indian Institute of Technology, Madras, India. He held postdoctoral position at Johns Hopkins University School of Medicine, Baltimore, Maryland, USA, and faculty positions at the National University of Singapore and the University of New South Wales, Sydney, Australia. He is currently a professor with the Medical Image Computing and Signal Processing Group, Digital University Kerala (former IIITM-K), Thiruvananthapuram, India. His current interests include compressed sensing, image reconstruction for parallel MRI, and image processing for diagnostic applications.

Disclosures

There are no conflicts of interest.

Code and Data Availability

The datasets utilized in this article are publicly accessible at the following link: https://old.mridata.org/fullysampled. The code is available at the following link: https://drive.google.com/drive/folders/1TG7IgYomWLT_P_l1dAjYljd0ndiCPMwE?usp=sharing.