Dynamic MR Image Reconstruction based on Total Generalized Variation and Low Rank Decomposition

Abstract

Purpose:

Propose a novel decomposition based model employing the total generalized variation (TGV) and the nuclear norm, which can be used in compressed sensing based dynamic MR reconstructions.

Theory and Methods:

We employ the nuclear norm to represent the time-coherent background and the spatio-temporal TGV functional for the sparse dynamic component above. We first discuss the existence of solutions to the proposed model, and then design an algorithm using the classical first-order Primal-Dual method for solving the proposed model and give the norm estimation for the convergence condition. The proposed model is compared with the state-of-the-art methods on different datasets under different sampling schemes and acceleration factors.

Results:

The proposed model achieves higher SERs and SSIMs than kt-SLR, kt-RPCA, L+S and ICTGV on cardiac cine, perfusion and breast DCE-MRI datasets under both pseudo radial and Cartesian sampling schemes. In addition, the proposed model better suppresses the spatial artifacts and preserves the edges.

Conclusion:

The proposed model outperforms the state-of-the-art methods and generates high quality reconstructions under different sampling schemes and different acceleration factors.

Introduction

Dynamic magnetic resonance imaging (dMRI) is an important technique in MRI and has been used in a broad range of applications from cardiac cine imaging to breast DCE-MRI (Dynamic Contrast Enhanced) [1], which aims to visualize time dependent changes of morphology or contrast. However, traditional dMRI methods suffer from slow imaging speed and low signal-to-noise ratio, and it is not easy to balance the trade-off between spatial and temporal resolution [2–4].

Compressed sensing [5] [6] theory has been a hot topic in image processing for the past decade and has been widely employed in dMRI [7–13]. Compressed sensing theory demonstrates that it is possible to accurately reconstruct dynamic MR images from the undersampled Fourier data (known as k-space), which results in significantly reduced scanning time [7, 14]. Thus compressed sensing theory is a promising method that accelerates the dMRI process while protecting the temporal and spatial resolution.

Denote the spatio-temporal image as X ∈ ${\mathbb{\text{C}}}^{m\times n\times d}$, where the dynamic image has d dynamics and for each dynamic, the spatial dimension is m × n. The dynamic measurements correspond to the samples in k-space corrupted with noise

$$B=AX+ϵ,$$

where A = M $\mathcal{F}$ is the sampling operator, $\mathcal{F}$ is the Fourier transform on each temporal frame, M is the sampling mask on each temporal frame and $ϵ$ is additive white Gaussian noise. Then compressed sensing based reconstruction for dynamic MRI can be formulated as:

$$\underset{X}{\min }\frac{1}{2}\Vert AX-B{\Vert }_{\text{F}}^2+\alpha \Vert \mathcal{S}X{\Vert }_1,$$ (1)

where $\mathcal{S}$ is a certain sparse transform and α is a parameter that balances between data consistency and sparsity.

The choice of sparse regularizer for dynamic MR data is crucial and tricky. There are three main strategies for compressed sensing based dMRI. The first and simplest strategy is to utilise sparse regularizers alone in the model. kt-SPARSE [7] employs both temporal and spatial sparse regularizers in the model, which can be formulated as:

$$\underset{X}{\min }\frac{1}{2}\Vert AX-B{\Vert }_{\text{F}}^2+\alpha \Vert \mathcal{W}X{\Vert }_1+\beta \Vert {\mathcal{F}}_{t}X{\Vert }_1,$$ (2)

where $\mathcal{W}$ is the Wavelet transform in spatial direction and ${\mathcal{W}}_t$ is one-dimensional Fourier transform in temporal direction. This model works well on cardiac MRI data since Fourier transform can sparsify periodic movements like cardiac beating. However, other dynamic MR images, such as breast DCE-MRI, usually do not show periodicity in the breast or cancer area, kt-SPARSE has limitations on such datasets. iGrasp [15] combines the idea of compressed sensing, parallel imaging and golden-angle radial sampling and proposes a model that only contains one regularizer:

$$\underset{X}{\min }\Vert AX-B{\Vert }_{\text{F}}^2+\alpha {\Vert {\nabla }_{t}X\Vert }_1,$$ (3)

where ∇_t is the gradient operator in temporal direction. This model performs well when time sparsity is dominated in the dataset. For most dynamic MR images, however, both spatial and temporal resolution are of great importance – high spatial resolution helps doctors visually understand the images and high temporal resolution is essential in performing quantitative analysis for DCE-MRI. Thus using only temporal regularizer is inadequate.

The second strategy is to consider low rank in the model and aims to find a solution that is both sparse and low rank. In this case, one dynamic MR image is often reformed into a matrix, in which each column corresponds to one frame and each row corresponds to one voxel. This matrix is referred to as a Casorati matrix, which has the dimension of mn × d and is very likely to be approximately low rank because of the high correlation between temporal frames. kt-SLR [9] employs both spatial and temporal gradient operators as well as non-convex Schatten p-quasi-norm in the model:

$$\underset{X}{\min }\frac{1}{2}\Vert AX-B{\Vert }_{\text{F}}^2+\alpha \Vert X{\Vert }_{\text{TV}}+\beta \Vert X{\Vert }_p,$$ (4)

where 0 < p < 1 and the spatio-temporal gradient is defined as:

$$\Vert \cdot {\Vert }_{\text{TV}}={\Vert {\nabla }_3\cdot \Vert }_1={\int }_{\mathrm{\Omega }}\sqrt{{\left({\nabla }_x\cdot \right)}^2+{\left({\nabla }_y\cdot \right)}^2+\mu {\left({\nabla }_t\cdot \right)}^2},$$

and ∇_x, ∇_y and ∇_t are the gradient operators along x, y and t direction respectively, μ is the parameter that balances the trade-off between spatial and temporal sparsity, and $\Vert \cdot {\Vert }_p$ is the Schatten p-quasi-norm.

However as kt-SLR uses TV as the regularizer, the reconstructed images often suffer from the so-called staircase effects. Besides, due to the non-convexity of Schatten p-quasi-norm when 0 < p < 1, it is more difficult to compute a robust numerical solution.

The last strategy is to consider the dynamic MR image as the superposition of a sparse component and a low rank component, which can be regularized using different constraints. The main advantage of this idea is that after subtracting the background from the image, the remaining part becomes ‘sparser’ than the dynamic image itself, resulting in a more promising situation for compressed sensing based models. Otazo [12] employed this idea and proposed a model which is referred to as L+S:

$$\underset{L,S}{\min }\frac{1}{2}\Vert A(L+S)-B{\Vert }_{\text{F}}^2+\alpha {\Vert {\nabla }_{t}S\Vert }_1+\beta \Vert L{\Vert }_{*},$$ (5)

where $\Vert \cdot {\Vert }_{*}$ denotes the nuclear norm, and L and S are the low rank component and the sparse component respectively. Similarly, Tremoulheac [16] proposed the model kt-RPCA, which employs the temporal Fourier transform for the sparse component:

$$\underset{L,S}{\min }\frac{1}{2}\Vert A(L+S)-B{\Vert }_{\text{F}}^2+\alpha {\Vert {\mathcal{F}}_{t}S\Vert }_1+\beta \Vert L{\Vert }_{*}.$$ (6)

However as the case in iGrasp, both models only exploit temporal sparsity constraints. Besides, since kt-RPCA exploits temporal Fourier transform, it only works well on cardiac datasets. Schloegl [17] proposed a novel regularizer based on infimal convolution of total generalized variation (TGV) functionals, which is called ICTGV and defined as:

$${\mathrm{ICTGV}}_{\alpha ,\beta }^2(X)=\underset{X=X_1+X_2}{\inf }\mathrm{TG}{\text{V}}_{{\alpha }_1}^2\left(X_1\right)+\beta \mathrm{TG}{\text{V}}_{{\alpha }_2}^2\left(X_2\right),$$ (7)

and the model, which we also refer to as ICTGV, is

$$\underset{X}{\min }\frac{1}{2}\Vert AX-B{\Vert }_{\text{F}}^2+{\mathrm{ICTGV}}_{\alpha ,\beta }^2(X).$$ (8)

The well-known concept of TGV was introduced in [18], which can be regarded as a generalization of TV [19]. TGV contains high order derivatives and can better depict smooth areas in the image as well as suppress staircase effects. Although ICTGV does not explicitly decompose a dynamic image into low rank and sparse components, ICTGV optimally balances between spatial and temporal regularity and performs well on cardiac cine and perfusion images, and thus we still categorize ICTGV into this group. However the performance of ICTGV on breast DCE-MRI is unknown.

In general, the above compressed sensing based models for dynamic MR images either suffer from staircase effects or employ temporal sparsity only. Motivated by the purpose of suppressing staircase artifacts and improving reconstruction quality, under the framework of decomposition based model, we propose to employ the second order total generalized variation $({\text{TGV}}_{\alpha }^2)$ as the spatio-temporal regularizer for the sparse component and the nuclear norm for the low rank component. Thus the proposed model can be formulated as:

$$\underset{L,S}{\min }\frac{1}{2}\Vert A(L+S)-B{\Vert }_{\text{F}}^2+{\text{TGV}}_{\alpha }^2(S)+\beta \Vert L{\Vert }_{*}.$$ (9)

This paper is organized as follows. In section 2 we first give a detailed review of low rank and TGV, formulate the proposed model and discuss the existence of solutions to the proposed model. Then we show the discretization form of the proposed model and the corresponding first-order Primal–Dual algorithm. Section 3 gives the details of datasets and method of comparisons. In section 4 we compare the performance of the proposed model with state-of-the-art methods on different dynamic MR datasets under different sampling schemes and acceleration factors and present the results. Section 5 draws the discussions and conclusions.

Theory

Low rank

Recently low rank matrix completion has been applied to dMRI instead of just employing sparsity. The main idea is to consider one dynamic MR image as a space-time matrix, in which each column corresponds to one temporal frame and each row corresponds to one voxel. In [20], Candes presented the mathematical foundations of low rank and sparse decomposition, known as Robust PCA (RPCA). Given a Casorati matrix $X\in {ℂ}^{mn\times d}$, RPCA describes the minimization problem

$$\underset{L,S}{\min }\Vert L{\Vert }_{*}+\alpha \Vert S{\Vert }_{1}s.t.X=L+S,$$ (10)

where the nuclear norm is employed as the convex relaxation of the rank and is defined as the sum of singular values (σ_i) of a matrix:

$$\Vert \cdot {\Vert }_{*}=\sum _i{\sigma }_i.$$

Candes [20] pointed out that RPCA can perfectly recover the low rank and the sparse components under several assumptions, one of which is that the rank of L is low and the sparsity of S is reasonable small.

The combination of compressed sensing and low rank matrix completion is a promising idea for further accelerating the imaging speed. The decomposition model is reasonable for dynamic MRI: the low rank component can model the time-coherent background and the sparse component can model the dynamic information that lies on top of the background, such as cardiac beating in cardiac cine and tumor contrast change in breast DCE-MRI. Several works [12,16] have employed the RPCA scheme in dMRI, in which temporal gradient or Fourier transform is exploited as the sparse transform. Wang [21] pointed out that for breast DCE-MRI datasets, the nuclear norm can achieve the highest image quality among common temporal regularizers, which motivates the choice of nuclear norm in the proposed model.

Second order TGV

The mathematical foundation of TGV was presented by Bredies [18], which can be regarded as a generalization of TV [19]. The TGV functional can depict the smooth area of an image and is able to suppress the staircase effects efficiently while preserving the edges. Thus TGV has been used in many image processing problems, such as denoising [22], compressed sensing [22], etc.

In this paper, we will use the total generalized variation of second-order $({\text{TGV}}_{\alpha }^2)$ through-out the paper, which has the definition

$${\text{TGV}}_{\alpha }^2(u)=\underset{\omega \in \text{BD}(\mathrm{\Omega })}{\min }{\alpha }_1{\Vert {\nabla }_{3}u-\omega \Vert }_1+{\alpha }_0\Vert \mathcal{E}(\omega ){\Vert }_1,$$ (11)

where $\mathcal{E}(\omega )=\frac{1}{2}\left({\nabla }_3\omega +{\nabla }_3{\omega }^T\right)$ is the symmetric gradient and BD(Ω) is the space of functions with bounded deformation. For more mathematical background about ${\text{TGV}}_{\alpha }^2$, please refer to [18].

The proposed model

Now we present the proposed model for compressed sensing dMRI by exploiting both ${\text{TGV}}_{\alpha }^2$ and the nuclear norm under the decomposition scheme, which can be formulated as:

$$\underset{L,S}{\min }\frac{1}{2}\Vert A(L+S)-B{\Vert }_{\text{F}}^2+\mathrm{TG}{\text{V}}_{\alpha }^2(S)+\beta \Vert L{\Vert }_{*},$$ (12)

where the nuclear norm models the time-coherent background and ${\text{TGV}}_{\alpha }^2$ models the sparse dynamic information above. Note that the proposed model is convex and well-posed.

Now we discuss the existence of solutions to the proposed model. We treat the sparse component S and low rank component L separately. For the sparse component S, choose a linear projection operator $P:L^2(\mathrm{\Omega })\to \ker ({\text{TGV}}_{\alpha }^2)$ and a minimizing sequence {Sⁿ}, then let A : L²(Ω) → L²(Ω) be a linear and continuous operator which is injective on ker $({\text{TGV}}_{\alpha }^2)$, and B ∈ L²(Ω), we deduce that $\Vert A{S}^n-B\Vert$ is bounded. Due to ${\text{TGV}}_{\alpha }^2\left(P{S}^n\right)=0$, we have ${\text{TGV}}_{\alpha }^2\left(S^n-P{S}^n\right)={\text{TGV}}_{\alpha }^2\left(S^n\right)$ is bounded. Thus the sequence {Sⁿ − P Sⁿ} is bounded in L²(Ω). Since A is continuous, $\Vert A(S^n-P{S}^n)\Vert$ is bounded, then $\Vert A(S^n-P{S}^n)-B\Vert$ is also bounded. Moreover, there exists a constant c₁ > 0 such that $\Vert PS\Vert \le c_1\Vert APS\Vert$, then

$$\Vert P{S}^n\Vert \le c_1\Vert AP{S}^n\Vert \le c_1(\Vert A{S}^n-B\Vert +\Vert A(S^n-P{S}^n)-B\Vert )\le c_2,$$

for some c₂ > 0, which implies {Sⁿ} is bounded in L²(Ω). Then by the reflexivity of L²(Ω), there exists a subsequence {Sⁿ} which converges weakly to S* ∈ L²(Ω). According to the convexity and lower semi-continuity of the functional, we deduce that S* is a sparse solution to (12). For more details, please refer to [23].

Note that the choice of ${\text{TGV}}_{\alpha }^2$ and the nuclear norm are natural for dynamic MR images. On one hand, since MR images are usually sparse in their gradient domains, ${\text{TGV}}_{\alpha }^2$ which consists of first and second order derivatives can sparsify MR images. Besides, since the background is subtracted from one dynamic image, the remaining dynamic information should be sparser than the original image. On the other hand, the background of one dynamic image is time-coherent and hence low rank, the nuclear norm can perfectly model the background. Therefore the choice of ${\text{TGV}}_{\alpha }^2$ and the nuclear norm satisfy the conditions of Candes’s theorem [20] and thus can guarantee the recovery of the dynamic images.

Numerical Algorithm

In this subsection, we present the numerical algorithm to solve the proposed model. First we need the discrete setting of ${\text{TGV}}_{\alpha }^2$. For discretization, we use a three dimensional regular Cartesian grid of size m × n × d and mesh size 1. The reader interested in the mathematical background may find more information about the discretization in the Appendix.

Now we focus on the discrete version of the proposed model (12) and assume that the linear sampling operator A : U → Y, where $U={ℂ}^{m\times n\times d}$ and Y is a finite Hilbert space (kt-space in dMRI) which is already given in a discretized form. Then the proposed model (12) can be formulated as:

$$\begin{array}{r}\underset{(S,w,L)\in U\times V\times U}{\min }{\alpha }_1{\Vert {\nabla }_{3}S-w\Vert }_1+{\alpha }_0\Vert \mathcal{E}(w){\Vert }_1+\beta \Vert L{\Vert }_{*}\\ +\frac{1}{2}\Vert A(L+S)-B{\Vert }_{\text{F}}^2,\end{array}$$ (13)

where V = U × U × U = U³.

In this paper, we adopt the Primal-Dual [24] algorithm to solve the proposed model. The Primal-Dual algorithm is a first order algorithm and easy to implement, which has been widely used in many image processing problems [22]. Details about how this method is applied can be found in the Appendix.

Methods

Datasets

We apply the proposed model retrospectively on three different dynamic datasets. The first two datasets are the physiologically improved non uniform cardiac torso (PINCAT) numerical phantom and in vivo cardiac perfusion MRI data (cardiac), which are also used in kt-SLR [9]. For more information about the datasets, please refer to [25, 26]. The choice of these two datasets is to fairly compare with other models. The dimensions of PINCAT and cardiac perfusion are 128 × 128 × 50 and 190 × 90 × 70 respectively.

The last group of datasets are in vivo breast DCE-MRI data collected under a protocol approved by the institutional review board. The data were acquired using a spoiled gradientrecalled echo (SPGRE) sequence on a Philips (Best, Netherlands) Achieva 3T scanner with TR = 4.33 ms, TE = 2.12 ms, flip angle = 12°. The dimension of the coil-combined complex data is 192 × 192 × 10 × 105, which consists of 192 readout points by 192 phase encodes across 10 slices repeated over 105 dynamics. The spatial resolution is 1.33 mm by 1.33 mm by 5 mm, and the field of view is 256 mm by 256 mm by 50 mm. Further details about the protocol can be found in [27]. We denote the data as breast1 for simplicity. Another two breast datasets were collected under the same protocol with different imaging parameters, resulting in a different dimension of 192 × 192 × 20 × 25. We denote these two datasets as breast2 and breast3 respectively. Note that only magnitude images of breast2 and breast3 are available for the reconstruction. To narrow the focus of the paper, we look only at the slice that passes through the center of the tumor. For breast1, the center slice is the 6th slice and for breast2 and breast3, the center slices are the 10th slices. Thus the final dimensions are 192 × 192 × 105 for breast1 and 192 × 192 × 105 for breast2 and breast3.

Comparisons

We compare the proposed model with existing state-of-the-art methods, which are kt-SLR¹ [9], kt-RPCA² [16], L+S³ [12] and ICTGV⁴ [17]. All the models are performed in MATLAB and are available online. All the experiments are performed in MATLAB R2018a running on a dual 10-core Intel Xeon E5–2630 CPU 2.2 GHz with 128 GB of RAM and an Nvidia TITAN V GPU. We also implement the proposed model in CUDA C to accelerate the MATLAB code. Both versions can be found online https://github.com/chixindebaoyu/tgvnn. The runtime comparison between MATLAB and CUDA is also presented.

For all the models, we apply the pseudo radial sampling pattern which is also used in kt-SLR [9]. For each frame, the radial trajectory is uniformly spaced and is rotated by a small random angle in each temporal frame to increase incoherence. Figure 1 (left) shows an example of the pseudo radial sampling. The choice of pseudo radial sampling pattern is to simplify the computation, because all the sampling points lie on a Cartesian grid and fast Fourier transform suffices. For all the experiments, unless specified, we all use 32 sampling spokes for consistency. The total acceleration factor of the mask is around 6.6. Here the total acceleration factor is the ratio of total k-space points to total k-space samples. In addition, for testing the performance of the proposed model under different sampling schemes, we also compare the reconstructions under the Cartesian sampling mask, which is widely used in clinical MRI. For each frame, low frequency is fully sampled with the central window radius of 10 pixels. Outside the central window, we randomly pick a few lines along the phase encoding direction to guarantee the maximal orthogonality across frames. The total acceleration factor of the mask is around 5.1. Figure 1 (right) shows one frame of the Cartesian mask.

Figure 1:

Sampling masks (one frame). Left: Pseudo radial sampling mask. Right: Cartesian sampling mask.

The assessments used for the reconstructions are the signal to error ratio (SER) and the structural similarity index (SSIM), which are commonly used in compressed sensing based reconstructions. Specifically, the SER is defined as

$$\text{SER}=-10{\log }_{10}\frac{{\Vert X_{\text{rec}}-X_{\text{ori}}\Vert }_{\text{F}}^2}{{\Vert X_{\text{ori}}\Vert }_{\text{F}}^2},$$

where X_rec is the reconstructed image and X_ori is the original, fully-sampled data. For all the experiments, the dataset is scaled to [0,1]. We tune the parameters manually on the phantom dataset via an exhaustive search over each parameter in order to obtain the optimal parameters for the best SER and apply this set of parameters to the other datasets. Table 1 shows the parameters used for the proposed model. Note that the iteration number is fixed at 500 for the pseudo radial sampling and 1500 for the Cartesian sampling, which is sufficient for the convergence of the algorithm.

Table 1:

Parameters used for the proposed model.

Sampling ╲ Parameters	α0	α1	β	σ	τ	μ	iter
Pseudo radial	0.006	0.004	0.5	0.25	0.25	1	500
Cartesian	0.015	0.005	0.08	0.28	0.28	1	1500

Results

Table 2 shows the comparisons of different models on different datasets under the pseudo radial sampling pattern. We can see that the proposed model performs well on all the datasets with high SERs and SSIMs. Although kt-SLR achieves the highest SER on PINCAT, it does not achieve high SERs on breast datasets. Also although kt-RPCA performs well on breast datasets, it does not work well on PINCAT and cardiac perfusion.

Table 2:

Comparisons of different models on different datasets.

Models ╲ Datesets		PINCAT	cardiac	Breast1	Breast2	Breast3
Zerofilled	SER	20.54	14.80	11.32	11.57	14.85
	SSIM	0.8321	0.8855	0.4820	0.5897	0.7086
kt-SLR	SER	33.35	17.58	17.42	13.61	18.38
	SSIM	0.9880	0.9412	0.7712	0.6956	0.8461
kt-RPCA	SER	29.27	18.33	19.31	14.82	19.81
	SSIM	0.9700	0.9447	0.8857	0.7920	0.9068
L+S	SER	27.99	19.12	17.60	14.74	19.91
	SSIM	0.9514	0.9490	0.7675	0.7319	0.8791
ICTGV	SER	26.88	17.87	16.31	12.18	16.84
	SSIM	0.9435	0.9405	0.6735	0.6080	0.7693
Proposed	SER	32.74	19.57	20.56	16.24	21.08
	SSIM	0.9917	0.9514	0.9402	0.9091	0.9356

Figure 2 shows the comparisons of different models on PINCAT data. Note that the 1st frame is presented here. The red arrows indicate that although kt-SLR achieves the highest SER (33.35), there are still artifacts left in the reconstructed image, which do not appear in the proposed approach. Furthermore, the blue arrows show that the proposed model outperforms kt-RPCA, L+S and ICTGV in both preserving the edges and suppressing the artifacts.

Figure 2:

Comparisons on PINCAT (the 1st frame). The first row: reconstructed images. The second row: zoomed images in the red box. The third row: difference images with respect to the original image. The fourth row: zoomed difference images. The difference images are multiplied by 30 to increase visibility.

Figure 3 presents the reconstructions of cardiac perfusion. The 1st frame is presented here. The proposed model achieves the highest SER (19.58) and SSIM (0.9514), and produces the least artifacts. The red arrows show the poor performance of kt-SLR, kt-RPCA and ICTGV, especially on the edges. Although L+S performs well on cardiac perfusion, the blue arrows show that the result of the proposed model is smoother than L+S.

Figure 3:

Comparisons on perfusion (the 1st frame). The first row: reconstructed images. The second row: zoomed images in the red box. The third row: difference images with respect to the original image. The fourth row: zoomed difference images. The difference images are multiplied by 10 to increase visibility.

For a better view of the temporal direction, we also plot three different frames as well as time series for cardiac perfusion in Figure 4. We can see that the proposed model generates visually high quality reconstructions. The red arrow shows that kt-SLR produces blurry reconstructions, resulting in low SER. The blue arrow indicates that kt-RPCA suffers from motion blur, especially on the edge. Besides we can observe from the yellow arrows that the time series plots of both kt-RPCA and ICTGV are oversmooth, especially ICTGV. L+S performs well on cardiac perfusion in both SER and time series plot. However since L+S has no spatial constraint, all the frames appear noisy (green arrow).

Figure 4:

Comparisons on cardiac perfusion. Rows (a), (b) and (c) are the 11th, 21st and 54th frames respectively. Row (d) is the column through the cardiac (red line) across all the frames.

The comparisons of different models on breast1, breast2 and breast3 can be found in Supporting Information Figure S1, S2 and S3 respectively. Again the proposed model achieves the highest SERs and SSIMs and performs well in both removing spatial artifacts (blue arrows) and reconstructing the tumor (red arrows). kt-SLR, kt-RPCA and ICTGV do not work well on breast datasets. Interestingly, L+S does slightly better in reconstructing the tumor. However L+S achieves low SERs and SSIMs and brings more spatial artifacts in the background (blue arrows).

In order to show the temporal direction of the tumor, we also plot the mean intensity curves in the tumor area of breast1 in Figure 5. The mean intensity curve here is the mean pixel intensity of the tumor area (red rectangle) across all the frames. We can see from Figure 5 that the proposed model fits the mean intensity curve well before and after the injection of the contrast agent with low variance. On the contrary, both kt-SLR and ICTGV achieve high-variance mean intensity curves, especially after the injection of the contrast agent (blue arrows). The yellow arrow indicates that kt-RPCA fails to fit the first 10–15 frames. L+S performs poorly in fitting the last 5 frames, which can be observed more clearly in the pixel intensity curve (green arrows).

Figure 5:

Comparisons on breast1. Rows (a), (b) and (c) are the 1st, 53rd and 105th frames respectively. Row (d) is the mean intensity curve in the tumor area (red rectangle) across all the frames. Row (e) is the intensity curve of one pixel inside the tumor.

Table 3 shows the running time comparison on breast1 of different models and between CPU (MATLAB) and GPU (CUDA C). Note that ICTGV also has a GPU version code, but we can not make it work on our workstation, thus only CPU runtime is presented here. Note that kt-RPCA and L+S are relatively faster because both models employ only temporal regularizers. We can see that although the MATLAB code of the proposed model is slow (2812.90 s), GPU accelerates it by about 100× and it only takes 28.35 s on breast1 dataset.

Table 3:

Running time comparison on breast1.

Models	kt-SLR (CPU)	kt-RPCA (CPU)	L+S (CPU)	ICTGV (CPU)	Proposed (CPU)	Proposed (GPU)
Time (s)	5794.88	641.71	493.80	2572.04	2812.90	28.35

For testing the performance of the proposed model under different acceleration factors, we choose 5 different numbers of spokes, resulting in 5 sampling masks with different acceleration factors. The numbers of spokes are 12, 22, 32, 42 and 52, and the corresponding actual acceleration factors are 17.0, 9.5, 6.6, 5.1 and 4.2 respectively. Table 4 demonstrates that the proposed model outperforms the other methods under different acceleration factors, which implies consistency.

Table 4:

Comparisons of different sampling factors on breast1.

Models ╲ Spokes		12	22	32	42	52
Zerofilled	SER	5.98	9.06	11.32	12.93	14.33
	SSIM	0.3218	0.4108	0.4820	0.5350	0.5818
kt-SLR	SER	15.16	16.10	17.42	19.07	19.11
	SSIM	0.7323	0.7312	0.7712	0.8087	0.7989
kt-RPCA	SER	13.10	17.74	19.31	20.26	21.13
	SSIM	0.6241	0.8356	0.8857	0.9061	0.9245
L+S	SER	12.75	16.00	17.60	18.73	19.64
	SSIM	0.5885	0.7119	0.7675	0.8018	0.8275
ICTGV	SER	13.56	15.30	16.31	17.03	17.56
	SSIM	0.6001	0.6485	0.6735	0.6895	0.7007
Proposed	SER	16.49	19.06	20.56	21.82	22.96
	SSIM	0.8620	0.9119	0.9402	0.9535	0.9632

For evaluating the performance of the proposed model under different sampling schemes, we also compare all the models under the Cartesian sampling mask, which is shown in Figure 1. Table 5 shows the results of different models under the Cartesian sampling mask. Both the proposed model and L+S achieve the highest SER (28.46) while the other models perform relatively worse under the Cartesian sampling mask. Note that the SSIM (0.9839) of the proposed model is the higher than L+S, although they share the same SER.

Table 5:

Comparisons of reconstruction using the Cartesian mask on breast1.

Models	Zerofilled	kt-SLR	kt-RPCA	L+S	ICTGV	Proposed
SER	11.29	16.60	16.86	14.55	14.33	17.79
SSIM	0.7370	0.8942	0.9126	0.8379	0.7681	0.9169

Discussion and Conclusions

In this paper, we propose a novel decomposition based model exploiting spatio-temporal TGV2 and the nuclear norm for compressed sensing dynamic MR reconstruction. The nuclear norm can model time-coherent background and perform well in removing spatial artifacts, and the TGV functional represents the smooth part well and can preserve the edges as well as reduce staircase effects. We first discuss the existence of solutions to the proposed model, and then we employ the Primal-Dual algorithm to solve the proposed model and give the norm estimation for the convergence condition. We also implement the proposed model using GPU in CUDA C to accelerate the MATLAB code. Numerical experiments on PINCAT, cardiac perfusion and breast DCE-MRI datasets indicate that the proposed model outperforms the-state-of-art methods in both suppressing the spatial artifacts and preserving the edges under different acceleration factors and different sampling schemes, especially for breast DCE-MRI datasets.

Appendix

Discretization

In this subsection, we present the discretization of ${\text{TGV}}_{\alpha }^2$ and the algorithm adapting to the proposed model. In order to solve the proposed model, we first need to give the discretization of ${\text{TGV}}_{\alpha }^2$. In this paper, since we are dealing with dynamic MR images which have two spatial dimensions and one temporal dimension, we focus on second order TGV (${\text{TGV}}_{\alpha }^2$) in ${\mathcal{\text{C}}}^3$. Following essentially the presentation in [18, 24], the region Ω is discretized as the gid:

$$\mathrm{\Omega }=\{(i,j,k)|1\le i\le m,1\le j\le n,1\le k\le d\},$$

where m, n and d are the dimensions of the dynamic image. Denote three finite dimensional vector spaces $U={ℂ}^{m\times n\times d},V=U\times U\times U=U^3$ and $W=U\times U\times U\times U\times U\times U=U^6$. The space U will be endowed with the scalar product ${\langle u,u^{\prime }\rangle }_U=\mathrm{Re}{\sum }_{(i,j,k)\in \mathrm{\Omega }}{u}_{i,j,k}\overline{u_{i,j,k}^{\prime }}$ for u, u^′ ∈ U and the norm $\Vert u{\Vert }_U=\sqrt{{\langle u,u\rangle }_U}$. Similarly, the spaces V and W are equipped with the scalar products ${\langle v,v^{\prime }\rangle }_V={\sum }_{i=1}^3{\langle v_i,v_i^{\prime }\rangle }_U$ and ${\langle w,w^{\prime }\rangle }_W={\sum }_{i=1}^3{\langle w_i,w_i^{\prime }\rangle }_U+2{\sum }_{i=4}^6{\langle w_i,w_i^{\prime }\rangle }_U$, respectively.

The TGV functional is discretized by finite differences where the spatial grid sizes are chosen to be one and the temporal grid size is manually set by the parameter μ. The forward and backward operators are:

$${({\partial }_x^{+}u)}_{i,j,k}=\{\begin{array}{ll}{u}_{i+1,j,k}-u_{i,j,k},\hfill & 1\le i<m,\hfill \\ 0,\hfill & i=m,\hfill \end{array}$$

$${({\partial }_y^{+}u)}_{i,j,k}=\{\begin{array}{ll}{u}_{i,j+1,k}-u_{i,j,k},\hfill & 1\le j<n,\hfill \\ 0,\hfill & j=n,\hfill \end{array}$$

$${({\partial }_t^{+}u)}_{i,j,k}=\{\begin{array}{ll}\left(u_{i,j,k+1}-u_{i,j,k}\right)/\mu ,\hfill & 1\le k<d,\hfill \\ 0,\hfill & k=d,\hfill \end{array}$$

and

$${({\partial }_x^{-}u)}_{i,j,k}=\{\begin{array}{ll}{u}_{i,1,k},\hfill & i=1,\hfill \\ u_{i,j,k}-u_{i-1,j,k},\hfill & 1<i<m,\hfill \\ -u_{m-1,j,k},\hfill & i=m,\hfill \end{array}$$

$${\left({\partial }_y^{-}u\right)}_{i,j,k}=\{\begin{array}{ll}{u}_{i,1,k},\hfill & j=1,\hfill \\ u_{i,j,k}-u_{i,j-1,k},\hfill & 1<j<n,\hfill \\ -u_{i,n-1,k},\hfill & j=n,\hfill \end{array}$$

$${({\partial }_t^{-}u)}_{i,j,k}=\{\begin{array}{ll}{u}_{i,j,1}/\mu ,\hfill & k=1,\hfill \\ (u_{i,j,k-1}-u_{i,j,k})/\mu ,\hfill & 1<k<d,\hfill \\ -u_{i,j,k-1}/\mu ,\hfill & k=d.\hfill \end{array}$$

Furthermore, the gradient ∇₃, symmetric gradient $\mathcal{E}$ and their corresponding divergence operators are defined as follows:

$${\nabla }_3:U\to V,{\nabla }_{3}u=\left(\begin{array}{c}{\partial }_x^{+}u\\ {\partial }_y^{+}u\\ {\partial }_t^{+}u\end{array}\right),$$

$$\mathcal{E}:V\to W,\mathcal{E}(v)=\left(\begin{array}{c}{\partial }_x^{+}{v}_1\\ {\partial }_y^{+}{v}_2\\ {\partial }_t^{+}{v}_3\\ \frac{1}{2}\left({\partial }_y^{+}{v}_1+{\partial }_x^{+}{v}_2\right)\\ \frac{1}{2}\left({\partial }_t^{+}{v}_1+{\partial }_y^{+}{v}_3\right)\\ \frac{1}{2}\left({\partial }_t^{+}{v}_1+{\partial }_x^{+}{v}_3\right)\end{array}\right),$$

and

$$\mathrm{div}:V\to U,\mathrm{div}v={\partial }_x^{-}{v}_1+{\partial }_y^{-}{v}_2+{\partial }_t^{-}{v}_3,$$

$${\mathrm{div}}_2:W\to V,{\mathrm{div}}_{2}w=\left(\begin{array}{l}{\partial }_x^{-}{w}_1+{\partial }_y^{-}{w}_4+{\partial }_t^{-}{w}_6\hfill \\ {\partial }_x^{-}{w}_4+{\partial }_y^{-}{w}_2+{\partial }_t^{-}{w}_5\hfill \\ {\partial }_x^{-}{w}_6+{\partial }_y^{-}{w}_5+{\partial }_t^{-}{w}_3\hfill \end{array}\right).$$

By definition, the discrete divergences satisfy ${\nabla }_3^{*}=-\mathrm{div}$ and ${\mathcal{E}}^{*}=-{\mathrm{div}}_2$. In order to define the discrete version of ${\text{TGV}}_{\alpha }^2$, we also need the L¹ and L^∞ norms:

$$\begin{array}{l}u\in U:\Vert u{\Vert }_1=\sum _{(i,j,k)\in \mathrm{\Omega }}|u_{i,j,k}|,\\ \Vert u{\Vert }_{\infty }=\underset{(i,j,k)\in \mathrm{\Omega }}{\max }|u_{i,j,k}|,\end{array}$$

$$\begin{array}{l}v\in V:\Vert v{\Vert }_1=\sum _{(i,j,k)\in \mathrm{\Omega }}{\left(v_{1i,j,k}^2+v_{2i,j,k}^2+v_{3i,j,k}^2\right)}^{1/2},\\ \Vert v{\Vert }_{\infty }=\underset{(i,j,k)\in \mathrm{\Omega }}{\max }{\left(v_{1i,j,k}^2+v_{2i,j,k}^2+v_{3i,j,k}^2\right)}^{1/2},\end{array}$$

$$\begin{array}{l}w\in W:\Vert w{\Vert }_1=\sum _{(i,j,k)\in \mathrm{\Omega }}(w_{1,i,j,k}^2+w_{2i,j,k}^2+w_{3i,j,k}^2\\ {+2w_{4i,j,k}^2+2w_{5i,j,k}^2+2w_{6i,j,k}^2)}^{1/2},\end{array}$$

$$\begin{array}{l}\Vert w{\Vert }_{\infty }=\underset{(i,j,k)\in \mathrm{\Omega }}{\max }(w_{1i,j,k}^2+w_{2i,j,k}^2+w_{3i,j,k}^2\\ {+2w_{4i,j,k}^2+2w_{5i,j,k}^2+2w_{6i,j,k}^2)}^{1/2}.\end{array}$$

Algorithm description

In this paper, we use the Primal-Dual method [24] to solve the proposed model. The Primal-Dual method has been widely employed in finding the solution of the convex-concave saddle point problem:

$$\underset{x\in \mathcal{X}}{\min }\underset{y\in \mathcal{Y}}{\max }\langle Kx,y\rangle +F(x)-G(y),$$ (14)

where $\mathcal{X}$ and $\mathcal{Y}$ are Hilbert spaces, $K:\mathcal{X}\to \mathcal{Y}$ is a linear continuous mapping, and the functional $F:\mathcal{X}\to (-\infty ,\infty ]$ and $G:\mathcal{Y}\to (-\infty ,\infty ]$ are proper, convex and lower semi-continuous. In order to state the algorithm, we denote the resolvent operators as (I + τ∂F)⁻¹ and (I + σ∂G)⁻¹, which have the following closed-form representations:

$$x^{*}={(I+\tau \partial F)}^{-1}(\overline{x})=\underset{x\in \mathcal{X}}{\mathrm{argmin}}\frac{\Vert x-\overline{x}{\Vert }_{\mathcal{X}}^2}{2}+\tau F(x)$$

and

$$y^{*}={(I+\sigma \partial G)}^{-1}(\overline{y})=\underset{y\in \mathcal{Y}}{\mathrm{argmin}}\frac{\Vert y-\overline{y}{\Vert }_{\mathcal{Y}}^2}{2}+\sigma G(y),$$

respectively, where τ, σ > 0. Then given an initial point (x⁰, y⁰) ∈ $\mathcal{X}$ × $\mathcal{Y}$ and set ${\overline{x}}^0=x^0$, the Primal-Dual algorithm has the following iteration

$$\{\begin{array}{l}{y}^{n+1}={(I+\sigma \partial G)}^{-1}\left(y^n+\sigma K{\overline{x}}^n\right),\hfill \\ x^{n+1}={(I+\tau \partial F)}^{-1}\left(x^n-\tau K^{*}{y}^{n+1}\right)\hfill \\ {\overline{x}}^{x+1}=2x^{n+1}-x^n,\hfill \end{array}.$$ (15)

The algorithm converges if the condition $\tau \sigma \Vert K{\Vert }^2<1$ is satisfied with some τ, σ > 0.

Now we adopt the Primal-Dual algorithm to solve the proposed model. The corresponding Primal-Dual saddle point problem of model (13) is given by

$$\begin{array}{l}\underset{(S,w,L)\in U\times V\times U}{\min }\underset{(p,q,\lambda )\in V\times W\times Y}{\max }\langle {\nabla }_{3}S-w,p\rangle +\langle \mathcal{E}(w),q\rangle +\beta \Vert L{\Vert }_{*}\\ +\langle A(L+S)-B,\lambda \rangle -\frac{1}{2}\Vert \lambda {\Vert }_{\text{F}}^2\\ -{\mathcal{I}}_{\Vert \cdot {\Vert }_{\infty }\le {\alpha }_1}(p)-{\mathcal{I}}_{\Vert \cdot {\Vert }_{\infty }\le {\alpha }_0}(q),\end{array}$$ (16)

where p, q and λ are the dual variables, and $\mathcal{I}$ is the indicator function of a convex set. In order to make (16) coincide with the saddle-point structure (14), we choose

$$\mathcal{X}=U\times V\times U,\mathcal{Y}=V\times W\times Y,K=\left(\begin{array}{ccc}{\nabla }_3& -I& 0\\ 0& \epsilon & 0\\ A& 0& A\end{array}\right),$$

and

$$\begin{array}{l}F(x)=\beta \Vert L{\Vert }_{*},\\ G(y)=\langle B,\lambda \rangle +\frac{\Vert \lambda {\Vert }_{\text{F}}^2}{2}+{\mathcal{I}}_{\Vert \cdot {\Vert }_{\infty }\le {\alpha }_1}(p)+{\mathcal{I}}_{\Vert \cdot {\Vert }_{\infty }\le {\alpha }_0}(q).\end{array}$$

Thus the Primal-Dual method of the proposed model is shown in Algorithm 1.

Algorithm 1.

Primal-Dual Method for TGV and low rank decomposition.

Initialization: σ, τ, S0, L0, set L0 = A*B, S0 = 0

Iterations: Update (Sn, wn, Ln), (pn, qn, λn) and $\left({\overline{S}}^n,{\overline{w}}^n,{\overline{L}}^n\right)$ by the following steps:

$p^{n+1}={\mathcal{P}}_{{\alpha }_1}\left(p^n+\sigma \left({\nabla }_3{\overline{S}}^n-{\overline{w}}^n\right)\right)$

$q^{n+1}={\mathcal{P}}_{{\alpha }_0}\left(q^n+\sigma \mathcal{E}\left({\overline{w}}^n\right)\right)$

${\lambda }^{n+1}=\left({\lambda }^n+\sigma \left(A\left({\overline{L}}^n+{\overline{S}}^n\right)-B\right)\right)/(1+\sigma )$

$S^{n+1}=S^n-\tau \left(A^{*}{\lambda }^{n+1}-{\mathrm{div}}_1{p}^{n+1}\right)$

$w^{n+1}=w^n+\tau \left({\mathrm{div}}_2{q}^{n+1}+p^{n+1}\right)$

$L^{n+1}={\mathcal{S}}_{\beta }\left(L^n-\tau A^{*}{r}^{n+1}\right)$

${\overline{w}}^{n+1}=2w^{n+1}-w^n$

${\overline{L}}^{n+1}=2L^{n+1}-L^n$

Until convergence, return L + S

For updating dual variables p and q, the projection operator ${\mathcal{P}}_{\alpha }$ is given as

$$t^{*}={\mathcal{P}}_{\alpha }(\overline{t})=\underset{\Vert t{\Vert }_{\infty }\le \alpha }{\mathrm{argmin}}\frac{\Vert t-\overline{t}{\Vert }_{\text{F}}^2}{2}=\frac{\overline{t}}{\max \left(1,\frac{|\overline{t}|}{\alpha }\right)}.$$

For updating the primal variable L, the shrinkage operator ${\mathcal{S}}_{\beta }$ is defined as

$$t^{*}={\mathcal{S}}_{\beta }(\overline{t})=\underset{t}{\mathrm{argmin}}\frac{\Vert t-\overline{t}{\Vert }_{\text{F}}^2}{2}+\beta \Vert t{\Vert }_{*}=\text{U}{\mathcal{S}}_{\beta }(\mathrm{\Sigma }){\text{V}}^T,$$

where

$${\mathcal{S}}_{\beta }(\mathrm{\Sigma })=diag\left\{\max \left({\sigma }_i-\beta ,0\right)\right\},i=1,\dots ,r,$$

and U, Σ and V are a singular value decomposition of $\overline{t}$ of rank r.

Now we estimate the norm of operator K for the convergence condition $\tau \sigma \Vert K{\Vert }^2<1$. One can see that ${\Vert {\nabla }_3\Vert }^2<12$ and $\Vert \mathcal{E}{\Vert }^2<12$. Denote x = (S, w, L), then

$$\begin{array}{l}\Vert Kx{\Vert }^2={\Vert {\nabla }_{3}S-w\Vert }^2+\Vert \mathcal{E}w{\Vert }^2+\Vert AL+AS{\Vert }^2\\ <\left(\Vert A{\Vert }^2+12\right)\Vert S{\Vert }^2+2\sqrt{12}\Vert S\Vert \Vert w\Vert +13\Vert w{\Vert }^2+\Vert A{\Vert }^2\Vert L{\Vert }^2.\end{array}$$

Since $2\sqrt{12}\Vert S\Vert \Vert w\Vert \le a^2\Vert S{\Vert }^2+12\Vert w{\Vert }^2/a^2$ for all a > 0, we have

$$\Vert Kx{\Vert }^2<\left(\Vert A{\Vert }^2+12+a^2\right)\Vert S{\Vert }^2+\left(12/a^2+13\right)\Vert w{\Vert }^2+\Vert A{\Vert }^2\Vert L{\Vert }^2,$$

for a > 0. The smallest value is reached when $\Vert A{\Vert }^2+12+a^2=12/a^2+13$ which yields

$$\Vert K{\Vert }^2<\frac{\Vert A{\Vert }^2+25+\sqrt{{\left(\Vert A{\Vert }^2-1\right)}^2+48}}{2}.$$