Dynamic MRI using model-based deep learning and SToRM priors: MoDL-SToRM

Abstract

Purpose:

To introduce a novel framework to combine deep-learned priors along with complementary image regularization penalties to reconstruct free breathing & ungated cardiac MRI data from highly undersampled multi-channel measurements.

Methods:

Image recovery is formulated as an optimization problem, where the cost function is the sum of data consistency term, convolutional neural network (CNN) denoising prior, and SmooThness regularization on manifolds (SToRM) prior that exploits the manifold structure of images in the dataset. An iterative algorithm, which alternates between denoizing of the image data using CNN and SToRM, and conjugate gradients (CG) step that minimizes the data consistency cost is introduced. Unrolling the iterative algorithm yields a deep network, which is trained using exemplar data.

Results:

The experimental results demonstrate that the proposed framework can offer fast recovery of free breathing and ungated cardiac MRI data from less than 8.2s of acquisition time per slice. The reconstructions are comparable in image quality to SToRM reconstructions from 42s of acquisition time, offering a fivefold reduction in scan time.

Conclusions:

The results show the benefit in combining deep learned CNN priors with complementary image regularization penalties. Specifically, this work demonstrates the benefit in combining the CNN prior that exploits local and population generalizable redundancies together with SToRM, which capitalizes on patient-specific information including cardiac and respiratory patterns. The synergistic combination is facilitated by the proposed framework.

1 |. INTRODUCTION

Breath-held cardiac cine MRI is a key component in cardiac MRI exams, which is used for the anatomical and functional assessment of the heart. Unfortunately, several subject groups (e.g. children and chronic obstructive pulmonary disease (COPD) patients¹) cannot hold their breath and hence are excluded from breath-held MRI studies. In addition, breath-held protocols are also associated with long scan times. Several methods were introduced to reduce the breath-holding requirements or enable free breathing cardiac MRI protocols, including parallel MRI,^2,3 approaches that exploit the structure of x-f space,^4–6 compressed sensing schemes,^7,8 low-rank methods,^9,10 blind compressed sensing,^11,12 motion compensated methods,^13,14 and kernel low-rank methods.¹⁵ Motivated by recent deep learning frameworks for static imaging applications,^16–18 deep-learning based breath-held cardiac cine acceleration methods were also introduced.^19,20 Recently, several researchers have proposed to estimate cardiac and respiratory phases from the central k-space regions using band-pass filtering; the data are then binned to the respective phases, followed by reconstruction using compressed sensing^21,22 or low-rank tensor methods.^23,24 These methods depend on the accurate estimation of phases using prior information about the cardiac and respiratory rates, which may degrade with irregular respiration or arrhythmia. We have recently introduced the SToRM^25–28 framework as an alternative to explicit motion resolved strategies.^21–23 SToRM assumes that the images in the free-breathing dataset lie on a smooth and low-dimensional manifold parameterized by cardiac & respiratory phases. SToRM acquisition relies on navigator radial spokes, which are used to compute the manifold Laplacian matrix, to capture the structure of the manifold. Once the Laplacian matrix is available, the estimation of the dataset simplifies to a quadratic regularization scheme. SToRM can provide reliable free breathing reconstructions from around 40 seconds/slice of scan time, which ensures that the image manifold is well-sampled.

The main focus of this work is to further reduce the scan time of SToRM by combining the patient-specific SToRM prior with deep-learned priors, which are population generalizable. While the direct deep learning approaches^17,18 that estimate the images directly from the measured k-space data are computationally efficient, it is not straight forward for them to ensure data consistency or incorporate patient-specific priors. We hence rely on our recent model-based deep learning (MoDL) framework, which formulates the image recovery as an optimization scheme,^29,30 where the cost function is the combination of a data consistency term with a deep learned prior; the unrolling of an iterative algorithm to solve the above cost function translates to a deep network. This main difference of this scheme with other model based methods^16,19,31 is the use of embedded conjugate gradient (CG) blocks and the sharing of network parameters between iterations; our results in^29,30 show that sharing of trainable parameters across iterations reduces the training data requirement significantly, while the use of CG blocks within the deep network translates to improved results for a specified number of iterations. Note that the use of CG blocks results in a slightly longer run time compared to direct learning approaches.^17,18 However, the proposed framework still provides significantly shorter run times than classical compressed sensing strategies, thanks to the reduced number of iterations and reduced number of CG steps per iterations. In this work, we use this optimization-based framework for the seamless integration of the forward model (coil sensitivity information, sampling pattern) with SToRM priors and deep learning priors.

The proposed MoDL-SToRM cost consists of a data consistency term, a deep learned prior that learns population generalizable information, and the SToRM prior; the framework may also be used with other regularizers such as.^32–34 The CNN-based prior exploits local image redundancies of the 2D+time dataset.

By contrast, the SToRM prior exploits non-local redundancies between images in the dataset, which are specific to the cardiac and respiratory patterns of the subject. The regularization parameters that weigh the individual contributions of each term are also optimized during the training phase, eliminating the need for image-specific tuning of parameters. The combination of deep learning with other complementary priors in the context of free-breathing image reconstruction is not reported in the literature, to the best of our knowledge. These complementary priors enable the recovery from highly undersampled measurements, thus reducing the acquisition time by 5-10-fold over SToRM. While our focus is on free-breathing applications in this work, the algorithm may also provide good reconstructions of relatively less challenging breath-held applications.

2 |. METHODS

2.1 |. Acquisition scheme

Four healthy volunteers instructed to breathe normally were scanned at the Siemens Aera scanner in the University of Iowa hospitals to generate prospectively undersampled free-breathing ungated radial dataset. The data were acquired using a FLASH sequence with a 32 channel cardiac array. The scan parameters were TR/TE = 4.2/2.2 ms, number of slices = 5, slice thickness = 5 mm, FOV = 300 mm, spatial resolution = 1.17 mm. A temporal resolution of 42 ms was obtained by binning 10 consecutive lines of k-space per frame, including 4 uniform navigator lines. Each slice comprised of 10 000 radial lines of the k-space binned to 1000 frames, resulting in an acquisition time of 42s.

The raw k-space data were interpolated to a Cartesian grid and 7 virtual coils were approximated out of the initial 32 using a SVD-based coil-compression technique. The coil sensitivity maps were estimated from the compressed data using ESPIRiT.³⁵ The SToRM²⁵ reconstructed images were used as the reference to train the deep networks. We use subsets of the above data to demonstrate the utility of the proposed scheme.

2.2 |. MoDL-SToRM: formulation

We generalize the model-based deep learning framework (MoDL) by adding a SToRM prior:

$$\mathit{C}\left(\mathbf{X}\right)=\underset{\text{data}\text{consistency}}{\underbrace{{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_2^2}}+\frac{{\lambda }_1}{2}\underset{\text{CNN}\text{prior}}{\underbrace{{\Vert {\mathcal{N}}_{\mathbf{w}}\left(\mathbf{X}\right)\Vert }^2}}+\frac{{\lambda }_2}{2}\underset{\text{SToRM}\text{prior}}{\underbrace{\text{tr}\left({\mathbf{X}}^T\mathbf{LX}\right)}}.$$ (1)

Here, $\mathcal{A}$ is the multi-channel Fourier sampling operator, which includes coil sensitivity weighting. ${\mathcal{N}}_w$ is a 3-D CNN-based estimator that estimates the noise and alias patterns in the dataset from local neighborhoods of the 2D+time dataset; ${\Vert {\mathcal{N}}_{\mathbf{w}}\left(\mathbf{x}\right)\Vert }^2$ is a measure of the alias/noise contribution in the dataset X.²⁹ The denoised signal can thus be estimated from the data X as

$${\mathcal{D}}_{\mathbf{w}}\left(\mathbf{X}\right)=\left(\mathcal{I}-{\mathcal{N}}_w\right)\left(\mathbf{X}\right)=\mathbf{X}-{\mathcal{N}}_{\mathbf{w}}\left(\mathbf{X}\right).$$ (2)

Note that we can also express ${\mathcal{N}}_w\left(\mathbf{X}\right)=\mathbf{X}-{\mathcal{D}}_w\left(\mathbf{X}\right)$, where ${\mathcal{D}}_{\mathbf{w}}\left(\mathbf{X}\right)$ is the denoised version of X, when it is corrupted with noise and/or artifacts. However, we expect ${\mathcal{D}}_w\left(\mathbf{X}\right)=\mathbf{X}$ when X is an image free from noise and artifacts; i.e, ${\mathcal{N}}_w\left(\mathbf{X}\right)=0$ in this case. The above relation allows us to rewrite the second term in Equation (1) as the norm of the differences between the original and denoised images. The SToRM prior tr (X^TLX), exploits the similarities beyond the local neighborhood. The manifold Laplacian, L = D–W is estimated from the k-space navigators.²⁵ The diagonal matrix D is specified as D_(i,i) = Σ_j W_(i,j), where W is a weight matrix, such that, the weight W_(i,j) is high when x_i and xj have similar cardiac and/or respiratory phase. tr is the trace operator.

2.3 |. Alternating minimization algorithm

We expand the SToRM penalty as

$$\begin{gathered} 2\text{tr}\left({\mathbf{X}}^T\mathbf{L}\mathbf{X}\right)=2\text{tr}\left({\mathbf{X}}^T\left[\mathbf{D}-\mathbf{W}\right]\mathbf{X}\right) \\ =2\text{tr}\left({\mathbf{X}}^T\mathbf{D}\mathbf{X}\right)-2\text{tr}\left({\mathbf{X}}^T\underset{\mathbf{Q}}{\underbrace{\mathbf{W}\mathbf{X}}}\right) \end{gathered}$$

We consider temporary variables $\mathbf{Y}={\mathcal{D}}_{\mathbf{w}}\left(\mathbf{X}\right)$ and Q = WX and rewrite Equation (1) as:

$$\mathit{C}\left(\mathbf{X}\right)={\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_2^2+\frac{{\lambda }_1}{2}\underset{{\mathcal{N}}_w\left(\mathbf{X}\right)}{\underbrace{{\Vert \mathbf{X}-\mathbf{Y}\Vert }^2}}+{\lambda }_2\left(\text{tr}\left({\mathbf{X}}^T\mathbf{DX}\right)-\text{tr}\left({\mathbf{X}}^T\mathbf{Q}\right)\right),$$ (3)

We note that Equation (3) is equivalent to Equation (1) when $\mathbf{Y}={\mathcal{D}}_{\mathbf{w}}\left(\mathbf{X}\right)$ and Q = WX. Minimizing the objective with respect to X, assuming variables Y and Q to be fixed and determined from the previous iterations yields:

$${\nabla }_{\mathbf{x}}C={\mathcal{A}}^{\ast }\left(\mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\right)+{\lambda }_1\left(\mathbf{X}-\mathbf{Y}\right)+{\lambda }_2\left(\mathbf{D}\mathbf{X}-\mathbf{Q}\right)=0$$ (4)

where ${\mathcal{A}}^{\ast }$ is the adjoint of $\mathcal{A}$. This can be solved as

$$\mathbf{X}={\left({\mathcal{A}}^{\ast }\mathcal{A}+{\lambda }_1\mathbf{I}+{\lambda }_2\mathbf{D}\right)}^{-1}\underset{\mathbf{R}}{\underbrace{\left(\begin{array}{cc}{\mathcal{A}}^{\ast }\left(\mathbf{B}\right)+{\lambda }_1& \mathbf{Y}+{\lambda }_2\mathbf{Q}\end{array}\right)}}$$ (5)

As D is diagonal, ${\left({\mathcal{A}}^{\ast }\mathcal{A}+{\lambda }_1\mathbf{I}+{\lambda }_2\mathbf{D}\right)}^{-1}$ can be implemented on a frame-by-frame basis. We solve for Equation (5) for each frame of X using conjugate gradients algorithm. This provides us with an alternating algorithm:

$${\mathbf{Y}}_n={\mathcal{D}}_{\mathbf{w}}\left({\mathbf{X}}_n\right)$$ (6)

$${\mathbf{Q}}_n=\mathbf{W}{\mathbf{X}}_n$$ (7)

$${\mathbf{R}}_n=\left({\mathcal{A}}^{\ast }\left(\mathbf{B}\right)+{\lambda }_1{\mathbf{Y}}_n+{\lambda }_2{\mathbf{Q}}_n\right)$$ (8)

$${\mathbf{X}}_{n+1}={\left({\mathcal{A}}^{\ast }\mathcal{A}+{\lambda }_1\mathbf{I}+{\lambda }_2\mathbf{D}\right)}^{-1}{\mathbf{R}}_n.$$ (9)

Once the number of iterations is fixed, the network can be unrolled to yield a deep network as in Figure 1A. The parameters of ${\mathcal{D}}_{\mathbf{w}}$ and the optimization parameters λ₁ and λ₂ are trainable and shared throughout the iterations.

FIGURE 1.

Illustration of the proposed MoDL-SToRM framework. The proposed scheme is obtained by unrolling the iterations specified by Equations (6)–(9) as shown in (A). Each iteration consists of CNN denoiser ${\mathcal{D}}_w$, specified by Equation (6), SToRM update ${\mathbf{Q}}_i=\mathbf{W}{\mathbf{X}}_n$ specified by Equation (7), and data-consistency enforcement specified by Equation (9), as shown in (B). The CNN denoiser ${\mathcal{D}}_w$ is implemented as a residual network as shown in (C), where the architecture of ${\mathcal{N}}_w=\mathcal{I}-{\mathcal{D}}_w$ is shown in (D). Here, ${\mathcal{N}}_w$, the noise extractor operator. The main differences between this scheme and other model-based deep-learned schemes is the sharing of the weights across iterations as shown in (A) and the use of CG blocks to enforce the data-consistency in (B), when complex forward models such as multi-channel sampling is used. Note that unlike DC and ${\mathcal{D}}_w$ that involves local operations, the update of Q_n is global in nature; the direct implementation of the unrolled network in (A) is associated with high memory demand and is not feasible on current GPU devices. We use the training strategy in (E), where we use the lagged update of Q_n. Specifically, we perform a forward pass through the network to determine Q_n for all the frames in each training dataset. These Q_n parameters are stored. Batches of seven frames of X₀ and Q_n are fed into the network to update the network weights, which can be performed on the GPU. We propose to precompute Q_n in an outer-loop and update it less frequently than the network parameters

We initialize the iterations with the SToRM solution:

$${\mathbf{X}}_0=\text{arg}\underset{\mathbf{x}}{\text{min}}{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_2^2+\frac{\eta }{2}\text{tr}\left({\mathbf{X}}^T\mathbf{L}\mathbf{X}\right),$$ (10)

where η is fixed and chosen manually to produce the best SToRM results.

2.4 |. Network and training details

We note that the unrolled network described by Equations (6)–(9) is dependent on the weight matrix W, which captures the non-local similarities between image frames. For each training/testing dataset, we estimate W from navigators; the dataset and the corresponding W are used for training/testing. The rest of the variables including the regularization parameters λ₁, λ₂, and the weights of N_w are learned during training. Specifically, we train the unrolled network in an end-to-end fashion using different sets of {X₀, X_g, W} . Here X_g is the ground-truth, while W is fixed during training and is different for each training and test dataset. The CNN captures the local redundancies, which is independent from the non-local information in W. Since the end-to-end network sees different sets of training data, each with different W matrices, it learns the network parameters that are invariant/independent of the specific W.

2.4.1 |. Lagged update of Q_n

The training scheme requires the storage of Y_n, Q_n, and X_n. The straightforward training of the unrolled architecture in Figure 1A requires all these intermediate variables to be available on the GPU memory, which is often not feasible. We propose a lagged approach shown in Figure 1E, where Q_n is updated less frequently during training. We update Q_n iby making a forward pass through the network, assuming known network parameters. The Q_n, each corresponding to 200 frames, are then stored in the computer memory and assumed to be fixed during the inner iterations. The trainable network parameters specified by w, λ₁ and λ₂ are optimized in the inner loop on the GPU. We form batches of seven frames and the corresponding frames of the pre-computed Q_n for training. Following convergence of Equation (3) (inner-loop) for a fixed Q_n, we update Q_n and re-train the network, assuming the network parameters from the previous outer iteration as the initialization. We need multiple outer iterations for the training procedure to converge.

2.4.2 |. Training dataset

The data were acquired on four healthy volunteers, each but one with two different views–short axis and four chamber view–resulting in a total of seven datasets. We used the data from four datasets for training and remaining three for testing. We extracted three non-overlapping groups of 200 frames each from the above datasets, which were used for training. The SToRM reconstruction of the datasets from 1000 frames are considered as reference data, whereas, the input to the network was X₀, the solution to Equation (10) computed with reduced number of frames (200).

2.4.3 |. Trainable parameters of the network

The CNN block specified by ${\mathcal{D}}_{\mathbf{w}}$ consists of a 6 layer CNN with 64 filters of dimensions 3 × 3 × 3 in the first five layers, followed by two 3 × 3 filters in last layer. To deal with complex data, the real and imaginary part of the frames were passed as two channels of the input tensor. The total number of trainable parameters in the network is 151666 real variables. The sharing of the parameters across iterations provides good performance, while significantly reducing training data demand as shown in.²⁹

2.4.4 |. Training strategy

The network was trained with the Adam optimizer on mean squared error loss, implemented on TensorFlow and trained on a NVIDIA P100 GPU. We pre-trained the ${\mathcal{D}}_w$ to denoise various versions of SToRM-1000 reconstructions corrupted with different levels of noise, which took 18 hours (1200 epochs). Next, we trained a MoDL-SToRM with N = 1. Following a single iteration training, we considered a multiiteration model (N > 1), with the parameters initialized by the ones learned with N = 1.

We observed that a network with two iterations was sufficient to provide good reconstructions; the performance saturated beyond two repetitions. The total training time was 35 hours. The final inference for 8.4 s of data was from a single forward pass containing N = 2 repetitions, which takes around 28 second for all 200 frames. This is significantly faster than most compressed sensing reconstructions.

We also compare the proposed MoDL-SToRM reconstruction scheme against (a) SToRM alone and (b) Tikhonov-SToRM shown in Equation (11).

$$\mathit{C}\left(\mathbf{X}\right)=\underset{\text{data}\text{consistency}}{\underbrace{{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_2^2}}+\frac{{\lambda }_{\text{Tikh}}}{2}\underset{\text{Tikhonov}}{\underbrace{{\Vert \nabla \left(\mathbf{X}\right)\Vert }^2}}+\frac{{\lambda }_2}{2}\underset{\text{SToRM}\text{prior}}{\underbrace{\text{tr}\left({\mathbf{X}}^T\mathbf{L}\mathbf{X}\right)}}.$$ (11)

We consider reconstructions from 200 frames, corresponding to 8.4 seconds of acquisition time. All comparisons are made with SToRM reconstructions from 1000 frames (42 seconds) using the signal to error ratio metric (in addition to the standard PSNR and SSIM) defined as

$$\text{SER}=20{\text{log}}_{10}\left(\frac{\Vert {\mathbf{X}}_{1000}\Vert }{\Vert {\mathbf{X}}_{1000}-\mathbf{X}\Vert }\right),$$ (12)

where X₁₀₀₀ denotes the SToRM-reconstruction from 1000 frames and X is the specific reconstruction. The visual comparisons of the reconstructed images, their time profiles, and error images with SToRM reconstructions from 1000 frames as ground truth, are shown in Figures 2–4.

FIGURE 2.

Comparisons on the simulated dataset: A, Full view of a single frame from the simulated ground truth time series of 500 frames. Only (red) cropped myocardium region is shown in (B). B, Top row: Simulated ground truth time series of 500 frames. Following six rows are three sets of competing reconstructions and corresponding error (w.r.t to top row) images: (i) SToRM reconstruction with 100 frames, (ii) Tikhonov-SToRM reconstruction with 100 frames and (iii) proposed with 100 frames. First column is the time profile along a vertical cut across the myocardium shown in green in (A). Following three columns show three cardiac states at one respiratory stage. The position of the respiratory stage is marked blue on the time profile, in the first column. Three cardiac states are neighboring frames near the marked time point. The SER (dB) reported in the figure corresponds to the myocardium area

FIGURE 4.

Dataset 2: A, Full view of a single frame from the SToRM reconstruction using 1000 frames. Only (red) cropped myocardium region is shown. B, Top row: SToRM reconstruction using 1000 frames. Following six rows are three sets of competing reconstructions and corresponding error (w.r.t to top row) images: (i) SToRM reconstruction with 200 frames, (ii) Tikhonov-SToRM reconstruction with 200 frames and (iii) proposed with 200 frames. First column is the time profile along a vertical cut across the myocardium shown in green in (A). Following three columns show three cardiac states at one different respiratory stage. The position of the respiratory stage is marked blue on the time profile, in the first column. Three cardiac states are neighboring frames near the marked time point. The SER (dB) reported in the figure corresponds to the myocardium area

2.5 |. Simulated data

We generated simulated free breathing datasets by extracting three cardiac cycles of a SToRM-1000 reconstruction and deforming (using Bspline interpolation) them in space and time to generate six synthetic datasets. These simulated datasets (four training & two testing) were retrospectively undersampled using six golden angle radial lines & four uniform radial navigators. The results on a test dataset is shown in Figure 2, while quantitative comparisons are in Table 1.

TABLE 1.

Quantitative comparison of the methods on simulated dynamic datasets

Dataset	Method	SER (dB)	PSNR (dB)	SSIM
	SToRM	16.31	37.31	0.8868
Subject 1	Tikhonov-SToRM	18.78	39.77	0.9021
	Proposed	20.36	41.36	0.9386
	SToRM	14.54	33.33	0.8161
Subject 2	Tikhonov-SToRM	16.55	35.40	0.8783
	Proposed	20.12	38.97	0.9114
	SToRM	17.09	34.19	0.8345
Subject 3	Tikhonov-SToRM	19.02	36.17	0.8484
	Proposed	20.73	37.83	0.9021
	SToRM	15.04	34.60	0.6880
Simulated	Tikhonov-SToRM	15.29	34.85	0.6968
dataset 1	Proposed	23.43	42.98	0.9602
	SToRM	21.63	36.03	0.8641
Simulated	Tikhonov-SToRM	21.92	36.92	0.8696
dataset 2	Proposed	26.82	41.23	0.9721

We report the signal to error ratio (SER), peak signal to noise ratio (PSNR), and structural similarity index (SSIM). These metrics are reported for the entire field of view. By contrast, the SER (dB) metrics reported in the Figures are reported only for the myocardium area.

3 |. RESULTS

3.1 |. Selection of parameters

We first discuss how the parameters of the algorithm was selected.

3.1.1 |. Number of iterations N

We observe that the performance of MoDL-SToRM saturates with N. For example, for test dataset 1, we obtained PSNR of 37.13 dB, 40.68 dB, and 41.36 dB, with N=0, 1, and 2, respectively. The change in performance from N=2 to N=3 was negligible; we choose N=2 in the rest of the experiments.

3.1.2 |. Number of outer iterations in training N_out

We observe that few outer iterations were sufficient for the training to converge. We obtained PSNR of 40.62 dB 41.36 dB and 41.37 dB, when the number of outer iterations is 1, 2, and 3, respectively. We choose this setting for the rest of the experiments since the performance saturates at N_out = 2.

3.2 |. Comparisons with other methods

The comparisons on the simulated datasets in Figure 1 show that the proposed method provides the best reconstructions, which is also confirmed by the quantitative results shown in Table 1. The comparisons of the reconstructions from 200 frames in Figure 3 show that the proposed algorithm provides the most accurate reconstructions, revealed by the reduced errors and improved SER. We observe that the performance of SToRM suffers when the number of frames are reduced, evidenced by the high amount of noise like alias artifacts. Comparison of the proposed method with MoDL²⁹ (explained in Supporting Information Figure S1), is provided in the Supporting Information (Figure S3–S5, Table S1). MoDL only uses local information and is hence not able to provide high quality reconstructions for such high accelerations; however, we expect the MoDL to work well in breath-held applications such as.¹⁹ This signifies the need of the additional SToRM prior, which can exploit the non-local redundancy a simple CNN model cannot capture. The comparisons on prospective data shows that the proposed reconstruction from 8.4 seconds scan time is most comparable with SToRM from 42s of scan time, while the SToRM-alone reconstructions from 8.4 seconds scan time results in noise amplification.

FIGURE 3.

Comparisons on Dataset 1: A, Full view of a single frame from the SToRM reconstruction using 1000 frames. Only (red) cropped myocardium region is shown. B, Top row: SToRM reconstruction using 1000 frames. Following six rows are three sets of competing reconstructions and corresponding error (w.r.t to top row) images: (i) SToRM reconstruction with 200 frames, (ii) Tikhonov-SToRM reconstruction with 200 frames and (iii) proposed with 200 frames. First column is the time profile along a vertical cut across the myocardium shown in green in (A). Following three columns show three cardiac states at one respiratory stage. The positions of the respiratory stage is marked blue on the time profile, in the first column. Three cardiac states are neighboring frames near the marked time point. The SER (dB) reported in the figure corresponds to the myocardium area

4 |. DISCUSSION & CONCLUSION

We introduced a model-based framework, which can accommodate learnable CNN priors along with conventional SToRM regularizers, for the recovery of free breathing and ungated cardiac MRI data from radial acquisitions. The CNN exploits local population-generalizable redundancies, while the SToRM prior enables the use of patient specific nonlocal redundancies that depend on the cardiac and respiratory patterns. Our experiments show that very few iterations of Equations (6)–(9) is sufficient to provide good reconstructions. The fast saturation of performance with iterations is mainly due to the use of CG algorithm within the network. The improved performance in the context of limited training data can be attributed to the trainable parameters in the network, shared across iterations. The proposed scheme also provides a fast reconstruction time of around 30 seconds on a P100 GPU for the reconstruction of 200 frames.

We observe that from the error images in Figures 2 and 3 that the proposed approach results in reduced overall errors compared to competing methods, there are relatively higher residual errors around the image edges. Nevertheless, we note that the residual error is comparable or lower in magnitude than the ones obtained by other methods at all spatial locations, including at edges. The results also show that the MoDL-SToRM approach outperforms Tikhonov-SToRM, demonstrating the use of learnable priors. We observe from the figures in the supplementary information (S.3 & S.4) that the use of MoDL alone provides poor quality reconstructions, while its combination with SToRM provides improved results. We note that the undersampling factor needed to enable free breathing and ungated cardiac MRI is quite high (≈50 fold undersampling), compared to most deep learning-based acceleration schemes. MoDL uses local redundancies to recover these images, which results in poor reconstructions in this highly undersampled setting. The SToRM scheme facilitates the combination of information from different image frames. The reduced effective sampling resulting from reduced acquisition time causes increased errors, which the added MoDL regularization reduces. Although the CNN network architecture is same for MoDL and MoDL-SToRM, the two networks are trained to minimize different cost functions and hence the learned weights are expected to be different. We note that the size of the CNN in the proposed scheme is significantly smaller than those available in the literature; as shown in,^29,30 the sharing of weights between iterations allows us to significantly reduce the data demand required to avoid overfitting. The decay of validation error with training iterations as shown in the supplementary material also confirms that the model is not overfitting the data. This is also confirmed by the validation curves shown in the Supporting Information (Figure S2).

We rely on the alternating minimization strategy specified by Equations (6)–(9) to solve Equation (3). We have not rigorously studied the convergence of this algorithm to the objective in Equation (3), which is beyond the scope. Similarly, we have not studied the impact of the sampling pattern (e.g numbers of spokes) on the quality of the reconstructions in this note. The data were acquired using a sequence with four uniform radial navigators and six golden angle radial lines. This will be the focus of our future work. Each image is only sampled with 10 radial lines, which translates to 50 fold undersampling. MoDL-SToRM simultaneously exploits local and global redundancies to yield improved results. The SToRM scheme facilitates the combination of information from different image frames. With low number of frames, the SToRM-alone regularization results in increased errors, which the added MoDL regularization reduces significantly. MoDL and MoDL-SToRM share the network architecture (except for λ₂) but the learned weights differ significantly. In this work, we restricted our attention to L₂ loss metric to train the network. Several researchers have recently proposed alternate metrics for network training with improved results. We had experimented with L₂–L₁ losses with little improvement in quality. The performance of the algorithm may be improved using sophisticated training strategies including GAN, but is beyond the scope of this work.