Dynamic MRI using SmooThness Regularization on Manifolds (SToRM)

Abstract

We introduce a novel algorithm to recover real time dynamic MR images from highly under-sampled k-t space measurements. The proposed scheme models the images in the dynamic dataset as points on a smooth, low dimensional manifold in high dimensional space. We propose to exploit the non-linear and non-local redundancies in the dataset by posing its recovery as a manifold smoothness regularized optimization problem. A navigator acquisition scheme is used to determine the structure of the manifold, or equivalently the associated graph Laplacian matrix. The estimated Laplacian matrix is used to recover the dataset from undersampled measurements. The utility of the proposed scheme is demonstrated by comparisons with state of the art methods in multi-slice real-time cardiac and speech imaging applications.

I. Introduction

Dynamic MR imaging plays a central role in several applications such as structural and functional imaging of the heart, lung and liver, as well as vocal tract imaging in speech. While breath-held and ECG gated imaging is the default acquisition strategy in cardiac MRI, free-breathing un-gated acquisitions can enable the imaging of patients that have difficulty holding their breath [1] (e.g. COPD, obese, and paediatric subjects). Such free running sequences, where the acquisitions are not triggered by physiological signals, can also offer higher acquisition efficiency. The main challenge with free-breathing and ungated strategies (often termed as real-time (RT) imaging), is the slow nature of MR acquisition, which severely restricts the achievable spatial and temporal resolution.

Several model-based reconstruction algorithms that recover dynamic data from undersampled measurements have been introduced to improve the spatial and temporal resolution. The popular approaches include k-t SPARSE methods [2], [3], total variation (TV) regularization [4], and low rank methods such as k-t PCA [5] or partially separable functions (PSF) [6], [7]. k-t SPARSE methods model the intensity profiles as a sparse linear combination of exponentials. Temporal TV regularization relies on the similarity of each frame with its neighbours in time. PSF and k-t PCA methods exploit the linear dependencies between the intensity profiles by modelling them as a linear combination of basis functions, which are estimated from navigator signals. The main drawback of these schemes in the context of real-time MRI is the degradation in performance with extensive inter-frame motion.

We introduce a novel RT image acquisition and reconstruction method termed SToRM (SmooThness Regularization on Manifolds), which exploits the non-linear and non-local dependencies between images in the time series. In many RT applications, each image frame in the dataset is a non-linear function of a few physiological parameters (e.g. cardiac and respiratory phase in real-time cardiac cine). Thus the image frames can be modelled as points on a smooth and low dimensional non-linear manifold. Unlike motion resolved reconstruction strategies that bin the data to a few cardiac and respiratory phases and recover them, we propose to recover the entire dynamic dataset from the undersampled k-t data as a manifold smoothness regularized reconstruction problem. The proposed approach is inspired by the manifold regularization schemes that are widely used in machine learning applications [8], [9], [10]. This strategy requires the knowledge of the manifold structure, or equivalently the associated graph Laplacian operator. We introduce a navigator acquisition scheme to estimate the graph Laplacian matrix. We consider both ℓ₂ and ℓ₁ regularization penalties. We show that the ℓ₂-SToRM formulation can be solved analytically in the Fourier domain in the single receiver coil setting, while it can be solved efficiently using a simple conjugate gradients algorithm in the multi-channel case. We introduce a variable splitting based algorithm to solve for the ℓ₁-SToRM formulation. We demonstrate the utility of our method in accelerated cardiac and speech imaging. The comparisons of the proposed method with the state of the art methods show improved image quality. We expect that our proposed scheme can also be used to accelerate other MR imaging applications such as lung, bowel and liver imaging. The early version of the proposed algorithm was presented in the conference paper [11], and later extended in [12].

II. BACKGROUND

A. Acquisition scheme

We model the raw dynamic multi-channel MRI data from the i^th image frame x_i as:

$${\mathbf{b}}_{ij}=\underset{{\mathbf{A}}_{ij}}{\underbrace{{\mathbf{S}}_i{\mathbf{FC}}_j}}{\mathbf{x}}_i+{\eta }_{ij},j=1,\dots ,{\mathrm{N}}_{\text{coils}}$$ (1)

where C_j is the receive sensitivity of the j^th coil, S_i is the sampling pattern for the i^th frame and η_ij is the noise. F is the discrete Fourier transform matrix. The above can be simplified and re-written as:

$$\mathbf{B}=\mathcal{A}\left(\mathbf{X}\right)+\eta$$ (2)

where X = [x₁, . . ., x_k] is the Casorati matrix obtained by stacking the vectorized images as columns.

B. Manifold regularization

We model the images in the dynamic dataset as points on a smooth low-dimensional image manifold i.e. ${\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_k\in \mathcal{M}\subseteq {\mathbb{R}}^N$. Here $\mathcal{M}$ is a smooth m-dimensional manifold (m << N) and N specifies the number of pixels in an image. The regularized recovery of continuous multi-dimensional functions of a manifold has received considerable attention in the context of machine learning [9]. The problem is formulated as:

$$\widehat{f}=\arg \underset{f}{\min }\mathcal{V}\left(f\right)+\lambda {\int }_{\mathcal{M}}{\Vert {\nabla }_{\mathcal{M}}f\Vert }^{2}d\mathbf{x}$$ (3)

where f is the continuous function, $\mathcal{V}$ is the desired loss function and ${\nabla }_{\mathcal{M}}f$ is the derivative of f on $\mathcal{M}$. The second term contains the roughness prior on the manifold which can also be expressed as:

$${\int }_{\mathcal{M}}{\Vert {\nabla }_{\mathcal{M}}f\Vert }^{2}d\mathbf{x}=\langle {\nabla }_{\mathcal{M}}f,{\nabla }_{\mathcal{M}}f\rangle =\langle f,{\Delta }_{\mathcal{M}}f\rangle ={\int }_{\mathcal{M}}f{\Delta }_{\mathcal{M}}fd\mathbf{x}$$ (4)

where ${\Delta }_{\mathcal{M}}$ is the Laplace-Beltrami operator on the manifold. When one is only interested in recovering discrete function values specified by f = f₁, f₂, . . ., f_k at points x₁, x₂, . . ., x_k, the common practice is to approximate the problem as [9]:

$$\widehat{\mathbf{f}}=\arg \underset{\mathbf{f}}{\min }\mathcal{V}\left(\mathbf{f}\right)+\lambda \sum _i\sum _j{w}_{ij}{\Vert f_i-f_j\Vert }^2$$ (5)

where the weights w_ij are specified by:

$$w_{ij}={\mathbf{e}}^{-\frac{{\Vert {\mathbf{x}}_i-{\mathbf{x}}_j\Vert }^2}{{\sigma }^2}}$$ (6)

Note that the weights decay with distance. Specifically, w_ij will assume a high value if x_i and x_j are similar to each other, and a small value if they are different. The penalty term can also be expressed as:

$$\sum _i\sum _j{w}_{ij}{\Vert f_i-f_j\Vert }^2=2\mathrm{Tr}\left({\mathbf{fLf}}^H\right)$$ (7)

where Tr denotes the trace operator and L is the graph Laplacian operator. The L matrix is related to the weight matrix W (with entries defined by (6)) as:

$$\mathbf{L}=\mathbf{D}-\mathbf{W}$$ (8)

where D is a diagonal matrix with entries $\mathbf{D}(i,i)={\sum }_j{w}_{ij}$. For example, in a three node graph, the Laplacian is:

$$\mathbf{L}=\left[\begin{array}{ccc}\hfill w_{12}+w_{13}\hfill & \hfill -w_{12}\hfill & \hfill -w_{13}\hfill \\ \hfill -w_{12}\hfill & \hfill w_{12}+w_{23}\hfill & \hfill -w_{23}\hfill \\ \hfill -w_{13}\hfill & \hfill -w_{23}\hfill & \hfill w_{13}+w_{23}\hfill \end{array}\right]$$ (9)

Note the similarity between the discrete approximation (7) and (4). When the manifold is uniformly sampled, the discrete graph Laplacian operator converges to the Laplace Beltrami operator on the manifold in the limit (as the distance between samples tend to zero) [13]. When $\mathcal{M}={\mathbb{R}}^m$, then L is exactly the finite difference discretization of the continuous Laplacian operator on a regular lattice (up to a constant factor)[10]:

$${\Delta }_{\mathcal{M}}f\left(\mathbf{r}\right)=\sum _{i=1}^m\frac{f(\mathbf{r}+{\mathbf{e}}_i)+f(\mathbf{r}-{\mathbf{e}}_i)-2f\left(\mathbf{r}\right)}{{\delta }^2}=-\frac{{\left[\mathbf{Lf}\right]}_{\left(\mathbf{r}\right)}}{{\delta }^2}$$ (10)

where e₁, . . . e_m form an orthogonal basis for ${\mathbb{R}}^m$ with $\Vert {\mathbf{e}}_i\Vert =\delta$.

III. Proposed Scheme

We model the images in the dynamic dataset as points on a smooth low-dimensional manifold parameterized by a few variables. For example, the images in a free-breathing and ungated cardiac MRI dataset are non-linear functions of their cardiac and respiratory phases. The proposed framework is also general enough to be applied to other dynamic imaging applications like imaging of the vocal tract in speech, where there is no concept of phases equivalent to cardiac and respiratory phases in cardiac imaging. We propose to recover the dynamic dataset from its undersampled measurements (1) by exploiting the manifold structure of the data. Motivated by (5), we pose the recovery as:

$$\left\{{\mathbf{X}}^{\ast }\right\}=\arg \underset{\mathbf{X}}{\min }{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_F^2+\lambda \sum _i\sum _j{\left(\sqrt{w_{ij}}{\Vert {\mathbf{x}}_i-{\mathbf{x}}_j\Vert }_p\right)}^p$$ (11)

where we use the ℓ_p (p ≤ 2) norm of the image differences in the regularizer. We will consider the special cases p = 2 and p = 1 in the later subsections. The above optimization problem promotes solutions where each image frame is similar in the lp norm sense to its neighbours on the manifold and the degree of similarity is determined by the weights w_ij.

In classical manifold embedding applications, the weights are derived from the images themselves. This approach is not practical in our setting since we only have a few measurements available from each frame. We hence propose an acquisition strategy using navigators to estimate the weights to be used in (11). This approach is similar to [6], [7].

A. Estimation of Manifold Structure from Navigators

Consider that each of the k images is observed by the same M × N matrix Ψ (M < N). This mapping is a stable embedding if the distance between any two points x_i and x_j is preserved after the mapping Ψx_i. Wakin et al [14] have shown that a random orthoprojector Ψ provides a stable embedding of the manifold. Specifically, for some $0<ϵ\le \frac{1}{3}$ and a sufficient number of measurements M, the following holds with high probability for every pair x_i and x_j:

$$(1-ϵ)\Vert {\mathbf{x}}_i-{\mathbf{x}}_j\Vert \le \Vert \Psi {\mathbf{x}}_i-\Psi {\mathbf{x}}_j\Vert \le (1+ϵ)\Vert {\mathbf{x}}_i-{\mathbf{x}}_j\Vert$$ (12)

The number of measurements M required to yield stable embedding is independent of the ambient dimension N and is almost linearly proportional to the dimension of the manifold m. The required number of measurements also depends on the characteristics of the manifold which are captured by its condition number and volume [15].

Motivated by the above result, we propose to sample the same k-space locations in every temporal frame. We term the common measurements as navigator acquisitions, which are often used in many dynamic MRI applications for calibration [6], [7]. We define the measurement operator A_ij corresponding to the i^th frame and the j^th coil as (see (1)):

$${\mathbf{b}}_{i,j}=\underset{{\mathbf{A}}_{ij}}{\underbrace{\left[\begin{array}{c}\hfill \Phi \hfill \\ \hfill {\mathbf{B}}_i\hfill \end{array}\right]\mathbf{F}{\mathbf{C}}_j}}{\mathbf{x}}_i+{\eta }_{ij}$$ (13)

The first operator Φ samples the same k-space locations every frame, regardless of the frame number i; the corresponding samples (termed navigator signals) enable the estimation of the neighbours of each frame. The second operator B_i which samples different k-space locations every frame aids the image recovery algorithm by sampling the neighbours of a particular image frame at complementary k-space locations. We propose to estimate the inter-image distances as:

$$d_{ij}^2=\sum _{l=1}^{{\mathrm{N}}_{\text{coils}}}{\Vert {\mathbf{z}}_{il}-{\mathbf{z}}_{jl}\Vert }^2$$ (14)

where z_i,l are the navigator signals given by:

$${\mathbf{z}}_{i,l}=\Phi \mathbf{F}{\mathbf{C}}_l{\mathbf{x}}_i+{\eta }_{il},l=1,\dots ,{\mathrm{N}}_{\text{coils}}$$ (15)

We compute the weights as:

$$w_{ij}=\{\begin{array}{cc}{\mathbf{e}}^{-\frac{d_{ij}^2}{{\sigma }^2}}\hfill & ,\text{if}{\mathbf{x}}_i\text{and}{\mathbf{x}}_j\text{are neighbours}.\hfill \\ 0\hfill & ,\text{otherwise}.\hfill \end{array}\phantom{\}}$$ (16)

We set the neighbourhood of each frame to be a fixed number of nearest neighbours. For example, in order to retain the 5 nearest neighbours for each frame, the i^th and the j^th frames are considered to be neighbours if the i^th frame is among the 5 frames most similar to the j^th frame or the j^th frame is among the 5 frames most similar to the i^th frame.

B. Special case: ℓ₂ smoothness prior

When p = 2, the recovery using (11) simplifies to:

$${\mathbf{X}}^{\ast }=\arg \underset{\mathbf{X}}{\min }{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_F^2+2\lambda \mathrm{Tr}\left({\mathbf{XLX}}^H\right),$$ (17)

where the Laplacian matrix L is obtained from the weights using (8). We refer to this implementation as ℓ₂-SToRM. We can view (17) as an analysis formulation since the regularizer is based on the analysis of X (specified by XQ), where L = QQ^H. The problem (17) can be rewritten as:

$$\left\{{\mathbf{X}}^{\ast }\right\}=\arg \underset{\mathbf{X}}{\min }{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_F^2+2\lambda {\Vert \mathbf{XQ}\Vert }^2.$$ (18)

The k × k(k – 1)/2 matrix Q specifies a gradient operator. For example, in a 4 node graph, the matrix Q is specified by:

$${\mathbf{Q}}^T=\left[\begin{array}{cccc}\hfill {\sqrt{w}}_{12}\hfill & \hfill -{\sqrt{w}}_{12}\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\sqrt{w}}_{23}\hfill & \hfill -{\sqrt{w}}_{23}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill {\sqrt{w}}_{34}\hfill & \hfill -{\sqrt{w}}_{34}\hfill \\ \hfill {\sqrt{w}}_{13}\hfill & \hfill 0\hfill & \hfill -{\sqrt{w}}_{13}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill {\sqrt{w}}_{24}\hfill & \hfill 0\hfill & \hfill -{\sqrt{w}}_{24}\hfill \\ \hfill {\sqrt{w}}_{14}\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill -{\sqrt{w}}_{14}\hfill \end{array}\right]$$ (19)

Note that this approach is very similar to Tikhonov temporal regularization, when the sparse matrix Q is the temporal finite difference operator. The proposed scheme uses an operator that computes differences between the neighbours on the manifold, rather than the temporal neighbours. Since the neighbours on the manifold are expected to be more similar than the ones in time, we expect to obtain better recovery.

We will now show that this formulation is also equivalent to a synthesis formulation by a simple change of variables. In addition to providing additional insights, this offers an approach to represent the data efficiently, while working with large datasets. The Laplacian matrix has a singular value decomposition specified by:

$$\mathbf{L}=\mathbf{V}\Sigma {\mathbf{V}}^H$$ (20)

The eigen vectors of the Laplacian matrix denoted by v_i are ideally suited to represent smooth signals on the manifold. A simple special case worth discussing is when the graph has r disjoint clusters. In this case, spectral graph theory shows that L will have r zero singular values. The corresponding r singular vectors V₀ with an appropriate rotation matrix R will yield a set of sparse temporal basis functions:

$${\mathbf{E}}_0={\mathbf{RV}}_0$$ (21)

Each of the basis functions in E₀ will assume a value of zero for frames that are not in a particular cluster, and a constant value for all the frames in the cluster. This property is exploited in spectral clustering. If the images in the cluster are the same, these temporal basis functions are sufficient to represent the signal. Note that this representation is strikingly different from principle component analysis used in k-t PCA or PSF methods [6], [7]. Unlike the global subspace model used in these methods, the proposed approach captures the geometry of the data on the manifold, enabled by the non-linear mapping (16). By minimizing the cross talk between images in distinct cardiac/respiratory phases, it is expected to reduce temporal blurring.

In the general setting, one would need more basis functions to account for the variability of images within clusters/on the manifold. Substituting L in the regularization penalty term in (17), we obtain:

$$\mathrm{Tr}\left(\underset{\mathbf{U}}{\underbrace{\left(XV\right)}}\Sigma {\left(XV\right)}^H\right)=\sum _{i=1}^k{\sigma }_i{\Vert {\mathbf{u}}_i\Vert }^2,$$ (22)

where u_i = Xv_i is the projection of X onto the i^th singular vector v_i and σ_i is the i^th singular value of L. Substituting for X in terms of U in (17), we obtain the equivalent synthesis formulation:

$${\mathbf{U}}^{\ast }=\arg \underset{\mathbf{U}}{\min }{\Vert \mathcal{A}\left({UV}^H\right)-\mathbf{B}\Vert }_F^2+2\lambda \sum _{i=1}^k{\sigma }_i{\Vert {\mathbf{u}}_i\Vert }^2$$ (23)

Note that the above formulation is very similar to the k-t PCA or PSF [6] algorithms that are now widely used in dynamic MRI. The columns of U correspond to representative images, while the columns of V are the corresponding temporal basis functions.

C. Special case: ℓ₁ smoothness prior

We consider the ℓ₁ norm of the differences between neighbouring images on the manifold:

$$\left\{{\mathbf{X}}^{\ast }\right\}=\arg \underset{\mathbf{X}}{\min }{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_F^2+2\lambda {\Vert XQ\Vert }_1$$ (24)

We term this implementation ℓ₁-SToRM. Note that the above approach simplifies to the popular temporal total variation formulation when:

$$w_{ij}=\{\begin{array}{cc}1,\hfill & \text{if}j=i+1,i-1.\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}\phantom{\}}$$ (25)

We expect our method to achieve better reconstruction than temporal TV since it enforces the differences between the closest neighbours of a frame on the manifold to be sparse. These frames might not be the frames that are close to it temporally, especially in case of high motion between frames.

Considering that Q has a singular value decomposition:

$$\mathbf{Q}=\mathbf{V}{\Sigma }^{\frac{1}{2}}\mathbf{T}$$ (26)

we can also find the equivalent synthesis formulation for the ℓ₁ problem by a change of variable X = UV^H:

$${\mathbf{U}}^{\ast }=\arg \underset{\mathbf{U}}{\min }{\Vert \mathcal{A}\left({UV}^H\right)-\mathbf{B}\Vert }_F^2+2\lambda {\Vert \mathbf{U}{\Sigma }^{\frac{1}{2}}\mathbf{T}\Vert }_1,$$ (27)

Note that this approach has similarities to ℓ₁ regularized PSF regularization schemes [7], except that the ℓ₁ norm of $\mathbf{U}{\Sigma }^{\frac{1}{2}}\mathbf{T}$ is penalized rather than that of U.

IV. Implementation

We consider separately the solutions for the 2 cases described in the previous section: p = 2 and p = 1.

A. ℓ₂ smoothness prior

In the single coil case, problem (17) has an analytical solution in the Fourier domain. We rewrite (17) in this special case as:

$${\widehat{\mathbf{X}}}^{\ast }=\arg \underset{\widehat{\mathbf{X}}}{\min }\sum _i{\Vert {\mathbf{S}}_i{\widehat{\mathbf{x}}}_i-{\mathbf{b}}_i\Vert }_F^2+2\lambda \mathrm{Tr}\left(\widehat{\mathbf{X}}\mathbf{L}{\widehat{\mathbf{X}}}^H\right)$$ (28)

where the columns of X̂ = [x̂₁, . . ., x̂_k] are the Fourier coefficients of the images given by: x̂_i = Fx_i. The key observation is that the above expression can be decoupled into several independent subproblems, each involving the recovery of a row of X̂. Let X̂_(j,·) denote the j^th row of X̂ and B_(j,·) denote the vector of measurements corresponding to this row. Then, we can solve for X̂_(j,·) analytically as:

$${\widehat{\mathbf{X}}}_{(j,\cdot )}={({\mathbf{D}}_j^H{\mathbf{D}}_j+2\lambda \mathbf{L})}^{-1}{\mathbf{D}}_j^H{\mathbf{B}}_{(j,\cdot )}$$ (29)

where D_j is the sampling matrix corresponding to the j^th row. The solutions for the different rows of X̂ can be computed in parallel. This analytic approach can give us a significant speed-up over solving for the whole matrix X̂ using iterative algorithms such as conjugate gradient.

In the multi-channel setting, it is possible to solve for each coil using the above method and combine them using a sum-of-squares strategy. Since this approach is suboptimal, we propose to directly solve (17) using the conjugate gradient algorithm (accounting for the coil sensitivities) to obtain a more accurate solution. The gradient of the cost function in (17) can be computed as: $2{\mathcal{A}}^H\mathcal{A}\left(\mathbf{X}\right)+4\mathbf{XL}$. The computation of ${\mathcal{A}}^H\mathcal{A}\left(\mathbf{X}\right)$ can be broken down into blocks (each containing a few temporal frames of X) and the blocks can be processed in parallel in order to reduce computational complexity.

B. ℓ₁ smoothness formulation

We rely on a variable splitting strategy using an auxiliary variable Z to solve (24):

$$\{{\mathbf{X}}^{\ast },{\mathbf{Z}}^{\ast }\}=\arg \underset{\mathbf{X},\mathbf{Z}}{\min }{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_F^2+2\lambda {\Vert \mathbf{Z}\Vert }_{{\ell }_1}+\beta {\Vert \mathbf{XQ}-\mathbf{Z}\Vert }_F^2$$ (30)

We solve the above problem by alternating between minimization with respect to the 2 variables:

$${\mathbf{X}}^{\left(n\right)}=\arg \underset{\mathbf{X}}{\min }{\Vert \mathcal{A}\left(\mathbf{X}\right)-\mathbf{B}\Vert }_F^2+\beta {\Vert \mathbf{XQ}-{\mathbf{Z}}^{(n-1)}\Vert }_F^2$$ (31)

$${\mathbf{Z}}^{\left(n\right)}=\arg \underset{\mathbf{Z}}{\min }\beta {\Vert {\mathbf{X}}^{\left(n\right)}\mathbf{Q}-\mathbf{Z}\Vert }_F^2+2\lambda {\Vert \mathbf{Z}\Vert }_1$$ (32)

We use a homotopy continuation strategy on the parameter β, where β is initialized to a very small value and then increased gradually to a very large value till the algorithm converges. As in the ℓ₂ case, (31) can be solved analytically in the Fourier domain for single coil data. For multi-coil data, we use the conjugate gradient algorithm. (32) can be solved using shrinkage. The matrix Z is large and storing it explicitly will result in huge memory demands. We observe that the evaluation of (31) only requires ZQ^T, which is considerably smaller in dimension than Z. We perform in-place computation of the variable ZQ^T and store it instead of Z to reduce the memory demand of the algorithm.

C. Acquisition Scheme

The acquisition scheme used follows from the discussion in section III-A. We used a set of uniformly spaced radial navigator acquisitions (corresponding to Φ), separated by 180°/N_l degrees where N_l is the number of navigator lines per frame. The remaining k-space samples (corresponding to B_i) were acquired using a golden angle radial k-space trajectory, where each line was separated by an angle of 111.25° from the previous line. Thus, B_i varies from frame to frame. The acquisition and reconstruction pipeline is illustrated in Fig 1, where we consider the single coil setup for simplicity.

Fig. 1.

Summary of the proposed data acquisition and reconstruction scheme for the single coil case. The blue radial lines denote the navigators that sample the same k-space locations in every frame. The weight matrix is estimated from the k-space data acquired using these navigator lines as described in (16). The final images are recovered from the entire measurements by solving (11).

D. Datasets

We use a numerical cardiac phantom and a retrospectively undersampled speech dataset for quantitative comparisons. We also consider the recovery of prospectively undersampled real-time cardiac MRI data.

1) PINCAT phantom

A short axis view of the PINCAT phantom [16] heart with matrix size 128 × 128 and 500 frames was used for numerical simulations. The dataset has around 26 cardiac cycles and 5 respiration cycles.

2) Speech Imaging

We use the MR dataset titled ’F1’ in the USC-TIMIT database [17] to demonstrate our method. The raw k-space data for the images in the database was acquired using a spiral trajectory and this data was gridded to reconstruct the images. The reconstructed images have been made available in the dataset as a movie in the coil-combined form with matrix size 68 × 68 and frame-rate 23.18 frames/s. This corresponds to a temporal resolution of around 43 ms. The Fourier data corresponding to the first 6000 image frames was retrospectively undersampled using 9 golden angle radial lines and 1 spiral navigator per frame and used for our experiments.

3) Cardiac Imaging

A prospectively undersampled free-breathing ungated radial dataset was acquired using a SSFP sequence on a Siemens 3T TIM Trio scanner with a 18 channel cardiac array from a healthy volunteer who was asked to breathe normally. The scan parameters were TR/TE = 4.2/2.2 ms, number of slices = 5, slice thickness = 5 mm, FOV = 300 mm, spatial resolution = 1.17 mm. A temporal resolution of 42 ms was achieved by sampling 10 lines of k-space per frame, out of which 4 were navigator lines. 10000 radial lines of k-space were acquired per slice which resulted in an acquisition time of around 42 s per slice. For 5 slices this resulted in a total acquisition time of around 3.5 mins.

The raw k-space data was interpolated to a Cartesian grid and a SVD based coil-compression technique was used in order to create 4 virtual coil elements from the initial 18. We reconstructed low temporal resolution images for the original coils by binning k-space data from a large number of frames. We then performed an SVD on these images and retained only the 4 most significant singular vectors. The data from the original coils was coil-combined to form virtual coil data using the singular vectors obtained. This was done in order to reduce the computational complexity of the reconstruction procedure. The coil sensitivity maps were estimated from this compressed data using the method by Walsh et al [18]. To reduce computational complexity, the coil sensitivity maps were assumed to be constant over time.

4) Comparison between breath-held and free-breathing cardiac acquisitions

In order to compare the image quality obtained using our method to that obtained by a breath-held protocol, we acquired 2 cardiac datasets:

• A prospectively undersampled free-breathing ungated radial dataset.

• A fully-sampled breath-held ECG-gated radial dataset.

The first dataset was acquired using a SSFP sequence on a Siemens 3T TIM Trio scanner with a 5 channel cardiac array from a healthy volunteer who was asked to breathe normally. A TRUFI frequency scout was performed prior to data acquisition to prevent banding artifacts due to the presence of field in-homogeneity. The scan parameters were TR/TE = 3.2/1.62 ms, number of slices = 5, slice thickness = 5 mm, FOV = 300 mm, spatial resolution = 1.17 mm. A temporal resolution of 41.6 ms was achieved by sampling 13 lines of k-space per frame, out of which 4 were navigator lines. 13000 radial lines of k-space were acquired per slice which resulted in an acquisition time of around 42 s per slice. For 5 slices this resulted in a total acquisition time of around 3.3 mins.

The fully-sampled ECG-gated breath-held dataset was acquired by a SSFP sequence on the same subject immediately after the free-breathing scan. The sampling trajectory was uniform radial and the scan parameters were: TR/TE = 3.4/1.72 ms, number of slices = 5, slice thickness = 5 mm, number of channels = 5, FOV = 300 mm, spatial resolution = 1.17 mm, number of cardiac phases = 18, radial views per cardiac phase = 253. Each slice required a breath-hold of around 16 s followed by a resting period of around 25 s. For 5 slices this resulted in a total acquisition time of around 3 mins.

Pre-interpolation to a Cartesian grid, coil sensitivity estimation and coil compression were performed using the acquired k-space data as described in the previous section. 3 virtual coils were created in this case.

E. State of the art methods used for comparison

The in vivo data reconstructed using ℓ₂ and ℓ₁-SToRM was compared to the reconstructions by 3 other methods: kt-LR [19], temporal TV and PSF. The kt-LR and temporal TV methods do not require the acquisition of navigators. Thus, we did not include navigator lines in our sampling pattern for the speech data, for generating the results for these 2 methods. However, we could not do the same for the cardiac datasets since they were prospectively undersampled. For the PSF method, we used the Frobenius norm of the basis images as a regularizer. The approach followed was similar to [6], with the same weighting applied to all basis images. For all 3 competing methods, the regularization parameter giving the highest SER reconstruction was chosen in case of the speech dataset. For the cardiac dataset, since the ground-truth was not available, the regularization parameter which seemed to best preserve the features of the data was chosen. Spatial TV regularization was not used with any of the algorithms.

V. Results

A. Simulations using PINCAT phantom data

We first conducted some numerical simulations on the PINCAT phantom. Two frames of the phantom dataset are shown in Fig 2.(a) and Fig 2.(b).

Fig. 2.

Illustration of the weight matrix and the ability of the scheme to enable implicit motion resolved recovery. (a,b) Two frames from the PINCAT dataset. (c) Weight matrix computed from the fully sampled k-space data. The green and blue lines show the rows corresponding to the frames in (a) and (b) respectively. The neighbours of these frames can be obtained using the weight matrix. (d) Temporal intensity profile corresponding to the cut shown by the red dotted line in (a). Frames (a) and (b) and a few of their neighbours are marked.

1) Weight matrix estimate from fully sampled data

We computed the weight matrices from the fully sampled k-space data, corresponding to different σ values. These matrices were thresholded to retain only the 5 nearest neighbours for each frame. The k-space data was then under-sampled (10 lines per frame sampled on a pseudo golden angle trajectory). Images were reconstructed from this under-sampled data using ℓ₂-SToRM with the weight matrices corresponding to different σ values. The σ value giving the highest SER reconstruction was chosen to form the optimal weight matrix. This matrix is shown in Fig 2.(c). The temporal intensity profile of the original dataset (along the cut given by the red dotted line in Fig 2.(a)) is shown in Fig 2.(d). The frames in Fig 2.(a) and Fig 2.(b) and a few of their neighbours (obtained from the weight matrix) are marked along the profile. We observe that the frames in Fig 2.(a) and Fig 2.(b) are very similar to their neighbours estimated by the weight matrix.

2) Effect of navigator trajectory on weight matrix estimation

The effect of different navigator schemes on weight estimation is studied in Fig 3. The weights estimated from different trajectories were compared quantitatively to the ground-truth weights obtained from the fully sampled data (Section V-A1). The normalized ℓ₂ norm of the weight estimation error was used as the error metric. The optimal σ parameter varies from trajectory to trajectory, depending on the number of k-space points. We chose the best σ value in each case to obtain fair comparisons. We did not threshold the weight matrices for this experiment. We considered spiral and radial navigators with the same readout duration (TR = 4.3 ms). The percent errors in weight estimation (computed using ℓ₂ norm) are plotted in Fig 3.(a). We observe that 1 spiral shot (4.3 ms) is almost as accurate in estimating the weights as 5 radial lines (21.5 ms). The percent errors incurred in the two cases are 7.34% and 6.99% respectively. The 2^nd, 3^rd and 4^th eigen-vectors of the L matrix estimated from: (1) the fully sampled data, (2) 1 radial line, and (3) 1 spiral shot are shown in Fig 3.(b). The experiments show that the eigen vectors in all three cases are quite similar. We also observe that the 2^nd eigen-vector captures the respiratory motion of the data (5 respiratory cycles can be seen). The 3^rd eigen-vector shows the 2^nd harmonic of the respiratory motion. The dominant frequency of this eigen-vector is double that of the dominant frequency of the respiratory motion. The 4^th eigen vector captures the cardiac motion modulated by the respiratory frequency (26 cardiac cycles can be seen).

Fig. 3.

Effect of different navigator trajectories on weight matrix estimation. (a) Percentage error in the weight matrix estimation (computed using ℓ₂ norm), using different navigator trajectories. Spiral and radial trajectories are chosen such that the time taken to acquire 1 spiral shot is the same as that for 1 radial line. (b) The 2^nd, 3^rd and 4^th eigen vectors of the Laplacian matrix estimated from (1) fully sampled k-space, shown in blue (2) 1 radial spoke, shown in green (3) 1 spiral readout, shown in pink. We observe that these vectors capture the respiratory motion, the 2nd harmonic of the respiratory motion, and the cardiac motion modulated by the respiratory frequency respectively.

3) Effect of weight matrix on image reconstruction

The effect of different weight matrices (computed using the navigator trajectories described in Section V-A2) on image reconstruction quality is studied in Fig 4.(a). The phantom data was under-sampled in k-space using a golden angle trajectory with 10 lines per frame and this data was used for all reconstructions. The navigator data was used only for weight computation. The weight matrices were thresholded to retain only the 5 nearest neighbours for each frame. In Fig 4.(b), we show a single image frame from the time series, as reconstructed using different weight matrices. The weights computed using a 1 radial line navigator produced reconstructed images of comparable quality (31.87 dB) to the case of ground-truth weights (32.25 dB). The single shot spiral navigator trajectory, which takes the same acquisition time as 1 radial line, performed slightly better (32.18 dB) than the single radial line case. Estimation of weights using only the centre k-space signal gave very poor reconstructions (20.57 dB). Using a 3 × 3 patch around center k-space as the navigator signal (instead of the centre only) improved the results considerably (30.33 dB), though the error images show more artifacts than when using radial or spiral trajectories.

Fig. 4.

Effect of weight matrices estimated using different navigator trajectories on reconstruction. (a) Signal to error ratio of the reconstructions with the Laplacian matrix estimated from different navigator trajectories. The k-space samples used to reconstruct the images are the same for all cases (10 golden angle radial lines per frame). Only the navigator trajectory used to compute the weight matrix are varied. (b) A reconstructed frame is shown for a few of the trajectories reported in (a).

We clarify that for the above experiment we used the navigator data only for estimating the weights and not for reconstruction. However, the navigator data was used for reconstruction in all the subsequent in-vivo experiments on the speech and cardiac data. For the experiment in Sec V-A2, we were studying the relative merits of different sampling schemes on the weight computation. The analysis was extended in the above experiment, where we studied the effect of those computed weights on image reconstruction. If we included the navigator signals for the reconstruction step, then the quality of our reconstructed images would be dependent on: (1) The accuracy of the computed weights (2) The incoherence of the sampling patterns used for each trajectory. Since we were only studying effect (1), we used the same samples for reconstruction in each case.

B. Experiments on in vivo data

In the in vivo experiments, the parameter σ used for the calculation of the weight matrix W was automatically computed using the strategy described in [20]. For this purpose, we computed the weight matrix for a range of σ values and evaluated $l\left(\sigma \right)={\sum }_{i=1}^k{\sum }_{j=1}^k{W}_{ij}\left(\sigma \right)$ for each weight matrix obtained. A log-log plot of l(σ) revealed 2 constant asymptotes at σ → 0 and σ → ∞, smoothly connected by an approximately straight line. The approximate σ value at the middle of this linear portion was selected to form our weight matrix. The weight matrix was thresholded to retain only the 4-6 nearest neighbours for each frame. For the multi-slice cardiac datasets, we had to compute the weight matrix separately for each slice. This is because our acquisition scheme was 2D, i.e. the k-space samples from a particular slice were completely acquired before moving on to the next slice. The regularization parameter λ was chosen emperically. All reconstructions were done on a desktop computer (Intel Xeon E5-1620 CPU, 3.6 GHz, 32 GB RAM). The algorithms were implemented in MATLAB, and may be further optimized to produce lower reconstruction times.

1) Retrospective undersampling experiments on speech dataset

The speech dataset was reconstructed from under-sampled k-space data using different techniques, as shown in Fig 5. The first row (a) shows the ground-truth images, while each subsequent row corresponds to datasets reconstructed by different methods. The techniques used for reconstruction from under-sampled k-space data along with their reconstruction times are: (b) kt-LR (4.8 hrs) (c) temporal TV (21 mins) (d) PSF [6] (3 mins) (e) ℓ₂-SToRM (7 mins) (f) ℓ₁-SToRM (32 mins). For the kt-LR and temporal TV reconstructions, 10 golden angle radial lines of k-space were used per frame and no navigator lines were included. For the PSF and SToRM methods, k-space was undersampled using 9 golden angle radial lines and 1 spiral navigator per frame. SToRM produces reconstructions with higher SER than the other methods. Though the ℓ₁ and ℓ₂-SToRM reconstructions have comparable SER, it can be seen than the ℓ₁ formulation reduces blurring and preserves borders better. The competing techniques have more artifacts compared to the proposed methods, as pointed out in the figure. The ability to recover high quality images from under-sampled data indicates that our method can be used to improve the temporal resolution and also acquire multiple slices in a shorter scan.

Fig. 5.

Reconstruction of the speech dataset. (a) Ground-truth images. The subsequent rows correspond to reconstructions from under-sampled k-space data using (b) kt-LR, (c) temporal TV, (d) PSF, (e) ℓ₂-SToRM, and (f) ℓ₁-SToRM. The data used for (b) and (c) had a golden angle radial trajectory without navigators. The data used for (d), (e) and (f) had a spiral navigator. The arrows point out artefacts in the images reconstructed by the competing methods, which are not present in the images reconstructed by SToRM.

2) Recovery of prospectively undersampled RT cardiac dataset

The multi-slice free-breathing highly undersampled cardiac dataset described in Section IV-D3 was reconstructed using different methods, as illustrated in Fig 6. The techniques used for reconstruction in the different rows along with their reconstruction times are: (a) kt-LR (7.5 hrs) (b) temporal TV (4.7 hrs) (c) PSF (4 mins) (d) ℓ₂-SToRM (24 mins) (e) ℓ₁-SToRM (4.9 hrs). The temporal intensity profile along a vertical cut of the image frames (given by the red dotted line in Fig 6.(a)) is also shown for each method. The comparisons are only qualitative since the ground truth dataset was not available. We observe that SToRM reduces streaking artifacts and spatial blurring, compared to other state of the art methods. Specifically, we observe that the myocardial borders are well captured, while details such as the papillary muscles are better defined. We also note that while the image frames of the ℓ₁ and ℓ₂-SToRM reconstructions look similar, the temporal intensity profiles of the ℓ₁ formulation appear sharper.

Fig. 6.

Reconstruction of the free-breathing cardiac dataset. Selected image frames and temporal intensity profiles along a vertical cut given by the red dotted line in (a) are shown. The images were reconstructed from under-sampled k-space data using (a) kt-LR, (b) temporal TV, (c) PSF, (d) ℓ₂-SToRM, and (e) ℓ₁-SToRM. The arrows point out artefacts in the images reconstructed by the competing methods, which are not present in the images reconstructed by SToRM.

3) Comparison between free-breathing and breath-held cardiac reconstructions

The quality of the reconstructed free-breathing and breath-held cardiac datasets described in Section IV-D4 are compared in Fig 7. The breath-held dataset was reconstructed using CG-SENSE [21], while the free breathing dataset was reconstructed using ℓ₂-SToRM. We show the data corresponding to 2 out of the 5 reconstructed slices. The figure shows results from a particular slice of the breath-held dataset and also its best matching slice from the free-breathing dataset; it was difficult to find perfect matches between the breath-held and the free-breathing acquisitions. Fig 7 shows: (a) 3 cardiac phases from the breath-held cine reconstruction and temporal intensity profile along the yellow dotted line. (b) 3 frames from a single cardiac cycle of the free-breathing dataset and temporal intensity profile along a vertical cut (same cut as the breath-held dataset). Note that the breath-held dataset has a few cardiac phases averaged over many cardiac cycles, while the free-breathing dataset consists of several cardiac cycles. Images from the cardiac cycle of the free-breathing reconstructions which best matched the breath-held images are shown here. We observe that the dataset reconstructed using SToRM is of comparable quality to the breath-held cine datasets.

Fig. 7.

Comparison between proposed free-breathing (FB) reconstruction and breath-held (BH) reconstruction. The BH dataset was reconstructed using CG-SENSE. The FB dataset was recovered using ℓ₂-SToRM. Two matching slices from both datasets are shown. The rows represent different slices. (a) Images in different cardiac phases from the BH dataset. The voxel profiles along the yellow dotted line are also shown. (b) Image frames from a particular cardiac cycle of the FB dataset. The voxel profiles for a few cardiac cycles of the FB dataset are also shown (along the same cut as the BH dataset).

VI. DISCUSSION

The proposed dynamic imaging scheme estimates the proximity of the images on the manifold using navigator signals, followed by a manifold aware recovery of the images from highly undersampled measurements. The reconstructed image quality was observed to be superior to that achieved by other state-of-the-art ungated reconstruction methods. Moreover, the experiment on the speech dataset demonstrated that SToRM can recover images in case of repeating frames, irrespective of whether the repetitions are periodic. In fact, the method does not distinguish between periodic and aperiodic changes. The quality of our reconstructed images is quite dependent on the degrees of freedom of the underlying physiological process. If the degrees of freedom is low, then every frame will have a sufficient number of neighbours very similar to it with high probability (provided that our acquisition time is long enough). If the degrees of freedom is high, then many frames may not have any other frames very similar to it, and the recovered frames will be of poor quality. However, in such situations, other model-based reconstruction schemes should also perform poorly due to lack of redundancy in the data.

While the original stable embedding theory deals with random ortho-projectors [14], our empirical comparisons in section V show that the radial k-space sampling scheme can estimate the neighbourhood of each image frame quite accurately. Moreover, our experiments also show that approximate estimates of the weight matrix (using one radial line of k-space) are often sufficient to ensure good recovery of images. Our experiments also reveal that spiral navigators are more efficient than radial navigators. We used the radial acquisition scheme for ease of implementation on the scanner. We will investigate the utility of spiral navigators in the future, which may translate to improved temporal resolution or reconstruction quality.

The proposed scheme has a few free parameters: (1) σ (2) the number of neighbours (3) λ. The optimal σ value is dependent on the k-space trajectory as well as the number of points. However, we observed that the reconstruction quality is not very sensitive to the exact value of σ. Specifically, changing σ by a factor of 10 does not significantly affect the reconstruction quality. The number of neighbours is a data dependent parameter determined by the degree of redundancy in the dataset. If a sufficient number of similar frames is available for each frame, then a small increase in the number of neighbours will not affect the image quality. However, if the number of neighbours is made very high, then all the neighbours of a particular frame will not be very similar to it, and the resulting reconstructed image will have motion blur. If the number of neighbours is made very low, then we will have aliasing artefacts. Similarly, the regularization parameter λ is also data dependent.

We show that ℓ₂-SToRM has similarities to the k-t PCA and PSF methods, with the exception that the temporal basis functions are the singular vectors of the Laplacian matrix rather than that of the covariance matrix. These basis functions promote smoother solutions on the manifold, enabling the exploitation of the non-linear dependencies between images. ℓ₁-SToRM is similar to the temporal TV scheme, with the exception that the standard finite difference matrix is replaced by an adaptive finite difference operator; this enables the exploitation of non-local dependencies between images in the dataset. The ℓ₂-SToRM scheme has similarities to the recent work [22]. Specifically, their solution is a clever approximation of our analytic solution in the ℓ₂ setting for the single channel case. Our approach also has conceptual similarities to [23], where the cardiac and respiratory phase information is recovered from the singular vectors of the graph Laplacian. This approach has been inspired by dimensionality reduction methods such as ISOMAP and LLE [24], [25] that are used to embed the data on a manifold to a lower dimensional subspace. [23] identifies the cardiac and respiratory phases from the dimensionality reduced data, followed by explicit motion-resolved binned reconstructions similar to [26]. In contrast, SToRM performs an implicit motion-resolved recovery of the entire RT dataset. In addition, SToRM does not need the explicit identification of individual phases, which is difficult in applications with both cardiac and respiratory motion and require additional pre-processing steps [23], [27]. The estimation of the cardiac and respiratory phases using band-pass filtering as in [27], [26] may be challenging in cases with irregular respiratory motion and arrhythmia. In addition, many applications like speech imaging have no concept of phase equivalent to cardiac and respiratory phases in cardiac imaging. SToRM extends readily to such applications. The proposed scheme also has conceptual similarities to recent kernel PCA based approaches, introduced to exploit non-linear similarities between image patches. Specifically, [28] learns the basis functions using linear PCA on non-linearly transformed patches from low-resolution images. They then iterate between projecting each non-linearly transformed patch from the high-resolution images to this subspace, and solving for pre-images that satisfy data-consistency. This approach may be seen as a synthesis formulation of ℓ₂-SToRM, when re-engineered for image patches.

VII. CONCLUSION

We introduced a novel acquisition scheme and reconstruction algorithm for real-time dynamic MR imaging, termed SToRM. The central assumption is that the images in the dynamic dataset are points on a smooth, low dimensional manifold embedded in high dimensional space. We formulated the recovery of the dataset from highly under-sampled measurements as a manifold smoothness regularized optimization problem. The neighbours of each image on the manifold were estimated from the navigator acquisition. SToRM was demonstrated to be useful in accelerating free breathing cardiac imaging and speech imaging, without compromising on image quality and slice coverage. This approach improves the spatio-temporal resolution, while ensuring patient comfort and reducing the total scan time. It can be easily extended to other dynamic imaging applications like liver, bowel and lung imaging.