Extreme MRI: Large-scale volumetric dynamic imaging from continuous non-gated acquisitions

Abstract

Purpose:

To develop a framework to reconstruct large-scale volumetric dynamic MRI from rapid continuous and non-gated acquisitions, with applications to pulmonary and dynamic contrast-enhanced (DCE) imaging.

Theory and Methods:

The problem considered here requires recovering 100 gigabytes of dynamic volumetric image data from a few gigabytes of k-space data, acquired continuously over several minutes. This reconstruction is vastly underdetermined, heavily stressing computing resources as well as memory management and storage. To overcome these challenges, we leverage intrinsic three-dimensional (3D) trajectories, such as 3D radial and 3D cones, with ordering that incoherently cover time and k-space over the entire acquisition. We then propose two innovations: (a) A compressed representation using multiscale low-rank matrix factorization that constrains the reconstruction problem, and reduces its memory footprint. (b) Stochastic optimization to reduce computation, improve memory locality, and minimize communications between threads and processors. We demonstrate the feasibility of the proposed method on DCE imaging acquired with a golden-angle ordered 3D cones trajectory and pulmonary imaging acquired with a bit-reversed ordered 3D radial trajectory. We compare it with “soft-gated” dynamic reconstruction for DCE and respiratory-resolved reconstruction for pulmonary imaging.

Results:

The proposed technique shows transient dynamics that are not seen in gating-based methods. When applied to datasets with irregular, or non-repetitive motions, the proposed method displays sharper image features.

Conclusions:

We demonstrated a method that can reconstruct massive 3D dynamic image series in the extreme undersampling and extreme computation setting.

1 |. INTRODUCTION

Volumetric dynamic MRI has become an important component in a wide variety of applications, including pulmonary,^1,2 flow,^3,4 and dynamic contrast-enhanced (DCE) imaging.⁵ Among its many advantages, three-dimensional (3D) dynamic MRI enables high-quality multiplanar and temporal reformatting, which can greatly enhance clinical interpretations. Recent advances in parallel imaging,^6,7 compressed sensing⁸ and data sorting techniques⁹ have also substantially improved spatiotemporal resolution tradeoffs and motion robustness, which translate to many practical benefits in clinical settings.^10–12

Because volumetric dynamic MRI reconstruction is inherently underdetermined, most existing methods^2,9,13,14 rely on gating and data binning techniques to reduce reconstruction undersampling rates. In particular, these techniques exploit the periodicity or repeatability of the underlying dynamics, such as cardiac and respiratory motion. k-space samples are acquired over many cycles and are then sorted according to their corresponding phase in the cycle. Data sorting can be accomplished by leveraging external navigator signals, such as respiratory bellow or navigators derived from the MR data itself (self-gating). One can then either reconstruct a single phase (hard gating) or a weighted combination of the phases (soft gating).^5,15–17 All phases can also be jointly reconstructed to produce motion resolved images. This can be extended to include multiple dimensions, such as the two-dimensional parametrization of cardiac and respiratory phases in XD-GRASP.⁹ XD-flow¹⁸ further incorporates flow encoding. Recently, MR multitasking¹⁴ considers multiparametric mappings and bins measurements in five dimensions.

The main drawback of gating and data binning methods is that periodic assumptions may not hold in practice. Incompatible dynamics include bulk movements, coughing, and contrast enhancements. These transient dynamics can be lost in reconstruction or assigned incorrectly to one of the phases, causing image artifacts. Moreover, gating techniques require accurate estimations of the underlying respiratory signal, which can be challenging to obtain for patients with irregular motions. Finally, when integrated to applications like DCE, in which non-periodic dynamics are desired, these methods inherently limit the temporal resolution to be coarser than a respiratory cycle.

In this article, we aim to overcome limitations of periodicity constraints. In two-dimensional (2D) dynamic imaging, there are a wide range of works^19–25 on reconstructing non-gated real-time dynamic MRI at a high spatial resolution. However, 3D dynamic MRI reconstruction is vastly more underdetermined and demanding of computation and memory. While compressed sensing and low-rank reconstruction techniques enable better tradeoffs between spatial and temporal resolution, state of the art methods^5,26,27 can only achieve temporal resolution on the order of seconds. Alternatively, spatial resolution is usually compromised to achieve higher frame rates.^28,29 One reason for the limitations is that most existing techniques consider Cartesian or stack of two-dimensional (2D) non-Cartesian trajectories, such as stack-of-stars or stack-of-spirals, which do not sample efficiently in at least one dimension. These sampling trajectories are often preferred partly because of their computational efficiency: the reconstruction problem can be broken down into 2D sub-problems that can easily fit into memory of local servers and deployed concurrently across processors.

Instead, we leverage intrinsic 3D non-Cartesian trajectories, such as 3D radial¹ and 3D cones.³⁰ These trajectories with pseudo-random orderings can efficiently cover k-space in a short time interval, making them ideal for high resolution volumetric dynamic imaging. Moreover, their undersampling artifacts are diffused in the image domain along all directions, which fit well to the compressed sensing framework. However, even with these efficient trajectories, the reconstruction problem is still heavily underdetermined, and significantly more computationally and memory expensive. In particular, the problem requires recovering images on the order of a hundred gigabytes (GB) from a few GBs of measurements, and cannot be decoupled into smaller 2D sub-problems.

We propose two innovations to overcome these reconstruction challenges: (a) a compressed representation using the multiscale low-rank matrix factorization (MSLR)³¹ that both constrains the reconstruction problem and reduces its memory footprint, and (b) stochastic optimization to reduce computation, improve memory locality, and minimize communications between threads and processors.

MSLR was previously studied in³¹ and generalizes low rank (LR),^32,33 locally low rank (LLR)³⁴, and low rank + sparse (L+S)³⁵ matrix models. The representation considers all scales of correlations; local, global and sparse. Hence, it can obtain a more compact signal representation than other LR methods. In Section 2.1, we will combine the compact MSLR representation with the MRI acquisition model. A consequence of using this compressed representation is that the underlying dynamic sequence of images, which can require hundreds of GBs of storage, can be represented in mere few GBs. We propose an objective function that directly solves for the compressed representation. Such an approach makes it feasible to implement the reconstruction on local workstations. In the context of LR modeling, this is often referred to as the Burer-Monteiro factorization, which will be described in Section 2.3.

To further reduce computation, we propose using stochastic gradient descent (SGD)³⁶ for reconstruction. SGD is commonly used nowadays in machine learning to efficiently optimize over large-scale datasets for training neural networks. Inspired by this use, we apply SGD to large-scale volumetric dynamic MRI reconstruction. SGD allows us to reduce the number of non-uniform fast Fourier transforms (NUFFT) from thousands-per-iteration to a single one per iteration. In particular, incorporating stochastic optimization for the proposed method cuts down the reconstruction time from weeks to hours, and will be described in more detail in Section 2.4.

We demonstrate the feasibility of the proposed method in DCE imaging acquired with a golden-angle ordered 3D cones trajectory³⁰ and lung imaging acquired with a bit-reversed ordered 3D ultra-short time echo (UTE) radial trajectory.¹ Our results show that the proposed technique, reconstructed at near millimeter spatial resolution and sub-second temporal resolution, can visualize certain transient dynamics that are lost in gating-based reconstructions.

2 |. THEORY

2.1 |. Multiscale low rank

We begin by giving an overview of different LR models and motivate the use of MSLR for volumetric dynamic image reconstruction.

LR modeling^32,33 has been shown to be effective at representing static tissues, global contrast changes, or smooth dynamics in many dynamic imaging applications.^{24,28,37–39} The model assumes that images over time are linear combinations of a small number of principal components. Equivalently, the spatiotemporal matrix, formed by stacking images as column vectors, has low rank. Besides regularizing the reconstruction problem, explicit LR factorization also drastically reduces memory usage. This memory saving property was used in the many works^{24,28,37–39} for smaller data sizes and often for problems that can easily be decoupled into smaller 2D problems. In this article, the need for high compression rate becomes crucial as the problem cannot be separated into smaller 2D ones and is on the order of a 100 GB.

On the other hand, LR modeling does not exploit spatial localities of the underlying dynamics. For example, blood vessel dynamics would require basis vectors of the size of a full image to be represented. As a result, for problems requiring very low-dimensional representations, spatially localized dynamics are often lost in LR reconstruction, as shown in Section 4. To mitigate this, LLR was proposed by Trzasko et al³⁴ to exploit spatial locality by representing spatiotemporal matrices with block low-rank matrices. While effective for spatially localized dyamics, LLR no longer exploit global correlation, which can lead to flickering artifacts in practice. Another direction to improve LR is the L+S matrix factorization technique.³⁵ L+S separately represents static background dynamics as a LR matrix and fast transient dynamics as a sparse matrix. However, unlike LR or LLR, L+S cannot be used for compression as non-zero locations of sparse components are not known before reconstruction. In particular, sparse components in L+S would require storing the entire image sequence in order to perform reconstruction, which defeats the purpose of using a compressed representation for memory savings.

Here, we adopt the MSLR representation that generalizes the above mentioned LR models to represent dynamics at multiple scales. In MSLR, the spatiotemporal matrix is modeled as a sum of block-wise low-rank matrices with increasing scales of block sizes, as illustrated in Figure 1. This allows us to combine the benefits of both LR and LLR, using the decomposition technique introduced in L+S.

FIGURE 1.

Overview of the proposed method. We leverage intrinsic three-dimensional (3D) trajectories, such as 3D radial and 3D cones, with ordering that incoherently cover time and k-space over the entire acquisition. We then propose two innovations: A, a compressed representation using multiscale low-rank matrix factorization that both constrains the reconstruction problem and reduces its memory footprint. B, Stochastic optimization to reduce computation, improve memory locality, and minimize communications between threads and processors. This framework allows us to reconstruct large-scale volumetric dynamic MRI, which enables high-quality reformatting as shown in Supporting Information Video S1

Concretely, let T be the number of frames, N be the image size, and J be the number of scales for MSLR. And for scale j = 1, …, J, let N_j be the block size, B_j be the number of blocks and K_j be the maximum block matrix rank. Then for each block b = 1, …, B_j, let us define ${\mathbf{\text{L}}}_{jb}\in {ℂ}^{N_j\times K_j}$ be the block spatial bases, ${\mathbf{\text{R}}}_{jb}\in {ℂ}^{T\times K_j}$ be the block temporal bases, and ${\mathbf{\text{M}}}_{jb}\in {ℂ}^{N\times N_j}$ be a linear operator that embeds the input block matrix to the full image. The volumetric spatiotemporal matrix $\mathbf{\text{X}}\in {ℂ}^{N\times T}$ is represented as

$$\mathbf{\text{X}}=\sum _{j=1}^J\sum _{b=1}^{B_j}{\mathbf{\text{M}}}_{jb}{\mathbf{\text{L}}}_{jb}{\mathbf{\text{R}}}_{jb}^H$$ (1)

For notation simplicity, we consider stacked spatial and temporal bases ${\mathbf{\text{L}}}_j\in {ℂ}^{N_j{B}_j\times K_j}$ and ${\mathbf{\text{R}}}_j\in {ℂ}^{T{B}_j\times K_j}$, respectively, and a linear operator ${\mathcal{M}}_j:{ℂ}^{N_j{B}_j\times T{B}_j}\to {ℂ}^{N\times T}$ that embeds the stacked input to an image such that

$$\mathbf{\text{X}}=\sum _{j=1}^J{\mathcal{M}}_j\left({\mathbf{\text{L}}}_j{\mathbf{\text{R}}}_j^H\right).$$ (2)

2.2 |. Dynamic MRI forward model

We consider multichannel MRI acquisition systems with C coil channels and divide the overall scan into T frames. We assume the underlying image for each frame is approximately static for fine enough temporal resolution. That is, we consider view sharing with short non-overlapping time windows. Let M be the number of measurements for each frame, $\mathcal{A}:{ℂ}^{N\times T}\to {ℂ}^{MC\times T}$ be the overall sensing linear operator, which incorporates sensitivity maps and non-uniform Fourier sampling, and $\mathbf{\text{W}}\in {ℂ}^{MC\times T}$ represent the complex white Gaussian acquisition noise. Then, the k-space measurement matrix $\mathbf{\text{Y}}\in {ℂ}^{MC\times T}$ is given by

$$\mathbf{\text{Y}}=\mathcal{A}\left(\mathbf{\text{X}}\right)+\mathbf{\text{W}}.$$ (3)

Since the number of measurements is vastly smaller than the total dynamic image size, the reconstruction problem is severely underdetermined. Compressed sensing MRI⁸ proposed a framework of using an incoherent sampling acquisition, a sparsifying signal representation, and a non-linear algorithm for underdetermined MRI reconstruction. While actual measurement systems deviate from theoretical assumptions in,^40,41 the framework provides practical guidance on sampling designs and motivated the use of low-dimensional models. Following this structure, we leverage intrinsic 3D non-Cartesian trajectories, such as 3D radial,¹ and 3D cones,³⁰ with ordering that incoherently cover time and k-space over the entire acquisition. As for the underlying model, we use MSLR in the previous subsection for compactly representing the underlying volumetric image sequence. The overall forward model is then given by,

$$\mathbf{\text{Y}}=\mathcal{A}\left(\sum _{j=1}^J{\mathcal{M}}_j\left({\mathbf{\text{L}}}_j{\mathbf{\text{R}}}_j^H\right)\right)+\mathbf{\text{W}}.$$ (4)

2.3 |. Memory-efficient formulation using the Burer-Monteiro factorization

With the forward model, one way to impose low-rank constraints is through convex relaxation by minimizing the nuclear norm,^42–44 or equivalently the sum of singular values. In particular, let ${\Vert \mathbf{\text{X}}\Vert }_{*}$ denote the nuclear norm of a matrix X. Then for each scale j, let λ_j be the regularization parameter and ${\Vert \mathbf{\text{X}}\Vert }_{\left(j\right)}={\sum }_{b=1}^{B_j}{\Vert {\mathbf{\text{M}}}_{jb}^H\mathbf{\text{X}}\Vert }_{*}$ denote the block-wise nuclear norm, the convex formulation considers the objective function,

$$f\left(\mathbf{\text{X}}\right)=\frac{1}{2}{\Vert \mathbf{\text{Y}}-\mathcal{A}\left(\sum _{j=1}^J{\mathcal{M}}_j\left({\mathbf{\text{X}}}_j\right)\right)\Vert }_2^2+\sum _{j=1}^J{\lambda }_i{\Vert {\mathbf{\text{X}}}_i\Vert }_{\left(j\right)}.$$ (5)

When the measurement system is the identity matrix, theoretical properties of MSLR decomposition using the nuclear norm were studied in.³¹ While this setting substantially deviates from our forward model, it provides practical guidance on selecting regularization parameters. In particular, under the identity setup and certain idealized incoherent conditions on the basis vectors, MSLR decomposition can be recovered when ${\left\{{\lambda }_j\right\}}_{j=1}^J$ are chosen such that

$${\lambda }_j\propto \sqrt{N_j}+\sqrt{T}+\sqrt{2\text{log}{B}_j}.$$ (6)

This relationship is useful because we would only need to tune one scaling parameter instead of J regularization parameters. In our experiments, we chose ${\left\{{\lambda }_j\right\}}_{j=1}^J$ following Equation 6.

However, a significant downside of the convex formulation is that it uses significant amount of memory. This is because full sized spatiotemporal matrices are stored even when they are extremely low rank. In particular, for an image size of 320 × 320 × 320 and 500 frames, merely storing the image in complex single precision floats requires 125 GBs! Applying iterative algorithms would require a few times more memory for workspace, which can approach terabytes. Local workstations, or even computing clusters, have difficulties handling such memory demand.

Rather than minimizing the convex problem in Equation 5, we consider the Burer-Monteiro factorization^43,45 to directly solve for the compressed representation. It considers the following non-convex transformation,

$${\Vert \mathbf{\text{X}}\Vert }_{*}=\underset{\mathbf{\text{X}}\hspace{0.17em}=\hspace{0.17em}\mathbf{\text{L}}{\mathbf{\text{R}}}^H}{\text{min}}\frac{1}{2}\left({\Vert \mathbf{\text{L}}\Vert }_F^2+{\Vert \mathbf{\text{R}}\Vert }_F^2\right),$$ (7)

and minimizes the following equivalent objective function:

$$\begin{gathered} f\left(\mathbf{\text{L}},\mathbf{\text{R}}\right)=\frac{1}{2}{\Vert \mathbf{\text{y}}-\mathcal{A}\left(\sum _{j=1}^J{\mathcal{M}}_j\left({\mathbf{\text{L}}}_j{\mathbf{\text{R}}}_j^H\right)\right)\Vert }_2^2 \\ +\sum _{j=1}^J\frac{{\lambda }_j}{2}\left({\Vert {\mathbf{\text{L}}}_j\Vert }_F^2+{\Vert {\mathbf{\text{R}}}_j\Vert }_F^2\right). \end{gathered}$$ (8)

The primary benefit of this formulation is that it can use significantly less memory for low-rank matrices. For a typical 3D volume, the variables can even fit the variables in graphic processing units (GPU). However, the main drawback is that the problem becomes non-convex. Nonetheless, in practice, we observe that applying first-order gradient methods on the non-convex objective function reaches decent solutions. In the Discussion section, we point to recent theoretical results on the Burer-Monteiro factorization to motivate the non-convex formulation. However, we emphasize that there is currently no theoretical guarantees for the sensing system considered here.

Before moving to the algorithm, we note that the proposed formulation can easily incorporate priors of the temporal dynamics, such as smoothness. For example, given a finite difference operator $\mathbf{\text{D}}\in {ℂ}^{T-1\hspace{0.17em}\times T}$, the above objective function can be modified to enforce temporal smoothness as follows:

$$\begin{gathered} f\left(\mathbf{\text{L}},\mathbf{\text{R}}\right)=\frac{1}{2}{\Vert \mathbf{\text{y}}-\mathcal{A}\left(\sum _{j=1}^J{\mathcal{M}}_j\left({\mathbf{\text{L}}}_j{\mathbf{\text{R}}}_j^H\right)\right)\Vert }_2^2 \\ +\sum _{j=1}^J\frac{{\lambda }_i}{2}\left({\Vert {\mathbf{\text{L}}}_i\Vert }_F^2+{\Vert \mathbf{\text{D}}{\mathbf{\text{R}}}_j\Vert }_F^2\right). \end{gathered}$$ (9)

2.4 |. Stochastic reconstruction

Even with the explicit LR factorization, performing conventional iterative reconstruction is still prohibitively slow, requiring computing many NUFFTs per iteration, one for each time frame and channel. To address this, we propose to apply stochastic optimization techniques to accelerate the reconstruction, which update variables in each iteration with random subsets of measurements.

The idea of using subsets of measurements for iterative updates has been proposed in the general medical imaging community. Examples include ordered subset algorithms for positron emission tomography reconstruction^46,47 and the algebraic reconstruction technique for computed tomography.⁴⁸ In MRI, Muckley et al⁴⁹ applied incremental gradient methods to parallel imaging reconstruction. A recent work⁵⁰ also proposed to use stochastic optimization for dynamic MRI reconstruction, but only demonstrated on gated k-space datasets. Our contribution is to show how stochastic optimization can transform an almost computationally infeasible reconstruction to a practical one.

In particular, let us split the objective function into separate frames and coils,

$$\begin{gathered} f_{tc}\left(\mathbf{\text{L}},\mathbf{\text{R}}\right)=\frac{1}{2}{\Vert {\mathbf{\text{Y}}}_{tc}-{\mathcal{A}}_{tc}\left({\sum }_{j=1}^J{\mathcal{M}}_j\left({\mathbf{\text{L}}}_j{\mathbf{\text{R}}}_{jt}^H\right)\right)\Vert }_2^2 \\ +{\sum }_{j=1}^J\frac{{\lambda }_j}{2}\left(\frac{1}{TC}{\Vert {\mathbf{\text{L}}}_j\Vert }_F^2+\frac{1}{C}{\Vert \mathbf{\text{D}}{\mathbf{\text{R}}}_j\Vert }_F^2\right), \end{gathered}$$ (10)

for t = 1,…,T and c = 1,…C, then $f\left(\mathbf{\text{L}},\mathbf{\text{R}}\right)={\sum }_{t=1}^T{\sum }_{c=1}^C{f}_{tc}\left(\mathbf{\text{L}},\mathbf{\text{R}}\right)$.

In each iteration, SGD randomly selects a frame t and a coil c, and performs the following updates:

$$\begin{gathered} \mathbf{\text{L}}=\mathbf{\text{L}}-\alpha TC{\nabla }_{\mathbf{\text{L}}}{f}_{tc}\left(\mathbf{\text{L}},\mathbf{\text{R}}\right) \\ {\mathbf{\text{R}}}_t={\mathbf{\text{R}}}_t-\alpha C{\nabla }_{{\mathbf{\text{R}}}_t}{f}_{tc}\left(\mathbf{\text{L}},\mathbf{\text{R}}\right), \end{gathered}$$ (11)

where α is a step-size parameter. In the Methods section, we describe more in detail how to select α.

With SGD, the number of NUFFTs per iteration is drastically reduced to 1. Moreover, SGD can easily support parallel processing by performing mini-batch updates. Given G parallel processors, such as multiple GPUs, each processor g can choose a different frame t and a different coil c, calculate the gradients ${\nabla }_{\mathbf{\text{L}}}{f}_{tc}\left(\mathbf{\text{L}},{\mathbf{\text{R}}}_t\right)$ and ${\nabla }_{{\mathbf{\text{R}}}_t}{f}_{tc}\left(\mathbf{\text{L}},\mathbf{\text{R}}\right)$, and additively synchronize afterward.

Concretely, in each iteration, SGD randomly selects an index set $\mathcal{I}=\left\{\left(t_1,c_1\right),\left(t_2,c_2\right),\dots ,\left(t_G,c_G\right)\right\}$. Then, SGD performs

$$\begin{gathered} \mathbf{\text{L}}=\mathbf{\text{L}}-\frac{\alpha TC}{G}\sum _{\left(t,c\right)\in \mathcal{I}}{\nabla }_{\mathbf{\text{L}}}{f}_{tc}\left(\mathbf{\text{L}},\mathbf{\text{R}}\right) \\ {\mathbf{\text{R}}}_t={\mathbf{\text{R}}}_t-\frac{\alpha C}{G}\sum _{c:\left(t,c\right)\in \mathcal{I}}{\nabla }_{{\mathbf{\text{R}}}_t}{f}_{tc}\left(\mathbf{\text{L}},\mathbf{\text{R}}\right)\hspace{0.17em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}t\hspace{0.17em}\text{such}\hspace{0.17em}\text{that}\hspace{0.17em}\left(t,c\right)\in \mathcal{I}\hspace{0.17em}, \end{gathered}$$ (12)

where only the summations are synchronized across the processors, thus minimizing expensive communications.

3 |. METHODS

We demonstrated the feasibility of the proposed method on DCE datasets acquired with a golden-angle ordered 3D cones trajectory³⁰ and lung datasets acquired with a bit-reversed ordered 3D UTE radial trajectory.¹ In general, our experiments aim to show that the proposed techniques enables large-scale volumetric dynamic MRI reconstruction. On the other hand, we emphasize that targeted reconstruction resolution does not necessarily translate to the true apparent resolution. Dynamics can be blurred and features can be lost. Hence, another purpose of our comparisons is to see what additional dynamics can be seen in the proposed reconstruction and what artifacts arise.

We first describe implementation details used in both applications. The proposed reconstruction was implemented in Python using the package SigPy⁵¹ on workstations with two Intel Xeon Gold CPUs and four Titan Xp GPUs. All operations, except loading the data and splitting into frames, were performed on the GPUs.

The number of scales in MSLR was determined by the GPU memory constraint. In particular, we used three scales with block widths 32, 64, and 128 for small datasets (the first two DCE datasets and the first lung dataset) and block widths 64, 128, and 256 for large datasets (the third DCE dataset and the second lung dataset). Because MSLR, like orthogonal wavelet transforms, is not shift invariant, blocking artifacts can arise in practice. To prevent this, each block is overlapped by half the block size in all spatial dimensions. Each voxel in the end was represented by eight basis vectors in each scale, including the overlapped bases.

To ensure the same parameters provide similar performance for different datasets, the forward operators and k-space datasets were normalized appropriately. In particular, the NUFFT operators were normalized by the maximum singular value of the NUFFT operator of the first frame, which was estimated using 10 iterations of power method. The k-space data were scaled by ${\Vert {\mathbf{\text{x}}}_{\text{grid}}\Vert }_2/\sqrt{T}$, where x_grid is the time-averaged density compensated gridding reconstructed image. We found that this normalization scheme made regularization parameters less sensitive to dataset scaling and the number of frames. k-space data were prewhitened using additional noise measurements collected with each scan. Sensitivity maps were estimated using ESPIRiT⁵² on time-averaged k-space datasets.

The spatial bases {L_i}_i and temporal bases {R_i}_i were initialized as complex white Gaussian noise vectors with unit norm. To choose the step-size α, we first set it to 1, and run the iteration updates. If the iteration diverged with the gradient norm reaching the numerical limit, the reconstruction was restarted and the step-size was halved. In practice, very few iterations were needed to detect divergence, as SGD is known to diverge exponentially⁵³ when the step-size is too large.

The regularization parameters were chosen to be ${\lambda }_j=\lambda \left(\sqrt{N_j}+\sqrt{T}+\sqrt{2\text{log}{B}_j}\right)$, with λ being a tuned parameter. Due to the small image size of the second DCE dataset described below, we performed parameter search on it for a 20-frame reconstruction and used the same parameters for all other datasets. In particular, λ = 10⁻⁴ results in the best subjective tradeoff between noise-level and spatiotemporal blurring. Supporting Information Figure S1 shows an example of the tradeoffs. Following common practices in machine learning, SGD selects time frames and coils sequentially from shuffled instances of {1, …, T} × {1, …, C}. Objective values over iterations were also recorded to set the number of epochs. The number of epochs was 60, which means SGD goes through the entire k-space dataset 60 times, resulting in a total of 60 TC/G SGD update steps.

We compared the proposed reconstruction with gating-based methods, which required respiratory motion signals. We estimated them from filtered k-space center signals. There are two reasons for using k-space DC based navigators: (a) it is self-gated, which avoids drifting and asynchronous issues associated with respiratory bellows, and (b) completely automated, which does not need manual selections of region of interests as in image navigators. Symmetric extension was performed on the k-space center before filtering to prevent edge effects. The filter was optimized in the least squares sense to have a pass band between 0.1 and 1 Hz and stop band everywhere else with a transition width of 0.05 Hz. The filtered signal was then robustly normalized by subtracting the median, and dividing by the median absolute deviation.

When the patient scans were performed, the prescribed field-of-views (FOV) were all smaller than the patient bodies, often with patient’s arms outside the FOV. For iterative reconstruction, small FOVs result in artifacts from model mismatch. Instead of using the prescribed FOV, we estimated the matrix size from a low-resolution time-averaged image with a large reconstruction FOV. In particular, the image was reconstructed at twice the prescribed FOV with a gridding reconstruction using the first 100 samples along all readouts. We then applied a threshold of 0.1 of its maximum amplitude to estimate the FOV.

3.1 |. DCE with golden-angle ordered 3D cones trajectory

We applied the proposed reconstruction on three DCE datasets from pediatric patients to qualitatively evaluate its performance. All datasets were acquired on GE 3T scanners using a spoiled gradient echo (SPGR) sequence and a golden-angle ordered 3D cones trajectory. All human images were collected with appropriate IRB permissions. The first dataset is a chest scan with regular respiratory motion and little bulk motion. The second dataset is a chest scan with several large bulk movements throughout the scan. The third dataset is an abdominal scan with regular respiratory motion. Signal intensity curves were plotted at voxels at the left and right ventricles, and aorta for the first two datasets, and at the cortex, medulla, and aorta for the third dataset.

On the first dataset, we performed reconstructions with LR, LLR, and MSLR of 500 frames to evaluate how different LR model affects the reconstruction quality. Matrix ranks for LR and LLR were chosen so that they have similar number of parameters as MSLR. For all datasets, we compared the proposed reconstruction method of 500 frames to soft-gated dynamic reconstructions of 20 frames. The soft-gated dynamic reconstruction was similar to Zhang et al⁵ except with a MSLR regularization instead of LLR. Soft-gating weights were computed from the respiratory motion signal mentioned above with a threshold chosen from the 10th percentile of the signal, and a decay parameter of 1.

The first DCE dataset was acquired with a 16-channel coil array, scan time of 4 minutes 40 seconds, TE = 0.1 ms, TR = 5.8 ms, flip angle = 14°, and bandwidth = 125 kHz. The number of readout points was 624, and the number of interleaves was 48129. The spatial resolution was reconstructed at 1 × 1 × 2.8 mm³, and the matrix size was 323 × 186 × 332. The targeted reconstruction temporal resolution for the proposed method was 580 ms, and for the soft-gated reconstruction was 14.5 seconds.

The second DCE dataset was acquired with a 12-channel coil array, scan time of 5 minutes 11 seconds, TE = 0.1 ms, TR = 7.4 ms, flip angle 15°, bandwidth = 125 kHz. The number of readout points was 711, and the number of interleaves was 41861. The spatial resolution was reconstructed at 1 × 1 × 1.8 mm³, and the matrix size was 360 × 156 × 126. The targeted reconstruction temporal resolution for the proposed method was 614 ms, and for the soft-gated reconstruction was 15.49 seconds.

The third DCE dataset was acquired with a 32-channel coil array, scan time of 3 minutes 53 seconds, TE = 0.1 ms, TR = 8.3 ms, flip angle 14°, bandwidth = 125 kHz. The number of readout points was 1007 and the number of interleaves was 28083. The spatial resolution was reconstructed at 1 × 1 × 1.8 mm³, and the matrix size was 438 × 276 × 138. The targeted reconstruction temporal resolution for the proposed method was 467 ms, and for the soft-gated reconstruction was 11.68 seconds.

3.2 |. Pulmonary imaging with bit-reversed ordered 3D UTE radial trajectory

We applied the proposed reconstruction on two lung datasets from adult patients to qualitatively evaluate its performance. All datasets were acquired on GE 3T Discovery MR750 clinical scanners (GE Healthcare, Waukesha, WI) using an optimized UTE sequence¹ and a bit-reversed ordered 3D radial trajectory. The first dataset has regular respiratory motion and little bulk motion. The second dataset has abrupt bulk motions such as coughing throughout the scan.

We compared the proposed method with respiratory-resolved reconstruction to five motion states. The proposed technique was reconstructed with 500 frames, whereas for the third lung dataset. The respiratory-resolved reconstruction was performed with total variation regularization along the motion states, which is similar to Jiang et al² and Feng et al.⁵⁴ To exclude data corrupted by bulk motion, only k-space data with the respiratory signal between the 10th and 90th percentiles were used. They were then sorted into five equally sized bins, each representing a motion state.

More specifically, the first lung dataset was acquired with an eight-channel coil array, an overall scan time of 4 minutes 40 seconds, TE = 80 μs, TR = 2.81 ms, flip angle = 4°, and sampling bandwidth = 250 kHz. The number of readout points was 523, and the number of interleaves was 99680. The targeted reconstruction spatial resolution was 1.25 × 1.25 × 1.25 mm³, and the matrix size was 500 × 250 × 241. The targeted reconstruction temporal resolution was 560 ms.

The second lung dataset was acquired with an eight-channel coil array, an overall scan time of 4 minutes 18 seconds, TE = 80 μs, TR = 3.48 ms, flip angle 4°, and sampling bandwidth = 250 kHz. The number of readout points was 654, and the number of interleaves was 75768. The targeted reconstruction spatial resolution was 1.25 × 1.25 × 1.25 mm³, and the matrix size was 408 × 183 × 379. The targeted reconstruction temporal resolution was 515 ms.

4 |. RESULTS

4.1 |. DCE with Golden-angle Ordered 3D Cones Trajectory

4.1.1 |. First DCE dataset

Figure 2 and Supporting Information Videos S2–S4 show reconstruction results with LR, LLR, and MSLR. As pointed by the red arrows, reconstruction with LR shows unrealistic dynamics, with contrast enhancing in both left and right ventricles at the same time. The other reconstructions show contrast enhancing first in the right ventricle then the left ventricle, which is physiologically correct. On the other hand, LLR displays more flickering temporal artifacts, which can be seen more visibly from Supporting Information Video S3. These artifacts are also pointed out by the orange arrows in figure. MSLR achieves a balance between representing contrast enhancement dynamics and reducing artifacts.

FIGURE 2.

Reconstructions with LR, LLR, and MSLR on the first DCE dataset. Dynamics can be seen more clearly in Supporting Information Videos S2–S4. As pointed by the red arrows, reconstruction with LR shows unrealistic dynamics, with contrast enhancing in both left and right ventricles at the same time. The other reconstructions show contrast enhancing first in the right ventricle then the left ventricle, which is physiologically correct. On the other hand, LLR displays more flickering temporal artifacts, which can be seen more visibly from Supporting Information Video S3. These artifacts are also pointed out by the orange arrows in figure. MSLR achieves a balance between representing contrast enhancement dynamics and reducing artifacts

Figure 3 and Supporting Information Video S5 show the corresponding MSLR decomposition. The scale with the 128³-sized blocks mostly shows static background tissues. The scale with the 64³-sized blocks depicts mostly contrast enhancements in the heart and aorta, and respiratory motion. The scale with 32³-sized blocks displays spatially localized dynamics, such as those in the right ventricle, and oscillates more over time than other scales.

FIGURE 3.

MSLR decomposition of the first DCE dataset. Dynamics can be seen more clearly in Supporting Information Video S5. The scale with the 128³-sized blocks mostly shows static background tissues. The scale with the 64³-sized blocks depicts mostly contrast enhancements in the heart and aorta, and respiratory motion. The scale with 32³-sized blocks displays spatially localized dynamics, such as those in the right ventricle, and oscillates more over time than other scales

Figure 4 and Supporting Information Videos S4 and S6 compare the proposed method with the soft-gated reconstruction. Regular respiratory motion can be observed in the proposed reconstruction in Supporting Information Video S4. Slight movements of the subject’s left arm can be seen from the video right before the contrast enhancements and toward the end of the scan. This can also be visualized in the 3D rendered Supporting Information Video S1. Contrast enhancements can be seen starting from the right ventricle, to the lung, then the left ventricle, and to the aorta, whereas the soft-gated reconstruction merges together the temporal changes of the lung, the left ventricle, and the aorta. The signal intensity plots for the proposed reconstruction also show much higher peaks when contrast is injected, which is physiologically accurate. In comparison, the soft-gated reconstruction signal curves appear to be smoothed.

FIGURE 4.

Comparison of the proposed method with the soft-gated reconstruction of the first DCE dataset. Dynamics can be seen more clearly in Supporting Information Videos S4 and S6. Regular respiratory motion can be observed in the proposed reconstruction in Supporting Information Video S4. Contrast enhancements can be seen starting from the right ventricle, to the lung, then the left ventricle, and to the aorta, whereas the soft-gated reconstruction merges together the temporal changes of the lung, the left ventricle, and the aorta. The signal intensity plots for the proposed reconstruction also show much higher peaks when contrast is injected, which is physiologically accurate. In comparison, the soft-gated reconstruction signal curves appear to be smoothed

An instance of the proposed method takes about 14 hours and the soft-gated reconstruction takes about 40 minutes. The resulting image using the MSLR representation takes 1.7 GBs to store.

4.1.2 |. Second DCE dataset

Convergence plots for the second DCE dataset reconstructed with 20 frames are shown in Figure 5A,B. Each marker indicates a pass through the entire k-space dataset. Figure 5A compares SGD with one GPU to GD. After 60 passes of k-space data, SGD is mostly converged, whereas GD only attains the objective value of about the fourth pass for SGD. Figure 5B compares the convergence of SGD with multiple GPUs. The convergence rates are similar in terms of number of passes. As for overall computation time, there is a small overhead going from one GPU to two GPUs, resulting in a 1.7× speedup instead of the ideal 2× speedup. We conjecture that this is due to data transferring between GPUs. From two to four GPUs, there is not much additional overhead, resulting in a speedup of almost 2×. The overall speedup from one to four GPUs is about 3.4.

FIGURE 5.

Convergence plots for the second DCE dataset reconstructed with 20 frames. Each marker indicates a pass over the entire k-space dataset. A, Comparison between SGD with one GPU to GD. After 60 passes of k-space data, SGD is mostly converged, whereas GD only attains the objective value of about the fourth pass for SGD. B, Comparison of SGD convergence rates with multiple GPUs. The convergence rates for different GPUs are similar. The convergence rates are similar in terms of number of passes. As for overall computation time, there is a small overhead going from one GPU to two GPUs, resulting in a 1.7× speedup instead of the ideal 2× speedup. We conjecture that this is due to data transferring between GPUs. From two to four GPUs, there is not much additional overhead, resulting in a speedup of almost 2×. The overall speedup from one to four GPUs is about 3.4

Figure 6 and Supporting Information Videos S7 and S8 compare the proposed method with the soft-gated reconstruction. From Supporting Information Video S7, the proposed reconstruction shows regular respiratory motion in the beginning, but after contrast injection, breathing becomes more rapid and the patient body shifts to the right seven times. While the image quality during these bulk movements degrades, it improves as soon as the patient body returns to the original position. Similar to the first DCE dataset, distinct phases of contrast enhancement to different organs can be seen, whereas the soft-gated reconstruction merges all dynamics into one frame, including the bulk motion. From the signal intensity curves in Figure 7, the peak contrast enhancements are higher for the proposed reconstruction than for the soft-gated reconstruction. Dips in the signal intensity curves in later parts of the scan can be seen for the proposed reconstruction, which corresponds to when bulk motions occur. A motion adjusted plot was created by manually tracking the voxels over time. Variations due to bulk motions in the signal intensity plots are mostly removed after motion adjustment.

FIGURE 6.

Comparison of the proposed method with the soft-gated reconstruction of the second DCE dataset. Dynamics can be seen more clearly in Supporting Information Videos S7 and S8. From Supporting Information Video S7, the proposed reconstruction shows regular respiratory motion in the beginning, but after contrast injection, breathing becomes more rapid and the patient body shifts to the right seven times. While the image quality during these bulk movements degrades, it improves as soon as the patient body returns to the original position. Similar to the first DCE dataset, distinct phases of contrast enhancement to different organs can be seen, whereas the soft-gated reconstruction merges all dynamics into one frame, including the bulk motion

FIGURE 7.

Signal intensity curves for the proposed method and the soft-gated reconstruction of the second DCE dataset. From the signal intensity curves, the peak contrast enhancements are higher for the proposed reconstruction than for the soft-gated reconstruction. Dips in the signal intensity curves in later parts of the scan can be seen for the proposed reconstruction, which corresponds to when bulk motions occur. A motion adjusted plot was created by manually tracking the voxels over time. Variations due to bulk motions in the signal intensity plots are mostly removed after motion adjustment

An instance of the proposed method takes about 6 hours and the soft-gated reconstruction takes about 30 minutes. The resulting image using the MSLR representation takes 1 GB to store.

4.1.3 |. Third DCE dataset

Figure 8 and Supporting Information Videos S9 and S10 compare the proposed method with the soft-gated reconstruction. From Supporting Information Video S9, regular breathing motion can be seen in the proposed reconstruction. Contrast dynamics starting from the aorta, and slowly filling in the cortex and the medulla can be observed. The aortic temporal changes can be seen at a higher frame rate for the proposed reconstruction than for the soft-gated reconstruction.

FIGURE 8.

Comparison of the proposed method with the soft-gated reconstruction of the third DCE dataset. Dynamics can be seen more clearly in Supporting Information Videos S9 and S10. From Supporting Information Video S9, regular breathing motion can be seen in the proposed reconstruction. Contrast dynamics starting from the aorta, and slowly filling in the cortex and the medulla can be observed. The aortic temporal changes can be seen at a higher frame rate for the proposed reconstruction than for the soft-gated reconstruction

An instance of the proposed method takes about 42 hours and the soft-gated reconstruction takes about 2 hours. The resulting image using the MSLR representation takes 2.5 GBs to store.

4.2|. Pulmonary imaging with bit-reversed ordered 3D UTE radial trajectory

4.2.1 |. First lung dataset

Figure 9 and Supporting Information Videos S11 and S12 compare the proposed method with the respiratory-resolved reconstruction. From the cross section over time and Supporting Information Video S11, regular breathing with slight variable rates can be observed. Overall, Supporting Information Video S11 of the proposed reconstruction shows some temporal flickering artifacts. Looking at each frame individually, the proposed reconstruction shows similar image quality and sharpness as the respiratory-resolved reconstruction for the expiration phase. For other phases, the respiratory-resolved reconstruction is slightly sharper near the diaphragms.

FIGURE 9.

Comparison of the proposed method with the respiratory-resolved reconstruction of the first lung dataset. Dynamics can be seen more clearly in Supporting Information Videos S11 and S12. From the cross section over time and Supporting Information Video S11, regular breathing with slight variable rates can be observed. Overall, Supporting Information Video S11 of the proposed reconstruction shows temporal flickering artifacts. Looking at each frame individually, the proposed reconstruction shows similar image quality and sharpness as the respiratory-resolved reconstruction for the expiration phase. For other phases, the respiratory-resolved reconstruction is slightly sharper near the diaphragms

An instance of the proposed method takes about 45 hours and the respiratory-resolved reconstruction takes about an hour. The resulting image using the MSLR representation takes 4.4 GB to store.

4.2.2 |. Second lung dataset

Figure 10 and Supporting Information Videos S13 and S14 compare the proposed method with the respiratory-resolved reconstruction. From the cross section and Supporting Information Video S13, coughing can be observed in the beginning of the scan for the proposed reconstruction. The patient can be seen to return to a more regular breathing pattern after a while but still occasionally show abrupt motions. The proposed reconstruction does show flickering temporal artifacts when the patient coughs, but in general has less noise-like artifacts and much sharper features than the respiratory-resolved reconstruction.

FIGURE 10.

Comparison of the proposed method with the respiratory-resolved reconstruction of the second lung dataset. Dynamics can be seen more clearly in Supporting Information Videos S13 and S14. From the cross section and Supporting Information Video S13, coughing can be observed in the beginning of the scan for the proposed reconstruction. The patient can be seen to return to a more regular breathing pattern after a while but still occasionally show abrupt motions. The proposed reconstruction does show severe flickering temporal artifacts when the patient coughs, but in general has less noise-like artifacts and much sharper features than the respiratory-resolved reconstruction

An instance of the proposed method takes about 11.5 hours and the respiratory-resolved reconstruction takes about 15 minutes. The resulting image using the MSLR representation takes 2.8 GBs to store.

5 |. DISCUSSION

We have developed a method to reconstruct large-scale volumetric dynamic image sequences from rapid, continuous, and non-gated acquisitions. The reconstruction problem considered is vastly underdetermined, and computationally and memory demanding. Using MSLR, the proposed technique can greatly compress dynamic image sequence on the order of a 100 GB to only a few GBs. Using the Burer-Monteiro heuristic, the reconstruction can directly optimize for the compressed representation. Finally incorporating stochastic optimization, the runtime for the resulting method is drastically reduced.

In Figure 2, we compared the effect of different LR models. Consistent with observations in,^5,34 LR has difficulty representing spatially localized dynamics, and exhibits temporal blurring of contrast enhancements in the ventricles. LLR is able to depict spatially localized dynamics well and show distinct contrast enhancements in the left and right ventricles. However, compared to MSLR, LLR has more spurious temporal artifacts. Because LR and LLR are subsets of MSLR, MSLR in principle can always perform better with the suitably tuned regularization parameters. The purpose of the experiment is to show that with a fixed regularization relationship between scales described in Equation 6, MSLR can still represent dynamics with the appropriate scale. This is also supported qualitatively in the MSLR decomposition shown in Figure 3.

The benefit of using SGD compared to conventional iterative algorithms can be seen in Figure 5. For GD, each pass of the k-space data only computes one iteration, whereas for SGD, each pass computes TC many iterations. Even though each SGD iteration makes fewer progress than GD, SGD reaches an approximate minimum faster by performing many more iterations. In our experiment with 20 frames, we observe that GD is approximately 15× slower, and expect this difference to be even greater with 500 frames. However, the current SGD reconstruction time is still long, ranging from 6 to 42 hours, depending on the number of coils and the reconstruction matrix size. We remain hopeful that with additional computing devices, either with large number of GPUs or cloud computing, reconstruction time can be brought to a reasonable range. This is partly supported by Figure 5B, which shows that the speedup of using multiple GPUs is almost linear from one to four GPUs.

Figure 4, 6, and 8 all show that the proposed reconstruction displays much finer dynamics that are not represented in soft-gated reconstructions with low frame rates. Distinct phases of contrast enhancements in different organs can be seen, which are more physiologically accurate. The benefits of higher temporal resolution can also be seen from signal intensity curves. In particular, signal intensity peaks of the aorta are much higher in the proposed reconstruction, but are averaged out in the soft-gated reconstruction. While bulk motion still affects the overall image quality as shown in Figure 6, the proposed reconstruction allows us to retrospectively adjust for bulk motion when computing the signal intensity curves, which can be useful for quantification purposes.

Figure 10 shows that for pulmonary imaging, the proposed reconstruction really shines when there are non-periodic motions. Because Figure 9 mostly consists regular breathing, the proposed reconstruction does not offer much beyond unrolling the periodic dynamics. However, for the second dataset with irregular breathing, the proposed reconstruction provides substantially improved image quality, and also depicts the irregularities. In particular, coughing can be seen from the beginning of the dynamic image series.

While the proposed method does show transient dynamics that are not seen in gating-based reconstructions, flickering temporal artifacts can often be seen in reconstructions. We believe the main cause for the flickering artifacts is that block low-rank models are inherently sensitive to local fluctuations. Each block is almost independent with others, and small blocks are more easily fitted to aliasing artifacts and noise. This explains why LLR has flickering and LR does not. Ideally, MSLR would suppress the spurious blocks and use larger blocks instead. However, in the limited memory setting, we can only use three scales, and the model is forced to use smaller blocks for representation. This is why for larger datasets (the third DCE dataset and all lung datasets), we used larger block sizes for reconstruction. If more memory is available, we would do more scales and let the reconstruction choose the appropriate blocks.

Besides the flickering artifacts, the proposed technique also shows degraded image quality and temporal blurring for large bulk motions, such as those in Supporting Information Video S7. One reason is that MSLR, or in general LR representations, does not explicitly model for motions. Hence, for large displacements, the resulting dynamics may not be low rank even for small block sizes. While there are existing works⁵⁵ on incorporating motion information into LR modeling, it is still unclear how to do it in a computation and memory-friendly way for the extreme MRI setting.

We would also like to emphasize that the actual temporal resolution of our reconstruction is lower than the prescribed temporal resolution. This is especially true with the finite-difference temporal penalty applied on the temporal bases. Since we do not have access to the ground truths, we are unable to quantify the actual spatiotemporal resolution. A potential future direction is to create a realistic large-scale digital phantom to help quantify the actual resolution.

In terms of clinical applications, the proposed method is suitable when volumetric information with high temporal resolution is desired. In particular, it could potentially work well in the lower abdomen where conventional techniques are often affected by bowel motion. Also, functional imaging approaches that require large volume coverage, such as imaging of tracheomalacia, could benefit from the technique. Another potential application is MR spirometry⁵⁶ for irregular breathing, in which regional lung deformations over times are used to estimate pulmonary function. If the reconstruction time can be made short enough with more advanced hardware, the proposed method can also be useful for interventional MRI.

Finally, we would like to mention several theoretical results regarding the Burer-Monteiro factorization and point to potential directions to bridge the gaps between their assumptions and our setting. Existing works^45,57 have shown that any second-order stationary point (ie, solutions with zero gradient and positive semi-definite Hessian) of the low-rank factorized non-convex problem is equivalent to the global minimum of the convex problem using nuclear norm, as long as the solution matrix rank is smaller than what is prescribed. Recent theoretical results⁵⁸ have further shown that under idealized incoherence conditions (such as the restricted isometry property), any second-order stationary point of the non-convex formulation is also a global minimum. In order to generalize these results to our setup, it is crucial to extend inherent conditions to multichannel forward models. In addition, it remains to be shown that 3D radial and 3D cones trajectories with pseudo-randomized orderings satisfy these theoretical assumptions.

6 |. CONCLUSION

We demonstrated a method that can reconstruct massive 3D dynamic image series in the extreme undersampling and extreme computation setting. The proposed technique shows transient dynamics that are not seen in gating-based methods. When applied to datasets with irregular, or non-repetitive motions, the proposed method also displays sharper image features.

Supplementary Material

VIDEO S1 3D rendering of the proposed reconstruction of the first DCE dataset. The proposed method can recon- struct large-scale volumetric dynamic MRI, which enables high-quality reformatting.

VIDEO S1 3D rendering of the proposed reconstruction of the first DCE dataset. The proposed method can reconstruct large-scale volumetric dynamic MRI, which enables high-quality reformatting as shown

VIDEO S2 The proposed reconstruction with LR modeling of the first DCE dataset. LR shows contrast flowing into left and right ventricles at the same time, which is not physiolog- ical correct

VIDEO S2 The proposed reconstruction with LR modeling of the first DCE dataset. LR shows contrast flowing into left and right ventricles at the same time, which is not physiological correct

VIDEO S3 The proposed reconstruction with LLR modeling of the first DCE dataset. LLR shows contrast enhancing first in the right ventricle then the left ventricle, which is more physiological accurate, but displays more flickering temporal artifacts

VIDEO S3 The proposed reconstruction with LLR modeling of the first DCE dataset. LLR shows contrast enhancing first in the right ventricle then the left ventricle, which is more physiological accurate, but displays more flickering temporal artifacts

VIDEO S4 The proposed reconstruction with MSLR mod- eling of the first DCE dataset. MSLR achieves a balance be- tween representing contrast enhancement dynamics and re- ducing artifacts. Contrast enhancements can be seen starting from the right ventricle, to the lung, then the left ventricle, and to the aorta

VIDEO S4 The proposed reconstruction with MSLR modeling of the first DCE dataset. MSLR achieves a balance between representing contrast enhancement dynamics and reducing artifacts. Contrast enhancements can be seen starting from the right ventricle, to the lung, then the left ventricle, and to the aorta

VIDEO S7 The proposed reconstruction of the second DCE dataset. The proposed reconstruction shows regular respi- ratory motion in the beginning, but after contrast injection, breathing becomes more rapid and the patient body shifts to the right seven times. While the image quality during these bulk movements degrades, it improves as soon as the patient body returns to the original position

VIDEO S7 The proposed reconstruction of the second DCE dataset. The proposed reconstruction shows regular respiratory motion in the beginning, but after contrast injection, breathing becomes more rapid and the patient body shifts to the right seven times. While the image quality during these bulk movements degrades, it improves as soon as the patient body returns to the original position

VIDEO S5 MSLR decomposition of the first DCE dataset. The scale with the 1283-sized blocks mostly shows static background tissues. The scale with the 643-sized blocks de- picts mostly contrast enhancements in the heart and aorta, and respiratory motion. The scale with 323-sized blocks displays spatially localized dynamics, such as those in the right ventricle, and oscillates more over time than other scales

VIDEO S5 MSLR decomposition of the first DCE dataset. The scale with the 128³-sized blocks mostly shows static background tissues. The scale with the 64³-sized blocks depicts mostly contrast enhancements in the heart and aorta, and respiratory motion. The scale with 32³-sized blocks displays spatially localized dynamics, such as those in the right ventricle, and oscillates more over time than other scales

VIDEO S8 Soft-gated reconstruction of the second DCE dataset. The soft-gated reconstruction merges all dynamics during contrast injection into one frame, including the bulk motion

VIDEO S8 Soft-gated reconstruction of the second DCE dataset. The soft-gated reconstruction merges all dynamics during contrast injection into one frame, including the bulk motion

VIDEO S10 Soft-gated reconstruction of the third DCE data- set. Contrast dynamics starting from the aorta, and slowly filling in the cortex and the medulla can be observed

VIDEO S10 Soft-gated reconstruction of the third DCE dataset. Contrast dynamics starting from the aorta, and slowly filling in the cortex and the medulla can be observed

VIDEO S9 The proposed reconstruction of the third DCE dataset. Regular breathing motion can be seen in the pro- posed reconstruction. Contrast dynamics starting from the aorta, and slowly filling in the cortex and the medulla can be observed at a high frame rate

VIDEO S9 The proposed reconstruction of the third DCE dataset. Regular breathing motion can be seen in the proposed reconstruction. Contrast dynamics starting from the aorta, and slowly filling in the cortex and the medulla can be observed at a high frame rate

VIDEO S14 Respiratory-resolved reconstruction of the sec- ond lung dataset. The respiratory-resolved reconstruction in general more noise-like artifacts and much blurrier features than the proposed method

VIDEO S14 Respiratory-resolved reconstruction of the second lung dataset. The respiratory-resolved reconstruction in general more noise-like artifacts and much blurrier features than the proposed method

FIGURE S1 Effects of different regularization parameters on the second DCE dataset. Too large of a regularization parameter (λ = 10⁻³) blurs both the images and the temporal dynamics. Too small of a regularization parameter (λ = 10⁻⁵) results in noise-like artifacts. λ = 10⁻⁴ provides a balance between noise reduction and preserving dynamics

FIGURE S1 Effects of different regularization parameters on the second DCE dataset. Too large of a regularization parameter (λ = 10⁻³) blurs both the images and the temporal dynamics. Too small of a regularization parameter (λ = 10⁻⁵) results in noise-like artifacts. λ = 10⁻⁴ provides a balance between noise reduction and preserving dynamics

VIDEO S13 The proposed reconstruction of the second lung dataset. Coughing can be observed in the beginning of the scan for the proposed reconstruction. The patient can be seen to return to a more regular breathing pattern after a while but still occasionally show abrupt motions. The proposed re- construction does show severe flickering temporal artifacts during the patient coughing, but in general has less noise- like artifacts and much sharper features than the respiratory- resolved reconstruction

VIDEO S13 The proposed reconstruction of the second lung dataset. Coughing can be observed in the beginning of the scan for the proposed reconstruction. The patient can be seen to return to a more regular breathing pattern after a while but still occasionally show abrupt motions. The proposed reconstruction does show severe flickering temporal artifacts during the patient coughing, but in general has less noise-like artifacts and much sharper features than the respiratory-resolved reconstruction

VIDEO S12 Respiratory-resolved reconstruction of the first lung dataset. The respiratory-resolved reconstruction is slightly sharper near the diaphragms in most phases than the proposed reconstruction

VIDEO S12 Respiratory-resolved reconstruction of the first lung dataset. The respiratory-resolved reconstruction is slightly sharper near the diaphragms in most phases than the proposed reconstruction

VIDEO S6 Soft-gated reconstruction of the first DCE data- set. The soft-gated reconstruction merges together the tempo- ral changes of the lung, the left ventricle and the aorta

VIDEO S6 Soft-gated reconstruction of the first DCE dataset. The soft-gated reconstruction merges together the temporal changes of the lung, the left ventricle and the aorta

VIDEO S11 The proposed reconstruction of the first lung dataset. Regular breathing with slight variable rates can be observed. Overall, the proposed reconstruction shows tempo- ral flickering artifacts

VIDEO S11 The proposed reconstruction of the first lung dataset. Regular breathing with slight variable rates can be observed. Overall, the proposed reconstruction shows temporal flickering artifacts

DATA AVAILABILITY STATEMENT

Code that reproduces the results of this study is openly available as a Python package which can be downloaded from https://github.com/mikgroup/extreme_mri.git.