Nonrigid 3D motion estimation at high temporal resolution from prospectively undersampled k‐space data using low‐rank MR‐MOTUS

Abstract

Purpose

With the recent introduction of the MR‐LINAC, an MR‐scanner combined with a radiotherapy LINAC, MR‐based motion estimation has become of increasing interest to (retrospectively) characterize tumor and organs‐at‐risk motion during radiotherapy. To this extent, we introduce low‐rank MR‐MOTUS, a framework to retrospectively reconstruct time‐resolved nonrigid 3D+t motion fields from a single low‐resolution reference image and prospectively undersampled k‐space data acquired during motion.

Theory

Low‐rank MR‐MOTUS exploits spatiotemporal correlations in internal body motion with a low‐rank motion model, and inverts a signal model that relates motion fields directly to a reference image and k‐space data. The low‐rank model reduces the degrees‐of‐freedom, memory consumption, and reconstruction times by assuming a factorization of space‐time motion fields in spatial and temporal components.

Methods

Low‐rank MR‐MOTUS was employed to estimate motion in 2D/3D abdominothoracic scans and 3D head scans. Data were acquired using golden‐ratio radial readouts. Reconstructed 2D and 3D respiratory motion fields were, respectively, validated against time‐resolved and respiratory‐resolved image reconstructions, and the head motion against static image reconstructions from fully sampled data acquired right before and right after the motion.

Results

Results show that 2D+t respiratory motion can be estimated retrospectively at 40.8 motion fields per second, 3D+t respiratory motion at 7.6 motion fields per second and 3D+t head‐neck motion at 9.3 motion fields per second. The validations show good consistency with image reconstructions.

Conclusions

The proposed framework can estimate time‐resolved nonrigid 3D motion fields, which allows to characterize drifts and intra and inter‐cycle patterns in breathing motion during radiotherapy, and could form the basis for real‐time MR‐guided radiotherapy.

1. INTRODUCTION

Uncertainty in tumor and organs‐at‐risk locations due to unknown respiratory‐induced organ motion diminishes the efficacy of radiotherapy in the abdomen and thorax in two ways. First, tumors are irradiated with larger treatment margins, which results in increased radiation dose and toxicity to healthy tissue. Second, it prevents an accurate (retrospective) estimation of the actual dose accumulated in the targeted tumor and healthy surrounding tissue during the treatment.

Recently, the MR‐LINAC was introduced as the combination of an MR‐scanner and a linear accelerator (LINAC) in a single device, ¹ , ² , ³ , ⁴ which has the potential to address both points above. Achieving this goal, however, poses the following technical challenge: real‐time reconstructions at 5 Hz ⁵ , ⁶ of internal body motion during the treatments. A fundamental step toward real‐time reconstructions is the retrospective estimation of time‐resolved motion fields. Additionally, these retrospectively reconstructed motion fields are valuable for the calculation of accumulated dose and can be taken into account for more accurate radiation planning of subsequent treatments. To this extent, we focus on the retrospective reconstruction of time‐resolved 3D+t respiratory motion with a temporal resolution of 5 motion fields per second. We envision that this framework could eventually be adapted to prospective real‐time reconstructions. ⁷

In MR‐guided radiotherapy, tumor and organs‐at‐risk motion is typically estimated from cine‐MR‐images followed by image registration. For time‐resolved motion estimation, these cine‐MR‐images would thus require sufficient temporal resolution and spatial coverage to resolve the targeted motion. This is in general achievable in 2D, and also in 3D for slowly moving targets such as pelvic tumors. ⁸ However, in 3D, it is more challenging for faster moving targets like lung tumors, that require at least 5 motion fields per second. ⁵ , ⁶

Several strategies have previously been proposed to extract tumor and organ‐at‐risk motion from MR‐images, three of which will be reviewed below. With the first strategy, average respiratory motion is estimated from a respiratory‐resolved 3D+t MRI. This approach retrospectively sorts image slices or k‐space readouts in 3D acquisitions according to their respective respiratory phases, extracted using a respiratory motion surrogate (eg, pneumatic belt, self‐navigation signal or navigator). Examples include the works in Refs. [9, 10, 11, 12, 13] (see Ref. [14] for a more complete overview). Although the retrospective sorting in these methods allows for efficient use of all acquired data, it makes strong assumptions on the periodicity of respiratory motion and characterizes only average 3D+t breathing motion. Although this is useful to reduce treatment margins, it may not be sufficient for accurate accumulation of the delivered dose.

A different strategy uses multislice/orthogonal 2D+t cine‐MRI for 3D+t motion estimation. ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² The reduction in the spatial dimension allows for higher temporal resolution, and is combined with a model that links the lower dimensional image data to 3D motion fields. This strategy assumes, however, that a good fit on lower dimensional images implies a good fit in the full 3D domain. Although this is reasonable for small volumes, since slices cover a large fraction of the volume in such a case, it may be less valid for larger volumes which may be required for dose accumulation.

The third strategy does not rely on sorting, but reconstructs images from highly undersampled k‐space data. Even with parallel imaging, ²³ , ²⁴ this typically eventually results in lower SNR, lower spatial resolution, and/or undersampling artifacts. Nevertheless, it has been shown that motion fields can be estimated from these images with sufficient accuracy. ²⁵ , ²⁶ , ²⁷ , ²⁸ Additionally, iterative reconstructions based on compressed sensing ²⁹ have been proposed to exploit the spatiotemporal sparsity of images. However, for the intended application the reported temporal resolution was too low, ²⁸ , ³⁰ , ³¹ or the FOV was too small. ³² , ³³

Following a different approach, we have previously introduced MR‐MOTUS ³⁴ (Model‐based Reconstruction of MOTion from Undersampled Signal), a new framework that allows to reconstruct nonrigid 3D motion fields directly from k‐space data. The key ingredient of MR‐MOTUS is a signal model that explicitly relates dynamic k‐space data to the combination of a static reference image and dynamic motion fields. Assuming a reference image is available, and data are acquired in steady state, motion fields can be reconstructed directly from k‐space data by solving the corresponding nonlinear inverse problem. Since motion fields are spatially correlated and therefore compressible, few data are required for the reconstructions.

The possibility to reconstruct motion from few k‐space data makes MR‐MOTUS a natural candidate for time‐resolved 3D+t motion estimation, which is not directly restricted to the achievable temporal resolution in MR‐images. Our work presented in Ref. [34], however, represents a proof‐of‐concept, and demonstrates MR‐MOTUS in an experimental setting. Four points of improvement should be addressed for the extension of MR‐MOTUS to time‐resolved 3D+t motion estimation:

1. Only spatial correlation in motion fields was exploited, and a single static motion‐field was reconstructed for each single snapshot of k‐space data. Additionally exploiting temporal correlation, and jointly reconstructing the 3D+t motion‐field series at once, could improve the reconstruction quality and lower requirements of computing time and memory.

2. Only the body coil was used for data acquisition to obtain homogeneous coil sensitivity. Since typically multicoil acquisitions are favored, this did not represent a practical setting.

3. The required reference image was obtained from a separate MR‐scan during breath‐hold. Ideally, no breath‐holds are required and reconstructions can be performed on data acquired in free‐breathing conditions.

4. 3D motion fields were previously reconstructed from retrospectively undersampled Cartesian k‐space data, while the motion estimation application requires prospectively undersampled acquisitions with an efficient non‐Cartesian trajectory.

In this work, we address the aforementioned points of improvement and extend the framework to experiments in a realistic setting, in which reference image and time‐resolved 3D+t motion fields can be reconstructed from linearly combined multi‐coil, free‐breathing, prospectively undersampled non‐Cartesian 3D k‐space data. Time‐resolved 3D+t motion‐field reconstructions require the representation of 3D motion fields over a large number of timepoints (>100), and thereby introduce a large number unknowns. We propose to use a spatiotemporal low‐rank motion model to compress the representation of 3D+t motion fields. Several works have previously proposed low‐rank motion models for motion estimation, ¹⁹ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ and the analyses in Refs. [19, 35, 36] suggest that a rank‐2 motion model can accurately describe respiratory motion. Consequently, the low‐rank motion model can reduce the number of unknowns by two orders of magnitude, thereby introducing a regularization in both space and time and significantly reducing memory consumption and reconstruction times for 3D+t reconstructions. We will refer to the extended framework as low‐rank MR‐MOTUS.

We demonstrate and validate low‐rank MR‐MOTUS in a total of 6 in vivo experiments on 2 healthy subjects and several moving anatomies. 2D/3D abdominothoracic respiratory motion is included in view of the MR‐guided radiotherapy application, and 3D head‐and‐neck motion is included for additional validation and as a demonstration to handle different types of motion. The 2D respiratory motion reconstruction is validated against 2D time‐resolved compressed sensing, the 3D respiratory motion reconstruction against respiratory‐resolved 3D image reconstruction, and the 3D head‐and‐neck motion against 3D static images acquired right before and right after the motion.

2. THEORY

2.1. Background MR‐MOTUS

We assume a general d‐dimensional setting, with targeted case d = 3, and we follow the convention that bold‐faced characters denote vectorizations. We define ${\mathbf{x}}_0\mapsto {\mathbf{x}}_t$ as the mappings from coordinates ${\mathbf{x}}_0\in {\mathbb{R}}^d$ in a reference image to new locations ${\mathbf{x}}_t\in {\mathbb{R}}^d$ at time t. The mappings are characterized by the motion fields ${\mathbf{d}}_t$ through ${\mathbf{x}}_t={\mathbf{x}}_0+{\mathbf{d}}_t({\mathbf{x}}_0)$. This will be written in concatenated vector‐form as

$${\mathbf{X}}_t={\mathbf{X}}_0+{\mathbf{D}}_t,$$ (1)

where ${\mathbf{X}}_t,{\mathbf{X}}_0,{\mathbf{D}}_t\in {\mathbb{R}}^{Nd\times 1}$ denote the vertical concatenations over N spatial points in a d‐dimensional setup. The MR‐MOTUS forward model ³⁴ explicitly relates the motion fields ${\mathbf{D}}_t$ and a static reference image ${\mathbf{q}}_0\in {\mathbb{C}}^N$ to dynamic, single‐channel (and possibly non‐Cartesian) k‐space measurements ${\mathbf{s}}_t\in {\mathbb{C}}^{N_k}$:

$${\mathbf{s}}_t=\mathbf{F}({\mathbf{D}}_t|{\mathbf{q}}_0)+{\mathit{ϵ}}_t.$$ (2)

Here ${\mathit{ϵ}}_t\in {\mathbb{C}}^{N_k}$ is the complex noise vector and $\mathbf{F}:{\mathbb{R}}^{\mathit{Nd}}\mapsto {\mathbb{C}}^{N_k}$ is the vectorization of the forward operator defined as

$$F({\mathbf{d}}_t)[\mathbf{k}]={\int }_{\mathrm{\Omega }}{q}_0({\mathbf{x}}_0)e^{‐i2\pi \mathbf{k}·\left[{\mathbf{x}}_0+{\mathbf{d}}_t({\mathbf{x}}_0)\right]}\mathrm{d}{\mathbf{x}}_0,$$ (3)

where $\mathbf{k}\in {\mathbb{R}}^d$ denotes the k‐space coordinate. By fitting the nonlinear signal model in Equation (3) to acquired k‐space data, motion fields can be reconstructed directly from k‐space measurements.

2.2. Reconstruction problem formulation for space‐time reconstructions

In this work, we follow Ref. [40] and formulate the reconstruction problem for space‐time motion fields D as follows:

$$\underset{\mathbf{D}}{min}\sum _{t=1}^M{∥\mathbf{F}({\mathbf{D}}_t)‐{\mathbf{s}}_t∥}_2^2+{\lambda }_R\mathcal{R}(\mathbf{D}).$$ (4)

Here $\mathcal{R}(\mathbf{D})>0$ is a regularization functional, with corresponding parameter ${\lambda }_R>0$, which models a priori assumptions in order to exploit correlations in both space and time.

2.2.1. Parameterization with a low‐rank space‐time motion model

A straightforward parameterization of D considers one motion‐field per dynamic, ie, $\mathbf{D}=[{\mathbf{D}}_1,\cdots ,{\mathbf{D}}_M]\in {\mathbb{R}}^{Nd\times M}$. This is, however, impractical from a computational point‐of‐view, since the number of parameters scales with the number of dynamics: |D| = NMd ∼ M. For a typical scenario, $N\sim {10}^6,M\sim {10}^2$ and d = 3, in which case

$$|\mathbf{D}|\sim {10}^8.$$ (5)

Hence, this parameterization results in high memory consumption and long reconstruction times.

We observe that internal body motion is typically very rigid, and movement occurs along a similar angle over time. We therefore hypothesize that a representation of motion fields as the summation of motion directions per voxels with relative magnitude, multiplied by global scalings along these directions over time, can lead to an efficient representation with a reduced number of parameters. As an example, motion with fixed directions (and relative magnitude) per voxel can be represented with just one motion‐field and one global 1D scaling along these directions over time. This representation can mathematically be captured with a low‐rank motion model. The low‐rank model simultaneously reduces the number of parameters for the reconstruction and introduces a natural regularization in both space and time by enforcing the following factorization in spatial and temporal contributions:

$$\mathbf{D}=\left(\begin{array}{ccc}|& & |\\ {\mathbf{D}}_1& \cdots & {\mathbf{D}}_M\\ |& & |\end{array}\right)=\left(\begin{array}{ccc}|& & |\\ {\mathbf{\Phi }}^1& \cdots & {\mathbf{\Phi }}^R\\ |& & |\end{array}\right)\left(\begin{array}{ccc}‐& {\mathbf{\Psi }}^1& ‐\\ & ⋮& \\ ‐& {\mathbf{\Psi }}^R& ‐\end{array}\right)=\mathbf{\Phi }{\mathbf{\Psi }}^T.$$ (6)

Here R denotes the number of components of the model; $\mathbf{\Phi }\in {\mathbb{R}}^{Nd\times R}$ denotes the matrix with spatial components, and $\mathbf{\Psi }\in {\mathbb{R}}^{M\times R}$ denotes the matrix with temporal components. See also Figure 1 for a more intuitive explanation of the model. The model (6) will be referred to as the low‐rank model, since rank(D) ≤ R. The upper limit is achieved for R linearly independent components. A similar explicit low‐rank factorization was recently proposed in the context of image reconstructions in Ref. [30], with the same motivations as mentioned above.

FIGURE 1.

Overview of the low‐rank MR‐MOTUS framework. First, data are acquired during free‐breathing with a golden‐ratio radial trajectory (2D: golden‐angle radial, ⁴⁴ 3D: golden‐mean radial Koosh ball ⁴⁵ ). Then, DC‐based phase binning is performed on end‐inhale to reconstruct a motion‐free reference image. Finally, the reference image and free‐breathing data are fed into the low‐rank MR‐MOTUS reconstructions, resulting in time‐resolved 3D motion fields. The motion fields are reconstructed with an explicit constraint on the maximum rank. That is, as a sum of component motion fields with each a different temporal behavior. The number of such components is pre‐determined

The number of parameters in the low‐rank model is |D| = |Φ|+|Ψ| = (Nd+M)R. Analyses in Refs. [5, 19, 36] suggest that a motion model with rank 2 is sufficient to accurately model respiratory motion. For the typical scenario considered above ($N\sim {10}^6,M\sim {10}^2,d=3$), this would then imply

$$|\mathbf{D}|\sim {10}^6,$$ (7)

which is two orders of magnitude lower than Equation (5). In Supporting Information Section 1, analyses are performed to confirm that the low‐rank motion model with a small number of components R can indeed accurately represent motion fields.

We follow a standard approach in nonrigid medical image registration ⁴¹ and represent both the spatial components Φ and the temporal components Ψ of the motion fields in cubic B‐spline bases. This results in representation coefficients α, β for, respectively, Φ and Ψ.

2.2.2. Regularization functional

The motion‐field reconstruction problem in Equation (4) is typically ill‐posed, and requires incorporation of a priori knowledge of the motion fields. Since organs such as the liver, spleen and kidney consist of liquid filled tissue structures, they can be assumed incompressible and thus volume‐preserving under motion. ⁴² The Jacobian determinant $\mathcal{J}({\mathbf{D}}_t(\mathbf{r}))$ is the fraction of the volume at spatial coordinate r after deformation by ${\mathbf{D}}_t(\mathbf{r})$, with respect to the reference volume before deformation. Hence, values between 0 and 1 indicate shrinkage, values around 1 indicate no compression nor expansion, and values above 1 indicate expansion. We enforce the incompressibility assumption by penalizing deviations of the Jacobian determinant from unity ⁴³ :

$$\mathcal{R}(\mathbf{D}):=\sum _{t=1}^M{\Vert \mathbf{W}(\mathcal{J}({\mathbf{D}}_t)‐\mathbf{1})\Vert }_2^2.$$ (8)

Here $\mathcal{J}(·)$ computes the determinant of the Jacobian, and W is a diagonal matrix with weights per voxel. The weights are added to exclude regions where the regularization is less realistic, eg, in the lungs. As weights we have taken the magnitude of the reference image, scaled to unit norm. For the implementation, we follow Ref. [43] and compute spatial derivatives analytically using the spline parameterization of the motion fields.

2.2.3. Final reconstruction problem formulation

Substituting the spline representation, low‐rank model (6) and regularization (8) into the objective function (4) results in the following minimization problem to reconstruct space‐time motion fields:

$$\{{\mathit{\alpha }}^{†},{\mathit{\beta }}^{†}\}={\text{argmin}}_{\mathbf{\Phi }{\mathbf{\Psi }}^T=[{\mathbf{D}}_1,\cdots ,{\mathbf{D}}_M]}\sum _{t=1}^M{∥\mathbf{F}({\mathbf{D}}_t)‐{\mathbf{s}}_t∥}_2^2+{\lambda }_R\sum _{t=1}^M{\Vert \mathbf{W}(\mathcal{J}({\mathbf{D}}_t)‐\mathbf{1})\Vert }_2^2,$$ (9)

where ${\lambda }_R\in {\mathbb{R}}^{+}$ is the regularization parameter that balances the terms. Note that no temporal regularization is added, since the low‐rank model already acts as a strong regularization in both space and time.

3. METHODS

The following data were acquired in three different experiments per volunteer for two volunteers:

1. 2D+t abdominothoracic data;

2. 3D+t abdominothoracic data;

3. 3D+t head‐and‐neck data.

The 2D+t abdominothoracic data allow for a validation against time‐resolved image reconstruction at a high temporal resolution. The 3D+t abdominothoracic data are the targeted case for the application in MR‐guided radiotherapy. The 3D+t head‐and‐neck data are included as a demonstration to handle different types of motion, and for additional validation. All reconstructions are analyzed by comparison with image reconstructions on the same data. As an additional sanity check, the Jacobian determinant of the transformation corresponding to the motion fields is analyzed: ${\mathbf{x}}_0\mapsto {\mathbf{x}}_0+{\mathbf{d}}_t({\mathbf{x}}_0)$. More details regarding the experiments are provided below, organized per subsection.

All data were acquired on a 1.5T MRI scanner (Ingenia, Philips Healthcare, Best, The Netherlands) using a steady‐state spoiled gradient echo sequence (SPGR) with anterior and posterior receive arrays. We employed golden‐angle radial readouts for 2D, ⁴⁴ and golden‐mean Koosh ball radial readouts for 3D. ⁴⁵ The volunteers provided written informed consent prior to the scans, and all scans were approved by the institutional review board of the University Medical Center Utrecht and carried out in accordance with the relevant guidelines and regulations. See Table 1 for all relevant acquisition parameters.

TABLE 1.

Details of the in vivo experiments as described in Sections 3.1‐3.3: the top half lists acquisition details, and the bottom half lists reconstruction details for the time‐resolved experiments

Parameter	2D resp. motion	3D resp. motion	3D head‐and‐neck motion
FOV (m)	0.50 × 0.50 × 0.01	0.44 × 0.44 × 0.44	0.38 × 0.38 × 0.38
Acquisition matrix size	164 × 164 × 1	146 × 146 × 146	126 × 126 × 126
Number of samples on readout	264	264	232
Spatial acq. resolution (mm)	3.00 × 3.00 × 10.00	3.00 × 3.00 × 3.00	3.00 × 3.00 × 3.00
Repetition time (ms)	4.90	4.40	5.40
Echo time (ms)	2.30	1.80	2.30
Flip angle (${}^{\circ }$)	20	20	20
Bandwidth (Hz)	298.72	541.48	284.73
Trajectory	2D golden‐angle radial	3D golden‐mean radial Koosh ball	3D golden‐mean radial Koosh ball
Pulse sequence	2D SPGR	3D SPGR	3D SPGR
Coils (#Channels)	Anterior + Posterior (24)	Anterior + Posterior (24)	Anterior + Posterior (24)
Scanner	Philips Ingenia 1.5T	Philips Ingenia 1.5T	Philips Ingenia 1.5T
Motion model components	R = 3	R = 3	R = 6
Reference image resolution (mm)	6.70 × 6.70 × 10.00	6.70 × 6.70 × 6.70	9.05 × 9.05 × 9.05
Regularization parameter	${\lambda }_R=1.5·{10}^1$	${\lambda }_R=1.5·{10}^1$	${\lambda }_R=1.4·{10}^3$
Number of iterations	50	50	300
Splines per spatial dimension	18	16	3
Splines in time	1.28/second	8.25/second	5/second
Temporal motion resolution	40.8 Hz: 5 spokes/dynamic	7.6 Hz: 30 spokes/dynamic	9.3 Hz: 20 spokes/dynamic
Reconstructed motion duration (s)	20	33	40
Reconstruction time	4 minutes	50 minutes	2 hours

Notes: For the respiratory‐resolved reconstruction in Section 3.2, the same parameters were used as listed in the “3D resp. motion” column, but effectively resulted in a temporal motion resolution of about 5 Hz, with 18062 spokes per dynamic, due to the sorting.

We followed the approach outlined in Section 2.2, and reconstructed motion fields from linearly combined multicoil k‐space data acquired during motion by solving the minimization problem (9) with L‐BFGS, ⁴⁶ using the MATLAB implementation from Ref. (47). The low‐rank MR‐MOTUS workflow is schematically summarized in Figure 1. We refer to Table 1 for all parameter settings and to the Supporting Information of this work and the Supporting Information in Ref. (34) for more implementation details. Code that produces similar results as presented in this study is openly available at https://github.com/nrfhuttinga/LowRank_MRMOTUS.git.

3.1. Experiment 1: 2D+t in vivo respiratory motion reconstructions from abdominothoracic data

In the first experiment, a reference image and motion fields were reconstructed from the same 2D+t data acquired during 20 seconds of free‐breathing. The reference image was reconstructed from the end‐inhale bin after phase binning based on the self‐navigation signal of k = 0 values per readout (denoted as $k_0$‐values), see Supporting Information Section 3 for more details. The motion fields were reconstructed at 40.8 Hz, that is, 24.5 ms/frame, by assigning every 5 consecutive nonoverlapping spokes to one dynamic. The low‐rank model (6) was employed with R = 3, yielding motion fields with rank ≤3. Additional relevant reconstruction and acquisition parameters can be found in Table 1.

The motion fields were analyzed by comparison with a time‐resolved compressed sensing 2D+t reconstruction (CS2Dt) on the same free‐breathing data, and by means of the Jacobian determinant. For the comparison with CS2Dt, the MR‐MOTUS reference image was warped with the reconstructed motion fields to obtain a dynamic image sequence as follows. First, the motion fields are interpolated to the same spatial resolution as the image reconstruction using cubic interpolation. Second, the forward model (2) was evaluated on a Cartesian k‐space grid using the reconstructed motion fields ${\mathbf{D}}_t$. Finally, an inverse Fourier transform was performed to obtain one image per dynamic. The CS2Dt was reconstructed at a temporal resolution of 122.5 ms/frame by assigning every 25 consecutive nonoverlapping spokes to one dynamic, and was performed with the BART toolbox65 using spatial $L_1$‐wavelet and temporal total variation regularization. The temporal resolution of the CS2Dt was chosen as an integer multiple of the MR‐MOTUS resolution to allow comparison at the coarser CS2Dt temporal resolution. The comparison was performed by means of the relative error norm (REN). The REN between vectors a,b was defined as $\text{REN}(\mathbf{a},\mathbf{b})=\frac{\Vert \mathbf{a}‐\mathbf{b}\Vert }{\Vert \mathbf{b}\Vert }$.

3.2. Experiment 2: 3D+t in vivo respiratory motion reconstructions from abdominothoracic data

In the second experiment, we considered the targeted case for MR‐guided radiotherapy: a reference image and motion fields were reconstructed from 3D+t data acquired during 33 seconds of free‐breathing. The targeted high temporal resolution does not allow for a straightforward validation by comparison with dynamic 3D image reconstruction. For validation purposes, we therefore compared MR‐MOTUS with respiratory‐resolved image reconstruction by performing both reconstructions on respiratory‐sorted data.

Finally, we performed 3D+t time‐resolved motion reconstruction to demonstrate the ability to reconstruct motion at high temporal resolution from time‐resolved k‐space data. The reference image for both reconstructions was reconstructed from the end‐inhale bin after phase binning based on the $k_0$‐value per readout (see Supporting Information Section 3), and the low‐rank model (6) was employed with R = 3. See Table 1 for all reconstruction and acquisition parameters.

For the respiratory‐resolved reconstructions, phase binning was performed in 20 equal‐sized bins based on the $k_0$‐value per readout. The images were independently reconstructed for each bin using 28 iterations of CG‐SENSE. ⁵⁰ The motion fields were reconstructed over all bins simultaneously with low‐rank MR‐MOTUS by solving (9) with 20 dynamics. The quality of the MR‐MOTUS reconstruction was assessed by means of the Jacobian determinant and by comparison with the respiratory‐resolved image reconstruction. For the latter, a reference image was warped with the reconstructed motion fields to obtain a dynamic image sequence, as described in Section 3.1, and the two image sequences were compared in terms of REN. The reference image that was warped using the MR‐MOTUS motion fields was selected as the end‐inhale phase of the respiratory‐resolved image reconstruction (motion state #10) in order to reduce effects of image intensity, image quality, or contrast differences on the comparison of the two image sequences.

For the time‐resolved 3D+t reconstructions, motion fields were reconstructed at 7.6 Hz, ie, 132 ms/frame, by assigning every 30 consecutive nonoverlapping spokes to one dynamic. The reconstructions were analyzed by means of the Jacobian determinant and the average motion of the kidney was compared between the time‐resolved and respiratory‐resolved MR‐MOTUS reconstructions. This motion was computed as the mean of the displacements over a manually segmented mask of the right kidney. For comparison between respiratory‐resolved and time‐resolved, the motion magnitudes of each respiratory bin in the respiratory‐resolved reconstruction were assigned to the original, time‐resolved, spoke indices that were sorted into that particular bin.

3.3. Experiment 3: 3D+t in vivo head‐and‐neck motion reconstructions

With the third experiment, 3D+t motion fields were reconstructed from data acquired during head‐and‐neck motion. The subject was instructed to hold still in position 1 during the first 70 seconds of the acquisition, then move to position 2 and hold still for 70 seconds, then move freely for 40 seconds, and finally, hold still afterward in position 3 for 70 seconds. Data acquired in position 1 were used to reconstruct a reference image, data acquired during movement from position 2 to position 3 were used to reconstruct motion fields, and position 2 and 3 were used as fully sampled “checkpoints” to serve as validation; the beginning and end of the dynamic motion reconstruction should, respectively, coincide with positions 2 and 3. To verify this, the reference was warped with the reconstructed motion fields as described in Section 3.1, and the first and last dynamic of the resulting image sequence were visually compared with the fully sampled checkpoints. As a second analysis, the mean and standard deviation of the determinant of the Jacobian were computed for all dynamics, over all voxels within the body. The latter were determined by a threshold on the magnitude of the signal per voxel. The low‐rank motion model was employed with R = 6 to accommodate the head‐and‐neck motion which includes rotations in multiple planes. The motion fields were reconstructed at a temporal resolution of 9.3 Hz, that is, 108 ms/frame, by assigning every 20 consecutive nonoverlapping spokes to one dynamic. Additional reconstruction and acquisition parameters can be found in Table 1.

4. RESULTS

4.1. Experiment 1: 2D in vivo respiratory motion reconstructions from abdominothoracic data

The time‐resolved 2D respiratory motion was reconstructed with 40.8 motion fields per second. The Jacobian determinant and the comparison with CS2Dt is shown in Figure 2. The visual comparison with 2D+t compressed sensing image reconstruction corresponding to Figure 2B is shown in Supporting Information Video S1. It can be observed that good agreement is obtained for most phases of the respiratory cycle, with a small mismatch in end‐exhale in the upper back near the spine‐liver interface. The Jacobian determinants show small deviations from unity within the organs (green), and compression in the lungs (blue) except for the arteries. The qualitative results are supported by the quantitative results in Figure 2B, which show that the warped MR‐MOTUS images considerably reduce the REN.

FIGURE 2.

A, Jacobian determinants of the reconstructed motion fields in end‐inhale (left) and end‐exhale (right). The first end‐exhale and second end‐inhale positions were selected from all dynamics for this visualization. B, Relative error norm (REN) between MR‐MOTUS warped reference images and CS2Dt reconstruction over all dynamics (blue), and a baseline REN between the fixed MR‐MOTUS end‐exhale warped reference image and CS2Dt. The top row (I) shows the results for volunteer 1, whereas the bottom row (II) shows the results for volunteer 2. The comparison is also visualized in Supporting Information Video S1, and the reconstructed motion fields decomposed in the low‐rank model components are visualized in Supporting Information Videos S2 and S3

The warped reference images corresponding to the reconstructed motion‐field, overlayed with the motion‐field are shown in Supporting Information Videos S2 and S3. Moreover, these show the decomposition in the reconstructed low‐rank components. For volunteer 1, the first two components show pseudo‐periodic temporal behaviors, and the first is most prominent in magnitude. Both components show not only realistic movement of organs such as the liver and kidney, but also small unrealistic motion in the spine near the liver in end‐exhale. Interestingly, the third component shows a temporal behavior with a slight drift upwards, and the corresponding spatial motion‐field indicates a global rotation. Similar movement can also be observed in the ground‐truth CS2Dt reconstruction in Supporting Information Video S1. This movement could be caused by relaxation of the gluteus maximus muscle in the upper leg and buttocks. Similar motion patterns can be observed in Supporting Information Video S3 for volunteer 2, but the global rotation is less pronounced in the ground‐truth CS2Dt reconstruction.

4.2. Experiment 2: 3D in vivo respiratory motion reconstructions from abdomen/thorax data

The comparison between MR‐MOTUS and respiratory‐resolved image reconstruction is shown in Figures 3 and 4, Supporting Information Videos S4 and S5. It can be observed that good visual agreement is obtained between the two reconstructions for both volunteers. This is especially visible from the position of the top of the liver dome. The Jacobian determinants of the reconstructed motion fields are shown in Figure 4A. The lungs show compression (blue), except for the arteries, and small deviations from unity can be observed in the rest of the body. Deviation from unity can be observed at the spine‐liver interface, where a large volumetric compression is reconstructed. We expect this is related to the attachment of liver tissue to the spine during exhalation. The quantitative comparison in Figure 4B shows best agreement at motion state 10 (inhale) and worst agreement in motion state 19 (exhale). The sharp peak at motion state 10 can be explained by the fact that we took motion state 10 as the reference image to compute the warped reference images for MR‐MOTUS. The warped reference images reconstructed from the respiratory‐sorted data, overlayed with the motion‐field, are visualized for both volunteers in Supporting Information Videos S6 and S7. Moreover, these show the decomposition in the reconstructed low‐rank components. For both volunteers, the first component shows a pseudo‐periodic behavior in time and is most prominent in magnitude; the other components make only minor contributions. These large contributions of pseudo‐periodic components could be due to the periodicity assumption underlying the respiratory‐sorting. Small unrealistic motion can be observed for volunteer 1 at the spine‐liver interface and at the back of the spine, similar to the 2D reconstructions. Additionally, a small rotating motion can be observed in the motion‐field for volunteer 1 at the interface with the rib cage in the coronal slice on the bottom right. We expect the latter is caused by a combination of the volume‐preserving regularization and the inability of the motion model to resolve the sliding motion that is present in this area.

FIGURE 3.

Respiratory‐resolved image reconstruction (Resp. Resolved IR, left), MR‐MOTUS warped reference image (middle), and pixel‐wise absolute difference between the two reconstructions (right), as mentioned in Sections 3.2 and 4.2. The red and yellow horizontal lines indicated, respectively, end‐exhale and end‐inhale positions. A video corresponding to this figure of volunteer 1 is provided in Supporting Information Video S4. A similar video for volunteer 2 is provided in Supporting Information Video S5

FIGURE 4.

A, Jacobian determinants of the reconstructed respiratory‐resolved motion fields in end‐inhale (top) and end‐exhale (bottom). The first end‐exhale and second end‐inhale positions were selected from all dynamics for this visualization. B, Relative error norm (REN) with respiratory‐resolved image reconstruction (RR‐IR) for every motion state. The blue graph indicates the REN between MR‐MOTUS and respiratory‐resolved image reconstruction. The orange graph indicates a baseline comparison between the (fixed) end‐inhale image of the MR‐MOTUS reconstruction and the (dynamic) respiratory‐resolved image reconstruction. The sharp peak is caused by taking the 10th dynamic as the reference image for this comparison. The top row shows the results for volunteer 1, and the bottom row shows the results for volunteer 2. Videos corresponding to the comparisons in (B) are provided in Supporting Information Videos S4 and S5

The time‐resolved 3D respiratory motion was reconstructed with 7.6 motion fields per second. The warped reference images reconstructed from the time‐resolved data, overlayed with the motion‐field, are visualized for both volunteers in Supporting Information Videos S8 and S9. Similar motion is obtained as with the respiratory‐sorted data, but the reconstructed motion components are now similar in magnitude. All components show pseudo‐periodic temporal behavior, and the first component of volunteer 1 indicates a small drift. Similar to the respiratory‐resolved reconstructions, small unrealistic motion at the spine‐liver interface and anterior side of the spine can be observed for volunteer 1. Additionally, the same small rotation can be observed near the rib cage in the bottom right of the coronal slice. The Jacobian determinants of the reconstructed motion fields are shown in Figure 5. Similar patterns can be observed in end‐exhale as for the respiratory‐resolved motion reconstructions. Interestingly, the end‐inhale image for volunteer 1 shows a small expansion in the lungs, possibly indicating that a deeper inhale than the reference image was reconstructed while the reference image was obtained using respiratory‐sorting on end‐inhale. Finally, the comparison between the average kidney motion in the time‐resolved and respiratory‐resolved MR‐MOTUS reconstructions is visualized in Figure 6. The phase of the reconstructions are most similar in feet‐head (FH) and anterior‐posterior (AP), while in left‐right (LR) different patterns can be observed. However, it should be noted that the motion in FH and AP is two orders of magnitude higher than in LR. The motion magnitude is similar for both reconstructions, but the respiratory‐resolved reconstruction shows a constant amplitude over time since it only reconstructs an average breathing cycle. The time‐resolved reconstruction shows changing motion amplitudes over time. The phase difference between the two reconstructions may be explained by imperfect respiratory‐sorting.

FIGURE 5.

Jacobian determinants of the reconstructed time‐resolved motion fields in end‐inhale (top) and end‐exhale (bottom). The left figure shows the results for volunteer 1 and the right figure the results for volunteer 2. Videos corresponding to the reconstructions in this figure are provided in Supporting Information Videos S8 and S9

FIGURE 6.

Average motion of the right kidney over time, for both the respiratory‐resolved and the time‐resolved MR‐MOTUS reconstructions as mentioned in Sections 3.2 and 4.2. The respiratory‐resolved MR‐MOTUS reconstruction was projected back on the time axis, as described in Sections 3.2 and 4.2. The average motion magnitudes were computed over a manually segmented mask of the right kidney. Videos of reconstructions corresponding to these figures are provided in Supporting Information Videos S8 and S9

4.3. Experiment 3: 3D in vivo head‐and‐neck motion reconstructions

The time‐resolved 3D head‐and‐neck motion was reconstructed with 9.3 motion fields per second. The MR‐MOTUS warped reference images from 3D data acquired during head‐and‐neck motion are visualized for both volunteers in Supporting Information Videos S10 and S11. Clearly, rigid motion fields are reconstructed within the skull, and nonrigid motion fields at the neck. Figure 7 shows the Jacobian determinants of the reconstructed motion‐field over time (A), and the reconstructed temporal components (B) for both volunteers. The Jacobian determinant is close to 1 over the whole reconstructed time, with slightly more deviations for volunteer 1. These can be attributed to larger and more irregular motion than volunteer 2. The temporal components are relatively flat at the start and the end, corresponding to the static begin and end positions. The more extreme motion of volunteer 1 can also be observed from the larger magnitudes of the temporal components and from Supporting Information Video S10. Figure 8 shows the checkpoint validation for volunteer 2. It can be observed that good agreement is obtained between the fully sampled checkpoint images and the MR‐MOTUS reconstructions.

FIGURE 7.

Head‐and‐neck reconstructions as described in Sections 3.3 and 4.3. A, The mean (solid line) and standard deviation (shaded area) of the Jacobian determinants of the reconstructed motion fields over time. B, The reconstructed temporal profiles ${\mathbf{\Psi }}^i$, scaled by the norm of the corresponding ${\mathbf{\Phi }}^i$ to be able to compare their magnitudes. The top row and bottom rows, respectively, show the results for volunteers 1 and 2. Videos corresponding to the reconstructions in these figures are provided in Supporting Information Videos S10 and S11

FIGURE 8.

Checkpoint validation for the head‐and‐neck reconstructions of volunteer 2, as mentioned in Sections 3.3 and 4.3. The left columns shows the fully sampled checkpoint image, the middle column shows the MR‐MOTUS warped reference images and the right column shows the absolute pixel‐wise difference. The top part corresponds to the comparison with the checkpoint acquired right before the start of the motion, and the bottom part corresponds to the checkpoint acquired right after the start of the motion. A video corresponding to this figure is provided in Supporting Information Video S11. A similar video for volunteer 1 is provided in Supporting Information Video S10

5. DISCUSSION

We have previously introduced MR‐MOTUS, ³⁴ a framework to estimate motion directly from minimal k‐space data and a reference image by exploiting spatial correlation in internal body motion. In this work, we introduce low‐rank MR‐MOTUS: an extension of MR‐MOTUS from 3D to 3D+t reconstructions in a realistic experimental setting, where both reference image and motion fields are reconstructed from data acquired during free‐breathing. Low‐rank MR‐MOTUS employs a low‐rank motion model that constrains the degrees of freedom in space and time, thereby reducing memory consumption and functioning as a regularization in both space and time. It was demonstrated that the proposed method can reconstruct high quality 3D motion fields with a temporal resolution of more than 7.6 motion fields per second, while showing consistency with static, respiratory‐resolved and time‐resolved image reconstructions. Prospectively undersampled data were acquired with a non‐Cartesian trajectory and multichannel receivers, thereby bridging the gap toward clinical application.

The ability of the proposed framework to estimate time‐resolved rather than respiratory‐resolved motion is promising as it allows to characterize drifts and intra and inter‐cycle breathing patterns. This is in contrast with respiratory‐resolved methods that require sorting to obtain suitable images. ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁹ , ⁵⁰ , ⁵¹ , ⁵² , ⁶⁶ The sorting effectively results in (a motion model for) average breathing motion, which may have trouble capturing drifts and inter‐cycle variations. Some works have been proposed to reconstruct time‐resolved MR‐images without the need of retrospective sorting. However, the reported temporal resolution was too low, ²⁸ , ³⁰ , ³¹ or the FOV was too small. ³² , ³³ The time‐resolved motion estimation of low‐rank MR‐MOTUS in combination with an MR‐LINAC can be particularly beneficial for MR‐guided radiotherapy; the (retrospective) reconstruction of 3D+t time‐resolved tumor and organs‐at‐risk motion during treatment can be used for accurate dose accumulation, ³⁸ allowing for an accurate assessment of the treatments.

The resulting motion model explicitly separates a high‐dimensional static spatial component from a low‐dimensional dynamic temporal component. The low dimensionality of the dynamic behavior could be exploited to reduce the number of parameters and reconstruction times of future real‐time reconstructions, analogously to recently proposed approaches in Refs. [7, 19, 22, 53]. Our method could thereby form the basis for future work on real‐time MR‐based motion estimation, where reconstructions are performed on‐the‐fly to track tumor and organs‐at‐risk motion.

Low‐rank models in the context of motion estimation have been investigated before in several works, most of which retrospectively perform compression to a low‐rank model using principal component analysis. ¹⁹ , ³⁵ , ³⁷ , ³⁸ Others decouple the motion fields into spatial components and temporal components based on surrogate signals. ²¹ , ²² , ³⁹ The approach in this work is different in the sense that it explicitly and a priori enforces a structure that yields low‐rank motion fields, and does not assume dependence on surrogate signals for the motion model. Similar approaches have been studied in the context of image reconstruction. ³⁰ , ³² , ⁵⁶ , ⁵⁷ , ⁵⁸ , ⁵⁹ , ⁶⁰

This work includes some limitations and assumptions that should be addressed. Both the respiratory‐resolved and time‐resolved 3D respiratory motion reconstructions in Sections 3.2 and 4.2 look realistic in general. Yet, small unrealistic motion is reconstructed near discontinuities in the true motion fields that are present near sliding or attaching/detaching organ surfaces. This can be observed in for example Supporting Information Video S2, at the spine‐liver interface in end‐exhale. This could possibly be resolved with region‐specific ⁶¹ or nonparametric motion models, ⁶² but is beyond the scope of this work.

The respiratory motion reconstruction parameters in this work were fixed and were based on the results of a grid search for a single volunteer. We have empirically observed that the parameters from the grid search for a single volunteer are generalizable and yield acceptable results for all respiratory motion reconstructions in this work. Hence, in a realistic setting where grid searches may not be feasible due to time constraints, similar reconstruction parameter settings could be employed. The number of components R may be more subject‐dependent, and can be determined with analyses similar to the ones described in Supporting Information Section 1.

This work is based on the assumption that a motion‐free reference image is available, and that it can be warped with unknown motion fields into dynamic image series of a moving anatomy. For respiratory motion, these reference images could be extracted by means of respiratory binning, but this may be less straightforward for other body motion such as bladder filling, bowel movements, and prostate movement. However, these types of motion take place on a slower temporal scale, so more time would be available to gather information and update the reference image. Additionally, the modeling assumption regarding the warping of a single reference image into the dynamic image series may be partly violated due to unwanted contrast effects such as susceptibility‐induced $B_0$ variations. Yet, these effects can be assumed small at the targeted field strength of 1.5T. Nevertheless, future work will consider incorporating such additional contrast effects in the signal model, and performing joint reconstructions of the motion fields and the reference image.

Resulting from the grid search for optimal reconstruction parameters, we have used relatively low‐resolution reference images. This delivered the best performance, and allowed a factor 3 reduction of the reconstruction time. These low‐resolution reference images could allow for high‐resolution motion fields by means of interpolation, but do, however, provide less precise information regarding organ boundaries.

Contrary to standard coil compression techniques, the aim of the compression of multi‐channel data to a single channel in this work is homogeneous coil sensitivity. Consequently, this compression is suboptimal in terms of SNR. ⁶³ , ⁶⁴ Supporting Information Figure S3 analyzes the loss between a Roemer coil combination ⁶³ and the proposed coil compression on the 2D data. This shows an SNR loss factor between 1.5 and 2.5 in most of the body, which increases toward the boundary of the body. Good results were obtained with the coil compression introduced in this work, but more advanced techniques could possibly be used to improve the SNR after the compression. We refer to Supporting Information Section 2 for a more extensive discussion.

The last point of improvement is the validation of time‐resolved 3D+t motion fields. In general, this is not straightforward, and we considered three viable options for this: (a) in silico with a digital phantom, (b) with an MR‐compatible motion phantom, and (c) in vivo with respiratory‐resolved image reconstruction. We have opted for the third option, since this was considered the closest to a practical use‐case. The in silico validation does not consider real acquisition–related data corruption (eg, eddy currents, flow effects), and can, in case of for example, the XCAT phantom, ⁶⁵ yield unstable motion fields. ⁶⁶ MR‐compatible motion phantoms, although useful for proof‐of‐principle validations, represent simplified in vivo anatomies.

The intended application of MR‐MOTUS is MR‐guided radiotherapy, possibly in real‐time. However, the current reconstruction times in MATLAB on a desktop workstation are around 4 min for 2D+t with 40 motion fields/second, around 6 min for the respiratory‐resolved 3D reconstruction, and around 50 min for 3D+t time‐resolved respiratory motion with 7.6 motion fields/second. Hence, the current implementation of the method is not directly applicable for real‐time processing, but reconstruction times may be reduced with a different programming language, improved hardware, GPU‐accelerations, or deep learning.

6. CONCLUSION

We have introduced low‐rank MR‐MOTUS, an extension of MR‐MOTUS that allows to retrospectively reconstruct time‐resolved 3D+t motion fields from prospectively undersampled k‐space data and one reference image. Reconstructions were performed for 2D/3D respiratory motion and 3D head‐and‐neck motion. A temporal resolution of more than 7.8 motion fields per second was obtained, and the motion fields were consistent with image reconstructions. For MR‐guided radiotherapy, the time‐resolved 3D motion fields could be used to reconstruct the respiratory‐motion‐compensated accumulated dose during the treatment. Furthermore, the explicit decomposition of motion fields in static and dynamic components could form the basis for future work toward real‐time MR‐guided radiotherapy.

DATA AVAILABILITY STATEMENT

Code that produces similar results as presented in this study is openly available at https://github.com/nrfhuttinga/LowRank_MRMOTUS.git.

Supporting information

FIGURE S1 Results of the singular value analyses in Supporting Information Section 1 for 3D+t respiratory motion (left), and 3D+t head‐and‐neck motion (right). This figure clearly indicates that 3D+t motion fields possess the low‐rank property; models with R = 3 and R = 6 can, respectively, capture 97.9% and 99.9% of the variance of 3D+t respiratory motion and 3D+t head‐and‐neck motion, allowing for a significant reduction in the number of unknowns

FIGURE S2 This figure shows the effect of a different number of components R = 1, …, 20 on the respiratory motion reconstructions, as discussed in Supporting Information Section 1. The metrics were evaluated on respiratory‐resolved MR‐MOTUS and image reconstructions, and the means over all 20 reconstructed respiratory phases are visualized in this figure. Only a minimal change can be observed in both metrics, showing that the effect of R on the results is minimal for respiratory motion. NMI = Normalized Mutual Information, REN  =  Relative Error Norm

FIGURE S3 A, Roemer reconstruction. ⁶² B, The coil compression with ${\lambda }_{\mathit{CC}}=5·{10}^5$, as discussed in Supporting Information Section 2. C, SNR loss factor between (A) and (B). D, The histogram of the SNR loss factor in (C). The SNR loss factor is between 1.5 and 2.5 in most of the body and increases toward the boundary of the body

FIGURE S4 Results of the parameter search as mentioned in Supporting Information Sections 5 and 3.2. A, The effect of the reference image resolution and reference image respiratory binning phase on the reconstruction quality. B, The effect of the reference image resolution on the reconstruction time. In the figure, “InhaleBinned” refers to the binning phase for the reference image (InhaleBinned = 1 for inhale, InhaleBinned = 0 for exhale), “Resolution” denotes the spatial resolution of the reference image, and ‘SplineOrder’ denotes the number of spline basis functions defined per spatial dimension

VIDEO S1 This is an animated figure and should be viewed under Supporting Information. 2D‐t compressed sensing reconstruction (left), MR‐MOTUS warped reference images (middle), and pixel‐wise absolute differences between the two reconstructions (right), as mentioned in Sections 3.1 and 4.1. The top row shows reconstructions for volunteer 1, and the bottom row for volunteer 2

VIDEO S2 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images overlayed with reconstructed dynamic motion fields from 2D time‐resolved data, as mentioned in Sections 3.1 and 4.1. The image shows a decomposition in the reconstructed components ${\mathbf{\Phi }}^i$ (spatial) and Ψ (temporal) for volunteer 1. For visualization purposes, the components were scaled such that $\Vert {\mathbf{\Phi }}^i\Vert =1$

VIDEO S3 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images overlayed with reconstructed dynamic motion fields from 2D time‐resolved data, as mentioned in Sections 3.1 and 4.1. The image shows a decomposition in the reconstructed components ${\mathbf{\Phi }}^i$ (spatial) and Ψ (temporal) for volunteer 2. For visualization purposes the components were scaled such that $\Vert {\mathbf{\Phi }}^i\Vert =1$

VIDEO S4 This is an animated figure and should be viewed under Supporting Information. Respiratory‐resolved image reconstruction (Resp. resolved IR, left), MR‐MOTUS warped reference images (middle), and pixel‐wise absolute differences between the two reconstructions (right), as mentioned in Sections 3.2 and 4.2. The visualization shows data from volunteer 1

VIDEO S5 This is an animated figure and should be viewed under Supporting Information. Respiratory‐resolved image reconstruction (Resp. resolved IR, left), MR‐MOTUS warped reference images (middle), and pixel‐wise absolute differences between the two reconstructions (right), as mentioned in Sections 3.2 and 4.2. The visualization shows data from volunteer 2

VIDEO S6 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images overlayed with reconstructed dynamic motion fields from respiratory‐sorted data, as mentioned in Sections 3.2 and 4.2. The image shows a decomposition in the reconstructed components ${\mathbf{\Phi }}^i$ (spatial) and Ψ (temporal) for volunteer 1. For visualization purposes, the components were scaled such that $\Vert {\mathbf{\Phi }}^i\Vert =1$

VIDEO S7 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images overlayed with reconstructed dynamic motion fields from respiratory‐sorted data, as mentioned in Sections 3.2 and 4.2. The image shows a decomposition in the reconstructed components ${\mathbf{\Phi }}^i$ (spatial) and Ψ (temporal) for volunteer 2. For visualization purposes, the components were scaled such that $\Vert {\mathbf{\Phi }}^i\Vert =1$

VIDEO S8 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images overlayed with reconstructed dynamic motion fields from respiratory‐sorted data, as mentioned in Sections 3.2 and 4.2. The image shows a decomposition in the reconstructed components ${\mathbf{\Phi }}^i$ (spatial) and Ψ (temporal) for volunteer 1. For visualization purposes, the components were scaled such that $\Vert {\mathbf{\Phi }}^i\Vert =1$

VIDEO S9 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images overlayed with reconstructed dynamic motion fields from respiratory‐sorted data, as mentioned in Sections 3.2 and 4.2. The image shows a decomposition in the reconstructed components ${\mathbf{\Phi }}^i$ (spatial) and Ψ (temporal) for volunteer 2. For visualization purposes, the components were scaled such that $\Vert {\mathbf{\Phi }}^i\Vert =1$

VIDEO S10 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images resulting from the 3D head‐and‐neck motion reconstructions for volunteer 1, as mentioned in Sections 3.3 and 4.3

VIDEO S11 This is an animated figure and should be viewed under Supporting Information. MR‐MOTUS warped reference images resulting from the 3D head‐and‐neck motion reconstructions for volunteer 2, as mentioned in Sections 3.3 and 4.3

VIDEO S12 This is an animated figure and should be viewed under Supporting Information. Respiratory‐resolved image reconstruction (Resp. resolved IR, left), MR‐MOTUS warped reference images (middle), and pixel‐wise absolute differences between the two reconstructions (right), as mentioned in Supporting Information Sections 5 and 3.2. The four blocks show reconstructions with different reconstruction parameter settings. “InhaleBinned” denotes whether the reference image is binned in inhale (1) or exhale (0). ‘Ref. resolution” denotes the resolution of the reference image in millimeters. All motion fields were reconstructed without regularization and with 9 cubic spline functions in every direction

Huttinga NRF, Bruijnen T, van den Berg CAT, Sbrizzi A. Nonrigid 3D motion estimation at high temporal resolution from prospectively undersampled k‐space data using low‐rank MR‐MOTUS. Magn Reson Med. 2021;85:2309‐2326. 10.1002/mrm.28562