Magnetic Resonance SIGnature MAtching (MRSIGMA) for real-time volumetric imaging

Abstract

Purpose:

To propose a real-time 3D MRI technique called MR SIGnature MAtching (MRSIGMA) for high-resolution volumetric imaging and motion tracking with very low latency.

Methods:

MRSIGMA consists of two steps: (1) offline learning of a database of possible 3D motion states and corresponding motion signature ranges, and (2) online matching of new motion signatures acquired in real time with pre-learned motion states. Specifically, the offline learning step (non-real-time) reconstructs motion-resolved 4D images representing different motion states and assigns a unique motion range to each state. The online matching step (real-time) acquires motion signatures only and selects one of the pre-learned 3D motion states for each newly-acquired signature, which generates 3D images efficiently in real time. MRSIGMA was evaluated on fifteen golden-angle stack-of-stars liver datasets; and the accuracy of respiratory motion tracking with the online-generated real-time 3D MRI was validated against corresponding 2D projections acquired in real time.

Results:

The total latency of generating each 3D image during online matching was ~300ms, including acquisition of the motion signature data (~138ms) and corresponding matching process (~150ms). Linear correlation assessment suggested excellent correlation (R²=0.948) between motion displacement measured from the online-generated real-time 3D images and the 2D real-time projections.

Conclusion:

This proof-of-concept study demonstrated the feasibility of MRSIGMA for high-resolution real-time volumetric imaging, which shifts the acquisition and reconstruction burden to an offline learning step and leaves fast online matching for online imaging with very low imaging latency. MRSIGMA can potentially be used for real-time motion tracking in MRI-guided radiation therapy.

INTRODUCTION

Accurate and precise treatment delivery is required in radiation therapy to maximize irradiation in a tumor and to minimize toxicity in surrounding healthy tissue. The use of MRI to guide radiation therapy has gained substantial interest in recent years due to its superior soft-tissue contrast, absence of ionizing radiation, and flexibility in acquiring multi-contrast images that can be used both for pre-treatment planning and for post-treatment evaluation (1–4). The new integrated MR-Linac system, combining an MRI scanner with a linear accelerator (5,6), is now available for simultaneous imaging and radiation treatment. One of the main promises of the MR-Linac modality is to enable MRI-guided adaptive radiotherapy (1,7–9), which may particularly be useful for moving organs. However, MRI is relatively slow to image volumetric organ motion in real time. The term real-time imaging is used here to refer to an imaging system with a total imaging latency (acquisition time + reconstruction time) lower than the dynamic process of interest. For example, the respiratory rate in adults is ~12–18 breaths per minute, which translates to a respiratory cycle of ~3–5 seconds. Therefore, an imaging technique would need a latency no longer than 300ms to sample 10 respiratory phases per respiratory cycle in real time. Even with the latest acquisition and reconstruction techniques, real-time MRI is still primarily limited to 2D imaging, which suffers from through-plane motion misregistration and suboptimal interpretation of motion. Advanced iterative reconstruction, such as compressed sensing (10), has been increasingly used to improve spatiotemporal resolution and image quality in real-time imaging (11,12), but this can prolong reconstruction time and thus lead to increased imaging latency. Given the slow imaging speed to acquire volumetric MRI data in real time, moving organs, such as the liver and the lung, pose a significant challenge for MRI-guided adaptive treatment delivery (13).

Different approaches have been developed to image a moving organ in real time for potential applications in MRI-guided adaptive radiation therapy. For example, several studies have proposed to perform fast 2D cine imaging in three orthogonal directions simultaneously, from which pseudo-3D images can be estimated for deriving volumetric tumor motion information (14–17). More advanced approaches aim to calculate a patient-specific motion model that can link a 4D motion-resolved image-set acquired over multiple motion cycles (slow) with real-time 2D cine imaging (fast), so that real-time pseudo-3D images can be generated with low acquisition latency (18–22). However, the conversion of 2D images to pseudo-3D images often involves a computationally expensive interpolation process, which can result in high reconstruction latency without immediate feedback, thus restricting real-time adaptive treatment delivery. Meanwhile, since interpolation artifacts may occur during the conversion step, these errors can be propagated through the entire treatment workflow. Highly-accelerated motion-resolved 4D MRI approaches have also been proposed for MRI-guided radiation therapy (23–25). However, these approaches typically have limited real-time capability since they require long acquisition times over multiple respiratory cycles (non-real-time) and/or complex iterative image reconstruction that increases overall latency. Thus, high-resolution real-time 3D MRI remains an unmet need for motion adaptive radiation therapy.

This work presents a novel real-time 3D MRI technique called MR SIGnature MAtching (MRSIGMA), which aims to address the above-mentioned challenges by pre-learning 3D motion states in an offline step and performing fast signature matching for an online real-time imaging step. The method enables high-resolution real-time volumetric imaging with sub-second imaging latency, presenting good promise for MRI-guided adaptive radiation therapy. In the following sections, the idea behind MRSIGMA is described first, followed by a practical implementation of the framework using a 3D stack-of-stars golden-angle radial sampling scheme. The performance of MRSIGMA is then evaluated on numerical abdominal phantom and free-breathing liver datasets.

METHODS

General Idea of MRSIGMA

MRSIGMA consists of two steps: (1) offline learning of 3D motion states and corresponding motion signature ranges, and (2) online matching of high temporal resolution signature-only data acquired in real time with one of the pre-learned motion states. These two steps enable the generation of real-time 3D images with very low imaging latency.

Specifically, as shown in Figure 1a, the purpose of the offline learning step is to continuously acquire both imaging (blue blocks-D) and motion signature (red blocks-S) data alternatively during free breathing, followed by reconstruction of a motion-resolved 4D image-set to form a database of motion states (purple blocks, MS-1 to MS-n). From the acquired motion signature, a unique motion signature range (red blocks, SR-1 to SR-n) is generated and assigned to represent each motion state, where the signature range indicates the intrinsic range of motion displacement for a given motion state. This is performed based on the fact that respiratory motion occurs (pseudo-) periodically. The number of motion states can be adapted for different applications, and in practice, can also be determined according to the total acquisition time, motion range and reconstruction time. A larger number of motion states leads to improved motion resolution at the expense of increased computational cost, which prolongs reconstruction time to generate the motion database. The motion signature can be extracted directly from acquired data (i.e., self-navigation) or can be acquired explicitly as additional navigators. A key point for MRSIGMA is that the acquisition of each signature must be very fast (i.e., ~100–200 ms).

Figure 1.

General description of MRSIGMA. (1) Offline learning acquires motion signature and 3D imaging data over multiple respiratory cycles to create a database of high-resolution 3D motion states and motion signature ranges. (2) Online matching acquires signature data only at high temporal resolution. A 3D motion state whose signature best matches each newly-acquired signature data is selected as the output image. Real-time 3D motion tracking is accomplished by performing all the time-consuming acquisition and reconstruction tasks during offline learning and leaving just signature data acquisition and signature matching during online matching to minimize imaging latency.

As shown in Figure 1b, the online matching step, to be performed during the online treatment period in the context of radiation therapy, acquires signature data only. This ensures that data acquisition can be fast enough for tracking organ motion in real time. The 3D motion state whose motion signature range contains the newly-acquired online signature is then selected from the pre-learned database as the output volumetric image for this time point. This unique framework of MRSIGMA shifts the time-consuming data acquisition and reconstruction burden to an offline learning step, leaving simple and rapid operations (signature acquisition and simple signature matching) for online matching with substantially reduced imaging latency.

Implementation of MRSIGMA

We propose to implement MRSIGMA using a combination of 3D golden-angle radial k-space sampling (26) and eXtra-Dimensional Golden-angle RAdial Sparse Parallel (XD-GRASP) reconstruction (27). Golden-angle radial sampling enables continuous data acquisition and flexible data sorting to generate different motion states (27–31). The high level of incoherence along time provided by the golden-angle rotation scheme facilitates the use of compressed sensing reconstruction approaches (32–34), which speeds up the image acquisition process. Moreover, radial sampling allows for self-navigation, from which a motion signal can be directly extracted from the acquired data to serve as motion signatures (27,28).

In this work, continuous 3D acquisitions are performed using a stack-of-stars golden-angle sampling trajectory (Figure 2) (35), in which the in-plane k_x-k_y encoding is implemented using radial lines rotated by the golden angle (111.25^o) and the slice encoding (k_z) is implemented on a Cartesian grid. Since each radial line passes through the center of k-space, a respiratory motion signal, which can serve as motion signatures, can be extracted from the acquired data. Specifically, a 1D projection along the z dimension can be obtained for each rotation angle by applying an inverse fast Fourier transform (FFT) along each k_z line formed by the central k_x-k_y k-space points (27). Figure 2 shows an example in which the red dashed lines show the k_z profiles for each acquisition angle/time point in the k_z-t space (Figure 2a) and corresponding projection profiles in the z-t plane with detected respiratory motion signals (Figure 2b: red and blue curves for the offline learning and online matching, respectively). The respiratory motion signal is detected from each coil element with a principal component analysis (PCA) operation, and a clustering algorithm is used to generate a single motion signal from an ensemble of motion signals estimated from each coil. More details about the automated respiratory motion detection in stack-of-stars imaging was described in (28). In our current implementation, the online matching step is also implemented using the golden-angle imaging trajectory, and each radial stack at one rotation angle is treated as data for generating motion signature at the corresponding time point. For both the offline learning and the online matching steps, the detected respiratory motion signals serve as the motion signatures.

Figure 2.

(a) Stack-of-stars golden-angle radial sampling for MRSIGMA. (b) High temporal resolution signature data are generated by taking an FFT along the z-dimension for each angle to generate projections. A respiratory signal is extracted from the projections to serve as motion signatures during both the offline learning step (red curve) and the online matching step (blue curve).

Figure 3 shows k-space sorting, generation of motion signature ranges, XD-GRASP reconstruction of motion states database, and the procedure of online signature matching. Specifically, based on a respiratory motion signal extracted from the offline training data, 3D k-space data are first sorted into a number of undersampled motion states spanning from expiration to inspiration based on the motion amplitude, where the range of motion displacement for each motion phase is assigned as the signature range for this motion state as indicated in the figure. The sorted 5D k-space data (k_x-k_y-k_z-t-coil) are then reconstructed into a motion-resolved 4D image-set (x-y-z-t) using XD-GRASP, where the fourth dimension represents motion states. Reconstruction is performed using a multicoil compressed sensing approach that exploits temporal correlations along the motion dimension by minimizing differences between adjacent temporal frames (minimizing total variation along motion states), and the reconstruction algorithm aims to solve the following optimization problem:

$$m=\underset{m}{\arg \min }\frac{1}{2}{\Vert \sqrt{W}FCm-\sqrt{W}y\Vert }_2^2+\lambda {\Vert Sm\Vert }_1$$ [1]

Figure 3.

Offline learning and online matching steps separated by a break interval. An example of the break interval can be the transition from beam-off to beam-on for the MR-Linac. During offline learning, the motion database is generated where each entry is given by the pair of motion signature (red line) and motion state (3D image). During online matching, online signature data (blue line) are matched to the corresponding offline signature (red line). Here, the blue dashed rectangular box represents the offline motion signatures.

Here, F is the non-unform FFT (NUFFT) operator, C represents coil sensitivities and W is corresponding density compensation matrix for radial sampling. y is acquired multicoil k-space data sorted into multiple motion states (k_x-k_y-k_z-t-coil) and m is the motion-resolved 4D images (x-y-z-t) to be reconstructed. A regularization parameter λ controls the balance between the data consistency (the left term) and the promotion of sparsity (the right term). S is an operator for calculating first-order finite differences along the motion state dimension.

As mentioned above, only motion signatures (i.e., respiratory motion signal, blue curves in Figure 2&3, extracted from the radial stacks acquired at different rotation angles) are acquired during the online matching step, so that high temporal resolution real-time 3D images can be generated efficiently without an explicit iterative reconstruction step. Figure 3 presents the detailed matching process. For a given time point, corresponding online motion signature (e.g., the magenta star) is compared to the pre-generated motion signature ranges. A 3D motion state whose motion signature range contains the amplitude of the current online signature is selected as the output image at the current time point. The same procedure can be repeated for a different online motion signature (e.g., the green star). If an online motion signature cannot find a corresponding offline motion range (e.g., the brown star), this time point is slipped, and the same procedure moves to the next time point. The process of acquiring data (radial stacks for different rotation angles/time points) for online signature generation, computing online signatures (inverse Fourier transform on the central k_x-k_y points followed by motion signal extraction) and simple online signature matching minimizes total imaging latency by reducing both data acquisition time and computation time.

Numerical Abdominal Phantom Study

The idea of MRSIGMA was first demonstrated using a numerical phantom with realistic abdominal anatomy and respiratory motion that was previously developed by the Seiberlich Lab at Case Western Reserve University (36). The source code for the phantom simulation was downloaded from https://github.com/SeiberlichLab/Abdominal_MR_Phantom. The phantom experiment was carried out with the following steps. First, 4D images consisting of a total of 2000 time points were simulated with a consistent and uniform breathing pattern with ~4.5 seconds per respiratory cycle (corresponding to 28 time points/ respiratory cycle in our phantom image-set). The matrix size of each image volume was 256×256×48 with a voxel size of 1.56×1.56×5.0mm³. The largest deformation along the superior-inferior direction, the anterior-posterior direction and the left-right direction was 13mm, 6.5mm, and 2mm, respectively. After generation of the image-set, zero-mean Gaussian noise with a signal-to-noise ratio of 46dB was then added to all the time points to ensure a more realistic simulation. Second, for each image slice, 2000 single-coil radial spokes rotated by the golden angle (111.25^o) were generated using NUFFT with 2-fold oversampling. Each rotating spoke was generated from one image frame, which resembles a typical golden-angle radial acquisition scheme. This process was repeated for all the 48 slices to simulate stack-of-stars k-space data with dimensions of 512×2000×48.

In-Vivo Studies

Data Acquisition

Fifteen 3D liver datasets, previously acquired using a prototype fat-saturated T₁-weighted stack-of-stars golden-angle sequence on a 3.0T MRI scanner (TimTrio, Siemens Healthineers, Germany), were retrospectively used to test the MRSIGMA framework in this work. No specific patient inclusion or exclusion criterial were applied. Data acquisition was approved by the local Institutional Review Board (IRB) and written informed consent form was obtained from each subject prior the MR scan. Relevant imaging parameters included: in-plane matrix size = 256×256, FOV = 320×320mm², voxel size = 1.25×1.25×5mm³, TR/TE = 3.40/1.68ms, flip angle = 10^o and bandwidth = 600 Hz/pixel. A total of 1000 spokes were acquired during free-breathing for each k_z position and a total of 35 partitions were acquired. The total acquisition time (TA) was 138 seconds. The stack-of-stars acquisition was performed in a way that all the partitions for a given rotation angle were acquired linearly before moving to the next acquisition angle. The radial scan was added to the end of the clinical exam after the injection of Gd-EOB-DTPA (Eovist).

To test the potential influence of respiratory variations during longer scans, additional two liver datasets (one from a volunteer and the other one from a patient referred for radiation therapy, both with local IRB approval and written consent) were also prospectively acquired on a 3.0T MRI scanner (Ingenia, Philips Healthcare Netherlands) and the MR data were acquired for 8.5 minutes each. Datasets were acquired using a fat-saturated T₁-weighted stack-of-stars golden-angle imaging sequence. Relevant acquisition parameters were: in-plane matrix size = 256×256, FOV = 384×384mm², voxel size = 1.50×1.50×4mm³, TR/TE = 4.06/1.68ms, flip angle = 12^o and bandwidth = 618 Hz/pixel. A total of 1792 spokes were acquired during free-breathing for each k_z position and a total of 51 partitions were acquired.

Image Reconstruction

For the phantom study, the first 1800 spokes were used for offline training. XD-GRASP was performed to reconstruct motion-resolved 4D images with 10 motion states spanning from expiration to inspiration which serve as the motion database. A respiratory motion signal was extracted using the algorithm as described above and with more details in (28). The sorting process was performed in a way that each motion state had the same number of spokes (200 radial stacks for each motion state). Using the matching algorithm described in the “implementation” subsection above and outlined in Figure 3, online matching was performed on the last 200 radial stacks to generate one 3D image for each rotation angle. The generated real-time 3D images were then compared with the corresponding ground-truth images.

For all the short in-vivo scans (TA=138 seconds, n=15) and the long in-vivo scans (TA=8.5 minutes, n=2), a respiratory motion signal was first extracted from the acquired k-space. Simulation and test of MRSIGMA were performed on the short scans only. Offline learning was performed using radial stacks 6–905 (900 spokes per slice). The first 5 radial stacks were discarded since they were not at steady state. The 900 offline learning radial stacks were sorted into 10 motion states (90 radial stacks for each motion state) spanning from expiration to inspiration, and XD-GRASP was applied to reconstruct motion-resolved 4D images. Online matching was performed using radial stacks 906–1000 (95 spokes per slice) to generate a 3D image for each rotation angle at a temporal resolution of 138 ms per volume (138 seconds / 1000 radial stacks). Long scans were employed to assess respiratory motion variation over long acquisitions and XD-GRASP was applied to generate 10 motion states.

The reconstruction was implemented in MATLAB (Mathworks, MA) using a nonlinear conjugate gradient algorithm to solve Equation 1 and was performed in a workstation with OSX operation system, 8-core CPU and 64GB memory. For in-vivo datasets, coil sensitivity maps were estimated from a 3D image reconstructed by averaging all the acquired spokes using the adaptive combination method (37).

Data Analysis and Statistical Analysis

Given the challenges of obtaining high-quality real-time 4D images with our target resolution (1.25×1.25×5mm³, 138 ms per volume) and imaging latency for comparison and validation, we designed a special data analysis strategy for evaluating the accuracy of MRSIGMA in-vivo. The strategy involved correlating the motion displacement of the liver dome in the online-generated real-time 3D images with that in the corresponding x-z 2D real-time projections, which were obtained by applying a 2D inverse FFT on the online signature data (2D planes in k_x-k_z dimension for the last 95 radial stacks), as shown in Supporting Information Figure S1. Since these data were acquired in real time (for generating online motion signatures), they can be treated as a reference for motion displacement. The analysis was performed in MATLAB. Both the online-generated 3D images and real-time 2D projections were first interpolated (zero-filling in k-space) along the slice dimension for better visualization. In a following step, as shown in Figure 4a for two acquisition angles, the distance of the liver dome with respect to the top edge of the FOV (the yellow two-way arrows) was manually measured for both 3D images and 2D projection independently. Specifically, the pixel index of the liver dome was recorded using the MATLAB function “ginput”. The distance was then calculated by multiplying the reconstructed/interpolated voxel size along the head-to-foot direction with the number of pixels between the liver dome and the top edge of the FOV. For measurement in the 3D images, the distance was calculated in the coronal plane and a central slice that best represents the liver dome was used for the analysis. Since the liver dome cannot be visualized clearly in certain acquisition angles, such as those shown in Figure 4b, the distance was only measured in selected angles (0^o-45^o, 135^o-225^o and 315^o-360^o). The distances measured from the real-time 3D images and the real-time 2D projections were compared using linear correlation.

Figure 4.

Comparison of x-z 2D projections with corresponding 3D images obtained with MRSIGMA at different acquisition angles. The distance of the liver dome with respect to the top edge of the FOV (yellow two-way arrows) was measured for both online-generated 3D images and real-time 2D projections to validate the accuracy of MRSIGMA. These images also suggest that the liver dome can be seen in certain angles (a) and cannot be visualized clearly in some angles (b), and thus, the distance was only measured in the acquisitions angle from which the liver dome can be visualized.

RESULTS

Numerical Phantom Study

Figure 5a shows the respiratory motion signal extracted from the synthesized stack-of-stars k-space. The respiratory motion matches well with the underlying projections, from which the signal was extracted. Figure 5b&c show the comparison of phantom ground-truth images (simulated phantom images) with corresponding images generated using MRSIGMA in both axial and coronal planes at two different time points representing an expiratory state (Time Point 1) and an inspiratory state (Time Point 2). As highlighted by the yellow dashed lines, the MRSIGMA images show similar respiratory positions with the corresponding ground-truth images. Corresponding cine images for all the time points are shown in Supporting Information Video 1.

Figure 5.

(a) Respiratory motion signal extracted from the synthesized stack-of-stars k-space. The respiratory motion matches well with the underlying projections. (b-c) Comparison of phantom ground-truth images with corresponding images generated using MRSIGMA in axial and coronal planes at two different time points representing an expiratory state (Time Point 1) and an inspiratory state (Time Point 2). The yellow dashed lines highlight respiratory positions at the two time points that are similar between the MRSIGMA images and corresponding ground-truth images.

In-Vivo Studies

The average XD-GRASP reconstruction time to generate the database of motion states, using our preliminary implementation in MATLAB, was 73.82±7.10 minutes. The imaging latency for the online matching step, averaged over all the datasets, was 289.9±3.1ms, including the acquisition of online motion signature (138ms) and the matching process (151.9±3.1ms). Figure 6a shows four representative motion signals extracted from the short scans and Figure 6b shows motion signals extracted from the two long scans superimposed on the underlying projections. Although irregular breathing was observed in the motion signals, all the projections and motion signals do not show significant respiratory drift. Figure 7 compares the online-generated 3D images in two different motion states/time points (the right two columns, generated using the online matching algorithm) with corresponding real-time x-z projections (the left column, generated from online signature data as shown in Figure 4) for one patient dataset. The liver dome can be visualized in the real-time 2D projections, which provided an indirect reference to validate the online-generated real-time 3D images. Figure 8 shows the same comparison in another patient dataset, which also suggests that the motion displacement in the online-generated real-time 3D images matches well with that in the real-time 2D projections. Suspected lesions from this patient, as indicated by the green arrows, can be clearly visualized in the 3D images in both coronal and sagittal planes. Corresponding images in the axial plane are shown in Supporting Information Figure S2, in which green arrows highlight the lesions.

Figure 6.

Four representative motion signals extracted from the short scans (a: TA=138 seconds) and two motion signals extracted from the two long scans (b: TA=8:30 minutes) superimposed on the underlying projections. Irregular breathing was observed in the motion signals, but all the projections and motion signals do not show significant respiratory drift. TA: total acquisition time.

Figure 7.

Comparison of x-z 2D projections with corresponding 3D images obtained with MRSIGMA at different motion states in one patient. The x-z 2D real-time projections, which served as the online motion signature data, were treated as reference to validate the motion pattern in MRSIGMA. This example shows that MRSIGMA was able to generate high-resolution 3D images in real time, with its motion pattern well-correlated with the reference 2D projections (yellow dashed lines).

Figure 8.

Comparison of x-z 2D projections with corresponding 3D images obtained with MRSIGMA at different motion states in another patient. Same as in Figure 5, MRSIGMA was able to generate high-resolution 3D images with a motion pattern well-correlated with the reference 2D real-time projections (yellow dashed lines). Green arrows indicate two liver lesions that can be well visualized in both coronal and sagittal planes.

Supporting Information Video 2 shows the cine images representing the real-time liver motion displacement in different orientations obtained with MRSIGMA. Here, the rotation of corresponding real-time 2D projections are flicking because they were acquired with a golden-angle rotation scheme. Supporting Information Video 3 shows the same cine images sorted according to rotation angles, comparing the real-time liver motion displacement between MRSIGMA and re-sorted x-z projections. While it is hard to see detailed image information in the 2D x-z projections, they only served as online motion signature data and can be used to validate motion displacement in corresponding 3D high-resolution images. The average distance of the liver dome to the top edge of the FOV was 30.83±13.21 mm and 29.47±12.41 mm for the real-time 2D projections and the online-generated real-time 3D images, respectively. Corresponding linear correlation plot was shown in Figure 9, with a slope of 0.92, an intercept of 1.26 mm and an R-square of 0.948, suggesting excellent correlations of motion displacement of the liver dome, thus validating the accuracy of MRSIGMA. Figure 10 shows the results of 10-phase XD-GRASP performed on the two datasets acquired for 8.5 minutes.

Figure 9.

Linear correlation plot to assess the correlation of motion displacement (unit: mm) measured from online-generated 3D images and x-z 2D real-time projections. The slope of the plot was 0.92 and the intercept was 1.26 mm with an R-square of 0.948, suggesting excellent correlation of the motion displacement measured in the two types of images.

Figure 10.

Respiratory-resolved images (10 phases) reconstructed from the two 8.5 minutes datasets (one volunteer and one patient). Yellow arrows highlight several tumors in the patient images.

DISCUSSION

In this work, a novel real-time volumetric imaging technique called MRSIGMA is proposed for real-time volumetric imaging with low imaging latency (~300ms). MRSIGMA consists of two components, one for offline learning of motion states and signatures, and the other one for online matching. Different from prior 4D MRI techniques that rely on highly-accelerated data acquisition and faster reconstruction speed to reduce imaging latency, MRSIGMA shifts the long data acquisition and computationally expensive reconstruction to an offline learning step, leaving a signature-only acquisition and a simple matching operation for online matching to minimize imaging latency. The main application of MRSIGMA is targeted for adaptive radiation treatment using an integrated MR-Linac system, where offline learning can be performed prior to the actual beam delivery (e.g., pre-beam imaging). Once the offline learning process is completed, the online matching step can be performed to generate 3D images with an imaging latency of ~300ms, or less with further optimization, for adaptive beam delivery in real time.

Golden-angle radial acquisition and XD-GRASP image reconstruction are the two essential components in the MRSIGMA framework. On one hand, the golden-angle rotation scheme allows for a free-breathing continuous data acquisition over multiple respiratory cycles and flexible data sorting for reconstructing a motion-resolved dynamic image-set that serves as the database of offline motion states. Although only a stack-of-stars acquisition was demonstrated in this work, the concept of MRSIGMA can easily be extended to other golden-angle sampling schemes such as golden-angle Cartesian (38) or 3D “kooshball” golden-angle radial sampling (39,40). Compared to the stack-of-stars sampling, the “kooshball” geometry may be better suited for MRSIGMA, where isotropic volumetric coverage and isotropic spatial resolution can inherently be acquired to extract isotropic 3D motion information, thus allowing better delineation of the tumor during respiration. On the other hand, the use of XD-GRASP reconstruction can reduce the amount of data needed for generating motion states, so that the offline learning step can be performed more efficiently. However, careful selection of acceleration rates and optimization of the reconstruction algorithm is important to avoid over-smoothing of the motion to be reconstructed. Meanwhile, XD-GRASP reconstruction is an iterative process with heavy computational burden, and thus an optimized implementation of the reconstruction algorithm using a dedicated GPU server for fast reconstruction speed is highly-desired for MRSGIMA. In particular, XD-GRASP may benefit from recent advances in deep learning-based radial MR image reconstruction (41–43), which can enable reconstruction of accelerated MR data in the order of seconds.

The numerical abdominal phantom study was included as a first proof-of-concept demonstration for MRSIGMA. In particular, since a numerical phantom provides access to ground-truth images that are not available in-vivo, it is our goal to combine numerical phantom and in-vivo results (using 2D real-time projections as a reference) to demonstrate the initial feasibility of MRSIGMA. However, a single phantom simulation is insufficient for validating the precision or accuracy of MRSIGMA, given that a single phantom simulation cannot mimic patient-dependent breathing changes. In future work, we plan to further explore the performance of MRSIGMA with respect to simulation of a variety of realistic breathing patterns using the numerical phantom.

In this study, 10 motion states were generated during the offline learning step, which was because of a few reasons. First, the feasibility of MRISIGMA was demonstrated by retrospectively using prior radial liver datasets that were acquired for only 2 minutes and 18 seconds (1000 spokes in each partition and 35 acquired partitions). Thus, the number of reconstructed motion phases was limited to avoid excessive undersampling in each motion state. In practice, we expect that the data acquisition for offline learning can last for 5–8 minutes, so that respiratory motion can be better resolved with an increased number of motion states (e.g., 20 or 30 motion states) if needed. Meanwhile, the recently proposed GRASP-Pro reconstruction (44) may be further applied to improve reconstructed image quality with high temporal resolution. Second, the generation of 10 motion states was also based on the fact that 10 motion phases are typically used as a standard in 4D CT based radiation treatment planning. More than 10 phases in 4DCT would require an overly tight pitch and increased acquisition time to obtain appropriate data in each phase which may potentially increase radiation dose. Finally, increasing the number of respiratory phases may be counter-productive at the first glance, due to potential contouring uncertainty and increased burden on physicians who need to contour each motion phase. However, the development of advanced image processing algorithms that can automatically propagate the contour defined in one phase to others can alleviate this limitation.

With current MRI technology, it is significantly challenging to acquire fully-sampled 3D images with high spatial and temporal resolution for validation of MRSIGMA. Thus, in addition to the numerical phantom demonstration, the validation of MRSIGMA was performed indirectly using real-time 2D x-z rotating projections as our reference, where each 2D projection was obtained by performing a 2D FFT on each rotating radial stack that was used as online matching signature data. Despite reduced spatial information on these projections, they provide respiratory motion of large organs, such as the liver, in real time and can be treated as an indirect ground truth. Our quantitative analysis and comparison showed that the motion estimated from online-generated real-time 3D images is in a good agreement with that estimated from the real-time 2D projections. Meanwhile, these 2D projections can also provide certain spatial information for monitoring any change of the motion pattern in real time. Since these datasets were retrospectively used in this study, the online matching acquisition was also performed with a golden-angle rotation. A better implementation of MRSIGMA using stack-of-stars sampling would be to employ golden-angle rotation only for the offline learning and radial stacks with a fixed angle for online matching, so that corresponding 2D projections can provide a more consistent display of the respiratory motion for real-time monitoring and for evaluating the accuracy of motion tracking. Moreover, instead of acquiring a series of 2D projections, the online matching acquisition can be further improved by acquiring a 1D projection oriented along the head-to-foot direction for generating motion signature and a slice-selective coronal 2D image centered at the target location for each time point, so that the delineation of the target can be improved, and the monitoring of motion can be more reliable.

One challenge for MRSIGMA is the influence of inconsistencies between the offline learning step and the online matching step. The first potential inconsistency is the presence of motion baseline drifts and shifts. Baseline drift can occur as a patient gets more relaxed on the treatment table after initial setup. Baseline shift can be caused by patient movement during treatment or a change in breathing patterns when a patient takes a deep breath at any point during their treatment delivery process but the tumor does not come back to the original position. Although we did not notice large baseline motion drifts in our small liver datasets cohort, they can occur more often in the lung compared to the liver. However, MRSIGMA would allow for real-time monitoring of the breathing pattern in a patient based on the detected respiratory motion signal, and potentially based on online 2D projections or 2D images as described above. Therefore, a few potential strategies can be employed for overcoming this challenge. For example, if a minor drift is observed because a patient gets nervous, we can wait some time for the patient to relax before we start the beam delivery. If the drift remains or gets worse, it may be appropriate to re-learn the motion states. However, this will prolong the overall treatment procedure and fast reconstruction of motion states (e.g., using deep learning approaches) would be essential for completing this efficiently. These strategies can also be used to handle changes in breathing patterns from offline learning to online matching. The second potential inconsistency is patient movement during the treatment process. Depending on the direction of patient movement, this may not always be reflected in the detection of motion signal. Therefore, the use of 2D images as motion signatures can be useful for providing real-time monitoring of patient movement. Additional studies to investigate the influence of these inconsistencies in our proposed framework and these potential solutions would be necessary to further validate the performance of MRSIGMA. The third potential inconsistency is the deformation of the bowel or stomach, which in turn can cause deformation in other organs such as the pancreas. However, such kind of deformation is expected to be reduced in the liver, which is our primary target application in this study.

This work has several limitations that require discussion. First, given the difficulties of obtaining real-time high-resolution 3D images, we did not have a ground truth for in vivo validation. As a result, indirect validation was performed against manually measured displacements in the x-z 2D projection profiles that were used as online matching data. Although these measurements showed good correlations with the online generated real-time 3D images, further validation of the accuracy of MRSIGMA in a large patient cohort would be needed, particularly to test the influence of different respiratory patterns. Second, stack-of-stars sampling led to lower slice resolution, and interpolation was used to create 3D images with pseudo-isotropic voxel size for display purpose. This can be addressed by using a kooshball-based 3D golden-angle radial sampling trajectory at the expense of prolonged data acquisition and higher reconstruction burden in the offline learning step. Third, the respiratory motion signal was extracted using a previous described algorithm (28) that involves normalization. Thus, the motion signal does not provide quantitative motion amplitude and can only be used for data sorting. However, as demonstrated in the in-vivo analysis, motion tracking can be performed in the online-generated real-time 3D images, so that quantitative motion amplitude can be obtained retrospectively. For efficiently performing this task, an automated tracking algorithm would be important to avoid manual labelling. Fourth, MRSIGMA was implemented entirely in MATLAB with slow computational speed. Code optimization, for example in C++, and/or the use of deep learning techniques, can significantly increase computational efficiency and thus further reduce the overall imaging latency for improved real-time beam adaptation in an MR-Linac system. Fifth, MRSIGMA relies on the assumption that respiratory motion is periodic or pseudo-periodic, so that motion states can be learned from data acquired over multiple respiratory cycles. As a result, the existing framework may need to be modified for organs/tumors in the low abdomen, which can be affected by irregular motion (e.g., bowel movement).

Conclusion

This work proposes a novel low-latency 3D MRI technique called MRSIGMA, which pre-learns motion states and signatures in an offline step, and then efficiently generates real-time high-resolution 3D images during online matching. Compared to conventional rapid volumetric MRI approaches, MRSIGMA shifts the time-consuming data acquisition and image reconstruction burden to offline learning and leaves simple operations for online matching, which dramatically reduces the latency for real-time capabilities. MRSIGMA can be potentially applied to the integrated MR-Linac for real-time motion tracking and adaptive MR-guided radiation treatment.

Supplementary Material

fS1-S3

Supporting Information Figure S1 Generation of real-time 2D projections for validating the accuracy of MRSIGMA in-vivo. The projections were generated by applying a 2D inverse FFT on the online signature data (2D planes in k_x-k_z dimension). Despite reduced spatial information on these projections, they provide respiratory motion of large organs, such as the liver, during real time and can be treated as a pseudo ground truth.

Supporting Information Figure S2 Three slices of MRSIGMA images from the same patient as Figure 8 at two different time points. Green arrows indicate liver lesions.

vS1-S3

Supporting Information Video 1 Cine images comparing the ground-truth images with MRSIGMA images in a numerical abdominal phantom simulation.

Supporting Information Video 2 Cine images representing the real-time liver motion displacement in different orientations obtained with MRSIGMA. The corresponding real-time 2D x-z projections are flicking because they were acquired with a golden-angle rotation scheme.

Supporting Information Video 3 Cine images sorted according to rotation angles, comparing the real-time liver motion displacement between MRSIGMA and re-sorted x-z 2D projections. While it is hard to see detailed image information in the 2D x-z projections, they only served as online motion signature data and can be used to validate the motion displacement in the corresponding 3D high-resolution images.