Real-Time Magnetic Resonance Imaging

Abstract

Real-time magnetic resonance imaging (RT-MRI) allows for imaging dynamic processes as they occur, without relying on any repetition or synchronization. This is made possible by modern MRI technology such as fast-switching gradients and parallel imaging. It is compatible with many (but not all) MRI sequences, including spoiled gradient echo, balanced steady-state free precession, and single-shot rapid acquisition with relaxation enhancement. RT-MRI has earned an important role in both diagnostic imaging and image guidance of invasive procedures. Its unique diagnostic value is prominent in areas of the body that undergo substantial and often irregular motion, such as the heart, gastrointestinal system, upper airway vocal tract, and joints. Its value in interventional procedure guidance is prominent for procedures that require multiple forms of soft-tissue contrast, as well as flow information. In this review, we discuss the history of RT-MRI, fundamental tradeoffs, enabling technology, established applications, and current trends.

REAL-TIME MAGNETIC RESONANCE IMAGING (RT-MRI) enables rapid and continuous acquisition of images that allows visualization of dynamic processes as they occur. RT-MRI does not rely on any gating, synchronization, or repetition of the underlying movement or contrast dynamics. The quality of RT-MRI has experienced major leaps in the past 30+ years due to advances in MRI technology, including fast switching gradients, array receiver coils, and advanced reconstruction including parallel imaging, compressed sensing, and artificial intelligence. Over this same time window, RT-MRI has earned a substantial role in both diagnostic imaging and in the image guidance of invasive procedures. Diagnostic RT-MRI has proven most valuable in areas of the body that undergo substantial and irregular motion, such as the heart (eg, arrhythmia), upper airway vocal tract (eg, speech production), joints (eg, instability), and gastrointestinal system (eg, motility). RT-MRI for interventional guidance has proven most valuable for procedures that require multiple forms of soft-tissue contrast, as well as flow information (eg, right heart catheterization).

Historically, MRI has had a reputation of being a “slow” modality, especially compared to x-ray, computed tomography (CT), and ultrasound. This perception is changing. On modern commercial MRI equipment, RT-MRI is now feasible, practical, and readily available. It is compatible with most MRI sequences, and notably includes proton density (PD) and T₁-weighted (T₁w) spoiled gradient echo (GRE), balanced steady-state free precession (bSSFP), and single-shot rapid acquisition with relaxation enhancement (RARE). RT-MRI can provide more than adequate spatiotemporal resolution, contrast-to-noise efficiency, and image quality for a wide array of applications.

The purpose of this review is to summarize current state-of-the-art RT-MRI technology and clinical applications, predominantly focusing on imaging of dynamic motion. We begin with a discussion of the history of RT-MRI and fundamental tradeoffs. We then review enabling technology, which includes hardware, acquisition, reconstruction, interaction, and postprocessing. Next, we review the most common applications, including cardiac, interventional, upper airway, and musculoskeletal. Finally, we discuss current trends, including the use of machine learning and the use of high-performance low-field MRI systems.

There has been some recent debate regarding the nomenclature for RT-MRI.^1,2 In this article, RT-MRI will refer to real-time acquisition that does not use any gating, synchronization, or repetition of the movement. We use the term interactive RT-MRI when latency between acquisition and image display is short enough to permit interaction (eg, guidance of interventions, or calculation of real-time cardiac output).

Brief History

Real-time techniques are commonly used in medical imaging, including live video for gait analysis, fluoroscopy for diagnostic studies of the gastrointestinal tract, and ultrasound to guide interventional procedures such as biopsies. Development toward fast MRI began shortly after the first medical applications. This included the development of echo planar imaging (EPI) by Sir Peter Mansfield in 1977,³ as well as introduction of fast low angle shot magnetic resonance imaging (FLASH) by Frahm et al in 1986,⁴ and fast spin-echo imaging (RARE) by Hennig et al in 1986.⁵ Early clinical applications of real-time imaging include MR fluoroscopic images of the head by Farzaneh et al in 1989,⁶ followed by techniques to interactively control scan slice orientation and image contrast by Holzinger et al in 1990,⁷ and real-time flow measurements by Riederer et al in 1991.⁸

More recent developments have enabled substantial progress in MRI acquisition speed, which are discussed below. There has been tremendous growth in the number of publications on RT-MRI, as shown in Fig. 1. Many of the large vendors have adopted the use of interactive RT-MRI for localization and scan plane/volume prescription. And several diagnostic and interventional applications have developed and matured.

FIGURE 1:

Publications involving RT-MRI. PubMed search: ((“real-time MRI”) OR (“real-time NMR”) OR (“real-time magnetic resonance”) OR (“real-time interactive MRI”) OR (“RT-MRI”)).

Fundamental Tradeoffs

MRI in general must balance a tradeoff between spatial resolution, temporal resolution, signal-to-noise ratio (SNR), artifacts, reconstruction latency, and modeling assumptions. This tradeoff is put to the test in RT-MRI, where temporal resolution is at a premium. Figure 2 shows a scatterplot of spatial resolution (x-axis) vs. temporal resolution (y-axis) from 22 recent publications that utilize state-of-the-art 2D RT-MRI techniques. These publications, with acquisition details listed in Table S1, were selected to include a diversity of imaging methods and target applications, and to include work from different laboratories both with and without specialized hardware and software capability. The general tradeoff is illustrated by a gray shaded bar, with fine spatial resolution and coarse temporal resolution on the upper left (eg, 1×1 mm² with 80 msec temporal resolution, 12.5 fps), and coarse spatial resolution and fine temporal resolution on the right (eg, 3.5×3.5 mm² with 20 msec temporal resolution, 50 fps). Deviations from this line occur because the data span several different applications, reconstruction methods, coil geometries, field strengths, and modeling assumptions. The impact of these is discussed in detail in the Enabling Technology section below. In general, moving towards the lower left requires using more advanced technology, advanced modeling, and increased computational demand, or accepting worse SNR and/or more severe image artifacts.

FIGURE 2:

Scatterplot of 2D RT-MRI spatial and temporal resolution. Spatial resolution (x-axis) vs. temporal resolution (y-axis) is plotted from 22 recent publications that utilize state-of-the-art methodology, as selected by the authors of this review, summarized in Table S1. The gray shaded bar indicates the general spatiotemporal resolution tradeoff. All substantial deviations are due to variations in the FOV, use of parallel imaging, use of reconstruction constraints, and minimum acceptable SNR.

Enabling Technology

RT-MRI is made possible by several system components that are included in most modern high-end clinical MRI systems. These beneficial components are discussed in this section. Items that are nonstandard are clearly identified.

Hardware

The speed of MRI is fundamentally limited by nuclear magnetic resonance (NMR) relaxation, and the time required to cover k-space. RT-MRI therefore relies on time-efficient spatial encoding. One major technological advance has been high-fidelity fast-switching shielded gradients, with eddy-current precompensation. Modern systems offer this with gradients that can produce ≥40 mT/m amplitude and ≥150mT/m/msec slew rate on each physical axis with a 50–100% duty cycle. This enables GRE and bSSFP contrast with very short repetition times and enables efficient k-space sampling (eg, spiral or echo-planar). Spiral and EPI trajectories were among the first technical advances to substantially broaden the set of applications that could be addressed with RT-MRI.⁹

RT-MRI also benefits from the use of parallel imaging, which requires carefully designed receiver coil arrays that provide diverse coil sensitivity along the likely directions of spatial encoding and k-space undersampling. These coils should preferably preserve body-noise dominance.¹⁰ Several MRI manufacturers and third-party vendors manufacture such coils and optimized cardiac and torso coils are readily available. However, coils for nascent applications such as speech^11,12 and dynamic musculoskeletal imaging are not yet standardized, and there is substantial room for development.

Interactive RT-MRI requires the use of a flexible programmable console that can allow “on-the-fly” changes to the scan plane, shim, and many other imaging parameters. Several modern systems offer this capability, wherein many aspects of a pulse sequence can be modified without substantial interruption (ie, within a few milliseconds).

Data Acquisition

High frame-rate RT-MRI has been fundamentally enabled by two gradient echo sequences: GRE and bSSFP (Fig. 3a). RF-spoiled and gradient-spoiled GRE sequences are robust to artifacts and provide T₁-weighted contrast with a short repetition time (TR). bSSFP sequences¹⁴ provide higher SNR efficiency than GRE, and provide T₂/T₁ contrast, which is extremely advantageous in cardiac imaging because of the excellent blood–myocardium contrast (Fig. 3a, right). However, bSSFP has two important limitations. One is sensitivity to off-resonance, which manifests as banding artifacts.¹⁵ bSSFP RT-MRI is often used at ≤ 1.5T or at 3T with careful shimming over the region of interest (ROI) and with the shortest possible TR. The second issue is a transient approach to steady state, which can be problematic if the application requires frequent switching of the scan plane or volumes, or if there is flow or motion through regions that experience the banding artifact. Transient signal oscillations can be easily mitigated using catalyzation preparation schemes; however, it still takes time to reach steady-state contrast.

FIGURE 3:

Common sequences, sampling trajectories, and view orders used in 2D RT-MRI. (a) Sequence diagrams of spoiled GRE and bSSFP, and the steady-state signal amplitude as a function of off-resonance Δf; Simulation parameters: TR = 5 msec; flip angle = 5° for spoiled GRE; flip angle = 60° for bSSFP; myocardium T₁/T₂ = 950/50 msec; blood T₁/T₂ = 1500/250 msec (representative of 1.5T). (b) Non-Cartesian sampling trajectories of undersampled radial, single-shot spiral, and single-shot EPI. (c) View orders of multishot spiral of conventional 13-interleaf bit-reversed and golden-ratio, and unaliased FOV as a function of the number of interleaves [Reproduced from Ref. (13)].

RT-MRI has been made possible by efficient k-space sampling trajectories (eg, spiral and EPI, described in the preceding section) and clever temporal undersampling schemes. Although it is inefficient, 2DFT imaging may be used in both 2D and 3D imaging due to the simplicity of reconstruction and robustness to artifacts. A wide range of $\stackrel{⃑}{k}$ −t undersampling schemes exists for accelerated 2DFT imaging.^16–21 For example, one can use variable-density pseudo-random $\stackrel{⃑}{k}$ −t sampling, which creates incoherent aliasing artifacts in a certain transform domain that can be resolved by advanced reconstruction algorithms.

Figure 3b illustrates more efficient sampling trajectories. EPI³ is an alternative where multiple Cartesian lines are acquired after each excitation, therefore k-space can be filled only with one or fewer repetitions. However, its long readout time makes it vulnerable to ghosting and distortions from gradient waveform inaccuracy or off-resonance and near-RT correction methods^22,23 have been proposed.

Most preferred are radial and spiral samplings when higher spatial and/or temporal resolution is desired. Both spiral and radial sampling naturally oversample the center of k-space and this offers motion robustness and tolerance to undersampling. While radial sampling is π/2 less efficient than Cartesian, its motion robustness and tolerance to undersampling make it a popular RT-MRI acquisition method.²⁴ With moderate angular undersampling along with the golden angle scheme,²⁵ the streaking artifacts are usually mild in appearance and incoherent over time. Alternatively, spirals can be very efficient methods to cover k-space. Single-shot spirals can completely cover k-space, but the resultant long readout time increases sensitivity to off-resonance, resulting in spatial blurring. Multishot spiral acquisitions with a short readout time can be used to alleviate blurring. Both radial and spiral imaging can be accelerated with the use of $\stackrel{⃑}{k}$ −t sampling strategies such as undersampling and random angle order schemes, for example bit-reversed⁹ or golden angle^13,25 (Fig. 3c). For instance, radial or spiral imaging can be performed with an angle incremented by the golden angle (222.5°)²⁵ or tiny golden angles.²⁶ This, when undersampled, produces relatively incoherent aliasing in the spatial and temporal domain (or in a transform domain), which is well suited for advanced reconstruction algorithms. The view order is an added variable and the golden angle scheme is widely used, as it additionally has the retrospective field-of-view (FOV) tradeoff ability as opposed to the conventional scheme (right, Fig. 3c).

RT-MRI has also benefited from 2D multislice imaging,^11,27–31 which can be performed by utilizing time-interleaved sampling of acquisition schemes described above with a corresponding reduction in spatial or temporal resolution by the number of slices. Alternatively, simultaneous multislice imaging³² can be utilized to accelerate data acquisition. Especially, controlled aliasing in parallel imaging results in higher acceleration, also known as CAIPIRINHA,^33,34 has shown substantially reduced aliasing artifacts. This technique has recently been explored in some RT-MRI studies.^35–37 3D imaging has also been explored by extending 2D acquisition schemes to 3D or by using novel 3D sampling trajectories. Radial or spiral can readily be extended to 3D stack-of-stars³⁸ or 3D stack-of-spirals^39,40 by adding phase encoding steps along k_z direction. Echo-volume imaging,⁴¹ a 3D extension of EPI, can achieve time-efficient 3D imaging and has primarily been investigated for brain functional MRI.⁴² Combinations of two sampling schemes for 3D imaging has also been explored, including EPI⁴³ and Cartesian sampling,⁴⁴ each combined with golden–angle radial sampling. There has also been interesting literature where volumetric image can be efficiently obtained by RT-MRI acquisitions with automatic advancement of the slice position.⁴⁵ While 3D acquisitions allow for a more flexible $\stackrel{⃑}{k}$ −t sampling strategy and therefore much redundant information to be exploited along the additional dimension, the increased amount of data is challenging in terms of data processing and reconstruction.

Reconstruction

RT-MRI data sampling and reconstruction varies with application. This is because there is a trade-off between temporal resolution and reconstruction time (governed by the complexity of the algorithm). The simplest reconstruction techniques use data sharing strategies, for example, keyhole or sliding window reconstruction.⁴⁶ This simplicity enables low latency reconstruction; however, the true temporal resolution is coarse.

Higher levels of acceleration are achievable by undersampling; however, zero-filled reconstruction results in spatial aliasing that renders the images clinically unusable. One can use parallel imaging to recover usable images, which can be performed in the image domain; sensitivity encoding (SENSE),⁴⁷ or in k-space; generalized autocalibrating partial parallel acquisition (GRAPPA).⁴⁸ Parallel imaging of Cartesian data enables very low latency reconstructions; hence, its popularity in RT-MRI interventional applications.

Reconstruction of undersampled non-Cartesian data is substantially more complex. This is because the k-space points must be resampled onto a Cartesian grid in order to use the fast Fourier transform (FFT), which increases computation demand.⁴⁹ Additionally, in non-Cartesian SENSE each voxel in the image domain can potentially alias with all of the other voxels, resulting in the need for time-consuming iterative reconstructions.⁵⁰ In non-Cartesian GRAPPA the irregular gaps in k-space result in the need for geometry-specific GRAPPA weights,^51,52 requiring large amounts of calibration data. These drawbacks often restrict the use of non-Cartesian parallel imaging to applications where real-time visualization is not necessary. One exception is through-time GRAPPA, where multiple fully sampled non-Cartesian datasets are acquired and are used to learn the location-specific GRAPPA weights.^53–55

The level of acceleration achievable using parallel imaging is, in theory, equal to the number of independent coil elements along the direction of undersampling. In practice, acceleration rates are often limited to 2–3 using Cartesian trajectories, and 3–4 using non-Cartesian trajectories. Higher levels of acceleration can be achieved by combining temporal and spatial encoding schemes. These techniques often lever-age the fact that the MR data are sparse in $\stackrel{⃑}{x}$−f space, including; $\stackrel{⃑}{k}$ −t broad-use linear acquisition speed-up technique (BLAST), $\stackrel{⃑}{k}$−t SENSE, $\stackrel{⃑}{k}$−t GRAPPA.⁵⁶ In these schemes, the reconstruction is constrained using some prior information that can be used to determine the ground truth. These techniques often preclude real-time reconstruction, as this prior information is extracted from the data itself,⁵⁷ or as part of a prescan.

Even higher acceleration factors are possible using constrained reconstructions,⁵⁸ compressed sensing,⁵⁹ and regularized nonlinear inversion (NLINV).⁶⁰ These methods rely on object models, such as sparsity in a known transform domain, along with data sampling that produces incoherent aliasing in the sparsity domain. Some of the most popular sparsifying transforms for RT-MRI are finite differences, total variation, wavelet, and Fourier transform, which can be applied to exploit spatial and/or temporal sparsity. This category of methods enables acquisition of exceptionally high temporal and/or spatial resolution; however, they rely on nonlinear reconstruction that is computationally expensive, resulting in high reconstruction latency that often limits clinical adoption.

There has been substantial work towards reducing image reconstruction times. This includes the use of coil selection⁶¹ and coil compression⁶² techniques. Modern graphics processing units (GPUs) enable improvements in latency and throughput, through their massively parallel architecture,^63–65 and have enabled up to 27-fold reduction in latency compared to conventional CPUs,⁶⁶ making interactive RT-MRI feasible.

In addition, the efficient k-space trajectories often used in RT-MRI are susceptible to artifacts from linear time-invariant gradient distortions, including eddy currents. Correction of these has been shown to be feasible in real time.²³ However, artifacts from concomitant fields^67,68 are more challenging. At present, these require the use of approximations,⁶⁹ or the use of NMR field probes,^70,71 in conjunction with a more sophisticated offline reconstruction.⁷²

Vendor-agnostic raw data formats have been developed to promote reconstruction algorithm sharing and data sharing, for example ISMRM Raw Data (ISMRMRD)⁷³ and Raw Array (RA).⁷⁴ The ISMRMRD format is designed to support simultaneous streaming of raw data, pulse sequence waveforms, and physiology waveforms, to image reconstruction or processing software.⁷⁵ This streaming capability is important for fast reconstruction of real-time images.

Postprocessing

Offline processing, including segmentation, parameter quantification, and distortion correction, of RT-MRI images is quite similar to traditional offline processing of CINE imaging or even static imaging. However, there are many scenarios where inline postprocessing adds value, particularly for interactive RT-MRI. The unique features of these methods in RT-MRI are that they typically have to tolerate lower image SNR and have to satisfy maximum latency requirements. Inline segmentation has been shown to aid monitoring of ventricular function during cardiac interventions.⁷⁶ Inline flow quantification has been shown to expedite comprehensive cardiac examinations.⁷⁷ And inline off-resonance artifact correction (deblurring) has been shown to substantially improve the sharpness of speech articulator depiction.⁷⁸

Interaction/Visualization

RT-MRI applications often benefit from the synchronization with other complementary real-time inputs. For cardiopulmonary testing, physiological monitoring of ECG, respiration, and expiratory gases may be synchronized with real-time cardiovascular imaging.⁷⁶ MRI-guided catheterization requires electrophysiological recording of high-fidelity ECG waveforms, which can be challenging within the MRI environment, and hemodynamic recording of invasive pressure waveforms.^79,80 Real-time speech imaging requires synchronization of audio signals with imaging data, which can be achieved with commercial products (eg, FORMI II₊, Optoacoustics, Israel).^24,81

Additionally, for some applications user interaction is required to modify parameters during imaging using real-time feedback to the scanner. Most notably, for real-time MRI-guided intervention the modification of slice position, slice thickness, slice orientation, image contrast, frame rate, and device imaging modules can each be toggled and modified interactively without a pause in the continuous stream of real-time imaging. Interactivity requires that images are displayed and manipulated in real-time, and that pulse sequence parameters are accessible to be modified on-the-fly.

Major MRI vendors have prototype or product graphical user interfaces for interactive imaging (eg, Monte Carlo prototype, Siemens Healthcare, Erlangen Germany; iSuite, Philips, Best, The Netherlands; iDrive and MR Echo, GE Healthcare, Waukesha, WI). Interactive imaging platforms are also available through independent MRI software vendors, most notably the RTHawk platform (Heart Vista, Los Altos, CA).⁸² This platform is MRI vendor agnostic as long as a fully flexible stub sequence is available, but to date has been primarily developed on the GE platform. This system is compatible with interactive scan plane modification using a six degree-of-freedom 3D mouse.⁸²

Auxiliary equipment can also be used inside of the MRI suite for interactive modification of imaging. MRI-guided catheterization procedures use foot-pedals to leave interventionist hands available for device manipulation, which mimics traditional x-ray catheterization suites. Other auxiliary equipment including computer mice, keyboards, and tablets have been explored for interactive imaging,⁸³ as well as gesture-based scan control.⁸⁴ Furthermore, augmented reality and virtual reality equipment⁸⁵ may be attractive for interactive visualization of real-time imaging in the future.

Applications

RT-MRI benefits a broad range of diagnostic and interventional applications. Here we summarize the application-specific needs, imaging considerations, and the impact to date.

Cardiac

RT-MRI enables imaging of the cardiovascular system without the need for cardiac gating or respiratory compensation. This is particularly valuable in patients with cardiac arrhythmia^86,87 where cardiac gating fails (~10% of patients referred for diagnostic cardiac imaging), and in patients who find breath-holding difficult (~10% of patients). It is also extremely valuable in children with congenital heart disease (CHD), where it can be used to lessen the need for sedation⁸⁸ and its associated risks.

Real-time assessment of cardiovascular structures requires relatively high spatial and temporal resolution to ensure accurate visualization and quantification. For example, Setser et al⁸⁹ recommend a minimum temporal resolution of 50 msec (20 fps), and spatial resolution of 2 mm for functional RT-MRI of the left ventricle. Real-time ventricular function imaging has been shown to achieve an excellent agreement with reference standard breath-held, cardiac gated techniques, through a combination of undersampled radial trajectories with $\stackrel{⃑}{k}$−t SENSE reconstructions in CHD,⁹⁰ $\stackrel{⃑}{k}$−t SPARSE-SENSE in patients with tachycardia²¹ and myocardial infarction,⁹¹ regularized nonlinear inversion,⁹² and more recently, proof-of-concept studies using machine-learning reconstructions in CHD.⁹³ Figure 4a and Movie S1 contain a representative example of image quality. Spiral trajectories, although less popular, have been combined with compressed sensing to achieve high-resolution imaging in children with CHD.⁹⁶

FIGURE 4:

Illustration of cardiovascular RT-MRI. (a) Real-time cine imaging using tiny-golden angle radial bSSFP sequence at 1.5T, with 12x undersampling and compressive sensing reconstruction (TE/TR = 1.3/2.7 msec, flip angle = 70°, in-plane resolution = 2.1 mm, 32 msec temporal resolution, 31 fps). A movie can be found in Movie S1 [Adapted from Ref. (91)]. (b) Real-time PCMR using perturbed spirals at 1.5T, with 18x undersampling and compressive sensing reconstruction. Top: Magnitude images, Bottom: Phase images (TE/TR = 1.9/6.7 msec, VENC = 200 cm/s, flip angle = 20°, in-plane resolution = 1.8 mm, 27 msec temporal resolution, 37 fps). A movie can be found in Movie S2 [Adapted from Ref. (94)]. (c) Real-time imaging of the fetal heart (shown by arrow in first column) demonstrating gross fetal movement. Golden-angle radial bSSFP sequence at 1.5T, with 27x undersampling and compressive sensing reconstruction (TR = 5.0 msec, flip angle = 70°, in-plane resolution = 1.0 mm, 74 msec temporal resolution, 14 fps). A movie can be found in Movie S3 [Adapted from Ref. (95)].

Quantification of blood flow using phase contrast (PC) requires high spatial resolution to mitigate partial volume effects; Greil et al recommend that it is necessary to have 16 pixels in the cross-section of the vessel of interest to get accurate flow quantification.⁹⁷ PC RT-MRI has been used clinically to assess the respiratory and cardiac components of flow in patients with Fontan circulation,^98,99 to quantify flow in children with cardiac left-to-right shunts,¹⁰⁰ to investigate the effect of elevated intrathoracic pressure on blood flow,¹⁰¹ and in CHD.⁹⁴ Figure 4b and Movie S2 contain a representative example of image quality.

RT-MRI also enables imaging during exercise, which can be used to unmask subtle changes in early cardiovascular disease.¹⁰² Studies have shown the ability to measure ventricular volumes during exercise; to assess the effect of percutaneous pulmonary valve implantation,¹⁰³ to assess the mechanisms which augment cardiac output with exercise in repaired tetralogy of Fallot,¹⁰⁴ and to unmask right ventricular dysfunction in pulmonary arterial hypertension.¹⁰⁵ Real-time flow during exercise has also been used in patients with total cavopulmonary connection¹⁰⁶ and after surgical bypass of aortic arch obstruction.¹⁰⁷ Continuous acquisition of real-time flow during ramped exercise over 10 minutes has been shown,⁶³ as well as a combination with respiratory gas analysis to simultaneously measure peak oxygen consumption (VO₂), enabling assessment of exercise capacity.¹⁰⁸

Fetal cardiac RT-MRI is of interest as a potential secondary tool (after fetal echocardiography) to characterize congenital cardiac malformations.¹⁰⁹ It is one of the most challenging applications of RT-MRI because of the very high heart rates, need for high spatial resolution (tiny hearts), need for interaction (to follow movement of the fetus), and the desire to be conservative with specific absorption rate (SAR) and acoustic noise.¹¹⁰ It has been attempted with some success⁹⁵; however, RT-MRI acquisition with retrospective metric-optimized-gating has provided a promising approach for CINE MRI. Figure 4c and Movie S3 contain a representative example of image quality of RT-MRI in the fetal heart.

MRI-Guided Invasive Procedures

MRI-guidance of diagnostic and therapeutic invasive procedures employ intraprocedural imaging to navigate devices and to assess procedural outcomes. Compared to traditional image-guidance modalities such as x-ray and ultrasound, MRI offers flexible image contrast and 3D imaging capabilities that can improve tissue visualization during a procedure. Real-time MRI is particularly valuable during complex device maneuvers and previous review articles have described the applications of real-time imaging technologies to guide invasive procedures.^111–113 Figure 5 and Movies S4 and S5 contain illustrative examples.

FIGURE 5:

Illustrations of RT-MRI for MRI-guided invasive procedures. Cardiovascular procedures are the most technically demanding for RT-MRI, and therefore are provided. (a) The position and orientation of catheter devices with two embedded microcoils are tracked on a previously acquired 3D volume for an electrophysiology procedure. Real-time device tracing is achieved using 3D gradient echo projection imaging (resolution 0.83 mm, 10 Hz tracking rate) [Reproduced from Ref. (114)]. (b) Interactive RT-MRI used to navigate gadolinium-filled balloon wedge end-hole catheter during diagnostic right heart catheterization (bSSFP, TE/TR = 1.44/2.88 msec, flip angle = 40°, in-plane resolution = 1.8×2.4 mm², GRAPPA rate 2, 200 msec temporal resolution, 5 fps) [Adapted from Ref. (115)]. (c) Real-time MRI thermometry used to calculate thermal dose during therapeutic ablation procedure (gradient echo EPI, TE/TR = 18–20/110 msec, flip angle = 60, in-plane resolution = 1.6×1.6 mm², GRAPPA rate 2, 200 msec temporal resolution, 5 slices/s) [Adapted from Ref. (116)].

Many invasive procedures have been performed in patients in the MRI environment, including MRI-guided biopsy^117,118; radiotherapy¹¹⁹; thermal ablations such as radiofrequency ablation,^120,121 laser ablation, cryoablation,¹²² microwave ablation, and high-intensity focused ultrasound¹²³; chemoablation; drug injection; electrophysiology^114,124,125; and invasive pressure measurements.^80,115,126

MRI-guided invasive procedures have unique requirements. Low-latency reconstruction and in-room image display are essential, such that images can be used for real-time device navigation and procedural decision-making. Interactivity of imaging parameters is critical to control image contrast and frame rate throughout the procedure. Image processing, including image segmentation, registration, and distortion correction, must also be performed on-the-fly.¹¹² During many invasive procedures, devices (eg, needles, catheters, guidewires, sheaths) are imaged concurrently with target anatomy. Device imaging is performed using either “passive” visualization, exploiting the material properties, or “active” visualization, in which devices themselves are RF receivers, designed as loopless antennas for imaging¹²⁷ or microcoils for 3D device tracking¹²⁸ (Fig. 5a, Movie S4). Solenoid and loopless receiver coils positioned on invasive devices can also be applied for small-FOV local imaging in MRI-endoscopy applications.¹²⁹

Most in-bore biopsy procedures of prostate, liver, breast, and brain lesions use standard T₂-weighted, T₁-weighted, diffusion-weighted, or dynamic contrast-enhanced imaging for iterative confirmation of needle placement,^117,118,130 with a few studies employing high frame-rate dynamic needle guidance.^131,132 MRI-guided cardiovascular procedures are the most technically demanding, requiring rapid multiplanar imaging for device navigation. Most commonly, a 2D Cartesian bSSFP acquisition is used, in combination with parallel imaging, to achieve 5–10 frames/sec, with magnetization preparation pulses for contrast variation (Fig. 5b, Movie S5). 2D radial and spiral spoiled gradient echo acquisitions have also been applied for cardiovascular procedures,^133,134 and when combined with regularized nonlinear inversion reconstruction, have achieved a temporal resolution of 42 msec (24 fps) with reconstruction delay of 27 msec.¹³⁵

RT-MRI thermometry, using proton resonance frequency shift imaging, is important to monitor thermal ablations with high temporal resolution. Multislice single-shot EPI acquisitions with real-time image registration methods have been developed for thermometry and dosimetry during cardiac RF ablations^116,136 (Fig. 5c, Movie S6). The geometry of transcranial high-intensity focused ultrasound (HIFU) therapy devices restrict the number of receive coils that can be positioned around the head, and undersampled stack-of-spiral and stack-of-stars EPI thermometry acquisitions^40,43 have been developed for volumetric brain coverage with 75-msec temporal resolution (13 fps). 3D dynamic keyhole imaging has generated high spatiotemporal resolution (1.5×1.5×6 mm³ with 455 msec temporal resolution, 2 fps) imaging for dynamic guidance of radiotherapy.¹³⁷ This work utilizes a super-resolution generative model for high spatial reconstruction from low-spatial and high-temporal resolution images.

The unique set of requirements of MRI-guided invasive procedures continues to motivate the innovation of RT-MRI acquisitions, as well as rapid inline image reconstruction and image processing.

Upper Airway

RT-MRI enables imaging soft-tissue structures and muscles of the upper airway that are coordinated in space and time to perform essential human functions such as speech, respiration, and digestion. RT-MRI is preferred over other imaging and movement-tracking modalities because it allows for observing deep soft tissues such as the velum, pharyngeal wall, and the larynx in the arbitrary imaging plane without radiation or endoscopy. The nature of the movements of the upper airway is not necessarily periodic and is unrepeatable. RT-MRI can now be combined with intermittent tagging pulses to visualize internal deformation in the tongue muscles^138,139 and be exploited to reconstruct 3D tongue shape or model.^140,141 Real-time visualization and interaction are also advantageous for the operator to modify imaging parameters on-the-fly and to ensure subject compliance with stimuli.

Figure 6 and Movies S7–S9 illustrate representative research areas in speech production,^24,144 sleep apnea,^36,37,145 and swallowing.⁶⁰ Imaging is often performed along with synchronized recordings of physiological signals such as audio signals in speech,¹⁴⁶ polysomnography signals used in sleep studies,³⁶ and intraoral pressure sensor used in swallowing¹⁴³ to aid real-time or retrospective analysis. Several clinical applications have also been explored such as velopharyngeal insufficiency,¹⁴⁷ apraxia,^60,148 and postsurgical assessment of glossectomy^148,149 and cleft-palate repair.^150,151 Many other applications and technical aspects of the upper airway imaging have been described in review articles with focuses on speech,^24,144,152 speech and sleep,¹⁵³ sleep,¹⁴⁵ and image analysis techniques on RT-MRI of vocal tract motion.¹⁵⁴

FIGURE 6:

Illustration of upper airway RT-MRI. (a) Speech production imaging using 13-interleave spiral GRE sequence at 1.5T (TE/TR = 0.8/6.0 msec, flip angle = 15° in-plane resolution = 2.4 mm, 12 msec temporal resolution, 83 fps) [Adapted from Ref. (142)]. (b) Sleep apnea study using simultaneous multislice radial GRE sequence at 3T (TE/TR = 3.7/6.5 msec, flip angle = 5°, slice thickness/gap = 7/3 mm, 3 slices, in-plane resolution = 1 mm, 96 msec temporal resolution, 10 fps) [Adapted from Ref. (37)]. (c) Swallowing imaging of 10-ml pineapple juice using radial FLASH sequence (TE/TR = 1.33/2.10 msec, flip angle = 8°, in-plane resolution = 1.3 mm, 40 msec temporal resolution, 25 fps, 19 spokes) [Adapted from Ref. (143)].

The upper airway imaging generally requires high spatial and temporal resolution, although specific imaging parameters would be dictated by the applications shown in Fig. 1 in Lingala et al.²⁴ For speech, high temporal resolution, below 70 msec (greater than 14 fps) and spatial resolution of no more than 3.5 mm² are typically required to study a broad range of speech events. For sleep, pharyngeal airway motion is relatively slower than vocal tract motion, as it involves closure of the airway, requiring lower temporal resolution than imaging for speech and swallowing. For swallowing, it is valuable to track both the pharyngolaryngeal area and the lower esophageal sphincter and its surrounding area.¹⁵⁵

Current state-of-the-art techniques use non-Cartesian sampling (radial or spiral acquisition) and parallel imaging, combined with constrained reconstruction. This has enabled 2D dynamic images with spatial resolutions of 1.3–2.4 mm² at high temporal resolutions of 10–60 msec (100 to 17 fps) from highly undersampled MRI data.^{11,54,55,143,156–159} Imaging the 2D mid-sagittal plane is the most widely used, as it is most informative thanks to its entire vocal tract coverage from the lips to the glottis given a high temporal resolution. Imaging a few 2D planes,^11,29,30 simultaneous 2D planes,^36,37 or 3D³⁸ at the cost of temporal resolution have also been developed. Recently, 3D stack-of-spiral acquisition demonstrated imaging of the full vocal tract (FOV: 200×200×70 mm³) with high spatiotemporal resolution (2.4×2.4×5.8 mm³, with 72 msec temporal resolution, 14 fps) during natural speech.³⁹

Imaging the upper airway has a unique challenge―tissue surfaces along the upper airway are the main region of interest but are vulnerable to off-resonance effects. Those include the movements of the articulators during speech or swallowing, the closure of the velopharyngeal port in velopharyngeal insufficiency, and the collapse of the pharyngeal airway in sleep studies. Offresonance at the tissue surfaces manifest as blurring or signal loss with non-Cartesian sampling such as spirals and/or appear as banding artifact when bSSFP sequences are used. Careful shimming is usually carried out with a focus on the tissue boundaries. Current RT-MRI studies for speech production are most often conducted using short-duration spirals (2.5 msec) and/or at lower field strength (1.5T or lower) MRI scanners, or in conjunction with field inhomogeneity reconstruction^160–162 or, more recently, artificial intelligence (AI)-based deblurring.⁷⁸

Musculoskeletal

Musculoskeletal RT-MRI is used to reveal abnormal joint biomechanics, which are linked to joint disorders, usually in the context of pain, instability, or movement restriction. For example, abnormal knee kinematics is known to contribute to the development and progression of osteoarthritis. Common musculoskeletal applications include the knee, wrist, spine, temporomandibular joint, foot, and hip,¹⁶³ with a selected few illustrated in Fig. 7 and Movies S10–S12. These have vastly different spatiotemporal resolution and coverage needs.

FIGURE 7:

Illustration of three musculoskeletal RT-MRI applications. (a) Knee RT-MRI highlighting flexion, extension, and the measurement of rectus femoris knee muscle moment arms [Adapted from Ref. (164)]. (b) Wrist RT-MRI illustrating a radial–ulnar deviation maneuver, suitable for measuring dynamics of the scapholunate gap [Adapted from Ref. (165)]. (c) Temporomandibular joint RT-MRI illustrating the ability to track condyle movement during voluntary opening of the mouth [Adapted from Ref. (166)].

Early work used semi-static approaches (multiple static images in fixed postures). Systematic studies have since determined that biomechanical models derived from RT-MRI are substantially different from those derived from semi-static approaches,¹⁶⁷ making RT-MRI the current method of choice. Musculoskeletal movements are voluntary, and can sometimes be performed through controlled dynamics, for example where movements are repeated and synchronized with a metronome or periodic visual stimulus. However, voluntary synchronization is imprecise, subject-dependent, and impractical for patients with pain. Retrospective gating can be applied using additional sensors¹⁶⁸ or self-gating approaches,¹⁶⁹ assuming that the timing variations have a negligible impact on kinematic estimates. Movement can be more precisely controlled by a physical apparatus, but this does not lead to realistic physical effort, which is often needed to reproduce the abnormal movement and/or the symptom being studied (eg, pain). Finally, it is possible to synchronize using external muscle stimulation,¹⁷⁰ which adds experimental complexity. For all of these reasons, there is a need for RT-MRI, and often a need for an appropriate load.¹⁷¹

Musculoskeletal RT-MRI requires strong contrast between muscle, fat, and surrounding fluids, and benefits from high SNR efficiency, making bSSFP the sequence of choice.¹⁷² bSSFP can be augmented with phase-sensitive reconstruction for fat–water separation.¹⁷³ GRE may also be used with optional multiecho fat–water separation.¹⁷⁴ Most studies in the literature were performed at conventional field strengths (1.5T and 3T) in the supine position, which restricts movement and load bearing. For this reason, low-field open-bore, and upright MRI has been explored as an alternative to alleviate these restrictions.¹⁷⁵

In the knee, RT-MRI has provided valuable insight into patellofemoral pain, and used to quantify kinematics,¹⁷⁶ document the differences between the load-bearing and nonloadbearing kinematics,¹⁷⁷ and determine the effects of physical supports.¹⁷⁶ RT-MRI measurement of tibiofemoral kinematics can be further improved by using slice-to-volume registration with 3D static scans.^178,179 In the finger, 2D RT-MRI has provided insight into joint cavitation, which is responsible for the cracking sound.¹⁸⁰ In the wrist, 2D and 3D RT-MRI have provided insight into ligament insufficiency, instability, and how aberrant kinematics may contribute to wrist pain.^165,181 In the shoulder, low-resolution RT-MRI in conjunction with high-resolution static MRI was used to better characterize rotator cuff disease.¹⁸² In the temporomandibular joint, RT-MRI has been demonstrated with adequate spatiotemporal resolution to track disc and condyle kinematics.^166,183 In summary, RT-MRI is a promising early-stage technique for studying musculoskeletal kinematic insufficiency/abnormality and pain in several body regions.

Other

RT-MRI has also been used for other thorax and abdominal imaging applications. Real-time cine imaging has been used to assess abnormal bowel motility following ingestion of an oral contrast agent (eg, mannitol). Bowel imaging is typically performed in a coronal orientation and bSSFP imaging has been applied for imaging for 1–3 seconds temporal resolution per 3D volume (1 to 0.3 fps).^184,185 Dynamic esophageal imaging has been achieved using a radial spoiled gradient echo acquisition with nonlinear inverse image reconstruction for 40-msec temporal resolution (25 fps) following the ingestion of pineapple juice.¹⁸⁶ RT-MRI has also been used in urethrography, to assess urinary function during bladder emptying,¹⁸⁷ in defecography, to assess the pelvic floor in mechanical and functional rectal disorders,¹⁸⁸ and to asses pelvic floor disorder and pelvic organ prolapse.¹⁸⁹ These techniques often use turbo-spin-echo sequences to achieve T₂ weighting, with 1.0–1.5 mm spatial resolution, and a temporal resolution of 1–5 seconds (1 to 0.2 fps).

RT-MRI has also proven valuable for screening of fetal central nervous system abnormalities, specifically the structural malformations.^190–192 T₂-weighted RARE sequences (often with partial echo along the phase-encode direction) are used for beneficial contrast between gray and white matter.

This article focused on the use of RT-MRI in imaging physical motion; however, it can also be used to image contrast dynamics. Although this is beyond the scope of this article, this includes dynamic contrast-enhanced MRI,¹⁹³ time-resolved angiography,¹⁹⁴ as well as functional MRI,⁴² including arterial spin labeling (ASL)¹⁹⁵ and encephalography.¹⁹⁶

Current Directions

Increasing Role of ML/AI

In recent years, machine-learning (ML) methods, especially deep learning, have enabled breakthroughs in computer vision and image analysis. The astonishing success of deep-learning algorithms has penetrated areas of MR image reconstruction, artifact correction, automatic classification and segmentation, landmark detection, and so on.

Deep learning has huge potential in RT-MRI applications; for example, in real-time visualization and/or immediate downstream analysis. There have been several early works on ML with success for RT-MRI applications, as illustrated in Fig. 8 and Movies S13 and S14. This figure demonstrates the use of deep learning for various tasks; reconstruction of RT-MRI data (removing artifact from undersampled radial images, Fig. 8a), improving image quality of RT-MRI (performing off-resonance deblurring, Fig. 8b), and to enhance clinical impact (performing needle detection and segmentation, Fig. 8c). It has been shown to be popular for low-latency reconstructions of real-time MR data; in 2D cardiac imaging,^93,198,199 MR-guided radiotherapy images,²⁰⁰ and 3D functional MRI.²⁰¹ Additionally, it is popular for rapid postprocessing of real-time data; including identification and segmentation of the vocal tract,^202–205 as well as segmentation of the left and right ventricles from real-time cardiac MRI.²⁰⁶ Other applications include rapid needle detection and segmentation in MR-guided interventions,^197,207 enabling real-time localization of the fetal brain,²⁰⁸ spiral off-resonance deblurring in speech imaging,⁷⁸ and a combination of reconstruction and postprocessing for real-time MR thermometry.²⁰⁹

FIGURE 8:

Illustration of three ML/AI-based low-latency applications. (a) Image reconstruction of cardiovascular imaging; (left-to-right) the BH-bSSFP sequence and the RT radial sequence reconstructed with gridding, GRASP, and the residual U-Net [Adapted from Ref. (93)] (b) Spiral off-resonance deblurring of speech imaging; (left-to-right) GT, uncorrected, IR with GT field map, and the CNN [Adapted from Ref. (78)]. (C) Needle detection and segmentation for ex vivo tissue RT-MRI; (left-to-right) Original image, needle detection and segmentation result using Mask R-CNN, result comparison against a reference [Adapted from Ref. (197)]. Note that “processing time” shown here is the time to run the neural networks and does not include the time to do preprocessing of the data. BH: breath-hold, GRASP: Golden-angle radial sparse parallel imaging, PT: processing time, GT: ground truth, IR: iterative reconstruction.

Some commercially available software has started using ML in real-time imaging, including HeartVista,²¹⁰ which uses ML to automate the MRI exam, control the scanner and assist scan planning, and MeVisLab²¹¹ for segmentation and annotation.

High-Performance Low Field

Recent publications have suggested substantial opportunities for real-time imaging on high-performance low field (HPLF) MRI systems.²¹² These systems operate with a low static field strength (eg, 0.35T and 0.55T) and incorporate contemporary high-performance hardware and modern imaging methods. Unlike historic low-field MRI systems, these HPLF systems include fast shielded gradients and multichannel receiver arrays. They are capable of non-Cartesian sampling, parallel imaging, and compressed sensing, and can exploit readily available computational resources. These systems have produced new capability and exceptional performance for cardiac, abdominal, and pulmonary RT-MRI, illustrated in Fig. 9 and Movie S15.

FIGURE 9:

Demonstration of real-time bSSFP imaging using a high-performance low field (HPLF) MRI system (prototype 0.55T Aera, Siemens Healthcare, Erlangen, Germany). (a) Real-time bSSFP for MRI-guided invasive cardiovascular procedures (TE/TR = 2.0/4.0 msec, flip angle = 45°, in-plane resolution = 2 mm, slice thickness = 8 mm, GRAPPA rate 2, 250 msec temporal resolution, 4 fps). (b) Real-time bSSFP for dynamic intestinal imaging (TE/TR = 1.6/3.2 msec, flip angle = 90°, in-plane resolution = 1.2 mm, slice thickness = 10 mm, GRAPPA rate 3, 1.2 sec temporal resolution for 6 slices, 0.8 fps). No bSSFP banding around the intestines is observed using the HPLF system configuration. (c) Real-time bSSFP for dynamic respiratory imaging (TE/TR = 1.21/2.4 msec, flip angle = 70°, in-plane resolution = 1.8 mm, slice-thickness = 15 mm, GRAPPA rate 2, 250 msec temporal resolution, 4 fps). Due to the reduced susceptibility, higher-quality imaging of lung parenchyma is feasible.

The HPLF system configuration provides several advantages for real-time imaging. 1) Many RT-MRI acquisitions are limited by off-resonance (eg, bSSFP, spiral). Since absolute off-resonance (in Hz) scales linearly with field strength, lower field strength systems relax this constraint. For bSSFP acquisitions, longer TRs can be used without banding artifacts. For spiral sampling, longer readouts can be used without blurring artifacts. 2) T₂* is longer, making it possible for efficient data-sampling strategies, including spiral and EPI and to be implemented with prolonged duration. This allows increased flexibility in trajectory design. Moreover, artifacts related to susceptibility (eg, blurring and ghosting) are reduced. 3) T₁ is shorter at lower field strength, causing more rapid signal recovery between RF pulses. 4) Tissue heating due to RF is reduced at low field, which permits high flip angle excitation and magnetization preparation pulses, with diminishing concerns of patient safety during prolonged real-time imaging. The net result is a favorable system configuration for real-time imaging.

HPLF is especially beneficial for MR-guided interventions, due to the favorable low SAR properties. Reduced device heating will enable the application of commercial conductive devices in the MRI environment, and reduce the burden of design for new devices, which is substantial at conventional field strengths (≥1.5T).²¹³ HPLF is also beneficial for real-time speech imaging, where susceptibility gradients at air–tissue interfaces are the primary constraint.

Finally, an HPLF system could be cost-effective to manufacture, install, and maintain, which may increase accessibility for these real-time imaging applications outside of the conventional radiology environment (eg, cardiology and point-of-care settings). Currently, the B₀ subsystem comprises roughly 30% of system cost.²¹⁴ Therefore, a reduction in field strength and potential complexity (ie, simplified cooling and maintenance) could enable an attractive value system.

Concluding Remarks

MRI has historically been applied to static imaging of the brain, spine, and joints, due to its relatively slow speed. Leaps in MRI technology and information extraction over the last two decades have made it possible to image more broadly, including areas of the body that experience substantial motion and even irregular motion, and including the real-time guidance of interventions. RT-MRI has a rich history and an even brighter future. It is a key enabling technology for MRI to penetrate new diagnostic and interventional applications.