Computational MRI: Compressive Sensing and Beyond

The process of forming images from measured data using computational algorithms is referred to as computational imaging. Rapid advances in computational hardware and signal processing algorithms have resulted in a flurry of activity in computational imaging in several application areas, including medicine, biology, remote sensing, and seismic imaging. Medical imaging has witnessed extensive research in computational imaging, beginning with computed tomography (CT), which relies on algorithms to construct a 3D volume from X-ray measurements taken from X-ray projections at different angles; this work received the 1979 Nobel Prize in Medicine. Most of the current medical-imaging modalities [e.g., magnetic resonance imaging (MRI), CT, positron emission tomography] employ computational imaging in one form or another.

Advances in computational MRI were primarily driven in the last decade by parallel image acquisition using multiple receiver coils and compressed sensing (CS). The ability of these approaches to break the classical Nyquist sampling limit has been exploited to considerably reduce the acquisition time in static imaging applications and to significantly improve the spatial and temporal resolution in dynamic imaging applications. Extensive research in this area has facilitated developments of efficient transforms, novel regularization priors, smart acquisition strategies, fast optimization algorithms, and computational toolboxes. The recent U.S. Food and Drug Administration approval of CS products for clinical scans makes MRI one of the main benefactors of CS algorithms.

In this issue

While the application of CS-based algorithms in medical imaging is maturing, the recent research in this area has initiated a new computational way of thinking. The main focus of this special issue of IEEE Signal Processing Magazine (SPM) is on recent developments in computational MRI. These developments are pushing the frontier of computational imaging beyond CS. Similar to CS, most of these algorithms rely on image representation in one form or another. However, the common recent thread is the departure from handcrafted image representations (e.g., sparse wavelet model) to learning-based image representations. These learned representations are seamlessly combined with clever measurement strategies to significantly advance the state of the art in a number of areas. Several exciting applications including significantly improved spatial and temporal resolution, a considerable reduction in scan time, measurement of biophysical parameters directly from highly undersampled data, and direct measurement of very high-dimensional data are reviewed in this special issue of SPM.

This issue describes key ideas underlying the computational approaches used in MRI. These approaches range from CS algorithms that rely on fixed transforms or dictionaries, to adaptive or shallow-learning algorithms that adapt the image representation to the data (e.g., low-rank and dictionary-based methods), to recent deep-learning methods that learn a highly nonlinear representation from exemplar data. The articles provide insight into the capabilities of the current algorithms, their limitations, and their utility in challenging MRI problems. While the focus of this special issue is on medical imaging and in particular MRI, most of the problems, and hence solutions, are easily translatable to signal recovery applications in other areas.

Overview

The 11 articles in this issue represent three broad categories. Each article explains its specific problem setting and the applications where the problem occurs, the various solution approaches (algorithms), and experimental comparisons (either from existing work or new ones).

Model-based reconstruction in MRI using fixed transforms

The first article, “Mathematical Models for Magnetic Resonance Imaging Reconstruction,” by Doneva, is an overview of MRI acquisition schemes from a computational perspective. Doneva discusses the various challenges stemming from the nonideal nature of the acquisition. She also provides a brief overview of the various model-based reconstruction algorithms introduced to overcome these challenges.

In his article, “Optimization Methods for Magnetic Resonance Image Reconstruction,” Fessler covers the model-based algorithms from an optimization theory perspective in greater detail. Specifically, he poses the image reconstruction algorithm as an optimization algorithm and reviews various regularization penalties or priors designed to exploit specific image properties, including sparsity and data-adaptive regularizers. The article also reviews computationally efficient algorithms to solve the aforementioned optimization problems.

Shallow-learning methods that adapt the transform or representation to the data

The article “Transform Learning for Magnetic Resonance Image Reconstruction” by Wen et al. provides an overview of learning-based methods in MRI. These methods range from those that use fixed transforms/models, to shallow-learning methods that adapt the transform to the image content, to deep-learning methods.

The next article, “Structured Low-Rank Algorithms,” by Jacob, Mani, and Ye reviews advances in recovering images by lifting their Fourier data to a structured matrix. The low-rank nature of these matrices is used to recover them from undersampled measurements. The authors also discuss the ability of the framework to exploit image properties that are difficult to capture by image-domain methods, including continuous-domain sparsity, exponential time profiles, phase relations, and manifold structure of images, as well as connection to deep-learning methods.

In “Linear Predictability in Magnetic Resonance Imaging Reconstruction,” Haldar and Setsompop provide a historical review of linear prediction-based methods used in MRI, including the earliest methods in computational MRI; Fourier domain parallel MRI algorithms; and recent methods that can exploit limited spatial support, phase constraints, and sparsity. The authors also provide interesting examples of highly accelerated imaging by combining multiple constraints.

Christodoulou and Lingala next present “Accelerated Dynamic Magnetic Resonance Imaging Using Learned Representations,” a unified view of learning algorithms to dynamic MRI, designed to overcome challenges including motion, while capitalizing on opportunities including pseudo-repetitive motion and temporal correlations. These algorithms range from blind CS, low-rank reconstruction methods, higher-order multidynamic methods, explicit motion-estimation and compensated-recovery methods, and manifold regularized-recovery methods.

In the next article, “Computational MRI With Physics-Based Constraints,” Tamir et al. introduce physics-based models in MRI. These models are used as constraints for CS reconstruction and quantitative imaging. The article focuses on spin density and relaxation effects. The authors also discuss approaches to selecting user-controllable scan parameters based on the physical model. Several multicontrast imaging and quantitative mapping examples are presented.

In their article, “Plug-and-Play Methods for Magnetic Resonance Imaging,” Ahmad et al. review plug and play (PnP) methods, where an off-the-shelf denoising subroutine is used as a regularization prior in image reconstruction problems. The authors describe how the PnP method can be interpreted as a solution to an equilibrium equation, allowing convergence analysis from the equilibrium perspective. The article demonstrates applications of PnP methods in MRI applications.

Deep-learning algorithms for MRI

The next article, “Compressed Sensing: From Research to Clinical Practice With Deep Neural Networks,” by Sandino et al., reviews the challenges with the classical model-based algorithms using fixed transforms in the clinical perspective. It then demonstrates how these challenges can be overcome in a step-by-step fashion using a deep-learning-based reconstruction framework by applying unrolled neural networks.

In their article, “Deep-Learning Methods for Parallel Magnetic Resonance Image Reconstruction,” Knoll et al. review recent advances in deep learning applied to reconstructing highly undersampled MR images using neural networks. The article reviews image-domain-based techniques that introduce deep regularizers and describes Fourier domain methods that use neural network-based interpolation strategies.

Finally, in “Deep Magnetic Resonance Image Reconstruction,” Liang et al. provide an overview of deep-learning-based algorithms in MR inverse problems. The authors present two general classes of approaches: 1) algorithms that rely on a model-based framework with deep-learned priors, which are unrolled, and 2) deep architectures that do not rely on loop unrolling. They also discuss signal processing approaches to maximizing the potential, and they point out open problems.

Biographies

Mathews Jacob(mathews-jacob@uiowa.edu) received his Ph.D. degree from the Biomedical Imaging Group, ETH Zürich, Switzerland, in 2003. He was a Beckman postdoctoral fellow at the University of Illinois at Urbana–Champaign between 2003 and 2006. He is currently a professor in the Department of Electrical and Computer Engineering and is heading the Computational Biomedical Research Group at the University of Iowa, Iowa City. His research interests include image reconstruction, analysis, and quantification in the context of magnetic resonance imaging. He received the CAREER Award from the National Science Foundation and the Research Scholar Award from the American Cancer Society. He currently serves as associate editor of IEEE Transactions on Medical Imaging and IEEE Transactions on Computational Imaging and is the general chair of the 2020 IEEE International Symposium on Biomedical Imaging. He is a Senior Member of the IEEE.

Jong Chul Ye(jong.ye@kaist.ac.kr) is currently a professor in the Department of Bio/Brain Engineering and an adjunct professor in the Department of Mathematical Sciences at the Korea Advanced Institute of Science and Technology, Daejeon, South Korea. He serves as an associate editor for IEEE Transactions on Medical Imaging, a senior editor for IEEE Signal Processing Magazine, and a section editor for BMC Biomedical Engineering. He is vice chair of the IEEE Signal Processing Society Technical Committee for Computational Imaging and a general cochair of the 2020 IEEE Symposium on Biomedical Imaging. His group was awarded first place at the 2009 Recon Challenge at the International Society for Magnetic Resonance in Medicine workshop and second place at the 2016 Low Dose CT Grand Challenge organized by the American Association of Physicists in Medicine. His research interests include machine learning and sparse recovery for various imaging reconstruction problems in X-ray computerized tomography, magnetic resonance imaging, optics, ultrasound, and so on. He is a Senior Member of the IEEE.

Leslie Ying(leiying@buffalo.edu) received her B.S. Eng. degree in electronics engineering from Tsinghua University, China, in 1997 and her M.S. and Ph.D. degrees in electrical engineering from the University of Illinois at Urbana-Champaign in 1999 and 2003, respectively. She is a Clifford C. Furnas professor of biomedical engineering and electrical engineering at the University at Buffalo, State University of New York. She received the CAREER Award from the National Science Foundation in 2009. Her research interests include compressed sensing, image reconstruction, and machine learning for biomedical imaging, such as magnetic resonance imaging and superresolution microscopy. She served as an associate editor of IEEE Transactions on Biomedical Engineering from 2009 to 2012 and deputy editor of Magnetic Resonance in Medicine from 2012 to 2019. She served on the administrative committee of the IEEE Engineering in Medicine and Biology Society from 2013 to 2015. She is now the editor-in-chief of IEEE Transactions on Medical Imaging.

Mariya Doneva (mariya.doneva@philips.com) received her B.Sc. and M.Sc. degrees in physics from the University of Oldenburg, Germany, in 2006 and 2007, respectively, and her Ph.D. degree in physics from the University of Lübeck, Germany, in 2010. She is a senior scientist at Philips Research. Her research interests include methods for efficient data acquisition, image reconstruction, and quantitative parameter mapping in the context of magnetic resonance imaging. She was a research associate in the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley, from 2015 to 2016. She is a recipient of the 2011 Junior Fellow Award of the International Society for Magnetic Resonance in Medicine.