Abstract
Purpose
To develop a deep learning reconstruction approach to improve the reconstruction speed and quality of highly undersampled variable-density single-shot fast spin-echo imaging by using a variational network (VN), and to clinically evaluate the feasibility of this approach.
Materials and Methods
Imaging was performed with a 3.0-T imager with a coronal variable-density single-shot fast spin-echo sequence at 3.25 times acceleration in 157 patients referred for abdominal imaging (mean age, 11 years; range, 1–34 years; 72 males [mean age, 10 years; range, 1–26 years] and 85 females [mean age, 12 years; range, 1–34 years]) between March 2016 and April 2017. A VN was trained based on the parallel imaging and compressed sensing (PICS) reconstruction of 130 patients. The remaining 27 patients were used for evaluation. Image quality was evaluated in an independent blinded fashion by three radiologists in terms of overall image quality, perceived signal-to-noise ratio, image contrast, sharpness, and residual artifacts with scores ranging from 1 (nondiagnostic) to 5 (excellent). Wilcoxon tests were performed to test the hypothesis that there was no significant difference between VN and PICS.
Results
VN achieved improved perceived signal-to-noise ratio (P = .01) and improved sharpness (P < .001), with no difference in image contrast (P = .24) and residual artifacts (P = .07). In terms of overall image quality, VN performed better than did PICS (P = .02). Average reconstruction time ± standard deviation was 5.60 seconds ± 1.30 per section for PICS and 0.19 second ± 0.04 per section for VN.
Conclusion
Compared with the conventional parallel imaging and compressed sensing reconstruction (PICS), the variational network (VN) approach accelerates the reconstruction of variable-density single-shot fast spin-echo sequences and achieves improved overall image quality with higher perceived signal-to-noise ratio and sharpness.
© RSNA, 2018
Introduction
Fast spin-echo (SE) sequences are routinely used in clinical MRI to achieve T2 weighting. Compared with conventional fast SE sequences, single-shot fast SE sequences acquire the whole two-dimensional k-space in a single echo train (1), resulting in much faster acquisition compared with conventional fast SE. Single-shot fast SE has been increasingly used for T2-weighted imaging of the abdomen and pelvis to mitigate the influence of motion artifacts.
Single-shot fast SE sequences are often implemented as half-Fourier acquisitions (2) to shorten the echo train length, thus reducing the specific absorption rate and achieving clinically relevant echo times; the impact of the half-Fourier acquisition on blurring is complex because of a tradeoff between reduced T2 decay over the echo train and resolution loss associated with homodyne reconstruction. Variable density sampling has been used to achieve similar echo times with full k-space coverage (3,4). Combined with variable-refocusing flip angles (5), variable-density single-shot fast SE sequences have been shown to achieve improved image quality and speed compared with conventional single-shot fast SE with half-Fourier acquisitions (6).
Reconstructing variable density–sampled k-space usually requires a combination of parallel imaging and compressed sensing (PICS) (7–10). However, although acquisitions require around 500 msec per section, PICS reconstruction may take seconds to finish. Therefore, conventional PICS techniques may lead to reconstruction lags and queues that are unacceptable in a clinical setting. In addition, PICS reconstruction often requires empirical tuning of parameters, which can result in heterogeneous image quality, particularly in pediatric applications that have a wide range of patient sizes. Therefore, it is highly desirable to improve the reconstruction of variable-density single-shot fast SE in terms of both speed and consistency.
Recently, deep learning approaches (11–16) have been used to reconstruct undersampled k-space data by training deep neural networks, which take k-space measurements (or zero-filled reconstructions) as inputs and output reconstructed images. These approaches have been demonstrated to achieve faster reconstruction than conventional PICS while maintaining comparable image quality (11,15). Among these approaches, a variational network (VN) approach (11) has been proposed to reconstruct images of the knee acquired with standard multishot two-dimensional fast SE by unfolding a PICS reconstruction and learning the regularization functions and coefficients. This approach was shown to be faster than conventional PICS reconstruction, with reconstruction times of under 193 msec per section (11).
The purpose of this work is to improve the reconstruction speed and quality of variable-density single-shot fast SE sequences by using a deep learning–based VN reconstruction, and to clinically investigate its feasibility in comparison with conventional PICS reconstruction.
Materials and Methods
Our study had institutional review board approval and informed written consent was obtained. The authors from academic institutions designed the study and had control of the data and the information submitted for publication. V.T., I.M., and C.J.H are employees of GE Healthcare. The remaining authors declare no conflict of interest.
Variable-Density Single-Shot Fast SE Sequences and PICS Reconstruction
A variable-density sampling scheme (3,4) (Fig 1, A) was developed for single-shot fast SE sequences to enable full-Fourier acquisitions with variable-refocusing flip angles (6). Imaging parameters are shown in Table 1. To obtain training reference, images were reconstructed with a conventional PICS approach by using an iterative compressed sensing sensitivity encoding reconstruction with precomputed coil sensitivity maps and ℓ1-wavelet regularizations (3,17).
Table 1:
Image Reconstruction with a VN
A VN containing 10 recurrences of data-processing blocks (11) was implemented and trained to directly output variable-density single-shot fast SE reconstructions. Inputs to the network were raw k-space measurements and precomputed coil sensitivity maps (Fig 1, B). Each data-processing block contains a data-consistency bypass, a regularization bypass, and an input bypass (Fig 1, C). The output of one data-processing block can be expressed as:
, |
where and denote the input and output of the block. denotes an operator that transforms the acquired multichannel k-space to a channel-combined image by performing two-dimensional inverse Fourier transform inverse coil sensitivity multiplication and summation over channels. denotes the adjoint operator of . denotes the step size of the data-consistency bypass. denotes a weighted combination of Gaussian radial basis functions, that is,
, |
where denotes the weight of the basis function. is a linear operator that models convolutions with filter kernels of size 11 × 11, and denotes the number of convolutions used to model the regularization term. Twenty filters and 31 Gaussian radial basis functions were used, with about 50 000 learned parameters.
In the training stage, the difference between the output and the reference images was estimated by the mean squared error, and it was minimized by optimizing the values of and through stochastic gradient descent. After training the network for 784 complete passes through the training data set, a model with learned fixed values of these parameters was used to reconstruct images acquired during clinical imaging.
Experiments
The cohort of 157 patients referred for abdominal imaging was predominantly pediatric (mean age, 11 years; range, 1–34 years; 72 males [mean age, 10 years; range, 1–26 years] and 85 females [mean age, 12 years; range, 1–34 years]). Imaging was performed with a variable-density single-shot fast SE pulse sequence at 3.25 times acceleration on a 3.0-T imager (MR750; GE Healthcare, Waukesha, Wis) with a 32-channel torso coil (NeoCoil; Pewaukee, Wis) between March 2016 and April 2017. The acquisition field of view and number of acquired sections were prescribed based on the size of the patient. Coil sensitivity maps for both VN and PICS were precomputed with ESPIRiT (18) by using the Berkeley Advanced Reconstruction Toolbox (19). Conventional PICS reconstruction was performed with 30 iterations. An empirically tuned regularization factor of 0.001 was used for all iterations. These parameters were optimized based on the results of previous studies (3,4) and the radiologists’ feedback on image quality.
The VN was trained based on the PICS reconstruction of 130 patients (63 males and 67 females) on a single GeForce GTX 1080 Ti graphics processing unit (NVIDIA, Santa Clara, Calif) for 784 epochs (about 1 day) by using TensorFlow (https://www.tensorflow.org). A set of coronal images in each patient was used for training, and each image was treated as an individual training sample. Evaluation of the trained VN model was performed in the remaining 27 patients (nine males, 18 females) in comparison with the PICS reconstruction.
Image Grading
Reconstructed images with blinded reconstruction information were evaluated independently and consecutively by three radiologists (S.S.V., with 15 years of experience; J.S., with 5 years of experience; and S.T.C., with 7 years of experience in body MRI interpretation) in terms of overall image quality, perceived signal-to-noise ratio (SNR), image contrast, image sharpness, and residual artifacts with scores ranging from 1 (nondiagnostic) to 5 (excellent). Scoring criteria is shown in Table 2. Evaluations were performed for each patient based on a complete set of reconstructed coronal images.
Table 2:
Statistical Analysis
Wilcoxon tests were performed in Matlab (version 2015b; Mathworks, Natick, Mass) to test the null hypothesis that there was no significant difference between the VN approach and the conventional PICS reconstruction. A two-tailed P value of less than .05 was considered to indicate statistical significance. The 95% confidence intervals of proportions for the VN approach to outperform PICS were also calculated. Interobserver variability was assessed by using the intraclass correlation coefficient based on the normalized scores of each reader. This coefficient was interpreted as excellent (0.75–1.00), good (0.60–0.75), fair (0.40–0.59), and poor (<0.40) (20).
Results
Compared with conventional PICS reconstruction, the VN approach achieved improved perceived SNR (P = .01; mean score ± standard deviation, 4.2 ± 0.7 for VN and 3.9 ± 0.9 for PICS) and improved sharpness (P < .001; mean score, 4.3 ± 0.7 for VN and 4.0 ± 0.8 for PICS), with no difference in image contrast (P = .24; mean score, 4.1 ± 0.9 for VN and 4.0 ± 0.9 for PICS) and residual artifacts (P = .07; mean score, 3.6 ± 0.7 for VN and 3.4 ± 0.8 for PICS). In terms of overall image quality, VN performed better than did PICS (P = .02; mean score, 4.1 ± 0.8 for VN and 3.8 ± 0.9 for PICS) (Fig 2, B). As shown in the mean scores previously, VN achieved smaller standard deviation of scores than did PICS in perceived SNR, sharpness, residual artifacts, and overall image quality (Fig 2, B). 95% confidence intervals of proportions for the VN approach to outperform PICS were 0.41 ± 0.19, 0.70 ± 0.17, 0.37 ± 0.18, 0.33 ± 0.18, and 0.41 ± 0.19 for perceived SNR, sharpness, contrast, residual artifacts, and overall image quality, respectively.
The intraclass correlation coefficients of 0.70 in overall image quality, 0.70 in perceived SNR, and 0.57 in residual artifacts indicated good agreement among the three readers in overall image quality. Intraclass correlation coefficients of 0.40 in image sharpness and 0.44 in image contrast indicated fair agreement. The intraclass correlation coefficient of 0.62 for our entire study indicated good agreement among readers.
Representative images comparing the VN and PICS approaches in terms of perceived SNR, residual artifacts, and sharpness are shown in Figures 3, 4 and Figures E1–E6 (online). Six of 27 PICS reconstructions showed noticeable noise amplification, which was reduced with the VN approach (Figs 3, 4 and Fig E1, E2 [online]). In Figure 4, PICS reconstruction shows residual artifacts attributable to a prescription field of view that is smaller than is the patient. These artifacts were considerably reduced with the VN approach. Five of 27 VN reconstructions showed improved structural delineation and sharpness compared with PICS (Figs E3–E6 [online]).
Representative images comparing the VN and PICS approaches in the presence of pathologic disease are shown in Figure 5 and Figure E7 (online). Figure 5 shows a patient with a focal liver lesion, and Figure E7 (online) shows a patient with cirrhosis. In both patients, the VN approach showed comparable results as did PICS, with the VN approach achieving improved perceived SNR without sacrificing sharpness or lesion conspicuity. Residual ghosting artifacts were observed in four of 27 patients with the VN approach. In terms of motion-related ghosting artifacts, the VN approach showed similar results as PICS (Fig E8 [online]). In two of 27 patients, the VN approach resulted in ghosting artifacts that were not observed in the corresponding PICS reconstruction (Fig E9 [online]). Further investigation showed that the corresponding coil sensitivity maps estimated from ESPIRiT were responsible for these artifacts because of the presence of singularities at the same spatial locations (Fig E9 [online]).
Average reconstruction time was 5.60 seconds ± 1.30 per section for PICS in the Berkeley Advanced Reconstruction Toolbox and 0.19 second ± 0.04 per section for VN in TensorFlow on the same graphics processing unit.
Discussion
This work developed and clinically evaluated a VN approach for fast and robust reconstruction of variable-density single-shot fast SE acquisitions. This VN approach enables reconstruction speeds of approximately 0.2 second per section, which allows real-time image reconstruction of variable-density single-shot fast SE sequences for practical clinical deployment. Furthermore, compared with the conventional PICS reconstruction, it achieves improved overall image quality with higher perceived SNR and improved sharpness.
The primary goal of our study was to speed up conventional PICS reconstruction of variable-density single-shot fast SE imaging. By using PICS reconstruction as reference, the VN model was trained to output images comparable to the PICS reconstruction, with only raw k-space measurements and coil sensitivity maps as inputs. Compared with conventional PICS reconstruction that uses 30 steps of forward and backward operations with wavelet transform regularizations, the trained VN model only contains 10 steps of forward and backward operations, and the regularizations are represented by a series of two-dimensional convolutions and a weighted combination of Gaussian radial basis functions. Deep learning-based VN reconstruction is also highly accelerated with off-the-shelf software (TensorFlow) and libraries (cuDNN; NVIDIA).
The VN method is limited to learning the regularization coefficients and step sizes of the data consistency and regularization terms under a PICS reconstruction framework, rather than directly applying a neural network to the input. Although this is not as flexible as are other deep learning methods, it ensures that the entire network functions as an accelerated PICS reconstruction rather than as a pure deep neural network model learned from training samples. Therefore, the output of the VN approach is kept close to the conventional PICS reconstruction by design, which reduces the risk of reconstructing images that show artifacts or false features.
The VN approach allows different regularization coefficients and flexible regularization functions at each step, as it learns the regularization at each data processing block as a function of its own inputs. Compared with conventional PICS with constant regularization and number of iterations, this provides higher flexibility in applying more appropriate regularization levels to different images. Because the training set includes 130 different patients, the VN learns an average PICS reconstruction, effectively functioning as a de-noising network for images with low SNR or residual artifacts. For these images, the VN results in improved perceived SNR and reduced residual artifacts compared with the PICS reconstruction. For images with high SNR, this may also reduce over-regularization and improve image sharpness compared with the PICS reconstruction. Therefore, the VN approach enables more consistent image quality over conventional PICS reconstruction for image parameters pretrained in the training process.
The VN approach had several limitations. First, the design of the VN architecture and the training parameters require empirical tuning of the network architecture (number of steps, convolution kernels, and Gaussian radial basis functions). This tuning is relatively tractable because the VN generalizes the reconstruction for all training data sets, whereas the PICS is sensitive to individual images. Second, in two of the 27 reconstructions, small singularities in the coil sensitivity maps resulted in residual ghosting with the VN approach. In contrast, PICS produced clean reconstructions of magnitude images by using the same sensitivity maps. The cause of these artifacts in the coil sensitivity maps is the subject of ongoing research. Potential solutions include using sensitivity maps from prescans or enabling complex value operations. Third, the variable-density sampling pattern was optimized for conventional PICS reconstruction. For deep learning approaches, the optimal variable-density sampling pattern may be different from PICS. The optimal sampling pattern may also be data dependent. Therefore, optimizing the sampling pattern for the VN approach and even for each case may further improve the performance of the VN approach.
There were several limitations to our study. First, reported reconstruction times may be dependent on hardware models and implementations. Second, prescan noise statistics may also be available on some hardware platforms. This enables adaptive regularizations in PICS (21), which may improve the image quality over constant regularizations. Finally, no fully sampled data or external validation standard was used to evaluate the accuracy of both PICS and VN reconstructions; this is also not feasible because of T2 decay over the echo train.
In conclusion, the VN approach provides a fast and feasible way of reconstructing variable-density single-shot fast SE images. Learning-based regularizations help to improve perceived SNR, enabling improved overall image quality of variable-density single-shot fast SE imaging.
Summary
A deep learning reconstruction for variable-density single-shot fast spin-echo MRI achieves improved overall image quality with higher signal-to-noise ratio and sharpness than does conventional reconstruction methods.
Implications for Patient Care
■ The variational network approach allows fast and robust image reconstruction of variable-density single-shot fast spin-echo sequences for practical clinical deployment.
■ Learning a variational network enables real-time scanned image feedback for clinicians and improves perceived signal-to-noise ratio with better overall image quality for variable-density single-shot fast spin-echo images.
SUPPLEMENTAL FIGURES
Acknowledgments
Acknowledgment
The authors thank Marcus Alley, PhD, for helping with the online reconstruction.
Supported by National Institutes of Health (P41 EB015891, R01 EB009690, R01 EB019241) and GE Healthcare.
F.C. and V.T. contributed equally to this work.
Disclosures of Conflicts of Interest: F.C. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author has grants/grants pending with GE Healthcare. Other relationships: disclosed no relevant relationships. V.T. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author is an employee of GE Healthcare. Other relationships: disclosed no relevant relationships. I.M. disclosed no relevant relationships. J.Y.C. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author is a consultant for HeartVista. Other relationships: author has patents pending. J.I.T. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author has grants/grants pending with GE Healthcare. Other relationships: disclosed no relevant relationships. J.S. disclosed no relevant relationships. S.T.C. disclosed no relevant relationships. C.J.H. disclosed no relevant relationships. J.M.P. disclosed no relevant relationships. S.S.V. Activities related to the present article: disclosed no relevant relationships. Activities not related to the present article: author holds stock/stock options in Arterys. Other relationships: disclosed no relevant relationships.
Abbreviations:
- PICS
- parallel imaging and compressed sensing
- SE
- spin echo
- SNR
- signal-to-noise ratio
- VN
- variational network
References
- 1.Semelka RC, Kelekis NL, Thomasson D, Brown MA, Laub GA. HASTE MR imaging: description of technique and preliminary results in the abdomen. J Magn Reson Imaging 1996;6(4):698–699. [DOI] [PubMed] [Google Scholar]
- 2.Feinberg DA, Hale JD, Watts JC, Kaufman L, Mark A. Halving MR imaging time by conjugation: demonstration at 3.5 kG. Radiology 1986;161(2):527–531. [DOI] [PubMed] [Google Scholar]
- 3.Taviani V, Litwiller DV, Tamir JI, Loening AM, Hargreaves BA, Vasanawala SS. Variable Density Compressed Sensing Single Shot Fast Spin Echo [abstr]. In: Proceedings of the Twenty-Fourth Meeting of the International Society for Magnetic Resonance in Medicine. Berkeley, Calif: International Society for Magnetic Resonance in Medicine, 2016; 618. [Google Scholar]
- 4.Chen F, Taviani V, Tamir JI, et al. Self-calibrating wave-encoded variable-density single-shot fast spin echo imaging. J Magn Reson Imaging 2018;47(4):954–966. [DOI] [PubMed] [Google Scholar]
- 5.Busse RF, Brau AC, Vu A, et al. Effects of refocusing flip angle modulation and view ordering in 3D fast spin echo. Magn Reson Med 2008;60(3):640–649. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Loening AM, Saranathan M, Ruangwattanapaisarn N, Litwiller DV, Shimakawa A, Vasanawala SS. Increased speed and image quality in single-shot fast spin echo imaging via variable refocusing flip angles. J Magn Reson Imaging 2015;42(6):1747–1758. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lustig M, Donoho D, Pauly JM. Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn Reson Med 2007;58(6):1182–1195. [DOI] [PubMed] [Google Scholar]
- 8.Griswold MA, Jakob PM, Chen Q, et al. Resolution enhancement in single-shot imaging using simultaneous acquisition of spatial harmonics (SMASH). Magn Reson Med 1999;41(6):1236–1245. [DOI] [PubMed] [Google Scholar]
- 9.Pruessmann KP, Weiger M, Scheidegger MB, Boesiger P. SENSE: sensitivity encoding for fast MRI. Magn Reson Med 1999;42(5):952–962. [PubMed] [Google Scholar]
- 10.Griswold MA, Jakob PM, Heidemann RM, et al. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magn Reson Med 2002;47(6):1202–1210. [DOI] [PubMed] [Google Scholar]
- 11.Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med 2018;79(6):3055–3071. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Yang Y, Sun J, Li H, Xu Z. ADMM-Net: a deep learning approach for compressive sensing MRI. arXiv:1705.06869 [cs.CV], 2017. https://arxiv.org/abs/1705.06869. Accessed October 16, 2017. [DOI] [PubMed]
- 13.Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging. arXiv: 1611.03679 [cs.CV], 2016. https://arxiv.org/abs/1611.03679. Accessed October 20, 2017. [DOI] [PubMed]
- 14.Schlemper J, Caballero J, Hajnal JV, Price A, Rueckert D. A deep cascade of convolutional neural networks for MR image reconstruction. International Conference on Information Processing in Medical Imaging, June 25, 2017. Cham, Switzerland: Springer, 647–658. [Google Scholar]
- 15.Qin C, Schlemper J, Caballero J, Price A, Hajnal JV, Rueckert D. Convolutional recurrent neural networks for dynamic MR image reconstruction. arXiv:1712.01751 [cs.CV], 2017. https://arxiv.org/abs/1712.01751. Accessed January 22, 2018. [DOI] [PubMed]
- 16.Dar SU, Çukur T. A transfer-learning approach for accelerated MRI using deep neural networks. arXiv:1710.02615 [cs.CV], 2017. https://arxiv.org/abs/1710.02615. Accessed November 10, 2017. [DOI] [PubMed]
- 17.Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2009;2(1):183–202. [Google Scholar]
- 18.Uecker M, Lai P, Murphy MJ, et al. ESPIRiT: an eigenvalue approach to autocalibrating parallel MRI—where SENSE meets GRAPPA. Magn Reson Med 2014;71(3):990–1001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Uecker M, Ong F, Tamir JI, et al. Berkeley advanced reconstruction toolbox [abstr]. In: Proceedings of the Twenty-Third Meeting of the International Society for Magnetic Resonance in Medicine. Berkeley, Calif: International Society for Magnetic Resonance in Medicine, 2015; 2486. [Google Scholar]
- 20.Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess 1994;6(4):284–290. [Google Scholar]
- 21.Khare K, Hardy CJ, King KF, Turski PA, Marinelli L. Accelerated MR imaging using compressive sensing with no free parameters. Magn Reson Med 2012;68(5):1450–1457. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.