20-fold Accelerated 7T fMRI Using Referenceless Self-Supervised Deep Learning Reconstruction

Abstract

High spatial and temporal resolution across the whole brain is essential to accurately resolve neural activities in fMRI. Therefore, accelerated imaging techniques target improved coverage with high spatio-temporal resolution. Simultaneous multi-slice (SMS) imaging combined with in-plane acceleration are used in large studies that involve ultrahigh field fMRI, such as the Human Connectome Project. However, for even higher acceleration rates, these methods cannot be reliably utilized due to aliasing and noise artifacts. Deep learning (DL) reconstruction techniques have recently gained substantial interest for improving highly-accelerated MRI. Supervised learning of DL reconstructions generally requires fully-sampled training datasets, which is not available for high-resolution fMRI studies. To tackle this challenge, self-supervised learning has been proposed for training of DL reconstruction with only undersampled datasets, showing similar performance to supervised learning. In this study, we utilize a self-supervised physics-guided DL reconstruction on a 5-fold SMS and 4-fold in-plane accelerated 7T fMRI data. Our results show that our self-supervised DL reconstruction produce high-quality images at this 20-fold acceleration, substantially improving on existing methods, while showing similar functional precision and temporal effects in the subsequent analysis compared to a standard 10-fold accelerated acquisition.

I. INTRODUCTION

Functional MRI (fMRI) has been crucial in expanding our understanding of human perception and cognition [1]. Even though significant progress has been made in fMRI acquisition and reconstruction [2], additional improvements for coverage and resolution are imperative to contribute to our broader understanding of the brain function.

As fMRI acquisitions have become a standard part of large-scale studies, such as the Human Connectome Project (HCP) [3], several accelerated MRI methods have been utilized for improved resolution and coverage. Simultaneous multi-slice (SMS) imaging is commonly used in functional neuroimaging with its ability for rapid high-resolution with whole brain coverage [4], [5]. SMS imaging is typically further combined with in-plane acceleration for improving the resolution of fMRI at ultrahigh field strengths, while maintaining a reasonable echo time [2]. These accelerated acquisitions are then reconstructed using linear SMS and parallel imaging reconstruction [4]–[6]. However such reconstructions suffer from spatially varying noise amplification [7], [8], and may suffer from inter and intra-slice residual aliasing artifacts [9] which will further be exacerbated when operating at higher acceleration rates.

Recently, deep learning (DL) methods have gained immense interest as an alternative to improve accelerated MRI [10]–[13]. In particular, physics-guided DL methods have emerged as a powerful strategy due to their robustness and improved reconstruction quality [11]–[14], showing promising results at higher acceleration rates where conventional methods struggle to maintain high quality reconstruction. In general, DL reconstruction methods are trained in a supervised manner, using fully-sampled data as reference [10]–[13]. However, such reference data is not available in highly-accelerated fMRI acquisitions. Recently, self-supervised learning has been proposed for training physics-guided DL reconstruction using only undersampled data, showing similar performance to supervised learning [15].

In this work, we adapt a recent self-supervised DL [15] strategy to reconstruct an SMS-accelerated HCP-style retinotopic mapping acquisition and show that the results of subsequent fMRI analysis are unaltered by the use of such a regularized non-linear DL reconstruction. In doing so, we establish the feasibility of reconstructing 20-fold accelerated 7T fMRI data using self-supervised DL reconstruction. Results show that the proposed DL reconstruction substantially improves upon conventional methods at the 20-fold acceleration rate, while maintaining comparable signal fidelity to conventional methods at standard 10-fold acceleration for fMRI analysis.

II. METHODS

A. Self-supervised DL Reconstruction for SMS fMRI

The forward model for SMS acquisition is given as:

$$y_{SMS}^{\Omega }=\sum _{i=1}^S{\mathbf{\text{E}}}_i^{\Omega }{x}_i+n,$$ (1)

where $y_{\text{SMS}}^{\Omega }\in {ℂ}^P$ is the acquired SMS k-space, Ω is the in-plane undersampling pattern, S is number of simultaneously excited slices, $x^i\in {ℂ}^{M\times N}$ is the i^th slice image, $E_{\text{i}}^{\Omega }:{ℂ}^{\text{M}\times \text{N}}\to {ℂ}^P$ is the multi-coil encoding operator of the i^th slice, and n is measurement noise. The notation can be condensed by letting x_SMS be the concatenation of the individual slices ${\left\{x_i\right\}}_{i=1}^S$ along the readout direction [4], [16], and $E_{SMS}^{\Omega }=\left[E_1^{\Omega }\dots E_S^{\Omega }\right]$, which yields

$$y_{SMS}^{\Omega }=E_{SMS}^{\Omega }{x}_{SMS}+n.$$ (2)

For high acceleration rates, this system is generally ill-conditioned. Thus, a regularized least squares formulation may be used for the inverse problem:

$$\text{arg}\underset{x_{SMS}}{\text{min}}{\Vert y_{SMS}^{\Omega }-E_{SMS}^{\Omega }{x}_{SMS}\Vert }_2^2+\mathcal{R}\left(x_{SMS}\right),$$ (3)

where the first term enforces data consistency (DC) with the acquired measurements and $\mathcal{R}(\cdot )$ is a regularizer. This least squares problem is typically solved by using iterative techniques that decouples the DC term and the regularizer into a series of sub-problems [17]. One such solution is variable splitting with quadratic penalty [13], [15]:

$$z_{SMS}^{(l-1)}=\text{arg}\underset{z_{SMS}}{\text{min}}{\mu \Vert x_{SMS}^{(l-1)}-z_{SMS}\Vert }_2^2+\mathcal{R}\left(z_{SMS}\right),$$ (4)

$$x_{SMS}^{(l)}=\text{arg}\underset{x_{SMS}}{\text{min}}{\Vert y_{SMS}^{\Omega }-E_{SMS}^{\Omega }{x}_{SMS}\Vert }_2^2+\mu {\Vert x_{SMS}-z_{SMS}^{(l-1)}\Vert }_2^2,$$ (5)

where $x_{SMS}^{(l)}$ is the reconstructed slices at iteration l, while $z_{SMS}^{(l)}$ is an auxiliary variable and μ is the penalty parameter. This iterative algorithm is unrolled for a fixed number of iterations in physics-guided DL reconstruction [10], and sub-problem (4) is solved implicitly using neural networks, while (5) is solved by conjugate gradient [13]. The network output for a given test dataset can be written as $f\left(y_{SMS}^{\Omega },E_{SMS}^{\Omega };\mathit{\theta }\right)$, where θ are the learnable parameters of the unrolled network.

Although physics-guided neural networks are conventionally trained in a supervised manner, the lack of reference training data for high-resolution fMRI studies prevents this strategy. However, a recent work entitled Self-Supervised learning via Data Undersampling (SSDU) has shown similar performance to supervised learning by splitting the acquired locations, Ω, into two disjoint sets, Θ and Ω, where Θ is used in the DC units of the unrolled network and Λ is used to define the k-space loss [15]. A multi-mask version of SSDU has also been proposed to improve reconstruction quality at very high acceleration rates by utilizing these disjoints sets multiple times [18], [19]. In this study, multi-mask SSDU is used with the following loss function:

$$\underset{\mathit{\theta }}{\text{min}}\frac{1}{N\cdot K}\sum _{n=1}^N\sum _{k=1}^K\mathcal{L}\left(y_{SMS}^{{\Lambda }_k,n},E_{SMS}^{{\Lambda }_k,n}\left(f\left(y_{SMS}^{{\Theta }_k,n},E_{SMS}^{{\Theta }_k,n};\mathit{\theta }\right)\right)\right),$$ (6)

where N is the number of training data in the database, K is the number of multi-maks (Ω = Θ_k ∪Λ_k, k ∈ 1, …, K), $\mathcal{L}(\cdot ,\cdot \cdot )$ is the k-space loss between the unseen acquired points and the network output and the network is parametrized by θ. The k^th training and loss masks used on the n^th training data sample are represented as Λ_k, n and Θ_k, n, respectively. A schematic of the implementation is shown in Figure 1.

Fig. 1:

A schematic of the self-supervised approach used in this study that does not require fully-sampled data. The iterative algorithm in Equations (4)–(5) is unrolled for a fixed number of iterations. Each iteration consists of a regularizer unit which is implemented by a ResNet, and a data consistency unit that enforces fidelity with the acquired k-space. Self-supervised training splits the acquired samples into K pairs of disjoint sets (Ω=Θ_k∪Λ_k), where Θ_k is used for data-consistency and Λ_k is used for training loss. This strategy allows a training without fully-sampled data, which has been a main hindrance for the adaptation of DL reconstruction for highly-accelerated fMRI. Multi-mask SSDU is utilized with K= 6.

B. Imaging Experiments

Imaging was performed on a 7T Siemens Magnex Scientific (Siemens Healthineers, Erlangen, Germany) system with a 32-channel receiver head coil-array. 8 subjects were imaged using the HCP-retinotopy protocol detailed in [20]. The study was approved by our institutional review board, and written informed consent was obtained before each examination.

For each subject, six 5-minute acquisitions were obtained with the protocol in [20]. In this study, the first 2 experimental runs, which are the rotating wedge (RETCCW and RETCW) retinotopic mapping paradigms, were utilized. RETCCW was collected with an anterior to posterior phase encoding direction, while RETCW was collected with a posterior to anterior phase encoding direction. The following imaging parameters were used similar to the 7T HCP fMRI protocol [2]: SMS factor = 5, in-plane acceleration = 2, resolution: 1.6mm isotropic, TR = 1s and whole-brain coverage. In both runs, the subjects viewed wedges which completed a rotation every 32 seconds (0.3125 Hz). These data were used as the retinotopic representation within the visual cortex.

The standard acquisition at 5-fold SMS and 2-fold in-plane acceleration (overall 10-fold acceleration), reconstructed with conventional split-slice GRAPPA (SPSG) [6] was used as the baseline for image quality and fMRI analyses. These acquired k-space data were further retrospectively undersampled to 5-fold SMS and 4-fold in-plane acceleration (overall 20-fold acceleration), which were then reconstructed using the proposed self-supervised DL reconstruction, as well as conventional SPSG. In the following, SMS5×R2 (as 10-fold acceleration) notation is used for the standard acquisition, while SMS5×R4 (as 20-fold acceleration) notation is used for the retrospectively sub-sampled datasets.

C. Functional Processing and Retinotopic Analyses

The reconstructions were each minimally processed identically using afni_proc.py, a tool available with AFNI [21]. To account for differences from the different phase encoding directions, distortion correction was performed. Subsequently, motion correction was performed using the distortion-corrected median image as a registration target. These distortion- and motion-corrected data were then scaled such that each voxel had a mean of 100. The processed data were then imported into MATLAB. The clockwise wedge data were time-reversed and both datasets were shifted to align the hemodynamic responses for identical wedge positions. Next, 9 nuisance regressors were projected out from each dataset. These were polynomials up to order 3 and the 6 (rigid-body) motion estimates from the motion-correction step. The mean of the two datasets were calculated and an FFT was performed to determine the amplitude and phase at 0.3125Hz. In order to threshold the data, the coherence between the two runs was also calculated using the MATLAB tool mscohere. The phase maps were compared for each reconstruction using a coherence threshold of 0.4.

A region of interest was then created from the baseline SMS5×R2 data reconstructed with SPSG using this threshold. A number of metrics were calculated for this region. This included the average absolute phase error, relative to the original SMS5×R2 SPSG reconstruction, of the deep learning and SPSG reconstructions of the SMS5×R4 accelerated data. Additionally, temporal signal to noise ratio (tSNR) was calculated for each reconstruction, estimated as the mean of the signal after processing divided by the standard deviation of the residual following the removal of nuisance regressors. Circular statistics were handled using CircStat [22].

D. Implementation Details

Multi-mask SSDU [15], [18], [19] was used with K = 6 masks to train a physics-guided DL reconstruction network. The sub-problems in (4) and (5) were unrolled for 10 iterations. Each iteration contains a DC unit and a regularizer. The former was solved by 10 conjugate gradient steps [13], while the latter was solved by a convolutional neural network that utilizes a ResNet structure [23]. ESPIRiT was used to generate sensitivity maps from low-resolution calibration scans [24]. The network was trained using Adam optimizer with an ℓ₁-ℓ₂ loss in k-space [15] with a learning rate of 3 · 10⁻⁴ over 80 epochs. The network had a total of 592,129 trainable parameters.

For SMS imaging, to avoid boundary artifacts during regularization, the CAIPI [5] shifts were removed to position each slice to the middle of the FOV before the regularizer ResNet unit, and re-arranged back to the acquired CAIPI shifts before the DC units. The ResNet unit operated on a 2-channel input (real and imaginary), and the SMS = 5 slices were concatenated along the readout direction, as detailed in Section II-A.

Training was performed on 17 slab groups from 4 subjects. Only one time-frame per subject was used. Therefore no temporal correlations/redundancies were exploited by the DL reconstruction. Testing was performed on the whole volume and whole time series of 4 subjects unseen by the network. All training was performed using TensorFlow in Python.

III. RESULTS

SPSG reconstruction at SMS5×R2 is shown in Figure 2 (first column) as the baseline image quality. For the SMS5×R4 accelerated data reconstructed with both SPSG and self-supervised DL, visible noise reduction is observed with the proposed self-supervised DL reconstruction. Compared to SPSG reconstruction at 20-fold acceleration, self-supervised DL removes residual aliasing artifacts leading to a closer match to the reference image quality. Substantial improvement is observed in thalamus region (third slice) with the proposed DL method.

Fig. 2:

Representative slices from the HCP-style retinotopy acquisition, reconstructed with SPSG at the standard 10-fold acceleration, and with SPSG and proposed self-supervised DL at a retrospective 20-fold acceleration. Baseline image quality is based on the 10-fold SPSG reconstruction. At 20-fold acceleration, SPSG suffers from visible noise amplification and residual artifacts, while the self-supervised DL reconstruction substantially suppresses these across all the slices shown.

Fig. 3 depicts tSNR maps of axial, sagittal and coronal slices of a subject reconstructed with SPSG and self-supervised DL reconstructions prior to any processing for distortion or motion. The first column depicts the baseline 10-fold accelerated acquisition using SPSG reconstruction. Self-supervised DL reconstruction substantially improves upon SPSG at SMS5×R4 acceleration showing good fit to reference tSNR maps. Mean and standard deviation of the tSNR values in a 10 × 10 × 10 voxel (gray boxes) agree with these visual observations. 10-fold acceleration has the highest tSNR value (18.99±0.32) while SPSG at SMS5×R4 acceleration has the lowest tSNR value (8.94±0.25). Self-supervised DL reconstruction at SMS5×R4 acceleration leads to a substantially improved tSNR value (17.38±0.33) close to baseline tSNR of the 10-fold accelerated acquisition.

Fig. 3:

tSNR maps of axial, sagittal and coronal slices using SPSG at standard 10-fold acceleration, and using SPSG and proposed self-supervied DL at retrospective 20-fold acceleration. Substantial tSNR gains are observed in the self-supervised DL reconstruction compared to SPSG. The mean and standard deviation of tSNR values in the grey boxes are 18.99±0.32, 8.94±0.25 and 17.38±0.33 for SMS5×R2 SPSG, SMS5×R4 SPSG and SMS5×R4 self-supervised DL, respectively, confirming these observations.

Following distortion and motion correction, the average of the absolute difference in phase estimates across subjects was 0.36±0.02 radians for the SMS5×R4 self-supervised DL and 0.46±0.40 radians for the SMS5×R4 SPSG reconstructions, within the ROI derived from the original SMS5×R2 SPSG reconstruction. In this setting, the average tSNR within the ROI was 32.5±3.2 for SMS5×R2 SPSG, 22.5±2.0 for SMS5×R4 self-supervised DL and 15.1±1.5 for the SMS5×R4 SPSG reconstructions.

IV. DISCUSSION

We proposed a self-supervised physics-guided deep learning reconstruction for highly-accelerated HCP-style fMRI. A retrospective study with SMS5×R4 acceleration was evaluated both for image reconstruction and subsequent fMRI analysis. Proposed self-supervised DL reconstruction showed improved image quality compared to existing SPSG reconstruction. tSNR improvement is observed with self-supervised DL reconstruction compared to SPSG. Furthermore, fMRI analysis shows that self-supervised DL reconstruction has higher coherence than the SPSG reconstruction in the visual cortex similar to standard acquisition.

For the 20-fold accelerated data, the self-supervised DL reconstruction strongly outperforms the SPSG reconstruction across multiple metrics, including tSNR and absolute difference in phase estimates. For the latter, substantial improvement in standard deviation is observed in addition to a 20.40±0.96% improvement in the mean, highlighting less extreme errors in phase estimations. This is also apparent from the images themselves, which possess visible artifacts for SMS5×R4 SPSG reconstruction. With the DL reconstruction, these artifacts are no longer visible, and the image structure resembles the original SMS5×R2 data. The self-supervised DL functional data are also associated with higher tSNR compared to the SMS5×R4 SPSG reconstruction.

While the SMS5×R4 DL reconstruction do not fully reproduce the SMS5×R2 case, the phase maps in Figure 4 suggest that much of the important temporal information is retained across the two runs. The activation patterns are robust, and retinotopic mapping is preserved, leading to highly correlated phase values and low error in phase estimation.

Fig. 4:

Phase maps for each reconstruction type with a coherence threshold of 0.4, limited to clusters greater than 40 voxels. Color indicates polar angle, in radians (please see the legend in top left figure). Right side of the brain is on right.

V. Conclusion

The proposed self-supervised DL reconstruction reduced residual artifacts and improved the tSNR compared to existing reconstruction technique at 20-fold accelerated fMRI acquisition. The fMRI analysis led to similar results compared to standard acquisition, showing that self-supervised DL reconstruction did not alter the temporal effects or functional spatial precision.