DeepSTI: Towards Tensor Reconstruction using Fewer Orientations in Susceptibility Tensor Imaging

Abstract

Susceptibility tensor imaging (STI) is an emerging magnetic resonance imaging technique that characterizes the anisotropic tissue magnetic susceptibility with a second-order tensor model. STI has the potential to provide information for both the reconstruction of white matter fiber pathways and detection of myelin changes in the brain at mm resolution or less, which would be of great value for understanding brain structure and function in healthy and diseased brain. However, the application of STI in vivo has been hindered by its cumbersome and time-consuming acquisition requirement of measuring susceptibility induced MR phase changes at multiple head orientations. Usually, sampling at more than six orientations is required to obtain sufficient information for the ill-posed STI dipole inversion. This complexity is enhanced by the limitation in head rotation angles due to physical constraints of the head coil. As a result, STI has not yet been widely applied in human studies in vivo. In this work, we tackle these issues by proposing an image reconstruction algorithm for STI that leverages data-driven priors. Our method, called DeepSTI, learns the data prior implicitly via a deep neural network that approximates the proximal operator of a regularizer function for STI. The dipole inversion problem is then solved iteratively using the learned proximal network. Experimental results using both simulation and in vivo human data demonstrate great improvement over state-of-the-art algorithms in terms of the reconstructed tensor image, principal eigenvector maps and tractography results, while allowing for tensor reconstruction with MR phase measured at much less than six different orientations. Notably, promising reconstruction results are achieved by our method from only one orientation in human in vivo, and we demonstrate a potential application of this technique for estimating lesion susceptibility anisotropy in patients with multiple sclerosis.

1. Introduction

The characterization of neural fiber pathways in the brain is important for understanding brain function and development, as well as for diagnosis and treatment of various neurological and psychiatric diseases (Glasser et al., 2016). Estimation of local white matter fiber directions and subsequent tracking of fiber pathways allows for noninvasive characterization of whole-brain neural networks as well as physical connections between different brain regions. Furthermore, it is often of great interest to understand not only the anatomical aspects of white matter fibers, but also their pathophysiological condition in terms of axonal integrity, myelination, and tissue composition, as those conditions have important implications for the study of brain development (Gilles et al., 1983; Van der Knaap et al., 1991; Pujol et al., 2006; Nave and Werner, 2014; Monje, 2018; Grotheer et al., 2022), and a variety of neurodegenerative diseases, for instance multiple sclerosis (MS) (Langkammer et al., 2013; Chen et al., 2014; Li et al., 2016; Wiggermann et al., 2017) and Alzheimer’s disease (Acosta-Cabronero et al., 2013; Bouhrara et al., 2018; Ayton et al., 2017; Kim et al., 2017; Chen et al., 2021b).

The development of diffusion tensor imaging (DTI), has allowed for noninvasive mapping of 3D fiber pathways of the human brain in vivo (Mori and Van Zijl, 2002; Wakana et al., 2004; Jung et al., 2009; Schilling et al., 2018; Jeurissen et al., 2019; Sotiropoulos and Zalesky, 2019). Broadly speaking, DTI, and more advanced approaches such as multi-shell multiorientation diffusion mapping, track white matter fibers by imaging the restricted movement of water molecules inside nerve fibers (Jeurissen et al., 2014, 2019). DTI still is the most commonly used method for imaging neural fiber pathways for clinical applications. While recent advances in pulse sequence design and scanner gradients (McNab et al., 2013; Fan et al., 2014) allow resolutions of about 1.5mm isotropic (Glasser et al., 2016), most clinical diffusion MRI data are still acquired with relative poor spatial resolution, with typical imaging voxel sizes around 2–2.5 mm isotropic. In addition, diffusion MRI is mainly sensitive to the axon cell membrane barrier with much less sensitivity to myelination changes in the brain (Beaulieu, 2002; Liu et al., 2011), which is of specific clinical interest.

A promising alternative to DTI-based tractography is susceptibility tensor imaging (STI) (Liu, 2010). Unlike DTI, STI measures the anisotropic magnetic susceptibility of brain tissue originating from aligned tissue micro-structures with anisotropic molecular susceptibility, e.g. the white matter myelin sheath (Li et al., 2012a, 2017). Using a gradient echo (GRE) sequence to measure the susceptibility induced MR phase changes, STI has the potential to achieve much higher spatial resolution (sub-millimeter) in imaging the fiber pathways than DTI, especially at high magnetic field strengths (3T and above). In addition, because tissue magnetic susceptibility is sensitive to changes in tissue components such as myelin and iron (Sibgatulin et al., 2021), STI derived measures can potentially provide new insights into the pathophysiological processes during brain development and degeneration as shown in fetal alcohol spectrum disorder (Cao et al., 2014) and multiple sclerosis (Wisnieff et al., 2015; Sibgatulin et al., 2022; Li et al., 2016; Wisnieff et al., 2013; Haacke et al., 2015; Liu et al., 2015).

Yet, current STI techniques suffer from long scanning time and impractical protocols, which pose major barriers for routine application of human STI in vivo. Obtaining one STI scan requires rotating the subject’s head multiple times inside the magnet and acquiring a series of measurements at each head orientation. Assuming the susceptibility tensor is real and symmetric, at least six orientations need to be acquired to solve the ill-posed dipole inversion problem for reconstruction of a STI image (Liu, 2010)—and several more are often desirable to improve image quality. Holding the head at non-supine positions for a long time is difficult and increases discomfort for patients. Beyond this, constraints based on head coil size limit the possible range of head rotation angles to typically within 25 degrees from the direction of main magnetic field for in vivo human experiments (Li et al., 2012b). Such a limitation results in a high condition number for the STI forward model, posing significant challenges for reliable image reconstruction even when a large number of orientations can be acquired (Li and Van Zijl, 2014; Bao et al., 2021). As a result, STI is yet to achieve fiber tracking results as consistent and reliable as DTI, especially for in vivo human measurements (Bilgic et al., 2016).

In this paper, we present an approach towards solving the above issues by developing new machine learning based methods for STI image reconstruction that allow for a reduced number of head orientations. Specifically, we developed a STI algorithm, called DeepSTI, that approximately minimizes a regularized STI reconstruction problem, where the regularization is based on a prior that is learned from data.¹ To this end, we leveraged advances in deep learning methods, which have recently demonstrated encouraging results in solving ill-posed inverse problems in medical imaging but have not yet been applied for the challenging problem of STI reconstruction, potentially due to its high-dimensional nature and the difficulty to incorporate MR phase measures acquired at varying number of orientations and arbitrary angles as input. Here, we exploited the learned proximal convolutional neural network (LPCNN) model, used previously for reconstructing isotropic magnetic susceptibility sources as in quantitative susceptibility mapping (QSM) (Lai et al., 2020), and we expanded it to bear in the context of this more challenging task of STI. We validated our method using simulations as well as applied it on in vivo human data, and compared it to existing state-of-the-art STI reconstruction methods. The proposed algorithm shows unprecedented results for STI reconstruction with fewer than 6 orientations, with superior performance over previous methods in the reconstruction of tensor image and principal eigenvector (PEV) maps. DeepSTI also produces more coherent results in fiber tractography from STI, and is not limited to using uniform resolution across samples. Figure 1 depicts an example result, demonstrating the improved resolution of STI over that of DTI, on a human subject.

Fig. 1:

Anisotropy weighted principal eigenvector (PEV) maps estimated from real measurements from a human subject. Left: fractional anisotropy (FA) weighted DTI PEV map acquired at 3T and 2.2 mm isotropic resolution (interpolated to 0.98×0.98×1 mm resolution). Right: magnetic susceptibility anisotropy (MSA) weighted STI PEV map estimated by DeepSTI from 4 GRE measurements at 7T acquired at 1 mm isotropic resolution (reconstructed to 0.98×0.98×1 mm resolution). The acquisition time for STI was 21 minutes (4 head orientations, 5 minutes 15 seconds per orientation). The acquisition time of DTI was 8 minutes 50 seconds.

The rest of this paper is organized as follows. Section 2 summarizes related works. Section 3 describes our proposed method for STI reconstruction, while Section 4 presents experiments and results. Section 5 includes a further discussion on advantages and limitations of our method before concluding in Section 6.

2. Related work

A variety of methods have been proposed for STI reconstruction (Liu, 2010; Liu et al., 2012; Li and Van Zijl, 2014; Bilgic et al., 2016; Dibb and Liu, 2017; Bao et al., 2021; Cao et al., 2021). The original approach (Liu, 2010) used a least square solver to tackle the dipole inversion problem in the frequency domain. Subsequently, Liu et al. (2012) proposed to penalize high spatial frequency anisotropic components, leading to improvements in reconstruction and less sensitivity to imperfect image registration. However, this can lead to over-smoothing (Cao et al., 2021), undermining the advantages of STI for high-resolution fiber tracking. Another study (Li and Van Zijl, 2014) proposed to limit the magnetic susceptibility anisotropy (MSA) to white matter regions as determined from a coregistered T1-weighted image and imposed morphology constraints on the mean magnetic susceptibility (MMS), allowing more accurate tensor estimation. Yet, this approach is naturally sensitive to errors in white matter segmentation and imposes a strong assumption on the anisotropy distribution. Joint estimation of mutually anisotropic relaxation tensor and susceptibility tensor regions from the same multi-echo acquisitions has also been proposed to better determine the principal eigenvectors of the susceptibility tensors (Dibb and Liu, 2017). More recently, Bao et al. (2021) proposed diffusion-regularized STI using the fiber directions estimated by DTI to regularize the STI solution. When using only 6 orientations, this approach achieved better reconstruction than traditional methods and displayed reduced artifacts in fiber direction maps. However, this method requires additional acquisition time for DTI and the regularization from low-resolution DTI can lead to over-smoothed STI solutions. Finally, Cao et al. (2021) proposed the asymmetric susceptibility tensor imaging (aSTI) approach in which the susceptibility tensor is fitted with 9 unknowns first and then decomposed into the symmetric and anti-symmetric components. This approach resulted in less noise and streaking artifacts in the reconstructed symmetric susceptibility tensor, while the anti-symmetric components captured mainly noise and artifacts. Further MMS and magnetic susceptibility anisotropy (MSA) based regularizations of the aSTI approach have also been tested very recently (Shi et al., 2022). Overall, all these approaches require MR phase images acquired at a large number of head orientations for satisfactory reconstruction, which hampers the wide application of STI in human studies in vivo. On the other hand, a number of machine learning based approaches have been proposed for dipole inversion in the simpler problem of QSM (Yoon et al., 2018; Bollmann et al., 2019; Kames et al., 2019; Jung et al., 2020; Chen et al., 2020; Polak et al., 2020; Lai et al., 2020; Jung et al., 2022; Gao et al., 2021, 2022), where the susceptibility is assumed to be isotropic and only the equivalent MMS is estimated. These methods are encouraging, but have not yet been extended to the more challenging setting of STI estimation.

In this work we leverage tools from learned proximal neural networks. The idea of learning a proximal operator using deep learning has been explored previously in a few studies for inverse tasks in medical imaging. For example, Adler and Öktem (2018) proposed a Learned Primal-Dual algorithm for tomographic reconstruction, which used a modified version of the primal dual hybrid gradient algorithm (Chambolle and Pock, 2011) replacing the proximal operator by convolutional neural networks learned from training data. Mardani et al. (2018) proposed to use generative adversarial networks (GAN) for compressed sensing MRI, leveraging the generator for aliasing removal by projecting an initial aliased estimate onto the learned low-dimensional manifold of high-quality MR images. Finally, in the related work of Lai et al. (2020) the authors proposed a learned proximal CNN method for QSM reconstruction (LPCNN-QSM), training a Wide ResNet (Zagoruyko and Komodakis, 2016) as the proximal operator within the proximal gradient descent algorithm for solving the dipole inversion. LPCNN-QSM (Lai et al., 2020) not only achieved state-of-the-art reconstruction accuracy, but also naturally handled an arbitrary number of phase inputs at arbitrary orientations with a single trained model, providing far more flexibility than other methods.

However, difficulties remain in applying learned proximal models in STI. First, the condition number of the forward operator in STI is high (> 50, (Bao et al., 2021)), which makes its inverse problem more ill-posed than QSM and the solution less stable and more sensitive to noise. Second, unlike QSM, where high-quality reconstructions can be obtained from human subjects in vivo through multiple orientation sampling (COSMOS) (Liu et al., 2009), high-quality ground-truth images for human STI are almost impossible to obtain with current technology, posing challenges for both training and evaluation. Furthermore, the underlying image dimension of the STI problem is 6 times higher than QSM. This allows quantification of susceptibility anisotropy and white matter fiber orientations, but it also increases the number of unknowns and makes training more difficult, both statistically and computationally (including GPU memory constraints). Finally, the need for incorporating a varying number of measurements at arbitrary orientation angles as input during training (and testing) is an additional difficulty for data-driven STI reconstruction methods in general.

3. Methods

When a material is placed in a magnetic field, it gets magnetized and creates a local magnetic field perturbation. The ratio of its magnetization to the applied magnetic field is defined as the magnetic susceptibility (Liu, 2010; Li et al., 2017). For a specific anisotropic material, e.g. white matter fibers, the magnetic susceptibility can be approximated by a real symmetric 3×3 tensor (Li and Van Zijl, 2014; Li et al., 2017; Bao et al.,, denoted by $\mathit{\chi }$:

$$\mathit{\chi }=\left[\begin{array}{lll}{\chi }_{11}& {\chi }_{12}& {\chi }_{13}\\ {\chi }_{12}& {\chi }_{22}& {\chi }_{23}\\ {\chi }_{13}& {\chi }_{23}& {\chi }_{33}\end{array}\right]\in {\mathbb{R}}^{3\times 3}.$$ (1)

For axial symmetry, e.g. in cylindrial structures, this tensor contains 6 independent elements: ${\chi }_{11},{\chi }_{12},{\chi }_{13},{\chi }_{22},{\chi }_{23}$ and ${\chi }_{33}$.

The goal of STI is to characterize the magnetic susceptibility tensor of tissue at every voxel. Assuming each voxel can be described by a 3×3 symmetric tensor, each susceptibility tensor image can be divided into 6 sub-volumes, where each sub-volume corresponds to one independent tensor element. Denote the ${\chi }_{ij}$ element at the v-th voxel as ${\chi }_{ij}^v$. Then, each sub-volume is given by

$${\mathbf{x}}_{ij}=\left[{\chi }_{ij}^1\cdots {\chi }_{ij}^v\cdots {\chi }_{ij}^n\right]\in {\mathbb{R}}^n,$$ (2)

where $i,j\in \{1,2,3\}$, $i\le j$, and $n$ is the number of voxels. Thus, the entire susceptibility tensor image $\mathbf{x}$ can be given by

$$\mathbf{x}={\left[\begin{array}{llllll}{\mathbf{x}}_{11}& {\mathbf{x}}_{12}& {\mathbf{x}}_{13}& {\mathbf{x}}_{22}& {\mathbf{x}}_{23}& {\mathbf{x}}_{33}\end{array}\right]}^T\in {\mathbb{R}}^{6n}.$$ (3)

Magnetized materials generate their own magnetic field that in turn perturbs the main magnetic field. Such local field changes cause variations in the material’s resonance frequency, which can be measured via phase images in MRI. The overall magnetic field perturbation, as measured by the phase image, represents the sum of all the field changes generated by each constituent magnetic dipole (Li et al., 2017; Ruetten et al., 2019), and can be calculated as the convolution of a susceptibility distribution with the magnetic field induced by a unit magnetic dipole (Liu, 2010; Liu et al., 2015; Li et al., 2017; Ruetten et al., 2019). Such convolution can be written in a simpler form in the spatial frequency domain using a Fourier Transform (Salomir et al., 2003; Marques and Bowtell, 2005). Therefore, we can express the relationship between the normalized local field change, $\delta B\in {\mathbb{R}}^n$, and the underlying susceptibility tensor image $\mathbf{x}$ as:

$$\delta B={ℱ}^{-1}\mathbf{A}ℱ\mathbf{x}+\mathbf{e},$$ (4)

where $ℱ$ denotes the discrete Fourier Transform², $\mathbf{e}$ represents noise in the measurements, and $\mathbf{A}\in {\mathbb{R}}^{n\times 6n}$ is the dipole kernel in the frequency domain. The dipole kernel is composed of 6 sub-matrices:

$$\mathbf{A}=\left[{\mathbf{A}}_{11},{\mathbf{A}}_{12}+{\mathbf{A}}_{21},{\mathbf{A}}_{13}+{\mathbf{A}}_{31},{\mathbf{A}}_{22},{\mathbf{A}}_{23}+{\mathbf{A}}_{32},{\mathbf{A}}_{33}\right],$$ (5)

where each sub-matrix ${\mathbf{A}}_{ij}\in {\mathbb{R}}^{n\times n}$ is diagonal and multiplies the Fourier transform of ${\mathbf{x}}_{ij}$. It can be shown (Liu, 2010) that the diagonal elements of $A_{ij}$ are given by

$$A_{ij}(\mathbf{k})=\frac{h_i{h}_j}{3}-\frac{{\mathbf{k}}^T\mathbf{h}{k}_i{h}_j}{{\mathbf{k}}^T\mathbf{k}},$$ (6)

where $\mathbf{k}={\left[k_1,k_2,k_3\right]}^T\in {\mathbb{R}}^3$ is the spatial frequency vector and $\mathbf{h}={\left[h_1,h_2,h_3\right]}^T\in {\mathbb{R}}^3$ is the direction of the main magnetic field in the subject frame of reference.

The above Eq. (4) represents the field perturbation induced by the susceptibility tensor acquired at a specific brain orientation. For measurements from multiple head orientations, we then have

$$\left[\begin{array}{c}\delta B^1\\ \delta B^2\\ ⋮\\ \delta B^m\end{array}\right]=\left[\begin{array}{c}{ℱ}^{-1}{\mathbf{A}}^1\\ {ℱ}^{-1}{\mathbf{A}}^2\\ ⋮\\ {ℱ}^{-1}{\mathbf{A}}^m\end{array}\right]ℱ\mathbf{x}+\mathbf{e}={ℱ}^{-1}𝒜ℱ\mathbf{x}+\mathbf{e}$$ (7)

where $\delta B^c$ and ${\mathbf{A}}^c$ are the normalized local field change and dipole kernel at the $c$-th orientation, respectively, and $m$ is the number of orientations. For simplicity, we denote $𝒜$ as the concatenation of the different dipole operators ${\mathbf{A}}^c$.

Given this forward model that characterizes the mapping from the susceptibility tensor image, $\mathbf{x}$, to the normalized local field change measurements $\left(\delta B^1,\dots ,\delta B^m\right)$, the goal of STI reconstruction, or dipole inversion, is to recover the former from the latter. Since the head rotation of human subjects in an MRI scanner is limited by the narrow head coil (to about 25° away from the z-axis, the direction of main magnetic field), the condition number of the system matrix $𝒜$ in STI is large (Bao et al.,2021), making the inverse solution exceptionally unstable, vul nerable to small errors and noise in the measurements. Hence, a good prior is critical for accurate estimation of $\mathbf{x}$. For simplicity, we henceforth denote the measurements at $m$ different orientations by

$$\mathbf{y}={\left[\delta B^1,\dots ,\delta B^m\right]}^T\in {\mathbb{R}}^{nm}.$$ (8)

Then, the dipole inversion problem can be written as the following regularized linear inverse problem:

$$\underset{\mathbf{x}}{\mathrm{m}\mathrm{i}\mathrm{n}}\frac{1}{2}{\parallel \mathbf{M}(\mathbf{y}-{ℱ}^{-1}𝒜ℱ\mathbf{x})\parallel }_2^2+R(\mathbf{x}),$$ (9)

where $\mathbf{M}$ is a brain mask³, $f(\mathbf{x})=\frac{1}{2}{\parallel \mathbf{M}(\mathbf{y}-{ℱ}^{-1}𝒜ℱ\mathbf{x})\parallel }_2^2:{\mathbb{R}}^{6n}\to \mathbb{R}$ is a data fidelity term that promotes the solution to match observed measurements $\mathbf{y}$, and $R\left(\mathbf{x}\right):{\mathbb{R}}^{6n}\to \mathbb{R}$ is a suitable regularization term that introduces prior knowledge⁴ and allows a robust reconstruction for the ill-posed inverse problem (Benning and Burger, 2018). All STI reconstruction methods (Liu et al., 2012; Li and Van Zijl, 2014; Bao et al., 2021) employ some form of the problem formulation in Eq. (9), while differing in the choice of the prior, $R\left(\mathbf{x}\right)$. In what follows, we expand on the choice of prior used in this work, which will be learned from data.

3.1. Learned Proximal Networks for STI Reconstruction

In this work, the prior $R\left(\mathbf{x}\right)$ will be defined implicitly, as we now describe. In order to minimize the problem in Eq. (9), we follow a proximal gradient descent approach by taking the gradient of the differentiable function $f\left(\mathbf{x}\right)$ and the proximal of the regularization function, which we define as ${𝒫}_R$, allowing for potentially non-smooth functions $R\left(\mathbf{x}\right)$. This algorithm produces the iterates given by:

$${\mathbf{x}}_{k+1}={𝒫}_R\left({\mathbf{x}}_k-{\alpha }_k\nabla f\left({\mathbf{x}}_k\right)\right),$$ (10)

where ${\alpha }_k\in \mathbb{R}$ is the step size at the $k$-th iteration. The proximal operator of a function $R$ is formally given by

$${𝒫}_R(\mathbf{x})=\mathrm{a}\mathrm{r}\mathrm{g}\underset{\mathbf{z}}{\mathrm{m}\mathrm{i}\mathrm{n}}\left(R(\mathbf{z})+\frac{1}{2}\parallel \mathbf{x}-\mathbf{z}{\parallel }_2^2\right),$$ (11)

thus providing estimates that are close to a given point while having a small value of the regularizer, $R\left(\mathbf{x}\right)$. To leverage the advantage of deep neural networks for learning powerful priors from large amounts of data, herein we parameterize the proximal operator ${𝒫}_R$ by a deep neural network $G_{\theta }$, so that the iterates in Eq. (10) become

$$x_{k+1}=G_{\theta }\left(x_k-{\alpha }_k\nabla f\left(x_k\right)\right):=S_{\theta }\left(x_k;y\right)$$ (12)

where we succinctly denote by $\theta$ all trainable parameters of the neural network. As we will shortly show, the model $G_{\theta }$ will learn the prior distribution of STI images implicitly by approximating the proximal operator for the regularizer. Importantly, this parameterization decouples the learned prior from the forward operator, allowing it to be applied with a variety of forward models. Indeed, a new forward model can be naturally incorporated in our algorithm by adjusting the forward function in Eq. 12 (i.e., by modifying the function $f\left(\mathbf{x}\right)$), without any modifications to $G_{\theta }$. In the context of STI, this will allow our model to deal with an arbitrary number of measurements at arbitrary head orientations without any retraining or adaptation of the network.

In this way, given an initial estimation ${\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_0$ and a trained model $G_{\hat{\theta }}$, the solution to the inverse problem is given by iteratively applying the following updates with the learned proximal operator:

$$\stackrel{\mathbf{ˆ}}{\mathbf{x}}=\left(S_{\hat{\theta }}\circ \dots \circ S_{\hat{\theta }}\right)\left({\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_0;\mathbf{y}\right)=S_{\hat{\theta }}^{\left(K\right)}\left({\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_0;\mathbf{y}\right):={\varphi }_{\hat{\theta }}(\mathbf{y}),$$ (13)

where $K$ is the number of iterations and o denotes the composition of functions.

In order to obtain a good model $G_{\hat{\theta }}$, we leverage a supervised learning framework. To this end, denote by $𝒳$ the distribution of susceptibility tensor images of human brains, and let $p(\mathrm{Y}\mid \mathbf{x})$ be the distribution of the measurements $\mathbf{y}$ for a given tensor image. We then seek for model parameters, $\theta$, that minimize the expected loss over susceptibility tensors and their measurements, that is

$$\underset{\theta }{\min }\underset{\mathbf{x}~\mathcal{X}}{\mathbb{E}}\underset{\mathbf{y}~p(\text{Y}\mid \mathbf{x})}{\mathbb{E}}L\left({\varphi }_{\theta }(\mathbf{y}),\mathbf{x}\right).$$ (14)

The loss $L$ penalizes large differences between $\mathbf{x}$ and their estimates, $\stackrel{\mathbf{ˆ}}{\mathbf{x}}={\varphi }_{\theta }\left(\mathbf{y}\right)$. In this work, we employ the ${\ell }_1$ norm to this end, i.e. $L\left({\varphi }_{\theta }\left(\mathbf{y}\right),\mathbf{x}\right)={∥{\varphi }_{\theta }\left(\mathbf{y}\right)-\mathbf{x}∥}_1$, to avoid the over-smoothing effect typically produced by the ${\ell }_2$ loss. In practice, the distributions above are unknown and thus training is done by minimizing an empirical version of the expected loss in Eq. (14). Given a set of training samples of ground truth STI (if they were known) and their corresponding measurements, ${\left\{{\mathbf{x}}_i,{\mathbf{y}}_i\right\}}_{i=1}^N$, we minimize this loss over training samples via the following optimization problem:

$$\widehat{\theta }=\arg \underset{\theta }{\text{min}}\frac{1}{N}\sum _{i=1}^N{‖{\varphi }_{\theta }\left({\mathbf{y}}_i\right)-{\mathbf{x}}_i‖}_1.$$ (15)

While this formulation can provide good learned predictors, ground-truth STI samples $\mathbf{x}$ cannot be easily obtained. To resolve this issue, we simulate this distribution by generating realistic brain phantoms using a combination of in vivo QSM and DTI measurements collected from human subjects. We can thus sample the measurements ${\mathbf{y}}_i$ for a given tensor image ${\mathbf{x}}_i$ with the forward model specified above in Eq. (7). We now move on to the generation of this training data.

3.2. Training Data Generation

One major challenge of applying the learned proximal network for STI is the lack of ground-truth data. As stated above, none of the existing methods can provide accurate STI estimation for human in vivo (unlike QSM) and, as a result, it is not feasible to acquire high-quality ground-truth STI from in vivo human subjects, even with large amounts of sampling. To tackle this issue, and to enable supervised learning with measurement-source image pairs, we resort to the available QSM and DTI datasets of in vivo human subjects for generating realistic brain phantoms for STI. In particular, multiple orientation QSM (COSMOS) can provide high-quality estimates of mean magnetic susceptibility. On the other hand, the fiber direction and fractional anisotropy (FA) estimated in DTI can provide good estimates for the principal eigenvector and scaled susceptibility anisotropy. Therefore, we can synthesize computational STI phantoms from QSM-DTI images of the same subjects. Local field measurements are then obtained from these using the forward model in Eq. (7). The simulated measurements and the susceptibility tensor source pairs are used as training samples for the learned proximal network.

More precisely, denote the mean magnetic susceptibility (MMS) value at each voxel as $q\in \mathbb{R}$, as obtained from QSM, and the principal eigenvector of DTI as ${\mathbf{v}}_D\in {\mathbb{R}}^3$ and the DTI FA as $a_D\in \mathbb{R}$. The magnetic susceptibility anisotropy (MSA), $a_S$, is modelled from a scaled DTI FA: $a_S=\gamma a_D$, where $\gamma$ is a scaling factor to map $a_D$ to the range of $a_S$. We set $\gamma =1/15$, such that the mean of MSA for each subject is approximately 0.02 ppm. Further, the eigenvalues of susceptibility tensor at each voxel are given by solving the following system:

$$\left\{\begin{array}{l}\left({\lambda }_1+{\lambda }_2+{\lambda }_3\right)/3=q\\ {\lambda }_1-\left({\lambda }_2+{\lambda }_3\right)/2=a_S\\ {\lambda }_2-{\lambda }_3=\mathrm{\Delta }\end{array}\right.$$ (16)

where ${\lambda }_1\ge {\lambda }_2\ge {\lambda }_3$ are the eigenvalues of the tensor at each location. Above, $\mathrm{\Delta }$ quantifies the difference between the two smaller eigenvalues, known as the tensor asymmetry, which is 0 under the ideal cylindrical symmetry assumption for myelinated nerve fibers. In our model, we randomly sample $\mathrm{\Delta }$ for each voxel from a small range $[0,ϵ]$ to capture the deviation from cylindrical symmetry in real world settings.

Moreover, we let the principal eigenvector ${\mathbf{v}}_1\in {\mathbb{R}}^3$ of the susceptibility tensor at each voxel to be ${\mathbf{v}}_1={\mathbf{v}}_D$, since the principal eigenvectors should align with white matter fiber direction in both STI and DTI. The remaining two eigenvectors, ${\mathbf{v}}_2$ and ${\mathbf{v}}_3$, are sampled randomly such that ${\mathbf{v}}_1,{\mathbf{v}}_2,{\mathbf{v}}_3$ are orthonormal. In this way, the susceptibility tensor at each voxel is given by

$$\chi =\mathbf{V}\mathbf{\Lambda }{\mathbf{V}}^T$$ (17)

where $\mathbf{\Lambda }=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}\left({\lambda }_1,{\lambda }_2,{\lambda }_3\right)$ and $\mathbf{V}=\left[{\mathbf{v}}_1,{\mathbf{v}}_2,{\mathbf{v}}_3\right]$. Finally, the local field maps are computed from computational phantoms via the STI forward model described in Eq. (7), with additive Gaussian variables $\mathbf{e}$ to account for measurement noise and discrepancies between the idealized STI forward model and the real-world imaging system.

More details on the network architecture, training protocol, phantom generation and other implementation details can be found in Appendix A.

3.3. Data

Previously published STI GRE data from 8 healthy young adults (1 Female, mean(std) age of 32(3) years) were used. The STI data for 3 subjects were scanned at 3T (Philips, Achieva) with (1.5 mm)³ isotropic resolution (data from Li and Van Zijl (2014)) and 5 subjects were scanned at 7T (Philips, Achieva), with (1 mm)³ isotropic resolution, reconstructed to 0.98 × 0.98 × 1 mm3 (data from Li et al. (2012b)). A varying number of 6 to 12 head orientations were sampled for each subject scanned at 3T, and 4 head orientations were sampled for each subject scanned at 7T. Phase preprocessing includes phase unwrapping using a Laplacian-based method (Schofield and Zhu, 2003; Li et al., 2011) and background field removal using the sophisticated harmonic artifact reduction on phase data with variable kernel size (VSHARP) method (Schweser et al., 2011; Wu et al., 2012b,a) and maximum kernel radius of 4.5 mm for the 3T data (as in Li and Van Zijl (2014)). For the 7T data, best-path based unwrapping (Abdul-Rahman et al., 2007) and LBV+VSHARP methods (Zhou et al., 2014) with maximum kernel radius of 8 mm were used to calculate the local field maps. For phase map pre-processing, we used already preprocessed data from previous studies. Since these data came from two different studies: the 3T data are from Li and Van Zijl (2014) and the 7T data are from Li et al. (2012b), they were processed using two different methods. In addition, it is worth noting that having slightly different phase pre-processing methods is a good variation to have in the training dataset to improve the generalization ability of trained model, given that real clinical STI datasets may have different pre-processing pipelines. Multiple orientation QSM images were reconstructed from all available GRE measurements using COSMOS. We also used previously published DTI data for all these 8 subjects (data from Li and Van Zijl (2014); Li et al. (2012b)). Unlike STI, DTI data were acquired at 3T for all the 8 subjects, since performing DTI at 7T is technically challenging due to the higher field inhomogeneity and shorter T2* caused by the higher field strength. DTI data were scanned with (2.2 mm)³ isotropic resolution, then reconstructed and interpolated to the same resolution as the STI GRE measurements of each subject.

For data simulation, STI phantoms were generated for each subject using the proposed pipeline in Section 3.2 and local field measurements from different head orientations were simulated from the generated phantoms. To imitate the limitation on rotation angle from the head coil in real situation, the simulated rotation angles were chosen to be uniformly distributed in a range of 25° with respect to the main magnetic field $B_0$. A total of 20 orientations were simulated for each subject. Gaussian noise at signal-to-noise-ratio (SNR) of 10dB (10:1) was added to all simulated measurements.

Experiments were performed by 5-fold cross validation, each time dividing the 8 subjects into 5, 1 and 2 subjects for network training, validation and testing, respectively. The testing data always included one low-resolution data acquired at 3T and one high-resolution data acquired at 7T.

3.4. Competing Methods

In this work we compare with the following state-of-the-art methods for STI reconstructions: 1) STIimag (Li et al., 2017): an image-space-based STI method that solves Eq. (9) with a simple regularization setting the non-brain area to be close to zero; 2) Mean Magnetic Susceptibility regularized (MMSR) STI (Li and Van Zijl, 2014): a regularized STI method which limits anisotropy to white matter area and imposes morphology constraint on MMS; and 3) asymmetric STI (aSTI) (Cao et al., 2021), which fits an asymmetric tensor using a least squares solver and then decomposes it into symmetric and anti-symmetric components; and 4) aSTI+ (Shi et al., 2022): an improved version of aSTI with an isotropic constraint inside cerebrospinal fluid (CSF) and a morphology constraint based on MMS inside white matter. All methods were implemented in Matlab using a least square (LSQR) solver (Paige and Saunders, 1982). The convergence tolerance of the LSQR solver was optimized independently for each method using simulated measurements from 6 orientations to maximize their performance. In addition to the above existing methods, we also implemented and compared to an ${\ell }_1$ regularized STI inversion algorithm based on a wavelet frame, that we dubb WaveletSTI. The method encourages sparsity in the STI reconstruction by adding an ${\ell }_1$ penalty on the wavelet coefficients of the estimated tensor image. The reconstruction problem can thus be formulated as: ${\mathrm{m}\mathrm{i}\mathrm{n}}_{\mathbf{x}}\frac{1}{2}{\parallel \mathbf{M}(\mathbf{y}-{ℱ}^{-1}𝒜ℱ\mathbf{x})\parallel }_2^2+\lambda \parallel \mathbf{W}\mathbf{x}{\parallel }_1$, where $\mathbf{W}$ is the Wavelet transform, implemented as a separable operator per tensor component. In this work, we use the db4 wavelet and set $\lambda =2\cdot {10}^{-4}$ based on preliminary results on simulation data with 6 orientations.

3.5. Fiber Tractography

To evaluate the performance of DeepSTI for white matter fiber tracking, we performed whole brain tractography and local fiber tracking through the corpus callosum (CC) using the STI results and compared these results to those using DTI. Wholebrain tracking of white matter fibers was performed using the FACT algorithm (Mori et al., 1999), and tracking was done using the MRtrix3 toolbox (Tournier et al., 2019). Seeds were randomly sampled from inside the brain mask until a million tracks have been generated. The maximum length of track was set as 250 mm and the anisotropy cut-off threshold for terminating tracks was set as 0.01 ppm for STI MSA and 0.12 for DTI FA. Then, we performed tracking of local fiber bundles passing through the CC. To do this, a mask of CC was manually delineated on a mid-sagittal slice and all fibers passing through the mask were selected from the whole brain tracking result.

We also performed tracking of a specific fiber bundle, i.e. the forceps major, the white matter fiber bundle that connects the left and right occipital lobes via the splenium of corpus callosum (Wakana et al., 2007). For this, we set the number of tracts for whole brain tracking to 10 million for both DTI and STI, and decreased the cutoff threshold for DTI FA to 0.08 to recover more tracks in the forceps major. Segmentation of the left and right occipital lobes was obtained using MRIstudio (Jiang et al., 2006) (http://www.mristudio.org) and combined with the gray matter mask from FSL FAST (Zhang et al., 2001). Tracks passing through the CC and both occipital regions were selected from the whole brain tracking as forceps major. Finally, the selected fibers were cleaned up based on Yeatman et al. (2012) to remove outlier deviating tracks from the central bundle.

3.6. Error Metrics

The following error metrics were used to quantitatively evaluate the reconstruction accuracy of different STI methods. First, to evaluate the accuracy of tensor image reconstruction, peak signal to noise ratio (PSNR) and structural similarity index measures (SSIM) were computed for the whole brain. SSIM was first computed for each tensor component separately and then averaged over all 6 components. To evaluate the accuracy of estimated fiber directions, we computed two angular metrics on PEV maps: eigenvector cosine similarity error (ECSE) and weighted-PSNR (wPSNR). ECSE is defined as one minus the cosine similarity between the estimated and ground-truth PEV, averaged over the anisotropic region (defined by voxels with ground-truth MSA >0.015 ppm). wPSNR is the PSNR of anisotropy-weighted PEV maps over the whole brain. The results are presented in Section 4.6.

4. Experimental results

4.1. Principal Eigenvector Estimation from Simulation

Figure 2 presents the estimated principal eigenvector (PEV) maps obtained from different numbers of head orientations (from 6 to 1) using simulated measurements of a brain phantom with 1.5 mm isotropic resolution. All STI PEV maps are weighted by the magnetic susceptibility anisotropy (MSA) estimated by each respective method. Note that while the visual results are affected by both MSA and PEV estimation, the ECSE metric reflects only PEV estimation accuracy. A significant improvement can be observed in the results yielded by DeepSTI. Compared to the existing methods, more accurate PEV estimation is achieved by DeepSTI across all numbers of head orientations. Even when the number of orientations decreases from 6 to 2, DeepSTI continues to produce highly accurate results (as measured by ECSE) that match well with the ground-truth, while the estimations from other methods deteriorate drastically. In the extreme case of reconstruction from only one head orientation, while the other methods fail to produce any useful reconstructions, DeepSTI can still recover a good amount of structures visible in the ground-truth PEV map, notably in the corpus callosum splenium (indicated by yellow arrow). These results demonstrate the potential of accurate fiber direction mapping using STI when sampling a much smaller number of head orientations than previously reported.

Fig. 2:

Magnetic susceptibility anisotropy (MSA) weighted PEV maps estimated from measurements at decreasing number of head orientations simulated from a brain phantom with 1.5 mm resolution. On the lower-right corner of each image shows the eigenvector cosine similarity error (ECSE) of estimated PEV as defined in Section 3.6.

4.2. Principal Eigenvector Estimation from in vivo Data

We further evaluated DeepSTI using in vivo data. Figure 3 shows the MSA-weighted STI PEV maps estimated from MR phase measurements acquired on a subject at 3T with 1.5 mm isotropic resolution, in comparison to the DTI PEV map. The DTI PEV map is weighted by the DTI FA, while all STI PEV maps are weighted by MSA estimated by each respective method. Note that the qualitative performance of all STI methods decreased as compared to the simulation, likely due to errors introduced in pre-processing steps such as co-registration, background field removal and potential MR phase/frequency contributions that are unaccounted for in the forward model, e.g. multi-compartment microstructure effects or chemical exchange etc. (Wharton and Bowtell, 2012; Sati et al., 2013; Wharton and Bowtell, 2015). Nonetheless, the results provided by DeepSTI still appear more anatomically coherent and better match the result of DTI visually. Note that with only one orientation, DeepSTI can still yield a reasonable estimation of the PEV map. To further evaluate the sensitivity of DeepSTI to different input angles, we visualize the variance in DeepSTI’s predictions across different angles in Section 4.5 and include an illustration of the PEV maps predicted by DeepSTI using different input angles in Fig. B.11 in the Appendix.

Fig. 3:

MSA-weighted PEV maps estimated from in vivo GRE measurements of a subject at 3T with 1.5 mm isotropic resolution as a function of number of head orientations, with FA-weighted DTI PEV map as reference. DTI is acquired at 2.2 mm isotropic resolution and interpolated to 1.5 mm isotropic resolution. The detailed head orientations are specified in the appendix (Table C.2).

4.3. Susceptibility Tensor Component Estimation

We further visualize the susceptibility tensor component images reconstructed by different methods in Fig. 4a and Fig. 4b using the same 3T subject as shown in Figs. 2 and 3. The three diagonal elements of the tensor are shown, defined in left-right $\left({\chi }_{11}\right)$, anterior-posterior $\left({\chi }_{22}\right)$ and superior-inferior $\left({\chi }_{33}\right)$ directions, respectively. Figure 4a shows results from numerical simulations, along with the ground-truth tensor images from the brain phantom. It can be observed that with 3 head orientations, all three methods reconstructed ${\chi }_{33}$ accurately, but DeepSTI outperforms other methods for ${\chi }_{11}$ and ${\chi }_{22}$ estimations. With only one head orientation, DeepSTI produced reasonably good results for all three diagonal elements, while the performance of other methods had deteriorated substantially. Figure 4b shows results from in vivo measurements, where DeepSTI again outperforms other methods for all numbers of measurements and degrades the least with decreasing number of head orientations. Results of eigenvalue, MMS and MSA predictions are shown in Fig. B.12 in the Appendix.

Fig. 4:

Diagonal elements of tensor images $\left({\chi }_{11},{\chi }_{22},{\chi }_{33}\right)$ reconstructed by different methods using (a) simulated and (b) in vivo phase measurements from different numbers of head orientations.

(a) Simulation results from computational phantom.

(b) Results from in vivo human measurements.

4.4. Tractography

Figure 5 depicts fiber tracking results achieved by different methods using in vivo data acquired at 7T with 0.98×0.98×1 mm resolution. With 4 head orientations, DeepSTI provides more complete fiber reconstructions as compared to other STI methods. Even with fewer head orientations, DeepSTI can still recover a good portion of major fiber pathways in whole brain tracking (top row of Fig. 5). Notably, the fibers passing through CC connecting the left and right hemispheres can still be nicely recovered even from only one head orientation (middle and bottom rows of Fig. 5).

Fig. 5:

STI-based fiber tractography results from phase measurements for a subject at 7T with 0.98 × 0.98 × 1 mm3 resolution. Rows 1–3: whole brain tracking results overlaid on a DTI-based FA map at different axial planes. Row 4: 3D volume rendering of neural fibers passing corpus callosum. Row 5: 2D visualization of neural fibers passing corpus callosum. Columns from left to right: DTI (acquired at 3T and 2.2 mm isotropic resolution and interpolated to 0.98 × 0.98 × 1 mm3 resolution), STIimag’s result from 4 orientations, WaveletSTI’s result from 4 orientations, MMSR’s result from 4 orientations, aSTI’s result from 4 orientations, aSTI+’s result from 4 orientations, DeepSTI’s results from 4, 3, 2 and 1 orientations. No FA or MSA weighting is used in visualizing the tractography results.

Fig. 6 depicts tractography results for the forceps major using the same 7T dataset as in Fig. 5. In this task, most existing STI methods (including MMSR, aSTI and aSTI+) failed to reconstruct the forceps major, yielding no tracts that satisfy the given anatomical constraint. STIimag was able to produce a small portion of tracts, but largely incomplete. DeepSTI yielded the most coherent and complete result among all STI methods, with a complete C-shape structure.

Fig. 6:

Tracking of callosal fiber bundle that connects the occipital lobes via the splenium of corpus callosum (forceps major) from measurements of a subject at 7T with 0.98×0.98×1(mm)3 resolution. From left to right: reference DTI reconstruction (acquired at 2.2 mm isotropic resolution and interpolated to 0.98×0.98×1(mm)3 resolution), STI reconstruction from STIimag using four orientations, STI reconstruction from DeepSTI using four orientations. Note that WaveletSTI, MMSR, aSTI and aSTI+ did not recover any tracts that satisfy the anatomical constraint of the forceps major.

4.5. Variance Analysis across Different Angles

Since DeepSTI can reconstruct the underlying susceptibility tensor using MR phase measurements obtained with different head orientations, it is important to understand the variation in the estimated tensor at various angle combinations for the reduced number of measurements. To this end, we further computed the variance in PEV maps estimated from different observation angles for each STI method. Figure 7 shows the result from a 3T data, where 6 head orientations in total were acquired. For each number of orientations less than 6, we computed STI reconstruction from all possible combinations of angles (6, 15, 20, 15 and 6 combinations for 5, 4, 3, 2 and 1 orientations out of 6, respectively) and presented the variance in the modulated PEV maps. As can be observed, the variance yielded by DeepSTI is smaller than for other methods, and it increases only moderately with a reduction in the number of orientations.

Fig. 7:

Variance in MSA-weighted PEV maps estimated from different observing angles using a 3T human in vivo measurements. A total of six head orientations were acquired from this subject. At each number of head orientations, all possible combinations of angles (6,15,20,15 and 6 combinations for 5,4,3,2 and 1 orientations, respectively) were tested.

4.6. Quantitative Results

We quantitatively compare different methods by computing errors of the tensor reconstruction and PEV estimation from simulated measurements of brain phantoms—where ground-truth STI tensors exist—using the error metrics defined in Section 3.6. To account for the variation in rotation angle, three different sets of head orientations were tested for each subject at each number of orientations. All orientations were uniformly sampled from within 25° with respect to the main magnetic field. Results are summarized in Table 1. DeepSTI outperforms the other methods in all metrics with 6 and 3 orientations and in SSIM, ECSE and wPSNR with 1 orientation, which quantitatively verifies the improvement seen visually in the qualitative results.

Table 1:

Quantitative metrics of STI reconstruction using simulated measurements from computational brain phantom, comparing DeepSTI to STIimag (Li et al., 2017), MMSR (Li and Van Zijl, 2014), aSTI (Cao et al., 2021), aSTI+ (Shi et al., 2022) and WaveletSTI. Results were obtained from all 8 subjects using cross validation. PSNR, SSIM and wPSNR were computed for the whole brain and ECSE was computed from the anisotropic regions. Detailed definitions of the metrics are provided in Section 3.6. Numbers in brackets denote standard deviation.

Number of orientations	Method	PSNR	SSIM	ECSE	wPSNR
6	STIimag	40.26(1.16)	0.88(0.01)	0.25(0.03)	15.78(1.24)
	WaveletSTI	41.39(1.24)	0.89(0.01)	0.23(0.03)	15.83(1.40)
	MMSR	44.14(0.82)	0.91(0.01)	0.26(0.03)	16.98(0.70)
	aSTI	40.38(1.20)	0.90(0.01)	0.28(0.03)	17.46(0.81)
	aSTI+	41.00(1.26)	0.90(0.01)	0.28(0.03)	17.65(0.91)
	DeepSTI (this work)	48.06(0.99)	0.96(0.01)	0.07(0.01)	23.13(1.08)
3	STIimag	39.24(1.24)	0.87(0.01)	0.30(0.03)	15.07(1.51)
	WaveletSTI	39.79(1.20)	0.88(0.01)	0.29(0.02)	15.23(1.45)
	MMSR	42.76(0.98)	0.90(0.01)	0.32(0.03)	15.94(0.83)
	aSTI	39.64(1.29)	0.89(0.01)	0.31(0.03)	17.27(0.86)
	aSTI+	39.99(1.25)	0.89(0.01)	0.32(0.03)	17.64(0.81)
	DeepSTI	46.50(1.19)	0.95(0.01)	0.10(0.02)	22.45(1.14)
1	STIimag	38.27(1.13)	0.87(0.01)	0.38(0.02)	16.92(0.81)
	WaveletSTI	38.27(1.13)	0.88(0.01)	0.39(0.02)	16.71(0.84)
	MMSR	40.77(0.88)	0.89(0.01)	0.39(0.01)	16.86(0.43)
	aSTI	38.35(1.15)	0.89(0.01)	0.38(0.02)	17.66(0.61)
	aSTI+	38.65(1.13)	0.89(0.01)	0.38(0.02)	17.60(0.64)
	DeepSTI	43.05(0.97)	0.93(0.01)	0.16(0.02)	20.72(1.04)

4.7. Multiple Sclerosis Patients

To further examine the generalization ability of DeepSTI to patients with neurological diseases, we tested DeepSTI using previously published GRE data from two multiple sclerosis (MS) patients (7T data from Li et al. (2016)). Fig. 8 depicts the MMS and MSA maps calculated from DeepSTI using only one head orientation for each patient. For the first patient (Fig. 8.a), DeepSTI predicted increased MMS and decreased MSA in two example lesions that are hyperintense in local frequency (and QSM as noted in Li et al. (2016)) and hypointense in R2* (indicating major demyelination), as indicated by red arrows. For the second patient (Fig. 8.b), two example lesions that appear invisible (isointense) in local frequency (and QSM as noted in Li et al. (2016)), but hypointense in R2* (indicating potential loss of both iron and myelin), and visible in T1 and FLAIR were selected. DeepSTI’s prediction presents isointense in MMS and shows decreased MSA (green arrow) in one lesion, and less distinct MSA changes (red arrow) in the other lesion.

Fig. 8:

DeepSTI predicted mean magnetic susceptibility (MMS) and magnetic susceptibility anisotropy (MSA) from MR phase measured at one head orientation in two multiple sclerosis patients. The gray scale range for R2*, frequency, MMS and MSA are [0, 70] Hz, [−11, 11] Hz, [−0.08, 0.08] ppm and [0, 0.04] ppm, respectively.

4.8. Ablation Studies and Limitations

To better understand the contribution of the learned prior to the reconstructed MSA of DeepSTI, we tested DeepSTI on simulated phase generated from a fully isotropic STI brain phantom (manually set MSA=0 across the whole brain). Furthermore, to mimic the more realistic case of partially isotropic regions caused by lesions, we also tested our method on a local isotropic phantom where two cubic isotropic regions of size (10 voxel)³ were inserted in the original anisotropic phantom (Fig. 10).

Fig. 10:

DeepSTI prediction for a locally isotropic brain phantom, in comparison with the corresponding prediction for the original anisotropic phantom. Top row: ground-truth MSA (left) and DeepSTI-predicted MSA (right) for the original anisotropic phantom. Bottom row: ground-truth MSA (left) and DeepSTI-predicted MSA (right) for the locally isotropic phantom with two manually selected isotropic regions (indicated by red arrows).

Fig. 9a shows the histogram of MSA values predicted by DeepSTI in major fiber regions (defined by voxels with ground-truth MSA >0.02 ppm) for the original anisotropic phantom and the fully isotropic phantom, while Fig. 9b shows corresponding MSA maps. Although DeepSTI naturally does not predict exactly zero MSA for the isotropic phantom, it is clear that the MSA values predicted by DeepSTI are significantly smaller in the isotropic phantom than in the original phantom. Fig. 10 depicts the MSA maps predicted by DeepSTI for the local isotropic phantom, where a notable decrease in anisotropy can be observed in DeepSTI’s prediction for both of the manually-defined isotropic local regions (indicated by red arrows).

Fig. 9:

DeepSTI predicted MSA for a fully isotropic phantom. (a) Histogram of MSA values in major fiber regions defined by voxels with ground-truth anisotropy > 0.02ppm. (b) DeepSTI predicted MSA for the original anisotropic phantom and the isotropic phantom.

5. Discussion

In vivo human STI has long been hampered by the requirement for acquisition at multiple head orientations and the difficulty in reliable dipole inversion from noisy phase measurements. In this paper, we proposed a new data-driven method for STI dipole inversion that combines the benefits of modern deep learning approaches for learning powerful priors and the flexibility of classic iterative gradient-based algorithms for handling variable forward models. The results show that DeepSTI provides estimations that are significantly more accurate (in the case of brain phantom data with ground truth) or more anatomically consistent in the case of real measurements, even though the model was trained on synthetic data, emphasizing its generalization ability. Notably, DeepSTI is able to manage diverse inputs at different resolutions, with different numbers of orientations, and allows for STI reconstructions with fewer phase measurements (less than 6). It is also worth noting that, even though this paper focuses on the task of STI reconstruction using fewer orientations, the proposed approach is not limited to the STI inversion problem. Indeed, the proposed proximal learning approach is a general framework that could be extended to many different types of inverse problems in medical imaging more broadly.

While DeepSTI shares similarities to the ideas proposed in Lai et al. (2020), there are several differences. First, instead of training on estimated ground-truth images from COSMOS, we developed a phantom-based training scheme that addresses the issue of lacking ground-truth samples from real measurements in STI. Moreover, a new network architecture based on a Residual Symmetric U-Net (Ronneberger et al., 2015; Lee et al., 2017) was used to learn the proximal operator instead of the Wide ResNet (Zagoruyko and Komodakis, 2016). This led to faster and more stable training as well as superior results in the reconstruction of the tensor image and estimation of white matter fiber directions, especially in the case of very few (e.g. single) head orientations. Finally, DeepSTI leverages a new multi-resolution training strategy that allows the model to handle measurements at different resolutions without the need for any ad-hoc re-sampling or re-training. Being able to handle data at different resolutions and different numbers of orientation sampling is central to STI reconstructions, where acquisition protocols between subjects and studies often differ.

5.1. Results of Different Tensor Elements

The tensor image results in Fig. 4 reveal that the existing methods are far less accurate in estimating left-right $\left({\chi }_{11}\right)$ and anterior-posterior $\left({\chi }_{22}\right)$ elements than the superior-inferior $\left({\chi }_{33}\right)$ elements, which is consistent with results reported in Bao et al. (2021). This is not unexpected because the main magnetic field is in the superior-inferior direction and the head rotation angles are limited to a relatively narrow range (< 25°) around that. Note that in the lab frame of reference that aligns with the main field, only $\left({\chi }_{33}\right)$ and some off-diagonal elements (${\chi }_{13}$ and ${\chi }_{23}$) contribute to the field shift and the MR phase measurement (Li et al., 2012b; Milovic et al., 2020). However, DeepSTI was able to produce highly accurate estimations for all three diagonal elements, explaining its better performance in estimating the principal eigenvectors that are used to indicate white matter fiber directions as shown in Fig. 2 and Fig. 3.

5.2. Results for Multiple Sclerosis Patients

Fig. 8 demonstrates the result of our method on two MS patients. While different disease presentations and their neurobiological bases in STI remain to be analyzed in the future, this simple case-study demonstrates that the developed method has the potential to generalize to different clinical settings, e.g. for investigating lesion iron and myelin changes through the measures of MMS and MSA in MS with a single phase measurement. Comparing these results to the positive and negative susceptibility sources using recent chi-separation methods may also be helpful to better interpret the DeepSTI results in MS lesions (Chen et al., 2021a; Shin et al., 2021). Moreover, these results might provide new insights on tissue microstructure changes, such as the manifestation of reduced anisotropy on some MS lesions. We believe this illustrates the potential of DeepSTI to study similar neurological disorders in the future.

5.3. Results for Isotropic Phantoms

Fig. 9 presents the results on an unrealistic but interesting case, where reconstruction of a completely isotropic brain phantom is attempted. Although the results indeed show decreased MSA in white matter areas compared to the anisotropic phantom, the predicted anisotropy values are not completely zero. This is expected, because the solution given by DeepSTI can be broadly regarded as a maximum a posteriori (MAP) estimate for the ill-posed STI reconstruction problem, maximizing the probability of the solution under the learned prior from data. A completely isotropic brain has probability zero in the distribution of human STI (and, by extension, in all the training data used to train the model) and thus a completely isotropic reconstruction should never be expected. Yet, this does not mean that regions of abnormal anisotropy, or areas that are mostly isotropic in white matter, could not be approximately reconstructed. On the contrary, this is demonstrated by the experiment with the locally-isotropic regions in Fig. 10, as well as by the results on the MS patients.

5.4. Single-Phase STI Reconstruction

Experimental results on both simulation and in vivo human data show good performance of DeepSTI with even just one head orientation. Naturally, reconstructing STI from one head orientation is impossible (without any other information), as in any other ill-posed problem. However, accurate reconstructions are possible by making distributional assumptions on the data. The good reconstruction of DeepSTI for even a single phase can be attributed to the prior learned by the proximal network. With good assumptions on the underlying data distribution (i.e. a good characterization of the prior), accurate reconstruction can be achieved with only a small number of measurements. If the proximal network can characterize the prior distribution of STI well, it is thus possible to reliably recover the underlying STI with good accuracy from very few measurements, even at only one head orientation, as we extensively show in our numerical experiments (Figs. 2 and 4a and Table 1).

5.5. Generalization to Out-of-Distribution Samples

It is certainly true that the DeepSTI method depends on samples being drawn from the same underlying data distribution as those in the training set – as in any supervised learning framework. However, this does not imply that when data is slightly different from that in the training data, reconstructions are necessarily impaired. As a concrete example of this observation, we tested the method on two MS patients, which are quite different from subjects in our training data (consisting of all healthy subjects). As our results on MS patients (Fig. 8) show, DeepSTI can still predict changes in MMS and MSA caused by some MS lesions. This suggests that DeepSTI has the potential to generalize to different clinical datasets, even if they differ from the distribution of samples seen during training. Of course, for samples very different from the data distribution of the present study, e.g. from other body parts or brains of other animals, it is expected that the current DeepSTI model would not work as well. However, these situations are out of the scope of this work and will be further investigated in future studies.

5.6. Limitations and Future Work

Lastly, there are several limitations and areas of improvement in our study. First, our evaluation is based on a small cohort of subjects. Cross validation was employed to alleviate this limitation, but a large-scale evaluation involving samples from a broader and larger population will allow for both better training and characterization of the generalization ability of our method. Second, the results from real GRE measurements from human subjects are encouraging and more anatomically feasible than other methods, but these still lack ground-truth for validation and error calculation. In future studies, animal models with available ground-truth images, e.g. from high quality, large number of orientation sampling with no orientation constraints (Chen et al., 2022; Gkotsoulias et al., 2022), could be utilized to allow for better error quantification and further improvement of DeepSTI in vivo. Third, a recent study (Cao et al., 2021) demonstrated that an asymmetric STI formulation—which removed the symmetric constraint on susceptibility tensors—could help separate noise and artifacts from the underlying susceptibility tensor sources, leading to better reconstruction results. DeepSTI can be incorporated with such asymmetric formulation as well, and other potential improvements in the physical model of STI might further improve the prediction of susceptibility tensor using DeepSTI. Finally, while the obtained networks have been proven to be very useful in regularizing this challenging image problem, it is still unclear to us how to precisely, analytically characterize and analyze the prior implicitly parameterized by these models. Further investigation is required to better understand these data-driven priors to increase interpretability and reliability.

6. Conclusion

In this paper, we presented DeepSTI for STI dipole inversion with the goal of reducing the need for current cumbersome and time-consuming STI acquisition schemes that require many head orientations. By combining the benefits of modern deep learning approaches for learning powerful priors and the flexibility of classic iterative gradient-based algorithms for handling variable forward models, DeepSTI enabled accurate and anatomically coherent tensor image reconstructions with fewer head orientations. Experimental results on both simulation and in vivo human brain data demonstrate the superiority of DeepSTI compared to state-of-the-art methods for tensor image reconstruction, fiber direction estimation and tractography. Our results shed light on potential large-scale application of high-resolution STI on human in vivo for better understanding brain functions and neurological diseases.

Highlights.

• We present DeepSTI, a method for STI reconstruction with data-driven priors

• DeepSTI allows for STI from significantly fewer orientations than previous methods

• Promising results are obtained from only one orientation in human in vivo

• High-resolution fiber tractography results are obtained from DeepSTI reconstructions

• We demonstrate an application of DeepSTI for multiple sclerosis characterization

Appendix A. Implementation Details

Our network architecture for proximal learning was implemented following the Residual Symmetric Unet Lee et al. (2017). It contains a contracting path with five downsampling layers and an expanding path with five upsampling layers. Downsampling is implemented by max-pooling, and upsampling is by transpose convolution. The basic module for feature extraction is a single convolution followed by a residual block. The residual block contains two concatenated convolution layers, each followed by group normalization and Exponential Linear Unit (ELU) activation. Skip connections are added between the same spatial scale of contracting and expanding paths and summation joining is used instead of concatenation.

For network training, we used the Adam optimizer (Kingma and Ba, 2014) with a batch size of two. Training was run for a total of 100 epochs (with 1000 batches, i.e., 2000 patches, per epoch). The learning rate was 10⁻⁴ and kept unchanged throughout training. At the end of each epoch, the trained model was evaluated on validation data and the one with the lowest PEV cosine similarity error (ECSE) among all epochs was selected as the final model. Phase measurements from 1–6 head orientations were used to train the network. To increase the diversity of training data and further improve the performance, data augmentation is performed, including random affine (scaling and rotation) transform, random elastic transform, and adding isotropic cuboid patches with random sizes and locations to simulate lesions. During training, images were cropped into 64³ patches with a stride of 32, given GPU memory constraints. At test time, no cropping was applied and the whole images were fed into the network for tensor reconstruction. The matrix size of whole image is 144 × 144 × 90 for 3T data and 224 × 224 × 110 for 7T data.

Five-fold cross validation was performed. In each fold, 5 subjects were used for training, 1 for validation and 2 for testing. Training data include two levels of spatial resolution: low-resolution (1.5 mm isotropic) and high-resolution (0.98 × 0.98 × 1mm), allowing for mixed multi-resolution training, with a two-stream batch sampler to ensure each batch contains samples of the same resolution for faster training. The training data of each fold always included two low-resolution subjects and three high-resolution subjects.

The number of iterations for learned proximal updates, $K$, is 4 in eq. 13.

Details for STI phantom generation.

In eq. 16, $\mathrm{\Delta }$ was uniformly sampled between [0,0.002] to represent the difference between the two smaller eigenvalues of the susceptibility tensor. The eigenvectors ${\mathbf{v}}_2$ and ${\mathbf{v}}_3$ are sampled such that ${\mathbf{v}}_1,{\mathbf{v}}_2,{\mathbf{v}}_3$ form an orthonormal basis. To achieve this, ${\mathbf{v}}_2$ and ${\mathbf{v}}_3$ are obtained via rotation of a coordinate system using Euler angles. The first two euler angles are chosen such that the z-axis $([0,0,1{]}^T)$ is rotated to ${\mathbf{v}}_1$. Then, the third Euler angle is randomly chosen between 0 and $2\pi .{\mathbf{v}}_2$ and ${\mathbf{v}}_3$ are the x- and y-axes of the coordinate system after rotation by the Euler angles. In Eqs. (4) and (7), Gaussian variable $\mathbf{e}$ serves as the noise term in the STI forward model. The standard deviation of $\mathbf{e}$ is chosen such that SNR of the simulated measurements is 10dB.

Appendix B. Complementary Figures

Fig. B.11:

MSA-weighted PEV maps predicted by DeepSTI using measurement at one head orientation with different rotation angles. Top row: results from simulation data; bottom row: results from in vivo data. The input angles (i.e., directions of main magnetic field in the subject frame of reference) are (from left to right): a) for simulation data: (−0.0010, −0.0250, 0.9997), (0.1196, 0.2541, 0.9597), (0.0854, −0.2788, 0.9565), (0.0090, 0.4195, 0.9077), (0.3411, 0.1648, 0.9254), (−0.2203, −0.0452, 0.9744); b) for in vivo measurements: (0.1342, 0.1898, 0.9726), (0.1102, 0.1024, 0.9886), (0.1854, 0.0502, 0.9814), (0.0519, 0.1327, 0.9898), (−0.0135, 0.3424, 0.9394), (0.2460, −0.0841, 0.9656).

Fig. B.12:

Eigenvalues $\left({\lambda }_1\ge {\lambda }_2\ge {\lambda }_3\right)$, MMS and MSA of the tensor images reconstructed by different methods using (a) simulated and (b) in vivo phase measurements from di_fferent numbers (6, 3 and 1) of head orientations.

Appendix C. Detailed Head Orientations

Table C.2:

Detailed head orientations for Fig. 3.

Number of orientations	Directions of main magnetic field in the subject frame of reference at different orientations
6	(−0.0010, −0.0250, 0.9997), (0.1196, 0.2541,0.9597), (0.0854, −0.2788, 0.9565), (0.0090, 0.4195, 0.9077), (0.3411,0.1648,0.9254), (−0.2203, -0.0452, 0.9744)
5	(−0.0010, −0.0250, 0.9997), (0.1196, 0.2541,0.9597), (0.0854, −0.2788, 0.9565), (0.0090, 0.4195, 0.9077), (0.3411,0.1648,0.9254)
4	(−0.0010, −0.0250, 0.9997), (0.1196, 0.2541,0.9597), (0.0854, −0.2788, 0.9565), (0.0090, 0.4195, 0.9077)
3	(−0.0010, −0.0250, 0.9997), (0.1196, 0.2541,0.9597), (0.0854, −0.2788, 0.9565)
2	(−0.0010, −0.0250, 0.9997), (0.1196, 0.2541,0.9597)
1	(−0.0010, −0.0250, 0.9997)