A machine learning-based method for estimating the number and orientations of major fascicles in diffusion-weighted magnetic resonance imaging

Abstract

Accurate modeling of diffusion-weighted magnetic resonance imaging measurements is necessary for accurate brain connectivity analysis. Existing methods for estimating the number and orientations of fascicles in an imaging voxel either depend on non-convex optimization techniques that are sensitive to initialization and measurement noise, or are prone to predicting spurious fascicles. In this paper, we propose a machine learning-based technique that can accurately estimate the number and orientations of fascicles in a voxel. Our method can be trained with either simulated or real diffusion-weighted imaging data. Our method estimates the angle to the closest fascicle for each direction in a set of discrete directions uniformly spread on the unit sphere. This information is then processed to extract the number and orientations of fascicles in a voxel. On realistic simulated phantom data with known ground truth, our method predicts the number and orientations of crossing fascicles more accurately than several classical and machine learning methods. It also leads to more accurate tractography. On real data, our method is better than or compares favorably with other methods in terms of robustness to measurement down-sampling and also in terms of expert quality assessment of tractography results.

Graphical Abstract

1. Introduction

Diffusion-weighted magnetic resonance imaging (DW-MRI) is a powerful tool for non-invasive probing of brain micro-structure, with important applications in studying brain connectivity, development, and degeneration (Johansen-Berg and Behrens, 2013). In brain connectivity studies, DW-MRI measurements are used to infer the number and orientations of major fascicles or a fiber orientation distribution function (fODF) in each voxel. Tractography techniques are then used to create a brain connectivity map from these local information. Early studies relied on the diffusion tensor imaging (DTI) model, which modeled the anisotropic diffusion in each voxel with a single tensor (Basser et al., 1994). Limitations of this model were evident from the outset, as it failed to accommodate voxels with multiple fascicles. Such voxels account for a large fraction of brain white matter (Alexander et al., 2001). Accurate tractography and connectivity analysis require more flexible models that can estimate the number and orientations of multiple fascicles in a voxel.

Existing methods for estimating the number and orientations of fascicles in multi-fascicle voxels can be divided into parametric and non-parametric methods (Seunarine and Alexander, 2014). Parametric methods model the diffusion signal as the sum of signals from different compartments and estimate the model parameters using a model fitting method. This is usually performed with the use of nonlinear optimization techniques. Non-parametric methods, on the other hand, typically estimate the probability distribution of diffusion or of fiber orientations on a sphere. Prominent peaks of the estimated distribution are assumed to correspond to major fascicles. However, both techniques have important shortcomings (Seunarine and Alexander, 2014). Parametric methods suffer from local minima and sensitivity to initialization as they involve solving a non-convex optimization problem. Furthermore, determining the correct number of fascicles in each voxel is not straight-forward and depends on additional methods, which are often complicated and computationally expensive (Schultz et al., 2010; Scherrer et al., 2013). For non-parametric models, the choice of the right representation for the distribution on sphere is unclear. Most methods use linear representations, which are not sufficiently complex (Seunarine and Alexander, 2014). Moreover, they often produce spurious peaks that can be hard to distinguish from the true ones (Tournier et al., 2008).

A less common but promising class of methods are data-driven and machine learning-based techniques. With the growing diversity and power of machine learning models and techniques, these methods have been gaining more attention in recent years. Several recent studies have attempted at estimating scalar diffusion parameters such as diffusion kurtosis measures and generalized fractional anisotropy using deep learning (Golkov et al., 2016; Gibbons et al., 2019; Ye et al., 2019; Aliotta et al., 2019). They have shown that deep learning methods can accurately estimate such parameters from highly under-sampled q-space data. However, the above-mentioned studies do not address estimation of the number and orientations of fascicles, which is the aim of this paper.

A number of studies have proposed machine learning methods for estimating the number and/or orientations of major fascicles or for estimating the complete fODF. Support vector regression (SVR) was used to estimate the number of fascicles in each voxel by Schultz (2012). Another study used convolutional neural networks (CNNs) for the same purpose (Koppers et al., 2017b). Another recent study proposed using CNNs for estimating the orientations of fascicles (Koppers and Merhof, 2016). However, they stacked one-dimensional diffusion measurements in an artificial manner to synthesize 2D input signals for their CNN. Other CNN-based methods work on small patches of DW-MRI images as input (Lin et al., 2019; Koppers et al., 2017a). In a different method, fODF was estimated in a sparse signal reconstruction framework, wherein a deep learning model was used to estimate the sparse reconstruction coefficients (Ye and Prince, 2017). One study learned an fODF prior using auto-encoders and incorporated this prior within more traditional optimization-based fODF estimation techniques (Patel et al., 2018). Some studies represent the diffusion signal and/or the fODF in spherical harmonic bases and use machine learning methods to estimate the coefficients of the fODF from those of the diffusion signal (Nath et al., 2019; Nat et al., 2019). The latter methods used ground truth training data with histological tracing of fiber orientations, which is very hard to come by.

Recently, several machine learning methods have also been proposed for tractography and white matter bundle segmentation (Poulin et al. (2019), Wasserthal et al. (2018a)). There is considerable diversity in terms of models used by different studies. Examples include random forests (Neher et al., 2015, 2017), convolutional neural networks (Wasserthal et al., 2018b), multi-layer perceptrons (Jörgens et al., 2018), and gated neural networks (Benou and Raviv, 2019; Poulin et al., 2017). Moreover, there is substantial variability among these studies in terms of problem formulation, data pre-processing, and the model output. For example, some of these works perform single-fiber tracking by formulating the problem as either a classification or regression task (Neher et al. (2017), Poulin et al. (2018), Jörgens et al. (2018)), while some other studies employ more complex machinery to segment complete bundles (Wasserthal et al. (2018a), Wasserthal et al. (2019)). The work of Wasserthal et al. (2019), for example, employs three separate CNN models to estimate tract orientations in each voxels, to predict bundle segmentation maps, and to predict the tract start and end voxels. These three CNN models work together to produce accurate segmentations of different bundles.

In this paper, we propose a novel machine learning-based method for estimating the number and orientations of fascicles in each voxel from DW-MRI measurements. Our method makes use of a deep learning model. However, it is quite different from all prior works. Unlike parametric methods that aim to optimize all model parameters jointly, and unlike non-parametric methods that estimate the orientation distribution function on the entire sphere at once, our method uses all measurements in a voxel to estimate one single parameter at a time. Specifically, as we explain in detail below, we consider a set of directions on the unit sphere. For each direction in this set, one at a time, we use all DW-MRI measurements in the voxel to estimate the angle to the closest fascicle for that direction. This information can be further processed to estimate the number and orientations of major fascicles or used directly for tractography. Our method can be trained using either simulated or real DW-MRI data. We show that our method achieves results that are comparable with or better than several competing methods.

2. Materials and methods

Figure 1 shows an overview of our method. In brief, for each voxel, we consider a set of directions on the unit sphere. For each direction, we estimate the angle to the closest fascicle. We then process this information to determine the number and orientations of major fascicles in that voxel. We explain the steps of our method below. To simplify the presentation of our proposed method, we assume a single-shell measurement scheme, i.e., only one gradient strength, b.

Figure 1:

A schematic summary of our proposed method for estimating the number and orientations of major fascicles. Given the diffusion measurements in an imaging voxel (a), we consider a set of directions on the unit sphere (b). For each direction on the unit sphere, u, we compute (c) a feature vector $F_u(\theta )$ using Equation (2). We use a multi-layer perceptron, (d), to estimate the angle to the closest fascicle, $\text{Φ}(u)$, based on this feature vector. This is performed separately for each direction on the unit sphere, (e). The estimated angles, $\text{Φ}(u)$, are smoothed using bivariate spline smoothing to obtain a more accurate estimation of the angle to the closest fascicle ${\text{Φ}}_s(u)$, (f). We then threshold ${\text{Φ}}_s(u)$ and perform local minimum extraction to determine the number and approximate orientations of the fascicles, (g). Finally, the estimated orientations are further refined using Karcher mean to obtain a more accurate estimation of fascicle orientations, (h).

2.1. Feature vector computation

Let us denote the set of m DW-MRI measurements in a voxel with ${\left\{s_i\left(q_i,b\right)\right\}}_{i=1}^m$, where the unit vector q_i is the gradient direction for the i^th measurement. Given an arbitrary direction u on the unit sphere, our intermediate goal is to estimate the angle of the closest fascicle to u.

We begin by noting that, assuming axially-symmetric fascicles, the diffusion signal can be modeled as (Anderson, 2005):

$$s_i\left(q_i\right)=s_0\left(f_{\text{iso}}\exp \left(-b{\lambda }_{\text{iso}}\right)+\sum _{k=1}^K{f}_k\exp \left(-b({\lambda }_{\perp }^k+3({\overline{\lambda }}^k-{\lambda }_{\perp }^k){\cos }^2{\alpha }_i^k)\right)\right)$$ (1)

where s₀ is the signal when no diffusion gradient is applied. Also, ${\overline{\lambda }}^k=({\lambda }_{\parallel }^k+2{\lambda }_{\perp }^k)/3$, where ${\lambda }_{\parallel }^k$ and ${\lambda }_{\perp }^k$ denote the axial and radial diffusivities, respectively, for the k^th fascicle, and λ_iso is the diffusivity of the isotropic compartment. Moreover, ${\alpha }_i^k$ is the angle between the k^th fascicle and q_i. Finally, K is the number of fascicles crossing the voxel and f_ks denote the occupancy fraction of each compartment. From the above equation, because all parameters except for α’s are fixed in a voxel, we observe that the signal in any direction is mainly a function of the angles between that direction and the fascicles, with the closest fascicle having the largest influence.

Based on the above argument, we propose the following feature vector to be used for estimating the angle to the closest fascicle for an arbitrary direction u:

$$F_u\left({\theta }_j\right)=\sum _i\omega \left(\angle \left({\theta }_j,q_i\right)\right)s_i\left(q_i\right)/s_0{\theta }_j=j\pi /(2n),j=0:n$$ (2)

where $\theta$ is the angle away from u, and $\angle \left({\theta }_j,q_i\right)$ is the angle between ${\theta }_j$ and q_i. This is simply a weighted average of the diffusion measurements as a function of $\theta$. Please see Figure 2(a) for a schematic illustration. Simply put, for each angle ${\theta }_j$ we consider a cone with that angle around u, and compute the weighted average of the diffusion signal measurements s(q_i), with weights depending on the closeness of q_i to the cone. We used $\omega \propto 1/\left(\angle \left({\theta }_j,q_i\right)+ϵ\right)$ with = 0.1 rad, in order to give larger weights to measurements with q closer to ${\theta }_j$. We compute this feature vector for a set of n + 1 angles ${\theta }_j$ from 0 to $\pi /2$. We set n = 15 in this work. Figure 2(b)–(d) shows example feature vectors for arbitrary directions in voxels with 1, 2, and 3 fascicles.

Figure 2: (a).

This schematic shows how Equation (2) computes the feature vector for a direction u. For simplicity of illustration, the direction u is chosen as the vertical direction in this figure. Geometrically speaking, for each angle θ_j we consider a cone with that angle around u. An example cone is shown with dotted red lines. We compute the weighted average of the diffusion signal, $s\left(q_i\right)$, with weights depending on the closeness of q_i to the cone. The right side of the figure displays feature vectors for 50 randomly selected directions for voxels with 1–3 fascicles ((b)-(d), respectively). The colorbar shows the angle of u to the closest fascicle. In other words, the blue curves are feature vectors for directions u that are very close to a major fascicle, whereas the red curves are for directions that make an angle of almost 90° with the closest fascicle. It is clear that the proposed feature vector, $F_u\left({\theta }_j\right)$, is strongly related to the angle of u to its closest fascicle.

2.2. Estimation of the angle to the closest fascicle

As discussed above and illustrated in Figure 2, the computed feature vector is strongly related to the angle to the closest fascicle. Therefore, we propose to estimate the angle to the closest fascicle based on the computed feature vector. We used a multi-layer perceptron (MLP) for this purpose because MLPs have a high representational capacity and our experiments showed that they produced much more accurate predictions than competing models such as SVR. Based on preliminary cross-validation experiments with phantoms similar to those developed in Scherrer et al. (2013), we decided on a network with six hidden layers, with {30,60,80,80,60,30} neurons in the hidden layers. All layers, except for the last, included rectified linear units as activation functions. The last layer did not have any activation function, i.e., it used a linear activation. The MLP input is the feature vector, F, explained above. Therefore, the input layer is of size equal to the length of the feature vector, i.e., n + 1. The MLP output is a scalar, representing the angle to the closest fascicle.

2.3. Predicting the number and orientations of fascicles

We use the trained MLP to estimate the angle to the closest fascicle for a set of directions uniformly spread on the unit sphere. In this work we used 724 directions, resulting in a resolution of approximately 7°. We denote these predictions with $\text{Φ}(u)$. For each voxel, $\text{Φ}(u)$ has 724 scalar values, one for each of the 724 discrete directions, u, on the unit sphere. We process $\text{Φ}(u)$ in three steps to determine the number and orientations of fascicles in a voxel. These steps are illustrated in Figure 1(e)–(h) and they are explained below.

1. Smoothing to reduce the estimation error in $\text{Φ}(u)$. The error in the estimated angles, $\text{Φ}(u)$, can be reduced by exploiting the knowledge that the estimated angles for nearby directions should be close. For this step, we use bivariate spline smoothing in spherical coordinates (Dierckx, 1995). We used the same spline degree of 3 in all of our experiments. This operation results in a smoother and more accurate estimate of the angle to the closest fascicle, which we denote with ${\text{Φ}}_s(u)$.

2. Thresholding to identify candidate fascicle orientations. Directions u for which the estimated angle to the closest fascicle, ${\text{Φ}}_s(u)$, is close to zero are candidates for fascicle orientations. Hence, we define $u_t=\left\{u\mid {\text{Φ}}_s(u)<t\right\}$ as the candidates. Based on the resolution of our spherical grid (7°) and the accuracy of ${\text{Φ}}_s$ (discussed below), we chose a threshold of t = 30°. However, this procedure generates a large number of candidate fascicle orientations, many of which will be clustered together around the true fascicle orientations. Therefore, within the set u_t we perform a local-minimum extraction to detect the true fascicles. We mark a direction $u^{\ast }\in u_t$ as a local minimum if ${\text{Φ}}_s\left(u^{\ast }\right)$ is not larger than any of its neighbors and smaller than at least one of its neighbors. In this work, we used the routines available in the DIPY package (Garyfallidis et al., 2014) for local minimum extraction.

3. Estimating fascicle orientations. From the above step we obtain a set of estimated fascicle orientations, $\left\{u^{\ast }\right\}$. The number of vectors in this set is our estimate of the number of major fascicles in the voxel. One can propose the directions of $\left\{u^{\ast }\right\}$ as the estimated fascicle orientations. However, we can improve the estimated fascicle orientations by exploiting not only the directions of local minima, but also the nearby directions in u_t. Therefore, we further improve the estimated fascicle orientations as follows. For each $u_i^{\ast }\in \left\{u^{\ast }\right\}$, we identify all candidate directions in u_t that are closer to $u_i^{\ast }$ than to any other direction in $\left\{u^{\ast }\right\}$. Let us denote the set of these directions with $\left\{u_t^i\right\}$. We compute the Karcher mean (Pennec, 1999) of the directions in the set $\left\{u_t^i\right\}$ to obtain a more accurate estimate of the orientation of the fascicle $u_i^{\ast }$.

2.4. Compared methods and implementation details

We compared our method with several other methods explained below.

• Bayesian method of Behrens et al. (2007, 2003). This method uses automatic relevance determination (ARD) to estimate the number of major fascicles. All model parameters including the orientation of major fascicles are estimated in a Bayesian framework using Metropolis Hastings Markov Chain Monte Carlo sampling.

• Multi-tensor model fitting with F-test for model selection. F-test is a well-known statistical model selection method that has been used by previous studies in DW-MRI modeling (Alexander et al., 2002; Kreher et al., 2005; Scherrer and Warfield, 2012). In this work, we used it for selecting the number of fascicles in a multi-tensor model with an additional isotropic compartment, similar to Equation (1).

• Multi-tensor model fitting with selection of the number of fascicles based on generalization error. We used the method proposed by Scherrer et al. (2013). This method is based on representing each of the fascicles with a tensor. The number of fascicles is determined by estimating the generalization error using the 632+ bootstrap technique (Efron and Tibshirani, 1997).

• Constrained spherical deconvolution (CSD) (Tournier et al., 2007). This method estimates the fODF on a spherical grid. To be consistent, we used the same sphere as that used for our proposed method, which included 724 directions. In order to estimate the number of major fascicles from the estimated fODF, we used the method proposed by Schultz et al. (2010). This method depends on a set of thresholds to decide whether two or more fascicles exist in a voxel. We determined these thresholds using cross-validation on a digital phantom with known ground-truth.

• Sparse Fascicle Model (SFM) (Rokem et al., 2015). Similar to CSD described above, for SFM we used the method of Schultz et al. (2010) to determine the number of major fascicles.

• Self-oriented Diffusion Basis Functions proposed by Aranda et al. (2013). We refer to this method as SODBF. This is an extension of the method proposed by Ramirez-Manzanares et al. (2007). In brief, this method is based on representing the diffusion signal as a sum of signals originating from a set of candidate compartments, where each compartment is parameterized with a tensor and a set of rotation matrices that determine the angular rotation needed to map that tensor to a set of directions on a spherical grid.

• Method of Schultz (2012) for estimating the number of fascicles in a voxel. This method is based on a support vector regression (SVR) model. The input feature vector to the SVR in this method is the vector of sorted eigenvalues of the diffusion tensor model. The author proposes to combine his method of estimating the number of fascicles with the ball-and-stick model in order to estimate the orientations of fascicles.

• Deep learning methods of Koppers (Koppers and Merhof (2016), Koppers et al. (2017b)). Koppers et al. (2017b) proposed a CNN model for estimating the number of fascicles in each voxel. In a different work, the same authors proposed an ensemble of three CNNs for estimating the orientations of up to three fascicles in each voxel (Koppers and Merhof, 2016). To make their model generalizable to different acquisition gradient directions, they interpolate the diffusion measurements onto a hemispherical grid of size 120 using spherical harmonics. Their three CNNs predict the fascicle orientations as a classification problem on a hemisphere of size 250.

• End-to-end deep learning method of (Lin et al., 2019). This method is based on a CNN that accepts the diffusion signal of a 3 × 3 × 3 block around a voxel of interest and predicts the fODF for that voxel. The diffusion signal as well as the fODF are represented in spherical harmonics bases. We refer the interested reader to (Lin et al., 2019) for details.

Similar to the method of Schultz et al. (2010) that we used with CSD and SFM, both of the tensor fitting methods also involved thresholds for estimating the number of fascicles (Scherrer et al., 2013). We optimized the thresholds using data simulated on a phantom similar to the phantom used by Scherrer et al. (2013). We used the same phantom data to select the hyper-parameters of the Monte Carlo method used for the ARD method of Behrens et al. (2007) and to tune our proposed method, which mainly included setting the value of the threshold t.

We used either simulated or real diffusion data to train our model and the other competing machine learning methods. To generate simulated diffusion data, we used a multi-tensor model according to Equation (1) with K up to 3 and range of diffusivity values ${\lambda }_{\parallel }\in [0.0018,0.0024]$ and ${\lambda }_{\perp }\in [0.00035,0.00050]$. We simulated one million voxels with one, two, and three fascicles, for a total of three million voxels. For each voxel, we considered 724 directions, u, uniformly spread on the sphere and computed the feature vector F_u using Equation (2). The target output of the MLP for each direction u is the angle between u and the major axis of the closest tensor.

For training with real diffusion data, we used scans from the developing Human Connectome Project (dHCP) dataset (Bastiani et al., 2019). Subjects in this dataset are newborns imaged between 27 and 45 weeks gestational age in natural sleep. Diffusion scans in this dataset include 300 measurements with diffusion strengths, b, of 0 (n=20), 400 (n=64), 1000 (n=88), and 2600 (n=128). Some of the scan parameters are as follows: multiband 4x acceleration, SENSE factor 1.2 and Partial Fourier 0.86, acquired resolution 1.5 × 1.5 mm, 3 mm slices with 1.5 mm overlap, TR/TE= 3800/90ms. We applied the Multi-Shell Multi-Tissue Constrained Spherical Deconvolution (MSMT-CSD) (Jeurissen et al., 2014) on the full multi-shell measurements to estimate the fiber response function. We considered the white matter voxels in each scan. For each voxel, we determined the largest fODF peak. We then determined the number of other fODF peaks in that voxel that were larger than 0.50 times the largest peak. If the number of peaks in a voxel was more than 3, we ignored that voxel. We followed this approach based on the results of recent works that have highlighted the limitations of DW-MRI in detecting the number and orientations of multiple fibers (Schilling et al. (2016), Schilling et al. (2018)). Otherwise, we used that voxel to generate training data. Specifically, similar to the simulated data described above, we considered a set of 724 directions, u, and for each direction we computed the feature vector F_u and the angle of u to the closest CSD peak as the input and target output for training the MLP. We performed this on 75 scans from the dHCP dataset, obtaining a total 3.1 million voxels for training. We used a separate set of 20 test scans from the same dataset for testing our method and other methods. On the test scans, we only used the 88 measurements in the b = 1000 shell. To generate the ground-truth on these 20 test scans, we again applied the MSMT-CSD algorithm on the full multi-shell data.

We implemented our method in Python and TensorFlow. The output of the MLP is the estimate of the angle to the closest fascicle. We trained the MLP by minimizing the squared difference between the estimated angle and the ground-truth. Network weights were initialized using the method of He et al. (2015). We used the Adam optimizer (Kingma and Ba, 2014) with a batch size of 1000 and a starting learning rate of 0.0001. For both the simulated and real training data, we set aside 30% of the training data as validation set. We reduced the learning rate by a factor of 0.90 after a training epoch if the loss on the validation set did not decrease. If the validation loss did not decrease in two successive training epochs, we stopped the training. This usually occurred after approximately 200 training epochs. For both simulated and real training data, the training time on a Linux machine with an NVIDIA GeForce GTX 1080 GPU was approximately 8 hours.

2.5. Tractography and semi-quantitative analysis of fiber tracts

In some of our experiments with phantom and real data, we compared different methods in terms of tractography. In those experiments, we used the EuDX algorithm (Garyfallidis, 2013) for fiber tracking. We used a step size of 0.4 mm with all methods. For stopping criteria, we used the white matter mask (provided as part of the dHCP dataset, generated using the DRAW-EM algorithm (Makropoulos et al., 2014)) and the mask of generalized fractional anisotropy (Tuch, 2004) thresholded at 0.25. We found that using both stopping criteria together removed many false streamlines without stopping the true streamlines.

In our experiments with real DW-MRI data, an expert neuroanatomist (LV) compared our method with other competing methods in terms of the quality of generated tractograms. Specifically, we applied our method, SFM, and DTI on 20 subjects from the dHCP datasets and presented the tractography results to our expert. The tractograms generated by the three methods were randomly named “trk 01”, “trk 02”, and “trk 03”. The expert did not know which tractogram was generated by which of the three methods. This evaluation was based on assessment of commissural (corpus callosum and anterior commissure), projection (frontopontine fibers, corticospinal tract, and fornix), association (cingulum, inferior fronto-occipital fasciculus, and uncinate fasciculus), and cerebellar (middle cerebellar pedunculus) tracts. The tracts were first visualized using the default track group created with three coronal (Y) slice filters. The filters were situated in the frontal, central, and parietal regions of the brain. The frontal coronal slice filter was situated in the white matter rostral and adjacent to the most anterior tip of the lateral ventricles and was used to visualize cingulum, corpus callosum, frontopontine fibers, uncinate fasciculi, and inferior fronto-occipital fasciculi. The central slice filter was situated at the level of midline bundle component of the anterior commissure and was used to visualize the corticospinal tract, anterior commissure, and fornices. The parietal slice filter was situated in the white matter caudal and adjacent to the posterior tip of the splenium of the corpus callosum and was used to visualize middle cerebellar pedunculi and cingulum.

Next, the white matter tracts were visually inspected and graded based on their integrity (i.e., all components of the tract being visible) and bilateral presence. Grade 3 was characterized by the presence of all components of the given tract bilaterally. Grade 2 was characterized by the absence of at least one component of the given tract regardless of the side. Grade 1 was characterized by the complete absence of the tract in both hemispheres of the brain. One exception to this grading rule was the corticospinal tract. For the corticospinal tract, we assessed only its caudal part, coursing between cerebral peduncles and decussation of pyramids, which we treated as a separate tract.

3. Results and Discussion

3.1. Preliminary cross-validation experiments

We performed the preliminary validation of our proposed method on a digital phantom similar to that used by Scherrer et al. (2013). This is a 15×15-voxel phantom with 1, 2, or 3 fascicles per voxel. In these experiments, the root mean square of the error in the angle to the closest fascicle estimated by our MLP was 7.0, 7.4, and 8.2 degrees for voxels with 1, 2, and 3 fascicles, respectively. After refining the estimations with the steps explained in Section 2.3, these errors decreased to 3.2, 3.7, and 5.7 degrees, respectively. For detecting voxels with 1, 2, and 3 voxels, our method achieved accuracy of 0.98, 0.98, and 0.90, respectively.

3.2. Evaluation on the HARDI-2013 phantoms

One of the phantoms in this dataset contains voxels with up to five crossing fascicles (Caruyer et al., 2014). We used the HARDI scheme of this data, which included 64 gradient directions at b = 3000. The diffusion signal for this phantom was simulated using a composite hindered and restricted model (Assaf and Basser, 2005), with a diffusion tensor for the extra-axonal compartment and a cylinder for the intra-axonal compartment (Caruyer et al., 2014). The effect of longitudinal and transverse relaxation on the signal decay were taken into account. To account for spatial variability and correlations, T1 and T2 relaxation times for each voxel were sampled from random normal distributions $\mathcal{N}$(832,10) and $\mathcal{N}$(79.6,0.6), respectively, with a correlation of 0.20 between neighboring voxels (Caruyer et al., 2014). To train our MLP and the other learning-based methods, we simulated data using the multi-tensor model (Equation (1)). We used the parameter settings optimized in the training step above for all methods including ours.

Table 1 shows the comparison of different methods in terms of accuracy, sensitivity, and specificity in detecting voxels with one, two, and three fascicles. For this evaluation, we considered voxels for which the angle between neighboring fascicles was as low as 30◦ and fraction of each fascicle was as low as 0.15. Our method achieved higher accuracy values than all other methods in detecting voxels with one, two, and three voxels. In terms of sensitivity and specificity, our method achieved results that were better than or competitive with other methods. In detecting voxels with 2 or 3 fascicles our method performed much better than all other methods. Most other methods completely failed in detecting voxels with three fascicles, whereas our method was accurate. Method of Lin et al. (2019) was more accurate than other methods, except for our proposed method on voxels with two and three fascicles. As we described above, this is an end-to-end deep learning method that uses the diffusion signal in a 3 × 3 × 3-neighborhood of the voxel of interest. One interesting observation from this table is that overall the machine learning-based methods have achieved higher sensitivity in detecting voxels with three fascicles than the non-learning methods.

Table 1:

Accuracy, sensitivity, and specificity of different methods in detecting voxels with 1–3 fascicles. The best results in each column have been marked using bold type. Overall, our method achieved the best performance.

	One fascicle			Two fascicles			Three fascicles
Method	Accuracy	Sensitivity	Specificity	Accuracy	Sensitivity	Specificity	Accuracy	Sensitivity	Specificity
SFM	0.62	1.00	0.32	0.74	0.42	1.00	0.87	0.00	0.00
CSD	0.90	1.00	0.85	0.81	0.83	0.67	0.82	0.43	1.00
Tensor fitting- F-test	0.93	0.94	0.93	0.68	0.96	0.51	0.67	0.00	0.00
Tensor fitting- bootstrap	0.88	0.66	0.99	0.55	0.98	0.43	0.67	0.00	0.00
ARD	0.92	0.87	0.94	0.74	0.90	0.58	0.78	0.38	0.87
SODBF	0.94	0.96	0.94	0.70	0.71	0.68	0.58	0.40	0.67
Schultz (2012)	0.80	0.77	0.84	0.64	0.75	0.50	0.55	0.51	0.57
Koppers and Merhof (2016)	0.92	0.85	0.93	0.70	0.72	0.68	0.69	0.59	0.77
Lin et al. (2019)	0.83	0.86	0.80	0.86	0.80	0.88	0.81	0.78	0.90
Proposed method	0.97	0.99	0.96	0.93	0.98	0.85	0.92	0.75	0.98

We also compared the methods in terms of their accuracy in estimating fascicle orientations by computing the weighted average angular error (WAAE) proposed by Schultz (2012). WAAE is defined for each voxel as $\text{WAAE}={\sum }_i{w}_i\underset{j}{\min }\arccos \left(\left|v_i,{\widehat{v}}_j\right|\right)$, where v and $\widehat{v}$ denote the true and estimated fascicle orientation direction vectors and w_i is the true occupancy fraction of the i^th fascicle. In other words, WAAE weights the errors in estimating fascicle orientations in a way that more prominent fascicles are assigned larger weights. Table 2 reports the values of WAAE separately for all voxels containing one, two, and three fascicles in this phantom. For voxels with two or three fascicles, our method achieved more accurate estimation of fascicle orientations than all other methods, including the other three machine learning methods. For voxels with one fascicle, our method was slightly less accurate than some of the methods such as CSD, SFM, and method of Lin et al. (2019). However, it is likely that fascicle orientation estimation accuracy in voxels with multiple fascicles is more important for the purposes of tractography and connectivity analysis. It is also noteworthy that a finer spherical grid, i.e., a larger number of directions u on the unit sphere, would increase the spatial resolution, which may further improve the accuracy of our method in estimating fascicle orientations.

Table 2:

Comparison of different methods in estimating fascicle orientations in terms of WAAE, in degrees, in voxels with 1–3 fascicles. The best results in each column have been marked using bold type.

Method	One fascicle	Two fascicles	Three fascicles
SFM	2.99	11.7	17.8
CSD	2.85	11.8	12.8
Tensor fitting- F-test	5.76	10.6	15.4
Tensor fitting- bootstrap	3.00	10.2	15.1
ARD	4.58	9.31	12.8
SODBF	4.40	11.35	14.0
Schultz (2012)	4.54	13.20	16.4
Koppers and Merhof (2016)	4.28	11.92	13.0
Lin et al. (2019)	3.93	10.4	12.0
Proposed method	4.05	9.23	10.9

For another phantom in this dataset, 20 pairs of seed and target tractography regions of interest (ROIs) are known. On this phantom, we applied different methods to estimate the number and orientations of major fascicles or the fODF in each voxel, followed by tractography. For tractography, we launched one streamline from each seed voxel and computed the fraction of the streamlines that ended within a distance of at most two voxels from a corresponding target voxel. We refer to this fraction as success ratio. Table 3 compares different methods in terms of success ratio. Our method achieved a higher average success ratio than all other methods. SFM, CSD, and the methods of Koppers and Merhof (2016) and Lin et al. (2019) also achieved high success ratios in this experiment.

Table 3:

Mean, standard deviation and range of success ratio for HARDI-2013 phantom tractography.

Method	success ratio
	mean ± std,	[min, max]
SFM	0.59 ± 0.26,	[0.00,0.96]
CSD	0.58 ± 0.20,	[0.09,0.93]
Tensor fitting- F-test	0.47 ± 0.22,	[0.09,0.90]
Tensor fitting- bootstrap	0.49 ± 0.23,	[0.05,0.88]
ARD	0.56 ± 0.22,	[0.06,0.88]
SODBF	0.57 ± 0.22,	[0.00,0.84]
Schultz (2012)	0.50 ± 0.24,	[0.00,0.90]
Koppers and Merhof (2016)	0.58 ± 0.18,	[0.08,0.93]
Lin et al. (2019)	0.58 ± 0.15,	[0.06,0.90]
Proposed method	0.63 0.21,	[0.12,0.93]

3.3. Evaluation of estimated fODF with real DW-MRI data

We applied our method, CSD, method of Lin et al. (2019), and method of Koppers and Merhof (2016) on 20 subjects from the dHCP dataset (Bastiani et al., 2019). We applied each of these methods on either the diffusion measurements in the b = 1000 shell or the measurements in the b = 2600 shell. In the absence of ground truth, we first compared different methods visually in terms of the reconstructed fODF with the fODF estimated by MSMT-CSD using the diffusion measurements in all three shells. For our method, we propose $\text{fODF}(u)=1/{\text{Φ}}_s{(u)}^p$, where we set p = 2 in the results shown below. The justification for this definition is simple; directions u that are closer to major fascicles should have a higher fODF value. Increasing p makes the fODF pointier without changing the directions of the peaks. Note that our above-defined fODF does not posses all of the properties of an ideal fODF. Ideally, an fODF should represent a continuous fiber density probability distribution that incorporates such factors as fiber orientation dispersion and anisotropy. Although our definition is not guaranteed to follow these requirements, it does represent the number and orientations of major fascicles. A representative example of reconstructed fODFs in an area of crossing fibers has been shown in a coronal slice of a dHCP subject in Figure 3. In this example, we applied our method, CSD, and method of Lin et al. (2019) on the measurements in the b = 2600 shell. Overall, we observed that compared with CSD, the reconstructions of our method and method of Lin et al. (2019) were closer to the ground truth reconstructed with MSMT-CSD using full multi-shell measurements.

Figure 3:

An example of the fODF reconstructed by our method, CSD, and method of Lin et al. (2019). Top row: left image shows the corresponding slice from the T2-weighted image with the ROI marked in red; middle image shows the same slice of the color fractional anisotropy image; right image is the “reference” fODF reconstructed with MSMT-CSD using the measurements in all three shells. Bottom row: fODFs reconstructed by CSD, method of Lin et al. (2019), and our proposed method. In this example, the three methods in the bottom row were applied on the measurements in the b = 2600 shell.

For a quantitative comparison, we evaluated the robustness of different methods to measurement down-sampling. For each subject, we removed 25% and 50% of the measurements in a shell at random. We performed this experiment separately for the b = 1000 shell and for the b = 2600 shell. We applied our method, CSD, method of Lin et al. (2019), and method of Koppers and Merhof (2016) on these down-sampled measurements and determined the orientation of the most dominant fascicle (i.e., the largest peak of the fODF). We computed the angular error between this orientation and the orientation estimated using all measurements in that shell for all voxels in the white matter. This included a total of more than 1.4 million voxels. Results of this experiment are presented in Table 4. The results show that overall our method is more robust to measurement down-sampling than CSD and the two competing machine learning methods. We performed paired t-tests for this experiment using a significance level of α = 0.01. For the b = 1000 shell, the angular error for our method was significantly smaller than that of CSD (p = 0.0014) and the other two machine learning methods at both 25% and 50% down-sampling rates (p = 0.0005−0.0081). For the b = 2600 shell, at 25% down-sampling rate CSD resulted in lower error, but it was not significantly better than our method (p = 0.184). At 50% down-sampling rate, however, our method resulted in lower error than CSD and other methods and this difference was statistically significant (p = 0.0013 − 0.0073).

Table 4:

Comparison of the proposed method and CSD in terms of error in the estimated orientation of the most prominent fascicle in 20 subjects from the dHCP dataset at two different measurement down-sampling rates.

		Down-sampling by 25%		Down-sampling by 50%
		mean ± std.	maximum	mean ± std.	maximum
b = 1000	CSD	11.8 ± 3.1	22.0	16.6 ± 3.2	22.2
	Koppers and Merhof (2016)	12.4 ± 0.9	14.0	16.5 ± 1.5	20.4
	Lin et al. (2019)	12.1 ± 0.7	14.3	17.3 ± 1.3	22.0
	Proposed method	10.9 ± 0.7	12.6	15.8 ± 0.5	16.9
b = 2600	CSD	10.2 ± 2.6	16.0	15.2 ± 2.1	19.8
	Koppers and Merhof (2016)	13.1 ± 0.9	16.5	15.3 ± 1.2	18.1
	Lin et al. (2019)	11.3 ± 1.2	14.4	14.3 ± 1.0	17.7
	Proposed method	10.4 ± 1.2	14.4	12.8 ± 1.3	15.6

3.4. Evaluation of tractography with real DW-MRI data

We applied our method, SFM, and DTI on 20 subjects from the dHCP dataset and compared the three methods in terms of the quality of the whole-brain tractogram. Overall, as assessed by a neuroanatomist (LV), our method produced more accurate tracts than SFM and DTI. Figures 4, 5, and 6 show examples of tracts on which our method resulted in better tract reconstructions than both SFM and DTI.

Figure 4:

Visualization and scoring of white matter tracts for our method (left column), SFM (middle column), and DTI (right column) using the frontal slice filter (vertical yellow line in the left upper corner of each row). Upper row: Note the integrity and presence of inferior fronto-occipital fasciculus (blue arrow) and uncinate fasciculus (white arrow) when using our method (left column). In contrast, notice the less abundant tract and absence of components of inferior fronto-occipital fasciculus when using SFM (middle column) or DTI method (right column). Bottom row: Note the integrity and bilateral presence of cingulum subdivisions expected to be found in this region of the brain (white arrows in the left column) and the absence of spurious fibers of the body of corpus callosum when using our method (left column). In contrast, notice the unilateral absence (left hemisphere) and thin reconstruction (right hemisphere) of retrosplenial subdivision of the cingulum when using the DTI method (missing third row of white arrows in the right column) and the presence of spurious fibers of the body corpus callosum when using SFM (blue arrow in the middle column).

Figure 5:

Visualization and grading of white matter tracts for our method (left column), SFM (middle column), and DTI (right column) using the central slice filter (vertical yellow line in the left upper corner of each row). Upper row: Note the integrity and bilateral presence of all parts of the fornix (in particular columns of fornices, white arrows) and anterior commissure (middle bundle component shown with blue arrow) when using our method (left column). In contrast, notice the unilateral absence of the middle bundle component of the anterior commissure (blue arrow) when using SFM (middle column) or DTI method (right column). Bottom row: Note the integrity and bilateral presence of the caudal part of the corticospinal tract (white arrows) that reached the decussation of pyramids when using our method (right column). In contrast, notice the unilateral or bilateral absence of the caudal part of the corticospinal tract (white arrows) when using SFM (middle column) or DTI method (right column).

Figure 6:

Visualization and grading of white matter tracts for our method (left column), SFM (middle column), and DTI (right column) using the frontal (vertical yellow line in the left upper corner of the upper row) and parietal slice filters (vertical line in the left upper corner of the bottom row). Upper row: Note the integrity and bilateral presence of the frontopontine fibers (white arrows) when using our method (left column). In contrast, notice unilateral or absence of the frontopontine fibers (white arrows) when using SFM (middle column) or DTI method (right column). Bottom row: Note the integrity and bilateral presence of all the components of the cingulum (in particular beneath the isthmus of the cingulate gyrus, white arrows) when using our method (left column). In contrast, notice the unilateral absence of components of cingulum (white arrows) when using SFM (middle column) or DTI method (right column). Similarly, notice the bilateral presence of middle cerebellar pedunculi when using our method (left column), and unilateral or (middle column) bilateral absence (right column) when using SFM or DTI.

As we explained in Section 2.5, our expert evaluation of tractograms involved assigning a grade of 1, 2, or 3 to each of 12 separate tracts. For each subject, we summed the scores received by each method on the 12 tracts to arrive at a single overall score in the range [12,36] for the entire tractogram. On 9 of the 20 subjects, our method obtained a higher score than SFM and DTI. On 5 and 2 of the subjects, respectively, SFM and DTI obtained higher scores. On 3 and 1 of the subjects our method tied as best score with, respectively, SFM and DTI. For a statistical analysis, we applied the Wilcoxon signed-rank test. At a p value of 0.01, the score obtained by our method was significantly higher than both SFM and DTI, and the score of SFM was also significantly higher than DTI.

Different aspects of the methods proposed in this work may be improved and extended in future works. One important aspect is the generation of training data. One approach for generating reliable diffusion data and corresponding brain micro-structure is diffusion-weighted MRI scan followed by tissue dissection. This approach can generate small sets of high-quality data at high cost. In this work, we instead followed two different approaches. In one approach, we applied MSMT-CSD on high-quality multi-shell data with 300 measurements to estimate the ground-truth peaks in each voxel. In the other approach, we used a multi-tensor model to synthesize training data. None of these methods are perfect. With regard to our first approach, recent works have shown the limitations of spherical deconvolution methods for micro-structure estimation (Canales-Rodríguez et al. (2019), Schilling et al. (2018)). Work of Canales-Rodríguez et al. (2019) revealed the limitations of spherical deconvolution methods and work of Schilling et al. (2018) demonstrated the limitations of current diffusion-weighted MRI scanning protocols and reconstruction methods in resolving crossing fibers. The number of diffusion measurements in our in-vivo dataset were larger than the number of measurements used in those works. Nonetheless, DW-MRI reconstruction algorithms (including MSMT-CSD used in our work) are ultimately limited in resolving crossing fibers. Some of the decisions that we made in generating our training data were based on the results reported in those works and in order to reduce the less reliable data. For example, based on the results reported by Schilling et al. (2018), we ignored voxels with more than three peaks, peaks that were very small, and peaks that were too close together. By removing such data, our aim was to exclude less reliable data samples. Nonetheless, future works may improve our methods by proposing more effective methods for generating the training data and estimating the reliability of each data sample. Our second approach, based on a multi-tensor model, has been employed by many prior works (e.g., Canales-Rodríguez et al. (2019)) as it is a simple model that can be used to generate large datasets. However, more elaborate methods have been explored by prior studies (Hall and Alexander (2009), Rafael-Patino et al. (2020), Lee et al. (2020)). In our work, in addition to the multi-tensor model, we also experimented with the more complex CHARMED model (Assaf and Basser, 2005) to generate our training data, but did not obtain higher accuracy at test time. Nonetheless, other more accurate models of diffusion signal generation may be explored in future works.

Similar to other machine learning methods (e.g., Lin et al. (2019) and Koppers and Merhof (2016)), our model needs to be trained from scratch if the b-value used for data acquisition changes. However, our method is invariant to the number and directions of measurements in a shell. In other words, our method does not need to be re-trained if the number and/or orientations of diffusion gradients used for image acquisition change. Moreover, one can change the resolution of the spherical grid for estimating fascicle orientations without the need to re-train the MLP. As we mentioned in Section 2.3, in our experiments we used a grid of size 720. In our experience, increasing the resolution of the spherical grid only slightly improves the accuracy of the method. An experimental comparison of three different grid sizes with the phantom test data has been presented in Table 5. As shown in this table, reducing the grid size to 240 can significantly reduce the accuracy of the method. However, increasing the grid size to 2400 did not substantially improve the accuracy of the method.

Table 5:

An experimental evaluation of the effect of grid size on the accuracy of the proposed method. In this table, “grid resolution” is the average of the angle between two neighboring directions in the grid. Accuracy and WAAE refer to the same values reported in Tables 1 and 2

		One fascicle		Two fascicles		Three fascicles
spherical grid size	grid resolution	Accuracy	WAAE	Accuracy	WAAE	Accuracy	WAAE
240	12.5°	0.90	4.47	0.85	11.2	0.84	12.1
720	7.2°	0.97	4.05	0.93	9.23	0.92	10.9
2400	3.8°	0.97	4.01	0.95	9.10	0.92	10.6

We think one of the main reasons for the success of our method is the effectiveness of the proposed feature vector. As we described in Section 2.1, for each direction u, our proposed feature vector $F_u(\theta )$ quantifies the change in the diffusion signal as a function of the angle $\theta$ away from u. For each angle $\theta$, this feature vector was computed using a simple weighted averaging, where we used an inverse weighting, i.e., $\omega \propto \frac{1}{\angle \left({\theta }_j,q_i\right)+ϵ}$. As shown in Figure 2, this feature vector is highly correlated with the angle to the closest fascicle for voxels with one and more than one fascicles. We experimented with other schemes such as inverse-squared weighting $(\omega \propto \frac{1}{\angle {\left({\theta }_j,q_i\right)}^2+ϵ})$, but did not achieve better results. Nonetheless, we think other feature vector designs may be worth pursuing in future works.

4. Conclusions

In this work we proposed a novel machine learning method for estimating the number and orientations of fascicles in each voxel from DW-MRI measurements. Our method uses all diffusion measurements in a voxel to estimate the angle to the closest fascicle for every direction in a set of directions on the unit sphere. Importantly, this is done separately for each direction. This distinguishes our method from all previous methods that estimate all of their unknown parameters or the distribution function at the same time. For example, for a tensor fitting approach, even with a simplified tensor model as in Equation (1), a three-tensor model will have 16 unknown parameters. Tensor-fitting methods solve a non-convex optimization problem to estimate these parameters, which can produce sub-optimal solutions and be sensitive to initialization. This is the main advantage of our method, leading to its superior performance. Our method proved to be more accurate than several competing methods in detecting voxels with two or three fascicles, which are highly important for tractography and connectivity analysis. Our method is also accurate in estimating fiber orientations, which are also important in tractography. Our experiments with real data also showed that our method was more robust than other methods to the reduction of the number of measurements. This makes our method a better choice for applications such as fetal imaging where it is difficult to acquire a large number of high-quality measurements due to fetal movements as well as total acquisition time constraints. In experiments with real data, detailed expert evaluations showed that our method was able to accurately reconstruct various brain tracts. Overall, our experimental results show that the method proposed in this study can be used for accurate estimation of the number and orientations of major fascicles and, hence, for accurate tractography.

Highlights.

We propose a data-driven method for determination of orientations of brain fascicles.

We compute feature vectors that represent signal attenuation with orientation.

An MLP uses the feature vector to predict the angle to the closest fascicle.

Our method accurately estimates the number and orientations of fascicles in a voxel.

Our method also leads to superior tractography, assessed by experts.