Direct Segmentation of the Major White Matter Tracts in Diffusion Tensor Images

Abstract

Diffusion-weighted images of the human brain are acquired more and more routinely in clinical research settings, yet segmenting and labeling white matter tracts in these images is still challenging. We present in this paper a fully automated method to extract many anatomical tracts at once on diffusion tensor images, based on a Markov random field model and anatomical priors. The approach provides a direct voxel labeling, models explicitly fiber crossings and can handle white matter lesions. Experiments on simulations and repeatability studies show robustness to noise and reproducibility of the algorithm, which has been made publicly available.

1. Introduction

Diffusion-weighted imaging (DWI) has become a major tool for the in vivo study of connectivity in the human human brain in health and disease (Bihan et al., 2001; Basser and Jones, 2002). Clinical researchers have been increasingly interested in using DWI in their studies, and have widely adopted diffusion tensor derived indices of white matter (WM) integrity such as fractional anisotropy (FA) and mean, parallel and perpendicular diffusivity (Horsfield and Jones, 2002). Furthermore, recent works have shown that a tract-based analysis of the white matter has the promise to offer a deeper insight into its characteristics in health and disease (Reich et al., 2006; Yushkevich et al., 2008; O’Donnell et al., 2009).

Obtaining an anatomical segmentation of white matter tracts, however, has proven to be difficult for several reasons. DWI tractography, which provides information about the connections between remote regions of the brain (Mori and van Zijl, 2002), does not directly lead to the characterization of fiber bundles (tracts) corresponding to known anatomy, and requires grouping, trimming, and labeling of the numerically computed fibers (Mori et al., 2005; Kouby et al., 2005; O’Donnell and Westin, 2007; Maddah et al., 2007; Lawes et al., 2008; Ziyan et al., 2009; Mayer et al., 2011). Fiber tracking reliability can further vary with imaging resolution, noise and patient orientation (Wakana et al., 2007) as well as decreased anisotropy or lesions that occur with disease, and more advanced methods (Tuch, 2004; Behrens et al., 2007; Jones, 2008; Descoteaux et al., 2009; Reisert et al., 2011) require longer imaging times, often impractical for clinical imaging. Finally, the presence of pathology influences many of these methods and may confound the results in studies of diseases involving WM lesions (Reich et al., 2010).

Alternative approaches have been proposed to represent and study the white matter, most prominently a skeleton representation built on the FA map (Smith et al., 2006) and a region-based parcellation (Mori et al., 2008). Both methods rely on deformable registration strategies to align the diffusion images to labeled templates or group averages (Yang et al., 2008; Ceritoglu et al., 2009). These methods provide indirect information about the tracts and may be misled by WM lesions, which disrupt the FA map and provide undesirable features for registration algorithms to match between subjects.

In this work, we present a new approach to fiber tract segmentation capable of accurate performance on routine clinical acquisitions. With an atlas-based Markov random field (MRF) representation, we derive a fast, scalable estimation algorithm to reliably segment most of the major anatomical fiber tracts within the human brain. The approach bypasses tractography and associated issues and is therefore considered a “direct” tract segmentation approach. Our approach models overlapping and crossing fibers and robustly handles the presence of noise and white matter lesions without manual initialization or pre-filtering. The method was tested and validated on simulations and repeatability studies, and has been successfully applied to the analysis of WM tracts in multiple sclerosis (MS). The algorithm (coded in the cross-platform Java programming language) together with its statistical tract atlas are available for free on the NITRC.org website.

1.1. Related Works

Previous methods for direct tract segmentation include a method based on level sets (Lenglet et al., 2006), one using non-parametric fuzzy classification (Awate et al., 2007), and tensor-space clustering approaches (Goh and Vidal, 2008; Rodrigues et al., 2009). These methods required the specification of initial regions of interest (ROIs) corresponding to specific tracts of interest. Some probabilistic tractography methods evaluate the connection strength between regions or voxels (Parker and Alexander, 2003; Fletcher et al., 2007; Yendiki et al., 2008); however, the connection pathways they can define are not necessarily identifiable as known anatomical tracts.

Recent studies indicate that multi-atlas registration combined with image-based segmentation approaches have improved accuracy (Heckemann et al., 2010). Indeed, statistical atlas-based approaches incorporated in voxel classification techniques (Maddah et al., 2008; Hagler et al., 2009) or in parametric deformable models (Eckstein et al., 2009) have shown some promising results for segmenting anatomical tracts. In particular, (Hagler et al., 2009) proposed to use a statistical atlas essentially equivalent to the one presented here. However, these methods have so far considered each fiber tract as a separate structure without explicitly addressing either joint segmentation or crossing tracts. MRFs have been previously considered for regularizing diffusion tensors (Poupon et al., 1998), but have not been used for segmentation. Our work brings together MRF modeling and atlas-based segmentation in a global and computationally efficient algorithm that explicitly handles fiber crossings and lesions in clinical-quality images.

2. Methods

Our segmentation method combines the global information of location and direction contained in a probabilistic atlas of known white matter tracts with the local diffusion information obtained from the image within a MRF model (Li, 2009). In a MRF, the conditional probability distribution for the labels l at x is entirely determined by the conditional probability distribution in their neighborhood. Due to the Hammersley-Clifford theorem, the joint distribution follows a Gibbs distribution:

$$P(\{l\})=Z^{-1}\text{exp}\frac{1}{T}\left({\displaystyle \sum _x}{V}_1(x,l)+{\displaystyle \sum _{x,y}}{V}_2(x,y,l,m)\right),$$ (1)

for a MRF including a unary term V₁(x, l) and a pairwise term V₂(x, y, l, m), with y and m the location and label of a voxel in the neighborhood of x, l, T a scaling parameter and Z a normalization factor. The terms V₁ and V₂ are often referred to as energy terms, and we will also use U(x, l) = V₁(x, l) + ∑_y,m V₂(x, y, l, m) to denote the overall energy at voxel x. In our model, the unary term V₁(x, l) combines the tensor direction and diffusion characteristics of the data with the tract atlas to encode the likelihood of each label l at x, and the pairwise term V₂(x, y, l, m) encodes the local connections between neighboring diffusion tensors.

As it is well known that there is extensive overlap and crossing between multiple tracts at the current voxel resolution in diffusion MRI, it would be unrealistic to expect a given voxel to contain a single tract everywhere, and so our set of labels {l} must not only include labels for each separate tract but also overlapping tracts within a single voxel. To avoid combinatorial explosion, we limit ourselves to a maximum of two tracts per voxel and exclude unlikely pairs (see Section 2.4). The approach can in principle be extended to multiple overlaps, provided that the underlying diffusion information is sufficient to differentiate multiple crossings from isotropic regions.

To obtain a tract segmentation, we need to first model V₁ and V₂ as a function of the diffusion data for the possible labels and then to maximize P({l}) with respect to the label field {l}. The resulting segmented tracts are represented by membership functions (probabilistic assignments at each voxel) and can be analyzed in a variety of ways including computation of tract volume and determining average diffusion properties (e.g., FA, MD).

Our segmentation algorithm, referred to as Diffusion-Oriented Tract Segmentation (DOTS), is organized as follows. First, the diffusion weighted images are combined and pre-processed to obtain tensor images (Section 2.1). DOTS starts by registering our probabilistic atlas (described in Section 2.2) to the DTI data, and then using the MRF model to estimate the posterior probability of the voxel label given the data (Section 2.3). The model incorporates information about the local anisotropy (Section 2.3.1), the connectivity between neighboring voxels (Section 2.3.2) and the statistics of the atlas (Section 2.3.3). Section 2.3.4 brings all the different components of the MRF into the probability function P({l}) to be maximized by the algorithm. Additional optional modeling is provided to exclude anatomically irrelevant crossing configurations (Section 2.4) and to handle white matter lesions (Section 2.5). Additional implementation details are given in Section 2.6.

2.1. Diffusion-weighted image pre-processing

To process a series of diffusion-weighted images, we first perform a tensor reconstruction to get a set of diffusion eigenvectors and associated eigenvalues {υ⃗_n(x), λ_n(x)}_{1, N} at each voxel x with the standard linear reconstruction method (Basser and Jones, 2002). The images are first aligned to the b0 image and corrected for distortions against a structural image within the CATNAP software (Landman et al., 2007). Extra-cranial tissues are removed with an automatic skull-stripping method (Carass et al., 2007) applied to a co-registered structural image or using a semi-automatic method (Bazin et al., 2007) applied to the mean diffusivity image if a structural image is not available.

2.2. White Matter Tract Atlas

Our statistical atlas is based on the atlas of Mori and Wakana (Mori et al., 2005). Following the atlas tract definitions and delineation protocols, we obtained a set of semi-manual delineations of major tracts, which we augmented with additional expert delineations of a few other tracts of interest. All tracts were delineated using tractography and multiple regions of interest (ROI) selection in DTI Studio.¹ Our standard atlas currently includes the following fiber tracts (separated for left and right when applicable): anterior, superior, and posterior thalamic radiations, corpus callosum and tapetum, inferior and superior longitudinal and fronto-occipital fasciculi, cingulum, fornix, uncinate fasciculus, optic tract, optic radiation, cortico-spinal and cortico-pontine tracts, medial lemniscus, inferior, middle, and superior cerebellar peduncles (see Table 1).

Table 1.

Tract Labels

Acronym	Name
ATRL ATRR	Anterior thalamic radiation (left, right)
CCF CCP CCS	Corpus callosum (frontal, posterior and superior)
CGL CGR	Cingulum (left, right)
CPTL CPTR	Cortico-pontine tract (left, right)
CSTL CSTR	Cortico-spinal tract (left, right)
FNXL FNXR	Fornix (left, right)
ICPL ICPR	Inferior cerebellar peduncle (left, right)
IFOL IFOR	Inferior fronto-occipital fascicle (left, right)
ILFL ILFR	Inferior longitudinal fascicle (left, right)
MCP MLL MLR	Middle cerebellar peduncle Medial lemniscus (left, right)
OPRL OPRR	Optic radiation (left, right)
OPTL OPTR	Optic tract (left, right)
PTRL PTRR	Posterior thalamic radiation (left, right)
SCPL SCPR	Superior cerebellar peduncle (left, right)
SFOL SFOR	Superior fronto-occipital fascicle (left, right)
SLFL SLFR	Superior longitudinal fascicle (left, right)
STRL STRR	Superior thalamic radiation (left, right)
TAP	Tapetum
UNCL UNCR	Uncinate fascicle (left, right)

Our atlas includes both the spatial probability p_l(x) of the existence of label l at voxel x as well as the most probable diffusion direction d̄_l(x) along the tract. Deterministic tractography usually provides very conservative delineations, including only the portion of the tracts that can be reconstructed through multiple ROIs. For this reason, we extended the definition of the tracts in the vicinity of the delineated regions with a smoothing function:

$$\begin{array}{c}{p}_l^t(x)=k\star r_l^t(x)\hfill \\ {\overline{d}}_l^t(x)=\frac{{\displaystyle {\sum }_y}{p}_l^t(y){\overline{d}}_l^t(y)}{{\displaystyle {\sum }_y}{p}_l^t(y)}\forall y\in N(x)\mathrm{s}.\mathrm{t}.p_l^t(y)>p_l^t(x),\hfill \end{array}$$ (2)

where $r_l^t(x)$ is the delineated tract region, $p_l^t(x)$ the smoothed delineation, k a smoothing kernel, ${\overline{d}}_l^t(x)$ the principal direction at voxel x in image t for tract l, and N (x) is the neighborhood defined by the smoothing kernel. Note that the directions are meaningful at first only inside the delineated regions $r_l^t$, and must be extrapolated from higher probability to lower probability regions. The directions are orientation-independent (i.e. d̄ and −d̄ are the same direction), so the sums of directions are performed as follows:

$$\begin{array}{cc}{\overline{d}}_1+{\overline{d}}_2\hfill & ={\overrightarrow{d}}_1+{\overrightarrow{d}}_2\text{ if }{\overrightarrow{d}}_1·{\overrightarrow{d}}_2>0,\hfill \\ \hfill & ={\overrightarrow{d}}_1-{\overrightarrow{d}}_2\text{otherwise}\hfill \end{array}$$ (3)

where d⃗ is the oriented vector corresponding to direction d̄. The averaged direction in Eq. (2) may not have unit norm, unless all the directions are the same. This provides information about the uncertainty in direction at each location in the atlas. In this work we use a linear smoothing kernel of radius 5 mm to construct our atlas. The size of the kernel has some impact on the segmentation, as a larger scale is better at representing tracts with large individual variations but also increases the risk of mislabeling regions with too many overlapping candidates. Our experiments however indicate that a small but reasonable smoothing (5 mm corresponds to about two voxels at the atlas’s original resolution) improves the results.

Examples from the atlas are given in Figure 1. Although this atlas includes most of the major WM tracts in the human brain, the segmentation also includes isotropic regions like the gray matter structures or the ventricles, and some undefined regions of white matter, for instance the “U” fibers that connect neighboring gyri of the cortex. We model the isotropic regions by creating a low anisotropy mask (FA ≤ 0.1) and we capture the undefined white matter regions by subtracting from the opposite mask (FA > 0.1) any region included in one of the tracts defined above.

Figure 1.

Examples from the WM tract atlas: maximum intensity projection along the sagittal direction (except for MCP in the axial direction, and FNX, CPT in sagittal and coronal directions) of the shape prior multiplied by the color-coded direction prior.

2.3. Markov Random Modeling of Diffusion

Given the DTI data and the statistical atlas above, we derive below the terms of the MRF model of Eq. (12): unary terms representing local diffusion properties of the tensor and the contributions of the tract atlas in terms of shape and direction, and a binary term representing connectivity between neighboring tensors.

Our set of possible labels includes the individual tracts listed in Table 1, undefined fiber tracts and isotropic regions, as well as overlapping pairs of tracts. In the following, we describe general properties that apply to any individual tract, group of overlapping tracts or isotropic region independently of the specific label. Let us denote by T, O, and I general attributes of the single tract, overlapping tracts and isotropic regions respectively.

2.3.1. Diffusion Type

At every voxel x, we can have one of three types of structures in this model: isotropic diffusion, diffusion along a single tract, or diffusion along multiple tracts overlapping or crossing. To model the type of diffusion, we use indices of diffusion along a single tract (T), overlapping tracts (O), and isotropic regions (I) derived from the linear, planar and spherical indices of (Westin et al., 2002):

$$d_T(x)=\frac{{\mathrm{\lambda }}_1-{\mathrm{\lambda }}_2}{{\mathrm{\lambda }}_1}(x)d_O(x)=\frac{{\mathrm{\lambda }}_1-{\mathrm{\lambda }}_3}{{\mathrm{\lambda }}_1}(x)d_I(x)=\frac{{\mathrm{\lambda }}_3}{{\mathrm{\lambda }}_1}(x)$$ (4)

Note that water may still diffuse along a single direction for overlapping but aligned tracts, so we cannot differentiate between the two in regions of linear diffusion and d_O ≥ d_T (see Fig. 2b).

Figure 2.

Example data set depicting diffusion type and connectivity functions for single (top) and overlapping tracts (bottom): a) original image color map, b) d_T, d_O, c) s_T, s_O in the X direction, d) s_T, s_O in the Y direction, e) s_T, s_O in the Z direction. The diffusion functions are defined in [0, 1], whereas the connectivity functions are defined in [−1, 1]. Although the functions for single and overlapping tracts appear very similar, there are some subtle differences: d_T is darker in regions of crossing (e.g. the junction between ATR, IFO and CCF), and the single tract connectivity maps s_T appear sharper because they include more negative values (it is more likely to find similar directions when using first and second eigenvectors, as in s_O).

2.3.2. Local tensor connectivity

The evidence for fiber tracts comes from the diffusion tensors: if two tensors are aligned they likely correspond to the same tract or overlapping tracts. The connectivity between neighboring tensors is modeled as follows, assuming a single tract:

$$s_T(x,y)=(1-\text{min}(\theta ({\overrightarrow{\upsilon }}_1(x),{\overrightarrow{\upsilon }}_{\mathit{\text{xy}}}),\theta ({\overrightarrow{\upsilon }}_1(y),{\overrightarrow{\upsilon }}_{\mathit{\text{xy}}})))\times (1-2\theta ({\overrightarrow{\upsilon }}_1(x),{\overrightarrow{\upsilon }}_1(y)))$$ (5)

where υ⃗_xy represents the direction vector between voxels x and y, υ⃗₁(x) and υ⃗₁(y) represent the principal eigenvector directions at x and y, and $\theta ({\overrightarrow{\upsilon }}_1,{\overrightarrow{\upsilon }}_2)=\frac{2}{\pi }$ arccos(|υ⃗₁ · υ⃗₂|). The function θ(υ⃗₁, υ⃗₂) gives the angle between directions υ⃗₁ and υ⃗₂ normalized in [0, 1]. Because it is linear, it is more sensitive to small angular variations than the product υ⃗₁ · υ⃗₂. In some of the following cases, the vectors in this formula have less than unit norm as a way to encode uncertainty in their direction as a lower bound on angles. In such cases, the function θ(υ⃗₁, υ⃗₂) returns values in [α, 1], where $\alpha =\frac{2}{\pi }$ arccos(|υ₁| |υ₂|). Note that other measures involving the complete tensor are possible and they may be interesting to explore in the future.

The similarity function s_T(x, y) is close to +1 when the diffusion directions at x and y are aligned with each other and with the path from x to y. If both diffusion directions are orthogonal to that path, we cannot assume that they are related even if they are aligned: many fiber tracts have “kissing” fibers that follow the same direction before diverging. In such cases, s_T(x, y) goes to zero in order to model the uncertainty. When the diffusion directions are orthogonal and one of them is aligned with the path, then it is clear that both points cannot be part of the same tract, which translates into a negative value up to −1 (see Fig. 2c–e).

For crossing or overlapping fibers, the main diffusion direction can easily switch from υ⃗₁ to υ⃗₂ if the tensor is “flat”, and so we extend the connectivity to consider the secondary directions:

$$\begin{array}{cc}\{{\overrightarrow{\upsilon }}_O(x),{\overrightarrow{\upsilon }}_O(y)\}=\text{argmin}\hfill & \left\{\begin{array}{cc}\theta ({\overrightarrow{\upsilon }}_1(x),{\overrightarrow{\upsilon }}_1(y)),\hfill & \theta (\frac{{\mathrm{\lambda }}_2(x)}{{\mathrm{\lambda }}_1(x)}{\overrightarrow{\upsilon }}_2(x),{\overrightarrow{\upsilon }}_1(y)),\hfill \\ \theta ({\overrightarrow{\upsilon }}_1(x),\frac{{\mathrm{\lambda }}_2(y)}{{\mathrm{\lambda }}_1(y)}{\overrightarrow{\upsilon }}_2(y)),\hfill & \theta (\frac{{\mathrm{\lambda }}_2(x)}{{\mathrm{\lambda }}_1(x)}{\overrightarrow{\upsilon }}_2(x),\frac{{\mathrm{\lambda }}_2(y)}{{\mathrm{\lambda }}_1(y)}{\overrightarrow{\upsilon }}_2(y))\hfill \end{array}\right\},\hfill \\ \hfill s_O(x,y)=& (1-\text{min}(\theta ({\overrightarrow{\upsilon }}_O(x),{\overrightarrow{\upsilon }}_{\mathit{\text{xy}}}),\theta ({\overrightarrow{\upsilon }}_O(y),{\overrightarrow{\upsilon }}_{\mathit{\text{xy}}})))\times (1-2\theta ({\overrightarrow{\upsilon }}_O(x),{\overrightarrow{\upsilon }}_O(y))).\hfill \end{array}$$ (6)

The first part of this equation means that we select the pair of directions at x and y that are best aligned among the first two eigenvectors {υ⃗₁(x), υ⃗₂(x), υ⃗₁(y), υ⃗₂(y)}.

We penalize the υ⃗₂ by $\frac{{\mathrm{\lambda }}_2}{{\mathrm{\lambda }}_1}$ to account for the uncertainty due to the fact it is required to be orthogonal to υ⃗₁. Note also that these directions are not the true fiber directions, which cannot be retrieved with the tensor representation. However, the erroneous tensors will be similar and share directions as long as the two fibers are crossing along similar directions and the diffusion is slightly stronger in one direction. In practice, many regions of overlap between the tracts that we can observe at clinical resolution involve primarily kissing fibers as opposed to crossing ones, for which the principal direction of diffusion as estimated from a tensor reconstruction is still meaningful.

Finally, isotropic regions are assumed to have uniform connectivity s_I(x, y) = s_I, given as a prior parameter. The value of s_I will influence how labels from isotropic regions propagate into anisotropic ones as compared to fibers and overlapping regions. Because all the regions evolve simultaneously, we set s_I = 1/N_t in our experiments, where N_t is the total number of regions in the atlas.

2.3.3. Atlas information

The atlas provides a prior on tract location p_l and tract direction d̄_l, which is integrated in the MRF model through a shape prior term u_l and a direction coefficient c_l as follows.

Because we allow the presence of multiple tracts at a location, the prior probability for a particular label defined by the atlas cannot be used directly. Instead, we define the shape prior term u_l(x) for a single tract l as follows:

$$u_l(x)=p_l(x)·\frac{p_l(x)}{{\displaystyle {\sum }_m}{p}_m(x)},$$ (7)

giving high values where the prior probability p_l(x) is high and likely to be a single tract (∑_m p_m(x) ≈ p_l(x), for all tract labels m).

To compute c_l, we compare the direction prior d⃗_l(x) with the principal direction υ⃗₁(x) of the tensor in the image to segment:

$$c_l(x)=|{\overrightarrow{d}}_l(x)|\left(1-2\theta ({\overrightarrow{\upsilon }}_1(x),\frac{{\overrightarrow{d}}_l(x)}{|{\overrightarrow{d}}_l(x)|})\right).$$ (8)

The coefficient c_l is positive when the directions coincide, and negative when they are orthogonal. The uncertainty on the direction prior, represented by its norm, lowers both positive and negative values of c_l. For two crossing tracts l and m, the shape prior term combines the priors of both objects with the likelihood for two overlapping tracts:

$$u_{l,m}(x)=p_l(x)p_m(x)\frac{p_l(x)+p_m(x)}{{\displaystyle {\sum }_n}{p}_n(x)}.$$ (9)

If the tracts have different orientations, the reconstructed tensor will represent a mixture pointing to a different direction than the individual diffusion tensors for each tract. However, if we assume the true tensors with principal directions vectors d⃗_1,l(x) and d⃗_1,m(x) to be prolate tensors with high anisotropy, we can derive from the Stejskal-Tanner equation that the estimated tensor for the mixture will have for its principal direction the vector d⃗_l,m(x) = d⃗_1,l(x)±d⃗_1,m(x) with largest norm.

The composite tensor direction d⃗_l,m(x) is then renormalized so that its norm is $\frac{|{\overrightarrow{d}}_{1,l}(x)|+|{\overrightarrow{d}}_{1,m}(x)|}{2}$. From this composite vector, we can define the following direction coefficient as we did before in the single tract case:

$$c_{l,m}(x)=|{\overrightarrow{d}}_{l,m}(x)|\left(1-2\theta ({\overrightarrow{\upsilon }}_1(x),\frac{{\overrightarrow{d}}_{l,m}(x)}{|{\overrightarrow{d}}_{l,m}(x)|})\right).$$ (10)

Finally, isotropic regions use the same shape prior as individual tracts and assume no preferred direction, setting $c_I(x)=\frac{1}{2}$.

2.3.4. Propagation of the tract probabilities

Once we have the various elements of diffusion and atlas information above, we can derive the terms of the energy function. From the diffusion and prior-based terms, we derive the following unary term:

$$\begin{array}{c}\hfill V_1(x,l)=d_T(x)u_l(x)c_l(x)\hfill \\ \hfill V_1(x,l,m)=d_O(x)u_{l,m}(x)c_{l,m}(x)\hfill \\ \hfill V_1(x,I)=\frac{1}{2}{d}_I(x)u_I(x)\hfill \end{array}$$ (11)

for single tracts, overlapping tracts and isotropic regions respectively. We combine all the terms as a product since we require all three conditions to be met jointly in order to attribute a label l to a given voxel (unlike with more classical probabilistic models, we cannot separate the terms of V₁ into a sum of conditionally independent elements).

The neighborhood term propagates the unary energy values along the most likely fiber directions or isotropically depending on the type of label. Let x⁺ = argmax_y∈N⁺(x)s_T(x, y), where N⁺(x) is the half of the 26-neighborhood N(x) of x such that υ⃗_xy · υ⃗₁(x) > 0, and similarly x⁻ = argmax_y∈N⁻(x)s_T(x, y), with N⁻(x) such that υ⃗_xy · υ⃗₁(x) ≤ 0. The same definition holds for overlapping tracts, substituting s_O(x, y) to s_T(x, y). The binary term V₂(x, y, l, m) is only non-zero if y = x⁺ or y = x⁻ and l, m include one same label, leading to the following formula for the energy term U(x, l) = V₁(x, l) + ∑_y,m V₂(x, y, l, m) at x:

$$\begin{array}{c}U(x,l)=d_T(x)u_l(x)c_l(x)+s_T(x,x^{+}){\text{max}}_m(U(x^{+},l),U(x^{+},l,m))+s_T(x,x^{-}){\text{max}}_m(U(x^{-},l),U(x^{-},l,m))\hfill \\ U(x,l,m)=d_O(x)u_{l,m}(x)c_{l,m}(x)+s_O(x,x^{+})\text{ max }(U(x^{+},l,m),U(x^{+},l),U(x^{+},m))+s_O(x,x^{-})\text{ max }(U(x^{-},l,m),U(x^{-},l),U(x^{-},m))\hfill \\ U(x,I)=\frac{1}{2}{d}_I{u}_I(x)+\frac{1}{|N(x)|}{\displaystyle {\sum }_{y\in N(x)}}{s}_{I}U(y,I)\hfill \end{array}$$ (12)

These three equations list all the possible cases: single tract labels, pairs of overlapping labels, and isotropic regions. Because regions of overlap interact with single tract regions, the energy can propagate between different labels as long as they share a common tract.

With this model, only the neighboring voxels most likely to be along the same tract are considered for single or overlapping tracts, whereas the contribution of all neighbors is averaged for isotropic regions. By using such a subset of the neighborhood, we greatly simplify the structure of the MRF and improve computational stability, as the field is locally oriented along the tracts, removing many loops in the neighborhood graph. The probability function can be efficiently maximized through an iterated conditional modes algorithm which converges quickly in practice and requires little computational overhead (Besag, 1986). This simplified neighborhood structure enable us to estimate the MRF efficiently without the help of a more elaborate MRF solver (Bazin et al., 2009a; Kolmogorov and Zabih, 2004).

2.4. Anatomical constraints

Even with a limit of two overlapping tracts per voxel, the number of possible pairs grows quickly when including more white matter tracts. However, it is well known from anatomy that certain pairs of tracts will not overlap, even in the presence of deforming pathology or trauma. For instance, it is clear that the superior longitudinal fasciculus (SLF) should not interact with the uncinate tract (UNC), or the cortico-spinal tract (CST) with the frontal forceps (CCF). From this observation, we restrict the possible label pairs to a list set a priori from anatomical information. The list may depend on image resolution, as more tracts will overlap in a voxel of coarser resolution.

To obtain such information from atlases and anatomy experts is challenging when the number of tracts to be estimated grows. We define an overlap probability for each pair of tracts in the atlas as follows:

$$P_O(m,l)=\frac{{\text{max}}_x{p}_m(x)p_l(x)}{({\text{max}}_x{p}_m(x))({\text{max}}_x{p}_l(x))}$$ (13)

where p_m(x), p_l(x) are the spatial probabilities for tracts m, l. We then restrict the possible label pairs to those such that P_O > 1/2.

Similarly, we make the assumption that our blurred atlas priors should fully include the corresponding tracts, thus estimating only probabilities for tracts with non-zero prior at each voxel. These two modeling constraints greatly reduce the combinatorial increase and computational burden when additional tracts and potential crossings are added to the atlas, which allowed the method to scale well from an initial atlas of 10 tracts to our current atlas of 39 tracts.

2.5. Handling WM pathology

Despite the development of many diffusion MRI processing algorithms in recent years, studies of the WM structures have been difficult when pathology is present. In diseases like multiple sclerosis, Alzheimer’s disease, or even in normal aging, lesions appear inside the WM, changing the diffusion properties of the tissue (Reich et al., 2010; Wheeler-Kingshott and Cercignani, 2009).

Our approach is already robust to most lesions as it combines information coming from multiple directions around the area of lesion, however lesions with a sharp decrease in anisotropy will likely be segmented as isotropic regions rather than tracts. If we have an estimate of the lesion location from co-registered structural MRI (using (Shiee et al., 2010) in our case), we can use that information as an additional prior: to compensate for the low anisotropy inside lesions, we simply update the indices in all lesion voxels i by:

$$\begin{array}{c}{d}_O(i)\leftarrow d_O(i)+d_I(i)\hfill \\ d_T(i)\leftarrow d_T(i)+d_I(i)\hfill \\ d_I(i)\leftarrow 0\hfill \end{array}$$ (14)

This update enforces that DOTS segments lesions as part of at least one tract, provided that the local diffusion direction matches the atlas. Note that DOTS includes a tract label for unidentified WM regions and allows isotropic regions to spread to their neighborhood, so the lesions are not forced to merge with a nearby tract if their tensor directions are not compatible.

2.6. Algorithmic Details

The complete segmentation process is performed as follows. First we register the shape and direction atlas to the skull-stripped tensor image to be segmented with a multi-scale gradient descent method that maximizes E_R = ∑_x ∑_l ‖a(x)p_l(T(x))‖², where a(x) is the fractional anisotropy and T a rigid transform. The direction atlas is rotated accordingly. Next, the estimation algorithm is initialized with U(x, l) = V₁(x, l), and then refined using iterated conditional modes and registration until convergence. Convergence is measured by the proportion of changed labels for each iteration.

To reduce computational overhead, we only record the energy function for the N_B labels with highest energy at a given voxel, and approximate the others to zero. In our experiments, no significant differences could be observed in the results for N_B ≥ 8. Also note that the connectivities s_T(x, x⁺), s_T(x, x⁻), s_O(x, x⁺), s_O(x, x⁻) are functions of the data alone, and can be precomputed to improve computation speed. A 181×217×181 voxel image (1mm cubic resolution) is processed with this method in less than 30 minutes, and a more typical 256×256×60 voxel image takes about 10 minutes to converge on a modern workstation with 8GB of available memory.

Once the algorithm has converged, we obtain a hard segmentation by selecting the labeling of highest energy, and attributing the voxel to the underlying tract or tracts in case of overlap; see Fig. 3. The corresponding membership function is given by:

$$p(l)=\frac{\text{exp}(g_{0}U(x,l))+{\displaystyle {\sum }_m}\text{exp}(g_{0}U(x,l,m))}{{\displaystyle {\sum }_m}\text{exp}(g_{0}U(x,m))+{\displaystyle {\sum }_{m,n}}\text{exp}(g_{o}U(x,m,n))}$$ (15)

where g₀ is a parameter controlling the sharpness of the membership.

Figure 3.

DOTS segmentation example for one subject of our reproducibility study in axial, coronal, sagittal views and 3D rendering. Regions of overlapping tracts are displayed as a checkerboard pattern of the tracts’ colors in the 2D views. It is notable that many regions of overlap are found in practice. The recovered tracts boundaries are also somewhat irregular, as such boundaries are poorly defined on the tensor images.

The DOTS algorithm has been implemented as a plug-in for the MIPAV software package (McAuliffe et al., 2001) and the JIST pipeline environment (Lucas et al., 2010). The software is freely available on the NITRC repository².

3. Experiments

We present here a simple synthetic experiment as well as real data experiments on clinical-quality data sets using our 39 tract labels atlas. Additional experiments performed with preliminary atlases have been reported in (Bazin et al., 2009b).

3.1. Synthetic Crossing Experiments

We first investigate the ability to recover crossing fibers with a simulated image depicting crossing fiber tracts with various levels of noise; see Fig. 4. The noise increases the number of voxels treated as isotropic as they diverge from the directions learned in the atlas. The crossing is correctly estimated as belonging to both fiber tracts, although with lower certainty. Note that the tensors from the crossing region were not included in the atlas, as would be expected from a simple tractography-based delineation.

Figure 4.

Simulated crossing experiment at SNR 25 (top) and 5 (bottom). From left to right: a simulated image, the manual “fiber” delineation used in the atlas, the hard segmentation, showing the intersection in white, and the two membership functions for the tracts. Note that the memberships decrease smoothly inside the crossing, as both tracts can have membership values above 0.5 with our model.

3.2. Reproducibility study

Segmenting tracts in a way that is consistent for repeated scans of a given subject is a requirement for clinical studies. We tested the reproducibility of our segmentation on the publicly available Kirby21 dataset (Landman et al., 2010b), which consists of two sets of MR scans taken on the same day for 21 healthy subjects, following a “Jones 30” protocol (Jones et al., 1999). The scans for each subject were processed separately with DOTS before co-registering their FA maps for comparisons. Figure 5 gives the average and standard deviation of the Dice overlap, average boundary surface distance and volume difference ratio. We also report the fiber tract volumes, as the volume difference ratio and Dice coefficient become more variable for smaller structures (in particular, WM tracts are much smaller than GM structures for which volume measurements and Dice coefficients are usually reported).

Figure 5.

DOTS segmentation results for two successive acquisitions. Mean (bar) and standard deviation (line) for 21 subjects of the following measures (from top to bottom): Dice overlap, average boundary surface distance, volume difference (as a percentage of the average volume), and average volume.

This experiment indicates that DOTS is robust to changes in patient position and scanning noise. Most notably, the tract overlap is above 0.7 for the larger tracts, and stays above 0.6 even for the optic tract, the cerebellar peduncles and the uncinate fasciculus, despite their small sizes. The distance between estimated tract boundaries is about half the imaging voxel size of 2.2 mm.

The segmentations from the two separate scans are very similar for each subject, while the labels adapt well to different anatomies; see Figure 6. It is also noticeable that there is a large amount of overlap between estimated labels, confirming the need for modeling overlap at the current resolution of clinical DTI.

Figure 6.

DOTS segmentation examples for three different subjects in coronal views for two separate acquisitions (top and middle). The bottom row displays selected regions of overlap, which are indicated with a checkerboard pattern of the tracts’ colors. Regions of unlabeled white matter are indicated in red. Note that the labeling is very consistent even in the regions of overlap where the tensors are more likely to be influenced by noise.

3.3. Integration with fiber tractography

Although DOTS labels give a reliable representation of the white matter tracts, they are quite different from the segmentation one would obtain from manual bundling of tractography results. Because our MRF model does not require long range interactions between voxels, the DOTS labels are more inclusive. On the other hand, one can independently reconstruct individual sample paths through the white matter (generally referred to as “fibers,” although the term is misleading) with any appropriate tractography technique (Mori et al., 1999; Koch et al., 2002; Behrens et al., 2007; Reisert et al., 2011).

Given a set of fibers computed by any tractography method, we can associate a fiber f to a tract l using the following method:

$$\begin{array}{cc}f\in l\text{ if }\hfill & {\displaystyle {\int }_{x\in f}}\mathit{\text{dx}}>l_0\hfill \\ \hfill & \text{and }{\displaystyle {\int }_{x\in f}}{b}_l(x)\mathit{\text{dx}}>{\displaystyle {\int }_{x\in f}}{b}_m(x)\mathit{\text{dx}},\forall m\ne l\hfill \\ \hfill & \text{and }{\displaystyle {\int }_{x\in f}}{b}_l(x)\mathit{\text{dx}}>r{\displaystyle {\int }_{x\in f}}\mathit{\text{dx}}\hfill \end{array}$$

where l₀ is a minimum length parameter set to 20 mm, b_l a binary function equal to 1 inside tract l and 0 otherwise, and r a minimum inclusion ratio set to 75%.

We performed fiber tractography on the Kirby21 dataset using the FACT algorithm (Mori et al., 1999) with the following parameters: FA start 0.2, FA stop 0.1 and maximum angle 60 degrees, and labeled the generated fibers with the above method. As shown on Fig. 7, the labeled fibers form compact bundles with many fibers. It is notable that the fibers extend further toward the outer boundaries of the WM than the corresponding DOTS labels, as the deterministic tractography procedure extends fibers until they reach a FA of 0.1, whereas the DOTS model assigns lower probability to these regions of lower FA and higher variability. Note that DOTS labels can easily be extended to always include such regions if necessary using a white matter mask to define a priori the boundaries of the isotropic regions.

Figure 7.

Tractography results automatically labeled with DOTS for the three subjects of Fig. 6 in coronal view and 3D rendering. The bundles are overall rather homogeneous, and the included regions are similar in size to the original DOTS labels, indicating that the reconstructed fibers follow quite closely the DOTS segmentation.

3.4. Robustness to pathology

Finally, we assessed the application of DOTS on white matter lesions on a data set of 10 multiple sclerosis subjects. We identified white matter lesions on co-registered structural MRI (MPRAGE and FLAIR) with (lesion)TOADS, an automated brain and lesion segmentation tool (Shiee et al., 2010). DOTS was run with and without the prior lesion map, to measure the impact of lesions on the labeling.

We measured average fractional anisotropy (FA) and volume for the tracts, two measures of interest when assessing the impact of lesions on white matter. When comparing uncorrected and corrected measures with a t-test, we find small but significant differences (generally a decrease) in FA for ATRR, CCS, FNXL, PTRL and more important differences in volume (either increasing or decreasing by up to 5% with the correction) for CSTL, ICPL, MLR, OPTL, PTRL, SCPR, SLFL, SLFR, TAP and UNCL (p < 0.05). These results clearly show that the presence of lesions confounds DTI measurements unless they are properly accounted for in the model.

The individual segmentations, as depicted in Fig. 8 highlight the accumulation of lesions along the peri-ventricular tracts (especially the optic radiation) for subjects with large lesion loads, while the interplay of focal lesions with other tracts appears more complex. DOTS handles well the damaged regions on clinical quality DTI, and we hope it will offer a more systematic way to study the relationship of WM lesions with cerebral circuits and their relative impairment.

Figure 8.

DOTS segmentation examples for three different subjects with multiple sclerosis. The white matter lesions segmented from the co-registered structural images (T1 and FLAIR) are indicated with a white outline. Although the lesions do not follow given tracts, it is clear that certain tracts are largely spared while others are strongly affected.

4. Discussion

We presented here an atlas-based segmentation technique to reliably extract known anatomical tracts from clinical quality diffusion tensor MRI. As in the work of (Hagler et al., 2009), the method combines global statistical priors of shape and direction with the local connectivity information extracted from the tensors. The prior direction information in particular is important to disambiguate many possible tracts. However, we also noticed that the atlas alone was only partially successful, and that many parts of the tracts were not directly identified where eigenvector directions were not well aligned with the atlas. The MRF model plays a central role in solving this problem, as it propagates well-defined labels to the neighboring voxels along the tracts of the subject, reinforcing label probabilities.

The explicit modeling of crossings and overlapping regions is also a key element, as many tracts overlap at the current resolution. The algorithm is presently limited to crossings between two tracts, which may be a source of error in regions of triple-perpendicular crossings (Jeurissen et al., 2011). The main limitation resides in the tensor representation: because these crossings would be indistinguishable from isotropic regions, adding the extra degrees of freedom needed for triple intersection may reduce the accuracy of the results. However, the model could be extended easily from pairwise to multiple overlaps, given a more elaborate model of diffusion at higher angular resolution or even using regular DTI sequences (Landman et al., 2010a). Finally, the regions of multiple crossings are not necessarily estimated as isotropic in the current model, because neighboring regions from the converging tracts can propagate their labels into the crossing (s_T and S_O are usually lower but non-zero in isotropic voxels), and we did not observe any clear mislabeling in regions of multiple overlap.

Our experiments show that the DOTS segmentation is highly reproducible, even for small and variable tracts. The tracts segmented can be used readily as regions of interest for localized analysis of specific white matter pathways. Tract based statistics can be extracted separately for each tract, and effects of lesions or atrophy on anisotropy measures are less likely to confound the results. The segmented regions are somewhat larger than ones obtained through bundling tractography results, inherently because the MRF model does not require each voxel of the tract to be part of a complete path. On the other hand, a larger region of interest can be beneficial when computing statistics of the white matter. The results of DOTS can also be combined with any tractography technique to generate labeled fiber bundles in a fully automated fashion as well.

The atlas currently includes 39 separate tracts, covering most of the deep white matter pathways. Additional tracts can easily be added provided they can be identified reliably enough on diffusion MRI. Anatomical constraints are however important in order to avoid combinatorial explosion of the number of possible labels to estimate, and the atlas would benefit from incorporating high resolution anatomical delineations (Bürgel et al., 2006). The diffusion properties of a given tract are not fully captured by the principal eigenvector directions, and a more elaborate representation of the expected tensor (or higher order model) shape might further increase accuracy at the cost of higher computational demands and possibly a lowered robustness to noise and lesions.

Even with the few limitations discussed above, essentially inherited from the underlying tensor model of DTI, the DOTS software provides a novel and practical tool for DTI analysis in routinely acquired sequences, and we hope that making this method available to the community will help neuroscientists study in more detail individual white matter tracts in larger cohorts.