DeepBundle: Fiber Bundle Parcellation with Graph Convolution Neural Networks

Abstract

Parcellation of whole-brain tractography streamlines is an important step for tract-based analysis of brain white matter microstructure. Existing fiber parcellation approaches rely on accurate registration between an atlas and the tractograms of an individual, however, due to large individual differences, accurate registration is hard to guarantee in practice. To resolve this issue, we propose a novel deep learning method, called DeepBundle, for registration-free fiber parcellation. Our method utilizes graph convolution neural networks (GCNNs) to predict the parcellation label of each fiber tract. GCNNs are capable of extracting the geometric features of each fiber tract and harnessing the resulting features for accurate fiber parcellation and ultimately avoiding the use of atlases and any registration method. We evaluate DeepBundle using data from the Human Connectome Project. Experimental results demonstrate the advantages of DeepBundle and suggest that the geometric features extracted from each fiber tract can be used to effectively parcellate the fiber tracts.

1. Introduction

Diffusion MRI provides valuable insights into the 3D geometric structure of brain neural fiber tracts, allowing tract-based analysis (TBA) of brain white matter microstructure in vivo [1]. TBA usually focuses on specific fiber bundles that are extracted from the whole-brain tractograms. Therefore, effective methods for segmenting whole-brain tractograms into fiber bundles of interest are desirable.

Existing fiber parcellation approaches can be divided into two categories: ROI-based and streamline-based. ROI-based approaches [2,3] first parcellate the brain surface based on an atlas and then use the parcellation ROIs to extract different fiber bundles. Streamline-based approaches [4,5] employ streamline registration methods to directly transfer the fiber parcellation information from an atlas to tractograms of an individual. These methods are directly affected by registration accuracy, which is, however, hard to guarantee in practice due to factors such as inter-subject variability, noise, and artifacts. Moreover, the registration procedure is usually time-consuming, making it unsuitable for large-scale studies and real-time applications. Recently, Wasserthal et al. [6] proposed to predict the binary mask of a fiber bundle from the whole brain fiber peaks using convolutional neural networks. This method avoids the registration procedure but is limited to generating a binary tract segmentation mask rather than the parcellation labels of the fiber tracts. In addition, this method utilizes the fiber orientation information at each voxel rather than the actual geometries of the fiber tracts.

In this work, we propose a deep-learning approach, called DeepBundle, to parcellate whole-brain tractograms without the time-consuming registration. Specifically, we view the coordinates of the points on each fiber tract as functions defined on a graph. The point coordinates are then fed to a graph convolutional neural networks (GCNN) to extract the latent geometric features of fiber tracts for bundle recognition. Our network is trained end-to-end with the point coordinates as inputs and parcellation labels as outputs, thus avoiding having to manually craft features for the purpose of bundle parcellation. During the testing stage, the point coordinates extracted from fiber tracts of the whole brain are directly fed to the trained networks, thus avoiding the use of atlases and any registration method. Extensive experiments performed using data from the Human Connectome Project (HCP) indicate that DeepBundle yields fiber parcellation labels with remarkably improved accuracy, confirming that the geometric features extracted by GCNNs are effective for fiber bundle parcellation.

2. Methods

In this section, we will first show how fiber streamlines can be represented using graphs, and then, introduce the theory of spectral graph convolution and graph pooling. Finally, we will describe our network in detail.

2.1. Graph Representation of Fiber Tracts

Considering an undirected graph as $\mathcal{G}=(\mathcal{V},\mathcal{E},\mathcal{W})$, $\mathcal{V}=\left\{v_1,\cdots ,v_n\right\}$ is a set of n vertices, $\mathcal{E}\subseteq \mathcal{V}\times \mathcal{V}$ is the edge set, and $\mathcal{W}=\left(w_{(i,j)}\right)$ is the n × n adjacency matrix, which is symmetric, i.e., w_(i,j) = w_(j,i). We uniformly sample n points on each fiber tract, and thus this discrete fiber tract can be represented by a line-type graph, where the edge weights are given by

$$w_{i,j}=\{\begin{array}{ll}1,\hfill & \text{ if }i,j\text{ are connected,}\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}$$ (1)

The graph $\mathcal{G}$ now encodes the geometric relationships between sampling points on the fiber tract [7]. We then view the point coordinates extracted from a fiber tract as graph-structured data.

2.2. Spectral Graph Convolution

To extract the underlying geometry-invariance features of each fiber tract, we employ spectral graph convolution [8], which generalizes conventional convolution in Euclidean space using graph Fourier transformation. The transformation relies on the eigendecomposition of the graph Laplacian Δ, which is defined as

$$\Delta =\Phi \Lambda {\Phi }^{\text{T}}=I_n-{\mathcal{D}}^{-1/2}\mathcal{W}{\mathcal{D}}^{-1/2},$$ (2)

Where $\Phi =\left({\phi }_1,\cdots ,{\phi }_n\right)$ is a matrix of orthonormal eigenvectors $\left({\Phi }^{\text{T}}\Phi =I_n\right)$, $\Lambda =\text{diag}\left({\lambda }_1,\cdots ,{\lambda }_n\right)$ is a diagonal matrix of corresponding eigenvalues, I_n is an identity matrix, and $\mathcal{D}={\sum }_{j\ne i}{w}_{(i,j)}$ is the degree matrix. Such eigenvectors $\left({\phi }_1,\cdots ,{\phi }_n\right)$ can be interpreted as the Fourier bases, and ${\Phi }^T$ is used to transform the features from the spatial domain to spectral domain. The graph Fourier transformation is formulated by

$$\text{f}* \text{g}=\Phi \left({\Phi }^{\text{T}}\text{g}\right)\odot \left({\Phi }^{\text{T}}\text{f}\right)=\Phi \text{diag}\left({\widehat{g}}_1,\cdots ,{\widehat{g}}_n\right)\widehat{\text{f}},$$ (3)

where f = (f₁, ⋯, f_n)^T is the input signal, which denotes the geometric features of one streamline on the vertices of graph $\mathcal{G}$. Its Fourier transformation is given by $\widehat{\text{f}}=\left({\Phi }^{\text{T}}\text{f}\right)$; g is a convolutional filter in the spatial domain, and the spectral convolution can be defined as element-wise product $\left({\Phi }^{\text{T}}\text{g}\right)\odot \left({\Phi }^{\text{T}}\text{f}\right)$, which can also be written by diag $\left({\widehat{g}}_1,\cdots ,{\widehat{g}}_n\right)\widehat{\text{f}}$. Hereby, diag $\left({\widehat{g}}_1,\cdots ,{\widehat{g}}_n\right)$ is the corresponding convolutional filter in the spectral domain.

Utilizing the spectral definition of the convolution, GCNNs generalize CNNs to graphs. The ℓ-th spectral convolution layer can be written by

$${\text{f}}_k^{(\ell +1)}=\xi \left(\Phi \sum _{k^{\prime }=1}^p\text{diag}\left({\widehat{g}}_{\left(k,k^{\prime },1\right)}^{(\ell )},\cdots ,{\widehat{g}}_{\left(k,k^{\prime },n\right)}^{(\ell )}\right){\Phi }^T{\text{f}}_{k^{\prime }}^{(\ell )}\right),$$ (4)

where ${\text{F}}^{(\ell )}=\left({\text{f}}_1^{(\ell )},\cdots ,{\text{f}}_p^{(\ell )}\right)$ and ${\text{F}}^{(\ell +1)}=\left({\text{f}}_1^{(\ell +1)},\cdots ,{\text{f}}_q^{(\ell +1)}\right)$ are the n × p and n × q matrices, representing the input and output features of ℓ-th spectral convolution layer; ${\text{f}}_{k^{\prime }}^{(\ell )}$ and ${\text{f}}_k^{(\ell +1)}$ denote the k′-th column of the input matrix and the k-th column of the output matrix. $\Phi =\left({\phi }_1,\cdots ,{\phi }_n\right)$ is an n × n matrix, and diag $\left({\widehat{g}}_{\left(k,k^{\prime },1\right)}^{(\ell )},\cdots ,{\widehat{g}}_{\left(k,k^{\prime },n\right)}^{(\ell )}\right)$ is a n × n diagonal matrix which denotes learnable filters in the spectral domain; and ξ denotes the nonlinearity, e.g., ReLU.

2.3. Fast Graph Pooling

We utilize the Graclus [9,10] to coarsen the input graph into multi-scales, which is similar to the pooling operation in conventional CNNs. In practice, the multi-scale graph is organized as a binary tree, which becomes coarse from the leaf layer to the root layer. When constructing this tree, Graclus introduces fake nodes, rearranges the nodes of the streamline, and thus, we can perform graph pooling using fast 1D spatial pooling. Cousin nodes in one layer are aggregated into one parent node in the upper layer of the binary tree. The pooling process is illustrated in Fig. 1.

Fig. 1:

The architecture of the employed spectral GCNNs. Red arrows denote the spectral convolution layers, blue arrows denote the fast max-pooling layers, a green arrow denotes one fully connected layer, and the yellow arrow denotes the softmax layer.

2.4. Network Architecture of DeepBundle

Fig. 1 illustrates the network architecture of DeepBundle. We employ a spectral GCNNs with the architecture of GC32-P2-GC64-P2-FC512, where GCc denotes a graph convolution layer which has c filters, P2 denotes a graph pooling layer with a factor 2, and FC512 denotes a fully connected layer with 512 hidden nodes. Each graph convolutional layer is activated by the ReLU function. The last layer is the softmax regression layer, and we employ the cross-entropy loss with an l₂ regularization term. We separately trained networks for segmenting different fiber bundles of interest from the whole-brain tracts.

In Fig. 1, the length of the input is increased from n to k due to the introduction of fake nodes (black dots) by Graclus. We employed 32 and 64 filters in the first convolution and the second convolution layer respectively. In the pooling layer, two neighbor nodes are aggregated into one. The input with a length k is reduced to k/4 after going through two max-pooling layers.

3. Results

3.1. Implementation Details

For evaluation, we used the publicly available HCP fiber tract dataset [11]. This dataset contains 105 subjects. Each subject has 72 fiber bundles, containing streamlines of different lengths. We uniformly resampled them to have the same number of points, i.e., 100 points. We then represented all streamlines using a common graph. In our experiments, we randomly selected 25 subjects from the database for training, 2 for validation, and 11 for testing. In preparing the training data, a positive label is assigned to all streamlines from a bundle of interest. The negative samples are collected by 1) selecting streamlines from spatially neighboring bundles, and 2) randomly selecting an equal number of streamlines from all other bundles. For testing, all streamlines of each testing subject were fed separately into the trained network.

3.2. Experimental Results

Fig. 2 shows the fiber parcellation results of one randomly selected subject (#623844). 12 bundles of interest are parcellated from more than one million whole-brain fiber tracts. Although fiber bundles in the left and right hemispheres may share similar shapes, DeepBundle is able to distinguish between them. For some small-size bundles, e.g., FX, which have only hundreds of streamlines, DeepBundle consistently gives promising parcellation results. In addition, we show the quantitative parcellation results of the 12 fiber bundles in Table 1, further confirming the conclusions that can be drawn from Fig. 2.

Fig. 2:

Qualitative results of 12 fiber bundles: Corticospinal tract (CST), Commissure anterior (CA), Corpus callosum (Rostrum (CC 1), Genu (CC 2), Isthmus (CC 6)), Uncinate fascicle (UF), Arcuate fascicle (AF), Fornix (FX). Colored streamlines indicate the true positive (TP) streamlines, while black and white denote the false positive (FP) and false negative (FN) streamlines, respectively.

Table 1:

Classification results for 12 bundles of subject #623844.

Counts	CST left	CST right	CC 1	CC 2	CC 6	CA
TP	4755	7726	4419	51967	30082	1330
FP	244	395	143	1100	602	0
FN	54	382	29	153	320	0
Counts	UF left	UF right	AF left	AF right	FX left	FX right
TP	3535	4916	54448	42635	94	123
FP	0	0	199	495	6	7
FN	1	3	190	220	1	1

Table 2 shows the quantitative results computed across 10 testing subjects. A large precision/recall value indicates more accurate fiber parcellation. It can be observed that DeepBundle achieves high precision and recall values for all bundles of interest, indicating promising fiber parcellation accuracy. It is interesting to note that the recall numbers of left CST and right CST are smaller than the other fiber bundles. This is due to the fact their neighboring bundles, such as POPT and FPT, are very similar to CST in shape. Such streamlines induce ambiguities in fiber parcellation, even when performed manually.

Table 2:

Classification performance (mean ± standard deviation in %) on 12 fiber bundles from 11 subjects.

	CST left	CST right	CC 1	CC 2	CC 6	CA
Precision	90.5 ± 7.1	91.1 ± 7.9	96.5 ± 2.1	97.6 ± 2.2	95.6 ± 2.3	99.9 ± 0.1
Recall	88.4 ± 11.6	86.6 ± 11.6	95.5 ± 4.3	98.4 ± 1.3	98.1 ± 1.8	100 ± 0.0
	UF left	UF right	AF left	AF right	FX left	FX right
Precision	99.7 ± 0.6	99.8 ± 0.2	99.2 ± 1.0	99.4 ± 0.6	87.3 ± 12.7	90.7 ± 14.1
Recall	99.2 ± 2.1	99.6 ± 0.4	99.4 ± 0.8	96.5 ± 2.8	97.5 ± 3.9	96.6 ± 3.8

We also compared DeepBundle with a popular method called RecoBundles [12]. We mapped all parcellated fiber streamlines to created volumetric visitation maps and computed their Dice scores with respect to the gold standard. The results, shown in Table 3, indicate that DeepBundle significantly improves the Dice score and outperforms RecoBundles for all bundles of interest. RecoBundles gives significantly lower Dice scores CA, left FX, and right FX. DeepBundle yields relatively smaller Dice scores for left CST and right CST, but still significantly outperforms RecoBundles.

Table 3:

Dice scores (mean ± standard deviation in %) of 12 fiber bundles from 11 testing subjects.

	CST left	CST right	CC 1	CC 2	CC 6	CA
RecoBundles	45.3 ± 9.7	40.6 ± 6.1	51.3 ± 13.2	63.6 ± 8.6	60.7 ± 6.31	25.2 ± 16.1
DeepBundle	80.7 ± 10.1	86.9 ± 5.1	93.0 ± 3.3	96.8 ± 1.2	97.1 ± 1.4	99.1 ± 1.6
	UF left	UF right	AF left	AF right	FX left	FX right
RecoBundles	46.9 ± 6.3	52.1 ± 7.7	61.3 ± 9.1	63.1 ± 12.7	8.1 ± 2.8	9.5 ± 3.6
DeepBundle	96.4 ± 5.1	98.1 ± 1.3	95.8 ± 3.8	96.2 ± 4.0	82.4 ± 9.3	86.5 ± 8.8

4. Conclusion

In this paper, we have proposed a framework for fiber bundle parcellation using a GCNN. Our method directly predicts a tract parcellation label from the point coordinates extracted from a fiber tract. GCNNs are capable of extracting robust geometric features for tract parcellation in an end-to-end manner. Experiments on HCP data demonstrate that our method, DeepBundle, yields promising tract bundle parcellation results with high precision and recall rates. DeepBundle also achieves much higher Dice scores compared with RecoBundles. Our results also suggest that DeepBundle is even capable of effectively parcellating small tract bundles.