3D Segmentation of Rodent Brain Structures Using Hierarchical Shape Priors and Deformable Models

Abstract

In this paper, we propose a method to segment multiple rodent brain structures simultaneously. This method combines deformable models and hierarchical shape priors within one framework. The deformation module employs both gradient and appearance information to generate image forces to deform the shape. The shape prior module uses Principal Component Analysis to hierarchically model the multiple structures at both global and local levels. At the global level, the statistics of relative positions among different structures are modeled. At the local level, the shape statistics within each structure is learned from training samples. Our segmentation method adaptively employs both priors to constrain the intermediate deformation result. This prior constraint improves the robustness of the model and benefits the segmentation accuracy. Another merit of our prior module is that the size of the training data can be small, because the shape prior module models each structure individually and combines them using global statistics. This scheme can preserve shape details better than directly applying PCA on all structures. We use this method to segment rodent brain structures, such as the cerebellum, the left and right striatum, and the left and right hippocampus. The experiments show that our method works effectively and this hierarchical prior improves the segmentation performance.

1 Introduction

Magnetic resonance imaging (MRI) at a spatial resolution of at least 100um in one dimension is frequently referred to as MR microscopy (MRM) and is currently available with the use of high magnetic field images. This technical achievement has permitted the detailed anatomical study of the rodent brain, which is much smaller than the human brain and requires a small voxel size in order to be imaged. Rodents are often used as models of human disease not only because they frequently exhibit key features of abnormal neurological conditions but also because they are a convenient starting point for novel studies. The analysis of rodent brain image faces similar challenges to human imaging, with individual variation in size, morphology, and topology of the brain structures complicating the neuroanatomical studies. Such analysis is frequently performed by segmenting the regions of interest (ROI) in rodent brain images. The challenge of this segmentation task is threefold. 1) The image information is sometimes incomplete or misleading. For example, there is no obvious boundary in part of the striatum, and the cerebellum contains interleaving texture (Fig. 1), 2) It is important to discover and/or preserve some complex local shape details, such as the paraflocculi (i.e., two protruding features of the cerebellum), 3) Given limited number of training samples, it is desirable to segment all interested structures simultaneously by effectively learning a shape atlas for multiple structures.

Fig. 1.

The MR image of a rat brain used in experiments. The image information is misleading because 1) part of the boundary between striatum (top) and other structures is blurred; 2) the cerebellum (bottom) has complex textures and large gradient values inside.

In recent decades, many deformable model based segmentation methods have been proposed to solve these challenges and achieved tremendous success [6,2,7,11]. The traditional Snakes [6] solely rely on the image gradient information, so they may be trapped by noise and spurious edges. Region analysis strategies [13] have been incorporated in Snake-like models to improve their robustness to noise. Metamorphs [5] was proposed to be able to integrate shape and appearance in a unified space. The model has not only boundary shape but also interior appearance, making it more robust to ambiguous boundaries and complex internal textures. Its 3D version, Active Volume Model (AVM) [10], is proposed to perform volume segmentation. The AVM model’s shape is represented by a simplex mesh and its volumetric interior carries the various visual appearance feature statistics. However, the only shape prior used in AVM is the smoothness constraint. Thus it may not be able to preserve or discover small shape details. Another group of deformable models also considering region information is level set based methods [7,8]. These approaches have been widely used in tubular structure and 3D cortex segmentation tasks since they are topologically free and can be easily used in any dimension. Statistical modeling approaches such as Active Shape Model (ASM) [2] are also widely used. These methods constrain the intermediate shape by using the shape pattern from existing data. Thus they are able to recover or preserve local shape details. However, these methods may need a large amount of 3D training data, whose creation and maintenance can be difficult and time consuming in practice. Combining AVM and ASM is able to handle both complex texture and shape details without a large set of 3D training data [12]. However, this approach still cannot handle multiple structures simultaneously because, with multiple structures together, there is a larger variability and thus it usually requires more training samples in order to capture such variability.

In this paper, we propose a unified algorithm to alleviate these three challenges mentioned above. It includes a robust deformation model which uses both gradient and appearance information, and a shape prior model which can learn shape statistics from a small number of training data by employing a hierarchical modeling scheme. Note that the idea of hierarchical shape prior has been investigated much in the literature [3,4,1]. Our contribution is to combine such idea with a robust deformable model to effectively segment multiple structures of rodent brains. First, the shape statistics of each structure is obtained using topology-aware shape registration and Principal Component Analysis (PCA). The relative positions among structures are also obtained as high level statistics. This hierarchical prior module is able to effectively build shape statistics in two levels even with limited number of training data. Second, the deformation module is used to drive the 3D mesh based on the image information. Its intermediate result is refined and constrained by adaptively using the hierarchical shape priors. Our deformation module is relatively robust to image noise. However, in the presence of bad initialization or heterogeneous texture, the shape prior constraint can still prevent the model from getting stuck or leaking out of the target in the presence of bad initialization or heterogenous texture. We use this algorithm to segment complex structures in rodent brains, such as the cerebellum, the hippocampus and the striatum. Extensive experiments have been designed to evaluate this method.

2 Methodology

Shape Atlas for Multiple Structures

All brain scans are aligned to the same reference brain. Alignment is performed with rigid transformations (rotations and translations), thus after alignment the volume and shape of brain structures do not change. The brain structures are manually segmented by clinical experts. We focus on the cerebellum, the left and right striatum, and the left and right hippocampus. After extracting triangular meshes from the binary image, we use sophisticated geometry processing methods to improve the mesh quality, such as the mesh decimation, isotropic remeshing and detail-preserved smoothing. A topology-aware shape registration method based on Adaptive Focus Deformable Model (AFDM) [9] is then used to register a reference mesh to the others. Thus all processed meshes share the same topology and have one-to-one correspondence among vertices.

Once the training shapes are available, the generalized Procrustes analysis [2] is applied to align them together. PCA is then used to capture the shape statistics, such as the mean shape and its major variances. The shape statistics are captured in a hierarchical scheme. First, we apply PCA on each structure separately. Given limited training samples, this approach can learn the shape details better than using all structures. Fig. 2 shows the largest mode of the variations for rodent brain structures. Second, PCA is also used to learn the relative positions among structures, and their locations with respect to the mass centroid. This prior is built at the high level. Both statistics are used as prior information to guide the segmentation process. Since the hierarchical scheme builds priors in two levels, it is able to effectively model the multiple structures effectively even given a small number of training samples.

Fig. 2.

The largest mode (−3σ to 3σ) of the rat brain variations, which represents the changing of the structure size. Middle is the mean shape.

Deformation Module

Given a test 3D image, it is aligned to the reference brain. Then the mean mass centroid of all structures is used as the estimated center. The mean shape of each structure is placed according to the relative positions learned at the high level statistics. However, this initialization may not be close to the boundary of the testing data because of the variance. Thus deformable models are still needed for accurate segmentation. In order to find the boundary robustly, the initialized models are driven by both gradient and region information derived from the image. Region information alleviates the problems caused by unclear boundaries and complex textures. The overall energy function is: E = E_int + E_ext = E_int + (E_g + k_R · E_R), where E_int is the internal (smoothness) energy, E_ext is the external (image) energy, E_g is the gradient term, E_R is the region term, k_R is a constant to balance the contributions of the two external energy terms. The balance between the internal and external energies is naturally controlled by the smoothness factor in the stiffness matrix by using Finite Element Method (FEM) as the deformation scheme [10].

The traditional gradient data terms usually include the gradient map, edge distance map, or a combination of both, while the region data term encodes constraints for the model-interior appearance statistics. Considering a module using intensity statistics, the object region is predicted according to the current model-interior intensity distribution. Having both foreground object and background probabilities, we obtain a binary map that represents the predicted object region by applying the Bayesian Decision rule. Connected component analysis is then applied to the binary map to retrieve the connected component that overlaps the current model. This connected region is considered as the current ROI. Let us denote the signed distance transform of the current model’s surface shape as Φ_Λ, and the signed distance transform of the ROI boundary shape as Φ_R, the region-based external energy term is defined using voxels within a narrow band around the model surface as: $E_R={\displaystyle {\int }_{\Lambda }{\Phi }_{\Lambda }\left(\mathbf{v}\right){\Phi }_R\left(\mathbf{v}\right)d\Lambda }$The multiplicative term provides two-way balloon forces that deform the model toward the predicted ROI boundary. This allows flexible model initializations either overlapping the object or inside the object. Using these external energy terms as image forces, the FEM model is driven to find object boundaries.

Hierarchical Shape Priors and Model Evolution

The above model may still not be able to avoid the local minimum or preserve a specific shape, especially when the texture of the image is complex. Thus a shape prior energy is added to our method to constrain and refine the shape and position of all n structures during deformation: $E_{\mathit{prior}}={\displaystyle {\sum }_{i=1}^n{E}_{\mathit{local}}^i}+k_g\cdot E_{\mathit{global}}$, where $E_{\mathit{local}}^i$ is the shape constraint applied on the ith structure, and E_global is a constraint on the relative positions among different structures and distances with respect to the mass centroid.

In terms of the implementation, the energy optimization is achieved by using an expectation-maximization (EM) type of framework. Given intermediate segmentation results of all structures, in the “E” step, the locations of structures are adjusted as per the global statistics to prevent overlap or intersection. In the “M” step, each individual shape is constrained by aligning to the mean shape, and then mapping into PCA space to update the pose and shape parameters. Thus it guarantees that the shape deforms only into patterns consistent with the training data, which can refine the shape and prevent over-segmentation. This whole hierarchical framework is employed adaptively by defining k_g = 1−e^−|∇Λ|, where |∇Λ| is the magnitude of deformation change in surface shape. In the beginning, the shape deforms a lot so k_g is relatively large. It means that we trust more on the high level information, and put larger weights on the global statistics. After several iterations, the local statistics should be more important since the brain structure is nearby the boundary. At this time the shape deforms less so k_g is smaller. The whole segmentation framework, including training and testing, is summarized as follows:

1. Manually segment a small number of training data. Use PCA to capture shape statistics and location statistics of relative positions.

2. Given a testing data, it is aligned to the reference image. Then multiple models are placed using mean relative positions with respect to the mean mass centroid. Initialize these models, i.e., stiffness matrix and step size for FEM and the gradient magnitude or edge map.

3. Compute Φ_Λ based on the current model; predict object ROI R by applying the Bayesian Decision rule to binarizing the estimated object probability map, and compute Φ_R. Calculate the external force vector.

4. Deform the model using FEM and external forces derived from gradient and region terms. Smoothness constraint is implicitly incorporated by FEM.

5. Update the value of k_g = 1 − e^−|∇Λ|. Adjust the structure positions to prevent intersections using global prior.

6. Refine each intermediate result by local prior constraint (transform it to the mean shape, then update the pose and shape parameters).

7. Repeat steps 3–6 until convergence.

Alternating the deformation module and the shape constraint module is more robust to noise and can handle more complex textures than purely using a deformation module. Due to the constraint from the shape prior, our model converges fast and robustly towards the true boundary given the mean shape of the multiple objects as initialization. One more benefit of our multiple structure model is that the size of the training data can be small, because the shape prior is built hierarchically. The prior for each structure is obtained individually, which can better discover the shape statistics than using one prior for all structures.

3 Experiments

Experimental Settings

The heads of adult male Sprague-Dawley rats were scanned on a 21.1T Bruker Biospin Avance scanner. The protocol consisted of a 3D T2-weighted scan with echo-time (TE) 7.5ms, repetition time (TR) 150ms, 27.7 kHz bandwidth, field of view (FOV) of 3.4 × 3.2 × 3.0mm, and voxel size 0 08mm, isotropic. 3D annotation is manually performed on 58 volume data by an expert neurologist. 8 are used as training data since we only need a small number of samples for hierarchical statistics. The rest 50 are used as testing. The proposed method was implemented in C++ and Python 2.6 and tested on a 2.40 GHz Intel Core2 Quad computer with 8G RAM.

Visual comparisons

Fig. 3 shows the visual comparisons of segmentation results starting from the same initialization. In the first row, we segment each structure individually. The shape prior is the smoothness constraint. In the second row, the hierarchical shape prior energy is incorporated to segment all structures simultaneously. When segmenting the cerebellum, the shape prior helps to preserve the shape of paraflocculi and to avoid the local minima caused by complex textures inside. Without using our priors, the model is attracted by large gradient values inside and the paraflocculi are smoothed out. In the striatum and hippocampus cases, there is no obvious boundary between these structures and the brain. Thus the model can easily over-segment the ROI. Our priors alleviate this problem. Another benefit of the hierarchical prior is that the spatial constraint is incorporated. The distance between two structures has to follow the location statistics. Thus structures cannot intersect with each other.

Fig. 3.

Comparison of segmentation results starting from the same initialization. The first row: traditional shape prior constraint (smoothness). The second row: using hierarchical shape prior. From left to right: the cerebellum, the left and right striatum, the left and right hippocampus, and all five structures.

Quantitative comparisons

Tab. 1 compares different priors. Note that all deformation modules are the same and based on both gradient and region information. We compared the methods of combing the robust deformation model with a traditional smoothness shape prior [10], an independent shape prior [12] for each structure and hierarchical shape priors (i.e., the proposed method). The same parameters are used in all deformation modules. We reported the mean value and standard deviation of voxel distances between segmented surfaces and ground truth surfaces. Generally, the shape prior constraint improves the segmentation accuracy. In the cerebellum case, the spatial constraint in hierarchical priors only slightly improves the result, while it is much more important in the striatum and hippocampus cases. The reason is that the striatum and hippocampus are adjacent to each other. Spatial constraints can prevent the intersection of different structures, which implicitly alleviates the over-segmentation problem caused by the low contrast and ambiguous boundaries. The volume magnitude information is very important in rodent brain analysis. Thus we also reported the relative error of volume magnitude compared to the ground truth. Using the hierarchical shape prior achieves the best result, especially for the striatum and the hippocampus. This spatial constraint not only improves the segmentation accuracy for multiple structures, but also decreases the number of iterations and computation time to converge.

4 Conclusions

In this paper, we proposed a deformable segmentation method to segment multiple rodent brain structures simultaneously. The model is driven by external forces derived from both image gradient and appearance information. To improve the robustness, a hierarchical shape prior module is also employed for model initialization and shape refinement. We use this method to segment complex structures in rodent brains, which shows that our method works effectively.

Table 1.

Quantitative comparisons of different prior schemes. We reported the mean and standard deviation of voxel distances between segmented surfaces and ground truth surfaces, and relative errors of volume magnitude in proportions

Structures	Prior types	Voxel distance	Volume error
	Smoothness	4.35 ± 2.17	0.22 ± 0.12
Cerebellum	Independent	1.74 ± 1.18	0.05 ± 0.02
	Hierarchical	1.70 ± 1.13	0.04 ± 0.02
	Smoothness	3.79 ± 2.05	0.51 ± 0.19
Striatum	Independent	2.93 ± 1.81	0.19 ± 0.06
	Hierarchical	1.37 ± 1.09	0.07 ± 0.03
	Smoothness	3.82 ± 2.14	0.53 ± 0.18
Hippocampus	Independent	2.69 ± 1.83	0.17 ± 0.05
	Hierarchical	1.22 ± 1.05	0.06 ± 0.02