Regional Manifold Learning for Deformable Registration of Brain MR Images

Abstract

We propose a method for deformable registration based on learning the manifolds of individual brain regions. Recent publications on registration of medical images advocate the use of manifold learning in order to confine the search space to anatomically plausible deformations. Existing methods construct manifolds based on a single metric over the entire image domain thus frequently miss regional brain variations. We address this issue by first learning manifolds for specific regions and then computing region-specific deformations from these manifolds. We then determine deformations for the entire image domain by learning the global manifold in such a way that it preserves the region-specific deformations. We evaluate the accuracy of our method by applying it to the LPBA40 dataset and measuring the overlap of the deformed segmentations. The result shows significant improvement in registration accuracy on cortex regions compared to other state of the art methods.

1 Introduction

The analysis of deformation from non-rigid registration has become an important component in brain image applications such as morphometric analysis [1] and atlas-based segmentation [6]. To improve registration accuracy and thus the subsequent analysis, recent publications on registration [3, 5, 7, 11] first learn the manifold capturing the neighborhood relationship of a set of images before registering individual scans. Registration then consists of determining the geodesic path between the image pairs and decomposing the deformation into a series of small deformations along that path. Since each subject moves only towards its nearby subject, the resulting deformations can be more accurate.

However, the state-of-the-art in this domain faces several challenges. First, the accuracy of the manifold in capturing the neighborhood relationship of the underlying data structure highly depends on the metric used for measuring differences between images. Current methods typically use a single metric over the entire image domain. For example, ABSORB [7] measures intensity difference of the images and GRAM [11] computes the distance based on pairwise deformations between whole brain anatomies. However, two images might be very similar in certain image regions but very different in other regions. A single global metric generally will blur those differences and thus not accurately capture the neighborhood relations of localized brain regions. A second challenge relates to the sample size used for training the manifolds. Since manifolds are constructed directly from image samples, training based on a limited number of the image samples is likely to result in a poor approximation of the true data structure. This concern is especially relevant to medical image domain, where studies are usually limited to a few hundred samples. We address these issues by developing a deformable registration method based on regional manifolds.

Our method constructs the manifold over the entire image region in two steps. First, it separately learns the manifolds for individual brain regions and uses these manifolds to compute region-specific deformations. The construction of these regional manifolds is now based on metrics that are much more sensitive to local variations within that region than a single global metric. In addition, the anatomical variation within a specific region is smaller compared to one captures in the entire image domain so that our method can faithfully learn regional manifolds with a relatively small number of samples. In the second step, our method learns the manifold over the entire image domain so that moving along geodesics of that manifold does not interfere with the deformations inferred from the regional manifolds. Specifically, we use a Markov Random Field model to produce smooth deformation maps across the entire image domain while preserving the region-specific deformations. In other words, our approach determines the optimal geodesic path over the entire image region by gradually warping localized brain regions according to the regional manifolds.

We demonstrate the advantage of our method by performing atlas-based segmentation on LPBA40 dataset. The results show significant improvement in registration accuracy on cortex regions in terms of overlap score compared to other state-of-the art registration methods.

2 Regional Manifold Learning based Registration

We now describe our Regional Manifold Learning based Registration (RMLR) as illustrated in Fig. 1. RMLR first defines a set of regions of interests (ROIs) in training images. Then, RMLR independently learns the manifolds for individual regions and finds the region-specific deformations constrained by the regional manifolds. Next, RMLR learns the manifold for the whole brain image based on regional manifolds. Finally, RMLR determines the deformation in the entire image domain based on the global manifold while preserving region-specific deformations. We outline each step in more detail in the remainder of this section.

Fig. 1.

Illustration of the proposed method: It first defines ROIs and then learns the manifolds for each ROI. Next, the global manifold is constructed based on these regional manifolds. Then, the localized brain regions will be warped to their nearest neighbors based on the regional manifolds (red and green), however adjacent regions will deform in mutually compatible ways maintaining the smoothness of the overall deformation based on the global manifold (blue).

Defining the ROIs

To define the ROIs, we first set the ROIs in the template image I_T. Specifically, we separate the entire image domain Ω into R image regions {Ω^r: r = 1, …, R} so that the union of all regions is a subset of Ω(i.e. ${\cup }_{r=1}^{r=R}{\mathit{\Omega }}^r\subset \mathit{\Omega }$) and each region does not overlap with another region, (i.e. for ∀r,∄s ≠ r: {Ω^r ∩ Ω^s ≠ ∅}). We then automatically find the corresponding regions in training images {I_i: i = 1, …, N} using a non-rigid registration [10].

Computing the Regional Manifolds

For each Ω^r, we learn the regional manifolds as in GRAM [5], which was originally proposed for learning the manifold of an entire image domain. In order to reduce the risk of boundary artifacts during registration, we compute the manifold for a larger image block B^r ⊃ Ω^r. Let ϕ_B^r (I_i, I_j) be the diffeomorphic deformation which maps the image block in I_i to the corresponding block in I_j. Then, the distance d_B^r (I_i, I_j) between blocks is defined as the weighted sum of intensity difference and field smoothness [3, 5]:

$$d_{B^r}(I_i,I_j):=\sum _{u\in B^r}{\Vert I_i\circ {\varphi }_{B^r}(I_i,I_j)(u)-I_j(u)\Vert }_2^2+{\lambda }_r{\Vert \nabla {\varphi }_{B^r}(I_i,I_j)(u)\Vert }_2^2,$$ (1)

where λ_r is a weighting parameter between intensity difference and field smoothness term, and ||·|| is the L2 norm. To reduce the computational burden in computing the distances for all pairs of blocks, we use the symmetric diffeomorphic registration [10] and compute d_B^r(I_i, I_j) only for i < j. Also, we empirically set λ_r so that two terms have the same maximum value over all pairs of images.

Based on these pairwise distances, we construct k-NN graph for each region whose nodes correspond to the image blocks. Heuristically, we set the smallest value that makes the k-NN graph connected as k. From the k-NN graph, we find the shortest paths from the block in the template I_T to all other image blocks. Then, the shortest paths from one root node to the rest form a spanning tree with respect to B^r. We consider this spanning tree M_Ω^r as the regional manifold for Ω^r that represents the neighborhood relations in the specific region.

Registering the ROIs

After computing the regional manifold, we construct the region-specific deformation based on this regional manifold. Between two image blocks in I_i and I_j, we choose the shortest path [i, p₁, …, p_l, j] within the graph M_Ω^r and define ϕ̂_B^r(I_i, I_j):= ϕ_B^r(I_i, I_p1) ∘ ··· ∘ ϕ_B^r (I_p^l, I_j) by concatenating the corresponding deformations between neighbor blocks along that path. ϕ̂_B^r(I_i, I_j) is diffeomorphic because composition operator preserves this property. The region-specific deformation ϕ̂_Ω^r(I_i, I_j) is simply the ϕ̂_B^r(I_i, I_j) restricted to the region Ω^r. This process is performed on all R regions independently, producing region-specific deformation fields.

Computing the Global Manifold

Given registration results in the ROIs, we compute the manifold for the entire image domain Ω. Similar to regional manifold computation, we measure the distance based on pairwise deformations ϕ_Ω(I_i, I_j) between I_i and I_j. To preserve pre-computed region-specific deformations while maintaining the smoothness of overall deformation, it is desirable to compute the globally diffeomorphic deformation in such a way that it keeps the region-specific deformations. Toward this, we determine ϕ_Ω(I_i, I_j) using an MRF based registration method [4]. A typical MRF model optimizes the following energy function:

$$E:=\sum _{s\in \mathcal{V}}{\theta }_s(x_s)+\sum _{(s,t)\in \mathcal{E}}{\theta }_{st}(x_s,x_t),$$ (2)

where 𝒱 is a set of nodes on the image, ℰ is a set of edges between neighbor nodes, and x_s is the label of node s ∈ 𝒱. Each label x_s corresponds to a displacement vector v(x_s) by which s moves to a new position. The unary term θ_s(x_s) represents the data cost of assigning label x_s at node s in terms of image dissimilarity. The smoothness term θ_st(x_s, x_t) penalizes the cost of label discrepancy between two neighboring nodes s and t, i.e. θ_st(x_s, x_t):= γ_st ·min{||v(x_s) − v(x_t)||₁, T_st}, with γ_st being a regularization constant and T_st being a threshold for truncation. In order to preserve the region-specific deformations, we define a modified unary term θ̂_s(x_s) as following:

$${\widehat{\theta }}_s(x_s):=\{\begin{array}{ll}L(1-\delta (x_s-{\widehat{x}}_s))\hfill & \text{for}s\in {\cup }_{r=1}^{r=R}{\mathit{\Omega }}^r\hfill \\ 1-\mathit{NCC}(x_s)\hfill & \text{for}s\in \mathit{\Omega }\backslash {\cup }_{r=1}^{r=R}{\mathit{\Omega }}^r,\hfill \end{array}$$ (3)

where L is very large, x̂_s is the pre-defined label from region-specific deformations, δ(·) is a delta function, and NCC is a normalized cross correlation. We optimize this energy model via tree re-weighted message passing method [8] with γ_st = 3 and T_st = 20. In order to guarantee the diffeomorphism, we constrain the displacement at each node not to exceed 40% of the spacing between two adjacent nodes [2]. The MRF-based registration for each pair of subjects gives us ϕ_Ω which is diffeomorphic and preserves the region-specific deformations.

Based on these MRF registrations, we construct the k-NN graph whose nodes represent the images. From this graph, we find a spanning tree from one root node to all images and consider this spanning tree M_Ω as the global manifold.

Registering the Whole Brain Images

We now find the global deformation ϕ̂_Ω(I_i, I_j) between I_i and I_j constrained by the global manifold M_Ω. We denote [i, G₁, …, G_m, j] as the shortest path between I_i and I_j in M_Ω. Then, ϕ̂_Ω(I_i, I_j) is defined as ϕ̂_Ω(I_i, I_j):= ϕ_Ω(I_i, I_G1) ∘ ··· ∘ ϕ_Ω (I_Gm, I_j), where ϕ_Ω is the MRF propagated deformation from the previous step. We now show that ϕ̂_Ω(I_i, I_j) preserves the region-specific deformation ϕ̂_Ω^r(I_i, I_j) in Ω^r.

We prove the claim by contradiction. First, in Ω^r, the deformation ϕ_Ω between neighbor nodes on M_Ω is defined by ϕ̂_Ω^r (according to M_Ω^r) because our MRF registration preserves the region-specific deformation. Then, the deformation for any path on M_Ω^r with the same start and end node [b, …, b] is the identity as M_Ω^r is loop-free and ϕ̂_Ω^ris symmetric. Thus, the deformation of path passing through a node twice, such as [a, …, b, …, b, …, c], is equivalent to the deformation defined by the path [a, …, b, …, c] without that loop. Now, suppose that there is an image pair with ϕ̂_Ω^r(I_i, I_j) ≠ ϕ̂_Ω(I_i, I_j) on Ω^r. Let [i, p₁, …, p_l, j] and [i, G₁, …, G_m, j] be the shortest paths in M_Ω^r and M_Ω, respectively. Then, we define [i, $G_1^{\prime },\dots ,G_n^{\prime }$, j] as the corresponding equivalent path of [i, G₁, …, G_m, j] in M_Ω^r by first replacing two neighboring nodes [G_q, G_q+1] with the corresponding path in M_Ω^r and then removing all loops according to the previous observation. We note that a node in either path [i, p₁, …, p_l, j] or [i, $G_1^{\prime },\dots ,G_n^{\prime }$, j] can only appear once in that path. If now ϕ̂_Ω^r(I_i, I_j) ≠ ϕ̂_Ω(I_i, I_j) on Ω^r, then two paths have to differ in at least one position k, i.e. $p_k\ne G_k^{\prime }$. This implies that the loop-free graph structure M_Ω^r has a loop as there are two unique loop-free paths between node i and j. As this contradicts our assumption of M_Ω^r, it follows that ϕ̂_Ω(I_i, I_j) always has to be equivalent to ϕ̂_Ω^r(I_i, I_j) on Ω^r.

3 Experiments on LPBA40 dataset

We measure the accuracy of our RMLR method by applying it to the LPBA40 dataset [9]. The dataset consists of 40 linearly aligned brain images each with 54 manually labeled segmentations. From those 40 scans, we empirically choose one scan as a template. We then determine the registration accuracy by aligning all other subjects to the template and measuring the overlap between the aligned and template segmentations. For comparison, we measure the accuracy of direct diffeomorphic Demons [10] which aligns the images without any manifold learning. In addition, we compare our RMLR method with whole brain GRAM [5] which learns the manifold based on a single metric over the entire image domain. For all three methods, we used the same registration parameters with three levels of resolution and the smoothing kernel size of 1.5. The remainder of this section discusses the experimental results in further detail.

Choosing Regions

The proposed method is independent of the choices of ROIs. In this paper, we try out the two schemes shown in Fig. 2. First, we define regions simply as cubes over the entire image domain as shown in Fig. 2(a). The 151×188×136 image is divided into 36 cubic regions whose sizes are 37×32×32. In order to separate cubic regions, we empirically set a gap of 20-voxels between regions. The second scheme uses the segmentation of template image I_T to define regions according to anatomical brain structures. As illustrated in Fig. 2(b), we specify 10 regions including frontal lobe, parietal lobe, occipital lobe, temporal lobe, and sub-cortical regions of left and right hemispheres. Each structure region is refined through erosion to guarantee the non-overlap between regions.

Fig. 2.

The mid-axial slice of the template image with (a) cubic regions and (b) structure regions. Red boundaries indicate ROIs.

Comparing Regional and Global Deformations

First, we visually compare the region-specific deformation with the whole brain deformation from our RMLR method. Fig. 3(a) shows an example of region-specific deformations based on regional manifolds. We only display y-displacement vector field in a mid-coronal slice. Black boundaries represent structure ROIs including frontal lobe, temporal lobe and subcortical regions. Initially, there is only zero-displacement vector field outside the ROIs. Fig. 3(b) illustrates the whole brain deformation based on the global manifold which is constructed from regional manifolds. Note that the whole brain deformation is globally smooth and preserves the region-specific deformation from the Fig. 3(a). This supports our claim that MRF registration produces the globally smooth deformation and the region-specific deformation can be preserved during the MRF registration and composition along the path on the global manifold.

Fig. 3.

y-displacement in a coronal view for (a) region-specific and (b) whole brain deformations from RMLR. Black boundaries represent the ROIs. Notice that whole brain deformation is globally smooth while preserving region-specific deformations.

Comparing the RMLR and GRAM

Next, we compare the regional manifolds from RMLR with the manifold produced by GRAM based on a single metric over a whole brain image. To do so, we visualize the shortest paths for the left hippocampus and right angular gyrus in Fig. 4(a) and Fig. 4(b), respectively. Upper row represents the GRAM manifold and lower row represents the regional manifolds from RMLR. First, we note that the paths are the same for both regions according to the GRAM manifold (upper) but not so for RMLR (lower). This indicates that the anatomical variation for one region is generally different from that of another region and the manifold based on a single global metric may not capture this local variation with limited samples. Furthermore, regional manifolds reflect more gradual changes in both regions in terms of hippocampus size and gyrus appearance compared to the path from GRAM manifold. This also confirms that regional manifold can better capture the anatomical variation in the specific region with a small number of samples.

Fig. 4.

Shortest path for (a) left hippocampus and (b) right angular gyrus. The upper row is the path by GRAM and the lower row is the path by regional manifolds. The numbers on top is the subject ID. Notice gradual decreases in hippocampus size and sulcal depth in the paths of regional manifolds compared with those from GRAM.

Measuring DICE scores

To measure registration accuracy, we compute the DICE score between manual segmentation and atlas-based segmentation from all registration methods. The average DICE score over all 54 labels on the original dataset is 57.4% and the score after direct diffeomorphic Demons is 72%. Whole brain GRAM slightly improves the score to 73.1%. RMLR increases the score to 75.1% with the cubic and to 75.2% with the structural region setting. This shows that our RMLR achieves improvement in average DICE score over all labels and that this improvement is somewhat independent towards the region selection schemes. For further comparison, we display in Fig 5 the 25, 50 and 75 percentiles of the DICE scores for the four different registration methods with respect to selected structures. RMLR produces scores with higher median and lower variation than GRAM or Demons. In particular, RMLR achieves statistically significant improvement in cortex regions (p < 0.05). If we use the scores of Demons as an indicator regarding the degree of difficulty in registering the regions then the regional manifolds provides the biggest improvements in difficult regions, such as angular gyrus, and only slightly impacted the results in easy regions, such as the temporal gyrus and subcortical regions. In summary, these results agree with our initial intuition that manifolds based on a single metric over the entire image domain are not as accurate in capturing local brain variations than ones that are based on regional manifolds.

Fig. 5.

Structure-specific DICE scores. Upper and lower bar represent 75 and 25 percentiles of DICE scores, respectively. Midpoint indicates the median.

4 Conclusion

In this paper, we proposed a deformable registration based on learning the manifolds of individual brain regions. Our method first learns the manifolds for specific regions and then computes region-specific deformations from these manifolds. We then determine deformations for the entire image domain by learning the global manifold while preserving the region-specific deformations via a MRF model. Experimental results on the LPBA40 dataset show that the proposed method can significantly improve registration accuracy compared to direct pairwise or whole brain manifold learning based registration methods. In a future work, we will investigate an adaptive way of our method for overlapping ROIs.