Joint Segmentation of Image Ensembles via Latent Atlases

Abstract

Spatial priors, such as probabilistic atlases, play an important role in MRI segmentation. However, the availability of comprehensive, reliable and suitable manual segmentations for atlas construction is limited. We therefore propose a joint segmentation of corresponding, aligned structures in the entire population that does not require a probability atlas. Instead, a latent atlas, initialized by a single manual segmentation, is inferred from the evolving segmentations of the ensemble. The proposed method is based on probabilistic principles but is solved using partial differential equations (PDEs) and energy minimization criteria. We evaluate the method by segmenting 50 brain MR volumes. Segmentation accuracy for cortical and subcortical structures approaches the quality of state-of-the-art atlas-based segmentation results, suggesting that the latent atlas method is a reasonable alternative when existing atlases are not compatible with the data to be processed.

1 Introduction

Probabilistic atlases are crucial for most MR segmentation methods due to the absence of well defined boundaries. Derived from comprehensive sets of manually labeled examples, atlases provide statistical priors for tissue classification and structure segmentation [1,9,13,14,18]. Although atlas-based segmentation methods often achieve accurate results, the need for spatial priors can be problematic. First, the availability of suitable atlases is limited since manual segmentation of a significant number of volumes requires expensive effort of an experienced physician. Second, the suitability of existing atlases for images from different populations is questionable. Examples include using normal adult brain atlas for brain parcellation of young children or patients with severe brain pathologies.

Recently, a few methods have been proposed to reduce or avoid the dependency on possibly incompatible atlases. In the atlas-based segmentation method in [3], topological constraints are used to avoid possible bias introduced by the atlas. In [19], manually labeled structures are used to support the automatic segmentation of neighboring structures within the same image. Tu et al. [17] propose a discriminative approach for the segmentation of adjacent brain structures using a set of discriminative features learned from training examples. Lord et al. [11] suggest a group-wise smoothing, segmentation and registration method for cross sectional MR scans. Bhatia et al. [5] update an initial atlas constructed from a different population using the evolving segmentations of multiple images.

Here we propose and demonstrate a method that does not use a set of training images or probabilistic atlases as priors. Instead we extract an ensemble of corresponding structures simultaneously. The evolving segmentation of the entire image set supports each of the individual segmentations. In practice, a subset of the model parameters, called the spatial parameters, is inferred as part of the joint segmentation processes. These latent spatial parameters, which can be viewed as a ‘dynamic atlas’, are estimated exclusively from the data at hand and a single manual segmentation. The latent atlas is used as a Markov Random Field (MRF) prior on the tissue labels. The main novelty of the method with respect to other group-wise segmentation methods such as [11,5] is the consistent statistically-driven variational framework for MR ensemble segmentation.

Our contribution is two-fold. We introduce a level set framework, that is based on probabilistic principles, in which segmentation uncertainty is expressed by the logistic function of the associated level set values, similar to [15]. We then use it for group-wise segmentation. We evaluate our method by segmenting the amygdala, temporal gyrus and hippocampus in each hemisphere in 50 MR brain scans. The dice scores achieved by our method approach the atlas-based segmentation results of [14].

2 Problem Definition and Probabilistic Model

Our objective is to segment a particular structure or region of interest in N aligned MR images. Specifically, we consider the 2-partition problem where each voxel in image I_n (n = 1 … N) is assigned to either the foreground (structure of interest) or the background.

Let each image I_n: Ω → ℝ⁺, be a gray level image with V voxels, defined on Ω ⊂ ℝ³ and Γ_n: Ω → {0, 1} be the unknown segmentation of the image I_n. We assume that each Γ_n is generated iid from a probability distribution p(Γ| θ_Γ) where θ_Γ is a set of unknown parameters. We also assume that Γ_n generates the observed image I_n, independently of all other image-segmentation pairs, with probability p(I_n|Γ_n, θ_I,n) where θ_I,n are the parameters corresponding to image I_n. We assign a specific set of intensity parameters to each image since the acquisition conditions might vary across subjects.

Let {I₁ … I_N } be the given set of aligned images that form the observed variable in our problem and let Γ = {Γ₁ … Γ_N} be the corresponding segmentations. The joint distribution p(I₁ … I_N, Γ₁ … Γ_N|Θ) is governed by the composite set of parameters Θ = {θ_Γ, θ_I,1 … θ_I,N }. Our goal is to estimate the segmentations Γ.

We jointly optimize for the segmentations Γ and the parameters Θ, assuming that I₁ … I_N are independent:

$$\{\widehat{\mathrm{\Theta }},\widehat{\mathrm{\Gamma }}\}=arg\underset{\{\mathrm{\Theta },\mathrm{\Gamma }\}}{max}logp(I_1\dots I_N,{\mathrm{\Gamma }}_1\dots {\mathrm{\Gamma }}_N;\mathrm{\Theta })$$ (1)

$$=arg\underset{\{\mathrm{\Theta },\mathrm{\Gamma }\}}{max}\sum _{n=1}^{N}logp(I_n,{\mathrm{\Gamma }}_n;\mathrm{\Theta })$$ (2)

$$=arg\underset{\{\mathrm{\Theta },\mathrm{\Gamma }\}}{max}\sum _{n=1}^N[logp(I_n\mid {\mathrm{\Gamma }}_n;{\theta }_{I,n})+logp({\mathrm{\Gamma }}_n;{\theta }_{\mathrm{\Gamma }})].$$ (3)

We propose to alternate between estimating the maximum a posteriori (MAP) segmentations and updating the model parameters. For a given value of the model parameters Θ̂, Equation (3) implies that the segmentations can be estimated by solving N separate MAP problems:

$${\widehat{\mathrm{\Gamma }}}_n=arg\underset{{\mathrm{\Gamma }}_n}{max}[logp(I_n\mid {\mathrm{\Gamma }}_n;{\theta }_{I,n})+logp({\mathrm{\Gamma }}_n;{\theta }_{\mathrm{\Gamma }})].$$ (4)

We then fix Γ̂ and estimate the model parameters Θ = {θ_Γ, θ_I,1, … θ_I,N} by solving two ML problems:

$${\widehat{\theta }}_{I,n}=arg\underset{{\theta }_{I,n}}{max}logp(I_n\mid {\mathrm{\Gamma }}_n,{\theta }_{I,n})$$ (5)

$${\widehat{\theta }}_{\mathrm{\Gamma }}=arg\underset{{\theta }_{\mathrm{\Gamma }}}{max}\sum _{n=1}^{N}logp({\mathrm{\Gamma }}_n\mid {\theta }_{\mathrm{\Gamma }}).$$ (6)

In the following sections we present a level-set framework that is motivated by this probabilistic model. We reformulate the estimation problem stated in Eq. (4) such that the soft segmentations p(Γ_n) rather then the Γ_n are estimated.

3 Probabilistic View of the Level Set Framework

Now we draw the connection between the probabilistic model presented above and the level set framework for segmentation. Let φ_n: Ω → ℝ denote a level set function associated with image I_n. The zero level C_n = {x ∈ Ω| φ_n(x) = 0} defines the interface between the partitions of I_n. Vese and Chan [6] represent the image partitions by a regularized variant of the Heaviside function of φ_n, e.g., ${\stackrel{\sim }{H}}_{\epsilon }({\phi }_n)=\frac{1}{2}(1+\frac{2}{\pi }arctan(\frac{{\phi }_n}{\epsilon }))$. Alternatively, we can use the hyperbolic tangent to achieve the same goal:

$${\stackrel{\sim }{H}}_{\epsilon }({\phi }_n)=\frac{1}{2}\left(1+tanh\left(\frac{{\phi }_n}{2\epsilon }\right)\right)=\frac{1}{1+e^{-{\phi }_n/\epsilon }}.$$ (7)

For ε = 1, the function H̃_ε(·) is the logistic function. Similar to [15], we define the level set function φ_n using the log-odds formulation instead of the conventional signed distance function:

$${\phi }_n(\mathbf{x})\triangleq \epsilon \text{logit}(p)=\epsilon log\frac{p(\mathbf{x}\in w)}{1-p(\mathbf{x}\in \omega )}=\epsilon log\frac{p(\mathbf{x}\in \omega )}{p(\mathbf{x}\in \mathrm{\Omega }\backslash \omega )}.$$ (8)

The scalar ε determines the scaling of the level set function φ_n with respect to the ratio of the probabilities. Substituting this definition into Eq. (7) we obtain

$${\stackrel{\sim }{H}}_{\epsilon }({\phi }_n(\mathbf{x}))=\frac{1}{1+p(\mathbf{x}\in \mathrm{\Omega }\backslash \omega )/p(\mathbf{x}\in \omega )}=p(\mathbf{x}\in \omega ),$$ (9)

which implies that the function H̃_ε(φ_n(x)) can be viewed as the probability that the voxel in location x belongs to the foreground region. The functions H̃_ε(φ_n(x)) and H(φ_n(x)) therefore represent soft and hard segmentations, respectively. To simplify the notation we omit the subscript ε in the rest of the paper.

In the following subsections we relate the terms in Eq. (3) to the energy terms in the classical level set functional.

3.1 Image Likelihood Term

Let us first consider the image likelihood term in Eq. (3):

$$logp(I_n\mid {\mathrm{\Gamma }}_n,{\widehat{\theta }}_{I,n})=\sum _{\{v\mid {\mathrm{\Gamma }}_n^v=1\}}log{p}_{\text{in}}(I_n^v;{\theta }_{I,n})+\sum _{\{v\mid {\mathrm{\Gamma }}_n^v=0\}}log{p}_{\text{out}}(I_n^v;{\theta }_{I,n}),$$ (10)

where p_in and p_out are the probability distributions of the foreground and back-ground image intensities, respectively.

Let E_I ≅ −log p(I_n| Γ_n, θ̂_I,n) define the energy term associated with the image likelihood term. Using the level-set formulation and replacing the binary labels Γ_n in Eq. (10) with a soft segmentation represented by H̃(φ_n), we get:

$$E_I({\phi }_n,\mathrm{\Theta })=-{\int }_{\mathrm{\Omega }}\left[log{p}_{\text{in}}(I_n;{\theta }_{I,n})\stackrel{\sim }{H}({\phi }_n(\mathbf{x}))+log{p}_{\text{out}}(I_n;{\theta }_{I,n})\stackrel{\sim }{H}(-{\phi }_n(\mathbf{x}))\right]d\mathbf{x}.$$ (11)

If we use, for example, Gaussian densities for p_in and p_out we get the familiar minimal variance term [6,12]. Here, we use a Gaussian mixture to model the background, as described later in the paper.

3.2 Spatial Prior Term

We define the prior probability p(Γ_n|θ_Γ) to be a Markov Random Field (MRF):

$$p({\mathrm{\Gamma }}_n\mid {\theta }_{\mathrm{\Gamma }})=\frac{1}{Z({\theta }_{\mathrm{\Gamma }})}\prod _{v=1}^V{({\theta }_{\mathrm{\Gamma }}^v)}^{{\mathrm{\Gamma }}_n^v}{(1-{\theta }_{\mathrm{\Gamma }}^v)}^{(1-{\mathrm{\Gamma }}_n^v)}{e}^{-f({\mathrm{\Gamma }}_n^v,{\mathrm{\Gamma }}_n^{\mathcal{N}(v)})},$$ (12)

where Z(θ_Γ) is the partition function and (v) are the closest neighbors of voxel v. The function f (·) accounts for the interactions between neighboring voxels. If we omit the pairwise term in Eq. (12), the prior on segmentations p(Γ_n|θ_Γ) reduces to a Bernoulli distribution, where the parameters θ_Γ represent the probability map for the structure of interest. The introduction of the pairwise clique potentials complicates the model but encourages smoother labeling configurations.

Using the continuous level set formulation with soft segmentation, we define the spatial energy term. as follows:

$$E_S({\phi }_n,\mathrm{\Theta })=-{\int }_{\mathrm{\Omega }}\left[log{\theta }_{\mathrm{\Gamma }}(\mathbf{x})\stackrel{\sim }{H}({\phi }_n(\mathbf{x}))+log(1-{\theta }_{\mathrm{\Gamma }}(\mathbf{x}))\stackrel{\sim }{H}(-{\phi }_n(\mathbf{x}))\right]d\mathbf{x}$$ (13)

As in [4] we ignore the partition function, approximating the MRF model above. The logarithm of the pairwise clique potential term f (·) can be configured to act as a finite difference operator approximating the gradient of Γ_n at the voxel v [10]. It can be therefore viewed as an approximation of the continuous term

$$E_{\text{LEN}}({\phi }_n)={\int }_{\mathrm{\Omega }}\mid \nabla \stackrel{\sim }{H}({\phi }_n(\mathbf{x}))\mid d\mathbf{x},$$ (14)

which is the commonly used smoothness constraint as reformulated in [6].

3.3 The Unified Energy Functional

We now construct the cost functional for φ₁ … φ_N and the parameters Θ by combing Eqs. (11),(13) and (14):

$$E({\phi }_1\dots {\phi }_N,\mathrm{\Theta })=\gamma E_{\text{LEN}}+\beta E_I+\alpha E_S,$$ (15)

where α = 1 − β − γ. As in [16] we tune the weights such that the contributions of the energy terms E_LEN, E_I and E_S to the overall cost are balanced.

4 Alternating Minimization Algorithm

We optimize the unified functional (15) in a set of alternating steps. For fixed model parameters Θ, the evolution of each of the level set functions φ_n follows the gradient descent equation:

$$\frac{\partial {\phi }_n}{\partial t}=\stackrel{\sim }{\delta }({\phi }_n)\left\{\gamma \text{div}(\frac{\nabla {\phi }_n}{\mid \nabla {\phi }_n\mid })+\beta \left[log{p}_{\text{in}}(I_n(\mathbf{x});{\widehat{\theta }}_{I,n})-log{p}_{\text{out}}(I_n(\mathbf{x});{\widehat{\theta }}_{I,n})\right]+\alpha \left[log{\widehat{\theta }}_{\mathrm{\Gamma }}-log(1-{\widehat{\theta }}_{\mathrm{\Gamma }})\right]\right\},$$ (16)

where $\stackrel{\sim }{\delta }({\phi }_n)\triangleq {\stackrel{\sim }{\delta }}_{\epsilon }({\phi }_n)=\frac{d{\stackrel{\sim }{H}}_{\epsilon }({\phi }_n)}{d{\phi }_n}$ is the derivative of the Heaviside function, i.e.,

$${\stackrel{\sim }{\delta }}_{\epsilon }({\phi }_n)=\frac{1}{2\epsilon }\text{sech}(\frac{{\phi }_n}{2\epsilon })=\frac{1}{\epsilon cosh(\frac{{\phi }_n}{\epsilon })}.$$

For fixed segmentations φ_n the model parameters are recovered by differentiating the cost functional (15) with respect to each parameter.

4.1 Intensity Parameters

We assume that the intensities of the structure of interest are drawn from a normal distribution, i.e., $p_{\text{in}}(I_n;{\theta }_{I,n})=\mathcal{N}(I_n;{\mu }_n,{\sigma }_n^2)$. The distribution mean and variance of the foreground region of I_n are updated at every iteration. The intensities of the background tissues are modeled as a K-Gaussian mixture:

$$p_{\text{out}}(I_n;{\theta }_{I,n})=\text{GMM}({\mu }_n^1\cdots {\mu }_n^K,{\sigma }_n^1\cdots {\sigma }_n^K,w_n^1\cdots w_n^K),$$

where $w_n^k$ is the mixing proportion component k in the mixture. The Gaussian mixture model parameters are estimated using expectation maximization (EM) [7].

4.2 Spatial Parameters

We estimate the spatial function θ_Γ(x), constructing a dynamically evolving latent atlas, by optimizing the sum of the energy terms the depend on θ_Γ:

$${\widehat{\theta }}_{\mathrm{\Gamma }}=arg\underset{{\theta }_{\mathrm{\Gamma }}}{max}\sum _{n=1}^N{\int }_{\mathrm{\Omega }}[\stackrel{\sim }{H}({\phi }_n(\mathbf{x}))log({\theta }_{\mathrm{\Gamma }}(\mathbf{x}))+(1-\stackrel{\sim }{H}({\phi }_n(\mathbf{x})))log(1-{\theta }_{\mathrm{\Gamma }}(\mathbf{x}))]d\mathbf{x},$$

yielding ${\widehat{\theta }}_{\mathrm{\Gamma }}(\mathbf{x})=\frac{1}{N}{\sum }_{n=1}^N\stackrel{\sim }{H}({\phi }_n(\mathbf{x}))$.

5 Experimental Results

We test the proposed approach on 50 MR brain scans. Some of the subjects in this set are diagnosed with the first episode schizophrenia or affective disorder. The MR images (T1, 256 × 256 × 128 volume, 0.9375 × 0.9375 × 1.5 mm³ voxel size) were acquired by a 1.5-T General Electric Scanner. The data was originally acquired for brain morphometry study [8]. In addition to the MR volumes, manual segmentations of three structures (superior temporal gyrus, amygdala, and hippocampus) in each hemisphere were provided for each of the 50 individuals and used to evaluate the quality of the automatic segmentation results. MR images are preprocessed by skull stripping. The volumes were aligned using B-spline registration according to [2].

Assuming that the manual segmentation of a single instance is given, we initialize the latent atlas θ_Γ by the Heaviside function of a single manual segmentation smoothed with a Gaussian kernel of width σ (σ = .35 in our experiments). We ran the experiments twice, initializing the level-set functions of the signed distance function of the given manual segmentation or a sphere, with center and radius corresponding to the manually segmented structure. The results obtained using the first mode of initialization were slightly better. We excluded the image associated with the given manual segmentations from the ensemble. The algorithm was implemented in Matlab and ran on the average 7.5 minutes for 50 cropped volumes, excluding the time of the initial estimate of the background intensity of the Gaussian mixture. About 7 iterations were needed until convergence which was obtained when the update of the level-set functions did not induce changes in the corresponding boundaries.

We used the Dice score to evaluate segmentations. The results are shown in Figure 1. In contrast to [14] we do not use spatial priors, neither do we use hierarchical multi-stage segmentation model. Nevertheless, the Dice scores obtained by our method approache the state-of-the-art atlas-based segmentation results reported there. To exemplify the significance of a latent atlas, generated concurrently with the segmentation, we also show a comparison to the segmentation results obtained by using an atlas constructed from a single manual segmentation smoothed by a Gaussian kernel, without the update procedure. Figure 2 shows segmentation examples of the three pairs of structures in representative individual brains. Coronal views of the resulting 3D atlases for each pair of structures are shown in the fourth column of Figure 2.

Fig. 1.

The mean and standard deviation of the Dice scores calculated for six structures in the ensemble. The latent atlas segmentation (green) is compared to the atlas-based segmentation (blue) reported in [14] and to the segmentation obtained by using a single manual segmentation as an atlas (red).

Fig. 2.

Three cross-sections of 3D segmentations of Hippocampus, Amygdala and Superior Temporal Gyrus in the left and right hemispheres. Automatic segmentation is shown in red. Manual segmentation is shown in blue. Fourth column: Coronal views of the resulting atlases for each pair of structures.

6 Discussion and Conclusions

We presented a level set framework for segmentation of MR image ensembles, that is motivated by a generative probabilistic model. Unlike most previous methods, we do not use spatial priors in the form of a probabilistic atlas. Instead, spatial latent parameters, which form a ‘dynamic atlas’, are inferred from the data set through an alternating minimization procedure.

The quality of the segmentation results obtained for ensembles of brain structures shows that the proposed method presents a reasonable alternative to standard segmentation techniques when a compatible atlas is not available.

An on-going research is now conducted to demonstrate the ability of the proposed algorithm to handle pathological cases (e.g., in a longitudinal or a multimodal study) where the atlas-based approach still fails.