LOCUS: LOcal Cooperative Unified Segmentation of MRI brain scans

Abstract

We propose to carry out cooperatively both tissue and structure segmentations by distributing a set of local and cooperative models in a unified MRF framework. Tissue segmentation is performed by partitionning the volume into subvolumes where local MRFs are estimated in cooperation with their neighbors to ensure consistency. Local estimation fits precisely to the local intensity distribution and thus handles nonuniformity of intensity without any bias field modelization. Structure segmentation is performed via local MRFs that integrate localization constraints provided by a priori general fuzzy description of brain anatomy. Structure segmentation is not reduced to a postprocessing step but cooperates with tissue segmentation to gradually and conjointly improve models accuracy. The evaluation was performed using phantoms and real 3T brain scans. It shows good results and in particular robustness to nonuniformity and noise with a low computational cost.

1. Introduction

MRI brain scan segmentation is a challenging task and has been widely addressed in the last 15 years. Difficulties in automatic segmentation arise from various sources including the size of the data, the low contrast between tissues, the limitations of available a priori knowledge, local perturbations such as noise or global perturbations such as intensity nonuniformity. Current approaches share three main characteristics: first, tissue and structure segmentations are considered as two separate tasks whereas they are clearly linked. Second, for a robust to noise segmentation, the Markov Random Field (MRF) probabilistic framework is classically used to introduce spatial dependencies between voxels [1, 2]. Third, tissue models are generally estimated globally through the entire volume and do not reflect spatial intensity variations within each tissue, due mainly to biological tissue properties and to MRI hardware imperfections. Only the latter is generally addressed, modeled by the introduction of an explicit so called “bias field” model to estimate. Local segmentation is an attractive alternative. The principle is to compute models in various subvolumes to fit better to local image properties. However, the few local approaches proposed to date are clearly limited: they use local estimation as a preprocessing step only to estimate a bias field model [3], a training set for statistical local shape modelling [4], redondant information to ensure consistency and smoothnesss between local estimated models [5, 6], or an atlas providing a priori local spatial information [7] greedily increasing computational cost. We present in this paper an original LOcal Cooperative Unified Segmentation (LOCUS) approach which 1) performs tissue and structure segmentation by distributing a set of cooperating local MRF models through the volume, 2) segments structures by introducing prior localization constraints in a MRF framework and 3) ensures local models consistency and tractable computational time via specific cooperation and coordination mechanisms.

2. Method

2.1. MRF Segmentation

We consider a finite set of N voxels V = {1, …N} on a regular 3-D grid. Our aim is to assign each voxel i to one of K classes considering the observed greylevel intensity y_i at voxel i. Both observed intensities and unknown classes are considered to be random fields denoted respectively by Y = {Y₁, …, Y_N} and Z = {Z₁, …, Z_N}. Each random variable Z_i takes its value in {e₁,…, e_K} where e_k is a K-dimensional binary vector corresponding to class k. Only the k^th component of this vector is non zero and is set to 1. In a traditionnal Markov model based segmentation framework, it is assumed that the conditional field Z given Y = y is a Markov random field, ie. $p(\mathbf{z}\mid \mathbf{Y}=\mathbf{y},\mathbf{\Phi })=W_{y,\mathrm{\Phi }}^{-1}exp(-H(\mathbf{z}\mid \mathbf{y},\mathrm{\Phi }))$, where H (z |y, Φ) is an energy function depending on some parameters Φ = (Φ_y, Φ_z) and given by:

$$H(\mathbf{z}\mid \mathbf{y},\mathrm{\Phi })=H(\mathbf{z}\mid {\mathrm{\Phi }}_z)-\sum _{i\in V}logp(y_i\mid z_i,{\mathrm{\Phi }}_y).$$ (1)

This energy is a combination of two terms: the first term is a regularization term that accounts for spatial dependencies between voxels. Denoting by (i) the neighbors of voxel i and by ^t z_i the transpose of vector z_i, we will consider a Potts model with external field:

$$H(\mathbf{z}\mid {\mathrm{\Phi }}_z)=\sum _{i\in V}({{}^{t}z}_i{v}_i-\frac{\beta }{2}\sum _{j\in \mathcal{N}(i)}{{}^{t}z}_i{z}_j).$$ (2)

The second summation in (2) tends to favor neighbors that are in the same class when β is positive. This β parameter accounts for the strengh of spatial interaction. Other parameters are the v_i’s that are K-dimensional vectors defining the so-called external field. In this case Φ_z = {v₁, …, v_N, β}. The v_i’s can be related to a priori weights accounting for the relative importance of the K classes at site i. The introduction of these extra parameters in the standard Potts model enables us to integrate a priori knowledge on classes. The second term in (1) is a data-driven term based on intensities. For MRI we generally consider Gaussian probability density functions for each k, p (y_i |z_i = e_k, Φ_y) = g_{μk σk} (y_i), with Φ_y = {μ_k, σ_k, k = 1…K}. Segmentation is then performed according to the Maximum A Posteriori principle (MAP) by maximizing over z the probability p (z|y, Φ). This requires the evaluation of an intractable normalizing constant W_{y, Φ} and the estimation of the unknown parameters Φ. A standard approach is to use EM-based algorithms to globally estimate the parameters through the entire volume. We propose in the next subsection a LOcal and Cooperative version of EM (LOG-EM) for local segmentation approaches.

2.2. Local Cooperative Tissue Segmentation (LOCUS-T)

We partition the volume into a set of C non-overlapping local subvolumes V_c, c ∈ C and distribute one local MRF model M_c per subvolume. We consider K = 3 tissue classes: CSF (Cephalo-Spinal Fluid), GM (Grey Matter) and WM (White Matter). The hidden tissue classes t_i’s take their values in {e₁, e₂, e₃} respectively for classes {e_CSF, e_GM, e_WM}. Each local MRF model M_c is defined by the Gibbs distribution of energy (see Section 2.1):

$$H^c(\mathbf{t}\mid \mathbf{y},{\mathrm{\Phi }}^c)=\sum _{i\in V_c}\left[{{}^{t}t}_i{\lambda }_i^c-\frac{{\beta }^c}{2}\sum _{j\in \mathcal{N}(i)}{{}^{t}t}_i{t}_j-logp(y_i\mid t_i,{\mathrm{\Phi }}_y^c)\right],$$ (3)

where the parameters ${\mathrm{\Phi }}^c=\{{\mathrm{\Phi }}_t^c,{\mathrm{\Phi }}_y^c\}$ have to be estimated. However, the external field denoted by $\{{\lambda }_1^c,\dots {\lambda }_N^c\}$ is not estimated but used to incorporate information coming from structure segmentation to perform cooperation. ${\mathrm{\Phi }}_t^c$ reduces then to {β^c}, while ${\mathrm{\Phi }}_y^c$ are the estimated parameters of the local Gaussian tissue intensity models. The MRF model M_c introduces spatial dependencies between voxels in its subvolume V_c, providing consistent neighboring labels. Because the estimation is local, some tissue classes are likely to be under-represented in some subvolumes, leading to poor model estimations with a classical EM scheme. We propose a LOcal and Cooperative version of EM (LOC-EM) for spatially organized subvolumes to ensure a global consistency of local models. We denote by (M_c) the set of MRF models neighbouring M_c and introduce in EM a set of cooperation and coordination mechanisms as follows:

Cooperation between M_c and (M_C)

• - Model Checking: we compute for each M_c a model M̃_c averaging the models of (M_c). Then, for each class k, we compute the KullBack-Leibler distance ${\mathcal{D}}_k^c$ between intensity models of M_c and M̃_c.

• - Model Correction: if ${\mathcal{D}}_k^c$ is larger than a given threshold, we compute the corrected mean and variance of class k from a linear combination of intensity models in M_c and M̃_c using ${\mathcal{D}}_k^c$ to determine the linear coefficients.

• - Model Interpolation: from local estimations in neighbouring subvolumes we get then one intensity model per voxel by using cubic splines interpolation between corrected models of M_c and of (M_c). This results in a non-stationary field-like approach and has the advantage to ensure smooth model variation between neighboring subvolumes and to intrinsically handle nonuniformity of intensity inside each subvolume.

Coordination between MRF models

• - System starting: each local EM enters in idle mode after its local initialization. A global intensity model is computed using the Fuzzy C-Mean algorithm and then only the MRF models closest to the global model are activated.

• - Knowledge spreading: when the EM algorithm for M_c is stabilized, its neighbors are activated to perform estimation in turn. For already stabilized EM, model checking is performed. If it results in model correction and model interpolation, the corresponding EM are restarted to take into account the updated models modifications.

2.3. Cooperative Tissue and Structure Segmentation (LOCUS-TS)

We extend the approach above to segment both tissues and structures. We currently consider L = 9 subcortical structures: the ventricular system, the Frontal Horns, the Caudate Nuclei, the Thalamus, and the Putamens. For each target structure l we define a local Markov model M_l that labels voxels of its subvolume V_l in K = 2 classes referred to as structure and background. Denoting by s = {s_i, i ∈ V_l} the hidden classes, the energy function of M_l is given by:

$$H^l(\mathbf{s}\mid \mathbf{y},{\mathrm{\Psi }}^l)=\sum _{i\in V_l}\left[{{}^{t}s}_i{\alpha }_i^l-\frac{{\beta }^l}{2}\sum _{j\in \mathcal{N}(i)}{{}^{t}s}_i{s}_j-logp(y_i\mid s_i,{\mathrm{\Psi }}_{y,i}^l)\right],$$ (4)

with ${\mathrm{\Psi }}^l=\{{\beta }^l,{\alpha }_i^l,{\mathrm{\Psi }}_{y,i}^l,i\in V_l\}$ and s_i ∈ {e₁, e₂} = {e_B, e_S} for a voxel of the background or a voxel belonging to structure l.

Integration of prior localization constraints in the MRF

Automatic structure segmentation cannot rely only on radiometry information because intensity distributions of grey nuclei are largely overlapping. A priori knowledge should be introduced. A recent way to provide it is to describe brain anatomy with generic fuzzy spatial relations [8, 9]. Three kind of relations are generally considered: distance, symmetry and orientation relations. They are expressed as 3D fuzzy maps to take into account the generic nature of the provided knowledge. Each subcortical structure is described by a set of such generic fuzzy spatial relations provided by a brain anatomist. Fusion operators between fuzzy sets are then used to combine the knowledge provided by each spatial relation and provide a generic Fuzzy Localization Map (FLM) of the structure in the volume. The FLM f^l of structure l is used in two ways: first it dynamically provides the structure subvolume V_l containing the structure l by a simple thresholding. Second, it can be integrated as an a priori anatomical knowledge in the MRF framework via the external field {α_i, i ∈ V_l} We denote by $f_i^l$ the value of f^l at voxel i and propose to introduce the prior fuzzy knowledge of the FLM as relative prior weights for each voxel i, by setting ${\alpha }_i^l={}^t[{\alpha }_i^l(e_B),{\alpha }_i^l(e_S)]$ to ${\alpha }_i^l=\gamma {}^t[-log(1-f_i^l),-log{f}_i^l]$, where γ > 0 adjusts the influence of the external field. When $f_i^l\approx 0$, voxel i is unlikely to belong to the structure. It follows ${\alpha }_i^l(e_B)<{\alpha }_i^l(e_S)$ which favors in (4) the background class. When $f_i^l\approx 1$, voxel i is likely to belong to the structure. In that case ${\alpha }_i^l(e_B)>{\alpha }_i^l(e_S)$ and the class structure is favored.

Cooperation and coordination mechanisms between MRF models

Let _T→S (l) (resp. _S→T (c)) denotes the tissue (resp. structure) subvolumes that overlap with the structure subvolume V_l (resp. tissue subvolume V_c). _S→S (l) denotes structures using l as a reference in a spatial relation. MRF models cooperate to make the segmentations gradually more accurate as described below.

• - Structure segmentation starting: Structure models wait for their corresponding tissue models convergence to start their segmentation with sufficient reliable tissue knowledge.

• - Updating structure models via tissue models: each structure l being composed of a single tissue T^l ∈ {e_CSF, e_GM, e_WM}, we do not estimate intensity models of class structure and class background. We rather compute them from tissue intensity models by setting for i ∈ V_l:

  $$\{\begin{array}{l}p(y_i\mid s_i=e_S,{\mathrm{\Psi }}_{y,i}^l)=p(y_i\mid t_i=T^l,{\mathrm{\Psi }}_{y,i})\hfill \\ p(y_i\mid s_i=e_B,{\mathrm{\Psi }}_{y,i}^l)=\underset{t\in \{e_{\mathit{CSF}},e_{GM},e_{WM}\}}{max}p(y_i\mid t_i=t,{\mathrm{\Psi }}_{y,i}),\hfill \end{array}$$

  so that improvements in tissue intensity models estimation are dynamically taken into account by structure models.

• - Feedback of Structure Segmentation on Tissue Segmentation: conversely, results from structure models are integrated in the tissue model via the external field λ^c (see Eq. 3). We express it as the disjunctive fusion over l of posteriori probabilities p (s|y,Ψ^l) coming from structures l of _S→T (c). It follows that structure segmentation is not reduced to a second step but is combined to tissue segmentation to improve their performances.

• - Updating Fuzzy Maps: when the segmentation of structure l is updated the structure models of _S→S (l) take it into account by re-computing their spatial relations with respect to l, making the knowledge gradually more accurate.

A synthetic view of our approach is given in Fig 1.

Fig. 1.

Cooperative LOCUS-TS approach: for tissues (left), a LOG-EM cycle is distributed to each subvolume. For structures (right), each structure subvolume is associated to an EM cycle which cooperates with tissues.

3. Results

The evaluation was performed using both phantoms and real 3T brain scans. We choose not to estimate the β parameter but considered it as β = 1/T with T a decreasing temperature. Experimentally T : 10 → 5 provided good results. We first quantitatively compared LOCUS-T to two well known approaches, FAST [2] of FSL and SPM5 [10], with the Dice similarity metric on the BrainWeb phantoms with 40% of nonuniformity and different noise values (see Fig. 2). Fig. 3 shows a visual evaluation on a very high bias field real 3T brain scan⁴. Fig. 3 shows that SPM5 failed, probably due to the use of a priori information hard to match with a surface coil brain acquisition. Next, we evaluated the cooperative tissue and structure segmentations. Three experts have manually segmented on BrainWeb the left caudate nucleus, the left putamen and the left thalamus, from which we computed a ground truth segmentation using STAPLE [11]. Fig. 4 illustrates the gradual improvements of tissue and structure segmentations provided by LOCUS-TS. At the first convergence (time t₀) of EM, the Dice index is respectively 0.76, 0.77 and 0.72 for the caudate nucleus, the putamen and the thalamus. At the end, it reaches 0.76, 0.79 and 0.80. Fig. 5 shows qualitative evaluation of LOCUS-TS on a real 3T brain scan.

Fig. 2.

Comparison of LOCUS-T, FSL and SPM5 on the BrainWeb phantoms.

Fig. 3.

Tissue segmentations provided by SPM5 (b), FSL (c) and LOCUS-T (d).

Fig. 4.

Segmentation by LOCUS-TS on BrainWeb(a), 3D reconstruction(b), gradual improvement of putamen segmentation(c) and corresponding tissue segmentation(d).

Fig. 5.

Segmentation by LOCUS-TS on a real 3T image(a), 3D structure reconstruction(b), gradual improvement of thalamus segmentation(c) and tissue segmentation(d).

4. Discussion

Classical global approaches require to estimate an explicit bias field model to take into account the tissue intensity inhomogeneities [10, 12] due to MRI hardware imperfections. This model relies on the non realistic assumption of a single multiplicative bias field affecting all tissue classes equally. In contrast, the local estimation of MRF parameters in different subvolumes intrinsically handles the different sources of tissue intensity inhomogeneities. Our approach, with specific cooperation mechanisms between local models, appears to be an elegant and time efficient way to ensure global consistency of local models for tissue segmentation. It shows a significantly higher robustness to noise when compared to SPM5 (see Fig. 2), and more generally comparable results for a reduced computing time, namely, approximately 4min for LOCUS-T and respectively 8min and 14min for FSL and SPM5 on a 4Ghz Pentium, 1Go RAM. It illustrates that easy-to-segment subvolumes converge quickly, allowing the system to focus on other areas. LOCUS-T appears to be robust to very high intensity inhomogeneities as well (see Fig. 3), while SPM5, which uses an a priori atlas, fails in the segmentation and FSL does not estimate a correct bias field. In addition, instead of considering structure segmentation as a postprocessing, we propose to combine tissue and structure segmentations in a cooperative way: tissue and structure models are mutually updated, making both models gradually more accurate and providing optimal results (see Fig. 4 and 5). Improvements are particularly significant for structures such as thalamus or putamen for which contrast to noise ratio is low (see Dice index improvement in Section 3). As regards the additional use of a priori anatomical knowledge, standard structure segmentation approaches rely on a global atlas. Atlas warping methods are classicaly time consuming and more or less limited due to inter-subject variability. [7] introduced an atlas knowledge via the interaction energy term in a MRF. To be tractable, this solution considers that only the first order conditional dependence is important. This simplifying assumption, which reduces the Markov property, is not required with our approach. We consider instead an a priori description of brain anatomy based on fuzzy spatial relations. This was introduced in [8] with a region-based approach, while it is used in [9] in a deformable model framework. However, the image preprocessing steps required make this approach difficult to apply on high field images, with high intensity nonuniformity, or on non homogeneous structures such as putamen. Our solution consists in introducing fuzzy localization constraints as relative prior weights for each voxel in the MRF framework via an external field term. It does not suffer from such difficulties as can be illustrated on structures such as putamen, and is still time efficient (10 to 15min). Note that currently LOCUS segments only a subset of the 37 structures segmented by the method proposed in [7]. To conclude, the robustness and modularity of our LOCUS approach appear as interesting features when handling complex segmentation tasks. Efficiency is improved both in term of results quality and computing time.