Minimization of Region-Scalable Fitting Energy for Image Segmentation

Abstract

Intensity inhomogeneities often occur in real-world images and may cause considerable difficulties in image segmentation. In order to overcome the difficulties caused by intensity inhomogeneities, we propose a region-based active contour model that draws upon intensity information in local regions at a controllable scale. A data fitting energy is defined in terms of a contour and two fitting functions that locally approximate the image intensities on the two sides of the contour. This energy is then incorporated into a variational level set formulation with a level set regularization term, from which a curve evolution equation is derived for energy minimization. Due to a kernel function in the data fitting term, intensity information in local regions is extracted to guide the motion of the contour, which thereby enables our model to cope with intensity inhomogeneity. In addition, the regularity of the level set function is intrinsically preserved by the level set regularization term to ensure accurate computation and avoids expensive reinitialization of the evolving level set function. Experimental results for synthetic and real images show desirable performances of our method.

I. INTRODUCTION

Active contour models have been extensively applied to image segmentation [4], [7], [10], [19]. There are several desirable advantages of active contour models over classical image segmentation methods, such as edge detection, thresholding, and region grow. First, active contour models can achieve sub-pixel accuracy of object boundaries [3]. Second, active contour models can be easily formulated under a principled energy minimization framework, and allow incorporation of various prior knowledge, such as shape and intensity distribution, for robust image segmentation [5], [14]. Third, they can provide smooth and closed contours as segmentation results, which are necessary and can be readily used for further applications, such as shape analysis and recognition.

Existing active contour models can be categorized into two major classes: edge-based models [3], [10]–[12], [18], [19], [28], [30], and region-based models[4], [23], [25]–[27], [29]. Edge-based models use local edge information to attract the active contour toward the object boundaries. Region-based models aim to identify each region of interest by using a certain region descriptor to guide the motion of the active contour. However, popular region-based active contour models [4], [23], [25], [26] tend to rely on intensity homogeneity in each of the regions to be segmented. For example, the popular piecewise constant (PC) models are based on the assumption that image intensities are statistically homogeneous (roughly a constant) in each region.

In fact, intensity inhomogeneity often occurs in real images from different modalities. For medical images, intensity inhomogeneity is usually due to technical limitations or artifacts introduced by the object being imaged. In particular, the inhomogeneities in magnetic resonance (MR) images arise from the nonuniform magnetic fields produced by radio-frequency coils as well as from variations in object susceptibility. Segmentation of such MR images usually requires intensity inhomogeneity correction as a preprocessing step [9].

Intensity inhomogeneity can be addressed by more sophisticated models than PC models. Vese and Chan [29] and Tsai et al. [27] independently proposed two similar region-based models for more general images. Aiming at minimizing the Mumford–Shah functional [21], both models cast image segmentation as a problem of finding an optimal approximation of the original image by a piecewise smooth function. These models, widely known as piecewise smooth (PS) models, have exhibited certain capability of handling intensity inhomogeneity. However, the PS models are computationally expensive and suffer from other difficulties. Recently, Michailovich et al. [20] proposed an active contour model using the Bhattacharyya difference between the intensity distributions inside and outside a contour. Their model does not rely on the intensity homogeneity and, therefore, to some extent, overcome the limitation of PC models.

In this paper, we propose a new region-based active contour model in a variational level set formulation. We first define a region-scalable fitting (RSF) energy functional in terms of a contour and two fitting functions that locally approximate the image intensities on the two sides of the contour. The optimal fitting functions are shown to be the averages of local intensities on the two sides of the contour. The region-scalability of the RSF energy is due to the kernel function with a scale parameter, which allows the use of intensity information in regions at a controllable scale, from small neighborhoods to the entire domain. This energy is then incorporated into a variational level set formulation with a level set regularization term. In the resulting curve evolution that minimizes the associated energy functional, intensity information in local regions at a certain scale is used to compute the two fitting functions and, thus, guide the motion of the contour toward the object boundaries. As a result, the proposed model can be used to segment images with intensity inhomogeneity. Due to the level set regularization term in the proposed level set formulation, the regularity of the level set function is intrinsically preserved to ensure accurate computation for the level set evolution and final results, and avoid expensive reinitialization procedures.

Note that our model, originally termed as local binary fitting model, was first presented in [15], and published later in [16] as a full conference paper. Recently, local intensity averages were also introduced to active contour models in the context of geodesic active contour model [13] or piecewise smooth models [1], [2], [24]. These models exhibit certain capability of handling intensity inhomogeneity. In this paper, local intensity averages are derived as the minimizers of the proposed energy functional in a distinct variational formulation.

The remainder of this paper is organized as follows. In Section II, we first review some well known existing region-based models and their limitations. The proposed method is introduced in Section III. The implementation and results of our method are given in Section IV, followed by some discussions in Section V. This paper is summarized in Section VI.

II. REGION-BASED ACTIVE CONTOUR MODELS

Let Ω ⊂ ℜ² be the image domain, and I : Ω → ℜ be a given gray level image. In [21], Mumford and Shah formulated the image segmentation problem as follows: given an image I, find a contour C which segments the image into nonoverlapping regions. They proposed the following energy functional:

$${ℱ}^{\mathit{\text{MS}}}(u,C)={\displaystyle {\int }_{\mathrm{\Omega }}{(u-I)}^{2}d\mathbf{x}\text{+}\mu {\displaystyle {\int }_{\mathrm{\Omega }\backslash C}{|\nabla u|}^2}}d\mathbf{x}\text{+}\nu |C|$$ (1)

where |C| is the length of the contour C. The minimization of Mumford–Shah functional results in an optimal contour C that segments the given image I, and an image u that approximates the original image I and is smooth within each of the connected components in the image domain Ω separated by the contour C. In practice, it is difficult to minimize the functional (1) due to the unknown contour C of lower dimension and the nonconvexity of the functional.

Chan and Vese [4] proposed an active contour approach to the Mumford–Shah problem for a special case where the image u in the functional (1) is a piecewise constant function. For an image I(x,y) on the image domain Ω, they propose to minimize the following energy:

$$\begin{array}{ll}{ℱ}^{\mathit{\text{CV}}}(C,c_1,c_2)=\hfill & {\lambda }_1{\displaystyle {\int }_{\text{outside}(C)}|I(\mathbf{x})-c_1{|}^{2}d\mathbf{x}}\hfill \\ \hfill & +{\lambda }_2{\displaystyle {\int }_{\text{inside}(C)}|I(\mathbf{x})-c_2{|}^{2}d\mathbf{x}}\hfill \\ \hfill & +\nu |C|\hfill \end{array}$$ (2)

where outside(C) and inside(C) represent the regions outside and inside the contour C, respectively, and c₁ and c₂ are two constants that approximate the image intensity in outside(C) and inside(C). We call the first two terms in (2) the global fitting energy. This energy can be represented by a level set formulation, and then the energy minimization problem can be converted to solving a level set evolution equation [4].

The optimal constants c₁ and c₂ that minimize the above global fitting energy are the averages of the intensities in the entire regions outside(C) and inside(C), respectively. Such optimal constants c₁ and c₂ can be far away from the original image data, if the intensities within outside(C) or inside(C) are not homogeneous. They do not contain any local intensity information, which is crucial for segmentation of images with intensity inhomogeneity. As a consequence, the PC model [4] generally fails to segment images with intensity inhomogeneity. Likewise, more general piecewise constant models in a multiphase level set framework [23], [29] are not applicable for such images either.

The difficulties in segmenting images with intensity inhomogeneity can be seen from the following examples. The vessel image and a brain MR image in the first column in Fig. 1 are typical examples of images with intensity inhomogeneity. For such images, simple thresholding cannot segment them correctly. In fact, no matter what threshold value is selected, some part of the background/foreground is incorrectly identified as the foreground/background, as shown in the second column. The third column of Fig. 1 shows similar erroneous results obtained by applying Chan and Vese’s PC model [4]. These examples show the inability of the PC model and simple thresholding in segmenting images with intensity inhomogeneity.

Fig. 1.

Error of thresholding and Chan–Vese model for images with intensity inhomogeneity. Column 1: Original images. Column 2: Thresholding results. Column 3: Results of Chan–Vese’s PC model.

The PS models in [29] and [27] overcome the limitation of PC models in segmenting images with intensity inhomogeneity. In [29], Vese and Chan introduced an energy functional on a level set function ϕ and two smooth functions u⁺ and u⁻ that are defined on the regions outside and inside the zero level contour of a level set function ϕ, respectively. The energy functional has a data fitting term, which describes the approximation of the image by u⁺ and u⁻ in their corresponding subregions, and a smoothing term that forces u⁺ and u⁻ to be smooth.

The minimization of the energy functional in the PS model consists of the following three computational tasks. The first one is to solve the PDE of the main function ϕ by a sequence of iterations. Second, at every certain number of iterations for ϕ, the fitting functions u⁺ and u⁻ have to be updated by solving two elliptic PDEs. Third, the functions u⁺ and u⁻, which are defined on different regions, have to be extended to the entire image domain. In addition, periodic reinitialization is typically necessary to repair the level set function degraded by the evolution. Obviously, the involved computation in PS model is expensive, which limits its applications in practice.

III. REGION-SCALABLE FITTING MODEL

A. Region-Scalable Fitting Energy

In this section, we propose a region-based model using intensity information in local regions at a controllable scale. We first introduce a nonnegative kernel function K : ℜⁿ → [0, +∞) with the following properties:

1. K(−u) = K(u);

2. K(u) ≥ K(v), if |u| < |v|, and lim_|u|→∞ K(u) = 0;

3. ∫ K(x)dx = 1.

We call property 2) a localization property of the kernel K. The kernel function and its localization property play a key role in the proposed method.

Consider a given vector valued image I : Ω → ℜ^d, where Ω ⊂ ℜⁿ is the image domain, and d ≥ 1 is the dimension of the vector I(x). In particular, d = 1 for gray level images, while d = 3 for color images. Let C be a closed contour in the image domain Ω, which separates Ω into two regions: Ω₁ = outside(C) and Ω₂ = inside(C). For a given point x ∈ Ω, we define the following local intensity fitting energy:

$$\begin{array}{c}{\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}(C,f_1(\mathbf{x}),f_2(\mathbf{x}))\hfill \\ \hfill ={\displaystyle \sum _{i=1}^2{\lambda }_i{\displaystyle {\int }_{{\mathrm{\Omega }}_i}K(\mathbf{x}-\mathbf{y})}|I(\mathbf{y})-f_i(\mathbf{x}){|}^{2}d\mathbf{y}}\end{array}$$ (3)

where λ₁ and λ₂ are positive constants, and f₁(x) and f₂(x) are two values that approximate image intensities in Ω₁ and Ω₂, respectively. The intensities I(y) that are effectively involved in the above fitting energy are in a local region centered at the point x, whose size can be controlled by the kernel function K, as explained below. Therefore, we call the local intensity fitting energy in (3) a region-scalable fitting (RSF) energy of a contour C at a point x.

The choice of the kernel function K is flexible, as long as it satisfies the above three basic properties. In this paper, it is chosen as a Gaussian kernel

$$K_{\sigma }(\mathbf{u})=\frac{1}{{(2\pi )}^{n/2}{\sigma }^n}{e}^{-|\mathbf{u}{|}^2/2{\sigma }^2}$$ (4)

with a scale parameter σ > 0.

It is necessary to elaborate on the meaning of the fitting energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ defined by (3) in the following. First, ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ is a weighted mean square error of the approximation of the image intensities I(y) outside and inside the contour C by the fitting values f₁(x) and f₂(x), respectively, with K(x − y) as the weight assigned to each intensity I(y) at y. Second, due to the localization property of the kernel function, the contribution of the intensity I(y) to the fitting energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ decreases and approaches to zero as the point y goes away from the center point x. Therefore, the energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ is dominated by the intensities I(y) of the points y in a neighborhood of x. In particular, the Gaussian kernel K_σ(x − y) decreases drastically to zero as y goes away from x. Roughly speaking, the Gaussian kernel K_σ(x − y) is effectively zero when |x − y| > 3σ. Therefore, only the intensities in the neighborhood {y : |x − y| ≤ 3σ} are dominant in the energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$. In this sense, we say that the fitting energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ is localized around the point x.

The fitting energy in (3) is region-scalable in the following sense. The fitting values f₁(x) and f₂(x) approximate the image intensities in a region centered at the point x, whose size can be controlled by the scale parameters σ. The fitting energy (3) with a small σ only involves the intensities within a small neighborhood of the point x, while the fitting energy with a large σ involves the image intensities in a large region centered at x. Note that, in our preliminary work [16], the energy (3)was termed as a local fitting energy, as opposed to the global fitting energy (2) in Chan and Vese’s PC model [4]. However, it is more appropriate to call the energy (3) a region-scalable fitting energy, since the intensities for the fitting energy (3) are not restricted to a small local region. In fact, the intensities I(y) for the fitting energy (3) can be in a region of any size: from a small neighborhood to the entire image domain. This region-scalability is a unique and desirable feature of the proposed method.

Given a center point x, the fitting energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ can be minimized when the contour C is exactly on the object boundary and the fitting values f₁ and f₂ optimally approximate the local image intensities on the two sides of C. To obtain the entire object boundary, we must find a contour C that minimizes the energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ for all x in the image domain Ω. This can be achieved by minimizing the integral of ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ over all the center points x in the image domain Ω, namely, ${\displaystyle \int {\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}(C,f_1(\mathbf{x}),f_2(\mathbf{x}))d\mathbf{x}.}$. In addition, it is necessary to smooth the contour C by penalizing its length |C|, as in most of active contour models. Therefore, we define the following energy functional:

$$\mathit{Ɛ}(C,f_1(\mathbf{x}),f_2(\mathbf{x}))={\displaystyle \int {\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}(C,f_1(\mathbf{x}),f_2(\mathbf{x}))d\mathbf{x}+\nu |C|}.$$ (5)

This energy functional is defined for a contour C. To handle topological changes, we will convert it to a level set formulation in the next subsection.

B. Level Set Formulation

In level set methods [22], a contour C ⊂ Ω is represented by the zero level set of a Lipschitz function ϕ : Ω → ℜ, which is called a level set function. In this paper, we let the level set function ϕ take positive and negative values outside and inside the contour C, respectively. Let H be the Heaviside function, then the energy functional ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}(C,f_1(\mathbf{x}),f_2(\mathbf{x}))$ can be expressed as

$$\begin{array}{l}{\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}(\varphi ,f_1(\mathbf{x}),f_2(\mathbf{x}))\hfill \\ \hfill \hspace{1em}={\displaystyle \sum _{i=1}^2{\lambda }_i{\displaystyle \int K_{\sigma }(\mathbf{x}-\mathbf{y})|I(\mathbf{y})-f_i(\mathbf{x}){|}^2{M}_i(\varphi (\mathbf{y}))d\mathbf{y}}}\end{array}$$ (6)

where M₁(ϕ) = H(ϕ) and M₂(ϕ) = 1 − H(ϕ). Thus, the energy Ɛ in (5) can be written as (7), shown at the bottom of the next page, where the last term ∫ |∇H(ϕ(x))|dx computes the length of the zero level contour of ϕ. Note that this length term has been commonly used in variational level set methods for the regularization of the zero level contour [4], [29]. The length of the zero level contour can be equivalently expressed as the integral ∫ δ(ϕ)|∇ϕ|dxwith the Dirac delta function δ, which has often been used in variational level set methods [5], [18].

$$\begin{array}{ll}\mathit{Ɛ}(\varphi ,f_1,f_2)=\hfill & {\displaystyle \sum _{i=1}^2{\lambda }_i{\displaystyle \int \left({\displaystyle \int K_{\sigma }(\mathbf{x}-\mathbf{y})|I(\mathbf{y})-f_i(\mathbf{x}){|}^2{M}_i(\varphi (\mathbf{y}))d\mathbf{y}}\right)d\mathbf{x}}}\hfill \\ \hfill & +\nu {\displaystyle \int |\nabla H(\varphi (\mathbf{x}))|d\mathbf{x}}\hfill \end{array}$$ (7)

In practice, the Heaviside function H in the above energy functionals is approximated by a smooth function H_ϵ defined by

$$H_{ϵ}(x)=\frac{1}{2}\left[1+\frac{2}{\pi }\text{arctan}\left(\frac{x}{ϵ}\right)\right].$$ (8)

The derivative of H_ϵ is

$${\delta }_{ϵ}(x)=H_{ϵ}^{\prime }(x)=\frac{1}{\pi }\frac{ϵ}{{ϵ}^2+x^2}.$$ (9)

By replacing H in (7) with H_ϵ, the energy functional Ɛ in (7) is then approximated by (10), shown at the bottom of the page, where $M_1^{ϵ}(\varphi )=H_{ϵ}(\varphi )$ and $M_2^{ϵ}(\varphi )=1-H_{ϵ}(\varphi ).$

$$\begin{array}{ll}{\mathit{Ɛ}}_{ϵ}(\varphi ,f_1,f_2)=\hfill & {\displaystyle \sum _{i=1}^2{\lambda }_i{\displaystyle \int \left({\displaystyle \int K_{\sigma }(\mathbf{x}-\mathbf{y})|I(\mathbf{y})-f_i(\mathbf{x}){|}^2{M}_i^{ϵ}(\varphi (\mathbf{y}))d\mathbf{y}}\right)d\mathbf{x}}}\hfill \\ \hfill & +\nu {\displaystyle \int |\nabla H_{ϵ}(\varphi (\mathbf{x}))|d\mathbf{x}}\hfill \end{array}$$ (10)

To preserve the regularity of the level set function ϕ, which is necessary for accurate computation and stable level set evolution, we introduce a level set regularization term in our variational level set formulation. As proposed in [18], we define the level set regularization term as

$$𝒫(\varphi )={\displaystyle \int \frac{1}{2}{(|\nabla \varphi (\mathbf{x})|-1)}^{2}d\mathbf{x}}$$ (11)

which characterizes the deviation of the function ϕ from a signed distance function. Therefore, we propose to minimize the energy functional

$$ℱ(\varphi ,f_1,f_2)={\mathit{Ɛ}}_{ϵ}(\varphi ,f_1,f_2)+\mu 𝒫(\varphi )$$ (12)

where µ is a positive constant. To minimize this energy functional, its gradient flow is used as the level set evolution equation in the proposed method.

C. Energy Minimization

We use the standard gradient descent (or steepest descent) method to minimize the energy functional (12). For a fixed level set function ϕ, we minimize the functional ℱ(ϕ, f₁, f₂) in (12) with respect to the functions f₁(x) and f₂(x). By calculus of variations, it can be shown that the functions f₁(x) and f₂(x) that minimize ℱ(ϕ, f₁, f₂) satisfy the following Euler–Lagrange equations:

$${\displaystyle \int K_{\sigma }(\mathbf{x}-\mathbf{y})M_i^{ϵ}(\varphi (\mathbf{y}))(I(\mathbf{y})-f_i(\mathbf{x}))d\mathbf{y}=0,i=1,2.}$$ (13)

From (13), we obtain

$$f_i(\mathbf{x})=\frac{K_{\sigma }(\mathbf{x})*\left[M_i^{ϵ}(\varphi (\mathbf{x}))I(\mathbf{x})\right]}{K_{\sigma }(\mathbf{x})*M_i^{ϵ}(\varphi (\mathbf{x}))},i=1,2$$ (14)

which minimize the energy functional ℱ(ϕ, f₁, f₂) for a fixed ϕ. The functions f₁(x) and f₂(x) given by (14) are weighted averages of the intensities in a neighborhood of x, whose size is proportional to the scale parameter σ. Note that the denominators in (14) are always positive, due to the fact that $M_1^{ϵ}(\varphi )=H_{ϵ}(\varphi )>0$ and $M_2^{ϵ}(\varphi )=1-H_{ϵ}(\varphi )>0$ by the definition of H_ϵ in (8).

Keeping f₁ and f₂ fixed, we minimize the energy functional ℱ(ϕ, f₁, f₂) with respect to ϕ using the standard gradient descent method by solving the gradient flow equation as follows:

$$\begin{array}{c}\frac{\partial \varphi }{\partial t}=-{\delta }_{ϵ}(\varphi )({\lambda }_1{e}_1-{\lambda }_2{e}_2)+\nu {\delta }_{ϵ}(\varphi )\text{div}\left(\frac{\nabla \varphi }{|\nabla \varphi |}\right)\hfill \\ \hfill +\mu \left({\nabla }^2\varphi -\text{div}\left(\frac{\nabla \varphi }{|\nabla \varphi |}\right)\right)\end{array}$$ (15)

where δ_ϵ is the smoothed Dirac delta function given by (9), and e₁ and e₂ are the functions

$$e_i(\mathbf{x})={\displaystyle \int K_{\sigma }(\mathbf{y}-\mathbf{x})|I(\mathbf{x})-f_i(\mathbf{y}){|}^{2}d\mathbf{y},i=1,2}$$ (16)

where f₁ and f₂ are given by (14).

The above (15) is the level set evolution equation to be solved in the proposed method. The term −δ_ϵ(ϕ)(λ₁e₁ − λ₂e₂is derived from the data fitting energy, and, therefore, is referred to as the data fitting term. This term plays a key role in the proposed model, since it is responsible for driving the active contour toward object boundaries. The second term νδ_ϵ(ϕ)div (∇ϕ/|∇ϕ|) has a length shortening or smoothing effect on the zero level contour, which is necessary to maintain the regularity of the contour. This term is called the arc length term. The third term µ (∇²ϕ − div (∇ϕ/|∇ϕ|)) is called a level set regularization term, since it serves to maintain the regularity of the level set function.

D. Regularity of Fitting Functions and Level Set Function

In the proposed model, we introduce two fitting functions f₁ and f₂. They are different from the data fitting functions u⁺ and u⁻ in the PS model, due to the different natures of the data fitting energy terms in the two models. In the fitting energy ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ in (6), each integral is a weighted average squared distance from the fitting value f₁(x) (or f₂(x)) to all the image intensities I(y) the region outside(C) (or the region inside(C)), with the kernel K_σ(x − y) as the weight. Due to the localization property of the kernel K_σ, the values f₁(x) and f₂(x) that minimize ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$ are determined by all the intensities I(y) in a neighborhood of x. When the point x moves to an adjacent point x′ = x + Δx for a small displacement Δx, the majority of points in the neighborhood of x remain in that of x′. Therefore, the values of f_l(x′) and f₂(x′) which minimize ${\mathit{Ɛ}}_{\mathbf{x}\prime }^{\text{Fit}}$ are close tof_l(x) and f₂(x) that minimize ${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}$, due to the overlap of the neighborhoods of x and x′. This implies the smoothness of the functions f_l and f₂. The smoothness of f_l and f₂ is also confirmed by the Gaussian convolutions in (14).

The regularity of the level set function ϕ is also important for stable evolution and accurate computation in level set methods. In our method, the regularity of the level set function ϕ is inherently ensured by the level set regularization term in our level set formulation. Without the level set regularization term, the level set function ϕ typically grows to very large values on both sides of the zero level set, which yields an arbitrarily small value of δ_ϵ(ϕ) as a factor in the evolution (15). Thus, the motion of the contour becomes much slower and can even be stopped before it reaches the desired object boundaries. More importantly, the irregularity of ϕ make the involved computation inaccurate and, therefore, causes erroneous segmentation results. For more details about the significance and the mechanism of the level set regularization term, the readers are referred to [18].

Note that the irregularity of the level set function occurs in the PS model. The regularity of the level set function can be eventually violated by the level set evolution according to the PS model. Therefore, an extra numerical remedy, known as reinitialization, is usually needed to periodically stop the evolution and reshape the degraded level set function as a signed distance function for further computation. However, the practice of reinitialization may cause some undesirable side effects, such as preventing the detection of interior boundaries within an object, as pointed out in [4]. In our model, the regularity of the level set function is inherently maintained by the level set regularization term. This term is associated with the penalizing term 𝒫(ϕ) as a soft constraint on the regularity of the level set function ϕ, which regularizes the evolving level set function ϕ by penalizing its deviation from a signed distance function, instead of forcing ϕ to be a signed distance function.

IV. IMPLEMENTATION AND EXPERIMENTAL RESULTS

A. Implementation

All the partial derivatives ∂ϕ/∂x and ∂ϕ/∂y in (15) can be simply discretized as central finite differences. The temporal derivative is discretized as a forward difference. An iteration scheme is then obtained by discretizing the PDE (15). The level set function ϕ can be simply initialized as a binary step function which takes a negative constant value −c₀ inside a region R₀ and a positive constant value c₀ outside, for a constant c₀ > 0. We choose c₀ = 2 in the experiments shown in this paper. The advantage of using binary step function as the initial level set function is that new contours can emerge easily and the curve evolution is significantly faster than the evolution from an initial function as a signed distance map. In our implementation, the functions f₁ and f₂ are updated at every time step according to (14) before the update of the level set function ϕ.

To compute the convolutions in (14) more efficiently, the kernel K_σ can be truncated as a w × w mask, where w is the smallest odd number no less than 4σ. For example, given a scale parameter σ = 3.0, we have w = 13 and the size of the mask is 13 × 13. The difference between using the above specified size of the mask and a larger mask of K_σ, such as 41 × 41 for σ = 3.0, has been found to be neglectable in terms of segmentation results. Therefore, it is advisable to use the above given mask size to compute the convolutions efficiently.

The main computational cost in our method is for computing f₁ and f₂ in (14) and λ₁e₁ − λ₂e₂ in the level set evolution (15). There are four convolutions in the numerators and denominators in (14). However, we notice that f₂ in (14) can be written as

$$f_2=\frac{K_{\sigma }*I-K_{\sigma }*[H_{ϵ}(\varphi )I]}{K_{\sigma }*1-K_{\sigma }*H_{ϵ}(\varphi )}$$

where 1 is the constant function with value 1. In the above expression of f₂, the second terms in the numerator and the denominator are the same as the numerator and the denominator in f₁, while the terms K_σ * I in the numerator and K_σ * 1 in the denominator do not depend on the evolving level set function ϕ. Therefore, the two convolutions K_σ * I and K_σ * 1 can be computed only once before the iterations. During the iterations, only the two convolutions K_σ * [H_ϵ (ϕ)I] and K_σ * H_ϵ(ϕ) are computed for the evolving ϕ. The term λ₁e₁ − λ₂e₂ in (15) can be expressed as a combination of three convolutions, with one independent of ϕ, which can be computed only once before the iterations. Therefore, there are totally four convolutions to be computed at each iterations in the above implementation.

B. Results

The proposed method has been tested with synthetic and real images from different modalities. Unless otherwise specified, we use the following parameters in this paper: σ = 3.0, λ₁ = λ₂ = 1.0, time step Δt = 0.1, µ = 1, and ν = 0.001 × 255 × 255. We use relatively small scale parameter σ for the experiments in this section. In general, our method with a smaller scale σ can produce more accurate location of the object boundaries, while it is more independent of the location of the initial contour when a larger σ is used. The influence of different scale parameters on the behavior of our method will be discussed in Section V.

We first show the results for three synthetic images in Fig. 2. These images have the same objects but different distribution of intensities. The initial and the final contours are plotted on the images in the first column and the second column, respectively. To clearly see the profile of the functions f₁ and f₂ associated with the final level set function ϕ, we plot a 1-D cross section at the middle row of f₁ and f₂ and the input image I in the third column. The intensity of the image in the first row is piecewise constant. There are three distinct intensities in the three regions. The result in Row 1 shows that our method is able to segment images with multiple distinct means of image intensities.

Fig. 2.

Results of our method for three synthetic images. Column 1: Initial contour and original image. Column 2: Final contour. Column 3: 1-D cross section of the fitting functions f₁ (dashed lines), f₂ (dotted lines), and the original image I (solid lines), for the final level set function ϕ. Column 4: 1-D cross section of the fitting image $f={\displaystyle {\sum }_{i=1}^2{M}_i^{ϵ}(\varphi )f_i}$ (dashed lines) for the final level set function ϕ and the original image I (solid lines).

The second and third rows in Fig. 2 show the results for two images corrupted by intensity inhomogeneity. The image in third row was generated by adding Gaussian noise to the clean image in the second row. The standard deviation of the noise is 4.0, which is quite high relative to the image contrast. As explained in Section III-D, the functions f₁ and f₂ given by (14) are smooth enough, even in the presence of noise. This is con-firmed by the smoothness of f₁ and f₂ plotted in the second and third rows. In addition, we can see that the fitting functions f₁ and f₂ are only affected marginally by the added noise, even though the noise is quite high. As a result, the segmentation results for the clean image and the noise contaminated version are very close. This demonstrates the robustness of our method to the noise.

Fig. 3 shows the results for two synthetic images, two X-ray images of vessels, and a real image of a T-shaped object. All of them are typical images with intensity inhomogeneity. In particular, the vessel image in the third row has been used in Fig. 1, by which we have shown that the PC model fails to segment the images due to the intensity inhomogeneity. The image in the second row was created with intensity inhomogeneity and contaminated with high level noise. Our method successfully extracts the object boundaries for these two images, as shown in Fig. 3. The third and the fourth rows in Fig. 3 show the results of our method for two real images of blood vessels. In these two images, part of the vessel boundaries are quite weak, which renders it a nontrivial task to segment the vessels in the images. The image in the bottom row is a real image with intensity inhomogeneity due to nonuniform illumination. Satisfactory segmentation results have been obtained for these challenging images, as shown in Fig. 3.

Fig. 3.

Results of our method for synthetic images and real images. The curve evolution process from the initial contour (in the first column) to the final contour (in the fourth column) is shown in every row for the corresponding image.

Intensity inhomogeneity also often occurs in MR images, such as the one shown in the upper row in Fig. 4. Some intensities of the white matter in the upper part are even lower than those of the gray matter in the lower part. Nevertheless, our method successfully segments the white matter in these two images. For this image, we use the parameters σ = 3.0, λ₁ = 1.0, λ₂ = 2.0, time step Δt = 0.1, µ = 1, and ν = 0.003 × 255 × 255. Note that we choose a larger value λ₂ than λ₁ for this image to avoid the emergence of new contours far away from the initial contour, such as the skull boundaries, as explained in Section IV-C. In this experiment, we choose a larger ν than in other experiments to further penalize the length of the contour, which also discourages the expansion of the contour to some extent. The lower row in Fig. 4 shows the result for a CT image of a liver with a tumor (the dark area in the middle). For this image, we used the scale parameter σ = 4.0. This image is rather noisy and part of the tumor boundary is weak. Our method successfully extracts the object boundaries in this image.

Fig. 4.

Results of our method for MR and CT images. The curve evolution process from the initial contour (in the first column) to the final contour (in the fourth column) is shown in every row for the corresponding image.

Fig. 5 shows an application of our model to a color image of potatoes. With this experiment, we also demonstrate the smoothness of the fitting images f₁ and f₂ by showing the images of them. The first row shows the active contours on the original images from its initial to converged state. The second and third rows show the corresponding fitting images f₁ and f₂, computed by (14). It is clearly seen that these images are smooth, which experimentally verifies the regularity of the fitting functions f₁ and f₂ as mentioned in the previous section. It is interesting to note that their combination $f={\displaystyle {\sum }_{i=1}^2{M}_i^{ϵ}(\varphi )f_i}$ enhances the features in the images, such as object boundaries, when it converges to its final result (the last image in the fourth row).

Fig. 5.

Results of our method for a color image of potatoes. Row 1 shows the curve evolution process from the initial contour to the final contour. Row 2, Row 3, and Row 4 show f₁, f₂, and the fitting image $f={\displaystyle {\sum }_{i=1}^2{M}_i^{ϵ}(\varphi )f_i}$, respectively, at different time steps.

C. Remarks on the Behavior of Curve Evolution

Note that new contours can emerge during the curve evolution in our method. This has been observed in the experiments for the images in the second and third rows of Fig. 3 and the first row in Fig. 4. The emergence of new contours speeds up the curve evolution toward final results, and enables the detection of interior boundaries, such as the two interior boundaries within the white matter in Fig. 4.

The emergence of new contours is possible because the data fitting term −δ_ϵ(ϕ)(λ₁e₁ − λ₂e₂) in (15) has influence on the change of ϕ in the entire image domain, as the factor δ_ϵ(ϕ) is nonzero by the definition of δ_ϵ in (9). For points x far away from the zero level contour but near an object boundary, the values of (λ₁e₁ − λ₂e₂) may still be large. Although the factor δ_ϵ(ϕ) takes small values far away from the zero level set, the data fitting term −δ_ϵ(ϕ)(λ₁e₁ − λ₂e₂) is not zero and cannot be ignored, which can eventually change the value of ϕ. As a result of cumulative change of ϕ, new zero level contours may emerge at strong object boundaries. Especially, when the level set function ϕ is initialized to be small values, such as the binary step function used in our implementation (see Section IV-A), new contours can emerge more easily and quickly. In our experiments, the initial level set function ϕ₀ is defined as a binary step function that takes values of c₀ and −c₀ with a small value of c₀ = 2. As a result, the factor δ_ϵ(ϕ) takes relatively larger values in the beginning of the level set evolution, which leads to faster emergence of new contours at strong edges, even at locations far away from current zero level set. Of additional note, a larger ϵ gives a broader profile of δ_ϵ, which makes it easier for new contours to emerge. However, broader profile of δ_ϵ decreases accuracy in the final contour location.

The coefficients λ₁ and λ₂ are the weights of the two integrals in (3), or equivalently (6), over the regions outside(C) and inside(C), respectively. In most cases, we set the coefficients as the same constant λ₁ = λ₂, which leads to a fair competition between the regions inside and outside the zero level contour during the evolution. However, when different weights λ₁ and λ₂ are used, the amounts of penalty imposed on the integrals over outside(C) and inside(C) are different. For example, when λ₂ is larger than λ₁, larger penalty is imposed on the integral over the region inside(C) in the data fitting energy, which implies a larger penalty on the area of inside(C). Due to this larger penalty on the area of inside(C), the emergence of new contour outside the initial contour, which would increase the area of inside(C), is to some extent prevented. Especially, at locations far away from the current zero level set, we have e₁ ≈ e₂, and, thus, the signs of −δ_ϵ(ϕ)(λ₁e₁ − λ₂e₂) tend to be positive there, because λ₂ > λ₁. As a result, ϕ does not decrease at these locations, where ϕ is initially positive, and, thus, no new contours are created there.

D. Comparison With Piecewise Smooth Model and Mean Shift Algorithm

It is obvious that PC models by nature cannot address intensity inhomogeneity. Previous experimental results in Section II for Chan and Vese’s PC model shown in Fig. 1 and those of our method shown in Fig. 3 and Fig. 4 have demonstrated the advantage of our method over the PC model. We now compare our model with the PS model in [29] and the well-known mean shift algorithm [8], both of which possess certain capability of handling intensity inhomogeneity.

We first show the results of the comparison with the PS model. As described in Section II, the PS model consists of quite a few computationally expensive steps. By comparing the computational procedures in the PS model and our model, it is clear that our method is much simpler and more efficient than the PS model. This is demonstrated by the following experiments of comparing the computation time in both methods for five images.

Fig. 6 shows the results of our model and PS model using the same images and the same initial contours in the upper row and the lower row, respectively. The CPU times for these images are listed in Table I, which were recorded from our experiments with Matlab code run on a Dell Dimension 4600 PC, with Pentium 4 processor, 2.80 GHz, 1 GB RAM, with Matlab 7.4 on Windows XP. The sizes of these images are also shown in this table. In the experiments with the images in Fig. 6, our model is about 15 to 60 times faster than the PS model. This demonstrates the significant advantage of our model in terms of computational efficiency.

Fig. 6.

Comparison of our model with PS model. The initial contours and the final contours are plotted as the dashed green contours and solid red contours, respectively. Upper row: The results of our model. Lower row: The results of PS model.

TABLE I.

CPU Time (in Second) for Our Model and Vese-Chan’s PS Model for the Images in Fig. 6 in the Same Order. The Sizes Of Images 1–5 are 88 × 85, 75 × 79, 131 × 103, 110 × 110, and 78 × 119 Pixels, Respectively

	Image 1	Image 2	Image 3	Image 4	Image 5
Our model	0.84	1.23	8.58	5.78	5.92
PS model	30.76	75.51	266.25	142.24	86.07

Our model is also superior in terms of accuracy. This is obvious for the MR image in the fifth column in Fig. 6. Our method extracts the white matter boundary accurately, while the contour of the PS model skips some parts of the white matter boundary and is finally attracted to the outer boundary of the gray matter. The right most column in Fig. 6 is an enlarged view of the lower left portion of the figures in the fifth column, which clearly shows the advantage of our model over the PS model.

For comparison with the mean shift algorithm, we used the software EDISON downloaded from http://www.caip.rutgers.edu/riul/research/code/EDISON, which is based on a fast implementation of the mean shift algorithm using a speed-up scheme described in [6]. Fig. 7 shows the results of mean shift algorithm for the same images in Fig. 6 for the comparison with the PS model. The result of mean shift algorithm for the first image is similar to that of our method, showing certain ability of the mean shift algorithm in handling intensity inhomogeneity. However, for the first vessel image, a small portion of the vessel is missing. For the rest three images, the errors of mean shift algorithm are even more obvious. We notice that the segmentation result of mean shift algorithm is somewhat sensitive to the choice of two major parameters: spatial bandwidth h_s and range bandwidth h_r. We have tweaked these two parameters and other minor parameters for the best segmentation results for these five images.

Fig. 7.

Results of mean shift algorithm for five images with spatial bandwidth h_s and the range bandwidth h_r represented as a pair (h_s, h_r) = (3,8), (7,8), (4,3), (4,4), and (6,6) in the order from left to right.

V. FURTHER DISCUSSION

A. Discussion on Region-Scalability

It is necessary to examine the influence of the scale parameter σ on the segmentation results of the proposed method, although the same scale parameter σ = 3.0 has been used for most of the images previously in this paper. For this purpose, we apply our method using three different scale parameters σ = 3.0, 6.0, and 10.0 for the same MR image used in Fig. 4. The results for these three scale parameters are shown in Fig. 8. These results are grossly similar, while differences in fine details of the resulting white matter boundaries can be observed. The most accurate segmentation result is obtained for the smallest scale parameter σ = 3.0.

Fig. 8.

Results of our model with scale parameters σ =3.0, 6.0, and 10.0 (from the left to the right).

Our method with a larger scale parameter σ is more robust to the location of the initial contour. For sufficiently large scale σ, our method can be as insensitive to the initialization as the PC model [4]. In fact, the PC model [4] can be considered as an extreme case of the proposed RSF model for σ → ∞. This can be seen from the limit of the fitting function f₁ and f₂ in (14) as σ → ∞. From (14) and the definition of K_σ, it can be shown that

$$\underset{\sigma \to \infty }{\lim }{f}_i(\mathbf{x})=\frac{{\displaystyle \int M_i^{ϵ}(\varphi (\mathbf{y}))I(\mathbf{y})d\mathbf{y}}}{{\displaystyle \int M_i^{ϵ}(\varphi (\mathbf{y}))d\mathbf{y}}},i=1,2.$$ (17)

The right-hand sides in (17) are the means of the intensities in the regions {ϕ > 0} and {ϕ < 0}, respectively, which are the same constants that fit the intensities in these two regions in the PC model [4].

To examine the performances of our method using a relatively small and a sufficiently large scale parameters σ, we use σ = 3.0 and σ = 30.0 for an MR image of bladder with intensity inhomogeneity. The initial contour shown in Fig. 9(a) is used in both cases. The corresponding results for σ = 3.0 and σ = 30.0 are shown in Fig. 9(b) and (c), respectively. It can be seen that the result for σ = 3.0 is desirable, with the bladder and the surrounding minor structures segmented very well. The result of our method with σ = 30.0, however, is similar to the result of the PC model, shown in Fig. 9(d), both missing a significant part of the bladder.

Fig. 9.

Experiments for an MR image of bladder. (a) Initial contour and the original image. (b) Result of our method for σ = 3:0. (c) Result of our method for σ = 30:0. (d) Result of the PC model. (e)–(h) Results (white solid contours) of our model for four different initial contours (the black dashed rectangles or circles), with the same scale parameter σ = 10:0. Image courtesy of University of Graz.

An important advantage of the proposed model is its region-scalability: it allows the choice of the scale parameter σ to exploit intensity information in regions of different scales, from small neighborhoods to the entire image domain. For many real-world images, the intensity inhomogeneity is not so severe. In this situations, a reasonably large σ can be used in our model, so that it is more independent of initialization, while achieving satisfactory accuracy in segmentation. For example, we set σ = 10.0 in our model and tested it with four different initial contours (shown as black dashed rectangles or circles) for the same image in the lower row of Fig. 9. For these four diverse initializations, almost the same segmentation results (shown as white solid contours) were obtained, as shown in the lower row of Fig. 9. The boundary of the object of interest (the bladder) is extracted very well for these different initializations. We remark that using a larger scale parameter σ increases the computational cost of the convolutions in each iteration than using a smaller one. However, the number of total iterations can be reduced by using a larger σ. Therefore, the total computational costs are comparable for both cases.

B. Some Extensions

Note that the level set formulation proposed in this paper is a two-phase model, which cannot segment regions with multiple junctions. This limitation can be easily overcome by extending the formulation in this paper to a multiphase level set formulation. The extension to multiphase level set formulation is simply established by generalizing the local intensity fitting energy in (3) as

$${\mathit{Ɛ}}_{\mathbf{x}}^{\text{Fit}}={\displaystyle \sum _{i=1}^N{\lambda }_i{\displaystyle {\int }_{{\mathrm{\Omega }}_i}K(\mathbf{x}-\mathbf{y})|I(\mathbf{y})-f_i(\mathbf{x}){|}^{2}d\mathbf{y}}}$$

where f₁, …, f_N are N fitting functions associated with N disjoint regions Ω₁, …, Ω_N in Ω. These N regions can be represented with multiple level set functions as in [29] to define an energy functional of the level set functions. During the preparation of this paper, we have reported our results of the multiphase formulation for 3-D segmentation of white matter, gray matter, and cerebral-spinal fluid in brain MR images in [17].

The experiments in Section V-A show the advantages of using small and large scale parameters in the proposed RSF model. In this paper, we only use one scale parameter σ in the RSF energy for each given image. However, the proposed RSF model provides basic elements that can be used in more sophisticated ways to further improve both accuracy and robustness, such as combining RSF energy terms with different scale parameters. The computational efficiency can also be significantly improved by narrow band implementation of the proposed model. Due to the space limit, the detail of the above extensions are not included in this paper.

VI. CONCLUSION

We have presented a new region-based active contour model that draws upon intensity information in local regions at a controllable scale. The proposed model is able to segment images with intensity inhomogeneity, and has desirable performance for images with weak object boundaries. With the level set regularization term in the proposed level set formulation, the regularity of the level set function is intrinsically preserved to ensure accurate computation and avoid expensive reinitialization procedures. Experimental results have demonstrated the advantages of our method over several well-known methods for image segmentation.

Biographies

Chunming Li received the Ph.D. degree in electrical engineering from the University of Connecticut, Storrs, in 2005.

Between 2001 and 2003, he was a consultant for Robotics Technology Group, ABB Corporate Research, where he worked on image processing and camera calibration algorithms. From 2003 to 2005, he did his Ph.D. research on medical image processing and analysis, which was supported by Pfizer Global Research and Development for research on the measurement of a biomarker for atherosclerosis. He is currently a research fellow at Vanderbilt University Institute of Imaging Science. His research interests include the development and application of algorithms in image processing, medical imaging, and computer vision.

Chiu-Yen Kao received the Ph.D. degree in mathematics from the University of California, Los Angeles, in 2004.

She is currently an Assistant Professor at The Ohio State University, Columbus, and a long term visitor at the Mathematical Biosciences Institute. Her research expertise lies in level set methods, Hamilton–Jacobi equations, computational anatomy, and inverse problems. For biological applications, she is currently working on the structural study of human brains, automatic numerical extraction of ciliary muscle, and general image processing techniques based on partial differential equations.

John C. Gore received the Ph.D. degree in physics from the University of London, London, U.K., in 1976.

He is the Director of the Institute of Imaging Science and Chancellor’s University Professor of Radiology and Radiological Sciences, Biomedical Engineering, Physics, and Molecular Physiology and Biophysics at Vanderbilt University, Nashville, TN.

Dr. Gore is an elected fellow of the American Institute of Medical and Biological Engineering, the International Society for Magnetic Resonance in Medicine (ISMRM), and the Institute of Physics (U.K.). In 2004, he was awarded the Gold Medal from the ISMRM for his contributions to the field of magnetic resonance imaging. He is Editor-in-Chief of the journal Magnetic Resonance Imaging. His research interests include the development and application of imaging methods for understanding tissue physiology and structure, molecular imaging, and functional brain imaging.

Zhaohua Ding received the Ph.D. degree in biomedical engineering from The Ohio State University, Columbus, in 1999.

He was a postdoctoral research associate at Yale University, New Haven, CT, from 1999–2002. Currently, he is an Assistant Professor at the Institute of Imaging Science, Vanderbilt University, Nashville, TN. His major research interest is medical image analysis and biomedical applications.