TISSUE LEVEL SEGMENTATION AND TRACKING OF BIOLOGICAL STRUCTURES IN MICROSCOPIC IMAGES BASED ON DENSITY MAPS

Abstract

During embryogenesis, cells coordinate to form geometric arrangements. These arrangements are initially noticed as stereotypical clumps of cells that further divide to form a rigorous structure with a high density of cells. In this work, we explore density-based segmentation and tracking of cellular structures as observed in microscopy images. Using a new modified form of the Mumford-Shah energy functional, we derived a variational level-set for density-based segmentation. The novelty of the work lies in evolving an initialized contour that represents a salient structure on density maps to automatically generate novel cell structures upon convergence. We validate our methods and show results on confocal ear images of the zebrafish embryo.

1. INTRODUCTION

Biological microstructures, as observed at a microscopic scale, are composed of a finite number of components such as red blood cells, nuclei, cytoplasm and vacuoles etc. These components (or phases) are spatially distributed and relatively packed to create a signature texture for a novel tissue or organ. While textural pattern differences can get quite complicated, the earliest notion of such geometric packing is observed during embryogenesis when precursor cells form tissues of different cellular densities. These cells divide and migrate to further define the geometry of the organ.

Developmental biologists are interested in understanding how these rearrangements lead to and control the formation of structures. Advances in confocal and multi-photon fluorescence microscopy have paved the way for spatiotemporal imaging of embryonic development. Image analysis can now automatically segment out cellular structures in a high throughput manner. Hence, in this work, we focus on the segmentation and tracking of multi-cellular structures such as tissues and organs.

One of the challenges in defining a salient structure in embryonic images is the lack of boundaries or edges. Please refer to Figure 1(a). Most segmentation techniques in image processing rely on the presence of strong image gradients and boundary edges. We use the popular region-based level-set methods for image segmentation. Region-based level-sets as compared to edge-based do not require strong image gradients but instead rely on region statistics to perform a segmentation. Chan and Vese [1] introduced a piecewise-constant model of image intensities for image segmentation. This was based on minimizing an energy functional developed by Mumford and Shah [2] for segmentation.

Fig. 1.

(a) A optical section of the zebrafish embryo ear. Note the variable density of cells. (b) Color-coded density map of cell regions. (c) Two-class segmentation of the image using a level-set formulation.

In level-set literature, numerous texture segmentation algorithms have been proposed that model the image intensities in terms of Gabor functions [3], wavelets [4] and frequency, oscillating functions [5] or in terms of parametric histogram distributions [6]. Our approach fundamentally measures density differences of components, in this case cells, and can be generalized to spatial distributions and packing of microstructural phases in other fields such as microcrystals in metal alloys.

Note that the same 3D algorithm applied independently at each time-point creates discontinuities in structure especially at object edges. However, level-set approaches naturally can track changes and follow the development of structures. Dufour et al. [7] used the level-set formulation in [1] to track individual cells with confocal time-lapse microscopy. Several other approaches exist for tracking cells [8]. Our approach, instead requires tracking a region of cells with a stereotype density rather than individual ones. It is more related in purpose to the work by Mosaliganti et al. [9] on segmentation and tracking of clonal colonies in cell cultures. The authors used a density-based manifold with 2-point correlation functions (pcfs) to evolve a contour with fast marching methods to track regions. The fast marching methods solve the Eikonal heat equation and are less rigorous than our current variational formulation that naturally define a segmentation.

In our approach, we formulate an energy functional similar to the popular Mumford-Shah energy functional but changed to accommodate a density function instead of image intensities. We estimate cell densities by using point correlation functions computed on classified cell images (Figure 1(b)). By representing the region-of-interest as being enclosed within the zero-contour of an implicit function, the energy is minimized in a variational level-set formulation. Upon convergence, the enclosed region within the zero level-set defines our multi-cellular structure (Figure 1(c)). The resulting level-set formulation permits us to track development of tissues and organs by initializing with the implicit function generated from the previous time-point. Hence, we obtain robust and smooth segmentations of dense cellular structure.

The rest of the paper is organized as follows. Section 2 formulates the fitting energy in the context of active contour models. We derive the complete level-set formulation and describe our implementation in Section 3. Section 4 shows our results segmenting the zebrafish ear from confocal images. Finally, in Section 5, we provide a summary and describe our plans for the future.

2. MATHEMATICAL FORMULATION

The point correlation functions are adapted from material science literature to measure microstructural properties such as densities and packing [10]. The 2-pcfs have been initially introduced in medical imaging for histology image segmentation [11,12]. Preliminary studies have shown that these functions are more efficient in estimating densities arising from geometric arrangements than kernel-based methods which are more suited for groups of clusters. However, these studies are beyond the scope of this paper.

2.1. Cell Density - 2-Point Correlation Functions

Correlation functions describe the spatial distribution of material components in a multi-phase system. A phase represents a multi-component sample where the packing distribution of the different components is uniform throughout. Assume the presence of two phases in the image, say, phase 0 (black) and 1 (white) as shown in Figure 2(a). A 2-point correlation function (2-pcf) is the probability of a straight line segment of length k randomly placed in the image such that one end is in phase i₁ ∈ {0, 1} and the other end is in phase i₂ ∈ {0, 1}. For a 2-phase image, there are four possible 2-pcfs namely $P_{00}^k,P_{01}^k,P_{10}^k$ and $P_{11}^k$ and:

$$\begin{array}{ll}{P}_{00}^k+P_{01}^k+P_{10}^k+P_{11}^k=1\hfill & \hfill P_{01}^k=P_{10}^k\\ P_{00}^k+P_{00}^k=f_0\hfill & \hfill P_{10}^k+P_{11}^k=f_1\end{array}$$

where parameters f₀ and f₁ represent the volume fractions of the individual phases. In Figure 1(b), the function P₁₁ is computed for a value of k = 50 pixel units. The value of k is chosen to be equal to the linear separation distance between neighboring cells. We observe that the dense cell regions record a high measurement and vice-versa. It is now our goal to separate the image into two classes based on density differences.

Fig. 2.

The figure shows four different microstructure ensembles composed of 2-phase components, namely, phase 0 (black) and phase 1 (white). The images are geometrically sampled by placing polygons (red and blue) and noting the phases at their vertices. A plot of the 2-pcf $P_{01}^k$ for varying k reveals different characteristic signatures. Note that (a) and (c) textures have the same signature that overlaps.

2.2. Density-based active contours

The basic idea of active contour models relies on detecting salient objects by evolving a curve or a surface subject to image-based constraints. Let Ω ⊂ ℝ^d be the image domain and u₀ : Ω → ℝ be a given image function. In [2], Mumford and Shah made a seminal contribution to the literature on image segmentation problem by proposing an energy functional to be minimized by a suitable contour C in Ω.

$$F^{MS}(u,C)={\int }_{\mathrm{\Omega }}{(u-u_0)}^{2}dx+{\int }_{\mathrm{\Omega }\backslash C}{\mid \nabla u\mid }^{2}dx+\nu \mid C\mid$$

where |C| is the length of the contour C and ν is a nonnegative constant weight. The minimization of the above functional results in an optimal contour that segments the image and provides a piecewise smooth image u that approximates the original image u₀. Chan and Vese [1] proposed an active contour model for segmenting images containing objects that have poor boundaries. They proposed an energy that is an piece-wise constant approximation to the Mumford and Shah functional:

$$\begin{array}{l}{F}^{CV}(C,c_1,c_2)={\lambda }_1{\int }_{C_{in}}{\mid I(x)-c_1\mid }^{2}dx\\ +{\lambda }_2{\int }_{C_{\mathit{out}}}{\mid I(x)-c_2\mid }^{2}dx+\mu .\mathit{Area}(C)+\nu .\mathit{Volume}(C)\end{array}$$

where in(C) and out(C) represent the region inside and outside the contour C, respectively, and c₁ and c₂ are two scalar constants that approximate the image intensities. The first two terms are often referred as global binary fitting energy terms that seek to separate an image into two regions of constant image intensities. As an extension to the above model, we assume instead that the image consists of foreground and background with distinct and constant cellular densities c₁ and c₂. We now define our energy term for minimization in terms of the 2 − pc f.

$$\begin{array}{l}{F}^{\mathit{density}}(C,c_1,c_2)={\lambda }_1{\int }_{C_{in}}{\mid P_{01}(x)-c_1\mid }^{2}dx\\ +{\lambda }_2{\int }_{C_{\mathit{out}}}{\mid P_{01}(x)-c_2\mid }^{2}dx+\mu .\mathit{Area}(C)+\nu .\mathit{Volume}(C)\end{array}$$

3. IMPLEMENTATION

In level-set methods, a contour C ⊂ Ω is represented by the zero level-set of a Lipschitz function φ : Ω → ℝ. With the level-set representation, the energy functional F^density(u; c₁; c₂) can be rewritten as

$$\begin{array}{l}{F}^{\mathit{density}}(\phi ,c_0,c_1)={\lambda }_1{\int }_{C_{in}}{\mid P_{01}(x)-c_1\mid }^{2}H(\phi )dx\\ +{\lambda }_2{\int }_{C_{\mathit{out}}}{\mid P_{01}(x)-c_2\mid }^2(1-H(\phi ))dx\\ +\mu {\int }_{\mathrm{\Omega }}\delta (\phi )\mid \nabla \phi \mid dx+\nu {\int }_{\mathrm{\Omega }}H(\phi )dx\end{array}$$ (1)

where H is the Heaviside function. In order to ensure the stable evolution of the level-set function φ, we add the distance regularizing term to penalize its deviation from a signed distance function. The deviation is characterized by the following integral $D(\phi )={\int }_{\mathrm{\Omega }}\frac{1}{2}{(\mid \nabla \phi (x)\mid -1)}^2$ [13]. Therefore, the entire energy functional is defined as E(φ, c₁; c₂) = F^density(φ, c₁; c₂) + β D(φ ).

Inr practice, the Heaviside function and the Dirac function in Equation 1 are approximated by smooth function H_ε and δ_ε defined as

$$H_{\epsilon }(x)=\frac{1}{2}(1+\frac{2}{\pi }\mathit{arctan}(\frac{x}{\epsilon }));{\delta }_{\epsilon }(x)=\frac{1}{\pi }\frac{\epsilon }{{\epsilon }^2+x^2}$$

We then minimize the energy functional to obtain tangible segmentations. For a fixed level-set function φ, we minimize the functional F^density with respect to the remaining parameters. By calculus of variations, it can be shown that the following Euler-Lagrange equations need to be satisfied.

$$c_1=\frac{{\int }_{\mathrm{\Omega }}{P}_{01}(x)H(\phi (x))dx}{{\int }_{\mathrm{\Omega }}H(\phi (x))dx}{c}_2=\frac{{\int }_{\mathrm{\Omega }}{P}_{01}(x)\left(1-H(\phi (x))\right)dx}{{\int }_{\mathrm{\Omega }}\left(1-H(\phi (x))\right)dx}$$

By minimizing the energy functional with respect to φ and using H_ε and δ_ε , we derive the gradient descent flow:

$$\frac{\partial \phi }{\partial t}={\delta }_e(\phi )\left[\mu \mathit{div}(\frac{\nabla \phi }{\mid \nabla \phi \mid })-\nu -{\lambda }_1{(P_{01}-c_1)}^2+{\lambda }_2{(P_{01}-c_2)}^2\right]=0\text{in}(0,\infty )\times \mathrm{\Omega }$$ (2)

The above Equation 2 is the proposed implicit active contour model in this paper. For tracking a multi-cellular structure across time, we initialize the level-set function at a given time-point φⁿ⁺¹ with the level-set function determined at the previous time-point φⁿ. This ensure a continuous smooth evolution of the structure.

4. RESULTS AND DISCUSSION

Our data consists of 3D +t images of zebrafish development. A typical time-point in the dataset is obtained every 2 minutes and has pixel dimensions of 1024 × 1024 × 80. Each voxel has physical dimensions of 0.2 × 0.2× 1μm. We are particularly interested in the ear since its formation occurs quite early in development. The ear consists of a central vesicle (hollow sphere) flanked by other large structures such as the hind-brain, eye and head mesenchyma, all of which are objects of interest. In order to validate our algorithm and measure the accuracy of our implementation, we generated synthetic 3D datasets that mimic natural images. Please refer to Figure 3(a). In a typical dataset, we create a region-of-interest or the foreground region similar to a thick shell. The shell is filled with cells that have an average separation of a pixel units from neighbors. The background (region apart from the shell) is filled with cells with at least b pixel units of separation on average. Note that a is different from b. The shell region is representative of the otic vesicle and cells are densely packed within. We run our algorithm and compare the segmented region with the original shells. The segmentation quality is assessed using the Dice metric that penalizes both true negative and false positive regions showing in Figure 3(b). We list the results for 5 different sets of a = 10 and b = {15, 20, 25, 30, 35}. Our algorithm records a robust performance with percentage accuracies {96.76, 96.82, 97.71, 96.89, 96.31} respectively. Note that we uniformly set λ₁ = λ₂ = 1 and μ = ν = 0 for all the experiments although these parameters can be optimized for individual datasets. The k in the 2-pcfs is set to k = a = 10 pixel units. We note that at object boundaries, the calculation of densities is ambiguous and hence segmentation is not ac curate. We also execute our algorithm of the 3D+t images the zebrafish ear dataset. Figure 4 shows the segmentations obtained at t = 0 and 100 acquisitions.

Fig. 3.

Left: Two regions of different densities. Right: Automated segmentation (green) overlapped with true segmentation(maroon).

Fig. 4.

Tracking zebrafish ear otic vesicle development at t = 0 (left) and t = 139 datasets (right).

5. FUTURE WORK

We would like to use our current work in biological studies on zebrafish development to automatically segment and clarify different tissue types in the embryo. We would like to extend it to more generalized structures where the microenvironment includes entities other than cells alone. Such structures are evident in histochemically stained images consisting of red blood cells, nuclei, vacuoles, cytoplasm etc. Our density function will also have to be suitably extended to estimate multi-component spatial distributions and packing.