Automatic Cell Segmentation in Fluorescence Images of Confluent Cell Monolayers Using Multi-object Geometric Deformable Model

Abstract

With the rapid development of microscopy for cell imaging, there is a strong and growing demand for image analysis software to quantitatively study cell morphology. Automatic cell segmentation is an important step in image analysis. Despite substantial progress, there is still a need to improve the accuracy, efficiency, and adaptability to different cell morphologies. In this paper, we propose a fully automatic method for segmenting cells in fluorescence images of confluent cell monolayers. This method addresses several challenges through a combination of ideas. 1) It realizes a fully automatic segmentation process by first detecting the cell nuclei as initial seeds and then using a multi-object geometric deformable model (MGDM) for final segmentation. 2) To deal with different defects in the fluorescence images, the cell junctions are enhanced by applying an order-statistic filter and principal curvature based image operator. 3) The final segmentation using MGDM promotes robust and accurate segmentation results, and guarantees no overlaps and gaps between neighboring cells. The automatic segmentation results are compared with manually delineated cells, and the average Dice coefficient over all distinguishable cells is 0.88.

1. INTRODUCTION

With the rapid development of immunofluorescence microscopy for cell imaging, there is a strong and growing demand for image analysis software to quantitatively study cell morphology. Automatic cell segmentation, separating the individual cells from the background, is the initial step prior to any other automatic image analysis procedures. The segmentation result determines the quality of all subsequent analysis.

Various segmentation methods have been proposed for different types of cell images. The watershed algorithm and its many variants^1,2 have gained great popularity for their simplicity and efficiency in separating clusters of cells. Other more advanced algorithms, such as active contours³ and level sets⁴ offer more powerful tools for cell segmentation, as they offer a high degree of accuracy, robustness to noise and weak boundaries, and have the ability to incorporate all image features and prior information about the cell shape. Other promising approaches include the graph-cut methods⁵ and dynamic programming⁶.

In an ideal fluorescence image of a confluent monolayer, the cells cover the plane with no overlaps or gaps and the cell junctions are stained so they appear in a specific color in the image. One challenge is that the images often appear noisy due to various defects. Baggett⁷ and Mucllough⁶ used dynamic programming with user interactions to achieve accurate results on this kind of cell image. Frolkovic⁸ presented an automatic approach by first detecting the centers of nuclei as starting points and then using level set methods to segment individual cells. Since cells were segmented separately in this approach, it cannot guarantee that there’s no overlaps or gaps between multiple cells.

In this work, we propose a fully automatic whole cell segmentation method for the fluorescence images of confluent cell monolayers. First, cell nuclei are automatically detected as initial seeds for subsequent segmentation. To deal with different defects, the cell junctions are enhanced by applying an order-statistic filter and a principal curvature-based image operator. The multi-object geometric deformable model (MGDM)⁹ is used for the final segmentation, which promotes robust and accurate segmentation of the whole cells, and guarantees no overlaps and gaps between multiple cells.

The rest of the paper is organized as follows. Section 2 describes the proposed segmentation method, which includes three main steps: cell nucleus detection, cell junction enhancement, and final segmentation using MGDM. Section 3 shows qualitative and quantitative evaluation of the segmentation results. Section 4 gives the conclusion and a discussion of future work.

2. METHOD

The cells studied in this work are the human umbilical vein endothelial cells (HUVECs). The cells are immunostained for DAPI and VE-cadherin, so that the cell nuclei appear blue and cell junctions appear green in the fluorescence images. Fig. 1 shows an example of the fluorescence images and the individual green and blue channels. The following sections describe the three main steps of our method: 1) Cell nucleus detection, which provides an initial segmentation; 2) Cell junction enhancement; 3) Final segmentation using MGDM.

Figure 1.

Fluorescence images of HUVECs. (a) Merged DAPI (blue) and VE-cadherin (green) channel. (b) Cell nuclei. (c) Cell junction network.

2.1 Cell Nucleus Detection

In order to localize each cell in the image, the cell nuclei (see Fig. 1) are automatically detected and used as initial seeds for subsequent segmentation. The main challenge for cell nucleus detection is overlap due to clustering or multiple nuclei. The popular watershed methods often involve careful seed finding before (or region merging after) the watershed operation to prevent over-segmentation. To avoid this, we present a different approach by analyzing the contour of the nuclei cluster and separating the touching nuclei using the shape characteristic of nuclei. First, a connected region R representing a nucleus or a cluster of nuclei is detected by region growing. ∂R, the boundary of R, is represented by an ordered set of points {p_i}, i ∈ I = {1, 2, … , N_R}. Candidate cuts between touching nuclei are generated by detecting local minima in the contour distance map. The contour distance map D_R of a region R is a N_R × N_R matrix expressing the pair-wise distances between points in {p_i}, e.g. D_R(i, j) is the distance between p_i and p_j. As shown in Fig. 2, the true cut between two touching cells is one of the local minimas in D_R. Then the true cuts are identified by checking the similarity of the segmented contours to an ellipse shape. For a segmented contour c, represented by a set of points {p_ik}, i_k ∈ I_c ⊂ I, an ellipse E_c is fitted to the point set {p_ik}. The similarity of c to an ellipse is quantified by the mean distance from {p_ik} to E_c:

$$S\left(c\right)=\frac{1}{\mid I_c\mid }\sum _{i\in I_c}\mathrm{dist}(p_{ik},E_c),$$

where dist(p,E_c) is the shortest distance from a point p to the ellipse E_c.

Figure 2.

The cut between two touching cells and the corresponding local minima in the contour distance map D_R(i, j).

The automatic cell nucleus detection algorithm can be summarized as follows:

Fig. 3 shows the cell nuclei sequentially detected by running the above procedures. Fig. 3(g) is an image with zero in the background and the region of each cell nucleus labeled with a different color. This labeled image is used as the initialization for the final segmentation using the geometric deformable model in Sec 2.3.

Figure 3.

Cell nucleus detection and initial segmentation. (a) Image of cell nuclei. (b) First contour generated by region growing. (c) Possible cut. (d) First nucleus labeled. (e) Second nucleus labeled. (f) Second contour by region growing. (g) Third nucleus labeled. (h) The labeled image as an initial segmentation.

2.2 Cell Junction Enhancement

The image of the cell junction network is the main input for final segmentation. The quality of the cell junction network directly affects the segmentation results. Ideally, we expect the cell junctions to have a dark background and a bright curvilinear structure. However, real images have various kinds of defects. As shown in Fig. 4, there are bright spots scattered throughout the image and smooth intensity peaks around the cell nuclei. Cell junctions have an irregular appearance and are very weak or even indistinguishable in some areas. In our method, we suppress the defects and enhance the curvilinear structure by first applying an order-statistic filter and then computing the principal curvature of the image intensity surface.

Figure 4.

Cell junction enhancement. (a) Original image of cell junction network. (b) After order-statistic filtering; (c) After applying image operator C.

1) Order-statistic filtering

Order-statistic filtering replaces each pixel p in an image by the k-th largest intensity value in a certain neighborhood of p. Here we use a 3 × 3 neighborhood and choose k = [0.2n], where n is the number of pixels in the neighborhood. The bright spots are eliminated since they often have high intensities but small areas. On the other hand, the curvilinear structure is not affected much since the high intensity area can cover substantial area in the neighborhood of p. Fig. 4(b) shows a typical image of cell junction network after order-statistic filtering.

2) Curvilinear structure enhancement

Second derivatives of image intensity are often used for the detection and enhancement of curvilinear structures^10,11. The differential structure of the intensity surface at a point p = (x, y) is captured by the Hessian matrix,

$$H(x,y)=\left[\begin{array}{cc}\hfill r_{xx}(x,y)\hfill & \hfill r_{xy}(x,y)\hfill \\ \hfill r_{xy}(x,y)\hfill & \hfill r_{yy}(x,y)\hfill \end{array}\right],$$ (1)

where r..(x, y) are second order derivatives of a smooth version of the original image.

The two eigenvectors indicate the directions along which the local surface has the greatest and least curvature, and the corresponding eigenvalues, κ₁(x, y) and κ₂(x, y) (κ₁(x, y) ≥ κ₂(x, y)), are the respective curvature values, called the principal curvatures. For a point on the cell junction, the largest principal curvature κ₁(x, y) should be negative with large absolute value. As a result, κ₁(x, y) can be used to identify the curvilinear structure in the image and suppress other structures such as low frequency intensity variations. We define an image operator C o I(x, y) = max(−κ₁(x, y), 0), which will have a big value for a convex curvilinear structure. Fig. 4(c) shows the image of cell junctions after applying the operator C. The smooth intensity peaks at the cell nuclei are effectively suppressed, and the cell junction becomes a nice curvilinear structure and weak cell junctions are enhanced.

2.3 Final segmentation by MGDM

Geometric deformable models using the level set representations⁴ have gained great popularity in segmentation problems. Most of these methods have been proposed for single objects, however, which makes them unsuitable for the multiple cell segmentation task here. Various extensions of level set techniques to multiple objects have been proposed^9,12,13. The Multi-object Geometric Deformable Model (MGDM)⁹ is ideally suited for our problem because it is computationally efficient even with large numbers of objects and can apply different forces (called speeds in the level set segmentation literature) to different boundaries between objects. In our model, MGDM has N + 1 objects, with N cell objects and one background object. In the final segmentation, we would like: 1) the boundary of cell objects to coincide with the cell junction in the image; 2) no overlaps and gaps between neighboring cells ; 3) the boundary to be smooth and robust to noise in the image. In the following, we describe the external and internal forces used in MGDM to achieve these goals.

External force

Gradient vector flow (GVF)¹⁴ has been widely adopted in the active contour and level set methods to drive an object boundary to desired edges. It is computed as a diffusion of the gradient vectors of the edge map derived from the image. GVF has the advantages of large capture range compared to traditional gradient field, and can force object contours into boundary concavities. In our method, we use a weighted sum of GVF and the gradient field as the external forces. As shown in Fig. 5, the external force is a smoothly changing vector field pointing to the cell junction, with small magnitude in the middle of the cell and large magnitude near the cell junction.

Figure 5.

(a) The external force field and (b)–(f) the MGDM evolution.

Internal force

The internal force includes a balloon force¹⁵ and a curvature smoothing force¹⁶. In our method, we assume that each cell contains only one nucleus, and the nucleus is completely contained within the cell border. The outward balloon force, in the form $\frac{\partial \varphi }{\partial t}=\alpha \mid \Delta \varphi \mid$, makes each cell object expand from the initial nucleus region. It should be noted that, the balloon force is only applied to the cell objects (not to the background object). Therefore, the cell boundary will expand over the background, but once it touches the boundary of a neighboring cell, the touching boundary stops the expansion because the balloon forces from the two cell objects offset, and the boundary is then driven by other forces (external force and curvature force). The curvature smoothing force $\frac{\partial \varphi }{\partial t}=\beta \kappa \mid \Delta \varphi \mid$ preserves the smoothness of the boundary and is applied to all objects.

Considering both internal and external forces, the total force applied to the level set ϕ_i can be written as,

$$f_i=\{\begin{array}{cc}{f}_{GVF}\cdot \Delta {\varphi }_i+(\alpha +\beta \kappa )\mid \Delta {\varphi }_i\mid ,\hfill & \mathrm{if}i\in \{1,2{,}_{\cdots },N\},\left(\mathrm{cells}\right),\hfill \\ f_{GVF}\cdot \Delta {\varphi }_i+\beta \kappa \mid \Delta {\varphi }_i\mid ,\hfill & \mathrm{if}i=N+1,\left(\mathrm{background}\right).\hfill \end{array}\phantom{\}}$$ (2)

Figs. 5 (b)–(f) show the evolution of the segmented regions. For the initial iterations, the cell objects expand under the balloon forces. As the cell boundary gets close to the cell junction in the image, the GVF field dominates and drives the boundary to the cell junctions. Since there is no balloon force applied to the background object, it shrinks so that at convergence, there are no gaps between neighboring cells.

3. EXPERIMENT

The automatic segmentation algorithm is tested on six representative images, containing over 1300 cells in total. The “ground truth” segmentation was obtained by manual delineation of the boundaries of individual cells. Only the cells with no ambiguity and a distinguishable boundary are delineated and evaluated. We also exclude those cells having multiple nuclei, since our algorithm assumes that each cell has only one nucleus. In all, about 900 cells are manually delineated. As shown in Fig. 6, the automatically segmented boundary coincides with the manually delineated boundary in most places. The manual delineation may have gaps between neighboring cells, while the automatic segmentation does not have this problem. In order to quantitatively evaluate the segmentation result, for each manually delineated cell, we first identify the corresponding automatically segmented region, and then compute the Dice coefficient between the automatic segmented region and the manually delineated region. Let $R_i^m$, i = 1, 2, … , M be the region of a manually delineated cell. Let $N_j^a$, j = 1, 2, … , L be the region of a detected nuclei in the automatic segmentation and $R_j^a$ the corresponding whole cell region growing from $N_j^a$. For each manually delineated cell $R_i^m$, we assign

$$R_{k_i}^a:k_i=\arg \max \left(\mid N_{k_i}^a\cap R_i^m\mid \right),k_i\in \{1,2{,}_{\cdots },L\}$$

as the automatically segmented cell region, and the Dice coefficient between U_i and W_ki is computed as

$$d_i=\frac{2\mid R_i^m\cap R_{k_i}^a\mid }{\mid R_i^m\mid +\mid R_{k_i}^a\mid },i=1,2,\cdots ,M,$$

where | · | indicates the area of a region.

Figure 6.

Comparison between automatically segmented and manually delineated cell boundaries in one image.

Fig. 7 shows the histogram of the Dice coefficients computed from the M delineated cells. We can see that there is a peak in the Dice distribution around 0.95 and 81% of the cells had Dice over 0.80. The average Dice is 0.88. The errors are mainly due to the irregular appearance of the cell junctions. The resulting boundary at a weak or broken cell junction is somewhat arbitrary. It may stop at where the growing boundaries of the two neighboring cells touched, or may be driven to a strong junction nearby under the GVF force. In a few extreme cases, the whole cell object shrink and disappear, resulting in a Dice of zero. This problem can be alleviated by further improving the cell junction enhancement and incorporating prior knowledge about the region and shape of each cell in MGDM, e.g., the final segmented cell region should always contain the cell nucleus.

Figure 7.

Histogram of the Dice coefficients computed from the delineated cells in all test images.

4. CONCLUSION

We propose a fully automatic whole cell segmentation method for the fluorescence images of confluent cell monolayers, addressing several challenging problems. First the cell nuclei are automatically detected and used as initial seeds for segmentation. The cell junction is enhanced by order-statistic filtering and a principal curvature based image operator. The multi-object geometric deformable model is used for the final segmentation that promotes robust, accurate segmentation, and guarantees no overlaps and gaps between neighboring cells. Experiments show promising results of the automatic algorithm. Future work would involve: 1) incorporating region and shape prior of cells in MGDM to further improve the robustness and accuracy of the algorithm; 2) introduce a belief measure for each segmented cell quantifying to what degree the segmentation can be trusted; 3) making the algorithm adaptable to other types of cells or cell images.