Coupled Segmentation of Nuclear and Membrane-bound Macromolecules through Voting and Multiphase Level Set

Abstract

Membrane-bound macromolecules play an important role in tissue architecture and cell-cell communication, and is regulated by almost one-third of the genome. At the optical scale, one group of membrane proteins expresses themselves as linear structures along the cell surface boundaries, while others are sequestered; and this paper targets the former group. Segmentation of these membrane proteins on a cell-by-cell basis enables the quantitative assessment of localization for comparative analysis. However, such membrane proteins typically lack continuity, and their intensity distributions are often very heterogeneous; moreover, nuclei can form large clump, which further impedes the quantification of membrane signals on a cell-by-cell basis. To tackle these problems, we introduce a three-step process to (i) regularize the membrane signal through iterative tangential voting, (ii) constrain the location of surface proteins by nuclear features, where clumps of nuclei are segmented through a delaunay triangulation approach, and (iii) assign membrane-bound macromolecules to individual cells through an application of multi-phase geodesic level-set. We have validated our method using both synthetic data and a dataset of 200 images, and are able to demonstrate the efficacy of our approach with superior performance.

1 Introduction and motivation

Cell surface proteins control and regulate cell-cell interactions and the physical properties of tissue architecture. This paper focuses on the segmentation of membrane proteins that visualize with curvilinear diffused signal along the cell surface boundaries when imaged by fluorescence microscopy. For example, the cadherin family of membrane proteins have a diffused signature, while the connexin family of proteins are sequestered between neighboring cells. In this paper, we used samples that have been labeled with E-cadherin antibody as a proxy for validating computational steps for a wider class of cell surface proteins. The E-, R-, and Ncadherin families of adhesion molecules are known to regulate the dynamic properties of cell-cell adhesion. For example, E-cadherin is a calcium-dependent cell adhesion molecule that influences differentiation and tissue structure; it is a class of an adherent junction between epithelial cells with access to the actin cytoskeleton through cadherin attachment proteins. As an endpoint, E-cadherin has been studied extensively, since it appears to function as a barrier to cancer. Loss of Ecadherin has been associated with (i) increased motility, (ii) potential cancer progression and metastasis, and (iii) increased resistance to normal cell death. Since down-regulation of E-cadherin is a critical endpoint for quantitative cell systems biology, detailed segmentation of the E-cadherin signal provides important clues for understanding biological processes under different sets of experimental factors and perturbation. An example of the spatial organization of the E-cadherin adhesion molecules is shown in Figure 1.

Fig. 1.

An example of an E-cadherin signal: (a) A composite image of nuclei and membrane signals; (b) The corresponding nuclear channel; and (c) Membrane channel.

Even though cell-cell adhesion molecules express themselves as locally linear structures, these signals suffer from perceptual gaps, non-uniformity in intensity and scale (e.g., thickness), as well as noise. The expressions of these molecules are often examined in the context of nuclear substructures with both the nuclear and adhesion molecules being labeled with different fluorescent probes. Current literature is rich in terms of the delineation of nuclear morphology and its shape features on a cell-by-cell basis; however, the area of research on segmentation of cell-cell adhesion molecules or cell surface proteins remains largely unexplored. Here, we introduce an integrated method for the segmentation of the nuclear regions and membrane-bound macromolecules. Essentially, the overall theme of our research is to leverage geometric reasoning and complement it with evolving fronts, when appropriate. Examples of composite and individual channels are shown in Figure 1. In the case of the segmentation of the nuclear regions, potential ambiguities, such as overlapping adjacent nuclei, are resolved by exploiting convex properties of the nuclear shape. In the case of membrane-bound macro-molecules, local linear structures that delineate cellular boundaries are first regularized based on proximity and continuity. Then by leveraging the nuclear shape as a reference for initial condition, a multi-phase evolving front is designed for delineating membrane-bound macromolecules on a cell-by-cell basis. In short, our proposed method (i) delineates overlapping nuclear shapes through geometric analysis and partitioning, (ii) regularizes fluorescent signals at the cell-cell interfaces, and (iii) applies a multi-phase geodesic level-set to assign membrane signals to each cell. Each step is validated with synthetic data, compared with some of the existing methods, and evaluated on real data.

Organization of this paper is as follows: Section 2 provides a brief review of the previous research; Section 3 describes our approach and details our implementation of tangential voting; Section 4 validates our approach using synthetic data; Section 5 demonstrates the experimental results; and Section 6 concludes the paper.

2 Review of Previous Work

The difficulties in the segmentation of surface protein localization are often due to variations in scale, noise, and topology. Other complexities originate from missing data and perceptual boundaries that lead to diffusion and dispersion of the spatial grouping in the image space. Techniques for grouping local image features into globally salient structures have incorporated clustering and graph theoretical methods [1], Bayesian models for combining tangential representations of sparse contours [2], tensor voting [3] for grouping or interpolating distant features, etc. While these techniques differ in their concepts, they are all model-free and share a common thread of continuity and proximity along the minimum energy path, which infers global saliency. More recently, prior shape models have also been incorporated for boundary closure [4]. Since nuclear segmentation is one of the initial steps of the process, we provide a brief review on this topic and then proceed on quantification of membrane signals.

With respect to the nuclear segmentation, literature is rich, especially where the key issue is delineation of overlapping nuclei distribution as a result of fixation, and a recent comparative study examines traditional nuclear segmentation methods in this context [5]. The key concepts have been watershed[6], geometric methods [7,8], active surface models for segmentation of nuclear regions [9–11], and hybrid combination of these methods. In [12], detection of nuclear regions was modeled through Laplacian of Gaussian (LoG). The response from the LoG filter was then analyzed for peak detection corresponding to the center of mass for each nucleus. This method is non-iterative, incorporates a fixed filter size, and is appropriate when nuclear morphology is relatively homogeneous. This is essentially a detection technique, however, it does provide some constraints to bound the segmentation problem. In [13], segmentation and classification are tightly coupled for a model-based framework. Their process was initiated from a watershed-based method, which partitions the image and potentially fragments each nucleus. Through an efficient hierarchical strategy, two or more fragments are merged together to form a candidate. Each candidate is then scored to guide the merging process. The final blob is also classified against one of the models to reveal the nuclear type. However, the segmentation results are sensitive to initial conditions for the generation of “merge tree.” In a later study by [14], a method based on regularization of the gradient vector flow was proposed. This method was roughly an extension of the regularized centroid transform [15] to 3D data. The basic idea is to apply the gradient flow tracking algorithm to label each pixel with a converged sink position that corresponds to cell centroid. This method is iterative, computationally intensive, and has been applied to low resolution fluorescent images. The main limitations of this method are that (i) multiple sinks may be detected for elongated nuclei, and (ii) it assumes that the nuclear intensity is homogeneous and does not contain obvious inner structures. Whereas in [16], an efficient implementation of the level-set method [17] was proposed by reducing complexities associated with the original multi-phase implementation [18]. They leveraged the four-color property of the spatial organization of different objects to reduce the number of evolving fronts. The net result was an efficient implementation, by reducing the number of coupling terms, which has been applied to the tracking of cells in wound healing assays. These assays are imaged through phase contrast microscopy: however, it should be noted that initialization could be a potential problem for fixed cells with overlapping compartments. In [19], an interactive method with evolving fronts was proposed for cell segmentation. In [20], prior knowledge from distinct examples were integrated in the graphcut framework to tackle the batch effect in large dataset.

With respect to segmentation of the signals at the cell membrane, several key concepts have been introduced. [21,22] proposed a three-step strategy based on (i) tangential voting for regularizing and grouping membrane-bound signals, (ii) delineation of nuclear regions through a combination of zero-crossing and gradient filter, and (iii) a snake-based model based on gradient vector field (GVF) for the delineation membrane-bound signals. While [23] introduced the concept of iterative tensor voting for simultaneous denoising and perceptual grouping of membrane-bound signals. [24] proposed a method based on subjective surfaces, and applied it to confocal data that was collected from zebrafish. This method was preceded by spatial non-linear filtering of the images for improved performance. Similar strategy has also been proposed for delineating membranes in embryo [25]. [26] utilized nuclear region as a reference to approximate neighborhood for membrane regions. It then used the labeling information (e.g., a specific dye) to delineate membrane signals based on continuity. [27] utilized shape information for cell segmentation and classification. To a large degree, most of above methods leveraged perceptual grouping and evolving fronts with various degrees of complexities and robustness. The main contributions of our paper are (i) allow tangential voting to regularize and group membrane signal, and (ii) exploit multi-phase evolving fronts to relax to the regularized signal. The net results are robustness, invariance to missing boundaries, and quantification of the membrane signal on a cell-by-cell basis.

3 Approach

Figure 2 shows the computational steps in segmenting membrane-bound macromolecules, where the solution is constrained with the segmentation of nuclear regions as well as the gap filling of membrane-bound proteins. The application of multi-phase geodesic level-set provides the final assignment of surface proteins on a cell-by-cell basis. An visual example can be found in Figure 14. Below, we will summarize our approach to nuclear segmentation and then proceed with delineation of membrane-bound proteins.

Fig. 2.

Computational steps in delineating membrane-bound macromolecules

Fig. 14.

Computational steps in the assignment of cell surface protein markers to each nucleus: (a) original nuclear regions; (b) segmented nuclear features; (c) original cell surface marker; (d) voted cell surface proteins; (e) composite image of both nuclear channel and membrane channel; (f) segmentation of cell surface protein.

3.1 Nuclear Segmentation

A key observation, we made, is that the nuclear shape is typically convex. Therefore, ambiguities associated with the delineating overlapping nuclei could be resolved by detecting concavities and partitioning them through geometric reasoning. The process is shown in Figure 3. First, the original image was segmented into foreground and background by coupling gradient and zero-crossing filters [7]. Subsequently, the contours were extracted, points of maximum curvature were detected, and triangulation was performed between these points for hypothesizing convex blobs. This method is similar to the one proposed in our previous work [7], however, a significant performance improvement has been made through triangulation and subsequent geometric reasoning. In the remainder of this section, we outline the details of the method, evaluate the method on synthetic data, and compare the performance of our method with that of the watershed method.

Fig. 3.

Steps in delineating overlapping nuclei.

3.1.1 Delaunay Triangulation of Points of Maximum Curvature for Hypothesis Generation and Edge Removal

The curvature along the contour is computed by using $k=\frac{x\prime y″-y\prime x″}{{({x\prime }^2+{y\prime }^2)}^{3/2}}$, where x and y are coordinates of the boundary points. The derivatives are computed by convoluting the boundary with derivatives of Gaussian. An example of detected points of maximum curvature whose k values are larger than threshold λ_k are shown in Figure 4. The ith point of curvature maxima is denoted as v_i, and a set of M points of maximum curvature along a closed contours is denoted as $V={\cup }_{i=1}^M{\upsilon }_i$.

Fig. 4.

Points of maximum curvature: (a) detected maxima, and (b)curvature profile for one object with closed boundary.

We then applied the Delaunay Triangulation (DT) [28] to all points of maximum curvature to hypothesize all possible groupings. The main advantage of DT is that the edges are non-intersecting, and the Euclidean minimum spanning tree is a subgraph of DT. Let e_ij be the edge connecting two points of curvature maxima v_i and v_j, as shown in Figure 5. Let T_i and T_j be the unit vectors representing the tangent directions at v_i and v_j, and β_ij and β_ji be the angles formed by T_i, T_j and e_ij. From a grouping perspective, Let triangulated edges representing one connected component (e.g., a group of connected nuclei) be denoted by E = ∪e_ij for i, j ∈ {1, …, M}, and i ≠ j. Let θ ∈ E be a decomposition of the configuration space Ω. Therefore, the number of possible decompositions in this space is |Ω| = 2^M.

Fig. 5.

Geometric attributes of points of maximum curvature for a hypothesized edge for completing perception boundaries.

3.1.2 Geometric Reasoning

DT edges provide a natural way of perceptually grouping the points of maximum curvature to decompose a clump of nuclei. Nonetheless, the number of edges can incur a high computational costs for finding θ^*. In this section, our aim was to recover a decomposition of θ^* that best fits a set of geometric constraints. For a hypothesized edge e_ij and its attributes, these constraints are: (i) it must be inside the clump; (ii) that the angle between T_i and T_j should be maximized (e.g., they should be antiparallel); (iii) that β_ij,β_ji should be as close as possible to π/2; and (iv) it must not intersect other edges. Therefore, the following set of rules can reduce the search space: (i) deleting edge crossing the background; (ii) deleting edge e_ij if (T_i·T_j) > λ_T; and (iii) deleting edge e_ij if max(|T_i · e_ij/|e_ij||, |T_j · e_ji/|e_ji||) > λ_β, where λ_T and λ_β are very conservative thresholds to simply throw away extreme cases. An example of the application of these constraints is shown in Figure 6(b) and Figure 6(c).

Fig. 6.

Removal of triangulated edges through stepwise refinement and application of geometric constraints. (a) Delaunay Triangulation;(b) No background edges; (c) Edge pruning;(d) Edge inference.

An interesting observation was that such a simple set of constraints eliminated many of the incorrect hypotheses rapidly leaving only a few for further validation. Nevertheless, it is possible that more than one configuration of partitioning (e.g., θ) will satisfy the geometric constraints. In that case, our metric will then be to adopt a configuration with the best convexity, which is defined as $C=-N+{\sum }_{i=1}^N{\varphi }_i/\pi$. Here, ϕ_i is the sum of the tangent angles formed along the contour of the ith partition, and N is the total number of decomposed partitions of the clump. As a result, the number of edges for examination is reduced from M(M + 1)/2 to less than 3(M - 2) with a computational complexity of O(M log M).

After the geometric constraints have been applied, the resulting edge set becomes sparse and ready for the application of simple inference rules to get θ^∗. Denoting the input and output edge sets as E_in and E_out, with point set V_in and V_out, respectively, and deg(v_i) as the degree of point v_i in the edge set, the algorithm for edge inference is summarized as follows:

1. Let E_in be the edge set after edge pruning, and E_out ← ∅.

2. While E_in ≠ ∅

   1. In V_in, if deg(v_i) = 1, then E_in ← E_in \ e_ij and E_out ← E_out ∪ e_ij.

   2. In E_out, if e_ij ∈ E_out, then E_in ← E_in \ e_i* \ e_j*, where * stands for vertices.

   3. If e_ij ∈ E_in, e_jk ∈ E_in, and e_ki ∈ E_in, with deg(v_i) = 2, deg(v_j) = 2, and deg(v_k) = 2, then E_in ← E_in \ e_ij \ e_jk \ e_ki, and E_out ← E_out ∪ e_ij ∪ e_jk ∪ e_ki.

3. For v_i ∈ V with no e_i* ∈ E_out. Use its tangent normal direction to generate an edge into E_out.

4. For e_ij ∈ E_out, e_jk ∈ E_out, and e_ki ∈ E_out, with deg(v_i) = 2, deg(v_j) = 2,and deg(v_k) = 2, choose the two edges which produce the minimum convexity after decomposition, and delete the remaining one from E_out.

In the case where |V| = 1, E_in is set to be ∅. For the case where |V| = 2, we set E_in = {e₁₂} if e₁₂ passes the edge pruning test, or E_in = ∅ if not. As shown in Figure 6(d), the connected component is correctly decomposed into convex regions by the final edge set θ^* = E_out.

3.1.3 Experimental Results

Here, we (i) examined the behavior of the method with synthetic data; (ii) validated the technique with real data; and then (iii) compared the performance of the method with a traditional method. The empirical parameter settings were λ_β = 0.9, λ_T = -0.15, and λ_k = 0.1. With respect to comparison with the previous literature, we have opted to test the watershed method with distance transform [29], since it is widely used by the microscopy community.

In the synthetic test, objects were generated randomly, and noise was added, as shown in Figure 7(a). This experiment showed that the marker-guided watershed reduces over-segmentation as compared with the original watershed method. Nonetheless, the results lacked smoothness along the inferred edges, and there was an inherent loss of accuracy. In contrast, our proposed method partitions the clumps along the expected locations (e.g., points of maximum curvature) while eliminating over-segmentation.

Fig. 7.

Results of synthetic data: (a) original synthetic data; (b) results from watershed method; (c) results from the marker-guided watershed method; and (d) results from our method.

In the case of real data, a set of 10 DAPI-stained images were acquired, with each image containing roughly 100 cells. The watershed method with marker-based constraint significantly reduced over-segmentation; however, it still could not decompose some of the touching nuclei. Whereas, our method consistently performed better than the marker-based approach. Comparative results for three different images are shown in Figure 8.

Fig. 8.

Performance on real data: the original image (top row), watershed results (second row), marker-guided watershed results (third row), and our method results (bottom row).

3.2 Iterative Voting

The concept of iterative voting [30, 22], which has also been extended to iterative tensor voting [23], has already been introduced. To briefly review, the main theme of iterative voting is to infer saliency, which can be in the form closure, continuity, and symmetry. The inference is achieved by means of designing kernels that elucidate a specific feature through iterative refinement. For example, in [30, 31], we showed how the center of mass for a blob-like object can be inferred; thus, constraining the decomposition of overlapping nuclei. The center of mass is then inferred by projecting gradient features radially through kernels that are cone-shaped, and have their maximum strength at a specific distance from the vertex of the cone. The corresponding kernels and stepwise refinement of the center of mass are shown in Figures 9 and 10, respectively. Figure 11shows the iterative nature of the kernel voting, where at each iteration the kernel aperture (e.g., Figure 9) is reduced, and redirected in the direction of maximum voting landscape.

Fig. 9.

Kernel topography: (a-d) Evolving kernel for the detection of radial symmetries (shown at a fixed orientation) has a trapezoidal active area with Gaussian distribution along both axes.

Fig. 10.

Detection of radial symmetries for a multicellular system with overlapping nuclei: (a) original image; and (b-c) voting landscape at several intermediate steps indicating convergence to nuclear seeds.

Fig. 11.

(a) Redirection of the kernel in the next iteration where Q is locally maximum; and (b) general flow of the algorithm.

Alternatively, in [22], we demonstrated that through a different kernel design perceptual gaps can be completed. In this case, the kernels are still cone-shaped, but their strength decays as a function of the distance to the vertex, as shown in Figure 9. We have suggested that iterative voting and differential-based methods are similar in that they are both iterative, have denoising attributes, and can enhance a specific saliency for further processing. The nature of these kernels are shown in Figure 12.

Fig. 12.

Kernel topography: Oriented kernels for inference of continuity are bidirectional, and their energy dissipates as a function of distance. Initially, the energy is dispersed (top row), but becomes more focused (bottom row).

Specifically, in the application to the regularization of membrane-bound macromolecules (as shown in Figure 13), the membrane signals correspond to the negative curvature maxima at a given scale within the image space. But curvature features are noisy and may suffer from undesirable artifacts. The process is initiated by voting with a Gaussian kernel at each image feature point. Let F (x_o, y_o) be the curvature feature at location (x_o, y_o) in the image. Let (x_n, y_n) be a point in the neighborhood of (x_o, y_o) that can be influenced with a kernel applied at position (x_o, y_o). The initial voted image is then represented as

Fig. 13.

Iterative tangential voting: (a) a small region of the original image, (b) voting results after one iteration, and (c) voting results after 9 iterations. Iterative voting thins the location of the signal, reduces spurious noise, and improves continuity.

$$V(x_n,y_n)=\sum _{(x_n,y_n)\in \mathit{\text{Neighbor}}(x_o,y_o)}\{F(x_o,y_o)\ast G_{(x_o,y_o)}(\sigma )\}$$ (1)

The refinement of the voted image is iterative, involving application of a more focused kernel at the next iteration along the α direction.

$$\alpha =arctan\frac{V_{yy}-K_{\mathit{max}}}{V_{xy}}$$ (2)

Where V_yy and V_xy are the local derivatives of the voted image, and K_max is the maximum curvature computed from the Hessian of the voted image. The shape of the kernels, shown in Figure 12, indicates whether the energy distribution of the kernel is focused or dispersed. Initially, the energy is dispersed; however, at each consecutive iteration, the energy becomes more focused and at the same time the kernel orientation is redirected along the direction of maximum response, as shown in Figure 11(a), and the entire process is shown in Figure 11(b). These voting kernels are precomputed and indexed for rapid retrieval.

$$V(x_n,y_n)=\sum _{(x_n,y_n)\in \mathit{\text{Neighbor}}(x_o,y_o)}\{F(x_o,y_o)\ast \mathit{\text{Kernel}}(\sigma ,\theta ,\alpha )\}$$ (3)

The iterative voting algorithm is summarized as follows,

1. Initialize the parameters: Initialize r_max, Δ_max, and a sequence Δ_max = Δ_N < Δ_N-1 < … < Δ₀ = 0. Set n := N, where N is the number of iterations, and let Δ_n = Δ_max. r_max refers to the extent of the voting influence along a given orientation. r_max decays as a function of the distance to the voting pixel.

2. Initialize the saliency feature image: Define the feature image F (x, y) to be the local external force at each pixel of the original image. The external force is set to the maximum (negative) curvature, which corresponds to the membrane signal.

3. Initialize the voting magnitude: Apply the isotropic voting of Equation 1.

4. Update the voting direction: Compute the Hessian of the voted landscape and construct an orientation map based on Equation 2.

5. Refine the angular range: Let n := n - 1, apply Equation 3, and repeat steps 4-5 until n = 0.

3.3 Integration with multi-phase level-set

Our intent is to quantify membrane-bound protein localization on a cell-by-cell basis, and the nuclear channel is used as a context in which the segmentation problem is constrained. Here, we utilize the concept of multi-phase level-set framework [32,18,16], where the external energy is derived from membrane-bound macromolecules following the use of iterative tangential voting, which is outlined earlier. Within this framework, we have two principles:

1. The evolving contour (zero level-set) represents the membrane for each nucleus, which means the contour must attach to the membrane signal.

2. Each phase represents a unique cell, and different phase regions (cell regions) do not overlap.

Based on the principles listed above, and using the Heaviside function H, and the one-dimensional Dirac measure δ, defined by

$$H(z)=\{\begin{array}{c}1,\text{if}z\ge 0\\ 0,\text{if}z<0\end{array},\delta (z)=\frac{dH(z)}{dz}$$

The energy form can be written as:

$$E=\mu \sum _i^M{\int }_{\Omega }g(I)|\nabla {\mathrm{\Phi }}_i(x,y)|{\delta }_i(x,y)\mathit{dxdy}+\lambda \sum _{i=1}^M\sum _{j=1,j\ne i}^M{\int }_{\Omega }H({\mathrm{\Phi }}_i)H({\mathrm{\Phi }}_i)\mathit{dxdy}$$ (4)

in which, Ω is the image domain; M is the number of nuclei; I is the enhanced membrane image; Φ is the Lipschitz function, whose zero level-set is designated to attach to the membrane signal; µ and λ are constant coefficients, weighting different terms; and

$$g(I)=\frac{1}{1+{|\nabla I|}^p}$$ (5)

The first term is the geodesic length of the zero level-set, forcing the zero level-set to attach to the membrane signal; the second term is the penalty for overlapping. The minimization of the objective function is achieved by gradient descent based on the corresponding Euler-Lagrange equation:

$$\frac{\partial {\mathrm{\Phi }}_i}{\partial t}=\delta ({\mathrm{\Phi }}_i)\cdot (\mu \nabla g\frac{\nabla {\mathrm{\Phi }}_i}{|\nabla {\mathrm{\Phi }}_i|}+\mu \mathit{gdiv}(\frac{\nabla {\mathrm{\Phi }}_i}{|\nabla {\mathrm{\Phi }}_i|})-\lambda \sum _{j=1,j\ne i}^{M}H({\mathrm{\Phi }}_i))$$ (6)

To discretize the equation: let h be the space step, Δt be the time step, and (x_p,y_q) = (ph, qh) be the grid points. The finite differences are:

$$\begin{array}{l}{\mathrm{\Delta }}_{+}^x\mathrm{\Phi }(p,q)=\mathrm{\Phi }(p+1,q)-\mathrm{\Phi }(p,q)\\ {\mathrm{\Delta }}_{-}^x\mathrm{\Phi }(p,q)=\mathrm{\Phi }(p,q)-\mathrm{\Phi }(p-1,q)\\ {\mathrm{\Delta }}_{+}^y\mathrm{\Phi }(p,q)=\mathrm{\Phi }(p,q+1)-\mathrm{\Phi }(p,q)\\ {\mathrm{\Delta }}_{-}^y\mathrm{\Phi }(p,q)=\mathrm{\Phi }(p,q)-\mathrm{\Phi }(p,q-1)\end{array}$$

We compute Φⁿ+1 by the following discretization:

$$\begin{array}{l}\frac{{\mathrm{\Phi }}_i^{n+1}-{\mathrm{\Phi }}_i^n}{\mathrm{\Delta }t}={\delta }_h({\mathrm{\Phi }}_i^n)\left[\frac{\mu }{h^2}\frac{{\mathrm{\Delta }}_{+}^{x}g{\mathrm{\Delta }}_{+}^x{\mathrm{\Phi }}_i^n}{\sqrt{\frac{{({\mathrm{\Delta }}_{+}^x{\mathrm{\Phi }}_i^n)}^2}{h^2}+\frac{{({\mathrm{\Delta }}_{+}^y{\mathrm{\Phi }}_i^n)}^2}{h^2}}}\right]\\ +{\delta }_h({\mathrm{\Phi }}_i^n)\left[\frac{\mu }{h^2}\frac{{\mathrm{\Delta }}_{+}^{y}g{\mathrm{\Delta }}_{+}^y{\mathrm{\Phi }}_i^n}{\sqrt{\frac{{({\mathrm{\Delta }}_{+}^x{\mathrm{\Phi }}_i^n)}^2}{h^2}+\frac{{({\mathrm{\Delta }}_{+}^y{\mathrm{\Phi }}_i^n)}^2}{h^2}}}\right]\\ +{\delta }_h({\mathrm{\Phi }}_i^n)\left[\frac{\mu g}{h^2}{\mathrm{\Delta }}_{+}^x(\frac{{\mathrm{\Delta }}_{+}^x{\mathrm{\Phi }}_i^n}{\sqrt{\frac{{({\mathrm{\Delta }}_{+}^x{\mathrm{\Phi }}_i^n)}^2}{h^2}+\frac{{({\mathrm{\Delta }}_{+}^y{\mathrm{\Phi }}_i^n)}^2}{h^2}}})\right]\\ +{\delta }_h({\mathrm{\Phi }}_i^n)\left[\frac{\mu g}{h^2}{\mathrm{\Delta }}_{+}^y(\frac{{\mathrm{\Delta }}_{+}^y{\mathrm{\Phi }}_i^n}{\sqrt{\frac{{({\mathrm{\Delta }}_{+}^x{\mathrm{\Phi }}_i^n)}^2}{h^2}+\frac{{({\mathrm{\Delta }}_{+}^y{\mathrm{\Phi }}_i^n)}^2}{h^2}}})\right]\\ -{\delta }_h({\mathrm{\Phi }}_i^n)\cdot \lambda \cdot \text{∑}_{j=1,j\ne i}^{M}H({\mathrm{\Phi }}_j^n)\end{array}$$ (7)

4 Validation on Synthetic Data

To further validate the performance of our approach, we created synthetic nuclear channel and E-cadherin channel, together with the ground truth for both nuclear segmentation and E-cadherin detection, as shown in Figure 15. Experiments were carried out with the same parameter settings, upon the images with gaps in Ecadherin channel, and with different level of noise in both two channels; and comparisons were made between our approach and previous approaches for both nuclear segmentation and membrane segmentation. The performance is summarized in Table 1 and 2, in which E-cadherin detection accuracy was measured by the statistics of the distance(error) between detected points and the ground truth location, and nuclear segmentation was validated by the mean precision and mean recall compared with ground truth, where

Fig. 15.

Validation on synthetic nuclear and membrane bound signals: (a) Synthetic nuclear channel; (b) Ground truth of nuclear segmentation mask; (c) Synthetic E-cadherin channel; (d) Ground truth of E-cadherin center line; (e)(k)(q)(w) Nuclear channel with SNR = {0,20,10,5}db, respectively; (f)(l)(r)(x) Nuclear segmentation with proposed method; (g)(m)(s)(y) Nuclear segmentation with marker-guided watershed; (h)(n)(t)(z) E-cadherin channel with gaps and SNR = {0,20,10,5}db, respectively; (i)(o)(u)(aa) E-cadherin segmentation with proposed method; (j)(p)(v)(ab) E-cadherin segmentation with GVF snake; The last two columns, in each row, can best be visualized by enlarging the pdf file.

Table 1.

Mean precision and recall of synthetic nuclear segmentation at different noise level based on segmentation result and ground truth, as shown in Fig 15.

	Mean Precision / Mean Recall
SNR (dB)	Marker-guided Watershed	Proposed Method
∞	0.895/0.935	1.0 / 1.0
20	0.856/0.874	0.999 / 0.999
10	0.835/0.869	0.988 / 0.998
5	0.798/0.705	0.936 / 0.994

Table 2.

Mean and STD Distance between E-cadhern detection and ground truth location at different noise level compared with ground truth, as shown in Fig 15.

	Mean Distance ± STD Distance (pixel)
SNR (dB)	GVF Snake	Proposed Method
∞	1.212 ± 2.297	0.743 ± 0.824
20	1.280 ± 2.878	0.778 ± 1.015
10	3.553 ± 6.316	0.795 ± 1.018
5	10.629 ± 16.416	1.025 ± 1.571

$$\begin{array}{ccc}\hfill M& \hfill :\hfill & \text{Total number of nuclei in ground truth image}\hfill \\ \hfill N_i& \hfill :\hfill & \text{The}{i}^{th}\text{nucleus}\hfill \\ \hfill \text{mean precision}& \hfill =\hfill & \frac{1}{M}\text{∑}_{i=1}^M\frac{\text{Number of Pixels Correctly Labeled for}{N}_i}{\text{Total Number of Pixels Labeled for}{N}_i}\hfill \\ \hfill \text{mean recall}& \hfill =\hfill & \frac{1}{M}\text{∑}_{i=1}^M\frac{\text{Number of Pixels Correctly Labeled for}{N}_i}{\text{Total Number of Pixels Labeled for}{N}_i\text{in Ground Truth}}\hfill \end{array}$$

Figure 15 indicates that our method can easily deal with the discontinuity in Ecadherin signal, which is typical in the real data; and Tables 1 and 2 show that our method is immune to low signal to noise ratio.

It is obvious that the performance of our propose approach, for both nuclear segmentation and membrane segmentation, is superior to previous methods.

5 Evaluation on real data

An experiment has been designed and samples have been imaged to evaluate the performance of the method. In this experiment, samples are labeled with a counter-stain to visualize nuclear features, and with an antibody against the E-cadherin for visualizing adherents junctions. A total of 201 images (1344-by-1024 pixels) were collected, nuclear regions were segmented, and then E-cadherin signal was assigned to each nuclear region. The parameters were fixed for the entire dataset at µ = 100, λ = 5, Δt = 0.1, and iteration = 100. Table 3 shows the average computational time for each step. Figure 16 shows one example of our experimental result, processing steps, and comparison with an earlier method based on GVF snake [21,22]. The GVF snake has the property that its evolution can stop due to absence of attracting forces, which is illustrated in the red bounding boxes in Figure 16. In addition, Figure 16 shows intermediate results of tangential voting for smoothing the membrane-bound signal [33]. Tangential voting regularizes the membrane-bound signal through enhancement, noise reduction, and gap filling of small regions. In our experimental data set, the quality of segmentation of the membrane-bound protein is nearly perfect, and any erroneous segmentation is due to incorrect nuclear segmentation, which is typically caused by the following factors:

Table 3.

Average computational time for each step. Dataset: 201 images; Image size: 1344-by-1024 pixels; Average number of nuclei per image is around: 100; Platform: Intel(R) Xeon(R) CPU, X5356 @3.00GHz.

	Nuclear Segmentation	Iterative Voting	Multi-phase level-set
Average time cost	1 sec	60 sec	10 min

Fig. 16.

Performance of the proposed nuclear segmentation and multi-phase level set is compared with watershed segmentation and gradient vector field snake model: (a) composite image of both nuclear channel and membrane channel; (b) original nuclear region; (c) segmented nuclear features with method in [7]; (d) segmented nuclear features with our approach; (e) original E-cadherin signal; (f) result of tangential voting along the membrane-bound signal; (g) segmentation of cell surface protein via GVF snake [21]; and (h) segmentation of cell surface protein with our approach(multi-phase geodesic level-set); (i-l) intermediate results in level-set evolution. The red rectangles indicate that the evolution of GVF snake can stop due to the absence of attracting forces (i.e., the membrane signal).

1. Missing points of maximum curvature. During the detection of points of maximum curvature, the scale of Gaussian kernel is determined experimentally. However, there are cases that the touching nuclei do not form significant curvature along the contour, which leads to under-segmentation of the touching nuclei. This is a rare event based on our observation.

2. Sometimes, during the sample preparation and fixation, nuclear regions overlap and form large clumps; this is a quality control issue associated with sample preparation In these cases, even with correct segmentation of the nuclear regions, membrane-bound signals are meaningless, since it is not clear with which nuclei they are associated.

6 Conclusion and future work

A series of computational steps were proposed and tested on real data to delineate cell surface proteins. Cell surface proteins are heterogeneous in width and suffer from perceptual gaps. A multi-step process was proposed to couple the segmentation of both nuclear signal and membrane-bound macromolecules. First, the nuclear segmentation provided context and the necessary reference for initializing the evolving fronts for quantifying membrane-bound signal on the cell-by-cell basis. Next, iterative tangential voting was applied to enhance and regularize membrane proteins while diffusing noise. Lastly, a multi-phase geodesic level-set model with improved performance and geometric stability is designed and validated. The pipeline has been first profiled on synthetic data and then tested on real data. We show the superior performance of our approach when compared to alternative strategies. Our future work will focus on (i) improving the computational efficiency for multi-phase levelset based on [16], and (ii) extending the system to 3D.