Segmentation of multicolor fluorescence in situ hybridization images using an improved fuzzy C-means clustering algorithm by incorporating both spatial and spectral information

Abstract.

Multicolor fluorescence in situ hybridization (M-FISH) is a multichannel imaging technique for rapid detection of chromosomal abnormalities. It is a critical and challenging step to segment chromosomes from M-FISH images toward better chromosome classification. Recently, several fuzzy C-means (FCM) clustering-based methods have been proposed for M-FISH image segmentation or classification, e.g., adaptive fuzzy C-means (AFCM) and improved AFCM (IAFCM), but most of these methods used only one channel imaging information with limited accuracy. To improve the segmentation for better accuracy and more robustness, we proposed an FCM clustering-based method, denoted by spatial- and spectral-FCM. Our method has the following advantages: (1) it is able to exploit information from neighboring pixels (spatial information) to reduce the noise and (2) it can incorporate pixel information across different channels simultaneously (spectral information) into the model. We evaluated the performance of our method by comparing with other FCM-based methods in terms of both accuracy and false-positive detection rate on synthetic, hybrid, and real images. The comparisons on 36 M-FISH images have shown that our proposed method results in higher segmentation accuracy ($0.9382\pm 0.0250$) and a lower false-positive ratio ($0.1042\pm 0.1481$) than conventional FCM (accuracy: $0.9210\pm 0.0457$, and false-positive ratio: $0.1389\pm 0.1899$) and the IAFCM (accuracy: $0.8730\pm 0.2438$ and false-positive ratio: $0.2438\pm 0.2438$) methods by incorporating both spatial and spectral information from M-FISH images.

1. Introduction

Both numerical and structural chromosomal abnormalities, such as duplications, translocations, inversions, and deletions, are important factors associated with complex diseases. For example, Down syndrome is caused by having three copies of chromosome 21.¹ The detection of chromosomal abnormalities has been used for prenatal and postnatal diagnostics and for cancer cytogenetics research. Multicolor fluorescence in situ hybridization (M-FISH) is a powerful tool for simultaneous visualization of chromosomal abnormalities in a single cell by labeling chromosomes with different fluorophores, where the Boolean combinational labeling strategy is used. The number of the combinations of $M$ fluorophores is $2^M-1$; therefore, five different fluorophores are sufficient for differentiating 24 types of chromosomes (22 autosomes and two sex chromosomes). Figure 1 shows an image set with six-channel M-FISH images from a single metaphase cell. In Figs. 1(a)–1(e), the chromosomes labeled with different fluorophores are acquired by an epifluorescence microscopy equipped with a filter cube [including far red (F), spectrum green (G), spectrum aqua (A), spectrum red (R), and spectrum gold (Y)]. The other image, Fig. 1(f), is 4 in, 6-diamidino-2-phenylindole (DAPI), which is labeled as D. All chromosomes are labeled in DAPI channel, so they are all visible in the image D.

Fig. 1.

A demonstration of an M-FISH image set with six-channel M-FISH images from a single metaphase cell. In images (a)–(e), different subsets of chromosomes are labeled with different fluorophores and are acquired by an epifluorescence microscopy equipped with the filter cubes [far red (F), spectrum green (G), spectrum aqua (A), spectrum red (R), and spectrum gold (Y), respectively]. The other image (f) is DAPI channel, labeling all chromosomes in the image.

Accurate segmentation of M-FISH images has a significant impact on subsequent chromosomal classification and hence on clinical diagnosis. In our previous works on chromosome classification, including Bayesian classifier,² sparse representation-based classifier,³ regularized multinomial logistic classifier,⁴ structural sparse representation classifier,^5,6 and patch-based tensor decomposition classifier,⁷ several steps have been done in preprocessing^8,9 to facilitate the analysis of M-FISH images. Among them, image segmentation is a critical step and can significantly improve classification accuracy.^9,10 The other importance of segmentation is to take advantage of five fluorescence channels to code different chromosomes. With accurate segmentation of chromosome in each channel, the chromosomes can be better distinguished with the combination of all channels of information. Therefore, accurate segmentation is critical for M-FISH image analysis. By segmentation, a mask is generated to separate chromosome regions from the background. However, the quality of acquired M-FISH images is usually poor due to various factors,¹¹ such as uneven intensity between imaging channels, inhomogeneous intensity of intrachannel, and spectral overlap.⁸ It has been a daunting challenge to accurately segment chromosomal images under such conditions.

A number of machine learning methods have been proposed for image segmentation or classification, e.g., supervised method,¹² semisupervised method,¹³ unsupervised method,^14–16 and hybrid method.¹⁷ Since M-FISH images are sensitive to experimental platforms and are largely heterogeneous across subjects (e.g., cells), unsupervised methods (e.g., clustering) are usually applied to image segmentation or classification. The FCM-based clustering method, as one of the unsupervised techniques, has been widely used in the fields of geology, astronomy, medical imaging, and multimedia.^18–21 To reduce the effect of noise and intensity inhomogeneities in the image, several modified FCM approaches have been proposed by considering the spatial smoothness of neighboring pixel intensities in the image.^22,23 For example, Tolias and Panas²⁴ modified the prototype vectors as functions of the pixel location in the image for segmentation. Ahmed et al.²⁵ modified FCM to use the immediate neighborhood for labeling the fuzziness membership of a pixel in the segmentation of MRI data. Chuang et al.²⁶ incorporated the spatial information into the membership function to make the clustering be less sensitive to noise, which yielded a more homogeneous segmented region. Pham and Prince²⁷ developed AFCM to improve the objective function by adding the first and second order regularizations to the estimated cluster centroids to enable the estimated membership functions to be spatially smooth. In our recent work, we developed an improved adaptive fuzzy C-means (IAFCM) algorithm by adding a gain field to regularize the cluster center and showed better performance over other methods in M-FISH image segmentation.⁹ Despite the efficiency of these works, these methods mainly focused on a single channel image clustering, while ignoring multispectral information in M-FISH image. Multichannel information is complementary, which can be used to further improve the segmentation.

Segmentation of multispectral images has been studied in several fields, such as remote sensing, monitoring, and chemometrics. Tran et.al.²⁸ presented an interesting tutorial on multispectral image segmentation. The work demonstrated the benefit of utilizing spectral information in multivariate image segmentation. As suggested by the study, partitional clustering techniques (e.g., FCM) can be the best option for segmentation given the known or easily estimated number of clusters in the image. Recently, Saïd et al.²⁹ extracted speeded-up robust features from multispectral face images to be weighted in an FCM cost function to improve the segmentation. He et al.³⁰ imposed a total variation (TV) regularization on the membership function in FCM to reduce the noise effect in hyperspectral imaging segmentation. In addition, some works have involved the incorporation of both spatial and spectral information for multispectral image segmentation. For example, Paclık et al.³¹ applied clustering to spectral and spatial information separately and combined both posterior probabilities based on the product rule. Li et al.³² also applied a two-step strategy to estimate the maximum combined posterior possibility by integrating spectral and spatial prior information.

The previous works have shown the advantages of utilizing both spatial and spectral information in image segmentation; however, little or no work has been performed for the M-FISH imaging application. In this paper, we propose an improved fuzzy C-means clustering model, namely spatial- and spectral-FCM (ssFCM), to account for both spatial and spectral information simultaneously, with the use of TV and row-wise sparse constraints, respectively. The TV constraint uses gradient information to smooth the cluster by considering that the neighboring pixels share similar membership values. The row-wise sparse constraint ensures that the membership functions of the neighboring pixels share similar patterns across different channels. This new method is expected to deliver better results because it incorporates both the spectral information across multichannels and spatial information within a neighborhood simultaneously.

The rest of the paper is organized as follows. In Sec. 2, we first review the conventional FCM clustering method and then propose an improved model by incorporating both spatial and spectral information. In Sec. 3, we present the experimental results on both simulated and real M-FISH images. Finally, we conclude the paper with a brief discussion.

2. Method

2.1. FCM Clustering Method

The FCM clustering proposed by Refs. 15 and 16 is an unsupervised learning technique to organize the data into two or more clusters. FCM clustering can be obtained by the following optimization:

$$\underset{u,c}{\min }\sum _{k=1}^{N_C}\sum _{i=1}^N{u}_{ki}^q{\Vert x_i-c_k\Vert }^2\text{subject to}\sum _{k=1}^{N_C}{u}_{ki}=1,$$ (1)

where $x_i$ is the intensity value of the $i$’th pixel from an image with size $I_r$ by $I_c$, $i=\mathrm{1,2},\dots ,N$, $N=I_r\times I_c$, and $c_k$ denotes the center of the $k$’th cluster, $k=\mathrm{1,2},\dots ,N_c$. $N_c$ is set to 2 in this work, referring to background and chromosome. $u_{ki}$ is the membership of $x_i$ in the $k$’th cluster and $q$ is the weight on each membership to determine the degree of fuzziness. A higher value of membership $u_{ki}$ indicates a higher degree of probability for the $i$’th pixel to belong to the $k$’th cluster and vice versa. Therefore, a pixel will be assigned to a particular cluster if it has the highest membership value corresponding to the cluster. The optimization problem in Eq. (1) can be solved with the Lagrangian formulation as follows:

$$J_{\mathrm{FCM}}=\sum _{k=1}^{N_C}\sum _{i=1}^N{u}_{ki}^q{\Vert x_i-c_k\Vert }^2+\lambda (\sum _{k=1}^{N_C}{u}_{ki}-1),$$ (2)

where $\lambda$ is the Lagrange multiplier. The solution of $u_{ki}$ and $c_k$ can be derived by

$$\frac{\partial J_{\mathrm{FCM}}}{\partial u_{ki}}=0,\frac{\partial J_{\mathrm{FCM}}}{\partial c_k}=0.$$

As a result, we can approximate the membership $u_{ki}$ and the cluster centers $c_k$ iteratively with

$$u_{ki}=\frac{{\Vert x_i-c_k\Vert }^{-2/(q-1)}}{\sum _{j=1}^{N_C}{\Vert x_i-c_j\Vert }^{-2/(q-1)}},c_k=\frac{\sum _{i=1}^N{u}_{ki}^q{x}_i}{\sum _{i=1}^N{u}_{ki}^q}.$$ (3)

The iteration will stop when $|u_{ki}^{(m+1)}-u_{ki}^{(m)}|<ϵ$, where $ϵ$ is a threshold for the termination and $m$ is the iteration step. Figure 2(a) shows the schematic diagram of the conventional FCM model. With FCM, each pixel (e.g., A) will be assigned a membership value, belonging to either chromosome cluster ($u_{1,A}$) or background cluster ($u_{2,A}$). By comparing the two membership matrices ${\mathbf{U}}_1$ and ${\mathbf{U}}_2$, we can therefore assign all the pixels to different clusters.

Fig. 2.

(a) The comparison of the conventional FCM model and (b) our newly proposed FCM model. In the conventional FCM model, the membership functions of pixel $A$ assigned to 2 classes in the $d$’th channel, denoted by ($u_{k,A}^{(d)}$, $k=\mathrm{1,2}$; $d=\mathrm{1,2},\dots ,6$), are estimated individually. However, in the proposed model, the membership functions of neighboring pixels (i.e., B-E), denoted by $u_{k,B}^{(d)},u_{k,C}^{(d)},\dots u_{k,E}^{(d)}$ are also accounted for, estimating $u_{k,A}^{(d)}$ in the $d$’th channel. TVs of membership $u_{k,A}^{(d)}$ with each of other neighboring points (e.g., ${\mathrm{TV}}_{A1,B1}$) are used as the constraint in the model to account for spatial neighborhood information.

2.2. Proposed FCM Clustering Algorithm Incorporating Both Spatial and Spectral Information

Despite the success of FCM clustering-based methods in imaging segmentation or classification, many of them overlook the spatial and/or spectral information among pixels in multidimensional images (e.g., M-FISH image). Both spatial and spectral information have been widely used for noise reduction,³³ hyperspectral or color imaging classification and segmentation,^32,34 and magnetic resonance imaging segmentation.^30,35–37 Instead of analyzing each pixel individually, spatially or spectrally neighboring pixels can be used together to improve image analysis because these neighboring pixels will share similar characteristics, e.g., having similar intensities. Therefore, it is promising to incorporate this information into the FCM model.

Specifically, our newly proposed FCM clustering model is formulated as follows:

$$\begin{gathered} \underset{\mathbf{U},\mathbf{C}}{\min }\sum _{d=1}^M\sum _{k=1}^{N_c}\sum _{i=1}^N[{(U_{i,k}^{(d)})}^q{\Vert x_i^{(d)}-c_k^{(d)}\Vert }^2]+\lambda {\Vert \mathrm{\varphi }(\mathbf{D},\mathbf{U})\Vert }_{2,1} \\ \text{subject to}\sum _{k=1}^{N_c}{U}_{i,k}^{(d)}=1,i=\mathrm{1,2},\dots ,N;d=\mathrm{1,2},\dots ,M, \end{gathered}$$ (4)

where $\lambda$ is the tuning parameter. $M$ denotes the number of channels, which equals 6 for M-FISH images. The function $\mathrm{\varphi }(\mathbf{D},\mathbf{U})=\mathbf{DU}$ is regularized by the row-wise sparsity. $\mathbf{U}=[U_1^{(1)},U_2^{(1)},\cdots ,U_{Nc}^{(1)},\cdots ,U_k^{(d)},\cdots ,U_1^{(M)},U_2^{(M)},\cdots ,U_{N_c}^{(M)}]\in R^{N\times P}$ is the membership matrix with $P=Nc\times M$. $U_{k,i}^{(d)}$ is the membership value indicating the probability that the $i$’th pixel in the $d$’th channel image is assigned to the $k$’th cluster, and each column $U_k^{(d)}=[U_{i,k}^{(d)},i=\mathrm{1,2},\cdots ,N]\in R^{N\times 1}$ indicates the membership values for all pixels in the $d$’th channel image to be assigned to the $k$’th cluster.

$\mathbf{D}$ consists of difference operators to calculate the membership difference of neighboring pixels in $\mathbf{U}$ horizontally, vertically, and diagonally. Let $\mathbf{B}$ denote the $I_c\times I_c$ matrix defined by $\mathbf{B}=\left[\begin{array}{cccc}0& & & \\ -1& 1& & \\ & \ddots & \ddots & \\ & & -1& 1\end{array}\right]$ and choose $D_h=[\mathbf{B}\otimes \mathbf{I}]$, where $\mathbf{I}$ is the $I_r\times I_r$ identity matrix and the operator $\otimes$ denotes the Kronecker tenser product of matrices $\mathbf{B}$ and $\mathbf{I}$. $I_r$ and $I_c$ indicate the number of rows and columns, respectively, of the image matrix in each channel with $N=I_r\times I_c$. ${\mathbf{D}}_h$ is the difference operator to compute the difference of membership between a pixel and its neighboring pixels horizontally in the image. Similarly, let ${\mathbf{D}}_v$ and ${\mathbf{D}}_d$ be the difference operators in the vertical and diagonal directions, respectively. We can combine three difference operators as $\mathbf{DU}={[{\mathbf{D}}_h\mathbf{U},{\mathbf{D}}_v\mathbf{U},{\mathbf{D}}_d\mathbf{U}]}^T$, where each row in $\mathbf{DU}$ represents interchannel membership values. These difference operators are actually used to define row-wise anisotropic TV constraints.^38,39 The use of the TV can remove unwanted variations among neighboring pixels while preserving important details such as edges.⁴⁰ To incorporate spectral information, we enforce a row-wise sparse penalty ${\Vert ·\Vert }_{m,n}$ on matrix $\mathrm{\varphi }(\mathbf{D},\mathbf{U})$. Row-wise sparse penalization on a matrix $\mathbf{X}\in R^{r\times c}$ is defined by ${\Vert \mathbf{X}\Vert }_{m,n}={\{\sum _{i=1}^r{[{(\sum _{j=1}^c{|x_{ij}|}^n)}^{1/n}]}^m\}}^{1/m}$, where $\mathbf{X}$ is an $r\times c$ matrix and $m$ and $n$ are parameters for defining $L_{m,n}$ norm of matrix $\mathbf{X}$. In the ssFCM model, we use $L_{\mathrm{2,1}}$ norm ($m=2$, $n=1$), which is the sum of the Euclidean norms of the rows of the matrix. $L_{\mathrm{2,1}}$ norm is nondifferentiable on each row of matrix only when all elements in the row are zeros, resulting in a row-sparse solution matrix (the element values in the majority of rows are zeros), as introduced in Refs. 41 and 42. Combined with TV to calculate the membership difference of neighboring pixels in each channel, the additional row-sparse penalty is imposed on the membership difference across all channels, enforcing that the membership differences among neighboring pixels are close to zeros across spectral channels.

The proposed ssFCM model is illustrated in Fig. 2(b). The input is a multichannel image matrix with each image collected from one channel, e.g., six images in the M-FISH collected from six channels, respectively. For each pixel in one-channel image (e.g., pixel $A^{(1)}$ in the first channel), we will explore the spatial neighboring pixels of $A^{(1)}$ in the same channel (e.g., pixels $B^{(1)}\sim E^{(1)}$,) and the spectrally neighboring pixels from the other five channels (e.g., $A^{(i)}\sim E^{(i)}$, $i=2,\dots ,6$). Each pixel in each channel image is given a membership value to the chromosome (i.e., $U_{1,x}^{(i)}$) and to the background cluster (i.e., $U_{2,x}^{(i)}$), respectively, where $x$ indicates the pixel in each image (e.g., $x=A,B,\dots ,E$) and $i$ is the index of the channel (i.e., $i=\mathrm{1,2},\dots ,6$). Totally, we have 12 membership matrices to be estimated, which is combined as $\mathbf{U}=[U_1^{(1)},U_2^{(1)},U_1^{(2)},U_2^{(2)},\cdots ,U_1^{(6)},U_2^{(6)}]\in R^{N\times 12}$, where $N$ is the number of pixels in each channel image. As discussed before, we enforce a TV combined with a row-wise sparse constraint on the membership matrix $\mathbf{U}$ to utilize both spatial and spectral information. This way we will ensure that (1) the neighboring pixels will have similar membership functions and (2) the clustering patterns of neighboring pixels (e.g., A,…,E) will be similar across different channels. The constraints will have the potential to reduce noise using both spatial and spectral information of each pixel.

The alternating direction method of multipliers (ADMM)⁴³ can be used to solve Eq. (4). Specifically, we solve the following optimization problem:

$$\begin{gathered} \underset{\mathbf{U},\mathbf{C},\mathbf{V}}{\min }\sum _{d=1}^M\sum _{k=1}^{N_c}\sum _{i=1}^N[{(U_{i,k}^{(d)})}^q{\Vert x_i^{(d)}-c_k^{(d)}\Vert }^2]+\lambda {\Vert \mathbf{V}\Vert }_{\mathrm{2,1}} \\ \text{subject to}:\mathbf{V}=\mathbf{DU}; \\ \sum _{k=1}^{N_c}{U}_{i,k}^{(d)}=1,i=\mathrm{1,2},\dots ,N;d=\mathrm{1,2},\dots ,M, \end{gathered}$$ (5)

which can be solved with the following cost function:

$$L_{\rho }(\mathbf{U},\mathbf{V},z)=\sum _{d=1}^M\sum _{k=1}^{N_c}\sum _{i=1}^N[{(U_{i,k}^{(d)})}^q{\Vert x_i^{(d)}-c_k^{(d)}\Vert }^2]+\sum _{i,d}{\lambda }_1^{(i,d)}(\sum _{k=1}^{N_c}{U}_{i,k}^{(d)}-1)+\lambda {\Vert \mathbf{V}\Vert }_{\mathrm{2,1}}+\frac{\rho }{2}{\Vert \mathrm{vec}(\mathbf{V})-({\mathbf{I}}_{K\times K}\otimes \mathbf{D})\mathrm{vec}(\mathbf{U})+\mathbf{Z}\Vert }_2^2,$$ (6)

where ${\lambda }_1^{(i,d)}$ is the Lagrange multiplier and $\lambda$ is the parameter for controlling the row sparsity in matrix $\mathbf{V}$, i.e., the proportion of nonzero rows. The fuzziness $q$ is set to 2. $\mathrm{vec}(\mathbf{DU})=\mathrm{vec}(\mathbf{DUI})=({\mathbf{I}}_{\mathbf{K}\times \mathbf{K}}\otimes \mathbf{D})\mathrm{vec}(\mathbf{U})$, where ${\mathbf{I}}_{K\times K}$ is an identity matrix and $\mathrm{vec}(·)$ is an operator to reformulate a matrix to a vector by columns; $\rho >0$ is the augmented Lagrangian parameter to approximate $\mathbf{D}$, $\mathbf{U}$ by $\mathbf{V}$; and $\mathbf{Z}$ is the scaled dual variable. The parameter $\rho$ is updated by iterations, as suggested in Ref. 43, to improve the convergence and make the solution be less dependent on the initial value. The solution of Eq. (6) is derived by

$$\frac{\partial L_{\rho }}{\partial c_k^{(d)}}=0\Rightarrow c_k^{(d)}=\frac{{\sum }_{i=1}^N{(U_{i,k}^{(d)})}^q{x}_i^{(d)}}{{\sum }_{i=1}^N{(U_{i,k}^{(d)})}^q},$$ (7)

$$\frac{\partial L_p}{\partial U}=0\Rightarrow {[\mathrm{vec}(U)]}_j=\frac{\rho \times {\{{({\mathbf{I}}_{K\times K}\otimes \mathbf{D})}^T[\mathrm{vec}(\mathbf{V})+\mathbf{Z}]\}}_j-{\sum }_i({\nabla }_{i,i\ne j}{U}_{i\ne j})-{\lambda }_1^{(i,d)}}{2Q_j+\rho {\nabla }_{j,j}},$$ (8)

where $Q_j=Q_{i,k}^{(d)}={\Vert x_i^{(d)}-c_k^{(d)}\Vert }^2$, $U_i=U_{i,k}^{(d)},k=1,\dots ,N_c;d=\mathrm{1,2},\dots ,M$, $\nabla ={({\mathbf{I}}_{K\times K}\otimes \mathbf{D})}^T({\mathbf{I}}_{K\times K}\otimes \mathbf{D})$. In Eq. (8), ${\lambda }_1^{(i,d)}$ will be solved by the constraint $\sum _{k=1}^{N_c}{U}_{k,i}^{(d)}=1$, as in Eq. (3). Table 1 shows the algorithm of solving Eq. (6) via ADMM. It uses a decomposition-coordination procedure, and the solutions to small local subproblems are coordinated to find the solution to a large global problem. In the algorithm, at each iteration, the parameters $\mathbf{U}$, $\mathbf{V}$, $\mathbf{C}$, and $\mathbf{Z}$ are updated sequentially with other parameters fixed. This iteration will stop until a stop criterion is satisfied.

Table 1.

The application of the ADMM algorithm to the proposed FCM clustering model.

Algorithm: the solution of the proposed FCM clustering via ADMM algorithm
Input: the original image $\mathbf{X}$, linear operator $\mathbf{D}$, $q=2$, $n=0$
Output: Matrix $\mathbf{U}$ of membership functions, matrix $\mathbf{C}$ of cluster centers
Initialization: $U(0)=0$, $V(0)=0$, $Z(0)=0$
While stopping criteria false do
Update center $\mathbf{C}$: $C(n+1)\leftarrow \arg \min L_{\rho }[U(n)]$
Update $\mathbf{U}$: $U(n+1)\leftarrow \arg \min L_{\rho }[C(n+1),U(n),V(n),Z(n)]$
Update $\mathbf{V}$: $V(n+1)\leftarrow \arg \min [\lambda {\Vert V(n)\Vert }_{\mathrm{2,1}}+\rho /2{\Vert V(n)-\mathbf{D}\mathbf{U}(n+1)+Z(n)\Vert }_2^2]$
Update $\mathbf{Z}$: $Z(n+1)\leftarrow V(n+1)-\mathbf{D}\mathbf{U}(n+1)+Z(n)$
End while
Return:$\mathbf{U}$ and $\mathbf{C}$

3. Application of the Proposed FCM to both Simulated and Real M-FISH Images

3.1. Results on Synthetic Image Sets

Two three-channel synthetic image sets with $128\times 128\text{pixels}$, which have the intensity values of 0 and 100, were generated. The first image set (Fig. 3) consists of four squares belonging to three different classes: the first class contains the pixels with an intensity value of 100 in the first channel and 0 in the second channel; the second class consists of pixels with an intensity of 0 in the first channel and 100 in the second channel; the remaining pixels (background pixels) are clustered as the third class with an intensity 0 in both channels. The third channel (DAPI channel) has the intensity value of 100 for the pixels from the first two classes and 0 for background pixels. The ground truth image is used to evaluate the performance. The Gaussian noise with a standard deviation $\sigma =40$ was added to the three channels, as shown in Figs. 3(a)–3(c). The FCM, IAFCM, our proposed ssFCM-DAPI only (i.e., only the DAPI channel is segmented by ssFCM model), and ssFCM model (i.e., all channels are used for segmentation) were then applied, respectively. All programs were run on a Linux system-based CCS cluster at Tulane,⁴⁴ and the average computational time was around 200 s. The results of segmentation using four different models were shown in Figs. 3(d)–3(g), respectively. In addition, they were further assessed by the following metrics as used in our previous work:⁹

$$\begin{gathered} \text{Correct ratio}=\frac{\#\text{of pixels in interested areas being correctly classified}}{\#\text{of pixels in interested areas}}, \\ \text{False ratio}=\frac{\#\text{of pixels in background being classified as interested areas}}{\#\text{of pixels in interested areas}}. \end{gathered}$$

Fig. 3.

The segmentation of the synthetic image set consisting of four squares with Gaussian noise ($\sigma =40$) added. (a)–(c) show the images from the first, second, and third channels, respectively. (d)–(g) are the segmentation results by FCM model, IAFCM, ssFCM-DAPI only model, and ssFCM model, respectively.

The proposed ssFCM model shows higher correct ratio $\mathrm{CR}=1$ with lower false ratio $\mathrm{FR}=0$ and the IAFCM model shows lower $\mathrm{CR}=0.7057$ with higher $\mathrm{FR}=0.4351$ compared with the other methods. The comparison of four methods in the segmentation of synthetic images with different noise levels is listed in Table 2. The results demonstrate that the ssFCM model is more accurate and more robust to noise than the other three models.

Table 2.

A comparison of the segmentation in terms of CR/FR Using FCM, IAFCM, ssFCM-DAPI only and the ssFCM models on the first synthetic image set with three different noise levels.

Noise levels	FCM(CR/FR)	IAFCM(CR/FR)	ssFCM-DAPI only (CR/FR)	ssFCM (CR/FR)
$\sigma =20$	0.9933/0.006	0.9711/0.2913	1/0.0002	1/0
$\sigma =40$	0.8975/0.1075	0.7057/0.4351	1/0.0008	1/0
$\sigma =60$	0.7927/0.2047	0.6335/0.4652	0.9950/0.0088	0.9974/0.0026

The second synthetic image set includes two annuluses with the same center, including three different classes: small annulus, big annulus, and background. The first two channels contain a small and big annulus, respectively, while the third channel contains both of them. Figures 4(a)–4(c) show the second synthetic image set with Gaussian noise. The four models (i.e., FCM, IAFCM, ssFCM-DAPI only, ssFCM) were then applied to the image set with noise level (e.g., $\sigma =40$), and the segmentation results are presented in Figs. 4(d)–4(g), compared with the ground truth image. In Fig. 4, the segmentation of the proposed ssFCM is better than the three others with the $\mathrm{CR}=0.9974$ and $\mathrm{FR}=0.0033$. Table 3 summarized the segmentation results of FCM, IAFCM, ssFCM-DAPI only and the ssFCM models in terms of CR and FR. The results show that the performance of the ssFCM model is better than the others with higher CR and lower FR at all noise levels.

Fig. 4.

The segmentation of the synthetic image set consisting of two annuluses with Gaussian noise ($\sigma =40$). (a)–(c) The images from the first, second, and third channels, respectively. (d)–(g) The segmentation results using FCM model, IAFCM, ssFCM-DAPI only model, and ssFCM model, respectively.

Table 3.

A comparison of the segmentation of FCM, IAFCM, ssFCM-DAPI only, and the ssFCM models on the second synthetic image set consisting of two annuluses with three different noise levels.

Noise levels	FCM (CR/FR)	IAFCM (CR/FR)	ssFCM-DAPI only (CR/FR)	ssFCM (CR/FR)
$\sigma =20$	0.994/0.0047	0.995/0.0076	1/0.0002	1/0
$\sigma =40$	0.9239/0.1464	0.96147/0.3028	0.9944/0.0047	0.9974/0.0033
$\sigma =60$	0.8512/0.2777	0.8341/0.4088	0.9551/0.0939	0.9896/0.0324

3.2. Results with Hybrid Simulation Images

First, two hybrid simulation datasets were generated from real images. The simulated image sets correspond to five different classes and four different channels. The first three channels contain chromosomes partially, while the fourth channel is the DAPI image showing all chromosomes. The ground truth image is also simulated, where each pixel in the chromosome is labeled correctly, which is used to verify the accuracy of image segmentation. For simulated dataset I, the background is labeled 0 and the chromosomal region is labeled 255; for simulated dataset II, the background is labeled 0 and the intensity level of the chromosomal region is the same as the real chromosomal image. To test the robustness of different models, Gaussian noise with 30 different levels ($\sigma$ is from 0 to 150) is added. Figure 5 shows the simulated image dataset I mixed with Gaussian noise, and the segmentation results of different FCM models. Figures 5(a)–5(d) are the first, second, third, and fourth channel of simulated image dataset I, respectively. Figures 5(i)–5(l) are the first, second, third, and fourth channel of simulated image dataset II, respectively. Figures 5(e)–5(h) are the segmentation results of applying FCM model, IAFCM model, ssFCM-DAPI only, and ssFCM models to simulated image dataset I, respectively, and Figs. 5(m)–5(p) are the segmentation results of applying FCM model, IAFCM model, ssFCM-DAPI only, and ssFCM models to simulated image dataset II, respectively. The results show that the proposed ssFCM model gives the highest segmentation accuracy, which demonstrates better robustness to the noise than the other two models.

Fig. 5.

The simulated image datasets I and II and segmentation results of different FCM models. (a)–(d) The images from the first to the fourth channel in simulated image set I, respectively. (e)–(h) The segmentation results of simulated image dataset I using FCM model, IAFCM, ssFCM-DAPI only model, and ssFCM model, respectively. (i)–(l) The images from the first channel to the fourth channel in simulated image set II, respectively. (m)–(p) The segmentation results of simulated image dataset II using FCM model, IAFCM, ssFCM-DAPI only model, and ssFCM model, respectively.

Figure 6 compares the segmentation results of four FCM-based models with different levels of noise, in terms of CR and FR. The noise follows the normal distribution with 30 levels ($\sigma$ varying from 0 to 150). In Fig. 6, FCM and IAFCM were tested only on the DAPI channel as we did in Ref. 6. To evaluate the benefit of using spectral information, we also applied our FCM method to all six channels (ssFCM). The results show that our ssFCM models (ssFCM-DAPI only and ssFCM) largely outperform the other two FCM methods with higher CR and much lower FR, especially when the noise level is higher (e.g., $\sigma >50$), which indicates that our methods are more robust than the other two FCM methods. For the ssFCM and ssFCM-DAPI only, when the noise level $\sigma <75$, both have no significant difference. However, as the noise level grows higher, ssFCM has higher CR than ssFCM-DAPI only, while FR is still similar in both methods, which shows the benefit of incorporating multichannel information to reduce the noise effect.

Fig. 6.

The comparisons of (a) CR and (b) FR using different models on the simulated image set I by changing the noise level from 0 to 30 (corresponding $\sigma$ is from 0 to 150).

In simulation set II, the images are taken the same as the real chromosomal images. Figure 7 shows the simulated image dataset II mixed with Gaussian noise ($\sigma$ is from 0 to 150) and the segmentation results of different FCM models. Figures 5(i)–5(l) are the top four channels in simulated image dataset II. Figures 5(m)–5(p) show the segmentation results of applying the FCM model, IAFCM model, ssFCM-DAPI only and ssFCM models to the simulated image dataset II, respectively. The same levels of noise as in set I are added to test model robustness, as shown in Fig. 7. It can be seen that the results are similar to those in set I, and our ssFCM model generally gives the highest CR and lowest FR compared with the other models. Therefore, both simulations indicate that ssFCM method can take advantage of both spatial information to improve segmentation accuracy and multichannel spectral information to reduce the noise effect.

Fig. 7.

The comparisons of (a) CR and (b) FR using different segmentation models on the simulated image set II by changing the noise level from 0 to 30 (corresponding $\sigma$ is from 0 to 150).

3.3. Results with Real M-FISH Images

Besides the simulated datasets, we also applied the proposed FCM model to real M-FISH images with six different channels from Advanced Digital Imaging Research (League City, Texas).⁴⁵ The images of first five channels were labeled with different fluorophores, and the sixth channel was the DAPI image. The ground truth image was also provided, which was labeled by an experienced cytogeneticist. The results of testing on real M-FISH images are shown in Fig. 8. It shows that the CR of the proposed ssFCM models has the highest value at almost all noise levels ($\sigma$ from 0 to 150). As the noise level increases, the proposed ssFCM models deliver higher CR value and lower FR value than that of other models. Therefore, the newly proposed FCM model is more robust to the noise than the other two models.

Fig. 8.

The comparison results of (a) CR and (b) FR using different segmentation models on the real M-FSIH images by changing the noise level from 0 to 30 (corresponding $\sigma$ is from 0 to 150).

All models have been applied to the images of whole cells for further comparison. Figure 9 shows an example of M-FISH dataset and the segmentation results as well as the ground truth. It can be seen that there is more noise around edges of chromosomes with the conventional FCM model and even more noise on the background with the IAFCM model than our ssFCM-based models (ssFCM-DAPI only and ssFCM models). Furthermore, the differences between the ground truth and the results of three models are shown in Fig. 10, where the blue, red, and cyan indicate the pixels correctly segmented into chromosomes and background, background pixels wrongly segmented into chromosomes, and chromosome pixels wrongly segmented into background, respectively. In addition, Fig. 11 shows a quantitative analysis of the segmentation results in terms of CR and FR with 15 levels of noise ($\sigma$ from 0 to 150). It is worth noting that the CRs of ssFCM-based models are generally comparable and higher than those by FCM and IAFCM models. It is also shown that FRs of ssFCM-based models are lower than the other two models under high noise levels. Overall, the results indicate that the proposed ssFCM models have improved accuracy and more robustness than FCM and IAFCM models because both spatial and spectral information is incorporated into the model.

Fig. 9.

An example of M-FISH dataset and the comparison of segmentation results by different FCM methods. (a)–(f) Six channels of M-FISH dataset, (g) the ground truth used for model evaluation, (h)–(k) the segmentation results of FCM model, IAFCM, ssFCM-DAPI only model, and ssFCM model, respectively.

Fig. 10.

(a) The difference between the ground truth and the segmentation results by FCM, (b) IAFCM, (c) ssFCM-DAPI only model, and (d) ssFCM models, respectively, where the blue, red, and cyan indicate the pixels correctly segmented into chromosomes and background, background pixels wrongly segmented into chromosomes, and chromosome pixels wrongly segmented into background, respectively.

Fig. 11.

The segmentation results [mean values of (a) CR and (b) FR] on six cells with 15 noise levels (corresponding σ is from 0 to 150) using different FCM methods.

4. Conclusion and Discussion

In this paper, we propose an improved fuzzy C-means clustering algorithm for chromosome segmentation, namely ssFCM, by taking advantage of both multichannel spectral information of M-FISH imaging and the spatial information within neighboring pixels in each channel. The model is designed by imposing both TV and row-wise sparse constraints on the membership matrix in the FCM model. The model is then applied to the segmentation of chromosome images from the M-FISH image set. To evaluate the performance, we compare the proposed ssFCM models with the conventional FCM and an IAFCM on synthetic, hybrid, and real M-FISH images, respectively. Table 4 summarizes all tested images and the quantitative performance using different models. The segmentation results shows that (1) the proposed FCM model outperforms the other two models (e.g., FCM and IAFCM) in terms of both CR and FR on the synthetic and hybrid image sets and (2) the proposed model is generally better than the other two models, e.g., having lower FR and being more robust to the noise on real M-FISH images. Nevertheless, further work can be conducted by modeling noises with Gaussian mixture distribution. Also, more datasets should be tested to show the reproducibility of the ssFCM model. More accurate segmentation of M-FISH images can increase subsequent classification of chromosomes and the detection of chromosomal abnormalities, which will, in turn, translate into improved diagnosis of genetic diseases and cancers.

Table 4.

The summary of testing on hybrid simulation and real M-FISH images and quantitative performance using different models.

Image sets	Noise level	Method	CR ($\text{mean}\pm \mathrm{Std}$)	FR ($\text{mean}\pm \mathrm{Std}$)
Simulation set I (four images)	30	FCM	$0.8814\pm 0.1245$	$0.1427\pm 0.1332$
		IAFCM	$0.9184\pm 0.1005$	$0.3235\pm 0.0876$
		ssFCM-DAPI only	$0.9012\pm 0.1168$	$0.1053\pm 0.1246$
		ssFCM	$0.9203\pm 0.1096$	$0.0977\pm 0.1235$
Simulation set II (four images)	30	FCM	$0.8813\pm 0.1239$	$0.1413\pm 0.1339$
		IAFCM	$0.9173\pm 0.1011$	$0.3015\pm 0.1068$
		ssFCM-DAPI only	$0.9090\pm 0.1126$	$0.0968\pm 0.1212$
		ssFCM	$0.9169\pm 0.1097$	$0.0909\pm 0.1197$
Real M-FISH images (36 images)	15	FCM	$0.9210\pm 0.0457$	$0.1389\pm 0.1899$
		IAFCM	$0.8730\pm 0.2438$	$0.2438\pm 0.2438$
		ssFCM-DAPI only	$0.9368\pm 0.0242$	$0.0919\pm 0.0250$
		ssFCM	$0.9382\pm 0.0250$	$0.1042\pm 0.1481$

Biographies

Jingyao Li received her BS degree in measurement and control technology and instrumentation in 2006 from Hebei University of Technology, China, her MS degree in biomedical engineering in 2009 from Tianjin University, China, and her PhD in biomedical engineering from Tulane University United States, in 2015. Her current research interests include medical image analysis, sparse representation modeling, and machine learning.

Dongdong Lin received his BS degree in biomedical engineering from Chongqing University in China in 2007 and his PhD in biomedical engineering from Tulane University, in 2015. He is currently a postdoctoral researcher in the Mind Research Network Institute. His primary research is on mathematical models for integrative analysis of high-dimensional biological data and multimodal medical imaging to deepen the insights of the pathophysiology underlying complex disorders.

Yu-Ping Wang received his BS degree in applied mathematics from Tianjin University, China, in 1990, and his MS degree in computational mathematics and PhD in communications and electronic systems from Xi’an Jiaotong University, China, in 1993 and 1996, respectively. He is currently a professor in the Department of Biomedical Engineering and Global Biostatistics and Data Sciences at Tulane University. His research interests include signal processing and machine learning with applications to biomedical imaging and bioinformatics.

Disclosures

The authors declare that they have no competing interests.