Analysis of Cellular Feature Differences of Astrocytomas with Distinct Mutational Profiles Using Digitized Histopathology Images

Abstract

Cellular phenotypic features derived from histopathology images are the basis of pathologic diagnosis and are thought to be related to underlying molecular profiles. Due to overwhelming cell numbers and population heterogeneity, it remains challenging to quantitatively compute and compare features of cells with distinct molecular signatures. In this study, we propose a self-reliant and efficient analysis framework that supports quantitative analysis of cellular phenotypic difference across distinct molecular groups. To demonstrate efficacy, we quantitatively analyze astrocytomas that are molecularly characterized as either Isocitrate Dehydrogenase (IDH) mutant (MUT) or wildtype (WT) using imaging data from The Cancer Genome Atlas database. Representative cell instances that are phenotypically different between these two groups are retrieved after segmentation, feature computation, data pruning, dimensionality reduction, and unsupervised clustering. Our analysis is generic and can be applied to a wide set of cell-based biomedical research.

I. INTRODUCTION

Large-scale microscopic pathology images have been used in the practice of diagnostic pathology and understanding of disease mechanisms. Cells are fundamental pathology objects, as they capture rich information on disease characteristics that have been used for diagnostic purposes for over a century. Cell size and shape are controlled by many factors including the genomic, biochemical, and metabolic status, signaling networks engaged, physical properties of the plasma membrane, and underlying cytoskeletal properties [1]. Therefore, quantitative morphometric analysis may provide information on the genetic or molecular properties of cell populations. Although the prognostic significance of subjective cell features are well known in some diseases, it is often challenging to manually segment and quantitatively analyze cell population features due to their overwhelmingly large numbers in histologic sections. As a result, numerous automated image analysis methods for cell analytics have been proposed [2], [3], [4]. However, there is a lack of complete, self-reliant, modularized processing functions to facilitate the study of cellular phenotypic differences in distinct disease states. To address this, we present a complete and automated cellular feature analysis framework that includes cell segmentation, feature computation, and feature analytics for distinguishing cell populations with distinct molecular signatures.

As a driving use case for our study, we quantitatively analyze tumor cells from Grade III astrocytomas that are molecularly characterized as either Isocitrate Dehydrogenase (IDH)-mutant (MUT) or wildtype (WT). IDH mutations in infiltrating gliomas identify biologically distinct disease subsets with substantially younger age at presentation, slower clinical progression and longer overall survival compared to those that are IDH wildtype. IDH mutations represent an early, and likely initiating event that is present in more than 80% of grades II and II astrocytomas and secondary glioblastomas [5]. These genetically and clinically distinct forms of astrocytoma were only recently recognized and their morphologic differences have not been described, providing an excellent test case for framework development.

II. METHOD

Our cell analysis workflow consists of a sequence of steps, including tumor region annotation, tumor region image extraction, image stain normalization, stain color deconvolution, cell segmentation, cellular feature computation, and feature comparison analysis with representative cell retrieval.

A. Tumor Cell Segmentation

The overall schema for cell segmentation is presented in Figure 1. As not all tissue regions in a whole-slide microscopy image are tumor regions of interest, neuropathologists manually select Regions of Interest (ROIs) from each slide [6]. The resulting annotation results are captured and exported in xml files. Next, our analysis module programmatically reads xml files and uses openSlide API to retrieve the annotated image regions from whole-slide images [7].

Fig. 1:

Overall schema of cell analysis is presented.

To begin the cell segmentation analysis, we normalize colors of all images to mitigate the impact of stain variation on the follow-up segmentation. Aiming for color channel correlation minimization, we convert each image from the RGB to LAB color space where each channel is mapped by mean and standard deviation of the corresponding image channel from target image, respectively [8]. The transformed color channels are then mapped back to RGB space [9]. All slides are stained by Hematoxylin and Eosin (H&E) stain, resulting in regions of nuclei and cytoplasms in purple and pink. To target signals useful for segmentation, we decouple these two distinct stains from the original color image by Lambert-Beer’s law [10]. As nuclei in cells are highlighted by Hematoxylin, we next process the deconvolved Hematoxylin image channel for further segmentation.

We explore the resulting Hematoxylin image channel and find that pixels tend to have gradually decreased Hematoxylin stain signal intensity as they approach to nuclear centroid. To enhance signal contrast for more accurate segmentation, we determine the likelihood of a given pixel in cell regions by analyzing eigenvalues of the Hessian matrix from local image neighbors [11]. Given the prior knowledge about cell shape, we search for circular structures in Hematoxylin channel and compute likelihood for cellular pixels based on geometric structures characterized by the neighboring pixel intensity profiles. For any arbitrary pixel at (x₀, y₀), its local image intensity change can be represented by Taylor expansion:

$$\begin{gathered} h_{\sigma }(x_0+\delta x,y_0+\delta y)=h_{\sigma }(x_0,y_0)+(\delta x,\delta y)D(h_{\sigma }){\mid }_{(x_0,y_0)} \\ +(\delta x,\delta y)D^2(h_{\sigma }){\mid }_{(x_0,y_0)}(\delta x,\delta y{)}^T+\mathcal{O}(({\delta }^3) \end{gathered}$$ (1)

where h_σ is the convolution of the deconvolved Hematoxylin image channel h and a Gaussian filter G_σ with standard deviation σ; D² (h) is Hessian of h that is symmetric and thus diagonalizable with two resulting eigenvalues λ_i, i = 1, 2. Due to the Hematoxylin channel property, it is straightforward to have 0 ≪ λ₁ ⪅ λ₂ for pixels within cells. To improve the intensity contrast between cells and background, we use the following cell enhancement function [11]: $f(h_{\sigma }(x,y),\alpha ,\beta )=\left(1-exp\left(-\frac{({\lambda }_1∕{\lambda }_2{)}^2}{2{\alpha }^2}\right)\right)\left(1-exp\left(-\frac{{\lambda }_1^2+{\lambda }_2^2}{2{\beta }^2}\right)\right)$ where α and β are sensitivity parameters for two product terms. As cell size is variable, we take the maximum response with the optimal scale σ*. The resulting enhanced likelihood map is further processed with hysteresis thresholding [12]. In the postprocessing step, we exclude the resulting candidate either too small or too bright to be a cell. For those candidates with internal holes, we fill up holes. The cellular boundary is smoothed by a low pass filter in the end.

B. Feature Computation

To remove all erroneous results generated by machine analysis, the resulting cellular boundaries are manually corrected by domain experts before they are used for feature computation. Manual corrections included annotations for mitotic figures, apoptotic nuclei, corpora amylacea, neurons, and endothelial cells. Abnormally segmented cells, such as those clumped cells and partial cells on image edges, are removed. After this validation and correction process by human experts, we proceed with cellular feature analysis by computing quantitative features related to cell size, shape, intensity, and hyper-chromaticity. Specifically, for each segmented cell, Area, Perimeter, Max Distance, and Equivalent Diameter are computed to represent cell size; Eccentricity, Circularity, and Extent are derived to describe cell shape; Intensity Standard Deviation, and Intensity Entropy are used to characterize Hyper-chromaticity; Intensity Mean is used to represent cell pixel intensity.

C. Feature Analysis

To visually perceive cellular feature difference, we project data to a lower dimensional subspace with Multi-Dimensional Scaling (MDS) [13] after feature normalization.

MDS is a non-linear dimensionality reduction method that detects the underlying dimensions of data by finding the similarity or dissimilarity between pairs of data. Given the dissimilarity matrix Δ = [∥y_i − y_j ∥ = δ_i,j] derived from the original feature space with dimensionality of p, our goal is to find $\{x_i\in {\mathbb{R}}^d\}$ with d < p such that ∥x_i − x_j∥ ≈ δ_{i, j}. $x_i\in {\mathbb{R}}^d$ is the low dimensional data representation of $y_i\in {\mathbb{R}}^p$ in a higher dimensional space. MDS generalizes the optimization method by minimizing the cost function: $S=\sum _{i\ne j}({\delta }_{i,j}-\Vert x_i-x_j\Vert {)}^2)$. For visual assessment, we apply MDS and project data to a lower space (i.e. d = 3) with MDS and present data in Figure 2 (Left) where data from MUT and WT are observed to be severely overlapped. As there is a large number of cells from multiple cell populations with large phenotypic and biological variation in each image, severe noise is embedded in data representation as suggested by the nonlinear dimensionality reduction method MDS. Therefore, it is more promising to find true cell phenotypic feature signatures by data pruning than exhaustively formulating additional cell feature representations in this scenario. Following this idea, we measure discriminating power of cells in an ensemble classification manner and robustly find representative instances distinctive across two groups.

Fig. 2:

MDS derived scatter plots (Left) before and (Right) after data pruning are demonstrated.

1). Data Pruning:

We prune data with a mixed collection of generative and discriminative classifiers at this stage: Logistic Regression (LR), Random Forest (RF), AdaBoost (AB), Naive Bayes (NB), Quadratic Discriminant Analysis (QDA) and Neural Net (NN) [15], [18]. Generative models are used to model the distribution of individual class. These methods derive the posterior probability on class label y given observation x using Bayes rule after modeling p(y) and $p(x\mid y):\underset{y}{\text{arg max}}p(y\mid x)=\underset{x}{\text{arg max}}p(x\mid y)p(y)$. By contrast,

Algorithm 1: Algorithm for representative cell retrieval

$$\begin{gathered} \begin{array}{cc}\hfill & \mathrm{Input}:D:datafeaturematrix;\text{N: number of rows in}D;F\leftarrow 9;C:\text{cluster}\hfill \\ \hfill & \text{number}\hfill \\ \hfill & \mathrm{Output}:P_i:cellpanelforclusteriofeachgroup\hfill \\ \hfill & \hfill \\ \hfill & \\ 1:\mathbf{for}\mathbf{all}\text{groups}G\in (0,1)\mathbf{do}\hfill \\ \hfill & \\ 2:D_0\leftarrow D[G];\text{count}\leftarrow 0\hfill \\ \hfill & \\ 3:\mathbf{for}\mathbf{all}c\in (0,1,\dots ,C)\mathbf{do}\hfill \\ \hfill & \\ 4:\mathbf{for}\mathbf{all}i\in (0,1,\dots ,N-1)\mathbf{do}\hfill \\ \hfill & \\ 5:\text{count}[c]\leftarrow \text{count}[c]+1\hfill \\ \hfill & \\ 6:\overline{c}\leftarrow avg(D_0,\text{count})∕∕\overline{c}:\text{cluster centroid}\hfill \\ \hfill & \\ 7:\mathbf{foa}\mathbf{all}i\in (0,1,\dots ,N-1)\mathbf{do}\hfill \\ \hfill & \\ 8:\delta [i]\leftarrow \text{euclidDist}(\overline{c},D_0[i])\hfill \\ \hfill & \\ 9:\text{append}(D_0,\delta )\hfill \\ \hfill & 10:D_0^s\leftarrow sort(D_0,\delta )\hfill \\ \hfill & 11:∕∕\text{Find the}{w}_{max}\text{and}{h}_{max}\text{of MBR from 100 cells}\hfill \\ \hfill & 12:w_{\max }\leftarrow 0;h_{\max }\leftarrow 0\hfill \\ \hfill & 13:\mathbf{for}\mathbf{all}j\in (1,2,\dots ,100)\mathbf{do}\hfill \\ \hfill & 14:x_{\text{min}}\leftarrow \text{min}(N_B[0]);y_{\max }\leftarrow \text{min}(N_B[1])\hfill \\ \hfill & 15:x_{\max }\leftarrow \max (N_B[0]);y_{\max }\leftarrow \max (N_B[1])\hfill \\ \hfill & 16:w\leftarrow x_{\max }-x_{\text{min}};h\leftarrow y_{\max }-y_{\text{min}}\hfill \\ \hfill & 17:w_{\max }\leftarrow \max (w,w_{\max });h_{\max }\leftarrow \max (h,h_{\max })\hfill \\ \hfill & 18:w_{\text{final}}\leftarrow w_{\max }+x_{\text{margin}};h_{\text{final}}\leftarrow h_{\max }+h_{\text{margin}}\hfill \\ \hfill & 19:\mathbf{for}\mathbf{all}j\in (1,2,\dots ,100)\mathbf{do}\hfill \\ \hfill & 20:x_{\text{min}}\leftarrow \text{min}(N_B[0];)y_{\text{min}}\leftarrow \text{min}(N_B[1])\hfill \\ \hfill & 21:N_{\text{MBR}}[j]\leftarrow crop(I,[x_{\text{min}},y_{\text{min}},w_{\text{final}},h_{\text{final}}])\hfill \\ \hfill & 22:\mathbf{for}\mathbf{all}j\in (1,2,\dots ,100)\mathbf{do}\hfill \\ \hfill & 23:P_i:\text{image panel generated by adding}{N}_{\text{MBR}}[j]\hfill \end{array} \end{gathered}$$

discriminative methods predict the class label y from the training example x, by evaluating: $f(x)=\underset{y}{\text{arg max}}p(y\mid x)$.

We use ensemble classifiers to vote for discriminating cell instances in a robust way [16]. We have trained all these classifiers and classified data into two groups: MUT and WT. Five-fold cross validation is used to mitigate over fitting problem with the training data. We prune data iteratively in a few successive steps. First, those instances that are correctly classified by any of the above classifiers are kept. In the next iteration, we train classifiers with the remaining data and only keep those instances correctly classified by at least two updated classifiers. We increment the number of classifiers that produce correct classification results in each step. Following this method, the remaining dataset correctly classified by at least five classifiers are retained in the last iteration. We stop from increasing the number of correct classifiers further as we notice the classification accuracies of all classifiers get saturated up to this iteration. Finally, those confusing instances that are misclassified by any of the classifiers are removed.

2). Data Clustering Analysis:

With the retained data, we next analyze the hidden structure of group MUT and WT and find the most representative cell instances of each group. Specifically, we recover the structure of the unlabeled data of each group and partition them into K sets by K-means clustering [17]. We change K from 1 to 10. For each value, we compute the Sum of Squared Errors (SSE). Overall, SSE decreases as we increase K. We aim to choose as a small K as possible that produces a sufficiently low SSE. The “elbow” point usually represents the point after which SSE does not reduce much by increasing cluster number. We detect the “elbow” point where the graph begins to flatten significantly (i.e. the decrease percentage of SSE is < 15% in our case) [19]. Following this approach, we find K = 3 and K = 4 as the optimal cluster number for MUT and WT group, respectively. Clustering analysis for each group is next performed with the optimal K value.

3). Retrieval of Representative Cells:

With data clusters from each group, representative cells are found with Algorithm 1 that computes the centroid of each cluster and identifies the first 100 most representative cells based on their Euclidean distances to the cluster centroid. In Algorithm 1, variable δ represents a distance vector that captures Euclidean distances from data points to the cluster centroid; I is the image for cell analysis; D₀ and N_B capture the cellular features and boundary coordinates. Representative cells capturing group difference signatures are extracted from corresponding images {I} by fitting minimum bounding rectangles from {N_B} and assembled in a 10 × 10 cellular panel for each group, respectively.

III. EXPERIMENTAL RESULTS

As a driving use case, we test the developed analysis pipeline with histopathology images of astrocytoma tissues (i.e. a malignant brain tumor) from The Cancer Genome Atlas database. This dataset includes 200 images (1024 × 1024) of tumor regions from 50 patients equally from two molecular groups: Isocitrate Dehydrogenase (IDH) mutant (MUT) and wildtype (WT). In aggregation, 50,588 nuclei are automatically analyzed for feature computation, with 26,871 and 23,717 from MUT and WT, respectively.

To visualize data separability, we project data to a lower three-dimensional feature space with MDS and produce scatter plots with data before and after pruning in Figure 2. It is clear that the mapped feature data from two groups are highly clumped before data pruning, whereas post-pruning data present substantially improved separability between two groups. In Figure 4, we further visualize the decision boundary with the projected data from the two groups with QDA. The mode of the distribution of each group is specified by a confidence ellipse, with blue and red region for MUT and WT, respectively. The dataset after data pruning process includes 20,653 nuclei instances (i.e. about 41% of the original data) of which 8,257 and 12,396 cells are from group MUT and WT, respectively.

Fig. 4:

Decision boundaries from quadratic discriminant analysis (Left) before and (Right) after data pruning are presented.

Table I presents mean and standard deviation of the most discriminating features of cells between MUT and WT after data pruning. After statistical analysis and feature histogram study, we find Area and Perimeter are most discriminating cellular features between group MUT and WT.

TABLE I:

Mean and standard deviation of most discriminating cellular features for distinguishing cells in group MUT from WT are presented.

Group	Area	Perimeter	MeanIntensity	MaxDistance
MUT	448.28±162.88	76.36±14.96	52.50±16.00	28.33±6.16
WT	198.97±82.12	54.09±13.61	71.63±24.29	22.09±6.43

To visualize representative cells that carry discriminating information from each group, we cluster each group with K-means algorithm with optimal cluster number. We present scatter plot for each group in Figure 3. It is observed that relatively well compact clusters are formed with clear cluster separation for each group. Additionally, cells from same clusters have very similar feature representations and those from different clusters present distinct feature signature.

Fig. 3:

Scatter plots of clustered data from (Left) MUT and (Right) WT group are presented.

After clustering data for each group, we retrieve the first 100 representative cells based on the Euclidean distance to each cluster centroid for each group. A minimum bounding rectangle is found for each cell. The resulting cell image regions are extracted from the associated images and assembled into a 10 by 10 image panel. We repeat this process for each specific cluster of each group. In this way, we can facilitate the visualization of representative cells that capture discriminating information for differentiation of cell populations with distinct genetic or molecular properties. In Figure 5, we present such cell panels of one randomly selected cluster from MUT and WT, respectively. With these cell panels, we observe that cells from MUT are more circular in shape and larger in size than those from WT group. Additionally, cells from group MUT tend to have lower intensity than those from WT. The substantial difference in cellular phenotypic features related to the shape, size, and intensity between the two groups suggests the efficacy of our complete analysis pipeline for cell phenotypic feature comparison with large-scale molecularly distinct cell populations from histopathology images.

Fig. 5:

Two representative panels of retrieved cells from (Left) MUT and (Right) WT group are presented.

IV. CONCLUSIONS

We present a complete workflow that facilitates quantitative histopathology imaging investigations of phenotypic feature comparison for cells from different molecular groups. The developed analysis framework consists of image color normalization, deconvolution, segmentation, feature computation, and feature comparison analysis with representative cell retrieval. As our driving use case, we test our method on histopathology microscopy images of astrocytoma brain tumors from database of The Cancer Genome Atlas. With derived cellular features, we identify and retrieve representative cell instances that are phenotypically different between Isocitrate Dehydrogenase (IDH)-mutant (MUT) or wildtype (WT) groups by data pruning, dimensionality reduction, and unsupervised clustering. Our analysis is generic to a wide set of cell-based biomedical research.