Convolutional neural network initialized active contour model with adaptive ellipse fitting for nuclear segmentation on breast histopathological images

Abstract.

Automated detection and segmentation of nuclei from high-resolution histopathological images is a challenging problem owing to the size and complexity of digitized histopathologic images. In the context of breast cancer, the modified Bloom–Richardson Grading system is highly correlated with the morphological and topological nuclear features are highly correlated with Modified Bloom–Richardson grading. Therefore, to develop a computer-aided prognosis system, automated detection and segmentation of nuclei are critical prerequisite steps. We present a method for automated detection and segmentation of breast cancer nuclei named a convolutional neural network initialized active contour model with adaptive ellipse fitting (CoNNACaeF). The CoNNACaeF model is able to detect and segment nuclei simultaneously, which consist of three different modules: convolutional neural network (CNN) for accurate nuclei detection, (2) region-based active contour (RAC) model for subsequent nuclear segmentation based on the initial CNN-based detection of nuclear patches, and (3) adaptive ellipse fitting for overlapping solution of clumped nuclear regions. The performance of the CoNNACaeF model is evaluated on three different breast histological data sets, comprising a total of 257 H&E-stained images. The model is shown to have improved detection accuracy of F-measure 80.18%, 85.71%, and 80.36% and average area under precision-recall curves (AveP) 77%, 82%, and 74% on a total of 3 million nuclei from 204 whole slide images from three different datasets. Additionally, CoNNACaeF yielded an F-measure at 74.01% and 85.36%, respectively, for two different breast cancer datasets. The CoNNACaeF model also outperformed the three other state-of-the-art nuclear detection and segmentation approaches, which are blue ratio initialized local region active contour, iterative radial voting initialized local region active contour, and maximally stable extremal region initialized local region active contour models.

1. Introduction

In recent years, a large focus of histopathological image analysis has been on the automated identification of different types of nuclei, such as epithelial,¹ lymphocytes,² and cancerous nuclei.³ For a number of different cancers, cancer grading is highly correlated with the appearance and morphology of individual nuclei as seen on a routine Hematoxylin and Eosin–stained histopathology image.^4,5 The presence, extent, size, shape, cellular organization, and other morphological appearances of nuclei are important indicators for presence or severity of disease.^6–11 In some circumstances, the populations of isolated cells and cell clusters in the tumor¹² may reflect some diagnostic relevance. In breast cancer grading, nuclear atypia refers to how size and appearance of cell nuclei tend to vary in terms of shape and appearance due to irregular chromatin texture and presence of nucleoli.^4,13 Normal or benign nuclei are typically small and uniform in appearance while malignant cells are larger and vary in size and shape.¹⁴ The modified Bloom–Richardson grading system comprises three factors, including glandular (acinar) or tubular differentiation, nuclear pleomorphism, and mitotic count. Therefore, the grading system is highly correlated with the morphological and topological features of the nuclei in breast cancers. Explicitly extracting nuclear features pertaining to nuclear shape, architecture, and topology are critical considerations in the construction of automated nuclear grading systems.¹⁵

Therefore, for the purposes of computerized nuclear grading, both nuclear detection and nuclear segmentation are important prerequisites. The goal of nuclear detection algorithms is to identify the centroid of the nuclei. Accurate nuclei detection results can enable the automated characterization of spatial architecture of nuclei in tumor regions.¹⁶ Features that reflect the spatial arrangement of nuclei (e.g., via graph algorithms such as the Voronoi, delaunay triangulation, minimum spanning tree) have been shown to be strongly associated with grade¹⁷ and cancer progression.¹⁸ Many cell-based research studies require automated counting of lymphocytes and nuclei from histological sections to quantitatively assess changes within microenvironment of cells, tissues, and organs.^19–21 Nuclear segmentation is a higher level approach to extract the contours of a nucleus. Precise nuclear segmentation is a critical prerequisite step for extraction of nuclear shape features in tumor tissue to classify nuclear atypia within tumor regions.⁷ This is important because the shapes of nuclear contours are valuable for cancer grading. Additionally, in conjunction with machine learning classifiers, nuclear shape features can be used to predict patient outcome and disease aggressiveness.²² The nuclear detection step can serve as an initialization phase for the subsequent nuclear segmentation step.^2,23 If the nuclear detection phase is suboptimal, the detection errors will be propagated to the segmentation phase causing inaccurate segmentation results. Therefore, it is important to develop an accurate detection method for accurately detecting the nuclei prior to the segmentation step. This is the reason why the segmentation models are usually paired with a nuclei detection method.^2,24

However, automated detection and segmentation of nuclei from high-resolution histopathological image typically has challenges with the following five reasons:²⁵ (1) in histology image, a large number of nuclei and different tissue structures congested together, meaning that there is a need for highly efficient and accurate approaches, (2) high-resolution whole slide images (WSI) are normally the size of $\mathrm{220,000}\times \mathrm{90,000}\text{pixels}$, (3) the variability in size, shape, appearance, and texture of the individual nucleus and lack of prominent and obvious boundaries for most nuclei, (4) noisy, nonhomogenous backgrounds, staining artifacts, and staining variations, and (5) nuclei being clustered closely, overlapped, and occluded.

To tackle these challenges, we present a three-phase scheme that couples convolutional neural network (CNN) for accurate nuclei detection, region-based active contour (RAC) model for initial nuclei segmentation, and adaptive ellipse fitting (AEF)–based overlapping resolution for handling overlapped nuclei. The idea of combining an initial nuclear detection step with shape-based segmentation is attractive since it dramatically reduces the real estate over which the more computationally expensive segmentation methods needs to be applied. Additionally, since it provides an initialization of the active contour can be more quickly converged to the true nuclear boundary, it makes the approach more computationally feasible.

The rest of the paper is organized as follows: a review of previously related works and the contribution of this paper are presented in Sec. 2. A detailed description of detection, segmentation, and overlapping resolution components is presented in Sec. 3. The experimental setup and comparative strategies are discussed in Sec. 4. The experimental results and discussions are reported in Sec. 5. Concluding remarks are presented in Sec. 6.

2. Previous Work and Contributions

In this section, we provide a brief overview of the state of the art in nuclear detection and segmentation from histopathologic images and discuss some of the limitations of these previous approaches, motivating the need for our approach. The authors in Refs. ²⁶ and ²⁷ developed CNNs–based approaches for nuclear detection and segmentation from high-resolution histological images. Current nuclear detection approaches include voting based,^28–32 LoG filter based,^33–35 intensity based,^36–41 mathematical morphology based,^41–43 H-minima transform based,⁴⁴ watershed based,^15,45–48 gradient based,^43,48 fuzzy C-means,⁴⁰ region growth and MRF,⁴⁹ Gaussian mixture model,² and other color-based^33,50,51 approaches. Although current detection approaches show their efficiency in detecting nuclei, finding proper seed points or deciding initial contours on histological images is still an open and challenging problem; deep learning (DL) is a data-driven and end-to-end learning approach,⁵² which attempts to learn high-level structural features from just pixel intensities.^53,54 These DL-based approaches had evoked great interest from the histological image analysis community since CNN won the ICPR 2012 contest and MICCAI 2013 grand challenge on mitotic detection.⁵⁵ Deep convolutional neural network (DCNN) is one of the most popular DL architecture approaches that had shown great achievements in various applications, especially in image analysis. The DCNN involves convolutional and subsampling operations to learn a set of locally connected neurons through local receptive fields for feature extraction.⁵⁶ Authors in Ref. 57 presented a CNN with three layers and eight feature maps per hidden layer for nuclei classification from histopathological images. In Ref. 58, we employed the sparse stacked autoencoder framework for learning high-level features corresponding to different regions of interest containing nuclei. These low-dimensional, high-level features were subsequently fed into a Softmax classifier (SMC) for discriminating nuclei from non-nuclear regions of interests within an independent testing set. In Ref. 59, a sparse reconstruction and adaptive dictionary learning method was presented for automatic cell detection, where a sparse reconstruction-based approach was employed to split touching cells,^26,27 developed convolutional neural networks approaches for nuclear detection and segmentation from high-resolution histological images.

Active contour models (ACMs) or level set–based approaches,^{2,15,23,28,38,45,51} watershed-based approaches,^{29,37,44,46–48,50,60} and region-growing⁴⁹ remain three effective approaches that had been widely used in segmenting desired objects from histopathological images. However, watershed-based models are prone to over-segmentation. Also, without proper initialization with seed points, watershed approaches are easily disrupted by local intensity fluctuation. Similarly, ACMs or level set–based approaches are usually sensitive to the initial placement of seed points or initial contours. However, due to the complicated nature and sheer size of histological images, the curve for model initialization must be placed near the desired boundary. Region-scalable fitting-based ACM was presented in Ref. 61 to overcome the limitation of global region-based ACM,⁶² where the intensity information of the local region was described by a Gaussian kernel, where the covariance of the kernel controls the scale of local regions.

The other drawbacks of ACMs are that they are prone to undersegmentation of the clumped nuclei, in which the nuclei are touching or overlapping with each other. Therefore, overlapping resolution is usually needed for solving the undersegmentation problem. There are two main types of overlapping resolution schemes. The first type of scheme is essentially integrated with the nuclear detection results.^28,42,46,63 Therefore, the performance of this type of scheme depends on a nuclei detection result. In Ref. 63, the clumped nuclei regions are separated by a marked watershed algorithm. Before separating the clumped nuclei regions, an improved seed detection technique based on voting is used to detect the nuclei. Other related works include marker-controlled watershed;⁴² integration of H-minima transform for detecting seed and the outer distance transform for separating cluster nuclei;⁴⁶ the single-pass voting algorithm for nuclear detection and repulsive level set method for segmenting cluster nuclei.²⁸ In Ref. 7, radial symmetry scheme was used to detect candidate nuclei locations. Then, watershed and ellipse fitting schemes were used for segmenting nuclei. One deficiency of the ellipse fitting scheme is that the scheme simply assumes every nucleus to be elliptical. The second class of approaches is based on shape analysis, especially concavity detection. In Ref. 2, we presented a heuristic splitting of contours via identification of high concavity points. In Ref. 64, an iterative, concave-point and radial-symmetry-based splitting algorithm was used for separating touching-cell clumps. In Ref. 15, prior shape was learned on top of a region-based ACM for segmenting overlapped nuclei.

Figure 1 illustrates the flowchart showing the work for the convolutional neural network initialized active contour model with adaptive ellipse fitting (CoNNACaeF) for nuclear detection and segmentation on histological images. As the flowchart shows, the model mainly includes two components. The detection component aims to accurately detect the centers of nuclei from high-resolution histological images. It employs the SAE for learning an initial filter with training patches. The SAE learns an initial filter with training patches, in a data-driven fashion, which extracts high-level features of input training patches. It then provides the initial filter bank to a CNN. Next, the CNN plus SMC model is trained with a back-propagation approach layer by layer. We then use CNN + SMC to represent the detection component, which comprises CNN and SMC. After the CNN + SMC is trained, it will subsequently be employed for detecting nuclear patches from the patches selected by a sliding window–based detector. The sliding windows sweep the windows over every possible location of nuclei on the histological images for selecting candidate patches. Based off the initially detected nuclear patches, circular contours are initialized at every detected potential nuclear centroid and subsequently employed for the initialization of the RAC model. The region-based active contour (RAC) model evolves to segment the boundaries of all the nuclei in the image based off the initial contours. To solve the undersegmentation problem, the AEF scheme is presented to segment clumped nuclei.

Fig. 1.

The flowchart illustrating the CoNNACaeF for nuclear detection and segmentation.

The combination of the CNN and RAC models is important in this paper. The CNN-based detection component can allow for identifying nuclei fairly accurately. Hence, it provides a good initialization point for the RAC model. Moreover, the detection component gives fewer false-positive detection. Thus, it greatly reduces the number of initial contours for subsequent nuclei segmentation and greatly increases the computational efficiency. Moreover, the accurate detection of nuclei provides not only the initial contour for the RAC model but also accurate seed points for the AEF scheme for the nuclear overlap resolution step. The motivation to leverage CNN for nuclear detection was to accurately detect the location of a nucleus, which subsequently provided an initial contour for the ACM. It is well-known that the ACMs are sensitive to the initial contours. Therefore, to segment a large number of nuclei from histological images, automated initialization of multiple active contours is dependent on accurate initialization. Similar to our previous work,²³ we integrated the HNCut initialization scheme with an ACM for initially identifying the location of the glands and then selectively invoked ACMs in locations identified as candidate regions by NHCut to be able to segment the glands. In this work, we focused on the problem of nuclei segmentation where the initial detection was conducted with a deep convolutional network for identifying the initial location of nuclei. Hence, it provided a good initialization point/contour for the subsequent region-based active contour (RAC) model.

3. Methodology

3.1. Nuclear Detection

3.1.1. Sparse autoencoder for learning initial weights

Basically, an SAE is simply a multilayer, feed-forward, neural network trained to represent the input. By applying a greedy layer-wise backpropagation approach, the AE tries to decrease the discrepancy as much as possible between input and reconstruction by learning an encoder and a decoder network (see Fig. 2), which yields a set of weights $W$ and biases $b$.⁵⁸ For simplicity, in this paper, we use the same notations as those in Ref. 58.

Fig. 2.

The diagram illustrating the architecture of AE with “encoder” and “decoder” networks for initial filter bank learning of nuclei structures. The “encoder” network represents $11\times 11\times 3$ input pixel intensities corresponding to an image patch via a reduced 25-dimensional feature vector. Then, the “decoder” network reconstructs the pixel intensities within the image patch via the 25-dimensional feature vector.

The architecture of the basic SAE is shown in Fig. 2. In general, the input layer of the autoencoder consists of an encoder network, which transforms input $X={[x(1),x(2),\dots ,x(N)]}^T$ into the corresponding representation $h$, and the hidden layer $h(k)={[h_1(k),h_2(k),\dots ,h_{d_h}(k)]}^T$ can be seen as the feature representation of the input data. The output layer is an effective decoder network that is trained to reconstruct an approximation $\widehat{X}$ of the input from the hidden representation $h$. Basically, training an AE is the same as finding optimal parameters by minimizing the discrepancy between input $X$ and its reconstruction $\widehat{X}$.

3.1.2. CNN + SMC for nuclei detection

As shown in Fig. 3(a), the CNN is a hierarchical neural network that comprises a convolutional layer [see Fig. 3(b)], a max-pooling layer [see Fig. 3(c)], a full connection layer, and a final classification layer. The convolutional layers (or C layers) and max-pooling layers (or P layers) produce a convolutional and a max-pooling feature map via successive convolution and max-pooling operations, respectively. The max-pooling operation down-samples an input image patch, reducing its dimensionality and allowing for assumptions to be made about features contained in the binned subregions.

Fig. 3.

(a) Illustration of CNN plus SMC for identifying the presence or absence of nuclei in each image patch, (b) the convolutional operation, and (c) the max-pooling operation.

These feature maps extract and combine a set of appropriate features. This high-level feature is subsequently fed to a SMC, which produces a two-dimensional vector where each element can be interpreted as a probability distribution over two different possible outcomes, presence or absence of a nucleus, respectively. The final result 1 or 0 was determined by the higher of the two numbers in the two-dimensional probability vector associated with each image patch.

3.1.3. CNN + SMC with sliding window detector for nuclei detection

Figure 4 illustrates the architecture and procedure of CNN + SMC for nuclei detection.

Fig. 4.

The diagram illustrating CNN + SMC for nuclei detection on histological images. With the sliding window scheme, the selected image patches from the histological image are fed into the trained CNN + SMC model for detecting nuclei presence or absence. If the nuclei is found to be present, a green dot is then placed in the center of each image patch.

To detect whether or not a nucleus is present at any given location on the image, a sliding window scheme involving a $34\times 34$ window sliding across the entire image is used to select candidate patches. The window size is defined as $34\times 34$, which is big enough to contain a nucleus within the patch under $40\times$ optical magnification resolutions images. The size is given in pixels. This choice is justified in Ref. 58, and the overall performance with such window size was evaluated in Ref. 58. It is therefore omitted in this paper. Since the approach involves the use of a pixel-by-pixel sliding window, the sliding window detector will detect a large number of nuclear centroids. Moreover, the sliding window detector typically results in multiple responses around the target nucleus. To avoid such multiple detections of the same nucleus, nonmaxima suppression (NMS) is applied to all detections in the image with confidence above certain threshold. Thus, any detector responses in the neighborhood of a nucleus with less than locally maximal confidence scores are removed. The specific procedure involving NMS was as follows. Several sliding windows were applied for nuclear detection within small local $34\times 34\text{pixel}$ regions. These windows are then sorted according to their confidence value in containing a candidate nucleus. Here, 0.8 was empirically defined as the threshold of confidence across the entire image for the presence of a nucleus within the sliding window. All windows with a confidence value of $>0.8$ were retained while windows with a confidence of less than that value were suppressed. Then, the window with the highest confidence value was chosen and other windows were suppressed when the intersection over union (IoU) was $>30\%$. Here, IoU is computed as

$$s(i,j)=\frac{|i\cap j|}{|i\cup j|}-1.$$ (1)

Here, the $i$, $j$ indices refer to the area of the two sliding windows, respectively.

Figure 5 shows the detection results with CNN before and after employing NMS, respectively. The detector recognizes well-centered nuclei in its input field, even in the presence of an adjacent nucleus. The approach thus is able to reject images containing no centered nuclei. If the nuclei presence in the input patches is detected by a nuclei detector, the patches will then be considered during the segmentation phase.

Fig. 5.

The illustration of the intermediate detection results by the CNN model for nuclear detection on a magnified region, which is selected from the black square region in Fig. 9(b). The detection results with CNN before and after applying NMS method are shown in (a) and (b), respectively.

3.2. Nuclear Segmentation

3.2.1. Initial contour generation for region-based active contour model

The performance of ACMs is sensitive and dependent on the initial contour of the models. Each candidate nucleus identified via the CNN is then used for generating octagon-like contours and subsequently employed as the initialization for the RAC model. As most of the nuclei are roughly circular in shape, octagon-like contours were fitted to the nuclear regions identified in the vicinity of the detected nuclear centroids. The choice of octagon-contours was on account of the relatively easy implementation of these shapes. The initial contour map for the entire image is generated after applying this process for all of the detected nuclei patches. The examples of initial contour maps for WSI are shown in Figs. 11(a) and 11(b), respectively. Then, beginning with these initial contours, the RAC model evolves to segment the boundaries of each nucleus on each of the images.

Fig. 11.

The illustration of the initial contour map for an entire image with detected nuclear patches by the CNN model is shown on representative images from (a) $D_1$ and (b) $D_2$.

3.2.2. Region-based active contour for nuclei segmentation

Assume an image $\mathrm{\Omega }$ is partitioned into two regions: ${\mathrm{\Omega }}_1$ nuclei (foreground) and ${\mathrm{\Omega }}_2$ non-nuclei (background). The distribution of local intensity statistics in each region ${\mathrm{\Omega }}_{\lambda }(\lambda =\mathrm{1,2})$ can be represented via a truncated Gaussian distribution as

$$p_{\lambda ,u}[I(v)]=\frac{1}{\sqrt{2\pi }{\sigma }_{\lambda }(u)}\exp \{-\frac{{[m_{\lambda }(u)-I(v)]}^2}{2{\sigma }_{\lambda }{(u)}^2}\},$$ (2)

where $m_{\lambda }(u)$ and ${\sigma }_{\lambda }(u)$ are the mean and variance of the local Gaussian distribution. Here, $u$ and $v$ are the two pixels in the image $\mathrm{\Omega }$ and ${\{{\mathrm{\Omega }}_{\lambda }\}}_{\lambda =1}^2$, $\lambda \in \{1=\text{foreground},2=\text{background}\}$, and are two disjointed regions such that $\mathrm{\Omega }={\cup }_{\lambda =1}^2{\mathrm{\Omega }}_{\lambda }$.

We define a kernel function:

$$K_{\mathrm{\Sigma }}(d)=\{\begin{array}{ll}\frac{1}{a}\exp (-\frac{{|d|}^2}{2{\mathrm{\Sigma }}^2}),& \text{if},|d|\le \rho ;\\ 0,& \text{if},|d|>\rho ,\end{array}$$ (3)

where $a$ and $\rho$ are the two predefined constants and $\int K(d)=1$. Here, $d$ represents the spatial location of the contour in the image, and $\mathrm{\Sigma }$ is the scale parameter that controls the localization property of the kernel. The curve evolution function can be derived with the theory of the calculus of variations as⁶⁵

$$\{\begin{array}{cc}\frac{\partial \varphi }{\partial t}=& -{\delta }_{\epsilon }(\varphi )(e_1-e_2)+\nu {\delta }_{\epsilon }(\varphi )\mathrm{div}\left(\frac{\nabla \varphi }{|\nabla \varphi |}\right)\\ & +\mu [{\nabla }^2\varphi -\mathrm{div}(\frac{\nabla \varphi }{|\nabla \varphi |}\left)\right]\\ {\varphi }_0,& \end{array},$$ (4)

where ${\varphi }_0$ is the initial contour determined by the CNN + SMC model in the detection phase, $e_1(u)={\int }_{\mathrm{\Omega }}{K}_{\mathrm{\Sigma }}(v-u)\{\log [{\sigma }_1(v)]+\frac{{[m_1(v)-I(u)]}^2}{2{\sigma }_1{(v)}^2}\}\mathrm{d}v$ and $e_2(u)={\int }_{\mathrm{\Omega }}{K}_{\mathrm{\Sigma }}(v-u)\{\log [{\sigma }_2(v)]+\frac{{[m_2(v)-I(u)]}^2}{2{\sigma }_2{(v)}^2}\}\mathrm{d}v$. Here, $\nu$ and $\mu$ are the positive constants. Here, ${\delta }_{\epsilon }$ is the smoothed Dirac delta function. The parameters of the kernel function $K_{\mathrm{\Sigma }}(·)$ were predefined as $\mathrm{\Sigma }=3.0$ and $\rho =6$, as previously suggested in Ref. 65.

3.2.3. Adaptive ellipse fitting for overlap resolution

Figure 6 illustrates the flowchart for the AEF for overlap resolution for reconciling clumped nuclear regions.

Fig. 6.

The flowchart illustrates the adaptive ellipse-fitting approach for overlap resolution on (a) clumped nuclei region (in white) generated by RAC model where red dots are detected nuclear centers by the CNN-based scheme. (b) The boundary is divided into different curve sections, with different colors representing different attributes pertaining to their respective nuclei. (c) The clumped region is divided into different subregions, illustrated via different gray scale regions, each region reflecting attributes pertinent to their respective nucleus. (d) The boundary of each subregion is estimated based on (c) where the pink contour is the boundary of the subregion attributed to the pink dot. (e) The ellipse-fitting algorithm operates based on the boundary or curve of each subregion obtained in (d).

Let $R_i$, $i\in \{\mathrm{1,2},\dots ,r\}$, be an $i$’th clumped nuclear region in the tissue section and ${\overline{R}}_i$ is the corresponding boundary of the $i$’th region, where $r$ is the total number of clumped regions in the image $\mathrm{\Omega }$ of the tissue section. $c_{ij}$, $j\in \{\mathrm{1,2},\dots ,n_i\}$ is the $j$’th nucleus in $R_i$, where $n_i$ is the total number of nuclei in $R_i$. The subregion $P^{ij}$ and curve ${\overline{P}}^{ij}$ that attribute to nuclei $c_{ij}$, $j\in \{\mathrm{1,2},\dots ,n_i\}$ is defined as

$$P^{ij}=\{p_{ijl}|\underset{j}{\min }|p_{ijl}-c_{ij}|,p_{ijl}\in R_i,l\in \{\mathrm{1,2},\dots ,r_j\},j\in \{\mathrm{1,2},\dots ,n_i\}\},$$ (5)

$${\overline{P}}^{ij}=\{{\overline{p}}_{ijk}|\underset{j}{\min }|{\overline{p}}_{ijk}-c_{ij}|,{\overline{p}}_{ijl}\in {\overline{R}}_i,k\in \{\mathrm{1,2},\dots ,{\overline{r}}_j\},j\in \{\mathrm{1,2},\dots ,n_i\}\},$$ (6)

where $r_j$ and ${\overline{r}}_j$ are the total number of pixels in the subregion $R^{ij}$ and on the boundary of curve ${\overline{R}}^{ij}$, respectively. We define $p_j$ and ${\overline{p}}_j$ as the total number of pixels in the sets $P^{ij}$ and ${\overline{P}}^{ij}$, respectively. The detailed description of the proposed AEF algorithm is given as a pseudocode in Algorithm 1. Figure 7 shows how the AEF algorithm deals with the undersegmentation problem raised by the RAC model. Figure 7(a) is a histopathological image with heavily clumped nuclei. The red dots are the nuclear centers detected by the preceding detection module. Figure 7(b) shows the initial circular-like contours generated at every detected nuclear centroid (the red dots). As shown in Fig. 7(c), the RAC model causes serious undersegmentation problems. Based on the initial segmentation results and detected nuclear centers, the AEF was then applied to separate nuclei in clumped nuclear regions. As shown in Fig. 7(d), the clumped nuclei are well separated.

Algorithm 1.

The adaptive ellipse fitting.

Require: Binary image $\mathrm{\Omega }$ with multiple clumped regions $R_i$, $i\in \{\mathrm{1,2},\dots ,r\}$; the detected nuclear centers $c_{ij}$, $j\in \{\mathrm{1,2},\dots c_i\}$; threshold $\theta =30$. Define$p_j$and${\overline{p}}_j$are the total number of pixels in the sets$P^{ij}$and${\bar{P}}^{ij}$, respectively.

Ensure: Ellipse fitting on each subregion in the image

1: Find all the clumped regions in the image $\mathrm{\Omega }$

2: for Clumped region $R_i$$i\in \{\mathrm{1,2},\dots ,r\}$do

3:  for Pixels ${\overline{p}}_{ijk}$ on the boundary ${\overline{R}}_i$$k\in \{\mathrm{1,2},\dots r_j\}$, $j\in \{\mathrm{1,2},\dots n_i\}$do

4:   Compute ${\overline{P}}^{ij}$ by calculating pixel-wise distance between nucleus $c_{ij}$ and the pixels ${\overline{p}}_{ijk}$ on ${\overline{R}}_i$ based on Eq. (6)

5:  end for

6:  for Pixels $p_{ijl}$ in the region $R_i$$j\in \{\mathrm{1,2},\dots c_i\}$, $l\in \{\mathrm{1,2},\dots r_j\}$do

7:   Compute $P^{ij}$ by calculating pixel-wise distance between nucleus $c_{ij}$ and the pixels $p_{ijl}$ on ${\overline{R}}_i$ based on Eq. (5) $k\in \{\mathrm{1,2},\dots ,r_j\}$

8:  end for

9:  for Each curve ${\overline{P}}^{ij}$do

10:   if${\overline{p}}_j<\theta$ and $p_j<\theta$then

11:    compute the ellipse for which the sum of the squares of the distances to nucleus$j$based on the boundary pixel of$j$’th region$P^{ij}$associated with the nucleus is minimal

12:   end if

13:   if${\overline{p}}_j\ge \theta$then

14:    compute the ellipse for which the sum of the squares of the distances to nucleus$j$based on the curve${\overline{P}}^{ij}$associated with the nucleus is minimal

15:   end if

16:  end for

17: end for

Fig. 7.

The illustration of AEF for nuclear overlap resolution on heavily clumped nuclei. (a) The original image with the detected nuclear centroid (in red dots), (b) the initial round contours (in green curves) shown on top of original image, (c) the initial segmentation results (in green curves) by RAC model, and (d) the final segmentation results (in green curves) after applying the AEF algorithm on (c).

4. Experimental Design

4.1. Datasets

The histological images in the paper were generated by digitization of standard Hematoxylin and Eosin–stained (H&E stain) slides. To address the issue of staining variation, all the images from each dataset were normalized using the color normalization approach described in Ref. 66. The method is based off a nonlinear mapping of a source image to a target image using a representation derived from color deconvolution.

4.1.1. Data set 1 (D1): H&E lymph node-negative and estrogen receptor–positive BC data set

A total of 37 H&E-stained histopathological glass slides were obtained from a cohort of 17 lymph node-negative and estrogen receptor–positive breast cancer (LN-, ER + BC) patients. The size of each histopathological image is about $2000\times 2000\text{pixels}$, with an average of 1500 nuclei in each image. The H & E-stained breast histopathology glass slides were scanned into a computer using a high resolution whole slide scanner Aperio ScanScope digitizer at $40\times$ optical magnification. For all 37 images in this study, the objective was to automatically detect the location of nuclear regions and segment the boundary of nuclei. Since it was impossible to have an expert pathologist manually detect and segment each and every nucleus in each of 37 images (to provide ground truth for quantitative evaluation), the expert was asked to randomly pick regions of interest on the digitized images where nuclei clusters were visible. The expert then proceeded to meticulously segment the boundary of each nuclei within these visually identified regions of interest on each image. The randomly picked image regions comprised both clumped nuclear regions and individual nuclei. Although the overlapping resolution component was only applied to clumped nuclei regions, the individual nuclei were segmented via the segmentation module. Therefore, the segmentation performance of different models was compared on both clustered and nonclustered nuclei. Part of the reason for having an expert manually pick ROIs for evaluation was to avoid inadvertent selection of primarily stromal regions. These regions would likely have yielded scattered and isolated nuclei, considerably easier to detect and segment compared to clusters of overlapping nuclei.

4.1.2. Data set 2 (D₂): H & E Lymphocyte human epidermal growth factor receptor-2 (HER2+) BC data set

A total of 100 images were obtained from 47 H & E–stained biopsy samples of HER2 + BC patients. Each images is about $100\times 100\text{pixels}$ and contains an average of about 100 individual lymphocytes. The goal is to automatically detect and segment lymphocytes from these images. We direct the interested readers to Ref. 67 for a detailed description on how the ground truth was generated for the data set.

4.1.3. Data set 3 (D₃): Ductal Carcinoma in Situ (DCIS) BC data set

Histopathological images of breast-tissue for this study were collected on a retrospective basis from the Indiana University Health Pathology Lab (IUHPL) according to the protocol approved by the Institutional Review Board (IRB). All the slides were imaged using an Aperio ScanScope digitizer (Aperio, Vista, California) available in the tissue archival service at IUHPL. 120 images (around 2250K pixels for each image) were gathered from 40 patients, three images per patient. The expert was asked to manually annotate the centroid of each nuclei from each image. Note that these data only provided the ground truth of the nuclear center. Therefore, we only evaluated the detection accuracy on $D_3$.

For all of $D_1-D_3$, the manual annotations of the nuclear centroid and associated boundary (if provided) served as the ground truth of location and boundary of nuclear region. The quantitative evaluation results of automated detection and segmentation algorithms are based on these manual annotations. To address the issue of staining variation, all the images from each dataset were normalized using the color normalization approach described in Ref. 66. The method is based on a nonlinear mapping of a source image to a target image using a representation derived from color deconvolution.

In our experiment, we randomly generated 8000 patches ($34\times 34\text{pixels}$) from $D_1$, which comprised 2000 nuclei patches and 6000 non-nuclei patches from histopathological images, from which 1000 patches (500 nuclei and 500 non-nuclei) as a validation set for tuning of the hyper parameters. From each patch, 50 local receptive fields, whose sizes are $11\times 11\times 3$, are randomly extracted for training and validation.

4.2. Implementation Details for CoNNACaeF

All the experiments were carried out on a PC [Intel Core(TM) 3.4 GHz processor with 16 GB of RAM]. The implementation of SAE and CNN is based on the UFLDL Tutorial.⁶⁸ Our implementation was based on “matconvnet,” which is part of the vlfeat library. Our goal in this work was to use the SAE for determining the initial weights of the CNN. Moreover, this model is simple and easily implementable. The deeper architectures (more than three layers) resulted in over-fitting. The implementation of the RAC model is based on Ref. 61.

We performed a sensitivity analysis to analyze the effect of window size on the detection accuracy of CoNNACaeF. Figure 8 shows the sensitivity of window size ($X$ axis) on the nuclear detection accuracy ($Y$ axis) of the CoNNACaeF model. The curve shows that the model achieves the best F-measure and average precision value when the window size is 34 pixels. As the resolution of the images is comparable to what was employed in our previous work,⁵⁸ the procedure to optimize the parameters for the detection components of the CoNNACaeF model, is modeled on the strategies previously employed by us in Ref. 58. The window size was chosen to be big enough to contain a nucleus at $40\times$ optical magnification. The size of the individual image patches is in pixels. This choice was justified in Ref. 58, and the overall performance on account of this window size was evaluated in Ref. 58. We refer the interested reader to Sec. 4 in Ref. 58.

Fig. 8.

$F$-measure and AveP on the detection accuracy of CoNNACaeF with variable window size.

For the SAE model, as shown in Fig. 2, we randomly extracted local receptive fields with three color channels with size of $11\times 11\times 3$ from each training patch. Each of these patches was then input to the SAE. Each local receptive field yields a $11\times 11\times 3=363$ input vector to SAE. Therefore, $d_x=363$. For hidden layers, the number of units is set as $d_h=25$. The number of hidden layers was chosen on a trial and error basis, involving experimental evaluation with different numbers of hidden layers.

For the CNN, as shown in Fig. 3, we used one convolutional layer (one convolution layers and one pooling layer), one full connection layer, and an output layer. For the convolutional layer, 25 fixed $11\times 11$ convolutional and $8\times 8$ pooling kernels were used, respectively. For tunable hyper-parameters, a coarse-to-fine sweep approach was used for choosing kernel sizes, number of filters, learning rate, and weight decay. We employed a hierarchical search algorithm, involving first searching for a specific range of parameters and then honing in on the subspace of optimal parameters for the model. For other hyperparameters, we tried to obtain stable values that do not degrade our results.

For the NMS method employed in this paper, we only considered windows whose detection confidence is greater than or equal to a threshold, where the threshold value and overlapping rate are empirically defined as 0.8% and 30%, respectively. The threshold and overlap rate parameters were determined based on a trial and error basis, evaluating different thresholds and overlap rates. Below is the specific procedure involving NMS. Several nuclear windows were first detected in a local $34\times 34$ region. These windows are sorted according to their confidence value. Here, 0.8 was defined as the threshold of confidence across the entire image. All the detected potential nuclear centroids with a confidence value $\ge 0.8$ were retained. Then, the one with highest confidence value was chosen and other windows were suppressed, ones with overlapping rates $>30\%$.

When generating the initial contour map based on detected nuclear centers in the detection component, circular-like initial contours for each nucleus is implemented as a regular octagon centered by the nucleus center and whose side length is 38 pixels. It is almost the same size of a nucleus.

4.3. Comparative Strategies

To show the effectiveness of the proposed CoNNACaeF model, the model is compared with three other nuclear detection and segmentation strategies (see Table 1). The performance of the proposed model and comparative models are evaluated on image datasets $D_1$ to $D_3$.

Table 1.

Models considered in this work for comparative evaluation.

	Components
Acronym	Detection	Segmentation
		Initial segmentation	Overlap resolution
BRACaeF	BR	Extremal region-based AC model	aeF
IRVACaeF	IRV
MSERACaeF	MSER
CoNNACaeF	CoNN

We compared CNN with extant nuclei detection methods, including blue ratio (BR),³³ iterative radial voting (IRV),³² and maximally stable extremal region (MSER).³⁶ The implementation of BR is based on Ref. 33. The implementation of IRV and MSER is based on the source codes provided by the authors in the paper. We direct the interested readers to relevant references for detailed descriptions on the algorithms.

We also compared CoNNACaeF with blue ratio initialized local region active contour (BRACaeF), iterative radial voting initialized local region active contour (IRVACaeF), and maximally stable extremal region initialized local region active contour models (MSERACaeF) (see Table 1) models. The implementation of these three compared models is similar to CoNNACaeF model, where the detection results of BR, IRV, and MSER are leveraged to provide the initialization for the RAC model. More detailed description of these comparative models is provided in Table 1.

4.4. Performance Evaluation

4.4.1. Evaluating detection performance

The quantitative performance of nuclear detection by CNN and compared models shown in Table 1 is analyzed by using the metrics given in Ref. 58.

The performance of automatic nuclear detection is quantified in terms of precision, recall or true positive rate, F-measure, and Average Precision (AveP). Here, true positive (TP) is defined as the number of nuclei correctly identified as such by the model. In the paper, the correct detection of nuclear patches (TP) was identified as those instances in which the distance between the center of the detected nuclear window and the closest annotated, pathologist-identified nucleus was less than or equal to 17 pixels. FP and FN refer to false-positive and false-negative errors, respectively. AveP involves computing the average value of $p(r)$ over interval between $r=0$ and $r=1$ and the precision $p(r)$ is a function of recall $r$. Therefore, AveP shows the average area under precision-recall curve [see Figs. 13(a), 13(c), and 13(e)].

Fig. 13.

(a, c, e) The precision-recall curve and (b, d, f) ROC curves on detection accuracy of CNN compared to BR, MSER, and IRV on (a, b) $D_1$, (c, d) $D_2$, and (e, f) $D_3$. The values in the legend represents the $F$-measure of different detection methods on $D_1-D_3$, respectively.

We also mapped the precision-recall curves [see Figs. 13(a), 13(c), and 13(e)] and receiver operating characteristic (ROC) curves [see Figs. 13(b), 13(d), and 13(f)] to assess the performance of nuclear detection on three data sets provided by explicitly listing the models here.

4.4.2. Evaluating segmentation performance

The quantitative performance of nuclear segmentation by CoNNACaeF and comparative models shown in Table 1 was analyzed by using the metrics in Table 2, respectively. We report the pixel-wise mean precision, recall, and F-measure,⁶⁹ which are computed on a per nucleus basis, thus allowing a comparison to different models. We also reported computational time (CT) for detection and segmentation components for CoNNACaeF and comparative models.

Table 2.

The quantitative evaluation of the detection and segmentation results on $D_1$, $D_2$, and $D_3$ with BRACaeF, IRVACaeF, MSERACaeFC, and CoNNACaeF models. Note that $D_3$ only had manual annotations of the centroid of nuclei. Consequently, no quantitative evaluation of the segmentation results was performed for the four models for the images in $D_3$.

		Detection					Segmentation
Models	Datasets	Precision (%)	Recall (%)	F1 (%)	AveP (%)	CT (s)	Precision (%)	Recall (%)	F1 (%)	CT (s)
BRACaeF	$D_1$	69.41	50.24	58.29	35.80	2.51	36.87	28.89	30.51	150.43
	$D_2$	97.18	69.09	80.15	67.80	7.81	85.60	78.78	81.31	263.34
	$D_3$	83.49	45.67	59.04	40.02	22.43
IVTACaeF	$D_1$	62.88	47.35	54.02	32.08	6.32	33.76	30.43	30.92	148.15
	$D_2$	60.37	65.14	61.63	52.91	30.76	75.30	63.53	67.91	219.56
	$D_3$	66.21	52.96	58.85	38.58	182.89
MSERACaeF	$D_1$	83.58	63.91	72.44	53.96	13.30	57.25	50.47	51.43	154.38
	$D_2$	92.12	79.22	82.67	74.10	58.08	91.32	80.04	84.81	267.32
	$D_3$	71.70	78.03	74.74	57.03	230.51
CoNNACaeF	$D_1$	73.36	88.39	80.18	76.92	4.63	85.03	71.64	74.01	59.89
	$D_2$	83.91	88.54	85.71	81.73	23.51	90.33	82.33	85.36	199.96
	$D_3$	76.88	84.17	80.36	74.08	166.16

Note: The bold values represent the best performances.

5. Results and Discussion

5.1. Qualitative Results

The detection results of the CNN model on a large breast histological image [Fig. 9(a)] from D1 is shown in Fig. 9(b). The detection results of BR, IRV, MSER, and CNN models on $D_1$, $D_2$, and $D_3$ are illustrated in Figs. 10(a)–10(d) and 10(m)–10(p), and Figs. 12(a), 12(d), 12(g), and 12(j), respectively. For $D_1$, the detection results of different models are illustrated on a magnified region that is selected from the black square region in Fig. 9(b). In these detection results [see Figs. 10(a)–10(d) and 10(m)–10(p), and Figs. 12(a), 12(d), 12(g), and 12(j)], the green dots, yellow triangle, and red squares represent the nuclei that had been correctly detected (true-positive detection), the non-nuclei that had been wrongly detected as the nuclei (false-positive detection), and the nuclei that were missed with respect to the manually ascertained ground truth delineations, respectively. The CNN model was found to outperform the other three models with respect to the ground truth. Figure 5 shows the results of NMS method in improving the detection results from the CNN. The initial contour maps on $D_1$ and $D_2$ for detected nuclei patches with CNN models are illustrated in Figs. 11(a) and 11(b), respectively.

Fig. 9.

The nuclei detection results (b) with the CNN model for a breast cancer histopathology image (a) from a patient in $D_1$. The green dots, yellow triangles, and red squares represent the TP, FP, and FN with respect to the ground truth, respectively. The Pre, Rec, and F1 in the legend are the quantitative performance metrics for evaluation of the detection performance on this image.

Fig. 10.

The illustration of the nuclei detection results on (a–d) $D_1$ and (m–p) $D_2$, segmentation results by RAC model on (e–h) $D_1$ and (q–t) $D_2$, and overlap resolution by AEF on (i–l) $D_1$ and (u–x) $D_2$ with (a, e, i, m, u) BRACaeF, (b, f, j, n, v) IRVACaeF, (c, g, k, o, w) MSERACaeF, and (d, h, l, p, x) CoNNACaeF models. The detection and segmentation results with different models on a magnified patch (size $800\times 800$) in (a–h) is selected from the black square region in Fig. 9(a) of $D_1$. Green contours in (e–h) and (q–t) are segmentation results while (i–l) and (u–x) are further results of overlapping resolution with different models, respectively.

Fig. 12.

(a, d, g, j) Detection results, (b, e, h, k) initial segmentation by RAC model, and (c, f, i, l) final segmentation results after AEF algorithm for overlap resolution with (a, b, c) BRACaeF, (d, e, f) IRVACaeF, (g, h, i) MSERACaeF, and (j, k, l) CoNNACaeF on $D_3$.

The initial segmentation results by RAC model on $D_1$, $D_2$, and $D_3$ with BRACaeF, IRVACaeF, MSERACaeF, and CoNNACaeF models are shown in Figs. 10(e)–10(h), 10(q)–10(t), and Figs. 12(b), 12(e), 12(h), and 12(k), respectively. The green contours are segmentation results with different models. As the initial contour maps are highly dependent on the detection results, those nuclei that are missed by detection component will be missed by the subsequent segmentation component. Also, the undersegmentation problems are presented in clumped nuclear regions across different models. By employing the AEF algorithm on these results, most of overlapping and touching nuclei are separable. The final segmentation results after applying AEF by different models are shown in Figs. 10(i)–10(l), 10(u)–10(x), and Figs. 12(c), 12(f), 12(i), and 12(l), respectively. As expected, the CNN model was shown to outperform the other three models in terms of segmentation accuracy.

5.2. Quantitative Results

Figure 13(a) shows the precision-recall curves corresponding to nuclear detection accuracy with respect to the BRACaeF, IRVACaeF, MSERACaeF, and CoNNACaeF models across all the images from $D_1-D_3$, respectively. Each point on the $X$- and $Y$-axes represents precision and recall, respectively. Each model is quantitatively evaluated using AveP, as shown in Table 2. The results appear to suggest that CoNNACaeF achieves the highest AveP. Each of the ROC and precision-recall curves [Figs. 13(a), 13(c), 13(e), and 13(b), 13(d), 13(f), respectively] were generated by sequentially plotting the confidence scores (in descending order) associated with the various nuclear detection methods considered in this work, and across all the images from three datasets. High precision or true-positive rate corresponds to a method with more accurate nuclear detection results. For the nuclear detection problem, we only had information pertaining to the total number of manually identified nuclear patches (or positive patches). However, information on the total number of patches without nuclei (or negative patches) was not available. Therefore, to compute the false-positive rate (FPR), we estimated the total number of negative patches with the sliding window scheme across the randomly chosen ROIs on each image. The window slides across each ROI image, row by row, from the upper left corner to the lower right (the step size was fixed at six pixels). The number of negative patches equals the sum of all the patches across all of the images from $D_1-D_3$, respectively, excluding well-centered and annotated patches, as well as, those instances in which the distance between the center of the patch window and the closest annotated pathologist identified nucleus was $\le 17\text{pixels}$. Also, for Figs. 13(b), 13(d), and 13(f), since the total number of FP detections is always smaller than the estimate of the total number of negative patches, the FPR can never reach 1. The trajectory of the ROC curves is therefore only plotted for a false-positive fraction of 0.2. The ROC curve [see Figs. 13(b), 13(d), and 13(f)] shows that CoNNACaeF results in a superior detection performance compared to the other comparative models.

In terms of both detection and segmentation performance, the mean of precision, recall, and F-measure of CoNNACaeF and comparative models are shown in Table 2. As expected, in terms of detection results on $D_1$, $D_2$, and $D_3$, CoNNACaeF yields the highest F-measure at 80.18%, 85.71%, and 80.36%, respectively. In terms of segmentation results on $D_1$ and $D_2$, CoNNACaeF yields the highest F-measure at 74.01% and 85.36%, respectively.

The method integrates nuclear detection, segmentation, and overlapping resolution components in an efficient way. The detection component showed good performance in detecting the location of nuclei. However, the detection strategy still had some deficiencies. First, the sliding window scheme requires the window slides across the image in a step-size of six pixels, a high computational burden. More efficient detection strategies such as regression-based approaches are needed. Second, although several false-positive detections were avoided with the NMS scheme, some nuclei were missed, especially when multiple nuclei are proximal or clumped together. Third, the CoNNACaeF model, especially for segmentation, involves some manually adjusted parameters. This is also one of the limitations of ACM–based schemes.

5.3. Computational Consideration

All the experiments were carried out on a PC [Intel Core(TM) 3.4 GHz processor with 16 GB of RAM] and a Quadro 2000 NVIDIA Graphics Processor Unit. The software implementation was performed using MATLAB 2014a. The training set comprised 2500 nuclei and 6500 nonnuclei patches. The size of each patch was $34\times 34\text{pixels}$. The training time for the deep convolutional network was 288 s. The average CT for detection and segmentation of all nuclei on each image in $D_2$ was 23.51 and 199.96 s, respectively. The detailed CTs for the images in $D_1$, $D_2$, and $D_3$ are shown in Table 2.

6. Concluding Remarks

We presented CoNNACaeF model for the simultaneous, automated, detection, and segmentation of nuclei from breast histopathological images. The model takes advantage of the initialization by the CNN classifier, which provides accurate model initialization for the ACM. The pairing of the CNN with an active contour enables accurate and efficient nuclear segmentation on whole slide imagery. Additionally, the overlap resolution module enables separation of intersecting and clumped nuclei from each other.

To show the effectiveness of the model, we compared CoNNACaeF with BRACaeF, IRVACaeF, and MSERACaeF for the problems of nuclei detection and segmentation. Both qualitative and quantitative evaluation results on three data sets show that CoNNACaeF outperforms three state-of-the-art methods in terms of detection and segmentation accuracy. In future work, we will look to integrate the CoNNACaeF model with cell-graph and nuclear morphometric feature extraction methods for quantifying heterogeneity of breast tumors in histopathological images.

Biographies

Jun Xu is a full-time professor at Nanjing University of Information Science and Technology, China. He received his MS degree in applied mathematics from the University of Electronic Science and Technology of China and his PhD in control science and engineering from Zhejiang University, China. His other academic experiences include work as post-doc associate and research assistant at Rutgers University and a visiting professor at Case Western Reserve University. His research interests include medical image analysis, computational pathology, and digital pathology.

Lei Gong received his BS and MS degrees from Nanjing University of Information Science and Technology, Nanjing, China, in 2013 and 2016, respectively. His current research interests are machine learning and its application in computer-aided diagnosis on cancers.

Guanghao Wang received his BS and MS degrees from Nanjing University of Information Science and Technology, Nanjing, China, in 2012 and 2015, respectively. His research interest is pattern recognition. He is now working at Novatek Inc. in Shanghai, China.

Cheng Lu is a senior research associate at the Center for Computational Imaging and Personalized Diagnostics (CCIPD), Case Western Reserve University, and an associate professor in Shaanxi Normal University, China. He has authored over 20 peer-reviewed journal publications and over 20 conference papers and abstracts in the filed of pattern recognition, translational medicine, and image analysis.

Hannah Gilmore is the director of surgical pathology and director of the breast pathology service at University Hospitals Case Medical Center. She is an assistant professor at the Department of Pathology at Case Western Reserve University and is a member of the Case Comprehensive Cancer Center. She is a clinical expert in the diagnosis of breast diseases and is a member of the College of American Pathologists Advanced Multidisciplinary Breast Program. Her research in breast disease spans the clinical, translational, and basic science spectrum and has been published in numerous peer-reviewed journals. In addition to receiving research and teaching awards, her work has been funded by the National Cancer Institute, the Department of Defense, the American Cancer Society as well as from Industry.

Shaoting Zhang is an assistant professor in the Department of Computer Science at the University of North Carolina at Charlotte, since 2013. Before joining UNC Charlotte, he was a research assistant professor in the Department of Computer Science at Rutgers-New Brunswick, 2012 to 2013. He received his PhD in computer science from Rutgers in 2012, his MS degree from Shanghai Jiao Tong University in 2007, and his BE degree from Zhejiang University in 2005. His research is on the interface of medical imaging informatics, computer vision, and machine learning.

Anant Madabhushi is the director of the Center for Computational Imaging and Personalized Diagnostics (CCIPD) and the F. Alex Nason Professor II in the Departments of Biomedical Engineering, Pathology, Radiology, Radiation Oncology, Urology, General Medical Sciences, and Electrical Engineering and Computer Science at Case Western Reserve University. He is also a research health scientist at the Louis Stokes Cleveland Veterans Administration Medical Center. He has authored over 150 peer-reviewed journal publications and over 180 conferences papers and delivered over 250 invited talks and lectures both in the US and abroad.

Disclosures

No conflicts of interest, financial or otherwise, are declared by the authors.