Extending U-Net Network for Improved Nuclei Instance Segmentation Accuracy in Histopathology Images

Abstract

Analysis of morphometric features of nuclei plays an important role in understanding disease progression and predict efficacy of treatment. First step towards this goal requires segmentation of individual nuclei within the imaged tissue. Accurate nuclei instance segmentation is one of the most challenging tasks in computational pathology due to broad morphological variances of individual nuclei and dense clustering of nuclei with indistinct boundaries. It is extremely laborious and costly to annotate nuclei instances, requiring experienced pathologists to manually draw the contours, which often results in the lack of annotated data. Inevitably subjective annotation and mislabeling prevent supervised learning approaches to learn from accurate samples and consequently decrease the generalization capacity to robustly segment unseen organ nuclei, leading to over- or under-segmentations as a result. To address these issues, we use a variation of U-Net that uses squeeze and excitation blocks (USE-Net) for robust nuclei segmentation. The squeeze and excitation blocks allow the network to perform feature recalibration by emphasizing informative features and suppressing less useful ones. Furthermore, we extend the proposed network USE-Net not to generate only a segmentation mask, but also to output shape markers to allow better separation of nuclei from each other particularly within dense clusters.

The proposed network was trained, tested, and evaluated on 2018 MICCAI Multi-Organ-Nuclei-Segmentation (MoNuSeg) challenge dataset. Promising results were obtained on unseen data despite that the data used for training USE-Net was significantly small. The source code of the USE-Net is available at https://github.com/CIVA-Lab/USE-Net

I. Introduction

In digital microscopic tissue images, nuclear segmentation may allow high-quality features to be extracted for nuclear morphometrics and other analysis in computation pathology. In particular, segmentation of nuclei instances, that helps not only gather information on position and density, but also rich morphology features, nuclei density, magnitude and nucleus-to-cytoplasm ration, which is important to determine cancer grades, diagnosis and predict efficacy of treatment. However, it remains difficult to automatically segment the nuclei at instance-level because of the several factors. One of the factors having over or under segmentation can easily be caused by the overwhelming presence of nuclei occlusions and clusters, which makes precise morphological measurement of nuclei instances difficult. The other factor is having blurred border and inconsistent staining makes images hard for indistinguishable features to be used and therefore misleading and subjective annotations occurs, which makes it difficult to obtain robust results. Another factor, is the diversity in cell magnitude, appearance and density among different cell types and organs, makes it hard to have an approach with good generalization potential for robust cell analysis.

Methods proposed earlier were based on thresholding and morphological operations [1], [2] and it was hard for those approaches to find reliable threshold for complex background. To overcome those early approaches the techniques based on deep learning started to become widely used for medical image segmentation [3]–[5]. For instance, [6] proposed as deep contour-aware network (DCAN) for instance segmentation that first uses the supplementary contour and instance information to distinguish the attached objects. BESNet [7] specifically concatenates the output contour frames with nuclei features in decoders to use contour-specific features to aid nuclei prediction. It only learns additional knowledge in the nuclei branch, but lacks the possible reversed advantages of nuclei to contour, which is more significant since the appearance of the contour is more complicated and has greater intra-variance than that of nuclei. BESNet had a similar architecture to U-Net, but two decoders were used to boost cell boundaries and segment entire cells. Inspired by BESNet, a multi-level knowledge aggregation module was proposed by [8] to fuse the features extracted by the two BESNet decoders. The proposed architecture, Contour-aware Informative Aggregation Network (CIA-Net), achieved outstanding nuclei instance segmentation accuracy and won the first prize for the challenge of multi-organ nuclei segmentation (MoNuSeg). Since the annotated data is lacking for nuclei segmentation and making those annotation is costly and time consuming the following paper [9] proposed an instance-aware self-supervised learning approach to reduce the requirement of manual annotation in deep convolutional neural network. The proposed method involves two sub-tasks, which enforce the neural network to autonomously learn the prior knowledge on nuclei size and quantity by deeply exploiting the rich information contained in raw data.

In this paper, we leverage the encoder-decoder deep learning network architecture USE-Net [10], which extends a state-of-the-art network architecture U-Net [11] with a squeeze and excitation blocks [12] for robust nuclei segmentation. The squeeze and excitation blocks allow the network to perform feature recalibration by emphasizing informative features and suppressing less useful ones. The encoder part of the network extract features from an input H&E tissue images using a pre-trained SE-ResNet-50 [12] backbone. The extracted features are used to reconstruct the segmentation mask of the nuclei using similar structure such as U-Net architecture with the skip connections coming from each block of SE-ResNet-50 architecture. Moreover, we extend the USE-Net not to generate only a segmentation mask, but also to output shape markers to allow better separation of nuclei from each other particularly within dense clusters.

The overall proposed pipeline starts initially with preprocessing step to decrease the color variation of the inputs. Structure preserving color normalization (SPCN) proposed by [13] is a commonly used technique for color normalization of histology images, where the structure of the image is preserved while the color is normalized to a common color space. By using SPCN as a preprocessing step for color normalization in our pipeline we are transforming the histology images to a common color spaces. Through this step, we are making our network not to deal with the color variation of the input data. Afterward, the preprocessed data are augmented and fed into the USE-Net network. Finally, the output mask and shape marker obtained from the network are post-processed using mathematical morphology operations and the watershed method is applied to separate attached nuclei from each other to get the labeled mask as a final output from pipeline, where each nuclei has their unique labels. Fig. 1 demonstrates sample results of the proposed network, where top-left represents original image, top-right represents output mask from USE-Net, bottom-left represents output shape marker from USE-Net and bottom-right represents final output of the proposed pipeline.

Fig. 1:

Sample result on prostate tissue type. Top-left: raw image, top-right: proposed USE-Net mask, bottom-left: proposed USE-Net shape marker, bottom-right: proposed pipeline labeled result

The main contributions of this paper are (i) a multi-task variation of the encoder-decoder deep learning network architecture USE-Net, which extend U-Net with squeeze and excitation blocks; (ii) a complete pipeline for generating labeled nuclei mask from an H&E images using small amount of training data; and (iii) demonstrating the strength of using shape marker instead of center marker as a second output of the proposed network to allow better separation of nuclei from each other particularly within dense clusters. The proposed network was trained, tested, and evaluated on 2018 MICCAI Multi-Organ-Nuclei-Segmentation (MoNuSeg) [14] challenge dataset. Promising results were obtained on unseen data despite that the data used for training USE-Net was significantly small. The rest of the paper is organized as follows. In Section II, details of the proposed methods are described. In Section III, experimental results are presented. Finally, Section IV concludes the paper.

II. Methods

The overall flow of the proposed pipeline is detailed in this section as well as the USE-Net architecture. Initially, the proposed pipeline begins with a preprocessing step to reduce color variations in input images. Afterward, the preprocessed data is augmented and fed into the network. Later, the output masks from the network are post-processed using mathematical morphology. Finally, the watershed method is applied to separate attached nuclei from each other and get the labeled mask as a final output from pipeline, where each nuclei has its unique id. Fig. 2 demonstrates the proposed pipeline.

Fig. 2:

Proposed pipeline consisting of preprocessing, segmentation and post-processing steps.

A. Preprocessing

By increasing the input samples and reducing the color variation in the training images, our network will be able to produce more accurate segmentation results. The color variation in histology images can be reduced by using color normalization techniques. Moreover, different data augmentation strategies are used during training to improve the network’s generalizability.

1). Color Normalization:

It is almost inevitable to have color variation in the obtained histology images due to three main reasons: (1) usage of different microscopes or scanners for image acquisition (demonstrated in Fig. 3); (2) different staining solutions from different chemical manufacturers; (3) variation of staining protocols across different labs [13].

Fig. 3:

Same tissue image scanned by different scanners [13].

Color variance is not a major challenge for experienced pathologists in general, since they are trained in their routine diagnoses to deal with this visual variability. For several automated image processing and quantification techniques, however, it has become more of a challenge. For instance, two studies have shown that the output of tumor segmentation techniques based on learning deteriorates, because color of testing images are different than the training dataset [15], [16].

Structure preserving color normalization (SPCN) [13] is commonly used for color normalization of histology images, where the structure of the image is preserved while the color is normalized to a common color space. Consequently, the learning process for segmentation is streamlined.

To use SPCN, the target image needs to be selected first, where the remaining training and testing data are going to be color normalized. As a target the following training image of liver tissue ”TCGA-B0-5711-01Z-00-DX1” was selected arbitrarily. The Fig. 4 demonstrates the example of color normalizing using SPCN technique.

Fig. 4:

SPCN color normalization, left to right: input, target, SPCN output.

2). Data Augmentation:

The MoNuSeg challenge provides 30 images as a training data, which limits the potential of deep networks that require large amounts of data. To overcome this issue different augmentation strategies are used to increase the number of training data using 30 initial images provided. In particular three popular augmentation methods are used: (1) horizontal and vertical flip; (2) rotation; and (3) translation. Random combination of those methods have been used to obtain more variation of input data for training.

B. Nuclei Segmentation Network

The used encoder-decoder deep learning network architecture USE-Net [10], [17], is similar to the state-of-the-art network architecture U-Net [11], where in the encoder side the SE-ResNet-50 [12] is used as backbone instead of normal U-Net encoder, and squeeze and excitation blocks are used after each residual block of the ResNet-50. The squeeze and excitation blocks allow the network to perform feature recalibration by emphasizing informative features and suppressing less useful ones. The encoder part of the network extract features from an input H&E tissue images using a pre-trained SE-ResNet-50 [12] backbone, which is trained on ImageNet [18]. The extracted features are used to reconstruct the segmentation mask of the nuclei in decoder side using similar structure such as U-Net architecture with the skip connections coming from each block of SE-ResNet-50 architecture. As final outcome segmentation mask and shape marker are obtained from the proposed network. Fig. 5 shows the overall architecture of the used network USE-Net for nuclei segmentation.

Fig. 5:

The overall architecture of the USE-Net network [10] with SE-ResNet-50 backbone, where each residual block uses a final squeeze and excitation. (a) shows the general architecture, (b) shows the detailed conv block in the decoder, (c) shows the detailed conv block in the encoder.

C. Postprocessing

USE-Net gives segmentation mask and shape marker as the class label probabilities. Thresholding those probabilities leads to foreground/background segmentation mask and shape marker. To remove small spurious detection and fill the gap morphology operations are applied in the segmentation mask. After thresholding shape marker, connected component is applied to it, since it has better nuclei separation. Obtained labeled shape marker map along with the binarized mask are given as inputs to the watershed algorithm to generate a final instance segmentation of the nuclei.

III. Experimental Results

In this section we present details on training and test datasets, evaluation metrics, ablation study, qualitative and quantitative results of the extended USE-Net deep learning nuclei segmentation network.

A. Benchmark evaluation datasets

The benchmark dataset provided by MoNuSeg 2018 challenge [14] is used to train and test our extended USE-Net network. The MoNuSeg 2018 training data was the same as that previously released by [19], which included 30 tissue images, each 1000 × 1000 in size, containing 21,623 hand-annotated nuclear boundaries. Each 1000 × 1000 image in this dataset was extracted from an individual patient downloaded from TCGA [20] from a separate whole slide image (WSI) (scanned at 40×). Seven different organs, such as breast, liver, kidney, prostate, bladder, colon and stomach, were represented in the dataset and both benign and diseased tissue samples were included to ensure the variety of nuclear appearance. In addition, the training images came from 18 different hospitals, which, due to the disparities in staining procedures and image acquisition equipment (scanners) across laboratories, introduced another source of appearance variations. The testing data contains 14 images each of size 1000 × 1000 pixels, spanning 7 organs (kidney, lung, colon, breast, bladder, prostate, brain), several disease states (benign and tumors at different stages), and approximately 7,223 annotated nuclei with the same method of annotation as of training data. The training and testing sets provided by the MoNuSeg 2018 challenge along with the number of samples for different organ presented in the TABLE I.

TABLE I:

MoNuSeg 2018 training and testing dataset composition [14].

Data subset	Nuclei	Images
	Total	Total	Breast	Liver	Kidney	Prostate	Bladder	Colon	Stomach	Lung	Brain
Training set	21,623	30	6	6	6	6	2	2	2	-	-
Testing set	7,223	14	2	-	3	2	2	1	-	2	2
Total	28,846	44	8	6	9	8	4	3	2	2	2

B. USE-Net Training Details

Weights for the SE-ResNet-50 backbone used in USE-Net initialized with pre-trained weights on ImageNet. Input size of the network is 992 × 992. Adam optimizer is used during training with an initial learning rate of 1e-4 that is reduced by a factor of 10 after every 30 epochs. The training data is shuffled and split into 90% for training and 10% for validation. Since there is an imbalance between the foreground and background classes, a combined loss function consisting of Dice loss (Eq. 1) [21] and binary cross entropy loss (Eq. 2) is used to train the network. The Dice loss is defined as,

$$L_{\mathit{\text{Dice}}}=\frac{2{\sum }_i^N{p}_i{g}_i}{{\sum }_i^N{p}_i^2+{\sum }_i^N{g}_i^2}$$ (1)

in which pi and gi represent pairs of corresponding pixel values of prediction and ground truth labels. The binary cross entropy (BCE) loss is defined as,

$$L_{BCE}=-(y\times \text{log}(p(y))+(1-y)\times \text{log}(1-p(y)))$$ (2)

where, y is the label and p(y) is the predicted probability. The final network loss is,

$$\mathit{\text{Loss}}=L_{BCE}*\lambda +L_{\mathit{\text{Dice}}}*(1-\lambda )$$ (3)

where λ is the weight parameter and is empirically chosen as 0.5.

The extended USE-Net model is implemented in PyTorch [22], and trained for 60 epochs with a mini-batch size of 8. For each epoch the training and validation samples are shuffled again. It took ≈5 hours (each epoch is about 5 mins) to finish the whole training process for the USE-Net on an NVIDIA Tesla V100 GPU.

C. Evaluation Metric

Because of its proven benefits over other segmentation metrics, the average aggregated Jaccard Index (AJI) was used as the metric to assess the performance of the competing algorithms in nucleus segmentation challenge MoNuSeg 2018 [14]. Average AJI is a unified evaluation metric that penalizes both object-level and pixel-level errors to overcome the shortcomings of other evaluation criteria. The value of AJI varies from 0 to 1, where higher is better. By maximizing the Jaccard index, computing AJI involves matching all ground truth nuclei to one observed nuclei. The AJI is then equivalent to the ratio of the intersection and union sums of the cardinals of these matched ground truths and projected nuclei. Additionally, the denominator is applied to all detected components that are not paired. Let G = {G1, G2, … Gn} denotes the set of instance ground truths, S = {S1, S2, … Sm} denotes the set of segmented objects and N denotes the set ofsegmented objects with none intersection to ground truth.

$$AJI=\frac{{\sum }_i^N{G}_i\cap S_j}{{\sum }_i^N{G}_i\cup S_j+{\sum }_{k\in N}{S}_k}$$ (4)

where,

$$j={\text{argmax}}_k\frac{G_i\cap S_k}{G_i\cup S_k}$$ (5)

The AJI computation code is available on the website of the challenge [14].

D. Experiment on MoNuSeg 2018 Challenge Dataset

Using the train data provided by MoNuSeg 2018, we have trained a couple of networks using different variations of backbone in the encoder part, losses and network outcomes. As a network outcomes two different strategies were used, first one having network to output mask and center marker and second one having network to output mask and shape marker. By doing ablation study using those different experiments we can understand which combination gives the best accuracy and allow better separation of nuclei from each other particularly within dense clusters. Fig. 6 demonstrates the difference between markers given as network output. Center marker is obtained from input image by finding the center of each nuclei as point pixel and dilating it with a constant value for all data. Shape marker is obtained from input image by eroding each nuclei, where erosion value changes according to the size of the nuclei. Moreover, to demonstrate the advantage of using squeeze and excitation block in the encoder side of the network, different backbones such as ResNet-18, ResNet-50 and SE-ResNet-50 were used in the ablation study.

Fig. 6:

Difference between center and shape marker.

We obtained 14 labeled nuclei from 14 test images using the overall pipeline that was proposed. Those labeled nuclei were used to obtain individual AJI for each nuclei separately and to get a mean AJI result, the average of the 14 obtained individual AJI was taken. Ablation study results are presented in the TABLE II, where each row represents different experiments. It can be observed from experiment 1 and experiment 2 that by adding SPCN as a normalization step, the mean AJI increased almost in a half percent. Also it can be observed that by just using mask as an output from network and obtaining marker using mathematical morphology operation from that mask gives the mean AJI of 51.91%. However, instead of using mathematical morphology operation to obtain marker from mask, if the marker is also provided by the network the mean AJI increased by almost 1%. Moreover, using shape marker instead of center marker improved mean AJI result by almost 5%. There was no significant improvement when ResNet-50 is used instead of ResNet-18 as a backbone. However, adding squeeze and excitation block in the backbone increased the mean AJI by almost 4% with respect of not using squeeze and excitation block in the backbone. By increasing the number of data using augmentation method and using both BCE and DICE loss with ResNet-18 backbone the mean AJI increased more than 1% with respect of using small data and only BCE loss with the same backbone. The best result obtained from ablation study has 65.87 % in mean AJI was from the extended USE-Net network where SE-ResNet-50 is used as a backbone, 4500 samples used for training, both BCE and DICE losses are used, and the segmentation mask and shape marker are given as an output, which is more than 6% in mean AJI with respect to the same configuration, but with ResNet-18 backbone. The obtained result from proposed network is on par with human performance, which is 65.3% [14].

TABLE II:

Ablation study using different normalization, augmentation, backbone, loss, and output.

	Normalization		Data Augmentation		Backbone			Loss		Output
Exp	None	SPCN	N=1500	N=4500	RN-18	RN-50	SE-RN-50	BCE	DICE	Mask	Center	Shape	Mean AJI
Exp 1	×		×		×			×		×			0.5146
Exp 2		×	×		×			×		×			0.5191
Exp 3		×	×		×			×		×	×		0.5254
Exp 4		×	×		×			×		×		×	0.5743
Exp 5		×	×			×		×		×		×	0.5812
Exp 6		×		×	×			×	×	×		×	0.5945
Exp 7		×	×				×	×		×		×	0.6189
Exp 8		×	×				×	×	×	×		×	0.6256
Exp 9		×		×			×	×	×	×		×	0.6587

We compared the performance of the extended USE-Net method against the top five methods of the MoNuSeg 2018 challenge [14] result as shown in Table III. It can be observed that there is not to much gap between the top five methods of the MoNuSeg 2018 challenge. Moreover, the difference between the top performing one and our proposed method is just 3%. As a future improvements we are planning adding contour information to our proposed network and also using affine deformation for data augmentation similar to the challenge winner team. Also for mask and marker estimation we are planning to either use cascaded network as team ranked fourth is doing or using two separate network one for mask and one for marker prediction. The qualitative results of USE-Net are shown in the Fig. 7, where individual AJI obtained from pipeline of each tissue is, 64% for prostate, 60% for brain and 60% for breast.

TABLE III:

Comparison of USE-Net to top five performing methods on MoNuSeg 2018 challenge [14].

Methods	Rank	Mean AJI
CUHK&IMSIGHT	1	0.6907
BUPT.J.LI	2	0.6868
pku.hzq	3	0.6852
Yunzhi	4	0.6788
Navid Alemi	5	0.6779
USE-Net		0.6587

Fig. 7:

Qualitative results of the proposed pipeline using USE-Net to generate mask and shape markers. The AJI of prostate is 64%, AJI of brain is 60%, and AJI of breast is 60%

IV. Conclusions

We have proposed a multi-task encoder-decoder deep learning network architecture, USE-Net, which extends U-Net with a squeeze and excitation blocks to improve nuclei instance segmentation accuracy. The overall proposed pipeline start initially with preprocessing step to decrease the color variation of the inputs using SPCN. Afterwards, the preprocessed data are augmented and feed into the network. Later the output mask and shape marker from the network are post-processed using mathematical morphology operations. Finally, the watershed method is applied to separate attached nuclei from each other and get the labeled mask as a final output from pipeline, where each nuclei has their unique pixel value. After doing ablation study using different normalization, augmentation, backbone, loss and outputs the following observations were made; (i) Using SPCN as a normalization step increased mean AJI almost in a half percent. (ii) Instead of using mathematical morphology to obtain marker from mask, if the marker is provided by network the mean AJI improved by almost 1%. (iii) Using shape marker instead of center marker improved the mean AJI by almost 5% with respect to using center marker. (iv) Adding squeeze and excitation block in the backbone increased the mean AJI by almost 4% with respect of not using squeeze and excitation block in the backbone. (v) Increasing training data and combining BCE and DICE losses improved mean AJI by more than 6% if SE-ResNet-50 backbone is used instead of ResNet-18. The obtained result from proposed network is on par with human performance, which is 65.3%.