Automated Segmentation of Abnormal Tissues in Medical Images

Abstract

Nowadays, medical image modalities are almost available everywhere. These modalities are bases of diagnosis of various diseases sensitive to specific tissue type. Usually physicians look for abnormalities in these modalities in diagnostic procedures. Count and volume of abnormalities are very important for optimal treatment of patients. Segmentation is a preliminary step for these measurements and also further analysis. Manual segmentation of abnormalities is cumbersome, error prone, and subjective. As a result, automated segmentation of abnormal tissue is a need. In this study, representative techniques for segmentation of abnormal tissues are reviewed. Main focus is on the segmentation of multiple sclerosis lesions, breast cancer masses, lung nodules, and skin lesions. As experimental results demonstrate, the methods based on deep learning techniques perform better than other methods that are usually based on handy feature engineering techniques. Finally, the most common measures to evaluate automated abnormal tissue segmentation methods are reported

Introduction

Medical imaging refers to the set of techniques and methods for visual reconstruction of different body organs and their corresponding compartments, especially those covered by skin and bony structures. The reconstructions are appeared in form of digital images. These images are used by physicians in different analysis and diagnostic processes. There are variety of medical imaging techniques, including, magnetic resonance (MR) imaging, computed tomography (CT) imaging, X-ray imaging, ultrasound imaging, and positron emitting tomography (PET) imaging. Each imaging technique is sensitive to a specific type of tissue; for example, MR images are sensitive to soft tissues while X-ray images are sensitive to hard and bony structures.

An important analysis based on digital medical images is the study of abnormal tissues both in diagnostic and follow-up sessions. In these cases, physicians look for organ abnormalities in digital image of that organ. Quantitative measurement of these abnormalities, including counts and volume are very important for monitoring disease progression and optimal treatment. In order to perform mentioned quantitative analysis, proper segmentation of abnormalities in medical images is necessary.

Image segmentation refers to such task of assigning a label to pixels of digital images in which pixels with the same label share same visual characteristics or computed properties. Segmentation has abound applications in the field of medical imaging. In both computer aided diagnosis (CADx) and detection (CADe), segmentation is inevitable for locating tumors and pathologies, diagnoses of colon, prostate, liver, and breast cancers, studying anatomical structures, measuring tissue volume, long nodule detection, and many other applications. As a result, segmentation based applications of medical imaging can be widely used in clinics for diagnostic and treatment monitoring by physicians as well as in medical research institutes for task such as drug development. As analysis and monitoring of abnormal tissues in medical images is very important, accurate segmentation of these abnormalities is of interest for many of CAD systems.

Manual segmentation of diverse abnormal tissues in medical images is usually time-consuming, difficult, error prone, and subjective. Moreover, in some medical research projects, multitude images of various patient groups should be segmented timely, accurately, and under the same protocol. Therefore, automated segmentation of abnormal tissues in medical images is necessary to facilitate clinical trials.

Automated segmentation of abnormal tissues in medical images is considered as a challenging and sophisticated task in computer vision. Reasons of this complexity are related to the inherent characteristics of body tissues. Inhomogeneity which means surface of a specific anatomical structure or pathology is not identical throughout that structure or pathology is one source of complexity. These changes in the gray level tone affect correct interpretation about the tissue properties. Moreover, abnormal tissues may look similar to the normal ones. In another word, gray levels of different soft tissues are close together. Another source of complexity is partial volume effect stating that some regions have properties of more than one tissue type. In addition, border of some abnormalities is fuzzy which causes their shapes not to have accurate description. Noise and artifacts are another reality which compound the puzzle. Abnormalities as complex and subtle objects justify the need for more sophisticated methods and techniques for tackling challenges of abnormal tissue segmentation.

There are different candidate approaches for automated segmentation of abnormalities in medical images that some of them are different thresholding methods [ 1 ], active contours, mode finding methods, normalize cut [ 2 ], and graph cut [ 3 ] techniques. These segmentation methods are not able to segment abnormalities in clinical level because they are not able to tackle previously mentioned complexities. In other words, abnormalities in medical images are potentially too complex to be represented accurately by a simple model. In order to facilitate some difficulties related to segmentation of abnormal tissues such as fuzzy borders and partial volume effect, pixel-based segmentation approaches are the most promising path to be followed. A pixel-based segmentation method of abnormalities judge all of the pixels in the input image based on the features describe those pixels. These features are obtained directly or computed from a subimage surrounding each individual pixel. Later, pixels with the same judgment (label) are considered as a region either normal or abnormal [ 4 ].

As it is depicted in Figure 1, a typical CAD system for pixel-vise segmentation of abnormal tissues in medical images potentially has seven phases. The procedure starts with image acquisition followed by pre-processing, feature extraction and transformation, feature selection, classification, post-processing, and ends with evaluation. In image acquisition phase, a digital image of a specific organ is obtained by various previously mentioned medical imaging techniques. The tasks related to pre-processing stage depend on the organ under analysis and also imaging technique. Noise elimination and histogram normalization are common tasks of pre-processing stage. Feature extraction and transformation refer to the description and abstraction of subimages surrounding a pixel under judgment. As there are many feature extraction methods with different properties, too many quantifiers describing surface intensity of a typical subimage are obtained. In practice, huge amounts of quantifiers are dangerous and cause curse of dimensionality as a bad phenomenon. Feature selection stage allows a subset of most informative features describing subimages best to be selected. Moreover, feature selection stage prevents passing random errors and noises from being in the next stages by selecting the most discriminative features. Classification stage attempts to build a model with capability of assigning a label to the middle pixel of subimage under analysis. This label can identify whether this pixel belongs to a normal tissue or abnormal one. Post processing stage attempts to connect pixels with the same label and highlights abnormal regions. In addition, as many of the segmentation methods potentially come up with many false positives and negatives, post-processing phase attempts to increase both sensitivity and specificity of the segmentation by reducing these false positives and negatives, respectively [ 5 ]. Evaluation as final stage of the pipeline measures performance of segmentation method quantitatively by utilizing relevant performance measures. These measurements specify appropriateness of the method for clinical trials.

Figure 1.

Block diagram of a typical computer aided detection (CAD) system for abnormal tissue segmentation in medical images.

In the rest of this paper, important techniques for segmenting abnormalities related to different diseases and pathologies are reviewed. These include multiple sclerosis lesion segmentation, breast cancer abnormal mass segmentation, lung nodule segmentation, and skin lesion segmentation. In the section entitled “evaluation methods of tissue segmentation techniques”, some common performance evaluation measures are explained. Discussion and conclusion section discuss and conclude the paper.

Multiple Sclerosis Lesion Segmentation

Multiple Sclerosis (MS) is one of degenerative diseases of central nervous system. MS patients have some dead tissues in their brains called MS lesions. These lesions are appeared in MR images as hyper-intense or hypo-intense signals in different image modalities. Load of these lesions correlates with disease progression. There are a variety of methods for automatic segmentation of MS lesions on MR images. Some methods work based on single modality and others based on multiple modalities. Some segmentation methods are supervised and others are unsupervised. Fuzzy c-means [ 6 ], fuzzy connectedness [ 7 ], expectation maximization [ 8 ], and anomaly detection [ 9 , 10 ] are unsupervised techniques of MS lesion segmentation. Supervised methods are categorized into two main categories: First category is based on handy feature engineering techniques. The second category works based on representation learning techniques.

Methods working based on handy feature engineering techniques have two important phases of feature extraction and classification. In feature extraction phase, a small image patch surrounding a pixel is described using variety of features, including intensity [ 8 , 11 ], spatial [ 12 , 13 ], shape [ 14 , 15 ], size [ 15 ], texture [ 16 ], atlas [ 17 ], and context rich features [ 18 ]. In classification phase, the middle pixel of image path or the image patch itself as a block is judged as normal tissue or lesion. In this phase, different classifiers, including k-nearest neighbor [ 19 , 20 ], support vector machines (SVM) [ 21 , 22 ], and decision trees [ 23 , 24 ] are used. One representative study of this category uses 34 different features for describing texture of image patches. These features are mainly related to the family of Haralick features [ 25 ] and gray level run length features [ 26 ]. Later, SVM is used to decide if the block under analysis is lesion or normal tissue. Finally, segmentation results are refined in post-processing stage. Performance of the proposed method in term of Dice similarity coefficient is reported as 0.79 [ 16 ].

Second category of supervised MS lesion segmentation methods are based on representation learning techniques. These methods do not have feature extraction phase. In other words, feature extraction is performed automatically; As a result, image patches are directly fed to the model to be judge. One of these studies uses massive training artificial neural network (MTANN) technique for segmentation of MS lesions [ 27 ]. Recently, many techniques based on deep neural networks especially convolutional neural network (CNN) have been proposed. In these techniques, the focus is on proposing new network architectures. Among these network architectures, encoding CNNs are massively used for segmentation of MS lesions [ 28 ]. One architecture based on encoding CNN uses shortcut connections leading to segment lesions in different scales and sizes [ 29 ]. Another network architecture attempts to use different size kernels to consider voxels in different resolutions [ 30 ]. According to the fact that MS lesions are more probable to be appeared in special regions of the brain, location sensitive CNN has been proposed [ 31 ].

Experimental results of previously mentioned segmentation methods of MS lesions on MR images reveal that representation learning based methods are the most accurate and promising ones for segmentation. Another advantage of these methods is that they do not relay on any handy feature extraction techniques.

Breast Cancer Abnormal Mass Segmentation

Breast cancer is one of the main causes of cancer death all over the world, especially on women. However, early diagnosis of breast cancer prevents its progress and also considerably increases success of treatment. There are different methods and techniques for diagnosis of breast cancer in its early stages, including ultrasound, digital mammography (DM), MR imaging [ 32 ], electrical impedance spectroscopy, and biopsy. Among these methods, DM gains more popularity for screening of breast cancer. In order to diagnose breast cancer before severe conditions, proper analysis of DM images is very important. The analysis yields to detect a cancerous mass in DM images which usually come up with many false positives and false negatives. Moreover, in screening programs which a large number of population is observed, analysis error increased. As a result, a proper CAD system can assist radiologists in screening programs [ 33 ]. As DM images are noisy and full of artifacts, their automatic analysis is challenging. Analysis methods for breast cancer are categorized into two main groups, including category performing lesion detection in two steps of candidate mass detection and classification of these masses as benign or malignant and category directly detecting malignant masses.

One of the recent methods belonging to the first category of breast cancer detection works as follow: after pre-processing, candidate regions are segmented via Otsu’s method. Next, gray level co-occurrence matrix and shape features are extracted from segmented candidate regions; then some of those extracted features are selected via correlation analysis. Finally, these features are fed to a random forest classifier to decide if candidate regions are normal, benign, or malignant masses [ 34 ]. This pipeline is repeated in other studies [ 35 ]. Here, a region of interest segmentation again is based on Otsu’s method; feature extraction phase utilizes some extra statistical features such as gray level run length features and those based on first order statistics. The best features are selected via CART method in the feature selection phase, and finally, feed-forward artificial neural networks are used as classifier. In another effort, a novel texture filter for detecting calcifications as first biomarkers of breast cancer is proposed [ 36 ].

As mentioned previously, second category of breast cancer detection methods bypasses feature extraction from candidate regions of interests. One representative method of this category uses a modified deep network architecture known as u-net [ 37 ] for segmentation of masses in digital mammograms [ 38 ]. Output of the network is a probability map binarized by thresholding. The binarized image represents malignant breast lesions. The literatures demonstrate breast cancer detection and segmentation methods based on deep architectures are very promising. These methods can assist radiologists in clinics as second opinion for improving diagnostic performance.

Lung Nodule Segmentation

Lung cancer is the first cause of cancer led to death in the male world widely. Early diagnosis of lung cancer is necessary for optimal treatment. Lung nodules are early signs of lung cancer which are observed in CT images. Segmentation of lung nodules are considered as one of the challenging tasks in automated medical image analysis [ 39 ]. One of early successful methods for segmentation of lung nodules is based on MTANN [ 40 ]. Main focus of this method is based on automated feature extraction [ 41 ]. Some people extend the idea of MTANN to support vector machines for segmenting lung nodules by substitution of neural network module with support vector module [ 42 ].

Some solutions for segmentation of lung nodules based on representation learning family especially deep learning methods have been proposed during the last years [ 43 ]. One recent work of these family extracts multiple two-dimensional views of three-dimensional volume of lung; later, for each view, CNN extracts features and detects nodules. Finally, a fusion module combines results of each individual CNN for final nodule detection [ 44 ]. In another study, convolutional restricted Boltzmann machines are used for lung nodule segmentation. In this study, discriminative and generative information is combined together for better segmentation [ 45 ]. Experimental results demonstrate the methods based on CNN perform better than other methods.

Skin Lesion Segmentation

Melanoma is one of the deadliest type of skin cancer with highest mortality rate. Conventional diagnosis methods are expensive due to intervention of experienced experts. Therefore, CAD systems can facilitate diagnostic process of melanoma. Skin lesion segmentation as the first stage for building a CAD system for melanoma detection in dermoscopic images is essential. Segmentation of skin lesions is a challenging task because of some items, including low contrast between lesions and normal skin, variabilities in shape, size, color, texture, light reflection, and fuzzy border of lesions.

A variety of methods have been proposed for automated segmentation of skin lesions. Many of them are based on active contour methods of segmentation and density estimation. In one representative density-based method, probability density function of lesion and normal skin areas are estimated. Later, maximum likelihood classifier is employed to decide whether each pixel belongs to lesion or normal skin area [ 46 ]. Another study segments skin lesions based on uniform distribution. In addition, as change turnover between lesion area and healthy skin area is smooth, uniform based segmentation may fail to segment lesion areas correctly. In order to refine segmentation, active contour method is employed. Finally, segmentation results of two segmentation methods are fused for more accurate segmentation [ 47 ]. Another method purely based on active contours also attempts to segment skin lesions. In this method, after proper pre-processing of skin images, multi Otsu method of segmentation is utilized for initialization of active contour method of segmentation [ 48 ].

From the fact that texture is an important feature, many researches based on texture for segmentation of skin lesions have been conducted. One of these studies extracts texture features based on Gabor filters and co-occurrence matrices from candidate regions. These features are classified using a support vector classifier [ 49 ].

As same as other applications of abnormal tissue segmentation, deep neural networks are massively applied on segmentation of skin lesions. A method based on U-net [ 37 ] for segmentation of these lesions are proposed [ 50 ]. One recent study based on a convolutional neural network with 19 layers performs segmentation of skin lesions [ 51 ]. This architecture utilizes a specified loss function based on Jaccard index for better segmentation.

Evaluation Methods of Tissue Segmentation Techniques

In order to evaluate performance of abnormal tissue segmentation methods, various measures are used in different studies. These measures usually quantify how the output of a typical segmentation method which is a lesion mask in form of a binary image corresponds to ground truth which also is a binary image. Sensitivity, specificity, accuracy, Dice similarity coefficient, and Jaccard index are the most popular measures for assessing this correspondence. Equations 1 to 5 describe how these measures are calculated. In these equations, TP and TN are the number of pixels that are correctly identified as belonging to abnormal and normal tissues, respectively. FP and FN are the number of pixels that are incorrectly identified as belonging to lesion and normal tissues, respectively. Range of these performance measures are in the interval [0, 100]. For all of these measures, 100 is the best and 0 is the worst value.

$\mathrm{Sensitivity}=\frac{\mathrm{TP}}{\mathrm{TP+FN}}\times 100$ (1)

$\mathrm{Specificity}=\frac{\mathrm{TN}}{\mathrm{TN+FP}}\times 100$ (2)

$\mathrm{Accuracy}=\frac{\mathrm{TN+TP}}{\mathrm{TN+TP+FN+FP}}\times 100$ (3)

$\mathrm{Dice}=\frac{\mathrm{2TP}}{\mathrm{2TP+FP+FN}}\times 100$ (4)

$\mathrm{Jaccard}=\frac{\mathrm{TP}}{\mathrm{TP+FN+FP}}\times 100$ (5)

To measure performance of abnormal tissue segmentation methods, sensitivity measures the proportion of abnormal pixels that are correctly identified as abnormal; specificity measures the proportion of normal pixels that are correctly identified as normal. A perfect segmentation method has the value of 100 percent for both sensitivity and specificity. A result of 100 percent sensitive segmentation method means all lesion pixels are identified as lesion; A result of 100 percent specific method means no normal pixel is incorrectly identified as lesion. In other words, sensitivity tries to avoid false negatives while specificity tries to avoid false positives. As any attempts to increase sensitivity may not decrease specificity; these two are reported together for better understanding of segmentation performance.

In order to see true positives and true negatives together, other performance measures are needed that accuracy is one of such measures. In fact, an increase in both sensitivity and specificity leads to increasing accuracy and vice versa. Accuracy measure suffers from the fact that its measured value is reliable if the number of abnormal pixels is approximately equal to the number of normal pixels. This imbalance situation is common in abnormal tissue segmentation methods because the abnormal areas are much smaller than normal areas. Jaccard index is a quantity measuring similarity in case of asymmetry. Dice is very similar to Jaccard and measures percentage of overlap between two binary images which one of them is ground truth and the other one is the result of segmentation method.

Discussion

In the previous sections, we demonstrate abnormal tissues are segmented using a variety of techniques from simple traditional to complex ones. As abnormalities are inherently complex structures and medical images usually come up with many artifacts, conventional segmentation techniques are unable to correctly segment abnormalities. As a result, more sophisticated methods are employed to perform segmentation of abnormalities in medical images. These sophisticated methods attempt to represent tissues in a way that separation of abnormal tissues from healthy ones become an easy task.

Conclusion

In this study, segmentation techniques of abnormal tissue for MS lesion, breast cancer abnormal masses, lung nodules, and skin lesions are reviewed. As reported by the literatures, the methods based on deep learning techniques are very successful and promising for segmentation of abnormalities in medical images.