Diagnostic techniques for improved segmentation, feature extraction, and classification of malignant melanoma

Hyunju Lee; Kiwoon Kwon

doi:10.1007/s13534-019-00142-8

. 2019 Dec 7;10(1):171–179. doi: 10.1007/s13534-019-00142-8

Diagnostic techniques for improved segmentation, feature extraction, and classification of malignant melanoma

Hyunju Lee ¹, Kiwoon Kwon ^1,^✉

PMCID: PMC7046850 PMID: 32175137

Abstract

A typical diagnosis of malignant melanoma involves three major steps: segmentation of a lesion from the input color image, feature extraction from the separated lesion, and classification to distinguish malignant from benign melanomas based on features obtained. We suggest new methods for segmentation, feature extraction, and classification compared. We replaced edge-imfill method with U-Otsu method for segmentation, the previous features with new features for the criteria ABCD (asymmetry, border irregularity, color variegation, diameter) criteria, and the median thresholding with weighted receiver operating characteristic thresholding for classification. We used 88 melanoma images and expert’s segmentation. All the three steps in the suggested method were compared with the steps in the previous method, with respect to sensitivity, specificity, and accuracy of the 88 samples. For segmentation, the previous and the suggested segmentations were also compared assuming the skin cancer expert’s segmentation as a ground truth. All three steps resulted in remarkable improvement in the suggested method.

Keywords: Malignant melanoma, ABCD criteria, Image segmentation, Classification

Introduction

The incidence of melanoma has increased in the USA over the past 30 years and melanoma is dreaded for its rapid metastasis. Invasive melanoma constitutes about 1% of all skin cancers but accounts for a large proportion of skin cancer-related deaths. In the United States, 91,270 people were diagnosed with melanoma in 2018, which represents more than 91% of all skin cancers diagnosed, resulting in 9320 deaths [1]. Therefore, it is important to develop technologies to be used in the early diagnosis of malignant melanoma, since even experienced dermatologists are fatigued and have difficulty in diagnosing malignant melanoma. Various studies since the late 1990s for the automated analysis of dermatoscopy images have the potential to serve as adjuncts to physicians improving clinical managements [2].

Automated diagnosis of malignant melanoma usually requires three important steps: segmentation of lesion from the input color image, feature extraction from the separated lesion, and the differentiation malignant from benign melanomas based on the unique features. There are many works for these automated diagnosis [2–5].

Several segmentation algorithms have been used to classify melanoma [6–11]. In this study, the Otsu thresholding segmentation [12] has been used to separate lesions from the background. To improve the contrast, we applied Otsu method to U channel for YUV color representation of the image. The k-means method with k = 2 and k = 3 were also used to cluster two to three groups, respectively, inside the lesion to quantify color variegation.

For feature extraction from the segmented lesion, we used ABCD criteria [13], which are more popular than Menzis [14] and 7-point checklist [15] methods. The ABCD criteria can be used to distinguish malignant from benign melanoma based on the following four features: (A) asymmetric shape of the lesion (asymmetry), (B) irregular border of the lesion (border irregularity), (C) non-unifrom lesion color (Color variegation), and (D) lesion diameter greater than 6 mm (diameter). We did not investigate criterion D (Diameter) in further detail since the computation of the parameter is straightforward if the pixel size is known. The authors performed mathematical analysis of ABCD criteria [16] and compared their efficiency with previous studies [17, 18]. The quantification of ABCD criteria [16] was further improved in this study for B and C criteria.

The features extracted from the segmented lesion can be used in various methods of classification such that total dermatoscopy score (TDS) [19], k-nearest neighbor (kNN) [20], support vector machine (SVM) [21] and artificial neural networks (ANN) [20]. We used simple thresholding-based classification reported previously [16] and in this study. The previous median thresholding values for the three features derived from criteria A, B, C were improved to select the best thresholding value in receiver operating characteristic (ROC) space for the weighted sum of the features.

Overall, we suggest Otsu method for U-channel for segmentation, new features for criteria B and C, and thresholding-based classification considering ROC space and weighted sum of the three features. We compared the classification of the three new steps with the previous method [16]. The segmentation step was also compared with respect to the expert’s segmentation [22] as a ground truth.

Materials and methods

The outlines of the previous and the present major algorithms are as follows:

Previous method [16]

Segmentation: Edge-imfill method
Feature extraction: ABC criteria $(m_{a}, m_{b}, m_{c})$
Classification: Median thresholding

Current method

Segmentation: U-Otsu method
Feature extraction: ABC criteria $(m_{a}, m_{b}^{*}, m_{c}^{*})$
Classification: Weighted ROC thresholding

In the segmentation step of both methods, a median filter was used as a preprocessing step and morphological close and hole-filling operations were used for postprocessing analysis. In the feature extraction step, $m_{b}^{*}$ and $m_{c}^{*}$ were suggested afresh for B and C criteria.

In the previous case [16], we used 24 (13 benign and 11 malignant) melanoma images [23, 24]. By contrast, in the present study, we used 88 (52 benign and 36 malignant) melanoma images obtained from the international skin imaging collaboration website [22].

For the evaluation of the suggested method, we used the accuracy, the sensitivity, and the specificity, which is widely used as a binary classification test. For the evaluation of segmentation methods, we used the skin cancer expert’s segmentation [22] as a ground truth. To compare the efficiency of two segmentation methods, ‘U-Otsu method’ and ‘Edge-imfill method’, we used the following terminologies:

G: Ground truth segmentation of the expert.

$T_{1} :$ Segmentation by ‘Method 1’.

$T_{2} :$ Segmentation by ‘Method 2’

\begin{matrix} S_{i} & = \frac{|T_{i} \cap G|}{|T_{i} \cup G|}, i = 1, 2, \\ R & = \frac{S_{1} - S_{2}}{max (S_{1}, S_{2})} \end{matrix}

$1 - S_{i}, i = 1, 2$ is known as Jaccard Index, which is well known segmentation similarity measure [25]. Let us call R the synchro rate of comparison parameter between ‘U-Otsu method’ and ‘Edge-imfill method’. Based on this terminology and the assigned value $δ$ , we will call.

‘U-Otsu method’ is better than ‘Edge-imfill method’ if $R > δ .$

‘Edge-imfill method’ is better than ‘U-Otsu method’ if $R < - δ .$

‘U-Otsu method’ is similar to ‘Edge-imfill method’ if $|R| \leq δ .$

Using these two efficiency measures, we compared ‘the previous method’ and ‘the suggested method’.

Segmentation

In the previous study, we first regularized an intensity matrix (R + G + B)/3 of the input color image with the $9 \times 9$ average filter and converted the regularized matrix to a black and white image by taking threshold value as the average of the minimum and maximum values of the matrix. Then the edge was found by using the Canny method and possible holes inside of the edge was filled. For the criterion C, we further clustered two groups inside the segmented lesion by using k-means method with k = 2. We designated the previous segmentation method as Edge-imfill method.

As shown in Fig. 1, segmentation by Edge-imfill method is usually smaller than the expert’s segmentation. To overcome this shortcoming, we changed RGB color image into the YUV color system that was created to represent video and image color space.

We segmented the input color image applying the Otsu method [12] to the U channel. We found that the background carried uniform U channel values as shown in Fig. 1 rather than Y or V channel. Therefore, we suggest U-Otsu method in this study. The efficiency of Otsu method to U-channel is shown in Sect. 3.

Feature extraction: ABC criteria

Based on the segmented lesion, we extracted features to distinguish malignant from benign melanomas. The quantification of ABC criteria attracted many researchers investigating the diagnosis of malignant melanoma. In the previous method [16], we used features $(m_{a}, m_{b}, m_{c})$ for the quantification of ABC criteria. In this study, we extracted new features $m_{b}^{*}$ and $m_{c}^{*}$ for the criteria B and C.

Asymmetry $m_{a}$

Assume $D, p_{D}, D^{*}$ as the segmented lesion, the center of the circumscribed rectangle of $D$ , and a region symmetric to $D$ with respect to $p_{D}$ , respectively. Then, $m_{a}$ is defined as follows:

m_{a} = \frac{1}{2} \frac{|D \ D^{*}| + |D^{*} \ D|}{|D|}

The quantity $m_{a}$ for the criterion A is a real number varying between 0 and 1. If $m_{a}$ is close to 1 the lesion is considered malignant melanoma, and if $m_{a}$ is close to 0 then the lesion is a benign melanoma. This quantity $m_{a}$ was also used in the present study. Note that we use the center of the circumscribed rectangle of $D$ for simplicity of computation, but it is the same as the center of $D$ if $D$ is symmetric [16].

Border irregularity $m_{b}$ and new parameter $m_{b}^{*}$

Assume $\hat{D}$ as the convex hull of the lesion $D$ and $\partial D, \partial \hat{D}$ as the boundary of $D$ and $\hat{D}$ , respectively, then

m_{b} = \frac{1}{|\partial \hat{D}|} \sum_{x \in \partial D} dist (x, \partial \hat{D})

where $dist (x, B) = min_{y \in B} |x - y|$ . Assuming that $\partial D$ is irregular, it is probable that several boundary pixels exist inside the convex hull $\hat{D}$ and the average distance between $\partial D$ and $\partial \hat{D}$ has been reported before [16].

The feature $m_{b}$ focuses on the difference between the lesion and the convex hull of the lesion. However, in this study, we considered the distance from the center of gravity of the lesion. The distance function from the center may fluctuate strongly for irregular boundary suggesting the need to replace $m_{b}$ with the new feature $m_{b}^{*}$ .

In the present method, $m_{b}^{*}$ is defined by the average difference of nearby extreme values for the distance function from the center $p_{D}$ to $\partial D$ ignoring a difference less than $ϵ *dist (p_{D}, \partial D)$ such that the computation does not depend on the possible noise when the boundary from the segmentation step is detected. We assumed that $D$ is star convex with respect to $p_{D}$ , that is, $D$ contains all straight lines from $p_{D}$ to $\partial D$ except the end point on $\partial D$ . The central $p_{D}$ is represented as a red star in Fig. 2a. The algorithm used to calculate is as follows:

Calculate the distance function $d (x) = |x - p_{D}|$ for $x \in \partial D$ as the blue lines in Fig. 2b.
Find extreme points set for the distance function $d (x)$ . The extreme values for corresponding extreme points are plotted as red lines in Fig. 2c.
Store the absolute values of the difference between two nearby extreme values into the set $E$ . For example, find the difference between the red line and the green line, the difference between the green line and the blue line, and so on as shown in Fig. 2d.
Let $F = \{y \in E |y > ϵ * dist (p_{D}, \partial D))\}$ . Then, $m_{b}^{*}$ is defined as follows:
$m_{b}^{*} = \{\begin{matrix} m e a n (F) & i f F \neq ϕ, \\ m e a n (E) & i f F = ϕ . \end{matrix})$

Fig. 2 — Process of obtaining $m_{b}^{*}$ a a lesion D and the center $p_{D}$ (red dot), b the distance from $p_{D}$ to $\partial D$ (blue line), c extreme values (red lines) and nonextreme values (dotted blue lines) for the distance function $d (x)$ , d extreme values (red, green, and blue lines). (Color figure online)

Note that $m_{b}^{*} > ϵ d$ if $F \neq ϕ$ and $m_{b}^{*} < ϵ d$ if $F = ϕ$ . Due to discretization, the distance function on the boundary $\partial D$ fluctuate for every pixel from nearby pixels. Therefore, it is possible that the number of the difference of the extreme values set E could be the number of $\partial D$ itself. To eliminate small perturbation from discretization error, we introduced the set F which exclude the difference of extreme values whose distance from the center is smaller than $ϵ$ times maximum distance. We selected $ϵ = \frac{1}{7}$ in this study. It is expected that the greater the oscillation of the distance $d (x)$ of the boundary $\partial D$ from the central $p_{D}$ , the greater the $m_{b}^{*}$ value.

Color variegation $m_{c}$ and new parameter $m_{c}^{*}$

The segmented lesion $D$ was further clustered into two groups $G_{1}$ and $G_{2}$ using 2-means clustering and into three groups $G_{3}, G_{4},$ and $G_{5}$ using 3-means clustering as shown in Fig. 3.

Fig. 3 — Result of k-means clustering for k = 2 (c, d) and k = 3 (e–g)

Let $c_{i}$ be the center of pixel values in each group $G_{i} (i = 1, 2, 3, 4, 5)$ and $c^{*}$ be the center of pixel values in the background region $D^{c} .$ Let the brightness follow an increasing order $|c_{3}| > |c_{4}| > |c_{5}|$ Let $c_{45}$ be the center of $G_{4} \cup G_{5},$ resulting in $|c_{4}| > |c_{45}| > |c_{5}|$ .

In the previous method, we quantified the parameter $m_{c}$ as the distance between $c_{1}$ and $c_{2}$ as follows: $m_{c} = \frac{∥c_{1} - c_{2}∥}{2^{n} \sqrt{3}},$ where $n$ denotes the bit number of the input color image and $m_{c}$ is a real number ranging between 0 and 1. The bigger the $m_{c}$ , the more variegated was the color of the lesion.

When we use k-means method, the required cluster number should be verified. In [16], the silhouette value for k = 2 was remarkably better than k = 3, 4, 5, 6 for all the 8 samples we tested in the study. The 88 samples used in this study were more diverse than the 24 samples used in the paper. Therefore, we suggest more advanced method for criterion C not only considering k = 2 but also k = 3. If the difference between $c_{3}$ and $c^{*}$ was less than the difference between $c_{3}$ and $c_{45}$ , the segmented lesion was considered a part of the background $D^{c}$ , which may be missed when we segment the lesion via k-means with k = 2. Therefore, we excluded $G_{3}$ from the lesion and computed $m_{c}^{*}$ only using $c_{4}$ and $c_{5}$ as follows:

m_{c}^{*} = \{\begin{matrix} \frac{∥c_{4} - c_{5}∥}{2^{n} \sqrt{3}} & i f ∥c_{3} - c^{*}∥ \leq ∥c_{3} - c_{45}∥, \\ m_{c} & o t h e r w i s e . \end{matrix})

The higher the $m_{c}^{*}$ value, the more different were the color in the lesion.

Classification

In feature extraction step, all the previous features $m_{a}, m_{b}, m_{c}$ and new features $m_{b}^{*}, m_{c}^{*}$ were designed to suggest an increased possibility of malignant melanoma if the values were higher. The next step is to select the appropriate thresholding value to determine the malignancy of the lesion. In the previous method, the median thresholding values were calculated for each extracted feature ( $m_{a}, m_{b}, m_{c}$ ) from an image and if the three features were less than the corresponding median thresholding values at the same time, the image suggested benign melanoma. Otherwise, it was diagnostic of malignant melanoma.

To improve the classification efficiency, we used ROC curve to display performance efficiency pairs, the specificity and the sensitivity, obtained by increasing the thresholding values from 0 to 1, which is frequently used in decision making algorithm [26–28]. The horizontal axis (x-axis) represents specificity and the vertical axis (y-axis) represents sensitivity. The area under this curve is called the Area Under Curve (AUC). The closer the AUC value is to 1, the more accurate the model is and we selected the best threshold with the sensitivity and specificity pair closest to (1, 1). That is, the ROC thresholding value was determined as follows: We increase the threshold values from 0 to 1 by 0.01 for the features $m_{a}, m_{b}^{*}, m_{c}^{*}$ , respectively, to select the ROC thresholding value closest to (1, 1).

Total Dermatoscopy Score (TDS) is used in ABCD criteria [19]. But in this paper, we select the weight for ( $m_{a}, m_{b}^{*}, m_{c}^{*}$ ) by using ROC analysis explained above. In the present Weighted ROC method, we selected the thresholding value along with the positive weights $w_{1}, w_{2}$ for the highest sensitiviy among the ROC thresholding values for all the weighted features $W = w_{1} m_{a} + (1 - w_{1}) (w_{2} m_{b}^{*} + (1 - w_{2}) m_{c}^{*})$ .

In summary, the thresholding value was determined by the following thresholding methods:

Median threshold

The average of maximum and minimum of extracted features, respectively, which is used in [16].
ROC threshold

The best thresholding values for each feature, respectively, with specificity and sensitivity close to (1, 1) in ROC space.
Weighted ROC threshold (suggested)

The best thresholding value along with positive weights $w_{1}, w_{2}$ whose sensitivity was highest among ROC thresholding values for the features $W = w_{1} m_{a} + (1 - w_{1}) (w_{2} m_{b}^{*} + (1 - w_{2}) m_{c}^{*}) .$

Results

Our suggested methods were U-Otsu segmentation, $(m_{a}, m_{b}^{*}, m_{c}^{*})$ feature extraction based on ABC criteria, and classification with weighted ROC thresholding value. For comparison with the previous method or other methods, we just replaced the step of interest. For example, to compare the suggested U-Otsu with the previous Edge-imfill segmentation, the other steps were fixed as suggested such as $(m_{a}, m_{b}^{*}, m_{c}^{*})$ feature extraction and weighted ROC thresholding.