Detection of Granularity in Dermoscopy Images of Malignant Melanoma Using Color and Texture Features

Abstract

Granularity, also called peppering and multiple blue-gray dots, is defined as an accumulation of tiny, blue-gray granules in dermoscopy images. Granularity is most closely associated with a diagnosis of malignant melanoma. This study analyzes areas of granularity with color and texture measures to discriminate granularity in melanoma from similar areas in non-melanoma skin lesions.

The granular areas in dermoscopy images of 74 melanomas and 14 melanomas in situ were identified and manually selected. For 200 non-melanoma dermoscopy images, those areas which most closely resembled granularity in color and texture were similarly selected. Ten texture and twenty-two color measures were studied. The texture measures consisted of the average and range of energy, inertia, correlation, inverse difference, and entropy. The color measures consisted of absolute and relative RGB averages, absolute and relative RGB chromaticity averages, absolute and relative G/B averages, CIE X, Y, Z, X/Y, X/Z and Y/Z averages, R variance, and luminance. These measures were calculated for each granular area of the melanomas and the comparable areas in the non-melanoma images.

Receiver operating characteristic (ROC) curve analysis showed that the best separation of melanoma images from non-melanoma images by granular area features was obtained with a combination of color and texture measures. Comparison of ROC results showed greater separation of melanoma from benign lesions using relative color than using absolute color. Statistical analysis showed that the four most significant measures of granularity in melanoma are two color measures and two texture measures averaged over the spots: relative blue, relative green, texture correlation, and texture energy range. The best feature set, utilizing texture and relative color measures, achieved an accuracy of 96.4% based on area under the receiver operating characteristic curve.

I. INTRODUCTION

Contact dermoscopy [1] allows detection of granularity, [1,2], which may be seen in images of malignant melanoma. Granularity has been described variously as “granules” [3], “grey-blue to black dust-like dots” [4], “grey-blue pepper-like granules” [3], and multiple, very small, roundish dots of blue-grey color [2]. This feature is better seen with nonpolarized-light contact dermoscopy than with polarized-light dermoscopy [5].

Granularity has been found to be significantly associated with a diagnosis of melanoma, with a sensitivity of 85%, specificity of 99%, and a positive predictive value of 78% [2]. Granular areas can be seen in several types of skin lesions other than melanoma, including lichenoid keratoses. Zaballos et. al. found the granularity pattern in all of the lichenoid keratosis that they examined, however they described it as “diffuse” and “brownish-grey, reddish-brown, bluish-grey or whitishgrey diffuse granules distributed throughout the lesion” [6]. Pastar et al. described a granularity pattern in a regressing seborrheic keratosis [7]. It is also a frequent finding in desmoplastic melanoma, with half of such lesions found to display the feature [8]. Fully regressive melanoma showed regions of granularity in 43% of the lesions studied [9].

The granular, pepper-like appearance of granular areas is a visible form of texture. In dermatology imaging, this research extends texture and color analysis for characterizing semitranslucent areas for skin lesion discrimination [10]. Ten texture co-occurrence matrix characteristics introduced by Haralick et al [11] were examined in this study for analysis of granular areas, including energy average, energy range, inertia average, inertia range, correlation average, correlation range, inverse difference average, inverse difference range, entropy average, and entropy range, which are denoted as T1-T10, respectively. The average and range of these measures in four directions, including 0°, 45°, 90°, and 135°, were calculated using CVIPtools [12].

It is the primary purpose of this study to determine if melanoma can be differentiated from other skin lesions by application of color and texture measures from granular areas. The measures chosen are the texture measures defined above and standard color measures determined from the granular areas belonging to melanomas and those areas in non-melanomas which most closely resemble the granular areas in melanomas. A second purpose of this study is to determine which measures, among texture, relative color and absolute color measures, best separate melanoma from other skin lesions.

II. METHODS

The image set used for this study includes contact, non-polarized dermoscopy images [5] of 88 melanomas with granular areas and 200 non-melanoma lesions from four clinics: Skin and Cancer Associates, Plantation, FL, Columbia Dermatology and Skin Cancer, Columbia, MO, Sheard & Drugge, Stamford, CT, and the Dermatology Center, Rolla, MO, selected for dermoscopy study during January–November, 2007. All lesions with any dermoscopy features of malignancy and any benign lesions for which there was any uncertainty were biopsied. All images were contact, non-polarized dermoscopy images taken with a Sony DSC-W70 7.2 megapixel digital camera with a 3Gen DermLite Fluid dermoscopy attachment (3Gen LLC, San Juan Capistrano, CA).

The 200-image competitive set to which the melanoma set was compared consisted of a variety of lesions encountered in the clinic during the same period in which the melanoma images were acquired. These 200 lesions were both malignant and benign and consisted of a variety of diagnoses: 40 dysplastic nevi, 38 basal cell carcinomas, 21 seborrheic keratoses, 53 nevi (including 13 intradermal nevi, 12 compound congenital nevi, 11 compound nevi, 7 congenital nevi, 5 junctional nevi, 3 acral nevi, 1 compound congenital dysplastic nevus with spitzoid features, and 1 Reed nevus), 8 actinic keratoses, 7 lentigines, 6 squamous cell carcinomas in-situ, 5 invasive squamous cell carcinomas, 2 benign collision tumors and one collision tumor between a lichen planus like keratosis and squamous cell in-situ, and one between a basal cell carcinoma and seborrheic keratosis, 3 hemangiomas, 2 nevi recurrens, 2 sebaceous hyperplasias, and one from each of 11 other benign lesions.

For the 88 melanomas, a total of 74 invasive melanomas with a median Breslow depth of 0.3 mm and 14 melanomas in-situ, students (JX, DC, and AB) manually marked the most typical granular portion of all images with an approximate oval shape smoothed by a second-order spline. A dermatologist (WVS) adjusted all marked areas as needed. An invasive malignant melanoma with a typical granular area marked is shown in Figure 1. The 200 non-melanoma lesions were manually marked using the same procedure. The areas marked on the non-melanoma lesions were similar in size and as close as possible to the same color and texture as the granular spots in the melanomas. The marked areas in both melanoma and non-melanoma lesions were then analyzed for RGB and chromaticity color features as well as various texture measures based on the co-occurrence matrix.

Figure 1.

Melanoma with sample granular area marked. The pepper-like granules can be seen in other areas, especially above and to the right of the selected spot.

Color features obtained for all lesions included the following standard measures over the marked spots: average red, green, and blue pixel (R,G,B) values; average relative color (relR, relG, relB) values; average absolute R,G, B chromaticity, where chromaticity in red = R/(R+G+B) and blue and green chromaticity are similarly defined; average G/B, average relG/relB; luminance = 0.30R+0.59G+0.11B; and variance in R. The relative color values on the spots are defined as relR = Rspot - Rskin, where Rskin is the average red color of a small area surrounding the lesion, proportional to the lesion size, and relG and relB are similarly defined [14]. The Commission internationale de l'Eclairage (CIE) trichromatic CIE-XYZ averages [13] for X, Y, Z and ratios X/Y, X/Z and Y/Z for the spots were also calculated, using methods in [10]. Three additional novel relative values calculated were average relative R, G, and B chromaticity, where relative red chromaticity = relR/(relR+relG+relB), and relative blue and green chromaticity are similarly defined. There are nine absolute color features computed over the spot, including the variance in red, average R, average G, average B, average G/B, average R chromaticity, average G chromaticity, average B chromaticity, and average luminance. There are nine relative color features computed over the spot, including the variance in red, average relR, average relG, average relB, average relG/relB, average relR chromaticity, average relB chromaticity, average relG chromaticity, and average luminance.

The ten texture measures and nine color measures, either absolute or relative, were used as inputs to a standard back-propagation neural network implemented in Matlab (MathWorks Inc, Natick MA). The various network architectures used were ten texture and nine absolute or relative color measures, 19x12x6x1, ten texture measures only, 10x6x4x1 and nine absolute or relative color measures only, 9x6x4x1 (4 layers, with 9, 10, or 19 inputs, two hidden layers with 12 and 6 nodes or 6 and 4 nodes in each layer and one output node). Because of the relatively small data set, a leave-one-out methodology was used for training/test set generation. The neural network was trained up to 15 epochs or until root-mean-square error was less than 0.001. For the granular area data gathered, statistical analysis was performed using SAS software (SAS Corp., Cary, NC) [15].

III. RESULTS

Diagnostic accuracy expressed by the area under the receiver operating characteristic (ROC) curve for detection of melanoma using all 19 texture and relative color measures for granular spots using the back-propagation neural network above, is 0.964. Figures 2 and 3 show the ROC results for the 10 texture features along with the absolute and relative color features, respectively. The vertical and horizontal axes in the ROC curves refer to sensitivity and 1-specificity, respectively, for the ROC curve plots. No improvement was observed with the addition of CIE-XYZ data.

Figure 2.

ROC curves for granular results using absolute color for the combined, color-only, and texture-only feature cases using a standard back-propagation neural network with leave-one-out methodology.

Figure 3.

ROC curves for granular results using relative color for the combined, color-only, and texture-only feature cases using a standard back-propagation neural network with leave-one-out methodology.

Diagnostic results for different texture pixel distances were compared. Using only the 10 texture measures obtained at pixel distances of 2, 6, and 10, and no color measures, area under the ROC curve was respectively 0.939, 0.958 and 0.946. This optimized pixel distance of 6 gives a texel size of 0.06 mm for granular areas.

Using the SAS Proc Logistic stepwise logistic regression procedure, the measures were selected in the order shown in Table 1. The final model retains five color features, with p-values < 0.0001 for relative green and relative blue over the spots, and six texture features, with p-values 0.0065 for correlation average at a pixel distance of six and 0.0084 for texture energy range. The two relative chromaticity features retained in the logistic regression model, relative green and relative blue chromaticity, are both significant with a p-value = 0.02.

Table 1.

Results of logistic regression procedure, 288 lesions, SAS Proc Logistic, for 19 measures with relative color and texture, pixel distance = 6.

Order of Final 12 features Chosen in SAS Proc Logistic Stepwise Selection in 13 Steps
Step	Measure	Description (all measures taken over selected spot)	p-values, comments
1	Texture Entropy Average	Average information content of the texture	<0.0001 First measure chosen, highest Chi-sq. score, removed at step 10
2	Relative Blue	Granular area blue minus background blue	<0.0001
3	Relative Green	Granular area green minus background green	<0.0001
4	Texture Entropy Range	Variation of entropy along the four orientations	0.0155
5	Texture Correlation Average	Average measure of similarity between pixels at a specified distance = 6 pixels	0.0065
6	Texture Energy Range	Variation of the homogeneity along the four orientations.	0.0084
7	Relative Green Chromaticity	Relative G / (Rel. R + Rel. G + Rel. B)	0.0203
8	Texture Correlation Range	Variation of correlation along the four orientations	0.1134
9	Red Variance	Variance in red pixel value	0.0997
10	Texture Entropy Average Removed	Average information content of the texture removed	0.6378* 1st feature selected is now removed from model
11	Texture Inertia Average	Average contrast	0.0218
12	Relative Blue Chromaticity	Relative B / (Rel. R + Rel. G + Rel. B)	0.0210
13	Texture Inertia Range	Variation of contrast along the four orientations	0.1515

p-value, which measures the importance of the variable to the overall model fit, was low for texture entropy average at step 1 but by step 10, this variable no longer contributed significantly to the multivariate model and it was removed.

IV. DISCUSSION

The moderately high success rate in separation of melanoma from non-melanoma lesions on the basis of features extracted from granular areas appears promising for early detection of malignant melanoma and warrants pursuit of fully automatic granularity detection. Granularity has a sensitivity for melanoma as high as 85% and a relative risk for melanoma as high as 1.07 [2]. Our ROC analysis indicates that the use of both color and texture measures on granular spots and granular mimics are needed for optimal separation of melanoma from competing diagnoses. In the final model chosen by logistic regression for best melanoma discrimination, there are five color features and six texture features, indicating a significant contribution from both color and texture measures in this model. The most significant of all retained measures were relative blue and relative green colors, with p < 0.0001 for both measures. The curves show that diagnostic accuracy as measured by ROC curves is slightly better for relative colors, subtracting background skin color from lesion color, than the results using unmodified absolute color. Relative color was also found to give superior melanoma discrimination in a previous study on blotches in dermoscopy images [16]. Of all ten texture features, correlation average was the most significant texture feature. Correlation average was also the most significant texture feature for discrimination of melanomas from benign lesions using atypical pigment network areas [17].

Our finding of the importance of combining color and texture features to improve discrimination of dermoscopy images confirms other studies on dermoscopy images of pigmented lesions [18,19]. Studies involving other types of images including fiberoptic [20], wound [21], corneal [22], and meat images [23] have also employed combinations of color and texture features. Other studies confirm superior separation by color. For example, one texture and color-based image retrieval system determined weights for color to be higher than for texture, with red, green, blue, variance and co-occurrence matrix weights respectively 2.4, 2.0, 2.4, 0.8 and 0.9 [17], in agreement with our higher importance for color statistics than texture statistics.

A significant limitation of the study is the use of manual selection of the granular areas. Another limitation is the relatively small number of lesions in the study. A further limitation in the study is the definition of relative chromaticity. If an image has the sum of relative colors relR+relG+relB given in Methods close to zero, the denominator can become vanishingly small, yielding a very large quotient. Replacing the signed quantities with their absolute values yielded worse separation, and the definition of relative chromaticity given in Methods above was retained.

For future work, automatic detection of granular areas will be undertaken using automatic segmentation techniques based on color and texture. In addition, the techniques reported here will be applied to a larger set of lesions.

We made no attempt to select all areas of granularity. Although the presence or absence of granularity (along with other regression structures) is reproducible among experienced dermoscopy practitioners, with an interobserver agreement value к = 0.44 determined over 85 lesions [1], it is difficult to reproducibly determine the extent of granularity (Figure 1). We also made no attempt to distinguish “regular granularity” (granules that are homogeneous in size, shape and distribution) from irregular granularity (granules that are heterogeneous in size, shape, distribution and color and associated with white color), which can aid in the differentiation of melanomas from benign lesions [2]. To better accomplish differentiation of granularity in melanomas from granularity in benign lesions, future work could explore the distribution of the granular areas as well as variance in particle size, shape and color in granular areas.

V. CONCLUSION

These results show that granularity in malignant melanomas can be distinguished from similar areas in benign lesions by the application of color and texture measures. The detection of melanoma with a high degree of accuracy and differentiation from a wide variety of lesions, both malignant and benign, is also encouraging, as most studies focus on the more limited problem of discrimination of melanoma from benign pigmented lesions.