Skin color classification of Koreans using clustering

Abstract

Background/purpose

Skin color is used as an index for diagnosing and predicting skin irritation, dermatitis, and skin conditions because skin color changes based on various factors. Therefore, a new method for consistently and accurately evaluating skin color while overcoming the limitations of the existing skin color evaluation method was proposed, and its usefulness was demonstrated.

Methods

Skin color was quantified using the RGB (Red, Green, Blue), HSV (Hue Saturation Value), CIELab, and YCbCr color spaces in the acquired Korean skin images, which were classified through clustering. In addition, the classification performances of the existing visual scoring method and the proposed skin color classification method were compared and analyzed using multinomial logistic regression, support vector machine, K‐nearest neighbor, and random forest.

Results

After quantifying the skin color through the color space conversion of the skin image, the skin color classification performance according to the number of quantified features and the classifier was verified. In addition, the usefulness of the proposed classification method was confirmed by comparing its classification performance with that of the existing skin color classification method.

Conclusion

In this study, a method was proposed to objectively classify skin color values quantified from skin images of Koreans acquired using a digital camera through clustering. To verify the proposed method, its classification performance was compared with that of the existing classification method, and an optimized classification method was presented for the classification of Korean skin color. Thus, the proposed method can objectively classify skin color and can be used as a cornerstone in research to quantify skin color and establish objective classification criteria.

1. INTRODUCTION

Skin color is the most noticeable feature of the skin, and as more people want to lead a young and healthy life socially, interest in healthy skin is increasing, and the interest in skin color, as well. Skin color, which is mainly determined by melanin and hemoglobin, changes depending on various factors such as carotene, bilirubin, blood flow, and the thickness of the stratum corneum. The degree of change in skin color varies from person to person. ¹

Changes in skin color that vary depending on various factors and people are used as indicators to predict diseases and confirm treatment reactions in the medical field, such as evaluating the possibility of skin cancer using pigmentation and establishing a light treatment plan based on erythema reactions caused by ultraviolet exposure. ² Because skin color changes depending on tissue damage or metabolic abnormalities in trauma or chronic disease, it is also used as an index to evaluate disease severity. ³ Therefore, if skin color is not accurately evaluated, it is highly possible to lead to problems such as worsening of skin diseases, misdiagnosis, and lack of timely treatment, resulting in internal diseases or infectious complications. ⁴

However, in clinical practice, skin color evaluation is mainly performed by visual scoring, which is a subjective evaluation based on the diagnostic experience of a specialist; therefore, even the same skin color may be evaluated differently by each specialist. ⁵ , ⁶ In addition, because the visual evaluation recognizes a visible skin color differently depending on lighting conditions, such as the distribution of the light source and illuminance level, the skin color can be evaluated differently depending on the lighting environment. Therefore, an objective and accurate evaluation of skin color is required, and it is necessary to quantify skin color in the same environment and conduct research to establish and verify the objective evaluation criteria.

Measurement tools developed to quantify skin color can provide objective skin color information by accurately measuring the color values. ⁷ However, it has disadvantages of being expensive and inconvenient to be widely used in clinical practice and can be measured only at a single point. In addition, to measure skin color using a measuring tool, the probe must be in contact with the skin. Because the pressure applied to the skin during contact has a significant effect on skin color measurement, objectivity is not guaranteed. ⁸ , ⁹ While overcoming the limitations of these skin color measurement tools, there is a need for a method suitable for use in situations where it is difficult to use the tool, or when it is difficult to spend time such as visiting a specialized institution for skin color evaluation. Therefore, it is necessary to propose a new method for objectively evaluating and classifying skin color using skin images captured by a device such as a digital camera, which is common to the general public, and to verify its usefulness for this purpose.

In this study, prior to establishing objective classification criteria using quantified skin color, skin surface images were acquired from Koreans using a digital camera, and skin color was quantified through color space conversion from the acquired skin images. We attempted to verify the superiority and usability of the proposed method by performing clustering using quantified skin color values and comparing the classification performance with existing skin color evaluation methods.

2. MATERIALS AND METHODS

2.1. Image acquisition device

Skin images were acquired using a digital single‐lens reflex camera (Canon EOS 30D, Canon Inc., Tokyo, Japan) and a macro lens (Tamron SP AF60mm F/2 Di II LD 1:1 Macro lens, Tamron Co., Ltd., Saitama, Japan). The camera specifications were set at ISO 4.0, aperture of F4.0. During shooting, an LED (Light‐Emitting Diode) ring light (Aputure Amaran AHL‐C60 Halo LED Ring Flash, Aputure Imaging Industries Co., Ltd., Shenzhen, China) was used to maintain uniform illuminance. In addition, by connecting a cylindrical light‐blocking module with a diameter of 10 cm and length of 12 cm in front of the LED ring light, it was illuminated only by the LED ring light without being disturbed by ambient light, and it was possible to shoot maintaining a certain distance.

2.2. Subjects and measurement criteria

A total of 595 skin images were obtained from Korean subjects (24.2 years old ± 2.36) having no skin diseases. When taking pictures of a subject's body to measure skin color, the body parts were not restricted to acquire various skin color data. Although the skin surface was photographed by irradiating with light of uniform brightness using an LED ring light, shadows occurred in the acquired image owing to the curved body. Therefore, photography was conducted after attaching a tape, with a circular hole of diameter 5 mm, to the flat part of the skin that received light uniformly (Figure 1).

FIGURE 1.

An example of a skin image of a Korean obtained after attaching a tape with a 5 mm round hole to the skin

2.3. Skin color classification

A method for quantifying skin color in images obtained from a digital camera and classifying skin color based on quantified values was proposed. The proposed method consists of five steps:

2.3.1. Step 1. Region of interest

When photographing the skin surface, to increase the accuracy of the skin color, a tape with circular hole of diameter 5 mm, was attached to a relatively flat and uniform area that received light. In the image, the area, except for the circular hole, was separated and removed. In the original image of 3504 × 2336 pixels, 450 × 450 pixels were set as the region of interest (ROI) based on the center of the area where the circular hole was perforated (Figure 2).

FIGURE 2.

The region of interest (ROI) area set in the skin image. (A) Selected ROI region; (B) cropped ROI region

2.3.2. Step 2. Skin color quantification using color space conversion

The color space is a mathematical model that expresses color information in terms of different color components and expresses color correlation by recognizing it as a three‐dimensional space. ¹⁰ In this study, skin color was quantified by converting ROI images obtained from 595 24‐bit full‐color images into four color spaces: RGB, HSV, CIELab, and YCbCr.

2.3.3. (a) RGB

The RGB color space is a basic color space consisting of three elements: red, green, and blue. Because of its simplicity, it is used to store and represent digital images, and different color spaces can be obtained through linear or nonlinear transformations. ¹⁰

2.3.4. (b) HSV

The HSV color space is expressed by three components: hue, saturation, and value. This is similar to the way humans judge colors and is more intuitive than the RGB color space because it expresses the color itself rather than a combination of colors. Therefore, it is useful for detecting objects using the colors in an image. ¹¹ , ¹²

2.3.5. (c) CIELab

The CIELab color space consists of brightness (L*) and color components (a*, b*), where a* and b* represent the green‐red and blue‐yellow components, respectively. ¹³

2.3.6. (d) YCbCr

The YCbCr color space is expressed by separating Y, representing luminance, and Cb and Cr, denoting color‐difference information in the RGB color space. ¹⁰

2.3.7. Step 3. Principal component analysis

Because 12 channels of RGB, HSV, CIELab, and YCbCr color spaces were used to quantify the skin color in the skin images, the quantified data consisted of 12 dimensions. The higher the dimensionality of the data, the greater the possibility that the processing performance deteriorates owing to the complexity. Principal component analysis (PCA) aims to find and project an axis that preserves the variance of the existing data as much as possible. ¹⁴ As shown in Equation (1), converting to a value with a Gaussian normal distribution with an average value of 0 and variance value of 1 through data standardization. Then generate the covariance matrix. Set the appropriate number of principal components by calculating the eigenvectors, eigenvalues and cumulative contribution rate using the covariance matrix.

$$X_{i\_\mathrm{new}}=\frac{x_i-\mathrm{mean}\left(x\right)}{stdev\left(x\right)}$$ (1)

After data standardization, generate the covariance matrix “Cov” of feature vector shown in Equation (2).

$$Cov=\left[\begin{array}{ccc}\begin{array}{cc}{\sigma }_{11}& {\sigma }_{12}\\ {\sigma }_{21}& {\sigma }_{22}\end{array}& \begin{array}{c}\cdots \\ \text{…}\end{array}& \begin{array}{c}{\sigma }_{1n}\\ {\sigma }_{2n}\end{array}\\ \begin{array}{cc}⋮& ⋮\end{array}& \ddots & ⋮\\ \begin{array}{cc}{\sigma }_{n1}& {\sigma }_{n2}\end{array}& \cdots & {\sigma }_{nn}\end{array}\right]$$ (2)

We perform PCA for two cases: (1) each color space; (2) all four color spaces. Therefore, feature vector consists of 3 and 12 features when PCA performed for each color space and all four color spaces respectively. And each covariance matrix formed as an [3 × 3] and [12 × 12] square matrix.

2.3.8. Step 4. Hierarchical clustering

Hierarchical clustering is an algorithm that forms a cluster by sequentially integrating each entity into a similar entity by using a hierarchical tree model. To classify similar data into the same group among the color values quantified through color space conversion in the skin image, bottom‐up agglomerative clustering was performed in which all data started as a cluster and merged with the surroundings. ¹⁵

2.3.9. Step 5. Evaluation of classification performance

To objectively classify skin color in the skin color range of Koreans, we measured and compared the classification performance using four classifiers to propose an optimized method: multinomial logistic regression (MLR), support vector machine (SVM), K‐nearest neighbor (K‐NN), and random forest (RF). In addition, the significance of the proposed method was verified by measuring the classification performance of visual evaluation, which is an existing skin color evaluation method.

MLR

Logistic regression is a supervised learning algorithm that applies a linear regression method for classification to predict which category the data belong to as a probability and classifies it with a higher probability. ¹⁶ Because this study corresponds to multiple classifications that classify skin color into five categories, the classification performance was evaluated using the multiple logistic regression method with the softmax function, as shown in Equation (3). ¹⁷

$$\mathrm{softmax}\left(x\right)=\frac{e_i^x}{{\sum }_{j=0}^k{e}^{x_i}}\left(\mathbf{i}=\mathbf{0},\mathbf{1},\text{…},\mathbf{k}\right)$$ (3)

SVM

The SVM is a binary linear classification model that determines the category to which a given dataset belongs. In this study, a one‐to‐one approach was used for classification into five categories, and a radial basis function kernel was used. ¹⁸

$$K\left(x,x^{\prime }\right)=\mathbf{exp}\left(\frac{x-{x^{\prime }}^2}{2{\sigma }^2}\right)$$ (4)

K‐NN

The K‐NN algorithm finds K pieces of data that are close to the current data in the entire dataset and classifies the current data with the largest number of labels among the data labels. ¹⁹ , ²⁰ The Euclidean distance is generally used for the distance between data points, and the result depends on the K value. In this study, classification performance was measured by setting the K value to 5

RF

RF is classified by learning the sampled data by bootstrapping according to the bagging method, as there are several decision trees, predicting individually and determining the final prediction value through the average or majority vote of the prediction results. ²¹ , ²² In this study, classification performance was measured by setting the number of decision trees to 100.

3. RESULTS

3.1. PCA

In this study, the color values were quantified from skin images acquired using four existing color spaces: RGB, HSV, CIELab, and YCbCr. Hence, one data had a total of 12 features. The more features the data have, the more likely the processing performance may decrease; hence, the number of features of the data was reduced through PCA in this study. PCA was performed for each of the four color spaces. When PCA was performed, eigenvalue and cumulative contribution rates were calculated. In this case, the appropriate number of principal components was set based on principal components with an eigenvalue of 0.7 or more and a cumulative contribution rate of ≥70%. ²³

When performing PCA for each color space, each color space has three‐dimensional data. Table 1 presents the results of calculating the eigenvalue and cumulative contribution rate for each color space to obtain the appropriate number of principal components. First, as a result of principal component analysis of the RGB color space, when only the first principal component was used, the eigenvalue was 0.7 or more, and the cumulative contribution rate was ≥70%. When PCA was performed on the HSV, CIELab, and YCbCr color spaces, when used until the second principal component in all three color spaces, the eigenvalue was 0.7, or more and the cumulative contribution rate was 70% or more. Consequently, when PCA is performed for each color space, it is appropriate to reduce the three‐ to one‐dimension for RGB and the three to two‐dimensions for HSV, CIELab, and YCbCr. Table 2 shows the principal components of each color space, which is composed of linear combinations.

TABLE 1.

Eigenvalue and cumulative contribution rate for each color space

	RGB			HSV
	Eigenvalue	Cumulative contribution rate		Eigenvalue	Cumulative contribution rate
PC1	2.6001	0.8652	PC1	1.3655	0.4544
PC2	0.3545	0.9832	PC2	0.9988	0.7868
PC3	0.0505	1.0000	PC3	0.6407	1.0000

	CIELab			YCbCr
	Eigenvalue	Cumulative contribution rate		Eigenvalue	Cumulative contribution rate
PC1	1.7668	0.5880	PC1	1.8419	0.619
PC2	0.7918	0.8514	PC2	0.9226	0.9199
PC3	0.4464	1.0000	PC3	0.2406	1.0000

TABLE 2.

Linear combination of principal components of each color space: RGB, HSV, CIELab, YCbCr

	RGB	HSV
PC1	(−0.5495× R)+(−0.6096× G)+(−0.5713× B)	(0.5140× H)+(0.7057× S)+(−0.4876× V)
PC2	(0.7722× R)+(−0.1056× G)+(−0.6258× B)	(0.6879× H)+(0.0004× S)+(0.7258× V)
PC3	(−0.3189× R)+(0.7851× G)+(−0.5310× B)	(0.5124× H)+(−0.7085× S)+(−0.4852× V)

	CIELab	YCbCr
PC1	(−0.4707× L)+(0.6375× a)+(0.6099× b)	(−0.2945× Y)+(−0.6729× Cb)+(0.6786× Cr)
PC2	(0.8722× L)+(0.2319× a)+(0.4307× b)	(0.9552× Y)+(−0.2298× Cb)+(0.1867× Cr)
PC3	(−0.1331× L)+(−0.7347× a)+(0.6652× b)	(−0.0303× Y)+(−0.7031× Cb)+(−0.7104× Cr)

Next, when PCA is performed on all the four color spaces, because all four color space channels of RGB, HSV, CIELab, and YCbCr are used, 12‐dimensional data are obtained. Table 3 presents the results of calculating the eigenvalue and cumulative contribution rate for each color space to obtain the appropriate number of principal components. When used up to the second or third principal component, the eigenvalue was 0.7 or more, and the cumulative contribution rate was 70% or more. Table 4 shows the principal components for all four color spaces, which is composed of linear combinations.

TABLE 3.

Eigenvalue and cumulative contribution rate for all four color spaces

	Eigenvalue	Cumulative contribution rate
PC1	6.5166	0.5421
PC2	3.7605	0.8550
PC3	1.8310	0.9699
PC4	0.3580	0.9996

TABLE 4.

Linear combination of the principal components of multicolor spaces

	All four color spaces
PC1	(−0.2578× R)+(−0.3678× G)+(−0.3892× B)+(0.0601× H)+(0.3462× S)+(−0.2578× V)+ (−0.3505× L)+(0.2269× a)+(0.2534× b)+(−0.3571× Y)+(−0.2139× Cb)+(0.2121× Cr)
PC2	(0.3847× R)+(0.1576× G)+(−0.0085× B)+(0.1237× H)+(0.2357× S)+(0.3847× V)+ (0.2290× L)+(0.2677× a)+(0.3603× b)+(0.2113× Y)+(−0.3933× Cb)+(0.3938× Cr)
PC3	(−0.0900× R)+(0.1209× G)+(−0.0773× B)+(0.7461× H)+(0.0480× S)+(−0.0900× V)+ (0.0437× L)+(−0.4748× a)+(0.2029× b)+(0.0359× Y)+(−0.2360× Cb)+(−0.2806× Cr)

3.2. Skin color classification using traditional method

To objectively classify skin color and compare it with the method proposed in this study, the skin color was evaluated through visual scoring using the existing skin color evaluation method. A general experimenter performed the classification twice for a total of 595 skin images obtained, and the visual grade (VG) was evaluated through a review by a specialist (Figure 3).

FIGURE 3.

An example of classification results according to the existing visual evaluation method. (A) Very light 0, (B) light 1, (C) medium 2, (D) dark 3, (E) very dark 4

3.3. Skin color classification performance

In this study, 80% of the total data were randomly chosen as the training dataset and 20% as the test dataset. Accordingly, of total 595 skin color data, 476 were used as training data, and 119 were used as test data. Data partitioning was equally applied to each method.

The VG of the traditional method, quantified data that reduced each color space to an appropriate number of principal components (PCA 3toprop), quantified data that reduced all four color spaces to two dimensions (PCA 12to2), and quantified data that reduced all four color spaces to three dimensions (PCA 12to3) were measured using MLR, SVM, K‐NN, and RF. Because the test datasets were not uniformly distributed, the classification performance was measured by obtaining the accuracy, F1 score, and area under the curve; the results are shown in Table 5.

TABLE 5.

The results of the classification performance measurement

	VG			PCA 3 to prop
	Accuracy	F1 score	AUC	Accuracy	F1 score	AUC
MLR	0.6218	0.5593	0.7163	0.8992	0.9159	0.9420
SVM	0.4706	0.2156	0.5501	0.9328	0.9375	0.9694
K‐NN	0.6555	0.6558	0.7804	0.9748	0.9750	0.9845
RF	0.8067	0.7898	0.8766	0.9412	0.9336	0.9593

	PCA 12 to 2			PCA 12 to 3
	Accuracy	F1 score	AUC	Accuracy	F1 score	AUC
MLR	0.9328	0.9423	0.9634	0.8655	0.8679	0.9099
SVM	0.9496	0.9589	0.9711	0.9496	0.9561	0.9694
K‐NN	0.9748	0.9555	0.9640	0.9832	0.9774	0.9845
RF	0.8908	0.9037	0.9322	0.9328	0.9429	0.9593

Abbreviations: AUC, Area Under the ROC Curve; K‐NN, K‐nearest neighbor; MLR, multinomial logistic regression; PCA, principal component analysis; RF, random forest; SVM, support vector machine; VG, visual grade.

The classification performance of the visual evaluation, the traditional method, was lower than the four classifiers which used in this study. This means that the evaluation of skin color by visual scoring is based on inconsistent criteria. In addition, as skin color classification using quantified color data is better than the classification performance of visual evaluation, it indicates that skin color can be objectively classified based on skin color values quantified using the color space. Moreover, when classifying using the proposed method, the classification performance of K‐NN was the highest among the classifiers used. This means that the K‐NN classifier is the most optimized method when attempting to objectively classify skin color in the skin color range of Koreans considered in this study.

4. DISCUSSION

As lifestyles change from living long to living young, interest in disease‐free and healthy skin, especially skin color that is quickly noticeable externally, is increasing. Changes in skin color caused by various factors are used as an index to confirm disease progression or response to treatment. However, because the evaluation of current skin color is mainly made through visual scoring, it can be evaluated differently depending on the diagnostic experience of a specialist, and the perspective of the person affected by light recognizes different colors according to changes in the lighting environment.

Therefore, to evaluate skin color objectively and accurately, it is necessary to conduct studies to quantify and establish standards for classifying skin color. However, research related to skin color is mainly conducted on the detection and division of areas, such as the face and hands, and studies to establish classification standards through skin color quantification are insignificant. ²⁴

Therefore, in this study, skin color quantification was performed using a color space commonly used in Korean skin images acquired with a digital camera, and a method for objectively classifying skin color by clustering quantified skin color values was proposed. By comparing with the classification performance of the existing visual evaluation method, the feasibility of the proposed method was proven, and an optimized classifier was presented for the classification of skin color in Koreans.

In this study, visual evaluation was performed using the existing evaluation method for comparison with the proposed method. However, in this case, an unskilled general experimenter first classified the skin color through a visual evaluation. And then, the classification was determined through review by a specialist. This is limited by the fact that, even if a specialist's review was conducted, the initial evaluation was performed by an unskilled general public; therefore, it is less reliable than the evaluation performed by a specialist in clinical practice. Therefore, by performing a visual evaluation of Korean skin images obtained and comparing and presenting visual evaluation results of specialists and the general public, we would like to provide evidence for the possibility of clinical use of the method proposed in this study.

In addition, this study did not consider melanin and hemoglobin levels, which are the main factors that determine skin color. Because it is difficult to investigate the relationship between these markers and skin color in various parts of the body, it is difficult to establish a classification standard for these markers for skin color classification. Follow‐up studies are necessary to overcome these limitations.

This study confirms that it is feasible to classify skin colors using multicolor space values obtained from skin images acquired with a digital camera. This is expected to establish more precise and objective classification criteria by applying them to existing studies that use only a single color space to objectively classify the severity of erythema, an indicator of psoriasis or atopic dermatitis. ²⁵ This study can contribute to the development of smart healthcare technology and the universalization of skin color diagnosis by facilitating continuous change observation without time and space constraints by presenting as a prior study to quantify skin color.

5. CONCLUSIONS

In this study, skin surface images were acquired using a digital camera, which is relatively common to the general public. A method for classifying skin color based on skin color values was proposed before establishing a standard for objective classification of skin color. The color values in the skin image were quantified using a color space, and similar skin color data were classified into groups by clustering based on the quantified color values. In addition, the usefulness of the proposed method was demonstrated by measuring the classification performance by classifying skin color according to an existing visual evaluation method. Consequently, a new skin color evaluation classification method was proposed that overcomes the limitations of the existing skin color evaluation method using a digital camera, objectively classifies skin color, and is optimized for the classification of skin color in Koreans. Furthermore, applying it to the classification of skin disease severity can be used as an auxiliary tool to help doctors objectively evaluate severity.

CONFLICT OF INTEREST

The authors declare that there is no conflict of interest.

Kye S, Lee O. Skin color classification of Koreans using clustering. Skin Res Technol. 2022;28:796–803. 10.1111/srt.13201

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.