Feature Selection from a Facial Image for Distinction of Sasang Constitution

Abstract

Recently, oriental medicine has received attention for providing personalized medicine through consideration of the unique nature and constitution of individual patients. With the eventual goal of globalization, the current trend in oriental medicine research is the standardization by adopting western scientific methods, which could represent a scientific revolution. The purpose of this study is to establish methods for finding statistically significant features in a facial image with respect to distinguishing constitution and to show the meaning of those features. From facial photo images, facial elements are analyzed in terms of the distance, angle and the distance ratios, for which there are 1225, 61 250 and 749 700 features, respectively. Due to the very large number of facial features, it is quite difficult to determine truly meaningful features. We suggest a process for the efficient analysis of facial features including the removal of outliers, control for missing data to guarantee data confidence and calculation of statistical significance by applying ANOVA. We show the statistical properties of selected features according to different constitutions using the nine distances, 10 angles and 10 rates of distance features that are finally established. Additionally, the Sasang constitutional meaning of the selected features is shown here.

Introduction

Sasang Constitutional Medicine (SCM) is based on the concept that pharmaceuticals should be used differently according to the balance between one's internal organs. SCM recommends that the major pharmaceuticals used in oriental medicine are classified into four groups according to the different constitution, and that only those pharmaceuticals suitable for one's constitution should be taken. Over the 100-year history of SCM, sufficient records of reactions to these drugs have been amassed to allow for the classification of the types of constitution. However, because some patients with one constitution are inversely sensitive to the pharmaceuticals corresponding to another constitution, it remains important in the successful practice of SCM to confirm the constitution of the patient.

The establisher of SCM, Lee Jema, suggested personality profiles, symptoms, body shape and other factors as important criteria in the classification of the constitution. However, facial shape was excluded from the criteria listed in Donguisusebowon (1) [‘Longevity and Life Preservation in Oriental Medicine’ (2) in its English translation], a fundamental text of SCM. Despite this omission, books by Lee's disciples used by modern SCM clinicians frequently referred to studies concerning the connections between facial shape and constitution. The basic paradigm of oriental medicine is that each part of the body is affected by and exerts an influence on every other part of the body. Many textbooks of oriental medicine suggest a visual inspection as a means to judge the strength and the weakness of internal organs. This leads us to study the facial feature as criteria for the classification of one of the Four Constitution types.

However, clinical use of the analysis of facial shape in the field is a limited qualitative technique, as this type of analysis is not based on quantified and standardized methods. Recently, new methods are available to analyze facial image data (3–6). These studies attempted to quantize the facial shape according to the Sasang constitution. We obtained the distance, angle and the distance ratios between facial feature points from 493 subjects who were previously classified according to their reaction to oriental medicines and analyzed the statistical significance of these measurements corresponding to SCM.

In this study, data-mining tools are used to find statistically meaningful features with respect to SCM. Data-mining techniques provide useful information from very large datasets and help to understand the features in the sets. Figure 1 shows a flowchart of the data process of finding the significant features. These methods are usually applied in data mining. Twenty-nine statistically significant features were found, including the distance, angle and distance ratios using the aforementioned method. Some of them are consistent with traditional literature, while the remaining features are newly introduced in terms of their relationships with SCM.

Figure 1.

The flowchart shows the data-mining procedure. Under the SOP, operators take a picture of a subject and fill feature points using a Microsoft Excel file. Let us calculate the quantitative measures. The next step is a data preprocess of controlling outliers and missing values. The next part of the procedure involves determining the significant features considering ANOVA and correlation.

Materials and Methods

We acquired 493 subjects from five sites in Korea. These were the Dong-eui University Oriental Medical Hospital, the Kyungwon University Oriental Medical Hospital, the Kyunghee University Hospital of Oriental Medicine, the Woosuk Medical Center and the Korea Institute of Oriental Medicine. The study protocol was approved by the Institutional Review Board of the Korea Institute of Oriental Medicine. The constitutions of all subjects were confirmed by SCM doctors after scanning reactions and observing patient improvements after the administration of constitution-specific pharmaceuticals. Before this series of experiments commenced, a standard operating procedure (SOP) was established to act as a safeguard against differences in individual operators. All processes then followed this SOP.

We used 50 feature points from a frontal facial image. Supplementary Figure S1 represents the positions on the face. After verifying the reproducibility of the work of three operators in independently marking selected feature points for five repetitions, one operator was chosen after demonstrating a high level of reproducibility performance. That operator took the coordinates of each feature point in pixel units, and those pairs of coordinates were recorded using Microsoft Excel files.

Quantitative Measures: Distance, Angle and Distance Ratios

The quantitative measures for the statistical analysis included the distance, angle and the ratio of the two distances as calculated from the 50 previously recorded feature points. The distances could be calculated using two points from the facial images. However, it was necessary to convert relative distances to absolute distances, as distance depends on photographic conditions such as the actual distance between the patient and camera. Each picture has a ruler marked with a unit distance, allowing a realistic distance measure to be calculated by the ratio of the relative distance to the relative unit distance. Here, the two coordinates (x_i, y_i), (x_j, y_j) are the i, j-th pixel positions from the x- and y-axis, respectively, for i, j = 1, … ,50. The realistic distance from the i-th to the j-th feature points is then defined by the following equation:

(1)

where (m_x1 - m_y1) and (m_x2 - m_y2) represent the pixel positions of the unit distances for x and y coordinates, respectively.

The second quantitative measure is angle , which is formed by the two rays and sharing as a common endpoint of the j-th feature point, where the rays and are lines containing the i, j- and j, k-th feature points, respectively. The calculation form of angle is reflected in the following equation:

(2)

where z_i, z_j, z_k represent the points in a complex plane: they are z_i = x_i + iy_i, z_j = x_j + iy_j and z_k = x_k + iy_k, respectively.

It is believed that body size is one of the most influential factors affecting facial distance; that is, larger people have a larger face. The important property of facial angles is more independent of body size compared with the facial distance feature. In Equation (2), the adjustive term of a unit ruler is unnecessary; thus, this equation shows that the angles remove the difference in the individual body shapes. The third measure is the rate of the two distances, which is less sensitive to body size due to the use of a numerical correction accounting for body size. The rate of distance r(i, j/k,l) is the distance ratios d(i, j) divided by the distance d(k,l), as in the following equation:

(3)

The unit distance does not affect the distance ratios as much as the angle, as the final forms of Equations (2) and (3) do not contain the unit rule information. It is expected that both the angle and the distance ratios will be the most efficient quantitative measures for classifying the Sasang constitution.

Removing Outliers

As the operator obtains the coordinates from the feature points, an unintended mistake can cause an outlier that is inconsistent with the probabilistic distribution of the rest of the data and is considered as meaningless (7). Furthermore, a statistical analysis produced from data that include outliers may lead to erroneous results. Observation data having a very weak possibility of occurrence are regarded as outlier data. Under the assumption that the data are independent and identically distributed with a normal distribution having a mean μ and a standard deviation σ for each quantitative feature, observation data outside the interval with a radius of 4σ at the center μ are considered as outlying data. The complete process of removing outliers is given in Appendix 1.

Controlling for Missing Values

While filling out the coordinates of feature points, an operator is likely to miss some values due to the unclear characteristics of the feature points of individual patients and due to other unintended operator errors. Accumulation of these missing values collected for the purpose of quantitative measurement detracts from the validity of the statistical analysis. Although many methods have been suggested to deal with the problem of missing values (8), we use the simple classical method rather than current complex methods. The selected method, which removes features having >20% missing values, is widely used in general data preprocessing, and performance can be achieved independent of the properties of the data. We chose 20% as the threshold in order to guarantee the minimal number of data subset, which is one of the disjoint subsets of the original data randomly divided into two subsets, ensuring statistical power without missing values in the next ANOVA test step.

Table 1.

Table of significant distance features associated with the Four Constitution types

Index	d(i, j)	P-value	Missing rate (%)	Mean ± SD
				All	TE	SY	SE
1	d(39,44)	8.63E-04	3.4836	46.86 ± 7.74	48.78 ± 7.07	44.66 ± 7.74	45.76 ± 8.14
2	d(38,44)	3.24E-04	3.0737	56.84 ± 7.86	59.07 ± 7.48	54.70 ± 9.08	54.99 ± 5.52
3	d(40,44)	3.66E-03	2.8688	35.76 ± 6.63	37.37 ± 6.21	34.38 ± 6.95	34.28 ± 6.38
4	d(44,49)	5.59E-04	3.0737	58.06 ± 8.21	60.40 ± 7.77	56.16 ± 9.26	55.71 ± 6.28
5	d(37,44)	6.76E-04	3.8934	69.49 ± 8.83	71.76 ± 8.47	67.37 ± 10.27	67.55 ± 6.14
6	d(44,48)	9.61E-04	2.8688	71.05 ± 8.82	73.31 ± 8.44	69.03 ± 10.20	69.05 ± 6.31
7	d(35,44)	1.38E-03	2.8688	62.98 ± 8.33	65 ± 7.85	61.18 ± 9.96	61.17 ± 5.77
8	d(18,44)	3.27E-03	2.8688	97.11 ± 9.09	98.98 ± 9.18	95.88 ± 8.01	94.92 ± 9.39
9	d(15,44)	5.04E-03	3.6885	105.02 ± 8.56	107.10 ± 8.12	103.33 ± 9.26	102.97 ± 7.59

The term d(i, j) denotes the distance from the i-th and j-th feature points. The P-value comes from the ANOVA test.

Table 2.

Table of significant angle features associated with the Four Constitution types

The term denotes the angle with the vertex j-th point and the endpoints i-th and k-th feature points. The P-value comes from the ANOVA test.

Table 3.

Table of significant rate of distances features associated with Four Constitution type

Index	r(i,j/k,l)	P-value	Missing rate (%)	Mean ± SD
				All	TE	SY	SE
1	r(26,39/44,49)	6.80E-07	5.7377	1.116 ± 0.142	1.082 ± 0.136	1.133 ± 0.141	1.165 ± 0.137
2	r(17,38/40,44)	6.60E-08	3.8934	2.013 ± 0.344	1.923 ± 0.338	2.06 ± 0.349	2.137 ± 0.301
3	r(17,35/34,39)	2.99E-06	7.7868	0.84 ± 0.103	0.815 ± 0.099	0.862 ± 0.101	0.865 ± 0.103
4	r(17,35/34,37)	4.15E-11	5.9426	0.692 ± 0.086	0.665 ± 0.082	0.708 ± 0.081	0.727 ± 0.082
5	r(21,36/44,49)	8.04E-07	4.5081	0.826 ± 0.113	0.798 ± 0.108	0.844 ± 0.115	0.861 ± 0.108
6	r(27,46/40,44)	3.89E-08	12.704	2.458 ± 0.395	2.346 ± 0.368	2.523 ± 0.383	2.599 ± 0.401
7	r(26,35/44,48)	7.84E-07	4.3032	0.542 ± 0.072	0.524 ± 0.069	0.553 ± 0.073	0.564 ± 0.068
8	r(17,37/38,44)	1.94E-08	4.0983	0.98 ± 0.135	0.943 ± 0.131	1.005 ± 0.133	1.026 ± 0.123
9	r(19,26/44,49)	2.84E-09	5.7377	0.15 ± 0.029	0.143 ± 0.026	0.15 ± 0.023	0.164 ± 0.036
10	r(17,35/18,50)	4.60E-07	4.0983	0.386 ± 0.029	0.38 ± 0.029	0.388 ± 0.029	0.397 ± 0.025

The term r(i, j/k,l) denotes the ratio of distance of line segment i-th and j-the feature points to distance of line segment k-th and l-the feature points. The P-value comes from the ANOVA test.

ANOVA Test

The goal of this study is to determine the facial features of distance, angle and the ratio of distance having statistical significance related to Sasang constitutions. The statistical analysis result will indicate how the Four Constitution types are associated with the quantitative features. As there are four factors (TY, TE, SE and SY types), it is possible to use a one-way ANOVA to test the null hypothesis that there is no difference among the Four Constitution types.

Choosing the Significant Features

After the earlier mentioned process, there remain 621 distances, 20 275 angles and 192 510 rates of distance as quantitative measures to analyze statistically. Each feature is simultaneously tested with an ANOVA test. False positive errors, which incorrectly reject the null hypothesis in statistical inference hypothesis test, are likely to occur when one tests a very large bundle as a whole. As the number of comparisons increases, more null hypotheses will become incorrectly rejected. To solve this problem, many statistical techniques have been developed to control significant levels (9,10). However, a few statistical conditions for the facial image data are inconsistent with the assumptions of these statistical techniques. In order to solve this problem, we proposed the use of resampling methods, where the features that are selected many times in the iterative statistical analysis will lead to a lower probability of having a true null hypothesis rejected by chance. Such selected features have a statistically significant association with the Four Constitution types. The feature selection algorithm is discussed in Appendix 2.

Considering Correlation

A number of features are too significant to understand in terms of whether these features have statistical meaning and how to use them for the classification of Sasang constitutions. Furthermore, there are relationships among features; that is, features within geometric areas of the face are highly correlated with one another. For this reason, it is necessary to reduce the number of features that have the same geometrical and statistical properties. The hierarchical clustering method has been suggested to solve such correlation problems (11,12). This method clarifies the linkage of individual features by gradually merging groups. The linkages can be constructed through the correlation of groups and individual features, where the largest correlation can be connected. Therefore, shared features among groups are highly correlated with each other and have the same geometrical properties. The representation of each group is selected as the lowest P-value of the ANOVA test with respect to the Four Constitution types, which produces the maximum difference in the group associated with the Four Constitution types.

Results

Data Acquisition

Among the 493 subjects, with the frequency distribution as shown in Table S1, five TY type and unidentified people were ruled out for the reason that their presence may have led to unexpected statistical bias, as commonly occurs with small data sample sets. Thus, the frontal view face images of 488 people were used in the test.

Data Process

After removing the features having >20% missing values, the numbers of features pertaining to the distance, angle and rates of distance were reduced to 621, 20 275 and 192 510, respectively. These figures represent 50.69, 33.11 and 25.68%, respectively, of the initial number of features. In this work, we used a resampling method coupled with an iterative ANOVA test to determine the false positive features that were irrelevant in characterizing the Four Constitution types. As a result, out of 10 random subsets comprising half of the entire dataset, we are left with 20.72% out of 621 distance features, 11.25% out of 20 275 angle features and 25.68% out of 192 510 distance ratio features. Each subset is listed in Supplementary Table S2. In reducing the distance features, only one was chosen as significant for each instance in the iterative analysis. For instance, only two and six features out of the total number of distance features yielded significant differences according to the Four Constitution types in 9 and 8 out of 10 subsets, respectively. For the angle features, the features common to all, 9 or 8 out of the 10 random subsets number 57, 130, 127, respectively, and 1702, 2526, and 2873 in the case of the distance rates. It is not by chance that the features commonly selected in all of the random subsets in the angle and the distance rates are significantly related to the Four Constitution types at α = 0.05. To run the pattern classification algorithm efficiently, usually a sufficient number of features is needed. For this reason, we adopted eight common distance features in eight and nine random subsets as well, despite the low reliability of the significance from the statistical test. Moreover, only one distance feature is insufficient for analyzing the constitutional properties using the facial image. For these two reasons, 9 distance features, 57 angle features and 1702 distance rate features were chosen as the effective features for the next clustering process.

Hierarchical Clustering

An effective method for a further reduction in the number of features can be the extraction of the representative features from the block or group of interest using a hierarchical clustering technique. We find this procedure to provide a good understanding of the geometrical and correlative relationships among the features. Supplementary Figure S2 shows the correlations between features as a function of the distance rates. By linking the group and individual features, in the figure, we find that each red- or blue-colored block shows similar geometrical and correlative properties. Specifically, the red block of the (i, j)-th element in the array represents a close connection between the i-th and j-th features, while the blue block in the same element shows a very low correlation between them. Supplementary Figure S3 presents the correlation between each of the remaining 10 representative features after removing the features in each coloured block. Here, red denotes the same meaning as in Supplementary Figure S2, whereas blue now indicates no correlation, with a value close to 0.

Analysis of Selected Features

In Supplementary Tables 13, the statistically significant features of the distance, angle and distance rate are listed, respectively. For instance, in Supplementary Table 1, we find that the values of the selected distance features in the TE constitutional type are all larger than those in the other types. A consistent result is shown in Supplementary Figure S1 in that these nine distance features contain the 44th feature point. These results show that the TE type has a wide chin and face, in accordance with the traditional understanding. Regarding the angle features, in Supplementary Figure S1 by comparing the angles in the 34th feature point, we see that the SE types have larger angles than the TE and SY types. Similarly, through the 32nd feature point, we find that the TE types feature larger angles compared with the other types. All of these features demonstrate that the TE type of the Four Constitution types has the largest angle, which is consistent with the traditional literature.

Discussion

The goal of this study is to utilize scientific methods to detect significant features to help determine the Sasang constitution of a patient. SCM doctors refer to various standards for the judgment of the Sasang constitution, but facial characteristics are commonly considered as an important factor in determining the Sasang constitution. The scientific methods used in this study borrow data-mining techniques such as data preprocessing and significant feature selection. The quantitative measures used for the statistical analysis were the distances, angles and distance ratios. Data preprocessing accounted for the missing data and removed outliers and the statistically meaningful features associated with the Four Constitution types were repeatedly selected by iterative ANOVA testing. A feature reduction process was then used to simplify the variables. In the end, nine distances, 10 angles and 10 rates of distance were selected as statistically significant features.

We compared the results of the measurements and the statistical process with those in an article (13) from eight publications, reviewed and summarized by SCM doctors. The results of this study show that the width over the length was larger in a face of the TE type compared with that of the SE type. The tendency was similar for the distance, angle and distance ratio. Mainly, it was clearer when the width of the jaw served as a standard. In the case of the angle, the eye of the SE type has a round shape, and although those of the SY type and the TE type had similar shapes, the shape of the SY type was more round than that of the TE type. The article (13), the results of which were found to be in good agreement with those of this study, reports that the TE type has a wider jaw than the SE and the SY types, the facial length of the SY type is shorter, and the SE type usually has a smaller but longer face. The eyes of both the SY and SE types are round. However, the article fails to mention whether the features of the eyes of the SY type are closer to those of the TE type or those of the SE type.

In this research, we found statistically important features by using data-mining tools. We expect that the selected features will be helpful to distinguish Sasang constitutions for individuals, which is left as a future study.

Supplementary Data

Supplementary data are available at eCAM online.