Two-dimensional Whitening Reconstruction for Enhancing Robustness of Principal Component Analysis

Abstract

Principal component analysis (PCA) is widely applied in various areas, one of the typical applications is in face. Many versions of PCA have been developed for face recognition. However, most of these approaches are sensitive to grossly corrupted entries in a 2D matrix representing a face image. In this paper, we try to reduce the influence of grosses like variations in lighting, facial expressions and occlusions to improve the robustness of PCA. In order to achieve this goal, we present a simple but effective unsupervised preprocessing method, two-dimensional whitening reconstruction (TWR), which includes two stages: 1) A whitening process on a 2D face image matrix rather than a concatenated 1D vector; 2) 2D face image matrix reconstruction. TWR reduces the pixel redundancy of the internal image, meanwhile maintains important intrinsic features. In this way, negative effects introduced by gross-like variations are greatly reduced. Furthermore, the face image with TWR preprocessing could be approximate to a Gaussian signal, on which PCA is more effective. Experiments on benchmark face databases demonstrate that the proposed method could significantly improve the robustness of PCA methods on classification and clustering, especially for the faces with severe illumination changes.

I. Introduction

PRINCIPAL component analysis (PCA) is a popular feature extraction and data representation method that is widely applied in various areas, such as machining learning, pattern recognition and computer vision, because of its simplicity and efficiency. One of the typical application areas is face recognition, on which many characteristics of PCA have been explored and numerous variants of PCA have been proposed. Sirovich and Kirby [1], [2] first applied PCA to efficiently represent pictures of human face, and argued that any face image could be approximately reconstructed by a weighted sum of a small collection of images. Later, Turk and Pentland [3] presented the well-known Eigenfaces method for face recognition in 1991, after that PCA has been extensively explored and become one of the most successful approaches in face recognition [4], [5], [6]. PCA only considers second-order statistic information, however, a lot of important information are often contained in high-order relationships among pixels. Thus, Bartlett et al. [7], [8] proposed two different independent component analysis (ICA) [9], [10] architectures, i.e. ICA Architecture I and II, and their experimental results suggested that ICA outperformed PCA. However, Yang et al. [11], [12] pointed out that both ICA Architecture I and II contained PCA and a whitening process that were main reasons for the superior performance of ICA to PCA, while the ICA projection had little effect on the performance of face recognition. More specifically, ICA Architecture I involves a vertically centered PCA process (PCA I) and ICA Architecture II contains a whitened horizontally centered PCA process (PCA II).

In above mentioned PCA-based approaches, they all transform the 2D face matrix into a concatenated 1D vector before feature extraction, which leads to a high-dimensional image vector space. Usually, it is difficult to evaluate the covariance matrix accurately due to the large size of the covariance matrix, and the relatively small number of training samples. In order to overcome this problem, two-dimensional PCA (2DPCA) [13] was proposed, which could evaluate covariance matrix accurately and determine the corresponding eigenvectors quickly.

One of the major disadvantages of PCA is that it is sensitive to grossly corrupted entries in the matrix [14], [15], [16]. Unfortunately, gross errors, like the variations in lighting, are ubiquitous in face images especially in real applications, often significantly decrease the recognition performance of PCA. In addition, many approaches such as PCA I, PCA II and 2DPCA that can be regarded as the variants of PCA, are also sensitive to those gross errors. Therefore, it is a necessary and challenge task to solve this problem to enhance the robustness of PCA.

In order to reduce the influence of gross errors, many techniques have been proposed. One of the most widely used techniques is preprocessing. Many preprocessing methods have been developed, these can be roughly classified into two categories: (1) The subspace methods ([14], [17], [18], [19]), which are used to remove the corrupted entries in the matrix based on an assumption that the data matrix approximately have low intrinsic dimensionality [20], [21]; (2) The local feature descriptors ([22], [23], [24]), such as Gabor [25], [26] or local binary patterns (LBP) [27], etc., to capture local structures to obtain discriminative capability while preserving the robustness to illumination, expression and occlusion. However, these methods are not specifically designed for PCA and its variants, because their outputs might not be approximate to Gaussian distribution, on which PCA is more effective.

In this paper, a novel and simple unsupervised preprocessing method, called two-dimensional whitening reconstruction (TWR), is developed to significantly reduce the influence of gross errors on PCA via reducing the pixel redundancy of the internal image and maintaining important intrinsic features, and meanwhile making each face image to be close to a Gaussian signal. By comparison with PCA and its variants, we first do a whitening process on the original 2D face image matrix rather than on a concatenated 1D vector, then, we reconstruct a 2D matrix which is named as whitening reconstruction in this paper. After that, PCA or its variants is applied to the reconstructed image. Experimental results on several benchmark databases demonstrate that our method can significantly improve the classification and clustering performance of PCA and its variants, especially for face images with severe illumination changes.

The rest of paper is structured as follows. Section II mainly presents and analyzes the proposed method TWR, which reconstructs the image via whitening the 2D face image matrix. Section III reports and discusses experimental results on benchmark databases. Finally, Section IV gives the conclusion and points out future research work.

II. Two-dimensional Whitening Reconstruction Algorithm and Analysis

In the process of recording images, it is inevitable to encounter kinds of noises, like variations in lighting. Although the visual effect is different [28], in essence, the number of dimensions of the image is small and stable [29]. Thus, the noise can be effectively removed by preprocessing methods. Generally, a whitening process that reduces the influence of variance of the data matrix can improve the recognition performance of PCA [12]. Furthermore, 2DPCA, which projects an image matrix onto a projection matrix, outperforms PCA that projects an image vector onto a projection vector [13]. Motivated by these findings, we propose a preprocessing method which reconstructs the image matrix via whitening the 2D image matrix.

A. Two-dimensional Whitening Reconstruction

Given an image X = [x₁, x₂, ⋯, x_n] ∈ ℜ^p×n, similar to PCA I [11], let Y = X^T, Y = [y₁, y₂, ⋯, y_p], then calculate the mean vector of Y as follows:

$$\mathbf{u}=\frac{1}{p}\sum _{j=1}^p{\mathbf{y}}_j.$$ (1)

Subtracting the mean vector u from Y, we can get Y_h = [y₁ − u, y₂ − u, ⋯, y_p − u], then let $X_{\upsilon }=Y_h^T=[{\tilde{\mathbf{x}}}_1,{\tilde{\mathbf{x}}}_2,\cdots ,{\tilde{\mathbf{x}}}_n]$, the covariance matrix is:

$$S_{\upsilon }=\frac{1}{n}{X}_{\upsilon }{X}_{\upsilon }^T.$$ (2)

Let P_υ = [α₁, α₂, ⋯, α_m] ∈ ℜ^p×m represent the orthonormal eigenvectors of S_υ corresponding to the m largest positive eigenvalues $\frac{{\lambda }_1}{n}\ge \frac{{\lambda }_2}{n}\ge \cdots \ge \frac{{\lambda }_m}{n}$. Then letting Λ = diag (λ₁, λ₂, ⋯, λ_m), we can obtain the whitened matrix $P_w=\sqrt{n}{P}_{\upsilon }{\mathrm{\Lambda }}^{-\frac{1}{2}}$, so that:

$$P_w^T{S}_{\upsilon }{P}_w=I_m.$$ (3)

Thus X_υ can be whitened as follows:

$$R_I=P_w^T{X}_{\upsilon }.$$ (4)

As for R_I, we can get Theorem I as follows:

Theorem I

For any image matrix X_υ = UDV^T, where U^TU = I_p, V^TV = I_n and D ∈ ℜ^p×n is a diagonal matrix with the diagonal elements in descending order, R_I becomes:

$$R_I=\sqrt{n}V{(:,1:m)}^T.$$ (5)

Proof

According to singular value decomposition (SVD) [30], the image matrix X_υ can be written as:

$$X_{\upsilon }=UD{V}^T,$$ (6)

where U ∈ R^p×p, U^TU = I_p, D ∈ ℜ^p×n is a diagonal matrix and its nonzero diagonal elements are d = [d₁, d₂, ⋯, d_t] (d₁ ≥ d₂ ≥ ⋯ ≥ d_t), t = min(p, n), V ∈ ℜ^n×n and V^TV = I_n. Thus Eq. (2) can be rewritten as:

$$S_{\upsilon }=\frac{1}{n}U{D}^2{U}^T.$$ (7)

Then we can get:

$$P_{\upsilon }=U(:,1:m),$$ (8)

which corresponds to the diagonal matrix ${\mathrm{\Lambda }}^{\frac{1}{2}}=D(1:m,1:m)$, hence,

$$P_w=\sqrt{n}{P}_{\upsilon }{\mathrm{\Lambda }}^{-\frac{1}{2}}=\sqrt{n}U(:,1:m)D{(1:m,1:m)}^{-1}.$$ (9)

Substituting Eq. (6) and Eq. (9) into Eq. (4), we can get $R_I=\sqrt{n}V{(:,1:m)}^T$.

Therefore, Theorem I is proved.

Next we use P_υ to reconstruct the X_υ, we can obtain

$${\tilde{X}}_{\upsilon }=P_{\upsilon }{R}_I.$$ (10)

Based on Eq. (10), we can get Theorem II.

Theorem II

Eq. (10) is equivalent to:

$${\tilde{X}}_{\upsilon }=\sqrt{n}U(:,1:m)V{(:,1:m)}^T.$$ (11)

Proof

Substituting Eq. (5) and Eq. (8) into Eq. (10), we can get:

$${\tilde{X}}_{\upsilon }=P_{\upsilon }{R}_I=\sqrt{n}U(:,1:m)V{(:,1:m)}^T.$$ (12)

Therefore, Theorem II is proved.

For clear illustration, we present the detailed procedure of TWR in Algorithm 1. Also, we present a visual block diagram in Fig. 1. As shown in Fig. 1b and Fig. 1c, the original image is changed by Step 2 and 4 (listed in Algorithm 1). Step 2 (Fig. 1b) does not change the original image significantly. The final whitened image (Fig. 1c) is generated by Step 4. It is worth noting that Step 4 actually contains two stages: A whitening process and 2D face matrix reconstruction.

Fig. 1.

Visual block dragram of TWR in different steps (m=30). (a) Original face, (b) the face after subtracting the mean vector u, (c) whitened face.

Since the image size of all images in one set is equal usually, we can discard $\sqrt{n}$ during reconstructing. Thus the reconstructed image of X is:

Algorithm 1.

Two-dimensional Whitening Reconstruction (TWR)

Input: An image matrix X = [x1, x2, ⋯, xn] ∈ ℜp×n, the rank m.

1. Calculate the mean vector $\mathbf{u}=\frac{1}{p}{\sum }_{j=1}^p{\mathbf{y}}_j$, where yj ∈ Y and Y = XT = [y1, y2, ⋯, yp];

2. Calculate the matrix $X_{\upsilon }=Y_h^T$, where Yh = [y1 − u, y2 − u, ⋯, yp − u];

3. Calculate the SVD of Xυ = UDVT;

4. Calculate the matrix ${\tilde{X}}_{\upsilon }=\sqrt{n}U(:,1:m)V{(:,1:m)}^T$.

Output: Reconstructed matrix X̃υ ∈ ℜp×n.

$$\tilde{X}=U(:,1:m)V{(:,1:m)}^T,$$ (13)

which is named as the whitened image. For brevity, the whitened face image is called whitened face in this paper.

B. Computational Complexity of TWR

Given an image matrix X ∈ ℜ^p×n and the rank m, the computation of the mean vector u in Step 1 and the matrix X_υ in Step 2 requires O(np) operations. The computational complexity of the SVD of X_υ in Step 3 is O(min(n²p, np²)), and the computation of the matrix X̃_υ in Step 4 requires at most O(mnp) operations. Because of m ≤ min(n, p), the total computational complexity of TWR is O(min(n²p, np²)).

C. Relations to PCA and its variants

We introduce the relations among PCA and its variants in this section. Compared PCA I with PCA, the biggest difference between them is that they calculate their mean vectors in different ways so that they have different projection vectors, which can be seen from Proposition 1 in [12]. PCA II includes a whitening process in which projection vectors affect the variance distribution among different images, while the whitening process is neglected by PCA and PCA I. 2DPCA is derived from PCA, and it projects the 2D image matrix into a low-dimensional space directly, unlike PCA, PCA I and PCA II which need to convert the 2D matrix to a 1D vector first.

Different from PCA, PCA I, PCA II and 2DPCA which are feature extraction methods for dimension reduction, the proposed method TWR is a preprocessing method to whiten the 2D image matrix. Besides, compared with PCA II, TWR does a whitening process on the original 2D face image matrix and focuses on reducing the pixel redundancy of the internal image and meanwhile maintaining important intrinsic features. Therefore, the preprocessed image after TWR can be used as the input for PCA and its variants.

D. Face Representation

In order to better display the influence of TWR on face representation, we present the faces calculated by PCA and PCA II in Fig. 2. Fig. 2a uses original faces as the input for PCA and PCA II; Fig. 2b uses whitened faces as the input for PCA and PCA II. The original faces with strong illumination changes are from Extended Yale-B face database [31] and all the original face images are cropped and resized to 32 × 32.

Fig. 2.

Face representation with different methods. (a) Using original face images as the input (From top to bottom: original faces, the faces represented by PCA, the faces represented by PCA II); (b) using the whitened face images as the input (From top to bottom: whitened faces, the faces represented by PCA, the faces represented by PCA II).

Fig. 2 shows that whitened faces can get the rough profile of original faces with preserving their texture information. Because TWR works by reducing the pixel redundancy via a whitening process that will change the singular values of the original 2D face image matrix. In addition, Fig. 3 illustrates that TWR can transfer the original image data to be a signal close to Gaussian distribution, on which PCA is more effective [14], [32] [33]. Comparing TWR with PCA II, PCA II only focuses on reducing the overall pixel redundancy among all faces, while TWR reduces the pixel redundancy of the internal faces, and more important, it maintains important intrinsic features such as textures of the objects. Furthermore, the texture information in whitened faces can be enhanced by PCA and PCA II (Fig. 2b).

Fig. 3.

A sample of pixel value distribution of the original face and its whitened face, the width of each bar is 0.05. (a) Original face (from Extended Yale-B database), (b) pixel value distribution of the original face, (c) whitened face, (d) pixel value distribution of the whitened face.

E. Projection for Separation

To illustrate the benefits of TWR for linear projection, we randomly select three human faces with lighting variability from Extended Yale-B database. TWR is first applied to obtain their whitened faces. The original and whitened faces will then be transferred to vectors and projected into 2D feature space via PCA and PCA II, respectively. Fig. 4 is a comparison of original and whitened faces using PCA and PCA II to project them into 2D plane (each point represents a face image). From this example, it is clear that both PCA and PCA II are sensitive to grossly corrupted entries in the original faces. By contrast, PCA and PCA II with whitened faces can make the classes linearly separable in the projected space. The possible main reasons are that TWR decrease the influence of gross errors via reducing the pixel redundancy of the internal face (Fig. 2) and make a face image be close to a Gaussian signal (Fig. 3), which is more suitable for PCA processing. For the faces under variations in lighting, expression or occlusion, their corresponding whitened faces are also close to Gaussian signals, and we can obtain similar projection results, for simplicity, we do not show them.

Fig. 4.

A comparison of original faces and whitened faces using PCA and PCA II for a three-class problem, the labels of the three human are 7, 23 and 33 in Extended Yale-B database, respectively. (a) PCA with original faces, (b) PCA II with original faces, (c) PCA with whitened faces, (d) PCA II with whitened faces.

III. Experimental Results and Analysis

Since PCA can be applied to both classification and clustering, we present and analyze the performance of the proposed method TWR on PCA and its variants via classification and clustering experiments. There are three typical face databases used in our experiments as follows:

Extended Yale-B face database [31], [34] contains 16,128 face images of 38 human subjects under 9 poses and 64 illumination conditions. We chose the frontal pose with different illumination images in this paper, there were 2,414 face images in total. All the face images were manually aligned and cropped, and were resized to 32 × 32 pixels. Several original faces and their whitened faces are shown in the first row of Fig. 2a and Fig. 2b, respectively.

CMU PIE database [35] contains 41,368 face images of 68 human subjects under 13 different poses and 43 illumination conditions, and with 4 different expressions. Here, all images were also manually cropped and resized to 32 × 32 pixels. Eight original face image samples of one subject and their corresponding whitened faces are shown in Fig. 5a.

Fig. 5.

The sample faces and corresponding whitened faces from different databases. (a) Original faces from CMU PIE database and corresponding whitened faces, (b) original faces from AR database and corresponding whitened faces.

AR database [36] contains over 4,000 color images of 126 human subjects (70 men and 56 women), and includes frontal view faces with different facial expressions, illumination conditions and occlusions. In this paper, we chose 2,600 face images of 100 individuals (50 men and 50 women) under different facial expressions, illumination conditions and occlusions. All the face images were manually cropped and resized to 33 × 30 pixels. Eight original face images of one human and their whitened faces are shown in Fig. 5b.

A. Classification Experiments

In classification experiments, we mainly present the superior performance of TWR on the faces under lighting variability, and apply TWR to PCA, PCA I, PCA II and 2DPCA to demonstrate that TWR is effective for PCA and its variants. Here, we adopt Extended Yale-B and CMU PIE databases in classification experiments.

1) Recognition with Training Faces under Illumination Changes

For Extended Yale-B database, we randomly selected q=[2, 4, 6, 8] images as the training set, and the remaining images were used for testing, where q is the number of training images of each person; for CMU PIE database, we firstly selected the images from the frontal pose (C27) and each subject had 42 facial images under different illumination conditions [37], then we randomly chose q=[1, 2, 3, 4] images as the training set and used the remaining images for testing. We repeated this process 20 times and calculated the mean recognition accuracy. In addition, we first got the best classification results of PCA and its variants for original face images, then we selected corresponding feature dimensions of each method and extracted the same number of features for whitened faces. For the rank of whitened face, we empirically set m=28 for Extended Yale-B and CMU PIE databases in our experiments.

To clearly show the performance of TWR on PCA and its variants, we also present two popular face recognition methods fisherface [38] and collaborative representation based classification (CRC) [39] in our experiments, the latter can nearly get the same classification results as the popular face recognition approach, sparse representation based classification (SRC) [40], with much less computation time. Moreover, we also illustrate the results of three preprocessing methods called unsupervised low-rank representation (ULR) [18], illumination normalization (IN) [22] and discriminant face descriptor (DFD) [24], all of them can improve face recognition accuracy under lighting variability. Here, except CRC, the nearest centroid classifier [41] is used.

Table I and II show that all of ULR, IN, DFD and TWR can boost the classification results of PCA and its variants on Extended Yale-B and CMU PIE databases, but TWR preprocessing gets more improvement than ULR and IN, and obtains superior performance to DFD especially for small q. Besides, PCA II reducing the pixel redundancy among different faces can enhance the robustness of PCA. Compared with PCA II, TWR can further improve this robustness via reducing the pixel redundancy of the internal face. Therefore, TWR+PCA II is superior to other PCA methods. Moreover, TWR+PCA II obtains the best classification accuracy among all methods on two databases. Interestingly, PCA and PCA I are inferior to 2DPCA when the faces are non-preprocessed or preprocessed by IN, but these three methods obtain almost the same accuracy as the faces are preprocessed by ULR, DFD or TWR. A possible main reason is that 2DPCA, ULR, DFD and TWR can reduce the pixel redundancy of the internal face.

TABLE I.

Classification results (mean accuracy ± standard derivation (%)) on Extended Yale-B database under lighting variability

Method	q = 2	q = 4	q = 6	q = 8
fisherface	38.5±1.3	63.8±2.2	74.7±1.4	80.0±1.5
CRC	52.7±2.6	76.4±2.2	86.1±1.0	90.1±0.8
PCA	11.2±1.0	11.9±1.1	11.6±1.0	11.7±1.1
PCA I	14.9±1.2	17.0±1.3	17.9±1.1	19.2±1.2
PCA II	48.1±2.0	67.3±1.2	75.4±1.6	80.6±1.7
2DPCA	19.4±1.5	24.3±1.9	26.8±1.8	29.3±2.0
ULR+PCA	48.0±2.3	62.9±1.1	70.0±1.7	74.0±1.9
ULR+PCA I	48.0±2.3	68.8±1.1	70.0±1.7	74.0±1.9
ULR+PCA II	52.2±2.1	70.0±1.0	78.3±1.7	84.0±1.4
ULR+2DPCA	49.4±2.2	64.7±1.3	71.8±2.0	75.9±2.2
IN+PCA	46.1±2.5	59.9±2.9	66.2±3.0	71.7±2.7
IN+PCA I	46.1±2.5	60.0±2.9	66.3±3.0	71.8±2.8
IN+PCA II	68.4±2.0	76.3±2.4	90.1±1.1	94.2±1.4
IN+2DPCA	56.6±2.3	71.8±2.7	78.1±2.8	83.3±2.1
DFD+PCA	48.7±2.9	80.4±2.7	85.5±1.7	88.6±1.5
DFD+PCA I	48.6±2.9	80.3±2.7	85.5±1.7	88.6±1.5
DFD+PCA II	46.7±2.2	80.3±2.2	83.5±1.5	85.6±1.8
DFD+2DPCA	46.0±2.8	80.1±2.8	85.5±1.7	88.8±1.4
TWR+PCA	62.6±2.5	77.9±2.0	84.1±1.8	88.6±2.1
TWR+PCA I	62.4±2.5	77.9±2.1	84.0±1.8	88.6±2.1
TWR+PCA II	72.6±2.2	88.8±1.3	93.3±0.7	96.0±0.9
TWR+2DPCA	63.3±2.7	78.5±2.0	85.0±1.8	89.6±2.0

TABLE II.

Classification results (mean accuracy ± standard derivation (%)) on CMU PIE database under variations with lighting

Method	q = 1	q = 2	q = 3	q = 4
fisherface	28.4±1.8	87.9±1.4	95.2±1.1	95.2±1.1
CRC	88.1±1.0	99.3±0.3	99.9±0.1	100
PCA	24.1±1.2	28.8±1.4	31.3±1.3	32.7±1.9
PCA I	37.0±1.2	43.9±1.1	46.7±1.5	48.6±1.5
PCA II	78.8±1.3	93.9±1.8	96.9±0.6	97.7±0.6
2DPCA	45.1±1.3	58.3±1.6	63.7±2.0	67.6±2.1
ULR+PCA	88.2±1.8	95.3±0.7	97.3±0.5	98.4±0.4
ULR+PCA I	88.3±1.8	95.3±0.6	97.3±0.5	98.4±0.4
ULR+PCA II	90.6±1.3	99.0±0.3	99.6±0.2	99.7±0.2
ULR+2DPCA	88.7±1.7	95.5± 0.7	97.6±0.5	98.5±0.4
IN+PCA	63.6±3.0	72.1±2.2	78.0±2.2	80.4±1.9
IN+PCA I	63.6±3.0	72.1±2.2	78.0±2.1	80.5±1.9
IN+PCA II	76.4±2.4	95.3±0.6	97.0±0.4	97.6±0.5
IN+2DPCA	68.5±2.8	79.2±1.6	84.3±1.7	86.5±1.3
DFD+PCA	45.1±4.0	95.6±0.7	97.0±0.7	97.8±0.6
DFD+PCA I	44.9±4.0	95.6±0.7	97.0±0.7	97.8±0.6
DFD+PCA II	54.7±3.2	97.9±0.4	98.8±0.3	99.2±0.3
DFD+2DPCA	44.5±3.7	96.0±0.6	97.2±0.7	97.8±0.5
TWR+PCA	93.3±1.1	97.0±0.7	98.4±0.4	99.2±0.3
TWR+PCA I	93.1±1.0	97.0±0.7	98.4±0.4	99.2±0.3
TWR+PCA II	96.4±0.5	99.7±0.2	99.9±0.1	100
TWR+2DPCA	93.4±1.0	96.9±0.7	98.4±0.4	99.2±0.4

2) Recognition with Training Faces under Normal Lighting Condition

In order to better show the strength of TWR on lighting variability, in this section, we show the performance of TWR with training faces under normal lighting condition while test faces under lighting variations from slight to severe. In this experiment, similar to [42], we adopted Extended Yale- B database and divided the database into five subsets, we chose Subset 1 as the training set, and used Subsets 2–5, with illumination changes from slight to severe, for testing. Besides, each image was cropped and resized to 96 × 84 pixels, some sample images of each subset are shown in Fig. 6. Here, we present the performance of some popular methods for comparison: LRC [42], SRC [40], CRC [39], RSC [43], NMR [44], HQ_A and HQ_M [45]. In addition, we also apply TWR to these methods for better showing the performance of TWR. Among PCA methods, we just present the performance of TWR+PCA II in this experiment, because Table I and Table II have demonstrated that TWR+PCA II is the best choice. As for the rank in TWR, we empirically set m=80. Table III shows that TWR can largely improve the classification performance of above mentioned methods, especially on Subset 5, which suggests that TWR is more suitable for handling the faces under severe illumination changes.

Fig. 6.

From top to bottom, each row shows the sample images of subsets 1, 2, 3, 4 and 5, respectively.

TABLE III.

Classification results (%) on faces under illumination changes from slight to severe

Method	Subset 2	Subset 3	Subset 4	Subset 5
LRC	100	100	87.6	42.2
CRC	100	100	88.0	35.7
SRC	98.5	93.4	78.4	28.8
RSRC	100	100	80.3	36.7
HQ_A	99.8	96.3	67.9	31.3
HQ_M	99.8	96.3	75.8	36.8
NMR	100	100	90.2	47.9
TWR+PCA II	100	100	92.2	76.4
TWR+LRC	100	100	98.5	88.4
TWR+CRC	100	100	95.1	76.8
TWR+SRC	100	100	97.9	86.8
TWR+RSRC	100	100	97.2	85.2
TWR+HQ_A	100	99.6	90.7	60.2
TWR+HQ_M	100	99.6	90.5	58.0
TWR+NMR	100	100	98.5	88.2

3) Rank Selection

TWR has an essential parameter: the rank m, which is empirically set to 28 and 80 in above experiments. In this section, we examine the impact of this parameter on the performance of TWR.

Since the impact of this parameter on different methods is similar, for brevity, we just show the results of PCA. For comparison, we randomly selected q=2 and q=1 images of each person as the training set from Extended Yale-B and CMU PIE databases, respectively, then we repeated this process 20 times and calculated the mean accuracy. In addition, each image in Extended Yale-B and CMU PIE databases was resized to 32 × 32 pixels. Fig. 7 shows that the images from different databases have different optimal values for m, but neither of them can get the best results with full rank. Thus we usually discard several vectors corresponding to the smallest eigenvalues to get optimal or sub-optimal results when constructing whitened faces. The main reason is that small eigenvalues might be corresponding to the noise. Because of the same reason, we chose m=80 for Extended Yale-B database when each image was resized to 96 × 84 pixels.

Fig. 7.

Accuracy with different m on two face databases when each image is resized to 32×32 pixels. (a) Extended Yale-B (q=2), (b) CMU PIE (q=1).

We also show the impact of the rank m on the pixel value distribution of the whitened face. The distribution of one whitened face with different dimensions is shown in Fig. 8, which suggests that the larger rank m, the closer to Gaussian distribution.

Fig. 8.

The pixel value distribution of one whitened face with different m, the width of each bar is 0.05. (a) Original face, (b) m=10, (c) m=20, (d) m=30.

B. Clustering Experiments

In clustering experiments, we present the performance of TWR on faces under variations in lighting, expression and occlusion. Here, we adopt Extended Yale-B, CMU PIE and AR databases. As for CMU PIE database, we select the images from the frontal pose (C27) and each subject has around 49 images with varying illuminations and facial expressions [46].

1) Experimental setting

In the following experiments, we adopt the popular cluster method K-Means for clustering. The methods K-Means, PCA+ K-Means, PCA I +K-Means and PCA II +K-Means are briefly named as KM, PCA, PCA I and PCA II, respectively. Since 2DPCA cannot be directly applied to K-means, we discard it in clustering experiments.

We randomly selected k=[2, 4, 6, 8, 10] individuals from three databases for clustering, and ran the K-Means method 10 times for the same samples. We repeated this process 20 times and then calculated the average cluster accuracy. The cluster accuracy is measured by

$$\mathit{\text{Cluster accuracy}}=\frac{\mathit{\text{number of correct classified images}}}{\mathit{\text{number of total images}}}.$$ (14)

Here, we set the rank m = [30, 30, 28] to reconstruct whitened faces on Extended Yale-B, CMU PIE and AR databases, respectively.

2) Experimental Results and Analysis

Tables IV–VI show that TWR can significantly improve the clustering performance of PCA and its variants. The possible reasons are: (1) TWR decreases the influence of gross errors via reducing the pixel redundancy of the internal face image and meanwhile maintaining the important intrinsic features; (2) TWR makes each face image be close to a Gaussian signal, which might promote different images to be approximate to a Gaussian distribution that is more suitable for K-means processing [47]. In addition, PCA II has much better performance than KM, PCA and PCA I as the faces are not preprocessed, while the gap of the clustering accuracy of KM, PCA, PCA I and PCA II becomes small as the faces are preprocessed by TWR, probably because decreasing the pixel redundancy of each internal image can reduce the pixel redundancy of overall images.

TABLE IV.

Clustering results on Extended Yale-B database with faces under lighting variation

k	Non-preprocessed (%)				Preprocessed by TWR (%)
	KM	PCA	PCA I	PCA II	KM	PCA	PCA I	PCA II
2	52.2	52.1	51.8	71.9	97.0	96.7	97.0	89.7
4	28.5	28.5	29.1	54.7	81.2	80.2	81.5	86.2
6	22.2	22.2	20.3	49.2	74.7	75.5	75.1	67.8
8	19.3	19.4	17.0	43.5	68.2	67.2	68.1	71.0
10	16.8	16.7	14.4	44.2	63.0	63.2	63.8	60.4
Avg.	27.8	27.8	26.5	52.7	76.8	76.6	77.1	75.0

TABLE VI.

Clustering results on AR database with faces under variations with lighting, facial expression and occlusion

k	Non-preprocessed (%)				Preprocessed by TWR (%)
	KM	PCA	PCA I	PCA II	KM	PCA	PCA I	PCA II
2	58.8	58.6	64.7	73.1	98.9	99.1	99.1	95.4
4	38.9	39.2	44.1	51.2	96.1	96.1	95.8	89.4
6	29.5	29.4	31.3	56.0	83.4	81.7	83.0	90.3
8	22.7	22.7	25.0	49.5	75.0	74.7	75.4	88.5
10	19.6	19.5	22.6	45.5	68.2	67.8	67.8	85.7
Avg.	33.9	33.9	37.5	55.1	83.8	83.9	84.2	89.9

IV. Conclusion

In this paper, we present a simple and efficient preprocessing method, two-dimensional whitening reconstruction (TWR), that enhances the robustness of PCA and its variants via a whitening process and 2D image reconstruction. TWR works by reducing the pixel redundancy of the internal face and maintaining the significant intrinsic features, so the rough profile and texture information of the face are preserved while some noises are removed. Beyond that, TWR can transfer each face image to be close to a Gaussian signal, which is more suitable for PCA processing. Classification experiments on both Extended Yale-B and CMU PIE databases demonstrate that TWR could greatly improve the robustness of PCA and its variants on faces under lighting variety. Clustering experiments on three databases suggest that TWR could also significantly improve the clustering performance of PCA methods. Although TWR+PCA II could get very encouraging classification and clustering results on the faces with lighting variations, we find that the classification performance of TWR+PCA II on the faces under variations in expression and occlusion is not as well as under lighting variations. Therefore, in our future work, we will explore the possible reason and further improve the classification performance of TWR+PCA II on expression and occlusion problems.

TABLE V.

Clustering results on CMU PIE database with faces under variations with lighting and facial expression

k	Non-preprocessed (%)				Preprocessed by TWR (%)
	KM	PCA	PCA I	PCA II	KM	PCA	PCA I	PCA II
2	61.9	62.0	55.0	66.4	93.0	93.1	93.1	89.4
4	31.8	32.1	38.6	65.6	87.0	86.5	86.4	88.1
6	24.3	24.5	29.7	54.1	79.0	80.3	78.1	78.4
8	25.3	25.7	31.3	55.1	76.5	77.4	78.0	75.1
10	21.4	21.5	27.4	53.8	74.6	74.3	74.2	73.7
Avg.	32.9	33.2	36.4	66.4	82.0	82.3	82.0	80.9