Improvement of illumination-insensitive feathers for face recognition under complex illumination conditions

Complex illumination condition is one of the most critical challenging problems for practical face recognition. In this paper, we propose a novel method to improve the illumination invariants for solving this challenge. Firstly, a new method based on the Lambert reflectance model is proposed to extract illumination invariant, which is less insensitive to complex illumination variations. Secondly, in order to repair the defects caused by process of illumination invariants extraction, Fast Mean Filter is utilized to smooth and remove noise. Lastly, for raising the richness of information in output image, a nonlinear normalization transformation is proposed. Compared with the state-of-the-arts, experimental results show that the proposed method can extract more robust illumination invariants. Apart from it, the richness of information in processed image is greater and superior performance in face recognition rate is superior.


Introduction
As an important research field of image processing and pattern recognition, face recognition has made remarkable progress in recent years.However, slight illumination changes can lead to great difference among the same target image, which increases the difficulty of image analysis and pattern recognition.Improvement of the illumination robustness for face recognition in complex light environment is a difficult problem in the field of artificial intelligence and computer vision [1,2].
At present, domestic and overseas researchers have proposed numerous illumination processing algorithms for face images under complex illumination conditions, and achieved good experimental results.
They can be roughly divided into two categories.One approach deals with image processing modeling techniques, which are helpful to normalize faces with different lighting effects, such as logarithm transforms (LT) [3], gamma correction (GC) [4], block-based histogram equalization (BHE) [5], adaptive histogram equalization (AHE) [6], oriented local histogram equalization (OLHE) [7], the small-and large-scale (S&L) features [8] and enhanced local texture feature [9].LT and GC are of low computational complexity due to global techniques of image enhancement, but uneven illumination variation is too tricky even by using these global processing methods.On the other hand, AHE, BHE, AHE and OLHE also cannot achieve satisfactory performance on face recognition under complex illumination.
The reason of this is that they tend to destroy overall distribution of essential image characteristics and enhancement of local texture feature tends to lose many image detail informations.
The other category directly or indirectly extracts illumination invariants, and it copes with illumination variation based on a common hypothesis that the reflectance changes quickly and the illumination varies slowly.This type of methods mainly consists of multiscale retinex (MSR) [10], DCT-based nor-malization technique (DCT) [11], self-quotient image (SQI) [12], logarithmic total variation (LTV) [13,14], gabor based methods [15], wavelet-based methods [16,17], contourlet-based methods [18], Bidimensional empirical mode decomposition (BEMD) [19], Gradient [20] and Weber [21].MSR [10] combines several low-pass filters (i.e., Gaussian filters) for illumination estimation in a logarithm domain, which can cause serious halo effect.Gradient [20] is employed to reduce the halo effect in a certain degree based on an anisotropic filter.It is difficult for SQI [12] to keep sharp edges due to use of the weighted Gaussian filters.LTV [13,14] improves SQI [12] based on logarithmic total variation.Furemore, illumination invariants extracted by wavelet-based [16,17] methods have obvious Gibbs phenomena and contourlet-based [18] methods tend to destory much useful information.BEMD [19] method has quite high computational expense.Gradient-based methods completely discard intensity information of facial reflectance.Cheng [22] proposes an illumination estimation algorithm to extract illumination invariants to meet the actual lighting situations and image acquisition model.In all, most of those methods based on the common hypothesis only extract high-frequency characteristics of illumination invariants (i.e., facial reflectance), neglect its low-frequency characteristic causing false edge phenomenon.On the other hand, most of them are sensitive to illumination variation.In addition, many image detail informations are easy to lose during the procedure of processing by most of those methods.
Therefore, the enhancement of a facial texture feature is still unsatisfactory due to the superfluous information of facial skin.
In order to overcome the adverse effect of complex illumination variations on face recognition, an enhanced facial texture illumination normalization method is introduced in this paper.A novel illumination estimation algorithm is developed to extract illumination invariants, which can meet the actual lighting situations, image acquisition mode based on the analysis of the complex lighting conditions and the classical Lambertian reflectance mode.Fast mean filter is utilized to smooth processed image and remove noise.A nonlinear normalization transformation is developed to raise the richness of information in processed image.An overall flowchart for our proposed method is shown in Fig. 1.The main contributions of this paper are as follows: Firstly, the proposed method can more effectively extract illumination invariants, and it is less sensitive to complex illumination variation compared with other methods.Secondly, high-frequency and low-frequency characteristics are considered during the process of illumination invariants extraction, which can avoid loss of detail information.Thirdly, the richness of information is greater compared with other methods in the output image.Furthermore, experimental results indicate that the proposed method is a better approach compared with the state-of-the-arts.This paper is organized as follows: Section 2 describes the proposed method.Section 3 shows experimental results and comparisons with state-of-the-art methods, and our concluding remarks are given in In this case, facial reflectance R can be looked upon as illumination invariants.From the above model defined by Eq. (1), we can know that if R and L are not known in advance, and it will be an illposed question to deduce R and L from an image.To deal with the problem, a common assumption is that L varies slowly and R changes quickly.Under this assumption, an image is segmented into two frequency bands: low-frequency and high-frequency band, where L and R correspond to the low-frequency band and high-frequency, respectively.When illumination invariants are being extracted, only highfrequency characteristics of R is considered while its low-frequency characteristics is ignored by many existing methods.However, in effect, a facial reflectance not only has high frequency characteristics, but also has low-frequency ones such as the reflectance in areas of eyes, eyebrow and skin.Both of them play important roles in face recognition.Therefore, assuming rapid changes of surface reflectance on human face is inappropriate, which easily causes extracted illumination invariants to lose important low-frequency information of facial reflectance. (1)

Illumination Invariants
It is usually assumed that illumination changes slowly by the conventional illumination estimation method.However, it causes failure to obtain good estimation result on images under complex illumination.Under ideal lighting conditions (parallel light without shielding, scattering or diffusing reflection phenomenon and so on), each image pixel usually has the similar illumination, and the reflectance value is between 0 and 1.In this case, the maximum value of I is closer to the illumination L than other values of I.But in real environment the ideal illumination condition usually does not exist.For each image patch in the regions with slow illumination changes, the local maximum value of I is a reasonable illumination estimation of its central pixel.The mode MAX is defined as

I x y L x y I x y = −
The functions min(•) calculates the minimum value of a local window.After calculating L m and L s of I(x, y), illumination fusion is proposed to refine illumination estimation.In the fusion, we first distinguish between light shielding edges and other regions by image segmentation, and then define fused illumination estimation result Lms(x, y) of I(x, y) by ( 5) Where mean(•) denotes the average value of an image and k∈[0, 1] is a parameter, which is set 0.6 according to [22].For strong relationship among neighboring pixels of illuminations and better preservation of image edge information, anisotropic diffusion filter is proposed [24].Equation of anisotropic diffusion is presented as


Where (x, y) denotes the pixel position in the discrete 2-D grid; t presents the iteration step; K is the gradient threshold parameter; λ determines the rate of diffusion and ( , ) neighborhood of pixel (x, y): ( , ) x y η = {N, S, E, W}, where N, S, E and W denote the North, South, East and West neighbors of pixel (x, y), respectively.The symbol ∇ which in the continuous form is used for the gradient operator now represents a scalar defined as the difference between neighboring pixels in each direction: gK is the conductance function, and it is defined as ( ) The function ( 10) is chosen to satisfy 0 lim ( ) 1 x g x → = , so that the diffusion is maximal within uni- form regions, and lim ( ) 0 x g x →∞ = , so that the diffusion is stopped across edges.According to [24], K λ and t, respectively, are 15, 0.15 and 20.
Based on the classic Lambertian reflectance model, illumination invariant at the coordinate of (x, y)

Fast Mean Filter
The invariants of illumination are extracted after illumination estimation, but unfortunately, the processed images suffer from loss of information, unsmooth texture and interference of noise.For further improvement of processed images, fast mean filter is used to smooth images and remove noise, which can repair the defects caused by process of illumination invariants extraction.[each column contains m elements.]proved by about 50%, and filter template size is empirically set 3×3 in regard to [25].After fast mean filter processing, R FMF can be obtained.As shown in Fig. 3, it is clear that the FMF results are smoother and have less noise compared with the results of illumination invariants.

Normalization Transformation
After smoothing the results of illumination invariants by utilizing fast mean filter, a normalization transformation is developed, which can raise the richness of information of the processed images.The transformation is defined as following: The transformation (12) would make the processed image have more details information and look more natural compared with the FMF results, as indicated in Fig. 4. Fig. 5 shows the final results of different people.Original images selected from Yale B+ [27], CMU-PIE [28] and CAS-PEAL-R1 [29] databases, which are under complex illumination conditions, as displayed in Fig. 5(a).The corresponding results by our method are given Fig. 5(b).The results show that the detail informations in final output images are greatly preserved by our method.
Entropy is a measure of the richness of information in the image, and the larger entropy value the image has, the higher the information contained in the output image.
The entropy for the whole image can be defined as

Experimental Results
In this section, Yale B+ [27,28], CMU-PIE [29], CAS-PEAL-R1 [30] and ORL [31] databases are selected to evaluate the performance of the method introduced in this paper.Face recognition on those databases are still a challenging task owing to their complex lighting conditions.Yale B+ consists of 38 and facial details (glasses / no glasses).And all images are taken under a dark homogeneous background.
All selected images are resized to 64 × 56.According to the serial number order, one image of each subject is used as training samples, other images are used for testing, and the final result is calculated by the average value.PCA [32], KFA [33] and LDA [34] are used for testing in the same conditions, because they are highly sensitive to illumination variation.MSR [10], DCT [11], Gradient [20], Weber [21] and Cheng [22], as the state-of-the-art methods, are selected to implement performance comparison.

Experiments on Yale B+ database
In our experiments, we choose the frontal images under varying lighting conditions.The statistical results of recognition rates of PCA [32], KFA [33] and LDA [34] based on all the methods on Yale B+ database are shown in Fig. 4.
It can be seen from Fig. 6 that our introduced method performs the best in comparison with all other approaches, because it can more effectively preserve the image detail information and improve richness of illumination invariant.The recognition rates of PCA, KFA and LDA based on our method are 95.62%,97.78% and 98.72%, respectively.Specifically speaking, the recognition rate of performing the other approaches: MSR [10], DCT [11], Gradient [20], Weber [21] and Cheng [22] are discriminatively lower than performing our method on the Yale B+ dataset containing with full illumination variation.

Experiments on CMU-PIE database
In this section, we choose the frontal images under varying lighting conditions from CMU-PIE database.The statistical results of recognition rates of PCA [32], KFA [33] and LDA [34] on CMU-PIE database are displayed in Fig. 8.
As indicated in Fig. 8, most of the methods can obtain high recognition.The recognition rate of performing our method is better compared with the other state-of-arts.The recognition rates of KPA based on the mentioned methods are bigger than 90% except MSR, and the proposed method can get the highest recognition rate (99.05%).Additionally, the method introduced by us can also get the highest recognition rate by LDA.The recognition results of PCA [32], KFA [33] and LDA [34] are shown in Fig. 10.It can be known from Fig. 10 that the overall recognition rates by all the method is declined sharply.One reason is that the difference of poses and scale changes for the same person in CSA-PEAL-R1 database.One can find that our proposed approach performs the best by comparison with all other methods.Some different lighting images of subjects and the corresponding results of different methods are indicated in Fig. 11.From Fig. 11(b) to Fig. 11(g), we can know that our method can more effectively retain facial details and eliminate the influence of illumination change on digital images, which are under complex illumination conditions.And the final illumination invariants processed by our method look more natural and have more detail information.

Experiments on ORL database
We use 400 face images of 40 human subjects in our experiments on ORL database.As indicated in Fig. 12, the results of recognition rates of PCA [32], KFA [33] and LDA [34] on ORL database are displayed.Compared with other methods, we can know that our proposed approach performs the best, as shown in Fig. 12.The recognition rates of PCA based on our method can reach 50.31%, and recognition rates of the others are lower than 50%.Specifically speaking, the recognition rates of KFA and LDA based on our method are also perform best in comparison with the other approaches, which are 63.27% and 66.87%, respectively.Some images of subjects from ORL dataset and the corresponding results of different methods are indicated in Fig. 13.As shown in Fig. 13(a), we can see that the original images are under complex conditions, such as varying lighting, facial expressions and facial details (glasses / no glasses).The proposed methods can surpass the other methods in retaining facial details and eliminating the influence of illumination change on digital images under complex conditions, in Fig. 13(g), To sum up, as indicated in Fig. 6, 8, 10, and 12.The MSR method has the lowest recognition rates on Yale B+, CMU-PIE, CAS-PEAL-R1 databases.But on the ORL database, the recognition rates of MSR are higher than DCT by KFA and LDA.DCT makes unsatisfactory performances in the four public databases, but in most cases, it performs better than MSR.The performance of Gradient is better than that of MSR and DCT on all the tested databases, however, the performance is still not ideal.Weber can obtain higher recognition rates than MSR, DCT and Gradient on Yale B+, and it is mostly lower than Gradient on the other three databases.The Cheng method ranks as the second in the four databases except on ORL by KFA.Through the results of the comparison mentioned above, it is clear that our method can obtain the best performance on the databases used in this paper. 4

Conclusions
An effective method is used to solve the critical issue of complex illumination variation for face recognition in this paper.The proposed method is less insensitive to complex illumination variation and repairs the defects caused by process of illumination invariants extraction by using fast mean filter.
Improvement of illumination invariant by our method can make the output image get more detail information than other methods, thus a greater richness of information and higher recognition rate under complex illumination conditions can be obtained by the proposed method.Experimental results have shown that the proposed method has superior performance in face recognition by comparing with some state-of-the-arts.
Fig. 1 The flowchart of proposed method an image local window centered at pixel(x, y), O i, j denotes a neighboring pixel in Ω, and The functions max(•) calculates the maximum value of a local window.Illumination estimation by local maximum pixel value is unsuitable for the pixels on the light shielding edge with quickly changed illumination.(3) is used to estimate illumination of pixels in the regions with quick illumination changes.The mode MIN is described as , ,

Fig. 3 (
Fig. 3 (a) Results of illumination invariants used in Fig. 2 (b) FMF results of (a) is the element of R FMF , a=mean(R FMF ), b=std(R FMF ), and c=mean(R FMF ) / std(R FMF ).Where the function mean (•) calculates the mean value of RFMF, and std (•) calculates the standard deviation value of RFMF.

Fig. 5 (
Fig. 5 (a) Original images (b) Final results of illumination invariants subjects under 64 lighting conditions.The CMU-PIE includes 68 subjects under 21 illumination conditions with a little hair and background in the face images.The illumination of CAS-PEAL-R1 face database is different from Yale B+, and the number of illumination conditions is not the same for each subject.Besides, there exists illumination interference in this database.Moreover, it is more difficult for face recognition because of the difference of poses and scale changes for the same person.The ORL face database contains 400 face images of 40 human subjects with varying lighting, facial expressions Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 26 January 2018 doi:10.20944/preprints201801.0250.v1

Fig. 6
Fig. 6 Comparison of recognition rate on Yale B+ database Fig. 7(a) shows some original images of subjects from Yale B+ dataset, and the corresponding results of different methods are indicated in Fig. 7(b) -Fig.7(g).We can see that the original images are under complex illumination conditions.It is clear that the proposed method can more effectively retain facial

Fig. 9
Fig. 9 shows some complex illumination images of subjects from CMU-PIE database and the corresponding results of different methods.Compared with some state-of-the-arts, it is clear that our approach performs best on retaining facial details and eliminating the influence of illumination change on digital images retaining facial details, which are displayed in Fig. 9(b) to Fig. 9(g).

Fig. 8 3 . 3
Fig. 8 Comparison of recognition rate on CMU-PIE database 3.3 Experiments on CSA-PEAL-R1 database In order to build a fair comparison among the investigated approaches, 233 subjects under 10 different illuminations are chosen from CAS-PEAL-R1 database.Just like experiments on Yale B+ and CMU-PIE databases, one illumination of each subject is selected randomly as a training sample, and the others are taken as a testing set for face recognition, the final result is calculated by the average value.

Table 1 .
Entropy value comparisons in different methodsDatabasesEntropy value of different methods MSR