Bilateral Line Local Binary Patterns for Face Recognition

Local binary pattern is one of the visual descriptors and can be used as a powerful feature extractor for texture classification. In this paper, a novel representation for face recognition is proposed, called it Bilateral Line Local Binary Patterns (BL-LBP). This scheme is an extension of Line Local Binary Patterns descriptors in the statistical learning subspace. The present bilateral descriptors are fused with an ensemble learning of calibrated SVM models. The performance of this scheme is evaluated using 5 standard face databases. It is found that it is robust against illumination variation, diverse facial expressions and head pose variations and its recognition accuracy reaches 98 percent, running on a mobile device with a processing speed of 63 ms per face. Results suggest that our proposed method can be very useful for the vision systems that have limited resources where the computational cost is critical.


Introduction
Biometrics utilizes unique patterns of human body parts or its behaviors such as face, eye, hand, fingerprint, voice, signing for an identification purpose and it has been an important research area. Most of these biometric attributes maintain their characteristic over the lifetime. The question is how to pick them up safely and reliably because some of their setups are intrusive and the data acquisition is often difficult. In contrast, acquisition of facial data is relatively easy due to several factors such as non-intrusiveness, availability of diverse tools and low cost, and then face recognition becomes a viable technology in our modern life. Some interesting face recognition applications are introduced in public environments (driving license, military application, sporting event, airport, etc.) and in the private one (online service, banking, embedded application, mobile device security, etc) [1].
The standard face recognition system generally consists of three main stages: face detection, facial feature representation, and classification. The focus of the present study is on the facial representation stage. In fact, the key to successful face recognition should be how one can extract valuable and crucial facial features. The face recognition system cannot obtain good results when the input data are insufficient for any feature descriptors, even if the best classifier is utilized for the classification task. The aim of facial feature representation, of course, is to reduce the dimensionality of data while preserving their discriminative information for the classification task. The chosen descriptor should perform well in both criteria such as compactness and discriminative power. Finally, the processing speed has to be fast enough especially for mobile use. Many outstanding studies have been proposed and they can be grouped into two categories: the holistic-based methods and the local-based methods. The basic idea of holistic-based methods is to utilize all pixels on the whole face image. Several subspace methods such as Eigenfaces and its improvement [2], Principal Component Analysis (PCA) [3], Fisher Discriminant Analysis (FDA) [4,5] and Sparse Representation-based Classification (SRC) [6] belong to the holistic approach. However, this approach encounters some limitations including several unconstrained cases such as illumination, head pose or facial expression. In contrast, the local-based approach which is known as feature-based method utilizes the local facial feature for image representation. It considers mainly the geometry structure of each patch within the given image. The obtained features of this arrangement often cover more useful information than the holistic-based case. Some examples are Gabor-wavelet features [7], Local Binary Pattern (LBP) features [8][9][10][11] or the fusion of LBP and Gabor [12]. For instance, the Gabor-wavelet feature is widely applied to either the whole face or some small face regions to describe facial features with a bank of Gabor filters. Gabor algorithm requires high computational resources since a large number of Gabor filters is required to compute and store multi-scale and multi-orientation coefficients. With this similar approach, LBP is known as a non-parametric method efficiently summarizing the local structures of an image and has been popular in facial image analysis and LBP descriptors are tolerant against illumination or scale changes while keeping its computational cost low. Since the facial recognition system is basically to classify a given face image according to the facial features extracted using the above methods, it adopts several representative classifiers such as k-Nearest Neighbors (kNN), Support Vector Machines (SVM), Random forest, etc. at its last stage. For the best face recognition, a feature descriptor has to accompany with a high-performing classifier.
A lot of excellent works have been proposed to improve the performance of the system by solving some challenges in face problems. Recent challenges remain such as variations of facial expression, pose angle, illumination as well as background complexity (Fig. 1). Other difficulties are occlusion, aging, makeups, image quality, etc. It seems to be hard to have a method that can deal well with all these obstacles. Therefore, the face recognition problem is still an interesting topic.  Among those challenges, illumination variation is one of the most difficult obstacles since it can change the appearance of the face through increasing or decreasing the intensities of face images 3 of 18 by various light source directions. It requires a robust face representation not only works well with illumination variation but works well with facial expression. Thus, this study aims to overcome the drawback of representing face images against these variations. Inspired by the success of LBP for face recognition [13,14], we proposed a novel framework which is an extension of EL-LBP [15] called it BL-LBP (Bilateral Line Local Binary Pattern) for the face image presentation and recognition. This proposed framework aims to satisfy a number of challenging criteria: 1. Robustness and Compactness: The algorithm should be flexible and robust against variation of illumination, facial expression, pose and image quality. 2. Processing Speed: The time of the face representation stage and classification should be fast enough. 3. Discriminant Capability: For an unseen face image, the system can extract the efficient facial features and these features should represent for that image with a discriminant capability. Figure 2 shows a conceptual diagram of our method for the face recognition system. With this scheme, we have two primary contributions. The first contribution is to build a group of descriptors for the feature representation task. The fusion of LLBP descriptors by horizontal and vertical direction with Bilateral 2D-PCA subspaces produces compact and robust descriptors that effectively adapt to various angles of the light sources of the image input. The second contribution is to apply an ensemble learning strategy based on the late fusion of SVM models for the classification task which leads to higher accuracy performance. The evaluation results suggest that our method outperforms state-of-the-art LBP approaches, including a deep learning approach. The paper is organized as follows. Section 2 introduces briefly the principle of LBP and statistical subspace learning. Section 3 describes a few techniques on feature selection for the image representation since they are the main motivation of the present study. Our proposed framework will be described in Section 4. The experimental result using four face databases is presented in Section 5. In section 6, a conclusion is given.

Local Binary Patterns (LBP)
The basic LBP operator encodes the relationship between a central pixel and its surrounding neighborhood 3 × 3 pixels to make a binary bit string. That bit string can be easily converted to its corresponding decimal value to produce the LBP code. The process of encoding an LBP bit string is illustrated in Fig. 3. The intensity of each pixel g p in the neighborhood is compared to that of the central pixel g c (x c , y c ). If g p is either equal or greater than g c , then it is encoded as 1 otherwise as 0. These binary codes are concatenated in the clockwise direction to build a binary number, and its corresponding decimal value is given as a label. The original LBP operator can be extended by utilizing different sizes of a neighborhood to capture some dominant features underlying in the large scale structure [10]. The extended version of LBP can be described as Eq. (1), where (P, R) denotes the neighborhood of P sampling points on a circle of radius R at a given central pixel (x c , y c ): where s(x)= 1, x ≥ 0 0, x < 0 With this operator, LBP operator can work as invariant against the monotonic gray-scale transformations. Based on P pixels in the LBP neighborhood, 2 P different output values, corresponding to 2 P different binary patterns, are formed. Thus, we can construct a 2 p bins histogram of the LBP pattern as an occurrence frequency vector. However, there is a small subset of patterns, called "uniform patterns", which occur frequently and have the discriminative power. These special patterns contain at most two bitwise transitions from 0 to 1 or vice versa when the corresponding bit string is considered in a circular shape. With P = 8, there are 58 cases of these, and the counts for the remaining patterns are pooled into a single "non-uniform" bin, then LBP uniform (denoted as LBP u2 P,R ) obtains a 59-bins output histogram instead of a 256-bins histogram. From this definition, the number of different uniform output labels for mapping patterns of P bits in LBP u2 P,R is P(P − 1) + 3.

LBP Descriptor
For LBP-based face recognition, a face image is partitioned into m small regions S (j) (j = 0..m − 1 and m = k 2 , k ∈ N) having the same size and non-overlapping regions where LBP descriptors can be extracted and then concatenated into a single feature vector to represent the texture of the given face image. A histogram for j th local region S (j) is computed as follow: i is the number of pixels (with LBP label i) in the local region S (j) . With Eq. (2), the regional LBP descriptor for S (j) can be expressed: where T stands for the matrix transpose operator and the standard column vector H (j) belongs to L-dimensional real space. Note that in Eq. (3), H (j) provides regional LBP histogram information for S (j) . Finally, LBP histogram for all S (j) can be obtained by concatenating all H (j) into a single column vector: The LBP histogram H is applied to extract LBP feature descriptor of the given face image. Further detail for this process will be described in the following subsection.

LBP Feature Selection
For face recognition using the LBP based method, many studies propose to boost the discriminative capability within feature vectors such as: (1) increase the number of blocks within an image; (2) increase the number of neighbors to capture more dominant textures; (3) combine LBP with other features extraction method. However, these methods lead to a significant increase in feature vector length as well as a redundant representation including unnecessary information [16]. In addition, the issues for LBP feature selection have been studied. According to a recent survey [17], LBP feature selection techniques are classified into two main categories: (1) exploitation of some feature extraction techniques to find the best discriminant feature, although such initiative could boost LBP feature capacity it requires an off-line training processing; (2) reducing the feature-length of LBP descriptors by using some rules, although this approach has limited feature selection ability and yet it is simple and has the low computational cost.

LBP subspace learning
Subspace learning aims to reduce the dimension of face images by projecting the original data into a new subspace wherein one can obtain an optimal and lower-dimensional representation [18,19]. It tries to boost LBP features by combining many individual LBP features with other holistic/ local features to take advantage of each feature type without caring about the dimension. Then, several well-known dimension reduction methods such as PCA, FDA and Gabor wavelet are applied to obtain a low-dimensional compact representation.
For instance, Linear Discriminant Analysis (LDA) [20] was utilized to convert a high dimensional space using multi-scale LBP features into a discriminant space, often yielding very promising results. Although a combination of the Gabor wavelet and LBP feature could lead to a high dimensional space, it was successfully performed by Tan and Triggs [21] by using PCA to obtain an optimal space. The Kernel Discriminative Common Vectors (KDCV) was applied to extract discriminant nonlinear compact features for face recognition. Zhao et al. [22] introduced a fusion of Laplacian PCA (LPCA) and LBP for feature selection by proving that LPCA outperformed PCA and KPCA.

Rule-based strategy
The rule-based strategy aims to reduce the dimension of feature vectors within the algorithm of LBP using a certain rule. For instance, the number of LBP labels such as Local Line Binary Patterns (LLBP) [14] or Center-Symmetric Local Binary Patterns (CS-LBP) [23] can be reduced for a compact histogram while considering the large neighborhood. Or, selecting a subset of available bins instead of using the whole set of patterns is another way.
Lahdenoja et al [24] introduced a new concept of symmetry for dimensionality reduction as an LBP-based face descriptor. Here, patterns were assigned different levels of symmetry based on the number of ones and zeros they contain. A high level of symmetry patterns proved to occur more frequently in facial features having the best discriminative efficiency. These patterns seem to have a certain invariance property against environmental factors such as illumination or rotation. In [25], the authors showed that a properly selected subset of patterns encoded in LBP forms an efficient and robust texture description which can achieve better classification rates in comparison with the whole LBP histogram. They introduced a method based on a beam search. The classification of the optimization Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 27 May 2020 doi:10.20944/preprints202005.0451.v1 set within each iteration was made using a reduced LBP histogram that contained one bin for each pattern chosen so far. The contributions from all remaining patterns (out of 256) were combined into a single bin. In their experiments, the method based on feature selection by beam search performed better than the whole set of patterns or the uniform patterns when classifying tilted textures and using non-tilted samples for training. Smith et al. [26] proposed a fusion method by using LBP to generate a large number of candidate features and then selecting from them using fast correlation-based filtering (FCBF). FCBF operates by repeatedly choosing the feature that is most correlated with class, excluding those features already chosen or rejected, and rejecting any features that are more correlated with it than with the class. As a measure of correlation, the information-theoretic concept of symmetric uncertainty is used. When applied to the LBP features, FCBF reduced their number from 107,000 down to 120. Guo at el. [27] presented a learning framework which is formulated into a three-layered model. By a simultaneous consideration of the robustness, discriminant power and representation capability of features, this model can produce the optimal pattern subset of interest. With this improvement, this learning model can be integrated with existing LBP variants to derive new image features for texture representation and classification. Our present approach is a fusion of LBP-based rule and subspace learning to boost the discriminant of these compact descriptors.

Line Local Binary Patterns (LLBP)
The algorithm of LLBP considers only the pixel neighbors of a given central pixel in the horizontal or the vertical direction instead of the grid shape of the neighborhood. The basic idea is illustrated in Fig. 4 and it is similar to the original LBP but there are some differences: 1. The shape of neighborhood is a straight line either in the horizontal or the vertical direction. 2. For a given central pixel, the number of neighbors to the horizontal direction should be identical to that in the vertical direction. If the number of one direction is less than that of the other direction, a 0-padding mask is inserted to balance the neighbor to make the total number, called it N, to be even. 3. A bit string comes by comparing all neighbors to the central value: the binary bit is to be 1 if the neighbor pixel is greater or equal to the current element, otherwise it is to be 0. Each element in the bit string is multiplied by weight according to its position (described in Fig. 4, Eq. (5) and Eq. (6)) and the current pixel is replaced by the sum of all values.
4. The range of values in LLBP representation (A LLBP ) must be in a range from 0 to 255 by Eq. (7) A short where N is the number of neighbors, c is a given central pixel, g c is the value of center c, h n is neighborhood pixel on the horizontal line and v c is the neighborhood pixel on the vertical line, and s function is same as in LBP. Then, this descriptor is defined by the histogram of LLBP patterns and the number of bins depends on the size of the neighborhood. the histogram. However, since the number of bins in this way is too long, we propose a strategy to reduce the number of bins and described its detail in the next section. The algorithm of LLBP calculates the binary code along the horizontal and the vertical direction separately. Figure 5 shows an example of step 4 in the LLBP algorithm to convert a bit-string to decimal values in a new range of values. Thereafter, the example of LLBP images for these directions is shown in Fig. 6.

The facial matrix (LLBP descriptor)
After applying the LLBP operation on an input image, as a result, we obtain two LLBP images by horizontal and vertical direction. Each of them is manipulated to produce an LLBP descriptor respectively, called it the facial matrix as displayed in Fig. 7  Firstly, we find all uniform patterns corresponding to the number of neighbors and convert all of them into decimal numbers as shown in Table 1. A local binary pattern is called 'uniform' if its uniformity measure is at most 2. Here, BIT denotes the number of bits in a bit string (the number of neighbors) and BIN indicates the number of bins that can be obtained by counting LLBP patterns. As Eq. (7) only covers LLBP values from 0 to 255, we consider the range from 2 to 8 and the neighborhood must be an even number. With this strategy, we obtain 57 bins when the number of the neighborhood is 8, whereas the traditional LBP uniform approaches produce 59 bins.

Bilateral 2D-PCA Subspace Learning
2D-PCA method [28] is a well known dimension reduction method since it has two important advantages over PCA: (1) it is easier to compute the covariance matrix accurately; (2) less time is required to determine the corresponding eigenvectors. A bilateral 2D-PCA [28,29] is a variation where one encodes the spatial information embedded in both column and row directions of facial matrix.
Let A k M k=1 be a training set, where A k denotes k th an (m × BI N) training sample and M the total number of training set. The average matrix is denoted A = 1 M ∑ k A k . Then a non-negative definite matrix G row BI N×BI N by the row direction can be evaluated as follow: Similarly, we consider the column direction of FM matrix.
and A (j) denote the j th column vectors of A k and A, respectively. Then the covariance matrix G col m×m by column direction can be evaluated as follow: We obtain 2 optimal projection matrices X opt = [X 1 , ..., X d ] and Z opt = [Z 1 , ..., Z q ] by computing the eigenvectors of Eq. (9) and Eq. (10) corresponding to the d and q largest eigenvalues, respectively. The Bilateral 2D-PCA learns two optimal matrices X and Z from a set of FM training descriptors, and then project input matrix A onto X and Z respectively, yielding coefficient matrices Y row and Y col The vector Y q×d in Eq. (11) represents an optimal feature of input matrix A m×BI N . Therefore, with two LLBP descriptors by horizontal and vertical direction, we obtain two optimal features by directions respectively:

Ensemble learning
The importance of facial features and classifiers has been emphasized above and selecting the right classifier among many plays a crucial role in achieving a good result [30,31]. Since it is known that SVM works very efficiently in the multi-classification case with the presence of various constraints, we adopt an SVM based calibrated classification model.
Let the output of a learning method be f(x). To get the calibrated probabilities, pass the output through a sigmoid: where the parameters A and B are fitted using the maximum likelihood estimation from a training set ( f i ; y i ). Gradient descent is used to find A and B by minimizing the negative log likelihood of the training data: where The Platt algorithm [32,33] is used for the mapping model prediction to the posterior probabilities by an additional sigmoid function. However, to avoid overfitting to the sigmoid train set, output of the sample model is used. If there are N + positive examples and N − negative examples in the training set, the Platt calibration uses the target values y + and y − (instead of 1 and 0), respectively, for each training example, where Individual Calibrated SVM models are trained with LLBP descriptors by vertical and horizontal direction respectively. Each model obtains a prediction vector P = (p 1 , p 2 , p j , ..., p n ) T , where p j is the probability which belongs to j th -class of model and n is the number of classes for classification. The fusion is operated as below:P = (p 1 ,p 2 ,p j , ...,p where P h and P v are probabilities of calibrated SVM classifiers by horizontal and vertical direction respectively. Thereafter, a majority voting is considered for the fusion vector and the final decision depends on the class with the highest probabilities.

The BL-LBP features algorithm and its characteristic
This section summarizes the proposed method as Algorithm 1 below.  Table 1  The BL-LBP feature has several advantages over traditional LBP methods. Given that BL-LBP is an extension of the LLBP method, it inherits the characteristics of LLBP and so it is robust against changes of illumination and scale, especially when a source of light lies in one part of a face (either left or right and either top or bottom). The overflow transformation of Eq. (7) and counting the occurrence frequency of the value in matrix A (i)−short LLBP based on series of intervals would produce a descriptor with a constant size, even though the number of neighbors varies. Although dividing the original image into many smaller blocks can lead to the larger dimension, an arrangement is made to fuse sub-histograms for a facial matrix. The computation cost of the Bilateral 2D-PCA on the facial matrix is handy and it is faster than any method that uses a long vector. By using 98% Contribution Rate of Eigenvalues (CRE) in the Bilateral 2D-PCA, principal components of the row and column direction subspace are kept and yet the number of principal components is very small. Such maneuvering makes this feature descriptor efficient and fast for the classification task.

Configuration for experiments
Our BL-LBP scheme was evaluated using four standard face databases such as ORL, YALE, AR, and Aberdeen as shown in Fig. 5. Each database was divided into training and testing set and each set occupied 50% of the total samples. The model ran on both a PC and a mobile device. The configurations for both cases are detailed in Table 2. The experiment for a PC was prepared to evaluate its performance, while that for a mobile device was for the real-time face recognition application.   Figure 8 illustrates some images on four databases.
A cross-validation method was designed for the measurement of the accuracy and processing time for each database. Half of the samples were randomly selected for training and the remaining half for testing, and this evaluation was repeated 100 times by shuffling data. Therefore, recognition  results were the average values for 100 random permutations. Moreover, each original image was preprocessed by applying denoising techniques for ORL and by applying histogram equalization for the others after converting images into gray levels, respectively.

Face recognition based on deep features
For a fair evaluation between methods, an experiment based on deep features is carried. With the advantages of pre-trained deep learning models on the general ImageNet database which has more 1.2 million images with 1000 categories, the transfer learning of deep models has been applied to many recent application regarding face images such as Race recognition [34], or smile detection [35]. Therefore, a fine-tuned strategy of pretrained model ResNet-18 [36] is considered in this experiment for comparisons. The network is a good trade-off between depth (compared to AlexNet, VGG-16, ResNet-50) and performance. The ResNet-18 architecture which is summarized in Fig. 9 consists of 18 layers. The size of feature vector of the ResNet-18 model is 512 at the average pool layer. The input of the network is an RGB image of size 224 × 224. For our classification task, we modify the network by adding average-pooling and softmax layers to the origin ResNet model. The optimization goal is to maximize the accuracy performance by a SGD optimizer with a learning rate of 0.01. The input images are also augmented such as rotation, flipping, scaling for overfitting avoidance. The results of several datasets are reported in the next section as a comparison.

Experiment 1: Accuracy performance
In this section, BL-LBP scheme was evaluated and compared to the previous LBP methods as well as deep learning. The recognition rate of several methods were shown in the Table 3 where our BL-LBP showed the highest accuracy for several databases and it reached 98%. Table 3 shows that the recognition rate of LLBP is higher than those of LBP u2 8,1 and LBP u2 16,2 . For effective descriptors of the face recognition, each image of LBP methods was partitioned into m(m = k 2 ) regions, thus the dimension of feature descriptor was very huge m × q (q is the number of bins in the histogram). For instance, m = 49, the feature descriptor for LBP u2 8,1 required 2891 (= 49 × 59) bins and for LBP u2 16,2 required 9555 (49 × 195) bins. The dimension of LLBP which was described in parentheses corresponding to each database could reach to 25600 (= 1024 × 5 2 ) bins. However, BL-LBP, which were extended by LLBP, overcame this drawback since it required very few feature descriptors since the longest feature descriptor had only 728 (= 13 × 56). Without caring the parameter k and BI NS, BL-LBP descriptors considered two principal component directions, thus its dimension was reduced by at least 3 times compared to LLBP. Compared to the deep learning approach (ResNet-18), our BL-LBP framework produced higher accuracy performance on YALE, AR, and Arberdeen databases except for a little bit worse accuracy performance on ORL. Generally speaking, BL-LBP outperformed the others and it ran very well on an embedded machine.  was lost, making it a tough database. The larger databases with facial expressions such as AR and Aberdeen databases were actually a challenge for face recognition. Besides expression variation, Aberdeen database was made under strict lighting conditions while AR database was collected by changing head poses extremely. However, BL-LBP was on the top in term of accuracy: 98.33% on AR and 98.84% on Aberdeen, respectively. Also, it handled well the pose variation problem of the ORL database. Such result suggests that BL-LBP has some characteristics that is robust against the facial expression variations and the extreme head poses.
For the details, we explored the accuracy of various methods by considering m = k 2 local facial regions as shown in Fig. 10 while the input image of the deep learning approach was not partitioned. Therefore, we showed only the result of deep learning as a point to compare methods. In fact, the accuracy of each method dramatically increased when the value of parameter k was in a range of 1 to 5, and yet that of BL-LBP slightly increased. However, BL-LBP was always on the top accuracy when the value of parameter k was in the range of 6 to 8. In this case, BL-LBP outperformed LLBP and the others in terms of the recognition rate and the dimension of feature descriptors.

Experiment 2: Evaluation on an extension of YALE face database under illumination variations
The extension of YALE face database [37] is a combination of the YALE face database B with 10 subjects and the extended YALE face database B with 29 subjects. Both databases contain face images under 64 illumination conditions with 9 poses. In this experiment, we conduct only frontal face images for each subject with various illumination, thus this data contains 2,467 frontal face images of 39 subjects. The database which is divided into 5 subsets is displayed in Table 4. The goal of this split is to check the tolerance of several methods according to various angles of the light source.
The impact of illumination variations was evaluated by picking two images/subject from all subsets for the training set, 5 subsets with the remaining images were 5 test sets. Table 5 compared the results of each subset as well as the overall performance. LLBP method showed the worst result among methods since its descriptors were huge and contained redundant information and decreased the performance. Contrastly, the feature descriptors of BL-LBP and ResNet-18 were rather small and can produce higher accuracy. Our BL-LBP scheme obtained the highest and lowest accuracy on Subset 2 and Subset 5, respectively. With Subset 5, the accuracy of the deep learning approach was a little bit better than BL-LBP. However, the overall performance of BL-LBP was higher. This experiment proved that our BL-LBP scheme can be tolerant of illumination variations very well and works better than the deep learning approach (ResNet-18).

Experiment 3: Processing Time
The processing speed for the BL-LBP scheme was measured using a PC and a mobile device (Galaxy S8). The processing time is an important factor for the real-time application. As the mobile devices have a limited computational resource, the processing time is considered as a crucial criteria. For a fair comparison, all experiments was carried out under the similar conditions for both machines. The detail information for libraries and hardware specifications is given in Table 2. Table 6 described the total processing time for the whole process of face recognition, i.e the average time for a testing query on an image on several methods. Note that the average time for LBP u2 16,2 , LLBP and LBP u2 8,1 was 193, 153, and 106 ms, respectively, whereas that for BL-LBP was 63 ms, indicating that BL-LBP was 2.4 times faster than LLBP. The deep learning approach was faster than BL-LBP on the PC machine, mainly because ResNet-18 ran on a GPU for its manipulation while BL-LBP on a CPU. The processing time of ResNet-18 on a mobile device could not report here since this deep learning model was not fitted for a machine with limited resources. Under the CPU configuration, the processing time of our BL-LBP was the fastest one.

Conclusions
Local Binary Pattern is one of the favored visual descriptors in recent years. It has been used as a powerful feature extractor for diverse computer vision problems. In this paper, a novel BL-LBP scheme is proposed for face image recognition which is easily suited to a mobile application. The present encoding scheme for face recognition is extended from Line LBP descriptors where bilateral statistical learning is applied for efficient descriptors. Such descriptors are then boosted with an ensemble learning strategy of SVM models for the classification task. Performance evaluation using 5 face image databases suggests that the BL-LBP scheme is robust and flexible under various conditions such as variations of illumination, extreme head poses, scale change, and facial expressions. The proposed scheme outperforms the previous LBP-based methods as well as a deep learning approach. The major advantage of it is that it allows making bilateral feature descriptors from a raw image. Since the dimension of feature is smaller than other LBP methods, this representation scheme makes a better sense in the mobile vision system where the computational resources are critical as we demonstrate here.

Conflicts of Interest:
The authors declare no conflict of interest.