Recognition of Faces using Efficient Multiscale Local Binary Pattern and Kernel Discriminant Analysis in Varying Environment

: Face recognition involves matching face images with different environmental conditions. Matching face images with different environmental conditions is not a easy task. Also matching face images considering variations such as changing illumination, pose, facial expression and that with uncontrolled conditions becomes more difficult. This paper focuses on accurately recognizing face images considering all the above variations. The proposed system is based on collecting features from face images using Multiscale Local Binary pattern (MLBP) with eight orientations out of 59 crucial ones and then finding similarity using a kernel linear discriminant analysis. Literature suggested that MLBP can give up to 256 orientations for a single radius considered around a pixel and its neighborhood. The paper uses only 8 orientations for a single radius and four such radii (1, 3, 5 and 7) are considered around a single pixel with (8x4) 32 histogram features thus reducing the computational complexity. Various face image databases are considered in this paper namely, Labeled Faces in Wild (LFW), Japanese Female Facial Expression (JAFFE), AR and Asian. Results showed that the proposed system correctly identified 9 out of 10 subjects. The proposed system involves preprocessing including alignment and noise reduction using a Gaussian filter, feature extraction using MLBP based histograms and matching based on kernel linear discriminant analysis.


Introduction
The main function of face recognition is to identify faces from a given image or a sequence of images. There are many such systems used in security and surveillance applications. The task of face recognition is simple but becomes challenging when encountered with various challenges such as variation in illumination, pose, expression and somewhat occlusion etc. These challenges make face recognition an active area of research in both computer vision and pattern recognition. The progress of face recognition technology over past decades is substantial as benchmarked by National Institute of Standards and Technology (NIST).
Face recognition techniques give more accurate results for the images which are collected under controlled environmental conditions. The compressed facial images such as passport, visa and ID card are subject to losses; while less compressed images such as mug shot are generally of higher resolution, also exhibit considerable pose, illumination and expression variations. To identify faces in an uncontrolled environment is still a challenge for facial recognition reliability. The performance of FRT deteriorates when encountered with variations in illumination, pose, expression and somewhat occlusion. In this paper a method is proposed for matching the faces in unconstrained conditions. It also considers various challenges such as changing illumination, pose, facial expression and somewhat occlusion of eyeglasses by using the faces from four different databases.
Face recognition is done by various researchers as reported by Beham and Roomi (2013) using various methods and techniques. Features are extracted by combining PCA and LDP is proposed by Luo et al (2015). In this PCA is used for collecting global features and LDP is used for collecting local features. Both features are combined and classified using SVM to improve the accuracy for facial expression. The representative information is collected via dictionary learning without prior knowledge as reported by Wei and Wang (2015). A learning based unsupervised method encodes local microstructures of face to code histogram is presented by Cao et al. (2010). Using global and local features a face recognition method is proposed based on Zernike moments and Hermite kernels to deal with changing illumination, pose, facial expression and somewhat occlusion by Farokhi et al. (2015).
Classic image representation methods include HOG by Wu et al. (2012), Gabor by Hernandez and Romero (2003), SIFT by Infantino et al. (2009). Among these LBP (Local Binary Pattern), has prominent advantages. LBP is an image descriptor able to capture image texture. It is simple, computationally efficient and powerful in terms of its performance. Hence it is widely used in face recognition and texture analysis. The proposed method uses LBP with various scales. LBP extracts the data from small microstructures so it is not able to capture large scale structures which may contain dominant textural features. Therefore LBP is used with various scales for extracting the data from face image. These features of multiple scales are concatenated using histogram features. The proposed method collects the features from face images using multiscale local binary pattern considering only 4 radii. Only 32 histogram features are obtained from single face image reducing the computational complexity.

Related Work
Face recognition of humans is a very active research area comprising various descriptors and classifiers to tackle the problems of variation in pose, illumination, occlusion etc. due to non-linear effects. Existing improvements and applications of LBP are studied and summarized by Zhao (2012). A fast face recognition system is proposed using LBP on symbian platform by Sum et al. (2012) although the computational ability of symbian platform is limited, the experimental results show good performance. For reducing speckle noise a novel local binary pattern based adaptive diffusion is presented by Mandava and Regentova (2012). The discriminative features of LBP and HOG descriptor are proposed by Bhele and Mankar (2015). The problem with conventional LBP is that it is sensitive to noise in near uniform image regions. It can be overcome by using Local ternary pattern proposed by Tan and Triggs (2007). LBP descriptors are sensitive to rotation and scaling. To tackle this problem Ojala et al. (2002) considered circular LBP with varying radius and increasing sampling points. As the number of sampling points increases LBP becomes to sparse and is not able to discriminate image texture properly. Liao et al. (2009) selected rotation invariant patterns instead of using uniform patterns. Guo et al. (2010) presented hierarchical multiscale LBP which improve performance by extracting data from non-uniform bins. The extension of LBP texture analysis operator to multiple scales is proposed by Maenpaa and Pietikainen (2003). In this method increase in classification accuracy is not large enough as compared to increased computational burden.

Proposed System
In the proposed system face is recognized by extracting the features from filtered image and classified as shown in Fig. 1.

Preprocessing
Database considered in this work consists of images which are well posed and need not require alignment since the line joining the pupil was nearly horizontal. But some of the database face images require alignment for better matching. Therefore, a separate module was implemented in MATLAB 2010a to align the images. The module merely provide a pointer by which a line joining the pupil would give an angle with respect to horizontal line and then the face image would be rotated by the desired obtained angle in either clockwise or anticlockwise direction to correct the alignment. Some of the images are shown in Fig. 2.

Gaussian Filter
Face images are prone to noise. Some of them are inhomogeneous illumination and random noise. As per database available mentioned in the abstract, experimentally it is found that Gaussian filter is able to correct most of the noise to have good features after filtering. A 3x3 filter is used to reduce the noise effect over the image with sigma=2. The Gaussian filtered image is shown in Fig. 3.  Local Binary Pattern Ojala et al. (1996) presented Local Binary pattern which extracts features are extensively used in many applications due to its ruggedness and simplicity. It involves thresholding neighboring pixels based on center pixel value. That is, if the neighboring pixel has value less than center pixel then it is assigned a value 0, otherwise a value 1 as shown in Fig. 4.
After thresholding, the neighbors can be arranged in 256 different ways for 8 values and then the corresponding decimal value is calculated for each combination of the 8 pixels. The 8 combinations considered in this work are shown in Fig. 5 for a radius of 1 around a center pixel.
The first pixel considered was left corner pixel for our work.
These binary patterns are extracted and collected into histogram and then it is used as a texture descriptor. The disadvantage of LBP is its small spatial area. Features calculated for 3x3 neighborhood area is not able to extract large scale features which may be the important features of some textures. Along with this the operator is not robust against local changes in the texture, which may be due to variation in illumination or viewpoints. Therefore operator is needed with large spatial support.
Therefore the operator is extended at multiple scales for rotation invariant image analysis.
Multiscale Local Binary Pattern (MLBP) In MLBP, four radius are considered around a center pixel with R=1, R=3, R=5 and R=7. Eight pixels are considered for pattern around the center pixel as shown in the Fig.  6. Only eight pixels are considered for each radius as it reduces the computational time and complexity. Eight orientations for each radius correspond to 8x4=32 decimal values around a single pixel in a 10x10 window. A histogram was considered for each radius. To correct border pixels, they were padded correctly to avoid losing them. All images are resized to 200x150 pixel images. 10x10 windows are considered for extracting features. Therefore total windows along column sides are 15+14=29 and along row side 20+19=39. Total windows obtained are 29×39=1131 for a single image. Therefore, each image was featured with an array of 1132×32=36192 histograms.

Classifier
The following steps of classification are applied on each column of both training and testing images.
Consider Ds corresponds to an array of 36192×256 for training face image and D p corresponds to an array of 36192×256 for test image, then similarity is given Fig. 6: . / S Num Den = If the similarity value is 1, then both the training and test images are identical, if similarity value is 0, they differ in all respect.

Results and Discussion
Challenges of face recognition such as variation in pose, illumination, facial expression and somewhat occlusion in unconstrained conditions is considered in this paper. The experimental evaluation is done by considering the face images from various databases such as Labeled Faces in Wild as reported by Huang et al. (2011) Japanese Female Face Expression by Lyons et al. (1998), Asian by Jalal and Kim (2014) and AR by Martinez and Benavente (1998) are considered for evaluation. The total subjects corresponding to four databases available were 13000,213,200 and 4000. The proposed work is carried out on 500 subjects from the databases for this paper. The face image of size 200×150 is considered for experiments which is filtered using Gaussian filter with sigma=2 for removing noise and in homogeneity. Then LBP is applied to 10x10 window of each image and the window is moved without using overlapping concept as overlapping of windows provides redundant features for face image. The concept of overlapping of window can be effectively used for extracting features from sketches. The LBP is applied on various scales with radii R=1, 3, 5 and 7. From total 256 orientations only 57 orientations are important and from each radius only 8 orientations are considered in this paper as it reduces the computation time and complexity. Total 1131 windows are obtained from single image. From these windows an array of 36192 features are obtained. These features are then classified using the classifier mentioned above. Some of the sample images from various databases and some of the recognized images are shown respectively in Figure 8 and 9. The results of classification are obtained as similarity values which is shown in Table 2. Each column indicates the similarity value, the highest value in the column designates the matched image. The diagonal value shows matching correctly in bold text while non matched images are shown in red color. The proposed method is compared with other methods as shown in Table 1.
Based on the experiments performed on databases the proposed method achieves considerably good classification accuracy. It can also be inferred that the proposed method using MLBP is robust to illumination changes, pose variation, change in facial expression and somewhat occlusion in unconstrained conditions. The computational complexity and time required by the proposed method is less as compared to Gabor based methods.   Bhele and Mankar (2015) Proposed Method using MLBP LFW, AR, Asian, JAFFE Approx. 99% Table 2. Table indicating

Conclusion
The complete system was implemented using MATLAB 2010a. Even if the histogram features considered for each radius was 8, the system requires 5-6 minutes execution time for single face image matching over all 500 images (face images) in the database. The faces from various databases suffer from changing pose, expression, illumination and somewhat occlusion. In spite of all these changes we got more accurate results which increases the recognition rate. We had increased the orientation from 8 to 12, 15 and 20, but there was no significant change in classification results. Also the filtering no matter enhanced the quality of image but at the cost of feature loss. The window size considered in this work was 10x10 keeping the computational complexity low, better results can be obtained by reducing the window size. The similarity value between the training and test images is calculated as per the equation 1, 2 and 3 a better kernel can be used to improve the results. As compared to other technique in (Ojala et al., 1996), our results are superior.