Facial Features for Template Matching Based Face Recognition

.


INTRODUCTION
In past decade, face recognition has made great strides. It has attracted global attention and rapidly maturing growth in researches. Nowadays, face recognition has been widely applied in security system, surveillance system, credit card verification and other applications. Automatic face and facial features detection have become important for face recognition. Automated face recognition has been applied in holistic and feature-based methods. The holistic approach [4,5] treats a face as two dimensional pattern of intensity variation. Eyes and mouth are important facial landmarks which significantly affect the performance of face recognition. The feature-based approach [3,12] recognizes a face using the geometrical measurements taken among facial features.
Many researches have proposed methods to find the eyes [9] , mouth regions [5,6] and locate the face region [3,4,8] in an image or video [11] . These methods have shown the popularity of using information such as template matching, geometrical and intensity features. Brunelli and Poggio [3] and Beymer [4] located eyes using template matching. Template matching and eigenspace methods require normalization of the face for variation of size and orientation. A large number of templates are needed for template matching to accommodate varying poses.
The vertical and horizontal projections are one of the popular methods used in feature based approaches. The projections can be applied on the first derivative of the image [3] or directly on the intensities values. Projection based methods [7,9] have been used particularly to find coarsely the position of the facial features. The projection method is being combined with other methods such as genetic algorithm [8] and gabor transformation [18] . The amount of work that gave qualitative results about the performance of features localization is limited as most of the previous studies [7,9,12,13] excluded the effects of beard or moustache.
In the most recent researches, a fast facial features detection algorithm has been developed based on generative frameworks [19] . Adaboost is used to search for features through the entire face to get all possible combination of facial features. The final combination is determined by maximum likelihood. The detection rate is arguable as a detection of more than three features is considered as successful detection rather than calculate the rates separately. There is a need to manually zoon the image before running detection for features which are too small. Besides, a new facial features localization technique [20] has been proposed based on multi-stream Gaussian Mixture Modal (GMM) to locate regions of eyes, nose and mouth corners. PCA is used for feature extraction but the detection of eyes at the initial stage was not fully automated.
As for classification method, template matching [3] and neural networks [15] have been widely implemented. Template matching is sensitive to variation of poses while neural networks consume time in training when dealing with bigger database. Fuzzy clustering has been combined with parallel neural networks [14] to classify human faces. In this study, an algorithm has been proposed to reduce the effects of thick beard, moustache and facial expressions on facial features detection. The proposed algorithm will first locate the face region using skin-color. Iris candidates are extracted and selected from valleys based on computed costs. Then, the algorithm detects the lips region using the RGB color space. Image processing and connected component labeling have been used to eliminate unwanted noise and extract the lips region. SUSAN algorithm is applied to detect the mouth corners in order to obtain the accurate position of mouth. Finally, template matching is used to recognize faces in database based on extracted eye and mouth templates.

Face region extraction:
In this experiment, color images for males and females with beards, moustaches, long hair and facial expressions are specifically chosen from the online face database [16] . The images are assumed to be head-shoulder images with plain background and head rotation on y-axis within the interval ±30°. The proposed algorithm first creates a skin-color model from color images: Using the r,g color space as in Eq. 1, a skin-color model is created. First, select skin-color pixels whose color value v = (r,g) satisfies the set threshold. Second, estimate the mean and covariance matrix of the pixels in the selected regions [2] . Then, the Gaussian distribution model is computed using the probability density function. Mathematical morphology: opening and closing are applied to the region of skin-color pixels. Finally, the largest area of connected component of the region is defined as face region.

Detection and selection of iris:
The proposed algorithm first converts the color image to grayscale image and applies grayscale closing [1] to the face region to extract intensity valleys: where, g(x,y) and i(x,y) denote the value obtained by applying grayscale closing and its intensity value. Then, region consisting of pixels which is greater than or equal to a threshold value are determined to be valleys. Preprocessing steps such as illumination normalization [17] and light spot deletion are performed to enhance the quality of image and reduce illumination effect in this algorithm.
This algorithm performs the similar method [2] for iris selection. For iris candidates' selection, the costs for all pixels in the valleys are computed and select m pixels according to non-increasing order that give the local maxima. First, the eye template is being placed at each candidate location and measures the separability [2] between outer and inner regions. Second, Canny edge detector [10] is applied to the face region. The fitness and maximum vote ratio of iris candidates are measured using Hough transform. Third, the fitness of the iris candidates to the intensity image is measured by placing two different templates [1] . Fourth, the average intensity of each iris candidate over the average intensities for all candidates has to be calculated. The sum of all the four calculated costs for each iris candidate as mentioned is being used for final determination as in Eq. 3: Where: C(i) and C(j) = The sum of the computed costs for each pair of iris candidates R(i,j) = The normalized cross-correlation value computed by using eye template which is manually cut from a face image t = The weight to adjust two terms of the cost Fig. 1: The detected mouth corners after applying the image processing steps and SUSAN corner detector The lowest cost F(i,j) is selected as the pair of irises.
Mouth detection and refinement: First, the detection process starts by using the detected irises location as the left X l -5 and right X r +15 boundaries in estimating the mouth region. Then, the vertical positions of both irises are used to calculate a mean value, Y min : Where: Y 1 and Y 2 = The upper and lower boundaries R = The height ratio H = The preference height of the region The region of interest with the calculated width and height from Eq. 4 is being clipped out of the original color image.
The proposed algorithm applies RGB color space on estimated mouth region. RGB color space is being applied to the clipped image to allocate only the red lips region. So, the red component is compared with the green and blue in RGB color space as in Eq. 5 with a threshold being set: where, I is the detected lips which appeared in white region.
Among all the image processing steps, connected component labeling plays an important role to refine the mouth location. In this case, it is being used not only to eliminate unwanted regions but also detected the lips region as the biggest area in the estimated region. Opening, closing and dilation of the mathematical morphologies are applied according to the thresholds being set for various areas of connected components to have a more complete lips region. Median filter is also being used to eliminate noises and small clusters. After applying all the image processing methods, the largest member of the connected component is being chosen as lips.
SUSAN corner detector is then applied on the processed image. Corners will be detected at the boundary of the white region. The two corners located at the left most and right most are determined as mouth corners as shown in Fig. 1. When there is more than one corner with the same x-position at each side, the point with the nearest y-position to the middle of the image will be considered as a corner. Finally, the average values of y-coordinates form both sides can help in minimizing the error rate and adjusting the mouth corners' positions. This can be very useful when there is only one failure corner detection with small deviation at either side. The result of mouth corners detection is used to refine and center the position of mouth.

RESULTS
There are 72 color images from AR face database which have been chosen to evaluate the performance of the proposed algorithm. The size of each image is 768×576 and the database of 12 subjects for male and female are used respectively. These images chosen are covered variety of poses, head orientations and different face contours. Most of these images are with beard, moustache, long hair, facial expressions, mouth closed and opened conditions.

DISCUSSION
The size of the face database used in this experiment is not large but the variety of faces and subjects are more than adequate to test the robustness of this algorithm. From Table 1, the overall performance of this algorithm in iris detection has achieved 93.06%. The unsuccessful cases are caused by the color of female's irises. When the intensity difference between iris and outer region is lower, failure detection might happen. Besides this, when the subject frowns too hard, the eyebrows will come closer to iris. If the iris is almost covered by the eyelid, a failure might also occur. However, the algorithm has shown good accuracy in iris detection dealing with these types of situations. The iris detection rate can still be improved by using an eye template with more similarity compared with most of the images in the face database. For mouth detection, the proposed algorithm achieved 95.83% in detecting the mouth corners for refinement purpose. This algorithm combining RGB color space and SUSAN corner detection techniques has shown significant detection rate over the images with beards, moustache, facial expressions and orientation angles. The performance of this method has been proven better than template matching (83.33%) and integral projection (81.94%) by testing with the same face database. From this experiment, template matching has generated bad detections when dealing with certain head orientation angles and facial expressions. It is found not effective in refining the mouth location. Projection method has failed in most of the images with thick beard, moustache and 'mouthopened' conditions. Both of the methods will be tested again with a bigger database to perform under external lighting from various directions.
The failure cases from the proposed method are mostly caused by 'mouth-opened' but there are also successful detections in this condition. The examples of successful and failed mouth detections are shown in Fig. 2 and 3. The image processing steps carried out in the proposed method to eliminate beard, moustache and unwanted region work effectively. The processing time for mouth detection is 0.46 sec image −1 while 2.04 sec for iris detection as shown. The simplicity of the mouth detection's algorithm has an advantage of processing speed over algorithm for iris detection which is much more complicated. There are successful iris detections from incomplete detected face region and successful mouth detection from failure iris detection even the geometric information of these irises is being used to estimate the mouth region.
From Table 2, the combination of eye and mouth templates which is used for face recognition has recorded a success rate of 86.11%. Vividly, eye template contains more useful information than mouth template with 79.17% successful face recognition. However, mouth template can be used as additional information to further enhance the success rate of face recognition due to its low processing time, 0.10 sec. A total of 0.36 sec has clocked by using eye and mouth templates for face recognition in this experiment. The success rate of face recognition still can be improved by adding more images per person rather than three. In order to have a better recognition rate, the mouth corners need to be detected precisely as it is very useful in calculating correlation values for template matching method.

CONCLUSION
Referring to previous researches, there are lack of mouth detection results specifically dealing with thick beard, moustache and open mouth. The aim of this proposed algorithm is to effectively detect the facial features from different faces with head tilts, poses, facial expressions, beard and moustache. Face recognition using template matching can still be improved by obtaining more precise location of mouth corners as discussed. In future, this proposed algorithm will be tested with a larger face database, additional images with external lighting from different directions and more facial expressions. Artificial neural networks will be implemented for face recognition as a comparison to template matching technique and to improve the recognition rate.