Eye Detection Using Composite Cross-Correlation

This study presents a new eye detection method depending on composite template matching for facial images. The objective of this study is to utilize template match method to detect the eyes from given images and to improve this method to obtain higher rate of detection. The idea of our method is to integrate cross correlations of various eye templates. Thus, the correct values of single template matching based eye detection dominated the final output. It also contributed to the re-correct the detection in the event of failure of all single templates. The study also presents a method to obtain candidate eye pixels which contribute to abbreviate the time required to implement up to 91%. The formula of composite cross correlation has been generalized taking into account the differences between the sizes, shifts and irregular single templates. The experiments applied on PICS database reported 98.76% as eye detection rate.


INTRODUCTION
Human facial features play a significant role for face recognition. According to studies it is determined that eyes, mouth and nose are amongst the most important features for recognition. Furthermore, the extraction of facial feature points, (eyes, nose and mouth) plays an important role in face detection, model based image coding, expression recognition, facial animation and head pose determination. Different conditions different brightness, shadows, clearness can be obstacles in the process of facial feature extraction. In addition to small variations of face size and orientation can be adversely affected the result. Facial features may be partially covered by glasses, hat and hairs. Thus, facial feature extraction methods are sensitive to various non-idealities such as variations illumination, noise, orientation, time-consuming and color space used (Bagherian et al., 2009). Also continuous feature extraction will increase performance of face recognition system.
Facial feature extraction based on Geometry utilizes relative positions and sizes of the important components of face. It concentrates in detecting edges, directions or regions of important components in the face image to build feature vectors according to this information. Many filters such as Canny filter and gradient filter can be applied to detect feature region of face image. Adaboost has been utilized to redistribute the grayscales into the feature. In LBP method, face image has been divided up to regions or blocks and combined histogram of these regions to feature vector of the image. Gabor wavelets transform is one of feasible method in feature extraction which provide analysis of the input image in both spatial and frequency domains simultaneously (Lim et al., 2009).

AJAS
The advantage of facial feature extraction based on Geometry is concentration on important components of face such as eyes, nose, mouth. but the disadvantage is not to represent face global structure and face texture.
Facial Feature extraction based on color segmentation utilizes skin color as referential color to debate other nonskin color region. In the fact, non-skin color region in the face image is considered as potential eyes or mouth region (Do and Le, 2009). Various color models have been used to extract facial feature such as RGB, HSV or YCbC (Liu and Peng, 2010). After determining skin color and facial feature region, a threshold is applied to get accurate determination (Neagoe and Neghina, 2010). By detecting the two eyes and the mouth, it is easy to get the coordinates of the potential facial features region (Rajpathaka et al., 2009). It should be noted that diversity ethnical backgrounds is rather lamination in this approach.
Facial feature extraction based on appearance seeks to represent the face image by vector by using linear transformation and statistical methods such as PCA and ICA. However, the important information in the face image is contained in the high-order relationship among the image. While PCA uses second-order, ICA debates the high-order to enhance the result (Tirkaz and Albayrak, 2009). By this way, important information of face image is kept while reflect face global structure and redundant information is rejected.
Facial Feature Extraction Based on Template is used to detect the feature which the most related to template from face image. All facial features can be fit into templates such as eye template, nose template and mouth template. After normalizing the template and face image, the template is compared to all regions of the face image. By calculating the Mean Squared Error (MSE) of correlation between the template and different regions, the region with the lowest MES represents the desired feature (Bhoi and Mohanty, 2010). Facial Feature Extraction Based on Template does not require complex mathematical calculation and prior knowledge about the features geometry but it represent global face structure.

RELATED WORK
Template match based approaches is widely used in eye detection. These approaches based on comparison an eye template with all parts of given face image. Dramatically, comparison is calculated by the total differences, Euclidean dimension or crosscorrelation techniques. Bhoi and Mohanti employed cross correlation technique in template match based eye detection (in both cases open eye and close eye) by simple method does not require any complex mathematical calculation and prior knowledge about the eye (Bhoi and Mohanty, 2010). While in the earlier study used template match based approach in order to determine the fatigue drivers but by tracking video sequences and comparing changes on the distance of the eyelids during driving (Dong and Wu, 2005).
Since template match based methods are searching in the entire give face image, it is useful to use the initial methods or hybridizing template match based methods with other methods to predict the rough area of the eyes. By this way, we accelerate the implementation time to perform this method and avoid unexpected results. A robust algorithm has hybridized template match based method with feature based method to improve its effectiveness (Peng et al., 2005). Firstly, gradient image has been obtained form face image. Then, by applying horizontal and vertical projection, the two rough areas of eyes have been determined. Finally, template match based method is simply used to detect eyes. The experiment applied on ORL database demonstrated detection accuracy is 95.2%. In the study of component-based face detection, facial area has detected depending on the skin color (Lee, 2008) and this is what obviates searching in the non facial parts in the images. The abstract eye template has been proposed to use component-based method in two phrases. Firstly, to segmenting face area; Secondly, to segmenting eyes area (Guo et al., 2010). Rectangle of this template is divided into three horizontal parts, namely respectively upper eyelid, eye core and lower eyelid. each parts is processed separately and eye region is detected when the eye core part gives similarity ratio of not less than 90% and eyelids parts gives similarity ratio of not less than 60%. However, the experiments applied on CalTech Color Face Database showed 94.68% of detection accuracy.
The behavior of creation templates plays a crucial role in the performance of template match based approaches. For example, the various sizes between eye templates and the eyes in the given face images produce unsatisfactory results. Therefore, a method has been proposed to re-sizing eye template according to the expected size of the eye in the given facial image .
Resizing based templates are known as a multi-scale Science Publications AJAS template and it is effectively used to determine the position of the eye in the study aimed to analyze the upper facial expressions (Florea et al., 2011).
It is also useful to curtailing the templates of symmetric shapes. This is what has been achieved in the study suggested employing template based method to detect the human face (Chen et al., 2009). Depending on the symmetry of human face, template has been formed from only half face. This way accelerate the implementation of the method and gave the same results for the frontal face images. It is worth mentioning the half face based templates gave better results in the face images with angle comparing to the fall face based templates. In the face images with 30°, the detection accuracy increased from 22.5 to 95% while in the face images with 45°, the detection accuracy increased from 5 to 52.5%.
One of the ideas that have been realized in creation a template of the eye is creating eye template of only the pupil. This template has been proposed in a study meant to detect the pupil in vehicle operations (Kuronumam and Suzuki, 2011) and achieved better result than previous studies in same field (Kuronuma et al., 2010) depends on Otsu's method and the reason has been concluded is to the tolerance of template match methods and to ease of preparation template due to the regular circular shape of pupil.
Furthermore, a study has been proposed, combining template iris and a template that contains both eyes in order to detect more precisely the eyes (Yuen et al., 2009). Where this study proposed the candidate irises and assigns a cost for each pair of irises. Depending on the cost of irises pair in addition to the cost of cross correlation between the candidate irises and the template of two eyes, the correct position of iris has been detected. The results showed that 94.7% as detection accuracy for the experiments applied on the database of University of Bern and 96.8% as detection accuracy for the experiments applied the AR database.

PROPOSED METHOD
We describe our proposed method in four steps. First step is how to find candidate eye pixels from face images. Second step describes eye template based detection and cross correlation of temples. In third step we integrate single eye templates in one composite eye template. Finally, we generalize the composite eye to take into account shifts and irregular eye templates.

Candidate Eye Pixels
Applying eye template match based method on the entire image is computationally expensive and takes long time to implement .Therefore it is useful to apply a method to propose areas that are likely includes eye and ignore the rest areas of the face image during applying the eye template.
In general, facial image composed of pixels belong to the human face area and the pixels belong to the background area. To determine these areas, we study the color of image represented by YCbCr instead of RGB.
For given RGB face image I with the size of n × m, we find TCbCr Image I' with same size according to (1) As shown in Fig. 1, the background area in I" shown in black color {0}, skin area shown in white color {1} and there are black areas surrounded by the skin area and these what we look for, due to these indicate to facial features, including eyes. In general, background area surrounds facial skin area. This is what distinguishes between background area and the facial feature areas. To detect the background we pass lines perpendicular beginning from the border of face image and ending to first pixel belongs to the facial skin area. We applied this step for all border's pixels (Upper, lower, left and right). We assign a value of {0.5} to each pixel belongs to these lines. In (3), we formulated this step for the upper pixels in the border. In Fig. 1 Science Publications AJAS Fig. 1. The steps to detect the candidate eye areas Then, we form I"" which merge the background area with the facial skin area. According to mathematical morphology we apply dilation and erosion on features areas in I"" to detect the candidate eye areas which labeled in black color in I""'

Cross Correlation Based Matching
In order to find the eye in given face image I with a size of n×m .We first offer eye template T of size of q × p and the center (cx, cy). Then, we find the correlation matrix Co which represents the values of the cross correlation for all parts of the face image with this template. The element that contains the maximum value in the correlation matrix refers to the center of the eye in the face image I.
The cross correlation coefficient is defined as Equation (4)

∑∑
Values of matrix elements resulting from the (4) belongs to the range of [-1, 1]. We neglected mount values of some elements, which form a border with the size 0.5q × 0.5p surrounded the face image. Therefore, it is preferable to assign the value of zero for the border as shown in Equation (5):

Composite Cross Correlation
The use of cross correlation in eye template based detection method, gives high values for pixels in eye area in spite of wrong detection in some cases. Some researchers have suggested the use of more than an eye taken in different conditions and then adopt the template that gave the highest value. The problem lies in if all the templates do not detect the eye or the cross correlation value of wrong detection in a template was more than the value of correct detection of other templates. In our proposed method, we invested high values for eye positions in spite of wrong detection in some cases by multiply correlation matrices for several templates with each other. In this way, the correct values of eye detection dominated the final output. It also contributed to the re-correct the detection in the event of failure of all templates, as shown in the result section.
In our proposed method, we use three templates of right eye, a template for eye and surrounded skin area T 1 , a template for cropped eye T 2 and a template for cropped Iris T 3 . We calculate correlation matrices for a give image according (4 and5a) Then we normalize the values in correlation matrices to take the highest possible values according to Equation (6) We reformulate single template based cross correlation in (4, 5 and 6) to generalize it on multi templates (k templates) based cross correlation as shown in Equation (7): Where: [ ] 1 2 k T T ,T , ,T = K Figure 2 shows the cross correlation matrices for single templates T 1 ,T 2 and T 3 and composite templates [T 1 ,T 2 ], [T 1 , T 2 ], [ T 1 ,T 3 ] and [T 1 ,T 2 ,T 3 ]. All these templates concentrate on the left eye. The maximum value of the correlation matrix elements is represented by brightest pixel. We note that the greater the number of templates used, it reduced the brightness of the image while increasing the brightness of the pixel representing center of iris and therefore gives a more accurate result.

Shifts and Irrgular Templates
In fact, experience has shown effective eye detection using cross correlation of frontal eye templates. The problem is when the eye template taken from frontal face image while the given face image with angle. In this case, the Effectiveness of cross correlation descends and may have a negative impact on the eye detection. Some proposed methods suggest taking the templates at different angles but this is what adversely affects on the results of frontal face images. In other studies, taking half face template increases the accuracy of face detection. This is what we apply in our algorithms proposed on the eye taking into account the conservation good eye detection of front face image using another full eye template.
The value of the correlation matrix element Co' (I[T h ]) (x,y) is expresses the value of cross correlation between the template T h and a part of the given face image i with the same template dimensions q h ×p h . Since the center of this part of the face image is (x,y).
In previous cases, the center templates are the center of the iris. In templates that contain half a part of the eye, there is a shift between the center of the template and the center of iris. This may not been concern in the case of single eye templates but in composite eye templates we should take it into consideration because the center of all the templates must be the center of the eye. The center of the iris must been supplied during applying the eye templates. We In general, eye templates take a rectangle shape. To use non-rectangular templates, we use an abstract eye template with the same size of eye template. Abstract eye template contains elements of the value of {1} for the parts to be preserved and a value of {0} for the parts to be ignored. Then we multiply elements in the abstract eye template. Furthermore, the use of these templates abstract is additional proposal to treating the shifts in the templates. For example, instead of taking a half eye template, we take a template for a full eye and take an abstract template assigning the value of 1 for the half templates and the values of {0} for the another half. It is worth noting that the actual size of the template will change. Depending on (4 and 8), we deduce (9). Which takes abstract eye templates into account given in I(x s, y t); q p 1 T I(cx s,cy t) q p

EXPERIMENTAL RESULTS
We use the images in the aberdeen set of Psychological Image Collection at Stirling (PICS) database (PICS), a well-known free database of faces, to implement our experiments. In this database, there are completely 687 color face images from Ian Craw at Aberdeen, between 1 and 18 images of 90 individuals. Some variations in lighting, 8 have varied viewpoint and the resolution is varied from 336×480 to 624×544.
According to the experiments, using our method on (PICS) database to find candidate eye pixies, suggests average of candidate aye pixels up to 9% of face image. In other word, we abbreviated 91% of the cross-correlation processes should be implemented. Table 1. The experimental result for applying cross correlation based matching using single and composite eye template Detection rate without using Detection rate with using candidate eye pixels (%) candidate eye pixels (%)  (Peng et al., 2005) ORL 95.20 96.75 (Guo et al., 2010) Caltech 94.68 98.66 (Yuen et al., 2009) AR 96.80 97.62 In general, the number of candid eye is small in the face images with good lighting, while it increases in the in the face images with dark lighting because of the inability to separate skin color and facial features pixels. The experimental results applied on (PICS) database using tripartite right eye templates [T 1 ,T 2 ,…T 3 ] is shown in Table 1. However Fig. 2 shows the templates of T 1 ,T 2 and T 3 . These experiments aimed at finding the left eye in the face images. All face images in the database containing the left eye in pose form 0 to 90° (front face image to profile face image). Some erroneous results were a detection of the right eyes instead the left eye. We separate these results in order to determine the attitude of each template. For use of single templates, we find that the best results obtained by the template with a larger size, which represents the eye with the surrounding facial area where it can characterizing eye as much as possible from the template data. While the worst results obtained by the template with the smaller size, which represents iris due to slight contrast in this template as well as it to detect the right eye instead the left eye up to 18 % due to the similarity of the iris of both eyes.

AJAS
The experiments also showed that the use of a composite of many single templates inevitably produces better results than individually use of these single templates. Also, the most single templates differentiated from each other, generates better results composite templates. Therefore, the results of the composite template [T 1 , T 3 ] is better than the results of the composite template [T 1 ,T 2 ] and [T 1 ,T 3 ] because of the difference between T 1 and T 3 is greater than the difference between T 1 and T 2 and difference between T 2 and T 3 .
Some erroneous eye detection results pointed to wrong pixels in the background outside the facial area. So the ignorance the background by employing candid eye pixels has contributed to increase the rate of eye detection. Finally, use the composite template [T 1 ,T 2 ,T 3 ] to detect left regarding the candidate eye position gave a reasonable detection rate reached 98.76%.
The first three experiments in Table 1 which indicate to using single templates without using candidates eye pixels are the same experiments since we apply (Bhoi and Mohanty, 2010) on (PICS) database. In Table 2, we compare between the result of our proposed method and the results of previous methods by using same database after ignorance the face image which did not include explicit left eyes.

CONCLUSION
Template match method is widely used in order to detection the human face or the facial features from given image. The objective of this study is to utilize template match method in order to detect the eyes from given images and improve this method in order to obtain higher rate of detection. However previous generally, applying eye template match method on face image somehow indicates to the eye even through the decision of detection was wrong. In previous studies, multitemplate is proposed and the detection decision is deduced from the strongest cross correlation of them. The problem lies in if all the templates do not detect the eye or the cross correlation value of wrong Science Publications AJAS detection in a template was more than the value of correct detection of other templates.
In our proposed method, we proposed composite template invested high values for eye positions in spite of wrong detection in some cases by multiply correlation matrices for several single templates with each other. In this way, the correct values of eye detection dominated the final output. It also contributed to the re-correct the detection in the event of failure of all templates. The process of using composite template requires longer time than using single template. Therefore, we proposed a method to present candidate eye pixels and ignore the back ground and skin area from face images. By this method we reduce the time required to implement up to 91%.
This study also discussed the use of a vertical half of the eye template in order to better results with face image with angle, taking into account the shift between iris center and eye template center. This shift is necessary to parameterize in order to unify whole centers of the single templates in composite template. Furthermore, we presented how to treat irregular shapes of templates in composite template. Finally, we proposed a generalized formula for multi use of composite templates.
The experiments have applied on the Aberdeen set of Psychological Image Collection at Stirling (PICS) database. The composite eye template is compound of 3 single eye templates in deferent sizes. All these templates represent left eye. The results showed that the use of composite templates significantly contributed to raise the rate of eye detection up to 98.76 while it reported as 90.62% using single template. In other word, composite of many single templates inevitably produces better results than individually use of these single templates. Also, the most single templates differentiated from each other, generates better results composite templates.
However, the wrong detection caused due to the bright reflection of spectacle, the dim lighting on the face and the closed eyes.