Detection of Occluded Face Image using Mean Based Weight Matrix and Support Vector Machine

: Problem statement: Face occlusion is a very challenging problem in face recognition. The performance of face recognition system decreases drastically due to the presence of partial occlusion on the face. Extracting discriminative features to achieve accurate detection versus computational overhead in extracting the features, which affects the classification speed, would be a sustained problem. The objective of this study is to segment the human face into non-occluded and occluded part of the occluded human face image. In General, for face detection special facial features are extracted. In the proposed study a simplified algorithm to extract the features is developed. Approach: An algorithm which enables the automatic detection of the presence of occlusions on the face would be a useful tool to increase the performances of the system. The face image was preprocessed to enhance the input face images in order to reduce the loss of classification performance due to changes in facial appearance. The experiment also balances both illumination and facial expression changes. Results: In this study, a Mean Based Weight Matrix (MBWM) algorithm has been proposed to enhance the performance by 4.25% than the LBP method. Conclusion: The proposed model has been tested on occluded face images with a dataset obtained from the MIT face database.


INTRODUCTION
The most important goal of computer vision today is to achieve visual recognition ability akin to that of human. The face recognition problem has been researched intensively for the past few decades, due to its great potential in various practical applications such as Human Computer Interface (HCI), intelligent robot, surveillance and so on. Face Recognition can benefit the areas of Law Enforcement, Airport Security, Access Control, Driving Licenses Passports, Defense, Customs, Immigration and Scene analysis.
This has resulted in the development of successful algorithms and the introduction of commercial products. But, face recognition is still in the research and development phase, several commercial systems are currently available and research organizations are working on the development of more accurate and reliable systems. Using the present technology it is impossible to completely model human recognition system and reach its performance and accuracy. However, the human brain has its shortcomings in some aspects. The benefit of a computer system would be its capacity to handle large amount of data and ability to do a job in a predefined and repeated manner. The observations and findings about human face recognition system will be a good starting point for automatic face attribute analysis.
When face recognition is diversified further, obviously problem of occlusion by other objects or apparels such as sunglasses, carves, mask becomes eminent. Thus a robust algorithm for occluded faces is required for real applications Fig. 1. Show the outline of the occlusion detection process.

Related works:
The conventional holistic approaches, such as PCA, LDA and ICA, are not robust to partial occlusions. Also, local feature based methods are less sensitive to occlusion detection. A number of local feature-based and component-based methods were proposed for dealing with the occlusion problem. Shang et al. (2010) proposed an occlusion detection model in which the given face image is accomplished for each local patch independently by employing pattern classification framework. The high dimensional data set of the local patch is then reduced using PCA.  Lee et al. (2009) proposed a probabilistic approach (AMM) which can compensate for partially occluded faces. Kanan et al. (2009) extended Martinez's work by using the Self-Organizing Map (SOM) to learn the subspace instead of using the mixture of Gaussians. (Ali et al., 2011) proposed Locally Salient ICA (LS-ICA) method. This method employs local salient information to create part based local basis images and imposes additional localization constraint in the process of computing discriminating features for the images. He et al. (2010) presented a method which combines the reconstructive and discriminative models. A basis is constructed which contains the complete discriminative information. Nakamura et al. (2009) proposed to use a line feature based face Attributed Relational Graph (ARG) model to encode all the geometric information and local features of a face. Zhang et al. (2007) proposed to use Kullback-Leibler Divergence (KLD) to estimate the probability of occlusion in the feature space and improves the standard Local Gabor Binary Patterns (LGBP) for partially occluded face. Zhang et al. (2005) proposed the gabor filter to enhance the facial features and then extract the LGBPH sequence which improves much the LBP's roboutness.. Recently, (Wright et al., 2009) applied sparse signal representation for faces to deal with the corruption and occlusion problem. Jia and Martinez (2009) proposed to use Partial Support Vector Machines (PSVM) in places where occlusions may occur.

Occlusion detection:
The occlusion detection is based on Mean Based Weight Matrix (MBWM) algorithm. In order to get the discriminative features for face detection process, face image is segmented into different regions in a face image so that the features of only the non-occluded region which alone can show variation will be used for face detection which would improve the efficiency of the face detection system. Here,an occlusion detection algorithm based on subspace division is proposed.
Subdivision of a face image: Partial occlusions in face images usually occurs when the subjects wear adornments like sunglass or scarf, or when faces are covered by other objects such as hands, cup, mask and so on. In order to detect the locally occluded regions in a face image, the face image is subdivided into number of facial components. The number and the shape of the components depend on the nature of the occlusions. The face image is subdivided into 2, 4 and 6 segments. The discriminative features for each segment are then calculated. This study focuses on the occlusion detection on the face covered by medical mask. When the face image is subdivided into two equally halves as shown in Fig. 2 the lower Segment is used for detecting mask. Once the face is divided discriminating features are extracted from both the occluded and non-occluded part of the face image. These features are then fed to an SVM classifier for determining whether an occlusion is present or not.
Occlusion detection using LBP: Occlusion detection of a given face image is accomplished for each local patch independently by employing pattern classification framework.
Local Binary Pattern (LBPs) which was first proposed by (Ahonen et al., 2006) was originally designed for texture description. Still, it has shown very good performance in many other tasks and one of the most important application areas is facial image description. The basic LBP operator labels the pixels of an image (I p ) by thresholding each 3×3 pixel neighborhood of the input image with the center pixel value (I c ), multiplying the threshold values by weight (powers of two) and summing them. The operation of the basic LBP operator is illustrated in Fig. 3.
Thresholding is done using the centre pixel as in Eq. 1: Weights are then assigned and the LBP values are obtained using Eq. 2 and the values are summed to obtain the LBP values for the 3×3 matrix. The LBP feature thus obtained are considered for classification purpose: LBPs have been very effective for image representation as it is being applied to visual inspection, motion detection and outdoor scene analysis. The most important properties of LBP features are their tolerance against monotonic illumination changes and their computational simplicity. The LBP operator mainly detects many texture primitives as spot, line end, edge and corner typically accumulated into a histogram over a region to capture local texture information.
Occlusion detection using SLBM: The simplified local binary mean as proposed by (Priya et al., 2012) involves three steps which include subdividing, thresholding and weighing. First, a 3×3 sub image is cropped .The pixel values are represented as I p .
Thresholding is done using the mean of the 9 elements of the 3×3 sub image (I m ). Thresholding is done based on the rule given in Eq. 3: Weights are then assigned and summed to obtain the SLMB values for the 3×3 matrix using Eq. 4. The SLMB feature thus obtained are considered for classification purpose: The SLMB features are thus calculated. Many images of different types can have similar histograms, because, histograms provide only a coarse characterization of an image. This is the main disadvantage of using histograms. So, the statistical features such as mean and standard deviation of the SLMB features are calculated.
Occlusion detection using MBWM: In SLBM, thresholding is exactly at the value of the central pixel ic. This makes it to be sensitive to noise, especially in near-uniform image regions. Many facial regions are relatively uniform; it is potentially useful to improve the robustness of the underlying descriptors in these areas. So, the SLBM is extended to Mean Based Weight Matrix (MBWM) The mean based weight matrix involves three steps which include subdividing, thresholding and weighing. The 3×3 pixels of the image are replaced by a 3-valued functionas given in Eq. 5: The MBWM features are thus calculated. The first order statistical features such as mean and standard deviation of the MBWM features are calculated. These features are used for classification. Support Vector Machine (SVM) classifier: As a powerful machine learning technique for data classification, SVM performs an implicit mapping of data into a higher (maybe infinite) dimensional feature space and then finds a linear separating hyperplane with the maximal margin to separate data in this higher dimensional space (Chang and Lin, 2012). Given a training set of labeled examples {(x i , y i ), i = 1, 2 ……l} where x i Є R n and y i Є {-1, 1} a new test example x is classified by the function as in Eq. 7: where, α i is the Lagrange multiplier of a dual optimization problem that describes the separating hyperplane K (x i , x) is a kernel function and b is the threshold parameter of the hyperplane. The training sample x i with α i > 0 is called support vectors and SVM finds the hyperplane that maximizes the distance between the support vectors and the hyperplane. Given a non-linear mapping Φ that embeds the input data into the high dimensional space, kernels have the form of K (x i , x j ) = (Φ (x i ), Φ (x j ). SVM allows domain-specific selection of the kernel function. Though new kernels are being proposed, the most frequently used kernel functions are the linear, polynomial and Radial Basis Function (RBF) kernels. SVM makes binary decisions. With regard to the parameter selection of SVM, the mean and standard deviation are chosen. These parameters provided the best accuracy. The generalization performances achieved using the two different kernels is discussed.

RESULTS
The performance comparisons of the classifier: The performances of the occlusion detection schemes LBP, SLBM and MBWM are quantitatively evaluated. The LBP, SLBM, MBWM features are extracted for all the patches. The SVM classifier is used to detect the occlusion in the face region .Each training normal face image is divided into 2, 4 or 6 disjoint patches as shown in Fig. 4 and 5 represents the examples of synthetic occluded faces. However, using the occluded data labeled as the normal data in the recognition phase lowers the performance seriously since it delivers disturbed information. The features of the training images (unoccluded) are shown in Table 1. The features of the training image (synthetic occluded using medical mask) are in Table 2. Now, these two classes are tested on test images with synthetic occlusion pattern as shown in Fig. 5. Here the occluded patches are examined and compared to the trained database.
Subdivision of face image: Proposed partial occlusion detection and face recognition algorithm are developed based on local patches of a face image. Thus, different division methods may result in different performances on both occlusion detection and recognition. In this study, different subdivision method (2, 4, 6 segments) of face image is examined. The Support vector machine classifier is used for the comparison of the occlusion detection performances. 440 occlusion-free training face images in the MIT face database are examined. The features of the occluded regions are detected using LBP, SLBM and MBWM algorithms described.    Experiments on synthetic occlusion: A face images occluded by medical mask in MIT face database is used for testing the algorithm. First, the LBP based recognition algorithm is tested on synthetically occluded images as shown in Fig. 5. Occlusion-free images as in Fig. 4 were used for training. Similarly, the proposed SLBM and MBWM features are extracted from the face images. Comparisons of the performance of all the three algorithms are presented.
The recognition rate, defined as the percentage of correctly recognized faces, is used as the performance measure. The features of the occluded and nonoccluded face image subdivided into two equal halves with the upper segment focusing on the eyes and the lower segment focusing on the lips for the nonoccluded image and the medical mask for the occluded image for the MBWM algorithm is in Table 1 and 2. Figure 6 and 7 shows the classified output using linear and RBF SVM classifier using the first order statistical features, mean and standard deviation. The sign indicates test image classified as occluded face image and * sign indicates test image classified as nonoccluded face image. The + sign indicates the occluded training face image and the * sign indicates the nonoccluded training face image. The '○' indicates the support vectors selected for the hyperplane. Table 3-6 show the recognition rate for 2, 4 and 6 segments of training image. Figure 8 and 9 show the accuracy of classification using linear SVM as classifier.The classification result with RBF SVM as classifier is shown in Fig. 10 and 11. Experimental results show that the proposed algorithm achieved the highest detection rate with 2 segments. The performance decreased drastically as the size of the occluded region became larger.

DISCUSSION
All 40 persons (with 11 distinct poses) in the MIT face database were used. All the images have a bright homogeneous background and the subjects are in an upright, frontal position. For each individual, looking front, looking left, looking right, looking up and looking up towards left and looking up towards right, looking down are included. In addition to this variation in pose and images with four emotionsneutral, smile, laughter, sad/disgust are also included for every individual. Among these, all 440 normal face images and 60 occluded face images (medical mask) were used for the training the occluded and the non-occluded class. The 100 synthetic masked images were used for probes and all the normal frontal faces were used for the gallery. Table 7 shows the classification rate when various training sets are used. From this it is evident that as the number of training images increase the recognition rate decreases which is represented in Fig. 12.

CONCLUSION
In this study, occlusion problem, which has been researched relatively less than illumination and pose problems in face recognition, is discussed. SLBM and MBWM are the two algorithms proposed for partial occlusion detection. Locally occluded areas in faces are detected using SVM classifier and hence classified as occluded and non-occluded face images. Experimental results demonstrated that the proposed algorithm could reliably recognize partially occluded faces with higher recognition rate than the existing methods.
Future scope: This study focuses on the occlusion detection problem. This can be extended to recovery of face image from the occluded face. The same feature extraction methods may also be applied for classification of different expressions as happy, sad, angry using multiclass SVM. The same algorithm may also be extended to pose estimation. In the field of security, this study can be further enhanced by extending the same. Hardware implementation of the testing process may also be carried out considering the cost constraint.