Facial Expressions Recognition Using Eigenspaces

: A challenging research topic is to make the Computer Systems to recognize facial expressions from the face image. A method of facial expression recognition, based on Eigenspaces is presented in this study. Here, the authors recognize the user’s facial expressions from the input images, using a method that was customized from eigenface recognition. Evaluation was done for this method in terms of identification correctness using two different Facial Expressions databases, Cohn-Kanade facial expression database and Japanese Female Facial Expression database. The results show the effectiveness of proposed method.


INTRODUCTION
The human face gives one of the influential, flexible and usual ways of communication, during communal interaction. As per (Mehrabian, 1968), Facial Expressions gives significant communicative signals, which represent more than 50% of the outcome of a conversed message; so identification of facial expressions turn out to be a most important modality in human computer interaction. For example, in a Human-Computer Interface, if the system can sense and recognize the users' objectives from their facial expressions, it is possible for the computer to help them by providing implications and schemes according to sensed circumstance.
The possibility of making Computers to recognize facial expressions and use the information in Human Computer Interaction has increased significant research interest over the last few years. This has given increase to a number of automatic methods to recognize facial expressions in images or video (Zhi and Ruan, 2008;Pantic and Rothkrantz, 2000). In (Murthy and Jadon, 2007), the authors have suggested the usage of Eigenspaces in recognition of facial expressions. In this study, the authors have extended the set of Facial expressions from three expressions to six expressions ( Fig. 1) and established the effectiveness of the algorithm on two different databases, Cohn-Kanade database and JAFFE database. The famous Eigenface identification technique (Turk and Pentland, 1991) is customized and used for recognizing facial expressions. The goal of the authors is to use dimensionality methods on a huge and diversed dataset to study a sensible space for expressions and identify the facial expressions of the user. The innovation of the proposed approach is its performance in categorization, strength and need of preprocessing of images.
Facial expressions: Mehrabian (1968) indicated that only 7% of message is because of languages, 38% is through paralanguage and more than 50% of message is communicated by facial expressions. This shows that the facial expression is a most important modality in human communication. Thus the authors can imagine that, when designing the human computer interfaces, the facial expressions seems to be a most important factor for improving the communicability of message, even in human-machine communication. In recent years, a large amount of research has been done on recognition of facial expressions (Lin, 2006;Pantic and Patras, 2006). Cross cultural psychological research on facial expressions specifies that there may be a small set of facial expressions that are universal. This was first suggested by Darwin in his revolutionary work on "The origin of species". Psychologists Ekman and Friesen (1978;Dalgleish and Power, 2000) conducted the first methodologically sound studies and concluded that the emotions "Happiness, Anger, Sadness, Disgust, Surprise and Fear" are shown and interpreted in all human cultures in the same way. Figure 1 shows the universal expressions.

Related works:
In recent years, the research of developing automatic facial expression recognition systems has attracted a lot of attention from many different fields. A more recent and complete overview can be found in (Pantic and Rothkrantz, 2000;Fasel and Luettin, 2003). The approaches to facial expression recognition can be, roughly divided into two classes: geometrical feature-based approaches and appearance-based approaches (Tian et al., 2002). The geometrical feature-based approaches rely on the geometric facial features, which represent the shapes and locations of facial components such as eyebrows, eyes, nose, mouth. As for the appearance-based approaches, the whole-face or specific regions in a face image are used for the feature extraction via optical flow or some kinds of filters.
Many of the previous works (Pantic and Rothkrantz, 2004;Kapoor et al., 2003) on facial expression recognition are based on the existence of six universal expressions (anger, disgust, fear, joy, sorrow and surprise). These universal expressions are based on the psychological results of Dalgleish and Power (2000) and the Facial Action Coding System (FACS), developed by Ekman and Friesen (1978), which codes expressions as a combination of 46 facial movements called Action Units. The combination of these action units result in a large set of possible facial expressions. For example, smile expression is considered to be a combination of "pulling lip corners (AU12+13) and/or mouth opening (AU25+27) with upper lip raiser (AU10) and bit of furrow deepening (AU11)." However, this is only one type of a smile; there are many variations of the above motions, each having a different intensity of actuation. Despite its limitations, this method is the most widely used method for measuring human facial motion for both human and machine perception.
Conventional methods extract features of facial organs, such as eyes and a mouth, in gray or color images of front faces and recognize the expressions from changes in their shapes or their geometrical relationships by different facial expressions (Ioannou et al., 2007;Feng et al., 2006;Terzopoulos and Waters, 1993). However, estimation of their precise positions and shape attributes in real images is difficult, because of the wide variety of the face features, skin color/brightness, illumination conditions and geometrical variations such as head orientations. As a result, many of the systems need human assistance such as attaching marks on the subject's face or specifying windows covering each organ in the image. Neural networks seem promising for recognizing facial expressions (Kobayashi and Hara, 1991), but the methods using Neural Networks assumes locations of facial organs are to be provided as its input. Another idea is to estimate the movement of muscles from optical flow to recognize facial expression (Fleet et al., 2000;Yacoob and Davis, 1994). Its success depends on the reliability of optical flow estimation from image sequences and its accurate estimation seems difficult because of the complexity of facial images. Moreover, the method should compensate the flow vectors for the head movements of which estimation is not easy. The above systems work under many restricted conditions. One of the conditions is, the neutral expression of the subject is given first as control, the human operator specifies the windows covering the organs of the input face, or the marks are attached on the subject's face.
The success of facial expressions recognition system depends heavily on how well the movement of the key features points, like eyeballs and mouth corner, are tracked on the human face. To facilitate the tracking process, in the current practice the face of the human performer is painted with color make-up or attached with some small illuminative balls (Lo and Churig, 2001). However, the artificial attachment is intrusive to humans, possibly jeopardizing the quality of the human act. To eliminate such restrictions and to take the advantage of similarity measure between face and facial expression, in (Murthy and Jadon, 2007) the authors have proposed a method to recognize facial expressions from the whole face, rather than from changes in the shape of the facial organs such as eyes and a mouth, or their geometrical relationships. In other words, the expressions can be recognized without extracting the individual facial features. The idea is similar as the face identification method proposed by Turk and Pentland (1991) and method used in Frank and Noth (2003), but the characteristics of the problem domain are quite different. Our system was designed to recognize the expression of an unknown subject from a single front view of his/her face.

MATERIALS AND METHODS
Proposed method: In the proposed method, facial expressions of the human face are identified from the input image using Eigenspaces method. The method is modified from the famous Eigenface identification technique.
To illustrate the feasibility of using Eigenspace for facial expression recognition, the PCA reconstruction method is modified and the modified PCA reconstruction method is shown in Fig. 2. If the input image is similar to some expression training set, the reconstructed image will have less distortion than the image reconstructed from other eigenvectors of training expressions. Based on this idea, the authors divided the training set into six classes according to universal expressions as shown in Fig. 1 and computed the Eigenspaces of each class. For a test face image, they first project it onto the Eigenspace of each class independently and then derive reconstructed image from each Eigenspace.
By measuring the similarity between input image and the reconstructed image of each class, they can identify the class of input image whose reconstructed image is most similar to the input image.
Eigenspaces for facial expression recognition: When using Eigenspaces for facial expression recognition of unidentified faces, one option is to compute one Eigenspaces for each facial expression from a taged database of different persons. The categorization process corresponds to that of face recognition: Project a new image to each Eigenspaces and select the Eigenspaces, which best depicts the input image. This is done by computing the Mean Square Error. However, the problem with facial expression categorization is that the person is unrecognized. Each person uses a different smile. Nonetheless, each smile of each person be supposed to be classified as smile.
In order to deal with this reality, the authors have modified the Eigenface recognition method so that a separate subspace is created for each facial expression of the human, instead of having a single subspace for all expressions as in the original method ie., all six facial expressions will have their individual expression space as a subspace of the image space as shown in Fig.  2. With the expression subspaces available, the authors could then continue for recognition of expression in any given image. Similar to the images for getting the expression subspaces, the new image is first twisted to the corresponding column vector. The authors then consider the expression subspaces one at a time and compute the distances between the new image vector and the subspaces. Whichever expression space having the shortest distance to the input image, the corresponding expression will be designated as the facial expression contained in the input image.

RESULTS AND DISCUSSION
The authors have tested this method using the Cohn-Kanade facial expression database (Kanade et al., 2000). The database consists of more no. of university students with six expressions (Happiness, Sad, Surprise, Fear, Anger and Disgust). From the database, the authors have collected 120 images for training. They have used one expression of one student for training. For testing, they considered 300 images other than that of training. The results are obtained in the form of confusion matrices. The values in a cell shows, how frequently a particular error happened. The confusion matrices obtained with testing sets of Cohn-Kanade Facial Expressions Database are given in Table 1. In the study (Murthy and Jadon, 2007), the authors have reported the usefulness of this scheme for three expressions only (happiness, sad and surprised).  In this study, the authors are expanding the set of expressions to six universal expressions. They have also amplified the training images from 10-20 for each expression and expression subspaces are amplified from three to six expressions. They have also improved the testing images from 25-50.The accuracy rates are increased by 10% for Happiness and by 28-30% in case of surprised and sad respectively when compared with the results of (Murthy and Jadon, 2007). One of the reasons for this enhancement in identification rate is that there is an association between performance and the total number of samples. An analysis of the Confusion matrix for Cohn-Kanade Facial Expression Database (Table 1) suggests that the most excellent recognized group is Surprise 90% followed by Happiness 83%. The system is puzzled and recognized happiness as surprise four times and similarly recognized surprise as happiness two times. Probable reason is, in both situation the mouth is opened widely. Out of all the expressions, fear is the most confused expression, which is confused with all the other expressions. As per Zhang (1995), the expressers who are posing for expressions found that it is very hard to pose fear expressions exactly. There is a proof behind this suggestion that fear expressions are processed differently from the other basic facial expressions. Another significant examination to note is sad, anger and disgust are the expressions, which are extremely confused each other. Roughly, thirteen times they were confused with the other two expressions, which is very high. Even from the universal expressions shown in Fig. 1 the authors can observe all the three expressions are very close to each other. The difference between such expressions is tough to distinguish.
Further, the authors have also used the JApanese Female Facial Expressions database (Lyons et al., 1998) to test the efficiency of the this new method for facial expression recognition. The database contains 213 images of seven facial expressions posed by ten Japanese female models. For training, the authors have used 30 expressions and for testing, they have considered 30 images for each expression. They have used all expressions except neutral expression. The confusion matrices for JAFFE are given in Table 2. Analysis of the confusion matrix of JAFFE (Table 2) suggests that the system is again maximum confused between Happy and surprise. In case of surprise, it has eight expressions classified as happy, it is the only largest confusion when compared with other expressions' confusions.

CONCLUSION
When analyzing facial expressions, the authors always consider perspective information of the circumstances, knowledge about the observed person, voice, speech and hand and body signals. Even though humans recognize facial expressions without attempt or wait, consistent expression recognition by system is still a huge challenge. When compared with the facial expression recognition method based on the video sequence, the method based on the static image is further difficult because of the lack of chronological information. The most important role of this study is to examine the facial expression recognition based on static image and to suggest a new recognition method using Eigenspaces. The planned system is tested using Cohn-Kanade Facial Expression database and JAFFE database. The investigational results shows the effectiveness of the suggested method. In future, the study can be done to develop facial expression recognition system, which will make use of bothe neural network and wavelet transformations.

ACKNOWLEDGMENT
The researcher thank the team of Cohn-Kanade facial expression database and JAFFE for permitting to use the databases for their research. The authors also thank the authors of (Murthy and Jadon, 2009) because in this study, the work of (Murthy and Jadon, 2009) was implemented, tested with different inputs and expanded.