Eigenface Based Recognition of Emotion Variant Faces

In present, the automatic machine based face recognition has become significant due to its urgency in potential application and current scientific challenges of industries. However, most of the existing systems designed up to now can only effectively distinguish the faces when source images are collected under numerous constrained conditions. The success rate or positive impact of face recognition systems depend on a variety of information imposed in images of human faces. Pose of face, facial expression, angle, occlusion and state of structural components are some of those. Emotions can be expressed in different ways that can be seen such as facial expression, speech, written text and gestures. This model propose an efficient approach for the recognition of expression or emotion variant faces since there are very few emotion recognition software tools to handle such problems and there is a significant importance to this research area in the field of face recognition. Especially an approach proposed here to face recognition where the facial expression in the training image set and in the testing image set diverge and only one sample image per class is existing in the system. The input to the system is a frontal neutral expression oriented face image with unique background. In this image the hair is tied away from the face and facial hair should be removed. Principal Component Analysis approach was used as a primary mechanism in the proposed model. This approach has been applied purely on a set of face images in order to extract a set of eigenface images as the output. Here weights of the representation or image are used for recognition of emotions. One of the distance metric approaches Euclidean Distance used to discover the distance with the weight vectors which was associated with each of the training images for the existence of classification task.


Introduction
As one of the most successful applications of image analysis and understanding, face recognition has recently received significant attention, especially during the past several years [Zhao, 2003]. Face has gained significant importance in identifying the state and expressions of a human. Race, sex, age and emotions are some major information can be extracted or identified by analyzing human faces. Human faces are complex set of objects with so many features that can differ all the time based on influence of external environment. Due to technological advancements; there is an arousal of the world where human being and intelligent robots live together [Khandait et al., 2011]. Recognizing the emotional state of a person by analyzing him/her facial expression appear challenging but also essential many times due to the fact that most of the times facial expressions are not only the direct emotional outcome but also result from the extraction of the meaning of concepts. However, humans have the natural and inbuilt ability to detect, recognize, differentiate and identify or pinpoint a person's face. Computer is a machine which made by man and now he is trying to give artificial brain to go hand in hand with the human mind [Thuseethan, 2014].On the other hand unfortunately machines do not have this kind of capability, but can be imposed through learning.
The human sciences comprise a bank of literature based on emotion which is huge, but can be fragmented. Expression and feeling of people differs under different situations with different context. Recognizing the emotion of the face plays a significant role in the human-computer interaction. One of the key challenges for face recognition is finding efficient and discriminative facial appearance descriptors that are resistant to large variations in illumination, facial expression, pose, ageing, partial occlusions and other changes [Tan and Triggs, 2007]. The success of face recognition systems depend on a variety of information included in images of human faces such as pose, facial expression, occlusion, condition of image and presence or absence of structural components [Khashman, 2008]. Current systems have advanced to be fairly accurate in recognition under constrained scenarios, but extrinsic imaging parameters such as pose, illumination, and facial expression still cause much difficulty in correct recognition [Huang et al.,2002]. The important sources which are significant to this approach are in human psychology and linguistics, with some shadow from biology. A less studied problem Computer Engineering and Intelligent Systems www.iiste.org ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online) Vol. 5, No.7, 2014 is the recognition of expression variant faces; this problem can be formulated as follows: how can we robustly identify a person's face for which the learning and testing images differ in facial expressions? [Martinez, 2003].
The importance of recognizing emotion or expression variant faces is its ability to recognize criminals, suspects and employees of an organization. Even with one and only image it can recognize the emotion. The main objective of this research is to develop an Automatic Facial Expression Recognition System (AFERS) which automatically gathers human facial images containing some expression as input and recognize and classify it into appropriate expression class such as happy, angry, fear, disgust, neutral, surprise, and sad. One person may have too many different expressions or emotions at different times [Piquepaille, 2008]. Finally it identifies the image which has same person's face with different expressions or emotions. For example in a security division the police officer able to match the face image extracted from video or security cameras with available face images stored in system, for the purpose of identify a criminal or suspect. As mentioned previously, few drawbacks exist in newly developed system. But the proposed approach overcomes some of those drawbacks completely such as supports for single training face image.

Proposed Model
The proposed system comprises of four major modules. Those are Face Library, Pre-processing, Generate eigenfaces and Process Recognition. Figure 1 shows the block diagram of interrelation and flow of process between four major modules.

Face Library
In the training phase user should be able to add images to face library, the user can load the image from any area of disk and he/she will give personal details. These images will be saved into a specific folder and personal details regarding the image will be saved in a database developed in Microsoft Access. Here a real time database used instead of download the images from an existing database. The reason behind this is because real time databases noise free and easy use. The loaded training images will be converted into a vector format data and will be saved in MATLAB by using an ASCII file with the ".dat" extension. This generated data file will be moderated with two columns to store one image vector and a class number. Here the class number represents the unique number for each training image belongs to a particular person.

Pre-processing
The frontal face images with same background are the acceptable input format for the system. Further the image should be converted in such a way that the system understands or supports. The systems detect face under controlled conditions, such as without glasses, facial hair and any rigid head movement. Therefore, the input image should satisfy the above conditions. First the image has to be converted to grayscale by get rid of the hue (color) and diffusion information while retaining the luminance. All images are represented in matrix form and to gain maximum efficiency in matrix operations all of them attained as square matrix with the same size and are homogenized. The image inputs by user may vary in practice, and it is not a good restrict the user to use images in a pre-given size all the time. This system let the user to input image with any size and handle the resize mechanism internally after the insertion of image. System resizes all images to 480*480 pixel scales for further processing.

Generate Eigenfaces
The author has identified eigenface approach as an appropriate method to recognize expression variant faces after conducting a broad literature review because it has advantages over other face recognition methods such as speed, simplicity and learning capability [Turk & Pentland, 1991]. The Eigen face method is based on linearly projecting the image space to a low dimensional feature space [Peter N. Belhumeur et al., 1997]. Before eigenfaces were introduced in face recognition literature, most of the face recognition researches dealt with local Computer Engineering and Intelligent Systems www.iiste.org ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online) Vol. 5, No.7, 2014 features such as distance between ears, eyes and similar other features, this was not effective. eigenfaces is a Principal Component Analysis (PCA) approach, which uses a small set of characteristic pictures to describe the variations between face images. These eigenfaces will be derived from the covariance matrix of the distribution of high dimensional vector space of training set of face images. However, the identified eigenface method, which uses principal components analysis (PCA) for dimensionality reduction, yields projection directions that maximize the total scatter across all classes, i.e., across all images of all faces [Belhumeur et al., 1997]. Here, the original face image of training set will be transformed into a corresponding Eigen face and every face image is represented by a linear combination of these eigenvectors. The steps to find eigenfaces are: Step 1: Call the vector of all the training images saved in the ASCII file and stores in a matrix.
The above equation indicates a set T which comprises all the training images and N is number of available or existing training images stored in database.
Step2: Find the mean of loaded training images using the equation given below. Here, C is the covariance matrix of matrix which has subtracted image vectors; it can be simply specified as like this, Step 5: Calculate Eigen vectors and Eigen values, eigenvectors and corresponding eigenvalues should be calculated from covariance matrix C.

Here, C is a V2xV2 matrix and A is a V2 x N matrix
Step6: Obtaining the appropriate eigenfaces In this step, the Eigenvectors that have highest Eigen values will be chose as the well fit eigenface because it describes more characteristic features of the particular selected face. Eigenfaces with low Eigen values could be omitted since they matched only a small part of characteristic features of the faces.

Process Recognition
Find the weight vector of the stored images: To initialize the recognition task, at first the weight vector of the training images should be derived and calculated. And these vectors have to be stored corresponding to respective training image. Load testing image: Then the testing image will be loaded by the user and the system will convert this into a vector. Before do this some pre-processing operations should be handled and done to the testing image in order to improve the quality of the image to support to do better recognition of face. Find feature vector: After loaded the training image, the feature vector of the training image can be extracted and found. Here the new facial image is transformed into its eigenface components by a well-defined simple operation. Calculate Euclidean Distance: Classification will be performed by comparing the feature vectors of the training images with feature vector of input testing image. Distance measure is an important part of a vector model [Qian, 2004]. It can be done by using the distance measures; Euclidean Distance is probably the most widely used distance metric. The following algorithms illustrates this clearly, || || min Here, the weight vector of the testing image has been considered to take and find its distance with the weight vectors associated with each of the training image.
Choose Threshold: An optimal threshold is selected by the discriminant criterion, namely, so as to maximize the separability of the resultant classes in gray levels [Otsu, 1975]. The threshold used in a scenario where the face belongs to a testing image does not have a training image exist in database. The distance for each training image will be found out with the input testing image with respect to given image information. And even if the training image for a person does not exist in the database; still it will say the person is recognized as the training image with which scores is the lowest. Clearly this indicates an obstacle, to overcome this issue the threshold must be decided as Ө. A threshold should be chosen that defines the maximum allowable distance from any face image. If di < Ө, then the testing image is recognized as the image with which it gives the lowest distance. If d i > Ө, then the person belonging to that testing image does not belong to the database.
Display Results: If the comparison satisfies the threshold or find suitable image for at least one member then the face image will be classified as "Known face", otherwise it will be classified as "Unknown face". Once the face image is classified as "Known face" then the personal details are those saved in database will be displayed for further clarification through interface. Well organized interfaces used here to represent the result; it will increase the usability of the system [Thuseethan and Kuhanesan, 2014]. If it is classified as "unknown face" then the message "Match is not found" will be displayed.

Testing and Discussion
Publically available face data set JAFFE (Japanese Female Facial Expressions) used in order to test the prototype. JAFFE database is commonly used in measuring the performance of facial expression recognition systems [Shih et al., 2008]. Although other face databases with large number of subjects exist, they are inappropriate to test this prototype under some criteria such as they do not have enough emotion variant faces.
Computer Engineering and Intelligent Systems www.iiste.org ISSN 2222-1719 (Paper) ISSN 2222-2863 (Online) Vol. 5, No.7, 2014 35 database. All images are of size 256×256 pixels. The experimental photos give frontal face views and hair was tied away from the face to make all areas of the face visible as required by the system. Light also was controlled. The following figure 2 shows the images of eight people with neutral expression used for training.

Figure 2: Images of sample females used in testing
The below Figure 3 shows six of the basic expressions such as fear, happy, angry, disgust, sadness and surprise of a random sample person. Further it shows the neutral face of that particular person.  Here, C(r) is testing image set recognized correctly in the set T In this paper, we investigate the recognition of feature representation or expression variant faces schemes with seven different facial expressions on the JAFFE database. The overall efficiency of this system is good while having some deviations in recognition. The following table 2 shows the details of recognition rates of each and every expression with success percentage.

Limitations
The accuracy is not sufficient while considering about try to recognize a person for whom there is no training image is stored in database. This prototype currently has been tested with limited file types such as jpg, gif and tiff. Unable to test with other currently use types of image files such as bmp and png types. Due to limitation and lack of available facial images the achieved outcomes may differ slightly with other image types. The prototype has been tested with JAFFE facial image database; those databases have Chinese or Japanese faces and no other nationality faces. Due to this limitation in testing the system should be tested with other nationalities especially with some local faces.

Enhancements
Further real-time face recognition using a real time video input such as web cam can be made available to the user in order to reduce the human interaction with the system in the sense the implementation of suggested design improves the efficiency [Thuseethan and Kuhanesan, 2014]. Due to the existence of sunglasses, scarves, beards and mustaches there is a requirement to a mechanism to recognize real occluded face images. Implementing multiple recognition feature embedded with the system such as voice, so that the user can get both visual and audio of the user verified result together, which would give ultimate secured login feature and trust on the system. Implementing the system with the capabilities to recognize the face with dissimilar poses as well as detect the face in fair and complex and different backgrounds will make a complete system.

Test Set Result
PR (1)  Implement or expand the system in order to recognize different nationalities will produce an ideal system.