A Simple Method for Face Normalization Based on
Novel Normal Facial Diagram
Master Prince
Department of Computer Science, Qasim University, Buraydha, Qasim, 6688, Kingdom of Saudi Arabia. [email protected]
Abstract--- The human face is very special. It is the crucial part of the body by which one recognizes a person visually. In this paper, a simple and an efficient method of emotion normalization for faces recognition is introduced. This approach is also used to reduce the computational overhead by 1/n (n is number of images for an individual in a training set) and minimize distance between source image and target images. This method is mainly based on deforming a source face to represent corresponding normal face using proposed Normal Face Diagram (NFD) in order to normalize the source face. An imaginary NFD based on the location of the eyes pair of the source face is drawn, the diagram consists of edges connecting vertex of the pair of eye brows, eyes, nose tip, end points of the lips and mid of the bottom lip. So source facial diagram (SFD) is drawn based on extracted feature points. The genetic algorithm is used to search for possible face region, while the Eigen face technique is used to determine the fitness of the region. After alignment of an eye point (in our method left eye is aligned) our method obtain the vectors of displacement between SFD and NFD then project the vertex of the SFD along with the corresponding vertex of the NFD by adjustment of the magnitude and direction of these motion vectors.
Index Term-- Eigen face, Genetic Algorithms, Motion vectors, Magnitude and Direction, Normal facial diagram
1. INTRODUCTION
Human face recognition is one of the most emerging areas for the researchers in the last decade. In face recognition process we have a training set of images which contain feature (Eigen) vectors of different person’s images as well as a number of images of the same person keeping different expression on the face and taken images from different angles. Usually six expression is classified (joy, anger, surprised, disgust, fear and sorrow), instead of these expression person can ovaries facial organs in number of ways including jaw, lips, nose, eyebrow and iris and that may not belong to any one of the classified expressions. The idea is to draw NFD for the source face based on the location of the eyes pairs [14] and actual diagram of the source face SFD. The vertex representing any one of the eyes of NFD is aligned with the same eye (left eye in my case) of the SFD and then other eye is aligned by stretching or shrinking and/or rotating the face after that apply projective transformation in order to align rest of the vertexes of the SFD with corresponding vertex of the NFD of the source face.
Many methods have been proposed to handle the normalization of a facial image in terms of illumination, shirring angle, poses (camera angle). Our main focus is on minimizing the difference between test image and training set images for perfect recognition and minimizing the processing over head. For this we keep number of images of the same individual in the database and normalize the image as well.
1.1 Problem in recognition
There are several variables that effect the detection performance, including wearing glasses, illumination, camera angle, shirring angle, and facial expressions. There are lot of efforts have been taken to cope up with these except expression normalization. Not only recognized expressions, a human being can move their facial organs in different ways that may not belong to any of the recognized expressions. As a result the true face may not be detected. Moreover, it’s a time consuming process due to lack of constraint on the number, location, size, and orientation of the face in the image.
1.2 Existing work on facial feature extraction and emotion recognition and cloning
Table I
Recent Facial Landmarking Methods In The Literature
References Number of Test
Landmarks Database
Valstar et al. [14] (2010) 22 FERET+MMI
Kozakaya et al. [16] (2009) 14 FERET
Area et al. [14] (2006) 16 XM2VTS
In my case only eight landmarks are needed to draw NFD and SFD.
A number of methods are proposed for emotion recognition. Jun-yong Noh & Ulrich Neumann [5] present method for cloning expression on face using radial basis functions (RBF) and cylindrical projection through motion vectors direction and magnitude adjustment. K. C. Huang et al. [6] presents emotion recognition based on triangular facial feature extraction method. Rolf M. Koch et al. [13] proposed emotion editing using finite elements. They use both medical data for the simulation and the consideration of facial anatomy during the definition of muscle groups.
2. METHODOLOGY
Processing in the proposed method consists of four major stages, these are face detection, feature extraction, drawing SFD and NFD for the test image, projection of SFD vertexes on NFD vertexes.
Fig. 1. Proposed facial image emotion normalization and recognition system model.
For the feature extraction K. W Wong et al., [14] presents, the possible eye regions are detected by testing all the valley regions in an image. A pair of eye candidates are selected by means of the genetic algorithm [15] to form a possible face candidate. For drawing Based on the locations of the eye pairs, a population of possible face regions of different locations, sizes, and orientations can be generated [14].
2.1 Face Detection and Feature Extraction
Since the human iris in a gray-level image is of low intensity, a valley exists at an eye region. The valley field ɸv, can be extracted using morphological operators [10]. The equation for valley field extraction is
ɸv = f(x,y). B - f(x,y), (1)
where f(x,y) is the image and B is the structuring element, a pixel at (x,y) is considered as a possible eye candidate if the following criteria is satisfied:
f(x,y) < tl and ɸv(x,y) > tv, (2)
where tl and tv are thresholds.
In our approach two entries are selected from the buffer to form a possible face candidates in the genetic algorithm.
Fig. 2. The defined geometry of our head model
hface = 1.8 deye 3(a)
heye = 1/5 hface 3(b)
weye = 0.225 hface 3(c)
d = 1/3 hface 3(d)
Normalization of the possible face region
Shirring effect approximation and normalization.
Fig. 3. Shirring angle approximation
Normalization process for the shirring effect is performed using this transformation
(4)
In order to reduce the lighting effect, the possible face candidate would be normalized by transforming their histograms to the histogram of the reference face image [11] before calculating the fitness value.
Here I proposed a novel approach for emotion normalization. First, to determine whether the normalized face candidate is a face or not, the fitness value of the possible face region is computed by means of eigenface [18]. The eigenface are obtained by extracting the principle components from a training-set of pre-processed face images. By calculating Euclidian distance of each of the training set image from the weight of the mean image the maximum and minimum difference can be set as a fitness value.
Euclidian distance:
εn=||Ω-Ωn||2 (5)
Fitness function is defined as:
f(n) = { εmax(n) >= εmax , εmin(n) >= εmin & <= εmax } (6)
Where f(n) is a measure of the Euclidian distance between the input candidate and the training images. The possible face candidate with the correct fitness value passed through the next phase to extract the respective facial feature.
Feature extraction
After detecting the edge (canny) a combination of Global and Grid features are used to extract features. The Global features includes inter ocular distance, the distance between lips to the nose tip, the distance between nose tip to the line joining two eyes, the distance between lips to the line joining two eyes, eccentricity of the face, ratio of dimension, width of lips. The grid features used are the skin color, moustache region, lip region, eye tail, fore head, cantus, eyelid, and nose wing of the face image[13].
Now, SFD and NFD for the test image can be drawn:
2.2 Drawing of NFD and SFD
(a) (b)
Fig. 4. (a) NFD based on location of the eyes pair , (b) Source Facial Diagram (SFD) connecting eight landmarks.
2.3 Alignment of SDF with NFD
(a)
(b)
Fig. 5. (a) Mapping of SFD with NFD based on base line (b) example of determining a motion vector
The displacement vectors are v0 to v7
Ex. v3 = vn3 vs3 (7)
The maximum displacement.
2.4 Projective Transformation
(a) (b)
Fig. 6 (a). before transformation of the image, the red area stretched and has to be omitted (b) after transformation shows the displacement of neighborhood
pixels.
For a projective transformation:
[ up vp wp ] = [ x y w ] T (8)
u = up / wp, v = vp / wp
T is a 3-by-3 matrix, where all nine elements can be different.
(9)
The above matrix equation is equivalent to these two expressions:
(10)
(11)
3. EXPERIMENTAL RESULTS
(i)
(ii)
(iii)
(iv)
(a) (b) (c)
Fig. 7 (a). Sample test images (b) deformed images (c) training set images.
Table II
Using NFD and without using NFD the accuracy in recognition process with different emotions on the Olivetti Research Laboratory (ORL) face
database.
Emotions Accuracy
(NDF)
Accuracy (Without NDF)
Anger 98% 97%
Disgust 99% 98%
Fear 98% 97%
Joy 99% 99%
Sorrow 99% 98%
Surprise 97% 97%
Others 89.0% 96%
Table III
Using NFD and without using NFD the accuracy in recognition process with different emotions on the Facial Recognition Technology (FERET) database.
Emotions Accuracy
(NDF)
Accuracy (Without NDF)
Anger 98.4% 97.5%
Disgust 99.0% 98.0%
Fear 98.2% 97.0%
Joy 99.1% 99.0%
Sorrow 99.0% 98.2%
Surprise 97.0% 97.5%
Others 91.0% 97.0%
images in the database. So as the images increase processing overhead increases dramatically.
4. CONCLUSION
This paper presented an algorithm for detecting, normalizing and fast recognition of face images. The normalization includes shirring effect, lighting effect and most importantly removing emotions. In order to eliminating the emotions form the facial images the facial organs including eye brows, eyes, nose tip, end points of the lips and mid of the bottom lip are replaced to its original using the base line connecting eyes and the dimension given by the equation 3(a), 3(b), 3(c) and 3(d). The further approach will deals with the images taken from different camera angle, because drawing of NFD for non frontal facial image is difficult. This approach can deals with the slight camera angle.
REFERENCES
[1] A. A. Salah, H. Cinar, L. Akarun, and B. Sankur, “Robust facial landmarking for registration, “ Ann. Telecommun., vol. 62 no. 1/2., pp. 1608- 1633, 2007.
[2] A. Al-Oayedi, A.F. Clark, An algorithm for face and facial-feature location based on gray-scale information and facial geometry, International Conference on Image Processing and Its Applications, Vol. 2, pp. 625-629, 1999.
[3] A.M. Alattar, S.A. Rajala, Facial features localization in front view head and shoulders images, Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Vol. 6, pp. 3557-3560,1999.
[4] D.E. Goldberg, Genetic algorithms in search, optimization, and machine learning, Addison- Wesley, Reading, MA, 1989. [5] Jun-yong Noh & Ulrich Neumann, “Expression Clonong”,
ACM SIGGRAPH , Los Angeles, CA, USA, pp. 12-17 August 2001.
[6] Kuan-Chieh Huang, Sheng-Yu Huang, and Yau- Hwang Kuo, “Emotion Recognition Based on a Novel Triangular Facial Feature Extraction Method”, National Science Council, Taiwan, Republic of China under grant no. NSC97-2221- E-006-144-MY3.
[7] Kwok-Wai Wong, Kin-Man Lam, Wan-Chi Siu, “An efficient algorithm for human face detection and facial feature extraction under different conditions”, Pattern Recognition, vol. 34, pp. 1993-2004, 2001.
[8] M. Turk, A. Pentland, Eigenfaces for recognition, J Cognitive Neurosci. 3 (1), pp. 71- 86, 1991.
[9] M. Valstar, B. Martinez, X. Binefa, and M. Pantic, “Facial point detection using boosted regression and graph models,” in Proc. CVPR, pp. 2729–2736, 2010.
[10] P. Maragos, Tutorial on advances in morphological image processing and analysis, Opt. Engng. 26 (7), pp. 623-632, 1987.
[11] P.J. Phillips, Y. Vardi, Efficient illumination normalization of facial image”, Pattern Recognition Lett. 17, pp. 921-927, 1996.
[12] Ramesha K1, K B Raja2, Venugopal K R2 and L M Patnaik3, “Feature Extraction based Face recognition, Gender and Age Classification”, International Journal on Computer Science and Engineering, Vol. 02, No.01S, pp. 14-23, 2010.
[13] Rolf M. Koch, Markus H. Gross, Albert A. Bosshard, “Emotion Editing using Finite Elements”, CS Technical Report 281, January 9, 1998.
[14] S. Arca, P. Campadelli, and R. Lanzarotti, “A face recognition system based on automatically determined facial fiducial points,” Pattern Recognit.,Conference on Image Processing and Its Applications, Vol. 2, pp. 625-629, 1999.
[15] S. H. Jeng, H. Yuan, M. Liao, C.C. Han, M.Y. Chern, Y.T. Liu, Facial feature detection using geometrical face model: an efficient approach, Pattern Recognition 31 (3), 273-282,1998.
[16] S. M. Smith and J. M. Brady, “Susan—A new approach to low level image processing,” Int. J. Comput. Vis., vol. 23, pp. 45–78, 1997.
[17] S. Lyu and E. P. Simoncelli, “Statistical modeling of images with fields of gaussian scale mixtures,” in Advances in Neural Information Processing Systems 19, B. Schölkopf, J. Platt, and T. Hoffman, Eds. Cambridge, MA: MIT Press, pp. 945–952,2007.
[18] T. Kozakaya, T. Shibata, M. Yuasa, and O. Yamaguchi, “Facial feature localization using weighted vector concentration approach,” Image Vis. Comput., vol. 28, no. 5, pp. 772–780, May 2010.
