Kernel Eigenfaces Framework for Feature Extraction and Face Recognition

(1)

International Journal of Emerging Technology and Advanced Engineering

Website: www.ijetae.com (ISSN 2250-2459,ISO 9001:2008 Certified Journal, Volume 5, Issue 6, June 2015)

506

Kernel Eigenfaces Framework for Feature Extraction and Face

Recognition

Megha A. Kamble

1

, Dr. Sanjay L. Nalbalwar

2

, Prof. Swarali P. Sheth

3 1,2,3

Electronics and Telecommunication Engineering Department, Dr. Babasaheb Ambedkar Technological University Lonere, Raigad, M.H., India.

Abstract— In this paper, three methods, namely, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Kernel Principal Component Analysis (KPCA) are implemented successfully for feature extraction and recognition of 2-dimensional face images. PCA linearly transforms the original image space to an orthogonal eigenspace with reduced dimensionality whereas LDA performs linear transformation by maximizing the ratio of between class variance and within class variance. Non-linear subspace derived using kernel method, by adopting Gaussian kernel, has been found to be superior compared to linear subspaces. The ORL and JAFFE face image databases are used to perform and test experimental analysis and results presented in this paper.

Keywords— JAFFE,Kernel Principal Component Analysis (KPCA), Linear Discriminant Analysis (LDA), ORL, Principal Component Analysis (PCA)

I. INTRODUCTION

Face recognition has been studied for many years in the context of biometrics and is one the most successful applications of image analysis and understanding. Face recognition, which is recognizing human faces from digital images and videos, has been important research area in past three decades [1], [5]. Automatic face recognition has been widely used for biometric authentication, video surveillance. Biometric recognition refers to the automatic identification of a person based on his/her behavioral or anatomical characteristics or traits. Most commonly used biometrics are face, fingerprint, iris, retinal pattern, hand and finger geometry, signature, DNA etc. Among all of these, face is the most acceptable and reliable method of identification because of non-intrusive (without participant’s cooperation) nature of acquisition it is very successful identification and recognition technique [3]. In general, facial recognition system consists main three phases:

1. Pre-processing 2. Feature extraction 3. Recognition

This paper focuses on the feature extraction and recognition task. For feature extraction principal component analysis (PCA) [2], linear discriminant analysis (LDA) [5] [7-8], and kernel principal component analysis (KPCA) [3-6] algorithms have been applied.

The paper is organized as follows; section II describes principal component analysis (PCA), and linear discriminant analysis (LDA) method, and their algorithms whereas section III describes kernel principal component analysis (KPCA) method for face recognition problem. In section IV, description of ORL and JAFFE face databases used for experimentation is given. Finally, section V, presents experimental results and conclusions on ORL, and JAFFE face image databases to compare and test the effectiveness of each method.

II.PCAAND LDA

A. Principle Component Analysis (PCA)

The PCA algorithm is based on an eigenfaces approach in which small set of most significant features are used to describe the variation between face images. In PCA projection vectors are computed such that projected data retains the most information about the input image space. PCA reduces the dimensionality of input data space by projecting data from high dimensional space N to low dimensional space M, where M<< N.

1)PCA Algorithm:

Step 1: Prepare the data set

Get training face images I1, I2, I3, I4,…IM and transform

each training face image Ii(i = 1, 2, ..., M) in database into

a vector of length N=mn and form training set S. S = {Г1, Г2, Г3, …,ГM}

For example Let M=30 such vectors Гi (i = 1, 2, ...,M) of

(2)

International Journal of Emerging Technology and Advanced Engineering

507

Step 2: Compute average face vector.

Step 3: Subtract average face vector from each training image vector.

A new training matrix A = (Ф1, Ф2, Ф3,…, ФM) (of size

N×M) is formed by arranging averaged vectors.

Step 4: Compute covariance matrix and find its eigenvectors and eigenvalues .

Covariance matrix C has dimensions N×N, it gives N eigenvalues and eigenvectors. Therefore for an image of size 92×112, we have to calculate the matrix of dimensions 10304×10304 and find 10304 eigenvectors, which is not good to do efficiently since we do not require most of these vectors. Rank of covariance matrix is limited by the number of images in training set, if we have M images; we will have M-1 eigenvectors corresponding to non-zero eigenvalues. In linear algebra, one of the theorem states that the eigenvectors and eigenvalues can be obtained by finding eigenvectors of matrix L=ATA (size of M×M). If

i

v

and are eigenvectors and eigenvalues of matrix ATA then,

Multiply both sides of above equation by A from left, we have:

Comparing equations we can conclude that the first M-1 eigenvectors and eigenvalues of matrix C are given by

i

Av

and



_irespectively. Eigenvector corresponding with the highest eigenvalue gives highest variance whereas one associated with lowest eigenvalue represents smallest variance. Therefore sort vectors by eigenvalues such that the first vector corresponds to highest eigenvalue. Form the eigenfaces matrix E of dimension N × D, where D is the number of eigenvectors associated with non-zero eigenvalue. Then project the training data into eigenfaces.

Y = ETA

Step 5: read test image of the person we want to recognize in training set is transformed into a vector B, and mean value vector Ψ is subtracted from it and projected into eigenfaces E.



Step 6: Compute the Euclidean distance between test feature vector and all the training feature vectors.

=‖ ‖

The training face with minimum Euclidean distance shows similarity to test image.

2) PCA Flowchart:

Fig.1. Flowchart for PCA method











M i i M 1 1











i i









M i T T i i M

AA

C

1 1 i i i T v Av

A 



i i i T

v

A

Av

AA





)

(

)

(

_i _i _i

T

Av

AA





)

(

)

(

Av

_i _i

Av

_i

(3)

International Journal of Emerging Technology and Advanced Engineering

508

B. Linear Discriminant Analysis (LDA)

Linear Discriminant Analysis (LDA) is based on a linear projection into a low dimensional space from image space by maximizing the between class scatter and minimizing within class scatter by using linear discriminant criterion. This criterion groups the images of same class and separates images of different classes.

1) LDA Algorithm:

Step 1: project data in Eigen space.

Step 2: Calculate mean of each class.

Step 3: Compute within class scatter matrix which represents how face images are distributed closely within classes.

∑ ∑

Where, , the ith sample of class j, is the mean of class j,

is the number of samples in class j, c is the number of classes.

Step 4: Compute between class scatter matrix which describes how classes are separated from each other.

∑

Where, µ is the mean of all classes.

Step 5: then the subspace for LDA is spanned by vectors W= [W1, W2,…, Wd], can be computed by the eigenvectors of .

Step 6: Project the data in fisherfaces

Step 7: Project test image into eigenfaces and fisherfaces and recognition is performed.

[image:3.612.326.578.131.564.2]

2) LDA Flowchart:

(4)

International Journal of Emerging Technology and Advanced Engineering

509

III. KERNEL PRINCIPAL COMPONENT ANALYSIS (KPCA)

Face recognition is still very challenging due to following problems:

1. Illumination variation problem 2. Pose variation problem 3. Occlusion

These problems can cause serious performance degradation due to non-linearity in face recognition.

Illumination: Face appears different due to variation in lighting.

Pose variation: The performance of the face recognition systems reduces due to large pose variations in the input images.

Occlusion: it means any object comes over the face in images so the face images are partially occluded.

Because of these problems performance of the face recognition system drops significantly and because of this it becomes difficult to recognize a person from an image in the database.

KPCA

Kernel Principal Component Analysis is nonlinear method in which input space is mapped into feature space using nonlinear mapping by adopting Gaussian kernel, computed by equation,

|| ||

Then find principal components in that feature space, which is nonlinearly related to the input space.

[image:4.612.328.562.145.489.2]

KPCA Flowchart:

(5)

International Journal of Emerging Technology and Advanced Engineering

510

IV. DATABASE USED

A. The ORL database

The ORL [9] face image database, contains images of 40 distinct subjects, each subject having 10 different images. For some subjects, the images were taken at different time instances, varying the lightning conditions, facial expressions (open/closed eyes, smiling/not smiling) and facial details (glasses/no glasses). All the images were taken against dark homogeneous background with the subjects in an upright, frontal position (with tolerance for some side movement) [9]. All images grayscale size of each image is 92×112 pixels.

B. The JAFFE database

The Japanese Female Facial Expression (JAFFE) [10] face image database contains 213 images of 7 facial expressions (6 basic facial expressions + 1 neutral) posed by 10 Japanese females. Each image has been rated on 6 emotion adjectives by 60 Japanese subjects.

V. EXPERIMENTAL RESULTS AND CONCLUSIONS

The PCA, LDA, and KPCA methods have been implemented for feature extraction and recognition and tested on two well-known face image databases (ORL, and JAFFE). For experiments, each one of the two databases are randomly partitioned into a training dataset and test dataset with no overlap between these two sets.

A. Experiments on the ORL database

For first experiment, the ORL database is partitioned to create training set in which randomly five images per person are chosen and remaining five images of per person are chosen to form test set. Thus, training set of 200 images and testing set of 200 images are formed.

B. Experiments on the JAFFE database

For second experiment, from JAFFE database 20 images of each subject were chosen. To form training set of 100 images, randomly ten images per person are chosen and remaining ten images of per person are chosen to form test set of 100 images.

Fig. 4. Shows the results obtained after applying the three algorithms on ORL, and JAFFE database. The recognition accuracy is calculated for each of these algorithms as follows:

[image:5.612.325.561.110.315.2]

Accuracy rate in % =

Fig. 4. Experimental results with PCA, LDA, and KPCA methods on ORL, and JAFFE database

TABLE I

COMPARISON OF PERFORMANCE ON THE ORLDATABASE

Strategy Method Recognition Accuracy

using 5 images per person for training set

PCA 87.50%

LDA 89.50%

KPCA 92.50%

TABLE II

COMPARISON OF PERFORMANCE ON THE JAFFEDATABASE

Strategy Method Recognition Accuracy

using 10 images per person for training set

PCA 93%

LDA 94%

KPCA 99%

From Table I and II, we have observed that the performance of KPCA algorithm is found to be better than PCA and LDA algorithms. One of the reason being nonlinear nature of kernel based method. It is also observed that there is trade of between computation complexity and recognition accuracy.

Further, it is seen that as we increase the size of training database i.e., number of images per person the recognition accuracy increases at the cost of execution time.

Out of two linear methods PCA and LDA, LDA found to be better as far as accuracy is concerned.

(6)

International Journal of Emerging Technology and Advanced Engineering

511

REFERENCES

[1] M. Turk and A. Pentland, “Eigenfaces for recognition” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.

[2] Xiaofei He; Shuicheng Yan; Yuxiao Hu; Niyogi, P.; Hong-Jiang Zhang, "Face recognition using Laplacianfaces," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.27, no.3, pp.328,340, March 2005.

[3] Schölkopf, Bernhard, Alexander Smola, and Klaus-Robert Müller. "Nonlinear component analysis as a kernel eigenvalue problem." Neural computation 10.5 (1998): 1299-1319.

[4] Kwang In Kim; Keechul Jung; Hang Joon Kim, "Face recognition using kernel principal component analysis," Signal Processing Letters, IEEE , vol.9, no.2, pp.40,42, Feb. 2002.

[5] Jian Yang; Frangi, A.F.; Jing-Yu Yang; Zhang, D.; Zhong Jin, "KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.27, no.2, pp.230, 244, Feb. 2005.

[6] Muller, K.; Mika, S.; Ratsch, G.; Tsuda, K.; Scholkopf, Bernhard, "An introduction to kernel-based learning algorithms," IEEE Transactions on Neural Networks, vol.12, no.2, pp.181, 201, Mar 2001.

[7] Jing Peng; Peng Zhang; Riedel, N., "Discriminant Learning Analysis," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol.38, no.6, pp.1614, 1625, Dec. 2008. [8] Lu, J.; Plataniotis, K.N.; Venetsanopoulos, A.N., "Face recognition

using LDA-based algorithms," IEEE Transactions on Neural Networks, vol.14, no.1, pp.195,200, Jan 2003.

[9] AT & T: The database of faces (formerly: The ORL database of faces).

Website.http://www.cl.cam.ac.uk/Research/DTG/attarchive/facedata base. Html.