Comparative Evaluation of Self-Organizing
Feature Map and Back Propagation Neural
Network for Multimodal Biometric Systems
Olabode, A.O, Ajao, T.A , Ganiyu, R.A , Amusan, D.G, Yusuff, A.K and Folowosele, A.O
Abstract-Multimodal system is capable of increasing the scope and variety of input information the system takes from the users for authentication. However, Face, Ear and fingerprint have compatibility formation but little research have been done in the area of comparing biometric features in order to determine their overall accuracy in multimodal systems. A total of 2160 datasets were used for this experiment. 1260 datasets were used for the training and 900 were used for testing. These images were preprocessed using histogram equalization and feature extraction was carried out using Principal component analysis (PCA). Self-organizing feature map (SOFM) and back propagation neural network (BPNN) was used for classification. The performance of the developed multimodal biometric systems (face, ear and finger) was compared and evaluated in MATLAB environment. The results showed that SOFM has high recognition accuracy and time than BPNN.
Index term- Back Propagation Neural Network, Multimodal Biometric System and Self Organizing Feature Map.
1. INTRODUCTION
Multimodal biometrics system which combines two or more unimodal recognition systems into one single method can be used to overcome the limitations of individual biometrics and it has recently got major attention due to the increased level of security it provides which are known to be difficult to manipulate and hard to hark or bypass [1]. However, different biometrics features or traits collected from different sources of same person such as face, ear, fingerprint, palmprint, iris, voiceprint, signatures and DNA can be used in multimodal biometrics system and each biometric trait has its distinctive application, need and benefits [2].
Ear recognition has been identified as one of the attracting research areas and it has drawn the attention of many researchers due to its varying applications such as security systems and medical systems. Ears have gained attention in biometrics due
to the robustness of its shape, the images can be easy capture and its structure does not change radically over time [3]. Fingerprint-based recognition has been the longest serving, most successful and popular method for person identification and it can be considered as a pattern formed on the epidermis layer of a fingertip obtained whenever the finger is pressed against any surface [4]. Self-organizing feature map also known as a Kohonen map is a well-known artificial neural network and has been widely utilized in pattern recognition area.
modify the connection weights to gradually reduce the error. [5] concluded in his work that back propagation learning algorithm is one of the best algorithms among the Multi-layer perceptron algorithms.
Most of the existing multimodal biometrics system still encounters challenges which include large variability, high dimensionality, small sample size and average recognition time [6]. However, face, ear and fingerprint have compatibility formation but little research have been done in the area of comparing the biometric features in order to determine their overall accuracy in multimodal systems. The research evaluated the performance of self-organizing feature map and back propagation neural network as a classifier on multimodal biometrics system comprised of face, ear and fingerprint for better recognition and accuracy.
II. REVIEW OFRELATED WORKS
The term multimodal biometric system refers specifically to those biometric systems where multiple biometric modalities are used [7] Multimodal biometric systems rely on the evidence presented by multiple sources of biometric information, although there are several design issues that are associated with the multimodal biometric system development process including source of information, choice of biometric traits, choice of algorithm, classification, information fusion, cost benefit, processing sequences and level of robustness [8]. A typical multimodal biometric system is illustrated in Fig 2.1. This multimodal recognition system should meet the specified recognition accuracy, speed, and resource requirements, be harmless to the users, be accepted by the intended population, and be sufficiently robust to various fraudulent methods and attacks to the system [9]. The multimodal biometric system has image acquisition module, image processing module, feature extraction module and classification module.
Image pre-processing takes place for images at the lowest level of abstraction. These operations do not cause increment in image information content but they decrease it when entropy is used as an
information measure. The aim of pre-processing is to enhance some image features relevant for further
uses the redundancy in images and neighboring pixels corresponding to one real object have the same or similar brightness value [10]. Image
pre-processing method are classified into categories as explained below according to the size of the
neighborhood pixel that is used for the calculation of new pixel and this can be realized in MATLAB environment. The image pre-processing involves three processes;
i. Image cropping or filtering ii. Conversion to grayscale iii. Normalization of images.
Feature extraction is a type of dimensionality reduction that efficiently represents interesting parts of an image as a compact feature vector. This approach is useful when image sizes are large and a reduced feature representation is required to quickly complete tasks such as image matching and retrieval. Feature extraction technique better recognize images which also useful as a statistical technique that has found application in fields such as face recognition and image compression, and is a common technique for finding patterns in data of high dimension [12]. There are different algorithms for feature extraction used in biometric recognition systems such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Discrete Cosine Transform (DCT) [13].
such as artificial neural network, Self-organizing feature map and back propagation neural network were considered among many other techniques [15].
Fig 2.1: A typical overview of Multimodal Biometric Recognition System (Atalay, 2006) Classified as “known” or “unknown
”
Feature Concatenation and
fusion Feature extractor
Feature Normalization Pre-processing
Template Database Fingerprint image
Face image Acquisition
Ear image Acquisition
III. METHODOLOGY
This research combines the ear, face and fingerprints to develop a multimodal biometric system. Self-organizing feature map (SOFM) and Back propoagation neural network (BPNN) were used for classification. The overall performance of these algorithms were compared and evaluated on the developed multimodal system in terms of recognition accuracy, recognition time, sensitivity and specificity. The developed multimodal system was implemented in MATLAB environment. The following are the stages involved in the methodology;
a. Acquisition of the image datasets b. Image pre-processing
c. Feature extraction using PCA d. Feature level fusion
e. Classification (training and testing) of the biometric systems using SOFM and BPNN. A. Image Classification
Images of face, ear and fingerprint of 120 subjects were captured using Samsung galaxy S4 under the same lightning conditions in the size of 1200 X 1600 pixels. The original images captured were downsized into 100x100 dimensions without any alteration in the images. 120 subjects consist of total dataset of 2160 images.1260 of the images were used for training and the remaining 900 were used for testing and saved in Jpeg format.
B. Image pre-processing stage
This stage removes noise and unwanted element from the images. This converts original images obtained from camera which is in 3D form to 2D grayscale form with pixel values (0-225) which is black and white images. Normalization was used to remove any common features the images shared together and this is done by using Histogram equalization. Histogram equalization was used and this ensured that the input pixel intensity, X was transformed to new intensity value, X1 by T.
C. Feature extraction stage
Principal Component Analysis was used as feature extraction technique by converting the set of correlated images into set of uncorrelated eigenvectors and was also used for dimension
reduction of the image vector space. The PCA approach was used to reduce the dimension of the data by means of data compression basics and this revealed the most effective low dimensional structure of image patterns. Each image was represented as a weighted sum (feature vector) of the eigenvector, which was stored in an array. PCA eigenvector method considered each pixel in an image as a separate dimension, that is, N x N image has N2pixels or
N2dimensions. To calculate eigenvector, there is
need to calculate the covariance matrix. D. Feature level fusion stage
The feature set originated from three different sources (face, ear and finger) were initially pre-processed and the extracted features from each of the dataset formed a feature vector. These features from each of the dataset were then concatenated to form a new feature vector. Feature level fusion employed some feature selection technique on the concatenated feature vector. This research considered the simplest form of feature level fusion and this was done by concatenating the extracted features. Concatenation of feature set increased the dimensionality of the fused feature vector.
E. Training and Testing of multimodal system by SOFM and BPNN
i. Self-Organizing Feature Map
The input vectors were presented to the network based on the initial weights that was chosen at random and the neuron with weights closest to the input image vector was declared as the winner. Then, weights of all of the neurons in the neighborhood of the winning neuron was adjusted by an amount inversely proportional to the
For each node, the number of “wins” was recorded along with the label of the input sample. The weight vectors for the nodes were updated as described in the learning phase. During the testing phase, SOFM classified individual image based on correctly or incorrectly identified images. The fused input vector was compared with all nodes of the SOFM, and the best match found based on minimum Euclidean distance and was further subjected to some selected threshold values such as 0.24, 0.35, 0.47 and 0.58. The final output of the classified image was displayed as known or unknown and the results were recorded. The Self-Organizing Feature Map algorithm that was considered in this work is broken up into 6 steps and flowchart is represented in Fig 2.2:
i. Each node's weights was initialized.
ii. A vector was chosen at random from the set of training data and presented to the network.
iii. Every node in the network was examined to calculate which ones' weights are most like the input vector. The winning node is commonly known as the Best Matching Unit (BMU).
iv. The radius of the neighborhood of the BMU was calculated. This value started large. Typically it was set to be the radius of the network, diminishing each time-step. v. Any nodes found within the radius of the BMU, calculated in (iv.), was adjusted to make them more like the input vector. The closer a node is to the BMU, the more its' weights are altered.
vi. Repeat (ii) for N iterations.
vii. alculate error value for every neuron i in every layer in backward order j =M, M –1,…..,2, 1, from output to input layer, followed by weight adjustments. For the output layer, the error value and for hidden layers.
ii. Back Propagation Neural Network
BPNN trained the network to achieve a balance between the network’s ability to respond and the ability to give a reasonable response to the input that is similar, but not identical to the one used in the training. The training of a back propagation network used involved the feed forward of the input training
pattern, the back propagation of the associated error and the weighted adjustment. During the training phase, the data acquisition, pre-processing, feature extraction process and feature concatenation process took place concurrently and the fused image sample was presented to the BPNN at a stretch. In this implementation, back propagation neural network algorithm for error correction was used and the classification accuracy of the fused image vector for different number of hidden layer neurons was estimated.
The cross validation was implemented for different number of hidden layer neurons. Further, the testing samples were fed to neural network to determine the classification accuracy. The activation functions used in each case was hyperbolic tangent sigmoid transfer function and logarithmic sigmoid transfer function and was further subjected to some selected threshold values such as 0.24, 0.35, 0.47 and 0.58. The final output of the system was then displayed as shown on the flowchart in Fig 2.2. The following are the outline of the learning algorithm as shown below:
i. Initialize connection weights into small random values.
ii. Present the pth sample input vector of pattern and the corresponding output target
to the network.
iii. Pass the input values to the first layer, layer 1. For every input nodeiin layer 0, perform: iv. For every neuron i in every layer j =1,2….,M from input to output layer, find the output from the neuron.
Evaluate Classification Result (TP, FP, FN, TN)
NO
Stop
Load Test Image
Feature concatenation and fusion
Test another Image YES
Acquire Face, Ear, and Fingerprint for training Start
Convert each dataset into grayscale
Feature extraction and dimension reduction using
PCA
Convert each dataset into grayscale
Feature extraction and dimension reduction using
PCA
BPNN or SOFM Classification
Normalize feature Vector to single vector Normalize feature Vector to single vector
IV.RESULTS AND DISCUSSION
The comparative evaluation of Self-organizing feature map (SOFM) and Back propagation neural network (BPNN) was done on the developed multimodal biometrics system that comprised of faces, ears and fingerprint. The following parameters were used to measure or evaluate the overall performance of the system:
False Positive Rate = 삨삨삨 (1)
Sensitivity = 삨삨 (2)
Specificity = 삨
(3)
Overall Accuracy = 삨 삨 삨
(4)
Where TP is true positive, FP is false positive, FN is false negative, TN is true negative.
The total training time generated by SOFM for the finger-ear-face multimodal biometrics system is 897secs while total training time generated by BPNN is 776secs. The results shown that percentage accuracy increased as the dataset features increase. Tables 1-2 presented the performance evaluated based on FP, TP, TN, recognition accuracy and recognition time at threshold values of 0.24, 0.35.0.47 and 0.58 with respect to SOFM and BPNN respectively.
The recognition accuracy at Face-Ear-Fingerprint datasets (Multimodal level) with SOFM
generated 94% of 0.24 threshold, 95% of 0.35 threshold, 96% of 0.47 threshold, and 97% of 0.58 threshold while the recognition accuracy at Face-Ear-Fingerprint (Multimodal level)with BPNN generated 93% at 0.24 threshold, 94% of 0.35 threshold, 94% of 0.47 threshold, and 95% of 0.58 threshold as shown in Table 1 and 2. The average recognition time produced at Face-Ear-Fingerprint datasets (Multimodal level) with SOFM are 97.05s at 0.24 threshold, 92.10s at 0.35 threshold, 92.05s at 0.47 threshold, 91.60s at 0.58 threshold while BPNN produced 98.25s at 0.24 threshold, 92.15s at 0.35 threshold, 92.70s at 0.47 threshold and 93.30s at 0.58 threshold. Therefore, It is concluded that SOFM classified faster than BPNN.
In Table 1, SOFM generated false positive rate of 8 at 0.24 threshold, 8 at 0.35 threshold, 6 at 0.47 threshold, 4 at 0.58 thresholds, while in Table 2 BPNN produced 10 at 0.24 threshold, 10 at 0.35 threshold, 6 at 0.47 threshold and 6 at 0.58 threshold which showed that SOFM decreased in value compared with BPNN. Consequently, sensitivity and specificity are on the increase with SOFM than with BPNN when subjected to threshold values of 0.24, 0.35, 0.47 and 0.58. Receiver operating characteristics (ROC) also attested to the superiority of SOFM over BPNN as illustrated in Fig 2.3 and 2.4.
Table 1: FACE-EAR-FINGERPRINT (MULTIMODAL LEVEL) for SOFM
Threshold TP FP FN TN FPR
(%) Sensitivity(%) Specificity(%) Accuracy(%) Recognition Time (Sec)
0.24 50 4 0 46 8 100.00 92.00 94.00 97.05
0.35 49 4 1 46 8 98.00 92.00 95.00 92.10
0.47 48 3 2 48 6 98.00 94.00 96.00 92.05
0.58 49 2 3 48 4 96.00 96.00 97.00 91.60
Table 2: FACE-EAR-FINGERPRINT (MULTIMODAL LEVEL) for BPNN
Threshold TP FP FN TN FPR
(%) Sensitivity(%) Specificity(%) Accuracy(%) Recognition Time (Sec)
0.24 50 5 0 45 10 100.00 90.00 93.00 98.25
0.35 49 5 1 45 10 98.00 90.00 94.00 92.15
0.47 47 3 3 47 6 96.00 94.00 94.00 92.70
[image:8.612.65.578.331.412.2]Fig 2.4:
Graph showing ROC curve with BPNN
V. CONCLUSION
Based on the results, it can be concluded from this research that SOFM produced more efficient results in terms of specificity, sensitivity and recognition accuracy than BPNN. Also, in terms of recognition time, SOFM generated higher accuracy at a lesser time than the BPNN. Future work can be carried out by comparing the effect of ear biometric features with other features that have not been considered with ear biometrics to know their overall performance in terms of authentication and verification in multimodal biometrics system
.
ACKNOWLEDGEMENT
WE WISH TO THANKS EVERYONE WHO HAS ONE WAY OR THE OTHER CONTRIBUTED TO THE SUCCESS OF THIS PAPER.WE SAY THANK YOU SO MUCH.
REFERENCES
[1] Ramadan Gad, Ayman Sayed, Nawal El-Fishawy, M. Zorkany, (2015): “Multi-Biometric Systems: A State of the Art Survey and Research Directions” (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 6, No. 6, pp.128–138.
Learning System: International Journal of Engineering and TechnologyVol 2 no 11. [3] Dasari, N. S. (2006). “A Simple Geometric
Approach For Ear Recognition”, A Thesis Submitted İn Partial Fulfillment of The Requirements For The Degree of Master of Technology.
[4] Anil, K., Jain, L., Ajay, Kumar. (2010). “Biometrics of Next Generation: An Overview”, Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 488241226, USA.
[5] Alsmadi, M., Omar, K., and Noah, S. 2009. “Back Propagation Algorithm: The Best Algorithm among the Multi-layer Perceptron Algorithm,” IJCSNS International Journal of Computer Science and Network Security9 (4): 378-83.
[6] Jain, A. K. and Li, S. Z.
(2010).Eds.“Handbook of Face Recognition”.Springer-Verlag, Secaucus, New York, USA.pp. 32-40.
[7] Nandakumar, K., Chen, Y., Dass, S. C., and Jain, A. K. (2009). Likelihood ratio-based biometric score fusion. IEEE Transactions on Pattern Analysais and Machine Intelligence, vol. 30(2), pp. 342-347. [8] Ross, A., Nandakumar, K., and Jain, A. K.
(2006). “Handbook of multibiometrics” New York: Springer-Verlag.
[9] Jain, A. K., Nandakumar, K., and Ross, A. (2004). “Multibiometric systems”. Communications of the ACM, vol. 47, pp. 34-40, 2004.
[10] Sonka, M .,Hlavac, V and Boyle, R (1999) Image Processing Analysis and Machine Vision Optical Data Processing Technology
and Engineering University of Michigan : Michigan.pp.510.
[11] Atalay, I., (2006). Face Recognition Using Eigenfaces. Istanbul:Istanbul Technical University, pp. 1-20
[12] Wang, Z. and Li, X. (2010). “Face Recognition Based on Improved PCA Reconstruction,” in Intelligent Control and Automation (WCICA), 2010 8th World Congress on, pp. 6272- 6276.
[13] Boualleg, A.H., Bencheriet, Ch. and Tebbikh, H., (2011). Automatic Face Recognition UsingNeural Network- PCA. In: Proc. 2nd Information and Communication Technologies ICTTA '06, Syria: Damascus, pp. 1920 – 1925.
[14] Linas, J., Bowman, C., Rogova, G., Steinberg, A., Waltz, E, and White, F. (2004). “Revisiting the JDL data fusion model II”. In proc. of 7th International Conference on Information Fusion,
Stockholm, Sweden.
[15] Vailaya, A., M. Figueiredo, A. Jain, and H. Zhang. (2001). “Image classification for content based indexing” IEEE Transactions on Image Processing, 10(1): pp.117-130.
i. Olabode, A.O is currently a research scholar from Ladoke Akintola University of Technology, with M.Tech in Computer Science from the same University. His area of interests are Soft computing and Optimization, Numerical Modelling and Simulation,Artificial Intelligence and Biometrics.
iii. Ajao, T.A is currently a research scholar from Ladoke Akintola University of Technology, with M.Tech in Computer Science from the same University. His areaof interests are Biometrics, Artificial Intelligence and Information security. iv. Amusan, D.Gis currently a research scholar
from Ladoke Akintola University of Technology, with M.Tech in Computer Science and currently doing his Ph.D from the same University. His areaof interests are Software Engineering, Internet of Thing (IoT), Biometrics and Traffic Engineering.
v. Folowosele, A.O is currently a research scholar from Ladoke Akintola University of Technology, with M.Tech in Computer Science from the same University. His areaof interests are Biometrics, Artificial Intelligence and Soft Computing.
Dr R.A, Ganiyu is a senior lecturer in the department of Computer Science and Engineering, Ladoke Akintola University of Technoology, Ogbomoso,
