International Journal of Research in Information Technology International Journal of Research in Information Technology International Journal of Research in Information Technology International Journal of Research in Information Technology
(IJRIT) (IJRIT) (IJRIT) (IJRIT)
www.ijrit.com www.ijrit.com www.ijrit.com www.ijrit.com ISSN 2001-5569
Hand Gesture Recognition for performing Computer Mouse operations using K- Nearest Neighbor approach
Priyavarsha.S1, Arokia Magdaline.S2, Saranya.G3
1P.G student, Parisutham institute of technology & science, Affiliated to Anna university Thanjavur, Tamil Nadu, India
2Asst.professor, Parisutham institute of technology and science, Affiliated to Anna university Thanjavur, Tamil Nadu, India
3P.G student, Parisutham institute of technology & science, Affiliated to Anna university Thanjavur, Tamil Nadu, India
Abstract
Human computer interaction can be performed using hand gesture recognition. By this, computer devices such as mouse or keyboard can be operated with various hand gestures. In this proposed approach, it is been possible to perform more number of mouse operations which includes the basic operations such as left click, right click, double click, drag, refresh. Therefore it provides major support to the dumb people, those who can’t operate the speech based recognition systems. Once the image is acquired, it has been given to the movement tracking and gesture recognition modules. Movement tracking can be done using background subtraction algorithm. Gesture recognition can be performed using K-Nearest Neighbor classifier. This hand gesture recognition can be done by using low cost web camera instead. This proposed system encounters various drawbacks in various approaches are uniform illumination, uniform background and high cost.
Keywords: Human computer interaction, gesture recognition, movement tracking, K-nearest neighbor.
1. Introduction
Hand gesture recognition is an efficient way for Human – Computer Interaction (HCI). It provides major advantage to the dumb people those who cannot operate the speech based recognition systems. It also provides support to the blind people to keep their hand in the correct especially in front of the web camera. In Previous works, lighting has to be controlled in an efficient way. By [2], classification of the various gestures for, performing appropriate event is the major problem. To overcome this Classification problem, K-Nearest Neighbor (KNN) technique has been implemented in this proposed approach. The inputs given to perform the mouse action are limited in [4]. With these major drawbacks, uniform background and uniform illumination causes major problem in detecting and recognizing the various gestures. Apart from all these limitations, cost is also an important factor to be considered. In [1], very expensive kinect sensor has been used to capture the hand gesture and motions. The resolution of this sensor is low in case of kinectic depth map. To reduce the cost, instead of using this sensor, low cost web camera has been used in the proposed approach. The hand gesture recognition system framework comprises of three modules such as image acquisition (pre processing), movement
tracking and gesture recognition. For tracking the hand region, background subtraction algorithm has been used here. For recognizing the various hand gestures, KNN classifier has been used. The framework of the robust hand gesture recognition system is shown in the fig [1].
In the proposed system, the various hand gestures have been captured by the web camera. Once the gestures had been captured by the web camera, all the pre processing steps needs to be performed. After this process, movement of the hand region has to be tracked. This can be possible using Background subtraction Algorithm (BSA). Once it has been tracked, recognition of the given input gesture has to be performed. This recognition can be made as possible using K-Nearest Neighbor classification technique.
Fig 1.Frame work for this proposed hand gesture recognition
In fig 1, Dataset consists of the various hand gestures and comparison has to be made only after the process of detection and recognition of the input hand gesture. Once the given input gesture correlates with any of the hand gestures stored in the dataset, corresponding output events such as left click, double click, and right click and so on. The proposed system has been organized in a hierarchical manner which has been shown as follows:
section II deals with the process of the image acquisition (pre processing). Section III explains about the movement tracking of the hand. Section IV describes the gesture recognition between the recognized gestures with the gestures stored in the database. Section V deals with the experimental results. Section VI describes the conclusion part. The above frame work of hand gesture recognition system comprises of two main blocks such as movement tracking and gesture recognition.
2. Image acquisition
In this image acquisition module, input can be considered as the hand gesture which has been captured by the web camera. Following that various pre-processing steps have been performed to extract the features of the hand. Once the input gesture is captured by the web camera, the color of the image has to be normalized mainly to adapt to the varying lighting conditions as shown in the following fig 2. The main aim of this color normalization is to change the brightness level of the image. After this color normalization, RGB to HSV color conversion has been done to obtain the gray scale image. Because hand colors may get differ for different persons.
Fig 2.Pre processed output for the captured image This HSV conversion has to be made using the following formulas:
V=max{R,G,B}
δ=V-min{R,G,B}
S=δ/V
To obtain value for hue, following are the cases
(i)ifR=V,thenH=1/6(G-B)/δ (ii)ifG=V,thenH=1/6(2+(B-R)/δ) (iii) if B=V then H=1/6(4+(R-G))
Components Minimum value Maximum value
Hue 0 20
Saturation 30 150
Value 80 255
Table 1.Minimum and maximum values for the color components
Here ‘Hue’ and ‘Saturation’ are the components related to the color. Similarly ‘Value’ can be related to the brightness. Once the gray scale image is obtained, noise filteration has to be performed. The filter used here to perform this filteration is ‘Median filter’ which is mainly used to remove the holes and also to sharpen the edges. This median filter assigns each pixel a median value of its neighbors. Then the filtered image has been converted to the binary image format. After this process, BLOB (Binary Linked Objects) analysis needs to be done. A BLOB is where group of pixels share the same label due to their connectivity in the binary image. For each and every linked object, Region of Interest has to be defined. Thus the image has been segmented into a single connected skin tone region with black everywhere.
3. Movement tracking
For movement tracking of the hand, background subtraction algorithm has been used. There are two main processes which needs to be performed for tracking the hand region from the background part. Those steps are as follows:
Step 1: Background modeling Step 2: data validation
The data flow diagram for this tracking module can be shown in the following figure 3.
Fig 3. Flow diagram for hand tracking process
3.1 Background modeling:
• Background subtraction: process of separating foreground objects from the background.
Frame Difference Method (FDM) has been used here.
FDM is the process of taking the difference between the correct frame and the reference frame.
|frame(i)-frame(i-1)|> Th Where,
Frame(i) = current frame Frame(i-1) = previous frame
Th = threshold value = 0.002 3.2 Data validation
Data validation is the process of eliminating the pixels those are not satisfying the above mentioned threshold condition.
4. Gesture recognition
The following are the steps can be followed for performing the gesture classification using KNN classifier.
Step 1: Euclidean distance calculation using the following formula,
Where,
P = feature points in the input image
Q = feature points in the images which are stored in the trained database i=number of reference images, it varies from 1,2,…n.
Step 2: Sorting the nearest neighbors based on the above Euclidean distance calculation.
Step 3: Grouping the neighbor points based on the nearest point.
Step 4: Identifying the appropriate gesture based on the majority value obtained in the previous step.
Step 5: Performing the assigned mouse operations for the identified gesture.
5. Experimental Results
Pre-processed image
Background modeling
Data validation Tracked
handregion
For movement tracking module, the output obtained can be shown in the following figure 5.
After recognizing this gesture, the assigned numerical value for this gesture can be shown as 1.
And for the above gesture, the operation assigned here to perform the mouse operation is cursor movement.
6. Conclusion
Hence in the existing works for performing mouse operations has been limited in case of number of events such as left click, right click, double click, etc. But in this proposed work, maximum of 14 mouse events have to perform. And also it avoids the need of wearing any particular color taps/gaps to be wearer in the hand to segment the hand region.
7. References
[1]. Zhou Ren, Junsong Yuan , Jingjing Meng, Zhengyou Zhang, ‘Robust part-based hand gesture recognition using kinect sensor’, IEEE transactions on Multimedia , vol.15, no.5, August 2013.
[2]. Ehsan ul haqI, Syed Jahanzeb Hussain Pirzadcl, Mirza Waqar Bailand Hyunchul Shin, ‘New Hand Gesture Recognition Method for Mouse Operations’, School of Electrical and Computer Engineering, Hanyang University, Erica Campus, Korea.
[3]. Junhao Jiang, Junji Ma, Yiye Jin ‘Computer Music Controller Based on Hand Gestures Recognition Through Web-cam’, Department of Electrical Engineering, Stanford University.
[4]. X. Zabulisy, H. Baltzakisy, A. Argyroszy y, ‘Vision-based Hand Gesture Recognition for Human- Computer Interaction’, Institute of Computer Science Foundation for Research and Technology - Hellas (FORTH) Heraklion, Crete, Greece zComputer Science Department, University of Crete Heraklion, Crete, Greece.
[5]. Hervé Lahamy and Derek Litchi, ‘Real-time hand gesture recognition using range cameras’, Department of Geomatics Engineering, University of Calgary, 2500 University Dr NW, Calgary, Alberta, T2N1N4.
[6]. Ketki P. Kshirsagar, ‘Segmentation of One and Two Hand Gesture Recognition using Key Frame Selection’, Department of Electronics Engineering , Ph.D. student of S.G.G.S. Institute and Technology, Nanded Working in P.D.E.A.College of Engineering, Manjari (BK), Pune, India.
[7]. Elena Sánchez-Nielsen, Luis Antón-Canalís, ‘Hand Gesture Recognition for Human-Machine Interaction’, Department of Statistic,O.R. and Computer Science,University of La Laguna Edificio de Física y Matemáticas38271, La Laguna, Spain. Mario Hernández- Tejera Institute of Intelligent Systems and Numerical Applications in Engineering. Campus Universitario de Tafira35017, Las Palmas, Spain.
[8]. Ajita Satheesh, Ravindra Patel, ‘Dynamic Nearest Neighbours Claasifier for Integrated Data Using Object Oriented Concept Generalization’, IJSST, Vol11, No.1, MP, India.
[9]. Anand.H.Kulkarni, Sachin.A.Urabinahatti, ‘Performance Comparison of Three different classifiers for HCI Using Hand Gestures’, IJMER, vol2, issue 4 – 2012, Karnataka, India.
[10]. Dmitry Kostyrev, Sergey anishchenko, Mikhail Petrushan, ‘Time Invariant Hand Gesture Recognition for Human Computer Interaction’ Rostov-on-Don, Russia.
[11]. Harsha Varwani, Heena Choithwani, Kajal Sahatiya, Shruti Gangan, Tina Gyanchandani, Dashrath Mane, ‘ Understanding Various Techniques for Background Subtraction and Implementation of Shadow detection’, IJCTA, vol 4(5) 2013, Chembur.
[12]. Harshith.C, Karthik.R.Shastry, Manoj Ravindran, M.V.V.N.S Srikanth, Naveen Lakshmikhanth,
‘Survey on Various Gesture Recognition Techniques for Interfacing Machines Based on Ambient Intelligence’, IJCSES vol 1, no.2,Nov-2010, Coimbatore.
[13]. Mahmoud Abdulwahab Alawi, Othman O.Khalifa, M.D.Rafiqul Islam, ‘Performance comparison of Background Estimation algorithms for detecting moving vehicle’, IDOSI Publications, 109-114,2013.
[14]. Oleg Rumyantsev, Matt Merati, Vasant Ramachandran, ‘ Hand Sign recognition through palm gesture and movement’ ,Stanford University, EE368, Spring 2012.
[15]. Oleksiy Busaryev, John Doolittle, ‘Gesture Recognition with Applications’, Columbus, OH 43210,USA.
[16]. Priti P.Kuralkar, V.T.Gaikwad, ‘Human Object tracking using background Subtraction and Shadow Removal techniques’, IJARCSSE, vol 2, Issue 3, Mar 2012, India.
[17]. Qingchen, Nicolas D. Georganas, ‘Hand Gesture Recognition using Haar like features and a stochastic context free grammar’, IEEE Transaction on Insrumentation and Measurement, Vol 57, No.8, Aug 2008.
[18]. Q.Peter He, Jin Wang, ‘Fault detection using the K nearest neighbour rule for semiconductor manufacturing processes’ IEEE Transaction on Semiconductor manufacturing, Vol 20, No.4, Nov 2007.
[19]. R.Manikandan, R.Ramakrishnan, ‘Human object detection and tracking using Background subtraction for Sports Applications’, IJARCCE, vol 2, Issue 10, October 2013, Chennai, India.
[20]. Regina Lionnie, Ivanna K.Timotius, Iwan Setyawan, ‘Performance Comparison of several pre- processing methods in a Hand Gesture Recognition System based on nearest neighbour for different Background conditions’ITBJ.ICT,Vol 6, No.3 2012.
