Gesture Command Recognition System for Human Machine Interaction

(1)

AUSTRALIAN JOURNAL OF BASIC AND

Open Access Journal

Published BY AENSI Publication

This work is licensed under the Creative Commons Attribution International License (CC BY).

http://creativecommons.org/licenses/by/4.0/

To Cite This Article: B.Raj Narain and Maria.S.Priscilla.D., Gesture Command Recognition System for Human Machine Interaction

Basic & Appl. Sci., 10(5): 41-45, 2016

Gesture Command Recognition System for Human Machine Interaction

1_{B.Raj Narain and}2_{Maria.S.Priscilla.D}

1_{Assistant Professor, Dept. of Electronics & Instrumentation Engineering, Velammal}

2_{PG student, Dept. of Electronics & Instrumentation Engineering, Velammal Engineering College, Chennai}

Address For Correspondence:

Maria S Priscilla D, Electronics and Instrumentation Department, Velamm Engineering College.Email :[email protected], Ph.no: +918056132070

A R T I C L E I N F O

Article history:

Received 12 January 2016 Accepted 22 February 2016 Available online 1 March 2016

Keywords:

Hand gesture recognition system, Image processing, Colour detection algorithm, MATLAB.

Communication plays a vital role in human life. Communication is the process of transferring data from one entity to another over a particular medium. The message to be communicated is encoded and transmitted over a medium and is received at the receiving

research efforts are made to extend this communication over human in industrial and domestic systems that must be controlled, therefore it is c

of control are needed. In a world where humans and machines coexist, a more natural and easy communication mechanism is developed for

human-gestures which can be either through the face or hand. A gesture is a non-verbal form of communication

place of speech or together with spoken words,the rest of the body indicates an emotional state, therefore

have specific linguistic content. The types of gestures are emblems, iconic, beat, metaphoric, regulators etc. Over the years gestures have been used to communicate feelings and thoughts by humans. It has been used by actors to communicate emotions, a communication mediu

dumb.

Nowadays hand gestures are commonly used in various fields where complex machines can be operated with ease replacing mouse and keyboardand enabling navigation in virtual environment an

distance. It is mainly used in sign language recognition, remote control, pervasive game technology etc. Previously hand gestures for remote control were done using accelerometer sensors. Nowadays hand gestures are recognized using Digital Image Processing.

Image processing is processing of images using various mathematical operations by using

AUSTRALIAN JOURNAL OF BASIC AND

APPLIED SCIENCES

ISSN:1991-8178 EISSN: 2309-8414 Journal home page: www.ajbasweb.com

rights reserved

This work is licensed under the Creative Commons Attribution International License (CC BY).

http://creativecommons.org/licenses/by/4.0/

B.Raj Narain and Maria.S.Priscilla.D., Gesture Command Recognition System for Human Machine Interaction

Gesture Command Recognition System for Human Machine Interaction

Assistant Professor, Dept. of Electronics & Instrumentation Engineering, Velammal Engineering College, Chennai PG student, Dept. of Electronics & Instrumentation Engineering, Velammal Engineering College, Chennai -66, India.

Maria S Priscilla D, Electronics and Instrumentation Department, Velammal Engineering College.Email :[email protected], Ph.no: +918056132070

A B S T R A C T

This paper deals with the development of a vision based Gesture Command Recognition System that recognizes hand gestures and uses it for Human Machine Interaction (HMI). Gesture recognition is interpreting human gestures that commonly originate from the face or hand through mathematical algorithms. Hand gestures are recognized by Image Processing using Colour detection algorithm in MATLAB. In the colour method, primary-colours (red, green, blue) objects/hand gestures are detected. Hand gesture recognition technology allows operations of complex machines eliminating the need for physical contact between operator and machine. The recognized hand gestures are used for navigation of a machine in a virtual environment.

INTRODUCTION

Communication plays a vital role in human life. Communication is the process of transferring data from one entity to another over a particular medium. The message to be communicated is encoded and transmitted over a medium and is received at the receiving end and decoded to the desired message. Over the past few years, research efforts are made to extend this communication over human-machine interaction. There is a rapid increase in industrial and domestic systems that must be controlled, therefore it is clear that new and more natural methods In a world where humans and machines coexist, a more natural and easy communication -machine interaction. This communication mechanism is through human body res which can be either through the face or hand.

verbal form of communication where bodily actions transfer a particular message either in or together with spoken words,the rest of the body indicates an emotional state, therefore

inguistic content. The types of gestures are emblems, iconic, beat, metaphoric, regulators etc. Over the years gestures have been used to communicate feelings and thoughts by humans. It has been used by actors to communicate emotions, a communication medium among the illiterate and also to communicate with the deaf and

Nowadays hand gestures are commonly used in various fields where complex machines can be operated with ease replacing mouse and keyboardand enabling navigation in virtual environment and to communicate at a distance. It is mainly used in sign language recognition, remote control, pervasive game technology etc. Previously hand gestures for remote control were done using accelerometer sensors. Nowadays hand gestures are

gital Image Processing.

is processing of images using various mathematical operations by using

B.Raj Narain and Maria.S.Priscilla.D., Gesture Command Recognition System for Human Machine Interaction. Aust. J.

Gesture Command Recognition System for Human Machine Interaction

Engineering College, Chennai -66, India. 66, India.

This paper deals with the development of a vision based Gesture Command Recognition System that recognizes hand gestures and uses it for Human Machine Interaction (HMI).

human gestures that commonly originate from the face or hand through mathematical algorithms. Hand gestures are recognized by Image Processing using Colour detection algorithm in MATLAB. In the colour-detection bjects/hand gestures are detected. Hand gesture recognition technology allows operations of complex machines eliminating the need for physical contact between operator and machine. The recognized hand gestures

ual environment.

Communication plays a vital role in human life. Communication is the process of transferring data from one entity to another over a particular medium. The message to be communicated is encoded and transmitted over a end and decoded to the desired message. Over the past few years, There is a rapid increase lear that new and more natural methods In a world where humans and machines coexist, a more natural and easy communication machine interaction. This communication mechanism is through human body

where bodily actions transfer a particular message either in or together with spoken words,the rest of the body indicates an emotional state, therefore gestures inguistic content. The types of gestures are emblems, iconic, beat, metaphoric, regulators etc. Over the years gestures have been used to communicate feelings and thoughts by humans. It has been used by actors to m among the illiterate and also to communicate with the deaf and

Nowadays hand gestures are commonly used in various fields where complex machines can be operated with d to communicate at a distance. It is mainly used in sign language recognition, remote control, pervasive game technology etc. Previously hand gestures for remote control were done using accelerometer sensors. Nowadays hand gestures are

(2)

processing techniques for which the input is an image, such as photograph or a video frame. Image processing usually treats an image as a signaland applies various signal processing techniques to it. The image of the hand gestures are captured using a camera and these images are converted into signals, hence called as gesture signal processing. There are many limitations to most current systems for gesture detection like non-uniform lighting conditions, less- than-ideal camera resolution and depth of color limits the accuracy and number of possible gestures. These hand gestures are used for Human Machine Interaction. This paper is organized as follows, Gesture signal processing is briefly explained in section 2. Colour detection algorithm is presented in section 3 and the results are shown and discussed in sections 4 and 5. Finally the last section concludes the paper.

Gesture Signal Processing:

A gesture is an analog activity which can be seen by vision or by observing physically. The sensors that can be used for perceiving these gesture signals are: accelerometer, the gyroscope, cameras, and infrared or ultrasonic proximity sensors etc. In this paper, camera is used to capture these signals. The block diagram for gesture recognition system is presented in figure 1.

Fig. 1: Block diagram of the hand gesture recognition system

Use of accelerometer chip to control the gestures, causes user inconvenience, because the accelerometer needs to be placed on the user’s hand and any jerks damages the chip. In order to communicate the desired information to a digital machine for example a robot, the human gestures are converted into a signal. A detectable physical quantity or impulse (as a current, voltage or a magnetic field strength) by which messages or information can be passed is called a signal. First the input is given as images (analog data). These analog signals are captured by the sensors mentioned above and are then converted to digital signals. These gesture command signals are generated using a software platform for image processing, MATLAB.

The gesture signals are generated by the following steps, the hand gestures are first captured using a laptop camera. This step is called as image acquisition. Next the background subtraction is performed over the image, it is a technique in fields of image processing where an image’s foreground is extracted for processing. Digital images are usually prone to different types noise. Errors in the image acquisition process causes noise, that result in pixel values that do not show the original intensities of the real image. Therefore these noises are filtered out through various filters like median filters, adaptive filters etc. The image is next converted to a binary image. A binary image is a black and white image having only two possible pixel values, that 1 and 0. From this image the centroids for gesture recognition are obtained. From the position of the centroids the type of gesture will be recognized.

Colour Detection Algorithm:

(3)

returned with only the portions with that particular colour are white, while the background is black. This reduces the information of the image to only the relevant portions. A red colour band will be attached to the finger, using which the finger will be recognized.

The details of the MATLAB commands used are listed in Table 1. The laptop webcam is used to acquire the video. Find formats supported by the camera and device ID by using the MATLAB code. YUY2_320x240 is the resolution supported by the camera. There are also other formats available in MATLAB. The video is timed for 200 frames after which a snapshot is taken and converted to a greyscale. A greyscaleimage’s pixels represents intensity information, having black at the least intensity (0) to white at the highest (255). Subtract the red-colour components from the grey image.

This red colour will be extracted based on the pixel values ranging from 0 to 255 in the grey scale. Since, any signal always contains some noise, this noise in the gesture image signal is filtered using a Median Filter, a type of Spatial Filters. After filtering the image is then converted into a binary image. The Red colour used for tracking the gesture is thus highlighted enough. Next, the highlighted areas are located by computing the centroids.The X and Y coordinates represent the position of the red colour object. The values of the axes are from 0 to 255. By obtaining these coordinates the position of the finger is obtained .

Table 1: MATLAB commands and output

MATLAB command Output

vid = videoinput('winvideo',1,'YUY2_320x240') Capturing the video

vid.FramesAcquired<=120 Take snapshot after 120th frame

diff_im = imsubtract(data(:,:,1), rgb2gray(data))

Obtaining the red component from the image

diff_im = medfilt2(diff_im, [3 3]); Noise filtering from the image

diff_im = im2bw(diff_im,0.18); Conversion of filtered image to a binary image

stats = regionprops(bw, 'BoundingBox', 'Centroid'); Obtaining and displaying centroids

Results:

The results for the colour detection algorithm is shown here. Figure 2 shows the snapshot of the gesture taken after 120 frames after the video is started. Figure 3 shows the image where the red component has been subtracted from the background.

Fig. 2: Snapshot of the hand gesture Fig. 3: Image showing red component extraction

(4)

Fig. 6: Image with X and Y coordinates

Figure 4 is the filtered image using the median filter and figure 5 shows the filtered image being converted to a binary image from which the pixel values are obtained. Figure 6 shows the centroids obtained, they represent the position of the finger on the screen. The coordinates are X: 115, Y: 71.

Discussion:

Using this colour detection algorithm this gesture recognition can be done for the three colours Red, Green and Blue. The red colour area alone is recognized by subtracting the background and its coordinates are obtained using which each gesture is recognized. Generally, gestures used, vary depending on the number of fingers that are displayed. More the number of gestures, there are restrictions on the generic property of gesture control as people have to remember the different gestures for different types of controls. The maximum range for maximum probability of recognition from the camera is 40 cm. Any Red coloured component in the background can pose a risk of leading to false recognition. Therefore hand gestures for a particular purpose is thus recognized.

Conclusion:

In this paper a hand gesture recognition system is developed using a colour detection algorithm. These recognized hand gestures will be used for real time motion control in fields like robotics or human computer interaction. Consider the navigation of a robot using these gesture signals, if the finger is moved to the top of the screen then the gesture is recognized as forward motion. Similarly if the finger is moved to the bottom of the screen then the robot moves backwards.

REFERENCES

Guangqi Ye, J.J. Corso, G.D. Hager, 2004. Gesture Recognition Using 3D Appearance and Motion Features,Computer Vision and Pattern Recognition Workshop, CVPRW Conference.

Harshith, C., R. Karthik, Shastry, ManojRavindran, M.V.V.N.S Srikanth, Naveen Lakshmikhant, 2010. Survey on various gesture recognition techniques for interfacing machine based on ambient intelligence, International Journal of Computer Science & Engineering Survey (IJCSES) 1: 2.

Liao, Chung-Ju,Su, Shun-Feng; Chen, Ming-Chang, 2015. Vision-Based Hand Gesture Recognition System for a Dynamic and Complicated Environment.Systems, Man, and Cybernetics (SMC), IEEE International Conference.

Moni, M.A., A.B.M.S. Ali, 2009. HMM based hand gesture recognition: A review on techniques and approaches, Computer Science and Information Technology, ICCSIT, 2nd IEEE International Conference.

Murugeswari, M., S. Veluchamy, 2014. Hand gesture recognition system for real-time application, Advanced Communication Control and Computing Technologies (ICACCCT).

Narejo, G.B., S. Pervez Bharucha, 2013. Real time finger counting and virtual drawing using color detection and shape recognition, Information & Communication Technologies (ICICT), 5th International Conference.

Rafael, C. Gonzalez, Richard E. Woods, Steven L. Eddins, 2011. Digital image processing using MATLAB, 2nd Edition, McGraw-Hill Education (Asia).

RaquibBuksh, SoumyajitRouth, ParthibMitra, SubhajitBanik, 2014. MATLAB based image editing and color detection, International Journal of Scientific and Research Publications, 4: 1.

RuizeXu, Shengli Zhou, W.J. Li, 2012. MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition, Sensors Journal, IEEE , 12: 5.

(5)

Siddharth, S. Rautaraya, Anupam Agrawala, 2012. Real time gesture recognition system for interaction in dynamic environment, Sciverse Science direct, Procedia technology, 4.

Shukla, J., A. Dwivedi, 2014. A Method for Hand Gesture Recognition, Communication Systems and Network Technologies (CSNT), Fourth International Conference,.

Smitha, B., R. Abayadev, R. Anurag, N. Arun, A.D. Avinash, 2015. Real time gesture recognitions for dynamic applications, International Journal of Emerging Technology and Advanced Engineering (IJETAE), 5:4.