Hand Gesture Recognition Using Flex Sensors

(1)

ISSN 2395-1621 Hand Gesture Recognition Using Flex

Sensors

#1D. K. Barbole, ^#2Dr. D. V. Jadhav

#12BSCOER, Narhe, Pune

ABSTRACT ARTICLE INFO

In this system an electro-mechanical robot is designed and controlled using hand gesture in real time. The system is designed on microcontroller platform using Keil and MPLAB tools. Hand gesture recognition is done on the principle of resistance change sensed through flex sensor. These sensors are integrated in a hand gloves from which input to the system is given. The designed system is divided into two sections as transmitter and receiver. The transmitter section will be in hand gloves from which the data is sensed and processed through PIC16F7487 and send serially to the receiver section. RF technology is used to transmit the data at the receiver section at the frequency of 2.4 GHz. ARM 7 (LPC2148) processor is used to receive the data. Here from the received data, the character is predicted and matched with the closet character from which the character is identified and displayed on LCD. The various case studies is prepared for the designed system and tested in real time. The proposed system can be used for the various applications such as in unmanned machines, industries, handicapped personnel etc.

Keywords— a Sensor gloves, flex sensors, PIC controller, ARM processor etc

Article History

Received:20^thSeptember 2015 Received in revised form : 22th September 2015 Accepted:26^th September 2015 Published online :

1^st October 2015

I. INTRODUCTION

The system architecture for hand gesture recognition system using flex sensor is get divided into two parts, those are:

 Transmitter section

 Receiver section

Transmitter section consists of PIC microcontroller, hand gloves with flex sensors mounted on it, LCD display and RF module. While receiver section consists of ARM processor, DC motor, motor driver IC, LCD display and RF module. In this system transmitter gives input signal to receiver via wireless RF module. The various gestures made by user using hand gloves with flex sensors are taken as input signals. With this gesture’s signal, the vehicle will move from one place to other place. For each gesture, specific command is already defined in code like forward, stop, left, right.

II. LITRATURESURVEY

In the recent years, there has been tremendous research on the hand sign recognition. The technology of gesture recognition is divided into two categories-

2.1 Vision-based

In vision-based methods [11], computer camera is the input device for observing the information of hands or fingers.

The Vision Based methods require only a camera, thus realizing a natural interaction between humans and computers without the use of any extra devices. These systems tend to complement biological vision by describing artificial vision systems that are implemented in software and/or hardware. This poses a challenging problem as these systems need to be background invariant, lighting insensitive, person and camera independent to achieve real time performance. Moreover, such systems must be optimized to meet the requirements, including accuracy and robustness.

The vision based hand gesture recognition system is shown in fig.1:

(2)

Vision based analysis, is based on the way human beings perceive information about their surroundings, yet it is probably the most difficult to implement in a satisfactory way. Several different approaches have been tested so far.

 One is to build a three-dimensional model of the human hand. The model is matched to images of the hand by one or more cameras, and parameters corresponding to palm orientation and joint angles are estimated. These parameters are then used to perform gesture classification.

 Second one to capture the image using a camera then extract some feature and those features are used as input in a classification algorithm for classification.

2.2 Glove-based

In glove based systems [11], data gloves are used which can archive the accurate positions of hand gestures as its positions are directly measured. The Data-Glove based methods use sensor devices for digitizing hand and finger motions into multi-parametric data. The extra sensors make it easy to collect hand configuration and movement.

However, the devices are quite expensive and bring much cumbersome experience to the users some of the earlier gesture recognition systems attempted to identify gestures using glove-based devices that would measure the position and joint angles of the hand. However, these devices are very cumbersome and usually have many cables connected to a computer. This has brought forth the motivation of using non-intrusive, vision-based approaches for recognizing gestures Also the sensors used for the detection of the sign language and the gesture recognition in the system that are available in the market are quite costly. In computer recognition of spoken language, speech data is captured using a microphone connected to an ADC.

Similarly a data-capturing device is also required in order to recognize sign language; in this case measuring the position and movement of the signer's hands [12].

2.3 Comparison between vision-based hand gesture recognition and gloves- based hand gesture recognition

In case of vision- based gesture recognition systems, a lot of the digital signal processing has to be done.Also there require large programming. Because of this the response of the system is quite slow. Also the electric, magnetic fields, and any other disturbances may affect the performance of the system. In case of glove-based sign language recognition system that is available in the market lot of the hardware is required. Large numbers of cables have to be connected to the computers for monitoring the data. Hence the systems require lot of space. Also the system is not handy.

In case of our project flex sensors that we are using are of low cost. Also the ARM processor that we use in our process is very compact. Hence the space required for our system is very less compared to the other projects that are available in the market. Thus the system is portable.

Performance of the system is not affected by the disturbances. Here we are converting hand signs in to the corresponding speech signal; hence the system is the proper means of effective communication. As the hardware required in designing are low cost, the overall cost of the system is less compared to the other systems available in the market, and it is the system is flexible enough for a user to add, modify or delete a hand sign.

III. RESEARCH ELABORATION

Methodology-

Hand gesture can be best operated by wearing a glove. A glove-based system makes the user to be connected to the machine. Even wireless systems currently offered in the market makes the user to compulsory wear a glove.

Moreover, accurate devices are expensive and hard to calibrate. The block diagrams for transmitter and receiver sections are given below in fig .2 and fig .3 respectively.

Fig .2: Transmitter section for hand gesture recognition system

Image acquisition Pre-processing Feature extraction Classification

PIC micro- controll

er Power supply

Flex

sensors

16*2 LCD display

MA X 232

RF

mod

ule

(3)

System for Hand gesture recognition using flex sensor consists of Flex sensor, PIC microcontroller, ARM processor, Encoder and decoder, RF transmitter and receiver, LCD display, MAX 232. Description of each block is given as follows:

 Flex sensor

The Flex sensors are normally glued to the glove using either a needle or a thread. It generally requires a 5-volt Input power supply and Output between 0 and 5 V, the resistivity altering with the sensors degree of bend and the voltage output changing consequently. The sensors are usually connected to the device via three pin connectors namely ground, live, and output. The device can turn on the sensor from off mode, enabling it to power down when not in use and greatly decreasing power consumption.

The ex sensor Photograph shown changes resistance when twisted. It will only change resistance in single direction.

The flexed sensor has a resistance of near about 10,000 ohms. As the flex sensor is turned, the resistance rises up to 30- 40 kilo ohms at approximately 90 degrees. The dimensions of the sensor are inch wide, 4-1/2 inches long and 0.19 inches thick. The physical flex sensor is shown below in fig. 4.

Fig 4: Flex sensor

Fig 5: Circuit diagram of flex sensor

As seen in the above fig. 5 , that two or three sensors are connected serially and the output from the sensors is sent to the Analog to digital converter (ADC) in the controller. The outputs from the sensors are sent to LM258/LM358 op- amps and used as a non-inverted style setup to amplify their voltage. The more the degree of bending, the least will be the output voltage.

The output voltage is unwavering based on the equation Vin

* R1 / (R1 + R2), where R1 is the other input resistor to the non-inverting terminal. Using the voltage divider concept the output voltage is determined and it ranges from 1.30V to 2.8V.The gloves with flex sensors mounting on it are shown below in fig. 6.

Fig. 6: Gloves with flex sensor mounted on it

The characteristics of flex sensors are shown below in fig.

7(a) and fig. 7(b). When bending angle of flex sensor is increases, resistance also increases. And when resistance increases, the output voltage from flex sensor will decrease.

Bending (degree)

Fig. 7: (a) Graph of resistance Vs bending, (b) voltage Vs resistance

RF mo dul e

MA X 232

Batter y 12V

16*2 LCD displ

ay

Driv er IC

D C m ot or

1 D C m ot or

2 ARM

proces sor

Vout

Vin

(4)

 PIC microcontroller

All output signals generated from ex sensors are in analogue form and these signals need to be digitized before they can be transmitted to encoder. Therefore microcontroller PIC16F877A is used as the main controller. It has inbuilt ADC module, which digitizes all analogue signals from the sensors and inbuilt multiplexer for sensor signal selection. It supports both serial and parallel communication facilities.

 ARM processor

ARM7TDMI-S CPU with real-time emulation and embedded trace support, that combines the microcontroller with 32 kB, 64 kB, 128 kB, 256 kB and 512 kB of embedded high speed Flash memory. A 128-bit wide memory interface and a unique accelerator architecture enable 32-bit code execution at maximum clock rate. For critical code size applications, the alternative 16-bit Thumb mode reduces code by more than 30.

 Encoder and decoder

The output from the PIC microcontroller is encoded by using HT12E-212 series of encoder. It is used to correct the error at the receiver end, if any error had occurred. In the receiver it is decoded by using HTI2D212 series of decoder.

 RF transmitter and receiver

The RF module namely suggests, operates at radio frequency. Corresponding frequency range varies between 30kHz to 300GHz. In this RF system, the digital data is represented as variations in amplitude of carrier wave.

Transmission through RF module is better than IR (infrared) because of many reasons. Firstly, signals through RF can travel for larger distances making it suitable for long range applications. Also, while IR mostly operates in line-of-sight mode, RF signals can travel even if there is an obstruction between transmitter and receiver. Next, RF transmissions are more strong and reliable than IR transmission. RF transmitter and receiver operate at the frequency of 434 MHz. An RF transmitter receives serial data and transmits it wirelessly through RF its antenna connected at pin4. The transmission occurs at the rate of 1kbps -10kbps. The transmitted data is received by an RF receiver operating at the same frequency as that of the transmitter.

 LCD display

The system will recognize the gestures made by user and it will search for the instruction in the program database given by PIC microcontroller and ARM processor, to display it on the LCD. Suppose the user has to make a gesture for instruction Move forward then he will make the required gesture using his hand glove. All the sensors will be scanned for their analog voltage and it will be given to the controller to convert it into equivalent digital signals these values will be stored in internal memory. So, if entry is found then it will send command to LCD to display previously stored instruction.

 MAX 232

The MAX220/MAX249 family of line drivers/receivers is intended for all EIA/TIA- 232E and V.28/V.24 communications interfaces, particularly applications where 12V is not available. These parts are especially useful in battery-powered systems, since their low-power shutdown mode reduces power dissipation to less than 5W. The

MAX225, MAX233, MAX235, and

MAX245/MAX246/MAX247 use no external components and are recommended for applications where printed circuit board space is critical.

IV. RESULT

Results-

Here four different gestures are used to convey the commands to vehicle. Accuracy of these gestures for specific command is checked by displaying command on LCD display for that specific gesture. As any defined gesture is done, flex sensor will start bending. Due to bending, there will be some changes in resistance value of flex sensors. The result of change in resistance value after gestures are given is shown below in table 1.

Sr.

No.

Gesture Resistance value before gesture is

given

Resistance value

after gesture is

given

Command displayed

on LCD display

1. Gesture 1

536 ohm 900 ohm Forward

2. Gesture 2

199 ohm 420 ohm Left

3. Gesture 3

250 ohm 556 ohm Right

4. Gesture 4

391 ohm 568 ohm Stop

These commands are transmitted to receiver section via a RF module. With this command robot will start its movement.

Conclusion-

• This system will reduce the hectic interfaces like keyboard, mouse or any other equipment.

• Hand gesture recognition technique can provide user-friendly human and machine interfaces.

• Lifetime of gesture recognition system is high with greater precision and with efficient size.

Advantages-



speech recognition.



devices

 The prototype can be enhanced to produce full alphabet series & numbers

 Module on wrist makes it portable

 It doesn’t need translator or attendant

(5)

• Robotics

• Artificial Intelligence

• Games

• Desktop and PC Applications.

• Neural Networks.

V. FUTURE SCOPE

Gesture recognition technology can also be used to make the robots understand the human gestures and makes them work accordingly. We can extend this topic future such that, we could produce voice information based on hand gesture of user. The area of this technology is very vast and never ending. Hand recognition system can be useful in many fields like Artificial Intelligence, Embedded Systems and Neural networks as well as Fuzzy logics.

Gestures are intended to play an increasingly important role in human-computer interaction in the future.

Facial Gesture Recognition Method can also be used as an alternative while driving, especially when the driver is intended to sleep. As this technology is very user and environment friendly, it will surely create a good impact as well as curiosity between the researchers.

REFRENCES

[1] Zhaojie Ju, Member, IEEE, and Honghai Liu, Senior Member, IEEE,” Human Hand Motion Analysis With Multisensory Information”, IEEE transactions on mechatronics, vol. 19, no. 2, April,2014.

[2] Zhou Ren, Junsong Yuan, Member, IEEE, Jingjing Meng, Member, IEEE, and Zhengyou Zhang, Fellow, IEEE,

“Robust Part-Based Hand Gesture Recognition Using Kinect Sensor”, IEEE transactions on multimedia, vol. 15, no. 2, August,2013.

[3] Rory A. Cooper, Thomas A. Corfman, Shirley G.

Fitzgerald, Michael L. Boninger, Donald M. Spaeth, William Ammer, and Julianna Arva, “Performance Assessment of a Pushrim-Activated Power-Assisted Wheelchair Control System”, IEEE transaction on control systems technology, vol. 10,no. 1,January 2002.

[4] Naveen Kumar Toshawar, Chandni Jain, “Intelligent Control System using Gesture Sensing”, International Journal of Engineering and Technical Research (IJETR), STET-2014.

[5] Gunasekaran. K, Mnikandan. R, “Sign Language to Speech Translation System Using PIC Microcontroller, International Journal of Engineering and Technology (IJET)”, Vol 5 No 2 Apr-May.

[6 ] K.C.Sriharipriya, “Flex sensor based non-specific user hand gesture recognition”, Interantional journal of innovative research and studies, vol. 2 issue 5, May 2013.

[7] Prapat Parab1, Sanika Kinalekar, Rohit Chavan, Deep Sharan, Shubhadha Deshpande, “Hand Gesture Recognition using Microcontroller Flex Sensor”, International Journal Of Scientific Research And Education Volume 2 Issue 3 March 2014.

[8] Harish Kumar Kaura, Vipul Honrao, Sayali Patil, Pravish Shetty,” Gesture Controlled Robot using Image

Processing” (IJARAI) International Journal of Advanced Research in Artificial Intelligence, Vol. 2, No. 5, 2013.

[9] Miss. Kawade Sonam P, Prof. V. S. Ubale, “Gesture Recognition-A Review”, IOSR Journal of Electronics and Communication Engineering (IOSR-JECE),2012.

[10] Anupma Yadav, “Gesture recognition technology”, The Journal of Computer Science and Information Technology , ISSN 0973-4872, Vol. 3, No.1 (2006) pp. 86- 88.

[11] Ajinkya Raut, , Vineeta Singh, , Vikrant Rajput, , Ruchika Mahale, “Hand sign interpreter”, The International Journal of Engineering And Science (IJES) , Volume-1 , Issue-2 2012.

[12] Peter Vamplew, “Recognition of sign language gestures using neural networks”, Department of Computer Science, University of Tasmania

[13] Fels and G Hinton (1993), “Glove-Talk: A Neural Network Interface Between a Data-Glove and a Speech Synthesiser”, IEEE Transactions on Neural Networks, 4, 1, pp. 2-8 [3].

[14] Johnston (1989), Auslan: “The Sign Language of the Australian Deaf Community”, PhD thesis, Department of Linguistics, University of Sydney

[15] Kramer and L Leifer (1989), “The Talking Glove: A Speaking Aid for Nonvocal Deaf and Deaf-Blind Individuals”, RESNA 12th Annual Conference, New Orleans, Louisiana