Abstract – Nowadays, through the advancement of science and technology, possibility of human finger provide information into computer is no longer question. Fingers movement and hand motion continuously being center of research in human computer interaction (HCI) and robotic control s. Using self-develop DataGlove, an experiment was conducted by using motion capture S ystem (MOCAP) equipped with five motion capture cameras to capture human finger movements. The purpose of this paper is to analyze voltage output from DataGlove and angle obtains from motion capture system while constructing relationship concerning both outcomes. Polynomial equation is considered toward the construction of fitting curve line i n scatter data. Through the end of project, differences between finger graphs slopes will be clarify. Preliminary result of experiment exposed the newly develop DataGlove output might closely relate into angle of finger bending movement.
Index Term— DataGlove, Finger movements, Human Computer Interaction, Motion Capture S oftware (MOCAP), Polynomial Regression
I. INTRODUCTION
Human Computer Interaction (HCI) is term used to refer an understanding and designing of differences relationship between people and computer [1]. According to [2], HCI involve in various features such as command line, menus,
Papper submitted on 10 May 2013. T his work is supported by the ScienceFund Grant by the Ministry of Science, Technology and Innovation to
Universiti Malaysia Perlis (01-01-15-SF0210).
M. Haz wan Ali, Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia Perlis
KampusPauh Putra, 02600 Arau, Perlis, MALAYSIA (e-mail: [email protected]).
KhairunizamWAN, Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia Perlis
KampusPauh Putra, 02600 Arau, Perlis, MALAYSIA (e-mail: [email protected]).
Naz rul H. ADNAN, Bahagian Sumber Manusia, T ingkat 17 & 18, IbuPejabat MARA Jalan Raja Laut, 50609 Kuala Lumpur, MALAYSIA & Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia Perlis KampusPauh Putra, 02600
Arau, Perlis, MALAYSIA (e-mail: [email protected]).
Y.C Seah, Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia Perlis KampusPauh Putra,
02600 Arau, Perlis, MALAYSIA.
Juliana Aida Abu Bakar, School of Multimedia Tech & Communication College of Arts and Sciences Universiti Utara Malaysia 06010 Sintok, Kedah,
MALAYSIA (e-mail: [email protected]).
Zuradz man M Raz lan, Advanced Intelligent Computing and Sustainability Research Group, School of Mechatronic, Universiti Malaysia
Perlis Kampus Pauh Putra, 02600 Arau, Perlis, MALAYSIA. (e-mail: [email protected]).
natural language, direct manipulation, and form fill. Through direct manipulation, gesture of human body that co ntains meaningful information [3] will be interpreted through pointing device/graphical display [2]. DataGlove is an example of HCI which provide information about human finger and motion frequently used for gesture recognitions [4]. DataGlove is input device wears identical to standard glove capable to capture physical data such as bending of finger [5]. Due to that characteristic, DataGlove often use in Virtual Reality [6] and hand gesture application [7]-[10].
This research concern about characteristic of finger motion achieve from preliminary experiment while modeling finger movement into voltage and angle correlation signal. Effect of voltage on angle result will be revising using polynomial regression method. According to [11], polynomial regression is form of linear regression in which the relationship between variable x and dependent variable y is modeled as an nth order polynomial. Consequently, assessing variable voltage and angle using polynomial regression would estimate the relationship among variable.
This research paper organized as subsequent; Section 2 encompasses literature review of the related researches, problem and approach acquiring finger data. Section 3 presents the methodologies of applied procedure. Section 4 divided into 2 sections, first section states about experiment setup whereas second section demonstrations the result of experiments. Final section 5 expresses the conclusion over current research.
II. LITERATURE REVIEW
The modeling of finger motions in this research is based on GloveMAP and motion capture data. GloveMAP on the contrary is DataGlove construct using strain gauge sens or to measure finger flexion [12]. Assessing output voltage from GloveMAP with proficient signal analysis and programming, excellent result could be demonstrated. GloveMAP achievement has been verified by numerous experiments revolving around GloveMAP such as virtual interaction [13] whereby the waveform produce by GloveMAP are processes and display into virtual reality as an alternative by means of regular Graphical User Interface (GUI) [14]. Furthermore through GloveMAP, PCA-based finger movement and grasping classification development [15] has been presented successful. Angle contradictory to GloveMAP required dissimilar procedure to obtain the coordinate and magnitude of
Analysis of Finger Movement by Using Motion
Information from
GloveMAP and Motion
Capture System
finger flexion.
Motion capture system by Qualisys Track Manager Software (QTM) [16] otherwise present alternative approach in angle analysis whereby the software already equipped with calculation to acquire both magnitude and components for position, angle, velocity and acceleration. Motion Capture System (MOCAP) is generally system with capability to record the movements of human, and others motion, and then using recorded data to animating graphics. MOCAP mostly use in widespread commercial such as video game and movie while as well to other area such as biomedics, biomechanic, education and artistic. Notable usage of MOCAP is in the study of skeletal parameter by Adam G. Kirk et al. [17] University of California. Jonathan Maycock, et al. [18] in 2011 also manipulating MOCAP and DataGlove on robust tracking of human hand postures for robot teaching. While on International Joint Conference 2006, Young-Il Oh et al. [19] display a promising research in low cost motion capture system for PC-based immersive Virtual Environment (PIVE) system.
III. METHODOLOGY
Fig. 1. Flowchart of GloveMAP and QT M Software
A. Flow Chart of work s
Flow Chart of works shown in fig. 1 provides overview of the proposed system. The works start with the calibration of MOCAP before measuring finger movements . MOCAP data are directly transferred to MOCAP’s control computer whereas GloveMAP data transferal over microcontroller through serial communication port. Both finger movements data obtained from MOCAP and GloveMAP are analyzed by using Polynomial regression method.
B. Kinematic of finger
According to [20], kinematic is the branch of conventional mechanics that describe the motion points, bodies and systems of bodies without consideration of the cause’s motion. Salvador Cobos et al. [21] stated that kinematic model of human skeleton comprised of 19 links that initiate th e corresponding human bones, and 24 degrees of freedom (DOF) that represent the joint. That would mean four links and five DOF for index, middle, ring and little whereas three links and four DOF for thumb.
Fig. 2 shows detail of kinematic model for human index finger. In this kinematic finger model, Metacarpophalangeal joint (MCP) is modeled by two DOF universal joint label as ϴMCP1 and ϴMCP2 however proximal Interphalangeal (PIP) and distal Interphalangeal (DIP) similarly have one DOF label as ϴPIP and ϴDIP individually. Although, both kinematic modeling and MOCAP provide method to analyze angle, the nature constrains of human hand have to be taken into account as it refrain finger flexion in certain angle degree. Finger motion constrain typically classified into Intrafinger constrains and Angle range constraints. Intrafinger constrains is a common constraint occur to same finger joint and can be calculate by refer to (1), whereas ϴDIP is stated as finger bending angle of Distal Phalange joint and ϴPIPis structure for the Proximal Interphalangeal joint bending angle.
( ⁄ ) (1)
Angle range constraints otherwise a types of difficult constraints rising to the boundary of the range concerning finger motions of hand anatomy which follow by refer to (2).
( )
And,
(2) START
GloveMAP start capturing data
Motion Capture Start capturing Data
Computer Process Data
END
Microcontroller
Cubic Polynomial Marker Detection Calibration
Data
Trigonometry Function Matlab
Motion Capture Data
Yes
No
Fig. 2. Kinematic model human index finger
C. GloveMAP
GloveMAP shown in fig. 3 is adopted flex sensors in the construction of DataGlove. The resistivity of the sensor corresponds to the increasing distance between each of carbon element inside the thin strip of flex sensor. With the change of the resistance values , the voltage outputs can be calculated referring to voltage the divider equation, where Vin is the flex sensor supply voltage, R refer as the resistance of flex sensors, whereas Vout is voltage output resulting from referring to (3). By analyzing voltage data into Matlab, waveform as shown in fig. 4 is obtained.
(3)
Fig. 3. GloveMAP attached with the flex sensor
Fig. 4. Voltage waveform outputted from flex sensor
D. Motion Capture
Qualisys Track Manager (QTM) is used as tracking software due to fact that QTM is built around set of advanced motion capture algorithms to ensuring high performan ce, accuracy and low latency [16]. QTM measures the finger movements in 3D space. Fig. 5 shows the magnitude’s trajectories of finger movements for marker #1, #2 and #3.
Fig. 5. Magnitude of marker trajectories
E.Trigonometry Function
Trigonometric according to [22], is a branch of mathematics that studies triangles and the relationships between the lengths of their sides and the angle between those sides. In this research, trigonometry function is used to model finger movements of human. Trigonometry used in a calculation is Pythagorean Theorem and Point-Slope Equation as written in (4). Whereas ( , ) is a known point, m is a slope of the line and (x, y) is any point on the line.
( ) (4)
Fig. 6 illustrates the angle ϴ°, which is determined in the calculation. Thru expending of the equation (4), equation (5) is produced.
ϴMCP1
ϴMCP2
ϴPIP
ϴDIP
Flex sensor #1 (index finger)
LED Power indicator
Control Circuit
( ) ( )
( )
( )
And, ( )
( )
( )
( )
(5)
Fig. 6. Angle, Ɵ
Based on the 3 coordinates x, y and z, the correspondence Ɵ
is calculated. Fig. 7(a) shows the magnitude of marker trajectories for each marker #1, #2 and #3. Fig. 7 (b) shows the correspondence angles Ɵ calculated by the system.
Fig. 7. (a) Magnitude of marker trajectories for marker #1, #2 and #3 ; (b) Angle, Ɵ
F. Polynomial Regression
Polynomial regression method has been used attentively to nonlinear functions for modeling real-life phenomena and usually used in mathematical model to expecting dependent variable y on independent variable x. First degree of regression analysis (nonconstant linear function) can be u sed in constructing best fit straight line in scatter plot data. Second degree polynomial is a quadratic polynomial, with better data fit than first degree polynomial. A cubic function is a polynomial with degree of 3 and has form as refer to (6) [23].
( ) (6)
Cubic polynomial usually provides superior data fit than first and second order while ensuring high coefficient of determination on scatter plots. Fig. 8 shows the result of employing cubic polynomial into the Ɵ signal in fig. 7(b). The cubic polynomial curve display great comparable with original signal while expecting the subsequent signal sequence.
Fig. 8. Cubic polynomial plotted graph
IV. EXPERIM ENT
Experiments were carried out in MOCAP environment. Both GloveMAP and QTM Software were simultaneously run to read finger movements .
A. Experiment Setup
In the experiment, 3 markers were attached to GloveMAP as shoen in fig. 11. The markers positions were at distal phalanges, proximal phalanges and metacarpals of the index finger. The cameras used in the experiment were built on Oqus 100 with average residual of 0.3 to 0.9 mm at 3.7 m at distance. The experiments were conducted by doing various movements of index finger. Each experiment was run in 2 s.
ϴ
Marker#1 Marker#2
Fig. 9. MOCAP environment with 5 Oqus 100 cameras
Fig. 10. Anatomy of hand [24]
Fig. 11. GloveMAP and the location of markers placement
B. Experiment Result
For the experimental result, all data obtained was analyzed by using cubic polynomial. Angle and voltage were plotted into same graph which voltage performs as dependent variable y whereas angle as independent variable x. Although both
systems run for 2 s, data sampling rate for GloveMAP and QTM was difference. The sampling rate of QTM was 100 fps whereas GloveMAP contain 20 fps. Fig.12 (a) shows the voltage signal of GloveMAP and fig.12 (b) shows angle signal of QTM Software. Fig. 12 (c) shows the resampled signal of angle based on the obtained voltage data.
Fig. 12. Signals (a) Voltage signal (b) Angle signal (c) resampling angle signal
Finger movements have the directions, which were bending and straighten movements. Fig. 13(a) shows signal gradient decline as angle increase indicating the finger was bent meanwhile fig.13 (b) shows slope rise with angle decreasing indicate that finger was straighten. Fig.13 (c) and (d) show the results after employing polynomial regression of finger movements data. The value of correlation coefficient indicates linearity relationship between the correlation Marker#3
Marker#2
Marker#1
coordinate points . Fig.13 (c) and (d) have r2 values of 0.99 and 0.991, respectively. The strong correlation or relationship is defined that has a value ranging between 0.85 to 1 [25].
Norm of residual value was also observed. Norm of residual is defined as the difference between observe value with the estimate function value of the unobservable statically error [26]. It also may refer as measure of the deviation between the correlation and data. A lower norm of residual value implies a better fit of regression to the observe data. Norm of residual for fig. 13 (a) and fig. 13 (b) is 0.056058 and 0.04691, respectively. The values show that a small degree of error when finger bending and straighten. Moreover, the patterns of dot slopes designate that relationship between GloveMAP output voltage with angle are may possibly perpendicular to each other.
Fig. 13. Graph of finger flexion with polynomial (a) finger bend (b) finger straighten (c) finger bend polynomial (d) finger straighten polynomial
V. DISCUSSIONS
Based on the results, the correlation between voltage signals outputted from the DataGlove and the angle calculated from motion data acquired from MOCAP is established. The scope of research works is to find the correlation between the voltages outputted from the proposed DataGlove with the angle of finger’s movement. The correlation could be used in future experiments for the acquisition of finger movement’s data of the proposed DataGlove. However, human fingers have many degrees of freedom, and in the future various experiments need to be done as a further analysis to evaluate the performance of the proposed DataGlove.
VI. CONCLUSIONS
could be produced. Furthermore, finger movements include bending and straighten. The experimental results show that bending and straighten movements have a similar characteristic. In the future, the results will be used to acquire various signals of grasping activities. The classifier will be employed to train computer knows how to grab various objects based on shapes and patterns.
ACKNOWLEDGM ENT
Special thanks to all members of UNIMAP Advanced Intelligent Computing and Sustainability Research Group and School Of Mechatronics Engineering, Universiti Malaysia Perlis (UNIMAP) for providing the research equipment’s and internal foundations. This work is supported by the ScienceFund Grant by the Ministry of Science, Technology and Innovation to Universiti Malaysia Perlis (01-01-15-SF0210).
REFERENCES
[1] Richard Harper et al., “Being Human: Human-Computer
Interaction in the year 2020,” Microsoft Research Ltd, 2008, pp. 43.
[2] Sean Bechhofer, “Human Computer Interaction,” Lecture note University of Manchester, 2010, pp. 13-16.
[3] Mark Billinghurst , “ Gesture Based Interaction,” Lecture note Chapter 14, Aug.24, 2011, pp.1.
[4] David L. Quam, “Gesture Recognition with a DataGlove,”
National Conference on Aerospace and Electronics (NAECON 1990), IEEE, 1990, PP. 775-760.
[5] Information on http://en.wikipedia.org/wiki/Wired_glove [6] S. Sayeed, N. S. Kamel, and R. Besar, “ Virtual Reality Based
Dynamic Signature Verification Using Data glove,” International Conference on Intelligent and Advanced System s 2007 (ICIAS 2007), Nov.25-28, 2007, pp. 1260-1264.
[7] M. Ishikaws and H. Matsumnra, “Recognition of a Hand-Gesture Based on Self-organization Using a DataGlove,” 6th International Conference on Neural Information Processing (ICONIP '99), vol. 2, 1999, pp. 739-745.
[8] S. Saengsri, V. Niennattrakul, and C. A. Ratanamahatana, “ TFRS: T hai Finger-Spelling Sign La Recognition System,” 2012 Second International Conference on Digital Inform ation and Communication Technology and it’s Applications (DICTAP),
2012, pp. 457-462.
[9] Wu jiangqin et al., “A Simple Sign Language Recognition System
Based on DataGlove,” 1998 Fourth International Conference on Signal Processing Proceedings (ICSP '98), vol. 2, 1998, pp. 1257-1260.
[10] T . T . Swee et al., “ Wireless Data Gloves Malay Sign Language Recognition System,” Inform ation, Com m unications & Signal Processing (ICICS 2007), Dec. 10-13, 2007, pp. 1-4. [11] Information on http://en.wikipedia.org/wiki/Polynomial_regression [12] Nazrul H. ADNAN et al., “The Development of a Low Cost Data
Glove by Using Flexible Bend Sensor for Resistive Interfaces,”
The 2nd International Malaysia-Ireland Joint Sym posium on Engineering, Science and Business 2012 (IMiEJS2012), 2012, pp. 579-587.
[13] Nazrul H. ADNAN et al., “ Measurement of the Flexible Bending Force of the Index and Middle Fingers for Virtual Interaction,”
International Symposium on Robotics and Intelligent Sensors 2012 (IRIS 2012), Procedia Engineering 41, 2012, pp. 388-394. [14] Y. A. A. Refaat , and A. A. Ahmed, “ Introduction to Graphical
User Interface (GUI) MATLAB 6.5,” IEEE UAEU Student Branch UAE University, pp. 2-35.
[15] Nazrul H. ADNAN et al, “PCA-based Finger Movement and Grasping Classification using Data Glove “Glove MAP”,”
International Journal of Innovative Technology and Exploring
Engineering (IJITEE), 3rd ed, vol. 2, Feb. 2013, pp. 66-71. [16] Information on http://www.qualisys.com/products/ Software/qtm/ [17] A. G. Kirk, J. F. O’Brien, D. A. Forsyth, “ Skeletal Parameter Estimation from Optical Motion Capture Data,” IEEE Com puter Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 2, June 20-25, 2005.
[18] J. Maycock, J. Steffen, H. Ritter, “ Robust Dataglove Mapping for Recording Human Hand Postures,” 4th International Conference (ICIRA 201)1, Aachen, Germany, Proceedings, Part II, Dec. 6-8, 2011, pp. 34-45.
[19] Young-Il Oh, Kyoung-Hwan Jo, and Jihong Lee, “ Low Cost Motion Capture System for PC-based Immersive Virtual Environment (PIVE) System,” International Joint Conference 2006 (SICE-ICASE), Bexco, Busan, Korea, Oct. 18-21, 2006, pp. 3527-3530.
[20] Information on http://physics.tutorvista.com/motion/ kinematics-equations.html
[21] S. Cobos, M. Ferre, R. Aracil, J. Ortego and M. A. Sanchez-Uran, (2010), “Simplified Human Hand Models for Manipulation T asks,” Cutting Edge Robotics 2010 Chapter 10, pp. 156, Available: http://www.intechopen.com/books/cutting-edge- robotics-2010/simplified-human-hand-models-for-manipulation-tasks
[22] Information on http://en.wikipedia.org/wiki/ T rigonometry [23] Information on http://dufu.math.ncu.edu.tw/calculus/
calculus_pre/node7.html
[24] Information on http://www.yalemedicalgroup.org/stw/
Page.asp?Page ID=ST W023547
[25] [25]Information on http://www.isixsigma.com/tools- templates/graphical-analysis-charts/understanding-scatter-diagrams-and-correlation-analysis.html
[26] [26]Information on http://en.wikipedia.org/wiki/Errors_and_ [27] residuals_in_statistics.
M.Haz wan Ali received his Bachelor Engineering degree in Mechatronic Engineering from University Malaysia Perlis in 2012. He is currently a MSc student at University Malaysia Perlis. His research interest is in Robotics, Human-Computer Interaction (HCI), Virtual Reality and Artificial Intelligence.
Khairunizam WAN received his B. Eng. degree in Electrical & Electronic Eng. from Yamaguchi University and Ph.D. in Mechatronic Eng. from Kagawa University, in 1999 and 2009 respectively. He is currently a Senior Lecturer at School Of Mechatronic Engineering, University Malaysia Perlis. He is member of Board of Engineer and Institute of Engineer, Malaysia. His research interest is in Human-Computer Interaction (HCI), Intelligent T ransportation System, Artificial Intelligence and Robotics.
Y. C . Se ah is currently a Bachelor Degree student in Mechatronic Engineering at University Malaysia Perlis. His research interest is in Robotic, Artificial Intelligence and Mechatronic System,
Juliana A. AbuBakar currently lectures virtual reality and multimedia technology courses at University Utara Malaysia (UUM). She received B.Eng. degree in Electronic Engineering from University of Leeds, UK in 1999 and MSc. degree in Information Technology from UUM in 2003. She was awarded Ph.D from International Islamic University Malaysia in 2012 where her Ph.D thesis covers a complete cycle of design, development, and user evaluation of a virtual reality application for architectural heritage learning. She is passionate in virtual reality research and development projects since her first involvement in the academic world and has secured several national grants and published many articles in th e area.
