• No results found

Study on Stereo Vision based Parameterized Virtual Hand Modeling

N/A
N/A
Protected

Academic year: 2020

Share "Study on Stereo Vision based Parameterized Virtual Hand Modeling"

Copied!
6
0
0

Loading.... (view fulltext now)

Full text

(1)

2017 International Conference on Computer, Electronics and Communication Engineering (CECE 2017) ISBN: 978-1-60595-476-9

Study on Stereo Vision-based Parameterized Virtual Hand Modeling

Qiong-li LIU

1

, Hui-hua CHENG

1

, Wei-qing TIAN

1

, Zhi-guo LI

2

,

Peng ZHOU

1

and Yong-xia XU

2

1Wuhan Mechanical Technology College, Wuhan, Hubei, China

2Taiyuan Satellite Launch Center, Taiyuan, Shanxi, China

Keywords: Human computer interaction, Virtual hand model, Stereo vision, Finger tips feature.

Abstract. Virtual hand modeling is one fundamental part of the model-based hand tracking system. The virtual hand model-based tracking system can be valuable in the field of the immersive human machine interaction simulation such as virtual surgery and virtual maintenance training. In this paper, a method of automatic stereo vision-based virtual hand modeling system is presented. We firstly built a template of the improved virtual hand model, then extracted the hand features from images captured by the binocular stereo vision system, and completed modeling the virtual hand in accordance with the estimated parameters. We propose a new method of combining local distance maximum and curvature vector to extract the hand features and introduce a method of recognizing the different finger tips. The advantages of feature extraction method loosen the high quality of contour extraction from images and enforce the robustness of feature extraction.

Introduction

Human hand motion tracking and recognition has been an active research topic in the field of human computer interaction field. As a widely used way of communication in people's daily life, hands gestures have obvious advantages such as naturally interaction, convenience and so on. Along with the improvement of computer computing ability and interactive equipment performance, the technology of gesture tracking and recognition based on computer vision will greatly improve the efficiency of human-computer interaction. This technology has broad application prospects in the aspects of virtual environment interaction, sign language recognition, multimodal human-computer interaction, robot grasping, etc.

Due to difference of human hands’ shape and size, the human hand motion tracking based on the 3D model needs to model the user's hand in advance. At home and abroad, modeling of virtual hand model has been widely investigated, A use camera to obtain image data to hand motion, analyze 3D hand gesture automatically, use contour to extract boundary feature for recognition, successfully extracted 27 hand gesture interaction parameters, having 3D model reconstructed. B apply image recognition, feature extraction and surface fitting method , having real-time gesture modeled. Stinger’ steam use parametric quadratic surface in C to build the virtual hand model, the parametric model make hand shape size of the virtual hand model close to the experimenter’s hand. Duse stereo vision to obtain hand position, and combined with the data glove LED color to mark hand model. The current research situation shows, the method of virtual hand modeling can be summarized into two categories, base-on-appearance modeling and 3D modeling.

(2)

Virtual Hand Model

In this paper, the human hand model is realized by using surface model, which includes two layers: skin layer and framework layer. The skin layer is defined as a geometric object that is attached to the skeleton layer. The skeleton model is for deformation and motion of human hands that is the internal structure of hand model; hand skin layer model is used to describe the external shape, fingers and palm connected part is deformable mesh, the other part is rigid model.

Skeleton Layer Model

[image:2.612.219.396.230.409.2]

The human hand model is given by E, as shown in Graphic1, the model contains 27 degrees of freedom. Among them, the wrist has 6 degrees of freedom, the thumb TM joint has 2 degrees of freedom, the rest finger CMC joint fixation, each finger MCP joints has 2 degrees of freedom (adduction and yaw), each finger IP finger joint contains 1 degrees of freedom.

Figure 1. The hierarchy of the virtual hand skeleton model.

Appearance Model

C uses the parametric quadric surface to model human hand each segment. However; this method does not take deformation part in-between the fingers and the palm into consideration, it requires the palm direction toward the camera during the tracking. And if the finger is directly connected to the palm, the use of IK method to solve the joint angle will not be accurate. To this end, this paper considers the deformation of the joints between the fingers and the palm, modeling of the joint between the fingers and the palm based on the deformation grid technology. As green deformed part shown in Graphic2.The improved model not only has high precision in the inverse kinematics solution, but also image projection basically consistent with the actual hand shape.

Human Hand Modeling Based on Binocular Stereo Vision Human Hand Segmentation

(3)

Figure 2. The virtual hand appearance model. Figure 3. The projection of model to the image.

[image:3.612.130.472.64.263.2]

Figure 4. The original image (left) and skin segment results(right).

Hand Feature Extraction and Recognition

[image:3.612.246.368.402.488.2]

The fingertip and the palm root are selected as the features. G is proposed to detect the feature of human finger tip by using curvature vector, which requires that the contour extraction result has a certain smoothness. Due to the presence of local irregularities when the extraction results are affected by noise, it is easy to make the other points to be detected as figure tip by the curvature vector detection. Therefore, this paper proposed improvement on G algorithm, this paper proposes to extract finger character in image through combination of a regional maximum distance and curvature vector, and based on this proposed, come up with an algorithm which can recognize different finger tip features. Feature extraction results are shown in Figure 5.

Figure 5. The feature extraction results.

Set AA as the manual contour extraction result point set, pcas hand shape regional center of the

hand skin color segmented image, and define the distance from point pkon the contour to point

c

p in center as follows:

2

2 ( )

)

( c k c k

k

cp x x y y

p     (1)

Set vectorpipjdefined as AB, AC as a pair of points on the contour sequence equal interval to

k

p front and back, then curvature

between vectorspkpkj and pkpkjis defined as:

) arccos(

j k k j k k

j k k j k k

p p p p

p p p p

 

 

  

(2) Then the feature extraction algorithm and recognition algorithm are as follows:

Feature Extraction Algorithm:

(4)

Step 2 Ifkn, then turn to step4. Otherwise, to determine whether pk is regionally

maximum. To find out pk the left and right neighborhood points sets ( , )

2

1 k d

d k p

p

and( , )

1

2 k d

d

k p

p   in contour points set M, calculate the distance between points in these two

region to the centerpc, at the same time, determine whether they are less than pkpc , if so, then

turn to Step3; otherwise,kk1, turn to step2.

Step 3 To judge whether the vector curvature values generated respectively by pk and equal interval points at left and right are in the threshold range, The threshold range 0~30 is selected in the experiment. If it is in this range, then recordpkas feature point.kk1 , turn step2.

Step 4 End, showing the extraction from the finger tips feature points.

Feature Recognition Algorithm:

Set{p1,p2,p3,p4,p5}as extracted 5 finger feature points, then each figure point can be identified through algorithm as following.

Step 1 To determine the thumb tip and the little fingertip. Calculate the distance between the two points among the feature points. Select two longest distance points as the thumb tip and the little finger tip{ , '}

2 ' 1 p

p .

Step 2 To recognize the middle fingertip. Among the remaining three points { , , '}

5 ' 4 ' 3 p p

p distance

to the other two points, the smallest sum one is the point for the middle fingerP3, mark the other two points as{p11,p22}.

Step 3 To identify the thumb tip and little finger tip. Among{ , '}

2 ' 1 p

p that distance toP3, the longer one is the thumb tipP1, and the shorter one is the little finger tipP5.

Step 4 To identify the index finger tip and ring finger. Among{p11,p22}that distance toP5, the closer one is the ring finger tip P4, the further one is index finger tipP2.

Step 5 To calculate the palm root positionPr. The middle fingerP3 and the center pointpc

connection marked asl,intersection point of l and the outline of M, notP3, are the palmPr. Step 6 To show the feature points recognition results, then end.

Estimation of 3D Human Hand Model Parameter

AssumeH1,H2are projection matrix from left and right cameras world coordinate system to image

coordinate system respectively. SetP(X Y Z 1)Tas homogeneous coordinates of a point in the

space, T

v u

p1 ( 1 1 1) and p2 (u2 v2 1)T are the projected points homogeneous

coordinates in the left and right images respectively;

P H k p P H k p 2 2 2 1 1 1   (3) Provide the base matrix between the two cameras isF,then the corresponding poles of the left image pointp1 in the right image are:

1 1 Fp

l (4) The parametric modeling process mainly includes two steps: Space position calculation of feature points and Estimation of the model parameters.

Space Position Calculation of Feature Points:

(5)

6 ,.., 2 , 1 ,i

PiR as corresponding matching points in the right image, then the position of the feature

points in the world coordinate system can be estimated through3D reconstruction algorithm, assumePiw,i1,2,..,6, thenP6wis the palm root feature points position in the world coordinate.

Estimation of the Human Hand Model Parameters:

[image:5.612.238.376.183.297.2]

Firstly, calculate the length of the hand finger according to the relationship between each feature points of the hand, and then obtain the user hand model by updating the template model parameters. The specific algorithm steps are as follows:

Figure 6. The illumination of hand parameters estimation.

As shown in Figure 6, assume L

i

P (i=1,2,..5) is human fingertip position in the left image ,P6L is palm root position in the image, N1, N3, N4, N5are position in between each fingers respectively,

which connect the finger tips and the palm root, distance from N1to N3N5is N1N2, make a parallel

straight line from N1N2midpoint with N3N5, this straight line cross with connecting line of finger

tips and palm root respectively atN11,N12,N13,N14. Similarly, can obtain corresponding points in the

right image asN’11,N’12,N’13,N’14, then 3D reconstruction algorithm can estimate the position of the

feature points in the world coordinate system, which provide as w

i

N1 , i1,2,..,5, and each finger tips position in the world coordinate are w

i

P ,i1,2,..,5.

The thumb length is summation of second and third knuckles length, the rest fingers length are three fingers length summation. Assume feature point of space between fingers position in the

world coordinate system as w

i

N , the finger width estimatedwf 0.7* N2wN3w , thenN2w,N3ware the space between index finger and middle finger and the space between middle finger and ring finger respectively.

AssumePw

6 as the palm root position,Ncw is the midpoint of w wN

N2 3 , then the palm length is

estimated to be w

c w p P N

w6 , the width is estimated to belp 4wf .

After the parameters of the human hand model is estimated out, the user's virtual hand model can be obtained by updating the relevant parameters of the pre constructed virtual hand template model.

Conclusion

In this paper, an improved hand modeling method is proposed, and a parameterized virtual hand model modeling method based on binocular stereo vision is presented in this paper. The method can automatically extract and recognize the feature points of finger tips from the open hand image, and then estimate the parameters of the hand model by using the calibrated binocular camera, realize the modeling of the user's hand. Experiment result shows that this method can quickly build a personalized virtual hand model in a certain precision range, which is easy to operate.

References

(6)

[2] Sun Hanqiu, Yuan Xiaobu, Baciu G., et a1. Direct virtual hand interface in robot assembly programming [J]. Journal of Visual Languages and Computing, 1999, (10): 55 68.

[3] Stenger B., Thayananthan A., Torr P H, et al. Model-based hand tracking using a hierarchical Bayesian filter [J]. IEEE transactions on pattern analysis and machine intelligence. 2006, 28(9): 1372-1384.

[4] H. Hu, X. Gao, J. Li, J. Wang, H. Liu, Calibrating human hand for teleoperating the hit/dlr hand, in: IEEE International Conference on Robotics and Automation, vol. 5, 2004, pp. 4571–4576.

[5] J. Lee, T. Kunii, Constraint-based hand animation, in: Models and Techniques in Computer Animation, Springer, Tokyo, 1993, pp. 110–127.

[6] P. Kakumanu, S. Makrogiannis, N. Bourbakis, "A Survey of Skin-Color Modeling and Detection Methods", Pattern Recognition 40, 2007, pp 1106-1122.

Figure

Figure 1. The hierarchy of the virtual hand skeleton model.
Figure 5. The feature extraction results.
Figure 6. The illumination of hand parameters estimation.

References

Related documents

Virus induced gene silencing (VIGS) for functional analysis of wheat genes involved in Zymoseptoria tritici susceptibility and resistance.. Wing-Sham Lee,

The charges for support services have been allocated in accordance with CIPFA Accounting guidelines, as a result reductions are reflected in the 2014/15 Probable Outturn and

The secondary aims of this study were to investigate factors influencing lower limb vascular assessment practices including levels of clinical experience and education,

Information on maternal characteristics (age, gravidity, parity, human immunodeflciency virus status, delivery type and complications) was abstracted, as were birth data on the

In this paper domain decomposition methods for radia- tive transfer problems including conductive heat trans- fer are treated. The paper focuses on semi-transparent materials,

Luo D, Bernard DG, Balk J, Hai H, Cui X (2012) The DUF59 family gene AE7 acts in the cytosolic iron-sulfur cluster assembly pathway to maintain nuclear genome integrity in

Therefore, this investigation evaluates the immuno-expression of VEGFR-3 in placental conducting and exchange villi from normotensive, preeclamptic and HIV+ African women,

Then the emphasis is put on the system recovery/reconfiguration sequence, including (1) when the recovery control should be started after the fault isolation; (2) the sequence