School of Electrical Engineering and Computer Science
February 21 and February 28 of Spring 20
3.5 Parametric Equations in Gesture Recognition
Recognizing gestures, whether they be 2D using a stylus on a Tablet PC or 3D using a Nintendo Wiimote, is an important part of many applications in pen computing and video games. There are many different approaches to performing gesture recognition that make use of a variety of different algorithms in computer science. One common method is to create templates for each gesture and then test any incoming gesture against the templates. If the incoming gesture closely matches a particular template, then it is classified as that gesture. There are many different ways to create gesture templates. One approach is to create a collection of features, information that describes a particular gesture. These features are primarily based on computing statistical or geometric quantities from the data that represents the gesture. To ensure accurate recognition, these features must be good discriminators or provide quantities that can be used to tell the difference between one gesture from another.
As it turns out, the data we need to compute on for recognizing gestures can be described using parametric equations. For example, a Nintendo Wiimote provides a series of acceleration values in thex,y, andzdirections through time. In another example, a stylus on a Tablet PC provides the location of the stylus point on the surface of the screen as points in x and y through time. Recall that parametric equations can be used to describe a particle in motion in 2D or 3D through time. Thus, we get a collection of points from these devices that we can use to compute the features we need for gesture recognition.
3.5.1
Arc Length
Of the many possible features that one could compute to create a gesture template, one of the most common is arc length, the length of a curve. We can define arc length in terms of parametric equations. Thus, if we have a curve C defined by x = f(t) and y = g(t), then its arc length can be computed as L= Z β α v u u u t dx dt !2 + dy dt !2 dt. (3.15)
3.5. PARAMETRIC EQUATIONS IN GESTURE RECOGNITION
For example, consider the parametric equations x = 3 sin(3t) and y = 3 cos(3t), where 0 ≤ t ≤ 2π. To compute the arc length of this curve, we first find the derivatives of x and y,
dx
dt = 9 cos(3t) (3.16)
dy
dt =−9 sin(3t). (3.17)
Plugging in these derivatives into the arc length formula we obtain
L = Z 2π 0 q (9 cos(3t))2+ (−9 sin(3t))2dt (3.18) = Z 2π 0 q 81 cos2(3t) + 81 sin2(3t)dt (3.19) = Z 2π 0 √ 81dt (3.20) = Z 2π 0 9dt (3.21) = 18π. (3.22)
Unfortunately, when we are dealing with gesture recognition, the input devices we use do not provide functions of time. Rather, they provide points through time. So, when calculating arc length in gesture recognition applica- tions, we need to compute an approximation. Suppose we are recognizing pen gestures. A pen ink stroke is defined as a sequence of points s = p1p2...pnin
thexy-plane wherepi= (xi, yi),1≤i≤n,p1is the pen-down point,pn
is the pen-up point, and n is the number of points in the stroke. Thus, with these xypoints we have an approximation to the curve generated by the pen stroke. The largernis, the smaller the distance between the points and the better the approximation will be. To approximate arc length for anys, we compute
ˆ L= n X i=2 kpi−pi−1k (3.23) wherekpi−pi−1k = q (xi−xi−1) 2+ (y i−yi−1)
2. Asngoes to infinity, ˆLgoes to L. Note that this formula works for 3D points as well, the z component simply needs to be added to the distance calculation.
3.5.2
Linear Classification
We can use arc length and other features, such as aspect ratio, point and angle histograms, to create templates for each gesture in the gesture recognition process. Once we have the features calculated, we can train a machine to learn an algorithm, so it can tell us how incoming gestures should be classified. There are a wide variety of machine learning algorithms used in gesture recognition. In this chapter, we focus on one of the simplest, yet powerful approaches, the linear classifier.
Given a feature vector ~,associated with a given gesture g in a gesture
alphabet C, a linear evaluation function is derived over the features. The linear evaluation function is given as
gc =wc0+
F X
i=1
wcifi (3.24)
where 0 ≤ c < C, F is the number of features, and wci are the weights
for each feature associated with each gesture in C. The classification of the correct gestureg is the c that maximizes gc.
Training of this classifier is done by finding the weights wci from the
gesture samples. First, a feature vector meanf~c is calculated using
¯ fci= 1 Ec Ec−1 X e=0 fcei (3.25)
where fcei is the ith feature of the eth example of gesture c and 0 ≤ e < Ec where Ec is the number of training samples for gesture c. The sample
covariance matrix for gesture cis
Σcij = Ec−1
X
e=0
(fcei−f¯ci)(fcej−f¯cj). (3.26)
The Σcij are averaged to create a common covariance matrix
Σij = C−1 P c=0Σcij −C+CP−1 c=0 Ec . (3.27)
Parametric Equations for Fun and Profit 3-14
The inversion of Σij then lets us calculate the appropriate weights for the linear evaluation functions,
3.5. PARAMETRIC EQUATIONS IN GESTURE RECOGNITION wcj = F X i=1 (Σ−1) ijf¯ci (3.28) and wc0 =− 1 2 F X i=1 wcif¯ci (3.29) where 1≤j ≤F.
3.5.3
Recognizing Gestures
Figure3.10: The 25 gestures used in the 3D gesture database.
To test the gesture recognition algorithms, the Interactive Systems and User Experience Lab at UCF created a 3D gesture database. The 3D gestures were collected using a Nintendo Wii Remote. A total of 8500 gestures were collected spanning 25 different gestures (see Figure3.10). We tested the
linear classifier described above on this database, by examining how many training samples per gesture would aff ect the overall gesture recognition accuracy. A total of 5, 10 and 15 samples were used to create templates for each gesture. The algorithm achieved 93.5%, 96.3%, and 98.5% accuracy, using 5, 10, and 15 training samples per gesture respectively.
3.6
Conclusion
In this chapter, we have examined parametric equations and how they can be used to represent interesting curves in 2D and 3D space. We have ex- plored how these equations are used to create mathematical art and how we can use Calculus on parametric equations to compute arc length. Arc length is an important feature that is used to describe gestures for gesture recognition. These features are used in machine learning algorithms to cre- ate classifiers that take the feature data and create templates used in the recognition process. Finally, we showed a gesture recognition example for recognizing Nintendo Wiimote gestures. These gestures can be used to cre- ate more naturally expressive interfaces in video games.