Motion Analysis and Classification

The recorded 14-DOF of motion data from the five trials of all 15 objects gives seventy- five 14-DOF sampled motion functional data. With the classic PCA, the DOF of the hand is reduced from 14 to 3 by finding three principal components as principal hand postures and by representing the motion data using these principal postures. At this point, the motion data are reduced to seventy-five 3 DOF motion principal components. The entire set of PCs for every

time point of each grasping motion during the grasping procedure was stored, and these 3 DOF PC sets were kept for further analysis. For each PC set, the FPCA was applied to process the 75 functions to find a set of fPCs that can represent the basic functional dynamics of the temporal changes in these projected data. After the FPCA, each of the original seventy-five 14 DOF motion functions was represented using three independent variables.

The PCA is a good measure for extracting related empirical variables. Each orthogonal principal component linearly combines the empirical variables with the highest level of variance contribution on that dimension. Their level of variability correlates the variables that have a significant contribution on each component. Table 3.2 shows the components’ scores or coefficients for the entire motion. The first two components cover 83% of the overall variance. The coefficients in each component are a measure of the variability of the empirical variables on that component (see [44]). Values that fall further away from 0 indicate a large variance of that variable on that dimension, with the distance from 0 being proportional to the level of variance. In Table 3.2, these values are shown in bold typeface. The first component, PC 1, has most of its contributions from the MP joints of the hand’s fingers; contrastingly, the second component, PC 2, shows a higher contribution from the thumb’s joints. As expected, the variance from the entire set of grasps is mostly a combination of the fingers’ motion acting oppositely to the thumb. Lastly, in PC 3, the variability is given by the spread motion of the thumb and two other fingers, confirming this type of motion as the third most significant one.

The focus here is to analyze the entire set of grasps. However, the individual analysis of each object grasp showed similar distribution of the empirical variables. Most of the differences among objects were the level of significance of the thumb’s joints over the four fingers as well as the number of principal components necessary to cover the grasps’ variability range.

Once the principal postures of the grasping data have been extracted, analysis continues by evaluating the dynamics of these components. As explained before, the advantages of FPCA reside on the description of the dataset on a temporal graph, as a function of time. It relates the correlation of empirical variables throughout the whole motion instead of individual snapshots. FPCA was used to analyze the temporal functional dynamics of grasping motion by projecting the original sampled motion data to the vector corresponding to individual principal posture. In FPCA, B-spline functions were used as the function basis to create the functions from the sampled motion datasets because the observed grasping motion is not periodic. Figures 3.7 and 3.8 show the eigenfunctions and how they vary over the mean function during the entire raw datasets. In the top graphs, the black line refers to the mean function of the first principal component obtained from the PCA procedure; similarly, the bottom graphs show the second component’s mean function. The dashed and point-dashed blue lines describe the first (left graphs) and the second component (right graphs) variability over the mean based on derived eigenfunctions.

Because the first PC has most of its contributions from all the MP joints of the hand, they can be used as an indicator when all fingers stop adducting. Then, the segmentation of the PC set was performed automatically by finding the peak point in the first PC set. DTW was applied to the three most important segmented PC sets to align features among each five trials of the same object. The fPC scores can also be used as projections of the dynamic requirements for a motion. The functions were created using the B-spline basis from the segmented PC sets. FPCA was applied on these functions, and the projections yield the first fPCs with the corresponding scores that can be used to classify the motions regarding objects grasped. The result was clustered using

the three fPCs for the three most important principal components to reflect motion similarities with k-means.

Table 3.2. Principal component scores: entire motion.

Joint PC1 PC2 PC3 PC1 D1 MP 0.08 0.08 0.26 0.24 D1 PIP -0.13 -0.14 -0.44 0.15 D1-D2 MP 0.07 0.42 0.42 -0.07 D2 MP -0.34 -0.28 0.58 0.23 D2 PIP -0.09 0.65 0.11 0.14 D2-D3 MP -0.38 -0.13 0.00 0.16 D3 MP -0.19 0.06 0.00 0.49 D3 PIP 0.17 0.39 -0.34 0.37 D3-D4 MP -0.05 0.13 0.11 -0.21 D4 MP -0.07 0.18 -0.13 -0.26 D4 PIP 0.22 -0.21 -0.10 0.32 D4-D5 MP 0.68 -0.16 0.19 -0.04 D5 MP -0.35 0.07 -0.12 -0.37 D5 PIP 0.04 -0.03 0.03 -0.30 Percentage of variance 74.84% 8.07% 4.56% 3.67%

Figure 3.7. Plot of the first harmonics of the first and the second PC. Plot of the first harmonics of the first and the second PC showing the effects of adding and subtracting such principal component curve on the overall mean functions.

Figure 3.8. Plot of the second harmonics of the first and the second PC. Plot of the second harmonics of the first and the second PC showing the effects of adding and subtracting such principal component curve on the overall mean functions.

Overall, the k-mean clustering algorithm was implemented on functional principal component scores in three dimensions. The plot of k equals 5, 6, and 7 (see Figure 3.11). With further classification, which means increasing the k in clustering algorithm, except for group “Thumb-4 Finger,” “Thumb-3 Finger,” and group “Tripod” and “Thumb-2 Finger,” all other grasping motions have been clustered into separate classes. In the datasets, there are 15 objects, that is, five individual grasping trials for each object in total; however, when class number exceeds eight, most of them have some trials among five been clustered into other classes. However, for three to seven classes, the analysis apparently shows very clear groups among all grasping motions. Also, PCA approach can be used to categorize these grasps. First, final poses were found manually. One of those frames during the subject is holding the object firmly was chosen. After that, according to more than 85% of the variance covered by the first three PCs, scores of these PCs were used as parameters to cluster all 75 trials of grasping motion. However, the clustering result was not very promising. Most types of grasp will form new branches in the

hierarchy when the number of classes increases. The result has a different hierarchy than the Cutkosky taxonomy. Instead of two dimensions, which are the complexity and the power needed to finish the grasp, the result distinguished different types of grasping, or object, from the difference in the shape of hand during the grasping procedure without segmenting these motions. The Cutkosky taxonomy is shown in Figure 3.9, and the hierarchy developed in this paper is shown in Figure 3.10. In short, this hybrid approach is from a human hand point of view. The resultant clusters are shown in Table 3.3.

Figure 3.10. Grasp taxonomy according to the classification result.

The clustering results shown in Figure 3.11 have various coloring because the initial centroids are randomly chosen. In each plot, instances having the same coloring circles are in the same class. Only the results of k with values 5, 6, and 7 are displayed. The result according to other numbers of k can be found in Table 3.3.