Query for sequences with segmentation

In this experiment I considered each sequence in the database as a query and seg- mented the sequence before searching in the indexing space. The segmentation of a query motion sequence can be done either manually by the user or automatically by the algorithm. In my experiment I automatically segmented the query sequences into motion segments by the PPCA segmentation algorithm. The query performance on 609 sequences is given in Table 6.3. A query example of soccer kicking motion sequence was shown in Figure 6.5. As compared to the query for simple sequences, there was no

Returns Percentage of Average query Segmentation per query behaviorally time (Sec.) time (Sec.)

similar motions

10 88.26% 7.26 6.06

Table 6.3: Summary of querying complex motions with segmentation.

significant change in the percentage of returned similar motions and the rank of queried motion in the returning list (see Table 6.3). Although the average query time did in- crease to 7.26 seconds compared to about 1 second in the single behavior motion query, most of the query time (6.06 seconds) was spent on the PPCA segmentation. PPCA segmentation with larger steps ∆ may shorten the segmentation time with no signifi- cant drops in query performance. In addition, a user can always choose to segment the motion manually before querying the database.

I ran my experiments in Matlab 7 on a Dell Inspiron Laptop, with 1.4GHz CPU and 512M physical memory. A more powerful computer and more efficient code imple- mentation may push the performance higher to make the motion segmentation.

6.5

Conclusions

I applied a data-driven, piecewise linear approach to modeling human motions at a behavioral level. Based on this modeling approach, I designed an indexing scheme for efficient retrieval of behaviorally similar motion sequences from a large motion database. I believe that my method is a step forward in the study of human-perceived similarities among human motions. My method breaks down long motion sequences into segments of single-behavior motions. It then extracts equal-length feature vectors from the distributions of poses in the motion segments and uses them as the modeling primitives in a newly parameterized space. By doing so I can encapsulate the essence

of motions in a very compact but effective data structure. This model is immune to spatio-temporal variation among similar motions and is able to group together similar motions with different styles. The process of model construction and data query is efficient and scales well with data size and dimensionality.

Although my method does not guarantee that the returns for any particular query are in an absolute right order, more similar motion sequences do rank higher than the othernot-so-similar motion sequences in most cases. I believe that my method not only provides an efficient way to organize and categorize a larger human motion database, but also can be used independently for motion retrieval tasks such as assisting the composition of animation sequences in video games.

Figure 6.5: A snapshot of a soccer motion query. The first sequence is the query. The other two sequences are the returning results.

Chapter 7

Segment-Based Human Motion Compression

Human motion data have been used in many research fields and applications, such as animating human-like computer characters in video games, driving avatars in virtual re- ality environments, and generating special effects in movies. In particular, online video games often use motion data to interactively control the game characters from a remote site across the internet. As more and more motion data become available, compressing motion data for compact storage and fast transmission becomes imperative.

In order to achieve a greater compression ratio while still being able to retain high fidelity to the original motion sequences, I propose a motion compression method that exploits both spatial and temporal coherences inherent in a human motion sequence. First, I segment a motion sequence into segments of simple motions. Poses from each motion segment lie near a space with low linear dimensionality. I then compress these motion segments individually by PCA approximation. I compute PCA from each mo- tion segment and approximate pose position vectors by their projections onto the space spanned by the principal components. This segment-based PCA compression typically needs fewer principal components than compression using global PCA on a whole se- quence to achieve similar distortion ratio. To take advantage of temporal coherence and further compress the PCA projections of the poses of each motion segment, I adap- tively select and store only the key frames from each motion segment and use them as the control points for the cubic spline interpolation in the principle component space. This motion sequence compression method is efficient and easy to implement, with a corresponding decompression process that is simple and fast as well.

The rest of the chapter is organized as follows. In Section 7.1 I give an overview of the proposed method. In Section 7.2 I develop an algorithm to compress motion segments by PCA. I then describe how to achieve further compression by selecting the key frames in Section 7.3. In Section 7.4 I provide a decompression algorithm. I present

the experimental results in Section 7.5 and finally conclude with discussions in Section 7.6.

7.1

Overview

Figure 7.1 shows a flow chart of the compression and decompression pipeline. An overview of each component in the compression/decompression process is given below.

Figure 7.1: A diagram of motion data compression.

Normalization. In order to achieve high compression performance, I apply a nor- malization procedure (already described as a key component in PLM modeling pipeline in Chapter 3) to convert all the poses from different coordinated systems onto a uni- versal, model-rooted homogeneous coordinate system, where all the translational and orientational effects are removed from the poses through appropriate transformations. I used three markers for normalization, namely, the markers at STRN, the left and right shoulders. These threenormalization markers are crucial to the quality of the full pose configuration. I compress these three special markers separately from the rest of the markers, which I call thenon-normalization markers.

Motion segmentation. I segment the normalized motion data sequence into subsequences of simple motions whose poses lie near a low-dimensional linear space.

Compression of segments by PCA. For each motion segment, I approximate the pose position vectors by their projections onto the space spanned by the leading principal components.

Key frame selection for spline interpolation. Given PCA projection data of each frame of a motion segment, I adaptively select and store only the key frames as control points, such that the spline interpolation of the rest of the frames yields an approximation error below a preset threshold.

Decompression. In decompression I use spline interpolation to recover the po- sitions of the normalization markers, as well as the PCA projections of the non-key frames for the non-normalization markers. I then reconstruct the positions of thenon- normalization markers in the normalized coordinate system.

Denormalization. Denormalization transforms all the normalized poses back to their original coordinate systems.

Since normalization and segmentation have been presented in Chapter 3, I will not discuss those components again in this chapter. Instead, I will describe in more detail the rest of the key components in the following sections.