Human motion synthesis from captured data.

7311387

All rights reserved

INFORMATION TO ALL USERS

The q u ality of this re p ro d u ctio n is d e p e n d e n t upon the q u ality of the copy subm itted.

In the unlikely e v e n t that the a u th o r did not send a c o m p le te m anuscript

and there are missing pages, these will be n o te d . Also, if m ateria! had to be rem oved,

a n o te will in d ica te the deletion.

Published by ProQuest LLC (2017). C op yrig ht of the Dissertation is held by the Author.

All rights reserved.

This work is protected against unauthorized copying under Title 17, United States C o d e

M icroform Edition © ProQuest LLC.

ProQuest LLC.

789 East Eisenhower Parkway

P.O. Box 1346

: i

it

Human Motion Synthesis from Captured Data

L. Molina Tanco

Submitted for the Degree of

Doctor of Philosophy

from the

University of Surrey

UniS

Centre for Vision, Speech and Signal Processing

School of Electronics and Physical Sciences

University of Surrey

Guildford, Surrey GU2 7XH, U.K.

September 2002

Summary

A n im a tio n o f h u m a n m o tio n is o n e o f th e m o s t c h a lle n g in g to p ic s in c o m p u te r g ra p h ic s. T h is is d u e to th e la rg e n u m b e r o f d eg re e s o f fre e d o m o f th e b o d y a n d to o u r a b ility to d e te c t u n n a t u r a l m o tio n . K e y fra m in g a n d in te r p o la tio n re m a in s t h e fo rm o f a n im a tio n t h a t is p re fe r re d b y m o s t a n im a to r s b e c a u s e o f th e c o n tro l a n d fle x ib ility it p ro v id e s. H ow ever th is is a la b o u r in te n s iv e p ro c e ss t h a t re q u ire s sk ills t h a t ta k e y e a rs to a c q u ire . H u m a n m o tio n c a p tu r e te c h n iq u e s p ro v id e a c c u r a te m e a s u re m e n t o f th e m o tio n o f a p e rfo rm e r t h a t c a n b e m a p p e d o n to a n a n im a te d c h a r a c te r to p ro v id e s trik in g ly n a t u r a l a n im a tio n . T h is ra ise s th e p ro b le m o f h o w to allow a n a n im a to r to m o d ify c a p tu r e d m o v e m e n t to p r o d u c e a d e s ire d a n im a tio n w h ils t p re s e rv in g th e n a t u r a l q u a lity . T h is th e s is in tro d u c e s a n e w a p p ro a c h to t h e a n im a tio n o f h u m a n m o tio n b a s e d o n c o m b in in g th e fle x ib ility o f k e y fra m in g w ith th e v is u a l q u a lity o f m o tio n c a p tu r e d a ta . I n p a r tic u la r it a d d re s s e s th e p ro b le m o f s y n th e s is in g n a t u r a l inbetw een m o tio n for s p a rs e k e y fra m e s. T h is th e s is p ro p o s e s to o b ta in th is m o tio n b y s a m p lin g h ig h q u a lity h u m a n m o tio n c a p tu r e d a ta .

T h e p r o b le m o f k e y fra m e in te r p o la tio n is f o r m u la te d as a s e a rc h p r o b le m in a g ra p h . T h is p re s e n ts tw o d ifficu lties: T h e c o m p le x ity o f th e s e a rc h m a k e s it im p r a c tic a l fo r th e la rg e d a ta b a s e s o f m o tio n c a p tu r e r e q u ir e d to m o d e l h u m a n m o tio n . T h e se c o n d d iffic u lty is t h a t th e g lo b a l te m p o r a l s t r u c t u r e in th e d a t a m a y n o t b e p re s e rv e d in th e se a rc h .

T o a d d re s s th e s e d ifficu ltie s th is th e s is in tr o d u c e s a la y e re d fra m e w o rk t h a t b o t h r e ­ d u c e s th e c o m p le x ity o f th e s e a rc h a n d p re se rv e s th e g lo b a l te m p o r a l s tr u c tu r e o f th e d a ta . T h e first la y e r is a s im p lific a tio n o f th e g r a p h o b ta in e d b y c lu s te rin g m e th o d s . T h is la y e r e n a b le s efficient p la n n in g o f th e s e a rc h fo r a p a t h b e tw e e n s t a r t a n d e n d k e y fra m e s. T h e se c o n d la y e r d ir e c tly s a m p le s se g m e n ts o f th e o rig in a l m o tio n d a t a to s y n th e s is e r e a lis tic in b e tw e e n m o tio n for th e k e y fra m e s. A n u m b e r o f a d d itio n a l c o n ­ tr ib u tio n s a re m a d e in c lu d in g n ovel r e p r e s e n ta tio n s fo r h u m a n m o tio n , p o se s im ila r ity c o st fu n c tio n s , d y n a m ic p r o g r a m m in g a lg o rith m s fo r efficient s e a rc h a n d q u a n tita tiv e e v a lu a tio n m e th o d s . R e s u lts o f re a lis tic in b e tw e e n m o tio n a r e p r e s e n te d w ith d a ta b a s e s o f u p to 120 se q u e n c e s (35000 fra m e s ).

K e y w o r d s : H u m a n M o tio n S y n th e s is , M o tio n C a p tu r e , C h a r a c te r A n im a tio n , G r a p h S e a rc h , C lu s te rin g , U n s u p e rv is e d L e a rn in g , M a rk o v M o d e ls, D y n a m ic P ro g ra m m in g .

Acknowledgements

M a k in g th is th e s is w o u ld n o t h a v e b e e n p o ssib le w ith o u t th e a d v ic e , c o m m o n sen se, a n d p o s itiv e n e s s o f m y s u p e r v is o r D r. A d r ia n H ilto n . I n e v e ry o n e o f th e w eek ly m e e tin g s t h a t we h a d fo r th e la s t th r e e y e a rs I w o u ld leave h is office h a p p ie r t h a n I w as b efo re: M o re c o n fid e n t, w ith m o re id e a s a n d m o re energy. H is c o m m itm e n t a n d c o rd ia lity to h is s tu d e n ts h a v e c o n v in c e d m e t h a t h e re g a rd s s u p e rv is in g as b o th th e m o s t i m p o r ta n t a n d th e m o st e n jo y a b le p a r t o f h is jo b . I c a n n o t b u t re c o m m e n d h im to a n y p ro s p e c tiv e P h D s tu d e n ts !

I w o u ld a lso lik e to th a n k m y fo rm e r s u p e rv is o r, D r. A n d re w S to d d a r t, a s w ell as J e a n J o h n s o n -J o n e s , P ro fe s s o r J o h n Illin g w o rth a n d P ro fe s s o r J a n e t L a n s d a le , fo r co n c e iv in g th e p ro je c t “H u m a n M o tio n S y n th e s is for V ir tu a l P e rfo rm a n c e ” , w h ic h s u p p o r te d m y re se a rc h . I a m e q u a lly g ra te fu l to th e E n g in e e rin g a n d P h y s ic a l S cien ces R e s e a rc h C o u n c il for p ro v id in g th e fu n d in g for it.

D u rin g m y re s e a rc h I h av e u s e d so ftw a re w r itte n b y o th e r p e o p le , in s id e a n d o u ts id e o u r C e n tr e fo r V isio n , S p e e c h a n d S ig n a l P ro c e ssin g . W ith o u t th e ir g e n e ro s ity I w o u ld n o t h av e b e e n a b le to fin ish th is w o rk o n tim e! T h a n k s to D o rie n v a n d e B e lt, R o b e r t C rid a , B ill C h r is tm a s , A d r ia n H ilto n , P a r is L y ritis , K ie ro n M esse r, R a d e k M a rik , G eo rg e M a ta s , A n d re w S to d d a r t, a n d a ll th e p e o p le t h a t h av e c o n tr ib u te d to th e g o o d o ld A M M A so ftw a re lib ra ry , e sp e c ia lly C h a rle s G a la m b o s, w h o h a s m a in ta in e d i t fo r y ears a n d h e lp e d m e in in n u m e ra b le o cc a sio n s, as h a v e G ra e m e W ilfo rd a n d S im o n A lc o tt, o u r s y s te m a d m in is tr a to r s . I a m also g r a te f u l to A a ro n H e r tz m a n n a n d M a tth e w B r a n d for m a k in g p u b lic ly a v a ila b le m o sey, a p iece o f so ftw a re t h a t h a s b e e n u s e d to g e n e r a te m a n y fig u res in th is th e s is . I a m also v e ry g ra te fu l to M a tth e w B r a n d fo r le ttin g m e v is it h is la b a n d p a tie n tly e x p la in in g to m e h is w o rk o n h u m a n m o tio n s y n th e sis. I a m v ery g ra te fu l to th e b e s t w o rk m a te s a n d frie n d s in t h e w o rld : B ill a n d R a tn a . I a m n o t s u re I w o u ld h a v e b e e n a b le to su c c e e d w ith o u t th e ir g e n e ro u s frie n d s h ip , h e lp , a d v ic e a n d s p a r e b e d ro o m s! S h e n k a , A n ju , R ite s h , A lla n a n d N ili h a v e also b e e n e x tre m e ly k in d a n d p a tie n t w ith m e, e sp e c ia lly d u r in g th e la s t m o n th s o f m y P h D . I m u s t also t h a n k m y c o u sin N o rb e r to , w h o firs t k n e w a b o u t th e P h D o p p o r tu n ity a n d s e n t to m e a n e le c tro n ic m a il t h a t c h a n g e d m y life. M y w a rm e s t th a n k s go also to m y frie n d s in th e C e n tr e fo r V isio n , S p e e c h a n d S ig n a l P ro c e s s in g , a n d th o s e in Im a g in e e r S y ste m s, for m a k in g th e m b o th su c h s tim u la tin g a n d e n jo y a b le p la c e s to w o rk in: A lla n , A lis ta ir, B a r b a r a , D a n , D o rie n , E le n i, E n g - J o n , F ra n k , G o rd o n , J o e l, J o n a th a n , M a n o lo , M a ria , P h il, R a c h e l, R o b e r ta , S a n je e v , T a so s I a n d T a so s II. I h o p e to see m u ch m o re o f you!

I a m e x tre m e ly g ra te fu l to m y f a th e r fo r h e lp in g m e a n d s u p p o r tin g m e d u r in g a ll th e s e lo n g s tu d e n t y ea rs, fo r a lw ay s h a v in g m y e d u c a tio n a s h is first p rio rity . I h o p e h e th in k s it h a s b e e n w o rth -w h ile ! E v e ry o n e w h o k n o w s m e also k n o w s t h a t I w o u ld n e v e r h av e b e e n a b le d o th is w ith o u t m y d e a r Is a b e l. H e r co m p a n y , e n c o u ra g e m e n t a n d in fin ite p a tie n c e over th e s e y e a rs h av e g iv e n m e th e s tr e n g th to fin ish . I a m lo o k in g fo rw a rd to e m b ra c e g ro w n -u p life w ith h e r - th a n k s fo r w a itin g fo r me!

I would like to dedicate this thesis to m y m other, fo r teaching m e how to read, add and substract, fo r always stim u la tin g m y curiosity and always answ ering all m y questions, fo r m aking m e who I am.

Contents

1 I n t r o d u c t i o n 1 1.1 M o tiv a tio n ... 1 1.2 M e t h o d o l o g y ... 4 1.3 F r a m e w o r k ... 4 1.4 S t r u c t u r e ... 5 2 L i t e r a t u r e R e v i e w 7 2.1 T r a d itio n a l C o m p u te r A n i m a t i o n ... 8 2.1.1 I n t e r p o l a t i o n ... 8 2.1.2 In v e rse k i n e m a t i c s ... 9 2.2 M o tio n c a p tu r e s y s t e m s ... 10 2.3 S im u la tio n o f c h a r a c te r m o t i o n ... 11 2.3.1 D y n a m ic s i m u l a t i o n ... 12 2.4 R e u se o f m o tio n c a p tu r e d a t a ... 13 2.4.1 S ig n a l p r o c e s s i n g ... 14 2.4.2 I n t e r p o l a t i o n ... 15 2.4.3 C o n s tr a in t b a s e d e d itin g m e th o d s ... 15 2.4.4 S ta tis tic a l m o d e ls o f m o t i o n ... 17 2.5 S u m m a r y ... 21 3 T h e M o t i o n S y n t h e s i s P r o b l e m 2 3 3.1 M o tio n s y n th e s is b y k e y fra m e i n t e r p o l a t i o n ... 23 3.2 K e y fra m in g o n a d a ta b a s e o f m o tio n c a p tu r e d a t a ... 24 3.3 A g r a p h se a rc h f o r m u l a t i o n ... 25 3.4 D ire c t s o l u t i o n ... 25

3.5 A s ta tis tic a l fra m e w o rk fo r h u m a n m o tio n s y n th e s is ... 26

v

vi

Contents

4 A F r a m e w o r k f o r M o t i o n S y n t h e s i s 2 9

4.1 D a t a r e p r e s e n ta tio n ... 30

4.2 O v e r v i e w ... 31

4.3 A c o m p a c t r e p r e s e n ta tio n fo r m o tio n c a p tu r e d a t a ... 34

4.4 L evel one: M o d e llin g t h e te m p o r a l s tr u c t u r e o f th e m o tio n d a ta b a s e . . 36

4.4.1 M a rk o v m o d e ls a n d v e c to r q u a n t i s a t i o n ... 38

4 .4.2 E s tim a tio n o f th e tr a n s itio n p r o b a b i l i t i e s ... 39

4.5 L evel tw o: E n fo rc in g lo c a l s m o o th n e s s in th e d a t a ... 41

4.5.1 M o tio n s e g m e n t s ... 41

4 .5.2 D is c re te H id d e n M a rk o v m o d e ls fo r se g m e n t o p tim is a tio n . . . . 42

4.6 R e a lis tic M o tio n S y n t h e s i s ... 45

4.6.1 O b ta in in g s t a r t a n d e n d m o tio n s e g m e n t s ... 46

4.6 .2 O b ta in in g th e m o s t lik e ly se q u e n c e o f c l u s t e r s ... 46

4.6.3 S o lv in g for th e o p tim a l se q u e n c e o f m o tio n s e g m e n t s ... 47

4.6 .4 B le n d in g th e fin a l a n i m a t i o n ... 49 4 .7 A S im p le E x a m p l e ... 52 4.7.1 D im e n s io n a lity r e d u c t i o n ... 52 4 .7.2 T h e ch o ice o f re fe re n c e c o o rd in a te s y s t e m ... 54 4 .7.3 M o d e l c o n s tr u c tio n a n d m o d e l o rd e r s e l e c t i o n ... 56 4 .7 .4 S y n th e tic m o tio n g e n e r a t i o n ... 58 4.8 R e s u l t s ... 59 4.9 E v a lu a tio n o f s y n th e tic m o t i o n s ... 62

4.9.1 Q u a n tita tiv e e v a lu a tio n fo r d a ta b a s e D B 2 ... 66

4.9.2 Q u a lita tiv e e v a lu a tio n fo r d a ta b a s e D B 2 ... 67

4.10 F ra m e w o rk c o m p l e x i t y ... 68

4.11 D i s c u s s i o n ... 70

Contents

vii

5 M o t i o n S y n t h e s i s f o r L a r g e D a t a b a s e s 7 3

5.1 R o ta tio n a l v e rsu s p o s itio n a l d a t a r e p r e s e n t a t i o n ... 74

5.1.1 E x p e r im e n ts ... 77

5.1.2 C o n c l u s i o n ... 77

5.2 I n tr o d u c in g a n e x p lic it c o st fu n c tio n in th e f r a m e w o r k ... 78

5.2.1 T h e d is ta n c e b e tw e e n jo in t p o s itio n s as a c o st f u n c t i o n ... 81

5.2.2 T h e f o r m a tio n o f m o tio n s e g m e n t s ... 82

5.2.3 E n s u r in g c o n tin u o u s m o tio n th r o u g h re -in d e x in g m o tio n s e g m e n ts 83 5 .2.4 T h e E x p lic it V ite r b i a l g o r i t h m ... 85 5.2.5 E x p e r im e n ts ... 86 5.2.6 M o d ific a tio n s to c o m p l e x i t y ... 88 5 .2 .7 S u m m a r y ... 90 5.3 I n c o r p o r a tin g v e lo c itie s in to th e F r a m e w o r k ... 90 5.3.1 In c lu s io n o f v e lo c ity in t h e s t a t e v e c t o r ... 91 5.3.2 A n e w c o st f u n c tio n t h a t in c lu d e s v e l o c i t y ... 93 5.3.3 E x p e r im e n ts ... 97 5.3.4 S u m m a r y ... 99 5.4 E x te n d in g level o n e o f th e m o d e l ... 100 5.4.1 C lu s te r in g w ith m ix tu r e s o f G a u s s i a n s ... 100 5.5 F in a l re s u lts a n d e v a l u a t i o n ... 105 5.5.1 C o m p a r a tiv e e v a lu a tio n o f f r a m e w o r k s ... 106 5.5.2 M o tio n s y n th e s is fro m la rg e d a t a b a s e s ... 107 5.5.3 C o n c l u s i o n ... 108 5.6 S u m m a r y ...110 6 C o n c l u s i o n s , D i s c u s s i o n a n d F u t u r e W o r k 1 1 3 6.1 S u m m a r y ... 113 6.2 C o n c l u s i o n s ... 115 6.3 D i s c u s s i o n ... 117 6.3.1 G e n e ra l d i s c u s s i o n ... 117 6.3.2 R e c e n t w o r k ... 118 6.4 F u tu r e W o rk ...119

viii

Contents

A T h e R o t a t i o n v e c t o r a n d t h e d i s t a n c e b e t w e e n r o t a t i o n s 1 2 3 A .l D is ta n c e b e tw e e n s k e le to n p o se s ...124 B M o t i o n C a p t u r e D a t a b a s e s 1 2 9 B .l D a ta b a s e D B 1 ... 129 B .2 D a ta b a s e D B 2 ... 130 B .2.1 K e y f r a m e s ... 133 B .2 .2 R e s u lts ...133 B .3 D a ta b a s e D B 3 ... 134 B .4 D a ta b a s e D B 4 ... 135 B .4.1 K e y f r a m e s ... 136 B .4 .2 R e s u lts ... 137 B .5 D a ta b a s e D B 5 ... 139 i C D i j k s t r a ’s A l g o r i t h m 1 4 1

1.1 S p a rs e k e y f r a m e s ... 3

1.2 T h e P r o je c t “S y n th e s is in g H u m a n M o tio n for V ir tu a l P e rfo rm a n c e ” . . 6

2.1 S ta tis tic a l m o d e ls fo r m o tio n s y n t h e s i s ... 18

4.1 A n a r tic u la te d s k e le to n ... 30

4.2 O v e rv ie w o f th e m o d e llin g c o n s tr u c tio n ... 32

4.3 O v e rv ie w o f t h e s y n th e s is p r o c e s s ... 33

4 .4 A ty p ic a l sk e le to n u s e d fo r m o tio n c a p t u r e ... 34

4.5 S ta n d a r d K -m e a n s a l g o r i t h m ... 40

4.6 E s tim a tio n o f t r a n s itio n p r o b a b i l i t i e s ... 41

4.7 D ia g r a m o f t h e d is c re te h id d e n M a rk o v m o d e l in level t w o ... 45

4.8 U sin g level o n e fo r s y n t h e s i s ... 46

4.9 T h e D ir e c te d V ite r b i A l g o r i t h m ... 49

4.10 T h e o p e r a tio n o f D ire c te d V ite r b i a l g o r i t h m ... 50

4.11 A n g le b e tw e e n d ire c tio n a n d o r i e n t a t i o n ... 51

4.12 D a ta b a s e D B 1 ... 53

4.13 D is tr ib u tio n o f v a r ia tio n fo r D B 1 ... 53

4.1 4 D B 1 p r o je c te d to th e p r in c ip a l s u b s p a c e ... 55

4.15 Q u a n tis a tio n e rro rs fo r D B 1 ... 57

4.16 V a lid ity in d e x fo r D B 1 ... 58 4.17 I llu s tr a tio n o f M a rk o v c h a in m o d e l fo r D B 1 ... 59 4.18 E x a m p le k e y fra m e s fo r D B 1 ... 59 4.19 I llu s tr a tio n o f th e s y n th e s is p ro c e ss fo r d a ta b a s e D B 1 . . ! ... 60 4.20 S y n th e s is e x a m p le fo r D B 1 ... 61

List of Figures

ix

X

List of Figures

4.21 V a lid ity in d e x fo r D B 2 ... 61 4.22 S y n th e s is e x a m p le for D B 2 - p rio r to b le n d in g (1) 62 4.23 S y n th e s is e x a m p le fo r D B 2 - a f te r b le n d in g ( 1 ) ... 63 4.24 S y n th e s is e x a m p le fo r D B 2 - p r io r to b le n d in g (2) 63 4.25 S y n th e s is e x a m p le fo r D B 2 - a f te r b le n d in g ( 2 ) ... 64

4.26 Q u a n tita tiv e e v a lu a tio n - b a sic f r a m e w o r k ... 67

4.27 E x a m p le o f b a d r e s u lt - b a sic f r a m e w o r k ... 72

5.1 B e h a v io u r o f r o ta tio n a l p a r a m e t e r i s a t i o n ... 74

5.2 B e h a v io u r o f p o s itio n a l p a r a m e te ris a tio n ... 76

5.3 D B 3 in th e p r in c ip a l s u b s p a c e - r o ta tio n a l p a r a m e t e r i s a t i o n ... 78

5.4 D B 3 in th e p r in c ip a l s u b s p a c e - p o s itio n a l p a r a m e t e r i s a t i o n ... 79

5.5 S y n th e s is w ith r o ta tio n a l vs. s y n th e s is w ith p o s itio n a l p a r a m e te r is a tio n 80 5.6 B e h a v io u r o f D ire c te d V ite r b i a l g o r i t h m ... 81

5.7 E x p lic it c o st fu n c tio n in t h e tr e llis d i a g r a m ... 83

5.8 E x a m p le o f n o n -re p r e s e n ta b le c o n fig u ra tio n - b a sic fra m e w o rk (1) . . . . 84

5.9 E x a m p le o f n o n -re p re s e n ta b le c o n fig u ra tio n - b a s ic fra m e w o rk (2) . . . . 84

5.10 E x a m p le o f re -in d e x in g for n o n - re p re s e n ta b le c o n fig u ra tio n s - e x te n d e d fra m e w o rk ( 1 ) ... 85

5.11 E x a m p le o f re -in d e x in g fo r n o n -re p re s e n ta b le c o n fig u ra tio n s - e x te n d e d fra m e w o rk (2) . ... 85

5.12 T h e E x p lic it V ite r b i A l g o r i t h m ... 87

5.13 S y n th e s is e x a m p le w ith b a s ic f r a m e w o r k ... 88

5.14 S y n th e s is e x a m p le w ith e x te n d e d f r a m e w o r k ... 89

5.15 I llu s tr a tio n o f re q u ire m e n ts for a d y n a m ic c o st f u n c t i o n ... 94

5.16 A H e rm ite p o l y n o m i a l ... 95

5.17 I llu s tr a tio n o f d y n a m ic c o st f u n c t i o n ... 97

5.18 D y n a m ic vs. s ta tic c o st f u n c tio n - a c c u m u la te d j e r k ... 98

5.19 D y n a m ic vs. s ta tic c o st fu n c tio n - a v e ra g e excess o f j e r k ... 98

5.20 D y n a m ic vs. s ta tic c o st f u n c tio n - a v e ra g e n u m b e r o f j u m p s ... 99

5.21 E x a m p le o f b a d r e s u lt - C lu s te rin g u s in g G a u s s ia n m i x t u r e s ... 104

5.22 I llu s tr a tio n o f p o in t a s s ig n m e n ts in c lu s te rin g w ith G a u s s ia n m ix tu re s . 105 5.23 V a lid ity in d e x fo r D B 2 - p o s itio n a l p a r a m e t e r i s a t i o n ...107

List of Figures

xi

5.24 Q u a n tita tiv e e v a lu a tio n - e x te n d e d f r a m e w o r k ... 108

5.25 Q u a n tita tiv e e v a lu a tio n - e x te n d e d vs. b a sic f r a m e w o r k ... 109

5.26 S y n th e s is e x a m p le fo r D B 4 - p rio r to b le n d in g (1) ... 110 5.27 S y n th e s is e x a m p le fo r D B 4 - a f te r b le n d in g (1) 110 5.28 S y n th e s is e x a m p le fo r D B 4 - p rio r to b le n d in g ( 2 ) ... I l l 5.29 S y n th e s is e x a m p le fo r D B 4 - a f te r b le n d in g (2) I l l 6.1 A c o m p le te k e y fra m e -b a s e d a n im a tio n s y s t e m ...121 A .l O n e h a n d e d h a n d s t a n d s e q u e n c e ... 126

A. 2 I llu s tr a tio n o f a p p r o x im a te r o ta tio n a l d i s t a n c e ...127

B . l M o tio n e x a m p le O neH andedH andstand ( D B 1 ) ... 129

B .2 M o tio n e x a m p le W alkT rip ( D B 1 ) ... 129

B .3 M o tio n e x a m p le W alkT rip r o t a t e d (D B 1) ...130

B .4 M o tio n e x a m p le F a stR u n 2 F a stW a lk (D B 2) 130 B .5 M o tio n e x a m p le F a stW a lk2 R u n ( D B 2 ) ... 130 B .6 M o tio n e x a m p le G etU p2 ( D B 2 ) ...130 B .7 M o tio n e x a m p le L ieD o w n S (D B 2) ... 131 B .8 M o tio n e x a m p le Sh o t-B a ckS ( D B 2 ) ...131 B .9 M o tio n e x a m p le SitOG ( D B 2 ) ...131 B .1 0 M o tio n e x a m p le S i t l 7 ( D B 2 ) ...131 B . l l M o tio n e x a m p le S n e a k -S itW a lk ( D B 2 ) ... 131 B .1 2 M o tio n e x a m p le Sneak- W a lkS it ( D B 2 ) ... 132 B .13 M o tio n e x a m p le S p r in t ( D B 2 ) ...132 B .1 4 M o tio n e x a m p le S ta n d f ( D B 2 ) ... 132 B .15 M o tio n e x a m p le S ta n d 7 ( D B 2 ) ... 132 B .16 M o tio n e x a m p le W a lk2 5 -fsllll-l ( D B 2 ) ... 132 B .1 7 K e y fra m e s fo r d a ta b a s e D B 2 ... 133

B .1 8 S y n th e tic m o tio n fro m S it to F lat flo o r (D B 2 )... 134

B .1 9 S y n th e tic m o tio n fro m W alk R . Foot to L ie R ig h t (D B 2 )... 134

B .20 M o tio n e x a m p le F a stW a lk2 R u n ( D B 3 ) ... 135

xii

List of Figures

B .22 M o tio n e x a m p le S ta n d s ( D B 3 ) ...135

B .23 M o tio n e x a m p le W alk25-fsllll-l ( D B 3 ) ...135

B .2 4 K e y fra m e s fo r d a ta b a s e D B 4 ...137

B .25 S y n th e tic m o tio n fro m W alk R . Foot to S it ( D B 4 . ) ... 137

B .26 S y n th e tic m o tio n fro m S p rin t to F la t flo o r (D B 4 )... 138

B .27 S y n th e tic m o tio n fro m S rp in t to B a llet pose (D B 4 )... 138

B .28 S y n th e tic m o tio n fro m B a llet pose to L ie left (D B 4 ). ... 138

4.1 Q u a lita tiv e e v a lu a tio n - b a sic f r a m e w o r k ... 68

4.2 C o m p le x ity o f m o d e l c o n s tr u c tio n - b a sic f r a m e w o r k ... 69

4.3 C o m p le x ity o f s y n th e s is - b a sic f r a m e w o r k ... 70

5.1 C o m p le x ity o f m o d e l c o n s tr u c tio n - e x te n d e d f r a m e w o r k ... 89

5.2 C o m p le x ity o f s y n th e s is - e x te n d e d f r a m e w o r k ... 90

List of Tables

List of Symbols

(j) F ra m e in a m o tio n seq u en ce .

if) M o tio n se g m e n t, i.e. fra m e s u b s e t in a m o tio n se q u e n c e $ M o tio n seq u en ce.

ip I n s ta n ta n e o u s v e lo c ity in a m o tio n se q u e n c e

C C o st o f a se q u e n c e o f fra m e s (o r m o tio n se g m e n ts), d N u m b e r o f d im e n s io n s a f te r d im e n s io n a lity re d u c tio n . D N u m b e r o f d im e n s io n s b e fo re d im e n s io n a lity re d u c tio n .

V Q u a n tis a tio n e rro r (o r a v e ra n g e d is to r tio n ) e In d e x fo r e n d m o tio n se g m e n t,

j In d e x fo r fra m e s in a m o tio n se q u e n c e .

j J e r k v e c to r

J N u m b e r o f jo in ts in th e a r tic u la te d figure. K N u m b e r o f c lu s te rs in L ev el o ne.

I N u m b e r o f tim e s K -m e a n s is in itia lis e d w ith d iffe re n t c lu s te r c e n tre s. 1 In d e x fo r in te r m e d ia te s ta g e in d y n a m ic p ro g ra m m in g a lg o rith m s , m In d e x fo r s te p s in d y n a m ic p r o g r a m m in g a lg o rith m s .

M N u m b e r o f fra m e s in a m o tio n seq u en ce.

M N u m b e r o f s te p s in d y n a m ic p ro g r a m m in g a lg o rith m s . N N u m b e r o f m o tio n se q u e n c e s in a d a ta b a s e ,

n I n d e x fo r m o tio n se q u e n c e s in a d a ta b a s e , p I n d e x for fra m e s in a d a ta b a s e .

P T o ta l n u m b e r o f fra m e s in a d a ta b a s e .

R M a x im u m n u m b e r o f r e p e titio n s allo w ed in K -m e a n s c lu s te rin g , s In d e x fo r s t a r t m o tio n se g m e n t.

C h ap ter 1

Introduction

1.1

M o tiv a tio n

T h e s y n th e s is o f r e a lis tic h u m a n m o tio n is o n e o f th e m o s t c h a lle n g in g to p ic s in c o m ­ p u t e r g ra p h ic s . T h e r e a re tw o m a in re a s o n s fo r th is . O n e is c o m p le x ity : th e la rg e n u m b e r o f d e g re es o f fre e d o m in th e h u m a n b o d y . H u m a n s h a v e a r o u n d tw o h u n d r e d b o n e s a n d s ix h u n d r e d m u scles to m ove th e m . S im p lific a tio n s a r e n e e d e d to m a k e th e t a s k t r a c t a b l e fro m th e p o in t o f view o f t h e in te rfa c e w ith th e a n im a to r . T h e se c o n d re a s o n is s u b je c tiv e a n d h a s to d o w ith th e w ay w e p e rc e iv e h u m a n m o tio n . O u r b r a in is e x tr a o r d in a r y w ell sk ille d a t ju d g in g h u m a n m o v e m e n t a n d d is c e rn in g a n y u n n a t u r a l a r tif a c ts . W e a r e a ll e x p e r ts in th is field, b e c a u s e we t r a i n h a r d e v e ry d a y in p e rfo rm in g ta s k s t h a t r a n g e fro m d e te c tin g a s u b tle m o v e m e n t o f d is a p p ro v a l in a p e r s o n ’s eyes, to re c o g n isin g o u r fa v o u rite f o o tb a ll p la y e r o n te le v isio n b y t h e w ay h e ru n s .

T h e s y n th e s is o f h u m a n m o tio n p r e - d a te s c o m p u te r g ra p h ic s b y s e v e ra l d e c a d e s. F ro m t h e e a rly c a r to o n s o f W a lt D isn e y in th e 1930s, t r a d itio n a l a n im a tio n h a s sp e c ia lise d in p r o d u c in g a c a r ic a tu re o f m o tio n to c o m m u n ic a te m e a n in g w ith v e ry g o o d re s u lts . S e v e ra l te c h n iq u e s h a v e b e e n d e v e lo p e d fo r th is p u rp o s e , like s q u a s h a n d s tr e tc h , a n tic ­ ip a tio n , follow th r o u g h , e x a g g e ra tio n a n d o th ers[4 2 ]. T r a d itio n a l a n im a tio n h a s se ld o m tr i e d to ach ie v e r e a lis tic h u m a n m o tio n th r o u g h p h y s ic a lly c o rre c t p o se s o f th e ir c h a r­ a c te rs . I n s te a d it h a s u se d a ll s o r ts o f tric k s to con v ey th e c h a r a c te r ’s e m o tio n a n d in te n t.

2

Chapter 1. Introduction

S e v e ra l te a m s o f a n im a to r s m a y w o rk o n t r a d itio n a l a n im a te d p r o d u c tio n s over m a n y m o n th s o r y ears. A fte r th e c h a r a c te rs a n d t h e s to r y h av e b e e n o u tlin e d in th e s to r y ­ b o a rd , th e s o u n d tr a c k is re c o rd e d a n d m a n u a lly c o rre la te d w ith a m o re d e ta ile d la y o u t o f t h e a n im a tio n . S e n io r a n im a to r s d ra w th e key fra m e s o f th e sto ry , th o s e in w h ic h th e c h a ra c te rs a r e in d is tin c tiv e o r e x tr e m e p o se s w h ic h c o n s tr a in th e r e s t o f t h e fra m e s. O th e r less e x p e rie n c e d a n im a to r s fill in th e g a p s la te r b y d ra w in g t h e fra m e s b e tw e e n th e keyfi'am es, in a p ro c e ss c a lle d inbetweening.

T h is p ro c e ss h a s b e e n e m u la te d in c o m p u te r a n im a tio n p ack ag es. T h e a n im a to r lays do w n th e key fra m e s in th e c o m p u te r b y m a n ip u la tin g th e s y n th e tic c h a r a c te r ’s jo in ts . T h is tim e th e c o m p u te r p ro v id e s th e in b e tw e e n in g b y in te r p o la tin g th e k e y fra m e s w ith s m o o th fu n c tio n s t h a t a r e th e n re -s a m p le d to g e n e ra te t h e in b e tw e e n fra m e s. W ith c u r r e n t so ftw a re a n d h a rd w a re th is is a re a l-tim e p ro c e ss; if th e r e s u lt p ro v id e d b y th e in te r p o la tio n is n o t c o n v in c in g t h e a n im a to r p la c e s m o re k ey fra m e s b e tw e e n th e in itia l ones, r e p e a tin g th e p ro c e ss over a n d over a g a in u n til th e r e s u lt is s a tis fa c to ry . T o ach iev e th e d e s ire d q u a lity , it is lik ely t h a t th e a n im a to r w ill e v e n tu a lly h av e to lay d o w n a d e n se s e t o f k e y fra m e s. A lth o u g h th is p r o c e d u re gives t o t a l c o n tro l to th e a r t i s t over a ll a s p e c ts o f th e m o tio n , a n d is th e re fo re p re f e rre d b y m a n y a n im a to r s , th e p r o d u c tio n o f o n ly a few se c o n d s c a n ta k e w eeks , a n d re q u ire s a sk ill t h a t ta k e s y e a rs to a c q u ire .

K e y fra m in g is n o t ex c lu siv e to a n im a tio n . M o v em en t is fre q u e n tly c o m m u n ic a te d b y a se q u e n c e o f s ta tic p o ses. A s p o r ts tr a in e r , a c h o re o g ra p h e r, a p h y s io th e r a p is t o r a p a r e n t sh o w how a m o v e m e n t h a s to b e p e rfo rm e d b y d e c o m p o sin g it in to a series o f s ta tic key p o s itio n s . T h e p e rs o n le a rn in g t h e m o v e m e n t d o e s so b y im ita tin g th o s e s ta tic p o s itio n s in o rd e r, u n til t h e se q u e n c e is le a rn e d a n d s to r e d so m e w h e re in h is o r h e r b r a in as a u n it o f m o v e m e n t. I t c o u ld b e s a id t h a t k e y fra m in g , in a g e n e ra l sen se, is a n a t u r a l m e th o d to c re a te a n d convey h u m a n m o tio n .

W h a t is f r u s tr a tin g fo r th e no v ice a n im a to r is n o t to h a v e to u se a k e y fra m in g in te rfa c e , b u t r a th e r th e d e n s ity o f th e s e t o f k e y fra m e s t h a t m u s t b e sp e c ifie d in o r d e r to o b ta in a c c e p ta b le re s u lts .

1.1. Motivation

3

Figure 1.1: Sparse keyframes. No an im ato r would expect to get credible m otion from only these two keyframes. T he result provided by th e com puter th rough interpolation in this case will not be a n a tu ra l m otion sequence, and m any m ore keyframes will have to be inserted to achieve n a tu ra l m otion.

In th is th e s is we a d d r e s s th e p ro b le m o f how to im p ro v e th e r e s u lt t h a t th e c o m ­ p u te r p ro v id e s w h e n p r e s e n te d w ith sparse k e y fra m e s. B y s p a rs e k e y fra m e s we m e a n k e y fra m e s t h a t a re v is u a lly v e ry d iffe re n t, su c h as th o s e o f F ig u re 1.1, w h e re sim p le in te r p o la tio n sch em es w o u ld fail to p ro d u c e a lifelike a n im a tio n (see C h a p te r 2 for a re ­ v iew o f tr a d itio n a l c o m p u te r a n im a tio n m e th o d s ). O u r g o al is to m a k e th e in b e tw e e n in g g e n e ra te d b y th e c o m p u te r c o rr e s p o n d to n a t u r a l h u m a n m o tio n .

A c h ie v in g th is g o al w o u ld m a k e a n im a tio n a c c essib le for th e no v ice o r o c c a sio n a l a n ­ im a to r s , w h o u se a n im a tio n a s a m e a n s to a n e n d , a n d d o n o t h av e th e tim e o r th e w illin g n e ss to b e tr a in e d in th e a r t o f d e n se k e y fra m in g . In p a r tic u la r , o u r w o rk is m o tiv a te d b y c h o re o g ra p h e rs w h o u se t h e c o m p u te r to sk e tc h o r v isu a lise th e ir c h o re ­ o g ra p h ie s u sin g c o m p u te r a n im a tio n p ack ag es like L ifefo rm s [37]. C h o re o g ra p h e rs a re f r u s tr a te d fro m u sin g c o m p u te r a n im a tio n pack ag es w h e n th e y re a lise t h a t th e ir e x p e r ­ tis e in h u m a n b o d y m o tio n , w h ic h allo w s th e m to lay d o w n d e ta ile d k e y fra m e s, is n o t tr a n s la te d in to n a t u r a l m o tio n b y th e c o m p u te r [40].

4

Chapter 1. Introduction

1.2

M e th o d o lo g y

F o r t h e c o m p u te r t o g e n e r a te n a t u r a l h u m a n m o tio n , w e n e e d to feed it w ith in fo r m a tio n a b o u t how th e b o d y m oves, i.e. w e n e e d to b u ild a c o m p u te r m odel o f h u m a n m o tio n . T h e C o n cise O x fo rd D ic tio n a ry gives t h e follow ing d e fin itio n o f t h e w o rd m o d el:

2 a sim plified (o ften m athem atica l) d escription o f a system etc., to assist calculations and predictions.

S im p lific a tio n is e s s e n tia l to m o d e llin g . T o b u ild a m o d e l, w e n e e d to sim p lify th e o b je c t o f o u r m o d e llin g , i.e. fin d th e u n d e r ly in g s tr u c t u r e a n d rem o v e d e ta il fro m it. H e re lies t h e f u n d a m e n ta l d iffic u lty in s y n th e s is in g r e a lis tic h u m a n m o tio n . A s w as s a id e a rlie r, w e a re v e ry w ell tr a in e d in ju d g in g h u m a n m o tio n . I f w e b u ild a m o d e l,‘a s im p lific a tio n ’ o f h u m a n m o tio n , a n d s y n th e s is e m o tio n u s in g th is m o d e l, w e risk lo o sin g th e im p o r ta n t d e ta il in h u m a n m o tio n , a n d as a r e s u lt t h e m o tio n s t a r t s to lo o k u n n a t u r a l v e ry quickly.

T h is re s e a rc h p ro p o s e s to s y n th e s is e n o v el m o tio n s b y s a m p lin g h ig h q u a lity h u m a n

m o tio n capture d a t a 1. G iv e n a la rg e s e t o f u n c la ssifie d m o tio n c a p tu r e seq u e n c e s, we le a rn a m o d e l t h a t c a n s u m m a riz e t h e s ta tis tic a l p r o p e r tie s a n d t h e te m p o r a l s tr u c tu r e o f th e d a ta , id e n tify in g se g m e n ts o f th e o rig in a l m o tio n se q u e n c e s t h a t c a n b e c o n c a te ­ n a te d a n d b le n d e d to g e n e ra te in b e tw e e n m o tio n g iv e n u s e r sp e c ifie d k e y fra m e s. T h e a d v a n ta g e o f th is a p p ro a c h is t h a t i t c o m b in e s t h e fle x ib ility o f k e y fra m e a n im a tio n w ith th e r e a lis m o f m o tio n c a p tu r e d a ta .

1.3

F ram ew ork

T h is w o rk w as fu n d e d b y th e E P S R C g r a n t “S y n th e s is in g H u m a n M o tio n fo r V ir tu a l P e rfo rm a n c e ” , a jo in t p r o je c t b e tw e e n t h e C e n tre fo r V isio n , S p e e c h a n d S ig n a l P ro c e s s ­ in g a n d th e d e p a r tm e n t o f D a n c e S tu d ie s o f S u rre y U n iv e rsity . T h e p r o je c t’s g e n e ra l a im s w ere to in v e s tig a te th e a p p lic a tio n o f p a t t e r n r e c o g n itio n a n d s ig n a l p ro c e s s in g

1.4. Structure

5

te c h n iq u e s to th e a n a ly s is o f h u m a n m o tio n d a t a co lle c te d b y m a rk e r b a s e d , o p tic a l m o tio n c a p tu r e d ev ices, a n d to u se th e d e v e lo p e d re p r e s e n ta tio n s to sy n th e s is e novel, n a t u r a l lo o k in g , h u m a n m o tio n s fo r u se in d ig ita l c h o re o g ra p h y a n d o th e r a n im a tio n a p p lic a tio n s . I t w as n ic k n a m e d Coppelia, a f te r t h e fa m o u s b a lle t b y D e lib e s.

C o m p le te c h a r a c te r a n im a tio n c o m p rise s se v e ra l a re a s o f re se a rc h . Coppelia w as c o n ­ c e rn e d w ith a f u n d a m e n ta l p a r t: s k e le ta l a n im a tio n . I n F ig u re 1.2 w e p r e s e n t a d ia g ra m d e s c rib in g t h e p r o je c t in t h e c o n te x t o f v ir tu a l p e rfo rm a n c e a n d illu s tr a tin g w h ic h p a r ts o f it c o rr e s p o n d to th is th e s is .

1.4

S tr u c tu re

I n C h a p te r 2 we w ill p u t th is re s e a rc h in to c o n te x t b y re v ie w in g p re v io u s a p p ro a c h e s to h u m a n m o tio n s y n th e s is . T h e a p p ro a c h e s in c o m p u te r g ra p h ic s a re v a rie d a n d b o rro w fro m c o n tro l e n g in e e rin g , d y n a m ic s im u la tio n , s ig n a l p ro c e s s in g , s ta tis tic s a n d n u m e r ic a l o p tim iz a tio n .

C h a p te r 3 p re s e n ts t h e fo rm u la tio n o f t h e p ro b le m o f h u m a n m o tio n s y n th e s is by k e y fra m in g o n a d a ta b a s e o f h u m a n m o tio n c a p tu r e d a ta . T h e p ro b le m o f a n im a ­ tio n is f o rm u la te d as a p a t h se a rc h in a c o m p le x g ra p h . W e p ro p o s e to re d u c e th e c o m p le x ity o f th e s e a rc h b y p la n n in g . T h e k n o w led g e re q u ire d fo r th is p la n n in g w ill b e o b ta in e d in a n u n s u p e rv is e d , d a t a d riv e n m a n n e r , b y b u ild in g a tw o -lev el s ta tis tic a l m o d e l o f th e d a ta .

C h a p te r 4 in tro d u c e s t h e fra m e w o rk p r o p o s e d in th is th e s is to a d d re s s th e p r o b le m o f h u m a n m o tio n sy n th e s is . A tw o -lev el s ta tis tic a l m o d e l is b u ilt fro m t h e d a ta . T h is m o d e l c a n t h e n b e u s e d to s y n th e s is e n o v el in b e tw e e n m o tio n g iv en u s e r sp ecified k ey fra m e s. R e s u lts a re p r e s e n te d w ith d a ta b a s e s o f 5000 fra m e s.

C h a p te r 5 e x te n d s a n d re fin e s th e b a sic fra m e w o rk b y in tr o d u c in g n e w r e p r e s e n ta tio n s o f m o tio n a n d in tr o d u c in g n e w te c h n iq u e s fo r a d y n a m ic p r o g r a m m in g se a rc h . T h e r e s u lts p r e s e n te d h e re in c lu d e a d a ta b a s e w ith 120 se q u e n c e s a n d 32000 fra m e s.

F in a lly , C h a p te r 6 c o n c lu d e s th is w o rk a n d offers a s u m m a r y a n d d is c u s s io n o f its re s u lts a n d c o n trib u tio n s . S o m e p o te n tia l a v en u e s fo r f u tu r e re s e a rc h a re a lso p re s e n te d .

Chapter 1. Introduction

THIS THESIS

mocap DB

Model

User Skeletal Key Frames Motion

PROJECT COPPELIA Solver Constraints Refined Skeletal Motion Character MESH Motion ANIMATION Virtual ENVIRONMENT Performance MODELLING

C h ap ter 2

Literature Review

A c o m p re h e n siv e o v erv iew o n t h e s u b je c t o f h u m a n m o tio n a n im a tio n is a n e n o rm o u s ta s k . T h e s u b je c t a t t r a c t s re s e a c h e rs in m a n y fields: b io m e c h a n ic s, r o b o tic s , c o m p u te r g ra p h ic s , c o m p u te r v isio n , te le c o m m u n ic a tio n s , a n d p e o p le in th e e n te r ta in m e n t in d u s ­ try . I n d iv id u a l s ta te - o f - th e - a r t re v ie w s in e a c h o f th e s e fields w o u ld fill m a n y p ag e s.

I n c o m p u te r g ra p h ic s t h e a p p ro a c h e s a re v a rie d a n d th e m e th o d s b o rro w fro m c o n tro l e n g in e e rin g , d y n a m ic s im u la tio n , s ig n a l p ro c e ssin g , s ta tis tic s a n d n u m e ric a l o p tim iz a ­ tio n . H e re t h e g o al is to r e p r o d u c e h u m a n m o tio n in a w ay t h a t n o t o n ly gives h ig h v is u a l q u a lity , b u t a lso p ro v id e s t h e a n im a to r w ith as m u c h c o n tro l as p o ssib le . A g r e a t d e a l o f re s e a rc h goes in to im p ro v in g in te r a c tiv ity a t th e in te rfa c e b e tw e e n th e a n im a to r a n d t h e c o m p u te r.

I n c o m p u te r v isio n , t h e in te r e s t is o n a n a ly s is a n d p r e d ic tio n o f h u m a n m o tio n . A p r e d ic tio n h a s to b e generated b y a m o d e l. I t is th is g e n e ra tiv e fu n c tio n w h ic h h a s in tr ig u e d so m e re s e a rc h e rs as to h o w m o d e ls t h a t a re b u ilt to a n a ly s e m o tio n c a n b e u s e d to s y n th e s is e it.

T h e in te n tio n o f th is c h a p te r is to p u t th is w o rk in c o n te x t. C o m p u te r g ra p h ic s te c h ­ n iq u e s w ill b e re v ie w e d t h a t s h a r e t h e g o a l o f th is th e s is : th e in tr o d u c tio n o f so m e level o f a u to m a tio n in th e a n im a tio n o f a c h a r a c te r ’s m o tio n . I n a d d itio n , re s e a rc h in h u m a n m o tio n a n a ly s is is p r e s e n te d fro m w h ic h th is th e s is b o rro w s m o s t o f its b u ild in g m a te ria l.

8

Chapter 2. Literature Review

2.1

T ra d itio n a l C o m p u te r A n im a tio n

A p p ro x im a te ly tw e n ty y e a rs h av e p a s s e d sin ce t h e first sk e le ta l a n im a tio n s y s te m w as d e v e lo p e d a t O h io S ta te U n iv ersity . S in c e th e n , a c o n tin u u m o f n e w te c h n iq u e s h av e a u to m a te d ta s k s t h a t in th e e a rly y e a rs o f c o m p u te r a n im a tio n h a d to b e p ro g r a m m e d fro m s c ra tc h e v e ry tim e a n e w a n im a tio n w as p ro d u c e d . T h is h a s o p e n e d th e in d u s tr y to a n a r tis tic a lly in c lin e d u s e r c o m m u n ity .

C o m p u te r a n im a tio n o f h u m a n m o tio n c o m p rise s se v e ra l a re a s o f re se a rc h . A c o m p le te s im u la te d h u m a n s h o u ld b e c o m p o se d o f s im u la te d m u scles [69] (m a n y o f th e m in th e face [76]), o p tio n a lly so m e c lo th [4] a n d , w h a t w ill b e m o re re le v a n t t o th is w o rk , a sk e le to n to s u p p o r t a ll o f th e p re v io u s c o m p o n e n ts.

A sk e le to n in c o m p u te r g ra p h ic s is o fte n r e p re s e n te d b y a c o lle c tio n o f rig id ‘b o n e s ’ lin k e d b y jo in ts a n d o rg a n is e d in a h ie r a rc h ic a l s tr u c tu r e . E a c h jo in t c a n h av e a p a r e n t a n d se v e ra l c h ild re n . F o r in s ta n c e , th e fo re a rm c a n h a v e t h e u p p e r a r m as p a r e n t a n d th e h a n d a s ch ild . A p o se o f th e s k e le to n c a n b e d e s c rib e d as th e r o ta tio n s o f e a c h jo in t w ith re s p e c t to its p a r e n t jo in t. A jo in t p o ssesses o n e to th r e e d e g rees o f freed o m . T h e m a th e m a tic s to d e s c rib e th e g e o m e try a n d t h e m o tio n o f a n a r tic u la te d s k e le to n in c o m p u te r g ra p h ic s w ere b o rro w e d fro m r o b o tic s [25].

2 . 1 . 1 I n t e r p o l a t i o n

T h e tr a d itio n a l d iv is io n o f a n im a tio n w o rk b e tw e e n k e y fra m in g a n d in b e tw e e n in g [78] h a s b e e n e m u la te d in c o m p u te r a n im a tio n p ack ag es. O n c e t h e m o d e llin g , t h e sk e tc h o f th e s to r y a n d th e re c o rd in g o f th e d ia lo g u e is d o n e , th e a n im a to r lay s d o w n th e key fra m e s b y a d ju s tin g t h e jo in ts o f t h e a r tic u la te d figure. T h is p ro c e ss is te d io u s b e c a u s e o f th e la rg e n u m b e r o f d e g re e s o f fre e d o m (D O F s) a s k e le to n c a n h av e. J o in ts h ig h e r in th e h ie ra rc h y h av e to b e in itia lly c a re fu lly p la c e d b e c a u s e a n y f u r th e r c o rre c tio n w ill affect a ll th e ch ild jo in ts .

I n te r p o la tio n is u se d to av o id r e p e a tin g th is p ro c e ss fo r e a c h fra m e . A fte r t h e key fra m e s h a v e b e e n se t, th e c o m p u te r in te r p o la te s b e tw e e n th e m b y f ittin g s m o o th p a r a m e tr ic fu n c tio n s to th e D O F s o f t h e c h a ra c te r. I n te r p o la tin g r o ta tio n s is n o t tr iv ia l a s th e y

2.1. Traditional Computer Animation

9

a re p e rio d ic b y n a tu r e , a n d d o n o t c o m m u te . M o st c o m p u te r a n im a tio n pa ck ag es u se q u a te r n io n s p h e ric a l in te r p o la tio n in tr o d u c e d b y S h o e m a k e [70].

K e y fra m in g a n d in te r p o la tio n r e m a in s th e fo rm o f a n im a tio n t h a t is u s e d in m a in ­ s tr e a m film p r o d u c tio n s s u c h as T o y S to r y w h e re th e q u a lity is t h e p r im a r y issu e over p r o d u c tio n c o st a n d tim e . K e y fra m in g , a lth o u g h tim e c o n s u m in g , gives th e a n im a to r th e c o n tro l th e y r e q u ir e to a c h iev e th e d e s ire d m o v e m e n t se q u e n c e .

2 . 1 . 2 I n v e r s e k i n e m a t i c s

In v e rse k in e m a tic s is w id e ly u s e d as a to o l to a id t h e a n im a to r in p o s itio n in g a n a r tic ­ u la te d sk e le to n . I n ro b o tic s , a c h a in o f rig id o b je c ts lin k e d b y jo in ts (a r o b o t a rm ) is k n o w n a s a m a n ip u la to r . T h e la s t lin k in t h e m a n ip u la to r is c a lle d t h e e n d -e ffe cto r. G iv e n q, a v e c to r o f k n o w n jo in t v a ria b le s, th e fo rw a rd k in e m a tic s p ro b le m solves for

x , t h e p o s itio n a n d o r ie n ta tio n o f th e e n d -effecto r: x = f ( q), w h e re / is a m a tr ix c o n c a te n a tio n . T h e p ro b le m o f in v erse k in e m a tic s c a n b e f o r m u la te d as follow s: G iv e n a d e s ire d p o s itio n a n d o r ie n ta tio n o f th e e n d -e ffe c to r x, c o m p u te t h e v alu es o f th e tr a n s f o r m a tio n s n e e d e d in t h e r e s t o f th e m a n ip u la to r lin k s, i.e. t h e in v e rse m a p p in g / “ 1 : x —» q. A d iffic u lty is t h a t / is a n o n -lin e a r f u n c tio n o f q, a n d th e r e is in g e n e ra l n o u n iq u e s o lu tio n to th e in v erse m a p p in g . A g e n e ra l a n a ly tic a l s o lu tio n fo r a r b i t r a r y m a n ip u la to r s d o e s n o t e x is t, a n d th e re fo re ite r a tiv e n u m e ric a l m e th o d s a re c o m m o n ly u s e d to o b ta in a p p r o x im a te s o lu tio n s . W e lm a n [85] gives a n e x c e lle n t o v erv iew o n th e s o r t o f m e th o d s t h a t a re u s e d in c o m p u te r a n im a tio n to solve th e p ro b le m . In v e rse k in e ­ m a tic s c a n b e c o m b in e d w ith t h e in c o r p o r a tio n o f c o n s tr a in ts to sp e c ify fo r in s ta n c e t h a t p a r ts o f th e b o d y s h o u ld n o t m ove o r r e m a in in c o n ta c t w ith su rfa c e s.

C u r r e n t c o m m e rc ia l a n im a tio n p ac k ag es c o m m o n ly in c lu d e a n in v erse k in e m a tic s solver. T h is allow s th e a n im a to r to in te r a c tiv e ly m ove a ll o r so m e o f th e lin k s in th e k in e m a tic c h a in b y d ra g g in g t h e e iid -e ffe c to r w h ile m a in ta in in g c e r ta in s p a tia l c o n s tr a in ts a t th e s a m e tim e . I f th e n u m b e r o f lin k e d se g m e n ts to solve for is la rg e , it c a n y ie ld p o s itio n s t h a t lo o k u n n a tu r a l, so t h e c h a in w ill ty p ic a lly b e o n ly a n a r m o r a leg. T h is p ro v id e s a n o th e r m e th o d fo r th e a n im a to r t o c r e a te k ey fra m e s w h ic h is o fte n f a s te r a n d m o re n a t u r a l t h a n m a n ip u la tio n o f in d iv id u a l jo in ts . I n o rd e r to c re a te t h e in -b e tw e e n fra m e s,

10

Chapter 2. Literature Review

e a c h o f th e d e g re e s o f fre e d o m o f t h e c h a in is in te r p o la te d in d e p e n d e n tly . T h is re s u lts in th e e n d -e ffe cto rs m o v in g in a w ay t h a t is n o t n e c e ssa rily c o n s is te n t w ith th e s p a tia l c o n s tr a in ts t h a t th e a n im a to r se ts a t t h e k ey fra m e s. A w ay a r o u n d th is p ro b le m is to ta k e th e o u t p u t o f th e in te r p o la tio n a n d th e n a p p ly in v e rse k in e m a tic s a t e a c h fra m e to fulfill th e c o n s tr a in ts [85, 20], T h is a p p ro a c h lin k s w ith m o re re c e n t m e th o d s o f c o n s tr a in t b a s e d m o tio n e d itin g , w h ic h w ill b e re v ie w e d la te r in S e c tio n 2.4.3.

In v e rse k in e m a tic s a d d s s u p p o r t fo r k e y fra m in g , b u t d o e s n o t a d d re s s th e p r o b le m o f how to a u to m a tic a lly o b ta in n a t u r a l in b e tw e e n m o tio n for th e k ey fra m e s.

2.2

M o tio n ca p tu r e sy ste m s

M o tio n c a p tu r e te c h n o lo g y allow s u s to c a p tu r e a n d s to r e in th e c o m p u te r th e m ove­ m e n t o f a p e rs o n . T h e p e rfo rm e r o f th e m o v e m e n ts is o fte n sp e c ia lly tr a in e d , a n d t h e a c c u ra c y in th e r e g is tr a tio n o f th e m o tio n is so h ig h t h a t th e sk ills o f t h e p e rfo rm e r c a n b e p a s s e d o n to th e a n im a te d c h a r a c te r . I n m o st cases th is p ro c e ss is fa s te r t h a n tr a d itio n a l k e y fra m e d a n im a tio n , a n d th e re fo re it h a s b e e n a d o p te d b y c o m p u te r g am e s c o m p a n ie s, w h e re re d u c in g d e v e lo p e m e n t tim e is e s s e n tia l to b u sin e ss. I n o th e r cases m o tio n c a p tu r e is th e o n ly p o s s ib le a p p ro a c h , w h e n th e p e rf o r m e r ’s p a r tic u la r s ty le m u s t b e p re se rv e d fo r a r tis tic re a so n s, o r w h e n t h e a n im a tio n m u s t b e p r o d u c e d in r e a l tim e for a s y n th e tic c h a r a c te r in a p ro c e ss s im ila r to p u p p e te e rin g .

T h e u se o f d iffe re n t te c h n iq u e s to m a k e a n im a te d c h a ra c te r s m ove in a m o re n a t u r a l fa sh io n h a s its r o o ts in th e in tr o d u c tio n o f s p e c ia l effects in th e film in d u s tr y in th e 1920s [19]. T h is a r t is fa s t c h a n g in g a n d th e te c h n iq u e s u s e d n o w a d a y s a re re la tiv e ly new : T h e U .S. m ilita r y b e g a n to u se m a g n e tic s y s te m s in t h e 1970s to m e a s u re th e h e a d m o v e m e n t o f p ilo ts . M o tio n p e r c e p tio n a n d e x tr a c tio n fro m m o v in g lig h t d isp la y s h a s b e e n s tu d ie d for m a n y y e a rs [17], a n d its g en ealo g y d a te s b a c k to t h e tim e s o f M u y b rid g e [54], B io m e d ic a l a p p lic a tio n s p re c e d e d th e u se o f o p tic a l s y s te m s in th e e n te r ta in m e n t' in d u s try . I n m a g n e tic sy s te m s , th e u s e r is e n c u m b e re d b y th e w ires a tta c h e d to h is b o d y . W ire le ss s y s te m s also e x is t, b u t h av e a lim ite d r a n g e o f a c tio n , w h ich re d u c e s th e c a p tu r in g sp a c e a v a ila b le to t h e p e rfo rm e r. F ro m a m a g n e tic s y s te m p o s itio n a l a n d r o ta tio n a l in fo rm a tio n c a n b e o b ta in e d d ir e c tly fro m t h e in p u t d evice.

2.3. Simulation of character motion

11

I n a n o p tic a l s y s te m o n ly p o s itio n a l in fo r m a tio n is p ro v id e d b u t a t g r e a te r a c c u ra c y a n d g r e a te r re s o lu tio n . T h e firs t r e a l- tim e s y s te m s to a p p e a r w ere m a g n e tic a l, b u t c u r r e n t o p tic a l s y s te m s [23, 49] c a n a lso p ro v id e r e a l-tim e r o ta tio n a l d a ta . M o tio n c a p tu r e d a t a s y s te m s a re e m p lo y e d in m u lti-m illio n p o u n d H o lly w o o d p r o d u c tio n s like “T it a n i c ” , “T h e M u m m y ” o r “G la d ia to r ” , a s w ell as in m o s t c o m p u te r g a m e s o n th e m a rk e t t h a t involve h u m a n -lik e c h a ra c te rs .

M o tio n c a p tu r e p ro v id e s n a t u r a l m o tio n , a n d s o m e tim e s a t a lo w er c o st t h a n t r a d i ­ tio n a l c o m p u te r a n im a tio n [19]. H ow ever, m o tio n c a p tu r e c a n b e n o is y a n d n e e d p o s t p ro c e s s in g to m a k e th e d a t a u s e fu l fo r a n im a tio n [7]. P r o b a b ly th e m a in w eak n ess o f m o tio n c a p tu r e is its lac k o f flex ib ility . I t is v e ry d ifficu lt to e d it t h e c a p tu r e d m o tio n w ith o u t lo o sin g th e n a t u r a l q u a lity o f t h e m o v e m e n t. F o r e x a m p le , it c o u ld h a p p e n t h a t a sin g le p ir o u e tte is c a p tu r e d , a n d la te r a d o u b le p ir o u e tte is re q u ire d fo r th e a n im a te d c h a ra c te r; i t is v e ry lik ely t h a t th e r e s u lt o f r e p e a tin g t h e sin g le t u r n tw ice to o b ta in th e d o u b le o n e w ill lo o k u n n a t u r a l u n le ss g re a t e ffo rt is in v e s te d (w h ich is w h a t m o tio n c a p tu r e tr ie s to av o id ). M a n y a n im a to r s d o n ’t b e lie v e in m o tio n c a p tu r e , th e m a in re a s o n b e in g th e d is a d v a n ta g e o f h a v in g to in v e st tim e to g e t th e d e sire d fin a l a n im a tio n a n y w a y [57, 32].

2.3

S im u la tio n o f ch a ra cter m o tio n

S im u la tio n m e th o d s a t t e m p t to fre e th e o p e r a to r o f a c o m p u te r a n im a tio n to o l fro m h a v in g to h a v e a r tis tic sk ill a n d / o r b io m e c h a n ic a l k n o w led g e a n d s till p ro d u c e u se fu l a n im a tio n s . H ig h level d e fin itio n s o f a c tio n s a re tr a n s f o r m e d b y ‘s lid e r s ’ o n in tu itiv e p a r a m e te rs like v elo city , d ir e c tio n o r d u r a tio n . T h e m e th o d s c a n b e k in e m a tic o r d y n a m ic . K in e m a tic s im u la tio n a t t e m p t s to ach iev e a u to m a tic g e n e r a tio n o f h u m a n m o tio n th r o u g h b io m e c h a n ic a l k n o w led g e, fo rw a rd k in e m a tic s a n d in v e rse k in e m a tic s . S im u la tio n m e th o d s f a c ilita te t h e ta s k o f g e n e ra tin g m o tio n s o u t o f a fa m ily fo r w h ic h a m o d e l h a s b e e n c o n s tr u c te d . A w a lk in g g e n e r a to r c a n g e n e ra te w a lk in g m o tio n s w ith d iffe re n t s te p le n g th s o r d iffe re n t v elo c itie s [14]. B u t if a r u n n in g m o tio n is re q u ire d , a r u n n in g g e n e r a to r h a s to b e c o n s tr u c te d [15]. R e c e n t a p p ro a c h e s to d y n a m ic s im u la tio n like [24] u se c o m p o s itio n o f c o n tro lle rs to in tr o d u c e fle x ib ility in th e w ay s im u la tio n

12

Chapter 2. Literature Review

m e th o d s c a n a d a p t to n ew a n im a tio n ta s k s .

2.3.1

D y n am ic sim u la tio n

T h e p ro b le m o f r o b o tic c o n tro lle r d e sig n o r inverse dynam ics is t h a t o f fin d in g t h e forces a n d to rq u e s n e e d e d to a c c o m p lis h a c e r ta in k in e m a tic goal. I n th e ca se o f r o b o tic s , th e g o al is a d e te r m in e d tr a je c to r y o f th e m a n ip u la to r . D y n a m ic m o d e ls o f t h e r o b o t a r m h av e to b e d e v e lo p e d , a n d th e n c o rre s p o n d in g c o n tro l a lg o rith m s a re d e sig n e d to ach iev e th e p re sp e c ifie d ta s k .

T h e g ro u p le a d b y J e s s ic a H o d g in s in G e o rg ia I n s t i t u t e o f T ech n o lo g y h a s b e e n a c tiv e in a p p ly in g th e s e te c h n iq u e s to th e s y n th e s is o f h u m a n m o tio n [34]. T h e s e m e th o d s g e n e ra te a n im a tio n s w h ich d o n o t v io la te th e p h y sic a l law s t h a t w ere in c lu d e d in th e m o d el. F o r in s ta n c e , in th e r u n n e r s im u la tio n s in [35], s ix c o n s tr a in ts a v o id a fo o t fro m p e n e tr a tin g t h e g ro u n d , s lid in g o n it a n d ro llin g o r y a w in g a r o u n d its c e n te r o f m ass. A n o th e r in te r e s tin g f e a tu r e o f d y n a m ic s im u la tio n m e th o d s is t h a t so m e h ig h level c o n tro l is offered to th e u se r, like v e lo c ity o f th e r u n n e r in [35] o r h e ig h t o f th e ju m p for th e d iv e r in [90].

R e se a rc h e rs in th is a r e a te n d to u se c o m e rc ia lly a v a ila b le p ack ag es like S D /F A S T 1 t h a t c a n g e n e ra te th e e q u a tio n s o f m o tio n a u to m a tic a lly fro m a d e s c rip tio n o f t h e g eo m e­ t r y o f th e a r tic u la te d s y s te m . H ow ever th is d o es n o t solve t h e p ro b le m o f d e sig n in g c o n tro lle rs t h a t , g iv en e q u a tio n s for th e a r tic u la te d c h a ra c te r , w ill m a k e it r u n , fa ll o r ju m p . E a c h a n im a tio n uses s e v e ra l c o n tro lle rs fo r e a c h p h a s e o f th e a c tio n a n d a s t a t e m a c h in e to s w itc h a m o n g th e m . D iffe re n t a n im a tio n s re q u ire d iffe re n t s t a t e m a c h in e s w ith d iffe re n t s ta te s a n d it is n o t e v id e n t ho w to m o d ify th e s e c o m p le x m o d e ls to g en ­ e r a te o th e r a c tio n s . T h is m ak es it d iffic u lt to in c o r p o r a te th e s e m e th o d s in to c o m p u te r a n im a tio n to o ls.

F a lo u ts o s a n d v a n d e P a n n e p ro p o s e d m e th o d s to av o id h a v in g to d e sig n th e c o n tro lle rs. In [82] th e c o n tro lle r s y n th e s is w as a c h iev ed th r o u g h S e n s o r-A c tu a to r-N e tw o rk s . T h e e q u a tio n s o f m o tio n a re g e n e ra te d a u to m a tic a lly u s in g S D /F A S T . P a r a m e te r s o f th e

2.4. Reuse of motion capture data

13

n e tw o rk (d e la y s o f th e n o d e s , w e ig h ts a n d p a r a m e te rs o f t h e P a r t i a l D e riv a tiv e C o n ­ tro lle rs ) a r e v a rie d s to c h a s tic a lly to o p tim iz e p a r tic u la r b e h a v io u rs , like for e x a m p le a d v a n c in g as m u c h d is ta n c e a s p o ssib le . T h e s e m e th o d s h av e b e e n a p p lie d to s im p li­ fied 2D c h a r a c te rs b e c a u s e t h e s e a rc h sp a c e b e c o m e s to o la rg e fo r m o re c o m p lic a te d m o d e ls. A n o th e r a p p r o a c h to a v o id in g t h e d e sig n o f n e w c o n tro lle rs is th e re u s e o f e x is tin g ones. F a lo u ts o s e t al. [24] in tr o d u c e d a fra m e w o rk to c o m p o se c o n tro lle rs b y d e fin in g t h e in te rfa c e b e tw e e n th e s e a n d a c o n tro lle r s u p e rv is o r. T h e in te rfa c e c o n ­ s is ts o f p re c o n d itio n s , p o s tc o n d itio n s a n d e x p e c te d p e rfo rm a n c e in h a n d lin g th e c u r r e n t d y n a m ic s t a t e o f th e c h a ra c te r. T h e y p re s e n t a n a u to m a tic w ay o f le a r n in g th e s e p r e ­ c o n d itio n s u s in g s u p e rv is e d le a rn in g a n d d e m o n s tr a te a series o f fa irly c o m p le x d y n a m ic b e h a v io u rs fo r a th r e e d im e n s io n a l c h a r a c te r like b a la n c in g , fa llin g to t h e floor, g e ttin g u p a n d s ittin g .

T h e s e m e th o d s w o rk b e t t e r fo r h ig h d y n a m ic r a n g e a c tio n s like v a u ltin g in w h ic h th e r e c a n n o t b e m u c h v a r ia tio n a m o n g ju m p e r s [35]. V an d e P a n n e [8 1 ]p ro p o sed a w ay o f p a r a m e tr is in g w a lk in g m o tio n s fo r p a ssiv e ly s ta b le 3D c h a ra c te rs . T h e m e th o d c o n sists o f a n in te r p o la tio n in t h e p a r a m e te r sp a c e o f t h e c o n tro lle rs. T h is in te r p o la tio n is o b ta in e d b y ta k in g o n e c o n tro lle r X a n d o p tim iz in g w ith r e s p e c t to so m e c rite r io n to o b ta in a c o n tro lle r Y.

I f in itia l c o n d itio n s c o n s tr a in t h e m o tio n , as in p la tf o r m d iv in g [90], t h e r e s u lts a re im p ro v e d . H ow ever, fin d in g t h e fo rces o r to rq u e s t h a t w o u ld m a k e a c h a r a c te r d a n c e a r o u tin e w ith a sp ecific q u a lity is v e ry d iffic u lt, if n o t im p o ssib le . T h e v is u a l q u a lity o b ta in e d w ith m o tio n c a p tu r e d a t a c a n n o t y e t b e m a tc h e d w ith s im u la tio n m e th o d s .

2.4

R e u se o f m o tio n ca p tu r e d a ta

T h e a p p e a lin g v is u a l q u a lity o f m o tio n c a p tu r e , c o m b in e d w ith its a fo re m e n tio n e d lack o f fle x ib ility a n d h ig h c o st, h a s le d re s e a rc h e rs to in v e s tig a te te c h n iq u e s t h a t p e r m it re u s e a n d a d a p t a t i o n o f e x is tin g m o tio n d a ta . M o tio n c a p tu r e lib ra rie s a re c o m e rc ia lly a v a ila b le 2. B u t t h e p ro b le m r e m a in s h o w to re u s e th e d a t a to m e e t n e w g o als a n d

2See for instance http://www.charactermotion.com, http://www.viewpoint.com /, http://elelctrashock.com/ or http://www.m otek.org/

14

Chapter 2. Literature Review

a n im a te d iffe re n t c h a ra c te rs .

I n te r p o la tio n o r b le n d in g te c h n iq u e s [67, 86] a t t e m p t to g e n e ra te n e w m o tio n s b y b le n d ­ in g a m o n g tw o o r m o re e x is tin g m o tio n s. S ig n a l p ro c e s s in g m e th o d s [16, 80, 89] a p p ly g lo b a l tr a n s f o r m a tio n s to t h e m o tio n s o r th e ir fre q u e n c y re p re s e n ta tio n s . C o n s tr a in t b a s e d te c h n iq u e s [47, 29, 62, 88, 61, 91] e x p re ss e x p lic itly th r o u g h c o n s tr a in ts th o s e fe a tu re s o f th e m o tio n s t h a t h a v e to b e p re s e rv e d a n d th o s e t h a t h a v e to b e m o d ifie d . F in a lly , s ta tis tic a l m o d e ls o f h u m a n m o tio n c a n b e se e n a s a w ay to le a r n c o n s tra in ts fro m d a ta . D iffe re n t te c h n iq u e s c a n b e u s e d to m a k e s ta tis tic a l m o d e ls u se fu l fo r th e s y n th e s is o f m o tio n [9, 12, 27, 43, 64, 72, 73]. T h e re s e a rc h p r e s e n te d in th is th e s is falls in to th is c a te g o ry b y in tr o d u c in g a n ew w ay o f u s in g a s ta tis tic a l m o d e l fo r h u m a n m o tio n s y n th e s is t h a t c a n b e c o n tro lle d b y a k e y fra m in g in te rfa c e . I n S e c tio n s 2.4.1 to 2.4.4 a c r itic a l re v ie w o f m o tio n c a p tu r e d a t a re u s e m e th o d s is offered.

2.4.1

S ignal p ro cessin g

In th e m id 1990s, p r o b a b ly as a c o n se q u e n c e o f th e a v a ila b ility o f m o tio n c a p tu r e d a t a to re se a rc h e rs, se v e ra l a t t e m p t s w ere m a d e a t t r e a tin g h u m a n m o tio n a s a m u ltid im e n ­ sio n a l sig n a l o n w h ic h to a p p ly d iffe re n t sig n a l p ro c e s s in g te c h n iq u e s [16, 89]. S o m e o f th e s e te c h n iq u e s h av e th e ir o rig in in im a g e p ro c e s s in g a n d o th e r s in s p e e c h re c o g n itio n . M u lti-r e s o lu tio n filte rin g [16] is u s e d to d e v elo p a ‘m o tio n e q u a lis e r ’, i.e. a b a n k o f filte rs t h a t allow s a m p lific a tio n o r a tte n u a tio n o f th e m o tio n ’s a m p litu d e a t d iffe re n t freq u en cies. T h is a p p r o a c h is b a s e d o n th e h y p o th e s is t h a t fine d e ta il in m o tio n is fo u n d in th e h ig h fre q u e n c ie s, a n d c o a rse m o tio n in th e low fre q u e n c ies. I n te r p o la tio n o f tw o m o tio n s a n d m o tio n w a v e sh a p in g a re a lso a d d re s s e d . M o tio n d is p la c e d m a p p in g [16, 89] a p p lie s s m a ll lo c a l m o d ific a tio n s to th e m o tio n sig n al. I n th is a p p ro a c h , ch a n g e s to th e m o tio n a r e k e y fra m e d , as o p p o s e d to k e y fra m in g n ew p o ses. O n c e th e s e keychanges a re s u ita b ly p la c e d a n d in te r p o la te d , a s m o o th m o d ific a tio n o f th e w h o le m o tio n re s u lts . U n u m a e t al. [80] ta k e a d v a n ta g e o f th e p e rio d ic b e h a v io u r o f h u m a n lo c o m o tio n to a p p ly m o d ific a tio n s to th e F o u rie r e x p a n s io n s o f th e m o tio n sig n a l. A b a s ic c o m p o n e n t a n d a q u a lita tiv e c o m p o n e n t a re id e n tifie d a n d th e n u se d to in te r p o la te o r e x tr a p o la te m o tio n s in th e fre q u e n c y d o m a in .