Computer Generated Animation of Faces Frederick I. Parke, University of Utah
T h i s p a p e r d e s c r i b e s the r e p r e s e n t a t i o n , a n i m a t i o n a n d d a t a c o l l e c t i o n t e c h n i q u e s t h a t h a v e b e e n u s e d to p r o d u c e " r e a l i s t i c "
c o m p u t e r g e n e r a t e d h a l f - t o n e a n i m a t e d se- q u e n c e s of a h u m a n face c h a n g i n g e x p r e s - sion. It w a s d e t e r m i n e d t h a t a p p r o x i m a t i n g the s u r f a c e of a face w i t h a p o l y g o n a l s k i n c o n t a i n i n g a p p r o x i m a t e l y 250 p o l y g o n s de- f i n e d by a b o u t 400 v e r t i c e s is s u f f i c i e n t
to a c h i e v e a r e a l i s t i c face. A n i m a t i o n w a s a c c o m p l i s h e d u s i n g a c o s i n e i n t e r p o l a t i o n s c h e m e to fill in the i n t e r m e d i a t e f r a m e s b e t w e e n e x p r e s s i o n s . T h i s a p p r o a c h is g o o d e n o u g h to p r o d u c e r e a l i s t i c f a c i a l m o t i o n . The t h r e e - d i m e n s i o n a l d a t a u s e d to d e s c r i b e the e x p r e s s i o n s of the face w a s o b t a i n e d p h o t o g r a m m e t r i c a l l y u s i n g p a i r s of p h o t o g r a p h s .
K E Y W O R D S A N D P H R A S E S : c o m p u t e r g r a p h i c s , h a l f - t o n e r e n d e r i n g , s m o o t h s h a d i n g , com- p u t e r a n i m a t i o n , f l e x i b l e s u r f a c e s , p o l y - g o n a l s u r f a c e s , f a c i a l t o p o l o g y , c o s i n e i n t e r p o l a t i o n , £ h r e e - d i m e n s i o n a l d a t a a c q u i s i t i o n .
C R C A T E G O R I E S : 8.2, 3.41, 4.41, 6.35 I N T R O D U C T I O N
T h e h u m a n f a c e is a c h a l l e n g e for c o m - p u t e r a n i m a t i o n for a t l e a s t two r e a s o n s . F i r s t the face is n o t a r i g i d s t r u c t u r e b u t is a c o m p l e x f l e x i b l e s u r f a c e . H o w is the m o t i o n of s u c h a s u r f a c e s p e c i f i e d ? Sec- o n d l y f a c e s are v e r y f a m i l i a r to us, we h a v e a w e l l d e v e l o p e d s e n s e of w h a t e x p r e s - s i o n s and m o t i o n s are n a t u r a l for a face.
We n o t i c e s m a l l d e v i a t i o n s f r o m o u r c o n c e p t of h o w a face s h o u l d a p p e a r .
T h i s p a p e r d e s c r i b e s a f a i r l y s i m p l e w a y of r e p r e s e n t i n g the face a n d an a n i m a - t i o n t e c h n i q u e t h a t a l l o w s the p r o d u c t i o n of r e a l i s t i c h a l f - t o n e a n i m a t e d s e q u e n c e s of the face c h a n g i n g e x p r e s s i o n . T h e p a p e r
T h i s r e s e a r c h w a s s u p p o r t e d in p a r t b y the U n i v e r s i t y of U t a h C o m p u t e r S c i e n c e D i v i s i o n and by the A d v a n c e d R e s e a r c h P r o - jects A g e n c y of the D e p a r t m e n t of D e f e n s e , m o n i t o r e d by R o m e A i r D e v e l o p m e n t C e n t e r , G r i f f i s A i r F o r c e Base, N e w Y o r k 13440, u n d e r c o n t r a c t F 3 0 6 0 2 - 7 0 - C - 0 3 0 0 .
a l s o d e s c r i b e s the m e t h o d u s e d to c o l l e c t the d a t a for the faces. T h e s e t e c h n i q u e s c o u l d be u s e d to a n i m a t e o t h e r f l e x i b l e s u r f a c e s .
R E P R E S E N T A T I O N OF T H E F A C E
The f a c e is a y e r y c o m p l e x t h r e e - d i m e n s i o n a l surface. T h i s s u r f a c e is f l e x - ible. It u s u a l l y c o n t a i n s c r e a s e s , a n d it has c o l o r v a r i a t i o n . W h a t is the b e s t w a y to r e p r e s e n t s u c h a s u r f a c e t h a t a l l o w s b o t h a n i m a t i o n a n d h a l f - t o n e r e n d e r i n g ? O n e p o s s i b i l i t y w o u l d be to f i n d an a n a l y - tic s u r f a c e or c o l l e c t i o n of a n a l y t i c sur- face p a t c h e s (i) to a p p r o x i m a t e the sur- face o f the face. A s s u m i n g t h i s w e r e f e a s i b l e , t h e r e r e m a i n s the p r o b l e m of a n i m a t i n g this s u r f a c e or c o l l e c t i o n of p a t c h e s . A g a i n a s s u m i n g t h a t a p p r o p r i a t e a n i m a t i o n t e c h n i q u e s w e r e a v a i l a b l e f o r s u c h s u r f a c e s , t h e r e s t i l l r e m a i n s the p r o b l e m of p r o d u c i n g h a l f - t o n e r e n d e r i n g s of the s u r f a c e s . H i d d e n s u r f a c e a n d h a l f - tone a l g o r i t h m s e x i s t for q u a d r i c s u r f a c e s (2,3) b u t t h e y t e n d to be q u i t e e x p e n s i v e . O n e c a n i m a g i n e s i m i l a r a l g o r i t h m s for s u r f a c e s of h i g h e r d e g r e e , b u t w o u l d ex- p e c t t h e m to i n c r e a s e r a p i d l y in e x p e n s e as the d e g r e e of the s u r f a c e i n c r e a s e d . F o r this r e a s o n , w h e n h a l f - t o n e r e n d e r - ings are d e s i r e d , s u r f a c e s of h i g h d e g r e e are u s u a l l y a p p r o x i m a t e d b y a s k i n of p o l y g o n s .
In o r d e r to a p p r o x i m a t e the face w i t h a n a l y t i c p a t c h e s , one w o u l d e x p e c t t h e s e p a t c h e s to be at l e a s t q u a d r i c , and p r o b e a b l y of h i g h e r d e g r e e . T h e a p p r o a c h of a p p r o x i m a t i n g the face w i t h a n a l y t i c sur- f a c e s l e a d s to a p p r o x i m a t i n g the a p p r o x i - m a t e s u r f a c e s w i t h p o l y g o n s . T h i s s e e m s a r a t h e r c o m p l e x a n d r o u n d a b o u t a p p r o a c h .
T h e a p p r o a c h t a k e n in t h i s p a p e r is one f i r s t u s e d b y H e n r i G o u r a u d (4). His a p p r o a c h w a s to d i r e c t l y a p p r o x i m a t e the s u r f a c e of the face w i t h a n o n - a n a l y t i c skin of p o l y g o n s . T h i s s k i n w a s c o n s t r u c t - ed b y s a m p l i n g the s u r f a c e of the f a c e at a n u m b e r of p o i n t s a n d c o n n e c t i n g t h e s e p o i n t s to f o r m a s k i n o f p o l y g o n s .
In o r d e r to p r o d u c e a h a l f - t o n e r e n d - e r i n g of o b j e c t s in a t h r e e - d i m e n s i o n a l s p a c e s e v e r a l p r o b l e m s m u s t be solved.
T h e f i r s t of t h e s e p r o b l e m s is u s u a l l y re- f e r r e d to as c l i p p i n g . T h i s is the p r o b - l e m of d e t e r m i n i n g if all or p a r t of an o b j e c t is w i t h i n a v i e w i n g space. T h e v i e w i n g s p a c e is a p y r a m i d d e f i n e d b y the p o s i t i o n of the view, the d i r e c t i o n the v i e w e r is l o o k i n g a n d h i s v i e w i n g angle.
A n e x a m p l e of a v i e w i n g s p a c e w o u l d be t h a t p a r t of the u n i v e r s e v i s i b l e t h r o u g h a w i n d o w , a s s u m i n g t h a t all o b j e c t s ex- c e p t the w i n d o w f r a m e w e r e t r a n s p a r e n t and c o u l d n o t o c c l u d e o t h e r o b j e c t s . T h e s e c o n d p r o b l e m is the d e t e c t i o n of h i d d e n s u r f a c e s . In o t h e r w o r d s , w h i c h s u r f a c e s are in f r o n t of o t h e r s u r f a c e s w h e n s e e n f r o m a g i v e n p o s i t i o n . T h e l a s t p r o b l e m is o n e of d e t e r m i n i n g the s h a d i n g of t h e v i s i b l e s u r f a c e s a n d p r o d u c i n g the s h a d e d i m a g e on some o u t p u t d e v i c e , n o r m a l l y a CRT. T h e s h a d i n g u s u a l l y d e p e n d s on the o r i e n t a t i o n of the s u r f a c e w i t h r e s p e c t to the v i e w e r a n d the l i g h t source.
A p p r o x i m a t i o n w i t h p o l y g o n s has sev- e r a l a d v a n t a g e s . F o r p o l y g o n a l s u r f a c e s , the p r o b l e m s l i s t e d a b o v e h a v e b e e n s o l v e d by a n u m b e r of a l g o r i t h m s (5-11).
T h e s e a l g o r i t h m s are f a s t a n d i n e x p e n s i v e w h e n c o m p a r e d to a l g o r i t h m s for s u r f a c e s of h i g h e r d e g r e e . A t l e a s t one of t h e s e a l g o r i t h m s (9) is i m p l e m e n t e d in h a r d w a r e a n d a n o t h e r (ii) is c u r r e n t l y b e i n g im- p l e m e n t e d in h a r d w a r e . A l s o , the d e v e l - o p m e n t by G o u r a u d (4) of a s m o o t h s h a d i n g a l g o r i t h m for p o l y g o n a l s u r f a c e s m a k e s it p o s s i b l e to g i v e a c o n t i n u o u s l y c u r v e d a p p e a r a n c e to a s u r f a c e m a d e up of p o l y - gons.
F o r p o l y g o n a l s h a d i n g the s h a d e of e a c h p o l y g o n is c o n s t a n t a c r o s s the p o l y - gon. T h i s s h a d e is a f u n c t i o n of the a n g l e b e t w e e n the n o r m a l to the p o l y g o n and a line f r o m the l i g h t s o u r c e to the p o l y g o n . F o r G o u r a u d ' s s m o o t h s h a d i n g , h o w e v e r , the s h a d e is n o t c o n s t a n t a c r o s s the p o l y g o n . It is a f u n c t i o n of the a n g l e b e t w e e n the n o r m a l at e a c h v e r t e x of the p o l y g o n a n d a line f r o m the l i g h t s o u r c e to the p o l y g o n , a n d the p o s i t i o n w i t h i n in the p o l y g o n . The n o r m a l at a v e r t e x is the a v e r a g e of the n o r m a l s of the p o l y g o n s t h a t h a v e this v e r t e x in c o m m o n .
In s m o o t h s h a d e d r e n d e r i n g s a s p e c i a l p r o c e d u r e is n e c e s s a r y ~f c r e a s e s are to be v i s i b l e . C r e a s e s c a n be m a d e v i s i b l e b y " d o u b l i n g " v e r t i c e s . S i n c e c r e a s e s c a n o c c u r o n l y a l o n g the b o u n d a r y b e t w e e n a d j o i n i n g p o l y g o n s , e a c h v e r t e x a l o n g the c r e a s e is d o u b l e d . O n e v e r t e x of the p a i r b e l o n g s to p o l y g o n s on one s i d e of the c r e a s e a n d the o t h e r v e r t e x of the p a i r b e l o n g s to p o l y g o n s on the o t h e r s i d e of the crease. T h i s c a u s e s s e p a r a t e n o r m a l s to be c o m p u t e d . W h e n the p o l y g o n s are shaded, t h e r e w i l l be a s h a d i n g d i s c o n t i n - u i t y a l o n g the b o u n d a r y a n d the c r e a s e w i l l be v i s i b l e .
F i g u r e 1 s h o w s a face r e n d e r e d w i t h p o l y g o n a l s h a d i n g a n d a d i f f e r e n t e x p r e s - s i o n of the s a m e face r e n d e r e d u s i n g the s m o o t h s h a d i n g a l g o r i t h m .
F i g u r e 1
T w o e x p r e s s i o n s of the same face. T h e top one w a s r e n d e r e d u s i n g p o l y g o n a l s h a d - ing. T h e b o t t o m one w a s r e n d e r e d u s i n g G o u r a u d ' s s m o o t h s h a d i n g a l g o r i t h m .
H a v i n g d e c i d e d to u s e a p o l y g o n a l re~
p r e s e n t a t i o n , h o w d o e s o n e go a b o u t a P p r o x - i m a t i n g a face w i t h p o l y g o n s ? T h e r e are s e v e r a l t h i n g s to k e e p in mind.
i. To g e t g o o d s m o o t h s h a d i n g , the d e n s i t y of p o l y g o n s s h o u l d be h i g h e s t in the a r e a s of h i g h e s t c u r v a t u r e (the nose, m o u t h , a r o u n d the e y e s a n d the e d g e of the chin) a n d l o w e s t in the a r e a s of l o w e s t c u r v a t u r e (the f o r e h e a d , c h e e k s a n d n e c k ) .
2. W h e r e c r e a s e s o c c u r on a f a c e (under the eyes, the s i d e of the nose, the e d g e of the lips and the c o r n e r of the m o u t h ) , e d g e s of p o l y g o n s m u s t c o i n c i d e w i t h the c r e a s e s . A p o l y g o n m a y n o t s p a n
a c r e a s e .
3. U s e the s m a l l e s t n u m b e r of p o l y g o n s c o n s i s t e n t w i t h g o o d r e s u l t s . T h e r e a s o n s for t h i s are o b v i o u s : a s m a l l e r a m o u n t of data, f a s t e r p i c t u r e g e n e r a t i o n a n d m i n i - m i z a t i o n of the d a t a a c q u i s i t i o n p r o b l e m . 4. If a n i m a t i o n is d e s i r e d , the p o l y - g o n s m u s t be l a y e d o u t in a w a y t h a t a l l o w s the f a c e to f l e x n a t u r a l l y . T h e p o l y g o n s s h o u l d r e m a i n a p p r o x i m a t e l y p l a n a r as the face flexes.
F i g u r e 2
T w o p i c t u r e s s h o w i n g the s k i n of p o l y g o n s u s e d to r e p r e s e n t the face.
N o t e t h a t the p o l y g o n s of the s k i n c h a n g e s h a p e and p o s i t i o n as the face c h a n g e s e x p r e s s i o n .
5. S i n c e the face is a p p r o x i m a t e l y s y m m e t r i c , w e n e e d w o r r y o n l y a b o u t o n e side of the face. T h e o t h e r s i d e is o b t a i n e d b y " m i r r o r i n g " or r e f l e c t i n g a b o u t the p l a n e of s y m m e t r y .
6. E a c h p o l y g o n w i l l h a v e a s s o c i a t e d w i t h it a color. W h e r e c o l o r b o u n d a r i e s o c c u r on the face, the lips a n d e y e b r o w s , for e x a m p l e , p o l y g o n e d g e s m u s t c o i n c i d e w i t h t h e s e b o u n d a r i e s . A p o l y g o n m a y n o t
s p a n a c o l o r b o u n d a r y .
K e e p i n g t h e s e t h i n g s in m i n d the n e x t s t e p is to f i n d a c o o p e r a t i v e a s s i s t a n t w h o w i l l a l l o w y o u to d r a w or p a i n t a s e t of p o l y g o n s on his or h e r face. A f t e r d r a w i n g the p o l y g o n s k i n on one h a l f of the face, a s k the a s s i s t a n t to a s s u m e a n u m b e r of d i f f e r e n t e x p r e s s i o n s . F o r e a c h e x p r e s s i o n o b s e r v e h o w w e l l t h e s e p o l y g o n s a p p r o x i m a t e the face. A f t e r m o d i f y i n g the p o l y g o n set s e v e r a l t i m e s y o u s h o u l d a r r i v e at a r e a s o n a b l e set of p o l y g o n s to r e p r e s e n t the face. F i g u r e 2 s h o w s the s k i n of p o l y g o n s u s e d to p r o d u c e the f a c e s s h o w n in f i g u r e s 1,3 a n d 4.
O n e - h a l f of t h i s s k i n c o n t a i n s 124 p o l y - g o n s d e f i n e d b y 202 v e r t i c e s .
A u n i q u e p o i n t n u m b e r is a s s i g n e d to e a c h v e r t e x of the skin. T h e s k i n is t h e n s p e c i f i e d b y g o i n g a r o u n d e a c h p o l y g o n in a c l o c k w i s e d i r e c t i o n a n d r e c o r d i n g the p o i n t n u m b e r s of its v e r t i c e s .
T h e d e t a i l s of the face are v e r y i m p o r t a n t in a c h i e v i n g r e a l i s t i c r e s u l t s . F i g u r e 3 s h o w s the e f f e c t d e t a i l s , s u c h as the eyes, e y e b r o w s , e y e l a s h e s a n d t e e t h h a v e on the r e a l i s m of the face. T h e e y e - b r o w s a n d t e e t h w e r e i n c l u d e d s i m p l y b y a d d i n g p o l y g o n s of the a p p r o p r i a t e color.
The i l l u s i o n of e y e l a s h e s w a s a c h i e v e d b y c h a n g i n g the c o l o r of e x i s t i n g p o l y g o n s d i r e c t l y a b o v e the eyes. T h e face b y i t s e l f is n o t v e r y r e a l i s t i c . It w a s n e c - e s s a r y to c o m p l e t e the h e a d in o r d e r to be c o n v i n c i n g . F i g u r e 3 s h o w s h o w a " b o n n e t "
of h a i r w a s u s e d to c o m p l e t e the head.
C o l o r is an i m p o r t a n t f e a t u r e of the face. E a c h p o l y g o n h a s a c o l o r a s s o c i a t e d w i t h it. T h i s c o l o r is m a d e up of t h r e e c o m p o n e n t s ; red, g r e e n and blue. By
I
F i g u r e 3
F o u r p i c t u r e s t h a t i l l u s t r a t e the e f f e c t of d e t a i l s on the r e a l i s m of a face. T h e f i r s t p i c t u r e s h o w s the face alone. T h e n e x t p i c t u r e s h o w s the face w i t h e y e s a n d n o s e trils. In the n e x t p i c t u r e , teeth, e y e l a s h e s , e y e b r o w s a n d the i n s i d e of the m o u t h w e r e added. The last p i c t u r e s h o w s the c o m p l e t e head.
s p e c i f y i n g the v a l u e of e a c h c o m p o n e n t it is p o s s i b l e to a c h i e v e the d e s i r e d colors.
C o l o r h a l f - t o n e r e n d e r i n g s are p r o d u c e d by s c a n n i n g o u t the p i c t u r e t h r e e times, o n c e for e a c h of the p r i m a r y c o l o r s . The a p p r o - p r i a t e c o l o r f i l t e r is p l a c e d in f r o n t of the c a m e r a lens b e f o r e e a c h scan.
A f t e r s o m e e x p e r i m e n t a t i o n the c o m p o n - e n t v a l u e s for f l e s h - t o n e and the o t h e r c o l o r s of the f a c e w e r e d e t e r m i n e d . T h e s e c o m p o n e n t v a l u e s d e p e n d on a n u m b e r of v a r i a b l e s , i n c l u d i n g : the p h o s p h o r of the CRT, the t y p e of f i l t e r s used, the type of f i l m used, the i n t e n s i t y s e t t i n g of the CRT, a n d the c o m p e n s a t i o n f u n c t i o n u s e d to o v e r c o m e the n o n - l i n e a r c h a r a c t e r i s t i c s of the CRT.
A N I M A T I O N
A s s u m i n g t h a t we h a v e a s a t i s f a c t o r y s k i n of p o l y g o n s for the face, h o w do w e a n i m a t e it? We w o u l d like to s p e c i f y the m o t i o n of the s u r f a c e in the s i m p l e s t w a y c o n s i s t e n t w i t h n a t u r a l m o t i o n .
T h e a p p r o a c h t a k e n in this r e s e a r c h is s o m e w h a t s i m i l a r to the a p p r o a c h t a k e n b y the c o n v e n t i o n a l a n i m a t o r . T h e a n i m a t o r s p e c i f i e s the d e s i r e d m o t i o n b y b l o c k i n g it o u t w i t h a s e r i e s of k e y ~ r a w i n g s . He t h e n g i v e s t h e s e k e y d r a w i n g s to the as- s i s t a n t a n i m a t o r s w h o g e n e r a t e the r e q u i r e d i n t e r m e d i a t e frames. F o r the c o m p u t e r a n i m a t i o n , the k e y d r a w i n g s are r e p l a c e d by d a t a f i l e s d e s c r i b i n g the face for e a c h of a n u m b e r of d i f f e r e n t e x p r e s s i o n s . T h e d a t a for e a c h e x p r e s s i o n or "phase" of the f a c e c o n s i s t s of the t h r e e - d i m e n s i o n a l p o s i t i o n of e a c h p o i n t d e f i n i n g the p o l y g o n s k i n u s e d to r e p r e s e n t the face. F i g u r e 4 s h o w s two p h a s e s of a face.
T h e a n i m a t i o n p r o g r a m t a k e s the p l a c e of the a s s i s t a n t a n i m a t o r s and g e n e r a t e s the r e q u i r e d i n t e r m e d i a t e f r a m e s b e t w e e n the p h a s e s as the face c h a n g e s e x p r e s s i o n .
T o c h a n g e the face f r o m o n e e x p r e s - s i o n to a n o t h e r is a m a t t e r of m o v i n g e a c h p o i n t a s m a l l d i s t a n c e in s u c c e s s i v e frames. T h e p o s i t i o n of e a c h p o i n t of the s k i n in e a c h f r a m e is d e t e r m i n e d b y i n t e r p o l a t i n g b e t w e e n the p r e v i o u s p h a s e p o s i t i o n a n d the n e x t p h a s e p o s i t i o n . F i g u r e 2 s h o w s h o w the p o l y g o n s of the s k i n c h a n g e s h a p e a n d p o s i t i o n as the f a c e c h a n g e s e x p r e s s i o n .
S i n c e the f a c e is g o v e r n e d b y p h y s i c a l laws, its m o t i o n is n o t l i n e a r b u t t e n d s to a c c e l e r a t e a n d d e c e l e r a t e . A c o s i n e i n t e r p o l a t i o n s c h e m e w a s u s e d to a p p r o x i - m a t e the a c c e l e r a t i o n a n d d e c e l e r a t i o n of the f a c i a l m o t i o n s . E a c h f r a m e h a s a s s o c i a t e d w i t h it a p h a s e n u m b e r . T h i s p h a s e n u m b e r is a r e a l n u m b e r w h o s e i n t e g e r p a r t r e f e r s to the p r e v i o u s p h a s e a n d w h o s e f r a c t i o n a l p a r t i n d i c a t e s the po- s i t i o n of this f r a m e b e t w e e n the p r e v i o u s p h a s e a n d the n e x t p h a s e . F o r e x a m p l e , if p h a s e 2 is a s m i l e a n d p h a s e 3 is a f r o w n t h e n the p h a s e n u m b e r 2.5 m e a n s an e x p r e s - s i o n h a l f w a y b e t w e e n a s m i l e a n d a frown.
E a c h c o m p o n e n t of a p o i n t ' s p o s i t i o n is
c o m p u t e d u s i n g the f o l l o w i n g a l g o r i t h m . c u r r e n t p o s i t i o n = p o s i t i o n in the p r e v i o u s p h a s e + C * d i f f e r e n c e w h e r e
d i f f e r e n c e = p o s i t i o n in n e x t p h a s e - p o s i t i o n in p r e v i o u s p h a s e C = ( i . 0 - cos (~))/2.0
a n d
= p h a s e f r a c t i o n * 3 . 1 4 1 5 9 D A T A A C Q U I S I T I O N
M e a s u r i n g the t h r e e - d i m e n s i o n a l p o s i - t i o n of p o i n t s on the s u r f a c e of a f a c e or a n y o t h e r c o m p l e x o b j e c t is a s i g n i f i c a n t p r o b l e m .
If w e s t i l l h a v e o u r a s s i s t a n t w h o s e f a c e is p a i n t e d w i t h p o l y g o n s , w e a s k the a s s i s t a n t to a s s u m e a n u m b e r of d i f f e r e n t e x p r e s s i o n s . W e " f r e e z e " e a c h e x p r e s s i o n p h o t o g r a p h i c a l l y . F o r e a c h e x p r e s s i o n a
F i g u r e 4 T w o p h a s e s of a face.
F i g u r e 5
A t y p i c a l p a i r of d a t a p h o t o g r a p h s . p a i r of o r t h o g o n a l v i e w s of the face is taken, one f r o m d i r e c t l y in f r o n t a n d o n e f r o m the side. F i g u r e 5 s h o w s a t y p i c a l p a i r of t h e s e p h o t o g r a p h s . U s i n g t h e s e p a i r s of p h o t o g r a p h s w e e s t a b l i s h an o r i g i n a n d a c o o r d i n a t e system. T h e t h r e e - d i m e n - s i o n a l p o s i t i o n of e a c h p o i n t is m e a s u r e d d i r e c t l y f r o m the p h o t o g r a p h s .
N o t e t h a t the c o o r d i n a t e s y s t e m s h o u l d be c h o s e n s u c h t h a t two of the c o o r d i n a t e a x e s d e f i n e the synunetry p l a n e of the face.
T h i s f a c i l i t a t e s the m i r r o r i n g or r e f l e c - t i o n o p e r a t i o n n e c e s s a r y to o b t a i n the d a t a for the o t h e r h a l f of the face.
T h i s d a t a c o l l e c t i o n m e t h o d has s o m e s h o r t c o m i n g s . P h o t o g r a p h s a r e n o t o r t h o - g r a p h i c p r o j e c t i o n s b u t are p e r s p e c t i v e p r o j e c t i o n s . T h e r e f o r e , the i m a g e s on the p h o t o g r a p h s are s o m e w h a t d i s t o r t e d . T h i s d i s t o r t i o n can be r e d u c e d by u s i n g long f o c a l l e n g t h l e n s e s w h e n the p i c - t u r e s are taken. S o m e a d j u s t m e n t of the d a t a m a y be n e c e s s a r y d u e to this d i s - tortion. A n o t h e r s h o r t c o m i n g is t h a t some p o i n t s on the face m a y n o t be v i s i b l e in b o t h views. A b e s t g u e s s m u s t be m a d e for at l e a s t o n e of the c o o r d i n a t e s of t h e s e o c c l u d e d p o i n t s .
T H E A N I M A T I O N P R O G R A M
The a n i m a t i o n p r o g r a m c o n t a i n s a r r a y s to s t o r e the t o p o l o g y a n d p h a s e d a t a for the face. Up to t h r e e p h a s e s m a y be s t o r e d in the p r o g r a m . T h e t o p o l o g y and p h a s e d a t a is r e a d in f r o m d a t a files. T h e u s e r of the p r o g r a m i n t e r a c t s w i t h it to s p e c i f y w h i c h d a t a f i l e s he w a n t s r e a d in.
The p h a s e d a t a files c o n s i s t of the d a t a for a s e q u e n c e of p o i n t s . F o r e a c h
p o i n t t h e r e is a p o i n t n u m b e r a n d a t h r e e - d i m e n s i o n a l p o s i t i o n . T h e t o p o l o g y d a t a c o n s i s t s of a s p e c i f i c a t i o n for e a c h p o l y - g o n of the face. T h e p o l y g o n s p e c i f i c a - t i o n is m a d e up of the p o i n t n u m b e r s of the v e r t i c e s of the p o l y g o n a n d its color.
A f t e r the d e s i r e d d a t a is r e a d in, the d a t a for the o t h e r h a l f of the f a c e is c o n s t r u c t e d b y m i r r o r i n g or r e f l e c t i n g the d a t a f o r the f i r s t h a l f of the face.
F o r e a c h f r a m e of a s e q u e n c e a n u m b e r of t a s k s m u s t be a c c o m p l i s h e d in o r d e r to c o m p u t e the d a t a n e e d e d to p a s s o n to the h i d d e n s u r f a c e and s h a d i n g a l g o r i t h m s .
U s i n g the p h a s e n u m b e r a s s o c i a t e d w i t h e a c h frame, the p r o g r a m i n t e r p o l a t e s the p h a s e d a t a to g e t th e p o s i t i o n of e a c h v e r t e x of the skin for t h i s f r a m e .
T h e s e p o i n t p o s i t i o n s a r e s p e c i f i e d in a c o o r d i n a t e s y s t e m c e n t e r e d n e a r the c e n t e r of the head. T h e h i d d e n s u r f a c e a l g o r i t h m r e q u i r e s the d a t a to b e s p e c i - f i e d in a d i f f e r e n t c o o r d i n a t e system.
T h e n e w c o o r d i n a t e system, r e f e r e d to as the v i e w i n g system, h a s its o r i g i n at the p o s i t i o n w e w i s h to l o o k from. T h e Z a x i s of the v i e w i n g s y s t e m m u s t b e p o i n t - ing in the d i r e c t i o n w e w i s h to look.
T h e p o s i t i o n d a t a m u s t be t r a n s f o r m e d (12, 13) i n t o this n e w c o o r d i n a t e system. T h e a n i m a t i o n p r o g r a m f i r s t t r a n s l a t e s the d a t a so the o r i g i n m o v e s f r o m the c e n t e r of the h e a d to the p o s i t i o n w e w i s h to look from. It t h e n r o t a t e s the c o o r d i n a t e s y s t e m so t h a t the Z a x i s of the v i e w i n g s y s t e m is p o i n t i n g t o w a r d the p o s i t i o n w e w i s h to look at.
A f t e r the d a t a is t r a n s f o r m e d i n t o the v i e w i n g system, the n o r m a l to e a c h p o l y g o n is c o m p u t e d . U s i n g t h e s e n o r m a l s , the n o r m a l at e a c h v e r t e x of the s k i n is c o m p u t e d . T h i s is d o n e for e a c h v e r t e x b y a v e r a g i n g the n o r m a l s of the p o l y g o n s t h a t h a v e the v e r t e x in c o m m o n .
F o r e a c h f r a m e of a s e q u e n c e the fol- l o w i n g p a r a m e t e r s are p a s s e d o n to the h i d d e n s u r f a c e a n d s h a d i n g a l g o r i t h m s :
i. T h e v i e w i n g a n g l e (this is u s e d to d e t e r m i n e the v i e w i n g s p a c e a n d is a l s o u s e d in the p e r s p e c t i v e t r a n s f o r m a t i o n (12,13)).
2. T h e c o l o r of e a c h p o l y g o n . 3. T h e p o s i t i o n of the b e g i n n i n g a n d e n d i n g p o i n t s of e a c h e d g e of e a c h p o l y g o n a n d the n o r m a l s at t h e s e p o i n t s .
4. T h e d e s i r e d r e s o l u t i o n .
The a n i m a t i o n p r o g r a m w a s w r i t t e n in S A I L (14), an e x t e n d e d a l g o l for the P D P - 1 0 .
I M P L E M E N T A T I O N
T h e s y s t e m u s e d to p r o d u c e a n i m a t e d s e q u e n c e s , a n d the h a l f - t o n e r e n d e r i n g s i n c l u d e d in this p a p e r is s h o w n in F i g u r e 6. T h i s s y s t e m u s e s t w o P D P - 1 0 c o m p u t e r s . O n e of t h e s e is a d e d i c a t e d m a c h i n e t h a t a l l o w s o n l y one u s e r at any g i v e n time.
T i m e - S h a r e d P D P - 1 0
B u f f e r
A n i m a t i o n P r o g r a m
L i n k
S i n g l e - U s e r P D P - 1 0
I W a t k i n s ' ~ G o u r a u d ' s V i s i b l e S m o o t h
S u r f a c e S h a d i n g
I Alqorithm Al~lorithm
F i l m A d v a n c e /
S h u t t e r C o n t r o l
C o l o r F i l t e r B o x
x 0 1 A
Y D/A I
Z D / A
St°•r•Tube
\ C R T / "
i
- - - J
F i g u r e 6 S y s t e m c o n f i g u r a t i o n .
T h i s p r o c e s s o r is i n t e r f a c e d to the s p e c i a l e q u i p m e n t n e e d e d to p r o d u c e h a l f - t o n e p i c - tures. T h e o t h e r P D P - 1 0 is t i m e - s h a r e d , a n d r u n s u n d e r the T E N E X o p e r a t i n g system.
T h e r e is a link b e t w e e n the m a c h i n e s t h a t a l l o w s d a t a to be t r a n s f e r e d b e t w e e n them.
T h i s s y s t e m a l l o w s us to t a k e a d v a n - t a g e of the T E N E X o p e r a t i n g s y s t e m , p a r t i c - u l a r l y the f i l e system, on one m a c h i n e and the s p e c i a l h a l f - t o n e d i s p l a y e q u i p m e n t on the o t h e r m a c h i n e .
W h e n o n e w i s h e s to u s e this system, he c o n n e c t s the t e r m i n a l to the s i n g l e - u s e r P D P - 1 0 and l o a d s a p r o g r a m w h i c h c o n t a i n s ; a s o f t w a r e v e r s i o n of W a t k i n ' s v i s i b l e sur- face a l g o r i t h m , G o u r a u d ' s s m o o t h s h a d i n g a l g o r i t h m , a p r o c e d u r e to c a l i b r a t e the h a l f - t o n e d i s p l a y s a n d p r o c e d u r e s to h a n d l e the s i n g l e - u s e r side of the d a t a t r a n s f e r s a c r o s s the link. W h e n this p r o g r a m b e g i n s e x e c u t i o n , it f i r s t a l l o w s the u s e r to cal- i b r a t e the d i s p l a y e q u i p m e n t . It t h e n ini- t i a l i z e s the r e c e i v i n g s i d e of the d a t a link a n d g o e s to sleep. A t this p o i n t the u s e r s w i t c h e s the t e r m i n a l to the time- s h a r e d m a c h i n e . A f t e r l o g g i n g in, the u s e r l o a d s a n d s t a r t s the e x e c u t i o n of the ani- m a t i o n p r o g r a m . T h i s p r o g r a m a s k s the u s e r a n u m b e r of q u e s t i o n s , i n c l u d i n g : w h i c h p h a s e d a t a f i l e s he w a n t s r e a d in, the d e s i r e d n u m b e r of f r a m e s b e t w e e n p h a s e s , the r e s o l u t i o n to use, w h e r e he w a n t s to look, w h e r e he w a n t s to l o o k from, and w h e t h e r he w a n t s s m o o t h or p o l y g o n a l shading. A f t e r r e c e i v i n g this i n f o r m a t i o n , the p r o g r a m b e g i n s p r o c e s s i n g the f i r s t f r a m e of the s e q u e n c e .
D a t a to be p a s s e d to the h i d d e n sur- face a n d s h a d i n g a l g o r i t h m s is s t o r e d i n t o one of two b u f f e r s . W h e n a b u f f e r is full,
a flag is set. T h i s c a u s e s the a n i m a t i o n p r o g r a m to t r a n s f e r to the o t h e r b u f f e r and w a k e s up the s i n g l e - u s e r p r o g r a m . W h e n the s i n g l e - u s e r p r o g r a m w a k e s up, it t r a n s f e r s the d a t a o u t of the f u l l b u f f e r i n t o its m e m o r y , r e s e t s the f l a g a n d g o e s b a c k to sleep. R e s e t t i n g the b u f f e r f l a g a l l o w s the a n i m a t i o n p r o g r a m to r e u s e the b u f f e r .
W h e n a l l the d a t a for a s i n g l e f r a m e h a s b e e n t r a n s f e r e d , the s i n g ! e - u s e r p r o - g r a m b e g i n s w o r k i n g on it to g e n e r a t e the h a l f - t o n e image. T h e a n i m a t i o n p r o g r a m g o e s on to the n e x t f r a m e of the s e q u e n c e . W h i l e the s i n g l e - u s e r p r o g r a m is in the p r o c e s s of g e n e r a t i n g the h a l f - t o n e image, it i g n o r e s the b u f f e r f l a g s set b y the a n i - m a t i o n p r o g r a m . T h i s m e a n s t h a t as s o o n
as the a n i m a t i o n p r o g r a m f i l l s b o t h b u f f e r s it m u s t w a i t u n t i l the s i n g l e - u s e r p r o g r a m c o m p l e t e s the p i c t u r e a n d e m p t i e s a
b u f f e r .
The o u t p u t of the s i n g l e - u s e r p r o g r a m g o e s to t h r e e d i g i t a l ~ t o - a n a l o g c o n v e r t e r s w h i c h in t u r n d r i v e a n y c o m b i n a t i o n of the d i s p l a y d e v i c e s s h o w n in F i g u r e 6.
T h i s s y s t e m w o r k s w e l l if the t i m e - s h a r e d s y s t e m is n o t h e a v i l y loaded. If the t i m e - s h a r e d s y s t e m is h e a v i l y l o a d e d the s i n g l e - u s e r is i d l e m u c h of the t i m e w a i t i n g for data.
A l l of the h a l f - t o n e r e n d e r i n g s s h o w n in this r e p o r t w e r e p r o d u c e d w i t h the h i g h p r e c i s i o n d i s p l a y u s i n g a r e s o l u t i o n of i 0 2 4 x i 0 2 4 . A t this r e s o l u t i o n it t a k e s a b o u t 2% m i n u t e s to s c a n o u t a s i n g l e b l a c k - a n d - w h i t e p i c t u r e .
A n i m a t e d s e q u e n c e s are r e c o r d e d u s i n g a 35 ram a n i m a t i o n camera. F i l m a d v a n c e and
shutter are under p r o g r a m control. Ani- mated sequences of the face are p r o d u c e d at the rate of about 20 frames per hour.
A C K N O W L E D G M E N T S
I am grateful to Professors R. E.
Stephenson and I. E. Sutherland and to Barry Wessler and Ed C a t m u l l for their"
help and encouragement, also to M i k e M i l o c h i k for his p h o t o g r a p h i c assistance.
REFERENCES
1. Coons, S. A., "Surfaces for C o m p u t e r Aided Design of Space Forms", M.I.T., Cam- bridge, Mass., P r o j e c t MAC Report MAC-TR- 41, June 1967.
2. Mahl, R., "Visible Surface A l g o r i t h m for Quadric Patches", Computer Science, U n i v e r s i t y of Utah, Technical Report UTEC- CSc-70-iii, December 1970.
3. Weiss, R. A., "Be Vision, A Package of IBM 7090 F o r t r a n Programs to Draw Ortho- graphic Views of C o m b i n a t i o n s of Plane and Quadric Surfaces", JACM, vol. 13, April 1966, pp. 194-204.
4. Gouraud, H., "Computer Display of Curved Surfaces", C o m p u t e r Science, Univer- sity of Utah, T e c h n i c a l Report UTEC-CSc- 71-113, June 1971.
5. Wylie, C., Romney, G., Evans, D., and Erdahl, A., "Half-tone P e r s p e c t i v e D r a w i n g by Computer", Proc FJCC, vol. 31, pp.
49-58, 1967.
6. Appel, A., "The Notion of Q u a n t i t a t i v e Invisibility and the M a c h i n e R e n d e r i n g of Solids", Proc ACM, vol. 14, pp. 387-393, 1967.
7. Kelley, K. C., "A Computer P r o g r a m for the G e n e r a t i o n of Half-Tone Images w i t h Shadows", C o o r d i n a t e d Science Laboratory, U n i v e r s i t y of Illinois, Report R-444, N o v e m b e r 1969.
8. Romeny, G. W., "Computer A s s i s t e d A s s e m b l y and R e n d e r i n g of Solids", Rome Air D e v e l o p m e n t Center, Griffiss A i r Force Base, New York, Technical Report RADC-TR-69-365, September 1969.
9. Rougelot, R. S. and Shoemaker, R.,
"G. E. Real Time Display", General Elec- tric Co., Syracuse N. Y., NASA Report NAS 9-3916.
10. Warnock, J. E., "A Hidden Surface A l g o r i t h m for Computer G e n e r a t e d Halftone Pictures", Computer Science, U n i v e r s i t y of Utah, Technical Report 4-15, June 1969.
ll. Watkins, G. S., "A Real-Time V i s i b l e Surface Algorithm", Computer Science,
U n i v e r s i t y of Utah, Technical Report UTECH-CSc-70-101, June 1970.
12. Coons, S. A., "Transformations and Matrices", Notes for the 1967 Summer School on C o m p u t e r Graphics for Designers,
U n i v e r s i t y of Michigan, June 5-16, 1967.
13. Ahuja, D. V. and Coons, S. A., "Geo- m e t r y for C o n s t r u c t i o n and Display", IBM S [ s t e m s J o u r n a l , vol. 7, pp. 188-205,
1968.
14. Swinehart, D. and Sproull B., "SAIL", Stanford A r t i f i c i a l I n t e l l i g e n c e P r o j e c t O p e r a t i n g Note No. 57.1, April 1970.
