On-Road Driver Eye Movement Tracking Using Head-Mounted Devices
M. Sodhi,* B. R e i m e r , J. L. C o h e n E. V a s t e n b u r g , It. K a a r s T h e U n i v e r s i t y of R h o d e I s l a n d , K i n g s t o n H o g e s c h o o l V a n A m s t e r d a m , H o l l a n d
S. K i r s c h e n b a u m
N a v a l U n d e r s e a W a r f a r e L a b , N e w p o r t
Abstract
It is now evident from anecdotal evidence a n d p r e l i m i n a r y research t h a t d i s t r a c t i o n s can h i n d e r the t a s k of o p e r a t i n g a vehicle, a n d c o n s e q u e n t l y reduce driver safety. However with increasing wireless c o n n e c t i v i t y a n d t h e p o r t a b i l i t y of office devices, the vehicle of t h e f u t u r e is visualized as a n extension of t h e s t a t i c work place - i.e. a n office-on-the- move, w i t h a phone, a fax m a c h i n e a n d a c o m p u t e r all w i t h i n the reach of the vehicle operator. For this research a Head m o u n t e d E y e - t r a c k i n g Device (HED), is used for t r a c k i n g the eye movements of a driver n a v i g a t i n g a test route in a n a u t o m o b i b l e while c o m p l e t i n g various d r i v i n g tasks. Issues arising from d a t a collection of eye m o v e m e n t s d u r i n g t h e c o m p l e t i o n of various d r i v i n g tasks as well as t h e analysis of this d a t a are discussed. Methods for collecting video a n d s c a n - p a t h data, as well as difficulties a n d l i m i t a t i o n s are also reported.
Keywords: P e r c e p t u a l reasoning, Ergonomics, C a m e r a cal- ibration, Tracking
1 Introduction
Because of t h e increasing presence of In-Vehicle I n f o r m a t i o n Systems (IVIS) i n m o d e r n vehicles, questions are now b e i n g raised a b o u t t h e i m p a c t of these systems u p o n vehicle safety.
M a n u a l , visual, a n d a u d i t o r y m e t h o d s are b e i n g used to in- teract with various in-vehicle devices such as radios, com- pact disk players, cell phones, laptop, p a l m t o p computers, collision avoidance, global p o s i t i o n i n g n a v i g a t i o n systems, speech based e-mail a n d other m o d e r n in.formation equip- m e n t . These devices provide obvious benefits to the driver - however costs associated with changes i n driver workload a n d m o n i t o r i n g efficiency are n o t so clear. W h i l e thereis lit- tle a r g u m e n t t h a t visual, auditory, biomechanical, a n d cog- nitive distractions result from the use of these devices, t h e safety implications are difficult to quantify. To improve ve- hicular safety, some m e t h o d for t h e assessment of these dis- t r a c t i o n s is required ([Ranney et al. 2000] a n d [Serafin et al.
1993]). However, t h e exact p a r a m e t e r s with which a mini- m u m threshold can be defined are n o t yet fully u n d e r s t o o d
*aodhiOegr.uri.edu
Permission to make digital or hard copies of all or pad of this work for personal or classroom use is granted without fee provided that copies am not made or distnbuted for profit or commer'cJal advantage and ~ a t copies bear this notice and the full citation on the tirst page. To copy otherwise, to republish, to pest on servers or to redistribute to lists, requires pdor specific permission and/or a fee.
ETRA'02 N e w Odeans Louisiana USA
Copyright ACM 2002 1.-58113-4,47-3/02/03...$5.00
([Lee et al. 2000] a n d [Hankey et al. 2000]). A recent s t u d y [Wierwille a n d T i j e r i n a 1998] on developing formal defini- tions of t h e level of a t t e n t i o n required in o p e r a t i n g in-vehicle devices f o u n d t h a t "the a m o u n t a n d frequency of visual at- t e n t i o n to in-vehicle devices is directly safety relevant". A n - other s t u d y ([Vollrath a n d Totzke 2000]) on in-vehicle com- m u n i c a t i o n m e t h o d s concludes t h e m o d e of c o m m u n i c a t i o n w i t h in-vehicle devices does result i n differences i n levels of e n c r o a c h m e n t on t h e driver's a t t e n t i o n . C a r radios have b e e n largely accepted as a n acceptable d r i v i n g d i s t r a c t i o n while other in-vehicle i n f o r m a t i o n systems have n o t b e e n so readily accepted ([McKnight a n d Mclmight 1991] a n d [Cain and Burris 1999]). F u n d a m e n t a l l y , a d i s t r a c t i o n is a n y t h i n g t h a t takes a t t e n t i o n away from t h e p r i m a r y task. I n t h e case of a driver, this is a n y t h i n g t h a t takes t h e driver's a t t e n t i o n away from t h e d r i v i n g task. I n t h e absence of knowledge a b o u t t h e levels of d i s t r a c t i o n t h a t result from s e c o n d a r y tasks, it is difficult to predict what t h e effect of m u l t i p l e tasks is on t h e p r i m a r y (driving) task. W h i l e most drivers are risk adverse, t h e y are n o t fully aware of t h e involved risks w h e n m a k i n g decisions to use a n in-vehicle device. M a n y researchers have addressed problems r e l a t i n g to driver dis- t r a c t i o n s a n d a comprehensive review of t h e l i t e r a t u r e a n d s t u d i e s on various in-vehicle devices, completed i n [l:teimer a n d Sodhi 2002], is s u m m a r i z e d below.
1 . 1 E x i s t i n g R e s e a r c h I n t o T h e D i s t r a c t i o n P r o b l e m It has long b e e n recognized t h a t a n overload of informa- t i o n processing capacity causes problems with d r i v i n g per- formance [Matthews a n d Sparkes 1996]. E x i s t i n g research provides a n u m b e r of examples of overload. I n [Brown et al, 1969] it is d e m o n s t r a t e d t h a t c o n c u r r e n t p e r f o r m a n c e of a n a u d i t o r y t a s k impairs j u d g m e n t s of whether t h e car c a n be d r i v e n t h r o u g h a narrow gap. I n [Harms 1991] m e n t a l arith- metic p e r f o r m a n c e is shown to be sensitive to t h e d e m a n d s of t h e d r i v i n g task. I n recent years, there has b e e n a n increase in the variety of in-vehicle e q u i p m e n t available to drivers.
Cassette players a n d radios are s t a n d a r d in most cars. Mo- bile telephones are now widespread, a n d lately n a v i g a t i o n a l a n d r o u t e guidance e q u i p m e n t , real-time i n f o r m a t i o n sys- terns etc. have b e e n i n t r o d u c e d into the "driver space". A n u m b e r of investigations have b e e n directed at e v a l u a t i n g t h e effects of such e q u i p m e n t o n d r i v i n g performance. Cog- nitive load problems have b e e n related to t h e use of mobile phones i n various ways - actions such as p h o n e conversa- tions, holding t h e phone, a n d dialing while d r i v i n g ([McK- n i g h t a n d Mcknight 1991] a n d [Cain a n d Burris 1999]) have a n i m p a c t u p o n t h e driver's a t t e n t i o n in different ways. I n [McKnight a n d Mcknight 1991] a n i n t e n s e business conver- s a t i o n is shown to differ from a social c o n v e r s a t i o n in t h e cognitive load placed on t h e driver while o p e r a t i n g a ve- hicle. It appears t h a t h a n d s free conversations do result i n reduced cognitive loads, nevertheless there is a n increase i n the load compared w i t h n o r m a l d r i v i n g ([Pachiaudi a n d
61
C h a p o n 1994] a n d [Cain a n d B u r r i s 1999]). T h e r e l a t i v e risk of d r i v i n g w i t h a cell p h o n e has b e e n r e p o r t e d as c o m p a r a - ble w i t h t h e h a z a r d a s s o c i a t e d w i t h d r i v i n g w h i l e i n t o x i c a t e d ([1997]) a n d it is s t a t e d t h a t " t h e r e l a t i v e risk is s i m i l a r t o t h e h a z a r d a s s o c i a t e d w i t h d r i v i n g w i t h a b l o o d a l c o h o l level a t t h e l e g a l l i m i t " . I t has also b e e n s h o w n t h a t t h e r i s k as- s o c i a t e d w i t h a p h o n e c o n v e r s a t i o n w h i l e d r i v i n g d o e s n o t e n d w i t h t h e call ( [ R e d e l m e i e r a n d T i b s h i r a n i 1997]), a n d it is c o n j e c t u r e d t h a t t h e s u s t a i n e d risk is d u e t o t h e d r i v e r b e i n g m e n t a l l y o c c u p i e d w i t h t h e p a s t c o n v e r s a t i o n a f t e r it is a c t u a l l y over.
T h i s p a p e r focuses on t h e use of eye t r a c k i n g m e t h o d s t o m o n i t o r h o w v a r i o u s d i s t r a c t e r s affect a driver, a s s u m i n g a r e l a t i o n s h i p b e t w e e n eye m o v e m e n t s a n d a t t e n t i o n . W h i l e t h i s m o d e o f t r a c k i n g a t t e n t i o n h a s b e e n u s e d in t h e p a s t , r e c e n t a d v a n c e s in t r a c k i n g h a r d w a r d a n d s o f t w a r e , as well as i n c r e a s e s in c o m p u t i n g p o w e r h a v e n o w m a d e m a d e at- t e m p t s at m o n i t o r i n g d r i v e r a t t e n t i o n v i s - a - v i s e y e m o v e - m e n t t r a c k i n g feasible for r e a l t i m e analysis. E v e n so, t h e r e are s e v e r a l issues t h a t m u s t b e a d d r e s s e d w h e n u s i n g eye m o v e m e n t t r a c k i n g , a n d s o m e o f t h e s e a r e d e t a i l e d in t h i s p a p e r .
1.2 Eye Movements and Attention
T h e h u m a n o c u l o m o t o r s y s t e m is c o n t r o l l e d d i r e c t l y f r o m a s e c t i o n o f t h e b r a i n s t e m t h r o u g h t h r e e p a i r s of e x t r a o c u l a r muscles~ e a c h r e s p o n s i b l e for o n e o f t h e t h r e e d i r e c t i o n s of eye m o v e m e n t ; h o r i z o n t a l , v e r t i c a l , a n d t o r s i o n a l . T h e m o s t b a s i c m o v e m e n t , for w h i c h no s e l e c t i v e f u n c t i o n exists, is c a l l e d p h y s i o l o g i c a l n y s t a g m u s w h i c h is c a u s e d i n v o l u n t a r - ily b y t r e m o r s in t h e e x t r a o c u l a r muscles, r e s u l t i n g in slight shifts of t h e o c u l a r i m a g e in r e l a t i o n t o t h e r e t i n a . A l l t h e r e m a i n i n g eye m o v e m e n t s h a v e o n e t h i n g in c o m m o n - t h e y h a v e s o m e f u n c t i o n in c o n t r o l l i n g w h e r e t h e eyes fixate. A S a c c a d i c eye m o v e m e n t is t h e f u n d a m e n t a l s e a r c h m o v e m e n t for t h e eye. I t c a n b e b e s t d e s c r i b e d as a b a l l i s t i c m o t i o n t o p r o p e l t h e eye t o a n e w o b j e c t of i n t e r e s t . S a c c a d e s a r e f u r t h e r d e s c r i b e d as p r e p r o g r a m m e d m o v e m e n t s , a n d o n c e i n i t i a t e d , t h e i r p a t h or d e s t i n a t i o n c a n n o t b e v o l u n t a r i l y al- t e r e d . A c e r t a i n a m o u n t of t i m e , b e t w e e n 150 a n d 200 m s c a n b e a t t r i b u t e d to p l a n n i n g a n d e x e c u t i n g a s a c c a d e , w h i l e t h e a c t u a l m o v e m e n t o n l y t a k e s a m a x i m u m o f 30 m s w h i l e r e a c h i n g a m a x i m u m s p e e d o f u p t o 900 d e g p e r sec. T h e t i m e b e t w e e n s a c c a d e s is w h e n t h e eye p r o c e s s e s v i s u a l in- f o r m a t i o n b y f i x a t i n g o n a t a r g e t .
W h e n t h e o b j e c t o f i n t e r e s t is in m o t i o n , a t h i r d t y p e of eye m o v e m e n t k n o w n as a s m o o t h p u r s u i t m o v e m e n t is u s e d as o p p o s e d t o a s e q u e n c e o f s a c c a d e s a n d f i x a t i o n s . T h e p u r p o s e o f a s m o o t h p u r s u i t m o v e m e n t is t o t r a c k a m o v - ing o b j e c t a n d keep it in foveal v i e w o n c e a s a c c a d e p l a c e t h e o b j e c t in focus. T h i s m o v e m e n t allows for v i s u a l infor- m a t i o n t o b e e x t r a c t e d f r o m a m o v i n g t a r g e t . I t f u n c t i o n s u s i n g a f e e d b a c k p r o c e s s t h a t c o n s t a n t l y uses i n f o r m a t i o n r e l a t e d t o t h e s p e e d o f t h e m o v i n g o b j e c t t o p r e d i c t w h e r e t o m o v e t h e eyes. I n o r d e r t o keep a n o b j e c t c e n t e r e d in t h e f o v e a o f b o t h eyes a f o u r t h t y p e of m o v e m e n t k n o w n as v e r g e n c e m o v e m e n t s are u s e d i n c o n j u n c t i o n w i t h s m o o t h m o v e m e n t s . V e r g e n c e m o v e m e n t s are slow, 10 d e g p e r sec, d i s c o n j u g a t e m o v e m e n t s u s e d t o select t h e d i s t a n c e o f t h e t a r g e t b y a l i g n i n g t h e o b j e c t i n t h e c e n t e r o f b o t h fovea.
A d i s c o n j u g a t e m o v e m e n t is d e s c r i b e d as a s i t u a t i o n w h e r e e a c h eye is l o o k i n g in d i f f e r e n t d i r e c t i o n s . T w o final invol- u n t a r y e y e m o v e m e n t s a r e v e s t i b u l a r a n d o p t o k i n e t i c m o v e - m e n t s , w h i c h w o r k in c o n j u n c t i o n t o k e e p a n o b j e c t i n v i e w w h e n t h e h e a d m o v e s . V e s t i b u l a r m o v e m e n t s are t r i g g e r e d b y signals f r o m t h e i n n e r e a r t o o p p o s e r o t a t i o n a l m o v e m e n t ~
w h i l e o p t o k i n e t i c m o v e m e n t s are t r i g g e r e d b y o p t i c a l t r a n s - l a t i o n s o p p o s i n g u n i f o r m m o v e m e n t s in t h e v i s u a l field.
E y e m o v e m e n t s in m a n y cases are a s s u m e d t o b e p r e d i c - t o r s of a t t e n t i o n . A n e y e - t r a c k i n g s y s t e m s c a n t h e r e f o r e b e u s e d t o collect i n f o r m a t i o n a b o u t h o w a d r i v e r r e s p o n d s t o d i f f e r e n t s i t u a t i o n s o n or off t h e r o a d [ R e c a r t e a n d N u n e s 2000]. T w o different m o d e l s for eye m o v e m e n t s a n d a t t e n - t i o n a r e d e s c r i b e d in [ H e n d e r s o n 1993]. I n t h e first a s e q u e n - t i a l a t t e n t i o n m o d e l d e s c r i b e s a t t e n t i o n as b e i n g " d i r e c t e d t o t h e specific l o c a t i o n t o w a r d w h i c h t h e eyes will m o v e p r i o r t o a s a c c a d i c eye m o v e m e n t " . I n t h e s e c o n d m o d e l , a t t e n t i o n is d e s c r i b e d as b e i n g " a l l o c a t e d t o all l o c a t i o n s in t h e g e n e r a l d i r e c t i o n of t h e i m p e n d i n g s a c c a d e r a t h e r t h e n t o t h e specific t a r g e t l o c a t i o n " . S i m i l a r f i n d i n g s r e l a t i n g eye m o v e m e n t s d i r e c t l y t o a t t e n t i o n a l shifts for " o r d i n a r y a c t i v i t i e s " h a v e b e e n f o u n d b y [ R e c a r t e a n d N u n e s 2000]. S a a r i n e n [Saari- n e n 1993] f o u n d t h a t " o b s e r v e r s c o u l d n o t s h i f t a t t e n t i o n a w a y f r o m f i x a t i o n t o an e x t r a f o v e a l p o s i t i o n as efficiently as t h e y c o u l d m a i n t a i n a t t e n t i o n at f i x a t i o n " . H o w e v e r t h e r e is o n e difficulty t h a t n e e d s t o b e r e m e m b e r e d . U n t i l s o m e p e r i o d b e f o r e t h e s a c c a d i c m o v e m e n t s t a r t s , m o d e l s of eye m o v e m e n t a n d a t t e n t i o n p r e s u m e t h a t t h e m o v e m e n t c a n b e c a n c e l e d [Salvucci 2000]. It is t h e r e f o r e e a s y t o c o n c e i v e t h a t s i t u a t i o n s e x i s t w h e r e a t t e n t i o n is d i r e c t e d o n o b j e c t s b u t no eye m o v e m e n t is e v e r e x e c u t e d t o b r i n g it i n t o v i e w . I n o r d e r t o s i m p l i f y t h e p r o b l e m o f u n d e r s t a n d i n g w h e r e at- t e n t i o n is f o c u s e d t h e s e s i t u a t i o n s are a s s u m e d t o b e u n i m - p o r t a n t . W h i l e eye m o v e m e n t s are n o t a p e r f e c t i n d i c a t i o n o f c o g n i t i v e p r o c e s s , t h e y are a " g o o d i n d e x o f t h e m o m e n t t o m o m e n t o n - l i n e p r o c e s s i n g a c t i v i t i e s t h a t a c c o m p a n y vi- s u a l c o g n i t i v e t a s k s " [ R a y n e r 1992]. I n t h e p a s t , r e s e a r c h e r s h a v e u s e d eye m o v e m e n t s as a n i n s i g h t t o p e r s o n ' s t h o u g h t s a n d i n t e n d e d a c t i o n s [ M c K n i g h t a n d M c k n i g h t 1991]. M o r e recently, t h e focus h a s s h i f t e d i n t o m o d e l i n g b e h a v i o r p a t - t e r n s b a s e d u p o n eye m o v e m e n t s [Salvucci a n d A n d e r s o n 1998]. I n e i t h e r case i n f o r m a t i o n a b o u t w h e r e a s u b j e c t is l o o k i n g m u s t b e c o l l e c t e d a t h i g h f r e q u e n c y , t o c a p t u r e a n y s u d d e n c h a n g e s in a p e r s o n s a c t i o n s .
E y e m o v e m e n t s r e c o r d e d a t h i g h f r e q u e n c i e s c a n give i m - p o r t a n t clues t o h u m a n b e h a v i o r . A g r e a t e r u n d e r s t a n d i n g o f w h a t i n f o r m a t i o n p e o p l e use in p r o b l e m s o l v i n g c a n b e de- t e r m i n e d b y h o w l o n g it t a k e s t o p r o c e s s in_formation. B a s i c w o r k in t h e field was c o m p l e t e d in [ J u s t a n d C a r p e n t e r 1976]
w h e r e t h e r e l a t i o n s h i p b e t w e e n t h e locus, d u r a t i o n , a n d se- q u e n c e of eye f i x a t i o n s a n d t h e a c t i v i t y o f t h e c e n t r a l p r o c e s - sor was i n v e s t i g a t e d . E y e m o v e m e n t d a t a l o o k s a t w h e r e a n i n d i v i d u a l is c o l l e c t i n g v i s u a l i n f o r m a t i o n o v e r a v e r y s m a l l scale c u r r e n t l y in t h e r a n g e of 50 t o 400 H Z . H i g h e r c o l l e c t i o n r a t e s a r e o n l y a v a i l a b l e in s y s t e m s w h e r e t h e s u b j e c t s h e a d is fixed. C o m m e r c i a l h e a d m o u n t e d eye t r a c k e r s c a n c u r r e n t l y collect eye m o v e m e n t s in t h e r a n g e o f 50 t o 240 H Z w h i l e a v i d e o r e c o r d i n g o f t h e f o r w a r d s c e n e c a n b e m a d e a t 30 HZ. T r a d i t i o n a l m e t h o d s o f a n a l y z i n g e y e m o v e m e n t s h a v e f o c u s e d l a r g e l y o n s e p a r a t i n g f i x a t i o n s f r o m s a c c a d e s b a s e d u p o n v e l o c i t i e s , a g g r e g a t i o n of c o n s e c u t i v e p o i n t s w i t h d u r a - t i o n m i n i m u m s , a n d d i g i t a l f i l t e r i n g [Salvucci 1999]. M a n u a l m e t h o d s c a n t h e n b e u s e d t o i d e n t i f y w h a t a d r i v e r is f i x a t - i n g on. A r e c e n t t e c h n i q u e t o a u t o m a t e t h i s p r o c e s s i n v o l v e s t r a c i n g f i x a t i o n s . F i x a t i o n t r a c i n g is " t h e p r o c e s s of m a p p i n g o b s e r v e d a c t i o n p r o t o c o l s t o t h e s e q u e n t i a l p r e d i c t i o n s o f a c o g n i t i v e p r o c e s s m o d e l " [Salvucci 1999]. S a l v u c e i ( [ S a l v u c c i a n d A n d e r s o n 1998], [Salvucci 1999]) p r e s e n t s a n e x t e n s i v e r e v i e w of c u r r e n t m e t h o d s o f t r a c i n g eye m o v e m e n t s , a n d d e v e l o p s t h r e e n e w t e c h n i q u e s b a s e d u p o n M a r k o v m o d e l s . T h e m o d e l s h o w e v e r are l i m i t e d t h e i r a p p l i c a t i o n for s t u d y - i n g i n - v e h i c l e d e v i c e s b e c a u s e of t h e i r a s s u m p t i o n t h a t " t h e t a s k e n v i r o n m e n t in w h i c h e y e - m o v e m e n t d a t a are c o l l e c t e d
is (at least for t h e m o s t p a r t ) s t a t i c " . In t h e c o n t e x t of t h e a u t o m o b i l e t h e scenery o u t s i d e t h e vehicle is c o n s t a n t l y m o v i n g . T h e driver is t r a c k i n g o t h e r vehicle, signs, and ob- j e c t s o u t s i d e t h e vehicle w i t h s m o o t h m o v e m e n t s . To d e t e c t p a t t e r n s in d r i v e r ' s eye m o v e m e n t s new m e t h o d s of analysis n e e d to be developed.
To develop a new m e t h o d of a n a l y z i n g d y n a m i c scenes a n u n d e r s t a n d i n g of visual search in a d y n a m i c s e t t i n g is needed. W i t h t h e a d d i t i o n of s m o o t h eye m o v e m e n t s , t h e analysis of scan p a t h s for d y n a m i c scenes is m o r e difficult t h a n t h e a s s o c i a t e d p r o b l e m in a s t a t i c scene, since conclu- sions a b o u t eye m o v e m e n t s and t h e i r r e l a t i o n to t h e a c t u a l scene can only b e m a d e at discrete intervals r e l a t i n g to t h e r e c o r d i n g of t h e scene c a m e r a [Stark and Ellis 1981]. T h e low r e c o r d i n g r a t e s of scene cameras m a k e it necessary t o rely on eye m o v e m e n t s r e c o r d e d at higher frequencies to un- d e r s t a n d w h e r e t h e s u b j e c t is looking. D r i v i n g is one of m a n y tasks t h a t occurs in a d y n a m i c s e t t i n g a n d t h e r e f o r e exhibits this p r o b l e m . Drivers, l i m i t e d by t h e i r visual re- sources, c a n only focus on a single s t i m u l u s a n d search up to t h r e e t a r g e t s a second effectively [Moray 1990]. F r e q u e n t l y t h e n e e d arises to c o n c u r r e n t l y m o n i t o r m a n y different vi- sual stimuli such as t h e s p e e d o m e t e r , r e a r v i e w mirror, a car in front, t o t h e side, or o t h e r aspects of t h e visual scene n o t r e l a t e d to driving. W h e n visual resources axe a l l o c a t e d t o s e c o n d a r y tasks a decrease in t h e a m o u n t of visual resources a l l o c a t e d t o t h e d r i v i n g t a s k has t o o c c u r [ R u m a r 1988].
T i m e - s h a r i n g is u s e d as a m e t h o d of p a r t i a l l y o v e r c o m i n g this l i m i t a t i o n . W i t h t i m e - s h a r i n g i n d i v i d u a l v i s u a l tasks are c o m p l e t e d by sequences of saccadic m o v e m e n t s and fixa- tions. A f t e r e n o u g h i n f o r m a t i o n has b e e n a c q u i r e d f r o m one stimulus, a saccadic m o v e m e n t is e x e c u t e d , aligning a n o t h e r stimuli w i t h t h e c e n t r a l region of t h e fovea. T h e s e q u e n c e is r e p e a t e d over again u n t i l one of t h e tasks is c o m p l e t e [Wier- wille 1993]. T h e p r i m a r y s t i m u l u s in m a n y instances is t h e forward v i e w of t h e a u t o m o b i l e w i t h a r a n g e of s e c o n d a r y s t i m u l i c o m p e t i n g for t h e s p a r e visual c a p a c i t y [Rockwell 1988]. A p r o b l e m can o c c u r w h e n a driver chooses t o m o n i - t o r t o o m a n y s e c o n d a r y stimuli i n s t e a d of t h e p r i m a r y task, r e s u l t i n g in a lack of a t t e n t i o n t o t h e p r i m a r y task. T h e driver t h e r e f o r e c a n n o t i n t e r p r e t e n o u g h i n f o r m a t i o n f r o m t h e road.
Drivers n a t u r a l l y develop a safety m e c h a n i s m to c o u n t e r - act this p r o b l e m , by l i m i t i n g t h e a m o u n t of t i m e focus is d i r e c t e d off t h e r o a d for comfort, t o a m a x i m u m of approxi- m a t e l y 1.6 seconds [Wierwille 19931. D u e t o this l i m i t a t i o n a difficulty exists w h e n i n f o r m a t i o n needs to b e e x t r a c t e d f r o m highly complex, or u n k n o w n s e c o n d a r y tasks, such as t h e c l u t t e r e d d a s h b o a r d of a n e w car or dialing a cell phone.
In this s i t u a t i o n t h e t i m e to search and c o m p l e t e a t a s k m a y have t o b e longer t h a n t h e c o m f o r t limit. W h e n i n f o r m a t i o n is e x t r a c t e d f r o m c o m p l e x scene e x p e r i e n c e d drivers e x h i b i t a larger n u m b e r of eye m o v e m e n t s w i t h decreased f i x a t i o n l e n g t h [ C h a p m a n a n d U n d e r w o o d 1998].
O t h e r p r o b l e m s can be identified in a drivers visual field.
T h e visual field is a region of flexible size and s h a p e t h a t includes b o t h areas of direct focus a n d indirect focus. T h e useful area of t h e visual field or f u n c t i o n a l field of v i e w has b e e n d e s c r i b e d as " t h e area a r o u n d t h e fixation p o i n t f r o m which i n f o r m a t i o n is b e i n g briefly s t o r e d a n d r e a d o u t dur- ing a visual t a s k " [Williams 1988]. A r e l a t i o n s h i p b e t w e e n t h e size of t h e visual field and w o r k l o a d also exists, w h e n t o o m u c h i n f o r m a t i o n is b e i n g p r o c e s s e d t h e useful field of v i e w c o n t r a c t s t o p r e v e n t o v e r l o a d i n g of t h e visual s y s t e m ( [ R a n t a n e n a n d G o l d b e r g 1999], a n d [Miura 1990]). In ad- dition, a r e d u c t i o n in t h e m e a n gaze d u r a t i o n can be f o u n d [Miura 1990]. T h e r e d u c t i o n in visual field size c a n be r e l a t e d
t o two s e p a r a t e d p h e n o m e n a . T u n n e l vision, r e p r e s e n t e d by a clear r e d u c t i o n in a p e r t u r e angle of t h e visual cone, a n d a general decrease in p e r i p h e r a l visual p e r f o r m a n c e i n d e p e n - d e n t of t h e visual cone angle [l:tantanen and G o l d b e r g 1999].
Drivers affected by t h e s e changes rely on a g r e a t e r n u m b e r of s h o r t e r fixations t o d e t e c t and acquire i n f o r m a t i o n f r o m t a r g e t s ([Miura 1990] a n d [CrundaU et al. 1998]). C r u n d a l l et al. [Crundall et al. 1997] concludes t h a t slower r e a c t i o n t i m e s result. A n on t h e r o a d d r i v i n g study, using a c o m m e r - cial eye tracker as a m e t h o d of d e t e r m i n i n g w h e r e a s u b j e c t s a t t e n t i o n is focused, was c o m p l e t e d to f u r t h e r u n d e r s t a n d t h e effects t h a t d i s t r a c t i o n s h a v e on drivers as well as veri- fying t h e results of s o m e p r e v i o u s e x p e r i m e n t s .
2 Tracking Drivers Eye Movements in an On The Road Setting
T h e p u r p o s e of this i n v e s t i g a t i o n was t o a n a l y z e t h e gaze p a t t e r n of drivers w h e n t h e y are b e i n g d i s t r a c t e d . In t h e e x p e r i m e n t 24 people, w h o are all in possession of a valid d r i v e r ' s license, were asked t o drive a r o u t e of a b o u t 20 miles w i t h t h e H e a d m o u n t e d E y e - t r a c k i n g D e v i c e H E D on. It was r e c o m m e n d e d t h a t s u b j e c t s use t h e vehicle t h a t t h e y m o s t o f t e n drive, so t h a t t h e y would a l r e a d y be familiar w i t h t h e controls. D u r i n g t h e ride t h e s u b j e c t s were c o n f r o n t e d w i t h several distractions, w h i c h were p r e s e n t e d by one of t h e investigators. W h e n d r i v i n g p a s t a p r e - d e t e r m i n e d p o s i t i o n on t h e r o u t e for each d i s t r a c t i o n , t h e r e s e a r c h e r p l a y e d b a c k a p r e - r e c o r d e d C D t r a c k i n i t i a t i n g t h e specific d i s t r a c t i o n . T h e d i s t r a c t i o n s w h i c h t h e s u b j e c t s were c o n f r o n t e d w i t h were:
1. T u r n i n g on t h e radio a n d c h a n g i n g t h e s t a t i o n to 1610 A M .
2. N o t e t h e prices of gasoline f r o m a p p r o a c h i n g gas sta- tions.
3. A n s w e r i n g a p h o n e call w i t h o u t a h a n d s free device a n d c o m p l e t i n g a c o m p u t a t i o n a l task.
4. L o o k i n g in t h e r e a r v i e w m i r r o r and describe t h e vehicle t h a t is following.
5. A n s w e r i n g a h a n d s free p h o n e call a n d c o m p l e t i n g a m e m o r y task.
6. B e a d i n g t h e o d o m e t e r .
7. S t a r t l e s o u n d of a cellular p h o n e (3 rings).
2.1 System Setup
To o p e r a t e t h e s y s t e m in a car t h e following i t e m s were re- quired:
1. I V i e w o p e r a t o r P C .
2. C o m p u t e r m o n i t o r for eye level tracking.
3. H E D .
4. C D player and speakers.
5. U P S p o w e r pack.
T h e iView o p e r a t o r records a n d stores all t h e i n c o m i n g d a t a from t h e H E D . T h e m o n i t o r is necessary t o m a k e real t i m e a d j u s t m e n t s to t h e p u p i l a n d corneal reflection thresh- old. W h e n d r i v i n g in n a t u r a l light conditions, t h e light in- t e n s i t y changes w i t h cloud cover, t i m e of day, t r e e / b r i d g e cover etc. T h i s affects t h e c o n t r a s t of t h e pupil a n d corneal reflection. T h i s effect is m o s t p r o n o u n c e d w h e n it is sunny.
63
T h e r e f o r e , c l o u d y d a y s will g i v e b e t t e r eye d a t a . T w o re- s e a r c h e r s t r a v e l w i t h t h e s u b j e c t , inside t h e car. O n e re- s e a r c h e r o p e r a t e s t h e i V i e w P C a n d t h e o t h e r r e s e a r c h e r c o n t r o l s t h e d i s t r a c t i o n p r e s e n t a t i o n . D u r i n g an e x p e r i m e n - t a l r u n , t h e d i s t r a c t i n g r e s e a r c h e r is a s s i g n e d t h e f o l l o w i n g t a s k s :
1. A c t i v a t i n g t h e d i s t r a c t i o n s . 2. N o t e t h e traffic c i r c u m s t a n c e s . 3. A s k i n g p a r t i c u l a r q u e s t i o n s . 4. N o t i n g t h e a n s w e r s .
2.2 System Calibration
I n o r d e r t o c a p t u r e t h e g a z e p o s i t i o n c o r r e c t l y , t h e s y s t e m has t o b e c a l i b r a t e d for e a c h d r i v e r . T h i s is d o n e b y d e f i n i n g a n u m b e r of p o i n t s w h e r e t h e s u b j e c t m u s t f o c u s on. T h e n u m b e r o f p o i n t s a n d t h e i r p o s i t i o n s c a n b e d e f i n e d w i t h t h e i V i e w s o f t w a r e . T h e c a l i b r a t i o n p r o c e d u r e gives t h e o p t i o n o f b e i n g a b l e t o d e f i n e a 2-points~ 5 - p o i n t s , 9 - p o i n t s a n d a 13- p o i n t s c a l i b r a t i o n . T h e m o r e p o i n t s u s e d , t h e m o r e a c c u r a t e is t h e c a l i b r a t i o n a n d t h u s t h e g a z e p o s i t i o n . W h e n e n c o u n - t e r i n g difficulties d u r i n g c a l i b r a t i o n , a c o u p l e o f t h i n g s w e r e t r i e d t o f a c i l i t a t e c a l i b r a t i o n . F i r s t , t h e n u m b e r o f p o i n t s u s e d for c a l i b r a t i o n c a n b e d e c r e a s e d . A n o t h e r o p t i o n is t o d e c r e a s e t h e c a l i b r a t i o n a c c e p t level. T h e h i g h e r t h e a c c e p - t a t i o n level, t h e h a r d e r it is t o a c c e p t a g a z e p o s i t i o n . A s a r e s u l t t h e g a z e p o s i t i o n is m o r e a c c u r a t e . C o r r e s p o n d i n g l y , t h e lower t h e a c c e p t a t i o n level, t h e f a s t e r t h e s y s t e m will ac- c e p t a g a z e p o s i t i o n , b u t t h e c a l i b r a t i o n is less a c c u r a t e . T h e p o i n t s a r e d i s p l a y e d o n t h e c o m p u t e r screen. D r a g g i n g a n d d r o p p i n g t h e p o i n t s m a n u a l l y c a n c h a n g e t h e p l a c e m e n t o f t h e c a l i b r a t i o n p o i n t s . T h i s is v e r y c o n v e n i e n t w h e n a p o i n t is p l a c e d o n a l o c a t i o n w h e r e t h e s u b j e c t c a n h a r d l y see it or h a s t o t u r n t h e i r eyes in a n u n n a t u r a l p o s i t i o n . W i t h t h i s o p t i o n t h e p o i n t c a n b e s h i f t e d t o a b e t t e r l o c a t i o n . T o s o l v e t h e p r o b l e m o f c a l i b r a t i n g t h e s y s t e m w i t h t h e d r i v e r s e a t e d in t h e car, t h e c a r was d r i v e n in f r o n t o f a flat s u r f a c e s u c h as a g a r a g e d o o r . T h i s p r o v i d e s a fiat u n i f o r m w a l l w h e r e t h e c a l i b r a t i o n p o i n t s c o u l d b e easily l o c a t e d w i t h a l a s e r p o i n t e r or stick.
2.3 Data Recording
A s m e n t i o n e d earlier, t h e H E D uses t h r e e d i f f e r e n t c a m e r a s : t h e s c e n e c a m e r a , t h e e y e c a m e r a a n d t h e Ill. c a m e r a . T h e d a t a c o m i n g f r o m t h e s c e n e c a m e r a a n d t h e I R c a m e r a are s t o r e d digitally, as a n M P E G file a n d A S C I I file r e s p e c t i v e l y in t h e P C , w h i l e t h e eye c a m e r a d a t a c a n b e s t o r e d v i a a v i d e o r e c o r d e r . T h e d i g i t a l d a t a is u s e d for t h e a n a l y s i s a n d t h e e y e c a m e r a d a t a is u s e d for a d j u s t i n g t h e c o n t r a s t o f t h e p u p i l a n d c o r n e a l r e f l e c t i o n d u r i n g t h e c a l i b r a t i o n a n d t h e drive. F o r t h i s r e a s o n , o n l y t h e s c e n e c a m e r a a n d I R c a m e r a d a t a was r e c o r d e d for t h i s r e s e a r c h .
W h e n t h e eyes are c l o s e d t h e II:t c a m e r a is u n a b l e t o d e t e c t t h e p u p i l a n d t h e c o r n e a l r e f l e c t i o n . A s a r e s u l t , a n u m b e r o f z e r o s c o r r e s p o n d i n g t o t h e eye p o s i t i o n j u m p i n g f r o m t h e c u r r e n t p o s i t i o n t o t h e u p p e r left c o r n e r o f t h e s c e n e v i d e o will b e r e c o r d e d in t h e A S C I I file. T h e u p p e r left c o r n e r is c o n s i d e r e d t h e o r i g i n o f t h e c o o r d i n a t e space, w i t h r i g h t m o v e m e n t s r e s u l t i n g i n h i g h e r h o r i z o n t a l v a l u e s a n d u p w a r d s m o v e m e n t s r e s u l t i n g i n g r e a t e r v e r t i c a l values.
W h e n t h e c a l i b r a t i o n p r o c e d u r e is s u c c e s s f u l l y c o m p l e t e d t h e s u b j e c t is r e a d y t o drive, t h e s c e n e c a m e r a a n d I R c a m - e r a d a t a b e g i n r e c o r d i n g . T h e s u b j e c t is n o w a s k e d t o l o o k s t r a i g h t f o r w a r d a n d close h i s / h e r eyes for t h r e e s e c o n d s .
T h i s is d o n e t o s y n c h r o n i z e t h e t w o c a m e r a s y s t e m s . T h e s y n c h r o n i z a t i o n c a n l a t e r b e u s e d w h e n t r y i n g t o i d e n t i f y t h e s c e n e t h a t r e l a t e s t o m e a s u r e m e n t s r e c o r d e d in t h e A S C I I file. T h i s p r o c e d u r e p r o v e d t o b e c r i t i c a l in u n d e r s t a n d i n g h o w t h e r e c o r d e d s c e n e r e l a t e s t o t h e r e c o r d e d A S C I I file.
3 Results
T o e v a l u a t e t h e effect o f i n - v e h i c l e t a s k s o n t h e d r i v e r , f o c u s h a s b e e n p l a c e d o n a n a l y z i n g eye m o v e m e n t s as r e c o r d e d in t h e A S C I I file u s i n g t h e r e c o r d e d s c e n e as a m e t h o d o f de- t e r m i n i n g t h e p h y s i c a l l o c a t i o n o f a m o v e m e n t . S i n c e t h e a v e r a g e l e n g t h o f t h e d r i v e was 39 m i n u t e s , a n a v e r a g e o f 39 * 60 * 50 ---- 1 1 7 , 0 0 0 eye p o s i t i o n s m e a s u r e m e n t s w e r e m a d e for e a c h s u b j e c t . B e f o r e a d e t a i l e d a n a l y s i s c o u l d b e g i n , p r o b l e m s a s s o c i a t e d w i t h e n v i r o n m e n t a l c o n d i t i o n s t h a t l i m i t t h e e f f e c t i v e n e s s of t h e eye t r a c k e r w h i l e r e c o r d - i n g d a t a n e e d e d t o b e a d d r e s s e d . T w o l i m i t a t i o n s o f e y e t r a c k i n g w e r e o b s e r v e d w h e n a c o m p a r i s o n of t h e c o l l e c t e d d a t a a n d r e c o r d e d v i d e o w a s m a d e . F i r s t t h e eye t r a c k i n g e q u i p m e n t c a n n o t o b t a i n a p o s i t i o n m e a s u r e m e n t w h e n t h e eye is closed, s a y w h e n b l i n k i n g - I n t h i s s i t u a t i o n t h e e y e t r a c k e r a s s u m e s t h e e y e ' s p o s i t i o n is a t t h e o r i g i n t h e u p - p e r left h a n d c o r n e r . S e c o n d , s u n l i g h t i n t e r f e r e s w i t h t h e I R r e c o r d i n g d e v i c e , r e s u l t i n g in c o l l e c t e d p o s i t i o n s o u t s i d e of t h e f e a s i b l e r e g i o n o f m e a s u r e m e n t . B e f o r e a n y in d e p t h a n a l y s i s c a n b e c o m p l e t e d b o t h t y p e s o f false d a t a n e e d t o b e r e m o v e d f r o m t h e d a t a s t r e a m . T h e d a t a w a s f i l t e r e d t o r e m o v e b o t h p o s i t i o n s r e c o r d e d a t t h e o r i g i n a n d p o i n t s o u t - side t h e f e a s i b l e r e g i o n . W h e n t h e h o r i z o n t a l a n d v e r t i c a l eye p o s i t i o n s a r e p l o t t e d a g a i n s t t i m e , t h e f i l t e r e d s e c t i o n s a r e m i s s i n g as i l l u s t r a t e d in ( F i g u r e 1). A d e t a i l e d i n v e s t i g a t i o n i n t o eye m o v e m e n t p a t t e r n s c o u l d t h e n b e c o m p l e t e d .
B a s i c p l o t s o f r e c o r d e d d a t a p o s i t i o n s for a p a r t i c u l a r s u b - j e c t are s h o w n a g a i n s t t i m e for f o u r o f t h e t a s k s p r e s e n t e d t o t h e d r i v e r i n t h i s s t u d y : c h a n g i n g t h e r a d i o ( F i g u r e 1), c h e c k i n g t h e r e a r v i e w m i r r o r ( F i g u r e 2), r e a d i n g t h e o d o m e - t e r ( F i g u r e 3), a n d a h a n d h e l d cell p h o n e c o n v e r s a t i o n ( F i g - u r e 4). T h e figures s h o w t h e p a t t e r n of eye m o v e m e n t s b e - fore, d u i i n g , a n d a f t e r a t a s k is g i v e n t o t h e d r i v e r . S i n c e t h e s u b j e c t h a d no i n d i c a t i o n w h e n t h e i n s t r u c t i o n s for a t a s k w e r e t o b e g i n , eye m o v e m e n t s p a t t e r n s b e f o r e t h e s t a r t o f i n s t r u c t i o n s in a n y f i g u r e c a n b e c o n s i d e r e d c o n t r o l s . H o w - e v e r it is difl~icult t o classify a n y m o v e m e n t s r e c o r d e d as b e - i n g r e p r e s e n t a t i v e o f " n o r m a l driving"~ as w i l l b e d i s c u s s e d l a t e r . T w o b a s i c p a t t e r n s c a n b e i d e n t i f i e d f r o m t h e r e s u l t s , o n e in w h i c h glances a r e m a d e b e t w e e n t h e r o a d w a y a n d t h e d e v i c e (radio, r e a r v i e w m i r r o r , a n d o d o m e t e r ) ( F i g u r e s 1, 2, a n d 3) a n d a n o t h e r w h e r e t h e d r i v e r is in a s t a t e of s t a t i c f i x a t i o n o n t h e c e n t e r o f t h e r o a d ( F i g u r e 4). E a c h o f t h e s e s i t u a t i o n s n e e d s t o b e a n a l y z e d in a d i f f e r e n t m a n n e r , w i t h r e l a t i o n t o t h e effects o n d r i v i n g p e r f o r m a n c e .
I n t h e r a d i o t a s k it is e a s y t o see h o w t h e d r i v e r s eye m o v e m e n t s follow t h e t a s k s h a r i n g m o d e l . T h e i n s t r u c t i o n p e r i o d , t h e p e r i o d w h e r e t h e t a s k is c o m p l e t e d , a n d a n in- d i v i d u a l eye m o v e m e n t off t h e r o a d are i l l u s t r a t e d in ( F i g - u r e 1). W h e n t h e d r i v e r ' s eyes m o v e t o t h e r a d i o , w h i c h is l o c a t e d d o w n a n d t o t h e r i g h t o f t h e f o r w a r d v i e w , t h e h o r i z o n t a l p o s i t i o n a n d v e r t i c a l p o s i t i o n s i n c r e a s e , w i t h a d e l a y o c c u r r i n g at t h e p o s i t i o n of t h e r a d i o w h e r e s o m e ac- t i o n is c o m p l e t e d u n t i l a m o v e m e n t b a c k t o t h e f o r w a r d v i e w is c o m p l e t e d , u p a n d t o t h e left is. t h e h o r i z o n t a l a n d v e r - t i c a l p o s i t i o n s d e c r e a s e . T h e p r o c e s s is r e p e a t e d u n t i l t h e t a s k is c o m p l e t e d . T h e r e a r v i e w m i r r o r t a s k , ( F i g u r e 2), is v e r y s i m i l a r t o t h e r a d i o t a s k , e x c e p t i n s t e a d of a d o w n w a r d m o v e m e n t t o t h e r a d i o , a n u p w a r d m o v e m e n t t o t h e m i r r o r
is c o m b i n e d w i t h a m o v e m e n t t o t h e right. T h e o d o m e t e r reading, ( F i g u r e 3), shows m o r e of j u s t a one d i m e n s i o n a l m o v e m e n t to t h e d a s h b o a r d . W h e n t h e d r i v e r ' s eyes are n o t focused on t h e roadway, u n e x p e c t e d s t i m u l i will n o t b e focused close t o t h e fovea, r e q u i r i n g a n o t h e r eye m o v e m e n t a n d f i x a t i o n b e f o r e an u n d e r s t a n d i n g of t h e s i t u a t i o n c a n be m a d e . T h e driver, w h e n e n g a g e d in t h e cycle of glances b e t w e e n t h e device a n d t h e roadway, also loses t h e ability t o m o n i t o r s i t u a t i o n s t h a t c o u l d b e o c c u r r i n g a r o u n d t h e vehicle p e r i p h e r a l b u t n o t d i r e c t l y in front. T h e s e t y p e s of m o v e m e n t s are r e s p o n s i b l e for m a n y of t h e spikes f o u n d in t h e control s p a c e a r o u n d t h e i l l u s t r a t e d tasks.
A lack of eye m o v e m e n t s t o s u r r o u n d i n g l o c a t i o n s is e v e n m o r e p r o n o u n c e d d u r i n g t h e c o g n i t i v e p h o n e t a s k w h e r e t h e drivers eye " w a n d e r " a r o u n d t h e c e n t e r of t h e f o r w a r d view.
T h e lack o f m o v e m e n t possibly c o r r e s p o n d s t o visual t u n - neling a r e d u c t i o n in t h e useful field of v i e w o b s e r v e d d u r i n g p e r i o d s of i n c r e a s e d i n f o r m a t i o n processing. In this s i t u a t i o n it is a g a i n unlikely t h a t t h e d r i v e r w o u l d n o t i c e s i t u a t i o n s o c c u r r i n g a r o u n d t h e vehicle. It was also o b s e r v e d t h a t t h e r e d u c t i o n in eye m o v e m e n t s d i d n o t e n d w i t h t h e p h o n e call ie. e n d of t h e i n s t r u c t i o n period. R e l a t i n g t o t h e situa- t i o n d e s c r i b e d in [Redelmeier a n d T i b s h i r a n i 1997], w h e r e a s u s t a i n e d risk after t h e end o f a cell p h o n e c o n v e r s a t i o n is r e l a t e d t o after t h o u g h t s . E v e n w i t h t h e ability t o iden- tify d i s t r a c t i o n s t h a t cause a r e d u c t i o n in eye movements~
risk m a t r i c e s to c a t e g o r i z e t h e i r safety effects are difficult t o c o m p u t e . T h e ability t o develop t h e s e m a t r i x e s requires an u n d e r s t a n d i n g of a " n o r m a l " d r i v e r ' s eye m o v e m e n t p a t - tern. M e t h o d s of c o m p a r i n g h y p o t h e s i z e d " n o r m a l " d r i v e r ' s eye m o v e m e n t s are n e e d e d t o fully u n d e r s t a n d t h e w h e n a driver is u n d e r t h e effects of a c o g n i t i v e task.
"1
7 5 u
650 -
1
45o -1.~0 •
i ]
F i g u r e 1: H o r i z o n t a l (top line) Vertical ( b o t t o m line) eye m o v e m e n t s p l o t t e d a g a i n s t t i m e d u r i n g t h e r a d i o t a s k
3,1 Glance Measurements
A d e t a i l e d analysis of glances has b e e n c o m p l e t e d for t h e radio, r e a r v i e w mirror, a n d o d o m e t e r task. If a glance is defined as t h e s t a t i c t i m e w h e n a d r i v e r is likely t o b e inter- p r e t i n g i n f o r m a t i o n f r o m e i t h e r t h e r o a d w a y or s o m e in-car device, some h y p o t h e s i s can b e f o r m u l a t e d o n how long a d r i v e r ' s eyes are off t h e r o a d t o c o m p l e t e a p a r t i c u l a r task.
To simplify t h e analysis it is a s s u m e d t h a t t h e driver is e i t h e r i n t e r p r e t i n g i n f o r m a t i o n f r o m t h e r o a d w a y or I V I S device.
a~O
7 ~
65o
550
II "
25o -
~ o -
L I i I
F i g u r e 2: H o r i z o n t a l ( t o p line) Vertical ( b o t t o m line) eye m o v e m e n t s p l o t t e d against t i m e d u r i n g t h e r e a r v i e w m i r r o r t a s k
USo -
550 •
~ 0 •
. ~ 4 5 0 '
- 5 o Z 4 ~ U lO 12 14 l ~ 10 20 2
F i g u r e 3: H o r i z o n t a l (top line) V e r t i c a l ( b o t t o m line) eye m o v e m e n t s p l o t t e d against t i m e d u r i n g t h e o d o m e t e r t a s k
M o v e m e n t t i m e s and t h e influence of p e r i p h e r a l vision are c u r r e n t l y o m i t t e d f r o m this p r e l i m i n a r y analysis for simplic- ity. T h e e l i m i n a t i o n of m o v e m e n t t i m e s could h a v e s o m e effect o n glance times, since a slow m o v e m e n t w o u l d b e b e t - t e r classified as p a r t of t h e t r a n s i t i o n as o p p o s e d t o a glance.
For t h e r a d i o t a s k i l l u s t r a t e d in ( F i g u r e 1), it is possible t o c o n s t r u c t a table, i d e n t i f y i n g w h e r e t h e eyes are d i r e c t e d d u r i n g t h e task. Glances can b e c h a r a c t e r i z e d as e i t h e r o n r o a d or off r o a d glances. T h e d a t a c o r r e s p o n d i n g t o t h e ra- dio change t a s k is i l l u s t r a t e d in T a b l e 1. Glances can t h e n b e used as an i n d i c a t i o n of h o w o f t e n a d r i v e r ' s eyes are n o t on t h e road. T a b l e 2 s u m m a r i z e s t h e glances for t h e radio, rear v i e w m i r r o r , a n d o d o m e t e r tasks. T h e d a t a seems to verify the, 1.6 second rule discussed in [Wierwille 1993], since all b u t four glances for all t h r e e t a s k s were u n d e r t h e l i m i t a t i o n , a n d only one glance for 2.32 seconds e x c e e d e d 1.6 seconds for m o r e t h a n 0.08 seconds. W i t h o u t safety m a t r i x e s it is difficult to discuss h o w t h e d r i v e r is affected by different p a t - t e r n s of glances. H o w e v e r at 25 M P H t h e 2.32 second glance w o u l d r e p r e s e n t no direct f i x a t i o n or s m o o t h m o v e m e n t t o
55
850 - 750 -
~Su
~ 0
~mlSU j lSu 250 -
5u-
F i g u r e 4: H o r i z o n t a l ( t o p line) V e r t i c a l ( b o t t o m line) eye m o v e m e n t s p l o t t e d a g a i n s t t i m e d u r i n g t h e c o g n i t i v e h a n d h e l d p h o n e c o n v e r s a t i o n
t h e r o a d w a y for a d i s t a n c e of 85 feet, r a i s i n g s o m e b a s i c q u e s t i o n s of safety t h a t c a n n o t b e answered. G l a n c e m e a - s u r e m e n t s c a n n o t b e c o m p l e t e d for t h e c o g n i t i v e t a s k since it w o u l d r e p r e s e n t o n e l o n g g l a n c e t o t h e roadway. S i m i l a r l y c o g n i t i v e effects o f t h e radio, r e a r v i e w m i r r o r , a n d o d o m e t e r tasks c a n n o t b e i n c l u d e d in t h e analysis. As in t h e case of t h e c o g n i t i v e p h o n e c o n v e r s a t i o n , a c a l c u l a t e d " n o r m a l " eye m o v e m e n t p a t t e r n c o u l d allow for a f u r t h e r u n d e r s t a n d i n g of t h e c o g n i t i v e effects o f t h e s e tasks.
R o a d L o c a t i o n R o a d L o c a t i o n
G l a n c e O n Off G l a n c e O n Off
1 1.64 16 0.34
2 1.40 17 1.26
3 1.44 18 0.32
4 0.84 19 1.00
5 1.38 20 0.46
6 0.36 21 1.06
7 1-44 22 0-36
8 0.30 23 2.32
9 1.16 24 0.54
10 0.28 25 0.44
11 1.10 26 0.44
12 0.32 27 0.92
13 1.18 28 1.40
14 0.36 29 0.82
15 1.22
T a b l e 1: G l a n c e s for t h e r a d i o t a s k
3.2 Problems Encountered When Analyzing Eye Movement Data
T r a d i t i o n a l m e t h o d s of a n a l y z i n g eye m o v e m e n t s h a v e fo- c u s e d l a r g e l y on s e p a r a t i n g fixations f r o m s a c c a d e s b a s e d u p o n velocities, a g g r e g a t i o n of c o n s e c u t i v e p o i n t s w i t h du- r a t i o n m i n i m u m s , a n d d i g i t a l f i l t e r i n g [Salvucci 1999]. I n a d r i v i n g s i t u a t i o n , s m o o t h m o v e m e n t s are also p r e s e n t a n d t h e s e m e t h o d s w o u l d n e e d t o b e m o d i f i e d t o b e useful in t h e g e n e r a l case. M e t h o d s of s e p a r a t i n g fixations, s m o o t h
M e a s u r e T a s k c o m p l e t i o n
Off t h e r o a d M a x off t h e r o a d A v e r a g e off t h e r o a d g l a n c e
A v e r a g e g l a n c e G l a n c e s t o c o m p l e t e t a s k
R a d i o M i r r o r
26.12 26.56
18.38 18.72
2.32 1.68
1.23 1.17
0.90 0.86
15 16
O d o m e t e r 9.24
5.9 1.16 0.84 0.71
T a b l e 2: S u m m a r y o f glances for t h e radio, r e a r v i e w m i r r o r , a n d o d o m e t e r t a s k
m o v e m e n t s , a n d s a c c a d e s , p r o b a b l y c a n b e c o m p l e t e d w i t h a v e l o c i t y t h r e s h o l d . T h e v e l o c i t y of saccaclic m o v e m e n t is always s i g n i f i c a n t l y l a r g e r t h a n t h a t of a s m o o t h m o v e m e n t . A m e t h o d b a s e d u p o n t h i s t y p e of s e p a r a t i o n is discussed in [Sauter et al. 1991].
O n c e a s e p a r a t i o n of s m o o t h a n d s a c c a d i c m o v e m e n t is m a d e a n o t h e r p r o b l e m arises. I f a t r a c i n g p r o g r a m is t o m a p t h e s e m o v e m e n t s t o a c o g n i t i v e p r o c e s s m o d e l , a n un- d e r s t a n d i n g of t h e e n v i r o n m e n t t h e s u b j e c t is i m m e r s e d in is r e q u i r e d . I n t h e case o f a car driver, t h e e n v i r o n m e n t is c o n t i n u o u s l y chmaging, m a k i n g it difficult t o i d e n t i f y w h a t is a c t u a l l y o c c u r r i n g , e v e n if it is p o s s i b l e t o i d e n t i f y w h e r e t h e s u b j e c t ' s v i s u a l a t t e n t i o n is f o c u s e d in a c o o r d i n a t e s p a c e a u t o m a t i c a l l y . M a n u a l m a p p i n g is o n e t e c h n i q u e c o m m o n l y u s e d t o i d e n t i f y t h e scene. I n m a n u a l m a p p i n g a h u m a n o p e r a t o r d e t e r m i n e s , f r o m a v i d e o r e c o r d e d d u r i n g t h e t e s t , w h a t is a c t u a l l y o c c u r r i n g . I n a d d i t i o n t o t h e d y n a m i c scene, h e a d m o v e m e n t s f u r t h e r c o m p l i c a t e t h e analysis o f eye m o v e - m e n t d a t a r e c o r d e d u s i n g a h e a d m o u n t e d eye t r a c k e r .
I f a h e a d m o u n t e d eye t r a c k e r is used, r e c o r d e d d a t a o n l y r e p r e s e n t s t h e eye m o v e m e n t s in r e l a t i o n t o t h e s u b j e c t ' s h e a d . I n m a n y s i t u a t i o n s a r e f e r e n c e t o t h e v e h i c l e or o u t s i d e s p a c e is d e s i r e d so s o m e c a l c u l a t i o n of t h e gaze p a t h n e e d s t o b e m a d e . T o c a l c u l a t e t h e gaze p a t h s o m e c o m b i n a t i o n o f h e a d a n d eye p o s i t i o n is r e q u i r e d . S o m e eye t r a c k e r m o d e l s are a v a i l a b l e w i t h a n o p t i o n a l h e a d t r a c k e r t o e l i m i n a t e t h e h e a d m o v e m e n t p r o b l e m . However, t h e s e s y s t e m s c a n h e p r o b l e m a t i c in vehicles b e c a u s e o f t h e i r use of m a g n e t s .
Difficulties w i t h h e a d m o v e m e n t s w h e n u s i n g a H E D are o n e r e a s o n for t h e i r l i m i t e d use. In s i t u a t i o n s w h e r e a h e a d t r a c k e r c a n n o t b e used, c h a r a c t e r i s t i c s b e t w e e n h e a d a n d eye m o v e m e n t s m a y b e useful in s e p a r a t i n g o u t t h e h e a d m o v e m e n t c o m p o n e n t o f t h e gaze p a t h . I n [Stern a n d R a n - n e y 1999] h e a d m o v e m e n t is s h o w n t o b e d e p e n d e n t on eye m o v e m e n t . L a n d [Land 1992], discusses t h e s e m i - p r e d i c t a b l e r e l a t i o n s h i p b e t w e e n fast m o v i n g s a c c a d i c m o v e m e n t s a n d slower h e a d m o v e m e n t s . W h e n b o t h t h e eyes a n d h e a d re- ceive c o m m a n d s t o t u r n , a n eye m o v e m e n t o c c u r s q u i c k l y w i t h t h e h e a d following slowly, if free t o do so. I n situa- t i o n s w h e r e t h e h e a d is n o t free t o m o v e , a d d i t i o n a l m o v e - m e n t is a d d e d t o t h e eyes, r e s u l t i n g in t h e s a m e line of gaze [Land a n d H o r w o o d 1996]. S t e r n a n d R a n n e y [Stern a n d R a n n e y 19991 d e s c r i b e two t y p e s o f h e a d m o v e m e n t s , p r e - d i c t i v e h e a d m o v e m e n t , w h e r e t h e h e a d m o v e m e n t s u s u a l l y p r e c e d e s t h e eye m o v e m e n t , a n d r e a c t i v e h e a d m o v e m e n t s w h e r e t h e eye m o v e m e n t "lags b e h i n d t h e h e a d m o v e m e n t " . P r e d i c t i v e h e a d m o v e m e n t s u s u a l l y o c c u r in s i t u a t i o n s w h e r e we a n t i c i p a t e l o o k i n g at s o m e t h i n g i n o u r p e r i p h e r a l , w h i l e r e a c t i v e h e a d m o v e m e n t s o c c u r w h e n we do n o t a n t i c i p a t e looking. O t h e r m e t h o d s h a v e b e e n d e v e l o p e d for i d e n t i f y - ing h e a d m o v e m e n t , i n c l u d i n g v i d e o analysis t e c h n i q u e s us- ing face t r a c k i n g ( [ M a t s u m o t o a n d Z e l i n s k y 2000] a n d [Park a n d Lira 2001]) a n d a m o r e c o m p l e x l i g h t p r o j e c t i o n s y s t e m [ S c h u m a c h e r a n d K o r n 1983]. F u t u r e w o r k o n s e p a r a t i n g
h e a d m o v e m e n t s could include a c t i o n r e c o g n i t i o n a l g o r i t h m s similar t o t h o s e p r o p o s e d in ([Bobick 1996] and [Laya 1998]).
T h e w o r k in [Laya 1998] aims to develop a i m a g e processing solution t o t h e h e a d m o v e m e n t problem. Schill et al. [Schill et al. 2001] look at m e t h o d s for a u t o m a t i c a l l y d e t e c t i n g re- gions of interest in images using a c o m p l e x l e a r n i n g b a s e d s y s t e m b a s e d u p o n eye m o v e m e n t s , while d i s r e g a r d i n g t h e a c t u a l h e a d position.
In order t o f u r t h e r u n d e r s t a n d w h a t effects d i s t r a c t i n g tasks have on t h e driver, some u n d e r s t a n d i n g of w h a t is considered " n o r m a l " d r i v i n g is needed. S u c h a m e a s u r e is i n h e r e n t l y difficult t o c a l c u l a t e in a real r o a d s e t t i n g since t h e e n v i r o n m e n t is c o n s t a n t l y changing. T h e p r o b l e m is akin to t h e s t e a d y s t a t e p r o b l e m in r a n d o m process. However, in t h e case of drivers, it m a y be possible t o e s t i m a t e " n o r m a l "
eye m o v e m e n t s in an artificial e n v i r o n m e n t w h e r e t h e r o a d is straight, w i t h o n c o m i n g cars. H o w e v e r here t o o it is easy t o i m a g i n e t h e n u m b e r of possible d e r i v a t i o n s t h a t could b e discussed t o describe w h a t a driver w o u l d be e x p e c t e d t o look at. I n a s i t u a t i o n w i t h a o n c o m i n g cars a p p r o a c h i n g at 25 M P H , t h e driver needs to give t h e less a t t e n t i o n t o t h e a p p r o a c h i n g car t h a n at 40 M P H since unconsciously t h e driver knows things are h a p p e n i n g slower. W h i l e signs on t h e side of t h e r o a d r e q u i r e quick glances t h a t m a y h a p p e n w i t h s o m e p r o b a b i l i t y etc.
4 Conclusions
T h i s p a p e r d e m o n s t r a t e s how various eye m o v e m e n t s can b e collected and a n a l y z e d t o c o m p a r e a d r i v e r ' s p e r f o r m a n c e w i t h a v a r i e t y of in-vehicle tasks. Conclusions verify some basic m e a s u r e s t h a t h a v e already b e e n p u b l i s h e d in t h e lit- erature. E y e m o v e m e n t s for t h e radio, r e a r v i e w mirror, and o d o m e t e r tasks all show p a t t e r n s of t i m e sharing by drivers.
A t t e n t i o n is d i v i d e d b e t w e e n t h e tasks of d r i v i n g a n d t h e sec- o n d a r y task, s w i t c h i n g back a n d f o r t h b e t w e e n t h e m . T h e basic m e t h o d of analysis d e m o n s t r a t e d here shows only how a d r i v e r divides a t t e n t i o n , cognitive effects of t h e t a s k c a n n o t be a n a l y z e d using this m e t h o d . Results also concur w i t h t h e a p p a r e n t m a x i m u m glance t i m e of a p p r o x i m a t e l y 1.6 seconds off t h e r o a d as discussed in [WierwiUe 1993]. I n t h e t h r e e t e s t cases shown only four v i o l a t i o n s of this rule o c c u r r e d for t h e 38 glances off t h e r o a d a n d only one of t h e s e v i o l a t i o n s e x c e e d e d t h e rule by m o r e t h a n 0.08 seconds. T h e s m a l l v a r i a t i o n m a y r e l a t e t o how m o v e m e n t t i m e s were a s s u m e d t o be negligible. T h e one glance for 2.32 seconds d u r i n g t h e radio t a s k is however well a b o v e t h e t o l e r a n c e limit. If t h e driver was o b s e r v i n g t h e p o s t e d speed limit of 25 M P H , such a glance off t h e r o a d represents t r a v e l i n g a d i s t a n c e of a b o u t 85 feet. A l t h o u g h it is t e m p t i n g to conclude t h a t t r a v e l i n g 85 feet w i t h o u t looking d i r e c t l y at t h e r o a d is a d a n g e r o u s d r i v i n g practice, a d d i t i o n a l work is n e e d e d to s u p p o r t t h i s hypothesis~ and to develop a similar m a t r i x for t h e effects of c o g n i t i v e distractions.
T h e decrease in eye m o v e m e n t s shown w i t h t h e cognitive t a s k concurs w i t h w h a t has b e e n published, and t h e basic p r i n c i p a l of visual t u n n e l i n g t h a t o c c u r u n d e r cognitive load.
T h e dangers involved w i t h t h e cognitive t a s k axe shown to continue p a s t t h e end of t h e p h o n e conversation, agreeing w i t h t h e s i t u a t i o n d e s c r i b e d by [Hedelmeier and T i b s h i r a n i 1997], t h a t t h e risk of a cognitive c o n v e r s a t i o n is still higher t h a n " n o r m a l " after c o n v e r s a t i o n ends. Since t h e cognitive t a s k is r e p r e s e n t e d as a single glance at t h e roadway, it m a y also be i n t e r p r e t e d as a n unsafe d r i v i n g practice. It is how- ever a m o r e difficult p r o b l e m t o identify a level of risk f r o m t h e d a t a collected in this s t u d y w i t h o u t f u r t h e r m e t h o d s of
c o m p a r i n g s o m e m e a s u r e of " n o r m a l " driving. W i t h o u t a
" n o r m a l " d a t a s t r e a m difficulties also exist i d e n t i f y i n g if t h e effects of t h e t a s k axe c o m p l e t e l y e l i m i n a t e d .
5 The Future
To f u r t h e r develop driver p r e d i c t i o n models for r e a l - t i m e w a r n i n g s y s t e m s a n d a d v a n c e d m o d e l i n g systems, fast m e t h - ods of a n a l y z i n g h u m a n eye m o v e m e n t s are needed. T h e n e x t logical step is t o develop m a t r i x for i n t e r p r e t i n g a n d e v a l u a t i n g e y e - m o v e m e n t s as a m e t h o d of p r e d i c t i n g a d r i v e r ' s intent. Since h u m a n eye m o v e m e n t s are a s s u m e d t o follow a shift in a t t e n t i o n , this m e t h o d shows t h e m o s t p r o m i s e for p r e d i c t i n g driver actions. F u t u r e vehicles could easily be f i t t e d w i t h eye t r a c k i n g e q u i p m e n t , m o n i t o r i n g a drivers fixation p a t t e r n s , thus p r e v e n t i n g unsafe distrac- tions.
References
Bobick, A. 1996. C o m p u t e r s seeing action. In P r o c e e d i n g s of B M V C , , 12-22.
Brown, I., Tickner, A., and Simmonds, D. 1969. Interfer- ence between concurrent tasks of driving and telephoning.
Journal of Applied Psychology 53, 5,419--424.
Cain, A., and Burris, M. 1999. Investigation of the use of mobile phones while driving. Center for U r b a n Trans- portation Research.
C h a p m a n , P., and Underwood, G. 1998. Eye Guidance in Hea~ling, Driving, and Scene Perception. G. Underwood, Elsevier, ch. Visual search of dynamic scenes: event types and the role of experience in viewing driving situations, 369-393.
Crundall, D., Underwood, G., and C h a p m a n ~ P. 1997. Pe- ripheral detection rates in drivers. In Paper presented at the Seventh International Conference on Vision in Vehi- cles, .
Crundall, D., Underwood, G., and C h a p m a n , P. 1998. Eye Guidance in Heading, Driving, and Scene Perception. G.
Underwood, Elsevier, ch. H o w m u c h do novice drivers see?
T h e effects of d e m a n d on visual search strategies in novice and experienced drivers, 395--417.
Hankey, J., Dingus, T., Hanowslci, H., Wierwille, W., Monk, C., and Moyer, M. 2000. T h e development of a design evaluation tool and model of attention demand.
Harms, L. 1991. Variation in drivers' cognitive load: Ef- fects of driving through village areas and rural junctions.
Ergonomics 34, 151-160.
Henderson, J. 1993. Perception and cognition. Advances in eye-movement research. G. d'Ydewalle and J. van Hens- bergen, North-Holland, ch. Visual attention and saccadic eye-movements, 37-50.
Just, M., and Carpenter, P. 1976. Eye fixations and cognitive processes. Cognitive Psychology 8, 441---480.
Land, M., and Horwood, J. 1996. Vision in vehicles. A G . Gale and ID. B r o w n and C M . I-Iaslegrave and SP. Taylor~
North-Holland, ch. T h e relations between head and eye movements during driving, 153-160.
67
