How is Eye Movement related to Workload and Behaviour

2.2.2.1 The Strategic Nature of Eye Movements

Eye movement reveals the behaviour strategy in reading, picture viewing and piano playing (Land, 2006), where top-down behaviour patterns were found. It was only after portable eye movement monitoring equipment was invented in recent years that eye movement has been studied in daily activities like walking, tea and sandwich making, and in sport. A considerable amount of research have suggested that eye movement having the ability to reveal detailed

behaviour strategy when organising activities and performing tasks. Yarbus (Yarbus, 1967) investigated visual searching patterns when looking at pictures with a number of different questions in mind. These questions were related to different aspect of the image, for example the people, or the clothes they wear. The results showed that each question evoked a different pattern of eye movements, related to the information required to answer the question. This meant that eye movements are not simply related to the structure of the picture itself, but also to top-down instructions from the brain. Similarly, drivers also view the scenes with different questions, for example: is it safe to change lanes? Yarbus’ work proved that eye movements were reflex actions, but much more strategic in character. Such a strategy of eye movement determines that it is close related to workload and behaviour.

2.2.2.2 Eye Movements, Workload, and Behaviour

The previous study shows that eye movements can be both the workload and performance measures, for example, Recarte (Recarte et al., 2008) has taken blink rate as a reliable and nonintrusive workload measurement; while in some researches, eye movement was used as part of the task performance measurement (Angell et al., 2006). The physiological nature of eye movement provides the opportunity of building up a bridge between drivers’ workload and their performance in a detailed level.

Although it was argued that it is not always necessary to move the eyes to identify every object and human beings can move attention without necessarily moving their eyes (Posner, 1980), it is still believed by researchers that with complex stimuli, it is more efficient to move eyes than to move attention (He & Kowler, 1992; Rayner, 1998). In complex information processing tasks such as driving, where the link between the two is significant, analysing eye movement behaviour can provide the information on attention allocation. “Even though there is limited

systematic study of where drivers look in traffic, from observations it is clear that drivers visually detect the places from which they need to obtain information, e.g. on the car in front, the in-vehicle display, the pedestrians and cyclists, the road signs and traffic lights, etc.” (Land,

2006). How this information is taken, especially when dual-tasking and how drivers cope with the extra task demand can be revealed by observing drivers’ eye movement.

2.2.2.3 Eye Movements in Driving

Eye movement research started about 20 years ago. Huey (Huey, 1908) cited in (Rayner, 1998) first observed the role of eye movements in reading. Later the research interest spread out into reading in playing music, daily activities, sports, etc. Findings from these researches revealed that more complicated visual content typically caused more and longer fixations. Since 1990s,

research on the effects of cognitive load (e.g. solving math and physics problems) on eye movement lead to the concept that eye movement can reflect not only visual, but also mental workload. It is only since head-mounted eye trackers became available in the 1980s that it has been possible to study active tasks such as driving (Land, 2006).

Driving a vehicle in a real traffic situation is predominately visually demanding and, according to Wickens (Wickens, 1984a), the resources required in driving were multiple and the

concurrent processing of visual sources would cause structural interference. Even though the commonly believed statement that 90% of the information required for driving is visual was questioned by a review of vision in driving (Sivak, 1996), it is still true that most driving- critical information is collected visually. The driving task involves dealing with the road itself (steering, speed control), other road users (vehicles, cyclists, moving and stationary pedestrians), paying attention to road signs and other relevant sources of information. It is thus a varied task, and it is expected that a range of eye movement strategies will be employed (Land & Horwood, 1995; Land, 1992; Mourant & Rockwell, 1970; Rockwell, 1988).

Early studies on eye movement were mainly conducted on road sections with low curvatures (e.g. typically US roads), therefore only a weak correlation between gaze direction and steering was found (Zwahlen et al., 2003). The relationship between steering and gaze was later clarified by findings from sharp curves demanding more attention (Land & Lee, 1994). Land and Lee (Land & Lee, 1994) conducted their research on a very winding road, and the results revealed that the recorded gaze direction and steering wheel angle were very similar, as shown in Figure 2.5. In particular, drivers spent much of their time looking at the “tangent point” on the

approaching bend.

Figure 2.5 Gaze angle on winding road (Land & Lee, 1994)

In normal driving, drivers spend most of their time looking ahead with about 5° down from the horizontal, in vertical gaze angle. It was reported (Hughes & Cole, 1986) that when watching a

movie film of driving view, 25% of fixations were located in the central region, and 80% of the remaining were within 6°of the visual centre, i.e. only 15% of fixations were beyond 6°. It is believed that looking away from the road ahead for any period of time is detrimental, as Summala has found (Summala, 1998) the lane keeping on a straight road deteriorated as a function of drivers fixating away from the view ahead.

To keep a good steering, drivers need the information of further vision: tangent point, as well as near road region: road edge (Land & Horwood, 1995); when only the further region was visible but the near region was not, curvature matching was still accurate but the lane control was poor; On the other hand, with only the near road region visible but without the further region, lane maintenance was good, but curvature matching was poor, mainly due to large and abrupt steering induced by the short time available for reaction to the road edges (<0.5 s). Therefore neither the further vision, nor the near road lane edge input is sufficient on their own, but the combination of the two allows fluent accurate driving. The result suggested that about 5° down from the horizon gives a good result on both criteria, explaining why 5° down from the horizon was found in normal driving.

Peripheral vision can be developed to monitor lane position (Mourant & Rockwell, 1970). Learner drivers first use foveal vision for lane keeping, then increasingly move foveal gaze to more distant road regions, and learn to use their peripheral vision to stay in lane. Summala et al. (Summala, Nieminen & Punto, 1996) reached similar conclusions. This may also be related to the fact that experienced drivers show a much larger searching area (Crundall, Underwood & Chapman, 1999a).

2.2.2.4 The Effects of Secondary Tasks on Eye Movement

Typically, the percent of concentration on the road centre, searching area and fixation on the in- vehicle display are used to measure the demands of secondary tasks (Victor et al., 2005). When visual tasks became more difficult, drivers looked less at the road centre area ahead, and looked at the display more often and for longer durations with higher deviation; while for the auditory task, gaze concentration on the road centre area increased with the increase of task complexity. A similar effect of gaze concentration due to mental demand (as measured by a visual perimeter) were previously reported by Rantanen and Goldberg (Rantanen & Goldberg, 1999). As a

consequence of shrunken gaze area, mental tasks can also cause decreases of visual detection. Olsson and Burns (Olsson & Burns, 2000) found that counting backwards interfered with the detection of peripheral lights. Strayer, Drews and Johnston (Strayer et al., 2003) also reported that when participants were involved in a hands-free phone conversation, they responded slower

to the leading vehicle’s brake lights. Horberry and colleagues (Horberry et al., 2006) related to the visual impairment caused by mental workload directly with road accidents. They found in a study conducted in a simulator that engaging simulated hands-free mobile phone conversation impaired drivers’ responses to pedestrians crossing the road.

More evidence of the linking of eye movement parameters and performing secondary task were proposed by other research. For mental tasks, Recarte and Nunes (Recarte & Nunes, 2000; Recarte & Nunes, 2003) found that the fixations were longer and saccades were smaller for participants. Drivers were found to check mirrors more often (Recarte & Nunes, 2000), and their saccade rate decreased (Harbluk et al., 2007b). As for visual tasks, Harbluk, Noy and Eizenman (Harbluk et al., 2002) found that additional tasks reduced the overall number of saccades, scanning to the periphery, and checking instruments and mirrors, and that these changes were related to an increase in hard braking. In one of the researches carried out in an aviation simulator by Federal Aviation Administration, USA (Ahlstrom & Friedman-Berg, 2005), it was found that when detecting a higher visual condensed task, operators’ blink

duration and pupil diameter increased, but no significant changes were found in blink frequency, saccade frequency and saccade distance.

In summary, a review on the effect of secondary tasks on eye movement suggests that secondary tasks increased driver workload, decreased the area of visual space, and caused drivers to perceive less of visual objects. It should be noted that different types of tasks have very different effects on eye movement. In order to use eye movement measurements to investigate the workload and performance, tasks which cause the visual behaviour change need to be categorised and the characters of various eye movement parameters in each type of task have to be studied carefully.

2.2.2.5 The Effects of Different Driving Situations on Eye Movement

Apart from the effects due to secondary tasks, eye movements are also affected by different driving situations. For example, when driving at a lower speed (e.g. 30 mph or less), eyes are much more freed up for the multiple demands of dealing with other road users and potential obstacles, than driving at a higher speed (Land, 2006). Because at low speed, steering only requires peripheral lane edge information, and the need for monitoring distant tangent points is much reduced. Therefore, driving at a lower speed (normally accompanied by less workload) is observed to produce a wider visual searching area.

Visual searching behaviour in driving is similar to any other daily activity, showing the

“top-down” strategy of behaviour. Drivers’ visual fixation patterns were found to be affected by various demands in a systematic and predictable fashion (Land & Horwood, 1995; Recarte & Nunes, 2000; Velichkovsky et al., 2002; Victor, Harbluk & Engstrom, 2005). Searching for required information (e.g. road signs) is also influenced by the complexity and familiarity of the environment. In a more complex visual context, targeting and extracting the useful information can be more demanding. As the demands of traffic situations increase, peripheral vision is sacrificed in order to provide greater attention for information uptake by the fovea (Miura, 1986). This is confirmed by Shinoda et al. (Shinoda, Hayhoe & Shrivastava, 2001), who found that the detection of stop signs requires an active search, and the visibility of the signs depends on the concurrent visual background. Active searching also depends on what information is needed by the driver, for example, on a familiar route the need for checking road signs may be not as much as on an unfamiliar one. As Shinar and Gurion has found (Shinar & Gurion, 2008), drivers’ fixation were less dispersed on the above and right of the roadway (where the traffic signs are located in USA), and were more concentrated further down the drive lane when they became more familiar with the route. During driving, time pressure can also affect drivers’ visual searching strategies in different driving conditions. In free-driving, vision focus is on a location further down the road; however, when following other vehicles, drivers tend to concentrate on the rear of the leading vehicles (Shinar & Gurion, 2008). The visual drift was explained as the drivers’ initiative strategy of obtaining information with maximal lead time.

Previous research showed the sensitivity of visual behaviour on road environment, which implied that eye movements cannot be solely investigated, but needs to be considered under a specific driving situation. For any on-road research, to obtain high quality eye movement data, the road environment need to be controlled carefully.