Event Segmentation

Evidence that the attentional blink occurs during film perception can be found in recent work on Event Segmentation and Disruption Blindness. Event Segmentation refers to our tendency to ‘parse’ continuous visual actions into discrete events both intentionally and during normal viewing (Newtson, 1973; Newtson, Engquist, & Bois, 1977; Newtson & Enqguist, 1976; Zacks et al., 2001; Zacks, 2004; Zacks & Tversky, 2001). Disruption Blindness refers to our inability to recall whole-field disruptions (such as blank frames) inserted into a film depicting such events (Baldwin, Baird, Saylor, & Clark, 2001; Levin & Varakin, 2004; Newtson & Enqguist, 1976; Saylor & Baldwin, 2005). As this section will attempt to show, the time point at which one event is perceived as ending and another beginning are similar to the points identified by film editors as valid edit points. Editor’s intuition that placing a cut at these points will make the cut invisible to viewers is supported by evidence from disruption blindness.

The ability to identify structure within events is an important part of perceptual behaviour. Our knowledge of event structures influences how we read, remember, and plan (Zacks & Tversky, 2001). This ability to decompose continuous activities into discrete events develops in infancy (Wynn, 1996) and, by adulthood, has developed to a level of consistency that if asked to repeatedly segment the same activity into its constituent events the same events will be identified (Newtson, Engquist, & Bois, 1976). Viewers can segment activities into events of different sizes (fine or coarse) and these are hierarchically related: groups of fine events corresponding to single coarse events (Zacks & Tversky, 2001). For example, if asked to segment the activity of “ironing a shirt”, each stroke of the iron across the shirt, setting the iron down, picking the iron up, or lifting the shirt might be identified as individual fine events. By comparison, coarse events would probably be identified for groups of these events such as “ironing the left sleeve”, “ironing the chest”, or “folding the shirt”. These events do not appear to be arbitrary as there is considerable agreement between the location of breakpoints across viewers (a breakpoint is the

70 point at which one event is identified as ending and another beginning; Newtson et al., 1977).

There is also recent evidence indicating that the same perceptual segmentation occurs during normal viewing when the viewer is not instructed to perform segmentation (Zacks et al., 2001). Neuroimaging studies have shown that the same brain regions are active during active and passive segmentation (normal viewing conditions when the viewer is uninformed of the segmentation task; (Speer, Swallow K. M., & Zacks, 2003; Zacks et al., 2001). This indicates that the segmentation of continuous visual activities into discrete events is a natural part of visual perception (Zacks et al., 2001).

Darren Newtson first developed the event segmentation task as a method for investigating the differences and similarities between how different viewers perceived visual activities (Newtson, 1973). His initial experiments indicated that viewers were very capable at performing segmentation (Newtson, 1973) and seemed to base their choice of breakpoints on significant changes in depicted motion (Newtson et al., 1977). Given that the films used depicted an actor performing a simple task, the changes in motion corresponded to a large number of changes in the position of the actor’s body or limbs (Newtson et al., 1977). This correspondence between movement and breakpoints was most significant for fine breakpoints (points between small events; (Newtson et al., 1977). This relationship between changes in visual motion and perceived breakpoints has also been found by a more recent study (Zacks, 2004). Zacks created abstract animations in which two objects either followed random paths or exhibited intentional behaviour. He found that changes in motion reliably predicted where viewers would identify both fine and coarse breakpoints. This relationship was stronger for fine breakpoints and weakened as the size of the events increased (i.e. as the breakpoints became coarser). Zacks also found that viewer’s inferences about the intentionality of the behaviours weakened the influence of movement features, specifically with larger events (Zacks, 2004).

71 Neuroimaging evidence also supports the involvement of changes in motion in the identification of breakpoints (Speer et al., 2003; Zacks et al., 2001). Neural activation was observed in brain regions identified as the Medial Temporal complex (MT+) and Frontal Eye Field (FEF) both during active and passive segmentation (Zacks et al., 2001). The MT+ complex is a visual area containing cells known to be sensitive to direction and speed of visual motion. The FEF is known to be involved in guiding saccadic and smooth pursuit eye movements. Its involvement in automatic and intentional event segmentation is less clear as subsequent neuroimaging studies in which the FEF was more precisely located have shown weaker activation under passive segmentation conditions (Speer et al., 2003). There is also no recorded activation in the brain regions normally associated with controlling shifts of attention (Speer et al., 2003). This seems to suggest that eye movements are not critical to the perception of breakpoints although Speer et al (Speer et al., 2003) do not rule out the connection.

This absence of a clear relationship between breakpoints and overt shifts of attention suggests that the end of an event may not provide a point at which the cut can be hidden. The evidence of the attentional blink seemed to suggest that the end of a visual event such as that identified during the event segmentation task would be accompanied by an absence of attention. This absence would provide a period during which the visual scene could be changed by a cut without the viewer becoming aware. However, the weak activation observed in the brain region controlling eye movements (the FEF) could be an indication that eye movements accompany some, but not all breakpoints (Speer et al., 2003)30.

It is possible that a relationship between breakpoints and eye movements might be detected using a different recording technique. Tentative evidence for this relationship has been recently shown in an eye tracking study (Smith, Whitwell, &

30 _{As the neural activation patterns are averaging across all breakpoints, different activation patterns} for individual breakpoints is lost. This combined with the already weak activation observed under passive viewing might obscure the contribution of the FEF to event segmentation.

72 Lee, in press; Whitwell, 2005)31. By replicating Zacks et al (2001) methodology but replacing fMRI recordings with eye tracking, a significant decrease in saccade frequency was found 260ms prior to fine passive breakpoints followed by a significant increase in saccade frequency 140ms after the breakpoint (Smith et al., in press; Whitwell, 2005). No such effect of saccade frequency was observed after coarse breakpoints. The increase in frequency of saccades after fine breakpoints has been interpreted as visual search performed in response to the onset of a new visual event (Smith et al., in press). The time delay between the breakpoint and the increase in saccade frequency (140ms) is compatible with the average time taken to perform a saccadic eye movement in response to a sensory event (typically 150-200ms; Palmer, 1999). The potential for these eye movements to limit viewer awareness of cuts will be discussed in more detail in a later section (see 3.4.2).

As well as overt shifts of attention there is also the possibility that covert shifts of attention32 or the reallocation of attention to cognitive processes, as in the attentional blink could occur during event perception. To find evidence of these shifts in attention, viewers’ sensitivity to events need to be tested.