Saccade Identification

The main data output supplied by the eye-tracker is a set of X and Y coordinates. A new coordinate is produced every 4ms. There is additionally an automatic calculation that computes whether the participant is fixating or not for a given coordinate (1 = they are fixating and 2 = they are not). Time stamps are also produced for each of these points. Given the X and Y coordinates it is possible to determine the distance between the location of gaze and any given point on the screen using Pythagoras’s Theorem. In a right angled triangle the square of the hypotenuse is equal to the sum of the squares of the other two sides, see figure 5 below). Given the X coordinate (the adjacent) and the Y coordinate (the opposite), the distance between point A and point B (the hypotenuse) can be determined, the square root of X2 _{+ Y}2 _{= the} hypotenuse. Building on this, we can specify in a task script that if the distance between point A and point B is greater than a certain value, or is less than a certain value, a particular command should be executed. For example, to ensure that participants are fixating at the centre of the screen before each trial we can use this algorithm to determine a participant’s gaze location with respect to the centre of the screen. We can stipulate that if the gaze shifts away from the centre of the screen by more than a specified value within a given period then the trial will not start but instead the previous set of commands will be repeated. This algorithm will

continually repeat until the participant holds his or her gaze stable at the centre of the screen.

82

Figure 5. Method used for calculated gaze direction

7.2.1.1 Regions of interest and velocity based methods Two broad sets of saccade detection algorithms were developed for the tasks used. The first is a region of interest algorithm. Regions of interest were defined based on distance from the centre and angle. While a participant is completing a task during any period of interest within a trial the eye-tracker outputs their current gaze location and whether they are fixating or not. Based on this it is possible to

determine either the distance of their gaze from a point of interest or whether their position is within a predefined distance from a given point (e.g. in figure 6 greater than C but less that C + B) and within a given degree of visual angle (e.g. in figure 10 angle a). By defining regions of interest and the periods of time within which these regions are of interest saccadic movements were inferred. For example, participants must first fixate at the central fixation cross for a set period at the beginning of a trial. The trial is classified as a failed trial if there is any movement greater than approximately 1 degree away from the fixation cross before a defined period. Following this a target, or distractor or both will appear. If the direction of gaze is then directed at a region of interest within a defined period a saccade is

83

inferred. This method was used mainly for real-time gaze contingent programs.

Figure 6. Illustration of the method used to calculate if the direction of gaze falls within the region of interest.

The second method used to identify saccades is based on the velocity of eye

movements, that is, changes in distance over time. These calculations were typically conducted off-line. When processing the data off-line a number of technical issues were identified when importing the raw data. Loading the saved Matlab files back into Matlab created errors in some cases such that for some trials for some

participants a portion of the X-Y gaze position row vector was shifted to a new row. This rendered the data impossible to work with. The solution developed was to import the raw data file into excel, save it as a worksheet, and then import it into Matlab. The cause of this problem has not been identified but may relate to the length of the X-Y array. Upon loading the data into Matlab a further problem was identified. In some instances segments of data were inappropriately ordered. The solution was to reorder data based on the time stamp variable.

Once the data is correctly loaded and ordered the periods of interest are defined. Stimulus onsets time markers saved while the task is being completed are used for this purpose. The periods of interest typically segment the data into (1) the period before the arrival of the target (pre-jump period), (2) the period in which a saccade

84

should be made to the target, the target interval, and (3) the period after the target interval, post-target period. The pre-jump period is first analysed to identify if the trial contained a premature saccade or if the participants have failed to hold fixation at the central fixation cross before the target arrived. If neither of these is true the target interval is analysed to identify if a saccade was made during this period and the start and end point of this saccade. If no saccade is identified in either the pre- jump period or target period, the post target period was then analysed to determine if a late saccade was made, and if so to where.

Saccades are identified by firstly calculating “Distances”, all distances between adjacent gaze coordinates. This is calculated as the square root of the difference between all adjacent X locations squared plus the difference between all the adjacent Y locations squared, equation 1 below is the Matlab code used.

Equation 1

Lengths= sqrt((X(2:end)-X(1:end-1)).^2+(Y(2:end)-Y(1:end-1)).^2)

This gives N-1 number of “Distance” points. These distances constitute the distance of movement in eye gaze over 4 millisecond periods. A sliding window of size 7 was then run the length of the “Distances” variable. Within each window the average is calculated and subtracted from the adjacent window to give smoothed velocity between points (see equation 2). The degree of smoothing is determined by the size of the shifting window. The greater its size the more noise it removes but the less sensitive it becomes to changes.

Equation 2

For bb=WindowSize+1:length(Lengths)-WindowSize-1

Eye_velocity(1,bb)=mean(Lengths(bb:bb+WindowSize-1))-mean(Lengths(bb WindowSize:bb- 1));

End

The script then checks for changes in velocity (acceleration and deceleration) in order to identify the point of saccade onset and offset. A peak detection algorithm,

85

for which a threshold value is set (typically set between 0.5 and 0.7), is then run on the list of velocities to determine the location of velocities greater than the threshold value. This helps to identify the onset of the saccade which is the period just before the point of highest peak velocity. We can similarly determine the point of saccade offset as the period just after the lowest peak velocity (a velocity reduction). This can be seen in figure 7 which shows a plot of generated velocities. It can be seen that there is a spike increase and immediate decrease between samples 70 to 80. Given a number (N) of X and Y coordinate that were collected for an unbroken period of time, the number of coordinate points will equal the period of time in millisecond divided by 4 (given that the sampling rate of the eye-tracker is 250Hz). Therefore, if the peak is identified as occurring 50 sample points the appearance of the stimulus the peak of the saccade is 200 ms after the target appearance (50 sample points by 4 ms per sample point). In addition, by examining the post-

saccade location it can then determine whether the saccade was made to the target location, distracter location, or some other location.

Figure 7. Smoothed increases and decreases in saccade velocity across samples.

Early on in task development post-test identification of saccades based on a velocity based method was conducted in a semi-automated manner. This ensured the

accuracy of the method and helped to establish the identification parameters. Subsequently, data was processed with the use of a fully automated saccade

86

identification algorithm. A visual inspection of data plots for a subset of trials was conducted to validate the method used.

7.2.1.2 Dealing with missing data

Due to the nature of eye-tracking technology missing data is an issue. There are a number of reasons why one will have missing data. In the majority of cases the problem arises due to the fact that the pupil or Purkinje Images are not clearly visible within the video image. If there is a period of missing data around the period at which a saccade is initiated the trial must be eliminated as there is no way to determine an accurate reaction time or location landing point. It is important to then calculate the proportion of correct response trials and proportion of incorrect trials as opposed to the absolute number of trials. Such data loss will occur with current eye- tracking technology and the best solution is to collect many trials and to ensure that steps are taken to maximise the quality of the data collected.