Our Calibration Method

Our calibration method involves individually showing calibration points, of which true positions on the screen are known, distributed equally on the screen. The user focuses on each of the points and fixation data is recorded by the eye tracker. Calibration sites

{si}i=1,···,13 in the spatio-temporal domain are where our shifts{∆x(si),∆y(si)}i=1,···,13

are recorded.

(a) Empirial variogram (b) Fitted variogram

Figure 5.4: Example empirical and fitted variogram on real data from a particpant.

The next section will discuss the experiment that was set up to test and assess the new calibration method and the results obtained from this experiment.

5.4

Experimental Results

The experiment was divided into two stages. The first stage was performed using the built-in calibration tool that comes with eye trackers. The second stage was identical to the first except that it was performed using our kriging calibration apart from the initial calibration at the start of the recording that was performed with the built-in calibration tool. The order of these stages was randomised for each run of the experiment. For calibration we used 13 points for both stages. Fewer points can be used but accuracy is degraded as the number of calibration points decreases.

Each of the two stages was divided into parts consisting of the following: a calibra- tion step, three test points shown of known position, a short film clip (25 secs), three test points shown again, a short film clip (25 secs), and the same three test points shown again. Each part was repeated to the participant four times with no breaks. The three test points were points chosen to assess the accuracy of kriging interpolation.

The adjustment performed via interpolation on the fixations on these points was com- pared to the real known positions of the points on the screen. One point was chosen to be in the middle of three calibration points, one point was chosen to be in the middle of two calibration points and one test point was chosen to be at exactly the same site as a calibration point. Figure 5.5 shows the positions of the calibration and test points used in the experiment. The built-in calibration tool was used only once for the second stage of the experiment - at the very beginning to align the eye tracker with the screen.

Figure 5.5: The 13 calibration points (red) and 3 test points (blue) used in the second stage of the experiment. The red-blue point was used as a calibration and test point.

Eye movements were recorded using the eye tracker and setup described in Section 5.1. The experiment was conducted 11 times on four different people. Since both eyes were tracked, this gave us de facto 22 separate runs of the experiment.

Firstly, we show how the accuracy of the eye tracker degrades over time. The data used for this was taken from the second stage of the experiment where we use our own calibration method. If we opt not to use interpolation, the raw fixation data can be used on the known positions of the three test points to see how this difference (error) from the real positions of the points changes over time. We use the Euclidean distance as a measure of the error. For each group of three test points shown, the average error was calculated for these and plotted over time. Figure 5.6 shows the average error with its standard deviation that is depicted as error bars. Next we compare our method

0 50 100 150 200 250 300 350 400 20 30 40 50 60 70 80 90 100 110 Time (secs)

Avg. Error (pixels)

Accuracy Degradation

Figure 5.6: Average degradation of accuracy of the eye tracker over the 22 experiments. Standard deviation shown as error bars.

of calibration with the standard method. Figure 5.7 shows the error along with the standard deviation (error bars). There is a clear improvement of accuracy with the kriging method.

Figure 5.8 shows the interpolated shift ∆x(x, y, t) at a chosen timet over the entire

screen. We see that the shift varies significantly from approx. -10 to over 60 pixels. Figure 5.9 shows the interpolated shifts ∆x(·,·, t) and ∆y(·,·, t) over time for one

selected pixel (the left-most, middle calibration point from the start until the end of the experiment) for two people. It can be seen in this figure how the shift values generally increase as the experiment progresses. The shift values increase to compensate for the increase of error. Note that kriging interpolation also provides a measure of the uncertainty for each interpolated value and the confidence intervals are also shown: the confidence intervals are zero for calibration times (no uncertainty). The further away the point is from a calibration site, the larger the confidence interval.

0 2 4 6 8 10 0 10 20 30 40 50 60 70

Test point groups

Avg. Error (pixels)

Std Calib. Method VS Our Calib. Method

Figure 5.7: Average error for each showing of the three test points with standard de- viations obtained using the standard calibration method (red) and our kriging method (blue) over the 22 experiments.

The kriging calibration approach presented here also allows for the time taken to conduct experiments to be reduced compared to standard calibration. This is because only one initial calibration step with the built-in tool is required with our kriging method. With the standard method, sometimes calibration with the built-in tool has to be repeated several times during a calibration stage. On average, the experiments using the standard method of calibration took 19% longer than the kriging approach. Shorter recording sessions both save time and also increase the comfort level of participants. The comfort level for the operator of the eye tracker is also improved since he/she is not tied to the computer controlling the eye tracker through the duration of the experiments.

Although the eye tracking recording sessions are shorter for the participants it should be mentioned that additional work is passed down to the experimentor in the

0 100 200 300 400 500 600 700 800 900 1000 0 200 400 600 800 1000 1200 1400 1600 1800 x y −10 0 10 20 30 40 50 60 ∆x

Figure 5.8: Example image representing ∆x(x, y, t = 143) interpolated on the entire

screen.

post-processing stage. Scripts need to be written to extract calibration point fixations of participants from eye tracking data. These fixations then need to be fed into the application that performs kriging. Finally, another script needs to be used to combine the original eye tracking data with shift calculations to yield the final interpolated eye position values. This additional effort can be deemed acceptable, however, considering the benefits of our method discussed above.

5.5

Conclusion

In this chapter we have proposed to use kriging interpolation to accurately correct eye tracking data. This new approach limits the need to use the black-boxed built- in calibration tool during the recordings. Kriging calibration has been shown to be more accurate than standard calibration, it provides a measure of uncertainty for all

0 50 100 150 200 250 300 350 400 −30 −20 −10 0 10 20 30 40 50 60 70 time (sec) ∆ x (pixels) (a) 0 50 100 150 200 250 300 350 400 −30 −20 −10 0 10 20 30 40 50 60 70 time (sec) ∆ y (pixels) (b) 0 50 100 150 200 250 300 350 400 −30 −20 −10 0 10 20 30 40 50 60 70 time (sec) ∆ x (pixels) (c) 0 50 100 150 200 250 300 350 400 −30 −20 −10 0 10 20 30 40 50 60 70 time (sec) ∆ y (pixels) (d)

Figure 5.9: (a) ∆x for one pixel (a calibration point) for one person; (b) ∆y for the

same person and point; (c) ∆x for one pixel (a calibration point) for another person;

interpolated data and also significantly eases the eye tracking recording process. Since eye tracking is used to validata VSAs, improving eye tracking equipment accuracy with our kriging interpolation will have a positive knock-on effect on this field of research as well as on other fields that use eye tracking. Moreover, since eye trackers are expensive (prices range frome4000-e30,000), improving them will avoid the infeasible (for most) notion of constantly upgrading them with newer, more accurate models when they become available.