Experiment 4: Discussion

The overall purpose of Experiment 4 was to further elucidate the processes involved in successful performance on the Latin Square Task. As it turns out, most of the AOI predictors ended up being related to unsuccessful performance, with longer gaze on certain metrics related to higher failure rates. The predicted impact of RC and Steps was found, with a large detriment as these item characteristics increased. This is not at all surprising given the earlier experiments on the LST but was nonetheless necessary to account for what would otherwise be noise in our more subtle analysis of the eye tracking metrics. Of the gaze metrics, we found the same toggling findings as Laurence et al. (2018), where the number of revisits to the response options was inversely related to success. This may indicate that unsuccessful participants are more unsure of their answer, frequently returning to the

response options to search for possible solutions. Successful participants meanwhile,

understand the rules, and are only looking to the response options to confirm their response. The analyses on gaze duration metrics were more novel, with no work (until now) being done that specifically identifies areas of interest related to success on the LST. Of particular interest was the relational cells, which are necessary to solve the item. These were, contrary to the hypotheses, not related to item success overall. This could be because the clear, singular rule of the LST (that each row and column must contain only one of each element) and the presence of the ‘?’ in the target cell makes it easy for all participants to eventually identify the relational cells, though only successful participants then know what to do with these important cells. Although the relational cells become less obvious at the higher complexity of 4D items, there are often fewer filled cells in general in 4D items, so

identifying the appropriate cells is still common to all participants. This conclusion is

somewhat incomplete, particularly when considering the important findings on the distractor cells. Distractor cells are cells that are filled with shapes but are in no way necessary to solve the item. Perhaps the most insightful finding in this experiment is that gaze time on these distractor cells was significantly negatively related to item success, with (on average) every 1 second of gaze time spent on distractor cells leading to a substantial 16.3% decrease in the chance of solving the item correctly (when controlling for item characteristics and other AOIs). Taken in isolation, the distractor cells finding seems to indicate that success may indeed be related to identifying the important cells (relational cells) among the filled cells. However, taken together with the non-significant relational cells finding, the results suggest that all participants eventually identify the important cells, but those who linger on the distractor cells are the ones that fail. Successful participants can identify the important cells and swiftly disregard the distractors, making their solution process more effective. Failing

participants can also identify the important cells, but they appear to have more trouble disregarding the distracting information given in the matrix.

The results of the interim steps on 2-step items were also insightful. The fact that neither the interim target cell nor the interim relation cells predicted success seems to indicate that failing participants could identify the initial step despite the target ‘?’ not helping them with this identification. In this case though, the final relation cells did predict success. This is particularly interesting since the interim target cell is not marked, so if participants were struggling to find the solution pathway, the gaze data on this cell would indicate this is where unsuccessful participants are getting stuck. However, it appears that successful participants are the ones that not just identify the interim cell, but successfully move on from it to the second step. Either that, or they are working backwards – using the ‘?’ to identify what cells are required to solve the item. Whichever it is, it is something unsuccessful participants are failing to do. Of course, once again, the distractor cells also related negatively to success.

Experiment 4 provided another perspective from which to analyse the LST. Analysing eye tracking results necessarily involves some assumptions about the data being made. For instance, although it appears distractor cells are causing problems for failing participants, it is possible that failing participants are simply looking all over the matrix. This could also explain why relational cells were not significantly related to success – both successful and unsuccessful participants look at the relational cells, but for different reasons: successful participants identify the relational cells are integral to the solution while unsuccessful participants are simply looking everywhere. Thus, it may not necessarily be the distractor cells causing them issues but rather, gaze time on distractor cells are an outcome of their poor capability to solve the item. It should also be said that the data here is limited to a small sample size, and how these gaze metrics relate to individual differences variables, such as performance on Gf tasks, would be of interest in future research. Nonetheless, these eye

tracking results are (to the best of my knowledge) the first to be conducted on the LST and certainly the breadth of metrics that can be considered in the LST indicate it may be a fruitful task for future gaze analysis (in comparison to Laurence et al. (2018) who was largely limited to just response option revisits due to the choice of task). In addition, they certainly

contribute additional insight into the DC effect found throughout Experiments 1-3, which we turn to in the next section.