Reflection on the RedEye Interface

RedEye technology provides a great opportunity to gather data which represents usage of a website. Additionally, the criterion which may be selected from the RedEye interface allows the person creating the report to narrow down data to explore certain pages and pathways of the site which may be of interest.

There are however limitations of the interface which reduces the value of web analytics data when attempting to understand user experience.

It was difficult to make a start on analysis of the live data due to the quantity of data which covered the whole website. In this study I had input from Andrew Stockwell, who informed me of the client’s interests in the site which was helpful to direct me in the analysis. Conversation with clients who may be aware of problems in the site, or areas which are critical to success of the site may be useful to highlight areas of the data which should be explored in more detail. If it is not possible to discuss details with clients, findings from exploration of the site by experts, or using very quick user tests may be used to uncover major issues with the site. These findings may then be explored in more detail within the web analytics to present realistic user behaviour within the pages of the site which are expected to cause difficulty.

The RedEye interface does not provide a breakdown of the number of seconds spent on each page by users, even though the full pathways are timed. The separate timings for each page would be useful to provide more information of what users may be doing on each page, and

how long they spend on each page. This may help analysts infer whether users are spending excessive periods on one page, and therefore may be having difficulties locating required information. Additionally, timings may help to decide whether pages are visited simply as a through route to another page, or even mistakenly if they are shown to only be viewed for a few seconds. Earlier in this paper it was mentioned that Shneiderman (1998) proposed web analytics as a method which may be used to examine the efficiency of regularly used aspects of a website. Although the total time taken to complete pathways may be used to infer this type of information, a more detailed breakdown of the step by step timings would be more useful. Despite this suggestion, it must be remembered that because web analytics data is collected without observation, it is impossible to know whether the timings presented by the data are true to the actual time spent by users on each page of a website. Users may be distracted whilst on pages, therefore time number of seconds logged as spent on each page may be much longer than the actual duration of time spent concentrating on each page.

Further restrictions of web analytics and use of the Red Eye interface are caused by limited tagging of the website. As mentioned by Weischedel & Huizingh (2006), web analytics data may not cover the whole site traffic because of untagged areas. At Red Eye, site owners can choose to tag either full pages, or tag individual links within pages. The latter option gathers more detailed data about not only the pathways taken by users, but also the specific links that users chose to access each page. This is more informative for analysts and can provide greater detail of the visibility of certain aspects of the website and the popularity of navigation links used. This information may also be useful to explore the success of promotional panels which may be used for marketing purposes. William Hill is tagged per page, rather than per link which limits the detail of information which may be collected.

According to Arroyo et al. (2006), server logs which present data similar to that shown in the RedEye interface, do not provide vital detailed information about what users are doing at a more granular level than the page-view level. Atterer et al. (2006) concur that information collected in classic log files needs to be more detailed and fine-grained to obtain a meaningful understanding of how users interact with web applications. The problem with such data is that information about how users actually interact with a web page and what they see is unavailable from such logs (Arroyo et al. 2006 and Atterer et al. (2006). As an alternative to page and link tags, eye tracking may be used to provide more detail of user experience when moving through a website. Modern eye tracking equipment now makes it possible to record and analyse; which elements are actually seen, where users look first and what users look at the most (Johansen and Hansen, 2006). Arroyo et al. (2006) also suggest that such data can be used

to quantify which sections of a webpage are read, glanced at, or skipped/ignored which can be vital in web evaluation.

Despite these advantages of eye tracking, data analysis can be very time consuming and the equipment is expensive (Arroyo et al., 2006 and Johansen & Hansen, 2006). Additionally, eye tracking equipment may reduce test validity – for example by notably slowing the system response or by requiring users to re-calibrate between tasks (Johansen & Hansen, 2006). Arroyo et al. (2006) also make the very valid point that eye tracking does not represent frequent visitors to a site, nor replicate their surfing conditions because the participant is aware of the testing and such tracking usually takes place in lab conditions. The fact that eye tracking can not be used in users’ natural environment makes it impossible to use with web analytics. However, where the benefits of eye tracking may be utilized is during think aloud sessions to help determine whether users’ comments truly represent what they see.

Having reviewed the use of live web analytics data in understanding user experience, the next phase of this study involved gathering and analysing controlled web analytics data.