Experiments in Chapters 8, 9 and 10 were performed using an eye-tracking ma-chine. Eye-tracking is the technique used to capture and measure eye movements, allowing analysis to be performed on the patterns of visual attention of a user performing a series of specific tasks. From these analyses, inferences can be made about the user’s cognitive processes [Olsen et al., 2010]. Eye-movement is
typi-cally divided into two types 1) saccades (quick movements of the eye from one location to another) and 2) fixations (cessation of the eye movement, pausing on an object of interest for a period of time). Eye-tracking data is collected and then interpreted through visual representations such as gaze plots (a presentation of saccades between fixations, showing the eye scan path), or heat maps showing time spent at each fixation (example Figure 8.2 on page 129for an example of a gaze plot).
Experiments were performed using a Tobii T60 eye-tracker. This has an opti-mal operating distance of 65 cm, so the participant must be place approximately this far away from the eye-tracker. The T60 can capture stimuli at a maximum radius of 35 degrees and is integrated with a high resolution 17” monitor.
Figure 6.3 on the following page shows the pipeline process which was fol-lowed using the Tobii studio software to complete experiment design and record-ing through to statistical data exportation. Tobii studio was used to input the experimental design with selected test media and manage the participants and experiment counter-balancing. During the experiment process, participant eye-movement was recorded. Areas of interest (target tags) were defined within the tag cloud images and the software collected metrics regarding those areas of in-terest (such as the time to first fixation). The software was used to play back video recordings with eye-movements, to manually analyse eye movement search patterns within the media.
6.5 Summary and discussion
Our systematic mapping study of tag cloud research showed us that to broadly evaluate Taggle we should consider multiple targeted evaluation strategies. To that end, we followed a procedure of mapping out our overall evaluation strategy based on evaluation approaches presented byLam et al.[2012]. From this map we selected evaluations and appropriate methodologies. Experimentation conducted from this map required generation of realistic datasets sourced from both general and software engineering domains. Three of our experiments were run on eye-tracking machines, allowing us to perform analysis on patterns of user visual attention. The process of conducting an eye-tracking experiment also required
Figure 6.3: Pipeline of an eye-tracking experiment
carefully management and adherence to a pipeline process from experiment design to statistical data exportation.
Heuristic Evaluation of Taggle 7
An inexpensive and popular method for evaluating the usability of an interactive tool is to employ an evaluation using a set of heuristics such as Nielsen’s Ten Usability Heuristics for user interface design [Nielsen, 1992]. In this type of eval-uation, a number of experts review a tool and determine how well it follows some predefined guidelines. As part of our overall evaluation strategy, we conducted a heuristic evaluation of interactive tag cloud visualisation tool Taggle using a heuristic set specially designed for information visualisation tools. Subjective user feedback was sought to clarify research questions regarding the comprehen-sibility of the visualisation technique, as well as assessing the system usability. In
§7.1, we outline the set of heuristics used to evaluate Taggle. The methodology of the evaluation is detailed from §7.2 through §7.5. Results from the heuris-tics themselves, the guideline checklist and questionnaire are presented in §7.6.
Finally, subsequent adaptations resulting from the evaluation are presented and summarised in §7.7 and §7.8.
7.1 Heuristics
Heuristic evaluation is intended as a “discount usability engineering” method as opposed to an expensive user trial, in particular because so few participants are needed. Nielsen’s recommendation was to use three to five evaluators since that number is sufficient to identify most of the issues.
Information visualisation tools require a set of heuristics specifically tailored to finding usability issues that focus on the process of data exploration and visu-alisation techniques. Existing research has identified a number of heuristic sets and guidelines. The well-known ‘information seeking mantra’ by Shneiderman [1996] guides successful data exploration. Luzzardi et al. [2004] proposed an ex-tended set of ergonomic criteria for information visualisation techniques which were designed to assess both visualisation and interactivity techniques for hierar-chical data representations. Zuk and Carpendale[2006] outlined a set of heuristics based on previous theories and principles in perception and cognition by Bertin [1983], Tufte [1990] and Ware [2004]. Finally, Forsell and Johansson [2010] took these and other developed heuristic sets such as [Nielsen, 1992] and used em-pirical methods to synthesize them into a new set of ten heuristics which could provide the widest possible explanatory cover proportionally for all 63 heuristics presented in their study. It is this set of heuristics which was used to evaluate Taggle. Artefacts used in the heuristic evaluation sessions (including a complete description of the heuristics with additional guidelines on how to apply them) are available in Appendix B.
The purpose of the heuristic evaluation was to elicit user feedback which resulted in design refinements, and to check the tool was sufficiently detailed to be used in further trials. Subjective feedback was gained on certain points of interest to help shape tasks for future experiments. In particular, we wished to clarify the following research questions:
RQ: Is the visualisation technique itself instinctively comprehensible? Can the visualisation supply the user with a good overall picture of the data?
RQ: Can users infer general information about the data from interacting with the visualisation? What kind of information?
RQ: Is the data mapping process understandable?
RQ: Is the supplied system interactivity sufficient for accumulating knowledge about the data?
By evaluating our software through the use of a heuristic set, we wished to elicit user opinion on the system usability with regards to visual representation, interactivity, flexibility and consistency in design choices.