Comparison of data elements

Comparison of data points is another key task useful in any software engineering or multi-variate dataset. In Figure 4.22 on the next page font size and order are dually mapped to field ‘conditional complexity’. It is obvious for example, that

Figure 4.22: Finding classes with the highest value of a metric using dual order and font size mappings

tag ‘JUnitSystem’ has a higher conditional complexity than tag ‘JUnitMatchers’.

When font size and order are not mapped to the same data field it may not be so obvious. Using drag and drop a user may pull out the relevant tags and put them next to one another for easier comparison — see Figure 4.23 on the following page. Additionally, a user may hover over the tags and view the class metric information in the pop-up details-on-demand for comparison.

4.5 Summary and discussion

In this chapter, we introduced the tag cloud visualisation tool Taggle. This tool was inherited from previous research and extensively modified to create a stable system able to explore software quality metrics and other more general multi-variate data. Most notably, a variety of implementation changes were made as a result of considering design aspects from software engineering and tag cloud

Figure 4.23: Comparison of ‘JUnitSystem’ and ‘JUnitMatchers’

visual variable analysis (Chapter 3) as well as improvements generated from a heuristic evaluation by domain experts (Chapter 7).

Data can be gathered from a variety of sources and then converted (through XML transform or conversion utility) into a special Taggle XML format. This XML file is then input into Taggle through the user interface, where the user can view generated tag clouds and customise the visual property mappings as they explore the dataset. Interface and visual encoding design choices such as the visual properties included, constraints and options for each property, and fil-tering selections were made with respect to the software engineering and design considerations listed in previous chapters. Some choices, such as the data sum-mary screen and categorical data options, were incorporated by request during the heuristic evaluation.

In Chapter 3, a list of tasks was presented that represented meaningful ways users could interact with software data. These included tasks from data mining task types — clustering, summarising, associating and classifying — and involved activities such as identifying similar data characteristics, distribution and

corre-lations. In §4.4, we showed how a user could interact with a sample software engineering dataset to complete each task using the system. Taggle’s design was intended to cope specifically with the challenges of software engineering datasets, by using appropriate visual mappings available in tag clouds to render the data.

Systematic Mapping Study 5

of Tag Cloud Research

Information visualisation techniques can be challenging to evaluate [Plaisant, 2004]. This is because, in addition to general evaluation challenges (such as choosing appropriate questions and tasks, defining methods and then executing the evaluation correctly) the visualisation focus on the data exploration process is difficult to capture and quantify. We embarked on a systematic mapping study of previous research evaluating the tag cloud technique or interactive tools that included tag cloud visualisation. This study identified topics, fields or domains that had not been extensively researched, and approaches and methods which had been used for evaluation. To classify our work, we used a set of information visualisation guiding scenarios which are outlined in§5.1. Research questions and goals for the study are presented in §5.2. The research methodology including data sources, study selection and data extraction are outlined in§5.3. Results for research topic, methods, domains and approaches are discussed in §5.4. Finally,

summary and conclusions are presented in §5.5 and §5.6.

5.1 Strategies for evaluation

In 2012, Lam et al. [2012] identified seven guiding scenarios for information vi-sualisation evaluation. These scenarios were gathered from a systematic review of 803 information visualisation papers (345 of which included evaluations). Of these scenarios, four can be roughly defined as evaluation of the data analysis pro-cess (EWP, VDAR, CTV, CDA – described in §5.3.3) and the remaining three evaluate the visualisation use (UP, UE, VA). These two types of strategies have different goals and use different methodologies.

Evaluation of the data analysis process has a goal of understanding the un-derlying process and roles played by the visualisation itself, and captures a more whole-tool holistic view. The results from this type of analysis may be more meaningful as realistic tasks and scenarios are used. However, results can be more difficult to quantify. Also, the whole tool is evaluated so evaluation may require full featured and mature tool.

The visualisation use type strategies do not evaluate the whole tool but a system slice or technique. They are used to evaluate design decisions, explore the design space, benchmark existing systems or test usability. For these strategies, outputs are easier to quantify generated insight. There is a need to break the evaluation into techniques or visual encoding types, so careful prioritisation is needed. Because of this breaking off into sections, more than one experiment may be needed. Tasks may also need to be heavily abstracted which impacts realism.

In the systematic review performed by Lam et al. [2012], only 15 percent of papers used data analysis process type strategies in their evaluation. They concluded that evaluation in the information visualisation sector has been fol-lowing in the footsteps of evaluations for Human Computer Interaction (HCI) and Computer Graphics (CG), both of which are traditionally focused on con-trolled experiments and usability evaluations. The data process strategy research questions (such as a tools support for reasoning, knowledge discovery or decision making) are of high relevance and practical value. Lam et al. [2012] highlighted

the need to think critically about the goals of the types of evaluations needed for information visualisation.

We were interested to find out what types of evaluations had been performed for tools utilising tag cloud visualisation techniques, and what research topics and domains the evaluations focused on so we performed a systematic mapping study on 60 selected primary studies from 2007 to 2012.