Graphics engine visual mappings

The graphics engine and visual mapping transform and present the pre-processed data to the analyst. The use of terms such as “graphics engine”, to the modern mind, immediately implies the use of a computer and this is usually the case in the modern world. It should be remembered however that this is not the only means of creating visualisations and many other techniques (such as model building, painting or sketching and sonification)

Page | 32

remain valid forms of visualisation beyond the 2D / 3D graphics abilities of the modern computer. The visual mapping phase selects appropriate symbols to represent data items; the study of symbols and how they convey meaning is known as semiotics. The field originated in the United States in 1868 with the work of logician and mathematician Charles Sanders Peirce (Brent, 1998; Peirce, 1868). Semiotics as a field is defined as “The study of signs and symbols and their use or interpretation” (Oxford English Dictionary, 2014) and is closely related to linguistics as the use of signs and symbols effectively forms a visual language. Attempts have been made to develop a general classification for all signs & symbols (Bertin, 1983) but no single system has gained wide acceptance with most seeming to be based on “arguments by example” rather than formal experiment (Ware, 2012). Despite this there is broad agreement that symbols can be divided into three areas:

i. Semantic – The relation between signs and the things to which they refer (their meaning).

ii. Syntactic - Relations among signs in formal structures (a set of symbols that may be constrained by rules that are specific to it).

iii. Pragmatic - Relation between signs and the sign using agents (usually humans). Reviewing the above areas it might be concluded that visualisation will be a tool only of use to those from the same or similar social / cultural background. Clearly all three elements seem to require some agreed social or cultural context for proper interpretation of the visualisation. To illustrate this point consider the semiotic breakdown shown in Table 2-1 where a complex UK road sign / symbol has been analysed by semiotic area.

Semiotic area Sign / Symbol

Semantic – “Meaning” - Water / River / Sea - Warning / Danger

- Wall / Barrier - Car / Road user(s) Syntactic – “Combination of symbols to give

new meaning” - Quayside or river bank

Pragmatic – The using agents - Road user(s)

Table 2-1: An analysis of a UK road traffic sign by semiotic area.

It is certain that an understanding of the social / cultural background can greatly aid the effective interpretation of a visual representation. However there remains a difference between “greatly aid” and “is necessary / required”. In the first case we should proceed by establishing a set of conventions which are learned and adhered to whereas in the second it can be expected that different people from different social and cultural backgrounds will interpret an image in the same way. Fundamentally the question becomes whether or not symbols represents a learned language or a universal language that can express concepts across social and cultural divides. Research is divided on this point but the majority seem to favour the interpretation of symbols as a universal language. For example Deregowski has reported that both adults and children in a remote region of Zambia with little graphic art could still easily match photographs of toys to the actual toys (Deregowski, 1968). Similarly Hochberg and Brooks raised their own daughter to the age of two in a picture free house; they never read to her from picture books nor indicated that a picture was a representation of anything. Despite the lack of input telling her that pictures had any kind of meaning and any instruction on how to interpret pictures she could correctly identify objects in line drawings and black-white photographs demonstrating that interpretation of visual

Chapter 2: Visualisation

Page | 33

images / symbols is not a learned skill (Hochberg & Brooks, 1962). Nevertheless counter arguments can be raised especially when non-pictorial images are used. In Table 2-3 we identify the red triangle as a warning / danger symbol. It is difficult to see how this designation can be anything but a learned social / cultural rule applied to the context of road signs. It bears little resemblance to any real world object. How then do we explain the difference between pictorial representations of a real world object as a universal language when other non-pictorial representations appear to be learned? Pearson et al., argue that “The most probable explanation is that, at some stage in visual processing, the pictorial outline of an object and the object itself excite similar neural processes”(Pearson, Hanna & Martinez, 1990). Ware argues that this view is plausible as “one of the most important products of early visual processing is the extraction of linear features in the visual array. These may be either the visual boundaries of objects or the lines in a line drawing” (Ware, 2012).

From the studies discussed above we may ultimately make a case that the interpretation of symbols used on visualisations ultimately depends on both the human visual system, the natural environment in which it has evolved and the social / cultural learning of the individual. This fact allows the division of symbols into the following categories:

Symbol Type Description

Sensory Refers to symbols / visualisations that express meaning by using the perceptual processing power of the brain without learning.

Arbitrary Refers to symbols / visualisations that express meaning through learned social / cultural conventions (and therefore lack a perceptual basis).

Table 2-2: Colin Wares symbol classification scheme based on the symbols learned or un-learned meaning (Ware, 2012).

A graphical language can, therefore, be seen as a combination of symbols that fall into both the sensory and arbitrary categories. Whether or not any graphical language will represent a “universal language” or a “learned language” will depend on which category the majority of its symbols fall. There are probably very few graphical languages composed of entirely of only sensory or arbitrary symbols and therefore no completely universal language or completely learned language. In light of this we may re-examine our assessment that the language of road traffic signs is a learned one dependent on society and cultural training. Sign / Symbol Symbol

Type

Comments

- Water / River / Sea Sensory In our environment we encounter waves most frequently in liquids. The most common liquid we encounter is water. Our brains develop to pattern recognise waves and associate them with water. - Warning / Danger Arbitrary Both the use of a triangle and the adoption of the

colour red are associated – in western culture – with warnings and danger.

- Wall / Barrier Sensory Walls and barriers abound in the natural environment and our visual system adapts to recognise them. This symbol combines elements of a cliff face and a man-made wall.

Page | 34

Sign / Symbol Symbol Type

Comments

- Car / Road user(s) Sensory Our environment conditions our visual systems to recognise the pattern of a car / vehicle in much the same way as a dangerous predatory animal while associating this with a particular environment (the road).

Table 2-3: Quayside or river bank road traffic sign analysed using Ware's symbol classification scheme.

As can be seen in Table 2-3 when Ware’s classification scheme is applied to the elements of the Quayside or river bank road traffic sign we can see that, of the signs four elements, three are categorised as sensory and one is arbitrary. From this we may categorise this symbol as a ‘sensory’ symbol that expresses meaning through the perceptual processing power of the brain without learning.

It would be inappropriate to infer that all road traffic signs attempt to fall into the sensory category and hence form a ‘universal’ graphical language from an examination of a single road sign. Fortunately the development of road traffic signs is a well-documented area and some research will quickly show that the designers employed seven guiding principles when designing traffic signs. These are (Shinar et al., 2003):

i. Spatial Compatibility ii. Conceptual Compatibility iii. Physical Representation iv. Frequency

v. Standardisation vi. Singular Functionality vii. Visibility

The most relevant of these are items (ii) and (iii). Conceptual Compatibility is defined as “a driver will know the meaning of a symbol without having to reflect and interpret its meaning”. Clearly then the designers are targeting ‘sensory’ symbols where meaning arises from visual processing rather than the cognitive application of learned knowledge. Physical Representation is defined as “a driver will experience what is shown on the sign”. This immediately requires the sign to reflect the physical world / reality that the user (driver) is or is about to encounter. Inevitably then the mental model being built in the drivers mind by the symbol should reflect the physical world. As we observed earlier the drivers most powerful perceptions of the physical world arise from the visual sense.

Given this it seems appropriate to classify the ‘graphical language’ of road signs as an attempt to create a universal graphical language. It also serves to inform us that if we wish our own visualisations to be universally understood by a wide audience we should endeavour to match data with a visual representation that falls into Ware’s ‘sensory’ category.

Ware has examined the processes of the human visual system and has distilled from his studies eight guidelines for mapping data to a representation.

i. Design graphic representations of data by taking into account human sensory capabilities in such a way that important data elements and data patterns can be quickly perceived.

Chapter 2: Visualisation

Page | 35

ii. Important data should be represented by graphical elements that are more visually distinct than those representing less important information.

iii. Greater numerical qualities should be represented by more distinct graphical elements.

iv. Graphical symbol systems should be standardised within and across applications. v. Where two or more tools can perform the same task, choose the one that allows for

the most valuable work to be done per unit time.

vi. Consider adopting novel design solutions only when the estimated payoff is substantially greater than the cost of learning to use them.

vii. Unless the benefit of novelty outweighs the cost of inconsistency, adopt tools that are consistent with other commonly used tools.

viii. Effort spent on developing tools should be in proportion to the profits they are expected to generate. (This means that small market custom solutions should be developed only for high value cognitive work).

The impact of Ware’s guidelines can be appreciated by contracting two visualisations of the same data. In Figure 2-13 the grades achieved by a group of university students has been visualised as a histogram that follows Ware’s guidelines. In Figure 2-14 the same grades are visualised as a data table that breaks several of Ware’s guidelines. In each figure whether guideline has been followed or broken is indicated ( or ).

Page | 36

Figure 2-14: Data table representing student grades the breaks several of Ware’s guidelines.

2.3.4 Visual and Cognitive Processing, Data Exploration and View Manipulation