ABSTRACT
BAE, JUHEE. A Framework for Communicating Visual Structure using Gestalt Principles. (Under the direction of Benjamin Watson.)
Graphic designers have excelled at visual communication for decades. One of the primary methods they use is visual grouping, which organizes related items together into a complex, often hierarchical structure. Such structure is a crucial aid to understanding. Yet in their applied focus, designers have not generalized their practice to enable non-experts to use visual grouping successfully.
We study the use of visual grouping through two transforms: from informational to visual, and from visual to cognitive. Informational structure is the organization of knowledge; visual structure is the organization of the visual itself; and cognitive structure is the organization that the viewer creates from the visual. When expert or non-expert designers create visuals, they are defining a mapping from informational to visual structure. When viewers create an understanding of a visual, they are defining a mapping from visual to cognitive structure.
Researchers described visual grouping with Gestalt laws, including proximity, similarity, closure, good continuation, common fate, common region, and connectedness. Unfortunately, this research has not studied how these laws may be combined in any depth, particularly in the applied, graphical settings that concern us.
Our research begins with two applied studies of visual communication of structure in very constrained and applied settings. First, we propose a new method for depicting layered graphs,
Quilts, and compare it to two existing depictions. We find that Quilts support path finding, a crucial measure of graph understanding, much more successfully than those previous depictions. Second, we proposeGraphTiles, a method for depicting the local graph neighborhoods resulting from search on mobile devices, and compare it to a standard search display. We find that it enables more effective search.
We then examine visual communication in a more general and basic fashion. To understand how Gestalt grouping cues can be used to communicate informational structure effectively, we construct visuals depicting real world content with controlled combinations of grouping cues, and measure their effectiveness in communicating the structure of that content. Among many other lessons, we find that grouping cues are most effective when used to reinforce one another, that combining more cues generally improves structural communication, and that common region is a particularly effective grouping cue.
matching visual structure. The designer can then improve one or more of these prototypes. Our evaluation shows that visuals created by the tool were most effective at communicating knowledge structure, visuals created by the tool and improved by designers were most appealing, and that visuals created by designers without the tool were never better than the other two types of visuals.
© Copyright 2014 by Juhee Bae
A Framework for Communicating Visual Structure using Gestalt Principles
by Juhee Bae
A dissertation submitted to the Graduate Faculty of North Carolina State University
in partial fulfillment of the requirements for the Degree of
Doctor of Philosophy
Computer Science
Raleigh, North Carolina
2014
APPROVED BY:
Patrick FitzGerald Christopher Healey
Robert St.Amant Benjamin Watson
DEDICATION
With love, to my awesome God and my parents, Young Moon Bay and Dong Sook Kim.
BIOGRAPHY
Juhee Bae was born in Daegu, Korea, on November 4, 1979. She received a B.S. degree
from Ewha Womans University, Seoul, Korea, in 2003 and a M.S. degree from Seoul National
University in 2005, both in Computer Science. She studied at Central Washington University,
Ellensburg, WA, as an exchange student in her undergraduate junior year. She had worked at
IBM Korea, in Seoul, as a software engineer for three and one half years from 2005.
Juhee has joined the Ph.D. program in the Computer Science at North Carolina State
University since fall 2008. She has served for a Teaching Assistant in mostly computer graphics
courses since 2009 and for a Research Assistant in a project funded by the U.S. National Security
Agency since 2013. During her graduate career, she applied her studies as a research intern at
Nokia Research Center, Palo Alto, CA for 6 months in 2011 and at SAS Institute Inc., Cary,
NC from summer 2012 to 2013. She was awarded a graduate scholarship from KSEA-KUSCO
in 2012 and has been a member of Honor Society of Phi Kappa Phi.
Juhee’s research interests include data and information visualization, human-computer
in-teraction, and design. She enjoys mountain hiking, traveling, and drawing pictures with colored
ACKNOWLEDGEMENTS
I am most grateful to my advisor, Dr. Benjamin Watson, for his guidance and endless
sup-port during my graduate career. I also appreciate my committee members professor Patrick
FitzGerald, Dr. Robert St. Amant, and especially Dr. Christopher Healey for their support in
times of need, helpful advice, and invaluable feedback.
I would like to acknowledge Ji Hae Kim, Dr. Richard Kim, Dr. Sung-Eun Yoo, Dr. Sung
Woo Kim, and all the members of Mahanaim in the First Korean Baptist Church for their
prayers, love, and warm support. I am grateful and thank God that I was able to meet them
here in Raleigh.
Many friends and colleagues have helped me in various ways throughout the journey and
would like to thank them as well. They are — Qian, Adam, Lauren, and Chris from the
Vi-sual Experience Lab for their feedback and humor; Tomer and Barry for helping me out with
the experiments; Agnes, Dr. Anne McLaughlin, and Dr. Douglas Gillan from the psychology
department for their statistical feedback; Vidya Setlur and Ravi Devarajan for providing
intern-ship opportunities; and all staff members in the Department of Computer Science for taking
care of all administrative needs. I also thank Ja Yeon Jeong at Samsung and Young Jin Park at
SAS who have helped me in part of my research experiments. It will be a long list to mention
all but would like to thank all the friends (Korean, American, and from other countries) I met
during my graduate studies. I would like to extend my thanks to many students, especially
computer science students, at North Carolina State University and anonymous Turkers who
participated in my online experiments.
I sincerely thank my parents for their unconditional love, support, prayers, and patience,
along with my two younger brothers, Yo Seob and especially Jae Hyun. With their
encourage-ment and positive thinking, I was able to finish this dissertation.
TABLE OF CONTENTS
LIST OF TABLES . . . vi
LIST OF FIGURES . . . vii
Chapter 1 Introduction . . . 1
Chapter 2 Goal and Contributions . . . 4
2.1 Goal . . . 4
2.1.1 Applied experience with visual grouping . . . 4
2.1.2 Basic study of complex grouping . . . 5
2.1.3 A tool to automate informational to visual transform . . . 5
2.2 Contributions . . . 6
2.2.1 A new technique for visualizing layered graphs . . . 6
2.2.2 A new technique for visualizing graphs on mobiles . . . 6
2.2.3 A new technique for measuring structural understanding of visuals . . . . 6
2.2.4 Improved understanding of the visual to cognitive mapping . . . 6
2.2.5 A design tool for automating the informational to visual transform . . . . 7
Chapter 3 Collecting and summarizing prior work . . . 9
3.1 The informational to visual mapping . . . 9
3.1.1 Design . . . 9
3.1.2 Visualization . . . 11
3.1.3 Graphics . . . 11
3.2 The visual to cognitive mapping . . . 12
3.2.1 Psychology . . . 12
3.2.2 Education . . . 19
Chapter 4 Applied experience with visual grouping . . . 22
4.1 Depicting layered graphs . . . 22
4.1.1 Depicting skip links . . . 23
4.1.2 Comparing Quilts to other graph depictions . . . 30
4.2 Supporting mobile search with graph browsing . . . 40
4.2.1 Imprecise search on mobile devices . . . 40
4.2.2 Mobile Visualization . . . 41
4.2.3 Contributions . . . 41
4.2.4 The Graphtiles Interface . . . 41
4.2.5 Diary Study . . . 43
4.2.6 Link design comparison . . . 46
4.2.7 Evaluative comparison . . . 50
4.2.8 GraphTiles in non-bipartite graphs . . . 51
Chapter 5 Basic Study of Combining Visual Grouping Cues to Communicate
Complex Information Structure . . . 55
5.1 Related Work . . . 55
5.2 Studying disjoint combinations of grouping cues . . . 56
5.2.1 Methods . . . 56
5.2.2 Hypotheses . . . 61
5.2.3 Results and Discussion . . . 61
5.3 Studying reinforcing combinations of grouping cues . . . 63
5.3.1 Methods . . . 65
5.3.2 Hypotheses . . . 70
5.3.3 Results . . . 71
5.3.4 Discussion . . . 80
5.4 General Discussion . . . 81
5.5 Lessons for designers . . . 82
5.5.1 Lessons by grouping cues . . . 82
5.5.2 Lessons by content . . . 83
5.6 Preparing the Tool . . . 83
Chapter 6 A tool automating the informational to visual transform . . . 85
6.1 The design tool . . . 85
6.1.1 Informational structure: Input from web pages . . . 85
6.1.2 Visual Structure: Applying a combination of grouping principles to create prototype designs . . . 88
6.2 Collecting Data for Evaluation . . . 89
6.3 Evaluating the design tool . . . 90
6.3.1 Our goals . . . 92
6.3.2 Methods . . . 92
6.3.3 Hypotheses . . . 102
6.3.4 Results . . . 103
6.3.5 Discussion . . . 129
Chapter 7 Conclusion and Future Work . . . .131
LIST OF TABLES
Table 4.1 Significant main effects and significant two-way interactions on path finding time in the skip link study. . . 26 Table 4.2 Significant main effects and significant two-way interactions on path finding
accuracy in the skip link study. . . 29 Table 4.3 Significant main effects and interactions on path finding time in the depiction
study. . . 33 Table 4.4 Significant main effects and interactions on path finding accuracy in the
depiction study. . . 36
Table 5.1 Tree edit distance for each disjoint combination of grouping cues, with lower distances indicating viewers understand structure better. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 62 Table 5.2 Significant main effects on accuracy of structural communication using
dis-joint combinations of grouping cues, as measured by tree edit distance. . . . 63 Table 5.3 Significant effects on edit distance and preferences for the per cue
combina-tion analysis. . . 73 Table 5.4 Significant effects on edit distance and preferences for the reinforcement
anal-ysis. . . 74 Table 5.5 Distance, trial time, and preference means for the per grouping cue
anal-ysis. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 76 Table 5.6 Significant main effects and interaction of grouping cue, visual density, and
visual type on the edit distance for the per grouping cue analysis. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 77 Table 5.7 Significant main effects and interaction of grouping cue, visual density, and
visual type on the time for the per grouping cue analysis. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 78 Table 5.8 Significant main effects and interaction of grouping cue, visual density, and
visual type on the preference for the per grouping cue analysis. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 78
Table 6.1 Modifications student designers made to the tool-generated prototypes. . . . 91 Table 6.2 The two best and worst designs displaying varying individual content for
each tool type, as measured by edit distance in the more balanced study. Designs marked with the same letter contain the same content. Note the large effect of content complexity. . . 106 Table 6.3 The two best and worst designs displaying varying individual content for
Table 6.4 The two best and worst designs displaying varying individual content for each tool type, as measured by rated understandability in the more bal-anced study. Designs marked with the same letter contain the same content. Alignment and symmetry remain important, while simplicity in content and color schemes gain importance. . . 108 Table 6.5 The two best and worst designs displaying varying individual content for each
tool type, as measured by rated overall quality in the more balanced study. Designs marked with the same letter contain the same content. Attributes effective for both appeal and understandability seem effective here. . . 109 Table 6.6 The two best and worst designs displaying common content for each tool
type, as measured by edit distance. Unlike the important effect of content complexity in results with varying individual content (Table 6.2), with iden-tical content, here effective communication of structure seems paramount. . . 110 Table 6.7 The two best and worst designs displaying common content for each tool
type, as measured by rated visual appeal. As with individual content, align-ment, symmetry and color choice seem important here. Clearly viewers do not find the tool’s use of connectedness appealing, though it is quite under-standable. . . 111 Table 6.8 The two best and worst designs displaying common content for each tool
type, as measured by rated understandability. Different from individual content, symmetry seems less important, and communicating structure more so. . . 112 Table 6.9 The two best and worst designs displaying common content for each tool
type, as measured by rated overall quality. As with individual content, these examples seem to exhibit characteristics that worked for rated appeal and understandability. . . 113 Table 6.10 Means for each tool type. Italicized measures had a significantly main effect
on tool type. (dep. var. = dependent variable, mb = more balanced, ub = unbalanced, -a = all data, -b = best tool version) . . . 114 Table 6.11 Means for each cue combination. Italicized measures had a significantly main
effect on cue combination. (cmn = common region, cnc = connectedness, col = color similarity, prx = proximity, aln = alignment, dep. var. = dependent variable, mb = more balanced, ub = unbalanced, -a = all data) . . . 115 Table 6.12 Means for each complexity. Italicized measures had a significantly main
effect on complexity. (dep. var. = dependent variable, mb = more balanced, ub = unbalanced, -a = all data) . . . 115 Table 6.13 Means for each page type. Italicized measures had a significantly main effect
on page type. (dep. var. = dependent variable, mb = more balanced, ub = unbalanced, -a = all data) . . . 116 Table 6.14 Significant main effects and interactions on edit distance. We list more
bal-anced, unbalbal-anced, all data, best tool version for only tool type related analyses. (mb = more balanced, ub = unbalanced, -a = all data, -b = best tool version) 117 Table 6.15 Significant main effects and interaction on the trial time. (mb = more balanced,
Table 6.16 Significant main effects and interaction on the rated visual appeal. (mb = more balanced, ub = unbalanced, -a = all data, -b = best tool version) . . . 119 Table 6.17 Significant main effects and interaction on the rated understandability. (mb
= more balanced, ub = unbalanced, -a = all data, -b = best tool version) . . . 120 Table 6.18 Significant main effects and interaction on the rated overall quality. (mb =
LIST OF FIGURES
Figure 1.1 Informational to visual transform. Visual to cognitive transform. . . 2 Figure 1.2 Illustration of complexity of American military strategy. . . 3
Figure 2.1 Hierarchical clustering of fish. Excerpt from Medin [43]. . . 8
Figure 3.1 Examples of gestalt principles. (a) No grouping. Grouping by (b) proximity, (c) color similarity, (d) closure, (e) connectedness, (f) common region, (g) good continuation, and (h) common fate. . . 13 Figure 3.2 Connectedness operates prior to (a) proximity, (b) similar color, (c) similar
shape, and (d) similar size. . . 15 Figure 3.3 Different hierarchies formed by shape and color similarity. . . 16 Figure 3.4 Is there competition among proximity, common region, and connectedness?
(a) Grouping with common region and connectedness. (b) Grouping closer elements with common region and further elements with connectedness. (c) Grouping closer elements with connectedness and further elements with com-mon region. . . 17 Figure 3.5 (a) Integrated presentation and (b) separated presentation. Excerpt from
Mayer 2001 [42]. . . 20
Figure 4.1 Quilts shown in color-only, mixed, and text-only depictions. The red high-light indicates a successful path from source to destination, while the source and destination nodes are highlighted with white dots (color-only), white letters (mixed) or red letters (text). . . 22 Figure 4.2 Skip links depicted in (a) color, (b) mixed, and (c) text. The transparent
red lines indicate a path from source to destination with two skip links. . . . 24 Figure 4.3 Path finding times as affected by (a) depiction, (b) nodes, and (c) layers.
Bars show standard error. . . 27 Figure 4.4 Path finding times in the skip link study, as affected by (a) depiction-nodes,
and (b) depiction-layers. Bars show standard error. . . 28 Figure 4.5 Example of a 50 node, quarter link density, quarter skip link density, and 5
layered graphs. The left most depicts a node-link diagram, middle shows a centered matrix, and right most is Quilts. Red and black boxes and white numbers respectively indicate the source and destination node. Blue links in a node-link diagram indicate backward links. The node-link diagram is relatively much larger in experimental display. . . 31 Figure 4.6 The main effects of (a) depiction and (b) nodes on path finding time in the
depiction study. Bars indicate standard error. . . 34 Figure 4.7 Path finding times in the depiction study, as affected by (a) depiction-links,
Figure 4.8 Different explicit link representations for GraphTiles. (a) text: nodes with the same name are linked. (b) color: nodes with the same color are linked. (c) connectedness: nodes with lines between them are linked. (d) proximity: nodes at/containing the same vertical position are linked. (e) texture: nodes with/containing the same image are linked. . . 42 Figure 4.9 Answering imprecise movie queries on GraphTiles. (a) View after searching
for Robert Duvall. (b) Scroll right column to locate other person. (c) Long press makes a red boundary box on John Cazale. (d) Resulting movies are reordered in middle column. (e) Non-related movies are dimmed. . . 42 Figure 4.10 Search type by success/failure, easy/difficult, open/specific, and hit/miss.
(Blue: success, red: failure, saturated color: easy, bright color: difficult, empty dot: open, filled dot: specific, smaller dot: miss, bigger dot: hit) . . . 45 Figure 4.11 Average task completion times per depiction for the explicit link
representa-tion experiment. . . 48 Figure 4.12 ComparingGraphTileswith IMDb’s mobile website. (a) and (b): The Person
QueryType; (c) and (d): the Movie QueryType. . . 53 Figure 4.13 ApplyingGraphTilesto Seattle’s music band data. (a) Band and artist
rela-tionship, (b) band relarela-tionship, and (c) interactive reordering with dimmed image when not related. . . 54
Figure 5.1 The five grouping cues used in disjoint combination in our first study (dis-joint combination indicated in ‘outer visual cue, inner visual cue’). These visuals are simplified for illustration, experimental visuals were more com-plex. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 57 Figure 5.2 Grouping letters with same colors (A and F) in a practice task. . . 58 Figure 5.3 Toothpaste label stimuli in color similarity and proximity. (a) Color
simi-larity in outer group while proximity in inner group. (b) Proximity in outer group while color similarity in inner group. . . 60 Figure 5.4 The five grouping cues used in isolation and in reinforcing combination in
our second study. These visuals are simplified for illustration, experimental visuals were more complex. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, and aln=alignment) . . . 64 Figure 5.5 Some text and imagery visuals in low and high density. Content here shows
an Amazon example for the text-dominant visual and Google Images for the image-dominant visual. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 66 Figure 5.6 (a) Google’s image search result with common region, connectedness, color
Figure 5.9 Tree edit distance, trial time (in seconds), and preference means for the per cue combination analysis. Darker red, yellow, and blue indicate more ac-curate, quicker, and preferred forms of communication. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 72 Figure 5.10 Distance and trial time decline (i.e. understanding improves) as
reinforce-ment increases. Preference grows with more reinforcereinforce-ment. . . 75
Figure 6.1 The design tool receives an informational structure and generates design prototypes. Examples fromcommon webpage. . . 86 Figure 6.2 The design tool receives an informational structure and generates design
prototypes. Examples fromindividual webpage. . . 87 Figure 6.3 Histograms of how many student designers selected which design prototypes
for each page type. (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment) . . . 90 Figure 6.4 The practice test environment for the tool evaluation study. At left, it shows
a practice stimulus, and at right, shows three rating questions and a grouping task. Three rating questions were displayed before this grouping task. . . 95 Figure 6.5 Six example individual web page stimuli. All the stimuli in each row display
the same content. The left column shows student-generated stimuli, the middle tool-generated stimuli, and the right student-modified stimuli. The tool-generated and student-modified stimuli in the top row were created using the cmn-col-prx-aln cue combination, while the same at the bottom were created using cmn-col-aln. . . 96 Figure 6.6 Six example individual web page stimuli. All the stimuli in each row display
the same content. The left column shows student-generated stimuli, the middle tool-generated stimuli, and the right student-modified stimuli. The tool-generated and student-modified stimuli in the top row were created using thecmn-col-prxcue combination, while the same at the bottom were created using cnc-col-prx. . . 97 Figure 6.7 Three example common web page stimuli. The left column shows
generated stimuli, the middle tool-generated stimuli, and the right student-modified stimuli. The tool-generated and student-student-modified stimuli were cre-ated using the cmn-col-prx-aln cue combination. . . 98 Figure 6.8 Three example common web page stimuli. The left column shows
generated stimuli, the middle tool-generated stimuli, and the right student-modified stimuli. The tool-generated and student-student-modified stimuli were cre-ated using the cmn-col-aln cue combination. . . 99 Figure 6.9 Three example common web page stimuli. The left column shows
Figure 6.10 Six example common web page stimuli. The left column shows generated stimuli, the middle tool-generated stimuli, and the right student-modified stimuli. The tool-generated and student-student-modified stimuli in the top row were created using the cmn-col-prx cue combination, while the same at the bottom were created using cnc-col-prx. . . 101 Figure 6.11 Significant main effects of tool type and pairwise comparisons for all the
measures. Best tool version (mb-b and ub-b) includes cmn-col-prx-aln and
cmn-col-aln cue combinations. Sm indicates student-modified, Sg indicates
Chapter 1
Introduction
Researchers in neurophysiology and psychology have studied how our visual system organizes basic visual elements into groups [56, 45, 47, 12, 24, 46]. Several more applied researchers also recognize the importance of grouping:
“Grouping techniques are important in helping to organize information” [11].
“A graphic should not show only the leaves; it should show the branches as well as the entire tree” [13].
Psychologists have recently begun examining the relative strengths of disjoint or competing grouping cues [12, 47, 24, 23]. Given that cognitive scientists have begun studying these complex grouping cue combinations, we believe it is time to consider the use of such combinations in more applied way in visualization.
One of the motivations for this research comes from the question of how people group elements into a structure. We ask whether Gestalt principles (3.2.1.1) describing grouping in natural settings can more generally apply to information presentations, especially in terms of computer-aided presentations such as PowerPoint demonstrations, posters, and webpages; and if so, whether a tool might be developed that aids novice designers by exploiting our knowledge of visual grouping.
Figure 1.1: Informational to visual transform. Visual to cognitive transform.
to apply it to convey the intended message throughout the whole flow of informational, visual, and cognitive structure.
Chapter 2
Goal and Contributions
2.1
Goal
Our goal is to better understand visual communication of knowledge structure by studying how Gestalt grouping cues can be used effectively, and to apply that understanding by building a tool that generates design prototypes to help novice designers to create effective and well-structured visuals.
2.1.1 Applied experience with visual grouping
The research on depicting layered graphs and browsing graphs on a mobile device are two applied studies of visual communication of structure.
Depicting layered graphs
Quilts(Figure 4.1) are alternative matrix-based depictions, which the chain matrices represent proper links in a cascading series connected by layers [9]. A primary weakness inQuiltsis their depiction of skip links, links that do not simply connect to a succeeding layer. We compared
Quilts using color-only, text-only, and mixed (color and text) skip link depictions, finding that path finding with the color-only depiction is significantly slower and less accurate, and that in certain cases, the mixed depiction offers an advantage over the text-only depiction. We confirmed that symbolic and color information depicted together support the task best in time and accuracy.
Browsing graphs on mobiles
them is represented as a link for Internet Movie Database (IMDb). A fundamental challenge is to depict which people worked on which movies. We focused on drawing connections between the elements of each column to demonstrate groups of elements in the graph. A basic method is to draw direct lines between the related elements. We used grouping by connectedness, texture similarity, color similarity, and position similarity to depict the relationship. We use touch mobile phone to enable flicking interaction on the screen for each column. Through the interaction, it enables to view all the locally related elements despite the graph size, and indicates groups by grouping principles mentioned above. We applied Graphtiles to IMDb and music data and measured accuracy, time performance, and preferences for each grouping method.
2.1.2 Basic study of complex grouping
Designers have long been exploiting the Gestalt laws of visual grouping to deliver viewers those cues using visual hierarchy, often communicating structures much more complex than the simple organizations studied in psychological research. Unfortunately, designers are largely practical in their work, but have not paused to build a complex theory of structural communication. If we are to build a tool to help novices create effective and well-structured visuals, we need a better understanding of how to create them. Our work takes a first step toward addressing this lack, studying how five of the many grouping cues (proximity, color similarity, common region, connectedness, and alignment) can be effectively combined to communicate structured text and imagery from real world examples. To measure the effectiveness of this structural communication, we applied a digital version of card sorting, a method widely used in anthro-pology and cognitive science to extract cognitive structures. We then used tree edit distance [64] to measure the similarity between perceived and communicated structures. The trial time and preference of visuals were also measured.
2.1.3 A tool to automate informational to visual transform
With the knowledge obtained from the basic study, we built a design tool that receives infor-mational structure (e.g. an outline) and produces several design prototypes applying many grouping combinations. We generated five visual prototypes with four top ranked combinations from the basic study and one combination with connectedness. The cue combinations used are
cmn-col-prx-aln,cmn-col-aln,cmn-prx-aln,cmn-col-prx, andcnc-col-prx (cmn=common region, cnc=connectedness, col=color similarity, prx=proximity, aln=alignment).
how well a viewer understands the information. From the results, we expected to study whether tool involvement improves understanding of the content.
2.2
Contributions
In line with our goals, the work we describe here makes two applied contributions, and three more basic contributions.
2.2.1 A new technique for visualizing layered graphs
Quilts are a new compact matrix-based depiction for layered graphs that does not suffer from link crossing problems. Through experiments, we confirmed that Quilts support faster path-finding behavior than two well-known alternatives.
2.2.2 A new technique for visualizing graphs on mobiles
GraphTilesare a novel method for displaying and browsing through local graph neighborhoods resulting from search on mobile devices. In our experiments examining an imprecise search task,GraphTiles enabled participants to find information twice as quickly as a standard search display.
2.2.3 A new technique for measuring structural understanding of visuals For our more basic work, we required an easily scalable technique for measuring structural understanding of visuals. Inspired by the work of Medin [43], who used hierarchical card sorting and matrix analysis to compare cognitive organizations of fish in different ethnic communities, we developed a method and interactive system that allows participants to describe hierarchies directly, rather than through recursive clustering. Because the activity is supported through an interactive interface, it can be performed remotely, for example through Mechanical Turk. We then perform pairwise comparisons of those hierarchies using tree edit distance [64]. To our knowledge, we are the first to measure structural understanding in visualization.
2.2.4 Improved understanding of the visual to cognitive mapping
each of them communicates the entire structure (reinforcing one another), that combinations with more cues are better communicators than those with fewer, and that common region is a particularly effective cue.
Chapter 3
Collecting and summarizing prior
work
In this chapter, we present previous work to understand visual structure in relation to design, psychology, education, visualization, and graphics.
3.1
The informational to visual mapping
3.1.1 Design
Hierarchy is used in different senses in design and psychology. In design, it is a system or organization that helps to visualize a structure and the relationships between elements. Some examples include the tree, nest, and stair structure [38]. Other ways to organize the information into hierarchies are by node-link diagrams, matrices, and classification diagrams. A specialized form of hierarchy is a network graph that can be displayed in layered or unlayered layouts. On the other hand, some researchers in psychology use hierarchy to refer to the order of operations. For example, previous studies have found that proximity is perceived dominantly over shape similarity, and connectedness is perceived before both proximity and shape similarity. In this case, hierarchy is conceived as an order of pattern or analysis of Gestalt principles (3.2.1.1). In the present research, we use the term hierarchy to refer to the visual structure. Here, visual hierarchy or visual structure refers to the informational structure and grouping precedence in terms of the order of grouping principles.
the elements in relation to other elements to show visual connection. Proximity is to put related elements closer together which is described to help organize information and reduce clutterness. The author provides cognitive guidelines as well as what to avoid. She also suggests regrouping the elements if there are more than 3 to 5. Regarding to Gestalt’s grouping principles, repetition seems to relate with grouping by similarity and proximity.
A previous study attempted to provide guidelines in terms of user learning. The goal was to help the designers to create displays through knowledge of the acquisition, organization, and processing of information by users [22]. This research focused on organizing text elements by line length, directive cues, paragraph indication, status bar, line spacing, functional area, text columns, and illustrations. Although this study mainly highlights text elements, it also emphasizes the value of organizing the information.
Another study [44] identified a set of screen characteristics to develop a model that mea-sures the acceptance and performance of the user. The format characteristics include balance, equilibrium, symmetry, sequence, cohesion, unity, proportion, simplicity, density, regularity, economy, homogeneity, rhythm, and order and complexity. The researchers defined computa-tional formula for each characteristic by objective measures such as the area of object, distance between objects, width and height of the frame and objects, coordinates of the center of objects, number of objects, etc. The inputs were six models of different screen layouts drawn over the original screen using an editor, and the program provided calculated results of the 14 measure-ments. With the results, they validated the aesthetics of a screen by comparing users’ ratings in high, medium, and low. They concluded that screens that had higher computational results were rated higher on aesthetics by the viewers. The screen with lowest computed values was rated low. However, the researchers compared a limited number of layouts, and the program did not provide optimized layouts based on the computational measures.
to reduce search time, this model predicted that the number of groups on a screen has an upper limit of 40. Tullis also discovered that grouping the related elements with frames reduced the complexity of the screen. However, the factors examined in the study only describe the formats of the screen, not the semantics of the screen.
Most of the prior work emphasizes the importance of organizing the information in order to deliver its message well.
3.1.2 Visualization
Although not extensive, there are work exploring applications of visual grouping in the field of visualization. In his bookVisual Thinking for Design, Ware discusses widespread use of Gestalt grouping to communicate semantic structure, but he does not discuss the complex combination of grouping cues [61]. Ziemkiewicz and Kosara [65] experimentally verify this mapping of the visual to the semantic, and suggest that it may also be influenced by viewers’ physically based interpretation (e.g. gravity) of what they see.
Wattenberg and Fisher built an analysis engine that extracts grouping from imagery [62]. They point out that the structure of visualization should match the structure of the data and convey its intention. Rosenholz et al. [54] improved on Wattenberg and Fisher’s analysis engine by generalizing it to sense color and orientation, and making it simpler to extend the tool to detect other perceptual features as well. The tool responded to Gestalt proximity, similarity, and good continuation stimuli much like people would, and was able to convincingly extract structure from text, textures, interface, information graphics, and visualizations. Later, Rosenholz et al. [53] studied the use of the tool in the design process, finding that it could be helpful. However, as Wattenberg and Fisher point out, these algorithmic models need experimental validation.
While the above research is certainly useful, it does not provide a complete solution to the problem of helping novice designers synthesize good visuals. For example, how should grouping cues be combined to communicate complex informational structures containing groups within subgroups? Existing research does not provide a clear answer.
3.1.3 Graphics
Lok and Feiner [39] created multimedia presentation tools using abstract/spatial constraint solvers with the position and size of the elements, grid-based approach, machine learning, evaluation techniques, and relational grammars. Several researches attempt to evaluate the effectiveness of layouts according to aspects of aesthetics [40, 44]. However, they do not focus on successful communication of information structure in visualizations. Even with the presentation tools, novice users struggle to deliver their message because most tools are tailored to designers and visual experts. Sometimes, novice designers may assume that their message is delivered by their own way of presentation but the audience may not, in fact, understand what the message is about.
Jacobs [29] introduces adaptive grid-based document layout that divides the screen into grid cells. This composes a page of a document by selecting adaptive templates that include the style, layout elements, and constraint-based relationships among them. Then, the contents and several alternative images are adaptively selected and dynamically formatted based on the page orientation. Although the layout considers individual lines, words, and hyphenation units as atomic units, there is a limitation in using the entire block of text as comprising atomic units.
3.2
The visual to cognitive mapping
3.2.1 Psychology 3.2.1.1 Gestalt Theory
Gestalt Laws In the early 20th century, the Gestalt school of psychology developed Gestalt theory, which attempted to understand perception in terms of organizational laws or principles [37, 34]. The German word, Gestalt means a structure, configuration, or pattern of physical, biological, or psychological phenomena that is integrated so as to constitute a functional unit with properties not derivable by summation of its parts [4]. Some Gestalt laws are illustrated in Figure 3.1.
They include law of proximity (Figure 3.1 (b)) — nearby elements are perceived together; law of similarity (Figure 3.1 (c)) — similar elements tend to be grouped together; law of closure (Figure 3.1 (d)) — closed contour tends to be grouped; law of continuity (Figure 3.1 (g)) — well-aligned contours are perceived to be grouped; law of common fate (Figure 3.1 (h)) — elements moving together tend to be grouped; and law of symmetry — symmetrically arranged pairs of elements are perceived as a group. Additional studies added connectedness (Figure 3.1 (e)) [47] and common region (Figure 3.1 (f)) [45] as grouping principles.
(a) (b)
(c) (d)
(e) (f)
(g) (h)
Figure 3.1: Examples of gestalt principles. (a) No grouping. Grouping by (b) proximity, (c) color similarity, (d) closure, (e) connectedness, (f) common region, (g) good continuation, and (h) common fate.
shows an example of grouping by similarity in hue which divides the figure into two groups of black squares and two groups of grayish squares. Figure 3.1 (d) depicts grouping by closure which brackets surrounding two squares form four groups. Figure 3.1 (e) shows four groups of connected squares. Figure 3.1 (f) shows four groups of squares which each group has a common region by a boundary and grayish background. With continuity, we perceive two continual lines in Figure 3.1 (g), not four discontinued line elements. With common fate, Figure 3.1 (h) shows two groups of upward squares and two groups of downward squares towards the same direction.
Precedence of Gestalt Laws Previous researchers performed numerous experiments that provided clues as to which Gestalt laws viewers perceive before others. This essentially describes the central focus of our work, that is, how to combine grouping principles to create a visual structure.
curve grouped), and closure (segments of the same container are grouped). Relatively recently, Palmer and Rock [47, 45] introduced connectivity (linked items are grouped) and common region (items in the same container are grouped).
In the natural world, the grouping cues described by Gestalt laws exist in abundance, and often conflict, with different cues suggesting competing organizations of our environment. How do people combine multiple cues, and resolve those conflicts? We organize the research addressing these questions into three sets: work studying grouping cue combinations including proximity, connectedness, and common region.
Research shows that proximity often dominates similarity [12, 46]. Yet domination depends on cue strength, with proximity weakening as inter-item distance increases, and similarity weak-ening with reduced viewing time. Kubovy and van den Berg [35, 60] review a great deal of the literature on the simultaneous presence of these two cues and suggest that in fact they operate additively: they reinforce or interfere with one another in direct proportion to their strength.
Han’s work [24] showed that reinforcing similarity cues with agreeing connectedness cues made groups much easier to perceive, while combining proximity with connectedness had little effect. Palmer and Beck [46] found that conflicting similarity and connectedness cues made groups harder to perceive. Interestingly however, when they reinforced similarity with connect-edness, groups became slightly more difficult to perceive. To explain this, Palmer and Beck introduced the notion of intrinsic grouping cues, which depend on properties of the grouped items themselves and include similarity (shape and color) and proximity (position); as well as
extrinsic cues, which depend on other items and include connectedness (linking contours) and common region (surrounding contours). Reliance on extrinsic grouping cues therefore implied additional items and more complex visuals, making groups a bit more difficult to perceive.
Palmer and Beck [46] also found that conflicting similarity and common region cues made groups much more difficult to perceive, while reinforcing cues made groups slightly more difficult to perceive. Recently, Luna and Montoro investigated combinations of common region with proximity or similarity [41]. Conflicting cues made grouping more difficult, while reinforcing cues made grouping easier. Their results supported an additive model for combinations with extrinsic grouping cues, much like the model for intrinsic cues proposed by Kubovy and van den Berg [35].
First, some researchers support the notion that grouping by proximity dominates similarity [12, 24]. Responses are faster to a task involving global structures formed by proximity rather than similarity of shape and luminance. Others found that proximity precedes similarity and global similarity operates before local similarity of geometric features (e.g., viewers perceive a global diamond shape, not squares, when small squares consist of a diamond) [47]. Next, researchers conducted experimental studies to determine whether grouping by connectedness is perceived prior to grouping by proximity and similarity [47, 24, 23]. Palmer and Rock [47] proposed that connectedness operates prior to grouping principles such as proximity and similarity. For example, people tend to group two rectangles that are connected with a line than two rectangles closer to each other (Figure 3.2).
(a)
(b)
(c)
(d)
Figure 3.2: Connectedness operates prior to (a) proximity, (b) similar color, (c) similar shape, and (d) similar size.
However, Han [24] partially refuted this implication of connectedness always winning prox-imity. Grouping by connectedness performed better than weak proximity but when elements were grouped by strong proximity, it showed equally fast reaction times as grouping by connect-edness. Nevertheless, the advantage of grouping by connectedness revealed when the number of elements in the group increased. Han asked the participants to find a specific letter or to discriminate horizontal or vertical alignments, and found that response times were faster with grouping by connectedness than by weak proximity and similarity of shape, but not with strong proximity.
implications of connectedness by matching tasks and visual search tasks with various numbers and sizes of elements. The study found that global configuration reveals when there are smaller and many local elements in the pattern. This is why we perceive a circle when we see a circular configuration consisting of square-shaped elements.
While there are some principles that have obvious precedence over others, there are other grouping principles that compete against each other. Because of this ambiguity, the elements may be grouped differently depending on the viewer. For example, Figure 3.3 (b) and Figure 3.3 (c) show how people can draw different hierarchy from given elements (Figure 3.3 (a)) according to different grouping orders. The features of the elements that can be observed are shape and color differences. Figure 3.3 (b) shows an hierarchy perceived by grouping with shape similarity. Since there are three rectangles and two circles (Figure 3.3 (a)), three rectangles are grouped at the left part and two circles are grouped at the right part of the tree. Then, among the three rectangles at left, two rectangles depicted in red can be grouped again. On the other hand, Figure 3.3 (c) shows a hierarchy perceived by grouping with color similarity. Since there are two rectangles and one circle in red and one rectangle and one circle in blue (Figure 3.3 (a)), elements in red are located at the left part and elements in blue are located at the right part of the tree. Then, among the red elements at left, two red elements shaped in rectangle are grouped again.
(a) Given elements (b) Hierarchy by shape similarity (c) Hierarchy by color similarity
Figure 3.3: Different hierarchies formed by shape and color similarity.
and set two hypotheses that observers assess the cue more reliable when it is less ambiguous and when it is correlated with other cues in the environment. According to experiments in [33], global shape is more robust to noise or blur than the local shape, thus, people may consider it as more reliable. When one cue is weaker than the other, then the stronger cue wins when multiple cues exist simultaneously.
(a)
(b)
(c)
Figure 3.4: Is there competition among proximity, common region, and connectedness? (a) Grouping with common region and connectedness. (b) Grouping closer elements with common region and further elements with connectedness. (c) Grouping closer elements with connected-ness and further elements with common region.
According to the researchers’ finding, viewers were still able to read the sentence despite the misplacement. On the other hand, an incremental grouping task to find letters with same colored dots requires more time since people need time to follow the letter strokes individually.
3.2.1.2 Measures of grouping
Grouping methods Numerous grouping analyses have been conducted in anthropological studies. A study on folkbiology of fish allowed experts from different cultures to classify fish species [43]. Initially, the researchers prepared 44 species of fish on name cards. After sorting out the names that were not recognized by the participants, the researchers asked the participants to group the fish that live together and share a common habitat. They also asked for positive or negative relations (e.g., help or eat) between different kinds of fish using 19 categories of relationships. The participants were asked to form hierarchy of fish by repeatedly grouping and dividing existing groups. They used principal component analysis and multi-dimensional scaling to assess consensus across the experts. This research on the classification of fish demonstrated a cultural difference, whereby Native American fish experts sort more ecologically by habitat, while majority-culture fish experts sort more by the characteristics of the fish.
Consensus analysis addresses anthropologists’ investigation of an unknown culture and de-pends on informants’ responses [52]. It derives a formal mathematical model to analyze the informants’ consensus on questionnaire data, which provides individual competencies and an estimate of the correct answer to each question asked to the informants.
Another study on texture classification has helped to identify how people perceive textu-ral features in three dimensions [50]. The researchers used 12 characteristics (e.g., contrast, repetitive, random, fine, etc.) to classify the group of 56 textures on 9-point Likert scales. Then, the participants sorted the items into similar groups and described why the items in each group were similar. The researchers applied hierarchical cluster analysis and non-parametric multi-dimensional scaling to the similarity matrix produced by the users’ grouping selections of textures.
Another study of visual search on features and objects compared search times [56]. This study investigates the relationship between grouping and pre-attentive features such as color and shape. Pre-attentive features appear despite the grouping, however, if the task is to search conjunctive targets with more than two perceptual features, attention is required. Thus, search-ing conjunctive targets takes serial time as the number of group grows. The search becomes harder as each of the conjunctive features is located between two groups. The study is related with grouping by similarity in color and shape.
A study that developed a numerical model to evaluate graphical user interface screen mea-sured participants’ search time in looking for a given control (e.g., buttons, check boxes, radio buttons, etc.) to perform an instructed action [48]. The model combines screen factors such as element size, local density, alignment, and grouping, and produces a complexity score for the screen given. The researchers found that poor alignment and poor local density negatively af-fected time performance. They concluded that screens with grouping indications and alignment resulted in shorter search times, emphasizing the importance of grouping in visual layouts.
Ratings Most of the studies that gathered data on response time also gathered data on preference ratings [46, 48]. Palmer and Beck [46] maintained to use ratings in most of their measures of grouping. The researchers asked the participants to rate the strength or degree of the grouping between the elements of the target pair. Parush et al [48] not only measured the search time, but also let the participants rate their pair-wise preferences on a 7-point scale given the design screens. Among the design factors, they found that alignment and grouping influenced subjective preferences, while density had a lower preference rate. Eventually, the weights obtained by the users’ preferences were applied to recalculate the complexity score from their evaluation model.
3.2.2 Education
3.2.2.1 Multimedia Learning
(a) (b)
Figure 3.5: (a) Integrated presentation and (b) separated presentation. Excerpt from Mayer 2001 [42].
the content and makes use of it to solve other problems. There can be a redesign question, troubleshooting question, prediction question, and conceptual questions. The students were given a paper to write down the answers to the questions. Again, the score is computed by percentage of the number of acceptable answers compared to the answer key divided to the total. In both tests, there were two people who scored the answer by consensus for objectivity.
3.2.2.2 Visual Difficulties
Although most of us believe that success in learning comes from an easier learning process, studies claim that this is not always the case. A study on cognitive engagement found that making the learning material harder and enhancing engagement can improve long-term memory and retention [19]. They gave participants certain amount of time to learn about the features of aliens with easy- and hard-to-read fonts. Then, after performing unrelated tasks, the partic-ipants were asked to memorize randomly sampled features. The particpartic-ipants who learned from hard-to-read fonts recalled 14% more information than the other group.
Another research claims the advantage of visual difficulties [28]. They summarize research in psychology and education that supports benefit of difficulty in visuals along with active process-ing, which they condense as engagement. The researchers refer to non-efficient visual elements, irrelevant information and distracting visual elements, and extraneous elements (e.g., chartjunk) as examples of visual difficulties. According to their study, the features that we avoided before — such as depth cues, angle, shadow, hard-to-read font, and pie charts — increase cognitive load, thus, can lead to improved comprehension. They also comment on complex visual or-ganization to be preferable under some conditions which increases user engagement with the content.
Chapter 4
Applied experience with visual
grouping
4.1
Depicting layered graphs
A graph is a very complex grouping structure that loses understandability as the graph com-plexity grows. We introduce Quilts (Figure 4.1), a matrix-based depiction for layered graphs (like trees or DAGs) that communicates complex graphs more clearly. A significant challenge in graph depiction is representation of links, which can grow quadratically with graph nodes. Matrix graph depictions use a spatial coding of links to manage this complexity, but in layered graphs, links primarily connect nodes in one layer to the succeeding layer, leaving matrix de-pictions quite sparse. Our insight is to use matrices only to depict links between succeeding layers, making quilts much more compact than matrices.
4.1.1 Depicting skip links
We then require a technique for depicting links that do not simply connect to the next graph layer, or skip links. We created and evaluated three alternative designs forQuiltswith symbolic and color information. For example, Figure 4.2 shows skip links depicted in three conditions of symbolic and color: color, mixed (color and text), and text. Color depiction (Figure 4.2 (a)) distinguishes each layer with different chromaticity and each node within a layer with a different brightness. The mixed depiction (Figure 4.2 (b)) assigns a color to each layer, and a number to each node within a layer. Finally, text depiction (Figure 4.2 (c)) assigns a letter to each layer, and a number to each node within a layer.
The following evaluation section describes the details about the procedure and results.
4.1.1.1 Evaluation
Path finding is a high-level activity that involves basic tasks such as node finding, link finding, finding the most connected (highest degree) node, and finding neighbors. From the source node on the top layer to the destination node on bottom layer, the viewer finds a path by connecting the elements between the two nodes through the grouping of neighbor nodes and links.
The goal of our evaluation was to discover which of our three alternative skip link designs communicated grouping most clearly.
Methods We used a five-factor (3 depictions × 3 nodes × 2 links × 2 skips × 3 layers) within-subjects design. Depiction of skip links had three levels: color, text, and mixed. The number of nodes also had three levels: 50, 100, and 200. The density of proper links (links connecting directly to the next layer — i.e. not skip links) varied between 25% and 50% of all proper links possible. For example, given 400 cells in a Quilt’s matrices, a density of 25% would result in 100 proper links, and 50% in 200 proper links. The density of skips varied between 25% and 50% of the number of proper links (were skip density much higher, the graph would not be a good candidate for layering). Finally, the number of layers varied across 5, 10, and 15. 18 college students (11 male, 7 female, aged from 19 to 38) participated in the experiment. All had normal or corrected-normal vision and passed a color-blindness test. With 36 different graph treatments (3×2×2×3) and 3 different depictions, participants worked with 108 graphs per experimental session. We asked viewers to find a path between the source and destination nodes. The experiment was performed on a Dell workstation with NVIDIA GeForce 7950 GX2, an Intel Core Duo CPU 2.4 GHz processor, 4GB of RAM, Windows 7 OS, and a 1920 ×1200 pixel Dell 24 inch monitor. The graphs were displayed in a full screen mode. Participants sat on an office chair in any fashion they found comfortable.
(a)
(b)
(c)
above in generate and test stages. In the generate stage, a Python script given an experimentally specified number of nodes, layers, links and skips generated a graph mapping group of sequential nodes to a specific layer number, calculated needed number of proper and skip links, and created links that connected two nodes. Note that skip links could connect to nodes on the same layer, or a higher or lower numbered layer, and so sometimes moved “upward” in the three depictions. In the second test stage, a Python script validated four experimental constraints. First, the graph had to contain at least one “good” path between a randomly chosen source node in the first layer, and a randomly chosen destination node in the last layer. Second, since we were evaluating skip link depictions, any “good” path from source to destination had to contain at least one skip link, and therefore could not contain more links than the number of layers minus two. Third, to enforce a minimum difficulty, any “good” path had to contain at least three links. Fourth, a Quilts depiction had to fit on our display (we rejected only six graphs this way). If a graph did not meet the constraints, it regenerated and tested another graph with the same experimental variables until it fulfilled the requirements.
We wrote custom software to displayQuilts to support and record path-finding interaction. Participants traced a path on the graph by clicking on nodes or links with the mouse, and a valid link was highlighted in red. We automatically detected successful completion of a path by verifying that it connected the source and destination nodes, and met the length constraints of a “good” path. If a path was not completed within four minutes, the path-finding task was halted. In either case, we displayed the time used. The next graph appeared when the participant pressed a key, offering participants a break after every displayed graph.
Brightness in colored Quilts varied per layer from maximum to a non-black floor and did not vary in mixed Quilts. Initially, we varied chromaticity in colored Quilts algorithmically by distributing samples evenly in RGB color space. However in piloting, these results proved unsatisfactory, likely due to perceptual nonlinearities. Rather than implementing a complex color mapping optimization algorithm, we defined a color mapping manually, and tuned it in pilot studies to maximize visual differences between 5, 10, and 15 layers. The final result defined a different chromaticity for each layer.
To avoid confounding our experimental variables with learning and fatigue, we varied the order of our experimental treatments. We used complete counterbalancing of depiction. Within each depiction, we sampled the orders of the remaining treatments with a Latin Square. We obtained informed consent from the participants, and gave them written instructions explaining
As dependent measures, we recorded time to trace the path, and whether or not the path was indeed found (“accuracy”).
We assumed that the color-only depiction would be least legible in both time and accuracy. This would be because color matching becomes harder as the number of nodes increase. We also expected that the mixed depiction would support the path finding task best, since color enables individuals to find layers more quickly than textual matching alone (as is required by the text-only depiction), and layer to layer color matching is much easier than skip link to skip link. As to effects not directly related to skip link depictions, we predicted that more nodes would make following paths harder, since this would increase the graph complexity and search space. With more links, it would make finding paths easier because the number of paths would increase.
Results
We analyzed our results with a five-factor repeated-measures analysis of variance (ANOVA). We detail significant effects on the time measure in Table 4.1, and on the accuracy measure in Table 4.2. All the pairwise comparisons were evaluated with contrasts.
Depictionshad significant effect on the time measure and accuracy, with the means depicted in Figure 4.3. Pairwise comparisons showed that participants found paths more slowly with the color depiction than with the mixed and text depictions. Depiction also had a significant effect on accuracy, with means of 93%, 98% and 97% for the color, mixed and text depictions respectively. Pairwise comparisons showed that participants were less likely to find paths with the color depiction than with the mixed and text depictions.
All of the remaining variables also had significant main effects. Nodes had a significant
Table 4.1: Significant main effects and significant two-way interactions on path finding time in the skip link study.
independent variable ANOVA of time
F p
main effect
depictions F(2,34)=25.492 p<.00005
nodes F(2,34)=67.118 p<.00005
links F(1,17)=83.238 p<.00005
skips F(1,17)=6.461 p<.05
layers F(2,34)=25.586 p<.00005
2-way interaction
(a)
(b)
(c)
(a)
(b)
effect on the time measure, with path finding time increasing as nodes increased, also shown in Figure 4.3. Pairwise comparisons showed that each increase in the number of nodes made path-finding slower. Links had a significant effect on time, with means of 60.83 s and 44.65 s for link densities of 25% and 50% respectively. Links also had a significant effect on accuracy, with means of 95% and 97% for the 25% and 50% link densities. Participants were faster and more accurate when proper links were denser. Skips had a significant effect on time, with means of 50.7 s and 54.77 s for skip link densities of 25% and 50% respectively. Participants found paths faster when there were fewer skip links. Finally, layers had a significant effect on time, with path finding time decreasing as the number of layers increased, as shown in Figure 4.3. Pairwise comparisons showed that participants found paths more quickly with each increase in the number of layers. Layers also had a significant effect on accuracy, with means of 94%, 97% and 98% for 5, 10, and 15 layers. Pairwise comparisons showed that participants found paths more reliably with 10 and 15 layer graphs than with 5 layer graphs.
The interaction between depictions and nodes was significant by both the time (Table 4.1) and accuracy (Table 4.2) measures. We show the two-way time means in Figure 4.4 (a). As the number of nodes increased, participants using the color skip link depiction took longer to find paths, and were less successful at finding paths, than those using the mixed and text depictions. Moreover, two-way comparisons show that at 50 and 100 nodes, performance with the mixed depiction was faster than text depiction, but at 200 nodes, performance with the two depictions did not differ. The interaction between depiction and layers was also significant by the time (Table 4.1) and accuracy (Table 4.2) measures. We show the two-way time means in Figure 4.4 (b). As the number of layers decreased, participants using the color skip link depiction took longer to find paths, and were less successful in finding paths, than those using the mixed and text depictions. In other words, there was drastic improvement in time and accuracy with color depiction as the number of layers increased.
Table 4.2: Significant main effects and significant two-way interactions on path finding accuracy in the skip link study.
independent variable ANOVA of accuracy
F p
main effect
depictions F(2,34)=12.218 p<.0005
links F(1,17) =14.825 p=.001
layers F(2,34)=17.335 p<.005
2-way interaction
Discussion The results of our first experiment clearly show that Quilts using the color-only skip link depiction do not communicate the structure of large graphs well. Moreover, pairwise comparisons show thatQuilts using the mixed depiction have a slight advantage over the text-only depiction. It may be that color supports quicker identification of layers than text, since it accesses a lower perceptual level than text.
The main effects of most of our other variables are easily understood. Increasing nodes makes the path-finding task more difficult and complex. Increasing links eases the path-finding task by increasing the number of paths. It is a bit surprising that more skip links increased the time to find paths, until one recalls that in our design, use of a skip link was required, and having more skip links generally made a useful skip link harder to find. Most intriguing, however, is that more layers dramatically improves path finding time. This is likely due to a number of factors. First, with our constraints on path length for the first experiment, a 5-layer graph could only have “good” paths of length three: shorter and longer paths were rejected. With 10- and 15-layer graphs, a larger range of path lengths were accepted. In addition, as the number of layers decreased, the average number of nodes in each layer increased, increasing the difficulty of color matching. And more generally, more layers make skip links more helpful: they can skip closer to the path’s end.
4.1.2 Comparing Quilts to other graph depictions
With the knowledge that Quilts using mixed skip link depictions are the easiest to read, we wanted to compare them to two better known graph depictions: node-link diagrams and cen-tered matrices, illustrated in Figure 4.5. We chose to use cencen-tered (with nodes at the diagonal rather than the row and column headings) rather than uncentered matrices because we ex-pected that they would provide better support for path finding. We anticipated that Quilts
would support faster and more accurate path finding than these alternatives.
Methods Our second experiment has many similarities to our first. Below, we note only the differences between the second and first experiments. We again used a five-factor (3depictions ×3nodes ×2 links×2skips ×2layers) within-subjects design. In this experiment, however, the three levels of depictions were: node-link diagram, centered matrix, andQuilts. The density of skips varied between 0% and 25% of the number of proper links. This change allowed us to examine the effect of having skip links at all. Because the overall trend of layers was linear in the skip link study, we eliminated the middle level, leaving only graphs with 5 and 15 layers.
24 college students (17 male, 7 female, aged from 20 to 56) participated in the second experiment. All had normal or corrected-normal vision and passed a color-blindness test.
Figure 4.5: Example of a 50 node, quarter link density, quarter skip link density, and 5 layered graphs. The left most depicts a node-link diagram, middle shows a centered matrix, and right most is Quilts. Red and black boxes and white numbers respectively indicate the source and destination node. Blue links in a node-link diagram indicate backward links. The node-link diagram is relatively much larger in experimental display.
Apple 30 inch monitor. The graphs were displayed in a full screen mode. Participants sat on an office chair in any fashion they found comfortable.
We reused the 25% link density graphs from the skip link study and removed skip links from the 50% link density graphs for non-skip link graphs. We retained the constraints requiring at least one “good” path, and a minimum path length in all good paths of at least three links. Because our experiment was no longer focused on skip links, we allowed the maximum path length to rise to 1.5 times of the number of layers. To fit the node-link diagram and matrix depictions onto our display, we were forced to use the 2560 × 1600 pixel monitor. With 24 different graph treatments (3 × 2 × 2 × 2) and 3 different graph depictions, we showed 72 graphs to the participants.