Hypotheses

It was hypothesized that

 (H1) without the aid of public and private cues (non-CROSSBOARD condition), the participant would perform better at indexed task retrieval than un-indexed, as they can utilize the board ordering to effectively scan for the indexed item. Items that were at the beginning of the ordering would be found more quickly than those at the end.

164

 (H2) For the aid of public and private cues (CROSSBOARD condition), there would be less difference between the indexed and un-indexed retrieval tasks, as the cues drill down to an item irrespective of whether the indexed information is known or not. It was still expected that the indexed task will be completed in less time, as once at a suitable level of subdivision, the user can leverage the existing board ordering to find the item within a division without needing cues.

 (H3) Using public and private cues (CROSSBOARD) would be much quicker than using a static display when accessing non-indexed information.

 (H4) The highlighted board without auditory private cues (flash display) would not affect the participants‟ time performance, although they might distract them who do not receive the audio cues.

8.5.2 Participants

Eight volunteers (6 male, 2 female) with normal vision took part in this study. They were students or staff from Newcastle University, aged between 20 and 32. The participants have no previous experience in such experiment, and did not know the aim of the experiment. The content and the method of the experiment were agreed by the participants and approved by the supervisor of this research who is in the School of Computing Science at Newcastle University.

8.5.3 Display configuration

The test display was based on an airport departure board, with each item giving the flight number, destination and departure time. Items were indexed by departure time, as is the norm. The display consisted of 240 items, spread across three large screens which are the same size (7X8 feet each, 7X24 feet in total). The first cycle of cues selects the relevant screen. A binary area subdivision was used to partition each screen into 16 cells, and then each cell contains 5 items, highlighted by a row subdivision. This setup is shown in figure 8.7, and the sequence of subdivisions in figure 8.8.

In the first two seconds of a display sequence one audio cue will sound in one of four 500 milliseconds that four different regions of the display flash, this indicates a spatial

165

subdivision of the display in which the user‘s information resides. In the next two seconds a second cue (and coordinated flash) indicates a sub-region of the first subdivision further localizing the information.

The 80 items on each screen are arranged as 20 rows and 4 columns, which can be subdivided in a number of ways (‗a‘, ‗r‘ and ‗c‘ refer to area, row and column respectively):

• two levels of quartering plus row division

(4a) + (4a) + (5r) = 3 levels, 3 cycles, 13 steps (1)

• one level of row/column division

(20r + 4c) = 1 level, 2 cycles, 24 steps (2)

• quartering plus row/column division

(4a) + (10r + 4c) = 2 levels, 3 cycles, 18 steps (3)

• row/column plus row division

(4c + 4r) + (5r) = 2 levels, 3 cycles, 13 steps (4)

As can be seen (1) and (4) are equivalent in terms of number of time step and cycles, but (1) utilizes more levels of subdivision. Style (1) was chosen in order to investigate both area subdivision and row/column subdivision while minimizing the total number of steps.

166 8.5.4 Procedure

The experiment consisted of 30 tasks. Each participant performed 10 retrieval tasks for each display condition, in a random order one by one: 5 indexed retrievals and 5 non- indexed retrievals. Each task has two steps, in the first step, a two-part flight information item (e.g. 10:00 London) was shown on the screen, a participant was asked to write down to remember and retrieve the remaining part of the information item on the display shown in the next step as quickly as he/she could. In the second step, the visual information display was started immediately (the cycle of audio cue started simultaneously in the CROSSBOARD condition) after a participant informed the experimenter that he/she was ready. Once the participant located the target information item (e.g. 10:00 KLM161 London), he/she indicated his/her response by informing experimenter and reading the remaining part of the information item aloud, then a task ended.

The order of presentation of the displays for each subject was randomized. The flight information was also generated randomly, and there were flights to the same destination with different numbers leaving at different times, and also flights leaving at the same time, but to different destinations. A different board of flight times was presented for each task‟s iteration. All users received the same boards in the same order.

Before starting the experiment, each participant was asked to stand at a fixed location which was 5 meters away from the middle screen and informed that he/she was free of orientation during the experiment. Participants were not given an extensive training on using CROSSBOARD, but they were shown one example at the beginning of the experiment.

8.5.5 Results of Experiment 5

The time performance was calculated from being shown a display to finding the required item (time taken to read out the requested information was ignored).

Treating each task as a one-way repeated measures ANOVA revealed no significant difference across the three conditions for the indexed information retrieval (F(2,14)=2.286; df=2,14; p=0.138). However, the mean value and standard deviation of the Crossmodal Display is the minimum among the three conditions. The performance for un-indexed information retrieval revealed a significant difference across conditions (F(2,14)=24.659;

167

df=2,14; p<0.001). Again, the mean value and standard deviation of the Crossmodal Display is the minimum.

The descriptive statistics (Table 8.1) reveals that the performance for un-indexed tasks in CROSSBOARD condition is 448% better than in plain display condition ((Pu- Cu)/Cu = 448%); 417% better than in flash display condition ((Fu-Cu)/Cu = 417%).

The performance for indexed tasks in CROSSBOARD condition is 71% better than in plain display condition ((Pi-Ci)/Ci = 71%); 182% better than in flash display condition ((Fi-Ci)/Ci = 182%).

N Minimum Maximum Mean Std. Deviation

Index(1) 8 6.25 67.11 21.48 21.41 Index(2) 8 4.96 19.44 12.56 4.63 Index(3) 8 7.52 120.55 35.48 38.58 Un-indexed(1) 8 27.84 82.97 53.04 19.52 Un-indexed(2) 8 5.04 18.72 9.68 4.91 Un-indexed(3) 8 27.68 69.48 50.05 16.21 Valid N (listwise) 8

Table 8.1 Descriptive statistics of the results of Experiment 5

8.5.6 Discussion of Experiment 5

These results here support our original hypotheses and suggest that the Crossmodal Display is effective at reducing retrieval time for information on a high-density information display. (H3) Especially when the information is arranged in a fashion unordered by the searched value, the performance for un-indexed information retrieval was significantly better using CROSSBOARD than using a traditional high-density information display (corresponding to H3). Besides in the CROSSBOARD condition, participants performed better at indexed task retrieval than un-indexed (H1). This demonstrates that locating information by index is efficient on traditional high-density information display, while in the CROSSBOARD condition, the difference of time performance between the indexed and un-indexed retrieval tasks is relative small (H2). It was interesting that the un-indexed tasks

168

were completed in less average time than indexed tasks. This result demonstrates that retrieving information only using crossmodal cues is more efficient than using the combination of crossmodal cues and time index. (H4) Although the time performance on a highlighting display without cues was comparable to a non-highlighted display, the participants did comment that the flashing of the highlighting was distracting (H4). All users completed all the tasks using CROSSBOARD successfully without an extensive training though they were shown one example at the start of the experiment, which demonstrates that CROSSBOARD is easy to learn and use.

8.6 Conclusions

This chapter has reported our exploration of the design and an initial evaluation of CROSSBOARD, a Crossmodal Display prototype system that combines hierarchical spatial multiplexed visual public cues with coordinated hierarchical temporal multiplexed audio and/or haptic private cues to highlight the locations of information items relating to multiple simultaneous users on a large high-density information display. The cues are associated with regions of the display that are flashed in sequence. Depending on the number of items displayed, these regions may then be divided into sub-regions, which also have crossmodal cue combination associated with them. As the cued regions become smaller, it allows a user to rapidly search the region for the required information. This hierarchical cueing is designed to not affect the use of display in a traditional manner without cues, but allows those users with cues both to narrow down a region of the display that is searchable by some indexed part of the information item (e.g., flight departure time) and also to quickly locate an item that is not searchable by an index, which would otherwise require systemic scanning of the whole display.

The initial user study on CROSSBOARD (Experiment 5) mainly investigates the usability of CROSSBOARD. The results have demonstrated the effectiveness and efficiency of CROSSBOARD as a solution to help to solve the problems identified in Chapter 2 and Section 8.3, in particular, validated the following distinct advantages over the previous solutions we reviewed:

169

 Facilitate information retrieval on high-density information displays;

We have demonstrated that hierarchical crossmodal cues in CROSSBOARD can be utilized to provide greatly enhanced retrieval performance for un-indexed information on dense public information displays without adversely affecting the performance of conventional display usage (for the retrieval of indexed or unindexed information).

 Not use or only use limited sensing or tracking;

This is one of the significant advantages of Crossmodal Displays over previous display-based systems that highly rely on tracking technologies, which is demonstrated by the CROSSBOARD system.

 Large user capacity.

This is another significant advantage of Crossmodal Displays. In theory, a Crossmodal Display such as CROSSFLOW or CROSSBOARD can support a large amount of users as it is only limited by the processing capability of the server handling users' synchronization requests.

Although the following aspects are not validated directly through this evaluation, we have planned the methods to do them in future work.

 Support multiple simultaneous users accessing different information content without resorting to tracking users;

A study of multiple simultaneous users can validate this, and the experimental method has been demonstrated in Chapter 5.

 Preserve privacy and anonymity in physical public spaces;

The research into the user experience of multiple simultaneous users afforded by CROSSBOARD can validate this.

 Impose lower mental workload demands on users than traditional static displays.

The subjective task workload on users could be compared between CROSSBOARD and a static display in a user study so as to validate this.

170

How this chapter provides the answers to the research questions identified in Chapter 2 as well as the contributions to the design and evaluation suggestions for CROSSBOARD and Crossmodal Displays in general are summarized and discussed in Chapter 9.

171