Summary

Participant performance on the Object Perspective Taking test is comparable with Hegarty and Waller (2004). Participants tended to perform well and as a result the distribution of scores is highly skewed to the left. No speed-accuracy trade was evident in the results. Scores on the Santa-Barbara Sense of Direction Scale (SDS) are normally distributed and a weak negative correlation was found between the two scales.

Significant differences were found in participant route completion time between the paper-plan condition and all other conditions. No significant differences were found between the three conditions that used the PDA based support. This supports the hypothesis that different representations will result in differences in route completion time. SDS scores emerged as a significant covariate, explaining variation in route completion time and reducing error in the model.

Significant differences were also found between conditions for the mental demand and effort sub-scales of the NASA-TLX. The paper plan condition generated less workload on these scales than any of the three PDA based conditions for both scales; pairwise tests confirmed that these differences were significant. A weak trend emerged indicating that the allocentric segment condition elicited less mental demand than either the egocentric segment or plan overview condition. Pairwise comparisons to test this difference did not reach significance.

Participant’s rated their performance as best in the paper plan condition however no significant main effect of presentation method was found. Frustration was rated lowest in the paper plan condition and higher ratings were given in the three PDA based conditions. Significant differences in frustration were found between the

No significant main effect of presentation method was found for temporal demand. Significant differences were found between the levels of physical demand

experienced by participants. Differences in physical demand were found between all conditions.

These results support the hypothesis that different representations will result in differences in participant workload experienced.

Significant correlations between both psychometric tests (OPT and SDS) were found with route completion time. Generally poor performance on the OPT and poor self-perception of navigational ability predicted higher route completion times. These results support the directional hypothesis that lower perceived and measured spatial ability will result in higher route completion times.

Significant correlations between route completion time and two subscales of the NASA-TLX were found. The mental demand and effort sub-scales are positively correlated with completion time suggesting higher completion time is related to higher mental demand. This result supports the hypothesis that there will be relationships between route completion time and workload.

Using the refined version of the experience questionnaire developed, significant differences in experience were found between the paper plan condition and all other conditions. Exploratory analysis revealed significant relationships between the questionnaire and mental demand scales and route completion time.

Results of the behavioural analysis are generally consistent with the empirical data. Results indicate less confusion in the paper plan condition than in other conditions.

5.11 Discussion

Results indicate that participant’s performance was better and their workload lower when using the whole paper map when compared to any of the electronic methods of presentation used.

One explanation for these results is that the spatial information displayed using a paper plan is more closely aligned with participant’s expected image on presentation of a new environment. With the paper map participants can develop expectations in advance of the immediate environment given the overview presented. Using the paper plan, participants may have been able to use their own preferred strategy to encode information on the plan bringing the expected and perceived images close together, enabling efficient navigation.

The three electronic conditions are bound by the fact that display and presentation is more under the control of computer rather than driven by the participants’

themselves. This is a necessary trade-off since the entire plan cannot be displayed on the small screen of the mobile device. Participants must encode small segments of the plan presented, matching them to the local environment. Frequently,

perception of the environment extended beyond the spatial information displayed which may increase the time and cognitive resources required to match the expected image due to the requirement to perceive increased information in the environment. Participants in the plan overview condition frequently performed better than participants in either of the plan segment conditions. Again this finding may be connected with greater control over interacting with the plan overview condition than the plan segment conditions. Despite strong empirical evidence suggesting that egocentric presentation of spatial information is the preferred orientation (for example Aretz and Wickens, 1992), no significant differences were found between these conditions. One explanation for this difference between the results found and the literature that the extent to which participants were able to match the perceived and expected images may depend more on the type of information presented rather than method of presentation itself. The type of information used to test the different methods, floor-plan information, is designed to be used as an entire overview of an

across the building. When presented in this way, the paper plan information may be used in a familiar way by participants unlike the segment views. The plan

information presented in the electronic conditions may be too cluttered or

fragmented to allow encoding and efficient generation of an appropriate expected image which can be matched to the environment as in the model proposed by Passini (1980). When presented in short bursts, under the control of the computer, only the most relevant information should be presented.

Familiarity with paper based maps may also explain the significant difference in navigation experience reported by participants in the paper-plan condition. Shorter completion times, lower workload and positive experience may reflect an improved ability to orient using the paper map. Orientation may be more challenging when presented with parts of a map at specific locations. This observation is supported by the behavioural data where very low levels of confusion are recorded in the paper plan condition. Rotation in this condition is far more strategic; rotation only occurs when required by the route to maintain a track-up plan alignment. This observation is consistent with Aretz and Wickens (1992) who report that egocentric plan

alignments are preferred in navigation. In the other conditions rotation is more diffuse and at times inappropriate. One explanation for this behaviour is that participants are comparing the environment with different orientations of the plan information to find a match rather than rotate the plan or PDA according to the direction travelled and the number of turns made. Only one participant did not rotate the plan at all during the route. In post-experiment interviews this participant

reported expert levels of map use in the military. The highest levels of confusion are seen in the electronic plan and in the allocentric segment conditions. One

unexpected finding is the failure to find significant differences between the

allocentric and egocentric segment conditions. The experiment may not have been sensitive enough to find differences in these conditions, the unfamiliarity of both the

method of presentation and the mobile device may have masked any advantage in the egocentric condition. Increased levels of confusion were found in the allocentric condition compared to the egocentric condition, supporting the egocentric alignment as the preferred plan orientation. Confusion is evident in the egocentric condition but appears to reduce towards the end of the route suggesting that participants become accustomed to this form of presentation.

Interaction problems were observed by the experimenter in the plan overview condition which may account for increased confusion recorded in this condition. Despite giving participants the opportunity to practice the technique, a wide variation in ability was observed which was not evident in the pilot studies. Many participants found the stylus difficult to use in this way. Others found that attending to the method of interaction interfered with the task of navigation. The lack of clear difference between this condition and the other electronic methods of presentation may be explained in terms of the large between-subject variation observed. Measuring usability in future studies could control for this effect.

In this study, information displayed on the PDA was simply a replication of

information originally on the paper plan of the building. Much of the information may not have been necessary and may even have detracted from supporting users’ navigation requirements. This experiment suggests that in order to compete with a traditional approach of using paper based plans or maps, significant input from human factors is needed when designing location-based services viewed on handheld devices. As only limited information can be shown on the screen at any one time, new representations of space are required that draw users’ attention to the relevant parts of the environment. Effective presentation and use of spatial

information would ensure that information can be quickly and accurately transformed into appropriate navigation decisions. In the following experiments the type of

information that is displayed on a plan will be manipulated in order to improve performance when navigating.