Results

In this section we review the major outcomes of our testing.

Issues Related to Computer Literacy

Despite working in pairs, in each case a leader emerged who controlled the computer mouse and keyboard, with the other only offering suggestions. We preferred not to interfere with the dynamics between partners, but this did have the consequence that we could only observe the hands-on computer skills of the leaders.

All users seemed comfortable with basic web browsing skills, following hyperlinks and using the browser back button. One participant demonstrated facility with the mouse middle button scroll wheel.

3. Cycle One – First Website

We observed two difficulties shared across groups. First, none of the users demonstrated touch-typing capabilities. The need to look at keyboard and not screen while typing resulted in errors. For instance, if the cursor was not in a text box before typing, the users could type several words before realising that their typing had no effect. Second, fine positioning of the mouse was not easy. This became an issue when clicking on the central blocks on the home page, where only the text was hyperlinked (see Figure 18), as opposed to the whole block.

The users adapted by being more deliberate in their actions as the sessions continued. These difficulties slowed them down but we believe that other issues caused greater obstacles for the users.

Visibility of List Controls

Participants spent long periods reading without interacting with the site. When a task required that specific information be found, the participants most often read through lists rather than searching.

The first page of each section (a list) allowed access to all the entries in the site, as long as page navigation controls were used. Unfortunately these were hard to spot. In Figure 24 we show a typical list, with the page navigation controls circled in red.

Only the ten most recent entries were visible on the first page. Had we not prompted participants to search, they would have missed a large percentage of the available data. After searching, the page controls were still not used, but search ranking by relevance made navigating beyond the first page less important.

Figure 24 Website list navigation controls were unused in Cycle One testing. Here they are highlighted on a typical list in the site, but users did not easily spot them unassisted.

3. Cycle One – First Website

67 Menu Navigation and Visual Cues

All attempts to complete tasks were made with the help of the scoped sections of the site (see Section 0); the global search box on the home page was ignored. This may have been a conscious decision, or it may have been that positioning and design made it invisible. Figure 25 shows the home page with the search box circled in red.

The scoped sections offer the advantages and disadvantages of a hierarchical data organisation:

fewer distractions from irrelevant data, but also the possibility of missing a useful option if the wrong section is entered. Accordingly it was important that users understand which section of the site they were browsing, and that they be able to navigate back to the home page or into a different section.

Once in a scoped section of the site, the menu bar was the only means of navigating to other sections and the only visual cue regarding the current site section. We highlight the menu bar on a scoped search page in Figure 26. The faint blue background colour on the “study” menu item indicates the current section. Clicking on any other item in the menu navigates to a different section of the site.

Figure 25 The global search controls on the home page were unused in Cycle One testing. Here they are shown circled in red.These could have alleviated some navigation difficulties we observed, had they been used.

3. Cycle One – First Website

Figure 26 The menu navigation was not visible at crucial junctures. Here it is highlighted in red. The menu was intended to provide navigation controls and context to the user but was not considered for important tasks.

The given cues were insufficient. For example, after completing a task in the skills section, we asked one pair to begin the job search task. They did not spot the “jobs” menu button which would lead them into the jobs section, despite having previously used the “home” button to its left. After a long pause the pair discussed the possibility that they might enter the text, “jobs”

into the search box on the skills page, which would not provide useful data. Stronger indications of context and navigation affordances were necessary.

Visibility of Related Information

An important function of the site was its ability to lead users to new information based on searches they made and entries they viewed. The “related” lists of entries on the right hand side of every search and individual entry page (circled in Figure 27) were created for this purpose.

Unfortunately no users spotted the information.

3. Cycle One – First Website

69 Figure 27 Unused related information elements are highlighted on an entry detail page

Data Capture Policies

We had agreed the fields that would be captured for each different type of content, but the existence of free text fields allowed for variation in data capture approach. The tasks we chose were designed to elicit users’ reactions to all available content types, with the intention of informing the Link team’s approach.

We discovered a discrepancy between participants’ interests and the available entrance requirement and fee data for tertiary courses. Instead of displaying this data directly on our site, the Link team had captured links to academic institution websites, where our users would theoretically be able to find the information for themselves.

Unfortunately, pages from a sample external web site proved intimidating. We had to offer explanations about the purpose of the information in front of students and prompt heavily before they would take action. Using the external pages required combining information on our site with information from the external pages. Our users were not adept at this: they struggled, for instance, to use the information on our site about a course to identify which of several fees listed on an external page (see Figure 28) was relevant.

Students Level of Readiness

We hoped to learn whether our participants would be able to apply the content usefully. This question proved difficult to answer. On the one hand, the pair in Lavender Hill decided that a hypothetical friend should not apply for a job based on the dates listed on our site and the

3. Cycle One – First Website

knowledge of when the friend would be writing exams. In so doing they proved their ability to combine their own knowledge and the contents of our site to reach new conclusions. On the other hand, one of our Manenberg groups recommended one job entry over another because “it has more information”, rather than reading the contents of each and thinking about the task.

In another scenario, what we thought was a simple task proved difficult. Investigating university courses for a hypothetical student whose favourite school subject was accounting would, we hoped, result in an immediate search using the word “accounting”. Unfortunately the students hesitated. One group requested more information about the person, while others just looked at the items in the list without performing any action and would not commit to a recommendation.

Figure 28 List of annual fees on a university web page external to our website. Users struggled to use this page to determine course fees of a course on our site.