Tasks Completion Time

The aim of the trial was to compare how participants performed a set of tasks on the Prioritised version and on the Facebook mobile version. The way participants performed the tasks and the path taken could affect the time to complete the tasks. Task completion time could also suggest the ease of use and the ease of navigation of a particular site.

Realizing that participants may still have difficulties with the touch screen phones and that the completion time was not the main factor studied; we did not limit the time to complete each task. From our observations, most participants were determined to complete the tasks.

The time taken and measured in this section may not be accurate due to the loss of connection during the trials or participants’ inexperience with the mobile phone used. Although there were only two occurrences, these factors may introduce threat to validity as discussed in Section 6.10.

As described in Section 6.9, when there was an incomplete task, the time participants ended the task was used as the completion time. This method, however, may introduce issue with the validity of the measured time as the longer or shorter time participants took before giving up will skew the time. To overcome this issue, where there was any incomplete task (participant gave up or unsuccessfully completed the task), we also analysed the completion time for only the successfully completed task (see Section 6.10).

On average, participants completed tasks faster on the Prioritised version than on the Facebook mobile version. Figure 7.10 illustrates the mean task completion times of 13 participants for the Prioritised version and the Facebook mobile version.

105

Figure 7.10 The mean tasks completion time for the Prioritised version and the Facebook mobile version.

Results suggest that the Prioritised version is easier to use and that common or frequent tasks could be completed faster. Results showed that, although the overall layout of the Prioritised version is different than that of the Facebook full site, having the most common items at the top of the page and similar to the Facebook full site helped users find the items faster. In contrast, the more unfamiliar Facebook mobile confused participants and resulted in more time spent locating items.

Completion time for Task 3 (showing photo albums) was the longest for both versions trialled regardless of whether the time participants gave up the task was included or excluded from the result, but probably due to different reasons. On the prioritised version, the links to photo albums were initially removed. On the Facebook mobile, the links were visible but were hard to find due to their location. Although the link ‘more...’ was a new concept for the participants, they were still able to find items via the ‘more...’ link more quickly than on the actual Facebook mobile site.

A paired-samples t-test was conducted to examine whether there were significant differences of mean completion time between the tasks performed on the Prioritised version and the tasks performed on the Facebook mobile version. The t-test results revealed completion time for each task was significantly shorter (p < 0.05) on the Prioritised version than on the Facebook mobile version as shown in Table 7.1.

0 50 100 150 200 250

Task1 Task2 Task3 Task3-

Completed C o m p le ti o n T im e ( se c)

Mean Completion Time for Each Task

Prioritised version Facebook mobile

106

Table 7.1 The paired-samples t-test results

The high standard deviations suggest the data distribution deviates from a normal distribution. The high standard error means there are high variation between participants’ completion time. This is caused by the longer time taken by a few participants to complete or to give up on a task. In order to confirm the result, we also run a non-parametric Wilcoxon test. Results confirmed those of the t-test. On average, each task was completed faster on the Prioritised version than on the Facebook mobile version - tasks 1 (p=.043), task 2 (p= 0.007), and task 3 (p =.007).

As stated in Section 7.2.1, five participants accessed Facebook from their mobile. Figure 7.11 illustrates the mean time taken to complete each task on both the Prioritised version and the Facebook mobile version for those five participants; the task3-completed columns show the mean time for only the successfully completed Task3.

Figure 7.11 The mean task completion time for the five participants who have been accessing Facebook on their mobile. Error bars show the standard error.

These results showed that participants performed faster on the Prioritised version although they have experience with the Facebook mobile version. The results indicate that, it was

0 20 40 60 80 100 120 140 160

Task1 Task2 Task3 Task3-

Completed C o m p le ti o n T im e ( se c)

Mean Completion Time for Five Participants with Facebook Mobile Experience

Prioritised version Facebook mobile

107 helpful that the Prioritised version was similar to the Facebook full site even for users with experience of the mobile site. We found this result particularly encouraging.

As described earlier, in order to avoid the bias of order effect, the order of the versions trialled was alternated. We had half of the participants (group A) trialling the Facebook mobile version first and the other half (group B) the Prioritised version first. The result of each group’s completion time for each task is illustrated in Figure 7.12.

Figure 7.12 The mean time taken to complete each task for each group. A trialled the Facebook mobile first, and B trialled the Prioritised version first.

Results showed that participants completed the tasks more quickly on the Prioritised version than on the Facebook mobile regardless of the order the version was trialled.The mean times for the Prioritised version in Group A (the Prioritised version was trialled second) were less than in Group B (the Prioritised version was trialled first). This was due to a few individual participants who had forgotten how to update status (for task1) and were confused and having problems with tasks (task 2 and 3) as discussed in the previous section. Excluding the outliers gave a similar mean completion time for group A and B.

A few biases were observed on the completion time, for example, for task 3 for group B. The high mean times were due to a small group of individuals who were having problems with the task as described in the previous section. Also, it was caused by the longer time taken by participants who gave up the task for the Facebook mobile version or unsuccessfully completed it on the Prioritised version. Excluding the time taken for the incomplete or unsuccessfully completed task 3 as illustrated by task3-completed showed the difference.

0 50 100 150 200 250 300 A B A B A B A B

Task1 Task2 Task3 Task3-Completed

C o m p lp e ti o n T im e ( se c)

Mean Completion Time for Each Task

Prioritised version Facebook mobile

108 The time taken to complete each task suggests that although participants are new to the Prioritised version, they learnt to use it easily, and performed the tasks more quickly than on the Facebook mobile version.

The participants’ comments indicate that the similarity to the familiar Facebook full site allowed them to apply their knowledge and experience to make sense of the version and tasks.