RESULTS 114 not be clear to new users and that rewording the option to ‘What service delivery problem

would you like to report?’ would be more meaningful. Four participants (3, 8, 11, 23) said that insufficient feedback was provided after answering a text response, poll.

6.7.5 Task 3: Dirty water

As with previous tasks, participants were first required to log in to the client application. Of the 15 participants who performed Task 3, 86.67% managed to log in on their first attempt; one participant (6.67%.) was successful on their second attempt; while another participant (6.67%.) took three attempts. Two-thirds (66.67%.) of participants completed Task 3, while the remaining third (33.33%) did not.

Finding correct pell Completing poll Charts easy to confusing straightforward understand

Question presented

Strongly agree

■ Agree

■ Neutral

■ Disagree

■ Strongly disagree

Figure 6.11: Summary of Task 3 responses (n = 45)

Participants who completed the task using the feature phone took a mean duration of 64.62s, while those who used the smartphone took slightly less, as reflected in Table 6.7. While the median number of button presses made by participants when viewing the Results screen was less than those made by an expert user, this is likely due to the fact that two of the three participants (18, 25) using the feature phone did not consult the chart visualisations. Instead, they read values from the numerical table appearing directly on the Results screen, hence only requiring a single button press. In addition, a single participant (Participant_1) who used the smartphone consulted the numerical table instead of the chart visualisations. Excluding this participant when calculating the median brings the number to 4 button presses, which corresponds to the minimum required. Figure 6.11 presents a summary of all the task responses.

6.7. RESULTS 115

Task 3 Results screen

Feature phone Smartphone Feature phone Smartphone

Participant mean duration 64.62s 48.57s 14.57s 38.81s

Participant median 5 presses 7 presses 1 press 3.5 presses

Expert user ^{5 presses 3 presses 4 presses}

Table 6.7: Average duration and number of button presses made when performing Task 3 Ql: “Finding the correct poll was confusing” Again, varied responses were received from participants (n = 15). One participant, or 6.67% responded that they ‘Strongly agree’; two par­

ticipants (13.33%) responded ‘Agree’ and ‘Neutral’ respectively; 33.33%; responded ‘Disagree’;

while the remaining 33.33%; chose ‘Strongly disagree’. The resulting usability rating for the state­

ment was 74.67%;. There were a few noteworthy written responses, with Participant_ 14 stating that they “think fit should] specify that I’m reporting water quality”. Conversely, Participant _ 12 wrote: “[there were] few options to select through so fit] wasn’t hard finding the poll”.

Q2: “Completing the correct poll was straightforward” The second statement was answered positively bv all but one participant, resulting in a usability rating of 90.67% (n = 15). Just under half (46.67%) of the participants responded with ‘Strongly agree’, while the same proportion (46.67%;) responded with ‘Agree’. One participant (6.67%;) did not share these sentiments, instead responding with ‘Strongly disagree’. There were no written responses.

Q3: “The charts were easy to understand” Responses to statement 3 were positive, with 53.33%; of participants stating that they ‘Strongly agree’. The remaining 46.67%; responded with ‘Agree’ resulting in a usability rating of 90.67% (n = 15). Written comments from two participants (22, 24) who had used the smartphone handset (with interactive chart visualisa­

tions) when performing the study suggested including the actual value/percentage on each chart visualisation, instead of only making them viewable when ‘tapping’ the chart. Participant_22, for example, wrote: “Show the percentages and number of votes on the pic/bar chart” while Participant_24 stated: “Numbcrfcd] figures in the pie charts would have been better”.

Q4: “Which chart did you find easiest to understand?” The final question aimed to assess which chart type participants best understood. Of the 15 participants who answered the question, 46.67%; stated that they found the pie chart easiest to understand; 20%; stated that they preferred the bar chart; while 33.33%; indicated that they were both equally understandable.

Two noteworthy written comments were provided, the first by Participant_25 indicated that displaying the percentage value on the bar chart’s y-axis would be preferred to the actual number of votes. The second comment, by Participant_ 10 posed a similar suggestion, stating: “the bar

6.7. RESULTS 116

chart is a bit confusing because a user would expect the numbers on the y-axis to be percentages”.

Figure 6.12 depicts the misinterpreted bar chart visualisation. Participant_9 stated that they preferred the bar chart, and that “pie charts confuse some people”. Conversely, Participant_ 12 maintained that the pie chart was “visually more appealing”.

Figure 6.12: Misinterpreted bar chart visualisation

Discussion The usability rating for Task 3 was calculated for each participant, resulting in a mean usability rating of 85.33%. Observational notes made by the researcher highlighted that two participants (2, 10) did not notice the interactivity within the chart visualisations. When asked to determine how many other people were also experiencing brown water, Participant_25 was seen reading values from the numerical table instead of from the charts. Participant _ 14 had to perform the task again as they mistakenly dismissed the R esults screen without noting the required values.

6.7.6 Task 4: Change application language

The fourth task began by asking users to log in to the MobiSAM client application. A vast majority (80%.) of the 15 participants were able to successfully log in during their first attem pt;

two participants (13.33%;) managed on their second attem pt; while one participant (6.67%;) took three attem pts. All participants were able to complete the task, although two needed some guidance.

6.7. RESULTS ¹¹⁷

Finding where to change Update profile screen easy to languge straightforward understand

As shown in Table 6.8, Task 4 took participants on average 59.15s to complete when using the feature phone, and 26.26s using the smartphone. A median of three button presses were made by participants, irrespective of the handset used during the evaluation.

Feature phone Smartphone Participant mean duration 59.15s 26.26s

Participant median 3 presses

Expert user 3 presses

Table 6.8: Average duration and number of button presses made by participants performing Task 4

Ql: “Finding where to change the MobiSAM app’s language was straightforward”

The first statement presented to participants (n = 15) scored highly with 80% of participants stating that they ‘Strongly agree’; one participant (6.67%;) stating ‘Neutral’; while two parti­

cipants (13.33%) answered with ‘Disagree’. The statement received a usability rating of 89.33%.

Three participants (14, 15, 30) provided written comments, all of which highlighted the ease at which they were able to perform the task. Participant_30 wrote: “fit] was very easy [for me] to navigate to profile and change the language” while Participant_14 felt that previous experience helped, stating: “It was easy for me because I know about profiles”.

Q2: “The MobiSAM app’s Update profile screen was easy to understand” The next statement, again, scored favourably, achieving a usability rating of 93.33% (n = 15). Nearly three-quarters (73.33%;) of participants stated that they ‘Strongly agree’; 20%; stated ‘Agree’;

while one participant (6.67%;) chose ‘Neutral’. One written comment was provided by Parti­

cipant _ 15 who stated that the UI was “very straight forward”.

6.7. RESULTS 118