VulaMobi Usability Testing
Preliminaries
xiii. Participation in this survey is voluntary
xiv. Research participants may withdraw from the research at any time without compromise to their well-‐being
xv. The objective of this testing is to evaluate usability and usefulness of the VulaMobi tool xvi. The tool will not be used to evaluate ones academic ability
xvii. Research participants are required to sign consent forms
xviii. Research participant’s identities will be coded and as such their responses will be unidentifiable / anonymous
xix. Data collected during this research will be treated / handled in a confidential manner xx. Feel free to comment on any aspect of the evaluation in the comments section provided
should you require additional paper please let me know.
xxi. If any instructions are unclear please feel free to ask me.
xxii. Please note any peculiarities or useful functions you encounter during this evaluation.
xxiii. This evaluation is limited to the access of course websites which a student may be enrolled in. Societies and clubs are not included.
VulaMobi Usability Testing
Consent form
Researcher: _________________________
I agree to participate in this experiment.
I agree to my responses being used for education and research.
I understand that my personal information will only be used in aggregate form, so that I will not be personally identifiable.
I understand that any material captured by this experiment is to be treated confidentially and none of it will be released to the public.
I understand that I am under no obligation to participate in this project.
I understand that I have the right to withdraw from this experiment at any stage.
I have read this consent form and the information it contains and had the opportunity to ask questions about them.
Name of Participant:
__________________________
Signature of Participant:
__________________________
Date:
__________________________
Introductory Instruction Form
Thank you for agreeing to participate in this study! This form explains the experimental procedure, so please read it carefully.
The duration of the experiment is approximately 30 minutes. The experiment will present you with a set of tasks to perform using the Vulamobi app and the SAKAI mobile access portal. You will be told which of the two interfaces to start with. At the end of each set of tasks you will be required to answer some questions on the experience you had with each interface. You are encouraged to make any comments relating to the system in between tasks.
Please do not commence a task until told to do so and continue until either you are asked to stop, feel you have completed the task or you feel the need to stop for any reason.
Please answer all questions and feel free to share comments on any aspect of the test.
Pre Test Questions
Please select your choice with an ‘X’ where appropriate (use mark the area of your choice with an ‘X’):
1. What academic program are you registered for (e.g. BSc Computer Science)
2. Which year of study are you in?
3. How often do you use Vula on a desktop computer?
1 2 3 4 5 6 7
Never Daily
4. How often do you access Vula on a mobile device?
1 2 3 4 5 6 7
Never Daily
5. How familiar are you with the android phone environment?
1 2 3 4 5 6 7
Very Unfamiliar Very
Familiar
Participant Tasks
Section A: Vulamobi App Task 1
Please login to the service using the following details:
Username: “ test”
Password: “test”
Please make an indication when you feel you have successfully completed the task.
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 2
Where would you go to view the courses you are enrolled in? When you have identified the option please select it?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 3
Can you please view an announcement from a course?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 4
Can you view an announcement from a different course?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 5
Can you go to a view showing all your enrolled courses?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 6
Check the grades for ‘Cookery101?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 7
Can you logout of the system?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Vulamobi interface usability scale form
1. I think that I would like to use this system frequently.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
2. I found the system unnecessarily complex.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
3. I thought the system was easy to use.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
4. I think that I would need the support of a technical person to be able to use this system.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
5. It is useful?.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
6. I thought there was too much inconsistency in this system.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
7. I would imagine that most people would learn to use this system very quickly.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
8. I felt very confident using the system.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
9. Is it user friendly?
1 2 3 4 5 6 7
Strongly disagree Strongly agree
10. It helps me be more effective?
1 2 3 4 5 6 7
Strongly disagree Strongly agree
11. It makes the things I want to accomplish easier to get done?
1 2 3 4 5 6 7
Strongly disagree Strongly agree
1
Section B: SAKAI MOBILE ACCESS Task 1
Please login to the service using the following details:
Username: “ test”
Password: “test”
Please make an indication when you feel you have successfully completed the task.
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 2
Where would you go to view the courses you are enrolled in? When you have identified the option please select it?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 3
Can you please view an announcement from a course?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 4
Can you view an announcement from a different course?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 5
Can you go to a view showing all your enrolled courses?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 6
Check the grades for ‘Cookery101?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
Task 7
Can you logout of the system?
How difficult do you feel this task was to perform?
1 2 3 4 5 6 7
Very difficult Very easy
SAKAI mobile access interface usability scale form
1. I think that I would like to use this system frequently.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
2. I found the system unnecessarily complex.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
3. I thought the system was easy to use.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
4. I think that I would need the support of a technical person to be able to use this system.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
5. I found the various functions in this system were well integrated.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
6. I thought there was too much inconsistency in this system.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
7. I would imagine that most people would learn to use this system very quickly.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
8. I felt very confident using the system.
1 2 3 4 5 6 7
Strongly disagree Strongly agree
Post Test Interview
1. What are your overall impressions of the VulaMobi app?
1 2 3 4 5 6 7
Very bad Very good
2. If you had to give the app a grade, from A to F, where A was exemplary and F was failing, what grade would you give it, and why?
3.
4. Name three words or characteristics that describe this app?
5. What are the three things you like best about the app?
6. What are the three things you like least about the app?
7. If you could make one significant change to this app, what change would you make?
8. Are there tools or functionality you would like to see added to the Application? Which ones?
9. Would you recommend this app? To a friend?
10. Do you have any other questions or comments about the app or your experiences with it?