The first prototype

Having incorporated the two learning style into the system design, a prototype, which will examine the planning, requirements and show how the final system will look when completed, was built. The first prototype allows flexibility in reorganising the future work of the project. At this point, as discussed in the previous chapter, eight PhD colleagues were the evaluators, offering their assistance to act as users. They were

encouraged to speak out-loud while carrying out the test, discuss and criticise the problems that they experienced. The second stage involved developing the first prototype to get an initial image of what the structure would be, how it would be used, and to gain an idea of the work that would be required. As outlined above, the system offered two tests, both of which had to be completed. The computer then took the results of these two tests and built a profile for the user. After that, the system offered a lesson according to the user‘s profile.

Initialising Profile

The system creates a profile for the user. Information about the user is stored, including the results of the two learning styles tests presented at the initial login.

Checking the profile

The system loads information from the profile. Some of this information will instruct the system in the creation of the presentation.

Presentation

Based on the information stored in the profile and the content stored in the lesson, the system creates a customised presentation for the user. The presentation would be primarily of the preferred presentation style, or at least the explanation/support would be of the preferred style, based on the results of the initial tests.

4.7.1 Purpose

There were three goals behind developing the prototype. The primary goal was to examine the usability of the web-based system that will be used for the ongoing PhD project. The second goal is to obtain a general idea of the difficulties that might arise in the future for the main experiment. The third one is to gain and explore the opinions of the users. The importance of evaluation is a necessity at this stage of the research. It provides a lot of information that helps in measuring the progress and tests the expectations. It helps to enhance and improve the model. It reflects the true picture of the system from the user‘s point of view.

4.7.2 Evaluation tools

The instrument used for the evaluation study was a web-based questionnaire. Users interacted with the system and at the same time they filled in the survey. The survey questions were answered by utilising a five-point Likert scale, ranging from strongly agree to strongly disagree.

4.7.3 Data

The data was gathered in several ways. Following a registration screen where the demographic data was gathered, there were two learning styles tests to be completed. The results of these tests were stored in the database. The feedback of the tests and the feedback of using the system in general were also stored in the database of the system.

4.7.4 Participants

The target users were students at higher education level. Of the 300 people who were emailed, 157 responded. After reviewing the responses only 70 questionnaires were accepted due to partial or total incompletion. A total of 70 participants engaged and accepted to complete the tests as part of this usability evaluation study. Table 4-2 below shows the demographic information collected from participants through the web-based system. 53 males (75.7%) and 17 females (24%) participated. 49% of the users‘ main work or area of study was computing (49%), compared to 51% who were not studying computing (51%).

Table 4-2:Summary of demographic information of the participants Total % Sex Males 53 76 Females 17 24 Discipline Computer 34 49 Non Computer 36 51

Educational Status High school 3 4.3

Undergraduate 25 36 MSc 25 36 PhD student 12 17 Lecturer 2 2.9 Ages <18 2 2.9 18-24 19 27 25-34 39 56 35-44 6 8.6 54-55 1 1.4 >55 3 4.3 4.7.5 Results

Various aspects were addressed in the evaluation in order to gain as much information as possible to assist with developing and improving the prototype.

4.7.5.1 Learnability

Table 4-3 shows the questions that were asked to test learnability:

Table 4-3: Learnability

The questions included in this section of the questionnaire, were designed to determine the opinions of the participants on the learnability of the website, i.e. if there was

Questions Ag ree Neut ral Disa g ree No answ er

It was easy to learn to use the system and there was enough

information to show how to use the system. 55.7 15.7 2.8 25.7 The information retrieved by the system was effective in helping

me to complete the tasks. 44.2 25.7 2.8 27.1 Whenever I made a mistake using the system, I could recover

enough information to show them how to use the system, if the information retrieved by the system was helpful to the participants completing the tasks, and whether it was easy and quick to recover when a mistake was made.

Table 4-3 shows that a over 55% responses agreed that the website was easy to learn from for new users. Positive feedback focused on many aspects of the learnability of the website such as good presentation, and clear and well set out objectives. Some problems were encountered, although they sometimes appeared to be to do with personal choice, as some users liked the layout of the website, whilst some thought it could be improved. In general, the results support that the website is learnable.

4.7.5.2 Ease of use and usability

This section of the questionnaire was included to obtain the opinions of the participants about ease of use of the website, the organisation and layout, and the navigation. It also allowed respondents to include suggestions for improvements or changes.

Navigation is one of the main elements in the success of a website. Users should be led easily to the required information and back to the homepage. Good navigation allows the user to see where they are located on the site. The layout and the organisation of a website are important features, which should be taken into consideration. A well designed website conveys a greater feeling of organisation for the user. The following questions were asked to test ease of use and usability.

Table 4-4: Ease of use and usability Questions Ag ree Neut ral Disa g ree No answ er

The organisation of information presented by the system was clear and

pleasant to use. 61.4 10 2.8 25.7

It was easy to navigate and to find the information I needed. 51.4 17.1 2.8 27.1 I was able to complete the Learning styles Tests easily. 81.4 17.1 0.0 01.4

A large percentage of responses were positive, as Table 4-4 shows. Users who did not find the site organised included helpful suggestions on technical issues (directions of use and instructional complications), the layout (the positioning of the options on the

page), and presentational issues (making the website more fun and enjoyable by using more colour or different fonts).

4.7.5.3 Functionality

This section of the questionnaire was useful in ascertaining the opinions of the participants about the functionality of the system. Opinions about these operational elements varied greatly. Positive feedback regarding the second lesson focused on it being easier to understand, less time-intensive, and that the content was often more suited to, and therefore preferred by, the user. Negative feedback was rare, but when provided was constructive, and tended to be from those participants whose two lessons had been similar or the same.

Table 4-5 below shows that a large percentage of responses shows that users who responded were able to discover their learning style from the website, and were convinced about the value, for them, of the information produced by the system.

Table 4-5: Functionality Questions Ag ree Neut ral Disa g ree No answ er

The system was able to convince me that the recommendations and

presentation are of value. 38.6 18.6 05.8 37.1 The lesson was presented according to my learning styles (when the

lesson was according to their style) 35.8 20 04.3 40.0 The lesson was presented according to my learning styles (when the

lesson was not according to their style) 32.9 20 11.4 35.7 I was able to discover my learning styles 64.2 27.1 07.1 01.4

Of the total number of participants who answered the questionnaire, a large percentage of responses noticed that an effort was made to differentiate between a lesson given in accordance with a users learning style and a lesson given against a users learning style. An unexpected finding was that users did not reject the lesson that went against their learning styles. This may indicate that the users did not have sufficient knowledge about learning styles and as such were unable to identify when the system did or did not present a lesson according to their learning style. Indeed, 40% of users gave no answer to the question that the lesson was presented according to their learning style even when the lesson was according to their learning style.