Method of Data Collection

After the development stage of the system, the opinion of real users was explored to make sure that the system was achieving its desired purpose, i.e. whether or not users recognised that the system was presenting the lesson according to two of their learning styles. Analyses were carried out in order to determine whether the users were able to differentiate between lessons that were presented to them according to a personalised learning style and those which were not.

5.3.1 Participants

Initially, the experiment was intended to be conducted at Heriot Watt University. Unfortunately the timing was not suitable for most of the students as it was set during the last two weeks of the term and all of the students were busy with project deadlines. Since students would be leaving after the end of term, another location was chosen. Five Saudi Arabian higher education schools were contacted, of which two responded: The College of Technology in AlBaha, and The Department of Education in Jeddah, with the added cooperation of its computer school.

The initial number of students from one of the schools was 102, but only 91 fully completed the survey. More participants carried out the experiment from another city in Saudi Arabia bringing the total number of participants to 129.

As indicated in the Table 5-2 below, male respondents make up 67.4 percent of the total dataset, with 97.7 percent indicating that computer science was their chosen field of study. The majority of participants were undergraduate students, with the remainder of users making up only 14.7 percent of the total. From these results it becomes clear that the majority of respondents were male, computer science undergraduate students. This has both positive and negative implications. As a whole, analysing such a group means that the inferences can be kept narrowly defined. Given the small size of the dataset, this makes it easier to infer results. On the negative side, the system would benefit from an analysis of a broader range of individuals as a way of ensuring usability for the population at large. This may be something to consider for further research and investigation.

Table 5-2: Gender Frequencies

Gender Frequency Percent

Male 87 67.4

Female 42 32.6

Total 129 100.0

Table 5-3: Discipline Frequencies

Discipline _{Frequency Percent}

Computer field 126 97.7

Non computer field 3 2.3

Table 5-4: Status Frequencies Frequency Percent Undergraduate 110 85.3 Master 4 3.1 Diploma 9 7.0 Lecturer 6 4.7 Total 129 100.0

Table 5-5: Age Frequencies

Age Frequency Percent

18-24 years 110 85.3

25-34 years 13 10.1

35-44 years 4 3.1

44-55 years 2 1.6

Total 129 100.0

From Table 5-4 and Table 5-5, it can be seen that the status variable skewed to the undergraduate group, as is the age variable skewed toward the 18-24 age group. A bivariate correlation of the age and status variables indicated that the correlation is statistically significant with a Pearson correlation coefficient of r=0.613, which is significant at p<0.01. This positive correlation indicates that as the number of participants in age group 1 – 4 decreases (from 18-24 years in group 1 to 44-55 years in group 4), the number of participants in each status group from 1 - 4 also decreases (from undergraduates in group 1 to lecturers in group 4). As a result of the correlation analysis, status will be used exclusively, and age will be eliminated as the two variables appear to be measuring the same thing. This kind of correlation is not surprising, however, there would be no advantage in analysing them separately; the results would be the same.

5.3.1.1 Division of participants

The overall aim of this experiment is to test the hypotheses stated in the introduction of this study: “There will be a difference between users’ recognition of a lesson that is presented according to two classifications of learning styles, one and none of the users’ learning styles”. Participants were therefore, randomly assigned to one of four test groups (see Table 5-6 below):

Table 5-6: The groups that participants were randomly assigned to by the system

A VARK

B Honey & Mumford

C VARK and Honey & Mumford

D No tailoring

Group A was learning a lesson presented according to only one of the participant's learning styles (VARK); Group B was learning a lesson presented according to only one of the participant's learning styles (Honey & Mumford); Group C was learning a lesson which was presented according to both of the participant's learning styles (VARK and Honey & Mumford); Group D was learning a lesson presented with no personal customisation. The system considered the users as a queue of people. The first one would be in group A, the second would be placed in group B, third - C, fourth - D. This cycle would be repeated until the last user. Participants did not know which of the groups they were assigned to. The dependent variable, group, is central to this analysis. This variable indicates whether or not participants were given the system lesson in consideration of their learning styles and as a result of comparing the answers from each group, the research question will be discussed.

5.3.2 Procedure

The users were prepared for the experiment with a short fifteen-minute presentation on the importance of learning styles and the rest of the lecture time provided was used for utilisation of the system. During the presentation, the following ideas were discussed: overview of what learning styles are, why they are significant, and where more information on learning styles is available. After that the system was introduced and it was discussed how it could help them discover their own individual learning styles.

Two models of learning styles were considered. One was VARK, based on biological or physical learning considerations, and the other was Honey & Mumford, based on psychological learning considerations. The participants were presented with the two learning styles tests in order to determine their learning style, The system then used this information to generate a lesson based on the results of the learning styles test.

During the experiment, some of the students were having problems with some difficult English words in the test and asked for assistance. The lecturer was eager to be of assistance; he encouraged and helped the students. Upon completion of the task, the participants completed a survey and expressed their opinion of the system and its functionality.

5.3.3 Differences in methodology

Due to different experimenters undertaking the studies, there were differences between the two cities in terms of how the studies were conducted:

In the first city (Albaha) the researcher himself gave the lecture and introduced the experiments, while in the second city (Jeddah), a local member of staff was in charge of the experiments. The researcher provided him with the details of the first experiment, and the questions he was asked by students during that session. Another difference was the time allocated to the session. In the first school, the time given was a full lecture; 50 minutes. While in the second school the given time was about twenty five minutes; which the person who was in charge believed was enough.

Despite these differences, the order in which the lecture, test and lesson were given was the same for both cities: students were introduced to learning styles and to the research experiment, they then used the system, and finally completed the session with the survey. The differences were minor, but to err on the side of caution, statistics were conducted to check if these differences caused any significant differences in the results related to the research questions and hypotheses. This was to rule out the possibility of the data having to be analysed separately. The results of the tests will be shown later in this chapter.

The following sections will present the survey questions and detail the variables and methods used for analysis.