Cognitive Load Rating Scale (CLRS)

3.5.2.1 Validity of the Cognitive Load Rating Scale (CLRS).

The Cognitive Load Rating Scale (CLRS) was used with permission from Fred Paas of the Erasmus University, Rotterdam, Netherlands. In using the CLRS, Paas, Tuovinen, Tabbers, and Van Gerven (2003) and Pass and Van Merriënboer (1994) (see Appendix O) compared the instrument with the physiological measures like the one where heart beats or the observation of the pupil dilation were used in the different researches. They found that the CLRS was quite stable, meaning how a person perceived difficulty was more stable and valid to just a mere observation of the pupil dilation and the other measures. It was thus found to be valid. The CLRS was also easier and cheaper to use in this particular research 77

situation. One was able to use it with the large group and was more appropriate than other tests, for example, the testing of the heart beat for every question answered or done.

When this instrument was administered in the schools, it was explained to the learners what the questions meant. It was emphasised to the learners that the understanding was referring to how they perceived what had been taught. The questions had to be read again to the learners in the research situation, so that it was clear what the Cognitive Load (see also Appendix P) was referring to.

3.5.2.2 Reliability of the Cognitive Load Rating Scale (CLRS).

The reliability of the CLRS was determined wherein the internal consistency of the instrument was found to be 0.90 (Paas & Van Merriënboer, 1994). The instrument has been used for over 20 studies and was found to be more consistent over a big range than use of pupil dilation and heartbeat (Paas, Tuovinen, Tabbers, & Van Gerven, 2003).

For the local situation extra explanatory phrases were added to the instrument to make it more understandable to the learners to make sure that all ambiguity is clarified.

3.5.3 Split timer.

3.5.3.1 Validity of the split timer.

The timer would be started when the learners started writing the test. It was important that all the learners who were in the class would be started off at the same time. As soon as the learner indicated that he had finished then the split timer would be stopped for that particular learner. The order in which he had finished would be written on the script. It was important that the research assistants were given the same training and the same

understanding of how to take readings and use the split timer. This was done to ensure that the times written would be the same. There were practice times to check if it was done in the same way to make sure there was consistency.

Sources of not being valid would be where the learners arrive after the test had started. In order not to affect the actual times, the learner would have to have his own starting time so that the actual finishing time would be taken for the individual participant. This happened only in School A when three learners arrived after the test had started for the second post-test where each of the learners’ times were taken separately.

3.5.3.2 Reliability of the split timer.

The reliability of the split timer would be the user starting it on time and also stopping it when it would be necessary. There could be sources of error with regard to the timer not being started on time and not being stopped on time. Other sources of error included (a) the users not taking the reading correctly (b) the zero error of the split timer was + - 0.0005 seconds. The systematic error be in starting late would be larger however as indicated all these were very small. Since the times being discussed were of the order of 17 minutes, the split timer readings were very reliable.

3.6 Research Procedure

Intact groups were used in the schools and they were named as A, B, C and D in order not to ensure protection of their identity.

“Pace setters” had to be adhered to as this was the prescribed content by the Province where the study was done. With this constraint, the research was mainly in two phases where in the first week school A and B were not using computer simulations and the educators were teaching as they normally did. They were told to teach in the same way as they had been doing in the previous year(s). However, one educator was teaching for the first time and had no previous experience.

The other two schools C and D used computer simulations in the first week. The simulations used were the PhET simulations on Geometrical optics which have been

described in section 2.5.2. A teacher centred approach was followed with the use of CS. The same content was taught.

The TDRV-GO instrument was administered as a pre-test before the intervention and then again after week 1 as post-test 1 and finally when the second week was over the learners were given the post-test2 in the week after they were done with the second week. It was the same instrument.

The CLRS was administered twice in week 1 for each of the control and treatment groups. It was administered at the beginning of the week and at the end of the week. In the second week, the same procedure was done as in week 1 for the CLRS. The CLRS was designed in such a way that the questions asked about how the participant experienced the cognitive load, the strain on his mind with regard to what was being taught (Paas, 1992).

A general summary of the research procedure is given (see Figure 11). The Research procedure clearly indicates when the different activities were done and administered. Before the intervention, a pre-test of the TDRV-GO was administered to all learners and a split timer was also used to determine the speed they wrote the test. Both groups used a teacher centred approach and one school used computer simulations additionally.

The cognitive load was measured twice during week 1 as well as twice during the second week when the activities switched. Post-test 2 was written and was also measured.

Figure 11. Research procedure.

3.7 Pilot Study

The pilot study took place in three stages. 3.7.1 Stage 1.

At this stage after the instruments had been constructed, the language of the

instruments was tested by four learners to determine whether the language was appropriate. What was not clear to the learners was adjusted and put in easier sentence construction. Adjustments were made namely with regard to the question of measuring the focal length.

Pr e- tes t Teacher-centred School C & D Using computer simulations Po st- te st1 CLRS1& 2 Po st- te st2 Split timer1 _{Teacher-centred} School A & B NOT using Teacher-centred School C & D NOT using computer simulations Teacher-centred School A & B Using computer Split timer2 Split timer3 CLRS3 & 4 Week 1 Week 2 81

3.7.2 Stage 2.

A mock pilot where pre-service student teachers were introduced to the use of the PhET software was done which also included using the overhead projector. While they were using it they were observed by the researcher. The pre-service teachers were then asked to use it in a teaching demonstration in such a way that this could help in the actual training of the educators.

What was learnt from this exercise was not to take the use of the data projector for granted as the simulations were expected to be used using a data projector. This was then considered while training the educators.

3.7.3 Stage 3.

The TDRV-GO, CLRS instruments and the split-timer were tried and pilot tested in an environment akin to the study which it was going to take place. It was a tried out in a school 60 km away from where the four schools who were involved in the study were situated. The number of learners who participated was 39 in grade 11. During this pilot study, the learners were asked to sit for the pre-test and after a week they re-wrote the test. It was actually a test- retest. The school however was very keen in learning more about simulations and was visited after the research to give them opportunity to learn about all the computer simulations.

The results of the test-retest were analysed and the reliability of the test determined as seen in section 3.5.1.2. It was also helpful in determining how the split timer would be used. It also showed the ease of having groups of not more than 40 especially while using the split timer.

3.8 Ethical Considerations

Permission for ethical clearance was sought and granted from the University ethics committee for this research to be done see Appendix D

Permission was also sought and granted from the Vhembe District Manager (see Appendix E). The condition on which it was granted by the district manager was that not to interrupt the day to day “running” of the schools and to inform the circuit managers (see Appendix F). For that reason, two circuit managers were notified and they accepted and granted me access to the schools.

Permission was sought and consent was given by the following: Vhembe District Manager (Appendix E ); the principals (Appendix G and Appendix H); educators involved directly with the study (Appendix I and Appendix J); parents of the learners who were younger than 18 (Appendix K and Appendix L); and learners older than 18 (Appendix M). It was not straight forward as indicated by the principal of one of the schools that most of the learners were living in child-headed families and in their invitations to the parents they get several learners coming as parents. This was a condition for one of the schools; it is for these and other learners that we hope that the outcomes of this study will make a difference. Acknowledgements were also requested from the Circuit managers, (Appendix F), to comply with the District Manager’s instruction.