Illuminative evaluation studies internationally

The specific conceptual distinctions of Illuminative Evaluation, the ‘instructional system’ and ‘learning milieu’ in combination with Bernstein’s (1971) conceptual distinctions between ‘collection’ and ‘integrated’ curricula have been employed by Hamilton (1976) to evaluate the implementation of Science curriculum at two Scottish schools. Using Bernstein’s (1971) distinction between ‘collection’ and ‘integrated’ curricula as a conceptual framework for his study, and applying the principles of illuminative evaluation, Hamilton observed the science classes at the two schools and combined this method with teacher interviews and six questionnaires administered to pupils. Typical of an illuminative approach study, Hamilton priotised studying the learning milieu as he believes, ‘when an instructional system is adopted, it undergoes modifications that are rarely trivial’ (Hamilton, 1976, p.182). The instructional system in the study was

‘Integrated Science” gazetted in the Scottish government document commonly known by schools as, Curriculum Paper Seven.

Through his lesson observations, Hamilton picked up several issues pertaining to the learning milieu which he identified as negatively effecting the implementation of the curriculum. Some of the issues uncovered through observation of the learning milieu at Simpson school which decreased the success chances of the innovation include; teacher absences, relative shortage of apparatus and lack of laboratory accommodation appropriate to the needs of the curriculum innovation. Hamilton thus uncovers mismatches between the instructional system and the learning milieu and concludes that ‘the innovation resulted in serving ends directly opposed to the intended’ (1976, p.2005). It is the illuminative evaluation approach that revealed that the Scottish Integrated Science national innovation as actually taught is in reality undermining the central premises of the innovation.

b. Primary education in Malaysia

The Illuminative Evaluation principles of ‘instructional system’ and ‘learning milieu’ have also been employed by Tuah (1982) in Malaysia to evaluate the implementation of a new curriculum innovation called “Kurikulum Bersepadu Sekolah Rendah (KBSR) in rural primary schools of Sarawak in Malaysia. The new curriculum was aimed at improving the quality of primary education. The researcher conducted an evaluation of this curriculum innovation to establish if it was working out as intended. Following the methods of illuminative evaluation, the researcher analysed the ‘instructional system’ of the curriculum innovation. The instructional system in the study was the curriculum policy documents. Through the study of the policy documents, the researcher established that some of the practices that were planned for teachers to adopt in putting the curriculum into practice include; an integrated approach in teaching and learning of curriculum subjects, use of a variety of teaching methods by the teachers such as group teaching methods,

whole class teaching and individual instruction, use of a variety of learning activities, flexibility in the choice of content and teaching and learning methods and aids and use of continuous assessment that is incorporated into the teaching and learning processes. Also using the principle of ‘learning milieu’, the researcher conducted naturalistic observations of lessons with follow-up probing interviews with teachers to establish how this teaching actually took place in classrooms. Thus using illuminative evaluation approach, the researcher looked for ‘matches’ and ‘mismatches’ or ‘gaps’ between what was planned in the curriculum with what ‘actually happens’ in the classroom.

Data revealed mismatches between the plans in the curriculum documents on how the curriculum was planned to be taught and what happened in the classroom in the actual teaching of the curriculum. Data revealed that the curriculum was not being taught as intended. For example, although the curriculum recommended the teachers to embark on creative and a variety of innovative approaches in their teaching, teachers were however observed to be employing teacher dominated methodology. Similarly, the new element of continuous school-based assessment was still observed to place emphasis on the evaluation of students’ achievement and performance at the end of a school term and year instead of incorporating it into the teaching and learning processes to facilitate learning of the children.

c. Thinking skills in England

The specific conceptual distinctions of Illuminative Evaluation, the ‘instructional system’ and ‘learning milieu’ have also been employed by Nichols (1998) to evaluate the ‘Thinking Skills’ curriculum innovation at Devon school in the United Kingdom. In the ‘Thinking Skills’ study, the evaluators used a combination of tools of illuminative evaluation. The researcher studied the instructional system. The instructional system in the study was the organisational structure of the school. The researcher thus gathered

information about the organisational structure of the school. Applying the principle of the ‘learning milieu,’ the researcher observed some lessons and noted the limited nature of the training that the teachers had received in the process of teaching children how to think critically and creatively. The researcher asked the pupils, both by means of a questionnaire and in individual interviews, what they thought of the new subject and how it compared to other subjects. The researcher further examined the pupils’ scores on a variety of different cognitive and academic tests. He also interviewed the school principal and the key teachers involved in introducing the new curriculum subject. Following the methods of illuminative evaluation, the researcher looked for ‘matches’ and ‘mismatches’ between what was planned in the ‘Thinking Skills’ curriculum with what ‘actually happens’ in classroom teaching to gauge if the curriculum was being implemented as intended.

Data revealed that the curriculum was not being taught as intended in the sense that the format of the lessons was very different from what was intended in the curriculum documents. For example, didactic teaching was very much the order of the day and little opportunity, if any, was provided for reflection and discussion with the pupils (Nichols, 1998, p.22). No marks or grades were awarded in thinking skills lessons, so there appeared to be no way in which individual students could judge their progress. By using illuminative evaluation, the evaluator was able to demonstrate quite clearly where and why the curriculum innovation was not working as intended and the principal of the school was forced to face up to the issue of either abandoning the curriculum project altogether or ignoring the evaluation and allowing it to continue in exactly the same form, or instigating change at the level of implementation in the classroom. He elected for the third of these options (Nichols, 1998, p.22).

The studies cited above have shown that they are methodologically congruent to illuminative evaluation. Illuminative evaluation has been widely used both in Africa

and internationally to evaluate education programmes. The studies find matches and mismatches or ‘gaps’ between the intentions of programmes and what happens in the actual implementation of a programme.

It is also my assumption that teachers may or may not be teaching Expressive Arts as intended by the national government, hence my attempt to establish, using tools of illuminative evaluation, if the learning area at this early stage is being implemented more or less as intended. However, the illuminative evaluation research studies cited in this chapter only indicate the discrepancy between the instructional system and the learning milieux and not much about the constraints that implementers of the programmes or projects face in the implementation of the programmes for that discrepancy to occur. Thus, this study was conceived to address this knowledge gap on the possible reasons for discrepancies between instructional systems and learning milieus especially in curriculum innovations. Thus discussions with teachers and classroom observations provided better avenues for this study in the exploration of how primary school teachers in Malawi made decisions in implementation of Expressive Arts curriculum. Such an exploration unearthed some of the reasons for the discrepancies that commonly occur between the instructional systems of programmes and their learning milieus.