CHECKING FOR RELIABILITY

There are aspects of the original research method that are fairly obviously problematic. Specifically, it could be asked whether the assessment process by which the data were interpreted was really adequate. We chose to make the majority of judgements on the developmental level of the compositions ourselves without an extensive use of independent judges, a form of ‘product analysis’ that could be said to lack objectivity. There are also questions about the sample of children: for instance, could these findings be repeated in another culture? There were worries too about the thinness of data from

older children. For these and other reasons, replication of the original study became essential and the research trail led to an investigation in a different cultural setting, Cyprus.

The music curriculum in Cyprus schools is in process of positive development, with much encouragement being given by Michael Stavrides and his colleagues to the development of composing in the classroom. This work is still relatively recently begun and lacks the history of established practices, such as those promoted elsewhere by various curriculum development projects and endorsed particularly in Britain by the assessment requirements of the 16-plus General Certificate of Secondary Education examination (GCSE) and the legally binding National Curriculum. However, there is very developed work in some Cyprus schools and a number of curriculum activities have been devised with composing as an essential element. It was possible to draw on this work during 1990 to assist in replicating crucial aspects of the earlier London study, testing the reliability of the criteria and, to some extent, the validity of the theoretical model underpinning the developmental spiral.2

There were three Cyprus music curriculum projects. These were based on the materials of pentatonic scales, drones and the idea of ‘contrast’. We decided to investigate in particular the musical outcomes of the ‘contrasts’ project, mainly because the idea can be handled by even the youngest children, who might find scales and drones problematic. ‘Contrasts’ can be interpreted at any level, from materials to form. Twelve teachers in kindergarten, primary and secondary schools were involved and the children worked for three sessions on their pieces. In the third session the compositions were recorded. Over 600 recordings were collected and from these 28 were selected at random. The only sorting rule was that there should be seven items from each of four age groups: 4/5; 7/8; 10/11; 14/15. These 28 items were then assembled in random order on one cassette tape. This may sound a little ‘clinical’ but we were looking for greater procedural rigour than in the previous London study and wanted to be sure that this particular ‘product analysis’ was carried out without obvious bias.

This taped sample of 28 compositions was later played to seven primary and secondary music teachers in Britain. They were first given time to study the criterion statements and then were asked to assess the Cyprus compositions, placing each into a criterion category. They were not asked to identify the ages of the children but to assign each of the 28 compositions to one of the criterion statements given above.

These teacher-judges were told only of the ‘contrasts’ project starting point and of the Cypriot origins of the recordings. All judgments were made quite independently by each individual without discussion, usually after one hearing—unless a second hearing was requested. Time was allowed between

each item for reflection on the composition and to consider the criterion statements.3

The relationship between the actual ages of the children and the placing of compositions by the spiral criteria was analysed and found to be statistically significant.4 _{There was a clear ascending relationship, between}

age and the order of the criteria abstracted from the phases of the spiral. The sequence of developmental levels we originally mapped is reasonably accurate and it follows that the overall theoretical framework has predictive power.

Figure 12 shows the proportion of the 196 judgements made in each of the spiral categories at each age level. It can be seen that the spiral modes arrive on cue and exactly in the predicted sequence. The Sensory, Manipulative and Personal Expression levels are already in place by age 4/5; by age 7/8 the Vernacular is established; by 10/11 the Speculative appears; and compositions at age 14/15 show the first emergence of the Symbolic mode.

Differences between the data

There is one apparent difference that can be easily observed by inspection of the two sets of data. This prediction was originally signalled thus.

We suspect that, if children are in an environment where there are musical encounters, then this sequence may be followed more quickly. The

opposite may also, unfortunately, be true: in an impoverished environment, development is likely to be minimal, arrested.

(Swanwick and Tillman 1986:338)

Children in Cyprus schools are certainly not in an impoverished musical environment but there is unquestionably less of a tradition of composing in the classroom. We were also assessing compositions from several Cyprus schools where children had different teachers, unlike earlier data-gathering in Britain, where one teacher was systematically engaged in developing a rich musical curriculum during the four-year period of the research. We might therefore expect more variability in the Cyprus compositions and possible lower levels of achievement as assessed by the criteria. For example, among the UK children we could anticipate more confidence with the musical vernacular and greater fluency in structural speculation with a greater dispersion of the lower criterion levels in the higher age groups among the Cyprus sample. An inspection of the data suggests this may be so. By analysis we can determine that, although the sequential order of development may be identical, the compositions of the UK children appear generally to be more advanced than those of their Cyprus peers and that there is greater variance among the Cypriot children.5

This evidence gives credence to the structure of the original music developmental map and fulfills the projections made for the older age group. Furthermore, in the light of the differences between the two samples it is tempting provisionally to hypothesise that music education in schools is able to make a difference to the musical development of children. However, the research was not designed to test this and it would be unwise to rely on these results to support what would be, for music educators, an attractive thesis.

Several questions are raised by these findings and further work would be helpful. A comment by one of the British teacher-judges was to the effect that the criterion statements would be an improvement on some current ways of assessing children’s compositions. There is indeed a need and some potential here for developing more reliable, more musically valid and cross- cultural assessment techniques and the necessity of sensitising judges responsible for the assessment of musical compositions. In music education there is a central problem of how we validly assess the music-making and music-taking of ourselves and others. How do we become sensitive critics? Can what we learn from studying children’s composing be applied to other musical activities—to performance and audience-listening? It is to the processes of assessment along a wider front that the research trail now leads us, not to satisfy the clamour for public accountability but to better understand the nature of transactions in music and music education and therefore to be able to respond more appropriately to the music-making and musical responses of students and others who may be engaged in music.

102