Analysing elicitation data.

6.1.5.1 Analysing conceptions of geography

There is no one scoring form la that is most appropriate for concept maps, since concept maps ma be sed for a ide ariet of p rposes. (No ak, 1995: 84)

It has already been established that concept mapping can be used for a variety of purposes for teaching & learning, assessment, to monitor changes in conceptions and of the articles read, no two seem to use exactly the same method for analysing these maps. A range of qualitative and quantitative methods can be used and, in some cases, a

combination of both is appropriate. It therefore seems important to ask h do I ant to anal se the concept maps? as a first step, rather than ho ill I anal se the concept maps? . Once clear about the purpose (or purposes) for analysis it will then be possible to select from the range of techniques as appropriate.

The use of concept mapping, as indicated above, was for two reasons. Firstly to provide students with a means of examining their conceptions and beliefs about geography as a subject and to provide a basis for each student to consider if, and how, their conception changed as a result of experiences during the PGCE course. The focus here is on personal learning and development and therefore qualitative methods of analysis seem most appropriate. Secondly, concept mapping was used as a research tool to gather data that would enable conceptions to be analysed in order to generate categories into which

st dents concept maps co ld be placed according to how sophisticated their conception of geography appeared to be. Once all the concept maps were assigned to a category, I would be able to select a smaller sample to interview and observe teaching in school whose conceptions would be representative of the range of levels of sophistication evident in the cohort as a whole.

Chapter 6: Discussion of research tools

97 To make comparisons between students as fair as possible I was aware of the possibility of imposing my own values if a solely qualitative method of analysing the concept maps was used. It therefore seemed appropriate to develop a set of criteria against which to score each map. Many articles (Haseman and Mansfield, 1995; Ghaye & Robinson, 1989; Artiles et. al., 1994) describe the use of some sort of system for scoring elements of a concept map. Typically this involves either developing a, or using an established, set of criteria against which the maps might be scored. It is then possible to make a quantitative

comparison between concept maps either those of the same student over time, or between st dents and/or others at the same time. At the simplest le el concept maps can be scored according to the number of nodes or concepts shown, and the number of links made

between nodes. Judgements can be more sophisticated if the following features or characteristics of nodes and links are scored:

the number of appropriate / relevant concepts included

the number of appropriate relationships indicated along the links (this can only be done if the links are labelled, which is not always the case)

higher scores may be given for relationships that show a high degree of specificity or precision than a more generic connection

the degree of complexity (number of cross-links or interrelationships shown between nodes for example, within as well as between levels in a hierarchical concept map) One of the major criticisms of quantitative approaches is that a concept map is designed to present a holistic view, the meaning of which may become obscured or fragmented if over- detailed analysis is done of its component parts. For this reason some research projects have used a combination of techniq es. In addition, the concepts of appropriate and

rele ant are not al e free so hile the scoring s stem might, on the s rface, appear to lead to more objective judgements, the data are still subject to the interpretation of the person analysing them.

In the conte t of researching Y6 children s conceptions of an aspect of geograph Gha e and Robinson (1989), developed a scoring system for classifying the links between nodes (or concepts). In the course of scoring for links and concepts and then comparing each

Chapter 6: Discussion of research tools

98 child s score ith that of their teacher, the became a are that there ere some q alitati e distinctions between maps that the scoring system did not capture and so they developed a

t polog of reconstr ctions that ser ed as an indicator of the extent to which the children s maps sho ed comple it , q alit , ariet , match, d namism and approach (Ghaye & Robinson, 1989: 129) with descriptions of what each might look like for a more able child and a less able child. It was then possible to make a more qualitative judgement about a child on a continuum from the more able to the less able descriptors. A particular indicator that became sef l for this research as the notion of match . Gha e and

Robinson noted that more able pupils tended to diverge from the elements provided by the teacher and to show a degree of originality, whereas less able pupils tended to try to stick to the elements provided and to replicate what they had been taught.

In a similar vein Morine-Dershimer (1993), working with student maps that revealed their ideas about teacher planning, developed a category system based on elements in the st dents maps. The maps ere sed to identif elements that disting ished one from another as well as those that were common to all. On this basis the category system then enabled her to sort the concept maps into four broad groups on a best fit basis.

6.1.5.2 A combination of both techniques

For the purposes of this project a combination of techniques seemed to be most appropriate, the advantages of one helping to offset the disadvantages of the other. As mentioned above, the interpretations made of the concept maps are bound to be affected by the conceptions and values of the researcher. In an attempt to minimise these effects I used a scoring system2, developed from the maps themselves (as by Morine-Dershimer, 1993)

that foc sed on both the maps str ct re and content, and the relationship sho n bet een nodes (as for Ghaye and Robinson, 1989). The scoring system (table 6.2) was then used to give a score for each of the 79 maps and discussed with a colleague in order to check the validity of the judgements made. This enabled me to identify areas where the scoring

2Scoring is an unusual move for grounded theory research. On this occasion it was used as a mechanism for

generating categories for st dents concept maps indicating the degree of sophistication of their conception. This then enabled selection of a representative group of students to take part in the stimulated recall interviews and the school-based phase of the research.

Chapter 6: Discussion of research tools

99 system was leading to (a) different interpretations and (b) weighting the score of some elements of the concept map above others.

Structure of concept map Hierarchical

Relationships between nodes Cross links at each level General concepts added Specific concepts added

Frequent explanations along links Types of concepts / relationships Spatial / locational

Time / changes

Causes / effects unidirectional Processes

Interactions Issues

Active geography (investigations) Skills (mapwork, fieldwork)

Apparent structure / no apparent structure What type of relationship?

Linear links / cross links

Statement along links descriptive or

explanatory? Consider a threshold of 3 or more in order to score for explanatory?

Same as causes / changes Inter-relationships

These are represented in persuasion so remove

Table 6.2: First set of Criteria used for scoring concept maps

Structure of concept map

1. Does the map have an apparent ordered structure? 2. Is there a linear relationship between nodes?

3. Is there a more comple s stem of links ithin and bet een different le els in the map? 4. Have some general geographical concepts been added in the nodes (e.g. key concepts such as

processes, location, global warming)?

5. Have some specific concepts been added (e.g. places might be divided into countries, cities, rural / urban etc)?

6. Are there descriptive statements along the links?

7. Are there statements along the links that are explanatory / interesting geographically? (i.e. go be ond sa ing affects to explain in what way one thing affects the other etc)

Types of concepts / relationships between nodes

1. Does the map have a spatial / locational element?

2. Does the map have a time / changes element? (perhaps related to geographical processes) 3. Does the map focus on causes /effects, influences?

4. Are complex inter-relationships (e.g. human-physical, physical-human, human-human interactions etc) represented on the map?

5. Is there a sense of active geography? (through investigations, enquiry etc?) 6. Are such things as map ork / field ork sho n as geographical tools ? Table 6.3: Criteria used to score concept maps after revisions

The criteria for scoring were revised to take account of the points raised in the literature. One of the factors to take into consideration was that not all students chose to represent

Check key themes / concepts in geog are represented

Chapter 6: Discussion of research tools

100 their conception of geography as a hierarchy, but this did not mean that their conception

as an less comple than others . Indeed, one of the challenges as to gi e credit to complex ideas that might be represented in a seemingly simple model. A cluttered map with many nodes and links was not necessarily a map that represented complex ideas. The final version (table 6.3) was also expressed in the form of questions, to aid analysis

undertaken by others as part of the validation process. The system gave one point for each element that was present in a map, the total score for any map therefore being 13. Careful consideration had been given to the idea of differential scoring for each element (for example, 1 point = any representation of the element; 2 = detailed representation of the element; 3 = element represented in a way that adds complexity) but this was thought unwieldy and too open to subjective interpretation. The overall purpose of scoring was to enable distinctions to be made between students in terms of the overall complexity or degree of sophistication demonstrated in their concept maps so a single point for each element seemed appropriate. It then remained to consider at what point representation of an element o ld trigger the score of a point. In this respect an representation of the

element would be given a point as long as it made sense, geographically.

All concept maps were scored for a second time using the revised criteria from which four broad categories ere de eloped to represent the degree of sophistication of a st dent s concept map. Clearly this was a fairly crude tool but it served the purpose of making distinctions bet een maps more objecti el than the e e-ball techniq e sed d ring the pilot study. It also served the purpose of creating categories in which all the maps could be placed and from which it was then possible to select a sample for follow-up interview. An obvious drawback of the criteria is that in the application they will still be open to interpretation. In order to check the validity of the interpretations I made, a sample of maps were scored separately by two colleagues a secondary and a primary geography tutor without them knowing what scores I had allocated. I then compared my scores with each of theirs and any differences were discussed. While there were some differences in the detail of the interpretation made by each individual, there were few differences in relation to the four broad categories that were developed (see chapter 7, p 122-125). A further drawback is that the scoring system also did not allow for quality within any one element, or within the map as a whole, to be shown. However, some qualitative analysis was done by comparing the o erall, holistic conception portra ed against Barratt Hacking s (1996)

Chapter 6: Discussion of research tools

101 geographical persuasions (appendix 3). The purpose of this was to provide a sense of what

sort of geographer each st dent seemed to be and a basis for e ploring hether, hen interviewed, they would categorise themselves in the same way.

6.1.5.3 Analysing conceptions of teaching and learning

The purpose of analysing the written accounts of how students might teach an aspect of geography they had chosen and what the benefits of this might be for the pupils was to give the researcher some insight into their conception of teaching and learning and, to some extent, what they appeared to value in geographical education. These are qualitative data that were analysed accordingly using the theories of teaching categories (appendix 14).

The concept maps showing the relationship between content, pupil and teacher were then analysed qualitatively by looking at three key elements:

balance - is one element emphasised more than the others? interconnectedness - are they all linked to each other?

language - what terms are used along the links to describe the relationship?

These combined to provide a holistic conception that could be compared with that analysed using the theories of teaching. As such, the analyses of these concept maps was as much about validating the data from the written accounts as anything else.

6.2 Interviews

This section will begin by discussing the rationale for selection of stimulated recall as an interview technique, and some of the issues associated with use of interview as a means of gathering data. It will then discuss stages in the interview process designing the interview schedule, conducting interviews, analysis and writing up and make distinctions between the two types of interview conducted.

Chapter 6: Discussion of research tools

102 McAleese (1999) holds the ie that concept maps are an artefact of a process b hich learning occ rs (p. 8). In this respect he is s ggesting that the concept map acts as a record of a process and it is the process itself that is important not the product. His research

focuses on the use of concept mapping as a tool for learning rather than a tool for eliciting conceptions and beliefs, ho e er this ie , alongside No ak s that a concept map can onl represent an indi id al s conception at the moment in time when it is created, supported the idea that it would be necessary to conduct interviews that explored the process students went through at the time of making the concept map, and the extent to which they felt their map represented their conception of geography.

This is not to say that the products the concept maps themselves were not also useful. Analysis of these, along with the written accounts of teaching and learning, enabled me to place students in categories according to the sophistication of their conception of

geography, their geographical persuasion and their theory of teaching. However, it was clear that the interpretations of these data were open to question. Would others, including the students themselves, accord similar categories? Thus it was seen to be important to do follow up interviews with a sample of students (n=11) and the purpose of the interview would be to (a) use the data as a stimulus for discussion during which they would be invited to elaborate on their conceptions and the process they went through to create their concept map, (b) offer them the opportunity to validate interpretations made and (c) to explore a further aspect of conceptions, namely their conception of the nature of

knowledge, which initial analyses of the data and further reading seemed to be indicating was an important aspect to explore.