Data generation beyond the interviews

In addition to the interviews as the primary data source for this project four other research methods were planned; the Most Significant Change technique, a collaborative forum, observation, and document analysis. The Most Significant Change technique (Dart & Davies, 2003) was included in the ARC research methods and I undertook to carry out the Most Significant Change process with the teachers during their final interview. The nature of the teachers’ engagement at the end of the program and the overriding active interviewing approach meant that the Most Significant Change technique was not strictly implemented but stories of change were a critical component of the interviews (see chapters 7 and 8). I planned a forum which would bring together all the teachers from the different schools at the end of the program to share their stories and collaboratively interpret Global Connections. The use of stories in this way can be important for allowing people to construct and reconstruct their interests and meaning and overlapping stories allow for a collective agenda that can be powerful (Putnam & Feldstein, 2003). Although the teachers all agreed at the outset to participate in a forum, by the end of the program interest had waned and the forum did not eventuate.

5.7.2.1 Observation

Yin (2003) includes observation as one of the methods which can be added to the variety of sources used in a well designed multi-method case study. Observations take place in real time and so have

56 All of the interviews were transcribed and analysed using the qualitative research software program NVivo 8 and the coded interviews have been made available to the ARC project database.

101

particular advantages with regard to the development of the case study context. The disadvantage of observational data is that it is necessarily subjective and selective (Yin, 2003). Observation as a research tool can consist of a number of different methods⁵⁷ which can be used to provide different types of case study evidence (Bryman, 2004).

In this project, I was involved in a wide variety of activity as part of the ARC team. As a result, there were many opportunities to observe Global Connections in operation and the working environments of both Plan and the schools. There were also multiple opportunities to observe the interactions of the facilitators and the teachers and on a few occasions one-on-one interactions between the Plan coordinators and teachers. Yin (2003) suggests that the reliability of observational evidence can be enhanced if there are multiple observers. The majority of ARC activity was conducted with other research team members and I participated in a collective debriefing after each such event. I wrote up field notes after each event recording the team observations and my notes form part of the research evidence.

Yin (2003) also indicates that informally structured observation is a useful way of supplementing evidence generated during interviews. To this end, I also took notes after each interview that detailed observations of the interview context including such things as; where the interview took place (staffroom, shared office, empty classroom, busy classroom), the distractions and other activity that was taking place, and also impressions formed as to the extent and manner in which the

participant was engaged in the process.

Also included among the field notes which I classified as ‘observational data’ were several de facto

‘micro-interviews’ that I considered important. These arose when I had occasion to phone one of the research participants to ask them a particular question. The phone call was not recorded and was not treated like an interview but I did note down the relevant responses. Most commonly it involved calling the Plan program coordinator about something that was said in an interview with a teacher.

On one occasion it was to relate and discuss what I thought from an interview was a major misunderstanding that threatened to derail the whole program in the school (see section 8.2).

5.7.2.2 Document Analysis

Documents are relevant to virtually all case studies. They have the advantage of being able to be reviewed repeatedly and are generally exact (Yin, 2003). Additionally, the documents used were not prepared for this project or prepared by me and so were independent of my influence and bias that permeated both the interview and observational data. However, it is possible that some of the

57 Bryman (2004, p.167) lists and describes; Structural, Systematic, Participant, Non-participant, Unstructured, Simple, and Contrived observational methods. The data generated ranges from formal to casual using these different methods.

102

documents carried a selective bias from the people that prepared them. This element is particularly problematic in that such bias cannot be identified and acknowledged.

Nevertheless, with regard to this project the majority of documents were uncontroversial and therefore bias was considered inconsequential. The variety of documents that were used as background and as direct evidence in the case studies and their analysis included:

 Program documents related to the running of Global Connections, and the training of facilitators.

 Contract documents relating to the facilitators roles.

 Organisational documents that detailed Plan’s organisational structure.

 School mission statements and school newsletters.

 A large variety of ARC project documents including data transcripts and artefacts.

accumulated by other researchers in the ARC team.

 Email communications.

In most instances the documents were used to build understanding of the context of both the schools and Plan but in some instances they were used to extend the primary interview data – for example on one occasion when a teacher and the Plan coordinator had different recollections with regard to some of the pre-program activity the email trail sent between the two participants was used to categorically establish the sequence and timing of events.

Figure 5.3 on the following page summarises the data generation methods and the theory used to inform their specific design:

103 .

Figure 5.3: School Case Study Data sources and methods 5.8 Analysis Framework

Informal analysis was undertaken throughout the research process. Even before the first interview signalled the start of the active data generation, documents relating to the program operations were being considered in terms of ‘What is this program structure likely to mean for the schools?’

Similarly, immediately after the first interview was conducted reflection on the process included such questions as ‘What did I just hear?’, ‘What did it mean?’, ‘What did we miss?’, ‘What does it mean for the next interview?’ The inductive theory process that characterises qualitative work and case

 Interviews

 ARC research contexts

 Shared observations

 Plan documents

 School documents

 ARC project documents

 emails

Observation Document

Analysis

School Case Study

Interviews

Teachers School Management Facilitators Plan Staff

Active Interviewing

Action Research

Appreciative Inquiry Informed by Principles of

104

studies in particular was initiated. It is the initial development of theory prior to data generation which differentiates a case study approach from other strategies like ‘ethnography’ and ‘grounded theory’ (Yin, 2003).

Analysis and theory building was an ongoing process that iteratively fed back into the data

generation phases of the project. The embedded nature of analysis as part of the research process once again draws attention to reflexivity. There is no ‘outside’ from which to interpret the activities of social research (Geelan, 1997). The fact that tentative theory development began before the first data were generated suggested that the data analysis phase of the project should continue and augment this process. To this end, the data analysis began with an approach adapted from the coding principles of Grounded Theory.