Validity and reliability

Yin (2003) identified different forms of validity and reliability, and how to address them in the presentation of a case study. He distinguished between:

Construct validity: establishing correct operational measures for the concepts being studied

External validity: establishing the domain to which a study’s findings can be generalized Reliability: demonstrating that the operations of a study – such as the data collection procedures – can be repeated, with the same results (p. 34)

89 I address each of these now in turn.

For construct validity, there is a need to define the issues of interest. In the context of my data this includes issues related to:

 talking about and appropriation of subject discourses,

 use of dialogue more generally in meaning making,

 knowledge building as a cumulative process,

 multimodality of communication,

 a temporal view of the use of objects.

There is also a need to demonstrate how the data presented is illustrative of and relevant to addressing these issues. My data collection context provided a rich arena in which to focus on, and pan across, these issues, as they occurred in the natural course of lessons. I also wrote a summary of initial findings for the dance organisation, and presented initial findings at their own conference (based on data from all three classes), offering participants an opportunity to review and feed back on my interpretations. Through such sharing activity, Yin (2003) claimed that ‘the likelihood of falsely reporting an event should be reduced’ (p. 159), even in instances where there is no ‘truth’ to be revealed but different perspectives as gathered from different sources (as in my data), which thus increases the construct validity.

Regarding external validity, Yin identified that the case study is concerned with ‘analytic

generalization’ – linking phenomena with theory – whereas experimental designs are concerned with ‘statistical generalization’ – linking phenomena with the wider population. For single-case studies, where possible, external validity of claims can be enhanced by indicating where

phenomena are observed or referenced across the units of analysis, and how theory can be used to support findings made against practice (‘replication logic’). This is the approach I adopted to enhance the external validity of my analytic claims.

90

One further means of enhancing the validity of a qualitative study is to use different data sources to evidence and illustrate a claim, as suggested in my use of interviews, focus groups and VSRD sessions. This is not to conclude that the teachers and pupils would have the same experiences when talking about the same event, a conclusion that would not fit with my sociocultural framework, but such differences in interpretation themselves can be analytically interesting.

The validity of research design is another potential area of concern, whereby there is a trade-off between the advantages and disadvantages of a participant or non-participant observational study, as well as an ethnographic approach. Adopting a participant observation would potentially add more weight and validity to any claims, by coming from an ‘insider’ member of the

community. To do this however I would have needed to be immersed in all three classes, and such an approach potentially raises some ethical issues regarding informed consent, as well as issues of biased interpretation. Non-participant observation leaves me one step removed from the

community, and potentially makes participants wary particularly of being recorded, but means that informed consent can be gained before data collection. I also observed (but did not record) three lessons each with two of the classes a few months earlier (and with a different class in the third school), so they were relatively accustomed to me being there (see figure 3.2). In then recording events for the main data collection period I did so in as unobtrusive a manner as

possible. This was helped by having a wireless microphone, so that the microphone could be close to the activity I was focusing on, without me having to stand directly nearby (see section 3.5). An ethnographic approach would allow access to more dimensions and environments of participants’ lives, but would be very difficult to organise and gain consent for as well as being immensely time- consuming, given the large number of participants within three primary school classes in which I observed lessons. It would also involve a greater invasion into participants’ lives, which was not necessary and may indeed have detracted from my focus in my research questions to address the meaning-making activities that occurred and were resourced during their school-based topic

91

lessons. Thus a non-participant approach was deemed the most appropriate to the concerns of my participants and my own research needs.

By adopting a non-participant observer stance, I did not directly intervene in the activities as they happened, other than being in the same room. Words and actions recorded were therefore the spontaneous contributions of the participants in their real-life setting. In attempting to gain a participant perspective of those who were legitimate members of the teaching-and-learning communities, I conducted interviews, focus groups, and VSRD sessions, to give a further opportunity for participants to express their views so that I could use and apply their

interpretations when forming my analytic ideas. Having addressed notions of validity, reliability was the final key issue of concern outlined by Yin (2003, above).

Regarding reliability, Yin (2003) emphasised the need for transparency of research design: The objective is to be sure that if a later investigator followed the same procedures as described by an earlier investigator and conducted the same case study all over again, the later investigator should arrive at the same findings and conclusions. (p. 37)

As this situation would not be possible in case study research, as from a sociocultural view the exact same conditions would not re-present themselves for observation, a thorough and

transparent laying out of research procedures is the common way to establish reliability (which I offer in sections 3.5 and 3.6). The research design could therefore be repeated, but on different data.

Also in response to the challenge levelled at case study research that it perpetuates bias and views already held by researchers, Flyvbjerg (2006) emphasised the importance of maintaining ‘rigour’ within the case study method, alongside the ability to use it to ‘“close in” on real-life situations and test views directly in relation to phenomena as they unfold in practice’ (p. 235). Flyvbjerg also commented that through the process of in-depth data collection and analysis, a

92

case study researcher often generates findings that are substantially different to their initial interpretations of their data. This latter point was what I found regarding my ideas about use of subject terms in the lesson data, and in-depth analysis around this issue revealed some

interesting, unexpected findings (see chapter 5). Such findings would not have been possible without the detailed data collection that a case study involves, and detailed analysis that it allows. The quotation above also highlights the aspect of addressing ‘phenomena as they unfold in practice’, which would not be available within the controlled requirements of an experiment, and thus identifies an advantage of the case study in addressing the often unexpected nature of real- life events as they occur in real-time.

Having outlined the background and issues related to use of a case study method, I now offer more detail on how I adopted this method relative to recent research.