Reliability and validity in relation to the case studies

At a simple level, reliability can be understood as ‘the extent to which a research instrument such as a test will give the same result on different occasions’ (Thomas 2009: 105). Validity is the extent to which a test or procedure ‘measures or describes what it is supposed to measure or describe’ (Bell 1993: 65). Behind these simple definitions lie complex issues, particularly in relation to their applicability within qualitative research. Even in the simple definitions provided above, a problem of the priorities and purpose conveyed by the language used is evident. Measurement, for example, is typically not a major preoccupation for

147

qualitative researchers (Bryman 2008), and the notion of a test or testing could be viewed as belonging to the positivist tradition.

The concepts of reliability and validity are contested areas within qualitative research. Arguing that the field of qualitative research needed to employ different criteria rather than drawing on those used by quantitative researchers, Lincoln and Guba (1985) put forward the four terms of credibility, transferability, dependability and confirmability as equivalents for the conventional terms internal validity, external validity, reliability and objectivity. They assert that the introduction of these terms is ‘not simply to add to naturalism’s mystique or to provide it with its fair share of arcane concepts, but to make clear the inappropriateness of the conventional terms when applied to naturalism and to provide alternatives that stand in a more logical derivative relation to the naturalistic axioms’ (Lincoln and Guba (1985: 301)

These four terms have formed the guiding principles in relation to this study. Credibility relates to the degree to which findings make sense (Lincoln and Guba 1985). This was supported through prolonged engagement with the participants and opportunities within the sequence of interviews to revisit topics discussed on previous occasions. The group of participants comprised male and female teachers of different ages, working in different phases, in different schools and in different geographical locations. This reflects Shenton’s (2004) view that credibility can be supported through involving different types of participants and different sites.

Transferability is viewed by Lincoln and Guba (1985) as the extent to which the findings and any conclusions drawn can be applied to other settings or situations. This, in itself, is not that dissimilar to the concept of external validity. However, the difference lies in the responsibility of the researcher and the reader. The responsibility of the researcher is to ensure that sufficient contextual information about the fieldwork sites is provided to enable the reader to make decisions about transferability. It is the responsibility of the reader, having read the description within the research report of the context in which the work was undertaken, to determine how

148

far they can be confident in transferring findings and conclusions. In approaching the process of writing up the research I was very conscious of the need to provide an audit trail to ensure that the reader was able to trace the course of the research step-by-step via the decisions made and procedures described.

Dependability is linked to transferability and involves the detailed reporting of the processes used within the study so that there is the potential for a future researcher to repeat the work, even though they would not necessarily gain the same results or make the same interpretation. Through the level of detail provided, the reader is also able make their own assessment of the extent to which proper research practices have been followed. In this chapter, and throughout the thesis, I have sought to set out clearly what was done.

Confirmability involves incorporating measures to help ensure as far as possible that the findings within the research are the result of the experiences and ideas of the participants, rather than the characteristics and preferences of the researcher. The process adopted of recording interviews, transcribing them verbatim and quoting interviewees’ actual words is one way in which confirmability was pursued in this study. This means that the reader is able to make a judgement about whether my interpretation of a point made by a particular interviewee appears reasonable. The notion of an audit trail that provides detailed accounts of the decisions taken at every step of the data gathering process and in relation to interpretations made at the analysis stage is once again relevant. Reflecting this concern with capturing the twists and turns within the research process, Miles and Huberman (1994) consider that a key criterion for confirmability is the extent to which the researcher admits their own predispositions. Throughout this study I have attempted to make clear the reasoning behind particular decisions, including open acknowledgement of any of my beliefs and assumptions that may have exerted an influence. I have also endeavoured to offer reasons for favouring one approach when others could have been adopted and acknowledge weaknesses in the techniques actually employed. It is also hoped that the detailed

149

methodological description allows the integrity of research results to be scrutinised.

Triangulation is often referred to in relation to confirmability. This can take a number of forms. It may involve the use of different methods to investigate the same phenomenon. The idea behind this approach is that this can both compensate for the individual limitations of particular methods and capitalise on their respective benefits. Though this study used questionnaires and interviews, the purpose was not triangulation; these were two separate phases of the research. However, as described earlier, within the sequence of interviews conducted different approaches were used so that participants encountered a number of different stimuli to elicit their perspectives on behaviour related issues.

Another form of triangulation involves the use of a wide range of participants. Shenton (2004: 66) suggests that this means that individual viewpoints and experiences can ‘be verified against others and, ultimately, a rich picture of the attitudes, needs or behaviour of those under scrutiny may be constructed based on the contributions of a range of people.’ Though the selection of seven case study participants was not underpinned by a desire to generalise, involving a range of individuals was important in the context of a collective case study intending to shed some light on the thinking of teachers in the early stages of their careers. It allowed for some cross checking to determine whether an individual within the group represented a lone voice in expressing a particular view or attaching significance to an issue or there was some commonality amongst the group in how they were interpreting their experiences as they moved through their early years as teachers.