VALIDITY AND RELIABILITY IN THE RESEARCH PROCESS

Maxwell (1992) argues that qualitative researchers need to be cautious not to be working within the agenda of the positivists in arguing for the need for research to demonstrate concurrent, predictive convergent, criterion-related, internal and external validity. Like Guba & Lincoln (1989) he argues for the need to replace positivist notions of validity in qualitative research with the notion of authenticity. Maxwell (1992) echoing Mishler (1990) suggests that “understanding” is a more suitable term than “validity” in qualitative research. We, as researchers, are part of the world that we are researching and we cannot be completely objective, hence other peoples’ perspectives are equally as valid as our own and the task of research is to uncover these. My possible bias as a teacher of many years standing, as well as my strong views on imposed curriculum policy changes, was potentially a form of bias. I was however aware of this. I was vigilant about not letting such bias dilute the trustworthiness and authenticity of the data produced. Kelly (1999: 433-434) notes that the criteria for social science research should surpass categories of reliability, validity and generality, which he considers to be remnants from a modernist correspondence theory of truth. He refers instead to principles of 1) congruence, consisting of internal consistency and coherence and 2) plenitude. In developing the accounts of the focus group workshops and analysis of the questionnaires and interviews transcripts, I strived to adhere to such principles and to develop comprehensive accounts that maintained what he calls “a balance between generality and contextual detail.”

Therefore the peculiarity of my study, particularly its inquiry into disparate areas such as education policy, arts education, curriculum determinants and culture, made it necessary for me to employ a variety of methods best suited to my intent and rationale. I needed therefore to show that the assumptions on which my research methodology rested were rigorous and trustworthy but, like Lather (2001a) I questioned whether the discourse of ‘validity’ was still adequate to the task. The soundness of qualitative research say Guba & Lincoln (1989) depends upon criteria such as credibility, transferability and dependability. All these criteria were addressed in this study in a number of different ways. For example, my selection of documents that most closely identified the Arts and Culture phenomenon, my full transcriptions of interviews and the use of verbatim extracts in the analysis and the corroboration of the literature also established the credibility of this study. Another way in which I could attest to the rigour of the study was to show how it was guided and shaped by its theoretical framing.

Interestingly, according to Bogdan & Biklen (1992), researchers need to locate discussions of validity within the research paradigm that is being used. Hammersley (1992: 50) suggests that validity in qualitative research is what allows us to have confidence in our results, but that as reality is independent of the claims made by researchers, our accounts will be representations of that reality rather than a reproduction of it. Hammersley (1992: 179) further notes that the validity of qualitative research lies in its internal rather than its external validity. Therefore in this research it is the relationships of the observed responses that matter.

According to Bogdan & Biklen (1992: 48) reliability in qualitative research can be regarded as a fit between what researchers’ record as data and what actually occurs in the natural setting that is being researched, i.e. there should be a high degree of accuracy and comprehensiveness in the coverage. This is not to strive for uniformity; two researchers who are studying a single setting may come up with very different findings but both sets of findings might be reliable. As

Bogdan & Biklen (1992: 45) point out generalisability, construed differently from its usage in positive methodologies, can be addressed in qualitative research. Positivist researchers, they argue are more concerned to drive universal statements of general social processes rather than to provide accounts of the degree of commonality between various social settings (e.g. schools and classrooms). Bogdan & Biklen (1992) are more interested not with the issue of whether their findings are generalisable in the widest sense but with the question of the settings, people and situations to which they might be generalisable.

According to Cohen, Manion & Morrison (2007: 141) it is vital to be systematic about establishing the trustworthiness of data. One way of improving the trustworthiness of data is through triangulation. Triangulation is a powerful way of demonstrating concurrent validity, particularly in qualitative research. The use of multiple methods, or the multi-method approach as it is sometimes called, contrasts with the ubiquitous but generally more vulnerable single-method approach. Furthermore, the more the methods contrast with each other, the greater the possible trustworthiness. Guba & Lincoln (1989: 315) suggest that triangulation is intended as a check on data, while member checking? and elements of credibility, are to be used as a check on member’s ?construction of data. The researcher needs to be confident that the data generated is not simply an artefact of one specific method of collection. Such confidence can be achieved when different methods of data collection yield substantially the same results. Triangular techniques are suitable when a more holistic view of educational outcomes is sought or where a complex phenomenon requires elucidation.

Although I see triangulation as a rather positivist construct, in working within the constraints I have accepted here, I am obliged to refer to it. Triangulation in qualitative research usually refers to how data from different sources can be used to corroborate or elaborate a claim (de Vos et al, 2005). In my study triangulation is readily applicable to the various methods of data production. For example, in the current research the triangulation of different data sources such as interview

and questionnaire summaries was done, in keeping with the view of Miles & Huberman (1994: 438) that “triangulation is less a tactic than a mode of enquiry.” They add “by self-consciously setting out to collect and double-check findings using multiple sources and modes of evidence, the researcher builds the triangulation process into on-going data collection.” I produced data from different sources, using focus groups, one-to-one interviews and questionnaires and constantly squaring the findings by means of triangulation.

It is through all of the steps described above that the research establishes its credibility and dependability. The theoretical and conceptual frameworks of the research support all of this further. Finally, I offer the following criteria proposed by Riessman, (2002) and cited in Singh (2007) for making it possible for others to determine the trustworthiness of my work by:

• Describing how the interpretations were produced

• Specifying how I achieved successive steps of analysis and representation.

(Riessman, 2002: 261)

In adhering to these prescriptions of interpretive analysis, I met the internal constructs of rigour, as well as making my study trustworthy.

3.5 CONCLUSION

The research methodology and research process defined in this chapter informed the investigation of the research questions. I showed how I linked the cognitive theory to the methods and techniques that I used to produce the data and analyse it. The research design used made provision for the fluidity and pragmatic nature of qualitative processes, which are concerned with not compromising the validity of the data generated. Care was also taken to ensure that the research design and

process, the research method and data collection techniques were appropriate. As such, data was produced through a variety of interview modes and open-ended questionnaires. I described how the instruments for analysis were created. In sum, every care was taken to ensure that I could do a focused and critical study in order to arrive at credible findings. In the next chapter, the data is presented and analysed.