Ensuring validity and reliability

A good novel tells us the truth about its hero; but a bad novel tells us the truth about its author.

G.K. Chesterton It is said that “there are almost as many different lists of suggested criteria for judging research as there are writers about the issue” (Hodkinson, 2004, p. 11). A criticism levelled at the hermeneutic phenomenological approach is that it cannot produce universally valid knowledge, this is true, but it is not a weakness. This thesis details situations where “it is useful to know how people interpret what has happened to them, irrespective of other’s accounts, and irrespective of what more ‘objective’ observers might regard as ‘true’ or ‘accurate’’’ (Szarycz, 2009). The ‘experiences’ volunteered by the participating teachers in this research are by definition individualised and located in context; they provide thick descriptions rich with meaning. To counter the suggestion that the traditional evaluative canon of validity and reliability may be compromised by the researcher’s approach and practice I offer the following reply.

Validity is regarded as “the accuracy and truthfulness of findings” (Atheide & Johnson, 1994. p. 487). Interviewer bias during the interviewing process and processing stages of the data are cited in critiques of this choice of method (Kvale, 1996, Smith, 1998). Opinions regarding the capacity or desirability of measures to avoid bias are rife. I chose not to attempt a formulaic approach or ‘prescriptions for practice’ for my interviews as Bishop (1997) asserts that it is unworkable because researchers need to draw on their knowledge, experience, feelings and intuitions during interviews. In keeping with the American or ‘continental’ version of phenomenology developed by Heidegger, Gadamer and van Manen I have acknowledged and drawn on prior knowledge and experience throughout the interviewing process. I also decided to rely on the questions from the questionnaires as prompts to begin conversations and then to be open and responsive to the direction in which the teachers wished to proceed.

It was my goal to ensure that participants interpreted the questions in each questionnaire as I had intended. The terms used were jargon free and in plain English. The first questionnaire was trialled with non-participating colleagues and amendments made following their constructive criticism. At the beginning of each interview I sought confirmation that the questions had been understood. By organising interviews within a week of each questionnaire I gave respondents the opportunity to expand and clarify their written responses if they wished. It was also possible to immediately correct misinterpretations as they became apparent; this was especially germane for respondents for whom English was an additional language. With regard to establishing the accuracy and truthfulness of replies Cohen et al. (2007) suggest ‘convergent validity’ may be achieved by comparing the interview measure with another, pre-validated, measure. Triangulation by using multiple-methods is another approach that is also advised (Campbell & Fiske, 1959). The combined use of questionnaire and follow- up interview was consistent for all participants through each phase. This allowed for rich data to be collected that represented the individual

experiences of each respondent and provided enough commonalities in the approach to make comparative analysis of the data possible (Kelchtermans & Ballet, 2002, p. 109). The practice of checking questionnaire replies during interviews and returning transcripts for checking by participants was a consistent means of achieving “respondent validation” (Hammersley & Atksinson, 2003, p. 227).

Reliability refers to the “stability of the methods and findings” (Altheide & Johnson, 1994, p.487) the repeatability of the research or, in other words, the amount of variation that occurs when research is repeated. Every effort has been made to ensure that the data gathered is in response to the variable being scrutinised and not as result of the research method or application. By detailing the research methodology and its application through the methods of interviewing and questionnaires I have provided a template for future research.

These methods combine to gather a progression of snap shots of participants’ early experiences, and their understandings of those experiences. Many factors affect perception, and perception changes over time and with context, hence participants are unlikely to maintain the same interpretation of an event/situation as it fades from memory or is reinforced by subsequent events. The teachers’ responses were rendered by their authors on several occasions and so they may be expected to construct multiple perspectives and these may even appear contradictory. In addition, the informant’s choice of detail and purport will be affected by context and audience. Yet, I believe that by gathering information at intervals throughout the dynamic process of each teacher’s induction and integration into their new school community has been possible to map reifying and endorseable self-told lifeworlds and discover OTT’s journeys towards professional socialisation in New Zealand (Sfard & Prusak, 2005).

3.9 Conclusion

This research was designed to investigate the professional socialisation of ten OTTs employed urban secondary schools in 2011. It has been conducted over the course of three terms and details the experiences of the participants with macro-, exo- meso- and microsystems. The research philosophy and methodology have been informed by hermeneutical phenomenology (Gadamer, 1989; Heidegger, 1996; van Manen, 2011). The approach chosen for this research has enabled new perspectives and new knowledge to be gained by the gathering of contemporary commentaries on the process of professional socialisation as it is experienced by OTTs.

This chapter has presented the philosophy, methodology and methods used for this research. I have detailed the strengths and the weaknesses of approaches that I have rejected and I have explained those that I have selected. An acknowledged feature of a qualitative study design is that it must be “emergent and flexible, responsive to changing conditions of the study in progress” (Merriam, 2010, p. 16). With this in mind it must be acknowledged that expeditious adjustments to the application of the methods were made; these are explained and discussed in Chapter five. The next chapter presents the findings of this research in the words of the participants.