Part 6. Maximising the Benefit of Resources
3.5. Validity and Reliability
3.5.1. Creating and validating new knowledge
T
he aim of research is to create public knowledge which is not just new to researchers as individuals, but new to a research community. In all research, consensus within the research community establishes whether there is sufficient critical mass of data to draw conclusions about its validity. A filtering system in the form of adjudication, either through presentation of research findings at conferences, or publication in peer reviewed journals renowned for demonstrating rigour, evaluates the plausibility of educational claims and assesses the rigour of research projects. This manuscript has been presented as being of archivalsignificance, contributes to a body of knowledge and is responsive to the intended purpose. It discusses procedures and research results in terms of contribution to theory, and clearly identifies and acknowledges limitations and speculations (Elliot, Fischer, & Rennie, 1994).
64
There are several aspects of research validity available for consideration, the choice of which is located within the research paradigm used. Whereas validity within quantitative research is based on ‘certainty’, qualitative research accepts that hidden variables may affect its validity (Atkinson & Delamont, 2006;
Cresswell et al., 2007). Although Maxwell (1992) agrees with Cohen et al. (2011) that qualitative research generally cannot be replicated, Maxwell proposes that validity is based on five aspects of understanding. These five aspects, which have been carefully considered within this project, comprise descriptive validity
(providing an objectively factual account), interpretive validity (interpretation of respondents meaning), theoretical validity (research explains phenomena), generalisability (internal/external validity) and evaluative validity (judgement based on critical/theoretical perspectives rather than researcher interpretation).
Of key importance is consideration of internal and external validity. Research quality and internal validity of data is strongly dependent on research design which “integrates research, critical reflection and action” (Lincoln, 1995, p. 281). Validity and quality of interview procedures call for as much transparency, clarity and explicitness as possible, free of coercion and bias (Cohen et al., 2007;
Heshusius, 1992; Mentor et al., 2011). Content and face validity develops from interview questions based on sound theoretical framework.
Semi structured interviews encouraged reciprocal negotiation between interviewer and interviewee to accurately portray the interviewee’s voice, thus providing credible conclusions which inform readers that the research data is trustworthy and accurately measures what it was supposed to measure (Bishop, 1997; Cohen et al., 2011). Purposeful sampling and careful preparation of interviews negated any bias of gender, race and ethnicity and ensured that culture and cultural
differences were acknowledged and taken into consideration. To ensure there was no change in contextual situation which might impact on the validity of data, all interviews were conducted in participants’ natural contextual setting (Cresswell, 1994; Mentor et al., 2011).
External validity requires research results to be generalised based on the
65
al., 2007). This report demonstrates a thorough understanding of methodology behind data collection and analysis, and provides sufficient thickness of
descriptions for adjudicators to make informed judgment. The inclusion of a multitude of citations used in the literature review to support my research added credibility to findings. This provides readers with confidence that knowledgeable researchers have given due consideration to reliability and conclusiveness of previous research findings (Bell, 2005; Boote, 2008; Cohen et al., 2011; Yin, 2003).
Examination of interview recordings, observational notes and information from initial surveys permitted accurate interpretation and analysis of data, exposing dichotomies, examining silences, disruptions and contradictions (Yin, 2003, 2006). I have taken some resource documentation from the MOE and from school board meetings and used this in conjunction with literature and data gathered in
interviews, and so that tends to triangulate. Data saturation and confirmatory triangulation ensured quality, reliability and authenticity of informed descriptions. This allows transferability of findings to other research which is essential for validation of new knowledge (Bell, 2008; Bell, 2005; Cohen et al., 2007; Cresswell, 1994; Desimone & Le Floch, 2004; Hernon & Schwartz, 2009;
Heshusius, 1992; Koro-Ljungberg, 2010; Kvale & Brinkmann, 2009; Lather, 2006; Maykut & Morehouse, 2001; Mentor et al., 2011; Sapsford & Jupp, 1996).
Finally, construct validity assesses how well an interviewer’s thoughts match that of the interviewee. As interviewer I needed to be knowledgeable, skilled in communicating, listening and taking notes, sensitive and open to new ideas and flexible in steering an interview. It was important to be critical enough to challenge what was said, have the ability to remember what had previously been said, and to be good at interpreting information (Brewerton & Millward, 2001; Mentor et al., 2011). Given the interpretive nature of interviews, I was aware that my prior experiences could shape the interpretation of data, and that I could influence and distort responses of participants with my personal bias, values and judgements. Therefore, every effort was made to minimise bias.
66
Misunderstanding between interviewer and interviewee such as seeking answers that supported preconceived notions, misperception on my part about what the respondent was saying, or misunderstandings by the respondent about what was being asked could have resulted in intended meaning being lost in the transfer from oral to written accounts. Other sources for bias could have arisen from nonverbal expressions or tone of voice which could create indirect information and influence the way research findings were interpreted. This may have included changes to wording of questions, poor or biased prompting or alteration to the sequence of questions (Cohen et al., 2007). Verification and confirmation of interview transcripts by respondents prior to release for publication provided opportunity for any bias to be addressed and helped ensure accuracy of data (Bishop, 1997; Cohen & Manion, 1989; Cohen et al., 2007; Cresswell, 1994; Lincoln et al., 2011; Lodico, Spaulding, & Voegtle, 2006; Mentor et al., 2011).
3.5.2. What counts as evidence
Evidence stems from research built on “collaborative, reciprocal, trusting,
mutually accountable relationships with those studied”, and informs us whether or not to accept knowledge (Denzin et al., 2006, p. 776). The labelling of some research as evidence-based implies some research fails to provide evidence. Evidence included a formal framework of school records and documents, interview information, (formal transcribing and informal note taking) and information from the brief survey. Other forms of observation and collection of artefacts provide research evidence, but given the data collection methods used in this project, were not applicable (Bill, 2012).
Effective interview techniques and engagement in professional dialogue with principals combined with my experience as teacher, SENCO, and mother of two visually impaired children provided subjective evidence (Corbin & Strauss, 2008; Maykut & Morehouse, 2001; Mutch, 2005). Gillham (2010) claims that different kinds of data sources reporting on the same issues commonly yield contradictory or discrepant results which may complicate the establishment of evidence. I am mindful that education is an activity that is undertaken for public good. Therefore,
67
despite Kvale’s (1996) suggestions that semi structured interviews and other qualitative methods do not necessarily lend themselves to triangulation, I have attempted to ensure the validity of the data by drawing on multiple sources. These include text from interviews transcripts which clearly represented key findings, evidence gathered from national and international literature, as well as MOE and school board reports and surveys. Note taking recorded additional information during interviews such as body language and facial and hand gestures, adding meaning to the data gathering (Bell, 2010; Cohen et al., 2007; Stake, 2008; Yin, 2003). This report revealed strengths and weaknesses of the research,
explored and clarified ambiguous findings and either qualified or contradicted findings, minimizing the chance of shutting out evidence due to theoretical notions (Bell, 2010; Ryan & Hood, 2006).
I was aware that that the role of power relationships in constructing knowledge claims could influence findings, rendering some evidence invalid (Cohen et al., 2000; Lather, 2006; Moss, 1996). In addition, data presented by respondents as evidence for meeting the learning requirements of students with SEN, was often gathered through standardized tests which may fail to accurately assess student progress. Gauging assessment of a student’s achievement against National Standards relies on teacher overall judgment which could vary between teachers, resulting in evidence neither valid nor accurate when assessing whether their learning needs are being met (Nespor, 2006).