Validity

Validity for Ritchie and Lewis (2003) lies in the correctness or precision of the research findings. Basit (2010, p. 64) reminds us that “no research is totally valid as threats to validity cannot be totally removed” but “validity is a vital element of effective research because if a particular study is invalid, then it is worthless”

128 (ibid, 2010, p. 64). Validity as applied to qualitative research findings is often

perceived through the challenge and defence of believable data whilst

simultaneously considering whether the data is plausible, credible and reliable. Contrastingly, validity is frequently achieved in quantitative research through the use of mathematics or statistics where results are often definite and provable in nature. Qualitative research validity tends to be less statistical and relies more on data analysis that produces rich, deep narratives. Some qualitative researchers believe that the concept of validity is not synonymous to validity as understood by quantitative researchers and therefore a different perspective or approach should be employed. Those researchers arguing from this standpoint still make the case that every effort should be made to ensure validity so that the results of

qualitative research are to be believed. Other researchers do not make any distinction between achieving validity via different methodological approaches, for example Hammersley (1987, p. 69) argues that “an account is valid or true if it represents accurately those features of the phenomena, that it is intended to describe, explain or theorise”.

As an initial teacher trainer of the newly qualified teachers, some of whom were selected for involvement in this research, I openly acknowledge that I may have some preconceptions, for example about participant’s mathematical subject knowledge and even value judgements about their pedagogical approaches. Consequently the potential for bias is acknowledged and an open, honest, sharing approach already employed during their initial teacher training was continued to help guard against such influences. Shipman (1997) argues that total objectivity in social research may not be achievable, but by providing robust evidence trails of the research methodology, and also of the procedures and processes that underpin the analysis, then a degree of rigour can be achieved. In an effort to alleviate bias I needed to consider an open, honest and transparent approach in this study which could easily be evidenced through the design, discussions and research with colleagues.

Remarkably Lather (1986) and Morley (1991) argue for the rejection of

researcher neutrality, as most research is often undertaken by stakeholders with an interest in resolving a problem. They also argue that neither qualitative nor quantitative methodologies are sufficient epistemological structures to guard

129 against bias. A praxis paradigm where knowledge is derived from practice, and

practice informed by knowledge, in an ongoing process, is a basis of research and this might be an appropriate approach for openness, honesty and

transparency. I agree with Lather (1986) and Morley (1991) that total neutrality and total impartiality are almost impossible to achieve, especially where

researchers have both a professional and emotional involvement with the participants, never-the-less they should be goals that are to be achieved by the researcher.

I further acknowledge that demonstrating credibility and validity when carrying out research in a single school and with a single group of teachers that I had trained would be an important consideration. I would further argue that external validity as a cause-effect relationship between the independent and dependent variables is more normally applied to quantitative research, whereas internal credibility as a means of demonstrating believability is more appropriate for my qualitative study. However, Greenwood and Levin (2006, p. 80) argue that knowledge from

research has an “internal credibility” when generated collaboratively, but equally they warn that external credibility is required to convince “someone who did not participate in the inquiry that the results are believable” (ibid, p. 81). Such internal credibility, I would argue, emerges and is created through reflection and

discourse when “participants and the researchers negotiate the meanings created by their experiences during the research process” (ibid, p. 114). The nature of this qualitative case study is well suited to discourse and reflection from all involved. External credibility leads to Guba and Lincoln’s (1984) view that for the findings to be trustworthy they should also be transferable and applicable in other contexts.

Therefore, in this study I take the view of Lecompte and Goets (1982) that the two forms of validity most applicable to qualitative research are

1 internal validity, the degree of connection between observed investigations and the theoretical ideas

and

130

3.3.5.1 Internal validity

A researcher’s views, bias and values have an impact on the whole research process and they need to be acknowledged and carefully considered. It is also important, as a researcher, to declare professional experiences and

preconceptions of classrooms and the relationships with other participants that may influence classroom interpretations and consequently the internal validity of the research. Such experiences and preconceptions were outlined in chapter 1. However, I would argue that my professional experiences, classroom

conceptions together with strong professional relationships enabled me to better understand the classroom actions that I observe and allow me to interpret the views of participating teachers. Research needs to reflect, capture and critically analyse the perceptions of participants as “one of the assumptions of qualitative research is that reality is holistic, multidimensional, and ever-changing” (Merriam, 1998, p. 202). Several factors helped to increase the internal validity of this study including:

 long-term observations and a data gathering phase;

 the use of a participatory research model

 and, the triangulation of multiple methods of data collection as a method to confirm the findings

(Merriam, 1998).

Johnson (1997) and Newman and Benz (1998) support these three strategies as a means of demonstrating internal validity and suggest that researchers should also consider reflexivity (reflecting on ourselves and our involvement in the research. This is what May (2001, p. 44) calls “a consideration of the practice of research, our place within it” as the bidirectional relationship between cause and effect as well as to guard against their values creeping into the interpretation of data. Researchers should critically examine their interpretations to detect any potential bias which may influence the conclusions made about the data. Whilst it is recognised and acknowledged that conclusions can never be a hundred

percent value-free, a researcher should strive to achieve high levels of honesty, transparency and openness in order to convince others of the merit of their study. The keeping of an audit trail consisting of accurate and fully documented data records allows others to have a degree of confidence in the evidence, results, and conclusions thereby providing an opportunity for challenge as well as validation of the interpretation of data.

131 My study can be said to be valid because I gathered data from different sources using different methods over an extended period of time. I planned to fully and accurately document the data gathered and record the date of all interactions. However, a weakness might be perceived in that with only one school setting used to collect the classroom based data, but two different classrooms were sampled. In terms of internal validity, a real strength of the study comes from the participation by professionals in the fieldwork, and their intimate knowledge of the subject material, the pupils and their full engagement during the whole process. Carrying out a range of checks to see if what I observe and record, equates to what the teachers observe and believe, is a real strength in demonstrating internal validity, especially given the participatory nature of the professionals involved. Taking data from multiple sources using a range of methods such as questionnaires, classroom observations, interviews, teacher lesson plans and outcomes from specifically designed teaching materials has both grounded and helped triangulate the results.

3.3.5.2 External validity

As stated earlier the external validity of the research is concerned with the generalizability of the findings from one context to another (Merriam, 1998). While notions of external validity in qualitative research are debatable, I would argue from my experiences that it is possible to enhance the generalizability of a study. Guba and Lincoln (1994) suggest two key criteria for assessing external validity in qualitative studies, credibility and transferability. Credibility viewed in terms of whether the findings are believable and transferability if the outcomes can be applied to other contexts. Silverman (2013) claims that external validity in qualitative research can be achieved through triangulation by comparing different kinds of data from quantitative and qualitative methods (questionnaires and classroom observations) to ascertain if they substantiate each other. Newman and Benz (1998) argue that findings can be externally verified if the study results are applied to other studies, or if in-depth description enables the researcher to generalise findings. Lincoln and Guba (1985) warn qualitative researchers to guard against making claims that the study can be generalised or transferred as this is not a necessity, these claims should be left to the reader. This study takes the combined views of Guba and Lincoln (1994) and Silverman (2003) and aims

132 to achieve external validity by means of triangulation to demonstrate

transferability and believability.

3.3.6 Reliability

Lincoln and Guba (1985) remind us that for qualitative researchers issues of reliability relate to dependability or consistency. Lincoln and Guba (1985) imply that research results need to make sense to outsiders and that looking at the same datasets would allow them to arrive at the same conclusions. In this study, reliability is increased by making explicit the theoretical underpinnings of the study, how the researcher is positioned, who the participants are and how they are selected, the context in which the data was collected, and how the

triangulation of data was achieved (Merriam, 1998).

Reliability has its roots in quantitative research as a concept for testing or the evaluation of the research, however, this idea is now often applied to all research paradigms. The single most important assessment of a qualitative study is

quality. Eisner (1991, p. 58) reminds us that reliable qualitative studies help us “understand a situation that would otherwise be enigmatic or confusing” with the intention of explaining and generating understanding. The difference between reliability as a testing mechanism in quantitative research and as a means of generating understanding leads Stenbacka (2001, p. 552) to argue that “the concept of reliability is even misleading in qualitative research”. Patton (2001) takes the opposing viewpoint reminding researchers that reliability as a measure of quality is a factor that should be carefully considered when designing any study and analysing the results. Joppe (2000) defines reliability as:

The extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable (p. 1) .

Different paradigms should judge quality by their own terms (Healy and Perry, 2000). Lincoln and Guba (1985) suggest that reliability as a fundamental prerequisite for quality and closely correspond to credibility, dependability or transferability in qualitative paradigms. Clont (1992) and Seale (1999) support the concept of dependability as a means of defining reliability and Lincoln and Guba (1985, p. 316) state that: "Since there can be no validity without reliability, a

133 demonstration of the former [validity] is sufficient to establish the latter

[reliability]". Therefore, according to Guba and Lincoln (1985), the demonstration of a study being reliable through the use of appropriate methods, such as

triangulation, may help to ensure or even guarantee that the research is valid. The use of triangulation as a concept is justified as a means of confirming, demonstrating completeness and indicating the trustworthiness of data sets (Denzin, 1973; Jick, 1983; Bryman, 2008). Approaching research questions from differing viewpoints, using a range of methods, creates a deeper, fuller

understanding as well as alternate interpretations. The use of triangulation is therefore defensible as a concept and as a method to underpin the exploration of the complex situations, as in this study, to check for validity and reliability

(Bryman, 2008). Whilst triangulation is often used to support reliability, for consistent deep understanding we do need to be mindful of the temptation “to make inconsistent data sets artificially compatible in order to produce a more coherent account” (Arksey and Knight, 1999, p. 25). Triangulation cannot therefore be seen and a panacea to ensure reliability as it is possible to manipulate the data in such a way to achieve the result required.

The inherent “risk”, as an experienced professional, of prejudging the outcomes whilst undertaking the role of a researcher needed to be carefully thought through. Eliminating all risk is probably not feasible, but piloting all data

instruments that were to be used in the research, is seen as an important step in the thinking and reflective processes and as a risk avoidance measure.

134

3.3.7 Triangulation

The concept of triangulation in research refers to the use of multiple, different approaches to generate a better understanding of a particular theory or phenomenon that is being studied (Burton and Obel, 2011). Fusch and Ness (2015) argue that “the application of triangulation (multiple sources of data) will go a long way towards enhancing the reliability of results” (p. 1411). Triangulation of data is therefore a critical factor for the reliability, trustworthiness,

transparency, decision making and assumptions that are made (O’Brien et al., 2014; Arriaza et al., 2015), and this includes how the interpretations of data have been checked, cross-checked and any inferences that are made (Avenier and Thomas, 2015). Researchers using a qualitative methodology often use multiple data collection methods (triangulation) as it is assumed “that the use of a single method can never adequately shed light on a phenomenon” (Abrar et al., 2017, p. 17). Creswell (2009) reminds us that triangulation is also an important

technique for a researcher when trying to ensure reliability and validity of the data. From a qualitative methodological standpoint data collection and analysis is unavoidably subjective in nature. So to enhance credibility and trustworthiness of the analytical processes when identifying meaningful interpretations of patterns “located in the subjective interpretation of data” (Levitt, 2015, p. 456) triangulation is a necessity.

The use of triangulation by methods such as interviews, lesson video

observation, questionnaires and inspection of artefacts were all employed in this research so as to achieve a “richer, deeper, more robust, and also more well- developed data and to strengthen [their] validity and reliability” (Abrar et al., 2017, p. 19). This approach was important as Santiago-Delefosse et al. (2016) argue that data from different participants and at different times as well as using multiple data sources enhances reliability.

Therefore a primary purpose of using triangulation in this study was to eliminate or reduce biases and increase the reliability and validity of the data collected for the study. However, I also wanted to increase the comprehensiveness and confidence in the data to provide richness which would help with the

understanding of the phenomenon under study. Twining et al. (2017) remind us that giving participants the opportunity to comment and correct interview

135 manuscripts is also a valuable method of achieving credibility and trustworthiness with O’Brien et al. (2014) adding that emerging findings should also be shared for checking. Both of these suggestions were used in this research. The concept and use of triangulation of the data through the use multiple data collection methods enriches this study and the findings. This research therefore used two forms of triangulation – methodological triangulation (multiple methods) and respondent triangulation (different types of participants).

3.3.8 Ethics

Ethics in this research involved far more than a mere compliance with ethical codes and guidelines. There is an increasing recognition of the need to view ethics as a process throughout the entire life span of a project (Cutcliffe and Ramcharan, 2002; Guillemin and Gillam, 2004) rather than the process of mere compliance once codes have been agreed.

Groundwater-Smith and Mockler (2007) suggests that a set of ethical guidelines for this type of project should involve more than a mere observation of ethical codes and processes but include:

1. transparency and accountability to the community of learners; 2. collaborative approach to the work;

3. transformative intentions.

The collaborative nature of the project, its interactions with teachers and pupils and its transformative intentions permeate this thesis. The research proposal would be need to scrutinized through the University’s ethical approval system (appendix 37). There were a number of ethical issues that need to be considered:

1. All the children that were to be involved were under 16 years old. 2. The use of both audio and video recording for capturing data with both

children and adults.

3. The disruption to the normal classes.

4. The use of data collected from PGCE cohorts via the questionnaire 5. The use of data collected from mentors via the questionnaire 6. The use of words spoken by teachers in the semi-structured

interviews.

7. Finally the privacy and confidentiality of interviews, lesson plans and informal conversations with the study school staff would need to be explained and protocols agreed.

Consent would be needed to be requested and received from the head teacher of the participating school both verbally and via email before the study can take

136 place. However, because of the nature of the study informed consent from all of the participants (teachers and pupils) would also be needed. As (O’Leary, 2017) reminds us that consent to take part in a research study can only be given if a full understanding of the nature of their participation is explained. Verbal and written explanations of the study describing the level of participant involvement are two methods often used. In this research separate information letters and consent forms were given to the participant teachers, and the parents / carers of the pupils in the selected classes (appendix 36). These letters and consent forms outline the time commitment, the types of activities involved in the research and were endorsed by the school (head teacher) and provided to all concerned on official school headed paper. Informal consent (ie verbal rather than written) from PGCE students and their mentors was sought prior to the completion of the survey questionnaire by these two groups.

I requested and received written consent from all of the teachers who worked with me on the project. All parents / carers of the pupils involved in the two classes were contacted in writing to request their consent for their children’s involvement in the research lessons, the videoing of the lessons and in the subsequent interviews. Completed consent forms were received from every parent. I considered written consent letters for pupils to fill in, but prior to the research I had spent considerable time in class talking with, and explaining to, the two classes what the research was about and their right to withdraw. As another level of safety pupils were given the opportunity to speak to their regular teacher if they wished to withdraw from the research. I did consult with the teachers and the senior leadership as to their views as to whether a signed consent form from pupils would add any additional ethical value or robustness to the research (eventually we decided against a signed form from pupils).

The pupil’s right to withdraw from research was explored by Doyle (2007), citing pupil vulnerability as a major concern, with Campbell and Groundwater-Smith (2007) highlighting concerns about practitioner researchers being accountable to the pupils. So, on the morning of the lessons I explained to each class that they can ask for the filming to stop if they did not want to be filmed, additionally even though parental consent had been given they could still decide not to participate. The children were told how the video would be used, and they would be invited to