Reliability and validity of this study

As mentioned in the introduction to this chapter, this study has been conducted whilst I have been a practising teacher at the FEC, rather than an independent researcher. This can have advantages, as suggested by proponents such as McNiff (1993), McNiff and Whitehead (2007), who support the concept of educational research being conducted by practising teachers. It has several drawbacks, including the fact that, at times, I felt somewhat restricted in making criticisms of the institution or the wider political organisation of Further Education in the UK.

139

As a teacher at the FEC, I am in a position of trust, both directly due to my role as teacher or indirectly due to the relationships built up with students over a period of time. I have therefore had the opportunity to find out information that may be difficult for an independent researcher to determine. Hargreaves suggests that being a participant-observer eliminates potential resistance towards the research and ‘permits the investigator to experience and observe the group’s norms, values, conflicts and pressures’ (Hargreaves, 1967, p.193).

According to Cohen: ‘Quantitative research possesses a measure of standard error which is inbuilt and which has to be acknowledged. In qualitative data, the subjective of respondents, their opinions, attitudes and perspectives together contribute to a degree of bias’ (Cohen, 2007, p.133). This is borne out by the fact that in the quantitative analyses of examinations (Section 4 of the portfolio), I used statistical methods and showed the accuracies with error bars on the graphs. With qualitative methods, it is more difficult, in fact impossible, to present a ‘percentage error’ calculation, so qualitative methodologies require a different approach.

Stake (1995) attacks the criticisms of qualitative methodologies in a very direct manner: ‘qualitative study has everything wrong with it that its detractors claim’. He explains that this type of methodology is subjective, results in ‘little advancement of social practice’, ethical risks are higher, as well as costly in terms of time and money (Stake, 1995, p.45). By recognising these problems, qualitative researchers have developed respectable techniques for validation and an ethical obligation to minimise misrepresentation and misunderstandings. Stake suggests the use of

140

triangulation in order to check that what we are observing and reporting has the same meaning when found under different circumstances (Stake, 1995, p.113).

Hitchcock and Hughes discuss triangulation in more detail, explaining that there are four main methods for triangulation. Data triangulation is when the investigation is repeated over a period of time with the same source, Investigator triangulation is when there is more than one observer involved, Theory triangulation is when there is more than one approach to the problem and finally, Methodological triangulation when there is more than one source of obtaining information (Hitchcock and Hughes, 1995, p.324).

From these four types of triangulation, methodological triangulation was employed within my study, as there were several occasions where I could cross check information by using different sources. For example, some of the responses from the focus group with students in Section 1 of the portfolio could be checked by confirming with other sources. The students provided factual information on the number of science teachers at their respective schools which was then checked with staff from these schools as well as with other students who were unconnected with the focus group. The consultant from the Institute of Physics worked with our partner schools and she further confirmed that the information the students provided was correct.

Hitchcock and Hughes suggest that one method for triangulating information obtained from interviews is to provide a transcript of the interview for the respondent to check in order to ensure that it is a correct representation. They suggest that a second interview may be useful to clarify

141

any further issues or misconceptions. (Hitchcock and Hughes, 1995, p.182). When conducting the case study in Section 10, I provided the opportunity for the respondents to check and amend, if necessary, their responses to each question.

When conducting surveys and interviews, care was taken to explain to students the nature of the research. As the questions related to the topics and styles of teaching and learning they preferred, I have no reason to doubt their responses to these questions. When asking questions about family background, I interviewed each student separately. As this latter investigation was conducted with second year students, I had built up trust with them over an eighteen month period and met with most of the parents. I have discussed in Section 7 of the portfolio that due to the small number of students in this latter survey, it cannot be generalised and simply presents a representative sample of A level students at the FEC at that time.

Educational Research texts suggest that peer examination is another method of ensuring validity, by encouraging others to read, review and comment upon findings (Hitchcock and Hughes, 1995, p.325). This is borne out by the fact that each of the ten reports were reviewed by internal colleagues and managers at the FEC, as well as external professionals. I have included the original feedback forms in the portfolio, which show the suggestions, comments and opinions of the reviewers. I believe that adopting this process of peer review has helped to shape and improve the studies within the portfolio, but increased the validity of the body of work.

Cohen discusses external validity as the extent that results can be generalised to the wider population (Cohen, 2007, p.136). Clearly the

142

findings from this study do not extend to those of all young people who study physics in post-compulsory education, only those who study within further education colleges. At this point in time, this is the only option for young people in this area unless they select faith or independent schools.

The fact that I was studying for a course at university in parallel with the students was surprisingly well received. Whether it was due to the fact that I could share my learning experiences with my students, or that I was striving to be a better teacher, the students were always very helpful and eager to participate.