Validity and reliability

The view held by Taylor, Gibbs and Lewins (2005) is that validity refers to the idea that the account truly reflects what actually happened, or put simply that it is accurate.

According to Taylor, Gibbs and Lewins (2005) reliability means that the results of the analysis would also be obtained if different researchers repeated the research and analysis on another occasion. The respondents or participants involved may be

170

different from those in the original research though they will be similar and be doing similar things.

As researchers it is important to persuade audiences that the findings of enquiries are worth taking account of. Consideration also needs to be given to what it is that makes the study believable or trustworthy and the kinds of arguments that have been used. Validity is concerned with whether the findings are really what they appear to be about. Whereas generalisation refers to the extent to which the findings of the enquiry are more generally applicable outside the specifics of the situations studied.

It is important for example, to establish trustworthiness in fixed design research and ask questions such as: Have we done a good, thorough and honest job? We must also consider whether we have tried to explore, describe or explain in an open and unbiased way or whether we are more concerned with delivering the required answers or selecting the evidence to support a case. If these questions cannot be answered then in effect our research is worthless. Validity refers to the accuracy of a result. We need to ask question as to whether or not the findings really correspond to or adequately capture the actual state of affairs and whether any relationships established in the findings are true or due to the effect of something else. ‗Generalisability‘ refers to the extent to which the findings of the enquiry are more generally applicable; for example, in other contexts, situations or times, or to persons other than those directly involved.

Robson (2003) says: Reliability is the consistency or stability of a measure. For example if we were to repeat the exercise, whether we would get the same results. Reliability is the stability or consistency with which we measure something; for example, how are we going to assess educational achievement? It might include a formal achievement test administered at the end of the primary stage of schooling. Unless a measure is reliable, it cannot be valid and may not be sufficient to ensure validity. Unreliability may have various causes one of which is participant error. For example, pupils‘ performance might fluctuate widely from occasion to occasion on a more or less random basis. This could be due to tiredness from too many late nights which could produce changes for different times of the day, or pre menstrual tension

171

in girls or hay fever across genders. There are strategies which can be used to ensure that these kinds of fluctuations do not influence the findings, for example not testing participants during hay fever seasonal times. The actions of the participants can also affect validity i.e. pupils may seek to please the teacher knowing the importance of good results.

In contrast, with construct validity, one looks for what seems reasonable, (fact validity) or looks at possible links between scores on a test and then measure pupils actual educational achievement in their later life.

As situations change and because students move on, the most reliable form of validation used for this study was to use a teacher colleague, Dr. Allen to cross check the questions to the responses given by students in the study. This individual was chosen because he has worked closely with me as a teacher and had knowledge of what the work was about, the process I had gone through and the instruments used in the study; therefore his validation confirmed the reliability of the study. The issues relating to validity in the study of African-Caribbean boys and their teachers are discussed in Chapter Five.

The view held by Yin (2009) is that external validity is concerned with the ability to apply the findings in general. This means whether the results are generalisable beyond the immediate case. Validity refers to the quality of measurement. The view is that by using the same methodology repetitively the results of the study are the same and thus the research instrument is considered to be reliable and therefore consistency or repeatability can be maintained. (Bassey, 1999; Lincoln & Guba, 1985, Mishler, 2000). For example Lincoln and Guba (1985) argues that trustworthiness is distinct from the standard experimental precedent of trying to show validity, soundness and significance and that the aim of trustworthiness is to support the argument that findings of the study are worth paying attention to. Mishler (2000) asserts that the idea of discovering truth through measures of reliability and validity is replaced by the idea of trustworthiness. In Chapter Six I discuss validity in relation to the study of African-Caribbean boys.

172

It is important to decide how to test or maximise the validity, reliability and trustworthiness of a study. Bassey (1999) suggests that the researcher asks themselves eight questions as a checklist to demonstrate trustworthiness of the research

1) Has there been prolonged relative engagement with the data source.

In this study this is so, the study was conducted with groups of year 10, 11, 12 and 13 students, their teachers and the headteacher over a period of one year. 2) Has there been persistent observation of emerging issues?

– in this study observation was not used as a method of collecting data. However the emerging issues in this study is related to policies, racism, underachievement and gender gap.

3) Has raw data been adequately checked with their sources?

The boys and their teacher had the opportunity to check the interview transcripts to verify them.

4) Has there been sufficient triangulation of raw data leading to analytical statements?

The issue of triangulation was discussed earlier in this section in 4.44. 5) Has the working hypothesis, or evaluation or emerging story been

systematically tested against the analytical statement?

The working hypothesis and emerging statements was tested against the responses/answers of the respondents.

6) Has a critical friend thoroughly tried to challenge the findings?

I have discussed the contribution of my critical friend in Chapter Six and in the appendices.

Also both my supervisors continually reviewed the research at different stages and provided constructive criticism.

7) Is the account of the research sufficiently detailed to give the reader confidence in the findings? In Chapter Five and Six, I have provided a full account of the study to give readers confidence in the study.

173

Taped interviews, transcripts, questionnaire analysis and summaries are available.