Validity and Reliability

Validity and reliability are concerned as two key issues in research. Validity is concerned with whether the research methods and data are “right” which as Denscombe (2003) argued “the idea of validity hinges around the extent to which research data and the methods for obtaining the data are deemed accurate, honest and on target” (p.301). Wray (2006) echoed Denscombe, when he defined validity as:

“In general, validity is an indication of how sound a piece of research is. More specifically, validity applies to both the design and the methods of research. Validity in data collection means that your findings truly represent the phenomenon you are claiming to measure. Valid claims are solid claims so validity is one of the main concerns in research.”

In order to achieve this goal, Cohen et al(2007) suggested that validity in qualitative data might be addressed through the honesty, depth, richness and scope of the data achieved. Validity in quantitative data might be achieved through careful sampling, appropriate instrumentation and appropriate statistical treatment of the data collected (Cohen, Manion et al. 2007). Cohen et al(2000) indicated that the most practical way to achieve greater validity is to “minimise the amount of bias” as much as possible (p.105), however, it is impossible to achieve perfect validity and objectivity, as each person’s life experience and educational background would affect his or her point of view and behaviors during the research design and data collection process. In order to reduce bias and achieve greater validity, it is important for researchers to state clearly the potential bias, and be aware of it when conducting research (Wu 2010). In addition, researchers should aspire to as high a degree of validity as possible and consider other people’s perspectives as valid as their own (Hitchcock and Hughes 1995).

128

Compared to validity, reliability is reliant on data gathering and analysis technologies. According to Cohen et al(2007), reliability is a synonym for consistency over time, over instruments and over groups of respondents. Wray (2006) echoing Cohen et al

(2007), suggested that:

“Reliability has to do with the quality of measurement. In research, the term reliability means "repeatability" or "consistency". A measure is considered reliable if it would give us the same result over and over again (assuming that what we are measuring is not changing!).”

In qualitative research, reliability can be viewed as a match between what researchers record as data and what actually occurs in the natural setting that is being researched, i.e. a degree of accuracy and comprehensiveness of coverage (Bogdan & Biklen, 1992, cited in Cohen et al.2000). Hence, researchers have to devise the research procedures as accurately as possible eliminating possible ambiguities. While for quantitative research, if the same research tool is used with the same sample, the result should be the same (Bogdan & Biklen, 1992, cited in Cohen et al.2000).

The current study was designed with both qualitative and quantitative approaches in mind. A qualitative approach was adopted in this research in order to explore and understand a variety of aspects to schools’ motivations in introducing Mandarin Chinese as a new language, teachers’ teaching experiences abroad, both teachers’ and pupils’ attitudes and perceptions towards teaching and learning Mandarin Chinese. Regarding interview sessions for example, Cohen et al (2007) believed that interviews are the most practical way for the researcher to minimize the amount of bias in order to achieve the greater validity data. They further described the sources of bias that

129

exist within characteristics of the interviewers, respondents and the content of the questions. Silverman (1993) pointed out the importance of ensuring a consistent way for each interviewee to understand the questions in interviews in order to achieve the high degree of reliability of data they supply.

The questionnaire, as a measurement instrument, possesses adequate reliability and validity. In this study it is used to explore pupils’ attitudes and perceptions towards their learning experience of Mandarin. The design of questions included both open- ended and closed questions, single choice and multi-item scales. According to Dornyei (2003) “even in cases where there are no resources and opportunities for elaborate validation exercises, we should strive for a questionnaire that has appropriate and well-documented reliability in at least one aspect: internal consistency” (p.93), he further explained the definition of “internal consistency” as “the homogeneity of the items making up the various multi-item scales within the questionnaire” (p.94) later. Furthermore, in respect of reliability, Dornyei (2003) stated that two other issues need to be ensured in a questionnaire; firstly, the questionnaires would need to have multi-item scales and avoid applying single items; and secondly, the items would need to work together with other homogeneous items which they share the same target. In this study, while gaining valid data from 84 respondents, the data gained from open-ended questions and multi-items scale would meet the requirement of internal consistency. In order to address the second issue, on the one hand when questions were designed I invited a native speaker to read through the questions carefully to prevent some questions not targeting what they were not meant to ask; on the other hand, as with my other research methods, I discussed my work with two supervisors and revised them, when necessary, to ensure that

130

everything would work satisfactorily.

In this study, in respect of reliability, the research design was laid out in plain terms, conducting clear data collection and data analysis in order to minimize bias and subjectivity. During the fieldwork, I refrained from soliciting any answer to prevent the data from being skewed by any form of preconception and after gathering the data, I interpreted data with the assistance of two supervisors. Moreover, for the interview with the teacher of Mandarin who preferred to use Chinese, I took extra care to ensure its accuracy by asking the interview participant to verify both the initial and final transcripts.