Reliability and Validity

In general, research is mostly concerned with two requirements, reliability and validity (Cohen et al, 2011). They are important keys to effective research. Validity refers to the integrity of the conclusions generated from the research (Bryman, 2008). It means the accuracy of the research findings, and to what extent the research findings accurately represent facts (ibid).

Reliability refers to the consistency of research findings (Cohen et al, 2011). Cohen et al (2011, p.199) suggests that reliability concerns the degree of repeatability of the research results:

“For research to be reliable it must demonstrate that if it were to be carried out on a similar group of respondents in a similar context (however defined), then similar results would be found.”

Cohen et al (2011) points out that reliability concerns whether similar results would be found with different samples or different methods in a similar context.

In this study, regarding the stage of data collection through semi-structured interviews and questionnaire surveys, the key issue was to ensure the honesty of the participants. The participants were encouraged to provide true information. This was particularly important when I was interviewing the government officers. The officers might think that some of the topics are sensitive, and feel uncertain about the consequences of telling the truth. They might feel insecure in providing information

107

to a researcher studying abroad and might be afraid that the interviews would harm their political career. For this reason, government officers were likely to use ‘official lines’ when they were talking. If they used a lot of ‘official lines’ and avoided telling the truth, the officers might provide very little substantial content in the interview. In this case, the validity of the data would be undermined.

Moreover, the participants, either government officers or students, may have wanted to present themselves in the best light, either consciously or unconsciously. This was the main reason for students distorting facts in the interviews and questionnaire. This could harm the validity of the data.

In order to ensure validity and reliability, three measures have been taken. The first measure was to inform the participants that the data would be used for academic purpose only and not for any other purpose. None of the participants’ personal information was revealed to the public and it was confirmed that the participants’ identities would be kept anonymous if the research was published. This was to dispel any fear or doubt that participants might have and to improve the probability of the participants presenting the truth. It is believed that the anonymous questionnaire survey and interview encouraged the honesty of the participants.

The second measure was to build a friendly relationship with the interviewees. During the course of government officer interviews, I talked with the participants about some facts that interviewers already knew from other officers, especially the facts known from their superiors. This was to convey such a signal to the interviewees that I was aware of the facts and obtained permission from their superiors to ask them questions. Thus, interviews were more confidential and encouraged the interviewees to discuss what they genuinely knew. During the course of the student interview, the interviews were started in a flexible chatting way. I kept encouraging students to present the truth in a friendly way. For example, “Just say what you want to say, it does not matter if it was wrong or right”. This was effective in encouraging students to be honest in the interviews.

The third measure was to ask for the teacher’s help. During the course of the student questionnaire survey and student interview, I asked the teacher to help me to invite the students to take part in the research. Before each questionnaire survey, the

108

teacher asked the students to carefully answer the questionnaire and be faithful in all questions. The students who took part in the student interviews were asked by their teachers to answer the questions as honestly as possible.

Regarding the data analysis in this research, triangulation was used to improve validity and reliability of the research. According to Cohen et al (2011), triangulation is a powerful way of demonstrating validity, and triangulation refers to the use of two or more methods of data collection within a research. The validity of the research findings can be improved by presenting more evidence with other methods to support them. Firstly, triangulation of data sources was used. The findings based on qualitative data were triangulated with findings based on quantitative data. For example, in terms of the educational equity issue in undergraduate admission, the data from interviews was triangulated with statistical data. Secondly, triangulation of interviewees was used. For a specific claim or point, the number of interviewees who mentioned the claim or point was counted. More interviewees mentioned, stronger the claim or point was. For example, I asked the government officers and students about the application procedure of ‘Offsite NCEE’. This enabled me to cross check their answers and ensure the validity of the data.