Reliability, validity and credibility

The use of reliability and validity is common in both quantitative and qualitative research paradigms. Also, the use of them has been seen as central issues in all scientific measurements (Golafshani, 2003; Neuman, 2004). Reliability and validity are important in social research as constructing social research is abstract and unobservable. Thus, “Reliability and validity are ideas that help to establish the truthfulness, credibility, or believability of findings” (Neuman, 2011, p. 208). Reliability means that “the information provided by indicators (e.g., a questionnaire) does not vary as a result of characteristics of the indicator, instrument, or measurement device itself” (Neuman, 1997, p. 138). It suggests that the research result should not change when the research project is repeated or recurs under similar situations (Neuman, 2003). Validity addresses “the question of how well the social reality being measured through research matches with the constructs researchers use to understand it” (Neuman, 2003, p. 179). Compared with reliability, validity is more difficult to achieve, as the researcher “cannot have absolute confidence about validity” (Neuman, 2011, p. 211). The reason the researcher cannot achieve absolute validity is that constructing in social theory is abstract while indicators refer to concrete observation. According to Bohrnstedt (2002), the validity is part of a dynamic process and as it grows, the evidence will be accumulated over time. Without it, the measurement becomes meaningless. Thus, validity is difficult to obtain but it is essential while undertaking a research. This study uses both quantitative and qualitative methods to collect and analyze data, so the following section discusses how reliability and validity were achieved at two stages.

This study was considered as both reliable and valid. According to Neuman (2011), there are few chances of obtaining perfect reliability. Thus, some principles are provided by him for the researchers to follow to increase the credibility in the research. This research adopted these principles to increase the credibility, which will be introduced later in this chapter. Meanwhile, the researcher reviewed literature and found similar measures that had been used in the previous studies. In this way, the research measurement was more credible although it took more time and effort (Neuman, 2011). Moreover, it is proposed that one of the most important measurements to test validity is “Statistic Validity” (Neuman, 2011, p. 217) , which suggests the researcher choose a correct statistical procedure to fully meet the assumptions. This method will be explained in detail in the following chapter.

This research is also regarded as credible. Credibility has been examined across various fields, ranging from communication, psychology, and science to

86

interdisciplinary efforts in human-computer interaction (Rieh & Danielson, 2007). This research focuses on the information source (text) in the questionnaire and communication in the semi-structured interview. All the participants involved in the research were volunteers and felt comfortable to participate in it. They had been informed of the whole process of the research before they took part in this research as all the information sheets related to the research had been in the reception desk of the International Office of the subject university. Therefore, their opinions were regarded as credible when they expressed what they thought and believed in the questionnaire and in the interviews.

4.6.1 Reliability and validity at the quantitative stage

The researcher adopted a questionnaire as a tool for collecting data at the quantitative stage. At this stage, the researcher made efforts to ensure the result is replicable with regards to reliability, and the means of measurement are accurate with regards to validity (Golafshani, 2003). Thus, a clear instrument design, appropriate analysis strategies, and pilot tests are essential principles to increase reliability followed by the researcher (Neuman, 2004).

Firstly, the questionnaire items were made clearly and briefly to the point in order to avoid ambiguousness of the result. The questionnaire was designed according to the pre-designed outline that addressed four parts clearly: background information, English learning and teaching in high schools and in the university, students’ preferred English learning and teaching, and an open-ended question, which was coordinated with the research aim. Also, the scale of each variable was designed appropriately with numbers so that the participants could understand these indicators.

Secondly, the multiple sources of responses were gained to ensure its validity and reliability. The outline and the draft of the questionnaire were first viewed by the chief investigator and colleagues in the department. Later, they were passed to the academic staff and peers to review. Also, the students shared their opinions towards them. Opinions from these diverse perspectives were tested and compared, which could avoid bias and prejudice. In this way, the reliability and validity were ensured in the study.

Thirdly, the pilot study was regarded as an essential process to ensure the reliability and validity at this stage. The initial version of the questionnaire was tested with 100 university students. Then a meeting was arranged by the

researcher and six other researchers to discuss the questions that emerged in the pilot study. As it gained valuable recommendations and suggestions from the six researchers, the questionnaire items and interview questions were adjusted in the final version.

4.6.2 Reliability and validity at the qualitative stage

Viewing the literature, it provides different perceptions and definitions of reliability and validity in correspondence with how they are considered in qualitative research (Angen, 2000; Creswell, 2009; Eisner, 1991; Lather, 1993; Lincoln & Guba, 1985). Although the researchers’ understandings and the terms used are different, the reliability is one of the most important criteria for measuring a qualitative research, and the idea is most often used in all kinds of research (Golafshani, 2003). According to Golafshani (2003), reliability is a concept related to the quality as an idea to test a qualitative research. Also, Eisner (1991) states that a good qualitative research can assist the readers in understanding the situation. Therefore, reliability is an important factor to ensure that this study is a good qualitative research.

Although the understanding of reliability is various, “to ensure reliability in qualitative research, examination of trustworthiness is crucial” (Golafshani, 2003, p. 601). As the instrument is vital to ensure the trustworthiness in this study, the researcher designed the interview questions according to the outline. After the pilot study, she gained different perspectives from the students, academic staff, and the other researchers in the subject university in order to increase the trustworthiness at this stage. Also, the suggestions that involved the chief investigator and researcher’s colleagues were seriously taken. These various dimensions of feedback helped the reliability that has been achieved in the research. Based on these suggestions and recommendations, the researcher adjusted some questions in the final version of interview questions. It is believed that if the semi-interview is conducted again, the results would not be significantly different. Therefore, the qualitative stage is considered as reliable in this study.

The concept of validity is various in qualitative studies. This concept is not a fixed but “rather a contingent construct, inescapably grounded in the processes and intensions of particular research methodologies and projects” (Winter, 2000, p. 1). Although the term has not reached consensus, the central issue the validity concerns is whether the research findings can accurately reflect the phenomenon under investigation and the interpretations derived (Henn, Weinstein, & Foard,

88

2006). Several validation strategies have been proposed by Creswell (2009) and it is suggested at least two of them should be engaged in this research. At this stage, the researcher used triangulation and peer review strategies proposed in the literature (Creswell, 2007, 2009). The researcher asked an open-ended question in the questionnaire draft from multiple dimensions in order to obtain the external check of the research process. Using this review can further keep the research valid and provide the researcher with an opportunity to express her true feelings (Lincoln & Guba, 1985).