Reliability and Validity

Throughout the process of data collection and analysis, I ensured that the research findings and interpretations remained accurate. Validating findings means that I need to determine the accuracy or credibility of the findings through strategies such as

triangulation, member checking and external audit. 1. Triangulation

Qualitative inquirers triangulate among different data sources to enhance the accuracy of a study. Triangulation is the process of corroborating evidence from different individuals (e.g., a vice principal, a teacher and a student), types of data (e.g., observation data and interview data), or methods of data collection (e.g., observation and interview) in descriptions and themes in qualitative research. In this research, I examined each information source and found evidence to support a theme in order to develop a report that is both accurate and credible. For example: teacher B‘s dialogue with students showed a lot of her struggle, such as (See Chapter IV segment six, reference h7):

B: I have to say this, your minds must open. I always feel like I have no choice and don't know what to do. I particularly hope you can discuss in class. However, there are some students are very quiet, I don't know what to do. I especially hope to give more class time to you, let you think, inquiry and discuss, but we have to take

examination, we have no choice. I hope you can think more and inquiry more, even under the current education and examination

122

system, when you learn history, don't just memorize mechanically. After the pre-class quiz, I found out, I believe you yourselves already notice that you forget a lot of stuff. We have to take the exam, thus you must think more in class so that you can remember well.

In interview with her, teacher B revealed to me that the previous semester she asked students to explore, construct and teach the class by themselves, it was student-centered and the teacher was a helper and protector who could help students to prepare the class and add or correct some points. However, the scores of the exam were not very

satisfying and she therefore had to change her teaching method back to a more

traditional approach; despite the fact that she admitted preferring the student-centered approach to teaching. She pointed out that now she changed to exam-orientated

approach again which she does not like but has to. She is still exploring, unlike teacher C whose subject is not an exam subject, she needs to balance the exam and her

educational thoughts, to use her words, ―many times I wish to let students to obtain the knowledge actively, and however, it apparently could not work out with exam. I didn‘t find the solution yet.‖

Thus, both observation data and interview data showed teacher B‘s struggles. Participant‘s opinion or perception of certain topic could provide accurate and credible evidence for this study as well. For example, in interviews with students, they mentioned that ―sometimes teacher‘s action was not consistent with her words‖. Teacher C also mentioned and agreed with this opinion in the interview.

123 2. Member checking

I asked participants (teacher A and teacher B, vice principal-teacher C and students) in this research to check the accuracy of the account. This check involved taking the findings back to the participants and asking them (in writing or in an interview) about the accuracy of the report. In this research, I asked participants about many aspects of the observations, such as whether the description was complete and realistic, if the themes were accurate to include, and if the interpretations were fair and representative. I sent the participants the interview transcripts and part of the report which involved them for checking. They sent the transcripts back to me with comments, corrections and add- on ideas.

3. External audit.

I also obtained the services of an editor outside the study to review different aspects of the research. I asked an editor outside the project to conduct a thorough review of the study and report back in writing the strengths and weaknesses of the project (See Appendix J).

Besides this editor, I asked a lecturer to comment on the finding, she is an associate professor and teaches basic qualitative research and advanced qualitative research in the Faculty of Education; I audited her course of advanced qualitative research. She read through my thesis, and gave comments and suggestions on themes and sub-themes I was working with by that time. For example, she suggested that I refined some of the categories, such as change the category of ―unneeded-answered question‖ to ―self-answered question‖; put categories of ―self-answered question‖, ―talk

124 to self‖, ―discipline‖ and so on to theme of ―monologue‖, and put categories of

―facilitating question‖, ―share experience‖, and so on to theme of ―dialogue‖. Despite the strengths of qualitative analysis, the big problem for qualitative analysis it that there are often ―few agreed-on canons for qualitative data analysis, in the sense of shared ground rules for drawing conclusions and verifying their sturdiness‖ (Miles & Huberman, 1984, p. 16, cited in Patton, 2002). Patton (2002) notes that there are no absolute rules for determining significance, no shortcut of coping with other researchers‘ analytical processes, no straightforward tests that can be applied for reliability and validity. However, he points out that researchers can do their very best with their full intellect to fairly represent the data and communicate what the data reveals given the purpose of the study. When dealing with the data, the researcher must answer four riddles (Alberty, pp. 538-539, cited in Patton, 2002, p. 598):

Riddle Number One: Who Am I? Observer Riddle Number Two: Who Am I? Interviewer

Riddle Number Three: Who Am I? Participant in field settings Riddle Number Four: Who Am I? Interpreter

When I was conducting this research study, I was the observer, the interviewer, the non-participant in the field and interpreter of findings. The clearer I know what my role is, the more capable I am in achieving the research objectives without personal bias.

125

Figure 3.5. Framework of data analysis (adapted from Saat, 2003). First Stage of Analysis

Transcript of audio data during data collection initial findings led to reflection on data collection and data analysis

Second Stage of Analysis

Transcript of observation data Transcript of interview data

Matrix of observation data Matrix of interview data

Reviewed by supervisor

Refinement of themes and subthemes Area of Inquiry Strategies for building rapport Perception of teacher- student rapport Factors affect teacher-student rapport Teacher-student rapport represented through dialogue Characteristics of dialogue

126