Reliability and validity

The extent to which what is measured in a study is consistent reflects the reliability of a research procedure and thus concerns whether similar procedures and findings can be replicated (McMillan, 2003). This indirectly relates to validity in data collection, which refers to the interpretation and generalizability of the results (ibid, 2003).

In order to increase reliability, this study used multiple data-collection procedures. A combination of both qualitative methods of data collection and data analysis through

semi-structured interviews, stimulated recall of critical moments based on recorded lessons, discourse analysis of lesson transcriptions and document analysis and quantitative analysis through systematic classroom interaction observation allowed for methodological triangulation of the consistency between teachers’ perceptions and understanding of the recent English Language curriculum reform for primary schools and the actual implementation of the curriculum in the classrooms. The triangulation of data collected for this study thus compensates for the strengths and weaknesses of each data source and serves to validate and cross check the research findings.

Apart from that, the use of video recording of the observed lessons also allowed repeated viewings of how the curriculum was implemented in the classroom and hence provided opportunities for consistency checks. The interpretations derived from the analysis of data were also cross-checked with another trained research assistant to avoid disagreements and differences of opinion, to ensure consistency and validity. In addition, in the VSRD session, the process of the retrospection that was conducted as immediate as possible after the recorded event enhances validity. As mentioned earlier, in this study the recordings take place on the Day 1 and the stimulated recall session was conducted on Day 2. Although, there was a one day time interval to allow teachers to view the video footage first, but what had happened in the classroom was still fresh in the teachers’ mind.

Besides, the entire 32 lessons were observed and recorded from the back of the class in order to avoid unnecessary disruptions and minimise the effect of the ‘observer’s paradox’ that can affect the reliability of a study. Although, the teachers were at first hesitant and a bit apprehensive of the fact that the interview and their lessons were to be recorded, but as the interview and the lessons continued, they soon forgot the presence of the tape and video recorders. Providing information on the purpose of the study before the recording and the assurance that their identity would be kept anonymous and the data collected would remain confidential helps to lessen their anxiety. The fact that the components that are being observed were not described to the teachers reduces the chances of placebo-effect (Best & Kahn, 2005) where teachers modify the lessons according to what is expected, enhances the realibility and validity of the findings.

Apart from that, the use of interview protocol for both the interviews and the stimulated recall reinforced the validity of the data collection as it ensures that the

questions/prompts do not alter the cognitive process being employed at the time of the event. Best and Kahn (ibid) reinforced that “validity is greater when the interview is based on a carefully designed structure thus ensuring that the significant information is elicited (content validity)” (p. 336). Restating a question in slightly different form at a later time or repeating the interview at another time enhances the realiability or the consistency of response in the interview. Besides, interview is the most effective mean to measure areas where human motivation is revealed through actions, feelings, perceptions and attitudes. A depth of response is possible that is quite unlikely to be achieved through any other means.

A copy of the transcription was also sent to each of the participants for them to add or revise as necessary to increase accuracy and reliability. The goal is to seek confirmation that the researcher’s interpretations are congruent with the views expressed of those on whom the research was conducted (Bryman, 2012).

The focus on classroom interaction and discourse is relevant because of its centrality to the act of teaching and learning and that the teaching repertoire needs to include instructional variety, using and incorporating pupil ideas, appropriate and varied questioning, probing for knowledge and formative feedback to pupils. The application of the discourse analysis approach to analyse classroom interaction is pertinents as discourse analysis is concenred with the investigation of language (Gillen & Petersen, 2006). The employment of mixed-method approach consisted of lesson observations, systematic interaction analysis of digitally recorded lessons, discourse analysis of lesson transcripts, stimulated recall of critical moments selected from lessons, and teacher interviews allowed for the interplay of multiple analytic lenses and procedures and for the lessons to be analysed at the macro and micro level. Such methodological triangulation also allowed for a crosschecking of the reliability and validity of the classroom observation data.

Hence, the instruments used for data collection were carefully designed and piloted before they were administered. This was done to minimize any possibility of producing misleading, ambiguous or vague questions to the interviewees. The equipment used for video recording and audio recording were also piloted to avoid any technical problems that might affect the reliability of the findings.

5.13 Summary of the chapter

This chapter has discussed the research methodology used in the current study. Firstly, the mixed methods approach employed in this study was described, the rationale for the choice of the approach was given and an overview of the research design was provided. Then the selection of the location was justified, sampling strategies in the selection of participants were discussed and the profiles of the participants described. This was followed by an explanation of the instruments used in the interviews, video-stimulated reflective dialogues and lesson observations. To conclude, a detailed explanation of data collection procedures and a discussion of data analysis methods were provided.

CHAPTER SIX