Data analysis

Having collected all the data, I started on the analysis. Analysing the data obtained from both the quantitative and the qualitative methods was extremely time consuming. The analysis of the data obtained from the qualitative and quantitative methods is described in the following two separate sub-sections.

4.4.1 Qualitative analysis

Thematic analysis was employed with the data obtained from the interviews, field notes, open-ended questions of the questionnaires and answers written in spaces following closed questions in the questionnaire.

The interview transcriptions and questionnaire answers were all in Arabic, the learners’ mother tongue. Extracts and passages that referred to a single idea were identified, and this idea might be developed later into smaller themes or units (see Ellis and Barkhuizen, 2005; Robson, 2011 for details on thematic analysis). Final themes were then translated into English. Two dominant codes relevant to the main research question were identified: the merits and challenges of implementing critical thinking as a pedagogy to promote the quality of classroom dialogue, and building positive attitudes towards this pedagogy. After identifying merits and challenges, the codes were divided into smaller themes to fit under the two main codes (e.g., types of thinking strategy that could raise/lower the quality of dialogue). Some of these units were based on themes identified in the relevant literature. For example, the relevance of topics to learners’ lives, which reflects the positive aspect of the pedagogy, was identified by Dantas-Whitney (2002), Fairley (2009) and Chen (2010), and tolerance of ambiguity was identify by Lin and Mackay (2004) (see Table 5.1). With regards to themes associated with the negative aspect of this

92

intervention, I developed two units related to the barriers of reflection: Fears of losing marks and seeing reflection as an unnecessary strategy (see Table 5.2). These units were developed through my interaction with the data. The reason of such coding emerged from the fact that challenges of the critical thinking pedagogies have been overlooked in available literature; thus, I had to create my own themes during the analysis. I kept visiting the data from time to time and listened to the audio recordings while reading the transcribed texts to help me make sense of the data. To accompany the presentation of the data in the results chapters, certain extracts from the participants’ interviews or comments they made in the questionnaire were selected and translated into English. These extracts were then transcribed following the orthographic transcription method. Field notes were analysed in the same way. After interpreting observed descriptions by participants or colleagues, themes were identified and added as evidence to support the data obtained from the interviews, questionnaires and classroom dialogue.

The transcription of the 12 lessons, which formed part of the qualitative analysis of the audio-recorded classroom dialogue, was time-consuming. CA conventions were used to transcribe the classroom dialogue (see Appendix H). The analysis of dialogue was informed by CA methodology, which means that the analysis did not go deeper into CA features. The reason for using CA conventions here was that they lead to a more accurate understanding of classroom interaction and minimise the risk of bias, since things like interruptions and pauses are included. The specific focus in applying a qualitative method for analysing talk was on identifying in which way an interaction pattern was or was not different from an IRF sequence and examining examples of how thinking and learning opportunities were created or obstructed during talk. It should be noted, however, that transcription will never be an identical representation of the real talk that has taken place (Jenks, 2011). The reason is that researchers look for particular interaction details, and do not have time to add all details. Adding to this, it might be difficult to capture every single word produced by interlocutors, more particularly when overlaps occur.

93 4.4.2 Quantitative analysis

The data obtained from the closed questions in the questionnaire, from the pre- and post-tests and from the observation checklists were analysed by means of quantitative methods. Starting with the questionnaires, responses to closed questions were inserted into SPSS software and the results were laid out in a descriptive manner.

For the pre and post-tests, the Mean Turn Length (MTL) was measured for each test. The mean value was calculated by taking the total number of words employed by each individual and dividing this by the number of turns she had (Ellis and Barkhuizen, 2005). The types and frequencies of pragma-linguistic markers, ideas communicated and thinking levels in both tests were also calculated. Results are presented in tables and graphs were created using Excel.

The quantitative analysis of the ad hoc observation was the most time-consuming task in this study. The initial aim was to use Nemeth and Kormos’ (2001) framework for analysing quality of argument. However, the nature of the data I had obtained meant that this framework needed to be replaced. This was because I had found that in some lessons, the students were doing more than simply indicating agreement or disagreement with supporting reasons. For instance, some students initiated new ideas, which provoked activity in the classroom. Also, the teacher talk needed to be taken into consideration, and Nemeth and Kormos’ (ibid.) framework does not include teacher talk. A more detailed framework that included both teacher and student talk needed to be found. I searched education and linguistics databases and came across Brown and Kennedy’s (2011) framework.Thisframework examines the quality of classroom dialogue through identifying types and frequencies of teacher and student utterances. It codes utterances into various types (see Appendix G), and is thus more comprehensive than Nemeth and Kormos’ (ibid.) framework. Although Brown and Kennedy’s (2011) framework was designed for investigating the quality of child talk based on the philosophy inquiry approach, it was adaptable to my study

94

because it involves both the quality of talk and critical thinking. I developed some codes to fit my data (see Appendix K). After deciding on the framework, I started filling in the checklist through reading the transcriptions and listening at the same time to the recordings. The results were inserted into Excel to calculate frequencies.

To ensure the reliability of coding the quantitative data, I carried out an intra-rater reliability check four months after analysing the tests and observational checklists of all lessons. The agreement results of the pre- and post-tests were 87.5% for coding the levels of thinking and 94.28% for coding the types of utterances. With regard to the intra-rater check of the 12 lessons, the agreement levels were 91.94% for coding the utterances produced by the teacher and 87.95% for coding the utterances produced by learners (see Appendices Q and R).

The second step for enhancing the reliability of coding was conducting inter-rater reliability check for both the observation checklists and tests. The codes were discussed with two researchers. Results obtained from coding utterances found in 6 lessons by one researcher showed that the agreement levels were 95.90 % for coding the types of utterance produced by the teacher and 98. 38% for coding the types of utterance produced by learners. With regard to the reliability check of the pre- and post-tests, agreement results obtained from the inter-rater check conducted by another researcher were 89.65% for coding the levels of thinking and 90% for coding the types of utterance (see Appendices Sand T).