Data analysis

This section describes the analysis carried out for both quantitative and qualitative data. The quantitative data refers to the results of pre- and post-tests while the qualitative refers to the interview data from the teacher and student-participants.

3.8.1 Quantitative Data Analysis

The quantitative data of the study from the pre- and post-tests were analysed to accept or reject the research hypothesis. The software package used to assist in keying in and analysing the data was the Statistical Package for the Social Sciences (SPSS) Version 23.0. The data were first typed and coded before going through the data screening process to correct as many errors and inaccuracies as possible. Possible errors are usually the result of errors when entering the data. The screening also dealt with outliers – values that were inconsistent with the remaining dataset. As they can distort statistical results particularly when running a paired sample t-test, outliers need to be identified and corrected to avoid reaching unreliable conclusions from the findings.

The completion of these steps led to the beginning of data analysis. The quantitative data in the study were analysed using the paired sample t-test to look for the p- value<0.05 which would determine if the change from pre-test to post-test was

significant enough to suggest that the changes did not happen by accident but rather by an external stimulus (the intervention).

The paired sample t-test is used to compare two population means where researchers have two samples in which the observation in one sample can be paired with another observation in the second sample. Shier (2004: online) suggests that the situation may occur in one of these examples:

a. Before-and-after observations on the same group of subjects

b. When a comparison of two different methods of measurement or two different treatments is made where the measurements/treatments are applied to the same subjects

Thus, the first situation described the quantitative phase of the study where the same group of participants were measured twice, before and after a treatment. One of the ways to carry out a paired sample t-test is through SPSS by computing the differences (using Transform, Compute) and then carrying out a one-sample t-test as follows:
—

Analyze
— Compare Means.

The confidence interval for the true mean difference was also calculated. Even though the difference between pre- and post-test might be statistically significant, the size of the increase needs to be calculated via confidence interval to inform readers within what limits the true difference is likely to lie. The formula for a 95% confidence interval for the true mean difference is:

̄∗sd ̄∗ ̄ d ± t √

n or, equivalently d ± (t × SE(d))

where t∗ is the 2.5% point of the t-distribution on n − 1 degrees of freedom.

Another means that was used to calculate the impact of the mean difference is the effect size. Cohen’s d is a common tool and the effect size can be calculated using the paired t- test value as recommended by Rosenthal (1991).

3.8.2 Qualitative Data Analysis

The qualitative data, on the other hand, were managed manually. Despite an initial plan to use Qualitative Data Analysis Software (QDAS), the number of volunteers who participated in the interview was small (6 student-participants) in addition to the teacher-participant and would be manageable without any special software programme.

The interview protocol was developed using the deductive approach as the items were developed to confirm and double-check issues that arose in the quantitative part of the study. In other words, the structure of the interview had already been predetermined that allowed only a slight addition to the interview items. There were four phases involved in the qualitative data analysis. It began with the process of transcribing the data. The interviews were type-written in full as they were recorded. However, in the second phase as I organised the data, the parts where the participants and I were engaged in small talks were removed, leaving only the part related to the interview questions and responses.

In the third phase, the processed data were perused for themes. The students’ responses were tabulated and arranged according to each question to be compared side by side The themes were initially driven by the conceptual framework of the study formulated from the literature of differentiated lessons, but the data gave rise to several other themes when analysed using the constant comparison approach as suggested by Glazer and Strauss (1967). In the end these were merged into only four main themes: observable behavioural changes in the classroom, learner autonomy, equity, and potential of differentiated instruction in Malaysia. The constant comparison approach was employed because a) there were two different points of view – from the teacher and students and b) there were six students who answered the same questions, and thus the responses were compared. Even though the teacher was asked a different set of questions, the same themes (as driven by the literature) were embedded throughout the interview protocol. In the fourth and final phase, the data were summarised and linked to the research aims and questions as the data were arranged according to the themes and where relevant, supported with excerpts from the interview transcripts.

3.9 SUMMARY

In this chapter I reviewed several key components related to the methodology of the research. The first part describes the research design and the rationales for choosing classroom research and the mixed methods approach before explanation about the pre-experimental design and recapitulation of the research aims and hypotheses are given. The second part deals with discussion of the practical aspects beginning with the research procedures, from the needs analysis stage up to the post-intervention stage. The research instruments are later described and followed by a discussion of how two instruments (language learning style preference test and language attitude inventory) were piloted. The two instruments were piloted first with a group of Malaysian students who resembled the characteristics of the actual research participants to check for the test internal consistency reliability and solutions for item improvement were discussed where relevant. The chapter ends with a description of analysis for both the quantitative and qualitative data. In the following chapter, a key part of the study – the intervention is discussed where the teaching module is described along with the checklists for differentiated lessons and critical thinking inculcation.