Using context to understand quantitative findings

The teaching and learning process is hugely complex (Hikmet et al., 2008) and is impacted greatly by the “power of contexts” (Berliner, 2002, p18). Evidence from this study highlighted the importance of culture and context in understanding quantitative findings from the TIMSS 2011 international assessment with respect to Ireland and Northern Ireland. Almost all of the major differences in teacher-related factors between teachers in Ireland and Northern Ireland as reported in TIMSS 2011 were explained through gaining a better understanding of the fourth class context in both countries, using qualitative data (see Table 6.1 in the next chapter for a comprehensive summary). For example, upon analysis of the fourth class TIMSS 2011 quantitative data for Ireland and Northern Ireland, one might have initially drawn the conclusion that students in Northern Ireland outperformed students in Ireland in mathematics due to having significantly more experienced teachers, as this was the most notable difference in teacher-related factors between the two countries, with 71% of Northern Irish students taught by a teacher with eleven or more years of experience in comparison to 35% of Irish students. However, when this study qualitatively explored why teachers of fourth class in Northern Ireland were more experienced than their Irish counterparts, a clearer picture of the context of fourth class in both countries emerged, which provided an alternative, more multifaceted understanding of the differential student achievement scores in the two countries.

The qualitative data suggested that more experienced teachers may be chosen to teach fourth class (P6) in Northern Ireland due to the existence of unregulated external transfer tests, which secondary schools currently require students to take during their final year in primary school, P7. The transfer test is a high stakes test in which students are motivated to achieve highly. This is because transfer test scores influence whether or not a student will be accepted into the secondary school of their

Chapter 5: Discussion of Findings 155

choice. The existence of this test was perceived to impact upon the teachers chosen to teach P6 in this study, because teachers needed to have the expertise to teach the entire P6 mathematics curriculum, while also being expected to prepare P6 students for this transfer test. For example, Dervla (School E, Northern Ireland) described that teaching P6 in Northern Ireland is “quite a specialism”, as teachers are “legally bound to teach the curriculum but yet there is an expectation that you will also cater for preparation [for the transfer test].” The existence of the transfer test is also likely to explain the surprising finding within the TIMSS data that students spent a mean of 6.3 hours per week learning mathematics in Northern Ireland in comparison to 4.1 hours in Ireland. Evidence from the qualitative data also revealed that in addition to spending extra time learning mathematics in school, many fourth class students in Northern Ireland attended extra tuition in mathematics outside of school and that their parents were highly motivated in ensuring that they achieved well in the transfer test. Furthermore, Gareth (School B, Northern Ireland) noted that students experienced stress due to the academic pressure of the transfer test. While all of these factors relate directly to the existence of the transfer test in Northern Ireland, it is highly likely that they impacted upon Northern Ireland’s higher score in TIMSS 2011.

On the other hand, analysis of the data showed no evidence that fourth class students in Ireland experienced academic pressure to perform well on high stakes exams in mathematics, that they routinely spent extra-curricular time studying mathematics, or that more experienced teachers were selected by school leadership to teach them. Therefore, it is likely that these differences between the mathematics learning experiences of Irish and Northern Irish students may explain the fact that Northern Irish students outperformed Irish students in TIMSS 2011. This supports Wang’s

(2001, p20) argument that TIMSS findings should be “scrutinized carefully” due in part to the fact that exposure to mathematics content may differ across nations, as is the case regarding Ireland and Northern Ireland. By including qualitative data in this study, the transfer test was suggested to be an unobservable variable with respect to the quantitative TIMSS 2011 data, which impacted variables at the student, classroom and school levels of the conceptual framework (see Figure 2.1). An edited version of this framework is shown below in Figure 5.1. Variables which data analysis suggests were influenced by Northern Ireland’s high stakes transfer test context are highlighted in red.

Chapter 5: Discussion of Findings 157

STUDENT BACKGROUND

Prior Learning Socio-Economic Status

Special Needs Status English Additional Language

Parental Involvement

STUDENT EXPERIENCES

Student Learning Student Attitude & Motivation

Time on Task STUDENT OUTCOMES Achievement Learning Engagement CLASS FEATURES Class Composition Class Size Classroom Resources

TEACHER CLASSROOM PRACTICES

Classroom/Time/Behaviour Management Communicating High Expectations

Questioning Lesson Planning & Delivery Assessment (Formal/Informal) & Feedback

Teaching Maths Vocabulary Problem Solving Use of ICT Use of Concretes CLASS OUTCOMES Achievement Learning Engagement TEACHER QUALIFICATIONS Years of Experience Level of Education Mathematics Major Academic Ability Pedagogical Knowledge

Professional Development TEACHER ATTITUDES & BELIEFS

Maths Confidence/Efficacy

Attitude Towards Maths Maths Belief Orientation

SCHOOL OUTCOMES Achievement Learning Engagement SCHOOL FEATURES

School Composition (SES) School Composition (EAL)

School Resources

School Size

SCHOOL DYNAMICS

Leadership

School Climate (Social/Academic)

CPD Opportunities Collective Teacher Efficacy