Explanatory variables

As introduced before (see Section 3.1 and Section 3.4), according to the school effectiveness model used in PISA assessment framework (Purves, 1987; Scheerens, 1990; Scheerens and Bosker, 1997; OECD, 2013a), explanatory variables could be classified as input which is related to student background or school background, processes of teaching and learning (e.g. OTL, teaching practices, and teaching quality), and outcomes (e.g. non- cognitive outcomes such as motivation and self-beliefs). As suggested by Hughes’ (Hughes, 1993, cited in Bailey, 1996) washback model (see Table 3.1 in Section 3.6.1), participants’ actions would influence the process of teaching and learning so as to influence students’ learning outcomes. Therefore, as the conceptual framework indicates (see Section 3.6),

in MLM. According to the conceptual framework developed earlier (see Section 3.6), neither the school effectiveness model nor the washback model explicitly suggests students’ learning outcomes in reading as

variables explaining mathematics performance. However, students’ reading performance was still employed as an explanatory variable in MLM, since interview data indicate that students’ reading ability may have effect on their mathematics performance in PISA (see Section 5.2.2.2.1). Student and school background were involved for controlling for background effects in interpreting results of MLM.

Selection of explanatory variables also took into consideration the potential influence of students’ response styles. The processes variables (e.g. exposure to applied mathematics, cognitive activation) and non-cognitive outcomes variables (e.g. interest, self-efficacy) involved in PISA student questionnaires are generally constructed based on Likert-type scales which are commonly used in a range of surveys. On this kind of scale, students are asked to place themselves into a category of the level of their agreement with the statements about their mathematics learning process or non-

cognitive outcomes. It is argued that this question format poses the issue of interpersonal incomparability, since the ways that individuals understand the ‘same’ question might be vastly different and the perceived cut-points

between categories (e.g. strongly agree/agree/disagree/strongly disagree) may differ across individual respondents (Brady, 1985). This incomparability may not only exist across individuals, but also across cultures. For example, it has been evidenced that East Asian students compared to students from other background are more likely to show midpoint responding (MPR) style (Chen et al., 1995; Cheung et al., 2018). Since response styles, such as acquiescence response style (ARS), extreme response style (ERS) and MPR, make observed scale scores deviate from true scores, they can bring bias estimation of relationships between variables (Baumgartner and

Steenkamp, 2001).

To adjust the possible bias caused by these issues, PISA 2012 anchored some of the variables with the anchoring vignette technique introduced by King et al. (2004) (OECD, 2014a). In this technique, along with the self- reports, students are additionally given vignettes which describe hypothetical individuals (e.g. teachers) and asked to assess their level of agreement with the statements about these hypothetical individuals with the same scale as used in student self-report items (King et al., 2004; King and Wand, 2007;

Hopkins and King, 2010; OECD, 2014a). Students’ responses to the vignette assessments which reflect individual differences in interpreting scales and response styles are then used to adjust for students’ self-report responses (King et al., 2004; OECD, 2014a). As vignettes were involved in two of the three forms of PISA 2012 student questionnaire, anchored variables are available for those constructed with the items in these two forms (OECD, 2014a).

With investigation of the validity of anchoring vignettes by using PISA 2012 data of all 64 participating education systems, He et al. (2017) find that students’ responses to the anchoring vignettes were valid for representing individual and cultural differences. The official analyses of PISA 2012 also showed that compared with original (unanchored) variables, anchored variables tend to have higher correlations with students’ performance (OECD, 2014a). Besides, the paradoxical phenomena, for example,

students in high-performing education systems tended to have relatively low mathematics self-concept, were no longer found when anchored variables are used (OECD, 2014a). In consideration of the above, amongst the

explanatory variables, for those having corresponding anchored indicators in Fangshan PISA 2012 China Trial database, the anchored ones instead of the original ones were employed in the MLM analysis in my research.

Regarding students’ responses to the measures of their familiarity with mathematics concepts, it is considered that students may overstate what they know about the concepts due to their response tendency (OECD, 2014a). The overclaiming technique was therefore used in PISA 2012 particularly to address this issue (OECD, 2014a). This technique employs some questions asking students about their familiarity with concepts that do not exist to adjust students’ response tendency to overclaim (OECD, 2014a). In my research, the adjusted variable of familiarity with mathematics

concepts was employed in MLM analysis.

The explanatory variables and their descriptive statistics are shown by level 1 and level 2 respectively in Table 4.5. I will further describe them in the following.

Table 4.5 Descriptive statistics of explanatory variables

Note: missing is almost caused by design (see Section 4.6.2.2.2).

Catergories Variables Description N Min Max Mean (SD) Missing (%)

Level 1

Input Centred_ESCS Grand mean of ESCS _{614 -2.291 2.469 0 (0.816) 0}

Centred_AGE Grand mean of AGE 614 -0.753 0.577 0 (0.269) 0

male Dummy variable of gender 1=male, 0=female

614 0 1 0.528 (0.500) 0

highersec Dummy variable of educational level 1=Upper-secondary, 0=Lower-secondary

614 0 1 0.616 (0.487) 0

Processes OTL EXAPPLM Experience with applied mathematics tasks at school _{413 -2.987 3.204 0.095 (1.014) 32.74}

EXPUREM Experience with pure mathematics tasks at school 413 -2.733 0.796 -0.068 (0.937) 32.74 FAMCONC Familiarity with mathematical concepts (adjusted) 413 -3.440 2.920 0.481 (1.163) 32.74 Teaching

practices

TCHBEHFA Teacher behaviour: formative assessment 411 -2.392 2.630 0.601 (1.033) 33.06 TCHBEHSO Teacher behaviour: student orientation 410 -1.600 3.311 0.605 (1.209) 33.22 TCHBEHTD Teacher behaviour: teacher-directed instruction _{410 -3.653 2.563 1.021 (1.116) 33.22}

Teaching quality

TEACHSUP Teacher support in mathematics classes _{409 -2.885 1.577 0.594 (0.861) 33.39}

ANCCOGACT Cognitive activation (anchored) _{403 -3.067 3.460 0.363 (0.828) 34.36}

DISCLIMA Disciplinary climate 411 -2.508 1.870 0.629 (0.969) 33.06

ANCMTSUP Mathematics teacher support (anchored) 403 -2.749 2.638 0.725 (1.020) 34.36 ANCCLSMAN Classroom management (anchored) 403 -2.822 2.973 0.446 (0.969) 34.36 Non-cognitive outcome ANCINTMAT Mathematics interest (anchored) _{194 -1.696 2.998 0.584 (0.965) 68.40}

ANCINSTMOT Instrumental motivation for mathematics (anchored) _{194 -2.217 2.531 0.388 (0.887) 68.40}

MATHEFF Mathematics self-efficacy _{404 -1.535 2.298 0.860 (1.064) 34.20}

ANCSCMAT Mathematics self-concept (anchored) 403 -2.013 2.807 0.064 (0.796) 34.36

ANXMAT Mathematics anxiety 411 -2.243 2.467 0.223 (0.914) 33.06

Cognitive outcome pv1read Reading performance 614 308 712 530 (69) 0

Level 2

As displayed in Table 4.5 above, the input variables involved in multilevel analysis include students’ family socioeconomic status (ESCS), students’ age, gender (male), educational level (highersec), and student mean ESCS within schools (SCH_ESCS). Among them, male and highersec are dummy variables of gender and educational level respectively. Processes variables of interest in this research include opportunity to learn content (EXAPPLM, EXPUREM, FAMCONC), teaching practices (TCHBEHFA, TCHBEHSO, TCHBEHTD), and teaching quality (TEACHSUP, ANCCOGACT, DISCLIMA, ANCMTSUP, ANCCLSMAN). Outcomes variables contain students’ non- cognitive outcomes, that is, interest (ANCINTMAT) and instrumental motivation (ANCINSTMOT) to learn mathematics, self-beliefs (MATHEFF, ANCSCMAT), and students’ cognitive outcomes in terms of reading

performance.

Except the two dummy variables, male and highersec, the variables listed in Table 4.5 are all continuous variables. ESCS has a mean=0 and SD=1 for OECD countries’ average on PISA international scale (OECD, 2014a). AGE was calculated based on students’ year of birth and month of birth. To make the intercept of regression meaningful or observable, ESCS and Age, as labelled as “Centred” in Table 4.5, were linear transformed by subtracting their respective means from their values. This approach of centring is called grand mean centring. It does not change the interpretation of explanatory variables’ effects and is commonly used in multilevel analysis (Hox, 2010). With this transformation, Centred_ESCS=0, for example, refers to the average ESCS background. For the processes variables and non-cognitive outcomes variables listed above were all scaled with the mean=0 and SD=1 for OECD countries’ average (OECD, 2014a). As I introduced in Section 4.6.2.1, cognitive outcomes in each assessment domain of PISA 2012, for example, reading, mathematics, and science, five sets of PVs were drawn from the distribution of the estimation of students’ ability, representing their performance in the domain (OECD, 2014a). PVs of each domain have a mean=500 and SD=100 for the average of OECD countries in the cycle in which the domain was assessed as the majority domain for the first time (OECD, 2014a). pv1read which is displayed in Table 4.5 is the first PV of students’ reading performance. Here I just listed pv1read as an example. However, in the following multilevel analyses all the five sets of reading PVs were involved.