3. HOW DOES INEQUALITY OF SCHOOL ACHIEVEMENT AFFECT
3.4. Data Analysis and Results
3.4.3. Fixed-effects estimates
Estimates reported above might have been affected by a number of issues concerning OLS, the main being omitted variable bias. A sensitive issue is the lack of information on the main institutional settings in secondary and HE as well as cultural attitudes towards education, which are likely to remain constant across time.
As the dataset contains information for four periods, I take advantage of the data structure. Variables dealing with the wider institutional settings of educational systems and structural characteristics at a societal level are not likely to change over a 10-year period, although educational policies might well change in short time periods. Omitted variable bias could, in part, be addressed by using country fixed effects. A fixed effects approach will show how robust OLS results remain when dealing with omitted variable bias coming from time-invariant unobserved variables.
Variables have been introduced in blocks as in OLS models. Results are presented in Table 3.4. The basic specification only includes inequality of school achievement and the coefficient is statistically significant. An increase of one standard deviation in inequality
77
Table 3.4. Fixed effects regressions for higher education’s GER (+)
Base Add PISA variance composition Add Socio economic variables Add Secondary expenditure Add Tertiary expenditure
Add Sec & Tert expenditure
b/se b/se b/se b/se b/se b/se
Ineq Sch Ach -3.645** -3.767** -2.142 -2.482* -2.451* -2.738** (1.403) (1.445) (1.400) (1.257) (1.314) (1.290) ESCS 4.441 5.033 5.597 2.961 6.534 3.374 (7.231) (7.040) (6.331) (6.878) (5.927) (6.527) PISA Reading (z scores) 4.824 4.951 1.797 1.750 1.907 1.478 (3.579) (3.592) (3.180) (3.215) (3.194) (3.161) % within variance -0.012 -0.024 -0.085 -0.007 -0.062 (0.083) (0.068) (0.065) (0.065) (0.068) ln (GDP per capita PPP) 15.102** 11.490* 18.986*** 14.863*** (6.650) (6.485) (6.011) (5.482)
% Lab force with
HE 0.347*** 0.260** 0.330*** 0.256** (0.126) (0.119) (0.119) (0.118) HE Unemp (% total unemployment) 0.234 0.157 0.254* 0.164 (0.155) (0.168) (0.142) (0.159) GER Secondary 0.141 0.245*** 0.143* 0.235*** (0.087) (0.082) (0.086) (0.084) Exp in secondary (%GDP) 0.718*** 0.651*** (0.198) (0.200) Exp in tertiary (%GDP) 0.192 0.116 (0.149) (0.126) Constant 59.884*** 60.693*** 112.313*- -95.779 -156.673** -131.067** (1.918) (5.506) (66.142) (65.024) (60.219) (55.684) N Observations 228 228 228 228 228 228 N Groups 61 61 61 61 61 61 R2 within 0.236 0.239 0.408 0.447 0.427 0.459 Rho 0.827 0.827 0.881 0.869 0.912 0.897 AIC 1501.8 1505.1 1461.6 1450.0 1458.3 1449.2 BIC 1522.4 1532.5 1513.0 1508.3 1516.6 1514.4
(+) All models control for missing dummies
When controlling for PISA variance composition, the coefficient rises to 3.8. By adding more control variables, the coefficient for inequality of school achievement drops to 2.1-2.8 and still remains significant, but at a lower significance level (p=0.10), except for the specification including both secondary and HE expenditure, which remains significant at p=0.05. In comparison to OLS estimates, which remained unchanged as control variables were introduced, fixed effects estimates for inequality of school
78 achievement become smaller. Common to both approaches, nonetheless, is the fact that it seems that inequality of school achievement does affect HE participation.
Fixed effects and OLS models also differ in effect sizes of some control variables. In fixed effects models, coefficients for PISA reading scores are no longer significant. This means that PISA scores’ OLS estimates were affected by omitted variable bias as most of the effect has been taken by time-invariant cultural, historical, and institutional characteristics for instance. Equally important is that PISA coefficients effect size is massively reduced. Estimates for tertiary education expenditure are no longer negative nor statistically significant, as in OLS estimates.
Power estimates are high enough for the first two fixed-effects models, in the range 0.8-0.85, whereas I find power estimates in the range of 0.49-0.60 in the more complex models as in the last three columns of Table 3.4. This may crucially affect the reliability of the results because there is a high probability of the last estimates not being a true effect. Thus, results have to be taken with the necessary caution.
With regards to models’ appropriateness, unlike OLS models, the introduction of more variables in fixed-effects estimates does not improve goodness of fit – as assessed by BIC – but the opposite, so there is ground, if not to consider overfit to some extent, to contend more complex models.
However, in more substantive terms, I could still underline that there is no significant relationship between a cross-country absolute value of school achievement, as measured in PISA, and GER in HE. Indeed, there are, in fact, countries with comparatively poor performance in PISA reading with high GER as well as high PISA performing countries with lower GER in HE. This is what fixed-effect estimates show when comparing to OLS’s. Thus, for a country, the absolute value of PISA is not relevant for HE participation, although the relative school performance within that country seems to be relevant in order to access HE. Therefore, the relationship between prior achievement and access to HE should be regarded as idiosyncratic or country specific.
79 Nevertheless, the distribution of school achievement seemingly matters at a cross- country level and affects HE participation negatively. The more the persistence of SES on school achievement, the fewer chances of increasing participation in HE. Although this indicator has not been used by researchers and may be hard to interpret, my estimates show that weakening the association between SES and school achievement might be of key importance for countries to increase participation in HE. Moreover, a finding like this highlights the importance of counting on variables measuring social selection mechanisms in education systems, as their influence might be crucial in order to study social reproduction mechanisms through education. More importantly, however, is that this sort of finding may alert countries designing interventions aimed at achieving equity of HE access to look outside HE policy.
I have checked my results’ robustness by running the models with different imputation methods and several alternatives of lagged periods as well as including complete case analysis. These estimates are shown in Table 3.5. Column 1 shows the same coefficients for inequality of school achievement shown in Table 3.4.
Estimates do not vary dramatically, but coefficients are bigger in columns 2 and 4 (PISA variables lagged by 6 years and full case, respectively). Full case estimates are similar to those reported in Table 3.4 (column 1), so are estimates using within-country means as imputation technique (column 3).
Table 3.5. Fixed effects coefficients for inequality of school achievement
1 2 3 4 5
b b b b b
Base -3.645*** -4.394*** -3.386** -3.061* -3.206**
Add PISA variance components -3.767** -4.323*** -3.553** -2.689* -3.210** Add Socio economic variables -2.142 -3.888*** -2.280 -1.431 -2.174 Add Secondary expenditure -2.482* -3.545** -2.362 -2.688** -2.809**
Add HE expenditure -2.451* -3.644*** -2.474* -3.299** -2.468*
Add Sec & Tert expenditure -2.738** -3.301** -2.699* -2.673** -2.968** * p<.10, ** p<.05, *** p<.01
1: Variable of interest and all controls lagged by 3 years from outcome; missing values interpolated 2: Variable of interest and PISA variables lagged by 6 years, all the remaining variables lagged by 3 years from outcome; missing values interpolated
3: Variable of interest and all controls lagged by 3 years from outcome; missing values imputed by using within-country means
4: Variable of interest and all controls lagged by 3 years from outcome; full case
80 Coefficients in column 2 are estimated from a smaller sample as lagging PISA variables by 6 years means missing one observation point so the total variance is reduced. Nevertheless, the effect size is bigger and statistical power stays in the 0.80 area. Detailed tables reporting estimates for imputed data and full case are presented in Appendix A, as well as the full outputs for the estimates shown in Table 3.5.
In summary, results suggest that an increase of one standard deviation in inequality of school achievement is associated with a decrease of 2.5 to 4 percentage points in HE GER. The coefficient value is relevant because, as noted above, this range of values represents between a half and three-quarters of the average GER increase in the last three years.
However, I introduce an additional robustness check by re-running the models using inequality of school achievement but using PISA maths instead. Although the proportion of PISA maths variance explained by ESCS is not available for PISA 2009, I used 2000, 2003, and 2006 tests to re-run the estimates in Table 3.4. In Table 3.6., I compare the estimates for the same specifications in Table 3.4, but this time using the predictor values for 2000, 2003, and 2006 and the outcome value for 2003, 2006, and 2009.
Table 3.6. Estimates of inequality of school achievement using reading and maths PISA scores
Reading Maths
b b
Base (Only R-sq ESCS and ESCS) -3.465*** 0.971
Add PISA variables -3.414*** 0.715
Add Socio economic variables -2.152 -0.202
Add Secondary expenditure -2.791* -1.219
Add HE expenditure -2.188 -0.368
Add Sec & HE expenditure -2.692* -1.046
* p<.10, ** p<.05, *** p<.01
I find quite different estimates conditional to which subject score is used. There are good reasons either supporting the use of maths or reading when it comes to measuring school achievement. Firstly, maths skills might be regarded as not being culturally specific, as mathematical concepts might be universal so measures of different
81 countries should be more transparent, easier, and more comparable than reading. In other words, there are substantive and methodological reasons supporting math scores. Secondly, one may also conjecture that the distribution of maths skills might also be less socially determined than reading. Thus, obtaining coefficients not significantly different from zero for the variable measuring inequality of school achievement is unsurprising.
Nevertheless, there are also good reasons favouring reading scores. First, during the period covered in this study, reading has been the principal subject in 2000 and 2009 whereas maths was the principal subject only in 2003. Scores of non-principal subjects are estimated on the basis of a reduced set of items, so they may be less accurate. Second, the comparison shown in Table 3.5 is made from data having less variance, which may induce unstable coefficients. Third, there are some pieces of research in the UK showing a higher relative importance of reading skills when it comes to access to university (Aucejo and James 2015). In the US some research on academic under preparedness of freshmen suggests that a lack of literacy skills is harder to address with remediation courses than a lack of maths skills (Adelman 2004). Although this work might reflect the fact that a higher proportion of university courses require reading over maths skills, it may also reflect the fact that literacy skills are more likely to be determined by social class than maths skills. This is well in line with classical developments of socio-linguistics because, as Bernstein (2003) noted, working-class children, unlike their middle and upper-class counterparts, were socialised within a more restricted linguistic code.