• No results found

Chapter 2. Do student mobility grants lead to “more and better jobs”?

2.6 Results

This section reports and discusses the final results of this chapter. Each of the following sub-sections concerns one of the outcomes studied: odds of employment, the first one, and net monthly income at PPP, the second one. As outlined earlier, the estimates were calculated separately for each call, since they presented slightly different both selection rules and eligibility criteria16, thus reducing pre-treatment

15

The balance diagnostics was checked through “standardized difference in means”, a balance diagnostics test relying on the difference in means of each covariate between treated and control group, divided by the standard deviation in the full treated group: Xt−Xc

σt . This measure is compared before and

after matching in order to see to what extend the so-called “standardised bias” has been reduced by the matching (Rosenbaum and Rubin, 1985).

16

E.g. an individual who in 2008 was 35 and therefore was eligible for the call 2008, in 2009 was 36 and therefore was not eligible for the call 2009.

95 differences with the treated groups. This procedure explains why the size of the control group decreases for more recent calls.

For each outcome, the results are summarised in a table which, for every call, reports the estimates calculated through different algorithms. More precisely, for every call, the first row reports the so-called naïf effect, which is a simple t-test on the difference in means between treated and control groups, without controlling for selectivity. The second row shows the result with Nearest Neighbour Matching (NNM) with single matching (n=1), the third row with NNM with multiple matching (n=3), the fourth row with NNM with single matching (n=1) and exact matching on gender, the fifth row with NNM with single matching (n=1) and exact matching on level of education (the options being undergraduate degree, Master’s and Ph.D.).

It is worth emphasizing that the last two rows provide additional information not present in the previous ones. In short, they tell us how the programme would have fared had gender and levels of education, respectively, remained constant. However, we acknowledge that the variables accounting for the current level of education of the interviewees are post-treatment and therefore might be endogenous.

Moreover, for further robustness, an additional check has also been done for every outcome, by calculating a logit model for the odds of employment outcome, and an OLS regression model for the net monthly income at PPP outcome. In other words, by using the same specification as the one used to calculate the PS, the impact of the treatment has been re-estimated for each call. Naturally, the consistency of these estimates with those provided by the PSM further increases the reliability of the results.

2.6.1 Odds of employment

By studying the first outcome of interest, odds of employment, the analysis reported in Table 1.1 shows that in general the treatment has no statistically significant effect on the recipients, for every call and for every matching algorithm. In other words, it has been unable to enhance their chances of finding an employment.

Taking a closer look at the individual calls, the results for the call in 2006 show that the programme has no effect. Moreover, it is interesting to note that there is little evidence of sample selectivity, since the naïf effect does not vary considerably when selection bias is controlled for through the PSM. The only exception emerges when performing

96 exact matching on sex: in that case a negative effect of 7.5%, statistically significant at 10%, can be observed.

Table 2.11 – PSM estimates of odds of employment

Algorithm Effect Standard

error Treat on support. Treat off support (trimmed) Control Call 2006 Naïf effect1 -.021 .045 190 21 1260 Nnm(m1)2 -.020 .042 190 21 1260 Nnm(m3)3 -.038 .036 190 21 1260 Nnm(m1) sex4 -.075* .042 190 21 1260 Nnm(m1) edu.5 -.024 .040 190 21 1260 Calls 2007 and 2008 Naïf effect1 .012 .051 162 17 1211 Nnm(m1)2 .015 .048 162 17 1211 Nnm(m3)3 -.017 .040 162 17 1211 Nnm(m1) sex4 .011 .045 162 17 1211 Nnm(m1) edu.5 -.025 .044 162 17 1211 Call 2009 Naïf effect1 .014 .060 144 15 1030 Nnm(m1)2 .017 .060 144 15 1030 Nnm(m3)3 -.019 .050 144 15 1030 Nnm(m1) sex4 -.010 .060 144 15 1030 Nnm(m1) edu.5 .031 .058 144 15 1030

Stars indicate significance: *** 1%-level, ** 5%-level, * 10%-level.

The estimates have been calculated with the Stata module NNMATCH (Abadie et al., 2004). They are based on Nearest Neighbour Matching algorithms (NNM), repetitions are allowed and standard errors are calculated allowing for heteroskedasticity (robust=3).

1 Effect without matching: simple difference in means. 2 NNM is implemented with single matching (n=1). 3 NNM is implemented with multiple matching (n=3).

4 NNM is implemented with single matching (n=1) and exact matching is carried out on the covariate sex.

5 NNM is implemented with single matching (n=1) and exact matching is carried out on the covariate education level which can take the only the values either Ph.D. or Master’s (i.e. lower levels of education have not been considered)

Similarly, the call 2007&2008 is also characterised by the absence of statistically significant effects. In addition, as in the previous case the naïf effect does not differ significantly from the PSM estimates. Therefore, selectivity does not seem an issue for this call either. Also exact matching on gender and education level does not change significantly the results, suggesting that the observed impact of the programme is not significantly influenced by gender or by level of education.

97 Likewise, for call 2009 we observe results that are very similar to the previous call: no statistically significant results and no difference between naïf estimate and PSM estimates. Moreover, the results do not change even when exact matching on gender and level of education is performed.

Our estimates can be considered statistically robust since very similar results are obtained by using different matching algorithms. Nevertheless, to further minimize potential bias, another additional robustness check was performed, consisting of a logit regression model. The results, shown in Appendix 2.5, Table A-2.16, confirm that the treatment has no effect on the odds of employment of the recipients for every call.

2.6.2 Net monthly income

The second outcome of interest to assess the effectiveness of the M&B programme is net monthly income at PPP. In this regard, as reported in the Table 2.12, the results show the absence of any effect for the calls 2006 and 2009, but a statistically significant positive effect for the call 2007&2008.

Table 2.12 – PSM estimates for net monthly income at PPP (in euros)

Algorithm Effect Standard

error Treat on support. Treat off support (trimmed) Control Call 2006 Naïf effect1 161 95 145 16 832 Nnm(m1)2 165 108 145 16 832 Nnm(m3)3 119 93 145 16 832 Nnm(m1) sex4 -29 118 145 16 832 Nnm(m1) edu.5 109 102 145 16 832 Calls 2007 and 2008 Naïf effect1 237 138 116 12 794 Nnm(m1)2 237* 121 116 12 794 Nnm(m3)3 239** 103 116 12 794 Nnm(m1) sex4 180* 104 116 12 794 Nnm(m1) edu.5 258** 117 116 12 794 Call 2009 Naïf effect1 -133 124 97 10 674 Nnm(m1)2 -142 92 97 10 674 Nnm(m3)3 -63 73 97 10 674 Nnm(m1) sex4 -148 93 97 10 674 Nnm(m1) edu.5 -124 89 97 10 674

98

The estimates were calculated with the Stata module NNMATCH (Abadie et al., 2004). They are based on Nearest Neighbour Matching algorithms (NNM), repetitions are allowed and standard errors are calculated allowing for heteroskedasticity (robust=3).

1 Effect without matching: simple difference in means. 2 NNM is implemented with single matching (n=1). 3 NNM is implemented with multiple matching (n=3).

4 NNM is implemented with single matching (n=1) and exact matching is carried out on the covariate sex.

5 NNM is implemented with single matching (n=1) and exact matching is carried out on the covariate education level which can take the only the values either Ph.D. or Master’s (i.e. lower levels of education have not been considered)

In the results shown in Table 2.12 there is no evidence that the first call (2006) of the programme had an impact on the income of the recipients, since all of the estimates are statistically non-significant. In addition, there is no evidence of self-selection, since the naïf effect is not significantly different from the PSM estimates. Further, the estimates based on exact matching on gender and level of education do not provide statistically different results either, suggesting that the ineffectiveness of the programme does not hinge on either of these two variables. Exactly the same conclusions can be drawn with respect to the call 2009, which shows no significant effects irrespective of the matching algorithm.

As mentioned previously, the call 2007&2008 distinguishes itself since it provides evidence of positive effects of the programme on the net monthly income of the recipients, ranging from 180 to 258 euros per month. In this case, the results are statistically significant at 5-10%. The naïf effect displays very similar results to the PSM estimates, suggesting that (self-)selection is not an issue. Moreover, the algorithm performing exact matching on gender produces a smaller coefficient than the others, which might indicate that the programme impacts differently on men and women.

One might wonder why only the call 2007&2008 achieved the expected result of enhancing the income of the recipients. In this regard, there is no simple and clear answer, especially since the PSM does not provide much information on the mechanisms underlying the effect of the programme. However, in our opinion, the most likely explanation lies in the selection of the recipients. In fact, recall from Table 2.1 that in 2008 a higher number of applications than in the other calls had been rejected: 27% as compared to an average of 17%. This higher selectivity of the call 2008 might have increased both the quality of the degrees and of the applicants, resulting in better net monthly incomes. The issue of the scheme’s selectivity and how this can impact on the outcomes of interest is further discussed in the next section.

99 In addition to the application of multiple matching algorithms, a further robustness check relying on OLS regression was performed in order to improve the reliability of our findings. As can be seen in Appendix 2.5, Table A-2.17, this check confirms that the call 2007&2008 has a statistically significant positive effect on the net monthly income of the recipients, corresponding to 265 euros per month. Moreover, it also provides evidence of a positive and statistically significant effect concerning the call 2006, corresponding roughly to 200 euros per month. However, this result should be taken with great caution, since it is in contradiction with the PSM estimates.