Robustness tests

To check the robustness of the results, I employ a set of standard strategies in an RD with local randomisation approach. It comprises of predetermined outcomes and placebo outcome balance–test, the density of the running variable–test, and the placebo cut-off–test. I do not perform the sensitivity to window choice–test since the window selected is the lowest one and no window smaller than W0 in the analysis. In addition to these procedures, I also use two datasets of IFLS–5 and SUSENAS of Social and Cultural Module 2012 to check for consistency of the results.

The idea behind the predetermined outcomes test is that there should be no systematic differences between treated and control group concerning outcomes be- fore the treatment at the cut-off. Proving such condition is necessary to show that the treatment or the programme could not have affected these variables. In a local randomisation approach, both predetermined covariates and placebo outcome anal- ysis are analysed withinW0, the window that is being used in the primary analysis (Frandsen, 2017). Table4.6shows the predetermined covariates–test, and we can infer that most of these covariates when the subject was 12 years old are not sta- tistically different relative to the control group. The only exception is the number

TABLE 4.6– PREDETERMINED COVARIATES–TEST

Panel A - Household condition

Dep. Vars.: # of room HH size Live w mother Live w father

RDD-estimate 0.042 0.107 0.019 0.002

P-value small 0.635 0.363 0.216 0.930

P-value large 0.645 0.354 0.25 0.941

Panel B - Number of brothers/sisters

Dep. Vars.: # older brother # older sister # younger brother # younger sister

RDD-estimate 0.056 0.051 -0.012 0.016

P-value small 0.298 0.330 0.780 0.718

P-value large 0.325 0.356 0.799 0.708

Panel C - Proxy for wealth

Dep. Vars.: Have electricity Have piped water Own toilet Have up to 25 books RDD-estimate -0.037 -0.017 -0.011 0.002 P-value small 0.061* 0.444 0.672 0.930 P-value large 0.054* 0.481 0.693 0.941 Obs-left 666 666 666 666 Obs-right 650 650 650 650 Window-left -1 -1 -1 -1 Window-right 1 1 1 1

Note: ***, **, and * indicate 1, 5, and 10% significant levels, respectively. The estimates utiliserdrandinfcommand developed byCattaneo et al.(2016) with the default–a poly- nomial of order zero and use a Kernel type ofuniformfor observations withinW0. The

estimates use IFLS–5 data.

of older sister and probability of having electricity that has a statistical difference yet the magnitudes are negligible.

To implement the placebo test, I replicate the RDD-estimate using exit test score at the junior high school for Bahasa and Science subject as the outcome. The requirement for a good placebo outcome is that it occurs after the treatment, but by programme causal mechanism it could not possibly have been affected by the treatment. The exit test, known as EBTANAS occurs at the final year of junior high school, three years after the P4 workshop and at best of my knowledge its test design is stable at around cut-off. Also, the Bahasa and Science subjects are the

4.6. RESULTS 95 TABLE 4.7– PLACEBO OUTCOME—EBTANASSCORE FORBAHASA AND SCIENCE

ln(Bahasa score) ln(Science score) RDD-estimate 0.006 0.027 P-value small 0.720 0.178 P-value large 0.740 0.185 Obs-left 293 230 Obs-right 251 237 Window-left -1 -1 Window-right 1 1

Note: ***, **, and * indicate 1, 5, and 10% significant levels, respectively. The estimates utiliserdrandinfcommand developed byCattaneo et al.(2016) with the default–a poly- nomial of order zero and use a Kernel type ofuniformfor observations withinW0. The

estimates use IFLS–5 data. The sample size is smaller than the main estimate due to lower response rate for variables related toEBTANASscore.

ones that have no relation to the content of the P4 workshop, unlike other subjects such as PancasilaMoral Education (PMP) which is closely related. The insignifi- cant coefficients for Bahasa and Science score in Table4.7confirm the results that the state ideology indoctrination courses only affect individual scores for PMP sub- ject.

The third step in the robustness analysis is the running variable density test. In an RD design with discrete running variable—local randomisation approach, the researcher cannot use a standard McCrary (McCrary,2008) density test which based on a continuity assumption (Frandsen, 2017). Instead, the density test ex- ploits whether the discrete running variable’s probability mass function (pmf) sat- isfies a certain smoothness condition. If yes, then the observed frequency at the threshold has a known conditional distribution and allows the use of mass point adjacent to the cut-off (Frandsen, 2017). The general similarity of the test with standard McCrary density test is that it seeks whether the number of observa- tions at around the cut-off within the windowW0 are roughly similar. Specifically, I use Stata command rddisttestk developed by Frandsen (2017) to implement the test.

Figure4.5plots the density of observations by running variable and shows the test results. Three chosen value ofk which determines the maximal degree of non-

linearity in the pmf that is still considered to be compatible with no manipulation fails to reject the null of no difference (no manipulation). One possible explanation for a dip in the density at X = 0 is that the admission to school drops when the Asian financial crises hit in 1998. Thomas et al.(2004) demonstrate that the finan- cial crisis has had a dramatic negative effect on school attendance among young Indonesians. Among 8-13year-olds, the fraction of individuals that were not in school in 1998 was nearly 20% higher. In my sample, Figure 4.6 also shows the junior high schools’ admissions drop by about 5% in 1998.

To what extent this crisis effect confounds the impact estimates are twofold. First, the control group are dominated by the student from crisis—survival family. If social capital measures are positively related to the survival, then the impact es- timates are overstated. Second, the non-surviving students might postpone their enrolment and enter later which adds the density in the control group. Thus, the control group is contaminated with lower social capital students—assuming that social capital measures are positively related to the survival. This later case rein- forces the overstated impact estimates. Given that most of the estimates are not statistically significant, the required correction from this density result, if any, is to revise the coefficients downward.

Last, I implement the placebo outcome test using two artificial cut-offs: 1988 and 2008. These two years are picked up arbitrarily with a gap of ten from original cut-off both above and below it. I expect no effect on the outcomes of interest since at these two cut-offs the probability of treatment assignment does not change. Furthermore, each of artificial cut-off contains observations of treated only (for 1988) and control only (for 2008), respectively. Table 4.8 shows the RDD-estimate with these fake cut-offs. All coefficients are statistically insignificant, including the coefficients on trust and participation outcomes. The coefficients of these outcomes were statistically significant with the real cut-off.