Estimating effects heterogeneity

The results presented in the previous two subsections suggested little support of the implicit hypothesis in conventional accounts of peacebuilding influence on political participation–

results I argued should be expected if variation in local contexts is not taken into account. In the theoretical section, I drew on insights from social science studies to formulate specific hypotheses about how the strength of local cultures plays an important role in terms of moderating different peacebuilding strategies’ influence on liberal political participation at the local level and argued that these should provide a better accounting of the empirical relationship between peacebuilding strategies and political participation in local settings of different cultural strength, at least as measured in this chapter. I now turn to empirical data for answers.⁵ To ascertain effects, I fit a generic regression equation of the following

5My research hypotheses were formulated in a pre-analysis plan prior to conducting data analysis. The type of analysis I carry out is typically referred to as ”heterogeneous treatment effects” or ”interaction effects,” which Gerber and Green (2012) define as ”the change in treatment effect that occurs from one subgroup to the next.” The quantity of interest here is typically referred to as Conditional Average Treatment Effect (CATE), and it ascertains whether, and the extent to which, pre-specified covariates (in this case, cultural strength) lead to higher or lower ATE estimates.

type:

Y_it= β₀+ β₁S_t+ β₂C_t+ β₃P_t+ β₄SXP_t+ β₅CXP_t+ γ_c[t]+ δ_b[c[t]]+ �_it (5.2) Where:

Yit: is the level of participation for person in in town t (also town average);

S_t : is a dummy variable indicating whether the town received at security committee treatment;

C_t: is a dummy variable indicating whether the town received at civic education treat-ment;

P_t: is a variable indicating that indicate the strength of local culture t;

SXPt : is an interaction term between the security committee treatment and the strength of measure;

CXPt: is an interaction term between the civic education treatment and the strength of culture measure;

γ_c[t]: indicates wether the clan of security committee town was a high or low density;

δ_b[c[t]]: is a fixed effect for the block (and field facilitator) to which the town was assigned;

and

�_it: is the disturbance term for the regression.

This equation is similar to equation (1), except for the addition variable P_t, which indicates the level of cultural strength for town t and its interactions with the security committee and curriculum treatment (i.e. SXP_t : and CXP_t :).⁶ The parameters of interest in this equation are: β₄ and β₅, which basically provide estimates of ATEs across culturally established settings and across culturally fragmented settings.

The coefficient on the interaction terms (in model one) or the coefficient on the primary factors (in models 2 and 3 where the sample is already split) will provide an answer.⁷ That

6It should be noted that for the purpose of this part of the analysis, the civic education treatment combines the civic education only condition and the civic education plus security committee for both conceptual, statistical power and practical reasons. Though I plan to conduct sensitivity analysis with these measures employed separately to see if there will be significant variation in the results.

7In the full model, only the coefficients on interaction terms are of substantive and statistical importance.

is, these coefficients (in the full model at least) should be interpreted as the predicted differences in increase (or decrease) of levels of liberal participation between heterogeneous communities that have undergone similar treatment interventions. Regressions estimates are summarized in Table 5.3 below⁸ Model 22 presents results on a full sample, with two interaction terms based on each of the two treatment variables and a dummy variable indicating cultural strength (1 is the category for weak valuation). Models 23 and 24 present interaction effects in a different way, by looking at the effects of the treatment variables in split samples for local settings with and without an established culture. (I used the mean as a threshold to discriminate between the two types). Figure 5.1 provides compelling visualization of these heterogeneous results.

The results in Table 5.3 and visualizations shown in Figure 5.1 are striking and re-veal that the effects of the two peacebuilding strategies under study–the village security committee program and the civic education curriculum–depend critically on local context measured in terms of cultural strength. The results provide support for theoretical expec-tations outlined in this theory. Over the entire population of our communities, as model 22 indicates, the civic education had very large positive effects on political participation of the communities with fragmented established cultures in the order of .69 standard deviations, whereas the security community had a negative effect of -.48 standard deviations on the same set of communities. These are as substantively meaningful as they are statistically so, having achieved significance at the 99 percent confidence level.

Statistical analyses in split samples provide clearer support of each of the hypotheses outlined. In local settings that have established cultures (model 23), the curriculum inter-vention led to a .25 standard deviation decrease in levels of liberal political participation (a prediction made in H1), whereas the security committee intervention increased liberal participation by nearly twice this effect size (.41 standard deviations), the prediction made

In other words, the importance and significance of the primary factors are different when interaction terms are taken into account (Jaccard and Turrisi 2003), and for the purpose of the analysis at hand, they are not of primary concerns.

8I also estimated interaction effects between the two treatment variables and the full index measure of cultural strength, with values centered at mean 0 and the significance of the results is unchanged.

Table 5.3: Heterogeneous effects of peacebuilding strategies

Overall political participation

Full model Established Settings Fragmented Settings

(22) (23) (24)

Security committee .40*** .41*** -.18

(.10) (.12) (.16)

Civic education -.27** -.25* .50***

(0.10) (.10) (.10)

Cultural Strength -.02

(.17) Sec com X Cultural Strength -.48***

(.19) Cived X Cultural Strength .69***

(.17)

Constant -.08 -.14 -.06

(.11) (.14) (.12)

N= 1172 N=618 N= 554

F= .000 F= .000 F= .000

Adj. R² = .18 Adj. R²= .19 Adj. R²= .19 Table displays robust, clustered standard errors in parentheses. *, **

and *** indicate significance at the 90, 95 and 99 percent confidence levels respectively.

Figure 5.1: Heterogeneous effects of peacebuilding strategies

CivEd

SecCo

Strategy Type

-.5 0 .5 1

Standardized Coefficients

Established Settings

CivEd

SecCo

Strategy Type

-.5 0 .5 1

Standardized Coefficients

Fragmented Settings

Red dots indicate the difference in coefficients on the primary factors. Lines represent CI at 95 percentille.

Peacebuilding Strategies, Political Participation and Local Setting Types

in H3. By contrast, in culturally fragmented settings (model 24), the curriculum interven-tion is shown to have contributed to a .50 standard deviainterven-tion increase in levels of liberal participation, an effect statistically significant at the .01 level, which is consistent with the prediction made in H2, whereas the security committee intervention did poorly–as predicted by H4–and appears to have reduced levels of liberal political participation by 18 standard deviations, though this result is not statistically significant at the conventional level. These are important results as they suggest that different aspects of the same peacebuilding strat-egy can actually work at counter-purposes. Interestingly, the results also demonstrate that

differences in cultural strength do not automatically translate into a startup ”bump” with respect to political participation, suggesting that these effects do in fact come from the work of international interventions.⁹ Yet with this heterogeneity in local settings, average effects of the peacebuilding strategies can look no different from zero, as found in a number of the macro-level empirical studies discussed previously.

But how robust are these results? From a statistical standpoint, the results are quite robust, as a series of robustness checks confirm.¹⁰ Yet before drawing definitive conclusions about the established relationships, one needs to demonstrate that the results presented here are not a statistical artifact of the particular set of data employed. This is especially important since for my primary measure I relied heavily on auxiliary data provided by the national census (collected in 2008). While such data is typically more reliable than self-reported data, it may raise questions about whether it accurately captures people’s own valuation of their culture. Therefore, it is important to cross-check this construction with self-reported information from the respondents.

One way to do this is by examining whether the patterns of results hold with respect to a host of measures that are theoretically related to cultural strength, but that are not part of the primary measure. Here, I look at the saliency of six alternative indicators: greater trust in a co-tribesmen or co-religious, active participation in community works and in koko and susu, which are forms of rotating credit (in money, goods or labor), greater trust in the traditional leader, and frequency of interaction with the traditional leader. These measures get at qualitative aspects of cultural strength discussed earlier (i.e., stronger social identity and norms such as trusts and frequency of interaction with the chief) and therefore do serve as a good proxy of cultural strength. If the arguments advanced here are correct, we should expect the two treatments to have differential effects across individuals who have high and low values on these indicators. I recoded these indicators as dummy (0 below the mean and 1 above the mean) and estimated regressions of the type equation (2) on each subsample.

9The relevant variable (i.e., strength of culture) in the model does not have a meaningful or statistically significant association with liberal participation.

10I ran 100,000 basic simulations for each of these models, and the means estimates produced by these were virtually similar to those in the original models.

Regression estimates are summarized in Table 5.4 below, in the order listed above and with each pair of models showing results for individuals who value an existing culture strongly and those who do not.

The results are surprisingly consistent with those reported in the preceding paragraph.

In particular, it is noteworthy to see the consistency of strong positive effects of the security committee treatment on liberal political participation with respect to proxies of estab-lished cultures, and virtually no effects with regard to proxies for less estabestab-lished cultures.

By contrast, the civic education treatment does not achieve any effect of significance on participation with respect to cultural strength, which is again consistent with theoretical expectations. On these particular proxies of weak settings, the effects of the curriculum intervention are not consistently positive, but some are (see, for example the results on community works). These results increase confidence in the hypothesized relationship. I discuss these results in greater detail below.