Using Nonparametric Model Discrimination Test

To compare the conflict models using SNPI (Structural Network Power Index) variables against those using CINC (Composite Index of National Capability) or GNP variables, I first use Clarke’s (2001a, 2001b, 2003) pair-signed test of nonnested model discrimination60 _{(see also Conover 1980). Clarke}

60 _{The two sets of conflict models tested in this chapter are nonnested because one model cannot be}

reduced to the other model by imposing a set of linear restrictions on the parameter vector (see Clarke 2001a and 2001b for the definition of nonnested model).

(2001a, 2001b) posits that traditional methods of model discrimination such as likelihood ratio tests, F- test, and artificial nesting fail when applied to nonnested models. Clarke (2001a, 2003) also argues that a nonparametric approach for model discrimination such as the pair-signed test is more robust and performs better than other approaches of nonnested model comparison (e.g., the Vuong test). His model

discrimination test compares two nonnested models by examining the predictions of each model. If both models produce similar predictions, they cannot be distinguished. If one model produces better

predictions (statistically significant) than the other, we can conclude that the former model performs better (or has the greater explanatory power) than the latter. So, for example, in applying his tests to conflicts models examined in this chapter, if we find that the conflict models using SNPI variables produce the better predictions than the models using CINC variables, we can conclude that the former models performs better than the latter models. If we find that the opposite is true, we can argue that the latter models have greater explanatory power than the former models.

Clarke’s model comparison tests proceed in two steps (Clarke 2001a, 2003). First, each model’s predictions are generated for all available data points (calculating individual log-likelihood ratio for each case). Second, the predictions of each model are compared based on the median log-likelihood ratio to determine whether there is a statistically significant difference between the two models.61 _This

nonparametric nonnested model discrimination approach has been applied to international relations research. For example, Clarke (2003) applies this approach to compare a political norms explanation against a political structure explanation on foreign policy decision-making (Huth and Allee, 2002) and

61_{Clarke (2003, 77–78) details the algorithm for applying his pair-signed test as follows:}

a. Run model f, saving the individual log-likelihoods. For a binary choice model, the individual log- likelihoods are calculated by yilog(^pi) + (1 – yi)log(1 - ^pi). For a linear dependent variable model,

they are calculated by –log(2 * pi* sum((residuals(x))^2)/N)/ 2- (1/2) * ((residuals(x)) /

sqrt(sum((residuals(x)) ^2)/N)^2). The former is in Clarke (2003, 77) and the latter is in Souva (2005, 159), and also has been confirmed by personal communication with Clarke.

b. Run model g, saving the individual log-likelihoods.

c. Compute the differences and count the number of positive and negative values. d. The number of positive differences is distributed binomial (n, p = .5).

The test determines whether the median log-likelihood ratio is statistically different from zero. If the models are equally close to the true specification, half the log-likelihood ratios should be greater than zero and half should be less than zero. If model f is “better” than model g, more than half the log-likelihood ratios should be greater than zero. Conversely, if model g is “better” than model f, more than half the log- likelihood ratios should be less than zero.

finds that the former has greater explanatory power than the latter. He also finds that the two systemic long-cycle explanations of great power wars (research focused on global economic activities, such as Goldstein (1991), and those on global political order, such as Wallerstein (1983), Modelski and Thompson (1987), and Gilpin (1981)) cannot be discriminated from each other. Souva (2005) applies the test to compare the systemic realist explanation to the domestic-politics explanation of foreign policy decision- making and finds that the former model is statistically better than the latter.

Tables 4.9–4.10 present the results of pair-signed tests for both systemic and dyadic conflict models (comparing the models with the attribute-based power variables against those with the structural network power variables). For the systemic conflict study (systemic dispute and crisis onset models), Table 4.9 shows that ten of twelve models with SNPI variables have greater explanatory power than those with CINC variables (in the remaining two models, the model with SNPI performs equally well compared to the model with CINC). In other words, the model comparison test confirms that the models with SNPI

generally account for more variation in systemic conflict (both dispute and crisis) onsets and are statistically better than the models with CINC. For example, in 59.5–69.0% of all the systemic dispute cases (depending on the models), the model with SNPI outperforms the model with CINC (if the two models perform equally, each should account for 50%). For all of the systemic crisis cases, four of six models with SNPI outperform the model with CINC in 54.8–69.0% of the cases. In all the models where the SNPI conflict models outperform the CNIC model, the null hypothesis of equality between the two models is rejected at the 0.001 level.

For the dyadic conflict study, Table 4.10 shows that for all six models, those with SNPI variables have greater explanatory power than those with either the CINC or GNP variable. This means that the model comparison test confirms that the models with structural network power variables account for more variation in dyadic conflict onsets and are statistically better than the models with the attribute-based power variables. For example, in 53.9–62.9% of all dyadic dispute cases, the model with SNPI outperforms the model with CINC. The same results are found when the SNPI conflict models are compared with the GNP model; all six models with SNPI outperform the model with GNP in 53.3–61.6% of the cases. In all of the conflict models, the null hypothesis of equality (between the two sets of models,

with SNPI versus CINC and with SNPI versus GNP) is rejected at the 0.001 level. After comparing the conflict models, we conclude that the models with SNPI variables have greater explanatory power than (or statistically outperform) those with CINC (or GNP) variables in both systemic and dyadic conflict onsets.