Multiple-Group Analysis and Moderation Testing

The impact of moderators on the relations among the variables in a theoretical model is investigated through the use of measurement invariance and multiple-group analysis. Measurement invariance is defined as the extend to which items or subscales have equal meanings across different groups (French and Finch, 2008). This is examined through the equivalence of the psychometric properties of the instrument and the steps are the configural invariance followed by metric invariance, factor covariance invariance and measurement error invariance. Configural invariance refers to the investigation of whether the same item is an indicator of the same latent factor for each group, although factor loadings are allowed to differ across the different groups (French and Finch, 2008). Thus, in case there is similar but not identical indicators per construct, allows for variable presentation in each group and therefore configural invariance is achieved. Metric invariance (also known as factor loading invariance) is one step higher than the configural invariance and requires the loadings of each item on the underlying construct to be equal (identical) across the two groups. Similarly factor covariance invariance requires identical covariances among the different factors for both groups and measurement error invariance requires similar error structures for both groups (Hair et al., 2010). Those two final levels of invariance are very hard to achieve and indeed are rarely achieved in most empirical studies (Hair et al., 2010).

Multiple-group or multi-sample confirmatory factor analysis (MCFA) is a common method for examining the different levels of invariance discussed above (French and Finch, 2008). MCFA extends the traditional CFA into a multi-group situation where separate samples for each separate group are collected and then comparisons are made in order to determine their equivalence (or invariance). The basis of the process evolves through a series of empirical comparisons of different models with increasingly restrictive constraints (Hair et al., 2010). The fundamental measure of difference used is the difference in the chi-square values obtained for each comparing model denoted by Δχ². This measures allows for a comparison between the model that has no restrictions and the restricted one. The basic idea is that if the constrained model has a chi-square that is not substantially higher than the chis-square value of the less constrained model (or if the Δχ² value among the two comparing models is statistically non-significant) then the constraints can be accepted.

The procedure for the MCFA testing starts with the estimation of the most unconstrained models. These are separate and unique CFA models estimated for each sub-group. Those two models should have χ²/df, CFI and RMSEA values that indicate a good fit of the model in order to proceed with the rest of the analysis. Then the second step involves the configural invariance testing. In this step estimation of a common model for both groups at the same time is estimated which is free of any constraints.

This model should meet appropriate levels of CFA model fit as for the two separate models estimated before. Through this test, researchers confirm that the constructs are congeneric among the groups (Hair et al., 2010). This model sometimes is referred to as the totally free (TF) model because it allows all parameters to be estimated freely of any constraints. This model is also the baseline model that is used for the rest of the

comparisons. In the next steps restrictions are imposed to the TF or baseline model and the difference in the Δχ² is statistically examined. The first restriction involves the equivalence of factor loadings and it is called metric invariance. Here, first all loadings are restricted to be equal between the two competing groups. If the constraints are accepted then full-metric invariance is achieved. However, if the constrained is rejected, the researcher should proceed by freeing a set of parameter constraints (as less as possible) in an attempt to achieve partial-metric invariance (i.e. at lease some set of factor loadings is the same among the different groups). The next step in MCFA puts equality constraints (for the two groups) on the covariance among the different constructs (called factor covariance invariance) and the final step restricts the error variances to be equal among the different groups as well (called error variance invariance). Throughout all those steps the values of the Δχ² between the two models are compared and if these values are non-significant then the invariance hypothesis is accepted.

The level of invariance that is necessary to be achieved depends on the nature of the research (Hair et al., 2010). Since the aim of the current research is to examine whether the relationship between two constructs is the same or different across groups, the theory suggests that full configural invariance must be first achieved, followed by full or at least partial metric invariance. Metric invariance is necessary because if it is not achieved then the researcher does not know whether the differences in the relationships among the constructs are due to the nature of the relationship for the two different group or due to differently measured constructs for the two groups. Thus, in the first step of the multi-group empirical analysis (the MCFA), the aim is to achieve at least partial metric invariance.

After the MCFA, when measurement invariance is established, the structural model estimate is assessed for moderation by another series of model comparison, which is quite similar to the invariance testing described above (Hair et al., 2010). The procedure involves first the estimation through SEM of an unconstrained baseline model that is totally free (TF). This means that this first model has path estimates calculated separately for each group and it is identical to the TF model described above.

Another SEM model is then estimated that constraints all measurement weights (factor loadings) to be equal between the two groups. Then, the third model involves constraining the structural weights among the different groups, followed by models constraining the structural covariances of the independent variables, the structural residuals and finally the measurement residuals. All these models are examined in terms of assessing a good model fit as well as the χ² values are obtained. Differences among the chi-square values (Δχ²) are then calculated for each competing model and if the difference is statistically non-significant then the constraints are accepted. Again here, as with the MCFA the level of desired invariance is up to either full or partial metric invariance.

After that level of invariance is achieved the test for the moderating effect continues with the estimation of two additional models. The first is the model that has reached the required level of metric invariance and the second is the model that includes the specific constraint in the path estimate of interest that needs to be tested according to the hypothesis stated by the theory. Comparison of the differences of the two models, through the Δχ² value, indicates whether the model fit significantly decreased (i.e. an increase in chi-square) when the estimates were constrained to be equal for the two groups (Hair et al., 2010). Here, a statistically significant difference between the two

models indicates that the path estimates are indeed different between the two groups and therefore that moderation exists. Contrary, if the models are not significantly different, then there is no support for the moderation hypothesis, since the path estimates are found to be equal among the different groups.