Model fit indices

In SEM, it is generally accepted practice to report the χ² goodness of fit statistic and approximate fit indices to evaluate the fit of the proposed model with the covariance matrix of the data (Kline, 2011). In the results of this thesis the following approximate fit statistics are reported: the comparative fit index (CFI), the Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), the standardised root mean square residual (SRMR), and the expected cross-validation index (ECVI). These indices were selected because they are widely accepted. As well as giving detail on the absolute model fit, these approximate fit statistics also give detail of the how well the model fits relative to the most restrictive and least restrictive models and allow for the fact that no model can be absolutely “correct” (Blunch, 2008; Kline, 2011). All analysis will be conducted with Satorra-Bentler robust maximum likelihood estimation, in which these fit indices are adjusted to allow for it variations of multivariate normality (Raykov & Marcoulides, 2006). When assessing differences between models’ fit,χ² goodness of fit statistics with Satorra-Bentler adjustment are not directly comparable. A “scaled

difference” χ² statistic needs to be calculated that incorporates the non-normality adjustment (Satorra & Bentler, 2001). This statistic is calculated for comparisons between models’ fit in this thesis.

Although suggested cut off points for adequate model fit are outlined in the following sections, because each of these indices have strengths and weaknesses it is generally recommended that these be seen as guidelines. It is suggested that a range of indices are reported and that these are interpreted by looking at whole picture with the knowledge of each of the fit index’s biases (Kline, 2011).

5.6.3.1 χ² likelihood ratio test

This most widely used test of model fit tests the exact fit hypothesis with the null hypothesis that the proposed model perfectly fits the population covariance matrix (Kline, 2011). If the p value associated with the χ² statistic is less than a certain level (usually .05) then the null hypothesis of exact fit is rejected (Raykov & Marcoulides, 2006). It has been widely noted that because the χ² statistic is calculated by multiplying the minimised discrepancy function by the sample size, there is a tendency for models

with a large number of participants to spuriously inflate the χ² value leading to a greater likelihood of rejection of the null hypothesis in large samples (Kline, 2011; Raykov & Marcoulides, 2006). It has also been argued that the standard of perfect fit is not common or realistic in social science research (Miles & Shevlin, 2007). Because of these limitations of the χ² likelihood ratio test, a number of more pragmatic fit indices have been developed. Nonetheless, it is still recommended that χ² value be reported (Blunch, 2008; Kline, 2011).

5.6.3.2Comparative fit index (CFI) and Tucker-Lewis index (TLI)

The CFI and TLI are indices comparing the proposed model to a model with no relationships between observed variables (the independence model; Raykov &

Marcoulides, 2006). The CFI is the ratio of improvement of fit of the observed model over the independence model. The TLI is a similar index that also takes into account the number of degrees of freedom to advantage more parsimonious models (Kenny & McCoach, 2003). The CFI and TLI indices generally range from 0 to 1. It was originally suggested that scores greater than .90 are indicative of good model fit. However, it has more recently been argued that a score .95 is a more appropriate cut off (Byrne, 2010; Hu & Bentler, 1999).

5.6.3.3Root mean square error of approximation (RMSEA)

The RMSEA assesses the extent to which the proposed model fits the data “reasonably” well (T. A. Brown, 2006), rather than the “exact” fit being assessed by the χ² likelihood ratio test. The RMSEA is calculated using the non-central χ² distribution which is used to calculate the degree to which the model is incorrectly specified (T. A. Brown, 2006). It has been suggested that a RMSEA score less than .05 indicates good model fit, up to .08 it can be concluded that the model has reasonable fit, and RMSEA scores between .08 and .10 indicate mediocre model fit (Byrne, 2010; Hu & Bentler, 1999; MacCallum, Browne, & Sugawara, 1996). Confidence intervals for RMSEA can also be calculated, and these are reported in this thesis in accordance with suggestions that this is good practice (Kelley & Lai, 2011; Kline, 2011).

5.6.3.4Standardised root mean square residual (SRMR)

The SRMR is a measure of the discrepancy between the correlation matrix in the data and those proposed by the model (T. A. Brown, 2006). Lower values indicate better model fit. Hu and Bentler (1999) proposed that values less than .08 suggest good

model fit, although Byrne (2010) suggested a more stringent qualification of .05. Yu (2002) suggested that SRMR scores of .07 or less indicate good model fit.

5.6.3.5Expected cross-validation index (ECVI)

This index examines the input covariance matrix against and expected covariance matrix of another sample from the same population (Byrne, 2010). The ECVI is used compare different models—the model with the lowest value is most likely to be replicable across other samples from the same population (Byrne, 2010). This statistic will be used when models are being compared.

5.6.4Data analysis procedure