Structural Equation Modelling (SEM)

Structural Equation Modelling (SEM) is used here to examine householder decision-making in a bushfire preparedness context. SEM is particularly suited to the confirmation of theory and enables assessment of causal relationships between variables that are based on qualitative assumption and analysis (Anderson & Gerbing, 1988; Byrne, 2001; Goldberger & Duncan, 1973; Nachtigall, Kroehne, Funke, & Steyer, 2003). The benefit of using SEM lies in the ability to estimate multiple dependence relationships simultaneously and to identify how well empirical data fits with a hypothesised theory (Goodness-of-fit). SEM is a multivariate data analysis technique that combines aspects of multiple regression (structural path analysis) and factor analysis (measurement of latent constructs with multiple items) to estimate a series of interrelated dependence relationships (Kline, 2005). While the ability to examine the role of latent constructs in multivariate analysis is a key advantage of SEM, only observed variables are examined here using a structural path analysis. SEM is used here because it can test theory as a whole, rather than by looking at individual causal relationships. Bollen (1989),

Byrne (2001) and Kline (2005) provide detailed descriptions of SEM and its application in the social sciences.

In the current study, causal relationships between variables have been determined from in- depth analysis of qualitative interview data (chapter 4). SEM is applied here using a confirmatory path analysis method (using only observed variables) to test these causal relationships, and ultimately to determine how well the hypothesised theory predicts bushfire preparedness (by assessing model-fit to empirically collected data). The specification of a structural equation model was based on variables that corresponded to those described in the hypothesised theory (Figure 4.4). Confirmation of this theory is carried out here using a two- step process with three large data sets collected over time and in different locations. The first step involves validating the theory: Does a theory developed from qualitative data collected from at-risk householders display a strong goodness-of-fit to quantitative data collected from the same householder population? The second step requires the validated model to be re- tested with data from the same locations at a different point in time, and from a different location. Using this two-step method provides an extremely robust means by which to test the suitability of the proposed bushfire theory in a variety of situations. Quantitative data used to validate and re-test the theory were collected in two peri-urban locations. In Hobart, data were collected in October 2006 (N=482) and 2007 (N=349) creating a longitudinal data set. In Sydney, data were collected during October and November 2007 (N=221).

The null hypothesis tested in all SEM analyses states that the data collected fits the proposed model of bushfire preparedness (Chi-square). This method provides an absolute test of goodness of fit (with non-significance indicating a strong model fit). However, because SEM analyses are sensitive to effects of sample size and non-normality, other goodness of fit indices can also be examined when a significant Chi-square result is returned (on which basis the null hypothesis would normally be rejected). Where appropriate, subjective indices of fit can be considered if the Chi-square test returns a significant result. Alternative indices reported when appropriate include the Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), Normed Fit Index (NFI), Global Fit Index (GFI), and Global Fit Index adjusted to degrees of freedom (AGFI). Descriptions of these fit indices are provided by Long and Perkins (2003) and Byrne (2001).

In order to conduct SEM analyses, data were checked to ensure it met several statistical assumptions. Firstly, SEM requires a relatively large number of cases per variable to provide reliable results. Bentler and Chou (1987) recommend 15 cases per measured variable as a minimum. The model tested here has eight measured variables, meaning 120 cases would be the minimum required to gain accurate parameter estimates (particularly standard error). The smallest data set used in the current chapter allowed 27 cases for each measured variable, with the largest set allowing 60. SEM demands complete data sets and missing value analysis and replacement was conducted prior to undertaking SEM (see section 2.3.4.1), which prevented the necessity of using the AMOS software’s inbuilt maximum likelihood technique for missing data replacement, which imposes some restrictions on the analyses and interpretation of results. SEM also requires continuous and normally distributed data, and while the data used here were not strictly continuous (because data were collected using Likert scales), the underlying distributions of the scales used is continuous. Data were found to be normally distributed. Lastly, the model specified in the SEM analysis was over-identified (where more than one possible solution to each parameter estimate exists, but where each has only one best or optimal solution determined during exhaustive qualitative analyses during the theory development), and is explicitly based on substantive qualitative data, which ensures a strong theoretical basis for determining the causal relationships. All SEM analyses were conducted using AMOS version 17.

5.2.1.1 Structural model validation

Data collected from Hobart in October 2006 were used to validate the proposed bushfire preparedness theory. In validating this substantive theory, having a quantitative data set that corresponds directly to the qualitative set is extremely important as it acts as a control against which validation can be conducted. Because both sets of data have been collected from the same population of at-risk householders, quantitative validation of the qualitative theory provides a means by which to cross-examine that theory. It was anticipated that the Hobart 2006 quantitative data analysed using SEM would provide a strong fit to the substantive theory and offer confirmation for this theory.

5.2.1.2 Testing the structural model

Once confirmed, two secondary data sets were used to test the ability of the confirmed theory to predict bushfire preparedness under different conditions (time and location). Testing the model in this way allows an experimental assessment of how well the model could be applied to predict bushfire preparedness under conditions removed from those in which the model was initially developed. Testing was conducted using two new data sets. The first was collected in the same locations in Hobart as the initial validation data set, but during the following bushfire season (October 2007 - with the inclusion of several additional suburbs to the original Hobart sample). This data set permitted a longitudinal test of the theory and enables more solid conclusions to be drawn about the process leading to bushfire preparedness, overcoming the constraints imposed by cross-sectional examinations of decision-making for bushfire preparedness (Paton, et al., 2008a). In addition, secondary testing was conducted with data collected from householders living in peri-urban locations around Sydney. To be considered strong, the proposed theory should also reflect the decision process followed by individuals living in different environmental and social conditions, and with different bushfire regimes.

5.3 Results

Figure 5.1. Bushfire decision-making measurement model - Validation of structural model of bushfire

preparedness decision-making against data collected from Hobart householders in 2006 (χ2=8.519, df=10,

p=0.578; RMSEA=0.001, 90% 0.000  0.044). Showing R2 value (blue) and standardised regression