Data analysis techniques

The main purpose of the Quantitative Strand (Phase 2) was to pilot-test a self-compiled data collection instrument. (See Table 3.1: Flowchart of the Basic Procedures in implementing an exploratory design). In order to evaluate the psychometric properties associated with the constructs, reliability analysis, confirmatory factor analyses (CFA), and exploratory factor analysis (EFA) are required. The following section will discuss these data analysis techniques in detail.

3.4.7.1 Reliability analysis.

The purpose of investigating reliability and inter-item correlations is to determine which of the items in a scale, if any, have a negative effect on the overall reliability of the scale due to their inclusion in the particular scale. If an improvement in overall scale reliability occurs as a result of excluding a particular item, such an item could

be excluded from subsequent analysis (Hair, Black, Babin, Anderson, & Tatham, 2006).

3.4.7.1.1 General guidelines for interpreting reliability coefficients.

Good practice suggests that at least three items per dimension are required to provide adequate identification for a construct. In addition it seems as if the above guideline provides the minimum coverage of a variable’s theoretical domain (Hair, Black, Babin, Anderson, & Tatham, 2011, p. 698).

Nunnally’s (1967) guidelines were used to determine the critical levels of reliability for the scales and sub-scales and are indicated in Table 3.25 below.

Table 3.25

General Guidelines for Interpreting Reliability Coefficients Reliability coefficient value Interpretation

.90 and above excellent

.80 - .89 good

.70 - .79 adequate

below .70 may have limited applicability

In order to determine the applicability of the proposed factor structure (associated with each of the constructs), confirmatory factor analysis should be used. This technique is the focus of the following section.

3.4.7.2 Confirmatory factor analysis (CFA).

In the following section the purpose of confirmatory factor analysis is described.

3.4.7.2.1 Purpose of CFA.

To evaluate the quality of the measurements in terms of the data obtained (i.e.

measurement models), confirmatory factor analysis must be conducted.

The purpose of carrying out a CFA is to provide statistical evidence on whether each of the identified variables is adequately defined in terms of the common variance among the indicators (i.e. items) in a measurement model (MacKenzie, Podsakoff, &

Jarvis, 2005).

Confirmatory factor analysis is a way of testing how well measured variables represent a smaller number of constructs (Hair et al., 2006). In CFA, the researcher must specify the number of factors that exist within a set of variables and also on which factor each variable will load highly, before results can be computed. This information is obtained from the theory, and therefore the CFA serves to confirm the theoretical structure of the construct. Structural equation modelling is then used to test how well the theoretical pattern of factor loadings fits the actual data. Therefore, CFA assists researchers to either reject or accept their pre-conceived theory related to the factor structures associated with each construct.

There are several constructs used in this study. However, these constructs are measured through several indicators (i.e. items in a questionnaire). Thus, the latent variables are equivalent to the variables used in the study. The indicator variables (also known as manifest/observed variables) are equivalent to the items or parcels that are used to measure these constructs (Tabachnick & Fidell, 2001).

In this study, CFA was used to confirm the factor structure of each of the variables (transformational leadership, past leadership, job resources, past job resources supportive organisational climate, psychological empowerment, psychological capital, objective career success and subjective career success) and to provide a confirmatory test of the measurement theory. Only once this is done and the factor structure is accepted with confidence, can the researcher continue to evaluate the research questions.

3.4.7.3 Method of estimation of models.

Once the measurement models have been specified, the next step is to determine how the measurement model will be estimated. In the present study, the method of estimation used in CFA was robust diagonally weighted least squares (WLS), also

called generalised least squares. The least squares method is widely used to find or estimate the numerical values of parameters to fit a function to a set of data and to characterise the statistical properties of estimates. Robust WLS approach allows for a combination of binary ordered polytomous and continuous outcome variables and allows for multi-group analysis (Muthén, 1993).

After the measurement model has been specified and the parameters have been estimated, the following step is the assessment of the quality of each of the measurement models, using a number of goodness-of-fit statistics. The following section provides an overview of the goodness-of-fit statistics to be used to determine the validity of the measurement models in the current study.

3.4.7.4 Evaluating goodness-of-fit through confirmatory factor analysis.

In evaluating the goodness-of-fit for the constructs used in the current study, several approximate fit indices may be consulted. Hence, the degree to which the observed matrix fits the sample matrix is determined through goodness-of-fit statistics, discussed in the following section.

3.4.7.5 Goodness-of-fit statistics.

Goodness-of-fit indices are numerical indices that evaluate how well the model accounts for the data. These indices can be compared for a series of models with an increasing number of common factors (Fletcher, 2007). For the purposes of this study, only the following goodness-of-fit statistics are discussed, as they are the most widely reported and used fit statistics (Byrne, 1998; Hair et al., 2006): Satorra-Bentler chi-square (S-B χ2), standardised root mean chi-square residual (SRMR), root mean chi-square error of approximation (RMSEA), normed fit index (NFI), and comparative fit index (CFI).

3.4.7.5.1 Satorra-Bentler Scaled Chi-Square(S-B X²).

A family of scaling corrections aimed to improve the chi-square approximation of goodness-of-fit test statistics in small samples, large models and non-normal data was proposed by Satorra and Bentler (1994). The Satorra-Bentler scaled chi-squareis used when robust estimation techniques are employed. The reason why robust estimation

techniques are used is when data deviates from the normal distribution. If the data departs markedly from multivariate normality, the Satorra-Bentler scaled chi square statistic (S-B X²) should be used to provide an improved estimate of the fit of a model (Satorra et al., 1994).

3.4.7.5.2 Standardised Root Mean Square Residual (SRMR).

The SRMR is the standardised square root of the mean of the squared residuals, in other words, an average of the residuals between individual observed and estimated covariance and variance terms. Lower SRMR values represent better fit and higher values represent worse fit. The average SRMR value is 0, meaning that both positive and negative residuals can occur (Hair et al., 2006). An arbitrary cut-off of between .05 and .08 can be suggested for SRMR (Byrne, 1998; Hair et al., 2006).

3.4.6.5.3 The Root Mean Square Error of Approximation (RMSEA).

The RMSEA is a good representation of how well the model fits the population, not just the sample used for estimation. Lower RMSEA values indicate a better fit (Hair et al., 2006). In general, as with SRMR, values below .10 for the RMSEA are indicative of acceptable fit, with values below .05 suggesting a very good fit (Byrne, 1998; Hair et al., 2006). An arbitrary cut-off of between .05 and .08 can be suggested for RMSEA (Byrne, 1998; Hair et al., 2006).

3.4.7.5.4 Normed Fit Index (NFI).

A general guideline for the interpretation of the NFI is that values of .90 and higher indicate satisfactory fit between the postulated model and empirical data (Hair et al., 2006).

3.4.7.5.5 Comparative Fit Index (CFI).

The CFI is an improved fit statistic of NFI. Values above .90 are indicative of acceptable fit (Byrne, 1998; Hair et al., 2006).

The following section focuses on the second major approach to factor analysis, namely exploratory factor analysis.