Preparatory Procedures

This section motivates and describes the preparatory procedures that were followed before conducting the SEM analyses. Therefore this section will a) specify the respective models that were subjected to confirmatory factor analyses, b) identify the measurement models that were evaluated, c) indicate how missing values were approached, d) clarify the necessity of performing item and dimensionality analyses and e) discuss and explain the procedure that was followed for investigating measurement equivalence and measurement invariance.

5.6.1.1 Model specification

This section gives a detailed specification of the measurement model in SEM notation. Specification allows for a clear understanding of the complexity of the model as well as the number of parameters that needed to be estimated.

Null hypotheses H01ii=1, 2, 3 and H02ii=1, 2, 3 were tested by fitting the following basic single-group model to the data of each of the three groups:

Xi = i + Λ^xiξi+ δi --- (3) Where:

- Xi is the column vector of observable indicator scores for group i;

- Λ^xi is the matrix of factor loadings for group i;

- i is the vector of intercept terms;

- ξi is the column vector of latent factors for group i;

- δi is the column vector of unique/measurement errors components for group i comprising the combined effect on X of systematic non-relevant influences and random measurement error (Jöreskog & Sörbom, 1996a).

The above indicated measurement model includes two additional matrices. Firstly it includes a symmetrical variance-covariance matrix Φi and secondly a diagonal variance-covariance matrix i. The symmetrical variance-covariance matrix Φi

describes the variance in and covariance/correlations between the latent variables and the diagonal variance-covariance matrix i variance-covariance matrix Φi

84 describes the variance in and covariance/correlations between the latent variables and the diagonal variance-covariance matrix i. In contrast to the normal single-group measurement model the variances in Φi are also estimated. The fact that i is specified as a diagonal matrix implies that the measurement error terms are assumed to be uncorrelated across the indicator variables (Donnelly, 2009). Freeing off-diagonals in the diagonal matrix would imply that the error terms may be correlated indicating the possibility of additional common factors (Donnelly, 2009).

Taking into account the design intentions of the test developers and the confirmatory nature of this study freeing the off-diagonals would be impossible to justify.

Null hypotheses H03 and H0jj=4, 5, 6, 7 were tested by fitting the following basic multi-group model to the data of the three groups:

X^gi = ^gi + Λ^xgiξ^gi+ δ^gi --- (3) Where:

- X^gi is the column vector of observable indicator scores for group i;

- Λ^xgi is the matrix of factor loadings for group i;

- ^gi is the vector of intercept terms;

- ξ^gi is the column vector of latent factors for group i;

- δ^gi is the column vector of unique/measurement errors components for group i comprising the combined effect on X of systematic non-relevant influences and random measurement error (Jöreskog & Sörbom, 1996a).

The variance-covariance matrix Φ^gi again describes the variance in and covariance/correlations between the latent variables and the diagonal variance-covariance matrix ^g i variance-covariance matrix Φi describes the variance in and covariance/correlations between the latent variables and the diagonal variance-covariance matrix i. The variances in Φ^gi are estimated. The measurement error terms are assumed to be uncorrelated across the indicator variables.

5.6.1.2 Model identification

Model identification allows for determining whether sufficient information is available in order to attain a unique solution for the parameters to be estimated in the measurement model (Diamantopoulos & Siguaw, 2000). The suggestion is to

85 approach model specification in such a manner that a) a definite scale is allocated to each latent variable and b) the number of model parameters to be estimated do not exceed the number of unique variance/covariance terms in the sample observed covariance matrix (MacCallum, 1995). Both requirements have been met in both the single-group and multi-group measurement models. A definite scale has been allocated to each latent variable by fixing the factor loading of the first indicator variable of each latent variable to unity. The scale of the latent variable is thereby set to be equal to that of the first indicator variable of each subscale. The degrees of freedom for each measurement model that was fitted is shown in Table 5.1.

Table 5.1 clearly shows that all measurement models had positive degrees of freedom. The number of model parameters to be estimated therefore did not exceed the number of unique variance/covariance terms in the sample observed covariance matrix.

5.6.1.3 Treatment of missing values

The data might be incomplete due to missing values which can potentially present a problem that will have to be solved. Therefore missing values had to be identified and dealt with prior to conducting the analyses. The method used to impute missing values depended on the number of missing values as well as the nature of the data.

86

Table 5.1

DEGREES OF FREEDOM FOR EACH OF THE FITTED 15FQ+ MEASUREMENT MODELS

Total # of # Unique

Model/ parameters

to #

Indicator information

Hypothesis #

Lambda's # Tau's #

Theta-delta's #

Phi's be estimated variables #

Groups pieces Df

Single group measurement model 80 96 96 136 408 96 1 4752 4344

Configural invariance [Ha] 240 288 288 408 1224 96 3 14256 13032

Weak invariance [H01] 80 288 288 408 1064 96 3 14256 13192

Strong invariance [H02] 80 96 288 408 872 96 3 14256 13384

Strict invariance [H03] 80 96 96 408 680 96 3 14256 13576

Complete invariance [H04] 80 96 96 136 408 96 3 14256 13848

Stellenbosch University http://scholar.sun.ac.za

Missing values could be dealt with in different ways, these included: (1) listwise deletion, (2) pairwise deletion, (3) mean substitution, (4) group mean substitution, (5) imputation by regression, (6) structural equation modelling approach, (7) hot-deck imputation, (8) expectation maximization, (9) full information maximum likelihood and (10) multiple imputation (Du Toit & Du Toit, 2001).

The most appropriate method to use in this study was the listwise deletion method.

All items with missing values were identified through visual inspection and deleted accordingly, leaving only cases with complete data. This method might result in dramatically reducing the sample size which may negatively affect the data (Kline, 2005; Mels, 2003). The success of the statistical analyses is a function of sample size; therefore smaller samples could reduce the power of the statistical analyses (Olinsky, Chen & Harlow, 2003). Listwise deletion can also cause oversight of non-ignorable patterns of missing data (Olinsky et al, 2003). Therefore when data is missing completely at random listwise deletion will be unbiased (Olinsky, 2003).

Using listwise deletion in this study still resulted in an effective sample size of 10019 cases and no pattern of missing values was identified. The most appropriate method to satisfy the treatment of missing values for this study was therefore listwise deletion.

5.6.1.4 Item analysis

In this study the overarching purpose of item analysis was to gain a deeper and more penetrating understanding of the 15FQ+. According to Kline (1994) item analysis is a procedure where the correlations between each item and a total score are evaluated as well as the inter-item correlations. The intention of test developers is to construct items of a test in such a way that items allocated to the same subscale correlate higher amongst themselves than with items from others subscales (Donnelly, 2009). Nunnally (1978) indicates that item analysis is the first procedure used in item selection; the selected items will then be subjected to factor analysis.

The 15FQ+ was developed to measure a personality construct carrying a specific constitutive definition. In terms of this definition specific first and second-order latent dimensions are identified. Items have been written to indicate the standing of respondents on these specific latent variables. The items were developed to serve

88 as stimuli to which respondents react with observable behaviour that is a relatively uncontaminated expression primarily of the specific underlying latent variable. The observed behavioural response to these various scale stimuli are recorded on the response sheet. If these design intentions were successful it should reflect in a number of item statistics. Therefore the item analysis facilitates the process of identifying whether the observed variables are consistent measures of the intended latent variable. High reliability of the provided observed latent variable manifestations would give credence to the design intentions of the test developers. If the design intentions succeeded high internal consistency reliability, high item-total correlations, and high inter-item correlations and high squared multiple correlations should be observed for the items of a given subscale. The converse is, however, not true.

When high internal consistency reliability, high item-total correlations, high inter-item correlations and high squared multiple correlations are obtained it does not conclusively mean that the design intentions succeeded. It simply means that the design intentions could have succeeded. It means that the position that the design intentions succeeded is a permissible position. If, however, low internal consistency reliability, low item-total correlations, low inter-item correlations and low squared multiple correlations should be observed for the items of a given subscale it does conclusively mean that the design intentions failed (Popper, 1972).

This study utilized item analysis to determine whether the items comprising the various subscales successfully operationalise the latent variables they were tasked to reflect, according to the scoring key, as a forerunner to fitting the a priori model to the data. The intention was to retain all items but report on poor items that fail to discriminate between the different levels of latent variables they were designed to reflect, or that fail to respond in harmony with their partner items in the same subscale, both of which could be reasons for poor model fit in subsequent confirmatory factor analyses. Poor items will be identified based on different psychometric evidence. The evidence will include, amongst others, the following classical measurement theory item statistics: the item-total correlation, the squared multiple correlation, the change in subscale reliability when item is deleted, the change in sub-scale variance if the item is deleted, the inter-item correlations, the item mean and the item standard deviation (Murphy & Davidshofer, 2005). In addition, the analyses will also provide initial information regarding the homogeneity

89 of each sub-scale. For these analyses, each ethnic group’s data were analysed separately providing information regarding reliability of the observed variables across the ethnic groups. This procedure should provide valuable information regarding the measurement properties of the instrument across the Black, Coloured and White groups. The SPSS Scale Reliability Procedure was used to analyse the sub-scale items.

5.6.1.5 Dimensionality analysis

The 15FQ+ defines the first-order factors that it measures in a manner that does not allow for a splitting of the personality sub dimensions into finer, more specific personality dimensions. It does make provision for factor fusion into second-order factors but not factor fission. Uni-dimensionality occurs when the items selected for each scale, to represent the first-order personality factors, do in fact all measure a single common underlying latent variable (Hair, Black, Babine, Anderson & Tatham, 2006). The architecture of each scale used to measure the latent variables reflects the intention to construct essentially one-dimensional sets of items. These items are meant to operate as stimuli to which test respondents react with observable behaviour that is primarily an expression of a specific uni-dimensional latent variable.

It is, however, very difficult to isolate behaviour in such a manner that the response to an item only reflects the latent variable of interest. The behavioural response to each item is never only a reflection of the latent variable of interest but is also influenced by a number of other latent variables and random error influences that are not relevant to the measurement objective (Guion, 1998). Therefore strict uni-dimensionality will seldom, if ever, be achieved. The non-relevant latent variables that influence respondent’s reaction to item i do not, however, operate to affect respondent’s reaction to item j. The assumption is that only the relevant latent variable is a common source of variance across all the items comprising a scale.

Hence, uni-dimensionality would be achieved if the partial inter-item correlations would become negligibly small when controlling for a single underlying factor (Hair et al., 2006). In most other measuring instruments the only source of common variance amongst a set of items is meant to be the latent variable the set of items were designed to measure. Once that single common variable is controlled for the (partial) correlations between the items are meant to approach zero. In such cases one would expect to extract a single underlying common factor on which all the items

90 show reasonably high loadings. In the case of the 15FQ+, however, the response to an item in a specific subscale to varying degrees also reflects the remaining 15 latent variables constituting the personality domain but cancel each other out in a suppressor action. The question is what factor structure should emerge if the design intention of the developers of the 15FQ+ succeeded in developing subscales of items that predominantly reflect a single factor but also, albeit to a much lesser extent reflect the remaining factors comprising the personality space? One position to take is that for all subscales the exploratory factor analysis of the inter-item correlation matrix should result in the extraction of 16 factors but that in the rotated solution all items load strongly on a single (most probably the first) factor. All items display small positive and negative loadings, close to zero on all remaining factors.

The other possible position to take is that for all subscales the exploratory factor analysis of the inter-item correlation matrix should result in the extraction of a single factor on which all items load strongly. If, however, exploratory factor analysis of the inter-item correlation matrix would result in the extraction of more than one factor and the items of a specific subscale would load strongly on different factors this would comment unfavourably of the extent to which the design intentions succeeded.

Those scales failing the uni-dimensionality assumption would imply that multiple dimensions are specified for the instrument. Testing this assumption does not work against the need for the CFA. It rather provides further insight into the internal function of the a priori specified factor structure of the 15FQ+ and reasons for possible poor model fit.

To examine the uni-dimensionality assumption exploratory factor analyses (EFA) were performed on each of the scales of the 15FQ+. Unrestricted principle axis factor analysis was used as extraction technique (Tabachnick & Fidell, 2001) with oblique rotation. This analysis was performed on each of the 16 basic scales individually for all three ethnic groups (Black, Coloured and White). Principle axis factor analysis was chosen over principle components analysis as the former only analyses common variance (Tabachnick & Fidell, 2001). Principle axis analysis allows for the presence of measurement error while according to Kline (1994) principle components analysis does not separate error and specific variance.

Measurement of human behaviour and characteristics without measurement error is unlikely (Steward, 2001), consequently principal axis analysis is the preferred

91 method. When uni-dimensionality was not met, the possibility of meaningful factor fission was investigated. The ability of a single factor to account for the observed inter-item correlation matrix was also investigated when the uni-dimensionality assumption was challenged, irrespective of whether meaningful factor fission was found. This investigation allowed the determination of the magnitude of the factor loadings when a single factor (as per the a priori model) was forced, and the examination of the magnitude of the residual correlations. The magnitude of the latter can be regarded as reflecting on the credibility of the extracted single factor solution as an explanation for the observed correlation matrix. To meet the requirements of the suppressor principle the extraction of a single factor or the extraction of multiple factors with satisfactory loadings on the first factor was considered sufficient. The latter was considered to be the more realistic possibility.

SPSS was used for the principal factor analyses as described above. The eigenvalue-greater-than-unity rule of thumb was used to determine the number of factors to extract. A factor loading will be considered acceptable if λij .50. Hair et al.

(2006) recommended in the context of confirmatory factor analysis that factor loadings should be considered satisfactory if λij 0.71. The critical factor loading cut-off value suggested by Hair et al. (2006) is considered somewhat stringent in the case of individual items. EFA results for the separate ethnic group samples will be presented. Differences between each ethnic group sample will also be discussed.

While this does not provide information regarding the configural invariance of the 15FQ+, it does provide valuable information that could be returned to when wanting to identify reasons for poor CFA model fit.

5.6.2 Evaluation of the 15FQ+ Measurement model