CHAPTER 5: VALIDATION TESTING OF SPIRIT AT WORK SCALE
5.5 Study 1 with 155 UK Samples
5.5.3 Analysis and Results
In all, 400 questionnaires were distributed, 155 fully completed were returned, a response rate of 38.75%. Although there is a widely-cited rule of thumb from Nunnally (1978) that the subject to item ratio for exploratory factor analysis (EFA) should be at least 10:1, Osborne & Costello (2004) argue that recommendation was not supported by published research. Also, there is no one ratio that will work in all cases; the number of items per factor and communalities and item loading magnitudes can make any particular ratio overkill or hopelessly insufficient (MacCallum, Widaman, Preacher, & Hong, 2001). Given that the number of SAWS items was 18, the total sample of 155 participants satisfies the absolute minimum ratio of five individuals per each variable suggested by Gorsuch (1983) and Hatcher (1994). It also has been indicated that in most cases a sample size of 150 sufficient to obtain an accurate solution in EFA, provided that item intercorrelations are reasonably strong (Guadanoli & Velicer, 1988).
The participants consisted of males (26%) and females (74%). The sample reported their ages as either between 26-35 (36%), 46-55 (19%), 36-45 (18%), over 55
(14%) and under 25 (13%), respectively. Most of them described their marital status as married (45%), single (36%), cohabiting (10%), and separated or divorced (9%), respectively. They reported their tenure years in the organisation as either 4-9 (30%), 1- 3 (27%), under 1 (18%), 10-20 (16%), and over 20 (9%), respectively. Lastly, the respondents described their current positions as either administrative/clerical (54%), professional including academic staff (24%), management (12%), service (7%), and technical (3%), respectively.
After several extraction attempts, using multiply techniques (i.e., principal component, maximum likelihood, and unweighted least squares) and attempts at forcing a four-factor solution, only three factors appeared. The Spirit at Work construct in 155 UK samples produced only a three-factor solution (explained 63.41% of the variance) versus the four-factor solution expected. Surprisingly, the three items of engaging work construct were appearing in sense of community factor and the two items of mystical experience loaded in spiritual connection factor. Moreover, all the remaining items of both engaging work and mystical experience were loaded in the third combined factor. An explanation of this occurrence may be identified within the definition of SAWS used herein, which explains engaging work by using terms such as a profound feeling of well-being, a belief that one is engaged in meaningful work that has a higher purpose, an awareness of alignment between one’s values and beliefs and one’s work, and a sense of being authentic. These terms are intuitively related to the other two factors in some extent: (1) a mystical experience characterised by a positive state of energy or vitality, a sense of perfection, transcendence, and experiences of joy and bliss and (2) a sense of community characterised by a feeling of connectedness to others and common purpose. Also, the two items of mystical experience (ME1 “At times, I experience a
“high” at my work” and ME2 “At moments, I experience complete joy and ecstasy at work”) can possibly interpret in some part of spiritual connection which characterised by a sense of connection to something larger than self. Thus, table 5.1 depicts the factor loadings of the SAWS for 155 UK samples.
Table 5.1: Factor Loadings of SAWS for 155 UK Samples
Note: SPC = Spiritual Connection, ME = Mystical Experience, SOC = Sense of Community, and EW=Engaging Work
Extraction Method: Maximum Likelihood.
Rotation Method: Promax with Kaiser Normallization.
Spirit at Work Items Factor Loading
SPC/ME SOC/EW EW/ME
SPC2 .96 SPC3 .74 SPC1 .73 ME3 .72 ME4 .62 SOC2 .85 SOC1 .80 SOC3 .70 EW2 .68 EW3 .67 EW1 .58 EW4 .77 EW5 .77 EW6 .74 ME2 .74 ME1 .72 EW7 .69 ME5 .47
Further, the Bartlett test of sphericity was significant (p < .000), indicating that the 18- item matrix was significantly different from a matrix of essentially uncorrelated items. Moreover, the Kaiser-Meyer-Olkin (KMO) value was 0.89. It is argued that values above 0.6 are required for good factor analysis solutions (Tabachnick & Fidell, 2007). Finally, an analysis of the scree plot was confirmed the selection of three factors, as shown in figure 5.1. The slope decreases sharply between the third and fourth factor, suggesting that the three initial factors accounted for the major part of the variance. Factor 1, spiritual connection and mystical experience, explained 44.11% of the variance; factor 2, sense of community and engaging work, accounted 12.75% of the variance; and finally factor 3, engaging work and mystical experience, was responsible for 6.55% of the variance.
Figure 5.1: Scree Plot Factor Analyses for 155 UK Samples
Exploratory factor analysis is useful as an initial test of the theoretical assumptions about the constructs under investigation, since these do not have to be
declared and consequently the analysis is not influenced by them. Thompson (2004) suggests that when the theory has already been developed Confirmatory Factor Analysis (CFA) is more useful, as it allows the theory to be directly tested and the degree to which the data fits the model can be quantified in several ways. After having reviewed the EFA outcomes, CFA was used to directly test the underlying theory and examine construct validity. Since CFA was used to estimate the adequacy of the measurement model for structural equation modelling (SEM). However, it is still difficult to find a consensus in the literature concerning the most adequate fit indices to be used (Byrne, 2001). Model evaluation is one of the most unsettled issues related to SEM and many different statistics have been proposed as measures of the adequacy of a model (Arbuckle, 2007).
Typically, a common indicator of the adequacy of the SEM model is the chi- square statistical significant test. If the model has an adequate fit, chi-square (χ2) should be non-significant and it means that the model is not rejected. Nevertheless, the χ2 significant test is highly affected by sample size and normally large sample sizes tend to present significant levels (Loehlin, 1992; Byrne, 2001). Regarding the χ2 indicator there is not only checking on a significant test, but the size of χ2. It has been suggested that a χ2 two or three times as large as the degrees of freedom (df) is acceptable (Carmines & McIver, 1981) which means the closer the χ2 value is to the degrees of freedom, the better the model. Therefore, researchers have recommended reporting the χ2/df ratio (Marsh, Balla & McDonald, 1988). There is no general agreement about the optimal or adequate magnitude of the χ2/df ratio although a ratio below 3.0 is considered acceptable, but with ratios below 2.0 indicating a reasonable fit (Buss & Perry, 1992).
Another indicator examining the adequacy of the SEM model is a series of goodness-of-fit statistics, which can be classified as incremental or comparative indices. These indices are based on a comparison of the hypothesized model against a baseline model. One of the popular applied indices is the comparative fit index (CFI; Bentler, 1990) which was modified from the normed fit index (NFI; Bentler & Bonnett, 1980), as the NFI has shown a tendency to underestimate the fit in small samples. The incremental fit index (IFI; Bollen, 1989) was developed to address the issue of parsimony and sample size which was associated with the NFI. The Tucker-Lewis index (TLI referred to as non-normed fit index, NNFI; Tucker & Lewis, 1973) was also developed to overcome one of the weaknesses of the NFI. Whereas in the NFI there is no penalty for adding parameters, the TLI has such a penalty. For these indices (CFI, IFI, TLI) vary from 0 to 1, closer coefficients to unity indicate good fit, with acceptable levels of fit being above 0.9 (Marsh, Balla & McDonald, 1988), but values above 0.95 are preferred (Tucker & Lewis, 1973). Next, the root-mean-square error of approximation (RMSEA; Steiger & Lind, 1980) estimates how well the model parameters are able to reproduce the population covariance. Usually, a RMSEA value around 0.05 is considered a sign of good fit, between 0.05 – 0.10 an acceptable fit, and larger than 0.10 should not be accepted (Hu & Bentler, 1999). Lastly, the Akaike’s Information Criterion (AIC: Akaike, 1987) was developed to compare non-nested models, adjusting for the number of parameters estimated. If the models to be compared are not nested models, the principle that model should be as simple as possible, indicates that we should generally keep the simpler model. The model that generates the lowest AIC value is optimal. The absolute value of AIC has relatively little meaning; rather the focus is on the relative size, the model with the smaller AIC being preferred.
Considering the above analysis, the CFI, IFI, TLI, RMSEA and AIC are the indices used to evaluate the fit of the SAWS models for UK sample.
The models were examined using Confirmatory Factor Analysis (CFA) with AMOS software version 7.0 (Arbuckle, 2007). Principally, the purpose of the CFA is to compare the goodness-of-fit of rival models. Therefore the following 3 initial models were tested: (1) a null model where all items load on separate factors; (2) a single common model where including all 18 items assuming SAWS has only one factor; and (3) a four-factor theoretical model. The results are presented in the table 5.2.
Table 5.2: Fit Indices of Confirmatory Factor Analysis for Initial 18-item of SAWS: UK University Employees
Model χ2 Df χ2/df ∆χ2,(∆df) CFI IFI TLI RMSEA AIC
1) Null Factor 1757.29** 153 11.49 - .00 .00 .00 .26 1793.29 2) One Factor 647.27** 135 4.79 1110.02(18)** .68 .68 .64 .16 719.27 3) Four Factors 367.30** 129 2.85 279.97(6)** .85 .85 .82 .11 451.30
Note: N=155, **p < .001 CFI=Comparative Fit Index; IFI=Incremental Fit Index; TLI= Tucker-Lewis Index; RMSEA=Root-Mean-Square Error of Approximation; AIC= Akaike’s Information Criterion.
The CFA showed that a structure with only one factor did not fit the data adequately (model 2: χ2 = 647.27, df = 135; p < .001; χ2/df = 4.79; CFI = .68; IFI = .68; TLI = .64; RMSEA = .16; AIC = 719.27), neither did a structure with a four-factor theoretical model (model 3; χ2 = 367.30, df = 129; p < .001; χ2/df = 2.85; CFI = .85; IFI = .85; TLI = .82; RMSEA = .11; AIC = 451.30). The difference in fit between model 2 and 3 was highly significant (D2 = 279.97, df = 6, p < .001), indicating that four factors captured the covariation among the 18 items much better than a single common factor. However, initial fit statistics indicated that the four-factor theoretical model was very poor fit with the data and none of the criteria of acceptable model fit were met. Several options exist for modification of the measurement model to obtain adequate fit statistics (Diamantopoulos & Siguaw, 2006). Anderson & Gerbing (1998) state that this would be normally require modification by removing problem reflective indicators; however, one must be cautious to avoid using only statistical selection for removal of items which appear to be problematic.
Firstly, due to two factors between engaging work and mystical experience being loaded in the same factor in EFA and found highly correlated (r = .81), the researcher decided to combine these two factors into one called ‘EW+ME’. Secondly, since low squared multiple correlations (R2) values identified items that were poor indicators of their target factor; a minimum loading of 0.4 was required (Hair, Anderson, Tatham, & Black, 1995; Hinkin, 1998). Lastly, after reviewing all items concerned by starting the items with the highest modification index (MI) and then the items with the lower R2; some problem reflective indicators were considered to be eliminated. Taken together, six items were dropped [ME5 (saw16, R2 = 0.20); EW1 (saw1, R2 = 0.24); EW3 (saw6, R2 = 0.36); EW7 (saw18, MI = 20.60); EW2 (saw5, MI = 19.50), and ME3
(saw12, MI = 11.16]. Finally, the final CFA model of SAWS for 155 UK university employees contained a total of 12 items, six capturing combined engaging work and mystical experience [EW4(saw10), EW5(saw13), EW6(saw15), ME1(saw4), ME2(saw9), and ME4(saw14)], three capturing a sense of community [(SOC1(saw2), SOC2(saw7), and SOC3(saw17)] and three capturing spiritual connection [(SPC1(saw3), SPC2(saw8), and SPC3(saw11)] and with all items loading significantly on their respective factors, no cross-loadings and no correlated measurement errors.
Again the reduced twelve-item scale was subsequently treated with CFA. Given that in CFA, multiple models may fit the same dataset, it is best practice to not only test the single postulated model, but also a number of plausible rival models (Thompson, 2000). Therefore, the modified second-order model (representing the three sub- dimensions of Spirit at Work) was tested against a three-factor first order model, a one factor model (assuming respondents do not differentiate between the sub-dimensions, but Spirit at Work factor does exist) and a null-factor model (the data does not yield a single factor). Further, the three-factor first order model was tested with both correlated factors and uncorrelated factors. Table 5.3 details the results from the CFA of the 12- item SAWS.
The results showed that the modified three-factor correlated model’s overall fit was greatly improved and satisfactory as well as the second order three-factor model (model 4 = model 5: χ2 = 106.08, df = 51; p < .001; χ2/df = 2.08; CFI = .94; IFI = .94; TLI = .92; RMSEA = .08; AIC = 160.08) and were much better than the one-factor model (model 2: χ2 = 323.09, df = 54; p < .001; χ2/df = 5.98; CFI = .72; IFI = .72; TLI = .65; RMSEA = .18; AIC = 371.09) and the 3-factor uncorrelated (model 3: χ2 = 227.28,
df = 54; p < .001; χ2/df = 4.21; CFI = .82; IFI = .82; TLI = .78; RMSEA = .14; AIC = 275.28) which the CFA showed that they did not fit the data adequately. Moreover, the χ2 index to degree of freedom (χ2/df) ratio of both the model 4 and 5 fell marginally to 2.08, indicating a reasonable fit of the models. The difference in fit between model 3 and model 4 and 5 also was highly significant (D2 = 121.20, df = 3, p < .001), indicating that three factors correlated and captured the covariation among the 12 items much better than the three-factor uncorrelated factor. Therefore, the 3-factor solution for SAWS was eventually confirmed and consistent with the previous EFA outcomes. The results are presented in the table 5.3.
Table 5.3: Fit Indices of Confirmatory Factor Analysis for 12-item of SAWS: UK University Employees
Model χ2 Df χ2/df ∆χ2,(∆df) CFI IFI TLI RMSEA AIC
1.Null Factor 1009.61** 66 15.30 - .00 .00 .00 .31 1033.61 2.One Factor 323.09** 54 5.98 686.52(12)** .72 .72 .65 .18 371.09 3. Three- factor (Un- Correlated) 227.28** 54 4.21 95.81(0)ns .82 .82 .78 .14 275.28 4.Three- factor (Correlated) 106.08** 51 2.08 121.20(3)** .94 .94 .92 .08 160.08 5.Second- order Three- factor 106.08** 51 2.08 NA .94 .94 .92 .08 160.08
Note: N=155, **p < .001 CFI=Comparative Fit Index; IFI=Incremental Fit Index; TLI= Tucker-Lewis Index; RMSEA=Root-Mean-Square Error of Approximation; AIC= Akaike’s Information Criterion.
After reviewing all 6 items of ‘EW+ME’ dimension; EW4 I am fulfilling my calling through my work. (saw10) EW5 I feel grateful to be involved in work like mine. (saw13) EW6 I am passionate about my work. (saw15)
ME1 At times, I experience a “high” at my work. (saw4)
ME2 At moments, I experience complete joy and ecstasy at work. (saw9) ME4 I experience moments at work where everything is blissful. (saw14)
The researcher decided to name this emerging combined factor as ‘Meaning in Work’ because this term is consistent with the meaningful work aspect of workplace spirituality definition of Ashmos & Duchon (2000, p.141), the aspect of meaning in work reflects “a sense of what is important, energizing, and joyful about work”. Of which this meaning, as can be seen the meaning of the six items covered: the first two items (EW4, EW5) are addressed “a sense of what is important about work”; the next two items (EW6, ME1) are expressed “a sense of what is energizing about work” and the last two items (ME2, ME4) are included the meaning “a sense of what is joyful about work”.
The path diagram with standardised regression weights is depicted in figure 5.2. As can be seen, all latent factors load moderately/highly and significantly onto the second-order factor, suggesting that the three sub-dimensions accurately represents the higher latent construct of Spirit at Work. The mean of the first order loadings was 0.76, denoting that an average of 70% of the variance in the first-order factors was attributable to the SAWS (James & James, 1989); thus confirming that the conceptualisation of a second-order factor is reasonable. Therefore, the second-order of
SAWS measurement model for 155 UK samples is believed to be the most appropriate model for further study.
Figure 5.2: The Second-Order of SAWS Model for UK Sample with Statistically Significant Loading Standardised Coefficients
1.27 Meaning in Work .54 saw4 e4 .53 saw13 e13 .56 saw10 e10 .31 Sense of Community .53 saw17 e17 .73 saw7 e7 .60 saw2 e2 .35 Spiritual Connection .52 saw11 e11 .93 saw8 e8 .55 saw3 e3 .64 saw9 e9 .80 .48 saw14 e14 .70 .85 .78 .97 .74 .73 .73 .73 .72 .75
Spirit at
Work
1.13 r1 r2 r3 .60 .55 .45 saw15 e15 .67Another step in the evaluation of the adequacy of a measure is the assessment of reliability. Hinkin (1995, 1998) argues that the most common measure of reliability is Cronbach’s Alpha or internal consistency. Overall the higher-order of the SAWS with the UK sample a Cronbach α = .89 and all three subscales exhibited a Cronbach α above .80: meaning in work = .87; a sense of community α = .83; and. spiritual connection α = .84. Nunally (1978) suggested that alpha values of above .70 are acceptable, with values between .80 and .90 being very good. Therefore, these values showed the scale had very acceptable reliability. Lastly, the overall SAWS and three subscale means, standard deviations, correlations, and reliability estimates are presented in Table 5.4. Moderate to strong correlations between the three factors again support the notion of an over-riding Sprit at Work factor.
Table 5.4: Means, Standard Deviations, Correlations and Reliability Estimates: Overall SAWS and Three Subscales for 155 UK Samples
Notes: Reliability estimates are in parentheses; ** p < .01
Scales Mean SD 1 2 3 4
1.Overall SAWS 3.51 0.97 (0.89)
2. Meaning in Work 3.48 1.10 .93** (0.87)
3. Sense of Community 4.11 1.07 .67** .54** (0.83)