Chapter 5: Validity and Reliability of the Wagner (1995) Individualism-
6.2 Construction Process for the Nurses’ Motivation Measure
6.2.3 Method for this validation study
6.2.3.4 Model 4: The final version of the Nurses’ Motivation Measure (
number of items from 14 to 12 because of the theoretical consistency of item content and ease of interpretation (see Appendix K). The Influence of Others factor contained two items, the Status factor contained five items and the Job Rewards factor contained five items. The subscale alphas indicated good internal consistency (see Table 6.5). This section examines the psychometric properties of the final version, from internal
consistencies through to model fit indices, and evaluates the final model with regard to the research questions.
CFA was performed on the final 12-item, three-factor model. This model fitted the best of all the CFA models assessed, χ2 (51, N = 436) = 116.55, p < .001; NFI = 97; NNFI = .98; CFI = .98; RMSEA = .053, and confirmed that the model that emerged during four CFA runs was suitable for use in this research (see Figure 6.2). Specifically, the comparative fit index CFI compared the proposed model with a null model,
assuming that there were no relationships between the measures. A CFI value greater than 0.90 indicates an acceptable fit to the data (Bentler, 1992). The CFA model developed in this analysis indicated a CFI value of 0.98, which suggested a very good model fit. As an index used to assess residuals and adjusts parsimony in the model, RMSEA values must be equal to, or less than, 0.08 for an adequate model fit (Hu & Bentler, 1999). In the CFA model developed, the RMSEA value was 0.053, indicating good model fit.
Figure 6.2. Final 12-item three-factor model of the NMM.
6.2.3.4.1 Reliability of Model 4, the final version of the Nurses’ Motivation Measure (12 items). Cronbach’s alpha was .72 for the 12-item scale, with subscale
reliabilities ranging from .84 to .87 (see Table 6.5). According to Gable and Wolf (1993), good cognitive measures have alpha reliability ranging from the high .80s to the low .90s, but they went on to say that good affective instruments frequently report reliabilities ranging in the .70s. Therefore, based on the Cronbach’s alpha, it was found that the overall scale, as well as the individual factor-based subscales, all demonstrated adequate internal consistency levels.
Table 6.5
Model 4: Final Version NMM Subscale Alpha Scores, Item Means and Standard Deviations Factor Α M SD Influence of Others .84 4.08 2.29 Item 1 4.03 2.55 Item 2 4.13 2.40 Status .87 6.18 1.40 Item 1 5.98 1.83 Item 2 5.57 2.03 Item 3 6.36 1.76 Item 4 6.25 1.62 Item 5 6.72 1.40 Job Rewards .86 6.58 1.14 Item 1 5.89 2.01 Item 2 6.93 1.20 Item 3 6.59 1.32 Item 4 6.72 1.40 Item 5 6.78 1.22 Total measure .72 6.52 .66
6.2.3.4.2 Item and subscale correlations. Discriminant validity measures the degree to which a construct and its indicators are different from another construct and its indicators (Bagozzi, Yi, & Phillips, 1991). Evidence of discriminant validity can be determined by comparing the Cronbach’s alpha of a factor (subscale) to its correlations with other model variables (Sila & Ebrahimpour, 2005). According to Ghiselli,
Campbell and Zedeck (1981), if the value of alpha is sufficiently larger than the average of its correlations with other variables, this is evidence of discriminant validity.
The intercorrelation of items and subscales were considerably improved in the final version of the NMM. All single items now correlated substantially more with the subscale they were designed to measure than with any other subscale. Significant correlations (> .3) existed for all subscales, confirming that while the subscales were measuring a common construct, the correlations were small enough to indicate measurement of unique aspects of that construct (see Table 6.6).
6.2.3.4.3 Construct validity of Model 4, the final version. Although the revisions
of the NMM did not change the individual item wording, the overall model structure was now best represented in the final 12-item, three-factor version. The proportion of variance explained by three factors had risen from 42% to over 62% in the final version. Throughout the development and testing of the NMM, CFAs were conducted,
scrutinising for model estimations at the item and factor structure level. The final version of the NMM demonstrated a vast improvement in the factor structure. Specifically, all factor loadings in the final model are .60 or higher (see Table 6.6) which demonstrates strong convergent validity (Cole, 1987).The amount of squared multiple correlations for the three factors in the model developed range from 0.38 to 0.64 thus indicating acceptable squared factor loadings.
Table 6.6
Model 4: Fit Indices, Factor Correlations and Factor Loadings for the 12-item Three- factor NMM Factor Correlations (φ) Factor 1 2 3 1. Influence of Others 1 2. Status .38 1 3. Job Rewards .62 .41 1
Parameter Estimates (Factor Loadings)
Item 1 .60 .76 .73 Item 2 .89 .79 .71 Item 3 .80 .70 Item 4 .81 .87 Item 5 .72 .72 Model Fit
N χ2 df NNFI CFI RMSEA
436 116.00 55 .98 .98 .053
6.2.3.4.4 Evaluation of Model 4, the final model. Throughout the development of the NMM, the internal consistency was measured and assessed. The main intention of this stage of the research was to refine the instrument from a 34-item instrument, which had intentionally constructed to be long and more encompassing, to a final instrument that was short enough to be appealing and useful, but retain psychometric integrity. In this respect, this initial stage of the research was highly successful. In particular, the initial 34-item instrument was reduced to a much more manageable 12-item instrument. Three CFAs were conducted. The process continued until satisfactory goodness of fit statistics were obtained. In total, 22 of an initial 34 items were deleted, meaning the final scale contained one-third of the original items. In the process of reducing the number of items, the goodness of fit indices for the factor solution improved
total scale coefficient alpha estimates of reliability remained consistently greater than .7 for each stage of the instrument refinement. Despite the loss of items in each scale (up to five in most scales), the alpha scores remained constant. This contrasted with the effect that the loss of items in a scale usually produces on the alpha scores. These results were indicative of strong internal consistency throughout the development of the NMM and provided good support for the internal consistency aspect of the hypothesis.
The assessment of correlations among factors at each development stage presented the greatest challenge. To provide a strong multidimensional measure of nurses’ Career Choice Motivation, it was desirable for the individual factors not to be too highly correlated. In particular, correlations above .8 (after controlling for
unreliability) suggested that the two factors could not be well differentiated. The initial model demonstrated unacceptably high correlational results (as measured by Pearson’s r) at the factor level. The conclusion reached from these results was that a number of scales in the initial model could not be distinguished from other motivation factors easily. The final 12-item model demonstrated greatly improved scores that revealed the distinctiveness of each factor in the NMM, while also being significantly related to each other. The improvement in model fit provided by the three revisions was substantial. Factor correlations improved (see Table 6.6) compared with the initial version (34
items), in which nearly all factors correlated greater than .8. The multidimensionality of
nurses’ Career Choice Motivation was demonstrated in the final model and further
examination of these relationships is explored in Chapter 9.
CFA was utilised to provide confirmation of the a priori model based on SDT (Deci & Ryan, 1985) and a review of past nursing literature. The results showed that the hypothesised pattern of relations from the initial 34-item five-factor model was not confirmed. According to Nunnally (1978), content validity depends on how well the
researchers create measurement items using the relevant literature to cover the content domain of the variable being measured. As mentioned previously, the selection of dimensions and items in this study was based on SDT (Deci & Ryan, 1985).
6.3 Summary
An essential element of this thesis was the development of a valid and reliable measure of nurses’ Career Choice Motivation. This was achieved through an extensive review of the existing literature and theoretical development, as well as review by an expert panel, which resulted in a final item pool for testing. The instrument
respecifications were made in response to the findings of sophisticated analytical procedures addressing construct validity and internal consistency. These ultimately resulted in a valid and reliable 12-item instrument to measure nurses’ Career Choice Motivation (the NMM).
The key strength of the developmental process was the establishment of good construct validity based on CFAs. The initial data provided only a limited view of the overall structure of nurses’ Career Choice Motivation. Consequently, each refinement of the NMM provided a clearer understanding of the overall structure, as well as internal consistency. The assessment of the final 12-item version was achieved through statistical analysis. An examination of the coefficient alphas, mean score distributions, correlations factor structure and the model fit evidence of the NMM final version provided good support for the psychometric properties. The NMM not only provided a strong basis for subsequent research to be pursued in this thesis, it provided a strong instrument that could be confidently utilised for future nursing research.