Development of the LSA

Based on the data obtained from interviews with leaders, a new measure of job satisfaction was created, the LSA. The development of this measure followed the

recommendations and best practices detailed by Hinkin (1998), Jackson (1971), and Lane et al. (2016). As such, specific facets of job satisfaction were defined based on the themes that emerged from structured interviews and open-ended survey responses. Care was taken to not only describe examples of satisfaction and dissatisfaction associated with each facet, but also to describe how the facets differed from one another. Then, a pool of items two to five times the size of the final scale length was created for each facet. Item pools were subjected to several rounds of revision based on psychometric analyses and SME review. The final result was a 90 item measure assessing 18 facets of satisfaction, where each facet demonstrated sufficient internal consistency (Kline, 2000).

Because one of the goals of this dissertation was to create a measure that would have utility for both researchers and practitioners, a condensed version of the LSA was also created, named the LSA-Brief. Since each facet was defined to reflect a relatively narrow construct, single-item indicators of each facet were selected (Sackett & Larson, 1990). These single-item indicators were selected based on the degree to which they had

high conceptual overlap with their respective facet definitions, demonstrated high corrected-item total correlations with their intended scale, and for which SMEs sorted onto their respective facets with 100% accuracy. Setting these criteria helped to ensure that the items would be unambiguous when rated by leaders, thereby supporting their use as single-item indicators (Sackett & Larson).

As a result of the test development procedures followed in this dissertation, the resultant LSA measures have the potential to serve two important functions. First, the extended version of the LSA provides an option for those looking for a comprehensive, reliable, and psychometrically sound measure of satisfaction when time permits for the administration of the full item pool. For example, researchers who are interested in advancing theories of leader satisfaction, or for those examining how existing research on job satisfaction extends to the experiences of leaders, may benefit from administering the full version of the assessment. In contrast, the brief version of the LSA may be useful for those interested in gaining a quick but robust understanding of leaders’ attitudes, or when time constraints would otherwise not permit the collection of such information. For example, organizations who are interested in identifying areas of dissatisfaction among their leadership team, or executive coaches who would like more information on the greater context in which their coaching clients work may find utility administering the LSA-Brief. Irrespective of which version of the assessment is administered, users of the LSA are likely to gain a more comprehensive understanding of the satisfaction of leaders than if existing popular measures of job satisfaction were used.

10.1.3 Dimensionality of Job Satisfaction

An important step in the test development process is to examine the

done examining the dimensionality of job satisfaction in the literature. Those studies that have been conducted have tended to focus on exploratory methods (Hancer & George, 2004; Tan & Hawkins, 2000). As such, a major contribution of the present study was the application of various advanced modeling techniques, including CFA, hierarchical, bifactor, and ESEM analyses to examine the dimensionality of job satisfaction.

A lack of previous research using the abovementioned methods in the study of job satisfaction meant it was not possible to examine validation evidence of the

dimensionality of the new LSA-Extended by comparing it against previous research findings. To overcome this limitation, the present study also examined the dimensionality of job satisfaction using the MSQ, an extremely popular and well-researched measure of job satisfaction whose items, rating scale, and facet structure closely matched those of the LSA-Extended. Best-fitting models of the LSA were compared to those obtained from the MSQ to determine if the two measures of satisfaction suggested a similar underlying structure of the construct.

It is well-known that facets of job satisfaction tend to be highly correlated with one another (Scarpello & Campbell, 1983). As a result, the few studies that have examined the factor structure of facet-level measures of satisfaction using confirmatory methods have often struggled to find acceptable model fit. This has often been due to exceptionally high correlations among latent variables. In order to achieve convergence of these models, researchers have typically included a higher-order factor of general satisfaction (Bowling et al., 2018; Heritage et al., 2015; McIntyre & McIntyre, 2010). The issue with the application of these hierarchical models is that their implementation

seems to have been data-driven, without much discussion of their theoretical implications.

As discussed by Markon (2019), there are a number of theoretical assumptions that are associated with hierarchical models. In particular, their application to the study of job satisfaction implies that global satisfaction is the sum of facet satisfaction. Proponents of a hierarchical model of job satisfaction would argue that individuals tend to form global attitudes about their jobs, which in turn impacts their level of satisfaction with various facets or components of their job. This would seem to fit with the argument that facets of job satisfaction can and should be aggregated to form a global measure of satisfaction. However, as discussed earlier in this dissertation, the prevailing conclusion within the literature is that facets of satisfaction should not be aggregated, and that facet- level satisfaction is meaningfully distinct from global satisfaction. As such, the use of hierarchical models to explain the dimensionality of job satisfaction seems to contradict the theoretical understanding of the nature of satisfaction in the field.

The shared variance among facets of job satisfaction may be more appropriately modeled using a bifactor approach. A bifactor model would allow for the specification of an overall level of satisfaction to which all facets contribute. Importantly, the bifactor approach also allows for the modeling of unique facet variance. From a theoretical perspective, this suggests that individuals may form general impressions of their job, but that facets remain important predictors that contribute unique variance to relevant

outcomes. This implies that although overall satisfaction is relevant to consider, facets are important in their own right. This explanation is more in line with the way job

As such, this dissertation was one of the first known studies to systematically compare the statistical and theoretical implications of various confirmatory techniques in the study of the dimensionality of job satisfaction. In particular, the bifactor model was found to be associated with improved model fit over the unidimensional and hierarchical models. In addition, this pattern of results was observed for both the LSA and the MSQ, across two independent samples of leaders.

It is important to note that although the bifactor models reported in this

dissertation were associated with incremental statistical improvements in model fit, there was also evidence to support a single factor of satisfaction. The unidimensional model was found to fit the data well, and within the bifactor model all items had strong loadings on the general factor. Those researchers who may be inclined to advocate for a more parsimonious model of job satisfaction would be supported in this position based on the data presented here. However, a strong argument could be made that the added

complexity of the bifactor approach is warranted, given that many items did demonstrate unique variance associated with the facets of satisfaction. As such, an argument could also be made for the fact that a unidimensional approach to job satisfaction may lead researchers to miss potentially meaningful sources of information, and that the bifactor approach allows for a more nuanced approach to measurement.

There are two important implications of this investigation of the dimensionality of job satisfaction. First, it was found that the dimensionality of the new measure of job satisfaction was closely aligned with a well-validated and popular measure of

satisfaction, thereby demonstrating support for the new measure. Further support for the importance of facets as meaningfully distinct constructs was obtained from SME ratings.

When asked to sort LSA items into their relevant facets, SMEs were able to do so for all items included in the final version of the extended LSA with at least 80% accuracy. For those instances where at least 80% rater agreement was not obtained for the majority of items on a given facet, the reasons for inaccuracy in ratings were reviewed and items and/or facets were modified or dropped from the scale as necessary. This resulted in a final LSA with 18 conceptually distinct facets of job satisfaction, each of which were found to significantly contribute to a general factor of satisfaction.

Second, the best-fitting model that was identified for both measures was also the one that more closely aligned with the conceptualization of job satisfaction in the literature. Despite the fact that it traditionally has not been studied in this manner, the data from the present studies seemed to support the theoretical rationalizations of past researchers (Ironson et al., 1989; Spector, 1997). As such, future researchers who examine the dimensionality of job satisfaction may wish to consider including an investigation of bifactor models among their planned analyses.