Procedures Used for Assessing the Reliability and Validity of Reflective

The reliability and validity assessment of a reflective construct followed a standard approach, which was based on classical test theory and conventional scale development procedures. This approach consists of several steps that are outlined below.429

The first step examines the internal consistency reliability of the set of reflective indicators that was previously defined to measure the latent variable construct. It is assessed by calculating Cronbach's alpha (coefficient alpha), which is by far the most frequently used reliability coefficient.430 _{Moreover, Churchill notes that "coefficient alpha absolutely} should be the first measure one calculates to assess the quality of the instrument."431 Cronbach's alpha can take values between 0 and 1 with higher values indicating higher reliability. Thus, low values of Cronbach's alpha indicate that the set of indicators performs poorly in capturing the construct that it is supposed to measure. Following the most

common recommendation in the literature, this dissertation requires a minimum value of 0.7 for the set of reflective indicators to be deemed reliable.432

In addition, item-to-total correlations were calculated for the indicators making up the latent construct. The item-to-total correlation of an individual indicator is generally defined as its correlation with the total score of all the indicators that are measuring the same latent variable construct. Compared to indicators with relatively low correlations with total scores, those that have higher correlations with total scores have more variance relating to the common factor among the indicators and add more to the internal consistency

reliability. Therefore, it is advisable to use item-to-total correlations as a criterion to identify indicators that should be eliminated.433 _{Consequently, as long as Cronbach's alpha} does not fulfill the minimum value, the indicator with the lowest item-to-total correlation is successively dropped until the internal consistency reliability reaches a satisfying level.434 As stated previously, eliminating a reflective indicator is defendable because reflective indicators are essentially interchangeable. Moreover, according to Bearden et al, each indicator should have an item-to-total correlation above 0.5.435

429 Cf. Zinnbauer and Eberl (2004), pp. 15-17. 430 Cf. Homburg and Giering (1996), p. 8. 431 Churchill (1979), p. 68.

432 Cf. Nunnally (1978), p. 245.

433 Cf. Homburg and Giering (1996), p. 8.

434 This approach is based on the recommendations by Churchill (1979), p. 68. 435 Bearden et al. (1989), p. 475.

6.1 Conceptual Differences between Formative and Reflective Constructs 141 Subsequently, an exploratory factor analysis (EFA) of the purified set of indicators is conducted to ensure that these indicators are linked to a single underlying factor. This second step is necessary because a reflective measurement perspective initially only assumes that the indicators of the latent variable construct are all influenced by the sample underlying factor. The number of factors to be extracted is determined by the Kaiser criterion, which recommends retaining all factors with eigenvalues greater than 1. This criterion is based on the idea that the eigenvalue represents the amount of variation explained by a factor and that an eigenvalue below 1 would indicate that the whole factor explains less variance than a single indicator.436 _{A single common factor resulting from the} analysis does not only confirm the construct's unidimensionality but also indicates

convergent validity.437 _{However, it is commonly required that the extracted factor explains} at least 50% of the total variance in the indicators. Moreover, the communality of each indicator should be above 0.16.438

Cronbach's alpha, item-to-total correlations, and exploratory factor analysis are all considered first generation criteria for assessing construct reliability and validity. These criteria were calculated using SPSS software version 14.0. However, several researchers suggest supplementing the first generation criteria with those of the second generation because the latter are expected to be more powerful.439

Therefore, the third step is a confirmatory factor analysis (CFA) of the purified set of reflective indicators. The confirmatory factor analysis was performed using the AMOS software package and – based on the results of the exploratory factor analysis – assumes a one factorial structure. AMOS provides several methods for estimating the parameters (factor loadings) of the one-factor model that represents the reflective latent variable construct. The most widely used estimation method is the Maximum Likelihood (ML) estimator which requires that the observed indicators have a multivariate normal distribution.440 _{However, previous examination of the indicators used to measure the} latent variables in this study revealed that the vast majority did not follow a normal distribution.441 _{Therefore, the parameters (factor loadings) of the one-factor model are}

436 Cf. Backhaus et al. (2005), p. 295; Field (2005), p. 633.

437 Cf. Cadieux et al. (2006), p. 416; Zinnbauer and Eberl (2004), p. 7. 438 Cf. Zinnbauer and Eberl (2004), p. 7.

439 Cf. Fornell (1987), pp. 407-449; Homburg and Giering (1996), p. 8. 440 Cf. Bolllen (1989), p. 107; Backhaus et al. (2005), pp. 369-370. 441 See chapter 5.2.3.3.

estimated using the Unweighted Least Squares (ULS) estimator, which does not assume a particular distribution of the observed indicators.442 _{To obtain a comprehensive impression} of the model fit, information on the Goodness-of-Fit Index (GFI) and the Adjusted GFI (AGFI) were obtained from the results of the confirmatory factor analysis. These two indices represent global fit criteria, and the value of both should be greater than or equal to 0.9.443 _{More importantly, however, the results of the CFA are used to calculate three local} fit criteria: individual item reliability, composite reliability, and average variance extracted (AVE). These local fit criteria primarily assess the degree to which the measurement of the latent variable through its assigned indicators is reliable and valid.444 _{By definition the} individual item reliability is concerned with the measurement reliability of a single indicator and is calculated using the following formula445_:

ii jj ij jj ij i x rel + = ₂ 2 ) ( (Eq.2)

where Zij = estimated factor loading; jj = estimated variance of the latent variable [j; and ii = estimated variance of the measurement error in the indicator variable. In contrast to the individual item reliability, composite reliability and the average variance extracted indicate how well the latent variable (factor) is measured by the composite of its indicators. Therefore, these two criteria can be used to assess the convergent validity of the

indicators.446 _{Composite reliability and AVE are calculated using the following formulas:}

= = = + = k i ii jj k i ij jj k i ij j rel 1 2 1 2 1 ) ( (Eq.3) and = = = + = _k i k i ii jj ij k i jj ij j AVE 1 1 2 1 2 ) ( (Eq.4)

where the summation is over the k indicators comprising the focal latent variable [j.447 The three local fit criteria generally can take values between 0 and 1. However, the

442 Cf. Bollen (1989), p. 112; Backhaus et al. (2005), pp. 369-371. 443 Cf. Homburg and Baumgartner (1995), p. 172.

444 Cf. Homburg and Baumgartner (1995), p. 170.

445 Cf. Backhaus et al. (2005), p. 378; Zinnbauer and Eberl (2004), p. 7. 446 Cf. Homburg and Giering (1996), p. 11.

6.1 Conceptual Differences between Formative and Reflective Constructs 143 literature commonly suggests minimum values of 0.4 for the individual item reliability, 0.6 for the composite reliability, and 0.5 for the average variance extracted.448

As a result of the first three steps, each reflective construct contains a set of indicators for which reliability and convergent validity has been shown. To complete the construct validation process, the fourth step examines the discriminant validity between two or more constructs. Discriminant validity refers to the extent to which indicators of a given

construct differ from indicators of other constructs in the same model.449 _{That is,} discriminant validity is the degree to which constructs that should not be related

theoretically are, in fact, not interrelated in reality. The discriminant validity of reflective constructs is assessed using the Fornell/Larcker criterion, which states that the average variance extracted (AVE) in a construct measurement scale should be greater than the squared correlation of that construct with every other construct in the model.450

Table 9 provides a summary of the various criteria discussed above.

Criteria Minimum Value Source

Internal Consistency Reliability

Cronbach's Alpha W 0,7 Nunnally (1978), p. 245 Item-to-Total Correlation W 0,5 Bearden et al. (1989), p. 475

Exploratory Factor Analysis

Percentage of Variance Explained W 0,5 Zinnbauer and Eberl (2004), p. 7 Communalities (in case of a one-

factor solution) W 0,16 Zinnbauer and Eberl (2004), p. 7

Confirmatory Factor Analysis

Individual Item Relaibility W 0,4 Homburg and Baumgartner (1995), p. 172 Composite Reliability W 0,6 Bagozzi and Yi (1988), p. 82

Average Variance Extracted (AVE) W 0,5 Bagozzi and Yi (1988), p. 82

Discriminant Validity

Fornell/Larcker criterion AVE( i), AVE( j) > r2(i j) Fornell and Larcker (1981), p. 46

GFI W 0,9 Homburg and Baumgartner (1995), p. 172

AGFI W 0,9 Homburg and Baumgartner (1995), p. 172

Local Goodness- of-Fit Global Goodness- of-Fit

Table 9: Overview of Criteria Used for Assessing the Reliability and Validity of Reflective Constructs (Source: own illustration)

448 Cf. Homburg and Baumgartner (1995), p. 172; Bagozzi and Yi (1988), p. 82. 449 Cf. Hulland (1999), p. 199.

6.1.2 Procedures Used for Assessing the Reliability and Validity of Formative