Reliability, Validity and Normality

Both reliability and validity refer to related, desirable aspects of measurements as they are concerned with how concrete measures are connected to constructs (Neuman, 2006). These are major criteria for evaluating measurements (Zikmund, 2003). On the other hand, normality is important because it provides the underlying basis for many of the inferences made by business researchers (Hair, et al., 2003).

5.7.1.1 Reliability

Reliability is defined as “the degree to which measures are free from error and therefore yield consistent results” (Zikmund, 2003, p. 300). It is to ensure the consistency and stability of measurement when measuring the same thing each time. A reliable instrument could be used repeatedly in different time and different

87

conditions. Two dimensions underlying the concept of reliability: stability and internal consistency.

Stability measures the reliability of an instrument over time even though under uncontrollable testing conditions. Stability could be examined by test-retest reliability. Tests-retest reliability refers to the conduct of the same test, administered twice to the same subjects at intervals between several weeks to 6 months later. The higher the correlation of the two tests the more stable is the instrument.

Internal consistency measure the degree of homogeneity of the items in the instrument. In other words, the items in the instrument should be capable of measuring the construct. The most popular tests for internal consistency is Cronbach’s coefficient alpha and Kuder-Richardson formulae. A better instrument should have higher coefficients. Generally, a measure with a Cronbach’s alpha of above 0.7 is considered to be highly reliable (Hair, et al., 2003).

5.7.1.2 Validity

“Validity is the ability of a measure (for example, an attitude measure) to measure what it is supposed to measure” (Zikmund, 2003, p. 302). In other words, instrument or measurement should be able to measure what it is designated to measure. There are three validity tests that are used to test the goodness of measures; content validity, criterion-related validity and construct validity.

Content validity is also known as face validity referred to the adequacy and representativeness of the items in an instrument to measure what they are supposed to measure. In other words, the content of scale appears to be adequate to measure the construct. Zikmund (2003, p. 302) defined content validity as a “professional

88

agreement that a scale logically appears to accurately measure what it is intended to measure”. The content validity is greater if more scale items are used to measure the construct. For this study, content validity should not be a threat as the instruments in this study were adopted from previous studies. The adequacy of the items in the instruments used had been rigorously examined by previous research works.

Construct validity assesses the underlying construct or scale to determine how well the results obtained from the use of the construct fit with theory. Construct validity means that the empirical evidence generated by a measure is in line with the theoretical logic about the concept. It can be evaluated by using convergent validity technique and discriminant validity. As this study uses Structural Equation Modeling (SEM), it is important to measure the construct validity. The model must not only provide acceptable fit, but also must show evidence of construct validity (Hair, Black, Babin, Anderson, & Tatham, 2006). Convergent validity occurs when indicators of a specific construct share a high proportion of variance in common (Hair, et al., 2006), whereas discriminant validity reflects the extent to which the constructs in a model are different (Holmes-Smith, 2005). Convergent validity is similar to criterion validity (Zikmund, 2003). Further discussion of validity tests are explained in the data analysis chapter in Chapter 6.

5.7.1.3 Normality

Data screening and transformation techniques are used to ensure that data have been correctly entered and that the distributions of variables are normal. The results may be biased or even invalid if the variable departs significantly from its normal distribution. The assumption of normality is a pre-requisite for many inferential statistical techniques. Thus, it is important the data is normally distributed.

89

However, if the data is not normally distributed, it is necessary to transform the values of a variable in order to satisfy the distribution requirements for the use of a particular statistic by using some mathematical transformation such as using the logarithm, square root or reciprocal (Greenhalgh, 1997; Zikmund, 2003). But, problems with such transformations can provide an incorrect specification (Shook, Ketchen Jr, Hult, & Kacmar, 2004) and often violate the theoretical logic underpinning the original dataset (Hult et al., 2006). Another alternative is by using the non-parametric test. Non-parametric tests are also known as assumption-free tests because they have fewer assumptions about the type of data (Field, 2009). The most common non-parametric procedures used are the Mann-Whitney test, the Wilcoxon signed-rank test, Friedman’s test and the Kruskal-Wallis test. On the other hand, SEM offers estimation methods for non-normal data. The SEM estimation methods for non-normal data are discussed in detail in Subsection 5.7.3.2.

The normality assumption could be examined graphically and/or statistically. Graphically, it could be examined through histogram, stem-and-leaf plot, boxplot, normal probability plot and detrended normality plot. For the latter, a number of statistical approaches are available to test normality such as Kolmogorov-Smirnov statistics with a Lilliefors significance level and the Shapiro-Wilks statistic, Skewness and Kurtosis. This study employs both methods, the graphical plots and statistical analysis (Kolmogorov-Smirnov, Skewness and Kurtosis) to assess the normality of the data.