Introduction
This chapter presents an analysis of the findings from the quantitative data collected. The main analytical technique employed in this study was factor analysis. The suitability of the data for factor analysis is addressed early on in the chapter and this is then followed by an in-depth discussion of the factor structure extracted from the data. Additional analyses involving correlations, t-tests and one-way analysis of
variance are addressed in the second half of the chapter.
Factor Analysis Suitability of the Data for Factor Analysis
The 27 items of the TCoA-IIIA were subjected to exploratory factor analysis using SPSS version 23.0. Prior to performing the analysis, the suitability of the data for factor analysis was assessed. Notwithstanding the absence of any real consensus in relation to sample size for factor analysis, it is acknowledged that the sample must be “large enough that correlations are reliably estimated” (Tabachnick & Fidell, 2007, p.613). Field (2009) refers to a minimum amount of “10-15 participants per variable” (p.647) while Tabachnick and Fidell (2007) suggest that “it is comforting to have at least 300 cases for factor analysis” (p.613). In line with these general recommendations in the literature, the sample size of 489 in this study was deemed sufficient. Secondly, inspection of the correlation matrix revealed the presence of many coefficients of 0.3 and above, thereby fulfilling another key recommendation for factor analysis
98
0.825, exceeding the recommended value of 0.6 (Kaiser, 1974) and Bartlett’s test of Sphericity (Bartlett, 1954) reached statistical significance, supporting the factorability of the data.
Using maximum likelihood extraction and direct oblimin rotation as recommended by Costello and Osborne (2005) and Fabrigar and Wegener (2012), exploratory factor analysis (EFA) was conducted to assess the underlying structure of the 27 items on the TCoA-IIIA. The initial analysis extracted 8 factors with
eigenvalues greater than 1. Inspection of the scree plot, however, showed inflexions that would justify retaining either 3, 4 or 5 factors. Three further factor analyses were then conducted forcing 3, 4 and 5 factors respectively. The pattern matrices from each of these outputs were compared in order to find the solution of best fit (Costello & Osborne, 2005). A solution of best fit is a conceptually sound factor structure, with strong factor loadings and few item crossloadings (Costello & Osborne, 2005; Fabrigar & Wegener, 2012). Comparison of the pattern matrices suggested that the 5 factor structure was the cleanest and most conceptually sensible representation of the data. This result was supported by a separate parallel analysis which was carried out in conjunction with the main factor analysis. The parallel analysis involved conducting a Principal Component Analysis (PCA) and comparing the resulting eigenvalues with random eigenvalues generated using Watkins (2006) Monte Carlo PCA for parallel analysis. As explained by Pallant (2013), if the eigenvalue from the PCA is larger than the criterion value from parallel analysis, then this factor is retained. If the eigenvalue is smaller than the criterion value, then the factor is rejected. The results of the parallel analysis, displayed in Table 9, supported the decision to retain five factors.
99 Table 9
Comparison of eigenvalues from PCA and criterion values from parallel analysis Component Number Actual eigenvalue
from PCA
Criterion value from parallel analysis Decision 1 5.777 1.4541 Accept 2 2.869 1.3877 Accept 2 2.013 1.3414 Accept 4 1.623 1.2953 Accept 5 1.338 1.2542 Accept 6 1.164 1.2212 Reject Five-Factor Model
As illustrated in Table 10, the five-factor EFA result accounted for 40% of the cumulative variance before rotation. After rotation, the cumulative variance accounted for remained the same but this variance was redistributed across the factors (Cohen et al., 2012; Fabrigar & Wegener, 2012). SPSS does not report this redistributed
percentage variance for EFA. It does, however, provide some data on the rotation sums of square loadings and these figures give a sense of the explanatory power of the factors after rotation. The five factors extracted from the data differed from Brown’s (2006) original 4-factor model except for the school accountability factor which was fully recovered. As presented in Table 11, the five factors extracted from the data were named as follows: (1) Assessment is a diagnostic and formative tool; (2) Assessment is irrelevant (bad, ignored and inaccurate); (3) Assessment is a school accountability tool; (4) Assessment is a measurement and categorisation tool; (5) Assessment is a valid grading tool. In the case of factors three and four, the substantive loadings were negative and were reversed for ease of interpretation. This is an accepted practice within exploratory factor analysis. As explained by Thompson (2004), while referring to (Gorsuch, 1983, p.181), “because most people find it easier to think in positive terms, if the larger pattern and structure coefficients have negative signs on a given
100
factor, it is completely appropriate for the analyst to "reflect" the factor by reversing the signs of the coefficients on any given factor” (p.96). Fabrigar and Wegener (2012) also refer to this practice by explaining that “the scaling direction of common factors is arbitrary. Programs simply scale solutions on the basis of computational convenience rather than as a function of some fundamental conceptual property of the common factors. Thus, for any factor solution, it is permissible to reverse the signs of all factor loadings in a given column of the factor loading matrix. Such reversals (as long as they are applied to all elements in the column) do not affect the fit of the model or the communalities of the measured variables” (p.79).
In advance of discussing each factor individually, due regard must be given to the communality values provided for each variable (see table 12). The communality values after extraction represent the “variance in items accounted for by the extracted factors” (Fabrigar & Wegener, 2012, p.131). These values are important in that they provide information on the degree of association between variables whilst also alerting the researcher to variables which may prove problematic in the interpretation of factors. Pallant (2013) notes that “low values (e.g. less than .3) could indicate that the item does not fit well with the other items in its component” (p.206). Five of the twenty seven items in this study had communality values of less than 0.3. The items in question are Assessment has little impact on teaching (.135), Assessment is assigning a grade or level to student work (.169), Assessment places students into categories (.218), Teachers should take into account the error and imprecision in all assessment (.222), and Assessment determines if students meet qualifications standards (.244). These values will be considered as part of the discussion on each individual factor.
101