Component structure

variables. Using the test sample (N = 1,361), I conducted a PCA on 34 teaching items (see

Appendix C) with oblique rotation (Promax). Various tests were used to determine the number of components to extract (i.e., Kaiser’s criterion, Velicer’s Revised Minimum Average Partial (MAP) test, and Horn’s parallel analysis). With no consensus between the tests, I examined each

5 _{Missing data ranged from 0.1% to 0.7% for all variables. Little’s missing completely at random (MCAR) test}

confirmed that an imputation method can be used by illustrating that there was a random pattern of missing data (𝜒2

was 77.07 (104), p > 0.05). To calculate missing data for the quantitative variables, the expectation maximization (EM) approach was used.

of the solutions recommended by each test, i.e., three-, four-, five-, and six-component solutions were tested. A three-component structure produced the clearest structure with stronger

components than the other alternatives and was seen as theoretically closer to the structure proposed by Bolkan and Goodboy (2011). For the PCA, component loadings were expected to be greater than .30 because the very large sample size made small loadings statistically meaningful (Field, 2009).

Several re-specifications were conducted and 14 items were deleted in an iterative process due to poor representation by the component structure (see Appendix C for a list of all items). In the first iteration, items 3, 5, 7, 20 and 34 had communalities that were less than .3, and thus were removed because of their poor representation by the component structure. After these modifications, the PCA was re-ran a second time and the re-specified structure illustrated that all items had communalities greater than .3. Next, Cross-loading issues were removed in an iterative process with the most problematic cross-loading being removed in each iteration. A cross-loading meant that a given item loaded at .32 or higher on two or more components

(Tabachnick & Fidell, 2005). The most problematic cross-loading was clearly item 31 because it loaded on all three components. Item 14 also had dual loadings. After removing these two items, a third iteration revealed that items 11, 30, and 38 had problematic cross-loadings, and thus these items were removed. In this third iteration, items 25 and 36 also had cross-loading issues. Of the two latter items, item 25 was important for theoretical purposes because it was the only item that tapped into the support aspect of consideration leadership behaviour. In contrast, item 36 covered a topic that was already addressed by other items (i.e. assessment). For these reasons, item 36 was also dropped in this third iteration. After removing these items, the fourth iteration was

markedly clearer with only item 12 not significantly loading on any of the components. Therefore, this item was removed from the analysis.

As a final check for the unidimensionality of each component, an orthogonal rotation was employed and the results were compared to the nonorthogonal solution. The Varimax rotation revealed that items 10, 23, and 33 were the most problematic. The content of items 10 (i.e., students being prompted to think about their learning and ways to improve) and 33 (i.e., students seeing how set work matched what they were supposed to learn) were somewhat represented by other items in the component structure. However, item 23 captured a unique behaviour of staff sharing their enthusiasm with students and, for that reason, this item was kept. In a fifth iteration, items 10 and 33 were removed, and the resulting component matrices provided an acceptable structure using the Varimax procedure and a clear simple structure for Promax procedure. This final re-specified component solution was represented by 20 items and explained 48.67% of the variance. Horn’s parallel analysis was repeated for the reduced number of items and, again, three components were confirmed as appropriate.

I then conducted a PCA on the remaining 20 items using the validation portion of the sample (N = 1,343) (See Table 7). Both the communalities and component loadings for each variable between the test and validation samples were similar. However, one cross-loading issue with item 39 was highlighted in the validation sample. The primary loading for this item

switched from intellectual stimulation to consideration. The reason for this cross-loading is that item 39 taps into set work as well as making connections to knowledge and experience. These two aspects are indicative of consideration and intellectual stimulation respectively. In addition to the cross-loading issue, item 39 was the weakest loading variable on both of its components and had a relatively low communality. Hence, I deleted item 39. Overall, as shown in Table 7,

Table 7

Summary of (a) Test and Validation Subsamples’ Factor Loadings and Communalities for Principal Component Analysis With Promax Rotation and (b) Total Sample’s Standardized Factor Loadings for Confirmatory Factor Analysis of ETLQ’s Teacher-Evaluation Items (Study 1)

Principal component analysis Confirmatory factor analysis

Test components Validation components Constructs

Item descriptions 1 2 3 C 1 2 3 C Co DC IS IR

The feedback given on my set work helped to clarify things I hadn’t fully

understood .90 .62 .67 .50 .50 .25

The feedback given on my work helped me to improve my ways of learning

and studying .87 .60 .69 .54 .50 .25

Staff gave me the support I needed to help me complete the set work for this

course unit .82 .60 .66 .49 - -

I was encouraged to think about how best to tackle the set work .57 .40 .50 .41 - -

Staff were patient in explaining things which seemed difficult to grasp .50 .43 .78 .66 .44

Staff helped us to see how you are supposed to think and reach conclusions

in this subject .48 .48 .69 .57 .73

.54

Students’ views were valued in this course unit .46 .40 .71 .49 .66 .43

Staff tried to share their enthusiasm about the subject with us .40 .38 .71 .50 .63 .40

It was clear to me what I was supposed to learn in this course unit .88 .57 .79 .56 .55 .30

What we were taught seemed to match what we were supposed to learn .81 .60 .74 .57 .71 .50

The topics seemed to follow each other in a way that made sense to me .81 .54 .73 .52 .57 .32

The course unit was well organised and ran smoothly .63 .46 .63 .48 .63 .40

How this unit was taught fitted in well with what we were supposed to learn .59 .57 .60 .55 .75 .56

The handouts and other materials we were given helped me to understand

the unit .47 .36 .48 .35 .53 .28

Plenty of examples and illustrations were given to help us to grasp things better

.40 .38 .36 .36 - -

The teaching in this unit helped me to think about the evidence

underpinning different views .82 .61 .74 .61 .70 .50

This unit has given me a sense of what goes on ‘behind the scenes’ in this

subject area .77 .55 .77 .56

.61 .37

This unit encouraged me to relate what I learned to issues in the wider world .76 .47 .71 .52 .57 .33

The teaching encouraged me to rethink my understanding of some aspects

of the subject .66 .47 .30 .46 .47 .63 .40

Variance extracted (%) 38.43 39.12 39.85

Construct reliability .79 .79 .73

Note. Loadings less than .30 are not shown. C = communalities; Co = Consideration; DC = Direction and congruence; IS = Intellectual stimulation; IR = Item reliabilities.

the validation sample showed very good support for the component structure that was derived from the test sample.

The components were named as follows:

Component 1. Consideration: The items that loaded on this component related to constructive feedback and support given on assessments; staff’s support in the classroom, including patience and helping students to think; valuing students’ views; and sharing enthusiasm with students.

Component 2. Direction and congruence: The items on this component related to students being taught in an organized manner in order to achieve learning objectives. To guide students towards learning objectives, students were provided with examples, handouts, and other materials.

Component 3. Intellectual Stimulation: The items that loaded on this component contained some element of students being encouraged to think and be aware of varying evidence and issues in the subject matter. Students were also encouraged to not only apply their learning to the wider world, but also to challenge their

understanding of subject aspects.

The three emergent dimensions reinforced the notion that transformational leadership was applicable to the HEI module context albeit in limited ways. First, consideration was similar to Bass’ individualized consideration dimension because it included leader behaviours that focused on relationship building and follower development, e.g., support and constructive feedback (Bass, 1990). However, consideration included more generalized behaviours instead of

These generalized consideration behaviours entail more one-to-many than one-to-one communications, and the former may be more relevant to the distant HEI module context. Second, intellectual stimulation was identical to Bass’ conceptualization of this dimension because it described leader behaviours that invigorated followers’ thought processes as well as developed their ability to apply learning to tackle problems (Bass, 1990). Third, direction and congruence aligns with Rafferty and Griffin’s vision dimension reflecting the short-term nature of HEI modules. That is, for direction and congruence, students reported on the degree to which they were taught to match learning objectives, whereas vision describes the degree to which followers inspired towards achieving a long-term goal. Overall, H1 was supported based on EFA.

4.5.2 Confirmatory factor analysis. Confirmatory factor analysis (CFA) was