Materials

stated earlier, both of these measures were developed via literature reviews as well as staff and student interviews (Entwistle, 2005). The ETLQ was used to tap into students’ feedback on teaching (see Appendix C), facets of student engagement4, and student achievement (see

4 _{In a preliminary analysis, I assessed unidimensionality of all of the facets of engagement. Problematic items from}

both the deep approach and the deep approach orientation subscales were removed prior to the analysis. These problematic items either had cross-loading issues or standardized residuals exceeding the threshold of |4.0|.

Appendix D). For the ETLQ, items measuring students’ feedback on teaching, interest and enjoyment, student collegial support, and deep approach to learning were represented on a 5- point continuum (1 = disagree, 2 = disagree somewhat, 3 = unsure, 4 = agree somewhat, 5 = agree).

Students’ feedback on teaching. Thirty four items from the ETLQ was used to measure

students’ feedback on teaching with higher scores indicating more positive evaluations of teaching for a particular module. Six teaching-related subscales were identified on the

questionnaire cover page, including (a) aims and congruence (5 items), (b) choice allowed (2 items), (c), teaching for understanding (5 items), (d) set work and feedback (5 items), (e) assessing understanding (2 items), and (f) staff enthusiasm and support (2 items). These

subscales closely resembled those used by McCune (2003) and, in her report, coefficient alphas were good (ranging from 0.73 to 0.84). As mentioned before, the teaching items were phrased to capture overall course unit or module evaluations rather than personal leadership. Nonetheless, these items provided a foundation for understanding the links between classroom instruction and leadership. In later studies, these items would be further refined to tap into personal leadership.

Interest and enjoyment. The ETLQ was used to measure interest and enjoyment (3 items;

α = 0.77). For this measure, higher scores indicated higher levels of interest and enjoyment. The three items used to measure interest and enjoyment include, “I enjoyed being involved in this course unit”, “I found most of what I learned in this course unit really interesting”, and “I can imagine myself working in the subject area covered by this unit”. All three items indicated that students were passionate about the module, and thus engaged.

Student collegial support. The ETLQ was used to measure student collegial support (3

three items used to measure student collegial support include, “Students supported each other and tried to give help when it was needed”, “Talking with other students helped me to develop my understanding”, and “I found I could generally work comfortably with other students on this unit”. All three items indicated that students were activated in a positive way because they collaborated with and supported each other. Ideally, the student collegial support measurement items should specify a frame of reference (Macey & Schneider, 2008), e.g., students supported each other and tried to give help in this module ‘more than they normally would in the typical module’. Unfortunately, the use of a secondary dataset was limiting in this regard, and thus student collegial support was reflective of a limited view of students’ behavioural engagement.

Deep approach to learning. The ETLQ was used to measure deep approach to learning

(4 items; α = 0.64). For this measure, higher scores indicated a greater use of a deep approach to learning. The four items used to measure deep approach to learning include, “In reading for this course unit, I’ve tried to find out for myself exactly what the author means”, “I’ve tried to find better ways of tracking down relevant information in this subject”, “I’ve looked at evidence carefully to reach my own conclusion about what I’m studying”, and “It has been important for me to follow the argument, or to see the reasons behind things”. These four items were all indicative of students being immersed in the module, and thus engaged.

Student achievement. Student achievement can be measured both objectively and

subjectively. In this study, students were asked to rate their module performance. This evaluation was an interpretation of their actual achievement in which success or failure can be seen as a psychological state (Stephanou et al., 2011). Hence, student achievement was subjective because it was estimated by students. Studies showed that perceived achievement, although open to subjectivity, was related to students’ actual achievement (Stephanou et al., 2011). The ETLQ

measured students’ perception of how well they were doing in the module as a whole. Student achievement was measured using one item on a 9-point continuum with six labels, including very well, well, quite well, about average, not so well, and rather badly.

Control variables. In this study, I controlled for students orientation towards a deep

approach to learning, gender, and age.

Orientation towards a deep approach to learning. Students’ approaches to learning for a module were greatly influenced by their orientation towards learning in general. Therefore, I measured students’ orientation towards a deep approach to learning in order to determine whether instructors can deepen students’ learning beyond the students’ predisposition towards a deep approach to learning (Lizzio et al., 2002). I controlled for a deep approach orientation by using the LSQ’s measure of students’ deep approach to learning orientation (4 items; α = 0.71). This subscale was represented on a 5-point continuum (1 = disagree, 2 = disagree somewhat, 3 = unsure, 4 = agree somewhat, 5 = agree) with higher scores indicating a greater orientation

towards a deep approach to learning.

Gender. I used gender as a control variable for predicting student achievement. In using a dataset containing student records from the ‘old’ universities in England and Wales for the period 1973 to 1993, McNabb, Pal, and Sloane (2002) found that, on average, female students performed better than their male counterparts. At the same time male students were significantly more likely to obtain a first class degree. This so called gender gap in degree performance was thought to exist because of (a) differences in the type and nature of subjects chosen between genders, (b) gender differences in background characteristics associated with attainment, (c) psychological and/or biological factors, or (d) assessment biases that favoured male students.

However, the results for such reasons for the gender gap in student achievement were inconsistent (see McNabb et al., 2002; Mellanby, Martin, & O’Doherty, 2000). Based on the findings reported by McNabb et al. (2002), female students were expected to perform better than male students. Gender was coded as ‘1’ for males and ‘2’ for females.

Age. I used actual age as a control variable in predicting student achievement. The relationship between age and performance was not straightforward. Traditionally, mature students (usually ages 21 and over) were thought to be poorer performers than their younger counterparts due to age-related impairments in intellectual abilities and their lack of learning or studying skills. In a review of mature student achievement in higher education, Richardson (1994) explained that there was sufficient evidence for rejecting these stereotypes about mature students. Although mature students might be less likely to complete their programme, such withdrawal was likely due to personal or financial reasons and not academic failure. The mature students who completed their programme did not seem to perform any differently to the younger students in terms of their final degree (J. Richardson, 1994). Some researchers showed that mature students can even perform better than younger ones (e.g., McNabb et al., 2002; J.

Richardson, 1994). Additionally, McNabb et al. (2002) showed that the relationship between age and performance for their very large data-set was concave. Given the lack of justification for a relationship between age and student achievement, no relationship was expected.

4.4.4 Procedures. The ETLQ and the LSQ were distributed across 17 university