A quantitative approach to SE

A large number of studies focus on one or two dimensions of engagement. In countries such as US, New Zealand, Australia and South Africa, the development and discussion of SE is heavily influenced by those practices and types of research which focus on the behavioural and cognitive

dimensions of SE. The impetus in the US came from concerns arising from an increasingly dominant debate about quality in US higher education. This debate included firstly, questions about the elevated status of reputation, resources and research over learning and teaching, and secondly, the lack of agreement on how to assess quality in HE (McCormick and Kinzie, 2014). Quality in institutions was determined by the accreditation system carried out by peers (a system faulted for its emphasis on infrastructure and resources over learning and teaching) and the introduction of the National Survey of Student Engagement (NSSE) developed by George Kuh in 1998 was intended to shift the attention from quality to teaching and learning – SE in this context focuses on the amount of time and effort students put into their studies and other educationally purposeful activities, as well as the

institution’s effort to support student success (Kuh, 2001). The development of the NSSE is drawn from Pace’s (1982) concept of quality of student effort and the work of Astin (1984) on student ‘involvement’ which suggests that more involvement equals better learning. It also draws on: theories put forward by Pascarella and Terenzini (1991; 2005) which suggested that the extent to which students engage determines their success; and on

Chickering and Gamson’s (1987) ‘Seven Principles of Good Practice in Undergraduate Education’. These principles brought together the notion that HEIs need to be cognisant of the whole student experience both in and outside of the classroom. The principles promote active and collaborative learning that respects the heterogeneity of student cohorts whilst

encouraging a challenging learning climate predicated on trust relationships. These principles, have been used and adapted over the last three decades across the HE sector and at different levels and the first two principles in particular, which encourage contact between students and faculty and

develop reciprocity and cooperation among students anticipate key elements of most ‘students as partners’ paradigms, such as those referred to earlier.

The primary activity associated with NSSE is annually surveying college students to assess the extent to which they engage in educational practices associated with high levels of learning and development. It does this using five benchmarks namely:

1. Level of Academic Challenge 2. Active and Collaborative Learning.

3. Student Interactions with Faculty Members 4. Enriching Educational Experiences

5. Supportive Campus Environment

Kuh emphasised that in benchmarking engagement it is what students do that matters:

Student engagement is defined as students’ involvement in activities and conditions that are linked with high-quality learning. A key assumption is that learning outcomes are influenced by how an individual participates in educationally purposeful activities. While students are seen to be

responsible for constructing their own knowledge, learning is also seen to depend on institutions and staff generating conditions that stimulate student involvement.

(2001, p12)

NSSE annually collects information at over 1500 colleges and universities in the US and Canada about first-year and senior students' participation in programs and activities that institutions provide for their learning and

personal development (NSSE, Indiana University, 2015). The results provide an estimate of how undergraduates spend their time and what they gain from attending college. Having been in operation for over 15 years and reviewed in 2013, the quantitative evidence base gathered has been influential in shaping strategy and policy. A key output from NSSE is the ‘high-impact practices’ identified by Kuh (2008) that educational research suggests increase rates of student retention and SE. These are:

1. First-Year Seminars and Experiences; 2. Common Intellectual Experiences; 3. Learning Communities;

4. Writing-Intensive Courses;

6. Undergraduate Research; 7. Diversity/Global Learning;

8. Service Learning, Community-Based Learning; 9. Internships;

10. Capstone Courses and Projects.

Whilst NSSE has been critiqued as a methodological tool by some over the validity of the scales and the focus on staff and student behaviours (Weller, 2016; Bryson, 2014a; Kahu, 2013) it has ‘demonstrated that its most powerful contribution may be as a conversation starter, or a catalyst for more

intensive, varied and nuanced efforts to examine educational effectiveness’ (McCormick and Kinzie, 2014, p25).

The NSSE has been modified for use in Australia and New Zealand (as AUSSE), South Africa in 2009 (as SASSE), China in 2009 (as NSSE-China) and Ireland in 2013 (as ISSE). It was used for the first time in Australia and New Zealand in 2007 with one additional dimension – work integrated learning. Some of the key SE literature from Australia focuses on a social concept of engagement. Williams (1982) developed an index of ‘Institutional Belongingness, Social Involvement and ‘alienation’. Subsequently, the focus of SE in Australia was on the first-year experience. A wide-scale survey was administered every five years since 1994 and the survey was reviewed in 2004 to include more items. The work of McInnis (2001) using the data gathered, posited that connectedness was an important factor in determining student retention and success. Debates about SE gathered momentum, in Australia, leading to further, perhaps contested, definitions of SE which emerged from a largely positivist discourse. Two examples are:

The time, energy and resources students devote to the activities designed to enhance learning at university.

(Krause et al., 2005, p31)

The concept of student engagement is based on the

constructivist assumption that learning is influenced by how an individual participates in educationally purposeful

activities. Learning is seen as a ‘joint proposition,’ however which also depends on institutions and staff providing

students with the conditions, opportunities and expectations to become involved. However, individual learners are ultimately the agents in discussions of engagement.

(Coates, 2006, p26)

Krause and Coates (2008), whilst bringing together the data from the first- year experience survey and the AUSSE to produce seven scales of SE, acknowledged that qualitative and on-going measures were required in addition to surveys in order to understand engagement. These scales include both attitudinal and behavioural dimensions: transition engagement; academic engagement; peer engagement; student-staff engagement;

intellectual engagement, online engagement, and beyond-class engagement.

Since 2012, the Australian government has introduced a mandatory student survey, the University Experience Survey (UES). The survey’s introduction calls into question the future of the AUSSE. The UES has gained responses from over 100,000 students in each of the years 2012, 2013 and 2014. The results from these surveys were used to help both HE providers and

government improve teaching and learning in Australia (UES, 2014). In 2015, the UES was renamed the Student Experience Survey (SES) to include students from non-university higher education institutions. The questionnaire remains relatively unchanged to the 2014 UES. Approximately 145,000 students participated in the 2015 SES.

According to Bryson (2014a) relying on surveys alone raises problems in conceptualising SE and cautions that such surveys lose the context-specific nature of the student experience. Trowler (2010) has also highlighted that what works in one discipline, or context or for one specific cohort of students does not necessarily work in another. Indeed, we could also go as far as stating that individual students experience SE differently. Scholars have argued that the discipline-specific nature of engagement in particular is overlooked in evaluation instruments and that disciplinary factors can determine how students engage (differently) even within institutions (Brint, Cantwell and Hanneman, 2008; Weller, 2016). Bryson (2014a) and Trowler (2010) suggest that a positivist approach to measuring SE through surveys using the aforementioned instruments does not uncover the richness and

diversity of the student experience and does not give a voice to the student at all. The affect or emotional dimension is neither sought nor captured,

something that more recent SE models and conceptual frameworks seek to address (see e.g. Kahu, 2013, Pittaway, 2012; Solomonides, 2013; Thomas, 2012). These will be discussed below, after a review of the UK approach, which in contrast to the above has mainly taken a qualitative approach to researching SE.