Addressing the ‘language’ in e-assessment practice

It has long been observed in the literature that there is a lack of consensus in education domain regarding the terminology for technology related practices (see Chapter 3.2), with some practices often described differently depending on the context (Gikandi et al., 2011; Guri-Rosenblit, 2010; Moore et al., 2011; Sim et al., 2004; Timmis et al., 2016; Twigg, 2001). So, it only serves to hold true to see a similar phenomenon exists at OldU when speaking to academics and PS staff, as they describe e-assessment fulfilling a variety of roles (see Chapter 5.2). For PS staff, who see e-assessment as a method of assessment, their definition is

136 consistent those from the turn of the century, where e-assessment is often

contrasted with paper-based assessment, with the benefits and drawbacks e- assessment presented to aid readers in the decision making process (c.f. Bull & McKenna, 2004; Clariana & Wallace, 2002; Ricketts & Wilks, 2002; Sim et al., 2004).

But the biggest variation in understanding is between academics, with clear boundaries being drawn dependent on the faculty that they reside in. Academics in FSoSH tend to align with the PS definition, whilst FE&S view it as a tool that supports the process of assessment – an academic activity - in that technology facilitates the grading of student work, or the detection of plagiarism. Such a definition is consistent with those presented in later literature, as the benefits of technology use are presented in terms of the efficiencies for those involved (c.f. Cox, Schleyer, Johnson, Eaton, & Reynolds, 2008; Derrick & Stevens, 2005; Youmans, 2011). In FLHS, where a local e-learning team comprising academics and PS staff resides, there appears to be even more variation. For those

operating in a research role, there is a view that whilst e-assessment is a method, it also involves the automated marking of student work by a computer, often seen in the use of online MCQ tests which are used within the modules they interact with.

For those academics within a specific e-Learning academic role in FLHS, who are praised by many across the institution for supporting e-assessment at the local level and aiding policy enactment, they vary from viewing e-assessment as a method, through to e-assessment as a process, which is much more consistent with the work on the electronic management of assessment (EMA) process derived by JISC, in collaboration with the Heads of e-eLearning Forum (HeLF) and the University and Colleges Information Systems Association (UCISA). HeLF suggested that EMA covers four processes, including: e-submission, e-marking, e- feedback and the electronic return of work (Newland et al., 2013), whilst Ferrell and Gray (2013), writing for JISC, built on the work of the TRAFFIC Project

137 (Manchester Metropolitan University, 2014), have since expanded the term to encompass the whole assessment life cycle:

“The term electronic management of assessment (EMA) is

increasingly being used to describe the way in which technology is used across the assessment life cycle to support the electronic submission of assignments, as well as marking and feedback.” (Ferrell & Gray, 2013, What is EMA, para. 1)

Many UK HE institutions are now adopting the JISC definition of EMA as part of their own formal documentation (c.f. Canterbury Christ Church University, 2017; University of Bristol, 2017; University of Central Lancashire, 2016; University of Liverpool, 2016; University of Worcester, 2015; York St John University, 2015), given the focus on the whole assessment lifecycle (Figure 6.1). However, Voce (2015) makes an interesting observation of EMA, noting that it appears to be only concerned with text based assignments that require human input. Whilst Voce does not expand on why tests/quizzes that can be automatically marked by a computer are excluded from the definition, it does keep in line with JISC’s description and guidance for EMA. And although the JISC documentation and guidance does not specifically discount them, there is no real mention of them being considered, with much of the focus in relation to automation is regarding the benefits of common comment reuse and batch upload of student grades (Ferrell & Gray, 2013).

So why not use the term ‘EMA’ as the basis for this thesis? Indeed, several participants indicated during their interview that they would have like to have received the Interview Protocol (see Appendix Two – Interview Protocol) in advance, giving them the opportunity to refine their thoughts ahead of time. In this instance, I refer to my earlier discussions on the synonymous use of terminology in the field, and why this, which one could argue, may have introduced an element of bias into the research. Whilst EMA is becoming commonplace in the literature, Ferrell (2014)

138 Figure 6.1 The assessment and feedback lifecycle (adapted from Ferrell & Gray, 2013)

notes that the term is “often used interchangeably” (p. 5) with the term ‘e-

assessment management’ (EAM). Whilst EMA is a much broader term to cover the use of technology to support assessment related processes, Ferrell argues that EAM considers only the management of digital assessments, evident in EAM literature, which focuses on the electronicsubmission of assessment (c.f. Ellis, 2012; Ellis & Reynolds, 2013; Farisi, 2013; Hafeez-Baig, De George-Walker, Gururajan, & Danaher, 2011; Skrabal, Turner, Jones, Tilleman, & Coover, 2012).

To further ‘muddy the waters’, Canterbury Christ Church University’s (2017)

documentation for supporting EMA is entitled “electronic assignment management”, which is perhaps appropriate given that their focus is on the electronic submission of work.

139 Figure 6.2 Using Google to search for content related to 'electronic assessment' (left) and 'e-assessment' (right)

Regarding the participants’ request for the questions in advance of the interview, I can perhaps further illustrate the synonymous nature of terminology. A simple search on Google, at the time of writing, for ‘electronic assessment’ returned information about EMA, with JISC’s assessment lifecycle appearing at the top of the list. Conversely, a search for ‘e-assessment’ returned specific definitions more in keeping with definition used as the basis for this thesis (see Chapter 3.2). Itis also interesting to see that that the term ‘electronic assessment’ is used in the definition of ‘e-assessment’, creating an, almost, circular ambiguity (Figure 6.2). Whilst one could provide an explanation of this phenomena based on the

intricacies search engine functionality and the use of key words, what is evident here is that depending on the term used in the initial search, two very different paths could have been forged by the participants in, perhaps, a naïve quest to clarify their understanding in preparation for participation in this study. Whilst the variation in understanding is a key issue, the added confusion amongst participants regarding how academics and central PS departments should and could work together in the design and implementation of e-

assessment is also of interest. Voce (2015) has suggested that the role of support staff is not well documented in existing e-assessment policies, thus potentially

140 masking the important role that they may indeed play from academics also

involved in the process. This is indeed evident from OldU’s CS team, who noted

that historically, they were often left out of conversations relating to the use of technology in TLA activities, until the very ‘last minute’, whilst there was also evidence that PS staff supporting the implementation of e-assessment believe they are viewed as ‘inferior’ by their academic counterparts. However,

Whitchurch (2013) has suggested that there is growing body of evidence for co- operation between academics and PS staff in the support of university functions, which Duke (2003) believes is happening as universities recognise they need to engage in collaborative work for innovation to take place. Through a more collaborative process between academic and PS staff in the design and implementation of e-assessment, Bulpitt (2012) argues that it offers the

potential to bring together fragmented aspects of an institution to offer a more holistic, and less conflicting, approach to practice.