Downloaded from: http://insight.cumbria.ac.uk/id/eprint/3845/
Usage of any items from the University of Cumbria’s institutional repository ‘Insight’ must conform to the following fair usage guidelines.
Any item and its associated metadata held in the University of Cumbria’s institutional repository Insight (unless stated otherwise on the metadata record) may becopied, displayed or performed, and stored in line with the JISC fair dealing guidelines (available here) for educational and not-for-profit activities
provided that
• the authors, title and full bibliographic details of the item are cited clearly when any part of the work is referred to verbally or in the written form
• a hyperlink/URL to the original Insight record of that item is included in any citations of the work • the content is not changed in any way
• all files required for usage of the item are kept together with the main item file. You may not
• sell any part of an item
• refer to any part of an item without citation
• amend any item or contextualise it in a way that will impugn the creator’s reputation • remove or alter the copyright statement on an item.
The full policy can be found here.
Citation .
Engaging practical students through audio feedback
Practitioner Research in Higher Education Special Assessment Issue Copyright © 2018 University of Cumbria Vol 11(1) pages 87-94
John Pearson
University of Cumbria
Abstract
This paper uses an action research intervention in an attempt to improve student engagement with summative feedback. The intervention delivered summative module feedback to the students as audio recordings, replacing the written method employed in previous years.
The project found that students are keen on audio as an alternative to written feedback, perceiving it to be clearer, and more comprehensive and accessible. The use of spoken language allowed inflection and context in a manner absent from written feedback. Additionally, students stated that they were more likely to revisit feedback recordings in conjunction with their written materials, indicating a willingness to reflect upon their work. The project found that challenges of practical implementation around both the creation and dissemination of audio feedback can be a hurdle to engagement with this technique.
Keywords
Feedback; ITDEM; audio; engagement; student; practical.
Introduction
How do you encourage greater student engagement with module feedback? It is widely recognised that feedback is an essential part of learning and achievement (Bloxham and Boyd, 2007), and there is increasing evidence that alternatives to the typical written text (Hennessy and Forrester, 2014) are effective methods of engaging students with the feedback provided.
The focus for this action research project is upon increasing student engagement with feedback, in order to increase their learning, understanding and performance in future modules. The project was undertaken on a module assessment which had been delayed due to exceptional weather events in the UK during December 2015. As a result of this, alongside the inevitable pressure on students of beginning semester two projects whilst semester one assessments are still outstanding, I was keen to be able to produce feedback on the assessments quickly and in an accessible manner.
The foundation degree in Technical Theatre involved in this study attracts students from a widening participation background, with a particularly practical programme of delivery. Students work practically for 80% of the programme, and are far more likely to be learning in theatre spaces or workshops by constructing sets or rigging lights than they are to be in a classroom environment. This provides particular challenges to staff in engaging students in the more academic elements of their foundation degree, including myself. Providing students with informative and engaging feedback is one critical element for improving their learning process during these modules.
Literature Review
Alternate methods of feedback for enhancing learning & understanding
programme delivery. Whilst some studies have suggested that the summative results for learners may change little when a change of feedback format is undertaken (Gleaves and Walker, 2012), there is also sig ifi a t e ide e that lear ers opi io s a d engagement with feedback are appreciably improved by the use of audio (Lunt & Curran, 2010). More recently, McCarthy (2015) has considered and advocated for a mixture of multimedia feedback formats, including video-based material.
Gould and Day observe that nursing students valued audio feedback as more detailed, personalised and supportive than written feedback (2013:554), but also noted potential issues and support needs for some students (2013:564). Conversely, Chalmers et al. (2014) noted that in their quantitative study of science students, they did not find a quantifiable difference in the outcome of test results following different formats of feedback (p.70).
Time cost & efficiency
Returning feedback to students in a timely manner is essential for learning; the longer a student waits from submission to feedback, the less relevant the feedback offered may become. Audio feedback is one method by which this timeliness can be addressed. As Carruthers (2015) noted in relation to Rodway-D er, K ight & Du e s stud , the audio feedback received was timely, recognised as feedback, understood and facilitated feed-forward learning.
Similarly, Lunt and Curran (2010) posited that using audio feedback for formative and summative work could overcome the timeliness issue around returning feedback to students, something corroborated by Hennessy and Forrester (2014).
Ongoing engagement with feedback
When considering learning through feedback, student reflection on the material points raised is very i porta t. Parki et al. fou d that stude ts did alue hard- op feed a k […ho e er…] the majority rarely referred back to it after an initial read-through a d so its alue as tra sitor , p.966). In contrast, Carruthers et al. (2014, p. otes that o e of the ke features of audio feedback is the ability to re-access it, which is hugely significant for factors such as clarification, re ie , a ilit to ide tif areas that a e i pro ed upo , et … This work investigated whether audio feedback was effective by virtue of being revisited as Carruthers suggests.
Methodology
The project utilises an action research approach re o e ded Li Norto . Norto s ITDEM acronym refers to a five-part process: Identifying a problem, Thinking of ways to tackle it, Doing it, Evaluating it, and then Modifying your future practice based upon the results (2009:70).
In this case, ITDEM is being applied to the topic of providing feedback to a group of second year undergraduate (level five) foundation degree students studying Technical Theatre. In this case, the Evaluation stage is being undertaken using a focus group analysis.
During semester one of their second year, my students study Designing Performance, a module introducing theatre design principles. The module involves a summative assessment by presentation to their tutors and peers, upon which feedback is given. The presentation is intended to simulate the situation whereby a designer would pitch their conceptual ideas for the realisation of a show to a director.
consequently improve the quality of the design work produced for the shows undertaken during semester two.
Change
The major change undertaken during the project is to alter the method of feedback delivery from written PDF, to recorded audio delivered via the Blackboard VLE.
When planning the format of the feedback recording, I have included the grade of the work as the last item of substance within the recording as advised by Lunt & Curran (2010), and avoided writing it within the portal students use to access the feedback material. This has the effect of requiring the students to either skip through the audio track to reach the grade, or to listen through the file to the e d. I ha e also adopted a u er of re o e datio s fro He ess & Forrester s 4, p. considerations for staff giving audio feedback; a quiet room, consideration to tone of voice, supportive but critical language and explaining and clarifying holes and gaps in the material presented.
Alongside local anecdotal evidence, a body of research suggests that students are significantly grade-centric and can fail to engage effectively with feedback provided (Nesbit & Burton, 2006). When faced with written feedback, students may look at the grade and skim-read the page of written feedback at best. This Action Research project seeks to increase engagement with feedback by changing the format and delivery style in order to emphasise the substantive content of the feedback.
Ethics
The process was managed in accordance with BERA guidelines (2011), and ethical approval was gai ed through the U i ersit s ethi s pro edures. E er stude t as pro ided ith detailed feedback, and invited to request a written copy of their feedback should they wish to do so.
All students who volunteered to take part in the subsequent focus group completed a consent form, which was explained to them before completion. Participants understood that they were free to ask questions, that their anonymised responses would be analysed for the project, and that they were free to withdraw from the focus group study at any time.
Location and format
The Designing Performance presentations took place within the stude ts usual teaching and design
space at the university.
A focus group was chosen as the data collection format to encourage discourse between the students involved. A group discussion and learning environment is a common environment for these students, and the familiar environment is helpful in encouraging honest engagement and feedback. The group was composed of six students from the cohort who responded to my request for volunteers to take part: just over half the cohort.
The venue for the recording of the audio feedback was my office. Practically, the recording of feedback was undertaken using technology already at my disposal; audio recording using the built-in microphone on an Apple iMac, with Adobe Audition software. Processing of the audio was minimal with a preset speech processing effect applied to the entirety of each recording, and editing limited to removing any false starts and extended silences.
Delivery
students for accessing feedback already, and ensures that only the student concerned can access the recording intended for them.
The project intervention occurred during January 2016, when the Designing Performance module assessments took place. Following ethical approval, data collection for the project was by focus group, which took place in early May 2016. My audio recording of the focus group was subsequently transcribed for analysis.
Data and Analysis
Several negative points were raised at the very beginning of the focus group; the first relating to the e perie e of heari g their tutor s oi e through headphones, and the second in relation to the technology and ease of accessing the feedback files.
This perception may well be related to the unfamiliarity of the situation – the very close quarters of a feedback recording on headphones is unusual to this group of students. It is very possible that further use of audio feedback in this manner could lead to a greater familiarity with the situation, a d stude ts e o i g o forta le ith heari g their tutor s oi e i this a . I raised this poi t later in the discussion with the same students, and through discussion it did seem that their concerns were allayed somewhat through repeated listening to the recordings.
Technical Issues
The major, substantive negative issue for students with the feedback was in accessing the recordings. Most students in the focus group use mobile devices or tablets to access the VLE system, and were unable to successfully listen to the recording through these. One student relayed the experience of hearing the first few sentences of the recording, only to have the sound return to the start of the file; presumably due to download limitations on his device.
Following on from student observations of issues accessing the feedback on mobile devices, some students specifically mentioned that accessing and downloading worked without problem from a laptop or PC. The issue for some of these students was related to file management on their computer, where the file would be added (automatically or otherwise) to their music manager, and then appear in playlists.
With the propensity of music managers such as iTunes to automatically add files to their libraries, a number of students reported unexpectedly having their feedback played to them. One recounted that whilst testing sound systems as a DJ, the naming scheme used for the file meant that it appeared at the top of the list in his iTunes app and would occasionally be triggered whilst setting up in a club! Another student reports that his feedback file ended up synced with his mobile device, and startled him in the middle of a train journey to Liverpool.
Engagement & Content
Engagement with feedback appears to be significantly increased, along with perception of that feedback. Some students reported the feedback to be far more accessible in audio format and
found it to be a lot easier [to understand] instead of just reading through the documents .
Students perceptions of their engagement with the content of the feedback is remarkably improved by the audio feedback format. Members of the focus group referred to a blasé attitude to written feedback:
ou go straight to the arks a d that s it, ou re do e
and:
i o pariso to a pie e of paper, liste i g to it is a lot etter e ause ho a ti es […] I e literall just gla ed at it, just go e straight to the arks a d go e I e passed, great . I pretty sure a lot of us ha e just do e that…
By contrast, referring to audio feedback students relayed that they:
liste [ed] to it agai a d the […] took i hat as a tuall ei g said i stead of just reacting to it
Effort and input
During the focus group, students reported feeling that I had spent more time, and invested more effort in marking the work. Students felt that with paper feedback:
ou do t k o if ou e [i di ati g self] do e it, or [a colleague has] done it, or someone outside s do e it hereas ith audio feed a k ou k o that ou e go e through the ork.
In contrast to the skim reading confession above, with the audio format there is a sense from the students that the tutor has both engaged and invested more in producing the recording than they would with writing text feedback. Additionally, there is a suspicion from the students that it almost feels like you could put one copy and just change the name at the top of the paper ork he
riti g a page, hereas he it s spoke ou […] tailor it ore to the perso .
A related point of note was that as the discussion progressed, students expressed that they would have less of an issue with approaching me for a tutorial to discuss their work, as they felt the audio feedback initiated discussion in a way which the written feedback did not.
This perception of staff investment in their work, no matter the amount of time and effort involved, is undoubtedly important to building a sense of community and thereby student achievement longer term. It also leads to another observation from the group; that praise received through audio feedback was taken to have more meaning than the same words did when written down.
Access
Students expressed a perception that in addition to audio feedback being clearer to understand, it was easier to re-access feedback delivered in an audio format. Since both the audio and written feedback files were originally accessed through exactly the same interface, this may well have more to do ith stude ts per eptio s of the effort le el that should e i ol ed ith ea h t pe of file than the practicality. Perhaps it is not considered a burden to re-listen to an audio file to listen again, where it is to do the same to re-read a written page?
When questioned, four out of the six students in the focus group directly confirmed they had intentionally re-accessed and listened to the feedback received since the first time they heard it. Commonly, students reported listening again to the feedback once they had their work in front of them so they could listen and also look at the documents at the same time. This relates well to the experience of Carruthers et al, who list the ability to re-access feedback as important to students in their study (2014:6). They note that whilst 28% of students surveyed had re-accessed their feedback at that poi t, 69% indicated that they would be likely to refer back to this feedback when preparing other pieces of coursework.
Feedback formats
One student expressed the view that a one-to-one tutorial would be a preferable alternative to audio feedback, relating back to the ability to respond to comments made. Offering this kind of feedback by default would have a number of i pli atio s. Positi el , it ould address the stude t s immediate concern that they could not respond to points raised, however there are a number of potential pitfalls with this method as well. Unless either the tutor or the student makes a record of the tutorial, there is no concrete material to refer back to later. Giving feedback in this way allows no time for the student to reflect on their work before considering a response. Such a method would also prevent the student directly revisiting their feedback comments later.
I sort of had a argu e t ith ou [laughter] the I realised it as t real…
As several of the students who initially raised negative feelings towards the feedback observed that they wanted the ability to respond, I believe that time to reflect prior to being able to discuss feedback with their tutor is constructive. Future audio feedback could build upon this by actively inviting a student to discuss any questions they have.
Readability
Related to the point above about wanting to argue or discuss their feedback as they listened to it, it transpired from the focus group that some students reading written text hear it in their own voice. By contrast, with audio they hear the material in the voice of the recorder, their tutor. This may make for increased understanding when taken alongside the inflection and tone of voice you add with an audio file.
Practical Considerations
Towards the end of the focus group discussion, students reiterated the technical issues they had faced with accessing the files. Essentially, it seems that mobile devices and apps did not work for accessing the audio file downloads; it was necessary to go to a web browser on a computer to get hold of the file.
This practicality of access is one that could be problematic in terms of adopting the technique locally in a wider range of feedback situations. All students have access to laptops or computers through the University so access is possible, however the majority will attempt to use a tablet or mobile device in the first instance.
Fro the tutor s ie poi t, e perie es are i o trast to those of Carruthers et al. (2015) who tell us that they did not particularly find any noticeable time savings (p.366), in that it took significantly less time for me to compose rough notes into an audio file than into a presentable written page of feedback notes. It is worth noting in passing however, that my background in digital audio provides a platform to work from which cannot be assumed of most lecturing staff.
It was important to me that the feedback was completed quickly, partly due to the rescheduling of the initial assessment from December to the beginning of January. Following the presentations on a Friday, and having consulted with the colleague who was second marking alongside me to produce rough notes for each student, I was able to publish the audio feedback the following Monday afternoon; one and a half working days after the assessments took place. This compares favourably to the university-mandated requirement to return feedback within twenty working days, preferably fifteen.
Conclusion
This project finds evidence of significant positive effects when implementing audio feedback with small cohorts. It must be acknowledged that this is a small-scale qualitative study with a particular focus on students with a pre-existing technical mindset, and that one should be wary of extrapolating too greatly to the wider student community. It should be noted that there is a body of evidence with somewhat similar findings in relation to large cohort audio feedback, such as that by Fitzgerald (2011).
Another point worth noting is that the students involved deal with digital audio as part of their programme of study, and so are at least somewhat familiar with managing audio files.
Students emphasised a number of very positive points, not least simply the quality and quantity of feedback that they felt they received. This quality, alongside evidence that students have listened to the entirety of the feedback more than once, is the most important finding of the project.
The note of caution within the project is similar to that found by other projects; technical issues can impede student access to the material, which may lead to a negative view of the format. As Carruthers et al. (2014) note within their reflection, thirty-t o per e t […] i di ated that the had experienced some initial technical difficulties (2014:9).
The findings of this project serve to emphasise indications from the existing body of knowledge in relation to implementing audio feedback for student learning. With appropriate technical systems and support in place, students perceive the actual feedback delivery by audio overwhelmingly positively. However, should those supporting systems be inadequate and require students to jump through hoops in order to access the material, an otherwise positive system can be compromised.
I can, however, conclude that future research into common, accessible yet secure platforms for the dissemination of feedback would be a boon in aiding the adoption of this format more widely.
References
Bloxham, S. and Boyd, P. (2007) Developing Effective Assessment in Higher Education: A Practical
Guide. McGraw-Hill Education (UK).
Carruthers, C., McCarron, B., Bolan, P., Devine, A. and McMahon-Beattie, U. 4 Listening and Learning: Reflections on the use of Audio Feedback. An Excellence in Teaching and Learning Note', Business & Management Education in HE, 1(1), pp.4-11.
Carruthers, C., McCarron, B., Bolan, P., Devine, A., McMahon-Beattie, U. and Burns, A. (2015) I like the sound of that – an evaluation of providing audio feedback via the virtual learning environment for summative assessment', Assessment & Evaluation in Higher Education, 40(3), pp.352-370.
Chalmers, C., MacCallum, J., Mowat, E. and Fulton, N. (2014) 'Audio feedback: richer language but no measurable impact on student performance', Practitioner Research in Higher Education, 8(1), pp.64-73.
Fitzgerald, R. (2011) 'To What Extent Does a Digital Audio Feedback Strategy Support Large Cohorts?', Proceedings of the European Conference on e-Learning, pp.274-283.
Gleaves, A. and Walker, C. (2013) 'Richness, redundancy or relational salience? A comparison of the effect of textual and aural feedback modes on knowledge elaboration in higher education students' work', Computers & Education, 62, pp.249-261.
Gould, J. and Day, P. (2013) 'Hearing you loud and clear: student perspectives of audio feedback in higher education', Assessment & Evaluation in Higher Education, 38(5), pp.554-566.
Hennessy, C. and Forrester, G. (2014) Developing a framework for effective audio feedback: a case study,Assessment & Evaluation in Higher Education, 39(7), pp. 777-789.
Lunt, T. and Curran, J. (2010) Are you listening please? The advantages of electronic audio feedback compared to written feedback', Assessment & Evaluation in Higher Education, 35(7), pp.759-769.
McCarthy, J. (2015) 'Evaluating written, audio and video feedback in higher education summative assessment tasks', Issues in Educational Research, 25(2), pp.153-169.
Nesbit, P. and Burton, S. (2006) 'Student justice perceptions following assignment feedback',
Assessment & Evaluation in Higher Education, 31(6), pp.655-670.
Norton, L.S. (2009) Action research in teaching and learning: A practical guide to conducting
pedagogical research in universities. London: Routledge.
Parkin, H., Hepplestone, S., Holden, G., Irwin, B. and Thorpe, L. (2012) 'A role for technology in e ha i g stude ts e gage e t ith feed a k', Assessment & Evaluation in Higher
Education, 37(8), pp.963-973.
