Post study surveys

Two post study surveys were carried out with the sample participants: one for those receiving audio feedback by audio player (see Appendix D.2), and another for those students receiving video feedback by animated avatar (see Appendix D.3). The differences between the survey questions were minimal. In all questions but one, the wording was only changed to make it appropriate to the media in question ie: where one survey said, ‘audio player’ the other said ‘avatar’. There was only one other difference, where students receiving feedback by avatar were additionally asked if they would like to select their own avatar.

Having only a quarter of the student cohort assigned to be marked by the marker using audio feedback (50), this number was then further reduced by non-submissions or unacceptable submissions (31 submissions to be marked). This group was then split in two with each group receiving a different format. Response numbers were therefore expected to be low, although the percentage response rate is similar to that of the pre- study survey (26.66%).

Audio player Avatar Received feedback in format (count) 15 16

Responded to survey (count) 4 5

Response rate (%) 26.66% 31.25%

Table 5.1: Audio feedback - Sample size and response rates by delivery format

Students receiving feedback via audio player

All respondents receiving feedback by audio player found their feedback easily. One student could not listen to it, and another student found that the player did not work when off campus. Perhaps the student who could not listen was also off campus.

Only one student said they would not like to have audio feedback by audio player in future. They prefer to be able to scan the text as a whole, rather than needing to listen to the whole thing to find particular points. This same student does not think audio

feedback will improve the chances of review or applying suggested changes to future work.

The other 3 respondents would prefer audio feedback in future and do think audio feedback will improve the chances of review, or applying suggested changes to future work. When invited to comment one of these respondents suggested the use of screen cast video as preference to audio, “Maybe a screen video, with your cursor highlighting

problems as you talk through what we did wrong and right“.

Students receiving feedback via avatar

Survey respondents receiving audio feedback via animated avatar were not so positive. Lack of compatibility with the Chrome (Google 2008) browser software was a significant contributor, since it was favoured by computing students of the day. When asked about which format they preferred for future feedback, the same three students who had difficulty accessing it, expressed a preference for written feedback. Of those who

preferred only written feedback two expressed contentment with what they were used to,

“I am comfortable with the written feedback given”. Only one student preferred the

avatar, and another wanted to keep the text with the avatar. When asked if students would prefer feedback by avatar in future two students expressed concern for how long it would take to return feedback to students, anticipating that it would take longer than writing text, yet the feedback in this case was delivered in the normal time frame at the same time as the rest of the cohort.

Two students who successfully accessed their avatar feedback without difficulty both said they thought it would improve the chances of both reviewing feedback and applying learning from it to future work. Although it was thought that students might enjoy

selecting their own avatar, none did so, and when asked about it in the survey, none wanted to.

5.5.6 Conclusion

The evaluation of the original propositions are shown in Table 5.2, and these are then explained in more detail. Of the original propositions it has been possible to show that creating audio feedback for assessments of programming code is feasible (P1) and that students do prefer audio feedback (P2) as long as it is easily accessible. It was expected that computing students might enjoy the presentation of feedback by animated avatar, however, this did not appear to be the case (P3). The experience may have been tainted by implementation problems. Once the first student had been marked by audio, and the

Proposition Valuation P1) Audio feedback on programming code assignments is technically

feasible

TRUE

P2) Students prefer audio feedback delivered by an audio player em- bedded into the feedback area of the VLE, to digital text feedback

TRUE

P3) Students prefer audio as feedback delivered by avatar to both text feedback and audio feedback delivered by embedded audio player

FALSE

P4) There will be no increase in time spent by staff completing the mark- ing process

TRUE

P5) It is possible to deliver greater depth and detail by using audio than by text

TRUE

Table 5.2: Audio Feedback: Evaluation of propositions

process was familiar, the time taken to create the feedback for the other students was no longer than it took to deliver the written feedback (P4), and sometimes shorter, although usually only insignificantly shorter. It was anticipated that pasting in the links to the media might take a while, or that waiting for uploading of media may extend the process, but the upload can occur as a background process, and with shorter recording times than writing times, pasting in the links still had no impact on the overall workload.

The idea suggested in literature, that it is possible to deliver more detailed feedback using audio than written word (P5) certainly occurred in this study. In this case the text was used as a basis for the audio and there was a natural inclination to elaborate. This did not necessarily result in more detail per se, but it was felt that it lent itself to a more encouraging tone, rather than sticking to the exact words of the written version.