Pre-study survey

Firstly, an anonymous survey was carried out across the cohort of 200 students enrolled on the programming unit (see Appendix D.1), to gauge student perception of the unit and current feedback methods.

The survey was developed to cover aspects which were considered influential on student attitude to the feedback. The number of questions was limited by the number available free of charge on the survey tool. To mitigate against this limitation, two free format comments boxes were included at key points allowing students the opportunity to express themselves, in case they had not been given the opportunity to get across points important to them. Students might have also used this space to express

frustration with poor questions, however there was no indication of this. Multiple choice answers were used for 8 of the questions. The order of these was organised along a scale of positive to negative. The order was reversed on some questions to make it easy to identify where a student selected options in the same position vertically, indicating superficial participation and lack of sincere consideration of the questions. For similar reasons, in an attempt to ensure considered opinion was obtained, there was no neutral

option available as an answer to questions.

It was felt that it was important to gauge student attitude to the unit, in case this was impacting the results pertaining to feedback. Apart from general attitude towards

programming as a subject in the context of the unit, other factors identified as potentially impacting student opinion of the feedback were a) student opinion of their own progress compared to the pace of the unit and b) the grades awarded.

Fifty-two students responded taking on average 2 mins 43 seconds each. When asked about the pace of the unit (see Appendix D.1, Q1), 45% of students felt they were keeping pace or racing ahead of the unit delivery, and only 5 students out of 52 respondents felt they were getting left behind. When asked about their grades (see Appendix D.1, Q2) students were even more positive with 82.3% of students feeling their grades were ’ok’ or ’good’. The student attitude to the unit is indicated as positive on the whole, with 74.51% claiming to ‘love’ or ‘like’ it. Another 19.61% are putting up with it safe in the knowledge that if they can pass the first year they don’t have to do anymore programming to pass the degree. Only 3 students say they ‘dislike’

programming. Although there is a natural tendency towards telling staff what they want to hear, this survey is anonymous, and students could have taken this opportunity to express frustration. Students were considerably less positive about their progress, and more positive about their grades, indicating a considered response. Therefore, it was unlikely that negative attitude to feedback was significantly being tainted by the attitude to the unit.

It was important to find out, as a point of comparison, how students felt about the electronic text feedback on their programming assessments. When asked about how useful students found their feedback (see Appendix D.1, Q4) 47% found it at least some use and another 49% found it helpful or very helpful. Only 2 students graded it as ‘pointless’.

Since the status quo is to issue feedback, and students are expected to understand that they should independently use that feedback for learning, two questions focussed on how students use their feedback. Students were asked how often they read feedback and everyone claimed to read some part of it at least sometimes (see Appendix D.1, Q5). 82.35% said they read some part of it either ’often’ or ’always’. When asked about how much of it they read (see Appendix D.1, Q6) nearly half claimed to read nearly all of

it in detail. Clearly answers to these questions will largely return what students believe staff want to hear. However, this at least indicates, that students understand that they are expected to review their feedback.

The purpose of feedback is to guide student learning, and so students were asked if they felt they learned from feedback (see Appendix D.1, Q7). Only 3 students said they never did, with the majority learning from it sometimes (59%) or often (25%). The remaining 10% claimed to always learn from feedback. When asked about the

frequency with which students applied learning to future work (see Appendix D.1, Q8) results followed a predictable pattern of being similar, but always slightly less positive than the question about learning. Only 2 students said they never applied feedback to future work with the majority learning from it sometimes (35.29%) or often (39.22%). The remaining 21.57% claimed to always apply what is learned in future work. However, these responses to questions 7 and 8 do not make any useful contribution to knowledge as they contain contradictory results. More students claim to always feedforward

learning than those who always learn, although it may be interpreted as, students always feedforward learning when it occurs. However, the 3 students who say they never learn does not contain the subset of 2 students who never feedforward. That means 3 students who claim they never learn, claim that they do feed forward learning.

The final question was an opportunity for students to feedback what they would change about their feedback (see Appendix D.1, Q9). The most common criticisms were that students wanted more detail, and some expressed a desire for more contact time with staff. The second most frequent comment was that nothing should change, or it was already good.

“Nothing, [should change] the feedback is nearly always tailored to your code when necessary and extremely useful at explaining something when I’ve messed something up.”

Many other comments suggested improvements in areas, to which the literature said audio feedback may contribute positively

• Feedback is confusing

• Marking is too harsh

• Marking needs to be faster

• Feedback needs to show students where to improve

Based on these results going ahead with the audio feedback study seemed to be the next logical step.