This thesis is organised into a set of chapters as described below.
• Chapter 1: Introduction
This chapter has described the problem which motivated this research, and the resulting aims and objectives. To ensure shared meaning of the domain, it also
defines and discusses the key concepts related to assessment and feedback. The background and context in which the following studies took place is also discussed. Finally, the key contributions to knowledge have explicitly been identified.
• Chapter 2: Methodology
The decisions for the selection of the methods of the studies comprising this work are explained.
• Chapter 3: Literature Review
Publications regarding the use of video feedback in practice were reviewed, thematically analysed and the results synthesised.
• Chapter 4: Taxonomy Development
A taxonomy of video feedback is developed which is later validated by application in the studies of video feedback in practice and expert panel (see 8).
• Chapter 5: Pilot Study: Audio Feedback on Trial
A pilot study was run to determine if the benefits being reported by providers of video feedback could be achieved using audio feedback, and therefore, determine if it is possible to achieve the same positive results using fewer resources. Finally, a trial iteration of validation of the Taxonomy of Video Feedback is completed in this study applying only the facets relevant to audio feedback.
• Chapter 6: Video Feedback on Trial
This study investigates the development of a system for the production of video feedback according to the considerations identified in the taxonomy. It then puts video feedback into practice for a sample of computing students. Finally, the first full iteration of validation of the Taxonomy of Video Feedback is completed in this study.
• Chapter 7: Video Feedback in Practice
This study examines the perceptions of several cohorts of students in receipt of video feedback for all assessment submissions for an undergraduate unit. Finally, the second iteration of validation of the Taxonomy of Video Feedback is completed in this study.
• Chapter 8: Taxonomy Validation
This chapter describes the validation of the Taxonomy of Video Feedback by utilisation in practice and by expert panel.
• Chapter 9: Final Discussion and Future Work
This final chapter explores the extent to which the contributions to knowledge address the original research questions, critically evaluates this work, and makes recommendations for future work.
• Appendices
Material supporting other chapters.
1.7
Summary
This introductory chapter describes the motivation and objectives for this research. To ensure shared meaning key concepts related to assessment and feedback have been explained as relevant to the domain. It has laid out the background and context in which the following studies took place, and how the key contributions to knowledge were achieved. Finally, there is a description of the structure of this thesis. The next step is to select the methods to be used to achieve the research aims and objectives.
Chapter 2
Methodology
2.1
Introduction
The aim of this chapter is to explain the research plan. It begins by describing the context of the work, followed by the approach to research in terms of the perspective of the researcher, and how that is identified as a paradigm. Next is an exploration of potential methodologies in terms of their features, strengths, weaknesses, and which are the most appropriate fit to achieve the research objectives. Each study is then explained in terms of the potential methods, the rationale behind the method selection process and the selected method.
2.2
Research Context
It is intended that this research takes place in practice to ensure both its relevance and validity for the intended audience. With that context comes constraints and requirements necessary to prevent any detrimental impact to the students learning experience.
2.2.1 Audience
The audience for this work are the academics who have not yet tried to use video feedback but would like to do so, and would like the benefit of experience to guide their decisions before they attempt to go live in practice. Locally this includes colleagues who are taking on this practice for the purposes of the study. They may be open to modifying their instinctive practice based on evidence provided here. Globally it could be any teaching academic and/or assisting technologist, setting up a system to create and distribute video feedback for the first time, and seeking advice to ensure an easy transition. There are also the researchers publishing papers on the use of video
feedback, many of whom may be inclined to omit apparently insignificant figures or points which, if they can be encouraged to include them, may contribute to the global picture.
The most important audience of all according to Zuber-Skerritt (1996) is ourselves. Any researcher must be open to the findings and willing to explore ideas and interpretations that do not fit with their own ideas and intuition.
2.2.2 Requirements
The first requirement is that the proposed solution includes implementing a workable system for the creation and distribution of video feedback. This must be technology based, as is current practice for feedback creation and distribution. Analogue
alternatives have historically proven less efficient, and such a retrograde step would be unwelcome. Having established feasibility, the system needs be tested under load for the production of video feedback for large numbers of students (see Research Question 1 in section 1.2). Feedback must still delivered in a timely manner to a normally large cohort with the resources available. If adjustments are required to cope with the requirements of practice these must then be recorded, and the extent of their impact evaluated.
Tools need to be generated to guide staff in their endeavour to improve the feedback they create (see Research Question 4 in section 1.2) and these tools need to be validated in practice to prevent differences in simulated conditions influencing advice given, and to hold credibility with academic staff. (There may be some merit in conducting small discrete studies out of context, perhaps to enable the isolation of individual variables, but to ensure the guidance offered to staff is practical the study should largely be completed in a real-world situation.)
The study methodology requirements are: -
• Those concerning the impact on research context
– The research must be completed in practice
– The research must not be detrimental to the student experience
– The research must not increase the burden of staff marking student work – The intervention must be compatible with current institutional policies on
– Any intervention must be technology based and compatible with current IT
systems eg the virtual learning environment.
• Therefore, the methodology must have the following qualities:
– It must be appropriate for study of a contemporary phenomenon
– It must accept that data may be gathered from literature and/or any available
relevant source in practice
– The knowledge may be need to be constructed from several data sources of
different types
– It must allow the contribution to emerge from the knowledge in the form of
artefacts such as models and documentation.
– It must allow for validation of artefacts in practice
– It must permit numerous iterations to allow for validation producing
amendments and/or new knowledge, which may be applied and evaluated in subsequent iterations
2.2.3 Constraints
Conducting the research in practice constrains the resources available. The constraints are those imposed by the context.
Current student experience The most important constraint of this research is that nothing must be implemented that causes disruption to the student experience, or that results in feedback that is of less value to the students than is currently being received. Should anything be found to cause such an effect it must be removed from practice at the earliest opportunity.
Staff Workload Timely return of feedback to students is essential to maximising its learning potential. Creating feedback for normally large cohorts is a significant burden and may take several days for several staff, especially when completed in tandem with other normal duties, such as teaching, research and management. This research must not add to the burden of the marking team.
Data Sources The sources available in normal practice conditions are the opinions of the students and staff involved, and whatever information is already recorded by the information systems used.
Literature A number of studies examine the use of video feedback. Although conducted in practice they normally consider only small numbers of students over a short time span and are usually published following a first attempt. Their findings can be compared to the findings in this study, particularly for validation purposes, however, the context must be considered carefully in each case.
Stakeholder opinion It is important that, as well as gaining answers to targeted questions, this research offers students space to express their ideas on how they perceive, and use, video feedback. The newness of the use of video in this way means that an instrument which only offers selection from a fixed set of responses may prevent emergence of new and useful ideas, which in turn may improve the usefulness of the feedback to students. Students may have determined how to use the video feedback in new and exciting ways that staff have not even considered yet. They may even perceive their ideas as natural, instinctive and insignificant and yet, knowledge of those ideas may facilitate the improvement of the quality of the feedback. Therefore, it is important that the instrument used for data gathering stakeholder opinions allows for a free expression and emergence of ideas.
Video production and distribution system logs It is possible that the system used for video production, distribution, or both, may have system logs which record the level of user engagement. (It is already known that the current VLE does not record student activity in the feedback area.) For example, the number of times a student views a video. These systems may provide useful data available for quantitative analysis.
2.2.4 Role of the researcher
To complete the research in practice, the researcher is best positioned as part of the marking team creating feedback for students. This is the best position to gain understanding of both the system used, and the feelings and views of the feedback creators. The rapport with students built as a member of the teaching team may
increase the chances of revelations about its use, or may just as easily hinder criticism. Therefore, the position of the researcher must be reflected on with consideration of its impact on objectivity.
2.2.5 Ethical considerations
Clearly ethical considerations also place constraints on the work done here but they also maintain its credibility and relevance to practice. The intent of this research is to improve
the student experience when in receipt of feedback on assessed work. It must in no way be detrimental to that experience, nor have any influence over marks awarded.
All modes of data gathering and all related documentation, must receive ethical approval from Bournemouth University Ethics Committee. All participation must be voluntary. Due to the role of the researcher as a participant in the marking process, the relationship between the researcher and the participants, must be considered throughout. Any potential negative impact, on students or staff, must be mitigated against and openly discussed as part of the findings of this research. Records of data must be carefully stored with restricted access, and deleted on completion of the research, according to institution guidelines.