program were initially engaged to discuss the parameters of the action research model. These leaders included the Associate Dean (Education) of the College, the program convenor and two full-time academics who'd had experience in convening individual subjects. This was arranged as a facilitated discussion by the researcher and ran over a three-hour period. The session was recorded with the consent of the group and notes were taken by the researcher of then subsequently circulated to the group for
The program leaders first considered the anticipated focus of the action research and the potential for the elevated use of qualitative student feedback. It was quickly apparent that there was a shared disengagement from quantitative student feedback used broadly within the university. A consensus emerged that the ANUSET model, though broadly informative as a metric of relative student opinion, had failed to provide useful or consistent insights for program improvement. This is well reflected in this observation by one of the leaders captured in this session:
…there is no doubt the student voice is important, however what this voice is saying and what it is expressing needs to be more explicit if it is to be acted on. Too much emphasis is currently placed on knee-jerk reactions to numbers, sometimes doing more harm than good. (ML-1-13)
Participants drew reference to the elevating significance of student feedback surveys in the university, with a recent policy introduced requiring programs to respond where ratings were beneath the universities average level. In discussion, the comparative value of qualitative student opinion surveys was also debated. Some differences became apparent about what were the appropriate levels of accountability to student opinion that academics should have to demonstrate. This is captured in these contrasting responses below, where the tension between educational judgement and program reputation with students is evident:
…the reality is that students are more than capable of making judgments about teaching, however the real question is how this is understood, and by who, and what it is weighed up against…it is one important input into judging teaching
effectiveness, but one of many. (ML-1-6)
….students pay a considerable amount of money to study programs like ours and we need to know promptly and clearly when teachers are not meeting their expectations. Otherwise, we will lose students as word-of-mouth will undermine this program very quickly. (ML-1-8)
However, there was general agreement that current quantitative student feedback surveys lacked sufficient depth to effectively and consistently guide program development decisions. A number of examples were offered where conclusions had been drawn from student evaluation reports that had proven misleading and had
danger where those without an intimate knowledge of the program, its trajectory or limitations may reach forms of arbitrary judgement. This sentiment is captured well by the commentary of one participant:
….there is a great temptation to simplify complex teaching and assessment situations to a number and make an equally simplistic judgement about the quality of the teacher or whatever….we have subjects (like on legislation) that are challenging that are always rated more negatively because students have preferred other types of subjects. (ML-1-44)
This led to a substantial discussion around how to maintain a reputation for quality in the university, within the industry and with the education provider regulator (MARA) if conventional forms of student feedback-based evaluation were disrupted. This was seen as a serious impediment to the action research approach being proposed. The group then debated the divergent research on quantitative student evaluation that was circulated by the researcher in advance of the discussion (summarised in Chapter Two). It was broadly agreed that there were grounds for the developmental use of student feedback using an action research model (as outlined in Chapter Three). However, this model needed to represent a highly credible alternative to be accepted within the institution and by stakeholders outside the university. Lingering concerns clearly remained as to how be seen to be genuinely accountable in the absence of a defined metric.
Nevertheless, consensus was reached around the need for enhancing student learning to be the primary object of the action research model, rather than just the outcomes student feedback of itself. The tone of this consensus is reflected in this observation:
In the end, student opinion is just reflecting the student view….we need to respect it of course, but we need to focus on the primary issue of creating the context for good curriculum, good teaching and good assessment. Student reactions are the outcome, not the core of our work here. (ML-1-7)
In considering the specific nature of the action research response, program leaders returned to how vulnerable the program was to the effect of inadequate student learning. Although the ANUSET results the program had received were on or above institutional averages, some sharp anxiety about the actual quality of student learning prevailed in the group. Based on anecdotal evidence and several significant student complaints
(ironically outside conventional student feedback mechanisms), the intuitive sense of these leaders was that such elements as the design of online teaching, assessment and feedback had the potential for improvement. They also aspired to broaden the pedagogy of the program to offer more innovative forms of learning, whilst also wanting to assure the educational foundations on which the program currently rested. Yet there was also continuing unease on the viability of an ongoing action research project and the time demand it may place on teachers. There were several reasons for this. Most academics teaching on the program were engaged on part time contracts and given their
predominant roles as migration law practitioners spent limited time at the university. Similarly, despite some attempts at professional development, most were ‘accidental’ teachers, engaged primarily (though not exclusively) for their discipline expertise. These teachers were also dispersed across three capital cities outside Canberra from where the program was offered. One participant observed this paradox as:
…a fragile balance between raising educational expectations whilst keeping these teachers on board….they have the capacity to simply not continue if they feel expectations of them by us exceed what they believe to be reasonable. (ML-1-7)
Equally, there were concerns about the potential resource implications of greater
developmental imperatives being identified as a result of elevated student feedback data. Several participants raised another paradox here: making more visible the limitations of student learning could increase academic dissatisfaction if these issues could not be effectively addressed.
Ultimately, a shared commitment developed to test broadened engagement with qualitative student feedback as a means of potential pedagogical improvement. It was also hoped that the action research may incite further engagement of the largely part time teaching workforce in the collective task of program enhancement. It was agreed that an introductory seminar involving all program teachers would be convened prior to the commencement of the following semester (Semester One, 2010). This seminar would be designed along the lines described in the previous chapter, with participating teachers being provided material in advance on the proposed CHAT-based, action research model and its motivation to enhance student learning and provide the opportunity for situated academic development.