Evaluating quality teaching

The understanding that institutional perceptions of teaching excellence has influence over the direction for change and improvement efforts, combined with the complex nature of teaching makes the evaluation of teaching a challenging task that requires critical scrutiny.

i) Student evaluation of teachers (SET)

Student evaluation of teachers (SET) is a common tool used to evaluate teaching in higher education. The current trend in quality assurance at higher education in many universities views

students as “customers” (Douglas, Douglas, & Barnes, 2006, p. 251) or “direct receivers of the

teaching service” (Douglas & Douglas, 2006, p. 6). Consequently, SET has become an obvious choice for evaluating, maintaining and enhancing teaching quality supposedly, in the best

interest of students’ learning experiences. Although students are argued to be immature (Heilman & Matsuzaki, 2009; Pounder, 2007), biased (Douglas & Douglas, 2006; MacFarlane, 2007; Pounder, 2007), and ill-informed about what is current in teaching excellence (2007), they are the ones who can best describe their learning experiences of teaching.

Issues in SET are many. The focus here is on those related to the complex and multi- dimensional nature of quality teaching. Capturing the complexity of teaching in one single summative evaluation is a challenging task. The question of what constitutes excellent or effective teaching poses particular problems in SET content. More often than not, the criteria for evaluation contained in SET are those perceived by people other than the students, resulting

in evaluation of teaching that does not truly reflect students’ own learning experiences (Bie & Meng, 2009; Heilman & Matsuzaki, 2009; Shevlin, Banyard, Davies, & Griffiths, 2000).

Working around institutional ideologies and the institution’s perceptions and judgment of teaching excellence can play a significant role in determining how a higher education institution designs and manages teaching evaluation (Skelton, 2005; Tennant et al., 2010). A performative understanding of excellence for instance, tends to associate quality teaching with

on-stage performance which can reduce teaching to a technical activity (MacFarlane, 2007) and ignore the multi-dimensional and complex nature of teaching (Skelton, 2005).

There needs to be an opportunity for consideration of a variety of teaching methods, and make room for teaching innovations and creativity as opposed to traditional lecture. A broader concept of teaching and classroom experience has been suggested (Pounder, 2007; Tom, Tong, & Hesse, 2010). The inclusion of open-ended questions in SET is designed to encourage genuine and in-depth student feedback which provides insightful information better than numerical scores can (acknowledging potential student bias).

Beyond the assessment of teaching, it is important that the assessment is followed through with necessary support to improve and enhance teaching. Firstly, teachers should be informed of their assessment outcomes. Direct access to students’ feedback has been suggested as teachers are in the best position to understand the comments within a given context (Rowley, 2003). The use of SET to make personnel decisions that are often found in quality assurance culture should be avoided (Bie & Meng, 2009; Roche & Marsh, 2000).

Secondly, teachers need to be assisted in interpreting the results and developing the appropriate improvement strategies (Marsh, 2007; Roche & Marsh, 2000; Smith, 2008). Negative SET

feedback was found to be potentially damaging to a teacher’s self-concept (Roche & Marsh, 2000). Further, teachers were also found to have the will and desire to improve their teaching but often did not know where to begin (Budge et al., 2007; Postareff & Lindblom-Ylänne, 2008). Marsh (2007) suggests the need for a “systematic intervention” or the use of external consultants, emphasising the need to link teaching evaluation practices with professional development activities. Smith (2008) proposes building a structured model to create this link, arguing for a holistic approach that promotes institution-wide involvement of teachers and institution-wide support at multiple levels.

Finally, keeping a longitudinal database of individual teachers’ SET records can help the

institution monitor one single teacher’s development of teaching over time, rather than

comparing results between different teachers (Marsh, 2007). The latter does not provide an accurate picture of teaching performance due to various factors that can influence SET outcomes of different teachers (Ghedin & Aquario, 2008; Pounder, 2007). A longitudinal record of one single teacher can facilitate teaching development efforts that are more genuine.

Despite the variety of issues with using SET to evaluate and enhance teaching, it is indeed, the

only mechanism to collect students’ feedback in large quantities. We could be pragmatic and be mindful of its limitations (Rowley, 2003). To maximise the use of SET data, relating it to staff

development is essential. Also, supplementing SET with other mechanisms of evaluating teaching can help balance its shortcomings. The following discusses another mechanism for teaching evaluation.

ii) Peer observation of teaching (POT)

Peer observation of teaching (POT) is another common tool used (Gosling, 2002; Hammersley- Fletcher & Orsmond, 2004; Peel, 2005; Shortland, 2004). Among the recognised benefits of POT are the opportunity to have another teacher observe and comment on one’s teaching for

the purpose of development, to observe other teachers teach and gain ideas from people’s

approaches, as well as to review, reflect and discuss with another teacher, what was observed and to plan strategies for improvement.

One key issue in POT is who gets to have a sense of ownership in the teaching observation process? Gosling (2002) described three common models of POT: (i) the evaluation model (superior observing teachers); (ii) the development model (expert observing teachers), and (iii) the peer review model (teacher colleagues observing each other), also known as the

“collaborative model” (Gosling & O'Connor, 2006). The purpose of the evaluation model is mainly for management purposes, usually as part of the institutional quality management systems for formal report of accountability and performance. Despite its good intentions to facilitate teaching developments in the evaluation model issues include exposure of confidential

information that could be used and abused, outcome of reviews that may impact on teacher’s

career progress, as well as the lack of follow-up after quality reviewer processes are completed (McMahon, Barrett, & O'Neill, 2007; Shortland, 2004; C. Taylor, 2009). All these possibilities

could result in teachers’ suspicion of and resistance towards POT practices.

The purpose of both Gosling’s (2002) and Gosling and Connor’s (2006) development and peer review model is to improve teaching competencies. However, the former involves expert

feedback while the latter takes place between “genuine peers” of mutual respect (Gosling, 2002, p. 2). Only the peer review model involves letting the person being observed have full control over the structural design and information flow of the observation exercise (McMahon et al., 2007). This is so that the teachers involved could act as the agents of change when they are able to take ownership and control over the processes (Shortland, 2004; J. Taylor, 2007). McMahon et al. (2007) suggest five key dimensions of observee control: (i) having the liberty to choose the observer, (ii) having the opportunity to choose the focus of observation, (iii) deciding how feedback should be given, (iv) deciding what to do with observation information and (v) deciding how to act on it. It is believed that when teachers are given the opportunity to identify what will be most useful to them in professional practice, the review processes can

become more meaningful and promote active engagement (Gosling & O'Connor, 2006). Further, several observations over a longer duration provide teachers with the opportunity to try various teaching techniques, avoid one-time summative evaluation and builds teaching confidence (McMahon et al., 2007).

Faith in teaching observation rests on its transformative potential. It is important to develop a culture of criticism in which teachers are able to criticise constructively and in return, be

receptive to others’ (Peel, 2005) instead of letting it end up in mere friendly chats of what was observed (C. Taylor, 2009). Cross-discipline observation has also been suggested as it provides

the opportunity to see how teachers of other disciplines approach “the craft of teaching”

(Gosling & O'Connor, 2006, p. 500; Hammersley-Fletcher & Orsmond, 2004).

For follow-up, POT outcomes should be kept well within the cluster of practice, shared in the form of scholarship and learning community, or developed into resource books or multimedia resources (Hammersley-Fletcher & Orsmond, 2004; Nixon, Vickerman, & Maynard, 2010). In short, POT can be a meaningful tool for teaching enhancement if it is structured, well- resourced, collaborative, constructive (not repetitive), well-disseminated and streamlined across the institution (Atkinson & Bolt, 2010; Hammersley-Fletcher & Orsmond, 2004; Peel, 2005).

Both SET and POT are common mechanisms to evaluate teaching. Used together, they can

offset either’s weaknesses and provide complementary feedback on teaching from different perspectives; from students’, peers’, subject coordinators’, professional developers’ or pedagogical experts’.