One of the underlying problems in the assessment and demonstration of teaching and learning standards is the movement between explicit and tacit knowledge. Some activities can be assessed against absolute, quantifiable criteria, but many activities rely on qualitative judgments of quality. Therefore, judging student outcomes against absolute standards within certain disciplines (such as mathematics) is likely to be considered more objective than assessing standards in art or creative writing. This is not to suggest that the judgments in art or creative writing are arbitrary, but to the external stakeholder they may be less objective. It is also accepted that assessment is more valid when conducted by experts who can make informed judgments. This may also be externally validated through peer review, external assessment or moderation. While peer review or external examination has the advantage of relevance and reflects some kind of ‘authority, custom or consensus’ (referring to Sadler’s definition of standards), the process has also been shown to encourage conventionality and discourage innovation.99 Furthermore, when external examiners have a largely technical, standards- referencing role they tend to focus on quantifiable indicators, which in turn reduces the allowance for diversity and constrains the notion of excellence.100 External examination, which is more common in the UK, has also been shown to be inefficient and time consuming to those involved.101
It is clear that the assessment of student achievement and the grades given to students reflecting those judgments, are an important component in understanding the relationship between the expected standards and the achievement of those standards. Having external reference points, such as consensual benchmark statements of minimum and/or ideal expectations, does provide some confidence that standards are in some way calibrated, but they do not provide precise thresholds to measure standards against, or provide a basis for relative achievement between institutions. Subjectivity remains at the heart of many standard setting and assessment practices and complete objectivity is unlikely to be achieved. As McTaggart suggests, ‘objectivity is a seductive illusion, but even its more realistic semantic alternatives such as impartiality, neutrality, consistency and consensus present dangers to academic work and integrity.’102
Subjectivity and the reliable, accurate measurement of teaching and learning sit at the heart of the standards debate. It is evident that governments and institutions are increasingly using a range of qualitative and quantitative criteria and indicators to demonstrate standards. To counter balance
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
97 Wyatt-Smith et al., 2010, p. 59. 98 Woolf & Turner, 1997. 99 Williams, 1986. 100 Langfeldt et al., 2009. 101 Bloxham, 2009. 102 McTaggart, 2009, p.21.
47
potential subjectivity in assessment practice, quantitative measurement of teaching and learning has gained momentum. This is a practice that several academics are skeptical about. As Broadfoot suggests, ‘the idea that if we calibrate our instruments more finely, we can somehow achieve perfect measurement of quality is a modern illusion, created by computers and statisticians who make a living out of it.’103
Nevertheless, quantitative indicators are in precise terms what we value as a proxy for quality and standards.104 The criteria and indicators used to measure student achievement may never be precise and perfect, but they provide opportunities to clarify what is being a measured and how a particular threshold should be determined.
The nature and type of indicator used depends largely on the purpose of what is being measured. In determining a broad representation of academic standards there has been a gradual shift away from input indicators to throughput indicators (processes supporting teaching and learning), to output indicators.105 Historically, high academic standards have been represented by high achievement of input indicators. In other words, high entry standards and the selectivity of students into a program (or highly qualified staff) generally imply high academic standards per se. However, input indicators have become closely associated with an elitist notion of standards and this goes against the widely held agenda for increased participation in higher education. These input indicators are more associated to institutional reputation than the actual standards of its graduates.
The increased drive towards quantitative indicators does create additional problems. It is important that the data suits the intention, rather than letting existing data dictate the approach.106 Indicators need to drive behaviour that is intentional, appropriate and if possible, agreed by everyone involved. As Ramsden suggests, we are falling into a ‘trap, made seductive by the magic of valuations to which numbers are attached. The procedures we use to measure standards may become more important than the things we are measuring, so that we lose sight of the purpose of measuring’.107 Without a full understanding of indicators and how they are used, the standards set by governments or institutions and the achievement against those standards may be based on what data are available, rather than what is relevant, accurate or important.
It is seductive to think that having agreed performance indicators would provide a suitable standards framework and a means for objective judgements. It also undermines the informed judgement of academic staff. However, given the shift in discourse away from quality to standards, and the need for greater external, explicit evidence of achievement, it is inevitable that more quantitative indicators will be used to measure teaching and learning standards. Standards-based assessments of education necessitate the use of quantitative indicators. However, it is likely that a standards-based system will require a combination of indicators supported by informed judgements from expert assessment panels. The key question is how deep and precise is Australia willing to go? Are we, for example, likely to have discipline-specific assessments or external examinations, which would be costly and probably
unmanageable? Or are we likely to have agreed generic outcomes for graduates? Either way, peer review is still necessary to make sense of standards. Peer review allows us to understand the contextual factors that affect the measure of achievement. Thus, a combination of quantitative indicators and qualitative judgements, framed within a particular context, is likely to be a way forward that appeases everyone.
Conclusions
It is apparent that with any discussion about academic standards, different types of standards are being conflated, represented and interpreted by different stakeholders. While the Higher Education
Standards Framework categorises five separate standards, the setting, monitoring and assessment of each of them is confusing to most. Certainly TEQSA as a national regulator has a clear role in
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
103 Broadfoot, 1998, p.175. 104 James, 2008. 105 Brennan, 1997. 106 Coates, 2007b. 107 Ramsden, 1986, p.107.48
monitoring and assessing certain types of standards, such as provider standards, but their role in assessing teaching and learning standards is less clear.
Teaching and Learning standards consist of a range of different activities that have dynamic effects on each other. Institutions set their own teaching and learning standards, which are largely tied to the strategies, policies, processes, curriculum, staffing and students that are unique to each institution. This is not to suggest they are set or assessed in isolation but the external reference points are not always explicit. Setting prescribed national standards in teaching and learning is challenging, but will be required if institutions need to explicitly demonstrate their standards against a national threshold. However, any teaching and learning standards framework needs to balance national expectations against institutional expectations that are aligned to their unique missions. It needs to balance national standards with contextual parameters. This entails a balance of both quantitative measures of quality indicators alongside qualitative judgements of teaching and learning. The former requires extensive consultation and agreement from a wide range of stakeholders. To develop robust, quantitative indictors of teaching and standards will be challenging given the diversity of institutions and the dynamic processes that are inherent in a teaching and learning environment. The latter is best done through external peer review or moderation-type processes but the financial viability of such a model may also prove to be main challenge, especially if it is at the discipline level. A number of more generic, principles-based, graduate standards are likely to be more palatable, manageable, cost effective and less intrusive. It is, however, a challenge that must be overcome. Mass higher education in
Australia requires more robust processes and methods to monitor and demonstrate academic standards.
49