I-Probenet
Comenius 3 network on self-evaluation at school
CREATE A MIRROR
FOR YOUR SCHOOL
2
CONTENT
PREFACE 2
I-Probenet 2
1 AN INTRODUCTION TO SELF-EVALUATION AT SCHOOL 4
THE BACKGROUND OF SELF-EVALUATION 4
Evaluation: a history of approaches 4
Self-evaluation and School Autonomy 4
Self-evaluation and Learning 5
Autonomy, autonomous learning and self-evaluation 6
Evaluation and negotiation 6
Implications 7
A FRAMEWORK FOR SELF-EVALUATION 8
Self-evaluation for learning 8
Explicit self-evaluation 9
Levels of self-evaluation 9
THE CONTENT OF SELF-EVALUATION 12
2 SELF-EVALUATION: STATE OF THE ART IN 14 EUROPEAN COUNTRIES/REGIONS 15
1. Internal and external evaluation 15
2. The ESSE countries in search for balancing internal and external evaluation. 16
3 WHAT MAKES SELF-EVALUATION SUCCESSFUL? 28
1. Evaluation is not only a technique but also a process 28 2. Teachers´ participation and ownership in self-evaluation is crucial 29 3. Teachers are reassured by obligation, ability and willingness 30 4. Negotiating contracts for the self-evaluation process is necessary 30 5. There must be protection for reflection in a constructive culture of communication 31 6. Confidence in the management of the process especially regarding energy 32 7. Self-evaluation must have visible effects 33
4 IMPROVING TEACHING WORK BY ACTION RESEARCH 34
5 SNAPSHOTS FROM SELF – EVALUATION 38
Children at the centre of self-evaluation 38
Teachers at the centre of self-evaluation 49
Self-evaluation involving heads, leaders and organisations 56
Self-evaluation of projects 72
6 LOOKING INTO YOUR OWN MIRROR 78
7 NETWORK PARTNERS 79
3
PREFACE
I-Probenet
Self-evaluation is an essential step to quality improvement in education. Introducing a culture of self-evaluation at school is not easy. It should be carried out in an open, participative, democratic atmosphere. For most countries and schools this culture of self evaluation is new. Introducing it requires the right knowledge, the right material and the right strategies. On the other hand a lot of experience is available in some countries, in some schools and on some aspects and angles. Bringing all this together to share, experience, generate useful material and further European cooperation in this field has been the main aim of I-Probenet .
I - ProbeNet is a Comenius 3 thematic network. It creates an opportunity for the exchange of good ideas, best practice and new contacts through its website, national workshops and international conferences. The target groups are: school teachers, project coordinators, school leaders, teacher trainers, policy makers, inspectors, academics and others interested in the process of self- evaluation at school.
Self-evaluation has become a key element in school innovation. Schools have always been responsible for their own quality, but since in most countries schools are moving towards greater autonomy, the process of caring for that quality should be embedded in school’s daily life. The image of the school as a learning organisation with teachers and management learning from their practice and students engaged in autonomous learning and life-long learning also places self-evaluation at the centre.
In order to distinguish it from external evaluation or inspection, self-evaluation is seen to be school-initiated, internally organised and self-regulated. It
should aim at the professionalisation of decision-making, and improving the realisation of the school’s own objectives and the quality of education at school. Evaluation processes, where legitimacy and accountability based on externally imposed indicators are at stake, are referred to as external, even when they are partly internally organised.
The network organized three international conferences on self-evaluation at school. During the first conference: “Create a mirror for your school” in Sibiu, Romania, September 2002, we paid attention to instruments, steps and techniques. The second conference “Self-evaluation is a learning process” in Brdo, Slovenia,
September 2003, focused on the learning processes in self-evaluation and on the school as a learning organization.
“Self-evaluation is a dialogue” was the title of the third conference in Graz, Austria, September 2004. In Graz we emphasized the social processes involved in self-evaluation. These three titles also cover the vision of the network partners to see self-evaluation as an optimal learning opportunity for all actors involved and for the institution as a whole.
This book is a compilation of selected articles, written by network members, and of presentations done during the conferences. It starts with some theoretical input, but moves quickly into practical guidelines and practical
4
examples. With it we want to offer low threshold material for teachers and school leaders to help them walk the path of self-evaluation. It is meant as a start, a teaser, to motivate further reading and practice. More theory, more guidelines and more examples can be found in the ‘library’ and ‘your story’ sections of the network website: www. I-Probenet.net. The book is also translated into English, Dutch, French, Spanish, German, Romanian, Slovenian and Lithuanian and can be downloaded for free in .pdf from the network website.
I would like to thank all partners and members for their work and for their contributions to this book, especially the members of the editorial board: Mary Ann Halton, Marjolijn Smith-Voerman and Chris Williams. This is also the right place to thank the European Commission for their financial support within the frame of Comenius 3 and for their guidance during the life span of the network.
As project manager of the Landcommanderij Alden Biesen it was a pleasure for me to coordinate this network holding 17 partners from 10 countries. It was a challenge to bring so many different institutions, people and visions together. Moreover it was also thrilling to see the many results of the work done and specially to be part of the ‘learning process’ the group went through over the years. Alden Biesen, a cultural centre of the Flemish Community, tries to be a meeting place for Europeans and to further international cooperation in education and culture. The I-Probenet programme certainly has contributed to that goal.
Please enjoy reading and browsing through this book. Let it be a starting-point for learning.
Guy Tilkin Lies Kerkhofs
5
1 AN INTRODUCTION TO SELF-EVALUATION AT
SCHOOL
Jaap van Lakerveld, Leiden University, The Netherlands Christa Bauer, Schulverbund Graz West, Austria
Chris Williams, Lincoln Christ’s Hospital School, UK
THE BACKGROUND OF SELF-EVALUATION
Evaluation: a history of approaches
Over the last century a series of evaluation approaches has been developed, tried out and more or less
implemented. Evaluation started as merely measurement of individual learning outcomes. Later the definition was widened and emphasis was put on goal- oriented evaluation. In the seventies of the last century a number of new models and approaches was introduced, among them decision-making models, consumer models, art criticism-like approaches, legal models, democratic models, etc.
After the seventies a long period followed in which the approaches became more and more eclectic. No longer did evaluators think of only one model or one approach as the best. The challenge became to find the best match between what the school, or the people, asking for evaluation needed and the approach chosen.
In the late eighties and early nineties, when it became more and more popular to consider schools as learning organisations, evaluators became more and more aware of the necessity to organise evaluation in such a way that it provides an optimal learning opportunity for those for whom, and probably with whom, the evaluation is carried out. In this article this last position is chosen. Evaluation, no matter whether it focuses on pupil learning, on the quality of teaching, on organisational or management matters, or school leadership, or what ever else will always have to provide those who wish to evaluate with the best possible opportunities to learn from their experiences.
Evaluation of education is the process of systematic collection, analysis and exchange of data concerning educational processes of either individuals, groups or organisations (institutions, etc.) in order to facilitate learning among all parties concerned so value judgement and decision-making may be based on evidence.
Evaluation covers all areas of provision of education, teaching methods, finance, management, general direction and leadership and the pursuit of long-term objectives. It gives the parties involved a better understanding of what they are doing. It may enhance the innovative potential by highlighting successful initiatives and by explaining the context of these successes. At a deeper level, its findings may lead to new priorities and/or higher compatibility between choices and resources. Basically it all comes down to learning from one’s professional experience. Since these processes of evaluation are school initiated, school based and self regulated, they are referred to as self-evaluation to distinguish them from processes of inspection or external evaluation
Even though some forms of self-evaluation have long been every-day practices at schools, the overall culture of self-evaluation needs improvement even in the most developed countries. However, the urgency of elaborating appropriate educational evaluation strategies may be more vital in the new partner states. Educational reforms of more or less the same nature take place both in EU countries and in other partner states. For some countries reform actually means a series of dramatic alterations to be managed in a very short time scale.
Self-evaluation and School Autonomy
All over Europe school systems are developing in directions that allow schools more autonomy. This means that the school will be given more freedom to make policy decisions of its own. Schools increasingly are held responsible for the quality of the education provided. The consequence of growing autonomy within education is a change of the role that schools, authorities and other stakeholders play in quality development, and in quality control. Schools become more and more the initiators of their development. In many countries schools nowadays have more freedom to determine their programmes and plans to their own preference, or rather to the preference of the immediate stakeholders such as parents and students. At the same time, they have to make their views, their choices, and their decisions more transparent. They have to be able to prove that quality standards are being met, that drop-out rates are kept low, that results are good, that parents appreciate the school, that measures for improvement are taken if necessary... to name just a few quality indicators.
6
The trend towards more school autonomy has had its implications in various areas within education. In school development it led to more school-based initiatives and policies. In-service training shifted from
university-based (or institution-based) to school-based. Schools identify their policies, in-service needs are derived from these policies and in-service training, or rather in-service learning, is planned accordingly. Learning has become the central issue at all levels. Traditionally schools were considered places where pupils learn and teachers teach. Nowadays this idea is gradually extending to the view that schools should also be learning environments for teachers and managers or technical or administrative staff. ‘Life long learning’, ‘learning teams’ and ‘action learning’ are concepts that are gaining support among educationalists. The development of new knowledge and new technologies requires that workers in schools are capable learners, who are able to share their capabilities with young people. At the level of the organisation of schools this trend has brought new concepts such as the learning organisation, self-directing teams, and more recently the concept of knowledge productivity. Knowledge productivity is referring to the potential of an organisation (a school) to identify, absorb, generate, spread and apply (new) knowledge. Here ‘knowledge’ is used in the broadest sense of the word. Knowledge includes subject expertise, problem solving skills, communicative skills, reflective and meta-cognitive skills, the ability to regulate motivation, emotions and beliefs. All these competencies are supposed to develop optimally, if a balance is to be found between the degree of stability and security in an organisation and a certain level of creative turmoil. Too much stability may make an organisation lose its energy; too much turmoil may make the organisation explode, collapse, turn to apathy etc.
Relatively autonomous schools develop their own plans within the framework of national standards. Such schools choose to some extent their own materials and methods; they identify and implement their own staff development policies; they choose their own school organisation structure; they seek their own partners and establish their own partnerships with other schools or institutions of other kinds, such as research institutions, advisory bodies, partners from other areas of society (social work, libraries, IT agencies etc.)
Schools having a higher degree of professional autonomy now seek their own partners to develop their views and their policies. During the last few years a great number of school partnerships have been established to share experiences, to develop ideas or to implement common policies. Stimulated by the European Union, many school partnerships have been, or will be established, to promote the European dimension in schools in European countries (EU member states and non-member states). In some countries national or regional school partnerships are promoted to support processes of change and improvement at all levels. (Trans) national school partnerships may be considered to be part of the powerful learning environments schools nowadays need to create for themselves in order to continue their learning and to extend their own ability to learn.
Self-evaluation and Learning
Learning has become the key issue at all levels in schools. The learning of pupils always was a key issue; today the learning of all others involved is the challenge of modern educational systems. Feedback is the most powerful component in processes of learning. Individuals need to know what the result of their actions is and what impact it has, in order to know what to change, what to emphasise, what to add or to avoid in future. In organisations and particularly in schools, evaluation and assessment fulfil that feedback role at all levels and in different ways. The classroom practice, the school curriculum, the learning materials, the staff development, the school development, the management of the school the co-operation between schools... evaluations may take place within each of these areas and form the input for processes of learning and professional growth within the school. More specifically evaluation of these areas includes:
• Evaluation of pupils (tests, assessment, selection)
• Evaluation of staff (selection, appraisal, in-service feedback)
• Evaluation of programmes/curricula (evaluation of learning books, courses, organisation of the school year, subject areas, school curricula, national curricula)
• Evaluation of schools (quality control, consumer's information/public relations, inspection)
• Evaluation of school development (school audit, evaluation of school's policies or change processes, evaluation of implementation, evaluation of short term and long term progress etc.)
• Evaluation of (transnational) school partnerships (evaluation of the quality of the co-operation, evaluation of shared projects, evaluation of impact in the school, evaluation of effects on pupils, staff and parents).
7
Autonomy, autonomous learning and self-evaluation
As schools develop towards a greater autonomy, students are trained to engage in self-directed learning. They are stimulated to develop their learning competence in order to prepare themselves for a future in which learning will be lifelong and in which learning abilities will be the most selective features in any area of professional life. In this context of autonomisation of both the schools and students, teachers need to find answers as well. They will have to choose a position that enables them to contribute to the new requirements, but at the same time they must find a way to take a professional stand, to defend what was worthwhile and bring about what is even better. This means the teachers themselves will have to seek a kind of autonomy. Not the kind of autonomy that leads to isolation, but the awareness that the teaching force has its own position, its own responsibilities, its own ideals and its own professionalism within schools, within projects, within one’s discipline within the educational system and within society.
The new educational scenery now is beginning become clear. At all levels autonomy increases, but the
requirements change with it. This goes for leaders, students and teachers and all others active in the education.
Leaders and managers
Schools and school principals and managers play their part. They lead, they manage, and they account for what they are doing and if they learn from their experience, they will further improve their performance. Leaders are responsible for creating and maintaining a learning climate in which professional growth, knowledge productivity, quality and an innovative potential come to flourish.
Students
The students learn how to learn. They have to develop the competence to acquire new knowledge to apply new knowledge and to reflect on it and on its utility or value. In order to be able to do so they will need room for research, for discovery, for trial and error, for reflection. Eventually they will have to report and prove their achievements. The new media, the rich learning environment modern times offer to students have created a “zapping generation” of students that need to be able to seek their own way in the abundance of information available. These new conditions require highly skilled learners. Traditional teaching no longer provides the answer to these newly developed needs.
Teachers
Teachers are the people who must play the most important part in creating the powerful learning environment students need. Teachers form the group that creates the learning conditions and that provides the professional input into the school curriculum. Teachers have to meet standards, e.g. national exams, co-ordinated tests. Their role is changing over the recent decades from primarily transmitting knowledge towards a more facilitating role. Mentoring, coaching and tutoring, gained importance along teaching. The changing demands and new challenges for teachers put them in the positions of learners too. Only those willing to learn, to adapt or to change will be able to keep in control and fulfil the task teachers are facing in a way that meets the needs of the modern younger generation.
Evaluation and negotiation
So far we have tried to explain that evaluation serves a feedback function in the learning processes of all parties concerned. We took the position that evaluation is mainly aiming at creating opportunities for systematic learning from one’s experiences. Furthermore it was made clear that the present emphasis on school autonomy and the importance of self-regulation, or autonomous study skills among students, forces schools and teachers into a direction in which learning becomes a core activity and a core challenge for all the people involved. Another element in our philosophy is the idea that evaluation as part of a professional learning environment requires interaction among all the stakeholders or actors playing a part in the school, in the teaching/mentoring and in the learning processes.
This interaction serves two purposes. First it makes people put their views into words. By doing so it adds to their understanding of the situation. As such it is a way of processing the information people gather about their own situation. It makes things explicit and it reveals the relations between the observed facts. It supports the learning processes of the people involved in evaluation.
8
parallel each other, or be in conflict. This fact may be seen as an obstacle for smooth evaluations. However, from a learning perspective it also may be perceived as an advantage. Those involved in evaluation will be motivated to become active participants in the evaluation, if evaluation really matters to them. Like in politics people need to be able to identify with certain positions and interests in order to feel motivated to step forward, to take a stand, to defend their interests, to seek coalitions, to vote, elect and be elected. Those who get involved in evaluations in such a way will benefit most from it. The discussions, the concerns and their motivation will support their learning to the maximum. To them evaluation matters!
Implications
Evaluation is learning
Basically evaluation is about but learning. Processing information about one’s own performance (individually or as an organisational unit or even an organisation as a whole) in order to improve or change it, is basically a learning activity.
Learning as social constructivism
With changing concepts of learning the concept of evaluation has changed. Evaluation and learning developed along the same lines from a test psychological past through the era of behaviourism into the cognitive era, finally leading to the present social constructivist approach. Today learning is perceived as a process in which the learner is actively creating his or her own knowledge. Knowledge at present is seen as a whole range of learning outcomes varying from cognitive knowledge including meta-cognitive and reflective skills, to behavioural skills, social skills and affective outcomes such as attitudes, norms and values.
Evaluation is predominantly adult learning
In most situations in which evaluations are set up and carried out, the audiences and the target groups consist of adult workers, staff or managers. This makes it relevant to consider the conditions known to be essential in the learning environment of adult learners.
• Adults are known to learn best, or to gain most from learning experiences in which a connection is made with real events and real experiences derived from their working situation.
• Furthermore adults like to have opportunities to have input in the discussions, rather than being lectured or taught.
• A safe and secure environment is an important condition, since it reduces the threat of being exposed to, and damaged by a criticism.
• Adults tend to judge the value of learning by the applicability of the results. They feel a need for practical and applicable knowledge. Of course this varies with the dominant learning styles a person is characterised by.
• Adults like to be acknowledged and recognised.
Self-evaluation is organisational learning
Evaluation usually deals with projects or programmes in which people work together. After the evaluation, they will have to continue to work together and to improve their performance together. Collective or organisational learning is vital for raising collective performance. Therefore evaluation should be organised in such a way that the insights gained from it will find their way in the school organisation. The individual learning thus may affect others through direct transfer of the acquired knowledge and experience, or it may be transformed into changes or improvements of structures, methods, technologies, rules and regulations within the organisation.
Self-evaluation is best planned and organised as a learning process
For an evaluation to offer the optimal opportunities for learning of everyone involved in it, the evaluation should:
• be motivating both for the issues being reflected upon and for the activities included;
• provide a rich internal and external learning environment;
• give opportunities for experimentation;
• promote dialogue and feedback;
9
• feed back data about learning outcomes, effects and impact.
Self-evaluation will have to clarify the variety of interests and perspectives
Self evaluation should clarify the various interests of different parties concerned and provide the participants with the opportunity to clarify and negotiate their positions. This helps them to understand the dynamics of the school and its development and their own positions within it. Furthermore here one may find a great source of motivation for evaluation.
In the next chapter of this booklet these implications will be further elaborated and translated into a frame work for a future oriented, self-initiated, highly active, interactive, efficient, kind of evaluation. This framework will not be prescriptive in a narrow sense because it will give a sense of direction in the numerous considerations that may be involved in planning an evaluation.
A FRAMEWORK FOR SELF-EVALUATION
Self-evaluation for learning
Self- evaluation may serve various purposes. It may be done for purposes of accounting. Such evaluations are executed in order to convince either the community or the inspectorate or other bodies that the school operates in a responsible way. Self-evaluation may also be assisting decision making. In order to enable people to make decisions such evaluations provide the decision makers with the necessary data to underpin their decisions. Usually such evaluations focus on pre-set options such as a go- no go decision, or a choice between two alternatives (for instance: will we proceed with our mixed ability groups or do we prefer to work with homogeneous groups?)
In our context of self-evaluation we focus on the evaluations that aim at learning.
Evaluation is often seen a retrospective activity. This however does not imply that evaluation only occurs after a process is completed. Evaluation may well take place in any of the stages one can distinguish in a project, a programme or any kind of initiative.
Phase of the project Evaluation questions Time(scale)
Audit What is the state of affairs? What can we do to improve it? Before the project
Planning Are we making a good plan? In the beginning of the project Introduction Do those who will be involved understand/ support the plan? During the project/programme Pilot Does the plan appear to be feasible?
Can we do it? During the project/programme Implementation Is the project running smoothly? During the project/programme
Output evaluation
Does the project/ programme work?
Has everybody learned what they were supposed to learn?
After the project/programme
Effect evaluation Do those that were part of the project
10
Impact/ Transfer
Has the project/programme affected
the work of others/ the curriculum, careers, the organisation?
Long after the project when the programme has become common practice
Explicit self-evaluation
In discussing evaluation one may come to think that evaluation activities already take place. We consider it of vital importance that evaluation is an integral part of the work, a programme or a project. However we also would like to emphasise that evaluation takes something extra. Gathering data and processing them is one, but analysing them and reflecting on them is another. This takes time and room for analysis of the data, sharing the perceptions and opinions, time to search for meaningful solutions, new ways or follow up, and last but certainly not least, a reflection on the value of the processes studied and lessons learnt, both individually and collectively.
This time and room for evaluation and reflection, takes deliberately chosen moments in time, which was included in the school planning or in the planning of projects.
Levels of self-evaluation
If a school decides to do a self evaluation or of members of a project decide to do so they need to be aware that they may evaluate in a number of areas. For example they may focus on classroom activities or on school management issues. To give an overview of options we tried to distinguish between various levels of evaluation. It is helpful to be aware of the levels you wish to emphasise. Often one will choose a number of levels together. The list of levels may be considered as shifting from a classroom focus towards a more organisational/societal focus:
• Student learning
• Teaching
• School curriculum
• Curriculum development
• Professional development
• School organisation and management
• Leadership
11
• Broader societal context
Issues to be evaluated
Apart from levels of evaluation one may seek to find another way of focusing as well. In international projects, in other school projects or in analysing the school as such, one cannot evaluate everything. It will be necessary to focus, either from the start or to focus progressively while evaluating (“zooming”). Just to give a few possible emphases to put in an evaluation, we mention some focuses chosen in former projects we know of.
• The European dimension
• Equal opportunities
• Self regulated learning skills
• Moral development
• Multicultural issues
• Levels of performance in subjects
• School attendance
Goals of evaluation
Once the area of evaluation (level and focus) has been identified, one has to be aware of the goals of the evaluation
• improving a current project
• learning in preparation for future initiatives
• personal learning and professional growth,
• sharing findings and making them more transparent for democratic purposes
Target group(s)
Evaluation may serve different audiences/target groups. How evaluation is done and how it is reported, depends on the target group you have in mind. So it is good to know in advance for whom the evaluation is meant. Identify your choice considering the following options (again not exhaustive):
• Yourself
• Colleagues
• The whole staff
• Parents
• Students
• Local authority
• Inspectorate
• Related agencies
• The wider community
Information needed
Often evaluation focuses on judgements only: In our view it is very important to make a distinction between 1. describing how things went
2. identifying the context in which it took place 3. identifying the output
4. and learning what everybody’s opinion is on why things went this way and on how they value the project, the programme, or the school.
In order to use these categories it is important to distinguish between data about:
• Intentions, expectations and plans;
12
• Norms and values of those involved;
• Judgements, opinions and evaluations.
Methods and instruments of evaluation
Various methods and instruments may be used for gathering the information needed. We will just mention a number of them and give a brief explanation:
• Questionnaires. List of questions to be administered to the respondents. The list may consist of open or closed questions. They may be more or less structured.
• Observations. Evaluation data may be gathered by observing the behavior of the people (teachers, students, others). It will be necessary to identify carefully which is to be observed and to decide what observations stand for what meaning.
• Interviews, oral questionnaire, either structured semi-structured or open. Interviews may be held individually or with groups
• Meetings, Meetings stimulate people to reflect on things and to discuss them. While doing so evaluation implicitly occurs. It is vital however that this implicit process is made explicit at one point, for instance by summarising what’s been said or by analysing and feeding back the output of such meetings.
• Snowballing/Delphi, People may be asked individually what they think of particular things, but it is certainly illuminating for them to hear or read what the same things meant for others. By making a list of the issues raised, or of the opinions given, the opportunity is created for others to comment. In this way an overview is created of possible opinions and the support for each of them.
• Written products. Writing about ones experiences is a powerful way of reflecting upon them. So in either minutes, reports, articles etc. an evaluator will find a lot of condensed data about the reflections of those who wrote them. Alternatively by asking people to write things the evaluation i.e. learning-process has triggered.
• Presentations, Telling others what happens in your school or in your project forces you to reflect as well. So do the critical questions that may be raised by your audience. Again this method can prove to be a powerful stimulator of learning from experiences.
• Critique/reviews by internal or external experts/colleagues. Inviting others to come and give their views on what they see happening in projects, school, or programmes is a nice way of triggering the internal debate about the quality of it all. The external critic may be praised or lynched afterwards. The result however will be reflective and activating.
• Unobtrusive measures, Some data may be gathered without bothering anybody. How often do students visit the library, how many parents come to meetings or answer to mailings. By looking at the speed in which the chairs of the study room deteriorate one may derive something about the intensity with which this room is used (not for what purpose alas).
• Reports/minutes. Some documents are produced even if no evaluation takes place. Nevertheless they may have their function in an evaluation and serve the reflective process evaluation is supposed to trigger. Furthermore the data in these documents may be considered as data for further analysis.
Organisation of the evaluation process
In evaluations the people concerned or the people involved may all have different roles. That is why evaluators should wonder how to organise the evaluation and to consider the question how each person should be involved. Several roles may be considered:
• Member of the evaluation committee
• Respondent
• Member of the "audience" or target group to be reported to
• Evaluator
• Critic
• Stakeholder/representative
• Data processor
13
Information processingToo often the processing of the data appears to be the work only medieval monks were willing to do. Therefore it is wise to prepare for this processing in order to keep things as simple as possible. Choose categories for which to categorise the responses or data. Take care the process is transparent and easy. Slow process lead to slow feedback. Slow feedback in turn is usually frustrating and perceived as useless it is only available after the event lost its momentum.
Identification of Indicators and Criteria
Identify indicators and criteria in advance so the discussion about the meaning of the outcomes of an evaluation can be smooth and to the point instead of confusing and susceptible to multiple interpretations. This problem cannot be solved completely but good preparation certainly helps.
Analysis/interpretation
Take care to make a proper application of the indicators and criteria. Furthermore, by being careful and strategic in choosing the ones with which you do the analysis, half of the support for the conclusions may already be gained.
Reporting
Different audiences want different reports. Be aware of whom you are reporting to and what that implies for the format, the level of conclusions and the way priorities will be set.
Time schedule
Without a time schedule any project will fail and so will any evaluation. Try to take care that the time invested in evaluation never will be more than 10 % of the time invested in the development of what is evaluated. If there seems to be a need to evaluate even more intensively we believe enough is already known about what could be done to improve things. In such case it is better to emphasise further development first and start evaluating later. After having decided on all the issues sketched in this chapter. It will be possible to describe your evaluation plan. Many readers will probably think this requires an amount of preparation that is unrealistic and that the author must have lost touch with reality. Evaluation however has so often been an investment with no return, that one has to realise that it is either a well prepared and thought through enterprise, or it won’t serve its purpose. This chapter is no plea for evaluations in great depth in any situation. On the contrary it is a plea for evaluation only when it is needed; it is a plea for well focused evaluation, it is a plea for efficient evaluation, it is a plea for effective evaluation. A carefully designed evaluation plan may simply consist of the elements mentioned above as shown by the titles of the paragraphs (in italics).
THE CONTENT OF SELF-EVALUATION
Both the philosophy and the framework of self-evaluation gave an idea on how evaluation may be approached, they however did not give any idea yet on what to evaluate, nor on the criteria to be used.
To these vital matters this part of this chapter will be devoted. First of all we will focus on areas of quality one may distinguish. The attention will be paid to indicators of quality one may keep in mind in defining, describing and evaluating the schools quality. Of course the areas and the criteria mentioned are just mentioned as suggestions. Essential for self-evaluation is that those involved choose their own indicators of quality and define their own priorities and criteria.
In order to describe and evaluate ones own school it is important to be aware of the aspect of the school that may be included in such self evaluation. In a Comenius project on benchmarking named 2EQBS
(http://www.utv.mh.se/projekt/2eqbs/tool/) a tool was developed for schools self-evaluation including four broad areas of study:
Contextual factors
• School identity, what is the mission of the school?
• Autonomy, what freedom does the school have to make decisions on various matters?
14
Teaching and learning
• Curriculum, what subjects, what content, what activities and what programme does the school have?
• Pedagogy, what is the approach chosen in educational matters beyond instruction, more specifically in the area of values and moral, and social development?
• ICT, how does the school involve new technology in its teaching and learning processes?
• Tasks and assignments for pupils/students, what kinds of assignments or tasks are typical for the kind of schooling the school stands for?
• Learning materials, what kinds of learning materials are in use that are typical for the kind of teaching and learning in school?
• Teaching strategies, what models of teaching, teaching approaches, instructional ideas, the school is known for?
• Assessment, how are children assessed in order to identify their progress, or attainments?
School development
• Roles, positions and responsibilities, what kinds of work and what kinds of responsibilities are distinguished in the school?
• Schedules, how is the schools programme translated into a schedule? How many weeks a year, how many hours a day, interrupted by breaks or continuous, a schedule per year or per semester or other period?
• Staff structure, what is the organigram of the organisation of the school, who has what authority or mandate?
• Working culture/ethos, how do people work together and in what kind of professional as well as informal atmosphere?
• Information and communication, through what channels and media do people get information about work and school issues? How is information distributed and shared?
• Values and norms, what is, or is not done in school. What values and norms does the school hold as far as the internal work processes are concerned?
Leadership and change
• Vision and Mission, what philosophy serves as a compass for the school?
• Strategic planning, what specific strategic plans do exist within the school in order to prepare the school for the future?
• HRM policies, does a policy exist on how the staff will be trained and developed in order to face the challenges of the immediate and longer term future
• Curriculum policies, are there plans in action to reform, update or develop curriculum elements to be included in the schools programme?
• Innovation and change policies, what specific plans do exist to improve, innovate or change the school as an organisation?
• Committees, task forces, project groups etc. What temporary or semi- permanent groups are appointed that have a position outside the regular formal structure of the organisation?
In trying to describe the school for self-evaluative purposes these aspects of the school reality may serve as an overview of possible areas to cover.
More formal are the sixteen quality indicators the European commission came up with in the European Report on the Quality of school Education in may 2000. These sixteen indicators serve as an intermediate concept between areas to keep in mind and criteria to be applied to those areas.
15
Indicators on attainment 1. mathematics 2. reading 3. science 4. ICT 5. foreign languages 6. learning to learn 7. civicsIndicators on success and transition
8. drop out rates
9. completion of upper secondary education 10. participation in tertiary education
Indicators on monitoring of education
11. Evaluation and steering of school education 12. parent participation
Indicators on resources and structures
13. education and training of teachers 14. participation in pre-primary education 15. number of students per computer 16. Educational expenditure per student
These indicators may be helpful in further specifying the content of the self-evaluation and in focussing the final debate: the one on criteria. This is where the school will have to determine itself, of course in close contact with its parents, the unions, the experts, the inspectorate, the authorities. What targets, what goals, what results, degrees of satisfaction or approval will be the criteria in each of the areas included in the self-evaluation.
This discussion will have to lead to a scheme that shows the areas covered, the indicators used and the criteria to be met all in one.
Areas of evaluation Quality indicators Criteria for evaluation
...
...
...
...
...
...
...
...
...
Once such an overview of the content of an evaluation will be determined and the plan for evaluating this content will be composed, the evaluation may start. It is important to keep in mind that evaluation may be quite a lot of work. Because of that it is recommended to prioritise. Focus the evaluation on the issues that matter most. Evaluation that focuses on priorities, that is planned efficiently, that has support and that is organised as a learning process will be perceived as a logical and integrated part of a project or a change process. One could even leave the word evaluation out of it. It is rather learning, planning, reflecting, revising, optimising one is involved in. The more people perceive self-evaluation as an activity on top of all the things that need to be done the more resistance it will generate, and the less effective it will be. The central message of our philosophy and framework is that self-evaluation should be nothing more than the cleverest way of working (i.e. open, reflective, interactive, transparent).16
2 SELF-EVALUATION: STATE OF THE ART IN 14
EUROPEAN COUNTRIES/REGIONS
Bart Maes, Department for Educational Development, Ministry of Education, Flanders.
This article is developed from a paper presented at the Brdo conference of September 2003
1. Internal and external evaluation
1.1 External evaluation: accountability and improvement
External evaluation may be justified on the grounds of a need at centralised level to control and guide schools. It attempts to ensure that quality education is provided, that schools use resources efficiently and that they provide value for money. It has the task of ensuring that differences in school standards are not too discrepant and that agreed outcomes are met. External evaluation also raises the public’s general awareness of quality issues by publishing reports on the general health of the education system or of specific schools. While external evaluation is driven primarily by a need for (political) accountability, it may combine this with an improvement perspective. External evaluation can offer feedback to schools on their strengths and weaknesses, drawing up action points, offering support or resources to meet their targets. It can give impetus to school improvement by providing comparative data which can then be used as a management tool for focusing on a school’s performance in comparison with others (benchmarking) (MacBeath, 2000, 91).
1.2 Self-evaluation: improvement and accountability
Self-evaluation is underpinned by a different rationale. While it also has an accountability purpose, its primarily impulse is developmental. It is an intrinsic feature of effective schools and professional practice but acquires an extra urgency from decentralisation.
From a policy perspective, it is viewed as a mechanism for empowering schools themselves to improve quality from within, helping them to monitor their progress and to report accurately to their external constituencies - parents and wider public. It is seen as contributing to the democratic debate as to what constitutes quality at school and classroom level, whilst also complementing the work of external agencies.
From a school perspective, self-evaluation has a more immediate purpose. Dialogue is focused more on the internal stakeholders and their contribution to planning and improvement at classroom, school and community levels. To achieve this effectively requires the involvement of all relevant actors, and access to instruments which can best support decision making, learning and teaching (MacBeath, 2000, 91-92).
1.3 The relationship between internal and external evaluation
A successful and lasting marriage between internal and external evaluation is the goal towards which many European systems are striving. The nature and process of the evaluation system within any one country can be plotted as a point along the dimension of internal/external evaluation (see figure below). In addition, the further dimensions of pressure/support and bottom up/top down should also be considered in order to have an idea of what ‘systems’ have at their disposal.
The internal-external dimension represents a continuum from self-evaluation to evaluation from an outside source. The external end of the dimension relates to systems where the monitoring of quality and standards rests solely with an external body (e.g. inspectorate). On the other hand of the dimension lie systems where there is no external body and quality assurance is exclusively the province of the school itself.
The pressure-support axis describes a continuum with, at the one end, a high level of support from ‘the system’ and extremely strong pressure on the other end. While this dimension can be observed objectively, it also concerns a more subjective reality. That is, pressure and support are best understood in terms of what people experience: do they feel they are under pressure or do they rather feel supported? The balance can be found here by taking individual and collective experiences into account. People can do their job most effectively when a balance is reached here because they feel intrinsic satisfaction and motivation as well as extrinsic recognition and reward. The top down to bottom up axis represents how a system sees and implements change. At one extreme, it is
17
delivered from above, by dictation, by legislation, by national structure. Alternatively, it can come entirely frombelow, from class teachers, from pupils and parents.
Finding the ideal point of balance within and between these three dimensions is the challenge for all systems which seek to evaluate quality in a way that enhances the capacity of schools and teachers. Top down approaches need bottom up responses. External expectations have to meet internal needs, and pressure will not work without support (MacBeath, 2000, 93). Van Petegem (1996-’97, 470) agrees that the balance between external pressure and internally driven quality assurance is very subtle. External pressure will always be necessary to maintain the attention for this internal care for quality, but it should not result in too much pressure.
With this framework in mind, we will now look at what the ESSE-project taught us about the participating
countries’ search for balancing internal and external evaluations. How do they use the means at their disposal? In order to do that correctly, we need to consider the whole range of external support. More specifically, we will look at each of these external support mechanisms and reflect on their contribution in finding a balance between the dimensions mentioned here.
2. The ESSE countries in search for balancing internal and external
evaluation.
2.1 Provision of legislation and statutory position of school self-evaluation
In four countries, there is a formal obligation to implement self-evaluation. This is the case in Ireland, the region of Upper Austria, the French-speaking community of Belgium and Denmark.
In Ireland, self-evaluation is compulsory as it is an essential part of the school development planning process. Schools have to ensure that the school development process is regularly reviewed and updated.
In the French-speaking Community of Belgium, schools have to annually assess the implementation of the school plan and have to suggest adjustments and recommendations in an activity report, including whether or not the targets have been reached.
In Denmark, self-evaluation is so far only compulsory in upper secondary business colleges. In these colleges school heads and teachers are obliged twice a year to ask the students about their experiences concerning the quality of teaching in the different subjects, the educational organisation, etc.
In Upper Austria (which is one region of nine) self-evaluation is only compulsory in general secondary schools (comprising years 5 – 8).
In eight countries self-evaluation is not fully compulsory as there is no formal obligation to implement self-evaluation, but schools are strongly encouraged to perform (a kind of) self-evaluation. In fact, in some cases they almost have no choice. In England, Scotland, Northern Ireland, the Netherlands, France, Austria,
18
Flanders and Hesse the implementation of self-evaluation is highly recommended by the authorities and/or the inspectorate. In most countries, schools (or school governing bodies) are encouraged in different ways to perform self-evaluation.
In England, Scotland, Northern Ireland, Ireland, the Netherlands, France, Austria, the French-speaking Community of Belgium and Hesse schools have to produce and implement a school development plan/ project. In some of these countries, schools also are obliged to evaluate their progress in meeting the development plan objectives. In addition, in most of these countries schools also have to determine a strategy to improve school development (planning).
In England, Scotland, Northern Ireland and Flanders schools have to complete a self-evaluation form or self-evaluative profile before an inspection takes place. In Flanders, in the process file that schools have to fill out, the schools are invited to judge and assess themselves on different issues such as vision, participation policy, communication, co-ordination, data gathering, quality analysis and improvement.
In England, Scotland and Northern Ireland schools have to set targets as a way of working towards self-evaluation and improvement in schools, and/or to monitor and improve pupil performance.
In England and the Netherlands, schools have to monitor and safeguard standards of professional performance.
In most of the countries schools are also expected to formally report on one or more of the above mentioned aspects.
In the Czech Republic, Portugal, Denmark and Saxony there is no formal obligation to implement a self-evaluation process.
In most countries where school development planning or school improvement is increasingly emphasised or are compulsory, self-evaluation is recommended to schools and headteachers.
Self-evaluation is, depending on the country, partially or indirectly required by law albeit with the focus on reporting on progress towards stated development targets. Whilst a broader process of school self-evaluation is not specifically required by law, there is a very strong (national) expectation that this should be undertaken by all schools. In some cases, aspects will be directly controlled or evaluated by the inspectorate, the ministry or indirectly in the context of the obligation of reporting. Therefore, many schools have implemented some kind of self-evaluation.
In the different countries, there is a large variety in procedures and approaches in school improvement and school development planning which eventually stimulate and initiate self-evaluation. Some of the countries are in a gradual evolution towards more mandatory systems.
The link between self-evaluation outcomes and school planning
In 11 countries, the use of self-evaluation outcomes is an essential element for school development planning, action planning and improvement. Therefore, there is a clear link between school (development) planning and the self-evaluation outcomes.
• In England, Scotland and Hesse only school development planning is compulsory, while schools are relatively free to implement a self-evaluation process. However, as in these countries self-evaluation is considered as an essential part of school develop¬ment planning and school improvement, schools actually do not have much choice.
In England, self-evaluation is not compulsory but it is the key to target setting, school development planning and school improvement. Target setting is compulsory and implicitly requires some kind of self-evaluation. Furthermore, the Department for Education and Skills strongly encourages schools to work on a school development plan and to use self-evaluation as a key to school improvement. Self-evaluation must lead to effective action planning to build on a school’s strengths and remedy existing weaknesses.
• In Ireland, France, Denmark and the French-speaking Community of Belgium both self-evaluation and school (development) planning are compulsory. Self-evaluation in itself is directly compulsory because it is seen as an essential tool for school improvement, strate¬gic planning, action planning.
19
In France the annual assessment of the outcomes is part of the process of the school project (both schoolproject and self-evaluation are compul¬sory). The outcomes of the annual assessment are a basis for correcting the school project or for modifying the scope if the outcomes suggest that the objectives are unrealistic.
• In Northern Ireland, Portugal, Flanders and Saxony there is also a clear link between self-evaluation out comes and development planning. However, schools are free to implement self-evaluation and/or school (development) planning processes.
Some Portuguese schools normally use self-evaluation outcomes for the school development plan, especially to determine the priorities for action. This is current practice in schools with strategic planning skills. School planning, nor self-evaluation, nor the link between them is compulsory.
In five countries, schools are free to choose whether they link their self-evaluation outcomes to the school development planning and improvement or not. This is the case in Austria, the Czech Republic, the Netherlands, Hesse and Denmark (with the exception of the upper secondary business colleges) where self-evaluation is not compulsory. In these countries it is up to the individual school management to decide whether they want to make a link between self-evaluation outcomes and school (development) planning and improvement or not.
Schools’ reports on self-evaluation
Schools can report in different ways on the outcomes of their self-evaluation process. • Relatively compulsory reports including self-evaluation outcomes
In England, Scotland, the Netherlands, the French-speaking community of Belgium, France and Denmark schools have to write a report on their self-evaluation outcomes or a report including their self-evaluation outcomes.
All English schools have to publish the “Governors’ Annual Report” which should be based on a self-evaluation process regarding target setting.
• School reports on self-evaluation outcomes as a starting point for meta-evaluation
(i.e. evaluation of the quality and effectiveness of self-evaluation). This is or will be the case in three countries Northern Ireland, Austria, and the Netherlands.
In Northern Ireland the school performs an internal audit and reports on the results of this audit to their staff. This audit report is the starting point for a quality assurance inspection to assess the efficacy of the internal quality assurance procedures and to comment on the validity of the outcomes.
• Reports which are (also) meant for the inspectorate but not for meta-evaluative purposes are written in Northern Ireland, Hesse, Saxony, France and Denmark.
In Saxony and Hesse, the inspectorate uses this information to provide support and advice on the next steps of school development and/or self-evaluation.
• Internal reporting to the relevant stakeholders
In eight countries schools report to the relevant stakeholders, such as teachers, students, parents, e.g. through a prospectus, (planning) documents, meetings, conferences. In all these countries, this is
not compulsory. This is the case in Ireland, the Czech Republic, Portugal, Hesse, Saxony, Denmark, England and the Netherlands.
In the Netherlands the school prospectus, which contains information on a school’s objectives, its educational activities and the results achieved, is meant as a public record enabling parents to influence the quality of education. In Denmark some teachers also report to the headteacher on the quality of teaching. These reports are based on remarks provided by the students.
• Leaflets, brochures, informative portfolios about the school to the wider public
In some countries schools use self-evaluation outcomes in their leaflets (e.g. in France), brochures (e.g. in Hesse, distributed by the Hessian institute of In-service training ‘HeLP’), informative portfolios (e.g. in Portugal) and other documents (e.g. through an annual report in the Czech Republic) which are meant for the wider public.
20
the one hand some countries have a strong tradition on political accountability through publication of school results, often in a climate of comparability and competition. On the other hand there is a concern about openness towards all stakeholders. Furthermore, some countries have no tradition at all in taking into account forms of school and pupil results and in reporting on them.
2.2 Provision of statistical data for comparison and benchmarking
In most countries (12) national/regional bodies or other organisations provide schools with regular benchmarking data to help them judge how their performance compares to that of other schools, and to help them set
appropriate targets for improvement. Only in Austria, Hesse and the Czech Republic real “national” benchmarking data are not (or only to a minimal extent) available.
Different kinds of benchmarking and statistical data are available in these countries: • National performance benchmarks
This information allows schools to compare their results with recent national results and with the national data of the previous years. It shows the range of performance of similar schools, grouped together for example on the basis of free school meals or pupils’ prior attainment. National
performance benchmarks are provided in England, Scotland, Northern Ireland, Ireland, the Netherlands, Portugal, France, Flanders and the French-speaking Community of Belgium, the Czech Republic,
Denmark and Saxony.
• National value-added indicators
Value added-indicators are quantitative indicators used to assess what schools add to pupils’
performance, in comparison with expectations based on pupils’ backgrounds and prior attainments. This is an indication of the relative progress pupils have made. Value added shows the progress made by an individual or a group compared with the average progress made by similar pupils. Schools can use this information to assess how their pupils are performing in comparison with pupils of similar starting attainments nationally. Schools may also use the information to consider the performance of different classes or groups of pupils (e.g. boys, ethnic minorities, children with special educational needs) as well as the performance of a whole cohort. These indicators are provided in England (as part of the Autumn Package and through the LEA), Scotland, the Netherlands (through the use of a pupil monitoring system) and France.
• Data from the “annual” report of the inspectorate on the state of the education system
In six countries (the Netherlands, the Czech Republic, England, Scotland, Flanders and the French-speaking community of Belgium), the inspectorate has to report regularly (e.g. each year) on the state of the education system. In most cases, this report is based on the inspection findings gathered during school inspections of the previous year(s). These reports can also provide a kind of national benchmarking data to schools.
• Other data
The provision of other kinds of specific benchmarking is mentioned by Scotland, Denmark, the Netherlands, Flanders, Hesse, Saxony and Austria.
In most of the countries that provide benchmarking data, these data are considered as an essential foundation and starting point for all school self-evaluation, especially with regard to the analysis of attainment. Schools are invited to use the national data to assess their performance and to define their objectives accordingly.
Benchmarking data provide them with a minimal objective framework. In some countries the use of benchmarking data is even compulsory (England, Scotland, France,) or strongly encouraged by the inspectorate (the Netherlands, Flanders, the French-speaking community of Belgium) in the context of self-evaluation, school development planning and/or target setting (performance management).
However, in Ireland, Portugal and Saxony, benchmarking data are not seen as an essential part of schools’ self-evaluation (yet), in spite of the presence of the necessary statistical and benchmarking data.
21
It is quite clear that countries that have examination systems on the national level provide (in most cases) a lot ofinformation to schools and/or other stakeholders in the form of benchmarking data. These data are more or less sophisticated.
In England there is a wealth of statistical data and detailed benchmarking analysis provided both nationally and by the LEA. To assist schools in performance management and to provide a basis for performance comparisons and self-evaluation by looking at school and pupil results, the Department for Education and Skills (DfES), Ofsted, and the Qualifications and Curriclum Authority (QCA) produce jointly an annual Autumn Package of Pupil Performance Information, published by DfES. This package consists of National Summary Results (overview of pupils’ results on national test and examination performance at system level), National Value Added Information, National Benchmark Information (benchmarking data bringing the results of similar schools together) and a Performance and Assessment Report (PANDA, i.e. a package of data which is unique for each school and set in the context of the school’s socio-economic environment). The use of this package is compulsory and essential for the process of target setting, school improvement and self-evaluation.
Furthermore, we can state that the more self-evaluation is incorporated in the whole evaluation system and the more attainment results are available on the national level, the more benchmarking is used at school level. In a number of countries the inspectorate also has a significant role in gathering, analysing and/or disseminating benchmarking and statistical data.
2.3 Provision of a set of quality standards/indicators
Systematic self-evaluation processes in schools should be based on a comprehensive framework of quality indicators or standards. We investigated whether there is a nationally agreed or stimulated framework consisting of indicators, standards, and/or criteria available to schools in the different countries.
In most countries current inspection frameworks for evaluating schools are or can be used by schools to monitor their own performance. This is already the case in England, Scotland, Northern Ireland, the Netherlands, and Flanders. The use of the inspection frameworks for school self-evaluation is not compulsory but in some cases strongly encouraged e.g. by the government.
HM Inspectorate of Education in Scotland has developed “How Good is Our School”, a nationally accepted evaluation framework consisting of 33 quality indicators, which can be used for external evaluation by HMIE as well as for self-evaluation by the school. The use of this framework is not compulsory but it is endorsed by local Education Authorities and by the independent schools.
Ireland, the Czech Republic and Portugal are planning to make the inspection framework available to schools in the future.
“Evaluation Criteria” have also been developed for external evaluation of schools by the Irish Inspectorate as part of the development of Whole School Evaluation (WSE)-system (a quality indicator framework for full inspections), on which a pilot project has been completed. Although planning for the introduction of the WSE-model in primary and second level education is still incomplete, it is intended to publish and issue these criteria, as standards of quality, to schools in the near future as a resource for school self-evaluation. The use of the WSE-evaluation criteria as a basis for self-evaluation is optional.
In four countries (Austria, Hesse, France, the French-speaking Community of Belgium) the key areas of the school development plan can be used by schools as a framework of quality indicators. In most of these countries there is a kind of obligation with respect to the school development plan (and consequently to the use of the key areas of the school development plan as a framework for school self-evaluation).
Hessian schools are strongly encouraged to create a school development plan as a starting point for self-evaluation. The key areas of the school development plan can be considered as a quality framework for school self-evaluation.
In four cases (Austria, Denmark, Hesse and Saxony), a framework for school self-evaluation is developed in the context of a project stimulating the implementation of development planning or self-evaluation.
22
Education and Cultural Affairs has developed a number of quality areas and indicators for schools in the Quality in School-project (QIS). The use of these QIS-indicators is optional.
In Saxony, tests which compare general standards with student performance achievements also record standards. Saxony has recently introduced
the school portraits. In a school portrait, a school has to describe its situation in a number of areas and provide that electronically to the ministry’s school data bank. The areas of the school portrait can function as a framework for school self-evaluation. Working with school portraits is compulsory.
Common key areas and
indicators
Based on the available information, we have analysed the key areas and indicators in the 13 different nationally stimulated frameworks (most of them from the inspectorates) according to the broad lines of the conceptual framework which was developed by Deketelaere and discussed at a SICI-workshop (Deketelaere, 1999). Frequently used indicators in the Key Area “teaching-learning processes” are ‘teaching and learning’ (in 12 frameworks), ‘support and guidance’ (in 11 frameworks), ‘curriculum’ (in 11 frameworks), and ‘climate and/or ethos’ (in 11 frameworks).
In the Key Area “management processes” indicators about ‘management’ (which often also cover self-evaluation, and organisational aspects), and indicators about ‘links with parents and the community’ are mentioned in 11 frameworks, while indicators about the ‘management of resources’ are found in 10 frameworks.
The indicator “attainment and achievement of pupils” is the most popular one (in 12 frameworks) in the Key Area “output”.
Alternative quality frameworks
In 10 countries, schools are free to use other frameworks than the ones that are nationally promoted. The most popular alternative self-evaluation framework is “The Model of the European Foundation for Quality Management” (the “EFQM”-model) also known as “the European Business Excellence Model” which is used by some schools in most UK-regions, in the Netherlands, Flanders and Portugal. In Northern Ireland the use of this model is even supported by the Regional Training Unit.
In the UK-regions and Flanders schools can also strive for quality awards of a more specific nature e.g. “Investors in People”, “Charter Mark/Service first”, “ISO 9000”.
In England schools are also encouraged by the Department for Education and Skills to become involved in these quality systems. A small research has been conducted on the benefits of these quality awards for schools. One of the results was that recognised schools and schools working towards recognition as “Investors in People” received higher assessments of teaching quality at Ofsted inspections than schools not using the Standard.
23
In some countries schools make use of research-based instruments.Examples are the “SETAQ-questionnaire” (developed by the University of Strathclyde) in Northern Ireland, some instruments that are used by schools in the Netherlands (e.g. the “Diagnostic Instrument for School Improvement-DIS”, the “SAS-Instrument for School Diagnosis), frameworks from Austrian specialists such as Schratz and Posch, instruments and frameworks developed by Flemish researchers (e.g. IZES by Van Petegem).
In the Netherlands, Portugal, and Denmark schools use frameworks and instruments for school self-evaluation which were developed in the context of a project.
In Portugal some schools use the model of the “European project on school quality evaluation” (called Quality XXI). In the Netherlands different projects have been set up to stimulate and support the implementation of systematic school self-evaluation, e.g. the Q5-project. The Q5-network –an initiative of some school boards -works together with different other organisations such as the inspectorate, an institute for curriculum development, an association for educational assessment, regional advisory services for schools, pedagogical centres. QPO (quality in Primary Education), which is another Dutch project has published an inventory and description of possible frameworks and indicators on the website.
Flanders, England and the Netherlands report that alternative frameworks and instruments for school self-evaluation were developed by educational counselling services and/or (private) consultancy services or firms.
2.4 Provision of guidance and training for self-evaluation
Some encouragement and support at national level is important to