An evaluation can be defined as “the systematic investigation of merit or worth” (Guskey, 2000: 41). The evaluation of programmes can be used to distinguish between those that are effective and those that are ineffective. In other words, evaluation helps to determine whether it is worth committing further resources in a specific programme (Patton, 2008: 4). Adler (2002: 2) emphasises that “we need to identify, and then describe and explain, the kinds of programmes that are beneficial, or in the language of finance, add value….” A central question in this regard is how to identify PD programmes that are “beneficial” to teachers?
Another, more detailed, definition of evaluation is provided by Weiss (1998: 4) who characterises it as a “systematic assessment of the operation and/or the outcomes of a programme or policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy”. This definition highlights five important aspects. The first one, systematic, implies that the evaluation procedures are of a scientific nature, that is, evaluation is a purposeful, intentional, and thoughtful process (Guskey, 2000: 42). The next two aspects that need to be focused on are the operations and outcomes of the programme. These are referred to as process-based and outcomes-based evaluations. Process-based evaluations are done when one wants to fully understand how a process works. These evaluations are usually used for long-standing programmes about which complaints have been received, or programmes which seem ineffective (Weiss, 1998: 5). Outcomes-based evaluations seek to establish whether a programme is using the appropriate activities to achieve its outcomes. The fourth aspect highlighted in the definition relates to comparison standards. This means that the merit of a programme is measured against some set of expectations, after the evidence relating to its process and outcomes has been collected (Weiss, 1998: 5). The fifth aspect of the definition is concerned with whether the evaluation contributes to the betterment of the programme. Generally, an evaluation is done so as to improve a particular programme/policy. However, whether this is the case in practice
30 still needs to be established. Loots (2008: 1213) claims that evaluation is a means to ensure quality in programme delivery and outcomes. It is this understanding of evaluation that the researcher will adopt in this study. Hence, the intended outcomes of the programme should be clearly stated. Friedman claims that the “pre- specification of intended outcomes allows their assessment to be more clearly defined and independently judged” (2012: 185). General statements such as ‘improving the quality of life’, ‘empowering’, or ‘changing’ are insufficient descriptions of the goals or outcomes of programmes (Loots, 2008: 1214). Indeed, although such outcomes formulations do state values, they can neither be measured nor evaluated (ibid).
Education PD programmes are costly investments made by governments, business, and non-profit organisations. Hence, it becomes imperative to establish whether these programmes are in fact doing what they are meant for, that is, if they yield the expected results or if better results can be achieved – even with a smaller budget. The best means of determining if PD programmes are reaching the desired outcomes – and if the same outcomes can be achieved with a smaller budget – is by evaluating these programmes (Weiss, 1998: 6). Evaluation helps to measure possible change and assess results (Zepeda, 2008: 37). In addition to asking whether programmes worked or not, there must be questions relating to the factors that make them work, why some programmes succeed while others fail, or even how the successful ones can be improved (Weiss, 1998: 55).
To assess whether a PD programme is valuable, its design must be examined to determine the likelihood of it producing the intended results (Killion, 2005: 5). Besides the design of the programme, its goals, objectives, standards of success, and indicators of success should also be scrutinised (ibid).
When a PD programme is initiated, there should be a set of objectives that it is meant to achieve; ideally, these objectives should be informed by a needs-analysis of the intended participants. The evaluation of the programme is therefore the systematic process of determining whether these objectives were achieved (Luneta,
31 2013: 109). However, it is not enough to know what the programme aims to achieve, it is also important to establish how it intends to achieve its stated objectives (Weiss, 1998: 55). In order to measure this, each objective should have a performance indicator that assists in determining the success of the programme. A PD evaluation, in particular, “requires examination of programme goals, duration of PD activities, levels of implementation, change in beliefs and practices” (Zepeda, 2008: 36), to name but a few.
Three broad types of evaluation can be distinguished, namely, planning, formative, and summative evaluation (Guskey, 2000: 56). Planning evaluation takes place at the beginning stages, before the programme or its activities begin. The intent of this type of evaluation is to assist decision-makers in determining whether they are choosing the correct direction and if their desired results can be achieved (Guskey, 2000: 57). Formative evaluation occurs throughout the programme. The aim of this type of evaluation is to regularly feed information back into decision-making for the purposes of modification and improvement. The teachers and administrators are the audience of the evaluation (Zepeda, 2008: 45). Summative evaluation happens at the end of the programme and aims to assess the overall worth of the programme, to enable decision-makers to make a pronouncement on the continuation, termination, or revision of the PD programme (Zepeda, 2008:45). This study used summative evaluation, because the researcher undertook this investigation at the end of the lifespan of the programme. Thus, the researcher looked back to the PD programme to assess its overall worth.
The successes of the professional development of teachers are measured in terms of increases in their knowledge and skills, changes in classroom practice, and improvement in the students’ learning (Kilion, 2002; Sparks, 2002). This is in contrast to documenting activities completed by participating teachers or measuring teachers’ satisfaction with the PD programme (Blazer, 2005: 14; Killion, 2005: 5; King, 2014: 2). This can, however, only be done if the way in which the attainment of the programme’s goals will be assessed is clearly outlined (Blazer, 2005: 14). Cochran- Smith (2005: 302) argues that there is a need for better and more research on the
32 outcomes of teacher education. This includes separating the impact of preparation from the entering characteristics of participating teachers. It also requires careful consideration of how the teachers use what has been learnt in the classroom. Ultimately, teacher and classroom quality is embedded in what teachers do in the classroom with learners, and not just in their credentials or in course work (Pianta, 2005: 8; Bansilal, 2012: 254).
2.3 EFFECTIVENESS OF A PROFESSIONAL DEVELOPMENT