P ROGRAMME E VALUATION

Programme evaluation is a field of social science that uses a wide range of scientific methods in assessing or evaluating programme or policy intervention (Babbie and Mouton, 2001). According to Frechtling and Sharp (1997), programme evaluation is a systematic and objective process for determining project effectiveness. Programme evaluation is essential in assessing whether goals of the project are met (Frechtling, 2002) and can also help to have objective information on programme performance and how it can be improved (Bless and Higson-Smith, 2000).

Equally important is that evaluation helps identify changes, and enables progressive learning at the individual, community, institutional and policy levels (Bellamy et al., 2001). Project evaluation also provides information on how different aspects of the project are working and on the project outcomes that were not anticipated. The information provided could be used for continuous improvement process (Frechtling, 2002).

In this respect, evaluation is critical to the success of policy development and implementation. Nonetheless, it has been reported that evaluation of natural resource management policy has been neglected, leading to emergence of substantial gap

between theory and practice (Wallace et al., 1995; Bellamy et al. (1999a: cited in Bellamy, 2001)). Despite the claims that participatory forest management can contribute significantly to the improvement of forest condition and people’s livelihoods, few efforts have been made to review the performance of such policy interventions.

2.6.1 Types of Programme Evaluation

There are many types of programme evaluation, but the main types as categorized by many authors are: programme monitoring; programme performance assessment;

process evaluation; outcome or impact evaluation; and programme cost-effectiveness and cost-benefit assessments (Babbie and Mouton, 2001; USDJ, 1997; Frechtling and Sharp, 1997; Holmes, 2007).

Monitoring evaluation focuses on continued monitoring of the programme through selected indicators of the project activities as a tool for effective programme management. This type of evaluation is usually integrated into the routine programme implementation with the aim of providing information and improving performance (Babbie and Mouton, 2001). It is an ongoing collection of information to determine whether the programme is operating according to the formulated plan and focuses on programme implementation and function (USDJ, 1997; Frechtling and Sharp, 1997).

Programme performance assessment is also an ongoing collection of information with a focus on whether the programme is meeting its goals and objectives. It deals with programme activities and delivery of its services. Programme process on the other hand focuses on programme implementation and operations. It addresses programme operation and performance, identifies processes or procedures used in carrying out the programme functions, and answers questions regarding programme efforts.

Programme outcome or impact evaluation is another form of evaluation. It is employed to measure programme performance and determines whether programme activities produced the desired outcome or whether the programme achieved its intended objectives. Impact evaluation is aimed at providing an estimate of the impact

of the intervention or programme that are without the influence of other outside factors or events. The outcome may be divided into short-term, intermediate and long-term, with long-term outcome being the programme goal. As outlined by Holmes (2007), impact assessment requires that the programme objectives be adequately and well articulated to make it possible to measure the expected outcome. The programme should be sufficiently implemented before impact assessment can be conducted.

Programme cost-effectiveness and cost-benefit assessment focuses on assessment of programme effectiveness in terms of costs. It does not determine whether the programme worked but the results of this type of evaluation are used to compare programme economic outcomes and costs (USDJ, 1997; Frechtling and Sharp, 1997;

Holmes, 2007).

All the above types of programme evaluation are relevant and important to ensure successful programme implementation. However, application of all the types of evaluation in evaluating a programme or an intervention is rarely done due to constraints in logistics and resources for evaluation studies.

2.6.2 Programme Evaluation Methods

There are different ways of defining and measuring any particular evaluation process.

The choice of a measurement method is critical to programme evaluation process (Bless and Higson-Smith, 2000; USDJ, 1997). The programme should be well-understood and conceptualized before evaluation, and the understanding and conceptualization are best achieved through the use of programme logical framework.

According to Renger and Titcomb (2002), logic model is a tool for describing various components of a programme in a systematic and structured manner. Most evaluators make use of programme logical framework to assist in establishing whether the programme goals and objectives are well formulated, and whether programme activities and outputs are clearly specified, and whether the outcome and associated indicators are provided.

This study employed programme outcome or impact evaluation in order to evaluate programme success and accomplishment. The evaluation examined programme effectiveness, achievement of goals and objectives, and other unintended consequences. According to Rossi and Freeman (1999: cited in Holmes 2007), the aim is to produce an estimate of impact of the intervention not influenced by other events or processes. The prerequisites for outcome or impact evaluation are that the objectives are well articulated and activities are sufficiently executed (Holmes, 2007).

According to Babbie and Mouton (2001), there should also be change over time after the introduction of an intervention and that change should be attributed to that particular intervention and not other unrelated causes. The outcome may be divided into short-term, intermediate and long-term, with long-term outcome being the programme goal (Bless and Higson-Smith, 2000; USDJ, 1997).

2.6.3 Programme Evaluation Criteria and Indicators

Policy initiatives, such as participatory forest management need to be evaluated, linking the objective of the evaluation and the rationale to the performance of the project. The fundamental basis for evaluation is the establishment of practical criteria by which change can be monitored and assessed in order to assess progress and impact of the policy initiative (Bellamy et al., 2001). Programme or project evaluation must be clear, with measurable project goals and objectives that outline what the project planned to accomplish. The success indicators enables evaluation of what was set to be accomplished and what has been the impact of the project (PHAC, 1996).