TABLE 2.2 REVIEW OF EVALUATION THEORISTS AND THEIR CONCEPTS

Theorist Concept Concept Description

Barry MacDonald

Democratic Evaluation

The democratic evaluator recognizes multiple

perspectives of stakeholders and seeks to represent them without making judgment. The evaluator only collects definitions, perspectives, and judgment from

stakeholders (MacDonald, 1979).

Daniel

Stufflebeam CIPP Model

CIPP stands for context, input, process, and product. Context evaluation involves an assessment to understand needs, problems, and opportunity to decide program goals and objectives. Input evaluation identifies strategies and action plan. Process evaluation identifies shortcomings and refines implementation. Product evaluation measures outcomes for making decision. CIPP is cyclical to ensure continuous improvement of the program being delivered (Stufflebeam, 2007).

David Fetterman

Empowerment Evaluation

Participants conduct their own evaluations with outside evaluator as a coach or facilitator providing knowledge and tools for continuous self-assessment and accountability (Fetterman, 1998). Donald Campbell, Edward Such Man, &Thomas Cook Quasi- Experimental design

It is used to study impacts of programs when random assignment of participants is not possible. A control group and an experimental group is selected and compared with the attempts to uncover a causal relationship, even though the researcher cannot control all the factors that might affect the outcome (Alkin, 2004)

Egon Guba and Yvonne Lincoln

Fourth Generation Evaluation

The approach is based on constructivist methodology, aiming to develop judgmental consensus from an interaction among stakeholders with different, conflicting, or constructive opinions. The evaluator either moderate or engage in the negotiation (Guba & Lincoln, 1989).

Hallie Preskill Transformational

Learning

Preskill places importance of organization diagnosis on culture, communication style, values, leadership by the evaluator to get a contextual clue to be able to set realistic goals, frame finding, & set expectations for evaluation use. Diagnosis needs are shared to provide opportunity for reflection, transformation of learning, and utilization (Preskill & Torres, 1998)

Huey-Tsyhchen Theory-driven

evaluation

Involves making explicit why and how the program is supposed to achieve its outputs and outcomes, then analyse and investigate likely causal factors and/or alternative explanations for alleged program outcomes (Chen, 2005). J. Bradley Cousins Practical Participatory Evaluation

This approach was grounded in an organizational learning framework focusing on enhancing learning capacity of organization. Evaluation responsibilities are equally shared between staff and the evaluator. The staff are practically trained and supervised by the evaluator so eventually they can coordinate new evaluation efforts (Cousin & Earl, 1995).

Theorist Concept Concept Description

Jean King

Interactive Evaluation Practice (IEP)

Kings encourages communication and discussion to create shared meaning and trust among participants. This collaborative effort leads to tailoring of evaluation methods for the program. Major role of evaluator is to identify and foster leaders during evaluation process. Leaders are necessary to attract, recruit, and maintain collaboration from participants (King & Pechman, 2003).

John Owen Form and

Approach

Focusing on utilization, Owen provides a framework consisting of five major categories or ‘forms’ – proactive, clarificative, interactive, monitoring, and impact – according to state or orientation of the program, the typical issues attribute of the program being focused, and timing. Within these forms, he identifies a range of evaluation ‘approaches’ logically connected to each form each of which has implications for data collection and analysis (Owen, 2006).

Marvin Alkin User-oriented

Evaluation

Alkins focuses on identifying primary users and working with them from the start of evaluation process to enhance the possibility of utilization. The evaluator interacts with the users by present evaluation data as factual as possible without imposing value judgment, unless is demanded (Alkin, 1991).

Michael Patton Development

Evaluation

This involves the evaluator as an integral part of a team whose member collaborate to conceptualize, design and test new intervention. The evaluator elucidates the team with evaluative questions, data and logic to facilitate evidence-based decision-making (Patton, 1997, pp. 103- 106).

Michael Scriven Goal-Free

Evaluation

All of the outcomes of a program are examined, rather than limiting the investigation to the outcomes identified only by the objectives (Scriven, 1972).

Peter Rossi Tailored

Evaluation

Evaluation methods implemented are tailored to the stage of the program; that is, “fitting evaluation to program” (Rossi & Freeman, 1985, p.102)

Robert Boruch Randomized field

experiment

Attempts to estimate the relative effectiveness of different treatments that have a common aim, but depend on methods other than randomization to rule out the competing explanation for the treatment differences that may be uncovered (Boruch, 1997).

Robert Stake Responsive

Evaluation

Key feature is responsiveness to key issues, problems, or concern especially those experienced by people at the sites. (Stake, 2004, p. 89). Stake believes that there are multiple realities and stakeholder perspectives need to be represented.

Thomas Owens & Robert Wolf

Adversary Evaluation

There are two groups of evaluators collect and analyse data, then cross-examine the findings. This model was suggested to reduce bias on value interpretation (Alkin, 2004).

• an increase in ownership and a strengthening of commitment of stakeholders to implement corrective actions;

• creating joint learning to improve performance and outcomes;

• results that lead to better information flow;

• increase accountability and transparency.

Despite the popularity of participative approaches, it does not mean they are more superior to other non-participative methods; the best approach is that which best satisfies the aims of the study.

In practice, frameworks, models, or procedures are put forward in an effort to enhance program evaluation. Recognizing the complexity of the evaluation field, many practitioners have attempted to simplify these clustered theories, approaches, and methods into more logical and practical forms. Compared with other works, the framework developed by John Owen (Owen with Rogers, 1999; Owen, 2006), working in Australia, appears outstanding in terms of evaluation ‘forms’. Owen has developed an original, comprehensive conceptual framework of the existing evaluation approaches into five major categories or ‘forms’: proactive, clarificative, interactive, monitoring, and impact. Owen identifies the basic tenets of the evaluation form, then classifies and groups the evaluation approaches accordingly. The basic tenets include the following three aspects:

1. purpose or orientation of evaluation;

2. typical issues or broad questions that are consistent with each purpose; and

3. major approaches.

Other dimensions determining the evaluation form include the state of program, the major attributes on which the program is focused, and the timing of evaluation – prior, during, or after. Within these forms, he identifies the evaluation ‘approaches’ logically connected to each form and the implication of data collection and evidence analysis. These evaluation forms and their key features are outlined in Table 2.3. These dimensions are

TABLE 2.3 OWEN’S EVALUATION FORMS