Introduction
This study is a case study, not of the community development programme as
implemented by the Centre for Community Development, but of the evaluation of the programme implementation. It is also not a case study methodology employed to evaluate a programme but a methodological vehicle employed to focus on a particular approach to programme evaluation. A brief but particular reading and interpretation of the case study literature and its applicability to this study were provided in chapter one.
This chapter concerns itself with the meta-theoretical placement of the study in the shape of an analysis of the epistemological and methodological considerations and choices made by the researcher. This analysis provides the paradigmatic landscape wherein this study is located. It addresses the second research question namely, how do the debates in the philosophy of the social sciences support the theory-based
approach? There is an initial discussion on the meta-theoretical choices that
researchers make, can make or are exposed to. This is followed by an explication of the ontological, epistemological and methodological considerations and choices for this study. A detailed design for the evaluation of a community development programme is then provided. The design is outlined in terms of linkages to specific research
questions and the overall purpose of the study. It is presented within a ‘domain’
framework as conceptualised for the evaluation. Thereafter, and within the domains context, the choices and selection of research techniques are outlined and justified.
Lastly the data analysis procedures employed in this study are explained.
Epistemological and methodological considerations
This study takes Weiss’ (1998) position that programme evaluation applies the methods of social science research, both quantitative and qualitative. The principles and methods normally applied to all other types of research, must be applied in programme evaluation as well (Weiss, 1998; Clarke and Dawson, 1999). “What distinguishes evaluation research is not method or subject matter, but intent – the purpose for which it is done.” (Weiss, 1998, p.15) As stated in a previous chapter, the purpose of the evaluation could be to assess if social programmes are needed, if they are effective and/or if they are likely to be used. The efficacy of the programmes will be assessed, given the above purposes, in human and social terms. The research principles and methods will come into play when the evaluator has to make decisions about the particular methodological approach for the overall evaluation. Potter (1999) believes that while the evaluator will consider a methodology that suits the particular context of the programme as well as the requirements of those commissioning the evaluation, the evaluator will also rely on a set of personal assumptions to determine the methodology. According to Potter (1999), “particular evaluation practices reflect different methodological, epistemological as well as ideological assumptions” (p. 211).
Mouton (1996) describes it differently, as when researchers find themselves in a
‘particular intellectual milieu’, which involves the meta-theoretical assumptions held as valid within a discipline at a particular point in time. It is the intention here to take up this charge and to explore the intellectual milieu and through that, explicate the methodological, epistemological as well as ideological assumptions that support the choice of a theory-driven evaluation design.
These assumptions, according to Lincoln and Guba (1985), Potter (1999) and
Sarantakos (1998), are not merely in terms of methodology and methods, but related to a deeper worldview held by the researcher. The assumptions are the reflections of how the researcher sees the world and is referred to as a paradigm. A paradigm also
provides a philosophical framework for the study of that world. As such, paradigms serve a positive function by guiding the process of inquiry and forming a basis for the practice of science (Kuhn 1970). However, the dominance of any particular paradigm can lead to the stifling of creativity and the suppression of new lines of inquiry, thus
having negative effects on the processes of inquiry. It is precisely the positive and negative possibilities that have led to the ‘paradigm wars’ since the early 1900s. The
‘paradigm wars’ refer to attempts by scholars, opposed to the dominant positivist paradigm, to articulate alternative ‘paradigms’ based on different ontological, epistemological and even methodological principles. Protagonists for an alternative paradigm within the social sciences, argued that the natural science was conceived in positivistic terms and studied the objective, inanimate and non-human world. The social sciences instead were, according to them, concerned with:
Society, a product of the human mind, was subjective, emotive as well as intellectual… Human social behaviour was always imbued with values, and reliable knowledge of culture could only be gained by isolating the common ideas, the feelings, or goals of a particular historical society. It was these that made each social act subjectively meaningful. (Hughes, 1990, p. 90)
The ‘battle’ against the orthodoxy of positivism spawned alternative conceptions of searching for meaning such as “hermeneutics with its abiding question: how is an understanding of the past to be gained through its texts and other remains?” (Hughes, 1990, p. 90). The search for alternative conceptions of meaning resulted in various methodological approaches that argued against the narrow, positivist conception of meaning. Examples would include, among others, Marx’s material conception of history ( Mclellan, 1976), the Frankfurt School movement in search of critical theory (Bottomore, 1984), Habermas’ arguments for new foundations of critical theory (Roderick, 1986), and the more phenomenological, constructivist and feminist arguments that followed later (Lincoln and Guba, 1985). Hughes (1990) however, points to contemporary realists and empiricists such as Quine, Putnam and Hacking who present a revised conception of science and knowledge as a response to the critique of the positivist view of science.
What was rejected was not science, or indeed its eminence as a form of knowledge, but a view of science requiring epistemological and ontological unity. What is emphasised is the diversity and the disunity of science… the
theory-laden nature of observation is a feature of scientific work that natural scientists find unremarkable and obvious. (Hughes, 1990, p. 84)
There are clear signs that these wars are dissipating as more and more social scientists realise that it is more prudent for chosen methods to be determined by the nature of the research problem rather than by the methodological preferences of the researcher.
Clarke and Dawson (1999) maintain that, “ knowledge of the paradigmatic nature of decision-making will help make evaluators more aware of their methodological biases and paradigmatic assumptions so that they can make flexible, sophisticated, and adaptive methodological choices” (p. 38). The insight into the paradigms debate is essential, according to Clarke and Dawson, if evaluators are to avoid becoming
slavishly attached to one particular paradigm. Babbie (1989) states that a paradigm is a scheme that organises our view of the world. The paradigm does not “answer
important questions, it tells us where to look for the answers…where you look largely determines the answers you’ll find” (p. 47).
Sarantakos (1998) also draws his definition of a paradigm from Kuhn which states that
“a paradigm is a set of beliefs, values and techniques which is shared by members of a scientific community, and which acts as a guide or map, dictating the kinds of
problems scientists should address, and the type of explanations that are acceptable to them” (p. 32). Terre Blanche and Durrheim (1999) refer to paradigms as
“all-encompassing systems of interrelated practice and thinking that define the nature of enquiry” (p. 6). However, there is no complete agreement about the usage of the term.
Historically, some used the term very widely, referring to all scientific groupings, while others used it more narrowly to identify only the major theoretical directions in the social sciences. As an example, Sarantakos states that the first group (the wider perspective of paradigm), referred to phenomenology, symbolic interactionism, Marxism, feminism, postmodernism hermeneutics, etc., as examples of paradigms.
This is not the understanding of the term used in this study. The second group identified only two paradigms, namely positivism and post -positivism. The two paradigms had distinct features and were also referred to as the positivist and the interpretive paradigms. This dyadic interpretation of paradigm did not prevail for long and was expanded to include critical theory as a third paradigm, “and in this triadic
form it seems to have been accepted by most social scientists” (Sarankatos, p. 32). A description of these paradigms, taken from Sarankatos, is given below:
Table 3.1. Main paradigms in the social sciences
Positivistic Interpretive Critical
Positivism Symbolic interactionism Critical sociology
Neopositivism Phenomenology Conflict school of thought Methodological Ethnomethodology Marxism
Positivism Hermeneutics Feminism Logical Positivism Psychoanalysis
Ethnology Ethnography Sociolinguistics Sarankatos, 1998, p. 33
Potter (1999) makes a similar distinction but refers to the third paradigm as critical-emancipatory. Of interest to this study is Potter’s interpretation of the paradigms in terms of the kinds of evaluation research activities that would resort under each of these paradigms. According to Potter, the evaluation research activities resorting under the positivist banner would be limited to those aspects of social programmes that can be objectively observed and tested. He states further, erroneously one may add, that the activities are usually applied within a systematic framework and examples of such activities would include needs assessments, programme planning, formative
evaluations and summative evaluations. The interpretive paradigm according to him accommodates approaches that are more responsive to the programmes. It involves outsider evaluators who are mandated by all stakeholders. The evaluator structures regular report-back sessions and involve stakeholders in the design and implementation of the evaluation. Under the critical emancipatory banner, Potter locates evaluation research practices where there is a shared value system between the evaluator and the programme. It is generally based on action research for the purpose of improvement and involvement. The evaluator plays a central role in capacity building of programme staff as well as advocacy for the programme. The participants, through their
involvement in the process start learning to speak the language of power.
This outline by Potter is not very helpful in the present study. The examples used by him can be found and utilised across the so-called paradigms. He later states however that, “there is no single correct approach to programme evaluation, and evaluators typically choose an appropriate methodology to fit the pragmatic requirements of each programme, rather than being guided by one particular model or approach” (p. 211).
This statement resonates with an earlier one by Clarke and Dawson about the dangers of slavish loyalty to one particular paradigm. What is clear from Potter’s articulation of the paradigm-linked evaluation research activities is that the role of the evaluator will differ within each paradigm, the focus of the evaluation may differ and the involvement of the stakeholders will differ across the paradigms. The paradigmatic stance on each of the aforementioned issues is informed by a particular theoretical perspective of; (a) reality, (b) human beings, (c) the nature of science and (d) the purpose of science (Sarankatos, 1998).
Guba and Lincoln (1998) on the other hand argue that the basic beliefs that define inquiry can be summarised by the responses given by proponents of any given
paradigm to three fundamental questions. They are; the ontological question – what is the form and nature of reality?; the epistemological question – what is the nature of the relationship between the knower or would-be knower and what can be known?; and the methodological question – how can the inquirer go about finding out whatever he or she believes can be known? They provide the following table to illustrate the positions of each paradigm with respect to the three questions.
Table 3.2. Basic beliefs (metaphysics) of alternative inquiry paradigms
Item Positivism Post-positivism Critical Theory et al Constructivism
Ontology naïve critical historical relativism-local realism-“real” realism-“real” realism-virtual and specific reality but reality but only reality shaped by constructed realities apprehendable imperfectly and social, political
probabilistically cultural, economic apprehendable ethnic, and gender
values; crystallized
over time
Epistemology dualist/ modified transactional/ transactional/
objectivist; dualist/ subjectivist; subjectivist; created findings true objectivist value-mediated findings
critical findings
tradition/
community;
findings probably true
Methodology experimental/ modified dialogic/dialectical hermeneutical/
manipulative experimental/ dialectical
verification of manipulative;
hypotheses; critical multiplism;
chiefly falsification of quantitative hypotheses; may methods include qualitative methods.
Guba and Lincoln (1998, p. 203)
Note that Guba and Lincoln have added another paradigm, namely post-positivism, in their articulation of the broad range of paradigmatic considerations. While Guba and Lincoln sometimes accuse post-positivism as being merely a revision or slight
adjustment of the positivist position, proponents of post-positivism see it as a
“wholesale rejection of the central tenets of positivism” (Trochim, 2001, p.2). The above summary of basic beliefs will be used as a framework to explore the
paradigmatic concepts more fully and to anchor the present study in relation to the broader research paradigmatic considerations.
Ontological considerations
Guba and Lincoln (1998) maintain that the “term positivism denotes a ‘received view’
that has dominated the formal discourse in the physical and social sciences for some 400 years” (p. 202). Post-positivism according to them, represents attempts over the past few decades to respond in a limited way to the criticisms of positivism, while essentially remaining within the same set of beliefs. Critical theory (for them) denotes a set of alternative paradigms/approaches that include neo-Marxism, feminism and participatory inquiry. “The term constructivism denotes an alternative paradigm whose breakaway assumption is the move from ontological realism to ontological relativism”
(p. 203). This paradigm divide is further complicated by a conceptual split between a quantitative paradigm and a qualitative paradigm, where positivism and
post-positivism would be lumped under the quantitative banner and critical theory and constructivism under the qualitative banner. This is a false dichotomy and will be pursued later in the section concerned with the methodological dimensions of research.
The term ‘ontology’ means the study of reality (Mouton & Marais, 1990; Mouton, 1996; Sarankatos, 1998). Positivists, according to Mouton (1996), believe that there are enough similarities between the social and natural worlds to justify a common
epistemology and methodology in all the sciences, meaning that they ‘borrowed’ their underlying theoretical assumptions from the natural sciences. In the social sciences, they would hold that there exists a stable reality ‘out there’ that can be objectively captured through the senses. All who abide by particular rules of ‘observation’ can also uniformly perceive this reality and it (reality) is governed by a set of universal laws.
The ontology therefore is based on the premise of the existence of a stable external reality. The aim of research here “would be to provide an accurate description of the laws and mechanisms that operate in social life” (Terre Blanche and Durrheim, 1999,
p. 60). In a positivist view of the world, science was seen as the way to get at truth, to understand the world well enough so that we might predict and control it. The world and the universe were deterministic – they operated by laws of cause and effect that we could discern if we applied the unique approach of the scientific method. May (1993) makes the additional point that, positivism shares with empiricism “the belief that there are ‘facts’ which we can gather on the social world, independently of how people interpret them” (p. 6). The data collected, through meticulous and accurate
measurement will themselves constitute an end for the research and ‘will speak for themselves’. This ‘correspondence theory’ of reality is highly contested.
The ‘constructed reality’ as promoted by the constructivists, is an ontological position that maintains that reality is a construction in the minds of individuals. Since there are a lot of individuals, there will be multiple realities and the realities will be interpreted differently by different people. “Under this ontological position, the constructed realities ought to match the tangible entities as closely as possible, not, however, in order to create a derivative or reconstructed single reality or fulfil the criterion of objectivity, but rather to represent the multiple constructions of individuals (or fulfil the criteria of fairness)” (Lincoln and Guba, 1985, p.84). Interpretive science tries to capture reality as it is, namely as seen and experienced by respondents.
Constructivism, for Lincoln and Guba, hangs together with relativism. This is
particularly so in their promotion of ‘fourth generation evaluation’ (Guba and Lincoln, 1989). “For Guba and Lincoln there is no reality except that created by people as they attempt to make sense of their surroundings, and evaluation creates the reality that it presents, rather than discovering some objectively existing reality” (Shaw, p. 39). The ontological axiom therefore, that under-gird constructivist inquiry is the following;
There is no single reality on which inquiry may converge, but rather there are multiple realities that are socially constructed, and that, when known more fully, tend to produce diverging inquiry. These multiple and constructed
realities cannot be studied in pieces (as variables, for example), but holistically, since the pieces are interrelated in such a way as to influence all other pieces.
Moreover, the pieces are themselves sharply influenced by the nature of the immediate context. (Lincoln and Guba, 1986, p. 75)
Since the natural belief systems of constructivism are formed of ‘oppositional and conflicting’ assertions to that of positivism, Guba (1987) believes that the possibility of some form of conceptual (not methods level) accommodation does not exist. This is also termed the ‘incommensurability’ of different perspectives.
For critical theorists complex realites are formed in people’s minds. However, very often these realities are not what they seem. They hold that powerful people for the purpose of manipulation construct reality and to serve their own needs. Also, instead of reality being orderly, it is always in a state of conflict, tension and contradiction, resulting in a constantly changing world (Sarankatos, 1998). “Critical theory reached its zenith in the 1960s and 1970s. Its emphasis upon alienation, the domination of nature, the regressive components of progress, the mutability of human nature, and the stultifying effects of the culture industry and advanced industrial society made the enterprise relevant for young intellectuals” (Bronner, 1994, p. 9). Critical theorists believe that subjective meanings are relevant and important but they do not deny the existence of objective relations. For them there are also appearances and reality.
Reality often does not reflect the conflicts and contradictions that are prevalent in society, and appearance is based on illusion and distortion. “The interest of the critical theorists is to uncover these myths and illusions, to expose real structures and present reality as it is” (Sarankatos, 1998, p. 36). An example of this task would be the feminist agenda that sets out to expose patriarchal attitudes and gender divisions that work against the interest of women. Critical theorists further hold that society operates on the basis of oppression and exploitation. Masses of people are alienated from the corridors of power and this is a manifestation of a situation of domination. Freire (1998) states that “the more the alienated culture is uncovered, the more the oppressive reality in which it originates is exposed” (p. 476).
At this point, before looking at the ontological position of the post-positivist paradigm, it is possible to insert examples of programme evaluation approaches under each of the three paradigms discussed above. However, the literature informing and promoting programme evaluation practices of various kinds, starts and remains at a
methodological dimension. The exception would include among others, Guba and
methodological dimension. The exception would include among others, Guba and